When an ACTIVE batch takes over the active writer role from a SUBMITTED
batch, the written BO has the syncobj from the latter even though the
writer is the former. This is correct and an intended state, but it
means that then we can't gate the syncobj cleanup in agx_batch_cleanup
on being the active writer, since the SUBMITTED batch won't be.
Fixes: asahi/mesa#18
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
This code was originally based on the Panfrost implementation, but has been
improved in a number of ways.
1. Transform feedback programs are dispatched generically with Gallium calls,
rather than emitting something hardware-specific. This is cleaner and
portable to future GPUs.
2. Transform feedback with indexed draws is now fixed, by lowering to an index
buffer pull.
3. Transform feedback with buffer overflows is now fixed, by correctly
bounds checking in transform feedback programs.
4. Transform feedback with strips/fans/loops are fixed, by correctly
tessellating to the underlying primitives as required by OpenGL.
5. Transform feedback with QUADS is fixed, by tessellating to triangles as
required by OpenGL.
That said, the code is still not in its final form.
1. It still does not support indirect draws. This will require a substantial
overhaul to do tracking on the GPU instead of the CPU. Currently we force
unroll indirect draws (slow but kosher in GL, treif in Vulkan). This isn't
hard to solve but I'm not going to duplicate the code until the algorithms
are otherwise complete because it's a lot easier to hack on the CPU versions
than the GPU versions.
2. It still does not support primitive restart. This has especially nasty
interactions with transform feedback. Again we force unroll to non-primitive
restart forms, again slow but kosher in GL but treif in Vulkan. This is a lot
harder to deal with. I sketched out something really nasty in my notebook
(hinging on efficient GPU prefix sums) but I'm not in a hurry to type this
out.
3. There will be interactions with geometry and tessellation shaders and I don't
think I can get the core code here future-proofed without actually bringing
up the new shader stages.
As such, this is a hard fork of the panfrost code for now, I'm not trying to
share the code (although it *would* clear out almost all of panfrost's transform
feedback related piglit failures).
Passes dEQP-GLES3.functional.transform_feedback.* and most of the relevant
piglits.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
This shortcuts all headaches about how big this should be. It does increase
memory usage a bit if there are lots of shader variants compiled, but this
should be tolerable, and can be optimized later if so required. Thanks to the
previous commit, the disk cache size should be unaffected.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
We were being sloppy with the sizes before. It mostly worked out, but there were
some corner cases where we would end up with mixed sized collects and that won't
end well for us. Let's rework the logic to make all the sizes explicit in NIR --
32-bit for depth and 16-bit stencil -- and then do the needed promotions to make
it happen in the AGX IR side.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
In case .x isn't read, it'll be null which has the wrong size and will fail
the validation added later in this series. We fix this by padding with sized
undefs (something that exists of defined size but undefined value) rather than
nothingness (of undefined size).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
Sometimes a shader might index with a non-power-of-two stride. For example, if
it's indexing into an array of structures where the structure size is not a
power of two, we'll get a multiply with a constant as opposed to a shift. We
want to handle these cases, too. To do so, we generalize our pattern matching to
look for any kind of multiply (with our new helper), rather than hardcoding
logic for ishl. This eliminates right-shifts in a pile of compute shaders, which
makes me happy from a "I read lots of shader assembly when debugging"
perspective.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
Currently, we hardcode logic in the addressing chasing code to look for ishl
instructions that shift by constants. We can generalize this to looking for
integer multiplies by constants to optimize more addressing patterns. Add a
helper to do so.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
This is totally pointless. This saves some waits at the ends of compute kernels
(waiting for stores to complete before terminating the thread). I don't know
how much this would matter for performance, since the hardware may have to do
these waits internally, but it makes the generated code less silly which is
always nice.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
This lets us shadow textures updated in the middle of rendering in Quake3.
They're big memcpys, but as long as the texture memory is cached it's ok. We use
a heuristic to avoid too many memcpys from uncached memory, which would cause
slideshow performance in quake.
We need to be careful to avoid shadowing shared resources, though, that's
invalid and would break WSI pretty hard.
It would be better to blit on the GPU for large shadowing, but that's more
involved and left for future work.
Reduces stuttering in Quake3.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
Comparison with PowerVR's XML shows that this is the actual name... And it needs
to be set a bit more carefully than "no colour output" in order to get correct
behaviour for depth-only passes that use sample mask / discard. Fix the name
first, the extra conditions will come when they're needed for multisampling.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
When possible. Only occassionally possible because the loads are pretty limited
in the addressing arithmetic. This probably doesn't matter for performance but
it saves some noise in dEQP tests which makes for nicer debugging, plenty of
optimizations end up worth it for that alone.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
They should already work, we just need it enabled. The comment here claimed
(incorrectly) that there is no hardware support for linear 2D arrays. In fact,
there is support, it's just not advertised in the public Metal API. With some
awful tricks, I managed to reverse-engineer the hardware interface and hooked it
up, so we can take advantage of it now.
In fact, we can stop checking the target explicitly at all. The only case where
we can't compress is 1D/buffer textures, which are necessarily less than 16
height so will be dropped in the next check.
When I originally wrote this cuhange, dolphin's MeltyMoltenGalaxy trace with
specialized shaders at 4K was helped from 28fps to 43fps, which is massive :-)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
Staging resources need to be linear for efficient CPU side mapping. This is a
problem for access to 3D and cube textures, since we don't have linear 3D
textures or linear cube textures. But we do have linear 2D array textures, which
can be reshaped to the same effect. So use a 2D array staging resource even for
3D textures and cube maps.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
We handle linear 2D arrays internally for blit shaders, so we need textureSize
to work for these. That requires some special casing, because there's a line
stride where the layer count would otherwise be. But it's not too bad.
Fixes
dEQP-GLES3.functional.shaders.texture_functions.texturesize.sampler2darray_*
when forcing linear textures.
Since we clamp array access to the maximum layer, we need textureSize() to work
for even the most basic array texturing. So this should fix blits from linear 2D
arrays as well, which finally unlocks support for compressed arrays/cubes/3D
textures.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
From the commit message of 94f7c011d6 ("v3d: Track write reference to the
separate stencil buffer."), anholt says:
Otherwise, a blit from separate stencil may fail to flush the job that
initialized it, or new drawing could fail to flush a blit reading from
stencil.
Fixes
dEQP-GLES3.functional.fbo.blit.depth_stencil.depth32f_stencil8_stencil_only
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
Our staging resources need to be LINEAR, however we don't support LINEAR with
DEPTH_STENCIL. The APIs don't actually require this, we just need to make sure
we don't generate internal staging blits to linear depth/stencil resources. For
uploading to compressed depth/stencil textures, we could use a depth/stencil
staging (since we can read from linear depth/stencil). However, for downloading
from compressed depth/stencil, we can't use a depth/stencil staging (since we
can't write linear depth/stencil). So, to handle both cases in a unified way,
just use colour blits for depth/stencil resources, using a compatible colour
format. This wouldn't be ok for an application to do itself, but within the
driver we know that it's safe, since there's no difference in memory between
depth/stencil and colour on AGX. In particular, Z16 is compressed exactly the
same as R16, Z8 as R8, and so on.
Fixes depth/stencil compression.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
We go to initialize the disk cache before we've compiled any shaders so
agx_compiler_debug is 0 at this point. Don't try to read it, instead go through
sa safe getter that will do the right thing.
Fixes: 5e9538c12e ("agx: isolate compiler debug flags")
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
When an empty batch is submitted, nothing happens. However, this batch
may have taken over writer status for some BOs which still have a
pending submitted batch that hasn't finished yet. If we drop writer
status at this point, two bad things happen:
- We spuriously clear bo->writer_syncobj, which breaks syncing on
post-facto BO exports
- We break agx_sync_writer(), since we no longer know about the old
writer to properly block on it.
To fix this (hopefully rare) case, take advantage of bo->writer_syncobj
to find the currently submitted writer batch again, and revert the
writer to it. If this turns out to be common and a performance issue
iterating through submitted batches for each written BO, we could
implement it with two writer batch arrays instead, one for active
writers and one for submitted writers... but hopefully that isn't
necessary.
This splits the cleanup path in agx_batch_cleanup() depending on whether
the cleanup is for a reset or proper completion. Since this is only used
within agx_batch.c, drop the public prototype while we're at it.
Signed-off-by: Asahi Lina <lina@asahilina.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22891>
