Alyssa Rosenzweig
b45d54a388
agx: optimize b2x(inot)
...
easier to do as algebraic rules.
total instructions in shared programs: 2915611 -> 2910477 (-0.18%)
instructions in affected programs: 285251 -> 280117 (-1.80%)
helped: 1245
HURT: 5
Instructions are helped.
total bytes in shared programs: 19439752 -> 19400010 (-0.20%)
bytes in affected programs: 1881556 -> 1841814 (-2.11%)
helped: 1244
HURT: 6
Bytes are helped.
total halfregs in shared programs: 519253 -> 519553 (0.06%)
halfregs in affected programs: 5509 -> 5809 (5.45%)
helped: 24
HURT: 107
Halfregs are HURT.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
34ca925064
agx: don't inline imms into stack_store
...
we need opt_large_consts..
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
f6c17cbe48
ail: use template for tiled memcpy
...
less macro hell
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
075e595058
ail: port tiling routines to c++
...
to fix macro hell
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
104ba72547
ail: add tests for linear<-->twiddled copies
...
based on the panfrost tests.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
2c9189434f
docs/asahi: fix strided linear note
...
turns out linear 2d arrays work
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
f836d573e4
agx: only run early tests if needed
...
if it'd be early anyway don't bother, speeds up dolphin with spec shaders
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
7825044c39
asahi: honour discard_whole | persistent
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
793ac04077
asahi: handle read-only SSBOs
...
reduces silliness in Yuzu.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
97a945246c
asahi: fix valid buffer tracking for SSBO/image/XFB
...
smaller ranges.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
6689b083d8
asahi: don't sync for uninitialized buffer
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
9e8ce4550c
asahi: enable tcs caching
...
works fine
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
19da2f6d30
asahi: sync with query mismatches
...
otherwise the following assertion can fail. fixes crash in yuzu trace.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Alyssa Rosenzweig
3b927567ac
Revert "asahi: don't canonicalize nans/flush denorms when copying"
...
This reverts commit ab7fb3d400
.
This breaks ./arb_copy_image-formats -auto --samples=4 since we need the blit
path for compression transitions. The correct thing to do is cast the formats.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Asahi Lina
74110ef21d
asahi: batch: Trace before waiting for syncobj
...
If the render hangs, at least this way we get the trace.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Asahi Lina
ff900ceffe
asahi: Enable scratch debugging
...
Via ASAHI_MESA_DEBUG=scratch. This will assert if enabled and the
scratch workgroup allocation count turns out not to be sufficient (that
is, there were failed allocations), to help debug the max occupancy
calculation.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Asahi Lina
8845702f8c
asahi: Allocate scratch for shaders
...
Finally, now that scratch is available, tell the hardware how much we
need and allocate the scratch buffers as necessary.
We only do the max subgroup calculation for compute right now, and it's
probably conservative (we could probably calculate a reduced occupancy
depending on register pressure). Getting this too low won't break, the
GPU will just be less efficient if there is not enough scratch
allocated. Vertex and fragment just get the max 96 for now.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Asahi Lina
bb5277aa3d
asahi: Hook up scratch
...
Instantiate our 3 scratch managers for each shader type, and pass them
to the hardware if a batch uses scratch.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Asahi Lina
fa475c1b56
asahi: scratch: Add feature to debug core IDs
...
Define SCRATCH_DEBUG_CORES to a raw core count to bypass the mapping
machinery and allocate memory for many cores, so we can work out how the
core IDs are mapped.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:30 +00:00
Asahi Lina
494399c65c
asahi: Implement scratch allocation
...
Implement our helper program to map scratch blocks into stack memory,
and the driver side that allocates these blocks as necessary.
Alloction is grow-only right now. Drivers are expected to instantiate
scratch memory managers for each shader type (VS, FS, CS) and the same
buffers are reused across commands for each one, growing as necessary.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
da9677f14b
asahi: cmdbuf: Fix scratch bucket offset/size
...
This is 4 bits. It is an opaque value that gets passed to the
helper, except 0 means no scratch.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
d6cc820d1a
asahi: libagx: Move PACKED and GLOBAL macros to libagx.h
...
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
eec7c4beb8
agx: compiler: Enable stack_adjust
...
We'll need this once we start using scratch.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
1f1751d258
agx: compiler: Export scratch size to the driver
...
So it can configure the pipeline correctly, and allocate the required
scratch space.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
b89da92a5e
agx: compiler: Add fence_helper_exit_agx barrier
...
This is used by the helper program on exit.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
c406ce793b
agx: Add scaffolding to build the helper shader at device init
...
Add the scaffolding to compile our helper program, and load it at device
init time.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
05c8b59f5b
agx: Hook up helper intrinsics into CL
...
So we can use these intrinsicts from libagx.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
30862d7189
agx: Hook up AGX helper NIR intrinsics
...
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
b07dbf7b0f
nir: Add AGX-specific helper opcodes
...
These opcodes are used by the helper program to fetch the current
operation info and core ID.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
5d0f1ca77f
agx: Rename some SRs
...
I think these better represent what these SRs actually do.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Asahi Lina
28d34f6352
asahi: libagx: introduce AGX_STATIC_ASSERT
...
Using the array size trick, this works in both OpenCL and C.
Signed-off-by: Asahi Lina <lina@asahilina.net >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
8661006ef0
agx: add some more bitop tests
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Mary Guillemard
923767a968
agx: Add a bitop optimizer pass
...
Signed-off-by: Mary Guillemard <mary@mary.zone >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Mary Guillemard
2b89eb979e
agx: Fuse not into and/or/xor
...
Signed-off-by: Mary Guillemard <mary@mary.zone >
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Mary Guillemard
ba508fe854
agx: Remove and/or/xor pseudo ops
...
Switch back to bitop while keeping the aliases on agx_print_instr.
Also add all variants of 2 args of agx_bitop_table.
Signed-off-by: Mary Guillemard <mary@mary.zone >
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Mary Guillemard
a5b6ff3ccc
agx: Add more bitops in agx_bitop_table
...
Signed-off-by: Mary Guillemard <mary@mary.zone >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
62001e175f
asahi: fix UB in qbo's
...
upper bits may be uninitialized. fixes
KHR-GL46.transform_feedback_overflow_query_ARB when built with gcc (passed by
chance with clang).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
a06b51559b
asahi: plug geometry heap leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
877c180677
asahi: plug glsl type leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
06cf222830
asahi: plug so target leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
e2c5a93358
asahi: plug early_serialized_nir leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
ffac4e7179
asahi: plug pre-gs leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
7b44d15e5f
asahi: plug geometry shader leaks
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
1e973dd5a9
asahi: rework meta shader infra
...
this fixes the leaks, and is also a lot more pleasant to work with.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
7eda3da983
asahi: plug passthrough tcs leak
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
e91c0b6423
asahi: delete bogus assertion
...
dEQP-GLES3.functional.fragment_ops.scissor.outside_render_line at certain render
sizes.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
5261962186
asahi: fix prim restart unrolling with indirects
...
need to account for draw->start.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
c0d3bf3608
agx: optimize vote_eq
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
0de7018879
agx: optimize first_invocation
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00
Alyssa Rosenzweig
fb81201a34
agx: implement active_subgroup_invocation_agx
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27616 >
2024-02-14 21:02:29 +00:00