Georg Lehmann
e0bcab953d
nir: add amd shared append/consume
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31075 >
2024-09-19 16:21:47 +00:00
Alyssa Rosenzweig
4941d71846
nir/divergence_analysis: handle load_agx
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30981 >
2024-09-02 23:27:14 +00:00
Ian Romanick
c160ed212e
nir/divergence: resource_intel is less divergent than you thought
...
When the non_uniform flag is not set, the result is never divergent.
v2: Remove redundant assignment to is_divergent. Suggested by Caio.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org >
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30251 >
2024-08-30 03:39:30 +00:00
Karol Herbst
fc88f04ba1
vtn, nir: handle OpImageQueryLevels on images
...
This is needed for cl_khr_mipmap_image, specifically the OpenCL C
function get_image_num_mip_levels.
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Reviewed-by: Jesse Natalie <jenatali@microsoft.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30834 >
2024-08-27 15:06:17 +00:00
Lionel Landwerlin
2158fe2ae2
nir/divergence: add missing load_constant_base_ptr
...
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com >
Cc: mesa-stable
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30712 >
2024-08-27 01:33:52 +00:00
Konstantin Seurer
ce24486ee4
nir: Introduce nir_debug_info_instr
...
Adds a new instruction type that stores metadata that might be useful
for debugging purposes. Passes must ignore these instructions when
making decisions.
Reviewed-by: Jesse Natalie <jenatali@microsoft.com >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18903 >
2024-08-25 10:26:33 +00:00
Lionel Landwerlin
cf986dd589
nir: remove unused intel intrinsics
...
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com >
Reviewed-by: Ivan Briano <ivan.briano@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30713 >
2024-08-22 19:44:40 +00:00
Lionel Landwerlin
fbafa9cabd
intel/nir: remove load_global_const_block_intel intrinsic
...
load_global_constant_uniform_block_intel is equivalent in terms of
loading, then for the predicate we just do a bcsel afterward in places
where that is required.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com >
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30659 >
2024-08-16 11:12:39 +00:00
Job Noorman
e0bad1dd20
ir3: replace @load_uniform by new @load_const_ir3 intrinsic
...
Uniforms are a legacy thing and this intrinsic was only used to load
from const registers so the new naming is less confusing.
Signed-off-by: Job Noorman <jnoorman@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28341 >
2024-08-15 12:07:27 +00:00
Alyssa Rosenzweig
f04ae930d9
nir,agx: add "active threads in subgroup" intrinsic
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30633 >
2024-08-12 18:45:58 -04:00
Alyssa Rosenzweig
0566e9a51f
nir/divergence_analysis: handle derivative intrinsics
...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Reviewed-by: Jesse Natalie <jenatali@microsoft.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30565 >
2024-08-08 15:26:07 +00:00
Alyssa Rosenzweig
340831dbcc
nir/divergence_analysis: handle AGX stuff
...
bunch of vendor intrinsics, plus some standard intrinsics used in weird shader
stages.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Acked-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30488 >
2024-08-06 11:48:18 -04:00
Marek Olšák
b2d32ae246
nir: add nir_intrinsic_load_per_primitive_input, split from io_semantics flag
...
Instead of having 1 bit in nir_io_semantics indicating a per-primitive
FS input, add a dedicated intrinsic for it.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29895 >
2024-07-23 16:13:16 +00:00
Georg Lehmann
2d3f536174
aco,nir: add dpp16_shift_amd intrinsic
...
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24650 >
2024-07-17 15:04:38 +00:00
Marek Olšák
1b2cd628b8
nir: rename ordered_xfb_counter_add_gfx12_amd -> ordered_add_loop_gfx12_amd
...
because it can also be used by compute.
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30063 >
2024-07-13 01:32:48 +00:00
Connor Abbott
ec37e65a2d
ir3: Introduce elect_any_ir3
...
For preambles, we don't actually care which invocation we get, so we
don't have to enable helper invocations when the preamble uses "getone."
Introduce a new intrinsic with the right semantics and plumb it through.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29914 >
2024-06-26 17:40:15 +00:00
Faith Ekstrand
7e3d157bee
nak,nir: Drop r2ur_nv in favor of as_uniform
...
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29737 >
2024-06-15 06:14:27 +00:00
Faith Ekstrand
b107240474
nir: Add some new _nv intrinsics
...
The ldc_nv and ldcx_nv intrinsics correspond to the index and bindless
forms of NVIDIA's LDC instruction, respectively. ldc_nv is pretty much
load_ubo without some of the unnecessary constant bits while ldcx_nv
takes a 64-bit bindless handle instead of an index. The other two give
us a little control over register allocation at the NIR level to ensure
that LDCX handles are placed in uniform registers.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29591 >
2024-06-13 20:43:45 +00:00
Konstantin Seurer
a93f95c69c
radv/rt: Remove load_rt_dynamic_callable_stack_base_amd
...
Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28619 >
2024-05-28 12:23:45 +00:00
Lionel Landwerlin
2be28ee58a
nir: add a base offset for printf indexing
...
This will allow a driver to use a single table of printf strings
across all shaders.
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com >
Reviewed-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Ivan Briano <ivan.briano@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25814 >
2024-05-15 13:13:37 +00:00
Lionel Landwerlin
8d336f069e
nir/divergence: add missing load_printf_buffer_address
...
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com >
Cc: mesa-stable
Reviewed-by: Karol Herbst <kherbst@redhat.com >
Reviewed-by: Ivan Briano <ivan.briano@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25814 >
2024-05-15 13:13:37 +00:00
Juan A. Suarez Romero
87cd11ecd2
nir,v3d: rename tlb_color_v3d intrinsic
...
As this is intended to be used also by VC4, change the suffix to
something more convenient, like tlb_color_brcm.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com >
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29119 >
2024-05-13 10:44:17 +00:00
Marek Olšák
b06a71b3cd
nir: add streamout intrinsics for AMD GFX12
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-By: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28889 >
2024-04-30 17:17:25 +00:00
Karol Herbst
d22f936019
nir: remove workgroup_id_zero_base
...
This removes the need for drivers to handle both versions. The base will
get added once in nir_lower_system_values when converting from deref to
intrinsic and will be replaced by a zero for users not supporting it.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Signed-off-by: Karol Herbst <kherbst@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26800 >
2024-04-24 20:18:49 +00:00
Karol Herbst
3217838fef
nir: remove global_invocation_id_zero_base
...
This removes the need for drivers to handle both versions. The base will
get added once in nir_lower_system_values when converting from deref to
intrinsic and will be replaced by a zero for users not supporting it.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Signed-off-by: Karol Herbst <kherbst@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26800 >
2024-04-24 20:18:49 +00:00
Karol Herbst
3625a44dcc
nir/divergence_analysis: handle load_base_global_invocation_id
...
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Signed-off-by: Karol Herbst <kherbst@redhat.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26800 >
2024-04-24 20:18:48 +00:00
Marek Olšák
c1f750eed9
nir: add nir_intrinsic_optimization_barrier_sgpr_amd
...
for radeonsi
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28606 >
2024-04-13 16:45:08 +00:00
Connor Abbott
42ddbf4ebe
nir/divergence_analysis: Fix load_view_index divergence in VS
...
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28573 >
2024-04-08 23:20:08 +00:00
Connor Abbott
223e3727d5
nir/divergence_analysis: Add uniform_load_tears option
...
This "tear" is similar to the original concept of loads/stores tearing,
but across invocations in a wave instead of bytes. Qualcomm seems to
have this problem, at least for some GPUs. This fixes
spec@arb_shader_storage_buffer_object@execution@ssbo-atomiccompswap-int
on a630 once we start relying on divergence analysis for computing
reconvergence properties. For backends that have readFirstInvocation(),
it should be possible to fix the problem by inserting
readFirstInvocation() instead, but a5xx doesn't have it so we can't rely
on it in freedreno.
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28573 >
2024-04-08 23:20:08 +00:00
Connor Abbott
291e6d2940
nir/divergence_analysis: Add ir3-specific intrinsics
...
Reviewed-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28573 >
2024-04-08 23:20:08 +00:00
Rhys Perry
543ca160a5
nir,aco: add test intrinsics
...
These don't really do anything. They're just a source and user of SSA
defs.
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28301 >
2024-04-08 18:38:39 +00:00
Timur Kristóf
411de8488c
nir: Add two new AMD specific tess intrinsics.
...
These will be needed to implement some tessellation dynamic
states within the TCS as opposed to using an epilog.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28408 >
2024-03-28 23:44:03 +00:00
Mike Blumenkrantz
2b2f74ab35
nir/divergence: add zink intrinsics
...
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28429 >
2024-03-27 20:55:07 +00:00
Marek Olšák
1585a5cc6d
nir,amd: add nir_intrinsic_load_debug_log_desc_amd and its use
...
for shader debugging
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27952 >
2024-03-22 21:58:02 +00:00
Marek Olšák
6773595ed0
nir: rename AMD XFB intrinsics to *_gfx11_amd
...
to indicate it's only for gfx11.
Acked-by: Alyssa Rosenzweig <alyssa@rosenzweig.io >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27952 >
2024-03-22 21:58:02 +00:00
Marek Olšák
813f37a8ed
nir: add nir_block::divergent to indicate a divergent entry condition
...
to be used by nir_opt_varyings
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28049 >
2024-03-12 00:29:03 +00:00
Marek Olšák
867a0a7db9
nir/divergence_analysis: handle derefs of system values
...
needed by GLSL compiler optimizations that have unlowered sysvals
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com >
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28049 >
2024-03-12 00:29:03 +00:00
Marek Olšák
eb670d6eaf
nir/divergence_analysis: load_instance_id is convergent within a primitive
...
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com >
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28049 >
2024-03-12 00:29:03 +00:00
Marek Olšák
310b13b7f0
nir/divergence_analysis: load_primitive_id is convergent within a primitive
...
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com >
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28049 >
2024-03-12 00:29:03 +00:00
Marek Olšák
1621d4a0d3
nir/divergence_analysis: change function prototypes
...
for following commits
Reviewed-by: Konstantin Seurer <konstantin.seurer@gmail.com >
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28049 >
2024-03-12 00:29:03 +00:00
Juan A. Suarez Romero
62e1dff256
v3d: add load_fep_w_v3d intrinsic
...
This intrinsic helps to read the W coordinate stored in the QPU register
when initializing the input data for the fragment shaders.
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com >
Signed-off-by: Juan A. Suarez Romero <jasuarez@igalia.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/28072 >
2024-03-11 12:42:49 +00:00
Georg Lehmann
1d8b2b159e
nir/divergence_analysis: fix subgroup mask
...
These depend on the subgroup invocation id, so they are divergent.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Reviewed-by: Faith Ekstrand <faith.ekstrand@collabora.com >
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com >
Fixes: df86c5ffb3
("nir: add divergence analysis pass.")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27962 >
2024-03-05 14:52:17 +00:00
Ian Romanick
75de4458a1
nir: Mark nir_intrinsic_load_global_block_intel as divergent
...
This is divergent because it specifically loads sequential values into
successive SIMD lanes.
No shader-db or fossil-db changes on any Intel platform.
Fixes: 9f44a26462
("nir/divergence: handle load_global_block_intel")
Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27044 >
2024-02-27 08:36:42 -08:00
Timur Kristóf
cc1501628f
nir: Clean up divergence analysis for TES patch input loads.
...
Just make the code a little bit easier to follow.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27680 >
2024-02-26 14:53:23 +00:00
Timur Kristóf
870a2e4197
nir: Cleanup divergence analysis for mesh shaders.
...
1. Mesh shaders don't have inputs (only task payload),
so remove them from handling load_input.
2. Clarify in comments that loading any mesh shader
output is an NV_mesh_shader only feature.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27680 >
2024-02-26 14:53:23 +00:00
Timur Kristóf
9553d67373
nir: Fix divergence analysis of load_patch_vertices_in.
...
load_patch_vertices_in can only occur in tessellation shaders,
and contains the number of vertices in an input patch.
* TCS: patch_vertices_in is equal to the input patch size
* TES: patch_vertices_in is equal to the TCS output patch size
The patch sizes may be set by a pipeline or dynamic states,
however in both cases it is definitely uniform within a subgroup.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27680 >
2024-02-26 14:53:23 +00:00
Timur Kristóf
537c0029dd
nir: Fix divergence of reductions.
...
By accident, the function would return without setting
the divergence information.
Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27680 >
2024-02-26 14:53:23 +00:00
Bas Nieuwenhuizen
c7b2ac3377
radv: Remove ray_launch_size_addr_amd system value.
...
Not used anymore, so clean it up.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27664 >
2024-02-17 11:08:16 +00:00
Marek Olšák
e98bbcad17
nir: add vertex divergence into nir_divergence_analysis
...
This is a prerequisite for the new nir_opt_varyings pass.
It reuses the same divergent field in nir_def and nir_loop.
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26918 >
2024-02-02 16:45:52 -05:00
Georg Lehmann
1cb5bf7009
nir: add ballot_relaxed and as_uniform intrinsics
...
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27116 >
2024-01-19 20:13:33 +00:00