Timur Kristóf
85eab189ee
ac/nir: Move ac_nir_opt_pack_half to separate file.
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:46:15 +01:00
Timur Kristóf
88c951bd46
ac/nir: Move ac_nir_lower_legacy_gs to separate file.
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:46:11 +01:00
Timur Kristóf
6dd3f53204
ac/nir: Move ac_nir_lower_legacy_vs to separate file.
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:46:10 +01:00
Timur Kristóf
d0e71ac9cd
ac/nir: Move ac_nir_lower_intrinsics_to_args to separate file.
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:46:08 +01:00
Timur Kristóf
a0b226bafb
ac/nir: Expose ac_nir_unpack_value in ac_nir_helpers.h
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:46:06 +01:00
Timur Kristóf
1181348e80
ac/nir: Move ac_nir_create_gs_copy_shader to separate file.
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:46:04 +01:00
Timur Kristóf
4cad0bc438
ac/nir: Rename emit_streamout to ac_nir_emit_legacy_streamout
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:46:01 +01:00
Timur Kristóf
305fdfddb5
ac/nir: Move ac_set_nir_options to ac_nir.c
...
And rename it to ac_nir_set_options to match other functions.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:45:34 +01:00
Timur Kristóf
855de0483f
ac/nir: Move ac_nir callback functions to ac_nir.c
...
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:45:32 +01:00
Timur Kristóf
cc0166462e
ac/nir: Move ac_nir_get_mem_access_flags to ac_nir.c
...
And change its name to indicate that it is NIR specific.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:45:30 +01:00
Timur Kristóf
ad5c0b7103
ac/nir: Move ac_nir_lower_bit_size_callback to ac_nir.c
...
ac_shader_util should not concern itself with NIR stuff.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32966 >
2025-01-14 13:45:28 +01:00
Marek Olšák
4f2e2e10bc
ac/nir: vectorize streamout stores for legacy pipeline optimally
...
Walk the whole vertex stride thanks to XFB info sorted by offset, gather
individual components from same or different outputs, and once we have
gathered 4, store them as vec4.
It also removes the COHERENT flag from VMEM stores because NGG streamout
doesn't use it either and I don't think it's needed.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32686 >
2025-01-09 20:47:16 +00:00
Marek Olšák
e399f3bed9
ac/nir: sort xfb info to facilitate vectorization of xfb stores
...
xfb stores are not vectorized properly, leading to generating random soup
of b32, b64, b96, and b128 stores.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32686 >
2025-01-09 20:47:16 +00:00
Samuel Pitoiset
f09f31d093
ac/nir: fix a comment typo in load_subgroup_id_lowered()
...
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32940 >
2025-01-09 08:02:19 +00:00
Samuel Pitoiset
44ba856089
ac/nir: fix lowering subgroup ID for compute shaders on GFX12
...
This is lowered in backend compilers (LLVM or ACO) because it needs
to access ttmp registers which aren't exposed to NIR.
Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32940 >
2025-01-09 08:02:19 +00:00
Marek Olšák
7fbca998b1
amd: optimize atomics before lowering intrinsics
...
ac_nir_lower_intrinsics_to_args will lower most system values.
I have to keep the divergence analysis in ACO, otherwise it goes haywire.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:56 +00:00
Marek Olšák
5dd9171765
ac/nir: set upper ranges for range analysis while lowering system values
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
0d5b03f2b9
ac/nir: split local_invocation_ids to 3 separate VGPR inputs
...
so that we can set the upper range per VGPR.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
65d241c947
ac/nir: set arg_upper_bound_u32 for vs_rel_patch_id
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
1d9fbe5387
ac/nir: add helper ac_nir_load_arg_upper_bound
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
cfeaa45dc6
ac/nir: clean up ac_nir_lower_indirect_derefs
...
IO variables can't occur here anymore.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
ae22da2ff8
ac/nir: lower more loads in ac_nir_lower_intrinsics_to_args instead of drivers
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
ceb6f8fc32
amd: lower load_tess_rel_patch_id/primitive_id/tess_coord and overwrite.. in NIR
...
The overwrite instruction complicates it a little, which is why these
intrinsics are lowered together.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
61bfb4fa06
amd: lower load_subgroup_invocation in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
e69f47faee
amd: lower load_local_invocation_index in NIR
...
This is the last intrinsic that needed the LS VGPR bug workaround in ACO
and ac_nir_to_llvm.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
433ca6ba38
ac/nir: extract a load_subgroup_id lowered helper
...
this will be used in the next commit
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
342dcbdc8b
amd: lower load_vertex_id/instance_id and overwrite_vs_arguments in NIR
...
2 things complicate this:
- overwrite_vs_arguments_amd
- the LS VGPR bug workaround
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
66dd70adc5
amd: lower load_gs_wave_id_amd in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
923f59c971
amd: lower load_barycentric_at_offset in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
16ab05fad1
amd: lower load_barycentric_pixel/centroid/sample in NIR
...
radeonsi needs to preserve interp_mode in the arg load.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
7e83f6ca8b
amd: lower load_front_face in NIR
...
radeonsi must do this after si_lower_nir_abi, which optimizes front_face,
but doesn't lower it.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
6ad5225b2a
amd: lower load_frag_shading_rate in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
6d2e29ff6e
amd: lower load_sample_pos in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
110e474b4f
amd: lower load_sample_id in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
684c8da553
amd: lower load_invocation_id in NIR
...
ACO can't look for it because it's lowered there.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
d281240c57
amd: lower load_first_vertex/base_instance/draw_id/view_index in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
0d372b043b
amd: lower load_local_invocation_id in NIR
...
This is based on ACO.
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
13cb5c7b72
amd: lower load_frag_coord in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
58cb155068
amd: lower load_pixel_coord in NIR
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Marek Olšák
85c3b5159a
ac/nir: handle disabled PS VGPRs in ac_nir_load_arg_at_offset
...
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32782 >
2025-01-02 17:36:55 +00:00
Timur Kristóf
de2cb4a7d3
ac/nir: Only store params to attribute ring that are varying.
...
On GFX11+, varying outputs from the last pre-rasterization stage
are implemented by storing the outputs to the so-called
attribute ring.
Make sure to only store them when necessary.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640 >
2024-12-28 10:31:41 -06:00
Timur Kristóf
13234a8a8a
ac/nir: Only export parameters when they are actually varying.
...
In AMD terminology, varying outputs are implemented by
parameter export instructions on GFX6-10.3 GPUs.
Only emit those when actually necessary.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640 >
2024-12-28 10:31:38 -06:00
Timur Kristóf
4d6c00944b
ac/nir: Only export positions when they are really system values.
...
In AMD terminology, a system value is implemented by
position export instructions.
Make sure to only emit those when they are needed.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640 >
2024-12-28 10:31:36 -06:00
Timur Kristóf
f5981e8c0b
ac/nir: Split GS output usage masks to varying and sysval masks.
...
To keep track which output is used for what purpose.
Note that this commit just adds the capability to track this
separately in ac/nir. The drivers will need to be updated
in the future to take advantage of this.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640 >
2024-12-28 10:31:33 -06:00
Timur Kristóf
92464109e3
ac/nir: Mark when pre-rast output is used as varying or sysval.
...
In this commit, just collect the info.
It will be taken into use by subsequent commits.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640 >
2024-12-28 10:31:29 -06:00
Timur Kristóf
f38680aa1c
ac/nir: Introduce ac_nir_store_parameters_to_attr_ring.
...
This function is going to be used for storing parameter outputs
to the attribute ring, instead of the current implementation.
It is going to be taken into use in the following commits.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640 >
2024-12-28 10:31:06 -06:00
Timur Kristóf
c4b45f1ec8
ac/nir: Pass ac_nir_prerast_out to ac_nir_export_position.
...
In a subsequent commit, ac_nir_export_position will
start using other fields from ac_nir_prerast_out.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640 >
2024-12-28 10:31:04 -06:00
Timur Kristóf
3d291a98c4
ac/nir: Pass ac_nir_prerast_out to ac_nir_export_parameters.
...
In a subsequent commit, ac_nir_export_parameters will
start using other fields from ac_nir_prerast_out.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com >
Acked-by: Marek Olšák <marek.olsak@amd.com >
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32640 >
2024-12-28 10:31:01 -06:00
Qiang Yu
5f601361ed
ac/nir: lower access for shared and scratch memory
...
OpenCL may load and store vec16 data, while ACO only
support <=32byte. Radeonsi is going to use
ac_nir_lower_mem_access_bit_sizes() for lowering these
access.
Reviewed-by: Marek Olšák <marek.olsak@amd.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32781 >
2024-12-27 01:58:38 +00:00
Rhys Perry
033e76a82a
ac/nir: have ac_nir_lower_mem_access_bit_sizes preserve >128 bit SMEM
...
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com >
Reviewed-by: Georg Lehmann <dadschoorse@gmail.com >
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32408 >
2024-12-09 16:56:29 +00:00