Commit Graph

169680 Commits

Author SHA1 Message Date
Timur Kristóf
b51e858d68 radv/amdgpu: Place secondary CS without IB2 in non-WC GTT.
When using a secondary CS without IB2, we have to memcpy the
contents into the primary CS. Use these flags to improve perf
by preventing memcpy from VRAM.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
7c5bf91350 radv/amdgpu: Extract radv_amdgpu_cs_bo_create function.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
0c1d4130ca radv/amdgpu: Add bool is_secondary argument to cs_create function.
Also save is_secondary to the CS object.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
c0fe6bbef1 radv: Allow task/mesh shaders with RADV_DEBUG=noibs.
The new submit code path should now be able to handle gang submit
even when chaining is not enabled.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
dc59a6b2ee radv/amdgpu: Respect maximum number of submitted IBs per IP type.
In a gang submit, the maximum number of IBs is per IP type,
and is different for each queue. Let's respect that.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
e4a37547a5 radv/amdgpu: Clean up submission functions.
Remove a useless extra function call, and rename fallback to
internal because it is the only code path left.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
50e6b16855 radv/amdgpu: Use fallback submit for queues that can't use IBs.
This commit consolidates the queue submit code paths into one.

Now we always allocate BOs for every CS, but when IBs aren't
allowed, we simply submit every BO to the kernel.

A microbenchmark done by Bas indicated that submitting more IBs to
the kernel only adds a negligible overhead. Additionally, this
allows us to stop copying the command buffer contents in system
memory and get rid of a lot of legacy code.

In order to be able to submit every BO, we make sure to add the
last BO to the old_ib_buffers array on cs_finalize. This also
necessitates some changes in cs_execute_secondary and other
functions.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
958ea2ec2a radv: Split submission in winsys instead of radv_queue.
This will still make it so that RADV_DEBUG=hang will only submit
one command buffer at a time, but otherwise let's pass all CS
objects into one submission and let the winsys split them if
necessary.

The winsys can do a better job at splitting them because
radv_queue has no knowledge of IBs and ignores chaining in the
splitting logic.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
41a9bced31 radv: Fill continue preambles and postambles properly.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
1bdc1bd36c radv/amdgpu: Split gang submissions correctly when not chained.
In a gang submit, the follower (typically ACE) and leader
(typically GFX) can have synchronization between each other.
We must ensure that these end up in the same submission,
otherwise we can deadlock the GPU.

We rely on radv_queue here to order follower before the leader
in the submitted CS array.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
813885f682 radv/amdgpu: Add ability to submit non-chained CS to fallback.
When submitting a CS whose IP type doesn't support chaining,
let's submit all the IBs that this CS allocated.

This is going to be better than the sysmem code path because it
doesn't require the winsys to memcpy the contents of the command
buffers.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:03 +00:00
Timur Kristóf
e4dd6a8eb3 radv/amdgpu: Add postambles to fallback submit.
Some new features, such as perf counters and gang submit use
postambles. Implement these properly.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
103ebe96c0 radv/amdgpu: Add continue preambles to fallback submit.
When splitting a larger submission into several smaller ones,
the flushing at the beginning of the initial preambles is not
really necessary, so it'll be better to use the continue
preambles for this purpose.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
6844506c3d radv/amdgpu: Allow multiple continue preambles.
For feature parity with initial preambles.
Previously, continue preambles were for GFX6 only, but this is
about to change in the next commits.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
8cea452bda radv/amdgpu: Rewrite fallback code path so it can split submissions.
Currently, radv_queue already splits submissions but we want to
change this and be able to split them in the winsys code as well.
Necessary because we want to split based on number of actual
IBs not number of command buffers, but radv_queue is not
aware of IBs.

Note that this commit does not actually take this new split into
use yet, that will be done in a following commit when it is ready,
this is why we set the max IB count higher than radv_queue here.

This commit is the first step in making "fallback" the default and
only submission code path.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
df90102a15 radv/amdgpu: Remove hw_can_chain in favour of use_ib.
They have the same meaning, but use_ib makes more sense because it
also takes the gfx_level into account.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
ccb4b78840 radv/amdgpu: Add a few assertions during submit.
These are useful when debugging the submission code.

- Ensure correct IB alignment
- Ensure we don't submit empty IBs

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
e5201987b1 radv/amdgpu: Extract radv_amdgpu_cs_add_old_ib_buffer.
This function will be used elsewhere in a following commit.

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
44de40fb5f radv/amdgpu: Use correct alignment when creating CS BOs.
Shouldn't matter in practice because the kernel will likely
give us a page-aligned BO, but better to specify it just
in case.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
e214059120 radv/amdgpu: Only allow IB BOs on graphics and compute queues.
This disallows IB BOs on eg. SDMA queues which was previously
mistakenly left out.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
a7080fe4ef radv/amdgpu: Fix mismatching return type of radv_amdgpu_cs_submit.
Fixes a warning that is reported by GCC 13.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
daeb409559 ac: Add maximum number of submitted IBs.
The number of IBs per submit isn't infinite, it depends on the IP type
(ie. some initial setup needed for a submit) and the packet size.
It can be calculated according to the kernel source code as:
(ring->max_dw - emit_frame_size) / emit_ib_size

Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
Timur Kristóf
87d22933ae radv: Create continue preamble on GFX6 even when no shader rings are used.
Skipping the continue preamble can allow other processes to mess
up some	registers set by the current process.

Originally, we could omit generating the continue preamble when
no shader rings were used, because the register initialization
happened at the beginning of every main cmdbuf. However, this
isn't the case anymore.

Cc: mesa-stable
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22354>
2023-04-11 17:05:02 +00:00
xurui
0bba139767 zink: bs->dd.push_pool[1].pool should be freed
cc: mesa-stable

Signed-off-by: xurui <xurui@kylinos.cn>
Reviewed-by: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22389>
2023-04-11 16:16:05 +00:00
Patrick Lerda
f9401a515a egl: fix memory leak related to _eglRefreshDeviceList()
Indeed, the unnecessary drmDevice objects were not freed.

For instance, this issue could be triggered with: "piglit/bin/egl_ext_platform_device -auto -fbo":
SUMMARY: AddressSanitizer: 2796 byte(s) leaked in 12 allocation(s).

Fixes: e39d72aec2 ("egl: only take render nodes into account when listing DRM devices")
Signed-off-by: Patrick Lerda <patrick9876@free.fr>
Reviewed-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22408>
2023-04-11 15:48:26 +00:00
David Heidelberg
1fa1c285fc ci/amd: add draw.dynamic_rendering flake
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22411>
2023-04-11 15:10:37 +00:00
Eric Engestrom
4b182dbcf2 v3dv/ci: mark known dEQP-VK.wsi.xlib.surface.query_formats failure
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22410>
2023-04-11 13:39:31 +00:00
Bas Nieuwenhuizen
a8cb837703 radv: Align atomic values.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22121>
2023-04-11 12:57:15 +00:00
Bas Nieuwenhuizen
02c46fa1e8 util/disk_cache: Align atomic size.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22121>
2023-04-11 12:57:15 +00:00
Bas Nieuwenhuizen
1c2d90f17a util: Add aligned int64_t types for x86(non 64).
To avoid split locks.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22121>
2023-04-11 12:57:15 +00:00
Samuel Pitoiset
9ccaf5583f radv: fix re-emitting vertex user SGPRs when binding a graphics pipeline
The base SGPR and the number of SGPRs can be equal but it was incorrect
because one VS can have draw_id and one can have base_instance. Fix
this by invalidating the vertex user SGPRs unconditionally.

Though they should also be invalidated after executing secondaries,
otherwise nothing is invalidated if the same pipeline is bind to the
primary again.

This fixes dEQP-VK.dynamic_rendering.primary_cmd_buff.random.seed*.

Signed-off-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21652>
2023-04-11 12:12:05 +00:00
Daniel Schürmann
35805d172a radv/ci: add 2 more Flakes for Navi21
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
dcfca69e67 radv: clean up pipeline-cache interface
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
c2b9279227 radv: use vk_pipeline_cache
Replaces the RADV pipeline cache with an implementation
based on the common vk_pipeline_cache.

We use a dual-layer approach with two types of cache entries.

  1. radv_shader:
    - serialized as radv_shader_binary
    - uses SHA1 of the binary as key
  2. radv_pipeline_cache_object:
    - contains pointers to associated radv_shaders
    - serialized as list of SHA1
    - uses the pipeline hash as key

In combination with single-file disk-cache, this reduces the cache size by ~60%.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
9b3679aff2 radv: implement radv_shader_create_cached()
This function takes a radv_shader_binary and writes it to the
disk cache before creating and returning a radv_shader cache entry.
The key of the cache entry is the full SHA1 hash of the binary.
This way, we will be able to deduplicate identical shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
55cc2fb088 radv: add struct radv_pipeline_cache_object
This patch implements a per pipeline cache-entry which contains
pointers to the associated shaders.
The serialized format contains the binary hashes of the shaders.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
577d76f60f radv: add radv_shader_serialize() and radv_shader_deserialize() functions
These functions convert between radv_shader and radv_shader_binary.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
efbf0e70eb radv: unconditionally store the binary code in radv_shader
We will (temporarily) use it to serialize shader binaries
with the common vk_pipeline_cache.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Daniel Schürmann
5303a57964 radv: derive struct radv_shader from vk_pipeline_cache_object
We will use radv_shader directly as cache entries.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22030>
2023-04-11 11:38:15 +00:00
Lionel Landwerlin
daa8003e45 intel/fs: use nomask for setting cr0 for float controls
The instructions manipulation cr0 use the default mask on lane0. So if
for some reason that lane is disabled in some of the dispatchs, we can
end up not executing the instructions.

Fixes flakyness in dEQP-VK.spirv_assembly.instruction.graphics.16bit_storage.uniform_float_32_to_16.uniform_matrix_float_rtz_frag

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Francisco Jerez <currojerez@riseup.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22314>
2023-04-11 11:01:31 +00:00
Corentin Noël
45a682650c mesa: Add EXT_instanced_arrays support
The support for ARB_instanced_arrays already exists for big GL, reuse the same
PIPE_CAP_VERTEX_ELEMENT_INSTANCE_DIVISOR for this extension.

The OES_draw_elements_base_vertex and EXT_draw_elements_base_vertex both define
the DrawElementsInstancedBaseVertex method when EXT_instanced_arrays is defined.

Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22167>
2023-04-11 10:22:35 +00:00
Corentin Noël
8aa4d37b83 glapi: Make EXT_draw_instanced functions available for GLES 2.0
The EXT suffixed functions should be available in the GLES 2.0 context.

Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22167>
2023-04-11 10:22:35 +00:00
Corentin Noël
b3793dc442 mesa: OpenGL ES 3.0 requires EXT_instanced_arrays
Add it to the list of the required extensions.

Signed-off-by: Corentin Noël <corentin.noel@collabora.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Acked-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22167>
2023-04-11 10:22:35 +00:00
Lionel Landwerlin
cff71ae8ff anv: fixup streamout write barriers
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Cc: mesa-stable
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/8796
Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22336>
2023-04-11 09:53:10 +00:00
Konstantin Seurer
2bf9ae78c5 radv: Remove radv_bvh_aabb_node::aabb
It was only read by RRA which can infer it from the parenbt internal
node.

Change in average build time (Control):
84.69471 ms -> 84.25319 ms

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22400>
2023-04-11 07:57:34 +00:00
Konstantin Seurer
1ce50f0774 radv/bvh: Remove calculate_node_bounds
It is unused and will prevent removing the aabb field from
radv_bvh_aabb_node.

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22400>
2023-04-11 07:57:34 +00:00
Konstantin Seurer
f791cd9e43 radv: Pack and encode geometry id and flags on the CPU
There is no need to do it on the GPU.

Change in average build time (Control):
84.80691 ms -> 84.69471 ms

Reviewed-by: Friedrich Vock <friedrich.vock@gmx.de>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22400>
2023-04-11 07:57:34 +00:00
Konstantin Seurer
8391639b5f radv: Remove has_previous_stage
MESA_SHADER_NONE implies that has_previous_stage is false.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22382>
2023-04-11 06:24:17 +00:00
Konstantin Seurer
b4bae8a2c0 aco: Remove is_gs_copy_shader
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22382>
2023-04-11 06:24:17 +00:00
Konstantin Seurer
58ab783153 radv: Remove some dead radv_shader_args setup
radv_declare_shader_args already sets them up.

Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22382>
2023-04-11 06:24:17 +00:00