Commit Graph

5452 Commits

Author SHA1 Message Date
David Stevens
d8fdb8dab4 nir: Add colorspace support to YUV lowering pass
This change adds support for BT709 and BT2020 colorspace to the YUV
lowering pass. The default remains BT601.

This change also fixes minor imprecision in the last digits of the BT601
offsets due to computation from rounded values when the math was
simplified.

Reviewed-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6122>
2020-07-31 07:27:03 +00:00
Boris Brezillon
e1b114868a nir: Get rid of __[u]int64_to_fp32() and __fp32_to_[u]int64()
Those are now handled by nir_lower_int64() which has native NIR
implementations.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5588>
2020-07-30 16:54:24 +00:00
Boris Brezillon
936c58c8fc nir: Extend nir_lower_int64() to support i2f/f2i lowering
That's an attempt at replacing the complex __int64_to_float() and
__float_to_int64() implementations found in float64.glsl by a simpler
native NIR equivalent.

Thanks to that, we can have lower those conversion without having to
compile a GLSL shader, which would be quite annoying for OpenCL
kernels.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5588>
2020-07-30 16:54:24 +00:00
Boris Brezillon
bfee35b45c nir: Stop passing an options arg to nir_lower_int64()
This information is exposed through shader->options->lower_int64_options.
Removing the extra arg forces drivers to initialize this field correctly.

This also allows us to check the int64 lowering options from each int64
lowering helper and decide if we should lower the instructions we
introduce.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5588>
2020-07-30 16:54:24 +00:00
Boris Brezillon
dd155aef44 nir: Allow casts in nir_deref_instr_get[_const]_offset()
Allow casts in a deref chain so we can calculate an offset from
a base pointer dereference or have pointer type casts in the
middle of the chain (both are pretty common in CL).

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5682>
2020-07-30 17:11:43 +02:00
Boris Brezillon
6a1382399c nir: Use a switch in build_deref_offset()/deref_instr_get_const_offset()
We are about to add support for casts when calculating offset, but let's
first turn the if()/else if()/else block into a switch() statement to
ease addition of new cases.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5682>
2020-07-30 17:11:35 +02:00
Marcin Ślusarz
59bb0ff945 glsl: catch out of bounds access in the debug version
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6067>
2020-07-30 10:41:00 +00:00
Rhys Perry
6b99cf6064 nir/load_store_vectorize: fix indentation
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5980>
2020-07-29 17:57:13 +00:00
Caio Marcelo de Oliveira Filho
1a42e7dae9 nir: Filter modes of scoped memory barrier in nir_opt_load_store_vectorize
Otherwise a scoped memory barrier containing nir_var_mem_ubo (which
memoryBarrier() does lower to) would incorrectly prevent the
optimization to happen in UBOs.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5980>
2020-07-29 17:57:13 +00:00
Jason Ekstrand
5c5555a862 nir: Add a find_variable_with_[driver_]location helper
We've hand-rolled this loop 10 places and those are just the ones I
found easily.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
fc1363cc60 nir/gl_nir_linker: Call add_vars_with_modes once for GL_PROGRAM_INPUT
Now that nir_foreach_variable_with_modes can handle multiple modes at
one time, we can simplify things a bit and only walk the list once.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
789ec95dcd nir/split_per_member_structs: Inline split_variables_in_list
This lets us do one list walk instead of three.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
d70fff99c5 nir: Use a single list for all shader variables
Instead of having separate lists of variables, roughly sorted by mode,
use a single list for all shader-level NIR variables.  This makes a few
list walks a bit longer here and there but list walks aren't a very
common thing in NIR at all.  On the other hand, it makes a lot of things
like validation, printing, etc. way simpler.  Also, there are a number
of cases where we move variables from inputs/outputs to globals and this
makes it way easier because we no longer have to move them between
lists.  We only have to deal with that if moving them from the shader to
a nir_function_impl.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
77c41ce04b nir/gl_nir_linker: Use nir_foreach_variable_with_modes
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
c256cd900e nir/lower_variable_initializers: Restrict the modes we lower
This is not a functional change because these are the only modes we
handle.  All others get silently ignored.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
4d7e064623 nir/split_per_member_structs: Use nir_variable_with_modes_safe
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
90cf4709d9 nir/lower_uniforms_to_ubo: Use nir_foreach_variable_with_modes
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
d0d5ef6139 nir/lower_two_sided_color: Use nir_variable_create
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
bb3994cfe7 nir/io_to_vector: Use nir_foreach_variable_with_modes
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
0a77c67442 nir/lower_io_to_temporaries: Use a separate list for new inputs
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
cd66005b23 nir: Use a nir_shader and mode in lower_clip_cull_distance_arrays
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
d5d15c301e nir/lower_amul: Add a variable mode check
This loop should only apply to UBOs and SSBOs because max_slot is never
used for normal uniforms.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
3be0be7d54 nir: Split nir_index_vars into two functions
We also very slightly change the semantics.  It no longer is one index
per list for global variables and is a single index over-all.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
86c9303814 nir/split_vars: Add mode checks to list walks
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
4c3a30393d nir/linking: Rework some internal helpers
Instead of taking a variable list, take a nir_shader and mode.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
caab46c1e9 nir: Take a shader and variable mode in nir_assign_io_var_locations
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
964c1c4b87 nir: Take a nir_shader and variable mode in assign_var_locations
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
6f6f7a34c5 nir: Add and use a nir_variable_list_for_mode helper
We also add a new list iterator which takes a modes bitfield and
automatically figures out which list to use.  In the future, this
iterator will work for multiple modes but today it assumes a single mode
thanks to the behavior of nir_variable_list_for_mode.  This also doesn't
work for function_temp variables.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
e3e1c50067 nir: Add a nir_foreach_gl_uniform_variable helper for GL linking
There are a bunch of cases where we really do want to walk the list that
is nir->uniforms because we want all things declared "uniform" in the
GLSL.  Add a helper for this but restrict it to the GL linking code.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Timothy Arceri <tarceri@itsqueeze.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
feb32f898c nir: Add a nir_foreach_uniform_variable helper
This one's a bit more complex because it filters off only those
variables with mode == nir_var_uniform.  As such, it's not exactly a
drop-in replacement for nir_foreach_variable(var, &nir->uniforms).

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
92dcda5ce9 nir: Add a nir_foreach_function_temp_variable helper
Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:58 +00:00
Jason Ekstrand
2956d53400 nir: Add nir_foreach_shader_in/out_variable helpers
Reviewed-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:57 +00:00
Jason Ekstrand
9bf8572222 nir/dead_variables: Respect the modes passed to remove_dead_vars
For the most part, this doesn't actually matter today.  We already only
call remove_dead_vars on the lists that are specified in the modes.  The
only functional change here is for the uniform, mem_ubo, and mem_ssbo
modes because they share a list.  If nir_remove_dead_variables is called
with a mode of nir_var_uniform, it will no longer remove UBOs or SSBOs,
for instance.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:57 +00:00
Jason Ekstrand
5746af4446 nir: Take a mode in remove_unused_io_vars
Reviewed-by: Jose Maria Casanova Crespo <jmcasanova@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5966>
2020-07-29 17:38:57 +00:00
Rhys Perry
2adb337256 nir,radv/aco: add and use pass to lower make available/visible barriers
Lower them to ACCESS_COHERENT to simplify the backend and
probably give better performance than invalidating or writing back the
entire L0/L1 cache.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4905>
2020-07-28 16:56:34 +00:00
Caio Marcelo de Oliveira Filho
12dd5455f4 spirv: Handle most execution modes earlier
For convenience in e68871f6a4 ("spirv: Handle constants and types
before execution modes") we moved all execution mode parsing after the
constants and types, so that those using OpExecutionModeId could be
handled together.

Later in 84781e1f1d ("spirv/nir: keep track of SPV_KHR_float_controls
execution modes") we had to parse certain non-ID execution modes
before handling constants.

Instead of handling just the float controls related execution modes
early, handle all modes that don't need an ID.  This is a more
"natural" split and will allow other type handling to rely on
execution mode in the future.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6062>
2020-07-27 17:10:23 +00:00
Iago Toral Quiroga
17fd191eed nir/lower_clip: make the pass compatible with Vulkan semantics
Vulkan allows fragment shaders to read gl_ClipDistance[], in which
case the SPIR-V compiler inserts a single compact array variable for
VARYING_SLOW_CLIP_DIST0 and the lowering should not try to inject
its own variables, but instead work in terms of the existing one.
Vulkan drivers are expected to call this with use_clipdist_array set
to true to be consistent with this setup.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6022>
2020-07-27 08:25:57 +02:00
Karol Herbst
e2e89fb137 nir/lower_io: assert that offsets are used for shader_in
Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6059>
2020-07-25 08:51:48 +00:00
Jason Ekstrand
63cf8adb12 spirv: Also copy over binding information for atomic counters
I missed this if statement so atomic counters weren't getting bindings
and, when you have more than one of them, that meant they were all
getting combined into one.

Fixes: 3584cb09bc15 "spirv: Give atomic counters their own variable mode"
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6060>
2020-07-24 18:17:38 +00:00
Neil Roberts
56846a2b68 nir/schedule: Add an option for a fallback scheduling algorithm
The current scheduling algorithm favors parallelism a bit too
aggressively and sometimes generates shaders that fail register
allocation. This happens even if the threshold is set to zero to force
it to always use the CSR instruction choosing algorithm.

This patch adds an option to use an even more aggressive fallback that
just always picks the instruction with the shortest maximum delay in the
hope that that will generate the least register pressure. The intention
is to use this as a last resort after register allocation fails in order
to at least have a working shader.

Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Acked-by: Iago Toral Quiroga <itoral@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
2020-07-24 12:26:26 +02:00
Neil Roberts
bafd259177 nir/schedule: Add a callback for backend-specific dependencies
Adds a callback function to nir_schedule_options to give the backend a
chance to add custom dependencies between certain intrinsics. The
callback can assign a class number to the intrinsic and then set a read
or write dependency on that class.

v2: Use a linked-list of schedule nodes for the dependency classes
    instead of a fixed-sized array.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
2020-07-24 09:21:11 +02:00
Neil Roberts
260a8f759a nir/schedule: Store a pointer to the options struct in scoreboard
Instead of copying the individual members of nir_schedule_options into
the scoreboard, it now just keeps a pointer to the options. This avoids
the duplicated comments and makes it easier to add more options later.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
2020-07-24 09:21:11 +02:00
Neil Roberts
7665398e6c nir/scheduler: Move nir_scheduler to its own header
nir_schedule already has a struct for options which makes it more than
just a function declaration. Later patches intend to add more structs to
complement these options. In order to make the code easier to manage,
this moves the nir_scheduler-related parts out of nir.h to their own
header.

Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alejandro Piñeiro <apinheiro@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5953>
2020-07-24 09:21:11 +02:00
Jason Ekstrand
14a12b771d spirv: Rework our handling of images and samplers
Previously, objects of type OpTypeImage or OpTypeSampler were treated as
vtn_pointers and objects of type OpTypeSampledImage were a special-use
vtn_sampled_image struct.  This commit changes that so that all of those
objects are stored in vtn_ssa_values.  Each of images, samplers, and
sampled images, are stored as a scalar or vector nir_ssa_def whose
components are NIR deref values.  We now use vtn_type_get_nir_type to
re-resolve those as-needed into GLSL sampler types for NIR.

This simplification has a number of benefits:

 1. We can git rid of the rest of our special-cases for handling images
    and samplers in function arguments.  Now that they're treated as
    structs at the glsl_type level, the generic paths can handle images
    and samplers.

 2. We can now construct composite values containing images and samplers
    internally.  It's unclear from the SPIR-V spec whether or not this
    is allowed and it's not a pattern that GLSLang currently generates
    thanks to GLSL rules.  However, if we do start seeing SPIR-V that
    contains such composites, we should now be able to handle it.

 3. SPIR-V OpNull and OpUndef instructions can now create samplers,
    images, and sampled images.  The NIR generated won't likely be fully
    valid but, given a NIR pass to do something sensible, it should be a
    thing we can compile.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
2020-07-23 22:43:21 -05:00
Jason Ekstrand
66c8628b65 spirv: More heavily use vtn_ssa_value in function parameter handling
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
2020-07-23 22:43:21 -05:00
Jason Ekstrand
32ba23f897 spirv: Remove a dead case in function parameter handling
Ever since 31a7476335, we've set something for vtn_type::type for
all pointer types.  For logical pointer types, it's uint32_t.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
2020-07-23 22:43:21 -05:00
Jason Ekstrand
9e3213ad30 spirv: Add a helper for getting the NIR type of a vtn_type
There are a few cases, atomic counters being one example, where the type
used by vtn_ssa_value is not the same as the type we want NIR to use in
derefs and variables.  To solve this, we add a helper which converts
between the types for us.  In the next commit, we'll be adding another
major user of this: images and samplers.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
2020-07-23 22:43:21 -05:00
Jason Ekstrand
051f8d3d1c spirv: Give atomic counters their own variable mode
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
2020-07-23 22:43:21 -05:00
Jason Ekstrand
8a7932b095 spirv: Drop the sampled boolean from vtn_type
It was set but never used.  We always check the glsl_type instead.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
2020-07-23 22:43:21 -05:00
Jason Ekstrand
d0be2fed4e spirv: Add better checks for SSA value types
Primarily, we check for two things:

 1. That we only ever add SSA values via vtn_push_ssa_value and
    vtn_copy_value.

 2. That the type of the SSA value matches the SPIR-V destination type.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5278>
2020-07-23 22:43:21 -05:00