Commit Graph

5452 Commits

Author SHA1 Message Date
Jesse Natalie
b778e7bd6c nir/vtn: Convert constant samplers to variables with data
Reviewd-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
397730edd5 nir/vtn: Add intrinsics for CL image format/order queries
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
5ce6821900 nir/vtn: ImageSizeLod op can be applied to images
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
701cb9d60c nir/vtn: Handle integer sampling coordinates
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
a196f05fc2 nir/vtn: Use return type rather than image type for tex ops
Since OpenCL images don't have types, we can't use the image type here.
Rather than special-casing and only using SPIR-V return type for CL images,
we can just always use the return type to fill out the tex info.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
de36b5b805 nir/vtn: Add support for kernel images to SPIRV-to-NIR.
There's a few quirks: kernel images are untyped, whether they're
sampled is unknown, and they're passed as inputs to the kernel even though
SPIR-V declares their address space as UniformConstant.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:39 +00:00
Jesse Natalie
ce6f66242a nir/vtn: Add type constant to image intrinsics
Since OpenCL supports untyped images, backends might need type info
to be able to support the load/store ops.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5242>
2020-08-31 21:17:38 +00:00
Jesse Natalie
865a2ad086 clover/nir/spirv: Use uniform rather than shader_in for kernel inputs
The semantics of inputs for CL are a closer match to the semantics of uniforms for graphics.
Rather than cross-stage data, it's data that every thread sees uniformly.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6497>
2020-08-31 19:58:14 +00:00
Eric Anholt
73d2c6cdce nir: Switch the indexing of block->live_in/out arrays.
In nir-to-tgsi, I want to free temps storing SSA values when they go dead,
and NIR liveness has most of the information I need.  Hoever, when I reach
the end of a block, I need to free whatever temps were in liveout which
are dead at that point.  If liveout is indexed by live_index, then I don't
know the maximum live_index for iterating the live_out bitset, and I also
don't have a way to map that index back to the def->index that my temps
are stored under.

We can use the more typical def->index for these bitsets, which resolves
both of those problems.  The only cost is that ssa_undefs don't get merged
into a single bit in the bitfield, but there are generally 1-4 of them in
a shader and we don't track liveness for those anyway so splitting them
apart is fine.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6408>
2020-08-31 18:28:36 +00:00
Eric Anholt
5d2b2b59c4 nir: Invalidate live SSA def information when making new SSA defs.
Noted by @jekstrand in review of the SSA live index changes.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6408>
2020-08-31 18:28:36 +00:00
Italo Nicola
ee288f293b nir: add shared/global atomics to nir_get_io_offset_src()
Signed-off-by: Italo Nicola <italonicola@collabora.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6521>
2020-08-31 17:36:12 +00:00
Eric Anholt
e281ee04df nir: Look up the shader when printing a single instruction.
If you've inserted the instruction into a block, then we can get to the
shader.  This improves our instruction output, giving you i/o semantics
and variable names in intrinsics.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6496>
2020-08-31 17:20:43 +00:00
Eric Anholt
a3a8322dcd nir: Fix printing of individual instructions with io semantics.
The state->shader is missing when used outside of nir_print_shader, just
drop these details in that case.  We can fix nir_print_instr() to look up
the shader, but let's also make sure that an instr detached from a shader
(such as one you're constructing but haven't yet inserted) still works.

Fixes: 2b1ef5df4e ("nir: print IO semantics (v2)")
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6496>
2020-08-31 17:20:43 +00:00
Jason Ekstrand
ac6b8e42ce nir: Take a variable remap parameter in nir_inline_function_impl
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6411>
2020-08-29 18:32:46 +00:00
Jason Ekstrand
b6f31898a2 nir: Use a switch in nir_inline_function_impl
Acked-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6411>
2020-08-29 18:32:46 +00:00
Eric Anholt
31f75aaeef nir/lower_discard_to_demote: Use nir_shader_instructions_pass().
Cleans up indentation, and clears the metadata tracking flag that would
break if this pass was used in in NIR_PASS().

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6412>
2020-08-27 23:01:52 +00:00
Eric Anholt
265dcb3836 nir/nir_lower_wrmasks: Use the nir_lower_instructions_pass() helper.
This fixes the invalidation of metadata when we didn't modify the shader
and unindents a bunch of code.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6412>
2020-08-27 23:01:52 +00:00
Eric Anholt
c5e64c041f nir/lower_io_to_scalar: Convert to use nir_shader_instructions_pass().
This unindents a whole bunch of code, and fixes the lack of metadata
tracking in the pass (which wasn't called in an opt loop so it hadn't been
caught before).

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6412>
2020-08-27 23:01:52 +00:00
Eric Anholt
dfb8465341 nir/opt_undef: Convert to use nir_shader_instructions_pass().
We can't use nir_lower_instructions because we operate on stores which
don't have an SSA def.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6412>
2020-08-27 23:01:52 +00:00
Eric Anholt
e5d4bbd840 nir/lower_vec_to_movs: Convert to use nir_shader_instructions_pass().
Less pass code, less indenting, should be the same perf.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6412>
2020-08-27 23:01:52 +00:00
Eric Anholt
a62098fff2 nir: Add a helper for general instruction-modifying passes.
The nir_shader_lower_instructions() is really nice, but it's only for SSA
operations, and sometimes you want something more general.  I've put it in
nir_builder.h so it can be inlined and retain the same performance
characteristics we're used to in our lowering passes even in the absence
of LTO.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6412>
2020-08-27 23:01:52 +00:00
Eric Anholt
a82f664d0a nir/opt_copy_prop_vars: Quiet valgrind warning about overlapping memcpy.
The warning is kind of silly:

Test case 'dEQP-GLES2.functional.shaders.indexing.tmp_array.vec3_const_write_static_read_vertex'..
==1874780== Source and destination overlap in memcpy(0xa261690, 0xa261690, 160)
==1874780==    at 0x484D498: __GI_memcpy (vg_replace_strmem.c:1037)
==1874780==    by 0x596FC07: copy_entry_remove (nir_opt_copy_prop_vars.c:296)

The "memcpy is undefined if they overlap" thing is surely meant to be
"memcpy with *partial* overlap is undefined", but let's keep anyone else
from having to debug this.

Reviewed-by: Rob Clark <robdclark@chromium.org>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6178>
2020-08-27 21:39:50 +00:00
Jesse Natalie
608c87afdd nir/vtn: SPIR-V bit count opcodes (core and extension) dest size mismatches nir
SPIR-V dest sizes match the input, while nir is always int32. Insert
casts from the nir op to the expected SPIR-V dest.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>
2020-08-27 16:57:42 +00:00
Jesse Natalie
a54695ddcb nir: Add bit_count to lower_int64 pass
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>
2020-08-27 16:57:42 +00:00
Jesse Natalie
d91f85f16e nir: Remove 32bit restriction for uadd_carry optimization
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>
2020-08-27 16:57:42 +00:00
Jesse Natalie
9232887c69 nir: Implement mul_high lowering for bit sizes other than 32
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>
2020-08-27 16:57:42 +00:00
Jesse Natalie
ea715741b5 nir_lower_bit_size: Support lowering ops with differing source/dest sizes
Specifically the bit-finding routines always return int32. Don't complain
about the dest already being 32 bits when lowering to 32 bits, and
don't bother casting the dest if it's already right.

Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>
2020-08-27 16:57:42 +00:00
Jesse Natalie
0d595f72b2 nir: Relax opt_if logic to prevent re-merging 64bit phis for loop headers
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>
2020-08-27 16:57:42 +00:00
Jesse Natalie
43d22c8f20 nir: Add a lowering pass to split 64bit phis
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6313>
2020-08-27 16:57:41 +00:00
Jason Ekstrand
27e6117ee9 nir: Report progress properly in nir_lower_bool_to_*
All three passes have the same bug where, in the mov/vec case they
unconditionally return true even if they don't change anything.  Throw
in a bit size check so they return false properly.

Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6435>
2020-08-25 22:39:08 +00:00
Kenneth Graunke
2a9ffc24cc nir: Move new edgeflag assert into the io_lowered case
We only need to assert this in the `io_lowered` case, which actually
uses num_outputs.  This assert also doesn't appear to hold on iris,
where num_outputs is showing up as 0 (because it's likely not yet set).

Fixes assertion failures in edgeflag related tests on iris, which
doesn't use the io_lowered path currently.

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3456
Fixes: 484a60d547 ("nir: generate lowered IO in nir_lower_passthrough_edgeflags")
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6450>
2020-08-25 16:43:19 +00:00
Kenneth Graunke
2fcfcca842 nir: Copy semantics to nir_intrinsic_load_fs_input_interp_deltas
When using nir_lower_interpolation, we need to propagate the IO
semantics from the load_interpolated_input to the new
load_fs_input_interp_deltas intrinsics.  nir_lower_io assumes
they will be filled out.

This fixes assertions in most tests on iris since commit
01ab308edc, where nir_lower_io
started reading this field.

Fixes: 01ab308edc ("nir: update IO semantics in nir_io_add_const_offset_to_base")
Fixes: 502abfce7f ("nir: save IO semantics in lowered IO intrinsics")
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6450>
2020-08-25 16:43:19 +00:00
Daniel Schürmann
a79dad950b nir,amd: remove trinary_minmax opcodes
These consist of the variations nir_op_{i|u|f}{min|max|med}3 which are either
lowered in the backend (LLVM) anyway or can be recombined by the backend (ACO).

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6421>
2020-08-24 20:56:11 +00:00
Marek Olšák
484a60d547 nir: generate lowered IO in nir_lower_passthrough_edgeflags
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6442>
2020-08-24 19:07:18 +00:00
Marek Olšák
7980f3e519 nir: add interpolation qualifiers for color sysvals into shader_info
needed by radeonsi

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6442>
2020-08-24 19:07:18 +00:00
Marek Olšák
4d36dceeab nir: add shader_info::io_lowered
This will drive decisions in many NIR passes and st/mesa.

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6442>
2020-08-24 19:07:18 +00:00
Marek Olšák
844167d100 nir: properly identify texcoords for lowered IO in nir_lower_drawpixels
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6442>
2020-08-24 19:07:18 +00:00
Marek Olšák
2b1ef5df4e nir: print IO semantics (v2)
v2: print GS streams readably

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6442>
2020-08-24 19:07:18 +00:00
Marek Olšák
01ab308edc nir: update IO semantics in nir_io_add_const_offset_to_base
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6442>
2020-08-24 19:07:18 +00:00
Marek Olšák
17af07024d nir: gather all IO info from IO intrinsics
nir_io_add_const_offset_to_base will shrink num_slots, so it's better to
call it before nir_shader_gather_info.

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6442>
2020-08-24 19:07:18 +00:00
Marek Olšák
502abfce7f nir: save IO semantics in lowered IO intrinsics
This enables drivers and utils to get all IO information from intrinsics,
so that they don't have to walk the complex types of NIR variables to find
out other information about IO intrinsics.

NIR in/out variables can be removed after nir_lower_io. We could remove
the variables in the pass, but for now I just decided to remove
the variables in radeonsi before shaders are returned to st/mesa.
(st/mesa just needs adjustments to work without NIR in/out variables)

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6442>
2020-08-24 19:07:18 +00:00
Eric Anholt
a9019d14ae nir/load_store_vectorizer: Clean up unit test swizzle assertions.
They're hard to write and read, so put together a little helper to clean
up both the code and the gtest output on failure.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6431>
2020-08-24 17:16:05 +00:00
Eric Anholt
2b25240993 freedreno/ir3: Replace our custom vec4 UBO intrinsic with the shared lowering.
This gets us fewer comparisons in the shaders that we need to optimize
back out, and reduces backend code.

total instructions in shared programs: 11547270 -> 7219930 (-37.48%)
total full in shared programs: 334268 -> 319602 (-4.39%)

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6378>
2020-08-24 09:53:36 -07:00
Eric Anholt
73616598bd nir: Add a lowering pass for backends wanting load_ubo with vec4 offsets.
This is very common for backends -- r600, freedreno, and nir_to_tgsi all
needed versions of it.  Make a common intrinsic to use for it with a
shared, slightly-tuned-from-ir3 lowering pass.

Reviewed-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6378>
2020-08-24 09:53:35 -07:00
Eric Anholt
d84a16e481 nir: Add nir_[iu]shr_imm and nir_udiv_imm helpers and use them.
I was doing math manually in a lowering pass for converting a division to
a ushr, and this will let the pass be expressed more naturally.

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6378>
2020-08-24 09:53:17 -07:00
Eric Anholt
e78b887f76 nir: Make the nir_builder *_imm helpers consistently handle bit size.
We always want to demote the y to the bit size of the ssa def, but also
want to sanity check that our input and our masking is big enough.

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6378>
2020-08-24 09:53:09 -07:00
Erik Faye-Lund
5e841e8b4f nir: add iabs-lowering code
Microsoft's DXIL is based on LLVM, which doesn't have an integer ABS
opcode, but instead needs it lowered to NEG + MAX. We need to do this
with an option, to prevent an already existing optimization rule from
undoing this.

Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5211>
2020-08-24 10:02:47 +00:00
Karol Herbst
47e52d84ea nir: fix nir_variable_create for kernels
We ended up with INTERP_MODE_SMOOTH but we really just want to have NONE.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5891>
2020-08-21 22:07:05 +00:00
Jesse Natalie
368e9a0b54 spirv: Use new global invocation offset system value
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5891>
2020-08-21 22:07:05 +00:00
Jesse Natalie
d3faac7a15 nir: Add options to nir_lower_compute_system_values to control compute ID base lowering
If no options are provided, existing intrinsics are used.
If the lowering pass indicates there should be offsets used for global
invocation ID or work group ID, then those instructions are lowered to
include the offset.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5891>
2020-08-21 22:07:05 +00:00