Commit Graph

154 Commits

Author SHA1 Message Date
Rhys Perry
46d14abeae nir/builder: add nir_{ine,ibfe,ubfe}_imm() helper
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15854>
2022-04-15 23:56:11 +00:00
Emma Anholt
d506d910e4 nir: Switch to using nir_vec_scalars() for things that used nir_channel().
This should reduce follow-on optimization work to copy-propagate and
dead-code away the movs generated in construction of vectors.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14865>
2022-03-02 22:28:58 +00:00
Emma Anholt
d95f9d189a nir: Introduce a nir_vec_scalars() helper using nir_ssa_scalar.
Many users of nir_vec() do so by nir_channel()-ing a new ssa defs as movs
from other vectors to put the new vector together, which then just have to
get copy-propagated into the ALU srcs and DCEed away the temporary movs.
If they instead take nir_ssa_scalar, we can avoid that extra work.

Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14865>
2022-03-02 22:28:58 +00:00
Lionel Landwerlin
b8f087b0e6 nir/builder: add nir_ior_imm() helper
Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13719>
2022-02-08 12:55:25 +00:00
Lionel Landwerlin
e227bb9fd5 nir/builder: add ishl_imm helper
v2: add (y >= x->bit_size) condition (Caio)

Signed-off-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Reviewed-by: Caio Oliveira <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13739>
2022-02-02 17:09:46 +00:00
Emma Anholt
06fe04b4d7 nir: Make nir_build_alu() variants per 1-4 arg count.
This saves a bunch of generated code to pack up the extra NULLs to get to
4 args, and saves executing the conditions in nir_build_alu() to then skip
those NULLs.

Saves another 27kb on disk.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13916>
2021-12-01 22:12:19 +00:00
Emma Anholt
7603187aec nir: Un-inline more of nir_builder.h.
Cuts another 470KB of libnir.a in my release build.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13889>
2021-11-22 20:40:47 +00:00
Emma Anholt
d9bfcf5f5b nir: Un-inline nir_builder_alu_instr_finish_and_insert()
This function is big and I don't think it will won't get meaningfully
constant-propagated during inlining without LTO.  Move it to a .c file so
we just have one copy, saving 2.8MB from libnir.a on an amd64 release
build.

      text       data        bss      total filename
before:
  18953406    7768312     687260   27408978 build-release/driver-symlinks/iris_dri.so
   9734366    5542453     481692   15758511 build-release/lib/libvulkan_intel.so
  28687772   13310765    1168952   43167489 (TOTALS)

after:
  15478350    7767864     687260   23933474 build-release/driver-symlinks/iris_dri.so
   6810366    5541685     481692   12833743 build-release/lib/libvulkan_intel.so
  22288716   13309549    1168952   36767217 (TOTALS)

No statistically significant performance difference on iris shader-db, n=8.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13889>
2021-11-22 20:40:47 +00:00
Boris Brezillon
641bed3103 nir: Make sure src->num_components < dst->num_components in nir_ssa_for_src()
The NIR validation complains if the swizzle accesses a component that's
not present in the source.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13060>
2021-09-30 16:54:42 +02:00
Rhys Perry
ed70b256ce nir: add ffma creation helpers
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8056>
2021-08-16 17:19:45 +00:00
Emma Anholt
673cc9323a nir: Move phi src setup to a helper.
Cleans up the ralloc/list push code all over the tree.

Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11772>
2021-08-13 16:11:57 +00:00
Marcin Ślusarz
e1b325f587 nir/builder: invalidate metadata per function
Fixes: a62098fff2 ("nir: Add a helper for general instruction-modifying passes.")
Signed-off-by: Marcin Ślusarz <marcin.slusarz@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12324>
2021-08-11 11:23:30 +00:00
Mike Blumenkrantz
ec66c58138 nir: add imm_vec3 to round these out
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12253>
2021-08-09 14:45:30 +00:00
Mike Blumenkrantz
3ab74d0ffa nir: add nir_imm_ivec3 builder
the other ones exist, so why not this one too

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Rob Clark <robdclark@chromium.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11983>
2021-07-21 13:57:14 +00:00
Connor Abbott
e4e79de2a4 nir/subgroups: Support > 1 ballot components
Qualcomm has a mode with a subgroup size of 128, so just emitting larger
integer operations and then lowering them later isn't an option. This
makes the pass able to handle the lowering itself, so that we don't have
to go down to 64-thread wavefronts when ballots are used.

(The GLSL and legacy SPIR-V extensions only support a maximum of 64
threads, but I guess we'll cross that bridge when we come to it...)

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6752>
2021-07-08 16:02:41 +00:00
Mike Blumenkrantz
f9ecbb1e1d nir/builder: add nir_mask
it's handy to have functions for generating masks

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10620>
2021-05-26 04:06:27 +00:00
Mike Blumenkrantz
6df187df13 nir/builder: add nir_pad_vector and nir_pad_vec4 util functions
these pad a given value to vec4 or arbitrary number of components

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10630>
2021-05-16 14:15:14 +00:00
Mike Blumenkrantz
37545418cd nir: add nir_isub_imm
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10654>
2021-05-06 13:01:03 +00:00
Jason Ekstrand
e1edf74dde nir/builder: Move clamp helpers to nir_builder.h
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10631>
2021-05-04 22:51:34 +00:00
Alyssa Rosenzweig
899dd8e60a nir: Update some comments referring to imov
This was renamed when I was in high school. I remember updating the
Midgard compiler while sitting in AP Physics.

Signed-off-by: Alyssa Rosenzweig <alyssa@collabora.com>
Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10296>
2021-04-19 20:07:35 +00:00
Alyssa Rosenzweig
e4da24bd24 nir: Add {i2f, u2f, f2i, f2u} helpers
Convenient for bitsize independent lowerings, will be used in the idiv
lowering.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8339>
2021-04-07 15:48:15 +00:00
Alyssa Rosenzweig
6b19711645 nir: Add nir_type_convert
Generalizes nir_convert_to_bit_size, which we implement as a
special-case.

v2: Take a sized dest type but allow unsized or sized source to address
Jason's feedback. Shorten name.

Signed-off-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8339>
2021-04-07 15:48:15 +00:00
Timur Kristóf
9f9b0f583b nir: Add nir_builder helper for I/O address offset calculations.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Timur Kristóf
f6f68d5cf1 nir: Add new nir_builder helpers for iadd with no_unsigned_wrap.
Signed-off-by: Timur Kristóf <timur.kristof@gmail.com>
Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9201>
2021-03-17 12:42:23 +00:00
Rhys Perry
95819663b7 nir: allow 5 component vectors
These will be useful for sparse texture instructions and image load
intrinsics.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7774>
2021-01-06 20:36:38 +00:00
Rhys Perry
2846d7811e nir: fix nir_builder.h on MSVC C++ and GCC7.
MSVC C++ can't do designated initializers without /std:c++latest. These
helpers will likely be removed soon anyway, so just don't use the
intrinsic builders here.

This should also fix the GCC7 build, which doesn't implement non-trivial
designated initializers.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Fixes: c9bcad2573 ("nir: add generated intrinsic builders")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7808>
2020-11-27 10:51:54 +00:00
Rhys Perry
c9bcad2573 nir: add generated intrinsic builders
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>
2020-11-26 17:50:38 +00:00
Rhys Perry
b3c65f97ad nir: move nir_load_system_value() to nir_builder.h
To make review easier.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>
2020-11-26 17:50:38 +00:00
Rhys Perry
f8ee599dfb nir: remove useless nir_builder_opcodes.h include
This is included earlier in the file.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6587>
2020-11-26 17:50:38 +00:00
Jason Ekstrand
9fa1cdfe7f intel/rt: Implement push constants as global memory reads
They're not really "push" anymore but that's because there is no such
thing as push constants in bindless shaders on Intel.  They should be
fast enough, though.  There is some room for debate here as to whether
we want to do the pull in NIR or push it into the back-end.  The
advantage of doing it in the back-end is that it'd be easier to use
MOV_INDIRECT for indirect push constant access rather than falling back
to a dataport message.

Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356>
2020-11-25 05:37:10 +00:00
Eric Anholt
eda3e4e055 nir/builder: Add a name format arg to nir_builder_init_simple_shader().
This cleans up a bunch of gross sprintfs and keeps the caller from needing
to remember to ralloc_strdup.  I added a couple of '"%s", name ? name :
""' to radv where I didn't fully trace through whether a non-null name was
being passed in.

I also took the liberty of adding a basic name to a few shaders (pan_blit,
unit tests)

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7323>
2020-11-11 08:50:29 -08:00
Eric Anholt
5f992802f5 nir/builder: Drop the mem_ctx arg from nir_builder_init_simple_shader().
This looks a lot more simple now!

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7323>
2020-11-11 08:50:29 -08:00
Eric Anholt
4e9328e3b6 nir_builder: Return a new builder from nir_builder_init_simple_shader().
It's a little inline function, so we can just RAII it for better
ergonomics.

Reviewed-by: Rhys Perry <pendingchaos02@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7323>
2020-11-11 08:49:49 -08:00
Jason Ekstrand
07635a3284 nir/builder: Add a select_from_ssa_def_array helper
This is an operation we have to do already for nir_vector_extract and
I'm about to do something very similar for matrix columns.  Having a
more generic helper is useful.

Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6479>
2020-11-05 23:36:46 +00:00
Rhys Perry
b90063201a nir: use nir_alu_src_is_trivial_ssa() in nir_ssa_for_alu_src()
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7426>
2020-11-03 22:35:57 +00:00
Jason Ekstrand
a8e53a772f spirv: Add generic pointer support
Most of this is fairly straightforward; we just set all the modes on any
derefs which are generic.  The one tricky bit is OpGenericCastToPtrExplicit.
Instead of adding NIR intrinsics to do the cast, we add NIR intrinsics
to do a storage class check and then bcsel based on that.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
2020-11-03 22:18:28 +00:00
Jason Ekstrand
9d377c01d0 nir: Make nir_deref_instr::mode a bitfield
We rename it to "modes" to make it clear that it may contain more than
one mode and adjust all the uses of nir_deref_instr::modes to attempt to
handle multiple modes.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
2020-11-03 22:18:28 +00:00
Jason Ekstrand
3f0a29fffb nir/builder: Add a nir_ieq_imm helper
This shows up surprisingly often.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6332>
2020-11-03 22:18:28 +00:00
Rhys Perry
37b6b0967c nir: allow divergence information to be updated when inserting instruction
Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Daniel Schürmann <daniel@schuermann.dev>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6558>
2020-10-13 12:47:21 +00:00
Jason Ekstrand
588bb6686b nir: Add a conversion and rounding intrinsic
This new intrinsic is capable of handling the full range of conversions
from OpenCL including rounding modes and possible saturation.  The
intention is that we'll emit this intrinsic directly from spirv_to_nir
and then lower it to ALU ops later.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6945>
2020-10-01 18:36:53 +00:00
Jason Ekstrand
b2899f7265 nir: Add a new memcpy intrinsic
This matches SPIR-V's OpCopyMemorySized

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jesse Natalie
89401e5867 nir: More NIR_MAX_VEC_COMPONENTS fixes
Cc: mesa-stable@lists.freedesktop.org
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6655>
2020-09-09 20:19:42 +00:00
Jason Ekstrand
3135984ad0 spirv: Propagate alignments to deref chains via casts
This commit propagates the alignment information provided either through
the Alignment decoration on pointers or via the alignment mem operands
to OpLoad, OpStore, and OpCopyMemory to the NIR deref chain.  It does so
by wrapping the deref in a cast.  NIR should be able to clean up most
unnecessary casts only leaving us with the useful alignment information.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6472>
2020-09-03 18:02:50 +00:00
Jason Ekstrand
a06955f4ef nir/builder: Add load/store_global helpers
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6244>
2020-09-02 19:48:44 +00:00
Jason Ekstrand
91becd84ae intel/fs: Add support for a new load_reloc_const intrinsic
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6244>
2020-09-02 19:48:44 +00:00
Eric Anholt
a62098fff2 nir: Add a helper for general instruction-modifying passes.
The nir_shader_lower_instructions() is really nice, but it's only for SSA
operations, and sometimes you want something more general.  I've put it in
nir_builder.h so it can be inlined and retain the same performance
characteristics we're used to in our lowering passes even in the absence
of LTO.

Reviewed-By: Mike Blumenkrantz <michael.blumenkrantz@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6412>
2020-08-27 23:01:52 +00:00
Eric Anholt
d84a16e481 nir: Add nir_[iu]shr_imm and nir_udiv_imm helpers and use them.
I was doing math manually in a lowering pass for converting a division to
a ushr, and this will let the pass be expressed more naturally.

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6378>
2020-08-24 09:53:17 -07:00
Eric Anholt
e78b887f76 nir: Make the nir_builder *_imm helpers consistently handle bit size.
We always want to demote the y to the bit size of the ssa def, but also
want to sanity check that our input and our masking is big enough.

Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6378>
2020-08-24 09:53:09 -07:00
Karol Herbst
e5899c1e88 nir: rename nir_op_fne to nir_op_fneu
It was always fneu but naming it fne causes confusion from time to time. So
lets rename it. Later we also want to add other unordered and fne, this is
a smaller preparation for that.

Signed-off-by: Karol Herbst <kherbst@redhat.com>
Reviewed-by: Bas Nieuwenhuizen <bas@basnieuwenhuizen.nl>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6377>
2020-08-21 17:26:21 +00:00
Jesse Natalie
42d7bbfc22 nir: Use 'unsigned' instead of enum types in nir_variable::data
MSVC treats enums as signed, so storing values that use the topmost
bit of the explicitly sized field loads as a negative value instead.

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6393>
2020-08-20 22:22:06 +00:00