Commit Graph

5452 Commits

Author SHA1 Message Date
Rhys Perry
ee073cb543 nir/load_store_vectorize: improve vectorization with identical operations
We might have several identical options to vectorize an entry with, but
only one might be vectorizable because of writes interfering.

An example of this is a pattern found in some CTS tests:
a = load(0)
b = load(4)
store(0, a)
store(4, b)
a = load(0)
b = load(4)
store(0, a)
store(4, b)
...

It might have attempted to vectorize the first load(0) with the second
load(4) without attempting the second load(4) when the first fails. This
changes vectorize_entries() to continue even if the first try_vectorize()
failed.

fossil-db (Navi):
Totals from 117 (0.09% of 137413) affected shaders:
SGPRs: 7040 -> 7088 (+0.68%)
CodeSize: 276504 -> 276308 (-0.07%); split: -0.08%, +0.01%
Instrs: 51152 -> 51111 (-0.08%); split: -0.09%, +0.01%

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/5415>
2020-09-28 12:06:30 +00:00
Louis-Francis Ratté-Boulianne
f2b94ae085 glsl/linker: Add support for XFB varying lowering in geometry shader
Reviewed-by: Erik Faye-Lund <erik.faye-lund@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6723>
2020-09-28 07:57:28 +00:00
Jason Ekstrand
8323c03bbf spirv: Add support for OpCopyMemorySized
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jason Ekstrand
a3177cca99 nir: Add a lowering pass to lower memcpy
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jason Ekstrand
b2899f7265 nir: Add a new memcpy intrinsic
This matches SPIR-V's OpCopyMemorySized

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jason Ekstrand
8cbb6640f0 spirv: Use the new types helpers
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jason Ekstrand
de295b61da compiler/types: Add glsl_baseN_t_type(bit_size) helpers
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6713>
2020-09-25 23:48:03 +00:00
Jesse Natalie
5f1855d896 vtn/opencl: Switch fma to conditionally use libclc for 32bit floats
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 13:20:39 -07:00
Jesse Natalie
8fb25c48a4 vtn/opencl: Only use libclc ldexp when lower_ldexp is set
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 13:14:46 -07:00
Jesse Natalie
09bca4cb95 vtn/opencl: Switch some nir-sequence ops to use libclc
All of these are pretty well-defined. Rather than implementing them
as a sequence of nir ops, we can just use the libclc implementation.

v2 (idr): Delete functions that are now unused.

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 13:14:45 -07:00
Jesse Natalie
03fd217d21 vtn/opencl: Switch division-related ops to use libclc
Specifically, fmod only uses libclc if it was going to be lowered.
Also, add missing half_divide and half_recip handling.

Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
2624faf137 vtn/opencl: Switch exp/pow/log to use libclc
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
4cc9ad4eeb vtn/opencl: Switch non-native trig to use libclc
Acked-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
6aee49578f vtn/opencl: Hook up OpenCL async copy and group wait opcodes via libclc
Adds an additional hook for spirv_to_nir to handle a core opcode via
the OpenCL libclc infrastructure, and adds handling for SpvOpGroupAsyncCopy and
SpvOpGroupWaitEvents.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
00261d883d vtn/opencl: Rework handle_instr to be able to handle core SPIR-V opcodes via libclc
The OpenCL async copy/wait opcodes are core SPIR-V, rather than OpenCL extension opcodes.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
b08fd45be0 vtn/opencl: Implement a lot of opcodes via libclc
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
6436e3ac18 vtn/opencl: Add infrastructure for calling out to libclc
This patch adds a function remap table with name mangling, which
can convert a SPIR-V OpenCL extension opcode to a call to the external
libclc shader, which will be lowered/inlined after conversion.

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
2013903bc2 vtn/opencl: Rework type handling for CL extension opcodes
- Change dest type information to be vtn_type rather than glsl_type
 - Add source type information, also in vtn_type form

Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
938be4d6fa nir/vtn: Add handling for SPIR-V event variables
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Dave Airlie
c9a6b94f9e vtn: add an option to create a nir library from spirv
This adds an options to turn a spir-v library into a set of nir
functions. There is no entry point and all function implentations
are emitted.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
2826a5a923 nir: Add glsl_base_type unsigned -> signed version helper
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com>
Reviewed-by: Dave Airlie <airlied@redhat.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jesse Natalie
93db59e066 nir: Add an internal flag to shader_info
Don't print the shader if it's marked internal, unless NIR_PRINT
has been explicitly set to 2 (or higher).

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6035>
2020-09-25 20:09:08 +00:00
Jason Ekstrand
0206fb3941 nir/liveness: Consider if uses in nir_ssa_defs_interfere
Fixes: f86902e75d "nir: Add an SSA-based liveness analysis pass"
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/3428
Reviewed-by: Eric Anholt <eric@anholt.net>
Reviewed-by: Yevhenii Kharchenko <yevhenii.kharchenko@globallogic.com>
Reviewed-by: Connor Abbott <cwabbott0@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6824>
2020-09-25 14:16:15 +00:00
Rhys Perry
a18c84ecce nir/instr_set: hash intrinsic sources
ministat (CSE only):
Difference at 95.0% confidence
	-9.80325 +/- 0.173089
	-41.4434% +/- 0.461972%
	(Student's t, pooled s = 0.0763653)

ministat (entire run):
Difference at 95.0% confidence
	-3.13667 +/- 0.61519
	-5.11107% +/- 0.990737%
	(Student's t, pooled s = 0.271416)

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Matt Turner <mattst88@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6860>
2020-09-25 10:18:36 +00:00
Marek Olšák
ea77958fea nir: gather information about fbfetch and dual source color
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
a6abf175ef nir: fix input/output info gathering for lowered IO
Ooops.

Fixes: 17af07024d - nir: gather all IO info from IO intrinsics

Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
ef98c175c0 nir: gather fs.uses_sample_qualifier from lowered IO
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
7b108e6ac4 nir: set system_values_read for all intrinsics
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
abe9588ff0 nir: gather tess.tcs_cross_invocation info from lowered IO intrinsics
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Marek Olšák
10be706778 nir: gather indirect info from lowered IO intrinsics
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6758>
2020-09-25 02:29:30 -04:00
Jason Ekstrand
3173367a47 spirv: vtn_fail with a nice message on unsupported rounding modes
Reviewed-by: Caio Marcelo de Oliveira Filho <caio.oliveira@intel.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6845>
2020-09-25 01:43:28 +00:00
Kenneth Graunke
140f53e646 Revert "nir: replace lower_ffma and fuse_ffma with has_ffma"
This reverts commit 939ddf3f67.

Intel has a separate pass for fusing FFMAs selectively.  We split
these flags in commit 1b72c31e1f and
the reasoning still stands.  The patch being reverted was just a
cleanup, so there should be no issue with reverting it.

Acked-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6849>
2020-09-24 13:11:50 -07:00
Marek Olšák
939ddf3f67 nir: replace lower_ffma and fuse_ffma with has_ffma
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6756>
2020-09-24 12:29:11 +00:00
Marek Olšák
771aad3027 nir: split lower_ffma into lower_ffma16/32/64
AMD wants different behavior for each bit size

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6756>
2020-09-24 12:29:11 +00:00
Marek Olšák
21174dedec nir: split fuse_ffma into fuse_ffma16/32/64
AMD wants different behavior for each bit size

Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6756>
2020-09-24 12:29:11 +00:00
Jesse Natalie
924e27647e nir_lower_system_values: Fix load_global_invocation_id to use base_work_group_id even with no base_global id
Reviewed-by: Karol Herbst <kherbst@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6668>
2020-09-22 21:22:26 +00:00
Eric Anholt
ee4cee6dbd android: Disable trying to read/write to the disk cache.
We need the disk cache enabled in Android to get EGL_ANDROID_blob_cache's
callbacks called, but we don't actually want to store anything on disk.
Fixes "Failed to create //.cache for shader cache (Read-only file
system)---disabling." spam on init.

Reviewed-by: Tapani Pälli <tapani.palli@intel.com>
Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6762>
2020-09-22 20:41:25 +00:00
Jason Ekstrand
9750164c09 nir: Rename get_buffer_size to get_ssbo_size
This makes it explicit that this intrinsic is only for SSBOs.  For the
v3dv driver, we'll be adding a get_ubo_size intrinsic and we want to be
able to distinguish between the two.

Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Iago Toral Quiroga <itoral@igalia.com>
Reviewed-by: Alyssa Rosenzweig <alyssa.rosenzweig@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6812>
2020-09-22 13:34:12 +00:00
Rhys Perry
f100cf0d30 aco: stop multiplying driver_location by 4
This didn't really serve any purpose, doesn't match how FS inputs are
currently done, and prevented us from using
nir_io_add_const_offset_to_base in the future.

Signed-off-by: Rhys Perry <pendingchaos02@gmail.com>
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6689>
2020-09-22 12:38:43 +00:00
Danylo Piliaiev
f2b17dec12 nir/lower_samplers: Clamp out-of-bounds access to array of samplers
Section 5.11 (Out-of-Bounds Accesses) of the GLSL 4.60 spec says:

"In the subsections described above for array, vector, matrix and
 structure accesses, any out-of-bounds access produced undefined
 behavior.... Out-of-bounds reads return undefined values, which
 include values from other variables of the active program or zero."

Robustness extensions suggest to return zero on out-of-bounds
accesses, however it's not applicable to the arrays of samplers,
so just clamp the index.

Otherwise instr->sampler_index or instr->texture_index would be out
of bounds, and they are used as an index to arrays of driver state.

E.g. this fixes such dereference:
 if (options->lower_tex_packing[tex->sampler_index] !=
in nir_lower_tex.c

CC: <mesa-stable@lists.freedesktop.org>
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6428>
2020-09-22 09:06:52 +00:00
Danylo Piliaiev
0ba82f78a5 nir/large_constants: Eliminate out-of-bounds writes to large constants
Out-of-bounds writes could be eliminated per spec:

Section 5.11 (Out-of-Bounds Accesses) of the GLSL 4.60 spec says:

"In the subsections described above for array, vector, matrix and
 structure accesses, any out-of-bounds access produced undefined
 behavior.... Out-of-bounds writes may be discarded or overwrite
 other variables of the active program."

Fixes: 1235850522
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6428>
2020-09-22 09:06:52 +00:00
Danylo Piliaiev
66669eb529 nir/lower_io: Eliminate oob writes and return zero for oob reads
Out-of-bounds writes could be eliminated per spec:

Section 5.11 (Out-of-Bounds Accesses) of the GLSL 4.60 spec says:

 "In the subsections described above for array, vector, matrix and
  structure accesses, any out-of-bounds access produced undefined
  behavior....
  Out-of-bounds writes may be discarded or overwrite
  other variables of the active program.
  Out-of-bounds reads return undefined values, which
  include values from other variables of the active program or zero."

GL_KHR_robustness and GL_ARB_robustness encourage us to return zero
for reads.

Otherwise get_io_offset would return out-of-bound offset which may
result in out-of-bound loading/storing of inputs/outputs,
that could cause issues in drivers down the line.

E.g. this fixes such dereference:
 int vue_slot = vue_map->varying_to_slot[intrin->const_index[0]];
in brw_nir.c

CC: <mesa-stable@lists.freedesktop.org>
Signed-off-by: Danylo Piliaiev <danylo.piliaiev@globallogic.com>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6428>
2020-09-22 09:06:52 +00:00
Vinson Lee
3022cf3bac glsl: Initialize ir_constant member const_elements in all constructors.
Fix defects reported by Coverity Scan.

Uninitialized pointer field (UNINIT_CTOR)
uninit_member: Non-static class member const_elements is not
initialized in this constructor nor in any functions that it calls.

Signed-off-by: Vinson Lee <vlee@freedesktop.org>
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6770>
2020-09-21 18:29:46 -07:00
Jason Ekstrand
e1fc23265f nir: Add a pass for lowering CL-style image ops to texture ops
In CL 1.2, images are required to be either read-only or write-only.  We
can always translate the read-only image ops to texture ops.  In CL 2.0
(and an extension), the ability is added to have read-write images but
sampling (with a sampler) is only allowed on read-only images.  As long
as we only lower read-only images to texture ops, everything should stay
consistent.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6578>
2020-09-20 14:28:13 +00:00
Jason Ekstrand
3fc425b355 spirv: Plumb access qualifiers through from image types
In SPIR-V, the access qualifiers for an image are provided on the image
type.  Assuming no one swaps the types around on us (I think that should
be illegal), this means we can reliably fetch the access qualifiers from
the type itself.  The ops which don't really have an easy-to-fetch type
are the atomics because they use OpImageTexelPointer.  However, those
are only allowed on read/write images and that's the default.

Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6578>
2020-09-20 14:28:13 +00:00
Jason Ekstrand
1e902102c4 spirv: Access qualifiers are not a bitfield
They're an actual enum.  My bad.

Fixes: de36b5b805 "nir/vtn: Add support for kernel images to SPIRV-to-NIR"
Reviewed-by: Jesse Natalie <jenatali@microsoft.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6578>
2020-09-20 14:28:13 +00:00
Jesse Natalie
9aa86eb61a glsl_type: Add packed to structure type comparison for hash map
Fixes: 659f333b3a "glsl: add packed for struct types"
Reviewed-by: Marek Olšák <marek.olsak@amd.com>
Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6767>
2020-09-18 19:33:00 +00:00
Tapani Pälli
5805f5ab01 glsl: take EXT_gpu_shader4 in to account when adding round
GL_EXT_gpu_shader4 adds truncate() and round() builtins.

Fixes: 12567de2be ("glsl: mark some builtins with correct glsl(es) version check")
Signed-off-by: Tapani Pälli <tapani.palli@intel.com>
Acked-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6731>
2020-09-18 05:49:51 +00:00
Gert Wollny
6f2b6952be nir: remove ubo_r600 instrinsic since ubo_vec4 is used now
As suggested by Eric.

Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
eviewed-by: Eric Anholt <eric@anholt.net>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6743>
2020-09-17 10:11:11 +00:00
Alejandro Piñeiro
2aaa1564ad nir/lower_io: don't reduce range if parent length is zero
When handling arrays, range is increased based on the array size minus
one. But if such is zero, it has the effect of reducing the
range. Handle that case by returning the unknown range value.

v2:
  * Add missing braces.
  * Return unknown range in this case, instead of keeping the initial
    range.
v3: Simplify code, using existing "fail" label. (Jason)

Fixes the following using v3dv:
  dEQP-VK.graphicsfuzz.cov-simplify-clamp-max-itself

Reviewed-by: Jason Ekstrand <jason@jlekstrand.net>
Reviewed-by: Eric Anholt <eric@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6737>
2020-09-16 23:24:28 +02:00