Commit Graph

291 Commits

Author SHA1 Message Date
Mike Blumenkrantz
809e9462ce zink: be even more granular with optimal_key program updates
since the bits of each key are easily and efficiently comparable,
the draw-time updating here can be made extremely granular to
update exactly the stages that have changed since the last
time the program was used, further reducing overhead instead of
updating every possible key value

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
e4e9461403 zink: simplify bit tracking in zink_gfx_program_update()
much nicer now that gfx dirty bits are isolated

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
76ae28e69c zink: split dirty_shader_stages for gfx and compute
this simplifies/clarifies a lot of related code

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
49866c4f77 zink: use optimal_keys to massively reduce size of pipeline cache comparisons
since the entire pipeline module key is a uint32_t, this value can be used
for comparisons instead of comparing every shader module

ideally in the future more drivers will support the required features for
optimal_keys, allowing the other variant functions to be deleted

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
325c703624 zink: add 'optimal_keys' handling for shader keys
if it's known on init that there are no variants for nonseamless cubes,
or inlined values, or decomposed vertex attrs, then shader keys can be
compressed more optimally to reduce the work needed on program updates

more importantly, this reduces the total hash value for all the shader
modules to a single uint32_t (technically 24 bits), which is much better
than having to manage and incrementally add all the separte module hashes

...but for now using this is incompatible with gpl, so disable that

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
06e8acbc47 zink: ALWAYS_INLINE bind_gfx_stage
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
32f50630d6 zink: move to AoS for gfx program shader modules
this matches up better with the actual usage; the zink_shader_module
is still stored in the shader cache

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
745efa1231 zink: copy the non-dirty shader stages when updating gfx program
typo during refactoring

Fixes: d8455e5ebf ("zink: change u_foreach_bit to regular for loop in zink_gfx_program_update")

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
6df9bcaa10 zink: handle inline uniforms during first-time shader generation
this is unlikely but possible

Fixes: a0e69e7601 ("zink: split out first-time shader module creation")

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18786>
2022-09-26 17:12:31 +00:00
Mike Blumenkrantz
10ed67867e zink: use screen param for zink_shader_free()
the context is no longer used

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18689>
2022-09-23 17:02:11 +00:00
Mike Blumenkrantz
275e4a9c89 zink: add locking for program caches
these are context-based, so destroying them from another context needs locks

locks are not currently able to be contended

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18689>
2022-09-23 17:02:11 +00:00
Mike Blumenkrantz
eab18cb7a2 zink: use screen params for program reference functions
these never used the context anyway

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18689>
2022-09-23 17:02:11 +00:00
Mike Blumenkrantz
697eb99d69 zink: track the owner context for gfx programs
Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18689>
2022-09-23 17:02:11 +00:00
Mike Blumenkrantz
2fa7091263 zink: track a mask of gfx stages remaining in gfx programs
this provides a hint to improve program deletion

cc: mesa-stable

Reviewed-by: Adam Jackson <ajax@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18689>
2022-09-23 17:02:11 +00:00
Mike Blumenkrantz
dcc5261895 zink: implement async nir creation
this just punts to the compile threads directly

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18198>
2022-09-10 19:01:56 +00:00
Mike Blumenkrantz
7fc1ab4928 zink: defer all compute shader creation
with other various changes, this should enable the entirety of compute
shader creation to occur in the compile thread

it also enables (slightly) simplifying shader create by moving cube check
out to the caller

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
5489b1a8ff zink: generate sha1 for compute programs
this got broken in the move to u_live_shader_cache, which does not
handle compute programs and thus does not populate the shader's sha1

cc: mesa-stable

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
68f319a9b8 zink: init fbfetch/bindless ctx stuff on shader create
deferring until later will be problematic for threads

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
7ddd8e428b zink: add a context pointer to zink_program struct
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
a327ff90d2 zink: add partial async shader compile support
when compute shaders can be precompiled, they can be precompiled asynchronously
which allows the implementation of the parallel shader compile hooks

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
eb9ec4baa7 zink: add precompilation for compute shaders
for compute shaders that don't need spec constants or cube lowering,
precompiles are possible and can be performed immediately after disk
cache lookup completes

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
b2696ca58e zink: add a param to allow zink_screen_update_pipeline_cache to run directly
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
e1ad7bf0b7 zink: move compute pipeline cache update to caller
this is more flexible

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
48ebc60e80 zink: use different key comparison for local_size-using compute pipelines
this should be more correct

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
8731ffa238 zink: stop zeroing local size if current compute doesn't use it
this just complicates things for later compute updates

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
ad96d21239 zink: remove use_local_size from compute pipeline state
this is redundant since pipeline state gets flagged on bind

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
bbd58d1703 zink: don't add compute base pipeline to hash table
this is pointless since it has no lookup data

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
d0c8e70a64 zink: merge compute program hash table onto program struct
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
3e9a108c78 zink: add a pipeline shortcut for basic compute programs
if there are no inline uniforms, nonseamless cubes, or local size use,
then this is the "base" pipeline object that can be reused without checking
the hash table

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
8cdb227fe7 zink: add param to allow cache_get jobs to run directly
this will be useful for precompiles

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
3a674fc6eb zink: add use_local_size flag to compute programs
Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
4cb4bb555e zink: create compute programs from compute shaders directly
this simplifies the whole compute shader/program architecture and
also compiles compute shaders when apps maybe expect them to be compiled

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
06e0c05842 zink: use screen param for zink_descriptor_program_deinit
this better enables late destruction of programs

Reviewed-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18197>
2022-08-30 04:11:16 +00:00
Mike Blumenkrantz
6a836eaff5 zink: track which stages in gfx programs are using inlined uniforms
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
bc46d44ba5 zink: constify shader module variable
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
98fdcd9f9b zink: pass shader stage as param for gfx module update
less register pressure

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
145137df89 zink: use dynarray instead of list for internal shader cache
this is less likely to obliterate cpu caches

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
d8455e5ebf zink: change u_foreach_bit to regular for loop in zink_gfx_program_update
too hot to be using this

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
47d83e609a zink: cache dynamic render vs renderpass pipelines separately
removes comparisons from hash lookup

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
fbade5c71a zink: add templates for pipeline stage comparison
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
a0e69e7601 zink: split out first-time shader module creation
this simplifies the code a bit and makes it more flexible

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
f5008d4370 zink: split out gfx shader module creation from cache searching
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
db00e794b2 zink: split out nonseamless/inline shader module info calculation
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
4de0f7acd9 zink: plumb screen info through shader key comparison
Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
b8a7b912d3 zink: plumb some screen info through gfx shader creation
this should allow optimizing out some parts of the code

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
190e5bda11 zink: simplify shader key comparison by splitting out non-generated tcs
non-generated tcs has unique mechanics in that it doesn't have a base shader key,
so split that out to avoid unnecessary complexity

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
59fa328589 zink: rework bit iteration in update_gfx_shader_modules to use for loop
this is roughly 1% faster in drawoverhead case 7

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
d862871682 zink: move pipeline state comparison to c++ template
the other part of the pipeline update hotpath

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
c886a7a423 zink: move gfx pipeline recalc to c++
this is one of the hottest paths in the driver, and having to load
a function variant with all the extra dynamic state paths is not optimal
when they can never be used

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00
Mike Blumenkrantz
5457a70ce3 zink: move pipeline cache structs to zink_program.h
these shouldn't be widely used, so they're not in zink_types.h

Reviewed-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/18135>
2022-08-20 01:27:13 +00:00