ir3: Fix detection of nontrivial continues

We may still need to insert a continue block even if there is only one
backedge, in a situation like:

for (...) {
   if (...) continue;
   foo();
   break;
}

We want foo() to be executed before reconverging. This is important for
the BVH encoding kernel, which launches an invocation for each node in
the tree and does a preorder traversal:

while (true) {
   if (!ready[node]) continue;
   encode();
   for (child node)
      ready[child] = true;
   break;
}

For the first few nodes, which will be in the same wave, we need
encode() for the root node to be called first, then its children spin
until ready, then the children call encode(), and so on. This can only
work if the children that aren't ready yet are parked while the parent
executes encode(), which requires the continue block.

This is also required because divergence analysis will assume that
uniform values written before the continue are still uniform after it,
which isn't the case now and causes an RA validation failure with Godot.

Fixes: 0fa93fb662 ("ir3: Fix convergence behavior for loops with continues")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/31905>
This commit is contained in:
Connor Abbott
2024-03-25 11:18:40 -04:00
committed by Marge Bot
parent c245609b64
commit d3533716f9

View File

@@ -4493,6 +4493,32 @@ emit_if(struct ir3_context *ctx, nir_if *nif)
emit_cf_list(ctx, &nif->else_list);
}
static bool
has_nontrivial_continue(nir_loop *nloop)
{
struct nir_block *nstart = nir_loop_first_block(nloop);
/* There's always one incoming edge from outside the loop, and if there
* is more than one backedge from inside the loop (so more than 2 total
* edges) then one must be a nontrivial continue.
*/
if (nstart->predecessors->entries > 2)
return true;
/* Check whether the one backedge is a nontrivial continue. This can happen
* if the loop ends with a break.
*/
set_foreach (nstart->predecessors, entry) {
nir_block *pred = (nir_block*)entry->key;
if (pred == nir_loop_last_block(nloop) ||
pred == nir_cf_node_as_block(nir_cf_node_prev(&nloop->cf_node)))
continue;
return true;
}
return false;
}
static void
emit_loop(struct ir3_context *ctx, nir_loop *nloop)
{
@@ -4502,12 +4528,11 @@ emit_loop(struct ir3_context *ctx, nir_loop *nloop)
struct nir_block *nstart = nir_loop_first_block(nloop);
struct ir3_block *continue_blk = NULL;
/* There's always one incoming edge from outside the loop, and if there
* is more than one backedge from inside the loop (so more than 2 total
* edges) then we need to create a continue block after the loop to ensure
* that control reconverges at the end of each loop iteration.
/* If the loop has a continue statement that isn't at the end, then we need to
* create a continue block in order to let control flow reconverge before
* entering the next iteration of the loop.
*/
if (nstart->predecessors->entries > 2) {
if (has_nontrivial_continue(nloop)) {
continue_blk = create_continue_block(ctx, nstart);
}