panfrost: Remove silly assertion

We don't assert the rest of the Gallium contract here either.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10869>

src/gallium/drivers/panfrost/pan_context.c

@@ -420,9 +420,6 @@ panfrost_direct_draw(struct panfrost_batch *batch,
         int mode = info->mode;
 
         /* Fallback for unsupported modes */
-
-        assert(ctx->rasterizer != NULL);
-
         if (!(ctx->draw_modes & (1 << mode))) {
                 if (draw->count < 4) {
                         /* Degenerate case? */