Add an argument to ci_run_n_monitor specifying certain stages to be
excluded from consideration, defaulting to the one with post-merge and
performance jobs. This allows, e.g., to run all Panfrost pre-merge jobs:
./ci_run_n_monitor.py --target 'panfrost.*'
or to run all Freedreno pre-merge jobs:
./ci_run_n_monitor.py --target '.*' --include-stage freedreno
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30784>
The skip_follow_statuses variable, used to check if we need to stay
monitoring the pipeline instead of jumping to the target job traces, is
based on COMPLETED_STATUSES set. But, in Python, we do shallow copies by
default, and changes on skip_follow_statuses reflected on
COMPLETED_STATUSES, which was making manual dependencies stop playing
when --force-manual was not given.
Fixes: 84d401aebf0832741716f947dd7e2e9aac1221ac
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30526>
Some recurrent messages are only cluttering the ci_run_n_monitor's logs,
so let's use the easily provided @cache decorator to print stuff that
the user will only needs to see once.
Also removes some junk that can happen during the pipeline monitoring
loop, like the "----" line break, and newlines.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30361>
We stop monitoring pipeline changes when we find out that the target job
is already running, or is in a complete state.
But when we `--force-manual` we should consider that the job is not
complete and we should iterate a little more in the monitor pipeline
stage until it gets ready for trace following.
Closes: #11552
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30361>
Jobs that we want to run can get the `created` status when its
dependencies are still running.
So let's gather this information and ensure that we will wait these jobs
to reach the `manual` or `running` status before jumping to monitor
target jobs trace.
Closes: #11517
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30361>
This reverts commit 032d4a20f9.
The `if not to_cancel: return` was a red herring as what actually matters
is the job status, which is checked in each cancel_job() call, so we
can't know in advance whether anything will be cancelled, so let's just
drop this text explanation.
In the meantime we've also improved the emoji next to cancelled jobs, so
let's hope there is no longer any need to explain what this long list of
job names means.
Fixes: 032d4a20f9 ("bin/ci_run_n_monitor: explain that the 'Universal Recycling symbol' ♲ emoji means these jobs were cancelled")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30265>
IMO emojis are nice to add next to the information to recognize things
easily in cases where they are visible, but they should not *replace*
the information.
This adds a readable text next to all emojis (some already had one,
like the "job duration" ones).
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30223>
A couple of these were trivially missing the `created` status which
usually doesn't stay for long enough for this very slow script to notice.
The `cancel_job()` function will no longer cancel manual jobs waiting to
be started.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29917>
When the tool is used to stress test a pipeline, if there are jobs already ran,
use their information like it does when the stress flag is not set.
This provides consistency between the behavior when stress argument is not set,
to when it is set. When it is not set, it uses the information about jobs that
are already done. When it is set, it has to use the information about the
already ran jobs. Also, it saves resources by triggering the minimum required.
Signed-off-by: Sergi Blanch Torne <sergi.blanch.torne@collabora.com>
Reviewed-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29432>
When running on a detached head (eg. checkout of a tag or a specific
commit), there is no active branch, so we can't perform this check; just
skip it and assume the user knows what they're doing.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27578>
This avoids the surprising behaviour where `--target jobname` works for
some jobs but not others, because gitlab adds `X/N` at the end of these
job names.
If the user does specify something like `jobname 1/.*` to only run the
first, the extra `\d+/\d+` is ignored, just like if the job isn't
`parallel:` and therefore doesn't end with `X/N`.
If the user really wants to fail to match parallel jobs (previous
behaviour), they can simply add a `$` at the end of the job name/regex
(but also, I don't see why someone would want that behaviour).
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27530>
Fix an issue in `ci_run_n_monitor.py` where the token was not
being correctly propagated to the GitlabGQL abstraction. This addresses
misbehavior in scenarios like running pipelines in a private fork,
ensuring proper functionality.
Also document `find_dependencies` function.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27206>
When generating the list of targets from a script, being able to just
pass `--target "${list[@]}"` is very convenient.
The list of targets is simply converted to an "or" regex, matching any
of the `--target`s given.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/27252>
We already print all the detected target jobs from regex and its
dependencies. But for more complex regexes the list can be cumbersome,
and an aggregate list of dependencies and targets can be more value, so
add these prints as well.
This is what looks like:
```
Running 10 dependency jobs:
alpine/x86_64_lava_ssh_client, clang-format, debian-arm64,
debian-testing, debian/arm64_build, debian/x86_64_build,
debian/x86_64_build-base, kernel+rootfs_arm64, kernel+rootfs_x86_64,
rustfmt
Running 15 target jobs:
a618_gl 1/4, a660_gl 1/2, intel-tgl-skqp, iris-amly-egl, iris-apl-deqp
1/3, iris-cml-deqp 1/4, iris-glk-deqp 1/2, iris-kbl-deqp 1/3,
lima-mali450-deqp:arm64, lima-mali450-piglit:arm64 1/2,
panfrost-g52-gl:arm64 1/3, panfrost-g72-gl:arm64 1/3,
panfrost-t720-gles2:arm64, panfrost-t860-egl:arm64, zink-anv-tgl
```
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
Modify the GraphQL query used to fetch all jobs within a pipeline,
transitioning from fetching data via stage nodes to a direct job
retrieval approach.
The prior method was not paginated, potentially overloading the server
and complicating result parsing due to the structure of stage nodes. The
new approach simplifies data interpretation and handles job lists
exceeding 100 elements by implementing pagination with helper functions
to concatenate paginated results.
- Transitioned from extracting jobs from stage nodes to a direct query
for all jobs in the pipeline, improving data readability and server
performance.
- With the enhanced data clarity from the updated query, removed the
Dag+JobMetadata tuple as it's now redundant. The refined query
provides a more comprehensive job data including job name, stage, and
dependencies.
- The previous graph query relied on a graph node that will (or should)
be paginated anyway.
Closes: #10050
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
