Currently marge will ignore an error message if it is just the word
"error" without any further information. However, if she never finds a
more informative message, then she will just print that meaningless
error message.
Instead of an empty error message, just leave the message blank.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32773>
If marge doesn't find deqp-runner results or trace errors, she will parse
a failed job log for error messages. The parsing ignores certain phrases
in an attempt to find more relevant errors.
Add a test to check marge's error log parsing to make sure that we don't
create regressions when adding new phrases to ignore.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32773>
Currently the update_traces_checksum script prints a label verification
request with a line that is 167 characters long.
Split the long line to make it more readable. Update the flake8
configuration to enforce a maximum line length of 159 characters to ensure
consistency.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32602>
Currently the ci scripts and tests don't have any linting checks. Add
.flake8 linting to start adding some consistency to the scripts. Ignore
most of the existing errors until they can be addressed on an individual
basis.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32602>
When the gantt chart service was first implemented, the read_token()
function would attempt to read from the default token file if the token
argument was missing.
Now it's necessary to call a separate function to look in the default
token file, so if the token argument is not provided, expressly call
get_token_from_default_dir().
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32637>
The gantt charts have a vertical line indicating the time when marge-bot
stops waiting for CI to finish. Currently the ci-timeout is hard-coded at
60 minutes. But marge-bot's timeout is configurable, so allow this value
to be configured as well.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32637>
If a job is still running when the gantt chart is generated, the job's
phases won't be represented as a bar on the chart.
To show these jobs on the gantt chart, introduce a new phase "In-Progress"
to capture the current status of these jobs. Rename the "Running" phase
as "Time spent running" to emphasize that the running phase is complete.
This provides a more complete picture of the pipeline jobs when CI times
out.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32637>
The gantt chart script currently expect the pipeline url to be the last
line of marge-bot's comment on a failed pipeline. The pipeline summary
hook now adds additional information that was preventing the gantt chart
script from finding the pipeline url.
Change the regex for finding the pipeline url to account for pipeline
summary comments.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32637>
Add a pipeline-id to the gantt filename so that the filename is unique.
Reduce the additional info provided in the gantt comments since the tool
is now more familiar to developers.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32637>
Currently the gantt chart script will post a comment to any project that
uses the default marge bot user id. Although the gantt chart is flexible
enough to work for any project, restrict it to mesa/mesa by default. Add
the --project-id option to add a custom project or list of projects to
analyze.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32637>
The gantt chart currently hard-codes the marge user id, but since the
script can be used for any marge-bot user, add an option to specify the
marge-bot user id. If no option is specified, the default marge-bot user
id is still used.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32637>
Move the tool to summarize a failed pipeline to a generic .marge/hooks
directory. This will allow the fdo-bots repo to handle all marge hooks in
a consistent way across repositories that use this service.
Add a symlink to the bin/ci directory so that the pipeline summary tool
can still be run locally as well.
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32413>
Add an argument to ci_run_n_monitor specifying certain stages to be
excluded from consideration, defaulting to the one with post-merge and
performance jobs. This allows, e.g., to run all Panfrost pre-merge jobs:
./ci_run_n_monitor.py --target 'panfrost.*'
or to run all Freedreno pre-merge jobs:
./ci_run_n_monitor.py --target '.*' --include-stage freedreno
Signed-off-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30784>
The skip_follow_statuses variable, used to check if we need to stay
monitoring the pipeline instead of jumping to the target job traces, is
based on COMPLETED_STATUSES set. But, in Python, we do shallow copies by
default, and changes on skip_follow_statuses reflected on
COMPLETED_STATUSES, which was making manual dependencies stop playing
when --force-manual was not given.
Fixes: 84d401aebf0832741716f947dd7e2e9aac1221ac
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30526>
Some recurrent messages are only cluttering the ci_run_n_monitor's logs,
so let's use the easily provided @cache decorator to print stuff that
the user will only needs to see once.
Also removes some junk that can happen during the pipeline monitoring
loop, like the "----" line break, and newlines.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30361>
We stop monitoring pipeline changes when we find out that the target job
is already running, or is in a complete state.
But when we `--force-manual` we should consider that the job is not
complete and we should iterate a little more in the monitor pipeline
stage until it gets ready for trace following.
Closes: #11552
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30361>
Jobs that we want to run can get the `created` status when its
dependencies are still running.
So let's gather this information and ensure that we will wait these jobs
to reach the `manual` or `running` status before jumping to monitor
target jobs trace.
Closes: #11517
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30361>
This reverts commit 032d4a20f9.
The `if not to_cancel: return` was a red herring as what actually matters
is the job status, which is checked in each cancel_job() call, so we
can't know in advance whether anything will be cancelled, so let's just
drop this text explanation.
In the meantime we've also improved the emoji next to cancelled jobs, so
let's hope there is no longer any need to explain what this long list of
job names means.
Fixes: 032d4a20f9 ("bin/ci_run_n_monitor: explain that the 'Universal Recycling symbol' ♲ emoji means these jobs were cancelled")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/30265>
