OpenHarmony/third_party_mesa3d
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
192,204 Commits 9 Branches 1 Tag
032d4a20f90006ce421a78ae30ff8c9f7469d27e
Commit Graph

118 Commits

Author SHA1 Message Date
Eric Engestrom
db6541a41a bin/gitlab_gql: fix --print-merged-yaml when --rev != HEAD 
Reading the local root config file and then asking gitlab to evaluate it
in the context of some other version will cause issues if they are not
identical.

Instead, the local document should be a simple include of whatever is
the root config file at that commit.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26074>
 2023-12-14 12:06:39 +00:00
Eric Engestrom
76be39ca6d ci_run_n_monitor: allow picking a pipeline by its MR 
It picks the latest pipeline from the MR; if you want another pipeline,
use `--pipeline-url`.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26381>
 2023-12-03 11:28:49 +00:00
Helen Koike
2999091287 ci/ci_run_n_monitor: abort when target gets skipped 
when a target receives skipped state it is because some of the
dependencies failed, abort the script and print the reason

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26155>
 2023-11-20 13:03:09 +00:00
Eric Engestrom
a5e20a5c31 ci_run_n_monitor: require user to add an explicit .* at the end if jobs like *-full are wanted 
Most of the time, these jobs are not wanted, so let's make this a full
match instead of prefix match so that users only get what they ask for.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26066>
 2023-11-17 08:12:31 +00:00
Faith Ekstrand
94ea8a5883 ci: Update the python env for ci_run_n_monitor.py 
Without this, it blows up on Fedora 39 because of changes introduced by
Python 3.12 which break a a couple of our dependencies.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24998>
 2023-11-14 00:48:15 +00:00
Eric Engestrom
1cb3c349ff bin/gitlab_gql: give a better name to the --print-job-manifest argument value than PRINT_JOB_MANIFEST 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26143>
 2023-11-13 20:03:44 +00:00
Eric Engestrom
22961fc45a bin/gitlab_gql: deduplicate fetch_merged_yaml() logic between print branches 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26143>
 2023-11-13 20:03:44 +00:00
Eric Engestrom
ef63cc6017 bin/gitlab_gql: rename get_job_final_definition() to print_...() since that's what it actually does 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26143>
 2023-11-13 20:03:44 +00:00
Eric Engestrom
c6ef161bc1 bin/gitlab_gql: only allow a single --print-* argument per invocation 
Printing multiple things as the same time is not really usable, so let's
be explicit and only allow one at a time, which allows a few cleanups in
the code.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26143>
 2023-11-13 20:03:44 +00:00
Eric Engestrom
f4f4d80096 bin/gitlab_gql: only get the pipeline when a pipeline is needed 
Otherwise, options like --print-merged-yaml are broken when their
commit's push didn't create a pipeline.

Fixes: b87e092489 ("ci/bin: Fix gitlab_gql methods that uses needs DAG")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26142>
 2023-11-11 06:19:44 +00:00
Eric Engestrom
656afd8ede bin/gitlab_gql: fix command in example 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26141>
 2023-11-10 10:37:07 +00:00
Eric Engestrom
2cf031155b gitlab_gql: make --rev optional, defaulting to HEAD 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26073>
 2023-11-10 10:05:46 +00:00
Eric Engestrom
cc37af8fbc bin/gitlab_gql: resolve sha locally to be able to use things like HEAD 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26073>
 2023-11-10 10:05:46 +00:00
Guilherme Gallo
f8b8352022 ci/bin: Print a summary list of dependency and target jobs 
We already print all the detected target jobs from regex and its
dependencies. But for more complex regexes the list can be cumbersome,
and an aggregate list of dependencies and targets can be more value, so
add these prints as well.

This is what looks like:

```
Running 10 dependency jobs:
 alpine/x86_64_lava_ssh_client, clang-format, debian-arm64,
 debian-testing, debian/arm64_build, debian/x86_64_build,
 debian/x86_64_build-base, kernel+rootfs_arm64, kernel+rootfs_x86_64,
 rustfmt

Running 15 target jobs:
 a618_gl 1/4, a660_gl 1/2, intel-tgl-skqp, iris-amly-egl, iris-apl-deqp
 1/3, iris-cml-deqp 1/4, iris-glk-deqp 1/2, iris-kbl-deqp 1/3,
 lima-mali450-deqp:arm64, lima-mali450-piglit:arm64 1/2,
 panfrost-g52-gl:arm64 1/3, panfrost-g72-gl:arm64 1/3,
 panfrost-t720-gles2:arm64, panfrost-t860-egl:arm64, zink-anv-tgl
```

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Guilherme Gallo
e74238af42 ci/bin: Fix mypy errors in gitlab_gql.py 
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Guilherme Gallo
b87e092489 ci/bin: Fix gitlab_gql methods that uses needs DAG 
Some gitlab_gql.py features like `--print-dag` were affected by recent
changes. Update those functions with the refactored data.

Fixes: c7b67d8619
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Guilherme Gallo
278fc1c22a ci/bin: gql: Improve queries for jobs/stages retrieval 
Modify the GraphQL query used to fetch all jobs within a pipeline,
transitioning from fetching data via stage nodes to a direct job
retrieval approach.

The prior method was not paginated, potentially overloading the server
and complicating result parsing due to the structure of stage nodes. The
new approach simplifies data interpretation and handles job lists
exceeding 100 elements by implementing pagination with helper functions
to concatenate paginated results.

- Transitioned from extracting jobs from stage nodes to a direct query
  for all jobs in the pipeline, improving data readability and server
  performance.
- With the enhanced data clarity from the updated query, removed the
  Dag+JobMetadata tuple as it's now redundant. The refined query
  provides a more comprehensive job data including job name, stage, and
  dependencies.
- The previous graph query relied on a graph node that will (or should)
  be paginated anyway.

Closes: #10050
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Guilherme Gallo
664e6addea ci/bin: gql: Implement pagination 
Make query support pagination by supplying the paginated key.

In the following toy example, the paginated key is:
["levels", "cars"]

```graphql
query vehicle_store($location: ID!) {
  levels {
     cars {
        pageInfo {
          hasNextPage
          endCursor
        }
     ...
     }
  }
}
```

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Guilherme Gallo
c4b8c03012 ci/bin: gql: Log the caching errors 
When using cache and it fails, log the errors, clear the cache and retry
the query bypassing the cache

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Guilherme Gallo
aa2586b315 ci/bin: gql: make the query cache optional 
Make the caching optional, as it can fail in some complex queries where
the transport middleware uses non-pickle-able objects.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Guilherme Gallo
609b4bfef8 ci/bin: Replace AIOHTTPTransport with RequestsHTTPTransport 
For some reason AIOHTTPTransport started to use MultiDict after doing
some adjustments in the GraphQL query, which made `filecache` fail
because MultiDict object are not pickle-able.

Changing the transport strategy from AIOHTTPTransport to
RequestsHTTPTransport, which dropped one requirement. We aren't doing
async anyway, all the calls were sync before.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Guilherme Gallo
46b033a9bf ci/bin: Fix find_dependency function calls 
`find_dependency` signature changed, so we need to update their callings
correctly.

Fixes: 94655ff811

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25940>
 2023-11-08 02:18:17 +00:00
Eric Engestrom
ba4f6e87c7 ci/gitlab_gql: stop re-compiling regex now that all users pre-compile it 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26070>
 2023-11-07 06:08:44 +00:00
Eric Engestrom
94655ff811 ci_run_n_monitor: compile target_jobs_regex only once 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26070>
 2023-11-07 06:08:44 +00:00
Eric Engestrom
a5cb331c64 gitlab_gql: strip newline at the end of the token file 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/26075>
 2023-11-07 06:06:22 +00:00
Guilherme Gallo
969ede4521 ci/bin: Refactor create_job_needs_dag 
The function is getting too big, let's add comments, docstrings to the
most important function, new type hints and extract methods from it to
make it easier to read.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25858>
 2023-10-27 00:36:48 +00:00
Guilherme Gallo
4e4743ec0a ci/bin: Do not forget to add early-stage dependencies 
In Mesa MR pipelines, we have the sanity job. This job is not an
explicit need for any one job, because only jobs with `- needs: []`
ignores previous jobs. If no `needs` is specified, the job will wait
until all the jobs of earlier stages are finished before starting.

See additional details section at:
https://docs.gitlab.com/ee/ci/yaml/index.html#stage-post

Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/10032
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25858>
 2023-10-27 00:36:48 +00:00
Guilherme Gallo
c7b67d8619 ci/bin: Use iid instead of SHA in gitlab_gql 
We were using sha to fetch the pipeline from GraphQL, but that leads to
wrong results when MR and branch pipelines exist. For example,
using pipeline-url as the MR pipeline:
- https://gitlab.freedesktop.org/gallo/mesa/-/pipelines/1017182
This would lead into the branch pipeline:
- https://gitlab.freedesktop.org/gallo/mesa/-/pipelines/1013189

Also simplify the GQL query, it had lots of unused data.

Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25858>
 2023-10-27 00:36:48 +00:00
Christian Gmeiner
a53a394d30 ci: Bump PyYAML to 6.0.1 
Fixes an issue I am seeing on Fedora 39. For more details
about the upstream fix see https://github.com/yaml/pyyaml/pull/702

Signed-off-by: Christian Gmeiner <cgmeiner@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25878>
 2023-10-25 10:35:47 +02:00
Eric Anholt
74315051a9 ci_run_n_monitor: Always resolve --rev arguments for looking up pipelines. 
Otherwise you'd loop forever never finding it with --rev <paste from
gitlab UI> or --rev <branch name>

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25824>
 2023-10-25 01:48:18 +00:00
Helen Koike
663ad191ca ci/ci_run_n_monitor: print job duration time 
If the job is running, it prints the elapsed time so far, if it finished
already, it prints the time it took.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25822>
 2023-10-24 12:57:14 +00:00
Helen Koike
69012e355e ci/marge_queue: add pretty_dutation() 
Add pretty_duration() function that prints time in format 6m23s and use
it for marge_queue.

Signed-of-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25822>
 2023-10-24 12:57:14 +00:00
Helen Koike
9f4f52f4f3 ci/ci_run_n_monitor: keep monitoring if a job is still running 
When running multiple targets, the script was returning if any of the
targets failed.
But sometimes some of the targets are still running and we want to
monitor it until it finishes, even if some of the targets failed.

So check if any targeted jobs are in pending or running before deciding
to finish.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25826>
 2023-10-22 09:30:56 +00:00
Helen Koike
46fc9decad ci/marge_queue: add missing python-dateutils to requirements.txt 
marge_queue.py requires python-dateutils to run, so add it.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25828>
 2023-10-20 18:27:44 +00:00
Eric Anholt
fb95f1d55c ci_run_n_monitor: Poll mesa/mesa and user/mesa for pipelines at the same time. 
Now you don't fail if you're trying to test a mesa/mesa MR pipeline and
gitlab takes more than 10s to create it.  And you don't have to wait 10
seconds to get things started (aka see if your regex was right) if you're
testing a user/mesa fork pipeline.

Fixes: 941d92408e ("bin/ci_run_n_monitor: automatically pick MR pipelines when they exist")
Closes: #9894
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25810>
 2023-10-20 08:23:24 +00:00
David Heidelberg
0b31cda678 ci: ci_marge_queue.py 
Show currently assigned jobs to Marge and return 0 when it's free.

Useful for combination with ci_run_n_monitor.py .

Signed-off-by: David Heidelberg <david.heidelberg@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20347>
 2023-10-14 11:28:56 +00:00
Eric Engestrom
d3e96aa02e ci_run_n_monitor: dependency jobs must always be started 
Fixes: 6b49b477ac ("ci/ci_run_n_monitor: simplify enable/cancel logic in monitor_pipeline()")
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25715>
 2023-10-13 18:37:56 +00:00
Helen Koike
51521e1f9d ci/ci_run_n_monitor: limit repetitions on --stress 
--stress options now receives the number of repetitions or -1 to behave
as before.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25473>
 2023-10-11 21:50:58 +00:00
Helen Koike
57fa35f19c ci/ci_run_n_monitor: allow <user>/<project> in --project 
Allow to monitor pipelines in any <user>/<project> on gitlab.fdo.

If developers want to monitor MR pipelines, they can run with
--project mesa/mesa to force it to be picked from mesa username.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25473>
 2023-10-11 21:50:58 +00:00
Helen Koike
6b49b477ac ci/ci_run_n_monitor: simplify enable/cancel logic in monitor_pipeline() 
leave to enable_job() and cancel_job() to do the right thing according
to the job status.

let target canceled jobs to be re-runned.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25473>
 2023-10-11 21:50:58 +00:00
Helen Koike
17843ad7c6 ci/ci_run_n_monitor: merge enable_job with retry_job 
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25473>
 2023-10-11 21:50:58 +00:00
Helen Koike
90da4a6845 ci/ci_run_n_monitor: make --target mandatory 
There is no point running without a target, just make it mandatory and
simplify the script.

Use ".*" to run all targets.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25473>
 2023-10-11 21:50:58 +00:00
Helen Koike
633c08688b ci/ci_run_n_monitor: merge print_job_status_change with print_job_status 
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25473>
 2023-10-11 21:50:57 +00:00
Helen Koike
7f6c339740 ci/ci_run_n_monitor: simplify with defaultdict 
Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25473>
 2023-10-11 21:50:57 +00:00
Helen Koike
f838499901 ci/ci_run_n_monitor: print stress test results per job 
Since we can monitor multiple jobs, print the result of the stress test
per job name.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25473>
 2023-10-11 21:50:57 +00:00
Helen Koike
fa884141dc ci/ci_run_n_monitor: add docs for multiple targets 
clarify how multiple targets can be used with the pipe character.

Signed-off-by: Helen Koike <helen.koike@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25472>
 2023-09-29 09:50:53 -03:00
Eric Engestrom
7eabe9bb40 ci_run_n_monitor: add comment to explain "MR > fork" logic 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25186>
 2023-09-12 18:37:40 +00:00
Eric Engestrom
941d92408e bin/ci_run_n_monitor: automatically pick MR pipelines when they exist 
When an MR has been created, we usually want to run the jobs in the MR
pipeline so that reviewers see that things work as expected.

Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25153>
 2023-09-12 17:42:34 +00:00
Eric Engestrom
f398f0fb44 bin/ci_run_n_monitor: print in which repo we're looking for the pipeline 
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/25153>
 2023-09-12 17:42:33 +00:00
Eric Engestrom
8366d77e4c bin/ci_run_n_monitor: error out if both --project and --pipeline-url are passed 
Signed-off-by: Eric Engestrom <eric@igalia.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24937>
 2023-08-29 19:27:22 +00:00