This commit fixes the following flaws in the implementation:
* when a resource was re-allocated, the guest side storage
was also allocated
* when a source needs a readback before being written to, then
the call would go through vws->transfer_get, thereby bypassing the
staging resource, and this would fail on the host, because no
the allocated IOV was too small (just one byte)
* if the texture write would need neither flush nor readback, the
old code path would be used expecting that guest side backing stogage
for the texture.
v2: - actually do a readback to the stageing resource when it is required
- fix typo (Lepton)
v3: Don't use stageing transfers if the host can't read back the data
by rendering to an FBO or calling getTexImage, because in this case
we rely on the IOV to hold the date.
v4: Also don't use staging transfers if the format is no readback
format. Otherwise we have to deal with the resolve blit, and
this is currently not working correctly.
v5: add a new flag that indicates whether non-renderable textures can
be read back (either via glGetTexImage or GBM)
v6: Restrict the use of staging texture transfers to textures that can
be read back, and on GLES also if the they are bound to scanout and
the host uses minigbm to allocate such textures.
For that replace the flag indicating the capability to read back
non-renderable textures with a cap that indicates whether scanout
textures can be read back.
v7: update virglrenderer version in the CI
v8: update use of stageing (Chia-I)
v9: remove superflous check and assignment (Chia-I)
v10: disable stageing textures for arrays with stencil format. This is a
workaround for failures of the CI.
Fixes: cdc480585c
virgl/drm: New optimization for uploading textures
Signed-off-by: Gert Wollny <gert.wollny@collabora.com>
Reviewed-by: Chia-I Wu <olvaffe@gmail.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14495>
Run crosvm as a background process in order to allow intercepting
interrupt signals (INT, TERM) and properly release/cleanup any allocated
resources.
This is particularly helpful when one or more crosvm tasks hang, which
will eventually prevent subsequent instances to be started - currently
we can handle up to 128 concurrent crosvm instances per runner.
Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/15238>
- Add the scripts to the prepared Mesa artifacts for use in later
runner stages.
- Add a template generator (generate_b2c.py) which reads and
validates (very lightly for now) the Gitlab job environment and then
spits out a YAML file describing the necessary test workload to be
sent to a Valve CI gateway.
Acked-by: Samuel Pitoiset <samuel.pitoiset@gmail.com>
Reviewed-by: Martin Roukala <martin.roukala@mupuf.org>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14660>
This commit adds support for Vulkan backend on a630_skqp job.
= Needed changes
- Needed to install libvulkan-dev package on system
- Refactored the way the available skqp reports are printed
tested in development builds with skia tools
Piglit expectations had to be updated in various drivers due to !14750 not
having bumped the tags when it tried to uprev.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14686>
The Android CTS 10 version is relative old when compared with skia main
branch, which was being used before. Some modifications in the skqp
build/runner scripts were needed to make it run on CI.
- skqp versions from android-cts have already all assets inside
platform_tools folder.
- along with the assets, are the render and unit files which are
expected to pass in the Android CTS execution.
- removed custom test files from the a630 folder, to make it comply
with the CTS expectations.
- include new patches to remove Python2 dependencies and avoid the
installation of it in rootfs.
- strip binariesthe built binaries `skqp` and `list_gpu_unit_tests`, as
`is_debug = false` gn argument did not work, maybe it is not well
tested in development builds with skia tools
- use Clang instead of GCC. The GCC support is not so graceful as it is
in the skia main branch, some NEON instructions needs to be turned off
in the GCC compilation, causing different tests result. This change
does not imply a bigger rootfs, since the built skqp binary uses GCC
libc++ and other library runtimes. So clang is just a build
dependency.
= Changes in skqp results =
Some errors were found for GL backend and unit tests. GLES and VK tests are green.
All the failed tests were classified as expected to fail in the render and unit tests list.
```
gl_blur2rectsnonninepatch
gl_bug339297_as_clip
gl_bug6083
gl_dashtextcaps
```
```
SRGBReadWritePixels (../../tests/SRGBReadWritePixelsTest.cpp:214 Could not create sRGB surface context. [OpenGL])
```
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14686>
The mechanism currently used to pass data from the dEQP child process
executed in a crosvm guest environment towards the deqp-runner wrapper
script that starts the crosvm instance is based on creating, writing
and reading regular files.
In addition to the main drawback of using the storage, this approach
is potentially unreliable because the data cannot be transferred in
real-time and there is no control on ending the transmission. It also
requires a forced sleep for syncing the content, while the minimum
amount of time necessary to wait cannot be easily and safely
determined.
Replace this with an IPC based on the virtio transport for virtual
sockets (virtio-vsock).
Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14995>
lava_job_submitter.py unit tests are written in pytest and uses
freezegun in order to simulate timeouts in some tests scenarios. So,
this commit adds the packages `python3-pytest` and `python3-freezegun`
to fulfill this dependencies.
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14876>
When the lava_job_submitter.py retry loop finishes normally (without
falling through break-loop) it means that the submitter has exceeded the
retry count limit. However, when it happens the script
finishes normally. This patch adds a treatment to this case, warning the
user what happened and forcing the job to fail.
Moreover, this commit will make retry configurations configurable by
CI job, as it can take the default value from the following variables:
- LAVA_DEVICE_HANGING_TIMEOUT_SEC
- LAVA_WAIT_FOR_DEVICE_POLLING_TIME_SEC
- LAVA_LOG_POLLING_TIME_SEC
- LAVA_NUMBER_OF_RETRIES_TIMEOUT_DETECTION
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14876>
The main thing is VK 1.3 testing, but also includes test bugfixes. The
1.3 CTS required an uprev of deqp-runner to handle a new style of test
output, and that deqp-runner brings in some neat new features, too (piglit
in your deqp-runner suite, and extension list checking).
A bunch of VK tests got renamed, so I replaced panvk's custom test list
with simple include filters on the main test list.
Acked-by: Lionel Landwerlin <lionel.g.landwerlin@intel.com>
Acked-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Boris Brezillon <boris.brezillon@collabora.com> (panvk)
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14920>
This brings in some interesting new vulkan tests and fixes for the
spurious KHR-GL TF failures. Also, reduces the runtime of
dEQP-GLES31.functional.ssbo.layout.random.all_shared_buffer.36 so that it
should stop timing out.
Acked-by: Dave Airlie <airlied@redhat.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13779>
crosvm-runner.sh was using `export -p` to create an environment script
for the virtualized system, but this command will dump every declared
environment variable in the system, which includes Gitlab's CI variables
with sensitive data, such as passwords and auth tokens.
Replacing `export -p` to `generate-env.sh`, which only exports the
necessary variables for Mesa CI jobs.
Extra changes:
* Stop changing ${PWD} variable programmatically in scripts. ${PWD} is a
variable used by most prolific coreutils and bash commands, such as `cd`
and `pwd`, besides it is set by subshells [1]; changing this variable
may lead to complex situations.
As drop-in replacement for ${PWD}, use ${DEQP_BIN_DIR} to flag that
there is a special folder where dEQP should be run.
* Double quote path and array variables. See: https://github.com/koalaman/shellcheck/wiki/SC2086
* Do not export variables directly from commands output. See: https://github.com/koalaman/shellcheck/wiki/SC2155
[1]
```
$ cd /tmp
$ export PWD=test; bash -c 'echo $PWD'
/tmp
```
v2:
- Revert $DEQP_BIN_DIR quoting in crosvm-runner.sh and crosvm-init.sh
- Log all the passed variables to stdout, to help with debugging when
new variable are needed to be put in `generate-env.sh`
v3:
- Revert $DEQP_BIN_DIR quoting leftovers
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Reviewed-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14626>
The first container stage ("build") is for dependencies of the build.
These are infrequently-changing things like Visual Studio, LLVM, git,
and also meson. The second container stage ("test") currently depends
on the first, and adds test dependencies like piglit.
This lets us rev piglit without having to rebuild LLVM.
Acked-by: Daniel Stone <daniels@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14637>
In some cases the file paths passed to crosvm for execution do not point
to dEQP binaries, but can be wrapper scripts, like deqp-runner.sh.
Detect such cases and skip changing the working directory.
Additionally, use the POSIX compliant command substitution syntax
instead of the obsolete variant based on backquotes.
Fixes: 81f25d8f27 ("virgl/ci: Run each dEQP instance in its own VM")
Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14413>
In some corner cases like the kernel oops, we do not get the relevant
log messages from crosvm process to help with debugging.
Note there is currently a double redirection of its stdout stream, but
the content eventually ends up in /dev/null.
Let's fix this by redirecting both stdout and stderr streams to a
dedicated file, to avoid clobbering the output from the script/program
running inside crosvm. This is particularly required for the scenario
that involves deqp-runner starting crosvm via a *.toml suite.
Additionally, drop the unnecessary usage of 'stdbuf' and set the 'quiet'
kernel command-line parameter to get rid of the noise generated during
crosvm boot process.
Although not directly related, do some cleanup by removing the
temporary folder on script exit.
Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14413>
Ensure virglrenderer library is built before crosvm in order to allow
dynamic linking. This is needed for the scenarios where a different
virglrenderer library must be provided before launching crosvm, e.g.:
the upcoming Virgl CI solution that shares Mesa CI containers.
Additionally, this provides the virgl_test_server binary which is
required by piglit-runner.sh and deqp-runner.sh scripts when using
the virpipe Gallium driver.
Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14413>
Add support for building and installing deqp-runner from a git source
repository to facilitate further development & testing of new features
or simply making use of the latest upstream changes which were not
already tagged as a new package version.
The git revision to be built must be specified by setting
'DEQP_RUNNER_GIT_REV' env variable. To specify a git tag name instead,
'DEQP_RUNNER_GIT_TAG' variable must be used. It is also possible to
indicate a custom git repository via 'DEQP_RUNNER_GIT_URL'.
If neither a git revision or tag name has been set, deqp-runner is
installed from the rust package registry (the default behavior).
v2: Make use of '--git' and '--rev' cargo args to automate the git
checkout operation (Rohan).
v3: Allow Git URL override by using the hardcoded URL only when
DEQP_RUNNER_GIT_URL is not already set.
v4: Override 'EXTRA_CARGO_ARGS' to optimize the script and avoid a
second call to 'cargo install'. Additionally, add support for
indicating git tags (Rohan).
Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@collabora.com>
Reviewed-by: Rohan Garg <rohan.garg@intel.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14413>
According to RFC 4189 CSV files should be encoded using CRLF newlines,
not LF. This helps compatibility with tools, like python's csv module,
who always uses CRLF.
While we're at it, normalize the one CSV that was CRLF in-repo to LF,
and let git do the newline-normalization when needed instead.
Reviewed-by: Eric Engestrom <eric@engestrom.ch>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/12405>
Brings in these changes:
af1785f31 occlusion_query_conform: skip GetQueryCounterBits test if needed
dad078717 occlusion_query_conform: convert to pilgit subtests
b52c1c761 glsl-1.30: test nested preprocessor concat
6c4da153b texture-storage: Fix subtest result handling of skips.
4343f19db fbo-integer: Remove the invalid DrawPixels test.
e3842f2fe arb_dsa: exclude stencil8 textures from test sets.
ce8649be7 spec/ext_external_objects: Fix build on Debian systems
4e553838f glsl: add basic tests for desktop GLSL invariant qualifier linking
7e61e5199 Tests for variable in and out of loop scope
f855ad1c8 fbo-mrt-alphatest: Only require GLSL 1.20
9be2fe999 glx: add glx-multi-display-single-pbuffer test
bfe290725 glx: add glx-swap-pbuffer test
efa64335e framework: Fix build on Windows when using waffle
Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Tomeu Vizoso <tomeu.vizoso@collabora.com>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14468>
This commit makes `kernel+rootfs_arm64` job build and install skqp on
ARM64 devices rootfs.
Skia repository has a tool to prepare skqp models located at
`tools/skqp/cut-release`, which get files from [Skia
Gold](https://skia.org/docs/dev/testing/skiagold/), generate
files.checksum, rendertests.txt and unittests.txt. One gives a range of
commits to let `cut-release` find the right resources to prepare skqp
for the user. However, it is failing, since it fails when trying to get
image packages from a range of commits via HTTPS from the host
https://public-gold.skia.org but it responds with error 404 every time.
I tried a range a thousand of commits, yet it still does not give
results. The workaround employed was to recover the most recent
`files.checksum` and `rendertests.txt` files from the git history and
generate `unittests.txt` from `list_gpu_unit_tests` binary.
`skqp` runs two lists of tests, `rendertests.txt` and `unittests.txt`.
Both must be located inside the `skqp` assets folder. The first list
uses GL and GLES to test rendering scenarios. The second runs some unit
tests that do not render an image per se.
In order to make the first `a630_skqp` to be green, the crashing tests
were removed from the test lists and the expectations of the failing
ones were updated.
It is worth noting that `rendertests.txt` can bring some detail about
each test expectation, so each test can have a max pixel error count, to
tell `skqp` that it is OK to have at most that number of errors for that
test. See also:
https://github.com/google/skia/blob/main/tools/skqp/README_ALGORITHM.md
As each render backend has a different error count, two different
`rendertests.txt` files were created,
`src/freedreno/ci/freedreno-a630-skqp-gl_rendertests.txt`,
`src/freedreno/ci/freedreno-a630-skqp-gles_rendertests.txt` and
, which one refers to GL and GLES tests respectfully.
The unit tests file for a630 is located at
`src/freedreno/ci/freedreno-a630-skqp_unittests.txt`
```
aaclip
domain
formats
highcontrastfilter
rectangle_texture
yuv_make_color_space
```
```
ProcessorOptimizationValidationTest
VkProtectedContext_CreateNonprotectedContext
VkYCbcrSampler_DrawImageWithYcbcrSampler
VkYCbcrSampler_NoYcbcrSurface
```
Each test was updated with the max_error count equal to the first run result.
```
analytic_antialias_inverse
async_rescale_and_read_dog_down
async_rescale_and_read_dog_up
async_rescale_and_read_rose
async_rescale_and_read_text_down
async_rescale_and_read_text_up
async_rescale_and_read_text_up_large
async_rescale_and_read_yuv420_rose
complexclip2_path_bw
encode-platform
imageblur_large
lcdtextsize
onebadarc
onefailarc
scale-pixels
surfaceprops
textfilter_color
textfilter_image
```
Considering all the following tests results as wrong.
```
async_rescale_and_read_no_bleed
backdrop_imagefilter_croprect_persp
complexclip2
imageblurrepeatmode
mixerCF
overdrawcolorfilter
patch_alpha
patch_primitive
rrect_clip_bw
scaledemoji_rendering
yuv_splitter
```
v2:
a) add link to HTML report on job log
b) remove extraneous spaces diff
c) remove unnecessary conditions from build-skqp.sh
d) use fixed skqp source commit SHA
v3:
a) Use only main skia repository to fetch models and build skqp
b) Use list_gpu_unit_tests binary to create a base unittests.txt file
c) Remove crashing tests
d) Set failing tests expectations for the first skqp run
v4:
a) Remove clang dependency
b) Separate each skqp backend result into its folder
c) Regroup a630_skqp in one job
v5:
a) Separate tests files per driver
Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5580
Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com>
Reviewed-by: Emma Anholt <emma@anholt.net>
Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14146>
