8992cf5ab8
This commit makes `kernel+rootfs_arm64` job build and install skqp on ARM64 devices rootfs. Skia repository has a tool to prepare skqp models located at `tools/skqp/cut-release`, which get files from [Skia Gold](https://skia.org/docs/dev/testing/skiagold/), generate files.checksum, rendertests.txt and unittests.txt. One gives a range of commits to let `cut-release` find the right resources to prepare skqp for the user. However, it is failing, since it fails when trying to get image packages from a range of commits via HTTPS from the host https://public-gold.skia.org but it responds with error 404 every time. I tried a range a thousand of commits, yet it still does not give results. The workaround employed was to recover the most recent `files.checksum` and `rendertests.txt` files from the git history and generate `unittests.txt` from `list_gpu_unit_tests` binary. `skqp` runs two lists of tests, `rendertests.txt` and `unittests.txt`. Both must be located inside the `skqp` assets folder. The first list uses GL and GLES to test rendering scenarios. The second runs some unit tests that do not render an image per se. In order to make the first `a630_skqp` to be green, the crashing tests were removed from the test lists and the expectations of the failing ones were updated. It is worth noting that `rendertests.txt` can bring some detail about each test expectation, so each test can have a max pixel error count, to tell `skqp` that it is OK to have at most that number of errors for that test. See also: https://github.com/google/skia/blob/main/tools/skqp/README_ALGORITHM.md As each render backend has a different error count, two different `rendertests.txt` files were created, `src/freedreno/ci/freedreno-a630-skqp-gl_rendertests.txt`, `src/freedreno/ci/freedreno-a630-skqp-gles_rendertests.txt` and , which one refers to GL and GLES tests respectfully. The unit tests file for a630 is located at `src/freedreno/ci/freedreno-a630-skqp_unittests.txt` ``` aaclip domain formats highcontrastfilter rectangle_texture yuv_make_color_space ``` ``` ProcessorOptimizationValidationTest VkProtectedContext_CreateNonprotectedContext VkYCbcrSampler_DrawImageWithYcbcrSampler VkYCbcrSampler_NoYcbcrSurface ``` Each test was updated with the max_error count equal to the first run result. ``` analytic_antialias_inverse async_rescale_and_read_dog_down async_rescale_and_read_dog_up async_rescale_and_read_rose async_rescale_and_read_text_down async_rescale_and_read_text_up async_rescale_and_read_text_up_large async_rescale_and_read_yuv420_rose complexclip2_path_bw encode-platform imageblur_large lcdtextsize onebadarc onefailarc scale-pixels surfaceprops textfilter_color textfilter_image ``` Considering all the following tests results as wrong. ``` async_rescale_and_read_no_bleed backdrop_imagefilter_croprect_persp complexclip2 imageblurrepeatmode mixerCF overdrawcolorfilter patch_alpha patch_primitive rrect_clip_bw scaledemoji_rendering yuv_splitter ``` v2: a) add link to HTML report on job log b) remove extraneous spaces diff c) remove unnecessary conditions from build-skqp.sh d) use fixed skqp source commit SHA v3: a) Use only main skia repository to fetch models and build skqp b) Use list_gpu_unit_tests binary to create a base unittests.txt file c) Remove crashing tests d) Set failing tests expectations for the first skqp run v4: a) Remove clang dependency b) Separate each skqp backend result into its folder c) Regroup a630_skqp in one job v5: a) Separate tests files per driver Closes: https://gitlab.freedesktop.org/mesa/mesa/-/issues/5580 Signed-off-by: Guilherme Gallo <guilherme.gallo@collabora.com> Reviewed-by: Emma Anholt <emma@anholt.net> Part-of: <https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14146>
86 lines
2.6 KiB
Bash
Executable File
86 lines
2.6 KiB
Bash
Executable File
#!/bin/sh
|
|
|
|
copy_tests_files() (
|
|
SKQP_BACKEND="${1}"
|
|
SKQP_FILE_PREFIX="${INSTALL}/${GPU_VERSION}-skqp"
|
|
|
|
if echo "${SKQP_BACKEND}" | grep -qE 'gl(es)?'
|
|
then
|
|
SKQP_RENDER_TESTS_FILE="${SKQP_FILE_PREFIX}-${SKQP_BACKEND}_rendertests.txt"
|
|
cp "${SKQP_RENDER_TESTS_FILE}" "${SKQP_ASSETS_DIR}"/skqp/rendertests.txt
|
|
return 0
|
|
fi
|
|
|
|
# The unittests.txt path is hardcoded inside assets directory,
|
|
# that is why it needs to be a special case.
|
|
if echo "${SKQP_BACKEND}" | grep -qE "unitTest"
|
|
then
|
|
cp "${SKQP_FILE_PREFIX}_unittests.txt" "${SKQP_ASSETS_DIR}"/skqp/unittests.txt
|
|
fi
|
|
)
|
|
|
|
set -ex
|
|
|
|
# Needed so configuration files can contain paths to files in /install
|
|
ln -sf "$CI_PROJECT_DIR"/install /install
|
|
|
|
INSTALL=${PWD}/install
|
|
|
|
if [ -z "$GPU_VERSION" ]; then
|
|
echo 'GPU_VERSION must be set to something like "llvmpipe" or "freedreno-a630" (the name used in .gitlab-ci/gpu-version-*.txt)'
|
|
exit 1
|
|
fi
|
|
|
|
SKQP_ASSETS_DIR=/skqp/assets
|
|
SKQP_RESULTS_DIR="${SKQP_RESULTS_DIR:-results}"
|
|
|
|
mkdir "${SKQP_ASSETS_DIR}"/skqp
|
|
|
|
SKQP_EXITCODE=0
|
|
for SKQP_BACKEND in ${SKQP_BACKENDS}
|
|
do
|
|
set -e
|
|
copy_tests_files "${SKQP_BACKEND}"
|
|
|
|
set +e
|
|
SKQP_BACKEND_RESULTS_DIR="${SKQP_RESULTS_DIR}"/"${SKQP_BACKEND}"
|
|
mkdir -p "${SKQP_BACKEND_RESULTS_DIR}"
|
|
/skqp/skqp "${SKQP_ASSETS_DIR}" '' "${SKQP_BACKEND_RESULTS_DIR}" "${SKQP_BACKEND}_"
|
|
BACKEND_EXITCODE=$?
|
|
|
|
if [ ! $BACKEND_EXITCODE -eq 0 ]
|
|
then
|
|
echo "skqp failed on ${SKQP_BACKEND} tests with ${BACKEND_EXITCODE} exit code."
|
|
fi
|
|
|
|
# Propagate error codes to leverage the final job result
|
|
SKQP_EXITCODE=$(( SKQP_EXITCODE | BACKEND_EXITCODE ))
|
|
done
|
|
|
|
set +x
|
|
|
|
# Unit tests produce empty HTML reports, guide the user to check the TXT file.
|
|
if echo "${SKQP_BACKENDS}" | grep -qE "unitTest"
|
|
then
|
|
# Remove the empty HTML report to avoid confusion
|
|
rm -f "${SKQP_RESULTS_DIR}"/unitTest/report.html
|
|
|
|
echo "See skqp unit test results at:"
|
|
echo "https://$CI_PROJECT_ROOT_NAMESPACE.pages.freedesktop.org/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/${SKQP_RESULTS_DIR}/unitTest/unit_tests.txt"
|
|
fi
|
|
|
|
for REPORT in "${SKQP_RESULTS_DIR}"/**/report.html
|
|
do
|
|
BACKEND_NAME=$(echo "${REPORT}" | sed 's@.*/\([^/]*\)/report.html@\1@')
|
|
echo "See skqp ${BACKEND_NAME} render tests report at:"
|
|
echo "https://$CI_PROJECT_ROOT_NAMESPACE.pages.freedesktop.org/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/${REPORT}"
|
|
done
|
|
|
|
# If there is no report available, tell the user that something is wrong.
|
|
if [ ! -f "${REPORT}" ]
|
|
then
|
|
echo "No skqp report available. Probably some fatal error has occured during the skqp execution."
|
|
fi
|
|
|
|
exit $SKQP_EXITCODE
|