gitlab-ci: Make the test job fail when bugs are unexpectedly fixed.
If people fix bugs without updating the expected-fails list, then we end up with a lack of coverage of those failures in the future. Also, some day down the line another developer ends up trying to figure out if the bug was actually fixed or their environment is just failing to reproduce it. Suggested-by: Kristian H. Kristensen <hoegsberg@google.com> Reviewed-by: Adam Jackson <ajax@redhat.com> Acked-by: Rob Clark <robdclark@chromium.org> Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com>
This commit is contained in:
@@ -123,3 +123,14 @@ if [ -s /tmp/new-fails.txt ]; then
|
||||
else
|
||||
echo "No new failures"
|
||||
fi
|
||||
|
||||
sort /tmp/case-list.txt > /tmp/sorted-case-list.txt
|
||||
comm -12 /tmp/sorted-case-list.txt /tmp/expected-fails.txt > /tmp/expected-fails-in-caselist.txt
|
||||
comm -13 $RESULTS/fails.txt /tmp/expected-fails-in-caselist.txt > /tmp/new-passes.txt
|
||||
if [ -s /tmp/new-passes.txt ]; then
|
||||
echo "Unexpected passes, please update $DEQP_EXPECTED_FAILS (or add flaky tests to $DEQP_SKIPS):"
|
||||
cat /tmp/new-passes.txt
|
||||
exit 1
|
||||
else
|
||||
echo "No new passes"
|
||||
fi
|
||||
|
Reference in New Issue
Block a user