| Message ID | 20200707070858.6622-29-alex.bennee@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | testing updates (vm, gitlab, misc build fixes) | expand |
On Tue, Jul 7, 2020 at 9:14 AM Alex Bennée <alex.bennee@linaro.org> wrote: > > As part of migrating things from Travis to GitLab add the acceptance > tests. To do this: > > - rename system1 to system-ubuntu-main > - rename system2 to system-fedora-misc > - split into build/check/acceptance > - remove -j from check stages > - use artifacts to save build stage > - add post acceptance template and use > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> > Message-Id: <20200701135652.1366-31-alex.bennee@linaro.org> > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index a7abc55a5c6a..93baf98232d7 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -1,8 +1,12 @@ > +# Currently we have two build stages after our containers are built: > +# - build (for traditional build and test or first stage build) > +# - test (for test stages, using build artefacts from a build stage) > stages: > - containers > - containers-layer2 > - containers-layer3 > - build > + - test > > include: > - local: '/.gitlab-ci.d/edk2.yml' > @@ -24,26 +28,85 @@ include: > ../configure --enable-werror $CONFIGURE_ARGS ; > fi > - make -j"$JOBS" > - - make -j"$JOBS" $MAKE_CHECK_ARGS > + - if test -n "$MAKE_CHECK_ARGS"; > + then > + make $MAKE_CHECK_ARGS ; > + fi > + > +.native_test_job_template: &native_test_job_definition > + stage: test > + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest > + script: > + - cd build > + - find . -type f -exec touch {} + > + - make $MAKE_CHECK_ARGS > + > +.post_acceptance_template: &post_acceptance > + after_script: > + - cd build > + - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat > + - du -chs $HOME/avocado/data/cache > > -build-system1: > +build-system-ubuntu-main: > <<: *native_build_job_definition > variables: > IMAGE: ubuntu2004 > TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-softmmu > moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc-softmmu > riscv64-softmmu sparc-softmmu > + artifacts: > + paths: > + - build > + > +check-system-ubuntu-main: > + <<: *native_test_job_definition > + needs: > + - job: build-system-ubuntu-main > + artifacts: true > + variables: > + IMAGE: ubuntu2004 > MAKE_CHECK_ARGS: check > > -build-system2: > +acceptance-system-ubuntu-main: > + <<: *native_test_job_definition > + needs: > + - job: build-system-ubuntu-main > + artifacts: true > + variables: > + IMAGE: ubuntu2004 > + MAKE_CHECK_ARGS: check-acceptance > + <<: *post_acceptance > + > +build-system-fedora-alt: > <<: *native_build_job_definition > variables: > IMAGE: fedora > TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-softmmu > riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-softmmu > xtensa-softmmu nios2-softmmu or1k-softmmu > + artifacts: > + paths: > + - build > + > +check-system-fedora-alt: > + <<: *native_test_job_definition > + needs: > + - job: build-system-fedora-alt > + artifacts: true > + variables: > + IMAGE: fedora > MAKE_CHECK_ARGS: check > > +acceptance-system-fedora-alt: > + <<: *native_test_job_definition > + needs: > + - job: build-system-fedora-alt > + artifacts: true > + variables: > + IMAGE: fedora > + MAKE_CHECK_ARGS: check-acceptance > + <<: *post_acceptance Apparently our acceptance image misses netcat: (22/45) tests/acceptance/migration.py:Migration.test_migration_with_exec: SKIP: 'nc' command not found
On 08/09/2020 21.54, Philippe Mathieu-Daudé wrote: > On Tue, Jul 7, 2020 at 9:14 AM Alex Bennée <alex.bennee@linaro.org> wrote: >> >> As part of migrating things from Travis to GitLab add the acceptance >> tests. [...] > > Apparently our acceptance image misses netcat: > (22/45) tests/acceptance/migration.py:Migration.test_migration_with_exec: > SKIP: 'nc' command not found > Yes, we need to add this to the dockerfiles. I thought I had already added it once to a couple of them, but apparently that patch got lost in the shuffle... Can you send a patch? Or shall I put it on my todo list again? Thomas
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index a7abc55a5c6a..93baf98232d7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -1,8 +1,12 @@ +# Currently we have two build stages after our containers are built: +# - build (for traditional build and test or first stage build) +# - test (for test stages, using build artefacts from a build stage) stages: - containers - containers-layer2 - containers-layer3 - build + - test include: - local: '/.gitlab-ci.d/edk2.yml' @@ -24,26 +28,85 @@ include: ../configure --enable-werror $CONFIGURE_ARGS ; fi - make -j"$JOBS" - - make -j"$JOBS" $MAKE_CHECK_ARGS + - if test -n "$MAKE_CHECK_ARGS"; + then + make $MAKE_CHECK_ARGS ; + fi + +.native_test_job_template: &native_test_job_definition + stage: test + image: $CI_REGISTRY_IMAGE/qemu/$IMAGE:latest + script: + - cd build + - find . -type f -exec touch {} + + - make $MAKE_CHECK_ARGS + +.post_acceptance_template: &post_acceptance + after_script: + - cd build + - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat + - du -chs $HOME/avocado/data/cache -build-system1: +build-system-ubuntu-main: <<: *native_build_job_definition variables: IMAGE: ubuntu2004 TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu lm32-softmmu moxie-softmmu microblazeel-softmmu mips64el-softmmu m68k-softmmu ppc-softmmu riscv64-softmmu sparc-softmmu + artifacts: + paths: + - build + +check-system-ubuntu-main: + <<: *native_test_job_definition + needs: + - job: build-system-ubuntu-main + artifacts: true + variables: + IMAGE: ubuntu2004 MAKE_CHECK_ARGS: check -build-system2: +acceptance-system-ubuntu-main: + <<: *native_test_job_definition + needs: + - job: build-system-ubuntu-main + artifacts: true + variables: + IMAGE: ubuntu2004 + MAKE_CHECK_ARGS: check-acceptance + <<: *post_acceptance + +build-system-fedora-alt: <<: *native_build_job_definition variables: IMAGE: fedora TARGETS: tricore-softmmu unicore32-softmmu microblaze-softmmu mips-softmmu riscv32-softmmu s390x-softmmu sh4-softmmu sparc64-softmmu x86_64-softmmu xtensa-softmmu nios2-softmmu or1k-softmmu + artifacts: + paths: + - build + +check-system-fedora-alt: + <<: *native_test_job_definition + needs: + - job: build-system-fedora-alt + artifacts: true + variables: + IMAGE: fedora MAKE_CHECK_ARGS: check +acceptance-system-fedora-alt: + <<: *native_test_job_definition + needs: + - job: build-system-fedora-alt + artifacts: true + variables: + IMAGE: fedora + MAKE_CHECK_ARGS: check-acceptance + <<: *post_acceptance + build-disabled: <<: *native_build_job_definition variables: diff --git a/.travis.yml b/.travis.yml index 74158f741b18..c24dfbe377f3 100644 --- a/.travis.yml +++ b/.travis.yml @@ -289,29 +289,6 @@ jobs: python: 3.6 - # Acceptance (Functional) tests - - name: "GCC check-acceptance" - dist: bionic - env: - - CONFIG="--enable-tools --target-list=aarch64-softmmu,alpha-softmmu,arm-softmmu,m68k-softmmu,microblaze-softmmu,mips-softmmu,mips64el-softmmu,nios2-softmmu,or1k-softmmu,ppc-softmmu,ppc64-softmmu,s390x-softmmu,sh4-softmmu,sparc-softmmu,x86_64-softmmu,xtensa-softmmu" - - TEST_CMD="make check-acceptance" - - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-acceptance" - after_script: - - python3 -c 'import json; r = json.load(open("tests/results/latest/results.json")); [print(t["logfile"]) for t in r["tests"] if t["status"] not in ("PASS", "SKIP")]' | xargs cat - - du -chs $HOME/avocado/data/cache - addons: - apt: - packages: - - python3-pil - - python3-pip - - python3-numpy - - python3-opencv - - python3-venv - - rpm2cpio - - tesseract-ocr - - tesseract-ocr-eng - - # Using newer GCC with sanitizers - name: "GCC9 with sanitizers (softmmu)" addons: diff --git a/tests/Makefile.include b/tests/Makefile.include index 3f4448a20bc7..c316e0d66475 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -944,7 +944,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images --show=$(AVOCADO_SHOW) run --job-results-dir=$(TESTS_RESULTS_DIR) \ --filter-by-tags-include-empty --filter-by-tags-include-empty-key \ $(AVOCADO_TAGS) \ - --failfast=on tests/acceptance, \ + $(if $(GITLAB_CI),,--failfast=on) tests/acceptance, \ "AVOCADO", "tests/acceptance") # Consolidated targets diff --git a/tests/acceptance/replay_kernel.py b/tests/acceptance/replay_kernel.py index 60621417ddf0..62d2db8c64be 100644 --- a/tests/acceptance/replay_kernel.py +++ b/tests/acceptance/replay_kernel.py @@ -73,7 +73,7 @@ class ReplayKernel(LinuxKernelTest): logger = logging.getLogger('replay') logger.info('replay overhead {:.2%}'.format(t2 / t1 - 1)) - @skipIf(os.getenv('CONTINUOUS_INTEGRATION'), 'Running on Travis-CI') + @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab') def test_x86_64_pc(self): """ :avocado: tags=arch:x86_64