[PULL,00/41] testing updates (vm, gitlab, misc build fixes)

Message ID 20200707070858.6622-1-alex.bennee@linaro.org
Headers show
Series
  • testing updates (vm, gitlab, misc build fixes)
Related show

Message

Alex Bennée July 7, 2020, 7:08 a.m.
There will be some docker failures until the official repository has
seeded but local builds should continue to work.

----

The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:

  Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 16:08:41 +0100)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-testing-and-misc-070720-1

for you to fetch changes up to 6a726e8ca0286e3ed69945abd447099f6f6a903c:

  tests/qht-bench: Adjust threshold computation (2020-07-07 07:57:41 +0100)

----------------------------------------------------------------
Testing and build updates:

  - tests/vm support for aarch64 VMs
  - tests/tcg better cross-compiler detection
  - update docker tooling to support registries
  - gitlab build docker images and store in registry
  - gitlab use docker images for builds
  - a number of skipIf updates to support move
  - linux-user MAP_FIXED_NOREPLACE fix
  - qht-bench compiler tweaks
  - configure fix for secret keyring
  - tsan fiber annotation clean-up

----------------------------------------------------------------
Alex Bennée (23):
      tests/vm: switch from optsparse to argparse
      tests/vm: allow us to take advantage of MTTCG
      tests/docker: check for an parameters not empty string
      tests/docker: change tag naming scheme of our images
      .gitignore: un-ignore .gitlab-ci.d
      gitlab: build containers with buildkit and metadata
      tests/docker: add --registry support to tooling
      tests/docker: add packages needed for check-acceptance
      tests/acceptance: skip s390x_ccw_vrtio_tcg on GitLab
      tests/acceptance: fix dtb path for machine_rx_gdbsim
      tests/acceptance: skip multicore mips_malta tests on GitLab
      tests/acceptance: skip LinuxInitrd 2gib with v4.16 on GitLab
      gitlab: add acceptance testing to system builds
      tests/tcg: add more default compilers to configure.sh
      tests/docker: add a linux-user testing focused image
      linux-user/elfload: use MAP_FIXED_NOREPLACE in pgb_reserved_va
      gitlab: enable check-tcg for linux-user tests
      gitlab: add avocado asset caching
      gitlab: split build-disabled into two phases
      gitlab: limit re-builds of the containers
      containers.yml: build with docker.py tooling
      testing: add check-build target
      shippable: pull images from registry instead of building

Daniel P. Berrangé (3):
      gitlab: introduce explicit "container" and "build" stages
      gitlab: build all container images during CI
      gitlab: convert jobs to use custom built containers

David Edmondson (1):
      crypto/linux_keyring: fix 'secret_keyring' configure test

Richard Henderson (2):
      tests/qht-bench: Adjust testing rate by -1
      tests/qht-bench: Adjust threshold computation

Robert Foley (10):
      util/coroutine: Cleanup start_switch_fiber_ for TSAN.
      tests/vm: pass args through to BaseVM's __init__
      tests/vm: Add configuration to basevm.py
      tests/vm: Added configuration file support
      tests/vm: Add common Ubuntu python module
      tests/vm: Added a new script for ubuntu.aarch64.
      tests/vm: Added a new script for centos.aarch64.
      tests/vm: change scripts to use self._config
      python/qemu: Add ConsoleSocket for optional use in QEMUMachine
      tests/vm: Add workaround to consume console

Thomas Huth (2):
      gitlab-ci: Fix the change rules after moving the YML files
      travis.yml: Test also the other targets on s390x

 configure                                          |  31 +-
 linux-user/elfload.c                               |  10 +-
 tests/qht-bench.c                                  |  40 ++-
 util/coroutine-ucontext.c                          |  52 ++--
 .gitignore                                         |   1 +
 .gitlab-ci.d/containers.yml                        | 263 ++++++++++++++++
 .gitlab-ci.d/edk2.yml                              |   5 +-
 .gitlab-ci.d/opensbi.yml                           |   5 +-
 .gitlab-ci.yml                                     | 281 ++++++++++-------
 .shippable.yml                                     |   8 +-
 .travis.yml                                        |  62 ++--
 python/qemu/console_socket.py                      | 110 +++++++
 python/qemu/machine.py                             |  23 +-
 tests/Makefile.include                             |  19 +-
 tests/acceptance/boot_linux.py                     |   2 +
 tests/acceptance/linux_initrd.py                   |   3 +
 tests/acceptance/machine_mips_malta.py             |   3 +
 tests/acceptance/machine_rx_gdbsim.py              |   2 +-
 tests/acceptance/replay_kernel.py                  |   2 +-
 tests/docker/Makefile.include                      |  17 +-
 tests/docker/common.rc                             |   2 +-
 tests/docker/docker.py                             |  46 ++-
 .../dockerfiles/debian-all-test-cross.docker       |  53 ++++
 tests/docker/dockerfiles/debian-alpha-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-amd64-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-amd64.docker       |   2 +-
 tests/docker/dockerfiles/debian-arm64-cross.docker |   2 +-
 .../dockerfiles/debian-arm64-test-cross.docker     |   2 +-
 tests/docker/dockerfiles/debian-armel-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-armhf-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-hppa-cross.docker  |   2 +-
 tests/docker/dockerfiles/debian-m68k-cross.docker  |   2 +-
 tests/docker/dockerfiles/debian-mips-cross.docker  |   2 +-
 .../docker/dockerfiles/debian-mips64-cross.docker  |   2 +-
 .../dockerfiles/debian-mips64el-cross.docker       |   2 +-
 .../docker/dockerfiles/debian-mipsel-cross.docker  |   2 +-
 .../docker/dockerfiles/debian-powerpc-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-ppc64-cross.docker |   2 +-
 .../docker/dockerfiles/debian-ppc64el-cross.docker |   2 +-
 .../docker/dockerfiles/debian-riscv64-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-s390x-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-sh4-cross.docker   |   2 +-
 .../docker/dockerfiles/debian-sparc64-cross.docker |   2 +-
 .../docker/dockerfiles/debian-tricore-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-win32-cross.docker |   2 +-
 tests/docker/dockerfiles/debian-win64-cross.docker |   2 +-
 tests/docker/dockerfiles/debian9-mxe.docker        |   2 +-
 tests/docker/dockerfiles/fedora.docker             |   7 +
 tests/docker/dockerfiles/ubuntu2004.docker         |  10 +-
 tests/tcg/Makefile.qemu                            |   4 +-
 tests/tcg/configure.sh                             |  19 +-
 tests/vm/Makefile.include                          |  22 ++
 tests/vm/aarch64vm.py                              | 106 +++++++
 tests/vm/basevm.py                                 | 344 +++++++++++++++------
 tests/vm/centos-8-aarch64.ks                       |  51 +++
 tests/vm/centos.aarch64                            | 227 ++++++++++++++
 tests/vm/conf_example_aarch64.yml                  |  51 +++
 tests/vm/conf_example_x86.yml                      |  50 +++
 tests/vm/fedora                                    |  17 +-
 tests/vm/freebsd                                   |  16 +-
 tests/vm/netbsd                                    |  19 +-
 tests/vm/openbsd                                   |  17 +-
 tests/vm/ubuntu.aarch64                            |  68 ++++
 tests/vm/ubuntu.i386                               |  46 +--
 tests/vm/ubuntuvm.py                               |  60 ++++
 65 files changed, 1839 insertions(+), 383 deletions(-)
 create mode 100644 .gitlab-ci.d/containers.yml
 create mode 100644 python/qemu/console_socket.py
 create mode 100644 tests/docker/dockerfiles/debian-all-test-cross.docker
 create mode 100644 tests/vm/aarch64vm.py
 create mode 100644 tests/vm/centos-8-aarch64.ks
 create mode 100755 tests/vm/centos.aarch64
 create mode 100644 tests/vm/conf_example_aarch64.yml
 create mode 100644 tests/vm/conf_example_x86.yml
 create mode 100755 tests/vm/ubuntu.aarch64
 create mode 100644 tests/vm/ubuntuvm.py

-- 
2.20.1

Comments

Alex Bennée July 7, 2020, 9:39 a.m. | #1
Alex Bennée <alex.bennee@linaro.org> writes:

> There will be some docker failures until the official repository has

> seeded but local builds should continue to work.


You can force the docker builds to use my registry for example by setting:

  make docker-test-build REGISTRY=registry.gitlab.com/stsquad/qemu
>

> ----

>

> The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:

>

>   Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 16:08:41 +0100)

>

> are available in the Git repository at:

>

>   https://github.com/stsquad/qemu.git tags/pull-testing-and-misc-070720-1

>

> for you to fetch changes up to 6a726e8ca0286e3ed69945abd447099f6f6a903c:

>

>   tests/qht-bench: Adjust threshold computation (2020-07-07 07:57:41 +0100)

>

> ----------------------------------------------------------------

> Testing and build updates:

>

>   - tests/vm support for aarch64 VMs

>   - tests/tcg better cross-compiler detection

>   - update docker tooling to support registries

>   - gitlab build docker images and store in registry

>   - gitlab use docker images for builds

>   - a number of skipIf updates to support move

>   - linux-user MAP_FIXED_NOREPLACE fix

>   - qht-bench compiler tweaks

>   - configure fix for secret keyring

>   - tsan fiber annotation clean-up

>

> ----------------------------------------------------------------

> Alex Bennée (23):

>       tests/vm: switch from optsparse to argparse

>       tests/vm: allow us to take advantage of MTTCG

>       tests/docker: check for an parameters not empty string

>       tests/docker: change tag naming scheme of our images

>       .gitignore: un-ignore .gitlab-ci.d

>       gitlab: build containers with buildkit and metadata

>       tests/docker: add --registry support to tooling

>       tests/docker: add packages needed for check-acceptance

>       tests/acceptance: skip s390x_ccw_vrtio_tcg on GitLab

>       tests/acceptance: fix dtb path for machine_rx_gdbsim

>       tests/acceptance: skip multicore mips_malta tests on GitLab

>       tests/acceptance: skip LinuxInitrd 2gib with v4.16 on GitLab

>       gitlab: add acceptance testing to system builds

>       tests/tcg: add more default compilers to configure.sh

>       tests/docker: add a linux-user testing focused image

>       linux-user/elfload: use MAP_FIXED_NOREPLACE in pgb_reserved_va

>       gitlab: enable check-tcg for linux-user tests

>       gitlab: add avocado asset caching

>       gitlab: split build-disabled into two phases

>       gitlab: limit re-builds of the containers

>       containers.yml: build with docker.py tooling

>       testing: add check-build target

>       shippable: pull images from registry instead of building

>

> Daniel P. Berrangé (3):

>       gitlab: introduce explicit "container" and "build" stages

>       gitlab: build all container images during CI

>       gitlab: convert jobs to use custom built containers

>

> David Edmondson (1):

>       crypto/linux_keyring: fix 'secret_keyring' configure test

>

> Richard Henderson (2):

>       tests/qht-bench: Adjust testing rate by -1

>       tests/qht-bench: Adjust threshold computation

>

> Robert Foley (10):

>       util/coroutine: Cleanup start_switch_fiber_ for TSAN.

>       tests/vm: pass args through to BaseVM's __init__

>       tests/vm: Add configuration to basevm.py

>       tests/vm: Added configuration file support

>       tests/vm: Add common Ubuntu python module

>       tests/vm: Added a new script for ubuntu.aarch64.

>       tests/vm: Added a new script for centos.aarch64.

>       tests/vm: change scripts to use self._config

>       python/qemu: Add ConsoleSocket for optional use in QEMUMachine

>       tests/vm: Add workaround to consume console

>

> Thomas Huth (2):

>       gitlab-ci: Fix the change rules after moving the YML files

>       travis.yml: Test also the other targets on s390x

>

>  configure                                          |  31 +-

>  linux-user/elfload.c                               |  10 +-

>  tests/qht-bench.c                                  |  40 ++-

>  util/coroutine-ucontext.c                          |  52 ++--

>  .gitignore                                         |   1 +

>  .gitlab-ci.d/containers.yml                        | 263 ++++++++++++++++

>  .gitlab-ci.d/edk2.yml                              |   5 +-

>  .gitlab-ci.d/opensbi.yml                           |   5 +-

>  .gitlab-ci.yml                                     | 281 ++++++++++-------

>  .shippable.yml                                     |   8 +-

>  .travis.yml                                        |  62 ++--

>  python/qemu/console_socket.py                      | 110 +++++++

>  python/qemu/machine.py                             |  23 +-

>  tests/Makefile.include                             |  19 +-

>  tests/acceptance/boot_linux.py                     |   2 +

>  tests/acceptance/linux_initrd.py                   |   3 +

>  tests/acceptance/machine_mips_malta.py             |   3 +

>  tests/acceptance/machine_rx_gdbsim.py              |   2 +-

>  tests/acceptance/replay_kernel.py                  |   2 +-

>  tests/docker/Makefile.include                      |  17 +-

>  tests/docker/common.rc                             |   2 +-

>  tests/docker/docker.py                             |  46 ++-

>  .../dockerfiles/debian-all-test-cross.docker       |  53 ++++

>  tests/docker/dockerfiles/debian-alpha-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-amd64-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-amd64.docker       |   2 +-

>  tests/docker/dockerfiles/debian-arm64-cross.docker |   2 +-

>  .../dockerfiles/debian-arm64-test-cross.docker     |   2 +-

>  tests/docker/dockerfiles/debian-armel-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-armhf-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-hppa-cross.docker  |   2 +-

>  tests/docker/dockerfiles/debian-m68k-cross.docker  |   2 +-

>  tests/docker/dockerfiles/debian-mips-cross.docker  |   2 +-

>  .../docker/dockerfiles/debian-mips64-cross.docker  |   2 +-

>  .../dockerfiles/debian-mips64el-cross.docker       |   2 +-

>  .../docker/dockerfiles/debian-mipsel-cross.docker  |   2 +-

>  .../docker/dockerfiles/debian-powerpc-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-ppc64-cross.docker |   2 +-

>  .../docker/dockerfiles/debian-ppc64el-cross.docker |   2 +-

>  .../docker/dockerfiles/debian-riscv64-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-s390x-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-sh4-cross.docker   |   2 +-

>  .../docker/dockerfiles/debian-sparc64-cross.docker |   2 +-

>  .../docker/dockerfiles/debian-tricore-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-win32-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian-win64-cross.docker |   2 +-

>  tests/docker/dockerfiles/debian9-mxe.docker        |   2 +-

>  tests/docker/dockerfiles/fedora.docker             |   7 +

>  tests/docker/dockerfiles/ubuntu2004.docker         |  10 +-

>  tests/tcg/Makefile.qemu                            |   4 +-

>  tests/tcg/configure.sh                             |  19 +-

>  tests/vm/Makefile.include                          |  22 ++

>  tests/vm/aarch64vm.py                              | 106 +++++++

>  tests/vm/basevm.py                                 | 344 +++++++++++++++------

>  tests/vm/centos-8-aarch64.ks                       |  51 +++

>  tests/vm/centos.aarch64                            | 227 ++++++++++++++

>  tests/vm/conf_example_aarch64.yml                  |  51 +++

>  tests/vm/conf_example_x86.yml                      |  50 +++

>  tests/vm/fedora                                    |  17 +-

>  tests/vm/freebsd                                   |  16 +-

>  tests/vm/netbsd                                    |  19 +-

>  tests/vm/openbsd                                   |  17 +-

>  tests/vm/ubuntu.aarch64                            |  68 ++++

>  tests/vm/ubuntu.i386                               |  46 +--

>  tests/vm/ubuntuvm.py                               |  60 ++++

>  65 files changed, 1839 insertions(+), 383 deletions(-)

>  create mode 100644 .gitlab-ci.d/containers.yml

>  create mode 100644 python/qemu/console_socket.py

>  create mode 100644 tests/docker/dockerfiles/debian-all-test-cross.docker

>  create mode 100644 tests/vm/aarch64vm.py

>  create mode 100644 tests/vm/centos-8-aarch64.ks

>  create mode 100755 tests/vm/centos.aarch64

>  create mode 100644 tests/vm/conf_example_aarch64.yml

>  create mode 100644 tests/vm/conf_example_x86.yml

>  create mode 100755 tests/vm/ubuntu.aarch64

>  create mode 100644 tests/vm/ubuntuvm.py



-- 
Alex Bennée
Peter Maydell July 9, 2020, 11:31 a.m. | #2
On Tue, 7 Jul 2020 at 08:09, Alex Bennée <alex.bennee@linaro.org> wrote:
>

> There will be some docker failures until the official repository has

> seeded but local builds should continue to work.

>

> ----

>

> The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:

>

>   Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 16:08:41 +0100)

>

> are available in the Git repository at:

>

>   https://github.com/stsquad/qemu.git tags/pull-testing-and-misc-070720-1

>

> for you to fetch changes up to 6a726e8ca0286e3ed69945abd447099f6f6a903c:

>

>   tests/qht-bench: Adjust threshold computation (2020-07-07 07:57:41 +0100)

>

> ----------------------------------------------------------------

> Testing and build updates:

>

>   - tests/vm support for aarch64 VMs

>   - tests/tcg better cross-compiler detection

>   - update docker tooling to support registries

>   - gitlab build docker images and store in registry

>   - gitlab use docker images for builds

>   - a number of skipIf updates to support move

>   - linux-user MAP_FIXED_NOREPLACE fix

>   - qht-bench compiler tweaks

>   - configure fix for secret keyring

>   - tsan fiber annotation clean-up


freebsd failed:

perl: warning: Please check that your locale settings:
perl: warning: Falling back to the standard locale ("C").
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
con recv: Loading /boot/defaults/loader.conf
con recv: Loading /boot/device.hints
con recv: Loading /boot/loader.conf
con recv: Loading /boot/loader.conf.local
con recv: \/  ```                        ` s` `.....---.......--.```
-/ +o   .--`         /y:`      +.  yo`:.            :o      `+-   y/
            -/`   -o/  .-                  ::/sy+:.  /
    `--  / `:                          :` `:
:`  /                          /  .-                        -.   --
                  -.    `:`                  `:`      .--
`--.         .---.....----.  ______               ____   _____ _____
|  ____|             |  _ \ / ____|  __ \  | |___ _ __ ___  ___ | |_)
| (___ | |  | | |  ___| '__/ _ \/ _ \|  _ < \___ \| |  | | | |   | | |
 __/  __/| |_) |____) | |__| | | |   | | |    |    ||     |      |
 | |_|   |_|  \___|\___||____/|_____/|_____/
..........................................................................................................................................................................................................................................................................................................................................Welcome
to FreeBSD1. Boot Multi user [Enter]2. Boot Single user3. Escape to
loader prompt4. RebootOptions:/\/\5. Kernel: default/kernel (1 of 1)6.
Boot OptionsAutoboot
con send: 3
con recv:  in 10 seconds, hit [Enter] to boot or any other key to stop
con recv:
con recv: Exiting menu!
con recv: Type '?' for a list of commands, 'help' for more detailed help.
con recv: OK
con send: set console=comconsole<enter>
console: *** read timeout ***
console: waiting for: 'OK'
console: line buffer:

con recv:  set console=comconso

Failed to prepare guest environment
Traceback (most recent call last):
  File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line 628, in main
    return vm.build_image(args.image)
  File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 163,
in build_image
    self.console_boot_serial()
  File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 76,
in console_boot_serial
    self.console_wait_send("OK", "boot\n")
  File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
400, in console_wait_send
    self.console_wait(wait)
  File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
340, in console_wait
    chars = vm.console_socket.recv(1)
  File "/home/peter.maydell/qemu-freebsd/tests/vm/../../python/qemu/console_socket.py",
line 96, in recv
    raise socket.timeout
socket.timeout


NetBSD failed:
con recv: postfix: rebuilding /etc/mail/aliases (missing /etc/mail/aliases.db)
con recv: Starting inetd.
con recv: Starting cron.
con recv: Thu Jul  9 10:40:07 UTC 2020
con recv: NetBSD/amd64 (localhost) (constty)
con recv: login:
con send: qemu<enter>
con recv:  Jul  9 10:40:09 localhost getty[756]: /dev/ttyE2: Device
not configured
con recv: Jul  9 10:40:09 localhost getty[703]: /dev/ttyE3: Device not
configured
con recv: Jul  9 10:40:09 localhost getty[753]: /dev/ttyE1: Device not
configured
con recv: qemu
con recv: Password:
con send: qemupass<enter>
con recv: Login incorrect or refused on this terminal.
console: *** read timeout ***
console: waiting for: 'localhost$'
console: line buffer:

con recv: login:

Failed to prepare guest environment
Traceback (most recent call last):
  File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 628, in main
    return vm.build_image(args.image)
  File "/home/peter.maydell/qemu-netbsd/tests/vm/netbsd", line 174, in
build_image
    self._config["guest_pass"])
  File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 408,
in console_ssh_init
    self.console_wait_send(prompt,      "mkdir .ssh\n")
  File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 400,
in console_wait_send
    self.console_wait(wait)
  File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 340,
in console_wait
    chars = vm.console_socket.recv(1)
  File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/console_socket.py",
line 96, in recv
    raise socket.timeout
socket.timeout

arm-linux-static check-tcg failed:
  TEST    pauth-4 on aarch64
  TEST    semihosting on aarch64
  SKIPPED semiconsole on aarch64 because MANUAL ONLY
  TEST    basic gdbstub support
/tmp/tmp6ftb0v61qemu-gdbstub/gdbstub.socket: No such device or address.
SKIPPING (not connected)
  BUILD   TCG tests for aarch64_be-linux-user
  BUILD   aarch64_be-linux-user guest-tests SKIPPED
  RUN     TCG tests for aarch64_be-linux-user
  RUN     tests for aarch64_be-linux-user SKIPPED
make: Entering directory
'/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'
  BUILD   debian10
Error response from daemon: manifest for
registry.gitlab.com/qemu-project/qemu/qemu/debian10:latest not found
make: Leaving directory
'/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'
make: Entering directory
'/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'
  BUILD   debian-alpha-cross
Error response from daemon: manifest for
registry.gitlab.com/qemu-project/qemu/qemu/debian-alpha-cross:latest
not found
registry.gitlab.com/qemu-project/qemu/qemu/debian10:latest not found
Traceback (most recent call last):
  File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
line 701, in <module>
    sys.exit(main())
  File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
line 697, in main
    return args.cmdobj.run(args, argv)
  File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
line 494, in run
    extra_files_cksum=cksum)
  File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
line 347, in build_image
    quiet=quiet)
  File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",
line 244, in _do_check
    return subprocess.check_call(self._command + cmd, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'build', '-t',
'qemu/debian-alpha-cross', '-f',
'/tmp/docker_buildsstz1jcy/tmp_7c70xjh.docker', '--build-arg',
'BUILDKIT_INLINE_CACHE=1', '--cache-from',
'registry.gitlab.com/qemu-project/qemu/qemu/debian-alpha-cross',
'/tmp/docker_buildsstz1jcy']' returned non-zero exit status 1.
/home/petmay01/linaro/qemu-for-merges/tests/docker/Makefile.include:58:
recipe for target 'docker-image-debian-alpha-cross' failed
make: *** [docker-image-debian-alpha-cross] Error 1


Also a compile failure on s390x, but since this isn't related
to changes you made afaict I wonder if it's the result of
a change in the build environment:
/home/ubuntu/qemu/block/ssh.c: In function ‘check_host_key_knownhosts’:
/home/ubuntu/qemu/block/ssh.c:281:28: error: storage size of ‘state’ isn’t known
     enum ssh_known_hosts_e state;
                            ^~~~~
/home/ubuntu/qemu/block/ssh.c:289:13: error: implicit declaration of
function ‘ssh_session_is_known_server’ [-Werror=implicit-funct
ion-declaration]
     state = ssh_session_is_known_server(s->session);
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~
[and other errors]


thanks
-- PMM
Philippe Mathieu-Daudé July 9, 2020, 12:24 p.m. | #3
On 7/9/20 1:31 PM, Peter Maydell wrote:
> On Tue, 7 Jul 2020 at 08:09, Alex Bennée <alex.bennee@linaro.org> wrote:

>>

>> There will be some docker failures until the official repository has

>> seeded but local builds should continue to work.

>>

>> ----

>>

>> The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:

>>

>>   Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 16:08:41 +0100)

>>

>> are available in the Git repository at:

>>

>>   https://github.com/stsquad/qemu.git tags/pull-testing-and-misc-070720-1

>>

>> for you to fetch changes up to 6a726e8ca0286e3ed69945abd447099f6f6a903c:

>>

>>   tests/qht-bench: Adjust threshold computation (2020-07-07 07:57:41 +0100)

>>

>> ----------------------------------------------------------------

>> Testing and build updates:

>>

>>   - tests/vm support for aarch64 VMs

>>   - tests/tcg better cross-compiler detection

>>   - update docker tooling to support registries

>>   - gitlab build docker images and store in registry

>>   - gitlab use docker images for builds

>>   - a number of skipIf updates to support move

>>   - linux-user MAP_FIXED_NOREPLACE fix

>>   - qht-bench compiler tweaks

>>   - configure fix for secret keyring

>>   - tsan fiber annotation clean-up

[...]
> Also a compile failure on s390x, but since this isn't related

> to changes you made afaict I wonder if it's the result of

> a change in the build environment:

> /home/ubuntu/qemu/block/ssh.c: In function ‘check_host_key_knownhosts’:

> /home/ubuntu/qemu/block/ssh.c:281:28: error: storage size of ‘state’ isn’t known

>      enum ssh_known_hosts_e state;

>                             ^~~~~

> /home/ubuntu/qemu/block/ssh.c:289:13: error: implicit declaration of

> function ‘ssh_session_is_known_server’ [-Werror=implicit-funct

> ion-declaration]

>      state = ssh_session_is_known_server(s->session);

>              ^~~~~~~~~~~~~~~~~~~~~~~~~~~

> [and other errors]


libssh is bugged on Ubuntu 18.04.
https://bugs.launchpad.net/qemu/+bug/1838763

We need to use 'configure --disable-libssh' there.
Peter Maydell July 9, 2020, 1:04 p.m. | #4
On Thu, 9 Jul 2020 at 13:24, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
> libssh is bugged on Ubuntu 18.04.

> https://bugs.launchpad.net/qemu/+bug/1838763

>

> We need to use 'configure --disable-libssh' there.


Ah, thanks. I guess libssh recently got installed on that
box.

-- PMM
Alex Bennée July 9, 2020, 3:46 p.m. | #5
Peter Maydell <peter.maydell@linaro.org> writes:

> On Tue, 7 Jul 2020 at 08:09, Alex Bennée <alex.bennee@linaro.org> wrote:

>>

>> There will be some docker failures until the official repository has

>> seeded but local builds should continue to work.

>>

>> ----

>>

>> The following changes since commit eb6490f544388dd24c0d054a96dd304bc7284450:

>>

>>   Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200703' into staging (2020-07-04 16:08:41 +0100)

>>

>> are available in the Git repository at:

>>

>>   https://github.com/stsquad/qemu.git tags/pull-testing-and-misc-070720-1

>>

>> for you to fetch changes up to 6a726e8ca0286e3ed69945abd447099f6f6a903c:

>>

>>   tests/qht-bench: Adjust threshold computation (2020-07-07 07:57:41 +0100)

>>

>> ----------------------------------------------------------------

>> Testing and build updates:

>>

>>   - tests/vm support for aarch64 VMs

>>   - tests/tcg better cross-compiler detection

>>   - update docker tooling to support registries

>>   - gitlab build docker images and store in registry

>>   - gitlab use docker images for builds

>>   - a number of skipIf updates to support move

>>   - linux-user MAP_FIXED_NOREPLACE fix

>>   - qht-bench compiler tweaks

>>   - configure fix for secret keyring

>>   - tsan fiber annotation clean-up

>

> freebsd failed:

>

<snip>
>

> con recv:  set console=comconso

>

> Failed to prepare guest environment

> Traceback (most recent call last):

>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line 628, in main

>     return vm.build_image(args.image)

>   File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 163,

> in build_image

>     self.console_boot_serial()

>   File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 76,

> in console_boot_serial

>     self.console_wait_send("OK", "boot\n")

>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line

> 400, in console_wait_send

>     self.console_wait(wait)

>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line

> 340, in console_wait

>     chars = vm.console_socket.recv(1)

>   File "/home/peter.maydell/qemu-freebsd/tests/vm/../../python/qemu/console_socket.py",

> line 96, in recv

>     raise socket.timeout

> socket.timeout


Hmm works for me, but I'm guessing you have V=1?

> NetBSD failed:

> con recv: postfix: rebuilding /etc/mail/aliases (missing /etc/mail/aliases.db)

> con recv: Starting inetd.

> con recv: Starting cron.

> con recv: Thu Jul  9 10:40:07 UTC 2020

> con recv: NetBSD/amd64 (localhost) (constty)

> con recv: login:

> con send: qemu<enter>

> con recv:  Jul  9 10:40:09 localhost getty[756]: /dev/ttyE2: Device

> not configured

> con recv: Jul  9 10:40:09 localhost getty[703]: /dev/ttyE3: Device not

> configured

> con recv: Jul  9 10:40:09 localhost getty[753]: /dev/ttyE1: Device not

> configured

> con recv: qemu

> con recv: Password:

> con send: qemupass<enter>

> con recv: Login incorrect or refused on this terminal.

> console: *** read timeout ***

> console: waiting for: 'localhost$'

> console: line buffer:

>

> con recv: login:

>

> Failed to prepare guest environment

> Traceback (most recent call last):

>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 628, in main

>     return vm.build_image(args.image)

>   File "/home/peter.maydell/qemu-netbsd/tests/vm/netbsd", line 174, in

> build_image

>     self._config["guest_pass"])

>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 408,

> in console_ssh_init

>     self.console_wait_send(prompt,      "mkdir .ssh\n")

>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 400,

> in console_wait_send

>     self.console_wait(wait)

>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 340,

> in console_wait

>     chars = vm.console_socket.recv(1)

>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/console_socket.py",

> line 96, in recv

>     raise socket.timeout

> socket.timeout


OK that I can replicate. So it looks like our console bypass has an
issue. I'll dig into it.

> arm-linux-static check-tcg failed:

>   TEST    pauth-4 on aarch64

>   TEST    semihosting on aarch64

>   SKIPPED semiconsole on aarch64 because MANUAL ONLY

>   TEST    basic gdbstub support

> /tmp/tmp6ftb0v61qemu-gdbstub/gdbstub.socket: No such device or address.

> SKIPPING (not connected)

>   BUILD   TCG tests for aarch64_be-linux-user

>   BUILD   aarch64_be-linux-user guest-tests SKIPPED

>   RUN     TCG tests for aarch64_be-linux-user

>   RUN     tests for aarch64_be-linux-user SKIPPED

> make: Entering directory

> '/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'

>   BUILD   debian10

> Error response from daemon: manifest for

> registry.gitlab.com/qemu-project/qemu/qemu/debian10:latest not found

> make: Leaving directory

> '/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'

> make: Entering directory

> '/home/petmay01/linaro/qemu-for-merges/build/all-linux-static'

>   BUILD   debian-alpha-cross

> Error response from daemon: manifest for

> registry.gitlab.com/qemu-project/qemu/qemu/debian-alpha-cross:latest

> not found

> registry.gitlab.com/qemu-project/qemu/qemu/debian10:latest not found

> Traceback (most recent call last):

>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",

> line 701, in <module>

>     sys.exit(main())

>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",

> line 697, in main

>     return args.cmdobj.run(args, argv)

>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",

> line 494, in run

>     extra_files_cksum=cksum)

>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",

> line 347, in build_image

>     quiet=quiet)

>   File "/home/petmay01/linaro/qemu-for-merges/tests/docker/docker.py",

> line 244, in _do_check

>     return subprocess.check_call(self._command + cmd, **kwargs)

>   File "/usr/lib/python3.6/subprocess.py", line 311, in check_call

>     raise CalledProcessError(retcode, cmd)

> subprocess.CalledProcessError: Command '['docker', 'build', '-t',

> 'qemu/debian-alpha-cross', '-f',

> '/tmp/docker_buildsstz1jcy/tmp_7c70xjh.docker', '--build-arg',

> 'BUILDKIT_INLINE_CACHE=1', '--cache-from',

> 'registry.gitlab.com/qemu-project/qemu/qemu/debian-alpha-cross',

> '/tmp/docker_buildsstz1jcy']' returned non-zero exit status 1.

> /home/petmay01/linaro/qemu-for-merges/tests/docker/Makefile.include:58:

> recipe for target 'docker-image-debian-alpha-cross' failed

> make: *** [docker-image-debian-alpha-cross] Error 1


This is it tripping up on missing registry (as I switched the registry
from my previous patch posts to point to the "official" gitlab). You can
work around this for now by passing:

  make check-tcg REGISTRY=registry.gitlab.com/stsquad/qemu

I also just posted:

  Subject: [PATCH  v1 11/13] tests/docker: fall back more gracefully when pull fails
  Date: Thu,  9 Jul 2020 15:13:25 +0100
  Message-Id: <20200709141327.14631-12-alex.bennee@linaro.org>

Maybe I should just squash it into:

  tests/docker: add --registry support to tooling

?

> Also a compile failure on s390x, but since this isn't related

> to changes you made afaict I wonder if it's the result of

> a change in the build environment:

> /home/ubuntu/qemu/block/ssh.c: In function ‘check_host_key_knownhosts’:

> /home/ubuntu/qemu/block/ssh.c:281:28: error: storage size of ‘state’ isn’t known

>      enum ssh_known_hosts_e state;

>                             ^~~~~

> /home/ubuntu/qemu/block/ssh.c:289:13: error: implicit declaration of

> function ‘ssh_session_is_known_server’ [-Werror=implicit-funct

> ion-declaration]

>      state = ssh_session_is_known_server(s->session);

>              ^~~~~~~~~~~~~~~~~~~~~~~~~~~

> [and other errors]


As Philippe said I don't think this was me.

-- 
Alex Bennée