| Message ID | 20220914155950.804707-1-alex.bennee@linaro.org |
|---|---|
| Headers | show |
| Series | testing/next pre-PR (testing update and mips deprecation) | expand |
On 9/14/22 16:59, Alex Bennée wrote: > It's becoming harder to maintain a cross-compiler to test this host > architecture as the old stable Debian 10 ("Buster") moved into LTS > which supports fewer architectures. For now: > > - mark it's deprecation in the docs > - downgrade the containers to build TCG tests only > - drop the cross builds from our CI > > Users with an appropriate toolchain and user-space can still take > their chances building it. > > Signed-off-by: Alex Bennée<alex.bennee@linaro.org> > Reviewed-by: Philippe Mathieu-Daudé<f4bug@amsat.org> > Reviewed-by: Huacai Chen<chenhuacai@kernel.org> > Message-Id:<20220826172128.353798-16-alex.bennee@linaro.org> > > --- > v2 > - explicit little endian instead of LE > - s/A/As/ > - restore mips to dockerfile > --- > docs/about/build-platforms.rst | 2 +- > docs/about/deprecated.rst | 13 +++++++ > .gitlab-ci.d/container-cross.yml | 1 - > .gitlab-ci.d/crossbuilds.yml | 14 ------- > tests/docker/Makefile.include | 5 +-- > .../dockerfiles/debian-mips-cross.docker | 38 +++++-------------- > 6 files changed, 26 insertions(+), 47 deletions(-) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> r~
I am using gcc230 machine for the gcc compile farm. This is a big endian mips64 machine runnig Debian Buster. When compiling the qemu 7.1.0 release source, the generated binaries are 32-bit mips binaries, and I did not find out how to generate a 64-bit versions of the executables. As mips32 seems to still be the default arch that gcc uses, I don't really understand the idea of depreciating big endian mips32. Is this solely related to cross-compilation issues? Pierre Muller More information on gcc230: muller@gcc230:~$ uname -a Linux gcc230 4.9.79-UBNT_E300 #9 SMP Tue Jul 13 13:04:47 BST 2021 mips64 GNU/Linux muller@gcc230:~$ cat /etc/os-release PRETTY_NAME="Debian GNU/Linux 10 (buster)" NAME="Debian GNU/Linux" VERSION_ID="10" VERSION="10 (buster)" VERSION_CODENAME=buster ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/" muller@gcc230:~$ gcc --version gcc (Debian 8.3.0-6) 8.3.0 Copyright (C) 2018 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. muller@gcc230:~$ gcc -print-libgcc-file-name /usr/lib/gcc/mips-linux-gnu/8/libgcc.a muller@gcc230:~$ gcc -mabi=64 -print-libgcc-file-name /usr/lib/gcc/mips-linux-gnu/8/64/libgcc.a
On 9/16/22 10:08, Pierre Muller wrote: > > I am using gcc230 machine for the gcc compile farm. > > This is a big endian mips64 machine runnig Debian Buster. > > When compiling the qemu 7.1.0 release source, > the generated binaries are 32-bit mips binaries, > and I did not find out how to generate a 64-bit versions > of the executables. Yes, that host seems to have been installed with the o32 abi instead of the n64 (or n32) abi. > As mips32 seems to still be the default arch that gcc uses, > I don't really understand the idea of depreciating big endian mips32. > > Is this solely related to cross-compilation issues? Yes. Even gcc230 is fairly small for actually compiling qemu, it takes many hours. So for many hosts we rely on cross-compilation from x86_64. For that, we rely on the set of cross-compilers built by Debian 11 (bullseye) plus (!) the host libraries for that platform. We cannot simply rely on crossbuild-essential-mips because building qemu requires many more system libraries than just libc. In https://www.debian.org/releases/bullseye/, you'll note that big-endian mips is not listed, so we are now missing those system libraries. We are not intending to *remove* support for big-endian mips, as 99% of the code paths are shared with little-endian mips, which we can continue to test. But we are now saying that big-endian mips is not "supported" and might break. r~
Pierre Muller <pierre@freepascal.org> writes: > I am using gcc230 machine for the gcc compile farm. > > This is a big endian mips64 machine runnig Debian Buster. That's still oldstable, the current release of Debian doesn't support BE mips in either 32 or 64 bit. As bullseye was released last year buster will drop out of QEMU support window by August 2023 at the latest (current LTS + 2 years or upstream drops support whichever comes first). > When compiling the qemu 7.1.0 release source, > the generated binaries are 32-bit mips binaries, > and I did not find out how to generate a 64-bit versions > of the executables. I don't think we've ever been able to cross build QEMU BE mips64 - it was only with buster we stopped relying on sid for access to working cross compilers for building TCG tests: 4575a701ea (tests/docker: move our mips64 cross compile to Buster) > As mips32 seems to still be the default arch that gcc uses, > I don't really understand the idea of depreciating big endian mips32. > > Is this solely related to cross-compilation issues? Decent cross-compilation support for building QEMU is the minimum we need to ensure things don't bitrot. Ideally we would have real HW running a non-bespoke OS with a gitlab runner so we could build *and* run tests. However finding such HW is even harder than keeping the cross compilation working. > > Pierre Muller > > > More information on gcc230: > muller@gcc230:~$ uname -a > Linux gcc230 4.9.79-UBNT_E300 #9 SMP Tue Jul 13 13:04:47 BST 2021 mips64 GNU/Linux > muller@gcc230:~$ cat /etc/os-release > PRETTY_NAME="Debian GNU/Linux 10 (buster)" > NAME="Debian GNU/Linux" > VERSION_ID="10" > VERSION="10 (buster)" > VERSION_CODENAME=buster > ID=debian > HOME_URL="https://www.debian.org/" > SUPPORT_URL="https://www.debian.org/support" > BUG_REPORT_URL="https://bugs.debian.org/" > muller@gcc230:~$ gcc --version > gcc (Debian 8.3.0-6) 8.3.0 > Copyright (C) 2018 Free Software Foundation, Inc. > This is free software; see the source for copying conditions. There is NO > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > muller@gcc230:~$ gcc -print-libgcc-file-name > /usr/lib/gcc/mips-linux-gnu/8/libgcc.a > muller@gcc230:~$ gcc -mabi=64 -print-libgcc-file-name > /usr/lib/gcc/mips-linux-gnu/8/64/libgcc.a
Le 16/09/2022 à 10:38, Richard Henderson a écrit : > On 9/16/22 10:08, Pierre Muller wrote: >> >> I am using gcc230 machine for the gcc compile farm. >> >> This is a big endian mips64 machine runnig Debian Buster. >> >> When compiling the qemu 7.1.0 release source, >> the generated binaries are 32-bit mips binaries, >> and I did not find out how to generate a 64-bit versions >> of the executables. > > Yes, that host seems to have been installed with the o32 abi instead of the n64 (or n32) abi. Indeed, >> As mips32 seems to still be the default arch that gcc uses, >> I don't really understand the idea of depreciating big endian mips32. >> >> Is this solely related to cross-compilation issues? > > Yes. Even gcc230 is fairly small for actually compiling qemu, it takes many hours. So > for many hosts we rely on cross-compilation from x86_64. > > For that, we rely on the set of cross-compilers built by Debian 11 (bullseye) plus (!) the > host libraries for that platform. We cannot simply rely on crossbuild-essential-mips > because building qemu requires many more system libraries than just libc. > > In https://www.debian.org/releases/bullseye/, you'll note that big-endian mips is not > listed, so we are now missing those system libraries. So this is not limited to mips32, as big endian mips64 is also not supported anymore in bulleye... > We are not intending to *remove* support for big-endian mips, as 99% of the code paths are > shared with little-endian mips, which we can continue to test. But we are now saying that > big-endian mips is not "supported" and might break. Thank you for your answer and the clarifications! Pierre
On Fri, Sep 16, 2022 at 12:10:30PM +0200, Pierre Muller wrote: > > > Le 16/09/2022 à 10:38, Richard Henderson a écrit : > > On 9/16/22 10:08, Pierre Muller wrote: > > > > > > I am using gcc230 machine for the gcc compile farm. > > > > > > This is a big endian mips64 machine runnig Debian Buster. > > > > > > When compiling the qemu 7.1.0 release source, > > > the generated binaries are 32-bit mips binaries, > > > and I did not find out how to generate a 64-bit versions > > > of the executables. > > > > Yes, that host seems to have been installed with the o32 abi instead of the n64 (or n32) abi. > > Indeed, > > > As mips32 seems to still be the default arch that gcc uses, > > > I don't really understand the idea of depreciating big endian mips32. > > > > > > Is this solely related to cross-compilation issues? > > > > Yes. Even gcc230 is fairly small for actually compiling qemu, it takes many hours. So > > for many hosts we rely on cross-compilation from x86_64. > > > > For that, we rely on the set of cross-compilers built by Debian 11 (bullseye) plus (!) the > > host libraries for that platform. We cannot simply rely on crossbuild-essential-mips > > because building qemu requires many more system libraries than just libc. > > > > In https://www.debian.org/releases/bullseye/, you'll note that big-endian mips is not > > listed, so we are now missing those system libraries. > > So this is not limited to mips32, as big endian mips64 is also not supported anymore in bulleye... Right, so as a general policy, for a platform target to be considered fully supported, there needs to be a non-EOL (end of life) OS distro with which we can run automated CI. In terms of unusual architectures, Debian has historically been our best bet for being able to put together container images suitable for running CI. When even Debian has dropped an architecture, as well as removing our ability to do CI, it is also a sign of the architectures diminishing importance to the ecosystem as a whole. This makes it hard to justify investing in figuring out new CI options, though we're open to suggestions and help if people can point to non-EOL platforms that can be used, especially if container based. We don't wish to use end of life platforms for CI, since we periodically intend to increase our minimum required versions of 3rd party libraries, based on what current OS ship & testing using EOL platforms would prevent that. FWIW: https://www.qemu.org/docs/master/about/build-platforms.html > > We are not intending to *remove* support for big-endian mips, as 99% of the code paths are > > shared with little-endian mips, which we can continue to test. But we are now saying that > > big-endian mips is not "supported" and might break. > > Thank you for your answer and the clarifications! In practical terms the lack of CI means that we can't guarantee that a new QEMU release will compile successfully. We're handing off responsbility for keeping it working to any interested users to do adhoc testing of their own. We can still accept bug reports & patches when people discover problems. With regards, Daniel
On 16/9/22 12:22, Daniel P. Berrangé wrote: > On Fri, Sep 16, 2022 at 12:10:30PM +0200, Pierre Muller wrote: >> Le 16/09/2022 à 10:38, Richard Henderson a écrit : >>> We are not intending to *remove* support for big-endian mips, as 99% of the code paths are >>> shared with little-endian mips, which we can continue to test. But we are now saying that >>> big-endian mips is not "supported" and might break. >> >> Thank you for your answer and the clarifications! > > In practical terms the lack of CI means that we can't guarantee that a > new QEMU release will compile successfully. We're handing off responsbility > for keeping it working to any interested users to do adhoc testing of their > own. We can still accept bug reports & patches when people discover problems. Indeed. This is the situation of (host) HPPA. 3 people still build QEMU tools on it and report (build system) bugs from time to time, or send patches.
Hi, This is the current state of the testing tree. It was slightly delayed from the last posting as I was waiting for the libvirt-ci update to get merged. Changes since last post: - added some more avocado timeouts - moved cflags hack in the --disable-pie patch into configure - usual addressing of review comments (see --- in patches) I'd like to roll the PR next week so please review the remaining un-reviewed patches: - tests/docker: remove FROM qemu/ support from docker.py - tests/docker: update and flatten debian-hexagon-cross - tests/docker: update and flatten debian-amd64-cross - tests/docker: flatten debian-riscv64-test-cross - configure: explicitly set cflags for --disable-pie - tests/docker: remove tricore qemu/debian10 dependency - tests/avocado: reduce the default timeout to 120s - tests/avocado: add explicit timeout for ppc64le TCG tests - tests/avocado: add explicit timeout for s390 TCG tests - tests/avocado: add explicit timeout for Aarch64 TCG tests - gitlab: reduce targets in cross_user_build_job Alex Bennée (27): gitlab: reduce targets in cross_user_build_job tests/avocado: add explicit timeout for Aarch64 TCG tests tests/avocado: add explicit timeout for s390 TCG tests tests/avocado: add explicit timeout for ppc64le TCG tests tests/avocado: split the AST2x00Machine classes tests/avocado: reduce the default timeout to 120s tests/docker: update and flatten debian-alpha-cross tests/docker: update and flatten debian-hppa-cross tests/docker: update and flatten debian-m68k-cross tests/docker: update and flatten debian-mips64-cross tests/docker: update and flatten debian-sh4-cross tests/docker: update and flatten debian-sparc64-cross tests/docker: flatten debian-powerpc-test-cross tests/docker: remove tricore qemu/debian10 dependency tests/docker: remove amd64 qemu/debian10 dependency configure: explicitly set cflags for --disable-pie gitlab-ci: update aarch32/aarch64 custom runner jobs Deprecate 32 bit big-endian MIPS tests/docker: flatten debian-riscv64-test-cross tests/docker: update and flatten debian-all-test-cross tests/lcitool: bump to latest version tests/docker: update and flatten debian-amd64-cross tests/docker: update and flatten debian-loongarch-cross tests/docker: update and flatten debian-hexagon-cross tests/docker: update and flatten debian-toolchain tests/docker: remove FROM qemu/ support from docker.py tests/docker: remove the Debian base images Richard Henderson (1): gitlab-ci/custom-runners: Disable -static-pie for ubuntu-20.04-aarch64 Thomas Huth (2): tests/avocado/boot_linux_console: Fix the test_aarch64_xlnx_versal_virt test tests/vm: Remove obsolete Fedora VM test docs/about/build-platforms.rst | 2 +- docs/about/deprecated.rst | 13 ++ docs/devel/testing.rst | 2 +- configure | 6 + .gitlab-ci.d/cirrus/freebsd-12.vars | 2 +- .gitlab-ci.d/cirrus/freebsd-13.vars | 2 +- .gitlab-ci.d/container-core.yml | 5 - .gitlab-ci.d/container-cross.yml | 12 -- .gitlab-ci.d/containers.yml | 5 - .gitlab-ci.d/crossbuild-template.yml | 5 +- .gitlab-ci.d/crossbuilds.yml | 14 -- .gitlab-ci.d/custom-runners.yml | 4 +- ...4-aarch32.yml => ubuntu-22.04-aarch32.yml} | 6 +- ...4-aarch64.yml => ubuntu-22.04-aarch64.yml} | 38 ++-- MAINTAINERS | 3 +- tests/avocado/avocado_qemu/__init__.py | 2 +- tests/avocado/boot_linux.py | 5 + tests/avocado/boot_linux_console.py | 6 +- tests/avocado/machine_aspeed.py | 18 ++ tests/docker/Makefile.include | 30 +-- tests/docker/docker.py | 38 +--- .../dockerfiles/debian-all-test-cross.docker | 18 +- .../dockerfiles/debian-alpha-cross.docker | 12 +- .../dockerfiles/debian-amd64-cross.docker | 178 ++++++++++++++-- .../dockerfiles/debian-hexagon-cross.docker | 19 +- .../dockerfiles/debian-hppa-cross.docker | 12 +- .../dockerfiles/debian-loongarch-cross.docker | 8 +- .../dockerfiles/debian-m68k-cross.docker | 12 +- .../dockerfiles/debian-mips-cross.docker | 38 +--- .../dockerfiles/debian-mips64-cross.docker | 12 +- .../debian-powerpc-test-cross.docker | 12 +- .../debian-riscv64-test-cross.docker | 10 +- .../dockerfiles/debian-sh4-cross.docker | 12 +- .../dockerfiles/debian-sparc64-cross.docker | 12 +- .../dockerfiles/debian-toolchain.docker | 5 +- tests/docker/dockerfiles/debian10.docker | 38 ---- tests/docker/dockerfiles/debian11.docker | 18 -- tests/docker/dockerfiles/opensuse-leap.docker | 3 +- tests/docker/dockerfiles/ubuntu2004.docker | 2 +- tests/lcitool/libvirt-ci | 2 +- tests/lcitool/refresh | 7 + tests/vm/Makefile.include | 3 +- tests/vm/fedora | 190 ------------------ 43 files changed, 365 insertions(+), 476 deletions(-) rename .gitlab-ci.d/custom-runners/{ubuntu-20.04-aarch32.yml => ubuntu-22.04-aarch32.yml} (86%) rename .gitlab-ci.d/custom-runners/{ubuntu-20.04-aarch64.yml => ubuntu-22.04-aarch64.yml} (82%) delete mode 100644 tests/docker/dockerfiles/debian10.docker delete mode 100644 tests/docker/dockerfiles/debian11.docker delete mode 100755 tests/vm/fedora