| Message ID | 20230215192530.299263-8-alex.bennee@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | testing/next: docker, avocado, unit, | expand |
On 2/15/23 09:25, Alex Bennée wrote: > The 22.04 LTS release has been out for almost a year now so its time > to update all the remaining images to the current LTS. We can also > drop some hacks we need for older clang TSAN support. > > Signed-off-by: Alex Bennée<alex.bennee@linaro.org> > --- > docs/devel/testing.rst | 4 ++-- > .gitlab-ci.d/buildtest.yml | 22 +++++++++---------- > .gitlab-ci.d/containers.yml | 4 ++-- > .../{ubuntu2004.docker => ubuntu2204.docker} | 16 +++++--------- > tests/docker/test-tsan | 2 +- > tests/lcitool/refresh | 10 +-------- > 6 files changed, 23 insertions(+), 35 deletions(-) > rename tests/docker/dockerfiles/{ubuntu2004.docker => ubuntu2204.docker} (91%) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> r~
On 15/02/2023 20.25, Alex Bennée wrote: > The 22.04 LTS release has been out for almost a year now so its time > to update all the remaining images to the current LTS. We can also > drop some hacks we need for older clang TSAN support. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > --- > docs/devel/testing.rst | 4 ++-- > .gitlab-ci.d/buildtest.yml | 22 +++++++++---------- > .gitlab-ci.d/containers.yml | 4 ++-- > .../{ubuntu2004.docker => ubuntu2204.docker} | 16 +++++--------- > tests/docker/test-tsan | 2 +- > tests/lcitool/refresh | 10 +-------- > 6 files changed, 23 insertions(+), 35 deletions(-) > rename tests/docker/dockerfiles/{ubuntu2004.docker => ubuntu2204.docker} (91%) Reviewed-by: Thomas Huth <thuth@redhat.com>
On Wed, Feb 15, 2023 at 2:25 PM Alex Bennée <alex.bennee@linaro.org> wrote: > > The 22.04 LTS release has been out for almost a year now so its time > to update all the remaining images to the current LTS. We can also > drop some hacks we need for older clang TSAN support. We still support Ubuntu 20.04 until 2024 though, don't we? Is it safe to not test this platform? I've long been uncertain about what our policy actually is for docker tests, if we want to test every platform we support or only some of them; and if it's only some of them, when do we choose the older and when do we choose the newer? > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > --- > docs/devel/testing.rst | 4 ++-- > .gitlab-ci.d/buildtest.yml | 22 +++++++++---------- > .gitlab-ci.d/containers.yml | 4 ++-- > .../{ubuntu2004.docker => ubuntu2204.docker} | 16 +++++--------- > tests/docker/test-tsan | 2 +- > tests/lcitool/refresh | 10 +-------- > 6 files changed, 23 insertions(+), 35 deletions(-) > rename tests/docker/dockerfiles/{ubuntu2004.docker => ubuntu2204.docker} (91%) > > diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst > index e10c47b5a7..309a575abe 100644 > --- a/docs/devel/testing.rst > +++ b/docs/devel/testing.rst > @@ -574,13 +574,13 @@ https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual > > Thread Sanitizer in Docker > ~~~~~~~~~~~~~~~~~~~~~~~~~~ > -TSan is currently supported in the ubuntu2004 docker. > +TSan is currently supported in the ubuntu2204 docker. > > The test-tsan test will build using TSan and then run make check. > > .. code:: > > - make docker-test-tsan@ubuntu2004 > + make docker-test-tsan@ubuntu2204 > > TSan warnings under docker are placed in files located at build/tsan/. > > diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml > index 8f332fc36f..be319ab2d0 100644 > --- a/.gitlab-ci.d/buildtest.yml > +++ b/.gitlab-ci.d/buildtest.yml > @@ -38,9 +38,9 @@ avocado-system-alpine: > build-system-ubuntu: > extends: .native_build_job_template > needs: > - job: amd64-ubuntu2004-container > + job: amd64-ubuntu2204-container > variables: > - IMAGE: ubuntu2004 > + IMAGE: ubuntu2204 > CONFIGURE_ARGS: --enable-docs --enable-fdt=system --enable-capstone > TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu > microblazeel-softmmu mips64el-softmmu > @@ -56,7 +56,7 @@ check-system-ubuntu: > - job: build-system-ubuntu > artifacts: true > variables: > - IMAGE: ubuntu2004 > + IMAGE: ubuntu2204 > MAKE_CHECK_ARGS: check > > avocado-system-ubuntu: > @@ -65,7 +65,7 @@ avocado-system-ubuntu: > - job: build-system-ubuntu > artifacts: true > variables: > - IMAGE: ubuntu2004 > + IMAGE: ubuntu2204 > MAKE_CHECK_ARGS: check-avocado > > build-system-debian: > @@ -459,10 +459,10 @@ avocado-cfi-x86_64: > tsan-build: > extends: .native_build_job_template > needs: > - job: amd64-ubuntu2004-container > + job: amd64-ubuntu2204-container > variables: > - IMAGE: ubuntu2004 > - CONFIGURE_ARGS: --enable-tsan --cc=clang-10 --cxx=clang++-10 > + IMAGE: ubuntu2204 > + CONFIGURE_ARGS: --enable-tsan --cc=clang --cxx=clang++ > --enable-trace-backends=ust --enable-fdt=system --disable-slirp > TARGETS: x86_64-softmmu ppc64-softmmu riscv64-softmmu x86_64-linux-user > MAKE_CHECK_ARGS: bench V=1 > @@ -471,10 +471,10 @@ tsan-build: > gcov: > extends: .native_build_job_template > needs: > - job: amd64-ubuntu2004-container > + job: amd64-ubuntu2204-container > timeout: 80m > variables: > - IMAGE: ubuntu2004 > + IMAGE: ubuntu2204 > CONFIGURE_ARGS: --enable-gcov > TARGETS: aarch64-softmmu ppc64-softmmu s390x-softmmu x86_64-softmmu > MAKE_CHECK_ARGS: check > @@ -539,9 +539,9 @@ build-tci: > build-coroutine-sigaltstack: > extends: .native_build_job_template > needs: > - job: amd64-ubuntu2004-container > + job: amd64-ubuntu2204-container > variables: > - IMAGE: ubuntu2004 > + IMAGE: ubuntu2204 > CONFIGURE_ARGS: --with-coroutine=sigaltstack --disable-tcg > --enable-trace-backends=ftrace > MAKE_CHECK_ARGS: check-unit > diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml > index 96d2a3b58b..8637a13d86 100644 > --- a/.gitlab-ci.d/containers.yml > +++ b/.gitlab-ci.d/containers.yml > @@ -13,10 +13,10 @@ amd64-debian-container: > variables: > NAME: debian-amd64 > > -amd64-ubuntu2004-container: > +amd64-ubuntu2204-container: > extends: .container_job_template > variables: > - NAME: ubuntu2004 > + NAME: ubuntu2204 > > amd64-opensuse-leap-container: > extends: .container_job_template > diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2204.docker > similarity index 91% > rename from tests/docker/dockerfiles/ubuntu2004.docker > rename to tests/docker/dockerfiles/ubuntu2204.docker > index f34d88d33d..30b9e56793 100644 > --- a/tests/docker/dockerfiles/ubuntu2004.docker > +++ b/tests/docker/dockerfiles/ubuntu2204.docker > @@ -1,10 +1,10 @@ > # THIS FILE WAS AUTO-GENERATED > # > -# $ lcitool dockerfile --layers all ubuntu-2004 qemu > +# $ lcitool dockerfile --layers all ubuntu-2204 qemu > # > # https://gitlab.com/libvirt/libvirt-ci > > -FROM docker.io/library/ubuntu:20.04 > +FROM docker.io/library/ubuntu:22.04 > > RUN export DEBIAN_FRONTEND=noninteractive && \ > apt-get update && \ > @@ -14,7 +14,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ > bash \ > bc \ > bison \ > - bsdmainutils \ > + bsdextrautils \ > bzip2 \ > ca-certificates \ > ccache \ > @@ -36,6 +36,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ > libasan5 \ > libasound2-dev \ > libattr1-dev \ > + libbpf-dev \ > libbrlapi-dev \ > libbz2-dev \ > libc6-dev \ > @@ -90,6 +91,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ > libtasn1-6-dev \ > libubsan1 \ > libudev-dev \ > + liburing-dev \ > libusb-1.0-0-dev \ > libusbredirhost-dev \ > libvdeplug-dev \ > @@ -100,6 +102,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ > llvm \ > locales \ > make \ > + meson \ > multipath-tools \ > ncat \ > nettle-dev \ > @@ -111,11 +114,9 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ > python3-opencv \ > python3-pillow \ > python3-pip \ > - python3-setuptools \ > python3-sphinx \ > python3-sphinx-rtd-theme \ > python3-venv \ > - python3-wheel \ > python3-yaml \ > rpm2cpio \ > sed \ > @@ -139,13 +140,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ > ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \ > ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc > > -RUN /usr/bin/pip3 install meson==0.56.0 > - > ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" > ENV LANG "en_US.UTF-8" > ENV MAKE "/usr/bin/make" > ENV NINJA "/usr/bin/ninja" > ENV PYTHON "/usr/bin/python3" > -# Apply patch https://reviews.llvm.org/D75820 > -# This is required for TSan in clang-10 to compile with QEMU. > -RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h > diff --git a/tests/docker/test-tsan b/tests/docker/test-tsan > index 53d90d2f79..f6d6590e39 100755 > --- a/tests/docker/test-tsan > +++ b/tests/docker/test-tsan > @@ -21,7 +21,7 @@ setup_tsan() > tsan_log_dir="/tmp/qemu-test/build/tsan" > mkdir -p $tsan_log_dir > /dev/null || true > EXTRA_CONFIGURE_OPTS="${EXTRA_CONFIGURE_OPTS} --enable-tsan \ > - --cc=clang-10 --cxx=clang++-10 \ > + --cc=clang --cxx=clang++ \ > --disable-werror --extra-cflags=-O0" > # detect deadlocks is false currently simply because > # TSan crashes immediately with deadlock detector enabled. > diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh > index a5ea0efc3b..eeee9a7b34 100755 > --- a/tests/lcitool/refresh > +++ b/tests/lcitool/refresh > @@ -69,13 +69,6 @@ def generate_cirrus(target, trailer=None): > generate(filename, cmd, trailer) > > > -ubuntu2004_tsanhack = [ > - "# Apply patch https://reviews.llvm.org/D75820\n", > - "# This is required for TSan in clang-10 to compile with QEMU.\n", > - "RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h\n" > -] > - > - > # Netmap still needs to be manually built as it is yet to be packaged > # into a distro. We also add cscope and gtags which are used in the CI > # test > @@ -113,8 +106,7 @@ try: > trailer="".join(debian11_extras)) > generate_dockerfile("fedora", "fedora-37") > generate_dockerfile("opensuse-leap", "opensuse-leap-153") > - generate_dockerfile("ubuntu2004", "ubuntu-2004", > - trailer="".join(ubuntu2004_tsanhack)) > + generate_dockerfile("ubuntu2204", "ubuntu-2204") > > # > # Cross compiling builds > -- > 2.39.1 >
On Thu, Feb 16, 2023 at 01:15:30PM -0500, John Snow wrote: > On Wed, Feb 15, 2023 at 2:25 PM Alex Bennée <alex.bennee@linaro.org> wrote: > > > > The 22.04 LTS release has been out for almost a year now so its time > > to update all the remaining images to the current LTS. We can also > > drop some hacks we need for older clang TSAN support. > > We still support Ubuntu 20.04 until 2024 though, don't we? Is it safe > to not test this platform? > > I've long been uncertain about what our policy actually is for docker > tests, if we want to test every platform we support or only some of > them; and if it's only some of them, when do we choose the older and > when do we choose the newer? Ideally we would test both the oldest & newest versions of each distro we support. Practically though, we're compromised by the limited CI resources available. Dropping older Ubuntu images is a reasonable tradeoff, since we still have Debian images covered in CI. Debian can be thought of as an older version of Ubuntu to some extent, giving coverage that will mitigate the risks of dropping 20.04. With regards, Daniel
On Thu, Feb 16, 2023, 2:44 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > On Thu, Feb 16, 2023 at 01:15:30PM -0500, John Snow wrote: > > On Wed, Feb 15, 2023 at 2:25 PM Alex Bennée <alex.bennee@linaro.org> > wrote: > > > > > > The 22.04 LTS release has been out for almost a year now so its time > > > to update all the remaining images to the current LTS. We can also > > > drop some hacks we need for older clang TSAN support. > > > > We still support Ubuntu 20.04 until 2024 though, don't we? Is it safe > > to not test this platform? > > > > I've long been uncertain about what our policy actually is for docker > > tests, if we want to test every platform we support or only some of > > them; and if it's only some of them, when do we choose the older and > > when do we choose the newer? > > Ideally we would test both the oldest & newest versions of each > distro we support. Practically though, we're compromised by the > limited CI resources available. > Yes, understood. > Dropping older Ubuntu images is a reasonable tradeoff, since we > still have Debian images covered in CI. Debian can be thought > of as an older version of Ubuntu to some extent, giving coverage > that will mitigate the risks of dropping 20.04. > Okay, I'll take your word for that. I am not personally familiar with how much those distros diverge; I know Ubuntu is debian-based but that's the extent of my knowledge as I don't daily-drive either. So, firstly: Reviewed-by: John Snow <jsnow@redhat.com> because I suspect we all have our reasons and I also agree testing newer is generally of higher value than testing older. However, would it be possible to keep the older Ubuntu test as a manual execution that we could invoke at will, only during RC testing phase? If it's not a lot of work, I could even check that in myself as a follow-up if it isn't unwanted. I find that "oldest version of x" is quite useful to me for testing Python stuff in particular, as that ecosystem moves pretty fast. It'd be mighty convenient to me in particular to keep an old Ubuntu test around to run manually as needed. (Heck, even if it wasn't on CI at all but was just a container I could run locally, that would still be quite useful.) Whaddaya think? > With regards, > Daniel > -- > |: https://berrange.com -o- > https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- > https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- > https://www.instagram.com/dberrange :| > >
On Fri, Feb 17, 2023 at 11:35:44AM -0500, John Snow wrote: > On Thu, Feb 16, 2023, 2:44 PM Daniel P. Berrangé <berrange@redhat.com> > wrote: > > > On Thu, Feb 16, 2023 at 01:15:30PM -0500, John Snow wrote: > > > On Wed, Feb 15, 2023 at 2:25 PM Alex Bennée <alex.bennee@linaro.org> > > wrote: > > > > > > > > The 22.04 LTS release has been out for almost a year now so its time > > > > to update all the remaining images to the current LTS. We can also > > > > drop some hacks we need for older clang TSAN support. > > > > > > We still support Ubuntu 20.04 until 2024 though, don't we? Is it safe > > > to not test this platform? > > > > > > I've long been uncertain about what our policy actually is for docker > > > tests, if we want to test every platform we support or only some of > > > them; and if it's only some of them, when do we choose the older and > > > when do we choose the newer? > > > > Ideally we would test both the oldest & newest versions of each > > distro we support. Practically though, we're compromised by the > > limited CI resources available. > > > > Yes, understood. > > > > Dropping older Ubuntu images is a reasonable tradeoff, since we > > still have Debian images covered in CI. Debian can be thought > > of as an older version of Ubuntu to some extent, giving coverage > > that will mitigate the risks of dropping 20.04. > > > > Okay, I'll take your word for that. I am not personally familiar with how > much those distros diverge; I know Ubuntu is debian-based but that's the > extent of my knowledge as I don't daily-drive either. > > So, firstly: > > Reviewed-by: John Snow <jsnow@redhat.com> > > because I suspect we all have our reasons and I also agree testing newer is > generally of higher value than testing older. > > However, would it be possible to keep the older Ubuntu test as a manual > execution that we could invoke at will, only during RC testing phase? If > it's not a lot of work, I could even check that in myself as a follow-up if > it isn't unwanted. > > I find that "oldest version of x" is quite useful to me for testing Python > stuff in particular, as that ecosystem moves pretty fast. It'd be mighty > convenient to me in particular to keep an old Ubuntu test around to run > manually as needed. > > (Heck, even if it wasn't on CI at all but was just a container I could run > locally, that would still be quite useful.) > > Whaddaya think? It would be pretty trivial to have tests/docker/dockerfiles contain Dockerfiles for *every* supported distro version we have, and then only build & test a subset in CI. It would merely suggest that we change our naming convention so the dockerfiles in that dir include the version. Basically adopting the standard libvirt-ci naming convention for targets of $OSNAME-$OSVERSION: $ lcitool targets almalinux-8 almalinux-9 alpine-315 alpine-316 alpine-edge centos-stream-8 centos-stream-9 debian-10 debian-11 debian-sid fedora-36 fedora-37 fedora-rawhide freebsd-12 freebsd-13 freebsd-current macos-12 macos-13 opensuse-leap-153 opensuse-leap-154 opensuse-tumbleweed ubuntu-1804 ubuntu-2004 ubuntu-2204 Contributors can then use 'make docker-XXXX' to run build tests locally on specific distros when they need to test something that isn't covered by default in out gating CI With regards, Daniel
On Fri, Feb 17, 2023, 12:14 PM Daniel P. Berrangé <berrange@redhat.com> wrote: > On Fri, Feb 17, 2023 at 11:35:44AM -0500, John Snow wrote: > > On Thu, Feb 16, 2023, 2:44 PM Daniel P. Berrangé <berrange@redhat.com> > > wrote: > > > > > On Thu, Feb 16, 2023 at 01:15:30PM -0500, John Snow wrote: > > > > On Wed, Feb 15, 2023 at 2:25 PM Alex Bennée <alex.bennee@linaro.org> > > > wrote: > > > > > > > > > > The 22.04 LTS release has been out for almost a year now so its > time > > > > > to update all the remaining images to the current LTS. We can also > > > > > drop some hacks we need for older clang TSAN support. > > > > > > > > We still support Ubuntu 20.04 until 2024 though, don't we? Is it safe > > > > to not test this platform? > > > > > > > > I've long been uncertain about what our policy actually is for docker > > > > tests, if we want to test every platform we support or only some of > > > > them; and if it's only some of them, when do we choose the older and > > > > when do we choose the newer? > > > > > > Ideally we would test both the oldest & newest versions of each > > > distro we support. Practically though, we're compromised by the > > > limited CI resources available. > > > > > > > Yes, understood. > > > > > > > Dropping older Ubuntu images is a reasonable tradeoff, since we > > > still have Debian images covered in CI. Debian can be thought > > > of as an older version of Ubuntu to some extent, giving coverage > > > that will mitigate the risks of dropping 20.04. > > > > > > > Okay, I'll take your word for that. I am not personally familiar with how > > much those distros diverge; I know Ubuntu is debian-based but that's the > > extent of my knowledge as I don't daily-drive either. > > > > So, firstly: > > > > Reviewed-by: John Snow <jsnow@redhat.com> > > > > because I suspect we all have our reasons and I also agree testing newer > is > > generally of higher value than testing older. > > > > However, would it be possible to keep the older Ubuntu test as a manual > > execution that we could invoke at will, only during RC testing phase? If > > it's not a lot of work, I could even check that in myself as a follow-up > if > > it isn't unwanted. > > > > I find that "oldest version of x" is quite useful to me for testing > Python > > stuff in particular, as that ecosystem moves pretty fast. It'd be mighty > > convenient to me in particular to keep an old Ubuntu test around to run > > manually as needed. > > > > (Heck, even if it wasn't on CI at all but was just a container I could > run > > locally, that would still be quite useful.) > > > > Whaddaya think? > > It would be pretty trivial to have tests/docker/dockerfiles contain > Dockerfiles for *every* supported distro version we have, and then > only build & test a subset in CI. It would merely suggest that we > change our naming convention so the dockerfiles in that dir include > the version. Basically adopting the standard libvirt-ci naming > convention for targets of $OSNAME-$OSVERSION: > > $ lcitool targets > almalinux-8 > almalinux-9 > alpine-315 > alpine-316 > alpine-edge > centos-stream-8 > centos-stream-9 > debian-10 > debian-11 > debian-sid > fedora-36 > fedora-37 > fedora-rawhide > freebsd-12 > freebsd-13 > freebsd-current > macos-12 > macos-13 > Wait, what? How does this work?? opensuse-leap-153 > opensuse-leap-154 > opensuse-tumbleweed > ubuntu-1804 > ubuntu-2004 > ubuntu-2204 > > Contributors can then use 'make docker-XXXX' to run build tests > locally on specific distros when they need to test something > that isn't covered by default in out gating CI > OK, I might follow up on this, then. I would find this useful for proving certain python build system changes are not disruptive. In contrast to C world, I find modern Pythonisms sneak in with quite an increased frequency, so having manual testing for the oldest platforms has some value there, but only every once in a while. Not worth our CI minutes. Carry on as normal for this series, please and thank you! > > With regards, > Daniel > -- > |: https://berrange.com -o- > https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- > https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- > https://www.instagram.com/dberrange :| > >
On 17/2/23 18:14, Daniel P. Berrangé wrote: > On Fri, Feb 17, 2023 at 11:35:44AM -0500, John Snow wrote: >> However, would it be possible to keep the older Ubuntu test as a manual >> execution that we could invoke at will, only during RC testing phase? If >> it's not a lot of work, I could even check that in myself as a follow-up if >> it isn't unwanted. >> >> I find that "oldest version of x" is quite useful to me for testing Python >> stuff in particular, as that ecosystem moves pretty fast. It'd be mighty >> convenient to me in particular to keep an old Ubuntu test around to run >> manually as needed. >> >> (Heck, even if it wasn't on CI at all but was just a container I could run >> locally, that would still be quite useful.) >> >> Whaddaya think? > > It would be pretty trivial to have tests/docker/dockerfiles contain > Dockerfiles for *every* supported distro version we have, and then > only build & test a subset in CI. It would merely suggest that we That would be great! Could be added as byte-sized issue. > change our naming convention so the dockerfiles in that dir include > the version. Basically adopting the standard libvirt-ci naming > convention for targets of $OSNAME-$OSVERSION: > > $ lcitool targets > almalinux-8 > almalinux-9 > alpine-315 > alpine-316 > alpine-edge > centos-stream-8 > centos-stream-9 > debian-10 > debian-11 > debian-sid > fedora-36 > fedora-37 > fedora-rawhide > freebsd-12 > freebsd-13 > freebsd-current > macos-12 > macos-13 > opensuse-leap-153 > opensuse-leap-154 > opensuse-tumbleweed > ubuntu-1804 > ubuntu-2004 > ubuntu-2204 > > Contributors can then use 'make docker-XXXX' to run build tests > locally on specific distros when they need to test something > that isn't covered by default in out gating CI > > > With regards, > Daniel
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst index e10c47b5a7..309a575abe 100644 --- a/docs/devel/testing.rst +++ b/docs/devel/testing.rst @@ -574,13 +574,13 @@ https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual Thread Sanitizer in Docker ~~~~~~~~~~~~~~~~~~~~~~~~~~ -TSan is currently supported in the ubuntu2004 docker. +TSan is currently supported in the ubuntu2204 docker. The test-tsan test will build using TSan and then run make check. .. code:: - make docker-test-tsan@ubuntu2004 + make docker-test-tsan@ubuntu2204 TSan warnings under docker are placed in files located at build/tsan/. diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index 8f332fc36f..be319ab2d0 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -38,9 +38,9 @@ avocado-system-alpine: build-system-ubuntu: extends: .native_build_job_template needs: - job: amd64-ubuntu2004-container + job: amd64-ubuntu2204-container variables: - IMAGE: ubuntu2004 + IMAGE: ubuntu2204 CONFIGURE_ARGS: --enable-docs --enable-fdt=system --enable-capstone TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu microblazeel-softmmu mips64el-softmmu @@ -56,7 +56,7 @@ check-system-ubuntu: - job: build-system-ubuntu artifacts: true variables: - IMAGE: ubuntu2004 + IMAGE: ubuntu2204 MAKE_CHECK_ARGS: check avocado-system-ubuntu: @@ -65,7 +65,7 @@ avocado-system-ubuntu: - job: build-system-ubuntu artifacts: true variables: - IMAGE: ubuntu2004 + IMAGE: ubuntu2204 MAKE_CHECK_ARGS: check-avocado build-system-debian: @@ -459,10 +459,10 @@ avocado-cfi-x86_64: tsan-build: extends: .native_build_job_template needs: - job: amd64-ubuntu2004-container + job: amd64-ubuntu2204-container variables: - IMAGE: ubuntu2004 - CONFIGURE_ARGS: --enable-tsan --cc=clang-10 --cxx=clang++-10 + IMAGE: ubuntu2204 + CONFIGURE_ARGS: --enable-tsan --cc=clang --cxx=clang++ --enable-trace-backends=ust --enable-fdt=system --disable-slirp TARGETS: x86_64-softmmu ppc64-softmmu riscv64-softmmu x86_64-linux-user MAKE_CHECK_ARGS: bench V=1 @@ -471,10 +471,10 @@ tsan-build: gcov: extends: .native_build_job_template needs: - job: amd64-ubuntu2004-container + job: amd64-ubuntu2204-container timeout: 80m variables: - IMAGE: ubuntu2004 + IMAGE: ubuntu2204 CONFIGURE_ARGS: --enable-gcov TARGETS: aarch64-softmmu ppc64-softmmu s390x-softmmu x86_64-softmmu MAKE_CHECK_ARGS: check @@ -539,9 +539,9 @@ build-tci: build-coroutine-sigaltstack: extends: .native_build_job_template needs: - job: amd64-ubuntu2004-container + job: amd64-ubuntu2204-container variables: - IMAGE: ubuntu2004 + IMAGE: ubuntu2204 CONFIGURE_ARGS: --with-coroutine=sigaltstack --disable-tcg --enable-trace-backends=ftrace MAKE_CHECK_ARGS: check-unit diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml index 96d2a3b58b..8637a13d86 100644 --- a/.gitlab-ci.d/containers.yml +++ b/.gitlab-ci.d/containers.yml @@ -13,10 +13,10 @@ amd64-debian-container: variables: NAME: debian-amd64 -amd64-ubuntu2004-container: +amd64-ubuntu2204-container: extends: .container_job_template variables: - NAME: ubuntu2004 + NAME: ubuntu2204 amd64-opensuse-leap-container: extends: .container_job_template diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2204.docker similarity index 91% rename from tests/docker/dockerfiles/ubuntu2004.docker rename to tests/docker/dockerfiles/ubuntu2204.docker index f34d88d33d..30b9e56793 100644 --- a/tests/docker/dockerfiles/ubuntu2004.docker +++ b/tests/docker/dockerfiles/ubuntu2204.docker @@ -1,10 +1,10 @@ # THIS FILE WAS AUTO-GENERATED # -# $ lcitool dockerfile --layers all ubuntu-2004 qemu +# $ lcitool dockerfile --layers all ubuntu-2204 qemu # # https://gitlab.com/libvirt/libvirt-ci -FROM docker.io/library/ubuntu:20.04 +FROM docker.io/library/ubuntu:22.04 RUN export DEBIAN_FRONTEND=noninteractive && \ apt-get update && \ @@ -14,7 +14,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ bash \ bc \ bison \ - bsdmainutils \ + bsdextrautils \ bzip2 \ ca-certificates \ ccache \ @@ -36,6 +36,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ libasan5 \ libasound2-dev \ libattr1-dev \ + libbpf-dev \ libbrlapi-dev \ libbz2-dev \ libc6-dev \ @@ -90,6 +91,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ libtasn1-6-dev \ libubsan1 \ libudev-dev \ + liburing-dev \ libusb-1.0-0-dev \ libusbredirhost-dev \ libvdeplug-dev \ @@ -100,6 +102,7 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ llvm \ locales \ make \ + meson \ multipath-tools \ ncat \ nettle-dev \ @@ -111,11 +114,9 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ python3-opencv \ python3-pillow \ python3-pip \ - python3-setuptools \ python3-sphinx \ python3-sphinx-rtd-theme \ python3-venv \ - python3-wheel \ python3-yaml \ rpm2cpio \ sed \ @@ -139,13 +140,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/g++ && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc -RUN /usr/bin/pip3 install meson==0.56.0 - ENV CCACHE_WRAPPERSDIR "/usr/libexec/ccache-wrappers" ENV LANG "en_US.UTF-8" ENV MAKE "/usr/bin/make" ENV NINJA "/usr/bin/ninja" ENV PYTHON "/usr/bin/python3" -# Apply patch https://reviews.llvm.org/D75820 -# This is required for TSan in clang-10 to compile with QEMU. -RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h diff --git a/tests/docker/test-tsan b/tests/docker/test-tsan index 53d90d2f79..f6d6590e39 100755 --- a/tests/docker/test-tsan +++ b/tests/docker/test-tsan @@ -21,7 +21,7 @@ setup_tsan() tsan_log_dir="/tmp/qemu-test/build/tsan" mkdir -p $tsan_log_dir > /dev/null || true EXTRA_CONFIGURE_OPTS="${EXTRA_CONFIGURE_OPTS} --enable-tsan \ - --cc=clang-10 --cxx=clang++-10 \ + --cc=clang --cxx=clang++ \ --disable-werror --extra-cflags=-O0" # detect deadlocks is false currently simply because # TSan crashes immediately with deadlock detector enabled. diff --git a/tests/lcitool/refresh b/tests/lcitool/refresh index a5ea0efc3b..eeee9a7b34 100755 --- a/tests/lcitool/refresh +++ b/tests/lcitool/refresh @@ -69,13 +69,6 @@ def generate_cirrus(target, trailer=None): generate(filename, cmd, trailer) -ubuntu2004_tsanhack = [ - "# Apply patch https://reviews.llvm.org/D75820\n", - "# This is required for TSan in clang-10 to compile with QEMU.\n", - "RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h\n" -] - - # Netmap still needs to be manually built as it is yet to be packaged # into a distro. We also add cscope and gtags which are used in the CI # test @@ -113,8 +106,7 @@ try: trailer="".join(debian11_extras)) generate_dockerfile("fedora", "fedora-37") generate_dockerfile("opensuse-leap", "opensuse-leap-153") - generate_dockerfile("ubuntu2004", "ubuntu-2004", - trailer="".join(ubuntu2004_tsanhack)) + generate_dockerfile("ubuntu2204", "ubuntu-2204") # # Cross compiling builds
The 22.04 LTS release has been out for almost a year now so its time to update all the remaining images to the current LTS. We can also drop some hacks we need for older clang TSAN support. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> --- docs/devel/testing.rst | 4 ++-- .gitlab-ci.d/buildtest.yml | 22 +++++++++---------- .gitlab-ci.d/containers.yml | 4 ++-- .../{ubuntu2004.docker => ubuntu2204.docker} | 16 +++++--------- tests/docker/test-tsan | 2 +- tests/lcitool/refresh | 10 +-------- 6 files changed, 23 insertions(+), 35 deletions(-) rename tests/docker/dockerfiles/{ubuntu2004.docker => ubuntu2204.docker} (91%)