mbox

[PULL,00/31] testing/next and other misc fixes

Message ID 20220112112722.3641051-1-alex.bennee@linaro.org
State New
Headers show

Pull-request

https://github.com/stsquad/qemu.git tags/pull-for-7.0-110122-1

Message

Alex Bennée Jan. 12, 2022, 11:26 a.m. UTC
The following changes since commit bf99e0ec9a51976868d7a8334620716df15fe7fe:

  Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2022-01-11 10:12:29 +0000)

are available in the Git repository at:

  https://github.com/stsquad/qemu.git tags/pull-for-7.0-110122-1

for you to fetch changes up to dbd30b7abee963f4fb08892a7d7f920bb76ece58:

  linux-user: Remove the deprecated ppc64abi32 target (2022-01-11 13:00:53 +0000)

----------------------------------------------------------------
Various testing and other misc updates:

  - fix compiler warnings with ui and sdl
  - update QXL/spice dependancy
  - skip I/O tests on Alpine
  - update fedora image to latest version
  - integrate lcitool and regenerate docker images
  - favour CONFIG_LINUX_USER over CONFIG_LINUX
  - add libfuse3 dependencies to docker images
  - add dtb-kaslr-seed control knob to virt machine
  - fix build breakage from HMP update
  - update docs for C standard and suffix usage
  - add more logging for debugging user hole finding
  - fix bug with linux-user hold calculation
  - avoid affecting flags when printing results in float tests
  - add float reference files for ppc64
  - update FreeBSD to 12.3
  - add bison dependancy to tricore images
  - remove deprecated ppc64abi32 target

----------------------------------------------------------------
Alex Bennée (6):
      hw/arm: add control knob to disable kaslr_seed via DTB
      monitor: move x-query-profile into accel/tcg to fix build
      docs/devel: update C standard to C11
      docs/devel: more documentation on the use of suffixes
      linux-user/elfload: add extra logging for hole finding
      linux-user: don't adjust base of found hole

Brad Smith (1):
      FreeBSD: Upgrade to 12.3 release

Daniel P. Berrangé (17):
      ui: avoid compiler warnings from unused clipboard info variable
      meson: require liburing >= 0.3
      ui: avoid warnings about directdb on Alpine / musl libc
      ci: explicitly skip I/O tests on alpine
      tests/docker: switch fedora image to release 35
      tests: integrate lcitool for generating build env manifests
      tests/docker: auto-generate centos8.docker with lcitool
      tests/docker: auto-generate fedora.docker with lcitool
      tests/docker: auto-generate ubuntu1804.docker with lcitool
      tests/docker: auto-generate ubuntu2004.docker with lcitool
      tests/docker: auto-generate opensuse-leap.docker with lcitool
      tests/docker: remove ubuntu.docker container
      .gitlab-ci.d/cirrus: auto-generate variables with lcitool
      tests/docker: updates to alpine package list
      tests/docker: fix sorting of alpine image package lists
      tests/docker: fully expand the alpine package list
      tests/docker: auto-generate alpine.docker with lcitool

John Snow (1):
      spice: Update QXLInterface for spice >= 0.15.0

Paolo Bonzini (2):
      tests/tcg: use CONFIG_LINUX_USER, not CONFIG_LINUX
      docker: include bison in debian-tricore-cross

Richard Henderson (2):
      tests/tcg/multiarch: Read fp flags before printf
      test/tcg/ppc64le: Add float reference files

Stefan Hajnoczi (1):
      tests/docker: add libfuse3 development headers

Thomas Huth (1):
      linux-user: Remove the deprecated ppc64abi32 target

 docs/about/deprecated.rst                          |   7 -
 docs/about/removed-features.rst                    |   8 +
 docs/devel/style.rst                               |  14 +-
 docs/devel/testing.rst                             | 104 ++-
 docs/system/arm/virt.rst                           |   8 +
 docs/user/main.rst                                 |   1 -
 configure                                          |  29 +-
 Makefile                                           |   2 +
 configs/targets/ppc64abi32-linux-user.mak          |   8 -
 meson.build                                        |   3 +-
 qapi/machine.json                                  |   1 +
 include/glib-compat.h                              |   6 +-
 include/hw/arm/virt.h                              |   1 +
 include/ui/qemu-spice.h                            |   6 +
 include/ui/sdl2.h                                  |  11 +
 linux-user/ppc/target_syscall.h                    |   4 +-
 linux-user/syscall_defs.h                          |   6 +-
 accel/tcg/cpu-exec.c                               |  31 +
 hw/arm/virt.c                                      |  32 +-
 hw/display/qxl.c                                   |  14 +-
 linux-user/elfload.c                               |  27 +-
 linux-user/ppc/signal.c                            |  11 +-
 monitor/qmp-cmds.c                                 |  31 -
 tests/tcg/multiarch/float_convs.c                  |   2 +-
 tests/tcg/multiarch/float_madds.c                  |   2 +-
 ui/clipboard.c                                     |   4 +-
 ui/spice-display.c                                 |  11 +
 .gitlab-ci.d/buildtest.yml                         |  29 +-
 .gitlab-ci.d/cirrus.yml                            |   5 +-
 .gitlab-ci.d/cirrus/freebsd-12.vars                |  11 +-
 .gitlab-ci.d/cirrus/freebsd-13.vars                |  11 +-
 .gitlab-ci.d/cirrus/macos-11.vars                  |  11 +-
 .gitlab-ci.d/containers.yml                        |   5 -
 .gitmodules                                        |   3 +
 hmp-commands-info.hx                               |   2 +
 tests/docker/dockerfiles/alpine.docker             | 176 +++--
 tests/docker/dockerfiles/centos8.docker            | 244 ++++---
 .../docker/dockerfiles/debian-ppc64el-cross.docker |   2 +-
 .../docker/dockerfiles/debian-tricore-cross.docker |   1 +
 tests/docker/dockerfiles/fedora.docker             | 263 +++----
 tests/docker/dockerfiles/opensuse-leap.docker      | 246 ++++---
 tests/docker/dockerfiles/ubuntu.docker             |  71 --
 tests/docker/dockerfiles/ubuntu1804.docker         | 255 ++++---
 tests/docker/dockerfiles/ubuntu2004.docker         | 258 ++++---
 tests/lcitool/Makefile.include                     |  17 +
 tests/lcitool/libvirt-ci                           |   1 +
 tests/lcitool/projects/qemu.yml                    | 116 ++++
 tests/lcitool/refresh                              |  96 +++
 tests/tcg/configure.sh                             |  10 +-
 tests/tcg/hexagon/float_convs.ref                  | 152 ++--
 tests/tcg/hexagon/float_madds.ref                  |  48 +-
 tests/tcg/multiarch/Makefile.target                |   2 +-
 tests/tcg/ppc64le/float_convs.ref                  | 748 ++++++++++++++++++++
 tests/tcg/ppc64le/float_madds.ref                  | 768 +++++++++++++++++++++
 tests/tcg/x86_64/Makefile.target                   |   2 +-
 tests/vm/freebsd                                   |   8 +-
 56 files changed, 2982 insertions(+), 963 deletions(-)
 delete mode 100644 configs/targets/ppc64abi32-linux-user.mak
 delete mode 100644 tests/docker/dockerfiles/ubuntu.docker
 create mode 100644 tests/lcitool/Makefile.include
 create mode 160000 tests/lcitool/libvirt-ci
 create mode 100644 tests/lcitool/projects/qemu.yml
 create mode 100755 tests/lcitool/refresh
 create mode 100644 tests/tcg/ppc64le/float_convs.ref
 create mode 100644 tests/tcg/ppc64le/float_madds.ref

Comments

Peter Maydell Jan. 12, 2022, 9:20 p.m. UTC | #1
On Wed, 12 Jan 2022 at 11:27, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit bf99e0ec9a51976868d7a8334620716df15fe7fe:
>
>   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2022-01-11 10:12:29 +0000)
>
> are available in the Git repository at:
>
>   https://github.com/stsquad/qemu.git tags/pull-for-7.0-110122-1
>
> for you to fetch changes up to dbd30b7abee963f4fb08892a7d7f920bb76ece58:
>
>   linux-user: Remove the deprecated ppc64abi32 target (2022-01-11 13:00:53 +0000)
>
> ----------------------------------------------------------------
> Various testing and other misc updates:
>
>   - fix compiler warnings with ui and sdl
>   - update QXL/spice dependancy
>   - skip I/O tests on Alpine
>   - update fedora image to latest version
>   - integrate lcitool and regenerate docker images
>   - favour CONFIG_LINUX_USER over CONFIG_LINUX
>   - add libfuse3 dependencies to docker images
>   - add dtb-kaslr-seed control knob to virt machine
>   - fix build breakage from HMP update
>   - update docs for C standard and suffix usage
>   - add more logging for debugging user hole finding
>   - fix bug with linux-user hold calculation
>   - avoid affecting flags when printing results in float tests
>   - add float reference files for ppc64
>   - update FreeBSD to 12.3
>   - add bison dependancy to tricore images
>   - remove deprecated ppc64abi32 target

This seems to fail the ubuntu-18.04-s390x-all-linux-static job
with segfaults running linux-user binaries (not always the same
binary), eg:
https://gitlab.com/qemu-project/qemu/-/jobs/1968789446
https://gitlab.com/qemu-project/qemu/-/jobs/1968080419


thanks
-- PMM
Alex Bennée Jan. 13, 2022, 10:13 a.m. UTC | #2
Peter Maydell <peter.maydell@linaro.org> writes:

(adding the s390x people to the CC if they have any clues)

> On Wed, 12 Jan 2022 at 11:27, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> The following changes since commit bf99e0ec9a51976868d7a8334620716df15fe7fe:
>>
>>   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2022-01-11 10:12:29 +0000)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git tags/pull-for-7.0-110122-1
>>
>> for you to fetch changes up to dbd30b7abee963f4fb08892a7d7f920bb76ece58:
>>
>>   linux-user: Remove the deprecated ppc64abi32 target (2022-01-11 13:00:53 +0000)
>>
<snip>
> This seems to fail the ubuntu-18.04-s390x-all-linux-static job
> with segfaults running linux-user binaries (not always the same
> binary), eg:
> https://gitlab.com/qemu-project/qemu/-/jobs/1968789446
> https://gitlab.com/qemu-project/qemu/-/jobs/1968080419

*sigh*

So the regression is caused by:

  linux-user: don't adjust base of found hole

However it only occurs when pgb_static starts base at a low address. For
example:

  pgb_find_hole: base @ 13dd000 for 17432080 bytes
  pgb_static: base @ 13dd000 for 17432080 bytes
  Locating guest address space @ 0x13dd000

fails whereas:

  pgb_find_hole: base @ 41f97000 for 17432080 bytes      
  pgb_static: base @ 41f97000 for 17432080 bytes        
  Locating guest address space @ 0x41f97000      

works.

What I find confusing is why we end up with different addresses when
both QEMU and the test binary are static allocations. However the
varying allocation occurs before the change but without triggering the
crash:

  pgb_static: base @ 3dd000 for 17432080 bytes
  pgb_static: base @ 3dd000 for 17432080 bytes
  pgb_static: base @ 41246000 for 17432080 bytes
  pgb_static: base @ 3dd000 for 17432080 bytes
  pgb_static: base @ 40a2a000 for 17432080 bytes
  pgb_static: base @ 3dd000 for 17432080 bytes
  pgb_static: base @ 3dd000 for 17432080 bytes
  pgb_static: base @ 4060c000 for 17432080 bytes
  pgb_static: base @ 3dd000 for 17432080 bytes
  pgb_static: base @ 3dd000 for 17432080 bytes
  pgb_static: base @ 3dd000 for 17432080 bytes

>
>
> thanks
> -- PMM
Alex Bennée Jan. 13, 2022, 12:49 p.m. UTC | #3
Alex Bennée <alex.bennee@linaro.org> writes:

> Peter Maydell <peter.maydell@linaro.org> writes:
>
> (adding the s390x people to the CC if they have any clues)
>
>> On Wed, 12 Jan 2022 at 11:27, Alex Bennée <alex.bennee@linaro.org> wrote:
>>>
>>> The following changes since commit bf99e0ec9a51976868d7a8334620716df15fe7fe:
>>>
>>>   Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2022-01-11 10:12:29 +0000)
>>>
>>> are available in the Git repository at:
>>>
>>>   https://github.com/stsquad/qemu.git tags/pull-for-7.0-110122-1
>>>
>>> for you to fetch changes up to dbd30b7abee963f4fb08892a7d7f920bb76ece58:
>>>
>>>   linux-user: Remove the deprecated ppc64abi32 target (2022-01-11 13:00:53 +0000)
>>>
> <snip>
>> This seems to fail the ubuntu-18.04-s390x-all-linux-static job
>> with segfaults running linux-user binaries (not always the same
>> binary), eg:
>> https://gitlab.com/qemu-project/qemu/-/jobs/1968789446
>> https://gitlab.com/qemu-project/qemu/-/jobs/1968080419
>
> *sigh*
>
> So the regression is caused by:
>
>   linux-user: don't adjust base of found hole
>
> However it only occurs when pgb_static starts base at a low address. For
> example:
>
>   pgb_find_hole: base @ 13dd000 for 17432080 bytes
>   pgb_static: base @ 13dd000 for 17432080 bytes
>   Locating guest address space @ 0x13dd000
>
> fails whereas:
>
>   pgb_find_hole: base @ 41f97000 for 17432080 bytes      
>   pgb_static: base @ 41f97000 for 17432080 bytes        
>   Locating guest address space @ 0x41f97000      
>
> works.
>
> What I find confusing is why we end up with different addresses when
> both QEMU and the test binary are static allocations. However the
> varying allocation occurs before the change but without triggering the
> crash:

Continuing with debug dumps:

  read_self_maps: heap at 2445000->24ab000
  pgb_find_hole: brk @ 24ab000
  pgb_find_hole: start:24ab000 align_start:24ab000 end:3ffa0000000
  pgb_find_hole: after brk tweak align_start:424ab000
  Created 10 threads
  Done
  3, 0, PASS, 0.251649, 2, 3, -
  read_self_maps: heap at 2d14000->2d7a000
  pgb_find_hole: brk @ 2d7a000
  pgb_find_hole: start:13dd000 align_start:13dd000 end:2d14000
  4, -11, FALSE, 0.251602, 2, 4, -
  read_self_maps: heap at 1e6c000->1ed2000
  pgb_find_hole: brk @ 1ed2000
  pgb_find_hole: start:1ed2000 align_start:1ed2000 end:3ff90000000
  pgb_find_hole: after brk tweak align_start:41ed2000
  Created 10 threads
  Done
  5, 0, PASS, 0.253451, 3, 5, -
  read_self_maps: heap at 2c32000->2c98000
  pgb_find_hole: brk @ 2c98000
  pgb_find_hole: start:13dd000 align_start:13dd000 end:2c32000
  6, -11, FALSE, 0.251998, 3, 6, -
  read_self_maps: heap at 29f2000->2a58000
  pgb_find_hole: brk @ 2a58000
  pgb_find_hole: start:13dd000 align_start:13dd000 end:29f2000
  7, -11, FALSE, 0.251922, 3, 7, -
  read_self_maps: heap at 1b1f000->1b85000
  pgb_find_hole: brk @ 1b85000
  pgb_find_hole: start:1b85000 align_start:1b85000 end:3ff78000000
  pgb_find_hole: after brk tweak align_start:41b85000
  Created 10 threads
  Done
  8, 0, PASS, 0.251691, 4, 8, -

It looks like that we occasionally fit in bellow the heap and location
of brk but we aren't asking for enough space. I would like to get a core
dump of the failure because of course using gdb moves the maps around
enough that everything always works.
Alex Bennée Feb. 7, 2022, 10:20 a.m. UTC | #4
Thomas Huth <thuth@redhat.com> writes:

> On 12/01/2022 12.27, Alex Bennée wrote:
>> From: Thomas Huth <thuth@redhat.com>
>> It's likely broken, and nobody cared for picking it up again
>> during the deprecation phase, so let's remove this now.
>> Since this is the last entry in deprecated_targets_list, remove
>> the related code in the configure script, too.
>> Signed-off-by: Thomas Huth <thuth@redhat.com>
>
>  Hi Alex!
>
> What happened to this patch here? Seems like it got lost in v2 of the
> pull request?

Looks like... I'll include it in the PR I'll roll later today.

>
>  Thomas