mbox

[PULL,v2,00/36] Testing, docs, semihosting and plugin updates

Message ID 20230202160109.2061994-1-alex.bennee@linaro.org
State New
Headers show

Pull-request

https://gitlab.com/stsquad/qemu.git tags/pull-jan-omnibus-020223-1

Message

Alex Bennée Feb. 2, 2023, 4:01 p.m. UTC
The following changes since commit 026817fb69414c9d3909d8b1a209f90180d777d6:

  Merge tag 'python-pull-request' of https://gitlab.com/jsnow/qemu into staging (2023-02-01 16:15:56 +0000)

are available in the Git repository at:

  https://gitlab.com/stsquad/qemu.git tags/pull-jan-omnibus-020223-1

for you to fetch changes up to b3ca9646b9a5c44dfd110c5db9b4a8b8497de34e:

  gitlab: cut even more from cross-win64-system build (2023-02-02 13:25:27 +0000)

----------------------------------------------------------------
Testing, docs, semihosting and plugin updates

  - update playbooks for custom runners
  - add section timing support to gitlab
  - upgrade fedora images to 37
  - purge perl from the build system and deps
  - disable unstable tests in CI
  - improve intro, emulation and semihosting docs
  - semihosting bug fix and O_BINARY default
  - add memory-sve test
  - fix some races in qht
  - improve plugin handling of memory helpers
  - optimise plugin hooks
  - fix some plugin deadlocks
  - reduce win64-cross build time by dropping some targets

----------------------------------------------------------------
Alex Bennée (13):
      scripts/ci: update gitlab-runner playbook to use latest runner
      gitlab: just use plain --cc=clang for custom runner build
      tests/unit: drop hacky race avoidance in test-io-channel-command
      gitlab: add lsan suppression file to workaround tcmalloc issues
      tests/docker: drop debian-tricore-cross's partial status
      tests/tcg: skip the vma-pthread test on CI
      gitlab: wrap up test results for custom runners
      docs: add hotlinks to about preface text
      docs: add a new section to outline emulation support
      semihosting: add semihosting section to the docs
      docs: add an introduction to the system docs
      tests/tcg: add memory-sve test for aarch64
      gitlab: cut even more from cross-win64-system build

Emilio Cota (9):
      cpu: free cpu->tb_jmp_cache with RCU
      util/qht: add missing atomic_set(hashes[i])
      thread: de-const qemu_spin_destroy
      util/qht: use striped locks under TSAN
      plugins: make qemu_plugin_user_exit's locking order consistent with fork_start's
      plugins: fix optimization in plugin_gen_disable_mem_helpers
      translator: always pair plugin_gen_insn_{start, end} calls
      tcg: exclude non-memory effecting helpers from instrumentation
      cpu-exec: assert that plugin_mem_cbs is NULL after execution

Evgeny Iakovlev (1):
      semihosting: add O_BINARY flag in host_open for NT compatibility

Keith Packard (1):
      semihosting: Write back semihosting data before completion callback

Marc-André Lureau (7):
      build-sys: fix crlf-ending C code
      .gitlab-ci.d/windows: do not disable opengl
      meson: replace Perl usage with Python
      docs: drop texinfo options
      Update lcitool and fedora to 37
      lcitool: drop perl from QEMU project/dependencies
      lcitool: drop texinfo from QEMU project/dependencies

Mark Cave-Ayland (1):
      gitlab: add FF_SCRIPT_SECTIONS for timings

Philippe Mathieu-Daudé (1):
      tests/docker: Install flex in debian-tricore-cross

Richard Henderson (2):
      tests/tcg: Use SIGKILL for timeout
      plugins: Iterate on cb_lists in qemu_plugin_user_exit

Thomas Huth (1):
      MAINTAINERS: Fix the entry for tests/tcg/nios2

 MAINTAINERS                                        |   2 +-
 docs/about/emulation.rst                           | 190 ++++++++++++++++++
 docs/about/index.rst                               |  17 +-
 docs/conf.py                                       |  13 --
 docs/devel/tcg-plugins.rst                         |   2 +
 docs/interop/live-block-operations.rst             |   2 +
 docs/interop/qemu-qmp-ref.rst                      |   2 +
 docs/system/arm/emulation.rst                      |   2 +
 docs/system/index.rst                              |   4 +-
 docs/system/introduction.rst                       | 220 +++++++++++++++++++++
 docs/system/multi-process.rst                      |   2 +
 docs/system/quickstart.rst                         |  21 --
 docs/tools/index.rst                               |   2 +
 docs/user/index.rst                                |   2 +
 meson.build                                        |   2 +-
 accel/tcg/plugin-helpers.h                         |   4 +-
 accel/tcg/tb-jmp-cache.h                           |   1 +
 include/qemu/plugin.h                              |  11 ++
 include/qemu/thread.h                              |   5 +-
 include/tcg/tcg.h                                  |   2 +
 accel/tcg/cpu-exec.c                               |   5 +-
 accel/tcg/plugin-gen.c                             |  26 ++-
 accel/tcg/translator.c                             |  15 +-
 cpu.c                                              |  11 +-
 plugins/core.c                                     |  23 ++-
 semihosting/syscalls.c                             |  28 +--
 tcg/tcg.c                                          |   6 +-
 tests/unit/test-io-channel-command.c               |  14 +-
 util/qht.c                                         |  97 +++++++--
 .gitlab-ci.d/base.yml                              |   5 +
 .gitlab-ci.d/buildtest.yml                         |   1 +
 .gitlab-ci.d/cirrus/freebsd-12.vars                |   2 +-
 .gitlab-ci.d/cirrus/freebsd-13.vars                |   2 +-
 .gitlab-ci.d/cirrus/macos-12.vars                  |   2 +-
 .gitlab-ci.d/crossbuilds.yml                       |   4 +-
 .gitlab-ci.d/custom-runners.yml                    |  11 ++
 .gitlab-ci.d/custom-runners/ubuntu-20.04-s390x.yml |  13 +-
 .../custom-runners/ubuntu-22.04-aarch32.yml        |   2 +-
 .../custom-runners/ubuntu-22.04-aarch64.yml        |  15 +-
 .gitlab-ci.d/windows.yml                           |   5 +-
 qemu-options.hx                                    |  25 +--
 scripts/ci/setup/build-environment.yml             |   1 -
 scripts/ci/setup/gitlab-runner.yml                 |  56 ++----
 scripts/ci/setup/vars.yml.template                 |   2 -
 scripts/oss-fuzz/lsan_suppressions.txt             |   2 +
 scripts/shaderinclude.pl                           |  16 --
 scripts/shaderinclude.py                           |  26 +++
 tests/docker/Makefile.include                      |   1 -
 tests/docker/dockerfiles/alpine.docker             |   2 -
 tests/docker/dockerfiles/centos8.docker            |   2 -
 tests/docker/dockerfiles/debian-amd64-cross.docker |   4 +-
 tests/docker/dockerfiles/debian-amd64.docker       |   2 -
 tests/docker/dockerfiles/debian-arm64-cross.docker |   4 +-
 tests/docker/dockerfiles/debian-armel-cross.docker |   4 +-
 tests/docker/dockerfiles/debian-armhf-cross.docker |   4 +-
 .../dockerfiles/debian-mips64el-cross.docker       |   4 +-
 .../docker/dockerfiles/debian-mipsel-cross.docker  |   4 +-
 .../docker/dockerfiles/debian-ppc64el-cross.docker |   4 +-
 tests/docker/dockerfiles/debian-s390x-cross.docker |   4 +-
 tests/docker/dockerfiles/debian-toolchain.docker   |   1 -
 .../docker/dockerfiles/debian-tricore-cross.docker |   2 +-
 tests/docker/dockerfiles/fedora-win32-cross.docker |   6 +-
 tests/docker/dockerfiles/fedora-win64-cross.docker |   6 +-
 tests/docker/dockerfiles/fedora.docker             |   6 +-
 tests/docker/dockerfiles/opensuse-leap.docker      |   2 -
 tests/docker/dockerfiles/ubuntu2004.docker         |   2 -
 tests/lcitool/libvirt-ci                           |   2 +-
 tests/lcitool/projects/qemu.yml                    |   2 -
 tests/lcitool/refresh                              |   6 +-
 tests/qapi-schema/meson.build                      |   7 +-
 tests/tcg/Makefile.target                          |   4 +-
 tests/tcg/aarch64/Makefile.softmmu-target          |   7 +
 tests/tcg/aarch64/system/boot.S                    |   3 +-
 tests/tcg/multiarch/Makefile.target                |   9 +
 tests/vm/centos.aarch64                            |   2 +-
 75 files changed, 754 insertions(+), 273 deletions(-)
 create mode 100644 docs/about/emulation.rst
 create mode 100644 docs/system/introduction.rst
 delete mode 100644 docs/system/quickstart.rst
 create mode 100644 scripts/oss-fuzz/lsan_suppressions.txt
 delete mode 100644 scripts/shaderinclude.pl
 create mode 100644 scripts/shaderinclude.py

Comments

Peter Maydell Feb. 3, 2023, 9:30 a.m. UTC | #1
On Thu, 2 Feb 2023 at 16:01, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> The following changes since commit 026817fb69414c9d3909d8b1a209f90180d777d6:
>
>   Merge tag 'python-pull-request' of https://gitlab.com/jsnow/qemu into staging (2023-02-01 16:15:56 +0000)
>
> are available in the Git repository at:
>
>   https://gitlab.com/stsquad/qemu.git tags/pull-jan-omnibus-020223-1
>
> for you to fetch changes up to b3ca9646b9a5c44dfd110c5db9b4a8b8497de34e:
>
>   gitlab: cut even more from cross-win64-system build (2023-02-02 13:25:27 +0000)
>
> ----------------------------------------------------------------
> Testing, docs, semihosting and plugin updates
>
>   - update playbooks for custom runners
>   - add section timing support to gitlab
>   - upgrade fedora images to 37
>   - purge perl from the build system and deps
>   - disable unstable tests in CI
>   - improve intro, emulation and semihosting docs
>   - semihosting bug fix and O_BINARY default
>   - add memory-sve test
>   - fix some races in qht
>   - improve plugin handling of memory helpers
>   - optimise plugin hooks
>   - fix some plugin deadlocks
>   - reduce win64-cross build time by dropping some targets


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/8.0
for any user-visible changes.

-- PMM
Philippe Mathieu-Daudé Feb. 3, 2023, 5:38 p.m. UTC | #2
On 2/2/23 17:01, Alex Bennée wrote:
> The following changes since commit 026817fb69414c9d3909d8b1a209f90180d777d6:
> 
>    Merge tag 'python-pull-request' of https://gitlab.com/jsnow/qemu into staging (2023-02-01 16:15:56 +0000)
> 
> are available in the Git repository at:
> 
>    https://gitlab.com/stsquad/qemu.git tags/pull-jan-omnibus-020223-1
> 
> for you to fetch changes up to b3ca9646b9a5c44dfd110c5db9b4a8b8497de34e:
> 
>    gitlab: cut even more from cross-win64-system build (2023-02-02 13:25:27 +0000)
> 
> ----------------------------------------------------------------
> Testing, docs, semihosting and plugin updates
> 
>    - update playbooks for custom runners
>    - add section timing support to gitlab
>    - upgrade fedora images to 37
>    - purge perl from the build system and deps
>    - disable unstable tests in CI
>    - improve intro, emulation and semihosting docs
>    - semihosting bug fix and O_BINARY default
>    - add memory-sve test
>    - fix some races in qht
>    - improve plugin handling of memory helpers
>    - optimise plugin hooks
>    - fix some plugin deadlocks
>    - reduce win64-cross build time by dropping some targets
> 
> ----------------------------------------------------------------
FTR I'm now seeing this warning on Darwin/Aarch64:

C compiler for the host machine: clang (clang 14.0.0 "Apple clang 
version 14.0.0 (clang-1400.0.29.202)")
C linker for the host machine: clang ld64 820.1
Host machine cpu family: aarch64
Host machine cpu: aarch64
...
[1/3253] Linking target tests/plugin/libbb.dylib
ld: warning: -undefined dynamic_lookup may not work with chained fixups
[2/3253] Linking target tests/plugin/libinsn.dylib
ld: warning: -undefined dynamic_lookup may not work with chained fixups
[3/3253] Linking target tests/plugin/libmem.dylib
ld: warning: -undefined dynamic_lookup may not work with chained fixups
[4/3253] Linking target tests/plugin/libsyscall.dylib
ld: warning: -undefined dynamic_lookup may not work with chained fixups
Philippe Mathieu-Daudé Feb. 23, 2023, 12:59 p.m. UTC | #3
On 3/2/23 18:38, Philippe Mathieu-Daudé wrote:
> On 2/2/23 17:01, Alex Bennée wrote:

>> ----------------------------------------------------------------
>> Testing, docs, semihosting and plugin updates

>> ----------------------------------------------------------------
> FTR I'm now seeing this warning on Darwin/Aarch64:
> 
> C compiler for the host machine: clang (clang 14.0.0 "Apple clang 
> version 14.0.0 (clang-1400.0.29.202)")
> C linker for the host machine: clang ld64 820.1
> Host machine cpu family: aarch64
> Host machine cpu: aarch64
> ...
> [1/3253] Linking target tests/plugin/libbb.dylib
> ld: warning: -undefined dynamic_lookup may not work with chained fixups
> [2/3253] Linking target tests/plugin/libinsn.dylib
> ld: warning: -undefined dynamic_lookup may not work with chained fixups
> [3/3253] Linking target tests/plugin/libmem.dylib
> ld: warning: -undefined dynamic_lookup may not work with chained fixups
> [4/3253] Linking target tests/plugin/libsyscall.dylib
> ld: warning: -undefined dynamic_lookup may not work with chained fixups

Googling, someone suggest to use --linkopt=-Wl,-no_fixup_chains:
https://github.com/bazelbuild/bazel/issues/16413#issuecomment-1270603343

This doesn't show up on GitLab which uses:

C compiler for the host machine: cc (clang 13.1.6 "Apple clang version 
13.1.6 (clang-1316.0.21.2.5)")
C linker for the host machine: cc ld64 764
Host machine cpu family: aarch64
Host machine cpu: aarch64

I guess I'll just ignore these warnings.
Akihiko Odaki Feb. 24, 2023, 4:51 a.m. UTC | #4
On 2023/02/23 21:59, Philippe Mathieu-Daudé wrote:
> On 3/2/23 18:38, Philippe Mathieu-Daudé wrote:
>> On 2/2/23 17:01, Alex Bennée wrote:
> 
>>> ----------------------------------------------------------------
>>> Testing, docs, semihosting and plugin updates
> 
>>> ----------------------------------------------------------------
>> FTR I'm now seeing this warning on Darwin/Aarch64:
>>
>> C compiler for the host machine: clang (clang 14.0.0 "Apple clang 
>> version 14.0.0 (clang-1400.0.29.202)")
>> C linker for the host machine: clang ld64 820.1
>> Host machine cpu family: aarch64
>> Host machine cpu: aarch64
>> ...
>> [1/3253] Linking target tests/plugin/libbb.dylib
>> ld: warning: -undefined dynamic_lookup may not work with chained fixups
>> [2/3253] Linking target tests/plugin/libinsn.dylib
>> ld: warning: -undefined dynamic_lookup may not work with chained fixups
>> [3/3253] Linking target tests/plugin/libmem.dylib
>> ld: warning: -undefined dynamic_lookup may not work with chained fixups
>> [4/3253] Linking target tests/plugin/libsyscall.dylib
>> ld: warning: -undefined dynamic_lookup may not work with chained fixups
> 
> Googling, someone suggest to use --linkopt=-Wl,-no_fixup_chains:
> https://github.com/bazelbuild/bazel/issues/16413#issuecomment-1270603343
> 
> This doesn't show up on GitLab which uses:
> 
> C compiler for the host machine: cc (clang 13.1.6 "Apple clang version 
> 13.1.6 (clang-1316.0.21.2.5)")
> C linker for the host machine: cc ld64 764
> Host machine cpu family: aarch64
> Host machine cpu: aarch64
> 
> I guess I'll just ignore these warnings.

I think this problem is tracked at:
https://gitlab.com/qemu-project/qemu/-/issues/1244

It is caused by linker flags Meson adds. For Meson, it is tracked at:
https://github.com/mesonbuild/meson/issues/10894
Philippe Mathieu-Daudé Feb. 24, 2023, 8:16 a.m. UTC | #5
On 24/2/23 05:51, Akihiko Odaki wrote:
> On 2023/02/23 21:59, Philippe Mathieu-Daudé wrote:
>> On 3/2/23 18:38, Philippe Mathieu-Daudé wrote:
>>> On 2/2/23 17:01, Alex Bennée wrote:
>>
>>>> ----------------------------------------------------------------
>>>> Testing, docs, semihosting and plugin updates
>>
>>>> ----------------------------------------------------------------
>>> FTR I'm now seeing this warning on Darwin/Aarch64:
>>>
>>> C compiler for the host machine: clang (clang 14.0.0 "Apple clang 
>>> version 14.0.0 (clang-1400.0.29.202)")
>>> C linker for the host machine: clang ld64 820.1
>>> Host machine cpu family: aarch64
>>> Host machine cpu: aarch64
>>> ...
>>> [1/3253] Linking target tests/plugin/libbb.dylib
>>> ld: warning: -undefined dynamic_lookup may not work with chained fixups
>>> [2/3253] Linking target tests/plugin/libinsn.dylib
>>> ld: warning: -undefined dynamic_lookup may not work with chained fixups
>>> [3/3253] Linking target tests/plugin/libmem.dylib
>>> ld: warning: -undefined dynamic_lookup may not work with chained fixups
>>> [4/3253] Linking target tests/plugin/libsyscall.dylib
>>> ld: warning: -undefined dynamic_lookup may not work with chained fixups
>>
>> Googling, someone suggest to use --linkopt=-Wl,-no_fixup_chains:
>> https://github.com/bazelbuild/bazel/issues/16413#issuecomment-1270603343
>>
>> This doesn't show up on GitLab which uses:
>>
>> C compiler for the host machine: cc (clang 13.1.6 "Apple clang version 
>> 13.1.6 (clang-1316.0.21.2.5)")
>> C linker for the host machine: cc ld64 764
>> Host machine cpu family: aarch64
>> Host machine cpu: aarch64
>>
>> I guess I'll just ignore these warnings.
> 
> I think this problem is tracked at:
> https://gitlab.com/qemu-project/qemu/-/issues/1244
> 
> It is caused by linker flags Meson adds. For Meson, it is tracked at:
> https://github.com/mesonbuild/meson/issues/10894

Indeed. Thank you Akihiko :)