[v1,00/24] fix building of tests/tcg

Message ID 20180410193919.28026-1-alex.bennee@linaro.org
Headers show
Series
  • fix building of tests/tcg
Related show

Message

Alex Bennée April 10, 2018, 7:38 p.m.
Hi,

We've talked about fixing this up for a long time and there have been
several RFC patches sent attempting to do that. This is yet another
RFC but hopefully shows a way forward which can build on our docker
support but also allow users to specify their own guest cross
compilers.

So far I've built arm, aarch64, ppc64 and s390x but adding support for
additional guests is simple, especially if we already have a docker
cross compile setup.

I've weeded out any tests that linked with QEMU (testpath) and reduced
the expectations of any tests/tcg test to just having access to libc.
This is the lowest common denominator for anything that can reasonably
be called a cross compiler.

The tests are all aimed at linux-user targets. We could probably come
up with something for building bare-metal softmmu tests but I suspect
that might be better served by using a different mechanism to import
existing tests into our build (e.g. kvm-unit-tests).

Let's breakdown the series:

configure:

As we need make magic we move the detection of docker and cross
compilers into configure. If any cross compiler is detected for a
given target we set CROSS_CC_GUEST in each config-target.mak. A
functioning cross compiler takes precedence over the docker fallback.
I think this makes sense for individual target sub-maintainers as they
likely already have a cross compile setup.

For docker we only set HAVE_USER_DOCKER in config-host.mak if the user
can run docker without sudo. We still need match up a docker image and
compiler once we build. This is handled by
tests/tcg/$(ARCH)/Makefile.include.

docker:

I've extended Fam's original patches to allow a bit more flexibility
for the "cc" command. I'm not overly wedded to using the docker.py
wrapper, we could just construct the command directly in Make if we
wanted to.

tests/tcg:

The top of the tree has been cleared out and everything moved into
sub-directories. There is a new multiarch sub-directory which is built
for every linux-user guest that has compiler support. Additional tests
are then included from tests/tcg/$(ARCH)/*.c or manually
added/modified by tests/tcg/$(ARCH)/Makefile.target.

Makefile:

As we are building tests/tcg for each target so Makefile.target now
invokes a sub-make with tests/tcg/Makefile while in the appropriate
build directory $(ARCH)-linux-user/tests/. The check-tcg and clean-tcg
targets unroll into all the configured FOO-linux-user targets and will
build the tests if cross compilation is available, otherwise the build
is skipped without failing the make.

Running tests:

Currently this is done manually from the build directory:

  ./qemu-arm tests/hello-arm

However once we have everything converted it shouldn't be too hard to
plumb into the normal make check sequence.

So what do people think? Is this a viable way to go forward?

Alex Bennée (20):
  configure: add test for docker availability
  configure: add support for --cross-cc-FOO
  configure: move i386_cc to cross_cc_i386
  docker: extend "cc" command to accept compiler
  docker: allow "cc" command to run in user context
  docker: Makefile.include introduce DOCKER_SCRIPT
  tests/tcg: move architecture independent tests into subdir
  tests/tcg: move i386 specific tests into subdir
  tests/tcg/i386: move test-i386-sse.c to tests/tcg/x86_64/test-sse.c
  tests/tcg/i386: fix test-i386
  tests/tcg/i368: fix hello-i386
  tests/tcg/i386: fix test-i386-fprem
  tests/tcg: move ARM specific tests into subdir
  tests/tcg/arm: fix hello-arm
  tests/tcg: move MIPS specific tests into subdir
  tests/tcg: enable building for s390x
  tests/tcg: enable building for ppc64
  tests/tcg/Makefile: update to be called from Makefile.target
  Makefile.target: add (clean-)guest-tests targets
  tests/Makefile.include: add (clean-)check-tcg targets

Fam Zheng (4):
  Makefile: Rename TARGET_DIRS to TARGET_LIST
  docker: Add "cc" subcommand
  tests/tcg/multiarch: Build fix for linux-test
  tests/tcg/i386: Build fix for hello-i386

 Makefile                                           |  20 +--
 Makefile.target                                    |   5 +
 configure                                          |  92 +++++++++-
 scripts/create_config                              |   2 +-
 tests/Makefile.include                             |  21 ++-
 tests/docker/Makefile.include                      |  10 +-
 tests/docker/docker.py                             |  40 +++++
 tests/tcg/Makefile                                 | 185 +++++----------------
 tests/tcg/Makefile.include                         |  79 +++++++++
 tests/tcg/README                                   |  69 +-------
 tests/tcg/arm/Makefile.target                      |   6 +
 tests/tcg/arm/README                               |  11 ++
 tests/tcg/{ => arm}/hello-arm.c                    |   0
 tests/tcg/{ => arm}/test-arm-iwmmxt.s              |   0
 tests/tcg/i386/Makefile.target                     |  35 ++++
 tests/tcg/i386/README                              |  38 +++++
 tests/tcg/{ => i386}/hello-i386.c                  |   1 +
 tests/tcg/{ => i386}/pi_10.com                     | Bin
 tests/tcg/{ => i386}/runcom.c                      |   0
 tests/tcg/{ => i386}/test-i386-code16.S            |   0
 tests/tcg/{ => i386}/test-i386-fprem.c             |   9 +-
 tests/tcg/{ => i386}/test-i386-muldiv.h            |   0
 tests/tcg/{ => i386}/test-i386-shift.h             |   0
 tests/tcg/{ => i386}/test-i386-vm86.S              |   0
 tests/tcg/{ => i386}/test-i386.c                   |   1 -
 tests/tcg/{ => i386}/test-i386.h                   |   0
 tests/tcg/mips/README                              |   7 +
 tests/tcg/{ => mips}/hello-mips.c                  |   0
 tests/tcg/multiarch/README                         |   1 +
 tests/tcg/{ => multiarch}/linux-test.c             |  68 +++-----
 tests/tcg/{ => multiarch}/sha1.c                   |   0
 tests/tcg/{ => multiarch}/test-mmap.c              |   0
 tests/tcg/{ => multiarch}/testthread.c             |   0
 tests/tcg/ppc64/Makefile.include                   |   2 +
 tests/tcg/s390x/Makefile.include                   |   2 +
 tests/tcg/test_path.c                              | 157 -----------------
 tests/tcg/{test-i386-ssse3.c => x86_64/test-sse.c} |   6 +-
 37 files changed, 424 insertions(+), 443 deletions(-)
 create mode 100644 tests/tcg/Makefile.include
 create mode 100644 tests/tcg/arm/Makefile.target
 create mode 100644 tests/tcg/arm/README
 rename tests/tcg/{ => arm}/hello-arm.c (100%)
 rename tests/tcg/{ => arm}/test-arm-iwmmxt.s (100%)
 create mode 100644 tests/tcg/i386/Makefile.target
 create mode 100644 tests/tcg/i386/README
 rename tests/tcg/{ => i386}/hello-i386.c (96%)
 rename tests/tcg/{ => i386}/pi_10.com (100%)
 rename tests/tcg/{ => i386}/runcom.c (100%)
 rename tests/tcg/{ => i386}/test-i386-code16.S (100%)
 rename tests/tcg/{ => i386}/test-i386-fprem.c (98%)
 rename tests/tcg/{ => i386}/test-i386-muldiv.h (100%)
 rename tests/tcg/{ => i386}/test-i386-shift.h (100%)
 rename tests/tcg/{ => i386}/test-i386-vm86.S (100%)
 rename tests/tcg/{ => i386}/test-i386.c (99%)
 rename tests/tcg/{ => i386}/test-i386.h (100%)
 create mode 100644 tests/tcg/mips/README
 rename tests/tcg/{ => mips}/hello-mips.c (100%)
 create mode 100644 tests/tcg/multiarch/README
 rename tests/tcg/{ => multiarch}/linux-test.c (91%)
 rename tests/tcg/{ => multiarch}/sha1.c (100%)
 rename tests/tcg/{ => multiarch}/test-mmap.c (100%)
 rename tests/tcg/{ => multiarch}/testthread.c (100%)
 create mode 100644 tests/tcg/ppc64/Makefile.include
 create mode 100644 tests/tcg/s390x/Makefile.include
 delete mode 100644 tests/tcg/test_path.c
 rename tests/tcg/{test-i386-ssse3.c => x86_64/test-sse.c} (93%)

-- 
2.16.2

Comments

no-reply@patchew.org April 10, 2018, 8:07 p.m. | #1
Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20180410193919.28026-1-alex.bennee@linaro.org
Subject: [Qemu-devel] [PATCH v1 00/24] fix building of tests/tcg

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
    echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
    if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
        failed=1
        echo
    fi
    n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]               patchew/20180410193919.28026-1-alex.bennee@linaro.org -> patchew/20180410193919.28026-1-alex.bennee@linaro.org
Switched to a new branch 'test'
90c03a56a7 tests/Makefile.include: add (clean-)check-tcg targets
4d10ecddd9 Makefile.target: add (clean-)guest-tests targets
419d213b73 tests/tcg/Makefile: update to be called from Makefile.target
42375e278e tests/tcg: enable building for ppc64
48844c4a0b tests/tcg: enable building for s390x
30148808c5 tests/tcg: move MIPS specific tests into subdir
2e1ca97890 tests/tcg/arm: fix hello-arm
7e47cb78d1 tests/tcg: move ARM specific tests into subdir
ae14cce45a tests/tcg/i386: fix test-i386-fprem
fff9127a52 tests/tcg/i368: fix hello-i386
32defdca05 tests/tcg/i386: fix test-i386
f29d705818 tests/tcg/i386: move test-i386-sse.c to tests/tcg/x86_64/test-sse.c
420cad3ca6 tests/tcg/i386: Build fix for hello-i386
b9c79e9220 tests/tcg: move i386 specific tests into subdir
d3d8bafea4 tests/tcg/multiarch: Build fix for linux-test
a0e4f21091 tests/tcg: move architecture independent tests into subdir
4b155dc50c docker: Makefile.include introduce DOCKER_SCRIPT
a8974c3037 docker: allow "cc" command to run in user context
2b879dfb5c docker: extend "cc" command to accept compiler
948d434a34 docker: Add "cc" subcommand
6464064827 Makefile: Rename TARGET_DIRS to TARGET_LIST
ca85716ee3 configure: move i386_cc to cross_cc_i386
d0197823de configure: add support for --cross-cc-FOO
ebd12f8f96 configure: add test for docker availability

=== OUTPUT BEGIN ===
Checking PATCH 1/24: configure: add test for docker availability...
Checking PATCH 2/24: configure: add support for --cross-cc-FOO...
Checking PATCH 3/24: configure: move i386_cc to cross_cc_i386...
Checking PATCH 4/24: Makefile: Rename TARGET_DIRS to TARGET_LIST...
Checking PATCH 5/24: docker: Add "cc" subcommand...
Checking PATCH 6/24: docker: extend "cc" command to accept compiler...
Checking PATCH 7/24: docker: allow "cc" command to run in user context...
Checking PATCH 8/24: docker: Makefile.include introduce DOCKER_SCRIPT...
Checking PATCH 9/24: tests/tcg: move architecture independent tests into subdir...
Checking PATCH 10/24: tests/tcg/multiarch: Build fix for linux-test...
ERROR: if this code is redundant consider removing it
#152: FILE: tests/tcg/multiarch/linux-test.c:323:
+#if 0

total: 1 errors, 0 warnings, 185 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 11/24: tests/tcg: move i386 specific tests into subdir...
Checking PATCH 12/24: tests/tcg/i386: Build fix for hello-i386...
ERROR: externs should be avoided in .c files
#20: FILE: tests/tcg/i386/hello-i386.c:23:
+void _start(void);

total: 1 errors, 0 warnings, 7 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 13/24: tests/tcg/i386: move test-i386-sse.c to tests/tcg/x86_64/test-sse.c...
ERROR: code indent should never use tabs
#36: FILE: tests/tcg/x86_64/test-sse.c:44:
+^I/* SSE4 popcnt r64, r/m64 */$

total: 1 errors, 0 warnings, 21 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 14/24: tests/tcg/i386: fix test-i386...
Checking PATCH 15/24: tests/tcg/i368: fix hello-i386...
Checking PATCH 16/24: tests/tcg/i386: fix test-i386-fprem...
Checking PATCH 17/24: tests/tcg: move ARM specific tests into subdir...
Checking PATCH 18/24: tests/tcg/arm: fix hello-arm...
Checking PATCH 19/24: tests/tcg: move MIPS specific tests into subdir...
Checking PATCH 20/24: tests/tcg: enable building for s390x...
Checking PATCH 21/24: tests/tcg: enable building for ppc64...
Checking PATCH 22/24: tests/tcg/Makefile: update to be called from Makefile.target...
Checking PATCH 23/24: Makefile.target: add (clean-)guest-tests targets...
Checking PATCH 24/24: tests/Makefile.include: add (clean-)check-tcg targets...
=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Philippe Mathieu-Daudé April 16, 2018, 10:37 a.m. | #2
On 04/10/2018 04:38 PM, Alex Bennée wrote:
> Hi,

> 

> We've talked about fixing this up for a long time and there have been

> several RFC patches sent attempting to do that. This is yet another

> RFC but hopefully shows a way forward which can build on our docker

> support but also allow users to specify their own guest cross

> compilers.

> 

> So far I've built arm, aarch64, ppc64 and s390x but adding support for

> additional guests is simple, especially if we already have a docker

> cross compile setup.


The current docker setup is limited to x86_64 hosts (amd64) and doesn't
work with armhf/arm64 hosts.

Although armhf (arm32v7) is easier for end-user, I believe at some point
it'll be useful to support aarch64 (arm64v8) hosts and eventually s390x.

> I've weeded out any tests that linked with QEMU (testpath) and reduced

> the expectations of any tests/tcg test to just having access to libc.

> This is the lowest common denominator for anything that can reasonably

> be called a cross compiler.

> 

> The tests are all aimed at linux-user targets. We could probably come

> up with something for building bare-metal softmmu tests but I suspect

> that might be better served by using a different mechanism to import

> existing tests into our build (e.g. kvm-unit-tests).

> 

> Let's breakdown the series:

> 

> configure:

> 

> As we need make magic we move the detection of docker and cross

> compilers into configure. If any cross compiler is detected for a

> given target we set CROSS_CC_GUEST in each config-target.mak. A

> functioning cross compiler takes precedence over the docker fallback.

> I think this makes sense for individual target sub-maintainers as they

> likely already have a cross compile setup.

> 

> For docker we only set HAVE_USER_DOCKER in config-host.mak if the user

> can run docker without sudo. We still need match up a docker image and

> compiler once we build. This is handled by

> tests/tcg/$(ARCH)/Makefile.include.

> 

> docker:

> 

> I've extended Fam's original patches to allow a bit more flexibility

> for the "cc" command. I'm not overly wedded to using the docker.py

> wrapper, we could just construct the command directly in Make if we

> wanted to.

> 

> tests/tcg:

> 

> The top of the tree has been cleared out and everything moved into

> sub-directories. There is a new multiarch sub-directory which is built

> for every linux-user guest that has compiler support. Additional tests

> are then included from tests/tcg/$(ARCH)/*.c or manually

> added/modified by tests/tcg/$(ARCH)/Makefile.target.

> 

> Makefile:

> 

> As we are building tests/tcg for each target so Makefile.target now

> invokes a sub-make with tests/tcg/Makefile while in the appropriate

> build directory $(ARCH)-linux-user/tests/. The check-tcg and clean-tcg

> targets unroll into all the configured FOO-linux-user targets and will

> build the tests if cross compilation is available, otherwise the build

> is skipped without failing the make.

> 

> Running tests:

> 

> Currently this is done manually from the build directory:

> 

>   ./qemu-arm tests/hello-arm

> 

> However once we have everything converted it shouldn't be too hard to

> plumb into the normal make check sequence.

> 

> So what do people think? Is this a viable way to go forward?

> 

> Alex Bennée (20):

>   configure: add test for docker availability

>   configure: add support for --cross-cc-FOO

>   configure: move i386_cc to cross_cc_i386

>   docker: extend "cc" command to accept compiler

>   docker: allow "cc" command to run in user context

>   docker: Makefile.include introduce DOCKER_SCRIPT

>   tests/tcg: move architecture independent tests into subdir

>   tests/tcg: move i386 specific tests into subdir

>   tests/tcg/i386: move test-i386-sse.c to tests/tcg/x86_64/test-sse.c

>   tests/tcg/i386: fix test-i386

>   tests/tcg/i368: fix hello-i386

>   tests/tcg/i386: fix test-i386-fprem

>   tests/tcg: move ARM specific tests into subdir

>   tests/tcg/arm: fix hello-arm

>   tests/tcg: move MIPS specific tests into subdir

>   tests/tcg: enable building for s390x

>   tests/tcg: enable building for ppc64

>   tests/tcg/Makefile: update to be called from Makefile.target

>   Makefile.target: add (clean-)guest-tests targets

>   tests/Makefile.include: add (clean-)check-tcg targets

> 

> Fam Zheng (4):

>   Makefile: Rename TARGET_DIRS to TARGET_LIST

>   docker: Add "cc" subcommand

>   tests/tcg/multiarch: Build fix for linux-test

>   tests/tcg/i386: Build fix for hello-i386

> 

>  Makefile                                           |  20 +--

>  Makefile.target                                    |   5 +

>  configure                                          |  92 +++++++++-

>  scripts/create_config                              |   2 +-

>  tests/Makefile.include                             |  21 ++-

>  tests/docker/Makefile.include                      |  10 +-

>  tests/docker/docker.py                             |  40 +++++

>  tests/tcg/Makefile                                 | 185 +++++----------------

>  tests/tcg/Makefile.include                         |  79 +++++++++

>  tests/tcg/README                                   |  69 +-------

>  tests/tcg/arm/Makefile.target                      |   6 +

>  tests/tcg/arm/README                               |  11 ++

>  tests/tcg/{ => arm}/hello-arm.c                    |   0

>  tests/tcg/{ => arm}/test-arm-iwmmxt.s              |   0

>  tests/tcg/i386/Makefile.target                     |  35 ++++

>  tests/tcg/i386/README                              |  38 +++++

>  tests/tcg/{ => i386}/hello-i386.c                  |   1 +

>  tests/tcg/{ => i386}/pi_10.com                     | Bin

>  tests/tcg/{ => i386}/runcom.c                      |   0

>  tests/tcg/{ => i386}/test-i386-code16.S            |   0

>  tests/tcg/{ => i386}/test-i386-fprem.c             |   9 +-

>  tests/tcg/{ => i386}/test-i386-muldiv.h            |   0

>  tests/tcg/{ => i386}/test-i386-shift.h             |   0

>  tests/tcg/{ => i386}/test-i386-vm86.S              |   0

>  tests/tcg/{ => i386}/test-i386.c                   |   1 -

>  tests/tcg/{ => i386}/test-i386.h                   |   0

>  tests/tcg/mips/README                              |   7 +

>  tests/tcg/{ => mips}/hello-mips.c                  |   0

>  tests/tcg/multiarch/README                         |   1 +

>  tests/tcg/{ => multiarch}/linux-test.c             |  68 +++-----

>  tests/tcg/{ => multiarch}/sha1.c                   |   0

>  tests/tcg/{ => multiarch}/test-mmap.c              |   0

>  tests/tcg/{ => multiarch}/testthread.c             |   0

>  tests/tcg/ppc64/Makefile.include                   |   2 +

>  tests/tcg/s390x/Makefile.include                   |   2 +

>  tests/tcg/test_path.c                              | 157 -----------------

>  tests/tcg/{test-i386-ssse3.c => x86_64/test-sse.c} |   6 +-

>  37 files changed, 424 insertions(+), 443 deletions(-)

>  create mode 100644 tests/tcg/Makefile.include

>  create mode 100644 tests/tcg/arm/Makefile.target

>  create mode 100644 tests/tcg/arm/README

>  rename tests/tcg/{ => arm}/hello-arm.c (100%)

>  rename tests/tcg/{ => arm}/test-arm-iwmmxt.s (100%)

>  create mode 100644 tests/tcg/i386/Makefile.target

>  create mode 100644 tests/tcg/i386/README

>  rename tests/tcg/{ => i386}/hello-i386.c (96%)

>  rename tests/tcg/{ => i386}/pi_10.com (100%)

>  rename tests/tcg/{ => i386}/runcom.c (100%)

>  rename tests/tcg/{ => i386}/test-i386-code16.S (100%)

>  rename tests/tcg/{ => i386}/test-i386-fprem.c (98%)

>  rename tests/tcg/{ => i386}/test-i386-muldiv.h (100%)

>  rename tests/tcg/{ => i386}/test-i386-shift.h (100%)

>  rename tests/tcg/{ => i386}/test-i386-vm86.S (100%)

>  rename tests/tcg/{ => i386}/test-i386.c (99%)

>  rename tests/tcg/{ => i386}/test-i386.h (100%)

>  create mode 100644 tests/tcg/mips/README

>  rename tests/tcg/{ => mips}/hello-mips.c (100%)

>  create mode 100644 tests/tcg/multiarch/README

>  rename tests/tcg/{ => multiarch}/linux-test.c (91%)

>  rename tests/tcg/{ => multiarch}/sha1.c (100%)

>  rename tests/tcg/{ => multiarch}/test-mmap.c (100%)

>  rename tests/tcg/{ => multiarch}/testthread.c (100%)

>  create mode 100644 tests/tcg/ppc64/Makefile.include

>  create mode 100644 tests/tcg/s390x/Makefile.include

>  delete mode 100644 tests/tcg/test_path.c

>  rename tests/tcg/{test-i386-ssse3.c => x86_64/test-sse.c} (93%)

>