[RISU,v6,02/10] build-all-archs: support cross building via docker

Message ID 20170621154244.28309-3-alex.bennee@linaro.org
State New
Headers show
Series
  • Record/replay patches
Related show

Commit Message

Alex Bennée June 21, 2017, 3:42 p.m.
If we want to link to any other libraries we might find using simple
cross toolchains doesn't work so well. One way around this is to use a
dockerised cross-toolchain which then won't clash with your host
system. If the user specifies --use-docker the obvious will be done.

By default we use the QEMU projects qemu:debian-FOO-cross images as
RISU hackers are likely to be QEMU developers too. However any docker
tag can be passed on the command line.

If none of the docker images have usable compilers we fall back to
checking the host path.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>


---
v5
  - swapped with --static patch so this can be dropped if desired
---
 build-all-archs | 46 +++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 39 insertions(+), 7 deletions(-)

-- 
2.13.0

Comments

Philippe Mathieu-Daudé June 29, 2017, 1:07 p.m. | #1
Hi Alex,

On 06/21/2017 12:42 PM, Alex Bennée wrote:> By default we use the QEMU 
projects qemu:debian-FOO-cross images as
> RISU hackers are likely to be QEMU developers too. However any docker

> tag can be passed on the command line.


Any thought about starting to push images?
Alex Bennée June 29, 2017, 1:27 p.m. | #2
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> Hi Alex,

>

> On 06/21/2017 12:42 PM, Alex Bennée wrote:> By default we use the QEMU

> projects qemu:debian-FOO-cross images as

>> RISU hackers are likely to be QEMU developers too. However any docker

>> tag can be passed on the command line.

>

> Any thought about starting to push images?


I think it comes down to how much we trust Docker Hub and how much we
trust whoever does the pushing. Would it be an automatic thing? From
shippable?

--
Alex Bennée
Philippe Mathieu-Daudé June 29, 2017, 1:43 p.m. | #3
On Thu, Jun 29, 2017 at 10:27 AM, Alex Bennée <alex.bennee@linaro.org> wrote:
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

>> On 06/21/2017 12:42 PM, Alex Bennée wrote:> By default we use the QEMU

>> projects qemu:debian-FOO-cross images as

>>> RISU hackers are likely to be QEMU developers too. However any docker

>>> tag can be passed on the command line.

>>

>> Any thought about starting to push images?

>

> I think it comes down to how much we trust Docker Hub and how much we

> trust whoever does the pushing. Would it be an automatic thing? From

> shippable?


Surely automatic.

Docker Hub offer free builds but you have to provide a full Dockerfile
(which is generated in QEMU).
We might keep those files in the repo... I'll give this a try.
The other possibilities are Travis and Shippable with a bit more
complicated setup.

Patch hide | download patch | download mbox

diff --git a/build-all-archs b/build-all-archs
index 581a1b4..63918e5 100755
--- a/build-all-archs
+++ b/build-all-archs
@@ -11,9 +11,6 @@ 
 # Contributors:
 #     Peter Maydell (Linaro) - initial implementation
 
-# So we notice risugen failing even though it's in a pipeline
-set -o pipefail
-
 # Simple usage
 usage() {
     cat <<-EOF
@@ -21,7 +18,10 @@  usage() {
 
         Options include:
             --static               build a static binary
+            --use-docker[=tags]    use docker cross compile
 
+        If specifying docker the default will be to use the any
+        qemu:debian-FOO-cross targets available on your system.
 EOF
     exit 1
 }
@@ -37,6 +37,14 @@  while [[ "$1" = -* ]]; do
         --static)
             CONF="--static"
             ;;
+        --use-docker)
+            if [ -z "$arg" ]; then
+                default_tags=$(docker images qemu --format "{{.Repository}}:{{.Tag}}" | grep "\(arm\|power\).*cross$")
+                docker_tags=$(echo $default_tags | sed 's/\n/\s/g' )
+            else
+                docker_tags="$arg"
+            fi
+            ;;
         --help)
             usage
             ;;
@@ -48,10 +56,24 @@  done
 
 # Debian stretch and Ubuntu Xenial have cross compiler packages for
 # all of these:
-# gcc-arm-linux-gnueabihf gcc-aarch64-linux-gnu gcc-m68k-linux-gnu
-# gcc-powerpc64le-linux-gnu gcc-powerpc64-linux-gnu
+#   gcc-arm-linux-gnueabihf gcc-aarch64-linux-gnu gcc-m68k-linux-gnu
+#   gcc-powerpc64le-linux-gnu gcc-powerpc64-linux-gnu
+# If docker is enabled we just brute force the various images until we
+# can set the one that has a workable cross compiler.
+
+DOCKER_RUN="docker run --rm -t -u $(id -u) -v $(pwd):$(pwd) -w $(pwd)"
 
 program_exists() {
+    if [ ! -z "$docker_tags" ]; then
+        use_docker_tag=""
+        for tag in $docker_tags; do
+            if ${DOCKER_RUN} ${tag} /bin/bash -c "command -v $1 >/dev/null"; then
+                use_docker_tag=$tag
+                return
+            fi
+        done
+    fi
+
     command -v "$1" >/dev/null 2>&1
 }
 
@@ -62,19 +84,29 @@  for triplet in aarch64-linux-gnu arm-linux-gnueabihf m68k-linux-gnu \
     if ! program_exists "${triplet}-gcc"; then
         echo "Skipping ${triplet}: no compiler found"
         continue
+    else
+        echo "Building ${triplet} on ${use_docker_tag:-host}..."
     fi
 
     # Do a complete rebuild from scratch, because it's cheap enough.
     rm -rf build/${triplet}
     mkdir -p build/${triplet}
 
-    (cd build/${triplet} && CROSS_PREFIX="${triplet}-"  ../../configure ${CONF})
-    make -C build/${triplet} EXTRA_CFLAGS=-Werror
+    CONFIGURE="cd build/${triplet} && CROSS_PREFIX="${triplet}-"  ../../configure ${CONF}"
+    MAKE="make -C build/${triplet} EXTRA_CFLAGS=-Werror"
 
+    if [ -z "$use_docker_tag" ]; then
+        /bin/bash -c "${CONFIGURE}"
+        ${MAKE}
+    else
+        ${DOCKER_RUN} $use_docker_tag /bin/bash -c "${CONFIGURE}"
+        ${DOCKER_RUN} $use_docker_tag /bin/bash -c "${MAKE}"
+    fi
 done
 
 # Now run risugen for all architectures
 mkdir -p build/risuout
+set -o pipefail # detect failures in pipeline
 
 for f in *.risu; do
     echo "Running risugen on $f..."