diff mbox

[RFC,2/2] add debian-bootstrap.docker target

Message ID 1464272863-2285-3-git-send-email-alex.bennee@linaro.org
State New
Headers show

Commit Message

Alex Bennée May 26, 2016, 2:27 p.m. UTC 
Together with some changes to the docker script you can now build an
arbitrary architecture of Debian using debootstrap. To achieve this I
introduce the concept of a HOST_CMD in the docker config file. While
copying the file into workspace the HOST_CMD is run in the docker build
context. This allows debootstrap to set up its first stage before the
container is built.

To build a container you need a command line like:

  DEB_ARCH=armhf DEB_TYPE=testing \
    ./tests/docker/docker.py build --qemu=qemu-arm debian:armhf \
    ./tests/docker/dockerfiles/debian-bootstrap.docker

The DEB_ARCH/DEB_TYPE are expanded in the docker file by the HOST_CMD:

  HOST_CMD fakeroot debootstrap --variant=minbase --foreign \
    --arch=$DEB_ARCH $DEB_TYPE . \
    http://httpredir.debian.org/debian

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
 tests/docker/docker.py                           | 27 +++++++++++++++++-------
 tests/docker/dockerfiles/debian-bootstrap.docker | 22 +++++++++++++++++++
 2 files changed, 41 insertions(+), 8 deletions(-)
 create mode 100644 tests/docker/dockerfiles/debian-bootstrap.docker

-- 
2.7.4

Comments

Alex Bennée May 31, 2016, 3:27 p.m. UTC | #1 
Fam Zheng <famz@redhat.com> writes:

> On Thu, 05/26 15:27, Alex Bennée wrote:

>> Together with some changes to the docker script you can now build an

>> arbitrary architecture of Debian using debootstrap. To achieve this I

>> introduce the concept of a HOST_CMD in the docker config file. While

>> copying the file into workspace the HOST_CMD is run in the docker build

>> context. This allows debootstrap to set up its first stage before the

>> container is built.

>

> Could you instead introduce the concept of $IMAGE.pre file (in this case

> debian-bootstrap.pre, aside debian-bootstrap.docker), and exec it in

> docker.py?  It would be much more flexible, and we we wouldn't need to inject a

> custom directive to filter it out.


I'm ambivalent about that. To be honest this is a bit of a gap in
docker's image creation (or if there is a better more docker-y way of
doing things I couldn't find it). It does have the benefit of keeping
everything in one place.

We are copying the dockerfile to the build environment anyway so it
seemed natural to do the operation while copying across. How would
envison the .pre setup? A #! script we just exec inside the temp
directory we create?

>

> Fam



--
Alex Bennée
diff mbox

Patch

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index e9242f3..3ba3d4e 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -21,7 +21,7 @@  import uuid
 import argparse
 import tempfile
 import re
-from shutil import copyfile
+from shutil import copy
 
 def _text_checksum(text):
     """Calculate a digest string unique to the text content"""
@@ -46,7 +46,6 @@  def _find_user_binary(binary_name):
     for x in linux_user:
         check_path = "%s/%s/%s" % (top, x, binary_name)
         if os.path.isfile(check_path):
-            print ("found %s" % check_path)
             return check_path
     return None
 
@@ -58,7 +57,7 @@  def _copy_with_mkdir(src, root_dir, sub_path):
     except OSError:
         print "skipping %s" % (full_path)
 
-    copyfile(src, "%s/%s" % (full_path, os.path.basename(src)))
+    copy(src, "%s/%s" % (full_path, os.path.basename(src)))
 
 class Docker(object):
     """ Running Docker commands """
@@ -117,11 +116,23 @@  class Docker(object):
         tmp_dir = tempfile.mkdtemp(prefix="docker_build")
 
         # Copy the dockerfile into our work space
-        tmp = dockerfile + "\n" + \
-              "LABEL com.qemu.dockerfile-checksum=%s" % \
-              _text_checksum(dockerfile)
+        # line by line, stripping and executing HOST_CMDs
+        #
         tmp_df = tempfile.NamedTemporaryFile(dir=tmp_dir, suffix=".docker")
-        tmp_df.write(tmp)
+
+        for l in open(dockerfile).readlines():
+            m = re.match("HOST_CMD ", l)
+            if m:
+                print "l=%s" % (l)
+                cmd = l[m.end():]
+                r = subprocess.check_call(cmd, cwd=tmp_dir, shell=True)
+                tmp_df.write("# HOST_CMD %s# HOST_RES = %d\n" % (cmd, r))
+            else:
+                tmp_df.write(l)
+
+        tmp_df.write("\n")
+        tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
+                     _text_checksum(dockerfile))
         tmp_df.flush()
 
         # Do we want to copy QEMU into here?
@@ -210,7 +221,7 @@  class BuildCommand(SubCommand):
                 print "Image is up to date."
             return 0
 
-        dkr.build_image(tag, dockerfile, quiet=args.quiet, qemu=qbin, argv=argv)
+        dkr.build_image(tag, args.dockerfile, quiet=args.quiet, qemu=qbin, argv=argv)
         return 0
 
 class CleanCommand(SubCommand):
diff --git a/tests/docker/dockerfiles/debian-bootstrap.docker b/tests/docker/dockerfiles/debian-bootstrap.docker
new file mode 100644
index 0000000..44d107d
--- /dev/null
+++ b/tests/docker/dockerfiles/debian-bootstrap.docker
@@ -0,0 +1,22 @@ 
+# Create Debian Bootstrap Image
+#
+# This is intended to be pre-poluated by:
+#  - a first stage debootstrap
+#  - a native qemu-$arch that binfmt_misc will run
+FROM scratch
+
+# HOST_CMD is executed by docker.py while building the context
+HOST_CMD fakeroot debootstrap --variant=minbase --foreign --arch=$DEB_ARCH $DEB_TYPE . http://httpredir.debian.org/debian
+
+# Add everything from the context into the container
+ADD . /
+
+# Patch all mounts as docker already has stuff set up
+RUN sed -i 's/in_target mount/echo not for docker in_target mount/g' /debootstrap/functions
+
+# Run stage 2
+RUN /debootstrap/debootstrap --second-stage
+
+# At this point we can install additional packages if we want
+#RUN apt-get update
+#RUN apt-get dist-upgrade