From patchwork Tue Jul 12 15:00:34 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 71863 Delivered-To: patch@linaro.org Received: by 10.140.29.52 with SMTP id a49csp526395qga; Tue, 12 Jul 2016 08:10:19 -0700 (PDT) X-Received: by 10.55.170.20 with SMTP id t20mr3719176qke.162.1468336218999; Tue, 12 Jul 2016 08:10:18 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id m33si2768156qtm.130.2016.07.12.08.10.18 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 12 Jul 2016 08:10:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41145 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bMzKE-0001tr-34 for patch@linaro.org; Tue, 12 Jul 2016 11:10:18 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50231) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bMzB9-0008Lc-Nz for qemu-devel@nongnu.org; Tue, 12 Jul 2016 11:00:57 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bMzB4-0007ko-IV for qemu-devel@nongnu.org; Tue, 12 Jul 2016 11:00:55 -0400 Received: from mail-wm0-x22e.google.com ([2a00:1450:400c:c09::22e]:37844) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bMzB4-0007kd-9f for qemu-devel@nongnu.org; Tue, 12 Jul 2016 11:00:50 -0400 Received: by mail-wm0-x22e.google.com with SMTP id i5so29035106wmg.0 for ; Tue, 12 Jul 2016 08:00:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wyKuZMQGq9AczkNjak43T99JWsMy2nyXM6U8PQfal98=; b=ckfi6tQkYn+u1I0uJtcAcL7BaDpSMCj5E+BhBhtGgNtFDk6tbGRT02DW6pIdY6Sb1F ydctyDH74uFflaOnT1tjnLoUA+6PpkHes4+IDsM/j1T27ctAZ77BSZxU4HsqXSVCIoBJ yMAyrsSCBc4NVzT+/xb7ldkWVcp/i9to9hql0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wyKuZMQGq9AczkNjak43T99JWsMy2nyXM6U8PQfal98=; b=R8RWBg+fM/AhYvgUiq0BFt36Q+2yS+8FWKWgqDpB88kzGfB2bhNypfyH0RTLfKTfAt 5YHn+K5YRglBdf7z3S/vyHQlLn9/ehq20SHQJHpIOxeBGigpQdSdArY5Vcsg4Ar1P+xD LhzqnKNjU0pYcmt5jpMm3UuhiR5CmndxOa5ywLXmCzYM7uLKq8C3IweLB+Bs0NSBfkc9 QIDWEtnbAOwv+2SFfVbMZWXw3XwPWxEvpMizUEb45efcCkmDSe/PB5s/u5Z/tuMXbBQ6 R+D27X+79jrxV0AUTpNT5UhZd6z3uBxcplSS7YCYd9MLEeeFTfvPoUQzXMGdoLgF8DT2 IC2w== X-Gm-Message-State: ALyK8tLl2MfR+4utaLLmSXvDdkgNhTiJpP+M5d+zMl2yG/PCfJNTkq9h5kxN5psBPormSXUY X-Received: by 10.28.152.5 with SMTP id a5mr4241387wme.76.1468335648539; Tue, 12 Jul 2016 08:00:48 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id f187sm4103093wmf.15.2016.07.12.08.00.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jul 2016 08:00:46 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 12C693E0404; Tue, 12 Jul 2016 16:00:49 +0100 (BST) From: =?UTF-8?q?Alex=20Benn=C3=A9e?= To: qemu-devel@nongnu.org Date: Tue, 12 Jul 2016 16:00:34 +0100 Message-Id: <1468335639-24582-3-git-send-email-alex.bennee@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1468335639-24582-1-git-send-email-alex.bennee@linaro.org> References: <1468335639-24582-1-git-send-email-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22e Subject: [Qemu-devel] [PATCH v5 2/7] tests/docker/docker.py: support --include-executable X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , riku.voipio@linaro.org, famz@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" When passed the path to a binary we copy it and any linked libraries (if it is dynamically linked) into the docker build context. These can then be included by a dockerfile with the line: # Copy all of context into container ADD . / This is mainly intended for setting up foreign architecture docker images which use qemu-$arch to do cross-architecture linux-user execution. It also relies on the host and guest file-system following reasonable multi-arch layouts so the copied libraries don't clash with the guest ones. Signed-off-by: Alex Bennée --- v2 - change name of option - require full path to executable - clean-up the copy code v3 - safely deal with static builds - quietly pass if makedirs not needed - split copy routine and so lib resolver --- tests/docker/docker.py | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) -- 2.7.4 diff --git a/tests/docker/docker.py b/tests/docker/docker.py index ae40bb3..96d906e 100755 --- a/tests/docker/docker.py +++ b/tests/docker/docker.py @@ -20,6 +20,7 @@ import atexit import uuid import argparse import tempfile +import re from shutil import copy, rmtree def _text_checksum(text): @@ -38,6 +39,54 @@ def _guess_docker_command(): raise Exception("Cannot find working docker command. Tried:\n%s" % \ commands_txt) +def _copy_with_mkdir(src, root_dir, sub_path): + """Copy src into root_dir, creating sub_path as needed.""" + dest_dir = os.path.normpath("%s/%s" % (root_dir, sub_path)) + try: + os.makedirs(dest_dir) + except OSError: + # we can safely ignore already created directories + pass + + dest_file = "%s/%s" % (dest_dir, os.path.basename(src)) + copy(src, dest_file) + + +def _get_so_libs(executable): + """Return a list of libraries associated with an executable. + + The paths may be symbolic links which would need to be resolved to + ensure theright data is copied.""" + + libs = [] + ldd_re = re.compile(r"(/.*/)(\S*)") + try: + ldd_output = subprocess.check_output(["ldd", executable]) + for line in ldd_output.split("\n"): + search = ldd_re.search(line) + if search and len(search.groups()) == 2: + so_path = search.groups()[0] + so_lib = search.groups()[1] + libs.append("%s/%s" % (so_path, so_lib)) + except subprocess.CalledProcessError: + print "%s had no associated libraries (static build?)" % (executable) + + return libs + +def _copy_binary_with_libs(src, dest_dir): + """Copy a binary executable and all its dependant libraries. + + This does rely on the host file-system being fairly multi-arch + aware so the file don't clash with the guests layout.""" + + _copy_with_mkdir(src, dest_dir, "/usr/bin") + + libs = _get_so_libs(src) + if libs: + for l in libs: + so_path = os.path.dirname(l) + _copy_with_mkdir(l , dest_dir, so_path) + class Docker(object): """ Running Docker commands """ def __init__(self): @@ -151,6 +200,10 @@ class BuildCommand(SubCommand): """ Build docker image out of a dockerfile. Arguments: """ name = "build" def args(self, parser): + parser.add_argument("--include-executable", "-e", + help="""Specify a binary that will be copied to the + container together with all its dependent + libraries""") parser.add_argument("tag", help="Image Tag") parser.add_argument("dockerfile", @@ -168,6 +221,11 @@ class BuildCommand(SubCommand): # Create a docker context directory for the build docker_dir = tempfile.mkdtemp(prefix="docker_build") + # Do we include a extra binary? + if args.include_executable: + _copy_binary_with_libs(args.include_executable, + docker_dir) + dkr.build_image(tag, docker_dir, dockerfile, quiet=args.quiet, argv=argv)