From patchwork Sat Aug 17 05:11:25 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Fathi Boudra X-Patchwork-Id: 19276 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-yh0-f71.google.com (mail-yh0-f71.google.com [209.85.213.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 635DE246AB for ; Sat, 17 Aug 2013 05:11:28 +0000 (UTC) Received: by mail-yh0-f71.google.com with SMTP id f64sf1618592yha.2 for ; Fri, 16 Aug 2013 22:11:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-gm-message-state:delivered-to:mime-version:to:from:subject :message-id:date:reply-to:sender:errors-to:precedence :x-original-sender:x-original-authentication-results:mailing-list :list-id:list-post:list-help:list-archive:list-unsubscribe :content-type; bh=ZwfEGTbtxxjiEtlINMCPOKrHpDZKoQHnQGC/CcvXtnk=; b=LhjBj0fqqi6LZ+YkdIfM3TmUpmSgGIYXRoODaoZjdAR0ubU8oDSKmWVFKKLRkBHlsq XJF4ZZFILecK5AkInMrISCF+da4mod8QKRrEX5206zkvovRfSh52UBw+eE5qq8uijOdu Dkwoe+dqV0HG2qidM5L1yHqZjljf/JZpqenLvJV+Y1xIIAdX4+ymOmwx1q5kTXe79Zma YwsjkiEeqjAlBEhGVDztlVxbbf3liViudQeAbu22aX6HwhMURUBGShr6TkDiASkyeVF4 F0Ok4lNoNlcQGEW/cU0pvfEqgX46Oiy3/4+Lq3iU6h2rJGXudvheNopXB0u/eSWudS/C HUEg== X-Received: by 10.236.46.72 with SMTP id q48mr445752yhb.30.1376716287689; Fri, 16 Aug 2013 22:11:27 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.49.18.199 with SMTP id y7ls1013937qed.78.gmail; Fri, 16 Aug 2013 22:11:27 -0700 (PDT) X-Received: by 10.58.152.3 with SMTP id uu3mr4667258veb.16.1376716287545; Fri, 16 Aug 2013 22:11:27 -0700 (PDT) Received: from mail-ve0-f182.google.com (mail-ve0-f182.google.com [209.85.128.182]) by mx.google.com with ESMTPS id vv9si270208vec.5.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 16 Aug 2013 22:11:27 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.182 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.182; Received: by mail-ve0-f182.google.com with SMTP id m1so1934316ves.27 for ; Fri, 16 Aug 2013 22:11:27 -0700 (PDT) X-Gm-Message-State: ALoCoQlyEOBQbUXn9ciyqE7VbPgQRUELkse9DsMcRuu24U40bZmbDzHFHc3LWmBWYLydTueGksMf X-Received: by 10.58.235.69 with SMTP id uk5mr1357998vec.17.1376716287443; Fri, 16 Aug 2013 22:11:27 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.220.174.196 with SMTP id u4csp105024vcz; Fri, 16 Aug 2013 22:11:26 -0700 (PDT) X-Received: by 10.180.184.107 with SMTP id et11mr782277wic.60.1376716285945; Fri, 16 Aug 2013 22:11:25 -0700 (PDT) Received: from indium.canonical.com (indium.canonical.com. [91.189.90.7]) by mx.google.com with ESMTPS id ll10si337883wjb.109.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Fri, 16 Aug 2013 22:11:25 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of bounces@canonical.com designates 91.189.90.7 as permitted sender) client-ip=91.189.90.7; Received: from ackee.canonical.com ([91.189.89.26]) by indium.canonical.com with esmtp (Exim 4.71 #1 (Debian)) id 1VAYnN-0004I7-AR for ; Sat, 17 Aug 2013 05:11:25 +0000 Received: from ackee.canonical.com (localhost [127.0.0.1]) by ackee.canonical.com (Postfix) with ESMTP id 3EC2CE0513 for ; Sat, 17 Aug 2013 05:11:25 +0000 (UTC) MIME-Version: 1.0 X-Launchpad-Project: linaro-image-tools X-Launchpad-Branch: ~linaro-image-tools/linaro-image-tools/trunk X-Launchpad-Message-Rationale: Subscriber X-Launchpad-Branch-Revision-Number: 637 X-Launchpad-Notification-Type: branch-revision To: Linaro Patch Tracker From: noreply@launchpad.net Subject: [Branch ~linaro-image-tools/linaro-image-tools/trunk] Rev 637: Add initial support for Fedora rootfs Message-Id: <20130817051125.19010.88452.launchpad@ackee.canonical.com> Date: Sat, 17 Aug 2013 05:11:25 -0000 Reply-To: noreply@launchpad.net Sender: bounces@canonical.com Errors-To: bounces@canonical.com Precedence: list X-Generated-By: Launchpad (canonical.com); Revision="16723"; Instance="launchpad-lazr.conf" X-Launchpad-Hash: 1f73c8ce0f171750c8b0d327d2669f5e1fd2fe6a X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: noreply@launchpad.net X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.182 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Merge authors: Fathi Boudra (fboudra) Related merge proposals: https://code.launchpad.net/~fboudra/linaro-image-tools/initial-fedora-support/+merge/180492 proposed by: Fathi Boudra (fboudra) review: Approve - Fathi Boudra (fboudra) review: Needs Fixing - Milo Casagrande (milo) ------------------------------------------------------------ revno: 637 [merge] committer: Fathi Boudra branch nick: linaro-image-tools timestamp: Sat 2013-08-17 08:10:14 +0300 message: Add initial support for Fedora rootfs modified: linaro-hwpack-install linaro-media-create linaro_image_tools/media_create/chroot_utils.py linaro_image_tools/media_create/rootfs.py linaro_image_tools/media_create/tests/test_media_create.py linaro_image_tools/media_create/unpack_binary_tarball.py linaro_image_tools/utils.py --- lp:linaro-image-tools https://code.launchpad.net/~linaro-image-tools/linaro-image-tools/trunk You are subscribed to branch lp:linaro-image-tools. To unsubscribe from this branch go to https://code.launchpad.net/~linaro-image-tools/linaro-image-tools/trunk/+edit-subscription === modified file 'linaro-hwpack-install' --- linaro-hwpack-install 2013-02-21 14:40:45 +0000 +++ linaro-hwpack-install 2013-08-08 07:44:38 +0000 @@ -1,8 +1,7 @@ #!/bin/sh # linaro-hwpack-install - Install a Linaro Hardware Pack. -# This script is meant to run inside a chroot containing nothing other than -# ubuntu-minimal, so it must not depend on anything that's not in -# there. +# This script is meant to run inside a chroot. It must not depend on anything +# that's not in there. # TODO: When upgrading to a newer hwpack, make sure packages and apt sources # that are no longer needed are removed. @@ -44,6 +43,7 @@ SUPPORTED_FORMATS="1.0 2.0 3.0" # A space-separated list of hwpack formats. FLASH_KERNEL_SKIP="true" export FLASH_KERNEL_SKIP # skip attempting to run flash-kernel-hooks +DISTRIBUTION=`grep '^ID=' /etc/os-release | sed 's/ID=//'` sudo="sudo" if [ $(id -u) -eq 0 ]; then @@ -129,7 +129,7 @@ "Try using a newer version of $(basename $0)." # Check the architecture of the hwpack matches that of the host system. - if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then + if [ "x$EXTRACT_KERNEL_ONLY" = "xno" -a "$DISTRIBUTION" = "ubuntu" ]; then # TODO: create a generic way to identify the architecture, without depending on dpkg [ "$HWPACK_ARCH" = `dpkg --print-architecture` ] || \ die "Hardware pack architecture ($HWPACK_ARCH) does not match the host's architecture" @@ -269,14 +269,49 @@ # We assume the hwpack is always available at the rootfs ROOTFS_DIR=$(dirname $HWPACK_TARBALL) + DEB_DIR="${TEMP_DIR}/extracted" + mkdir -p ${DEB_DIR} + ls ${HWPACK_DIR}/pkgs/linux-[ih]*.deb | while read pkg; do echo "Extracting package `basename $pkg`" - dpkg-deb -x ${pkg} $ROOTFS_DIR - done + if [ "$DISTRIBUTION" = "ubuntu" ]; then + dpkg-deb -x ${pkg} $ROOTFS_DIR + elif [ "$DISTRIBUTION" = "fedora" ]; then + ar x ${pkg} + tar -xf data.tar.* -C ${DEB_DIR} + rm -f debian-binary control.tar.gz data.tar.* + fi + done + + ls ${HWPACK_DIR}/pkgs/arndale-pre-boot_*.deb | while read pkg; do + echo "Extracting package `basename $pkg`" + if [ "$DISTRIBUTION" = "ubuntu" ]; then + dpkg-deb -x ${pkg} $ROOTFS_DIR + elif [ "$DISTRIBUTION" = "fedora" ]; then + ar x ${pkg} + tar -xf data.tar.* -C ${DEB_DIR} + rm -f debian-binary control.tar.gz data.tar.* + fi + done + + if [ "$DISTRIBUTION" = "fedora" ]; then + cp -a ${DEB_DIR}/boot/* /boot/ + cp -a ${DEB_DIR}/lib/modules/* /usr/lib/modules/ + cp -a ${DEB_DIR}/lib/firmware/* /usr/lib/firmware/ + fi + + if [ -d "${HWPACK_DIR}/u_boot" ]; then + cp -a ${HWPACK_DIR}/u_boot/usr/lib/* /usr/lib + fi + + if [ -d "${HWPACK_DIR}/uefi" ]; then + cp -a ${HWPACK_DIR}/uefi/usr/lib/* /usr/lib + fi # manually generate modules.dep ls $ROOTFS_DIR/lib/modules | while read kernel; do depmod -b $ROOTFS_DIR ${kernel} || true + [ "$DISTRIBUTION" = "fedora" ] && dracut /boot/initrd.img-${kernel} ${kernel} done; } @@ -284,7 +319,7 @@ # Ensure our temp dir and apt sources are removed. echo -n "Cleaning up ..." rm -rf $TEMP_DIR - if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then + if [ "x$EXTRACT_KERNEL_ONLY" = "xno" -a "$DISTRIBUTION" = "ubuntu" ]; then rm -f /usr/sbin/policy-rc.d mv -f /sbin/start-stop-daemon.REAL /sbin/start-stop-daemon if [ -x /sbin/initctl.REAL ]; then @@ -318,7 +353,7 @@ setup_hwpack # In case we only care about the kernel, don't mess up with the system -if [ "x$EXTRACT_KERNEL_ONLY" = "xno" ]; then +if [ "x$EXTRACT_KERNEL_ONLY" = "xno" -a "$DISTRIBUTION" = "ubuntu" ]; then setup_apt_sources setup_ubuntu_rootfs install_deb_packages === modified file 'linaro-media-create' --- linaro-media-create 2013-08-16 09:30:08 +0000 +++ linaro-media-create 2013-08-17 05:10:14 +0000 @@ -181,12 +181,6 @@ # The binary image is in the new live format. filesystem_dir = 'binary/boot/filesystem.dir' - # if not a debian compatible system, just extract the kernel packages - extract_kpkgs = False - if not path_in_tarfile_exists( - os.path.join(filesystem_dir, 'etc', 'debian_version'), args.binary): - extract_kpkgs = True - ROOTFS_DIR = os.path.join(BIN_DIR, filesystem_dir) try: @@ -209,6 +203,24 @@ unpack_binary_tarball(args.binary, BIN_DIR) + # if compatible system, extract all packages + os_release_id = 'linux' + os_release_file = '%s/etc/os-release' % ROOTFS_DIR + if os.path.exists(os_release_file): + for line in open(os_release_file): + if line.startswith('ID='): + os_release_id = line[(len('ID=')):] + os_release_id = os_release_id.strip('\"\n') + break + + if os_release_id == 'debian' or os_release_id == 'ubuntu' or \ + os.path.exists('%s/etc/debian_version' % ROOTFS_DIR): + extract_kpkgs = False + elif os_release_id == 'fedora': + extract_kpkgs = False + else: + extract_kpkgs = True + hwpacks = args.hwpacks lmc_dir = os.path.dirname(__file__) if lmc_dir == '': @@ -263,6 +275,6 @@ populate_rootfs(ROOTFS_DIR, ROOT_DISK, root_partition, args.rootfs, rootfs_id, create_swap, str(args.swap_file), board_config.mmc_device_id, board_config.mmc_part_offset, - board_config) + os_release_id, board_config) logger.info("Done creating Linaro image on %s" % media.path) === modified file 'linaro_image_tools/media_create/chroot_utils.py' --- linaro_image_tools/media_create/chroot_utils.py 2013-02-17 13:53:41 +0000 +++ linaro_image_tools/media_create/chroot_utils.py 2013-08-08 07:44:38 +0000 @@ -105,7 +105,7 @@ print "Installing (linaro-hwpack-install) %s in target rootfs." % ( hwpack_basename) - # Get infromation required by linaro-hwpack-install + # Get information required by linaro-hwpack-install with HardwarepackHandler([hwpack_file]) as hwpack: version, _ = hwpack.get_field("version") architecture, _ = hwpack.get_field("architecture") === modified file 'linaro_image_tools/media_create/rootfs.py' --- linaro_image_tools/media_create/rootfs.py 2013-02-17 13:53:41 +0000 +++ linaro_image_tools/media_create/rootfs.py 2013-08-08 07:44:38 +0000 @@ -43,7 +43,8 @@ def populate_rootfs(content_dir, root_disk, partition, rootfs_type, rootfs_id, should_create_swap, swap_size, - mmc_device_id, partition_offset, board_config=None): + mmc_device_id, partition_offset, os_release_id, + board_config=None): """Populate the rootfs and make the necessary tweaks to make it usable. This consists of: @@ -86,13 +87,14 @@ append_to_fstab(root_disk, fstab_additions) - print "\nCreating /etc/flash-kernel.conf\n" - create_flash_kernel_config( - root_disk, mmc_device_id, 1 + partition_offset) + if os_release_id == 'debian' or os_release_id == 'ubuntu': + print "\nCreating /etc/flash-kernel.conf\n" + create_flash_kernel_config( + root_disk, mmc_device_id, 1 + partition_offset) - if board_config is not None: - print "\nUpdating /etc/network/interfaces\n" - update_network_interfaces(root_disk, board_config) + if board_config is not None: + print "\nUpdating /etc/network/interfaces\n" + update_network_interfaces(root_disk, board_config) def update_network_interfaces(root_disk, board_config): @@ -136,7 +138,8 @@ not be world-readable. """ p = cmd_runner.run( - ['find', directory, '-maxdepth', '1', '-mindepth', '1'], + ['find', directory, '-maxdepth', '1', '-mindepth', '1', + '!', '-name', 'lost+found'], stdout=subprocess.PIPE, as_root=True) stdout, _ = p.communicate() return stdout.split() === modified file 'linaro_image_tools/media_create/tests/test_media_create.py' --- linaro_image_tools/media_create/tests/test_media_create.py 2013-08-16 08:01:22 +0000 +++ linaro_image_tools/media_create/tests/test_media_create.py 2013-08-17 05:05:41 +0000 @@ -3673,7 +3673,8 @@ populate_rootfs( contents_dir, root_disk, partition='/dev/rootfs', rootfs_type='ext3', rootfs_id='UUID=uuid', should_create_swap=True, - swap_size=100, mmc_device_id=0, partition_offset=0) + swap_size=100, mmc_device_id=0, partition_offset=0, + os_release_id='ubuntu', board_config=None) self.assertEqual( ['UUID=uuid / ext3 errors=remount-ro 0 1', === modified file 'linaro_image_tools/media_create/unpack_binary_tarball.py' --- linaro_image_tools/media_create/unpack_binary_tarball.py 2012-06-13 14:11:28 +0000 +++ linaro_image_tools/media_create/unpack_binary_tarball.py 2013-08-08 07:44:38 +0000 @@ -29,8 +29,11 @@ def unpack_binary_tarball(tarball, unpack_dir, as_root=True): + extract_opt = '-xf' + if tarball.endswith('.xz'): + extract_opt = '-Jxf' proc = cmd_runner.run( - ['tar', '--numeric-owner', '-C', unpack_dir, '-xf', tarball], + ['tar', '--numeric-owner', '-C', unpack_dir, extract_opt, tarball], as_root=as_root) proc.wait() return proc.returncode === modified file 'linaro_image_tools/utils.py' --- linaro_image_tools/utils.py 2013-08-06 07:25:33 +0000 +++ linaro_image_tools/utils.py 2013-08-08 07:44:38 +0000 @@ -89,11 +89,21 @@ def path_in_tarfile_exists(path, tar_file): exists = True try: - tarinfo = tarfile.open(tar_file, 'r:gz') + tarinfo = tarfile.open(tar_file, 'r:*') tarinfo.getmember(path) tarinfo.close() except KeyError: exists = False + except (tarfile.ReadError, tarfile.CompressionError): + exists = False + # Fallback to tar command + cmd = ['tar', '-tf', tar_file, '--wildcards', '*' + path] + proc = cmd_runner.run(cmd, + stdout=open('/dev/null', 'w'), + stderr=open('/dev/null', 'w')) + proc.wait() + if proc.returncode == 0: + exists = True finally: return exists