From patchwork Tue Nov 13 18:46:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Maydell X-Patchwork-Id: 151004 Delivered-To: patches@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp4757692ljp; Tue, 13 Nov 2018 10:46:47 -0800 (PST) X-Google-Smtp-Source: AJdET5dHPNV4McmuoGDNa3+QI1xG/CVZdQdjfv75THVn3TYqHfAqqVmMgIMQn4csVcl5WmpHDLI0 X-Received: by 2002:adf:80c8:: with SMTP id 66-v6mr6067807wrl.57.1542134807192; Tue, 13 Nov 2018 10:46:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542134807; cv=none; d=google.com; s=arc-20160816; b=jUOEyammXMrJoBL+o3MKSkPZPoUHP8VAPsyAwUHnxqrc76GGaKsMez9/w79tKDWWQ2 Qq5FK2FdyLqtwkGtIwtCerKrui26m5OVAPjOdvzulkDtaJ4MJ7vB+/EBWjqON9IJ1o7g 5jnG7L8MYegHYv3WYMG8poJrVt5ImKQxTzKB3Z7NNdQXolWH3efeDXkmH4W+nVaRYv43 VXSwSaio7GjrK5jQT9Cxl/B2EV1c4xNv96JlI2+7wJ6VlCKBQ603vrkjJzAMSisNhzNB PxT8ydzJbP/oUjiurz/pR1FNrut7wmy/QIvLPCsL5LCA3lk72CTGLNY9eDeLQQ21EdMF OpDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Ags+AJUoI7iifXmfZDxtNRqvTP4zKdwa2uX9A0FTgRc=; b=KqDFTpJ/qJKMvIIc7Qa8cFJm1ghyAR4is3kwSu/8SScHLw7pbAk/KhTXEXclcO3V3z D077wukLeVC2PrVhD+dgjxCwAO5ebAM8PCvIFmAbN8cbmuaAClXvoN37AsTNUz7usgyR Lro1XyiaaE8Y2tJYvAHBBDYmm27ikJHYLPGWMzH0Q1NIJIxA7y2NkjUY4HbqvhCQfgaC RByXSXpr2xiPI8ozziutul4FaKY1Hdt7UJiJyazbga+K6zYv9QDWLCgzkG6h/njqRI3i Jy1krh6XH31H7jpGbyqaVCs04H9DthrqaFBCcT0jOf/tjNfHDLJy1qF37lWCb8AU6+Ez nYfA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of pm215@archaic.org.uk designates 2001:8b0:1d0::2 as permitted sender) smtp.mailfrom=pm215@archaic.org.uk; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from orth.archaic.org.uk (orth.archaic.org.uk. [2001:8b0:1d0::2]) by mx.google.com with ESMTPS id y16-v6si16845060wrr.124.2018.11.13.10.46.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 13 Nov 2018 10:46:47 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of pm215@archaic.org.uk designates 2001:8b0:1d0::2 as permitted sender) client-ip=2001:8b0:1d0::2; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of pm215@archaic.org.uk designates 2001:8b0:1d0::2 as permitted sender) smtp.mailfrom=pm215@archaic.org.uk; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from pm215 by orth.archaic.org.uk with local (Exim 4.89) (envelope-from ) id 1gMdi2-0007Dl-GY; Tue, 13 Nov 2018 18:46:46 +0000 From: Peter Maydell To: qemu-devel@nongnu.org Cc: patches@linaro.org, Paolo Bonzini , =?utf-8?q?Alex_Benn=C3=A9e?= , Fam Zheng , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Markus Armbruster Subject: [PATCH 2/2] scripts/coverity-scan: Add Docker support Date: Tue, 13 Nov 2018 18:46:41 +0000 Message-Id: <20181113184641.4492-3-peter.maydell@linaro.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181113184641.4492-1-peter.maydell@linaro.org> References: <20181113184641.4492-1-peter.maydell@linaro.org> MIME-Version: 1.0 Add support for running the Coverity Scan tools inside a Docker container rather than directly on the host system. Signed-off-by: Peter Maydell --- scripts/coverity-scan/coverity-scan.docker | 120 +++++++++++++++++++++ scripts/coverity-scan/run-coverity-scan | 58 ++++++++++ 2 files changed, 178 insertions(+) create mode 100644 scripts/coverity-scan/coverity-scan.docker -- 2.19.1 diff --git a/scripts/coverity-scan/coverity-scan.docker b/scripts/coverity-scan/coverity-scan.docker new file mode 100644 index 00000000000..81f69459954 --- /dev/null +++ b/scripts/coverity-scan/coverity-scan.docker @@ -0,0 +1,120 @@ +# syntax=docker/dockerfile:1.0.0-experimental +# +# Docker setup for running the "Coverity Scan" tools over the source +# tree and uploading them to the website, as per +# https://scan.coverity.com/projects/qemu/builds/new +# We do this on a fixed config (currently Fedora 28 with a known +# set of dependencies and a configure command that enables a specific +# set of options) so that random changes don't result in our accidentally +# dropping some files from the scan. +# The work of actually doing the build is handled by the +# run-coverity-scan script. + + +FROM fedora:28 +ENV PACKAGES \ + alsa-lib-devel \ + bc \ + bison \ + bluez-libs-devel \ + brlapi-devel \ + bzip2 \ + bzip2-devel \ + ccache \ + clang \ + curl \ + cyrus-sasl-devel \ + device-mapper-multipath-devel \ + findutils \ + flex \ + gcc \ + gcc-c++ \ + gettext \ + git \ + glib2-devel \ + glusterfs-api-devel \ + gnutls-devel \ + gtk3-devel \ + hostname \ + libaio-devel \ + libasan \ + libattr-devel \ + libcap-devel \ + libcap-ng-devel \ + libcurl-devel \ + libepoxy-devel \ + libfdt-devel \ + libgbm-devel \ + libiscsi-devel \ + libjpeg-devel \ + libnfs-devel \ + libpng-devel \ + librbd-devel \ + libseccomp-devel \ + libssh2-devel \ + libubsan \ + libudev-devel \ + libusbx-devel \ + libxml2-devel \ + llvm \ + lzo-devel \ + make \ + mingw32-bzip2 \ + mingw32-curl \ + mingw32-glib2 \ + mingw32-gmp \ + mingw32-gnutls \ + mingw32-gtk3 \ + mingw32-libjpeg-turbo \ + mingw32-libpng \ + mingw32-libssh2 \ + mingw32-libtasn1 \ + mingw32-nettle \ + mingw32-pixman \ + mingw32-pkg-config \ + mingw32-SDL2 \ + mingw64-bzip2 \ + mingw64-curl \ + mingw64-glib2 \ + mingw64-gmp \ + mingw64-gnutls \ + mingw64-gtk3 \ + mingw64-libjpeg-turbo \ + mingw64-libpng \ + mingw64-libssh2 \ + mingw64-libtasn1 \ + mingw64-nettle \ + mingw64-pixman \ + mingw64-pkg-config \ + mingw64-SDL2 \ + ncurses-devel \ + nettle-devel \ + nss-devel \ + numactl-devel \ + perl \ + pixman-devel \ + pulseaudio-libs-devel \ + python3 \ + PyYAML \ + rdma-core-devel \ + SDL2-devel \ + snappy-devel \ + sparse \ + spice-server-devel \ + systemtap-sdt-devel \ + tar \ + usbredir-devel \ + virglrenderer-devel \ + vte3-devel \ + wget \ + which \ + xen-devel \ + xfsprogs-devel \ + zlib-devel +ENV QEMU_CONFIGURE_OPTS --python=/usr/bin/python3 + +RUN dnf install -y $PACKAGES +RUN rpm -q $PACKAGES | sort > /packages.txt +ENV COVERITY_TOOL_BASE=/coverity-tools +COPY run-coverity-scan run-coverity-scan +RUN --mount=type=secret,id=coverity.token,required ./run-coverity-scan --update-tools-only --tokenfile /run/secrets/coverity.token diff --git a/scripts/coverity-scan/run-coverity-scan b/scripts/coverity-scan/run-coverity-scan index 99495b04501..e89316c090d 100755 --- a/scripts/coverity-scan/run-coverity-scan +++ b/scripts/coverity-scan/run-coverity-scan @@ -29,6 +29,7 @@ # Command line options: # --dry-run : run the tools, but don't actually do the upload +# --docker : create and work inside a docker container # --update-tools-only : update the cached copy of the tools, but don't run them # --tokenfile : file to read Coverity token from # --version ver : specify version being analyzed (default: ask git) @@ -122,6 +123,7 @@ update_coverity_tools () { # Check user-provided environment variables and arguments DRYRUN=no UPDATE_ONLY=no +DOCKER=no while [ "$#" -ge 1 ]; do case "$1" in @@ -169,6 +171,10 @@ while [ "$#" -ge 1 ]; do SRCDIR="$1" shift ;; + --docker) + DOCKER=yes + shift + ;; *) echo "Unexpected argument '$1'" exit 1 @@ -199,6 +205,10 @@ PROJTOKEN="$COVERITY_TOKEN" PROJNAME=QEMU TARBALL=cov-int.tar.xz +if [ "$UPDATE_ONLY" = yes ] && [ "$DOCKER" = yes ]; then + echo "Combining --docker and --update-only is not supported" + exit 1 +fi if [ "$UPDATE_ONLY" = yes ]; then # Just do the tools update; we don't need to check whether @@ -229,6 +239,54 @@ if [ -z "$COVERITY_EMAIL" ]; then COVERITY_EMAIL="$(git config user.email)" fi +# Run ourselves inside docker if that's what the user wants +if [ "$DOCKER" = yes ]; then + # build docker container including the coverity-scan tools + # Put the Coverity token into a temporary file that only + # we have read access to, and then pass it to docker build + # using --secret. This requires at least Docker 18.09. + # Mostly what we are trying to do here is ensure we don't leak + # the token into the Docker image. + umask 077 + SECRETDIR=$(mktemp -d) + if [ -z "$SECRETDIR" ]; then + echo "Failed to create temporary directory" + exit 1 + fi + trap 'rm -rf "$SECRETDIR"' INT TERM EXIT + echo "Created temporary directory $SECRETDIR" + SECRET="$SECRETDIR/token" + echo "$COVERITY_TOKEN" > "$SECRET" + echo "Building docker container..." + # TODO: This re-downloads the tools every time, rather than + # caching and reusing the image produced with the downloaded tools. + # Not sure why. + # TODO: how do you get 'docker build' to print the output of the + # commands it is running to its stdout? This would be useful for debug. + DOCKER_BUILDKIT=1 docker build -t coverity-scanner \ + --secret id=coverity.token,src="$SECRET" \ + -f scripts/coverity-scan/coverity-scan.docker \ + scripts/coverity-scan + echo "Archiving sources to be analyzed..." + ./scripts/archive-source.sh "$SECRETDIR/qemu-sources.tgz" + (cd "$SECRETDIR" && mkdir qemu && cd qemu && tar xvf ../qemu-sources.tgz) + if [ "$DRYRUN" = yes ]; then + DRYRUNARG=--dry-run + fi + echo "Running scanner..." + # Arrange for this docker run to get access to the sources with -v. + # We pass through all the configuration from the outer script to the inner. + docker run -it --env COVERITY_EMAIL --env COVERITY_BUILD_CMD \ + -v "$SECRETDIR:/work" coverity-scanner \ + ./run-coverity-scan --version "$VERSION" \ + --description "$DESCRIPTION" $DRYRUNARG --tokenfile /work/token \ + --srcdir /work/qemu + echo "Docker work complete." + exit 0 +fi + +# Otherwise, continue with the full build and upload process. + check_upload_permissions update_coverity_tools