From patchwork Mon Apr 14 10:38:14 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 28310 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ve0-f200.google.com (mail-ve0-f200.google.com [209.85.128.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 412C52036A for ; Mon, 14 Apr 2014 10:40:06 +0000 (UTC) Received: by mail-ve0-f200.google.com with SMTP id oy12sf29727830veb.3 for ; Mon, 14 Apr 2014 03:40:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id :mime-version:cc:subject:precedence:list-id:list-unsubscribe :list-post:list-help:list-subscribe:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list :list-archive:content-type:content-transfer-encoding; bh=poXD67DBxQ94Gn+4lfUQnGs0tUPXnM1FvfE2itU0QBo=; b=L/yv16w06rOl5ff786HJjGIeCI24KUk+AjUzHNQbkjYT3d6RuEYno3wAfODV9JIE2z AB/VljptXSIIlhYQq26cfFqbAOlSrwzbTqMbmP2rBkdjdMNvQRb3Pv2B2IGv8xdhKw6L knJndY8wJez6H5CJUgksM610PBaWb10TBuTyzmJRR4rXL0JzZ7g/h8qod4snUMsuLgZL 6AAoNLBN75oPfj7nl50x+Mlsb4RqVVJLHReRDTV0s0LtfEBQfVOdIraFvVIjT452ygSm QQSlbSsN16cAPmNFD+hBeFei5x7HuX0CSyPM2Ah2NqvwATGbzvkIdF5PjP9IYeRTjmFQ Uq2A== X-Gm-Message-State: ALoCoQmgZMJvnQnl0LENvOjhU0f2MeEthHba/ynoy2kaGpuIKPxfnCnL70U3pkYg1Rp1O4uhex3o X-Received: by 10.236.138.113 with SMTP id z77mr16169644yhi.25.1397472005656; Mon, 14 Apr 2014 03:40:05 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.91.131 with SMTP id z3ls2581258qgd.59.gmail; Mon, 14 Apr 2014 03:40:05 -0700 (PDT) X-Received: by 10.220.190.197 with SMTP id dj5mr24595320vcb.19.1397472005501; Mon, 14 Apr 2014 03:40:05 -0700 (PDT) Received: from mail-ve0-f174.google.com (mail-ve0-f174.google.com [209.85.128.174]) by mx.google.com with ESMTPS id tz5si2673667vdc.7.2014.04.14.03.40.05 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 14 Apr 2014 03:40:05 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.174 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.174; Received: by mail-ve0-f174.google.com with SMTP id oz11so7338547veb.33 for ; Mon, 14 Apr 2014 03:40:05 -0700 (PDT) X-Received: by 10.220.159.4 with SMTP id h4mr36428257vcx.1.1397472005376; Mon, 14 Apr 2014 03:40:05 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp136826vcb; Mon, 14 Apr 2014 03:40:04 -0700 (PDT) X-Received: by 10.220.59.65 with SMTP id k1mr2366290vch.22.1397472004789; Mon, 14 Apr 2014 03:40:04 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id p8si2679005vef.122.2014.04.14.03.40.04 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 14 Apr 2014 03:40:04 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xen.org designates 50.57.142.19 as permitted sender) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WZeHO-0002Mo-0i; Mon, 14 Apr 2014 10:38:22 +0000 Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WZeHM-0002MP-7J for xen-devel@lists.xen.org; Mon, 14 Apr 2014 10:38:20 +0000 Received: from [85.158.137.68:25054] by server-14.bemta-3.messagelabs.com id 18/BE-30903-B9ABB435; Mon, 14 Apr 2014 10:38:19 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-12.tower-31.messagelabs.com!1397471896!5341075!1 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.11.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 23699 invoked from network); 14 Apr 2014 10:38:17 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-12.tower-31.messagelabs.com with RC4-SHA encrypted SMTP; 14 Apr 2014 10:38:17 -0000 X-IronPort-AV: E=Sophos;i="4.97,856,1389744000"; d="scan'208";a="119668010" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 14 Apr 2014 10:38:16 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.3.123.3; Mon, 14 Apr 2014 06:38:15 -0400 Received: from spare.cam.xci-test.com ([10.80.2.80] helo=kazak.uk.xensource.com.) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WZeHH-0003u8-2n; Mon, 14 Apr 2014 10:38:15 +0000 From: Ian Campbell To: , Date: Mon, 14 Apr 2014 11:38:14 +0100 Message-ID: <1397471894-19513-1-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.9.0 MIME-Version: 1.0 X-DLP: MIA1 Cc: Dario Faggioli , Jim Fehlig , Ian Campbell Subject: [Xen-devel] [PATCH osstest.git v3] Add libvirt build and test jobs. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.174 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: The ts-libvirt-build, sg-run-job and mfi-common changes have been tested in standalone mode. The ts-xen-install change has been tested in standalone mode, it is expected that the rest of test-*-*-libvirt will fail horribly at this time. The remaining changes are intended to introduce a libvirt push gate and have been tested as far as my limited ability to invoke cr-daily-branch by hand is able, but I am sure they are incomplete in some way or other. The libvirt.git initscript is horribly redhat specific, and the debian package one is large and complex, so ts-libvirt-build inserts its own simple initscript. The standalone wrapper is updated to handle path_libvirt in set-paths. libvirt.git's autogen.sh involves initialising/cloning a git submodule. AIUI the way git submodules work they are effectively versioned by the repo which references them and so there are no push gate implications here. Signed-off-by: Ian Campbell Acked-by: Ian Jackson Cc: Jim Fehlig Cc: Dario Faggioli --- v3: Disable uml, lxc and vbox backends at build time Create test job on armhf as well as x86 Don't run non-libvirt tests in libvirt flights. v2: Add a test job, which will clearly fail. Make ts-xen-install DTRT though --- Osstest/TestSupport.pm | 7 ++- ap-common | 5 ++ ap-fetch-version | 4 ++ ap-fetch-version-old | 5 ++ ap-print-url | 3 + ap-push | 5 ++ cr-daily-branch | 8 +++ cri-common | 1 + make-flight | 7 +++ mfi-common | 15 ++++- sg-run-job | 5 ++ standalone | 2 +- ts-libvirt-build | 159 +++++++++++++++++++++++++++++++++++++++++++++++++ ts-xen-install | 5 +- 14 files changed, 227 insertions(+), 4 deletions(-) create mode 100755 ts-libvirt-build diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm index b11eb17..87b8b2e 100644 --- a/Osstest/TestSupport.pm +++ b/Osstest/TestSupport.pm @@ -1764,7 +1764,12 @@ our %toolstacks= Command => 'xl', CfgPathVar => 'cfgpath', RestoreNeedsConfig => 1, - } + }, + 'libvirt' => { + NewDaemons => [qw(libvirtd)], + Dom0MemFixed => 1, + Command => 'virsh', + }, ); sub toolstack () { diff --git a/ap-common b/ap-common index cb0cfff..73bb90c 100644 --- a/ap-common +++ b/ap-common @@ -33,6 +33,10 @@ : ${TREEVCS_LINUX:=git} +: ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git} +: ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git} +: ${TREEVCS_LIBVIRT:=git} + : ${TREE_LINUXFIRMWARE:=git://xenbits.xen.org/osstest/linux-firmware.git} : ${PUSH_TREE_LINUXFIRMWARE:=$XENBITS:/home/osstest/ext/linux-firmware.git} : ${UPSTREAM_TREE_LINUXFIRMWARE:=$GIT_KERNEL_ORG/pub/scm/linux/kernel/git/firmware/linux-firmware.git} @@ -56,6 +60,7 @@ fi : ${LOCALREV_XEN:=daily-cron.$branch} : ${LOCALREV_LINUX:=daily-cron.$branch} +: ${LOCALREV_LIBVIRT:=daily-cron.$branch} : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27} diff --git a/ap-fetch-version b/ap-fetch-version index aaf884d..2990462 100755 --- a/ap-fetch-version +++ b/ap-fetch-version @@ -69,6 +69,10 @@ linuxfirmware) repo_tree_rev_fetch_git linuxfirmware \ $UPSTREAM_TREE_LINUXFIRMWARE master daily-cron.$branch ;; +libvirt) + repo_tree_rev_fetch_git libvirt \ + $TREE_LIBVIRT master $LOCALREV_LIBVIRT + ;; osstest) if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then git fetch $HOME/testing.git pretest:ap-fetch >&2 diff --git a/ap-fetch-version-old b/ap-fetch-version-old index 21ff786..a32de0e 100755 --- a/ap-fetch-version-old +++ b/ap-fetch-version-old @@ -27,6 +27,7 @@ select_xenbranch : ${BASE_TAG_LINUX2639:=tested/2.6.39.x} : ${BASE_LOCALREV_LINUX:=daily-cron.$branch.old} +: ${BASE_LOCALREV_LIBVIRT:=daily-cron.$branch.old} : ${BASE_TREE_QEMU_UPSTREAM:=${TREE_QEMU_UPSTREAM/\/staging\//\/}} @@ -73,6 +74,10 @@ linuxfirmware) repo_tree_rev_fetch_git linuxfirmware \ $TREE_LINUXFIRMWARE master daily-cron-old.$branch ;; +libvirt) + repo_tree_rev_fetch_git libvirt \ + $TREE_LIBVIRT master $BASE_LOCALREV_LIBVIRT + ;; osstest) if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then git fetch -f $HOME/testing.git incoming:ap-fetch diff --git a/ap-print-url b/ap-print-url index f2812d0..2b55fce 100755 --- a/ap-print-url +++ b/ap-print-url @@ -46,6 +46,9 @@ linux-2.6.39) linuxfirmware) echo $UPSTREAM_TREE_LINUXFIRMWARE ;; +libvirt) + echo $TREE_LIBVIRT + ;; osstest) echo none:; ;; diff --git a/ap-push b/ap-push index dc0a166..af76ecc 100755 --- a/ap-push +++ b/ap-push @@ -32,6 +32,7 @@ select_xenbranch TREE_LINUX=$PUSH_TREE_LINUX TREE_QEMU_UPSTREAM=$XENBITS:/home/xen/git/qemu-upstream-${xenbranch#xen-}.git TREE_XEN=$PUSH_TREE_XEN +TREE_LIBVIRT=$PUSH_TREE_LIBVIRT if info_linux_tree "$branch"; then cd $repos/linux @@ -72,6 +73,10 @@ linux-xen-*) cd $repos/linux git push $TREE_LINUX $revision:refs/heads/$branch ;; +libvirt) + cd $repos/libvirt + git push $TREE_LIBVIRT $revision:master + ;; osstest) git push $HOME/testing.git $revision:incoming git push $XENBITS:/home/xen/git/osstest.git $revision:master diff --git a/cr-daily-branch b/cr-daily-branch index 02fef15..a13d75a 100755 --- a/cr-daily-branch +++ b/cr-daily-branch @@ -133,6 +133,10 @@ if [ "x$REVISION_LINUX_ARM" = x ]; then export REVISION_LINUX_ARM fi fi +if [ "x$REVISION_LIBVIRT" = x ]; then + determine_version REVISION_LIBVIRT libvirt + export REVISION_LIBVIRT +fi if [ "x$REVISION_LINUXFIRMWARE" = x ]; then determine_version REVISION_LINUXFIRMWARE linuxfirmware export REVISION_LINUXFIRMWARE @@ -165,6 +169,10 @@ linuxfirmware) realtree=linuxfirmware NEW_REVISION=$REVISION_LINUXFIRMWARE ;; +libvirt) + realtree=libvirt + NEW_REVISION=$REVISION_LIBVIRT + ;; *) NEW_REVISION='' wantpush=false diff --git a/cri-common b/cri-common index 497d4e3..f472927 100644 --- a/cri-common +++ b/cri-common @@ -42,6 +42,7 @@ select_xenbranch () { linux) tree=linux; xenbranch=xen-unstable ;; linux-*) tree=linux; xenbranch=xen-unstable ;; linuxfirmware) tree=linuxfirmware; xenbranch=xen-unstable ;; + libvirt) tree=libvirt; xenbranch=xen-unstable ;; osstest) tree=osstest; xenbranch=xen-unstable ;; esac if [ "x$tree" = xlinux ]; then diff --git a/make-flight b/make-flight index 92b813a..aadf58b 100755 --- a/make-flight +++ b/make-flight @@ -64,6 +64,9 @@ job_create_test_filter_callback () { return 1;; esac ;; + libvirt) + if [ x$toolstack != xlibvirt ] ; then return 1; fi + ;; *) case "$job" in *-qemuu-*) @@ -238,6 +241,10 @@ test_matrix_do_one () { $xenarch $dom0arch \ $debian_runvars all_hostflags=$most_hostflags + job_create_test test-$xenarch$kern-$dom0arch-libvirt test-debian libvirt \ + $xenarch $dom0arch \ + $debian_runvars all_hostflags=$most_hostflags + # No further arm tests at the moment if [ $dom0arch = armhf ]; then return diff --git a/mfi-common b/mfi-common index 598aa02..a8e528a 100644 --- a/mfi-common +++ b/mfi-common @@ -162,6 +162,16 @@ create_build_jobs () { tree_linuxfirmware=$TREE_LINUXFIRMWARE \ revision_linuxfirmware=$REVISION_LINUXFIRMWARE + ./cs-job-create $flight build-$arch-libvirt build-libvirt \ + arch=$arch \ + tree_xen=$TREE_XEN \ + $RUNVARS $BUILD_RUNVARS $BUILD_LIBVIRT_RUNVARS $arch_runvars \ + $suite_runvars \ + host_hostflags=$build_hostflags \ + buildjob=${bfi}build-$arch \ + tree_libvirt=$TREE_LIBVIRT revision_libvirt=$REVISION_LIBVIRT\ + ${TREEVCS_LIBVIRT:+treevcs_libvirt=}${TREEVCS_LIBVIRT} + case "$arch" in armhf) continue;; # don't do any other kernel builds esac @@ -193,6 +203,7 @@ job_create_test () { xenbuildjob="${bfi}build-$xenarch" buildjob="${bfi}build-$dom0arch" + tsbuildjob= case "$xenbranch:$toolstack" in xen-3.*-testing:*) ;; @@ -203,11 +214,13 @@ job_create_test () { *:xend) xenbuildjob="$xenbuildjob-xend" buildjob="${bfi}build-$dom0arch-xend" ;; + *:libvirt) tsbuildjob="libvirtbuildjob=$buildjob-libvirt" + ;; esac ./cs-job-create $flight $job $recipe toolstack=$toolstack \ $RUNVARS $TEST_RUNVARS $most_runvars \ - xenbuildjob=$xenbuildjob buildjob=$buildjob "$@" + xenbuildjob=$xenbuildjob buildjob=$buildjob $tsbuildjob "$@" } # Iterate over xenarch, dom0arch and kernel calling test_matrix_do_one diff --git a/sg-run-job b/sg-run-job index d336fbd..3d44331 100755 --- a/sg-run-job +++ b/sg-run-job @@ -312,6 +312,7 @@ proc test-guest-nomigr {g} { proc need-hosts/build {} { return BUILD } proc need-hosts/build-kern {} { return BUILD } +proc need-hosts/build-libvirt {} { return BUILD } proc run-job/build {} { run-ts . = ts-xen-build @@ -321,6 +322,10 @@ proc run-job/build-kern {} { run-ts . = ts-kernel-build } +proc run-job/build-libvirt {} { + run-ts . = ts-libvirt-build +} + proc prepare-build-host {} { global jobinfo run-ts broken = ts-hosts-allocate + host diff --git a/standalone b/standalone index 37c0294..d10a223 100755 --- a/standalone +++ b/standalone @@ -166,7 +166,7 @@ case $op in job=$1; shift - for d in '' xen kern ; do + for d in '' xen kern libvirt ; do runvar="path_${d}dist" path="build/${d}dist.tar.gz" if [ -f "logs/$flight/$job/$path" ] ; then diff --git a/ts-libvirt-build b/ts-libvirt-build new file mode 100755 index 0000000..56b78f1 --- /dev/null +++ b/ts-libvirt-build @@ -0,0 +1,159 @@ +#!/usr/bin/perl +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2009-2014 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see . + +use strict qw(vars); +use DBI; +use Osstest; +use Osstest::TestSupport; + +tsreadconfig(); + +our ($whhost) = @ARGV; +$whhost ||= 'host'; +our $ho= selecthost($whhost); + +my $leaf= "build.$flight.$job"; +my $homedir = get_host_property($ho, 'homedir', '/home/osstest'); +my $builddir= "$homedir/$leaf"; +my $xendist= "$builddir/xendist"; + +sub libvirtd_init (); + +my $makeflags= get_host_property($ho, 'build make flags', '-j4'); + +sub checkout () { + target_cmd($ho, "rm -rf $builddir && mkdir $builddir && mkdir $xendist", 600); + + my $path = get_stashed("path_dist", $r{"buildjob"}); + my $distcopy= "$builddir/dist.tar.gz"; + target_putfile_root($ho, 300, $path, $distcopy); + target_cmd_root($ho, "tar -C $xendist -hzxf $distcopy", 300); + + build_clone($ho, 'libvirt', $builddir, 'libvirt'); +} + +sub prep() { + target_install_packages($ho, qw(autoconf automake libtool xsltproc + libxml2-utils libxml2-dev libnl-dev + libdevmapper-dev w3c-dtd-xhtml)); +} + +sub config() { + target_cmd_build($ho, 1000, $builddir, <&1 && touch ../build-ok-stamp) |tee ../log + test -f ../build-ok-stamp #/ + echo ok. +END +} + +sub install() { + target_cmd_build($ho, 300, $builddir, < +# based on the skeletons that comes with dh_make +# +### BEGIN INIT INFO +# Provides: libvirt-bin libvirtd +# Required-Start: $network $local_fs $remote_fs $syslog +# Required-Stop: $local_fs $remote_fs $syslog +# Should-Start: avahi-daemon cgconfig +# Should-Stop: avahi-daemon cgconfig +# Default-Start: 2 3 4 5 +# Default-Stop: 0 1 6 +# Short-Description: libvirt management daemon +### END INIT INFO + +PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin +export PATH +DAEMON=/usr/local/sbin/libvirtd +NAME=libvirtd +DESC="libvirt management daemon" + +test -x $DAEMON || exit 0 +. /lib/lsb/init-functions + +PIDFILE=/var/run/$NAME.pid + +case "$1" in + start) + [ "$VERBOSE" != no ] && log_daemon_msg "Starting $DESC" "$NAME" + start-stop-daemon --start --pidfile $PIDFILE \ + --exec $DAEMON -- -d $libvirtd_opts + [ "$VERBOSE" != no ] && log_end_msg $? + ;; + stop) + [ "$VERBOSE" != no ] && log_daemon_msg "Stopping $DESC" "$NAME" + start-stop-daemon --stop --pidfile $PIDFILE \ + --exec $DAEMON + [ "$VERBOSE" != no ] && log_end_msg $? + ;; + restart) + stop + start + ;; + reload|force-reload) + start-stop-daemon --stop --signal 1 --quiet --pidfile \ + /var/run/$NAME.pid --exec $DAEMON + ;; + *) + N=/etc/init.d/libvirtd + echo "Usage: $N {start|stop|restart|reload|force-reload}" >&2 + exit 1 + ;; +esac + +exit 0 +END +} diff --git a/ts-xen-install b/ts-xen-install index 4a810e4..435fc3e 100755 --- a/ts-xen-install +++ b/ts-xen-install @@ -63,7 +63,10 @@ sub packages () { } sub extract () { - foreach my $part ('', 'kern', 'xen') { + my @parts = ('', 'kern', 'xen'); + push @parts, 'libvirt' if $r{toolstack} eq "libvirt"; + + foreach my $part (@parts) { target_extract_jobdistpath($ho, $part, "path_${part}dist", $r{"${part}buildjob"}, \%distpath); }