From patchwork Tue May 28 09:49:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 165291 Delivered-To: patch@linaro.org Received: by 2002:ac9:2a84:0:0:0:0:0 with SMTP id p4csp7810922oca; Tue, 28 May 2019 03:08:45 -0700 (PDT) X-Google-Smtp-Source: APXvYqyMu387zBTGiv0jpd07WiMXiGStYx0NO7NJT0deluELj18MjiyoCXJpVkUvrEuf1aS1jbJv X-Received: by 2002:ac8:32e9:: with SMTP id a38mr4820395qtb.245.1559038125527; Tue, 28 May 2019 03:08:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559038125; cv=none; d=google.com; s=arc-20160816; b=haKFNq6cEK7uQmWNz+Ka6hznF4k2Jczq0+dqnDB5ytwV50jstA5p0MJEm7int0xsEI 4AZKpkQgLHOjSZGQFKIoRwgPYr476BzOBTwjuryJ356q8hP7NFu4UKFQsSr2AOByT9Rf SqWMfj4S2JtLb3QznPksqgMzHlyTfxSKJHwMbDYQ/X5ux921aMgJceQXA6EJH/lBgpdz fAOds4Tm7JIoeNdyWpyFfapZFOvvjFIG+en9h7SuEQGV2eQYbMaFRhk0q8aALZpwHosF ft3Ij92TmNJzinLcg+CiQziYCYdCyk564KVaJPsdgxIqK/lygOS/C7ywRSigGkcWxr7s /Y3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=JwuWb01uGinb5zd5Xhv8VSI/Lid9VvCyBHhhMXoCHSE=; b=eSDQiOWpO3xxDJ3UEsni4CfqhZcWvrdioRg9bwZ+o78UaNSynZvIKZ6r3I1emOnXEi g5BNxf7aEuyQvYsmV5CixElaksgjbBCdI/8rgNJuViTcZB8ZmmYmUaaGwGFPWDMqGi6Q LQL0SaNQWXyMJbuAoWmWEe14zNlkJ2l+gmxE+SI44AAM2rJcQGfpupv+MxRtq90xW6/1 yMOapxutMc4OhYWgTfDSPjQUYCykwfjPM6eU3O6lsFVOcUbTSX9/z2U/zgA7BCk9i07E 5W4Dww5WDZ9ajKss0d3EJHzh6MhYVg1GAoGmN6EvJrqkBnRLaVFiN6b7CRKed6TwGDU2 fvEg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="Oz/NMYnV"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s49si1317285qth.14.2019.05.28.03.08.45 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 28 May 2019 03:08:45 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="Oz/NMYnV"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:60219 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hVZ2D-0004s9-1p for patch@linaro.org; Tue, 28 May 2019 06:08:45 -0400 Received: from eggs.gnu.org ([209.51.188.92]:43614) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hVYtj-0006Q8-0L for qemu-devel@nongnu.org; Tue, 28 May 2019 06:00:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hVYth-0005bm-E7 for qemu-devel@nongnu.org; Tue, 28 May 2019 05:59:58 -0400 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]:52276) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hVYth-0005aZ-8C for qemu-devel@nongnu.org; Tue, 28 May 2019 05:59:57 -0400 Received: by mail-wm1-x341.google.com with SMTP id y3so2175723wmm.2 for ; Tue, 28 May 2019 02:59:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JwuWb01uGinb5zd5Xhv8VSI/Lid9VvCyBHhhMXoCHSE=; b=Oz/NMYnVy0VTC1Lo/BrVoHKWAILtjUPZJi2RBxCSWtaS8kNmA/TEeKfobWTw6kFYYo yqbG2AAgnK622jmnfhiUwgNNROaczaVRln/JWq66mh/mqWiw7RymhPXxz7DN9hFAB/m7 PQD+CsUe0rHeQRKRp/nb/gc4FpZjb2VDch75EO4w4FHGT58VUPlA8vZ/Hv4jYUajPueU YhIrtdaZnUtdrYf+5mtUF1fwZxB4wn/TjhS/G/WE1/FrgyLrwpB14YISIxgDCjVEiUNK PkarGQIx4gJe5ZshReIoLNkZrr0K/zHQ7W5L/6p/u3raIz5SeDkA1mo6N9QgYugdjW+J V/Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JwuWb01uGinb5zd5Xhv8VSI/Lid9VvCyBHhhMXoCHSE=; b=t+bDyAkLODJwXwUsMsEJ00mBTaxHIcWhXhYzs8I8aRoIGmz4f5xwFMV4f9J90/jjvi jwagK0hkAns9K1dLIEubLQA4AkDQ8sPdHMngQUA7HozwidXpEd5XdeXOy0IB6LBR7K3a tpLxZfHGrEPEsb94OaGw9ekej5QaQtM4/GjkQk6yTY2x6KgAOJTwQ9q23G3LVSaCuH3M 9BX4Qhl/UaW7+25pb6FXd2qcKlMqKSUAmWVMR+gpxZyJv9++rILGI6U8Ufu44P2LSllm k+0UzV7HGWjYH5/l7MlbpeQyTz67Meo4e42HMhUuauv3MA4FlTxUs3WmLJqXuhyJKfDM h7Rw== X-Gm-Message-State: APjAAAWvNb/9EX2d8pahlW/dvfDcAm5BPlUiHRMdFwX03NU4I3qzH8bv gJFBo99+Yz3x5nzHE6g3iaLUew== X-Received: by 2002:a1c:c909:: with SMTP id f9mr2496045wmb.115.1559037596166; Tue, 28 May 2019 02:59:56 -0700 (PDT) Received: from zen.linaroharston ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id g11sm11395367wrx.62.2019.05.28.02.59.50 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 28 May 2019 02:59:51 -0700 (PDT) Received: from zen.linaroharston. (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id E4A661FFB1; Tue, 28 May 2019 10:49:55 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: peter.maydell@linaro.org Date: Tue, 28 May 2019 10:49:53 +0100 Message-Id: <20190528094953.14898-29-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190528094953.14898-1-alex.bennee@linaro.org> References: <20190528094953.14898-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::341 Subject: [Qemu-devel] [PULL 28/28] tests: Run the iotests during "make check" again X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Thomas Huth , =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Thomas Huth People often forget to run the iotests before submitting patches or pull requests - this is likely due to the fact that we do not run the tests during our mandatory "make check" tests yet. Now that we've got a proper "auto" group of iotests that should be fine to run in every environment, we can enable the iotests during "make check" again by running the "auto" tests by default from the check-block.sh script. Some cases still need to be checked first, though: iotests need bash and GNU sed (otherwise they fail), and if gprof is enabled, it spoils the output of some test cases causing them to fail. So if we detect that one of the required programs is missing or that gprof is enabled, we still have to skip the iotests to avoid failures. And finally, since we are using check-block.sh now again, this patch also removes the qemu-iotests-quick.sh script since we do not need that anymore (and having two shell wrapper scripts around the block tests seem rather confusing than helpful). Signed-off-by: Thomas Huth Message-Id: <20190502084506.8009-8-thuth@redhat.com> [AJB: -makecheck to check-block.sh, move check-block to start&gate it] Signed-off-by: Alex Bennée -- 2.20.1 diff --git a/tests/Makefile.include b/tests/Makefile.include index 1865f6b3222..0c6d8c75941 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -140,7 +140,7 @@ check-unit-y += tests/test-uuid$(EXESUF) check-unit-y += tests/ptimer-test$(EXESUF) check-unit-y += tests/test-qapi-util$(EXESUF) -check-block-$(CONFIG_POSIX) += tests/qemu-iotests-quick.sh +check-block-$(call land,$(CONFIG_POSIX),$(CONFIG_SOFTMMU)) += tests/check-block.sh # All QTests for now are POSIX-only, but the dependencies are # really in libqtest, not in the testcases themselves. @@ -1077,8 +1077,8 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES) QEMU_IOTESTS_HELPERS-$(call land,$(CONFIG_SOFTMMU),$(CONFIG_LINUX)) = tests/qemu-iotests/socket_scm_helper$(EXESUF) -.PHONY: check-tests/qemu-iotests-quick.sh -check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) +.PHONY: check-tests/check-block.sh +check-tests/check-block.sh: tests/check-block.sh qemu-img$(EXESUF) qemu-io$(EXESUF) qemu-nbd$(EXESUF) $(QEMU_IOTESTS_HELPERS-y) $< .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y)) @@ -1152,7 +1152,7 @@ check-acceptance: check-venv $(TESTS_RESULTS_DIR) check-qapi-schema: $(patsubst %,check-%, $(check-qapi-schema-y)) check-tests/qapi-schema/doc-good.texi check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS)) check-block: $(patsubst %,check-%, $(check-block-y)) -check: check-qapi-schema check-unit check-softfloat check-qtest check-decodetree +check: check-block check-qapi-schema check-unit check-softfloat check-qtest check-decodetree check-clean: rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y) rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y)) diff --git a/tests/check-block.sh b/tests/check-block.sh index f3d12fd602d..c8b6cec3f68 100755 --- a/tests/check-block.sh +++ b/tests/check-block.sh @@ -1,24 +1,48 @@ #!/bin/sh -FORMAT_LIST="raw qcow2 qed vmdk vpc" +# Honor the SPEED environment variable, just like we do it for the qtests. +if [ "$SPEED" = "slow" ]; then + format_list="raw qcow2" + group= +elif [ "$SPEED" = "thorough" ]; then + format_list="raw qcow2 qed vmdk vpc" + group= +else + format_list=qcow2 + group="-g auto" +fi + if [ "$#" -ne 0 ]; then - FORMAT_LIST="$@" + format_list="$@" +fi + +if grep -q "TARGET_GPROF=y" *-softmmu/config-target.mak 2>/dev/null ; then + echo "GPROF is enabled ==> Not running the qemu-iotests." + exit 0 fi -export QEMU_PROG="$PWD/x86_64-softmmu/qemu-system-x86_64" -export QEMU_IMG_PROG="$PWD/qemu-img" -export QEMU_IO_PROG="$PWD/qemu-io" +if [ -z "$(find . -name 'qemu-system-*' -print)" ]; then + echo "No qemu-system binary available ==> Not running the qemu-iotests." + exit 0 +fi + +if ! command -v bash >/dev/null 2>&1 ; then + echo "bash not available ==> Not running the qemu-iotests." + exit 0 +fi -if [ ! -x $QEMU_PROG ]; then - echo "'make check-block' requires qemu-system-x86_64" - exit 1 +if ! (sed --version | grep 'GNU sed') > /dev/null 2>&1 ; then + if ! command -v gsed >/dev/null 2>&1; then + echo "GNU sed not available ==> Not running the qemu-iotests." + exit 0 + fi fi cd tests/qemu-iotests ret=0 -for FMT in $FORMAT_LIST ; do - ./check -T -nocache -$FMT || ret=1 +for fmt in $format_list ; do + ./check -makecheck -$fmt $group || ret=1 done exit $ret diff --git a/tests/qemu-iotests-quick.sh b/tests/qemu-iotests-quick.sh deleted file mode 100755 index 0e554bb9724..00000000000 --- a/tests/qemu-iotests-quick.sh +++ /dev/null @@ -1,8 +0,0 @@ -#!/bin/sh - -cd tests/qemu-iotests - -ret=0 -TEST_DIR=${TEST_DIR:-/tmp/qemu-iotests-quick-$$} ./check -T -qcow2 -g quick || ret=1 - -exit $ret