From patchwork Tue Sep 20 09:26:35 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Lyon X-Patchwork-Id: 76593 Delivered-To: patch@linaro.org Received: by 10.140.106.72 with SMTP id d66csp1383348qgf; Tue, 20 Sep 2016 02:27:05 -0700 (PDT) X-Received: by 10.66.149.138 with SMTP id ua10mr36233629pab.146.1474363625759; Tue, 20 Sep 2016 02:27:05 -0700 (PDT) Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id dc7si22967568pad.277.2016.09.20.02.27.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Sep 2016 02:27:05 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-436304-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org; spf=pass (google.com: domain of gcc-patches-return-436304-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-436304-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:from:date:message-id:subject:to:content-type; q= dns; s=default; b=PB+Cd+MzNrBAmRYGBeNVmMzPSfu10jKqyjf6oaXh7/0sNE GfxvCbTEsZhbTruU+8B9x0uheDsREV/yCNkbJJxzKopROvmVDVX20vfaxH0Wr4bZ 3j4aGRHisxqt3NGvHhN9q7H/g3JHl20LcWWG2vhQ4SF/tU933cyfFI7gG0Plo= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:from:date:message-id:subject:to:content-type; s= default; bh=cwVtBLVHJVy08m3Rh33URDDZWr8=; b=LwB4Pw0L6trgZvJXE5qi a1961Ae9BbDFpVZ2P6R8gRzETZ/6yIrZ7zZRe7/qIC93rz89XPMXCiT8nseQhf6J FrUoODAaC6GRP6d/cyXgYrGTa8+I4CMFwUN5BWIv2t+KJH8yo8pVOmcGISScze+H jDHQrJekGzUBRHUCS9mQw9M= Received: (qmail 42448 invoked by alias); 20 Sep 2016 09:26:49 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 42400 invoked by uid 89); 20 Sep 2016 09:26:48 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL, BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy=Running, H*c:sk:001a113, Take, 2016-09-20 X-HELO: mail-qt0-f173.google.com Received: from mail-qt0-f173.google.com (HELO mail-qt0-f173.google.com) (209.85.216.173) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 20 Sep 2016 09:26:38 +0000 Received: by mail-qt0-f173.google.com with SMTP id 93so4613514qtg.2 for ; Tue, 20 Sep 2016 02:26:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=a1MwLf6z3vPEYiF2XLk2jEUEQpRT39gDbwf1UJBGqYA=; b=ArYkk+tSfzTwGiGhqS5de2dfvENhQ9vJe6JcNMOrR+0kmcLwPJsBmf8+FJ38a5x49q BySi7b4XsTlktHCrfQ5Oe6LXXDbXJi9hlhXf9HqBA/F3/R5a+L+mRIvMVDf5dSgQj3Q6 qkmSWnRiG4NhEZ0d70YG+sN8K/6fRQBRF5qZOrtjV27N6/Ny1AEXSyRr259etFBEaIYK 1zdXC1Z4BENFyxfxgl12+cJP3lCN1ZoDspdRZC/TnDygFQC20S5pGjvQByen6rsb/6vv a68mfEjWWuns6ffVc9a+gttjWdc+89CzjM8mo7urP/qzp548kLcyOBc7ijrGAfdZSt58 hGiQ== X-Gm-Message-State: AE9vXwNBJAJwsQZcN1jcjsf5YvT4RYEmK09beatjuidMe815fWPO8lCVqJysERxBcfCANNV9no88uue+/heDSvH9 X-Received: by 10.200.47.3 with SMTP id j3mr33062066qta.27.1474363596727; Tue, 20 Sep 2016 02:26:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.21.80 with HTTP; Tue, 20 Sep 2016 02:26:35 -0700 (PDT) From: Christophe Lyon Date: Tue, 20 Sep 2016 11:26:35 +0200 Message-ID: Subject: Report DejaGnu ERROR messages in compare_tests To: "gcc-patches@gcc.gnu.org" X-IsSubscribed: yes Hello, We recently faced a problem where a DejaGnu error went un-noticed (https://gcc.gnu.org/ml/gcc-patches/2016-08/msg01879.html). To help identify these problems earlier, here is a patch for compare_tests that will report such cases as: -------------------------------- New tests that FAIL: arm-sim: (DejaGnu) proc "scan-dump-tree-not fail_test optimized" does not exist. -------------------------------- OK? Thanks, Christophe contrib/ChangeLog: 2016-09-20 Christophe Lyon * compare_tests: Take ERROR messages into account when comparing. diff --git a/contrib/compare_tests b/contrib/compare_tests index 8ecf824..d16e7e9 100755 --- a/contrib/compare_tests +++ b/contrib/compare_tests @@ -107,8 +107,8 @@ elif [ -d "$1" -o -d "$2" ] ; then usage "Must specify either two directories or two files" fi -sed 's/^XFAIL/FAIL/; s/^XPASS/PASS/' < "$1" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp1 -sed 's/^XFAIL/FAIL/; s/^XPASS/PASS/' < "$2" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp2 +sed 's/^XFAIL/FAIL/; s/^ERROR/FAIL/; s/^XPASS/PASS/' < "$1" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp1 +sed 's/^XFAIL/FAIL/; s/^ERROR/FAIL/; s/^XPASS/PASS/' < "$2" | awk '/^Running target / {target = $3} { if (target != "unix") { sub(/: /, "&"target": " ); }; print $0; }' | cut -c1-2000 >$tmp2 before=$tmp1 now=$tmp2