From patchwork Thu May 24 08:07:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 136715 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1884151lji; Thu, 24 May 2018 01:08:01 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqerMoH4pkKH00ptqeuk8RxrRwgWuKNS1DJ2s3dEfr5WShnj8jWKOhOml4sd5QmPaTej9az X-Received: by 2002:a65:4487:: with SMTP id l7-v6mr4914762pgq.194.1527149281671; Thu, 24 May 2018 01:08:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527149281; cv=none; d=google.com; s=arc-20160816; b=oWzU9Iyt8kYLkvN+FCxRZmyAtxIJavfqm9hYEWgICRYC45iBMHFYDJCEiJniF4cXMX xlkvDzWrfU849ErvwJ9zpK2i7/2v6tFqc9EiPxfSNdoaOmZ4bYXomIjX1TcdGNsrdJKt +NFYeKcpEFMJabBieZo0iNeQX9yOcpYpgF8376ROdTOqhKgCFyUen0FZzJHdX5ibQeqY AOZRvOubOqa/mj7roe3lva69Vv6CbRR4UiGXpLLwJvjqGe2MNrWzI1s0cXQc7IYCdjbq d1lhNSHMmqMm4plwrv0lWF8ztKMUsrd7UnBNBtq9rNgtVKaCxRP1d4O9nh8tQ16O2zOy yzFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:date:subject:mail-followup-to:to :from:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=ijLXpJ35YphEz6kgObnP5TYBOingsjFWJlMfSIwHxKk=; b=pBYn8BDhFqzu1f2TKOAxtPyZ74FDUoS/ppa00S5R/iyJgdFDB/NMjURyED9heTPE9I yl+Yf+GAprJ36wf5KcnYjy8vZdI0R0eOjbh4+PuvCr5Dpkk87Hqs83kRiulsSlvEZkyh P/VaFw/G17Tkf9NqLzNwEOYbozX1rEA+ZbNR2GXoz2FGFAUk7i1WtDNLIz6FNmnTtscV fdl0bmn61n4D3biPv3+iPAKIfZZmiAoGaVIxN45re8ylVci8SN/cxrPXoc1WG0H2IKCs YmIqt8KOZU8VIdKd0hiJb9XncfZ8FKaeSJfdW4nMRoNMV36And35zhAkd+jMHV2wk4DX QK3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=v4rXaYUp; spf=pass (google.com: domain of gcc-patches-return-478349-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-478349-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id v7-v6si1952154plz.86.2018.05.24.01.08.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 May 2018 01:08:01 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-478349-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=v4rXaYUp; spf=pass (google.com: domain of gcc-patches-return-478349-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-478349-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; q=dns; s= default; b=p5ygQLIIS5ndlYMZLJeh0Zre9/VPevLLcwzZeJxSELg0vNjtRFvY7 jGv5EIJ/VFtJ3vLQfyx9TsMwp/UgpdRc7PhBnAjq1UO8AGeJjz4sfzvuIG+KxNNF 0xG+kKaZ3A4eYSltAEg92TXsVwkQNPlfR9W1nO/pOd7Bv/ypScuJos= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; s= default; bh=cHGmhIY+dQpyXwNAMyMXK8d6SzA=; b=v4rXaYUpRLTSFAgpkZwz T3yvItvZFtuMq/xVsK41i9jyDVO6/NdBp6mDxBd0UYiwsX4CtcdAOp8HXfKqkSpv bkouOJ7nG6NB82X+BuzzBDIakP3oanRZPtxtZ33GUmx5u6NyxEJT4TjIz2Op1741 s6A4t17TFxuAHXlN27xOewY= Received: (qmail 108213 invoked by alias); 24 May 2018 08:07:49 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 108195 invoked by uid 89); 24 May 2018 08:07:48 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-11.2 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=Advanced, 2018-04-20, 46369131350, 46.369131350 X-HELO: mail-wr0-f179.google.com Received: from mail-wr0-f179.google.com (HELO mail-wr0-f179.google.com) (209.85.128.179) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 24 May 2018 08:07:46 +0000 Received: by mail-wr0-f179.google.com with SMTP id w18-v6so1349175wrn.6 for ; Thu, 24 May 2018 01:07:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:date:message-id :user-agent:mime-version; bh=ijLXpJ35YphEz6kgObnP5TYBOingsjFWJlMfSIwHxKk=; b=GYoutl2zufVOoz4DJ0GTjB0+shDHaoeLijxvd9A1I9eXwBmLaOGz0oTVoDWeMOWHqu C2zr4+acVqlSlOm+/lvYEoDll99mK+tBlr6LbKmq1rfLpgxm0O/aybjS4bi43YnS5gsZ Ua7Papw1fnk8w4VMEIgcfEKcssX7XTRVAQZNqwYQZkknMy0Oro2NO5Mx9CoB6OKhOeOw H3FJfPPdGxam7pMx0WceYkYnYgpCr8jJPe5jrwAiwH6XHqv3PbRJjBI3kFNjELyf0mNG 4tAtvLR6CYtZdfSGfEK3zyd2TOVsv7HrFhjiobBaRdjGxCD2wxP99WkzE6/5zYSqPNg5 Lp+Q== X-Gm-Message-State: ALKqPwd1wEEG1Wd7hzTpmHzCNReO2KOaD9R+pBvPAo2HkZQnpArsUfM0 Pc+oDls6T93nmKqrH+PxZQ1Nurtbx7U= X-Received: by 2002:adf:e686:: with SMTP id r6-v6mr6035129wrm.35.1527149264431; Thu, 24 May 2018 01:07:44 -0700 (PDT) Received: from localhost (201.69.7.51.dyn.plus.net. [51.7.69.201]) by smtp.gmail.com with ESMTPSA id h81-v6sm9631749wmd.0.2018.05.24.01.07.42 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 24 May 2018 01:07:43 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: Use canonicalize_math_after_vectorization_p for FMA folds Date: Thu, 24 May 2018 09:07:42 +0100 Message-ID: <87lgc9easx.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 The folds in r260348 kicked in before vectorisation, which hurts for two reasons: (1) the current suboptimal handling of nothrow meant that we could drop the flag early and so prevent if-conversion (2) some architectures provide more scalar forms than vector forms (true for Advanced SIMD) (1) is a bug in itself that needs to be fixed eventually, but delaying the folds is still needed for (2). Tested on aarch64-linux-gnu (with and without SVE), aarch64_be-elf and x86_64-linux-gnu. OK to install? (Patch is mostly just reindent.) Richard 2018-05-24 Richard Sandiford gcc/ * match.pd: Delay FMA folds until after vectorization. gcc/testsuite/ * gcc.dg/vect/vect-fma-1.c: New test. Index: gcc/match.pd =================================================================== --- gcc/match.pd 2018-05-18 09:26:37.735714314 +0100 +++ gcc/match.pd 2018-05-24 09:05:10.432158893 +0100 @@ -4703,59 +4703,60 @@ DEFINE_INT_AND_FLOAT_ROUND_FN (RINT) wi::to_wide (@ipos) + isize)) (BIT_FIELD_REF @0 @rsize @rpos))))) -(for fmas (FMA) +(if (canonicalize_math_after_vectorization_p ()) + (for fmas (FMA) + (simplify + (fmas:c (negate @0) @1 @2) + (IFN_FNMA @0 @1 @2)) + (simplify + (fmas @0 @1 (negate @2)) + (IFN_FMS @0 @1 @2)) + (simplify + (fmas:c (negate @0) @1 (negate @2)) + (IFN_FNMS @0 @1 @2)) + (simplify + (negate (fmas@3 @0 @1 @2)) + (if (single_use (@3)) + (IFN_FNMS @0 @1 @2)))) + + (simplify + (IFN_FMS:c (negate @0) @1 @2) + (IFN_FNMS @0 @1 @2)) (simplify - (fmas:c (negate @0) @1 @2) + (IFN_FMS @0 @1 (negate @2)) + (IFN_FMA @0 @1 @2)) + (simplify + (IFN_FMS:c (negate @0) @1 (negate @2)) (IFN_FNMA @0 @1 @2)) (simplify - (fmas @0 @1 (negate @2)) - (IFN_FMS @0 @1 @2)) + (negate (IFN_FMS@3 @0 @1 @2)) + (if (single_use (@3)) + (IFN_FNMA @0 @1 @2))) + + (simplify + (IFN_FNMA:c (negate @0) @1 @2) + (IFN_FMA @0 @1 @2)) (simplify - (fmas:c (negate @0) @1 (negate @2)) + (IFN_FNMA @0 @1 (negate @2)) (IFN_FNMS @0 @1 @2)) (simplify - (negate (fmas@3 @0 @1 @2)) + (IFN_FNMA:c (negate @0) @1 (negate @2)) + (IFN_FMS @0 @1 @2)) + (simplify + (negate (IFN_FNMA@3 @0 @1 @2)) (if (single_use (@3)) - (IFN_FNMS @0 @1 @2)))) + (IFN_FMS @0 @1 @2))) -(simplify - (IFN_FMS:c (negate @0) @1 @2) - (IFN_FNMS @0 @1 @2)) -(simplify - (IFN_FMS @0 @1 (negate @2)) - (IFN_FMA @0 @1 @2)) -(simplify - (IFN_FMS:c (negate @0) @1 (negate @2)) - (IFN_FNMA @0 @1 @2)) -(simplify - (negate (IFN_FMS@3 @0 @1 @2)) + (simplify + (IFN_FNMS:c (negate @0) @1 @2) + (IFN_FMS @0 @1 @2)) + (simplify + (IFN_FNMS @0 @1 (negate @2)) + (IFN_FNMA @0 @1 @2)) + (simplify + (IFN_FNMS:c (negate @0) @1 (negate @2)) + (IFN_FMA @0 @1 @2)) + (simplify + (negate (IFN_FNMS@3 @0 @1 @2)) (if (single_use (@3)) - (IFN_FNMA @0 @1 @2))) - -(simplify - (IFN_FNMA:c (negate @0) @1 @2) - (IFN_FMA @0 @1 @2)) -(simplify - (IFN_FNMA @0 @1 (negate @2)) - (IFN_FNMS @0 @1 @2)) -(simplify - (IFN_FNMA:c (negate @0) @1 (negate @2)) - (IFN_FMS @0 @1 @2)) -(simplify - (negate (IFN_FNMA@3 @0 @1 @2)) - (if (single_use (@3)) - (IFN_FMS @0 @1 @2))) - -(simplify - (IFN_FNMS:c (negate @0) @1 @2) - (IFN_FMS @0 @1 @2)) -(simplify - (IFN_FNMS @0 @1 (negate @2)) - (IFN_FNMA @0 @1 @2)) -(simplify - (IFN_FNMS:c (negate @0) @1 (negate @2)) - (IFN_FMA @0 @1 @2)) -(simplify - (negate (IFN_FNMS@3 @0 @1 @2)) - (if (single_use (@3)) - (IFN_FMA @0 @1 @2))) + (IFN_FMA @0 @1 @2)))) Index: gcc/testsuite/gcc.dg/vect/vect-fma-1.c =================================================================== --- /dev/null 2018-04-20 16:19:46.369131350 +0100 +++ gcc/testsuite/gcc.dg/vect/vect-fma-1.c 2018-05-24 09:05:10.432158893 +0100 @@ -0,0 +1,58 @@ +/* { dg-require-effective-target scalar_all_fma } */ + +#include "tree-vect.h" + +#define N (VECTOR_BITS * 11 / 64 + 3) + +#define DEF(INV) \ + void __attribute__ ((noipa)) \ + f_##INV (double *restrict a, double *restrict b, \ + double *restrict c, double *restrict d) \ + { \ + for (int i = 0; i < N; ++i) \ + { \ + double mb = (INV & 1 ? -b[i] : b[i]); \ + double mc = c[i]; \ + double md = (INV & 2 ? -d[i] : d[i]); \ + double fma = __builtin_fma (mb, mc, md); \ + a[i] = (INV & 4 ? -fma : fma); \ + } \ + } + +#define TEST(INV) \ + { \ + f_##INV (a, b, c, d); \ + for (int i = 0; i < N; ++i) \ + { \ + double mb = (INV & 1 ? -b[i] : b[i]); \ + double mc = c[i]; \ + double md = (INV & 2 ? -d[i] : d[i]); \ + double fma = __builtin_fma (mb, mc, md); \ + double expected = (INV & 4 ? -fma : fma); \ + if (a[i] != expected) \ + __builtin_abort (); \ + asm volatile ("" ::: "memory"); \ + } \ + } + +#define FOR_EACH_INV(T) \ + T (0) T (1) T (2) T (3) T (4) T (5) T (6) T (7) + +FOR_EACH_INV (DEF) + +int +main (void) +{ + double a[N], b[N], c[N], d[N]; + for (int i = 0; i < N; ++i) + { + b[i] = i % 17; + c[i] = i % 9 + 11; + d[i] = i % 13 + 14; + asm volatile ("" ::: "memory"); + } + FOR_EACH_INV (TEST) + return 0; +} + +/* { dg-final { scan-tree-dump-times "LOOP VECTORIZED" 8 "vect" { target vect_double } } } */