From patchwork Tue Feb 20 16:53:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 128925 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp4842175ljc; Tue, 20 Feb 2018 08:53:26 -0800 (PST) X-Google-Smtp-Source: AH8x226LQMp+aA612rpcmx/J6NT9f3dKFQPR2xW+Zq1G6s6qeaUJ2rovDQJ1DpmRhV52iuJG0XGJ X-Received: by 10.99.173.73 with SMTP id y9mr197943pgo.432.1519145606648; Tue, 20 Feb 2018 08:53:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519145606; cv=none; d=google.com; s=arc-20160816; b=0HBiBT1eZ28mxW6HLqfCJQk8GDgxtIwO+WjSk4Bnl76IAv+C7Cy75F1I6hbsmoGL19 JkjeiAvmQudMbSW5ApXHvKLVFusJBlEeXyVbfWDlK0bHN0GAEuNgs7Mw5+jj+FtE1CaC TDGvXHj8fW2AHgQF5hD/BBaJcQzJ0ruv/HIN2FBPoe2ZZLlAqP4LuSgbS6KHlSDgE2QL V3amrvN9O8LDkOhOf0mMGpM9rCvWYAR2zo05/jYiXmF4UfJ8V/Bn2C3aWoIMXXg99s8t sgqajuLePYh3lkCc7KKWnXzoAOD3X/BDqHfivE76OQbH1QNDX96cXqij3wAc1ztJqqGj /sSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:date:subject:mail-followup-to:to :from:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=v83KgrUrE5scWArfCq7mhlKxRqlmP9eOaEwMOVRRzAw=; b=qtm76P6vsUvmISOmpSSnjc78p6+Y1X3uLK7AmVrvrFDSVoimjJZUctMQzZkTagxouA 3vNQplyMyHwiGLRDtGZn8injWGoaDAEHFCBn8SbNuICvEz5OcJtomO6b/Km+/IncIKuC n7ghnGrV/H+sAc7o01U9fvHdqvLkEcPnkPmljbkkrwUpv0g8t3vp+Yr95D1GtTVnkTpL aZGMsxCdmsHlX7shP3bK9nNZxPl4VdeEn3ma7TxUmn28Btsi6LC0tbfSsye8lACgXaOb 3PBuHN9SON5HWwBxNiDqTi1YaC/g1UapKTuzHfRY3K+JjO7cxMolmdXCkioHCNT6r9kq L/pw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=cR/DiWW1; spf=pass (google.com: domain of gcc-patches-return-473605-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-473605-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id c62si1661935pfg.240.2018.02.20.08.53.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Feb 2018 08:53:26 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-return-473605-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=cR/DiWW1; spf=pass (google.com: domain of gcc-patches-return-473605-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-473605-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; q=dns; s= default; b=roLEtD/fPzsa4wkKJcnWGuewqnuPMan+gg+kGWW0Qd4qbeN6zae1V jQkX4kknNB55XOF9MMpMPXG/zXYxxnx4nBzXcqxcbGKZBoaen/fzGT2Pmn//Tfw7 /O5NhBnOU6RsJJTU2PmQE3C+QCPfTItqGVhx8CymldyzTKNoLYAOqY= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; s= default; bh=nLlTZ+GIQwGXXDYqPltcLNrrLJc=; b=cR/DiWW1VEBqVdqOoH3o r/l1wPb22vckkVuO86qHEP5+9CcThfEYGLBXXtfcVpTEZQk9MpfdaCAJHZBQvUsb nMspRJWnANEOrkseCErGoUBtpS2tRE/mR7HJSfRszgL/717fyR/W7JpkE28fzdvo Ok/4vFJnoG50xdqOxNL2cjI= Received: (qmail 44172 invoked by alias); 20 Feb 2018 16:53:14 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 44163 invoked by uid 89); 20 Feb 2018 16:53:14 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-12.2 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=keyed, dissolve X-HELO: mail-wr0-f170.google.com Received: from mail-wr0-f170.google.com (HELO mail-wr0-f170.google.com) (209.85.128.170) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Tue, 20 Feb 2018 16:53:12 +0000 Received: by mail-wr0-f170.google.com with SMTP id f14so11883034wre.8 for ; Tue, 20 Feb 2018 08:53:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:date:message-id :user-agent:mime-version; bh=v83KgrUrE5scWArfCq7mhlKxRqlmP9eOaEwMOVRRzAw=; b=qDxlfvqKXNjrLOflsBgb9h9mRGNdWFbOVKq6Uof700CElNrRjZqBttg9/NrXMzUjUs 7m0kMIj98CUn/raM+GzF2R++uFvCGIgXIwZJ+i0qbRArs/UD9IgAWj78CAN2lLRFj0p5 GdfiFfSzTqCkRs3aEECuyLLOzmSO/yAfouuM+K/uqT+Ml+UcT3drQrqUuwoNrkW9Lf15 1JbRbqdNtzAiBbtwfAgaRoEM0xgnsiU30lzW02YjDz2CdQ0C9L7QYuodicIzra7NZmV6 efBwACT0UDybNQl+OgMejUjgMfH1SMmWf4jcU8SPmdnoZ23mC1y4VhUTW+mFezXq+r1e Tqig== X-Gm-Message-State: APf1xPA4YM8G7BhPPbOBwodvGErrcDqdIYLWqstFx6EUZRYS4pTDGbu2 J9GcbLBmlJPw40ugO82bzqxbLAFpR4M= X-Received: by 10.223.158.193 with SMTP id b1mr270337wrf.156.1519145589813; Tue, 20 Feb 2018 08:53:09 -0800 (PST) Received: from localhost (79.58.7.51.dyn.plus.net. [51.7.58.79]) by smtp.gmail.com with ESMTPSA id q131sm12701613wmg.2.2018.02.20.08.53.08 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 20 Feb 2018 08:53:08 -0800 (PST) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: Make fix for PR 83965 handle SLP reduction chains Date: Tue, 20 Feb 2018 16:53:08 +0000 Message-ID: <87606rsivf.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 This patch prevents pattern-matching of fold-left SLP reduction chains, which the previous patch for 83965 didn't handle properly. It only stops the last statement in the group from being matched, but that's enough to cause the group to be dissolved later. A better fix would be to put all the information about the reduction on the the first statement in the reduction chain, so that every statement in the group can tell what the group is doing. That doesn't seem like stage 4 material though. As it stands, things seem to be a bit of a mess. In vect_force_simple_reduction we attach the reduction type and phi pointer to the last statement in a reduction chain: reduc_def_info = vinfo_for_stmt (def); STMT_VINFO_REDUC_TYPE (reduc_def_info) = v_reduc_type; STMT_VINFO_REDUC_DEF (reduc_def_info) = phi; and mark it as vect_reduction_type in vect_analyze_scalar_cycles_1: STMT_VINFO_DEF_TYPE (vinfo_for_stmt (reduc_stmt)) = vect_reduction_def; This code in vectorizable_reduction gave the impression that everything really is keyed off the last statement: /* In case of reduction chain we switch to the first stmt in the chain, but we don't update STMT_INFO, since only the last stmt is marked as reduction and has reduction properties. */ if (GROUP_FIRST_ELEMENT (stmt_info) && GROUP_FIRST_ELEMENT (stmt_info) != stmt) { stmt = GROUP_FIRST_ELEMENT (stmt_info); first_p = false; } But this code is dead these days. GROUP_FIRST_ELEMENT is only nonnull for SLP reduction chains, since we dissolve the group if SLP fails. And SLP only analyses the first statement in the group, not the last: stmt = SLP_TREE_SCALAR_STMTS (node)[0]; stmt_vec_info stmt_info = vinfo_for_stmt (stmt); [...] bool res = vect_analyze_stmt (stmt, &dummy, node, node_instance); So from that point of view the DEF_TYPE, REDUC_TYPE and REDUC_DEF are being attached to the wrong statement, since we only analyse the first one. But it turns out that REDUC_TYPE and REDUC_DEF don't matter for the first statement in the group, since that takes the phi as an input, and when the phi is a direct input, we use *its* REDUC_TYPE and REDUC_DEF instead of the statement's own info. The DEF_TYPE problem is handled by: /* Mark the first element of the reduction chain as reduction to properly transform the node. In the reduction analysis phase only the last element of the chain is marked as reduction. */ if (!STMT_VINFO_GROUPED_ACCESS (vinfo_for_stmt (stmt))) STMT_VINFO_DEF_TYPE (vinfo_for_stmt (stmt)) = vect_reduction_def; in vect_analyze_slp_instance (cancelled by: STMT_VINFO_DEF_TYPE (vinfo_for_stmt (first_element)) = vect_internal_def; in vect_analyze_slp on failure), with the operation being repeated in vect_schedule_slp_instance (redundantly AFAICT): /* Mark the first element of the reduction chain as reduction to properly transform the node. In the analysis phase only the last element of the chain is marked as reduction. */ if (GROUP_FIRST_ELEMENT (stmt_info) && !STMT_VINFO_GROUPED_ACCESS (stmt_info) && GROUP_FIRST_ELEMENT (stmt_info) == stmt) { STMT_VINFO_DEF_TYPE (stmt_info) = vect_reduction_def; STMT_VINFO_TYPE (stmt_info) = reduc_vec_info_type; } Tested on aarch64-linux-gnu, x86_64-linux-gnu and powerpc64-linux-gnu. OK to install? Richard 2018-02-20 Richard Sandiford gcc/ PR tree-optimization/83965 * tree-vect-patterns.c (vect_reassociating_reduction_p): Assume that grouped statements are part of a reduction chain. Return true if the statement is not marked as a reduction itself but is part of a group. (vect_recog_dot_prod_pattern): Don't check whether the statement is part of a group here. (vect_recog_sad_pattern): Likewise. (vect_recog_widen_sum_pattern): Likewise. gcc/testsuite/ PR tree-optimization/83965 * gcc.dg/vect/pr83965-2.c: New test. Index: gcc/tree-vect-patterns.c =================================================================== --- gcc/tree-vect-patterns.c 2018-02-20 09:40:41.843451227 +0000 +++ gcc/tree-vect-patterns.c 2018-02-20 16:28:55.636762056 +0000 @@ -222,13 +222,16 @@ vect_recog_temp_ssa_var (tree type, gimp } /* Return true if STMT_VINFO describes a reduction for which reassociation - is allowed. */ + is allowed. If STMT_INFO is part of a group, assume that it's part of + a reduction chain and optimistically assume that all statements + except the last allow reassociation. */ static bool vect_reassociating_reduction_p (stmt_vec_info stmt_vinfo) { return (STMT_VINFO_DEF_TYPE (stmt_vinfo) == vect_reduction_def - && STMT_VINFO_REDUC_TYPE (stmt_vinfo) != FOLD_LEFT_REDUCTION); + ? STMT_VINFO_REDUC_TYPE (stmt_vinfo) != FOLD_LEFT_REDUCTION + : GROUP_FIRST_ELEMENT (stmt_vinfo) != NULL); } /* Function vect_recog_dot_prod_pattern @@ -350,8 +353,7 @@ vect_recog_dot_prod_pattern (vec *s { gimple *def_stmt; - if (!vect_reassociating_reduction_p (stmt_vinfo) - && ! STMT_VINFO_GROUP_FIRST_ELEMENT (stmt_vinfo)) + if (!vect_reassociating_reduction_p (stmt_vinfo)) return NULL; plus_oprnd0 = gimple_assign_rhs1 (last_stmt); plus_oprnd1 = gimple_assign_rhs2 (last_stmt); @@ -1256,8 +1257,7 @@ vect_recog_widen_sum_pattern (vec