From patchwork Thu Sep 14 11:24:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 112544 Delivered-To: patch@linaro.org Received: by 10.140.106.117 with SMTP id d108csp626269qgf; Thu, 14 Sep 2017 04:24:51 -0700 (PDT) X-Received: by 10.84.241.201 with SMTP id t9mr24079134plm.154.1505388291632; Thu, 14 Sep 2017 04:24:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505388291; cv=none; d=google.com; s=arc-20160816; b=X08HyFMH9JqTl9aKcoUgNE5GjFwFrlQOZWGEycfHD7Uf0xtWcciJPJA/3v1HvqLmAQ g/WkWkMFQURgNozECf118kN1Mx6VcWqusWmcPJD5DiusjMP0/ikYTstWt7r/JFZEeWYF xYiDm682X2nDmRvZvvCthMtLrY1YxsLc/P9dCp+aVjOkxI9TK6NLnE5tzxCZX8FDeRhG 2MvjE3uBbneHpVN0U/c47fDfipGnoHNYVvZNx+phr2bYaG95kVmPJyKptYQ9WyeAODjS OPQ2ZBzPr2Wp8lnegSY7NUfLZn6HkNzcWQ6EPPdQvzyTMb0wS+4MOfVQWNONaEjdXhWw 3EMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:date:subject:mail-followup-to:to :from:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=YyjzhMZmiGMM3pOrL+zQaHPUSm3vVV6GEdVZaFtUz0I=; b=Yf7Fat2Szc3By+uUYr9NlYoULXvZQnQsJM5Zw8Y/cKdoxyLvpnxf51LL9Gk8RsFW8Z oRz2ORRCatqoyezNT0zp7APIf9NoFUBW03niwf32qWr9dk9568u70KqTcoMakasEfwUz P9gm8w2kKAebw/M0DnTLFuOedhy/zAVyovr6kfFmZsnuPVfl+gPCDbLB0y/NVk16yXXM W+aUWWcQV3dqA5S1rfgKpWgoaRA2/a2o6zI27hQAy0ZQaOp7JEX3HIb5o4YdKlc5A2VE hn4A00YWZhZvRCxnn+Zy+8sWyoJgijWJp2nu/Rlf9/mwgMM/d3O/2ak8v76D4xLmKV09 44Yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Lvdu20lB; spf=pass (google.com: domain of gcc-patches-return-462124-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-462124-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id x10si12076192plm.804.2017.09.14.04.24.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Sep 2017 04:24:51 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-462124-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Lvdu20lB; spf=pass (google.com: domain of gcc-patches-return-462124-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-462124-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; q=dns; s= default; b=ux2hJqNrwzuKq1JyHZ5CJiXBZbmw6geHWHFJ6oEP08bWYhR8Qid6w 01Ge+OBw6/UuyVzuU6A3VX/K+rtMsoEP5CeuZGzrCQZevl2c6vMDD72UcfhxbbWq b6aCIavOCKX+QmLkDiqTFsfLOoZZRvSfMFZ7ZB9KeVZBJZbW2FDq+Y= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:subject:date:message-id:mime-version:content-type; s= default; bh=ni5rSywYVqA+1CDwSKk+BrDi/DU=; b=Lvdu20lBx9JUvIBl+FFY uhxbDiBL7HqRDltFszinRJR/ZUpTgDWUO9fYITlccEAX3KW739yMQGCBSw21aBrf DE0lVmorNGzhYrXiaHZLaC0BK+CXxJWXRoTinbYm2Y97VxBpIX4XrHtgQLMDNSdJ DubOylHr0YSigQz5W+EdUkQ= Received: (qmail 127470 invoked by alias); 14 Sep 2017 11:24:37 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 116734 invoked by uid 89); 14 Sep 2017 11:24:27 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-11.0 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=vectorisation X-HELO: mail-wr0-f173.google.com Received: from mail-wr0-f173.google.com (HELO mail-wr0-f173.google.com) (209.85.128.173) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 14 Sep 2017 11:24:25 +0000 Received: by mail-wr0-f173.google.com with SMTP id u96so36354wrb.6 for ; Thu, 14 Sep 2017 04:24:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:subject:date:message-id :user-agent:mime-version; bh=YyjzhMZmiGMM3pOrL+zQaHPUSm3vVV6GEdVZaFtUz0I=; b=Ax7Le5BXcdysVfTq3huCW1cbKpciPeydvcF8wYASF0EYSeLUZ0RlvYHeY9eTlMB0Jg a1I09l26pk/PG+qlpjYFUO+wjnsMDy8rCrJ9bSgVYoICmRQGJM4kzWWqX+CVIqzJsOu0 hLoBfeUDf0R3kJeNzcqKaRkO3w+mPSj0uzpRpJK0IU26T1WQ2DWbWY28T9a4o5gqkAbq +ID7hOrCeVdtW0ihes043L8iJi7dQUVULb/KsQEP/wlyHFDjlpakeD0/fARsTLaHJehN cPfMK9S8QmjdCb1s0NQJmLySR68D9WuFeX9jBoIRLXaqXjuRxqcgfAn540Zj1WdtUpLY ZlNQ== X-Gm-Message-State: AHPjjUh6STFkI6BGT3I32U31q4Q5wkRgcTdw1ch5P5EFOn1in4bv43P1 q7Vzgdl5HzO3zHBS6EWwuA== X-Google-Smtp-Source: ADKCNb7wkBWBe6pRXnfA733iVtGtBsXwHjcNDxzP6+Z0t8N4Jy1RWzMpw7Ix8IipQfxgCmGIZ4iDbg== X-Received: by 10.223.164.71 with SMTP id e7mr18147829wra.4.1505388262260; Thu, 14 Sep 2017 04:24:22 -0700 (PDT) Received: from localhost (92.40.248.70.threembb.co.uk. [92.40.248.70]) by smtp.gmail.com with ESMTPSA id a69sm949615wme.40.2017.09.14.04.24.21 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 Sep 2017 04:24:21 -0700 (PDT) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.sandiford@linaro.org Subject: Add LOOP_VINFO_MAX_VECT_FACTOR Date: Thu, 14 Sep 2017 12:24:18 +0100 Message-ID: <87k211wmjh.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Epilogue vectorisation uses the vectorisation factor of the main loop as the maximum vectorisation factor allowed for correctness. That makes sense as a conservatively correct value, since the chosen vectorisation factor will be strictly less than that anyway. However, once the VF itself becomes variable, it's easier to carry across the original maximum VF instead. Tested on aarch64-linux-gnu, x86_64-linux-gnu and powerpc64le-linux-gnu. OK to install? Richard 2017-09-14 Richard Sandiford Alan Hayward David Sherwood gcc/ * tree-vectorizer.h (_loop_vec_info): Add max_vectorization_factor. (LOOP_VINFO_MAX_VECT_FACTOR): New macro. (LOOP_VINFO_ORIG_VECT_FACTOR): Replace with... (LOOP_VINFO_ORIG_MAX_VECT_FACTOR): ...this new macro. * tree-vect-data-refs.c (vect_analyze_data_ref_dependences): Update accordingly. * tree-vect-loop.c (_loop_vec_info::_loop_vec_info): Initialize max_vectorization_factor. (vect_analyze_loop_2): Set LOOP_VINFO_MAX_VECT_FACTOR. Index: gcc/tree-vectorizer.h =================================================================== --- gcc/tree-vectorizer.h 2017-09-14 11:28:27.080519923 +0100 +++ gcc/tree-vectorizer.h 2017-09-14 11:30:06.064254417 +0100 @@ -241,6 +241,10 @@ typedef struct _loop_vec_info : public v /* Unrolling factor */ int vectorization_factor; + /* Maximum runtime vectorization factor, or MAX_VECTORIZATION_FACTOR + if there is no particular limit. */ + unsigned HOST_WIDE_INT max_vectorization_factor; + /* Unknown DRs according to which loop was peeled. */ struct data_reference *unaligned_dr; @@ -355,6 +359,7 @@ #define LOOP_VINFO_NITERS_ASSUMPTIONS(L) #define LOOP_VINFO_COST_MODEL_THRESHOLD(L) (L)->th #define LOOP_VINFO_VECTORIZABLE_P(L) (L)->vectorizable #define LOOP_VINFO_VECT_FACTOR(L) (L)->vectorization_factor +#define LOOP_VINFO_MAX_VECT_FACTOR(L) (L)->max_vectorization_factor #define LOOP_VINFO_PTR_MASK(L) (L)->ptr_mask #define LOOP_VINFO_LOOP_NEST(L) (L)->loop_nest #define LOOP_VINFO_DATAREFS(L) (L)->datarefs @@ -400,8 +405,8 @@ #define LOOP_VINFO_NITERS_KNOWN_P(L) #define LOOP_VINFO_EPILOGUE_P(L) \ (LOOP_VINFO_ORIG_LOOP_INFO (L) != NULL) -#define LOOP_VINFO_ORIG_VECT_FACTOR(L) \ - (LOOP_VINFO_VECT_FACTOR (LOOP_VINFO_ORIG_LOOP_INFO (L))) +#define LOOP_VINFO_ORIG_MAX_VECT_FACTOR(L) \ + (LOOP_VINFO_MAX_VECT_FACTOR (LOOP_VINFO_ORIG_LOOP_INFO (L))) static inline loop_vec_info loop_vec_info_for_loop (struct loop *loop) Index: gcc/tree-vect-data-refs.c =================================================================== --- gcc/tree-vect-data-refs.c 2017-09-14 11:29:19.649870912 +0100 +++ gcc/tree-vect-data-refs.c 2017-09-14 11:30:06.063347272 +0100 @@ -509,7 +509,7 @@ vect_analyze_data_ref_dependences (loop_ was applied to original loop. Therefore we may just get max_vf using VF of original loop. */ if (LOOP_VINFO_EPILOGUE_P (loop_vinfo)) - *max_vf = LOOP_VINFO_ORIG_VECT_FACTOR (loop_vinfo); + *max_vf = LOOP_VINFO_ORIG_MAX_VECT_FACTOR (loop_vinfo); else FOR_EACH_VEC_ELT (LOOP_VINFO_DDRS (loop_vinfo), i, ddr) if (vect_analyze_data_ref_dependence (ddr, loop_vinfo, max_vf)) Index: gcc/tree-vect-loop.c =================================================================== --- gcc/tree-vect-loop.c 2017-09-14 11:28:27.079519923 +0100 +++ gcc/tree-vect-loop.c 2017-09-14 11:30:06.064254417 +0100 @@ -1111,6 +1111,7 @@ _loop_vec_info::_loop_vec_info (struct l num_iters_assumptions (NULL_TREE), th (0), vectorization_factor (0), + max_vectorization_factor (0), unaligned_dr (NULL), peeling_for_alignment (0), ptr_mask (0), @@ -1920,6 +1921,7 @@ vect_analyze_loop_2 (loop_vec_info loop_ "bad data dependence.\n"); return false; } + LOOP_VINFO_MAX_VECT_FACTOR (loop_vinfo) = max_vf; ok = vect_determine_vectorization_factor (loop_vinfo); if (!ok)