From patchwork Fri Sep 15 01:33:28 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kugan Vivekanandarajah X-Patchwork-Id: 112670 Delivered-To: patch@linaro.org Received: by 10.140.106.117 with SMTP id d108csp72953qgf; Thu, 14 Sep 2017 18:33:46 -0700 (PDT) X-Received: by 10.101.88.131 with SMTP id d3mr10209035pgu.209.1505439225967; Thu, 14 Sep 2017 18:33:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505439225; cv=none; d=google.com; s=arc-20160816; b=N588Ze3gzHKt308/kDHUPWtNE/uzos64j8pKK4vG2HaN9K6yfZlltJeii6bBPi4hSi Idde71K5ql5AZuJCc//V5B9xJhn9b0F7hWJ+uB+yibVcYdAux9nJbG1q71/upjaqMCRt mob77xFVwHO8v71XSvdPQkvA6h5Mkd4A2EgZORwaPK/s6AmY6EKsl0J9zvlqeMEICgt4 SI7RGchOjis7SZuFoz+HXYrSTHHNs3xGueegVII3lxL+hpxcq73wrxR6rbsfSMH55gOH 6e1HsDvcU6BU15gh1fjRGF8VGen5UTeEFe5xlXljcF7fF0JksAekb1xn7VBd440/v2Ha nsSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=to:subject:message-id:date:from:mime-version:delivered-to:sender :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:mailing-list:dkim-signature:domainkey-signature :arc-authentication-results; bh=UPKoBomRH9DpKoyyDXx1OccyzbaZXwsBuC1CczPaTl0=; b=osIujfxjcQEzE/mr7uKKnhYV2qhETaGnZuROcrsH7nJDu4sNu+WEDwRhE7mLFKs72B Iodn+01zpMoWAMgwGdKGSVX4cDCO8AD4hHt0uDVI/IungDjlc00oPgNPMCHlhPYb0etF RghGVk2aJOmtFIVNZ8S/YQ110VI32B232Sp/ZYCgwvBiUfdb79ILKTPiwJGyEHur7Sel s3K7ktSoP55wK9/21HxSljxEe2Fh6LpVKHYfSU+5XTf69mKDRcpJyqNUwWZZGIq3D4/i s3UkQIDgRP5lb+BDP29x1Sbzis+8AZB6IHWVE8uwq2G2NVsX8xik036qXQDCZtcMEWS1 L+Og== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=kc7n4hK8; spf=pass (google.com: domain of gcc-patches-return-462193-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-462193-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id m12si12985499pln.643.2017.09.14.18.33.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 14 Sep 2017 18:33:45 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-462193-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=kc7n4hK8; spf=pass (google.com: domain of gcc-patches-return-462193-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-462193-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:from:date:message-id:subject:to:content-type; q= dns; s=default; b=ZROEOgmFwtShwNz5R8YyHvd4FIXZkQxSzISUS3snyPTYAh fEwxENL8NgbfvDpYUfZ6aiaZ0DfejAi5tLu4ALqvMKMUgPNKsQXXZjkUibUHdOJH IH8oxBGjnxdeo1CdjqnNyr2izxha/373jHAxwuYSmYfD51u2kkjVPlCgh1rvs= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender :mime-version:from:date:message-id:subject:to:content-type; s= default; bh=S4uBgiqx1VMBhVx94dTvpPJog1Q=; b=kc7n4hK8I/DcRtZVtr/A Mv8AKJyppHsnxVZMkarv/w8XxCK88orUem39yvWMIEBh7O4uCVNC2FFMNjqx2WWZ 4Y3ScVSj6FRATv4c3pKjR1WDrakHXeMjwHAIAU0GxLyT8o1r59K3YcsCRNDnDh10 AwssMZMENdH3vtWn2PIdkf8= Received: (qmail 115753 invoked by alias); 15 Sep 2017 01:33:33 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 115744 invoked by uid 89); 15 Sep 2017 01:33:33 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-24.4 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM, SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-qt0-f175.google.com Received: from mail-qt0-f175.google.com (HELO mail-qt0-f175.google.com) (209.85.216.175) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 15 Sep 2017 01:33:31 +0000 Received: by mail-qt0-f175.google.com with SMTP id s18so1004889qta.3 for ; Thu, 14 Sep 2017 18:33:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=4oL5zEPTSVijPy+W3ivPCfh4qXJ/CxbylyiQoEfU5tU=; b=JN4rtPQ+rGCO6O5QMCChXjtp+qSpYpHrSnyuCVdeMRP2IXG1Ay96vUMvHnlUh1hiow ygiviMoYylp+MX4qy8vtywCb0sAL6PGA15QmwbxvH9luRdcNRL+L/2iykywxquAoR2YY mUgmQacpcFZZB5DyN6zhe/LRp/6pS21H7El7rF5a7zjpp5GntLJXqc8V/gikNFst4Mp6 LMzte0/wzLWGmzD5xQjt8p4xq7Pbes8nE0Nzu3MD4IHzQCgdiZXJ/OwYvl3smS6B+2ko Mj2YPX49j1WL8Oc2fFazsLhtxLOtbaBGIauh5QFaEyN0z59P6q3wIzgB6+HvSnaGr3BN 0eIw== X-Gm-Message-State: AHPjjUgDmtZFvvKXlUtGlt6pg8675W7mDGGnEFVbfg5rYdwTOf7PFf7I Par5iQBNnFvIEgPIu/W9JBdf0hdpD2SuN9WqQWfhSqZXYv4= X-Google-Smtp-Source: AOwi7QCjHQKUrFQlK0aYsNBaOFwL4XdfUswt7geiML8Gif6gxUmrCxcJGUQlmrSrcWd9Fpo5CCiXEMONASg6dSw5lyE= X-Received: by 10.237.37.228 with SMTP id y33mr25193921qtc.79.1505439210034; Thu, 14 Sep 2017 18:33:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.237.37.211 with HTTP; Thu, 14 Sep 2017 18:33:28 -0700 (PDT) From: Kugan Vivekanandarajah Date: Fri, 15 Sep 2017 11:33:28 +1000 Message-ID: Subject: [RFC][AARCH64][PATCH 5/5] add aarch64_loop_unroll_adjust to limit partial unrolling in rtl based on strided-loads in loop To: "gcc-patches@gcc.gnu.org" X-IsSubscribed: yes This patch adds aarch64_loop_unroll_adjust to limit partial unrolling in rtl based on strided-loads in loop. Thanks, Kugan gcc/ChangeLog: 2017-09-12 Kugan Vivekanandarajah * cfgloop.h (iv_analyze_biv): export. * loop-iv.c: Likewise. * config/aarch64/aarch64.c (strided_load_p): New. (insn_has_strided_load): New. (count_strided_load_rtl): New. (aarch64_loop_unroll_adjust): New. >From 10e02b026784798fff6a3513dc11b1cffb1cf78a Mon Sep 17 00:00:00 2001 From: Kugan Vivekanandarajah Date: Wed, 23 Aug 2017 12:35:14 +1000 Subject: [PATCH 5/5] add aarch64_loop_unroll_adjust --- gcc/cfgloop.h | 1 + gcc/config/aarch64/aarch64.c | 136 +++++++++++++++++++++++++++++++++++++++++++ gcc/loop-iv.c | 2 +- 3 files changed, 138 insertions(+), 1 deletion(-) diff --git a/gcc/cfgloop.h b/gcc/cfgloop.h index 2308e7a..a3876a2 100644 --- a/gcc/cfgloop.h +++ b/gcc/cfgloop.h @@ -479,6 +479,7 @@ extern bool iv_analyze_expr (rtx_insn *, rtx, machine_mode, extern rtx get_iv_value (struct rtx_iv *, rtx); extern bool biv_p (rtx_insn *, rtx); extern void find_simple_exit (struct loop *, struct niter_desc *); +extern bool iv_analyze_biv (rtx def, struct rtx_iv *iv); extern void iv_analysis_done (void); extern struct niter_desc *get_simple_loop_desc (struct loop *loop); diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index e88bb6c..624a996 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -15189,6 +15189,139 @@ aarch64_ok_to_unroll (struct loop *loop, unsigned HOST_WIDE_INT nunroll) return true; } +/* Return true if X is a strided load. */ + +static bool +strided_load_p (const_rtx x) +{ + struct rtx_iv iv; + rtx reg; + + if (!MEM_P (x)) + return false; + + reg = XEXP (x, 0); + if (REG_P (reg) + || UNARY_P (reg)) + { + if (!REG_P (reg)) + reg = XEXP (reg, 0); + if (REG_P (reg) + && iv_analyze_biv (reg, &iv)) + return true; + } + else if (BINARY_P (reg)) + { + rtx reg1, reg2; + reg1 = XEXP (reg, 0); + reg2 = XEXP (reg, 1); + if (REG_P (reg1) + && iv_analyze_biv (reg1, &iv)) + return true; + if (REG_P (reg2) + && iv_analyze_biv (reg2, &iv)) + return true; + } + return false; +} + + +/* Return true if X INSN is a strided load. */ + +static bool +insn_has_strided_load (rtx_insn *insn) +{ + subrtx_iterator::array_type array; + if (!INSN_P (insn) || recog_memoized (insn) < 0) + return false; + rtx pat = PATTERN (insn); + + switch (GET_CODE (pat)) + { + case PARALLEL: + { + for (int j = 0; j < XVECLEN (pat, 0); ++j) + { + rtx ex = XVECEXP (pat, 0, j); + FOR_EACH_SUBRTX (iter, array, ex, NONCONST) + { + const_rtx x = *iter; + if (GET_CODE (x) == SET + && strided_load_p (SET_SRC (x))) + return true; + } + } + } + break; + + case SET: + FOR_EACH_SUBRTX (iter, array, SET_SRC (pat), NONCONST) + { + const_rtx x = *iter; + if (strided_load_p (x)) + return true; + } + + default: + break; + } + return false; +} + +/* Count the strided loads in the LOOP. If the strided loads are larger + (compared to MAX_STRIDED_LOADS), we dont need to compute all of + them. This is used to limit the partial unrolling factor to avoid + prefetcher collision. */ + +static unsigned +count_strided_load_rtl (struct loop *loop, unsigned max_strided_loads) +{ + basic_block *bbs; + unsigned count = 0; + rtx_insn *insn; + iv_analysis_loop_init (loop); + bbs = get_loop_body (loop); + + for (unsigned i = 0; i < loop->num_nodes; ++i) + { + FOR_BB_INSNS (bbs[i], insn) + { + if (insn_has_strided_load (insn)) + count ++; + + if (count > (max_strided_loads / 2)) + { + free (bbs); + iv_analysis_done (); + return count; + } + } + } + free (bbs); + iv_analysis_done (); + return count; +} + +/* Target hook loop_unroll_adjust that limits partial loop unrolling + factor, if this would make the outer loop's prefetch streams more + than hardware can handle. */ + +static unsigned +aarch64_loop_unroll_adjust (unsigned n_unroll, struct loop *loop) +{ + int max_strided_loads; + max_strided_loads = aarch64_tune_params.prefetch->hw_prefetchers_avail; + + if (max_strided_loads == -1) + return n_unroll; + + unsigned count = count_strided_load_rtl (loop, max_strided_loads); + if (count > 0) + n_unroll = 1 << (floor_log2 (max_strided_loads/count)); + + return n_unroll; +} + /* Target-specific selftests. */ #if CHECKING_P @@ -15620,6 +15753,9 @@ aarch64_libgcc_floating_mode_supported_p #undef TARGET_OK_TO_UNROLL #define TARGET_OK_TO_UNROLL aarch64_ok_to_unroll +#undef TARGET_LOOP_UNROLL_ADJUST +#define TARGET_LOOP_UNROLL_ADJUST aarch64_loop_unroll_adjust + #if CHECKING_P #undef TARGET_RUN_TARGET_SELFTESTS #define TARGET_RUN_TARGET_SELFTESTS selftest::aarch64_run_selftests diff --git a/gcc/loop-iv.c b/gcc/loop-iv.c index 745b613..3a8c54e 100644 --- a/gcc/loop-iv.c +++ b/gcc/loop-iv.c @@ -852,7 +852,7 @@ record_biv (rtx def, struct rtx_iv *iv) /* Determines whether DEF is a biv and if so, stores its description to *IV. */ -static bool +bool iv_analyze_biv (rtx def, struct rtx_iv *iv) { rtx inner_step, outer_step; -- 2.7.4