From patchwork Fri Jul 27 09:37:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Richard Earnshaw \(lists\)" X-Patchwork-Id: 143025 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp600177ljj; Fri, 27 Jul 2018 02:40:44 -0700 (PDT) X-Google-Smtp-Source: AAOMgpfjd6q74wxGAJ3VCRetzBYu0T2B2L5AdtYw5kFLbtH2AXPS65F6W438eKrrQDxw6AspHhb+ X-Received: by 2002:a63:8c51:: with SMTP id q17-v6mr5525507pgn.236.1532684444875; Fri, 27 Jul 2018 02:40:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1532684444; cv=none; d=google.com; s=arc-20160816; b=D5lHVZlIBdrm2ZexaMeMIDUl7g4Jd8lRIo3SsHiX6FH3J+MbE1ytiYQ0o+DlXsp9qt 3GF6+F0sKSvK9Il2YAerbXPTnzKe3sfZwhwGs2LBAoQ+TSeaY0f17tJ485o68W4P2tLI OGUOO1GFy/I/jd+ffR8tObT08mwJqoisx0/Vb+ofm/VJrKbEzHVtqt+0FgunRrz34Pf+ 8sqP88FpgeFR5hTCEqxC8YwjCPm/jO7SYy7/Fb++FIV6bg6KAsISZJi3a7Z03BAnLoQi c6ApKTxxz5DS0NChc8m9WJwDXSHw6p8UvUurNUfh5K8O3GjN2WAt6cwfYaS9N1QIser1 1jeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=4/pAfxltGbLmiFNxVdiCJd4cT4lGGlEBErVX+jbkJdc=; b=GUl+lJMfVCKnBA31+iEvp0pk8FuvV2H9qFpQI5wM0pU1WUgm202IqwxxsRNsDkyxpW 67PjR//qvyI40XcuGwt0xku7X2q2AEFfJEVF94twDyn8om3XN5JdGF46GrgHEQm8Vb1W msUwk4MGBmkj5J4Y/NFNwaJH3vpJdiiDfzcAoDU9PYp93L8ycqeaYAa8auJETRt1AL6c tkzLFH7ILa2tQS5l6le0AjsdE5eCkAMNxTwQzrXuWUDFMbZUd+82LVSCHLusqVTMcQco 3vDT5OOWv3SRUeblZ4phNNsOXfm5VokykMIHxpwwQ01ZNGdHz77tLVA5zVci0q5lLNys MeYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=RZw0UWaq; spf=pass (google.com: domain of gcc-patches-return-482511-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-482511-patch=linaro.org@gcc.gnu.org" Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id d20-v6si3150697pls.477.2018.07.27.02.40.44 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jul 2018 02:40:44 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-482511-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=RZw0UWaq; spf=pass (google.com: domain of gcc-patches-return-482511-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-482511-patch=linaro.org@gcc.gnu.org" DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type; q=dns; s=default; b=rGTFZN1aYFpLZvM8 MF4HF9AhKYd2G0ZmKfi3jpg6YN3W0guncTxcDdkbas0Gl2iJWrazgE7pt2kNaK8F kxpUg9FgmepRgtf7ggIsjYDSdXZKQGFbBpN3EFCsPkY0yIh9BU3fd2gOsx2Xtf0y VG3kN5oXqGySwXydPsJyqOvdiM4= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type; s=default; bh=khxVvZ4VBSs63V6+E9vHOH S9fww=; b=RZw0UWaqPouj5aqHb8bDvMwU8Z4gQTiycpflrhDTcJSroAicd7tojj A0yXbG5QLMzXnQXYAUnMv8uQf8yaeM9PiUuQ4jYfJxPWg8Q6YY6opPSerdOwiu9A 7Xf4srMIqnVDhEko/sCf45stHCaVeQPbNxmGn0GbeCcAzciwPJacQ= Received: (qmail 48235 invoked by alias); 27 Jul 2018 09:38:40 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 48009 invoked by uid 89); 27 Jul 2018 09:38:39 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_FAIL autolearn=ham version=3.3.2 spammy= X-HELO: eggs.gnu.org Received: from eggs.gnu.org (HELO eggs.gnu.org) (208.118.235.92) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Fri, 27 Jul 2018 09:38:36 +0000 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fizCg-0006D5-JI for gcc-patches@gcc.gnu.org; Fri, 27 Jul 2018 05:38:34 -0400 Received: from foss.arm.com ([217.140.101.70]:47238) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fizCg-0006CH-74 for gcc-patches@gcc.gnu.org; Fri, 27 Jul 2018 05:38:30 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 87F561682; Fri, 27 Jul 2018 02:38:29 -0700 (PDT) Received: from e120077-lin.cambridge.arm.com (e120077-lin.cambridge.arm.com [10.2.207.74]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E46AF3F575; Fri, 27 Jul 2018 02:38:28 -0700 (PDT) From: Richard Earnshaw To: gcc-patches@gcc.gnu.org Cc: Richard Earnshaw Subject: [PATCH 07/11] AArch64 - use CSDB based sequences if speculation tracking is enabled Date: Fri, 27 Jul 2018 10:37:51 +0100 Message-Id: <1532684275-13041-8-git-send-email-Richard.Earnshaw@arm.com> In-Reply-To: <1532684275-13041-1-git-send-email-Richard.Earnshaw@arm.com> References: <1531154299-28349-1-git-send-email-Richard.Earnshaw@arm.com> <1532684275-13041-1-git-send-email-Richard.Earnshaw@arm.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 217.140.101.70 In this final patch, now that we can track speculation through conditional branches, we can use this information to use a less expensive CSDB based speculation barrier. * config/aarch64/iterators.md (ALLI_TI): New iterator. * config/aarch64/aarch64.md (despeculate_copy): New expand. (despeculate_copy_insn): New insn. (despeculate_copyti_insn): New insn. (despeculate_simple): New insn (despeculate_simpleti): New insn. * config/aarch64/aarch64.c (aarch64_speculation_safe_value): New function. (TARGET_SPECULATION_SAFE_VALUE): Redefine to aarch64_speculation_safe_value. (aarch64_print_operand): Handle const0_rtx in modifier 'H'. --- gcc/config/aarch64/aarch64.c | 48 ++++++++++++++++++++ gcc/config/aarch64/aarch64.md | 97 +++++++++++++++++++++++++++++++++++++++++ gcc/config/aarch64/iterators.md | 3 ++ 3 files changed, 148 insertions(+) diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index cca465e..fc6eb1c 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -6760,6 +6760,12 @@ aarch64_print_operand (FILE *f, rtx x, int code) break; case 'H': + if (x == const0_rtx) + { + asm_fprintf (f, "xzr"); + break; + } + if (!REG_P (x) || !GP_REGNUM_P (REGNO (x) + 1)) { output_operand_lossage ("invalid operand for '%%%c'", code); @@ -17638,6 +17644,45 @@ aarch64_select_early_remat_modes (sbitmap modes) } } +/* Override the default target speculation_safe_value. */ +static rtx +aarch64_speculation_safe_value (machine_mode mode, + rtx result, rtx val, rtx failval) +{ + /* Maybe we should warn if falling back to hard barriers. They are + likely to be noticably more expensive than the alternative below. */ + if (!aarch64_track_speculation) + return default_speculation_safe_value (mode, result, val, failval); + + if (!REG_P (val)) + val = copy_to_mode_reg (mode, val); + + if (!aarch64_reg_or_zero (failval, mode)) + failval = copy_to_mode_reg (mode, failval); + + switch (mode) + { + case E_QImode: + emit_insn (gen_despeculate_copyqi (result, val, failval)); + break; + case E_HImode: + emit_insn (gen_despeculate_copyhi (result, val, failval)); + break; + case E_SImode: + emit_insn (gen_despeculate_copysi (result, val, failval)); + break; + case E_DImode: + emit_insn (gen_despeculate_copydi (result, val, failval)); + break; + case E_TImode: + emit_insn (gen_despeculate_copyti (result, val, failval)); + break; + default: + gcc_unreachable (); + } + return result; +} + /* Target-specific selftests. */ #if CHECKING_P @@ -18110,6 +18155,9 @@ aarch64_libgcc_floating_mode_supported_p #undef TARGET_SELECT_EARLY_REMAT_MODES #define TARGET_SELECT_EARLY_REMAT_MODES aarch64_select_early_remat_modes +#undef TARGET_SPECULATION_SAFE_VALUE +#define TARGET_SPECULATION_SAFE_VALUE aarch64_speculation_safe_value + #if CHECKING_P #undef TARGET_RUN_TARGET_SELFTESTS #define TARGET_RUN_TARGET_SELFTESTS selftest::aarch64_run_selftests diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md index 528d03d..321a674 100644 --- a/gcc/config/aarch64/aarch64.md +++ b/gcc/config/aarch64/aarch64.md @@ -6129,6 +6129,103 @@ (define_insn "speculation_barrier" (set_attr "speculation_barrier" "true")] ) +;; Support for __builtin_speculation_safe_value when we have speculation +;; tracking enabled. Use the speculation tracker to decide whether to +;; copy operand 1 to the target, or to copy the fail value (operand 2). +(define_expand "despeculate_copy" + [(set (match_operand:ALLI_TI 0 "register_operand" "=r") + (unspec_volatile:ALLI_TI + [(match_operand:ALLI_TI 1 "register_operand" "r") + (match_operand:ALLI_TI 2 "aarch64_reg_or_zero" "rZ") + (use (reg:DI SPECULATION_TRACKER_REGNUM)) + (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))] + "" + " + { + if (operands[2] == const0_rtx) + { + rtx tracker; + if (mode == TImode) + tracker = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM); + else + tracker = gen_rtx_REG (mode, SPECULATION_TRACKER_REGNUM); + + emit_insn (gen_despeculate_simple (operands[0], operands[1], + tracker)); + DONE; + } + } + " +) + +;; Patterns to match despeculate_copy. Note that "hint 0x14" is the +;; encoding for CSDB, but will work in older versions of the assembler. +(define_insn "*despeculate_copy_insn" + [(set (match_operand:ALLI 0 "register_operand" "=r") + (unspec_volatile:ALLI + [(match_operand:ALLI 1 "register_operand" "r") + (match_operand:ALLI 2 "aarch64_reg_or_zero" "rZ") + (use (reg:DI SPECULATION_TRACKER_REGNUM)) + (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))] + "" + { + operands[3] = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM); + output_asm_insn ("cmp\\t%3, #0\;csel\\t%0, %1, %2, ne\;hint\t0x14 // csdb", + operands); + return ""; + } + [(set_attr "length" "12") + (set_attr "type" "block") + (set_attr "speculation_barrier" "true")] +) + +;; Pattern to match despeculate_copyti +(define_insn "*despeculate_copyti_insn" + [(set (match_operand:TI 0 "register_operand" "=r") + (unspec_volatile:TI + [(match_operand:TI 1 "register_operand" "r") + (match_operand:TI 2 "aarch64_reg_or_zero" "rZ") + (use (reg:DI SPECULATION_TRACKER_REGNUM)) + (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))] + "" + { + operands[3] = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM); + output_asm_insn + ("cmp\\t%3, #0\;csel\\t%0, %1, %2, ne\;csel\\t%H0, %H1, %H2, ne\;hint\t0x14 // csdb", + operands); + return ""; + } + [(set_attr "length" "16") + (set_attr "type" "block") + (set_attr "speculation_barrier" "true")] +) + +(define_insn "despeculate_simple" + [(set (match_operand:ALLI 0 "register_operand" "=r") + (unspec_volatile:ALLI + [(match_operand:ALLI 1 "register_operand" "r") + (use (match_operand:ALLI 2 "register_operand" ""))] + UNSPECV_SPECULATION_BARRIER))] + "" + "and\\t%0, %1, %2\;hint\t0x14 // csdb" + [(set_attr "type" "block") + (set_attr "length" "8") + (set_attr "speculation_barrier" "true")] +) + +(define_insn "despeculate_simpleti" + [(set (match_operand:TI 0 "register_operand" "=r") + (unspec_volatile:TI + [(match_operand:TI 1 "register_operand" "r") + (use (match_operand:DI 2 "register_operand" ""))] + UNSPECV_SPECULATION_BARRIER))] + "" + "and\\t%0, %1, %2\;and\\t%H0, %H1, %2\;hint\t0x14 // csdb" + [(set_attr "type" "block") + (set_attr "length" "12") + (set_attr "speculation_barrier" "true")] +) + ;; AdvSIMD Stuff (include "aarch64-simd.md") diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md index d846118..450edea 100644 --- a/gcc/config/aarch64/iterators.md +++ b/gcc/config/aarch64/iterators.md @@ -35,6 +35,9 @@ (define_mode_iterator SHORT [QI HI]) ;; Iterator for all integer modes (up to 64-bit) (define_mode_iterator ALLI [QI HI SI DI]) +;; Iterator for all integer modes (up to 128-bit) +(define_mode_iterator ALLI_TI [QI HI SI DI TI]) + ;; Iterator for all integer modes that can be extended (up to 64-bit) (define_mode_iterator ALLX [QI HI SI])