From patchwork Mon Jul 9 16:38:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Richard Earnshaw \(lists\)" X-Patchwork-Id: 141471 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp2872430ljj; Mon, 9 Jul 2018 09:40:33 -0700 (PDT) X-Google-Smtp-Source: AAOMgpeTWx6spqCIkDA2N+Kf/4lrWIrfh4kJP20G5c5E/fMYocagA1aOhcIrFRf9zetYX9RZxOkd X-Received: by 2002:a62:da07:: with SMTP id c7-v6mr22025388pfh.106.1531154433257; Mon, 09 Jul 2018 09:40:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531154433; cv=none; d=google.com; s=arc-20160816; b=Bk20DfKT8+EbB8UNY8Ov9RSDZBp5h3UVSIXIBalBlko3gi+Fbt6RcHbzSCMloOT57w gybBt0LJAe3HERBL737PlBlSSgVExQULaqs0PdmMFsAqVksvzOYMj5FhFHEF9bPJ6J+Z DuThw96Lo0yt89hQKXyq8obORT/N24VnV5bzZK2/h4V+ZDuCkfY+3Fkh1dWS+u6/5DoG igk6ri44ObX326U/AgpZ6DSfG2+AngD47KJWH0XBuWL6BK7hDPxvb8K3vrBRhwm2mIXq ieKgxBelco0C78C/fhMIOn5DTYZmWkslwrpAzZc+ucC3rfi0efspetqDuN2zPvgtJ0EO qtog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=FpJM0xqBkPDWHF9HHuR6UArIFGiauFAe09HCYtzIAVc=; b=SOTbji2VXZ2/NXWZucsk/MN2hs465WrFgwR5VzxdQnQUbJ9w0TQCVbH7SjYYGs8Bhn XMW05UcfcUL6hN/ZDtQxZLykplMPsnXDmlcZTEJluYOpyW52FiZdjL9KBpS24lfhfJGD nc42I6amc/J+iKHEshm2MDwhJxaG5fup5HH2Hw09eEldcYPee+tOqcRkD+iTmPeERdY4 ghP+JmK3gIddzm8RTGuhtzERIrZTmyYyizc/CXLnVwp9Aund/4nsf6vjhXizn/KXzDuj LAeEBUjgvZzlgu0EMpEtdGNwvrjTqLevF7bEczGFXYQgC/a0VR6xd4PndQD2aUK+TAVc QJXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=BlWPstOT; spf=pass (google.com: domain of gcc-patches-return-481231-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-481231-patch=linaro.org@gcc.gnu.org" Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id t16-v6si13934613pga.442.2018.07.09.09.40.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 09 Jul 2018 09:40:33 -0700 (PDT) Received-SPF: pass (google.com: domain of gcc-patches-return-481231-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=BlWPstOT; spf=pass (google.com: domain of gcc-patches-return-481231-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="gcc-patches-return-481231-patch=linaro.org@gcc.gnu.org" DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type; q=dns; s=default; b=rIeWJ9iugSgBAUn1 X73xbkR9P56nFeku8VSmUq/h3+Nv/X7fhcQsjiDKl8Ej5IL4xQJVKwkJKN5EB2Km LuRl36oHIYQKtyHEEU9xFSG329CRjiPMqR1aR2qW3OZeHOIebsxvzLEDrihgnX5S TkCB0bG0/5jV9hzsHAE4UYBjC1I= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-type; s=default; bh=VY3LNvFwtzFqdVoD1pwIR8 ol+pg=; b=BlWPstOTbgxIyOvRQkAj4F7zu2zixMjSxBsIkmvc1InFSqTrCrIYaa mEX1HME8pQ2R1tfH7TO6yllw3VLZgZgKGgQjbIJZ8Bl5dconvirenJzNL9WBlEvq TON5h4IDvA8qVzWrYYlhM2bFGq1Zzs38zzm7YugqT14+qKTA7PUOc= Received: (qmail 90769 invoked by alias); 9 Jul 2018 16:39:22 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 90641 invoked by uid 89); 9 Jul 2018 16:39:21 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS autolearn=ham version=3.3.2 spammy=falling, tracker X-HELO: foss.arm.com Received: from usa-sjc-mx-foss1.foss.arm.com (HELO foss.arm.com) (217.140.101.70) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Mon, 09 Jul 2018 16:39:19 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C98CA16A3; Mon, 9 Jul 2018 09:39:17 -0700 (PDT) Received: from e120077-lin.cambridge.arm.com (e120077-lin.cambridge.arm.com [10.2.206.23]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0EBFF3F589; Mon, 9 Jul 2018 09:39:16 -0700 (PDT) From: Richard Earnshaw To: gcc-patches@gcc.gnu.org Cc: Richard Earnshaw , richard.earnshaw@arm.com Subject: [PATCH 7/7] AArch64 - use CSDB based sequences if speculation tracking is enabled Date: Mon, 9 Jul 2018 17:38:19 +0100 Message-Id: <1531154299-28349-8-git-send-email-Richard.Earnshaw@arm.com> In-Reply-To: <1531154299-28349-1-git-send-email-Richard.Earnshaw@arm.com> References: <1531154299-28349-1-git-send-email-Richard.Earnshaw@arm.com> MIME-Version: 1.0 In this final patch, now that we can track speculation through conditional branches, we can use this information to use a less expensive CSDB based speculation barrier. * config/aarch64/iterators.md (ALLI_TI): New iterator. * config/aarch64/aarch64.md (despeculate_copy): New expand. (despeculate_copy_insn): New insn. (despeculate_copyti_insn): New insn. (despeculate_simple): New insn (despeculate_simpleti): New insn. * config/aarch64/aarch64.c (aarch64_speculation_safe_value): New function. (TARGET_SPECULATION_SAFE_VALUE): Redefine to aarch64_speculation_safe_value. --- gcc/config/aarch64/aarch64.c | 42 ++++++++++++++++++ gcc/config/aarch64/aarch64.md | 96 +++++++++++++++++++++++++++++++++++++++++ gcc/config/aarch64/iterators.md | 3 ++ 3 files changed, 141 insertions(+) diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index b11d768..b30b857 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -17648,6 +17648,45 @@ aarch64_select_early_remat_modes (sbitmap modes) } } +/* Override the default target speculation_safe_value. */ +static rtx +aarch64_speculation_safe_value (machine_mode mode, + rtx result, rtx val, rtx failval) +{ + /* Maybe we should warn if falling back to hard barriers. They are + likely to be noticably more expensive than the alternative below. */ + if (!aarch64_track_speculation) + return default_speculation_safe_value (mode, result, val, failval); + + if (!REG_P (val)) + val = copy_to_mode_reg (mode, val); + + if (!aarch64_reg_or_zero (failval, mode)) + failval = copy_to_mode_reg (mode, failval); + + switch (mode) + { + case E_QImode: + emit_insn (gen_despeculate_copyqi (result, val, failval)); + break; + case E_HImode: + emit_insn (gen_despeculate_copyhi (result, val, failval)); + break; + case E_SImode: + emit_insn (gen_despeculate_copysi (result, val, failval)); + break; + case E_DImode: + emit_insn (gen_despeculate_copydi (result, val, failval)); + break; + case E_TImode: + emit_insn (gen_despeculate_copyti (result, val, failval)); + break; + default: + gcc_unreachable (); + } + return result; +} + /* Target-specific selftests. */ #if CHECKING_P @@ -18117,6 +18156,9 @@ aarch64_libgcc_floating_mode_supported_p #undef TARGET_SELECT_EARLY_REMAT_MODES #define TARGET_SELECT_EARLY_REMAT_MODES aarch64_select_early_remat_modes +#undef TARGET_SPECULATION_SAFE_VALUE +#define TARGET_SPECULATION_SAFE_VALUE aarch64_speculation_safe_value + #if CHECKING_P #undef TARGET_RUN_TARGET_SELFTESTS #define TARGET_RUN_TARGET_SELFTESTS selftest::aarch64_run_selftests diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md index 528d03d..cbcada2 100644 --- a/gcc/config/aarch64/aarch64.md +++ b/gcc/config/aarch64/aarch64.md @@ -6129,6 +6129,102 @@ (define_insn "speculation_barrier" (set_attr "speculation_barrier" "true")] ) +;; Support for __builtin_speculation_safe_value when we have speculation +;; tracking enabled. Use the speculation tracker to decide whether to +;; copy operand 1 to the target, or to copy the fail value (operand 2). +(define_expand "despeculate_copy" + [(set (match_operand:ALLI_TI 0 "register_operand" "=r") + (unspec_volatile:ALLI_TI + [(match_operand:ALLI_TI 1 "register_operand" "r") + (match_operand:ALLI_TI 2 "aarch64_reg_or_zero" "rZ") + (use (reg:DI SPECULATION_TRACKER_REGNUM)) + (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))] + "" + " + { + if (operands[2] == const0_rtx) + { + rtx tracker; + if (mode == TImode) + tracker = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM); + else + tracker = gen_rtx_REG (mode, SPECULATION_TRACKER_REGNUM); + + emit_insn (gen_despeculate_simple (operands[0], operands[1], + tracker)); + DONE; + } + } + " +) + +;; Pattern to match despeculate_copy +(define_insn "*despeculate_copy_insn" + [(set (match_operand:ALLI 0 "register_operand" "=r") + (unspec_volatile:ALLI + [(match_operand:ALLI 1 "register_operand" "r") + (match_operand:ALLI 2 "aarch64_reg_or_zero" "rZ") + (use (reg:DI SPECULATION_TRACKER_REGNUM)) + (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))] + "" + { + operands[3] = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM); + output_asm_insn ("cmp\\t%3, #0\;csel\\t%0, %1, %2, ne\;csdb", + operands); + return ""; + } + [(set_attr "length" "12") + (set_attr "type" "block") + (set_attr "speculation_barrier" "true")] +) + +;; Pattern to match despeculate_copyti +(define_insn "*despeculate_copyti_insn" + [(set (match_operand:TI 0 "register_operand" "=r") + (unspec_volatile:TI + [(match_operand:TI 1 "register_operand" "r") + (match_operand:TI 2 "aarch64_reg_or_zero" "rZ") + (use (reg:DI SPECULATION_TRACKER_REGNUM)) + (clobber (reg:CC CC_REGNUM))] UNSPECV_SPECULATION_BARRIER))] + "" + { + operands[3] = gen_rtx_REG (DImode, SPECULATION_TRACKER_REGNUM); + output_asm_insn + ("cmp\\t%3, #0\;csel\\t%0, %1, %2, ne\;csel\\t%R0, %R1, %R2, ne\;csdb", + operands); + return ""; + } + [(set_attr "length" "16") + (set_attr "type" "block") + (set_attr "speculation_barrier" "true")] +) + +(define_insn "despeculate_simple" + [(set (match_operand:ALLI 0 "register_operand" "=r") + (unspec_volatile:ALLI + [(match_operand:ALLI 1 "register_operand" "r") + (use (match_operand:ALLI 2 "register_operand" ""))] + UNSPECV_SPECULATION_BARRIER))] + "" + "and\\t%0, %1, %2\;csdb" + [(set_attr "type" "block") + (set_attr "length" "8") + (set_attr "speculation_barrier" "true")] +) + +(define_insn "despeculate_simpleti" + [(set (match_operand:TI 0 "register_operand" "=r") + (unspec_volatile:TI + [(match_operand:TI 1 "register_operand" "r") + (use (match_operand:DI 2 "register_operand" ""))] + UNSPECV_SPECULATION_BARRIER))] + "" + "and\\t%0, %1, %2\;and\\t%R0, %R1, %2\;csdb" + [(set_attr "type" "block") + (set_attr "length" "12") + (set_attr "speculation_barrier" "true")] +) + ;; AdvSIMD Stuff (include "aarch64-simd.md") diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md index 949fd2a..d2b7fff 100644 --- a/gcc/config/aarch64/iterators.md +++ b/gcc/config/aarch64/iterators.md @@ -35,6 +35,9 @@ (define_mode_iterator SHORT [QI HI]) ;; Iterator for all integer modes (up to 64-bit) (define_mode_iterator ALLI [QI HI SI DI]) +;; Iterator for all integer modes (up to 128-bit) +(define_mode_iterator ALLI_TI [QI HI SI DI TI]) + ;; Iterator for all integer modes that can be extended (up to 64-bit) (define_mode_iterator ALLX [QI HI SI])