From patchwork Thu Jan 4 13:58:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Richard Earnshaw \(lists\)" X-Patchwork-Id: 123423 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp11460761qgn; Thu, 4 Jan 2018 05:59:22 -0800 (PST) X-Google-Smtp-Source: ACJfBovSypdO/4LPF79fWj5aarXf3PY6rJTBFNDKbMJa2o5PNt2SftEUFt1zXE7WqMl1go1B6y7y X-Received: by 10.84.195.36 with SMTP id i33mr4835323pld.189.1515074362833; Thu, 04 Jan 2018 05:59:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515074362; cv=none; d=google.com; s=arc-20160816; b=pVzQyQ9LpQzd0ewh5b8ARoi32PsFLWHPdC+hXcwhU1FFrRun8V0gYC2VKcHKfTR/pP HQGPGEgdTIcYq1gMrnoMRxSCOxsmO5V0rrxQCW9Jo6qife5sY7HP81lSB61zmcKTt0B8 Zci6BL4N9ojc3vkxy0BcqvgFv8rpIdMuGgAx+96LUqvP6/2PAF466UsYFQSSfJv2D6c6 v+5KRIlbJxaIf2A9bPXTR2T3p5zzyKS1hUp5ZqVPPkb3rbOIpVyHNvTneglIKh//n/MT ZQghsB6OIsPZZtgvYia92n2EOwA3TG4K2PXumYkXjLn56Dj5Ons2GoySTzKSD9bdO5Cz dOjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:references:in-reply-to :message-id:date:subject:cc:to:from:delivered-to:sender:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mailing-list:dkim-signature:domainkey-signature :arc-authentication-results; bh=sNuMIxqxUFx9RCUpyP6MB1B55KGE+icxRGmH5il7iGA=; b=vkx5JS/gqrvyqG0v9VTmFivDoAiTsTrkzeNkPy3pA8Q2nyJ8CYrmi2MUiJ08hI3Xjm WyquqsFxcQQdKb8miotaDlNwOGZ40KHtflWw0dDaFa9M5e/pO++cZTfJQgxSjImMwpkI xENj38YI5etmcL1AeorGvTeT6HL+hNP8nWtY4uW62GoNImnepSvOsSQd8RBW/SAU2USH 7akj4gk1p9GvvBwvYJG+sLR2Ll3637e8P/y59BR1OchqZDECmQcq8Qii+v9de6OBKBw3 Q5pFdh2N8oQ8MkA/Wt4WH+cCuE9Q5X6CMYFwY25x5dKnxHa1O03b72OLmBwRHIZLlUHU Kl9w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Gp05HKRh; spf=pass (google.com: domain of gcc-patches-return-470135-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-470135-patch=linaro.org@gcc.gnu.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id z63si2365834pfl.241.2018.01.04.05.59.22 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Jan 2018 05:59:22 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-return-470135-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=Gp05HKRh; spf=pass (google.com: domain of gcc-patches-return-470135-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-470135-patch=linaro.org@gcc.gnu.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references:mime-version:content-type; q=dns; s= default; b=X65++NsHGvDg8BLD1O7+5TouUw1mDoa0IkgJ5/to3GQ52sZ6IBe4q nD9se62uMtUQoQ4MLkpKEK/wZysKscMbZMW3FDwLuw56KpsPQKg0/u4dqpTz5/QP +qUEUFnfGKMXRHLwbCJOwRfqULObSET4ztiSNbOniwUY2scgVjhOpU= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references:mime-version:content-type; s=default; bh=uz5KN9Nr4vvXV9TdZeKJT/NLlXA=; b=Gp05HKRhBTR3rbjJYjBdpmvw4ZIx tQ+bcsJAI3MxOCp5bZJcDvIbTDiYQ4gZWhhhGN+eazrJFIhBui0k6VJz9SjQ5+N2 U9Bh6ekrlZ/CgwuhQuPdq9nlI3vb/SoeogZqK02SE7d7lnqIk5V1a80tC1dnUEPm fZmAMPtZZCpPwQI= Received: (qmail 87828 invoked by alias); 4 Jan 2018 13:58:58 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 87751 invoked by uid 89); 4 Jan 2018 13:58:57 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS, T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=inhibit X-HELO: foss.arm.com Received: from usa-sjc-mx-foss1.foss.arm.com (HELO foss.arm.com) (217.140.101.70) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 04 Jan 2018 13:58:55 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A6D3915AD; Thu, 4 Jan 2018 05:58:54 -0800 (PST) Received: from e105689-lin.cambridge.arm.com (e105689-lin.cambridge.arm.com [10.2.207.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DEE143F581; Thu, 4 Jan 2018 05:58:53 -0800 (PST) From: Richard Earnshaw To: gcc-patches@gcc.gnu.org Cc: Richard Earnshaw Subject: [PATCH 2/3] [aarch64] Implement support for __builtin_load_no_speculate. Date: Thu, 4 Jan 2018 13:58:42 +0000 Message-Id: <1197003135ea0b0aeecd038a07db6b787a7db6f6.1515072356.git.Richard.Earnshaw@arm.com> In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 This patch implements support for __builtin_load_no_speculate on AArch64. On this architecture we inhibit speclation by emitting a combination of CSEL and a hint instruction that ensures the CSEL is full resolved when the operands to the CSEL may involve a speculative load. * config/aarch64/aarch64.c (aarch64_print_operand): Handle zero passed to 'H' operand qualifier. (aarch64_inhibit_load_speculation): New function. (TARGET_INHIBIT_LOAD_SPECULATION): Redefine. * config/aarch64/aarch64.md (UNSPECV_NOSPECULATE): New unspec_volatile code. (nospeculate, nospeculateti): New patterns. --- gcc/config/aarch64/aarch64.c | 92 +++++++++++++++++++++++++++++++++++++++++++ gcc/config/aarch64/aarch64.md | 28 +++++++++++++ 2 files changed, 120 insertions(+) diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index 93e9d9f9..7410921 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -5315,6 +5315,14 @@ aarch64_print_operand (FILE *f, rtx x, int code) break; case 'H': + /* Print the higher numbered register of a pair (TImode) of regs. */ + if (x == const0_rtx + || (CONST_DOUBLE_P (x) && aarch64_float_const_zero_rtx_p (x))) + { + asm_fprintf (f, "xzr"); + break; + } + if (!REG_P (x) || !GP_REGNUM_P (REGNO (x) + 1)) { output_operand_lossage ("invalid operand for '%%%c'", code); @@ -15115,6 +15123,87 @@ aarch64_sched_can_speculate_insn (rtx_insn *insn) } } +static rtx +aarch64_inhibit_load_speculation (machine_mode mode, rtx result, rtx mem, + rtx lower_bound, rtx upper_bound, + rtx fail_result, rtx cmpptr) +{ + rtx cond, comparison; + rtx target = gen_reg_rtx (mode); + rtx tgt2 = result; + + if (!register_operand (cmpptr, ptr_mode)) + cmpptr = force_reg (ptr_mode, cmpptr); + + if (!register_operand (tgt2, mode)) + tgt2 = gen_reg_rtx (mode); + + if (upper_bound == NULL) + { + if (!register_operand (lower_bound, ptr_mode)) + lower_bound = force_reg (ptr_mode, lower_bound); + + cond = aarch64_gen_compare_reg (LTU, cmpptr, lower_bound); + comparison = gen_rtx_LTU (VOIDmode, cond, const0_rtx); + } + else if (lower_bound == NULL) + { + if (!register_operand (upper_bound, ptr_mode)) + upper_bound = force_reg (ptr_mode, upper_bound); + + cond = aarch64_gen_compare_reg (GEU, cmpptr, upper_bound); + comparison = gen_rtx_GEU (VOIDmode, cond, const0_rtx); + } + else + { + if (!register_operand (lower_bound, ptr_mode)) + lower_bound = force_reg (ptr_mode, lower_bound); + + if (!register_operand (upper_bound, ptr_mode)) + upper_bound = force_reg (ptr_mode, upper_bound); + + rtx cond1 = aarch64_gen_compare_reg (GEU, cmpptr, lower_bound); + rtx comparison1 = gen_rtx_GEU (ptr_mode, cond1, const0_rtx); + rtx failcond = GEN_INT (aarch64_get_condition_code (comparison1)^1); + cond = gen_rtx_REG (CCmode, CC_REGNUM); + if (ptr_mode == SImode) + emit_insn (gen_ccmpsi (cond1, cond, cmpptr, upper_bound, comparison1, + failcond)); + else + emit_insn (gen_ccmpdi (cond1, cond, cmpptr, upper_bound, comparison1, + failcond)); + comparison = gen_rtx_GEU (VOIDmode, cond, const0_rtx); + } + + rtx_code_label *label = gen_label_rtx (); + emit_jump_insn (gen_condjump (comparison, cond, label)); + emit_move_insn (target, mem); + emit_label (label); + + insn_code icode; + + switch (mode) + { + case E_QImode: icode = CODE_FOR_nospeculateqi; break; + case E_HImode: icode = CODE_FOR_nospeculatehi; break; + case E_SImode: icode = CODE_FOR_nospeculatesi; break; + case E_DImode: icode = CODE_FOR_nospeculatedi; break; + case E_TImode: icode = CODE_FOR_nospeculateti; break; + default: + gcc_unreachable (); + } + + if (! insn_operand_matches (icode, 4, fail_result)) + fail_result = force_reg (mode, fail_result); + + emit_insn (GEN_FCN (icode) (tgt2, comparison, cond, target, fail_result)); + + if (tgt2 != result) + emit_move_insn (result, tgt2); + + return result; +} + /* Target-specific selftests. */ #if CHECKING_P @@ -15554,6 +15643,9 @@ aarch64_libgcc_floating_mode_supported_p #undef TARGET_CONSTANT_ALIGNMENT #define TARGET_CONSTANT_ALIGNMENT aarch64_constant_alignment +#undef TARGET_INHIBIT_LOAD_SPECULATION +#define TARGET_INHIBIT_LOAD_SPECULATION aarch64_inhibit_load_speculation + #if CHECKING_P #undef TARGET_RUN_TARGET_SELFTESTS #define TARGET_RUN_TARGET_SELFTESTS selftest::aarch64_run_selftests diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md index f1e2a07..1a1f398 100644 --- a/gcc/config/aarch64/aarch64.md +++ b/gcc/config/aarch64/aarch64.md @@ -153,6 +153,7 @@ UNSPECV_SET_FPSR ; Represent assign of FPSR content. UNSPECV_BLOCKAGE ; Represent a blockage UNSPECV_PROBE_STACK_RANGE ; Represent stack range probing. + UNSPECV_NOSPECULATE ; Inhibit speculation ] ) @@ -5797,6 +5798,33 @@ DONE; }) +(define_insn "nospeculate" + [(set (match_operand:ALLI 0 "register_operand" "=r") + (unspec_volatile:ALLI + [(match_operator 1 "aarch64_comparison_operator" + [(match_operand 2 "cc_register" "") (const_int 0)]) + (match_operand:ALLI 3 "register_operand" "r") + (match_operand:ALLI 4 "aarch64_reg_or_zero" "rZ")] + UNSPECV_NOSPECULATE))] + "" + "csel\\t%0, %3, %4, %M1\;hint\t#0x14\t// CSDB" + [(set_attr "type" "csel") + (set_attr "length" "8")] +) + +(define_insn "nospeculateti" + [(set (match_operand:TI 0 "register_operand" "=r") + (unspec_volatile:TI + [(match_operator 1 "aarch64_comparison_operator" + [(match_operand 2 "cc_register" "") (const_int 0)]) + (match_operand:TI 3 "register_operand" "r") + (match_operand:TI 4 "aarch64_reg_or_zero" "rZ")] + UNSPECV_NOSPECULATE))] + "" + "csel\\t%x0, %x3, %x4, %M1\;csel\\t%H0, %H3, %H4, %M1\;hint\t#0x14\t// CSDB" + [(set_attr "type" "csel") + (set_attr "length" "12")] +) ;; AdvSIMD Stuff (include "aarch64-simd.md")