From patchwork Wed Jan 17 14:55:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Richard Earnshaw \(lists\)" X-Patchwork-Id: 124810 Delivered-To: patch@linaro.org Received: by 10.46.62.1 with SMTP id l1csp70998lja; Wed, 17 Jan 2018 06:57:01 -0800 (PST) X-Google-Smtp-Source: ACJfBouMRj9MY235wJ5YqUYY3qu6dU7IzYXqlqWg9Vg4jyakZEKwYcXZxjzulAOsK11GjesbYVHt X-Received: by 10.159.230.13 with SMTP id u13mr27535545plq.114.1516201020861; Wed, 17 Jan 2018 06:57:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516201020; cv=none; d=google.com; s=arc-20160816; b=L+oKxMTdqc9IpiyY6Fp1TciKJTPkQ6bLdav3iF1HjReHnZ3pLzjysc033SyL2ncQBz hw4OVBWyFBherJTIXqoC8YP5RfJHPS2rS7ZBZ4z9LSQZhW1/ANgv0EQ5JkD5TGP9BCnp Um/4SwdFhXtO0WSpVEpHym0MRnOoBLNFv7WJo/4sxQoW3RpklUZUDVaiIlQdnPUlEmRL dP0sPaoZMKVVScX7kU/paUjkfAyxh/cgXYwnQsSrlps/gxAArE6PgT4az4nQP8Jm9akO txj1yKcEqg0Azj61wz1bVcJwmYz/eV7wz2ZLHVrpnTJDkdHbYycrexzne1u/O5Iy3oN+ azEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:references:in-reply-to :message-id:date:subject:cc:to:from:delivered-to:sender:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mailing-list:dkim-signature:domainkey-signature :arc-authentication-results; bh=Iwgkxb3gLzKRVhzrtxjHSsxdNTrQwRYqB2ouFtS5/Yc=; b=APdhV6+uMpHy6c5RvE9/if3x5wVTqlJ1UoL/m3rvsCA5hKJRFi/4GLoe/CSi0bmB2q g5tNQ9Dzm3oQGxLQPDzyJq8ar/U902fzkbRXUavZGi0g7ZVpEhPwc1YxLHVSwdDeUkXC tH7it8Tofmm70VOrXOUIStQ0hOJeVGnfZu7uWXYqUYWXnn62ktLBhoCkt5bqUzXVjzF/ kxkjJ5mHZ/L76GP0pyrQj75ysg/MeoLpW+qVM7p8hS0CeMvWbynZf6LXa08MieSmxrkV 9+AaEA9TwunYnYHrWf4SvlROstU2e2fMWQc0bR2exckJFk9F9UaXjXFdARpalb3+5hsP n8Eg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=kJ8TB42T; spf=pass (google.com: domain of gcc-patches-return-471479-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-471479-patch=linaro.org@gcc.gnu.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id q10si3994023pgd.331.2018.01.17.06.57.00 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Jan 2018 06:57:00 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-return-471479-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=kJ8TB42T; spf=pass (google.com: domain of gcc-patches-return-471479-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-471479-patch=linaro.org@gcc.gnu.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references:mime-version:content-type; q=dns; s= default; b=KCRj/yrnW7RFkJ1bk2uTvz+EjpkvBGGjft5yCeSdQiBwOPYPrWJQq Jv/ssfB4GW9YUvacOQjZN1jVgqF3/XuZFTIraDh/nh0/mxZGdgNm15S+npO+AfJk l7Mtev5dFEGGbLCzLq8cWL6W+PTOkIjLHRyiEfdnoqg6nok//QZW/Y= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references:mime-version:content-type; s=default; bh=YTKpH+DROGzk5IXsI91r3sP+VDo=; b=kJ8TB42TxfmsAMypAeS7XoaE67oZ ZqHAcx5sSTTYeDGTh2uLI1Vfl3nprTFGEiY53lyg504HWSk1bzT8nv5tEmCvGXsP YGyieBu5pCYgj8nau3z5tPW/YDELSSlSTGymW4JqXF2vGCNBllySDW/Qrv72nuMr nRfjppBTT6Zp5Go= Received: (qmail 85001 invoked by alias); 17 Jan 2018 14:56:14 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 84874 invoked by uid 89); 17 Jan 2018 14:56:13 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS, T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=reveal, techniques, traces, fabricated X-HELO: foss.arm.com Received: from usa-sjc-mx-foss1.foss.arm.com (HELO foss.arm.com) (217.140.101.70) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 17 Jan 2018 14:56:09 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CADC315A2; Wed, 17 Jan 2018 06:56:07 -0800 (PST) Received: from e105689-lin.cambridge.arm.com (e105689-lin.cambridge.arm.com [10.2.207.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BA0C43F557; Wed, 17 Jan 2018 06:56:06 -0800 (PST) From: Richard Earnshaw To: gcc-patches@gcc.gnu.org Cc: Richard Earnshaw Subject: [PATCH 1/3] [builtins] Generic support for __builtin_speculation_safe_load() Date: Wed, 17 Jan 2018 14:55:34 +0000 Message-Id: <34222de202457b0a96265827da65ee0ca07cf99b.1516199099.git.Richard.Earnshaw@arm.com> In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 This patch adds generic support for the new builtin __builtin_speculation_safe_load. It provides the overloading of the different access sizes and a default fall-back expansion for targets that do not support a mechanism for inhibiting speculation. * builtin-types.def (BT_FN_I1_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR): New builtin type signature. (BT_FN_I2_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR): Likewise. (BT_FN_I4_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR): Likewise. (BT_FN_I8_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR): Likewise. (BT_FN_I16_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR): Likewise. * builtins.def (BUILT_IN_SPECULATION_SAFE_LOAD_N): New builtin. (BUILT_IN_SPECULATION_SAFE_LOAD_1): Likewise. (BUILT_IN_SPECULATION_SAFE_LOAD_2): Likewise. (BUILT_IN_SPECULATION_SAFE_LOAD_4): Likewise. (BUILT_IN_SPECULATION_SAFE_LOAD_8): Likewise. (BUILT_IN_SPECULATION_SAFE_LOAD_16): Likewise. * target.def (speculation_safe_load): New hook. * doc/tm.texi.in (TARGET_SPECULATION_SAFE_LOAD): Add to documentation. * doc/tm.texi: Regenerated. * doc/cpp.texi: Document __HAVE_SPECULATION_SAFE_LOAD. * doc/extend.texi: Document __builtin_speculation_safe_load. * c-family/c-common.c (speculation_safe_load_resolve_size): New function. (speculation_safe_load_resolve_params): New function. (speculation_safe_load_resolve_return): New function. (resolve_overloaded_builtin): Handle overloading __builtin_speculation_safe_load. * builtins.c (expand_speculation_safe_load): New function. (expand_builtin): Handle new speculation-safe builtins. * targhooks.h (default_speculation_safe_load): Declare. * targhooks.c (default_speculation_safe_load): New function. --- gcc/builtin-types.def | 16 +++++ gcc/builtins.c | 81 +++++++++++++++++++++++ gcc/builtins.def | 17 +++++ gcc/c-family/c-common.c | 152 ++++++++++++++++++++++++++++++++++++++++++++ gcc/c-family/c-cppbuiltin.c | 5 +- gcc/doc/cpp.texi | 4 ++ gcc/doc/extend.texi | 68 ++++++++++++++++++++ gcc/doc/tm.texi | 9 +++ gcc/doc/tm.texi.in | 2 + gcc/target.def | 34 ++++++++++ gcc/targhooks.c | 59 +++++++++++++++++ gcc/targhooks.h | 3 + 12 files changed, 449 insertions(+), 1 deletion(-) diff --git a/gcc/builtin-types.def b/gcc/builtin-types.def index bb50e60..492d4f6 100644 --- a/gcc/builtin-types.def +++ b/gcc/builtin-types.def @@ -785,6 +785,22 @@ DEF_FUNCTION_TYPE_VAR_3 (BT_FN_SSIZE_STRING_SIZE_CONST_STRING_VAR, DEF_FUNCTION_TYPE_VAR_3 (BT_FN_INT_FILEPTR_INT_CONST_STRING_VAR, BT_INT, BT_FILEPTR, BT_INT, BT_CONST_STRING) +DEF_FUNCTION_TYPE_VAR_3 (BT_FN_I1_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, + BT_I1, BT_CONST_VOLATILE_PTR, BT_CONST_VOLATILE_PTR, + BT_CONST_VOLATILE_PTR) +DEF_FUNCTION_TYPE_VAR_3 (BT_FN_I2_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, + BT_I2, BT_CONST_VOLATILE_PTR, BT_CONST_VOLATILE_PTR, + BT_CONST_VOLATILE_PTR) +DEF_FUNCTION_TYPE_VAR_3 (BT_FN_I4_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, + BT_I4, BT_CONST_VOLATILE_PTR, BT_CONST_VOLATILE_PTR, + BT_CONST_VOLATILE_PTR) +DEF_FUNCTION_TYPE_VAR_3 (BT_FN_I8_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, + BT_I8, BT_CONST_VOLATILE_PTR, BT_CONST_VOLATILE_PTR, + BT_CONST_VOLATILE_PTR) +DEF_FUNCTION_TYPE_VAR_3 (BT_FN_I16_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, + BT_I16, BT_CONST_VOLATILE_PTR, BT_CONST_VOLATILE_PTR, + BT_CONST_VOLATILE_PTR) + DEF_FUNCTION_TYPE_VAR_4 (BT_FN_INT_STRING_INT_SIZE_CONST_STRING_VAR, BT_INT, BT_STRING, BT_INT, BT_SIZE, BT_CONST_STRING) diff --git a/gcc/builtins.c b/gcc/builtins.c index 98eb804..c0a15d1 100644 --- a/gcc/builtins.c +++ b/gcc/builtins.c @@ -6602,6 +6602,79 @@ expand_stack_save (void) return ret; } +/* Expand a call to __builtin_speculation_safe_load_. MODE + represents the size of the first argument to that call. We emit a + warning if the result isn't used (IGNORE != 0), since the + implementation might rely on the value being used to correctly + inhibit speculation. */ +static rtx +expand_speculation_safe_load (machine_mode mode, tree exp, rtx target, + int ignore) +{ + rtx ptr, mem, lower, upper, cmpptr; + unsigned nargs = call_expr_nargs (exp); + + if (ignore) + { + warning_at (input_location, 0, + "result of __builtin_speculation_safe_load must be used to " + "ensure correct operation"); + target = NULL; + } + + tree arg0 = CALL_EXPR_ARG (exp, 0); + tree arg1 = CALL_EXPR_ARG (exp, 1); + tree arg2 = CALL_EXPR_ARG (exp, 2); + + ptr = expand_expr (arg0, NULL_RTX, ptr_mode, EXPAND_SUM); + mem = validize_mem (gen_rtx_MEM (mode, convert_memory_address (Pmode, ptr))); + + set_mem_align (mem, MAX (GET_MODE_ALIGNMENT (mode), + get_pointer_alignment (arg0))); + set_mem_alias_set (mem, get_alias_set (TREE_TYPE (TREE_TYPE (arg0)))); + + /* Mark the memory access as volatile. We don't want the optimizers to + move it or otherwise substitue an alternative value. */ + MEM_VOLATILE_P (mem) = 1; + + lower = expand_normal (arg1); + if (GET_MODE (lower) != ptr_mode && GET_MODE (lower) != VOIDmode) + lower = convert_modes (ptr_mode, VOIDmode, lower, + TYPE_UNSIGNED (TREE_TYPE (arg1))); + + /* Ensure that the upper bound is not NULL. The builtin is not portable + unless we enforce this. */ + if (integer_zerop (tree_strip_nop_conversions (arg2))) + error_at (input_location, "third argument (upper bound) cannot be NULL"); + + upper = expand_normal (arg2); + if (GET_MODE (upper) != ptr_mode && GET_MODE (upper) != VOIDmode) + upper = convert_modes (ptr_mode, VOIDmode, upper, + TYPE_UNSIGNED (TREE_TYPE (arg2))); + + if (nargs > 3) + { + tree arg3 = CALL_EXPR_ARG (exp, 3); + /* Ensure that cmpptr is not NULL. The builtin is not portable + unless we enforce this. */ + if (integer_zerop (tree_strip_nop_conversions (arg3))) + error_at (input_location, + "fourth argument, if present, must be non-NULL"); + + cmpptr = expand_normal (arg3); + if (GET_MODE (cmpptr) != ptr_mode && GET_MODE (cmpptr) != VOIDmode) + cmpptr = convert_modes (ptr_mode, VOIDmode, cmpptr, + TYPE_UNSIGNED (TREE_TYPE (arg3))); + } + else + cmpptr = ptr; + + if (target == NULL) + target = gen_reg_rtx (mode); + + return targetm.speculation_safe_load (mode, target, mem, lower, upper, + cmpptr, true); +} /* Expand an expression EXP that calls a built-in function, with result going to TARGET if that's convenient @@ -7732,6 +7805,14 @@ expand_builtin (tree exp, rtx target, rtx subtarget, machine_mode mode, folding. */ break; + case BUILT_IN_SPECULATION_SAFE_LOAD_1: + case BUILT_IN_SPECULATION_SAFE_LOAD_2: + case BUILT_IN_SPECULATION_SAFE_LOAD_4: + case BUILT_IN_SPECULATION_SAFE_LOAD_8: + case BUILT_IN_SPECULATION_SAFE_LOAD_16: + mode = get_builtin_sync_mode (fcode - BUILT_IN_SPECULATION_SAFE_LOAD_1); + return expand_speculation_safe_load (mode, exp, target, ignore); + default: /* just do library call, if unknown builtin */ break; } diff --git a/gcc/builtins.def b/gcc/builtins.def index 671097e..16fa3e3 100644 --- a/gcc/builtins.def +++ b/gcc/builtins.def @@ -1017,6 +1017,23 @@ DEF_BUILTIN (BUILT_IN_EMUTLS_REGISTER_COMMON, true, true, true, ATTR_NOTHROW_LEAF_LIST, false, !targetm.have_tls) +/* Suppressing speculation. Users are expected to use the first (N) + variant, which will be translated internally into one of the other + types. */ +DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_LOAD_N, "speculation_safe_load", + BT_FN_VOID_VAR, ATTR_NULL) + +DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_LOAD_1, "speculation_safe_load_1", + BT_FN_I1_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, ATTR_NULL) +DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_LOAD_2, "speculation_safe_load_2", + BT_FN_I2_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, ATTR_NULL) +DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_LOAD_4, "speculation_safe_load_4", + BT_FN_I4_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, ATTR_NULL) +DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_LOAD_8, "speculation_safe_load_8", + BT_FN_I8_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, ATTR_NULL) +DEF_GCC_BUILTIN (BUILT_IN_SPECULATION_SAFE_LOAD_16, "speculation_safe_load_16", + BT_FN_I16_CONST_VPTR_CONST_VPTR_CONST_VPTR_VAR, ATTR_NULL) + /* Exception support. */ DEF_BUILTIN_STUB (BUILT_IN_UNWIND_RESUME, "__builtin_unwind_resume") DEF_BUILTIN_STUB (BUILT_IN_CXA_END_CLEANUP, "__builtin_cxa_end_cleanup") diff --git a/gcc/c-family/c-common.c b/gcc/c-family/c-common.c index 197a71f..f8b1fdf 100644 --- a/gcc/c-family/c-common.c +++ b/gcc/c-family/c-common.c @@ -6456,6 +6456,134 @@ builtin_type_for_size (int size, bool unsignedp) return type ? type : error_mark_node; } +/* Work out the size of the object pointed to by the first arguement + of a call to __builtin_speculation_safe. Only pointers to + integral types and pointers are permitted. Return 0 if the + arguement type is not supported of if the size is too large. */ +static int +speculation_safe_load_resolve_size (tree function, vec *params) +{ + /* Type of the argument. */ + tree type; + int size; + + if (vec_safe_is_empty (params)) + { + error ("too few arguments to function %qE", function); + return 0; + } + + type = TREE_TYPE ((*params)[0]); + + if (!POINTER_TYPE_P (type)) + goto incompatible; + + type = TREE_TYPE (type); + + if (TREE_CODE (type) == ARRAY_TYPE) + { + /* Force array-to-pointer decay for c++. */ + gcc_assert (c_dialect_cxx ()); + (*params)[0] = default_conversion ((*params)[0]); + type = TREE_TYPE ((*params)[0]); + } + + if (!INTEGRAL_TYPE_P (type) && !POINTER_TYPE_P (type)) + goto incompatible; + + if (!COMPLETE_TYPE_P (type)) + goto incompatible; + + size = tree_to_uhwi (TYPE_SIZE_UNIT (type)); + if (size == 1 || size == 2 || size == 4 || size == 8 || size == 16) + return size; + + incompatible: + /* Issue the diagnostic only if the argument is valid, otherwise + it would be redundant at best and could be misleading. */ + if (type != error_mark_node) + error ("operand type %qT is incompatible with argument %d of %qE", + type, 1, function); + + return 0; +} + +/* Validate and coerce PARAMS, the arguments to ORIG_FUNCTION to fit + the prototype for FUNCTION. The first three arguments are + mandatory, but shouldn't need casting as they are all pointers and + we've already established that the first argument is a pointer to a + permitted type. The two optional arguments may need to be + fabricated if they have been omitted. */ +static bool +speculation_safe_load_resolve_params (location_t loc, tree orig_function, + tree function, + vec *params) +{ + function_args_iterator iter; + + function_args_iter_init (&iter, TREE_TYPE (function)); + tree arg_type = function_args_iter_cond (&iter); + unsigned parmnum; + tree val; + + if (params->length () < 3) + { + error_at (loc, "too few arguments to function %qE", orig_function); + return false; + } + else if (params->length () > 4) + { + error_at (loc, "too many arguments to function %qE", orig_function); + return false; + } + + /* Required arguments. These must all be pointers. */ + for (parmnum = 0; parmnum < 3; parmnum++) + { + arg_type = function_args_iter_cond (&iter); + val = (*params)[parmnum]; + if (TREE_CODE (TREE_TYPE (val)) == ARRAY_TYPE) + val = default_conversion (val); + if (TREE_CODE (TREE_TYPE (val)) != POINTER_TYPE) + goto bad_arg; + (*params)[parmnum] = val; + } + + /* Optional pointer to compare against. */ + arg_type = function_args_iter_cond (&iter); + if (params->length () == 4) + { + val = (*params)[parmnum]; + if (TREE_CODE (TREE_TYPE (val)) == ARRAY_TYPE) + val = default_conversion (val); + if (TREE_CODE (TREE_TYPE (val)) != POINTER_TYPE) + goto bad_arg; + (*params)[parmnum] = val; + } + + return true; + + bad_arg: + error_at (loc, "expecting argument of type %qT for argument %u", arg_type, + parmnum); + return false; +} + +/* Cast the result of the builtin back to the type pointed to by the + first argument, preserving any qualifiers that it might have. */ +static tree +speculation_safe_load_resolve_return (tree first_param, tree result) +{ + tree ptype = TREE_TYPE (TREE_TYPE (first_param)); + tree rtype = TREE_TYPE (result); + ptype = TYPE_MAIN_VARIANT (ptype); + + if (tree_int_cst_equal (TYPE_SIZE (ptype), TYPE_SIZE (rtype))) + return convert (ptype, result); + + return result; +} + /* A helper function for resolve_overloaded_builtin in resolving the overloaded __sync_ builtins. Returns a positive power of 2 if the first operand of PARAMS is a pointer to a supported data type. @@ -7110,6 +7238,30 @@ resolve_overloaded_builtin (location_t loc, tree function, /* Handle BUILT_IN_NORMAL here. */ switch (orig_code) { + case BUILT_IN_SPECULATION_SAFE_LOAD_N: + { + int n = speculation_safe_load_resolve_size (function, params); + tree new_function, first_param, result; + enum built_in_function fncode; + + if (n == 0) + return error_mark_node; + + fncode = (enum built_in_function)((int)orig_code + exact_log2 (n) + 1); + new_function = builtin_decl_explicit (fncode); + first_param = (*params)[0]; + if (!speculation_safe_load_resolve_params (loc, function, new_function, + params)) + return error_mark_node; + + result = build_function_call_vec (loc, vNULL, new_function, params, + NULL); + if (result == error_mark_node) + return result; + + return speculation_safe_load_resolve_return (first_param, result); + } + case BUILT_IN_ATOMIC_EXCHANGE: case BUILT_IN_ATOMIC_COMPARE_EXCHANGE: case BUILT_IN_ATOMIC_LOAD: diff --git a/gcc/c-family/c-cppbuiltin.c b/gcc/c-family/c-cppbuiltin.c index 9e33aed..61f1a2e 100644 --- a/gcc/c-family/c-cppbuiltin.c +++ b/gcc/c-family/c-cppbuiltin.c @@ -1361,7 +1361,10 @@ c_cpp_builtins (cpp_reader *pfile) cpp_define (pfile, "__WCHAR_UNSIGNED__"); cpp_atomic_builtins (pfile); - + + /* Show support for __builtin_speculation_safe_load (). */ + cpp_define (pfile, "__HAVE_SPECULATION_SAFE_LOAD"); + #ifdef DWARF2_UNWIND_INFO if (dwarf2out_do_cfi_asm ()) cpp_define (pfile, "__GCC_HAVE_DWARF2_CFI_ASM"); diff --git a/gcc/doc/cpp.texi b/gcc/doc/cpp.texi index 94437d5..e1980fe 100644 --- a/gcc/doc/cpp.texi +++ b/gcc/doc/cpp.texi @@ -2381,6 +2381,10 @@ If GCC cannot determine the current date, it will emit a warning message These macros are defined when the target processor supports atomic compare and swap operations on operands 1, 2, 4, 8 or 16 bytes in length, respectively. +@item __HAVE_SPECULATION_SAFE_LAOD +This macro is defined with the value 1 to show that this version of GCC +supports @code{__builtin_speculation_safe_load}. + @item __GCC_HAVE_DWARF2_CFI_ASM This macro is defined when the compiler is emitting DWARF CFI directives to the assembler. When this is defined, it is possible to emit those same diff --git a/gcc/doc/extend.texi b/gcc/doc/extend.texi index 2a553ad..cbee943 100644 --- a/gcc/doc/extend.texi +++ b/gcc/doc/extend.texi @@ -10971,6 +10971,7 @@ the built-in function returns -1. @findex __builtin_powi @findex __builtin_powif @findex __builtin_powil +@findex __builtin_speculation_safe_load @findex _Exit @findex _exit @findex abort @@ -11614,6 +11615,73 @@ check its compatibility with @var{size}. @end deftypefn +@deftypefn {Built-in Function} @var{type} __builtin_speculation_safe_load (const volatile @var{type} *ptr, const volatile void *lower_bound, const volatile void *upper_bound, const volatile void *cmpptr) + +Modern processors use sophisticated techniques to improve their +overall performance. For example, the processor may begin to execute +instructions that it guesses are likely to be needed in the near +future: this is known as speculative execution. If the guess turns +out to be correct then significant time can be saved. If the guess +turns out to be wrong the instruction's calculations are discarded and +the correct ones are then executed and the program will continue as +the programmer intended. However, in some circumstances it is +possible for the discarded operations to leave traces of what happened +in a manner that can later be discovered by timing subsequent +operations. Speculative memory operations, for example, interact with +the cache memory found on many processors in a way which could +allow further carefully controlled speculative instructions to reveal +information. + +The @code{__builtin_speculation_safe_load} function provides a means +to limit the extent to which a processor can continue speculative +execution with the result of loading a value stored at @var{ptr}. The +boundary conditions, described by @var{cmpptr}, @var{lower_bound} and +@var{upper_bound}, define the conditions under which execution after +the load can continue safely: + +@enumerate +@item +When the call to the builtin is not being speculatively executed the +result is @code{*ptr} if @code{lower_bound <= cmpptr < upper_bound}. +The behavior is undefined if cmpptr is outside of that range. + +@item +When code is being speculatively executed either: +@itemize +@item +execution of subsequent instructions that depend on the result will +be prevented until it can be proven that the call to the builtin is +not being speculatively executed (i.e.@: until execution can continue under +point 1), or + +@item +speculation may continue using @code{*ptr} as the result when +@code{lower_bound <= cmpptr < upper_bound}, or an unspecified constant +value (e.g.@: zero) if @code{cmpptr} lies outside that range. +@end itemize +@end enumerate + +The type of the result, @var{type}, may be any integral type (signed, +or unsigned, @code{char}, @code{short}, @code{int}, etc) or a pointer +to any type. + +The final argument, @var{cmpptr}, may be omitted if it is the same as +@var{ptr}. + +The builtin is supported for all architectures, but on machines where +target-specific support for inhibiting speculation is not implemented, +or not necessary, the compiler will emit a warning. + +@emph{Note:} it is important that the boundary conditions used +accurately describe the conditions under which speculation may be +occuring, otherwise the builtin will not provide protection against +speculative use of the result. + +The pre-processor macro @code{__HAVE_SPECULATION_SAFE_LOAD} is defined with the +value 1 on all implementations of GCC that support this builtin. + +@end deftypefn + @deftypefn {Built-in Function} int __builtin_types_compatible_p (@var{type1}, @var{type2}) You can use the built-in function @code{__builtin_types_compatible_p} to diff --git a/gcc/doc/tm.texi b/gcc/doc/tm.texi index 9793a0e..9f2206d 100644 --- a/gcc/doc/tm.texi +++ b/gcc/doc/tm.texi @@ -11922,6 +11922,15 @@ maintainer is familiar with. @end defmac +@deftypefn {Target Hook} rtx TARGET_SPECULATION_SAFE_LOAD (machine_mode @var{mode}, rtx @var{result}, rtx @var{mem}, rtx @var{lower_bound}, rtx @var{upper_bound}, rtx @var{cmpptr}, bool @var{warn}) +Generate a target-specific code sequence that implements @code{__builtin_speculation_safe_load}, returning @var{mem}, a @code{MEM} of type @var{mode} in @var{result}. + In the abstract machine, the built-in must only be called when @var{cmpptr} is greater than or equal to @var{lower} and less than @var{upper}. The behaviour is undefined otherwise (and may generate memory faults). + On real hardware, however, the builtin may be reached while the CPU is speculatively executing code. It is this built-in's responsibility to ensure that, if speculating when outside the specified boundary conditions, the contents of @var{mem} is not visible to subsequent instructions. + There are two common techniques that can be used to implement this builtin. The first is to emit a target-specific barrier instruction sequence that is guaranteed to ensure that speculative execution cannot continue. On architectures with such a sequence the bounds checks can be ignored and the expansion can be simply a load and the barrier. + On some targets, however, unsafe speculation can be inhibited by overwriting the result when the speculation bounds are exceeded. This must be done using a code sequence that does not introduce further speculative behavior. An example of a target using this approach is AArch64. + The default implementation implements the logic of the builtin but cannot provide the target-specific code necessary to inhibit speculation. If @var{warn} is true a warning will be emitted to that effect. +@end deftypefn + @deftypefn {Target Hook} void TARGET_RUN_TARGET_SELFTESTS (void) If selftests are enabled, run any selftests for this target. @end deftypefn diff --git a/gcc/doc/tm.texi.in b/gcc/doc/tm.texi.in index 7bcfb37..3393457 100644 --- a/gcc/doc/tm.texi.in +++ b/gcc/doc/tm.texi.in @@ -8075,4 +8075,6 @@ maintainer is familiar with. @end defmac +@hook TARGET_SPECULATION_SAFE_LOAD + @hook TARGET_RUN_TARGET_SELFTESTS diff --git a/gcc/target.def b/gcc/target.def index e9eacc8..0ba42a1 100644 --- a/gcc/target.def +++ b/gcc/target.def @@ -4214,6 +4214,40 @@ DEFHOOK hook_bool_void_true) DEFHOOK +(speculation_safe_load, + "Generate a target-specific code sequence that implements\ + @code{__builtin_speculation_safe_load}, returning @var{mem}, a @code{MEM} of\ + type @var{mode} in @var{result}.\ + \n\ + In the abstract machine, the built-in must only be called when @var{cmpptr}\ + is greater than or equal to @var{lower} and less than @var{upper}. The\ + behaviour is undefined otherwise (and may generate memory faults).\ + \n\ + On real hardware, however, the builtin may be reached while the CPU is\ + speculatively executing code. It is this built-in's responsibility to ensure\ + that, if speculating when outside the specified boundary conditions, the\ + contents of @var{mem} is not visible to subsequent instructions.\ + \n\ + There are two common techniques that can be used to implement this builtin.\ + The first is to emit a target-specific barrier instruction sequence that is\ + guaranteed to ensure that speculative execution cannot continue. On\ + architectures with such a sequence the bounds checks can be ignored and\ + the expansion can be simply a load and the barrier.\ + \n\ + On some targets, however, unsafe speculation can be inhibited by overwriting\ + the result when the speculation bounds are exceeded. This must be done\ + using a code sequence that does not introduce further speculative behavior.\ + An example of a target using this approach is AArch64.\ + \n\ + The default implementation implements the logic of the builtin\ + but cannot provide the target-specific code necessary to inhibit\ + speculation. If @var{warn} is true a warning will be emitted to that\ + effect.", + rtx, (machine_mode mode, rtx result, rtx mem, rtx lower_bound, + rtx upper_bound, rtx cmpptr, bool warn), + default_speculation_safe_load) + +DEFHOOK (can_use_doloop_p, "Return true if it is possible to use low-overhead loops (@code{doloop_end}\n\ and @code{doloop_begin}) for a particular loop. @var{iterations} gives the\n\ diff --git a/gcc/targhooks.c b/gcc/targhooks.c index 653567c..228b966 100644 --- a/gcc/targhooks.c +++ b/gcc/targhooks.c @@ -82,6 +82,7 @@ along with GCC; see the file COPYING3. If not see #include "params.h" #include "real.h" #include "langhooks.h" +#include "dojump.h" bool default_legitimate_address_p (machine_mode mode ATTRIBUTE_UNUSED, @@ -2307,4 +2308,62 @@ default_stack_clash_protection_final_dynamic_probe (rtx residual ATTRIBUTE_UNUSE return 0; } +/* Default implementation of the speculation-safe-load builtin. + This version does not have, or know of, the target-specific + mechanisms necessary to inhibit speculation, so it simply emits a + code sequence that implements the architectural aspects of the + builtin. */ +rtx +default_speculation_safe_load (machine_mode mode ATTRIBUTE_UNUSED, + rtx result, rtx mem, rtx lower_bound, + rtx upper_bound, rtx cmpptr, bool warn) +{ + rtx_code_label *done_label = gen_label_rtx (); + rtx_code_label *inrange_label = gen_label_rtx (); + + if (warn) + warning_at + (input_location, 0, + "this target does not support anti-speculation operations. " + "Your program will still execute correctly, but speculation " + "will not be inhibited"); + + /* We don't have any speculation barriers, but if we mark the branch + probabilities to be always predicting the out-of-bounds path, then + there's a higher chance that the compiler will order code so that + static prediction will fall through a safe path. */ + if (lower_bound == const0_rtx) + { + /* There's no point in checking against a lower bound of zero, simply + skip that case as it tells us nothing about the speculation + condition. */ + do_compare_rtx_and_jump (cmpptr, upper_bound, LTU, true, ptr_mode, + NULL, NULL, inrange_label, + profile_probability::never ()); + emit_move_insn (result, GEN_INT (0)); + emit_jump (done_label); + emit_label (inrange_label); + emit_move_insn (result, mem); + emit_label (done_label); + } + else + { + rtx_code_label *oob_label = gen_label_rtx (); + do_compare_rtx_and_jump (cmpptr, lower_bound, LTU, true, ptr_mode, + NULL, NULL, oob_label, + profile_probability::always ()); + do_compare_rtx_and_jump (cmpptr, upper_bound, GEU, true, ptr_mode, + NULL, NULL, inrange_label, + profile_probability::never ()); + emit_label (oob_label); + emit_move_insn (result, GEN_INT (0)); + emit_jump (done_label); + emit_label (inrange_label); + emit_move_insn (result, mem); + emit_label (done_label); + } + + return result; +} + #include "gt-targhooks.h" diff --git a/gcc/targhooks.h b/gcc/targhooks.h index e753e58..76ba150 100644 --- a/gcc/targhooks.h +++ b/gcc/targhooks.h @@ -286,4 +286,7 @@ extern enum flt_eval_method default_excess_precision (enum excess_precision_type ATTRIBUTE_UNUSED); extern bool default_stack_clash_protection_final_dynamic_probe (rtx); +extern rtx +default_speculation_safe_load (machine_mode, rtx, rtx, rtx, rtx, rtx, bool); + #endif /* GCC_TARGHOOKS_H */ From patchwork Wed Jan 17 14:55:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Richard Earnshaw \(lists\)" X-Patchwork-Id: 124808 Delivered-To: patch@linaro.org Received: by 10.46.62.1 with SMTP id l1csp70822lja; Wed, 17 Jan 2018 06:56:34 -0800 (PST) X-Google-Smtp-Source: ACJfBouTCD52BOQQLyY0grRGR9xYmXCxtlSTgSHCHxJowfOh10+kyV9a4revwCU3s/YaEwg3SZJa X-Received: by 10.84.232.202 with SMTP id x10mr5689842plm.367.1516200994438; Wed, 17 Jan 2018 06:56:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516200994; cv=none; d=google.com; s=arc-20160816; b=tSfYvdM2ERhH8YnQqaqas4J+WT2bJvgemhKkDf1s/cG6DlQVJx6lDDQiKwZr7Dfjvs xtC7DgHvQdHP3GrpNeH/ITIQKZBR8s5xVl09ciswTihiZbCPdM/1CfiIhcUqy2pD/HGH WIe2h72cQ4CLHam/aKzdCWVJzC+QFj5awhfqk12MWGmskVJ5wnI2P4GIrX5hitHo+2qA jblokV+yultLI7hgvzLk2LvglzcSLkwzekbV1QOIftDaSaTiJP6xS9m+RQdd5v0rq59p WH5Nr+ar4V0/o2HrlkPoip4s1Z7b7Ed8zHq/l8HNukWiseI5FKthwLssJDGZd2BlbUVN Se5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:references:in-reply-to :message-id:date:subject:cc:to:from:delivered-to:sender:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mailing-list:dkim-signature:domainkey-signature :arc-authentication-results; bh=568cOZwEGtggUnppCLEjbi15SvYToOhv7iLaG/MK0to=; b=Bm/K4SvyPoRSvbNr3scmhDHGX/50lopERonNQkpab3v0+awTp1UVon1jCzyqlCKtmt 1ExzYL5dRva7jSyKLopfzdmIrAjKLN3XfNRTCx5/oAbrRgr0gDNYQaeRt9k26NJLAZu7 1dzK2QCrV2UZPWHTZXi0hE/E6I397k1EaLsQJtNzJJ82Y0u1pPqsTR6VRVFIA4Bg/TWV 5iKF2FTnLAT4ctVHsUYdlJc4MW+F5xZr8uPOrL/FdAL4cQoOsbPa9fFdYFIodJt4hcJe jC/cRh0ELMU/odRvZKivUEbXon3mZOGzYPgVPy22aF5ratAaVXvUbRNm8DJWYs+x4lpK TmZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=lUH4RT5I; spf=pass (google.com: domain of gcc-patches-return-471477-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-471477-patch=linaro.org@gcc.gnu.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id e4si4518389pfb.84.2018.01.17.06.56.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Jan 2018 06:56:34 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-return-471477-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=lUH4RT5I; spf=pass (google.com: domain of gcc-patches-return-471477-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-471477-patch=linaro.org@gcc.gnu.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references:mime-version:content-type; q=dns; s= default; b=PrrQq20qupL1NFcgscaNjMqjnyCt6Enl2t2qrdmc6pxoHf3TMaP8v SVmcdb4WuQz8DPPz7bwKSVDuOjpY9GtqCD06kzatRQIRInloaiIPKrx6uBxpwA3k 9wePBJ9bSiwQ868T2TUh9TfOLJXge/jQ0qxxOjtR8tQ56ZRh/fG0mc= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references:mime-version:content-type; s=default; bh=28EjefTxx1zs6/kjFDaaQFwW2Zc=; b=lUH4RT5IUggTzPkavdBFZCbIIBLd ccDVLQZY95TKkjO0o1VJMFpEiF5ppdblWbXEvkh2DpzwEcNEj2eLSXmR4sIHWOF/ ueJPGztEllXqpMDSD0vZp3FoLt8G1iSBiWKMFrAok2BIS4ty7Z3Y/3Hr+c9lj44e 8UoQ19FYHawufsY= Received: (qmail 84721 invoked by alias); 17 Jan 2018 14:56:12 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 84627 invoked by uid 89); 17 Jan 2018 14:56:12 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS, T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=numbered X-HELO: foss.arm.com Received: from usa-sjc-mx-foss1.foss.arm.com (HELO foss.arm.com) (217.140.101.70) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 17 Jan 2018 14:56:10 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D0D6A15AD; Wed, 17 Jan 2018 06:56:08 -0800 (PST) Received: from e105689-lin.cambridge.arm.com (e105689-lin.cambridge.arm.com [10.2.207.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 14D9D3F557; Wed, 17 Jan 2018 06:56:07 -0800 (PST) From: Richard Earnshaw To: gcc-patches@gcc.gnu.org Cc: Richard Earnshaw Subject: [PATCH 2/3] [aarch64] Implement support for __builtin_speculation_safe_load Date: Wed, 17 Jan 2018 14:55:35 +0000 Message-Id: <38bc3149ec21722caba728bae9dab5381514a4e4.1516199099.git.Richard.Earnshaw@arm.com> In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 This patch implements support for __builtin_speculation_safe_load on AArch64. On this architecture we inhibit speclation by emitting a combination of CSEL and a hint instruction that ensures the CSEL is full resolved when the operands to the CSEL may involve a speculative load. * config/aarch64/aarch64.c (aarch64_print_operand): Handle zero passed to 'H' operand qualifier. (aarch64_speculation_safe_load): New function. (TARGET_SPECULATION_SAFE_LOAD): Redefine. * config/aarch64/aarch64.md (UNSPECV_NOSPECULATE): New unspec_volatile code. (nospeculate, nospeculateti): New patterns. --- gcc/config/aarch64/aarch64.c | 81 +++++++++++++++++++++++++++++++++++++++++++ gcc/config/aarch64/aarch64.md | 28 +++++++++++++++ 2 files changed, 109 insertions(+) diff --git a/gcc/config/aarch64/aarch64.c b/gcc/config/aarch64/aarch64.c index 93e9d9f9..6591d19 100644 --- a/gcc/config/aarch64/aarch64.c +++ b/gcc/config/aarch64/aarch64.c @@ -5315,6 +5315,14 @@ aarch64_print_operand (FILE *f, rtx x, int code) break; case 'H': + /* Print the higher numbered register of a pair (TImode) of regs. */ + if (x == const0_rtx + || (CONST_DOUBLE_P (x) && aarch64_float_const_zero_rtx_p (x))) + { + asm_fprintf (f, "xzr"); + break; + } + if (!REG_P (x) || !GP_REGNUM_P (REGNO (x) + 1)) { output_operand_lossage ("invalid operand for '%%%c'", code); @@ -15115,6 +15123,76 @@ aarch64_sched_can_speculate_insn (rtx_insn *insn) } } +static rtx +aarch64_speculation_safe_load (machine_mode mode, rtx result, rtx mem, + rtx lower_bound, rtx upper_bound, rtx cmpptr, + bool warn ATTRIBUTE_UNUSED) +{ + rtx cond, comparison; + rtx target = gen_reg_rtx (mode); + rtx tgt2 = result; + + if (!register_operand (cmpptr, ptr_mode)) + cmpptr = force_reg (ptr_mode, cmpptr); + + if (!register_operand (tgt2, mode)) + tgt2 = gen_reg_rtx (mode); + + if (lower_bound == const0_rtx) + { + if (!register_operand (upper_bound, ptr_mode)) + upper_bound = force_reg (ptr_mode, upper_bound); + + cond = aarch64_gen_compare_reg (GEU, cmpptr, upper_bound); + comparison = gen_rtx_GEU (VOIDmode, cond, const0_rtx); + } + else + { + if (!register_operand (lower_bound, ptr_mode)) + lower_bound = force_reg (ptr_mode, lower_bound); + + if (!register_operand (upper_bound, ptr_mode)) + upper_bound = force_reg (ptr_mode, upper_bound); + + rtx cond1 = aarch64_gen_compare_reg (GEU, cmpptr, lower_bound); + rtx comparison1 = gen_rtx_GEU (ptr_mode, cond1, const0_rtx); + rtx failcond = GEN_INT (aarch64_get_condition_code (comparison1)^1); + cond = gen_rtx_REG (CCmode, CC_REGNUM); + if (ptr_mode == SImode) + emit_insn (gen_ccmpsi (cond1, cond, cmpptr, upper_bound, comparison1, + failcond)); + else + emit_insn (gen_ccmpdi (cond1, cond, cmpptr, upper_bound, comparison1, + failcond)); + comparison = gen_rtx_GEU (VOIDmode, cond, const0_rtx); + } + + rtx_code_label *label = gen_label_rtx (); + emit_jump_insn (gen_condjump (comparison, cond, label)); + emit_move_insn (target, mem); + emit_label (label); + + insn_code icode; + + switch (mode) + { + case E_QImode: icode = CODE_FOR_nospeculateqi; break; + case E_HImode: icode = CODE_FOR_nospeculatehi; break; + case E_SImode: icode = CODE_FOR_nospeculatesi; break; + case E_DImode: icode = CODE_FOR_nospeculatedi; break; + case E_TImode: icode = CODE_FOR_nospeculateti; break; + default: + gcc_unreachable (); + } + + emit_insn (GEN_FCN (icode) (tgt2, comparison, cond, target, const0_rtx)); + + if (tgt2 != result) + emit_move_insn (result, tgt2); + + return result; +} + /* Target-specific selftests. */ #if CHECKING_P @@ -15554,6 +15632,9 @@ aarch64_libgcc_floating_mode_supported_p #undef TARGET_CONSTANT_ALIGNMENT #define TARGET_CONSTANT_ALIGNMENT aarch64_constant_alignment +#undef TARGET_SPECULATION_SAFE_LOAD +#define TARGET_SPECULATION_SAFE_LOAD aarch64_speculation_safe_load + #if CHECKING_P #undef TARGET_RUN_TARGET_SELFTESTS #define TARGET_RUN_TARGET_SELFTESTS selftest::aarch64_run_selftests diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md index f1e2a07..1a1f398 100644 --- a/gcc/config/aarch64/aarch64.md +++ b/gcc/config/aarch64/aarch64.md @@ -153,6 +153,7 @@ UNSPECV_SET_FPSR ; Represent assign of FPSR content. UNSPECV_BLOCKAGE ; Represent a blockage UNSPECV_PROBE_STACK_RANGE ; Represent stack range probing. + UNSPECV_NOSPECULATE ; Inhibit speculation ] ) @@ -5797,6 +5798,33 @@ DONE; }) +(define_insn "nospeculate" + [(set (match_operand:ALLI 0 "register_operand" "=r") + (unspec_volatile:ALLI + [(match_operator 1 "aarch64_comparison_operator" + [(match_operand 2 "cc_register" "") (const_int 0)]) + (match_operand:ALLI 3 "register_operand" "r") + (match_operand:ALLI 4 "aarch64_reg_or_zero" "rZ")] + UNSPECV_NOSPECULATE))] + "" + "csel\\t%0, %3, %4, %M1\;hint\t#0x14\t// CSDB" + [(set_attr "type" "csel") + (set_attr "length" "8")] +) + +(define_insn "nospeculateti" + [(set (match_operand:TI 0 "register_operand" "=r") + (unspec_volatile:TI + [(match_operator 1 "aarch64_comparison_operator" + [(match_operand 2 "cc_register" "") (const_int 0)]) + (match_operand:TI 3 "register_operand" "r") + (match_operand:TI 4 "aarch64_reg_or_zero" "rZ")] + UNSPECV_NOSPECULATE))] + "" + "csel\\t%x0, %x3, %x4, %M1\;csel\\t%H0, %H3, %H4, %M1\;hint\t#0x14\t// CSDB" + [(set_attr "type" "csel") + (set_attr "length" "12")] +) ;; AdvSIMD Stuff (include "aarch64-simd.md") From patchwork Wed Jan 17 14:55:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Richard Earnshaw \(lists\)" X-Patchwork-Id: 124809 Delivered-To: patch@linaro.org Received: by 10.46.62.1 with SMTP id l1csp70916lja; Wed, 17 Jan 2018 06:56:47 -0800 (PST) X-Google-Smtp-Source: ACJfBovrEXi1qajoz0/MVao0abPFqX4H7oPKUiuCsYhddFO7mpvDUcFMdVpamWtab4aliRHMyGEr X-Received: by 10.99.163.2 with SMTP id s2mr5590056pge.264.1516201007083; Wed, 17 Jan 2018 06:56:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516201007; cv=none; d=google.com; s=arc-20160816; b=Vk/xcnwZ4LtHgLxY3VrQbtbOrKeuFzJcn37GUnEwWuH1nJc1nPY1FxOudWADmFG0li 9EzWBCR5t49TV3aQJrMHt8FeOcmyvfwyPTl4cMRTv2U2BdKBCoif63XIEUALqcYd8z7D cjq68FA5hAUwUKttnTQpNChntp+5z+Er4K/vK1uQh3+KXmD7557oy7GV6xyXG3caque6 hjyr4LMStiNJJzDnOcUiSJVbKIdLh8bg48mezLdCCwHa4jezRArw/RERfPxkQRUj4wgU q9G0VWPChJP5MYdDf/2aNWpVpzs7HQPy53dWm9DYxZJg9rQEmjboSJu8g/oUG90JzwKb QdoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:references:in-reply-to :message-id:date:subject:cc:to:from:delivered-to:sender:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence :mailing-list:dkim-signature:domainkey-signature :arc-authentication-results; bh=WcOyiktWBnz5Sp3F/vB41+7HLiiVxXE7r80Wxiw6JuA=; b=eVQyrMqrMvcUoIoHKeoUD4lukfTK+27DEFIZN4PFByCpx3rR/OpJEZ/u62a+bGKVcz lw/FaZUZ2tPLXq5nhlGDo9NgreZaPNgtwdz/JA0XGmTsEomRZL2ci3ccPwTGT3DqZdX5 bjpKUBHBprHflZc/+dmP+e648rhppc1FM2SQEoB+oW5JDTZEmYg+ncT8VUofHVuwJLYB fxlh1Y9Tg+23qbNEl3ZMjIjJGXw8lm049fbgVuHng3AXbyjLui2jvXS0wblWRMn+n+3G /UnhcpdgXL1Eob1Ey7DnsHaS6vEXPLQGaL+vjdLyNvgwGP1a1PJsRa5cjKwkhozCjzs3 NqIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=a90iOO4k; spf=pass (google.com: domain of gcc-patches-return-471478-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-471478-patch=linaro.org@gcc.gnu.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id c4si4008575pgt.44.2018.01.17.06.56.46 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Jan 2018 06:56:47 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-return-471478-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=a90iOO4k; spf=pass (google.com: domain of gcc-patches-return-471478-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-471478-patch=linaro.org@gcc.gnu.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references:mime-version:content-type; q=dns; s= default; b=ucU+c8gGqoOzk63fU2ahsKnKaTG+N2sjcIXdYQwJabnHiUvCoZT0Z ICiHWdbRntmECcFLbUCJ0HRh+owzk5e1aQC8jNhmUuEhURZ4X3tBSp3vHFGTzOAJ 7YeUBqbY2KIpxtZKxXox3rNN2Fj5jWVPXvG0tKa3Dn3d1AvZGQlAiU= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:in-reply-to:references :in-reply-to:references:mime-version:content-type; s=default; bh=ggsJhnA0pQA2WMvZWYtsx/CfacA=; b=a90iOO4kDrG3dfiQC0/sd3Hz+gdA fGf/DZXM30vgRSZ8PmRrEyhbJ5xbN/w7t+e1ut8uN8mzN3aWTnhJfBXdrvPgTar9 obJ1M/gapHCtsL5m8cSK2gm+Gkik0pXXhBkyCKmEMx2VGUmWMvmQEVb5dQdScwvV +g2rkMLJyEI/3ts= Received: (qmail 84990 invoked by alias); 17 Jan 2018 14:56:14 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 84866 invoked by uid 89); 17 Jan 2018 14:56:13 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-26.9 required=5.0 tests=BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS, T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=multimedia, Multimedia, wireless, llr X-HELO: foss.arm.com Received: from usa-sjc-mx-foss1.foss.arm.com (HELO foss.arm.com) (217.140.101.70) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Wed, 17 Jan 2018 14:56:11 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D69AF1610; Wed, 17 Jan 2018 06:56:09 -0800 (PST) Received: from e105689-lin.cambridge.arm.com (e105689-lin.cambridge.arm.com [10.2.207.32]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1A7CC3F557; Wed, 17 Jan 2018 06:56:08 -0800 (PST) From: Richard Earnshaw To: gcc-patches@gcc.gnu.org Cc: Richard Earnshaw Subject: [PATCH 3/3] [arm] Implement support for the de-speculation intrinsic Date: Wed, 17 Jan 2018 14:55:36 +0000 Message-Id: In-Reply-To: References: In-Reply-To: References: MIME-Version: 1.0 This patch implements despeculation on ARM. We only support it when generating ARM or Thumb2 code (we need conditional execution); and we only support it for sizes up to DImode. For unsupported cases we fall back to the generic code generation sequence so that a suitable failure warning is emitted. * config/arm/arm.c (arm_speculation_safe_load): New function (TARGET_SPECULATION_SAFE_LOAD): Redefine. * config/arm/unspec.md (VUNSPEC_NOSPECULATE): New unspec_volatile code. * config/arm/arm.md (cmp_ior): Make this pattern callable. (nospeculate, nospeculatedi): New patterns. --- gcc/config/arm/arm.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++ gcc/config/arm/arm.md | 40 ++++++++++++++++++- gcc/config/arm/unspecs.md | 1 + 3 files changed, 140 insertions(+), 1 deletion(-) diff --git a/gcc/config/arm/arm.c b/gcc/config/arm/arm.c index 11e35ad..f28ad2b 100644 --- a/gcc/config/arm/arm.c +++ b/gcc/config/arm/arm.c @@ -321,6 +321,8 @@ static unsigned int arm_hard_regno_nregs (unsigned int, machine_mode); static bool arm_hard_regno_mode_ok (unsigned int, machine_mode); static bool arm_modes_tieable_p (machine_mode, machine_mode); static HOST_WIDE_INT arm_constant_alignment (const_tree, HOST_WIDE_INT); +static rtx arm_speculation_safe_load (machine_mode, rtx, rtx, rtx, rtx, rtx, + bool); /* Table of machine attributes. */ static const struct attribute_spec arm_attribute_table[] = @@ -804,6 +806,9 @@ static const struct attribute_spec arm_attribute_table[] = #undef TARGET_CONSTANT_ALIGNMENT #define TARGET_CONSTANT_ALIGNMENT arm_constant_alignment + +#undef TARGET_SPECULATION_SAFE_LOAD +#define TARGET_SPECULATION_SAFE_LOAD arm_speculation_safe_load /* Obstack for minipool constant handling. */ static struct obstack minipool_obstack; @@ -31523,6 +31528,101 @@ arm_constant_alignment (const_tree exp, HOST_WIDE_INT align) return align; } +static rtx +arm_speculation_safe_load (machine_mode mode, rtx result, rtx mem, + rtx lower_bound, rtx upper_bound, + rtx cmpptr, bool warn) +{ + rtx cond, comparison; + + /* We can't support this for Thumb1 as we have no suitable conditional + move operations. Nor do we support it for TImode. For both + these cases fall back to the generic code sequence which will emit + a suitable warning for us. */ + if (mode == TImode || TARGET_THUMB1) + return default_speculation_safe_load (mode, result, mem, lower_bound, + upper_bound, cmpptr, warn); + + + rtx target = gen_reg_rtx (mode); + rtx tgt2 = result; + + if (!register_operand (tgt2, mode)) + tgt2 = gen_reg_rtx (mode); + + if (!register_operand (cmpptr, ptr_mode)) + cmpptr = force_reg (ptr_mode, cmpptr); + + /* There's no point in comparing against a lower bound that is NULL, all + addresses are greater than or equal to that. */ + if (lower_bound == const0_rtx) + { + if (!register_operand (upper_bound, ptr_mode)) + upper_bound = force_reg (ptr_mode, upper_bound); + + cond = arm_gen_compare_reg (GEU, cmpptr, upper_bound, NULL); + comparison = gen_rtx_GEU (VOIDmode, cond, const0_rtx); + } + else + { + /* We want to generate code for + result = (cmpptr < lower || cmpptr >= upper) ? 0 : *ptr; + Which can be recast to + result = (cmpptr < lower || upper <= cmpptr) ? 0 : *ptr; + which can be implemented as + cmp cmpptr, lower + cmpcs upper, cmpptr + bls 1f + ldr result, [ptr] + 1: + movls result, #0 + with suitable IT instructions as needed for thumb2. Later + optimization passes may make the load conditional. */ + + if (!register_operand (lower_bound, ptr_mode)) + lower_bound = force_reg (ptr_mode, lower_bound); + + if (!register_operand (upper_bound, ptr_mode)) + upper_bound = force_reg (ptr_mode, upper_bound); + + rtx comparison1 = gen_rtx_LTU (SImode, cmpptr, lower_bound); + rtx comparison2 = gen_rtx_LEU (SImode, upper_bound, cmpptr); + cond = gen_rtx_REG (arm_select_dominance_cc_mode (comparison1, + comparison2, + DOM_CC_X_OR_Y), + CC_REGNUM); + emit_insn (gen_cmp_ior (cmpptr, lower_bound, upper_bound, cmpptr, + comparison1, comparison2, cond)); + comparison = gen_rtx_NE (SImode, cond, const0_rtx); + } + + rtx_code_label *label = gen_label_rtx (); + emit_jump_insn (gen_arm_cond_branch (label, comparison, cond)); + emit_move_insn (target, mem); + emit_label (label); + + insn_code icode; + + /* We don't support TImode on Arm, but that can't currently be generated + for integral types on this architecture. */ + switch (mode) + { + case E_QImode: icode = CODE_FOR_nospeculateqi; break; + case E_HImode: icode = CODE_FOR_nospeculatehi; break; + case E_SImode: icode = CODE_FOR_nospeculatesi; break; + case E_DImode: icode = CODE_FOR_nospeculatedi; break; + default: + gcc_unreachable (); + } + + emit_insn (GEN_FCN (icode) (tgt2, comparison, cond, target, const0_rtx)); + + if (tgt2 != result) + emit_move_insn (result, tgt2); + + return result; +} + #if CHECKING_P namespace selftest { diff --git a/gcc/config/arm/arm.md b/gcc/config/arm/arm.md index d60c5af..e700fdf 100644 --- a/gcc/config/arm/arm.md +++ b/gcc/config/arm/arm.md @@ -9488,7 +9488,7 @@ (set_attr "type" "multiple")] ) -(define_insn "*cmp_ior" +(define_insn "cmp_ior" [(set (match_operand 6 "dominant_cc_register" "") (compare (ior:SI @@ -12015,6 +12015,44 @@ [(set_attr "length" "4") (set_attr "type" "coproc")]) +(define_insn "nospeculate" + [(set (match_operand:QHSI 0 "s_register_operand" "=l,l,r") + (unspec_volatile:QHSI + [(match_operator 1 "arm_comparison_operator" + [(match_operand 2 "cc_register" "") (const_int 0)]) + (match_operand:QHSI 3 "s_register_operand" "0,0,0") + (match_operand:QHSI 4 "arm_not_operand" "I,K,r")] + VUNSPEC_NOSPECULATE))] + "TARGET_32BIT" + { + if (TARGET_THUMB) + return \"it\\t%d1\;mov%d1\\t%0, %4\;.inst 0xf3af8014\t%@ CSDB\"; + return \"mov%d1\\t%0, %4\;.inst 0xe320f014\t%@ CSDB\"; + } + [(set_attr "type" "mov_imm,mvn_imm,mov_reg") + (set_attr "conds" "use") + (set_attr "length" "8")] +) + +(define_insn "nospeculatedi" + [(set (match_operand:DI 0 "s_register_operand" "=r") + (unspec_volatile:DI + [(match_operator 1 "arm_comparison_operator" + [(match_operand 2 "cc_register" "") (const_int 0)]) + (match_operand:DI 3 "s_register_operand" "0") + (match_operand:DI 4 "arm_rhs_operand" "rI")] + VUNSPEC_NOSPECULATE))] + "TARGET_32BIT" + { + if (TARGET_THUMB) + return \"it\\t%d1\;mov%d1\\t%Q0, %Q4\;it\\t%d1\;mov%d1\\t%R0, %R4\;.inst 0xf3af8014\t%@ CSDB\"; + return \"mov%d1\\t%Q0, %Q4\;mov%d1\\t%R0, %R4\;.inst 0xe320f014\t%@ CSDB\"; + } + [(set_attr "type" "mov_reg") + (set_attr "conds" "use") + (set_attr "length" "12")] +) + ;; Vector bits common to IWMMXT and Neon (include "vec-common.md") ;; Load the Intel Wireless Multimedia Extension patterns diff --git a/gcc/config/arm/unspecs.md b/gcc/config/arm/unspecs.md index c474f4b..727a5ab 100644 --- a/gcc/config/arm/unspecs.md +++ b/gcc/config/arm/unspecs.md @@ -168,6 +168,7 @@ VUNSPEC_MCRR2 ; Represent the coprocessor mcrr2 instruction. VUNSPEC_MRRC ; Represent the coprocessor mrrc instruction. VUNSPEC_MRRC2 ; Represent the coprocessor mrrc2 instruction. + VUNSPEC_NOSPECULATE ; Represent a despeculation sequence. ]) ;; Enumerators for NEON unspecs.