From patchwork Fri Jan 5 14:57:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 123529 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp910905qgn; Fri, 5 Jan 2018 06:58:16 -0800 (PST) X-Google-Smtp-Source: ACJfBotUiFEde5oqbILDa0UT6C2fwlPFnAe92Keak0IMI3yrUAuhS9epHH/gNyX+GkoaEJszlv23 X-Received: by 10.101.92.66 with SMTP id v2mr2821012pgr.37.1515164296542; Fri, 05 Jan 2018 06:58:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515164296; cv=none; d=google.com; s=arc-20160816; b=VmRg3nwGaXxVRZ8grrzQ5eGQOEB9OVl3Un54C6oGP087LHk/ittLkjrHuDKIy2Urze KuvHp1ZbRsH0nllK+o2w+IRe9LQg6OUGHksXv1nxshATggYyyIKR+GotbMkXzYJRAdDF /4M06smO2dYdNIYXyujV63I41yXhYCQsA2B6Zg7UWqweTqcg91VNPYRZDrBirKA6VzaK viVtB+bk5TYikr/uivzaIZVOXA/ofjxg3/qJsYwJ6T4Yg6rx221WTy/aplVJr2SNLfIp X5sey+vWIlN0reYj4nli4+LQjO//9+pl9HhiPcWnGvjnBqijidLV6faZ+HtYfLEAUwVl y/oA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=MnZKqmMPmBImvWxbqYQitaeW6PpeTXGCjZlv633zhyQ=; b=d15A01IWu0oJLukk0to4yNFg5Ji/rqoW2poafY2cBU5tSxHlYH2m8VLvPlR60/zX3X MLHQeUuku4Hlt5q5sPX+fAkWU9478m4NtULC1ybdvYLBc6XhSuHWcgU6vkFKZow9dS4X P1dqCzZmecNqh2Qc9zhch6WHNMbpIYjFDIZOG6/Or2hwTHV6k2AKUVgT10qLo0YbGZEy g8hzX5+W6hxw4OpYDhn27hmeMLAX1QcSR0EcaE1MhOSV3mS6ZwsjjXDqPWPd+71d1Ve1 mSScPm9oZ6jshDFj9AEi/ZmWdz8HwcN260/Pn+k8a80KKqOsnyWwsKrjyuw88GofhLAW Scdw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 33si4077516ply.308.2018.01.05.06.58.16; Fri, 05 Jan 2018 06:58:16 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752240AbeAEO6O (ORCPT + 26 others); Fri, 5 Jan 2018 09:58:14 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:46086 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752195AbeAEO6J (ORCPT ); Fri, 5 Jan 2018 09:58:09 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D2EFE15BF; Fri, 5 Jan 2018 06:58:08 -0800 (PST) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E375E3F581; Fri, 5 Jan 2018 06:58:06 -0800 (PST) From: Mark Rutland To: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org Cc: dan.j.williams@intel.com, elena.reshetova@intel.com, corbet@lwn.net, alan@linux.intel.com, peterz@infradead.org, will.deacon@arm.com, gregkh@linuxfoundation.org, tglx@linutronix.de, Mark Rutland Subject: [RFCv2 3/4] arm64: implement nospec_ptr() Date: Fri, 5 Jan 2018 14:57:49 +0000 Message-Id: <20180105145750.53294-4-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180105145750.53294-1-mark.rutland@arm.com> References: <20180105145750.53294-1-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch implements nospec_ptr() for arm64, following the recommended architectural sequence. Signed-off-by: Mark Rutland Signed-off-by: Will Deacon Cc: Dan Williams Cc: Peter Zijlstra --- arch/arm64/include/asm/barrier.h | 55 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) -- 2.11.0 diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index 77651c49ef44..b4819f6a0e5c 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -40,6 +40,61 @@ #define dma_rmb() dmb(oshld) #define dma_wmb() dmb(oshst) +#define __load_no_speculate_n(ptr, lo, hi, failval, cmpptr, w, sz) \ +({ \ + typeof(*ptr) __nln_val; \ + typeof(*ptr) __failval = \ + (typeof(*ptr))(unsigned long)(failval); \ + \ + asm volatile ( \ + " cmp %[c], %[l]\n" \ + " ccmp %[c], %[h], 2, cs\n" \ + " b.cs 1f\n" \ + " ldr" #sz " %" #w "[v], %[p]\n" \ + "1: csel %" #w "[v], %" #w "[v], %" #w "[f], cc\n" \ + " hint #0x14 // CSDB\n" \ + : [v] "=&r" (__nln_val) \ + : [p] "m" (*(ptr)), [l] "r" (lo), [h] "r" (hi), \ + [f] "rZ" (__failval), [c] "r" (cmpptr) \ + : "cc"); \ + \ + __nln_val; \ +}) + +#define __load_no_speculate(ptr, lo, hi, failval, cmpptr) \ +({ \ + typeof(*(ptr)) __nl_val; \ + \ + switch (sizeof(__nl_val)) { \ + case 1: \ + __nl_val = __load_no_speculate_n(ptr, lo, hi, failval, \ + cmpptr, w, b); \ + break; \ + case 2: \ + __nl_val = __load_no_speculate_n(ptr, lo, hi, failval, \ + cmpptr, w, h); \ + break; \ + case 4: \ + __nl_val = __load_no_speculate_n(ptr, lo, hi, failval, \ + cmpptr, w, ); \ + break; \ + case 8: \ + __nl_val = __load_no_speculate_n(ptr, lo, hi, failval, \ + cmpptr, x, ); \ + break; \ + default: \ + BUILD_BUG(); \ + } \ + \ + __nl_val; \ +}) + +#define nospec_ptr(ptr, lo, hi) \ +({ \ + typeof(ptr) __np_ptr = (ptr); \ + __load_no_speculate(&__np_ptr, lo, hi, 0, __np_ptr); \ +}) + #define __smp_mb() dmb(ish) #define __smp_rmb() dmb(ishld) #define __smp_wmb() dmb(ishst)