From patchwork Thu Apr 12 11:11:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 133199 Delivered-To: patch@linaro.org Received: by 10.46.84.29 with SMTP id i29csp1571157ljb; Thu, 12 Apr 2018 04:12:04 -0700 (PDT) X-Google-Smtp-Source: AIpwx4/0cmaYkRHm1O0/EvLMhZUpKm0CPtNUV+fnqx68jyNXhZOY0hhcg91cc5tCxqFtvvkiPK5M X-Received: by 10.98.8.133 with SMTP id 5mr7226404pfi.154.1523531524835; Thu, 12 Apr 2018 04:12:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523531524; cv=none; d=google.com; s=arc-20160816; b=F1XSRHP2A9KK4FQnZpoIeo6Pc509aY7WUJkzsG9IP6lCwpY+Uaej3vSNLLOJCoGStQ pFpuX/6jZL8IjDm9ZG8q4t/SOhvdNQPCW256WO1CrI2grmdg53axyEn8dgAsd67uZ2c6 8Tu6kaMW8Io28p289Wc1sxfdp4CLH42FTbm2oMIwm5AGEgoA9epKmJuwUSkRU+ZpenMt Yc0T5zhl5vK9T/00JI+ZcA40fftfpkvicI5/+L5hZxsuPkT42VLbw/0YRRYPm20BotEO wD4dTy6SCmS16HjgE7/mBLTSEzDkk3itUMvjkVrcoluV/U10bgeIYggT/uiTst/I78Ny jurg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=tbdZnx8oT7TLvrCi4LBJppKVatJqGN+9CnuprN+ZIVk=; b=GZFW9aF6OvWkKsqvpuhPxjDxKQSRB17seBkl3UAUeNLA1LUNPAl1Ox4+SeKidTwyUY 5b4KIp/GAS35cfN6QDRqCPYIJRYBn+dmQ6IBMw2egtIYkcxSh0iaLvqNsSzqp6JoAEbi n0+I2ZTbu6eW5KluECBNqwZqk7OX7PfGuqmQ1g+jL/vlfBbDZYFR2LZCquE5FjrEuWag 1x1cl2RdBh6mbBTnjkqX6eCbC8tyGjASt7W5vZlk1LDohMoY9WEL4ggq+MXSPwOyZWVS lmtvWxMxiYLdXa+YvP2aBl8bcTpbKbkTys73rs4exsSjmxdOCtExq5At1ZzofGkt3hOS QZjA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id bg3-v6si3001807plb.118.2018.04.12.04.12.04; Thu, 12 Apr 2018 04:12:04 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752761AbeDLLME (ORCPT + 11 others); Thu, 12 Apr 2018 07:12:04 -0400 Received: from foss.arm.com ([217.140.101.70]:59326 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752428AbeDLLMD (ORCPT ); Thu, 12 Apr 2018 07:12:03 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 441F580D; Thu, 12 Apr 2018 04:12:03 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9A5F93F24A; Thu, 12 Apr 2018 04:12:01 -0700 (PDT) From: Mark Rutland To: stable@vger.kernel.org Cc: mark.brown@linaro.org, ard.biesheuvel@linaro.org, marc.zyngier@arm.com, will.deacon@arm.com, catalin.marinas@arm.com, ghackmann@google.com, shankerd@codeaurora.org Subject: [PATCH v4.9.y 04/42] arm64: Make USER_DS an inclusive limit Date: Thu, 12 Apr 2018 12:11:00 +0100 Message-Id: <20180412111138.40990-5-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180412111138.40990-1-mark.rutland@arm.com> References: <20180412111138.40990-1-mark.rutland@arm.com> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Robin Murphy commit 51369e398d0d33e8f524314e672b07e8cf870e79 upstream. Currently, USER_DS represents an exclusive limit while KERNEL_DS is inclusive. In order to do some clever trickery for speculation-safe masking, we need them both to behave equivalently - there aren't enough bits to make KERNEL_DS exclusive, so we have precisely one option. This also happens to correct a longstanding false negative for a range ending on the very top byte of kernel memory. Mark Rutland points out that we've actually got the semantics of addresses vs. segments muddled up in most of the places we need to amend, so shuffle the {USER,KERNEL}_DS definitions around such that we can correct those properly instead of just pasting "-1"s everywhere. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas [v4.9: avoid dependence on TTBR0 SW PAN and THREAD_INFO_IN_TASK_STRUCT] Signed-off-by: Mark Rutland [v4.9 backport] --- arch/arm64/include/asm/processor.h | 3 +++ arch/arm64/include/asm/uaccess.h | 46 ++++++++++++++++++++++---------------- arch/arm64/kernel/entry.S | 4 ++-- arch/arm64/mm/fault.c | 2 +- 4 files changed, 33 insertions(+), 22 deletions(-) -- 2.11.0 diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 4258f4d594b5..5917147af0c4 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -21,6 +21,9 @@ #define TASK_SIZE_64 (UL(1) << VA_BITS) +#define KERNEL_DS UL(-1) +#define USER_DS (TASK_SIZE_64 - 1) + #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 811cf16a65f9..0f05ba51531c 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -28,6 +28,7 @@ #include #include +#include #include #include #include @@ -59,10 +60,7 @@ struct exception_table_entry extern int fixup_exception(struct pt_regs *regs); -#define KERNEL_DS (-1UL) #define get_ds() (KERNEL_DS) - -#define USER_DS TASK_SIZE_64 #define get_fs() (current_thread_info()->addr_limit) static inline void set_fs(mm_segment_t fs) @@ -87,22 +85,32 @@ static inline void set_fs(mm_segment_t fs) * Returns 1 if the range is valid, 0 otherwise. * * This is equivalent to the following test: - * (u65)addr + (u65)size <= current->addr_limit - * - * This needs 65-bit arithmetic. + * (u65)addr + (u65)size <= (u65)current->addr_limit + 1 */ -#define __range_ok(addr, size) \ -({ \ - unsigned long __addr = (unsigned long __force)(addr); \ - unsigned long flag, roksum; \ - __chk_user_ptr(addr); \ - asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, ls" \ - : "=&r" (flag), "=&r" (roksum) \ - : "1" (__addr), "Ir" (size), \ - "r" (current_thread_info()->addr_limit) \ - : "cc"); \ - flag; \ -}) +static inline unsigned long __range_ok(unsigned long addr, unsigned long size) +{ + unsigned long limit = current_thread_info()->addr_limit; + + __chk_user_ptr(addr); + asm volatile( + // A + B <= C + 1 for all A,B,C, in four easy steps: + // 1: X = A + B; X' = X % 2^64 + " adds %0, %0, %2\n" + // 2: Set C = 0 if X > 2^64, to guarantee X' > C in step 4 + " csel %1, xzr, %1, hi\n" + // 3: Set X' = ~0 if X >= 2^64. For X == 2^64, this decrements X' + // to compensate for the carry flag being set in step 4. For + // X > 2^64, X' merely has to remain nonzero, which it does. + " csinv %0, %0, xzr, cc\n" + // 4: For X < 2^64, this gives us X' - C - 1 <= 0, where the -1 + // comes from the carry in being clear. Otherwise, we are + // testing X' - C == 0, subject to the previous adjustments. + " sbcs xzr, %0, %1\n" + " cset %0, ls\n" + : "+r" (addr), "+r" (limit) : "Ir" (size) : "cc"); + + return addr; +} /* * When dealing with data aborts, watchpoints, or instruction traps we may end @@ -111,7 +119,7 @@ static inline void set_fs(mm_segment_t fs) */ #define untagged_addr(addr) sign_extend64(addr, 55) -#define access_ok(type, addr, size) __range_ok(addr, size) +#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size) #define user_addr_max get_fs #define _ASM_EXTABLE(from, to) \ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index e10e9f2e48ad..3f1cd7a795bb 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -126,10 +126,10 @@ alternative_else_nop_endif .else add x21, sp, #S_FRAME_SIZE get_thread_info tsk - /* Save the task's original addr_limit and set USER_DS (TASK_SIZE_64) */ + /* Save the task's original addr_limit and set USER_DS */ ldr x20, [tsk, #TI_ADDR_LIMIT] str x20, [sp, #S_ORIG_ADDR_LIMIT] - mov x20, #TASK_SIZE_64 + mov x20, #USER_DS str x20, [tsk, #TI_ADDR_LIMIT] /* No need to reset PSTATE.UAO, hardware's already set it to 0 for us */ .endif /* \el == 0 */ diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 403fe9e57135..4df70c9fd762 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -332,7 +332,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } - if (is_permission_fault(esr) && (addr < USER_DS)) { + if (is_permission_fault(esr) && (addr < TASK_SIZE)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) die("Accessing user space memory with fs=KERNEL_DS", regs, esr);