From patchwork Tue Apr 3 11:09:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 132719 Delivered-To: patch@linaro.org Received: by 10.46.84.29 with SMTP id i29csp3666071ljb; Tue, 3 Apr 2018 04:10:16 -0700 (PDT) X-Google-Smtp-Source: AIpwx49WhIx169kyogPHy2cIqiuIIbKnBiaW/F58pAlJgtsOQqA774nHv+sMz+fFjxxawJzEZHOK X-Received: by 2002:a17:902:9a44:: with SMTP id x4-v6mr13642363plv.312.1522753816527; Tue, 03 Apr 2018 04:10:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522753816; cv=none; d=google.com; s=arc-20160816; b=AzlksoCO35/OrN6YtuQfW2M1O2aDlJJbtLroIqb1sZeFjlpurN7PseuouK3WtDjexD CcirJMv0Z6xb7YcXc0URLZagQ/uYz4Rpnf0QT2ePdMgQUX+fkIYcjmOVCAjOheSSnisv 4is9u4jpKOQ0+Iv+bZOcN4XiwnH4kSBMjjPo6hEqoSEECfVOnBH3FmysrGWd1R/Q3Wpc Cb5EWHZlaUaDcMXEiaTfJ2c0QsN7J4Niarz4fdBbezfMsZk2llyOw46vzd2FXG73gfzH ZjVIdEviPT2A/9ONTnU4cLWv7hvb6SsPPXzAdpbkkPPWtg2lP//avhudsXCxFdN5966F vtJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=rKcJsu4nXOhVOdZreRrI1YIOBRzprPuT+zJqzRfJL5g=; b=fX8PRGc5IKs6cOruuTCbsMJQYkA2MiEBb26XYqvaAhcN8CRWCUar3dtqajtQSdmcBn iJhZD6QtJZFnjQD9GXlaQvRxvZPwtofjBGguoagUGjoenTkpGCTi8K4zPcKHq06ZKaF6 4wl/xcyqCau7lSDWylqO2GQdDO01kLHmztE5faHluGApGld8jEpjvDEFuQNnC97vVDy9 X4Y4koSZSWCfqneFOTplhiT7lourizbn5m0B7aARscxOv8jkzbMVzPHTySBKj2uMzucM ETJuwPERI7SoF3z+Q0FIli9H6fE3z6Kzs2HXHcDqKgaihkKY63PRhe/KvK8WtfCX8TmR GXZA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l91-v6si306341plb.301.2018.04.03.04.10.16; Tue, 03 Apr 2018 04:10:16 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755483AbeDCLKN (ORCPT + 11 others); Tue, 3 Apr 2018 07:10:13 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:59352 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755432AbeDCLKM (ORCPT ); Tue, 3 Apr 2018 07:10:12 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 38E1C1435; Tue, 3 Apr 2018 04:10:12 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0858D3F587; Tue, 3 Apr 2018 04:10:10 -0700 (PDT) From: Mark Rutland To: stable@vger.kernel.org Cc: mark.brown@linaro.org, ard.biesheuvel@linaro.org, marc.zyngier@arm.com, will.deacon@arm.com Subject: [PATCH v4.9.y 13/27] arm64: entry: Add fake CPU feature for unmapping the kernel at EL0 Date: Tue, 3 Apr 2018 12:09:09 +0100 Message-Id: <20180403110923.43575-14-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180403110923.43575-1-mark.rutland@arm.com> References: <20180403110923.43575-1-mark.rutland@arm.com> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Will Deacon commit ea1e3de85e94 upstream. Allow explicit disabling of the entry trampoline on the kernel command line (kpti=off) by adding a fake CPU feature (ARM64_UNMAP_KERNEL_AT_EL0) that can be used to toggle the alternative sequences in our entry code and avoid use of the trampoline altogether if desired. This also allows us to make use of a static key in arm64_kernel_unmapped_at_el0(). Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon [Alex: use first free cpucap number, use cpus_have_cap] Signed-off-by: Alex Shi [v4.9 backport] Signed-off-by: Mark Rutland [v4.9 backport] --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/mmu.h | 3 ++- arch/arm64/kernel/cpufeature.c | 41 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/entry.S | 9 +++++---- 4 files changed, 50 insertions(+), 6 deletions(-) -- 2.11.0 diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 87b446535185..7ddf233f05bd 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -34,7 +34,8 @@ #define ARM64_HAS_32BIT_EL0 13 #define ARM64_HYP_OFFSET_LOW 14 #define ARM64_MISMATCHED_CACHE_LINE_SIZE 15 +#define ARM64_UNMAP_KERNEL_AT_EL0 16 -#define ARM64_NCAPS 16 +#define ARM64_NCAPS 17 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 279e75b8a49e..a813edf28737 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -34,7 +34,8 @@ typedef struct { static inline bool arm64_kernel_unmapped_at_el0(void) { - return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0); + return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) && + cpus_have_cap(ARM64_UNMAP_KERNEL_AT_EL0); } extern void paging_init(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 3a129d48674e..74b168c51abd 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -746,6 +746,40 @@ static bool hyp_offset_low(const struct arm64_cpu_capabilities *entry, return idmap_addr > GENMASK(VA_BITS - 2, 0) && !is_kernel_in_hyp_mode(); } +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */ + +static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, + int __unused) +{ + /* Forced on command line? */ + if (__kpti_forced) { + pr_info_once("kernel page table isolation forced %s by command line option\n", + __kpti_forced > 0 ? "ON" : "OFF"); + return __kpti_forced > 0; + } + + /* Useful for KASLR robustness */ + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + return true; + + return false; +} + +static int __init parse_kpti(char *str) +{ + bool enabled; + int ret = strtobool(str, &enabled); + + if (ret) + return ret; + + __kpti_forced = enabled ? 1 : -1; + return 0; +} +__setup("kpti=", parse_kpti); +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -829,6 +863,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .def_scope = SCOPE_SYSTEM, .matches = hyp_offset_low, }, +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + { + .capability = ARM64_UNMAP_KERNEL_AT_EL0, + .def_scope = SCOPE_SYSTEM, + .matches = unmap_kernel_at_el0, + }, +#endif {}, }; diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 623c160bf68e..b16d0534cda3 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -73,6 +73,7 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 .if \el == 0 .if \regsize == 64 mrs x30, tpidrro_el0 @@ -81,6 +82,7 @@ mov x30, xzr .endif .endif +alternative_else_nop_endif #endif sub sp, sp, #S_FRAME_SIZE @@ -208,10 +210,9 @@ alternative_else_nop_endif ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp -#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 - eret -#else .if \el == 0 +alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 bne 4f msr far_el1, x30 tramp_alias x30, tramp_exit_native @@ -219,10 +220,10 @@ alternative_else_nop_endif 4: tramp_alias x30, tramp_exit_compat br x30 +#endif .else eret .endif -#endif .endm .macro get_thread_info, rd