From patchwork Wed Oct 31 13:56:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Long X-Patchwork-Id: 149802 Delivered-To: patches@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp6826458ljp; Wed, 31 Oct 2018 06:57:27 -0700 (PDT) X-Received: by 2002:a1f:cc86:: with SMTP id c128mr1264935vkg.77.1540994247500; Wed, 31 Oct 2018 06:57:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540994247; cv=none; d=google.com; s=arc-20160816; b=oec9IzPr9oXxMVcq8E6ApQD18Pf04bPZsA97+vIznzOfKHhDKuweja15tgGCHB65Zn 7TLbhhiU0YotmPJ/MxdfP94nPaWB8hbCAAoR+xK+Y62A2Fy1WSCvGG2TuD9c9ckT3Z8h 9RYUMg1tR9Rcykn2rZvr0sCSM3ebDdCCZ6idz9J5gm1rj23LbiAiWwIEhQldqkoYn+gO +Eelk5jkaygIC/Kpbn83KhRY22nlkEwZAUt9//TDoRJdfqGFojX9QL32HVTDb7BwkDy2 ke1UOUvyGmsPMVFzfnoZYIjD0/JZhNHrSaY4nX2Msxlm/kUYY/ULo/CpgJGKUm5ddTJm 5GHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=RTypk7ytVSq6HRP039KKlUhlZd4ojNgO2ok3F47UQ/c=; b=RuUHEBcpLQppK3+fLpG/CcDuBPz9vDszRIUF7NreEpj+uUumOvpcoxWwKbwmSn742q 6lAL70vHuX5simOLcR3AkcZJl6Ic/KqxF4YXdPxPaew/IGgsmG/sA+xLNz+2RxZwbGOG Be+xprxTd4c+z9+nlp3UiYZLcazsWr83JHIj7zX9QfnJJqAo9aAxLKuoC3INWxWT5prJ +40UkaG+/LO6bvEpiTpAf79qita48vR4xBRaaDT5JZF6uvns6egPF8SMjykM9cM/l34M 0gL21rOHQDVOJIvAJCfULaJcYuPZbEgrG+ShIOEqU1eROtNF2RCZfCf9pTbglwLAsGOF Iwkg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bvTQZe4p; spf=pass (google.com: domain of dave.long@linaro.org designates 209.85.220.41 as permitted sender) smtp.mailfrom=dave.long@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id y24sor14583464uae.36.2018.10.31.06.57.27 for (Google Transport Security); Wed, 31 Oct 2018 06:57:27 -0700 (PDT) Received-SPF: pass (google.com: domain of dave.long@linaro.org designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bvTQZe4p; spf=pass (google.com: domain of dave.long@linaro.org designates 209.85.220.41 as permitted sender) smtp.mailfrom=dave.long@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RTypk7ytVSq6HRP039KKlUhlZd4ojNgO2ok3F47UQ/c=; b=bvTQZe4pXLgDaJiGOmig0jKwwXBgDbRmRqBou+V/wKc4f9XUYcbwnsQMHOrwOx7xRv 7x4o2IxHFelZxnpPo/Y5zsbeggFWht4jwAO7iiEhDrk+nUqdWqugmy3SiZFF9m1K90Hj L8ycjyf34WrUrEAjyUkt/roEQ2Xm6TGrMilxE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RTypk7ytVSq6HRP039KKlUhlZd4ojNgO2ok3F47UQ/c=; b=FuizAOM+ZX6fnQCRkCaYWNh3or988DMnbetRKvOAzMu5Vcc1YwjlsEJjrFt2TCvksc UeWFeVGwFj9d/vmzuDocfZxqULbubYn1XIEudhZ1uw5bzGOyGYXHlFGZ+0bFffAuJCGZ pMAxE1A6wbq8izpNR5mXzarVQ4Ps4O1QYyZQBos75XYm5trp9C2MFBbHCREmbzfoIUQH S3mvmU1h0pgR7BLzNcvtAlNIDQ8pEgq0c7we0IoNNFtqV3thtrVTW70/kig/I1Gx0PRH hC3+hpFVH/sPddSnSsVW4vEpwOCLtY31xEtSabj+oRZq9y01wc7VTrep8Xst1q0fw3Il aXfg== X-Gm-Message-State: AGRZ1gKQsAtcZAOzxsi323ZUt9qfcq+8brMEitOoGOMulhVoxsx151Xi mBjpNWlwlmtggxQlDYcB8He2PB4njbEOZQ== X-Google-Smtp-Source: AJdET5eLEGKjedllltHNV4qmQ91Rg16raLh0UMnc1ziqKhavHG6DaRRLq8ADPy7BTxrPcRzGON6Thw== X-Received: by 2002:ab0:526:: with SMTP id 35mr1333471uax.84.1540994246935; Wed, 31 Oct 2018 06:57:26 -0700 (PDT) Return-Path: Received: from dave-Dell-System-XPS-L502X.hsd1.nh.comcast.net ([2603:3005:3403:7100:2c71:8680:34e1:a6aa]) by smtp.googlemail.com with ESMTPSA id s85-v6sm2275624vse.29.2018.10.31.06.57.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 31 Oct 2018 06:57:26 -0700 (PDT) From: David Long To: stable@vger.kernel.org, Russell King - ARM Linux , Florian Fainelli , Tony Lindgren , Marc Zyngier , Mark Rutland Cc: Greg KH , Mark Brown Subject: [PATCH 4.9 08/24] ARM: spectre-v2: harden user aborts in kernel space Date: Wed, 31 Oct 2018 09:56:57 -0400 Message-Id: <20181031135713.2873-9-dave.long@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181031135713.2873-1-dave.long@linaro.org> References: <20181031135713.2873-1-dave.long@linaro.org> From: Russell King Commit f5fe12b1eaee220ce62ff9afb8b90929c396595f upstream. In order to prevent aliasing attacks on the branch predictor, invalidate the BTB or instruction cache on CPUs that are known to be affected when taking an abort on a address that is outside of a user task limit: Cortex A8, A9, A12, A17, A73, A75: flush BTB. Cortex A15, Brahma B15: invalidate icache. If the IBE bit is not set, then there is little point to enabling the workaround. Signed-off-by: Russell King Boot-tested-by: Tony Lindgren Reviewed-by: Tony Lindgren Signed-off-by: David A. Long --- arch/arm/include/asm/cp15.h | 3 ++ arch/arm/include/asm/system_misc.h | 15 ++++++ arch/arm/mm/fault.c | 3 ++ arch/arm/mm/proc-v7-bugs.c | 73 ++++++++++++++++++++++++++++-- arch/arm/mm/proc-v7.S | 8 ++-- 5 files changed, 94 insertions(+), 8 deletions(-) -- 2.17.1 diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h index dbdbce1b3a72..b74b174ac9fc 100644 --- a/arch/arm/include/asm/cp15.h +++ b/arch/arm/include/asm/cp15.h @@ -64,6 +64,9 @@ #define __write_sysreg(v, r, w, c, t) asm volatile(w " " c : : "r" ((t)(v))) #define write_sysreg(v, ...) __write_sysreg(v, __VA_ARGS__) +#define BPIALL __ACCESS_CP15(c7, 0, c5, 6) +#define ICIALLU __ACCESS_CP15(c7, 0, c5, 0) + extern unsigned long cr_alignment; /* defined in entry-armv.S */ static inline unsigned long get_cr(void) diff --git a/arch/arm/include/asm/system_misc.h b/arch/arm/include/asm/system_misc.h index a3d61ad984af..1fed41440af9 100644 --- a/arch/arm/include/asm/system_misc.h +++ b/arch/arm/include/asm/system_misc.h @@ -7,6 +7,7 @@ #include #include #include +#include extern void cpu_init(void); @@ -14,6 +15,20 @@ void soft_restart(unsigned long); extern void (*arm_pm_restart)(enum reboot_mode reboot_mode, const char *cmd); extern void (*arm_pm_idle)(void); +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +typedef void (*harden_branch_predictor_fn_t)(void); +DECLARE_PER_CPU(harden_branch_predictor_fn_t, harden_branch_predictor_fn); +static inline void harden_branch_predictor(void) +{ + harden_branch_predictor_fn_t fn = per_cpu(harden_branch_predictor_fn, + smp_processor_id()); + if (fn) + fn(); +} +#else +#define harden_branch_predictor() do { } while (0) +#endif + #define UDBG_UNDEFINED (1 << 0) #define UDBG_SYSCALL (1 << 1) #define UDBG_BADABORT (1 << 2) diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index f7861dc83182..5ca207ada852 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -163,6 +163,9 @@ __do_user_fault(struct task_struct *tsk, unsigned long addr, { struct siginfo si; + if (addr > TASK_SIZE) + harden_branch_predictor(); + #ifdef CONFIG_DEBUG_USER if (((user_debug & UDBG_SEGV) && (sig == SIGSEGV)) || ((user_debug & UDBG_BUS) && (sig == SIGBUS))) { diff --git a/arch/arm/mm/proc-v7-bugs.c b/arch/arm/mm/proc-v7-bugs.c index e46557db6446..85a2e3d6263c 100644 --- a/arch/arm/mm/proc-v7-bugs.c +++ b/arch/arm/mm/proc-v7-bugs.c @@ -2,7 +2,61 @@ #include #include -static __maybe_unused void cpu_v7_check_auxcr_set(bool *warned, +#include +#include +#include + +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +DEFINE_PER_CPU(harden_branch_predictor_fn_t, harden_branch_predictor_fn); + +static void harden_branch_predictor_bpiall(void) +{ + write_sysreg(0, BPIALL); +} + +static void harden_branch_predictor_iciallu(void) +{ + write_sysreg(0, ICIALLU); +} + +static void cpu_v7_spectre_init(void) +{ + const char *spectre_v2_method = NULL; + int cpu = smp_processor_id(); + + if (per_cpu(harden_branch_predictor_fn, cpu)) + return; + + switch (read_cpuid_part()) { + case ARM_CPU_PART_CORTEX_A8: + case ARM_CPU_PART_CORTEX_A9: + case ARM_CPU_PART_CORTEX_A12: + case ARM_CPU_PART_CORTEX_A17: + case ARM_CPU_PART_CORTEX_A73: + case ARM_CPU_PART_CORTEX_A75: + per_cpu(harden_branch_predictor_fn, cpu) = + harden_branch_predictor_bpiall; + spectre_v2_method = "BPIALL"; + break; + + case ARM_CPU_PART_CORTEX_A15: + case ARM_CPU_PART_BRAHMA_B15: + per_cpu(harden_branch_predictor_fn, cpu) = + harden_branch_predictor_iciallu; + spectre_v2_method = "ICIALLU"; + break; + } + if (spectre_v2_method) + pr_info("CPU%u: Spectre v2: using %s workaround\n", + smp_processor_id(), spectre_v2_method); +} +#else +static void cpu_v7_spectre_init(void) +{ +} +#endif + +static __maybe_unused bool cpu_v7_check_auxcr_set(bool *warned, u32 mask, const char *msg) { u32 aux_cr; @@ -13,24 +67,33 @@ static __maybe_unused void cpu_v7_check_auxcr_set(bool *warned, if (!*warned) pr_err("CPU%u: %s", smp_processor_id(), msg); *warned = true; + return false; } + return true; } static DEFINE_PER_CPU(bool, spectre_warned); -static void check_spectre_auxcr(bool *warned, u32 bit) +static bool check_spectre_auxcr(bool *warned, u32 bit) { - if (IS_ENABLED(CONFIG_HARDEN_BRANCH_PREDICTOR) && + return IS_ENABLED(CONFIG_HARDEN_BRANCH_PREDICTOR) && cpu_v7_check_auxcr_set(warned, bit, "Spectre v2: firmware did not set auxiliary control register IBE bit, system vulnerable\n"); } void cpu_v7_ca8_ibe(void) { - check_spectre_auxcr(this_cpu_ptr(&spectre_warned), BIT(6)); + if (check_spectre_auxcr(this_cpu_ptr(&spectre_warned), BIT(6))) + cpu_v7_spectre_init(); } void cpu_v7_ca15_ibe(void) { - check_spectre_auxcr(this_cpu_ptr(&spectre_warned), BIT(0)); + if (check_spectre_auxcr(this_cpu_ptr(&spectre_warned), BIT(0))) + cpu_v7_spectre_init(); +} + +void cpu_v7_bugs_init(void) +{ + cpu_v7_spectre_init(); } diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S index 4e4f794f17ce..2d2e5ae85816 100644 --- a/arch/arm/mm/proc-v7.S +++ b/arch/arm/mm/proc-v7.S @@ -527,8 +527,10 @@ __v7_setup_stack: __INITDATA + .weak cpu_v7_bugs_init + @ define struct processor (see and proc-macros.S) - define_processor_functions v7, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 + define_processor_functions v7, dabort=v7_early_abort, pabort=v7_pabort, suspend=1, bugs=cpu_v7_bugs_init #ifdef CONFIG_HARDEN_BRANCH_PREDICTOR @ generic v7 bpiall on context switch @@ -543,7 +545,7 @@ __v7_setup_stack: globl_equ cpu_v7_bpiall_do_suspend, cpu_v7_do_suspend globl_equ cpu_v7_bpiall_do_resume, cpu_v7_do_resume #endif - define_processor_functions v7_bpiall, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 + define_processor_functions v7_bpiall, dabort=v7_early_abort, pabort=v7_pabort, suspend=1, bugs=cpu_v7_bugs_init #define HARDENED_BPIALL_PROCESSOR_FUNCTIONS v7_bpiall_processor_functions #else @@ -579,7 +581,7 @@ __v7_setup_stack: globl_equ cpu_ca9mp_switch_mm, cpu_v7_switch_mm #endif globl_equ cpu_ca9mp_set_pte_ext, cpu_v7_set_pte_ext - define_processor_functions ca9mp, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 + define_processor_functions ca9mp, dabort=v7_early_abort, pabort=v7_pabort, suspend=1, bugs=cpu_v7_bugs_init #endif @ Cortex-A15 - needs iciallu switch_mm for hardening