From patchwork Thu Aug 1 08:16:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 170351 Delivered-To: patch@linaro.org Received: by 2002:a92:512:0:0:0:0:0 with SMTP id q18csp5087175ile; Thu, 1 Aug 2019 01:21:33 -0700 (PDT) X-Google-Smtp-Source: APXvYqxb9Ysi8+PtKwdydFe+/lHbyI0DruX3IWPgQu0Gk+z53Hex6yy9HzftAgYg48lB2SYhbmg9 X-Received: by 2002:a17:90a:8985:: with SMTP id v5mr7130892pjn.136.1564647693634; Thu, 01 Aug 2019 01:21:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564647693; cv=none; d=google.com; s=arc-20160816; b=kUdK6KNzX7yWtksWhcaEeMuQky7/4ngqZ6mx+dOyeIPA2m6n19TVDTFp7l4T5nzfin Ynkmg8rTjP0vUFj53Q3w2916sY/HbU60XejwUBIUk3meXlWXDV9S/ta3ZLFfkCv1+xxg KwP4y/PU2ElLYh2XUJ+kFLbVBzasdbkSr/yVzEgMQBWAzxEEcNIlLLsVT/55SzXJ2suZ SA0JWdgG7KFOvqqOMCOYdyWikqWk+XxsoR5VNwlVtlgDZw2UU+Q5NfSGUBnlwkkX+731 ra0Yq2YHdUC+w+jrXRt3zdkY3Tgo3bZ2x4DJWTOY1KsqxWU4GQcRvDn6lfuyDQ645BMu uVjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=H4B5LlN5Nm5PcO8sp/nxTYTyQFk96hnkKyUFVVoH87s=; b=SE39IvU83LjFTtGQd+WEHhYoNtawumIfMr1NpcyJG+K/gxteZKXsdteuRqU8l86Ueh bMeyM+6hSa+oGZJsM/MELW2cAKsvbTrw5nj9Xklf40k5JGZECWZPfaMnSjRjWOQHJ+Qe tPfEOEFw7Pj9+4vEDUkdSCUb34ulr0qrexCqNvsfoJFMzP2NdQsq3Ab0vzYptaJVXfec godnOxea0BWJOjFQcmq7mu9+RXImrlwO0zzfXqMGHJCWwJBiVLv8ZfVafnDYsoMNBce6 Nd02HLf0tAkXV90kt622iX6TAjLjqL+MAUKK73Sxqp21V5MV9daxJYGNXCDA2noFL8n7 igMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MaUgCHGQ; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z13si4013236pgj.476.2019.08.01.01.21.33; Thu, 01 Aug 2019 01:21:33 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MaUgCHGQ; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727490AbfHAIVc (ORCPT + 14 others); Thu, 1 Aug 2019 04:21:32 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:34636 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725958AbfHAIVc (ORCPT ); Thu, 1 Aug 2019 04:21:32 -0400 Received: by mail-pf1-f196.google.com with SMTP id b13so33609988pfo.1 for ; Thu, 01 Aug 2019 01:21:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=H4B5LlN5Nm5PcO8sp/nxTYTyQFk96hnkKyUFVVoH87s=; b=MaUgCHGQ6ZRUCIQBTSvrIEWWynAHGTPU5ZoAItdNZxUBR+fcHgjURqzz8mfkt5vOZV nwKXcoMT3zfoLnhKb9m/DLhWqhy6YxYBG1PsyLVL7M/7dOLRGGYQNnkguMBWr2Bd+Q9j zGUL5cqCpYnb6FoTarylw4LsyauiexM1aY3KYMJBNnbMcsEAphLMikrYHYJbS+BwcZ+/ EJV/8X8e3LxZVxVT3qH9DJlEid6CYeNzbjM/OpBjasc7t78Zz/3VyPrvY7b31GAzKYZm Za9yZBFICuAxsXpzLrZ8Jke/6ClA8b/rzCMDJlJckDEg0HTigxBSrf+rVLfNKk8n7VOF 1FAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=H4B5LlN5Nm5PcO8sp/nxTYTyQFk96hnkKyUFVVoH87s=; b=bonK2qHXb4PN5NLN7+7hu2kwmTAqvBH7uqfmweTIJwJnNg9ZnYwvQoI7PJq/SUAznN W4+93XO995oAv2SQ2vmuYno7k/6KgzzR0x49DCJaouCAlKW2EDdXZOkSSr2wQIHHcUEO aEDJ7XcRW3/xQQshiTXawNOnxKJVkBXKwZm+EEqXJIgc4PD9wf6PfaifkfaZmKynqjo1 3tWID1G7dyHS066JYbicg3St8uW+Xv4QoKt7sGV382ax049PRaLaSjts0aUlhs9JnhzV GQoM9jf/AyCe3KupduTmPKTHTzR24RNrmyo6GyRUcx8jnCjFKS9CzoO1P/K2ypgCCRJi nmoA== X-Gm-Message-State: APjAAAW2IbeJScCVqjy+fU3JIbPppR3oawO4N1jK0fMJQgaMOMHIyqAW U1B4a+tNYtAEG4auyu/cyWuTERK3/NM= X-Received: by 2002:a65:64cf:: with SMTP id t15mr114291628pgv.88.1564647691566; Thu, 01 Aug 2019 01:21:31 -0700 (PDT) Received: from localhost ([122.172.28.117]) by smtp.gmail.com with ESMTPSA id k22sm76945297pfk.157.2019.08.01.01.21.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Aug 2019 01:21:31 -0700 (PDT) From: Viresh Kumar To: stable@vger.kernel.org Cc: Viresh Kumar , Julien Thierry , linux-arm-kernel@lists.infradead.org, Catalin Marinas , Marc Zyngier , Mark Rutland , Will Deacon , Russell King , Vincent Guittot , mark.brown@arm.com, guohanjun@huawei.com Subject: [PATCH ARM32 v4.4 V2 45/47] ARM: spectre-v2: per-CPU vtables to work around big.Little systems Date: Thu, 1 Aug 2019 13:46:29 +0530 Message-Id: <06b369712f1c6cdb7358f245da8f641243750ccb.1564646727.git.viresh.kumar@linaro.org> X-Mailer: git-send-email 2.21.0.rc0.269.g1a574e7a288b In-Reply-To: References: MIME-Version: 1.0 Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Russell King Commit 383fb3ee8024d596f488d2dbaf45e572897acbdb upstream. In big.Little systems, some CPUs require the Spectre workarounds in paths such as the context switch, but other CPUs do not. In order to handle these differences, we need per-CPU vtables. We are unable to use the kernel's per-CPU variables to support this as per-CPU is not initialised at times when we need access to the vtables, so we have to use an array indexed by logical CPU number. We use an array-of-pointers to avoid having function pointers in the kernel's read/write .data section. Note: Added include of linux/slab.h in arch/arm/smp.c. Reviewed-by: Julien Thierry Signed-off-by: Russell King Signed-off-by: Viresh Kumar --- arch/arm/include/asm/proc-fns.h | 23 +++++++++++++++++++++++ arch/arm/kernel/setup.c | 5 +++++ arch/arm/kernel/smp.c | 32 ++++++++++++++++++++++++++++++++ arch/arm/mm/proc-v7-bugs.c | 17 ++--------------- 4 files changed, 62 insertions(+), 15 deletions(-) -- 2.21.0.rc0.269.g1a574e7a288b diff --git a/arch/arm/include/asm/proc-fns.h b/arch/arm/include/asm/proc-fns.h index a1a71b068edc..1bfcc3bcfc6d 100644 --- a/arch/arm/include/asm/proc-fns.h +++ b/arch/arm/include/asm/proc-fns.h @@ -104,12 +104,35 @@ extern void cpu_do_resume(void *); #else extern struct processor processor; +#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR) +#include +/* + * This can't be a per-cpu variable because we need to access it before + * per-cpu has been initialised. We have a couple of functions that are + * called in a pre-emptible context, and so can't use smp_processor_id() + * there, hence PROC_TABLE(). We insist in init_proc_vtable() that the + * function pointers for these are identical across all CPUs. + */ +extern struct processor *cpu_vtable[]; +#define PROC_VTABLE(f) cpu_vtable[smp_processor_id()]->f +#define PROC_TABLE(f) cpu_vtable[0]->f +static inline void init_proc_vtable(const struct processor *p) +{ + unsigned int cpu = smp_processor_id(); + *cpu_vtable[cpu] = *p; + WARN_ON_ONCE(cpu_vtable[cpu]->dcache_clean_area != + cpu_vtable[0]->dcache_clean_area); + WARN_ON_ONCE(cpu_vtable[cpu]->set_pte_ext != + cpu_vtable[0]->set_pte_ext); +} +#else #define PROC_VTABLE(f) processor.f #define PROC_TABLE(f) processor.f static inline void init_proc_vtable(const struct processor *p) { processor = *p; } +#endif #define cpu_proc_init PROC_VTABLE(_proc_init) #define cpu_check_bugs PROC_VTABLE(check_bugs) diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c index 8081f88bf636..f2833d7c5378 100644 --- a/arch/arm/kernel/setup.c +++ b/arch/arm/kernel/setup.c @@ -113,6 +113,11 @@ EXPORT_SYMBOL(elf_hwcap2); #ifdef MULTI_CPU struct processor processor __ro_after_init; +#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR) +struct processor *cpu_vtable[NR_CPUS] = { + [0] = &processor, +}; +#endif #endif #ifdef MULTI_TLB struct cpu_tlb_fns cpu_tlb __ro_after_init; diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c index c92abf791aed..03c11e021962 100644 --- a/arch/arm/kernel/smp.c +++ b/arch/arm/kernel/smp.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #include @@ -40,6 +41,7 @@ #include #include #include +#include #include #include #include @@ -96,6 +98,30 @@ static unsigned long get_arch_pgd(pgd_t *pgd) #endif } +#if defined(CONFIG_BIG_LITTLE) && defined(CONFIG_HARDEN_BRANCH_PREDICTOR) +static int secondary_biglittle_prepare(unsigned int cpu) +{ + if (!cpu_vtable[cpu]) + cpu_vtable[cpu] = kzalloc(sizeof(*cpu_vtable[cpu]), GFP_KERNEL); + + return cpu_vtable[cpu] ? 0 : -ENOMEM; +} + +static void secondary_biglittle_init(void) +{ + init_proc_vtable(lookup_processor(read_cpuid_id())->proc); +} +#else +static int secondary_biglittle_prepare(unsigned int cpu) +{ + return 0; +} + +static void secondary_biglittle_init(void) +{ +} +#endif + int __cpu_up(unsigned int cpu, struct task_struct *idle) { int ret; @@ -103,6 +129,10 @@ int __cpu_up(unsigned int cpu, struct task_struct *idle) if (!smp_ops.smp_boot_secondary) return -ENOSYS; + ret = secondary_biglittle_prepare(cpu); + if (ret) + return ret; + /* * We need to tell the secondary core where to find * its stack and the page tables. @@ -354,6 +384,8 @@ asmlinkage void secondary_start_kernel(void) struct mm_struct *mm = &init_mm; unsigned int cpu; + secondary_biglittle_init(); + /* * The identity mapping is uncached (strongly ordered), so * switch away from it before attempting any exclusive accesses. diff --git a/arch/arm/mm/proc-v7-bugs.c b/arch/arm/mm/proc-v7-bugs.c index 5544b82a2e7a..9a07916af8dd 100644 --- a/arch/arm/mm/proc-v7-bugs.c +++ b/arch/arm/mm/proc-v7-bugs.c @@ -52,8 +52,6 @@ static void cpu_v7_spectre_init(void) case ARM_CPU_PART_CORTEX_A17: case ARM_CPU_PART_CORTEX_A73: case ARM_CPU_PART_CORTEX_A75: - if (processor.switch_mm != cpu_v7_bpiall_switch_mm) - goto bl_error; per_cpu(harden_branch_predictor_fn, cpu) = harden_branch_predictor_bpiall; spectre_v2_method = "BPIALL"; @@ -61,8 +59,6 @@ static void cpu_v7_spectre_init(void) case ARM_CPU_PART_CORTEX_A15: case ARM_CPU_PART_BRAHMA_B15: - if (processor.switch_mm != cpu_v7_iciallu_switch_mm) - goto bl_error; per_cpu(harden_branch_predictor_fn, cpu) = harden_branch_predictor_iciallu; spectre_v2_method = "ICIALLU"; @@ -88,11 +84,9 @@ static void cpu_v7_spectre_init(void) ARM_SMCCC_ARCH_WORKAROUND_1, &res); if ((int)res.a0 != 0) break; - if (processor.switch_mm != cpu_v7_hvc_switch_mm && cpu) - goto bl_error; per_cpu(harden_branch_predictor_fn, cpu) = call_hvc_arch_workaround_1; - processor.switch_mm = cpu_v7_hvc_switch_mm; + cpu_do_switch_mm = cpu_v7_hvc_switch_mm; spectre_v2_method = "hypervisor"; break; @@ -101,11 +95,9 @@ static void cpu_v7_spectre_init(void) ARM_SMCCC_ARCH_WORKAROUND_1, &res); if ((int)res.a0 != 0) break; - if (processor.switch_mm != cpu_v7_smc_switch_mm && cpu) - goto bl_error; per_cpu(harden_branch_predictor_fn, cpu) = call_smc_arch_workaround_1; - processor.switch_mm = cpu_v7_smc_switch_mm; + cpu_do_switch_mm = cpu_v7_smc_switch_mm; spectre_v2_method = "firmware"; break; @@ -119,11 +111,6 @@ static void cpu_v7_spectre_init(void) if (spectre_v2_method) pr_info("CPU%u: Spectre v2: using %s workaround\n", smp_processor_id(), spectre_v2_method); - return; - -bl_error: - pr_err("CPU%u: Spectre v2: incorrect context switching function, system vulnerable\n", - cpu); } #else static void cpu_v7_spectre_init(void)