From patchwork Wed Feb 28 03:56:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129886 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp567478lja; Tue, 27 Feb 2018 19:57:53 -0800 (PST) X-Google-Smtp-Source: AH8x226XsIRhlnDuCHbq9vEK2QGBAnIJ48WOwJiQPt7UX2z48JzgWk6L6T+N2ssLfadVXHn7inQ+ X-Received: by 2002:a17:902:8501:: with SMTP id bj1-v6mr16900845plb.110.1519790273353; Tue, 27 Feb 2018 19:57:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790273; cv=none; d=google.com; s=arc-20160816; b=01rcmirMP/pszrmZNrpEiH7lD9lwUS10VS4jiNK2zfIE8q6jMX43MnOaQq9SejlLEm XUub00Vwsg3a3C8FAf3pIQ/NsvsWftmrxI/Iviiy86J020S1vg02qq0Fag0a6DDC1HJZ V0GqfZwKqqNKt2APDcpCznOC8JV16BIGf6wukf5y5u7drPcMfM+Ev94RKcz9DxWa67RM 4xeAdmoHbV4YEykl+YQt49Iqk+JyUqV7jIOAfojbOV1pIPcDJEBolWR8406ujiUABx8s JBNX1ta1WtaLMu2QfY7AvOB1PqJk2K8K2jd60djKXp2pWWrrqsak3XMFNem0sSFhEqFg Tptg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=JlI9DP1HlKjji9zn/7bpfmOr/uB1krsnDWBdsArXfLk=; b=X21xleGKICkey9YRq+w//tWaUi7lPcGUwDa3Z6UtYlghhgOn5gzg5V5D0gDl/GRWcn jvMihX3SjRmwSDFCrQS8lgJNk00DxN71Wu17DKjPos+lG4YSYkpy2wT9Nsk9zH1NGmgr X2veNL9+GT0erpZ8d2HYXTkGSC6sSxbhi5OTEE6R+WJBaUYS6ogUNaQFYQdsYmfZqoqu UFH8zT+SWnkCHOj6DBQCsLMdhpQWoMXWSgGUPQwLgZRd3mnoBxzP+7OuWI7JMI3USf6V t6FRtRn7UmELPV/rSYcHNylPh0vfuiCA2cYfU7+Wqjpx+tsbUvGk4uSSIC3erAZu6lzI Yycg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RQH5Gop9; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 32-v6si573381plg.555.2018.02.27.19.57.53; Tue, 27 Feb 2018 19:57:53 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RQH5Gop9; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751937AbeB1D5r (ORCPT + 28 others); Tue, 27 Feb 2018 22:57:47 -0500 Received: from mail-pl0-f66.google.com ([209.85.160.66]:46115 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751811AbeB1D5o (ORCPT ); Tue, 27 Feb 2018 22:57:44 -0500 Received: by mail-pl0-f66.google.com with SMTP id y8-v6so757754pll.13 for ; Tue, 27 Feb 2018 19:57:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JlI9DP1HlKjji9zn/7bpfmOr/uB1krsnDWBdsArXfLk=; b=RQH5Gop988mVavWcfYWliQXqF72jFL+LSV+W9ay8EmY3404fRDS+trv+19X3otPlGT iTus77JzCzC/3S4OPTrf4beE8/lW7yQJPSHh2EKyOVmP28dYgiYKPQeHR9zlOSl6O4TO N1xhBTeOrt5EmqDXYFrv4j/ht3cAz8hCX179E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JlI9DP1HlKjji9zn/7bpfmOr/uB1krsnDWBdsArXfLk=; b=e5DKAEfPpjxR48BXX3CIqigTIzX/HqH//8q4hh3HSOZmLZEDtUFmBnxxnZZG1yvsUo esNz5wPYQgXXZjtlrNLpLovQhyl8fs3XdpOLnENZyG9aI1/2UQeY4FOlme34BU6n6Xk2 gfEfroucT0ogec/mbdfwe5MkTZw7zKfHwbvDxOwuUVgSW9Y+H91T2aMjpTxJW8ofogS9 4InsSx+8b/tUY8tbhFeMhLeNOb9aQF6bVeLyLjJmGNVVkxNklXvC3SryKuXGGbQwpKd1 sw8H3NeHxwug1I6SwPkkb49fyiZ4V9XzMsdgW8aqOhHeZ6PXZiwx22AEh5WsjPG3oMk0 Pcxw== X-Gm-Message-State: APf1xPB+99VmCFRaF5SAhOx9ADg55mXpdsJxOg51n4ERmYfM/ywFg9b6 hOyWnTcAH0p5ZM42H0+2fTEFnQ== X-Received: by 2002:a17:902:7404:: with SMTP id g4-v6mr16383927pll.235.1519790263134; Tue, 27 Feb 2018 19:57:43 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.57.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:57:42 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 02/29] arm64: mm: Move ASID from TTBR0 to TTBR1 Date: Wed, 28 Feb 2018 11:56:24 +0800 Message-Id: <1519790211-16582-3-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 7655abb95386 upstream. In preparation for mapping kernelspace and userspace with different ASIDs, move the ASID to TTBR1 and update switch_mm to context-switch TTBR0 via an invalid mapping (the zero page). Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no pre_ttbr0_update_workaround in arch/arm64/mm/proc.S --- arch/arm64/include/asm/mmu_context.h | 7 +++++++ arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/include/asm/proc-fns.h | 6 ------ arch/arm64/mm/proc.S | 9 ++++++--- 4 files changed, 14 insertions(+), 9 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index a501853..b96c4799 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -50,6 +50,13 @@ static inline void cpu_set_reserved_ttbr0(void) isb(); } +static inline void cpu_switch_mm(pgd_t *pgd, struct mm_struct *mm) +{ + BUG_ON(pgd == swapper_pg_dir); + cpu_set_reserved_ttbr0(); + cpu_do_switch_mm(virt_to_phys(pgd),mm); +} + /* * TCR.T0SZ value to use when the ID map is active. Usually equals * TCR_T0SZ(VA_BITS), unless system RAM is positioned very high in diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index eb0c2bd..8df4cb6 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -272,6 +272,7 @@ #define TCR_TG1_4K (UL(2) << TCR_TG1_SHIFT) #define TCR_TG1_64K (UL(3) << TCR_TG1_SHIFT) +#define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) #define TCR_HA (UL(1) << 39) diff --git a/arch/arm64/include/asm/proc-fns.h b/arch/arm64/include/asm/proc-fns.h index 14ad6e4..16cef2e 100644 --- a/arch/arm64/include/asm/proc-fns.h +++ b/arch/arm64/include/asm/proc-fns.h @@ -35,12 +35,6 @@ extern u64 cpu_do_resume(phys_addr_t ptr, u64 idmap_ttbr); #include -#define cpu_switch_mm(pgd,mm) \ -do { \ - BUG_ON(pgd == swapper_pg_dir); \ - cpu_do_switch_mm(virt_to_phys(pgd),mm); \ -} while (0) - #endif /* __ASSEMBLY__ */ #endif /* __KERNEL__ */ #endif /* __ASM_PROCFNS_H */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 352c73b..3378f3e 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -132,9 +132,12 @@ ENDPROC(cpu_do_resume) * - pgd_phys - physical address of new TTB */ ENTRY(cpu_do_switch_mm) + mrs x2, ttbr1_el1 mmid x1, x1 // get mm->context.id - bfi x0, x1, #48, #16 // set the ASID - msr ttbr0_el1, x0 // set TTBR0 + bfi x2, x1, #48, #16 // set the ASID + msr ttbr1_el1, x2 // in TTBR1 (since TCR.A1 is set) + isb + msr ttbr0_el1, x0 // now update TTBR0 isb alternative_if ARM64_WORKAROUND_CAVIUM_27456 ic iallu @@ -222,7 +225,7 @@ ENTRY(__cpu_setup) * both user and kernel. */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ - TCR_TG_FLAGS | TCR_ASID16 | TCR_TBI0 + TCR_TG_FLAGS | TCR_ASID16 | TCR_TBI0 | TCR_A1 tcr_set_idmap_t0sz x10, x9 /* From patchwork Wed Feb 28 03:56:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129887 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp567526lja; Tue, 27 Feb 2018 19:57:57 -0800 (PST) X-Google-Smtp-Source: AG47ELtLGWYp4YespeEvuPr5XhxIIlM2YOdJ/ItcRPW39wH/A2ZC1MHdEWDloGkQfeX7yMaFSGIs X-Received: by 2002:a17:902:1c5:: with SMTP id b63-v6mr10136385plb.311.1519790277799; Tue, 27 Feb 2018 19:57:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790277; cv=none; d=google.com; s=arc-20160816; b=fSvlX7QH9uyXIesl0VI7M0/GbjAtrIMcBjQ2KXeyR70XiMLFyncs83Rj7RyxdAc56z 11f4oJDxUBYTbABOXx3v1QNJeW005pq3xm9aYl/7jgl3BYsJ+GBB9T/RJwxj+ynP68P+ D7ZoObsMAKWKSHwecHD/7Jss5joJzTR1ZKDxL3aF99ydpvytSQCKrbLDjVxX5QSfZo5G g6Bn688tzzdMlLvcNtPcT7FkKY6NpJXQY39HqOaRLuvoSX7B5xkEpKS6OkL4IKz7UG/z dVo624x7p3SYUm4k6bpGJIx9cfvkE49Avmu9Swwz4yO20WPgTh1hdLrwpo3ZgTcW3NcW lhJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=ei8K/u/Kw0usIbJStlCx/X9r89lOl79D4a+wZXmBbiM=; b=Vr0DPPVdBuxo31DlsfXQ//YlIqDC6VZ4FCEm4lXMDV7C2xDCJpnCv8ZHYzfp5wvSla xQbeQ1/2Tft15U7y+zUZVo3MYjZYOH0r5mlh5IuY0clT5FGOISzu462eGp1IyYyl07s+ dcM6gQqEOKU13x39dGhlVog4rrV2FRCDVJOapwdqOdjCYyhWxbGnsVCURNRYX3MWE39V 6Okpg3a+rVHVCc4kXEoBHdeKoDSxdBhigLmQF+sZoh58KuqJGLhqrBSjcHC/uq6T+Vfu B26U7Z2uszgMUhOAhndQPEjKW+7Ip6v4J66Wvr/woWzqkXlIGz8wbGJLDuHWHngO/JJN 5RlA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CZzvQxfG; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l7-v6si585896pls.331.2018.02.27.19.57.57; Tue, 27 Feb 2018 19:57:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CZzvQxfG; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751974AbeB1D5y (ORCPT + 28 others); Tue, 27 Feb 2018 22:57:54 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:33262 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751942AbeB1D5v (ORCPT ); Tue, 27 Feb 2018 22:57:51 -0500 Received: by mail-pg0-f67.google.com with SMTP id g12so488888pgs.0 for ; Tue, 27 Feb 2018 19:57:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ei8K/u/Kw0usIbJStlCx/X9r89lOl79D4a+wZXmBbiM=; b=CZzvQxfGCOfcVKhg6pOWxmK47Ud0F1tSf+xrA5i8+vKVVK/iB8PxIybac2G9IC3+/H s5Twa2tpfLCaY5kOJezW1ZenNaxqMW772gswWBWGf0B7ICIetliL4yPe/lfH7nbMR+JP YK6gVh2+8kyh/Ds57Us4sh6KDtkpTvDAQunmw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ei8K/u/Kw0usIbJStlCx/X9r89lOl79D4a+wZXmBbiM=; b=FxksF6Xd/xesEODLmFKIDfvZGp/yToDnclpllIv9GZWZ8wdDxIZcYgUh8iPIwDYj09 GN6FuOgiZ7+CXz88cZsI1PoX1xiGbMPIGIwFYoRKGtzWow3wJnz4XfjKFP0jIRFV/842 OxysxDYdn/27bAZH9IRgH8Wt43gs7gyk89kPXLLXq6Bjk1qONIqjlioSqnK/v5++6qEh 310EFV051yTZ7BN32A1gJ0UakJ0UXCM24//nJjbaCbafkApzy8SZ9L3MqqQ7ur+HAuYV REssWAN3W58eY+xF84+JY1g/8KGiaf1RvwRTHUl8e+Yabq1QBuNK+LNAcCm/TO84tjY2 EQiQ== X-Gm-Message-State: APf1xPD0hzpx0JGTz5YmOhcOrdRL6pDddNXuLMO/rEn7YIaj3grTy4Kw 3WRnLS4XNs7SYlQiqJ5e7xBiDQ== X-Received: by 10.101.81.204 with SMTP id i12mr13054782pgq.206.1519790270814; Tue, 27 Feb 2018 19:57:50 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.57.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:57:50 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 03/29] arm64: mm: Allocate ASIDs in pairs Date: Wed, 28 Feb 2018 11:56:25 +0800 Message-Id: <1519790211-16582-4-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 0c8ea531b774 upstream. In preparation for separate kernel/user ASIDs, allocate them in pairs for each mm_struct. The bottom bit distinguishes the two: if it is set, then the ASID will map only userspace. Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no MMCF_AARCH32 in arch/arm64/include/asm/mmu.h --- arch/arm64/include/asm/mmu.h | 2 ++ arch/arm64/mm/context.c | 25 +++++++++++++++++-------- 2 files changed, 19 insertions(+), 8 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 8d9fce0..49924e5 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -16,6 +16,8 @@ #ifndef __ASM_MMU_H #define __ASM_MMU_H +#define USER_ASID_FLAG (UL(1) << 48) + typedef struct { atomic64_t id; void *vdso; diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index efcf1f7..f00f5ee 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -39,7 +39,16 @@ static cpumask_t tlb_flush_pending; #define ASID_MASK (~GENMASK(asid_bits - 1, 0)) #define ASID_FIRST_VERSION (1UL << asid_bits) -#define NUM_USER_ASIDS ASID_FIRST_VERSION + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define NUM_USER_ASIDS (ASID_FIRST_VERSION >> 1) +#define asid2idx(asid) (((asid) & ~ASID_MASK) >> 1) +#define idx2asid(idx) (((idx) << 1) & ~ASID_MASK) +#else +#define NUM_USER_ASIDS (ASID_FIRST_VERSION) +#define asid2idx(asid) ((asid) & ~ASID_MASK) +#define idx2asid(idx) asid2idx(idx) +#endif /* Get the ASIDBits supported by the current CPU */ static u32 get_cpu_asid_bits(void) @@ -104,7 +113,7 @@ static void flush_context(unsigned int cpu) */ if (asid == 0) asid = per_cpu(reserved_asids, i); - __set_bit(asid & ~ASID_MASK, asid_map); + __set_bit(asid2idx(asid), asid_map); per_cpu(reserved_asids, i) = asid; } @@ -159,16 +168,16 @@ static u64 new_context(struct mm_struct *mm, unsigned int cpu) * We had a valid ASID in a previous life, so try to re-use * it if possible. */ - asid &= ~ASID_MASK; - if (!__test_and_set_bit(asid, asid_map)) + if (!__test_and_set_bit(asid2idx(asid), asid_map)) return newasid; } /* * Allocate a free ASID. If we can't find one, take a note of the - * currently active ASIDs and mark the TLBs as requiring flushes. - * We always count from ASID #1, as we use ASID #0 when setting a - * reserved TTBR0 for the init_mm. + * currently active ASIDs and mark the TLBs as requiring flushes. We + * always count from ASID #2 (index 1), as we use ASID #0 when setting + * a reserved TTBR0 for the init_mm and we allocate ASIDs in even/odd + * pairs. */ asid = find_next_zero_bit(asid_map, NUM_USER_ASIDS, cur_idx); if (asid != NUM_USER_ASIDS) @@ -185,7 +194,7 @@ static u64 new_context(struct mm_struct *mm, unsigned int cpu) set_asid: __set_bit(asid, asid_map); cur_idx = asid; - return asid | generation; + return idx2asid(asid) | generation; } void check_and_switch_context(struct mm_struct *mm, unsigned int cpu) From patchwork Wed Feb 28 03:56:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129888 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp567622lja; Tue, 27 Feb 2018 19:58:06 -0800 (PST) X-Google-Smtp-Source: AH8x224KhcmSCInsdrkv0EdNFMqYmk9gGj308GSvEmoud1Nci/vB9+WMguPcjywxCajpWtJsia2G X-Received: by 10.99.96.73 with SMTP id u70mr12693256pgb.199.1519790286185; Tue, 27 Feb 2018 19:58:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790286; cv=none; d=google.com; s=arc-20160816; b=VYNR6POkMlDHbVqWynALL3d3jki6JKP5LvPA6mOPmybMLIkHpxSQFwBZn2dcbCAkZY ZRb3QZXX+sBx1nxAcBx00rtf8wgHfx4AAKpJlpPye9LFCpVTIHD5CUiP9PTgFKO15w6i W3GNpdqtMqZD3Gj6tJJglxbr4XQdoXwq7Wq+TSHwYTwnVB0zqObCHe3UdqSh6klSTVUO i1LQxEmqIdDQQGzmCWyKgd0fKTgTwB+n0MK0B+PKHkPFHN7w8c2BzPtn96jxADmg8/Pd jhyodT6qmmjFBwTzatyJvXOJJKFyQ4el+/CXFkzqZwCHFjeSdgbzWuawLtaBS26j3lvz Snow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=fBaV2So5mLi7XEyJz2SzruPt01XLduxFQhFmGwjPC28=; b=I9Zi8BsOnRIzu21o9vSRKcMnY1eOX9yfQSpZfjDXmcH/hEbJJRtYwC2AydxnOC2GwS KyTi5yGQvFP3EEgL+e+N2U0vFbayWaOzNqxHm/qxK9Qaq9LJ/lTZrpUOwsezRnGME7d1 zd3Hwzqvp2GDZifJjtlkabTW1C6iccMG0X0K2MO35pmcHk18UNEYuvapDzc3vf8Qck6B gBmvjFKxStG4dBxheu26e5CbO+N3YahNhX4/L2Z7ibcsC+bpYDtTBUKT/sauZ5cRE0pw 2o/FUT3RbkzXPKpSc5E8m08Ph95jRlpiSIxtHk/8Rr2I75LvPZOxYDULhGvLbcknbugM WMCQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LtkiNWUt; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g4-v6si586704plb.114.2018.02.27.19.58.05; Tue, 27 Feb 2018 19:58:06 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LtkiNWUt; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752003AbeB1D6C (ORCPT + 28 others); Tue, 27 Feb 2018 22:58:02 -0500 Received: from mail-pl0-f66.google.com ([209.85.160.66]:36772 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751981AbeB1D56 (ORCPT ); Tue, 27 Feb 2018 22:57:58 -0500 Received: by mail-pl0-f66.google.com with SMTP id 61-v6so777082plf.3 for ; Tue, 27 Feb 2018 19:57:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=fBaV2So5mLi7XEyJz2SzruPt01XLduxFQhFmGwjPC28=; b=LtkiNWUtxuJbVzTPhxAZVglhCF0Rin+SCEqA2+LieCgm65oSSLCMJDrSm+PCdiXBcc OndWn9Rmwy+j3z33F7uP716uucFG4PN5+Bkvt/XV9V6xq1LPKIInHcIQXR1AhUqjaPll MoeXULksvI8rnTGiFyr4ogZVZ/TFW9Xnmx/Zc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=fBaV2So5mLi7XEyJz2SzruPt01XLduxFQhFmGwjPC28=; b=E38H+LgbuFW09Pnf7xRPIzlO9B+4qi9xBjyUtU0f2tD5b8pJnLGKFkZOv+zZybbGlE cI5qzW6j5KAe9F9EQWFxux8ddrt3vllNA4La3akaemPNLeHmfBNxaQoF7vIN9pU2TMPN zyzBCPZS/7vyG3r1AleOGwwZpMDsanxxINew/kfmHrI7bzjGmfbCerLpO+XM23YcCOih NbJaj4YCLYVuUOfmjZcO4EeCWeSdvCKY0yJSFRW0f9iNWW7f8Q/bY2e7aWWQiN6bsfze jSSe1VOCEsrBntMgqvPDAOyd7G45hvyCIqVzKkztmc22b+IqvwcUrpSLHDJq+9w1zMyT kzxA== X-Gm-Message-State: APf1xPAARbXRZ/gs7IUzpznNnD5yr07tNaq9C0ztS3fvesdKlpSPkz7o ZodQKZ/DcuCDjp0Y7GHARxYWTw== X-Received: by 2002:a17:902:2803:: with SMTP id e3-v6mr16411484plb.238.1519790278309; Tue, 27 Feb 2018 19:57:58 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.57.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:57:57 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 04/29] arm64: mm: Add arm64_kernel_unmapped_at_el0 helper Date: Wed, 28 Feb 2018 11:56:26 +0800 Message-Id: <1519790211-16582-5-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit fc0e1299da54 upstream. In order for code such as TLB invalidation to operate efficiently when the decision to map the kernel at EL0 is determined at runtime, this patch introduces a helper function, arm64_kernel_unmapped_at_el0, to determine whether or not the kernel is mapped whilst running in userspace. Currently, this just reports the value of CONFIG_UNMAP_KERNEL_AT_EL0, but will later be hooked up to a fake CPU capability using a static key. Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/include/asm/mmu.h | 8 ++++++++ 1 file changed, 8 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 49924e5..279e75b 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -18,6 +18,8 @@ #define USER_ASID_FLAG (UL(1) << 48) +#ifndef __ASSEMBLY__ + typedef struct { atomic64_t id; void *vdso; @@ -30,6 +32,11 @@ typedef struct { */ #define ASID(mm) ((mm)->context.id.counter & 0xffff) +static inline bool arm64_kernel_unmapped_at_el0(void) +{ + return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0); +} + extern void paging_init(void); extern void bootmem_init(void); extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt); @@ -39,4 +46,5 @@ extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, pgprot_t prot, bool allow_block_mappings); extern void *fixmap_remap_fdt(phys_addr_t dt_phys); +#endif /* !__ASSEMBLY__ */ #endif From patchwork Wed Feb 28 03:56:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129889 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp567691lja; Tue, 27 Feb 2018 19:58:13 -0800 (PST) X-Google-Smtp-Source: AH8x227w4Wpg3sHRPw7Y2etbtDWC0+Md0qx6LOog/uVwl5rojjmbDSH3xJk7rJYZctg+s16CFxnN X-Received: by 10.101.97.139 with SMTP id c11mr13101413pgv.433.1519790293019; Tue, 27 Feb 2018 19:58:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790293; cv=none; d=google.com; s=arc-20160816; b=G55pTaiOQ6AoYj0xFkw1S0V7w58oa7jBdKIpaEDN2O8GolMogbVLe3X3X2diZG6yoJ qb+dZ/d4AoxIpoPrqcbadrjuWcG2vt2I75rht0FdSa5HS6lwIBkyDFm3lUsVsLZOvuQa j8lzeg/nmXrJ8wqHxcJcNTVNFE5ulEshy7boYCu1qnm2/G3mSNisVq2VnkcNQI8+cHeA 8CxKto9C0s1sIh8RHjVcmIzSLLvz23SXCEhr3rC84XovEL6ZhDH6cmYWNsR3vDTvIs8M 9+aG1Mog67eNRnuEJOH/rZA5mYKYkeqSjTMEuxpcygMr2PtUe+Otut/J2aStV4y1k+s1 Ym7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=CnBAWrPsMy6/oPLDU3/iiTfetB2B2cU6YnCOS/tfJT8=; b=hWe+/7GQTeEMWIT0ZCfOtiyFdNrLk3uDaAtvMuFKwbdkB4LRxDub89Fl5RH1lTMzQw gadd9si3nUbu29/hOGDAdCpI004WzpR5tfMZy5UNpF/8q5j8R1nUQM52gDpz/rAnHbIp ykxuPmpWH8/rq3FRX36BsWnnvCvWGQ1UTofBglwoCzjN3C6LwpEH/ZuuRMnqVHUhO/bQ IKZSgLVEtVOkY3OdUjLPoGj9S8w0TgJpbcGRQZs+rAUH3a9xJaWulthu5Seh4eff2Ziy C4JnlD2YL5dLvAyJUec4qFLrpZ9yYrlT8hNHcdhv8DQRd1Q7OFdAR0k+sQizleYuLC5h GFvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=h7zHUAXD; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g4-v6si586704plb.114.2018.02.27.19.58.12; Tue, 27 Feb 2018 19:58:13 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=h7zHUAXD; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752035AbeB1D6J (ORCPT + 28 others); Tue, 27 Feb 2018 22:58:09 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:38830 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751814AbeB1D6G (ORCPT ); Tue, 27 Feb 2018 22:58:06 -0500 Received: by mail-pf0-f193.google.com with SMTP id d26so509733pfn.5 for ; Tue, 27 Feb 2018 19:58:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=CnBAWrPsMy6/oPLDU3/iiTfetB2B2cU6YnCOS/tfJT8=; b=h7zHUAXDuHAlWLwpykzWPz/ygRPjuK7F0BngMHbMl22nGuMAmRgw6bxlodXZZTVvWD CqVezrVF7TEwY8KnGkX5WPIsxx031bBWg53s4Ss+E3vot5fPBcFDqN+CVVUTvyPjoBMA ATI8KiP9f3Bkv0liPg9UjbYj3AIcOmANLMaq8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=CnBAWrPsMy6/oPLDU3/iiTfetB2B2cU6YnCOS/tfJT8=; b=KdSgpghzhDeLDmWPXS0OAQKpp60dhxOtyPyZXMytKJMXCCCOHGd5+UmIeiqAgEzECk 5vTAdR8uE2GB++dVbSGzhZzX2ZnKr2vvFlKpdYV/XfzplVqgWdjTRSopEqCGOe0dn5W6 /3n1A9E1d9DpcO9NW9acgQE/s18Kpr4Y7XN/1TEkpvZibblqcnk65sdibInxAow+489A 3HZufgqrqLS39MMb5Wf9mjHJrQ6vhB88Q0wNkJ9C/laxO+QG+sIY3YGpjU2JRefPCs+6 kUnqOrYHEBBNSKxjrLlEasQZT82xpJy8BZbdAmWRMlNgGMrewzeuL5MFjqSvAUzJPV0E R+ug== X-Gm-Message-State: APf1xPAtlP/YkY5sZrAk12JTnyYEfFsdqetHv3CekWtVDV6aOX8MHtck /9LltmD2bVOMfHyzAR828LHpiQ== X-Received: by 10.99.153.1 with SMTP id d1mr12934030pge.338.1519790285840; Tue, 27 Feb 2018 19:58:05 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.57.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:58:05 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 05/29] arm64: mm: Invalidate both kernel and user ASIDs when performing TLBI Date: Wed, 28 Feb 2018 11:56:27 +0800 Message-Id: <1519790211-16582-6-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 9b0de864b5bc upstream. Since an mm has both a kernel and a user ASID, we need to ensure that broadcast TLB maintenance targets both address spaces so that things like CoW continue to work with the uaccess primitives in the kernel. Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/include/asm/tlbflush.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index deab523..ad6bd8b 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -23,6 +23,7 @@ #include #include +#include /* * Raw TLBI operations. @@ -42,6 +43,11 @@ #define __tlbi(op, ...) __TLBI_N(op, ##__VA_ARGS__, 1, 0) +#define __tlbi_user(op, arg) do { \ + if (arm64_kernel_unmapped_at_el0()) \ + __tlbi(op, (arg) | USER_ASID_FLAG); \ +} while (0) + /* * TLB Management * ============== @@ -103,6 +109,7 @@ static inline void flush_tlb_mm(struct mm_struct *mm) dsb(ishst); __tlbi(aside1is, asid); + __tlbi_user(aside1is, asid); dsb(ish); } @@ -113,6 +120,7 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, dsb(ishst); __tlbi(vale1is, addr); + __tlbi_user(vale1is, addr); dsb(ish); } @@ -139,10 +147,13 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, dsb(ishst); for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12)) { - if (last_level) + if (last_level) { __tlbi(vale1is, addr); - else + __tlbi_user(vale1is, addr); + } else { __tlbi(vae1is, addr); + __tlbi_user(vae1is, addr); + } } dsb(ish); } @@ -182,6 +193,7 @@ static inline void __flush_tlb_pgtable(struct mm_struct *mm, unsigned long addr = uaddr >> 12 | (ASID(mm) << 48); __tlbi(vae1is, addr); + __tlbi_user(vae1is, addr); dsb(ish); } From patchwork Wed Feb 28 03:56:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129890 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp567778lja; Tue, 27 Feb 2018 19:58:21 -0800 (PST) X-Google-Smtp-Source: AH8x226MBiLrloViG42NJxv+G1PsOM4UH/uKozd9UA9B3zpi32Q7UHIrbj1xlLM9D9GZVloOqgcp X-Received: by 10.98.31.79 with SMTP id f76mr16375402pff.60.1519790301317; Tue, 27 Feb 2018 19:58:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790301; cv=none; d=google.com; s=arc-20160816; b=pgRqLgGXQYSKtTHgeUB/YmhAsZ0l8WydPxqR3A7g7jc38MaFXG+vzZou0t2FqxwA4r W/O2EesYaPDMileI4gPghKZil5QTDgRwQyt/K7Ldu+OpmgDBSOQp1cDWD5sQd2XWE0xf 8jqKapqQ3xNph8Crv3/MOGKAB1MYZw2i5X8RDOL1BWqOZh4mCllPq+oS7WafrwxqyFRG vM/QPAg8wi1VsAY8tSoaTDCbbg30k2GHvEzCnpVF7pCGrzaGHDAqJ208M46Booq7hTpU idN+Y6ImdFbIv8OK35i3LGw5tr/XHPIeQBJpszdoWG7OcXzsemRdBxJ64MuqE80y+Fjh sVDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=BbrhBHDEMFhCrPIdPhoFv8B5sSWqD0P7v6bm6++E2JU=; b=Tla5juHfWKGgouIlLCEa2od+mP1x/A9l4a96gBJHUqdGtLiHN9xp3ezjQg/GyMP2Jn gnPkDzS7FC/7HrqTA9XbKz+n3NJoeHW9/WzTOCKX4E9EB/hGtv+9b53rknyCL/rRRJv3 O4Npic27FVPsj5iExlx+8w1i8yug8GH5nbSLMXe1dvDyJ0RNW3AIOvzgeXzFRqT24voP wq2mM47TJXF34bRKhIjzc+gWcL5s3IcYNGCCwCpj8Nl0RSDMh77PjfTpl4LSHS3HUfjY duhIykXv3dfiEG88Bzi0YGc5uCA7e5XwMOG+lSGIIhs61KPqUlAIr/16+rxy2aAKAE0F F80g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=d6218NBb; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g4-v6si586704plb.114.2018.02.27.19.58.21; Tue, 27 Feb 2018 19:58:21 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=d6218NBb; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752074AbeB1D6R (ORCPT + 28 others); Tue, 27 Feb 2018 22:58:17 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:38662 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751814AbeB1D6P (ORCPT ); Tue, 27 Feb 2018 22:58:15 -0500 Received: by mail-pg0-f67.google.com with SMTP id l24so484433pgc.5 for ; Tue, 27 Feb 2018 19:58:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=BbrhBHDEMFhCrPIdPhoFv8B5sSWqD0P7v6bm6++E2JU=; b=d6218NBbdhhfvf2H33B6hbHFIBVfL3tx0mMoWLifIGuj/PiT9cAd2FhAYsnPE296er wvYOXson94njb9Td1TyLdwwiQtTFAFBh5dX2MCDtANK1AFZVrdQcAHYZtqoYdeapcAqk cA3kBYzF0l6ahvv1p4Ky3HeVcP57/r/+bCbQk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=BbrhBHDEMFhCrPIdPhoFv8B5sSWqD0P7v6bm6++E2JU=; b=QrFGSipsq5bFwE1Dq82cm4qOhYAByHBKlczW9BP9sDEV58rnhfQvo2Z+mTJlUJbldC D02SxWxDYiplSdjCE8NxcYlH0z3ELO4Y7Mz2Ar9tT92MYsCMmaDnZQ3O96qs7FuiJdoY FySf0j8/Qvqce5ZTlhGWozslLlJsasNpYzQoCPqLrkxl8Q/1GYprJxetX1gR6uwjWuod tpTCCCSjUcU7WKsYlQezhGiE24yCDs1xUnHsDtGXdgttsuXe9qVOnbxnlNa1B/2Iaq7J d+itvKUWWCh/B1WS8yOKVxxVODl5kWw1Ql768/PDwMVkoWTKfWkS2YCrMw9Cwfasp1Ky kkQg== X-Gm-Message-State: APf1xPD3DqVM9l0ry4upzAnZh9n1F04+opCZsixf+Jk4gILQaQBWfaqu IApPlJU0MxBh5o49kyT4bq8OE9Jq7VI= X-Received: by 10.98.237.12 with SMTP id u12mr4516684pfh.72.1519790294747; Tue, 27 Feb 2018 19:58:14 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.58.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:58:14 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Mark Rutland , James Morse , Alex Shi Subject: [PATCH 06/29] arm64: factor out entry stack manipulation Date: Wed, 28 Feb 2018 11:56:28 +0800 Message-Id: <1519790211-16582-7-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Mark Rutland commit b11e5759bfac upstream. In subsequent patches, we will detect stack overflow in our exception entry code, by verifying the SP after it has been decremented to make space for the exception regs. This verification code is small, and we can minimize its impact by placing it directly in the vectors. To avoid redundant modification of the SP, we also need to move the initial decrement of the SP into the vectors. As a preparatory step, this patch introduces kernel_ventry, which performs this decrement, and updates the entry code accordingly. Subsequent patches will fold SP verification into kernel_ventry. There should be no functional change as a result of this patch. Signed-off-by: Ard Biesheuvel [Mark: turn into prep patch, expand commit msg] Signed-off-by: Mark Rutland Reviewed-by: Will Deacon Tested-by: Laura Abbott Cc: Catalin Marinas Cc: James Morse Signed-off-by: Alex Shi --- arch/arm64/kernel/entry.S | 47 ++++++++++++++++++++++++++--------------------- 1 file changed, 26 insertions(+), 21 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index b4c7db4..f5aa8f0 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -68,8 +68,13 @@ #define BAD_FIQ 2 #define BAD_ERROR 3 - .macro kernel_entry, el, regsize = 64 + .macro kernel_ventry label + .align 7 sub sp, sp, #S_FRAME_SIZE + b \label + .endm + + .macro kernel_entry, el, regsize = 64 .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 .endif @@ -257,31 +262,31 @@ tsk .req x28 // current thread_info .align 11 ENTRY(vectors) - ventry el1_sync_invalid // Synchronous EL1t - ventry el1_irq_invalid // IRQ EL1t - ventry el1_fiq_invalid // FIQ EL1t - ventry el1_error_invalid // Error EL1t + kernel_ventry el1_sync_invalid // Synchronous EL1t + kernel_ventry el1_irq_invalid // IRQ EL1t + kernel_ventry el1_fiq_invalid // FIQ EL1t + kernel_ventry el1_error_invalid // Error EL1t - ventry el1_sync // Synchronous EL1h - ventry el1_irq // IRQ EL1h - ventry el1_fiq_invalid // FIQ EL1h - ventry el1_error_invalid // Error EL1h + kernel_ventry el1_sync // Synchronous EL1h + kernel_ventry el1_irq // IRQ EL1h + kernel_ventry el1_fiq_invalid // FIQ EL1h + kernel_ventry el1_error_invalid // Error EL1h - ventry el0_sync // Synchronous 64-bit EL0 - ventry el0_irq // IRQ 64-bit EL0 - ventry el0_fiq_invalid // FIQ 64-bit EL0 - ventry el0_error_invalid // Error 64-bit EL0 + kernel_ventry el0_sync // Synchronous 64-bit EL0 + kernel_ventry el0_irq // IRQ 64-bit EL0 + kernel_ventry el0_fiq_invalid // FIQ 64-bit EL0 + kernel_ventry el0_error_invalid // Error 64-bit EL0 #ifdef CONFIG_COMPAT - ventry el0_sync_compat // Synchronous 32-bit EL0 - ventry el0_irq_compat // IRQ 32-bit EL0 - ventry el0_fiq_invalid_compat // FIQ 32-bit EL0 - ventry el0_error_invalid_compat // Error 32-bit EL0 + kernel_ventry el0_sync_compat // Synchronous 32-bit EL0 + kernel_ventry el0_irq_compat // IRQ 32-bit EL0 + kernel_ventry el0_fiq_invalid_compat // FIQ 32-bit EL0 + kernel_ventry el0_error_invalid_compat // Error 32-bit EL0 #else - ventry el0_sync_invalid // Synchronous 32-bit EL0 - ventry el0_irq_invalid // IRQ 32-bit EL0 - ventry el0_fiq_invalid // FIQ 32-bit EL0 - ventry el0_error_invalid // Error 32-bit EL0 + kernel_ventry el0_sync_invalid // Synchronous 32-bit EL0 + kernel_ventry el0_irq_invalid // IRQ 32-bit EL0 + kernel_ventry el0_fiq_invalid // FIQ 32-bit EL0 + kernel_ventry el0_error_invalid // Error 32-bit EL0 #endif END(vectors) From patchwork Wed Feb 28 03:56:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129891 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp567860lja; Tue, 27 Feb 2018 19:58:30 -0800 (PST) X-Google-Smtp-Source: AH8x225RwmGQ3B2tBNXE6ubwSQiNeNXOKJErs2EDrrdaua6ZrtOtClPdDiqrUwRh+x7gSoWZAYFV X-Received: by 2002:a17:902:7b92:: with SMTP id w18-v6mr16092448pll.159.1519790309970; Tue, 27 Feb 2018 19:58:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790309; cv=none; d=google.com; s=arc-20160816; b=s4Kq3YfaDJJMMcyNW/5AIQMc2Mm/d9AhZdR/W/PvV+9hn6H7mvfkT9jwVLQ/me5+ww tHjmZwGoHGTZtaRHuCz0fiH3zsJr11DMuJxKN3nQCBiyXE0h3I8NoFzyk0uZmu/kKr/E n1CyA+2gNh9lc3ipyXI7UxKg3DU3YpwHa72QE1T9kj8Ttj/k9ofIgZogEQiVtTn/JOxV fG24mGI1A2oUl2LfGk8s4U5IwI2FklVpbef2UfGxiypFpSwXONzTlKNDQgU7OD2x4cvf dx0ugAKEhJJATO5iw5qKbNrua2AW1AFOMQAMzYztuAEXxRw6uW/TymnyT7zT5xKo8Z9w 3yUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=A0pfV/DwY+7OHAPDaT2NCm9JaxY5uj5pHP+9/1OoPcU=; b=lG3P5fk6jVZtt7RmDZPblJ2pFIwaZ5JiST2XUDwRn0xhUSN1HciuFQ+dWJuyxooYKX 4/KmCBPA8zqpue7AZLNGRJ1T9KFW+f64TDYh+RDZ52gtu2KZXNDYaC67iaEWpjV8rMrf Q2GkGXIa1pmXpDDKQAX0p0kbV9ipGvYT0Mf0ht+UchKwDiXXMYAoU74oNuvx8/Pa1/N7 vGpJc9RlVtSxrflRBKLLG2BYvc/AdqnJb95qAoG4gp2TagQ5llB6LVv0doqWAq3y8p2H iZIPQREak7oQoVZcoDxilcjR9AkS8c41o414CFqKFPtS01zPcV5fD2X5yZ86PyJHEuYa 39Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LtDf4XA4; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g4-v6si586704plb.114.2018.02.27.19.58.29; Tue, 27 Feb 2018 19:58:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LtDf4XA4; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752105AbeB1D61 (ORCPT + 28 others); Tue, 27 Feb 2018 22:58:27 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:39471 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752077AbeB1D6Z (ORCPT ); Tue, 27 Feb 2018 22:58:25 -0500 Received: by mail-pl0-f68.google.com with SMTP id s13-v6so772311plq.6 for ; Tue, 27 Feb 2018 19:58:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=A0pfV/DwY+7OHAPDaT2NCm9JaxY5uj5pHP+9/1OoPcU=; b=LtDf4XA4SNXxq8oIVHznFIsTr/jEO1OaospEOmJ72fplp7yhUm9pV0RWK7rhHTBHcJ 6XhPlXc+dIwqGt3iWTUSOu7cVzqub56pk7/HfDrwFKJ0O7vEOxEp4dRMceCHzGlxwj3F 1gRLcmfnz6IPqO8oJhq//YB8+Oq+dZIvpCBzE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=A0pfV/DwY+7OHAPDaT2NCm9JaxY5uj5pHP+9/1OoPcU=; b=BicL1L1U9rS9vqYKdc5Ctn9Xd/TOV65rsHOgyffUozDHM/DAOjHt42wEcgj6H2yIIS Bs8gKmlsMH9CqxuLsMdAklt01RIk+ySdI3XTu3Rb1vwUuZsDOuaUFWzEZk+xZGYnVhdu UG7ColZ9mq9Cl97FPecBQh1Y3+emW92GRpEmz0PvoLhmH67daPtQltPRkSiECdc/pnay cmOAcV+RxLLJM24Bs6BpyQ4bNGV4Tc/n/H3URY1ouYBX5uGLKPhKrLbTkkzuqvCxWUie ELS4IBMaoyn+fAuGDzxkbAwgKqkOTikMKaGelxHwxdO8x5hAP//xhEvaAPySXlzeJgOm mnAw== X-Gm-Message-State: APf1xPB3TJHsWhpla03jLmiAzmwcvob2TYH2WpkHiVicyeEKwibyD/S9 mekBg4/RiK2MvFlCLR9euZs3qQ== X-Received: by 2002:a17:902:6b4c:: with SMTP id g12-v6mr16279012plt.363.1519790304486; Tue, 27 Feb 2018 19:58:24 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.58.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:58:23 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Xie XiuQi , Wang Xiongfeng , James Morse , Alex Shi Subject: [PATCH 07/29] arm64: entry.S: move SError handling into a C function for future expansion Date: Wed, 28 Feb 2018 11:56:29 +0800 Message-Id: <1519790211-16582-8-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Xie XiuQi commit a92d4d1454ab upstream. Today SError is taken using the inv_entry macro that ends up in bad_mode. SError can be used by the RAS Extensions to notify either the OS or firmware of CPU problems, some of which may have been corrected. To allow this handling to be added, add a do_serror() C function that just panic()s. Add the entry.S boiler plate to save/restore the CPU registers and unmask debug exceptions. Future patches may change do_serror() to return if the SError Interrupt was notification of a corrected error. Signed-off-by: Xie XiuQi Signed-off-by: Wang Xiongfeng [Split out of a bigger patch, added compat path, renamed, enabled debug exceptions] Signed-off-by: James Morse Reviewed-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no vmap_stack in arch/arm64/kernel/traps.c using old enable_dbg_and_irq instead of enable_daif in arch/arm64/kernel/entry.S --- arch/arm64/kernel/entry.S | 36 +++++++++++++++++++++++++++++------- arch/arm64/kernel/traps.c | 14 ++++++++++++++ 2 files changed, 43 insertions(+), 7 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index f5aa8f0..60b202a 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -270,18 +270,18 @@ ENTRY(vectors) kernel_ventry el1_sync // Synchronous EL1h kernel_ventry el1_irq // IRQ EL1h kernel_ventry el1_fiq_invalid // FIQ EL1h - kernel_ventry el1_error_invalid // Error EL1h + kernel_ventry el1_error // Error EL1h kernel_ventry el0_sync // Synchronous 64-bit EL0 kernel_ventry el0_irq // IRQ 64-bit EL0 kernel_ventry el0_fiq_invalid // FIQ 64-bit EL0 - kernel_ventry el0_error_invalid // Error 64-bit EL0 + kernel_ventry el0_error // Error 64-bit EL0 #ifdef CONFIG_COMPAT kernel_ventry el0_sync_compat // Synchronous 32-bit EL0 kernel_ventry el0_irq_compat // IRQ 32-bit EL0 kernel_ventry el0_fiq_invalid_compat // FIQ 32-bit EL0 - kernel_ventry el0_error_invalid_compat // Error 32-bit EL0 + kernel_ventry el0_error_compat // Error 32-bit EL0 #else kernel_ventry el0_sync_invalid // Synchronous 32-bit EL0 kernel_ventry el0_irq_invalid // IRQ 32-bit EL0 @@ -321,10 +321,6 @@ ENDPROC(el0_error_invalid) el0_fiq_invalid_compat: inv_entry 0, BAD_FIQ, 32 ENDPROC(el0_fiq_invalid_compat) - -el0_error_invalid_compat: - inv_entry 0, BAD_ERROR, 32 -ENDPROC(el0_error_invalid_compat) #endif el1_sync_invalid: @@ -532,6 +528,10 @@ el0_svc_compat: el0_irq_compat: kernel_entry 0, 32 b el0_irq_naked + +el0_error_compat: + kernel_entry 0, 32 + b el0_error_naked #endif el0_da: @@ -653,6 +653,28 @@ el0_irq_naked: b ret_to_user ENDPROC(el0_irq) +el1_error: + kernel_entry 1 + mrs x1, esr_el1 + enable_dbg + mov x0, sp + bl do_serror + kernel_exit 1 +ENDPROC(el1_error) + +el0_error: + kernel_entry 0 +el0_error_naked: + mrs x1, esr_el1 + enable_dbg + mov x0, sp + bl do_serror + enable_dbg_and_irq + ct_user_exit + b ret_to_user +ENDPROC(el0_error) + + /* * Register switch for AArch64. The callee-saved registers need to be saved * and restored. On entry: diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index c743d1f..2ef7e33 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -637,6 +637,20 @@ asmlinkage void bad_el0_sync(struct pt_regs *regs, int reason, unsigned int esr) force_sig_info(info.si_signo, &info, current); } + +asmlinkage void do_serror(struct pt_regs *regs, unsigned int esr) +{ + nmi_enter(); + + console_verbose(); + + pr_crit("SError Interrupt on CPU%d, code 0x%08x -- %s\n", + smp_processor_id(), esr, esr_get_class_string(esr)); + __show_regs(regs); + + panic("Asynchronous SError Interrupt"); +} + void __pte_error(const char *file, int line, unsigned long val) { pr_err("%s:%d: bad pte %016lx.\n", file, line, val); From patchwork Wed Feb 28 03:56:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129892 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp567945lja; Tue, 27 Feb 2018 19:58:38 -0800 (PST) X-Google-Smtp-Source: AH8x225cmXoOwFqA7p+KMXQe71gDzQEW11jVu56zrerKp2hRFvjFH2yBMSdyTvKx/fv02gD3bRE7 X-Received: by 2002:a17:902:225:: with SMTP id 34-v6mr16566998plc.415.1519790318126; Tue, 27 Feb 2018 19:58:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790318; cv=none; d=google.com; s=arc-20160816; b=NdKitEeJIcE78JN5osLstIoYA7ZmmHjz7hzcazFEZ9M5Cz7yxDBOSZpnlMFOctBgCG RG7zFxOmamLqiWULMOYFmGKrRbpl0PNTmUjLIgAjf7sYuOO1dEFYv9rQDfbliYxvxlc/ kMpGkTnrw+nyEAuB8T37EejNWHPijUavd0lQEIWJLgG9S+S4mdAQVua7sy6sO5rr+OeP vSzr77dfoAegKSLMOABF073ACBHnpWB2uy+jt0xHxoZ89W1BgW4ldH/aZcykRiBn1psg GAZ/rWbPbvXXxvFU59ZLAlswlQ+vfuYwBp92auCxpGONiRYsVCswWo3TKqyLzMKdAQGe 6YMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=9AGPVua7FerW9MHbnW3GGajj8cb4hN8hkiOyT6YzHrQ=; b=HEooCQg8R4jHlp7AlfAJKZPb9L9VJiuGMzX5EtwkMxDjH1etL55iukRr/v2pSV8CG/ ZY+yxr9Kbie7oOHykvdWS86hF0bxjNVo69eofrkUpt36eGqH+eu3p016yxYDaBRGVfqP NCtEgR8wdrNEFA08mcHGrgmC6sYAzbq1+0PXnqcnjlQUfb6wHKQ7/w96taqWSQByc3LC BFpp2+5T38FEjrWDkQL2w8Hyn3kx611TJ2IWZLCQtTmZ9jEgHmni+KbHrM7F4U5aB+sW HVtRcd7otRbxcnKwtxzYRgsC2tq7FRrHnOB2AxajCKgG1gFZfE9rFq8eoCShnlsUTnOW PtdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DgNjoIR8; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g4-v6si586704plb.114.2018.02.27.19.58.37; Tue, 27 Feb 2018 19:58:38 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DgNjoIR8; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752130AbeB1D6f (ORCPT + 28 others); Tue, 27 Feb 2018 22:58:35 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:40640 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752110AbeB1D6c (ORCPT ); Tue, 27 Feb 2018 22:58:32 -0500 Received: by mail-pf0-f196.google.com with SMTP id m5so506915pff.7 for ; Tue, 27 Feb 2018 19:58:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=9AGPVua7FerW9MHbnW3GGajj8cb4hN8hkiOyT6YzHrQ=; b=DgNjoIR8+juvDroKwmeklTID+hCjbk6wTFRxycGBE9p45Ac7l+uBgbtXIfuSHDc1We 6yGWjLMdwBz1sl6OGQM2aQZJ39/4CHIDV1icB9uqM6B4coRgslHC6PYXZ+VsfC+24oft YNUEW8YDyp0ETa8LFoHka3tswCezYg5oHE/ks= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=9AGPVua7FerW9MHbnW3GGajj8cb4hN8hkiOyT6YzHrQ=; b=MZRwoqcwxAB7OJyLmLNTUoCKyMAU7wgwapPp6IaZPfG4jAJ1NlcqdADejABeKCxlYN MHD4P15aOuFOWQ7l7ITC2MSbKlcfzgegN4YgMNxMgXU9KaubuEEy1pFOxuqDtb8MvY5p LGS+jreX4cmqUtS/tBNHW2v6h6z/KeD6HjFe0RzHjUbLzGaU+tZx95S2joTukCGTdSFf QSg0ZsPgXpwj9DXsUSc2yOIwOqVuvd8QLR8TuL8AJlFc3CbuQ+F2PL0nWdK/6X95jq3n rRtDbgTHq/0CiDRM5mekcwpwxopt8ivWwZ2eDKnYXVH0iinS32pQPOVew75prm+caK4A dgVg== X-Gm-Message-State: APf1xPDmNO9VGBDQfZDhFgB5k8XohfVVLGOZ4tnNYiNQQqO69UP46UWs Tzwg0XWoS7AQUDbORTKxjMIOaA== X-Received: by 10.101.81.204 with SMTP id i12mr13056152pgq.206.1519790312181; Tue, 27 Feb 2018 19:58:32 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.58.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:58:31 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: AKASHI Takahiro , Jessica Yu , Alex Shi Subject: [PATCH 08/29] module: extend 'rodata=off' boot cmdline parameter to module mappings Date: Wed, 28 Feb 2018 11:56:30 +0800 Message-Id: <1519790211-16582-9-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: AKASHI Takahiro commit 39290b389ea upstream. The current "rodata=off" parameter disables read-only kernel mappings under CONFIG_DEBUG_RODATA: commit d2aa1acad22f ("mm/init: Add 'rodata=off' boot cmdline parameter to disable read-only kernel mappings") This patch is a logical extension to module mappings ie. read-only mappings at module loading can be disabled even if CONFIG_DEBUG_SET_MODULE_RONX (mainly for debug use). Please note, however, that it only affects RO/RW permissions, keeping NX set. This is the first step to make CONFIG_DEBUG_SET_MODULE_RONX mandatory (always-on) in the future as CONFIG_DEBUG_RODATA on x86 and arm64. Suggested-by: and Acked-by: Mark Rutland Signed-off-by: AKASHI Takahiro Reviewed-by: Kees Cook Acked-by: Rusty Russell Link: http://lkml.kernel.org/r/20161114061505.15238-1-takahiro.akashi@linaro.org Signed-off-by: Jessica Yu Signed-off-by: Alex Shi Conflicts: keeping kaiser.h in init/main.c --- include/linux/init.h | 3 +++ init/main.c | 7 +++++-- kernel/module.c | 20 +++++++++++++++++--- 3 files changed, 25 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/include/linux/init.h b/include/linux/init.h index 8e346d1..4dfe300 100644 --- a/include/linux/init.h +++ b/include/linux/init.h @@ -133,6 +133,9 @@ void prepare_namespace(void); void __init load_default_modules(void); int __init init_rootfs(void); +#if defined(CONFIG_DEBUG_RODATA) || defined(CONFIG_DEBUG_SET_MODULE_RONX) +extern bool rodata_enabled; +#endif #ifdef CONFIG_DEBUG_RODATA void mark_rodata_ro(void); #endif diff --git a/init/main.c b/init/main.c index 99f0265..f22957a 100644 --- a/init/main.c +++ b/init/main.c @@ -81,6 +81,7 @@ #include #include #include +#include #include #include @@ -914,14 +915,16 @@ static int try_to_run_init_process(const char *init_filename) static noinline void __init kernel_init_freeable(void); -#ifdef CONFIG_DEBUG_RODATA -static bool rodata_enabled = true; +#if defined(CONFIG_DEBUG_RODATA) || defined(CONFIG_SET_MODULE_RONX) +bool rodata_enabled __ro_after_init = true; static int __init set_debug_rodata(char *str) { return strtobool(str, &rodata_enabled); } __setup("rodata=", set_debug_rodata); +#endif +#ifdef CONFIG_DEBUG_RODATA static void mark_readonly(void) { if (rodata_enabled) diff --git a/kernel/module.c b/kernel/module.c index 07bfb99..0651f2d 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -1911,6 +1911,9 @@ static void frob_writable_data(const struct module_layout *layout, /* livepatching wants to disable read-only so it can frob module. */ void module_disable_ro(const struct module *mod) { + if (!rodata_enabled) + return; + frob_text(&mod->core_layout, set_memory_rw); frob_rodata(&mod->core_layout, set_memory_rw); frob_ro_after_init(&mod->core_layout, set_memory_rw); @@ -1920,6 +1923,9 @@ void module_disable_ro(const struct module *mod) void module_enable_ro(const struct module *mod, bool after_init) { + if (!rodata_enabled) + return; + frob_text(&mod->core_layout, set_memory_ro); frob_rodata(&mod->core_layout, set_memory_ro); frob_text(&mod->init_layout, set_memory_ro); @@ -1952,6 +1958,9 @@ void set_all_modules_text_rw(void) { struct module *mod; + if (!rodata_enabled) + return; + mutex_lock(&module_mutex); list_for_each_entry_rcu(mod, &modules, list) { if (mod->state == MODULE_STATE_UNFORMED) @@ -1968,6 +1977,9 @@ void set_all_modules_text_ro(void) { struct module *mod; + if (!rodata_enabled) + return; + mutex_lock(&module_mutex); list_for_each_entry_rcu(mod, &modules, list) { if (mod->state == MODULE_STATE_UNFORMED) @@ -1981,10 +1993,12 @@ void set_all_modules_text_ro(void) static void disable_ro_nx(const struct module_layout *layout) { - frob_text(layout, set_memory_rw); - frob_rodata(layout, set_memory_rw); + if (rodata_enabled) { + frob_text(layout, set_memory_rw); + frob_rodata(layout, set_memory_rw); + frob_ro_after_init(layout, set_memory_rw); + } frob_rodata(layout, set_memory_x); - frob_ro_after_init(layout, set_memory_rw); frob_ro_after_init(layout, set_memory_x); frob_writable_data(layout, set_memory_x); } From patchwork Wed Feb 28 03:56:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129893 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568029lja; Tue, 27 Feb 2018 19:58:46 -0800 (PST) X-Google-Smtp-Source: AG47ELvGTsda5dIS214UA7y4Dh+vfEBc2o16FL27k9j+6CtFY5kSBklCYqSrxemMV77lvWaP9O4I X-Received: by 2002:a17:902:59d3:: with SMTP id d19-v6mr12154596plj.356.1519790326764; Tue, 27 Feb 2018 19:58:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790326; cv=none; d=google.com; s=arc-20160816; b=HuiNbeDCrN6Xo359aFMJJxIH/QOPBjE65GAm1AGc2zYRwIInpCxAGKCNbMnvn14zek YRLp5HUOPKVssN7mozJvSbtYpUnqI+UrkzRhMmVuRMq2+YPG9V15fT2SKI8s5WzvNnyF 0Gp2h5EnDAmPrOAznQbzW3q4m2OHVRTlAeZXt6w5SFyyaNjR9+9goZVKQlr62jc4XClV /y/s38I5IM6M/T8zlPrdyV9n9hwrXJBuTOZ2vm2OgObCGgfL3ApisrL3WmR9iN8ZwDpk FB4GKkm6rdFMHj/34bM+wTEZw9rsA2g0/o27nZOuCceJnVeuWEzwgYPw3wi3lN67LfJN FH9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=xF6X5uN03sAA3yMb21/xX2XvZZVdgl9P5oiG1iJTS6M=; b=N9Myj/DbnWP7c3HkIAUe2ptqUd5QjLRrRlwJThqOEw28+f2vTTkpsC1i7icANO7bJS vehJeLVHBLTzOsJwzG5OqPyBvVojfHRLcAc9neTNcFQIHehVwvZDHvuOgKMKaKHY1Uyn RfP0jsB6lQwWwlABdonqDroRzIozwJaklizCwfIwKBCU36QpztcdVScQq8tBWPoWLhrv V00No4ChyRHrsmbLfid/NIAaXxW6blVADaSLuru/7IZk08zZuwzg94RxgZa+/Oo7EV9F 1M4O4fScmvEllY9mZXqfq+rrBnsWXBV9chj0B6+tK+zxzwG05bOUllEtDdGHGWOP2uTo pZ1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=B3gaaLmT; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e13si457384pgt.481.2018.02.27.19.58.46; Tue, 27 Feb 2018 19:58:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=B3gaaLmT; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752155AbeB1D6n (ORCPT + 28 others); Tue, 27 Feb 2018 22:58:43 -0500 Received: from mail-pg0-f65.google.com ([74.125.83.65]:41934 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752110AbeB1D6k (ORCPT ); Tue, 27 Feb 2018 22:58:40 -0500 Received: by mail-pg0-f65.google.com with SMTP id q27so478384pgn.8 for ; Tue, 27 Feb 2018 19:58:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xF6X5uN03sAA3yMb21/xX2XvZZVdgl9P5oiG1iJTS6M=; b=B3gaaLmTOgRlE44IL/W74Kw5rAhKmmpE9ZtR9//Rhp4GsjAIipRjhpqXeE1wU8MAFK 9hESja0KDXfvF1JD1yH6RWfpuf4WfNApE6TF6Tfr/qwHwIll/PkEZaaG2F856IyX57Oh JcBPwX79nMfSacISM/EiC73313tZxSyfyq6lU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xF6X5uN03sAA3yMb21/xX2XvZZVdgl9P5oiG1iJTS6M=; b=OKuRcMQl0bIzZncjKcQdWWoGwuq/BiZm+5K2BX5JLxZz88KI0FPCy75KBZBIur6uip uhcV0BIJryWov38Noz5ic0xHCmbQbqzl0xXsTcCVt4IiQn3p8vQgKnGtd8Vl7dYbV1PJ UF7EandX1ZKoPbx/4t+mq38E0i4/d/TmZMIJRB3EDcqQc4iSQp8L+Qu5JMT4lSFU8bvx 3dxxLQ9F//i2awsuTfDYNBf2/i5I5NyHhB/cK2iS/S6OuIiQ5g2eW3h0UMnTeNmpcRLk UVR7qD496ZMinxN3e89e+gATn81VmpnW1MmecKHfR//6V6+LmXxEtxokeZZkI/D5jrsc K2Sg== X-Gm-Message-State: APf1xPDjFQgmfoXTCLYvtspNayU4TXkK1vaDD5Ws/VH2I7wDd6L8Xykf 836QThVFb6PnHT3E3WRiCcIPnQ== X-Received: by 10.99.120.138 with SMTP id t132mr12978589pgc.54.1519790319443; Tue, 27 Feb 2018 19:58:39 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.58.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:58:38 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 09/29] arm64: entry: Add exception trampoline page for exceptions from EL0 Date: Wed, 28 Feb 2018 11:56:31 +0800 Message-Id: <1519790211-16582-10-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit c7b9adaf85f8 upstream. To allow unmapping of the kernel whilst running at EL0, we need to point the exception vectors at an entry trampoline that can map/unmap the kernel on entry/exit respectively. This patch adds the trampoline page, although it is not yet plugged into the vector table and is therefore unused. Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: add asm/mmu.h in entry.S for ASID marco add kernel-pgtable.h in entry.S for SWAPPER_DIR_SIZE and RESERVED_TTBR0_SIZE no SW PAN in vmlinux.lds.S --- arch/arm64/include/asm/kernel-pgtable.h | 2 + arch/arm64/kernel/entry.S | 86 +++++++++++++++++++++++++++++++++ arch/arm64/kernel/vmlinux.lds.S | 17 +++++++ 3 files changed, 105 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index e4ddac9..135e829 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -54,6 +54,8 @@ #define SWAPPER_DIR_SIZE (SWAPPER_PGTABLE_LEVELS * PAGE_SIZE) #define IDMAP_DIR_SIZE (IDMAP_PGTABLE_LEVELS * PAGE_SIZE) +#define RESERVED_TTBR0_SIZE (0) /*no CONFIG_ARM64_SW_TTBR0_PAN introduced */ + /* Initial memory map size */ #if ARM64_SWAPPER_USES_SECTION_MAPS #define SWAPPER_BLOCK_SHIFT SECTION_SHIFT diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 60b202a..f0c6b37 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -29,9 +29,11 @@ #include #include #include +#include #include #include #include +#include /* * Context tracking subsystem. Used to instrument transitions @@ -828,6 +830,90 @@ __ni_sys_trace: .popsection // .entry.text +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +/* + * Exception vectors trampoline. + */ + .pushsection ".entry.tramp.text", "ax" + + .macro tramp_map_kernel, tmp + mrs \tmp, ttbr1_el1 + sub \tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) + bic \tmp, \tmp, #USER_ASID_FLAG + msr ttbr1_el1, \tmp + .endm + + .macro tramp_unmap_kernel, tmp + mrs \tmp, ttbr1_el1 + add \tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) + orr \tmp, \tmp, #USER_ASID_FLAG + msr ttbr1_el1, \tmp + /* + * We avoid running the post_ttbr_update_workaround here because the + * user and kernel ASIDs don't have conflicting mappings, so any + * "blessing" as described in: + * + * http://lkml.kernel.org/r/56BB848A.6060603@caviumnetworks.com + * + * will not hurt correctness. Whilst this may partially defeat the + * point of using split ASIDs in the first place, it avoids + * the hit of invalidating the entire I-cache on every return to + * userspace. + */ + .endm + + .macro tramp_ventry, regsize = 64 + .align 7 +1: + .if \regsize == 64 + msr tpidrro_el0, x30 // Restored in kernel_ventry + .endif + tramp_map_kernel x30 + ldr x30, =vectors + prfm plil1strm, [x30, #(1b - tramp_vectors)] + msr vbar_el1, x30 + add x30, x30, #(1b - tramp_vectors) + isb + br x30 + .endm + + .macro tramp_exit, regsize = 64 + adr x30, tramp_vectors + msr vbar_el1, x30 + tramp_unmap_kernel x30 + .if \regsize == 64 + mrs x30, far_el1 + .endif + eret + .endm + + .align 11 +ENTRY(tramp_vectors) + .space 0x400 + + tramp_ventry + tramp_ventry + tramp_ventry + tramp_ventry + + tramp_ventry 32 + tramp_ventry 32 + tramp_ventry 32 + tramp_ventry 32 +END(tramp_vectors) + +ENTRY(tramp_exit_native) + tramp_exit +END(tramp_exit_native) + +ENTRY(tramp_exit_compat) + tramp_exit 32 +END(tramp_exit_compat) + + .ltorg + .popsection // .entry.tramp.text +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + /* * Special system call wrappers. */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 1105aab..466a43a 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -56,6 +56,17 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define TRAMP_TEXT \ + . = ALIGN(PAGE_SIZE); \ + VMLINUX_SYMBOL(__entry_tramp_text_start) = .; \ + *(.entry.tramp.text) \ + . = ALIGN(PAGE_SIZE); \ + VMLINUX_SYMBOL(__entry_tramp_text_end) = .; +#else +#define TRAMP_TEXT +#endif + /* * The size of the PE/COFF section that covers the kernel image, which * runs from stext to _edata, must be a round multiple of the PE/COFF @@ -128,6 +139,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + TRAMP_TEXT *(.fixup) *(.gnu.warning) . = ALIGN(16); @@ -216,6 +228,11 @@ SECTIONS swapper_pg_dir = .; . += SWAPPER_DIR_SIZE; +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + tramp_pg_dir = .; + . += PAGE_SIZE; +#endif + _end = .; STABS_DEBUG From patchwork Wed Feb 28 03:56:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129894 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568110lja; Tue, 27 Feb 2018 19:58:57 -0800 (PST) X-Google-Smtp-Source: AH8x226kD6BUICA0Z6qzFnZ7Uz+ZPoHwgZBGRjtEzIZyLiaBq+bG3xPDLcqb2BCt4W18NFq4P4VC X-Received: by 10.98.19.146 with SMTP id 18mr16418681pft.3.1519790337457; Tue, 27 Feb 2018 19:58:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790337; cv=none; d=google.com; s=arc-20160816; b=QLhAwGI8uboV7xiXvvNm8f1lCz5/VzxgWDRv9ZcW0pVRKkZqmO+Ul1Fjjed0Phc50y Y1t3lawQGGtEYobU8XEbj+aCjanXoPfhLwVweq8fgOJGTyf/mFLgeDkFlkJiXs6PaH9G 20BMk41OHlWL798LeUF/xZhYcg0iVQKF6qJ5N8X7kG+mlaGtDIW/gqfJlPh3Gr8EgqnH QJAlXotLTqPoORteetgPWsKd5hUiQIgX69IwDB/zUCnuQibW3QQ9bo4KohwL38xv9+Mu 1Qs96u5OlbjTQvCplu5Ccjb8mhnVoXVP1nAd4jl+09YsItwrM5/OSoT1Cwpl2RChuJHY GT/g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=Dc0kkeu44UfOi1EiXBfePyqu3iL2CZsHYUa6WCie3uU=; b=XUwJHHPW+pinwXJd+SGYEKxahLzZLDDYqhoR08QsJMaFgO0jJSDxMIC88XglTt3PS4 K0DiInG1/fMPA/HRowQfg3CJ33IC/pzbjByzQPV3j/RvAMlW5q76S75f+OTY7NxIl+3+ 3V80OQkhR4tQXOrk5YP2DupB5ienyBWPpN2ieR4mE83447RMlgHcuMBjQmmEN74vp8HM PHYHagLer8T+/G61KfQYbm+q8ID9FWlVSZ6U3lQjuotyonDVY11cVS7UY2bNFZOLModO 09NBgwThXpUHSP7x4CMUXhZ0+LxvpLBJ09PpFkXpRGZEmYLLqt9c4q4SzoRWytVgH9h0 aziQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=I2x2yCrd; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e13si457384pgt.481.2018.02.27.19.58.57; Tue, 27 Feb 2018 19:58:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=I2x2yCrd; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752175AbeB1D6v (ORCPT + 28 others); Tue, 27 Feb 2018 22:58:51 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:35663 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752110AbeB1D6r (ORCPT ); Tue, 27 Feb 2018 22:58:47 -0500 Received: by mail-pf0-f195.google.com with SMTP id y186so514075pfb.2 for ; Tue, 27 Feb 2018 19:58:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Dc0kkeu44UfOi1EiXBfePyqu3iL2CZsHYUa6WCie3uU=; b=I2x2yCrdloDJsDzGky18ImiprE3074QmGs0w/bU6KgN4DSY09E0MdREcIWl0Ltbmgl LJo4BMEz9o89GiWf/2NedS+MvCaCGozBaJRgQ2P72Xj7UV5zmMwtdfM5EQfrCfleOE+T 24Nca5qIOMRKqx65mKx9mfNLA+rd2DzLkGWis= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Dc0kkeu44UfOi1EiXBfePyqu3iL2CZsHYUa6WCie3uU=; b=qtq6T0IGYjWbi/NSKNHDjbveMIi4iUfoqtSBB+U7RX0t1fUDomUDskRH2h5zeDoRN3 yXaOuIeU4RCYLQD4F9sjhiCtfkksge5ek6JCOrKChp2tAsWC0DSAgDX+/Olx7Yb+wZOU F2tWJVHk3TZy63umB169sTU0358JZKgCS8JJaFJnELvnCoiXcmUhUqoUceqimWnR+5A+ bu9uN1fXJyQqWW3p5smK3GFZuhIeqto9HZAmg6gy+0TYOnwZyfC9MYUFY+UDjKcr9IKR vmOaKgpNK/4wmkzNPR6VtjQRXpcEA23S7CU7L+jCt6hDG0ehvUvXVXiD72cdfcNRKNCm rpFA== X-Gm-Message-State: APf1xPA5u1/p3Nst01uJK207cJKNr2emydD+ozUPL73PUKdFAPytsdLt S4x2PEFGqHp+IJ9NrmqXROUC6w== X-Received: by 10.101.66.196 with SMTP id l4mr13030606pgp.66.1519790326774; Tue, 27 Feb 2018 19:58:46 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.58.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:58:46 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 10/29] arm64: mm: Map entry trampoline into trampoline and kernel page tables Date: Wed, 28 Feb 2018 11:56:32 +0800 Message-Id: <1519790211-16582-11-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 51a0048beb44 upstream. The exception entry trampoline needs to be mapped at the same virtual address in both the trampoline page table (which maps nothing else) and also the kernel page table, so that we can swizzle TTBR1_EL1 on exceptions from and return to EL0. This patch maps the trampoline at a fixed virtual address in the fixmap area of the kernel virtual address space, which allows the kernel proper to be randomized with respect to the trampoline when KASLR is enabled. Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no acpi apei in arch/arm64/include/asm/fixmap.h no rodata in arch/arm64/mm/mmu.c --- arch/arm64/include/asm/fixmap.h | 5 +++++ arch/arm64/include/asm/pgtable.h | 1 + arch/arm64/kernel/asm-offsets.c | 6 +++++- arch/arm64/mm/mmu.c | 23 +++++++++++++++++++++++ 4 files changed, 34 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index caf86be..7b1d88c 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -51,6 +51,11 @@ enum fixed_addresses { FIX_EARLYCON_MEM_BASE, FIX_TEXT_POKE0, + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + FIX_ENTRY_TRAMP_TEXT, +#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT)) +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses, /* diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index 7acd3c5..3a30a39 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -692,6 +692,7 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm, extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; extern pgd_t idmap_pg_dir[PTRS_PER_PGD]; +extern pgd_t tramp_pg_dir[PTRS_PER_PGD]; /* * Encode and decode a swap entry: diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index c58ddf8..5f4bf3c 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -144,11 +145,14 @@ int main(void) DEFINE(ARM_SMCCC_RES_X2_OFFS, offsetof(struct arm_smccc_res, a2)); DEFINE(ARM_SMCCC_QUIRK_ID_OFFS, offsetof(struct arm_smccc_quirk, id)); DEFINE(ARM_SMCCC_QUIRK_STATE_OFFS, offsetof(struct arm_smccc_quirk, state)); - BLANK(); DEFINE(HIBERN_PBE_ORIG, offsetof(struct pbe, orig_address)); DEFINE(HIBERN_PBE_ADDR, offsetof(struct pbe, address)); DEFINE(HIBERN_PBE_NEXT, offsetof(struct pbe, next)); DEFINE(ARM64_FTR_SYSVAL, offsetof(struct arm64_ftr_reg, sys_val)); + BLANK(); +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + DEFINE(TRAMP_VALIAS, TRAMP_VALIAS); +#endif return 0; } diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d5cc6d7..84945c9 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -419,6 +419,29 @@ static void __init map_kernel_segment(pgd_t *pgd, void *va_start, void *va_end, vm_area_add_early(vma); } +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static int __init map_entry_trampoline(void) +{ + extern char __entry_tramp_text_start[]; + + pgprot_t prot = rodata_enabled ? PAGE_KERNEL_ROX : PAGE_KERNEL_EXEC; + phys_addr_t pa_start = __pa_symbol(__entry_tramp_text_start); + + /* The trampoline is always mapped and can therefore be global */ + pgprot_val(prot) &= ~PTE_NG; + + /* Map only the text into the trampoline page table */ + memset(tramp_pg_dir, 0, PGD_SIZE); + __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, PAGE_SIZE, + prot, pgd_pgtable_alloc, 0); + + /* ...as well as the kernel page table */ + __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); + return 0; +} +core_initcall(map_entry_trampoline); +#endif + /* * Create fine-grained mappings for the kernel. */ From patchwork Wed Feb 28 03:56:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129896 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568238lja; Tue, 27 Feb 2018 19:59:10 -0800 (PST) X-Google-Smtp-Source: AH8x224x/HSTj8bQQtU9ufwGvCMIjhWK5I0aNWIgLuUDKUJSuCWYdkSv6Ylr/uhiK9jaEWQaMlAE X-Received: by 10.98.9.5 with SMTP id e5mr16434620pfd.189.1519790350528; Tue, 27 Feb 2018 19:59:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790350; cv=none; d=google.com; s=arc-20160816; b=VsiQSNDev5TPlV3WX4nsB9yZNVopDc7B8MxpfAPcPVB2igi6/vlAlsI+MEDXKE00Ix /g+CJeMHxU7sMOJ65QzByY31+QbAohCA6LJ4I92Y8lmJKLkNf6u+V+Z3Lja9WExoomVP Ge0VIPdshJ2wkvyBsNgDHWONMSA2BhSynZHxPKbhoZZNeyBbE6z5tbu39DZB9vU9zsj1 g+1mgNv/AU1tkFEF559rVwj4NWZ7NhpDhoCP3pRKwJTFO9TW7S4rbz6dWqTcCh3jrh/1 9n1oOmQusANsokWP/mhbGfFNUNGA3d8MauWdQNbX+lPXnKzc4cAao+6NzFreaa1vLjKp v/ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=3EefTrlVGpCtN2KCozRnf/Q739j84WLPOQ3GnGLPWwc=; b=0SydzmFduYr7w/oZ0MyASdxwKJScihoR6/q1q+2xaikWKetHN9Fw7chfb8hSjlZxRx ZlI97K7bAH665GJXw4WxJKNRbrhd0B2SVrL4GHxFTCW0IUJVW2vUb2LCvBVXtfqlWjet wWOeAK7j9GIZEVNdgcrZdw4wBsriDlh/RYwOZURu0d90ErfTJxY8IgmATIG/TgJLYfnd ZSvwIaUOMj2f785KpTJ5PIu91SKwp2sLg+PxirbCnHfufQkfKf2dM9lrcdNbroXG2Ut2 IDBPmxB+jua4jxsvkcM2Rrx+GYyo1fVG4S8uw6I1dK8SAs8dMx+QW7d8DU3R2B65pXTh UuGA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=I5i7fWK9; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x2-v6si603010plr.334.2018.02.27.19.59.10; Tue, 27 Feb 2018 19:59:10 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=I5i7fWK9; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752244AbeB1D7F (ORCPT + 28 others); Tue, 27 Feb 2018 22:59:05 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:38485 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752219AbeB1D7C (ORCPT ); Tue, 27 Feb 2018 22:59:02 -0500 Received: by mail-pl0-f67.google.com with SMTP id d4-v6so775631pll.5 for ; Tue, 27 Feb 2018 19:59:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3EefTrlVGpCtN2KCozRnf/Q739j84WLPOQ3GnGLPWwc=; b=I5i7fWK90jD96UsGo0uqR6y4N2ezALYxV9gmWp5olA7bwSpBVt4RywMawse2zn4sJN Q6DCBw+XtUieaL7IqLPwhQXo/YqiS4FPUXdNDq72dyogy0UVCoxnTW7P1MOmcoePAtFR Qn0/CuSnOMlaiS9l0ut+ER656ENPug5QQf1A8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3EefTrlVGpCtN2KCozRnf/Q739j84WLPOQ3GnGLPWwc=; b=rHBUZJSzDZRgRZPSBoFp41du9IUb7KY1hiE296dnhLC527DhYmA4hSYI/WkKKl6aEa wAqVyt0Q07wpps/Th09wgSJY3rg/POVsr3fCTfr80LL0fNw4UaTMr8nqFcusu2zdrhQd RbE+3YklSGQKZ0KbdziinEFq5sbBDwaXP7WmKVIPlPxGPaWhWdLPTCKL+kOWyr2IoPvg J3gqnY8Mh1PTvtvzjWcG6sMV55UJVoYZCJEI7KKjIPA8jRag7Zz3H86jBtVoyf1bTekc 3KUs9w33GTIhOGggjJytEGmHGtvyvvtJNCgK34reDFl2kfnGOoATm0Z5rFV/1KqREB4z PWMQ== X-Gm-Message-State: APf1xPBkI0gQLwgXHR1y/VYj0AhHEksPOPvxyMSF1maQOFO+5rudxhVu fPL0GaRoX/t1dH2fwBSIUPnacHDxGfw= X-Received: by 2002:a17:902:4025:: with SMTP id b34-v6mr8396044pld.359.1519790341925; Tue, 27 Feb 2018 19:59:01 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.58.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:01 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 12/29] arm64: entry: Hook up entry trampoline to exception vectors Date: Wed, 28 Feb 2018 11:56:34 +0800 Message-Id: <1519790211-16582-13-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 4bf3286d29f3 upstream. Hook up the entry trampoline to our exception vectors so that all exceptions from and returns to EL0 go via the trampoline, which swizzles the vector base register accordingly. Transitioning to and from the kernel clobbers x30, so we use tpidrro_el0 and far_el1 as scratch registers for native tasks. Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kernel/entry.S | 39 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 36 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 7b1be51..eccb6d8 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -72,10 +72,26 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + .if \el == 0 + .if \regsize == 64 + mrs x30, tpidrro_el0 + msr tpidrro_el0, xzr + .else + mov x30, xzr + .endif + .endif +#endif + sub sp, sp, #S_FRAME_SIZE b el\()\el\()_\label .endm + .macro tramp_alias, dst, sym + mov_q \dst, TRAMP_VALIAS + add \dst, \dst, #(\sym - .entry.tramp.text) + .endm + .macro kernel_entry, el, regsize = 64 .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 @@ -157,18 +173,20 @@ ct_user_enter ldr x23, [sp, #S_SP] // load return stack pointer msr sp_el0, x23 + tst x22, #PSR_MODE32_BIT // native task? + b.eq 3f + #ifdef CONFIG_ARM64_ERRATUM_845719 alternative_if ARM64_WORKAROUND_845719 - tbz x22, #4, 1f #ifdef CONFIG_PID_IN_CONTEXTIDR mrs x29, contextidr_el1 msr contextidr_el1, x29 #else msr contextidr_el1, xzr #endif -1: alternative_else_nop_endif #endif +3: .endif msr elr_el1, x21 // set up the return data msr spsr_el1, x22 @@ -189,7 +207,22 @@ alternative_else_nop_endif ldp x28, x29, [sp, #16 * 14] ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp - eret // return to kernel + +#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 + eret +#else + .if \el == 0 + bne 4f + msr far_el1, x30 + tramp_alias x30, tramp_exit_native + br x30 +4: + tramp_alias x30, tramp_exit_compat + br x30 + .else + eret + .endif +#endif .endm .macro get_thread_info, rd From patchwork Wed Feb 28 03:56:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129898 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568361lja; Tue, 27 Feb 2018 19:59:24 -0800 (PST) X-Google-Smtp-Source: AG47ELseaD3Ef7n2Mor4dqeKenSazuYkZDGjB89HN7vpVyHvUH4naMRni440KhU5ZLiP1nnh9No5 X-Received: by 10.98.34.143 with SMTP id p15mr3584172pfj.101.1519790363679; Tue, 27 Feb 2018 19:59:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790363; cv=none; d=google.com; s=arc-20160816; b=vfkvMr5Mzriblt3aqtFZEk9Bw/+Eyal6v1dqckG2dfAn4GJbjzIqg+0jj1hcy+BSAz MhHiWG3cIjjriIWkX0BPjbgR80DXtWZtfuTrqM7dr1pl97Oz1LYzt/XbiwyZT7DjZ1En EWTixcD0oN73cGpDQrZzGZm8GtgUNBTwrH4hoyXh/p90lxtVDwhGLvuCVN7zi0z2vXTr yYe3gTE11tXr55AZiIc2eq1S2VGEgmNvaPNrNU2Ne2gbMcj2+p8jf0yn0wnv+kQb0OKY B835H99JSOK44PUlyb44o7xQM7r5AVt64xqGLygubvE15HT/qwdGbsIVAOqa/T6CGMVf T9Vg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=mw1ugMQkrRom83ynaVYoq/Tb8k4WnH3naAwpWK3t9VY=; b=WrK7Ims6qKXV+Xe5mBMJiivyDc9+uqKTPhlRLcaqShrDBD6DbyEo/D7hECsj3vNj4X Awun53l0tzPEYaOkfB5lEqmJte0XkCcmTbGElhPtrZOFK43zbCA/16zlBVK1MP1JSPSe UV5/gDYHKzMc1+HjVX17IFZnsssl+6ILY1JYo0hlYoSVf1ZrtASq+wjbUB2CGt2GzV2c gndUgowu1U8Ryq+pytVgkQrSw5KvH+5+4XchJOH5WnTL7OsFs0P9XW0Tvtgxg2CkwnsI vAUV0edxa9W9ql7uyhvkgiIIV/vHeNIDR+ctiZR6lGv/JQydd1SilZCGSyFfUmTSODon Gjpg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WjFaltI0; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w61-v6si571691plb.733.2018.02.27.19.59.23; Tue, 27 Feb 2018 19:59:23 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WjFaltI0; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932110AbeB1D7U (ORCPT + 28 others); Tue, 27 Feb 2018 22:59:20 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:43138 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932088AbeB1D7Q (ORCPT ); Tue, 27 Feb 2018 22:59:16 -0500 Received: by mail-pg0-f66.google.com with SMTP id e9so477992pgs.10 for ; Tue, 27 Feb 2018 19:59:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mw1ugMQkrRom83ynaVYoq/Tb8k4WnH3naAwpWK3t9VY=; b=WjFaltI08pTdm+kzBYOFtt4iDK7PidHLpFDoJAu4/JwHJxT17Wu1aphabty+nVIbvv VMcTErx8cc9VbtJVVk9seKpXKIWi4y+rfDC9opqCrxthXo+2bltaoZBqt38wN85P0EC9 r1+KxGwQbCZ2hVwAmrfjgUkwsDG6ejDyfBU5I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mw1ugMQkrRom83ynaVYoq/Tb8k4WnH3naAwpWK3t9VY=; b=fiG4uEdnF4Egt0qHA7FJ7zJhwuWg6dNMcrfFtGMKaxMNZnJgK8EEQNgLCBRX63vhck +tcOQO1U7px0c5iuvv3YLRbshiZhBR7xTcNcCTlyK+Qv2FPFkWFKDxNoGPNcg2BZlONY OZq1Q0AuNrgGhoV9OztVdu5o98aBNJlB8YLgGPrRdb36aGp/gGiW9SMKSGTaFl56dITu GVDVgQ/pvScc2OyujPZOQoQufgnGqSkRC/pRAx4vMIoupRfwudW+X4FBG3HZREaxPfUy sGkfPEPWNNsMY/GhgWT50iLpY/ihl9fSF+tFg1JbV7t8IxDLSAO8puGPmVPJWkqGNPWc CQvA== X-Gm-Message-State: APf1xPBdAggpyixZgVsDfOTbKyiRp66X5wBjMepHxLiaBMnz7RzOXvTh d+hB9oLGE54LecdOPJpewBzOPA== X-Received: by 10.99.9.195 with SMTP id 186mr12972734pgj.152.1519790356397; Tue, 27 Feb 2018 19:59:16 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:15 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 14/29] arm64: entry: Add fake CPU feature for unmapping the kernel at EL0 Date: Wed, 28 Feb 2018 11:56:36 +0800 Message-Id: <1519790211-16582-15-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit ea1e3de85e94 upstream. Allow explicit disabling of the entry trampoline on the kernel command line (kpti=off) by adding a fake CPU feature (ARM64_UNMAP_KERNEL_AT_EL0) that can be used to toggle the alternative sequences in our entry code and avoid use of the trampoline altogether if desired. This also allows us to make use of a static key in arm64_kernel_unmapped_at_el0(). Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: skip non enabled cpu features in arch/arm64/include/asm/cpucaps.h and arch/arm64/kernel/cpufeature.c using cpus_have_cap instead of cpus_have_const_cap in arch/arm64/include/asm/mmu.h --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/mmu.h | 3 ++- arch/arm64/kernel/cpufeature.c | 41 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/entry.S | 9 +++++---- 4 files changed, 50 insertions(+), 6 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 87b4465..7ddf233 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -34,7 +34,8 @@ #define ARM64_HAS_32BIT_EL0 13 #define ARM64_HYP_OFFSET_LOW 14 #define ARM64_MISMATCHED_CACHE_LINE_SIZE 15 +#define ARM64_UNMAP_KERNEL_AT_EL0 16 -#define ARM64_NCAPS 16 +#define ARM64_NCAPS 17 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 279e75b..a813edf 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -34,7 +34,8 @@ typedef struct { static inline bool arm64_kernel_unmapped_at_el0(void) { - return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0); + return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) && + cpus_have_cap(ARM64_UNMAP_KERNEL_AT_EL0); } extern void paging_init(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 3a129d4..74b168c 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -746,6 +746,40 @@ static bool hyp_offset_low(const struct arm64_cpu_capabilities *entry, return idmap_addr > GENMASK(VA_BITS - 2, 0) && !is_kernel_in_hyp_mode(); } +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */ + +static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, + int __unused) +{ + /* Forced on command line? */ + if (__kpti_forced) { + pr_info_once("kernel page table isolation forced %s by command line option\n", + __kpti_forced > 0 ? "ON" : "OFF"); + return __kpti_forced > 0; + } + + /* Useful for KASLR robustness */ + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + return true; + + return false; +} + +static int __init parse_kpti(char *str) +{ + bool enabled; + int ret = strtobool(str, &enabled); + + if (ret) + return ret; + + __kpti_forced = enabled ? 1 : -1; + return 0; +} +__setup("kpti=", parse_kpti); +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -829,6 +863,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .def_scope = SCOPE_SYSTEM, .matches = hyp_offset_low, }, +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + { + .capability = ARM64_UNMAP_KERNEL_AT_EL0, + .def_scope = SCOPE_SYSTEM, + .matches = unmap_kernel_at_el0, + }, +#endif {}, }; diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index eccb6d8..54f35cc 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -73,6 +73,7 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 .if \el == 0 .if \regsize == 64 mrs x30, tpidrro_el0 @@ -81,6 +82,7 @@ mov x30, xzr .endif .endif +alternative_else_nop_endif #endif sub sp, sp, #S_FRAME_SIZE @@ -208,10 +210,9 @@ alternative_else_nop_endif ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp -#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 - eret -#else .if \el == 0 +alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 bne 4f msr far_el1, x30 tramp_alias x30, tramp_exit_native @@ -219,10 +220,10 @@ alternative_else_nop_endif 4: tramp_alias x30, tramp_exit_compat br x30 +#endif .else eret .endif -#endif .endm .macro get_thread_info, rd From patchwork Wed Feb 28 03:56:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129899 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568413lja; Tue, 27 Feb 2018 19:59:29 -0800 (PST) X-Google-Smtp-Source: AH8x224Axl5my3CwF3neHyiVUNJJmUWnQN2oO2zZXrYWk1JE+h2fbxxOzg/ZcZoJDXX2GUxDNeYs X-Received: by 10.99.116.23 with SMTP id p23mr12812399pgc.178.1519790369393; Tue, 27 Feb 2018 19:59:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790369; cv=none; d=google.com; s=arc-20160816; b=tmafZV3pSLRrRxEEkvS1iIOgmMfUhfBMxF2wHvTuXmuvNRPQzCmFLZ+Bi6TzFw7OfF Il+CPLzDrbtHai6z3Nf+lfGi80m42h3MdKS+RhqgJsQPB2aMxxJAeJ6mZ7W9vCRccdJZ FxAJQ/7kbNe1fu/MIGvTYyUjW2Dx2E4b8agooGBWri6SBrYf2zYzzP+SDtVXuP49Bh8o O8egcCT9ftjnQGjWmbBX0xsqz8c9gxZV2R+OgSZHUPdcDyt5yrVqCCc+eyhJRAOeMbKv iaStgubXMWaJ/dKAPPOZPE51N5G7PBg5nqt6WFt+r2Ww2KC6JKHMyulaw/rT9dwIqYMn IbYw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=tyMA723Y2s+lrou1aTVC8Ls8RrTZ3RWM2Sb0Xk16jlw=; b=oa/aDMI8w5Ngcpb3CU2lplZsHyPMCTwG5GDYd2bNgIqbDiu7LxdFaoql3mIdnJPo52 C+YJg9zm+em+FsJMc7UL75bqoW/2FyIKN27h7e2z4e5DOD2RnAvDAzi1jeLjffUVcm24 I0zBWzmsrO82JmU2dFfHFqwmhz71PmYI3n7+hRHnVenApRAcMV8hvJDKrds9FweHWBDz 1DsRIlcJfhvUkMf04jnuZVgIix2lteONL5+E/m8zOM2/OYzHOaO+TtE7QiegDErjS26w iuaaKf4W/nmYfA/rzSF51Saurs2kqEDejaXT9cagmBUKJVtMdETvJSvBI/WyIPkCPcIj jNHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bucoZaPh; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w61-v6si571691plb.733.2018.02.27.19.59.29; Tue, 27 Feb 2018 19:59:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bucoZaPh; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932132AbeB1D70 (ORCPT + 28 others); Tue, 27 Feb 2018 22:59:26 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:41342 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932088AbeB1D7X (ORCPT ); Tue, 27 Feb 2018 22:59:23 -0500 Received: by mail-pl0-f65.google.com with SMTP id k8-v6so769196pli.8 for ; Tue, 27 Feb 2018 19:59:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=tyMA723Y2s+lrou1aTVC8Ls8RrTZ3RWM2Sb0Xk16jlw=; b=bucoZaPhbV5kF+4NP1O17dVl/sUzBJYljcbJxSIUqXqnmT7zLtBZUFJClo16+sk3+g OZeOflSRg5auqbe0VfCUex4pVTYuJcNdRzQwp8L70VCTIFKr3tbAbKE9fc/WhCKOQdXu eBehB/11D2olRtJfHyI+XNGVZM6qbZBRDSTtU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=tyMA723Y2s+lrou1aTVC8Ls8RrTZ3RWM2Sb0Xk16jlw=; b=nHYmeCNzdIUx7rYy96UQJiz1g0cc6wL45R8pLX78aE+yTLBSmcayDidqhn32X7nPJk nMBwd3rYPTLby/Q6/pGT3mC1scKh7HvbYcnSifVFBXtX72APc5BrrVnzr8lZ4iNNZoIH AjR/fEm7UihSGA917ueiHwzag0XocTlMmoK/RgYp7eDpUgcMYhrcr23qPlkzKJTR1ub5 PniAx9a8saZ/Nq2E3jUVkFE9YiPgVbVGu8jHNv8CcP4BO24wUIj7CdBo1Aca8F1R1aSL DyvDYkiKlvXiaR7Zcgnx8ZiSiwUvSqS+IdNz5Wx1gJvannwXiQEGpMBjLbbE7At0oIrm LWLA== X-Gm-Message-State: APf1xPDMD/zplcZD7MTRHmmmUD32aZX4J+fOBnhbT3gCsbpCA644eFpI HWk3S/xIlh9KQnTnI7QMh8VzPw== X-Received: by 2002:a17:902:bb96:: with SMTP id m22-v6mr16494917pls.17.1519790363353; Tue, 27 Feb 2018 19:59:23 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:22 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 15/29] arm64: kaslr: Put kernel vectors address in separate data page Date: Wed, 28 Feb 2018 11:56:37 +0800 Message-Id: <1519790211-16582-16-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 6c27c4082f4f upstream. The literal pool entry for identifying the vectors base is the only piece of information in the trampoline page that identifies the true location of the kernel. This patch moves it into a page-aligned region of the .rodata section and maps this adjacent to the trampoline text via an additional fixmap entry, which protects against any accidental leakage of the trampoline contents. Suggested-by: Ard Biesheuvel Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: remove ARM64_WORKAROUND_QCOM_FALKOR_E1003 fix in arch/arm64/kernel/entry.S --- arch/arm64/include/asm/fixmap.h | 1 + arch/arm64/kernel/entry.S | 13 +++++++++++++ arch/arm64/kernel/vmlinux.lds.S | 5 ++++- arch/arm64/mm/mmu.c | 10 +++++++++- 4 files changed, 27 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index 7b1d88c..d8e5805 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -53,6 +53,7 @@ enum fixed_addresses { FIX_TEXT_POKE0, #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + FIX_ENTRY_TRAMP_DATA, FIX_ENTRY_TRAMP_TEXT, #define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT)) #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 54f35cc..996c605 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -903,7 +903,12 @@ __ni_sys_trace: msr tpidrro_el0, x30 // Restored in kernel_ventry .endif tramp_map_kernel x30 +#ifdef CONFIG_RANDOMIZE_BASE + adr x30, tramp_vectors + PAGE_SIZE + ldr x30, [x30] +#else ldr x30, =vectors +#endif prfm plil1strm, [x30, #(1b - tramp_vectors)] msr vbar_el1, x30 add x30, x30, #(1b - tramp_vectors) @@ -946,6 +951,14 @@ END(tramp_exit_compat) .ltorg .popsection // .entry.tramp.text +#ifdef CONFIG_RANDOMIZE_BASE + .pushsection ".rodata", "a" + .align PAGE_SHIFT + .globl __entry_tramp_data_start +__entry_tramp_data_start: + .quad vectors + .popsection // .rodata +#endif /* CONFIG_RANDOMIZE_BASE */ #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ /* diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 466a43a..6a58455 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -252,7 +252,10 @@ ASSERT(__idmap_text_end - (__idmap_text_start & ~(SZ_4K - 1)) <= SZ_4K, ASSERT(__hibernate_exit_text_end - (__hibernate_exit_text_start & ~(SZ_4K - 1)) <= SZ_4K, "Hibernate exit text too big or misaligned") #endif - +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +ASSERT((__entry_tramp_text_end - __entry_tramp_text_start) == PAGE_SIZE, + "Entry trampoline text too big") +#endif /* * If padding is applied before .head.text, virt<->phys conversions will fail. */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 84945c9..10db4bf 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -435,8 +435,16 @@ static int __init map_entry_trampoline(void) __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, PAGE_SIZE, prot, pgd_pgtable_alloc, 0); - /* ...as well as the kernel page table */ + /* Map both the text and data into the kernel page table */ __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) { + extern char __entry_tramp_data_start[]; + + __set_fixmap(FIX_ENTRY_TRAMP_DATA, + __pa_symbol(__entry_tramp_data_start), + PAGE_KERNEL_RO); + } + return 0; } core_initcall(map_entry_trampoline); From patchwork Wed Feb 28 03:56:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129900 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568461lja; Tue, 27 Feb 2018 19:59:36 -0800 (PST) X-Google-Smtp-Source: AH8x225z/sUA8t3zvDQF2ew9L/YYCb3KelXca6zz7qCL8HnxPrSO04D5g+TTAIe9gidRDLerd1+a X-Received: by 2002:a17:902:2de4:: with SMTP id p91-v6mr16669192plb.405.1519790376120; Tue, 27 Feb 2018 19:59:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790376; cv=none; d=google.com; s=arc-20160816; b=TLe7cQn/K3fVYCoD6zzZIcTu3GlVwLPhHw7DhZsoub6GJcwsuM09tlTgVj5jsVrCFo RMy7aZWFbpevfzr7cj1UgvtirykhEynbGRKd0kI2VMZwgFQKkvgmdO2r4enFCj6b53Hg xL5a7/ouMGv/jFVl5P0znXiyGO6eggd0/+a1R89zL8PqPKciTwf9gRc22PvA1+vyvYkf syygcGH+QED9vb5lbJa1qxt9+9PjqJ5g9SJowaa19R/FAPkIZz3TXNuJaU24MvzrtQ30 iXbQVnAktrKFVo1N+klQHV47mVfHWbNs28TT0W8/0R4znXUaTJLSPYZeRpDmnkDIIOGn H3qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=maE1x+zHtUmA+lsl2Bxum/L+15vTg0UTcHBJ7wI7oeM=; b=CDT/W9m0DqsXPap7gFjVdnnM+lt9CX0MW8t0I3Z+tnyymQF7FspUbMkK6HAH2IYM8c HpDZ3vTDhEHEfTi7Gx6+oBwLU3KK6c3D50OGV8JACfODoQ6FewkM5F7tNQ6I5U1yqfRP Sz0ef/6ZKXh+Mi+uKhkMIgjYBtUNEY9JxmBc3B80dcRSl5salolvhIhRoTEqjHYcRFgq 2YqqWWvxSi3KyZCBdtRV2rOIi6o+i0CIXDuGT5t0fjpfUN0I50jbSpatHB8y08NxmuYt 67Ll4yhng8mWgnZOYZe1FdcmFj/Fgd9VOE3JW0m4RuLWzWSbnpFUD7hTi2Qk7MZVQlQQ kPog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CM93PBOv; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q25si464141pge.457.2018.02.27.19.59.35; Tue, 27 Feb 2018 19:59:36 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CM93PBOv; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932148AbeB1D7d (ORCPT + 28 others); Tue, 27 Feb 2018 22:59:33 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:42014 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932088AbeB1D7a (ORCPT ); Tue, 27 Feb 2018 22:59:30 -0500 Received: by mail-pf0-f193.google.com with SMTP id a16so504584pfn.9 for ; Tue, 27 Feb 2018 19:59:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=maE1x+zHtUmA+lsl2Bxum/L+15vTg0UTcHBJ7wI7oeM=; b=CM93PBOvq9YTR78mUyTMRhpb0Ktj4IgHME1yNysliZ/Ihlc2LSpzL1bGbyuulv1Rla BsTENFh85LjhkRGPMI9q4yET9zW3A9DydSCcsJ1Ahycs2PIbmyLYr6PAkJ5NPgs9y3ox QNQAz9xNQ5ZVeDqdN3KJsPa3OddGIWXJRC+JI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=maE1x+zHtUmA+lsl2Bxum/L+15vTg0UTcHBJ7wI7oeM=; b=BDPU1OGAPwmtY4NmXhEBpaECIdg7eSrRfcefGlztJHx2//czVG7WHJivly3XWIC48p mL4oAAHGT1tcadJ+MvXy6ZWKK/lggEwHUTMw1/lfszY/6yEqQ58kkR/leKL14QFnu1jm kFEQgrN8GYNteW6pRk9KGczV0z0LEAmu6NFBDVEotWY5ThBss/h+sSHFzBKQk6p/zi6F 9iIQKVEJBkPLI3OtnOQHEirzIbeFHTBB/N1SwVUfxmc3Oi72KY1HZfZFqahnKmYxsJpn cSwUrNZzGYiAW+2d/x5qc28y/iHf4qWrXFclHTj0Klwwg6qKknq4LemGZ7Zz8MI4SQRK j/6g== X-Gm-Message-State: APf1xPB+VxNYAd1RtDqZOayZ6c7e2FeRk0e6V8vEW2QwNBYNSNHPKs8g tILKqCtpA3BSxBZ0p81dIzV4dg== X-Received: by 10.98.67.78 with SMTP id q75mr16016208pfa.98.1519790370200; Tue, 27 Feb 2018 19:59:30 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:29 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 16/29] arm64: use RET instruction for exiting the trampoline Date: Wed, 28 Feb 2018 11:56:38 +0800 Message-Id: <1519790211-16582-17-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit be04a6d1126b upstream. Speculation attacks against the entry trampoline can potentially resteer the speculative instruction stream through the indirect branch and into arbitrary gadgets within the kernel. This patch defends against these attacks by forcing a misprediction through the return stack: a dummy BL instruction loads an entry into the stack, so that the predicted program flow of the subsequent RET instruction is to a branch-to-self instruction which is finally resolved as a branch to the kernel vectors with speculation suppressed. Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/kernel/entry.S | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 996c605..c00921e 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -902,6 +902,14 @@ __ni_sys_trace: .if \regsize == 64 msr tpidrro_el0, x30 // Restored in kernel_ventry .endif + /* + * Defend against branch aliasing attacks by pushing a dummy + * entry onto the return stack and using a RET instruction to + * enter the full-fat kernel vectors. + */ + bl 2f + b . +2: tramp_map_kernel x30 #ifdef CONFIG_RANDOMIZE_BASE adr x30, tramp_vectors + PAGE_SIZE @@ -913,7 +921,7 @@ __ni_sys_trace: msr vbar_el1, x30 add x30, x30, #(1b - tramp_vectors) isb - br x30 + ret .endm .macro tramp_exit, regsize = 64 From patchwork Wed Feb 28 03:56:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129901 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568599lja; Tue, 27 Feb 2018 19:59:48 -0800 (PST) X-Google-Smtp-Source: AH8x225WCcq1L3xpgeIfDGbvvsl4bsrxe6e/1RmoBTmCVc6Tt6XoPzzxQAQ5YrE08cl6lDyQjxfU X-Received: by 10.99.113.75 with SMTP id b11mr13321153pgn.271.1519790388382; Tue, 27 Feb 2018 19:59:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790388; cv=none; d=google.com; s=arc-20160816; b=UBvKTiUmqPIDLGvMYxam+VmIq+hplq1kDGQDBGQDWBtdl1gBpHt3oPxVW6YR0jVJLO DWMMNnQIlcmWSOCM6rlx4G2CGx5Y7yi3ZUWllRujA01Fx86JpNNTRjmuOpJHdEoEQEoI 7KSBe/BTnt3Po9nryn/x+BP3NzaPAxy+t5BY4lFbwUHhktNc7vlnicE2SvgCB3Sq4Pce 2Ht0t8zpQhIJuzgCnii3gjb165WRIDU8Fikhrzv6Kjt8z/cFPcEB2YD1mkCs6joKr1fI cnF/56GY/xp+UywPDmKJPRtVy/tdlkF5UlrQqomqclb7o9/GtyLBwfXxx1h4ckA5i9Hi zLVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=95Iv0nJp083o8oUFCW7XEWIw7Oc3xuvo6wFQkXWVNj4=; b=WA35VI1coVVjZ1m30P3TW6MMVLRivw3K5slAs/Z9uGO1C6RZ8sdViv5GMFXakxcUFj d0LD7pQVj3wJs/tEpkbqP03FwrGReOmyP3ynTmUUcMrlbjmYh8zV2FqxL/guKx+dSdWE WsONEzelIeMl17uo/grm23MMn05B+vXHKIFbjz4dm17yduxUVcifETBn4Lo6eX86ogKi I996bWn+8JHNcTDWCG5Au/kynD9jorQgpa7XVWbGKtByxDcNeZTwWXSN7bQr2MR/v1uK FslsR4djqxfFPikowS7VTSMt9iuk7nYSSGEw7iMa154SfBwacTK/hL1uYrUqaVtO1xwA m2hQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dnHsb0lk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q25si464141pge.457.2018.02.27.19.59.48; Tue, 27 Feb 2018 19:59:48 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dnHsb0lk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932190AbeB1D7o (ORCPT + 28 others); Tue, 27 Feb 2018 22:59:44 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:46265 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932151AbeB1D7h (ORCPT ); Tue, 27 Feb 2018 22:59:37 -0500 Received: by mail-pl0-f68.google.com with SMTP id y8-v6so760662pll.13 for ; Tue, 27 Feb 2018 19:59:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=95Iv0nJp083o8oUFCW7XEWIw7Oc3xuvo6wFQkXWVNj4=; b=dnHsb0lkyUAzCef0hhzZgt+tMQS2Wkc6UzQyLyPDkjmwGFPyA3FjnmeApi342yzp+W sb3C0gPIn+IcLSeyYfvSv99FPlms/Yr5yq6KpZgXTVVeN4olcrFgKoABaA4jyBlWd0uh FfmLGE3SCuzwfHiJUM6ar39xKpXq4OQjuh77s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=95Iv0nJp083o8oUFCW7XEWIw7Oc3xuvo6wFQkXWVNj4=; b=Gui/w+Og6HQS2HWwHNoz4k82ydEIqkdM43Sjo04krKJNfScwQdPlk1ojxKl81528re p73tUNGNrsy1RB1/RAMgD25AzQe6CchxWVsQRMXcwGYCG5JKSPbTj5+cHUa6oawGM5RV VtYO3ythuEprT+zknjJbdpcYrGxwNmA/aGSKa+9kxtTKEC1CBarvFQlrD6IQHcENdhBG U1vBqxavZMvgn8ZI0Gd8HnAIkAgfI8neAXZWq8JJDlA2MNY68+xvx5h1naGa6mtFF0Cf /x3cA7qWOamXfiw0lOmfB/cajaSIQfDTbJbFiT1q3cyu4ZKJU0G+zemqAZQ8Ag9n/TD8 553Q== X-Gm-Message-State: APf1xPAJS9bIk2MncKWdvWRElkTRQPX5FBgZgS4CNTta0ibYtV5IFMOV IIVeBmur/of6tfHvkXKbHiPJ1w== X-Received: by 2002:a17:902:f81:: with SMTP id 1-v6mr16025516plz.265.1519790377200; Tue, 27 Feb 2018 19:59:37 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:36 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 17/29] arm64: Kconfig: Add CONFIG_UNMAP_KERNEL_AT_EL0 Date: Wed, 28 Feb 2018 11:56:39 +0800 Message-Id: <1519790211-16582-18-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 084eb77cd3a8 upstream. Add a Kconfig entry to control use of the entry trampoline, which allows us to unmap the kernel whilst running in userspace and improve the robustness of KASLR. Reviewed-by: Mark Rutland Tested-by: Laura Abbott Tested-by: Shanker Donthineni Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/Kconfig | 13 +++++++++++++ 1 file changed, 13 insertions(+) -- 2.7.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7769c2e..6b6e9f8 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -733,6 +733,19 @@ config FORCE_MAX_ZONEORDER However for 4K, we choose a higher default value, 11 as opposed to 10, giving us 4M allocations matching the default size used by generic code. +config UNMAP_KERNEL_AT_EL0 + bool "Unmap kernel when running in userspace (aka \"KAISER\")" + default y + help + Some attacks against KASLR make use of the timing difference between + a permission fault which could arise from a page table entry that is + present in the TLB, and a translation fault which always requires a + page table walk. This option defends against these attacks by unmapping + the kernel whilst running in userspace, therefore forcing translation + faults for all of kernel space. + + If unsure, say Y. + menuconfig ARMV8_DEPRECATED bool "Emulate deprecated/obsolete ARMv8 instructions" depends on COMPAT From patchwork Wed Feb 28 03:56:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129902 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568641lja; Tue, 27 Feb 2018 19:59:51 -0800 (PST) X-Google-Smtp-Source: AH8x224okyEseDf1mhKps+rYhmKCXomQAKbxr51kOwQEtGiTmWvIsc1lTofaHb9sLdT5NLmHpIZP X-Received: by 2002:a17:902:63:: with SMTP id 90-v6mr16277597pla.125.1519790391335; Tue, 27 Feb 2018 19:59:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790391; cv=none; d=google.com; s=arc-20160816; b=LnO6A8lFQ+qWzlVfQpeCE5v89OmyyyGKaVvON69IB3bCQ9UcKAWc1I8ogLgPZZcj2f GAJ/jy6W5KE0x5SnTdWWdr8/W/2JvHTkjTXvl2Li3Z3kGIjs45daWdYyvK9kixlnNZZg +cSSHYy41savbumXxP6oYC37r62y3o5IB+oznWuYKYHmGf2cSC9+Cx0OWf0MU1t+asjy C40YBjJK6rzhjQMvm+ux9wLr48glEVyZvT+UK6eVL7i36k1jXnCOe5FUj9xu0f4/zD2Q D7tnYVqcNB1RrXW6tcruozF1HaHqAqH661BLl/ldjsl2GS6N1YCX0etpYIMLQFhQXgYV KF8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=3X/tJlgDOIhzeNz+KgV3mfSlCMlc7iRs7Xo6UX1dC64=; b=XEYzh5sbmwo1mirOrxvjHJhkRahzQ0mMUfRjF/lMLExzPwSDu2jZEnFLWzWow9HaD1 G4GGbq339KP9Fz+/T56jDwhHoXyNfQ4ssJJOL6ricRnk6M4q4YavEsUaWu3WaP5PmYrZ 2hY6HGDAwOYVnff2TfcW4yI4LL/aOUkynY341jwPH5ZJwggTW+DcdCfkXiV7k2Yo9O73 /0ZMsSZBvbu6RGoFIZQy41hcC7USLtpzoPyKhfbl9Ncw4nvtOR+kTSuyp0Mwb1+HgdSV 8uMjHhGDZ14UschlI3lXHBgaT4/H+AQtBJ/yfGZzWbJpQzp0+3mmeOSH1QDF4Ij2eAqs JM7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XU8+JuQA; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q25si464141pge.457.2018.02.27.19.59.50; Tue, 27 Feb 2018 19:59:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XU8+JuQA; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932210AbeB1D7r (ORCPT + 28 others); Tue, 27 Feb 2018 22:59:47 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:42771 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932151AbeB1D7p (ORCPT ); Tue, 27 Feb 2018 22:59:45 -0500 Received: by mail-pl0-f68.google.com with SMTP id 93-v6so769290plc.9 for ; Tue, 27 Feb 2018 19:59:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3X/tJlgDOIhzeNz+KgV3mfSlCMlc7iRs7Xo6UX1dC64=; b=XU8+JuQAhG632Xltrlwg6usmgZ3qVBV7cZP/Q3GPXP9wf2L1IzqpYeyS/rqr4IGR40 pE+zuKxp3sEsEctkCqk2eeMNIJipCzVdp86uAYRxM2yhDWMH/O1VJ2wmMlRO9I9BJbMQ DW6E3JVPaD/+JkhXNU66tXl4xGv1+b7Xv6xks= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3X/tJlgDOIhzeNz+KgV3mfSlCMlc7iRs7Xo6UX1dC64=; b=kNPOsZLfpKaBTmf35tLtBBi6DdM5gXyiPcDOshkPn/S9KhbT8eF0q1SCpD92rMm5dN vxOR5KW2iPRTo2+Erh/gm+QQuefWsXaDreFH/1tcUJs+ISIjUi2KT29npmoVJZDQY2pz SJ1pDOmcxf3S9ZMJRJgRcSWpoZy8dIkk2LYpX35d4oB4Wz3AUnkc7CIjv4vr7rj9RHba lGaLGTq+YbTBMSWQ2SQc/MOsWKy2hGuI7PSUPnBDB70CfpCZma709K3kDVS0PWYBVNEP rGGcKMD2Eeed13LEgGRixW4ZB9xQpf3ToXyp1reM5GW1JaZLYB0aOQ2SxI4/bI1AnBv+ z0Cw== X-Gm-Message-State: APf1xPD24+NYteaRYQngD0WeuseAd/LzMDmYISK+c1t7SX+l+gRYbWkt vFEi9kvJ24ztxatxpzWWltUTmg== X-Received: by 2002:a17:902:6b83:: with SMTP id p3-v6mr16327926plk.18.1519790384963; Tue, 27 Feb 2018 19:59:44 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:44 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 18/29] arm64: Kconfig: Reword UNMAP_KERNEL_AT_EL0 kconfig entry Date: Wed, 28 Feb 2018 11:56:40 +0800 Message-Id: <1519790211-16582-19-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 0617052ddde3 upstream. Although CONFIG_UNMAP_KERNEL_AT_EL0 does make KASLR more robust, it's actually more useful as a mitigation against speculation attacks that can leak arbitrary kernel data to userspace through speculation. Reword the Kconfig help message to reflect this, and make the option depend on EXPERT so that it is on by default for the majority of users. Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/Kconfig | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) -- 2.7.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 6b6e9f8..c8471cf 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -734,15 +734,14 @@ config FORCE_MAX_ZONEORDER 4M allocations matching the default size used by generic code. config UNMAP_KERNEL_AT_EL0 - bool "Unmap kernel when running in userspace (aka \"KAISER\")" + bool "Unmap kernel when running in userspace (aka \"KAISER\")" if EXPERT default y help - Some attacks against KASLR make use of the timing difference between - a permission fault which could arise from a page table entry that is - present in the TLB, and a translation fault which always requires a - page table walk. This option defends against these attacks by unmapping - the kernel whilst running in userspace, therefore forcing translation - faults for all of kernel space. + Speculation attacks against some high-performance processors can + be used to bypass MMU permission checks and leak kernel data to + userspace. This can be defended against by unmapping the kernel + when running in userspace, mapping it back in on exception entry + via a trampoline page in the vector table. If unsure, say Y. From patchwork Wed Feb 28 03:56:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129903 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568739lja; Tue, 27 Feb 2018 19:59:59 -0800 (PST) X-Google-Smtp-Source: AH8x227s1cx+Q0x6/IIur6ud55s8GxxhTsKTV09g3VjZUvrZPI8kikf7hEBVdLVKSX3JRWI/d3kW X-Received: by 10.101.76.13 with SMTP id u13mr12788888pgq.287.1519790399003; Tue, 27 Feb 2018 19:59:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790398; cv=none; d=google.com; s=arc-20160816; b=fXNcdHIE0V1Xd9nl3aB/HNujlZy708ZDKoEXiWmfNCOjmqNvm/BTzwKDNnFWOQf/ke +/zzuQPXomKE9T00KZudZz6yfHs7dFIOeo5AFBiU4e4vsoNL+s+W1wAOc52sFRAD9oUr TzFjMV3jBHA+fAASm05eiwSl6zF3L/esl3ZqzikiooxSctcEyzd7/ZLtEcFhyjWQLFrd oxsBs//JdJzWu2z251L1UrUUgDyzCBIHywxrHi5AU1893Zm1xcTfu/boj6Us0aAQpNqE 5ANHYx7Z7TUPz74Hv4oNrHFid9eGRcjC5bswl+muArOVTJz+Ucl+1hh0KTsI8RpeJJ/W 1S2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=Y7Yb5ubO8PqpEAZQ4vm3NWKhm2ulQzVizy5/nfrU5Tc=; b=ln1UbKMiL38Gr6PMU5z2AGAiUCsbTMcJb8dIzgmXGSIpZKJblx/hB7gu2yqJ31oL6r VA8tx2/c9Adf4zH/AZx6MyhC2zid83nt53OoHFqhsk1oZZ3SCTcEE6B3coEJl/xNC9Xm ufZC/n/joTs0F1+TbNCzfnV5LBZ1cfdwfmNVN9dXC1YPXJNNLxpI2Df6WPVUj4Bnca3y tqKPCnlr0sg0hqyFRzs49Mrov1kg7YPWOs9PxEkkqrU1EbjQ+H4sxtZ3tV48i2KX+hha gd/52PGggybMkNMXkdAdt5Pdwo+fEd71FGWy16RYt9WOLUU0IPbWQZWV+GS4SjzZDM+4 VOYA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kQsGIn/P; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a34-v6si597978pld.27.2018.02.27.19.59.58; Tue, 27 Feb 2018 19:59:58 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kQsGIn/P; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932226AbeB1D74 (ORCPT + 28 others); Tue, 27 Feb 2018 22:59:56 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:34125 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932197AbeB1D7w (ORCPT ); Tue, 27 Feb 2018 22:59:52 -0500 Received: by mail-pl0-f67.google.com with SMTP id u13-v6so780974plq.1 for ; Tue, 27 Feb 2018 19:59:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Y7Yb5ubO8PqpEAZQ4vm3NWKhm2ulQzVizy5/nfrU5Tc=; b=kQsGIn/PXxWANlzu2KuirJ298WQt6Lq2i490awihM/Q16+AXttHAsoFBG5ITJiWqJO 3DPwWPwtLJAmGW8vuww/+ESywolI1YJTkmQZxSqmwGEi6wvAfTaj3VwWvToKmWIPcYHE 3Szp6g8mcqAZWMgYemBRaxlm0CoFxPBoYnRkY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Y7Yb5ubO8PqpEAZQ4vm3NWKhm2ulQzVizy5/nfrU5Tc=; b=JJiQPSteVfGi8e96xlaRRvVBDDrGtLpdpedaZe5X26V/tJP744j65plK/FTVKSAjCt FV8R0nEN3RkfWAv5C0KTmyB3vG5eZZV+iGlClq6ng0IG8TJLPlzCRyp3FDRQcB6ucS+1 VI+cGzRn/ESdUGaaAeqXQdKmO4uJaTKLToWPDESrofXo8BYxciW46QfQclQ0WYBi+eXS 8YjEnpeG031QUUGcCFnS+dmT5FGse2CKVKDIYiVsoAMmqYhuVsJxNE5nI1zeuTMWDv14 X9inSRo2ZvJwrMZYKQPqeKZj7jRtWgchAeguERkjHTj3HnKNq47hiUP9/oex4FqnOfe9 Sbdw== X-Gm-Message-State: APf1xPD7ndhUBZwogqVosoENp8FWHGGqZimO5NoMvStCQaIekXmlX04c HyU5/TTimDHFJd9XW5llsxE1HQ== X-Received: by 2002:a17:902:678e:: with SMTP id g14-v6mr127770plk.440.1519790392423; Tue, 27 Feb 2018 19:59:52 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:51 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 19/29] arm64: Take into account ID_AA64PFR0_EL1.CSV3 Date: Wed, 28 Feb 2018 11:56:41 +0800 Message-Id: <1519790211-16582-20-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 179a56f6f9fb upstream. For non-KASLR kernels where the KPTI behaviour has not been overridden on the command line we can use ID_AA64PFR0_EL1.CSV3 to determine whether or not we should unmap the kernel whilst running at EL0. Reviewed-by: Suzuki K Poulose Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: skip cpu features like SVE etc. and use 5 paramaters function ARM64_FTR_BITS() replace read_sanitised_ftr_reg with old name read_system_reg arch/arm64/include/asm/sysreg.h arch/arm64/kernel/cpufeature.c --- arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/kernel/cpufeature.c | 8 +++++++- 2 files changed, 8 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 7393cc7..7cb7f7c 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -117,6 +117,7 @@ #define ID_AA64ISAR0_AES_SHIFT 4 /* id_aa64pfr0 */ +#define ID_AA64PFR0_CSV3_SHIFT 60 #define ID_AA64PFR0_GIC_SHIFT 24 #define ID_AA64PFR0_ASIMD_SHIFT 20 #define ID_AA64PFR0_FP_SHIFT 16 diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 74b168c..3ec6517 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -98,6 +98,7 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { ARM64_FTR_BITS(FTR_STRICT, FTR_EXACT, ID_AA64PFR0_GIC_SHIFT, 4, 0), S_ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI), S_ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI), + ARM64_FTR_BITS(FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0), /* Linux doesn't care about the EL3 */ ARM64_FTR_BITS(FTR_NONSTRICT, FTR_EXACT, ID_AA64PFR0_EL3_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_STRICT, FTR_EXACT, ID_AA64PFR0_EL2_SHIFT, 4, 0), @@ -752,6 +753,8 @@ static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, int __unused) { + u64 pfr0 = read_system_reg(SYS_ID_AA64PFR0_EL1); + /* Forced on command line? */ if (__kpti_forced) { pr_info_once("kernel page table isolation forced %s by command line option\n", @@ -763,7 +766,9 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) return true; - return false; + /* Defer to CPU feature registers */ + return !cpuid_feature_extract_unsigned_field(pfr0, + ID_AA64PFR0_CSV3_SHIFT); } static int __init parse_kpti(char *str) @@ -865,6 +870,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = { }, #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 { + .desc = "Kernel page table isolation (KPTI)", .capability = ARM64_UNMAP_KERNEL_AT_EL0, .def_scope = SCOPE_SYSTEM, .matches = unmap_kernel_at_el0, From patchwork Wed Feb 28 03:56:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129904 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp568869lja; Tue, 27 Feb 2018 20:00:06 -0800 (PST) X-Google-Smtp-Source: AH8x225aU58r+q10zC62j+aRVCvrAVO1eKUb18/vGalSIvgi7GucFp1GuNpRosmEBfJz7PkdeNTe X-Received: by 10.99.174.5 with SMTP id q5mr13113063pgf.170.1519790406039; Tue, 27 Feb 2018 20:00:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790406; cv=none; d=google.com; s=arc-20160816; b=ZriNEcibFgmaYqvs+6eY3RfXITpU9YeQNe84mtQy7EbmalE/+jcVkrXFaRPWdPBRdP FX+JFDuCMSe9/tgikIT4X32w4WSIDCk0XQHujmUBtTbmBoo4iknb5f6dCMI+vKrBIO6W CTnqxjmDYSoqI9/K1MzEECXSinMH0HoEzfPhGG5S4Dm0LU1DDUgGzrVmGgt2VoXg9236 f4OJ18DAdO1kslRJnCOoLgssaQAiI8kSXrgdF69Pbmz8hhziICEArJXM5vEV1fskbNBM ddzszC0fWhrn1dmd7l6f/123rlFUxZAbqjANMiUFY6bPSRh/8sQvL/dv87xSJQsxBOod ntjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=OMOLk3BE6r9QGQmIu+rjUcCTZUD9wK3TDcxdkw1PHjk=; b=uO2OkedMNXi3FZpedOA5TQHu1afEIfkuMbYgmKSpAPNURWVULtWdbHyyDLUZJCkbRm a2r6qXF1uCJsc7+NEACpRbfOpyyaTrISAbk8ID1RXoQXmVdWBBuB2x2aUo+sK+x1NPMM 2W42aKWK3utDsbrCjLh2RezNdVbtMwx7oox084w//rvWB+AfnPkr+/RhRZb1vep0ulXH CBU2CHgfT2gtmvpoeg5jMPrCmOhGHtW8D4VhtlH3XHS6O2aC36NxDKYIOJaMWVzgBCdM AX2DM6bxK2PrdLSdEws7fTMd76/FBK0+Mnzpe0TZDt6DeI/0h5/fkLMYjXmsydi0Z7bk JLyA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZujYVB5w; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g17-v6si571790plo.357.2018.02.27.20.00.05; Tue, 27 Feb 2018 20:00:06 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZujYVB5w; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932244AbeB1EAD (ORCPT + 28 others); Tue, 27 Feb 2018 23:00:03 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:37608 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751863AbeB1D77 (ORCPT ); Tue, 27 Feb 2018 22:59:59 -0500 Received: by mail-pg0-f67.google.com with SMTP id y26so486349pgv.4 for ; Tue, 27 Feb 2018 19:59:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=OMOLk3BE6r9QGQmIu+rjUcCTZUD9wK3TDcxdkw1PHjk=; b=ZujYVB5wMdevtAjc3qrHnnQuXTTYa6SjTqsk4vDArOAJWfStlc1mbwGsFT7HPwS7Ik OgRVPXh4YlGwfUzk6VQRoxmRkbDWtoB59rtOQvL1NixYxMT9E79ycRiED6KobVDCjsQn qAdDq1l1eEhENqJ89keRIM6kzvS0PeRI6War0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=OMOLk3BE6r9QGQmIu+rjUcCTZUD9wK3TDcxdkw1PHjk=; b=VJBS7FclaVUY6RqqgdqGdYgXpmLDZiTeCQrI/2cIf5Y4vLkKZ4w/5zNUIyRY8fQR4X 1/3DWp1pIVEabgPcDfEiXMf9xq1rDv2wWViiAjsoEU/SNBFA4sFxfBkBhOVBo2kzlsJo QQAAiDInU9Pw/ZqGQZ3IvIZtTY1UU+X2aj66MgolF27FvFFbUfG2xwEtPin7HJQ8d9m9 fw4UASX3wvXfF8sq55nQr4RTW3GhAN53AAonUX5fiLu1cKeAiLgmL2pUKDyznGYXJ6Z5 OCakC+GG9yiPoL1VRpSgUqwAvLFtC5lV5qHqpohDJ8DabT3Oh03l3G878236RUnH3BZo FGxw== X-Gm-Message-State: APf1xPDhHX+Hp/uCx+dydOii8mdlR96gpC7FbGOs0w5rK2XKgU2Vw904 xFN0H6WhXBRlc9j2G17ReIT0tg== X-Received: by 10.98.202.138 with SMTP id y10mr16234337pfk.184.1519790398714; Tue, 27 Feb 2018 19:59:58 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 19:59:58 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 20/29] arm64: Allow checking of a CPU-local erratum Date: Wed, 28 Feb 2018 11:56:42 +0800 Message-Id: <1519790211-16582-21-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 8f4137588261d7504f4aa022dc9d1a1fd1940e8e upstream. this_cpu_has_cap() only checks the feature array, and not the errata one. In order to be able to check for a CPU-local erratum, allow it to inspect the latter as well. This is consistent with cpus_have_cap()'s behaviour, which includes errata already. Acked-by: Thomas Gleixner Acked-by: Daniel Lezcano Reviewed-by: Suzuki K Poulose Signed-off-by: Marc Zyngier Signed-off-by: Alex Shi --- arch/arm64/kernel/cpufeature.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 3ec6517..4e5dff1 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1103,20 +1103,29 @@ static void __init setup_feature_capabilities(void) * Check if the current CPU has a given feature capability. * Should be called from non-preemptible context. */ -bool this_cpu_has_cap(unsigned int cap) +static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array, + unsigned int cap) { const struct arm64_cpu_capabilities *caps; if (WARN_ON(preemptible())) return false; - for (caps = arm64_features; caps->desc; caps++) + for (caps = cap_array; caps->desc; caps++) if (caps->capability == cap && caps->matches) return caps->matches(caps, SCOPE_LOCAL_CPU); return false; } +extern const struct arm64_cpu_capabilities arm64_errata[]; + +bool this_cpu_has_cap(unsigned int cap) +{ + return (__this_cpu_has_cap(arm64_features, cap) || + __this_cpu_has_cap(arm64_errata, cap)); +} + void __init setup_cpu_features(void) { u32 cwg; From patchwork Wed Feb 28 03:56:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129905 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp569017lja; Tue, 27 Feb 2018 20:00:14 -0800 (PST) X-Google-Smtp-Source: AH8x226dzu5LYbbySciJuncty6P+q4SHM/MwUKUDMKaKSd2jGboch7beK8gwBH775yMW7Jd68yZ3 X-Received: by 10.99.122.74 with SMTP id j10mr12682059pgn.84.1519790414380; Tue, 27 Feb 2018 20:00:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790414; cv=none; d=google.com; s=arc-20160816; b=FSw1V6Kjm6SiYX0qsUGbpbTHuTLGqvyHd3rCpiXco3j2PR4CBpDiJqLn6+CkOeosLC P9DmkqrDf+f5hH0aqlIv9/+IxI1Nnzx2HzOh5xbHIMhIDaet7nRrQ373TsYnIxIG0Hhh ucIBOQtPRR+0Nm+i4PAh2JyBFhQY/ajxoNhGZ03tgjCWqxzAr76g5E5jGaNbMOFJcEcd qyQ8oU5G7apVwjPXLLLaKJ4qhFVAABMPqu/xcYTHNtlJSw37h/TK9cuyLz79GCw9N6ao WiFkG+UppUWK16DpEBJbPS6ADp6LxJZo7UMjIEOu6dIIRkwNTvG0/4fkKCSaW55k+JPl KetA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=dHNDx9kw7MHipWQb550NFtLt7C6FAMAdmYc8+9G2iPE=; b=vStegD/B8UEcea6n1Ns0GrYV/7y4KAox+McdzgAP9edqUB800TC3X9PfMUUZ17UXsk LkCMABpejCiRZCbmGG74NhQgD6cD9Cy46EUsUzADE1Qcnk+ndTqoZlUEtBmrhGlVE1pJ MCB19HBX5xJHTfKqZLQ1JRdnnPe5I0Nf/oa435GBEdVBWIMilodOJak+rsQDqbQT4ikP +p2zACYyFU6q9vmiFrCdrULZ8BxeQeQwucXW2aet1rYI2IGZKccNu+CnE2SiD1jETOaV poCaDp7ZDFlLpxK0aei4Gj4UxRSfG+ZRRmX35cpx9HlF8POeQ2hrsShZdeBEbfo4Huc2 joBg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=h1S8wsae; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g17-v6si571790plo.357.2018.02.27.20.00.13; Tue, 27 Feb 2018 20:00:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=h1S8wsae; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752279AbeB1EAJ (ORCPT + 28 others); Tue, 27 Feb 2018 23:00:09 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:45776 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751863AbeB1EAG (ORCPT ); Tue, 27 Feb 2018 23:00:06 -0500 Received: by mail-pf0-f193.google.com with SMTP id j24so502509pff.12 for ; Tue, 27 Feb 2018 20:00:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dHNDx9kw7MHipWQb550NFtLt7C6FAMAdmYc8+9G2iPE=; b=h1S8wsaeFW8IVTBk8hc4XBeJLTJLvX4Ytnhji056gbM5KuO2UNLGhSn3UIv6PgA9n5 XBiZVGac/aiTRO6P5b5r2zJnN7cQaAlGNs+a76axCx7c1c7jvv+09fHEAYhaWAkSRTBO 92pwR4bJ5C4ZoITsGUaRLvhnt6D9WjaTCdhvg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dHNDx9kw7MHipWQb550NFtLt7C6FAMAdmYc8+9G2iPE=; b=eqxFcJpPRSxxIiZ2k2UQUYnqxqpHv5k3iWX/68sChyoALcB8Gq7F+cjQlqzusP5zax kQJhTTMkxYE/quj57Z1RXGqqX0a78XoQQQxD0Nd3PuIdxflaCvgYtvHtmFVURD1KJI/j E7zzXtOyVE1ASD7/gmlWGYYjLkQO5u7EpgTHjbhvhMzLmcevvs4bWEaIXBQlwTgrbHyG U2fkHXvQNijDLD7e0bMLqu2O+N9DJ8Z3SfBeuUJ2eptsDTMbhCyEaelIKQlG19qVX1U/ gc9txf9Ocr0SRfiY4rQVnMvOP4Gw0zTyj8hjmnWuz/G9pMlw6QjN/2YXOmMeM3gtJncp t89g== X-Gm-Message-State: APf1xPCmMzWHBXJADu8r31HOoKaOWO+TXm1Khfgr+O9L2VM5Sg9YRM7L /ZxLolpHnpSZpaej2/AlfG0t3g== X-Received: by 10.98.23.136 with SMTP id 130mr16186438pfx.43.1519790405671; Tue, 27 Feb 2018 20:00:05 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.19.59.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 20:00:05 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Suzuki K Poulose , Mark Rutland , Alex Shi Subject: [PATCH 21/29] arm64: capabilities: Handle duplicate entries for a capability Date: Wed, 28 Feb 2018 11:56:43 +0800 Message-Id: <1519790211-16582-22-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Suzuki K Poulose commit 67948af41f2e upstream. Sometimes a single capability could be listed multiple times with differing matches(), e.g, CPU errata for different MIDR versions. This breaks verify_local_cpu_feature() and this_cpu_has_cap() as we stop checking for a capability on a CPU with the first entry in the given table, which is not sufficient. Make sure we run the checks for all entries of the same capability. We do this by fixing __this_cpu_has_cap() to run through all the entries in the given table for a match and reuse it for verify_local_cpu_feature(). Cc: Mark Rutland Cc: Will Deacon Acked-by: Marc Zyngier Signed-off-by: Suzuki K Poulose Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: arch/arm64/kernel/cpufeature.c --- arch/arm64/kernel/cpufeature.c | 44 ++++++++++++++++++++++-------------------- 1 file changed, 23 insertions(+), 21 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 4e5dff1..6200b81 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -969,6 +969,26 @@ static void __init setup_elf_hwcaps(const struct arm64_cpu_capabilities *hwcaps) cap_set_elf_hwcap(hwcaps); } +/* + * Check if the current CPU has a given feature capability. + * Should be called from non-preemptible context. + */ +static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array, + unsigned int cap) +{ + const struct arm64_cpu_capabilities *caps; + + if (WARN_ON(preemptible())) + return false; + + for (caps = cap_array; caps->desc; caps++) + if (caps->capability == cap && + caps->matches && + caps->matches(caps, SCOPE_LOCAL_CPU)) + return true; + return false; +} + void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps, const char *info) { @@ -1037,8 +1057,9 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps) } static void -verify_local_cpu_features(const struct arm64_cpu_capabilities *caps) +verify_local_cpu_features(const struct arm64_cpu_capabilities *caps_list) { + const struct arm64_cpu_capabilities *caps = caps_list; for (; caps->matches; caps++) { if (!cpus_have_cap(caps->capability)) continue; @@ -1046,7 +1067,7 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps) * If the new CPU misses an advertised feature, we cannot proceed * further, park the cpu. */ - if (!caps->matches(caps, SCOPE_LOCAL_CPU)) { + if (!__this_cpu_has_cap(caps_list, caps->capability)) { pr_crit("CPU%d: missing feature: %s\n", smp_processor_id(), caps->desc); cpu_die_early(); @@ -1099,25 +1120,6 @@ static void __init setup_feature_capabilities(void) enable_cpu_capabilities(arm64_features); } -/* - * Check if the current CPU has a given feature capability. - * Should be called from non-preemptible context. - */ -static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array, - unsigned int cap) -{ - const struct arm64_cpu_capabilities *caps; - - if (WARN_ON(preemptible())) - return false; - - for (caps = cap_array; caps->desc; caps++) - if (caps->capability == cap && caps->matches) - return caps->matches(caps, SCOPE_LOCAL_CPU); - - return false; -} - extern const struct arm64_cpu_capabilities arm64_errata[]; bool this_cpu_has_cap(unsigned int cap) From patchwork Wed Feb 28 03:56:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129906 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp569121lja; Tue, 27 Feb 2018 20:00:20 -0800 (PST) X-Google-Smtp-Source: AG47ELuKByx/RDwpmdXQXV77RAhsy5ojVvv0kXQ/3Wfpk2cEF4C66YWHCTtKXw1VIvTRuAZVF02w X-Received: by 10.99.42.207 with SMTP id q198mr5137941pgq.186.1519790420260; Tue, 27 Feb 2018 20:00:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790420; cv=none; d=google.com; s=arc-20160816; b=OoovJof+DxCGsXjddcWT/eQ7KKoHpIrGb/Mv1ixKITzSEaAA8DmkmFEeFHlALevYjT EUEG7/ac8T6SBggNF+olUb+d/rQS0GrXP9hT4hRTE4q3A1iBgAP4ljqj8UpV/dfoqucG diDJK87NxnnqnEder6g82m1LgANWjzX1rMfHdzhcbLWSyImmHWM1Yx4MA0uYEpjzEvHg 9rfcPcJs05bOdVrG6H5CGDIjrZHD7gwww9wizQeyXFXPVnbueiAkJK4ukLxBYK2T2pmS bBIsvY3K3AmgrA+uggaGiSWuT4fEmnI3vlAqqD9pd+yNOVC+MTElmwXFwD9tCKTCyxch 0Eyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=B05qSaRVvuSwICdyeBBOpt9FC3ROQfdqk+Z0jYLXYB8=; b=z91bDuA7OPrkOxWRwa7EoduM7dnG+AoaZSYKYstWv54lOt2F5aSvDzhw1LEAXr3+dO 3D9nU4yvsATP/lmil16h2atMO5nBCPmVXE5TdWNtWAVpZCn29W9jOktBotuHG+rmya7m 8NzJ8n1BLyGGIxlJbxnEWkUy7FHjA8VzIgmFtmI58fI8JaYnXgokvpvtGbYJ/duICcBt V/IYUBnBZZgxMP5N4Bh9WD53EjRapKz0PvgBKlF2u9Mnn/NlZ2kAxtUy6W+3bjoo3TUO 6XESztD8GZJpqAGR0eZlkrb/gUaUUZhcPWR83XBKN2EZEZnS8+rft6dMOFetD/IdjuD3 k48A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E6ju6OjC; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g17-v6si571790plo.357.2018.02.27.20.00.19; Tue, 27 Feb 2018 20:00:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E6ju6OjC; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752312AbeB1EAR (ORCPT + 28 others); Tue, 27 Feb 2018 23:00:17 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:35920 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752290AbeB1EAN (ORCPT ); Tue, 27 Feb 2018 23:00:13 -0500 Received: by mail-pl0-f67.google.com with SMTP id 61-v6so780510plf.3 for ; Tue, 27 Feb 2018 20:00:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=B05qSaRVvuSwICdyeBBOpt9FC3ROQfdqk+Z0jYLXYB8=; b=E6ju6OjCiJsW5pkjT/fPxQvtIiQg+l/HfJJI5xOGT0mwtTCJ0hCLfApzIxg7WWD5xC cZ9HfaUX3XxMIdHBv6Yp0kZHW3MQejF1pMeOly5HA86nNjDTkrNdRZ0LSV9U9vfDsboy EeU/PTipJz1mHikULojEdYv+cNsNsc2lCVpxU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=B05qSaRVvuSwICdyeBBOpt9FC3ROQfdqk+Z0jYLXYB8=; b=N/22DgcQm1cBiFgMKc/S3SMDAZ9LCfgtTPKA0eAATYSW7d5PJ4q9GKQ1inATUMh9+6 T26T2i+Utf3o2uUsSavjaVeyoetjkrwvIkTpuAWFqGo9wFngbIG3+v/ehHGGOAxThkqN hKgPQUhhtUPgAFJP1ilYgYKNfSzkRJqtJTW+YAX1fORmmEaxi+z/c3sCCvFjWIQQoTsZ SOvpX/K5WcLAJ/ESIKX1bpPasqfp87SpSormrbE6bSTJx4TfkKZoyoWTsjvK6X6XZz9N ickxrRIW6gbA/KfpvbJ6ALMfzh85bBsDrI+9x6abMhxW6DSlNVQLVpdvzFbR3b//O255 cvGg== X-Gm-Message-State: APf1xPCM7YxE7qQevMhYYGFZSIc5dSVIo8o2GIpXXSOhRcdoLwKrqAQB Lq3eNBxjjH6sSDqOqGZ7R86hKQ== X-Received: by 2002:a17:902:b416:: with SMTP id x22-v6mr4295471plr.279.1519790412629; Tue, 27 Feb 2018 20:00:12 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.20.00.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 20:00:12 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 22/29] arm64: cputype: Add missing MIDR values for Cortex-A72 and Cortex-A75 Date: Wed, 28 Feb 2018 11:56:44 +0800 Message-Id: <1519790211-16582-23-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit f0be3364335d47267aa1f7c5ed5faaa59c70db13 upstream Hook up MIDR values for the Cortex-A72 and Cortex-A75 CPUs, since they will soon need MIDR matches for hardening the branch predictor. Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: add A73 type in arch/arm64/include/asm/cputype.h --- arch/arm64/include/asm/cputype.h | 6 ++++++ 1 file changed, 6 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index 26a68dd..0843b3f 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -75,7 +75,10 @@ #define ARM_CPU_PART_AEM_V8 0xD0F #define ARM_CPU_PART_FOUNDATION 0xD00 #define ARM_CPU_PART_CORTEX_A57 0xD07 +#define ARM_CPU_PART_CORTEX_A72 0xD08 #define ARM_CPU_PART_CORTEX_A53 0xD03 +#define ARM_CPU_PART_CORTEX_A73 0xD09 +#define ARM_CPU_PART_CORTEX_A75 0xD0A #define APM_CPU_PART_POTENZA 0x000 @@ -86,6 +89,9 @@ #define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53) #define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57) +#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72) +#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73) +#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75) #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX) #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX) From patchwork Wed Feb 28 03:56:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129907 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp569238lja; Tue, 27 Feb 2018 20:00:25 -0800 (PST) X-Google-Smtp-Source: AH8x224grW61XOSGypq1Izmny+qK3oF7sS6XzIuMf8I0gZiq7tvefc7xGmx6eurQ1ERWazokkI9e X-Received: by 10.99.113.94 with SMTP id b30mr13103480pgn.228.1519790425741; Tue, 27 Feb 2018 20:00:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790425; cv=none; d=google.com; s=arc-20160816; b=NUypNSBeEa7lysYKVdiPpHQVdWXFCm6Qgp13414aWINfg5Vy8UewVmfYUWnvF2Sh0Z zd1O4eqN0PPwDlR+ikQUvQB2dBhHR9ty2+JCcPVb1IzTvyeLTv3AaDM6/72rtsQCx5Cw V8/Eb5Xw8eZ/4pUp6E33GLsmBSkxaPddyc3hNMyL4IdsT9yP/b0LupK2TffLl54x2tp2 NMBI/9ONJdz6CzyrKpKd3UyGZR5OpdFNLCCeu1xXcnKeH0m6WdD48ZUpa9UqOp85h8Xl kiLd2s+XVPnrGrfnWoZxL0t7N2KONcljK+khEmuOG2YKWSjXt7H0sqIBFmAhAC1lERnM wG8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=PoybqqZSHPpCwyXKEdU/iRxK/k+IGVxzQYoov7ys+2E=; b=Vgjrvo4pX7UMbG+mXp4tL7nFoiJTi0KF5VoDsfJUK938+6hdrssLOgoE9NUm6bL1WZ uCPKmGOnOJYAi0DRn1YHCc8uRcv4x+7hWE1sGxmtGWeqYax1WIt6HQjxfRQTh2RWkhwv miHpP9YYce4WHeaCtXdyliQUMbkJPMdzlKZVaM+haPBWx4xBenGd0dxQ9/xy2VZvfSUi te1OMkFLYH3l3Q7z+h6ydMOco39JWc2FE3zYtbuKLQisDQ3WErA0etOEw1ZWn8hS/BYk olbWE+/8SAgZRWg86VDwotNJqaR9gwEO61Bz1hpr1cw1XgAe1fFCHJZSPFd4MtVb90Vp ggTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=V2xzAr5l; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g17-v6si571790plo.357.2018.02.27.20.00.25; Tue, 27 Feb 2018 20:00:25 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=V2xzAr5l; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752333AbeB1EAW (ORCPT + 28 others); Tue, 27 Feb 2018 23:00:22 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:33054 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752016AbeB1EAT (ORCPT ); Tue, 27 Feb 2018 23:00:19 -0500 Received: by mail-pl0-f68.google.com with SMTP id c11-v6so781780plo.0 for ; Tue, 27 Feb 2018 20:00:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PoybqqZSHPpCwyXKEdU/iRxK/k+IGVxzQYoov7ys+2E=; b=V2xzAr5ltQF3Hy5DzYI7m0s+Jq/7VIZsaznsw8u2ieV7wLDRS3JWdiR3zv5GOsnlxK v7AKjNgPvqlnp9pS4AEG+Os6QXOAm1k8qztqzfYmwlywHqlzmCPZ8Ly2U6wTULhAenUM UocwCi2bPX1pVi5fAC/66MX27GrzfEaggT7EA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PoybqqZSHPpCwyXKEdU/iRxK/k+IGVxzQYoov7ys+2E=; b=tBRhWXIyUv51cLim6l+Hb0XkVeKrKeGZdzAnge7tANOU1XErijKcJDePCq2rWdJvqm dIcfBfMazhGWLelPUJG7dgAVKZHrzbJAjtStKp3optEBooI8iri4ASsSzOpUZF8tBrT2 j7o6qCWECkM/wf4z1g4m0jNyBc9rnFib9ONfmoEhveXRVeQ6Ydm3OeK4SDD3dxNdsydi vwEB5tayZVbgoUuOaNXjGzRkJ5tA4GSocbzhVkpiDKg9vxq4d24JOCU6caE6QOHzdl93 yQ11yyAKol0RZZ32i2S7oFXmMizkuW+FDySKDMlGEbilISl8Ns0rUJj8jVJoEVJ6a4uT E/yA== X-Gm-Message-State: APf1xPAlb1mYJDwDUgy1bdVOxIrLYhF9x9hKKfgoB+q00WnLRhL0pTJ5 qSI7Osn+0ljIt95lw1gNkZYA9g== X-Received: by 2002:a17:902:7404:: with SMTP id g4-v6mr16390270pll.235.1519790419043; Tue, 27 Feb 2018 20:00:19 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.20.00.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 20:00:18 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jayachandran C , Alex Shi Subject: [PATCH 23/29] arm64: cputype: Add MIDR values for Cavium ThunderX2 CPUs Date: Wed, 28 Feb 2018 11:56:45 +0800 Message-Id: <1519790211-16582-24-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Jayachandran C commit 0d90718871fe upstream. Add the older Broadcom ID as well as the new Cavium ID for ThunderX2 CPUs. Signed-off-by: Jayachandran C Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: no falkor support in arch/arm64/include/asm/cputype.h --- arch/arm64/include/asm/cputype.h | 3 +++ 1 file changed, 3 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index 0843b3f..9ee3038 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -84,6 +84,7 @@ #define CAVIUM_CPU_PART_THUNDERX 0x0A1 #define CAVIUM_CPU_PART_THUNDERX_81XX 0x0A2 +#define CAVIUM_CPU_PART_THUNDERX2 0x0AF #define BRCM_CPU_PART_VULCAN 0x516 @@ -94,6 +95,8 @@ #define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75) #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX) #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX) +#define MIDR_CAVIUM_THUNDERX2 MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX2) +#define MIDR_BRCM_VULCAN MIDR_CPU_MODEL(ARM_CPU_IMP_BRCM, BRCM_CPU_PART_VULCAN) #ifndef __ASSEMBLY__ From patchwork Wed Feb 28 03:56:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129908 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp569352lja; Tue, 27 Feb 2018 20:00:32 -0800 (PST) X-Google-Smtp-Source: AG47ELt1MgH/6nSIzwugWDpdhccBLGmlBuZxZbGoI/CH7JKKAB9YsoFxQ0npEy3RzHdP564hf6Jo X-Received: by 10.98.69.196 with SMTP id n65mr3796841pfi.29.1519790432407; Tue, 27 Feb 2018 20:00:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790432; cv=none; d=google.com; s=arc-20160816; b=JoNkimZl266DEAE8gcGobAygyDob4XowcmaYmLBlHotvPrt3Oba2BrL+bp3ljQfs2d 1iboytEZi4wBEe9AIyhnfN7Qrp7Q0Osip+XLZ/rQQVNTN3tM4LRQ+XMpGq25UVBxPGY6 j7YOEIWugByZ4AUFLNyjCF/BAWH52r7WpKMVdOXFJ3ihBzoQbohnJ07dvujl+NccrZc1 ZmXLzo2/TF5mSo/sUVZFW0lgyg1PwcSQE/Hi1CghueBiiIf8XfGBrAPJ7RsTi9S6J69x lUK2EAJtstiXHk7c+ZyDCF3p4WD6NXY4Az7Sqpj5kdH5UlJUwx49W7PR8tLPYSasixV6 NFIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=JC5B9Ogq58f9r18lYBOpTdl94tTzOD0MxFQZk4nXoSI=; b=oOQV5WvqeJq516z1Tfr+VC7/Xlubb93RTsRAKDz45c+gifwGJAxvR9lPP4g4UFn6qt hC+munfRtGLUMgr/EBO2uG3wX4lKzjpQ5CZxQjdh2BQq9ntrTzWk/Yf+69IKkd1Lwgc3 xr+ACdPCdLD0WdndNsXNg3lroh/g1UvzN8lroktDsJo61xasrerkbUVMQBu6/kjl4Weu 3rHXq1qyuxw7pQFeCtlDfGcoLNTpeP6VAvtlxTzFL+42zvhvPfpHlv7l5ie2zQpuMg9y /dE4eWx5N5aBJ7G4snET0t2Rf535b7UykMMUnNXTDG7VUn1SH4hz3YDMuRvBexw/ex// B6rg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZrT0fHbE; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g17-v6si571790plo.357.2018.02.27.20.00.31; Tue, 27 Feb 2018 20:00:32 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZrT0fHbE; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932260AbeB1EAa (ORCPT + 28 others); Tue, 27 Feb 2018 23:00:30 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:40728 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752339AbeB1EA0 (ORCPT ); Tue, 27 Feb 2018 23:00:26 -0500 Received: by mail-pl0-f67.google.com with SMTP id i6-v6so772801plt.7 for ; Tue, 27 Feb 2018 20:00:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JC5B9Ogq58f9r18lYBOpTdl94tTzOD0MxFQZk4nXoSI=; b=ZrT0fHbEFbb7oYRmH3KIm9/AALDvmwozkFQKNAEaeKzOMxlQBWeT7OyNwpY4W8LKq5 IBUDK74MDFUfIV7zCB82GqeDQwQ00EgZovJxeI6x50uvefK5G6XFjav4tRaD0QO3FR6d TkemZUVjyExhuLszuBiE46WHvVcP9c5yHxJ6c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JC5B9Ogq58f9r18lYBOpTdl94tTzOD0MxFQZk4nXoSI=; b=mQyaUoMAYpUho/Q1A3dkXTleWPe9vaAAgAWSPvOA8Kg5QUvBLtTGnK5Y3/jtuxMA7D GmwKLolJHt2LpWIV42nv6bUfk+ZNA1pA0y/h0CuCYjx06aV2R4QMeWJxcIe2zmpJ+YU8 VaDqpxhfm3rOfGmAaVgarkwqHw1IJFiTGZnj6tvlMSLTTTR2flEEm5h9bWvg/HeFkPmC /4fAZMKdG+1BDG+2qUltWzEaLFFqZBKYMDKUpXEwlIqH8fdKSHa3lyMT+9+yuoHH83WE RaghQ99u4NteI23o1Jh92pSLWoqn5h7PcGfyrjfx7xfEcW8GmlOgDyBt/ch9b1Y75A9V m/Rw== X-Gm-Message-State: APf1xPAQp3zVU4l/P/r6zDuOZx3pRYOEpinQl2dkHrzCVRUEtVQXX5tQ zUVFv9ZyBW+XGIQ9Y5tG6PX5ZA== X-Received: by 2002:a17:902:f81:: with SMTP id 1-v6mr16027770plz.265.1519790425841; Tue, 27 Feb 2018 20:00:25 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.20.00.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 20:00:25 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Jayachandran C , Alex Shi Subject: [PATCH 24/29] arm64: Turn on KPTI only on CPUs that need it Date: Wed, 28 Feb 2018 11:56:46 +0800 Message-Id: <1519790211-16582-25-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Jayachandran C commit 0ba2e29c7fc1 upstream. Whitelist Broadcom Vulcan/Cavium ThunderX2 processors in unmap_kernel_at_el0(). These CPUs are not vulnerable to CVE-2017-5754 and do not need KPTI when KASLR is off. Acked-by: Will Deacon Signed-off-by: Jayachandran C Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kernel/cpufeature.c | 7 +++++++ 1 file changed, 7 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 6200b81..e62583d 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -766,6 +766,13 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) return true; + /* Don't force KPTI for CPUs that are not vulnerable */ + switch (read_cpuid_id() & MIDR_CPU_MODEL_MASK) { + case MIDR_CAVIUM_THUNDERX2: + case MIDR_BRCM_VULCAN: + return false; + } + /* Defer to CPU feature registers */ return !cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_CSV3_SHIFT); From patchwork Wed Feb 28 03:56:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129910 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp569548lja; Tue, 27 Feb 2018 20:00:46 -0800 (PST) X-Google-Smtp-Source: AH8x224s1EfjAOqLToi2gLNmLybTgVIRz2/qXmhHxBBMgfN03vK+gQiUJ546TgB6o0mv7et29jQZ X-Received: by 10.101.92.6 with SMTP id u6mr13112162pgr.440.1519790446677; Tue, 27 Feb 2018 20:00:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790446; cv=none; d=google.com; s=arc-20160816; b=UNdzhzGD7sSKdJdGcsDOd5M4WZ6ISmqo0qIfuaeRGXa8twXsj/cE8cRitHL+IQTJrP TyqfjSd5GewoFh4t5WR1t5jFHfl8QVUftB2kie0ihSRAg9Mc9vLwGi3PA3tCSDFrwHWq P0NUhR9hm/+7wnuPP378KUXcHxd2nvdclUOND7Y2yETY2A3UMxpmo+i230zPtDYLQJ6z 3kD3BwW7pm4zXC/s3+KToMKYGqpiuESiA3Gr8pbF6qeKWaPDbSjha8NpTDGqB8okr13t UHhk+mfv0205BZXSOMkNAfovGoh4LYxlrVykN1tVsxZ+V8r5nE3topDLRKdoCRG5t2f7 dOyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=nhekXz+j4qe8zsvb6aqEDZ9CqrRgNU1mr7x/gWhrze4=; b=RUymb8bwVUCxNqrq/w00ku8UcJD6fNDDPDhxGnw/MdIWOPQ2u/eJowXWR9BkBj0Bx1 gSwCO31Dl2GL2fllskhCQwk87p33uAWGhXYrVbRp0rVHXFipAwVL/3bgFmnS5oYn+7Zl jLqj6LwY7/wUkWZWk0dI95lLEVdmQ8LwIEOJSgfDP6FTFPOTe3lno1u4RpsnrQsEIpTw v1QaPL+sQyHrZdDPYCdBq4h1F1Rd88GvQS/zoAa1/B37MkiLvqhnCOjI3Uv2WJ+BteVO YXFLVvWJ2gRySenh+TYMZX+lAeb37Fc56n4UlvHuMtkdKpPiVKSabqWUz5NzJy8WvMrl ugOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E780oBEF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c5-v6si604191pll.90.2018.02.27.20.00.46; Tue, 27 Feb 2018 20:00:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E780oBEF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752393AbeB1EAn (ORCPT + 28 others); Tue, 27 Feb 2018 23:00:43 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:42078 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752372AbeB1EAk (ORCPT ); Tue, 27 Feb 2018 23:00:40 -0500 Received: by mail-pf0-f195.google.com with SMTP id a16so505875pfn.9 for ; Tue, 27 Feb 2018 20:00:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nhekXz+j4qe8zsvb6aqEDZ9CqrRgNU1mr7x/gWhrze4=; b=E780oBEFWMFxsyAm5Y5zfMmAPY7BTyzMh6d8ZKTAqpZyxun6PepFyUzOdddqXmUmho ixkkIzUaPaFBuLNXLK8TwX3sqsCoybJqxS1uVpKpjLfDDiLWv8wvHFgjThC/gKVQU18J ReOlA+EYdifiLzel6r9S3Pe4XoAB/JYK4jlmM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nhekXz+j4qe8zsvb6aqEDZ9CqrRgNU1mr7x/gWhrze4=; b=WJD0MF/yNjB8wkJrjrtmeYRRf8u12EguccX4hZd/z6QB5u4ASIaJvWqRoxw5Gs/0fb GfOmFR7lqupTEPo/vtCGrxStsqujEWSpt0JDzueJuchP0w4ujsDBlZB+LjtoyHDJk6bP 05xgco1fdrtCXnkksHwRitS3OFtoowdxJPQPKELwAYvTt/uF1fp8jpq9NXkRvYvG5CTL 5bxoKPtZmciSNpYhWuv/UDqxQk4lcBzbGbbyCzPf2w5Mnt1PPqad8X7cL8PU5qG0X+LU b12Nx7CRMp1ILKUnepYpvgygrW2pyQ+sQoJ4V7H8VFPOFojOyRlKz/MpWDOWDYojAgta ZIyw== X-Gm-Message-State: APf1xPAlLnpdVyttjz6uvRqosAMrKRii2HXvShBUc5rX6SIcvKd2NWEf OlcNnMHqV6jCoP+hvX0bYGlocA== X-Received: by 10.99.115.5 with SMTP id o5mr13161480pgc.226.1519790439602; Tue, 27 Feb 2018 20:00:39 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.20.00.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 20:00:39 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 26/29] arm64: kpti: Add ->enable callback to remap swapper using nG mappings Date: Wed, 28 Feb 2018 11:56:48 +0800 Message-Id: <1519790211-16582-27-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit f992b4dfd58b upstream. Defaulting to global mappings for kernel space is generally good for performance and appears to be necessary for Cavium ThunderX. If we subsequently decide that we need to enable kpti, then we need to rewrite our existing page table entries to be non-global. This is fiddly, and made worse by the possible use of contiguous mappings, which require a strict break-before-make sequence. Since the enable callback runs on each online CPU from stop_machine context, we can have all CPUs enter the idmap, where secondaries can wait for the primary CPU to rewrite swapper with its MMU off. It's all fairly horrible, but at least it only runs once. Nicolas Dechesne found a bug on this commit which cause boot failure on db410c etc board. Ard Biesheuvel found it writting wrong contenct to ttbr1_el1 in __idmap_cpu_set_reserved_ttbr1 macro and fixed it by give it the right content. Tested-by: Marc Zyngier Reviewed-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Ard Biesheuvel Signed-off-by: Alex Shi Conflicts: no get_thread_info/post_ttbr_update_workaround/pre_disable_mmu_workaround in arch/arm64/include/asm/assembler.h and arch/arm64/mm/proc.S --- arch/arm64/include/asm/assembler.h | 3 + arch/arm64/kernel/cpufeature.c | 25 +++++ arch/arm64/mm/proc.S | 201 +++++++++++++++++++++++++++++++++++-- 3 files changed, 222 insertions(+), 7 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 851290d..7193bf9 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -413,4 +413,7 @@ alternative_endif movk \reg, :abs_g0_nc:\val .endm + .macro pte_to_phys, phys, pte + and \phys, \pte, #(((1 << (48 - PAGE_SHIFT)) - 1) << PAGE_SHIFT) + .endm #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index e62583d..0fba610 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -778,6 +778,30 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, ID_AA64PFR0_CSV3_SHIFT); } +static int kpti_install_ng_mappings(void *__unused) +{ + typedef void (kpti_remap_fn)(int, int, phys_addr_t); + extern kpti_remap_fn idmap_kpti_install_ng_mappings; + kpti_remap_fn *remap_fn; + + static bool kpti_applied = false; + int cpu = smp_processor_id(); + + if (kpti_applied) + return 0; + + remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings); + + cpu_install_idmap(); + remap_fn(cpu, num_online_cpus(), __pa_symbol(swapper_pg_dir)); + cpu_uninstall_idmap(); + + if (!cpu) + kpti_applied = true; + + return 0; +} + static int __init parse_kpti(char *str) { bool enabled; @@ -881,6 +905,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .capability = ARM64_UNMAP_KERNEL_AT_EL0, .def_scope = SCOPE_SYSTEM, .matches = unmap_kernel_at_el0, + .enable = kpti_install_ng_mappings, }, #endif {}, diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 3378f3e..5c268f5 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -148,6 +148,16 @@ alternative_else_nop_endif ENDPROC(cpu_do_switch_mm) .pushsection ".idmap.text", "ax" + +.macro __idmap_cpu_set_reserved_ttbr1, tmp1, tmp2 + adrp \tmp1, empty_zero_page + msr ttbr1_el1, \tmp1 + isb + tlbi vmalle1 + dsb nsh + isb +.endm + /* * void idmap_cpu_replace_ttbr1(phys_addr_t new_pgd) * @@ -158,13 +168,7 @@ ENTRY(idmap_cpu_replace_ttbr1) mrs x2, daif msr daifset, #0xf - adrp x1, empty_zero_page - msr ttbr1_el1, x1 - isb - - tlbi vmalle1 - dsb nsh - isb + __idmap_cpu_set_reserved_ttbr1 x1, x3 msr ttbr1_el1, x0 isb @@ -175,6 +179,189 @@ ENTRY(idmap_cpu_replace_ttbr1) ENDPROC(idmap_cpu_replace_ttbr1) .popsection +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + .pushsection ".idmap.text", "ax" + + .macro __idmap_kpti_get_pgtable_ent, type + dc cvac, cur_\()\type\()p // Ensure any existing dirty + dmb sy // lines are written back before + ldr \type, [cur_\()\type\()p] // loading the entry + tbz \type, #0, next_\()\type // Skip invalid entries + .endm + + .macro __idmap_kpti_put_pgtable_ent_ng, type + orr \type, \type, #PTE_NG // Same bit for blocks and pages + str \type, [cur_\()\type\()p] // Update the entry and ensure it + dc civac, cur_\()\type\()p // is visible to all CPUs. + .endm + +/* + * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper) + * + * Called exactly once from stop_machine context by each CPU found during boot. + */ +__idmap_kpti_flag: + .long 1 +ENTRY(idmap_kpti_install_ng_mappings) + cpu .req w0 + num_cpus .req w1 + swapper_pa .req x2 + swapper_ttb .req x3 + flag_ptr .req x4 + cur_pgdp .req x5 + end_pgdp .req x6 + pgd .req x7 + cur_pudp .req x8 + end_pudp .req x9 + pud .req x10 + cur_pmdp .req x11 + end_pmdp .req x12 + pmd .req x13 + cur_ptep .req x14 + end_ptep .req x15 + pte .req x16 + + mrs swapper_ttb, ttbr1_el1 + adr flag_ptr, __idmap_kpti_flag + + cbnz cpu, __idmap_kpti_secondary + + /* We're the boot CPU. Wait for the others to catch up */ + sevl +1: wfe + ldaxr w18, [flag_ptr] + eor w18, w18, num_cpus + cbnz w18, 1b + + /* We need to walk swapper, so turn off the MMU. */ + mrs x18, sctlr_el1 + bic x18, x18, #SCTLR_ELx_M + msr sctlr_el1, x18 + isb + + /* Everybody is enjoying the idmap, so we can rewrite swapper. */ + /* PGD */ + mov cur_pgdp, swapper_pa + add end_pgdp, cur_pgdp, #(PTRS_PER_PGD * 8) +do_pgd: __idmap_kpti_get_pgtable_ent pgd + tbnz pgd, #1, walk_puds + __idmap_kpti_put_pgtable_ent_ng pgd +next_pgd: + add cur_pgdp, cur_pgdp, #8 + cmp cur_pgdp, end_pgdp + b.ne do_pgd + + /* Publish the updated tables and nuke all the TLBs */ + dsb sy + tlbi vmalle1is + dsb ish + isb + + /* We're done: fire up the MMU again */ + mrs x18, sctlr_el1 + orr x18, x18, #SCTLR_ELx_M + msr sctlr_el1, x18 + isb + + /* Set the flag to zero to indicate that we're all done */ + str wzr, [flag_ptr] + ret + + /* PUD */ +walk_puds: + .if CONFIG_PGTABLE_LEVELS > 3 + pte_to_phys cur_pudp, pgd + add end_pudp, cur_pudp, #(PTRS_PER_PUD * 8) +do_pud: __idmap_kpti_get_pgtable_ent pud + tbnz pud, #1, walk_pmds + __idmap_kpti_put_pgtable_ent_ng pud +next_pud: + add cur_pudp, cur_pudp, 8 + cmp cur_pudp, end_pudp + b.ne do_pud + b next_pgd + .else /* CONFIG_PGTABLE_LEVELS <= 3 */ + mov pud, pgd + b walk_pmds +next_pud: + b next_pgd + .endif + + /* PMD */ +walk_pmds: + .if CONFIG_PGTABLE_LEVELS > 2 + pte_to_phys cur_pmdp, pud + add end_pmdp, cur_pmdp, #(PTRS_PER_PMD * 8) +do_pmd: __idmap_kpti_get_pgtable_ent pmd + tbnz pmd, #1, walk_ptes + __idmap_kpti_put_pgtable_ent_ng pmd +next_pmd: + add cur_pmdp, cur_pmdp, #8 + cmp cur_pmdp, end_pmdp + b.ne do_pmd + b next_pud + .else /* CONFIG_PGTABLE_LEVELS <= 2 */ + mov pmd, pud + b walk_ptes +next_pmd: + b next_pud + .endif + + /* PTE */ +walk_ptes: + pte_to_phys cur_ptep, pmd + add end_ptep, cur_ptep, #(PTRS_PER_PTE * 8) +do_pte: __idmap_kpti_get_pgtable_ent pte + __idmap_kpti_put_pgtable_ent_ng pte +next_pte: + add cur_ptep, cur_ptep, #8 + cmp cur_ptep, end_ptep + b.ne do_pte + b next_pmd + + /* Secondary CPUs end up here */ +__idmap_kpti_secondary: + /* Uninstall swapper before surgery begins */ + __idmap_cpu_set_reserved_ttbr1 x18, x17 + + /* Increment the flag to let the boot CPU we're ready */ +1: ldxr w18, [flag_ptr] + add w18, w18, #1 + stxr w17, w18, [flag_ptr] + cbnz w17, 1b + + /* Wait for the boot CPU to finish messing around with swapper */ + sevl +1: wfe + ldxr w18, [flag_ptr] + cbnz w18, 1b + + /* All done, act like nothing happened */ + msr ttbr1_el1, swapper_ttb + isb + ret + + .unreq cpu + .unreq num_cpus + .unreq swapper_pa + .unreq swapper_ttb + .unreq flag_ptr + .unreq cur_pgdp + .unreq end_pgdp + .unreq pgd + .unreq cur_pudp + .unreq end_pudp + .unreq pud + .unreq cur_pmdp + .unreq end_pmdp + .unreq pmd + .unreq cur_ptep + .unreq end_ptep + .unreq pte +ENDPROC(idmap_kpti_install_ng_mappings) + .popsection +#endif + /* * __cpu_setup * From patchwork Wed Feb 28 03:56:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129911 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp569696lja; Tue, 27 Feb 2018 20:00:57 -0800 (PST) X-Google-Smtp-Source: AG47ELtz81LK9QxhMQCGOrUSckPefR7YZ59TkCn4Vbu28kpdXqEEP2JevSQZNdSUnOqjqb4iyh8m X-Received: by 10.101.75.18 with SMTP id r18mr7023775pgq.36.1519790457061; Tue, 27 Feb 2018 20:00:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790457; cv=none; d=google.com; s=arc-20160816; b=qU+LrhsooK3K544Ojmh6tTs9PVosiPWlZTSfij1jJLx58wqkI1B+Kfw0s9AMUTPXao nJXc1BWDXYGwLjWSDRZlzuOtgp5MZQb1CLAxC1Ooaj+fKw34EgEZcr05gjsVFJ/pwmD1 4MbL/KxXSfOdL3KMrSwWjaIbXpbOHTFOXmnAqSiQmLd9ULb+lh06evXDe/WTPMoUS8rV hmlOMhs4fE9oPe2vFQKt6ZVhly/3RLDlFv8k9xomu3wNsq4he9L66SzDhpPH6nUppdUS iQ18sWEqAPSyjoWQJrNkiUac92uFA4UejT20UkZXLbCWLtJdmiyqmbGOEd4E0yV3oYip zcCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=61Y8iHNGS/LyzooAj6849j03LY7srEwVDjkJR8E93VM=; b=K5K79XLuCgemYGMVpWuR8Z94Kiu5RfuvkT0XxxoA5sUh4PxgMxYXUN5Jb8ygPJQ7z6 +d3qnuQ754htidjTLSfyI7Fmh6kVIvGXngMxGvccMZacnimsbjxQ9t0ictRKJkV85t2T WFPTNylFWNCMh15nZBW1gyzANyxhMTssmvzOBvCH3Tu16038Dtt0VNKXjWWoE7///FvL D7MIC1/00bZNTZPW/Ca+37I/FRYVHXjpjuhLlR70e9ek11c2cOG19anKORw47AdgR/Q/ wdN/tvPDJ6Jl40qSfB4WJQR65T7BEmAtsq42Yznt0Tkm9p/kgJbTG/rCEkDC26STjcwp cGcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dNmpN4p3; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c5-v6si604191pll.90.2018.02.27.20.00.56; Tue, 27 Feb 2018 20:00:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dNmpN4p3; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752062AbeB1EAw (ORCPT + 28 others); Tue, 27 Feb 2018 23:00:52 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:43232 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752404AbeB1EAu (ORCPT ); Tue, 27 Feb 2018 23:00:50 -0500 Received: by mail-pg0-f66.google.com with SMTP id e9so479698pgs.10 for ; Tue, 27 Feb 2018 20:00:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=61Y8iHNGS/LyzooAj6849j03LY7srEwVDjkJR8E93VM=; b=dNmpN4p3DuNhT6OkTUsJidqsJqV3/2TZy78JIsvqzoZCZPfgInieDltAXrRq73IKJ4 KH5muFiLLSY8CKHCXpg5y2rVFDprDz0D0QLZGjzlYQIKvgMloEgruc5z//80VK/friq7 i6aVzoOHY2ggfaUy3cRdXoqM2Zvwoh4Eb+/bM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=61Y8iHNGS/LyzooAj6849j03LY7srEwVDjkJR8E93VM=; b=O6t1HDOMvkh5Rx0nLVWmYWlCqHUKxl+UpbwXfFto/ikGqIvwkH7MM0qzvJvBsv4/AX ALwnqBCSM4/CMjlyERgJTJxmU/RQDGbNaDFLMMbJH/vb1RCvP+7IV2LRPdwWl9RgUOrN qTGHVqNugIkoxMb9PY0zXAeOx0pm1Weh7o/MLrHc7mGCQhZWZ7f+W9tPawg0uGpJwAeP wrWwWCt3/OSFf9OcCLs3f9nJBoVfNecA+FKplGaXvgFlKBBJfRCDZXQS5+xfFRuwKENI UNAlDdWwJP3+llEikTwcuUZ7zWheaBxGLHiiM4tLLMZ/F9CvINhpnydKz01PunLKhzdx lCUQ== X-Gm-Message-State: APf1xPCLxxz1DurGxHvlqr96nvEInOJ8FFqVUd6P9uof7Ihc1HPFDHL4 kR9kuVlH7v1CVQTPigLyzGkZOA== X-Received: by 10.98.89.156 with SMTP id k28mr16092279pfj.130.1519790449826; Tue, 27 Feb 2018 20:00:49 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.20.00.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 20:00:49 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 27/29] arm64: Force KPTI to be disabled on Cavium ThunderX Date: Wed, 28 Feb 2018 11:56:49 +0800 Message-Id: <1519790211-16582-28-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 6dc52b15c4a4 upstream. Cavium ThunderX's erratum 27456 results in a corruption of icache entries that are loaded from memory that is mapped as non-global (i.e. ASID-tagged). As KPTI is based on memory being mapped non-global, let's prevent it from kicking in if this erratum is detected. Signed-off-by: Marc Zyngier [will: Update comment] Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: using old function read_system_reg/cpus_have_cap to replace read_sanitised_ftr_reg/cpus_have_const_cap in arch/arm64/kernel/cpufeature.c --- arch/arm64/kernel/cpufeature.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 0fba610..760c851 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -753,12 +753,23 @@ static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, int __unused) { + char const *str = "command line option"; u64 pfr0 = read_system_reg(SYS_ID_AA64PFR0_EL1); - /* Forced on command line? */ + /* + * For reasons that aren't entirely clear, enabling KPTI on Cavium + * ThunderX leads to apparent I-cache corruption of kernel text, which + * ends as well as you might imagine. Don't even try. + */ + if (cpus_have_cap(ARM64_WORKAROUND_CAVIUM_27456)) { + str = "ARM64_WORKAROUND_CAVIUM_27456"; + __kpti_forced = -1; + } + + /* Forced? */ if (__kpti_forced) { - pr_info_once("kernel page table isolation forced %s by command line option\n", - __kpti_forced > 0 ? "ON" : "OFF"); + pr_info_once("kernel page table isolation forced %s by %s\n", + __kpti_forced > 0 ? "ON" : "OFF", str); return __kpti_forced > 0; } From patchwork Wed Feb 28 03:56:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129912 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp569810lja; Tue, 27 Feb 2018 20:01:07 -0800 (PST) X-Google-Smtp-Source: AG47ELthaYlVfMqHrqMetpOG1yh3vwukFPp37zRxV0p6gRhF/4x/9I3KxDGi7atjRCIFYDzzQgXi X-Received: by 10.99.178.6 with SMTP id x6mr5366524pge.98.1519790466898; Tue, 27 Feb 2018 20:01:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790466; cv=none; d=google.com; s=arc-20160816; b=iyDx+bm2PTJg7TFgJptNjpC5mJJa0uEjzNswhohCdoyZnHlClfduEvK42wm9fvdEvR Gqj1sUpwY0chPucgaUmFjKJsxqNZwb/JuLlLJeh0Jg2kQPP7kyJdb+f+Ziq7sbmRqeUf JI15xjt1velO9cBNYPKSFKyq0DDVRq5RB2uMAOcc8XZE4vEseLJCv/cZ55MK5mpW9pYh xL9tgG2+Z0D3cVpEHZjmnM+T3N2s7tWRsfHoRr/Ty0Gah1n121aRiSWWWeorRsNcZZ2t FoWZ+3Y6k7t44SVLY/rsnGrfoPSacXbCxmhZNuFXIM45FxyLMZBBNW1Rv/kl2lCOoyon gY9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=mDAP47eyIT68Z920skzZ0tbKXBprTWayJOiSVpV8SAg=; b=RjS6QOq0RhfAfvmX/Fm7S0CdVpIoGpHv2gqDZhvJHQlyOSj+FbMz5geSIAd1IvlN58 RUKGsGdnCBOS4gl/fz/K5dEILgyjU2JYL+vahuM8Q/xjb2D/BRmMYTGTKlUP3rjU+thd 6/QLRKM3nO1YlDOTRAaABLHoirhBj2GMrFsN7AAJjEXnnXjXwY5g0mhe7DaaWudYN8xm LdvbqrcUsisgv/zO3xZUPNsq6rciB1MWQNhXlozecSrC6J8YYSzV05Xf4XZj9Kn2z2kN kspHib4eOpY8cED6UfZUsfaMfG625Tpk11aokkz43GnwAuR3gL2OFVmiKrMyeHigzdzk 76xA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Js+guy+h; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c5-v6si604191pll.90.2018.02.27.20.01.06; Tue, 27 Feb 2018 20:01:06 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Js+guy+h; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752444AbeB1EBD (ORCPT + 28 others); Tue, 27 Feb 2018 23:01:03 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:45545 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752422AbeB1EBA (ORCPT ); Tue, 27 Feb 2018 23:01:00 -0500 Received: by mail-pg0-f67.google.com with SMTP id i133so477814pgc.12 for ; Tue, 27 Feb 2018 20:01:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mDAP47eyIT68Z920skzZ0tbKXBprTWayJOiSVpV8SAg=; b=Js+guy+hMKxmFG4/egEFD9meAto9cmYTH9gXJ1RnPYpBfiYmD0SrrjfzxT5+f23OkG 7K5BnvIU1LwjqR5x8WJrR/d1p1zu0eQWKQU5UJklQ53L6a7eFFkOQSOpUYXL3gyrhDmQ 5r4HMTDCrs7xZLGrCsZmR56s0vla1r1u8kxys= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mDAP47eyIT68Z920skzZ0tbKXBprTWayJOiSVpV8SAg=; b=WE4mdrI3TzvFdZxZNBqBPi5/twHdCzBm5BnwghVv/HpIHjyaMbMZunhX0QKdYr/XS9 XLG3ZRhCswROlUYkPTwgtOyTVRofaSph348F78BeDzhBrZqevdhdAIODw0OGr/nZzyvG RbJGE9o3uiLEMR9e54ZCRhsrZ2kSghXYpHwqctdHgF7ywqcli3em9kw4zd9cxjMMzk57 eRtyEktiowG66bDPF/dm8qa3IROZG2EVzfkuWTUQ/ndLvbNxdHLtlJiY2zO15fSGiXOG kWeF8bYPBt9VCntjIHE1hBopX4skIyLjLKxpNENnPXwlM5spYcnQqD+ALObdA2QMkfI7 YI8A== X-Gm-Message-State: APf1xPDlW5WDEeX5lgBTOF3dzKyt0bK9QWqNGN0xC/azorSM9XfdJ4x2 ZXi3OgZ4RMmwox/saLer28/ofQ== X-Received: by 10.98.237.12 with SMTP id u12mr4523070pfh.72.1519790460145; Tue, 27 Feb 2018 20:01:00 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.20.00.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 20:00:59 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 28/29] arm64: entry: Reword comment about post_ttbr_update_workaround Date: Wed, 28 Feb 2018 11:56:50 +0800 Message-Id: <1519790211-16582-29-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit f167211a93ac upstream. We don't fully understand the Cavium ThunderX erratum, but it appears that mapping the kernel as nG can lead to horrible consequences such as attempting to execute userspace from kernel context. Since kpti isn't enabled for these CPUs anyway, simplify the comment justifying the lack of post_ttbr_update_workaround in the exception trampoline. Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/kernel/entry.S | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index c00921e..8d50b28 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -883,16 +883,9 @@ __ni_sys_trace: orr \tmp, \tmp, #USER_ASID_FLAG msr ttbr1_el1, \tmp /* - * We avoid running the post_ttbr_update_workaround here because the - * user and kernel ASIDs don't have conflicting mappings, so any - * "blessing" as described in: - * - * http://lkml.kernel.org/r/56BB848A.6060603@caviumnetworks.com - * - * will not hurt correctness. Whilst this may partially defeat the - * point of using split ASIDs in the first place, it avoids - * the hit of invalidating the entire I-cache on every return to - * userspace. + * We avoid running the post_ttbr_update_workaround here because + * it's only needed by Cavium ThunderX, which requires KPTI to be + * disabled. */ .endm From patchwork Wed Feb 28 03:56:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129913 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp569931lja; Tue, 27 Feb 2018 20:01:15 -0800 (PST) X-Google-Smtp-Source: AH8x224swO4iao4HUVwJZkPL6A+K4BEk6DNc1jGpNKFV9dvnIM6Cbk6tKlzmBUJZnrbOytqRoOYS X-Received: by 10.99.61.75 with SMTP id k72mr12732477pga.384.1519790475487; Tue, 27 Feb 2018 20:01:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519790475; cv=none; d=google.com; s=arc-20160816; b=0Z2TIEigeSvXSvXbtNAcBRWi5dRNxtQZuNyY0N7+H7fgSGQTous/aIaBekIoPCcjYt FCg1Rjl3S0TAUhFXAUDRJ5+WJkgETa9uAoNLe04bb9F3D6zK3cMs6M+sNfjM6VLxeJcS ZvG3pSYp6NP0hNtTodZZnkDwe4HrdVhU2uBUVLfQssLquJ7Uxl8td5yI4poym5pFGrW+ o1fMQsy6Wq8iEbF7d3piHBP07SRq5A6kFuK2zE1tjKuGov90Eju+Zwl9SzIH4Ha9i91n DXVEDq80eq8Wr+XoW9GPTMlyfRDjTPnCrvtaDI88az45OkDi6zwDao1Jtbm0ThpWLSHw b57A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=bmODirOIrB4Uq31RHiBfVSkaHJATLnMI0vQEAhunKG8=; b=W03fbo2MsOosTqq8OxkWGBU81T9b5IqkNctmuZARUWCqrI5qXjGlBp8oGw84a+pwJJ wD74MLCJhXJkRHzIrFyj2sfns9Gc9Ta1ztMEY+Z2iay0QciwDWFJK1SKnuMK2cAZmNB6 Jc27scO2oBZb2tNgDvjVMtZDsyPeeHENF/lk6mhUnOJ3QZTYADyvCggY1NMs+2WdBWcN exqFhKxIHH3auPS9/N6jclBSKL9s9iVFgRBgCiVcOmzBuQrZQmdBNDr9NX1bciAcW0eT 4gNm65v6hClczDjFqVQGhK6nx3JfEWKFrYjZ3SCjB10GRsWLLWgfUSdguaA9Vvw92eq0 2iqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OFGVOi3h; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c5-v6si604191pll.90.2018.02.27.20.01.15; Tue, 27 Feb 2018 20:01:15 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OFGVOi3h; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752461AbeB1EBM (ORCPT + 28 others); Tue, 27 Feb 2018 23:01:12 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:46724 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752235AbeB1EBK (ORCPT ); Tue, 27 Feb 2018 23:01:10 -0500 Received: by mail-pg0-f66.google.com with SMTP id r26so480310pgv.13 for ; Tue, 27 Feb 2018 20:01:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bmODirOIrB4Uq31RHiBfVSkaHJATLnMI0vQEAhunKG8=; b=OFGVOi3he6LzbyCn520fy0htDXYWk4kbNgIkyA8pCCclqXIPvz0RdVaYJZUfuMjKOP pSO0DUNMTxa1Z6BfQfxCctG2TL/hpA87CwrHZKXJUkkuf4bUk1QP9XwypRJWvzYFZEyR waXAonKRxDizWyqDhU4oyjXBQNKTBjUZJQuWI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bmODirOIrB4Uq31RHiBfVSkaHJATLnMI0vQEAhunKG8=; b=binrZ7uEZf1lkmoA3QVNwvA3sjtTKyZO0mkmm2ZwZ6H3LbWRpd2Y4C035EZvDwDl0p ZP8KtPwOVHDtQR7tZmAifFQbkflluoWiGF0102pkQtwniFMuRh8FTgleSC/k0ukFH47j 7e/8VnUOeNV2GkLAN1OcCADI7JgoMhK2myuc1uuBz+Xe1FvGEjB57+/ocKQXJMEFd1uW 6oTjEnFdx/zEdaKampsmif+1fU2lPF4dsQGDleO19BcJy5GJQDHwgCMJamSDo0fAUsV1 +qwnopD9ZNOEB3RSIsmS4v770B0KbF10r44QhqAv6DPF7X6hh5zIzYC8T+VjGiksmPMn 1osw== X-Gm-Message-State: APf1xPCQij4wR393IvaYUidSk2vazIDiKDDZaRauuXqHMEzuh1OBGnhW 82S0jLtS+2wUM6Dv6Y6tsCAXSBAhYh8= X-Received: by 10.98.202.138 with SMTP id y10mr16237682pfk.184.1519790469719; Tue, 27 Feb 2018 20:01:09 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id q17sm739911pgt.7.2018.02.27.20.01.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 27 Feb 2018 20:01:09 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 29/29] arm64: idmap: Use "awx" flags for .idmap.text .pushsection directives Date: Wed, 28 Feb 2018 11:56:51 +0800 Message-Id: <1519790211-16582-30-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> References: <1519790211-16582-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 439e70e27a51 upstream. The identity map is mapped as both writeable and executable by the SWAPPER_MM_MMUFLAGS and this is relied upon by the kpti code to manage a synchronisation flag. Update the .pushsection flags to reflect the actual mapping attributes. Reported-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/kernel/cpu-reset.S | 2 +- arch/arm64/kernel/head.S | 2 +- arch/arm64/kernel/sleep.S | 2 +- arch/arm64/mm/proc.S | 8 ++++---- 4 files changed, 7 insertions(+), 7 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index 65f42d2..f736a6f 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -16,7 +16,7 @@ #include .text -.pushsection .idmap.text, "ax" +.pushsection .idmap.text, "awx" /* * __cpu_soft_restart(el2_switch, entry, arg0, arg1, arg2) - Helper for diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 539bebc..fa52817 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -473,7 +473,7 @@ ENDPROC(__primary_switched) * end early head section, begin head code that is also used for * hotplug and needs to have the same protections as the text region */ - .section ".idmap.text","ax" + .section ".idmap.text","awx" ENTRY(kimage_vaddr) .quad _text - TEXT_OFFSET diff --git a/arch/arm64/kernel/sleep.S b/arch/arm64/kernel/sleep.S index 1bec41b..0030d69 100644 --- a/arch/arm64/kernel/sleep.S +++ b/arch/arm64/kernel/sleep.S @@ -95,7 +95,7 @@ ENTRY(__cpu_suspend_enter) ret ENDPROC(__cpu_suspend_enter) - .pushsection ".idmap.text", "ax" + .pushsection ".idmap.text", "awx" ENTRY(cpu_resume) bl el2_setup // if in EL2 drop to EL1 cleanly bl __cpu_setup diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 5c268f5..c07d9cc 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -83,7 +83,7 @@ ENDPROC(cpu_do_suspend) * * x0: Address of context pointer */ - .pushsection ".idmap.text", "ax" + .pushsection ".idmap.text", "awx" ENTRY(cpu_do_resume) ldp x2, x3, [x0] ldp x4, x5, [x0, #16] @@ -147,7 +147,7 @@ alternative_else_nop_endif ret ENDPROC(cpu_do_switch_mm) - .pushsection ".idmap.text", "ax" + .pushsection ".idmap.text", "awx" .macro __idmap_cpu_set_reserved_ttbr1, tmp1, tmp2 adrp \tmp1, empty_zero_page @@ -180,7 +180,7 @@ ENDPROC(idmap_cpu_replace_ttbr1) .popsection #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 - .pushsection ".idmap.text", "ax" + .pushsection ".idmap.text", "awx" .macro __idmap_kpti_get_pgtable_ent, type dc cvac, cur_\()\type\()p // Ensure any existing dirty @@ -368,7 +368,7 @@ ENDPROC(idmap_kpti_install_ng_mappings) * Initialise the processor for turning the MMU on. Return in x0 the * value of the SCTLR_EL1 register. */ - .pushsection ".idmap.text", "ax" + .pushsection ".idmap.text", "awx" ENTRY(__cpu_setup) tlbi vmalle1 // Invalidate local TLB dsb nsh