From patchwork Fri Nov 17 18:21:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119194 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp843501qgn; Fri, 17 Nov 2017 10:22:26 -0800 (PST) X-Google-Smtp-Source: AGs4zMarP9bwd04MS5QaM7O3GMAZLZ1R6SQIvrhSCPtGHcdyV1HvDYnlqwkPnfL9BYLXS4vT4SH/ X-Received: by 10.98.46.7 with SMTP id u7mr2957429pfu.37.1510942945928; Fri, 17 Nov 2017 10:22:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510942945; cv=none; d=google.com; s=arc-20160816; b=LuYJ9Ow3MjENdwDvtWg8eafYBLvye19FplmtD80JIEuOQ3n3RljpY51XX4shYgRPhj qIAY9t9jV46dHG7SLW4oIX5gNNCdr8pfRAO53CeEJt5jRP53sSpHSWLYGc/oaWd+5A0p /JHxaRQlcz6Msu2y0gYlzaWlJ8hiaBPLuo4uuug8Za+k4D5QJ/m0oYkg96OjwleSwctD bK4rW8ctD8A0XxB5VNHC4OMUAerSdZrgH5KcCeI6QljaVaCYgRFCfNwhgHL1gKB+rkLw OfmnBYyYHbTacU/7SSPVodk1hv6tOGgOvgeamGd9wDZnVOPd5cyEIt3k9C/drJr+FCmc Gg/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=bDYAh19mDeUOIt604CmIEM9tIuAbPFgsMJp7ZAGTsiY=; b=rlZQiRvGdeBpdpJ7FOCTCjZmWWcNkHAVllTl44tJ2L7x/AaAE1rHAKYvrX3IvOxU0W qRcLBzpa9ZG/qxnDVD7obUbfyHFet2YANC+zl+O0BwCPnJ1t0oQGnnt6/2pb6R6ughVn E1MMeY3MG/QwUcI0olfF0Ik1Qt+4eM3gSv6ktavez2ZKhHRTfGJdfzK0aR6y7YAGPybJ 7KmOjdqy4tt7lf/GIOmeYM/qd8GoYzp/m0VNOc6xUrHI08h1/2q4KsnGcft0CPYFHo7U mHyCbmYXdzvMrryHnAn6f5hM1634WvegiE5+pgAdUCH38g+j4zIlDL0qA25/zngki8ER mEBQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x5si3079420pgb.437.2017.11.17.10.22.25; Fri, 17 Nov 2017 10:22:25 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1161271AbdKQSWU (ORCPT + 28 others); Fri, 17 Nov 2017 13:22:20 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39288 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760273AbdKQSVy (ORCPT ); Fri, 17 Nov 2017 13:21:54 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 165011596; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DBE493F599; Fri, 17 Nov 2017 10:21:53 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id EB2A31AE10AF; Fri, 17 Nov 2017 18:22:02 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 01/18] arm64: mm: Use non-global mappings for kernel space Date: Fri, 17 Nov 2017 18:21:44 +0000 Message-Id: <1510942921-12564-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation for unmapping the kernel whilst running in userspace, make the kernel mappings non-global so we can avoid expensive TLB invalidation on kernel exit to userspace. Signed-off-by: Will Deacon --- arch/arm64/include/asm/kernel-pgtable.h | 12 ++++++++++-- arch/arm64/include/asm/pgtable-prot.h | 21 +++++++++++++++------ 2 files changed, 25 insertions(+), 8 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index 7803343e5881..77a27af01371 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -78,8 +78,16 @@ /* * Initial memory map attributes. */ -#define SWAPPER_PTE_FLAGS (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) -#define SWAPPER_PMD_FLAGS (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) +#define _SWAPPER_PTE_FLAGS (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) +#define _SWAPPER_PMD_FLAGS (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define SWAPPER_PTE_FLAGS (_SWAPPER_PTE_FLAGS | PTE_NG) +#define SWAPPER_PMD_FLAGS (_SWAPPER_PMD_FLAGS | PMD_SECT_NG) +#else +#define SWAPPER_PTE_FLAGS _SWAPPER_PTE_FLAGS +#define SWAPPER_PMD_FLAGS _SWAPPER_PMD_FLAGS +#endif #if ARM64_SWAPPER_USES_SECTION_MAPS #define SWAPPER_MM_MMUFLAGS (PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS) diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 0a5635fb0ef9..22a926825e3f 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -34,8 +34,16 @@ #include -#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) -#define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) +#define _PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) +#define _PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define PROT_DEFAULT (_PROT_DEFAULT | PTE_NG) +#define PROT_SECT_DEFAULT (_PROT_SECT_DEFAULT | PMD_SECT_NG) +#else +#define PROT_DEFAULT _PROT_DEFAULT +#define PROT_SECT_DEFAULT _PROT_SECT_DEFAULT +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ #define PROT_DEVICE_nGnRnE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRnE)) #define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRE)) @@ -48,6 +56,7 @@ #define PROT_SECT_NORMAL_EXEC (PROT_SECT_DEFAULT | PMD_SECT_UXN | PMD_ATTRINDX(MT_NORMAL)) #define _PAGE_DEFAULT (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL)) +#define _HYP_PAGE_DEFAULT (_PAGE_DEFAULT & ~PTE_NG) #define PAGE_KERNEL __pgprot(_PAGE_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE) #define PAGE_KERNEL_RO __pgprot(_PAGE_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_RDONLY) @@ -55,15 +64,15 @@ #define PAGE_KERNEL_EXEC __pgprot(_PAGE_DEFAULT | PTE_UXN | PTE_DIRTY | PTE_WRITE) #define PAGE_KERNEL_EXEC_CONT __pgprot(_PAGE_DEFAULT | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_CONT) -#define PAGE_HYP __pgprot(_PAGE_DEFAULT | PTE_HYP | PTE_HYP_XN) -#define PAGE_HYP_EXEC __pgprot(_PAGE_DEFAULT | PTE_HYP | PTE_RDONLY) -#define PAGE_HYP_RO __pgprot(_PAGE_DEFAULT | PTE_HYP | PTE_RDONLY | PTE_HYP_XN) +#define PAGE_HYP __pgprot(_HYP_PAGE_DEFAULT | PTE_HYP | PTE_HYP_XN) +#define PAGE_HYP_EXEC __pgprot(_HYP_PAGE_DEFAULT | PTE_HYP | PTE_RDONLY) +#define PAGE_HYP_RO __pgprot(_HYP_PAGE_DEFAULT | PTE_HYP | PTE_RDONLY | PTE_HYP_XN) #define PAGE_HYP_DEVICE __pgprot(PROT_DEVICE_nGnRE | PTE_HYP) #define PAGE_S2 __pgprot(PROT_DEFAULT | PTE_S2_MEMATTR(MT_S2_NORMAL) | PTE_S2_RDONLY) #define PAGE_S2_DEVICE __pgprot(PROT_DEFAULT | PTE_S2_MEMATTR(MT_S2_DEVICE_nGnRE) | PTE_S2_RDONLY | PTE_UXN) -#define PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_VALID) | PTE_PROT_NONE | PTE_RDONLY | PTE_PXN | PTE_UXN) +#define PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_VALID) | PTE_PROT_NONE | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN) #define PAGE_SHARED __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE) #define PAGE_SHARED_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_WRITE) #define PAGE_READONLY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN) From patchwork Fri Nov 17 18:21:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119195 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp843696qgn; Fri, 17 Nov 2017 10:22:38 -0800 (PST) X-Google-Smtp-Source: AGs4zMYbV5DhlFOfWYHv4HDPC8xL2zOmD+QZ/QLOpJS2Hd3kmzd5kHZS1o0DJRsi/0LyHB1pKiIr X-Received: by 10.159.252.1 with SMTP id n1mr6022175pls.415.1510942958128; Fri, 17 Nov 2017 10:22:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510942958; cv=none; d=google.com; s=arc-20160816; b=Z+6F0Mw2zENjlrTaZzQAQKTJmRLeJHfhhwwKaxhscc/79Y4wbZ2hWKaWrq+O2MFAAS DUUFkx8ORLyQjBnuS5v1JBXOtnsPmeufqbMuDOrAl2nymuHlhESjnPCcziYAqGBdtoQk 4J4TebLYOsMAJqfP1DM7NScs3FMPxzc6IpngzS4/xA6SIh3wmgExCByMDMmjcy4ypDWT hSBpHeFPFdyfalpl4sJZAY/DCTKCSu1U2DZ8M1+cSCry588WNrVlr3NCdlx3y6Q4D/kY XOdstdshgZTFryGGkJnH4VhoJn3b2UyNjujDxewk4/ranXawwcNKQgQFnV1jwjSw9gcY qAKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=FC0EjtZGpmqUEOVEdjfbY49Aq+9DDmM38+aEoqOkXH0=; b=bRcgw2o/PU/RrH2yqxXI2K/QzQzF4hAYd3KfhczYADVT52gkvBYo4dBNHazJri5S/r +8EulhcDZ2Xtx4AYFQSpH/QBsf8VhAq+Dh3lTO/4lHG5CzKr42fLtRi83eHHijrW3Rvd P+zWfHUm3jAqTxIvvwdjvX0KguedV0SV6LQl/vP9GKpwdThSMjwPMjlH2v/TXxE8Cwez B9iTG43VE7I7lHpRS9qbUSFi30OCrgubzXQffUoUr5nMScbfJj3DyY3nx06GHQJrZ6mu CyIr8Ye/oIk5K3YImKnwVC+4aXiYlJvG9rCVmAiYeVlhy6rRVXdjvVU1qRzfORMfaRba jDOA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x5si3079420pgb.437.2017.11.17.10.22.37; Fri, 17 Nov 2017 10:22:38 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1161277AbdKQSWg (ORCPT + 28 others); Fri, 17 Nov 2017 13:22:36 -0500 Received: from foss.arm.com ([217.140.101.70]:39296 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760276AbdKQSVy (ORCPT ); Fri, 17 Nov 2017 13:21:54 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 234C615BF; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E85993F5A0; Fri, 17 Nov 2017 10:21:53 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 056571AE10C2; Fri, 17 Nov 2017 18:22:02 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 02/18] arm64: mm: Temporarily disable ARM64_SW_TTBR0_PAN Date: Fri, 17 Nov 2017 18:21:45 +0000 Message-Id: <1510942921-12564-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We're about to rework the way ASIDs are allocated, switch_mm is implemented and low-level kernel entry/exit is handled, so keep the ARM64_SW_TTBR0_PAN code out of the way whilst we do the heavy lifting. It will be re-enabled in a subsequent patch. Signed-off-by: Will Deacon --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) -- 2.1.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 0df64a6a56d4..582bbd77c390 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -872,6 +872,7 @@ endif config ARM64_SW_TTBR0_PAN bool "Emulate Privileged Access Never using TTBR0_EL1 switching" + depends on BROKEN # Temporary while switch_mm is reworked help Enabling this option prevents the kernel from accessing user-space memory directly by pointing TTBR0_EL1 to a reserved From patchwork Fri Nov 17 18:21:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119211 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp847065qgn; Fri, 17 Nov 2017 10:26:13 -0800 (PST) X-Google-Smtp-Source: AGs4zMYm708OkWXB83rsrIg/2/fLLr1xrr5nHxkBtrIkZteyK+3fp5vR5vSh83RH7KuTqPU8dg2V X-Received: by 10.101.80.10 with SMTP id f10mr5949821pgo.408.1510943173515; Fri, 17 Nov 2017 10:26:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943173; cv=none; d=google.com; s=arc-20160816; b=W+pVBdm242n03qGRuz8qVFlVM7rXRGyI5tz15/LHh0CfMkwzOOMMpM7r1b7EHGsMc6 Et6GkbJF4kYCjwG0NfHbMbYHlczBCM84Aj0oDFNKui2kykLRPcUg0WqZxIFIadeR+SkW IwKGSaaCKEwJUlcI7bIv5br4Wc75O5AjobYR4PuinOK1/n4TbBsM8wDltdnZsCrjUcrQ W+JERxRMhlP94GL5hRiQ6CMxXNXF3CTKxPSrKYDlKTpVxIu5s8+S1rg5Rklvv3SdWpdf oY6m+ONQe4Se/Cugf5P0AY6sY1T0a+PHCS0cQkNLvNYEYKHavynsHAEA6+LCwK/CRXCf LzyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=jTDRWgyYREJkYO6ZVBLae8Gm6Wg8YvPPSr+hkriqxJo=; b=znec30tiyyXrJ7/yzd2UEVGeJ3xtzOXANIOXuZM3JI6vSsd1cPsJqH8yOEKqDfzN9L Xolpu7ENRuavcCyVb3gIhF1sWAo3WFHo3+H11UQ/4ABpRLPM/iaUbnru6eU9uLr0tptB NMdVrdnRnyK4qmgzho4iMU5ZqaEc2f0XYob90DyyJXjZ4jioMZv+xlLZ5/4fOOAsvnOv T97TFwTb0Wz34QLmDECN6GwSvKx5sx3MDt7SSUhQzc3iRVie4eUsSdQ/2iQh8PrhaTeL wJt5NoBJ2UVC6kxZSeBwufa5Sl6LkCTHMJLL/RuhCH3TghUtSjtpK8WLTzaztMUaTMOY hgSg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v18si3149802pge.275.2017.11.17.10.26.13; Fri, 17 Nov 2017 10:26:13 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760676AbdKQS0K (ORCPT + 28 others); Fri, 17 Nov 2017 13:26:10 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39304 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760277AbdKQSVy (ORCPT ); Fri, 17 Nov 2017 13:21:54 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2E8461610; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 000103F703; Fri, 17 Nov 2017 10:21:53 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 117A91AE1116; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 03/18] arm64: mm: Move ASID from TTBR0 to TTBR1 Date: Fri, 17 Nov 2017 18:21:46 +0000 Message-Id: <1510942921-12564-4-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation for mapping kernelspace and userspace with different ASIDs, move the ASID to TTBR1 and update switch_mm to context-switch TTBR0 via an invalid mapping (the zero page). Signed-off-by: Will Deacon --- arch/arm64/include/asm/mmu_context.h | 7 +++++++ arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/include/asm/proc-fns.h | 6 ------ arch/arm64/mm/proc.S | 9 ++++++--- 4 files changed, 14 insertions(+), 9 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 3257895a9b5e..56723bcbfaaa 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -37,6 +37,13 @@ #include #include +#define cpu_switch_mm(pgd,mm) \ +do { \ + BUG_ON(pgd == swapper_pg_dir); \ + cpu_set_reserved_ttbr0(); \ + cpu_do_switch_mm(virt_to_phys(pgd),mm); \ +} while (0) + static inline void contextidr_thread_switch(struct task_struct *next) { if (!IS_ENABLED(CONFIG_PID_IN_CONTEXTIDR)) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index eb0c2bd90de9..8df4cb6ac6f7 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -272,6 +272,7 @@ #define TCR_TG1_4K (UL(2) << TCR_TG1_SHIFT) #define TCR_TG1_64K (UL(3) << TCR_TG1_SHIFT) +#define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) #define TCR_HA (UL(1) << 39) diff --git a/arch/arm64/include/asm/proc-fns.h b/arch/arm64/include/asm/proc-fns.h index 14ad6e4e87d1..16cef2e8449e 100644 --- a/arch/arm64/include/asm/proc-fns.h +++ b/arch/arm64/include/asm/proc-fns.h @@ -35,12 +35,6 @@ extern u64 cpu_do_resume(phys_addr_t ptr, u64 idmap_ttbr); #include -#define cpu_switch_mm(pgd,mm) \ -do { \ - BUG_ON(pgd == swapper_pg_dir); \ - cpu_do_switch_mm(virt_to_phys(pgd),mm); \ -} while (0) - #endif /* __ASSEMBLY__ */ #endif /* __KERNEL__ */ #endif /* __ASM_PROCFNS_H */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 877d42fb0df6..0bd7550b7230 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -139,9 +139,12 @@ ENDPROC(cpu_do_resume) */ ENTRY(cpu_do_switch_mm) pre_ttbr0_update_workaround x0, x2, x3 + mrs x2, ttbr1_el1 mmid x1, x1 // get mm->context.id - bfi x0, x1, #48, #16 // set the ASID - msr ttbr0_el1, x0 // set TTBR0 + bfi x2, x1, #48, #16 // set the ASID + msr ttbr1_el1, x2 // in TTBR1 (since TCR.A1 is set) + isb + msr ttbr0_el1, x0 // now update TTBR0 isb post_ttbr0_update_workaround ret @@ -225,7 +228,7 @@ ENTRY(__cpu_setup) * both user and kernel. */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ - TCR_TG_FLAGS | TCR_ASID16 | TCR_TBI0 + TCR_TG_FLAGS | TCR_ASID16 | TCR_TBI0 | TCR_A1 tcr_set_idmap_t0sz x10, x9 /* From patchwork Fri Nov 17 18:21:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119210 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp846942qgn; Fri, 17 Nov 2017 10:26:04 -0800 (PST) X-Google-Smtp-Source: AGs4zMbROVtoVajzeFmwLZ0K7XGNpJ4Ev6vtSVorLAa8VJT/9nWgAMgALD4I1D7oR/sr6Le6rjOL X-Received: by 10.101.82.76 with SMTP id q12mr5924990pgp.140.1510943164375; Fri, 17 Nov 2017 10:26:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943164; cv=none; d=google.com; s=arc-20160816; b=kjicHX3hmBnJ1aM18yhRVM8XUCOGZv9L/PTMAxLYn7TYPuF7hXtwwHAgy9E7v6d9aj SxafAqj/W02m8dmG8xfRgt2tIFAmvNwiGO1rTujC5By5n59EI02DtWPu/aeTGXu7k8y1 GmwrAvMhMm5p0qwxHRWVNlbNYwOAo9cf3aOHab8aK6pAGwxSTA2JyA4CO/kCIhcctuTh 7GOFaxzEqDPS7aB4neeAPFOF1ywcT88Q4NXub9bk7xR5FZP4ZooNwLXXMkFCMTwKxZ1r CVkKzUpDKEQZUsWTLKUmEAAMUD0OJBnlelCs4BL+iM/HtSmyqhk9Zx6r5ZVkTwlvBNTz TEVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=PDA82ICLKTd0Jvo9IkJ1FTAUSgsrbCCCltt0pqKUFWk=; b=JN9B/8pg+cg1IeY6OTtdJOqf1T+P+3I2BzVhFOGlASh26+VEx+7qg84+2cRAtQ3mB4 lKls9JuhpZqaV1CEpbcE7qWrRdW7Hh/naYPDHTozcTxU8AGsM2i56Y/V5dKzkWl52QFt thakKmwb1XwcVPBgwmRvTFAhnryFJt1R8U3oKCJDuZMeI3BJWMI88ElUdLpDB4SV6s9p oTsxp5dD9P+hIIIQG1huYKtFI2KwIhJUVaobsw7mTEhlWxtoFQ0J9k7ghKKfzrVClrtU JRuRStn/ijEjOBbSQtjyUueEKxdglRna1hxPhivfAfkDbXkr01YePPx4abRdXFGCMq57 V3NA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 75si3462685pfz.240.2017.11.17.10.26.04; Fri, 17 Nov 2017 10:26:04 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760639AbdKQS0A (ORCPT + 28 others); Fri, 17 Nov 2017 13:26:00 -0500 Received: from foss.arm.com ([217.140.101.70]:39332 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760278AbdKQSVy (ORCPT ); Fri, 17 Nov 2017 13:21:54 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3C9BE164F; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0E3713F246; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 1DB3D1AE1185; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 04/18] arm64: mm: Remove pre_ttbr0_update_workaround for Falkor erratum #E1003 Date: Fri, 17 Nov 2017 18:21:47 +0000 Message-Id: <1510942921-12564-5-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The pre_ttbr0_update_workaround hook is called prior to context-switching TTBR0 because Falkor erratum E1003 can cause TLB allocation with the wrong ASID if both the ASID and the base address of the TTBR are updated at the same time. With the ASID sitting safely in TTBR1, we no longer update things atomically, so we can remove the pre_ttbr0_update_workaround macro as it's no longer required. The erratum infrastructure and documentation is left around for #E1003, as it will be required by the entry trampoline code in a future patch. Signed-off-by: Will Deacon --- arch/arm64/include/asm/assembler.h | 22 ---------------------- arch/arm64/include/asm/mmu_context.h | 2 -- arch/arm64/mm/context.c | 11 ----------- arch/arm64/mm/proc.S | 1 - 4 files changed, 36 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index d58a6253c6ab..8359148858cb 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -25,7 +25,6 @@ #include #include -#include #include #include #include @@ -465,27 +464,6 @@ alternative_endif .endm /* - * Errata workaround prior to TTBR0_EL1 update - * - * val: TTBR value with new BADDR, preserved - * tmp0: temporary register, clobbered - * tmp1: other temporary register, clobbered - */ - .macro pre_ttbr0_update_workaround, val, tmp0, tmp1 -#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003 -alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003 - mrs \tmp0, ttbr0_el1 - mov \tmp1, #FALKOR_RESERVED_ASID - bfi \tmp0, \tmp1, #48, #16 // reserved ASID + old BADDR - msr ttbr0_el1, \tmp0 - isb - bfi \tmp0, \val, #0, #48 // reserved ASID + new BADDR - msr ttbr0_el1, \tmp0 - isb -alternative_else_nop_endif -#endif - .endm - /* * Errata workaround post TTBR0_EL1 update. */ diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 56723bcbfaaa..6d93bd545906 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -19,8 +19,6 @@ #ifndef __ASM_MMU_CONTEXT_H #define __ASM_MMU_CONTEXT_H -#define FALKOR_RESERVED_ASID 1 - #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index ab9f5f0fb2c7..78816e476491 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -79,13 +79,6 @@ void verify_cpu_asid_bits(void) } } -static void set_reserved_asid_bits(void) -{ - if (IS_ENABLED(CONFIG_QCOM_FALKOR_ERRATUM_1003) && - cpus_have_const_cap(ARM64_WORKAROUND_QCOM_FALKOR_E1003)) - __set_bit(FALKOR_RESERVED_ASID, asid_map); -} - static void flush_context(unsigned int cpu) { int i; @@ -94,8 +87,6 @@ static void flush_context(unsigned int cpu) /* Update the list of reserved ASIDs and the ASID bitmap. */ bitmap_clear(asid_map, 0, NUM_USER_ASIDS); - set_reserved_asid_bits(); - /* * Ensure the generation bump is observed before we xchg the * active_asids. @@ -250,8 +241,6 @@ static int asids_init(void) panic("Failed to allocate bitmap for %lu ASIDs\n", NUM_USER_ASIDS); - set_reserved_asid_bits(); - pr_info("ASID allocator initialised with %lu entries\n", NUM_USER_ASIDS); return 0; } diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 0bd7550b7230..1623150ed0a6 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -138,7 +138,6 @@ ENDPROC(cpu_do_resume) * - pgd_phys - physical address of new TTB */ ENTRY(cpu_do_switch_mm) - pre_ttbr0_update_workaround x0, x2, x3 mrs x2, ttbr1_el1 mmid x1, x1 // get mm->context.id bfi x2, x1, #48, #16 // set the ASID From patchwork Fri Nov 17 18:21:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119209 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp846766qgn; Fri, 17 Nov 2017 10:25:53 -0800 (PST) X-Google-Smtp-Source: AGs4zMZb47hZcr4gStX6kIrt04p5q0/vpA6TeRE5Seet3omOPL7XLw+GJXb662WTbJCTZXYm4OPR X-Received: by 10.98.186.13 with SMTP id k13mr2947258pff.166.1510943153044; Fri, 17 Nov 2017 10:25:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943153; cv=none; d=google.com; s=arc-20160816; b=YtuRlhzBSIDhlAh7ldVLDox8R9KUu+rYNR8spq5c34RVc12MH8dK0T5fb3TDehULCo s1sazm/LE6cg4vstqSiI1JXpnQk+Ncix9KTrbFOXYTMqHV8vbDw1Dai0PRAhNTX8P+FI 1GZDlgGISddPdi75Tdq1D+aG6iMAS7Jn7uFYOn0eyjJQCHkKgAhwU0PRiDUa5WpsMzD/ Wki/5V65S8B9/TeWN5qBPx3/uY0dKI8oGf6hgDTuEY5+ozkYcmolmswno/g2f00C7Nti Opil8VS4yp5rucO0ZAgcdGHod4ufhjGqKUvVbZRBAa4Uh4jMNiKM7O5B7+ChztD7gt/U jK8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=nES7YSTX0iBPY/vJFLeLQOZ7UisrY/vSSQKxE2q4SNU=; b=EBk2WzikHlN0wtowy0IPf9UqjFBROZMvW688UX8tHgqQQmAK+ffh/W4dGOH2hASmiD 4bWjescbGLjSP79nEatoxdhy51zpTVi8VT1u1kigNptz7tHs8/3We+b2X53pXUOwg2SI eQJ9khjruP+GsFa6xHHfQCOfSO3AaHMbTMM09h9bNW1mZkWpKdNYQ3m9cSwQk3lN7gWR ONmLJaq/UoHqVLsjwK8XCWL4HsdNHdiJ4kwqr+yaeoctTp8Ml0nC3sFua0M9z/9iaiCQ Lrtxd2YaUIPmsY8xRjSrs+WUA67sOyQ8Qipw4C2hdKCeRrWD3dghfiaKWriy+97CUrpc NXJA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i76si3066875pgc.416.2017.11.17.10.25.52; Fri, 17 Nov 2017 10:25:53 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760668AbdKQSZu (ORCPT + 28 others); Fri, 17 Nov 2017 13:25:50 -0500 Received: from foss.arm.com ([217.140.101.70]:39416 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760284AbdKQSVz (ORCPT ); Fri, 17 Nov 2017 13:21:55 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E7C90165C; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B97733F246; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 2BF871AE11F5; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 05/18] arm64: mm: Rename post_ttbr0_update_workaround Date: Fri, 17 Nov 2017 18:21:48 +0000 Message-Id: <1510942921-12564-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The post_ttbr0_update_workaround hook applies to any change to TTBRx_EL1. Since we're using TTBR1 for the ASID, rename the hook to make it clearer as to what it's doing. Signed-off-by: Will Deacon --- arch/arm64/include/asm/assembler.h | 5 ++--- arch/arm64/kernel/entry.S | 2 +- arch/arm64/mm/proc.S | 2 +- 3 files changed, 4 insertions(+), 5 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8359148858cb..622316a8c82b 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -464,10 +464,9 @@ alternative_endif .endm /* -/* - * Errata workaround post TTBR0_EL1 update. + * Errata workaround post TTBRx_EL1 update. */ - .macro post_ttbr0_update_workaround + .macro post_ttbr_update_workaround #ifdef CONFIG_CAVIUM_ERRATUM_27456 alternative_if ARM64_WORKAROUND_CAVIUM_27456 ic iallu diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index e1c59d4008a8..dd5fa2c3d489 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -255,7 +255,7 @@ alternative_else_nop_endif * Cavium erratum 27456 (broadcast TLBI instructions may cause I-cache * corruption). */ - post_ttbr0_update_workaround + post_ttbr_update_workaround .endif 1: .if \el != 0 diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 1623150ed0a6..447537c1699d 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -145,7 +145,7 @@ ENTRY(cpu_do_switch_mm) isb msr ttbr0_el1, x0 // now update TTBR0 isb - post_ttbr0_update_workaround + post_ttbr_update_workaround ret ENDPROC(cpu_do_switch_mm) From patchwork Fri Nov 17 18:21:49 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119208 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp846726qgn; Fri, 17 Nov 2017 10:25:49 -0800 (PST) X-Google-Smtp-Source: AGs4zMYxfDskGnCItb45fdN/0jJV5TtH7cTrwoROFhk0W7mT9kwDDzwINAHDmYbPGUnvKoZhPMVB X-Received: by 10.84.248.68 with SMTP id e4mr6043815pln.0.1510943149738; Fri, 17 Nov 2017 10:25:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943149; cv=none; d=google.com; s=arc-20160816; b=bVYtVJ20NTxdTcmHW3dwzk182ypHzLnagf9G0rlf9zA8YjIfu0zB8VjKtzpdow5hwI p/XnnbYvD+oDMPIKBIFKgULMKih9PpumdWnyn1yyNsN9sTq54CgFmE8Py0H/U3+D2t6H 7NcvkGxcleeUf3Zs3o/GigmUakrH3ZMQweAeWm7hD6HE3fdTJ5pGVA7KdJ6iDXY/7Dk2 3xw2gHFvY9Yv7KB+8Zuliu2rabM5pG0SeTwvzsHinK8auv+y9WK6hH/5VFUmKj03aNy/ FyN8wjq/j5pgxDXCXgjeypxiNN+iy3AR8bZqkoqtm0EiJZCnKZp6L5jJl2xJzl+l5nXq ANOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=xMOinoBt6p0MXL/ZtnJNfpLYdtAieIxSsdBO3GK8+q8=; b=wj9d/OoI31ZyjB+RLJKK81rRNimcDTplDO/dWrxpzRfLq/9P6kscJPOdbF3dQeY7lI DH1mBsjlyIqoBBAIdZqimVbiiKUN1KKHD8QEatv7oASx/qJNc+0gNmWCknfCPhvSBfks lIwdzCIpuz+1svwziyRlH7KIIxoZ1peQnJE6onTTEsM3qunHeEcbDdQW9VZnVQLRqWcu RMVq4ThRE6WFa3k5YlHUNps0qBv/dpWeEq11rKmdWpye3n6AmYAY7C9+099ycqW7wwCZ 4nfncWD/YydHAOyx4w2DjtxUMcXdED7AbvnQfddIkqxobBZe98FLRSn0AC9tUk7IWvkJ fVQg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i76si3066875pgc.416.2017.11.17.10.25.49; Fri, 17 Nov 2017 10:25:49 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760600AbdKQSZn (ORCPT + 28 others); Fri, 17 Nov 2017 13:25:43 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39418 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760285AbdKQSVz (ORCPT ); Fri, 17 Nov 2017 13:21:55 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id F0EEA165D; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C27DC3F599; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 3C5371AE12CE; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 06/18] arm64: mm: Fix and re-enable ARM64_SW_TTBR0_PAN Date: Fri, 17 Nov 2017 18:21:49 +0000 Message-Id: <1510942921-12564-7-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org With the ASID now installed in TTBR1, we can re-enable ARM64_SW_TTBR0_PAN by ensuring that we switch to a reserved ASID of zero when disabling user access and restore the active user ASID on the uaccess enable path. Signed-off-by: Will Deacon --- arch/arm64/Kconfig | 1 - arch/arm64/include/asm/asm-uaccess.h | 25 +++++++++++++++++-------- arch/arm64/include/asm/uaccess.h | 21 +++++++++++++++++---- arch/arm64/kernel/entry.S | 4 ++-- arch/arm64/lib/clear_user.S | 2 +- arch/arm64/lib/copy_from_user.S | 2 +- arch/arm64/lib/copy_in_user.S | 2 +- arch/arm64/lib/copy_to_user.S | 2 +- arch/arm64/mm/cache.S | 2 +- arch/arm64/xen/hypercall.S | 2 +- 10 files changed, 42 insertions(+), 21 deletions(-) -- 2.1.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 582bbd77c390..0df64a6a56d4 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -872,7 +872,6 @@ endif config ARM64_SW_TTBR0_PAN bool "Emulate Privileged Access Never using TTBR0_EL1 switching" - depends on BROKEN # Temporary while switch_mm is reworked help Enabling this option prevents the kernel from accessing user-space memory directly by pointing TTBR0_EL1 to a reserved diff --git a/arch/arm64/include/asm/asm-uaccess.h b/arch/arm64/include/asm/asm-uaccess.h index b3da6c886835..21b8cf304028 100644 --- a/arch/arm64/include/asm/asm-uaccess.h +++ b/arch/arm64/include/asm/asm-uaccess.h @@ -16,11 +16,20 @@ add \tmp1, \tmp1, #SWAPPER_DIR_SIZE // reserved_ttbr0 at the end of swapper_pg_dir msr ttbr0_el1, \tmp1 // set reserved TTBR0_EL1 isb + sub \tmp1, \tmp1, #SWAPPER_DIR_SIZE + bic \tmp1, \tmp1, #(0xffff << 48) + msr ttbr1_el1, \tmp1 // set reserved ASID + isb .endm - .macro __uaccess_ttbr0_enable, tmp1 + .macro __uaccess_ttbr0_enable, tmp1, tmp2 get_thread_info \tmp1 ldr \tmp1, [\tmp1, #TSK_TI_TTBR0] // load saved TTBR0_EL1 + mrs \tmp2, ttbr1_el1 + extr \tmp2, \tmp2, \tmp1, #48 + ror \tmp2, \tmp2, #16 + msr ttbr1_el1, \tmp2 // set the active ASID + isb msr ttbr0_el1, \tmp1 // set the non-PAN TTBR0_EL1 isb .endm @@ -31,18 +40,18 @@ alternative_if_not ARM64_HAS_PAN alternative_else_nop_endif .endm - .macro uaccess_ttbr0_enable, tmp1, tmp2 + .macro uaccess_ttbr0_enable, tmp1, tmp2, tmp3 alternative_if_not ARM64_HAS_PAN - save_and_disable_irq \tmp2 // avoid preemption - __uaccess_ttbr0_enable \tmp1 - restore_irq \tmp2 + save_and_disable_irq \tmp3 // avoid preemption + __uaccess_ttbr0_enable \tmp1, \tmp2 + restore_irq \tmp3 alternative_else_nop_endif .endm #else .macro uaccess_ttbr0_disable, tmp1 .endm - .macro uaccess_ttbr0_enable, tmp1, tmp2 + .macro uaccess_ttbr0_enable, tmp1, tmp2, tmp3 .endm #endif @@ -56,8 +65,8 @@ alternative_if ARM64_ALT_PAN_NOT_UAO alternative_else_nop_endif .endm - .macro uaccess_enable_not_uao, tmp1, tmp2 - uaccess_ttbr0_enable \tmp1, \tmp2 + .macro uaccess_enable_not_uao, tmp1, tmp2, tmp3 + uaccess_ttbr0_enable \tmp1, \tmp2, \tmp3 alternative_if ARM64_ALT_PAN_NOT_UAO SET_PSTATE_PAN(0) alternative_else_nop_endif diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index fc0f9eb66039..750a3b76a01c 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -107,15 +107,19 @@ static inline void __uaccess_ttbr0_disable(void) { unsigned long ttbr; + ttbr = read_sysreg(ttbr1_el1); /* reserved_ttbr0 placed at the end of swapper_pg_dir */ - ttbr = read_sysreg(ttbr1_el1) + SWAPPER_DIR_SIZE; - write_sysreg(ttbr, ttbr0_el1); + write_sysreg(ttbr + SWAPPER_DIR_SIZE, ttbr0_el1); + isb(); + /* Set reserved ASID */ + ttbr &= ~(0xffffUL << 48); + write_sysreg(ttbr, ttbr1_el1); isb(); } static inline void __uaccess_ttbr0_enable(void) { - unsigned long flags; + unsigned long flags, ttbr0, ttbr1; /* * Disable interrupts to avoid preemption between reading the 'ttbr0' @@ -123,7 +127,16 @@ static inline void __uaccess_ttbr0_enable(void) * roll-over and an update of 'ttbr0'. */ local_irq_save(flags); - write_sysreg(current_thread_info()->ttbr0, ttbr0_el1); + ttbr0 = current_thread_info()->ttbr0; + + /* Restore active ASID */ + ttbr1 = read_sysreg(ttbr1_el1); + ttbr1 |= ttbr0 & (0xffffUL << 48); + write_sysreg(ttbr1, ttbr1_el1); + isb(); + + /* Restore user page table */ + write_sysreg(ttbr0, ttbr0_el1); isb(); local_irq_restore(flags); } diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index dd5fa2c3d489..e2afc15a1535 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -184,7 +184,7 @@ alternative_if ARM64_HAS_PAN alternative_else_nop_endif .if \el != 0 - mrs x21, ttbr0_el1 + mrs x21, ttbr1_el1 tst x21, #0xffff << 48 // Check for the reserved ASID orr x23, x23, #PSR_PAN_BIT // Set the emulated PAN in the saved SPSR b.eq 1f // TTBR0 access already disabled @@ -246,7 +246,7 @@ alternative_else_nop_endif tbnz x22, #22, 1f // Skip re-enabling TTBR0 access if the PSR_PAN_BIT is set .endif - __uaccess_ttbr0_enable x0 + __uaccess_ttbr0_enable x0, x1 .if \el == 0 /* diff --git a/arch/arm64/lib/clear_user.S b/arch/arm64/lib/clear_user.S index e88fb99c1561..8f9c4641e706 100644 --- a/arch/arm64/lib/clear_user.S +++ b/arch/arm64/lib/clear_user.S @@ -30,7 +30,7 @@ * Alignment fixed up by hardware. */ ENTRY(__clear_user) - uaccess_enable_not_uao x2, x3 + uaccess_enable_not_uao x2, x3, x4 mov x2, x1 // save the size for fixup return subs x1, x1, #8 b.mi 2f diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S index 4b5d826895ff..69d86a80f3e2 100644 --- a/arch/arm64/lib/copy_from_user.S +++ b/arch/arm64/lib/copy_from_user.S @@ -64,7 +64,7 @@ end .req x5 ENTRY(__arch_copy_from_user) - uaccess_enable_not_uao x3, x4 + uaccess_enable_not_uao x3, x4, x5 add end, x0, x2 #include "copy_template.S" uaccess_disable_not_uao x3 diff --git a/arch/arm64/lib/copy_in_user.S b/arch/arm64/lib/copy_in_user.S index b24a830419ad..e442b531252a 100644 --- a/arch/arm64/lib/copy_in_user.S +++ b/arch/arm64/lib/copy_in_user.S @@ -65,7 +65,7 @@ end .req x5 ENTRY(raw_copy_in_user) - uaccess_enable_not_uao x3, x4 + uaccess_enable_not_uao x3, x4, x5 add end, x0, x2 #include "copy_template.S" uaccess_disable_not_uao x3 diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S index 351f0766f7a6..318f15d5c336 100644 --- a/arch/arm64/lib/copy_to_user.S +++ b/arch/arm64/lib/copy_to_user.S @@ -63,7 +63,7 @@ end .req x5 ENTRY(__arch_copy_to_user) - uaccess_enable_not_uao x3, x4 + uaccess_enable_not_uao x3, x4, x5 add end, x0, x2 #include "copy_template.S" uaccess_disable_not_uao x3 diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 7f1dbe962cf5..6cd20a8c0952 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -49,7 +49,7 @@ ENTRY(flush_icache_range) * - end - virtual end address of region */ ENTRY(__flush_cache_user_range) - uaccess_ttbr0_enable x2, x3 + uaccess_ttbr0_enable x2, x3, x4 dcache_line_size x2, x3 sub x3, x2, #1 bic x4, x0, x3 diff --git a/arch/arm64/xen/hypercall.S b/arch/arm64/xen/hypercall.S index 401ceb71540c..acdbd2c9e899 100644 --- a/arch/arm64/xen/hypercall.S +++ b/arch/arm64/xen/hypercall.S @@ -101,7 +101,7 @@ ENTRY(privcmd_call) * need the explicit uaccess_enable/disable if the TTBR0 PAN emulation * is enabled (it implies that hardware UAO and PAN disabled). */ - uaccess_ttbr0_enable x6, x7 + uaccess_ttbr0_enable x6, x7, x8 hvc XEN_IMM /* From patchwork Fri Nov 17 18:21:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119206 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp846290qgn; Fri, 17 Nov 2017 10:25:25 -0800 (PST) X-Google-Smtp-Source: AGs4zMZzjwOE+FG5XTZw4PMpECFQMAqteNyVu/2foZAepeCYH793DLlsrt7M/T6hgfFnOCoyVGTA X-Received: by 10.101.93.141 with SMTP id f13mr5679457pgt.92.1510943125703; Fri, 17 Nov 2017 10:25:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943125; cv=none; d=google.com; s=arc-20160816; b=rXfkVowVel1If+h61qUft0hA+8N+pMQqKeZyKSXDPCOr7EpN0xuLPXIYZGHbu3t6q3 rQPBSInUnZEQX33YR2Z5Dpx23SHFp7BKHIl32mHsRegEh3zwVZDgyE8vpapL63dVauC+ 0SCNi4f+bO23lttGm/qsuos7E6ANIXibnbbEBAqLZnyu///GftkqkiOINhHb6wDV02fm FA6/3CJBdcALR8Hxh3OErlkZwcr1I+7YyhISJr2mPlvr3eptcHLbaHNZLFDSrhpqtJl0 ZUH7o0E40Go/bTIsb2zp88XwOrYmlFeRVjv7SzF85JkHJF5lHg0Xt4mwd3i56SFkgpPD konA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=c7dHcwSDUHqdwwQwJycOKWjLK7Mx3XaXcICWWym4L3k=; b=jndO4rmCVpF+bK4iF/62I3+Xe01MwOFiwIKLOCedoECxFgsk3/4pBs0wsYwoiAZOsx vXfpG1Z+8JAgmhlLqDLJfOaAFdOBcF+AjXQP9C3XF7t+Z7NpSqjhrDDpH6+dK2WsSESq d/IOitUIkYVCevFi77D+X4LJ/Cgc+pgnzk1quckTZvVhsk2Vc48pXh0mDAoELBOmncZ2 S5XH3iC9s9dulYLbdH3gPhKWEPTqytY4aXsi95RxuW15g1b02fomBLzgvO6vAT7R9WxU 2svI4EoXIXn+NME8UFhI6h9ZTLgbGxrQ9E6hgr2C6dbHNeC6NDsvBI9vjc95PY47yWRX h6mQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y62si3088438pgb.383.2017.11.17.10.25.25; Fri, 17 Nov 2017 10:25:25 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760534AbdKQSZX (ORCPT + 28 others); Fri, 17 Nov 2017 13:25:23 -0500 Received: from foss.arm.com ([217.140.101.70]:39450 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760290AbdKQSV4 (ORCPT ); Fri, 17 Nov 2017 13:21:56 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0858E1682; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id CB6B83F5A0; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 4B2281AE1352; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 07/18] arm64: mm: Allocate ASIDs in pairs Date: Fri, 17 Nov 2017 18:21:50 +0000 Message-Id: <1510942921-12564-8-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation for separate kernel/user ASIDs, allocate them in pairs for each mm_struct. The bottom bit distinguishes the two: if it is set, then the ASID will map only userspace. Signed-off-by: Will Deacon --- arch/arm64/include/asm/mmu.h | 1 + arch/arm64/mm/context.c | 25 +++++++++++++++++-------- 2 files changed, 18 insertions(+), 8 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 0d34bf0a89c7..01bfb184f2a8 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -17,6 +17,7 @@ #define __ASM_MMU_H #define MMCF_AARCH32 0x1 /* mm context flag for AArch32 executables */ +#define USER_ASID_FLAG (UL(1) << 48) typedef struct { atomic64_t id; diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 78816e476491..db28958d9e4f 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -39,7 +39,16 @@ static cpumask_t tlb_flush_pending; #define ASID_MASK (~GENMASK(asid_bits - 1, 0)) #define ASID_FIRST_VERSION (1UL << asid_bits) -#define NUM_USER_ASIDS ASID_FIRST_VERSION + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define NUM_USER_ASIDS (ASID_FIRST_VERSION >> 1) +#define asid2idx(asid) (((asid) & ~ASID_MASK) >> 1) +#define idx2asid(idx) (((idx) << 1) & ~ASID_MASK) +#else +#define NUM_USER_ASIDS (ASID_FIRST_VERSION) +#define asid2idx(asid) ((asid) & ~ASID_MASK) +#define idx2asid(idx) asid2idx(idx) +#endif /* Get the ASIDBits supported by the current CPU */ static u32 get_cpu_asid_bits(void) @@ -104,7 +113,7 @@ static void flush_context(unsigned int cpu) */ if (asid == 0) asid = per_cpu(reserved_asids, i); - __set_bit(asid & ~ASID_MASK, asid_map); + __set_bit(asid2idx(asid), asid_map); per_cpu(reserved_asids, i) = asid; } @@ -156,16 +165,16 @@ static u64 new_context(struct mm_struct *mm, unsigned int cpu) * We had a valid ASID in a previous life, so try to re-use * it if possible. */ - asid &= ~ASID_MASK; - if (!__test_and_set_bit(asid, asid_map)) + if (!__test_and_set_bit(asid2idx(asid), asid_map)) return newasid; } /* * Allocate a free ASID. If we can't find one, take a note of the - * currently active ASIDs and mark the TLBs as requiring flushes. - * We always count from ASID #1, as we use ASID #0 when setting a - * reserved TTBR0 for the init_mm. + * currently active ASIDs and mark the TLBs as requiring flushes. We + * always count from ASID #2 (index 1), as we use ASID #0 when setting + * a reserved TTBR0 for the init_mm and we allocate ASIDs in even/odd + * pairs. */ asid = find_next_zero_bit(asid_map, NUM_USER_ASIDS, cur_idx); if (asid != NUM_USER_ASIDS) @@ -182,7 +191,7 @@ static u64 new_context(struct mm_struct *mm, unsigned int cpu) set_asid: __set_bit(asid, asid_map); cur_idx = asid; - return asid | generation; + return idx2asid(asid) | generation; } void check_and_switch_context(struct mm_struct *mm, unsigned int cpu) From patchwork Fri Nov 17 18:21:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119207 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp846535qgn; Fri, 17 Nov 2017 10:25:37 -0800 (PST) X-Google-Smtp-Source: AGs4zMZ/x3HTxwKSQeB05J3HtzjSvE0YqAaDfMRAdiWsv/5/necJmGE8nUuNOHpKkpwW1zDO24wy X-Received: by 10.99.142.73 with SMTP id k70mr6017406pge.426.1510943137821; Fri, 17 Nov 2017 10:25:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943137; cv=none; d=google.com; s=arc-20160816; b=lrVPBp77ZjVURrk6GUSmgRGYgnjOUvrZKxSSGFgc028Kg1cz32mNS3pG/QSbsWTtcS UpD4I64akFEHZSwIB8nslhRvlIQ8N3fMOEbzUxkrXL/O3mD2/LYima8qkG1p+UscqQla 3L22qAzQCJ9Tubxc5CVHNTyD++LwCdAWy9S4MoOYeHQqWilfL/enRB69BVW1a6t7dvaw Ar4O+VPDQbvPWmIuQfMPNQnKkw1/HnLQU6Z3ovQNvUCjKGpSADtEw/yaiFpJI2mTfOWu rd0qdgNcKKvvqkF9vrzJrEgEVWcqsuI+mUUQsYtWQAgh/fuzsHHmubpUHeNKPN1zqjCJ fVSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=uscVK/pF4fgLrK54hpEJVm83yG7CHNfeg3AmEAcmx0w=; b=RG6zQr5eYpJ0jtHSJVQqECwl+H3CdVaF22tSQ+nmHLhEZsI0lx8KkSK5ZlnjoDpt3X 2KOT0V+nes7m/b5F/T2tLlHo9oWj+KJpIYF4Qw71e0I2WxHh8VvzY0X1wt6ol5y6RPnY qHoQOo+fsgWNB7SEpbAgjc8hImf+EpDgiKCSClz2RU8obwXjgcE2JvITujCKWamaTmyH LBpE7hizctH1zIuIkBOjsHCynva6TWZ0K+Pl++/qMLZ+hjL/TZ7OqJHRhRUuCk58Xkdw o8/Jo5pGyRwEil9cIZ2TYwwqNCzLMpk+NCuyThcmEVusiHEZ5MKqU7C13YjKkF39giW2 WdMA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 64si3169904plk.227.2017.11.17.10.25.37; Fri, 17 Nov 2017 10:25:37 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760577AbdKQSZe (ORCPT + 28 others); Fri, 17 Nov 2017 13:25:34 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39452 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760288AbdKQSV4 (ORCPT ); Fri, 17 Nov 2017 13:21:56 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0EB27169F; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D4F3F3F703; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 5CCF41AE1361; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 08/18] arm64: mm: Add arm64_kernel_mapped_at_el0 helper using static key Date: Fri, 17 Nov 2017 18:21:51 +0000 Message-Id: <1510942921-12564-9-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In order for code such as TLB invalidation to operate efficiently when the decision to map the kernel at EL0 is determined at runtime, this patch introduces a helper function, arm64_kernel_mapped_at_el0, which uses a static key that will later be hooked up to a command-line option. Signed-off-by: Will Deacon --- arch/arm64/include/asm/mmu.h | 11 +++++++++++ arch/arm64/mm/mmu.c | 5 +++++ 2 files changed, 16 insertions(+) -- 2.1.4 diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 01bfb184f2a8..a84f851409ca 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -19,6 +19,8 @@ #define MMCF_AARCH32 0x1 /* mm context flag for AArch32 executables */ #define USER_ASID_FLAG (UL(1) << 48) +#ifndef __ASSEMBLY__ + typedef struct { atomic64_t id; void *vdso; @@ -32,6 +34,14 @@ typedef struct { */ #define ASID(mm) ((mm)->context.id.counter & 0xffff) +DECLARE_STATIC_KEY_TRUE(__unmap_kernel_at_el0); + +static inline bool arm64_kernel_mapped_at_el0(void) +{ + return !IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) || + !static_branch_likely(&__unmap_kernel_at_el0); +} + extern void paging_init(void); extern void bootmem_init(void); extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt); @@ -42,4 +52,5 @@ extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, extern void *fixmap_remap_fdt(phys_addr_t dt_phys); extern void mark_linear_text_alias_ro(void); +#endif /* !__ASSEMBLY__ */ #endif diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index f1eb15e0e864..a75858267b6d 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -525,6 +525,11 @@ static int __init parse_rodata(char *arg) } early_param("rodata", parse_rodata); +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +DEFINE_STATIC_KEY_TRUE(__unmap_kernel_at_el0); +EXPORT_SYMBOL_GPL(__unmap_kernel_at_el0); +#endif + /* * Create fine-grained mappings for the kernel. */ From patchwork Fri Nov 17 18:21:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119205 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp846033qgn; Fri, 17 Nov 2017 10:25:12 -0800 (PST) X-Google-Smtp-Source: AGs4zMYak5j7JyYWd9IZ6TX1zDrpGQeT+QN2t71CpcBkQUMe38XydqVaZmXrYwrj4JGMQ9a1FyZ8 X-Received: by 10.101.73.8 with SMTP id p8mr6039339pgs.106.1510943112223; Fri, 17 Nov 2017 10:25:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943112; cv=none; d=google.com; s=arc-20160816; b=j5F4MeCH0lXlUOwc5loA7W7qL1+1KvU1tZ1SLs92otxp/9xXX+aZ+Y90naiSrFmjm3 wtQs4CaGuFZzf7t6iCht1snXL0qsP0lW/M+044fqZhnsWGl5oef9bU7UozTuoBzRTeEH bkW21BKkzZEK2jP4L8pM/Ergn7AeHKzPI2AtMXlQM/fUlSPVQ3xGQdr60+iVaRZCrSpP b48vPSeh+e3iN4KuD+EPpB+ERRjjjMCzCFWu0rN0sJMlzDD18YLsYuuhx2QDwbxcxoYa FQjLzr8/7cUUHlSHbrXbEDKR3dpLxFzKa77/Gn5N4SvjZQLBMdfumMPqol4antrTThh6 pAPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=leKaMMVmuSX47rIR+EIgMDAFUVUxDkOfi9X26xuXjDc=; b=tbzOYDsf7wRAYuWZHEW3NhJ2xQdmDd1hlRCLNBgK/ev2wxOn4MCrBqTNJ1bJC4SzgV bikz2N+0I+K9qFxun+26CacXIp+4t2ptPbbIwGCCDiQOi53QbMR7VFt5h5zJwo26tpwc OFOT83Gxvvyg3BIWi2XDfg0qapw1OuVB6qZ+W7nHiJcaBd9WmVIrPqZYcsplBoVQ1+to +HUrzS+1OM8j+GbUPjXU3j9A8jzVqSzDTLc6VVWh1x34vORTVJRF8JChMTk3bdBqQqT5 jVDrrSFRvGaKyNld76fObQm0peiwYwb7qxr5lMw9rDKruDIkvYd6N6Gr3nci9/vhx6PQ /FPA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y62si3088438pgb.383.2017.11.17.10.25.11; Fri, 17 Nov 2017 10:25:12 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760507AbdKQSZJ (ORCPT + 28 others); Fri, 17 Nov 2017 13:25:09 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39458 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760294AbdKQSV4 (ORCPT ); Fri, 17 Nov 2017 13:21:56 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 175F016BA; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id DDBC63F7B0; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 6D12A1AE1373; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 09/18] arm64: mm: Invalidate both kernel and user ASIDs when performing TLBI Date: Fri, 17 Nov 2017 18:21:52 +0000 Message-Id: <1510942921-12564-10-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Since an mm has both a kernel and a user ASID, we need to ensure that broadcast TLB maintenance targets both address spaces so that things like CoW continue to work with the uaccess primitives in the kernel. Signed-off-by: Will Deacon --- arch/arm64/include/asm/tlbflush.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index af1c76981911..42d250ec74b1 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -23,6 +23,7 @@ #include #include +#include /* * Raw TLBI operations. @@ -54,6 +55,11 @@ #define __tlbi(op, ...) __TLBI_N(op, ##__VA_ARGS__, 1, 0) +#define __tlbi_user(op, arg) do { \ + if (!arm64_kernel_mapped_at_el0()) \ + __tlbi(op, (arg) | USER_ASID_FLAG); \ +} while (0) + /* * TLB Management * ============== @@ -115,6 +121,7 @@ static inline void flush_tlb_mm(struct mm_struct *mm) dsb(ishst); __tlbi(aside1is, asid); + __tlbi_user(aside1is, asid); dsb(ish); } @@ -125,6 +132,7 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, dsb(ishst); __tlbi(vale1is, addr); + __tlbi_user(vale1is, addr); dsb(ish); } @@ -151,10 +159,13 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, dsb(ishst); for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12)) { - if (last_level) + if (last_level) { __tlbi(vale1is, addr); - else + __tlbi_user(vale1is, addr); + } else { __tlbi(vae1is, addr); + __tlbi_user(vae1is, addr); + } } dsb(ish); } @@ -194,6 +205,7 @@ static inline void __flush_tlb_pgtable(struct mm_struct *mm, unsigned long addr = uaddr >> 12 | (ASID(mm) << 48); __tlbi(vae1is, addr); + __tlbi_user(vae1is, addr); dsb(ish); } From patchwork Fri Nov 17 18:21:53 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119198 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp844604qgn; Fri, 17 Nov 2017 10:23:45 -0800 (PST) X-Google-Smtp-Source: AGs4zMa9HavMeZ9Jn3tWkT/ZMM+Kvr7KbA3R/qwUMqHz9OAQoTwDLW5fEZCrQgH8jnUuR79dFoE0 X-Received: by 10.98.185.10 with SMTP id z10mr2958376pfe.8.1510943025535; Fri, 17 Nov 2017 10:23:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943025; cv=none; d=google.com; s=arc-20160816; b=b5ulCgauaiWdaohoVJQq5l41eKB6KqCazqbg4/WZODXZHL4ihZWT7l2QReWYPNE8Rx fvfJfpA0XnU/NRtLirqb0rTrdSLtuO3iHTz6rrJfBo+HYmJgnkPxYGGZPQHHTs5jTSAX l0NE2yTuUswwethF3aX9kFvMOJaC+oNChY/wPIiwiYw0E3uhKhyCZ8BvjfMchv67Oixq c+n9YaiwN+5t2PdsxOsa5dBK4+dsd+IpwIeiseL6r+Mm9Y6Lb1mOiBxQnFqhx/SXX/wV Ah8tigpWZ1FOM7ca+R8akakbfoaj03RdZvQoj//8GuuONHS/12ShS8CooM6jHHR6WpEU z20Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=gnontl8fXQWIxTymo1Qv6zBVUhMLyS2Y22fjMnKZK4U=; b=RGlOiMsKiqHgEdhH1Yq87+sf/yuGIc/pbskNBJGuFfXjI3bNZzb1upFa8Iy/rVx0eJ HrQE69MDRpJNlLq3jOzh5cS8+zjyZUdw8B8arZHLmjCUpTU2nPjmuyLaU7nUok+K91Ep ZaYFu64CjvFVslCPR52eAJCO8Fpk+UTnJ4w09eqxFwm41uPF7Al0ACHcl06ZmJZ2gNJ4 BgY5Y+cAHOuJFkD5q5hsIt3pz8VrqAycfuvSLwYfNcTDGNQMOv7znZKLyd2ieuQF/7el h0aZOwfVeyPGLSPwILO5QK6DWGGaT3MHJTh3gsVQnYULDBYRvKDmcS02aqo19uljILOR 5iYQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 129si3038351pgi.726.2017.11.17.10.23.45; Fri, 17 Nov 2017 10:23:45 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751515AbdKQSWo (ORCPT + 28 others); Fri, 17 Nov 2017 13:22:44 -0500 Received: from foss.arm.com ([217.140.101.70]:39460 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760293AbdKQSV4 (ORCPT ); Fri, 17 Nov 2017 13:21:56 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 208A71715; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E672B3F7B5; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 7D6141AE137F; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 10/18] arm64: entry: Add exception trampoline page for exceptions from EL0 Date: Fri, 17 Nov 2017 18:21:53 +0000 Message-Id: <1510942921-12564-11-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To allow unmapping of the kernel whilst running at EL0, we need to point the exception vectors at an entry trampoline that can map/unmap the kernel on entry/exit respectively. This patch adds the trampoline page, although it is not yet plugged into the vector table and is therefore unused. Signed-off-by: Will Deacon --- arch/arm64/kernel/entry.S | 85 +++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/vmlinux.lds.S | 17 +++++++++ 2 files changed, 102 insertions(+) -- 2.1.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index e2afc15a1535..d850af724c8c 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -895,6 +896,90 @@ __ni_sys_trace: .popsection // .entry.text +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +/* + * Exception vectors trampoline. + */ + .pushsection ".entry.tramp.text", "ax" + + .macro tramp_map_kernel, tmp + mrs \tmp, ttbr1_el1 + sub \tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) + bic \tmp, \tmp, #USER_ASID_FLAG + msr ttbr1_el1, \tmp + .endm + + .macro tramp_unmap_kernel, tmp + mrs \tmp, ttbr1_el1 + add \tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) + orr \tmp, \tmp, #USER_ASID_FLAG + msr ttbr1_el1, \tmp + /* + * We avoid running the post_ttbr_update_workaround here because the + * user and kernel ASIDs don't have conflicting mappings, so any + * "blessing" as described in: + * + * http://lkml.kernel.org/r/56BB848A.6060603@caviumnetworks.com + * + * will not hurt correctness. Whilst this may partially defeat the + * point of using split ASIDs in the first place, it avoids + * the hit of invalidating the entire I-cache on every return to + * userspace. + */ + .endm + + .macro tramp_ventry, regsize = 64 + .align 7 +1: + .if \regsize == 64 + msr tpidrro_el0, x30 + .endif + tramp_map_kernel x30 + ldr x30, =vectors + prfm plil1strm, [x30, #(1b - tramp_vectors)] + msr vbar_el1, x30 + add x30, x30, #(1b - tramp_vectors) + isb + br x30 + .endm + + .macro tramp_exit, regsize = 64 + adr x30, tramp_vectors + msr vbar_el1, x30 + tramp_unmap_kernel x30 + .if \regsize == 64 + mrs x30, far_el1 + .endif + eret + .endm + + .align 11 +ENTRY(tramp_vectors) + .space 0x400 + + tramp_ventry + tramp_ventry + tramp_ventry + tramp_ventry + + tramp_ventry 32 + tramp_ventry 32 + tramp_ventry 32 + tramp_ventry 32 +END(tramp_vectors) + +ENTRY(tramp_exit_native) + tramp_exit +END(tramp_exit_native) + +ENTRY(tramp_exit_compat) + tramp_exit 32 +END(tramp_exit_compat) + + .ltorg + .popsection // .entry.tramp.text +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + /* * Special system call wrappers. */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7da3e5c366a0..6b4260f22aab 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -57,6 +57,17 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define TRAMP_TEXT \ + . = ALIGN(PAGE_SIZE); \ + VMLINUX_SYMBOL(__entry_tramp_text_start) = .; \ + *(.entry.tramp.text) \ + . = ALIGN(PAGE_SIZE); \ + VMLINUX_SYMBOL(__entry_tramp_text_end) = .; +#else +#define TRAMP_TEXT +#endif + /* * The size of the PE/COFF section that covers the kernel image, which * runs from stext to _edata, must be a round multiple of the PE/COFF @@ -113,6 +124,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + TRAMP_TEXT *(.fixup) *(.gnu.warning) . = ALIGN(16); @@ -214,6 +226,11 @@ SECTIONS . += RESERVED_TTBR0_SIZE; #endif +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + tramp_pg_dir = .; + . += PAGE_SIZE; +#endif + __pecoff_data_size = ABSOLUTE(. - __initdata_begin); _end = .; From patchwork Fri Nov 17 18:21:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119203 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp845649qgn; Fri, 17 Nov 2017 10:24:52 -0800 (PST) X-Google-Smtp-Source: AGs4zMbd6eRtVQ9bnwpvyvEu8/paZR63RcAwtkr6X0Ae/b6b4XJTkcqFbhsxq9Rgd2nSdy6Hefp4 X-Received: by 10.99.114.82 with SMTP id c18mr5930516pgn.221.1510943092082; Fri, 17 Nov 2017 10:24:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943092; cv=none; d=google.com; s=arc-20160816; b=qm/36nLbZBehr0Hjk5vghI/gka9d8HcTuUliRX8xXKpxqw+mWqMCC1qhFPoq1SX3oK XvcUbVih7hjtJ+HmqGFZRColTQ8MgnMU7ZAesvSKO7KmSp3WDnIC6Butqz8mY5pLi6mi Rgpv0luVrHB3JQYKM9uUvMcJYH1zN59Q6ZdXg4I3nQauWhHhBqQT10QZDwqldxVWUORg zQETKC80vyb+uYr1lkFB28622Nel/K0U9Ia/QizDfIZfb01CrMPs7fAkTY6JcrpuSx6T ZvH2BYEnAtmfeu8/oapRyjiw/F4EuXKwmutxuzUkjnsNyGqFdk/M+XKj+ixFVsFUX+I5 2qHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=TJiBhosd4G9tpeCgmcONfRfbFPWgmlbBPBrgwJO0jP0=; b=RC6TFM/Lx5t5uwMltv1NoBmBSVxOsvrnCGqqQceAq/NNBnZVem0LIn5aKjG6QLFl// riDOogh3vNN5bD/fWs9/9v5k+J+CDoJ0+D0sjr3/Z0OolwhZ3mpGED3IIJfxbP768Jdz mzces0bEb9usP5BUf9pTlAMdNeGqgsToMp1e31RIv9kbSry9VQtKN13NnTr0cFyvFYbv Mts2tJoAZXkICvmK//mDfVnAMYEn91A+sQ0NO/0BssklpT/flQJRTxPL8Tk3PNpviH/2 Yt2pPpg/5SlJjCrw6WVDlXtLgUCqQryU9elY3Yn2OkJ72DfHYGDBqS5sGrJ0OB0KT/j5 WvUw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y62si3088438pgb.383.2017.11.17.10.24.51; Fri, 17 Nov 2017 10:24:52 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760450AbdKQSYr (ORCPT + 28 others); Fri, 17 Nov 2017 13:24:47 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39464 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760299AbdKQSV5 (ORCPT ); Fri, 17 Nov 2017 13:21:57 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 29CB7174E; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id EF59E3F246; Fri, 17 Nov 2017 10:21:54 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 8DA9B1AE1382; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 11/18] arm64: mm: Map entry trampoline into trampoline and kernel page tables Date: Fri, 17 Nov 2017 18:21:54 +0000 Message-Id: <1510942921-12564-12-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The exception entry trampoline needs to be mapped at the same virtual address in both the trampoline page table (which maps nothing else) and also the kernel page table, so that we can swizzle TTBR1_EL1 on exceptions from and return to EL0. This patch maps the trampoline at a fixed virtual address (TRAMP_VALIAS), which allows the kernel proper to be randomized with respect to the trampoline when KASLR is enabled. Signed-off-by: Will Deacon --- arch/arm64/include/asm/memory.h | 1 + arch/arm64/include/asm/pgtable.h | 1 + arch/arm64/mm/mmu.c | 48 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+) -- 2.1.4 diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index f7c4d2146aed..18a3cb86ef17 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -70,6 +70,7 @@ #define PAGE_OFFSET (UL(0xffffffffffffffff) - \ (UL(1) << (VA_BITS - 1)) + 1) #define KIMAGE_VADDR (MODULES_END) +#define TRAMP_VALIAS (KIMAGE_VADDR) #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) #define MODULES_VADDR (VA_START + KASAN_SHADOW_SIZE) #define MODULES_VSIZE (SZ_128M) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index b46e54c2399b..2f3b58a1d434 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -667,6 +667,7 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm, extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; extern pgd_t idmap_pg_dir[PTRS_PER_PGD]; +extern pgd_t tramp_pg_dir[PTRS_PER_PGD]; /* * Encode and decode a swap entry: diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index a75858267b6d..5ce5cb1249da 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -528,6 +528,51 @@ early_param("rodata", parse_rodata); #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 DEFINE_STATIC_KEY_TRUE(__unmap_kernel_at_el0); EXPORT_SYMBOL_GPL(__unmap_kernel_at_el0); + +static void __init add_tramp_vma(void) +{ + extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; + static struct vm_struct vmlinux_tramp; + unsigned long size = (unsigned long)__entry_tramp_text_end - + (unsigned long)__entry_tramp_text_start; + + vmlinux_tramp = (struct vm_struct) { + .addr = (void *)TRAMP_VALIAS, + .phys_addr = __pa_symbol(__entry_tramp_text_start), + .size = size + PAGE_SIZE, + .flags = VM_MAP, + .caller = __builtin_return_address(0), + + }; + + vm_area_add_early(&vmlinux_tramp); +} + +static int __init map_entry_trampoline(void) +{ + extern char __entry_tramp_text_start[], __entry_tramp_text_end[]; + + pgprot_t prot = rodata_enabled ? PAGE_KERNEL_ROX : PAGE_KERNEL_EXEC; + phys_addr_t size = (unsigned long)__entry_tramp_text_end - + (unsigned long)__entry_tramp_text_start; + phys_addr_t pa_start = __pa_symbol(__entry_tramp_text_start); + + /* The trampoline is always mapped and can therefore be global */ + pgprot_val(prot) &= ~PTE_NG; + + /* Map only the text into the trampoline page table */ + memset((char *)tramp_pg_dir, 0, PGD_SIZE); + __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, size, prot, + pgd_pgtable_alloc, 0); + + /* ...as well as the kernel page table */ + __create_pgd_mapping(init_mm.pgd, pa_start, TRAMP_VALIAS, size, prot, + pgd_pgtable_alloc, 0); + return 0; +} +core_initcall(map_entry_trampoline); +#else +static void __init add_tramp_vma(void) {} #endif /* @@ -559,6 +604,9 @@ static void __init map_kernel(pgd_t *pgd) &vmlinux_initdata, 0, VM_NO_GUARD); map_kernel_segment(pgd, _data, _end, PAGE_KERNEL, &vmlinux_data, 0, 0); + /* Add a VMA for the trampoline page, which will be mapped later on */ + add_tramp_vma(); + if (!pgd_val(*pgd_offset_raw(pgd, FIXADDR_START))) { /* * The fixmap falls in a separate pgd to the kernel, and doesn't From patchwork Fri Nov 17 18:21:55 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119202 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp845441qgn; Fri, 17 Nov 2017 10:24:40 -0800 (PST) X-Google-Smtp-Source: AGs4zMb/sRw3wuHvcpQPJc0jzxlnOSv6TW7nIAYaoXTbNm207TwYatenNpiuSV7pMZ0MrQCaRf/2 X-Received: by 10.99.152.68 with SMTP id l4mr6006134pgo.208.1510943080336; Fri, 17 Nov 2017 10:24:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943080; cv=none; d=google.com; s=arc-20160816; b=wVPyuQcExvinzK8+s8LKsBkOdv2/VSXCXmoyiCF76WyenBHrEWTO/j1Qmx14WZBakt uFrcMLNg/Fo1xo0gpKMFUBIrHUVyoMiQT0Xqi1pCXLB56gy1+/oRIS0ObxfY/uXywhM8 pijtkzwNYpRYzc6o/XJnV86ZliGJux5ibpdLs796NF5AQ+Gf9B4+fR1xEhUn6ts7BaVG mP5vGe/W9ejvT/GdQTnug2F8MOI+O3Na/M4TugLViZwAsEkMfu+6c5os6NzAHB2ULPzD 5wEPcNHayJlgusWbe+SK5Wf3CIH7ZXcB8J+9E1u+S9s5z+Ly91jp6R8gvOnQTCLJKGLG swPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=rUJlN0Kz1Z+NoaEhUmAaI1XRcGMCMkwe2PvIgD5P3xw=; b=DMUdhyFevf19zic5euWqRm1cc7/oOd5hP1VIgujHaEDuagfNuYFCHeDrKdWdwXUVm3 A7fAvcpGQPsRQszKsUa0ewWRV/X5R2xsuaeKXyq4MlSb+cEgw9YIMVBR902sqh8d1tyl m52lKRVBSI0POwFn2qL706nQ2SRlvj6KI1ZzVo2NBgBbLT2fRdUpy0Snr5ae56wDXP0I MSsvcbJgFH1Kcsh5AHuEYDgWTJ+hHa83tX5sGJEv7f6iBlwxhMVsRwR41SWyNXeIrhAI WOEIjA0G+WxDiJ6fZb9oOIqq1kdKTAaItB9Pc4t/rLtkoEd5hNxYd6NKHY79PoS3oz22 f0jQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p8si2041682plo.63.2017.11.17.10.24.39; Fri, 17 Nov 2017 10:24:40 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760412AbdKQSYf (ORCPT + 28 others); Fri, 17 Nov 2017 13:24:35 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39474 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760303AbdKQSV7 (ORCPT ); Fri, 17 Nov 2017 13:21:59 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 32DFF1993; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 041EC3F599; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 9DF581AE1384; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 12/18] arm64: entry: Explicitly pass exception level to kernel_ventry macro Date: Fri, 17 Nov 2017 18:21:55 +0000 Message-Id: <1510942921-12564-13-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We will need to treat exceptions from EL0 differently in kernel_ventry, so rework the macro to take the exception level as an argument and construct the branch target using that. Signed-off-by: Will Deacon --- arch/arm64/kernel/entry.S | 46 +++++++++++++++++++++++----------------------- 1 file changed, 23 insertions(+), 23 deletions(-) -- 2.1.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index d850af724c8c..e98cf3064509 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -70,7 +70,7 @@ #define BAD_FIQ 2 #define BAD_ERROR 3 - .macro kernel_ventry label + .macro kernel_ventry, el, label, regsize = 64 .align 7 sub sp, sp, #S_FRAME_SIZE #ifdef CONFIG_VMAP_STACK @@ -83,7 +83,7 @@ tbnz x0, #THREAD_SHIFT, 0f sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp - b \label + b el\()\el\()_\label 0: /* @@ -115,7 +115,7 @@ sub sp, sp, x0 mrs x0, tpidrro_el0 #endif - b \label + b el\()\el\()_\label .endm .macro kernel_entry, el, regsize = 64 @@ -366,31 +366,31 @@ tsk .req x28 // current thread_info .align 11 ENTRY(vectors) - kernel_ventry el1_sync_invalid // Synchronous EL1t - kernel_ventry el1_irq_invalid // IRQ EL1t - kernel_ventry el1_fiq_invalid // FIQ EL1t - kernel_ventry el1_error_invalid // Error EL1t + kernel_ventry 1, sync_invalid // Synchronous EL1t + kernel_ventry 1, irq_invalid // IRQ EL1t + kernel_ventry 1, fiq_invalid // FIQ EL1t + kernel_ventry 1, error_invalid // Error EL1t - kernel_ventry el1_sync // Synchronous EL1h - kernel_ventry el1_irq // IRQ EL1h - kernel_ventry el1_fiq_invalid // FIQ EL1h - kernel_ventry el1_error_invalid // Error EL1h + kernel_ventry 1, sync // Synchronous EL1h + kernel_ventry 1, irq // IRQ EL1h + kernel_ventry 1, fiq_invalid // FIQ EL1h + kernel_ventry 1, error_invalid // Error EL1h - kernel_ventry el0_sync // Synchronous 64-bit EL0 - kernel_ventry el0_irq // IRQ 64-bit EL0 - kernel_ventry el0_fiq_invalid // FIQ 64-bit EL0 - kernel_ventry el0_error_invalid // Error 64-bit EL0 + kernel_ventry 0, sync // Synchronous 64-bit EL0 + kernel_ventry 0, irq // IRQ 64-bit EL0 + kernel_ventry 0, fiq_invalid // FIQ 64-bit EL0 + kernel_ventry 0, error_invalid // Error 64-bit EL0 #ifdef CONFIG_COMPAT - kernel_ventry el0_sync_compat // Synchronous 32-bit EL0 - kernel_ventry el0_irq_compat // IRQ 32-bit EL0 - kernel_ventry el0_fiq_invalid_compat // FIQ 32-bit EL0 - kernel_ventry el0_error_invalid_compat // Error 32-bit EL0 + kernel_ventry 0, sync_compat, 32 // Synchronous 32-bit EL0 + kernel_ventry 0, irq_compat, 32 // IRQ 32-bit EL0 + kernel_ventry 0, fiq_invalid_compat, 32 // FIQ 32-bit EL0 + kernel_ventry 0, error_invalid_compat, 32 // Error 32-bit EL0 #else - kernel_ventry el0_sync_invalid // Synchronous 32-bit EL0 - kernel_ventry el0_irq_invalid // IRQ 32-bit EL0 - kernel_ventry el0_fiq_invalid // FIQ 32-bit EL0 - kernel_ventry el0_error_invalid // Error 32-bit EL0 + kernel_ventry 0, sync_invalid, 32 // Synchronous 32-bit EL0 + kernel_ventry 0, irq_invalid, 32 // IRQ 32-bit EL0 + kernel_ventry 0, fiq_invalid, 32 // FIQ 32-bit EL0 + kernel_ventry 0, error_invalid, 32 // Error 32-bit EL0 #endif END(vectors) From patchwork Fri Nov 17 18:21:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119204 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp845772qgn; Fri, 17 Nov 2017 10:24:58 -0800 (PST) X-Google-Smtp-Source: AGs4zMZSKChmk8zjfmK9yKYvL4j+PT4dzDsJoku2+gsBbIkDqKcMHR34vjLNeSbQbRAya6E0INPz X-Received: by 10.159.218.66 with SMTP id x2mr4059656plv.326.1510943098820; Fri, 17 Nov 2017 10:24:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943098; cv=none; d=google.com; s=arc-20160816; b=i0IBgbXJEPXFxmQUtGm3YYSyALExlj9hPkT1+9XRGZWukqVJOIKef8+6cxALOzhdF6 XBh6kaIWUEdTNBSLQKAshkG5ViIld5J2rjxHODRRd/48HzqQ6dfdGa5l6m6o2j1xfcoE TBDI6NhKteftdq0knFrtBEiYiSHp/P7VpHKfJHCVxDp++4r0aqD5RuYVRBxLyN20aIn8 hXn4B6Nw1sJrXkFlkhtyQR0WlTXd+u5aXBlz4Qlka9FcQh7pF3x4/MYWMfly0qpZGoBH aTaMmQAX4hY/60M4Xs+eq7q4rPtL3tFv5WRtSo4N/O0JUXWowQYjb5XqWBqmkbe1TGnC LsWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=+UUaiSbTiwSkmk2v0Bra6AufRm8d5j9mO1jvW6Vz9UU=; b=h4N0e3D5dzDhSDCBOR5fZLdpVYR+jK1hsU9d6X7hFTZGYMUWrDWsOT4M4EzhKesFNL bzcUspMnS/sf/w5BFaeT+1YsQHC7mLOtpsINonhPLBvKb1kaNpf1n2BJaz02DV2U9zeQ xTz7CMVyy6uhnTbcRwxtwUqmC5nrjsfzjWLZShsFVK1HoIoEAdTQaDy7C3/dUFEEgWR5 +5WXOLL0qMPbAgnFkegUPJEHGgDe5qCpzhgYjVxhsRdxECsHWo5vennk1jdWIr5xtHIJ Hpq23+OvFbhPKOuiZrN/Y6we1DzfzE9tFmHP6kHfhY6rPB+f+kS0WNubiolws1dXJdy8 dKiA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y62si3088438pgb.383.2017.11.17.10.24.58; Fri, 17 Nov 2017 10:24:58 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760478AbdKQSY5 (ORCPT + 28 others); Fri, 17 Nov 2017 13:24:57 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39462 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760297AbdKQSV5 (ORCPT ); Fri, 17 Nov 2017 13:21:57 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3B11E199B; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0CFE63F5A0; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id AE3931AE1396; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 13/18] arm64: entry: Hook up entry trampoline to exception vectors Date: Fri, 17 Nov 2017 18:21:56 +0000 Message-Id: <1510942921-12564-14-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hook up the entry trampoline to our exception vectors so that all exceptions from and returns to EL0 go via the trampoline, which swizzles the vector base register accordingly. Transitioning to and from the kernel clobbers x30, so we use tpidrro_el0 and far_el1 as scratch registers for native tasks. Signed-off-by: Will Deacon --- arch/arm64/kernel/entry.S | 46 ++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 40 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index e98cf3064509..a839b94bba05 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -72,6 +72,16 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + .if \el == 0 + .if \regsize == 64 + mrs x30, tpidrro_el0 + .else + mov x30, xzr + .endif + .endif +#endif + sub sp, sp, #S_FRAME_SIZE #ifdef CONFIG_VMAP_STACK /* @@ -118,6 +128,11 @@ b el\()\el\()_\label .endm + .macro tramp_alias, dst, sym + mov_q \dst, TRAMP_VALIAS + add \dst, \dst, #(\sym - .entry.tramp.text) + .endm + .macro kernel_entry, el, regsize = 64 .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 @@ -265,25 +280,39 @@ alternative_else_nop_endif 2: #endif + msr elr_el1, x21 // set up the return data + msr spsr_el1, x22 + ldr lr, [sp, #S_LR] + .if \el == 0 ldr x23, [sp, #S_SP] // load return stack pointer msr sp_el0, x23 + tbz x22, #4, 3f + #ifdef CONFIG_ARM64_ERRATUM_845719 alternative_if ARM64_WORKAROUND_845719 - tbz x22, #4, 1f #ifdef CONFIG_PID_IN_CONTEXTIDR mrs x29, contextidr_el1 msr contextidr_el1, x29 #else msr contextidr_el1, xzr #endif -1: alternative_else_nop_endif #endif +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + tramp_alias x30, tramp_exit_compat + b 4f +3: + msr tpidrro_el0, xzr + msr far_el1, x30 + tramp_alias x30, tramp_exit_native +4: + prfm plil1strm, [x30] +#else +3: +#endif .endif - msr elr_el1, x21 // set up the return data - msr spsr_el1, x22 ldp x0, x1, [sp, #16 * 0] ldp x2, x3, [sp, #16 * 1] ldp x4, x5, [sp, #16 * 2] @@ -299,9 +328,14 @@ alternative_else_nop_endif ldp x24, x25, [sp, #16 * 12] ldp x26, x27, [sp, #16 * 13] ldp x28, x29, [sp, #16 * 14] - ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp - eret // return to kernel + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + .if \el == 0 + br x30 + .endif +#endif + eret .endm .macro irq_stack_entry From patchwork Fri Nov 17 18:21:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119196 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp843946qgn; Fri, 17 Nov 2017 10:22:57 -0800 (PST) X-Google-Smtp-Source: AGs4zMbsYjn3KYAFaaKjzulAhH1/XJ4fOmIgvC/v/wqUCgYcL3lk9/6xiCF8hBoQamdC38H0m2w2 X-Received: by 10.99.97.66 with SMTP id v63mr5723900pgb.84.1510942977634; Fri, 17 Nov 2017 10:22:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510942977; cv=none; d=google.com; s=arc-20160816; b=Ezeae+YqxkMf9x8wsTdCBy/UQwte4s8P/IHs6uIqc2sdj9l1n+cHAdh4D55UHXz+yP i2uzEf66+J3aST6DxAgnTym4ayVOjxhTdGs2rN2BfLpMemqMsRRSlzRCZXoT2T1qxMND kfVHxjYazzq6n00AlTd9H9+DdNbLXFvaFGNEazMLLIKrd79fghed+jFGopf+/wpEIQgi JHKFXQs2FYQScWqZZWdq1S8rPD497Lcc5ktNRaQI8jJuWMGbmVRDadNbynem8VFG/Ioy 1TF2CgNhPR/mm4UbJVAH7TGfjH8kDqpC9eu7emfqM4E8cAtCTWfba9/JnTnSbO4le80P aLxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=dYak26K3sstZnUZAlKJH7mYKPDTAe3NjCY/sd9YQXjA=; b=id42YAz2AHh+r6tn6nSTMQl+ny56IdygzNQkl/DxJ2kNL22exeOoLjkYGpWfbxDn+p R92Qf4NVQsyMxnIpOdMJt4BwpLka6albwfxyjfHJJwGJ/d1ZAJY5dguDdmiQAfkiK1q8 ZgYkN1h5MIVIZRZrJB8vTqYCGJKqmxBEbry3fAoVjCeTE8ijdrHf2/7VtiHgK8y/FShN WZNcONvXQ0E4nMRJP8CKIQzo/VJCYybuLyVpHKLurADfeNtgGx+CyVZKEQu4XLMvfs9N oWhBfUixlxUmUqypuRBlXAbJc69zuzU5Ww4sInTsP2pKZKg9nG7nS0mda7y07bEF8mZZ KmDA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e11si3071081pgr.80.2017.11.17.10.22.57; Fri, 17 Nov 2017 10:22:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760281AbdKQSWx (ORCPT + 28 others); Fri, 17 Nov 2017 13:22:53 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39472 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760302AbdKQSV7 (ORCPT ); Fri, 17 Nov 2017 13:21:59 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4412319CC; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 15F513F703; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id BE8081AE13AB; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 14/18] arm64: erratum: Work around Falkor erratum #E1003 in trampoline code Date: Fri, 17 Nov 2017 18:21:57 +0000 Message-Id: <1510942921-12564-15-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We rely on an atomic swizzling of TTBR1 when transitioning from the entry trampoline to the kernel proper on an exception. We can't rely on this atomicity in the face of Falkor erratum #E1003, so on affected cores we can issue a TLB invalidation prior to jumping into the kernel. There is still the possibility of a TLB conflict here due to conflicting walk cache entries, but this doesn't appear to be the case on these CPUs in practice. Signed-off-by: Will Deacon --- arch/arm64/Kconfig | 17 +++++------------ arch/arm64/kernel/entry.S | 8 ++++++++ 2 files changed, 13 insertions(+), 12 deletions(-) -- 2.1.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 0df64a6a56d4..f0fcbfc2262e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -504,20 +504,13 @@ config CAVIUM_ERRATUM_30115 config QCOM_FALKOR_ERRATUM_1003 bool "Falkor E1003: Incorrect translation due to ASID change" default y - select ARM64_PAN if ARM64_SW_TTBR0_PAN help On Falkor v1, an incorrect ASID may be cached in the TLB when ASID - and BADDR are changed together in TTBRx_EL1. The workaround for this - issue is to use a reserved ASID in cpu_do_switch_mm() before - switching to the new ASID. Saying Y here selects ARM64_PAN if - ARM64_SW_TTBR0_PAN is selected. This is done because implementing and - maintaining the E1003 workaround in the software PAN emulation code - would be an unnecessary complication. The affected Falkor v1 CPU - implements ARMv8.1 hardware PAN support and using hardware PAN - support versus software PAN emulation is mutually exclusive at - runtime. - - If unsure, say Y. + and BADDR are changed together in TTBRx_EL1. Since we keep the ASID + in TTBR1_EL1, this situation only occurs in the entry trampoline and + then only for entries in the walk cache, since the leaf translation + is unchanged. Work around the erratum by invalidating the walk cache + entries for the trampoline before entering the kernel proper. config QCOM_FALKOR_ERRATUM_1009 bool "Falkor E1009: Prematurely complete a DSB after a TLBI" diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index a839b94bba05..a600879939ce 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -941,6 +941,14 @@ __ni_sys_trace: sub \tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) bic \tmp, \tmp, #USER_ASID_FLAG msr ttbr1_el1, \tmp +alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003 + isb + movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS >> 12) + movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS >> 12) + movk \tmp, #:abs_g0_nc:(TRAMP_VALIAS >> 12) + tlbi vae1, \tmp + dsb nsh +alternative_else_nop_endif .endm .macro tramp_unmap_kernel, tmp From patchwork Fri Nov 17 18:21:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119201 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp845268qgn; Fri, 17 Nov 2017 10:24:28 -0800 (PST) X-Google-Smtp-Source: AGs4zMaV2d4xW7Z6SNRFKUB1VL1Ha2NFMQiifYOXKcmt8bzma4RtARrr3D89i2OgwLklFYNYaftZ X-Received: by 10.101.85.9 with SMTP id f9mr5947509pgr.263.1510943068066; Fri, 17 Nov 2017 10:24:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943068; cv=none; d=google.com; s=arc-20160816; b=KT90DAmJg3oG7tqKsjNQcvIKG9RQlQLjS0i4GnaGc1NGW0Y3YT9wrlqXxkCQOJf/ki auslLRpx7wsGo+jNv9GIVzaXFEtzmc1w7ZCfeodf55A5gscc3l2fCKCsOm0B19Iinq4/ +idh39oxw2035HDMiFi3UOMSPTkKZSNAun0qxe91ty/rxVeacoKaLhcLPQlXy5Qn5/6h JqUux29e8DpMJttEb+5Ob1Elg0aGijY4HS58HlDwpEgJi9b0YUJEWcx7LXPXh5JtW73B 5eMWeD9eGWu1jKXT1szvAhZMdYtkGl2kwgz8KTAUFxsofdAq1mdrjDN5/2qmGy28Zs3q lBnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=0ToogaDsbG1kH6QnLlZfj52hArCSoT6d86JumPJ2T0s=; b=VW/c4RaKS/nCFZNoCsj1uKFHEGzW0Tb9plPwfLjjRPlLrOQE1lHgbje0OGsYIZvt6y 4ySEY8fCZRMJVdc92fvuKsnawJy2B29fxkPSUGLgLb6cExnVx34NNSBKre6gVc/bIM6l 99UkY84NfOQotYF+ghaAdH+7Y33n7q0daH5RtLrpyCj/idPfPF3lK7Ir1P9dpy+1nfHW 2gBzEpHoJbk8b4bqYjBFKQb8x5PnpAnDjGf288H++hhV9Q4IYdyGoaA9xYoXQXXDly90 HlG09gQrdcnDxDBMWz2gD5BZbPSLXGL1zejnwbRpuAkI8bK3z7ZOrdtIZLHEJQG6L+LU HcMA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l5si3219397pli.480.2017.11.17.10.24.27; Fri, 17 Nov 2017 10:24:28 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760385AbdKQSYY (ORCPT + 28 others); Fri, 17 Nov 2017 13:24:24 -0500 Received: from foss.arm.com ([217.140.101.70]:39416 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760306AbdKQSV7 (ORCPT ); Fri, 17 Nov 2017 13:21:59 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6075F19E8; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 310FE3F7B0; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id CCC341AE13AC; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 15/18] arm64: tls: Avoid unconditional zeroing of tpidrro_el0 for native tasks Date: Fri, 17 Nov 2017 18:21:58 +0000 Message-Id: <1510942921-12564-16-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We zero tpidrro_el0 on return to the exception trampoline since it is used as a scratch register during exception entry. When the entry trampoline is being used, we can therefore avoid zeroing tpidrro_el0 in the context-switch for native tasks. Signed-off-by: Will Deacon --- arch/arm64/kernel/process.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) -- 2.1.4 diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 2dc0f8482210..c2841bda60be 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -305,16 +305,14 @@ void tls_preserve_current_state(void) static void tls_thread_switch(struct task_struct *next) { - unsigned long tpidr, tpidrro; - tls_preserve_current_state(); - tpidr = *task_user_tls(next); - tpidrro = is_compat_thread(task_thread_info(next)) ? - next->thread.tp_value : 0; + if (is_compat_thread(task_thread_info(next))) + write_sysreg(next->thread.tp_value, tpidrro_el0); + else if (arm64_kernel_mapped_at_el0()) + write_sysreg(0, tpidrro_el0); - write_sysreg(tpidr, tpidr_el0); - write_sysreg(tpidrro, tpidrro_el0); + write_sysreg(*task_user_tls(next), tpidr_el0); } /* Restore the UAO state depending on next's addr_limit */ From patchwork Fri Nov 17 18:21:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119199 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp844874qgn; Fri, 17 Nov 2017 10:24:03 -0800 (PST) X-Google-Smtp-Source: AGs4zMbyCxdG2uGcokVDHXiN369d8tAwew8qGsUC2boIr++rC0EEnhscWCMRcx9LhbHtUGiUMOjS X-Received: by 10.101.101.215 with SMTP id y23mr1833166pgv.391.1510943043598; Fri, 17 Nov 2017 10:24:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943043; cv=none; d=google.com; s=arc-20160816; b=qkcaNT4BYUHaMOXkrpk0Md+M9E7GRk33Z1eukKlXfaSh5GSNH1j4/eRJ0j/ymxe1b3 XidWf7cJrnmkdj1mLaj1vsuaE5/63pU+f/aER3nLPX7EWFiOQ+UeuN+mmpXEKZ+TQ8b6 m3y/nQAPnxqrgmcukjvwOLkN592G5rDo2PdSxO5rWJmME37Azte2JygUTVl3/YUkBpsa umPCocIIGhsxQUs/woL69Z/aGwoa2AEdgsd9Vd9Epbkrd6CsYSxN3nJUDN46bqUgyrAj N8WdAAIJjFkIBj0biEndXpvc79CKQuskvYdvD36NLaM+OvPAKBHEBBXAy5UYCGQ0yo5d z0JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=GkCEVdH/uyVDhAUKbxsjwuvUXBGmovacScxMmjviJ2o=; b=MvAzhV4ZPyk3EiJ1Wi0suECvEPjc/IpyVQE9RsmRXp0eRwynLa+cV0NWkda7Fpf/FJ +Elpmxg1KtkcVlu49YrlSV60ubfMDxQSAagie+qj5fYMwvbBNQEc09pscnNBg4N0EsCS ZwHN6kKOhZSfYBX2B3dTZwJLlUKMCEyHE0eZNmC4vO3gbHxVWZw9kjScoa0zAkNtHq+P lWsbo2tBesr3jSjZ9p/EPM7HbzHDyw3UWnbpTNxyILxLOg8qKE4dbGA9X3YZ+N1fpEo+ E1radSsPKmLkwll0Fz0hY5QiAL5r+s7h8nE9oJYUcHzGZUbiEQhd+wjHAiH70NJg558w wcXg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l5si3219397pli.480.2017.11.17.10.24.03; Fri, 17 Nov 2017 10:24:03 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752535AbdKQSX7 (ORCPT + 28 others); Fri, 17 Nov 2017 13:23:59 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39418 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760317AbdKQSWC (ORCPT ); Fri, 17 Nov 2017 13:22:02 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D903E1A9A; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id AA5E23F246; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id DB1601AE10B5; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 16/18] arm64: entry: Add fake CPU feature for mapping the kernel at EL0 Date: Fri, 17 Nov 2017 18:21:59 +0000 Message-Id: <1510942921-12564-17-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Allow explicit disabling of the entry trampoline on the kernel command line by adding a fake CPU feature (ARM64_MAP_KERNEL_AT_EL0) that can be used to apply alternative sequences to our entry code and avoid use of the trampoline altogether. Signed-off-by: Will Deacon --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/kernel/cpufeature.c | 11 +++++++++++ arch/arm64/kernel/entry.S | 6 +++++- arch/arm64/mm/mmu.c | 7 +++++++ 4 files changed, 25 insertions(+), 2 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 8da621627d7c..f61d85f76683 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -40,7 +40,8 @@ #define ARM64_WORKAROUND_858921 19 #define ARM64_WORKAROUND_CAVIUM_30115 20 #define ARM64_HAS_DCPOP 21 +#define ARM64_MAP_KERNEL_AT_EL0 22 -#define ARM64_NCAPS 22 +#define ARM64_NCAPS 23 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 21e2c95d24e7..aa6b90de6591 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -796,6 +796,12 @@ static bool has_no_fpsimd(const struct arm64_cpu_capabilities *entry, int __unus ID_AA64PFR0_FP_SHIFT) < 0; } +static bool map_kernel_at_el0(const struct arm64_cpu_capabilities *entry, + int __unused) +{ + return arm64_kernel_mapped_at_el0(); +} + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -883,6 +889,11 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .matches = hyp_offset_low, }, { + .capability = ARM64_MAP_KERNEL_AT_EL0, + .def_scope = SCOPE_SYSTEM, + .matches = map_kernel_at_el0, + }, + { /* FP/SIMD is not implemented */ .capability = ARM64_HAS_NO_FPSIMD, .def_scope = SCOPE_SYSTEM, diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index a600879939ce..a74253defc5b 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -73,6 +73,7 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if_not ARM64_MAP_KERNEL_AT_EL0 .if \el == 0 .if \regsize == 64 mrs x30, tpidrro_el0 @@ -80,6 +81,7 @@ mov x30, xzr .endif .endif +alternative_else_nop_endif #endif sub sp, sp, #S_FRAME_SIZE @@ -300,6 +302,7 @@ alternative_if ARM64_WORKAROUND_845719 alternative_else_nop_endif #endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if_not ARM64_MAP_KERNEL_AT_EL0 tramp_alias x30, tramp_exit_compat b 4f 3: @@ -308,6 +311,7 @@ alternative_else_nop_endif tramp_alias x30, tramp_exit_native 4: prfm plil1strm, [x30] +alternative_else_nop_endif #else 3: #endif @@ -332,7 +336,7 @@ alternative_else_nop_endif #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 .if \el == 0 - br x30 + alternative_insn "br x30", nop, ARM64_MAP_KERNEL_AT_EL0 .endif #endif eret diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 5ce5cb1249da..dab987f2912c 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -571,6 +571,13 @@ static int __init map_entry_trampoline(void) return 0; } core_initcall(map_entry_trampoline); + +static int __init parse_nokaiser(char *__unused) +{ + static_branch_disable(&__unmap_kernel_at_el0); + return 0; +} +__setup("nokaiser", parse_nokaiser); #else static void __init add_tramp_vma(void) {} #endif From patchwork Fri Nov 17 18:22:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119200 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp845037qgn; Fri, 17 Nov 2017 10:24:14 -0800 (PST) X-Google-Smtp-Source: AGs4zMbkefPfUTcDqYQYsex70H8gk5pOjqUnpqE5lYABWN0W9MyBSMLPAyINYoETqGZaOp/35gFd X-Received: by 10.159.252.11 with SMTP id n11mr6094196pls.207.1510943054310; Fri, 17 Nov 2017 10:24:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510943054; cv=none; d=google.com; s=arc-20160816; b=ciGEl4pgbZkXS0DiCBARDV4GDfvWbGi5kHqirimHPtA0QwL7K4v9MPwBgu/ni6I6sj j9MnHmsDK6z4h4WHWx7NUKV3+GPsOTm8l1hTMc0Sbz7WW3dNfQMRumU2an7MN8d2gy4k ZnraTVcbwmw9XgozZimJv6Acn2UthcnNvzkYhGB2RlPXi56WZpvoSiSsllXNJgsRSuhL 8IsS+NJdbj0CrqnkJFcCayg3NTl2djzai0FZaF3WMD+rT/1KnIc0UNeL4q+Hwo/HH8Bz qZTPo+aqxYlCK3hwQ3d4cnvzuzwv6owUEzv61G4ON0svZKwxTNnymghzejlGm4KQ4WwN DCZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=UrTemSpdZXVKugSbACUYRYNejcJnpujb/ZMFHTSiHDw=; b=HwhgTRcGbU31RSQhP7I7GHapb0Pl96OsMzNCYfUe/QUpAUEfu09DaLaOePyRwIiqLT 9tvamUFjRRFPRnqta2ol/tUSZkeIc60Utaj41s146haX/nIb7sx2Kb0/oOY7TXQA+0ab CAZnPU6KMGpKNNDFV1MlSUZMmH6ueCozYBa694vUAC/MvUxgr8ancsG0KIK9xbgQc6hc oW4COA1YJqWvsiofBgJ1LoOAsLACmO+TE5DU80ty4DleCAO2Fg0HhosctCntoA9Yp1mk 59Z/GpaIbXw16lnTpbnjd9roS7BJtnj06qvb2nhFEJUZ4OWa3rYWvXe2ElRukQ9cGAye USrA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l5si3219397pli.480.2017.11.17.10.24.14; Fri, 17 Nov 2017 10:24:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1760358AbdKQSYM (ORCPT + 28 others); Fri, 17 Nov 2017 13:24:12 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:39476 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760307AbdKQSWA (ORCPT ); Fri, 17 Nov 2017 13:22:00 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 685BE19F6; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3A3DC3F246; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id E7D211AE10AF; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 17/18] arm64: makefile: Ensure TEXT_OFFSET doesn't overlap with trampoline Date: Fri, 17 Nov 2017 18:22:00 +0000 Message-Id: <1510942921-12564-18-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When CONFIG_ARM64_RANDOMIZE_TEXT_OFFSET=y, we could end up with a TEXT_OFFSET of less than 2 * PAGE_SIZE, which would result in an overlap with the trampoline and a panic on boot. Fix this by restricting the minimum value of the random TEXT_OFFSET value so that it is not less than 2 pages when CONFIG_UNMAP_KERNEL_AT_EL0 is enabled. I do wonder whether we should just remove CONFIG_ARM64_RANDOMIZE_TEXT_OFFSET completely, since we're realistically never going to be able to change our offset from 0x80000, but this keeps the dream alive for now. Signed-off-by: Will Deacon --- arch/arm64/Makefile | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) -- 2.1.4 diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 939b310913cf..b60ac6c43ccd 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -87,9 +87,21 @@ head-y := arch/arm64/kernel/head.o # The byte offset of the kernel image in RAM from the start of RAM. ifeq ($(CONFIG_ARM64_RANDOMIZE_TEXT_OFFSET), y) -TEXT_OFFSET := $(shell awk "BEGIN {srand(); printf \"0x%06x\n\", \ - int(2 * 1024 * 1024 / (2 ^ $(CONFIG_ARM64_PAGE_SHIFT)) * \ - rand()) * (2 ^ $(CONFIG_ARM64_PAGE_SHIFT))}") +TEXT_OFFSET := $(shell awk \ + "BEGIN { \ + srand(); \ + page_size = 2 ^ $(CONFIG_ARM64_PAGE_SHIFT); \ + tramp_size = 0; \ + if (\" $(CONFIG_UNMAP_KERNEL_AT_EL0)\" == \" y\") { \ + tramp_size = 2 * page_size; \ + } \ + offset = int(2 * 1024 * 1024 / page_size * rand()); \ + offset *= page_size; \ + if (offset < tramp_size) { \ + offset = tramp_size; \ + } \ + printf \"0x%06x\n\", offset; \ + }") else TEXT_OFFSET := 0x00080000 endif From patchwork Fri Nov 17 18:22:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 119197 Delivered-To: patch@linaro.org Received: by 10.140.22.164 with SMTP id 33csp844116qgn; Fri, 17 Nov 2017 10:23:09 -0800 (PST) X-Google-Smtp-Source: AGs4zMZC5RJMgEHYDOvqiAkKUqilPoKnJ5/Qn9R5s2riQ4P0dDurCMYnbJdb5CDfSNpt2IAZWuPt X-Received: by 10.99.108.66 with SMTP id h63mr5962969pgc.362.1510942989102; Fri, 17 Nov 2017 10:23:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1510942989; cv=none; d=google.com; s=arc-20160816; b=P84rocyyRQyYesoKXUVhdLDOiq5in5aiSO9r71KMgM1R2yLsjm2Bz1VpWU0aE92xPj b7kSv/rcaFC98pNlLgZ7OOKoUYwV1MFBSby+wxu9NhGw8f+ncRLx37SVMxzWpCyk/zRw IjV4Mlp8TQJY4fuU5/nXEU3Eg9g+1BcFbd78/D++cNsDQKwFA6szaf6sJYM6joZLhY5H ERtesCr/BMGBsX/uA23dkkaViC5VI+Tg3jr5tbvQ+8Y5qmlvIVJZvA6CAHrwc+wgbTX+ +6MDPLc5t628Th4HwDJjdrdM/2K+b1Eer95DlN0d2T+jL5oCaLz6Bqle4nwcX4RxTkH2 ENmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=qsKfeItzhCYGiPJQ8gapOR1rLd52bThpe+fuui1MwIg=; b=T3hcCUPyzyXXe4BgEZLJDNwz7kyTwNEMIvikB856BH1Wm8RiDYbUpxHOY+5ERN+xvI fJsatv+uh2bMvnqm4MFs3udNydyBCSSm3l1W+owhjAr8ApQBoSu1326AKZ8blxWRjTKL EkrfkaxCt76XFhQjEdbhtomCam14w03sETHFJNLSVErh37OmoVI9lvh9CCJedtPBlTMn jZpJ68cDjsLmR9LE0OAtaPzzunbieJtX8NY8UcGF9jlCTBr57rHMYoAFH4fP38e/XUIV so7frw/NW8dCeFp7oynNn8i3/qQUnvO8BOXaVyMvkT4866btslz8DnENulbpOTX9q6IR dx6w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g12si3179518pla.762.2017.11.17.10.23.08; Fri, 17 Nov 2017 10:23:09 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752102AbdKQSXF (ORCPT + 28 others); Fri, 17 Nov 2017 13:23:05 -0500 Received: from foss.arm.com ([217.140.101.70]:39478 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760319AbdKQSWD (ORCPT ); Fri, 17 Nov 2017 13:22:03 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E1EE61AC1; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B37463F599; Fri, 17 Nov 2017 10:21:55 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 03F941AE17F5; Fri, 17 Nov 2017 18:22:03 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, Will Deacon Subject: [PATCH 18/18] arm64: Kconfig: Add CONFIG_UNMAP_KERNEL_AT_EL0 Date: Fri, 17 Nov 2017 18:22:01 +0000 Message-Id: <1510942921-12564-19-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1510942921-12564-1-git-send-email-will.deacon@arm.com> References: <1510942921-12564-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add a Kconfig entry to control use of the entry trampoline, which allows us to unmap the kernel whilst running in userspace and improve the robustness of KASLR. Signed-off-by: Will Deacon --- arch/arm64/Kconfig | 13 +++++++++++++ 1 file changed, 13 insertions(+) -- 2.1.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index f0fcbfc2262e..f99ffb88843a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -796,6 +796,19 @@ config FORCE_MAX_ZONEORDER However for 4K, we choose a higher default value, 11 as opposed to 10, giving us 4M allocations matching the default size used by generic code. +config UNMAP_KERNEL_AT_EL0 + bool "Unmap kernel when running in userspace (aka \"KAISER\")" + default y + help + Some attacks against KASLR make use of the timing difference between + a permission fault which could arise from a page table entry that is + present in the TLB, and a translation fault which always requires a + page table walk. This option defends against these attacks by unmapping + the kernel whilst running in userspace, therefore forcing translation + faults for all of kernel space. + + If unsure, say Y. + menuconfig ARMV8_DEPRECATED bool "Emulate deprecated/obsolete ARMv8 instructions" depends on COMPAT