From patchwork Thu Nov 30 16:39:29 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120221 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp13223qgn; Thu, 30 Nov 2017 08:39:52 -0800 (PST) X-Google-Smtp-Source: AGs4zMbW+HQOqeowsFylPrk7XrXkNTnsKFaOcsg+RGJKM2hAnD9+JTBoSXkclLGNMcKSPeZ4HKEs X-Received: by 10.99.110.201 with SMTP id j192mr2960346pgc.70.1512059992349; Thu, 30 Nov 2017 08:39:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512059992; cv=none; d=google.com; s=arc-20160816; b=hl1l3zm39ak50y3nGYzbg0su3/bDCrc6991A6hUHQOctJ+7F9ZyIuL2XYKL552+ukI Dv5ciYcdxFzVz95ob8JsTxlapMEUla1vyNF0RYY3yYlOi2ImolVv4s4jeG7tErZbfpcr 5KIPn4/M7iStSulTNVMmIiKaWMyTZmQvn3dNDMr3q15vcofpstx/l7vLay0X6NOoOgnu tZjU8EqhzfCVxHXxuc5qS24v8di3MIWiNOqfHp+CSUzymPrQueb0J2GdwGMebrud1Yua ceL/PlNQ+8TPTCvwBUmkN7H8EMZLEWbkRGMZFmHBQBa3/Y762sQ0lLBsb4/5MsIZSbET FcVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=bDYAh19mDeUOIt604CmIEM9tIuAbPFgsMJp7ZAGTsiY=; b=jcRTRry6kZs4NGSiCkPzESq0R5YKbQJuGPYp7lnW6P/RSPN2TuA5gWHCXSj2LrfVTD 9B04cvdBErLR07ogKEBQPao1zugtTO29aQ5TqxqG2F9DnPx+wx3qPEek9GmpcMomZOlo VZCAKxKHeh+fTYXH2t7KY6XqvT4/W9td7LxHIugRIejHvp3ACZD5ZuTlQckFA0ENOQpH lY3OB4vXY4Ox+yNWFPQHGlBgub7JMi7HKmBYqXz7Ec+nWnF3zV1mDksbxzfUACH6vAKK uHQdEq7XvbRL3XBMM+jpTwjM+hKdzBLFUCL/42w2MGnqdFM9NBpb6pYAJ7LYJqoT/J9n TmAA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id p3si3257145pga.135.2017.11.30.08.39.52; Thu, 30 Nov 2017 08:39:52 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753427AbdK3Qjv (ORCPT + 28 others); Thu, 30 Nov 2017 11:39:51 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57364 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752431AbdK3Qjo (ORCPT ); Thu, 30 Nov 2017 11:39:44 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 770F51596; Thu, 30 Nov 2017 08:39:44 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4844E3F5B3; Thu, 30 Nov 2017 08:39:44 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 878981AE3BB5; Thu, 30 Nov 2017 16:39:47 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 01/18] arm64: mm: Use non-global mappings for kernel space Date: Thu, 30 Nov 2017 16:39:29 +0000 Message-Id: <1512059986-21325-2-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation for unmapping the kernel whilst running in userspace, make the kernel mappings non-global so we can avoid expensive TLB invalidation on kernel exit to userspace. Signed-off-by: Will Deacon --- arch/arm64/include/asm/kernel-pgtable.h | 12 ++++++++++-- arch/arm64/include/asm/pgtable-prot.h | 21 +++++++++++++++------ 2 files changed, 25 insertions(+), 8 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index 7803343e5881..77a27af01371 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -78,8 +78,16 @@ /* * Initial memory map attributes. */ -#define SWAPPER_PTE_FLAGS (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) -#define SWAPPER_PMD_FLAGS (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) +#define _SWAPPER_PTE_FLAGS (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) +#define _SWAPPER_PMD_FLAGS (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define SWAPPER_PTE_FLAGS (_SWAPPER_PTE_FLAGS | PTE_NG) +#define SWAPPER_PMD_FLAGS (_SWAPPER_PMD_FLAGS | PMD_SECT_NG) +#else +#define SWAPPER_PTE_FLAGS _SWAPPER_PTE_FLAGS +#define SWAPPER_PMD_FLAGS _SWAPPER_PMD_FLAGS +#endif #if ARM64_SWAPPER_USES_SECTION_MAPS #define SWAPPER_MM_MMUFLAGS (PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS) diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h index 0a5635fb0ef9..22a926825e3f 100644 --- a/arch/arm64/include/asm/pgtable-prot.h +++ b/arch/arm64/include/asm/pgtable-prot.h @@ -34,8 +34,16 @@ #include -#define PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) -#define PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) +#define _PROT_DEFAULT (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED) +#define _PROT_SECT_DEFAULT (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S) + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define PROT_DEFAULT (_PROT_DEFAULT | PTE_NG) +#define PROT_SECT_DEFAULT (_PROT_SECT_DEFAULT | PMD_SECT_NG) +#else +#define PROT_DEFAULT _PROT_DEFAULT +#define PROT_SECT_DEFAULT _PROT_SECT_DEFAULT +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ #define PROT_DEVICE_nGnRnE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRnE)) #define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRE)) @@ -48,6 +56,7 @@ #define PROT_SECT_NORMAL_EXEC (PROT_SECT_DEFAULT | PMD_SECT_UXN | PMD_ATTRINDX(MT_NORMAL)) #define _PAGE_DEFAULT (PROT_DEFAULT | PTE_ATTRINDX(MT_NORMAL)) +#define _HYP_PAGE_DEFAULT (_PAGE_DEFAULT & ~PTE_NG) #define PAGE_KERNEL __pgprot(_PAGE_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE) #define PAGE_KERNEL_RO __pgprot(_PAGE_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_RDONLY) @@ -55,15 +64,15 @@ #define PAGE_KERNEL_EXEC __pgprot(_PAGE_DEFAULT | PTE_UXN | PTE_DIRTY | PTE_WRITE) #define PAGE_KERNEL_EXEC_CONT __pgprot(_PAGE_DEFAULT | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_CONT) -#define PAGE_HYP __pgprot(_PAGE_DEFAULT | PTE_HYP | PTE_HYP_XN) -#define PAGE_HYP_EXEC __pgprot(_PAGE_DEFAULT | PTE_HYP | PTE_RDONLY) -#define PAGE_HYP_RO __pgprot(_PAGE_DEFAULT | PTE_HYP | PTE_RDONLY | PTE_HYP_XN) +#define PAGE_HYP __pgprot(_HYP_PAGE_DEFAULT | PTE_HYP | PTE_HYP_XN) +#define PAGE_HYP_EXEC __pgprot(_HYP_PAGE_DEFAULT | PTE_HYP | PTE_RDONLY) +#define PAGE_HYP_RO __pgprot(_HYP_PAGE_DEFAULT | PTE_HYP | PTE_RDONLY | PTE_HYP_XN) #define PAGE_HYP_DEVICE __pgprot(PROT_DEVICE_nGnRE | PTE_HYP) #define PAGE_S2 __pgprot(PROT_DEFAULT | PTE_S2_MEMATTR(MT_S2_NORMAL) | PTE_S2_RDONLY) #define PAGE_S2_DEVICE __pgprot(PROT_DEFAULT | PTE_S2_MEMATTR(MT_S2_DEVICE_nGnRE) | PTE_S2_RDONLY | PTE_UXN) -#define PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_VALID) | PTE_PROT_NONE | PTE_RDONLY | PTE_PXN | PTE_UXN) +#define PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_VALID) | PTE_PROT_NONE | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN) #define PAGE_SHARED __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE) #define PAGE_SHARED_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_WRITE) #define PAGE_READONLY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN) From patchwork Thu Nov 30 16:39:30 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120222 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp14169qgn; Thu, 30 Nov 2017 08:40:42 -0800 (PST) X-Google-Smtp-Source: AGs4zMaB9Bl5Z5o0qasDlKm05u8iHEfvC2OeXKl1gyiMI4oQa1Poy7YvvDC2JOAA9Tu3RXD0k+Vq X-Received: by 10.98.223.217 with SMTP id d86mr7298109pfl.190.1512060042648; Thu, 30 Nov 2017 08:40:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060042; cv=none; d=google.com; s=arc-20160816; b=thDDNbrbZI0jeln7PHTP6/jIn8XCOQ3wLw+Ulb1voIsXtWRA/3T7TBdHZeGG+qCEEa QLMFdxRDr1BSGrqCROWQkBmwkSzLpPTRT+18eOewii1goNLyOIFB5Kx3hUtRcMWD08fB 7sSiK+dwjDrCNU7/FGiNr9z94PEqWnIvLK14QiGb4w2Z4+FnoMA7t7/SCG/MzhZnfVfb c7gAFbHysEkKALqby688Rq/2Rn9yU8ivJyN8LsIgkEHtXt7Y72xdQDnBcF9CyYDew/fj BCWVACNDGHa2a7CIwG3ZIl6nQ36gpO49dEW3mzbEfo+4awqFFkydBEQtdPmWRip0bXXJ bUGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=jSPGZ676VPTSO1PYJhvI0y7XNviHi6dmfs5116hVDXI=; b=hUjmGqLbyVH3goTclrlKa0GU+h2kN+B9uYf0+UbhMzHjx3ABtePbd3D2F04ekqw65u QuPEra7KfudnSLKqNKEp8cnOEi0/A7jCpeasyOPVGwEYYnuh7BT+dDomI9VuG1sjSkq0 QimTqKq4vLMeUISc3+ca4EHXNkxmCBLqnuh58zB/lQo1QLpbvcXGiKwhztFYnL+kY8wy ZeS1qW99oif1/nulve/SLFv84yZvswNJpfoyxYKlwX0TogEmChAe2vLBsSUEpvCvdudD uldOHsySMDcQdTCMAVSzHccW73hYs842G8Ca8XlWkpAPHGeXFwrHLrczP02Wl34XjluG 1oCA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h72si3532928pfj.20.2017.11.30.08.40.19; Thu, 30 Nov 2017 08:40:42 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753408AbdK3Qjt (ORCPT + 28 others); Thu, 30 Nov 2017 11:39:49 -0500 Received: from foss.arm.com ([217.140.101.70]:57384 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753037AbdK3Qjo (ORCPT ); Thu, 30 Nov 2017 11:39:44 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 85F5915A2; Thu, 30 Nov 2017 08:39:44 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 576E43F5B7; Thu, 30 Nov 2017 08:39:44 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 977161AE3BC9; Thu, 30 Nov 2017 16:39:47 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 02/18] arm64: mm: Temporarily disable ARM64_SW_TTBR0_PAN Date: Thu, 30 Nov 2017 16:39:30 +0000 Message-Id: <1512059986-21325-3-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We're about to rework the way ASIDs are allocated, switch_mm is implemented and low-level kernel entry/exit is handled, so keep the ARM64_SW_TTBR0_PAN code out of the way whilst we do the heavy lifting. It will be re-enabled in a subsequent patch. Signed-off-by: Will Deacon --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) -- 2.1.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a93339f5178f..7e7d7fd152c4 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -910,6 +910,7 @@ endif config ARM64_SW_TTBR0_PAN bool "Emulate Privileged Access Never using TTBR0_EL1 switching" + depends on BROKEN # Temporary while switch_mm is reworked help Enabling this option prevents the kernel from accessing user-space memory directly by pointing TTBR0_EL1 to a reserved From patchwork Thu Nov 30 16:39:31 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120235 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp20692qgn; Thu, 30 Nov 2017 08:46:34 -0800 (PST) X-Google-Smtp-Source: AGs4zMbo4ja6coeVihd/Y0pH7OREs4e/4cx1MWRYYK+06orm1tEvSCadzmfpVbmEVUWUTm7VH1xB X-Received: by 10.159.214.146 with SMTP id n18mr3351443plp.310.1512060394058; Thu, 30 Nov 2017 08:46:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060394; cv=none; d=google.com; s=arc-20160816; b=dzR6mNA4RhR29iw0H66Lc/QFdSE+3YlXkvd40ms34RDZxXz2qgXbfUnTn/vaCUIz9Q XhI36S34yYT8T0pjqcOEf9jCXZK6J985BcxCVtH8pFLq55cHeESWCKmasd26fTs48Fi/ HygySHFeDPTXpYY92HfWpbDeQUWD7czZ1o6aZDMH4IdWUK0M5zyS6MOb0o6YYz6Xb7Ki H/00kM8/siJPtU8OS48jQ5vhza3uP18o+Lp2nxiryDn2UoB1WHOrhU2o+7eMnChwtl6V VQeGuLup9Lgx8nYye5YjXIBdenilBwf2eN/F+K/VzGYidljOBmiAY7OoSppQu01+90n3 u6Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=EW5Xa6a6OclLFhj0heQvM6HaWM/n9wJa92XP/x+rHEI=; b=MqE5A6po0RI2s5oMJPtRDygcfqjEHHArrDUvhJYHE6ijw8K5/VdjStSK/00IjovOpY 0JUdQiNGNw+PLRtwBHYle1r2A822EPc5Ma8iFG3Z2fvX4Mv8UREOjacGXEzD/MYLzezn 3TEAciAJ/Uze9P1+qXskPPrE2q/2CHwOPth3nZoQZt+JgrCt0TLzWxzRm/SsY16RT3t6 MUfzYd1xwVF00OkBwOWTvQ+gwSP8nnCwdOPmByok6L8pTtL9wnBQ+b2dAJfO+sWs7ayH cxbf69V0FbkxkCAzhu76CfO10LP7F3L4AzkG56voymEDBClWYAgWmFUbXhJlBivDKI4K kukg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w5si3213637pgm.49.2017.11.30.08.46.33; Thu, 30 Nov 2017 08:46:34 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753732AbdK3QqR (ORCPT + 28 others); Thu, 30 Nov 2017 11:46:17 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57404 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753042AbdK3Qjp (ORCPT ); Thu, 30 Nov 2017 11:39:45 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 93ABC15BE; Thu, 30 Nov 2017 08:39:44 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6507B3F774; Thu, 30 Nov 2017 08:39:44 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id A6D3B1AE3BDF; Thu, 30 Nov 2017 16:39:47 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 03/18] arm64: mm: Move ASID from TTBR0 to TTBR1 Date: Thu, 30 Nov 2017 16:39:31 +0000 Message-Id: <1512059986-21325-4-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation for mapping kernelspace and userspace with different ASIDs, move the ASID to TTBR1 and update switch_mm to context-switch TTBR0 via an invalid mapping (the zero page). Signed-off-by: Will Deacon --- arch/arm64/include/asm/mmu_context.h | 7 +++++++ arch/arm64/include/asm/pgtable-hwdef.h | 1 + arch/arm64/include/asm/proc-fns.h | 6 ------ arch/arm64/mm/proc.S | 9 ++++++--- 4 files changed, 14 insertions(+), 9 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 3257895a9b5e..56723bcbfaaa 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -37,6 +37,13 @@ #include #include +#define cpu_switch_mm(pgd,mm) \ +do { \ + BUG_ON(pgd == swapper_pg_dir); \ + cpu_set_reserved_ttbr0(); \ + cpu_do_switch_mm(virt_to_phys(pgd),mm); \ +} while (0) + static inline void contextidr_thread_switch(struct task_struct *next) { if (!IS_ENABLED(CONFIG_PID_IN_CONTEXTIDR)) diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index eb0c2bd90de9..8df4cb6ac6f7 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -272,6 +272,7 @@ #define TCR_TG1_4K (UL(2) << TCR_TG1_SHIFT) #define TCR_TG1_64K (UL(3) << TCR_TG1_SHIFT) +#define TCR_A1 (UL(1) << 22) #define TCR_ASID16 (UL(1) << 36) #define TCR_TBI0 (UL(1) << 37) #define TCR_HA (UL(1) << 39) diff --git a/arch/arm64/include/asm/proc-fns.h b/arch/arm64/include/asm/proc-fns.h index 14ad6e4e87d1..16cef2e8449e 100644 --- a/arch/arm64/include/asm/proc-fns.h +++ b/arch/arm64/include/asm/proc-fns.h @@ -35,12 +35,6 @@ extern u64 cpu_do_resume(phys_addr_t ptr, u64 idmap_ttbr); #include -#define cpu_switch_mm(pgd,mm) \ -do { \ - BUG_ON(pgd == swapper_pg_dir); \ - cpu_do_switch_mm(virt_to_phys(pgd),mm); \ -} while (0) - #endif /* __ASSEMBLY__ */ #endif /* __KERNEL__ */ #endif /* __ASM_PROCFNS_H */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 95233dfc4c39..a8a64898a2aa 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -139,9 +139,12 @@ ENDPROC(cpu_do_resume) */ ENTRY(cpu_do_switch_mm) pre_ttbr0_update_workaround x0, x2, x3 + mrs x2, ttbr1_el1 mmid x1, x1 // get mm->context.id - bfi x0, x1, #48, #16 // set the ASID - msr ttbr0_el1, x0 // set TTBR0 + bfi x2, x1, #48, #16 // set the ASID + msr ttbr1_el1, x2 // in TTBR1 (since TCR.A1 is set) + isb + msr ttbr0_el1, x0 // now update TTBR0 isb post_ttbr0_update_workaround ret @@ -224,7 +227,7 @@ ENTRY(__cpu_setup) * both user and kernel. */ ldr x10, =TCR_TxSZ(VA_BITS) | TCR_CACHE_FLAGS | TCR_SMP_FLAGS | \ - TCR_TG_FLAGS | TCR_ASID16 | TCR_TBI0 + TCR_TG_FLAGS | TCR_ASID16 | TCR_TBI0 | TCR_A1 tcr_set_idmap_t0sz x10, x9 /* From patchwork Thu Nov 30 16:39:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120234 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp20438qgn; Thu, 30 Nov 2017 08:46:21 -0800 (PST) X-Google-Smtp-Source: AGs4zMacOcp1FuIvoXTW8gKS4VyGJysueUksfa+DvMZE2d96XrWdfDTjBWy3kbXLq5bJoh8cSsZO X-Received: by 10.98.35.92 with SMTP id j89mr7157082pfj.49.1512060380803; Thu, 30 Nov 2017 08:46:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060380; cv=none; d=google.com; s=arc-20160816; b=twE9P1jbzV5pEdxo5j+B2pcS+AkG/H+UnzpWHHvDcVSKSARuh6RXJACGctaykUG59C iQcXGbl8/OYDBlMtmruJYov1uLtiJdYeS5nY1E4bIyw9+5wrPpvGMQujZYS9ZLZp3Sbu zeR39HFH+utGkgAEC+lutKSjtjQV0WF18gu3yTyb8JGhrpZ5fEeEL6Lh7v2xq630FWjG cCL4rk1yg7u9sKPgZULaztSa11/bJjn+fBUv/ymt23+YsuAEBOIptoHJcXMbFWuLcc6D zen/VtqNgB+uZVNoX+GgbPE8NMdwo8MgCCcybo0h2FZ4B+QuqHSQmiN17R06uEseFx9m i++Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=pyLsIRtLfE8yhR4hkSuG7iKHcpOblB/s79N8032ZVCc=; b=hXhAOwSfREyPYG/1XRtwXSTh2l1uFofmnYg2vTy4mBb3MZZvqYoezKNRWTZ4iw97Uj Lub0Dj5udLMFlJnsxtjLXEZo8Np0xvDdomka/+nzxm2Sal0dZRjSKiSdLMe4dl4IHXBo svHl1guljYUuRm10Oo4KYrJZRwCsT/bzeIAQl/b47EsJp6AlTLloIInO6+bbXph5sP+c O6yoRBgLKIBvMpS3DODbn7rBhUP0ECsY7gLmF9DS5HqqDGRnzNB3wBaxwwyudVS5Qin9 3fv04JuQgXWEWM4QrL/VmkfRnzNjseVGXR2z8sGwRoFX/SH0DAOdP5MtRMMggrKLbkwl PHbg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w5si3213637pgm.49.2017.11.30.08.46.20; Thu, 30 Nov 2017 08:46:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753745AbdK3QqT (ORCPT + 28 others); Thu, 30 Nov 2017 11:46:19 -0500 Received: from foss.arm.com ([217.140.101.70]:57428 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753226AbdK3Qjo (ORCPT ); Thu, 30 Nov 2017 11:39:44 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A49451610; Thu, 30 Nov 2017 08:39:44 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 754EC3F318; Thu, 30 Nov 2017 08:39:44 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id B4DC01AE3C18; Thu, 30 Nov 2017 16:39:47 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 04/18] arm64: mm: Remove pre_ttbr0_update_workaround for Falkor erratum #E1003 Date: Thu, 30 Nov 2017 16:39:32 +0000 Message-Id: <1512059986-21325-5-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The pre_ttbr0_update_workaround hook is called prior to context-switching TTBR0 because Falkor erratum E1003 can cause TLB allocation with the wrong ASID if both the ASID and the base address of the TTBR are updated at the same time. With the ASID sitting safely in TTBR1, we no longer update things atomically, so we can remove the pre_ttbr0_update_workaround macro as it's no longer required. The erratum infrastructure and documentation is left around for #E1003, as it will be required by the entry trampoline code in a future patch. Signed-off-by: Will Deacon --- arch/arm64/include/asm/assembler.h | 22 ---------------------- arch/arm64/include/asm/mmu_context.h | 2 -- arch/arm64/mm/context.c | 11 ----------- arch/arm64/mm/proc.S | 1 - 4 files changed, 36 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index aef72d886677..e1fa5db858b7 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -26,7 +26,6 @@ #include #include #include -#include #include #include #include @@ -478,27 +477,6 @@ alternative_endif .endm /* - * Errata workaround prior to TTBR0_EL1 update - * - * val: TTBR value with new BADDR, preserved - * tmp0: temporary register, clobbered - * tmp1: other temporary register, clobbered - */ - .macro pre_ttbr0_update_workaround, val, tmp0, tmp1 -#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003 -alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003 - mrs \tmp0, ttbr0_el1 - mov \tmp1, #FALKOR_RESERVED_ASID - bfi \tmp0, \tmp1, #48, #16 // reserved ASID + old BADDR - msr ttbr0_el1, \tmp0 - isb - bfi \tmp0, \val, #0, #48 // reserved ASID + new BADDR - msr ttbr0_el1, \tmp0 - isb -alternative_else_nop_endif -#endif - .endm - /* * Errata workaround post TTBR0_EL1 update. */ diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 56723bcbfaaa..6d93bd545906 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -19,8 +19,6 @@ #ifndef __ASM_MMU_CONTEXT_H #define __ASM_MMU_CONTEXT_H -#define FALKOR_RESERVED_ASID 1 - #ifndef __ASSEMBLY__ #include diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index ab9f5f0fb2c7..78816e476491 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -79,13 +79,6 @@ void verify_cpu_asid_bits(void) } } -static void set_reserved_asid_bits(void) -{ - if (IS_ENABLED(CONFIG_QCOM_FALKOR_ERRATUM_1003) && - cpus_have_const_cap(ARM64_WORKAROUND_QCOM_FALKOR_E1003)) - __set_bit(FALKOR_RESERVED_ASID, asid_map); -} - static void flush_context(unsigned int cpu) { int i; @@ -94,8 +87,6 @@ static void flush_context(unsigned int cpu) /* Update the list of reserved ASIDs and the ASID bitmap. */ bitmap_clear(asid_map, 0, NUM_USER_ASIDS); - set_reserved_asid_bits(); - /* * Ensure the generation bump is observed before we xchg the * active_asids. @@ -250,8 +241,6 @@ static int asids_init(void) panic("Failed to allocate bitmap for %lu ASIDs\n", NUM_USER_ASIDS); - set_reserved_asid_bits(); - pr_info("ASID allocator initialised with %lu entries\n", NUM_USER_ASIDS); return 0; } diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index a8a64898a2aa..f2ff0837577c 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -138,7 +138,6 @@ ENDPROC(cpu_do_resume) * - pgd_phys - physical address of new TTB */ ENTRY(cpu_do_switch_mm) - pre_ttbr0_update_workaround x0, x2, x3 mrs x2, ttbr1_el1 mmid x1, x1 // get mm->context.id bfi x2, x1, #48, #16 // set the ASID From patchwork Thu Nov 30 16:39:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120236 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp20946qgn; Thu, 30 Nov 2017 08:46:46 -0800 (PST) X-Google-Smtp-Source: AGs4zMYmOUQzV3GeZcGkVTwgf+QwGCEcI0nK2agEw+7CeEKiR7kgPXoPsD1v//MwDaPSuKIUp+4T X-Received: by 10.159.205.131 with SMTP id v3mr3297712plo.139.1512060406801; Thu, 30 Nov 2017 08:46:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060406; cv=none; d=google.com; s=arc-20160816; b=SuXgmEF7JjwnPNhKBfbIZvn77xHVOKNI3LkmJmEzZRH7GbeHrTg/KAij8U0hz+NZPm myaYdb5kWWdyz83w3cDo8LuxQBWm2iNUzhfJ0ofMXVs7DphO1UZdh0AsyD0SiQR7kLFd zpx+V0+i6F9HERKHEuDY9wsKnGxKCH+f7RcAiQJoxJ7TiBnimwxJoOr5taA7q+6hv/5I 8VcNh6QjnjmjR4OdJ7K3w4XUswblSLtKNzPCs1S2atj2J9W2LnOZfUGPFSTVEG0sM74w 1EGxKvjB6aAJPS8oARHPrhEbUVlocATCtxFBOPU1XvWDUW46g/AC7GE9/kuPTXemMbNY o9Ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=VysFSLKlA8MSUliu3avseHdHs0kUjW7mQ1mSd0bpIlo=; b=HNXIc5+YmiYXQdZGB/DHNR3wwHlVs0xpRHnwm6+zJU9/KQpj2bj7SacyJUdADEb6PG PNKm9rrH+QtdzFAiMdY09Z0TS2cXz0HrPyDN6PyOEXHJ0MnJRBr2XQ5tb7+08o1QQIyw Z6TV7ImM200F/xb/cScBpAmUmaW04tv6lLQ+x7f+vIKY/oBo+H1Huy287u2tS0puLE3K graNNQS7/4MsRxnVIeowN2FIuqHa61Edt/aJBeaCt9jp98WJLTRSk3pOQo5lCMsZz5Jb uyivpjYFHaPOt2gn9Y1zrZ2Nv0ovdFra/JQvOX03Gw5ILJr5db9DhX8rAJUju0ZcroTi i4Nw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a62si3268320pli.333.2017.11.30.08.46.46; Thu, 30 Nov 2017 08:46:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753383AbdK3QqP (ORCPT + 28 others); Thu, 30 Nov 2017 11:46:15 -0500 Received: from foss.arm.com ([217.140.101.70]:57482 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753313AbdK3Qjp (ORCPT ); Thu, 30 Nov 2017 11:39:45 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5293F164F; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2416F3F318; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id C521C1AE3D59; Thu, 30 Nov 2017 16:39:47 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 05/18] arm64: mm: Rename post_ttbr0_update_workaround Date: Thu, 30 Nov 2017 16:39:33 +0000 Message-Id: <1512059986-21325-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The post_ttbr0_update_workaround hook applies to any change to TTBRx_EL1. Since we're using TTBR1 for the ASID, rename the hook to make it clearer as to what it's doing. Signed-off-by: Will Deacon --- arch/arm64/include/asm/assembler.h | 5 ++--- arch/arm64/kernel/entry.S | 2 +- arch/arm64/mm/proc.S | 2 +- 3 files changed, 4 insertions(+), 5 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index e1fa5db858b7..c45bc94f15d0 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -477,10 +477,9 @@ alternative_endif .endm /* -/* - * Errata workaround post TTBR0_EL1 update. + * Errata workaround post TTBRx_EL1 update. */ - .macro post_ttbr0_update_workaround + .macro post_ttbr_update_workaround #ifdef CONFIG_CAVIUM_ERRATUM_27456 alternative_if ARM64_WORKAROUND_CAVIUM_27456 ic iallu diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 6d14b8f29b5f..804e43c9cb0b 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -257,7 +257,7 @@ alternative_else_nop_endif * Cavium erratum 27456 (broadcast TLBI instructions may cause I-cache * corruption). */ - post_ttbr0_update_workaround + post_ttbr_update_workaround .endif 1: .if \el != 0 diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index f2ff0837577c..3146dc96f05b 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -145,7 +145,7 @@ ENTRY(cpu_do_switch_mm) isb msr ttbr0_el1, x0 // now update TTBR0 isb - post_ttbr0_update_workaround + post_ttbr_update_workaround ret ENDPROC(cpu_do_switch_mm) From patchwork Thu Nov 30 16:39:34 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120237 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp21191qgn; Thu, 30 Nov 2017 08:46:58 -0800 (PST) X-Google-Smtp-Source: AGs4zMYiAXFyAcxm8C0p9BkYKH4NmdUMptGw7EPHwL+MX//4/FeY2PY8BaCk+MQn30pNzGUu+Nb3 X-Received: by 10.98.24.207 with SMTP id 198mr7240412pfy.3.1512060418159; Thu, 30 Nov 2017 08:46:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060418; cv=none; d=google.com; s=arc-20160816; b=StyUy+UMnpJtiByZkWIwXKtSoeB8HY+5F/FgZVfpg5DFEFQMeFFiVcl+0z8F2QPPsX 6tgOHCEYTsTa0uYq35SjXitctRq7iz3quMDeCn5tt6As9AqQ822f4oSOuhfApUD+XYB4 kS06SYv9ar1dkEG0drnJvsf1YZS3bnLQnhZq8lOjXoRDH2S3q9x/eWRwhXgyZyrwXb+u hTphEiXkbIBYltY6LtIujEpjQwZou/a3FMCQyvrJ2U5boNM0Rx46rAldzvhm6C3YZk/x iRsrVkGYuw87YdWFXIixgR1qLeJmT6Bq+ySirwLeE01gf/VJQ5K/sj55oV16JhxKPdmN 9fzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=w/KSh3+bc3EQ1WA/Tobp6evhkFOMRfvr83/GnxWM+hk=; b=YTyG56EXn2XhSSuxW8B0S8eAUV/9uNAl9anJJOHXCx8GoWQu+BGWjEhBcEL54l7BH7 0x0mYxixXEtioD+VboGYlXwV2skw9kx4wm9Gqz1sW1HRkxZasMOeprtSCEouJ2/56JIR c+f46nsUf/WKRrh+9EuTJ5Gl+O7aEu68fnM6iUa8KkAPlDxv0gFGr8nW81ApelsEVG9n W3br8Ep+whtug1SQQ/dyQq0JGH2hQZmqzuvM4XU14pM57CuH2KBkIPI7zx+lLrnHSu/R 54hbGxHVvBcoq+tWk2/SWts9jqNKQ7iGVgs5Ho0uKoy6kIoGOE80GLNEaGpXJOhxX0MC uB3A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 91si3465739pla.69.2017.11.30.08.46.57; Thu, 30 Nov 2017 08:46:58 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753705AbdK3QqO (ORCPT + 28 others); Thu, 30 Nov 2017 11:46:14 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57480 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753275AbdK3Qjp (ORCPT ); Thu, 30 Nov 2017 11:39:45 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5B7A0165C; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 2D0C23F5B3; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id D56AD1AE3D5B; Thu, 30 Nov 2017 16:39:47 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 06/18] arm64: mm: Fix and re-enable ARM64_SW_TTBR0_PAN Date: Thu, 30 Nov 2017 16:39:34 +0000 Message-Id: <1512059986-21325-7-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org With the ASID now installed in TTBR1, we can re-enable ARM64_SW_TTBR0_PAN by ensuring that we switch to a reserved ASID of zero when disabling user access and restore the active user ASID on the uaccess enable path. Signed-off-by: Will Deacon --- arch/arm64/Kconfig | 1 - arch/arm64/include/asm/asm-uaccess.h | 25 +++++++++++++++++-------- arch/arm64/include/asm/uaccess.h | 21 +++++++++++++++++---- arch/arm64/kernel/entry.S | 4 ++-- arch/arm64/lib/clear_user.S | 2 +- arch/arm64/lib/copy_from_user.S | 2 +- arch/arm64/lib/copy_in_user.S | 2 +- arch/arm64/lib/copy_to_user.S | 2 +- arch/arm64/mm/cache.S | 2 +- arch/arm64/xen/hypercall.S | 2 +- 10 files changed, 42 insertions(+), 21 deletions(-) -- 2.1.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7e7d7fd152c4..a93339f5178f 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -910,7 +910,6 @@ endif config ARM64_SW_TTBR0_PAN bool "Emulate Privileged Access Never using TTBR0_EL1 switching" - depends on BROKEN # Temporary while switch_mm is reworked help Enabling this option prevents the kernel from accessing user-space memory directly by pointing TTBR0_EL1 to a reserved diff --git a/arch/arm64/include/asm/asm-uaccess.h b/arch/arm64/include/asm/asm-uaccess.h index b3da6c886835..21b8cf304028 100644 --- a/arch/arm64/include/asm/asm-uaccess.h +++ b/arch/arm64/include/asm/asm-uaccess.h @@ -16,11 +16,20 @@ add \tmp1, \tmp1, #SWAPPER_DIR_SIZE // reserved_ttbr0 at the end of swapper_pg_dir msr ttbr0_el1, \tmp1 // set reserved TTBR0_EL1 isb + sub \tmp1, \tmp1, #SWAPPER_DIR_SIZE + bic \tmp1, \tmp1, #(0xffff << 48) + msr ttbr1_el1, \tmp1 // set reserved ASID + isb .endm - .macro __uaccess_ttbr0_enable, tmp1 + .macro __uaccess_ttbr0_enable, tmp1, tmp2 get_thread_info \tmp1 ldr \tmp1, [\tmp1, #TSK_TI_TTBR0] // load saved TTBR0_EL1 + mrs \tmp2, ttbr1_el1 + extr \tmp2, \tmp2, \tmp1, #48 + ror \tmp2, \tmp2, #16 + msr ttbr1_el1, \tmp2 // set the active ASID + isb msr ttbr0_el1, \tmp1 // set the non-PAN TTBR0_EL1 isb .endm @@ -31,18 +40,18 @@ alternative_if_not ARM64_HAS_PAN alternative_else_nop_endif .endm - .macro uaccess_ttbr0_enable, tmp1, tmp2 + .macro uaccess_ttbr0_enable, tmp1, tmp2, tmp3 alternative_if_not ARM64_HAS_PAN - save_and_disable_irq \tmp2 // avoid preemption - __uaccess_ttbr0_enable \tmp1 - restore_irq \tmp2 + save_and_disable_irq \tmp3 // avoid preemption + __uaccess_ttbr0_enable \tmp1, \tmp2 + restore_irq \tmp3 alternative_else_nop_endif .endm #else .macro uaccess_ttbr0_disable, tmp1 .endm - .macro uaccess_ttbr0_enable, tmp1, tmp2 + .macro uaccess_ttbr0_enable, tmp1, tmp2, tmp3 .endm #endif @@ -56,8 +65,8 @@ alternative_if ARM64_ALT_PAN_NOT_UAO alternative_else_nop_endif .endm - .macro uaccess_enable_not_uao, tmp1, tmp2 - uaccess_ttbr0_enable \tmp1, \tmp2 + .macro uaccess_enable_not_uao, tmp1, tmp2, tmp3 + uaccess_ttbr0_enable \tmp1, \tmp2, \tmp3 alternative_if ARM64_ALT_PAN_NOT_UAO SET_PSTATE_PAN(0) alternative_else_nop_endif diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index fc0f9eb66039..750a3b76a01c 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -107,15 +107,19 @@ static inline void __uaccess_ttbr0_disable(void) { unsigned long ttbr; + ttbr = read_sysreg(ttbr1_el1); /* reserved_ttbr0 placed at the end of swapper_pg_dir */ - ttbr = read_sysreg(ttbr1_el1) + SWAPPER_DIR_SIZE; - write_sysreg(ttbr, ttbr0_el1); + write_sysreg(ttbr + SWAPPER_DIR_SIZE, ttbr0_el1); + isb(); + /* Set reserved ASID */ + ttbr &= ~(0xffffUL << 48); + write_sysreg(ttbr, ttbr1_el1); isb(); } static inline void __uaccess_ttbr0_enable(void) { - unsigned long flags; + unsigned long flags, ttbr0, ttbr1; /* * Disable interrupts to avoid preemption between reading the 'ttbr0' @@ -123,7 +127,16 @@ static inline void __uaccess_ttbr0_enable(void) * roll-over and an update of 'ttbr0'. */ local_irq_save(flags); - write_sysreg(current_thread_info()->ttbr0, ttbr0_el1); + ttbr0 = current_thread_info()->ttbr0; + + /* Restore active ASID */ + ttbr1 = read_sysreg(ttbr1_el1); + ttbr1 |= ttbr0 & (0xffffUL << 48); + write_sysreg(ttbr1, ttbr1_el1); + isb(); + + /* Restore user page table */ + write_sysreg(ttbr0, ttbr0_el1); isb(); local_irq_restore(flags); } diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 804e43c9cb0b..d454d8ed45e4 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -184,7 +184,7 @@ alternative_if ARM64_HAS_PAN alternative_else_nop_endif .if \el != 0 - mrs x21, ttbr0_el1 + mrs x21, ttbr1_el1 tst x21, #0xffff << 48 // Check for the reserved ASID orr x23, x23, #PSR_PAN_BIT // Set the emulated PAN in the saved SPSR b.eq 1f // TTBR0 access already disabled @@ -248,7 +248,7 @@ alternative_else_nop_endif tbnz x22, #22, 1f // Skip re-enabling TTBR0 access if the PSR_PAN_BIT is set .endif - __uaccess_ttbr0_enable x0 + __uaccess_ttbr0_enable x0, x1 .if \el == 0 /* diff --git a/arch/arm64/lib/clear_user.S b/arch/arm64/lib/clear_user.S index e88fb99c1561..8f9c4641e706 100644 --- a/arch/arm64/lib/clear_user.S +++ b/arch/arm64/lib/clear_user.S @@ -30,7 +30,7 @@ * Alignment fixed up by hardware. */ ENTRY(__clear_user) - uaccess_enable_not_uao x2, x3 + uaccess_enable_not_uao x2, x3, x4 mov x2, x1 // save the size for fixup return subs x1, x1, #8 b.mi 2f diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S index 4b5d826895ff..69d86a80f3e2 100644 --- a/arch/arm64/lib/copy_from_user.S +++ b/arch/arm64/lib/copy_from_user.S @@ -64,7 +64,7 @@ end .req x5 ENTRY(__arch_copy_from_user) - uaccess_enable_not_uao x3, x4 + uaccess_enable_not_uao x3, x4, x5 add end, x0, x2 #include "copy_template.S" uaccess_disable_not_uao x3 diff --git a/arch/arm64/lib/copy_in_user.S b/arch/arm64/lib/copy_in_user.S index b24a830419ad..e442b531252a 100644 --- a/arch/arm64/lib/copy_in_user.S +++ b/arch/arm64/lib/copy_in_user.S @@ -65,7 +65,7 @@ end .req x5 ENTRY(raw_copy_in_user) - uaccess_enable_not_uao x3, x4 + uaccess_enable_not_uao x3, x4, x5 add end, x0, x2 #include "copy_template.S" uaccess_disable_not_uao x3 diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S index 351f0766f7a6..318f15d5c336 100644 --- a/arch/arm64/lib/copy_to_user.S +++ b/arch/arm64/lib/copy_to_user.S @@ -63,7 +63,7 @@ end .req x5 ENTRY(__arch_copy_to_user) - uaccess_enable_not_uao x3, x4 + uaccess_enable_not_uao x3, x4, x5 add end, x0, x2 #include "copy_template.S" uaccess_disable_not_uao x3 diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 7f1dbe962cf5..6cd20a8c0952 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -49,7 +49,7 @@ ENTRY(flush_icache_range) * - end - virtual end address of region */ ENTRY(__flush_cache_user_range) - uaccess_ttbr0_enable x2, x3 + uaccess_ttbr0_enable x2, x3, x4 dcache_line_size x2, x3 sub x3, x2, #1 bic x4, x0, x3 diff --git a/arch/arm64/xen/hypercall.S b/arch/arm64/xen/hypercall.S index 401ceb71540c..acdbd2c9e899 100644 --- a/arch/arm64/xen/hypercall.S +++ b/arch/arm64/xen/hypercall.S @@ -101,7 +101,7 @@ ENTRY(privcmd_call) * need the explicit uaccess_enable/disable if the TTBR0 PAN emulation * is enabled (it implies that hardware UAO and PAN disabled). */ - uaccess_ttbr0_enable x6, x7 + uaccess_ttbr0_enable x6, x7, x8 hvc XEN_IMM /* From patchwork Thu Nov 30 16:39:35 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120256 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp26344qgn; Thu, 30 Nov 2017 08:51:55 -0800 (PST) X-Google-Smtp-Source: AGs4zMYs4oanpiEyNVYEiTN+F+zu9IQQ22kQKJv1+h9NhCFD9vyKePdPezMpu7OnYoGIrAopsKfZ X-Received: by 10.98.19.202 with SMTP id 71mr7245205pft.181.1512060715776; Thu, 30 Nov 2017 08:51:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060715; cv=none; d=google.com; s=arc-20160816; b=Ah/J5tUc/kCluQoh4W4hzBOsH8npRa/l4CIIgjZAlAbnkMw5l43Tr4t8S24k3r9r7a qjS8POAUXIvCWgaON6l3lA1TygvVuBDOJkj7OQjsyErS6D+f8BEJreLP0Yhkrl1bLXbd DHrw7OFbp3GUNc+NLNSLUv+5dacT5cHux4oJj/IeDp3zw8zom136Ebpnmw/yp/oS0msa BiWpfO/+0d2nmFcdHn1+mHG2TSg2Lnl9uWG2haSPezxEyp1Dtws49LbEnD7WiPXfQXGz GANEeMqeThJAhPmuDkb2MtkwQJ3QwauKixtI+usX94QSZBSmla5RlF01la56MJZ3reTQ vu3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=c7dHcwSDUHqdwwQwJycOKWjLK7Mx3XaXcICWWym4L3k=; b=mJ2Zr07o8aacV9mYqR7RL9EAgr50a9O7hkeDUQrs98GsF4FyvQV5AncrhSptRERmX0 ftGCG2VhPEYiYkZwqbZSLNvWggWAzSUrDjWBHdu2cYrI1NJGGOVDYUa57POyLNKmE7yW IfuoTnLWzQsXPozo/Xp0jJWw7pgQi7rMjUKk99mmMYFazeE6swB1oUtuNzSUrpsmT3QD I2yU18jqasaVvskNxXUXa1gU1ByRAIBfI0GGSTmNx0QPn0wr2j/Sj5am1cYUgqnp9+kk L2G06vMeQB02y3wf8RZq/BBCaCxgxYTDJIPKXyCmuysiN7sUn2IKOtJnLrjLQS3V2XHf PL+w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t134si3233867pgb.511.2017.11.30.08.51.55; Thu, 30 Nov 2017 08:51:55 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753628AbdK3QqL (ORCPT + 28 others); Thu, 30 Nov 2017 11:46:11 -0500 Received: from foss.arm.com ([217.140.101.70]:57496 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753336AbdK3Qjp (ORCPT ); Thu, 30 Nov 2017 11:39:45 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 64F13165D; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 35F823F5B7; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id E5B5B1AE3D5D; Thu, 30 Nov 2017 16:39:47 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 07/18] arm64: mm: Allocate ASIDs in pairs Date: Thu, 30 Nov 2017 16:39:35 +0000 Message-Id: <1512059986-21325-8-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In preparation for separate kernel/user ASIDs, allocate them in pairs for each mm_struct. The bottom bit distinguishes the two: if it is set, then the ASID will map only userspace. Signed-off-by: Will Deacon --- arch/arm64/include/asm/mmu.h | 1 + arch/arm64/mm/context.c | 25 +++++++++++++++++-------- 2 files changed, 18 insertions(+), 8 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 0d34bf0a89c7..01bfb184f2a8 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -17,6 +17,7 @@ #define __ASM_MMU_H #define MMCF_AARCH32 0x1 /* mm context flag for AArch32 executables */ +#define USER_ASID_FLAG (UL(1) << 48) typedef struct { atomic64_t id; diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 78816e476491..db28958d9e4f 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -39,7 +39,16 @@ static cpumask_t tlb_flush_pending; #define ASID_MASK (~GENMASK(asid_bits - 1, 0)) #define ASID_FIRST_VERSION (1UL << asid_bits) -#define NUM_USER_ASIDS ASID_FIRST_VERSION + +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define NUM_USER_ASIDS (ASID_FIRST_VERSION >> 1) +#define asid2idx(asid) (((asid) & ~ASID_MASK) >> 1) +#define idx2asid(idx) (((idx) << 1) & ~ASID_MASK) +#else +#define NUM_USER_ASIDS (ASID_FIRST_VERSION) +#define asid2idx(asid) ((asid) & ~ASID_MASK) +#define idx2asid(idx) asid2idx(idx) +#endif /* Get the ASIDBits supported by the current CPU */ static u32 get_cpu_asid_bits(void) @@ -104,7 +113,7 @@ static void flush_context(unsigned int cpu) */ if (asid == 0) asid = per_cpu(reserved_asids, i); - __set_bit(asid & ~ASID_MASK, asid_map); + __set_bit(asid2idx(asid), asid_map); per_cpu(reserved_asids, i) = asid; } @@ -156,16 +165,16 @@ static u64 new_context(struct mm_struct *mm, unsigned int cpu) * We had a valid ASID in a previous life, so try to re-use * it if possible. */ - asid &= ~ASID_MASK; - if (!__test_and_set_bit(asid, asid_map)) + if (!__test_and_set_bit(asid2idx(asid), asid_map)) return newasid; } /* * Allocate a free ASID. If we can't find one, take a note of the - * currently active ASIDs and mark the TLBs as requiring flushes. - * We always count from ASID #1, as we use ASID #0 when setting a - * reserved TTBR0 for the init_mm. + * currently active ASIDs and mark the TLBs as requiring flushes. We + * always count from ASID #2 (index 1), as we use ASID #0 when setting + * a reserved TTBR0 for the init_mm and we allocate ASIDs in even/odd + * pairs. */ asid = find_next_zero_bit(asid_map, NUM_USER_ASIDS, cur_idx); if (asid != NUM_USER_ASIDS) @@ -182,7 +191,7 @@ static u64 new_context(struct mm_struct *mm, unsigned int cpu) set_asid: __set_bit(asid, asid_map); cur_idx = asid; - return asid | generation; + return idx2asid(asid) | generation; } void check_and_switch_context(struct mm_struct *mm, unsigned int cpu) From patchwork Thu Nov 30 16:39:36 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120238 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp21431qgn; Thu, 30 Nov 2017 08:47:10 -0800 (PST) X-Google-Smtp-Source: AGs4zMboHMZWVfhXNGvYTUPLCm6eL38oFN9pF+85e751bi0+MDoclxqGzjpRFH3eycyR6KhZ+KEo X-Received: by 10.98.204.198 with SMTP id j67mr7326582pfk.41.1512060430382; Thu, 30 Nov 2017 08:47:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060430; cv=none; d=google.com; s=arc-20160816; b=vwWI2pO2StTNep5AVshuBHjvp1QuPALkUZVDZr776wzXb9g5rawda6D4+sv4p6t0xM mEQxmR3lH5i6GlqEifiD8KZeFLK9oHwMRh1+GOQF/7XknW13IPlVLAkF+/YLHxAzAqLV 5hzGL3yi2NNU5zXo5yBhkShKzZ0+y1Ya5cTpnvL3xh5z01NsTv0zk1nANosTVB37x6qw 062cNjstWuLHtlWzTenqmHHS2sCDlY6kSepdIIs1oNoppfOMx5zL3AUMiyPke8s5Akg/ k8NXOl3X8mm6Isdr9oGJtlRANdXVy0h2yiAJlGkgjVWXsZcDyuAGWSXdmeBEfh94Yk8M ZRQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=TgdORaSKm/G1vHHhDw2jvbayLquZGlE/C67o8a6S314=; b=NL/OTtqmp9ocpvCx1egeClwCo0Ssy+xmY3nnXEytpnX5Yp3ui7ys/LSQAlcnUh5u3Y ItIUoCjMkvj0Mdes/giHdPY4JHq8B7DHmXW6nYiHh5rpcGIFbl85MRGqODq1MLQDkJ9L E1vsMY/0AP0wXBVuDD7SVeJZ6mhn4U1bP8gCJpvDcBV90rE8Wte4YSfFQojw0os42Hax h+smoMPBP9k44vGq81hFfP83U8blo+GIfrjFaVRuXHmEEf2FZtdHNfV0wNpVbndAo1+d 01noAjVvS+HIuQvMXZZi5JKMsKAY8Nl5VWhrKonmEO44Clr6lNsrM2rnVUMbZqSYKDA6 k3EQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 33si3402730ple.802.2017.11.30.08.47.10; Thu, 30 Nov 2017 08:47:10 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753688AbdK3QqM (ORCPT + 28 others); Thu, 30 Nov 2017 11:46:12 -0500 Received: from foss.arm.com ([217.140.101.70]:57498 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753327AbdK3Qjp (ORCPT ); Thu, 30 Nov 2017 11:39:45 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6D90C169E; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3FA333F774; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 01D9B1AE3D63; Thu, 30 Nov 2017 16:39:47 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 08/18] arm64: mm: Add arm64_kernel_unmapped_at_el0 helper Date: Thu, 30 Nov 2017 16:39:36 +0000 Message-Id: <1512059986-21325-9-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In order for code such as TLB invalidation to operate efficiently when the decision to map the kernel at EL0 is determined at runtime, this patch introduces a helper function, arm64_kernel_unmapped_at_el0, to determine whether or not the kernel is mapped whilst running in userspace. Currently, this just reports the value of CONFIG_UNMAP_KERNEL_AT_EL0, but will later be hooked up to a fake CPU capability using a static key. Signed-off-by: Will Deacon --- arch/arm64/include/asm/mmu.h | 8 ++++++++ 1 file changed, 8 insertions(+) -- 2.1.4 diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 01bfb184f2a8..c07954638658 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -19,6 +19,8 @@ #define MMCF_AARCH32 0x1 /* mm context flag for AArch32 executables */ #define USER_ASID_FLAG (UL(1) << 48) +#ifndef __ASSEMBLY__ + typedef struct { atomic64_t id; void *vdso; @@ -32,6 +34,11 @@ typedef struct { */ #define ASID(mm) ((mm)->context.id.counter & 0xffff) +static inline bool arm64_kernel_unmapped_at_el0(void) +{ + return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0); +} + extern void paging_init(void); extern void bootmem_init(void); extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt); @@ -42,4 +49,5 @@ extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, extern void *fixmap_remap_fdt(phys_addr_t dt_phys); extern void mark_linear_text_alias_ro(void); +#endif /* !__ASSEMBLY__ */ #endif From patchwork Thu Nov 30 16:39:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120232 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp17810qgn; Thu, 30 Nov 2017 08:43:54 -0800 (PST) X-Google-Smtp-Source: AGs4zMbu9qt/ovQoVeaH4vHHxcJk8hDa5CJsaQpz9zkEsqqmNnSt+rvnqURG8Us6mZyoOn46TEvH X-Received: by 10.84.143.131 with SMTP id 3mr3301860plz.113.1512060234591; Thu, 30 Nov 2017 08:43:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060234; cv=none; d=google.com; s=arc-20160816; b=zk2M03ik61b9O+zlgsdoyWnHPLnFpBU5gYqxH+Ze4mM5ux3sh0Sn61AAJ2N+VKsAs+ mEl/A4Y5+pmYioabw7ByaePK+Jwm7Vf/aoweHspwna/u7B1jgwktXMCwntHGOzZLtpAY IlqstoAW3dx9BwbupcZcsRsD6Ws6LbWiCmY8aku86vFeJw1UE8N1NKU9QwihCQfXvR/e 7rWR/lcoHMctsiqYjSD4m3VwDVA1+ylYSkF9klvAXvF1R4JpDduKXuRUaAgFVRxBxaaf cfYyrVemXXJjUVzKLrbHENOiXhadhuSnhxgSjIt4TPyyIspKpIgq8vYi+kvu1YGfYhOC qy2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=jqR3myRn5mGW2iJHKA08kFGSaW2k9eSlwe/+eLbRp3M=; b=n11hLGE8wxTZIp954/AgMjh6xkO4N09dImD/OUaygXc7KSfPScf5o5M7Rcrq8MoIsM UunKnwujzQnpGmZI1NiDwgUlGHUWT49b9u20bDlrvag5tj5dtJBbBwBHQuuhk6DME8CL uAnzpDC5QjERtpLmtmmkXIsW1dnfy5QOf1wV1nvFoby5SmYnT6DB9SiCqJM0OBD0KOwo WpNMaMS6W6G8CrtZ1HXSlKbjvpWnKoi3UIqEihoXS4dnLaEPcuIwecFQMFCHdht3VR+F aBXpsOmpQvY1zlhAxbcc50kPDL0r09+nSiGS6nyr/RuroF7q7eH94qZAgSqS2MLZYuOV OZJg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h8si3220078pgc.1.2017.11.30.08.43.54; Thu, 30 Nov 2017 08:43:54 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753584AbdK3QnY (ORCPT + 28 others); Thu, 30 Nov 2017 11:43:24 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57544 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753346AbdK3Qjq (ORCPT ); Thu, 30 Nov 2017 11:39:46 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 766BA16A0; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 486EF3F7C3; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 121F61AE3D69; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 09/18] arm64: mm: Invalidate both kernel and user ASIDs when performing TLBI Date: Thu, 30 Nov 2017 16:39:37 +0000 Message-Id: <1512059986-21325-10-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Since an mm has both a kernel and a user ASID, we need to ensure that broadcast TLB maintenance targets both address spaces so that things like CoW continue to work with the uaccess primitives in the kernel. Signed-off-by: Will Deacon --- arch/arm64/include/asm/tlbflush.h | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index af1c76981911..9e82dd79c7db 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -23,6 +23,7 @@ #include #include +#include /* * Raw TLBI operations. @@ -54,6 +55,11 @@ #define __tlbi(op, ...) __TLBI_N(op, ##__VA_ARGS__, 1, 0) +#define __tlbi_user(op, arg) do { \ + if (arm64_kernel_unmapped_at_el0()) \ + __tlbi(op, (arg) | USER_ASID_FLAG); \ +} while (0) + /* * TLB Management * ============== @@ -115,6 +121,7 @@ static inline void flush_tlb_mm(struct mm_struct *mm) dsb(ishst); __tlbi(aside1is, asid); + __tlbi_user(aside1is, asid); dsb(ish); } @@ -125,6 +132,7 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, dsb(ishst); __tlbi(vale1is, addr); + __tlbi_user(vale1is, addr); dsb(ish); } @@ -151,10 +159,13 @@ static inline void __flush_tlb_range(struct vm_area_struct *vma, dsb(ishst); for (addr = start; addr < end; addr += 1 << (PAGE_SHIFT - 12)) { - if (last_level) + if (last_level) { __tlbi(vale1is, addr); - else + __tlbi_user(vale1is, addr); + } else { __tlbi(vae1is, addr); + __tlbi_user(vae1is, addr); + } } dsb(ish); } @@ -194,6 +205,7 @@ static inline void __flush_tlb_pgtable(struct mm_struct *mm, unsigned long addr = uaddr >> 12 | (ASID(mm) << 48); __tlbi(vae1is, addr); + __tlbi_user(vae1is, addr); dsb(ish); } From patchwork Thu Nov 30 16:39:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120231 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp17325qgn; Thu, 30 Nov 2017 08:43:27 -0800 (PST) X-Google-Smtp-Source: AGs4zMa0mJZn0eReoXyoq/OhE3QLeZENMqU/rQqzwOMbAZ4qiKLzq+xE4umm+1sywtewnncPyU8n X-Received: by 10.98.202.217 with SMTP id y86mr7270809pfk.118.1512060207743; Thu, 30 Nov 2017 08:43:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060207; cv=none; d=google.com; s=arc-20160816; b=JpGNTPzf0ZnxhsqiRSpcbE/1Af4BCk16rVV07cuF2ZqqeOa5Jq+C5ji5alNUgmyCGE CGvzxp+jtOF7S1mfM8cIIYkGKwHCRUOdD0nfl3/ayoT0L4ARxSJCMlTHbC6WsoMLRgGm 3mANj8XJCAP4FvOU8/oG3jSCLcWztx5DaYm7kCtF42QOFZPHWZs8uxaFwjUfs+JLoUnm bIs7rFyzlho7O1LYnsXbsQ9nkFXke6N4xZYDpU1ik9HVCInLeYOoLeDYkSl3dsHthnXL PjnB7erdUc86L0sMuOUiFvWZJtLuTUsvrYMPNNH+8vyDTA07GerZm0/PxmTLWeD6lm3r kTBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=BdZY4LEXUeRHz38zcuwZgnyEx4KXiXJ74TFhhmoEiFU=; b=f6ZAuszzkhhEiGdtKS9F6HbO6ujyi4d/96Bj/p5/EORYb7c1veMGjUrT1wnh+vg/wG zrmdYo9jUY0es5hZxBP7GAgiUHLD/MipxxJaC9wfx1M/zy7PQR1cWJxd9uDJkv+Uyp4x 4FXRln2DYgwAimBAhWsWXMfL+Iy9S/v3fa3vFW6sG2lOb10kaHscQ/2fMDe5kxy24cK8 2C0Mi9EpTu13Kv7nLHuHfbx91gLcGVwXGwm4oPmw7BGjd5Be/3v6ijkXYr0NgeFAQ9qE us4iH6kZglRsys9Au6bMrFFMPbByZ5Hooj6+ra4wkkr2E46TReO+uiN+seyV3lUGg807 bCiQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 13si2541112pfm.342.2017.11.30.08.43.27; Thu, 30 Nov 2017 08:43:27 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753603AbdK3QnZ (ORCPT + 28 others); Thu, 30 Nov 2017 11:43:25 -0500 Received: from foss.arm.com ([217.140.101.70]:57542 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753345AbdK3Qjq (ORCPT ); Thu, 30 Nov 2017 11:39:46 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7F53916BA; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 510C73F7C4; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 20E981AE3D6B; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 10/18] arm64: entry: Add exception trampoline page for exceptions from EL0 Date: Thu, 30 Nov 2017 16:39:38 +0000 Message-Id: <1512059986-21325-11-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To allow unmapping of the kernel whilst running at EL0, we need to point the exception vectors at an entry trampoline that can map/unmap the kernel on entry/exit respectively. This patch adds the trampoline page, although it is not yet plugged into the vector table and is therefore unused. Signed-off-by: Will Deacon --- arch/arm64/kernel/entry.S | 86 +++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/vmlinux.lds.S | 17 ++++++++ 2 files changed, 103 insertions(+) -- 2.1.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index d454d8ed45e4..dea196f287a0 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -28,6 +28,8 @@ #include #include #include +#include +#include #include #include #include @@ -943,6 +945,90 @@ __ni_sys_trace: .popsection // .entry.text +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +/* + * Exception vectors trampoline. + */ + .pushsection ".entry.tramp.text", "ax" + + .macro tramp_map_kernel, tmp + mrs \tmp, ttbr1_el1 + sub \tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) + bic \tmp, \tmp, #USER_ASID_FLAG + msr ttbr1_el1, \tmp + .endm + + .macro tramp_unmap_kernel, tmp + mrs \tmp, ttbr1_el1 + add \tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) + orr \tmp, \tmp, #USER_ASID_FLAG + msr ttbr1_el1, \tmp + /* + * We avoid running the post_ttbr_update_workaround here because the + * user and kernel ASIDs don't have conflicting mappings, so any + * "blessing" as described in: + * + * http://lkml.kernel.org/r/56BB848A.6060603@caviumnetworks.com + * + * will not hurt correctness. Whilst this may partially defeat the + * point of using split ASIDs in the first place, it avoids + * the hit of invalidating the entire I-cache on every return to + * userspace. + */ + .endm + + .macro tramp_ventry, regsize = 64 + .align 7 +1: + .if \regsize == 64 + msr tpidrro_el0, x30 + .endif + tramp_map_kernel x30 + ldr x30, =vectors + prfm plil1strm, [x30, #(1b - tramp_vectors)] + msr vbar_el1, x30 + add x30, x30, #(1b - tramp_vectors) + isb + br x30 + .endm + + .macro tramp_exit, regsize = 64 + adr x30, tramp_vectors + msr vbar_el1, x30 + tramp_unmap_kernel x30 + .if \regsize == 64 + mrs x30, far_el1 + .endif + eret + .endm + + .align 11 +ENTRY(tramp_vectors) + .space 0x400 + + tramp_ventry + tramp_ventry + tramp_ventry + tramp_ventry + + tramp_ventry 32 + tramp_ventry 32 + tramp_ventry 32 + tramp_ventry 32 +END(tramp_vectors) + +ENTRY(tramp_exit_native) + tramp_exit +END(tramp_exit_native) + +ENTRY(tramp_exit_compat) + tramp_exit 32 +END(tramp_exit_compat) + + .ltorg + .popsection // .entry.tramp.text +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + /* * Special system call wrappers. */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index 7da3e5c366a0..6b4260f22aab 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -57,6 +57,17 @@ jiffies = jiffies_64; #define HIBERNATE_TEXT #endif +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +#define TRAMP_TEXT \ + . = ALIGN(PAGE_SIZE); \ + VMLINUX_SYMBOL(__entry_tramp_text_start) = .; \ + *(.entry.tramp.text) \ + . = ALIGN(PAGE_SIZE); \ + VMLINUX_SYMBOL(__entry_tramp_text_end) = .; +#else +#define TRAMP_TEXT +#endif + /* * The size of the PE/COFF section that covers the kernel image, which * runs from stext to _edata, must be a round multiple of the PE/COFF @@ -113,6 +124,7 @@ SECTIONS HYPERVISOR_TEXT IDMAP_TEXT HIBERNATE_TEXT + TRAMP_TEXT *(.fixup) *(.gnu.warning) . = ALIGN(16); @@ -214,6 +226,11 @@ SECTIONS . += RESERVED_TTBR0_SIZE; #endif +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + tramp_pg_dir = .; + . += PAGE_SIZE; +#endif + __pecoff_data_size = ABSOLUTE(. - __initdata_begin); _end = .; From patchwork Thu Nov 30 16:39:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120230 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp17238qgn; Thu, 30 Nov 2017 08:43:22 -0800 (PST) X-Google-Smtp-Source: AGs4zMZMp/p6ix1N4vzTBIXhMRbl4+mq2l1rAsaqmfs69B+lGbRSqgUXNBclyqzdglQ7/3whiU8X X-Received: by 10.98.210.138 with SMTP id c132mr3021546pfg.54.1512060202149; Thu, 30 Nov 2017 08:43:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060202; cv=none; d=google.com; s=arc-20160816; b=vb4zz4Z9cJzThlq3ym0gBhKD2OKCj/rrg2rsHrCRRR2CyKIdhqq57Z2lx5+BUpJUKG xcDPgmsgUj9+TNZWa5Eczi068R+KKo/2te8j205ajx2Lr15YbrNZMi5EL+l+ma05SrsR GTGskJDtDPo93Ff1r07JGqjA1Xj4+0jm5dRQ172eDYb0eLmQaB8jUp3PdbiZqKMDlZmW 1PmUrWi0B8WnvuXy7obzmKW8ESV1wEhW/XlEUewkwEn8svALlrTJqgn/TH7Mv+bMrEJQ 3L3/fV0YCT+ZOlvXIJBciX7REtyKFfOFbooW+KJtwY8cl2LkvZyDmkP157+UKEnMmeij pFRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=gzsAfRDUh4/8um4iPZHkXzPQ44FZUd1EQt97nnUK1HY=; b=k1Pdzb3MFKIKnJ2z9AZjL05aGVe8oqlLysVXhLuNG7eynJHbnArRfhLKaYNkkraEpM Cyr9NlLPHvV2P7265X99445fMIC+15xs7m9lmRs1zZGle99xQYs3RZURE9w5iDDpuv2/ EOB/ccGuCeKIaPkMP6j+SW/E3LP1DAiMijicUT+cWOKzAP5kzF0XyeKqlmg+w8edOify GO7zLKQLxOJc5RgvLRwZrCwkReyvHlIapbPGojkeoqrJRO5lkQBWqyGZEaurQarkvGb1 e8Dmb4La1dOY7Z2loRIHzzFlZc2Ko7jweg9rxK+nkPmAcxnrBegmQmdrHei0I3XnpLdo Jeyg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 13si2541112pfm.342.2017.11.30.08.43.21; Thu, 30 Nov 2017 08:43:22 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753274AbdK3QnU (ORCPT + 28 others); Thu, 30 Nov 2017 11:43:20 -0500 Received: from foss.arm.com ([217.140.101.70]:57562 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753351AbdK3Qjq (ORCPT ); Thu, 30 Nov 2017 11:39:46 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8A26F1715; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 59EDB3F318; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 2D3931AE3DD7; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 11/18] arm64: mm: Map entry trampoline into trampoline and kernel page tables Date: Thu, 30 Nov 2017 16:39:39 +0000 Message-Id: <1512059986-21325-12-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The exception entry trampoline needs to be mapped at the same virtual address in both the trampoline page table (which maps nothing else) and also the kernel page table, so that we can swizzle TTBR1_EL1 on exceptions from and return to EL0. This patch maps the trampoline at a fixed virtual address in the fixmap area of the kernel virtual address space, which allows the kernel proper to be randomized with respect to the trampoline when KASLR is enabled. Signed-off-by: Will Deacon --- arch/arm64/include/asm/fixmap.h | 4 ++++ arch/arm64/include/asm/pgtable.h | 1 + arch/arm64/kernel/asm-offsets.c | 6 +++++- arch/arm64/mm/mmu.c | 23 +++++++++++++++++++++++ 4 files changed, 33 insertions(+), 1 deletion(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/fixmap.h b/arch/arm64/include/asm/fixmap.h index 4052ec39e8db..8119b49be98d 100644 --- a/arch/arm64/include/asm/fixmap.h +++ b/arch/arm64/include/asm/fixmap.h @@ -58,6 +58,10 @@ enum fixed_addresses { FIX_APEI_GHES_NMI, #endif /* CONFIG_ACPI_APEI_GHES */ +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + FIX_ENTRY_TRAMP_TEXT, +#define TRAMP_VALIAS (__fix_to_virt(FIX_ENTRY_TRAMP_TEXT)) +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ __end_of_permanent_fixed_addresses, /* diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index c9530b5b5ca8..c8f56b2ca414 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -681,6 +681,7 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm, extern pgd_t swapper_pg_dir[PTRS_PER_PGD]; extern pgd_t idmap_pg_dir[PTRS_PER_PGD]; +extern pgd_t tramp_pg_dir[PTRS_PER_PGD]; /* * Encode and decode a swap entry: diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 71bf088f1e4b..af247d10252f 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -148,11 +149,14 @@ int main(void) DEFINE(ARM_SMCCC_RES_X2_OFFS, offsetof(struct arm_smccc_res, a2)); DEFINE(ARM_SMCCC_QUIRK_ID_OFFS, offsetof(struct arm_smccc_quirk, id)); DEFINE(ARM_SMCCC_QUIRK_STATE_OFFS, offsetof(struct arm_smccc_quirk, state)); - BLANK(); DEFINE(HIBERN_PBE_ORIG, offsetof(struct pbe, orig_address)); DEFINE(HIBERN_PBE_ADDR, offsetof(struct pbe, address)); DEFINE(HIBERN_PBE_NEXT, offsetof(struct pbe, next)); DEFINE(ARM64_FTR_SYSVAL, offsetof(struct arm64_ftr_reg, sys_val)); + BLANK(); +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + DEFINE(TRAMP_VALIAS, TRAMP_VALIAS); +#endif return 0; } diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 267d2b79d52d..c2622525c4d6 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -525,6 +525,29 @@ static int __init parse_rodata(char *arg) } early_param("rodata", parse_rodata); +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static int __init map_entry_trampoline(void) +{ + extern char __entry_tramp_text_start[]; + + pgprot_t prot = rodata_enabled ? PAGE_KERNEL_ROX : PAGE_KERNEL_EXEC; + phys_addr_t pa_start = __pa_symbol(__entry_tramp_text_start); + + /* The trampoline is always mapped and can therefore be global */ + pgprot_val(prot) &= ~PTE_NG; + + /* Map only the text into the trampoline page table */ + memset((char *)tramp_pg_dir, 0, PGD_SIZE); + __create_pgd_mapping(tramp_pg_dir, pa_start, TRAMP_VALIAS, PAGE_SIZE, + prot, pgd_pgtable_alloc, 0); + + /* ...as well as the kernel page table */ + __set_fixmap(FIX_ENTRY_TRAMP_TEXT, pa_start, prot); + return 0; +} +core_initcall(map_entry_trampoline); +#endif + /* * Create fine-grained mappings for the kernel. */ From patchwork Thu Nov 30 16:39:40 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120229 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp16696qgn; Thu, 30 Nov 2017 08:42:51 -0800 (PST) X-Google-Smtp-Source: AGs4zMZ4btuvIAkdIJpQUVx/OzPuuRXrbbbcDvVOgQiQayamrG059vpTqUedLzbU/4lOZ3ci+TQC X-Received: by 10.159.231.15 with SMTP id w15mr3280211plq.410.1512060171188; Thu, 30 Nov 2017 08:42:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060171; cv=none; d=google.com; s=arc-20160816; b=I9+DNnhdgA+u4Quek1TOPtlWzJDJo5QcefHf3udZwGdmrXAn0ib0IC9H5pxN0/Xzz1 d8ObwnfkZ7Fl26OEZmdQvG8ICIvazI0/ypRuDfDYjS8FFYXO2bYI48atnKELYRpojcJT 9ekiu4tkpccZSpGwrHlTAiLG/tLpaDAiw+c77SFzEagwaA6K95xx/Z0ngnp5USQ+QEIj HtkS2gaF1l6U2bNj0frJeOD59lHv60L02LLVPJArfp7OvRUS2BXptFnKu61n5mCdxxzc tyvWzlCt9jcDayEPzwiRWMRh+wJGOQtXmNMJJhxBYgOoe+ihqGJoCWvkv4Z5WhewLKUR tjUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=0hXKtsyjIxkizEfp/WQCc4B9C8qtKlQcrYlUmg9xkXo=; b=Yy2PvfbTuLa++TcgyBb3+Y7pnsKi53fBJeMFxxSzFDMmMgfA6Kw07q+AvZooxqpVdZ fPc7I5JI0pvRZBeED2SXt3+A9H+PcK2gg/hHmNazs+fkhhn8pueyu4I/TeJOsg4jDPLL TKsy8ibjdtaYzMUAD3/wXYdQKSy+KpJGSzw+wyDk11gV2o3TL9XNY1fxcyF1mxmfaCj3 teuh6RgVOYYF6VmRFQpp2+y8eQpan8HfpTkTY2oHuJKz7MBE2cCzvki9RzmleLOhqQdv 55rlbFf7deBp7SFvdrUZ4WshZ1U6MLkfMQPMDUkT2AqajMTZxRoYW3/nZdyIc4qn6Tq4 ESng== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w31si3415245pla.711.2017.11.30.08.42.50; Thu, 30 Nov 2017 08:42:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753413AbdK3Qmt (ORCPT + 28 others); Thu, 30 Nov 2017 11:42:49 -0500 Received: from foss.arm.com ([217.140.101.70]:57572 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753362AbdK3Qjr (ORCPT ); Thu, 30 Nov 2017 11:39:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 91DE3174E; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 62DF83F5B3; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 3BE771AE3DFA; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 12/18] arm64: entry: Explicitly pass exception level to kernel_ventry macro Date: Thu, 30 Nov 2017 16:39:40 +0000 Message-Id: <1512059986-21325-13-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We will need to treat exceptions from EL0 differently in kernel_ventry, so rework the macro to take the exception level as an argument and construct the branch target using that. Signed-off-by: Will Deacon --- arch/arm64/kernel/entry.S | 46 +++++++++++++++++++++++----------------------- 1 file changed, 23 insertions(+), 23 deletions(-) -- 2.1.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index dea196f287a0..688e52f65a8d 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -71,7 +71,7 @@ #define BAD_FIQ 2 #define BAD_ERROR 3 - .macro kernel_ventry label + .macro kernel_ventry, el, label, regsize = 64 .align 7 sub sp, sp, #S_FRAME_SIZE #ifdef CONFIG_VMAP_STACK @@ -84,7 +84,7 @@ tbnz x0, #THREAD_SHIFT, 0f sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp - b \label + b el\()\el\()_\label 0: /* @@ -116,7 +116,7 @@ sub sp, sp, x0 mrs x0, tpidrro_el0 #endif - b \label + b el\()\el\()_\label .endm .macro kernel_entry, el, regsize = 64 @@ -369,31 +369,31 @@ tsk .req x28 // current thread_info .align 11 ENTRY(vectors) - kernel_ventry el1_sync_invalid // Synchronous EL1t - kernel_ventry el1_irq_invalid // IRQ EL1t - kernel_ventry el1_fiq_invalid // FIQ EL1t - kernel_ventry el1_error_invalid // Error EL1t + kernel_ventry 1, sync_invalid // Synchronous EL1t + kernel_ventry 1, irq_invalid // IRQ EL1t + kernel_ventry 1, fiq_invalid // FIQ EL1t + kernel_ventry 1, error_invalid // Error EL1t - kernel_ventry el1_sync // Synchronous EL1h - kernel_ventry el1_irq // IRQ EL1h - kernel_ventry el1_fiq_invalid // FIQ EL1h - kernel_ventry el1_error // Error EL1h + kernel_ventry 1, sync // Synchronous EL1h + kernel_ventry 1, irq // IRQ EL1h + kernel_ventry 1, fiq_invalid // FIQ EL1h + kernel_ventry 1, error // Error EL1h - kernel_ventry el0_sync // Synchronous 64-bit EL0 - kernel_ventry el0_irq // IRQ 64-bit EL0 - kernel_ventry el0_fiq_invalid // FIQ 64-bit EL0 - kernel_ventry el0_error // Error 64-bit EL0 + kernel_ventry 0, sync // Synchronous 64-bit EL0 + kernel_ventry 0, irq // IRQ 64-bit EL0 + kernel_ventry 0, fiq_invalid // FIQ 64-bit EL0 + kernel_ventry 0, error // Error 64-bit EL0 #ifdef CONFIG_COMPAT - kernel_ventry el0_sync_compat // Synchronous 32-bit EL0 - kernel_ventry el0_irq_compat // IRQ 32-bit EL0 - kernel_ventry el0_fiq_invalid_compat // FIQ 32-bit EL0 - kernel_ventry el0_error_compat // Error 32-bit EL0 + kernel_ventry 0, sync_compat, 32 // Synchronous 32-bit EL0 + kernel_ventry 0, irq_compat, 32 // IRQ 32-bit EL0 + kernel_ventry 0, fiq_invalid_compat, 32 // FIQ 32-bit EL0 + kernel_ventry 0, error_compat, 32 // Error 32-bit EL0 #else - kernel_ventry el0_sync_invalid // Synchronous 32-bit EL0 - kernel_ventry el0_irq_invalid // IRQ 32-bit EL0 - kernel_ventry el0_fiq_invalid // FIQ 32-bit EL0 - kernel_ventry el0_error_invalid // Error 32-bit EL0 + kernel_ventry 0, sync_invalid, 32 // Synchronous 32-bit EL0 + kernel_ventry 0, irq_invalid, 32 // IRQ 32-bit EL0 + kernel_ventry 0, fiq_invalid, 32 // FIQ 32-bit EL0 + kernel_ventry 0, error_invalid, 32 // Error 32-bit EL0 #endif END(vectors) From patchwork Thu Nov 30 16:39:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120233 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp18234qgn; Thu, 30 Nov 2017 08:44:18 -0800 (PST) X-Google-Smtp-Source: AGs4zMaTGqYHlk4uJYh/mw51d4QixWH8kNAvqSvebN9MhuIyY/3COkCms0Ei3RqKABTFKw8F81KW X-Received: by 10.98.34.199 with SMTP id p68mr7109008pfj.241.1512060258619; Thu, 30 Nov 2017 08:44:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060258; cv=none; d=google.com; s=arc-20160816; b=x6xEiIp43RP+M7ezYZXZ2kpUUTx1AbXCcQKwUic572Nrvw8PoGpXnZ4u7DxGITSUfH oJHP6SLuyuxu2LcTa8G/TE+4Zjc7WOj+UkiAcOtYiIvNM8J/ifzJDCladMFrOpwjUOY+ ZtAS3y06i9+csv13x+HtOc1cqED2zud8eHODQHU0X9WdOgLpiiT3DOZODyj2izYANdfE 12td0pcOkxqz0/1Y9OdS1hmA47cjGxntknSLPlUc7/5xez+sl1p1yHzL8ueSdIoxVM6y EYjL0NAQvc//tD7PA0n4QYe8AY/2d1EjF1ca71YpO68TZi7QxsFz1suAFG9IasgAH4Qw 8CkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=7n3TAd51rVdguA3E+x++XQjmkPp4kPr/lYjjB5J/AFo=; b=HYxmo3087cXv9rGBoCPd6gSobSuEFCk7sY1AZraDRaNTUpSj4ePEeLj9JvDPv7MDKK 2bJbKFqxnTiZm/EH9S99s4TjOm9LBcJMxKch6asAV1LzCsdfkJHaLdzJH5C+QIrZklpT aETMo/ZSIqf4OycyH5XRpg6XB/70cqOOPfHby/8Rvg6/9qDOPrCiHoBH+LuHndbyv0nT vYrbLR3WjXjGYAdF0y8EWFZGTYQRwD1nSJLrAKhcIK8nlAt7CLI3mwBSY1ly2ipIh37k eoUkhhxUk5yMiTcuFKtlFiruekEdtGjligabP5l81EFKl5P/7FMKar4tmWoIjWsT00Hc 8khg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g68si3231268pgc.304.2017.11.30.08.44.18; Thu, 30 Nov 2017 08:44:18 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753543AbdK3QnV (ORCPT + 28 others); Thu, 30 Nov 2017 11:43:21 -0500 Received: from foss.arm.com ([217.140.101.70]:57560 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753350AbdK3Qjq (ORCPT ); Thu, 30 Nov 2017 11:39:46 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9A11219CC; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 6BC933F5B7; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 47F501AE3DFB; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 13/18] arm64: entry: Hook up entry trampoline to exception vectors Date: Thu, 30 Nov 2017 16:39:41 +0000 Message-Id: <1512059986-21325-14-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hook up the entry trampoline to our exception vectors so that all exceptions from and returns to EL0 go via the trampoline, which swizzles the vector base register accordingly. Transitioning to and from the kernel clobbers x30, so we use tpidrro_el0 and far_el1 as scratch registers for native tasks. Signed-off-by: Will Deacon --- arch/arm64/kernel/entry.S | 39 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 36 insertions(+), 3 deletions(-) -- 2.1.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 688e52f65a8d..99d105048663 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -73,6 +73,17 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + .if \el == 0 + .if \regsize == 64 + mrs x30, tpidrro_el0 + msr tpidrro_el0, xzr + .else + mov x30, xzr + .endif + .endif +#endif + sub sp, sp, #S_FRAME_SIZE #ifdef CONFIG_VMAP_STACK /* @@ -119,6 +130,11 @@ b el\()\el\()_\label .endm + .macro tramp_alias, dst, sym + mov_q \dst, TRAMP_VALIAS + add \dst, \dst, #(\sym - .entry.tramp.text) + .endm + .macro kernel_entry, el, regsize = 64 .if \regsize == 32 mov w0, w0 // zero upper 32 bits of x0 @@ -271,18 +287,20 @@ alternative_else_nop_endif .if \el == 0 ldr x23, [sp, #S_SP] // load return stack pointer msr sp_el0, x23 + tst x22, #PSR_MODE32_BIT // native task? + b.eq 3f + #ifdef CONFIG_ARM64_ERRATUM_845719 alternative_if ARM64_WORKAROUND_845719 - tbz x22, #4, 1f #ifdef CONFIG_PID_IN_CONTEXTIDR mrs x29, contextidr_el1 msr contextidr_el1, x29 #else msr contextidr_el1, xzr #endif -1: alternative_else_nop_endif #endif +3: .endif msr elr_el1, x21 // set up the return data @@ -304,7 +322,22 @@ alternative_else_nop_endif ldp x28, x29, [sp, #16 * 14] ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp - eret // return to kernel + +#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 + eret +#else + .if \el == 0 + bne 4f + msr far_el1, x30 + tramp_alias x30, tramp_exit_native + br x30 +4: + tramp_alias x30, tramp_exit_compat + br x30 + .else + eret + .endif +#endif .endm .macro irq_stack_entry From patchwork Thu Nov 30 16:39:42 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120225 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp14195qgn; Thu, 30 Nov 2017 08:40:43 -0800 (PST) X-Google-Smtp-Source: AGs4zMbgJpSvjpRH+gmGQo32Mqgkf/0L20TPL/hSwNcSPkzhmikci7Fh0ax+L5CR2ESgyGk84IR4 X-Received: by 10.159.249.1 with SMTP id bf1mr3208226plb.401.1512060043822; Thu, 30 Nov 2017 08:40:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060043; cv=none; d=google.com; s=arc-20160816; b=CDJ1mwj6YaD3S8bjpBp8iS3L+qYeDsF9bF6KuKr+DkSREfO6f5eSd8COi8DJCkBW8K Yij7B9iXqpIiInioUFH1aUb5vF/B94h5C8SthS69Il220T4FnEPsYyxCW5v5DWP9ynN5 uCAroJIJMAy7qHy4K2h8MkVsIfwoeZaHuKYymMzIk1+iK9XRQ9ZC4enJ5UP7iLHusvik qtW04dOEzzK6QMDUgGuXun5SmvIlRa2b3Smug6SU0lqSwrWtL4F/ccSDiolCgsNenwKU otMN2kjvfaWnICByCmknAYwoPA+l4TIMcjPYWAvbY4pFHwAIDrhSP+soSTaF+fnmMWxM grJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=aH7JQKGvfCSRiFksL2Cp4GDO2T3Y52614K79dFdZ96Y=; b=aHBJhIHtWYQgbsAJ/6U3curGyxt7a3KFVtxg8VWhuOXn9TanNLr/6mewScRgqHRBcG Qo1AjnKugBvYqx8WfmKFm5I2BuZSY8W2F9g+2+F9qu6S1g0mXXn2rPom1b2Z4PcEYv4s RkUHPrHLy7RT/Y6SnhYLuNRIzOC/hR1Nt6sj8iJeH7nRuW7w0TwFa3fMOWdvFfhHg4ZL i1o6tRCqnd6V4s8zofhx0Hm7OCZ96TcKVH7mRj7wbxscbdAVKRXzWZ6CynSx5oioJ5v7 /9sxxkWhiUyObrIaM4BhuOsLlHe1lYKp5m70jOkYPJc9j16PcBCdLb0U+IzdaaYq/LWm LIyw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h72si3532928pfj.20.2017.11.30.08.40.43; Thu, 30 Nov 2017 08:40:43 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753510AbdK3Qkk (ORCPT + 28 others); Thu, 30 Nov 2017 11:40:40 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57574 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753357AbdK3Qjr (ORCPT ); Thu, 30 Nov 2017 11:39:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A317C1A25; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 74BF63F774; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 5456C1AE3DFC; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 14/18] arm64: erratum: Work around Falkor erratum #E1003 in trampoline code Date: Thu, 30 Nov 2017 16:39:42 +0000 Message-Id: <1512059986-21325-15-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We rely on an atomic swizzling of TTBR1 when transitioning from the entry trampoline to the kernel proper on an exception. We can't rely on this atomicity in the face of Falkor erratum #E1003, so on affected cores we can issue a TLB invalidation to invalidate the walk cache prior to jumping into the kernel. There is still the possibility of a TLB conflict here due to conflicting walk cache entries prior to the invalidation, but this doesn't appear to be the case on these CPUs in practice. Signed-off-by: Will Deacon --- arch/arm64/Kconfig | 17 +++++------------ arch/arm64/kernel/entry.S | 10 ++++++++++ 2 files changed, 15 insertions(+), 12 deletions(-) -- 2.1.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a93339f5178f..fdcc7b9bb15d 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -522,20 +522,13 @@ config CAVIUM_ERRATUM_30115 config QCOM_FALKOR_ERRATUM_1003 bool "Falkor E1003: Incorrect translation due to ASID change" default y - select ARM64_PAN if ARM64_SW_TTBR0_PAN help On Falkor v1, an incorrect ASID may be cached in the TLB when ASID - and BADDR are changed together in TTBRx_EL1. The workaround for this - issue is to use a reserved ASID in cpu_do_switch_mm() before - switching to the new ASID. Saying Y here selects ARM64_PAN if - ARM64_SW_TTBR0_PAN is selected. This is done because implementing and - maintaining the E1003 workaround in the software PAN emulation code - would be an unnecessary complication. The affected Falkor v1 CPU - implements ARMv8.1 hardware PAN support and using hardware PAN - support versus software PAN emulation is mutually exclusive at - runtime. - - If unsure, say Y. + and BADDR are changed together in TTBRx_EL1. Since we keep the ASID + in TTBR1_EL1, this situation only occurs in the entry trampoline and + then only for entries in the walk cache, since the leaf translation + is unchanged. Work around the erratum by invalidating the walk cache + entries for the trampoline before entering the kernel proper. config QCOM_FALKOR_ERRATUM_1009 bool "Falkor E1009: Prematurely complete a DSB after a TLBI" diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 99d105048663..a5ec6ab5c711 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -989,6 +989,16 @@ __ni_sys_trace: sub \tmp, \tmp, #(SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) bic \tmp, \tmp, #USER_ASID_FLAG msr ttbr1_el1, \tmp +#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003 +alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003 + movk \tmp, #:abs_g2_nc:(TRAMP_VALIAS >> 12) + movk \tmp, #:abs_g1_nc:(TRAMP_VALIAS >> 12) + movk \tmp, #:abs_g0_nc:((TRAMP_VALIAS & (SZ_2M - 1)) >> 12) + isb + tlbi vae1, \tmp + dsb nsh +alternative_else_nop_endif +#endif /* CONFIG_QCOM_FALKOR_ERRATUM_1003 */ .endm .macro tramp_unmap_kernel, tmp From patchwork Thu Nov 30 16:39:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120226 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp14468qgn; Thu, 30 Nov 2017 08:40:56 -0800 (PST) X-Google-Smtp-Source: AGs4zMah2TONTi90WQQlIR7jmZ8JCKDvgAbCT6UiNsCRurXArST9v1WyusrRVpdzecLrqGcIeiWq X-Received: by 10.99.134.73 with SMTP id x70mr2985999pgd.130.1512060056242; Thu, 30 Nov 2017 08:40:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060056; cv=none; d=google.com; s=arc-20160816; b=zBYGcbVGm6NNBZ9F2dWPPldtpanbGS2imUeWyX0vCiJVc+7W/ycDMYREhXZygJZ/LI JLbXpGlOWHhIAmACmbQvXyvKtSBC+VUaH+G3g/A7Bnofevt0rHUYneBnPNR0/h5gPyzE FFm5GgKbF2qv4vZYpkMhXNzu97Ce7lQYyXNgdgQYLTCQGb4gxLa5eB+z/dGbOpFeS1wY pVpbZe66mG9tWbIXi5o7u5sxjOavLpk6Ete5JRhAWSEbqx4Y7aLDwXLIsaZonvbWHnkD bulAiAbvAn2WLFU8LR9tvNCl2oOQyFeEsLj+dfK9ZEGwDgZT9sgvIERj8ymLjMBPmyqv 72ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=tGwqIzaw6YVQZJvN26AWqQo/vaSJU0HHNefiNUooruk=; b=LURDOf+vwdSIIDyZKRyNgD5mAlPEf1gWSqk/+r96ESvEE5/+/vPab03FQ7Cw/JqEYd qt4WRLsZuhdret9Cu/fMTbdo2qgoFeCcbiAcMQvAB5u008Pii1+mVWbkPsTH6PI3hwPp PYRgXN+/8GHx0jiowEzIXzaffe6wEhv+n3N8inxeBryXWXvw4iaYfypXRalZgZkKCpgw zCE6q+PkSZJCRLMPR2rWZD1ZvktYXD+qpAlI8NIGMRKfEmO7OzWz+riqHdr66x19FXzw hXwfoy+YDDoRrfxpbMkAr3pMAVjbHt1HHA7rQ0g+fX+x5IAng+lv5Rz2wDs3grxiV1Xr 2DEg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h72si3532928pfj.20.2017.11.30.08.40.55; Thu, 30 Nov 2017 08:40:56 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753496AbdK3Qkj (ORCPT + 28 others); Thu, 30 Nov 2017 11:40:39 -0500 Received: from foss.arm.com ([217.140.101.70]:57482 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753366AbdK3Qjr (ORCPT ); Thu, 30 Nov 2017 11:39:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BF5681A9A; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8FF593F7C3; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 600031AE3DFD; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 15/18] arm64: tls: Avoid unconditional zeroing of tpidrro_el0 for native tasks Date: Thu, 30 Nov 2017 16:39:43 +0000 Message-Id: <1512059986-21325-16-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When unmapping the kernel at EL0, we use tpidrro_el0 as a scratch register during exception entry from native tasks and subsequently zero it in the kernel_ventry macro. We can therefore avoid zeroing tpidrro_el0 in the context-switch path for native tasks using the entry trampoline. Signed-off-by: Will Deacon --- arch/arm64/kernel/process.c | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) -- 2.1.4 diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index b2adcce7bc18..aba3a1fb492d 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -361,16 +361,14 @@ void tls_preserve_current_state(void) static void tls_thread_switch(struct task_struct *next) { - unsigned long tpidr, tpidrro; - tls_preserve_current_state(); - tpidr = *task_user_tls(next); - tpidrro = is_compat_thread(task_thread_info(next)) ? - next->thread.tp_value : 0; + if (is_compat_thread(task_thread_info(next))) + write_sysreg(next->thread.tp_value, tpidrro_el0); + else if (!arm64_kernel_unmapped_at_el0()) + write_sysreg(0, tpidrro_el0); - write_sysreg(tpidr, tpidr_el0); - write_sysreg(tpidrro, tpidrro_el0); + write_sysreg(*task_user_tls(next), tpidr_el0); } /* Restore the UAO state depending on next's addr_limit */ From patchwork Thu Nov 30 16:39:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120224 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp14187qgn; Thu, 30 Nov 2017 08:40:43 -0800 (PST) X-Google-Smtp-Source: AGs4zMa7Hg6vGoaE3JjHeCIVYdhpS+wXqjUIxrmVFl4cf/d+5pY36XlKZdFSBokNAu5ysBxmvLMh X-Received: by 10.98.86.70 with SMTP id k67mr7188460pfb.214.1512060043461; Thu, 30 Nov 2017 08:40:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060043; cv=none; d=google.com; s=arc-20160816; b=tA5BBTIhtpmhg6RW4jKLW18I38mIE5jWfgDl/kfhy/Km0mUn3SDjWzMQn9DhA2ezRw cZFdzwgqURxfGCbWwrFsKsQAqGkHtI3iiyi7mMkA2CdDEhuE1RVi4krnyIRdq33hdpYL wMzNpgUb4ukqC2cKjDbQ8mS7m86a7oI98MabsanadqPn/i4rk3WhomIChvDsJB0i8dNp 3VsFX98xBehxiGjKgIn9MJDsQmkS8Vly9TbkGKuMvT5/IulP4EY0MYNvBm9gZVcPx2r4 oN+5wV59xaSUEbGugcqTLnj3e2Fu7P7LNss1eQbEofMXDdPDX41ZCK2djFOa6EvEJtg/ 2SOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=uGMAFHIDE6QccExS90aDXuF4sFb4LPp2pjv42dL57N0=; b=rTr9nb27lwBwJ2z8Avjz/LtTYkoJPedj6AIXNmM1vBKTGvaLZwyOilaI9mzVov/qq8 cs3XhCs3bnL9MRD+N2m+5Hfjhvy0hhmzOikhqBilWJCxy5DHMIuSIPGfvwd2B1UmL4EC hYgSEa6i4cjje6mMbsKYFh6GoEm8DC7y1IOGA7NmzZzv668DK376g26OqGsm8oYFJbp5 AzXH2JVXP0oSQBrdMaQi2DAJ0G60QjxRpvB0H9FAHMsDfN5jaEdJ5yRRL+NypQCQId/A 2mypSBiln2LpDhrWRqg/0toOjdcm4zrE+yHsVj7NYDfEH+e05EUwHvOykcv+SIVt61qU X0kA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h72si3532928pfj.20.2017.11.30.08.40.43; Thu, 30 Nov 2017 08:40:43 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753463AbdK3Qkf (ORCPT + 28 others); Thu, 30 Nov 2017 11:40:35 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57480 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753379AbdK3Qjr (ORCPT ); Thu, 30 Nov 2017 11:39:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 406EB1AFA; Thu, 30 Nov 2017 08:39:46 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 117483F318; Thu, 30 Nov 2017 08:39:46 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 6E3FD1AE3DFE; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 16/18] arm64: entry: Add fake CPU feature for unmapping the kernel at EL0 Date: Thu, 30 Nov 2017 16:39:44 +0000 Message-Id: <1512059986-21325-17-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Allow explicit disabling of the entry trampoline on the kernel command line (kaiser=off) by adding a fake CPU feature (ARM64_UNMAP_KERNEL_AT_EL0) that can be used to toggle the alternative sequences in our entry code and avoid use of the trampoline altogether if desired. This also allows us to make use of a static key in arm64_kernel_unmapped_at_el0(). Signed-off-by: Will Deacon --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/mmu.h | 3 ++- arch/arm64/kernel/cpufeature.c | 41 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/entry.S | 11 +++++++---- 4 files changed, 52 insertions(+), 6 deletions(-) -- 2.1.4 diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 2ff7c5e8efab..b4537ffd1018 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -41,7 +41,8 @@ #define ARM64_WORKAROUND_CAVIUM_30115 20 #define ARM64_HAS_DCPOP 21 #define ARM64_SVE 22 +#define ARM64_UNMAP_KERNEL_AT_EL0 23 -#define ARM64_NCAPS 23 +#define ARM64_NCAPS 24 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index c07954638658..da6f12e40714 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -36,7 +36,8 @@ typedef struct { static inline bool arm64_kernel_unmapped_at_el0(void) { - return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0); + return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) && + cpus_have_const_cap(ARM64_UNMAP_KERNEL_AT_EL0); } extern void paging_init(void); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index c5ba0097887f..72fc55d22ddb 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -845,6 +845,40 @@ static bool has_no_fpsimd(const struct arm64_cpu_capabilities *entry, int __unus ID_AA64PFR0_FP_SHIFT) < 0; } +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +static int __kaiser_forced; /* 0: not forced, >0: forced on, <0: forced off */ + +static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, + int __unused) +{ + /* Forced on command line? */ + if (__kaiser_forced) { + pr_info("KAISER forced %s by command line option\n", + __kaiser_forced > 0 ? "ON" : "OFF"); + return __kaiser_forced > 0; + } + + /* Useful for KASLR robustness */ + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + return true; + + return false; +} + +static int __init parse_kaiser(char *str) +{ + bool enabled; + int ret = strtobool(str, &enabled); + + if (ret) + return ret; + + __kaiser_forced = enabled ? 1 : -1; + return 0; +} +__setup("kaiser=", parse_kaiser); +#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ + static const struct arm64_cpu_capabilities arm64_features[] = { { .desc = "GIC system register CPU interface", @@ -931,6 +965,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .def_scope = SCOPE_SYSTEM, .matches = hyp_offset_low, }, +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 + { + .capability = ARM64_UNMAP_KERNEL_AT_EL0, + .def_scope = SCOPE_SYSTEM, + .matches = unmap_kernel_at_el0, + }, +#endif { /* FP/SIMD is not implemented */ .capability = ARM64_HAS_NO_FPSIMD, diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index a5ec6ab5c711..d8775f55e930 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -74,6 +74,7 @@ .macro kernel_ventry, el, label, regsize = 64 .align 7 #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 .if \el == 0 .if \regsize == 64 mrs x30, tpidrro_el0 @@ -82,6 +83,7 @@ mov x30, xzr .endif .endif +alternative_else_nop_endif #endif sub sp, sp, #S_FRAME_SIZE @@ -323,10 +325,10 @@ alternative_else_nop_endif ldr lr, [sp, #S_LR] add sp, sp, #S_FRAME_SIZE // restore sp -#ifndef CONFIG_UNMAP_KERNEL_AT_EL0 - eret -#else .if \el == 0 +alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 +#ifdef CONFIG_UNMAP_KERNEL_AT_EL0 +alternative_if ARM64_UNMAP_KERNEL_AT_EL0 bne 4f msr far_el1, x30 tramp_alias x30, tramp_exit_native @@ -334,10 +336,11 @@ alternative_else_nop_endif 4: tramp_alias x30, tramp_exit_compat br x30 +alternative_else_nop_endif +#endif .else eret .endif -#endif .endm .macro irq_stack_entry From patchwork Thu Nov 30 16:39:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120227 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp14902qgn; Thu, 30 Nov 2017 08:41:20 -0800 (PST) X-Google-Smtp-Source: AGs4zMb8N9w43GO5ruA/p+vjvhQBPZ1JFQBPjfVOpNQ60C6GWhnZd5r79gPLye6Z+RivGEckUpj+ X-Received: by 10.101.70.11 with SMTP id v11mr2910014pgq.306.1512060080089; Thu, 30 Nov 2017 08:41:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060080; cv=none; d=google.com; s=arc-20160816; b=N9Rf3i4utnWi+lmq14TM9LT/y3+rrJHHQ/QZvLrBBhEo+a9RULuSKSwbX7/LvX5jEL i6BEZafxL7DSBOawI17rVwxh2EnI6NSaikSELbgXmFKqbvyTTPtNNyhlnaRkMzFar4Xj 34/mERMy1IJdXK0gXZtQmj3H/lQmbwgWbTtlRJCrlkDYnlqoVjN45xufFfO98ko1DfmN srkRyrJRa408T6pKgAIjDDge//ZJiA4Dtvni4Dx69LpYWl3SxVEAFsJcZjsYhmRhkqLm h91ijAhetQ/d1doYy1xFRjFGFHBz67T+9ctHn3MyhNOY/ZSEI8g9lr++qZ1kH65SglEX 5ylA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=n70RXBSLuXhyWLspoI0EYQyEdySg+bxuJ4tPpiOFrUg=; b=OoYRQg62hOFEOXK5DKn5TqlYJtnQe7HW5EEGa71vs3X9nthsbiAhNMJZeU50q6x63y OAZO9acwo0+rT1Wu8kefyaTxgrAnaJw6kzfTFKIti6C55Lot/G61TXVAH6BGUocEwioe E/dz60V3UaREgegAe53r4MjW1cODCT2LbgTHKZKYy8OnlGd2kyWiUXa+WfK55fQ68RV3 xS1CLxUxOFKqfocvSF2WWIfikvAiPsvT/4FVuFuDUjDH2uBkTIm4nfqSVvuK/sA8FMfO dXb1Y9kjh2RfCVpLRQezxB45whdI0WmI3OkLHkFVkcRoSCln9p0kkigtG2p2BwzHuwvp ALxg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g5si3409671pll.15.2017.11.30.08.41.19; Thu, 30 Nov 2017 08:41:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753482AbdK3Qkh (ORCPT + 28 others); Thu, 30 Nov 2017 11:40:37 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:57576 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753369AbdK3Qjr (ORCPT ); Thu, 30 Nov 2017 11:39:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C740A1AED; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 98F863F318; Thu, 30 Nov 2017 08:39:45 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 7E86F1AE3E09; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 17/18] arm64: Kconfig: Add CONFIG_UNMAP_KERNEL_AT_EL0 Date: Thu, 30 Nov 2017 16:39:45 +0000 Message-Id: <1512059986-21325-18-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add a Kconfig entry to control use of the entry trampoline, which allows us to unmap the kernel whilst running in userspace and improve the robustness of KASLR. Signed-off-by: Will Deacon --- arch/arm64/Kconfig | 13 +++++++++++++ 1 file changed, 13 insertions(+) -- 2.1.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index fdcc7b9bb15d..3af1657fcac3 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -833,6 +833,19 @@ config FORCE_MAX_ZONEORDER However for 4K, we choose a higher default value, 11 as opposed to 10, giving us 4M allocations matching the default size used by generic code. +config UNMAP_KERNEL_AT_EL0 + bool "Unmap kernel when running in userspace (aka \"KAISER\")" + default y + help + Some attacks against KASLR make use of the timing difference between + a permission fault which could arise from a page table entry that is + present in the TLB, and a translation fault which always requires a + page table walk. This option defends against these attacks by unmapping + the kernel whilst running in userspace, therefore forcing translation + faults for all of kernel space. + + If unsure, say Y. + menuconfig ARMV8_DEPRECATED bool "Emulate deprecated/obsolete ARMv8 instructions" depends on COMPAT From patchwork Thu Nov 30 16:39:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 120228 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp16461qgn; Thu, 30 Nov 2017 08:42:38 -0800 (PST) X-Google-Smtp-Source: AGs4zMYKr88vIjG1ZycQ66PNf5ZTDlB91On5b03tQkYM6QtDOFvJBNHXvxdi4Y+9fJqpglaH6XUb X-Received: by 10.99.120.195 with SMTP id t186mr2969083pgc.62.1512060158144; Thu, 30 Nov 2017 08:42:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512060158; cv=none; d=google.com; s=arc-20160816; b=uG0R0NTJLlqpDVcObbx0uYnm6BBJYVx4Ok06mA2dt3Gyhda6MCuvP8oKRI/su76JzS hTBGLwE+8y8Hw6wuuZjg4pjC07As7yndkYdVtAq8CELjM2JLpM6QwU9fKYkn4McD8ueL oLTy0GHPxP6mrTMg8f4g7erC6vswEnrvyT+XtmqGADFK5fcMaOoZUrQtqFAHMtEE8Qxd jfw1DH2UYS/mzXk0z9M2hOjqQVTvV5l+PmaCaNhh5V8rJSaNy7nbU5NI8FIaDK5DUSb2 2pimJC6jcL2Olo5nLFyBMiXjo2BP+us7PCftiHgcJGyE77yuj5dEF5EZjPAl9WyOSNl1 opCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=sWLRcRShKWR4TB3jKdyMc0sCk28ScWZ5UxqSh2LbkYI=; b=HjEExsy6lOsAfvQC4OlIrec2Vz8yIBPZs3lebZIb1AwlhYpKGbVKaTY+2T08jcFrY/ yefypPh3V7M2pSch6FkzXHRBzVSqbPyo11wRmyd799b9Bt+IzNQEZLnXsKdTbrfa24bu 6ZjRRINdfXxMxRQ1FB1RkO/sL6nwu4NsVFFTDiDvfcy/pSGKBVaYZVks1+tuIryqzVnz gbsu/sJyg+52Lc83EFIiVUq93F62cst8aDWfKsTVlnmQzOx22/xyeHkQmmpuy8yijTuc 9BDGYLn0W53UvZp5qgvR3YvdXsoF3pTNLB8uT0+3hiCq1+IVA6/ALyygsn2sq3QHJj5w 5b5w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g1si3492329pfj.303.2017.11.30.08.42.37; Thu, 30 Nov 2017 08:42:38 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753326AbdK3Qkd (ORCPT + 28 others); Thu, 30 Nov 2017 11:40:33 -0500 Received: from foss.arm.com ([217.140.101.70]:57586 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753382AbdK3Qjr (ORCPT ); Thu, 30 Nov 2017 11:39:47 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 492251B33; Thu, 30 Nov 2017 08:39:46 -0800 (PST) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 1A6083F5B3; Thu, 30 Nov 2017 08:39:46 -0800 (PST) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id 8ED121AE3BC8; Thu, 30 Nov 2017 16:39:48 +0000 (GMT) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, catalin.marinas@arm.com, mark.rutland@arm.com, ard.biesheuvel@linaro.org, sboyd@codeaurora.org, dave.hansen@linux.intel.com, keescook@chromium.org, msalter@redhat.com, labbott@redhat.com, tglx@linutronix.de, Will Deacon Subject: [PATCH v2 18/18] perf: arm_spe: Disallow userspace profiling when arm_kernel_unmapped_at_el0() Date: Thu, 30 Nov 2017 16:39:46 +0000 Message-Id: <1512059986-21325-19-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1512059986-21325-1-git-send-email-will.deacon@arm.com> References: <1512059986-21325-1-git-send-email-will.deacon@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When running with the kernel unmapped whilst at EL0, the virtually-addressed SPE buffer is also unmapped, which can lead to buffer faults if userspace profiling is enabled. This patch prohibits SPE profiling of userspace when arm_kernel_unmapped_at_el0(). Signed-off-by: Will Deacon --- drivers/perf/arm_spe_pmu.c | 7 +++++++ 1 file changed, 7 insertions(+) -- 2.1.4 diff --git a/drivers/perf/arm_spe_pmu.c b/drivers/perf/arm_spe_pmu.c index 8ce262fc2561..c028db8973a4 100644 --- a/drivers/perf/arm_spe_pmu.c +++ b/drivers/perf/arm_spe_pmu.c @@ -675,6 +675,13 @@ static int arm_spe_pmu_event_init(struct perf_event *event) return -EOPNOTSUPP; /* + * If kernelspace is unmapped when running at EL0, then the SPE + * buffer will fault and prematurely terminate the AUX session. + */ + if (arm64_kernel_unmapped_at_el0() && !attr->exclude_user) + dev_warn_once(&spe_pmu->pdev->dev, "unable to write to profiling buffer from EL0. Try passing \"kaiser=off\" on the kernel command line"); + + /* * Feedback-directed frequency throttling doesn't work when we * have a buffer of samples. We'd need to manually count the * samples in the buffer when it fills up and adjust the event