From patchwork Mon Feb 26 08:19:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129613 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3367868lja; Mon, 26 Feb 2018 00:39:11 -0800 (PST) X-Google-Smtp-Source: AH8x227xiMATmviO3U48g/62CzfO1gqahaPFjcR1GiKiusJXBokwjQhe+LOlwsR7gMpf/LO/XtdE X-Received: by 10.98.33.204 with SMTP id o73mr9912304pfj.54.1519634351394; Mon, 26 Feb 2018 00:39:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519634351; cv=none; d=google.com; s=arc-20160816; b=yQF6MO/ZPmZPIq+jXrVvV+Dilf+575e4coegsx4Md/mQgo0PSrHtq037UHcF8IlUpY 9RQq7wJ37h8Q8+tpHlSlzlH3itsnOtsNkXTdvNOsCw9WCvwG1oWfpyCh3EvTIuF/hpnG sRKSHjVtpexrN/jGNl7ULop2Xo7KSTkGhlGoPXuTSN0z5dGpwEn/IuJG4Xo92xy/uuXA FWzIVsItgjKEWab0Ixj1MpkORcCM28HwfgjQeDD9xG0CnQuxZsjttZxwT6O1RjHSF+VB WrSqBAw7dM5yzwe55LN4VodMEJc6ggYNLkQIpd7ACwy/2JsgL74gFiwc0WhjubhkYOV6 GqrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=Bg7B0hfDb1LbDSK0bEabGd/Si+r1Y+auDWs+NQonZeA=; b=vODPmWDoYktC/pBIBRr42y65jxIZdnyyJIEIBFlJZ5g9mHSiqH5XrYQ94ZSzMVzzGl KyMpqaM8zu5wny4jAWmdGvcpkL+jHeCva8b7shmQQOk8uc9ogXFtwt6O95O6V7PHcFqn 3BB1JJ6ZITXBYiJytweN+M8Bf3JnQV2fUD0DhxBTWYhV1a8GkiOEQy70FF5U/nwuYnn8 0W1KVv3FVxDc3rs1rA61/7/A63gsCGMTuPSoLldi0Dbc/QGCy9MI/xTHLExe+XNsVSF3 Ui7TWQuO5HfOdLbqv0imC5U9h2x/H46kpT2p9vR75JGQEYJ8Abyo+gzpo0igMGQtQ9h/ k3qA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BMCqSGmr; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i11si1205565pfi.388.2018.02.26.00.39.11; Mon, 26 Feb 2018 00:39:11 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BMCqSGmr; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752056AbeBZIjK (ORCPT + 10 others); Mon, 26 Feb 2018 03:39:10 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:37479 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751567AbeBZIVP (ORCPT ); Mon, 26 Feb 2018 03:21:15 -0500 Received: by mail-pg0-f67.google.com with SMTP id y26so5909540pgv.4 for ; Mon, 26 Feb 2018 00:21:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Bg7B0hfDb1LbDSK0bEabGd/Si+r1Y+auDWs+NQonZeA=; b=BMCqSGmrderHIelq7P0/XNM0Gw1/I4t6N8Y2W0FKBy9v763WQZerCBfXz13oFsFayu yoFt35kIXJyQoufduZ1a2dSr2jPiOJOVjKqRHzdaNxaxvHqz2+VjAV3vgo2YZk5d9Lao V77TW6Qdzo7OR+yhOWhPZt8rBHgnlwCw8EkMU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Bg7B0hfDb1LbDSK0bEabGd/Si+r1Y+auDWs+NQonZeA=; b=DUSyLFO6q2vumzzhWzVq/fejHKmp27balzXo/+dwZd4FmaNA5gaLvZpV2xJp0plnrv RVEdsTr1kdKLGKn1WMFBu4JSI/wSpnTm2mja2NG4uCI9SlnjqrwAdB8SVDAfXpKsdcL7 ydxxs+/SAHD7k30PJj1TWTaMrMpk/uHfyO56W+PlHUIuITs55rw+b8Far0ZAqJwDeWq1 K8Bzj47d4hfNd3JJzDJR6NA/vOkjNRWxUCUT1d408X/IXOvfeNPkTdReHoBcpDOjGx+t JWaT0QdD4EJqH+QD2P00Jn6O9KmVE1yMhUU8vbzTSF+ShLdXmmmMqObb5BYa+rz56TCu swYQ== X-Gm-Message-State: APf1xPAkWQUfqjOLHIT+E/bIBaoEzM0KY2M4JwrZncO8zIrBoZ62AK07 07Rj0UTGV4esNeRh83nOCVkpNA== X-Received: by 10.99.9.195 with SMTP id 186mr7733855pgj.152.1519633275074; Mon, 26 Feb 2018 00:21:15 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.21.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:21:14 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-mm@kvack.org (open list:MEMORY MANAGEMENT), linux-kernel@vger.kernel.org (open list) Cc: Laura Abbott Subject: [PATCH 01/52] mm: Introduce lm_alias Date: Mon, 26 Feb 2018 16:19:35 +0800 Message-Id: <1519633227-29832-2-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Laura Abbott commit 568c5fe5a54 upstream. Certain architectures may have the kernel image mapped separately to alias the linear map. Introduce a macro lm_alias to translate a kernel image symbol into its linear alias. This is used in part with work to add CONFIG_DEBUG_VIRTUAL support for arm64. Reviewed-by: Mark Rutland Tested-by: Mark Rutland Signed-off-by: Laura Abbott Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- include/linux/mm.h | 4 ++++ 1 file changed, 4 insertions(+) -- 2.7.4 diff --git a/include/linux/mm.h b/include/linux/mm.h index 2217e2f..edd2480 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -76,6 +76,10 @@ extern int mmap_rnd_compat_bits __read_mostly; #define page_to_virt(x) __va(PFN_PHYS(page_to_pfn(x))) #endif +#ifndef lm_alias +#define lm_alias(x) __va(__pa_symbol(x)) +#endif + /* * To prevent common memory management code establishing * a zero page mapping on a read fault. From patchwork Mon Feb 26 08:19:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129562 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3355847lja; Mon, 26 Feb 2018 00:21:57 -0800 (PST) X-Google-Smtp-Source: AH8x224LH13JUaxN8ojyutLBX858GwSbVO+fFMdEbe6zcsqH2aON/tbPfeObczbAWj0nv34n7bbf X-Received: by 2002:a17:902:7509:: with SMTP id i9-v6mr10116945pll.220.1519633317572; Mon, 26 Feb 2018 00:21:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633317; cv=none; d=google.com; s=arc-20160816; b=FKcC9mf3dW954v2LByA6x7D/AwQythcB1giugYXFNz8XfqLUC+ISUtJsuQxMo72bmw N3g9o/H968N6yBvFfyeA6pbwBCtkL9wjSrYotQ7V/FnK6px9I0dBqsn0TdaUft8PkQPH AVfSnoIMFQkSnmRANFXtAfZrF1XtyKhKnvpIvr8YMtPZ8TTiuiL9ju6t93oF/7ZoD50g iIKG2E8WCLzwB3OQIprdXU7YYM+y4NArw9fmCnpyIjL4OtN1Z1sFYLx82AMQ6rF3u46q OU/Ze3zLpls8DCxfQJ9tGQctmBPnZzlWKJYxKt1e7ZaQm02tTMHA2uVXoWYd4jAAhkfq 1KoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=DuN6SyTg5l99yxUh5yRn/aUeDXw1S1qe26nBS+h01No=; b=0CwD8xgzuTN7KwKXi2d+os3xWC1+FL/PjBYg/Hz15Y6HDwLm+CpWe85kqTR742dK7i xZVrig4mMc7eZklVjKSY1Dq9HYpQCz4hJ+4JNJsHVr8WP+sdKJ9NRBum8xakSkTfkf7Y LPF2PkxKWZ8J/CmaROq317NxAF1lxdJsME0NOcroE9rIG7Ame2uh0nuv8iHka2f5jzXy ul5sn8gOPee31Hkwipw9YomkKLknqycbwiHIYJR12PruXSbMneM0klp8Rw3QWjmsUUGa FV5Grgg+L2Xlie9zejXEp3usp5FwBHXk3f9f7Olcnk32Gd2nBXDxd0EDdXyYbEockLVH xyYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AigfDg1D; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y8si5246129pgp.602.2018.02.26.00.21.57; Mon, 26 Feb 2018 00:21:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AigfDg1D; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752341AbeBZIVz (ORCPT + 10 others); Mon, 26 Feb 2018 03:21:55 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:40481 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752333AbeBZIVv (ORCPT ); Mon, 26 Feb 2018 03:21:51 -0500 Received: by mail-pl0-f65.google.com with SMTP id i6so8865314plt.7 for ; Mon, 26 Feb 2018 00:21:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DuN6SyTg5l99yxUh5yRn/aUeDXw1S1qe26nBS+h01No=; b=AigfDg1DPwo3XHYjpcgdgALqBsi/BmMdAak9pXjBPcwu5d8TKBLNK9QfCC3AQ5yQhe fvGA0vlo2oivswOtUbjM8yFI5nQlLhHzcfCIQKPGdJ2qFhUG6f+NhtZsloNXAcSEMiaY Jnt+GU7Ij7IJRMR/hocwiT2hCasJVH0HgV+6Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DuN6SyTg5l99yxUh5yRn/aUeDXw1S1qe26nBS+h01No=; b=ohjEmsphWPiClpPL6tKM+9wNY/zUdWIBtPCDUH3CiYqwqPmjzppHMp9UVsRIJGIro0 D09hDR14UTqh2JL8i8ka99gb2aot2QTRRfdyxtLYt/1O+7D6y3XIHi91QrA/88jBWNeG K+TLJ0+Kk4bS7dWQswKuzpnsfME/laLm5W70De93nOCLrteeOfD6I8sR3CObqGK3ZVtu lJvgWwoM5wqZwW8H6+N/DX2Qm068ZAqecloxG6DSgzveDYFLT5Nu1Isleq5ymd31lIHe FEtm0Yf22gPizFx/r1K5pRQuck4XTMV84FXZmxFMB4na3Lu7hmw1sLS5mpYf0ABHrQee 71yw== X-Gm-Message-State: APf1xPBiwAt4R3PfkPH3GXQEo8R87QWbzlDVWdzbL5T1tyV673RaMg6Q WAi7aVqz6fD7yBYXbayfkv6+NA== X-Received: by 2002:a17:902:bf44:: with SMTP id u4-v6mr6647841pls.68.1519633311154; Mon, 26 Feb 2018 00:21:51 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.21.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:21:50 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: James Morse , Kees Cook Subject: [PATCH 06/52] arm64: Factor out PAN enabling/disabling into separate uaccess_* macros Date: Mon, 26 Feb 2018 16:19:40 +0800 Message-Id: <1519633227-29832-7-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Catalin Marinas commit bd38967d406 upstream. This patch moves the directly coded alternatives for turning PAN on/off into separate uaccess_{enable,disable} macros or functions. The asm macros take a few arguments which will be used in subsequent patches. Note that any (unlikely) access that the compiler might generate between uaccess_enable() and uaccess_disable(), other than those explicitly specified by the user access code, will not be protected by PAN. Cc: Will Deacon Cc: James Morse Cc: Kees Cook Reviewed-by: Mark Rutland Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/futex.h | 17 ++++---- arch/arm64/include/asm/uaccess.h | 79 +++++++++++++++++++++++++++++++----- arch/arm64/kernel/armv8_deprecated.c | 11 +++-- arch/arm64/lib/clear_user.S | 11 ++--- arch/arm64/lib/copy_from_user.S | 11 ++--- arch/arm64/lib/copy_in_user.S | 11 ++--- arch/arm64/lib/copy_to_user.S | 11 ++--- 7 files changed, 93 insertions(+), 58 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index f2585cd..85c4a89 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -21,15 +21,12 @@ #include #include -#include -#include #include -#include #define __futex_atomic_op(insn, ret, oldval, uaddr, tmp, oparg) \ +do { \ + uaccess_enable(); \ asm volatile( \ - ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, \ - CONFIG_ARM64_PAN) \ " prfm pstl1strm, %2\n" \ "1: ldxr %w1, %2\n" \ insn "\n" \ @@ -44,11 +41,11 @@ " .popsection\n" \ _ASM_EXTABLE(1b, 4b) \ _ASM_EXTABLE(2b, 4b) \ - ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \ - CONFIG_ARM64_PAN) \ : "=&r" (ret), "=&r" (oldval), "+Q" (*uaddr), "=&r" (tmp) \ : "r" (oparg), "Ir" (-EFAULT) \ - : "memory") + : "memory"); \ + uaccess_disable(); \ +} while (0) static inline int futex_atomic_op_inuser (int encoded_op, u32 __user *uaddr) @@ -118,8 +115,8 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, if (!access_ok(VERIFY_WRITE, uaddr, sizeof(u32))) return -EFAULT; + uaccess_enable(); asm volatile("// futex_atomic_cmpxchg_inatomic\n" -ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, CONFIG_ARM64_PAN) " prfm pstl1strm, %2\n" "1: ldxr %w1, %2\n" " sub %w3, %w1, %w4\n" @@ -134,10 +131,10 @@ ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, CONFIG_ARM64_PAN) " .popsection\n" _ASM_EXTABLE(1b, 4b) _ASM_EXTABLE(2b, 4b) -ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, CONFIG_ARM64_PAN) : "+r" (ret), "=&r" (val), "+Q" (*uaddr), "=&r" (tmp) : "r" (oldval), "r" (newval), "Ir" (-EFAULT) : "memory"); + uaccess_disable(); *uval = val; return ret; diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 811cf16..94e1457 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -18,6 +18,11 @@ #ifndef __ASM_UACCESS_H #define __ASM_UACCESS_H +#include +#include + +#ifndef __ASSEMBLY__ + /* * User space memory access functions */ @@ -26,10 +31,8 @@ #include #include -#include #include #include -#include #include #include #include @@ -121,6 +124,44 @@ static inline void set_fs(mm_segment_t fs) " .popsection\n" /* + * User access enabling/disabling. + */ +#define __uaccess_disable(alt) \ +do { \ + asm(ALTERNATIVE("nop", SET_PSTATE_PAN(1), alt, \ + CONFIG_ARM64_PAN)); \ +} while (0) + +#define __uaccess_enable(alt) \ +do { \ + asm(ALTERNATIVE("nop", SET_PSTATE_PAN(0), alt, \ + CONFIG_ARM64_PAN)); \ +} while (0) + +static inline void uaccess_disable(void) +{ + __uaccess_disable(ARM64_HAS_PAN); +} + +static inline void uaccess_enable(void) +{ + __uaccess_enable(ARM64_HAS_PAN); +} + +/* + * These functions are no-ops when UAO is present. + */ +static inline void uaccess_disable_not_uao(void) +{ + __uaccess_disable(ARM64_ALT_PAN_NOT_UAO); +} + +static inline void uaccess_enable_not_uao(void) +{ + __uaccess_enable(ARM64_ALT_PAN_NOT_UAO); +} + +/* * The "__xxx" versions of the user access functions do not verify the address * space - it must have been done previously with a separate "access_ok()" * call. @@ -147,8 +188,7 @@ static inline void set_fs(mm_segment_t fs) do { \ unsigned long __gu_val; \ __chk_user_ptr(ptr); \ - asm(ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_ALT_PAN_NOT_UAO,\ - CONFIG_ARM64_PAN)); \ + uaccess_enable_not_uao(); \ switch (sizeof(*(ptr))) { \ case 1: \ __get_user_asm("ldrb", "ldtrb", "%w", __gu_val, (ptr), \ @@ -169,9 +209,8 @@ do { \ default: \ BUILD_BUG(); \ } \ + uaccess_disable_not_uao(); \ (x) = (__force __typeof__(*(ptr)))__gu_val; \ - asm(ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_ALT_PAN_NOT_UAO,\ - CONFIG_ARM64_PAN)); \ } while (0) #define __get_user(x, ptr) \ @@ -216,8 +255,7 @@ do { \ do { \ __typeof__(*(ptr)) __pu_val = (x); \ __chk_user_ptr(ptr); \ - asm(ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_ALT_PAN_NOT_UAO,\ - CONFIG_ARM64_PAN)); \ + uaccess_enable_not_uao(); \ switch (sizeof(*(ptr))) { \ case 1: \ __put_user_asm("strb", "sttrb", "%w", __pu_val, (ptr), \ @@ -238,8 +276,7 @@ do { \ default: \ BUILD_BUG(); \ } \ - asm(ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_ALT_PAN_NOT_UAO,\ - CONFIG_ARM64_PAN)); \ + uaccess_disable_not_uao(); \ } while (0) #define __put_user(x, ptr) \ @@ -332,4 +369,26 @@ extern long strncpy_from_user(char *dest, const char __user *src, long count); extern __must_check long strlen_user(const char __user *str); extern __must_check long strnlen_user(const char __user *str, long n); +#else /* __ASSEMBLY__ */ + +#include + +/* + * User access enabling/disabling macros. These are no-ops when UAO is + * present. + */ + .macro uaccess_disable_not_uao, tmp1 +alternative_if ARM64_ALT_PAN_NOT_UAO + SET_PSTATE_PAN(1) +alternative_else_nop_endif + .endm + + .macro uaccess_enable_not_uao, tmp1, tmp2 +alternative_if ARM64_ALT_PAN_NOT_UAO + SET_PSTATE_PAN(0) +alternative_else_nop_endif + .endm + +#endif /* __ASSEMBLY__ */ + #endif /* __ASM_UACCESS_H */ diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c index c0ede23..29d2ad8 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -14,7 +14,6 @@ #include #include -#include #include #include #include @@ -285,10 +284,10 @@ static void __init register_insn_emulation_sysctl(struct ctl_table *table) #define __SWP_LL_SC_LOOPS 4 #define __user_swpX_asm(data, addr, res, temp, temp2, B) \ +do { \ + uaccess_enable(); \ __asm__ __volatile__( \ " mov %w3, %w7\n" \ - ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, \ - CONFIG_ARM64_PAN) \ "0: ldxr"B" %w2, [%4]\n" \ "1: stxr"B" %w0, %w1, [%4]\n" \ " cbz %w0, 2f\n" \ @@ -306,13 +305,13 @@ static void __init register_insn_emulation_sysctl(struct ctl_table *table) " .popsection" \ _ASM_EXTABLE(0b, 4b) \ _ASM_EXTABLE(1b, 4b) \ - ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \ - CONFIG_ARM64_PAN) \ : "=&r" (res), "+r" (data), "=&r" (temp), "=&r" (temp2) \ : "r" ((unsigned long)addr), "i" (-EAGAIN), \ "i" (-EFAULT), \ "i" (__SWP_LL_SC_LOOPS) \ - : "memory") + : "memory"); \ + uaccess_disable(); \ +} while (0) #define __user_swp_asm(data, addr, res, temp, temp2) \ __user_swpX_asm(data, addr, res, temp, temp2, "") diff --git a/arch/arm64/lib/clear_user.S b/arch/arm64/lib/clear_user.S index 5d1cad3..d7150e3 100644 --- a/arch/arm64/lib/clear_user.S +++ b/arch/arm64/lib/clear_user.S @@ -17,10 +17,7 @@ */ #include -#include -#include -#include -#include +#include .text @@ -33,8 +30,7 @@ * Alignment fixed up by hardware. */ ENTRY(__clear_user) -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_enable_not_uao x2, x3 mov x2, x1 // save the size for fixup return subs x1, x1, #8 b.mi 2f @@ -54,8 +50,7 @@ uao_user_alternative 9f, strh, sttrh, wzr, x0, 2 b.mi 5f uao_user_alternative 9f, strb, sttrb, wzr, x0, 0 5: mov x0, #0 -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_disable_not_uao x2 ret ENDPROC(__clear_user) diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S index 4fd67ea..cfe1339 100644 --- a/arch/arm64/lib/copy_from_user.S +++ b/arch/arm64/lib/copy_from_user.S @@ -16,11 +16,8 @@ #include -#include -#include #include -#include -#include +#include /* * Copy from user space to a kernel buffer (alignment handled by the hardware) @@ -67,12 +64,10 @@ end .req x5 ENTRY(__arch_copy_from_user) -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_enable_not_uao x3, x4 add end, x0, x2 #include "copy_template.S" -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_disable_not_uao x3 mov x0, #0 // Nothing to copy ret ENDPROC(__arch_copy_from_user) diff --git a/arch/arm64/lib/copy_in_user.S b/arch/arm64/lib/copy_in_user.S index f7292dd0..718b1c4 100644 --- a/arch/arm64/lib/copy_in_user.S +++ b/arch/arm64/lib/copy_in_user.S @@ -18,11 +18,8 @@ #include -#include -#include #include -#include -#include +#include /* * Copy from user space to user space (alignment handled by the hardware) @@ -68,12 +65,10 @@ end .req x5 ENTRY(__copy_in_user) -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_enable_not_uao x3, x4 add end, x0, x2 #include "copy_template.S" -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_disable_not_uao x3 mov x0, #0 ret ENDPROC(__copy_in_user) diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S index 7a7efe2..e99e31c 100644 --- a/arch/arm64/lib/copy_to_user.S +++ b/arch/arm64/lib/copy_to_user.S @@ -16,11 +16,8 @@ #include -#include -#include #include -#include -#include +#include /* * Copy to user space from a kernel buffer (alignment handled by the hardware) @@ -66,12 +63,10 @@ end .req x5 ENTRY(__arch_copy_to_user) -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_enable_not_uao x3, x4 add end, x0, x2 #include "copy_template.S" -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_disable_not_uao x3 mov x0, #0 ret ENDPROC(__arch_copy_to_user) From patchwork Mon Feb 26 08:19:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129563 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3355945lja; Mon, 26 Feb 2018 00:22:07 -0800 (PST) X-Google-Smtp-Source: AG47ELvxdaMQTTgIQngO9WY7Xqagf1mtPva8zZjKk+kIGMffw+5WoDv8vt8hO5gBHBZL3uuyC2wM X-Received: by 10.98.65.72 with SMTP id o69mr7762030pfa.97.1519633327830; Mon, 26 Feb 2018 00:22:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633327; cv=none; d=google.com; s=arc-20160816; b=nAYL39sGmWp5yc02Om7ymrveLvR8nvGVXatxWReZO5hxj5bHlE3Twv22XPoaU6aZSl Di9E12sddAOkJysEdaaRQeYQdk+oE1YmRLvIxeJQG/nt9ELm8BzlypCt3Gq9HH5aPhmm drGG/8oDlMDXj+Kp9JmgDtMBgXQcqpRWNkaHHSA90AtOPE+a+2lAqxZxv9ODA8L93A4D wupK8HIXpSlzqVy9SX7hvXY/M4/SXHyA+XgTKbz4nDbikyMfoUSezrys/txJt4nvY0t/ k8/FBGV1cX3WVCwJOfrK3o6Wx84iTCcf4Ptujm5A1BON7+nJrFrT5AE7z1ZFTUrChec+ HOUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=1Y8BanaJo1dWq0GIhEtFd/7ONzz9WFIJLPbGyXUfe10=; b=CYOMUfOW9cVSz4Bi3RNAYutgxIz7pmqPT9+XdqlNmzfqA4/cUwEovjF9fIYW+SrQRU 7m40wpKUptxNZUqZDKlSsvp86oEX002XP3GqRq2+rgz5ZhYD+34U9LW3jytWpRDAwTMN lMCklYuDeKIfxBd0WArls4MdNYd0omzeIjAL6VgXCICv2Fu5ljHNjkHNfvdngDwaBQx4 P9TAovnD+xIMAPEMDspxZ6G6jevxExkeN0amC2XRF2PeUxYhePgIiU0oNFwaMhD7qnJ8 UQRdpeG45GX8H+L21NjSuT7MiOeNYDOtZfgrfzhGlMJzN8kc9OM28LOlpMyRggCvVtEU YGsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=d1w8YmAm; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o2si1022986pfg.286.2018.02.26.00.22.07; Mon, 26 Feb 2018 00:22:07 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=d1w8YmAm; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752250AbeBZIWG (ORCPT + 10 others); Mon, 26 Feb 2018 03:22:06 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:46754 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752354AbeBZIV7 (ORCPT ); Mon, 26 Feb 2018 03:21:59 -0500 Received: by mail-pf0-f195.google.com with SMTP id z10so1880523pfh.13 for ; Mon, 26 Feb 2018 00:21:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1Y8BanaJo1dWq0GIhEtFd/7ONzz9WFIJLPbGyXUfe10=; b=d1w8YmAmizyCUP7HKtkENUc3d3YvSxAw5AtnW6Afct7gea/L3eJM4pEQAf8/ARNI+V TN675bGaucTxnQTM82msEVVXFeVeVqeLu59ELxwAHdIlszH9UM5bYAU38oobUs575BDj xJKuXCfeHowqYhanam1zRPPesyj9gRdIGWMJk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1Y8BanaJo1dWq0GIhEtFd/7ONzz9WFIJLPbGyXUfe10=; b=Yapdj4R506LyGqSOeZtRd9r7qS5iAG3ftCQ9DePSay8ohdNn7+AYFuI8xaK9xbslte i4QkEtNGnRhclHGci52O755uUAY/fZRnu3rhHOCz95ERbLTttlE2GVht+8ne3/HzRPQc GBU3MhaBtJIkDu9fsKk8+PbR5YtsdSpcF1vsn8N5tI7DcLS9Y4KQRRntCi2OiQNZpT4r HwWxZuGMP/ifpdGimo9NmXa/i8Dil2GbDYgWxlqSnsL46KsKO/4n+MrV7yZ0OJrhVbho aeHUBl49xf02CBHo7lEHPLsE2+qpvRAaFizJ9uAHxnngVPrVue5brHO1Kghm3MjQcDQv kgSQ== X-Gm-Message-State: APf1xPCJlNkykNBCZaqkg5nh4xSGrPJmV1LA+0MO+7UpXEAiSMf62kXU JYytdsJiFNxjad3gNNxOaco2Sg== X-Received: by 10.98.204.69 with SMTP id a66mr9820419pfg.33.1519633318597; Mon, 26 Feb 2018 00:21:58 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.21.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:21:58 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: James Morse , Kees Cook Subject: [PATCH 07/52] arm64: Factor out TTBR0_EL1 post-update workaround into a specific asm macro Date: Mon, 26 Feb 2018 16:19:41 +0800 Message-Id: <1519633227-29832-8-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Catalin Marinas commit f33bcf03e6 upstream This patch takes the errata workaround code out of cpu_do_switch_mm into a dedicated post_ttbr0_update_workaround macro which will be reused in a subsequent patch. Cc: Will Deacon Cc: James Morse Cc: Kees Cook Reviewed-by: Mark Rutland Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/assembler.h | 13 +++++++++++++ arch/arm64/mm/proc.S | 6 +----- 2 files changed, 14 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8760300..ab3a179 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -420,4 +420,17 @@ alternative_endif movk \reg, :abs_g0_nc:\val .endm +/* + * Errata workaround post TTBR0_EL1 update. + */ + .macro post_ttbr0_update_workaround +#ifdef CONFIG_CAVIUM_ERRATUM_27456 +alternative_if ARM64_WORKAROUND_CAVIUM_27456 + ic iallu + dsb nsh + isb +alternative_else_nop_endif +#endif + .endm + #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 352c73b..c2adb0c 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -136,11 +136,7 @@ ENTRY(cpu_do_switch_mm) bfi x0, x1, #48, #16 // set the ASID msr ttbr0_el1, x0 // set TTBR0 isb -alternative_if ARM64_WORKAROUND_CAVIUM_27456 - ic iallu - dsb nsh - isb -alternative_else_nop_endif + post_ttbr0_update_workaround ret ENDPROC(cpu_do_switch_mm) From patchwork Mon Feb 26 08:19:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129574 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356985lja; Mon, 26 Feb 2018 00:23:41 -0800 (PST) X-Google-Smtp-Source: AH8x2274YtU2Hm1Z/7+g0bY6rxaTaIgLGzcwey4dRUCmcmnz2I04t7ozO9IqYq0CHCchHpssPulq X-Received: by 2002:a17:902:243:: with SMTP id 61-v6mr9829617plc.202.1519633421681; Mon, 26 Feb 2018 00:23:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633421; cv=none; d=google.com; s=arc-20160816; b=DsD+xta5UW3hWBFi1dZJHYm6p+1KEx4fRi0iSXxPg88csQPQANFjZlrSk6ICPj8/7I 2LV4yJKcOIf2jRWwkVElUs+dAJDVsYYcVLivpy2I0LxgOHwtmfxsri+DiGd4jFIBmxnH Kju230GHsljepUZCBt8n6lwX2N1H7fbRiKxxjUilQwZPOE48l/TlHTdI6HcxkVOEV4kD flgQ7zBq3Dwm6Xr0SRzkJS51K/dbW6kV7ZUbsV1gXbKCevnIgEhcRXM33H6DCieWpMes EpnqOgglaG75AJFpWVMZcQsAdP3DY7XlNe5CpUU884s4cxkwOZeMFG2X+MBI3UiUZ2xf 6Dkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=zPqSNl85WyC76XW1VxDfQh//hh1Bpcs4lU5Fv1YTRJ0=; b=l3+qq4Co4fEbGKo+55+ONBt/ZQ8+t46j6KlU4YRAE6yqv5rm9h6nKCJTQ934fxWAa4 opBk7zdKEWP4p3mUnLF2+fdvpRylmOjrp/LBC1HCLnilTLvfGU+Ko05RFfVD9MxlOkmH DsxuMR9fjf0VFWPIrmWR5lHv8PlC018gBDrGp0WRZz6FqUdBavt0kNNIG4UeRhA/gwQ3 vbt/mwasGZcAwildCg+v6jd+8uVCvtNnfwjG5ezBtUjr0DH4FzATf/vnUzOuZpZa5v6d UQT0G28NMQdlUAI1BkgCH5Z4LZFeReVXX/j6Ku45brgJDKjiEqQNbKcxV/UULPuOTbSw 3PYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KDWmoSqW; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m12-v6si3738150pll.351.2018.02.26.00.23.41; Mon, 26 Feb 2018 00:23:41 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KDWmoSqW; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751888AbeBZIXj (ORCPT + 10 others); Mon, 26 Feb 2018 03:23:39 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:34135 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751746AbeBZIXe (ORCPT ); Mon, 26 Feb 2018 03:23:34 -0500 Received: by mail-pg0-f68.google.com with SMTP id m19so5917893pgn.1 for ; Mon, 26 Feb 2018 00:23:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=zPqSNl85WyC76XW1VxDfQh//hh1Bpcs4lU5Fv1YTRJ0=; b=KDWmoSqWRpgPXOtDS87zbO1parC+aWPWEmMrTJNlXcMiRczSZo/OOvbh1VPNpMuwxg 7v2ODO9ISyamLeGUekYilApPW2HQB89TMbVyFN0Jq0h2vYG6Gj2q3Wpb1T9qmZmj8La2 Ed0iqoaMU0PcjP85X3dG6y0dVASeYBMXgv1vs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zPqSNl85WyC76XW1VxDfQh//hh1Bpcs4lU5Fv1YTRJ0=; b=mjwssidKe6P+r0W5p3ty3g/eKPGKA2YuFn++4xtbUn/Ltqz7qw1216WZgLb9tNM+AB NyC6RWE+Gamp1yX6d+afvP0j/KNnt05pS4eQdM1x+VFli1EuPfsy3X/1amhO8rt2/bQa NstSCMxuk+YPCEa5zNc47ZO0BkjQo1Ev1KB3KkaNh49VMyL/3WdrKKLjmxGGcNqnlaGJ eEcp4KB+feTt4FBJzGH39731XsOqQOfgfmL2x8O1aoFZKSF/j3uF2bFYIYpbycrH3WyJ OSd0pdfxhyNzlsl+y/pcvZ+4WnPS8404D+EbAkQfAOidi3NF72UkXa+vADexYr5gHzNd qvmA== X-Gm-Message-State: APf1xPCjkboMbHgWBHcNQZVZv/jKNuVz+drYEYRAaERQeX60q70sW1ZA SUn0YaGbEgM0SLvgufXql67vvQ== X-Received: by 10.98.25.10 with SMTP id 10mr9661936pfz.136.1519633414159; Mon, 26 Feb 2018 00:23:34 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.23.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:33 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: Suzuki K Poulose , Mark Rutland , Andre Przywara , Dave Martin Subject: [PATCH 20/52] arm64: Run enable method for errata work arounds on late CPUs Date: Mon, 26 Feb 2018 16:19:54 +0800 Message-Id: <1519633227-29832-21-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Suzuki K Poulose commit 55b35d070c25 upstream. When a CPU is brought up after we have finalised the system wide capabilities (i.e, features and errata), we make sure the new CPU doesn't need a new errata work around which has not been detected already. However we don't run enable() method on the new CPU for the errata work arounds already detected. This could cause the new CPU running without potential work arounds. It is upto the "enable()" method to decide if this CPU should do something about the errata. Fixes: commit 6a6efbb45b7d95c84 ("arm64: Verify CPU errata work arounds on hotplugged CPU") Cc: Will Deacon Cc: Mark Rutland Cc: Andre Przywara Cc: Dave Martin Signed-off-by: Suzuki K Poulose Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kernel/cpu_errata.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index c66a673c..8de43799 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -150,15 +150,18 @@ void verify_local_cpu_errata_workarounds(void) { const struct arm64_cpu_capabilities *caps = arm64_errata; - for (; caps->matches; caps++) - if (!cpus_have_cap(caps->capability) && - caps->matches(caps, SCOPE_LOCAL_CPU)) { + for (; caps->matches; caps++) { + if (cpus_have_cap(caps->capability)) { + if (caps->enable) + caps->enable((void *)caps); + } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) { pr_crit("CPU%d: Requires work around for %s, not detected" " at boot time\n", smp_processor_id(), caps->desc ? : "an erratum"); cpu_die_early(); } + } } void update_cpu_errata_workarounds(void) From patchwork Mon Feb 26 08:19:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129607 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3363622lja; Mon, 26 Feb 2018 00:33:02 -0800 (PST) X-Google-Smtp-Source: AH8x226g+1YHrkZZUcuEgDHV4/YY/N75i5yQYtcx3L7GVDTmtfCnBfkW9yXSRylq1diadA13uZUP X-Received: by 2002:a17:902:3001:: with SMTP id u1-v6mr8947583plb.254.1519633981976; Mon, 26 Feb 2018 00:33:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633981; cv=none; d=google.com; s=arc-20160816; b=OFdHY141bk5hnHAJHtJ9kkF0krlToG1prKLccA+1CQgT22PT6YQcyv8g8LWZ30+yqn 0SC7OkJGAhFgKTF+RutvYNngFG7iAxSjzfZ+R+LhHNuypUCQ/I9ws84oG7w9UpImlWe8 fLDFttDq7DNBumu6y59Qd8as0mnBHZkqVPk02KkMROkr21qu3x/gw0Izyov8wJaMgJcN XP93FXCbad6/r36KQgWmpuM0qjcSpDcmGhVSx5/oajKo5L/TiAc9n4AtQZZXVbuGwj9V +doZWvDHy/nbMZ1yq9ufLaUEzNfpj3CZCSTwbtuaBWJ6h69UJFspczunA4kVbVpmms4W QeHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=+xR+r110x92nrh5KbxC/qh/uHxgkGEZ0it5ORoIx2gk=; b=Vea75bNaZyC7ryj5mZfFB15JMAHgePbZJnbvsm4JgcfXbNmHryjscswsavJQORWCt4 QHFIS8A5y5MQx/BjN+MZb6rfXScYWD0IRWmgcqhJmG2V4OJcruE4zuCumFgckYy2VJ8R JboJodQfDlUbo1qYhnkdDmUZFPovq3Oy3A16ryl6l5Io8MsuhtuamJNs2xIyedBysboM C0Brbt8a+7u0V/D9Y2z/hl8stJxXqcUCVpgwqzrTkvFuSbyleuoclrsyICRFTRRGpOWK 15Y3VfnfiPYIbp1QAuqWpTMjW5vK3IljjV4o/hg4/tLNzsanDego4NQd/AQbmm6MLprS cLgw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gUwGegbt; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c14si980807pgu.341.2018.02.26.00.33.01; Mon, 26 Feb 2018 00:33:01 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gUwGegbt; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752436AbeBZIdA (ORCPT + 10 others); Mon, 26 Feb 2018 03:33:00 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:37123 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752541AbeBZIXk (ORCPT ); Mon, 26 Feb 2018 03:23:40 -0500 Received: by mail-pl0-f68.google.com with SMTP id ay8so8854359plb.4 for ; Mon, 26 Feb 2018 00:23:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=+xR+r110x92nrh5KbxC/qh/uHxgkGEZ0it5ORoIx2gk=; b=gUwGegbtxMM+J+Et1e7BilrCA4XcNO36qS4/uUgBZWPNdQefunecApbEEIR45RIPbv MFQa4xXTC2L2Oc2fnOXsByXb0GdTG/v3osZd4DZ9i3UxSsSGa8NKYx9Z/Plxd5wuV8xz Doc7OTJwb5fF+Tkz0OGAbtOli2VYcAuRfrut8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=+xR+r110x92nrh5KbxC/qh/uHxgkGEZ0it5ORoIx2gk=; b=WIOfNH2ggNbXVmwng7WvvZuvz+iy4j0dxxrb8XP4b44uGZrubi91CfHd9MVidVRZ7z m02ptIkPcYgXRhNrIHcR0/S80lDS1tkNL1OlZjbq3DRM7eLL8pGrQR4mBX4oZN5BmZ+S AMk7a313FejmJMIfwBCSTPeHbi7a5Uz9m86KV+5P+G2zI5VF2amFZV/0W9k3q1eJVKZT 40WD8QI03c8GKuPHpg43JPYfAKxjLpwbNbZOQoqAwLS82sTT3WuvIPsYxoSIFIvVMyrC Urh0NVAULgq+EmHAVTw/AFDJzMexCatlz0qBN96PI2DvedVSo7X4lvw9HUbpcFP/4nWU WVYw== X-Gm-Message-State: APf1xPCuirgaNTI70X9ysYns5vVvUxW2lmA4up/CZ20r4/B/sG1feDlQ jDe69XFvBPV8d+3HRlkEKGFMhg== X-Received: by 2002:a17:902:8341:: with SMTP id z1-v6mr10050718pln.386.1519633419920; Mon, 26 Feb 2018 00:23:39 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.23.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:39 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 21/52] arm64: cpufeature: Pass capability structure to ->enable callback Date: Mon, 26 Feb 2018 16:19:55 +0800 Message-Id: <1519633227-29832-22-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Will Deacon commit 0a0d111d40fd upstream. In order to invoke the CPU capability ->matches callback from the ->enable callback for applying local-CPU workarounds, we need a handle on the capability structure. This patch passes a pointer to the capability structure to the ->enable callback. Reviewed-by: Suzuki K Poulose Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: arch/arm64/kernel/cpufeature.c --- arch/arm64/kernel/cpufeature.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index fcb67e0..5c41ef6 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -949,7 +949,7 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps) * uses an IPI, giving us a PSTATE that disappears when * we return. */ - stop_machine(caps->enable, NULL, cpu_online_mask); + stop_machine(caps->enable, (void *)caps, cpu_online_mask); } /* @@ -1005,7 +1005,7 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps) cpu_die_early(); } if (caps->enable) - caps->enable(NULL); + caps->enable((void *)caps); } } From patchwork Mon Feb 26 08:19:59 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129578 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357367lja; Mon, 26 Feb 2018 00:24:16 -0800 (PST) X-Google-Smtp-Source: AH8x226D0nmJEFb95bhLnSIRDVpq8DKRIdEZ44Lzc0eJWci0fbWdvlKXgRvaCEb86+q0rwRZZLXe X-Received: by 2002:a17:902:3103:: with SMTP id w3-v6mr9423133plb.99.1519633455876; Mon, 26 Feb 2018 00:24:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633455; cv=none; d=google.com; s=arc-20160816; b=wgHMC3z0/mfcrD8K4eHgI7eu95oalUj3GO6ebBzVyMiID0J0NMHUL7Ylkh7Rfhswex i0H/kFPxyZ/Z5T6YIuGn1unq74W9Y5nffpVAtdvCBdUtu5RZxNZp+oq78nUwfjGKqoGL t9n+X/yZoWg3zAjZE3zjyvfhZHAnA1h2vf/ATz06quZLZIdlrvP/ZdZmUQDab6SoINSN km7EM3rx5dWwco8jkJFde2OR2OIybT69FT3nKHGHJA0zDTc250G4NevnLR7wxZSd78aC fAyls230hUy+dv/YwGI0k+LFK9srBpuJLtVi0S9AlhS+DHz1M66hKjqhXGdijA0E2aI9 m5GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=XF3NgDLv8+ueVB8Xvd1HYusPmZ7vqfE4Nzmt2Dh5Rd0=; b=UwlyFMwIvSqPw/zrtAixrKDH/Bz662t/RbrD21du2CPLozsJBOG6OG5oJQu+Jduv0+ kKQUWc1NBThhlD6rjFFoftc1yuXhz51zUPANclSdAfwvYJAi172Tpbn25oPWMCaNUdX5 mTrI4S8v/lwjkreYIir6PY9NuMIAaUS6tXFEBtpcB5dDHZN3AjLguQLqq9qSSap+0o0a LzVdCKfgp8t1bpW/c8y7Ilwp6qRtZFS/Az1VV+OeqHI5DWlJi9ddglf2qHK7rBeQ5EKh ozyhoK42zxtDCShA4LDJ1rb7gBJ9aBW+hQF31neUL3nteaP7WA3lGShAkQWlGku/0ZOj u55Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=inZdjk12; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b11si5249076pgr.612.2018.02.26.00.24.15; Mon, 26 Feb 2018 00:24:15 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=inZdjk12; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751891AbeBZIYN (ORCPT + 10 others); Mon, 26 Feb 2018 03:24:13 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:46862 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751883AbeBZIYK (ORCPT ); Mon, 26 Feb 2018 03:24:10 -0500 Received: by mail-pf0-f196.google.com with SMTP id z10so1882200pfh.13 for ; Mon, 26 Feb 2018 00:24:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=XF3NgDLv8+ueVB8Xvd1HYusPmZ7vqfE4Nzmt2Dh5Rd0=; b=inZdjk12xBzZXv9CV/YXKVmXhYgE7NuNrm930dN6daBFfls25rUUL0kpK+d0vBIDJN NfQr7ifN8BE0zYldx8EMDwliUCC/0MxZoLqxFPCMDnDNNmqj8+/mG/BJyFpDccggRbda Gqpt1e3rZog4/d+e8AomfQFXrrRW/RQv3E15U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=XF3NgDLv8+ueVB8Xvd1HYusPmZ7vqfE4Nzmt2Dh5Rd0=; b=TOfT9lDuOyUKp/xzVAnoaafL/sGXx3+Pvo5gXvj56AMjIJVEzkUnETP3WxhUbjIv5f zcWxCelAEi8VPGc5RJCe3xDavavptHRuzX4JGEmx2lfHDOW0N7A5KoqnVKj78ETYbtPT +DNpKgn1ZZTN9NcOvqxbhvEeKH8G1xnf5xNXpwoOUb392rdakmjow0PgPF6XRgEwcIjN Ggsh9Qsafd3oioR3SPjYe0n73ijBYOsz5wBCzKBTo+9b66molrEG83OFiK02Um0ory8Q H5XaBLmtxz5PnfopFna7aE5hmt4siV33VW67S6fVGKcV0PIWseCndVVIf29X4qjqZJ4d S/7w== X-Gm-Message-State: APf1xPCUAAaRAzffxxxGV/FE8yKa7c9gE6hZrvDrNg8NyciIxkNqyu27 7oa6GXiPjFUyCIbFGAiVgLnaxg== X-Received: by 10.98.152.205 with SMTP id d74mr9871501pfk.115.1519633449585; Mon, 26 Feb 2018 00:24:09 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.24.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:24:09 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Russell King , linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 25/52] arm64: KVM: Use per-CPU vector when BP hardening is enabled Date: Mon, 26 Feb 2018 16:19:59 +0800 Message-Id: <1519633227-29832-26-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier commit 6840bdd73d07 upstream Now that we have per-CPU vectors, let's plug then in the KVM/arm64 code. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: mv changes from virt/kvm/arm/arm.c to arch/arm/kvm/arm.c --- arch/arm/include/asm/kvm_mmu.h | 10 ++++++++++ arch/arm/kvm/arm.c | 8 +++++++- arch/arm64/include/asm/kvm_mmu.h | 38 ++++++++++++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/switch.c | 2 +- 4 files changed, 56 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index a58bbaa..d10e362 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -223,6 +223,16 @@ static inline unsigned int kvm_get_vmid_bits(void) return 8; } +static inline void *kvm_get_hyp_vector(void) +{ + return kvm_ksym_ref(__kvm_hyp_vector); +} + +static inline int kvm_map_vectors(void) +{ + return 0; +} + #endif /* !__ASSEMBLY__ */ #endif /* __ARM_KVM_MMU_H__ */ diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index c38bfbe..4e4ae07 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -1088,7 +1088,7 @@ static void cpu_init_hyp_mode(void *dummy) pgd_ptr = kvm_mmu_get_httbr(); stack_page = __this_cpu_read(kvm_arm_hyp_stack_page); hyp_stack_ptr = stack_page + PAGE_SIZE; - vector_ptr = (unsigned long)kvm_ksym_ref(__kvm_hyp_vector); + vector_ptr = (unsigned long)kvm_get_hyp_vector(); __cpu_init_hyp_mode(pgd_ptr, hyp_stack_ptr, vector_ptr); __cpu_init_stage2(); @@ -1345,6 +1345,12 @@ static int init_hyp_mode(void) goto out_err; } + err = kvm_map_vectors(); + if (err) { + kvm_err("Cannot map vectors\n"); + goto out_err; + } + /* * Map the Hyp stack pages */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 6d22017..80bf337 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -313,5 +313,43 @@ static inline unsigned int kvm_get_vmid_bits(void) return (cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR1_VMIDBITS_SHIFT) == 2) ? 16 : 8; } +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +#include + +static inline void *kvm_get_hyp_vector(void) +{ + struct bp_hardening_data *data = arm64_get_bp_hardening_data(); + void *vect = kvm_ksym_ref(__kvm_hyp_vector); + + if (data->fn) { + vect = __bp_harden_hyp_vecs_start + + data->hyp_vectors_slot * SZ_2K; + + if (!cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN)) + vect = lm_alias(vect); + } + + return vect; +} + +static inline int kvm_map_vectors(void) +{ + return create_hyp_mappings(kvm_ksym_ref(__bp_harden_hyp_vecs_start), + kvm_ksym_ref(__bp_harden_hyp_vecs_end), + PAGE_HYP_EXEC); +} + +#else +static inline void *kvm_get_hyp_vector(void) +{ + return kvm_ksym_ref(__kvm_hyp_vector); +} + +static inline int kvm_map_vectors(void) +{ + return 0; +} +#endif + #endif /* __ASSEMBLY__ */ #endif /* __ARM64_KVM_MMU_H__ */ diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 0c848c1..cf6d962 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -50,7 +50,7 @@ static void __hyp_text __activate_traps_vhe(void) val &= ~CPACR_EL1_FPEN; write_sysreg(val, cpacr_el1); - write_sysreg(__kvm_hyp_vector, vbar_el1); + write_sysreg(kvm_get_hyp_vector(), vbar_el1); } static void __hyp_text __activate_traps_nvhe(void) From patchwork Mon Feb 26 08:20:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129579 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357490lja; Mon, 26 Feb 2018 00:24:26 -0800 (PST) X-Google-Smtp-Source: AH8x226EtX6dyj+DwfhjUeIbEApEwlPGoHoV68pmUBFSJD5EfPvEgby7R0j/nPG/NnCZqL8ID58j X-Received: by 10.98.204.69 with SMTP id a66mr9825847pfg.33.1519633466796; Mon, 26 Feb 2018 00:24:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633466; cv=none; d=google.com; s=arc-20160816; b=IxIAfw/4MNSA8t7bPC3kpsXdHDRwWEEyo7Jf24uCz6zBrJt1mOL9Qpd2o40BMFz3IP 5L9c1LF6vP5+bEYh4gax80U3qUpIOG3SmzsFwdaqq9EWLBaGuWlgP9ni6AzX8qv/7HV8 RLplUftflCtlRHMxrj/PtKL07cMQ3ZFQmQKUYZEaFjZ/xnUnPvdsR+edftWOj/gQt0RX apGyYOs/RUPXz7rleRNohjibHC0Y7Kz9c+3Prg9HcOj5FU4SxfAYfYCxrkixVdY9btbk RLlXbG0YdtowWdWdlZRZg5wUm//K7A/EQK1de59W3xlz+zPhjycKiv8GL+ZsGdWub1IT Ozcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=aecMAEg4UDkugVp6jZTqZagnCNTuf8l+gwqIgnZ36z0=; b=pDXiY31dgjYaWnPwbbuN9tKjbwG6/AVbnEeZhYd0UeAK0ZCHITQPfmvkQbeI4u0DRy CkqFz0XwiD3NEn5WKurjlOalCq3GdHGSj2kQJarnq4XpHlrOYGTO49wyge6AuODGpO1Y G5sfOiXXaanrjdpsl5ig09C8erRSvezTsXj6MaJ+4XrEKmWcVwHuKxq35QAs/5fDa1aS JZRhI6sF3TcBK2EX38JjZfRlgGPtwe9o1ImPC77Xn7nqpzHLpQipBt8fDv4G3zG0KZC9 29vdP8/3QCe86n92DS4t61WQoH5QJ8Q/kNToH1dwTXm/0fZL4R6ajozsepq4IAm0efSz Cz6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NdY553qP; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b11si5249076pgr.612.2018.02.26.00.24.26; Mon, 26 Feb 2018 00:24:26 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NdY553qP; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752377AbeBZIYZ (ORCPT + 10 others); Mon, 26 Feb 2018 03:24:25 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:34164 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751900AbeBZIYQ (ORCPT ); Mon, 26 Feb 2018 03:24:16 -0500 Received: by mail-pg0-f67.google.com with SMTP id m19so5918379pgn.1 for ; Mon, 26 Feb 2018 00:24:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=aecMAEg4UDkugVp6jZTqZagnCNTuf8l+gwqIgnZ36z0=; b=NdY553qPJUY1vLu+Qho/mPsQGua5qj3L8fVD59qAjUipogrBfvW50yn9SHwpDBnm6z QoNvU4tjDhR21wnxkdQ2tvuNaxywJ/OT6cDGw/nBLp5lrjNG0Fu1hGGQz+LV7XKwuGUw eLMQ5QMZKwsuUrC/XMlGXIfIc6t62DqXLj8WY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=aecMAEg4UDkugVp6jZTqZagnCNTuf8l+gwqIgnZ36z0=; b=gV9zKvZ8s8WNG8TaXhLk9VdBjOWLrDUEiNi7Js6crVOXw5YWJMcN0QJiLHB5sBGkUT q7VAR+dDbhe5lQUOHeijU0mRVkWssxZVbf8sjEiXlp7tXMHcd39iAcsF16QlF+1jr5ZM albrEuMa6n4jpgG/zttpeo5XIHU7pA3tAm3CguPomXHS6BmZlwrbT8qNURQE5r5xa9TF pMUhVMw4eGwAcl0twIV4NNzliQpnZULPx7Jky1A5oMu1bGcdndcrCMoaPhiiJFpsaQxS 9lDWwDQGlH8HoSzX6U42WgP+sC84JDiQx/xT9G3YuD5xAbxN/Njjm6RtZ7D27F1cDfvw whCw== X-Gm-Message-State: APf1xPBy3gOeRGz94JFObvFRKSFJr+j4qDCStjLJScpiJfRIRtxWzMg0 OH5DzvjMF1ORA6RAqe+Iv3RfSA== X-Received: by 10.101.99.205 with SMTP id n13mr7929403pgv.345.1519633455649; Mon, 26 Feb 2018 00:24:15 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.24.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:24:15 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 26/52] arm64: entry: Apply BP hardening for high-priority synchronous exceptions Date: Mon, 26 Feb 2018 16:20:00 +0800 Message-Id: <1519633227-29832-27-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Will Deacon commit 5dfc6ed27710 upstream. Software-step and PC alignment fault exceptions have higher priority than instruction abort exceptions, so apply the BP hardening hooks there too if the user PC appears to reside in kernel space. Reported-by: Dan Hettena Reviewed-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: expand enable_da_f to 'msr daifclr, #(8 | 4 | 1)' in arch/arm64/kernel/entry.S --- arch/arm64/kernel/entry.S | 6 ++++-- arch/arm64/mm/fault.c | 9 +++++++++ 2 files changed, 13 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index bdb0139..d50c2fe 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -584,8 +584,10 @@ el0_sp_pc: * Stack or PC alignment exception handling */ mrs x26, far_el1 - // enable interrupts before calling the main handler - enable_dbg_and_irq + msr daifclr, #(8 | 4 | 1) +#ifdef CONFIG_TRACE_IRQFLAGS + bl trace_hardirqs_off +#endif ct_user_exit mov x0, x26 mov x1, x25 diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index c95b194..6120a14 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -617,6 +617,12 @@ asmlinkage void __exception do_sp_pc_abort(unsigned long addr, struct siginfo info; struct task_struct *tsk = current; + if (user_mode(regs)) { + if (instruction_pointer(regs) > TASK_SIZE) + arm64_apply_bp_hardening(); + local_irq_enable(); + } + if (show_unhandled_signals && unhandled_signal(tsk, SIGBUS)) pr_info_ratelimited("%s[%d]: %s exception: pc=%p sp=%p\n", tsk->comm, task_pid_nr(tsk), @@ -676,6 +682,9 @@ asmlinkage int __exception do_debug_exception(unsigned long addr, if (interrupts_enabled(regs)) trace_hardirqs_off(); + if (user_mode(regs) && instruction_pointer(regs) > TASK_SIZE) + arm64_apply_bp_hardening(); + if (!inf->fn(addr, esr, regs)) { rv = 1; } else { From patchwork Mon Feb 26 08:20:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129580 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357513lja; Mon, 26 Feb 2018 00:24:29 -0800 (PST) X-Google-Smtp-Source: AG47ELtp+2HeE2tZ2RAZDejy8YrUAteAU0wqT0PcMruZZwuiPiM5+iAfVK/NGzfQs+KdvLsQ/GEf X-Received: by 2002:a17:902:2ec1:: with SMTP id r59-v6mr6847637plb.416.1519633468951; Mon, 26 Feb 2018 00:24:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633468; cv=none; d=google.com; s=arc-20160816; b=ZP42TWw2o0EpLCA+DIxCvKASd/9anDjInHxloL7L3viDUSEWyHR48ePQONj278svPL fV3ioEa/kRKSBf43OaKOxaLnX+OPePlGz0T2Qx9Duchl9WjNyR14IsqxDo9xKpB7VqhL k34hny4Nc/+2Otxf0UUpo2ZtvihM5meW8QCdntJaoh3SvMDuYG9eCSwav4DsqsuXR8Fn J5lyeJRCI9yQNe8sdZBBzeU+Th0R7Mi2DNk3y+mBxy4CxZ0dK8Sl3r5O8xm25Bqi6zmN tCNgOux2SJV6U8GnrDS93uhOR6FEYjD0/8/cUF2tLNdYJ1GdTakdMTOPgkhjvwYRF55j hlKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=o9wt4qxRgviZ/SM2aAeUByaDVo70FE/AaFiMiokIiaM=; b=EOuDxLZhZV48S+smS3bcllYAFFDkkk0gRt/wDSzhd7A5Il0i3qtT5ooWjSBlYBtM+D zvMhZ5cNRaRYHlUA8Sa4fiX16g0Yp9wjXW+aZE1CMXDYrI2EjRY0UQP9JgCKeRvkaUlL 9ukJOSoUS6l1jYawoTuzYv4PcMu7WeZQeViQBBi0Wh6dxhADm68C3GLtEnggKOzX+7un UMQ1q94zf1K85/uxlW6FDeRjwDh71fkB5tFgIKCTgoqgInBcuis8z5xdczncZ1zSybbI KoLK1RZtMKonXGpFxnORDzalBhpKUw9Ocpgkh/BORbtvYdMbQnSXwpcHNRL7PT3oKG3Z ukRg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VvoK62Rd; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b11si5249076pgr.612.2018.02.26.00.24.28; Mon, 26 Feb 2018 00:24:28 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VvoK62Rd; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752588AbeBZIY1 (ORCPT + 10 others); Mon, 26 Feb 2018 03:24:27 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:46870 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752403AbeBZIYV (ORCPT ); Mon, 26 Feb 2018 03:24:21 -0500 Received: by mail-pf0-f193.google.com with SMTP id z10so1882356pfh.13 for ; Mon, 26 Feb 2018 00:24:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=o9wt4qxRgviZ/SM2aAeUByaDVo70FE/AaFiMiokIiaM=; b=VvoK62RdimsjFNioD0qO4HEfqCyUJIJ9M2jhDIjHESfeUVHOkEjXfgJMAXJZd5ck9+ qTgilUPggg9D7JIGYSvMKzvcPruQbr3MJXtGojoj4lpAukymCQJ8jXiiXihqyxYx0uwY DeSxtOjkxcnR5y9oPPdVdUuXhWC/y33qFfjWY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=o9wt4qxRgviZ/SM2aAeUByaDVo70FE/AaFiMiokIiaM=; b=b3IUJycbs4SDnTwfZ9hTeCS8nLfkKJ57w0oD2NkcssIs6xj0O7HIuTB39C/N7TjZTA RLm3pyUPALgxfjpR5wm6cLfwkwBbHL4nvWLqScjqzp1UpHyj+pGp4SQEl+wlojN/XrHn 4A0TcwRuUz1ZapJcAdb+Jc+ZC9xqyG3hY7Mgd/zwxdJuBOANs/jEknEWu4nt3TW2MkVM dkiAx6/SInjJePcn9RM6ZLLoDJDBJW0m1zXyLTliSnteEdHq0yJlKJmR35IiZx9F6tm4 McXZiE0TVH6Hvq1r+RO7ibRbRZs2MGaE4OfunN1Zf4wQ8GKqm2SwyyQLJlWPD9Ozq+p3 271A== X-Gm-Message-State: APf1xPCkpz6vwcSmYW3xzH1UANQd3EzboWDuXyLPUR1PkhEZnb6JCBuU 1/HfgSJKsk6lrvTMkv3oRrzxGA== X-Received: by 10.99.122.86 with SMTP id j22mr7824959pgn.351.1519633461042; Mon, 26 Feb 2018 00:24:21 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.24.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:24:20 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 27/52] arm64: entry: Apply BP hardening for suspicious interrupts from EL0 Date: Mon, 26 Feb 2018 16:20:01 +0800 Message-Id: <1519633227-29832-28-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Will Deacon commit 30d88c0e3ace upstream. It is possible to take an IRQ from EL0 following a branch to a kernel address in such a way that the IRQ is prioritised over the instruction abort. Whilst an attacker would need to get the stars to align here, it might be sufficient with enough calibration so perform BP hardening in the rare case that we see a kernel address in the ELR when handling an IRQ from EL0. Reported-by: Dan Hettena Reviewed-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/kernel/entry.S | 5 +++++ arch/arm64/mm/fault.c | 6 ++++++ 2 files changed, 11 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index d50c2fe..e26a114 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -646,6 +646,11 @@ el0_irq_naked: #endif ct_user_exit +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + tbz x22, #55, 1f + bl do_el0_irq_bp_hardening +1: +#endif irq_handler #ifdef CONFIG_TRACE_IRQFLAGS diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 6120a14..ad49ae8 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -590,6 +590,12 @@ asmlinkage void __exception do_mem_abort(unsigned long addr, unsigned int esr, arm64_notify_die("", regs, &info, esr); } +asmlinkage void __exception do_el0_irq_bp_hardening(void) +{ + /* PC has already been checked in entry.S */ + arm64_apply_bp_hardening(); +} + asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr, unsigned int esr, struct pt_regs *regs) From patchwork Mon Feb 26 08:20:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129581 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357539lja; Mon, 26 Feb 2018 00:24:32 -0800 (PST) X-Google-Smtp-Source: AH8x227r0sjGv2YsPknBc9iNgiWs1SU2A+f2slpiqphKnAHmlUf5uTk+XX41ivvsUyInD64tAxLP X-Received: by 10.98.200.131 with SMTP id i3mr9886392pfk.40.1519633472418; Mon, 26 Feb 2018 00:24:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633472; cv=none; d=google.com; s=arc-20160816; b=Dqm0GQZxxg0atDI/inA0KefnW5r18K263gvyZhku2MP/Ff2FjEnYw6c9Kn5cOJljB7 qb4eKw6s5CQr5e1KzAeMOdwkpMo4wmutqrHYryeNsF9gbwdzNhlE1/p5mGulO49i6SVn ixnAj5eM4s2EHJvgrKlFqyqct9jWzUJ6ylaLY4tvE6vxyIA9YGic9SQgQ8NpSgixYjB2 rHW9Mt9fp0mye7t1j6QHpypI76jfla5xeKgvQB7d0d+xEeBXFTbejf230xcJyCN6Z14a JthUFTTfcACeFVxCGkfbcz7uVobAvF0ZN7SupMAGzUsi1n22k4XIB5yEesB9qf/XNU2+ smKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=AHr70MZkEOaKFey+eCDJheezhRWSsRAnZ4+h4S16qZs=; b=Gmb2m4Uqzjbga87bmpi4UZdpHL4rH4WTC6Q4GGnqLCdQnGOIWsMqE6H3RBYPUk3dcq sV2ybe8AZDeYmQLk9WyM8LdWudkIohgeukgIW8iS40mrj+Kk0j/yJdJzMI7aQRGylU1Y rzDYRuX2RNqWZAGheldzM5S4zS/cCc1BM180EhjaFmcLH9v+bcv6UZhQbQA9Kzg8py/C l2StcrssGtFu7pYzj71O/I65lxgJSr+ReYxVuxglF/VuCOW1CkrzvFAns7poRkFozG79 KjkZFec2tZSY7ftO8pSPYR8qUBSNFJIe77g+yMh/3q3pBx/8pd7ZnirFGiGF5onVwWFz ZMSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZM7Kt5Bd; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o74si6465353pfj.75.2018.02.26.00.24.32; Mon, 26 Feb 2018 00:24:32 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZM7Kt5Bd; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752205AbeBZIYa (ORCPT + 10 others); Mon, 26 Feb 2018 03:24:30 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:33334 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752240AbeBZIY2 (ORCPT ); Mon, 26 Feb 2018 03:24:28 -0500 Received: by mail-pf0-f194.google.com with SMTP id q13so6223863pff.0 for ; Mon, 26 Feb 2018 00:24:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=AHr70MZkEOaKFey+eCDJheezhRWSsRAnZ4+h4S16qZs=; b=ZM7Kt5Bdvj8NELsICBVQ7nDKwMYi9ZlNKaAv3EikznM6+/ePrsOr/ROVS4++oqb/Xy wY+HNXQr0tbktWrC/H+mg79kvyW8KwA3J4MBFmnt00xA53ZKpraeQFlm6zTSeiFsWTPX O/uvJt/QWPc5RsqdAqavPX/SWAPPM4Edzi4QI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=AHr70MZkEOaKFey+eCDJheezhRWSsRAnZ4+h4S16qZs=; b=ietcv2gcsCQXtEMKGzNzd/S/ILjt22sE9wAzzPun707uyBXC1L993Qi17wnUt9B7eT kKi+DYOz5/rHgmBMy2r+Xvuo4GVIignie4c8/WRQHtAbveeNPAMjwbRRCaHw31yWdjwR dDg6rB73c/r7VS4fuTZ6qjicdcGRBayImM5UrLC5l01EGYS5DKr+j28bSW5ZpUeo5fiM jssi38kK7ZuOb8/7vW0+StLioGEQldnWKhFbZAXmET8LCumJp9MmVc+0ScQpdjmkiRa/ +8/Scy0SRWK/tE54FPbGpOziob8xPEK1Ta7dAi07Sz3RgT1oX/TaowWcpUtCOrSCu251 DPXg== X-Gm-Message-State: APf1xPDFOeJY5+meqDHsKyC5uYGJpay/peiAMMy44lR3ww1h39lNajXp oCMgQihikbilM7F/wmDiLNjnzmdK65o= X-Received: by 10.98.217.76 with SMTP id s73mr9761914pfg.209.1519633467434; Mon, 26 Feb 2018 00:24:27 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.24.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:24:26 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 28/52] arm64: cputype: Add missing MIDR values for Cortex-A72 and Cortex-A75 Date: Mon, 26 Feb 2018 16:20:02 +0800 Message-Id: <1519633227-29832-29-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Will Deacon commit a65d219fe5dc upstream. Hook up MIDR values for the Cortex-A72 and Cortex-A75 CPUs, since they will soon need MIDR matches for hardening the branch predictor. Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: pick A73 too in arch/arm64/include/asm/cputype.h --- arch/arm64/include/asm/cputype.h | 6 ++++++ 1 file changed, 6 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index 26a68dd..0843b3f 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -75,7 +75,10 @@ #define ARM_CPU_PART_AEM_V8 0xD0F #define ARM_CPU_PART_FOUNDATION 0xD00 #define ARM_CPU_PART_CORTEX_A57 0xD07 +#define ARM_CPU_PART_CORTEX_A72 0xD08 #define ARM_CPU_PART_CORTEX_A53 0xD03 +#define ARM_CPU_PART_CORTEX_A73 0xD09 +#define ARM_CPU_PART_CORTEX_A75 0xD0A #define APM_CPU_PART_POTENZA 0x000 @@ -86,6 +89,9 @@ #define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53) #define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57) +#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72) +#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73) +#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75) #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX) #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX) From patchwork Mon Feb 26 08:20:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129582 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357724lja; Mon, 26 Feb 2018 00:24:51 -0800 (PST) X-Google-Smtp-Source: AH8x227A011GKtUFwpIRDL4E95W8LqF+oxAx0QjBE1s2dYl+hiv94DGpQbxPGC6dIAzB4Y9PCNuM X-Received: by 10.99.120.201 with SMTP id t192mr7811123pgc.39.1519633491326; Mon, 26 Feb 2018 00:24:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633491; cv=none; d=google.com; s=arc-20160816; b=vFz/qY9i4pvCL+fq1UVHpGco+7+IwvLWzX1vlYbOpVBfZRsbtKbLXt1ha1WvVrO9GU S1IsQpcGtJu8b9tTCGKXDr7Vv4BOOjoV90EC82QWsq0mcBNPIWebp8ZBKDHt9li248td cq+wkw9IbDswr2YWj5kxGEsw8h4d5E8ccOstiFDpa9EnCfk/aI/41Oxfu2tRMC9BoOfk rBVVlS42BSkLsSOZ4CkCZf9FBkzFvk6dOIKTOylkCybrMizyiPo2tmEmLDVYQHQzBHOg iePxngoUx5yZJW5HMDZv2rcg+hMRQ5zNvMnOL4NlpwDENnWzpKxzVyxM/vn/OvL/Mycv Ltyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=/nksGLZqHQjfpQbfH68HEgt65Nc6WZIfEk50r4t0UeE=; b=VxvZVjLLoiNCOJctFwihiTZdlaOV6uBf+0Ts3XkyYTPHnxN9nEK646yjC62K5Lp/FQ Z8/pAOKOYoFEXIBFCumLLCVaScpnJYj7vKn29qcnrg8RuTS2y3k7xm85YCowVValGYGV JjhIaqPCnttHWe+l854iLxOju3BRrPL/1LwPyNlVlG3sHnUhkQrCIpWNxKbcldqPS3Ii DksuPU+6bAilFexn8Jhlur3Dn+PsPMTxZfW1pSjEx8wHz3xOAm1PCN8ZqLTFSckj/lIO ggaMgyydxisixXt/+OqBK82RegDrR9EhH8PLX/PjqGxcehMmx1V9Hij2xyWuPhw4DFo0 inYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aJOtU3Bb; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 1-v6si6389538plp.678.2018.02.26.00.24.50; Mon, 26 Feb 2018 00:24:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=aJOtU3Bb; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752433AbeBZIYs (ORCPT + 10 others); Mon, 26 Feb 2018 03:24:48 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:39892 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751979AbeBZIYn (ORCPT ); Mon, 26 Feb 2018 03:24:43 -0500 Received: by mail-pg0-f67.google.com with SMTP id w17so5909567pgv.6 for ; Mon, 26 Feb 2018 00:24:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=/nksGLZqHQjfpQbfH68HEgt65Nc6WZIfEk50r4t0UeE=; b=aJOtU3BbNimU/2ivLtPtjHbi4kjfA18KzNfLloYw4Q3hTZ1P/nyrFFdoe2RaFJ0yNF MdjSu08eySO3EW1vYNOzrRndv6SeEddJWe2AJ7y1/1vb6nXbxJlesLdRWkyFmdB2lNBZ nCwEwHEQ99xu4KOJq6VQKGWXTk6FtFMwzXSS0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=/nksGLZqHQjfpQbfH68HEgt65Nc6WZIfEk50r4t0UeE=; b=l34Y42/PTB0ArvcFTibTAgONiWCvVLIGT092Cy+YaDw4URJu1Ry2xZzroqZBFwlpw4 UtHTAOoJf6PqF7LR0sKnOEB8wwMhsxIKuOKM3wWmwhObctsAl/qkb/S5h8eagrZqa9dk oaFA3k0aC8vpNTTWsZ5RwOKEksCQTuRYH8I3IP9Ix3tpJn00Rjx1clelDgra6HVoywIO dfwcRI8ocUhDf/LDJO9ctL35ZuSOQaCu1u4rGYXIEWalmdg+S3OtI8MHUu6IXGr+eu/u sh17nY4I8cmkk69B5oq8lJs9yNxuAgBQMf+mxSNm6SBb/0XbzNHvWjsaLmyiiYFnyWx/ GFmA== X-Gm-Message-State: APf1xPA0GIkkL96QAzl1PqBrSFz9uWD+mOvJ3wlIJ06pXz9l9YQTysC1 xqeQo7xus65qFMt94lVXnFbY7A== X-Received: by 10.99.145.194 with SMTP id l185mr7597844pge.394.1519633483193; Mon, 26 Feb 2018 00:24:43 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.24.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:24:42 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Christoffer Dall , kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 30/52] arm64: KVM: Increment PC after handling an SMC trap Date: Mon, 26 Feb 2018 16:20:04 +0800 Message-Id: <1519633227-29832-31-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier commit f5115e8869e1 upstream. When handling an SMC trap, the "preferred return address" is set to that of the SMC, and not the next PC (which is a departure from the behaviour of an SMC that isn't trapped). Increment PC in the handler, as the guest is otherwise forever stuck... Cc: stable@vger.kernel.org Fixes: acfb3b883f6d ("arm64: KVM: Fix SMCCC handling of unimplemented SMC/HVC calls") Reviewed-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kvm/handle_exit.c | 9 +++++++++ 1 file changed, 9 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 2e6e9e9..5b56b09 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -53,7 +53,16 @@ static int handle_hvc(struct kvm_vcpu *vcpu, struct kvm_run *run) static int handle_smc(struct kvm_vcpu *vcpu, struct kvm_run *run) { + /* + * "If an SMC instruction executed at Non-secure EL1 is + * trapped to EL2 because HCR_EL2.TSC is 1, the exception is a + * Trap exception, not a Secure Monitor Call exception [...]" + * + * We need to advance the PC after the trap, as it would + * otherwise return to the same address... + */ vcpu_set_reg(vcpu, 0, ~0UL); + kvm_skip_instr(vcpu, kvm_vcpu_trap_il_is32bit(vcpu)); return 1; } From patchwork Mon Feb 26 08:20:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129586 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3358027lja; Mon, 26 Feb 2018 00:25:21 -0800 (PST) X-Google-Smtp-Source: AH8x226wfNusMvKS8R9F9NCfBHHDpI/g92qE7lqM55yJ/1Uvh+6u1oOv7gOPyYrg2qalIP0XbY/n X-Received: by 10.98.137.147 with SMTP id n19mr9919604pfk.193.1519633520870; Mon, 26 Feb 2018 00:25:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633520; cv=none; d=google.com; s=arc-20160816; b=0RG//emZ9iShK8JFg5uSsNFGON1wGtnemSfdSAZka6TjJ0JNgUxOsBnoJ9Awn/qk/c nJQHCngRWQqbchvyCnzZhlujyCYXWnAO7B7ERCkrNI2o7hcG21Kzf6e3wXUdPmJmW3ig S4IKThhpc4aS/EXU3rO/utIbdguiquVZtBhO95/0og2MaeL1v8lWadxAjYFBz0D7xSWC ifg6QTk9nZQ85qCnVNEwMUb9w8grQOqyGW/AXhaEo5HiIEa1hwDvx1l9cR4ImEdgLDbb kXemFsVJIdHMCPX4oVAllclqMMMyU0+v6Mj1lhCQ0i8Dd2imLLM+Zo7WfR1jokHDRWOR 8F5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=joq1LHzXEIs9X9WqC4BlIrH4LbSvJY9OeFalK8l0bao=; b=l4gE12Cx+WzzR165F9Q91i+QJet+6VDPPWfaw6jeWI0Bm2UV/e7akUdAZEsSWJ/Wcp xqALDl1un/ftyjvF/FoyAb9WIHMJUx6JtWhhu7vUSqlxl2o96YgosNSVitasJWtMtiOZ crk6mcTlQTUyQS75fXjjLnDS2cpnH16u1XqFGt1bKdhfC4cAeH9T7GPdbz+4ZoAJ/L9V L82vsiqwBwxZXFXudGw7o3GOWS1T3ii1l2OZtifk5nZoutc0eNslu+RjFuZ64uMlq18L jcxw0VMmCNvi0rkxewMu/w8o+z1SaBNKsXe4pS/rlJtSF5q0w7gBz5Neq3H7fkCnFpoM f/QA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SNpo7E61; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o64si1254960pga.160.2018.02.26.00.25.20; Mon, 26 Feb 2018 00:25:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SNpo7E61; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752664AbeBZIZS (ORCPT + 10 others); Mon, 26 Feb 2018 03:25:18 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:44644 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752284AbeBZIZN (ORCPT ); Mon, 26 Feb 2018 03:25:13 -0500 Received: by mail-pg0-f68.google.com with SMTP id l4so5907206pgp.11 for ; Mon, 26 Feb 2018 00:25:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=joq1LHzXEIs9X9WqC4BlIrH4LbSvJY9OeFalK8l0bao=; b=SNpo7E61DVpWIdqOveOEXVYZ8EErZAZw5te1BCQUaq1y3eaARi0QQo+n/ewB6Jw45g E3HK7ag/EnBvUN4H/ezxDxKUwdpijE3MDGYCH1DapoR5A2tm9V+DCM2G32KpM+Qa1TBR 7BEzdPLgQk1bbk/rTjGa49HJBxdF9QpTDjr6Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=joq1LHzXEIs9X9WqC4BlIrH4LbSvJY9OeFalK8l0bao=; b=rCry9+MOehfZDoBvEse4BwCFnEOQZluedb08ULDu+SW8ES8b7ybus9cXWbVSAMVIo0 7+goDW4smd1w6kxZ9FxgoaOT1v6ncgBwGuT9NDxvBygf9AIa97gNz0JhB7GKWi9ejDsH ckYtDYGVDl2EXpzpgLmvh+lhJhI0CWqSaZ2h4sYZTAWo3B+UdFMqJQhQomAx7smyXc00 RRHjFIsz0fmJdv7527wGAus+xj7l27rEuU7SX5zcU5VWtA0/MBtKzGg8yVFv8tl6vWAh FpQdZoV0zLiTQgyTLxqfiR2OmU24z+Ldgp0d3RDAaOHj8kqV5I/a5jdkqCptKqvcb4u+ OgRQ== X-Gm-Message-State: APf1xPCuZq+E4PovewzkGz76+OoFeDxCTrOF6PeDXq67o/w4NPnHqQtL /ef2iCVniSj2LvSfumB1kO7F3oeoRFg= X-Received: by 10.99.103.133 with SMTP id b127mr7722713pgc.155.1519633513030; Mon, 26 Feb 2018 00:25:13 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.25.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:25:12 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Christoffer Dall , Russell King , kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 33/52] arm/arm64: KVM: Add smccc accessors to PSCI code Date: Mon, 26 Feb 2018 16:20:07 +0800 Message-Id: <1519633227-29832-34-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier commit 84684fecd7ea upstream. Instead of open coding the accesses to the various registers, let's add explicit SMCCC accessors. Reviewed-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: mv change from virt/kvm/arm/psci.c to arch/arm/kvm/psci.c --- arch/arm/kvm/psci.c | 52 ++++++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 42 insertions(+), 10 deletions(-) -- 2.7.4 diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 4adfa28..bc334d6 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -33,6 +33,38 @@ #define AFFINITY_MASK(level) ~((0x1UL << ((level) * MPIDR_LEVEL_BITS)) - 1) +static u32 smccc_get_function(struct kvm_vcpu *vcpu) +{ + return vcpu_get_reg(vcpu, 0); +} + +static unsigned long smccc_get_arg1(struct kvm_vcpu *vcpu) +{ + return vcpu_get_reg(vcpu, 1); +} + +static unsigned long smccc_get_arg2(struct kvm_vcpu *vcpu) +{ + return vcpu_get_reg(vcpu, 2); +} + +static unsigned long smccc_get_arg3(struct kvm_vcpu *vcpu) +{ + return vcpu_get_reg(vcpu, 3); +} + +static void smccc_set_retval(struct kvm_vcpu *vcpu, + unsigned long a0, + unsigned long a1, + unsigned long a2, + unsigned long a3) +{ + vcpu_set_reg(vcpu, 0, a0); + vcpu_set_reg(vcpu, 1, a1); + vcpu_set_reg(vcpu, 2, a2); + vcpu_set_reg(vcpu, 3, a3); +} + static unsigned long psci_affinity_mask(unsigned long affinity_level) { if (affinity_level <= 3) @@ -75,7 +107,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) unsigned long context_id; phys_addr_t target_pc; - cpu_id = vcpu_get_reg(source_vcpu, 1) & MPIDR_HWID_BITMASK; + cpu_id = smccc_get_arg1(source_vcpu) & MPIDR_HWID_BITMASK; if (vcpu_mode_is_32bit(source_vcpu)) cpu_id &= ~((u32) 0); @@ -94,8 +126,8 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) return PSCI_RET_INVALID_PARAMS; } - target_pc = vcpu_get_reg(source_vcpu, 2); - context_id = vcpu_get_reg(source_vcpu, 3); + target_pc = smccc_get_arg2(source_vcpu); + context_id = smccc_get_arg3(source_vcpu); kvm_reset_vcpu(vcpu); @@ -114,7 +146,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) * NOTE: We always update r0 (or x0) because for PSCI v0.1 * the general puspose registers are undefined upon CPU_ON. */ - vcpu_set_reg(vcpu, 0, context_id); + smccc_set_retval(vcpu, context_id, 0, 0, 0); vcpu->arch.power_off = false; smp_mb(); /* Make sure the above is visible */ @@ -134,8 +166,8 @@ static unsigned long kvm_psci_vcpu_affinity_info(struct kvm_vcpu *vcpu) struct kvm *kvm = vcpu->kvm; struct kvm_vcpu *tmp; - target_affinity = vcpu_get_reg(vcpu, 1); - lowest_affinity_level = vcpu_get_reg(vcpu, 2); + target_affinity = smccc_get_arg1(vcpu); + lowest_affinity_level = smccc_get_arg2(vcpu); /* Determine target affinity mask */ target_affinity_mask = psci_affinity_mask(lowest_affinity_level); @@ -209,7 +241,7 @@ int kvm_psci_version(struct kvm_vcpu *vcpu) static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) { struct kvm *kvm = vcpu->kvm; - unsigned long psci_fn = vcpu_get_reg(vcpu, 0) & ~((u32) 0); + u32 psci_fn = smccc_get_function(vcpu); unsigned long val; int ret = 1; @@ -276,14 +308,14 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) break; } - vcpu_set_reg(vcpu, 0, val); + smccc_set_retval(vcpu, val, 0, 0, 0); return ret; } static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) { struct kvm *kvm = vcpu->kvm; - unsigned long psci_fn = vcpu_get_reg(vcpu, 0) & ~((u32) 0); + u32 psci_fn = smccc_get_function(vcpu); unsigned long val; switch (psci_fn) { @@ -301,7 +333,7 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) break; } - vcpu_set_reg(vcpu, 0, val); + smccc_set_retval(vcpu, val, 0, 0, 0); return 1; } From patchwork Mon Feb 26 08:20:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129587 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3358175lja; Mon, 26 Feb 2018 00:25:32 -0800 (PST) X-Google-Smtp-Source: AH8x226dZoabfi3bPBz36HdSkrubJIcVag0P/EnKFzAzzCqpaFj/J0iYL9ysxYTGEkEDoq0cCHNx X-Received: by 10.98.106.10 with SMTP id f10mr9804192pfc.222.1519633531969; Mon, 26 Feb 2018 00:25:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633531; cv=none; d=google.com; s=arc-20160816; b=XkoD3Y/JUAHs5Ep+9vzU37TR4QC/P8w3k5RXPzCxgDktM9jiPbWNMBC7Ch03wUy4GH Q+o2/fKLKgRgXCZV/rTAfLpQ03B9SU+P3yQKqZ7PG5lGLAy4J6wMPZVwX1v/U8qfKMvC lOPJvxNxGzbbpuK0ZQH4mGuDAD8PRysWDop4K9/YO9eEAHKXUpPy1BnYVQJpYPLngxsm 386SVISuAfnlIBaSOB8m8Qs+/E8j64xpoqkGd+CB4b74jzgTS2BFyFO5kOEry3EiDpZa Dg7Rhf0CWdRjW8R97ulFKYsZPt8hoYkRzUqOZw03CMHCPRk1w7LsR2zH9JT+hB/NTH5F Ku7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=dkPnOZNxm9seQPfv5c9vz4+Yjf5n6fqYResagBKyW1s=; b=itbh4XiuDlK7jwG+Ci+SLoCtW+vM/fl/fuqKuYZmU9X5t9LwaOnHs4X8kRV5LO9yWz BLoQ7Ull4/JoKPpDY+ma7AFrONL08u//9kh53S9E5eTgaXDferw5HijFeqMwOQGbRlhf p/Ixl7oqlNzmSB1PwBnQeClHu15NkCeCaG0AzYCdMm5pV3XxYOwY4Rlz18scBOtt4mpQ uVo3pX/qJGqlaS0FoN4w/BlM2LRs2kFCIYYoAwrbbsZXnFHbf75HapMAg0vW8sZ3c23E /JhKoWlaVXybZulPq6sns62RQrWXF/1VrNW0zQg8qOKTz3iy81JCx5WD2Q/Zz/MGAj41 S5cw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Z4Ilq6+W; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r26si6430932pfi.378.2018.02.26.00.25.31; Mon, 26 Feb 2018 00:25:31 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Z4Ilq6+W; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752537AbeBZIZa (ORCPT + 10 others); Mon, 26 Feb 2018 03:25:30 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:33382 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752420AbeBZIZY (ORCPT ); Mon, 26 Feb 2018 03:25:24 -0500 Received: by mail-pf0-f196.google.com with SMTP id q13so6224557pff.0 for ; Mon, 26 Feb 2018 00:25:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=dkPnOZNxm9seQPfv5c9vz4+Yjf5n6fqYResagBKyW1s=; b=Z4Ilq6+Wx3MQYchI6CMGFV4WmySMREfNZejZ5nSzez8eBD9pw806Mmx4t436Qyu0Ma FhEbLlngN7+DGr24QJe/4YMPLywi0DZYCZZVOeNYAa/Ru8x1iNplG3ZcGAsBuQl6m/aU wCarkFKmL+/KSNmWgLTAhrIIXvXsGOUqNgO5w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=dkPnOZNxm9seQPfv5c9vz4+Yjf5n6fqYResagBKyW1s=; b=Yb/VR0gUM7+opV5U8csp+jb7siSXVeUiM0qKprlPixQgekweynHYDoleC5NmlB+6lq E61c5dEf8pUvlsUmrGeUSgSWj/ntAqPCl4qPxwRdhTWLzfCwPh2WHNM7xjQDF0W+hVs/ YB3ZfZOvIEwdwrCXyu1D/TYZ1EDq/JReatY564s/FTEn0yOQaFeiytaBCPn2H7WcG9so HCRkVYroqQKAHPKpF/JvqJ8FfP5NK5MFo9MFZrZpaE6GzS818f0nSiqP9dy67AXslQTW 7jeRaCwpr9nXMda/RTCNr0453DXJVcqOHY6NdKzvsOckctNMjdg5UbSL8jaEOcymHK/4 DpWQ== X-Gm-Message-State: APf1xPA3bobAik7AyADH3voJSfNeQhXj/CRwHez1K0f2Rk8dgy9QzAul 7Hw3FIlZ74ZXl7hY3htOSONazW8s2vU= X-Received: by 10.99.5.197 with SMTP id 188mr7799923pgf.196.1519633523556; Mon, 26 Feb 2018 00:25:23 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.25.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:25:23 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Christoffer Dall , Russell King , kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 34/52] arm/arm64: KVM: Implement PSCI 1.0 support Date: Mon, 26 Feb 2018 16:20:08 +0800 Message-Id: <1519633227-29832-35-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier commit 58e0b2239a4d upstream. PSCI 1.0 can be trivially implemented by providing the FEATURES call on top of PSCI 0.2 and returning 1.0 as the PSCI version. We happily ignore everything else, as they are either optional or are clarifications that do not require any additional change. PSCI 1.0 is now the default until we decide to add a userspace selection API. Reviewed-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: mv chagnes from virt/kvm/arm/psci.c to arch/arm/kvm/psci.c --- arch/arm/kvm/psci.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- include/kvm/arm_psci.h | 3 +++ 2 files changed, 47 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index bc334d6..097632c 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -233,7 +233,7 @@ static void kvm_psci_system_reset(struct kvm_vcpu *vcpu) int kvm_psci_version(struct kvm_vcpu *vcpu) { if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) - return KVM_ARM_PSCI_0_2; + return KVM_ARM_PSCI_LATEST; return KVM_ARM_PSCI_0_1; } @@ -312,6 +312,47 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) return ret; } +static int kvm_psci_1_0_call(struct kvm_vcpu *vcpu) +{ + u32 psci_fn = smccc_get_function(vcpu); + u32 feature; + unsigned long val; + int ret = 1; + + switch(psci_fn) { + case PSCI_0_2_FN_PSCI_VERSION: + val = KVM_ARM_PSCI_1_0; + break; + case PSCI_1_0_FN_PSCI_FEATURES: + feature = smccc_get_arg1(vcpu); + switch(feature) { + case PSCI_0_2_FN_PSCI_VERSION: + case PSCI_0_2_FN_CPU_SUSPEND: + case PSCI_0_2_FN64_CPU_SUSPEND: + case PSCI_0_2_FN_CPU_OFF: + case PSCI_0_2_FN_CPU_ON: + case PSCI_0_2_FN64_CPU_ON: + case PSCI_0_2_FN_AFFINITY_INFO: + case PSCI_0_2_FN64_AFFINITY_INFO: + case PSCI_0_2_FN_MIGRATE_INFO_TYPE: + case PSCI_0_2_FN_SYSTEM_OFF: + case PSCI_0_2_FN_SYSTEM_RESET: + case PSCI_1_0_FN_PSCI_FEATURES: + val = 0; + break; + default: + val = PSCI_RET_NOT_SUPPORTED; + break; + } + break; + default: + return kvm_psci_0_2_call(vcpu); + } + + smccc_set_retval(vcpu, val, 0, 0, 0); + return ret; +} + static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) { struct kvm *kvm = vcpu->kvm; @@ -354,6 +395,8 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) int kvm_psci_call(struct kvm_vcpu *vcpu) { switch (kvm_psci_version(vcpu)) { + case KVM_ARM_PSCI_1_0: + return kvm_psci_1_0_call(vcpu); case KVM_ARM_PSCI_0_2: return kvm_psci_0_2_call(vcpu); case KVM_ARM_PSCI_0_1: diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index 5659343..3236043 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -22,6 +22,9 @@ #define KVM_ARM_PSCI_0_1 PSCI_VERSION(0, 1) #define KVM_ARM_PSCI_0_2 PSCI_VERSION(0, 2) +#define KVM_ARM_PSCI_1_0 PSCI_VERSION(1, 0) + +#define KVM_ARM_PSCI_LATEST KVM_ARM_PSCI_1_0 int kvm_psci_version(struct kvm_vcpu *vcpu); int kvm_psci_call(struct kvm_vcpu *vcpu); From patchwork Mon Feb 26 08:20:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129588 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3358349lja; Mon, 26 Feb 2018 00:25:43 -0800 (PST) X-Google-Smtp-Source: AH8x225lQnscGMdxEkOd2eGyvL8cl0qK1E7i/ypIhsQhk364qSu08m0jPK5f4R0FLobnIw0f7Yjd X-Received: by 10.98.242.65 with SMTP id y1mr9603598pfl.232.1519633543192; Mon, 26 Feb 2018 00:25:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633543; cv=none; d=google.com; s=arc-20160816; b=kt1lGKSoJ4zZFEs01sPfpnVOAPrk/OUX9VmRasjYfp6w86H1eQAqdf1et6J+Jbrx7l YqTs2fEjo2vn17gBpumbSq+1brAuFmRPF58L+dKr6NWCuh2Ys2bQtb83/KysPtROV6CH upcKsEmOAiRdLBbEc0K/NEBAg7CsV1GKFkExrBpxcFA1KeTUk6QKJ/8gzGvxztmEKFaY 3yFOaar18VYW6H/RWd8QeXK67nc8hbahqmWs30jxZq1s85solRf7BSLsujTxGF84jOA2 3XNps8cYDG31AI+N4MD4+xaGv/B0bd8lDbZxdE2XgBk41XzKZtd7PPqyTiIcMX3YC4BT CzHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=d7AzDfwawwGLQswEyJ47jtnJY3zEo49brcg9XtRqJ2I=; b=H7tfl5kZ5mY5Kp8PNNwAbTxKGdVCixLkrRsHCaFV5nCh0TRb4KDEl4kgXUcBdqdKaZ wc9xE5MaCTCH1BTuMr182zN0Wg0dYu66HYN94H6jGUNtBa1P6xT9rI9iSfQ5BHDA0x4d HpG4TjhfAjn+Rbg+XTxhqJQVUBmlPb9e/jJTcUjNC9aax2OocXwZjQ9p1WAEa0IW8vIE BtXD687YsuDmKO44A8eaJZbVrs89vAL321rSUMXoT9WnNeUG6CSF340O2mwQO2jCjgtF WjnicMcyPCKT6D1jYBTCzKTlxtoEQMirpPc979ASWyVU7pUDJFp9AeuxRahYGaKXLDGV HBUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F9bW8+dZ; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 73-v6si3704201ple.829.2018.02.26.00.25.42; Mon, 26 Feb 2018 00:25:43 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F9bW8+dZ; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752034AbeBZIZk (ORCPT + 10 others); Mon, 26 Feb 2018 03:25:40 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:46681 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752420AbeBZIZe (ORCPT ); Mon, 26 Feb 2018 03:25:34 -0500 Received: by mail-pg0-f68.google.com with SMTP id r26so678095pgv.13 for ; Mon, 26 Feb 2018 00:25:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=d7AzDfwawwGLQswEyJ47jtnJY3zEo49brcg9XtRqJ2I=; b=F9bW8+dZecvQMzbS8ejkFcwARWuvthBO3T9ZVb3U9hfhI0DCluBnRTP4W551EuhtPF y92fNzqCp2XcjpKm8YMsJQT+7e/gSXhD5rY+h0dspPhxWgTCCfYC2Q+vJAuwgnGN3Pcz Kg2Yhdpl4DDxphCk1jzyLY+poRQ7wROCQuMio= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=d7AzDfwawwGLQswEyJ47jtnJY3zEo49brcg9XtRqJ2I=; b=tpt9hNwe07hS9iSafWe7sixVY0yW/hTVld0klBOQ+t5zkr4aqze5jixstL2veX4tdr 245T4LpT8+3zzlINnsmRlHE4kbi/2Or+HtfJuM1ZaskjVMs3QbpVXjae5JQxjg6KOQJB H+n2Y/HRlNbo7MqV/bSxGH013HlT6NXrc25qc7L5mRzVt0Mfr0+XgAeqoQ1ZmhaHDyTh MGnw+EbyJhAoJ0dSkFDmmfLyLHV7czsf4K/3JEPzM3TDL8XbfnWRAj9ArHOYtRJWZw6u FYH5yGvqmzaNdgCEgBfOxBBKeav5c6DcQQroZTn+VgfUn+uZBBOTM8BRp1C1xEp/8yYK PkIw== X-Gm-Message-State: APf1xPD83Lysmu89diJnQhs5rblyMKEzAbqPdji4LMIAY0m/HoVUTBfR /uuqx9b+QtVki90vZHAXw6FFOg== X-Received: by 10.101.97.72 with SMTP id o8mr7934493pgv.119.1519633533597; Mon, 26 Feb 2018 00:25:33 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.25.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:25:33 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Russell King , linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 35/52] arm/arm64: KVM: Advertise SMCCC v1.1 Date: Mon, 26 Feb 2018 16:20:09 +0800 Message-Id: <1519633227-29832-36-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier commit 09e6be12effd upstream. The new SMC Calling Convention (v1.1) allows for a reduced overhead when calling into the firmware, and provides a new feature discovery mechanism. Make it visible to KVM guests. Tested-by: Ard Biesheuvel Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: mv change from virt/kvm/arm/psci.c to arch/arm/kvm/psci.c --- arch/arm/kvm/handle_exit.c | 2 +- arch/arm/kvm/psci.c | 24 +++++++++++++++++++++++- arch/arm64/kvm/handle_exit.c | 2 +- include/kvm/arm_psci.h | 2 +- include/linux/arm-smccc.h | 13 +++++++++++++ 5 files changed, 39 insertions(+), 4 deletions(-) -- 2.7.4 diff --git a/arch/arm/kvm/handle_exit.c b/arch/arm/kvm/handle_exit.c index 0a2b758..de1aedc 100644 --- a/arch/arm/kvm/handle_exit.c +++ b/arch/arm/kvm/handle_exit.c @@ -36,7 +36,7 @@ static int handle_hvc(struct kvm_vcpu *vcpu, struct kvm_run *run) kvm_vcpu_hvc_get_imm(vcpu)); vcpu->stat.hvc_exit_stat++; - ret = kvm_psci_call(vcpu); + ret = kvm_hvc_call_handler(vcpu); if (ret < 0) { vcpu_set_reg(vcpu, 0, ~0UL); return 1; diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 097632c..7b44253 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -15,6 +15,7 @@ * along with this program. If not, see . */ +#include #include #include #include @@ -338,6 +339,7 @@ static int kvm_psci_1_0_call(struct kvm_vcpu *vcpu) case PSCI_0_2_FN_SYSTEM_OFF: case PSCI_0_2_FN_SYSTEM_RESET: case PSCI_1_0_FN_PSCI_FEATURES: + case ARM_SMCCC_VERSION_FUNC_ID: val = 0; break; default: @@ -392,7 +394,7 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) * Errors: * -EINVAL: Unrecognized PSCI function */ -int kvm_psci_call(struct kvm_vcpu *vcpu) +static int kvm_psci_call(struct kvm_vcpu *vcpu) { switch (kvm_psci_version(vcpu)) { case KVM_ARM_PSCI_1_0: @@ -405,3 +407,23 @@ int kvm_psci_call(struct kvm_vcpu *vcpu) return -EINVAL; }; } + +int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) +{ + u32 func_id = smccc_get_function(vcpu); + u32 val = PSCI_RET_NOT_SUPPORTED; + + switch (func_id) { + case ARM_SMCCC_VERSION_FUNC_ID: + val = ARM_SMCCC_VERSION_1_1; + break; + case ARM_SMCCC_ARCH_FEATURES_FUNC_ID: + /* Nothing supported yet */ + break; + default: + return kvm_psci_call(vcpu); + } + + smccc_set_retval(vcpu, val, 0, 0, 0); + return 1; +} diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 0231ebc..d4047fc 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -43,7 +43,7 @@ static int handle_hvc(struct kvm_vcpu *vcpu, struct kvm_run *run) kvm_vcpu_hvc_get_imm(vcpu)); vcpu->stat.hvc_exit_stat++; - ret = kvm_psci_call(vcpu); + ret = kvm_hvc_call_handler(vcpu); if (ret < 0) { vcpu_set_reg(vcpu, 0, ~0UL); return 1; diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index 3236043..ed1dd80 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -27,6 +27,6 @@ #define KVM_ARM_PSCI_LATEST KVM_ARM_PSCI_1_0 int kvm_psci_version(struct kvm_vcpu *vcpu); -int kvm_psci_call(struct kvm_vcpu *vcpu); +int kvm_hvc_call_handler(struct kvm_vcpu *vcpu); #endif /* __KVM_ARM_PSCI_H__ */ diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index 4c5bca38..dc68aa5 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -60,6 +60,19 @@ #define ARM_SMCCC_QUIRK_NONE 0 #define ARM_SMCCC_QUIRK_QCOM_A6 1 /* Save/restore register a6 */ +#define ARM_SMCCC_VERSION_1_0 0x10000 +#define ARM_SMCCC_VERSION_1_1 0x10001 + +#define ARM_SMCCC_VERSION_FUNC_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_32, \ + 0, 0) + +#define ARM_SMCCC_ARCH_FEATURES_FUNC_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_32, \ + 0, 1) + #ifndef __ASSEMBLY__ #include From patchwork Mon Feb 26 08:20:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129590 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3358585lja; Mon, 26 Feb 2018 00:26:00 -0800 (PST) X-Google-Smtp-Source: AH8x226l3CnCG+ANOAgRPgKwDHKHZ7BnBTych7hwifmVN4sROlIZvvGE4mw2/ZZtaY9Lzyvzxp8s X-Received: by 10.99.159.10 with SMTP id g10mr7683183pge.407.1519633559869; Mon, 26 Feb 2018 00:25:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633559; cv=none; d=google.com; s=arc-20160816; b=oL+gl9nm9TQTEWzRomLcYE3BIfGzoGCTZAgdwidgQg7JYK9H4ZFqomyCCseoAUc6U6 RETL4Q3Q/PY3X9dX19C6hQyiRqDbUfgcilqOaL1uhExwkjn0hGrPNkDzrdeyGZAaeHQF yjrO8Zbj9TZb0AYBLof1ke13hdU8xg0FTDV+Wkcy8Mb7dpcssrZGf68+iTO4v3JSfS6C OiqjOujYTQGEZYptcdD61Qdd+Tc7FlygJRSGdOZcVCOFpTdc1eFSwL1h+nemHYJVLAsW 6SUtClWe/txk2mJCqHGztsuJXYTJPYmzuPh95WiCekC+wEP2wPbmxQLfh7tjdOwWkrHz 5ZZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=1RaPKZO6eEZg+WHRSpnHizT0jS04yyLzTCiVvYJ2E2U=; b=GYLVfy2NhMPaJh0XS3VYzUtPfRg41TeJvTd7q9Pr5/myzKLa45ICaIfmKFMrJAnBr0 uc5RVRvrkW4pbe1p+vAi5McVb9Sg/RDy2rV90ockgI3oHYkyUnR555JKWR1tDjkadLSV G9RuelkkQ+2GsbbxTc/PkRsUTI3mCzdYhoe7v9d/yJCPFM3Z5Vh5SocF0ZXacN1OQIxc CAI5v85FpJGfrF8e5WmNgw92jwd3M6kVdo8jdp+ADYBZgtz1WdE/HpsUw0H4YWFwikqm kLaSvYHZ9hDYR+syu9BQl3waeYe7FJ1PWQ4nBTXj40Dy7MJ7onwrCUJ2VE398susP7d/ 9/DA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=enGsPUns; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u62si5274966pgd.579.2018.02.26.00.25.59; Mon, 26 Feb 2018 00:25:59 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=enGsPUns; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752576AbeBZIZ6 (ORCPT + 10 others); Mon, 26 Feb 2018 03:25:58 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:40131 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752255AbeBZIZy (ORCPT ); Mon, 26 Feb 2018 03:25:54 -0500 Received: by mail-pf0-f193.google.com with SMTP id m5so6218492pff.7 for ; Mon, 26 Feb 2018 00:25:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=1RaPKZO6eEZg+WHRSpnHizT0jS04yyLzTCiVvYJ2E2U=; b=enGsPUnsm+rGZ3967eUBFmebBMV609UJ2zknLGBLShaV2vJ9LWGdLzFp9gqOiJ5xWh eqAWUacYx6U+Rek47L07Sr9PTS5zy84MhA5M4TkcSlD2NJDlYk0DqJUU0qoWKrFo0wXS oskIb/aQo82DYH127fazCPKXOwx34TNBQ5f6s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=1RaPKZO6eEZg+WHRSpnHizT0jS04yyLzTCiVvYJ2E2U=; b=rw3wt6LJruCuJe0qLNbuCOIViull+YzTUMufwM0QYK3A02Xu+AmEFipqJgY4z2i+Yn 2LQmJitUQFE2RsGhrzy8FTAID0UezbgVxr2Ilq58CnALX+VNsS5IP23fc9v4+b2Jgr2u uu4edPQLC5D65Wtk9deNWrOdx+8/fHxrDerOlb/ZJqZQ/aP29zLs2PCk0JgDkLEiMVAD 9P0PZApw3PhuVjtiEZ39185Tjd3KfpM7ro6BuDU5nkYFApb6PGf5t1OAxop6EpCROR04 rm5JxWTAr4o/Lm3HK8v0GovM/xGUTM49vAnRSmic6kg5tSOGrh2crVVFlvrQFC4iAdLs XX2Q== X-Gm-Message-State: APf1xPCin3G0GIIiEX4i7Pbt5VAMeDCmU0ppgWP0XqLA4264WtYYIzpr AkXZOx2F+DkEvqXWISt52yUiog== X-Received: by 10.98.74.67 with SMTP id x64mr9824654pfa.135.1519633553460; Mon, 26 Feb 2018 00:25:53 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.25.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:25:52 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Russell King , linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 37/52] arm/arm64: KVM: Turn kvm_psci_version into a static inline Date: Mon, 26 Feb 2018 16:20:11 +0800 Message-Id: <1519633227-29832-38-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier commit a4097b351118 upstream. We're about to need kvm_psci_version in HYP too. So let's turn it into a static inline, and pass the kvm structure as a second parameter (so that HYP can do a kern_hyp_va on it). Tested-by: Ard Biesheuvel Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: mv changes from virt/kvm/arm/psci.c to arch/arm/kvm/psci.c --- arch/arm/kvm/psci.c | 12 ++---------- arch/arm64/kvm/hyp/switch.c | 20 ++++++++++++-------- include/kvm/arm_psci.h | 21 ++++++++++++++++++++- 3 files changed, 34 insertions(+), 19 deletions(-) -- 2.7.4 diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 7b44253..396eb5d 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -121,7 +121,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) if (!vcpu) return PSCI_RET_INVALID_PARAMS; if (!vcpu->arch.power_off) { - if (kvm_psci_version(source_vcpu) != KVM_ARM_PSCI_0_1) + if (kvm_psci_version(source_vcpu, kvm) != KVM_ARM_PSCI_0_1) return PSCI_RET_ALREADY_ON; else return PSCI_RET_INVALID_PARAMS; @@ -231,14 +231,6 @@ static void kvm_psci_system_reset(struct kvm_vcpu *vcpu) kvm_prepare_system_event(vcpu, KVM_SYSTEM_EVENT_RESET); } -int kvm_psci_version(struct kvm_vcpu *vcpu) -{ - if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) - return KVM_ARM_PSCI_LATEST; - - return KVM_ARM_PSCI_0_1; -} - static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) { struct kvm *kvm = vcpu->kvm; @@ -396,7 +388,7 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) */ static int kvm_psci_call(struct kvm_vcpu *vcpu) { - switch (kvm_psci_version(vcpu)) { + switch (kvm_psci_version(vcpu, vcpu->kvm)) { case KVM_ARM_PSCI_1_0: return kvm_psci_1_0_call(vcpu); case KVM_ARM_PSCI_0_2: diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 3eab6ac..996328e 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -19,6 +19,8 @@ #include #include +#include + #include #include #include @@ -311,14 +313,16 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu) if (exit_code == ARM_EXCEPTION_TRAP && (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC64 || - kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32) && - vcpu_get_reg(vcpu, 0) == PSCI_0_2_FN_PSCI_VERSION) { - u64 val = PSCI_RET_NOT_SUPPORTED; - if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) - val = 2; - - vcpu_set_reg(vcpu, 0, val); - goto again; + kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32)) { + u32 val = vcpu_get_reg(vcpu, 0); + + if (val == PSCI_0_2_FN_PSCI_VERSION) { + val = kvm_psci_version(vcpu, kern_hyp_va(vcpu->kvm)); + if (unlikely(val == KVM_ARM_PSCI_0_1)) + val = PSCI_RET_NOT_SUPPORTED; + vcpu_set_reg(vcpu, 0, val); + goto again; + } } if (static_branch_unlikely(&vgic_v2_cpuif_trap) && diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index ed1dd80..e518e4e 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -18,6 +18,7 @@ #ifndef __KVM_ARM_PSCI_H__ #define __KVM_ARM_PSCI_H__ +#include #include #define KVM_ARM_PSCI_0_1 PSCI_VERSION(0, 1) @@ -26,7 +27,25 @@ #define KVM_ARM_PSCI_LATEST KVM_ARM_PSCI_1_0 -int kvm_psci_version(struct kvm_vcpu *vcpu); +/* + * We need the KVM pointer independently from the vcpu as we can call + * this from HYP, and need to apply kern_hyp_va on it... + */ +static inline int kvm_psci_version(struct kvm_vcpu *vcpu, struct kvm *kvm) +{ + /* + * Our PSCI implementation stays the same across versions from + * v0.2 onward, only adding the few mandatory functions (such + * as FEATURES with 1.0) that are required by newer + * revisions. It is thus safe to return the latest. + */ + if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) + return KVM_ARM_PSCI_LATEST; + + return KVM_ARM_PSCI_0_1; +} + + int kvm_hvc_call_handler(struct kvm_vcpu *vcpu); #endif /* __KVM_ARM_PSCI_H__ */ From patchwork Mon Feb 26 08:20:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129591 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3358761lja; Mon, 26 Feb 2018 00:26:12 -0800 (PST) X-Google-Smtp-Source: AH8x225KnYEeKAP4IOUHD9s3EyGNSEWXmPEVbHoL+VwDJlwlU1LmAAHBmUDCqIURikHANWcn0GVo X-Received: by 2002:a17:902:9a85:: with SMTP id w5-v6mr9771572plp.417.1519633572190; Mon, 26 Feb 2018 00:26:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633572; cv=none; d=google.com; s=arc-20160816; b=Nj0PUIC89uoPzVI+Db5i91WqfKzzMdDDrOaoGcqQSc/6Cf3p1KCv4BuBkm4NKJuLe6 esIhgX2tiNVdfzWmex+b8S6eXlz8v+oQGqsLsJBrfpTKqmbxvkvAtRk8mkcW21QReyQ6 Qq6xqbxtoLTDw2kWml7NO2wzEBRUpNGdo/nX7L3NJ2P90EJ0s/FKtrA45FYWEvrNvXoS GhzsxqYcX1DE2xK/79HsuEMHW/ny1kiH1XQO5a+g9+Fce4X2SWVMKOfN5DZ0/x8IHm9+ K0SdzYnVmKYsdDW1zLM/mYljXxluqXVhwzOfrrR5JHrTwE/ULauDJUOatsfiGR0IOxIB 5DfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=45rSmjFMKcw3BvYABEBt7fYl55v7nB64zwwP7vtw3XM=; b=XCIULhfJET/d3uQxEiJHbPbASoflYSdIwoPopTLMA0lPCgA+Jp5gjUL0kEHFN0j6bl Cuy3MTuGImXYXf6/LVB8i0VBqPZrbtO49KbEj8owJnXDEB4dCkvQBAQs2WhC0R54Wu/0 AKbBF0/mkSFcsAE8uaFBYDBtNYnWKRNimbh+RBTmE4O1+ibIyXCr7MuTgiitP1RbbXa6 o7yevcROcXgl5YzBJfDIIPKGxN4+X3JTF9rXwBMzTtGshGHH5SExCsTwr1BJ3wAbxar7 UX1dz+YF4qOGDhk9otKpWnnSwOmgx4N67w+Ge4cai4fz6J2+6aMSeIs8EWNGfpwsyF6p drfA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Qy/nBnHn; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j127si5283966pgc.653.2018.02.26.00.26.11; Mon, 26 Feb 2018 00:26:12 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Qy/nBnHn; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752544AbeBZI0K (ORCPT + 10 others); Mon, 26 Feb 2018 03:26:10 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:33450 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752253AbeBZI0E (ORCPT ); Mon, 26 Feb 2018 03:26:04 -0500 Received: by mail-pl0-f68.google.com with SMTP id c11-v6so5450952plo.0 for ; Mon, 26 Feb 2018 00:26:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=45rSmjFMKcw3BvYABEBt7fYl55v7nB64zwwP7vtw3XM=; b=Qy/nBnHn996E+ZwHe6hLkYYQlxHxsnIypZQj6cnYiIwBfmMuM8Pkdma37+9jCVk4Qa SQ9+VuQlIZif6zdMPAC/OGKrw+Whz9J9tv1C8KL5ehDLk+VAcPks1qZ3fc3Vc9WeiSp1 UK4vkuDxwglnmW7qgT0ob5kxf+qGnIE8kc+es= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=45rSmjFMKcw3BvYABEBt7fYl55v7nB64zwwP7vtw3XM=; b=kyXzMTX5m4e3fcC6GK6ZqWvEJK+u+kRP03txjxlJ+lEnVe6Fz7giNHf1f4EefNNFpl LCv3SuD8YHMsl/iykUUFWEkNsdA9nzaPgCe4Z3NJUwVOLAxSzuUsiP3DZklAvFkhP3Mf E6xkbCGbPs4FlFN8L+D12+KuX/E2Wmgg9NPCVCOHFS7IXc3Uy7POTX2dU90Nnd/poY4L 2fH/rCtjB+5Z2f2KSJ12t18sI4och3Z9XiOLDphADUQo9c/o7Vsw8zxLq/zogUWKJdZF 8DCKxYGExHkSC+pvn6R1kVbTXeU5fSq0i/ObHSqGcO7dHkp3MQaZgyWI9G+9R3gTS4VD /hnQ== X-Gm-Message-State: APf1xPABCwGg+Mrng6Jp33odOctn1DXT0NIFfQAfsMXRywsYJ2trcdQV TtxFf7YHk8f9dk9CyqQCIueAXH1Rfgw= X-Received: by 2002:a17:902:3341:: with SMTP id a59-v6mr4859451plc.20.1519633563557; Mon, 26 Feb 2018 00:26:03 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.25.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:26:02 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Russell King , linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 38/52] arm64: KVM: Report SMCCC_ARCH_WORKAROUND_1 BP hardening support Date: Mon, 26 Feb 2018 16:20:12 +0800 Message-Id: <1519633227-29832-39-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier commit 6167ec5c9145 upstream. A new feature of SMCCC 1.1 is that it offers firmware-based CPU workarounds. In particular, SMCCC_ARCH_WORKAROUND_1 provides BP hardening for CVE-2017-5715. If the host has some mitigation for this issue, report that we deal with it using SMCCC_ARCH_WORKAROUND_1, as we apply the host workaround on every guest exit. Tested-by: Ard Biesheuvel Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no sve support in arch/arm64/include/asm/kvm_host.h mv changes from virt/kvm/arm/psci.c to arch/arm/kvm/psci.c using cpus_have_cap instead of cpus_have_const_cap --- arch/arm/include/asm/kvm_host.h | 6 ++++++ arch/arm/kvm/psci.c | 9 ++++++++- arch/arm64/include/asm/kvm_host.h | 6 ++++++ include/linux/arm-smccc.h | 5 +++++ 4 files changed, 25 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h index d5423ab..9fe1043 100644 --- a/arch/arm/include/asm/kvm_host.h +++ b/arch/arm/include/asm/kvm_host.h @@ -318,4 +318,10 @@ static inline int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, return -ENXIO; } +static inline bool kvm_arm_harden_branch_predictor(void) +{ + /* No way to detect it yet, pretend it is not there. */ + return false; +} + #endif /* __ARM_KVM_HOST_H__ */ diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 396eb5d..a825638 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -404,13 +404,20 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) { u32 func_id = smccc_get_function(vcpu); u32 val = PSCI_RET_NOT_SUPPORTED; + u32 feature; switch (func_id) { case ARM_SMCCC_VERSION_FUNC_ID: val = ARM_SMCCC_VERSION_1_1; break; case ARM_SMCCC_ARCH_FEATURES_FUNC_ID: - /* Nothing supported yet */ + feature = smccc_get_arg1(vcpu); + switch(feature) { + case ARM_SMCCC_ARCH_WORKAROUND_1: + if (kvm_arm_harden_branch_predictor()) + val = 0; + break; + } break; default: return kvm_psci_call(vcpu); diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index e505038..7f5244a 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -393,4 +393,10 @@ static inline void __cpu_init_stage2(void) "PARange is %d bits, unsupported configuration!", parange); } + +static inline bool kvm_arm_harden_branch_predictor(void) +{ + return cpus_have_cap(ARM64_HARDEN_BRANCH_PREDICTOR); +} + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index dc68aa5..e1ef944 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -73,6 +73,11 @@ ARM_SMCCC_SMC_32, \ 0, 1) +#define ARM_SMCCC_ARCH_WORKAROUND_1 \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_32, \ + 0, 0x8000) + #ifndef __ASSEMBLY__ #include From patchwork Mon Feb 26 08:20:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129597 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359509lja; Mon, 26 Feb 2018 00:27:22 -0800 (PST) X-Google-Smtp-Source: AH8x2241yU/V+mv7JQxkby+likMC9pqciLFpgrc3pRfmJAdCSfHcSEtNFRMGpH/z5896X23+uBYs X-Received: by 2002:a17:902:bb89:: with SMTP id m9-v6mr9851576pls.423.1519633641958; Mon, 26 Feb 2018 00:27:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633641; cv=none; d=google.com; s=arc-20160816; b=xqYqjRdO+wpenpeXltnA7jFiWU5hPoARXxsrqxhWgCW/lD7wUytGHCwSKV36ogh1zT BgVf+Peh3+mA2Bk5lyZtRGgrFTT9W4BbMVnrkCH4+ppYW6G32Ma/uR3Lqx1W5c2Pdcqz Aeivf/yWEC8BLTYzE0hfKHLTVrHnSoDjpeDk2AXhMBCigPy5fDL+6JU0uV+jX97UM9de 5pNCFJKLY7Q6QzBKcvQAoLWyJBEVezV9QcJ98QwpRIOdQBk1eXUzrciNWvX6B0a2fNsE albpDHRKlM602KPMJ/Le2Wu53SlEfT2HOp5DCwNO3WMIRpqJMoZP3K5Nl+TyPiVYfp02 s/xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=E4PLUgLdMJiPt35VITpiyZYx813szBZN2dNVjcYWHUQ=; b=Hnp9d3F8pqTyEfC+9hXHT71dFAk68QMII6cB+bIfYiiBNax0UFUnNYUDIsKMHsSl8K SIiZvkvi2p1u99QbBJuSNHnbquKOKNWbI37CV75hyp9DJmD8rjAGdnSiMsdxsaCzZJi/ /BUVAU1KAIGRXv2g2qRSotBlGT8L8U8vEAB2kZcBFe2c9MtBiBkJImlZxrFCq8rgbXFT JFP9iet7VFM40AXj573QABlEj7PY66yM5LRoSIW9f6tIlEGONYXY3kNzg21I6ov1ywyG eX3UwiNcTWANQAzF1qEF/nFFn12fX9LvsL6GasVVe/aLv6XC7xW48NhPMO89RgbdkOcv wngg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=S3WGURxE; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s190si5281566pgc.510.2018.02.26.00.27.21; Mon, 26 Feb 2018 00:27:21 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=S3WGURxE; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752842AbeBZI1T (ORCPT + 10 others); Mon, 26 Feb 2018 03:27:19 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:46765 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752524AbeBZI1P (ORCPT ); Mon, 26 Feb 2018 03:27:15 -0500 Received: by mail-pg0-f68.google.com with SMTP id r26so679331pgv.13 for ; Mon, 26 Feb 2018 00:27:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=E4PLUgLdMJiPt35VITpiyZYx813szBZN2dNVjcYWHUQ=; b=S3WGURxEucXq78zQtar10w9gSolJRggmlVjVZHWaqQzcRNy1r0f1TFfnAJh461NhX2 HmxlrLhNwDPTjJsKLLilmgKBdyKjxPmej2hvQHf+HIMWMww10JBOgNtGzsE3Nup2yGV4 ucH/N7WsofJNcmquDuaoKFLBsrUdPtGGN033M= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=E4PLUgLdMJiPt35VITpiyZYx813szBZN2dNVjcYWHUQ=; b=BWymVXqTXR2mPmepFEyUqodMcIE7N8/Mg5hM94QqYYddvsPNj1mqaSCRjJvWJYOnjH 3SiumZUCfq3NwtIU/TqBh/HfLS3XwrBxCc2169z8dnrs0IXiGIAJhFr1qcQQhstiKjG/ qKpMe9RSjQwsb3V5S2L8JQgyDcs1gCokv4Bj+Fti/95YGHBF63HmfmnxupCFTT7neuJu i5M34LGIxriMLU7CewTfJKBi+WccTquofAXfUrFLifIkCGzMatjBdZocvo8UcXb/vCXe Lyi08pJnM1QTh1+RUnTcGTgIi/SccmIqSuVVxZ+kpm1lpnG2CVaBkgQRYn50FxMbBjrr DqSQ== X-Gm-Message-State: APf1xPAksGZYmb979PnsN19zzPlLj9ENUiNGhS61sqa/BPR7iz8ajfRJ uyCAbGwKihx5cOMgGx2IgYaLbQ== X-Received: by 10.99.103.133 with SMTP id b127mr7726353pgc.155.1519633635036; Mon, 26 Feb 2018 00:27:15 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.27.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:14 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Russell King , linux-arm-kernel@lists.infradead.org (moderated list:ARM PORT), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 47/52] arm: Invalidate BTB on prefetch abort outside of user mapping on Cortex A8, A9, A12 and A17 Date: Mon, 26 Feb 2018 16:20:21 +0800 Message-Id: <1519633227-29832-48-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier ** Not yet queued for inclusion in mainline ** In order to prevent aliasing attacks on the branch predictor, invalidate the BTB on CPUs that are known to be affected when taking a prefetch abort on a address that is outside of a user task limit. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/include/asm/cp15.h | 2 ++ arch/arm/mm/fault.c | 25 ++++++++++++++++ arch/arm/mm/fsr-2level.c | 4 +-- arch/arm/mm/fsr-3level.c | 69 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 98 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h index dbdbce1..0672ddc 100644 --- a/arch/arm/include/asm/cp15.h +++ b/arch/arm/include/asm/cp15.h @@ -64,6 +64,8 @@ #define __write_sysreg(v, r, w, c, t) asm volatile(w " " c : : "r" ((t)(v))) #define write_sysreg(v, ...) __write_sysreg(v, __VA_ARGS__) +#define BPIALL __ACCESS_CP15(c7, 0, c5, 6) + extern unsigned long cr_alignment; /* defined in entry-armv.S */ static inline unsigned long get_cr(void) diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index f7861dc..17fd0c7 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -20,6 +20,7 @@ #include #include +#include #include #include #include @@ -395,12 +396,36 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) __do_kernel_fault(mm, addr, fsr, regs); return 0; } + +static int __maybe_unused +do_pabt_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) +{ +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + if (addr > TASK_SIZE) { + switch (read_cpuid_part()) { + case ARM_CPU_PART_CORTEX_A8: + case ARM_CPU_PART_CORTEX_A9: + case ARM_CPU_PART_CORTEX_A12: + case ARM_CPU_PART_CORTEX_A17: + write_sysreg(0, BPIALL); + break; + } + } +#endif + return do_page_fault(addr, fsr, regs); +} #else /* CONFIG_MMU */ static int do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) { return 0; } + +static int +do_pabt_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) +{ + return 0; +} #endif /* CONFIG_MMU */ /* diff --git a/arch/arm/mm/fsr-2level.c b/arch/arm/mm/fsr-2level.c index 18ca74c..4cede9b 100644 --- a/arch/arm/mm/fsr-2level.c +++ b/arch/arm/mm/fsr-2level.c @@ -50,7 +50,7 @@ static struct fsr_info ifsr_info[] = { { do_bad, SIGBUS, 0, "unknown 4" }, { do_translation_fault, SIGSEGV, SEGV_MAPERR, "section translation fault" }, { do_bad, SIGSEGV, SEGV_ACCERR, "page access flag fault" }, - { do_page_fault, SIGSEGV, SEGV_MAPERR, "page translation fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_MAPERR, "page translation fault" }, { do_bad, SIGBUS, 0, "external abort on non-linefetch" }, { do_bad, SIGSEGV, SEGV_ACCERR, "section domain fault" }, { do_bad, SIGBUS, 0, "unknown 10" }, @@ -58,7 +58,7 @@ static struct fsr_info ifsr_info[] = { { do_bad, SIGBUS, 0, "external abort on translation" }, { do_sect_fault, SIGSEGV, SEGV_ACCERR, "section permission fault" }, { do_bad, SIGBUS, 0, "external abort on translation" }, - { do_page_fault, SIGSEGV, SEGV_ACCERR, "page permission fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_ACCERR, "page permission fault" }, { do_bad, SIGBUS, 0, "unknown 16" }, { do_bad, SIGBUS, 0, "unknown 17" }, { do_bad, SIGBUS, 0, "unknown 18" }, diff --git a/arch/arm/mm/fsr-3level.c b/arch/arm/mm/fsr-3level.c index ab4409a..2745f7f 100644 --- a/arch/arm/mm/fsr-3level.c +++ b/arch/arm/mm/fsr-3level.c @@ -65,4 +65,73 @@ static struct fsr_info fsr_info[] = { { do_bad, SIGBUS, 0, "unknown 63" }, }; +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +static struct fsr_info ifsr_info[] = { + { do_bad, SIGBUS, 0, "unknown 0" }, + { do_bad, SIGBUS, 0, "unknown 1" }, + { do_bad, SIGBUS, 0, "unknown 2" }, + { do_bad, SIGBUS, 0, "unknown 3" }, + { do_bad, SIGBUS, 0, "reserved translation fault" }, + { do_translation_fault, SIGSEGV, SEGV_MAPERR, "level 1 translation fault" }, + { do_translation_fault, SIGSEGV, SEGV_MAPERR, "level 2 translation fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_MAPERR, "level 3 translation fault" }, + { do_bad, SIGBUS, 0, "reserved access flag fault" }, + { do_bad, SIGSEGV, SEGV_ACCERR, "level 1 access flag fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_ACCERR, "level 2 access flag fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 access flag fault" }, + { do_bad, SIGBUS, 0, "reserved permission fault" }, + { do_bad, SIGSEGV, SEGV_ACCERR, "level 1 permission fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_ACCERR, "level 2 permission fault" }, + { do_pabt_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 permission fault" }, + { do_bad, SIGBUS, 0, "synchronous external abort" }, + { do_bad, SIGBUS, 0, "asynchronous external abort" }, + { do_bad, SIGBUS, 0, "unknown 18" }, + { do_bad, SIGBUS, 0, "unknown 19" }, + { do_bad, SIGBUS, 0, "synchronous abort (translation table walk)" }, + { do_bad, SIGBUS, 0, "synchronous abort (translation table walk)" }, + { do_bad, SIGBUS, 0, "synchronous abort (translation table walk)" }, + { do_bad, SIGBUS, 0, "synchronous abort (translation table walk)" }, + { do_bad, SIGBUS, 0, "synchronous parity error" }, + { do_bad, SIGBUS, 0, "asynchronous parity error" }, + { do_bad, SIGBUS, 0, "unknown 26" }, + { do_bad, SIGBUS, 0, "unknown 27" }, + { do_bad, SIGBUS, 0, "synchronous parity error (translation table walk" }, + { do_bad, SIGBUS, 0, "synchronous parity error (translation table walk" }, + { do_bad, SIGBUS, 0, "synchronous parity error (translation table walk" }, + { do_bad, SIGBUS, 0, "synchronous parity error (translation table walk" }, + { do_bad, SIGBUS, 0, "unknown 32" }, + { do_bad, SIGBUS, BUS_ADRALN, "alignment fault" }, + { do_bad, SIGBUS, 0, "debug event" }, + { do_bad, SIGBUS, 0, "unknown 35" }, + { do_bad, SIGBUS, 0, "unknown 36" }, + { do_bad, SIGBUS, 0, "unknown 37" }, + { do_bad, SIGBUS, 0, "unknown 38" }, + { do_bad, SIGBUS, 0, "unknown 39" }, + { do_bad, SIGBUS, 0, "unknown 40" }, + { do_bad, SIGBUS, 0, "unknown 41" }, + { do_bad, SIGBUS, 0, "unknown 42" }, + { do_bad, SIGBUS, 0, "unknown 43" }, + { do_bad, SIGBUS, 0, "unknown 44" }, + { do_bad, SIGBUS, 0, "unknown 45" }, + { do_bad, SIGBUS, 0, "unknown 46" }, + { do_bad, SIGBUS, 0, "unknown 47" }, + { do_bad, SIGBUS, 0, "unknown 48" }, + { do_bad, SIGBUS, 0, "unknown 49" }, + { do_bad, SIGBUS, 0, "unknown 50" }, + { do_bad, SIGBUS, 0, "unknown 51" }, + { do_bad, SIGBUS, 0, "implementation fault (lockdown abort)" }, + { do_bad, SIGBUS, 0, "unknown 53" }, + { do_bad, SIGBUS, 0, "unknown 54" }, + { do_bad, SIGBUS, 0, "unknown 55" }, + { do_bad, SIGBUS, 0, "unknown 56" }, + { do_bad, SIGBUS, 0, "unknown 57" }, + { do_bad, SIGBUS, 0, "implementation fault (coprocessor abort)" }, + { do_bad, SIGBUS, 0, "unknown 59" }, + { do_bad, SIGBUS, 0, "unknown 60" }, + { do_bad, SIGBUS, 0, "unknown 61" }, + { do_bad, SIGBUS, 0, "unknown 62" }, + { do_bad, SIGBUS, 0, "unknown 63" }, +}; +#else #define ifsr_info fsr_info +#endif From patchwork Mon Feb 26 08:20:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129598 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359669lja; Mon, 26 Feb 2018 00:27:39 -0800 (PST) X-Google-Smtp-Source: AG47ELuhYDD3NzytRi7hzK6t0ov/MHR1OtX+GxJeIRh4jzpy8HYU/u+9PIxgHvqqW92IbzZMUk8Z X-Received: by 2002:a17:902:14b:: with SMTP id 69-v6mr4526441plb.209.1519633659507; Mon, 26 Feb 2018 00:27:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633659; cv=none; d=google.com; s=arc-20160816; b=insN+c/ysRrguTQqXSNpaETCGOIEexWrEGCof7v7WYuO8esRQm5OtdoufH4OGd7Gg4 DR4m2nYTLLR57Bq70Lr0WeXJIe2Vo1bxLFO9RD/t7hAmNhJLsUmDql2gBmYY6/NS7Or9 h+59/40ghnNeO+Cwk9775cxcZBubpuTqoYYx8GSO0iwOpesgd/vHAr5BYjsRMHDf56vF jmlUaAFW2JOljI5v8of+a2B5mlSTWi4w23t6a5LCflPv/xzP/qvER+8r8R1UezoTlFit jLroRTnSc+CILNUV2kf8nJ/tHg7V560pR9CRtat0OTEEPYrJ8qMb+kbWGyb79wat0jKq O85g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=EGJE0Nu47ttv68Cv7IDoZdvRv6F03kpbdP4X9eVpyD0=; b=lwc1//tCXfu/lHDGONmMg37Mm0Ji2Sb9i2iG30DbazJ9xi5Xp/E5Z8NzQ4XEwD16A6 bpzQ+6GIDzRn8RV39ifrYaaEvsHrUZ0nLWP8z6bmLVN7RPRYiMxL45Tq8U9iTss67S7r 0HgdVFVtBxJpoEiJ5g0gH7puQpDD9QfMewGweI+bIE2msrJLJlLhx9gCbGaTkjChA7Hi DqAvZ3SOXkqWp2vR+92FwrexVYhvo4P2d/mHHHyXeLhueqNEzvOuPY14KgTx8Fe64T4n PjEzblL/jY8xhC186ErzuM2HFJxtpdbBVaYi3lU8zWcibiR6cRT2beyObB+e4yuORoSu e/mw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=db7VIA95; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f23-v6si6600728plk.810.2018.02.26.00.27.39; Mon, 26 Feb 2018 00:27:39 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=db7VIA95; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752855AbeBZI1h (ORCPT + 10 others); Mon, 26 Feb 2018 03:27:37 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:46002 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752865AbeBZI1e (ORCPT ); Mon, 26 Feb 2018 03:27:34 -0500 Received: by mail-pl0-f67.google.com with SMTP id v9-v6so6862904plp.12 for ; Mon, 26 Feb 2018 00:27:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=EGJE0Nu47ttv68Cv7IDoZdvRv6F03kpbdP4X9eVpyD0=; b=db7VIA95FUrzght6E19A2huPjfrsEijQBVn16eA8R5RQGOomHH9u+0qfmdUWnDa2N8 /sUlzXqzBC2XiVRukO7Bwe7eVTO9u/2MNNrH6Cqneypvkno4XYFjNQ7y43dqGwFnkvQm umTbWy8t92pJCQWT0LTFIfpr7zSHX8n6LuKDA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=EGJE0Nu47ttv68Cv7IDoZdvRv6F03kpbdP4X9eVpyD0=; b=Zu2T9Ebh01zG/nXzyHIX7pncIqeglYInJAjrGYDy/NKu5ZwXDCEMLPZ4BersxOffkO IeJPKH7WhwYoEfMKX0X2sVS28gn1oL+HBdHMmfeOtiWWvtaYyuRG3EByLQCc+NHhSHBi 0Q3wE5PpMh4A3Q/E1qNOgkVOAu45fNM76Etonbs4VdXQX204/o0nsrNFjbxWVSl7XeLX JCzJMtVTG143j7t5wlIImjsV3eqHjxS3T/dc+KpwCMwRv5n65+FdeArtfB1QTwQVKXd0 7IR5shs3d/gvw/263/S6DPRmDdSa9quP3BFFM0/RJ4bGkj4Peo7bosNRKxri0IiwcDCf i/fQ== X-Gm-Message-State: APf1xPB1eQjs5Pmum261YCe1lKdQNXOmaCKG9pqavx3VFdfE4KSHK88d fMaJfcgRyMJ/qAFBhifjiFQ7PQ== X-Received: by 2002:a17:902:a70f:: with SMTP id w15-v6mr9595442plq.79.1519633652951; Mon, 26 Feb 2018 00:27:32 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.27.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:32 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Russell King , linux-arm-kernel@lists.infradead.org (moderated list:ARM PORT), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 49/52] arm: Add icache invalidation on switch_mm for Cortex-A15 Date: Mon, 26 Feb 2018 16:20:23 +0800 Message-Id: <1519633227-29832-50-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier ** Not yet queued for inclusion in mainline ** In order to avoid aliasing attacks against the branch predictor, Cortex-A15 require to invalidate the BTB when switching from one user context to another. The only way to do so on this CPU is to perform an ICIALLU, having set ACTLR[0] to 1 from secure mode. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/mm/proc-v7-2level.S | 10 ++++++++++ arch/arm/mm/proc-v7-3level.S | 10 ++++++++++ arch/arm/mm/proc-v7.S | 23 ++++++++++++++++++++++- 3 files changed, 42 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/arch/arm/mm/proc-v7-2level.S b/arch/arm/mm/proc-v7-2level.S index 0422e58b..6d81ed7 100644 --- a/arch/arm/mm/proc-v7-2level.S +++ b/arch/arm/mm/proc-v7-2level.S @@ -40,7 +40,16 @@ * Note that we always need to flush BTAC/BTB if IBE is set * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. + * + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. */ +ENTRY(cpu_v7_icinv_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + /* Fall through to switch_mm... */ +#endif + ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0 @@ -67,6 +76,7 @@ ENTRY(cpu_v7_switch_mm) bx lr ENDPROC(cpu_v7_switch_mm) ENDPROC(cpu_v7_btbinv_switch_mm) +ENDPROC(cpu_v7_icinv_switch_mm) /* * cpu_v7_set_pte_ext(ptep, pte) diff --git a/arch/arm/mm/proc-v7-3level.S b/arch/arm/mm/proc-v7-3level.S index a25450b..af6d7b1 100644 --- a/arch/arm/mm/proc-v7-3level.S +++ b/arch/arm/mm/proc-v7-3level.S @@ -54,6 +54,15 @@ * Set the translation table base pointer to be pgd_phys (physical address of * the new TTB). */ +ENTRY(cpu_v7_icinv_switch_mm) +#ifdef CONFIG_MMU + /* + * Cortex-A15 requires ACTLR[0] to be set from secure in order + * for the icache invalidation to also invalidate the BTB. + */ + mcr p15, 0, r0, c7, c5, 0 @ ICIALLU + /* Fall through to switch_mm... */ +#endif ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mcr p15, 0, r0, c7, c5, 6 @ flush BTAC/BTB @@ -69,6 +78,7 @@ ENTRY(cpu_v7_switch_mm) ret lr ENDPROC(cpu_v7_switch_mm) ENDPROC(cpu_v7_btbinv_switch_mm) +ENDPROC(cpu_v7_icinv_switch_mm) #ifdef __ARMEB__ #define rl r3 diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S index ff7018a..f385933 100644 --- a/arch/arm/mm/proc-v7.S +++ b/arch/arm/mm/proc-v7.S @@ -227,6 +227,26 @@ ENDPROC(cpu_ca9mp_do_resume) globl_equ cpu_ca17_do_resume, cpu_v7_do_resume #endif +/* + * Cortex-A15 + */ + globl_equ cpu_ca15_proc_init, cpu_v7_proc_init + globl_equ cpu_ca15_proc_fin, cpu_v7_proc_fin + globl_equ cpu_ca15_reset, cpu_v7_reset + globl_equ cpu_ca15_do_idle, cpu_v7_do_idle + globl_equ cpu_ca15_dcache_clean_area, cpu_v7_dcache_clean_area + globl_equ cpu_ca15_set_pte_ext, cpu_v7_set_pte_ext + globl_equ cpu_ca15_suspend_size, cpu_v7_suspend_size +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + globl_equ cpu_ca15_switch_mm, cpu_v7_icinv_switch_mm +#else + globl_equ cpu_ca15_switch_mm, cpu_v7_switch_mm +#endif +#ifdef CONFIG_ARM_CPU_SUSPEND + globl_equ cpu_ca15_do_suspend, cpu_v7_do_suspend + globl_equ cpu_ca15_do_resume, cpu_v7_do_resume +#endif + #ifdef CONFIG_CPU_PJ4B globl_equ cpu_pj4b_switch_mm, cpu_v7_switch_mm globl_equ cpu_pj4b_set_pte_ext, cpu_v7_set_pte_ext @@ -568,6 +588,7 @@ __v7_setup_stack: @ define struct processor (see and proc-macros.S) define_processor_functions v7, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 + define_processor_functions ca15, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 define_processor_functions ca17, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 #ifndef CONFIG_ARM_LPAE define_processor_functions ca8, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 @@ -689,7 +710,7 @@ __v7_ca12mp_proc_info: __v7_ca15mp_proc_info: .long 0x410fc0f0 .long 0xff0ffff0 - __v7_proc __v7_ca15mp_proc_info, __v7_ca15mp_setup + __v7_proc __v7_ca15mp_proc_info, __v7_ca15mp_setup, proc_fns = ca15_processor_functions .size __v7_ca15mp_proc_info, . - __v7_ca15mp_proc_info /* From patchwork Mon Feb 26 08:20:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129600 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359792lja; Mon, 26 Feb 2018 00:27:51 -0800 (PST) X-Google-Smtp-Source: AH8x227qmjbGCz4fNoiDPGY++cGNHrGCd3SOEqoaCRvz9pYzD6dCjhkarQADsuj4hlWs8QVOq7iB X-Received: by 10.98.204.69 with SMTP id a66mr9833394pfg.33.1519633670892; Mon, 26 Feb 2018 00:27:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633670; cv=none; d=google.com; s=arc-20160816; b=IE0Z9jrpEGgpflnYGBjY6rmAJkdSqWxMiRecrsPIpKYqXXoKeuhK5DE3+1EObbFigZ VWrH52UK+esz+DhUQoixR6J0I8f1Vfe8+xGXNGW3fjRwEEYLfQ/Rwx2H12KlhtlvbbRJ jYDJjZ6shdtfe+R3mAu/PMT8ghBWHQdgr6BrjN+TIBR0dL2IwFHlWFt8of1EUtz3th8+ F6K6Vk15z6soK/qUn5WS6s8Jxdy4ft9HtdhVkk42uyc8eR30bddRFEw0sstLaoUWNz9H Yymkj4ISoR0dSS7jXizgthcrIuJzbYc30wKHq6HyuQ8ilQjEd71evqSaWuWoZWxHFZPG n8ww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=TZy7LbCE06w5P4ocO94iT/5aR/zA6c96V+vApCDKSFE=; b=P8s3b+Zkub02Ceg3dAsvGhia81q6uIWQBbhOhdXMYO2KqQ9fc2KBa4zN3FCBbJWVPR DhssfOLCX995qCpRRE4r30c5NmzRKv8ycNIm3WHaoRmrfSdLk3ZXo/Hw6wX3dt4Giyz5 zEMbOW1Al2c4bqUWWqHqD15EV2TWDKyc3QrZ0HZO+G6Rgy+r0E1o24XT0Pb2pL47Ov2s l3c+PPg61jHAVvjS71aIcQk8cVG/bDRXoD5k6SwXX2HBquJ5l5ConN0Zq6/q3p7PxwU5 2aIkoTUuSU3nglchfqfcZzdLCqk5gVg3ykj/JZ9pxbNmi/VhGIskltpLNvqyfu/iBhld znvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OfTMe6MG; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f23-v6si6600728plk.810.2018.02.26.00.27.50; Mon, 26 Feb 2018 00:27:50 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OfTMe6MG; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752879AbeBZI1n (ORCPT + 10 others); Mon, 26 Feb 2018 03:27:43 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:44756 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752205AbeBZI1k (ORCPT ); Mon, 26 Feb 2018 03:27:40 -0500 Received: by mail-pg0-f67.google.com with SMTP id l4so5908959pgp.11 for ; Mon, 26 Feb 2018 00:27:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=TZy7LbCE06w5P4ocO94iT/5aR/zA6c96V+vApCDKSFE=; b=OfTMe6MGoKUHFGHU9OxUtwkc8u9CS1aPau9TvxDAM3avnGYMHCgq0SDhAXDKkBwTwF 3HYJUrBFoBQh+OKmL1Xr3bomSyAxicLHBH3dPXAIhyp9NLaQELuePug3yhfuIHa9ScuH TO7tnHEAri4iGkfFggSbD3jz9wOYXqOeyQSs4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=TZy7LbCE06w5P4ocO94iT/5aR/zA6c96V+vApCDKSFE=; b=LTSdPiz/8MQGV2yReBVTe/YaJtYF03AAh2An4DiMP+CkLjFf7pZ5nMiVLwbqSbehxc 1lkIPnX/MGtoxTTXkXFDvZmI1lsV8vJyGN4334gvZxORsUZKnQn1qnGER5OeXaHDHnVJ qaGlPHMghrxsmYQ/lPCPShk2WxtpShoAchpsxJWPU8PdetidT1da8RKIY0rjLkC2QzWO tQ3NbNYypiP/omp/AIFq341nkL2uqwzog7nEvfi9Zbb9/w1XwmELu0wzdF6E0ROslF8P a0EsulJk7OILROAPw/e2ABb7+P8DbeuW5DHf+1yklnt+OoSHFEahnZ0EYxxl5uwnBFTs OBug== X-Gm-Message-State: APf1xPBjLULZHbsumDgyFL4TxktehYMI/yYKpJL2Wd7+rupD9KRVFOcz 7/w/TOjHtJ+bwZBRzIoyk8oM2Q== X-Received: by 10.99.150.2 with SMTP id c2mr7981687pge.424.1519633659812; Mon, 26 Feb 2018 00:27:39 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.27.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:39 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Russell King , linux-arm-kernel@lists.infradead.org (moderated list:ARM PORT), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 50/52] arm: Invalidate icache on prefetch abort outside of user mapping on Cortex-A15 Date: Mon, 26 Feb 2018 16:20:24 +0800 Message-Id: <1519633227-29832-51-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier ** Not yet queued for inclusion in mainline ** In order to prevent aliasing attacks on the branch predictor, invalidate the icache on Cortex-A15, which has the side effect of invalidating the BTB. This requires ACTLR[0] to be set to 1 (secure operation). Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/include/asm/cp15.h | 1 + arch/arm/mm/fault.c | 4 ++++ 2 files changed, 5 insertions(+) -- 2.7.4 diff --git a/arch/arm/include/asm/cp15.h b/arch/arm/include/asm/cp15.h index 0672ddc..b74b174 100644 --- a/arch/arm/include/asm/cp15.h +++ b/arch/arm/include/asm/cp15.h @@ -65,6 +65,7 @@ #define write_sysreg(v, ...) __write_sysreg(v, __VA_ARGS__) #define BPIALL __ACCESS_CP15(c7, 0, c5, 6) +#define ICIALLU __ACCESS_CP15(c7, 0, c5, 0) extern unsigned long cr_alignment; /* defined in entry-armv.S */ diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c index 17fd0c7..a9c924b 100644 --- a/arch/arm/mm/fault.c +++ b/arch/arm/mm/fault.c @@ -409,6 +409,10 @@ do_pabt_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs) case ARM_CPU_PART_CORTEX_A17: write_sysreg(0, BPIALL); break; + + case ARM_CPU_PART_CORTEX_A15: + write_sysreg(0, ICIALLU); + break; } } #endif From patchwork Mon Feb 26 08:20:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129601 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359829lja; Mon, 26 Feb 2018 00:27:54 -0800 (PST) X-Google-Smtp-Source: AH8x225p7gr/V2hI/QtTp1e2muSu/u6Qr0bbHLcrc8aa5ccTvfZRlejrokXRDNQ3D05Jl5KV8135 X-Received: by 2002:a17:902:a504:: with SMTP id s4-v6mr10155526plq.43.1519633674865; Mon, 26 Feb 2018 00:27:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633674; cv=none; d=google.com; s=arc-20160816; b=C0wVx3vGrmQhY1ePenvNeCbmlhwbgCRvNNVYMXIDJli8HKkpg5rYkUYWVXSLNjh/lW lavbhMDHi8R7YlXKOqunTfOqHZh9Miq2ShNctfvcCWw3SNFPdGt5ilcym81VN4hOTpFq 33QswwZ+P5WJl6O04XnTd/mTXySrwGt7G4suii8r1AOEDrnwdURv634Fky+vnMe0+rYE 0dzxpvTZpU9lNleImhJ6tvKl3S7ItaRXSO6s/adUNkjgPaxdp5FgWy4xN2S6BO7B1JZ+ nh6DZqAc6jl8KpEsLDy+rU1FdNKjaXPVC/siSJTwCB4HXVZlvIZRDZfvEFSKZOzUQEzm BKnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=TVG6WlEkeS+IhPzLyRKtulufYo3MBwa/boLRfDGGR6U=; b=WmjBr3+QGi0T9KITuYI4LhwStcOzvno7qyDBBY2kXzCHtJ6hfURKFsq+Bo+wqJ3m31 eUqbUnHt2bwf+Iy+VRA76w1Cc0UTmhdWg1z1vESMzRMjxprc5K+6O/q7OoaUJh41p+BE avFR93IUosQjGWUYUO/2hV+o4NY3rYWgIJgXyW2AjlTC23lREih7DGCIuvs6ykLSe0ut HLeQps/Mt16SUJZlcLxjP5Nkc4HogZMW6pL3/cTBKkF7T++VLay3F+u4G8S9EipWu4vn eUNXXFxWQAWvZcvqz4ZUEvIFGuzG+OmvCbCeRst3KLC84KMUAOenofkrGn7Tz7xwjZnn EZtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=D16PV8HT; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f23-v6si6600728plk.810.2018.02.26.00.27.54; Mon, 26 Feb 2018 00:27:54 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=D16PV8HT; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752433AbeBZI1x (ORCPT + 10 others); Mon, 26 Feb 2018 03:27:53 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:39720 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752890AbeBZI1t (ORCPT ); Mon, 26 Feb 2018 03:27:49 -0500 Received: by mail-pl0-f67.google.com with SMTP id s13so8864839plq.6 for ; Mon, 26 Feb 2018 00:27:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=TVG6WlEkeS+IhPzLyRKtulufYo3MBwa/boLRfDGGR6U=; b=D16PV8HTTATeyZl6Ni3Yo6Wioqh3zuHxq2xZnFTqUYLnViWRyw0VIfd2HrfiZuJs+G ogt+dGmsiCyIaXdIMNe3S+YzeN8SgV8C54djUtgwJviaAL8Ez88fUgWVWMEUQkfpSoxr Bra4SXrMnYnQr8PnS9P9bEH2zZaUG2mZO1qxQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=TVG6WlEkeS+IhPzLyRKtulufYo3MBwa/boLRfDGGR6U=; b=UWB0Vov8OqyJZxlhgp5PVmundOKwv1bfKJyV2kV+u2WpR0OxoNmk2Vpd4fu0e3Ez0T IFfDHlQPddrCcBDDdfxauNRJz8rNaEY0QiqVF4yMw6bx28xKenn8rWkM62jESLf+de9T xrd3GO5ORryzFXLCkJD+ivAC17rNgr2GE+SvuZATlhSCKcDBI27qF7x0WcR1tLdBbMHM UmiInk9spwRap1gV+r0mbtlyNWe4vhWbs1JAd0l7gFXqh04lWNRtdf6gb/tG8YuQjHgi 9TjwxEOazJ4ocqPkRRXxnW0ShjhRZYun3HqRmRD5N4ikar0vt+1s9/snYk4Vl1aqXpub 5r5w== X-Gm-Message-State: APf1xPBij0L2ig/PIzS7MtbVV2AV2KmOgz2kACAfGNuNsjnL/xjF4XA+ L70jfWSihR78Koxm9uJg6/i1qQ== X-Received: by 2002:a17:902:5e3:: with SMTP id f90-v6mr9954123plf.413.1519633669077; Mon, 26 Feb 2018 00:27:49 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.27.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:48 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Russell King , linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 51/52] arm: KVM: Invalidate icache on guest exit for Cortex-A15 Date: Mon, 26 Feb 2018 16:20:25 +0800 Message-Id: <1519633227-29832-52-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Marc Zyngier ** Not yet queued for inclusion in mainline ** In order to avoid aliasing attacks against the branch predictor on Cortex-A15, let's invalidate the BTB on guest exit, which can only be done by invalidating the icache (with ACTLR[0] being set). We use the same hack as for A12/A17 to perform the vector decoding. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/include/asm/kvm_mmu.h | 5 +++++ arch/arm/kvm/hyp/hyp-entry.S | 24 ++++++++++++++++++++++++ 2 files changed, 29 insertions(+) -- 2.7.4 diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index 2887129..a602467 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -235,6 +235,11 @@ static inline void *kvm_get_hyp_vector(void) return kvm_ksym_ref(__kvm_hyp_vector_bp_inv); } + case ARM_CPU_PART_CORTEX_A15: + { + extern char __kvm_hyp_vector_ic_inv[]; + return kvm_ksym_ref(__kvm_hyp_vector_ic_inv); + } #endif default: { diff --git a/arch/arm/kvm/hyp/hyp-entry.S b/arch/arm/kvm/hyp/hyp-entry.S index b6b8cb1..4492768 100644 --- a/arch/arm/kvm/hyp/hyp-entry.S +++ b/arch/arm/kvm/hyp/hyp-entry.S @@ -73,6 +73,28 @@ __kvm_hyp_vector: #ifdef CONFIG_HARDEN_BRANCH_PREDICTOR .align 5 +__kvm_hyp_vector_ic_inv: + .global __kvm_hyp_vector_ic_inv + + /* + * We encode the exception entry in the bottom 3 bits of + * SP, and we have to guarantee to be 8 bytes aligned. + */ + W(add) sp, sp, #1 /* Reset 7 */ + W(add) sp, sp, #1 /* Undef 6 */ + W(add) sp, sp, #1 /* Syscall 5 */ + W(add) sp, sp, #1 /* Prefetch abort 4 */ + W(add) sp, sp, #1 /* Data abort 3 */ + W(add) sp, sp, #1 /* HVC 2 */ + W(add) sp, sp, #1 /* IRQ 1 */ + W(nop) /* FIQ 0 */ + + mcr p15, 0, r0, c7, c5, 0 /* ICIALLU */ + isb + + b decode_vectors + + .align 5 __kvm_hyp_vector_bp_inv: .global __kvm_hyp_vector_bp_inv @@ -92,6 +114,8 @@ __kvm_hyp_vector_bp_inv: mcr p15, 0, r0, c7, c5, 6 /* BPIALL */ isb +decode_vectors: + #ifdef CONFIG_THUMB2_KERNEL /* * Yet another silly hack: Use VPIDR as a temp register. From patchwork Mon Feb 26 08:20:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129602 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359899lja; Mon, 26 Feb 2018 00:28:03 -0800 (PST) X-Google-Smtp-Source: AH8x22653Iztp5ebERDb9BzMoslzfIevic9byHvQmS6l3Bhy36TueiaLM6RxPcjMNo4BSWZ3eG+K X-Received: by 10.101.93.134 with SMTP id f6mr7875451pgt.293.1519633683379; Mon, 26 Feb 2018 00:28:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633683; cv=none; d=google.com; s=arc-20160816; b=XlccdhldCNn8szoY5JRB40uoMEbNZuqZm0bErUtB1abufexyehhN4eUkCphp1sMM8J B7AACDwKTJ6YGVKgSuxWhvAgCaUE48F1uqCSiH9994/qwdL2E/ITEEVR13x36F0PN4BZ 2qu9A39TJ6n3WNHJSW73v1Vr+Ahqo0wmppHpfA3h0rbaRxmdmECrwDNDwcgSawNvL9aq jtmxzKxnqVFyKrDYBQkKs/Cm4LsQOX9zkxpbPgeSS/aZ9SwF2iCXmxYYTwkL18riUXOY oy5xmCrfKiNaYoGTexwv91O2uFchOUpdbteUc7N976GyruYw5lt5pK5F+g76Ter1Y6dy krPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=V4bM2YvsCwu18ZG/KjQ3BYGqsjNxEw8lX9Jc6LRIgdo=; b=byBv0TdOG8giYQw93qRXMF5Gd1NWlGr55Pwdyk5O8k650jOyClqLmgVqoo3DSLdDCW BMJ1xJv1e8Ycw5naNdTnSDdf1879+Vs3rjKh+3xl9gkt1pwM+pQcNa5CakN79Mk14q4D Q98gXGGV+TXlIU2Nn39uGxtx8ZYtW+jQEtok3zzdzcq1dSaOy1XxFDBFJllB+9shAX70 B3mZFOzj0PTYZE/LDUQIIcyRYNXnXzor1lvjilqMxt+zlW/DAL6ypl2wyUzlF5vr7L65 6mmKiKvTW4etRkrjGULm/aiZ14CWNWDkZlSa828gx7y7oSq4aH1DexTVWG/xiqk8TuYn 3Kmg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KE6Jdb94; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u187si5254624pgc.451.2018.02.26.00.28.03; Mon, 26 Feb 2018 00:28:03 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KE6Jdb94; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752921AbeBZI17 (ORCPT + 10 others); Mon, 26 Feb 2018 03:27:59 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:35047 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752899AbeBZI14 (ORCPT ); Mon, 26 Feb 2018 03:27:56 -0500 Received: by mail-pl0-f68.google.com with SMTP id bb3so8871628plb.2 for ; Mon, 26 Feb 2018 00:27:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=V4bM2YvsCwu18ZG/KjQ3BYGqsjNxEw8lX9Jc6LRIgdo=; b=KE6Jdb94bVRfWVEf3QbkJzw0AjKEVxQ4PCud268cgWN4JlMiwxq1GuocPujbPUqmWh P9ZDMWr6aOEGopCu4p50+2Z8C9DlQ/88FYt5tQnFa4rsUnkdzYayruCvDjAqsKv1DId0 RUmHOCQIRSuSp+qNtWrwUf8IiZmKIuileRBVk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=V4bM2YvsCwu18ZG/KjQ3BYGqsjNxEw8lX9Jc6LRIgdo=; b=DnCXcE7q3cnPMr5/deYWa48TE+WSj2f2ZBcJkm5JbLwD3ebfkynIwnrBD1OcqJvBuU oF6Kl131kHuO4ZtE6IDZ8eHQv2CEAzQckKYVBcfrpKQUhDqbVbv/asBjWQB6AMzx2RQv Dr0fEx86FeRbnO5GNA5drq8tBh6fvnoQ8PTIhSN9J7nNIjcdH6HmgSu5cL3s6TtvsVY+ LldpMz7BEkgaq47JYouhzf/NszGCdYukK6Y/RoPD/anWJVvxI3yuQgJVfAqclU1twv1j vGL7QmyvUguiMYCaTIC2UY7L+qmVwGJbPMp+2+tcQPIu3ZyKhLj27vEM5tnBK6ELrBOg 2GWw== X-Gm-Message-State: APf1xPCKM/Rygo9p9fVOJNxrs5oBk2XEDBMJZklvhHxMYJ5LZ4mAYwfI JOXBqugUic5Poiqzyhwdkgtk1A== X-Received: by 2002:a17:902:a585:: with SMTP id az5-v6mr9952799plb.167.1519633676102; Mon, 26 Feb 2018 00:27:56 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.27.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:55 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-kernel@vger.kernel.org (open list) Subject: [PATCH 52/52] arm64: Add README describing the branch Date: Mon, 26 Feb 2018 16:20:26 +0800 Message-Id: <1519633227-29832-53-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Will Deacon Provide some basic guidance about this branch, its layout and how we plan to deploy changes in the future. Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- SPECTRE-MELTDOWN-MITIGATIONS.README | 41 +++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 SPECTRE-MELTDOWN-MITIGATIONS.README -- 2.7.4 diff --git a/SPECTRE-MELTDOWN-MITIGATIONS.README b/SPECTRE-MELTDOWN-MITIGATIONS.README new file mode 100644 index 0000000..7b71b81 --- /dev/null +++ b/SPECTRE-MELTDOWN-MITIGATIONS.README @@ -0,0 +1,41 @@ +INTRODUCTION +============ + +This branch is based on the v4.15 release of Linux and contains arm64 +mitigations against the cache speculation side-channels described at +http://www.arm.com/security-update and commonly referred to as "Spectre" and +"Meltdown". Mitigations against variant 2 are also included for the 32-bit +armv7 architecture. + + +LAYOUT +====== + +Each commit contains a subject prefix which clearly identifies the variant +which it is attempting to mitigate against: + + [Variant 1/Spectre-v1] + [Variant 2/Spectre-v2] + [Variant 3/Meltdown] + +The commit message begins with a reference to the upstream commit ID. The +arm64 patches are targeting the v4.16 release of Linux and so the upstream +commit IDs may not appear in mainline releases prior to v4.16, although they +should still be accurate. + + +NOTES TO INTEGRATORS +==================== + +Please note that this branch is intended to be used as the basis for backport +efforts into stable and distribution kernels. It is not intended to be +integrated directly into production kernels and will consequently be frozen +when v4.16 is released after which it *WILL NOT RECEIVE FURTHER UPDATES*. + +Subsequent fixes and mitigation work will be queued in the mainline kernel +and CC'd to the stable trees as per the usual kernel development process. +Integrators are advised to track these trees as closely as possible. + +The patches here are intended to be specific to the arm/arm64 architectures +and therefore mitigation efforts elsewhere in the kernel (for example, in the +eBPF subsystem) are not included as part of this branch.