From patchwork Thu Mar 1 12:53:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130182 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2760382edc; Thu, 1 Mar 2018 04:55:34 -0800 (PST) X-Google-Smtp-Source: AG47ELtKbmZW/FM6nfoVi2rre7Wd/IvYUt+LRTi/CpUMbosOj40FDY7hhX4cvkn9W/7HdI0f2I1I X-Received: by 2002:a17:902:aa5:: with SMTP id 34-v6mr1857255plp.429.1519908934449; Thu, 01 Mar 2018 04:55:34 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519908934; cv=none; d=google.com; s=arc-20160816; b=kOMakxOmIREeg0uvBGXP781emio4kKTw35jseVedXRQdPWzxgeUPpZG6JrxeSfFS7j EBSh8DXq9FLc4HAOzk7FgZPUFcEiyDgOc0U9U/p6SFW+oYogz/tJFPC61GPXi6gwXIAx rXi4/mI/HhuowOhM1zuwGAYfm3gtM0NTm0uFyIjZaBBF2o36OWojzyrdFc4uaG1fgrVV 8RNbwiHbaFzLb25VE/BrVjXdYNBC8PQESe+Pgzs6RygEa7ZonnuGC0n4asvfMjL2moAU Q58U/sQE497c2j7xQheiP99x/jlH3Cu0ww326TqjjAlz15tjXTIoGmMUGpvMUQVr4dB6 OWJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=GWUSuhRz/PUASjkFIqTWrSa96WhoDAqhFr+yeOukJXE=; b=HE2ZVG56R3QNrXLLI9MJC2hM5AaFCJBsZo96oZFKOQuVd+px9S1uydNaVMqerKmqy0 dSTI5T8zVDlO44sTFqiO4R1GxG361+aVYqbWPw7Gi4VDM1bVhEELfEDNX0pzdIYvaX4f GqDrWNpoQjVF0sOX0ogaMpFoggW/ahgwSntzMOsNGsTlxciK3DyrPe2RkRNhH01qH8PT nEkPmc2SywDM7y5VvCGP55eouX++iFFl7x123SKsQPSFn/xwpMQwbimbPotdW5vg1lEW eSmeZLxLnZ9BqCFfgF6wxruCaxV17e0512Pn5UN/t/8vKo+gDWZzBGiqHzsEenZvg3Td 7Erg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gPAwTb9+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g72si2998752pfd.2.2018.03.01.04.55.34; Thu, 01 Mar 2018 04:55:34 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gPAwTb9+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030616AbeCAMz3 (ORCPT + 28 others); Thu, 1 Mar 2018 07:55:29 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:45918 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030549AbeCAMzZ (ORCPT ); Thu, 1 Mar 2018 07:55:25 -0500 Received: by mail-pf0-f193.google.com with SMTP id h19so1517956pfd.12 for ; Thu, 01 Mar 2018 04:55:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=GWUSuhRz/PUASjkFIqTWrSa96WhoDAqhFr+yeOukJXE=; b=gPAwTb9+u4UmqxN6cbplBWZjmmZnXrijzuHVUDHlI2Zo5MMUlU+ka/izwCLmOH/3h2 hxgVD2QonplV/SnjgIXF7nvzzoq4xHDeEnNzLQMMiwaffviXUoPCckHLV1a+SzwNPDyl kaevhp/MoL77LQqKTryn4jiAwOPuW7C6WDoN4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=GWUSuhRz/PUASjkFIqTWrSa96WhoDAqhFr+yeOukJXE=; b=kNMXLwt2cmkZxzducqzTTpozyId9l9bR0cwWBmAGsbW+Mro1iYe+OGwAY49HbqBvDN eA6KGiS8Dl4Nt9/A+qJTQYhv7v4beh+U1iSKULBltB6DPCKXVGckL2iMT0csOnpl5Nbv Vi84PRbyDrfAFXNTF7Rfp/resYDkGcHseB0diZV7xyjhjQHNiNRSbdgtmC/9b3gE6w5r krRcudg/JKl+1PLm+2A7kw0DF5TV5YNwCu3SdgPQOzNaSYUYEu70t1w4eeLfRPl9RpNz kl65RKygALEbfLFPA5JyBNvaoOVWx4KniJt1fouW3Hl1WVve0mGeGVeSgLbXzpKjVret z12w== X-Gm-Message-State: APf1xPD/cYMDWctsh9CFrhsMIYpZ/wTNB8512iCF5ZjuP4i+k+PNrzqn 8Z8YDfKcsQj/p0g9DH8iKd7oTQ== X-Received: by 10.99.4.66 with SMTP id 63mr1472139pge.93.1519908923913; Thu, 01 Mar 2018 04:55:23 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.55.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:55:23 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 02/45] arm64: alternatives: apply boot time fixups via the linear mapping Date: Thu, 1 Mar 2018 20:53:39 +0800 Message-Id: <1519908862-11425-3-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Ard Biesheuvel commit 5ea5306c323 upstream. One important rule of thumb when desiging a secure software system is that memory should never be writable and executable at the same time. We mostly adhere to this rule in the kernel, except at boot time, when regions may be mapped RWX until after we are done applying alternatives or making other one-off changes. For the alternative patching, we can improve the situation by applying the fixups via the linear mapping, which is never mapped with executable permissions. So map the linear alias of .text with RW- permissions initially, and remove the write permissions as soon as alternative patching has completed. Reviewed-by: Laura Abbott Reviewed-by: Mark Rutland Tested-by: Mark Rutland Signed-off-by: Ard Biesheuvel Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/mmu.h | 1 + arch/arm64/kernel/alternative.c | 11 ++++++----- arch/arm64/kernel/smp.c | 1 + arch/arm64/mm/mmu.c | 22 +++++++++++++++++----- 4 files changed, 25 insertions(+), 10 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 8d9fce0..b075140 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -36,5 +36,6 @@ extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, bool allow_block_mappings); extern void *fixmap_remap_fdt(phys_addr_t dt_phys); +extern void mark_linear_text_alias_ro(void); #endif diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index 06d650f..8840c10 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -105,11 +105,11 @@ static u32 get_alt_insn(struct alt_instr *alt, u32 *insnptr, u32 *altinsnptr) return insn; } -static void __apply_alternatives(void *alt_region) +static void __apply_alternatives(void *alt_region, bool use_linear_alias) { struct alt_instr *alt; struct alt_region *region = alt_region; - u32 *origptr, *replptr; + u32 *origptr, *replptr, *updptr; for (alt = region->begin; alt < region->end; alt++) { u32 insn; @@ -124,11 +124,12 @@ static void __apply_alternatives(void *alt_region) origptr = ALT_ORIG_PTR(alt); replptr = ALT_REPL_PTR(alt); + updptr = use_linear_alias ? (u32 *)lm_alias(origptr) : origptr; nr_inst = alt->alt_len / sizeof(insn); for (i = 0; i < nr_inst; i++) { insn = get_alt_insn(alt, origptr + i, replptr + i); - *(origptr + i) = cpu_to_le32(insn); + updptr[i] = cpu_to_le32(insn); } flush_icache_range((uintptr_t)origptr, @@ -155,7 +156,7 @@ static int __apply_alternatives_multi_stop(void *unused) isb(); } else { BUG_ON(patched); - __apply_alternatives(®ion); + __apply_alternatives(®ion, true); /* Barriers provided by the cache flushing */ WRITE_ONCE(patched, 1); } @@ -176,5 +177,5 @@ void apply_alternatives(void *start, size_t length) .end = start + length, }; - __apply_alternatives(®ion); + __apply_alternatives(®ion, false); } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index a70f7d3..66db515 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -426,6 +426,7 @@ void __init smp_cpus_done(unsigned int max_cpus) setup_cpu_features(); hyp_mode_check(); apply_alternatives_all(); + mark_linear_text_alias_ro(); } void __init smp_prepare_boot_cpu(void) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d5cc6d7..5dc72c0 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -354,16 +354,28 @@ static void __init __map_memblock(pgd_t *pgd, phys_addr_t start, phys_addr_t end !debug_pagealloc_enabled()); /* - * Map the linear alias of the [_text, __init_begin) interval as - * read-only/non-executable. This makes the contents of the - * region accessible to subsystems such as hibernate, but - * protects it from inadvertent modification or execution. + * Map the linear alias of the [_text, __init_begin) interval + * as non-executable now, and remove the write permission in + * mark_linear_text_alias_ro() below (which will be called after + * alternative patching has completed). This makes the contents + * of the region accessible to subsystems such as hibernate, + * but protects it from inadvertent modification or execution. */ __create_pgd_mapping(pgd, kernel_start, __phys_to_virt(kernel_start), - kernel_end - kernel_start, PAGE_KERNEL_RO, + kernel_end - kernel_start, PAGE_KERNEL, early_pgtable_alloc, !debug_pagealloc_enabled()); } +void __init mark_linear_text_alias_ro(void) +{ + /* + * Remove the write permissions from the linear alias of .text/.rodata + */ + create_mapping_late(__pa_symbol(_text), (unsigned long)lm_alias(_text), + (unsigned long)__init_begin - (unsigned long)_text, + PAGE_KERNEL_RO); +} + static void __init map_mem(pgd_t *pgd) { struct memblock_region *reg; From patchwork Thu Mar 1 12:53:40 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130183 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2760466edc; Thu, 1 Mar 2018 04:55:39 -0800 (PST) X-Google-Smtp-Source: AG47ELvgjvmNidzZmifrJ9IR/DncX+pB0kCgKEWdXKrw/U88XYeKoKd9r1FkbgCl0kCjW1R5Yahb X-Received: by 10.99.96.66 with SMTP id u63mr1508497pgb.49.1519908939605; Thu, 01 Mar 2018 04:55:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519908939; cv=none; d=google.com; s=arc-20160816; b=nNs4wDDNTEmjocRDTqYd8wMHYB9dnoh9OwePKBqFl6FEsXRwF6lOeagob2T4DVm8OT yYqzcagu4KsrnXp4bhBiQaBCMnA4+3W+Px3dp3j38I1RJirjy7rRR/El8Y4ypCh6U3kR idcBnmCHllOMnhtZl5VvVUrv1kqv2Z7aJkp5A538XQ0Kb3qGaxfEUHc3eri6JrjytO8E 7tumSp01QEon36sFGcj7Ky9Ck+bHsxvioOjK9loZP3lZbfXTJFn1jEFadNmt/R24lgmi wfaCKN4za6B/Qbw2bQSvQ+TULAf3YFNqL9YaYnXQbBjmDf9eSmYJT77L8B6MzlEmw3YU ZfWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=nelMizWy+r+xl1PA+JMazlupbHfphoojSqCggA1RmD0=; b=y7qFhCG2mLhU9k1mGSv5J8zZ0oZvYM+MkcE8QJSpBo0og7nkPeMrdwKp1hJQ3siE5M KXrcVx4v5ey+malZlRWei3G0atBPImgY6po1ZbucMslAu2cpjrmx//R5VHsF7fg9ix8V siR5ZYbXrcWwDB2n9V2+1H6Tm0svLAdHbfA5R/o0mJxQkj+ARIbs2Y0WpFcXRmYZaQR6 xVix4LES8ZCML0hzDPhPHh44FTOapfAscCWyr/Ku2npPPi9vhpqXv4sq/Ilz+UMQRmOA i98V8bRZOGG64PgV0F7Gw9dKtmjXGe32+PUSIfItrzySHQRMGq82q1AqqFurkOvuDbz8 wiOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=e66C1+HX; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c10si2454231pgf.230.2018.03.01.04.55.39; Thu, 01 Mar 2018 04:55:39 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=e66C1+HX; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030631AbeCAMzg (ORCPT + 28 others); Thu, 1 Mar 2018 07:55:36 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:43043 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030618AbeCAMzc (ORCPT ); Thu, 1 Mar 2018 07:55:32 -0500 Received: by mail-pl0-f67.google.com with SMTP id f23-v6so3581571plr.10 for ; Thu, 01 Mar 2018 04:55:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nelMizWy+r+xl1PA+JMazlupbHfphoojSqCggA1RmD0=; b=e66C1+HXHngXFMiUjstd+ocmupaazHCvdOBrdx7qoT6x8ORTs1kkR6JPnDxe8n0KC+ sESV0oVwuXo3a8YDGftEZzM9SGaGihMBRJce6ENYnaYpcRuqzRNl3gsMz1aY0qXcbqcP NaTqD67Sj3KeLfEDgnHs6SqsFBVSn4cMevd/Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nelMizWy+r+xl1PA+JMazlupbHfphoojSqCggA1RmD0=; b=IuMuVG6XpYp2u6cu4xRkb/zMslSO3L28BueN0iy5hyVcwreY/R8Wm9WyeyNW40VfG9 vM3A7MqXq8NcNPf4lRDrjrHLxDaLX8CrxeX0h5hO0v5btX3ryjpcvVkNsOvMCYM0/v3+ SD6sVQTrD92uPwnk/jomO6G/wsyFxvvGPx4iNaj0hv1a7dXnpDg9I13fDkwKVORBEN3P GmRkwF2QBzTo3QQsQMGIMF0f5rVDohjAFvuMlCO2CChPTjhtYxEhLDo0KhuIaGvCpgVR X/yuQPrYWCRLsA9zlk6eDb+NF/cgsMreY6DrYCzn3nyVzfMMryDUXf/4KZKk4j/gm3nL 964g== X-Gm-Message-State: APf1xPCNq19vIzAQiyEzRjvwQim3pobJD1sNFYvBK2NQNcAQmBG2bdNC w4FopmUGSBLefKV3h7z0srhM4w== X-Received: by 2002:a17:902:22f:: with SMTP id 44-v6mr1835471plc.418.1519908931827; Thu, 01 Mar 2018 04:55:31 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.55.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:55:31 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 03/45] arm64: barrier: Add CSDB macros to control data-value prediction Date: Thu, 1 Mar 2018 20:53:40 +0800 Message-Id: <1519908862-11425-4-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 669474e772b9 upstream. For CPUs capable of data value prediction, CSDB waits for any outstanding predictions to architecturally resolve before allowing speculative execution to continue. Provide macros to expose it to the arch code. Reviewed-by: Mark Rutland Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/assembler.h | 7 +++++++ arch/arm64/include/asm/barrier.h | 2 ++ 2 files changed, 9 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 851290d..8760300 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -87,6 +87,13 @@ .endm /* + * Value prediction barrier + */ + .macro csdb + hint #20 + .endm + +/* * NOP sequence */ .macro nops, num diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index 0fe7e43..c68fdc5 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -31,6 +31,8 @@ #define dmb(opt) asm volatile("dmb " #opt : : : "memory") #define dsb(opt) asm volatile("dsb " #opt : : : "memory") +#define csdb() asm volatile("hint #20" : : : "memory") + #define mb() dsb(sy) #define rmb() dsb(ld) #define wmb() dsb(st) From patchwork Thu Mar 1 12:53:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130184 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2760559edc; Thu, 1 Mar 2018 04:55:46 -0800 (PST) X-Google-Smtp-Source: AG47ELvRjYifAMTSBczhImcKIzgjcGL+jRDvgPkZ8gv68hPW+e9KDcyWu+I89RKhIvIh8eo1XYCM X-Received: by 10.99.110.5 with SMTP id j5mr1458659pgc.65.1519908946004; Thu, 01 Mar 2018 04:55:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519908945; cv=none; d=google.com; s=arc-20160816; b=Tau/PzApld6rgCofLxwv6AEV3EYy6JjKTsMHqqOgWQIGMETL8Pbsl3aNM2dFuQrib6 3co7B8i7Jj+/CdI5wAJAoMdQE2QYlTiM1wQkTijMHDHLWcJUtxRg8D02kAD5ZO+RiTyP tVpRMoeoFM5w0QQxlG0XP8BZaSPsvuQ16iLrKiH/zFHqVq6QEUixPPxymIQt0TW0fBN7 tt3Ybo8EfOPhcjVDy7qN7NIAsZh47cn+KQ9NcTPaa6uk9fgK7Ha/UwUn1FI8mOVH/uDq 6k4tgZgrsxpDrjv2DwGzmn9fURZ6XOfNvSMzAY3L8fGR1LRKH63h46ki6XFwFwuRIlqf 7pKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=1urEszs1X1+fSgF2FtrvQb+AjsP1I5AehCNh9xZoYO0=; b=gMrfI1KWenz4ZjTyuwQQ91lYn40yaAJcRC0Hu1UQy+3cPrLmm/LECkiyPi539B9leo JLPJ98F0kUPk96KT3dPEPGqLKZfkFgoXG/H3m9MgQl1BHFLbmhdQoeHkD7DuvCL8tMhU Q3utfT9+tTwCicYlSyHUglh9IrkT7IxZgfY28h+8EviUAJjHO/9e2Jys4N6Q4RY7DqvS XPgnK8c/37R7uxF8PakOjsAV+qpeq6NGV1XFiPGKHt0jLAiT/X1ut6YrYyaurrOf+H79 nVR3OQjb/NJ0BmMpn7Fs8dx51QcZsz6AICPj69gIZyUbMpvHpYeibjxT6wFUDZKqBR1y sirQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Vm6pcX2U; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c10si2454231pgf.230.2018.03.01.04.55.45; Thu, 01 Mar 2018 04:55:45 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Vm6pcX2U; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030646AbeCAMzm (ORCPT + 28 others); Thu, 1 Mar 2018 07:55:42 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:32883 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030629AbeCAMzj (ORCPT ); Thu, 1 Mar 2018 07:55:39 -0500 Received: by mail-pl0-f65.google.com with SMTP id c11-v6so3602536plo.0 for ; Thu, 01 Mar 2018 04:55:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1urEszs1X1+fSgF2FtrvQb+AjsP1I5AehCNh9xZoYO0=; b=Vm6pcX2UzAbaWdmyVa0rPvXZyO2E8/n8CFR09KxGgLr7DE84EkFSOc58SyN7MgSzHs R140O1gA1/VhJMJRrfpYoHFqDGmBjG0KNF0Pz/9niTXSBo1iMAboRvBbN0DACyGbIVAP DuTd7fju1g+IeY/LScim2LVwWOpyXitzWnwIs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1urEszs1X1+fSgF2FtrvQb+AjsP1I5AehCNh9xZoYO0=; b=qzpoumiRtOtnJbFW8BfKHIz1EtNb/9Kkil/VwF7PicBWphvTGi0F1WLSjAPfAsl1l7 BKWUqHc/tWtxrK383f7RMpuCTFzfLA12BUR6rvd3j87TG6QCAKHphMEWGNRWtBX2yN6u em49STmEIuFZm315cTIbhufkAHKqRWUTPUn0Vd8NX8zwrX0vN2saGhSq6/ciChMrEvLH zL/NAYGGTJIdCmUhAbZJuxsWtJoOCWHm78BIsqBhnjPyAnE4el9/llVrUlZB6dh6JRvb TnUGZMlKFgRkwP3xR0kHzpqNB4GgduGSbIpAHkj0JidZON3J6wl6oG2Nut1VlThzhHEh RO6w== X-Gm-Message-State: APf1xPCmrZmKM5tEkHKeTCwnB9nITsT1cHi6JmL5FTrFgMknzb9ZzuOO 7sbxjSVPDZonVpKYhOZcTejqSA== X-Received: by 2002:a17:902:b605:: with SMTP id b5-v6mr1871768pls.354.1519908939113; Thu, 01 Mar 2018 04:55:39 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.55.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:55:38 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Robin Murphy , Alex Shi Subject: [PATCH 04/45] arm64: Implement array_index_mask_nospec() Date: Thu, 1 Mar 2018 20:53:41 +0800 Message-Id: <1519908862-11425-5-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Robin Murphy commit 022620eed3d0 upstream. Provide an optimised, assembly implementation of array_index_mask_nospec() for arm64 so that the compiler is not in a position to transform the code in ways which affect its ability to inhibit speculation (e.g. by introducing conditional branches). This is similar to the sequence used by x86, modulo architectural differences in the carry/borrow flags. Reviewed-by: Mark Rutland Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/barrier.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index c68fdc5..0b0755c 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -40,6 +40,27 @@ #define dma_rmb() dmb(oshld) #define dma_wmb() dmb(oshst) +/* + * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz + * and 0 otherwise. + */ +#define array_index_mask_nospec array_index_mask_nospec +static inline unsigned long array_index_mask_nospec(unsigned long idx, + unsigned long sz) +{ + unsigned long mask; + + asm volatile( + " cmp %1, %2\n" + " sbc %0, xzr, xzr\n" + : "=r" (mask) + : "r" (idx), "Ir" (sz) + : "cc"); + + csdb(); + return mask; +} + #define __smp_mb() dmb(ish) #define __smp_rmb() dmb(ishld) #define __smp_wmb() dmb(ishst) From patchwork Thu Mar 1 12:53:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130186 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2760817edc; Thu, 1 Mar 2018 04:56:04 -0800 (PST) X-Google-Smtp-Source: AG47ELtYx1nxaDRriQ9j/AerFYI6Vh6Uc1DmZrNrIqA834U/NWfqLSwLDlAKDzK5rBNbwkHc3wLn X-Received: by 10.99.110.131 with SMTP id j125mr1494115pgc.382.1519908964214; Thu, 01 Mar 2018 04:56:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519908964; cv=none; d=google.com; s=arc-20160816; b=eQQg6WP/4FL68uZOPUzKioVZ2UWn0jBmeAsRxQJ3NkfRI2u+lmB0IRAcCSTRkilswR WmeK/twX5nFxb+xzB0mHuvKbHVgP/tuoQcocl+0xGlPbIGaGX0uC9WI8Jav5xB205l0A beeozKwcwO2as6chaFHGUaxphX7YmnHdOv84pfP4F5Mkv/njwjif0LdPIHXx6ouyNFTj sNTmAIaa3yfy2qKUM/51CbFOmlSOXHbREB8ub6lXhAuufY15H1u9ma9+waPaubk7u/KP DItjsq7p7cxRgAguSkhCzXMp6yPdUIihvmlN2xYCWS7Q/Qr6XUuEPHuqaHVBrzLo6zU4 P5pQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=DuN6SyTg5l99yxUh5yRn/aUeDXw1S1qe26nBS+h01No=; b=pEyGHUGnEg0BTbHkwXuQcIfMhhQ8Rnj9kqEg0Cg/9mXbJvgwREi8Jn/LogS0EItg1B ZOH4pAG735yyJQNUBymeE/bRHmdFQR54OTgAgaqq/4XFzdgSnzEWFPENSsHOxGvzgT8J 1M6gIoqksPEACF8FB8jCv6pHPf/YAf0hPPJKvwI4KNjsFRtefD0P8/ziQYy2lSsmM8W4 WE44jAbjJJC372z1BUbHyKqQVvMmYXGFRp3MzjTACG9T5og9tsPxGEUwf+WyFrY9iUr8 BeRryTLsSPUvwikGOx8k2sFipz21dcVKYga0ubHoECepRRnd3atdYkv+B9/NRi5MMQQV uPYQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=H7HFHdWR; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z60-v6si3059575plh.306.2018.03.01.04.56.03; Thu, 01 Mar 2018 04:56:04 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=H7HFHdWR; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030679AbeCAM4B (ORCPT + 28 others); Thu, 1 Mar 2018 07:56:01 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:33460 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030664AbeCAMz6 (ORCPT ); Thu, 1 Mar 2018 07:55:58 -0500 Received: by mail-pf0-f195.google.com with SMTP id q13so2409822pff.0 for ; Thu, 01 Mar 2018 04:55:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DuN6SyTg5l99yxUh5yRn/aUeDXw1S1qe26nBS+h01No=; b=H7HFHdWRgsrd09iiWTHOVQwgyZGk+rzsk8cFJcaxpYImvt/zRIkkzC+1SyWEeWVS3G 32QE/fuJ5Mth1RH8UQUOwpCoI4lHUceSxp/4h99ehcfMUZmvuq+P2SkNysd1hSSblSAb OT6uuLQ6D6lKP3ZEUcPDl8ST2/LPpvIff9yyM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DuN6SyTg5l99yxUh5yRn/aUeDXw1S1qe26nBS+h01No=; b=ngNmUTtwitwlSMGGXJK59iRRk+FVVtdi6zu7v+F6RC/M+JkT9WPMcehIFYilhtY+GR vVZcMcrO7U6VQQX4WWmB7vgrKkEzNAHhLgGh3xXgioDL0mUO39Ennc1lacUklBj7VYAQ TatV4sbCj3TN45eyLNqpJ9E2JXY+zxH5H7pVqjaZKyrrOQ63pt5c20ojVg4s6y0nW9u3 OUqoJo/61b4xV/zcdQqJcLwgQeq32FCX2BfM1pWBazevrOiTe+uLlgPEONo1FrK2dm2K bSlMK526A3tz33o9Wx3LTqibwTYz0bA98GDnbg2uo7lGJ22nGng+YLmgJ4vmviKXN4uJ aS1g== X-Gm-Message-State: APf1xPAqS0kByJRnzxSovxSDDzTNfg/KPWjOICpOVeuvqTHjg/ETuQOI O9ELZMZgS0VjI3IITRoSy/YXjw== X-Received: by 10.99.178.6 with SMTP id x6mr1524185pge.98.1519908957462; Thu, 01 Mar 2018 04:55:57 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.55.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:55:56 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: James Morse , Kees Cook , Alex Shi Subject: [PATCH 06/45] arm64: Factor out PAN enabling/disabling into separate uaccess_* macros Date: Thu, 1 Mar 2018 20:53:43 +0800 Message-Id: <1519908862-11425-7-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Catalin Marinas commit bd38967d406 upstream. This patch moves the directly coded alternatives for turning PAN on/off into separate uaccess_{enable,disable} macros or functions. The asm macros take a few arguments which will be used in subsequent patches. Note that any (unlikely) access that the compiler might generate between uaccess_enable() and uaccess_disable(), other than those explicitly specified by the user access code, will not be protected by PAN. Cc: Will Deacon Cc: James Morse Cc: Kees Cook Reviewed-by: Mark Rutland Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/futex.h | 17 ++++---- arch/arm64/include/asm/uaccess.h | 79 +++++++++++++++++++++++++++++++----- arch/arm64/kernel/armv8_deprecated.c | 11 +++-- arch/arm64/lib/clear_user.S | 11 ++--- arch/arm64/lib/copy_from_user.S | 11 ++--- arch/arm64/lib/copy_in_user.S | 11 ++--- arch/arm64/lib/copy_to_user.S | 11 ++--- 7 files changed, 93 insertions(+), 58 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index f2585cd..85c4a89 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -21,15 +21,12 @@ #include #include -#include -#include #include -#include #define __futex_atomic_op(insn, ret, oldval, uaddr, tmp, oparg) \ +do { \ + uaccess_enable(); \ asm volatile( \ - ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, \ - CONFIG_ARM64_PAN) \ " prfm pstl1strm, %2\n" \ "1: ldxr %w1, %2\n" \ insn "\n" \ @@ -44,11 +41,11 @@ " .popsection\n" \ _ASM_EXTABLE(1b, 4b) \ _ASM_EXTABLE(2b, 4b) \ - ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \ - CONFIG_ARM64_PAN) \ : "=&r" (ret), "=&r" (oldval), "+Q" (*uaddr), "=&r" (tmp) \ : "r" (oparg), "Ir" (-EFAULT) \ - : "memory") + : "memory"); \ + uaccess_disable(); \ +} while (0) static inline int futex_atomic_op_inuser (int encoded_op, u32 __user *uaddr) @@ -118,8 +115,8 @@ futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, if (!access_ok(VERIFY_WRITE, uaddr, sizeof(u32))) return -EFAULT; + uaccess_enable(); asm volatile("// futex_atomic_cmpxchg_inatomic\n" -ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, CONFIG_ARM64_PAN) " prfm pstl1strm, %2\n" "1: ldxr %w1, %2\n" " sub %w3, %w1, %w4\n" @@ -134,10 +131,10 @@ ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, CONFIG_ARM64_PAN) " .popsection\n" _ASM_EXTABLE(1b, 4b) _ASM_EXTABLE(2b, 4b) -ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, CONFIG_ARM64_PAN) : "+r" (ret), "=&r" (val), "+Q" (*uaddr), "=&r" (tmp) : "r" (oldval), "r" (newval), "Ir" (-EFAULT) : "memory"); + uaccess_disable(); *uval = val; return ret; diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 811cf16..94e1457 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -18,6 +18,11 @@ #ifndef __ASM_UACCESS_H #define __ASM_UACCESS_H +#include +#include + +#ifndef __ASSEMBLY__ + /* * User space memory access functions */ @@ -26,10 +31,8 @@ #include #include -#include #include #include -#include #include #include #include @@ -121,6 +124,44 @@ static inline void set_fs(mm_segment_t fs) " .popsection\n" /* + * User access enabling/disabling. + */ +#define __uaccess_disable(alt) \ +do { \ + asm(ALTERNATIVE("nop", SET_PSTATE_PAN(1), alt, \ + CONFIG_ARM64_PAN)); \ +} while (0) + +#define __uaccess_enable(alt) \ +do { \ + asm(ALTERNATIVE("nop", SET_PSTATE_PAN(0), alt, \ + CONFIG_ARM64_PAN)); \ +} while (0) + +static inline void uaccess_disable(void) +{ + __uaccess_disable(ARM64_HAS_PAN); +} + +static inline void uaccess_enable(void) +{ + __uaccess_enable(ARM64_HAS_PAN); +} + +/* + * These functions are no-ops when UAO is present. + */ +static inline void uaccess_disable_not_uao(void) +{ + __uaccess_disable(ARM64_ALT_PAN_NOT_UAO); +} + +static inline void uaccess_enable_not_uao(void) +{ + __uaccess_enable(ARM64_ALT_PAN_NOT_UAO); +} + +/* * The "__xxx" versions of the user access functions do not verify the address * space - it must have been done previously with a separate "access_ok()" * call. @@ -147,8 +188,7 @@ static inline void set_fs(mm_segment_t fs) do { \ unsigned long __gu_val; \ __chk_user_ptr(ptr); \ - asm(ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_ALT_PAN_NOT_UAO,\ - CONFIG_ARM64_PAN)); \ + uaccess_enable_not_uao(); \ switch (sizeof(*(ptr))) { \ case 1: \ __get_user_asm("ldrb", "ldtrb", "%w", __gu_val, (ptr), \ @@ -169,9 +209,8 @@ do { \ default: \ BUILD_BUG(); \ } \ + uaccess_disable_not_uao(); \ (x) = (__force __typeof__(*(ptr)))__gu_val; \ - asm(ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_ALT_PAN_NOT_UAO,\ - CONFIG_ARM64_PAN)); \ } while (0) #define __get_user(x, ptr) \ @@ -216,8 +255,7 @@ do { \ do { \ __typeof__(*(ptr)) __pu_val = (x); \ __chk_user_ptr(ptr); \ - asm(ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_ALT_PAN_NOT_UAO,\ - CONFIG_ARM64_PAN)); \ + uaccess_enable_not_uao(); \ switch (sizeof(*(ptr))) { \ case 1: \ __put_user_asm("strb", "sttrb", "%w", __pu_val, (ptr), \ @@ -238,8 +276,7 @@ do { \ default: \ BUILD_BUG(); \ } \ - asm(ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_ALT_PAN_NOT_UAO,\ - CONFIG_ARM64_PAN)); \ + uaccess_disable_not_uao(); \ } while (0) #define __put_user(x, ptr) \ @@ -332,4 +369,26 @@ extern long strncpy_from_user(char *dest, const char __user *src, long count); extern __must_check long strlen_user(const char __user *str); extern __must_check long strnlen_user(const char __user *str, long n); +#else /* __ASSEMBLY__ */ + +#include + +/* + * User access enabling/disabling macros. These are no-ops when UAO is + * present. + */ + .macro uaccess_disable_not_uao, tmp1 +alternative_if ARM64_ALT_PAN_NOT_UAO + SET_PSTATE_PAN(1) +alternative_else_nop_endif + .endm + + .macro uaccess_enable_not_uao, tmp1, tmp2 +alternative_if ARM64_ALT_PAN_NOT_UAO + SET_PSTATE_PAN(0) +alternative_else_nop_endif + .endm + +#endif /* __ASSEMBLY__ */ + #endif /* __ASM_UACCESS_H */ diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c index c0ede23..29d2ad8 100644 --- a/arch/arm64/kernel/armv8_deprecated.c +++ b/arch/arm64/kernel/armv8_deprecated.c @@ -14,7 +14,6 @@ #include #include -#include #include #include #include @@ -285,10 +284,10 @@ static void __init register_insn_emulation_sysctl(struct ctl_table *table) #define __SWP_LL_SC_LOOPS 4 #define __user_swpX_asm(data, addr, res, temp, temp2, B) \ +do { \ + uaccess_enable(); \ __asm__ __volatile__( \ " mov %w3, %w7\n" \ - ALTERNATIVE("nop", SET_PSTATE_PAN(0), ARM64_HAS_PAN, \ - CONFIG_ARM64_PAN) \ "0: ldxr"B" %w2, [%4]\n" \ "1: stxr"B" %w0, %w1, [%4]\n" \ " cbz %w0, 2f\n" \ @@ -306,13 +305,13 @@ static void __init register_insn_emulation_sysctl(struct ctl_table *table) " .popsection" \ _ASM_EXTABLE(0b, 4b) \ _ASM_EXTABLE(1b, 4b) \ - ALTERNATIVE("nop", SET_PSTATE_PAN(1), ARM64_HAS_PAN, \ - CONFIG_ARM64_PAN) \ : "=&r" (res), "+r" (data), "=&r" (temp), "=&r" (temp2) \ : "r" ((unsigned long)addr), "i" (-EAGAIN), \ "i" (-EFAULT), \ "i" (__SWP_LL_SC_LOOPS) \ - : "memory") + : "memory"); \ + uaccess_disable(); \ +} while (0) #define __user_swp_asm(data, addr, res, temp, temp2) \ __user_swpX_asm(data, addr, res, temp, temp2, "") diff --git a/arch/arm64/lib/clear_user.S b/arch/arm64/lib/clear_user.S index 5d1cad3..d7150e3 100644 --- a/arch/arm64/lib/clear_user.S +++ b/arch/arm64/lib/clear_user.S @@ -17,10 +17,7 @@ */ #include -#include -#include -#include -#include +#include .text @@ -33,8 +30,7 @@ * Alignment fixed up by hardware. */ ENTRY(__clear_user) -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_enable_not_uao x2, x3 mov x2, x1 // save the size for fixup return subs x1, x1, #8 b.mi 2f @@ -54,8 +50,7 @@ uao_user_alternative 9f, strh, sttrh, wzr, x0, 2 b.mi 5f uao_user_alternative 9f, strb, sttrb, wzr, x0, 0 5: mov x0, #0 -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_disable_not_uao x2 ret ENDPROC(__clear_user) diff --git a/arch/arm64/lib/copy_from_user.S b/arch/arm64/lib/copy_from_user.S index 4fd67ea..cfe1339 100644 --- a/arch/arm64/lib/copy_from_user.S +++ b/arch/arm64/lib/copy_from_user.S @@ -16,11 +16,8 @@ #include -#include -#include #include -#include -#include +#include /* * Copy from user space to a kernel buffer (alignment handled by the hardware) @@ -67,12 +64,10 @@ end .req x5 ENTRY(__arch_copy_from_user) -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_enable_not_uao x3, x4 add end, x0, x2 #include "copy_template.S" -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_disable_not_uao x3 mov x0, #0 // Nothing to copy ret ENDPROC(__arch_copy_from_user) diff --git a/arch/arm64/lib/copy_in_user.S b/arch/arm64/lib/copy_in_user.S index f7292dd0..718b1c4 100644 --- a/arch/arm64/lib/copy_in_user.S +++ b/arch/arm64/lib/copy_in_user.S @@ -18,11 +18,8 @@ #include -#include -#include #include -#include -#include +#include /* * Copy from user space to user space (alignment handled by the hardware) @@ -68,12 +65,10 @@ end .req x5 ENTRY(__copy_in_user) -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_enable_not_uao x3, x4 add end, x0, x2 #include "copy_template.S" -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_disable_not_uao x3 mov x0, #0 ret ENDPROC(__copy_in_user) diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S index 7a7efe2..e99e31c 100644 --- a/arch/arm64/lib/copy_to_user.S +++ b/arch/arm64/lib/copy_to_user.S @@ -16,11 +16,8 @@ #include -#include -#include #include -#include -#include +#include /* * Copy to user space from a kernel buffer (alignment handled by the hardware) @@ -66,12 +63,10 @@ end .req x5 ENTRY(__arch_copy_to_user) -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(0)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_enable_not_uao x3, x4 add end, x0, x2 #include "copy_template.S" -ALTERNATIVE("nop", __stringify(SET_PSTATE_PAN(1)), ARM64_ALT_PAN_NOT_UAO, \ - CONFIG_ARM64_PAN) + uaccess_disable_not_uao x3 mov x0, #0 ret ENDPROC(__arch_copy_to_user) From patchwork Thu Mar 1 12:53:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130187 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2760952edc; Thu, 1 Mar 2018 04:56:14 -0800 (PST) X-Google-Smtp-Source: AG47ELtFuZ7I2x4cJOW9Otm3JMiqErPSYA+UeODD2sVx8tJHGu7+EWgA/Kykp2J2Td+WnQ8T+a8t X-Received: by 2002:a17:902:7a2:: with SMTP id 31-v6mr1840843plj.313.1519908974188; Thu, 01 Mar 2018 04:56:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519908974; cv=none; d=google.com; s=arc-20160816; b=Oi1vhBdonVmPytTTtavOzbMlVXvEWnpHUcIvAZ+8iTgLiPfVqtj1mxWJQwpCqzzJJv Yl2WjqHuV7AldbLzmE2H65qN4VrK0zywpcx+GU7uGzctvK6WkVXdcf60XjqzvoEgu4Fq gxNL1KH2uyidcBYHot+ActM0Ls0B7ij5dRUEIZQgXgKoyjz5KnYFxfPQWUBpBYJWz7LY gtZ0hWI+V1JdsCYEP1ANm50LlyYK+sWk3ULYqo59Wt1+7G5IpSdAZ2tfRNmxBs7XnWZ8 kjWQdbfVCBXb0Eg+rLp3+1iAB58FfSUeCtSVIjyQRydx0m1CpCM5zAtbLmwnwlhzzcBW SIqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=1Y8BanaJo1dWq0GIhEtFd/7ONzz9WFIJLPbGyXUfe10=; b=UQNdQlwQnuN07IWkGHfmuoXA792OsiDxxN9eYWJI2NC5VN5J730nBKZLegSF+ACUEs ozVBnF2I0tS+KUD93kC3nVxnDfJZWZLPt5dxsKKOteDOImBgqHea+BpLT4Eu+iRSg33C Vfk3y01qkWdVMe1/IxqjZ+b4CeCCuCkCpicnIpxfZyk5YQxWi1A9RWXG+3I2kYKO0wZq JIXBf09vokHIZBlux4MoCh2kPYB9O5cdDuKgxPkfpEdmznNugX01CDLyk7l+2ZQ+c0c7 ID3Srg0Joul1yTgsNEdxwN/ilCoIceSyzYBGvC8hFinXgRtTVKZiPoTObRqstpOSZaw2 CN2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iJNPwsm5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z60-v6si3059575plh.306.2018.03.01.04.56.13; Thu, 01 Mar 2018 04:56:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iJNPwsm5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030694AbeCAM4K (ORCPT + 28 others); Thu, 1 Mar 2018 07:56:10 -0500 Received: from mail-pl0-f66.google.com ([209.85.160.66]:36076 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030568AbeCAM4G (ORCPT ); Thu, 1 Mar 2018 07:56:06 -0500 Received: by mail-pl0-f66.google.com with SMTP id 61-v6so3593445plf.3 for ; Thu, 01 Mar 2018 04:56:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1Y8BanaJo1dWq0GIhEtFd/7ONzz9WFIJLPbGyXUfe10=; b=iJNPwsm5RYUN+7lvaEDqUyfARuBi++8KzenaenRDM7kaPMoFj481PAc6dhCteFwK1L LU561UBY1VHvfo3+VaVqVaKXGlrVVw8WVdDaVa2Gx4aCyiKQRq2yBmzA61CLX53af0Kv I5SiiqUZ+PUsF05KOXZQtAvslfxP4UGh7o+lM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1Y8BanaJo1dWq0GIhEtFd/7ONzz9WFIJLPbGyXUfe10=; b=OP/mbRQkjideO+3QjirgoIBpRy1sEeS8dcbGgjAMb5P30HATMV4C8ein3UaKREetcs uplQoWrFfThn1Ci3vPlpyg72ocAU470P5AyNPFoOxCLpvOGOI5LywXb1EdgtlU/rcUMa suOko61TkftDjK8B/HZ/MP4TVLu7ACft7w9JJl20GjIZZ5IMsfdalEVdn5UqikWoUL/k 6+VkTYT3QI9+PtYdU79lx5FWfho2UCeVYtd77Vhtnk+bl0yl6Wv8TQlGpdJG5+Wb4iHu 7MScsaWPd6NFgXG6ikdO+GHrv5ymx4LVdwN/8lP27y0Hl0Vt4MYnsAqsmgVYf0dwt++w GWpg== X-Gm-Message-State: APf1xPAfdeaqUOJp3fvkWXhLua3Uu/nXoU8TInGmW95tEljuxk650swU yTYFkrMYpVZJg6RT6eMcDjWBew== X-Received: by 2002:a17:902:b690:: with SMTP id c16-v6mr1888625pls.264.1519908966243; Thu, 01 Mar 2018 04:56:06 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.55.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:56:05 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: James Morse , Kees Cook , Alex Shi Subject: [PATCH 07/45] arm64: Factor out TTBR0_EL1 post-update workaround into a specific asm macro Date: Thu, 1 Mar 2018 20:53:44 +0800 Message-Id: <1519908862-11425-8-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Catalin Marinas commit f33bcf03e6 upstream This patch takes the errata workaround code out of cpu_do_switch_mm into a dedicated post_ttbr0_update_workaround macro which will be reused in a subsequent patch. Cc: Will Deacon Cc: James Morse Cc: Kees Cook Reviewed-by: Mark Rutland Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/assembler.h | 13 +++++++++++++ arch/arm64/mm/proc.S | 6 +----- 2 files changed, 14 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 8760300..ab3a179 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -420,4 +420,17 @@ alternative_endif movk \reg, :abs_g0_nc:\val .endm +/* + * Errata workaround post TTBR0_EL1 update. + */ + .macro post_ttbr0_update_workaround +#ifdef CONFIG_CAVIUM_ERRATUM_27456 +alternative_if ARM64_WORKAROUND_CAVIUM_27456 + ic iallu + dsb nsh + isb +alternative_else_nop_endif +#endif + .endm + #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 352c73b..c2adb0c 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -136,11 +136,7 @@ ENTRY(cpu_do_switch_mm) bfi x0, x1, #48, #16 // set the ASID msr ttbr0_el1, x0 // set TTBR0 isb -alternative_if ARM64_WORKAROUND_CAVIUM_27456 - ic iallu - dsb nsh - isb -alternative_else_nop_endif + post_ttbr0_update_workaround ret ENDPROC(cpu_do_switch_mm) From patchwork Thu Mar 1 12:53:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130188 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2761108edc; Thu, 1 Mar 2018 04:56:22 -0800 (PST) X-Google-Smtp-Source: AG47ELvCxmkbTQIAcN08jAm1h+DXvKS1e46NkVsWhECnyUoB7kW7XbkzWv+AcOli+51Eb6YEtDlt X-Received: by 10.98.171.24 with SMTP id p24mr1852406pff.71.1519908982765; Thu, 01 Mar 2018 04:56:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519908982; cv=none; d=google.com; s=arc-20160816; b=Qd7eHI1WofFuxBv2WBBqvUPEecXnxM21R7ogiDC/TA5zy54TnS2AVX03o6wzdWSGSz WjBGYKECvLUP+z7OLr1zebdIdwvq2sWP59cKPfB29TKQ46zWUc3PK9USzVCIGtKuX0SL kpobUkOC7q2bvS0OUMBlRJUIi8p9ScJ3BxpnQZAMf+McnB1WfuGsVauF3qySYQgCuZgt bZG3CPWRL8nPMWwlYl+P2LgJDcFK+yefsJgdd1XhOlgc8gEAXMa670zz7oNI514fixJV fX6RyX3yBMQIp7Pb57afTxIEhV7PcRL4xTUBYuXk+pzwIrj946JKqo8NWJVomhJ5wa96 d+0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=faQj8JqC4/KwGNZMuLXKQ1k5PdvnOQTh5XG4tCkdUZY=; b=fRR+fi6vDrrrRG/mXh9hiRouLK7GfuG9ys5ndMA3MdZNOlOZUph0Nbg63mgFSpkndQ QJhTMxXiuVcYsJkdCSwLuZyPmf5wq5dvOReYEw01XGqjrjYGMe3A220V+D+hSu3TuIH0 p8WO4cyIWXUx4t+vwOqnHF2v1xVFNejyXfSZrVm0GwfRzCHSAG8O7xUy+VQXsjwi874X JYkRsHUSiy1zESmBRBN9JJmWUFJtBfvd/96s0DniGc2Ywf9A+GAhQqbNxyisQa0Uv1GO 1gZUv9tey3UyDLeuu17/avXFsPel66OlkPqmWLcppgsx7uMmTXn7ibJ6f0cDMZAMKZxs zd1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VwStRMao; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v17si2956195pfe.186.2018.03.01.04.56.22; Thu, 01 Mar 2018 04:56:22 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VwStRMao; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030709AbeCAM4S (ORCPT + 28 others); Thu, 1 Mar 2018 07:56:18 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:43108 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030578AbeCAM4O (ORCPT ); Thu, 1 Mar 2018 07:56:14 -0500 Received: by mail-pl0-f68.google.com with SMTP id f23-v6so3582541plr.10 for ; Thu, 01 Mar 2018 04:56:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=faQj8JqC4/KwGNZMuLXKQ1k5PdvnOQTh5XG4tCkdUZY=; b=VwStRMaoDqp7alKKuHhPa/i2UlOlBpXVz03G+MB0JFEB9uQOWyfUMOW9ERCIBMMf1v HJ9vUCLKrPJHtgFA1H6fxNoUV1/dTPWaX9JYCRHo0F8dZOLndhMOK+uqDP/dOc0/Ssez Eb7ZCCCV9r9VdZXGQCOOta58FxQWBWGTAejpE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=faQj8JqC4/KwGNZMuLXKQ1k5PdvnOQTh5XG4tCkdUZY=; b=mv0D34gwtCNQ8vr5YUfWa2ZH5Jn6kfwwt0YGxPeq0pzak9cukMtBPqFAcHhm5Q9qGD 4vzaRC6ykisYyBl5PsO3bAF9CsObTrmAjMFKjlHQFsyGvwIP/fNRzWqDZWgL2VrHyWrb bfl/PGC9kRKueDwSICBaU9ZufX1eVO9zEseij/3ocGrep0QL3qNfdpJPv+UhVPALlgQG 5Lim1ZGJDOEPLHAuk3yW+qBUvRu+uT2b2AnwzK52o2TtsQIcx6l8WXSNOQLIeAaDPWta PAAptiTdCQqHDVRGokEjVtbmEUkcLuSDgmINUZPLg8c67WY4FfxpGWUWoomsXs7Rm9jW IuEg== X-Gm-Message-State: APf1xPAgnb89zLtxrGqacKoenY1j7MU4GNDMYThEsILf3FTaF04RjyjI EVsSGJkKYndfILTbyNhpfCk9wA== X-Received: by 2002:a17:902:22f:: with SMTP id 44-v6mr1837181plc.418.1519908974364; Thu, 01 Mar 2018 04:56:14 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.56.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:56:13 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Mark Rutland , Alex Shi Subject: [PATCH 08/45] arm64: uaccess: consistently check object sizes Date: Thu, 1 Mar 2018 20:53:45 +0800 Message-Id: <1519908862-11425-9-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Mark Rutland commit 76624175dca upstream. Currently in arm64's copy_{to,from}_user, we only check the source/destination object size if access_ok() tells us the user access is permissible. However, in copy_from_user() we'll subsequently zero any remainder on the destination object. If we failed the access_ok() check, that applies to the whole object size, which we didn't check. To ensure that we catch that case, this patch hoists check_object_size() to the start of copy_from_user(), matching __copy_from_user() and __copy_to_user(). To make all of our uaccess copy primitives consistent, the same is done to copy_to_user(). Cc: Catalin Marinas Acked-by: Kees Cook Signed-off-by: Mark Rutland Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/include/asm/uaccess.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 94e1457..09c9b59 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -326,9 +326,9 @@ static inline unsigned long __must_check copy_from_user(void *to, const void __u { unsigned long res = n; kasan_check_write(to, n); + check_object_size(to, n, false); if (access_ok(VERIFY_READ, from, n)) { - check_object_size(to, n, false); res = __arch_copy_from_user(to, from, n); } if (unlikely(res)) @@ -339,9 +339,9 @@ static inline unsigned long __must_check copy_from_user(void *to, const void __u static inline unsigned long __must_check copy_to_user(void __user *to, const void *from, unsigned long n) { kasan_check_read(from, n); + check_object_size(from, n, true); if (access_ok(VERIFY_WRITE, to, n)) { - check_object_size(from, n, true); n = __arch_copy_to_user(to, from, n); } return n; From patchwork Thu Mar 1 12:53:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130189 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2761244edc; Thu, 1 Mar 2018 04:56:31 -0800 (PST) X-Google-Smtp-Source: AG47ELsqbefAOnf2X9T+2PxAQoD7nq/9GP88W9nXCb48rqBdKruf0W/DN09VJ7COc2xEpUNy6ROy X-Received: by 2002:a17:902:7046:: with SMTP id h6-v6mr1770843plt.301.1519908991390; Thu, 01 Mar 2018 04:56:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519908991; cv=none; d=google.com; s=arc-20160816; b=OYPLTcjuYODZqLn4gLQLPO3ySBc1BfBDE2eVDv8QOEnmrEcIu8XRvZGSqHDSwginAA ZXy+E678szbzCdQBTbyasSStjtokiMLiBofg9vXBVMGuuYPR2E3bKafFOvPS8bx7TBX0 YwDxWMwFRSMg0w9Rts1+uecZN2RqsBT6nho6If/ByMNUtu9T33V3cDTRSeqqgUz7Yn4P KQSe1Sa6tFXg96aTFH/ZpBmpmYuMInVZcy7GFV6akB9Q02sajkrhNuiAXlAV5JPWI5A2 oZe6tCmiqH9cahprp/KbUHkHnt89p2KlGUzYsCZLZhgj+8QKWJfAV0SZtjCpt/3jyJu+ SQtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=pkSwvmKMHNNLRhymKkKAhX1GDqTiFiQOpUbZ8tKMBgk=; b=Ff0dHKV4QrJJIgDDSTIsQ8VreQpS5hgTZgvuhAlzsNSy8tNThmz8TAv5wpQLk3EMzv ZmOtoCEdMjimvUECZVAnv6TK5G2prLLjKNwCW9nq6AV0WUxPxwt02uRypjFg6/ZNuVQp lTuNzCKqbwUIFku3H+QcM7OxdBg/fkz0OP9WNUYWWaVxMAmikOpGUL1aQysvYj2JDgQA ZvvoYKCkryhUUT9Stur+sz3Em+yHnBG9OPvu0XvbhEJhExdgRBuVO7/4BMzMq08VXvlP SoJ4ac43gjuDEBdX/SaB2jzKOgQNt8aq81dWA+jLhlZpmAWN5Pi201RXCTSS9Ha+V7zp dwrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TWc8Mayd; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v17si2956195pfe.186.2018.03.01.04.56.31; Thu, 01 Mar 2018 04:56:31 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TWc8Mayd; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030729AbeCAM41 (ORCPT + 28 others); Thu, 1 Mar 2018 07:56:27 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:38757 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030577AbeCAM4X (ORCPT ); Thu, 1 Mar 2018 07:56:23 -0500 Received: by mail-pf0-f195.google.com with SMTP id d26so2405598pfn.5 for ; Thu, 01 Mar 2018 04:56:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pkSwvmKMHNNLRhymKkKAhX1GDqTiFiQOpUbZ8tKMBgk=; b=TWc8MaydIeTgVwL1bNsWb6speWePBk5cwbeu8tE+wxKd8SsD5cVfgBhwWptS4TtVbN wty/oSLnY4XVK8XAfxr+3+LIXVfEnx+pHfKFbR1prFS0wzbjhvJWGXHXEqmyLXloSg1Q VEnFT/xBJHIOyP5e6cVXuVBA4TQn4CnmF3Mt4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pkSwvmKMHNNLRhymKkKAhX1GDqTiFiQOpUbZ8tKMBgk=; b=aH4JabKBc8ETTRxdv5Ue6qId6A0dnu+Pu/qplx9OYAUc42Nn5ylca5sG0FW/oxk3B1 kkffcwM9cxvMOvlQD5wVfsITn92+LWbo5xhs0O9coNLEsmjbz62y9DP7p8YwAIWwpUlq c5aB65E6XoLgQI+RerUUakdz/JEqh91FZ6QgydXt+iI9Ksis2srYBF6jWea44h980d0E xueYGTuLFtMh3tSPMff/wn9f0PORQlgokGBWKNHN4Zy/9JQwJ8sxnAe4QiXLm1eilaKZ HGeSd3cpZdubSzFG8SelVRX3EZShjBAeMmElf+eSam3fqsxUjKTJx0YJvV+jIJsPZxEl IUUA== X-Gm-Message-State: APf1xPD/jlSi5eaA/Ssh0bmuh/YNX9DTILfAcU/VbI2fRmjBSx+TMN2B X0U5YClLm37F1M3lmT62W0NhbQ== X-Received: by 10.101.86.198 with SMTP id w6mr1462142pgs.434.1519908983165; Thu, 01 Mar 2018 04:56:23 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.56.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:56:22 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Robin Murphy , Alex Shi Subject: [PATCH 09/45] arm64: Make USER_DS an inclusive limit Date: Thu, 1 Mar 2018 20:53:46 +0800 Message-Id: <1519908862-11425-10-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Robin Murphy commit 51369e398d0d upstream. Currently, USER_DS represents an exclusive limit while KERNEL_DS is inclusive. In order to do some clever trickery for speculation-safe masking, we need them both to behave equivalently - there aren't enough bits to make KERNEL_DS exclusive, so we have precisely one option. This also happens to correct a longstanding false negative for a range ending on the very top byte of kernel memory. Mark Rutland points out that we've actually got the semantics of addresses vs. segments muddled up in most of the places we need to amend, so shuffle the {USER,KERNEL}_DS definitions around such that we can correct those properly instead of just pasting "-1"s everywhere. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas (cherry picked from commit 83b20dff71ea949431cf57c6aebaaf7ebd5c1991) Signed-off-by: Alex Shi --- arch/arm64/include/asm/processor.h | 3 +++ arch/arm64/include/asm/uaccess.h | 46 ++++++++++++++++++++++---------------- arch/arm64/kernel/entry.S | 2 +- arch/arm64/mm/fault.c | 2 +- 4 files changed, 32 insertions(+), 21 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 4258f4d..5917147 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -21,6 +21,9 @@ #define TASK_SIZE_64 (UL(1) << VA_BITS) +#define KERNEL_DS UL(-1) +#define USER_DS (TASK_SIZE_64 - 1) + #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 09c9b59..7b1eb49 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -36,6 +36,7 @@ #include #include #include +#include #define VERIFY_READ 0 #define VERIFY_WRITE 1 @@ -62,10 +63,7 @@ struct exception_table_entry extern int fixup_exception(struct pt_regs *regs); -#define KERNEL_DS (-1UL) #define get_ds() (KERNEL_DS) - -#define USER_DS TASK_SIZE_64 #define get_fs() (current_thread_info()->addr_limit) static inline void set_fs(mm_segment_t fs) @@ -90,22 +88,32 @@ static inline void set_fs(mm_segment_t fs) * Returns 1 if the range is valid, 0 otherwise. * * This is equivalent to the following test: - * (u65)addr + (u65)size <= current->addr_limit - * - * This needs 65-bit arithmetic. + * (u65)addr + (u65)size <= (u65)current->addr_limit + 1 */ -#define __range_ok(addr, size) \ -({ \ - unsigned long __addr = (unsigned long __force)(addr); \ - unsigned long flag, roksum; \ - __chk_user_ptr(addr); \ - asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, ls" \ - : "=&r" (flag), "=&r" (roksum) \ - : "1" (__addr), "Ir" (size), \ - "r" (current_thread_info()->addr_limit) \ - : "cc"); \ - flag; \ -}) +static inline unsigned long __range_ok(unsigned long addr, unsigned long size) +{ + unsigned long limit = current_thread_info()->addr_limit; + + __chk_user_ptr(addr); + asm volatile( + // A + B <= C + 1 for all A,B,C, in four easy steps: + // 1: X = A + B; X' = X % 2^64 + " adds %0, %0, %2\n" + // 2: Set C = 0 if X > 2^64, to guarantee X' > C in step 4 + " csel %1, xzr, %1, hi\n" + // 3: Set X' = ~0 if X >= 2^64. For X == 2^64, this decrements X' + // to compensate for the carry flag being set in step 4. For + // X > 2^64, X' merely has to remain nonzero, which it does. + " csinv %0, %0, xzr, cc\n" + // 4: For X < 2^64, this gives us X' - C - 1 <= 0, where the -1 + // comes from the carry in being clear. Otherwise, we are + // testing X' - C == 0, subject to the previous adjustments. + " sbcs xzr, %0, %1\n" + " cset %0, ls\n" + : "+r" (addr), "+r" (limit) : "Ir" (size) : "cc"); + + return addr; +} /* * When dealing with data aborts, watchpoints, or instruction traps we may end @@ -114,7 +122,7 @@ static inline void set_fs(mm_segment_t fs) */ #define untagged_addr(addr) sign_extend64(addr, 55) -#define access_ok(type, addr, size) __range_ok(addr, size) +#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size) #define user_addr_max get_fs #define _ASM_EXTABLE(from, to) \ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 478f0fe..6915697 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -103,7 +103,7 @@ /* Save the task's original addr_limit and set USER_DS (TASK_SIZE_64) */ ldr x20, [tsk, #TI_ADDR_LIMIT] str x20, [sp, #S_ORIG_ADDR_LIMIT] - mov x20, #TASK_SIZE_64 + mov x20, #USER_DS str x20, [tsk, #TI_ADDR_LIMIT] /* No need to reset PSTATE.UAO, hardware's already set it to 0 for us */ .endif /* \el == 0 */ diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 403fe9e..4df70c9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -332,7 +332,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } - if (is_permission_fault(esr) && (addr < USER_DS)) { + if (is_permission_fault(esr) && (addr < TASK_SIZE)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) die("Accessing user space memory with fs=KERNEL_DS", regs, esr); From patchwork Thu Mar 1 12:53:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130190 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2761337edc; Thu, 1 Mar 2018 04:56:37 -0800 (PST) X-Google-Smtp-Source: AG47ELsKBEq05rvBcdBgfs+1bf6DlKfgWLZ4kVotqzKoFbbWpPNLVJl9kRHNlDTaDxDFm5yJ8vgY X-Received: by 2002:a17:902:aa87:: with SMTP id d7-v6mr1831054plr.237.1519908997626; Thu, 01 Mar 2018 04:56:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519908997; cv=none; d=google.com; s=arc-20160816; b=GzV486tuRxIiH9gAqX9HRMmreuyn6t2wppWDYEVZOy54Q6q0osIkZMr/c7bKjDpcj4 9fx5mWzOJ9ytai5BvfvxTdgT7JU6hY+nPlLQHTreECC6X73H2F4z936w2ogx/86bRiQH ey880B4ROri1Cpe5kFXZOss98RJbPIzRAc/3VdB9gaY1JYm/1vnaU2ZtyeyFZmmv6hsl pN+UAqpWPBU7iqnSzmjgW5rwYoebwHG027mnGFTx2DquoRGApaVixFBg91eIXGylsexb 6/7tdlAsm9wzFyGFXHq0C5w9diJVWiT2DKSozJuRIUX90WqFNgMEbdcYUkn+KuTlSOhp 9S3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=i4j5LLlcJTrNi6a03osuM7FMBFfhyS0MAdksfWz86sU=; b=l55OG9Clip40FMSmaGspwWNvky6d+RB77UlF4HWJ75SK+uCD3LuUwXlrghzL6DvVfC rEXWU7JrY046xt5w3rmuDNCPGHpKqB1OlxLDvEY0GzfjoMe3/nF1u8F+A7rHGL1myO5K zQXRqE4V9xt1CFo+Vm4ppRxpXY8Rdt1+5JymqTht4Xw7goJrp7FTK5/WHb5YHH/ULwGO 18N25N4NymqzCtrSsFufBKhUbwG7/4uQrfFIgaO21FScmBSUKmMq1bu4Cx+uVqlOkJ7V 3UL1GuwSG7x+FdoyOFoMMnvxDLiF7l/CQALQ14hJ65ez7DlhmZKkeKVRv4VQqOOwbDfF o4gw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=R0saQ4fw; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 14-v6si3022704plb.444.2018.03.01.04.56.37; Thu, 01 Mar 2018 04:56:37 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=R0saQ4fw; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030743AbeCAM4e (ORCPT + 28 others); Thu, 1 Mar 2018 07:56:34 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:34518 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030577AbeCAM4b (ORCPT ); Thu, 1 Mar 2018 07:56:31 -0500 Received: by mail-pl0-f68.google.com with SMTP id u13-v6so3593880plq.1 for ; Thu, 01 Mar 2018 04:56:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=i4j5LLlcJTrNi6a03osuM7FMBFfhyS0MAdksfWz86sU=; b=R0saQ4fweThOggq/TobnvzHWJpLRton+5jJ0eFNYqO5Snpv2UioTyy6aGI6w4P0KOB ON5T9SkWwhlF0sSoZBtM07coOH5WOu7BJ3Wu1f+tjr6cxw9qZo+ztL/JTJLREOdqttzI 7BYcQCOeS00mxpxGtGWiH+X72LTRbi7E3+YJA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=i4j5LLlcJTrNi6a03osuM7FMBFfhyS0MAdksfWz86sU=; b=UjpEExiOXBkvuy0VqH3amxe99glcl4TPWYdK5trYpfMswF/yfvjg0GB0sW0ANtWtcQ MO/aT+5dyE9h15fAY95hXAK3Yaqq8uIH5ZQNo2Y17QAydHA2zljJ2lKcbCnzmggh0vdL pZ2AYlWYAgTElaF2ANMhcoklj+GZgyEV1nNDxcUSrIfevrZZMgh2wsLzHr+Q4U8F2Kqc jgI85AeI1lSpq+Mlrz+B9WSDdYkkzLtzkpIuzgR4trWdl7LcV1OMsKacp2NRFwTw7cds evAJWa1LgoYjXxXN+r/PJmgImDWGLoxteWitbs2dQm1Pf7Hh/VXgJaLIwgzGKG8Bs9bK NMgA== X-Gm-Message-State: APf1xPBdge0XkQFF6ZkKooqGp2TGQzxiRI3dyM2sIcyYxK5aI/u/GPCl sLKVLcEohJ+kRbkdXoZhkwWF6Q== X-Received: by 2002:a17:902:b605:: with SMTP id b5-v6mr1873928pls.354.1519908991099; Thu, 01 Mar 2018 04:56:31 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.56.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:56:30 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Robin Murphy Subject: [PATCH 10/45] arm64: Use pointer masking to limit uaccess speculation Date: Thu, 1 Mar 2018 20:53:47 +0800 Message-Id: <1519908862-11425-11-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Robin Murphy commit 4d8efc2d5ee4 upstream. Similarly to x86, mitigate speculation past an access_ok() check by masking the pointer against the address limit before use. Even if we don't expect speculative writes per se, it is plausible that a CPU may still speculate at least as far as fetching a cache line for writing, hence we also harden put_user() and clear_user() for peace of mind. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/uaccess.h | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 7b1eb49..3531fec 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -170,6 +170,26 @@ static inline void uaccess_enable_not_uao(void) } /* + * Sanitise a uaccess pointer such that it becomes NULL if above the + * current addr_limit. + */ +#define uaccess_mask_ptr(ptr) (__typeof__(ptr))__uaccess_mask_ptr(ptr) +static inline void __user *__uaccess_mask_ptr(const void __user *ptr) +{ + void __user *safe_ptr; + + asm volatile( + " bics xzr, %1, %2\n" + " csel %0, %1, xzr, eq\n" + : "=&r" (safe_ptr) + : "r" (ptr), "r" (current_thread_info()->addr_limit) + : "cc"); + + csdb(); + return safe_ptr; +} + +/* * The "__xxx" versions of the user access functions do not verify the address * space - it must have been done previously with a separate "access_ok()" * call. @@ -241,7 +261,7 @@ do { \ __typeof__(*(ptr)) __user *__p = (ptr); \ might_fault(); \ access_ok(VERIFY_READ, __p, sizeof(*__p)) ? \ - __get_user((x), __p) : \ + __p = uaccess_mask_ptr(__p), __get_user((x), __p) : \ ((x) = 0, -EFAULT); \ }) @@ -307,7 +327,7 @@ do { \ __typeof__(*(ptr)) __user *__p = (ptr); \ might_fault(); \ access_ok(VERIFY_WRITE, __p, sizeof(*__p)) ? \ - __put_user((x), __p) : \ + __p = uaccess_mask_ptr(__p), __put_user((x), __p) : \ -EFAULT; \ }) @@ -368,7 +388,7 @@ static inline unsigned long __must_check copy_in_user(void __user *to, const voi static inline unsigned long __must_check clear_user(void __user *to, unsigned long n) { if (access_ok(VERIFY_WRITE, to, n)) - n = __clear_user(to, n); + n = __clear_user(__uaccess_mask_ptr(to), n); return n; } From patchwork Thu Mar 1 12:53:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130192 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2761566edc; Thu, 1 Mar 2018 04:56:54 -0800 (PST) X-Google-Smtp-Source: AG47ELvwwaiNqWibhUS4ug0ojC4m0EV8O3i6JlQQ4L3WusE7igRY/VYbaUKl+xggutAz52lVqZM8 X-Received: by 10.98.245.131 with SMTP id b3mr1891728pfm.20.1519909014665; Thu, 01 Mar 2018 04:56:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909014; cv=none; d=google.com; s=arc-20160816; b=im7/lJxa2j/7ajYE/8rmOWlPsYSpBp1cLyXjuZXA5JVNZ9MkI974ZhhNCw0gzhec5r aK2Yw2CVzvYDqeSv6BJjmNEiUJCjhJfbAUHRQrhBJbSl8DsHJ8n2YB0tN++8PJGmFmKR wbT+lsnCnWW3JBcWjSddk5IrxOwaFXVEBLYtLDzT/gJ3bp+8pla0Wd5+uhWBGuU8ylJl 0FpOZfu01hzla7MtoTMhULbwFgS78ih9sJ4SjTqwGiQeE7x8PhqLLTN+mXIeuLWq4buG +RrJnrX1qYz++veegF0iEeKWTpac4ULeQnI+mTrLH2WDl8aamAKmtpUXB2DSeIG1gxWH OSig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=taE8mK2Ip25XXu3tH/TfrJ9TiKkrcCk7pgR93VQ9+cc=; b=MhWnm3ZMeZe8r9Gc9PLcIAlb4fe/x+5KmKB7R+ik2ySDozE1pUAtQ91e/057q6u/FI u+Qu1uMxk0jLVJIOiWRJIbhxCoIcAabLeb2ve7FTGgb5fH7uS+M2rseu7sdF+tlnd5DJ L+MtIDGLYYeEmOLPAoNXT6asqfv7w5hsT+et1vm2qbn7XDtfKundg3XzmiCZBaR3PmuR 4hUhxhIIVeFEfsjxo31P9Cr7ym27010l+1V85/CgQUtJ4CSwERcS2PR7WmeQ9H+TxKEg o06Ms9Yy9xYBnFWk5n0U+KkayCNqd+kAfHSjvC7N563evcro9hD+0Zcy76WZ85ziHO71 QOOg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kIYle+vW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q18si3011014pfi.105.2018.03.01.04.56.54; Thu, 01 Mar 2018 04:56:54 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kIYle+vW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030773AbeCAM4v (ORCPT + 28 others); Thu, 1 Mar 2018 07:56:51 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:37817 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030576AbeCAM4q (ORCPT ); Thu, 1 Mar 2018 07:56:46 -0500 Received: by mail-pf0-f195.google.com with SMTP id h11so2008080pfn.4 for ; Thu, 01 Mar 2018 04:56:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=taE8mK2Ip25XXu3tH/TfrJ9TiKkrcCk7pgR93VQ9+cc=; b=kIYle+vWacd0WWpBliXR7l16/dddPTbZmm1Rx4E2dG+3u/1jdUn/IZLWWUyyfxr2hD El+MuyFgxNPXf9XL3tvpmw9pZfB8wEs/RFZfiW5fCoYQvuuwfbryin7ygf9dAdPebEQ9 9QiUKfqJO8e4TUcz4GfrVMSI1MaJ+kfg5X4ZM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=taE8mK2Ip25XXu3tH/TfrJ9TiKkrcCk7pgR93VQ9+cc=; b=O3wTB5rSEA04gNhaXqUa8sYKqvu5bxeY3kSSQ0LmNNFEV+eJU29dkZAFlvS0IGLAE/ 9cxmNVZKZUDn2qUdNCwTVshaOkJMr6v9iac16anB6lYr+HhjVeQmyWyfJhHyaGh/YymZ jh8HpklqTAJGLpoZMDf28kkegLvl7OTc+K9v9nKJkxY2gw40ffRykxyfqHUpnXnkLjm/ KAQDgPRWhj1CxV/sSdt4QL0OiGYVlX1pyU/h697/NVYwpYqp7Mlz4seEDsxjLMc/yg8Y JEOnwIj33gFLiIa3GOCV+L+Qjy5cJMa32Oeojf/nAnQdsSwyhqBm+P/vzkCcWwK+S1C4 sIhw== X-Gm-Message-State: APf1xPAkM6tadQO4H6EwlmOVsdFF27IEM8OnUoX4Nf+/udglpwMEnDh3 mozFkcDlArF2HvAKvQlzBVDEG6pCRN4= X-Received: by 10.99.178.6 with SMTP id x6mr1526018pge.98.1519909006247; Thu, 01 Mar 2018 04:56:46 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.56.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:56:45 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 12/45] arm64: entry: Ensure branch through syscall table is bounded under speculation Date: Thu, 1 Mar 2018 20:53:49 +0800 Message-Id: <1519908862-11425-13-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 6314d90e6493 upstream. In a similar manner to array_index_mask_nospec, this patch introduces an assembly macro (mask_nospec64) which can be used to bound a value under speculation. This macro is then used to ensure that the indirect branch through the syscall table is bounded under speculation, with out-of-range addresses speculating as calls to sys_io_setup (0). Reviewed-by: Mark Rutland Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/assembler.h | 11 +++++++++++ arch/arm64/kernel/entry.S | 2 ++ 2 files changed, 13 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index ab3a179..a109d30 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -94,6 +94,17 @@ .endm /* + * Sanitise a 64-bit bounded index wrt speculation, returning zero if out + * of bounds. + */ + .macro mask_nospec64, idx, limit, tmp + sub \tmp, \idx, \limit + bic \tmp, \tmp, \idx + and \idx, \idx, \tmp, asr #63 + csdb + .endm + +/* * NOP sequence */ .macro nops, num diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 48c41ff..0a27e12 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -233,6 +233,7 @@ alternative_else_nop_endif * x7 is reserved for the system call number in 32-bit mode. */ wsc_nr .req w25 // number of system calls +xsc_nr .req x25 // number of system calls (zero-extended) wscno .req w26 // syscall number xscno .req x26 // syscall number (zero-extended) stbl .req x27 // syscall table pointer @@ -754,6 +755,7 @@ el0_svc_naked: // compat entry point b.ne __sys_trace cmp wscno, wsc_nr // check upper syscall limit b.hs ni_sys + mask_nospec64 xscno, xsc_nr, x19 // enforce bounds for syscall number ldr x16, [stbl, xscno, lsl #3] // address in the syscall table blr x16 // call sys_* routine b ret_fast_syscall From patchwork Thu Mar 1 12:53:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130193 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2761712edc; Thu, 1 Mar 2018 04:57:03 -0800 (PST) X-Google-Smtp-Source: AG47ELvWLvwSpIMroIOGghOVHkXei1hUS2jUDOlJvvoJvtqngUcZhINAJ8UgypOD2zr/BX5qyiHV X-Received: by 10.101.101.204 with SMTP id y12mr1509884pgv.450.1519909023553; Thu, 01 Mar 2018 04:57:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909023; cv=none; d=google.com; s=arc-20160816; b=VlQOzRbPR1+wcCSV9A7GeO56etKxemDdUEMjYA0+eW8JTDptpWv73Vt5uYE09pDlVs KBVNw++dkwbMl+U1M/E2GTTmiiCurDOrd77M3agOyyhVkqjpyQ4gWDyROzNteMw2VoI5 6kc4sup2JSqX19hXaB6VlKBkQCB7uPQtURbSzez5FEaeCLjmeebDcTxRu7IpK+VaAiBl ZDzz3mWV54/vqnfHXK3wGfjC+MdMPa3ZAI25yG4jEmBbYMjdaBfQc3i+f+DOrgsFfn5H 2JkWWG0ivLmD4JcCWhLEauuL0CU8rEkZ7kSTP/FrN+RJtJSGAwIT+cTS4ialIeOxDqeU 5Lrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=B9j5UF0uESTYsHgGAtUITtbvy9Mt1eVCWXup4v2dg9E=; b=jTKD3qiHqz+e1o8QLX463+ix2jx1jCTFFzkzTpdP3w+hUB6712CeEcBvzOJnnbPioj hJhLD2jim3rf19Jp7Hy3wnTLM/ymQeVOlRY/1liILV4v0O4dJnblm3YRPgAAJwgpynJ6 kvzadrMeLzId58hFDaextsrDEIOVv9ACrQgMPnV0g4RyJ13J2Gdd6Xtxd9XLcH0z+fby I4k77JeU/gOiQ7UOIZ8e/Q8Jedfpgx+aOH38mHj1hr7yF78RRhajYvx1ApEoAAQfF2io kTI3/5g6P5Ckpl+LAWTunGUa4xcyvAK3FbjhM3mT5DteeKNqmZzsdPgtdpQ0Z2TLRxOP qbaQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L/9E5Ltp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b185si2439066pga.370.2018.03.01.04.57.03; Thu, 01 Mar 2018 04:57:03 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L/9E5Ltp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030787AbeCAM46 (ORCPT + 28 others); Thu, 1 Mar 2018 07:56:58 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:34554 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030545AbeCAM4z (ORCPT ); Thu, 1 Mar 2018 07:56:55 -0500 Received: by mail-pl0-f65.google.com with SMTP id u13-v6so3594424plq.1 for ; Thu, 01 Mar 2018 04:56:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=B9j5UF0uESTYsHgGAtUITtbvy9Mt1eVCWXup4v2dg9E=; b=L/9E5LtpJm2HVM2hM3oZ4BRU2ItlnylsxSkB+i33/7KgxxYnknzyJd7p/POIFigpYd TEKeg92LwkyH/j9FPEJZUVnwTG9Rsypub2fGgBV80aHn9kjh5aE66gPNJncuu4LJ7w0/ IrZ66XFjwkpfm64Q1AWGMJ3lHm4MFNt2vaRhA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=B9j5UF0uESTYsHgGAtUITtbvy9Mt1eVCWXup4v2dg9E=; b=toyvw6udMxxpJL5gNpnuCoJIxPmKU0R/GF0iKIVTFHyau03syYe5Mv/7r/Pxz7YjTH w/AB3y5hOHcJORK9Bqy9KN4VgagXeHGMU1ZX0fuDtJeq+YmlcIDeNf+S9HEz88D/UWWh RH0f16KgWjlSebyLJKPSLwwSS7wd3Oidjdam4Teaem0gkznRuzFzu4QfTKylvWySr3gu yAccDGBplo7vRwtWnjdD7ecMh7EyRluh5KerJG2D+t1gd62d46WOw1YartfpoGxxIsCT UKoaGd76CNUTn6UA3I/hmLimv2MUk/X/TdHW1qWkRz8sFdkOt3Vkc1L4uIPZA7xikomO eyyw== X-Gm-Message-State: APf1xPASBbGQx05CpS7zKyBb4vRuRkSXetVCeBCWlrbKyNit5EikOxV2 yf60zmLR0wkP++OUSHidjAUkMA== X-Received: by 2002:a17:902:7182:: with SMTP id b2-v6mr1823353pll.331.1519909014428; Thu, 01 Mar 2018 04:56:54 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.56.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:56:53 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 13/45] arm64: uaccess: Prevent speculative use of the current addr_limit Date: Thu, 1 Mar 2018 20:53:50 +0800 Message-Id: <1519908862-11425-14-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit c2f0ad4fc089 upstream. A mispredicted conditional call to set_fs could result in the wrong addr_limit being forwarded under speculation to a subsequent access_ok check, potentially forming part of a spectre-v1 attack using uaccess routines. This patch prevents this forwarding from taking place, but putting heavy barriers in set_fs after writing the addr_limit. Reviewed-by: Mark Rutland Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/uaccess.h | 7 +++++++ 1 file changed, 7 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 3531fec..00025c5 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -71,6 +71,13 @@ static inline void set_fs(mm_segment_t fs) current_thread_info()->addr_limit = fs; /* + * Prevent a mispredicted conditional call to set_fs from forwarding + * the wrong address limit to access_ok under speculation. + */ + dsb(nsh); + isb(); + + /* * Enable/disable UAO so that copy_to_user() etc can access * kernel memory with the unprivileged instructions. */ From patchwork Thu Mar 1 12:53:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130194 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2761883edc; Thu, 1 Mar 2018 04:57:17 -0800 (PST) X-Google-Smtp-Source: AG47ELsYiW9Y28mSr/9HXv4J/Z4jy7bAr8onTa+P4yXCP+ti7SKD8SIwtiRu5rUx2QtF0L/7ie/q X-Received: by 10.98.224.208 with SMTP id d77mr1885490pfm.194.1519909036980; Thu, 01 Mar 2018 04:57:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909036; cv=none; d=google.com; s=arc-20160816; b=NABXXPmCHISWdAnUEoW8KOS5wf4eC4IGRPr/e5CXNmHCKaiZ2slsDtsc7Qqx+71uCq AnOK22bwHHeckC3w101UdyyGIL+XOb1Ig54QBM6NycHPq5lWZ7Jaonb5VrFOLFdzFVqj rVEU9l2lT2b2kdJEM7ehBHDya9Lt5alnZQSOdkzRf3RpVCTnB+gwa5m7XFZ4Dx3HlGP5 CW6GkpDn0R74dlwrYZ0RBhxyMLtvxvh9TTmGKYYXcdQc32/obcfRXWuJnx97Sf3RT1q1 byhAoVBH25PEWwf7mflSuYW77Sb3mplDc+uMXXEM1KADOmb1+jKdiA8MyLTWoRk/jVXn AIRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=B06UrOpFOhDhk8CXgd0QbaeXaNIGUHvFPN317fRlFQ4=; b=LUR/mVVPlqdIzfOdYZTMDCbAr3L1ntIQu3hJ/sY28hq9aCtdPPZjBoEoW/tehegUci gpsY1T6pUkpbeZ2f49ZSaZDJKXSF015LE9/E1wxEtD2COV2zwOwCQ9l9HG6iT+X2wlSi 6OhrC918IwkpIvvfN7gO/vvpSWEgjkv1/3z7lDPdw+rMuq/k2bYnWXjAtHfE/HgGEYif m+18aqAqurJIf0ooWwrlRi8I1AUXlcqSOcK0VzeZDPh5LexxGxrVqF83yxJbVtroKetq gdZPeIX3oaAsTRw1/3WTX2MFJBQUm/+cU8tbPqRCAzVrMhUCzpZJbewSrWCjuJQTFG5a KfJA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=D/+FhXt5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b185si2439066pga.370.2018.03.01.04.57.16; Thu, 01 Mar 2018 04:57:16 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=D/+FhXt5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030821AbeCAM5M (ORCPT + 28 others); Thu, 1 Mar 2018 07:57:12 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:36481 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030790AbeCAM5C (ORCPT ); Thu, 1 Mar 2018 07:57:02 -0500 Received: by mail-pg0-f66.google.com with SMTP id i14so2276325pgv.3 for ; Thu, 01 Mar 2018 04:57:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=B06UrOpFOhDhk8CXgd0QbaeXaNIGUHvFPN317fRlFQ4=; b=D/+FhXt5iKbRmF6usXiWGMZw6TVW3dhAcUiL4cOj1ExLjn8byPNEftelFZB6NgiL6m hDnmkPfXNAEK64ROsIPjHe3v4lDxFzNZOUU+ZAQLu/W69DNNMJx3CFpu5CyaGi6vQ0qG NeiU3VeNURVbqPLyJIEQ/wfKw/7LULtU41Qlk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=B06UrOpFOhDhk8CXgd0QbaeXaNIGUHvFPN317fRlFQ4=; b=THpjgMk0jWa5d302WTUCeFs4xM8dnbjOhZyj51Dbf+rMDytmgIc2CehahsFbqRKek/ 2Yqm2KuEZPm1MigJisHKzXWVcUJ1nv2a0JdH3JQJbrjCR779KCb4tjTVxEWo8HS9X3CP C4MlyDDcujD2fYmCCB/RauE841E96cPiWMxxjVFgr/ZCxRy5fGs4u78sI4c0JN1e0h1E LQHaxRqlCu16Vw1mra1bOYoKmcrDgZvozULXYyDAkMcCRa1QAzFoa5taNVv0kh9fi5Yk HjE9K8rIbsU4iCykgEtiJO3o8WG6Hvhv+8q4N/WHKeKKxaSA+J7pjDS2GE3tzgARoCQ/ xx2A== X-Gm-Message-State: APf1xPCKQmhE+B5qJLGNkrdUKWyA7PQ7yv+7dVesMZ2IAY7bOWF44wX8 nN1Y2mhZ4wrk8pnBT+LT+KUNFg== X-Received: by 10.101.69.4 with SMTP id n4mr1510244pgq.184.1519909022161; Thu, 01 Mar 2018 04:57:02 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.56.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:57:01 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 14/45] arm64: uaccess: Don't bother eliding access_ok checks in __{get, put}_user Date: Thu, 1 Mar 2018 20:53:51 +0800 Message-Id: <1519908862-11425-15-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 84624087dd7e upstream. access_ok isn't an expensive operation once the addr_limit for the current thread has been loaded into the cache. Given that the initial access_ok check preceding a sequence of __{get,put}_user operations will take the brunt of the miss, we can make the __* variants identical to the full-fat versions, which brings with it the benefits of address masking. The likely cost in these sequences will be from toggling PAN/UAO, which we can address later by implementing the *_unsafe versions. Reviewed-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/uaccess.h | 54 ++++++++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 22 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 00025c5..ffa4e39 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -248,30 +248,35 @@ do { \ (x) = (__force __typeof__(*(ptr)))__gu_val; \ } while (0) -#define __get_user(x, ptr) \ +#define __get_user_check(x, ptr, err) \ ({ \ - int __gu_err = 0; \ - __get_user_err((x), (ptr), __gu_err); \ - __gu_err; \ + __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (access_ok(VERIFY_READ, __p, sizeof(*__p))) { \ + __p = uaccess_mask_ptr(__p); \ + __get_user_err((x), __p, (err)); \ + } else { \ + (x) = 0; (err) = -EFAULT; \ + } \ }) #define __get_user_error(x, ptr, err) \ ({ \ - __get_user_err((x), (ptr), (err)); \ + __get_user_check((x), (ptr), (err)); \ (void)0; \ }) #define __get_user_unaligned __get_user -#define get_user(x, ptr) \ +#define __get_user(x, ptr) \ ({ \ - __typeof__(*(ptr)) __user *__p = (ptr); \ - might_fault(); \ - access_ok(VERIFY_READ, __p, sizeof(*__p)) ? \ - __p = uaccess_mask_ptr(__p), __get_user((x), __p) : \ - ((x) = 0, -EFAULT); \ + int __gu_err = 0; \ + __get_user_check((x), (ptr), __gu_err); \ + __gu_err; \ }) +#define get_user __get_user + #define __put_user_asm(instr, alt_instr, reg, x, addr, err, feature) \ asm volatile( \ "1:"ALTERNATIVE(instr " " reg "1, [%2]\n", \ @@ -314,30 +319,35 @@ do { \ uaccess_disable_not_uao(); \ } while (0) -#define __put_user(x, ptr) \ +#define __put_user_check(x, ptr, err) \ ({ \ - int __pu_err = 0; \ - __put_user_err((x), (ptr), __pu_err); \ - __pu_err; \ + __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (access_ok(VERIFY_WRITE, __p, sizeof(*__p))) { \ + __p = uaccess_mask_ptr(__p); \ + __put_user_err((x), __p, (err)); \ + } else { \ + (err) = -EFAULT; \ + } \ }) #define __put_user_error(x, ptr, err) \ ({ \ - __put_user_err((x), (ptr), (err)); \ + __put_user_check((x), (ptr), (err)); \ (void)0; \ }) #define __put_user_unaligned __put_user -#define put_user(x, ptr) \ +#define __put_user(x, ptr) \ ({ \ - __typeof__(*(ptr)) __user *__p = (ptr); \ - might_fault(); \ - access_ok(VERIFY_WRITE, __p, sizeof(*__p)) ? \ - __p = uaccess_mask_ptr(__p), __put_user((x), __p) : \ - -EFAULT; \ + int __pu_err = 0; \ + __put_user_check((x), (ptr), __pu_err); \ + __pu_err; \ }) +#define put_user __put_user + extern unsigned long __must_check __arch_copy_from_user(void *to, const void __user *from, unsigned long n); extern unsigned long __must_check __arch_copy_to_user(void __user *to, const void *from, unsigned long n); extern unsigned long __must_check __copy_in_user(void __user *to, const void __user *from, unsigned long n); From patchwork Thu Mar 1 12:53:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130195 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2762063edc; Thu, 1 Mar 2018 04:57:27 -0800 (PST) X-Google-Smtp-Source: AG47ELszMOJ2RuKi3YxdN90oJW1Z1KfzKiOOfFbRIp3g2LkfEvxfREROdJz2bDP1Yueu9ZIxFYEM X-Received: by 2002:a17:902:4827:: with SMTP id s36-v6mr1834533pld.269.1519909047375; Thu, 01 Mar 2018 04:57:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909047; cv=none; d=google.com; s=arc-20160816; b=WVBsazIUFTzmRPZrOGdQlIR4QWCC9Q1tNiy/Ut01ncBoANlsRksxSq9ZZidN+5jW7s g7jvE/0ZRnkldz6deyHmZls/GN/RZkjKWup1wZMsn3KXJ3+Ht82BIVFeTaxkog/DpuwH d9QY71AFSVCxB4OvmkMi6167lvEZngD8U00luyVDoRXb6KXw8L8neU/2L8GthjrJiU2W Zi18TXqXJ2tkeHcmrzoernOGiTij5UYyqw6UEHXYjiuycse+QkyREWC2kr2VV7fNpI/n eKoKHwBA4TQ8vBdHjtAR+VD1iBLdJNZjFBANWf0t8parZJ1eWN0vwZ1kpZlEsxmUodK7 Sdow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=RdpE7iykfSRyUHlyiTypUt5pG2kDjHqgdWq/qtP7cnM=; b=Xp+pDzZOSpk3nRnN3JW0LEQttWlBF1gSWvRVpeQS1+rS1v7JuF5Z6o/hBqIZsIqtG1 jMQN2w60Re4zfYp10hK0clvGVWaELcL1HeM4hyErvKwHl2mkrVlvy1X0wmzrKvX7gnDL lm/MtJBH9n5V3Wmrx5/YsD89UeS2ULX6KP4AWTPifFsSgSHeOCRLxyy92H285vASMiTW zVtuRcyz9K4x/iI2jHwmAjGPJhnDEz03OhfcOHW1Tmw/PWrZJ0DHGH2U73tFVJ4y+jl7 5cIY5ta5u7BPfJdU50QINsPH6U11b/ypQlanIck52BSmFboF1GWirls1pKZIxylFWkT0 nhVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=UCUlHyYS; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w2-v6si84737plq.739.2018.03.01.04.57.27; Thu, 01 Mar 2018 04:57:27 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=UCUlHyYS; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030854AbeCAM5X (ORCPT + 28 others); Thu, 1 Mar 2018 07:57:23 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:40556 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030790AbeCAM5S (ORCPT ); Thu, 1 Mar 2018 07:57:18 -0500 Received: by mail-pf0-f193.google.com with SMTP id x1so522983pfh.7 for ; Thu, 01 Mar 2018 04:57:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RdpE7iykfSRyUHlyiTypUt5pG2kDjHqgdWq/qtP7cnM=; b=UCUlHyYShZhZO4lfFa0NSZDxFqINN4HY50VWrgVssgdPoAFqIFx9sn76t1qw56t96d OZer8JiD+RjYGmlrjepmvtkjyg2Spy9b7GLzelTP1SWvbb9ADMxuYP3VstOaX1SnioSj EAtU7FAuZ82qqAxUMPeznAZ5FRz5GWy38zN4U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RdpE7iykfSRyUHlyiTypUt5pG2kDjHqgdWq/qtP7cnM=; b=CRSGjNCHeGLMSfHYDA12cE5de0THeG45PmofXy9hmX00/oWYiMaLO6r33yHBMCB9mG RQoRkifdXlHc0qt+T6eNN+kkYWEy6VXC/zEOn+of1dY5rEOs9NgABWA1tr93eN6XVnnh gSqp3NGKoSgC/BCDLSvMzI3vVX5OonFquTIVqV/wH9R4BLH6uvvXGVE4nCvXaoyfmD4n GehCImRfcC0T1EH6Gj7wQTqEgMe4/adLA5Jb2gJ/K6EDydCVMfU5FDlvUvi33kTATuUT 3hV+LGr1cneaCVcNBbFeR+H1gAhjeeBaczcsv0ywbnz8qFM8dgq4AxLsXUvLkykTouIe 03/A== X-Gm-Message-State: APf1xPA/NG1FlIAmuRnc0XxxwOn3sOncEYv3HkKdE1hinmBvBaPmpWLp 75eOUKiw9ywfYJCIx5eVvU6Z2gqqSNE= X-Received: by 10.98.35.68 with SMTP id j65mr1851262pfj.5.1519909037479; Thu, 01 Mar 2018 04:57:17 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.57.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:57:16 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 16/45] arm64: futex: Mask __user pointers prior to dereference Date: Thu, 1 Mar 2018 20:53:53 +0800 Message-Id: <1519908862-11425-17-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 91b2d3442f6a upstream. The arm64 futex code has some explicit dereferencing of user pointers where performing atomic operations in response to a futex command. This patch uses masking to limit any speculative futex operations to within the user address space. Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/futex.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index 85c4a89..1943cf6 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -48,13 +48,14 @@ do { \ } while (0) static inline int -futex_atomic_op_inuser (int encoded_op, u32 __user *uaddr) +futex_atomic_op_inuser (int encoded_op, u32 __user *_uaddr) { int op = (encoded_op >> 28) & 7; int cmp = (encoded_op >> 24) & 15; int oparg = (encoded_op << 8) >> 20; int cmparg = (encoded_op << 20) >> 20; int oldval = 0, ret, tmp; + u32 __user *uaddr = __uaccess_mask_ptr(_uaddr); if (encoded_op & (FUTEX_OP_OPARG_SHIFT << 28)) oparg = 1 << oparg; @@ -106,15 +107,17 @@ futex_atomic_op_inuser (int encoded_op, u32 __user *uaddr) } static inline int -futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, +futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *_uaddr, u32 oldval, u32 newval) { int ret = 0; u32 val, tmp; + u32 __user *uaddr; - if (!access_ok(VERIFY_WRITE, uaddr, sizeof(u32))) + if (!access_ok(VERIFY_WRITE, _uaddr, sizeof(u32))) return -EFAULT; + uaddr = __uaccess_mask_ptr(_uaddr); uaccess_enable(); asm volatile("// futex_atomic_cmpxchg_inatomic\n" " prfm pstl1strm, %2\n" From patchwork Thu Mar 1 12:53:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130196 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2762117edc; Thu, 1 Mar 2018 04:57:32 -0800 (PST) X-Google-Smtp-Source: AG47ELu96GKpLuXM7ehMuIiXc7D1A2YwK1Bx3ScAouJo7Ivs1tVBjtxnzmr3AmbfLdyshRhE3gfN X-Received: by 2002:a17:902:5609:: with SMTP id h9-v6mr1862555pli.302.1519909052081; Thu, 01 Mar 2018 04:57:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909052; cv=none; d=google.com; s=arc-20160816; b=HGR8hxIu3Q5Fhd59s6wEplgNgVh60TlozP78Y18VoiLKpX04E1joQm7a9ZAxqVHyZH aI3G2d4mQfVXd7rMB6Lud25Is4myzgYyb+tDDq02jWwe1fmFG3IRWxqZu2z/J00mQ6BB SkD5uoSIriAGCNq6PG0z1+kswuEV8Y9eVpIedmCE5sd/e3jot20yxo3+b6hSxTktpTTd I1AumHtTrNHnH00p2Zq4LavsY1/ALQTOPKJu8oQ8trQNLxCV4KfReWeiGIj5M3K3h/fW Tov/OHqKlzGcHykUjwMQ2H0wPEva5gCEYtBebRwBwmoTFQ60YIn/3erTF49E12jXiuUq 1i8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=tfc0ew0kAdAAuBXUjDfVzA2fzAypIwJmE3g0LacJX+Y=; b=K7q6BfB5dBksf8D1QVtCqyO3fy3wX4GiML1ulnp32jXOCZTh869thlaHa3ntjYGTJO go2YAlclRYySasnsbMeOoVMwUv6bW3O0SzWR0HoPJvJSWTbkwujvc/l+my46YN/vkBsk D/2xbv5EKC669Hf49Pb9Z0FPimSPqDJW30SwOe7tHYUZdUFqvgvjdbPC+dY2hF651+HW gqh4NTV1wf9zfICJXx5VYcrJsMWAtlNv4/RCDSVr8wozZZ1ATG5XsPaBHXWAo0YUdcpO B3TK9B67TRvf4wJrTZUmS+xXdplp8nlFA4e/FJp02Wsn5OrH64ve4/m6yurO2Cb9sPYH AkWw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K/FCKrmc; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r4si2967984pfb.394.2018.03.01.04.57.31; Thu, 01 Mar 2018 04:57:32 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K/FCKrmc; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030869AbeCAM53 (ORCPT + 28 others); Thu, 1 Mar 2018 07:57:29 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:38008 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030790AbeCAM5Z (ORCPT ); Thu, 1 Mar 2018 07:57:25 -0500 Received: by mail-pl0-f67.google.com with SMTP id m22-v6so71410pls.5 for ; Thu, 01 Mar 2018 04:57:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=tfc0ew0kAdAAuBXUjDfVzA2fzAypIwJmE3g0LacJX+Y=; b=K/FCKrmcqHlWnlFt2drTfmezPq8tliZx9Wb2Z1LSw+Q99Op4u61KrNuUPul8I+jcFn 6a5Mmqhp6tXl6vPDJgmmKVrWKIgEJB5w9kmkb2loIddvOoYiLXw/8/1ipWoh4qxZ7os8 FDLX+NMr/TRI+kOamuadUUE+dBIgLc8b8xa0o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=tfc0ew0kAdAAuBXUjDfVzA2fzAypIwJmE3g0LacJX+Y=; b=F4YeCTSRtGFWhMfuCU3PV7fSxdOHhLcD4pUh4GBQYTElIUYIExLdPXKZ89hI55+kvP +qGEnhE+sQdPTXtbf+bWQzZ6/WZca8LTB1BEhW4GCu9X4Jp5E19Dbit8jC2YABNGc/vF AriBuCPRF3YmN+EJIKJhM8rUhSMRiSsN6dS6MGo0YrV4Tzu/5wawvj/eelQJOPIl2qy7 FBvOEPHSSyP8JtszchBsPr+k6cLUZc7/MN8m4kSc32rj1ecLmiKIrBSxBJp9UoMPbcrK Gpcipj17artvdtjz0VGpFOSewElWzheVxkW09Nv0mXYtsDPVyOQL2+HFnXnO77Xl/gDg BAPw== X-Gm-Message-State: APf1xPBHOgmBEop3Vp7CPh0OuFqYqw2wS5P6HFdlMxZd7qHtiO/l4gCv QxsjrSVhkAqD1/66VQTJZy8Ejw== X-Received: by 2002:a17:902:9882:: with SMTP id s2-v6mr1910969plp.196.1519909044733; Thu, 01 Mar 2018 04:57:24 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.57.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:57:24 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 17/45] drivers/firmware: Expose psci_get_version through psci_ops structure Date: Thu, 1 Mar 2018 20:53:54 +0800 Message-Id: <1519908862-11425-18-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit d68e3ba5303f upstream. Entry into recent versions of ARM Trusted Firmware will invalidate the CPU branch predictor state in order to protect against aliasing attacks. This patch exposes the PSCI "VERSION" function via psci_ops, so that it can be invoked outside of the PSCI driver where necessary. Acked-by: Lorenzo Pieralisi Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- drivers/firmware/psci.c | 2 ++ include/linux/psci.h | 1 + 2 files changed, 3 insertions(+) -- 2.7.4 diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c index 8263429..9a3ce76 100644 --- a/drivers/firmware/psci.c +++ b/drivers/firmware/psci.c @@ -496,6 +496,8 @@ static void __init psci_init_migrate(void) static void __init psci_0_2_set_functions(void) { pr_info("Using standard PSCI v0.2 function IDs\n"); + psci_ops.get_version = psci_get_version; + psci_function_id[PSCI_FN_CPU_SUSPEND] = PSCI_FN_NATIVE(0_2, CPU_SUSPEND); psci_ops.cpu_suspend = psci_cpu_suspend; diff --git a/include/linux/psci.h b/include/linux/psci.h index bdea1cb..6306ab1 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -26,6 +26,7 @@ int psci_cpu_init_idle(unsigned int cpu); int psci_cpu_suspend_enter(unsigned long index); struct psci_operations { + u32 (*get_version)(void); int (*cpu_suspend)(u32 state, unsigned long entry_point); int (*cpu_off)(u32 state); int (*cpu_on)(unsigned long cpuid, unsigned long entry_point); From patchwork Thu Mar 1 12:53:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130197 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2762220edc; Thu, 1 Mar 2018 04:57:39 -0800 (PST) X-Google-Smtp-Source: AG47ELudxr3/gzIInr46S+mQzwTWzOSMLMG/ZrdwC8mwQh04J4Nco9hxT5jyGvJaxw89o6F+AeZZ X-Received: by 2002:a17:902:a504:: with SMTP id s4-v6mr1908737plq.43.1519909059604; Thu, 01 Mar 2018 04:57:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909059; cv=none; d=google.com; s=arc-20160816; b=SmyXZMP4cfxMLH0zUdtf76nyxpQz0POIDVvkEBYtEhnaO36zqNbY9MsafOtTcm6aKz TjNV9exBLJG7sOlkKZ4M6Sg8F8rglkUqcovvBJtcT8+HMMjEe9xrl4shP7L2GqQ3Kcci CEDTlVBAQK5kcDrq9TLZJEAnIkmdGzmxWF1xTQpjtIev8G6wuGn2orMrDtDDafjk+ekF LkD2bf3OMNS0OMbrdyIAil9Edi72fuDav6fT1lV8h5VJ6hreje7UpUjlWQfgAC1KsfVj /0hXMaLd9CIMz1Y7VuOmQj31tq/UzoXiXFhrs635tlrhtgwHNhjfgav/VswjWXaoz5zG 9GpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=kRECNLAGvLdU50wl/x4bZ8haUF1zIORex5wijx37hoU=; b=AesveFIxDDf4L9ORzO0RlyPbNatGVT+pNoiJe92rd0lsJEQ4CBe4sYFihwehjYF7WR DgTzciDFIpJAi+PREKAhG1ZSDXm/aoXjTZ/ptFbz9mwE1SoncHSv8/0fAPCFfb+fkCjc UD45BLzED8mmbBwktFRJLtLUm1LljNfvCgTiuFE3lrV9UbnnEPdaDhQm2yj4QS/I41M9 2o16GWeGUCNgEcK7yrKEszYjOwDbf8UUIkJaj9Fx4DzjPaZFQq5EqI/gb61ACYSf/eR/ d1qZamBZ7Xvc0v7TfA+4ZDdzJ9CWD5OzsLHRP9J8J1g6mRE3FkrODEHk3esvAsEGzDk/ bijQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=UAGp3y1C; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r4si2967984pfb.394.2018.03.01.04.57.39; Thu, 01 Mar 2018 04:57:39 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=UAGp3y1C; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030884AbeCAM5g (ORCPT + 28 others); Thu, 1 Mar 2018 07:57:36 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:46052 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030790AbeCAM5e (ORCPT ); Thu, 1 Mar 2018 07:57:34 -0500 Received: by mail-pf0-f193.google.com with SMTP id h19so1520044pfd.12 for ; Thu, 01 Mar 2018 04:57:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=kRECNLAGvLdU50wl/x4bZ8haUF1zIORex5wijx37hoU=; b=UAGp3y1C7/q49oNYuAlcjG3D6pnUNPdpWufRL/I4+mivfaZBj8V1kjdy/JtC+XcOmt IpWhLRm3hDx+XD874RyhGUAahjPDY86Sssfx0HsUy6gwt1rKokPBHnFdgDUQ3zDHd7CV yr5GzG64TNwh5aCEk8/EDXcneiaeoTUrR5ElM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kRECNLAGvLdU50wl/x4bZ8haUF1zIORex5wijx37hoU=; b=a8YsNGLe6qWLIbcHqNOf5lkanF9wMYsqxMH/YJQqPewjKL+9uLJM687JfPcgAqt9Ag ZBsFitb3xqNRf1YVakIWP+eJLUD4ZZYd87KKaAQj56z3t1P4PZP/pIz+KoTUO2V/zYSO MAyACOK8tY5ZPLGS2uI04VjJAd8kY+bMWGag45DoMTBAD92DaRjRHa8lAlgYq5zZyDlD 60soDntyUv4KIBDkXv7phUiWh3q/F8HVIeuF5m0G/uFg9VF8Im9km9FEPa0jmBk/X+U9 x0eHdPTBsYYB3swzN4Euu/C+g3I3ilqnsGEF7I+Zk5UDNLL4EqaBF13UQEBbQ03ZBFPx k6kQ== X-Gm-Message-State: APf1xPA9Ak5KAGPZnE8LG2kjuZ5vNrtqEHtsY4KgTbSGeb+204FQC6FJ J597j8B9S7X71p5/nqwzFTGjtg== X-Received: by 10.98.159.209 with SMTP id v78mr1842256pfk.49.1519909053608; Thu, 01 Mar 2018 04:57:33 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.57.25 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:57:33 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: James Morse , Suzuki K Poulose , Alex Shi Subject: [PATCH 18/45] arm64: cpufeature: __this_cpu_has_cap() shouldn't stop early Date: Thu, 1 Mar 2018 20:53:55 +0800 Message-Id: <1519908862-11425-19-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: James Morse commit edf298cfce47 upstream. Alex Shi rewrite this commit on func this_cpu_has_cap(). The following commit log is still meaningful. this_cpu_has_cap() tests caps->desc not caps->matches, so it stops walking the list when it finds a 'silent' feature, instead of walking to the end of the list. Prior to v4.6's 644c2ae198412 ("arm64: cpufeature: Test 'matches' pointer to find the end of the list") we always tested desc to find the end of a capability list. This was changed for dubious things like PAN_NOT_UAO. v4.7's e3661b128e53e ("arm64: Allow a capability to be checked on single CPU") added this_cpu_has_cap() using the old desc style test. CC: Suzuki K Poulose Reviewed-by: Suzuki K Poulose Acked-by: Marc Zyngier Signed-off-by: James Morse Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/include/asm/futex.h | 1 + arch/arm64/kernel/cpufeature.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index 1943cf6..718140a 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -20,6 +20,7 @@ #include #include +#include #include diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 3a129d4..fcb67e0 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1063,8 +1063,8 @@ bool this_cpu_has_cap(unsigned int cap) if (WARN_ON(preemptible())) return false; - for (caps = arm64_features; caps->desc; caps++) - if (caps->capability == cap && caps->matches) + for (caps = arm64_features; caps->matches; caps++) + if (caps->capability == cap) return caps->matches(caps, SCOPE_LOCAL_CPU); return false; From patchwork Thu Mar 1 12:53:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130198 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2762324edc; Thu, 1 Mar 2018 04:57:47 -0800 (PST) X-Google-Smtp-Source: AG47ELv6JrB8RiXiSw9/xLw93AD27UqOW6iGa1GtNn+jBC6IacxEQ0XuvkQJtdDBXord+Wk2voIP X-Received: by 10.98.14.70 with SMTP id w67mr1862247pfi.1.1519909067130; Thu, 01 Mar 2018 04:57:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909067; cv=none; d=google.com; s=arc-20160816; b=dkmcUhmkN9Ne1Hm98Emht9IICnQ3weQ+2PyzJXgxnytTjeTrqLM9POAExjS4//5R9I lc55nOzAafjz5K0JdnQ5BJBswUplYB8FcI8a/J0yKwHrj8zrLvBz+tr4UJeyuan96M4D MlOQ+NBzAXpkm9dAwOVNeNRxFtwbd3fMS8wkI2qXFzemeJTB0NP8BiI8/qCFffD1HtP2 T33qNY6GJW0RpbozKYk0AFihjLuOq9LcLS/3QaiSQvdmAY1GC9h8Pa5vIL3nmwCwKUHH rhDk1S+5lUPViaSfoK7Oh/aI6bCZU0y6wzlHGIo2UE3rCUh8jagq80dZLxT7hxK3DWhp CktA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=lZSZ7mN/jhpd6MgLblCtRDrj6c7/jUPbXNYPUP0WgH0=; b=izHA+q5YTSCd+eM2JJ+s56rs/9R3QITNyUHy0cTCaagSIg/4mt076NFNh75g2GM91M muANJHCBmY+4hZemAKpOG+4n6Nw4cBdaRGVIUX58hROej49AvU06LpukMa60wSbWg3SP 9rVEv2W83fz49idAq+B7Cgo+8SRjocYrCXI4L5uicktMCnzhJ7FeyvWsVXPweyiMwpp+ 95b2PkpSi/7Kk9tyQ+gn1k9mqXIIUpm9QgOfI9lWfw6t59ZmFeYzW2/qe8XgBfGBuWyo PD6fjA3eb0jFHVtsYIQW4jn5YMYAfYyPJuHwH/8RU9l4lZ+x3N559+An0Erq3STKOX10 u8ig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WWRAREQN; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m8si2431103pgc.663.2018.03.01.04.57.46; Thu, 01 Mar 2018 04:57:47 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WWRAREQN; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030896AbeCAM5o (ORCPT + 28 others); Thu, 1 Mar 2018 07:57:44 -0500 Received: from mail-pg0-f65.google.com ([74.125.83.65]:40865 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030539AbeCAM5l (ORCPT ); Thu, 1 Mar 2018 07:57:41 -0500 Received: by mail-pg0-f65.google.com with SMTP id g2so2271374pgn.7 for ; Thu, 01 Mar 2018 04:57:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lZSZ7mN/jhpd6MgLblCtRDrj6c7/jUPbXNYPUP0WgH0=; b=WWRAREQNZkTXQj5ciNBE0dkrJL69J0JdIyRLXcqz7OZT7uyyh1Z/zDSrQ0hZsZ/Zcj b6rOPe5OsZKGydbdlCCV7gG6rIisfdWuLqM6JqN2tDuhwG5+aw0oUtbv8eSIberXbxV+ 0A4Fcs/aNaKYt3FmEFvQ8SLhQenFY4BmX18og= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lZSZ7mN/jhpd6MgLblCtRDrj6c7/jUPbXNYPUP0WgH0=; b=e15UZPn6XzR55JUGyOGwqQTg9vZpJs/GCEIqTwX5fkfJ1MJZy3fSQR7kDhZ5EvEOOq cZOjN0pFVJeSnw6steBgQ+n0MvN2mytn86wn2EGJWA5bkYgIrfrlks+t9IgCa1wychAi zwiL4ccReo7spszqzll6fzzm1VyRTNQaefmbNzS80LXNpATuT8xDb5FdoPjYkl9ZVvEb W0VyeU69pvnOSAy90WvRF91x6smNi1h1chgng3fRFiak9W/E3P/ORw4cDImsxuXPPDSR XjN+2467ejPK7qwj8AFfQ5HRh5DSl1SHfMXSS53sNHGNwbZucmBF1p4jliMKZH8DdJiN UwHA== X-Gm-Message-State: APf1xPArTweU8Qyyj5uE6s52QQUbWa9vDVHx3k0Jj31ynlOGJqdYHeE3 9GpOUwo7U18DaL++yhluzdDWEWRiuT8= X-Received: by 10.101.85.71 with SMTP id t7mr1482928pgr.386.1519909061117; Thu, 01 Mar 2018 04:57:41 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.57.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:57:40 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 19/45] arm64: cpu_errata: Allow an erratum to be match for all revisions of a core Date: Thu, 1 Mar 2018 20:53:56 +0800 Message-Id: <1519908862-11425-20-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 06f1494f837 upstream. Some minor erratum may not be fixed in further revisions of a core, leading to a situation where the workaround needs to be updated each time an updated core is released. Introduce a MIDR_ALL_VERSIONS match helper that will work for all versions of that MIDR, once and for all. Acked-by: Thomas Gleixner Acked-by: Mark Rutland Acked-by: Daniel Lezcano Reviewed-by: Suzuki K Poulose Signed-off-by: Marc Zyngier Signed-off-by: Alex Shi --- arch/arm64/kernel/cpu_errata.c | 7 +++++++ 1 file changed, 7 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index b75e917..c66a673c 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -53,6 +53,13 @@ static int cpu_enable_trap_ctr_access(void *__unused) .midr_range_min = min, \ .midr_range_max = max +#define MIDR_ALL_VERSIONS(model) \ + .def_scope = SCOPE_LOCAL_CPU, \ + .matches = is_affected_midr_range, \ + .midr_model = model, \ + .midr_range_min = 0, \ + .midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK) + const struct arm64_cpu_capabilities arm64_errata[] = { #if defined(CONFIG_ARM64_ERRATUM_826319) || \ defined(CONFIG_ARM64_ERRATUM_827319) || \ From patchwork Thu Mar 1 12:53:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130199 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2762444edc; Thu, 1 Mar 2018 04:57:57 -0800 (PST) X-Google-Smtp-Source: AG47ELtBtwqivhpwZizTZe2i1cRMnoutP6ztBLqBILEvQeT3BUG7Ubgxf0l3o562Q5ugiAA9deN9 X-Received: by 10.99.112.20 with SMTP id l20mr1511829pgc.412.1519909077631; Thu, 01 Mar 2018 04:57:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909077; cv=none; d=google.com; s=arc-20160816; b=JnI9zwHenqpfesYacaPr3DJwD5CtvT1/6Gdgp0TAmfhJi446+C3ueBDBFLPi2bcqkT pq5/1GVUlbhfCpzWOzWdorXsYGPdLpuspLFNaOA6kbc/XFPJxRDkcaK3Lj2OG52XsDhK 3OzQhLhHaaFPbM+GowUOpUG17vI5/bpyvH7jz4ziCXDjUqXPGjRhzfHJC6NfWnFqVI41 fV+s2EnartI/8E54RYrWOlZIA/kI93W8Ro1+5NPVMvgLu3sg5/PdMfJpHquFxKrIf7R+ gS1d0WCuzJFVuGpuUqej91b/B79+iNRba2BuyGNBTSfiFtKp2n0dq00Fd7MWNRkbm8rS ayJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=zPqSNl85WyC76XW1VxDfQh//hh1Bpcs4lU5Fv1YTRJ0=; b=XsTsuOezolIpezf+cC73vK7shYCGpjbOluRsEHSSwK+kp29KhKOs9DgMvgqNPIJ9YS ITUfIjzdlCUA0DqQ5bocvl3yzgEiE3/oYlObQYcLXIyrHNCi7hdzSgBJY+niN651L3gl XtKLTcvlDvehoxQfOMtYrj9Irr51vndIu9P7uJrdQgRYPbXii5lC4mAlPF9tV8CZXJai JXxv66AvepEmdPbIlVLjpFqHw/y6NQgqgVJj02t0Bqmx2OzM1Hohwnm3Bj4LmFtX7sEX kY8QgmSMWwY57nMsUzIizGbGz1lyzQtO87deBo/ijyeuWEYF1IgSGChRSvVkTEavR0Y0 iICw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RMB40iPH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w62si2442652pgd.573.2018.03.01.04.57.57; Thu, 01 Mar 2018 04:57:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RMB40iPH; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030909AbeCAM5y (ORCPT + 28 others); Thu, 1 Mar 2018 07:57:54 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:44073 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030602AbeCAM5v (ORCPT ); Thu, 1 Mar 2018 07:57:51 -0500 Received: by mail-pf0-f196.google.com with SMTP id 17so2396797pfw.11 for ; Thu, 01 Mar 2018 04:57:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=zPqSNl85WyC76XW1VxDfQh//hh1Bpcs4lU5Fv1YTRJ0=; b=RMB40iPHOPC/sqXJgaYtq5AMH0hSeRu+UcbZkc4e6/xt5Cw/Y2oaDugnEC3sKmG/zZ o04lRqh/27Gd5X+/nW+KF7UO6ejVsz0fHnRWKl4YNk+9Muko30PiaJAMjFblZiaRX9ez f0D1U/YUKbe/GWk3KG2QxXdDWbJyztF1Q72vU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=zPqSNl85WyC76XW1VxDfQh//hh1Bpcs4lU5Fv1YTRJ0=; b=torthYo3+Ts2PDzYq1CRfW5CYZMXVws/OWEukHeBLLqoskmmvVjDUI++TSCVmxFXwT kbb78289St2tgSGgbn4qwTxZg5Sjj7R8u/q1LvUskXD4N/PL8XKUIqC2IYCsSQrWA+ze G9HkeO+80gWdgACtjZuo3X2LQ3cG1ySmxb5TDvgsQoLhn4cOTv2JfoSFL09lk0kQiP9f obAAsgRCnFMlcPV3gfpQAmfJg8fcSR0R7/1VeDLnAYNP90DDtf+mLoOzOMdrqGio5uSf GHWjlXQNSDk7R3G0wU5SX73iZMqFE2q/Lile5pc+qnOoeHEmHUqj8gR33fyEzzee3mFr tD3w== X-Gm-Message-State: APf1xPBeTQvpzYfCtwNKxnUkIm9vpMc+bfYb7gB5bUZYnbEDhZmDECMI a+0m0JNO1wgIJ89HgHmsdB7xIA== X-Received: by 10.101.69.4 with SMTP id n4mr1512030pgq.184.1519909070967; Thu, 01 Mar 2018 04:57:50 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.57.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:57:50 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Suzuki K Poulose , Mark Rutland , Andre Przywara , Dave Martin , Alex Shi Subject: [PATCH 20/45] arm64: Run enable method for errata work arounds on late CPUs Date: Thu, 1 Mar 2018 20:53:57 +0800 Message-Id: <1519908862-11425-21-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Suzuki K Poulose commit 55b35d070c25 upstream. When a CPU is brought up after we have finalised the system wide capabilities (i.e, features and errata), we make sure the new CPU doesn't need a new errata work around which has not been detected already. However we don't run enable() method on the new CPU for the errata work arounds already detected. This could cause the new CPU running without potential work arounds. It is upto the "enable()" method to decide if this CPU should do something about the errata. Fixes: commit 6a6efbb45b7d95c84 ("arm64: Verify CPU errata work arounds on hotplugged CPU") Cc: Will Deacon Cc: Mark Rutland Cc: Andre Przywara Cc: Dave Martin Signed-off-by: Suzuki K Poulose Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kernel/cpu_errata.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index c66a673c..8de43799 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -150,15 +150,18 @@ void verify_local_cpu_errata_workarounds(void) { const struct arm64_cpu_capabilities *caps = arm64_errata; - for (; caps->matches; caps++) - if (!cpus_have_cap(caps->capability) && - caps->matches(caps, SCOPE_LOCAL_CPU)) { + for (; caps->matches; caps++) { + if (cpus_have_cap(caps->capability)) { + if (caps->enable) + caps->enable((void *)caps); + } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) { pr_crit("CPU%d: Requires work around for %s, not detected" " at boot time\n", smp_processor_id(), caps->desc ? : "an erratum"); cpu_die_early(); } + } } void update_cpu_errata_workarounds(void) From patchwork Thu Mar 1 12:53:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130200 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2762576edc; Thu, 1 Mar 2018 04:58:06 -0800 (PST) X-Google-Smtp-Source: AG47ELssfj80015H/guO0VGMmjfHkmrTigr5Cy28Fy5UeKO3haRyjwtydfM4+GdApf2Ya0qvDHs9 X-Received: by 2002:a17:902:9885:: with SMTP id s5-v6mr1897097plp.400.1519909085958; Thu, 01 Mar 2018 04:58:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909085; cv=none; d=google.com; s=arc-20160816; b=s7RNPi14Vv3Jo/roYk4SS70dmrQ2GkCBpoFDqRRem1cBectg3meW8ZtHdDAg988J4Z mZBp7rdfb7KrjkvI/22yivpwob2/yLw3rprc64HPzY6mi/ixwcjX2C73Evi0/Br9Sa+G OUfE0XDjXDKw2RERUjMznh1WY8T9nWy+TD9aG3aZUJCFngX4qfzzQn2pZQ/4oxl3EXAu BDZ8UUZ1ARGclJ4jtt5LUTqjdqQeRrgYuyHVmnGC/KSk83/OpJFPlJqV7efbx81eG914 uwlseFfdFdTMAFuaIEMSXeZEfWvVpmmsAcM1J3sZlb99sRljl9xPRLsC/tLvEsanDNMN 5HJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=mp5ELbwEmYRLHeQ3bnHDHxexdMKPPHMhOHF8S+YF3Ac=; b=treIJuo5SMNow+d+yIEXrGsxbZjHvZrJUi4YAB/fJIK4pcX5HzSnScurtPT3XJo7WC MK/NiTmoen+uO+90a6chktpP46UCd71ulxF+NScC3EqAOtwWPbtkY9ZWHkPUNeVuImWN egEdzv1Akzbml9M3v8t+FXvAxO7fZhH034UPo3HEVFRYJkOYN0+/3olCRNb48wPXP2s2 GBPBHhsfyVch01V+JNneY5GAXSoNOiznbjHI/7N7+pomGWiVEUM7wuRxkjJVAGPW1oMQ j8n2rfF6503iGGVeh7T944+UE1NzRJ/KiMSmVYVBdA7J3nDRM9z9UyxO5liq7ArjUkMX l+pQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=c3I9UT93; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w62si2442652pgd.573.2018.03.01.04.58.05; Thu, 01 Mar 2018 04:58:05 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=c3I9UT93; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030924AbeCAM6B (ORCPT + 28 others); Thu, 1 Mar 2018 07:58:01 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:41700 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030911AbeCAM56 (ORCPT ); Thu, 1 Mar 2018 07:57:58 -0500 Received: by mail-pf0-f195.google.com with SMTP id f80so2399668pfa.8 for ; Thu, 01 Mar 2018 04:57:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=mp5ELbwEmYRLHeQ3bnHDHxexdMKPPHMhOHF8S+YF3Ac=; b=c3I9UT93a9gl4iZ5XmQ52VlxvjAEr/8R2RbP1GwsMm1BMHfJStTzlh3Nhgh2vRA0tm nXzXLXeT2lRrvjbdBcHQIgqM0iJMPBQL0B7dZpAcokZBrKfo2w/E5h4JAc2xc0iekS+q OOUiUBIDTU3jYUa3THNcMd/SvhIsCW1TP/APg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=mp5ELbwEmYRLHeQ3bnHDHxexdMKPPHMhOHF8S+YF3Ac=; b=PjD1Z5SqQAf1VMQWonkiAML7xjzPWPE568vOvvmdLN+yEv3UQvad1dHyFWGtpKPfno GMtW7MRhA0xjq69TOHAlFW9eZkr9MzBEl/heLBkCwbtb543BGnj6a+nM6cg3imZXUxnH JE5K1Ho1MHq6ybKlgo3kj9XEr5W3iMlFPmTzHHvrIDtaYbOG1FKDQyaOvcezv3sC+gYs 1dSHWC470HenM03lxDhzl4+A//sM4W7n+thg6qyUAEOzZDewtmKubLw56PurtOgMmzQr Xe46kCyMfSn3cq2QTBQGJoKqcCQvF7ls0X1Sgg1T3AA0McztedJRjxBUQzJwCEaEwbQA Qx4A== X-Gm-Message-State: APf1xPBmzFGbrWiC6aCAZkemK1+w759A/UupMDxaI6Dl5YbHke0x/gfh suZeZc1Jg7yN/CDH7hr/4FU2XQ== X-Received: by 10.101.77.69 with SMTP id j5mr1490545pgt.352.1519909077807; Thu, 01 Mar 2018 04:57:57 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.57.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:57:57 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 21/45] arm64: cpufeature: Pass capability structure to ->enable callback Date: Thu, 1 Mar 2018 20:53:58 +0800 Message-Id: <1519908862-11425-22-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 0a0d111d40fd upstream. In order to invoke the CPU capability ->matches callback from the ->enable callback for applying local-CPU workarounds, we need a handle on the capability structure. This patch passes a pointer to the capability structure to the ->enable callback. Reviewed-by: Suzuki K Poulose Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kernel/cpufeature.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index fcb67e0..5c41ef6 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -949,7 +949,7 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps) * uses an IPI, giving us a PSTATE that disappears when * we return. */ - stop_machine(caps->enable, NULL, cpu_online_mask); + stop_machine(caps->enable, (void *)caps, cpu_online_mask); } /* @@ -1005,7 +1005,7 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps) cpu_die_early(); } if (caps->enable) - caps->enable(NULL); + caps->enable((void *)caps); } } From patchwork Thu Mar 1 12:54:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130202 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2763126edc; Thu, 1 Mar 2018 04:58:48 -0800 (PST) X-Google-Smtp-Source: AG47ELvV6FBi6QsZhEST646Q8mkNLZOXDicNl0vBQG+cD9fgyXPqdGoqM9fjnL5Gqkkcpa1GbQGB X-Received: by 10.101.70.203 with SMTP id n11mr1422733pgr.377.1519909128108; Thu, 01 Mar 2018 04:58:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909128; cv=none; d=google.com; s=arc-20160816; b=HogS4hPvbe93swqk5PXdTNMXgJx9P2zirWmEx7q5TNrfH56PNr+Qv+T5JniUtDP6Lj axuW+QOXOrNcGwV2kqOfCiLHcG1ZW1KTbsQ+x3fetjz24DW9q1a4OCnkdTuMyoGLyknF qTnm5xwy6joOoxdWo2jPt5LjrMhK3XrCCNpqhDmQqQCjQTgBSui3NbESfGOV9cf8QmCg hnJ4n9yyKmYE8k0a7O4ZJ5Ykp0sgTMblqNGEzFLDO99PYZzwtbiZnmBeeI4D9Wx86x/T IjLreXCS3lcgxVJNeE4aXoCkgKGUqRtAx7O8djCux7xK52EO9BeBshljyXgi28p9Dclt DcmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=LTZ3hl6u/7+oQ/R6V0y/q5Po/IW9w8cPYog3KcYjLHo=; b=ZoeaxqybGFaJNuzSWxYapRuLc995/roP3norj44Ixu+imyfDh/Kun2OLOaeRnz85w/ lAn/jgvmmrYWK/icD4fstDjWcSI9bmJ74qbhoRiroNU4jbviNBtczs/0tvCUgPBqRRCd MewO02A+TzuCpNz3KwtbMUxk7AYJUOVu1Bn8xCMs4MOiijqPNLkJTv8W0E/tKTBjj3MX zObNK3fGY017L2p1XUvzQzXbnozMw9Kqb5gKJtc356o3AbgH+FL/0OYQSuPJkRagb075 oabOz58LqcTBr826kxZvByLAXbNUzrWe4Ak6WmYUNx7fWsX8vsZ+PrGmXFzItiqunSc6 RT0Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MiD+Ntqk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m10si2472888pgs.236.2018.03.01.04.58.47; Thu, 01 Mar 2018 04:58:48 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MiD+Ntqk; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030943AbeCAM6o (ORCPT + 28 others); Thu, 1 Mar 2018 07:58:44 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:33637 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030930AbeCAM6k (ORCPT ); Thu, 1 Mar 2018 07:58:40 -0500 Received: by mail-pf0-f194.google.com with SMTP id q13so2412488pff.0 for ; Thu, 01 Mar 2018 04:58:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LTZ3hl6u/7+oQ/R6V0y/q5Po/IW9w8cPYog3KcYjLHo=; b=MiD+NtqkzxqbZo/n+0pMTie4pwG33gZoi/5glcIsKMj44S/18XCxfaoTf/qQsLTxn1 xY//l+o8smVrWpVA7yN7Z8nmHIXMnPs386wtqzpyVWqzAmDsPRVSo7ZqkHYItqbN+kvq qO+Ged2R+zK26Rj2cXU1l1rx4oLPshPyaZ9tI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LTZ3hl6u/7+oQ/R6V0y/q5Po/IW9w8cPYog3KcYjLHo=; b=CGDCbi5JOWlPyTIl/MrprDYO3m2I5lfzeRAW1RGZFEBa+V+XSkBHqRXMYX03MovIey PgF4dzu812r0fsHhPUx7y4lj7M8a1JZWljtXtP3sF8L0dGVaOshhVjFOTJ98U+aPz9hh UkJsFeByzLXK1nPJ7KJWuufmSoMFNuytzkvYMtCsFUWX/ztlaDvOXaDEXSgqZVwHq26c 8EbOb5HdW4PGhI2oPpn0xTaQOkmJX0m9p6k6AVij8rotbccNIRJl3y3f09jtzYT1vcqm 2nWWwfJOq0FJLrHF9RMN3ZMHLpqoqqKbcNSLOAaNZTIcTf34jGPRvtS6+A3g31SxfdYj i8LA== X-Gm-Message-State: APf1xPDCcEol5oyTzzRF1zDMdlRabqaUE1i+1EuiqfNF1uw1m7ad5cyE ZvzDdZBU9cjuLTtDuULos2TTcjV+hPA= X-Received: by 10.98.49.135 with SMTP id x129mr1853816pfx.75.1519909119399; Thu, 01 Mar 2018 04:58:39 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.58.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:58:38 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 23/45] arm64: Add skeleton to harden the branch predictor against aliasing attacks Date: Thu, 1 Mar 2018 20:54:00 +0800 Message-Id: <1519908862-11425-24-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 0f15adbb2861 upstream. Aliasing attacks against CPU branch predictors can allow an attacker to redirect speculative control flow on some CPUs and potentially divulge information from one context to another. This patch adds initial skeleton code behind a new Kconfig option to enable implementation-specific mitigations against these attacks for CPUs that are affected. Co-developed-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/Kconfig | 17 +++++++++ arch/arm64/include/asm/cpucaps.h | 3 +- arch/arm64/include/asm/mmu.h | 38 +++++++++++++++++++++ arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/kernel/Makefile | 4 +++ arch/arm64/kernel/bpi.S | 55 +++++++++++++++++++++++++++++ arch/arm64/kernel/cpu_errata.c | 74 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/cpufeature.c | 1 + arch/arm64/kernel/entry.S | 8 +++-- arch/arm64/mm/context.c | 2 ++ arch/arm64/mm/fault.c | 17 +++++++++ include/linux/mm_types.h | 1 + 12 files changed, 217 insertions(+), 4 deletions(-) create mode 100644 arch/arm64/kernel/bpi.S -- 2.7.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7769c2e..0c4be63 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -733,6 +733,23 @@ config FORCE_MAX_ZONEORDER However for 4K, we choose a higher default value, 11 as opposed to 10, giving us 4M allocations matching the default size used by generic code. +config HARDEN_BRANCH_PREDICTOR + bool "Harden the branch predictor against aliasing attacks" if EXPERT + default y + help + Speculation attacks against some high-performance processors rely on + being able to manipulate the branch predictor for a victim context by + executing aliasing branches in the attacker context. Such attacks + can be partially mitigated against by clearing internal branch + predictor state and limiting the prediction logic in some situations. + + This config option will take CPU-specific actions to harden the + branch predictor against aliasing attacks and may rely on specific + instruction sequences or control bits being set by the system + firmware. + + If unsure, say Y. + menuconfig ARMV8_DEPRECATED bool "Emulate deprecated/obsolete ARMv8 instructions" depends on COMPAT diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 87b4465..f8b7799 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -34,7 +34,8 @@ #define ARM64_HAS_32BIT_EL0 13 #define ARM64_HYP_OFFSET_LOW 14 #define ARM64_MISMATCHED_CACHE_LINE_SIZE 15 +#define ARM64_HARDEN_BRANCH_PREDICTOR 16 -#define ARM64_NCAPS 16 +#define ARM64_NCAPS 17 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index b075140..203974c 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -28,6 +28,44 @@ typedef struct { */ #define ASID(mm) ((mm)->context.id.counter & 0xffff) + +typedef void (*bp_hardening_cb_t)(void); + +struct bp_hardening_data { + int hyp_vectors_slot; + bp_hardening_cb_t fn; +}; + +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +extern char __bp_harden_hyp_vecs_start[], __bp_harden_hyp_vecs_end[]; + +DECLARE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); + +static inline struct bp_hardening_data *arm64_get_bp_hardening_data(void) +{ + return this_cpu_ptr(&bp_hardening_data); +} + +static inline void arm64_apply_bp_hardening(void) +{ + struct bp_hardening_data *d; + + if (!cpus_have_cap(ARM64_HARDEN_BRANCH_PREDICTOR)) + return; + + d = arm64_get_bp_hardening_data(); + if (d->fn) + d->fn(); +} +#else +static inline struct bp_hardening_data *arm64_get_bp_hardening_data(void) +{ + return NULL; +} + +static inline void arm64_apply_bp_hardening(void) { } +#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */ + extern void paging_init(void); extern void bootmem_init(void); extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt); diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 7393cc7..e91710f 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -117,6 +117,7 @@ #define ID_AA64ISAR0_AES_SHIFT 4 /* id_aa64pfr0 */ +#define ID_AA64PFR0_CSV2_SHIFT 56 #define ID_AA64PFR0_GIC_SHIFT 24 #define ID_AA64PFR0_ASIMD_SHIFT 20 #define ID_AA64PFR0_FP_SHIFT 16 diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 7d66bba..74b8fd8 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -51,6 +51,10 @@ arm64-obj-$(CONFIG_HIBERNATION) += hibernate.o hibernate-asm.o arm64-obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o \ cpu-reset.o +ifeq ($(CONFIG_KVM),y) +arm64-obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o +endif + obj-y += $(arm64-obj-y) vdso/ probes/ obj-m += $(arm64-obj-m) head-y := head.o diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S new file mode 100644 index 0000000..06a931e --- /dev/null +++ b/arch/arm64/kernel/bpi.S @@ -0,0 +1,55 @@ +/* + * Contains CPU specific branch predictor invalidation sequences + * + * Copyright (C) 2018 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +.macro ventry target + .rept 31 + nop + .endr + b \target +.endm + +.macro vectors target + ventry \target + 0x000 + ventry \target + 0x080 + ventry \target + 0x100 + ventry \target + 0x180 + + ventry \target + 0x200 + ventry \target + 0x280 + ventry \target + 0x300 + ventry \target + 0x380 + + ventry \target + 0x400 + ventry \target + 0x480 + ventry \target + 0x500 + ventry \target + 0x580 + + ventry \target + 0x600 + ventry \target + 0x680 + ventry \target + 0x700 + ventry \target + 0x780 +.endm + + .align 11 +ENTRY(__bp_harden_hyp_vecs_start) + .rept 4 + vectors __kvm_hyp_vector + .endr +ENTRY(__bp_harden_hyp_vecs_end) diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 8de43799..0e07893 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -46,6 +46,80 @@ static int cpu_enable_trap_ctr_access(void *__unused) return 0; } +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +#include +#include + +DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); + +#ifdef CONFIG_KVM +static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start, + const char *hyp_vecs_end) +{ + void *dst = lm_alias(__bp_harden_hyp_vecs_start + slot * SZ_2K); + int i; + + for (i = 0; i < SZ_2K; i += 0x80) + memcpy(dst + i, hyp_vecs_start, hyp_vecs_end - hyp_vecs_start); + + flush_icache_range((uintptr_t)dst, (uintptr_t)dst + SZ_2K); +} + +static void __install_bp_hardening_cb(bp_hardening_cb_t fn, + const char *hyp_vecs_start, + const char *hyp_vecs_end) +{ + static int last_slot = -1; + static DEFINE_SPINLOCK(bp_lock); + int cpu, slot = -1; + + spin_lock(&bp_lock); + for_each_possible_cpu(cpu) { + if (per_cpu(bp_hardening_data.fn, cpu) == fn) { + slot = per_cpu(bp_hardening_data.hyp_vectors_slot, cpu); + break; + } + } + + if (slot == -1) { + last_slot++; + BUG_ON(((__bp_harden_hyp_vecs_end - __bp_harden_hyp_vecs_start) + / SZ_2K) <= last_slot); + slot = last_slot; + __copy_hyp_vect_bpi(slot, hyp_vecs_start, hyp_vecs_end); + } + + __this_cpu_write(bp_hardening_data.hyp_vectors_slot, slot); + __this_cpu_write(bp_hardening_data.fn, fn); + spin_unlock(&bp_lock); +} +#else +static void __install_bp_hardening_cb(bp_hardening_cb_t fn, + const char *hyp_vecs_start, + const char *hyp_vecs_end) +{ + __this_cpu_write(bp_hardening_data.fn, fn); +} +#endif /* CONFIG_KVM */ + +static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry, + bp_hardening_cb_t fn, + const char *hyp_vecs_start, + const char *hyp_vecs_end) +{ + u64 pfr0; + + if (!entry->matches(entry, SCOPE_LOCAL_CPU)) + return; + + pfr0 = read_cpuid(ID_AA64PFR0_EL1); + if (cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_CSV2_SHIFT)) + return; + + __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end); +} +#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */ + #define MIDR_RANGE(model, min, max) \ .def_scope = SCOPE_LOCAL_CPU, \ .matches = is_affected_midr_range, \ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 5c41ef6..6e7fda3 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -98,6 +98,7 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { ARM64_FTR_BITS(FTR_STRICT, FTR_EXACT, ID_AA64PFR0_GIC_SHIFT, 4, 0), S_ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI), S_ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI), + ARM64_FTR_BITS(FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0), /* Linux doesn't care about the EL3 */ ARM64_FTR_BITS(FTR_NONSTRICT, FTR_EXACT, ID_AA64PFR0_EL3_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_STRICT, FTR_EXACT, ID_AA64PFR0_EL2_SHIFT, 4, 0), diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 0a27e12..bdb0139 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -549,13 +549,15 @@ el0_ia: * Instruction abort handling */ mrs x26, far_el1 - // enable interrupts before calling the main handler - enable_dbg_and_irq + msr daifclr, #(8 | 4 | 1) +#ifdef CONFIG_TRACE_IRQFLAGS + bl trace_hardirqs_off +#endif ct_user_exit mov x0, x26 mov x1, x25 mov x2, sp - bl do_mem_abort + bl do_el0_ia_bp_hardening b ret_to_user el0_fpsimd_acc: /* diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 32eeabe91..afc9266 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -231,6 +231,8 @@ asmlinkage void post_ttbr_update_workaround(void) "ic iallu; dsb nsh; isb", ARM64_WORKAROUND_CAVIUM_27456, CONFIG_CAVIUM_ERRATUM_27456)); + + arm64_apply_bp_hardening(); } static int asids_init(void) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 4df70c9..c95b194 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -590,6 +590,23 @@ asmlinkage void __exception do_mem_abort(unsigned long addr, unsigned int esr, arm64_notify_die("", regs, &info, esr); } +asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr, + unsigned int esr, + struct pt_regs *regs) +{ + /* + * We've taken an instruction abort from userspace and not yet + * re-enabled IRQs. If the address is a kernel address, apply + * BP hardening prior to enabling IRQs and pre-emption. + */ + if (addr > TASK_SIZE) + arm64_apply_bp_hardening(); + + local_irq_enable(); + do_mem_abort(addr, esr, regs); +} + + /* * Handle stack alignment exceptions. */ diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index e8471c2..15a82f3 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include From patchwork Thu Mar 1 12:54:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130203 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2763250edc; Thu, 1 Mar 2018 04:59:00 -0800 (PST) X-Google-Smtp-Source: AG47ELvNDWP1bFNPknXk2NCedpmeKEazWS4uXzCH224RVe262ARfoXdgeLLphi5p4HsUod3I3anP X-Received: by 2002:a17:902:8289:: with SMTP id y9-v6mr1866365pln.242.1519909139946; Thu, 01 Mar 2018 04:58:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909139; cv=none; d=google.com; s=arc-20160816; b=uF8f4g1eE/950eCaceLNfjtOmAEcETf3DGAMOKloG+gvA4B1Ic4jU/xPnUru/bN2du 2Y77mdFK4a9glHlvyr6vlyutU6ZlGfUIURjEH67S+yJ36eaBUxpGSFQfGk3gkXNaFB/n sytc5IpVihmvNy13CGgVx3oQZuQdgEe5ZF3ldUkM/SCg1BbPCTYShYtFeSABvIWkn+Un SKRfUky9eMJ8z+0PKFRitguB/Pgb1CExOEvtFQiXc6dVYW1maZZmSUVOqfXQJl+AmC9u IQ3ERMv648tm5JsEfaSsGB8oyfd7J7dJweDWNTFwrVCU0Y5+8wXy2L1wWWJKr1OI46SU oIIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=YTi+3EgVxwTxApY3ucUH/g1LD9enZczbL8ouc6EbVOA=; b=TCovxmcR8up70SCjA7IjlV3iYTOBZKqFU7PuuOjoiqFXIo0tR3Hu4M5Y7OvqTEyslF Cv3MvI9KAG2me8KoVzSgbwoTNoW/vyufUwV36fVDpQAQwQ3eKNvaTQQYZ7dzlFa3DdwS T4TtYpZQrDdDFuBvLtmXTQQBFTErLWht6dnUQKhWRSqEdxVjDg/gcduU6W78keMTdLPU lkTvgzvALPFie/8n05zbza1fMNiLL3im6QhQ9ACcaBKf7sN3MEBA0gJxxeVHvSkJjack squxh1PSFlpBhJpdz5pxcpK7dsiX9EJP/lFtv7P7kriqLSyLa8m11hQ4Jk8AvTNP6aGT XuQA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YOD8Yuf8; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m10si2472888pgs.236.2018.03.01.04.58.59; Thu, 01 Mar 2018 04:58:59 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YOD8Yuf8; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030957AbeCAM64 (ORCPT + 28 others); Thu, 1 Mar 2018 07:58:56 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:36317 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030753AbeCAM6r (ORCPT ); Thu, 1 Mar 2018 07:58:47 -0500 Received: by mail-pl0-f65.google.com with SMTP id 61-v6so3597205plf.3 for ; Thu, 01 Mar 2018 04:58:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YTi+3EgVxwTxApY3ucUH/g1LD9enZczbL8ouc6EbVOA=; b=YOD8Yuf8Eht7WwsO2JikzfyCrD/zUq3mDMabSjHjiBmlZ7LEMeNgKyqNYRMTppc499 AKj8idWMxjeL7qN6Oo9fxNE8q7JAJdy0toCtrOanbS47W/vuAcgTaf9VCKVL27przxMd QeVDBxPnK9PsrR3SzNx+ME1pZBJfYGjrS1q+Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YTi+3EgVxwTxApY3ucUH/g1LD9enZczbL8ouc6EbVOA=; b=NQluhonxdhagYkkqTmOgJgruCgWL3CGl/M/Bdpb4RVydDdknidrrj2dg32wlp7CVqm +QmeRPBnjJmAg96GQO6UPPVUwn2iBXj34krJ5xLCNPQve8rHEU0+OzfVZAyApE2WqVGz QhAhdvptP8O9IK+sNXm9k/gAgW8oXqxEWMEoiAKy35SGAuMN5cbyWTOpqHXjdBm0lMVF xdH3aEiNxYeXeJir1QsLyAQk31/Nry70ohc7TGzq636jrJZwhU2uIAwFszS/cshXBoH2 F5o+EaoVIV3G/w3R+oljXEfQSHU1HXjZmTi3eCZtC5ZtylgLMfpEcuhuA19xD3UvhrYd LoHA== X-Gm-Message-State: APf1xPC8seGTCJLHwWfSadyVnXKxAglrtfKftz11viCs3ahy5n48rS2Q cQAVjjQp4S4+A+2/OPIJzh+wfg== X-Received: by 2002:a17:902:f44:: with SMTP id 62-v6mr1819010ply.27.1519909126862; Thu, 01 Mar 2018 04:58:46 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.58.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:58:46 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 24/45] arm64: Move BP hardening to check_and_switch_context Date: Thu, 1 Mar 2018 20:54:01 +0800 Message-Id: <1519908862-11425-25-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit a8e4c0a919ae upstream. We call arm64_apply_bp_hardening() from post_ttbr_update_workaround, which has the unexpected consequence of being triggered on every exception return to userspace when ARM64_SW_TTBR0_PAN is selected, even if no context switch actually occured. This is a bit suboptimal, and it would be more logical to only invalidate the branch predictor when we actually switch to a different mm. In order to solve this, move the call to arm64_apply_bp_hardening() into check_and_switch_context(), where we're guaranteed to pick a different mm context. Acked-by: Will Deacon Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/mm/context.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index afc9266..36416e4 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -221,6 +221,7 @@ void check_and_switch_context(struct mm_struct *mm, unsigned int cpu) raw_spin_unlock_irqrestore(&cpu_asid_lock, flags); switch_mm_fastpath: + arm64_apply_bp_hardening(); cpu_switch_mm(mm->pgd, mm); } @@ -231,8 +232,6 @@ asmlinkage void post_ttbr_update_workaround(void) "ic iallu; dsb nsh; isb", ARM64_WORKAROUND_CAVIUM_27456, CONFIG_CAVIUM_ERRATUM_27456)); - - arm64_apply_bp_hardening(); } static int asids_init(void) From patchwork Thu Mar 1 12:54:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130224 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2771073edc; Thu, 1 Mar 2018 05:04:35 -0800 (PST) X-Google-Smtp-Source: AG47ELuYK1dyQkaFtE1DKKy/W3q8KmxaZ2nzJwZmJqnV040wEqaEff2CE/obIdwvs5+BrHyRqgxc X-Received: by 10.99.96.200 with SMTP id u191mr1519271pgb.252.1519909475062; Thu, 01 Mar 2018 05:04:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909475; cv=none; d=google.com; s=arc-20160816; b=xgbmzkbtUbjkngg7czylbG676TEFWw60JZLMDPA91rxxdz7azQQl/qBfvs+OGyB1KQ OR2g+x5eF9J18JRryHCgmUqQCTVRJCSSlrtAhq6qnp5M/yALTLG30JglT8sd+kGyAjid S9T2dg2MOzZ4WV49ec+rqNng2Qm3JtytD672Fa2C8KgvoiyWIRKKnoZZNQYSeYyTYY44 mp4grbU/FVi4aXmVG+jZQT9f7PmlcKZwiDylOhQAfnwPqO9/doFAxXhsBs01Kdy33VLY 0Coq3GdqotwJ9xmMi0F8tRPh16PRLfq4b0l1pDJHRCrm35cRV2UfXZr1HAJqx8O8GV8L XgIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=NZ2qtdpybzZDjgWGLMAPxPIrjxBvuuwvSlz1/mXg4z0=; b=eFiduEAYuiYbWgNkVejf2M3PhXyCb1Nfiqr2Oh5ZJCbB/czzpvfIbJKumLSnEA35oz 8PH2cMbfUdWYB1GpPCMivfXfVtNrx2SkYR9brhys4c+2IvTH4kMqDmr2pfuKWi+HB7TY JTrCfbejSn/ioiyQ7pL8FoDCc5nEZ0BYQSH2TH6HELX2wNqPtAWdg/NY+etPPGNB2QCS hxSnQgRPa+7nqgEEfV8DevBQtp9MW+uOvhwFjbRgUHvTvleQ7jKFYOVGJsj7HpLv483b yHjfqggxXi/LJQ+TDUG6FcgE1kOOAI2cD3wlcTzQbGRbwDpDszAk0OobPmeggyI8dn2H Fr0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HRBfzix6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e13si2441046pgt.569.2018.03.01.05.04.34; Thu, 01 Mar 2018 05:04:35 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HRBfzix6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031114AbeCANEa (ORCPT + 28 others); Thu, 1 Mar 2018 08:04:30 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:36970 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030654AbeCAM6x (ORCPT ); Thu, 1 Mar 2018 07:58:53 -0500 Received: by mail-pg0-f66.google.com with SMTP id y26so2276093pgv.4 for ; Thu, 01 Mar 2018 04:58:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=NZ2qtdpybzZDjgWGLMAPxPIrjxBvuuwvSlz1/mXg4z0=; b=HRBfzix657WyZtW2j/hHUvw3T0vT+NmV9/CkteQKq78WzaWl7nAJTkr9ejDlnpVxLm lczwa0QTAidlAfX3q1YI/0SonGtjMuTeaYobaU2aGsOPGyk9vsbjggfgpY93gVotf6Nx ibKhfBQWu1vNK2rZkSz6m4XEIZXrpgSkk6Eqk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=NZ2qtdpybzZDjgWGLMAPxPIrjxBvuuwvSlz1/mXg4z0=; b=TUdrdVIya4xFQxy8sMHrJzhBlQKQMJJWDGEb4WS4mP5KpAfxhign+k1O0dwPdwQaz3 IavB/xcF7+v88d9kLHRbSsZEucf7C5cLP3hON0a1Z0u3GDy/lIpfRacihmDmaPQOk/zb Kd0+0e8FRM2nREbdzy6eASDyJZ4gLZrpILZKSwkOTu+JsgYvZMZMuul/zEuTHMFsVgOc q+OGaZvqJ2N2z/4FCU1csozlhcs29AnRj1F241KDs7ArjoNQqqUjkb7ysffQvAhXioyM Yk82GqRX+mFEUdyHH9usu8SzyvKKQMDs89Km4YIxk2aeXeVjTOEU9LUFvYviexoVCZEZ 0y8w== X-Gm-Message-State: APf1xPDCvc9aYI4oqGyead/KQNdrJIsIMkoEo10L1dojx40dCnYLR8n3 OLbG4qTCcv9WxjEKSyswGJJmJw== X-Received: by 10.99.53.193 with SMTP id c184mr1469471pga.180.1519909133280; Thu, 01 Mar 2018 04:58:53 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.58.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:58:52 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 25/45] arm64: KVM: Use per-CPU vector when BP hardening is enabled Date: Thu, 1 Mar 2018 20:54:02 +0800 Message-Id: <1519908862-11425-26-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 6840bdd73d07 upstream Now that we have per-CPU vectors, let's plug then in the KVM/arm64 code. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/include/asm/kvm_mmu.h | 10 ++++++++++ arch/arm/kvm/arm.c | 8 +++++++- arch/arm64/include/asm/kvm_mmu.h | 38 ++++++++++++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/switch.c | 2 +- 4 files changed, 56 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index a58bbaa..d10e362 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -223,6 +223,16 @@ static inline unsigned int kvm_get_vmid_bits(void) return 8; } +static inline void *kvm_get_hyp_vector(void) +{ + return kvm_ksym_ref(__kvm_hyp_vector); +} + +static inline int kvm_map_vectors(void) +{ + return 0; +} + #endif /* !__ASSEMBLY__ */ #endif /* __ARM_KVM_MMU_H__ */ diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index c38bfbe..4e4ae07 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -1088,7 +1088,7 @@ static void cpu_init_hyp_mode(void *dummy) pgd_ptr = kvm_mmu_get_httbr(); stack_page = __this_cpu_read(kvm_arm_hyp_stack_page); hyp_stack_ptr = stack_page + PAGE_SIZE; - vector_ptr = (unsigned long)kvm_ksym_ref(__kvm_hyp_vector); + vector_ptr = (unsigned long)kvm_get_hyp_vector(); __cpu_init_hyp_mode(pgd_ptr, hyp_stack_ptr, vector_ptr); __cpu_init_stage2(); @@ -1345,6 +1345,12 @@ static int init_hyp_mode(void) goto out_err; } + err = kvm_map_vectors(); + if (err) { + kvm_err("Cannot map vectors\n"); + goto out_err; + } + /* * Map the Hyp stack pages */ diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index 6d22017..80bf337 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -313,5 +313,43 @@ static inline unsigned int kvm_get_vmid_bits(void) return (cpuid_feature_extract_unsigned_field(reg, ID_AA64MMFR1_VMIDBITS_SHIFT) == 2) ? 16 : 8; } +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +#include + +static inline void *kvm_get_hyp_vector(void) +{ + struct bp_hardening_data *data = arm64_get_bp_hardening_data(); + void *vect = kvm_ksym_ref(__kvm_hyp_vector); + + if (data->fn) { + vect = __bp_harden_hyp_vecs_start + + data->hyp_vectors_slot * SZ_2K; + + if (!cpus_have_cap(ARM64_HAS_VIRT_HOST_EXTN)) + vect = lm_alias(vect); + } + + return vect; +} + +static inline int kvm_map_vectors(void) +{ + return create_hyp_mappings(kvm_ksym_ref(__bp_harden_hyp_vecs_start), + kvm_ksym_ref(__bp_harden_hyp_vecs_end), + PAGE_HYP_EXEC); +} + +#else +static inline void *kvm_get_hyp_vector(void) +{ + return kvm_ksym_ref(__kvm_hyp_vector); +} + +static inline int kvm_map_vectors(void) +{ + return 0; +} +#endif + #endif /* __ASSEMBLY__ */ #endif /* __ARM64_KVM_MMU_H__ */ diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 0c848c1..cf6d962 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -50,7 +50,7 @@ static void __hyp_text __activate_traps_vhe(void) val &= ~CPACR_EL1_FPEN; write_sysreg(val, cpacr_el1); - write_sysreg(__kvm_hyp_vector, vbar_el1); + write_sysreg(kvm_get_hyp_vector(), vbar_el1); } static void __hyp_text __activate_traps_nvhe(void) From patchwork Thu Mar 1 12:54:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130205 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2763414edc; Thu, 1 Mar 2018 04:59:14 -0800 (PST) X-Google-Smtp-Source: AG47ELtedbmf3rICrtGNafOg00jg2fTgp391HvAWzSWJ5hnJWcvE/ZPW6vSti0UeC1sZmZZxJGUy X-Received: by 10.98.35.195 with SMTP id q64mr1831459pfj.161.1519909154278; Thu, 01 Mar 2018 04:59:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909154; cv=none; d=google.com; s=arc-20160816; b=dqBrUcdBH0RcMa8KGn+Qd5U1AlisidRBPovoRonMUftXT8wAcELVCXCTkO5DQrfPD9 +r8skfoyWuNvQ2999PQ7rMiPnqP0R3rGCXqMEU1JRnpDXB9pZ/HxKOQ6BFZMdt77biBs Pn0SesA4T6dvVN/O4WzhGmQTApu2ewnjz8cgjnP0dpNgY2ajSpYcqAgnBCyD3UtJQor0 4sGQRnJ6IUoVSs6yTLiw5LRJx7ritT7aSHPONb3IN41a7uzFjeSTUNNihsAu4bL511WC ehZofjep9lCg1QnJWeSB/ChjJSiSOFKhg39TqmIbbPAWuHsb5nOgOoiFE1/sNxQr8TR8 q/aA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=o9wt4qxRgviZ/SM2aAeUByaDVo70FE/AaFiMiokIiaM=; b=uq2MRcidl4UTVuiRZ8kF9YKOeJJl9AoeZSzLZO276YMnExq5//zxLTz/2Pxtj+3qcn iGuSVAYinRhV/d7mHKJ2jZUMlWk9YUWklmdJ65vBThCb2fOZQLBiKqAkgoKnxqMBjD4D mc9ptVMUSvIFElWvSVwBkju5Qu3eRIrlwcr/KxUaSCWddW3WENnVFKtbVXyihJzTMn3T I52OVAUXzE6Im1QKZm45B+4h615tXcmAe8kbN5YYHv/6kX+luk7arYX2oEK/GWbjsMGP ZPCkvSMhMHW3MFVrzUxQpW04iIatENt3qIVmwKTTb7Z8RQmoh+Gqucoh2mo1u76yqPMn nrtA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=g3bSVM/I; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r12si2461227pgt.200.2018.03.01.04.59.14; Thu, 01 Mar 2018 04:59:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=g3bSVM/I; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030984AbeCAM7L (ORCPT + 28 others); Thu, 1 Mar 2018 07:59:11 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:43380 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030779AbeCAM7H (ORCPT ); Thu, 1 Mar 2018 07:59:07 -0500 Received: by mail-pl0-f65.google.com with SMTP id f23-v6so3586608plr.10 for ; Thu, 01 Mar 2018 04:59:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=o9wt4qxRgviZ/SM2aAeUByaDVo70FE/AaFiMiokIiaM=; b=g3bSVM/I2Ae2TbP7y3xsFIsoL2bwk7RAYi8r/DuRx7AjqOtNXe+Q6Xd1PH5lyobZL4 Mmx5Am3+envCjO0U75QBoyfZVpFiLmfHRHvLnM0rhcGlPyHIyrSg3lKYvbkMWojhlmoa IoYJWNjH8Z8T5/nDjbnU3kPco/m9bQ9NK8uzg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=o9wt4qxRgviZ/SM2aAeUByaDVo70FE/AaFiMiokIiaM=; b=Z1MS+9Qa84/S5ZgOM8kUmbAWwrX8qVgEOpCYUhV2P5UTrbBI4TLTtC8FxLWtIryriJ /IbfN+t6p7pQXRiddZ4OTQN3tLN+xezE+n3DYwIP4BoRAgLg6WjbWjXPbRqga2wa+hds FD0fTv//ljMFpXwIqajmnM/XBlsQvFcJshSWaUtQnRr3RHPfGIYOmPcL6kCBn+4dAMfm +ZmubG73T0nMKOrCGydKE4A0o+fIQy7C9bGAUCTS6FvnxQ3itOwJkTOJkujLwB6Efz6P 0iPR2vQHWw1PHxeWh+E4CbdAwyZ7FHaCEwRy5oBpa2D1MmMAB5UZNeXQF3nHZDLXcAEE njnA== X-Gm-Message-State: APf1xPB8fYzZca+47jdi9Dbte763tABu+eqSWSbPVDzuGpmlfHRxulRY FO0Gv/+1kpYLX0ULpDsMZaEYZQ== X-Received: by 2002:a17:902:22f:: with SMTP id 44-v6mr1843963plc.418.1519909147140; Thu, 01 Mar 2018 04:59:07 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.59.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:59:06 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 27/45] arm64: entry: Apply BP hardening for suspicious interrupts from EL0 Date: Thu, 1 Mar 2018 20:54:04 +0800 Message-Id: <1519908862-11425-28-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 30d88c0e3ace upstream. It is possible to take an IRQ from EL0 following a branch to a kernel address in such a way that the IRQ is prioritised over the instruction abort. Whilst an attacker would need to get the stars to align here, it might be sufficient with enough calibration so perform BP hardening in the rare case that we see a kernel address in the ELR when handling an IRQ from EL0. Reported-by: Dan Hettena Reviewed-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/kernel/entry.S | 5 +++++ arch/arm64/mm/fault.c | 6 ++++++ 2 files changed, 11 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index d50c2fe..e26a114 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -646,6 +646,11 @@ el0_irq_naked: #endif ct_user_exit +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + tbz x22, #55, 1f + bl do_el0_irq_bp_hardening +1: +#endif irq_handler #ifdef CONFIG_TRACE_IRQFLAGS diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 6120a14..ad49ae8 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -590,6 +590,12 @@ asmlinkage void __exception do_mem_abort(unsigned long addr, unsigned int esr, arm64_notify_die("", regs, &info, esr); } +asmlinkage void __exception do_el0_irq_bp_hardening(void) +{ + /* PC has already been checked in entry.S */ + arm64_apply_bp_hardening(); +} + asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr, unsigned int esr, struct pt_regs *regs) From patchwork Thu Mar 1 12:54:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130206 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2763530edc; Thu, 1 Mar 2018 04:59:21 -0800 (PST) X-Google-Smtp-Source: AG47ELsY9qwMR10sPaADL5ttRtW9NSud43CIK5Y+fODI/Xsi+3WkiX3H9Eg00t6CknoTr4PSBoJV X-Received: by 2002:a17:902:850a:: with SMTP id bj10-v6mr1898386plb.5.1519909161633; Thu, 01 Mar 2018 04:59:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909161; cv=none; d=google.com; s=arc-20160816; b=bTjqWiN0ChQByxqCRDX50Sc6++veaFjvox1z5KMMB3BJun3AfLebqFrGoycsXEXe6A P01YG0vK23HkCpdS/MR7T2pcq+gmtBcFYWktVdPpNjs7WmWxs7QD2csSBg+xlTpdYpPp P8+5Q8gjVwykPNgZAuevMjPsB7bFKC/mdDtJbFUXKkJZpnJlOPhsZa4gXAaioFLirRih SM45rCzjYfFOg4hSxLWMI55gdQW3MN1t/b22kbGSAsnUv78JfUJsulFXmuAnW7+1NHGD PWDyoyCC4AFlRvvRkpk8oXylmkkpljgdBdr9wPgJZI2UP8cTGyQQTpRpoIG+zTrmuFup Ussg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=D9AfP0d9wigJX9FEBgA9gUNC1gw2wouMr9JGYC6sYGw=; b=tKzPpw3I2x8VKTP9aaWILheT1T9zkVuJKLMPiXzSxbfCR15QIrc6JU70+G1KfinHnI +5luPnGqFPCiuUMSdsy2g+hUFwuEBvfSRkjvYnRRM5SlkBgmZwxicirU2zNP1A8590dN u9aToR2UxPXwUOLw9449hBCM3JC/or7edwlq/hYG5EH0GEVByTBR2Bxv2fRZtsSYHJfl y6m50g3Et0n03Y8cuAbik94e0aEy3q8HmjnC9dNSutWCcuM5FAEJEycDdANNAasrZPX5 s9D9dd76oyICmJQn1Wuq/NyoI3nzhZuO6WRUaM3QETwq3MrW4trtruJCs1GKHkVFyv4Q XyFg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jS2vTu6b; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r12si2461227pgt.200.2018.03.01.04.59.21; Thu, 01 Mar 2018 04:59:21 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jS2vTu6b; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030998AbeCAM7S (ORCPT + 28 others); Thu, 1 Mar 2018 07:59:18 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:36990 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030804AbeCAM7O (ORCPT ); Thu, 1 Mar 2018 07:59:14 -0500 Received: by mail-pg0-f68.google.com with SMTP id y26so2276410pgv.4 for ; Thu, 01 Mar 2018 04:59:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=D9AfP0d9wigJX9FEBgA9gUNC1gw2wouMr9JGYC6sYGw=; b=jS2vTu6bOQlqruSbnRgE+uS/ZW2u30ikahXIOhL3/VStRxKB/46rcg/fFzQjJM6+4v hcAvUoGynPo/MATUC2k84gncykeZWw6BXYSYOv9idddqs3ZEN2PhqPID3l54bCflW5Xe 7vNN0V8roSLbyGiYcJrNknX8ihnIvkjocp4Tw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=D9AfP0d9wigJX9FEBgA9gUNC1gw2wouMr9JGYC6sYGw=; b=fvtTp17UcRTf1dJV3jil2xFGisfg0q71EPaFTm+fx4cu8G+e8DngcvHJfqBq63F5Yd lBklkxg4DoB6E2KIdxc/sTQorUOzZzSTQR75dw6Q5cxK/tEeWa2rZckMMA/VFk/lVrDJ y0lzV/ETOMuGiV1oXaBYo7iDbUY7RKiPnJ3HLr3bhcO+IUGyjrjHpDb9zfO4Eghi2CQD hRg2WDdE8PH4Pe2L3meJh9F9To1BAB5pfLKSBHaGkaNvA8E29kUgT2HN2OctTJ+3aBHw dP++HZPR7HqG+k+QWS4X11BxoOLwFiZugMcIFLlJgPnJh3iMT8CJIACPqM19lj4PXZgT 4Thw== X-Gm-Message-State: APf1xPB1hIapwZdGnvhz6Rfzx89f9rs27H55KqWmkPKwV/p7VL9/y705 nyyCwnctmpB/VlHxvL1eji/BXw== X-Received: by 10.98.237.12 with SMTP id u12mr1874733pfh.72.1519909154314; Thu, 01 Mar 2018 04:59:14 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.59.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:59:13 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 28/45] arm64: cputype: Add missing MIDR values for Cortex-A72 and Cortex-A75 Date: Thu, 1 Mar 2018 20:54:05 +0800 Message-Id: <1519908862-11425-29-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit a65d219fe5dc upstream. Hook up MIDR values for the Cortex-A72 and Cortex-A75 CPUs, since they will soon need MIDR matches for hardening the branch predictor. Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/cputype.h | 6 ++++++ 1 file changed, 6 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h index 26a68dd..0843b3f 100644 --- a/arch/arm64/include/asm/cputype.h +++ b/arch/arm64/include/asm/cputype.h @@ -75,7 +75,10 @@ #define ARM_CPU_PART_AEM_V8 0xD0F #define ARM_CPU_PART_FOUNDATION 0xD00 #define ARM_CPU_PART_CORTEX_A57 0xD07 +#define ARM_CPU_PART_CORTEX_A72 0xD08 #define ARM_CPU_PART_CORTEX_A53 0xD03 +#define ARM_CPU_PART_CORTEX_A73 0xD09 +#define ARM_CPU_PART_CORTEX_A75 0xD0A #define APM_CPU_PART_POTENZA 0x000 @@ -86,6 +89,9 @@ #define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53) #define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57) +#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72) +#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73) +#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75) #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX) #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX) From patchwork Thu Mar 1 12:54:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130207 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2763717edc; Thu, 1 Mar 2018 04:59:33 -0800 (PST) X-Google-Smtp-Source: AG47ELsYT9s4FX/afgKbW7COVz2HJ86FK/gr842gI0GVpQFRjIJDVBQOs13F3VpYO7aAkHdB1Yov X-Received: by 2002:a17:902:4545:: with SMTP id m63-v6mr1823857pld.15.1519909173392; Thu, 01 Mar 2018 04:59:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909173; cv=none; d=google.com; s=arc-20160816; b=j6hEiRrnG5GunqfXO8Zoc9RtA8CFwncb1H3QIp7lHibLhwiBNdeB6MlosYKZATxoo8 A54vvVK+96Csn2TTgz1LBmIGVkAa1E/Zi1PImdIAs7tJ4zX1VXXN4uLbMTgkTsl2qxHf Uw0p8qqh47PNlRMik2mPOIHeDrmaxM86Yy7v+j2upIOuElyNuXM2EiYxcRigjkpzvphs cjQBdHXFyPkWtz4Fl6n4GNLUaM87iUYmrwiw38B8m+1A2TiAudAJz5WDIYx9zi60vnqj OPBssGVajI98EWxQbj9RruLR1WXxx9BqUrJ+u/IxSFiYCtzqBKLlGLEOuwq+FRLa7uX7 bA/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=LVyW4e3/ZiswQxGmhghNNTu5Pe0KH7+4scjkon+J7js=; b=Mg3sR0l1hru91rNhdBxa8BGUNOBG9vsmV1+3xLMgy6KRGcBAWBm8tERC+yEZOYiZA2 5djgmDb0mwxHvv7kxU1xg2n7Zyz+owZvtO2f48zyBSDWxpuFLXA7/Qj+AltPRLJuZsQv FO2reWGssgHufj43SEeJ3BE3hjLRabyakDAUhg/wkyhzzBQUqdsWS9TQsEAuid7ZpbRG 8AAngFm7Zar/7sLd7aEppo72OOLdGCt14ds/LTcstLG2YkWYdEsLmHqSIdrVnEvtlCae tnu8hShrZzgJW474bdZSWoID0a6WY5Mr8ksbNMR/vFTeSm4xDT39+D12kixQ1Ug6RMte UEEQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TSi4qw+P; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h9si2989351pfk.72.2018.03.01.04.59.33; Thu, 01 Mar 2018 04:59:33 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TSi4qw+P; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031008AbeCAM73 (ORCPT + 28 others); Thu, 1 Mar 2018 07:59:29 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:32779 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030804AbeCAM7W (ORCPT ); Thu, 1 Mar 2018 07:59:22 -0500 Received: by mail-pg0-f67.google.com with SMTP id g12so2279725pgs.0 for ; Thu, 01 Mar 2018 04:59:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=LVyW4e3/ZiswQxGmhghNNTu5Pe0KH7+4scjkon+J7js=; b=TSi4qw+PBRPwqxaK5h7uIK7sqDHPIq59aqoFS3cuBmSFfu62O14ncON6D1XvkgZ86g FiG07MpGA6rb1NlDEfQgmm4xsibPm3kLDs06hpiRD5G6kOmCzslEuelG9gGKvnFfFB87 bSEb1SIPy5LkLg521qXlNe8UfuG94rHDmADE4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=LVyW4e3/ZiswQxGmhghNNTu5Pe0KH7+4scjkon+J7js=; b=mA+jj4A76mT5nmvNsQH2/z/ZH8E+iJ7qz0j0TUMaxbQaqBWdtKcofJVYI4K0p4jr0x e8iea7QybRSLvxJygTdvqmO4vO3HXKfhngMHCEzxHquQ8Kll1be+OlunxoKrZv4vEIV/ Aw/4dy88ELB0BxTITJT6+gaNTtvlIukVC3iA0DddNl1Eh5WYA2oBwihWQAKMkjy5C1il YN7gvdQ0Z46d/HbUSpefa2gDo2gpcbxoIPXYqqkHV62gdP3H8GQPtK84EutTin39t/MY EThCs0KjH1GA+0JwtoloG1HHk5I/Ws9eIRQfUVE65dAzg44AtuokEP67F+MuH2QVlfFJ ZjpA== X-Gm-Message-State: APf1xPBJwyQBHbNoRI0pke8G1ot0JeyvpP5NDb5/Mx4iknDwtcqR+ZmR irvQia4Bewn0yt/nCoivbze15au2sqM= X-Received: by 10.101.87.199 with SMTP id q7mr1496822pgr.215.1519909161327; Thu, 01 Mar 2018 04:59:21 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.59.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:59:20 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 29/45] arm64: Implement branch predictor hardening for affected Cortex-A CPUs Date: Thu, 1 Mar 2018 20:54:06 +0800 Message-Id: <1519908862-11425-30-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit aa6acde65e03 upstream. Cortex-A57, A72, A73 and A75 are susceptible to branch predictor aliasing and can theoretically be attacked by malicious code. This patch implements a PSCI-based mitigation for these CPUs when available. The call into firmware will invalidate the branch predictor state, preventing any malicious entries from affecting other victim contexts. Co-developed-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/kernel/bpi.S | 24 ++++++++++++++++++++++++ arch/arm64/kernel/cpu_errata.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S index 06a931e..dec95bd 100644 --- a/arch/arm64/kernel/bpi.S +++ b/arch/arm64/kernel/bpi.S @@ -53,3 +53,27 @@ ENTRY(__bp_harden_hyp_vecs_start) vectors __kvm_hyp_vector .endr ENTRY(__bp_harden_hyp_vecs_end) +ENTRY(__psci_hyp_bp_inval_start) + sub sp, sp, #(8 * 18) + stp x16, x17, [sp, #(16 * 0)] + stp x14, x15, [sp, #(16 * 1)] + stp x12, x13, [sp, #(16 * 2)] + stp x10, x11, [sp, #(16 * 3)] + stp x8, x9, [sp, #(16 * 4)] + stp x6, x7, [sp, #(16 * 5)] + stp x4, x5, [sp, #(16 * 6)] + stp x2, x3, [sp, #(16 * 7)] + stp x0, x1, [sp, #(16 * 8)] + mov x0, #0x84000000 + smc #0 + ldp x16, x17, [sp, #(16 * 0)] + ldp x14, x15, [sp, #(16 * 1)] + ldp x12, x13, [sp, #(16 * 2)] + ldp x10, x11, [sp, #(16 * 3)] + ldp x8, x9, [sp, #(16 * 4)] + ldp x6, x7, [sp, #(16 * 5)] + ldp x4, x5, [sp, #(16 * 6)] + ldp x2, x3, [sp, #(16 * 7)] + ldp x0, x1, [sp, #(16 * 8)] + add sp, sp, #(8 * 18) +ENTRY(__psci_hyp_bp_inval_end) diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 0e07893..f8810bf 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -53,6 +53,8 @@ static int cpu_enable_trap_ctr_access(void *__unused) DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); #ifdef CONFIG_KVM +extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[]; + static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start, const char *hyp_vecs_end) { @@ -94,6 +96,9 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn, spin_unlock(&bp_lock); } #else +#define __psci_hyp_bp_inval_start NULL +#define __psci_hyp_bp_inval_end NULL + static void __install_bp_hardening_cb(bp_hardening_cb_t fn, const char *hyp_vecs_start, const char *hyp_vecs_end) @@ -118,6 +123,21 @@ static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry, __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end); } + +#include + +static int enable_psci_bp_hardening(void *data) +{ + const struct arm64_cpu_capabilities *entry = data; + + if (psci_ops.get_version) + install_bp_hardening_cb(entry, + (bp_hardening_cb_t)psci_ops.get_version, + __psci_hyp_bp_inval_start, + __psci_hyp_bp_inval_end); + + return 0; +} #endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */ #define MIDR_RANGE(model, min, max) \ @@ -211,6 +231,28 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .def_scope = SCOPE_LOCAL_CPU, .enable = cpu_enable_trap_ctr_access, }, +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + { + .capability = ARM64_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A57), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM64_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A72), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM64_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A73), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM64_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A75), + .enable = enable_psci_bp_hardening, + }, +#endif { } }; From patchwork Thu Mar 1 12:54:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130208 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2763853edc; Thu, 1 Mar 2018 04:59:41 -0800 (PST) X-Google-Smtp-Source: AG47ELt5OvSgt6fv81++ZScAOedAiWda0xwVG88lrpOGP5XOqrqMNCuRfnz5HELN/+xM/i9QlkoA X-Received: by 10.98.254.6 with SMTP id z6mr1840603pfh.155.1519909181467; Thu, 01 Mar 2018 04:59:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909181; cv=none; d=google.com; s=arc-20160816; b=u8suQdCaoTbwG3FgV2vT3t6/0bco1jYfBdalSmMU7Tz9iSgfAZaHnc+WMPn5x67lpz kRAFCaHNs0dNxBDaU1pFcW5N7V5tsqJgWAhWEZNw6up3dFuOb8qbhj+ihIXOjPALlMBo cgUVfJJ/k4gZQ4uM/QBE9YkHqx6vVx3DFf7gs+P8AT9mCdwCWcFQ13YnMwp2jznHOrDi IAZS2CLdkXQ5Axsb/ObK2h6CgRRWHUJfQfYYm5tW+6UXUb3YAQGIxzXKi7jV6xxb4RTV upjsZxmthAyTUlciLLmzvUldsj4ITOBIbQciLGeQa7/B8vbPw+f1faXFUzn4UMS/Yb7y aasA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=/nksGLZqHQjfpQbfH68HEgt65Nc6WZIfEk50r4t0UeE=; b=uGPrWzpsJFlLydUzbO28SteicmnRjqkROwbS1sK2u5vjfkIADMAM74jNzqHAMcpzYc +y2iPZ52DfAdv67rUemi+ppIJvQ8Jx4KdtsNofu7GaqHfKADNaKkRcnEOeftq9L38dfm tOD0su1w8pPWZR+lWS1JYX13dbXAIfVDY8JTFMmp/aiOTrYpeFHCB6c6Bs9whr6SETfv EiI5Bzkm0rj05yKv0WqYQr/9epMrqIpiOXNfn3eqQh4WsMJ7BKITMYlU5qFbsPdHBphu H3GM8ZRLRRZAK1lhNzoROFMMgx+o6vdKjGQhLDopHu9Dh2BjI2Dz+u8M/YSXtbaSlA9k 7B4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KX26itFa; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h9si2989351pfk.72.2018.03.01.04.59.41; Thu, 01 Mar 2018 04:59:41 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KX26itFa; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031018AbeCAM7g (ORCPT + 28 others); Thu, 1 Mar 2018 07:59:36 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:46325 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030881AbeCAM72 (ORCPT ); Thu, 1 Mar 2018 07:59:28 -0500 Received: by mail-pf0-f196.google.com with SMTP id z10so2394319pfh.13 for ; Thu, 01 Mar 2018 04:59:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/nksGLZqHQjfpQbfH68HEgt65Nc6WZIfEk50r4t0UeE=; b=KX26itFa6TWDAtd8NB+rYrugNbEWlF+pHs/zXVAPusKcpCZ1Yu/dj9IDJ0aaYxZZe9 cbh5KbeIX3EMOicPzUJD+vVsN8abRaPgSgXf8EH34CGpffQ5XmnVOWCzgqITfqvMGc8N wMMw5oF1PWtUflxwo4lMbqjXw14s/8YtGrJkQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/nksGLZqHQjfpQbfH68HEgt65Nc6WZIfEk50r4t0UeE=; b=kLqVex8XCkz4uPUO09IwbTNC2WcWSFLQJldcDSh+3DMJC5FHHLrg7UOaueX/Q7hp75 m9rFUkY62xv5L74/vVMggbFnmIBcOHfKcj1fRm5jXc/5Nt+de9NgNTKheChsjwpXbZr8 C2KLld95K16+umjHMV1mPjrlCbYEgxIq79N6MlfM2HK67ftBQNf+oa2Bxs+utQX1xPh6 JvMq8LTM0oC0FLFAWg8TLn/kG4FRJICqk1mxX8GRaKx1qLV7rtJJiBR5NLIyMMjjL2l9 EhGA+a1jErOQbxTIeDfARfBStL13Zywf4MjAMj3X9V4KiITmZglDy9viUSxwm+VDuI2d iM+Q== X-Gm-Message-State: APf1xPB5dUylVwwJSTQuDtZbibEJlgIgGZupBwD1DbOl59dnIENgVqDR BeClXe7j59q6WmOPEcvVFJvdAg== X-Received: by 10.99.117.24 with SMTP id q24mr1474368pgc.53.1519909168143; Thu, 01 Mar 2018 04:59:28 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.59.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:59:27 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 30/45] arm64: KVM: Increment PC after handling an SMC trap Date: Thu, 1 Mar 2018 20:54:07 +0800 Message-Id: <1519908862-11425-31-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit f5115e8869e1 upstream. When handling an SMC trap, the "preferred return address" is set to that of the SMC, and not the next PC (which is a departure from the behaviour of an SMC that isn't trapped). Increment PC in the handler, as the guest is otherwise forever stuck... Cc: stable@vger.kernel.org Fixes: acfb3b883f6d ("arm64: KVM: Fix SMCCC handling of unimplemented SMC/HVC calls") Reviewed-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kvm/handle_exit.c | 9 +++++++++ 1 file changed, 9 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 2e6e9e9..5b56b09 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -53,7 +53,16 @@ static int handle_hvc(struct kvm_vcpu *vcpu, struct kvm_run *run) static int handle_smc(struct kvm_vcpu *vcpu, struct kvm_run *run) { + /* + * "If an SMC instruction executed at Non-secure EL1 is + * trapped to EL2 because HCR_EL2.TSC is 1, the exception is a + * Trap exception, not a Secure Monitor Call exception [...]" + * + * We need to advance the PC after the trap, as it would + * otherwise return to the same address... + */ vcpu_set_reg(vcpu, 0, ~0UL); + kvm_skip_instr(vcpu, kvm_vcpu_trap_il_is32bit(vcpu)); return 1; } From patchwork Thu Mar 1 12:54:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130209 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2763886edc; Thu, 1 Mar 2018 04:59:44 -0800 (PST) X-Google-Smtp-Source: AG47ELvRvMjToJV3xynrgQG+rR6SHsfgXJj3E9BoIDvJbKbqTWa6hyFzuSAlfnqLG2pvnrGVoOmz X-Received: by 2002:a17:902:8341:: with SMTP id z1-v6mr1898561pln.386.1519909184184; Thu, 01 Mar 2018 04:59:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909184; cv=none; d=google.com; s=arc-20160816; b=PQMflVNAZSTAW6X9wb55dlzXDu9Nskn8ZOk0emkb5x8d6Nf1NOaGyyzT89pTuMd9Wj AIYS25U9L/xglRcoINlHxlKeDnN2mE7Gy9jNgXkvh2o/jxli13Q4DdP73LzZFeddCBmN ywD29lRPqNm6dg4+3BDeDcucL7Dqg8B19Jsu5lSwzPjYjb5sm55lz+jwrwMWpe7YIm3w ITCBGahY9lKUhdmIbeUllqT315vn32lmc3dXRzzVtzpjkKWkd0Nmhcw1qTT3esqd6adG bTrnndxmvL7jMvzfTz8u2YLr8DlqGK/jqAgnkIY9weJAldiK4DxuaQkEqj3x5Usf7H+8 aNDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=iXQ1ea6Hf/00SPSEn2cgvnnJnEbrG1bOJUxh9KRtS1o=; b=DsQEurfbXuv18H/spy6N3qxbEPPX2+uoTgyyGtGmCataUTGR3fyltx+pih2AhPRYRq qanCOY9sQPLNCLlQ8/gEUYOaufCfDlkbC3Mt2W0FrdTl9HE7WqL+xLD7/VjiBtm5PDG4 rH2Qou8Dx94AYsplcprWeYiLy4cYbekGDRBmvj/X24nZSCTLAwl0NIm8kyIjwrkY4/hJ 5lphe7nbW2oGJDm27Xwq7KXXXTr6jHJpuBgq+Zy+EJl8fpRZDdOEmYhdvULDd32FgtDn +Y9BQU3xUzyaBTkS0DkirHb2qbRlmiGAI0X8voimpj6ujaFC9//ocywoQgWzHuQvp8G6 3LSg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HPw8khHD; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h9si2989351pfk.72.2018.03.01.04.59.43; Thu, 01 Mar 2018 04:59:44 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HPw8khHD; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031029AbeCAM7k (ORCPT + 28 others); Thu, 1 Mar 2018 07:59:40 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:38727 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031015AbeCAM7g (ORCPT ); Thu, 1 Mar 2018 07:59:36 -0500 Received: by mail-pg0-f66.google.com with SMTP id l24so2275583pgc.5 for ; Thu, 01 Mar 2018 04:59:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=iXQ1ea6Hf/00SPSEn2cgvnnJnEbrG1bOJUxh9KRtS1o=; b=HPw8khHDIOr90kZmiRgPeRdSq2EXPf7Ue7ekWyGqkLS/x99eLh5AmuNzHEnVuc0Jy2 a6fKIW1LGIzjUe3uGhM7sSfPxEBaexkI3xr/2oR6rK2fSAx7FYAPDc7MVRMRZ+Jm5A9D qbGKofIEnTruBfA+aVHV+uie7D1YQ4gQEx5yw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=iXQ1ea6Hf/00SPSEn2cgvnnJnEbrG1bOJUxh9KRtS1o=; b=qOloISfm6PXC1q4MJbRGZ3prnYwOQuVaiGsrq6FkLDNKUrTiuTIkuJZFHTULlC4Fcu V95SK3cHfY6thkkfUgzlNhbU90sH0afXplaJAHDoYACETr6H7zFkYqPAuxKNJAfPDYrV pxVm8qU6zTR+dQwoeXTR8YupNWTFaf1vNER/qvMRgjNgQZYXzQrjkqZN8l1DvQDrUKs6 5rAEWIziN9DFIAB2KKq8/+SWp5PM+T+yMK1J2K4Z6PSwAKzR/yojLtdmFA75DuoMpV9n xwelF4D+ANz6BQ3iKwsTET3+QlcKzdzVw1hfsc8QgNkb6J8IPEhqJUapytue5OeVSgCv XPCQ== X-Gm-Message-State: APf1xPCnsSK3881YAXWZt6WodLBJ9awHqO6TJXM/7tLMTc27KwEbN3u2 aQfH0h2T/Qi0xnPal3J1fGGiKQ== X-Received: by 10.98.86.15 with SMTP id k15mr1877067pfb.187.1519909175506; Thu, 01 Mar 2018 04:59:35 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.59.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:59:34 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 31/45] arm/arm64: KVM: Consolidate the PSCI include files Date: Thu, 1 Mar 2018 20:54:08 +0800 Message-Id: <1519908862-11425-32-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 1a2fb94e6a77 upstream. As we're about to update the PSCI support, and because I'm lazy, let's move the PSCI include file to include/kvm so that both ARM architectures can find it. Acked-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/include/asm/kvm_psci.h | 27 --------------------------- arch/arm/kvm/arm.c | 2 +- arch/arm/kvm/handle_exit.c | 2 +- arch/arm/kvm/psci.c | 2 +- arch/arm64/include/asm/kvm_psci.h | 27 --------------------------- arch/arm64/kvm/handle_exit.c | 3 ++- include/kvm/arm_psci.h | 27 +++++++++++++++++++++++++++ 7 files changed, 32 insertions(+), 58 deletions(-) delete mode 100644 arch/arm/include/asm/kvm_psci.h delete mode 100644 arch/arm64/include/asm/kvm_psci.h create mode 100644 include/kvm/arm_psci.h -- 2.7.4 diff --git a/arch/arm/include/asm/kvm_psci.h b/arch/arm/include/asm/kvm_psci.h deleted file mode 100644 index 6bda945..0000000 --- a/arch/arm/include/asm/kvm_psci.h +++ /dev/null @@ -1,27 +0,0 @@ -/* - * Copyright (C) 2012 - ARM Ltd - * Author: Marc Zyngier - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -#ifndef __ARM_KVM_PSCI_H__ -#define __ARM_KVM_PSCI_H__ - -#define KVM_ARM_PSCI_0_1 1 -#define KVM_ARM_PSCI_0_2 2 - -int kvm_psci_version(struct kvm_vcpu *vcpu); -int kvm_psci_call(struct kvm_vcpu *vcpu); - -#endif /* __ARM_KVM_PSCI_H__ */ diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index 4e4ae07..8588e67 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -44,8 +44,8 @@ #include #include #include -#include #include +#include #ifdef REQUIRES_VIRT __asm__(".arch_extension virt"); diff --git a/arch/arm/kvm/handle_exit.c b/arch/arm/kvm/handle_exit.c index 4e57ebc..0a2b758 100644 --- a/arch/arm/kvm/handle_exit.c +++ b/arch/arm/kvm/handle_exit.c @@ -21,7 +21,7 @@ #include #include #include -#include +#include #include #include "trace.h" diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index a08d7a9..18f9e9d 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -21,8 +21,8 @@ #include #include -#include #include +#include #include diff --git a/arch/arm64/include/asm/kvm_psci.h b/arch/arm64/include/asm/kvm_psci.h deleted file mode 100644 index bc39e55..0000000 --- a/arch/arm64/include/asm/kvm_psci.h +++ /dev/null @@ -1,27 +0,0 @@ -/* - * Copyright (C) 2012,2013 - ARM Ltd - * Author: Marc Zyngier - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -#ifndef __ARM64_KVM_PSCI_H__ -#define __ARM64_KVM_PSCI_H__ - -#define KVM_ARM_PSCI_0_1 1 -#define KVM_ARM_PSCI_0_2 2 - -int kvm_psci_version(struct kvm_vcpu *vcpu); -int kvm_psci_call(struct kvm_vcpu *vcpu); - -#endif /* __ARM64_KVM_PSCI_H__ */ diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 5b56b09..0231ebc 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -22,12 +22,13 @@ #include #include +#include + #include #include #include #include #include -#include #define CREATE_TRACE_POINTS #include "trace.h" diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h new file mode 100644 index 0000000..2042bb9 --- /dev/null +++ b/include/kvm/arm_psci.h @@ -0,0 +1,27 @@ +/* + * Copyright (C) 2012,2013 - ARM Ltd + * Author: Marc Zyngier + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#ifndef __KVM_ARM_PSCI_H__ +#define __KVM_ARM_PSCI_H__ + +#define KVM_ARM_PSCI_0_1 1 +#define KVM_ARM_PSCI_0_2 2 + +int kvm_psci_version(struct kvm_vcpu *vcpu); +int kvm_psci_call(struct kvm_vcpu *vcpu); + +#endif /* __KVM_ARM_PSCI_H__ */ From patchwork Thu Mar 1 12:54:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130210 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2764000edc; Thu, 1 Mar 2018 04:59:50 -0800 (PST) X-Google-Smtp-Source: AG47ELv4JCEwlTCJvGqs9YsZRXC85Y2NNAba0vfkC9gckN1xMiGaDfV0kn0xxq1Ys6OhDlucf1hb X-Received: by 2002:a17:902:7445:: with SMTP id e5-v6mr1868633plt.204.1519909190597; Thu, 01 Mar 2018 04:59:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909190; cv=none; d=google.com; s=arc-20160816; b=N8fuwEzhWY0vnhrpdm4HGzSt0GBAV0dNVd9OkxP+eCu3OqKZPoI6WrJAGKOjRrWhfg yywYDt+uVOLIe6cUWxkwgKiKXoAfLJTZnqStvAjDFz4JGyxwsMgGqqvyv3Ah3UeJ3Vin +mJx5FZU7fSTH2hfC2bs+4lPXvEElQAZmNn2Ue13FlbHI5hD+yXo8BRqGugzm4Gdfpuz 0WrRpeFWdvde930nATby5InEuNDTZnSgi10au/atpdLkdONDeb426VDWEQdBoOZ0Iz9n y3l/NnPrZ07gb//2Hl6ioFoMJ/hAp/olOM9EmbYQKPPSgDi7QhkwefC8bHWvprlxuapR jtbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=Uco9yRsCAyX2xrhlinZnEESkEh5h0LM+vQPWF5wB2iw=; b=R81WhyqxqCM20evFTXnDa4L4JOjdLPKPGUg3PH6uSvujGMmTH6IjgrxntVEsk9Hu9y +5iLZbWSlb5iFLtEaPhpmXEMno0AHnF4ot5yHeANTitwZuxXUDx/d7RFdFmVTXyIkBaW 0RJLB7AG092Cy2e26o7yM80+Yh/lxLLzl0x4d4BRu3LMoEWs4BK/Nqif2Dw8peK4zhOP +H9iXnUlyHxNKfDWNLVTNhw9Z3FglEuOT/nGbymg+oylCQ+o+YSJETJaQlFtCWA0qbpl 20aEARmgkdMX2avc/miZ4XlSyG49mnEXalAbgySCx0QbQVSoVz22mJR3m6FKZ1kRDK4n GyKw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JrrO13SK; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a4si2448513pgn.327.2018.03.01.04.59.50; Thu, 01 Mar 2018 04:59:50 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JrrO13SK; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1030922AbeCAM7r (ORCPT + 28 others); Thu, 1 Mar 2018 07:59:47 -0500 Received: from mail-pg0-f65.google.com ([74.125.83.65]:42062 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030920AbeCAM7n (ORCPT ); Thu, 1 Mar 2018 07:59:43 -0500 Received: by mail-pg0-f65.google.com with SMTP id y8so2271933pgr.9 for ; Thu, 01 Mar 2018 04:59:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Uco9yRsCAyX2xrhlinZnEESkEh5h0LM+vQPWF5wB2iw=; b=JrrO13SKvcwBHQRGijSQbVpTUyBaiTjlMmRn1y2Jly9s2Sf4dLg5BRMnmJHbroefpB 0ZHLKaxgmP04Gfjn2ZRJ9dpcHP0If6aUUs7VPX42fm3+JymTDqOJH+MR5feJ4KfitE45 KgywRHwCYasOIWJRLFpTcjfoTJPMo9BZoXb00= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Uco9yRsCAyX2xrhlinZnEESkEh5h0LM+vQPWF5wB2iw=; b=LrgwLhX4GF8E2VL5iISaStFwKoYy9PndBdAh18kQOx87TCayMkmG5w3ZaavcnDMFUk EKXmMBh1FVX91Nu7v2Qgp8xe6PF94D9amO5ohlIdO/vNWPjyaiMKiup9i954P+0AZaWa yn3PeHcJQU6V0KoupcMm8i31liOApDZKytvgELzVku7Df8PCvZWfq1TQwheu+fNZYE1r IZrTKdAsWeKjTN20E51aVCmAREUjKpJg9SFPCqtOMjrVsqvKSZ4gwWhYNZctHTCH3bgX 3qmH3WQ2+RoVaa1w+wfoGVn6i3Bt9+q527SzHokCZfd3fptuPWE/BO6Wo49jPv7ob/BW iTVQ== X-Gm-Message-State: APf1xPAYIUUViCG3DvDbjo/omeFAMXGdl+Jp5+BhI28yCMe+OnGWkFDy BqgWhDQLWYPmf1qwugqsKulJfw== X-Received: by 10.99.140.29 with SMTP id m29mr1527259pgd.320.1519909182749; Thu, 01 Mar 2018 04:59:42 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.59.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:59:42 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 32/45] arm/arm64: KVM: Add PSCI_VERSION helper Date: Thu, 1 Mar 2018 20:54:09 +0800 Message-Id: <1519908862-11425-33-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit d0a144f12a7c upstream. As we're about to trigger a PSCI version explosion, it doesn't hurt to introduce a PSCI_VERSION helper that is going to be used everywhere. Reviewed-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/kvm/psci.c | 2 +- include/kvm/arm_psci.h | 6 ++++-- include/uapi/linux/psci.h | 3 +++ 3 files changed, 8 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 18f9e9d..4adfa28 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -219,7 +219,7 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) * Bits[31:16] = Major Version = 0 * Bits[15:0] = Minor Version = 2 */ - val = 2; + val = KVM_ARM_PSCI_0_2; break; case PSCI_0_2_FN_CPU_SUSPEND: case PSCI_0_2_FN64_CPU_SUSPEND: diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index 2042bb9..5659343 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -18,8 +18,10 @@ #ifndef __KVM_ARM_PSCI_H__ #define __KVM_ARM_PSCI_H__ -#define KVM_ARM_PSCI_0_1 1 -#define KVM_ARM_PSCI_0_2 2 +#include + +#define KVM_ARM_PSCI_0_1 PSCI_VERSION(0, 1) +#define KVM_ARM_PSCI_0_2 PSCI_VERSION(0, 2) int kvm_psci_version(struct kvm_vcpu *vcpu); int kvm_psci_call(struct kvm_vcpu *vcpu); diff --git a/include/uapi/linux/psci.h b/include/uapi/linux/psci.h index 3d7a0fc..39930ca 100644 --- a/include/uapi/linux/psci.h +++ b/include/uapi/linux/psci.h @@ -87,6 +87,9 @@ (((ver) & PSCI_VERSION_MAJOR_MASK) >> PSCI_VERSION_MAJOR_SHIFT) #define PSCI_VERSION_MINOR(ver) \ ((ver) & PSCI_VERSION_MINOR_MASK) +#define PSCI_VERSION(maj, min) \ + ((((maj) << PSCI_VERSION_MAJOR_SHIFT) & PSCI_VERSION_MAJOR_MASK) | \ + ((min) & PSCI_VERSION_MINOR_MASK)) /* PSCI features decoding (>=1.0) */ #define PSCI_1_0_FEATURES_CPU_SUSPEND_PF_SHIFT 1 From patchwork Thu Mar 1 12:54:11 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130212 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2764296edc; Thu, 1 Mar 2018 05:00:07 -0800 (PST) X-Google-Smtp-Source: AG47ELuO0dV20baV6jG9SgNay9xgSuj8xm+9kZGvqOGCWDDAdA4/HuxBQCXnfiYs7jHSHVupbhNb X-Received: by 2002:a17:902:a607:: with SMTP id u7-v6mr1819393plq.367.1519909207043; Thu, 01 Mar 2018 05:00:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909207; cv=none; d=google.com; s=arc-20160816; b=Rey+xV1B5Eb6cgzAU/omTTtRZNDIrV+zwBNM2aKsmRJsp3Iy7JxygyVQFnPxht3r+J b3BHirA5o4bEhV4iAgehk8O8nTDNU/EbspIi5Mw2VmbuG86YeYP5bRBoqNqwPtdZHFaV x1hMsER+Q4zO0O6OBh8rQ56qvB/T114uiPpcAOX/cyb+z8vrbTvg8MykvQQbYJthYk2b 4D0jwFGgrCacFiZ/NeiwEBTLzT50MW0boUGfmLSIQzvPycRWWwIYdsLfsGw3qDpG9H71 R+ZKpYLPYpyjtC0FUVy6dOEMPMWKVFAF3Ht8hF4T5VHuQQKdOKO7aPnAYMg0Y3WumgnI abpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=J+5mFS5YG0Z6B4feNaUOGsgL/hJiqYMdZY2VyVL7BU0=; b=PQPFDC9MJEUa+9+eS0OjyeLbYYJKeegXR9E88OFh79JFltRl8lPUzPFaYQZV3sHBNN tvMwODSiBEPVJDxo69/KKoTje86qeqLTTlBnva1s9ubi4Udqf9uNs82Shd220t6QmjeP 7EUvgsoYvnaWQNRYGftpQdGbnTUrp+b4MEmyfc+/zs5t+3lUu/5JdZMJrOPEsrbv86Go CH8ru41UWFRccIKQIG6nLeUni/LpF+64ZIJNIcuBPo7L5GdzE+QbNrOH8Iz9ODfrglFN c8AdzOVlS3VPuyJUXYqGJUGr4WCZzIy0SiuDC9HcdspNH7uoX0XsuQ1cGdlHVcu0ql8I jJMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cNliJQE+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e71si2448760pgc.209.2018.03.01.05.00.06; Thu, 01 Mar 2018 05:00:07 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cNliJQE+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031066AbeCANAC (ORCPT + 28 others); Thu, 1 Mar 2018 08:00:02 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:34475 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030735AbeCAM75 (ORCPT ); Thu, 1 Mar 2018 07:59:57 -0500 Received: by mail-pg0-f66.google.com with SMTP id m19so2279243pgn.1 for ; Thu, 01 Mar 2018 04:59:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=J+5mFS5YG0Z6B4feNaUOGsgL/hJiqYMdZY2VyVL7BU0=; b=cNliJQE+mVOBsVb8LMhpi351PjVm9gqEDlFXQWMMXNNaqbPBsy4oK/0vk9cVUXNkm2 I+ojSopeItSzHa2lqOac6u6cEXjJ5RXwip9bjvc1YILoj4Gfh/ernqEk7iOl0xYqUkDp 9ALGtgQxoadLu1NrtZs4AaZxjAlGW1urE2jNM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=J+5mFS5YG0Z6B4feNaUOGsgL/hJiqYMdZY2VyVL7BU0=; b=Em4oHr9PqUlioEhoZBxcK2z6q7Xb/W+yaRtqspZXg5OSHxqDB71TBp1m8jc/5dHYKK zfWu67rN+tiOoowAyEzZFhlH8dNJZ9HpoFp21hbrIF6JNZQRGKTdl3GcLSt2WXzxanJ4 AqHFEVy1Vo/JumEV1KDV3Yx0+wMqx2VFck9h67Dz0MK+vOuwd3KoFxS+0il4ZcIWQt1R UPr9fw6VHCpeuqF2OdPvW7rOJuVkHa1RAbAA+8U5PFUtp09YwUgSKsQ0O6GqjpzKpMDj cZAc8VOcWQavpCVdqbgPHy0S54LFiZW1745iW+7qpf56iYMXltw10pKsvRzAb7DM7duD NtAg== X-Gm-Message-State: APf1xPAjtuIMevFcsqOCP0sTyHoHoTRWLlOAYd4IzJUHJfrtK5/pIwY6 FsvhvVOHuTnkhfpQFqCsu4H43A== X-Received: by 10.101.77.69 with SMTP id j5mr1494347pgt.352.1519909196927; Thu, 01 Mar 2018 04:59:56 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.59.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:59:56 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 34/45] arm/arm64: KVM: Implement PSCI 1.0 support Date: Thu, 1 Mar 2018 20:54:11 +0800 Message-Id: <1519908862-11425-35-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 58e0b2239a4d upstream. PSCI 1.0 can be trivially implemented by providing the FEATURES call on top of PSCI 0.2 and returning 1.0 as the PSCI version. We happily ignore everything else, as they are either optional or are clarifications that do not require any additional change. PSCI 1.0 is now the default until we decide to add a userspace selection API. Reviewed-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/kvm/psci.c | 45 ++++++++++++++++++++++++++++++++++++++++++++- include/kvm/arm_psci.h | 3 +++ 2 files changed, 47 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index bc334d6..097632c 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -233,7 +233,7 @@ static void kvm_psci_system_reset(struct kvm_vcpu *vcpu) int kvm_psci_version(struct kvm_vcpu *vcpu) { if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) - return KVM_ARM_PSCI_0_2; + return KVM_ARM_PSCI_LATEST; return KVM_ARM_PSCI_0_1; } @@ -312,6 +312,47 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) return ret; } +static int kvm_psci_1_0_call(struct kvm_vcpu *vcpu) +{ + u32 psci_fn = smccc_get_function(vcpu); + u32 feature; + unsigned long val; + int ret = 1; + + switch(psci_fn) { + case PSCI_0_2_FN_PSCI_VERSION: + val = KVM_ARM_PSCI_1_0; + break; + case PSCI_1_0_FN_PSCI_FEATURES: + feature = smccc_get_arg1(vcpu); + switch(feature) { + case PSCI_0_2_FN_PSCI_VERSION: + case PSCI_0_2_FN_CPU_SUSPEND: + case PSCI_0_2_FN64_CPU_SUSPEND: + case PSCI_0_2_FN_CPU_OFF: + case PSCI_0_2_FN_CPU_ON: + case PSCI_0_2_FN64_CPU_ON: + case PSCI_0_2_FN_AFFINITY_INFO: + case PSCI_0_2_FN64_AFFINITY_INFO: + case PSCI_0_2_FN_MIGRATE_INFO_TYPE: + case PSCI_0_2_FN_SYSTEM_OFF: + case PSCI_0_2_FN_SYSTEM_RESET: + case PSCI_1_0_FN_PSCI_FEATURES: + val = 0; + break; + default: + val = PSCI_RET_NOT_SUPPORTED; + break; + } + break; + default: + return kvm_psci_0_2_call(vcpu); + } + + smccc_set_retval(vcpu, val, 0, 0, 0); + return ret; +} + static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) { struct kvm *kvm = vcpu->kvm; @@ -354,6 +395,8 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) int kvm_psci_call(struct kvm_vcpu *vcpu) { switch (kvm_psci_version(vcpu)) { + case KVM_ARM_PSCI_1_0: + return kvm_psci_1_0_call(vcpu); case KVM_ARM_PSCI_0_2: return kvm_psci_0_2_call(vcpu); case KVM_ARM_PSCI_0_1: diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index 5659343..3236043 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -22,6 +22,9 @@ #define KVM_ARM_PSCI_0_1 PSCI_VERSION(0, 1) #define KVM_ARM_PSCI_0_2 PSCI_VERSION(0, 2) +#define KVM_ARM_PSCI_1_0 PSCI_VERSION(1, 0) + +#define KVM_ARM_PSCI_LATEST KVM_ARM_PSCI_1_0 int kvm_psci_version(struct kvm_vcpu *vcpu); int kvm_psci_call(struct kvm_vcpu *vcpu); From patchwork Thu Mar 1 12:54:12 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130213 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2764444edc; Thu, 1 Mar 2018 05:00:14 -0800 (PST) X-Google-Smtp-Source: AG47ELtoB/2wZQN13QR5It8iAgGgWK1Yz1+sMtsyV5gwknuWN1aIjhpPRX5kiHSpDBTP7pLr090c X-Received: by 2002:a17:902:3383:: with SMTP id b3-v6mr1911066plc.224.1519909214093; Thu, 01 Mar 2018 05:00:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909214; cv=none; d=google.com; s=arc-20160816; b=GFMzmRJrILqc4CfTYKCQqx7chFGSRBhN5tISc0Cna9VdiZFNmuD4aisOJtamn1oo9g 0kRjP/q+hkXH9iEH+83KW607JAhJEnOuXiSxO2XJED4klFZkVjrkx+FI7K0dxPoYVr5n +jx4S7jr1UHl9JdHPBaU1rWFmfHzRcWWXTddTEq2/5BcLU8ZJc77Q3wYm4wFyJvr1B0n OC1wmqCXB+X0ltqE47fKG0XhTb6KOd/isDlMLhyb9R+aRAQWFA57WpjcQiLXiBsqWYIO gamvllQda4bFvAWyGq8XUI59etb1sqMfUlh9KZUyDtZwDytGSQ8BfMW484btgu2Pc/eS OXbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=vuv71HvEZ9HnAUJTQ52hE3U6SvzV4oqNvK9jHSVsfaA=; b=B1Szvo+hIDNstbCtnUn34jOREsJfQqEH2+uNf+yfK/kfd5JWtWnOUHGcLae+bJAme2 AXAju0n2H6dkVvjEpM9VVOsLsn0zJukqOzhtaSErfCSX9IehyGQGa5U/s7fvGrCr6DEc jJA9xyJ6zgq8T0nDmVpOAZmsNYdOINWf0oi4qOZ1uq/SiEs+DSj+Dsu3UtzZM9JSRLu6 /uQpwvJFuEIbbtgexvBP0gj6MZ4Z/YWqL2HpDYSniB8NHF2j+MRHFTY+vjcSbHK4P4JU kQdPX1YaG6N3SE1gYTmVpl184dNmGWZF23mSdYjrD3rCluXDXk1LllBL+6x74ly/tfww SawQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GqeIjuNj; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e71si2448760pgc.209.2018.03.01.05.00.12; Thu, 01 Mar 2018 05:00:14 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GqeIjuNj; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031077AbeCANAK (ORCPT + 28 others); Thu, 1 Mar 2018 08:00:10 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:43014 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030915AbeCANAF (ORCPT ); Thu, 1 Mar 2018 08:00:05 -0500 Received: by mail-pf0-f193.google.com with SMTP id j2so256561pff.10 for ; Thu, 01 Mar 2018 05:00:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vuv71HvEZ9HnAUJTQ52hE3U6SvzV4oqNvK9jHSVsfaA=; b=GqeIjuNjXZxor3GLsU6Y2KXmuKkjdWSV1yZYtSgDnkbJNcqWzu9QADjLNCLnwyQKcM vniLAn0Wk81/TRsPLSeVLLkci0uT81Ur6XsQKQbFo2lNHC9KqfmSjboosoDS7BkXfHl/ bX3LLQHZcZeoUGQxe08gQJhfvZUxFQHFeR2XU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=vuv71HvEZ9HnAUJTQ52hE3U6SvzV4oqNvK9jHSVsfaA=; b=UxSjAhj0EiwefT11j/4zncZGvdkgOsFDYyLls9uYf9RQsVePJ4SqCqK+i0ReXz6NTZ roMhdDC4unytTYB8SCVmHFsBt9E/M1HFrFQPzJuuwrmfxb443gTnykF/huTusLWaunlh 8eP4dLAiUQE+1ipH/tHSbcYO87CjCONTRUDmhUpYFPrXt1uig+EjLBTdc5T7BgQp2kRa rUY+UM28DAfCHWGdwIixnSXk1l2CFEAIG/SpYp4ksM961b1nsTiBoLWG/e9l50e6o29d abWF2OluZ98GWfvzEnW8GT6amvk3lo5MclaQZUjToMvxsDMQJO9Lj1ESl//K1LinbSXq BTMA== X-Gm-Message-State: APf1xPC6WIMVlwoChGS0uJy0+yNVroxA3s89iu7ay9TudjMvbE753zT6 9Kk2quUGn69cNkp+FQaX+/Yo1A== X-Received: by 10.99.109.77 with SMTP id i74mr1449559pgc.73.1519909204541; Thu, 01 Mar 2018 05:00:04 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.59.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:00:03 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 35/45] arm/arm64: KVM: Advertise SMCCC v1.1 Date: Thu, 1 Mar 2018 20:54:12 +0800 Message-Id: <1519908862-11425-36-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 09e6be12effd upstream. The new SMC Calling Convention (v1.1) allows for a reduced overhead when calling into the firmware, and provides a new feature discovery mechanism. Make it visible to KVM guests. Tested-by: Ard Biesheuvel Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/kvm/handle_exit.c | 2 +- arch/arm/kvm/psci.c | 24 +++++++++++++++++++++++- arch/arm64/kvm/handle_exit.c | 2 +- include/kvm/arm_psci.h | 2 +- include/linux/arm-smccc.h | 13 +++++++++++++ 5 files changed, 39 insertions(+), 4 deletions(-) -- 2.7.4 diff --git a/arch/arm/kvm/handle_exit.c b/arch/arm/kvm/handle_exit.c index 0a2b758..de1aedc 100644 --- a/arch/arm/kvm/handle_exit.c +++ b/arch/arm/kvm/handle_exit.c @@ -36,7 +36,7 @@ static int handle_hvc(struct kvm_vcpu *vcpu, struct kvm_run *run) kvm_vcpu_hvc_get_imm(vcpu)); vcpu->stat.hvc_exit_stat++; - ret = kvm_psci_call(vcpu); + ret = kvm_hvc_call_handler(vcpu); if (ret < 0) { vcpu_set_reg(vcpu, 0, ~0UL); return 1; diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 097632c..7b44253 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -15,6 +15,7 @@ * along with this program. If not, see . */ +#include #include #include #include @@ -338,6 +339,7 @@ static int kvm_psci_1_0_call(struct kvm_vcpu *vcpu) case PSCI_0_2_FN_SYSTEM_OFF: case PSCI_0_2_FN_SYSTEM_RESET: case PSCI_1_0_FN_PSCI_FEATURES: + case ARM_SMCCC_VERSION_FUNC_ID: val = 0; break; default: @@ -392,7 +394,7 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) * Errors: * -EINVAL: Unrecognized PSCI function */ -int kvm_psci_call(struct kvm_vcpu *vcpu) +static int kvm_psci_call(struct kvm_vcpu *vcpu) { switch (kvm_psci_version(vcpu)) { case KVM_ARM_PSCI_1_0: @@ -405,3 +407,23 @@ int kvm_psci_call(struct kvm_vcpu *vcpu) return -EINVAL; }; } + +int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) +{ + u32 func_id = smccc_get_function(vcpu); + u32 val = PSCI_RET_NOT_SUPPORTED; + + switch (func_id) { + case ARM_SMCCC_VERSION_FUNC_ID: + val = ARM_SMCCC_VERSION_1_1; + break; + case ARM_SMCCC_ARCH_FEATURES_FUNC_ID: + /* Nothing supported yet */ + break; + default: + return kvm_psci_call(vcpu); + } + + smccc_set_retval(vcpu, val, 0, 0, 0); + return 1; +} diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 0231ebc..d4047fc 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -43,7 +43,7 @@ static int handle_hvc(struct kvm_vcpu *vcpu, struct kvm_run *run) kvm_vcpu_hvc_get_imm(vcpu)); vcpu->stat.hvc_exit_stat++; - ret = kvm_psci_call(vcpu); + ret = kvm_hvc_call_handler(vcpu); if (ret < 0) { vcpu_set_reg(vcpu, 0, ~0UL); return 1; diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index 3236043..ed1dd80 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -27,6 +27,6 @@ #define KVM_ARM_PSCI_LATEST KVM_ARM_PSCI_1_0 int kvm_psci_version(struct kvm_vcpu *vcpu); -int kvm_psci_call(struct kvm_vcpu *vcpu); +int kvm_hvc_call_handler(struct kvm_vcpu *vcpu); #endif /* __KVM_ARM_PSCI_H__ */ diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index 4c5bca38..dc68aa5 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -60,6 +60,19 @@ #define ARM_SMCCC_QUIRK_NONE 0 #define ARM_SMCCC_QUIRK_QCOM_A6 1 /* Save/restore register a6 */ +#define ARM_SMCCC_VERSION_1_0 0x10000 +#define ARM_SMCCC_VERSION_1_1 0x10001 + +#define ARM_SMCCC_VERSION_FUNC_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_32, \ + 0, 0) + +#define ARM_SMCCC_ARCH_FEATURES_FUNC_ID \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_32, \ + 0, 1) + #ifndef __ASSEMBLY__ #include From patchwork Thu Mar 1 12:54:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130214 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2764801edc; Thu, 1 Mar 2018 05:00:27 -0800 (PST) X-Google-Smtp-Source: AG47ELuqZy05gGX5IIH70SA0IqbIks5QWdvkSu2H6/dpq4z+wEUFbpuAxb6ko54SOTVZHiWWFH5x X-Received: by 10.98.89.156 with SMTP id k28mr1856637pfj.130.1519909227468; Thu, 01 Mar 2018 05:00:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909227; cv=none; d=google.com; s=arc-20160816; b=yLtzzAoMiNwoIfkZM5AoxCu5O/M/epP0gvjjjSX4qjkf3O8hx7BOzJTzqYTPL1qOzH Vw5fn0j3KP4Pzs2c1VivmOvqOZQd+Bf/ByvYJTXUfXWfcar6XKoSDMF0P3pMzf6D6s2F oXI/UIzj+qF0Dwnjue4fF3u1xQW5sXCDXl8O7xcgdM3HiVjb2iEDNANuFwZi7flbM3fy KoxL6UvB7ZLRNhdryOHxXNXhp+fq4YXWPF5hi4QvIQ/ZggDiW4/z8f1tk9IDZQRGuMTV hSeyXO/sMp/h3vUH6BtittlW57m7hYPZ6J96dzpnTrGVAl0+HIWJudsOiu5j3uCwcLfe Y2hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=ccWpB6B5MEm475gq8sjuImnJLMpKY94QLsRYdXyhfpw=; b=xGNCJVfC7y0l2iGx2mX+dT6QeG2673qeY24WMPI/LQ5ZTWtK9MtVTVbIHcu71rZnBf e5iO1IO4BNx4uyapjQ6TwcX9Jadt68MkOkQS723Tl43wwKw7rkXpvQvYqaamth3WDSaL IypZQshgo3CQPZ5LTQi35NJgpwC8GCB+HyzrBQ91zHe58+DZqOvDwt95XA2QFo/X2Vr0 rWRNPzeJZNfWA58K9n9EfGEPXWuNro9+xLc6gyp+rmnFZ0kvZYT0ccAjI4vrn9gCXnW5 A6k4xJhbP8JfG3lZ6wxh9wasMzEDoba2/7N/7/Zcawze4LeeQy4JoRnllMgE3bWzuJ5X hwWw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=eBmCz7H/; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a5-v6si3003318plh.450.2018.03.01.05.00.22; Thu, 01 Mar 2018 05:00:27 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=eBmCz7H/; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031092AbeCANAS (ORCPT + 28 others); Thu, 1 Mar 2018 08:00:18 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:44205 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030915AbeCANAO (ORCPT ); Thu, 1 Mar 2018 08:00:14 -0500 Received: by mail-pg0-f67.google.com with SMTP id l4so2267941pgp.11 for ; Thu, 01 Mar 2018 05:00:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ccWpB6B5MEm475gq8sjuImnJLMpKY94QLsRYdXyhfpw=; b=eBmCz7H/8d4JQQbcrfHGWDJ3JynokepH2HqubcLqlc3m1OaXtObxOsE2XLy7O+HbA0 4rYWONP5Ps/H1G5zQpnv9Hvgg/CPGjM0tU+NlAJ2zzZjixcfYPGMv36Ipzmzhkc58lp3 0HYZPoQ7dSUd7csijiGK7lVrGddwCDEGZZokI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ccWpB6B5MEm475gq8sjuImnJLMpKY94QLsRYdXyhfpw=; b=RHkIn33Zk2P8/6RnP6mtGUKudVDV67If0AEUE8lWroyTLG+/USNPn7z6IA3Oy7bVN3 xY6TPcVQVEZKAqfbd7+OkSq675lDuZ4gIKSq4cyoGSCJBaTLWEyW2riqVMMMBjnp/72f WcHCKbEz4J5QTkKfGjTz7B3d/TWAD65e9EiP2qdJiHOmRd9orwn/XXVT6+MJl5cZOyGV 9FMnmRKA6m0QvNJimIz00zImQobLyrTZmWko4gd1xWOt22gt2rHneFiwie7c/TXvFHZG y3HgJjgcWbfcKcTtJCGaBoDRKAkXt+2cg3Jx/Etb6XENLAWZWtkyDTfiernLLAym1UkW JGew== X-Gm-Message-State: APf1xPCoTEFc1EmZMfYFYX2IzFTt4kP2dSCJatf40PR4bFgErciGXkUL sIMkW6cSE6iqDP9fHSiEV0iZkw== X-Received: by 10.101.73.197 with SMTP id t5mr1513480pgs.426.1519909212255; Thu, 01 Mar 2018 05:00:12 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.00.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:00:11 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 36/45] arm64: KVM: Make PSCI_VERSION a fast path Date: Thu, 1 Mar 2018 20:54:13 +0800 Message-Id: <1519908862-11425-37-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 90348689d500 upstream. For those CPUs that require PSCI to perform a BP invalidation, going all the way to the PSCI code for not much is a waste of precious cycles. Let's terminate that call as early as possible. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/kvm/hyp/switch.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index cf6d962..3eab6ac 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -17,6 +17,7 @@ #include #include +#include #include #include @@ -308,6 +309,18 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu) if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu)) goto again; + if (exit_code == ARM_EXCEPTION_TRAP && + (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC64 || + kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32) && + vcpu_get_reg(vcpu, 0) == PSCI_0_2_FN_PSCI_VERSION) { + u64 val = PSCI_RET_NOT_SUPPORTED; + if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) + val = 2; + + vcpu_set_reg(vcpu, 0, val); + goto again; + } + if (static_branch_unlikely(&vgic_v2_cpuif_trap) && exit_code == ARM_EXCEPTION_TRAP) { bool valid; From patchwork Thu Mar 1 12:54:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130215 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2764920edc; Thu, 1 Mar 2018 05:00:31 -0800 (PST) X-Google-Smtp-Source: AG47ELuLGPNQX4wcFMHEdudaSzphCnnHPX+COp1NLoJ+F8OhfQvFk47WUm8VaHj4s05bz1CZ6UL3 X-Received: by 2002:a17:902:5327:: with SMTP id b36-v6mr1855624pli.332.1519909231597; Thu, 01 Mar 2018 05:00:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909231; cv=none; d=google.com; s=arc-20160816; b=fKbI3ceyqkW+0l4gqPREikCTNnlyZAKcwAyLdcg3Z+9UuPLYuGzzmwr2+3PApbVIiL GjQl/j1+0OIkJ/F3jzW+L5S1u0ffLYF54qSW036bTGN5K0KDCI/dUuet4yFJ7r294+Sy 8Q18LTSTW8LrENlMt7hyHqvOQ4/m2YAwMzt7DKpKJUHB5xazIuNQxoUhK/AnjEIhXG6Z YxJ1Wgo/8MO4SevpSXec5L10QCTX/NpJJcrw2hQ2lMqMj7f3UtyvE3Jd78XsN7ofRUGt EzGooi1Si5Tc/Mu8roQl1DTMCP4XZeqqM8SR9mDB+RvlhZkMciThfXybGl6eUeuRwLL2 QABQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=/+g8Mla38uMrMfR6tapxbEKrs5n2o8KvDdig+FWlP0U=; b=GANCJeVg2CRY3nkGJONtjnXMrCb2XvFwtUnJHW7z0VuFVO+hlv00dRoTvtqBCpb9av E3uXDYO+HfJ/HUSeYQ1pNy2KPF8ptn2g7V9fhfT6lvKfXvXXCKdgv7z8iQx3yvrA8ePa VVQr845PifHr00imcU7KggwM7XyHnOLD/Wb5o6p6e3Y6npTjCdIexM9S5t7Yrk1TlO/w 7rUQ58q15nyH6kxUWbTps8hFxMxc1LM9EakOgoJNYhuog0kHLQcMrF7WVSqITjnRfaUa ZOCiZ2QeAbHGuONEVs2n7ue6aI49kvO7Cd4cQFbpEyIXsDQAz1b3uCzLWmuUSux65FsH EQUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DQBYvdGO; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j186si2460280pgc.175.2018.03.01.05.00.29; Thu, 01 Mar 2018 05:00:31 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DQBYvdGO; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031107AbeCANAZ (ORCPT + 28 others); Thu, 1 Mar 2018 08:00:25 -0500 Received: from mail-pl0-f65.google.com ([209.85.160.65]:38271 "EHLO mail-pl0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031094AbeCANAU (ORCPT ); Thu, 1 Mar 2018 08:00:20 -0500 Received: by mail-pl0-f65.google.com with SMTP id m22-v6so75269pls.5 for ; Thu, 01 Mar 2018 05:00:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/+g8Mla38uMrMfR6tapxbEKrs5n2o8KvDdig+FWlP0U=; b=DQBYvdGOnOaO7HF/5ssmRJUALIhN7DYHPWq7nj+87TgOt3gdCCDAxhYD5ljJmPXktW o2AeKIRGxN5gALi3mVK7npTIcZlDtdD1Xm5c9IPmUznpPSF/GkerHM6fkKa/aRNlgH7p eZN+KWeZOwh44hgnNH7lRYaY20Qi/KLrqS9dw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/+g8Mla38uMrMfR6tapxbEKrs5n2o8KvDdig+FWlP0U=; b=SEcsxdmDFhkY7i70Ca2OA5kxllhbgHiaHT8DY9SuH2/CcL2MuLYuLj6BScaWCS2Wjl j/LquTkozISM3Z3cbs/HFfpoGLRZGu9nySTceP9qzCv6NgcIF8sE+iAD8emzH/LNdRdL maTsYlevB77wZ0raKd/Ch/SUlCU9ahLBCEXcn9kkM3cANt0NYrse4TdQa7N8Br/dVzxo cqLMF2Qf5JEcd8lvRCwAqH1EEeSb0I8CdL/Bekp//O5iI+Si0E18DxUMXvixsLm3WRxK Q77qGjqqtMYIfCMaC7vw+Cmm8YTWyC5HFBylrMqcIijUOKafh32xlDY0CEqW2Ysoa8R+ B57Q== X-Gm-Message-State: APf1xPC+K6uY4ge8CdI0uPqEN7XTknjCVzG7ZtIs6TzjEwFGI6TruiIc WCH6PRrSVU7Nn/+V650nAGhdtQ== X-Received: by 2002:a17:902:968d:: with SMTP id n13-v6mr1912241plp.33.1519909220204; Thu, 01 Mar 2018 05:00:20 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.00.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:00:19 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 37/45] arm/arm64: KVM: Turn kvm_psci_version into a static inline Date: Thu, 1 Mar 2018 20:54:14 +0800 Message-Id: <1519908862-11425-38-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit a4097b351118 upstream. We're about to need kvm_psci_version in HYP too. So let's turn it into a static inline, and pass the kvm structure as a second parameter (so that HYP can do a kern_hyp_va on it). Tested-by: Ard Biesheuvel Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/kvm/psci.c | 12 ++---------- arch/arm64/kvm/hyp/switch.c | 20 ++++++++++++-------- include/kvm/arm_psci.h | 21 ++++++++++++++++++++- 3 files changed, 34 insertions(+), 19 deletions(-) -- 2.7.4 diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 7b44253..396eb5d 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -121,7 +121,7 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) if (!vcpu) return PSCI_RET_INVALID_PARAMS; if (!vcpu->arch.power_off) { - if (kvm_psci_version(source_vcpu) != KVM_ARM_PSCI_0_1) + if (kvm_psci_version(source_vcpu, kvm) != KVM_ARM_PSCI_0_1) return PSCI_RET_ALREADY_ON; else return PSCI_RET_INVALID_PARAMS; @@ -231,14 +231,6 @@ static void kvm_psci_system_reset(struct kvm_vcpu *vcpu) kvm_prepare_system_event(vcpu, KVM_SYSTEM_EVENT_RESET); } -int kvm_psci_version(struct kvm_vcpu *vcpu) -{ - if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) - return KVM_ARM_PSCI_LATEST; - - return KVM_ARM_PSCI_0_1; -} - static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) { struct kvm *kvm = vcpu->kvm; @@ -396,7 +388,7 @@ static int kvm_psci_0_1_call(struct kvm_vcpu *vcpu) */ static int kvm_psci_call(struct kvm_vcpu *vcpu) { - switch (kvm_psci_version(vcpu)) { + switch (kvm_psci_version(vcpu, vcpu->kvm)) { case KVM_ARM_PSCI_1_0: return kvm_psci_1_0_call(vcpu); case KVM_ARM_PSCI_0_2: diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 3eab6ac..996328e 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -19,6 +19,8 @@ #include #include +#include + #include #include #include @@ -311,14 +313,16 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu) if (exit_code == ARM_EXCEPTION_TRAP && (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC64 || - kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32) && - vcpu_get_reg(vcpu, 0) == PSCI_0_2_FN_PSCI_VERSION) { - u64 val = PSCI_RET_NOT_SUPPORTED; - if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) - val = 2; - - vcpu_set_reg(vcpu, 0, val); - goto again; + kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32)) { + u32 val = vcpu_get_reg(vcpu, 0); + + if (val == PSCI_0_2_FN_PSCI_VERSION) { + val = kvm_psci_version(vcpu, kern_hyp_va(vcpu->kvm)); + if (unlikely(val == KVM_ARM_PSCI_0_1)) + val = PSCI_RET_NOT_SUPPORTED; + vcpu_set_reg(vcpu, 0, val); + goto again; + } } if (static_branch_unlikely(&vgic_v2_cpuif_trap) && diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index ed1dd80..e518e4e 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -18,6 +18,7 @@ #ifndef __KVM_ARM_PSCI_H__ #define __KVM_ARM_PSCI_H__ +#include #include #define KVM_ARM_PSCI_0_1 PSCI_VERSION(0, 1) @@ -26,7 +27,25 @@ #define KVM_ARM_PSCI_LATEST KVM_ARM_PSCI_1_0 -int kvm_psci_version(struct kvm_vcpu *vcpu); +/* + * We need the KVM pointer independently from the vcpu as we can call + * this from HYP, and need to apply kern_hyp_va on it... + */ +static inline int kvm_psci_version(struct kvm_vcpu *vcpu, struct kvm *kvm) +{ + /* + * Our PSCI implementation stays the same across versions from + * v0.2 onward, only adding the few mandatory functions (such + * as FEATURES with 1.0) that are required by newer + * revisions. It is thus safe to return the latest. + */ + if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) + return KVM_ARM_PSCI_LATEST; + + return KVM_ARM_PSCI_0_1; +} + + int kvm_hvc_call_handler(struct kvm_vcpu *vcpu); #endif /* __KVM_ARM_PSCI_H__ */ From patchwork Thu Mar 1 12:54:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130216 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2765073edc; Thu, 1 Mar 2018 05:00:38 -0800 (PST) X-Google-Smtp-Source: AG47ELv/odQXiECAx3jUKa1uHpUgGNzyDRW9r3k5b/GCRgnsUPH0NwFSW+/lKXZo7iYEMaJt13Ow X-Received: by 10.99.64.198 with SMTP id n189mr1476017pga.191.1519909238098; Thu, 01 Mar 2018 05:00:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909238; cv=none; d=google.com; s=arc-20160816; b=RxvaIQGYm7b4LYJkum+wBhD9TWFbYsoDnZY7umD6AnlGPV8dvx1r2XygV8Wi6vcpjq 7mdInJigQgtitoszqyvsXRiajK4CwHqeWW2uumhSFgFd0/s5yBaltPHHYqGSLFCj7JU+ 22fTiaZGTqKe5S01/bLe6eJNB/6rJ/o//aDKYz/KiQQva0+GVPBo3K/EiSR+8JJszFQI 2Ko2hYggFSuHtrIWlJO4xbCpXJU8+Tofm3gpFzbqtknW2i8ajjiTMLu395yTngkxGpGs sDaHDDP3QUUYMvBIa4LzOdiUD66lrtB0LZOvP32PjrE4I+XKOywsO3lk4kdTlCusUkoa XbWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=ZmjCFMUWnpLHU7w3vwDJv0/nQVwyUfFzmKW+/5xa0BE=; b=mvwFwcWOJJzQexpN/Nq6QwuliDePXDfW3EzXL4IzOmx0IlO/c+ouqcwb9nokuv9Qtq peOwgdQInJ8K9m1UHXvt/dUYolbfQ2J8Hwz+lNM2lvysVFWDQgNYSHHOnw1a7MnaLxtl 6U9QGmBl79qXUbLDTmtOmaRaLNkmBpRAm892C2CS6sLgj380jL7RAbpn5ep3m0XDSLmr bZ65YNd4lFRLZ0yh8k9zCVwtd4VSwTd/LK+l0R26J400DD6ywNKwSdvPVssx+p75sVR8 +QVA0/E5iWInvM1Bpw0z7eHY6l7+BvskYPa7IyE1KZExMUX4DWxuDdyEqsgEkUSTZgOE UOtw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gHsG8ThO; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b92-v6si3021336plb.747.2018.03.01.05.00.37; Thu, 01 Mar 2018 05:00:38 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gHsG8ThO; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031121AbeCANAb (ORCPT + 28 others); Thu, 1 Mar 2018 08:00:31 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:41879 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031094AbeCANA1 (ORCPT ); Thu, 1 Mar 2018 08:00:27 -0500 Received: by mail-pf0-f194.google.com with SMTP id f80so2402415pfa.8 for ; Thu, 01 Mar 2018 05:00:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ZmjCFMUWnpLHU7w3vwDJv0/nQVwyUfFzmKW+/5xa0BE=; b=gHsG8ThOKmqnIMDNZ2JSmqQeEg7xqG9RdQOFCsJp1Q6Ef9LDvMrlzIvDp8AJUOXAr6 M0u2HfsWBjU9SLvNemokIG+YT3IVY54EOPgs/v69t6ABAzXgjmGrCzBOXQ1Y1/wJTCQg GOCRZa3om6G4IQdTsIMNLJRB1XNypZZ7RIJWE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=ZmjCFMUWnpLHU7w3vwDJv0/nQVwyUfFzmKW+/5xa0BE=; b=gcaRxQf3PMvLj2muSQzMu7eOh5Gy4jXW7X7JmS152jkAAviEXIyeWo8S/gS0xThYSW 1ZzaqIHlv97aF/9EIXRPYs20RQV+pGv6KqhTg8doTwskDJaJ4z31VoHRYapJQV9pyiNj Zv6brNPI9Hms+7YBW8BVTshCL4mcBNvhm3IU3NPOqKriWIvAOPYPkew64uRlYmstz9HQ b7CvFHXRuKLk6Uc4519ZfWton3Jvg4SmL7jsUxGiZvaRiwSWcKuWk/4C8JlqGmPYpsvy oEGd+cPym7vhr0x6egq8tHWHdFRvWCpFQjXkXiUT3HJbYy84BNUXTnWLdCHoVR79CKwJ 6r1w== X-Gm-Message-State: APf1xPAlIz36JOfZIazh0jNwePwKu+ambiE5EWx4D7IhBl/aLoE/MU2P PnfA/f52qgWCjSIsYBX7WF9FrQ== X-Received: by 10.99.175.87 with SMTP id s23mr1497684pgo.328.1519909227314; Thu, 01 Mar 2018 05:00:27 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.00.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:00:26 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 38/45] arm64: KVM: Report SMCCC_ARCH_WORKAROUND_1 BP hardening support Date: Thu, 1 Mar 2018 20:54:15 +0800 Message-Id: <1519908862-11425-39-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 6167ec5c9145 upstream. A new feature of SMCCC 1.1 is that it offers firmware-based CPU workarounds. In particular, SMCCC_ARCH_WORKAROUND_1 provides BP hardening for CVE-2017-5715. If the host has some mitigation for this issue, report that we deal with it using SMCCC_ARCH_WORKAROUND_1, as we apply the host workaround on every guest exit. Tested-by: Ard Biesheuvel Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/include/asm/kvm_host.h | 6 ++++++ arch/arm/kvm/psci.c | 9 ++++++++- arch/arm64/include/asm/kvm_host.h | 6 ++++++ include/linux/arm-smccc.h | 5 +++++ 4 files changed, 25 insertions(+), 1 deletion(-) -- 2.7.4 diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h index d5423ab..9fe1043 100644 --- a/arch/arm/include/asm/kvm_host.h +++ b/arch/arm/include/asm/kvm_host.h @@ -318,4 +318,10 @@ static inline int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu, return -ENXIO; } +static inline bool kvm_arm_harden_branch_predictor(void) +{ + /* No way to detect it yet, pretend it is not there. */ + return false; +} + #endif /* __ARM_KVM_HOST_H__ */ diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 396eb5d..a825638 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -404,13 +404,20 @@ int kvm_hvc_call_handler(struct kvm_vcpu *vcpu) { u32 func_id = smccc_get_function(vcpu); u32 val = PSCI_RET_NOT_SUPPORTED; + u32 feature; switch (func_id) { case ARM_SMCCC_VERSION_FUNC_ID: val = ARM_SMCCC_VERSION_1_1; break; case ARM_SMCCC_ARCH_FEATURES_FUNC_ID: - /* Nothing supported yet */ + feature = smccc_get_arg1(vcpu); + switch(feature) { + case ARM_SMCCC_ARCH_WORKAROUND_1: + if (kvm_arm_harden_branch_predictor()) + val = 0; + break; + } break; default: return kvm_psci_call(vcpu); diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index e505038..7f5244a 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -393,4 +393,10 @@ static inline void __cpu_init_stage2(void) "PARange is %d bits, unsupported configuration!", parange); } + +static inline bool kvm_arm_harden_branch_predictor(void) +{ + return cpus_have_cap(ARM64_HARDEN_BRANCH_PREDICTOR); +} + #endif /* __ARM64_KVM_HOST_H__ */ diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index dc68aa5..e1ef944 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -73,6 +73,11 @@ ARM_SMCCC_SMC_32, \ 0, 1) +#define ARM_SMCCC_ARCH_WORKAROUND_1 \ + ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, \ + ARM_SMCCC_SMC_32, \ + 0, 0x8000) + #ifndef __ASSEMBLY__ #include From patchwork Thu Mar 1 12:54:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130217 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2765459edc; Thu, 1 Mar 2018 05:00:52 -0800 (PST) X-Google-Smtp-Source: AG47ELval59Qny+fOBAN5qNZubSx0ALq+6RLyzEcSpkxmai3p1GdcTaLTtHeKE+lkJ9dv8DYRmXH X-Received: by 2002:a17:902:b10b:: with SMTP id q11-v6mr1869833plr.275.1519909251907; Thu, 01 Mar 2018 05:00:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909251; cv=none; d=google.com; s=arc-20160816; b=ySvkvsUWsfM4CdEhArzZZmiRMCZ3ISVoAU4Anr/e27cIZQa7jT/VksQvEXQ8MYwJuO zXRBLkGoFu+gMQi20/+bMSFQ6C49UT2JlOf461kGiMcXZkCfSwsArbYQLBlL4jN9JYm8 8Y7gAP0jQvKvk2Ry28KeI5Dgz53TE+0pjn9OYoVaUQZ1r+RkyC4x5e2V6VRurFGnMIuz e8I/Ke+H92vvF6s9ELNZCa6f1Uv/vEVlUEhRVsAKnsyDf9IfkvUN5qNA+WR8E7QbgO99 rPEyQMJpYrxo3jzzQBl9H4iJBWGOYdOWGejARXtjdxtI5LM5mhiFB9Sc+frzwUWQK6x/ 0lFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=Jl8J60UuHWw6QycIueIONGAI64GFje0f/SkXAPYopdo=; b=I2f8YQ1SgXNkx9GacryaSxkRDOflNb3jRMTwwn2Y74OPaRgkKuwL2e/rCBtRD7FxPK Kg55h2GzvuyshNFAy6TE2D1Qfrt/8p1IQybb92D421da1GjtNGaAEwU7fvh+NMyP5e4B ErcLQNFze1b2tmGGaHNGuiR1NXzaJ6ygisxuHb73bfTFArkX+a0XO55XuHzkrvvSffYX QjR6YseWUm5mKJ+KJ0iEYtmGscJ78T/6AFjfvbJ0/Sq52ElxHCkumOZdjO9xBvb80gp9 TjSWGQWTFT3PEHB6SneZ0anS/Q2w74RTFSOAC/LJkLZiw9HaI9PVeuf5jjLcCn0VhWkP KvKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gD/uH45g; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k3si2956523pff.157.2018.03.01.05.00.41; Thu, 01 Mar 2018 05:00:50 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gD/uH45g; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031132AbeCANAj (ORCPT + 28 others); Thu, 1 Mar 2018 08:00:39 -0500 Received: from mail-pl0-f66.google.com ([209.85.160.66]:41922 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031123AbeCANAf (ORCPT ); Thu, 1 Mar 2018 08:00:35 -0500 Received: by mail-pl0-f66.google.com with SMTP id k8-v6so3585999pli.8 for ; Thu, 01 Mar 2018 05:00:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Jl8J60UuHWw6QycIueIONGAI64GFje0f/SkXAPYopdo=; b=gD/uH45gJQjkdY3z8SkrUQfQi7/JGBkpWDviFFCxtDiNpFSn2DftipKVeDNFfxnx0j XjwYO8uDP/ncXEdrz77Rk7lXHvN4pUIKPhYJ/1DqChxlxEJAPlddN9iceNqp98Ueqyy3 S7jqfFEl2X14rc9fKk1R0ulLD488F7ag31+kA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Jl8J60UuHWw6QycIueIONGAI64GFje0f/SkXAPYopdo=; b=XOpnzBlGFAnyxoDqIPzvAGsTmNoK005q8pVw9RxuHQQqsdGzLQSxe7/Zhx4wlwa07d v6CSVwAMvC44CuLrY+gPR6XR+dTiX38ymPydSkeBPHxHpgXuxdbNustelEWEFVVhJYC5 iV6C35odUpJcokpfmSJMCmdgppWScJQCjotn/s/d59x4vxE0rquyFmxP01pkVa/o+lal I3186iSClKQoZUNOcGnhTINyd4Ew+4DSxoL62BQr0netzPQZ3sVIXbYVn0EARmrF3KKA XkAJ4EaHM45WT+c8oOSA394KQVi7b1mswc10WZBZeBjYT35SClK6qNpoK6HCZ+HfNiWo Rc4A== X-Gm-Message-State: APf1xPDyloFnteHbuk6gDencD9TJh4cX/oK8D7QC/nuCdPgh76VfX+zC 7LmpaXM2M86jFunJiYtLUZ5ywkhM000= X-Received: by 2002:a17:902:9882:: with SMTP id s2-v6mr1918937plp.196.1519909235300; Thu, 01 Mar 2018 05:00:35 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.00.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:00:34 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 39/45] arm64: KVM: Add SMCCC_ARCH_WORKAROUND_1 fast handling Date: Thu, 1 Mar 2018 20:54:16 +0800 Message-Id: <1519908862-11425-40-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit f72af90c3783 upstream. We want SMCCC_ARCH_WORKAROUND_1 to be fast. As fast as possible. So let's intercept it as early as we can by testing for the function call number as soon as we've identified a HVC call coming from the guest. Tested-by: Ard Biesheuvel Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kvm/hyp/hyp-entry.S | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 4e92399..4e9d50c 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -15,6 +15,7 @@ * along with this program. If not, see . */ +#include #include #include @@ -79,10 +80,11 @@ alternative_endif lsr x0, x1, #ESR_ELx_EC_SHIFT cmp x0, #ESR_ELx_EC_HVC64 + ccmp x0, #ESR_ELx_EC_HVC32, #4, ne b.ne el1_trap - mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest - cbnz x1, el1_trap // called HVC + mrs x1, vttbr_el2 // If vttbr is valid, the guest + cbnz x1, el1_hvc_guest // called HVC /* Here, we're pretty sure the host called HVC. */ ldp x0, x1, [sp], #16 @@ -101,6 +103,20 @@ alternative_endif 2: eret +el1_hvc_guest: + /* + * Fastest possible path for ARM_SMCCC_ARCH_WORKAROUND_1. + * The workaround has already been applied on the host, + * so let's quickly get back to the guest. We don't bother + * restoring x1, as it can be clobbered anyway. + */ + ldr x1, [sp] // Guest's x0 + eor w1, w1, #ARM_SMCCC_ARCH_WORKAROUND_1 + cbnz w1, el1_trap + mov x0, x1 + add sp, sp, #16 + eret + el1_trap: /* * x0: ESR_EC From patchwork Thu Mar 1 12:54:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130218 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2765550edc; Thu, 1 Mar 2018 05:00:55 -0800 (PST) X-Google-Smtp-Source: AG47ELt9is9lLY3bVrt39mAgSIiMceCZHYByz10JiCyblVAEJh5Yi2SPweJwQoJT7wL6xyyP/ajj X-Received: by 2002:a17:902:7009:: with SMTP id y9-v6mr1829989plk.395.1519909255022; Thu, 01 Mar 2018 05:00:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909255; cv=none; d=google.com; s=arc-20160816; b=qsF50ohKV8oOi93KWO4fg06lcBKAl6YMbhtVTivtNCUXSHNB9F0PSzf+mDVmAtu/73 aKv3A0kpBZgemV+0PnjmenLP/I9wRLnN/r0Z36TdAmOAY4Snv001Vw0T/gvTKeISNOw6 ++lJN/1+VM75l6a535t0jnyNhNd+yncxCkwXbom54AGlPHmw4dycW2gM9M0NKrnh2W72 2xnZ8Sz6M8XElTC0svg77HbXgAtCBVZHVURalgsBPL63PHw+1qzWl8W+W+cL4PYYO6Lf fN+ElxDMBEvRersxG0a3H/Yx4VrUe+pYxQDxXmPM/oXRFndvGHzIb3ba+Fc8t4ln1FYl KgJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=JR0dWBMHXzRrhmbkXPj+t5C5z62Z2EIDM9UumhZBlIo=; b=akDgzCa5YtuRJNj54LwJKZtDLDBl23n/7+9GevK/PdQXJwTDDXrK3EuVcbNZ7JWPHY gwH40FuH7UNrqguiLbFgu2Vn6KgcVS/C1FaWTB0XxebpYLlUuzLfSUuhZWusSjjapeRs dWzOIFcHp7N0r7TuBYHbtxuRRy/lrHtNzTWyANAQOWRLJQqP5iutUdeZIaIIUFT3TGsE w3cNy8zB2P2UrrHxuo4d2MBB5oiw+oQYzU1tghlUMVKus8nZiYM9iCboSG/eqJ71zJ8x U7L23O3kILeNqcThURD7NnW5sF+gKlAFS8SrbRkocNCpOkC1qliBcLeBx+q54Fzk0BMO 66Kw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cD6/Rp3l; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k3si2956523pff.157.2018.03.01.05.00.51; Thu, 01 Mar 2018 05:00:55 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cD6/Rp3l; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031141AbeCANAr (ORCPT + 28 others); Thu, 1 Mar 2018 08:00:47 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:34525 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030312AbeCANAn (ORCPT ); Thu, 1 Mar 2018 08:00:43 -0500 Received: by mail-pg0-f67.google.com with SMTP id m19so2280109pgn.1 for ; Thu, 01 Mar 2018 05:00:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JR0dWBMHXzRrhmbkXPj+t5C5z62Z2EIDM9UumhZBlIo=; b=cD6/Rp3lDnHqh0fT/wnmni2DoWjyUxCJ4YN8useKtwZWLpatFiXbOZShawbt2vHpJg UYsR0eM+Cd79Dh1YEXxfb4M6nQGRPTw0+BOCH1zJSpGgJKVGhN0EsEdGBNiPaIeC9bNg sNcxuE4Wei3Y20W19JD8UDoef1R636+6x+on8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JR0dWBMHXzRrhmbkXPj+t5C5z62Z2EIDM9UumhZBlIo=; b=ITT3q0cO2072nb6AcPsI8uVVz+eAjPWOJULoTj8F4S+H9tNTjZcn+S3EgqY/D5kdYV rHHRx5xQzl8GiAC2QdPrglcDwche27tD8mGk/hMszs+uskD3FhFBCTrtLe8WM6UolQNn fGSA4DGlnPfjeyfvggwJaMiYwkmd8orMgAaoDRDghaZZGivZtEb6RcBOOv/imykbtT7x SHuj8ERAsxcN3gqP9U8QY6bPKSAVuLEQ18FEUPImWk3cFy7P6VCoC964R8UpbSlWauEx /ScUUSolZnbQ8Vnh9ugVq4ejDjwc0dpaOAxiucpcfziA920NlIDzgBLjZ4V+bzQ6eVJp 7lZA== X-Gm-Message-State: APf1xPAYyHZIcu2V772iYITjlLCXX7oOBGXi8P19WwKL6oxEEIs+xinH XBpBueL5JJvoFhJjaiFa0XkBZ+Ke7x0= X-Received: by 10.98.15.72 with SMTP id x69mr1892866pfi.16.1519909241810; Thu, 01 Mar 2018 05:00:41 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.00.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:00:41 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 40/45] firmware/psci: Expose PSCI conduit Date: Thu, 1 Mar 2018 20:54:17 +0800 Message-Id: <1519908862-11425-41-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 09a8d6d48499 upstream. In order to call into the firmware to apply workarounds, it is useful to find out whether we're using HVC or SMC. Let's expose this through the psci_ops. Acked-by: Lorenzo Pieralisi Reviewed-by: Robin Murphy Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- drivers/firmware/psci.c | 28 +++++++++++++++++++++++----- include/linux/psci.h | 7 +++++++ 2 files changed, 30 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c index 9a3ce76..a49196a 100644 --- a/drivers/firmware/psci.c +++ b/drivers/firmware/psci.c @@ -59,7 +59,9 @@ bool psci_tos_resident_on(int cpu) return cpu == resident_cpu; } -struct psci_operations psci_ops; +struct psci_operations psci_ops = { + .conduit = PSCI_CONDUIT_NONE, +}; typedef unsigned long (psci_fn)(unsigned long, unsigned long, unsigned long, unsigned long); @@ -210,6 +212,22 @@ static unsigned long psci_migrate_info_up_cpu(void) 0, 0, 0); } +static void set_conduit(enum psci_conduit conduit) +{ + switch (conduit) { + case PSCI_CONDUIT_HVC: + invoke_psci_fn = __invoke_psci_fn_hvc; + break; + case PSCI_CONDUIT_SMC: + invoke_psci_fn = __invoke_psci_fn_smc; + break; + default: + WARN(1, "Unexpected PSCI conduit %d\n", conduit); + } + + psci_ops.conduit = conduit; +} + static int get_set_conduit_method(struct device_node *np) { const char *method; @@ -222,9 +240,9 @@ static int get_set_conduit_method(struct device_node *np) } if (!strcmp("hvc", method)) { - invoke_psci_fn = __invoke_psci_fn_hvc; + set_conduit(PSCI_CONDUIT_HVC); } else if (!strcmp("smc", method)) { - invoke_psci_fn = __invoke_psci_fn_smc; + set_conduit(PSCI_CONDUIT_SMC); } else { pr_warn("invalid \"method\" property: %s\n", method); return -EINVAL; @@ -654,9 +672,9 @@ int __init psci_acpi_init(void) pr_info("probing for conduit method from ACPI.\n"); if (acpi_psci_use_hvc()) - invoke_psci_fn = __invoke_psci_fn_hvc; + set_conduit(PSCI_CONDUIT_HVC); else - invoke_psci_fn = __invoke_psci_fn_smc; + set_conduit(PSCI_CONDUIT_SMC); return psci_probe(); } diff --git a/include/linux/psci.h b/include/linux/psci.h index 6306ab1..66ff547 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -25,6 +25,12 @@ bool psci_tos_resident_on(int cpu); int psci_cpu_init_idle(unsigned int cpu); int psci_cpu_suspend_enter(unsigned long index); +enum psci_conduit { + PSCI_CONDUIT_NONE, + PSCI_CONDUIT_SMC, + PSCI_CONDUIT_HVC, +}; + struct psci_operations { u32 (*get_version)(void); int (*cpu_suspend)(u32 state, unsigned long entry_point); @@ -34,6 +40,7 @@ struct psci_operations { int (*affinity_info)(unsigned long target_affinity, unsigned long lowest_affinity_level); int (*migrate_info_type)(void); + enum psci_conduit conduit; }; extern struct psci_operations psci_ops; From patchwork Thu Mar 1 12:54:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130219 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2765756edc; Thu, 1 Mar 2018 05:01:01 -0800 (PST) X-Google-Smtp-Source: AG47ELuXIud6X1IgrrHNHq7ET00btz/o09UGquRXKB9SSD6aIPXsuagHM1HQQieGDcH3m4wODCJ8 X-Received: by 2002:a17:902:59d3:: with SMTP id d19-v6mr1856601plj.356.1519909261701; Thu, 01 Mar 2018 05:01:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909261; cv=none; d=google.com; s=arc-20160816; b=QQzuckZrrORp2uhbK0amcr47/gtAZCZl4IBWGERiacFxaE/lgNjhRvv3VT/hAC6PkV /fu0EQQRqafZoPJTiueirOgsmHdhXwVzhxJ7QrkUp0hfV2EqzKR3/tjuW9UN02GlCbjR N5KcDoGtjGq11sZDz8lsQAZbHynj13Bwa92AnVDRE14j8RTsQYAaVHBEZ9AhyLA/y/He LRCm08LrIiT84IODBM5P2uObsCCd1o/fSdmh5AIsS4if0LrheZCwpj4ZOhcCrdPrR8pb H6jHHbP4XhxTbLqVn16byxfnz2xJmy2r0QAfyeJ7e7lUuSz2CwcLsETztJCCjkoZ82re nGUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=HEY4UErd4E33sLr2mfE4fjyOkfyGVz0DVWFLdDXHqoI=; b=to++RE3UVRzX7YkjlXZWF3EzB1Mcs0jLPWZXz88HJk/Vm4PqfdBrpC6rmbyNGuqlge xMX1QvNJyg+jgHqDAWHIAqmS53TNIp0lqaaTd8V9PVqj915gzUvPK1dEgBAA/vkNJI9d 4vOmYi9UOEGjoHQjVzVu1sldx+1TyWBVPP/DQcohYAy3/+9UGzruoHXxzX53ArmzyMxv Isp4kIH7nX9bSAafrjeinwPYMWlLnB49mdjfy6gWoXu6iLvgi4BfE2Ip6qJopl1vZ+LK yVpH4APxltLV0kDRo6vLiH25gJ47cuJF1H9DBxq9/aUt18iofJu//kDjB3q8fNgGXN4s JTig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OsId/04A; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 7-v6si3032649ple.721.2018.03.01.05.00.55; Thu, 01 Mar 2018 05:01:01 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OsId/04A; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031152AbeCANAw (ORCPT + 28 others); Thu, 1 Mar 2018 08:00:52 -0500 Received: from mail-pl0-f66.google.com ([209.85.160.66]:43546 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030312AbeCANAu (ORCPT ); Thu, 1 Mar 2018 08:00:50 -0500 Received: by mail-pl0-f66.google.com with SMTP id f23-v6so3589240plr.10 for ; Thu, 01 Mar 2018 05:00:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=HEY4UErd4E33sLr2mfE4fjyOkfyGVz0DVWFLdDXHqoI=; b=OsId/04AGQ27DlnpR64S/j77k57uAdPQFgXDPf14JyXeFKZxkiVyT9nhHiNMcafSi1 tqOR7MQ7KrO67uAcXwK2E220CyikDhZezJnPbzDDiYq6S3ZmHqx05VINlQL5MvqILVQ9 8C42EuIgMKlqy2cMOnnnZ5KzgEtddRKOiNwsc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=HEY4UErd4E33sLr2mfE4fjyOkfyGVz0DVWFLdDXHqoI=; b=XIwvfpjwTfxIErgydXs86/Q34+WQ6Y5QBRVAtAl6ZYodbj6agiMlEiBKtMaGDpM/85 D2oQ8KeRrZwenNTtOhvJ1xXDJFhIYIyhIsoz55fic17rV0Av2+FOJm6pGlvncQo+/9RU IetIbdY2fdH9n3RySrlGioECoXPqZhTWYGXrZVSrnp8ebW9SHDTuEVT/wnq6hnkeK5ac zxUMbejoDqsGgfXTmddm71apIP0BS7Yds+gUwSbMXgHwzDo9f7pxufNfoaQ/Hbsk0yzW g6bjiW4J09C4Lq3+jpmAU5Q3V0F4iO+mrpvSuEdH+q3v0M/3YMZCJ/6UpPKIzICtFYGa fViA== X-Gm-Message-State: APf1xPBHwGhFZXTRNlfhwD0cGHjFkG1Mb1B5QT/j5Jv0xjBQtLFuEq+U A7GzRsb2iNo2gHeUrq5TC/lSaw== X-Received: by 2002:a17:902:27:: with SMTP id 36-v6mr1885686pla.128.1519909249536; Thu, 01 Mar 2018 05:00:49 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.00.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:00:49 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 41/45] firmware/psci: Expose SMCCC version through psci_ops Date: Thu, 1 Mar 2018 20:54:18 +0800 Message-Id: <1519908862-11425-42-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit e78eef554a91 upstream. Since PSCI 1.0 allows the SMCCC version to be (indirectly) probed, let's do that at boot time, and expose the version of the calling convention as part of the psci_ops structure. Acked-by: Lorenzo Pieralisi Reviewed-by: Robin Murphy Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- drivers/firmware/psci.c | 27 +++++++++++++++++++++++++++ include/linux/psci.h | 6 ++++++ 2 files changed, 33 insertions(+) -- 2.7.4 diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c index a49196a..79a48c3 100644 --- a/drivers/firmware/psci.c +++ b/drivers/firmware/psci.c @@ -61,6 +61,7 @@ bool psci_tos_resident_on(int cpu) struct psci_operations psci_ops = { .conduit = PSCI_CONDUIT_NONE, + .smccc_version = SMCCC_VERSION_1_0, }; typedef unsigned long (psci_fn)(unsigned long, unsigned long, @@ -511,6 +512,31 @@ static void __init psci_init_migrate(void) pr_info("Trusted OS resident on physical CPU 0x%lx\n", cpuid); } +static void __init psci_init_smccc(void) +{ + u32 ver = ARM_SMCCC_VERSION_1_0; + int feature; + + feature = psci_features(ARM_SMCCC_VERSION_FUNC_ID); + + if (feature != PSCI_RET_NOT_SUPPORTED) { + u32 ret; + ret = invoke_psci_fn(ARM_SMCCC_VERSION_FUNC_ID, 0, 0, 0); + if (ret == ARM_SMCCC_VERSION_1_1) { + psci_ops.smccc_version = SMCCC_VERSION_1_1; + ver = ret; + } + } + + /* + * Conveniently, the SMCCC and PSCI versions are encoded the + * same way. No, this isn't accidental. + */ + pr_info("SMC Calling Convention v%d.%d\n", + PSCI_VERSION_MAJOR(ver), PSCI_VERSION_MINOR(ver)); + +} + static void __init psci_0_2_set_functions(void) { pr_info("Using standard PSCI v0.2 function IDs\n"); @@ -559,6 +585,7 @@ static int __init psci_probe(void) psci_init_migrate(); if (PSCI_VERSION_MAJOR(ver) >= 1) { + psci_init_smccc(); psci_init_cpu_suspend(); psci_init_system_suspend(); } diff --git a/include/linux/psci.h b/include/linux/psci.h index 66ff547..347077c 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -31,6 +31,11 @@ enum psci_conduit { PSCI_CONDUIT_HVC, }; +enum smccc_version { + SMCCC_VERSION_1_0, + SMCCC_VERSION_1_1, +}; + struct psci_operations { u32 (*get_version)(void); int (*cpu_suspend)(u32 state, unsigned long entry_point); @@ -41,6 +46,7 @@ struct psci_operations { unsigned long lowest_affinity_level); int (*migrate_info_type)(void); enum psci_conduit conduit; + enum smccc_version smccc_version; }; extern struct psci_operations psci_ops; From patchwork Thu Mar 1 12:54:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130220 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2765864edc; Thu, 1 Mar 2018 05:01:05 -0800 (PST) X-Google-Smtp-Source: AG47ELv7j4iQaCwROXUpOFGcf8/Ka9WL9I6YWEaQE6x0sXPlEN4hU5CdQ6efTsrT1KWfFnq911zC X-Received: by 10.99.124.7 with SMTP id x7mr1485758pgc.356.1519909265208; Thu, 01 Mar 2018 05:01:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909265; cv=none; d=google.com; s=arc-20160816; b=ynotJAfIy4MVURofBXPQHAJRe33Vxe4LDGPZF6SZK6Nz2hUb+b+mrDVEeb/QABEnvd lFxwvE+1IEZ6vboFBm1+Vbai1zO+PodouP3wxJ0Q4bykYW5GlzJp0gXXWW8Pv07ocyrh qoNM2EyL90a2tq3f4gn8LPiwpuXLJWNpctfZgppFpN3o6dAiDTCDOwLhyTlr5pl9nO2V CegMrB65pwZnUkf005SQaSIdGpwvf7Ycsx2l9we019m+Q60TMR70IMXRzKuJrLILw7JL hZamG0Q73KlikmCjZiw1nrsDC3kU2Ng0tslnlbQABkqKJWQBO5U+RFp3pVX+ZaSHTnWT J8Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=xjpJ8k7CKNbvTvoCWi2+8tjs1SrG8L5eeUCBFmDTG3o=; b=p9pnxKkEul6h9Z+XWV6aoTdKB79b1RZkrEAEA/xXzlVAYEwyC9k7c46mxyqX2JkKbj MEvFnKYiIjG2VLPG49Gm6tng+mLlPp43bl7PluabN4RQh+r7K0N22rSF34ZZrNYQ6b4A sXlQXhk8L/vbeuEFkiN9S3xDv2UhGoxo40XMdOEvZTDtqPToYm4FjFrcpnhIO3DOmdeH Yb5SKPhanFnxsX5lbpknmgScBer48R3RdA8iwZlZdR5Z8oe3Xkqcxxsu3WEpUjUtKPLm I+Zd46tWJwya7k2XGvOUPhwCEoawcPtSPMWBa49YTfuKLNBQKaAklQYDVPWxIQvvIksk 1yRg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ULhcJObd; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o5-v6si3032649plh.135.2018.03.01.05.01.04; Thu, 01 Mar 2018 05:01:05 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ULhcJObd; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031165AbeCANBB (ORCPT + 28 others); Thu, 1 Mar 2018 08:01:01 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:34805 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031154AbeCANA6 (ORCPT ); Thu, 1 Mar 2018 08:00:58 -0500 Received: by mail-pf0-f195.google.com with SMTP id j20so2411125pfi.1 for ; Thu, 01 Mar 2018 05:00:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=xjpJ8k7CKNbvTvoCWi2+8tjs1SrG8L5eeUCBFmDTG3o=; b=ULhcJObd7hh6GHhVGPDcF1aqBB2fpD46RptqYqg6FZVnAxO64acvPhAp+B82sOpc9N W1brkOsZBelO139nXlzRrF5YP/oQAuK+c1zwqf8/PA/+1k0AuaTsk51RhO37DC+kdxEs 0jD5orIB1QIJ3OBDgSaqE1zcz686DGtNsBDgc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=xjpJ8k7CKNbvTvoCWi2+8tjs1SrG8L5eeUCBFmDTG3o=; b=edZ13uWge3Vfzp9DQu/TBxqwhP+lQ1V90ElPFVr4lKEQlvxq9LOfcfmJpWFk00HAk1 HPqeMs4EdeKFKStYGuYForsnIJ8O5aCG1HgPQfYYXVEnxRezt3YUtmx35jFZ95OSOHR+ 4NIDW8XWxsCRcNSsNly9zidkBo+5SDrPHir6rriDwBa1xknQ/+NqxXkluMIIZE1V+4UA u+zFgv9I012S2oLN5nyqbB0eZYvpUTF1ImYjqcCDYu8IQ2Yx/egSVAmNh3pXUI8J8HGU n7oAEoMvZJQ0wfizoIXPtNKrntFWrCYcj1GS9MvnUXQqqL7BoD2TesvbgTImH7nSGXAi PHaw== X-Gm-Message-State: APf1xPCvfk58N7sbFooMyRZ3KZme+tzL+7e9faSR815A16N5J0///eCV 5V67kvGwGYGv4z9L9nvB5/02+Q== X-Received: by 10.101.92.66 with SMTP id v2mr1482378pgr.341.1519909256952; Thu, 01 Mar 2018 05:00:56 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.00.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:00:56 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 42/45] arm/arm64: smccc: Make function identifiers an unsigned quantity Date: Thu, 1 Mar 2018 20:54:19 +0800 Message-Id: <1519908862-11425-43-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit ded4c39e93f3 upstream. Function identifiers are a 32bit, unsigned quantity. But we never tell so to the compiler, resulting in the following: 4ac: b26187e0 mov x0, #0xffffffff80000001 We thus rely on the firmware narrowing it for us, which is not always a reasonable expectation. Cc: stable@vger.kernel.org Reported-by: Ard Biesheuvel Acked-by: Ard Biesheuvel Reviewed-by: Robin Murphy Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- include/linux/arm-smccc.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index e1ef944..dd44d84 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -14,14 +14,16 @@ #ifndef __LINUX_ARM_SMCCC_H #define __LINUX_ARM_SMCCC_H +#include + /* * This file provides common defines for ARM SMC Calling Convention as * specified in * http://infocenter.arm.com/help/topic/com.arm.doc.den0028a/index.html */ -#define ARM_SMCCC_STD_CALL 0 -#define ARM_SMCCC_FAST_CALL 1 +#define ARM_SMCCC_STD_CALL _AC(0,U) +#define ARM_SMCCC_FAST_CALL _AC(1,U) #define ARM_SMCCC_TYPE_SHIFT 31 #define ARM_SMCCC_SMC_32 0 From patchwork Thu Mar 1 12:54:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130221 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2766133edc; Thu, 1 Mar 2018 05:01:13 -0800 (PST) X-Google-Smtp-Source: AG47ELsrGJTIzIAmD1X9ucCzJ3DaYyPRUw7a3Z3f7sQkXx1KOHnmRsx2rMxuug36VvVyT0N2Dyjr X-Received: by 10.99.146.91 with SMTP id s27mr1474105pgn.367.1519909273587; Thu, 01 Mar 2018 05:01:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909273; cv=none; d=google.com; s=arc-20160816; b=SUgumPGMO155j5y435mrLJhpvEmQ2Pz/wK34CvZwqdtTwONmJQmJQqWpaXu7Jp47hz 0YaNHUluWVzO6Iyd1nTODiScU2vfkoR95p50rU3xNp3DSA5Tu77t/9Ml+auOv0LDutdG T+XW41VqdYqxtXuvVc3ucDahFzhHv/NwjNA+QrKj+b27bmaAzHdhYJci6OG9rdhXFXYh IbL47x11aOXOjFYmL1p6Hhw1bOHtd6BIWXjip4/1AwcOq8tLZk/WPdEuPJcdfm/T5d+2 3hNG3U6bXZVTPggG1Ne/l464zx/769oRYsgL34VWk1MtqNxXsGtgR02hh5gtzLmfGJch wMvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=Jk3msMDfhwzjnG4JcoVxRkkc9wQeB0h3mvpSGrDHyMI=; b=ZK2wqiPG3shRaZxDkHn7Bl2N+NEY2AL24iif1ACtVVyamEtudBIky5oez4rKrbdmpc eT0g0OndM84QVPYtCl4qychpVjetcegC5/PtfBZ632mjQ4Hb6aOzz4M/XMpchbDOwi3m 4cpH1ruhLS9ECkJHHq4mfBXxKr8lW0Sm3m+Rne1Cks3bxIC+O3k6PjNFysFVJBj4ubea o942NLXS5a9qb2cN5e0CEUAkPb/Nvm6dhwsrh0f8Xez4/RXJAe+2qIsGFE1I6h2rC5eF P6YM2PGJ8mgcOh+3tPnRQoCc+Ka5Q2ERSzIQO9pGs5fPVU8JiqPR+lgMvSbTG9LAKGOX y09A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=S/Gslc34; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o5-v6si3032649plh.135.2018.03.01.05.01.12; Thu, 01 Mar 2018 05:01:13 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=S/Gslc34; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031177AbeCANBJ (ORCPT + 28 others); Thu, 1 Mar 2018 08:01:09 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:34964 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030370AbeCANBF (ORCPT ); Thu, 1 Mar 2018 08:01:05 -0500 Received: by mail-pf0-f195.google.com with SMTP id y186so2412100pfb.2 for ; Thu, 01 Mar 2018 05:01:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Jk3msMDfhwzjnG4JcoVxRkkc9wQeB0h3mvpSGrDHyMI=; b=S/Gslc34SlTb78YvEBOME1UKYXgCZoaUxY8JM3kR30zoYmrl+MCLl6eSsW49mhcr8I xSaurD1sDZBCj6tbXLyrsvxU0C0wg8PCLoz7dOhWhL/B3JSjIl12YbrxK21K6xiGhE6E OXYca/S/VlmpYg1qtUEngNM+DiavWvXvuo4lY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Jk3msMDfhwzjnG4JcoVxRkkc9wQeB0h3mvpSGrDHyMI=; b=SizdZzCSSXJDQYTB/CmjGO94Sk4SMyzvtbENKnVbZDzJMKKHBQDaEvAR+d4Ueaqchg MOMePhEgXE4+qzgAT4BZM80M/RnA6ShQ30H1VaIy3gI+2DZ6FUrkeFsjuv0klcUHkdu8 LbqLKSRp1zq8Q7I4shCwnIu+YJSDkriDf6jKjzAqv0B5z8ueO4NPtH3DuqMMM6DgNR7a vvm/tnZJUpAwIt7gedZwBB7xP8LTI5y7sao9nubEY2sPrUhacilM0oa7WLa0LaPQLgHO Df8dYvjI8cVjK3MRYH+IRCtz4gVgCvIiIfgpdHQKXkEirdw3sJsFYhnCwvVLrbZ8IFvh n/+g== X-Gm-Message-State: APf1xPBZAGf87fqg2vULQjO0SHlXq0gWEpuvq2gArPuFJsiXkA1tCKDE 2yPXD2tTmcFjFGZFnYuUF9IRMg== X-Received: by 10.98.35.68 with SMTP id j65mr1862855pfj.5.1519909264654; Thu, 01 Mar 2018 05:01:04 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.00.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:01:04 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 43/45] arm/arm64: smccc: Implement SMCCC v1.1 inline primitive Date: Thu, 1 Mar 2018 20:54:20 +0800 Message-Id: <1519908862-11425-44-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit f2d3b2e8759a upstream. One of the major improvement of SMCCC v1.1 is that it only clobbers the first 4 registers, both on 32 and 64bit. This means that it becomes very easy to provide an inline version of the SMC call primitive, and avoid performing a function call to stash the registers that would otherwise be clobbered by SMCCC v1.0. Reviewed-by: Robin Murphy Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- include/linux/arm-smccc.h | 141 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) -- 2.7.4 diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index dd44d84..a031897 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -150,5 +150,146 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, #define arm_smccc_hvc_quirk(...) __arm_smccc_hvc(__VA_ARGS__) +/* SMCCC v1.1 implementation madness follows */ +#ifdef CONFIG_ARM64 + +#define SMCCC_SMC_INST "smc #0" +#define SMCCC_HVC_INST "hvc #0" + +#elif defined(CONFIG_ARM) +#include +#include + +#define SMCCC_SMC_INST __SMC(0) +#define SMCCC_HVC_INST __HVC(0) + +#endif + +#define ___count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x + +#define __count_args(...) \ + ___count_args(__VA_ARGS__, 7, 6, 5, 4, 3, 2, 1, 0) + +#define __constraint_write_0 \ + "+r" (r0), "=&r" (r1), "=&r" (r2), "=&r" (r3) +#define __constraint_write_1 \ + "+r" (r0), "+r" (r1), "=&r" (r2), "=&r" (r3) +#define __constraint_write_2 \ + "+r" (r0), "+r" (r1), "+r" (r2), "=&r" (r3) +#define __constraint_write_3 \ + "+r" (r0), "+r" (r1), "+r" (r2), "+r" (r3) +#define __constraint_write_4 __constraint_write_3 +#define __constraint_write_5 __constraint_write_4 +#define __constraint_write_6 __constraint_write_5 +#define __constraint_write_7 __constraint_write_6 + +#define __constraint_read_0 +#define __constraint_read_1 +#define __constraint_read_2 +#define __constraint_read_3 +#define __constraint_read_4 "r" (r4) +#define __constraint_read_5 __constraint_read_4, "r" (r5) +#define __constraint_read_6 __constraint_read_5, "r" (r6) +#define __constraint_read_7 __constraint_read_6, "r" (r7) + +#define __declare_arg_0(a0, res) \ + struct arm_smccc_res *___res = res; \ + register u32 r0 asm("r0") = a0; \ + register unsigned long r1 asm("r1"); \ + register unsigned long r2 asm("r2"); \ + register unsigned long r3 asm("r3") + +#define __declare_arg_1(a0, a1, res) \ + struct arm_smccc_res *___res = res; \ + register u32 r0 asm("r0") = a0; \ + register typeof(a1) r1 asm("r1") = a1; \ + register unsigned long r2 asm("r2"); \ + register unsigned long r3 asm("r3") + +#define __declare_arg_2(a0, a1, a2, res) \ + struct arm_smccc_res *___res = res; \ + register u32 r0 asm("r0") = a0; \ + register typeof(a1) r1 asm("r1") = a1; \ + register typeof(a2) r2 asm("r2") = a2; \ + register unsigned long r3 asm("r3") + +#define __declare_arg_3(a0, a1, a2, a3, res) \ + struct arm_smccc_res *___res = res; \ + register u32 r0 asm("r0") = a0; \ + register typeof(a1) r1 asm("r1") = a1; \ + register typeof(a2) r2 asm("r2") = a2; \ + register typeof(a3) r3 asm("r3") = a3 + +#define __declare_arg_4(a0, a1, a2, a3, a4, res) \ + __declare_arg_3(a0, a1, a2, a3, res); \ + register typeof(a4) r4 asm("r4") = a4 + +#define __declare_arg_5(a0, a1, a2, a3, a4, a5, res) \ + __declare_arg_4(a0, a1, a2, a3, a4, res); \ + register typeof(a5) r5 asm("r5") = a5 + +#define __declare_arg_6(a0, a1, a2, a3, a4, a5, a6, res) \ + __declare_arg_5(a0, a1, a2, a3, a4, a5, res); \ + register typeof(a6) r6 asm("r6") = a6 + +#define __declare_arg_7(a0, a1, a2, a3, a4, a5, a6, a7, res) \ + __declare_arg_6(a0, a1, a2, a3, a4, a5, a6, res); \ + register typeof(a7) r7 asm("r7") = a7 + +#define ___declare_args(count, ...) __declare_arg_ ## count(__VA_ARGS__) +#define __declare_args(count, ...) ___declare_args(count, __VA_ARGS__) + +#define ___constraints(count) \ + : __constraint_write_ ## count \ + : __constraint_read_ ## count \ + : "memory" +#define __constraints(count) ___constraints(count) + +/* + * We have an output list that is not necessarily used, and GCC feels + * entitled to optimise the whole sequence away. "volatile" is what + * makes it stick. + */ +#define __arm_smccc_1_1(inst, ...) \ + do { \ + __declare_args(__count_args(__VA_ARGS__), __VA_ARGS__); \ + asm volatile(inst "\n" \ + __constraints(__count_args(__VA_ARGS__))); \ + if (___res) \ + *___res = (typeof(*___res)){r0, r1, r2, r3}; \ + } while (0) + +/* + * arm_smccc_1_1_smc() - make an SMCCC v1.1 compliant SMC call + * + * This is a variadic macro taking one to eight source arguments, and + * an optional return structure. + * + * @a0-a7: arguments passed in registers 0 to 7 + * @res: result values from registers 0 to 3 + * + * This macro is used to make SMC calls following SMC Calling Convention v1.1. + * The content of the supplied param are copied to registers 0 to 7 prior + * to the SMC instruction. The return values are updated with the content + * from register 0 to 3 on return from the SMC instruction if not NULL. + */ +#define arm_smccc_1_1_smc(...) __arm_smccc_1_1(SMCCC_SMC_INST, __VA_ARGS__) + +/* + * arm_smccc_1_1_hvc() - make an SMCCC v1.1 compliant HVC call + * + * This is a variadic macro taking one to eight source arguments, and + * an optional return structure. + * + * @a0-a7: arguments passed in registers 0 to 7 + * @res: result values from registers 0 to 3 + * + * This macro is used to make HVC calls following SMC Calling Convention v1.1. + * The content of the supplied param are copied to registers 0 to 7 prior + * to the HVC instruction. The return values are updated with the content + * from register 0 to 3 on return from the HVC instruction if not NULL. + */ +#define arm_smccc_1_1_hvc(...) __arm_smccc_1_1(SMCCC_HVC_INST, __VA_ARGS__) + #endif /*__ASSEMBLY__*/ #endif /*__LINUX_ARM_SMCCC_H*/ From patchwork Thu Mar 1 12:54:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130222 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2766687edc; Thu, 1 Mar 2018 05:01:32 -0800 (PST) X-Google-Smtp-Source: AG47ELsEeuIVqfmWP0aG2GwqrmHtLCrGxTJ6pWCrnoMTI5IU1V+DT/0zG8cSekoqGPeb9GKJ/F9j X-Received: by 10.99.95.142 with SMTP id t136mr1473890pgb.94.1519909292066; Thu, 01 Mar 2018 05:01:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909292; cv=none; d=google.com; s=arc-20160816; b=sXPf59rkKtzfzo6e1h9pMmHHiZ5ZQt2/9JKmHp4yZxEWI0kH8FZbUwgrgiztorfYzT OIB5VhpaVLyh861p6DJNYEKPqWjBeTfWWcZ0bQlYhuE4n1LAHM0obhY18iGRu+tLPyc9 OGoga871qC7ugkxFwf/N3QK3DUXKDJOKdwhOoZRF2S/WH7P4Un7krlLWCtrgPp+gk5SM ZgzR7yjzlsWH3Dz4ruFQRYQTKVZ1SRkNTBfGvj8S4rgw2Ut0Y7A8jSsq7hWwgxgT619S CG9y1UyrUOSenGRqFbF3Qu7j9TBiiR3vQpfinC05V04h7paXXqVRqqVo6jUzvBc0zmqr 1w5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=3nJ4z4z9UJLDq0U9q5iOSITdh7PqyHF254/P1MMmfts=; b=yZbKrlDxRiN00DwyEjy1OJxOPYVTqNvxld2DXnOjGOlgRouR7BjQ63jcPL4FNOs+Hn bGYkqXuATttSPvCPQT/0jIM9NSz4M71Zi5fhahhbDP1GMdELSBzNgtUlwWGYV4H7Hf3g FN4Ri4jOxrgKyQwdOgHEbes+c6tnvWUYc1mg5eX6v5i1xfR7tdTIlECw5HlixEZ4PPW5 M9jNIjuenO11wzes4bc47rQkRz8pOqMOyBbwh3O18DUVp0M8yeJqPNOHvQVHV5DBhhJN 2s5ziMKlAyAEAU3HeTsOzIYX+LAjxyYCUA6pKwgMedtjWAmkLYvuUrwJdnarINAhKOk7 e3Aw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Wg1LOuBR; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d21si2998591pfk.328.2018.03.01.05.01.20; Thu, 01 Mar 2018 05:01:32 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Wg1LOuBR; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031027AbeCANBQ (ORCPT + 28 others); Thu, 1 Mar 2018 08:01:16 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:43296 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030370AbeCANBN (ORCPT ); Thu, 1 Mar 2018 08:01:13 -0500 Received: by mail-pg0-f67.google.com with SMTP id e9so2268205pgs.10 for ; Thu, 01 Mar 2018 05:01:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3nJ4z4z9UJLDq0U9q5iOSITdh7PqyHF254/P1MMmfts=; b=Wg1LOuBRMS668yOimNSYIM1saUZk4iDPb3GMf3tokkdrYk8o4bY5Cv01NVVNAinfvj Jv5EeeRoJTplN1p1Nuipz4/QAgvgcrew3/XGu1ka4ESSvztONt35QyLSLXOe6jGnz8UW YQC/+PGUI0A8YLIAHyGWcyRhQqXFEUj2yyO0Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3nJ4z4z9UJLDq0U9q5iOSITdh7PqyHF254/P1MMmfts=; b=K+QSOSmlGt9joAW5hlZ9VbzhnMBK/RLcBnCvMEX0E+EHLk2djZh0NuBZK4EALiiiXm f0hC0hx+7nm1PnfSYbVeXF4o2iUWQdwlsSWETCu/8Wz14l0aEZdcdN9yiJUTU9pLDH9D 6DqcN+4GVG79Tk+t0EpxU99mhFxT9Gw7nM4klkwfj1LnFrhmDaCSc8UkDDgHTNNXUyUp CtsvbUhpL1dPOSGfreYS01nhpz2/caGjuWEMv3fMmzG2kvzPWwWNV7SfhP3/OdlbofeQ zByD1ODnLMKSd2JnQlGUtvxKUJR+LgzlqeyMdCR7D8nXzKGKw2sn0fg6T8g/A4vbEwKm B9VA== X-Gm-Message-State: APf1xPBgP4Z3VBOxYOFt4LrjnkOiO0mpPo/qKXIjtrudk9G4+SyXnv9R Ug+ZpbO2re2dm/eis1hcXKsW9Q== X-Received: by 10.99.178.6 with SMTP id x6mr1536804pge.98.1519909272669; Thu, 01 Mar 2018 05:01:12 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.01.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:01:12 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 44/45] arm64: Add ARM_SMCCC_ARCH_WORKAROUND_1 BP hardening support Date: Thu, 1 Mar 2018 20:54:21 +0800 Message-Id: <1519908862-11425-45-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit b092201e0020 upstream. Add the detection and runtime code for ARM_SMCCC_ARCH_WORKAROUND_1. It is lovely. Really. Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kernel/bpi.S | 20 ++++++++++++ arch/arm64/kernel/cpu_errata.c | 72 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 89 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S index dec95bd..c72f261 100644 --- a/arch/arm64/kernel/bpi.S +++ b/arch/arm64/kernel/bpi.S @@ -17,6 +17,7 @@ */ #include +#include .macro ventry target .rept 31 @@ -77,3 +78,22 @@ ENTRY(__psci_hyp_bp_inval_start) ldp x0, x1, [sp, #(16 * 8)] add sp, sp, #(8 * 18) ENTRY(__psci_hyp_bp_inval_end) + +.macro smccc_workaround_1 inst + sub sp, sp, #(8 * 4) + stp x2, x3, [sp, #(8 * 0)] + stp x0, x1, [sp, #(8 * 2)] + mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1 + \inst #0 + ldp x2, x3, [sp, #(8 * 0)] + ldp x0, x1, [sp, #(8 * 2)] + add sp, sp, #(8 * 4) +.endm + +ENTRY(__smccc_workaround_1_smc_start) + smccc_workaround_1 smc +ENTRY(__smccc_workaround_1_smc_end) + +ENTRY(__smccc_workaround_1_hvc_start) + smccc_workaround_1 hvc +ENTRY(__smccc_workaround_1_hvc_end) diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index f8810bf..9632319 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -54,6 +54,10 @@ DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); #ifdef CONFIG_KVM extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[]; +extern char __smccc_workaround_1_smc_start[]; +extern char __smccc_workaround_1_smc_end[]; +extern char __smccc_workaround_1_hvc_start[]; +extern char __smccc_workaround_1_hvc_end[]; static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start, const char *hyp_vecs_end) @@ -96,8 +100,12 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn, spin_unlock(&bp_lock); } #else -#define __psci_hyp_bp_inval_start NULL -#define __psci_hyp_bp_inval_end NULL +#define __psci_hyp_bp_inval_start NULL +#define __psci_hyp_bp_inval_end NULL +#define __smccc_workaround_1_smc_start NULL +#define __smccc_workaround_1_smc_end NULL +#define __smccc_workaround_1_hvc_start NULL +#define __smccc_workaround_1_hvc_end NULL static void __install_bp_hardening_cb(bp_hardening_cb_t fn, const char *hyp_vecs_start, @@ -124,17 +132,75 @@ static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry, __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end); } +#include +#include #include +static void call_smc_arch_workaround_1(void) +{ + arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL); +} + +static void call_hvc_arch_workaround_1(void) +{ + arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL); +} + +static bool check_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *entry) +{ + bp_hardening_cb_t cb; + void *smccc_start, *smccc_end; + struct arm_smccc_res res; + + if (!entry->matches(entry, SCOPE_LOCAL_CPU)) + return false; + + if (psci_ops.smccc_version == SMCCC_VERSION_1_0) + return false; + + switch (psci_ops.conduit) { + case PSCI_CONDUIT_HVC: + arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, + ARM_SMCCC_ARCH_WORKAROUND_1, &res); + if (res.a0) + return false; + cb = call_hvc_arch_workaround_1; + smccc_start = __smccc_workaround_1_hvc_start; + smccc_end = __smccc_workaround_1_hvc_end; + break; + + case PSCI_CONDUIT_SMC: + arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, + ARM_SMCCC_ARCH_WORKAROUND_1, &res); + if (res.a0) + return false; + cb = call_smc_arch_workaround_1; + smccc_start = __smccc_workaround_1_smc_start; + smccc_end = __smccc_workaround_1_smc_end; + break; + + default: + return false; + } + + install_bp_hardening_cb(entry, cb, smccc_start, smccc_end); + + return true; +} + static int enable_psci_bp_hardening(void *data) { const struct arm64_cpu_capabilities *entry = data; - if (psci_ops.get_version) + if (psci_ops.get_version) { + if (check_smccc_arch_workaround_1(entry)) + return 0; + install_bp_hardening_cb(entry, (bp_hardening_cb_t)psci_ops.get_version, __psci_hyp_bp_inval_start, __psci_hyp_bp_inval_end); + } return 0; } From patchwork Thu Mar 1 12:54:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 130223 Delivered-To: patch@linaro.org Received: by 10.80.172.228 with SMTP id x91csp2766821edc; Thu, 1 Mar 2018 05:01:36 -0800 (PST) X-Google-Smtp-Source: AG47ELuEcGj6CZZUajRD6cIuQqjtclERbnaB6ijuNSrXj85MsebRzcMaDqLc7AghoL8hBjV9mY1g X-Received: by 10.101.82.70 with SMTP id q6mr1474865pgp.67.1519909296278; Thu, 01 Mar 2018 05:01:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519909296; cv=none; d=google.com; s=arc-20160816; b=J8UEu/bSqiY6znBUItibHvInfTsZNBSxOm2vAD2ZU6OvFQV1ti3Y7EsvodEjhz6OmK MpuPJqtJG1cMkRlrU71fanSv8jzwpVvx9Y/n9cFgdGObIMdPatvsACxZlfzH9mo4sd6T dtfOeXfddhNBOXhjPfEpKl8WAcPKXBnBc0A6wIXGC9kBtG2lJgnguQtWbV3GzC+hhT4a HwKUmXjJH7jAwJRV2Pe7szuWhsRZA/seA6YC20uyNX13M8JTmJvu7d9Qf3KTkSnacxKt z2/tHNzCe3JkQkzB9uUjx0f7M5q6r6QmBSZbVd9pij5rBE86d5YQPx66cYJE9PtsVk4l uFyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=7qDP6b/nHj7r0IkKyyPP7XAky1eZBzSNc9JAXbaxcxI=; b=Xeqku0RJiIzuXpGh3+Omo37l9zhc6Ojc+DWM0zkEqsqzbN1UDaAHqXH+HxWRQWEC5B XpOEpxSKzpxQgKyJi3gVycjVtMddknAnkrlNPIGNcpeblMWRZ5zQTWrP6QaMqEhFrBZU xn1Se/me4crg17KW/WIpvi0ZzsUmuWzkTO17mIvIGVqL53S1+1AzHoXGLtkfvXkOZE8x 6rVorQjqIwdXf7bFEWagC0uamsAJ21O6DpClxUrWu09zbVD401CtD5YI2v7GAZ26FEgu 3Ct95C5x8H2B0APtgue+9c6OrGsY4deNCIOfp1foi7yhmMhi9otlLJDNQzDJdNp1BnAg /MoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GKKHh5y+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d21si2998591pfk.328.2018.03.01.05.01.32; Thu, 01 Mar 2018 05:01:36 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GKKHh5y+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031203AbeCANBX (ORCPT + 28 others); Thu, 1 Mar 2018 08:01:23 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:39688 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031190AbeCANBU (ORCPT ); Thu, 1 Mar 2018 08:01:20 -0500 Received: by mail-pg0-f67.google.com with SMTP id w16so1378857pgv.6 for ; Thu, 01 Mar 2018 05:01:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=7qDP6b/nHj7r0IkKyyPP7XAky1eZBzSNc9JAXbaxcxI=; b=GKKHh5y+CPbNm2eY4dm+GroT3ChUCirOoRpib3RUW+ocDyyNEnrpV64afS4ASs8KUl 6GgOnvq6g6A8WQZ10QiEqmAf/WYnkGtj/YocNmh4WRhAEt3ijO/9+/I6rnBDkjztia/V t/BaGrZ8Idt3WpD/Dg9AYWADFnn7m6gxfvtqY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=7qDP6b/nHj7r0IkKyyPP7XAky1eZBzSNc9JAXbaxcxI=; b=cJJLglm4TjZX5vtojiZ1IdPBdptlUz2HY+S2Xdhm4Rf0BBikr3TEiGsS8Y7ePEm40f IXP8vAEt8APBGSCGTjxdt+lZzr2SCCRuC3jksVnBEFnjBQXoGkne1D75xAK6wK/bH3gF rrR00cf1JVXoP1ystyUhR3OrbpsRyGlfB5T7pQlifjxmyIRriH9o0YSbuiPpDV5WDcr4 P34GTMh4DYdq7FT9hk6WBPSQCqvh6xVdsGPUqSbAdewYJE2NHh9FhiKpRds/rjeqilPd /ek4TUYJYSJpxdBtgBdVPhsl52JpEpA5W9hFOeskZXN+2uAv0vXTn7VoXuetAdaM0EBL NR9Q== X-Gm-Message-State: APf1xPD2g/i5ZqTJeqBeBIfzN2RJa/X5UDmFDzB3lkQm2DsFY/XsEzSG pPF3MOUjT72TMGFkAOZmerOpxQ== X-Received: by 10.98.246.16 with SMTP id x16mr486890pfh.81.1519909279658; Thu, 01 Mar 2018 05:01:19 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.05.01.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 05:01:19 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Cc: Alex Shi Subject: [PATCH 45/45] arm64: Kill PSCI_GET_VERSION as a variant-2 workaround Date: Thu, 1 Mar 2018 20:54:22 +0800 Message-Id: <1519908862-11425-46-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 3a0a397ff5ff upstream. Now that we've standardised on SMCCC v1.1 to perform the branch prediction invalidation, let's drop the previous band-aid. If vendors haven't updated their firmware to do SMCCC 1.1, they haven't updated PSCI either, so we don't loose anything. Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kernel/bpi.S | 24 ------------------------ arch/arm64/kernel/cpu_errata.c | 41 +++++++++++------------------------------ arch/arm64/kvm/hyp/switch.c | 14 -------------- 3 files changed, 11 insertions(+), 68 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S index c72f261..dc4eb15 100644 --- a/arch/arm64/kernel/bpi.S +++ b/arch/arm64/kernel/bpi.S @@ -54,30 +54,6 @@ ENTRY(__bp_harden_hyp_vecs_start) vectors __kvm_hyp_vector .endr ENTRY(__bp_harden_hyp_vecs_end) -ENTRY(__psci_hyp_bp_inval_start) - sub sp, sp, #(8 * 18) - stp x16, x17, [sp, #(16 * 0)] - stp x14, x15, [sp, #(16 * 1)] - stp x12, x13, [sp, #(16 * 2)] - stp x10, x11, [sp, #(16 * 3)] - stp x8, x9, [sp, #(16 * 4)] - stp x6, x7, [sp, #(16 * 5)] - stp x4, x5, [sp, #(16 * 6)] - stp x2, x3, [sp, #(16 * 7)] - stp x0, x1, [sp, #(16 * 8)] - mov x0, #0x84000000 - smc #0 - ldp x16, x17, [sp, #(16 * 0)] - ldp x14, x15, [sp, #(16 * 1)] - ldp x12, x13, [sp, #(16 * 2)] - ldp x10, x11, [sp, #(16 * 3)] - ldp x8, x9, [sp, #(16 * 4)] - ldp x6, x7, [sp, #(16 * 5)] - ldp x4, x5, [sp, #(16 * 6)] - ldp x2, x3, [sp, #(16 * 7)] - ldp x0, x1, [sp, #(16 * 8)] - add sp, sp, #(8 * 18) -ENTRY(__psci_hyp_bp_inval_end) .macro smccc_workaround_1 inst sub sp, sp, #(8 * 4) diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 9632319..8b74f80 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -53,7 +53,6 @@ static int cpu_enable_trap_ctr_access(void *__unused) DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); #ifdef CONFIG_KVM -extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[]; extern char __smccc_workaround_1_smc_start[]; extern char __smccc_workaround_1_smc_end[]; extern char __smccc_workaround_1_hvc_start[]; @@ -100,8 +99,6 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn, spin_unlock(&bp_lock); } #else -#define __psci_hyp_bp_inval_start NULL -#define __psci_hyp_bp_inval_end NULL #define __smccc_workaround_1_smc_start NULL #define __smccc_workaround_1_smc_end NULL #define __smccc_workaround_1_hvc_start NULL @@ -146,24 +143,25 @@ static void call_hvc_arch_workaround_1(void) arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL); } -static bool check_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *entry) +static int enable_smccc_arch_workaround_1(void *data) { + const struct arm64_cpu_capabilities *entry = data; bp_hardening_cb_t cb; void *smccc_start, *smccc_end; struct arm_smccc_res res; if (!entry->matches(entry, SCOPE_LOCAL_CPU)) - return false; + return 0; if (psci_ops.smccc_version == SMCCC_VERSION_1_0) - return false; + return 0; switch (psci_ops.conduit) { case PSCI_CONDUIT_HVC: arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, ARM_SMCCC_ARCH_WORKAROUND_1, &res); if (res.a0) - return false; + return 0; cb = call_hvc_arch_workaround_1; smccc_start = __smccc_workaround_1_hvc_start; smccc_end = __smccc_workaround_1_hvc_end; @@ -173,35 +171,18 @@ static bool check_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *e arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, ARM_SMCCC_ARCH_WORKAROUND_1, &res); if (res.a0) - return false; + return 0; cb = call_smc_arch_workaround_1; smccc_start = __smccc_workaround_1_smc_start; smccc_end = __smccc_workaround_1_smc_end; break; default: - return false; + return 0; } install_bp_hardening_cb(entry, cb, smccc_start, smccc_end); - return true; -} - -static int enable_psci_bp_hardening(void *data) -{ - const struct arm64_cpu_capabilities *entry = data; - - if (psci_ops.get_version) { - if (check_smccc_arch_workaround_1(entry)) - return 0; - - install_bp_hardening_cb(entry, - (bp_hardening_cb_t)psci_ops.get_version, - __psci_hyp_bp_inval_start, - __psci_hyp_bp_inval_end); - } - return 0; } #endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */ @@ -301,22 +282,22 @@ const struct arm64_cpu_capabilities arm64_errata[] = { { .capability = ARM64_HARDEN_BRANCH_PREDICTOR, MIDR_ALL_VERSIONS(MIDR_CORTEX_A57), - .enable = enable_psci_bp_hardening, + .enable = enable_smccc_arch_workaround_1, }, { .capability = ARM64_HARDEN_BRANCH_PREDICTOR, MIDR_ALL_VERSIONS(MIDR_CORTEX_A72), - .enable = enable_psci_bp_hardening, + .enable = enable_smccc_arch_workaround_1, }, { .capability = ARM64_HARDEN_BRANCH_PREDICTOR, MIDR_ALL_VERSIONS(MIDR_CORTEX_A73), - .enable = enable_psci_bp_hardening, + .enable = enable_smccc_arch_workaround_1, }, { .capability = ARM64_HARDEN_BRANCH_PREDICTOR, MIDR_ALL_VERSIONS(MIDR_CORTEX_A75), - .enable = enable_psci_bp_hardening, + .enable = enable_smccc_arch_workaround_1, }, #endif { diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 996328e..154b471 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -311,20 +311,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu) if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu)) goto again; - if (exit_code == ARM_EXCEPTION_TRAP && - (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC64 || - kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32)) { - u32 val = vcpu_get_reg(vcpu, 0); - - if (val == PSCI_0_2_FN_PSCI_VERSION) { - val = kvm_psci_version(vcpu, kern_hyp_va(vcpu->kvm)); - if (unlikely(val == KVM_ARM_PSCI_0_1)) - val = PSCI_RET_NOT_SUPPORTED; - vcpu_set_reg(vcpu, 0, val); - goto again; - } - } - if (static_branch_unlikely(&vgic_v2_cpuif_trap) && exit_code == ARM_EXCEPTION_TRAP) { bool valid;