From patchwork Thu Jan 25 10:31:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 125782 Delivered-To: patch@linaro.org Received: by 10.46.66.141 with SMTP id h13csp1016518ljf; Thu, 25 Jan 2018 02:31:48 -0800 (PST) X-Google-Smtp-Source: AH8x226W5KaiTWjyeBtJnAsGAYjt3mnzFFDDFUPSxwgp52BQdTmqC2V+eOuwaQ9uW+E69XzpJM1M X-Received: by 2002:a17:902:2f41:: with SMTP id s59-v6mr11202271plb.422.1516876308693; Thu, 25 Jan 2018 02:31:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516876308; cv=none; d=google.com; s=arc-20160816; b=wvbfaSCxgcV5F0jVNae/ar+zsxxIkGCtz58HcJ097b7jIaB7jR6CRHNumtf+K3hPKe cEwqvhRoSbIpGM8Pcf4x2vFK4lH2L5Q0ZPGFc5qQpmN1VR1oLtiVTFoxhEosR8Nr2mcz jB3CNXcJdMMqBFf6nVA/Ftxax9oK2xCoozgQh/DYgq9kRtk9LXGNaYXzbeBgLH5R0Bzg M/zZ9W52n2G7F9qjFL7Dv6jIGrV1ifuLEPVuzgnfQEhFZb11mdaryy2tSGYHiIq/MY4M 09rbaXRNPj0znuLkMZ36OdY4c+1ACBWV4d3uvebo6WdVAxDrk0/GsP9VTu/PqX4ipNMS RdcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=BUq8BRuqV8cRYIGUCbSrFQcKCeTVc32Q5AKUjSSyrZ4=; b=GMukMdDR7d72VL7/D87l33a1FJZExudi4Rn02h6OwF/eu2AbA9TVET53Ht/b0PdY/d EHvpKUOx9+k4qoe3L2LDtq3FCej/YC5aL2w1kXFzVVefnoLp5Qm0+JL+1JyPLX9q9egk 5B1RasQwpjiDnekh/aZDOw1Fs4791JB8Gp9fk8VnYgUEI2abCgFhFMj21mZcsB4wMRct pqtI+KFXWYgfEIFk86b5oXp+1moS/VTI+9XoxFzflDFPkaygva4LfzFnitQ/ud49mqJl W8Zfs05R+eqRCPcWvyLuBFD4lYNlj+WKdq4pF02tUt4s3WeN7W7LEaRELMSX51SfPEcT hXxQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=ZvRTSsxk; spf=pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-efi-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n188si1088646pga.175.2018.01.25.02.31.48; Thu, 25 Jan 2018 02:31:48 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=ZvRTSsxk; spf=pass (google.com: best guess record for domain of linux-efi-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-efi-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751173AbeAYKbs (ORCPT + 2 others); Thu, 25 Jan 2018 05:31:48 -0500 Received: from mail-wr0-f194.google.com ([209.85.128.194]:46158 "EHLO mail-wr0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751154AbeAYKbr (ORCPT ); Thu, 25 Jan 2018 05:31:47 -0500 Received: by mail-wr0-f194.google.com with SMTP id g21so7075468wrb.13 for ; Thu, 25 Jan 2018 02:31:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oxpKvnWnu3AzKIA2gQBJX2ShOJ2rqfrO9V0GrV28NxA=; b=ZvRTSsxkPG+Q60tHUITQujanRucBkC9Un2d6alaSTAKgKCnsA7VQDZuAI7Q56e6dzN dcH4GHTRk5dOnhjXbYKp8BZhkHTOIYHpfCGYoeUAysgQV9AfUqsJjWQYcc39QMGoCfen QtUGwZHQOKvEsmKV3SRP3R2RextP3xhVudabs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oxpKvnWnu3AzKIA2gQBJX2ShOJ2rqfrO9V0GrV28NxA=; b=PiC68XKhM0NB0svAfbt464chBJv2f/B3bqex+6WOjA4Iwhi1lVPokVfCD2XUSmHVr3 dZZaFdejrBT+Gqry7gKxd9RseV/27YAZ+ZVRKaSS4VZafIIgX2cg0zeGo9GLMVViQLPg XM3ng1ljR74qhOYogHfPEs5oT9mHT9k2fnlpnOwHpVK68gdYvTNECM0hoe0cyzFNq6Ih t16+lLUdAWzYZi+JM1+tVWF6w/IHhNKeiBBr1WJhAny27fdCjki1gbABcu/jtUhq/X6+ fpeDJ3DlDSaIqPlU2lKf6zQxdxcP0bw8hzfPBvN0peF9IDaXJQeXd1OR+nBudK0+fQIE rGkA== X-Gm-Message-State: AKwxytfXSjua3tZZK44KONkjw/3xyZiTopfn+MtyORCd7J9OCbKzRqXi qAvSqJqklqSPQ8Ltl7jlbXQ+8ewR9YE= X-Received: by 10.223.135.187 with SMTP id b56mr8429409wrb.164.1516876305409; Thu, 25 Jan 2018 02:31:45 -0800 (PST) Received: from localhost.localdomain ([160.167.127.168]) by smtp.gmail.com with ESMTPSA id j77sm1199964wmf.37.2018.01.25.02.31.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Jan 2018 02:31:44 -0800 (PST) From: Ard Biesheuvel To: linux-efi@vger.kernel.org, linux-arm-kernel@lists.infradead.org, will.deacon@arm.com, catalin.marinas@arm.com, mark.rutland@arm.com, marc.zyngier@arm.com Cc: joakim.bech@linaro.org, leif.lindholm@linaro.org, graeme.gregory@linaro.org, Ard Biesheuvel Subject: [PATCH 2/4] efi/arm64: map the stack and entry wrapper into the UEFI page tables Date: Thu, 25 Jan 2018 10:31:29 +0000 Message-Id: <20180125103131.19168-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180125103131.19168-1-ard.biesheuvel@linaro.org> References: <20180125103131.19168-1-ard.biesheuvel@linaro.org> Sender: linux-efi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-efi@vger.kernel.org As a preparatory step towards unmapping the kernel entirely while executing UEFI runtime services, move the stack and the entry wrapper routine mappings into the EFI page tables. Also, create a vector table that overrides the main one while executing in the firmware so we will be able to remap/unmap the kernel while taking interrupts. Signed-off-by: Ard Biesheuvel --- arch/arm/include/asm/efi.h | 5 ++ arch/arm64/include/asm/efi.h | 23 ++++++++- arch/arm64/include/asm/stacktrace.h | 4 ++ arch/arm64/kernel/efi-rt-wrapper.S | 51 +++++++++++++++++++- arch/arm64/kernel/efi.c | 24 +++++++++ arch/arm64/kernel/entry.S | 1 + drivers/firmware/efi/arm-runtime.c | 2 + 7 files changed, 107 insertions(+), 3 deletions(-) -- 2.11.0 -- To unsubscribe from this list: send the line "unsubscribe linux-efi" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/arch/arm/include/asm/efi.h b/arch/arm/include/asm/efi.h index 17f1f1a814ff..3a63e7cc1dfa 100644 --- a/arch/arm/include/asm/efi.h +++ b/arch/arm/include/asm/efi.h @@ -99,4 +99,9 @@ static inline unsigned long efi_get_max_initrd_addr(unsigned long dram_base, return dram_base + SZ_512M; } +static inline int efi_allocate_runtime_regions(struct mm_struct *mm) +{ + return 0; +} + #endif /* _ASM_ARM_EFI_H */ diff --git a/arch/arm64/include/asm/efi.h b/arch/arm64/include/asm/efi.h index 192d791f1103..b9b09a734719 100644 --- a/arch/arm64/include/asm/efi.h +++ b/arch/arm64/include/asm/efi.h @@ -2,11 +2,13 @@ #ifndef _ASM_EFI_H #define _ASM_EFI_H +#include + +#ifndef __ASSEMBLY__ #include #include #include #include -#include #include #include #include @@ -30,8 +32,9 @@ int efi_set_mapping_permissions(struct mm_struct *mm, efi_memory_desc_t *md); #define arch_efi_call_virt(p, f, args...) \ ({ \ efi_##f##_t *__f; \ + typeof(__efi_rt_asm_wrapper) *__wrap = (void *)EFI_CODE_BASE; \ __f = p->f; \ - __efi_rt_asm_wrapper(__f, #f, args); \ + __wrap(__f, #f, args); \ }) #define arch_efi_call_virt_teardown() \ @@ -146,4 +149,20 @@ static inline void efi_set_pgd(struct mm_struct *mm) void efi_virtmap_load(void); void efi_virtmap_unload(void); +int __init efi_allocate_runtime_regions(struct mm_struct *mm); + +#endif /* __ASSEMBLY__ */ + +/* + * When running with vmap'ed stacks, we need the base of the stack to be aligned + * appropriately, where the exact alignment depends on the page size. Let's just + * put the stack at address 0x0, which is guaranteed to be free and aligned. + */ +#define EFI_STACK_BASE 0x0 +#define EFI_STACK_SIZE THREAD_SIZE + +/* where to map the pivot code in the UEFI page tables */ +#define EFI_CODE_BASE 0x200000 +#define EFI_CODE_SIZE PAGE_SIZE + #endif /* _ASM_EFI_H */ diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 472ef944e932..b1212b3b3df5 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -72,6 +72,8 @@ static inline bool on_overflow_stack(unsigned long sp) static inline bool on_overflow_stack(unsigned long sp) { return false; } #endif +bool on_efi_stack(unsigned long sp); + /* * We can only safely access per-cpu stacks from current in a non-preemptible * context. @@ -88,6 +90,8 @@ static inline bool on_accessible_stack(struct task_struct *tsk, unsigned long sp return true; if (on_sdei_stack(sp)) return true; + if (on_efi_stack(sp)) + return true; return false; } diff --git a/arch/arm64/kernel/efi-rt-wrapper.S b/arch/arm64/kernel/efi-rt-wrapper.S index 05235ebb336d..09e77e5edd94 100644 --- a/arch/arm64/kernel/efi-rt-wrapper.S +++ b/arch/arm64/kernel/efi-rt-wrapper.S @@ -7,7 +7,10 @@ */ #include +#include + .section ".rodata", "a" + .align PAGE_SHIFT ENTRY(__efi_rt_asm_wrapper) stp x29, x30, [sp, #-32]! mov x29, sp @@ -19,6 +22,12 @@ ENTRY(__efi_rt_asm_wrapper) */ stp x1, x18, [sp, #16] + /* switch to the EFI runtime stack and vector table */ + mov sp, #EFI_STACK_BASE + EFI_STACK_SIZE + adr x1, __efi_rt_vectors + msr vbar_el1, x1 + isb + /* * We are lucky enough that no EFI runtime services take more than * 5 arguments, so all are passed in registers rather than via the @@ -32,10 +41,50 @@ ENTRY(__efi_rt_asm_wrapper) mov x4, x6 blr x8 + /* switch back to the task stack and primary vector table */ + mov sp, x29 + ldr x1, 2f + msr vbar_el1, x1 + isb + ldp x1, x2, [sp, #16] cmp x2, x18 ldp x29, x30, [sp], #32 b.ne 0f ret -0: b efi_handle_corrupted_x18 // tail call +0: ldr x8, 1f + br x8 // tail call ENDPROC(__efi_rt_asm_wrapper) + .align 3 +1: .quad efi_handle_corrupted_x18 +2: .quad vectors + + .macro ventry + .align 7 +.Lv\@ : stp x29, x30, [sp, #-16]! // preserve x29 and x30 + mrs x29, elr_el1 // preserve ELR + adr x30, .Lret // take return address + msr elr_el1, x30 // set ELR to return address + ldr x30, 2b // take address of 'vectors' + msr vbar_el1, x30 // set VBAR to 'vectors' + isb + add x30, x30, #.Lv\@ - __efi_rt_vectors + br x30 + .endm + +.Lret: msr elr_el1, x29 + adr x30, __efi_rt_vectors + msr vbar_el1, x30 + isb + ldp x29, x30, [sp], #16 + eret + + .align 11 +__efi_rt_vectors: + .rept 8 + ventry + .endr + /* + * EFI runtime services never drop to EL0, so the + * remaining vector table entries are not needed. + */ diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c index af4f943cffac..68c920b2f4f0 100644 --- a/arch/arm64/kernel/efi.c +++ b/arch/arm64/kernel/efi.c @@ -130,3 +130,27 @@ asmlinkage efi_status_t efi_handle_corrupted_x18(efi_status_t s, const char *f) pr_err_ratelimited(FW_BUG "register x18 corrupted by EFI %s\n", f); return s; } + +bool on_efi_stack(unsigned long sp) +{ + return sp >= EFI_STACK_BASE && sp < (EFI_STACK_BASE + EFI_STACK_SIZE); +} + +int __init efi_allocate_runtime_regions(struct mm_struct *mm) +{ + static u8 stack[EFI_STACK_SIZE] __page_aligned_bss; + + /* map the stack */ + create_pgd_mapping(mm, __pa_symbol(stack), + EFI_STACK_BASE, EFI_STACK_SIZE, + __pgprot(pgprot_val(PAGE_KERNEL) | PTE_NG), + false); + + /* map the runtime wrapper pivot function */ + create_pgd_mapping(mm, __pa_symbol(__efi_rt_asm_wrapper), + EFI_CODE_BASE, EFI_CODE_SIZE, + __pgprot(pgprot_val(PAGE_KERNEL_ROX) | PTE_NG), + false); + + return 0; +} diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index b34e717d7597..3bab6c60a12b 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -204,6 +204,7 @@ alternative_if ARM64_HAS_PAN alternative_else_nop_endif .if \el != 0 + tbz x21, #63, 1f // skip if TTBR0 covers the stack mrs x21, ttbr0_el1 tst x21, #TTBR_ASID_MASK // Check for the reserved ASID orr x23, x23, #PSR_PAN_BIT // Set the emulated PAN in the saved SPSR diff --git a/drivers/firmware/efi/arm-runtime.c b/drivers/firmware/efi/arm-runtime.c index 1cc41c3d6315..e84f4d961de2 100644 --- a/drivers/firmware/efi/arm-runtime.c +++ b/drivers/firmware/efi/arm-runtime.c @@ -107,6 +107,8 @@ static bool __init efi_virtmap_init(void) if (efi_memattr_apply_permissions(&efi_mm, efi_set_mapping_permissions)) return false; + efi_allocate_runtime_regions(&efi_mm); + return true; }