From patchwork Tue Aug 15 12:50:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 110150 Delivered-To: patch@linaro.org Received: by 10.182.109.195 with SMTP id hu3csp5439608obb; Tue, 15 Aug 2017 05:53:07 -0700 (PDT) X-Received: by 10.98.160.146 with SMTP id p18mr28716307pfl.322.1502801587872; Tue, 15 Aug 2017 05:53:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502801587; cv=none; d=google.com; s=arc-20160816; b=Z2caEYv1k14GHSe3xdfI6FHUBMiUPmXan8A1sNmX/X2FjdBItEpBXTPvlDWr5ssayd d3gstmY4+2I46qD/n+joWKU30Yjpkf1NSZ4Shvx7ZYt2Kn83HSdLKyDTM9o0BRV3DI+h PLvHe0dA0yYGEUd8wAmxplgASUOFhr9fE6rnKS/8FNBuk09vTS9HPfP4VCPMosrPl6o5 L3pibQkBMZw1cIdVhjsq/VTtZncWld1Ji3SffE1Vy52WzJpg4ZhwTADAI1GEyjvF5fvD 1pwf5PLcEXz5yMaqS7R7XJDneZs2sZINJ7GNzdNzwVA8arlksTMsnUUGTuJBja2fYcMY D8Pw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=z7xMWS7KOL3/NKi2FkdpMmBh0y7x0vffhggJ6JY98Sw=; b=a6ak+WxzPkx5TkiCMAwk2RMwlhUxNSixQDJtfiMq5r3LCegGIkYyGw/oJBRCuH9fuH xzcxljY+kM5t1sbrTvNNZjb4wIg8qoNZO6FDsKyQq6+JjM8aJ18h+lesF+AYTpxdIAGP HAUiLBh7vwjxGbcLQwsknPGtk3BfD6rhvkO5CB6w/ZdXPUIn82KL3kEsrM7WcuG/P9VW dzlgBIA0yKwyH7h5FBvd1DbbT1tB2vOHdk1J+Yz4kwpByeAdeb7DdIvuLlpj9MrvtOX/ dD87eEUqTBFycP0XL6cPBe2D+gpkFgmiTMa+LF+aWCI5r+037r2daBE4frFAlu4EN7LM d4Hg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g6si6158954plm.924.2017.08.15.05.53.07; Tue, 15 Aug 2017 05:53:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752851AbdHOMxE (ORCPT + 25 others); Tue, 15 Aug 2017 08:53:04 -0400 Received: from foss.arm.com ([217.140.101.70]:51592 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752756AbdHOMxC (ORCPT ); Tue, 15 Aug 2017 08:53:02 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 8F7A916A3; Tue, 15 Aug 2017 05:53:02 -0700 (PDT) Received: from leverpostej.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7D4653F540; Tue, 15 Aug 2017 05:53:00 -0700 (PDT) From: Mark Rutland To: linux-arm-kernel@lists.infradead.org Cc: ard.biesheuvel@linaro.org, catalin.marinas@arm.com, james.morse@arm.com, labbott@redhat.com, linux-kernel@vger.kernel.org, luto@amacapital.net, mark.rutland@arm.com, matt@codeblueprint.co.uk, will.deacon@arm.com, kernel-hardening@lists.openwall.com, keescook@chromium.org Subject: [PATCHv2 11/14] arm64: use an irq stack pointer Date: Tue, 15 Aug 2017 13:50:46 +0100 Message-Id: <1502801449-29246-12-git-send-email-mark.rutland@arm.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1502801449-29246-1-git-send-email-mark.rutland@arm.com> References: <1502801449-29246-1-git-send-email-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We allocate our IRQ stacks using a percpu array. This allows us to generate our IRQ stack pointers with adr_this_cpu, but bloats the kernel Image with the boot CPU's IRQ stack. Additionally, these are packed with other percpu variables, and aren't guaranteed to have guard pages. When we enable VMAP_STACK we'll want to vmap our IRQ stacks also, in order to provide guard pages and to permit more stringent alignment requirements. Doing so will require that we use a percpu pointer to each IRQ stack, rather than allocating a percpu IRQ stack in the kernel image. This patch updates our IRQ stack code to use a percpu pointer to the base of each IRQ stack. This will allow us to change the way the stack is allocated with minimal changes elsewhere. In some cases we may try to backtrace before the IRQ stack pointers are initialised, so on_irq_stack() is updated to account for this. In testing with cyclictest, there was no measureable difference between using adr_this_cpu (for irq_stack) and ldr_this_cpu (for irq_stack_ptr) in the IRQ entry path. Signed-off-by: Mark Rutland Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: James Morse Cc: Laura Abbott Cc: Will Deacon --- arch/arm64/include/asm/stacktrace.h | 7 +++++-- arch/arm64/kernel/entry.S | 2 +- arch/arm64/kernel/irq.c | 10 ++++++++++ 3 files changed, 16 insertions(+), 3 deletions(-) -- 1.9.1 diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 000e2418..4c68d8a 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -36,13 +36,16 @@ extern void walk_stackframe(struct task_struct *tsk, struct stackframe *frame, int (*fn)(struct stackframe *, void *), void *data); extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk); -DECLARE_PER_CPU(unsigned long [IRQ_STACK_SIZE/sizeof(long)], irq_stack); +DECLARE_PER_CPU(unsigned long *, irq_stack_ptr); static inline bool on_irq_stack(unsigned long sp) { - unsigned long low = (unsigned long)raw_cpu_ptr(irq_stack); + unsigned long low = (unsigned long)raw_cpu_read(irq_stack_ptr); unsigned long high = low + IRQ_STACK_SIZE; + if (!low) + return false; + return (low <= sp && sp < high); } diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 58eba94..5234886 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -276,7 +276,7 @@ alternative_else_nop_endif and x25, x25, #~(THREAD_SIZE - 1) cbnz x25, 9998f - adr_this_cpu x25, irq_stack, x26 + ldr_this_cpu x25, irq_stack_ptr, x26 mov x26, #IRQ_STACK_SIZE add x26, x25, x26 diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c index 2386b26..5141282 100644 --- a/arch/arm64/kernel/irq.c +++ b/arch/arm64/kernel/irq.c @@ -32,6 +32,7 @@ /* irq stack only needs to be 16 byte aligned - not IRQ_STACK_SIZE aligned. */ DEFINE_PER_CPU(unsigned long [IRQ_STACK_SIZE/sizeof(long)], irq_stack) __aligned(16); +DEFINE_PER_CPU(unsigned long *, irq_stack_ptr); int arch_show_interrupts(struct seq_file *p, int prec) { @@ -50,8 +51,17 @@ void __init set_handle_irq(void (*handle_irq)(struct pt_regs *)) handle_arch_irq = handle_irq; } +static void init_irq_stacks(void) +{ + int cpu; + + for_each_possible_cpu(cpu) + per_cpu(irq_stack_ptr, cpu) = per_cpu(irq_stack, cpu); +} + void __init init_IRQ(void) { + init_irq_stacks(); irqchip_init(); if (!handle_arch_irq) panic("No interrupt controller found.");