From patchwork Fri Sep 18 15:40:00 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 53920 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f69.google.com (mail-la0-f69.google.com [209.85.215.69]) by patches.linaro.org (Postfix) with ESMTPS id 9EFD0218E3 for ; Fri, 18 Sep 2015 15:43:28 +0000 (UTC) Received: by lamf6 with SMTP id f6sf21181060lam.1 for ; Fri, 18 Sep 2015 08:43:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id :in-reply-to:references:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:cc:mime-version :content-type:content-transfer-encoding:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list; bh=B/f4nR+Uy/GCwvu2L2XJQJ5VrAdmWCg3EasMM3Gooy0=; b=OtsmCACCou4ho7h0ppBUayfaRwbNb5hhwHLS5FHtLGbj3ntASLytxC+uinHFOP/8OO 346BgPEaq96+EQag/wwOQ0uHaadcM2f56TPdoyuDM6YqioWPJdIETRiZRhdomf1EDBd0 kRvIfsvMB2+jHSfLYdo7w8hL4LngWG6tTShkqj8IMSxdZR7JyJYjnK3R0COXHhO7jvbt pn1irkMmyiSntNxsxGKy9Yjh2eKZXueyhNzFjdhchJwLjcnZk4t5Pnku9gz2ELcMuEzX U0U3uh2jStd6GmsN2pM5E0BKg9vwkSGohhdvtMtiSCZXmDktw3C9vVU/y8goJ7ONvxXX e3uw== X-Gm-Message-State: ALoCoQkx9mJB3DdgN8WdOE7r48exT7Io8H9UXoRJ0+TsiCPOi7ylZJ941qKfAOkX9DuambSKA/7a X-Received: by 10.112.158.202 with SMTP id ww10mr1041926lbb.13.1442591007486; Fri, 18 Sep 2015 08:43:27 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.37.138 with SMTP id y10ls243593laj.35.gmail; Fri, 18 Sep 2015 08:43:27 -0700 (PDT) X-Received: by 10.152.7.37 with SMTP id g5mr3483783laa.22.1442591007233; Fri, 18 Sep 2015 08:43:27 -0700 (PDT) Received: from mail-lb0-f174.google.com (mail-lb0-f174.google.com. [209.85.217.174]) by mx.google.com with ESMTPS id kq1si6554678lac.46.2015.09.18.08.43.27 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Sep 2015 08:43:27 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.174 as permitted sender) client-ip=209.85.217.174; Received: by lbbmp1 with SMTP id mp1so27203320lbb.1 for ; Fri, 18 Sep 2015 08:43:27 -0700 (PDT) X-Received: by 10.152.161.232 with SMTP id xv8mr3586601lab.32.1442591007112; Fri, 18 Sep 2015 08:43:27 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.59.35 with SMTP id w3csp375726lbq; Fri, 18 Sep 2015 08:43:26 -0700 (PDT) X-Received: by 10.50.79.129 with SMTP id j1mr32868906igx.7.1442591005912; Fri, 18 Sep 2015 08:43:25 -0700 (PDT) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id rt7si7296944igb.91.2015.09.18.08.43.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Sep 2015 08:43:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1Zcxma-0002IB-Vo; Fri, 18 Sep 2015 15:41:05 +0000 Received: from mail-wi0-f178.google.com ([209.85.212.178]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1Zcxm8-0001f1-Gz for linux-arm-kernel@lists.infradead.org; Fri, 18 Sep 2015 15:40:39 +0000 Received: by wicge5 with SMTP id ge5so38140180wic.0 for ; Fri, 18 Sep 2015 08:40:14 -0700 (PDT) X-Received: by 10.180.20.15 with SMTP id j15mr17363595wie.76.1442590810699; Fri, 18 Sep 2015 08:40:10 -0700 (PDT) Received: from localhost.localdomain ([5.90.15.235]) by smtp.gmail.com with ESMTPSA id lu5sm9478601wjb.9.2015.09.18.08.40.08 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 18 Sep 2015 08:40:10 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, will.deacon@arm.com, catalin.marinas@arm.com Subject: [PATCH v1 2/2] arm64: errata: add module load workaround for erratum #843419 Date: Fri, 18 Sep 2015 17:40:00 +0200 Message-Id: <1442590800-14645-3-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1442590800-14645-1-git-send-email-ard.biesheuvel@linaro.org> References: <1442590800-14645-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150918_084037_048853_13DD6F47 X-CRM114-Status: GOOD ( 23.36 ) X-Spam-Score: -2.6 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.6 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [209.85.212.178 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.212.178 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , Cc: andre.przywara@arm.com, Ard Biesheuvel , suzuki.poulose@arm.com MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.174 as permitted sender) smtp.mailfrom=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 In order to work around Cortex-A53 erratum #843419, this patch updates the module loading logic to either change potentially problematic adrp instructions into adr instructions (if the symbol turns out to be in range), or emit a veneer that is guaranteed to be at an offset that does not trigger the issue. Signed-off-by: Ard Biesheuvel --- arch/arm64/Kconfig | 21 ++++++++++++ arch/arm64/include/asm/mod_veneers.h | 20 ++++++++++++ arch/arm64/kernel/mod_veneers.c | 33 +++++++++++++++++++ arch/arm64/kernel/module.c | 34 ++++++++++++++++++++ 4 files changed, 108 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index b591aac06fed..0f9e1d1235cb 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -337,6 +337,27 @@ endmenu config ARM64_MOD_VENEERS bool +config ARM64_ERRATUM_843419 + bool "Cortex-A53: 843419: A load or store might access an incorrect address" + depends on MODULES + select ARM64_MOD_VENEERS + default y + help + This option applies special relocation logic when loading kernel + modules to prevent ADRP instructions from appearing in either of the + last two instruction slots in a 4 KB page. This can cause a subsequent + memory access to use an incorrect address on Cortex-A53 parts up to + r0p4. If such an instruction is subject to relocation, we either + change the ADRP instruction into an ADR instruction (if the symbol it + refers to is within 1 MB), or else we replace it by a call to a veneer + which executes the same instruction at a safe offset before branching + back past the original location. + + Note that the kernel itself must be linked with a version of ld which + fixes potentially affected ADRP instructions at build time. + + If unsure, say Y. + choice prompt "Page size" default ARM64_4K_PAGES diff --git a/arch/arm64/include/asm/mod_veneers.h b/arch/arm64/include/asm/mod_veneers.h new file mode 100644 index 000000000000..3cf36ce91549 --- /dev/null +++ b/arch/arm64/include/asm/mod_veneers.h @@ -0,0 +1,20 @@ + +#include + +struct veneer_erratum_843419 { + u32 adrp; + u32 branch; +}; + +static inline bool erratum_843419_affects_adrp_insn(void *addr) +{ + /* + * The workaround for erratum 843419 only needs to be + * applied if the adrp instruction appears in either of + * the last two instruction slots in the 4 KB page. + */ + return ((u64)addr % SZ_4K) >= (SZ_4K - 8); +} + +void *emit_erratum_843419_veneer(struct module *mod, u32 *insn, + Elf64_Shdr *sechdrs); diff --git a/arch/arm64/kernel/mod_veneers.c b/arch/arm64/kernel/mod_veneers.c index 9649c4f305bd..7da300e59749 100644 --- a/arch/arm64/kernel/mod_veneers.c +++ b/arch/arm64/kernel/mod_veneers.c @@ -10,6 +10,8 @@ #include #include +#include + static bool in_init(const struct module *mod, u64 addr) { return addr - (u64)mod->module_init < mod->init_size; @@ -32,6 +34,31 @@ static void *alloc_veneer(struct module *mod, u64 loc, int size, return ret; } +#ifdef CONFIG_ARM64_ERRATUM_843419 +void *emit_erratum_843419_veneer(struct module *mod, u32 *insn, + Elf64_Shdr *sechdrs) +{ + struct veneer_erratum_843419 *veneer; + + veneer = alloc_veneer(mod, (u64)insn, 2 * sizeof(*veneer), sechdrs); + if (erratum_843419_affects_adrp_insn(&veneer->adrp)) + /* + * We allocated a veneer that is susceptible to the same problem + * as the original location. We allocated twice the space, so + * just advance to the next slot. + */ + veneer++; + + veneer->adrp = *insn; + veneer->branch = aarch64_insn_gen_branch_imm( + (unsigned long)&veneer->branch, + (unsigned long)(insn + 1), + AARCH64_INSN_BRANCH_NOLINK); + + return veneer; +} +#endif + /* estimate the maximum size of the veneer for this relocation */ static unsigned long get_veneers_size(Elf64_Addr base, const Elf64_Rela *rel, int num) @@ -41,6 +68,12 @@ static unsigned long get_veneers_size(Elf64_Addr base, const Elf64_Rela *rel, for (i = 0; i < num; i++) switch (ELF64_R_TYPE(rel[i].r_info)) { + case R_AARCH64_ADR_PREL_PG_HI21_NC: + case R_AARCH64_ADR_PREL_PG_HI21: +#ifdef CONFIG_ARM64_ERRATUM_843419 + ret += 2 * sizeof(struct veneer_erratum_843419); +#endif + break; } return ret; } diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c index 67bf4107f6ef..5d46f63c0111 100644 --- a/arch/arm64/kernel/module.c +++ b/arch/arm64/kernel/module.c @@ -27,6 +27,7 @@ #include #include #include +#include #include #define AARCH64_INSN_IMM_MOVNZ AARCH64_INSN_IMM_MAX @@ -335,6 +336,39 @@ int apply_relocate_add(Elf64_Shdr *sechdrs, case R_AARCH64_ADR_PREL_PG_HI21_NC: overflow_check = false; case R_AARCH64_ADR_PREL_PG_HI21: +#ifdef CONFIG_ARM64_ERRATUM_843419 + /* + * TODO check for presence of affected A53 cores + */ + if (erratum_843419_affects_adrp_insn(loc)) { + struct veneer_erratum_843419 *v; + + /* + * This adrp instruction appears at an offset + * that may be problematic on older Cortex-A53 + * cores. So first, try to convert it into a + * simple adr instruction. + */ + ovf = reloc_insn_imm(RELOC_OP_PREL, loc, + val & ~(SZ_4K - 1), 0, 21, + AARCH64_INSN_IMM_ADR); + if (ovf == 0) { + /* success! convert adrp -> adr */ + *(u32 *)loc &= 0x7fffffff; + break; + } + + /* symbol out of range for adr -> emit veneer */ + v = emit_erratum_843419_veneer(me, loc, + sechdrs); + *(u32 *)loc = aarch64_insn_gen_branch_imm( + (unsigned long)loc, + (unsigned long)v, + AARCH64_INSN_BRANCH_NOLINK); + loc = &v->adrp; + /* fall through */ + } +#endif ovf = reloc_insn_imm(RELOC_OP_PAGE, loc, val, 12, 21, AARCH64_INSN_IMM_ADR); break;