From patchwork Sun Aug 6 12:46:24 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 109507 Delivered-To: patch@linaro.org Received: by 10.140.95.78 with SMTP id h72csp424841qge; Sun, 6 Aug 2017 05:47:32 -0700 (PDT) X-Received: by 10.84.212.144 with SMTP id e16mr10354069pli.115.1502023652347; Sun, 06 Aug 2017 05:47:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1502023652; cv=none; d=google.com; s=arc-20160816; b=JJOAmZwUZb2+BcdWGrZNZCCn8jLn2Aoavk6ljyHfj8UEIzYrnoiis7juUOv4/BFKwG ARe43PlE+D8M9t23nnYe/dR9r6aCDJ6SBLzC5V8QkhyCZdEvSI1kkJuaYgZlW/ErQtem C2BiWNjrDpOWGWVvwdILipN6v8oqFp+v26OnFQ67eSyqaPioBrv9+whw7cAf87uHkil5 m6AMytOSVbZWcG4om2xwx2SHMBinU8IcG25RC68vp1QGT0H4kHlvRqXJ714nHM7eBEtN ucXlYPXg+JkOIxIViIhfnwFxhk6p5wdFU4TWBOeak70HDNc7MXmvQJwbDPdXnGaTfOTw BitA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=/8BkdH3JHzzEwFJ0JBuYSd0PyjpiLFG8xT1AO1YEgcU=; b=GarKiMRqULVSb3+dCKwl1sB2NS0z1Q5C9oFklh8/ZTeX8gUey8Sj20YtmkqBp/UyR/ Z4gfPzzeBJUd2qj9HcydQHNhlTt16dS6GPyAIksyoxWq9cOdz4Ptay4MqlMuXAJaLKXF u+t2rEsy1PuGh4tUByPSZA+337T34H+Jhfsmq/fHT0bkNnQEwP7T8kHgdSGi3rUVQ0IB WUYG6z7hmlXoYYbkfsS+HXJBorAz8pzP+uQC12WfhNTZLsogUbdRQjS3z9TiCXFDjWGV SSc/PoQSWE9jm90PzCVq8+dMZ0RX25Rsl0GmCV6WiZWZVpYHtqwGTi9rAplhmvSaRVA9 +LiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.b=cUM+c+0Q; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y1si3360607pgr.461.2017.08.06.05.47.32; Sun, 06 Aug 2017 05:47:32 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.b=cUM+c+0Q; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751363AbdHFMr3 (ORCPT + 25 others); Sun, 6 Aug 2017 08:47:29 -0400 Received: from mail-wr0-f174.google.com ([209.85.128.174]:37216 "EHLO mail-wr0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751277AbdHFMrZ (ORCPT ); Sun, 6 Aug 2017 08:47:25 -0400 Received: by mail-wr0-f174.google.com with SMTP id 33so30678577wrz.4 for ; Sun, 06 Aug 2017 05:47:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=/8BkdH3JHzzEwFJ0JBuYSd0PyjpiLFG8xT1AO1YEgcU=; b=cUM+c+0QlbiRGY93nMKiiP46c3SMkQlKUPNKIZ1yZZ5P78j6dOcIs2tC/tvIEw4g9t XHOO0BUOnwZmcbKxB/tXWbk61M5wpU7uuayL3MFcgmkMSgMdSc0XsYS3VBb7f1XNYzUX wnvUpdQ2NYZLqVNtkTjlEOnIF6eXUxMTUN1rg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=/8BkdH3JHzzEwFJ0JBuYSd0PyjpiLFG8xT1AO1YEgcU=; b=JBoYPf/8km5yrwTACpzI9i50bxE89Ff4D3VMCmJdRKhxH6MIpRGUWNTvAOBf8GmxrM oPiVl/M7o5BQwL1oDiwnTjkY4Y+EPmGgsOXvjgBjW/FU2UZ0bkv8TpxSzsaC+uYVqbup 4NAxg8kPXSb1BfM3TMqY6pk6wZ9ERcJVtrmKv5cgcpIPW5kwdvwl9DkJY9xFhzBAXZ2I k8FCrNI4Ea1w/mdKzgMzht0Am77sbdRyf6xEsGLjc1OMgVSb7eFHX8/Sz/Muss/3JiNY j0lsei1HbS4VJR0DOaA6cYD6RXSbm0dVv3ue6evaQChL8sAGUjQbO9dTe0zpQlaIkaOy cfTQ== X-Gm-Message-State: AIVw1122jvnypKYucNtpnvhOjWgeB/xBybUjgWP0wiDJedYtVsF9Nt6Q fzmULSpVuTCHfXz5 X-Received: by 10.223.152.172 with SMTP id w41mr6045072wrb.202.1502023644190; Sun, 06 Aug 2017 05:47:24 -0700 (PDT) Received: from localhost.localdomain ([105.142.229.83]) by smtp.gmail.com with ESMTPSA id r108sm8463951wrb.29.2017.08.06.05.47.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Aug 2017 05:47:22 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Jessica Yu , Arnd Bergmann , Russell King , Andrew Morton , Ingo Molnar , Kees Cook , Thomas Garnier , Nicolas Pitre Subject: [PATCH v2] module: use relative references for __ksymtab entries Date: Sun, 6 Aug 2017 13:46:24 +0100 Message-Id: <20170806124624.22915-1-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org An ordinary arm64 defconfig build has ~64 KB worth of __ksymtab entries, each consisting of two 64-bit fields containing absolute references, to the symbol itself and to a char array containing its name, respectively. When we build the same configuration with KASLR enabled, we end up with an additional ~192 KB of relocations in the .init section, i.e., one 24 byte entry for each absolute reference, which all need to be processed at boot time. Given how the struct kernel_symbol that describes each entry is completely local to module.c (except for the references emitted by EXPORT_SYMBOL() itself), we can easily modify it to contain two 32-bit relative references instead. This reduces the size of the __ksymtab section by 50% for all 64-bit architectures, and gets rid of the runtime relocations entirely for architectures implementing KASLR, either via standard PIE linking (arm64) or using custom host tools (x86). Since EXPORT_SYMBOL() no longer refers to struct kernel_symbol, move the definition into module.c, which is its only user. Note that the binary search involving __ksymtab contents relies on each section being sorted by symbol name. This is implemented based on the input section names, not the names in the ksymtab entries, so this patch does not interfere with that. Cc: Jessica Yu Cc: Arnd Bergmann Cc: Russell King Cc: Andrew Morton Cc: Ingo Molnar Cc: Kees Cook Cc: Thomas Garnier Cc: Nicolas Pitre Signed-off-by: Ard Biesheuvel --- v2: - fix error in modpost due to missing local __ksymtab_xxx symbol names Note that ARM requires a little tweak to ensure that the unwind info related to the discarded code is discarded as well. I'm sure there will be an arch or two where something similar may be required. arch/arm/kernel/vmlinux.lds.S | 1 + include/asm-generic/export.h | 10 +------ include/linux/export.h | 28 ++++++++++-------- kernel/module.c | 31 ++++++++++++++++---- 4 files changed, 43 insertions(+), 27 deletions(-) -- 2.11.0 diff --git a/arch/arm/kernel/vmlinux.lds.S b/arch/arm/kernel/vmlinux.lds.S index c83a7ba737d6..4bdba75e510c 100644 --- a/arch/arm/kernel/vmlinux.lds.S +++ b/arch/arm/kernel/vmlinux.lds.S @@ -89,6 +89,7 @@ SECTIONS #endif *(.discard) *(.discard.*) + *(.ARM.exidx.discard) } . = PAGE_OFFSET + TEXT_OFFSET; diff --git a/include/asm-generic/export.h b/include/asm-generic/export.h index 719db1968d81..fac5b2e6df37 100644 --- a/include/asm-generic/export.h +++ b/include/asm-generic/export.h @@ -4,17 +4,9 @@ #ifndef KSYM_FUNC #define KSYM_FUNC(x) x #endif -#ifdef CONFIG_64BIT -#define __put .quad #ifndef KSYM_ALIGN #define KSYM_ALIGN 8 #endif -#else -#define __put .long -#ifndef KSYM_ALIGN -#define KSYM_ALIGN 4 -#endif -#endif #ifndef KCRC_ALIGN #define KCRC_ALIGN 4 #endif @@ -35,7 +27,7 @@ .section ___ksymtab\sec+\name,"a" .balign KSYM_ALIGN KSYM(__ksymtab_\name): - __put \val, KSYM(__kstrtab_\name) + .long \val - ., KSYM(__kstrtab_\name) - . .previous .section __ksymtab_strings,"a" KSYM(__kstrtab_\name): diff --git a/include/linux/export.h b/include/linux/export.h index 1a1dfdb2a5c6..943656d74f0c 100644 --- a/include/linux/export.h +++ b/include/linux/export.h @@ -24,12 +24,6 @@ #define VMLINUX_SYMBOL_STR(x) __VMLINUX_SYMBOL_STR(x) #ifndef __ASSEMBLY__ -struct kernel_symbol -{ - unsigned long value; - const char *name; -}; - #ifdef MODULE extern struct module __this_module; #define THIS_MODULE (&__this_module) @@ -60,17 +54,27 @@ extern struct module __this_module; #define __CRC_SYMBOL(sym, sec) #endif -/* For every exported symbol, place a struct in the __ksymtab section */ +/* + * For every exported symbol, place a struct in the __ksymtab section. + * Note that we have to visibly take the address of sym, so the compiler + * is forced to emit it, rather than inlining it or removing it + * altogether. Do so in a way that avoids taking the address statically, + * and emit that code into a section that is discarded by the linker. + */ #define ___EXPORT_SYMBOL(sym, sec) \ extern typeof(sym) sym; \ __CRC_SYMBOL(sym, sec) \ static const char __kstrtab_##sym[] \ - __attribute__((section("__ksymtab_strings"), aligned(1))) \ + __attribute__((section("__ksymtab_strings"), used, aligned(1))) \ = VMLINUX_SYMBOL_STR(sym); \ - static const struct kernel_symbol __ksymtab_##sym \ - __used \ - __attribute__((section("___ksymtab" sec "+" #sym), used)) \ - = { (unsigned long)&sym, __kstrtab_##sym } + static void * __attribute__((section(".discard"), used)) \ + __discard_##sym(void) { return (void *)&sym; } \ + asm(" .section \"___ksymtab" sec "+" #sym "\", \"a\" \n" \ + " .balign 8 \n" \ + VMLINUX_SYMBOL_STR(__ksymtab_##sym) ": \n" \ + " .long " VMLINUX_SYMBOL_STR(sym) "- . \n" \ + " .long " VMLINUX_SYMBOL_STR(__kstrtab_##sym) "- .\n" \ + " .previous \n") #if defined(__KSYM_DEPS__) diff --git a/kernel/module.c b/kernel/module.c index 40f983cbea81..f09529f695f9 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -88,6 +88,12 @@ /* If this is set, the section belongs in the init part of the module */ #define INIT_OFFSET_MASK (1UL << (BITS_PER_LONG-1)) +struct kernel_symbol +{ + signed int value_offset; + signed int name_offset; +}; + /* * Mutex protects: * 1) List of modules (also safely readable with preempt_disable), @@ -539,12 +545,23 @@ static bool check_symbol(const struct symsearch *syms, return true; } +static unsigned long kernel_symbol_value(const struct kernel_symbol *sym) +{ + return (unsigned long)&sym->value_offset + sym->value_offset; +} + +static const char *kernel_symbol_name(const struct kernel_symbol *sym) +{ + return (const char *)((unsigned long)&sym->name_offset + + sym->name_offset); +} + static int cmp_name(const void *va, const void *vb) { const char *a; const struct kernel_symbol *b; a = va; b = vb; - return strcmp(a, b->name); + return strcmp(a, kernel_symbol_name(b)); } static bool find_symbol_in_section(const struct symsearch *syms, @@ -2190,7 +2207,7 @@ void *__symbol_get(const char *symbol) sym = NULL; preempt_enable(); - return sym ? (void *)sym->value : NULL; + return sym ? (void *)kernel_symbol_value(sym) : NULL; } EXPORT_SYMBOL_GPL(__symbol_get); @@ -2220,10 +2237,12 @@ static int verify_export_symbols(struct module *mod) for (i = 0; i < ARRAY_SIZE(arr); i++) { for (s = arr[i].sym; s < arr[i].sym + arr[i].num; s++) { - if (find_symbol(s->name, &owner, NULL, true, false)) { + if (find_symbol(kernel_symbol_name(s), &owner, NULL, + true, false)) { pr_err("%s: exports duplicate symbol %s" " (owned by %s)\n", - mod->name, s->name, module_name(owner)); + mod->name, kernel_symbol_name(s), + module_name(owner)); return -ENOEXEC; } } @@ -2272,7 +2291,7 @@ static int simplify_symbols(struct module *mod, const struct load_info *info) ksym = resolve_symbol_wait(mod, info, name); /* Ok if resolved. */ if (ksym && !IS_ERR(ksym)) { - sym[i].st_value = ksym->value; + sym[i].st_value = kernel_symbol_value(ksym); break; } @@ -2532,7 +2551,7 @@ static int is_exported(const char *name, unsigned long value, ks = lookup_symbol(name, __start___ksymtab, __stop___ksymtab); else ks = lookup_symbol(name, mod->syms, mod->syms + mod->num_syms); - return ks != NULL && ks->value == value; + return ks != NULL && kernel_symbol_value(ks) == value; } /* As per nm */