From patchwork Wed Sep 19 06:51:41 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 147025 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp343845ljw; Tue, 18 Sep 2018 23:52:37 -0700 (PDT) X-Google-Smtp-Source: ANB0VdZ4ytiXnUZAz/Nn58G9fjmL2JCpKLebgOirs/ZyyPUoy3cLafP6S2sKRebrJGzFHaR0Yo1S X-Received: by 2002:a17:902:e004:: with SMTP id ca4-v6mr32106873plb.252.1537339957147; Tue, 18 Sep 2018 23:52:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1537339957; cv=none; d=google.com; s=arc-20160816; b=GT3HefypF0C9MC8a1pYqflKFNqNkl/AiW8oLHbT3occJNQqdKXYCury+NzgzvIMqBC znQeMjdJEAsX/YKYQ6xf9wtiumMWGcIpTph4KASqgbF92p7JQ6FoFNOhPnIuxj7vVDBv VHoJB4hE2UhTD4aXVj7r1Xcrtk7xchvdv2Y9AtZ7GrBxDlAVh0rQNJLvivwlg73RkJ8z ZNXNEANz8uB+X9GYi/K8OWtpIjF98aD908q44lsvSR0YyGhTi7Ila39qQakLWGOh8UOf xvUmpBzD5tnF/nKksrS2yn0qYA2Tp78CRMtllKV0NTcnIHnmnDIiaZWRlC9HvzGNlID6 NqJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=XGRY1T3h+gs2S6eNeT9ZaVSn/BnvjE5DrD6FeDRoeQ4=; b=IG65i4FVs7PwEJmWc0Ldtd43VPkN6TiYboDp+cYE/sZCgR6K0oEpzP5Fq69Av4JN1Z mxWAygAIIFKdORp7UavLNiUTUTo8GrwNMepgJOhK6AZrK4L9REUX/k132qXnpXOsEKJY qCRKntojkd6AdWN+3bVRip2P5Hsj8h+IYidUGV24R9be+simuqGmAuCTlzA+ndc+hdTe 2knC1UQE1m7c4zI4tPNS8TRmnnHsmxfGfNE7Mt4zdFrGuKWrYndp0SZE7or6NtE0EoYV 1U31nZAIA7Le2DcoTw5EW4SI0NQs7p9fv3lAMghLza4BV8j686bGOZyb2i5k+Sq/M7UF dS2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="NB7mskp/"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t64-v6si19375292pgd.176.2018.09.18.23.52.36; Tue, 18 Sep 2018 23:52:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="NB7mskp/"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731082AbeISM3E (ORCPT + 32 others); Wed, 19 Sep 2018 08:29:04 -0400 Received: from mail-io1-f68.google.com ([209.85.166.68]:45074 "EHLO mail-io1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731062AbeISM3D (ORCPT ); Wed, 19 Sep 2018 08:29:03 -0400 Received: by mail-io1-f68.google.com with SMTP id e12-v6so3584993iok.12 for ; Tue, 18 Sep 2018 23:52:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XGRY1T3h+gs2S6eNeT9ZaVSn/BnvjE5DrD6FeDRoeQ4=; b=NB7mskp/qv9FTKKiv/NIH3QElhQgT74KLCGGPtWIXEbKuGh+Mn7QgIY8P0rP2earA6 HGPls0WytLUlXWc6UX60x6rMJgjdziPfxccBg+oM7l/vTE9ydrroiYYiRC+mZAFx3Pl3 mkSPKyvp48KYUfPkN/+fCiXjZWlfxNVTGFUGg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XGRY1T3h+gs2S6eNeT9ZaVSn/BnvjE5DrD6FeDRoeQ4=; b=nhEVl0+Imvl2ss0frVxWykOEsNL4mm9DszTvNqXzLG6SBPShmGDhhsD9A48WyrLCvP FO6aHI890kojMK8tSQqdV++u/4qVHXffWBMngMPWxB9txa/lq1H08iukh9sXtsBDGr+7 Ly8MdN/d4D60jd6BwILfqxheprLKfNToGtBGSjFsgYvMZx1Jc4SEiDsz6/GZ81leO0Rf GIMyThYgtvEujrdzXAPybkbq4bE9gmaBVDB/Qu3AEkzTtsdQveFOqm5Mew19e7R577Rj 9NNvyf44bdp6nD33kExpn87oP6yjHTD/X72aHU+BeWXjvqVPu8WwwYzJgN0MtUcDMVXH Td1A== X-Gm-Message-State: APzg51Bj7oxrM3GcAzTlL69SySxhQ6Ts76SGBqyL+r0SA5ffjWiigQgb LxeDDSVKkHmMch7k/qpFZ+A/QLrc5g82Fw== X-Received: by 2002:a5e:db07:: with SMTP id q7-v6mr28850533iop.81.1537339953658; Tue, 18 Sep 2018 23:52:33 -0700 (PDT) Received: from localhost.localdomain ([209.82.80.116]) by smtp.gmail.com with ESMTPSA id x68-v6sm7939477ita.2.2018.09.18.23.52.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 18 Sep 2018 23:52:33 -0700 (PDT) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org, Ard Biesheuvel , Arnd Bergmann , Heiko Carstens , Kees Cook , Will Deacon , Thomas Gleixner , Catalin Marinas , Ingo Molnar , Steven Rostedt , Martin Schwidefsky , Jessica Yu , Peter Zijlstra Subject: [PATCH v3 6/9] x86/kernel: jump_table: use relative references Date: Tue, 18 Sep 2018 23:51:41 -0700 Message-Id: <20180919065144.25010-7-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180919065144.25010-1-ard.biesheuvel@linaro.org> References: <20180919065144.25010-1-ard.biesheuvel@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Similar to the arm64 case, 64-bit x86 can benefit from using relative references rather than absolute ones when emitting struct jump_entry instances. Not only does this reduce the memory footprint of the entries themselves by 33%, it also removes the need for carrying relocation metadata on relocatable builds (i.e., for KASLR) which saves a fair chunk of .init space as well (although the savings are not as dramatic as on arm64) Signed-off-by: Ard Biesheuvel --- arch/x86/Kconfig | 1 + arch/x86/include/asm/jump_label.h | 24 +++++++------------- tools/objtool/special.c | 4 ++-- 3 files changed, 11 insertions(+), 18 deletions(-) -- 2.17.1 diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 1a0be022f91d..603f374f871d 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -119,6 +119,7 @@ config X86 select HAVE_ARCH_AUDITSYSCALL select HAVE_ARCH_HUGE_VMAP if X86_64 || X86_PAE select HAVE_ARCH_JUMP_LABEL + select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KGDB select HAVE_ARCH_MMAP_RND_BITS if MMU diff --git a/arch/x86/include/asm/jump_label.h b/arch/x86/include/asm/jump_label.h index 8c0de4282659..21efc9d07ed9 100644 --- a/arch/x86/include/asm/jump_label.h +++ b/arch/x86/include/asm/jump_label.h @@ -37,7 +37,8 @@ static __always_inline bool arch_static_branch(struct static_key *key, bool bran ".byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t" ".pushsection __jump_table, \"aw\" \n\t" _ASM_ALIGN "\n\t" - _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t" + ".long 1b - ., %l[l_yes] - . \n\t" + _ASM_PTR "%c0 + %c1 - .\n\t" ".popsection \n\t" : : "i" (key), "i" (branch) : : l_yes); @@ -53,7 +54,8 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool "2:\n\t" ".pushsection __jump_table, \"aw\" \n\t" _ASM_ALIGN "\n\t" - _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t" + ".long 1b - ., %l[l_yes] - . \n\t" + _ASM_PTR "%c0 + %c1 - .\n\t" ".popsection \n\t" : : "i" (key), "i" (branch) : : l_yes); @@ -62,18 +64,6 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool return true; } -#ifdef CONFIG_X86_64 -typedef u64 jump_label_t; -#else -typedef u32 jump_label_t; -#endif - -struct jump_entry { - jump_label_t code; - jump_label_t target; - jump_label_t key; -}; - #else /* __ASSEMBLY__ */ .macro STATIC_JUMP_IF_TRUE target, key, def @@ -88,7 +78,8 @@ struct jump_entry { .endif .pushsection __jump_table, "aw" _ASM_ALIGN - _ASM_PTR .Lstatic_jump_\@, \target, \key + .long .Lstatic_jump_\@ - ., \target - . + _ASM_PTR \key - . .popsection .endm @@ -104,7 +95,8 @@ struct jump_entry { .endif .pushsection __jump_table, "aw" _ASM_ALIGN - _ASM_PTR .Lstatic_jump_\@, \target, \key + 1 + .long .Lstatic_jump_\@ - ., \target - . + _ASM_PTR \key + 1 - . .popsection .endm diff --git a/tools/objtool/special.c b/tools/objtool/special.c index 84f001d52322..50af4e1274b3 100644 --- a/tools/objtool/special.c +++ b/tools/objtool/special.c @@ -30,9 +30,9 @@ #define EX_ORIG_OFFSET 0 #define EX_NEW_OFFSET 4 -#define JUMP_ENTRY_SIZE 24 +#define JUMP_ENTRY_SIZE 16 #define JUMP_ORIG_OFFSET 0 -#define JUMP_NEW_OFFSET 8 +#define JUMP_NEW_OFFSET 4 #define ALT_ENTRY_SIZE 13 #define ALT_ORIG_OFFSET 0