From patchwork Wed Nov 28 08:54:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 152237 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp831600ljp; Wed, 28 Nov 2018 00:55:09 -0800 (PST) X-Google-Smtp-Source: AFSGD/WDXXxeiP8EA7QYF5Xy1XtAs1qbGAW4ep2AzeazvKejN75GMyPsZqbaPCaTIXgiqjfdqWkb X-Received: by 2002:a17:902:7443:: with SMTP id e3mr33230790plt.304.1543395309291; Wed, 28 Nov 2018 00:55:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543395309; cv=none; d=google.com; s=arc-20160816; b=qtz0BBUo4uwVJ8pPUdH0/CPJBBFUc6SKOVPVRmgh14R3aEKlZf9D88qzlb71oI9DBM aJVc/JT23tifyCYaKW4TFxaWEfwY4cqU+YPyy24U8eAhkk92H3UZ96B6SBfKcEAYEo5r qEIcTGTTKz6icoaJSwXQEGhIAdevdF6sQNypkwMQ1zjEQjQYmKASXEfDrbI+6z2R0F05 lmMDh8gGsSVZ1ethZVzhCMwV+bieRJjtwCmCE11HqDbJL9v4RnKwVAPy0NtcuFkoeS1a cL8083+mzcs+T6grNnBUslKu5tUqbpT9k6KPVE6b/6sFdQA6bPayi/XZcH8RHqB0QkZd gyUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from; bh=EiKAPg+v9Yos5ekdhWgmBYLQgr5jNSeL51S/xnUONaA=; b=VuDeXkyS4hvSFEc3RaFMsP69xsQS7AibtDLZRiKSGBFsOuWIOeQIu1S+iYsajlfnL2 zBBtFVmbDoRcuXxNs4DPzMRrKmO4Nyjy+NuozKLNvr7ST+yEkBwXTXyCCVbHAsncasJ6 nNVTwwi77i+7NzySxE61DvQUXF5dGNXZbjHYWZcYBQoeVEKUIoONzjWEWphlL+a9Ez9f 1dBJBZ26r/sRNq9s97RZxegyOUFEnD6pXXackQ+Qp6ePi0EjiC8dzWlm6RxmpoO6nbkM GvmRL0KCtoZ5/8Iy9mZNc2dkZn1JJA/1tEu/2JgkYMgsel1g/12vRG4qsWtzOif96YUy HKXA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t4si7356426pfj.183.2018.11.28.00.55.09; Wed, 28 Nov 2018 00:55:09 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727790AbeK1Tz7 (ORCPT + 32 others); Wed, 28 Nov 2018 14:55:59 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:59350 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727726AbeK1Tz6 (ORCPT ); Wed, 28 Nov 2018 14:55:58 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1E283359B; Wed, 28 Nov 2018 00:55:06 -0800 (PST) Received: from localhost.localdomain (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4E26D3F5A0; Wed, 28 Nov 2018 00:55:04 -0800 (PST) From: Mark Rutland To: linux-kernel@vger.kernel.org, mingo@kernel.org Cc: Mark Rutland , Andrew Morton , Boqun Feng , Borislav Petkov , Peter Zijlstra , Will Deacon Subject: [PATCHv2] locking/atomics: build atomic headers as required Date: Wed, 28 Nov 2018 08:54:55 +0000 Message-Id: <20181128085455.1164-1-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Andrew and Ingo report that the check-atomics.sh script is simply too slow to run for every kernel build, and it's impractical to make it faster without rewriting it in something other than shell. Rather than committing the generated headers, let's regenerate these as-required, if we change the data or scripts used to generate the atomic headers, or when building from a pristine tree. That ensures they're always up-to-date, allows them to be built in parallel, and avoid redundant rebuilds, which is a 2-8s saving per incremental build. Since the results are not committed, it's very obvious that they should not be modified directly. If we need to generate more headers in future, it's easy to extend Makefile.genheader to permit this. I've verified that this works in the cases we previously had issues with (out-of-tree builds and where scripts have no execute permissions), and have tested these cases for both x86_64 and arm64. The diffstat looks nice, at least... Signed-off-by: Mark Rutland Cc: Andrew Morton Cc: Boqun Feng Cc: Borislav Petkov Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Will Deacon Cc: linux-kernel@vger.kernel.org --- Kbuild | 18 +- Makefile | 8 +- arch/arm64/include/asm/atomic.h | 2 +- arch/x86/include/asm/atomic.h | 2 +- include/asm-generic/atomic-instrumented.h | 1787 ---------------------- include/asm-generic/atomic-long.h | 1012 ------------- include/linux/atomic-fallback.h | 2294 ----------------------------- include/linux/atomic.h | 4 +- scripts/Makefile.genheader | 26 + scripts/atomic/check-atomics.sh | 19 - 10 files changed, 38 insertions(+), 5134 deletions(-) delete mode 100644 include/asm-generic/atomic-instrumented.h delete mode 100644 include/asm-generic/atomic-long.h delete mode 100644 include/linux/atomic-fallback.h create mode 100644 scripts/Makefile.genheader delete mode 100755 scripts/atomic/check-atomics.sh Hi Ingo, Would you be happy to pick this up? Andrew and Peter seemed happy with v1 [1]. To save on ~5100 lines of preimage for the deleted files, I generated this patch with `git format-patch -D`, which git am will complain about. I've pushed this to my atomics/regenerate branch [2], based on the tip locking/core branch, in the hope you're happy to pick it from there. If you can let me know your preference I can resend this however you prefer. Thanks, Mark. Since v1 [1]: * Clarify commit message w.r.t. when headers get regenerated * Add missing dependencies to Makefile.genheader * Minor cleanups for Makefile.genheader [1] https://lkml.kernel.org/r/20181123153321.8561-1-mark.rutland@arm.com [2] git://git.kernel.org/pub/scm/linux/kernel/git/mark/linux.git atomics/regenerate diff --git a/scripts/atomic/check-atomics.sh b/scripts/atomic/check-atomics.sh deleted file mode 100755 index c30101cddf2d..000000000000 -- 2.11.0 diff --git a/Kbuild b/Kbuild index 780048056ac5..005304205482 100644 --- a/Kbuild +++ b/Kbuild @@ -6,8 +6,7 @@ # 2) Generate timeconst.h # 3) Generate asm-offsets.h (may need bounds.h and timeconst.h) # 4) Check for missing system calls -# 5) check atomics headers are up-to-date -# 6) Generate constants.py (may need bounds.h) +# 5) Generate constants.py (may need bounds.h) ##### # 1) Generate bounds.h @@ -73,20 +72,7 @@ missing-syscalls: scripts/checksyscalls.sh $(offsets-file) FORCE $(call cmd,syscalls) ##### -# 5) Check atomic headers are up-to-date -# - -always += old-atomics -targets += old-atomics - -quiet_cmd_atomics = CALL $< - cmd_atomics = $(CONFIG_SHELL) $< - -old-atomics: scripts/atomic/check-atomics.sh FORCE - $(call cmd,atomics) - -##### -# 6) Generate constants for Python GDB integration +# 5) Generate constants for Python GDB integration # extra-$(CONFIG_GDB_SCRIPTS) += build_constants_py diff --git a/Makefile b/Makefile index 6c40d547513c..fd871d2c2f54 100644 --- a/Makefile +++ b/Makefile @@ -224,7 +224,7 @@ clean-targets := %clean mrproper cleandocs no-dot-config-targets := $(clean-targets) \ cscope gtags TAGS tags help% %docs check% coccicheck \ $(version_h) headers_% archheaders archscripts \ - %asm-generic kernelversion %src-pkg + %asm-generic genheader kernelversion %src-pkg no-sync-config-targets := $(no-dot-config-targets) install %install \ kernelrelease @@ -1089,7 +1089,7 @@ endif # prepare2 creates a makefile if using a separate output directory. # From this point forward, .config has been reprocessed, so any rules # that need to depend on updated CONFIG_* values can be checked here. -prepare2: prepare3 outputmakefile asm-generic +prepare2: prepare3 outputmakefile asm-generic genheader prepare1: prepare2 $(version_h) $(autoksyms_h) include/generated/utsrelease.h $(cmd_crmodverdir) @@ -1113,6 +1113,10 @@ uapi-asm-generic: $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.asm-generic \ src=uapi/asm obj=arch/$(SRCARCH)/include/generated/uapi/asm +PHONY += genheader +genheader: + $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.genheader obj=include/generated + PHONY += prepare-objtool prepare-objtool: $(objtool_target) diff --git a/arch/arm64/include/asm/atomic.h b/arch/arm64/include/asm/atomic.h index 1f4e9ee641c9..57c73cde3a51 100644 --- a/arch/arm64/include/asm/atomic.h +++ b/arch/arm64/include/asm/atomic.h @@ -166,7 +166,7 @@ #define arch_atomic64_dec_if_positive arch_atomic64_dec_if_positive -#include +#include #endif #endif diff --git a/arch/x86/include/asm/atomic.h b/arch/x86/include/asm/atomic.h index ea3d95275b43..d1e3aca236e2 100644 --- a/arch/x86/include/asm/atomic.h +++ b/arch/x86/include/asm/atomic.h @@ -262,6 +262,6 @@ static inline int arch_atomic_fetch_xor(int i, atomic_t *v) # include #endif -#include +#include #endif /* _ASM_X86_ATOMIC_H */ diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h deleted file mode 100644 index b8f5b35216e1..000000000000 diff --git a/include/asm-generic/atomic-long.h b/include/asm-generic/atomic-long.h deleted file mode 100644 index a833d385a70b..000000000000 diff --git a/include/linux/atomic-fallback.h b/include/linux/atomic-fallback.h deleted file mode 100644 index 1c02c0112fbb..000000000000 diff --git a/include/linux/atomic.h b/include/linux/atomic.h index 4c0d009a46f0..bea7ceec44af 100644 --- a/include/linux/atomic.h +++ b/include/linux/atomic.h @@ -71,8 +71,8 @@ __ret; \ }) -#include +#include -#include +#include #endif /* _LINUX_ATOMIC_H */ diff --git a/scripts/Makefile.genheader b/scripts/Makefile.genheader new file mode 100644 index 000000000000..03aac5628f69 --- /dev/null +++ b/scripts/Makefile.genheader @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Generate common headers under include/generated/ + +include scripts/Kbuild.include + +.PHONY: all + +targets += atomic-fallback.h +targets += atomic-instrumented.h +targets += atomic-long.h + +all: $(addprefix $(obj)/,$(targets)) + +atomic-dir := $(srctree)/scripts/atomic +atomic-tbl := $(atomic-dir)/atomics.tbl + +atomic-deps := $(atomic-tbl) $(atomic-dir)/atomic-tbl.sh \ + $(atomic-dir)/fallbacks/* + +quiet_cmd_genatomic = GENHDR $@ +cmd_genatomic = $(CONFIG_SHELL) $< $(atomic-tbl) > $@ + +$(obj)/atomic-%.h: $(atomic-dir)/gen-atomic-%.sh $(atomic-deps) + $(call if_changed,genatomic) +