From patchwork Tue Mar 27 05:29:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,01/21] kbuild: remove kbuild cache X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132451 Message-Id: <1522128575-5326-2-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:15 +0900 From: Masahiro Yamada List-Id: The kbuild cache was introduced to remember the result of shell commands, some of which are expensive to compute, such as $(call cc-option,...). However, this turned out not so clever as I had first expected. Actually, it is problematic. For example, "$(CC) -print-file-name" is cached. If the compiler is updated, the stale search path causes build error, which is difficult to figure out. Another problem scenario is cache files could be touched while install targets are running under the root permission. We can patch them if desired, but the build infrastructure is getting uglier and uglier. Now, we are going to move compiler flag tests to the configuration phase. If this is completed, the result of compiler tests will be naturally cached in the .config file. We will not have performance issues of incremental building since this testing only happens at Kconfig time. To start this work with a cleaner code base, remove the kbuild cache first. Revert the following commits: Commit 9a234a2e3843 ("kbuild: create directory for make cache only when necessary") Commit e17c400ae194 ("kbuild: shrink .cache.mk when it exceeds 1000 lines") Commit 4e56207130ed ("kbuild: Cache a few more calls to the compiler") Commit 3298b690b21c ("kbuild: Add a cache for generated variables") Signed-off-by: Masahiro Yamada --- Changes in v2: None Makefile | 5 +-- scripts/Kbuild.include | 101 +++++++------------------------------------------ 2 files changed, 16 insertions(+), 90 deletions(-) -- 2.7.4 Reviewed-by: Kees Cook diff --git a/Makefile b/Makefile index ff62b8e..c55cf95 100644 --- a/Makefile +++ b/Makefile @@ -656,7 +656,7 @@ KBUILD_CFLAGS += $(call cc-ifversion, -lt, 0409, \ KBUILD_CFLAGS += $(call cc-option,--param=allow-store-data-races=0) # check for 'asm goto' -ifeq ($(call shell-cached,$(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC) $(KBUILD_CFLAGS)), y) +ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC) $(KBUILD_CFLAGS)), y) KBUILD_CFLAGS += -DCC_HAVE_ASM_GOTO KBUILD_AFLAGS += -DCC_HAVE_ASM_GOTO endif @@ -811,7 +811,7 @@ KBUILD_CFLAGS += $(call cc-option,-fdata-sections,) endif # arch Makefile may override CC so keep this after arch Makefile is included -NOSTDINC_FLAGS += -nostdinc -isystem $(call shell-cached,$(CC) -print-file-name=include) +NOSTDINC_FLAGS += -nostdinc -isystem $(shell $(CC) -print-file-name=include) CHECKFLAGS += $(NOSTDINC_FLAGS) # warn about C99 declaration after statement @@ -1615,7 +1615,6 @@ clean: $(clean-dirs) -o -name '.*.d' -o -name '.*.tmp' -o -name '*.mod.c' \ -o -name '*.symtypes' -o -name 'modules.order' \ -o -name modules.builtin -o -name '.tmp_*.o.*' \ - -o -name .cache.mk \ -o -name '*.c.[012]*.*' \ -o -name '*.ll' \ -o -name '*.gcno' \) -type f -print | xargs rm -f diff --git a/scripts/Kbuild.include b/scripts/Kbuild.include index cce31ee..9f7eb10 100644 --- a/scripts/Kbuild.include +++ b/scripts/Kbuild.include @@ -8,8 +8,6 @@ squote := ' empty := space := $(empty) $(empty) space_escape := _-_SPACE_-_ -right_paren := ) -left_paren := ( ### # Name of target with a '.' as filename prefix. foo/bar.o => foo/.bar.o @@ -82,71 +80,6 @@ cc-cross-prefix = \ echo $(c); \ fi))) -# Tools for caching Makefile variables that are "expensive" to compute. -# -# Here we want to help deal with variables that take a long time to compute -# by making it easy to store these variables in a cache. -# -# The canonical example here is testing for compiler flags. On a simple system -# each call to the compiler takes 10 ms, but on a system with a compiler that's -# called through various wrappers it can take upwards of 100 ms. If we have -# 100 calls to the compiler this can take 1 second (on a simple system) or 10 -# seconds (on a complicated system). -# -# The "cache" will be in Makefile syntax and can be directly included. -# Any time we try to reference a variable that's not in the cache we'll -# calculate it and store it in the cache for next time. - -# Include values from last time -make-cache := $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/,$(if $(obj),$(obj)/)).cache.mk -$(make-cache): ; --include $(make-cache) - -cached-data := $(filter __cached_%, $(.VARIABLES)) - -# If cache exceeds 1000 lines, shrink it down to 500. -ifneq ($(word 1000,$(cached-data)),) -$(shell tail -n 500 $(make-cache) > $(make-cache).tmp; \ - mv $(make-cache).tmp $(make-cache)) -endif - -create-cache-dir := $(if $(KBUILD_SRC),$(if $(cache-data),,1)) - -# Usage: $(call __sanitize-opt,Hello=Hola$(comma)Goodbye Adios) -# -# Convert all '$', ')', '(', '\', '=', ' ', ',', ':' to '_' -__sanitize-opt = $(subst $$,_,$(subst $(right_paren),_,$(subst $(left_paren),_,$(subst \,_,$(subst =,_,$(subst $(space),_,$(subst $(comma),_,$(subst :,_,$(1))))))))) - -# Usage: $(call shell-cached,shell_command) -# Example: $(call shell-cached,md5sum /usr/bin/gcc) -# -# If we've already seen a call to this exact shell command (even in a -# previous invocation of make!) we'll return the value. If not, we'll -# compute it and store the result for future runs. -# -# This is a bit of voodoo, but basic explanation is that if the variable -# was undefined then we'll evaluate the shell command and store the result -# into the variable. We'll then store that value in the cache and finally -# output the value. -# -# NOTE: The $$(2) here isn't actually a parameter to __run-and-store. We -# happen to know that the caller will have their shell command in $(2) so the -# result of "call"ing this will produce a reference to that $(2). The reason -# for this strangeness is to avoid an extra level of eval (and escaping) of -# $(2). -define __run-and-store -ifeq ($(origin $(1)),undefined) - $$(eval $(1) := $$(shell $$(2))) -ifeq ($(create-cache-dir),1) - $$(shell mkdir -p $(dir $(make-cache))) - $$(eval create-cache-dir :=) -endif - $$(shell echo '$(1) := $$($(1))' >> $(make-cache)) -endif -endef -__shell-cached = $(eval $(call __run-and-store,$(1)))$($(1)) -shell-cached = $(call __shell-cached,__cached_$(call __sanitize-opt,$(1)),$(1)) - # output directory for tests below TMPOUT := $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/) @@ -154,36 +87,30 @@ TMPOUT := $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/) # Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise) # Exit code chooses option. "$$TMP" serves as a temporary file and is # automatically cleaned up. -__try-run = set -e; \ +try-run = $(shell set -e; \ TMP="$(TMPOUT).$$$$.tmp"; \ TMPO="$(TMPOUT).$$$$.o"; \ if ($(1)) >/dev/null 2>&1; \ then echo "$(2)"; \ else echo "$(3)"; \ fi; \ - rm -f "$$TMP" "$$TMPO" - -try-run = $(shell $(__try-run)) - -# try-run-cached -# This works like try-run, but the result is cached. -try-run-cached = $(call shell-cached,$(__try-run)) + rm -f "$$TMP" "$$TMPO") # as-option # Usage: cflags-y += $(call as-option,-Wa$(comma)-isa=foo,) -as-option = $(call try-run-cached,\ +as-option = $(call try-run,\ $(CC) $(KBUILD_CFLAGS) $(1) -c -x assembler /dev/null -o "$$TMP",$(1),$(2)) # as-instr # Usage: cflags-y += $(call as-instr,instr,option1,option2) -as-instr = $(call try-run-cached,\ +as-instr = $(call try-run,\ printf "%b\n" "$(1)" | $(CC) $(KBUILD_AFLAGS) -c -x assembler -o "$$TMP" -,$(2),$(3)) # __cc-option # Usage: MY_CFLAGS += $(call __cc-option,$(CC),$(MY_CFLAGS),-march=winchip-c6,-march=i586) -__cc-option = $(call try-run-cached,\ +__cc-option = $(call try-run,\ $(1) -Werror $(2) $(3) -c -x c /dev/null -o "$$TMP",$(3),$(4)) # Do not attempt to build with gcc plugins during cc-option tests. @@ -203,23 +130,23 @@ hostcc-option = $(call __cc-option, $(HOSTCC),\ # cc-option-yn # Usage: flag := $(call cc-option-yn,-march=winchip-c6) -cc-option-yn = $(call try-run-cached,\ +cc-option-yn = $(call try-run,\ $(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) $(1) -c -x c /dev/null -o "$$TMP",y,n) # cc-disable-warning # Usage: cflags-y += $(call cc-disable-warning,unused-but-set-variable) -cc-disable-warning = $(call try-run-cached,\ +cc-disable-warning = $(call try-run,\ $(CC) -Werror $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1))) # cc-name # Expands to either gcc or clang -cc-name = $(call shell-cached,$(CC) -v 2>&1 | grep -q "clang version" && echo clang || echo gcc) +cc-name = $(shell $(CC) -v 2>&1 | grep -q "clang version" && echo clang || echo gcc) # cc-version -cc-version = $(call shell-cached,$(CONFIG_SHELL) $(srctree)/scripts/gcc-version.sh $(CC)) +cc-version = $(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-version.sh $(CC)) # cc-fullversion -cc-fullversion = $(call shell-cached,$(CONFIG_SHELL) \ +cc-fullversion = $(shell $(CONFIG_SHELL) \ $(srctree)/scripts/gcc-version.sh -p $(CC)) # cc-ifversion @@ -232,21 +159,21 @@ cc-if-fullversion = $(shell [ $(cc-fullversion) $(1) $(2) ] && echo $(3) || echo # cc-ldoption # Usage: ldflags += $(call cc-ldoption, -Wl$(comma)--hash-style=both) -cc-ldoption = $(call try-run-cached,\ +cc-ldoption = $(call try-run,\ $(CC) $(1) $(KBUILD_CPPFLAGS) $(CC_OPTION_CFLAGS) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2)) # ld-option # Usage: LDFLAGS += $(call ld-option, -X) -ld-option = $(call try-run-cached, $(LD) $(LDFLAGS) $(1) -v,$(1),$(2)) +ld-option = $(call try-run, $(LD) $(LDFLAGS) $(1) -v,$(1),$(2)) # ar-option # Usage: KBUILD_ARFLAGS := $(call ar-option,D) # Important: no spaces around options -ar-option = $(call try-run-cached, $(AR) rc$(1) "$$TMP",$(1),$(2)) +ar-option = $(call try-run, $(AR) rc$(1) "$$TMP",$(1),$(2)) # ld-version # Note this is mainly for HJ Lu's 3 number binutil versions -ld-version = $(call shell-cached,$(LD) --version | $(srctree)/scripts/ld-version.sh) +ld-version = $(shell $(LD) --version | $(srctree)/scripts/ld-version.sh) # ld-ifversion # Usage: $(call ld-ifversion, -ge, 22252, y) From patchwork Tue Mar 27 05:29:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,02/21] kbuild: remove CONFIG_CROSS_COMPILE support X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132452 Message-Id: <1522128575-5326-3-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:16 +0900 From: Masahiro Yamada List-Id: Kbuild provides a couple of ways to specify CROSS_COMPILE: [1] Command line [2] Environment [3] arch/*/Makefile (only some architectures) [4] CONFIG_CROSS_COMPILE [4] is problematic for the compiler capability tests in Kconfig. CONFIG_CROSS_COMPILE allows users to change the compiler prefix from 'make menuconfig', etc. It means, the compiler options would have to be all re-calculated everytime CONFIG_CROSS_COMPILE is changed. To avoid complexity and performance issues, I'd like to evaluate the shell commands statically, i.e. only parsing Kconfig files. I guess the majority is [1] or [2]. Currently, there are only 4 defconfig files that specify CONFIG_CROSS_COMPILE. arch/arm/configs/lpc18xx_defconfig arch/hexagon/configs/comet_defconfig arch/openrisc/configs/or1ksim_defconfig arch/openrisc/configs/simple_smp_defconfig Signed-off-by: Masahiro Yamada --- Changes in v2: None Makefile | 3 --- init/Kconfig | 9 --------- 2 files changed, 12 deletions(-) -- 2.7.4 Reviewed-by: Kees Cook diff --git a/Makefile b/Makefile index c55cf95..5c395ed 100644 --- a/Makefile +++ b/Makefile @@ -316,12 +316,9 @@ SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \ # CROSS_COMPILE can be set on the command line # make CROSS_COMPILE=ia64-linux- # Alternatively CROSS_COMPILE can be set in the environment. -# A third alternative is to store a setting in .config so that plain -# "make" in the configured kernel build directory always uses that. # Default value for CROSS_COMPILE is not to prefix executables # Note: Some architectures assign CROSS_COMPILE in their arch/*/Makefile ARCH ?= $(SUBARCH) -CROSS_COMPILE ?= $(CONFIG_CROSS_COMPILE:"%"=%) # Architecture as present in compile.h UTS_MACHINE := $(ARCH) diff --git a/init/Kconfig b/init/Kconfig index e37f4b2..df18492 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -54,15 +54,6 @@ config INIT_ENV_ARG_LIMIT Maximum of each of the number of arguments and environment variables passed to init from the kernel command line. - -config CROSS_COMPILE - string "Cross-compiler tool prefix" - help - Same as running 'make CROSS_COMPILE=prefix-' but stored for - default make runs in this kernel build directory. You don't - need to set this unless you want the configured kernel build - directory to select the cross-compiler automatically. - config COMPILE_TEST bool "Compile also drivers which will not load" depends on !UML From patchwork Tue Mar 27 05:29:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,03/21] kconfig: move and rename sym_expand_string_value() X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132455 Message-Id: <1522128575-5326-4-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:17 +0900 From: Masahiro Yamada List-Id: This helper expands symbols contained in a string. I am about to change it to expand environments instead of symbols. Also, I will add function expansion later. Rename it to expand_string_value(), and move it to util.c, which is a more suitable place. Signed-off-by: Masahiro Yamada --- Changes in v2: None scripts/kconfig/lkc.h | 1 + scripts/kconfig/lkc_proto.h | 1 - scripts/kconfig/symbol.c | 53 ------------------------------------------- scripts/kconfig/util.c | 55 ++++++++++++++++++++++++++++++++++++++++++++- scripts/kconfig/zconf.y | 2 +- 5 files changed, 56 insertions(+), 56 deletions(-) -- 2.7.4 Reviewed-by: Kees Cook diff --git a/scripts/kconfig/lkc.h b/scripts/kconfig/lkc.h index f4394af..c8d9e55 100644 --- a/scripts/kconfig/lkc.h +++ b/scripts/kconfig/lkc.h @@ -111,6 +111,7 @@ void menu_finalize(struct menu *parent); void menu_set_type(int type); /* util.c */ +char *expand_string_value(const char *in); struct file *file_lookup(const char *name); int file_write_dep(const char *name); void *xmalloc(size_t size); diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h index 9dc8abf..9884adc 100644 --- a/scripts/kconfig/lkc_proto.h +++ b/scripts/kconfig/lkc_proto.h @@ -31,7 +31,6 @@ extern struct symbol * symbol_hash[SYMBOL_HASHSIZE]; struct symbol * sym_lookup(const char *name, int flags); struct symbol * sym_find(const char *name); -char *sym_expand_string_value(const char *in); const char * sym_escape_string_value(const char *in); struct symbol ** sym_re_search(const char *pattern); const char * sym_type_name(enum symbol_type type); diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c index f0b2e3b..03143b2 100644 --- a/scripts/kconfig/symbol.c +++ b/scripts/kconfig/symbol.c @@ -906,59 +906,6 @@ struct symbol *sym_find(const char *name) return symbol; } -/* - * Expand symbol's names embedded in the string given in argument. Symbols' - * name to be expanded shall be prefixed by a '$'. Unknown symbol expands to - * the empty string. - */ -char *sym_expand_string_value(const char *in) -{ - const char *src; - char *res; - size_t reslen; - - /* - * Note: 'in' might come from a token that's about to be - * freed, so make sure to always allocate a new string - */ - reslen = strlen(in) + 1; - res = xmalloc(reslen); - res[0] = '\0'; - - while ((src = strchr(in, '$'))) { - char *p, name[SYMBOL_MAXLENGTH]; - const char *symval = ""; - struct symbol *sym; - size_t newlen; - - strncat(res, in, src - in); - src++; - - p = name; - while (isalnum(*src) || *src == '_') - *p++ = *src++; - *p = '\0'; - - sym = sym_find(name); - if (sym != NULL) { - sym_calc_value(sym); - symval = sym_get_string_value(sym); - } - - newlen = strlen(res) + strlen(symval) + strlen(src) + 1; - if (newlen > reslen) { - reslen = newlen; - res = xrealloc(res, reslen); - } - - strcat(res, symval); - in = src; - } - strcat(res, in); - - return res; -} - const char *sym_escape_string_value(const char *in) { const char *p; diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c index c6f6e21..22201a4 100644 --- a/scripts/kconfig/util.c +++ b/scripts/kconfig/util.c @@ -10,11 +10,64 @@ #include #include "lkc.h" +/* + * Expand symbol's names embedded in the string given in argument. Symbols' + * name to be expanded shall be prefixed by a '$'. Unknown symbol expands to + * the empty string. + */ +char *expand_string_value(const char *in) +{ + const char *src; + char *res; + size_t reslen; + + /* + * Note: 'in' might come from a token that's about to be + * freed, so make sure to always allocate a new string + */ + reslen = strlen(in) + 1; + res = xmalloc(reslen); + res[0] = '\0'; + + while ((src = strchr(in, '$'))) { + char *p, name[SYMBOL_MAXLENGTH]; + const char *symval = ""; + struct symbol *sym; + size_t newlen; + + strncat(res, in, src - in); + src++; + + p = name; + while (isalnum(*src) || *src == '_') + *p++ = *src++; + *p = '\0'; + + sym = sym_find(name); + if (sym != NULL) { + sym_calc_value(sym); + symval = sym_get_string_value(sym); + } + + newlen = strlen(res) + strlen(symval) + strlen(src) + 1; + if (newlen > reslen) { + reslen = newlen; + res = xrealloc(res, reslen); + } + + strcat(res, symval); + in = src; + } + strcat(res, in); + + return res; +} + /* file already present in list? If not add it */ struct file *file_lookup(const char *name) { struct file *file; - char *file_name = sym_expand_string_value(name); + char *file_name = expand_string_value(name); for (file = file_list; file; file = file->next) { if (!strcmp(name, file->name)) { diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y index ad6305b..262c464 100644 --- a/scripts/kconfig/zconf.y +++ b/scripts/kconfig/zconf.y @@ -547,7 +547,7 @@ void conf_parse(const char *name) tmp = rootmenu.prompt->text; rootmenu.prompt->text = _(rootmenu.prompt->text); - rootmenu.prompt->text = sym_expand_string_value(rootmenu.prompt->text); + rootmenu.prompt->text = expand_string_value(rootmenu.prompt->text); free((char*)tmp); menu_finalize(&rootmenu); From patchwork Tue Mar 27 05:29:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2, 04/21] kconfig: reference environments directly and remove 'option env=' syntax X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132443 Message-Id: <1522128575-5326-5-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org, Ingo Molnar Date: Tue, 27 Mar 2018 14:29:18 +0900 From: Masahiro Yamada List-Id: To get an environment value, Kconfig needs to define a symbol using "option env=" syntax. It is tedious to add a config entry for each environment given that we need more environments such as 'CC', 'AS', 'srctree' etc. to evaluate the compiler capability in Kconfig. Adding '$' to symbols is weird. Kconfig can reference symbols directly like this: config FOO string default BAR So, I want to use the following syntax to get environment 'BAR' from the system: config FOO string default $BAR Looking at the code, the symbols prefixed with 'S' are expanded by: - conf_expand_value() This is used to expand 'arch/$ARCH/defconfig' and 'defconfig_list' - expand_string_value() This is used to expand strings in 'source' and 'mainmenu' All of them are fixed values independent of user configuration. So, this kind of syntax should be moved to simply take the environment. This change makes the code much cleaner. The bounce symbols 'SRCARCH', 'ARCH', 'SUBARCH', 'KERNELVERSION' are gone. sym_init() hard-coding 'UNAME_RELEASE' is also gone. 'UNAME_RELEASE' should be be given from the environment. ARCH_DEFCONFIG is a normal symbol, so it should be simply referenced by 'default ARCH_DEFCONFIG'. The environments are expanding in the lexer; when '$' is encountered, it is expanded, and resulted strings are pushed back to the input stream. This makes the implementation simpler. For example, the following code works. [Example code] config TOOLCHAIN_LIST string default "My tools: CC=$CC, AS=$AS, CPP=$CPP" [Result] $ make -s alldefconfig && tail -n 1 .config CONFIG_TOOLCHAIN_LIST="My tools: CC=gcc, AS=as, CPP=gcc -E" Signed-off-by: Masahiro Yamada --- I tested all 'make *config' for arch architectures. I confirmed this commit still produced the same result (by my kconfig test tool). Changes in v2: - Move the string expansion to the lexer phase. - Split environment helpers to env.c Documentation/kbuild/kconfig-language.txt | 8 --- Kconfig | 4 -- Makefile | 3 +- arch/sh/Kconfig | 4 +- arch/sparc/Kconfig | 4 +- arch/tile/Kconfig | 2 +- arch/um/Kconfig.common | 4 -- arch/x86/Kconfig | 4 +- arch/x86/um/Kconfig | 4 +- init/Kconfig | 10 +--- scripts/kconfig/confdata.c | 31 +--------- scripts/kconfig/env.c | 95 +++++++++++++++++++++++++++++++ scripts/kconfig/kconf_id.c | 1 - scripts/kconfig/lkc.h | 8 +-- scripts/kconfig/menu.c | 3 - scripts/kconfig/symbol.c | 56 ------------------ scripts/kconfig/util.c | 75 ++++++++---------------- scripts/kconfig/zconf.l | 20 ++++++- scripts/kconfig/zconf.y | 2 +- 19 files changed, 158 insertions(+), 180 deletions(-) create mode 100644 scripts/kconfig/env.c -- 2.7.4 Reviewed-by: Kees Cook diff --git a/Documentation/kbuild/kconfig-language.txt b/Documentation/kbuild/kconfig-language.txt index f5b9493..0e966e8 100644 --- a/Documentation/kbuild/kconfig-language.txt +++ b/Documentation/kbuild/kconfig-language.txt @@ -198,14 +198,6 @@ applicable everywhere (see syntax). enables the third modular state for all config symbols. At most one symbol may have the "modules" option set. - - "env"= - This imports the environment variable into Kconfig. It behaves like - a default, except that the value comes from the environment, this - also means that the behaviour when mixing it with normal defaults is - undefined at this point. The symbol is currently not exported back - to the build environment (if this is desired, it can be done via - another symbol). - - "allnoconfig_y" This declares the symbol as one that should have the value y when using "allnoconfig". Used for symbols that hide other symbols. diff --git a/Kconfig b/Kconfig index 8c4c1cb..e6ece5b 100644 --- a/Kconfig +++ b/Kconfig @@ -5,8 +5,4 @@ # mainmenu "Linux/$ARCH $KERNELVERSION Kernel Configuration" -config SRCARCH - string - option env="SRCARCH" - source "arch/$SRCARCH/Kconfig" diff --git a/Makefile b/Makefile index 5c395ed..4ae1486 100644 --- a/Makefile +++ b/Makefile @@ -284,7 +284,8 @@ include scripts/Kbuild.include # Read KERNELRELEASE from include/config/kernel.release (if it exists) KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null) KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION) -export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION +UNAME_RELEASE := $(shell uname --release) +export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION UNAME_RELEASE # SUBARCH tells the usermode build what the underlying arch is. That is set # first, and if a usermode build is happening, the "ARCH=um" on the command diff --git a/arch/sh/Kconfig b/arch/sh/Kconfig index 97fe293..14f3ef1 100644 --- a/arch/sh/Kconfig +++ b/arch/sh/Kconfig @@ -57,7 +57,7 @@ config SUPERH . config SUPERH32 - def_bool ARCH = "sh" + def_bool "$ARCH" = "sh" select HAVE_KPROBES select HAVE_KRETPROBES select HAVE_IOREMAP_PROT if MMU && !X2TLB @@ -76,7 +76,7 @@ config SUPERH32 select HAVE_CC_STACKPROTECTOR config SUPERH64 - def_bool ARCH = "sh64" + def_bool "$ARCH" = "sh64" select HAVE_EXIT_THREAD select KALLSYMS diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig index 8767e45..86b852e 100644 --- a/arch/sparc/Kconfig +++ b/arch/sparc/Kconfig @@ -1,6 +1,6 @@ config 64BIT - bool "64-bit kernel" if ARCH = "sparc" - default ARCH = "sparc64" + bool "64-bit kernel" if "$ARCH" = "sparc" + default "$ARCH" = "sparc64" help SPARC is a family of RISC microprocessors designed and marketed by Sun Microsystems, incorporated. They are very widely found in Sun diff --git a/arch/tile/Kconfig b/arch/tile/Kconfig index ef9d403..acc2182 100644 --- a/arch/tile/Kconfig +++ b/arch/tile/Kconfig @@ -119,7 +119,7 @@ config HVC_TILE # Building with ARCH=tilegx (or ARCH=tile) implies using the # 64-bit TILE-Gx toolchain, so force CONFIG_TILEGX on. config TILEGX - def_bool ARCH != "tilepro" + def_bool "$ARCH" != "tilepro" select ARCH_SUPPORTS_ATOMIC_RMW select GENERIC_IRQ_LEGACY_ALLOC_HWIRQ select HAVE_ARCH_JUMP_LABEL diff --git a/arch/um/Kconfig.common b/arch/um/Kconfig.common index c68add8..07f84c8 100644 --- a/arch/um/Kconfig.common +++ b/arch/um/Kconfig.common @@ -54,10 +54,6 @@ config HZ int default 100 -config SUBARCH - string - option env="SUBARCH" - config NR_CPUS int range 1 1 diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 0fa71a7..986fb0a 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1,8 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 # Select 32 or 64 bit config 64BIT - bool "64-bit kernel" if ARCH = "x86" - default ARCH != "i386" + bool "64-bit kernel" if "$ARCH" = "x86" + default "$ARCH" != "i386" ---help--- Say yes to build a 64-bit kernel - formerly known as x86_64 Say no to build a 32-bit kernel - formerly known as i386 diff --git a/arch/x86/um/Kconfig b/arch/x86/um/Kconfig index 13ed827..d355413 100644 --- a/arch/x86/um/Kconfig +++ b/arch/x86/um/Kconfig @@ -16,8 +16,8 @@ config UML_X86 select GENERIC_FIND_FIRST_BIT config 64BIT - bool "64-bit kernel" if SUBARCH = "x86" - default SUBARCH != "i386" + bool "64-bit kernel" if $SUBARCH = "x86" + default $SUBARCH != "i386" config X86_32 def_bool !64BIT diff --git a/init/Kconfig b/init/Kconfig index df18492..b4814e6 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1,11 +1,3 @@ -config ARCH - string - option env="ARCH" - -config KERNELVERSION - string - option env="KERNELVERSION" - config DEFCONFIG_LIST string depends on !UML @@ -13,7 +5,7 @@ config DEFCONFIG_LIST default "/lib/modules/$UNAME_RELEASE/.config" default "/etc/kernel-config" default "/boot/config-$UNAME_RELEASE" - default "$ARCH_DEFCONFIG" + default ARCH_DEFCONFIG default "arch/$ARCH/defconfig" config CONSTRUCTORS diff --git a/scripts/kconfig/confdata.c b/scripts/kconfig/confdata.c index df26c7b..98c2014 100644 --- a/scripts/kconfig/confdata.c +++ b/scripts/kconfig/confdata.c @@ -81,39 +81,13 @@ const char *conf_get_autoconfig_name(void) return name ? name : "include/config/auto.conf"; } -static char *conf_expand_value(const char *in) -{ - struct symbol *sym; - const char *src; - static char res_value[SYMBOL_MAXLENGTH]; - char *dst, name[SYMBOL_MAXLENGTH]; - - res_value[0] = 0; - dst = name; - while ((src = strchr(in, '$'))) { - strncat(res_value, in, src - in); - src++; - dst = name; - while (isalnum(*src) || *src == '_') - *dst++ = *src++; - *dst = 0; - sym = sym_lookup(name, 0); - sym_calc_value(sym); - strcat(res_value, sym_get_string_value(sym)); - in = src; - } - strcat(res_value, in); - - return res_value; -} - char *conf_get_default_confname(void) { struct stat buf; static char fullname[PATH_MAX+1]; char *env, *name; - name = conf_expand_value(conf_defname); + name = expand_string_value(conf_defname); env = getenv(SRCTREE); if (env) { sprintf(fullname, "%s/%s", env, name); @@ -274,7 +248,8 @@ int conf_read_simple(const char *name, int def) if (expr_calc_value(prop->visible.expr) == no || prop->expr->type != E_SYMBOL) continue; - name = conf_expand_value(prop->expr->left.sym->name); + sym_calc_value(prop->expr->left.sym); + name = sym_get_string_value(prop->expr->left.sym); in = zconf_fopen(name); if (in) { conf_message(_("using defaults found in %s"), diff --git a/scripts/kconfig/env.c b/scripts/kconfig/env.c new file mode 100644 index 0000000..9702f5c --- /dev/null +++ b/scripts/kconfig/env.c @@ -0,0 +1,95 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// Copyright (C) 2018 Masahiro Yamada + +static LIST_HEAD(env_list); + +struct env { + char *name; + char *value; + struct list_head node; +}; + +static struct env *env_list_lookup(const char *name) +{ + struct env *e; + + list_for_each_entry(e, &env_list, node) { + if (!strcmp(name, e->name)) + return e; + } + + return NULL; +} + +static void env_list_add(const char *name, const char *value) +{ + struct env *e; + + e = xmalloc(sizeof(*e)); + e->name = xstrdup(name); + e->value = xstrdup(value); + + list_add_tail(&e->node, &env_list); +} + +static void env_list_del(struct env *e) +{ + list_del(&e->node); + free(e->name); + free(e->value); + free(e); +} + +/* the returned pointer must be freed when done */ +static char *env_expand(const char *name) +{ + struct env *e; + const char *value; + + e = env_list_lookup(name); + if (e) + return xstrdup(e->value); + + value = getenv(name); + if (!value) { + fprintf(stderr, "environment variable \"%s\" undefined\n", name); + value = ""; + } + + /* + * we need to remember all referenced environments. + * They will be written out to include/config/auto.conf.cmd + */ + env_list_add(name, value); + + return xstrdup(value); +} + +/* works like env_expand, but 'name' does not need to be null-terminated */ +char *env_expand_n(const char *name, size_t n) +{ + char *tmp, *res; + + tmp = xmalloc(n + 1); + memcpy(tmp, name, n); + *(tmp + n) = '\0'; + + res = env_expand(tmp); + + free(tmp); + + return res; +} + +void env_write_dep(FILE *f, const char *autoconfig_name) +{ + struct env *env, *tmp; + + list_for_each_entry_safe(env, tmp, &env_list, node) { + fprintf(f, "ifneq \"$(%s)\" \"%s\"\n", env->name, env->value); + fprintf(f, "%s: FORCE\n", autoconfig_name); + fprintf(f, "endif\n"); + env_list_del(env); + } +} diff --git a/scripts/kconfig/kconf_id.c b/scripts/kconfig/kconf_id.c index 3ea9c5f..b3e0ea0 100644 --- a/scripts/kconfig/kconf_id.c +++ b/scripts/kconfig/kconf_id.c @@ -32,7 +32,6 @@ static struct kconf_id kconf_id_array[] = { { "on", T_ON, TF_PARAM }, { "modules", T_OPT_MODULES, TF_OPTION }, { "defconfig_list", T_OPT_DEFCONFIG_LIST, TF_OPTION }, - { "env", T_OPT_ENV, TF_OPTION }, { "allnoconfig_y", T_OPT_ALLNOCONFIG_Y, TF_OPTION }, }; diff --git a/scripts/kconfig/lkc.h b/scripts/kconfig/lkc.h index c8d9e55..03d007f 100644 --- a/scripts/kconfig/lkc.h +++ b/scripts/kconfig/lkc.h @@ -58,7 +58,6 @@ enum conf_def_mode { #define T_OPT_MODULES 1 #define T_OPT_DEFCONFIG_LIST 2 -#define T_OPT_ENV 3 #define T_OPT_ALLNOCONFIG_Y 4 struct kconf_id { @@ -95,6 +94,10 @@ static inline void xfwrite(const void *str, size_t len, size_t count, FILE *out) fprintf(stderr, "Error in writing or end of file.\n"); } +/* env.c */ +char *env_expand_n(const char *name, size_t n); +void env_write_dep(FILE *f, const char *auto_conf_name); + /* menu.c */ void _menu_init(void); void menu_warn(struct menu *menu, const char *fmt, ...); @@ -135,9 +138,6 @@ void str_printf(struct gstr *gs, const char *fmt, ...); const char *str_get(struct gstr *gs); /* symbol.c */ -extern struct expr *sym_env_list; - -void sym_init(void); void sym_clear_all_valid(void); struct symbol *sym_choice_default(struct symbol *sym); const char *sym_get_string_default(struct symbol *sym); diff --git a/scripts/kconfig/menu.c b/scripts/kconfig/menu.c index 5c5c137..8148305 100644 --- a/scripts/kconfig/menu.c +++ b/scripts/kconfig/menu.c @@ -214,9 +214,6 @@ void menu_add_option(int token, char *arg) zconf_error("trying to redefine defconfig symbol"); sym_defconfig_list->flags |= SYMBOL_AUTO; break; - case T_OPT_ENV: - prop_add_env(arg); - break; case T_OPT_ALLNOCONFIG_Y: current_entry->sym->flags |= SYMBOL_ALLNOCONFIG_Y; break; diff --git a/scripts/kconfig/symbol.c b/scripts/kconfig/symbol.c index 03143b2..7c9a88e 100644 --- a/scripts/kconfig/symbol.c +++ b/scripts/kconfig/symbol.c @@ -33,33 +33,6 @@ struct symbol *sym_defconfig_list; struct symbol *modules_sym; tristate modules_val; -struct expr *sym_env_list; - -static void sym_add_default(struct symbol *sym, const char *def) -{ - struct property *prop = prop_alloc(P_DEFAULT, sym); - - prop->expr = expr_alloc_symbol(sym_lookup(def, SYMBOL_CONST)); -} - -void sym_init(void) -{ - struct symbol *sym; - struct utsname uts; - static bool inited = false; - - if (inited) - return; - inited = true; - - uname(&uts); - - sym = sym_lookup("UNAME_RELEASE", 0); - sym->type = S_STRING; - sym->flags |= SYMBOL_AUTO; - sym_add_default(sym, uts.release); -} - enum symbol_type sym_get_type(struct symbol *sym) { enum symbol_type type = sym->type; @@ -1348,32 +1321,3 @@ const char *prop_get_type_name(enum prop_type type) } return "unknown"; } - -static void prop_add_env(const char *env) -{ - struct symbol *sym, *sym2; - struct property *prop; - char *p; - - sym = current_entry->sym; - sym->flags |= SYMBOL_AUTO; - for_all_properties(sym, prop, P_ENV) { - sym2 = prop_get_symbol(prop); - if (strcmp(sym2->name, env)) - menu_warn(current_entry, "redefining environment symbol from %s", - sym2->name); - return; - } - - prop = prop_alloc(P_ENV, sym); - prop->expr = expr_alloc_symbol(sym_lookup(env, SYMBOL_CONST)); - - sym_env_list = expr_alloc_one(E_LIST, sym_env_list); - sym_env_list->right.sym = sym; - - p = getenv(env); - if (p) - sym_add_default(sym, p); - else - menu_warn(current_entry, "environment variable %s undefined", env); -} diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c index 22201a4..136e497 100644 --- a/scripts/kconfig/util.c +++ b/scripts/kconfig/util.c @@ -8,16 +8,18 @@ #include #include #include + +#include "list.h" #include "lkc.h" /* - * Expand symbol's names embedded in the string given in argument. Symbols' - * name to be expanded shall be prefixed by a '$'. Unknown symbol expands to + * Expand environments embedded in the string given in argument. Environments + * to be expanded shall be prefixed by a '$'. Unknown environment expands to * the empty string. */ char *expand_string_value(const char *in) { - const char *src; + const char *p, *q; char *res; size_t reslen; @@ -25,39 +27,28 @@ char *expand_string_value(const char *in) * Note: 'in' might come from a token that's about to be * freed, so make sure to always allocate a new string */ - reslen = strlen(in) + 1; - res = xmalloc(reslen); - res[0] = '\0'; - - while ((src = strchr(in, '$'))) { - char *p, name[SYMBOL_MAXLENGTH]; - const char *symval = ""; - struct symbol *sym; - size_t newlen; - - strncat(res, in, src - in); - src++; - - p = name; - while (isalnum(*src) || *src == '_') - *p++ = *src++; - *p = '\0'; - - sym = sym_find(name); - if (sym != NULL) { - sym_calc_value(sym); - symval = sym_get_string_value(sym); - } + res = xmalloc(1); + *res = '\0'; - newlen = strlen(res) + strlen(symval) + strlen(src) + 1; - if (newlen > reslen) { - reslen = newlen; - res = xrealloc(res, reslen); - } + while ((p = strchr(in, '$'))) { + char *new; + + q = p + 1; + while (isalnum(*q) || *q == '_') + q++; - strcat(res, symval); - in = src; + new = env_expand_n(p + 1, q - p - 1); + + reslen = strlen(res) + (p - in) + strlen(new) + 1; + res = xrealloc(res, reslen); + strncat(res, in, p - in); + strcat(res, new); + free(new); + in = q; } + + reslen = strlen(res) + strlen(in) + 1; + res = xrealloc(res, reslen); strcat(res, in); return res; @@ -87,8 +78,6 @@ struct file *file_lookup(const char *name) /* write a dependency file as used by kbuild to track dependencies */ int file_write_dep(const char *name) { - struct symbol *sym, *env_sym; - struct expr *e; struct file *file; FILE *out; @@ -107,21 +96,7 @@ int file_write_dep(const char *name) fprintf(out, "\n%s: \\\n" "\t$(deps_config)\n\n", conf_get_autoconfig_name()); - expr_list_for_each_sym(sym_env_list, e, sym) { - struct property *prop; - const char *value; - - prop = sym_get_env_prop(sym); - env_sym = prop_get_symbol(prop); - if (!env_sym) - continue; - value = getenv(env_sym->name); - if (!value) - value = ""; - fprintf(out, "ifneq \"$(%s)\" \"%s\"\n", env_sym->name, value); - fprintf(out, "%s: FORCE\n", conf_get_autoconfig_name()); - fprintf(out, "endif\n"); - } + env_write_dep(out, conf_get_autoconfig_name()); fprintf(out, "\n$(deps_config): ;\n"); fclose(out); diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l index 045093d..551ca47 100644 --- a/scripts/kconfig/zconf.l +++ b/scripts/kconfig/zconf.l @@ -35,6 +35,7 @@ struct buffer *current_buf; static int last_ts, first_ts; +static void expand_string(const char *in); static void zconf_endhelp(void); static void zconf_endfile(void); @@ -120,6 +121,7 @@ n [A-Za-z0-9_-] } { + "$".* expand_string(yytext); "&&" return T_AND; "||" return T_OR; "(" return T_OPEN_PAREN; @@ -157,12 +159,13 @@ n [A-Za-z0-9_-] } { - [^'"\\\n]+/\n { + "$".* expand_string(yytext); + [^$'"\\\n]+/\n { append_string(yytext, yyleng); yylval.string = text; return T_WORD_QUOTE; } - [^'"\\\n]+ { + [^$'"\\\n]+ { append_string(yytext, yyleng); } \\.?/\n { @@ -249,6 +252,19 @@ n [A-Za-z0-9_-] } %% +static void expand_string(const char *in) +{ + char *p, *q; + + p = expand_string_value(in); + + q = p + strlen(p); + while (q > p) + unput(*--q); + + free(p); +} + void zconf_starthelp(void) { new_string(); diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y index 262c464..4ff4ac9 100644 --- a/scripts/kconfig/zconf.y +++ b/scripts/kconfig/zconf.y @@ -534,7 +534,6 @@ void conf_parse(const char *name) zconf_initscan(name); - sym_init(); _menu_init(); if (getenv("ZCONF_DEBUG")) @@ -775,6 +774,7 @@ void zconfdump(FILE *out) } #include "zconf.lex.c" +#include "env.c" #include "util.c" #include "confdata.c" #include "expr.c" From patchwork Tue Mar 27 05:29:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,05/21] kconfig: remove string expansion in file_lookup() X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132450 Message-Id: <1522128575-5326-6-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:19 +0900 From: Masahiro Yamada List-Id: There are two callers of file_lookup(). [1] zconf_initscan() This is used to open the first Kconfig. However, it cannot contain environments in the file path because zconf_fopen() is called before file_lookup(). By swapping the call order, KBUILD_KCONFIG would be able to contain environments, but I do not see practical benefits to support it. [2] zconf_nextfile() This is used to open the next file from 'source' statement like source "arch/$SRCARCH/Kconfig" but this has already been expanded in the lexer phase. So, file_lookup() does not need to expand the given path. By the way, file_lookup() was already buggy; it expanded a given path, but it used the path before expansion for look-up: if (!strcmp(name, file->name)) { Signed-off-by: Masahiro Yamada --- Changes in v2: - Simplify the patch. Just remove text expansion. scripts/kconfig/util.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) -- 2.7.4 Reviewed-by: Kees Cook Reviewed-by: Ulf Magnusson diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c index 136e497..3d27c49 100644 --- a/scripts/kconfig/util.c +++ b/scripts/kconfig/util.c @@ -58,18 +58,16 @@ char *expand_string_value(const char *in) struct file *file_lookup(const char *name) { struct file *file; - char *file_name = expand_string_value(name); for (file = file_list; file; file = file->next) { if (!strcmp(name, file->name)) { - free(file_name); return file; } } file = xmalloc(sizeof(*file)); memset(file, 0, sizeof(*file)); - file->name = file_name; + file->name = xstrdup(name); file->next = file_list; file_list = file; return file; From patchwork Tue Mar 27 05:29:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2, 06/21] kconfig: remove string expansion for mainmenu after yyparse() X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132458 Message-Id: <1522128575-5326-7-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:20 +0900 From: Masahiro Yamada List-Id: Now that environments are expanded in the lexer, conf_parse() does not need to explicitly expand $ARCH and $KERNELVERSION in the mainmenu. The hack introduced by commit 0724a7c32a54 ("kconfig: Don't leak main menus during parsing") can go away. Signed-off-by: Masahiro Yamada --- Changes in v2: - Simplify the patch. Just remove the text expansion. scripts/kconfig/zconf.y | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) -- 2.7.4 Reviewed-by: Kees Cook Reviewed-by: Ulf Magnusson diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y index 4ff4ac9..d8120c7 100644 --- a/scripts/kconfig/zconf.y +++ b/scripts/kconfig/zconf.y @@ -109,7 +109,7 @@ static struct menu *current_menu, *current_entry; %% input: nl start | start; -start: mainmenu_stmt stmt_list | no_mainmenu_stmt stmt_list; +start: mainmenu_stmt stmt_list | stmt_list; /* mainmenu entry */ @@ -118,19 +118,6 @@ mainmenu_stmt: T_MAINMENU prompt nl menu_add_prompt(P_MENU, $2, NULL); }; -/* Default main menu, if there's no mainmenu entry */ - -no_mainmenu_stmt: /* empty */ -{ - /* - * Hack: Keep the main menu title on the heap so we can safely free it - * later regardless of whether it comes from the 'prompt' in - * mainmenu_stmt or here - */ - menu_add_prompt(P_MENU, xstrdup("Linux Kernel Configuration"), NULL); -}; - - stmt_list: /* empty */ | stmt_list common_stmt @@ -528,7 +515,6 @@ word_opt: /* empty */ { $$ = NULL; } void conf_parse(const char *name) { - const char *tmp; struct symbol *sym; int i; @@ -544,10 +530,10 @@ void conf_parse(const char *name) if (!modules_sym) modules_sym = sym_find( "n" ); - tmp = rootmenu.prompt->text; - rootmenu.prompt->text = _(rootmenu.prompt->text); - rootmenu.prompt->text = expand_string_value(rootmenu.prompt->text); - free((char*)tmp); + if (!menu_has_prompt(&rootmenu)) { + current_entry = &rootmenu; + menu_add_prompt(P_MENU, "Linux Kernel Configuration", NULL); + } menu_finalize(&rootmenu); for_all_symbols(i, sym) { From patchwork Tue Mar 27 05:29:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2, 07/21] kconfig: add function support and implement 'shell' function X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132453 Message-Id: <1522128575-5326-8-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:21 +0900 From: Masahiro Yamada List-Id: This commit adds a new concept 'function' to do more text processing in Kconfig. A function call looks like this: $(function arg1, arg2, arg3, ...) (Actually, this syntax was inspired by make.) Real examples will look like this: $(shell echo hello world) $(cc-option -fstackprotector) This commit adds the basic infrastructure to add, delete, evaluate functions, and also the first built-in function $(shell ...). This accepts a single command to execute. It returns the standard output from it. [Example code] config HELLO string default "$(shell echo hello world)" config Y def_bool $(shell echo y) [Result] $ make -s alldefconfig && tail -n 2 .config CONFIG_HELLO="hello world" CONFIG_Y=y Caveat: Like environments, functions are expanded in the lexer. You cannot pass symbols to function arguments. This is a limitation to simplify the implementation. I want to avoid the dynamic function evaluation, which would introduce much more complexity. Signed-off-by: Masahiro Yamada --- Reminder for myself: Update Documentation/kbuild/kconfig-language.txt Changes in v2: - Use 'shell' for getting stdout from the comment. It was 'shell-stdout' in the previous version. - Symplify the implementation since the expansion has been moved to lexer. scripts/kconfig/function.c | 170 ++++++++++++++++++++++++++++++++++++++++++++ scripts/kconfig/lkc_proto.h | 5 ++ scripts/kconfig/util.c | 46 +++++++++--- scripts/kconfig/zconf.y | 9 +++ 4 files changed, 222 insertions(+), 8 deletions(-) create mode 100644 scripts/kconfig/function.c -- 2.7.4 diff --git a/scripts/kconfig/function.c b/scripts/kconfig/function.c new file mode 100644 index 0000000..913685f --- /dev/null +++ b/scripts/kconfig/function.c @@ -0,0 +1,170 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// Copyright (C) 2018 Masahiro Yamada + +#include +#include +#include + +#include "list.h" + +#define FUNCTION_MAX_ARGS 10 + +static LIST_HEAD(function_list); + +struct function { + char *name; + char *(*func)(struct function *f, int argc, char *argv[]); + struct list_head node; +}; + +static struct function *func_lookup(const char *name) +{ + struct function *f; + + list_for_each_entry(f, &function_list, node) { + if (!strcmp(name, f->name)) + return f; + } + + return NULL; +} + +static void func_add(const char *name, + char *(*func)(struct function *f, int argc, char *argv[])) +{ + struct function *f; + + f = func_lookup(name); + if (f) { + fprintf(stderr, "%s: function already exists. ignored.\n", name); + return; + } + + f = xmalloc(sizeof(*f)); + f->name = xstrdup(name); + f->func = func; + + list_add_tail(&f->node, &function_list); +} + +static void func_del(struct function *f) +{ + list_del(&f->node); + free(f->name); + free(f); +} + +static char *func_call(int argc, char *argv[]) +{ + struct function *f; + + f = func_lookup(argv[0]); + if (!f) { + fprintf(stderr, "%s: function not found\n", argv[0]); + return NULL; + } + + return f->func(f, argc, argv); +} + +static char *func_eval(const char *func) +{ + char *expanded, *saveptr, *str, *token, *res; + const char *delim; + int argc = 0; + char *argv[FUNCTION_MAX_ARGS]; + + expanded = expand_string_value(func); + + str = expanded; + delim = " "; + + while ((token = strtok_r(str, delim, &saveptr))) { + argv[argc++] = token; + str = NULL; + delim = ","; + } + + res = func_call(argc, argv); + + free(expanded); + + return res ?: xstrdup(""); +} + +char *func_eval_n(const char *func, size_t n) +{ + char *tmp, *res; + + tmp = xmalloc(n + 1); + memcpy(tmp, func, n); + *(tmp + n) = '\0'; + + res = func_eval(tmp); + + free(tmp); + + return res; +} + +/* built-in functions */ +static char *do_shell(struct function *f, int argc, char *argv[]) +{ + static const char *pre = "("; + static const char *post = ") 2>/dev/null"; + FILE *p; + char buf[256]; + char *cmd; + int ret; + + if (argc != 2) + return NULL; + + /* + * Surround the command with ( ) in case it is piped commands. + * Also, redirect stderr to /dev/null. + */ + cmd = xmalloc(strlen(pre) + strlen(argv[1]) + strlen(post) + 1); + strcpy(cmd, pre); + strcat(cmd, argv[1]); + strcat(cmd, post); + + p = popen(cmd, "r"); + if (!p) { + perror(cmd); + goto free; + } + if (fgets(buf, sizeof(buf), p)) { + size_t len = strlen(buf); + + if (buf[len - 1] == '\n') + buf[len - 1] = '\0'; + } else { + buf[0] = '\0'; + } + + ret = pclose(p); + if (ret == -1) + perror(cmd); + +free: + free(cmd); + + return xstrdup(buf); +} + +void func_init(void) +{ + /* register built-in functions */ + func_add("shell", do_shell); +} + +void func_exit(void) +{ + struct function *f, *tmp; + + /* unregister all functions */ + list_for_each_entry_safe(f, tmp, &function_list, node) + func_del(f); +} diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h index 9884adc..09a4f53 100644 --- a/scripts/kconfig/lkc_proto.h +++ b/scripts/kconfig/lkc_proto.h @@ -48,5 +48,10 @@ const char * sym_get_string_value(struct symbol *sym); const char * prop_get_type_name(enum prop_type type); +/* function.c */ +char *func_eval_n(const char *func, size_t n); +void func_init(void); +void func_exit(void); + /* expr.c */ void expr_print(struct expr *e, void (*fn)(void *, struct symbol *, const char *), void *data, int prevtoken); diff --git a/scripts/kconfig/util.c b/scripts/kconfig/util.c index 3d27c49..218b051 100644 --- a/scripts/kconfig/util.c +++ b/scripts/kconfig/util.c @@ -13,9 +13,10 @@ #include "lkc.h" /* - * Expand environments embedded in the string given in argument. Environments - * to be expanded shall be prefixed by a '$'. Unknown environment expands to - * the empty string. + * Expand environments and functions embedded in the string given in argument. + * Environments to be expanded shall be prefixed by a '$'. Functions to be + * evaluated shall be surrounded by $(). Unknown environment/function expands + * to the empty string. */ char *expand_string_value(const char *in) { @@ -33,11 +34,40 @@ char *expand_string_value(const char *in) while ((p = strchr(in, '$'))) { char *new; - q = p + 1; - while (isalnum(*q) || *q == '_') - q++; - - new = env_expand_n(p + 1, q - p - 1); + /* + * If the next character is '(', it is a function. + * Otherwise, environment. + */ + if (*(p + 1) == '(') { + int nest = 0; + + q = p + 2; + while (1) { + if (*q == '\0') { + fprintf(stderr, + "unterminated function: %s\n", + p); + new = xstrdup(""); + break; + } else if (*q == '(') { + nest++; + } else if (*q == ')') { + if (nest-- == 0) { + new = func_eval_n(p + 2, + q - p - 2); + q++; + break; + } + } + q++; + } + } else { + q = p + 1; + while (isalnum(*q) || *q == '_') + q++; + + new = env_expand_n(p + 1, q - p - 1); + } reslen = strlen(res) + (p - in) + strlen(new) + 1; res = xrealloc(res, reslen); diff --git a/scripts/kconfig/zconf.y b/scripts/kconfig/zconf.y index d8120c7..feaea18 100644 --- a/scripts/kconfig/zconf.y +++ b/scripts/kconfig/zconf.y @@ -520,11 +520,19 @@ void conf_parse(const char *name) zconf_initscan(name); + func_init(); _menu_init(); if (getenv("ZCONF_DEBUG")) yydebug = 1; yyparse(); + + /* + * Currently, functions are evaluated only when Kconfig files are + * parsed. We can free functions here. + */ + func_exit(); + if (yynerrs) exit(1); if (!modules_sym) @@ -765,4 +773,5 @@ void zconfdump(FILE *out) #include "confdata.c" #include "expr.c" #include "symbol.c" +#include "function.c" #include "menu.c" From patchwork Tue Mar 27 05:29:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,08/21] kconfig: replace $UNAME_RELEASE with function call X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132457 Message-Id: <1522128575-5326-9-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:22 +0900 From: Masahiro Yamada List-Id: Now that 'shell' function is supported, this can be self-contained in Kconfig. Signed-off-by: Masahiro Yamada --- Changes in v2: None Makefile | 3 +-- init/Kconfig | 4 ++-- 2 files changed, 3 insertions(+), 4 deletions(-) -- 2.7.4 Reviewed-by: Kees Cook Reviewed-by: Ulf Magnusson diff --git a/Makefile b/Makefile index 4ae1486..5c395ed 100644 --- a/Makefile +++ b/Makefile @@ -284,8 +284,7 @@ include scripts/Kbuild.include # Read KERNELRELEASE from include/config/kernel.release (if it exists) KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null) KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION) -UNAME_RELEASE := $(shell uname --release) -export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION UNAME_RELEASE +export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION # SUBARCH tells the usermode build what the underlying arch is. That is set # first, and if a usermode build is happening, the "ARCH=um" on the command diff --git a/init/Kconfig b/init/Kconfig index b4814e6..0d6fe20 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2,9 +2,9 @@ config DEFCONFIG_LIST string depends on !UML option defconfig_list - default "/lib/modules/$UNAME_RELEASE/.config" + default "/lib/modules/$(shell uname --release)/.config" default "/etc/kernel-config" - default "/boot/config-$UNAME_RELEASE" + default "/boot/config-$(shell uname --release)" default ARCH_DEFCONFIG default "arch/$ARCH/defconfig" From patchwork Tue Mar 27 05:29:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2, 09/21] kconfig: add 'macro' keyword to support user-defined function X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132448 Message-Id: <1522128575-5326-10-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:23 +0900 From: Masahiro Yamada List-Id: Now, we got a basic ability to test compiler capability in Kconfig. config CC_HAS_STACKPROTECTOR def_bool $(shell (($CC -Werror -fstack-protector -c -x c /dev/null -o /dev/null) && echo y) || echo n) This works, but it is ugly to repeat this long boilerplate. We want to describe like this: config CC_HAS_STACKPROTECTOR bool default $(cc-option -fstack-protector) It is straight-forward to add a new function, but I do not like to hard-code specialized functions like this. Hence, here is another feature to add functions from Kconfig files. A user-defined function is defined with a special keyword 'macro'. It can be referenced in the same way as built-in functions. This feature was also inspired by Makefile where user-defined functions are referenced by $(call func-name, args...), but I omitted the 'call' to makes it shorter. The macro definition can contain $(1), $(2), ... which will be replaced with arguments from the caller. The macro works just as a textual shorthand, which is also expanded in the lexer phase. [Example Code] macro success $(shell ($(1) && echo y) || echo n) config TRUE bool "true" default $(success true) config FALSE bool "false" default $(success false) [Result] $ make -s alldefconfig $ tail -n 2 .config CONFIG_TRUE=y # CONFIG_FALSE is not set [Example Code] macro success $(shell ($(1) && echo y) || echo n) macro cc-option $(success $CC -Werror $(1) -c -x c /dev/null -o /dev/null) config CC_HAS_STACKPROTECTOR def_bool $(cc-option -fstack-protector) [Result] $ make -s alldefconfig $ tail -n 1 .config CONFIG_CC_HAS_STACKPROTECTOR=y Signed-off-by: Masahiro Yamada --- Reminder for myself: Update Documentation/kbuild/kconfig-language.txt Changes in v2: - Use 'macro' directly instead of inside the string type symbol. scripts/kconfig/function.c | 59 +++++++++++++++++++++++++++++++++++++++++++-- scripts/kconfig/lkc_proto.h | 1 + scripts/kconfig/zconf.l | 31 ++++++++++++++++++++++++ 3 files changed, 89 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/scripts/kconfig/function.c b/scripts/kconfig/function.c index 913685f..389bb44 100644 --- a/scripts/kconfig/function.c +++ b/scripts/kconfig/function.c @@ -15,6 +15,7 @@ static LIST_HEAD(function_list); struct function { char *name; char *(*func)(struct function *f, int argc, char *argv[]); + char *macro; struct list_head node; }; @@ -31,7 +32,8 @@ static struct function *func_lookup(const char *name) } static void func_add(const char *name, - char *(*func)(struct function *f, int argc, char *argv[])) + char *(*func)(struct function *f, int argc, char *argv[]), + const char *macro) { struct function *f; @@ -44,6 +46,7 @@ static void func_add(const char *name, f = xmalloc(sizeof(*f)); f->name = xstrdup(name); f->func = func; + f->macro = macro ? xstrdup(macro) : NULL; list_add_tail(&f->node, &function_list); } @@ -51,6 +54,7 @@ static void func_add(const char *name, static void func_del(struct function *f) { list_del(&f->node); + free(f->macro); free(f->name); free(f); } @@ -108,6 +112,57 @@ char *func_eval_n(const char *func, size_t n) return res; } +/* run user-defined function */ +static char *do_macro(struct function *f, int argc, char *argv[]) +{ + char *new; + char *src, *p, *res; + size_t newlen; + int n; + + new = xmalloc(1); + *new = 0; + + /* + * This is a format string. $(1), $(2), ... must be replaced with + * function arguments. + */ + src = f->macro; + p = src; + + while ((p = strstr(p, "$("))) { + if (isdigit(p[2]) && p[3] == ')') { + n = p[2] - '0'; + if (n < argc) { + newlen = strlen(new) + (p - src) + + strlen(argv[n]) + 1; + new = xrealloc(new, newlen); + strncat(new, src, p - src); + strcat(new, argv[n]); + src = p + 4; + } + p += 2; + } + p += 2; + } + + newlen = strlen(new) + strlen(src) + 1; + new = xrealloc(new, newlen); + strcat(new, src); + + res = expand_string_value(new); + + free(new); + + return res; +} + +/* add user-defined function (macro) */ +void func_add_macro(const char *name, const char *macro) +{ + func_add(name, do_macro, macro); +} + /* built-in functions */ static char *do_shell(struct function *f, int argc, char *argv[]) { @@ -157,7 +212,7 @@ static char *do_shell(struct function *f, int argc, char *argv[]) void func_init(void) { /* register built-in functions */ - func_add("shell", do_shell); + func_add("shell", do_shell, NULL); } void func_exit(void) diff --git a/scripts/kconfig/lkc_proto.h b/scripts/kconfig/lkc_proto.h index 09a4f53..48699c0 100644 --- a/scripts/kconfig/lkc_proto.h +++ b/scripts/kconfig/lkc_proto.h @@ -50,6 +50,7 @@ const char * prop_get_type_name(enum prop_type type); /* function.c */ char *func_eval_n(const char *func, size_t n); +void func_add_macro(const char *name, const char *macro); void func_init(void); void func_exit(void); diff --git a/scripts/kconfig/zconf.l b/scripts/kconfig/zconf.l index 551ca47..6a18c68 100644 --- a/scripts/kconfig/zconf.l +++ b/scripts/kconfig/zconf.l @@ -74,6 +74,36 @@ static void warn_ignored_character(char chr) "%s:%d:warning: ignoring unsupported character '%c'\n", zconf_curname(), zconf_lineno(), chr); } + +static void handle_macro(const char *text) +{ + char *p, *q; + + while (isspace(*text)) + text++; + + p = xstrdup(text); + + q = p; + while (isalnum(*q) || *q == '_' || *q == '-') + q++; + + if (q == p || !*q) { + yyerror("invalid\n"); + goto free; + } + + *q = '\0'; + + q++; + while (isspace(*q)) + q++; + + func_add_macro(p, q); +free: + free(p); +} + %} n [A-Za-z0-9_-] @@ -82,6 +112,7 @@ n [A-Za-z0-9_-] int str = 0; int ts, i; +"macro"[ \t].* handle_macro(yytext + 6); [ \t]*#.*\n | [ \t]*\n { return T_EOL; From patchwork Tue Mar 27 05:29:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,10/21] kconfig: add 'success' and 'cc-option' macros X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132437 Message-Id: <1522128575-5326-11-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:24 +0900 From: Masahiro Yamada List-Id: 'cc-option' will be the most frequently used macro. It evaluates to 'y' if the given argument is supported by the compiler, or 'n' otherwise. Signed-off-by: Masahiro Yamada --- Changes in v2: - Implement 'success' as a macro init/Kconfig | 6 ++++++ 1 file changed, 6 insertions(+) -- 2.7.4 Reviewed-by: Kees Cook Reviewed-by: Ulf Magnusson diff --git a/init/Kconfig b/init/Kconfig index 0d6fe20..c456030 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -8,6 +8,12 @@ config DEFCONFIG_LIST default ARCH_DEFCONFIG default "arch/$ARCH/defconfig" +# expanded to y if the command exits with 0, n otherwise +macro success $(shell ($(1) && echo y) || echo n) + +# expanded to y if the given compiler flag is supported, n otherwise +macro cc-option $(success $CC -Werror $(1) -c -x c /dev/null -o /dev/null) + config CONSTRUCTORS bool depends on !UML From patchwork Tue Mar 27 05:29:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2, 11/21] stack-protector: test compiler capability in Kconfig and drop AUTO mode X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132438 Message-Id: <1522128575-5326-12-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org, Ingo Molnar Date: Tue, 27 Mar 2018 14:29:25 +0900 From: Masahiro Yamada List-Id: Move the test for -fstack-protector(-strong) option to Kconfig. If the compiler does not support the option, the corresponding menu is automatically hidden. If _STRONG is not supported, it will fall back to _REGULAR. If _REGULAR is not supported, it will be disabled. This means, _AUTO is implicitly handled by the dependency solver of Kconfig, hence removed. I also turned the 'choice' into only two boolean symbols. The use of 'choice' is not a good idea here, because all of all{yes,mod,no}config would choose the first visible value, while we want allnoconfig to disable as many features as possible. X86 has additional shell scripts in case the compiler supports the option, but generates broken code. I added CC_HAS_SANE_STACKPROTECTOR to test this. I had to add -m32 to gcc-x86_32-has-stack-protector.sh to make it work correctly. Signed-off-by: Masahiro Yamada --- Changes in v2: - Describe $(cc-option ...) directly in depends on context Makefile | 93 ++----------------------------- arch/Kconfig | 29 +++------- arch/x86/Kconfig | 8 ++- scripts/gcc-x86_32-has-stack-protector.sh | 7 +-- scripts/gcc-x86_64-has-stack-protector.sh | 5 -- 5 files changed, 22 insertions(+), 120 deletions(-) -- 2.7.4 diff --git a/Makefile b/Makefile index 5c395ed..5cadffa 100644 --- a/Makefile +++ b/Makefile @@ -675,55 +675,11 @@ ifneq ($(CONFIG_FRAME_WARN),0) KBUILD_CFLAGS += $(call cc-option,-Wframe-larger-than=${CONFIG_FRAME_WARN}) endif -# This selects the stack protector compiler flag. Testing it is delayed -# until after .config has been reprocessed, in the prepare-compiler-check -# target. -ifdef CONFIG_CC_STACKPROTECTOR_AUTO - stackp-flag := $(call cc-option,-fstack-protector-strong,$(call cc-option,-fstack-protector)) - stackp-name := AUTO -else -ifdef CONFIG_CC_STACKPROTECTOR_REGULAR - stackp-flag := -fstack-protector - stackp-name := REGULAR -else -ifdef CONFIG_CC_STACKPROTECTOR_STRONG - stackp-flag := -fstack-protector-strong - stackp-name := STRONG -else - # If either there is no stack protector for this architecture or - # CONFIG_CC_STACKPROTECTOR_NONE is selected, we're done, and $(stackp-name) - # is empty, skipping all remaining stack protector tests. - # - # Force off for distro compilers that enable stack protector by default. - KBUILD_CFLAGS += $(call cc-option, -fno-stack-protector) -endif -endif -endif -# Find arch-specific stack protector compiler sanity-checking script. -ifdef stackp-name -ifneq ($(stackp-flag),) - stackp-path := $(srctree)/scripts/gcc-$(SRCARCH)_$(BITS)-has-stack-protector.sh - stackp-check := $(wildcard $(stackp-path)) - # If the wildcard test matches a test script, run it to check functionality. - ifdef stackp-check - ifneq ($(shell $(CONFIG_SHELL) $(stackp-check) $(CC) $(KBUILD_CPPFLAGS) $(biarch)),y) - stackp-broken := y - endif - endif - ifndef stackp-broken - # If the stack protector is functional, enable code that depends on it. - KBUILD_CPPFLAGS += -DCONFIG_CC_STACKPROTECTOR - # Either we've already detected the flag (for AUTO) or we'll fail the - # build in the prepare-compiler-check rule (for specific flag). - KBUILD_CFLAGS += $(stackp-flag) - else - # We have to make sure stack protector is unconditionally disabled if - # the compiler is broken (in case we're going to continue the build in - # AUTO mode). - KBUILD_CFLAGS += $(call cc-option, -fno-stack-protector) - endif -endif -endif +stackp-flags-y := -fno-stack-protector +stackp-flags-$(CONFIG_CC_STACKPROTECTOR) := -fstack-protector +stackp-flags-$(CONFIG_CC_STACKPROTECTOR_STRONG) := -fstack-protector-strong + +KBUILD_CFLAGS += $(stackp-flags-y) ifeq ($(cc-name),clang) KBUILD_CPPFLAGS += $(call cc-option,-Qunused-arguments,) @@ -1092,7 +1048,7 @@ endif # prepare2 creates a makefile if using a separate output directory. # From this point forward, .config has been reprocessed, so any rules # that need to depend on updated CONFIG_* values can be checked here. -prepare2: prepare3 prepare-compiler-check outputmakefile asm-generic +prepare2: prepare3 outputmakefile asm-generic prepare1: prepare2 $(version_h) $(autoksyms_h) include/generated/utsrelease.h \ include/config/auto.conf @@ -1118,43 +1074,6 @@ uapi-asm-generic: PHONY += prepare-objtool prepare-objtool: $(objtool_target) -# Check for CONFIG flags that require compiler support. Abort the build -# after .config has been processed, but before the kernel build starts. -# -# For security-sensitive CONFIG options, we don't want to fallback and/or -# silently change which compiler flags will be used, since that leads to -# producing kernels with different security feature characteristics -# depending on the compiler used. (For example, "But I selected -# CC_STACKPROTECTOR_STRONG! Why did it build with _REGULAR?!") -PHONY += prepare-compiler-check -prepare-compiler-check: FORCE -# Make sure compiler supports requested stack protector flag. -ifdef stackp-name - # Warn about CONFIG_CC_STACKPROTECTOR_AUTO having found no option. - ifeq ($(stackp-flag),) - @echo CONFIG_CC_STACKPROTECTOR_$(stackp-name): \ - Compiler does not support any known stack-protector >&2 - else - # Fail if specifically requested stack protector is missing. - ifeq ($(call cc-option, $(stackp-flag)),) - @echo Cannot use CONFIG_CC_STACKPROTECTOR_$(stackp-name): \ - $(stackp-flag) not supported by compiler >&2 && exit 1 - endif - endif -endif -# Make sure compiler does not have buggy stack-protector support. If a -# specific stack-protector was requested, fail the build, otherwise warn. -ifdef stackp-broken - ifeq ($(stackp-name),AUTO) - @echo CONFIG_CC_STACKPROTECTOR_$(stackp-name): \ - $(stackp-flag) available but compiler is broken: disabling >&2 - else - @echo Cannot use CONFIG_CC_STACKPROTECTOR_$(stackp-name): \ - $(stackp-flag) available but compiler is broken >&2 && exit 1 - endif -endif - @: - # Generate some files # --------------------------------------------------------------------------- diff --git a/arch/Kconfig b/arch/Kconfig index 8e0d665..b42378d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -535,13 +535,13 @@ config HAVE_CC_STACKPROTECTOR bool help An arch should select this symbol if: - - its compiler supports the -fstack-protector option - it has implemented a stack canary (e.g. __stack_chk_guard) -choice - prompt "Stack Protector buffer overflow detection" +config CC_STACKPROTECTOR + bool "Stack Protector buffer overflow detection" depends on HAVE_CC_STACKPROTECTOR - default CC_STACKPROTECTOR_AUTO + depends on $(cc-option -fstack-protector) + default y help This option turns on the "stack-protector" GCC feature. This feature puts, at the beginning of functions, a canary value on @@ -551,14 +551,6 @@ choice overwrite the canary, which gets detected and the attack is then neutralized via a kernel panic. -config CC_STACKPROTECTOR_NONE - bool "None" - help - Disable "stack-protector" GCC feature. - -config CC_STACKPROTECTOR_REGULAR - bool "Regular" - help Functions will have the stack-protector canary logic added if they have an 8-byte or larger character array on the stack. @@ -570,7 +562,10 @@ config CC_STACKPROTECTOR_REGULAR by about 0.3%. config CC_STACKPROTECTOR_STRONG - bool "Strong" + bool "Strong Stack Protector" + depends on CC_STACKPROTECTOR + depends on $(cc-option -fstack-protector-strong) + default y help Functions will have the stack-protector canary logic added in any of the following conditions: @@ -588,14 +583,6 @@ config CC_STACKPROTECTOR_STRONG about 20% of all kernel functions, which increases the kernel code size by about 2%. -config CC_STACKPROTECTOR_AUTO - bool "Automatic" - help - If the compiler supports it, the best available stack-protector - option will be chosen. - -endchoice - config LD_DEAD_CODE_DATA_ELIMINATION bool help diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 986fb0a..946bd9b 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -124,7 +124,7 @@ config X86 select HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD if X86_64 select HAVE_ARCH_VMAP_STACK if X86_64 select HAVE_ARCH_WITHIN_STACK_FRAMES - select HAVE_CC_STACKPROTECTOR + select HAVE_CC_STACKPROTECTOR if CC_HAS_SANE_STACKPROTECTOR select HAVE_CMPXCHG_DOUBLE select HAVE_CMPXCHG_LOCAL select HAVE_CONTEXT_TRACKING if X86_64 @@ -340,6 +340,12 @@ config PGTABLE_LEVELS default 2 source "init/Kconfig" + +config CC_HAS_SANE_STACKPROTECTOR + bool + default $(success $srctree/scripts/gcc-x86_64-has-stack-protector.sh $CC) if 64BIT + default $(success $srctree/scripts/gcc-x86_32-has-stack-protector.sh $CC) + source "kernel/Kconfig.freezer" menu "Processor type and features" diff --git a/scripts/gcc-x86_32-has-stack-protector.sh b/scripts/gcc-x86_32-has-stack-protector.sh index 6b2aeef..f5c1194 100755 --- a/scripts/gcc-x86_32-has-stack-protector.sh +++ b/scripts/gcc-x86_32-has-stack-protector.sh @@ -1,9 +1,4 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 -echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -O0 -fstack-protector - -o - 2> /dev/null | grep -q "%gs" -if [ "$?" -eq "0" ] ; then - echo y -else - echo n -fi +echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -m32 -O0 -fstack-protector - -o - 2> /dev/null | grep -q "%gs" diff --git a/scripts/gcc-x86_64-has-stack-protector.sh b/scripts/gcc-x86_64-has-stack-protector.sh index 4a48bdc..3755af0 100755 --- a/scripts/gcc-x86_64-has-stack-protector.sh +++ b/scripts/gcc-x86_64-has-stack-protector.sh @@ -2,8 +2,3 @@ # SPDX-License-Identifier: GPL-2.0 echo "int foo(void) { char X[200]; return 3; }" | $* -S -x c -c -O0 -mcmodel=kernel -fno-PIE -fstack-protector - -o - 2> /dev/null | grep -q "%gs" -if [ "$?" -eq "0" ] ; then - echo y -else - echo n -fi From patchwork Tue Mar 27 05:29:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,12/21] kconfig: show compiler version text in the top comment X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132449 Message-Id: <1522128575-5326-13-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:26 +0900 From: Masahiro Yamada List-Id: The kernel configuration phase is now tightly coupled with the compiler in use. It will be nice to show the compiler information in Kconfig. The compiler information will be displayed like this: $ make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- config scripts/kconfig/conf --oldaskconfig Kconfig * * Linux/arm64 4.16.0-rc1 Kernel Configuration * * * Compiler: aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011 * * * General setup * Compile also drivers which will not load (COMPILE_TEST) [N/y/?] If you use GUI methods such as menuconfig, it will be displayed in the top menu. This is simply implemented by using 'comment'. So, it will be saved into the .config file as well. This commit has a very important meaning. If the compiler is upgraded, Kconfig must be re-run since different compilers have different sets of supported options. All referenced environments are written to include/config/auto.conf.cmd so that any environment change triggers syncconfig, and prompt the user to input new values if needed. With this commit, something like follows will be added to include/config/auto.conf.cmd ifneq "$(CC_VERSION_TEXT)" "aarch64-linux-gnu-gcc (Linaro GCC 7.2-2017.11) 7.2.1 20171011" include/config/auto.conf: FORCE endif Signed-off-by: Masahiro Yamada --- Changes in v2: None Kconfig | 2 ++ Makefile | 2 ++ 2 files changed, 4 insertions(+) -- 2.7.4 Reviewed-by: Kees Cook diff --git a/Kconfig b/Kconfig index e6ece5b..99ed4b8 100644 --- a/Kconfig +++ b/Kconfig @@ -5,4 +5,6 @@ # mainmenu "Linux/$ARCH $KERNELVERSION Kernel Configuration" +comment "Compiler: $CC_VERSION_TEXT" + source "arch/$SRCARCH/Kconfig" diff --git a/Makefile b/Makefile index 5cadffa..3dc9eb8 100644 --- a/Makefile +++ b/Makefile @@ -450,6 +450,8 @@ export KBUILD_AFLAGS_MODULE KBUILD_CFLAGS_MODULE KBUILD_LDFLAGS_MODULE export KBUILD_AFLAGS_KERNEL KBUILD_CFLAGS_KERNEL export KBUILD_ARFLAGS +export CC_VERSION_TEXT := $(shell $(CC) --version | head -n 1) + # When compiling out-of-tree modules, put MODVERDIR in the module # tree rather than in the kernel tree. The kernel tree might # even be read-only. From patchwork Tue Mar 27 05:29:27 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,13/21] kconfig: add CC_IS_GCC and GCC_VERSION X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132454 Message-Id: <1522128575-5326-14-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:27 +0900 From: Masahiro Yamada List-Id: This will be useful to specify the required compiler version, like this: config FOO bool "Use Foo" depends on GCC_VERSION >= 408000 help This feature requires GCC 4.8 or newer. Signed-off-by: Masahiro Yamada --- Changes in v2: None init/Kconfig | 8 ++++++++ 1 file changed, 8 insertions(+) -- 2.7.4 Reviewed-by: Kees Cook diff --git a/init/Kconfig b/init/Kconfig index c456030..923cab5 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -14,6 +14,14 @@ macro success $(shell ($(1) && echo y) || echo n) # expanded to y if the given compiler flag is supported, n otherwise macro cc-option $(success $CC -Werror $(1) -c -x c /dev/null -o /dev/null) +config CC_IS_GCC + def_bool $(success $CC --version | grep -q gcc) + +config GCC_VERSION + int + default $(shell $srctree/scripts/gcc-version.sh -p $CC | sed 's/^0*//') if CC_IS_GCC + default 0 + config CONSTRUCTORS bool depends on !UML From patchwork Tue Mar 27 05:29:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,14/21] kconfig: add CC_IS_CLANG and CLANG_VERSION X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132447 Message-Id: <1522128575-5326-15-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:28 +0900 From: Masahiro Yamada List-Id: This will be useful to describe the clang version dependency. Signed-off-by: Masahiro Yamada --- Changes in v2: None init/Kconfig | 7 +++++++ scripts/clang-version.sh | 24 +++++++++++------------- 2 files changed, 18 insertions(+), 13 deletions(-) -- 2.7.4 Reviewed-by: Kees Cook diff --git a/init/Kconfig b/init/Kconfig index 923cab5..5998034 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -22,6 +22,13 @@ config GCC_VERSION default $(shell $srctree/scripts/gcc-version.sh -p $CC | sed 's/^0*//') if CC_IS_GCC default 0 +config CC_IS_CLANG + def_bool $(success $CC --version | grep -q clang) + +config CLANG_VERSION + int + default $(shell $srctree/scripts/clang-version.sh $CC) + config CONSTRUCTORS bool depends on !UML diff --git a/scripts/clang-version.sh b/scripts/clang-version.sh index 9780efa..d8002d2 100755 --- a/scripts/clang-version.sh +++ b/scripts/clang-version.sh @@ -10,24 +10,22 @@ # clang-5.0.1 etc. # -if [ "$1" = "-p" ] ; then - with_patchlevel=1; - shift; -fi - compiler="$*" if [ ${#compiler} -eq 0 ]; then - echo "Error: No compiler specified." - printf "Usage:\n\t$0 \n" + echo "Error: No compiler specified." >&2 + printf "Usage:\n\t$0 \n" >&2 + echo 0 + exit 1 +fi + +if !( $compiler --version | grep -q clang) ; then + echo "This is not clang." >&2 + echo 0 exit 1 fi MAJOR=$(echo __clang_major__ | $compiler -E -x c - | tail -n 1) MINOR=$(echo __clang_minor__ | $compiler -E -x c - | tail -n 1) -if [ "x$with_patchlevel" != "x" ] ; then - PATCHLEVEL=$(echo __clang_patchlevel__ | $compiler -E -x c - | tail -n 1) - printf "%02d%02d%02d\\n" $MAJOR $MINOR $PATCHLEVEL -else - printf "%02d%02d\\n" $MAJOR $MINOR -fi +PATCHLEVEL=$(echo __clang_patchlevel__ | $compiler -E -x c - | tail -n 1) +printf "%d%02d%02d\\n" $MAJOR $MINOR $PATCHLEVEL From patchwork Tue Mar 27 05:29:29 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,15/21] gcov: remove CONFIG_GCOV_FORMAT_AUTODETECT X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132456 Message-Id: <1522128575-5326-16-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , Peter Oberparleiter , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:29 +0900 From: Masahiro Yamada List-Id: CONFIG_GCOV_FORMAT_AUTODETECT compiles either gcc_3_4.c or gcc_4_7.c according to your GCC version. We can achieve the equivalent behavior by setting reasonable dependency with the knowledge of the compiler version. If GCC older than 4.7 is used, GCOV_FORMAT_3_4 is the default, but users are still allowed to select GCOV_FORMAT_4_7 in case the newer format is back-ported. On the other hand, If GCC 4.7 or newer is used, there is no reason to use GCOV_FORMAT_3_4, so it should be hidden. If you down-grade your compiler GCC 4.7 or older, oldconfig/syncconfig will display a prompt for the choice because GCOV_FORMAT_3_4 becomes visible as a new symbol. Signed-off-by: Masahiro Yamada --- Changes in v2: - Move the 'depends on' to the choice value. kernel/gcov/Kconfig | 17 +++++------------ kernel/gcov/Makefile | 2 -- 2 files changed, 5 insertions(+), 14 deletions(-) -- 2.7.4 Acked-by: Peter Oberparleiter Reviewed-by: Kees Cook diff --git a/kernel/gcov/Kconfig b/kernel/gcov/Kconfig index 1276aab..1e3823f 100644 --- a/kernel/gcov/Kconfig +++ b/kernel/gcov/Kconfig @@ -53,23 +53,16 @@ config GCOV_PROFILE_ALL choice prompt "Specify GCOV format" depends on GCOV_KERNEL - default GCOV_FORMAT_AUTODETECT ---help--- - The gcov format is usually determined by the GCC version, but there are + The gcov format is usually determined by the GCC version, and the + default is chosen according to your GCC version. However, there are exceptions where format changes are integrated in lower-version GCCs. - In such a case use this option to adjust the format used in the kernel - accordingly. - - If unsure, choose "Autodetect". - -config GCOV_FORMAT_AUTODETECT - bool "Autodetect" - ---help--- - Select this option to use the format that corresponds to your GCC - version. + In such a case, change this option to adjust the format used in the + kernel accordingly. config GCOV_FORMAT_3_4 bool "GCC 3.4 format" + depends on CC_IS_GCC && GCC_VERSION < 40700 ---help--- Select this option to use the format defined by GCC 3.4. diff --git a/kernel/gcov/Makefile b/kernel/gcov/Makefile index c6c50e5..ff06d64 100644 --- a/kernel/gcov/Makefile +++ b/kernel/gcov/Makefile @@ -4,5 +4,3 @@ ccflags-y := -DSRCTREE='"$(srctree)"' -DOBJTREE='"$(objtree)"' obj-y := base.o fs.o obj-$(CONFIG_GCOV_FORMAT_3_4) += gcc_3_4.o obj-$(CONFIG_GCOV_FORMAT_4_7) += gcc_4_7.o -obj-$(CONFIG_GCOV_FORMAT_AUTODETECT) += $(call cc-ifversion, -lt, 0407, \ - gcc_3_4.o, gcc_4_7.o) From patchwork Tue Mar 27 05:29:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2, 16/21] kcov: imply GCC_PLUGINS and GCC_PLUGIN_SANCOV instead of select'ing them X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132446 Message-Id: <1522128575-5326-17-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:30 +0900 From: Masahiro Yamada List-Id: As Documentation/kbuild/kconfig-language.txt notes, 'select' should be used with care - it forces a lower limit of another symbol, ignoring the dependency. In this case, KCOV can select GCC_PLUGINS even if 'depends on HAVE_GCC_PLUGINS' is unmet. 'imply' is modest enough to observe the dependency, and this makes sense. If you enable KCOV, you will probably want to enable GCC_PLUGIN_SANCOV, but it should not break the dependency. I also remove unneeded code, I just happened to notice. Signed-off-by: Masahiro Yamada - Drop depends on GCC_VERSION --- Changes in v2: None lib/Kconfig.debug | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) -- 2.7.4 Reviewed-by: Kees Cook diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 5be22e4..8243c9d 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -743,8 +743,8 @@ config KCOV bool "Code coverage for fuzzing" depends on ARCH_HAS_KCOV select DEBUG_FS - select GCC_PLUGINS if !COMPILE_TEST - select GCC_PLUGIN_SANCOV if !COMPILE_TEST + imply GCC_PLUGINS + imply GCC_PLUGIN_SANCOV help KCOV exposes kernel code coverage information in a form suitable for coverage-guided fuzzing (randomized testing). @@ -758,7 +758,6 @@ config KCOV config KCOV_ENABLE_COMPARISONS bool "Enable comparison operands collection by KCOV" depends on KCOV - default n help KCOV also exposes operands of every comparison in the instrumented code along with operand sizes and PCs of the comparison instructions. @@ -768,7 +767,7 @@ config KCOV_ENABLE_COMPARISONS config KCOV_INSTRUMENT_ALL bool "Instrument all code by default" depends on KCOV - default y if KCOV + default y help If you are doing generic system call fuzzing (like e.g. syzkaller), then you will want to instrument the whole kernel and you should From patchwork Tue Mar 27 05:29:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,17/21] gcc-plugins: always build plugins with C++ X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132441 Message-Id: <1522128575-5326-18-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Emese Revfy Date: Tue, 27 Mar 2018 14:29:31 +0900 From: Masahiro Yamada List-Id: If the target compiler is GCC 4.8 or newer, plugins are compiled with HOSTCXX. Otherwise, gcc-plugin.sh will select HOSTCC or HOSTCXX. To simplify things, drop the HOSTCC support for plugins. If you use GCC plugins, the requirement for the target compiler is GCC 4.8 (or older compiler with necessary features backported). This is a feature for advanced users, so this requirement would not be a big deal. Currently, building plugins requires GCC 4.5, so this is not a big jump. Remove the code that was needed to build plugins with HOSTCC as a preparation for Kconfig migration. Signed-off-by: Masahiro Yamada --- Changes in v2: None scripts/Makefile.gcc-plugins | 11 +++-------- scripts/gcc-plugin.sh | 38 +++----------------------------------- scripts/gcc-plugins/Makefile | 15 ++++----------- 3 files changed, 10 insertions(+), 54 deletions(-) -- 2.7.4 diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins index b2a95af..b0f9108 100644 --- a/scripts/Makefile.gcc-plugins +++ b/scripts/Makefile.gcc-plugins @@ -1,7 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 ifdef CONFIG_GCC_PLUGINS - __PLUGINCC := $(call cc-ifversion, -ge, 0408, $(HOSTCXX), $(HOSTCC)) - PLUGINCC := $(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-plugin.sh "$(__PLUGINCC)" "$(HOSTCXX)" "$(CC)") + PLUGINCC := $(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-plugin.sh $(HOSTCXX) $(CC)) SANCOV_PLUGIN := -fplugin=$(objtree)/scripts/gcc-plugins/sancov_plugin.so @@ -65,12 +64,8 @@ ifdef CONFIG_GCC_PLUGINS @echo "Cannot use CONFIG_GCC_PLUGINS: plugin support on gcc <= 5.1 is buggy on powerpc, please upgrade to gcc 5.2 or newer" >&2 && exit 1 endif endif - ifeq ($(call cc-ifversion, -ge, 0405, y), y) - $(Q)$(srctree)/scripts/gcc-plugin.sh --show-error "$(__PLUGINCC)" "$(HOSTCXX)" "$(CC)" || true - @echo "Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?" >&2 && exit 1 - else - @echo "Cannot use CONFIG_GCC_PLUGINS: your gcc version does not support plugins, you should upgrade it to at least gcc 4.5" >&2 && exit 1 - endif + $(Q)$(srctree)/scripts/gcc-plugin.sh --show-error $(HOSTCXX) $(CC) || true + @echo "Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?" >&2 && exit 1 endif endif endif diff --git a/scripts/gcc-plugin.sh b/scripts/gcc-plugin.sh index d3caefe..0edbdae 100755 --- a/scripts/gcc-plugin.sh +++ b/scripts/gcc-plugin.sh @@ -8,42 +8,10 @@ if [ "$1" = "--show-error" ] ; then shift || true fi -gccplugins_dir=$($3 -print-file-name=plugin) -plugincc=$($1 -E -x c++ - -o /dev/null -I"${srctree}"/gcc-plugins -I"${gccplugins_dir}"/include 2>&1 <= 4008 || defined(ENABLE_BUILD_WITH_CXX) -#warning $2 CXX -#else -#warning $1 CC -#endif -EOF -) - -if [ $? -ne 0 ] -then - if [ -n "$SHOW_ERROR" ] ; then - echo "${plugincc}" >&2 - fi - exit 1 -fi - -case "$plugincc" in - *"$1 CC"*) - echo "$1" - exit 0 - ;; - - *"$2 CXX"*) - # the c++ compiler needs another test, see below - ;; - - *) - exit 1 - ;; -esac +gccplugins_dir=$($2 -print-file-name=plugin) # we need a c++ compiler that supports the designated initializer GNU extension -plugincc=$($2 -c -x c++ -std=gnu++98 - -fsyntax-only -I"${srctree}"/gcc-plugins -I"${gccplugins_dir}"/include 2>&1 <&1 < X-Patchwork-Id: 132442 Message-Id: <1522128575-5326-19-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Emese Revfy Date: Tue, 27 Mar 2018 14:29:32 +0900 From: Masahiro Yamada List-Id: For PowerPC, GCC 5.2 is the requirement for GCC plugins. Move the version check to Kconfig so that the GCC plugin menus will be hidden if an older compiler is in use. Signed-off-by: Masahiro Yamada Acked-by: Andrew Donnellan --- Changes in v2: None arch/powerpc/Kconfig | 2 +- scripts/Makefile.gcc-plugins | 8 -------- 2 files changed, 1 insertion(+), 9 deletions(-) -- 2.7.4 diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 73ce5dd..b8474c2 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -195,7 +195,7 @@ config PPC select HAVE_FTRACE_MCOUNT_RECORD select HAVE_FUNCTION_GRAPH_TRACER select HAVE_FUNCTION_TRACER - select HAVE_GCC_PLUGINS + select HAVE_GCC_PLUGINS if GCC_VERSION >= 50200 select HAVE_GENERIC_GUP select HAVE_HW_BREAKPOINT if PERF_EVENTS && (PPC_BOOK3S || PPC_8xx) select HAVE_IDE diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins index b0f9108..25da4c0 100644 --- a/scripts/Makefile.gcc-plugins +++ b/scripts/Makefile.gcc-plugins @@ -56,14 +56,6 @@ gcc-plugins-check: FORCE ifdef CONFIG_GCC_PLUGINS ifeq ($(PLUGINCC),) ifneq ($(GCC_PLUGINS_CFLAGS),) - # Various gccs between 4.5 and 5.1 have bugs on powerpc due to missing - # header files. gcc <= 4.6 doesn't work at all, gccs from 4.8 to 5.1 have - # issues with 64-bit targets. - ifeq ($(ARCH),powerpc) - ifeq ($(call cc-ifversion, -le, 0501, y), y) - @echo "Cannot use CONFIG_GCC_PLUGINS: plugin support on gcc <= 5.1 is buggy on powerpc, please upgrade to gcc 5.2 or newer" >&2 && exit 1 - endif - endif $(Q)$(srctree)/scripts/gcc-plugin.sh --show-error $(HOSTCXX) $(CC) || true @echo "Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?" >&2 && exit 1 endif From patchwork Tue Mar 27 05:29:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,19/21] gcc-plugins: test GCC plugin support in Kconfig X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132445 Message-Id: <1522128575-5326-20-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org, kernel-hardening@lists.openwall.com, Emese Revfy Date: Tue, 27 Mar 2018 14:29:33 +0900 From: Masahiro Yamada List-Id: Run scripts/gcc-plugin.sh from Kconfig. Users can enable GCC_PLUGINS only when it is supported. Signed-off-by: Masahiro Yamada --- Changes in v2: None arch/Kconfig | 4 +++ scripts/Makefile.gcc-plugins | 82 ++++++++++++++++---------------------------- scripts/gcc-plugin.sh | 1 - 3 files changed, 33 insertions(+), 54 deletions(-) -- 2.7.4 diff --git a/arch/Kconfig b/arch/Kconfig index b42378d..88cc925 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -407,9 +407,13 @@ config HAVE_GCC_PLUGINS An arch should select this symbol if it supports building with GCC plugins. +config CC_HAS_GCC_PLUGINS + bool + menuconfig GCC_PLUGINS bool "GCC plugins" depends on HAVE_GCC_PLUGINS + depends on $(success $srctree/scripts/gcc-plugin.sh $HOSTCXX $CC) depends on !COMPILE_TEST help GCC plugins are loadable modules that provide extra features to the diff --git a/scripts/Makefile.gcc-plugins b/scripts/Makefile.gcc-plugins index 25da4c0..19d0d5b 100644 --- a/scripts/Makefile.gcc-plugins +++ b/scripts/Makefile.gcc-plugins @@ -1,71 +1,47 @@ # SPDX-License-Identifier: GPL-2.0 -ifdef CONFIG_GCC_PLUGINS - PLUGINCC := $(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-plugin.sh $(HOSTCXX) $(CC)) - - SANCOV_PLUGIN := -fplugin=$(objtree)/scripts/gcc-plugins/sancov_plugin.so +SANCOV_PLUGIN := -fplugin=$(objtree)/scripts/gcc-plugins/sancov_plugin.so - gcc-plugin-$(CONFIG_GCC_PLUGIN_CYC_COMPLEXITY) += cyc_complexity_plugin.so +gcc-plugin-$(CONFIG_GCC_PLUGIN_CYC_COMPLEXITY) += cyc_complexity_plugin.so - gcc-plugin-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) += latent_entropy_plugin.so - gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) += -DLATENT_ENTROPY_PLUGIN - ifdef CONFIG_GCC_PLUGIN_LATENT_ENTROPY - DISABLE_LATENT_ENTROPY_PLUGIN += -fplugin-arg-latent_entropy_plugin-disable - endif +gcc-plugin-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) += latent_entropy_plugin.so +gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_LATENT_ENTROPY) += -DLATENT_ENTROPY_PLUGIN +ifdef CONFIG_GCC_PLUGIN_LATENT_ENTROPY + DISABLE_LATENT_ENTROPY_PLUGIN += -fplugin-arg-latent_entropy_plugin-disable +endif - ifdef CONFIG_GCC_PLUGIN_SANCOV - ifeq ($(CFLAGS_KCOV),) - # It is needed because of the gcc-plugin.sh and gcc version checks. - gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV) += sancov_plugin.so +ifdef CONFIG_GCC_PLUGIN_SANCOV + ifeq ($(CFLAGS_KCOV),) + # It is needed because of the gcc-plugin.sh and gcc version checks. + gcc-plugin-$(CONFIG_GCC_PLUGIN_SANCOV) += sancov_plugin.so - ifneq ($(PLUGINCC),) - CFLAGS_KCOV := $(SANCOV_PLUGIN) - else - $(warning warning: cannot use CONFIG_KCOV: -fsanitize-coverage=trace-pc is not supported by compiler) - endif - endif + CFLAGS_KCOV := $(SANCOV_PLUGIN) endif +endif - gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += structleak_plugin.so - gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_VERBOSE) += -fplugin-arg-structleak_plugin-verbose - gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL) += -fplugin-arg-structleak_plugin-byref-all - gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += -DSTRUCTLEAK_PLUGIN - - gcc-plugin-$(CONFIG_GCC_PLUGIN_RANDSTRUCT) += randomize_layout_plugin.so - gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT) += -DRANDSTRUCT_PLUGIN - gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT_PERFORMANCE) += -fplugin-arg-randomize_layout_plugin-performance-mode +gcc-plugin-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += structleak_plugin.so +gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_VERBOSE) += -fplugin-arg-structleak_plugin-verbose +gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL) += -fplugin-arg-structleak_plugin-byref-all +gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_STRUCTLEAK) += -DSTRUCTLEAK_PLUGIN - GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) +gcc-plugin-$(CONFIG_GCC_PLUGIN_RANDSTRUCT) += randomize_layout_plugin.so +gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT) += -DRANDSTRUCT_PLUGIN +gcc-plugin-cflags-$(CONFIG_GCC_PLUGIN_RANDSTRUCT_PERFORMANCE) += -fplugin-arg-randomize_layout_plugin-performance-mode - export PLUGINCC GCC_PLUGINS_CFLAGS GCC_PLUGIN GCC_PLUGIN_SUBDIR - export SANCOV_PLUGIN DISABLE_LATENT_ENTROPY_PLUGIN +GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) - ifneq ($(PLUGINCC),) - # SANCOV_PLUGIN can be only in CFLAGS_KCOV because avoid duplication. - GCC_PLUGINS_CFLAGS := $(filter-out $(SANCOV_PLUGIN), $(GCC_PLUGINS_CFLAGS)) - endif +export GCC_PLUGINS_CFLAGS GCC_PLUGIN GCC_PLUGIN_SUBDIR +export SANCOV_PLUGIN DISABLE_LATENT_ENTROPY_PLUGIN - KBUILD_CFLAGS += $(GCC_PLUGINS_CFLAGS) - GCC_PLUGIN := $(gcc-plugin-y) - GCC_PLUGIN_SUBDIR := $(gcc-plugin-subdir-y) -endif +# SANCOV_PLUGIN can be only in CFLAGS_KCOV because avoid duplication. +GCC_PLUGINS_CFLAGS := $(filter-out $(SANCOV_PLUGIN), $(GCC_PLUGINS_CFLAGS)) -# If plugins aren't supported, abort the build before hard-to-read compiler -# errors start getting spewed by the main build. -PHONY += gcc-plugins-check -gcc-plugins-check: FORCE -ifdef CONFIG_GCC_PLUGINS - ifeq ($(PLUGINCC),) - ifneq ($(GCC_PLUGINS_CFLAGS),) - $(Q)$(srctree)/scripts/gcc-plugin.sh --show-error $(HOSTCXX) $(CC) || true - @echo "Cannot use CONFIG_GCC_PLUGINS: your gcc installation does not support plugins, perhaps the necessary headers are missing?" >&2 && exit 1 - endif - endif -endif - @: +KBUILD_CFLAGS += $(GCC_PLUGINS_CFLAGS) +GCC_PLUGIN := $(gcc-plugin-y) +GCC_PLUGIN_SUBDIR := $(gcc-plugin-subdir-y) # Actually do the build, if requested. PHONY += gcc-plugins -gcc-plugins: scripts_basic gcc-plugins-check +gcc-plugins: scripts_basic ifdef CONFIG_GCC_PLUGINS $(Q)$(MAKE) $(build)=scripts/gcc-plugins endif diff --git a/scripts/gcc-plugin.sh b/scripts/gcc-plugin.sh index 0edbdae..b18c69c 100755 --- a/scripts/gcc-plugin.sh +++ b/scripts/gcc-plugin.sh @@ -24,7 +24,6 @@ EOF if [ $? -eq 0 ] then - echo "$1" exit 0 fi From patchwork Tue Mar 27 05:29:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2,20/21] gcc-plugins: enable GCC_PLUGINS for COMPILE_TEST X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132439 Message-Id: <1522128575-5326-21-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , linux-kernel@vger.kernel.org Date: Tue, 27 Mar 2018 14:29:34 +0900 From: Masahiro Yamada List-Id: The plugin availability is checked in Kconfig, so all{yes,mod}config will not be bothered. Remove 'depends on !COMPILE_TEST'. Signed-off-by: Masahiro Yamada --- Changes in v2: - Remove more 'depends on' arch/Kconfig | 4 ---- 1 file changed, 4 deletions(-) -- 2.7.4 diff --git a/arch/Kconfig b/arch/Kconfig index 88cc925..b07094c 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -414,7 +414,6 @@ menuconfig GCC_PLUGINS bool "GCC plugins" depends on HAVE_GCC_PLUGINS depends on $(success $srctree/scripts/gcc-plugin.sh $HOSTCXX $CC) - depends on !COMPILE_TEST help GCC plugins are loadable modules that provide extra features to the compiler. They are useful for runtime instrumentation and static analysis. @@ -424,7 +423,6 @@ menuconfig GCC_PLUGINS config GCC_PLUGIN_CYC_COMPLEXITY bool "Compute the cyclomatic complexity of a function" if EXPERT depends on GCC_PLUGINS - depends on !COMPILE_TEST help The complexity M of a function's control flow graph is defined as: M = E - N + 2P @@ -487,7 +485,6 @@ config GCC_PLUGIN_STRUCTLEAK_BYREF_ALL config GCC_PLUGIN_STRUCTLEAK_VERBOSE bool "Report forcefully initialized variables" depends on GCC_PLUGIN_STRUCTLEAK - depends on !COMPILE_TEST help This option will cause a warning to be printed each time the structleak plugin finds a variable it thinks needs to be @@ -527,7 +524,6 @@ config GCC_PLUGIN_RANDSTRUCT config GCC_PLUGIN_RANDSTRUCT_PERFORMANCE bool "Use cacheline-aware structure randomization" depends on GCC_PLUGIN_RANDSTRUCT - depends on !COMPILE_TEST help If you say Y here, the RANDSTRUCT randomization will make a best effort at restricting randomization to cacheline-sized From patchwork Tue Mar 27 05:29:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: [v2, 21/21] arm64: move GCC version check for ARCH_SUPPORTS_INT128 to Kconfig X-Patchwork-Submitter: Masahiro Yamada X-Patchwork-Id: 132444 Message-Id: <1522128575-5326-22-git-send-email-yamada.masahiro@socionext.com> To: linux-kbuild@vger.kernel.org Cc: Sam Ravnborg , Linus Torvalds , Arnd Bergmann , Ulf Magnusson , Kees Cook , Thomas Gleixner , Greg Kroah-Hartman , Randy Dunlap , "Luis R . Rodriguez" , Nicolas Pitre , Masahiro Yamada , Will Deacon , Catalin Marinas , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Date: Tue, 27 Mar 2018 14:29:35 +0900 From: Masahiro Yamada List-Id: This becomes much neater in Kconfig. Signed-off-by: Masahiro Yamada --- Changes in v2: - Newly added arch/arm64/Kconfig | 1 + arch/arm64/Makefile | 2 -- 2 files changed, 1 insertion(+), 2 deletions(-) -- 2.7.4 Acked-by: Will Deacon Reviewed-by: Kees Cook diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7381eeb..0c97f40 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -43,6 +43,7 @@ config ARM64 select ARCH_USE_QUEUED_RWLOCKS select ARCH_SUPPORTS_MEMORY_FAILURE select ARCH_SUPPORTS_ATOMIC_RMW + select ARCH_SUPPORTS_INT128 if GCC_VERSION >= 50000 select ARCH_SUPPORTS_NUMA_BALANCING select ARCH_WANT_COMPAT_IPC_PARSE_VERSION select ARCH_WANT_FRAME_POINTERS diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index b481b4a..7318165 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -57,8 +57,6 @@ KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) KBUILD_CFLAGS += $(call cc-option,-mabi=lp64) KBUILD_AFLAGS += $(call cc-option,-mabi=lp64) -KBUILD_CFLAGS += $(call cc-ifversion, -ge, 0500, -DCONFIG_ARCH_SUPPORTS_INT128) - ifeq ($(CONFIG_CPU_BIG_ENDIAN), y) KBUILD_CPPFLAGS += -mbig-endian CHECKFLAGS += -D__AARCH64EB__