| Message ID | 1503132577-24423-1-git-send-email-yamada.masahiro@socionext.com |
|---|---|
| Headers | show |
| Series | kbuild: generate intermediate C files instead of copying _shipped files | expand |
On Sat, Aug 19, 2017 at 1:49 AM, Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > Here is one question. Is it acceptable to use those rules all the time? > That is, generate those C files by flex, bison, gperf during the > kernel building. Yeah, I think we probably should do that. However, when I just tested, I noticed that we have issues with re-generating those files. With gperf 3.1 installed, I get In file included from scripts/kconfig/zconf.tab.c:213:0: scripts/kconfig/zconf.gperf:147:1: error: conflicting types for ‘kconf_id_lookup’ scripts/kconfig/zconf.gperf:12:31: note: previous declaration of ‘kconf_id_lookup’ was here static const struct kconf_id *kconf_id_lookup(register const char *str, register unsigned int len); ^~~~~~~~~~~~~~~ because gperf now generates const struct kconf_id * -kconf_id_lookup (register const char *str, register unsigned int len) +kconf_id_lookup (register const char *str, register size_t len) and I'm not sure how to detect that automatically. It seems to be a gperf-3.1 change, and gperf doesn't seem to generate any version markers. Working around that, I hit: In file included from scripts/genksyms/lex.lex.c:1921:0: scripts/genksyms/keywords.gperf:54:1: error: conflicting types for ‘is_reserved_word’ static, STATIC_KEYW ^~~~~~~~~~~~~~~~ In file included from scripts/genksyms/lex.lex.c:1921:0: scripts/genksyms/keywords.gperf:6:30: note: previous declaration of ‘is_reserved_word’ was here static const struct resword *is_reserved_word(register const char *str, register unsigned int len); ^~~~~~~~~~~~~~~~ so we have at least two cases of this in the source tree. So one of the advantages of the pre-shipped files is that we can avoid that kind of crazy version issues with the tools. But if we can solve the versioning thing easily, I certainly don't mind getting rid of the pre-generated files. Having to have flex/bison/gperf isn't a huge onus on the kernel build system. Linus
On Thu, Sep 7, 2017 at 11:18 PM, Masahiro Yamada <yamada.masahiro@socionext.com> wrote: > > If CONFIG_MODVERSIONS is enabled, > I notice lots of error messages. > WARNING: EXPORT symbol "finish_open" [vmlinux] version generation > failed, symbol will not be versioned > > So, I think something was broken in scripts/genksyms/. > > Of course, it was a trivial conversion, so it should not be hard to fix... Indeed, hopefully it would be trivial, but I don't even see the error here. Of course, I only did a "make allmodconfig" to test the MODVERSIONS case, I didn't actually install the modules. Is that error perhaps only detected at install time? I did build and install a kernel with that patch, but that's my actual "real" config for the machine, and it didn't have MODVERSIONS enabled. Oh, how I hate modversions. But I'll take a look if I can see what I did wrong in the "Trivial and Obvious(tm)" conversion. Linus
On Fri, Sep 8, 2017 at 10:22 AM, Linus Torvalds <torvalds@linux-foundation.org> wrote: > > Of course, I only did a "make allmodconfig" to test the MODVERSIONS > case, I didn't actually install the modules. Is that error perhaps > only detected at install time? Oh, I take that back. I just got a ton of warnings with my allmodconfig after doing a "git clean -dqfx". So I guess there is a dependency issue - my normal build test after the merge didn't show any issues, simply because the change in ksymoops didn't actually cause the version information to be re-generated. It doesn't seem to happen for every exported symbol, though. Odd. Looking at it, but not making sense of it yet. Linus
On Fri, Sep 8, 2017 at 11:39 AM, Linus Torvalds <torvalds@linux-foundation.org> wrote: > > Strange. Does anybody see what the pattern to the failure is? Found it. Stupid special case for 'typeof()' that used is_reserved_word() in ways I hadn't realized. Fix committed. Linus
On Fri, Sep 08, 2017 at 02:38:23PM -0700, Linus Torvalds wrote: > On Fri, Sep 8, 2017 at 11:39 AM, Linus Torvalds > <torvalds@linux-foundation.org> wrote: > > > > Strange. Does anybody see what the pattern to the failure is? > > Found it. Stupid special case for 'typeof()' that used > is_reserved_word() in ways I hadn't realized. > > Fix committed. To get bonus points for this cleanup you should also remove the now unused gpref support in Makefile.lib Sam
In Linux build system convention, we do not run tools such as flex, bison, gperf during the kernel building. Instead, manage generated C files in the repository with _shipped suffixes. They are simply shipped (copied) removing the _shipped suffixes during the kernel building. Commit 7373f4f83c71 ("kbuild: add implicit rules for parser generation") added a mechanism to regenerate intermediate C files easily. The build rules are surrounded with ifdef REGENERATE_PARSERS. So, we need to pass REGENERATE_PARSERS=1 from the command line when we want to update them. Here is one question. Is it acceptable to use those rules all the time? That is, generate those C files by flex, bison, gperf during the kernel building. This means, the build system depends on more external tools. From the users' point of view, they will need to install flex, bison, gperf in order to build the kernel. From the developers' point of view, the advantage is we do not need to version-control generated files, i.e. _shipped files will be deleted. I'd like to know if this is acceptable or not. For example, currently some files are simply shipped (copied) when building the kconfig program. $ make mrproper; make defconfig HOSTCC scripts/basic/fixdep HOSTCC scripts/kconfig/conf.o SHIPPED scripts/kconfig/zconf.tab.c SHIPPED scripts/kconfig/zconf.lex.c SHIPPED scripts/kconfig/zconf.hash.c HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/conf *** Default configuration is based on 'x86_64_defconfig' # # configuration written to .config # With this series, they are created from *real* sources (*.y, *.l, *.gperf files). $ make mrproper; make defconfig HOSTCC scripts/basic/fixdep HOSTCC scripts/kconfig/conf.o YACC scripts/kconfig/zconf.tab.c LEX scripts/kconfig/zconf.lex.c GPERF scripts/kconfig/zconf.hash.c HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/conf *** Default configuration is based on 'x86_64_defconfig' # # configuration written to .config # Note: The tool versions in Documentation/process/changes.rst are just place-holders for now. We need to figure out the minimal versions if we like to switch to this approach. Masahiro Yamada (3): kbuild: generate *.hash.c during build kbuild: generate *.lex.c during build kbuild: generate *.tab.c and *.tab.h during build Documentation/process/changes.rst | 36 + scripts/Makefile.lib | 26 +- scripts/dtc/Makefile | 6 +- scripts/dtc/dtc-lexer.lex.c_shipped | 2259 --------------------------- scripts/dtc/dtc-parser.tab.c_shipped | 2303 ---------------------------- scripts/dtc/dtc-parser.tab.h_shipped | 125 -- scripts/genksyms/Makefile | 4 +- scripts/genksyms/keywords.hash.c_shipped | 230 --- scripts/genksyms/lex.lex.c_shipped | 2291 --------------------------- scripts/genksyms/parse.tab.c_shipped | 2394 ----------------------------- scripts/genksyms/parse.tab.h_shipped | 119 -- scripts/kconfig/Makefile | 1 + scripts/kconfig/zconf.hash.c_shipped | 297 ---- scripts/kconfig/zconf.lex.c_shipped | 2473 ------------------------------ scripts/kconfig/zconf.tab.c_shipped | 2471 ----------------------------- 15 files changed, 53 insertions(+), 14982 deletions(-) delete mode 100644 scripts/dtc/dtc-lexer.lex.c_shipped delete mode 100644 scripts/dtc/dtc-parser.tab.c_shipped delete mode 100644 scripts/dtc/dtc-parser.tab.h_shipped delete mode 100644 scripts/genksyms/keywords.hash.c_shipped delete mode 100644 scripts/genksyms/lex.lex.c_shipped delete mode 100644 scripts/genksyms/parse.tab.c_shipped delete mode 100644 scripts/genksyms/parse.tab.h_shipped delete mode 100644 scripts/kconfig/zconf.hash.c_shipped delete mode 100644 scripts/kconfig/zconf.lex.c_shipped delete mode 100644 scripts/kconfig/zconf.tab.c_shipped -- 2.7.4