Odd build breakage in 4.9-rc7

Message ID alpine.LFD.2.20.1611301726240.1715@knanqh.ubzr
State New
Headers show

Commit Message

Nicolas Pitre Nov. 30, 2016, 10:41 p.m.
On Wed, 30 Nov 2016, Linus Torvalds wrote:

> On Wed, Nov 30, 2016 at 10:50 AM, Prarit Bhargava <prarit@redhat.com> wrote:

> >

> > It comes back.  The steps to reproduce this are:

> >

> > 1.  checkout latest linux.git

> > 2.  make -j112

> >

> > (IOW, it occurs 100% of the time for me on a clean tree.)


I don't have access to such hardware where -j112 could ever make sense.  :-)
In other words, I can't reproduce regardless of the -j value I try.

> I suspect it's not new, it's just that you are able to hit the timing

> just right (and the new include presumable makes that just be much

> easier).


Here's the best fix I can think of. I can't convince myself any other 
location would be 100% safe.  Obviously I can't confirm if this actually 
fixes anything.

----- >8
Subject: kbuild: make sure autoksyms.h exists early

Some people are able to trigger a race where autoksyms.h is used before
its empty version is even created. Let's create it at the same time as
the directory holding it is created.

Signed-off-by: Nicolas Pitre <nico@linaro.org>

Comments

Prarit Bhargava Dec. 1, 2016, 12:47 p.m. | #1
On 11/30/2016 05:41 PM, Nicolas Pitre wrote:
> On Wed, 30 Nov 2016, Linus Torvalds wrote:

> 

>> On Wed, Nov 30, 2016 at 10:50 AM, Prarit Bhargava <prarit@redhat.com> wrote:

>>>

>>> It comes back.  The steps to reproduce this are:

>>>

>>> 1.  checkout latest linux.git

>>> 2.  make -j112

>>>

>>> (IOW, it occurs 100% of the time for me on a clean tree.)

> 

> I don't have access to such hardware where -j112 could ever make sense.  :-)


:)  I could push the builds onto the -j256 but I'm doing other stuff over there. :)

> In other words, I can't reproduce regardless of the -j value I try.

> 

>> I suspect it's not new, it's just that you are able to hit the timing

>> just right (and the new include presumable makes that just be much

>> easier).

> 

> Here's the best fix I can think of. I can't convince myself any other 

> location would be 100% safe.  Obviously I can't confirm if this actually 

> fixes anything.

> 

> ----- >8

> Subject: kbuild: make sure autoksyms.h exists early

> 


I'm building with this patch on top of latest now.  I will put it in a tight
loop and clear the drop_caches between builds to see if I can make it fail.

Thanks Nicolas -- your help is very much appreciated.

P.
Prarit Bhargava Dec. 1, 2016, 2:03 p.m. | #2
On 11/30/2016 05:41 PM, Nicolas Pitre wrote:
> On Wed, 30 Nov 2016, Linus Torvalds wrote:

> 

>> On Wed, Nov 30, 2016 at 10:50 AM, Prarit Bhargava <prarit@redhat.com> wrote:

>>>

>>> It comes back.  The steps to reproduce this are:

>>>

>>> 1.  checkout latest linux.git

>>> 2.  make -j112

>>>

>>> (IOW, it occurs 100% of the time for me on a clean tree.)

> 

> I don't have access to such hardware where -j112 could ever make sense.  :-)

> In other words, I can't reproduce regardless of the -j value I try.

> 

>> I suspect it's not new, it's just that you are able to hit the timing

>> just right (and the new include presumable makes that just be much

>> easier).

> 

> Here's the best fix I can think of. I can't convince myself any other 

> location would be 100% safe.  Obviously I can't confirm if this actually 

> fixes anything.

> 

> ----- >8

> Subject: kbuild: make sure autoksyms.h exists early

> 

> Some people are able to trigger a race where autoksyms.h is used before

> its empty version is even created. Let's create it at the same time as

> the directory holding it is created.

> 

> Signed-off-by: Nicolas Pitre <nico@linaro.org>

> 

> diff --git a/Makefile b/Makefile

> index 694111b43c..9f9c3b577c 100644

> --- a/Makefile

> +++ b/Makefile

> @@ -1019,8 +1019,6 @@ prepare2: prepare3 prepare-compiler-check outputmakefile asm-generic

>  prepare1: prepare2 $(version_h) include/generated/utsrelease.h \

>                     include/config/auto.conf

>  	$(cmd_crmodverdir)

> -	$(Q)test -e include/generated/autoksyms.h || \

> -	    touch   include/generated/autoksyms.h

>  

>  archprepare: archheaders archscripts prepare1 scripts_basic

>  

> diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile

> index ebced77deb..90a091b6ae 100644

> --- a/scripts/kconfig/Makefile

> +++ b/scripts/kconfig/Makefile

> @@ -35,6 +35,8 @@ nconfig: $(obj)/nconf

>  

>  silentoldconfig: $(obj)/conf

>  	$(Q)mkdir -p include/config include/generated

> +	$(Q)test -e include/generated/autoksyms.h || \

> +	    touch   include/generated/autoksyms.h

>  	$< $(silent) --$@ $(Kconfig)

>  

>  localyesconfig localmodconfig: $(obj)/streamline_config.pl $(obj)/conf

> 


The testing was successful.

After testing an hour of builds with different -j values, I'm no longer seeing
any compile issues when this patch is applied.  When I remove the patch the
compile error returns so I'm going to say that this patch fixed it.

Thanks again Nicolas.

Tested-by: Prarit Bhargava <prarit@redhat.com>


P.
Jarod Wilson Dec. 1, 2016, 3:22 p.m. | #3
On 2016-12-01 9:03 AM, Prarit Bhargava wrote:
>

>

> On 11/30/2016 05:41 PM, Nicolas Pitre wrote:

>> On Wed, 30 Nov 2016, Linus Torvalds wrote:

>>

>>> On Wed, Nov 30, 2016 at 10:50 AM, Prarit Bhargava <prarit@redhat.com> wrote:

>>>>

>>>> It comes back.  The steps to reproduce this are:

>>>>

>>>> 1.  checkout latest linux.git

>>>> 2.  make -j112

>>>>

>>>> (IOW, it occurs 100% of the time for me on a clean tree.)

>>

>> I don't have access to such hardware where -j112 could ever make sense.  :-)

>> In other words, I can't reproduce regardless of the -j value I try.

>>

>>> I suspect it's not new, it's just that you are able to hit the timing

>>> just right (and the new include presumable makes that just be much

>>> easier).

>>

>> Here's the best fix I can think of. I can't convince myself any other

>> location would be 100% safe.  Obviously I can't confirm if this actually

>> fixes anything.

>>

>> ----- >8

>> Subject: kbuild: make sure autoksyms.h exists early

>>

>> Some people are able to trigger a race where autoksyms.h is used before

>> its empty version is even created. Let's create it at the same time as

>> the directory holding it is created.

>>

>> Signed-off-by: Nicolas Pitre <nico@linaro.org>

>>

>> diff --git a/Makefile b/Makefile

>> index 694111b43c..9f9c3b577c 100644

>> --- a/Makefile

>> +++ b/Makefile

>> @@ -1019,8 +1019,6 @@ prepare2: prepare3 prepare-compiler-check outputmakefile asm-generic

>>  prepare1: prepare2 $(version_h) include/generated/utsrelease.h \

>>                     include/config/auto.conf

>>  	$(cmd_crmodverdir)

>> -	$(Q)test -e include/generated/autoksyms.h || \

>> -	    touch   include/generated/autoksyms.h

>>

>>  archprepare: archheaders archscripts prepare1 scripts_basic

>>

>> diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile

>> index ebced77deb..90a091b6ae 100644

>> --- a/scripts/kconfig/Makefile

>> +++ b/scripts/kconfig/Makefile

>> @@ -35,6 +35,8 @@ nconfig: $(obj)/nconf

>>

>>  silentoldconfig: $(obj)/conf

>>  	$(Q)mkdir -p include/config include/generated

>> +	$(Q)test -e include/generated/autoksyms.h || \

>> +	    touch   include/generated/autoksyms.h

>>  	$< $(silent) --$@ $(Kconfig)

>>

>>  localyesconfig localmodconfig: $(obj)/streamline_config.pl $(obj)/conf

>>

>

> The testing was successful.

>

> After testing an hour of builds with different -j values, I'm no longer seeing

> any compile issues when this patch is applied.  When I remove the patch the

> compile error returns so I'm going to say that this patch fixed it.

>

> Thanks again Nicolas.

>

> Tested-by: Prarit Bhargava <prarit@redhat.com>


Looks good here as well, can do parallel make w/o reverting Tony's 
patches again.

Tested-by: Jarod Wilson <jarod@redhat.com>


-- 
Jarod Wilson
jarod@redhat.com

Patch hide | download patch | download mbox

diff --git a/Makefile b/Makefile
index 694111b43c..9f9c3b577c 100644
--- a/Makefile
+++ b/Makefile
@@ -1019,8 +1019,6 @@  prepare2: prepare3 prepare-compiler-check outputmakefile asm-generic
 prepare1: prepare2 $(version_h) include/generated/utsrelease.h \
                    include/config/auto.conf
 	$(cmd_crmodverdir)
-	$(Q)test -e include/generated/autoksyms.h || \
-	    touch   include/generated/autoksyms.h
 
 archprepare: archheaders archscripts prepare1 scripts_basic
 
diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile
index ebced77deb..90a091b6ae 100644
--- a/scripts/kconfig/Makefile
+++ b/scripts/kconfig/Makefile
@@ -35,6 +35,8 @@  nconfig: $(obj)/nconf
 
 silentoldconfig: $(obj)/conf
 	$(Q)mkdir -p include/config include/generated
+	$(Q)test -e include/generated/autoksyms.h || \
+	    touch   include/generated/autoksyms.h
 	$< $(silent) --$@ $(Kconfig)
 
 localyesconfig localmodconfig: $(obj)/streamline_config.pl $(obj)/conf