mbox series

[v6,00/29] Warn on orphan section placement

Message ID 20200821194310.3089815-1-keescook@chromium.org
Headers show
Series Warn on orphan section placement | expand

Message

Kees Cook Aug. 21, 2020, 7:42 p.m. UTC
Hi Ingo,

Based on my testing, this is ready to go. I've reviewed the feedback on
v5 and made a few small changes, noted below.


https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=linker/orphans/warn/v6

v6:
- rebase to -tip x86/boot
- remove 0-sized NOLOAD
- move .got.plt to end with INFO (NOLOAD warns)
- add Reviewed-bys
v5: https://lore.kernel.org/lkml/20200731230820.1742553-1-keescook@chromium.org/
v4: https://lore.kernel.org/lkml/20200629061840.4065483-1-keescook@chromium.org/
v3: https://lore.kernel.org/lkml/20200624014940.1204448-1-keescook@chromium.org/
v2: https://lore.kernel.org/lkml/20200622205815.2988115-1-keescook@chromium.org/
v1: https://lore.kernel.org/lkml/20200228002244.15240-1-keescook@chromium.org/

A recent bug[1] was solved for builds linked with ld.lld, and tracking
it down took way longer than it needed to (a year). Ultimately, it
boiled down to differences between ld.bfd and ld.lld's handling of
orphan sections. Similar situation have continued to recur, and it's
clear the kernel build needs to be much more explicit about linker
sections. Similarly, the recent FGKASLR series brought up orphan section
handling too[2]. In all cases, it would have been nice if the linker was
running with --orphan-handling=warn so that surprise sections wouldn't
silently get mapped into the kernel image at locations up to the whim
of the linker's orphan handling logic. Instead, all desired sections
should be explicitly identified in the linker script (to be either kept,
discarded, or verified to be zero-sized) with any orphans throwing a
warning. The powerpc architecture has actually been doing this for some
time, so this series just extends that coverage to x86, arm, and arm64.

This has gotten sucecssful build testing under the following matrix:

compiler/linker: gcc+ld.bfd, clang+ld.lld
targets: defconfig, allmodconfig
architectures: x86, i386, arm64, arm
versions: -tip x86/boot

All three architectures depend on the first several commits to
vmlinux.lds.h. x86 depends on Arvind's GOT series (in -tip x86/boot now).
arm64 depends on the efi/libstub patch. As such, I'd like to land this
series as a whole. Ingo has suggested he'd take it into -tip.

Thanks!

-Kees

[1] https://github.com/ClangBuiltLinux/linux/issues/282
[2] https://lore.kernel.org/lkml/202002242122.AA4D1B8@keescook/

Kees Cook (28):
  vmlinux.lds.h: Create COMMON_DISCARDS
  vmlinux.lds.h: Add .gnu.version* to COMMON_DISCARDS
  vmlinux.lds.h: Avoid KASAN and KCSAN's unwanted sections
  vmlinux.lds.h: Split ELF_DETAILS from STABS_DEBUG
  vmlinux.lds.h: Add .symtab, .strtab, and .shstrtab to ELF_DETAILS
  efi/libstub: Disable -mbranch-protection
  arm64/mm: Remove needless section quotes
  arm64/kernel: Remove needless Call Frame Information annotations
  arm64/build: Remove .eh_frame* sections due to unwind tables
  arm64/build: Use common DISCARDS in linker script
  arm64/build: Add missing DWARF sections
  arm64/build: Assert for unwanted sections
  arm64/build: Warn on orphan section placement
  arm/build: Refactor linker script headers
  arm/build: Explicitly keep .ARM.attributes sections
  arm/build: Add missing sections
  arm/build: Assert for unwanted sections
  arm/build: Warn on orphan section placement
  arm/boot: Handle all sections explicitly
  arm/boot: Warn on orphan section placement
  x86/asm: Avoid generating unused kprobe sections
  x86/build: Enforce an empty .got.plt section
  x86/build: Assert for unwanted sections
  x86/build: Warn on orphan section placement
  x86/boot/compressed: Reorganize zero-size section asserts
  x86/boot/compressed: Remove, discard, or assert for unwanted sections
  x86/boot/compressed: Add missing debugging sections to output
  x86/boot/compressed: Warn on orphan section placement

Nick Desaulniers (1):
  vmlinux.lds.h: add PGO and AutoFDO input sections

 arch/alpha/kernel/vmlinux.lds.S               |  1 +
 arch/arc/kernel/vmlinux.lds.S                 |  1 +
 arch/arm/Makefile                             |  4 ++
 arch/arm/boot/compressed/Makefile             |  2 +
 arch/arm/boot/compressed/vmlinux.lds.S        | 20 +++----
 .../arm/{kernel => include/asm}/vmlinux.lds.h | 30 ++++++++--
 arch/arm/kernel/vmlinux-xip.lds.S             |  8 ++-
 arch/arm/kernel/vmlinux.lds.S                 |  8 ++-
 arch/arm64/Makefile                           |  9 ++-
 arch/arm64/kernel/smccc-call.S                |  2 -
 arch/arm64/kernel/vmlinux.lds.S               | 28 +++++++--
 arch/arm64/mm/mmu.c                           |  2 +-
 arch/csky/kernel/vmlinux.lds.S                |  1 +
 arch/hexagon/kernel/vmlinux.lds.S             |  1 +
 arch/ia64/kernel/vmlinux.lds.S                |  1 +
 arch/mips/kernel/vmlinux.lds.S                |  1 +
 arch/nds32/kernel/vmlinux.lds.S               |  1 +
 arch/nios2/kernel/vmlinux.lds.S               |  1 +
 arch/openrisc/kernel/vmlinux.lds.S            |  1 +
 arch/parisc/boot/compressed/vmlinux.lds.S     |  1 +
 arch/parisc/kernel/vmlinux.lds.S              |  1 +
 arch/powerpc/kernel/vmlinux.lds.S             |  2 +-
 arch/riscv/kernel/vmlinux.lds.S               |  1 +
 arch/s390/kernel/vmlinux.lds.S                |  1 +
 arch/sh/kernel/vmlinux.lds.S                  |  1 +
 arch/sparc/kernel/vmlinux.lds.S               |  1 +
 arch/um/kernel/dyn.lds.S                      |  2 +-
 arch/um/kernel/uml.lds.S                      |  2 +-
 arch/x86/Makefile                             |  4 ++
 arch/x86/boot/compressed/Makefile             |  2 +
 arch/x86/boot/compressed/vmlinux.lds.S        | 58 +++++++++++++------
 arch/x86/include/asm/asm.h                    |  6 +-
 arch/x86/kernel/vmlinux.lds.S                 | 39 ++++++++++++-
 drivers/firmware/efi/libstub/Makefile         |  9 ++-
 include/asm-generic/vmlinux.lds.h             | 49 +++++++++++++---
 35 files changed, 241 insertions(+), 60 deletions(-)
 rename arch/arm/{kernel => include/asm}/vmlinux.lds.h (84%)

Comments

Ingo Molnar Sept. 1, 2020, 7:11 a.m. UTC | #1
* Kees Cook <keescook@chromium.org> wrote:

> On Fri, Aug 21, 2020 at 12:42:41PM -0700, Kees Cook wrote:
> > Hi Ingo,
> > 
> > Based on my testing, this is ready to go. I've reviewed the feedback on
> > v5 and made a few small changes, noted below.
> 
> If no one objects, I'll pop this into my tree for -next. I'd prefer it
> go via -tip though! :)
> 
> Thanks!

I'll pick it up today, it all looks very good now!

Thanks,

	Ingo
Ingo Molnar Sept. 1, 2020, 8:16 a.m. UTC | #2
* Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Ingo Molnar <mingo@kernel.org> wrote:
> 
> > 
> > * Kees Cook <keescook@chromium.org> wrote:
> > 
> > > On Fri, Aug 21, 2020 at 12:42:41PM -0700, Kees Cook wrote:
> > > > Hi Ingo,
> > > > 
> > > > Based on my testing, this is ready to go. I've reviewed the feedback on
> > > > v5 and made a few small changes, noted below.
> > > 
> > > If no one objects, I'll pop this into my tree for -next. I'd prefer it
> > > go via -tip though! :)
> > > 
> > > Thanks!
> > 
> > I'll pick it up today, it all looks very good now!
> 
> One thing I found in testing is that it doesn't handler older LD 
> versions well enough:
> 
>   ld: unrecognized option '--orphan-handling=warn'
> 
> Could we just detect the availability of this flag, and emit a warning 
> if it doesn't exist but otherwise not abort the build?
> 
> This is with:
> 
>   GNU ld version 2.25-17.fc23

I've resolved this for now by not applying the 5 patches that add the 
actual orphan section warnings:

  arm64/build: Warn on orphan section placement
  arm/build: Warn on orphan section placement
  arm/boot: Warn on orphan section placement
  x86/build: Warn on orphan section placement
  x86/boot/compressed: Warn on orphan section placement

The new asserts plus the actual fixes/enhancements are enough changes 
to test for now in any case. :-)

Thanks,

	Ingo
Nick Desaulniers Sept. 1, 2020, 6:02 p.m. UTC | #3
On Tue, Sep 1, 2020 at 8:17 AM Kees Cook <keescook@chromium.org> wrote:
>
> On Tue, Sep 01, 2020 at 10:16:47AM +0200, Ingo Molnar wrote:
> >
> > * Ingo Molnar <mingo@kernel.org> wrote:
> >
> > >
> > > * Ingo Molnar <mingo@kernel.org> wrote:
> > >
> > > >
> > > > * Kees Cook <keescook@chromium.org> wrote:
> > > >
> > > > > On Fri, Aug 21, 2020 at 12:42:41PM -0700, Kees Cook wrote:
> > > > > > Hi Ingo,
> > > > > >
> > > > > > Based on my testing, this is ready to go. I've reviewed the feedback on
> > > > > > v5 and made a few small changes, noted below.
> > > > >
> > > > > If no one objects, I'll pop this into my tree for -next. I'd prefer it
> > > > > go via -tip though! :)
> > > > >
> > > > > Thanks!
> > > >
> > > > I'll pick it up today, it all looks very good now!
> > >
> > > One thing I found in testing is that it doesn't handler older LD
> > > versions well enough:
> > >
> > >   ld: unrecognized option '--orphan-handling=warn'
>
> Oh! Uhm, yikes. Thanks for noticing this.
>
> > > Could we just detect the availability of this flag, and emit a warning
> > > if it doesn't exist but otherwise not abort the build?
>
> Yeah, I'll respin those patches.
>
> > > This is with:
> > >
> > >   GNU ld version 2.25-17.fc23
>
> (At best, this is from 2015 ... but yes, min binutils in 2.23.)

Ah, crap! Indeed arch/powerpc/Makefile wraps this in ld-option.

Uh oh, the ppc vdso uses cc-ldoption which was removed! (I think by
me; let me send patches)  How is that not an error?  Yes, guilty,
officer.
commit 055efab3120b ("kbuild: drop support for cc-ldoption").
Did I not know how to use grep, or?  No, it is
commit f2af201002a8 ("powerpc/build: vdso linker warning for orphan sections")
that is wrong.
Nick Desaulniers Sept. 1, 2020, 11:54 p.m. UTC | #4
On Tue, Sep 1, 2020 at 4:18 PM Kees Cook <keescook@chromium.org> wrote:
>
> On Tue, Sep 01, 2020 at 11:02:02AM -0700, Nick Desaulniers wrote:
> > Uh oh, the ppc vdso uses cc-ldoption which was removed! (I think by
> > me; let me send patches)  How is that not an error?  Yes, guilty,
> > officer.
> > commit 055efab3120b ("kbuild: drop support for cc-ldoption").
> > Did I not know how to use grep, or?  No, it is
> > commit f2af201002a8 ("powerpc/build: vdso linker warning for orphan sections")
> > that is wrong.
>
> Eek, yeah, the vdso needs fixing; whoops. Lucky for my series, I only need
> ld-option! ;)
>

I didn't cc everyone here on that thread, but here's the series I sent
for it: https://lore.kernel.org/lkml/20200901222523.1941988-1-ndesaulniers@google.com/T/#u
.
Geert Uytterhoeven Oct. 26, 2020, 12:29 p.m. UTC | #5
Hi Kees,

On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> In preparation for warning on orphan sections, discard
> unwanted non-zero-sized generated sections, and enforce other
> expected-to-be-zero-sized sections (since discarding them might hide
> problems with them suddenly gaining unexpected entries).
>
> Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> Signed-off-by: Kees Cook <keescook@chromium.org>

This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
sections") in v5.10-rc1, and is causing the following error with
renesas_defconfig[1]:

    aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
`kernel/bpf/core.o' being placed in section `.eh_frame'
    aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
    aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!

I cannot reproduce this with the standard arm64 defconfig.

I bisected the error to the aforementioned commit, but understand this
is not the real reason.  If I revert this commit, I still get:

    aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
`arch/arm64/kernel/head.o' being placed in section `.got.plt'
    aarch64-linux-gnu-ld: warning: orphan section `.plt' from
`arch/arm64/kernel/head.o' being placed in section `.plt'
    aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
`arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
    aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
`kernel/bpf/core.o' being placed in section `.eh_frame'

I.e. including the ".eh_frame" warning. I have tried bisecting that
warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
placement"), which is another red herring.

Note that even on plain be2881824ae9eb92, I get:

    aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
    aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!

The parent commit obviously doesn't show that (but probably still has
the problem).

Do you have a clue!

Thanks!

> --- a/arch/arm64/kernel/vmlinux.lds.S
> +++ b/arch/arm64/kernel/vmlinux.lds.S
> @@ -121,6 +121,14 @@ SECTIONS
>                 *(.got)                 /* Global offset table          */
>         }
>
> +       /*
> +        * Make sure that the .got.plt is either completely empty or it
> +        * contains only the lazy dispatch entries.
> +        */
> +       .got.plt : { *(.got.plt) }
> +       ASSERT(SIZEOF(.got.plt) == 0 || SIZEOF(.got.plt) == 0x18,
> +              "Unexpected GOT/PLT entries detected!")
> +
>         . = ALIGN(SEGMENT_ALIGN);
>         _etext = .;                     /* End of text section */
>
> @@ -243,6 +251,18 @@ SECTIONS
>         ELF_DETAILS
>
>         HEAD_SYMBOLS
> +
> +       /*
> +        * Sections that should stay zero sized, which is safer to
> +        * explicitly check instead of blindly discarding.
> +        */
> +       .plt : {
> +               *(.plt) *(.plt.*) *(.iplt) *(.igot)
> +       }
> +       ASSERT(SIZEOF(.plt) == 0, "Unexpected run-time procedure linkages detected!")
> +
> +       .data.rel.ro : { *(.data.rel.ro) }
> +       ASSERT(SIZEOF(.data.rel.ro) == 0, "Unexpected RELRO detected!")
>  }
>
>  #include "image-vars.h"

[1] https://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-devel.git/log/?h=topic/renesas-defconfig

Gr{oetje,eeting}s,

                        Geert


--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
Geert Uytterhoeven Oct. 26, 2020, 1:29 p.m. UTC | #6
Hi Kees,

On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > In preparation for warning on orphan sections, discard
> > unwanted non-zero-sized generated sections, and enforce other
> > expected-to-be-zero-sized sections (since discarding them might hide
> > problems with them suddenly gaining unexpected entries).
> >
> > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > Signed-off-by: Kees Cook <keescook@chromium.org>
>
> This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> sections") in v5.10-rc1, and is causing the following error with
> renesas_defconfig[1]:
>
>     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> `kernel/bpf/core.o' being placed in section `.eh_frame'
>     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
>     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
>
> I cannot reproduce this with the standard arm64 defconfig.
>
> I bisected the error to the aforementioned commit, but understand this
> is not the real reason.  If I revert this commit, I still get:
>
>     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> `arch/arm64/kernel/head.o' being placed in section `.got.plt'
>     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> `arch/arm64/kernel/head.o' being placed in section `.plt'
>     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
>     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> `kernel/bpf/core.o' being placed in section `.eh_frame'
>
> I.e. including the ".eh_frame" warning. I have tried bisecting that
> warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> placement"), which is another red herring.

kernel/bpf/core.o is the only file containing an eh_frame section,
causing the warning.
If I compile core.c with "-g" added, like arm64 defconfig does, the
eh_frame section is no longer emitted.

Hence setting CONFIG_DEBUG_INFO=y, cfr. arm64 defconfig, the warning
is gone, but I'm back to the the "Unexpected GOT/PLT entries" below...

> Note that even on plain be2881824ae9eb92, I get:
>
>     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
>     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
>
> The parent commit obviously doesn't show that (but probably still has
> the problem).

Gr{oetje,eeting}s,

                        Geert
Geert Uytterhoeven Oct. 26, 2020, 4:01 p.m. UTC | #7
On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > > In preparation for warning on orphan sections, discard
> > > unwanted non-zero-sized generated sections, and enforce other
> > > expected-to-be-zero-sized sections (since discarding them might hide
> > > problems with them suddenly gaining unexpected entries).
> > >
> > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > > Signed-off-by: Kees Cook <keescook@chromium.org>
> >
> > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> > sections") in v5.10-rc1, and is causing the following error with
> > renesas_defconfig[1]:
> >
> >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > `kernel/bpf/core.o' being placed in section `.eh_frame'
> >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> >
> > I cannot reproduce this with the standard arm64 defconfig.
> >
> > I bisected the error to the aforementioned commit, but understand this
> > is not the real reason.  If I revert this commit, I still get:
> >
> >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> > `arch/arm64/kernel/head.o' being placed in section `.got.plt'
> >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> > `arch/arm64/kernel/head.o' being placed in section `.plt'
> >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
> >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > `kernel/bpf/core.o' being placed in section `.eh_frame'
> >
> > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > placement"), which is another red herring.
>
> kernel/bpf/core.o is the only file containing an eh_frame section,
> causing the warning.
> If I compile core.c with "-g" added, like arm64 defconfig does, the
> eh_frame section is no longer emitted.
>
> Hence setting CONFIG_DEBUG_INFO=y, cfr. arm64 defconfig, the warning
> is gone, but I'm back to the the "Unexpected GOT/PLT entries" below...
>
> > Note that even on plain be2881824ae9eb92, I get:
> >
> >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> >
> > The parent commit obviously doesn't show that (but probably still has
> > the problem).

Reverting both
b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
seems to solve my problems, without any ill effects?

Gr{oetje,eeting}s,

                        Geert
Ard Biesheuvel Oct. 26, 2020, 5:38 p.m. UTC | #8
On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > > > In preparation for warning on orphan sections, discard
> > > > unwanted non-zero-sized generated sections, and enforce other
> > > > expected-to-be-zero-sized sections (since discarding them might hide
> > > > problems with them suddenly gaining unexpected entries).
> > > >
> > > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > > > Signed-off-by: Kees Cook <keescook@chromium.org>
> > >
> > > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> > > sections") in v5.10-rc1, and is causing the following error with
> > > renesas_defconfig[1]:
> > >
> > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > >
> > > I cannot reproduce this with the standard arm64 defconfig.
> > >
> > > I bisected the error to the aforementioned commit, but understand this
> > > is not the real reason.  If I revert this commit, I still get:
> > >
> > >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> > > `arch/arm64/kernel/head.o' being placed in section `.got.plt'
> > >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> > > `arch/arm64/kernel/head.o' being placed in section `.plt'
> > >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> > > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
> > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > >
> > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > placement"), which is another red herring.
> >
> > kernel/bpf/core.o is the only file containing an eh_frame section,
> > causing the warning.
> > If I compile core.c with "-g" added, like arm64 defconfig does, the
> > eh_frame section is no longer emitted.
> >
> > Hence setting CONFIG_DEBUG_INFO=y, cfr. arm64 defconfig, the warning
> > is gone, but I'm back to the the "Unexpected GOT/PLT entries" below...
> >
> > > Note that even on plain be2881824ae9eb92, I get:
> > >
> > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > >
> > > The parent commit obviously doesn't show that (but probably still has
> > > the problem).
>
> Reverting both
> b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> seems to solve my problems, without any ill effects?
>

I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)

The presence of .data.rel.ro and .got.plt sections suggests that the
toolchain is using -fpie and/or -z relro to build shared objects
rather than a fully linked bare metal binary.

Which toolchain are you using? Does adding -fno-pie to the compiler
command line and/or adding -z norelro to the linker command line make
any difference?
Geert Uytterhoeven Oct. 26, 2020, 5:43 p.m. UTC | #9
Hi Ard,

On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > > > > In preparation for warning on orphan sections, discard
> > > > > unwanted non-zero-sized generated sections, and enforce other
> > > > > expected-to-be-zero-sized sections (since discarding them might hide
> > > > > problems with them suddenly gaining unexpected entries).
> > > > >
> > > > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > > > > Signed-off-by: Kees Cook <keescook@chromium.org>
> > > >
> > > > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> > > > sections") in v5.10-rc1, and is causing the following error with
> > > > renesas_defconfig[1]:
> > > >
> > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > >
> > > > I cannot reproduce this with the standard arm64 defconfig.
> > > >
> > > > I bisected the error to the aforementioned commit, but understand this
> > > > is not the real reason.  If I revert this commit, I still get:
> > > >
> > > >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> > > > `arch/arm64/kernel/head.o' being placed in section `.got.plt'
> > > >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> > > > `arch/arm64/kernel/head.o' being placed in section `.plt'
> > > >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> > > > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
> > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > >
> > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > placement"), which is another red herring.
> > >
> > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > causing the warning.
> > > If I compile core.c with "-g" added, like arm64 defconfig does, the
> > > eh_frame section is no longer emitted.
> > >
> > > Hence setting CONFIG_DEBUG_INFO=y, cfr. arm64 defconfig, the warning
> > > is gone, but I'm back to the the "Unexpected GOT/PLT entries" below...
> > >
> > > > Note that even on plain be2881824ae9eb92, I get:
> > > >
> > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > >
> > > > The parent commit obviously doesn't show that (but probably still has
> > > > the problem).
> >
> > Reverting both
> > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > seems to solve my problems, without any ill effects?
> >
>
> I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
>
> The presence of .data.rel.ro and .got.plt sections suggests that the
> toolchain is using -fpie and/or -z relro to build shared objects
> rather than a fully linked bare metal binary.
>
> Which toolchain are you using? Does adding -fno-pie to the compiler

gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)  from Ubuntu 20.04LTS.

> command line and/or adding -z norelro to the linker command line make
> any difference?

I'll give that a try later...

Gr{oetje,eeting}s,

                        Geert
Nick Desaulniers Oct. 26, 2020, 5:48 p.m. UTC | #10
On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
>
> Hi Ard,
>
> On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > > > > > In preparation for warning on orphan sections, discard
> > > > > > unwanted non-zero-sized generated sections, and enforce other
> > > > > > expected-to-be-zero-sized sections (since discarding them might hide
> > > > > > problems with them suddenly gaining unexpected entries).
> > > > > >
> > > > > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > > > > > Signed-off-by: Kees Cook <keescook@chromium.org>
> > > > >
> > > > > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> > > > > sections") in v5.10-rc1, and is causing the following error with
> > > > > renesas_defconfig[1]:
> > > > >
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > >
> > > > > I cannot reproduce this with the standard arm64 defconfig.
> > > > >
> > > > > I bisected the error to the aforementioned commit, but understand this
> > > > > is not the real reason.  If I revert this commit, I still get:
> > > > >
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> > > > > `arch/arm64/kernel/head.o' being placed in section `.got.plt'
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> > > > > `arch/arm64/kernel/head.o' being placed in section `.plt'
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> > > > > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > >
> > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > > placement"), which is another red herring.
> > > >
> > > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > > causing the warning.

When I see .eh_frame, I think -fno-asynchronous-unwind-tables is
missing from someone's KBUILD_CFLAGS.
But I don't see anything curious in kernel/bpf/Makefile, unless
cc-disable-warning is somehow broken.

> > > > If I compile core.c with "-g" added, like arm64 defconfig does, the
> > > > eh_frame section is no longer emitted.
> > > >
> > > > Hence setting CONFIG_DEBUG_INFO=y, cfr. arm64 defconfig, the warning
> > > > is gone, but I'm back to the the "Unexpected GOT/PLT entries" below...
> > > >
> > > > > Note that even on plain be2881824ae9eb92, I get:
> > > > >
> > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > >
> > > > > The parent commit obviously doesn't show that (but probably still has
> > > > > the problem).
> > >
> > > Reverting both
> > > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > > seems to solve my problems, without any ill effects?
> > >
> >
> > I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
> >
> > The presence of .data.rel.ro and .got.plt sections suggests that the
> > toolchain is using -fpie and/or -z relro to build shared objects
> > rather than a fully linked bare metal binary.
> >
> > Which toolchain are you using? Does adding -fno-pie to the compiler
>
> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)  from Ubuntu 20.04LTS.
>
> > command line and/or adding -z norelro to the linker command line make
> > any difference?
>
> I'll give that a try later...

This patch just got picked up into the for-next branch of the arm64
tree; it enables `-z norelro` regardless of configs.
https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/commit/?h=for-next/core&id=3b92fa7485eba16b05166fddf38ab42f2ff6ab95
If you apply that, that should help you test `-z norelro` quickly.
Geert Uytterhoeven Oct. 26, 2020, 5:53 p.m. UTC | #11
Hi Nick,

On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
> On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven
> <geert@linux-m68k.org> wrote:
> > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > > > > > > In preparation for warning on orphan sections, discard
> > > > > > > unwanted non-zero-sized generated sections, and enforce other
> > > > > > > expected-to-be-zero-sized sections (since discarding them might hide
> > > > > > > problems with them suddenly gaining unexpected entries).
> > > > > > >
> > > > > > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > > > > > > Signed-off-by: Kees Cook <keescook@chromium.org>
> > > > > >
> > > > > > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> > > > > > sections") in v5.10-rc1, and is causing the following error with
> > > > > > renesas_defconfig[1]:
> > > > > >
> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > > >
> > > > > > I cannot reproduce this with the standard arm64 defconfig.
> > > > > >
> > > > > > I bisected the error to the aforementioned commit, but understand this
> > > > > > is not the real reason.  If I revert this commit, I still get:
> > > > > >
> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> > > > > > `arch/arm64/kernel/head.o' being placed in section `.got.plt'
> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> > > > > > `arch/arm64/kernel/head.o' being placed in section `.plt'
> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> > > > > > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > > >
> > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > > > placement"), which is another red herring.
> > > > >
> > > > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > > > causing the warning.
>
> When I see .eh_frame, I think -fno-asynchronous-unwind-tables is
> missing from someone's KBUILD_CFLAGS.
> But I don't see anything curious in kernel/bpf/Makefile, unless
> cc-disable-warning is somehow broken.

Yeah, I noticed it's added in arch/arm64/Makefile, and verified that it is
actually passed when building kernel/bpf/core.o.

> > > > > If I compile core.c with "-g" added, like arm64 defconfig does, the
> > > > > eh_frame section is no longer emitted.
> > > > >
> > > > > Hence setting CONFIG_DEBUG_INFO=y, cfr. arm64 defconfig, the warning
> > > > > is gone, but I'm back to the the "Unexpected GOT/PLT entries" below...
> > > > >
> > > > > > Note that even on plain be2881824ae9eb92, I get:
> > > > > >
> > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > > >
> > > > > > The parent commit obviously doesn't show that (but probably still has
> > > > > > the problem).
> > > >
> > > > Reverting both
> > > > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > > > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > > > seems to solve my problems, without any ill effects?
> > > >
> > >
> > > I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
> > >
> > > The presence of .data.rel.ro and .got.plt sections suggests that the
> > > toolchain is using -fpie and/or -z relro to build shared objects
> > > rather than a fully linked bare metal binary.
> > >
> > > Which toolchain are you using? Does adding -fno-pie to the compiler
> >
> > gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)  from Ubuntu 20.04LTS.
> >
> > > command line and/or adding -z norelro to the linker command line make
> > > any difference?
> >
> > I'll give that a try later...
>
> This patch just got picked up into the for-next branch of the arm64
> tree; it enables `-z norelro` regardless of configs.
> https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/commit/?h=for-next/core&id=3b92fa7485eba16b05166fddf38ab42f2ff6ab95
> If you apply that, that should help you test `-z norelro` quickly.

Thanks, will give that a try, too.

Gr{oetje,eeting}s,

                        Geert
Geert Uytterhoeven Oct. 27, 2020, 8:37 a.m. UTC | #12
Hi Nick,

On Mon, Oct 26, 2020 at 6:53 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> > On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven
> > <geert@linux-m68k.org> wrote:
> > > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > > > > > > > In preparation for warning on orphan sections, discard
> > > > > > > > unwanted non-zero-sized generated sections, and enforce other
> > > > > > > > expected-to-be-zero-sized sections (since discarding them might hide
> > > > > > > > problems with them suddenly gaining unexpected entries).
> > > > > > > >
> > > > > > > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > > > > > > > Signed-off-by: Kees Cook <keescook@chromium.org>
> > > > > > >
> > > > > > > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> > > > > > > sections") in v5.10-rc1, and is causing the following error with
> > > > > > > renesas_defconfig[1]:
> > > > > > >
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > > > >
> > > > > > > I cannot reproduce this with the standard arm64 defconfig.
> > > > > > >
> > > > > > > I bisected the error to the aforementioned commit, but understand this
> > > > > > > is not the real reason.  If I revert this commit, I still get:
> > > > > > >
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> > > > > > > `arch/arm64/kernel/head.o' being placed in section `.got.plt'
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> > > > > > > `arch/arm64/kernel/head.o' being placed in section `.plt'
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> > > > > > > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > > > >
> > > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > > > > placement"), which is another red herring.
> > > > > >
> > > > > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > > > > causing the warning.
> >
> > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is
> > missing from someone's KBUILD_CFLAGS.
> > But I don't see anything curious in kernel/bpf/Makefile, unless
> > cc-disable-warning is somehow broken.
>
> Yeah, I noticed it's added in arch/arm64/Makefile, and verified that it is
> actually passed when building kernel/bpf/core.o.
>
> > > > > > If I compile core.c with "-g" added, like arm64 defconfig does, the
> > > > > > eh_frame section is no longer emitted.
> > > > > >
> > > > > > Hence setting CONFIG_DEBUG_INFO=y, cfr. arm64 defconfig, the warning
> > > > > > is gone, but I'm back to the the "Unexpected GOT/PLT entries" below...
> > > > > >
> > > > > > > Note that even on plain be2881824ae9eb92, I get:
> > > > > > >
> > > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > > > >
> > > > > > > The parent commit obviously doesn't show that (but probably still has
> > > > > > > the problem).
> > > > >
> > > > > Reverting both
> > > > > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > > > > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > > > > seems to solve my problems, without any ill effects?
> > > > >
> > > >
> > > > I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
> > > >
> > > > The presence of .data.rel.ro and .got.plt sections suggests that the
> > > > toolchain is using -fpie and/or -z relro to build shared objects
> > > > rather than a fully linked bare metal binary.
> > > >
> > > > Which toolchain are you using? Does adding -fno-pie to the compiler
> > >
> > > gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)  from Ubuntu 20.04LTS.
> > >
> > > > command line and/or adding -z norelro to the linker command line make
> > > > any difference?
> > >
> > > I'll give that a try later...
> >
> > This patch just got picked up into the for-next branch of the arm64
> > tree; it enables `-z norelro` regardless of configs.
> > https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/commit/?h=for-next/core&id=3b92fa7485eba16b05166fddf38ab42f2ff6ab95
> > If you apply that, that should help you test `-z norelro` quickly.
>
> Thanks, will give that a try, too.

Commit 3b92fa7485eba16b05166fddf38ab42f2ff6ab95 is part of v5.10-rc1,
so it was already included, and thus doesn't fix the issue.

Gr{oetje,eeting}s,

                        Geert
Geert Uytterhoeven Oct. 27, 2020, 8:51 a.m. UTC | #13
Hi Ard,

On Mon, Oct 26, 2020 at 6:43 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > > > > > In preparation for warning on orphan sections, discard
> > > > > > unwanted non-zero-sized generated sections, and enforce other
> > > > > > expected-to-be-zero-sized sections (since discarding them might hide
> > > > > > problems with them suddenly gaining unexpected entries).
> > > > > >
> > > > > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > > > > > Signed-off-by: Kees Cook <keescook@chromium.org>
> > > > >
> > > > > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> > > > > sections") in v5.10-rc1, and is causing the following error with
> > > > > renesas_defconfig[1]:
> > > > >
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > >
> > > > > I cannot reproduce this with the standard arm64 defconfig.
> > > > >
> > > > > I bisected the error to the aforementioned commit, but understand this
> > > > > is not the real reason.  If I revert this commit, I still get:
> > > > >
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> > > > > `arch/arm64/kernel/head.o' being placed in section `.got.plt'
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> > > > > `arch/arm64/kernel/head.o' being placed in section `.plt'
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> > > > > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
> > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > >
> > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > > placement"), which is another red herring.
> > > >
> > > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > > causing the warning.
> > > > If I compile core.c with "-g" added, like arm64 defconfig does, the
> > > > eh_frame section is no longer emitted.
> > > >
> > > > Hence setting CONFIG_DEBUG_INFO=y, cfr. arm64 defconfig, the warning
> > > > is gone, but I'm back to the the "Unexpected GOT/PLT entries" below...
> > > >
> > > > > Note that even on plain be2881824ae9eb92, I get:
> > > > >
> > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > >
> > > > > The parent commit obviously doesn't show that (but probably still has
> > > > > the problem).
> > >
> > > Reverting both
> > > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > > seems to solve my problems, without any ill effects?
> > >
> >
> > I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
> >
> > The presence of .data.rel.ro and .got.plt sections suggests that the
> > toolchain is using -fpie and/or -z relro to build shared objects
> > rather than a fully linked bare metal binary.
> >
> > Which toolchain are you using? Does adding -fno-pie to the compiler
>
> gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)  from Ubuntu 20.04LTS.
>
> > command line and/or adding -z norelro to the linker command line make
> > any difference?
>
> I'll give that a try later...

Adding -fno-pie to KBUILD_AFLAGS and KBUILD_CFLAGS doesn't
make a difference.

Same for adding -z norelno to the final link command:

    aarch64-linux-gnu-ld: warning: -z norelno ignored
    aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
`kernel/bpf/core.o' being placed in section `.eh_frame'
    aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
    aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!

Gr{oetje,eeting}s,

                        Geert
Jean-Philippe Brucker Oct. 27, 2020, 10:08 a.m. UTC | #14
Hi,

On Mon, Oct 26, 2020 at 06:38:46PM +0100, Ard Biesheuvel wrote:
> > > > Note that even on plain be2881824ae9eb92, I get:
> > > >
> > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > >
> > > > The parent commit obviously doesn't show that (but probably still has
> > > > the problem).
> >
> > Reverting both
> > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > seems to solve my problems, without any ill effects?
> >
> 
> I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)

I have the same problem with one of my debug configs and Linux v5.10-rc1,
and can reproduce with the Debian 8.3.0 toolchain, by using the arm64
defconfig and disabling CONFIG_MODULES:

ld -EL -maarch64elf --no-undefined -X -z norelro -shared -Bsymbolic -z notext --no-apply-dynamic-relocs --fix-cortex-a53-843419 --orphan-handling=warn --build-id=sha1 --strip-debug -o .tmp_vmlinux.kallsyms1 -T ./arch/arm64/kernel/vmlinux.lds --whole-archive arch/arm64/kernel/head.o init/built-in.a usr/built-in.a arch/arm64/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a arch/arm64/lib/built-in.a lib/built-in.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive --start-group arch/arm64/lib/lib.a lib/lib.a ./drivers/firmware/efi/libstub/lib.a --end-group
ld: Unexpected GOT/PLT entries detected!
ld: Unexpected run-time procedure linkages detected!

Adding -fno-pie to this command doesn't fix the problem.

Note that when cross-building with a GCC 10.2 and binutils 2.35.1 I also
get several "aarch64-linux-gnu-ld: warning: -z norelro ignored" in
addition to the error, but I don't get that warning with the 8.3.0
toolchain.

Thanks,
Jean

> 
> The presence of .data.rel.ro and .got.plt sections suggests that the
> toolchain is using -fpie and/or -z relro to build shared objects
> rather than a fully linked bare metal binary.
> 
> Which toolchain are you using? Does adding -fno-pie to the compiler
> command line and/or adding -z norelro to the linker command line make
> any difference?
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Geert Uytterhoeven Oct. 27, 2020, 10:20 a.m. UTC | #15
Hi Jean-Philippe,

On Tue, Oct 27, 2020 at 11:09 AM Jean-Philippe Brucker
<jean-philippe@linaro.org> wrote:
> On Mon, Oct 26, 2020 at 06:38:46PM +0100, Ard Biesheuvel wrote:
> > > > > Note that even on plain be2881824ae9eb92, I get:
> > > > >
> > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > >
> > > > > The parent commit obviously doesn't show that (but probably still has
> > > > > the problem).
> > >
> > > Reverting both
> > > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > > seems to solve my problems, without any ill effects?
> > >
> >
> > I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
>
> I have the same problem with one of my debug configs and Linux v5.10-rc1,
> and can reproduce with the Debian 8.3.0 toolchain, by using the arm64
> defconfig and disabling CONFIG_MODULES:
>
> ld -EL -maarch64elf --no-undefined -X -z norelro -shared -Bsymbolic -z notext --no-apply-dynamic-relocs --fix-cortex-a53-843419 --orphan-handling=warn --build-id=sha1 --strip-debug -o .tmp_vmlinux.kallsyms1 -T ./arch/arm64/kernel/vmlinux.lds --whole-archive arch/arm64/kernel/head.o init/built-in.a usr/built-in.a arch/arm64/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a arch/arm64/lib/built-in.a lib/built-in.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive --start-group arch/arm64/lib/lib.a lib/lib.a ./drivers/firmware/efi/libstub/lib.a --end-group
> ld: Unexpected GOT/PLT entries detected!
> ld: Unexpected run-time procedure linkages detected!
>
> Adding -fno-pie to this command doesn't fix the problem.
>
> Note that when cross-building with a GCC 10.2 and binutils 2.35.1 I also
> get several "aarch64-linux-gnu-ld: warning: -z norelro ignored" in
> addition to the error, but I don't get that warning with the 8.3.0
> toolchain.

Thanks, my config (renesas_defconfig) also had CONFIG_MODULES disabled.
Enabling that fixes the link error due to unexpected entries, but the
.eh_frame orphan section warning is still there.

Gr{oetje,eeting}s,

                        Geert
Ard Biesheuvel Oct. 27, 2020, 11:29 a.m. UTC | #16
On Tue, 27 Oct 2020 at 11:20, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Hi Jean-Philippe,
>
> On Tue, Oct 27, 2020 at 11:09 AM Jean-Philippe Brucker
> <jean-philippe@linaro.org> wrote:
> > On Mon, Oct 26, 2020 at 06:38:46PM +0100, Ard Biesheuvel wrote:
> > > > > > Note that even on plain be2881824ae9eb92, I get:
> > > > > >
> > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > > >
> > > > > > The parent commit obviously doesn't show that (but probably still has
> > > > > > the problem).
> > > >
> > > > Reverting both
> > > > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > > > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > > > seems to solve my problems, without any ill effects?
> > > >
> > >
> > > I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
> >
> > I have the same problem with one of my debug configs and Linux v5.10-rc1,
> > and can reproduce with the Debian 8.3.0 toolchain, by using the arm64
> > defconfig and disabling CONFIG_MODULES:
> >
> > ld -EL -maarch64elf --no-undefined -X -z norelro -shared -Bsymbolic -z notext --no-apply-dynamic-relocs --fix-cortex-a53-843419 --orphan-handling=warn --build-id=sha1 --strip-debug -o .tmp_vmlinux.kallsyms1 -T ./arch/arm64/kernel/vmlinux.lds --whole-archive arch/arm64/kernel/head.o init/built-in.a usr/built-in.a arch/arm64/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a arch/arm64/lib/built-in.a lib/built-in.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive --start-group arch/arm64/lib/lib.a lib/lib.a ./drivers/firmware/efi/libstub/lib.a --end-group
> > ld: Unexpected GOT/PLT entries detected!
> > ld: Unexpected run-time procedure linkages detected!
> >
> > Adding -fno-pie to this command doesn't fix the problem.
> >
> > Note that when cross-building with a GCC 10.2 and binutils 2.35.1 I also
> > get several "aarch64-linux-gnu-ld: warning: -z norelro ignored" in
> > addition to the error, but I don't get that warning with the 8.3.0
> > toolchain.
>
> Thanks, my config (renesas_defconfig) also had CONFIG_MODULES disabled.
> Enabling that fixes the link error due to unexpected entries, but the
> .eh_frame orphan section warning is still there.
>

Looks like this is caused by the VFIO driver doing nasty things with
symbol_get(), resulting in weak symbol references being emitted. Since
taking the address of a weak symbol can yield NULL, the only way for
the linker to accommodate this is to use GOT indirection for the
direct symbol reference, so that the GOT entry can be set to NULL if
the reference is not satisfied at link time.
Naresh Kamboju Oct. 27, 2020, 11:53 a.m. UTC | #17
On Tue, 27 Oct 2020 at 17:00, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Tue, 27 Oct 2020 at 11:20, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> >
> > Hi Jean-Philippe,
> >
> > On Tue, Oct 27, 2020 at 11:09 AM Jean-Philippe Brucker
> > <jean-philippe@linaro.org> wrote:
> > > On Mon, Oct 26, 2020 at 06:38:46PM +0100, Ard Biesheuvel wrote:
> > > > > > > Note that even on plain be2881824ae9eb92, I get:
> > > > > > >
> > > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > > > >
> > > > > > > The parent commit obviously doesn't show that (but probably still has
> > > > > > > the problem).
> > > > >
> > > > > Reverting both
> > > > > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > > > > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > > > > seems to solve my problems, without any ill effects?
> > > > >
> > > >
> > > > I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
> > >
> > > I have the same problem with one of my debug configs and Linux v5.10-rc1,
> > > and can reproduce with the Debian 8.3.0 toolchain, by using the arm64
> > > defconfig and disabling CONFIG_MODULES:
> > >
> > > ld -EL -maarch64elf --no-undefined -X -z norelro -shared -Bsymbolic -z notext --no-apply-dynamic-relocs --fix-cortex-a53-843419 --orphan-handling=warn --build-id=sha1 --strip-debug -o .tmp_vmlinux.kallsyms1 -T ./arch/arm64/kernel/vmlinux.lds --whole-archive arch/arm64/kernel/head.o init/built-in.a usr/built-in.a arch/arm64/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a arch/arm64/lib/built-in.a lib/built-in.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive --start-group arch/arm64/lib/lib.a lib/lib.a ./drivers/firmware/efi/libstub/lib.a --end-group
> > > ld: Unexpected GOT/PLT entries detected!
> > > ld: Unexpected run-time procedure linkages detected!

 ld: Unexpected GOT/PLT entries detected!
 ld: Unexpected run-time procedure linkages detected!

The arm64 build error fixed by (I have tested defconfig)

[PATCH] soc: qcom: QCOM_RPMH fix build with modular QCOM_RPMH
https://lore.kernel.org/linux-arm-msm/20201027111422.4008114-1-anders.roxell@linaro.org/
---

When building allmodconfig leading to the following link error with
CONFIG_QCOM_RPMH=y and CONFIG_QCOM_COMMAND_DB=m:

aarch64-linux-gnu-ld: drivers/clk/qcom/clk-rpmh.o: in function `clk_rpmh_probe':
  drivers/clk/qcom/clk-rpmh.c:474: undefined reference to `cmd_db_read_addr'
  drivers/clk/qcom/clk-rpmh.c:474:(.text+0x254): relocation truncated
to fit: R_AARCH64_CALL26 against undefined symbol `cmd_db_read_addr'

Fix this by adding a Kconfig depenency and forcing QCOM_RPMH to be a
module when QCOM_COMMAND_DB is a module. Also removing the dependency on
'ARCH_QCOM || COMPILE_TEST' since that is already a dependency for
QCOM_COMMAND_DB.

Fixes: 778279f4f5e4 ("soc: qcom: cmd-db: allow loading as a module")
Signed-off-by: Anders Roxell <anders.roxell@linaro.org>
---
 drivers/soc/qcom/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/soc/qcom/Kconfig b/drivers/soc/qcom/Kconfig
index 9b4ae9c16ba7..3bdd1604f78f 100644
--- a/drivers/soc/qcom/Kconfig
+++ b/drivers/soc/qcom/Kconfig
@@ -109,7 +109,7 @@ config QCOM_RMTFS_MEM

 config QCOM_RPMH
  tristate "Qualcomm RPM-Hardened (RPMH) Communication"
- depends on ARCH_QCOM || COMPILE_TEST
+ depends on QCOM_COMMAND_DB
  help
   Support for communication with the hardened-RPM blocks in
   Qualcomm Technologies Inc (QTI) SoCs. RPMH communication uses an
Ard Biesheuvel Oct. 27, 2020, 1:56 p.m. UTC | #18
On Tue, 27 Oct 2020 at 12:29, Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Tue, 27 Oct 2020 at 11:20, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> >
> > Hi Jean-Philippe,
> >
> > On Tue, Oct 27, 2020 at 11:09 AM Jean-Philippe Brucker
> > <jean-philippe@linaro.org> wrote:
> > > On Mon, Oct 26, 2020 at 06:38:46PM +0100, Ard Biesheuvel wrote:
> > > > > > > Note that even on plain be2881824ae9eb92, I get:
> > > > > > >
> > > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > > > >
> > > > > > > The parent commit obviously doesn't show that (but probably still has
> > > > > > > the problem).
> > > > >
> > > > > Reverting both
> > > > > b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section placement")
> > > > > be2881824ae9eb92 ("arm64/build: Assert for unwanted sections")
> > > > > seems to solve my problems, without any ill effects?
> > > > >
> > > >
> > > > I cannot reproduce the issue here with my distro GCC+binutils (Debian 8.3.0)
> > >
> > > I have the same problem with one of my debug configs and Linux v5.10-rc1,
> > > and can reproduce with the Debian 8.3.0 toolchain, by using the arm64
> > > defconfig and disabling CONFIG_MODULES:
> > >
> > > ld -EL -maarch64elf --no-undefined -X -z norelro -shared -Bsymbolic -z notext --no-apply-dynamic-relocs --fix-cortex-a53-843419 --orphan-handling=warn --build-id=sha1 --strip-debug -o .tmp_vmlinux.kallsyms1 -T ./arch/arm64/kernel/vmlinux.lds --whole-archive arch/arm64/kernel/head.o init/built-in.a usr/built-in.a arch/arm64/built-in.a kernel/built-in.a certs/built-in.a mm/built-in.a fs/built-in.a ipc/built-in.a security/built-in.a crypto/built-in.a block/built-in.a arch/arm64/lib/built-in.a lib/built-in.a drivers/built-in.a sound/built-in.a net/built-in.a virt/built-in.a --no-whole-archive --start-group arch/arm64/lib/lib.a lib/lib.a ./drivers/firmware/efi/libstub/lib.a --end-group
> > > ld: Unexpected GOT/PLT entries detected!
> > > ld: Unexpected run-time procedure linkages detected!
> > >
> > > Adding -fno-pie to this command doesn't fix the problem.
> > >
> > > Note that when cross-building with a GCC 10.2 and binutils 2.35.1 I also
> > > get several "aarch64-linux-gnu-ld: warning: -z norelro ignored" in
> > > addition to the error, but I don't get that warning with the 8.3.0
> > > toolchain.
> >
> > Thanks, my config (renesas_defconfig) also had CONFIG_MODULES disabled.
> > Enabling that fixes the link error due to unexpected entries, but the
> > .eh_frame orphan section warning is still there.
> >
>
> Looks like this is caused by the VFIO driver doing nasty things with
> symbol_get(), resulting in weak symbol references being emitted. Since
> taking the address of a weak symbol can yield NULL, the only way for
> the linker to accommodate this is to use GOT indirection for the
> direct symbol reference, so that the GOT entry can be set to NULL if
> the reference is not satisfied at link time.

This seems to do the trick for me.

diff --git a/include/linux/module.h b/include/linux/module.h
index 7ccdf87f376f..6264617bab4d 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -740,7 +740,7 @@ static inline bool within_module(unsigned long
addr, const struct module *mod)
 }

 /* Get/put a kernel symbol (calls should be symmetric) */
-#define symbol_get(x) ({ extern typeof(x) x __attribute__((weak)); &(x); })
+#define symbol_get(x) ({ extern typeof(x) x
__attribute__((weak,visibility("hidden"))); &(x); })
 #define symbol_put(x) do { } while (0)
 #define symbol_put_addr(x) do { } while (0)
Geert Uytterhoeven Oct. 27, 2020, 7:25 p.m. UTC | #19
Hi Nick,

CC Josh

On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
> On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven

> <geert@linux-m68k.org> wrote:

> > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:

> > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:

> > > > > > > In preparation for warning on orphan sections, discard

> > > > > > > unwanted non-zero-sized generated sections, and enforce other

> > > > > > > expected-to-be-zero-sized sections (since discarding them might hide

> > > > > > > problems with them suddenly gaining unexpected entries).

> > > > > > >

> > > > > > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>

> > > > > > > Signed-off-by: Kees Cook <keescook@chromium.org>

> > > > > >

> > > > > > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted

> > > > > > sections") in v5.10-rc1, and is causing the following error with

> > > > > > renesas_defconfig[1]:

> > > > > >

> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from

> > > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'

> > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!

> > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!

> > > > > >

> > > > > > I cannot reproduce this with the standard arm64 defconfig.

> > > > > >

> > > > > > I bisected the error to the aforementioned commit, but understand this

> > > > > > is not the real reason.  If I revert this commit, I still get:

> > > > > >

> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from

> > > > > > `arch/arm64/kernel/head.o' being placed in section `.got.plt'

> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from

> > > > > > `arch/arm64/kernel/head.o' being placed in section `.plt'

> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from

> > > > > > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'

> > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from

> > > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'

> > > > > >

> > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that

> > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to

> > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section

> > > > > > placement"), which is another red herring.

> > > > >

> > > > > kernel/bpf/core.o is the only file containing an eh_frame section,

> > > > > causing the warning.

>

> When I see .eh_frame, I think -fno-asynchronous-unwind-tables is

> missing from someone's KBUILD_CFLAGS.

> But I don't see anything curious in kernel/bpf/Makefile, unless

> cc-disable-warning is somehow broken.


I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged
with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).

Even if the function is trivially empty ("return 0;"), a ".eh_frame" section
is generated.  Removing the __no_fgcse tag fixes that.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
Ard Biesheuvel Oct. 27, 2020, 7:33 p.m. UTC | #20
On Tue, 27 Oct 2020 at 20:25, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Hi Nick,
>
> CC Josh
>
> On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> > On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven
> > <geert@linux-m68k.org> wrote:
> > > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > On Fri, Aug 21, 2020 at 9:56 PM Kees Cook <keescook@chromium.org> wrote:
> > > > > > > > In preparation for warning on orphan sections, discard
> > > > > > > > unwanted non-zero-sized generated sections, and enforce other
> > > > > > > > expected-to-be-zero-sized sections (since discarding them might hide
> > > > > > > > problems with them suddenly gaining unexpected entries).
> > > > > > > >
> > > > > > > > Suggested-by: Ard Biesheuvel <ardb@kernel.org>
> > > > > > > > Signed-off-by: Kees Cook <keescook@chromium.org>
> > > > > > >
> > > > > > > This is now commit be2881824ae9eb92 ("arm64/build: Assert for unwanted
> > > > > > > sections") in v5.10-rc1, and is causing the following error with
> > > > > > > renesas_defconfig[1]:
> > > > > > >
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > > > >     aarch64-linux-gnu-ld: Unexpected GOT/PLT entries detected!
> > > > > > >     aarch64-linux-gnu-ld: Unexpected run-time procedure linkages detected!
> > > > > > >
> > > > > > > I cannot reproduce this with the standard arm64 defconfig.
> > > > > > >
> > > > > > > I bisected the error to the aforementioned commit, but understand this
> > > > > > > is not the real reason.  If I revert this commit, I still get:
> > > > > > >
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.got.plt' from
> > > > > > > `arch/arm64/kernel/head.o' being placed in section `.got.plt'
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.plt' from
> > > > > > > `arch/arm64/kernel/head.o' being placed in section `.plt'
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.data.rel.ro' from
> > > > > > > `arch/arm64/kernel/head.o' being placed in section `.data.rel.ro'
> > > > > > >     aarch64-linux-gnu-ld: warning: orphan section `.eh_frame' from
> > > > > > > `kernel/bpf/core.o' being placed in section `.eh_frame'
> > > > > > >
> > > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > > > > placement"), which is another red herring.
> > > > > >
> > > > > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > > > > causing the warning.
> >
> > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is
> > missing from someone's KBUILD_CFLAGS.
> > But I don't see anything curious in kernel/bpf/Makefile, unless
> > cc-disable-warning is somehow broken.
>
> I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged
> with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).
>
> Even if the function is trivially empty ("return 0;"), a ".eh_frame" section
> is generated.  Removing the __no_fgcse tag fixes that.
>


Given that it was added for issues related to retpolines, ORC and
objtool, it should be safe to make that annotation x86-only.
Arvind Sankar Oct. 27, 2020, 8 p.m. UTC | #21
On Tue, Oct 27, 2020 at 08:33:00PM +0100, Ard Biesheuvel wrote:
> On Tue, 27 Oct 2020 at 20:25, Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > >

> > > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is

> > > missing from someone's KBUILD_CFLAGS.

> > > But I don't see anything curious in kernel/bpf/Makefile, unless

> > > cc-disable-warning is somehow broken.

> >

> > I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged

> > with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).

> >

> > Even if the function is trivially empty ("return 0;"), a ".eh_frame" section

> > is generated.  Removing the __no_fgcse tag fixes that.

> >

> 

> 

> Given that it was added for issues related to retpolines, ORC and

> objtool, it should be safe to make that annotation x86-only.


The optimize attribute is not meant for production use. I had mentioned
this at the time but it got lost: the optimize attribute apparently does
not add options, it replaces them completely. So I'm guessing this one
is dropping the -fno-asynchronous-unwind-tables and causing the eh_frame
sections, though I don't know why that doesn't cause eh_frame on x86?

https://lore.kernel.org/lkml/alpine.LSU.2.21.2004151445520.11688@wotan.suse.de/
Nick Desaulniers Oct. 27, 2020, 8:12 p.m. UTC | #22
On Tue, Oct 27, 2020 at 12:25 PM Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
>
> Hi Nick,
>
> CC Josh
>
> On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers
> <ndesaulniers@google.com> wrote:
> > On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven
> > <geert@linux-m68k.org> wrote:
> > > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > > > > placement"), which is another red herring.
> > > > > >
> > > > > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > > > > causing the warning.
> >
> > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is
> > missing from someone's KBUILD_CFLAGS.
> > But I don't see anything curious in kernel/bpf/Makefile, unless
> > cc-disable-warning is somehow broken.
>
> I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged
> with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).
>
> Even if the function is trivially empty ("return 0;"), a ".eh_frame" section
> is generated.  Removing the __no_fgcse tag fixes that.

That's weird.  I feel pretty strongly that unless we're working around
a well understood compiler bug with a comment that links to a
submitted bug report, turning off rando compiler optimizations is a
terrible hack for which one must proceed straight to jail; do not pass
go; do not collect $200.  But maybe I'd feel differently for this case
given the context of the change that added it.  (Ard mentions
retpolines+orc+objtool; can someone share the relevant SHA if you have
it handy so I don't have to go digging?)  (I feel the same about there
being an empty asm(); statement in the definition of asm_volatile_goto
for compiler-gcc.h).  Might be time to "fix the compiler."

(It sounds like Arvind is both in agreement with my sentiment, and has
the root cause).

--
Thanks,
~Nick Desaulniers
Ard Biesheuvel Oct. 27, 2020, 8:15 p.m. UTC | #23
On Tue, 27 Oct 2020 at 21:12, Nick Desaulniers <ndesaulniers@google.com> wrote:
>

> On Tue, Oct 27, 2020 at 12:25 PM Geert Uytterhoeven

> <geert@linux-m68k.org> wrote:

> >

> > Hi Nick,

> >

> > CC Josh

> >

> > On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers

> > <ndesaulniers@google.com> wrote:

> > > On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven

> > > <geert@linux-m68k.org> wrote:

> > > > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:

> > > > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that

> > > > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to

> > > > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section

> > > > > > > > placement"), which is another red herring.

> > > > > > >

> > > > > > > kernel/bpf/core.o is the only file containing an eh_frame section,

> > > > > > > causing the warning.

> > >

> > > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is

> > > missing from someone's KBUILD_CFLAGS.

> > > But I don't see anything curious in kernel/bpf/Makefile, unless

> > > cc-disable-warning is somehow broken.

> >

> > I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged

> > with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).

> >

> > Even if the function is trivially empty ("return 0;"), a ".eh_frame" section

> > is generated.  Removing the __no_fgcse tag fixes that.

>

> That's weird.  I feel pretty strongly that unless we're working around

> a well understood compiler bug with a comment that links to a

> submitted bug report, turning off rando compiler optimizations is a

> terrible hack for which one must proceed straight to jail; do not pass

> go; do not collect $200.  But maybe I'd feel differently for this case

> given the context of the change that added it.  (Ard mentions

> retpolines+orc+objtool; can someone share the relevant SHA if you have

> it handy so I don't have to go digging?)


commit 3193c0836f203a91bef96d88c64cccf0be090d9c
Author: Josh Poimboeuf <jpoimboe@redhat.com>
Date:   Wed Jul 17 20:36:45 2019 -0500

    bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()

has

Fixes: e55a73251da3 ("bpf: Fix ORC unwinding in non-JIT BPF code")

and mentions objtool and CONFIG_RETPOLINE.

>  (I feel the same about there

> being an empty asm(); statement in the definition of asm_volatile_goto

> for compiler-gcc.h).  Might be time to "fix the compiler."

>

> (It sounds like Arvind is both in agreement with my sentiment, and has

> the root cause).

>


I agree that the __no_fgcse hack is terrible. Does Clang support the
following pragmas?

#pragma GCC push_options
#pragma GCC optimize ("-fno-gcse")
#pragma GCC pop_options

?
Nick Desaulniers Oct. 27, 2020, 8:17 p.m. UTC | #24
On Tue, Oct 27, 2020 at 1:15 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> On Tue, 27 Oct 2020 at 21:12, Nick Desaulniers <ndesaulniers@google.com> wrote:
> >
> > On Tue, Oct 27, 2020 at 12:25 PM Geert Uytterhoeven
> > <geert@linux-m68k.org> wrote:
> > >
> > > Hi Nick,
> > >
> > > CC Josh
> > >
> > > On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers
> > > <ndesaulniers@google.com> wrote:
> > > > On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven
> > > > <geert@linux-m68k.org> wrote:
> > > > > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > > > > > > placement"), which is another red herring.
> > > > > > > >
> > > > > > > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > > > > > > causing the warning.
> > > >
> > > > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is
> > > > missing from someone's KBUILD_CFLAGS.
> > > > But I don't see anything curious in kernel/bpf/Makefile, unless
> > > > cc-disable-warning is somehow broken.
> > >
> > > I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged
> > > with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).
> > >
> > > Even if the function is trivially empty ("return 0;"), a ".eh_frame" section
> > > is generated.  Removing the __no_fgcse tag fixes that.
> >
> > That's weird.  I feel pretty strongly that unless we're working around
> > a well understood compiler bug with a comment that links to a
> > submitted bug report, turning off rando compiler optimizations is a
> > terrible hack for which one must proceed straight to jail; do not pass
> > go; do not collect $200.  But maybe I'd feel differently for this case
> > given the context of the change that added it.  (Ard mentions
> > retpolines+orc+objtool; can someone share the relevant SHA if you have
> > it handy so I don't have to go digging?)
>
> commit 3193c0836f203a91bef96d88c64cccf0be090d9c
> Author: Josh Poimboeuf <jpoimboe@redhat.com>
> Date:   Wed Jul 17 20:36:45 2019 -0500
>
>     bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()
>
> has
>
> Fixes: e55a73251da3 ("bpf: Fix ORC unwinding in non-JIT BPF code")
>
> and mentions objtool and CONFIG_RETPOLINE.
>
> >  (I feel the same about there
> > being an empty asm(); statement in the definition of asm_volatile_goto
> > for compiler-gcc.h).  Might be time to "fix the compiler."
> >
> > (It sounds like Arvind is both in agreement with my sentiment, and has
> > the root cause).
> >
>
> I agree that the __no_fgcse hack is terrible. Does Clang support the
> following pragmas?
>
> #pragma GCC push_options
> #pragma GCC optimize ("-fno-gcse")
> #pragma GCC pop_options
>
> ?

Put it in godbolt.org.  Pretty sure it's `#pragma clang` though.
`#pragma GCC` might be supported in clang or silently ignored, but
IIRC pragmas were a bit of a compat nightmare.  I think Arnd wrote
some macros to set pragmas based on toolchain.  (Uses _Pragma, for
pragmas in macros, IIRC).
Nick Desaulniers Oct. 27, 2020, 8:28 p.m. UTC | #25
(+ right linux-toolchains mailing list, apologies for adding the wrong
one, I'm forever doomed to have gmail autocomplete to the wrong one
now that I've sent to it before)

On Tue, Oct 27, 2020 at 1:15 PM Ard Biesheuvel <ardb@kernel.org> wrote:
>

> On Tue, 27 Oct 2020 at 21:12, Nick Desaulniers <ndesaulniers@google.com> wrote:

> >

> > On Tue, Oct 27, 2020 at 12:25 PM Geert Uytterhoeven

> > <geert@linux-m68k.org> wrote:

> > >

> > > Hi Nick,

> > >

> > > CC Josh

> > >

> > > On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers

> > > <ndesaulniers@google.com> wrote:

> > > > On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven

> > > > <geert@linux-m68k.org> wrote:

> > > > > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:

> > > > > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that

> > > > > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to

> > > > > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section

> > > > > > > > > placement"), which is another red herring.

> > > > > > > >

> > > > > > > > kernel/bpf/core.o is the only file containing an eh_frame section,

> > > > > > > > causing the warning.

> > > >

> > > > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is

> > > > missing from someone's KBUILD_CFLAGS.

> > > > But I don't see anything curious in kernel/bpf/Makefile, unless

> > > > cc-disable-warning is somehow broken.

> > >

> > > I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged

> > > with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).

> > >

> > > Even if the function is trivially empty ("return 0;"), a ".eh_frame" section

> > > is generated.  Removing the __no_fgcse tag fixes that.

> >

> > That's weird.  I feel pretty strongly that unless we're working around

> > a well understood compiler bug with a comment that links to a

> > submitted bug report, turning off rando compiler optimizations is a

> > terrible hack for which one must proceed straight to jail; do not pass

> > go; do not collect $200.  But maybe I'd feel differently for this case

> > given the context of the change that added it.  (Ard mentions

> > retpolines+orc+objtool; can someone share the relevant SHA if you have

> > it handy so I don't have to go digging?)

>

> commit 3193c0836f203a91bef96d88c64cccf0be090d9c

> Author: Josh Poimboeuf <jpoimboe@redhat.com>

> Date:   Wed Jul 17 20:36:45 2019 -0500

>

>     bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()

>

> has

>

> Fixes: e55a73251da3 ("bpf: Fix ORC unwinding in non-JIT BPF code")

>

> and mentions objtool and CONFIG_RETPOLINE.


Thanks for the context.  It might be time to revisit the above commit.
If I revert it (small conflict that's easy to fixup),
kernel/bpf/core.o builds cleanly with defconfig+GCC-9.3, so maybe
obtool did get smart enough to handle that case?  Probably regresses
the performance of that main dispatch loop for BPF, but not sure what
folks are expecting when retpolines are enabled.
-- 
Thanks,
~Nick Desaulniers
Arvind Sankar Oct. 27, 2020, 8:30 p.m. UTC | #26
On Tue, Oct 27, 2020 at 01:17:55PM -0700, Nick Desaulniers wrote:
> On Tue, Oct 27, 2020 at 1:15 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> >
> > On Tue, 27 Oct 2020 at 21:12, Nick Desaulniers <ndesaulniers@google.com> wrote:
> > >
> > > On Tue, Oct 27, 2020 at 12:25 PM Geert Uytterhoeven
> > > <geert@linux-m68k.org> wrote:
> > > >
> > > > Hi Nick,
> > > >
> > > > CC Josh
> > > >
> > > > On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers
> > > > <ndesaulniers@google.com> wrote:
> > > > > On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven
> > > > > <geert@linux-m68k.org> wrote:
> > > > > > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:
> > > > > > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
> > > > > > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that
> > > > > > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to
> > > > > > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section
> > > > > > > > > > placement"), which is another red herring.
> > > > > > > > >
> > > > > > > > > kernel/bpf/core.o is the only file containing an eh_frame section,
> > > > > > > > > causing the warning.
> > > > >
> > > > > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is
> > > > > missing from someone's KBUILD_CFLAGS.
> > > > > But I don't see anything curious in kernel/bpf/Makefile, unless
> > > > > cc-disable-warning is somehow broken.
> > > >
> > > > I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged
> > > > with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).
> > > >
> > > > Even if the function is trivially empty ("return 0;"), a ".eh_frame" section
> > > > is generated.  Removing the __no_fgcse tag fixes that.
> > >
> > > That's weird.  I feel pretty strongly that unless we're working around
> > > a well understood compiler bug with a comment that links to a
> > > submitted bug report, turning off rando compiler optimizations is a
> > > terrible hack for which one must proceed straight to jail; do not pass
> > > go; do not collect $200.  But maybe I'd feel differently for this case
> > > given the context of the change that added it.  (Ard mentions
> > > retpolines+orc+objtool; can someone share the relevant SHA if you have
> > > it handy so I don't have to go digging?)
> >
> > commit 3193c0836f203a91bef96d88c64cccf0be090d9c
> > Author: Josh Poimboeuf <jpoimboe@redhat.com>
> > Date:   Wed Jul 17 20:36:45 2019 -0500
> >
> >     bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()
> >
> > has
> >
> > Fixes: e55a73251da3 ("bpf: Fix ORC unwinding in non-JIT BPF code")
> >
> > and mentions objtool and CONFIG_RETPOLINE.
> >
> > >  (I feel the same about there
> > > being an empty asm(); statement in the definition of asm_volatile_goto
> > > for compiler-gcc.h).  Might be time to "fix the compiler."
> > >
> > > (It sounds like Arvind is both in agreement with my sentiment, and has
> > > the root cause).
> > >
> >
> > I agree that the __no_fgcse hack is terrible. Does Clang support the
> > following pragmas?
> >
> > #pragma GCC push_options
> > #pragma GCC optimize ("-fno-gcse")
> > #pragma GCC pop_options
> >
> > ?
> 
> Put it in godbolt.org.  Pretty sure it's `#pragma clang` though.
> `#pragma GCC` might be supported in clang or silently ignored, but
> IIRC pragmas were a bit of a compat nightmare.  I think Arnd wrote
> some macros to set pragmas based on toolchain.  (Uses _Pragma, for
> pragmas in macros, IIRC).
> 
> -- 
> Thanks,
> ~Nick Desaulniers

https://gcc.gnu.org/onlinedocs/gcc/Function-Specific-Option-Pragmas.html#Function-Specific-Option-Pragmas

#pragma GCC optimize is equivalent to the function attribute, so does
that actually help?

Btw, the bug mentioned in asm_volatile_goto seems like its been fixed in
4.9, so the hack could be dropped now?
Arvind Sankar Oct. 27, 2020, 8:32 p.m. UTC | #27
On Tue, Oct 27, 2020 at 01:28:02PM -0700, Nick Desaulniers wrote:
> > commit 3193c0836f203a91bef96d88c64cccf0be090d9c

> > Author: Josh Poimboeuf <jpoimboe@redhat.com>

> > Date:   Wed Jul 17 20:36:45 2019 -0500

> >

> >     bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()

> >

> > has

> >

> > Fixes: e55a73251da3 ("bpf: Fix ORC unwinding in non-JIT BPF code")

> >

> > and mentions objtool and CONFIG_RETPOLINE.

> 

> Thanks for the context.  It might be time to revisit the above commit.

> If I revert it (small conflict that's easy to fixup),

> kernel/bpf/core.o builds cleanly with defconfig+GCC-9.3, so maybe

> obtool did get smart enough to handle that case?  Probably regresses

> the performance of that main dispatch loop for BPF, but not sure what

> folks are expecting when retpolines are enabled.

> -- 

> Thanks,

> ~Nick Desaulniers


The objtool issue was with RETPOLINE disabled.
Nick Desaulniers Oct. 27, 2020, 8:36 p.m. UTC | #28
On Tue, Oct 27, 2020 at 1:32 PM Arvind Sankar <nivedita@alum.mit.edu> wrote:
>
> On Tue, Oct 27, 2020 at 01:28:02PM -0700, Nick Desaulniers wrote:
> > > commit 3193c0836f203a91bef96d88c64cccf0be090d9c
> > > Author: Josh Poimboeuf <jpoimboe@redhat.com>
> > > Date:   Wed Jul 17 20:36:45 2019 -0500
> > >
> > >     bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()
> > >
> > > has
> > >
> > > Fixes: e55a73251da3 ("bpf: Fix ORC unwinding in non-JIT BPF code")
> > >
> > > and mentions objtool and CONFIG_RETPOLINE.
> >
> > Thanks for the context.  It might be time to revisit the above commit.
> > If I revert it (small conflict that's easy to fixup),
> > kernel/bpf/core.o builds cleanly with defconfig+GCC-9.3, so maybe
> > obtool did get smart enough to handle that case?  Probably regresses
> > the performance of that main dispatch loop for BPF, but not sure what
> > folks are expecting when retpolines are enabled.
> > --
> > Thanks,
> > ~Nick Desaulniers
>
> The objtool issue was with RETPOLINE disabled.

Ah, sorry, in that case default-CONFIG_RETPOLINE+gcc-9.3:
kernel/bpf/core.o: warning: objtool: ___bpf_prog_run()+0x8d4: sibling
call from callable instruction with modified stack frame
Nick Desaulniers Oct. 27, 2020, 8:40 p.m. UTC | #29
On Tue, Oct 27, 2020 at 1:30 PM Arvind Sankar <nivedita@alum.mit.edu> wrote:
>

> On Tue, Oct 27, 2020 at 01:17:55PM -0700, Nick Desaulniers wrote:

> > > >  (I feel the same about there

> > > > being an empty asm(); statement in the definition of asm_volatile_goto

> > > > for compiler-gcc.h).  Might be time to "fix the compiler."

> > > >

> > > > (It sounds like Arvind is both in agreement with my sentiment, and has

> > > > the root cause).

> > > >

> Btw, the bug mentioned in asm_volatile_goto seems like its been fixed in

> 4.9, so the hack could be dropped now?


https://lore.kernel.org/lkml/20180907222109.163802-1-ndesaulniers@google.com/

For the life of me I can't find Linus' response.  Maybe he shot it
down in the PR, but I can't find it...Miguel do you recall?  I could
paraphrase, but might be better to not rely on my memory.
-- 
Thanks,
~Nick Desaulniers
Arvind Sankar Oct. 27, 2020, 9:24 p.m. UTC | #30
On Tue, Oct 27, 2020 at 01:40:43PM -0700, Nick Desaulniers wrote:
> On Tue, Oct 27, 2020 at 1:30 PM Arvind Sankar <nivedita@alum.mit.edu> wrote:
> >
> > On Tue, Oct 27, 2020 at 01:17:55PM -0700, Nick Desaulniers wrote:
> > > > >  (I feel the same about there
> > > > > being an empty asm(); statement in the definition of asm_volatile_goto
> > > > > for compiler-gcc.h).  Might be time to "fix the compiler."
> > > > >
> > > > > (It sounds like Arvind is both in agreement with my sentiment, and has
> > > > > the root cause).
> > > > >
> > Btw, the bug mentioned in asm_volatile_goto seems like its been fixed in
> > 4.9, so the hack could be dropped now?
> 
> https://lore.kernel.org/lkml/20180907222109.163802-1-ndesaulniers@google.com/
> 
> For the life of me I can't find Linus' response.  Maybe he shot it
> down in the PR, but I can't find it...Miguel do you recall?  I could
> paraphrase, but might be better to not rely on my memory.
> -- 
> Thanks,
> ~Nick Desaulniers

You couldn't find it in July either :)
https://lkml.org/lkml/2020/7/10/1026

Possibly he didn't like the version check? That should be unnecessary now.
Alexei Starovoitov Oct. 28, 2020, 9:36 p.m. UTC | #31
On Tue, Oct 27, 2020 at 09:15:17PM +0100, Ard Biesheuvel wrote:
> On Tue, 27 Oct 2020 at 21:12, Nick Desaulniers <ndesaulniers@google.com> wrote:

> >

> > On Tue, Oct 27, 2020 at 12:25 PM Geert Uytterhoeven

> > <geert@linux-m68k.org> wrote:

> > >

> > > Hi Nick,

> > >

> > > CC Josh

> > >

> > > On Mon, Oct 26, 2020 at 6:49 PM Nick Desaulniers

> > > <ndesaulniers@google.com> wrote:

> > > > On Mon, Oct 26, 2020 at 10:44 AM Geert Uytterhoeven

> > > > <geert@linux-m68k.org> wrote:

> > > > > On Mon, Oct 26, 2020 at 6:39 PM Ard Biesheuvel <ardb@kernel.org> wrote:

> > > > > > On Mon, 26 Oct 2020 at 17:01, Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > > On Mon, Oct 26, 2020 at 2:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > > > On Mon, Oct 26, 2020 at 1:29 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:

> > > > > > > > > I.e. including the ".eh_frame" warning. I have tried bisecting that

> > > > > > > > > warning (i.e. with be2881824ae9eb92 reverted), but that leads me to

> > > > > > > > > commit b3e5d80d0c48c0cc ("arm64/build: Warn on orphan section

> > > > > > > > > placement"), which is another red herring.

> > > > > > > >

> > > > > > > > kernel/bpf/core.o is the only file containing an eh_frame section,

> > > > > > > > causing the warning.

> > > >

> > > > When I see .eh_frame, I think -fno-asynchronous-unwind-tables is

> > > > missing from someone's KBUILD_CFLAGS.

> > > > But I don't see anything curious in kernel/bpf/Makefile, unless

> > > > cc-disable-warning is somehow broken.

> > >

> > > I tracked it down to kernel/bpf/core.c:___bpf_prog_run() being tagged

> > > with __no_fgcse aka __attribute__((optimize("-fno-gcse"))).

> > >

> > > Even if the function is trivially empty ("return 0;"), a ".eh_frame" section

> > > is generated.  Removing the __no_fgcse tag fixes that.

> >

> > That's weird.  I feel pretty strongly that unless we're working around

> > a well understood compiler bug with a comment that links to a

> > submitted bug report, turning off rando compiler optimizations is a

> > terrible hack for which one must proceed straight to jail; do not pass

> > go; do not collect $200.  But maybe I'd feel differently for this case

> > given the context of the change that added it.  (Ard mentions

> > retpolines+orc+objtool; can someone share the relevant SHA if you have

> > it handy so I don't have to go digging?)

> 

> commit 3193c0836f203a91bef96d88c64cccf0be090d9c

> Author: Josh Poimboeuf <jpoimboe@redhat.com>

> Date:   Wed Jul 17 20:36:45 2019 -0500

> 

>     bpf: Disable GCC -fgcse optimization for ___bpf_prog_run()

> 

> has

> 

> Fixes: e55a73251da3 ("bpf: Fix ORC unwinding in non-JIT BPF code")


That commit is broken.
I had this patch in my queue:
-#define __no_fgcse __attribute__((optimize("-fno-gcse")))
+#define __no_fgcse __attribute__((optimize("-fno-gcse,-fno-omit-frame-pointer")))

Sounds like you want to add -fno-asynchronous-unwind-tables to the above list?

> and mentions objtool and CONFIG_RETPOLINE.

> 

> >  (I feel the same about there

> > being an empty asm(); statement in the definition of asm_volatile_goto

> > for compiler-gcc.h).  Might be time to "fix the compiler."

> >

> > (It sounds like Arvind is both in agreement with my sentiment, and has

> > the root cause).

> >

> 

> I agree that the __no_fgcse hack is terrible. Does Clang support the

> following pragmas?

> 

> #pragma GCC push_options

> #pragma GCC optimize ("-fno-gcse")

> #pragma GCC pop_options


That will work too, but optimize("-fno...,-fno..,-fno..") is imo cleaner.