diff mbox series

[v2] vmlinux.lds: add PGO and AutoFDO input sections

Message ID 20200625184752.73095-1-ndesaulniers@google.com
State Accepted
Commit eff8728fe69880d3f7983bec3fb6cea4c306261f
Headers show
Series [v2] vmlinux.lds: add PGO and AutoFDO input sections | expand

Commit Message

Nick Desaulniers June 25, 2020, 6:47 p.m. UTC 
Basically, consider .text.{hot|unlikely|unknown}.* part of .text, too.

When compiling with profiling information (collected via PGO
instrumentations or AutoFDO sampling), Clang will separate code into
.text.hot, .text.unlikely, or .text.unknown sections based on profiling
information. After D79600 (clang-11), these sections will have a
trailing `.` suffix, ie.  .text.hot., .text.unlikely., .text.unknown..

When using -ffunction-sections together with profiling infomation,
either explicitly (FGKASLR) or implicitly (LTO), code may be placed in
sections following the convention:
.text.hot.<foo>, .text.unlikely.<bar>, .text.unknown.<baz>
where <foo>, <bar>, and <baz> are functions.  (This produces one section
per function; we generally try to merge these all back via linker script
so that we don't have 50k sections).

For the above cases, we need to teach our linker scripts that such
sections might exist and that we'd explicitly like them grouped
together, otherwise we can wind up with code outside of the
_stext/_etext boundaries that might not be mapped properly for some
architectures, resulting in boot failures.

If the linker script is not told about possible input sections, then
where the section is placed as output is a heuristic-laiden mess that's
non-portable between linkers (ie. BFD and LLD), and has resulted in many
hard to debug bugs.  Kees Cook is working on cleaning this up by adding
--orphan-handling=warn linker flag used in ARCH=powerpc to additional
architectures. In the case of linker scripts, borrowing from the Zen of
Python: explicit is better than implicit.

Also, ld.bfd's internal linker script considers .text.hot AND
.text.hot.* to be part of .text, as well as .text.unlikely and
.text.unlikely.*. I didn't see support for .text.unknown.*, and didn't
see Clang producing such code in our kernel builds, but I see code in
LLVM that can produce such section names if profiling information is
missing. That may point to a larger issue with generating or collecting
profiles, but I would much rather be safe and explicit than have to
debug yet another issue related to orphan section placement.

Cc: stable@vger.kernel.org
Link: https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=add44f8d5c5c05e08b11e033127a744d61c26aee
Link: https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=1de778ed23ce7492c523d5850c6c6dbb34152655
Link: https://reviews.llvm.org/D79600
Link: https://bugs.chromium.org/p/chromium/issues/detail?id=1084760
Reported-by: Jian Cai <jiancai@google.com>
Debugged-by: Luis Lozano <llozano@google.com>
Suggested-by: Fāng-ruì Sòng <maskray@google.com>
Tested-by: Luis Lozano <llozano@google.com>

Tested-by: Manoj Gupta <manojgupta@google.com>

Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>

---
Changes V1 -> V2:
* Add .text.unknown.*.  It's not strictly necessary for us yet, but I
  really worry that it could become a problem for us. Either way, I'm
  happy to drop for a V3, but I'm suggesting we not.
* Beef up commit message.
* Drop references to LLD; the LLVM change had nothing to do with LLD.
  I've realized I have a Pavlovian-response to changes from Fāng-ruì
  that I associate with LLD.  I'm seeking professional help for my
  ailment. Forgive me.
* Add link to now public CrOS bug.

 include/asm-generic/vmlinux.lds.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

-- 
2.27.0.111.gc72c7da667-goog

Comments

Sasha Levin July 1, 2020, 7:33 p.m. UTC | #1 
Hi

[This is an automated email]

This commit has been processed because it contains a -stable tag.
The stable tag indicates that it's relevant for the following trees: all

The bot has tested the following trees: v5.7.6, v5.4.49, v4.19.130, v4.14.186, v4.9.228, v4.4.228.

v5.7.6: Failed to apply! Possible dependencies:
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")

v5.4.49: Failed to apply! Possible dependencies:
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")

v4.19.130: Failed to apply! Possible dependencies:
    5c67a52f3da0f ("Compiler Attributes: always use the extra-underscores syntax")
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")
    67361cf807128 ("powerpc/ftrace: Handle large kernel configs")
    71391bdd2e9aa ("include/linux/compiler_types.h: don't pollute userspace with macro definitions")
    77b0bf55bc675 ("kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs")
    87b512def7925 ("objtool: Add support for C jump tables")
    96af6cd02a10b ("Revert "x86/objtool: Use asm macros to work around GCC inlining bugs"")
    989bd5000f360 ("Compiler Attributes: remove unneeded sparse (__CHECKER__) tests")
    a3f8a30f3f007 ("Compiler Attributes: use feature checks instead of version checks")
    c06c4d8090513 ("x86/objtool: Use asm macros to work around GCC inlining bugs")
    c2c640aa04cc4 ("Compiler Attributes: remove unneeded tests")
    ec0bbef66f867 ("Compiler Attributes: homogenize __must_be_array")

v4.14.186: Failed to apply! Possible dependencies:
    10259821ac47d ("objtool: Make unreachable annotation inline asms explicitly volatile")
    4c1d9bb0b5d3c ("powerpc: Allow LD_DEAD_CODE_DATA_ELIMINATION to be selected")
    5633e85b2c313 ("powerpc64: Add .opd based function descriptor dereference")
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")
    67361cf807128 ("powerpc/ftrace: Handle large kernel configs")
    71391bdd2e9aa ("include/linux/compiler_types.h: don't pollute userspace with macro definitions")
    7290d58095712 ("module: use relative references for __ksymtab entries")
    77b0bf55bc675 ("kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs")
    815f0ddb346c1 ("include/linux/compiler*.h: make compiler-*.h mutually exclusive")
    8793bb7f4a9dd ("kbuild: add macro for controlling warnings to linux/compiler.h")
    87b512def7925 ("objtool: Add support for C jump tables")
    94e58e0ac3128 ("export.h: remove code for prefixing symbols with underscore")
    96af6cd02a10b ("Revert "x86/objtool: Use asm macros to work around GCC inlining bugs"")
    a10726075dec4 ("powerpc/32: Add .data.rel* sections explicitly")
    ae30cc05bed2f ("powerpc64/ftrace: Implement support for ftrace_regs_caller()")
    b865ea64304ed ("sections: split dereference_function_descriptor()")
    c06c4d8090513 ("x86/objtool: Use asm macros to work around GCC inlining bugs")
    cafa0010cd51f ("Raise the minimum required gcc version to 4.6")
    ea678ac627e01 ("powerpc64/ftrace: Add a field in paca to disable ftrace in unsafe code paths")
    fda784e50aace ("module: export module signature enforcement status")

v4.9.228: Failed to apply! Possible dependencies:
    096ff2ddba83b ("powerpc/ftrace/64: Split further based on -mprofile-kernel")
    2f59be5b970b5 ("powerpc/ftrace: Restore LR from pt_regs")
    4546561551106 ("powerpc/asm: Use OFFSET macro in asm-offsets.c")
    5d451a87e5ebb ("powerpc/64: Retrieve number of L1 cache sets from device-tree")
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")
    67361cf807128 ("powerpc/ftrace: Handle large kernel configs")
    700e64377c2c8 ("powerpc/ftrace: Move stack setup and teardown code into ftrace_graph_caller()")
    7853f9c029ac9 ("powerpc: Split ftrace bits into a separate file")
    902e06eb86cd6 ("powerpc/32: Change the stack protector canary value per task")
    99ad503287daf ("powerpc: Add a prototype for mcount() so it can be versioned")
    ae30cc05bed2f ("powerpc64/ftrace: Implement support for ftrace_regs_caller()")
    b3a7864c6feb0 ("powerpc/ftrace: Add prototype for prepare_ftrace_return()")
    bd067f83b0840 ("powerpc/64: Fix naming of cache block vs. cache line")
    c02e0349d7e9e ("powerpc/ftrace: Fix the comments for ftrace_modify_code")
    e2827fe5c1566 ("powerpc/64: Clean up ppc64_caches using a struct per cache")
    ea678ac627e01 ("powerpc64/ftrace: Add a field in paca to disable ftrace in unsafe code paths")

v4.4.228: Failed to apply! Possible dependencies:
    0f4c4af06eec5 ("kbuild: -ffunction-sections fix for archs with conflicting sections")
    136cd3450af80 ("powerpc/module: Only try to generate the ftrace_caller() stub once")
    153086644fd1f ("powerpc/ftrace: Add support for -mprofile-kernel ftrace ABI")
    20ef10c1b3068 ("module: Use the same logic for setting and unsetting RO/NX")
    336a7b5dd80a2 ("powerpc/module: Create a special stub for ftrace_caller()")
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")
    67361cf807128 ("powerpc/ftrace: Handle large kernel configs")
    7523e4dc5057e ("module: use a structure to encapsulate layout.")
    a5967db9af51a ("kbuild: allow architectures to use thin archives instead of ld -r")
    ae30cc05bed2f ("powerpc64/ftrace: Implement support for ftrace_regs_caller()")
    b67067f1176df ("kbuild: allow archs to select link dead code/data elimination")
    cb87481ee89db ("kbuild: linker script do not match C names unless LD_DEAD_CODE_DATA_ELIMINATION is configured")
    f17c4e01e906c ("powerpc/module: Mark module stubs with a magic value")
    f235541699bcf ("export.h: allow for per-symbol configurable EXPORT_SYMBOL()")


NOTE: The patch will not be queued to stable trees until it is upstream.

How should we proceed with this patch?

-- 
Thanks
Sasha
Nick Desaulniers July 1, 2020, 9:54 p.m. UTC | #2 
Hi Arnd,
I usually wait longer to bump threads for review, but we have a
holiday in the US so we're off tomorrow and Friday.
scripts/get_maintainer.pl recommend you for this patch.  Would you
take a look at it for us, please?

On Thu, Jun 25, 2020 at 11:48 AM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>

> Basically, consider .text.{hot|unlikely|unknown}.* part of .text, too.

>

> When compiling with profiling information (collected via PGO

> instrumentations or AutoFDO sampling), Clang will separate code into

> .text.hot, .text.unlikely, or .text.unknown sections based on profiling

> information. After D79600 (clang-11), these sections will have a

> trailing `.` suffix, ie.  .text.hot., .text.unlikely., .text.unknown..

>

> When using -ffunction-sections together with profiling infomation,

> either explicitly (FGKASLR) or implicitly (LTO), code may be placed in

> sections following the convention:

> .text.hot.<foo>, .text.unlikely.<bar>, .text.unknown.<baz>

> where <foo>, <bar>, and <baz> are functions.  (This produces one section

> per function; we generally try to merge these all back via linker script

> so that we don't have 50k sections).

>

> For the above cases, we need to teach our linker scripts that such

> sections might exist and that we'd explicitly like them grouped

> together, otherwise we can wind up with code outside of the

> _stext/_etext boundaries that might not be mapped properly for some

> architectures, resulting in boot failures.

>

> If the linker script is not told about possible input sections, then

> where the section is placed as output is a heuristic-laiden mess that's

> non-portable between linkers (ie. BFD and LLD), and has resulted in many

> hard to debug bugs.  Kees Cook is working on cleaning this up by adding

> --orphan-handling=warn linker flag used in ARCH=powerpc to additional

> architectures. In the case of linker scripts, borrowing from the Zen of

> Python: explicit is better than implicit.

>

> Also, ld.bfd's internal linker script considers .text.hot AND

> .text.hot.* to be part of .text, as well as .text.unlikely and

> .text.unlikely.*. I didn't see support for .text.unknown.*, and didn't

> see Clang producing such code in our kernel builds, but I see code in

> LLVM that can produce such section names if profiling information is

> missing. That may point to a larger issue with generating or collecting

> profiles, but I would much rather be safe and explicit than have to

> debug yet another issue related to orphan section placement.

>

> Cc: stable@vger.kernel.org

> Link: https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=add44f8d5c5c05e08b11e033127a744d61c26aee

> Link: https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=1de778ed23ce7492c523d5850c6c6dbb34152655

> Link: https://reviews.llvm.org/D79600

> Link: https://bugs.chromium.org/p/chromium/issues/detail?id=1084760

> Reported-by: Jian Cai <jiancai@google.com>

> Debugged-by: Luis Lozano <llozano@google.com>

> Suggested-by: Fāng-ruì Sòng <maskray@google.com>

> Tested-by: Luis Lozano <llozano@google.com>

> Tested-by: Manoj Gupta <manojgupta@google.com>

> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>

> ---

> Changes V1 -> V2:

> * Add .text.unknown.*.  It's not strictly necessary for us yet, but I

>   really worry that it could become a problem for us. Either way, I'm

>   happy to drop for a V3, but I'm suggesting we not.

> * Beef up commit message.

> * Drop references to LLD; the LLVM change had nothing to do with LLD.

>   I've realized I have a Pavlovian-response to changes from Fāng-ruì

>   that I associate with LLD.  I'm seeking professional help for my

>   ailment. Forgive me.

> * Add link to now public CrOS bug.

>

>  include/asm-generic/vmlinux.lds.h | 5 ++++-

>  1 file changed, 4 insertions(+), 1 deletion(-)

>

> diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h

> index d7c7c7f36c4a..245c1af4c057 100644

> --- a/include/asm-generic/vmlinux.lds.h

> +++ b/include/asm-generic/vmlinux.lds.h

> @@ -560,7 +560,10 @@

>   */

>  #define TEXT_TEXT                                                      \

>                 ALIGN_FUNCTION();                                       \

> -               *(.text.hot TEXT_MAIN .text.fixup .text.unlikely)       \

> +               *(.text.hot .text.hot.*)                                \

> +               *(TEXT_MAIN .text.fixup)                                \

> +               *(.text.unlikely .text.unlikely.*)                      \

> +               *(.text.unknown .text.unknown.*)                        \

>                 NOINSTR_TEXT                                            \

>                 *(.text..refcount)                                      \

>                 *(.ref.text)                                            \

> --

> 2.27.0.111.gc72c7da667-goog

>



-- 
Thanks,
~Nick Desaulniers
Sasha Levin July 10, 2020, 2:02 p.m. UTC | #3 
Hi

[This is an automated email]

This commit has been processed because it contains a -stable tag.
The stable tag indicates that it's relevant for the following trees: all

The bot has tested the following trees: v5.7.6, v5.4.49, v4.19.130, v4.14.186, v4.9.228, v4.4.228.

v5.7.6: Failed to apply! Possible dependencies:
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")

v5.4.49: Failed to apply! Possible dependencies:
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")

v4.19.130: Failed to apply! Possible dependencies:
    5c67a52f3da0f ("Compiler Attributes: always use the extra-underscores syntax")
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")
    67361cf807128 ("powerpc/ftrace: Handle large kernel configs")
    71391bdd2e9aa ("include/linux/compiler_types.h: don't pollute userspace with macro definitions")
    77b0bf55bc675 ("kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs")
    87b512def7925 ("objtool: Add support for C jump tables")
    96af6cd02a10b ("Revert "x86/objtool: Use asm macros to work around GCC inlining bugs"")
    989bd5000f360 ("Compiler Attributes: remove unneeded sparse (__CHECKER__) tests")
    a3f8a30f3f007 ("Compiler Attributes: use feature checks instead of version checks")
    c06c4d8090513 ("x86/objtool: Use asm macros to work around GCC inlining bugs")
    c2c640aa04cc4 ("Compiler Attributes: remove unneeded tests")
    ec0bbef66f867 ("Compiler Attributes: homogenize __must_be_array")

v4.14.186: Failed to apply! Possible dependencies:
    10259821ac47d ("objtool: Make unreachable annotation inline asms explicitly volatile")
    4c1d9bb0b5d3c ("powerpc: Allow LD_DEAD_CODE_DATA_ELIMINATION to be selected")
    5633e85b2c313 ("powerpc64: Add .opd based function descriptor dereference")
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")
    67361cf807128 ("powerpc/ftrace: Handle large kernel configs")
    71391bdd2e9aa ("include/linux/compiler_types.h: don't pollute userspace with macro definitions")
    7290d58095712 ("module: use relative references for __ksymtab entries")
    77b0bf55bc675 ("kbuild/Makefile: Prepare for using macros in inline assembly code to work around asm() related GCC inlining bugs")
    815f0ddb346c1 ("include/linux/compiler*.h: make compiler-*.h mutually exclusive")
    8793bb7f4a9dd ("kbuild: add macro for controlling warnings to linux/compiler.h")
    87b512def7925 ("objtool: Add support for C jump tables")
    94e58e0ac3128 ("export.h: remove code for prefixing symbols with underscore")
    96af6cd02a10b ("Revert "x86/objtool: Use asm macros to work around GCC inlining bugs"")
    a10726075dec4 ("powerpc/32: Add .data.rel* sections explicitly")
    ae30cc05bed2f ("powerpc64/ftrace: Implement support for ftrace_regs_caller()")
    b865ea64304ed ("sections: split dereference_function_descriptor()")
    c06c4d8090513 ("x86/objtool: Use asm macros to work around GCC inlining bugs")
    cafa0010cd51f ("Raise the minimum required gcc version to 4.6")
    ea678ac627e01 ("powerpc64/ftrace: Add a field in paca to disable ftrace in unsafe code paths")
    fda784e50aace ("module: export module signature enforcement status")

v4.9.228: Failed to apply! Possible dependencies:
    096ff2ddba83b ("powerpc/ftrace/64: Split further based on -mprofile-kernel")
    2f59be5b970b5 ("powerpc/ftrace: Restore LR from pt_regs")
    4546561551106 ("powerpc/asm: Use OFFSET macro in asm-offsets.c")
    5d451a87e5ebb ("powerpc/64: Retrieve number of L1 cache sets from device-tree")
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")
    67361cf807128 ("powerpc/ftrace: Handle large kernel configs")
    700e64377c2c8 ("powerpc/ftrace: Move stack setup and teardown code into ftrace_graph_caller()")
    7853f9c029ac9 ("powerpc: Split ftrace bits into a separate file")
    902e06eb86cd6 ("powerpc/32: Change the stack protector canary value per task")
    99ad503287daf ("powerpc: Add a prototype for mcount() so it can be versioned")
    ae30cc05bed2f ("powerpc64/ftrace: Implement support for ftrace_regs_caller()")
    b3a7864c6feb0 ("powerpc/ftrace: Add prototype for prepare_ftrace_return()")
    bd067f83b0840 ("powerpc/64: Fix naming of cache block vs. cache line")
    c02e0349d7e9e ("powerpc/ftrace: Fix the comments for ftrace_modify_code")
    e2827fe5c1566 ("powerpc/64: Clean up ppc64_caches using a struct per cache")
    ea678ac627e01 ("powerpc64/ftrace: Add a field in paca to disable ftrace in unsafe code paths")

v4.4.228: Failed to apply! Possible dependencies:
    0f4c4af06eec5 ("kbuild: -ffunction-sections fix for archs with conflicting sections")
    136cd3450af80 ("powerpc/module: Only try to generate the ftrace_caller() stub once")
    153086644fd1f ("powerpc/ftrace: Add support for -mprofile-kernel ftrace ABI")
    20ef10c1b3068 ("module: Use the same logic for setting and unsetting RO/NX")
    336a7b5dd80a2 ("powerpc/module: Create a special stub for ftrace_caller()")
    6553896666433 ("vmlinux.lds.h: Create section for protection against instrumentation")
    67361cf807128 ("powerpc/ftrace: Handle large kernel configs")
    7523e4dc5057e ("module: use a structure to encapsulate layout.")
    a5967db9af51a ("kbuild: allow architectures to use thin archives instead of ld -r")
    ae30cc05bed2f ("powerpc64/ftrace: Implement support for ftrace_regs_caller()")
    b67067f1176df ("kbuild: allow archs to select link dead code/data elimination")
    cb87481ee89db ("kbuild: linker script do not match C names unless LD_DEAD_CODE_DATA_ELIMINATION is configured")
    f17c4e01e906c ("powerpc/module: Mark module stubs with a magic value")
    f235541699bcf ("export.h: allow for per-symbol configurable EXPORT_SYMBOL()")


NOTE: The patch will not be queued to stable trees until it is upstream.

How should we proceed with this patch?

-- 
Thanks
Sasha
diff mbox series

Patch

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index d7c7c7f36c4a..245c1af4c057 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -560,7 +560,10 @@ 
  */
 #define TEXT_TEXT							\
 		ALIGN_FUNCTION();					\
-		*(.text.hot TEXT_MAIN .text.fixup .text.unlikely)	\
+		*(.text.hot .text.hot.*)				\
+		*(TEXT_MAIN .text.fixup)				\
+		*(.text.unlikely .text.unlikely.*)			\
+		*(.text.unknown .text.unknown.*)			\
 		NOINSTR_TEXT						\
 		*(.text..refcount)					\
 		*(.ref.text)						\