| Message ID | 1468502169-15248-5-git-send-email-ard.biesheuvel@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
On 16 July 2016 at 15:54, Shi, Steven <steven.shi@intel.com> wrote: > Hi Ard, > Sorry for my late response. Setting all symbols visibility as hidden will block LTO on GCC. GCC LTO need symbols to be public to let linker freely do the link time optimization across local to external. Below is the GCC5 build failure in my side with your patch. > Do you have any references that support this? Hidden visibility has nothing to do with linkage. It only controls the symbols that are exported from a shared library. Knowing whether a symbol is ever exported from a shared library allows the compiler to generate better code. > "gcc" -o /home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll -Os -flto -nostdlib -Wl,-n -Wl,-q -Wl,--gc-sections -z common-page-size=0x40 -Wl,--entry,_ModuleEntryPoint -Wl,-u,_ModuleEntryPoint -Wl,-Map,/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.map -mcmodel=small -Wl,-melf_x86_64 -Wl,--oformat=elf64-x86-64 -Wl,--start-group,,@/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/OUTPUT/static_library_files.lst -Wl,--end-group -Wl,--defsym=PECOFF_HEADER_SIZE=0x228 -Wl,--script=/home/jshi19/edk2-fork3/BaseTools/Scripts/GccBase.lds > "objcopy" /home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll > objcopy: error: the input file '/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll' has no sections This looks like an issue with --gc-sections. Could you check the map file why everything gets discarded? > make: *** [/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll] Error 1 > GNUmakefile:478: recipe for target '/home/jshi19/edk2-fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain/DEBUG/PeiCore.dll' failed > > > Steven Shi > Intel\SSG\STO\UEFI Firmware > > Tel: +86 021-61166522 > iNet: 821-6522 > >> -----Original Message----- >> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] >> Sent: Thursday, July 14, 2016 9:16 PM >> To: Shi, Steven <steven.shi@intel.com>; Zhu, Yonghong >> <yonghong.zhu@intel.com>; Gao, Liming <liming.gao@intel.com>; Kinney, >> Michael D <michael.d.kinney@intel.com>; Justen, Jordan L >> <jordan.l.justen@intel.com>; lersek@redhat.com; afish@apple.com; edk2- >> devel@lists.01.org >> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> >> Subject: [PATCH 4/5] MdePkg X64: force 'hidden' visibility when building with >> -fpic >> >> When building position independent (PIC) ELF objects, the GCC compiler >> assumes that each symbol with external linkage may potentially end up >> being exported from a shared library, which means that each of those >> symbols may be subject to symbol preemption, i.e., the executable >> linking to the shared library at runtime may override symbols exported >> by the shared library, and every internal reference held by the shared >> library itself *must* be made to point to the overridden version instead. >> >> For this reason, PIC code symbol references always go via the Global >> Offset Table (GOT), even if the code in question references symbols that >> are defined in the same compilation unit. The GOT refers to each symbol >> by absolute address, and so each entry is subject to runtime relocation. >> >> Since not every symbol with external linkage is ultimately exported from >> a shared library, the GCC compiler allows control over symbol visibility >> using attributes, command line arguments and pragmas, where 'hidden' >> means that the symbol is only referenced by the shared library itself. >> Due to the poor hygiene in EDK2 regarding the use of the 'static' >> modifier, many symbols that are local to their compilation unit end up >> being referenced indirectly via the GOT when building PIC code. >> >> In UEFI, there are no shared libraries and so there is no need to deal >> with symbol preemption, and we can mark every symbol reference 'hidden'. >> The only method that applies to all symbol definitions as well as >> declarations is the #pragma. So set the visibility 'hidden' pragma when >> building PIC code for X64 using GCC. >> >> Contributed-under: TianoCore Contribution Agreement 1.0 >> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> >> --- >> MdePkg/Include/X64/ProcessorBind.h | 10 ++++++++++ >> 1 file changed, 10 insertions(+) >> >> diff --git a/MdePkg/Include/X64/ProcessorBind.h >> b/MdePkg/Include/X64/ProcessorBind.h >> index 705104af062a..96df78fca07d 100644 >> --- a/MdePkg/Include/X64/ProcessorBind.h >> +++ b/MdePkg/Include/X64/ProcessorBind.h >> @@ -27,6 +27,16 @@ >> #pragma pack() >> #endif >> >> +#if defined(__GNUC__) && defined(__pic__) >> +// >> +// Mark all symbol declarations and references as hidden, meaning they will >> not >> +// be exported from a shared library, and thus will not be subject to symbol >> +// preemption. This allows the compiler to refer to symbols directly using >> +// relative references rather than via the GOT, which contains absolute >> symbol >> +// addresses that are subject to runtime relocation. >> +// >> +#pragma GCC visibility push (hidden) >> +#endif >> >> #if defined(__INTEL_COMPILER) >> // >> -- >> 2.7.4 > _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
> On 16 jul. 2016, at 17:10, Shi, Steven <steven.shi@intel.com> wrote: > > You clone and can try the GCC LTO build on my below branch, and welcome directly debug on it. > https://github.com/shijunjing/edk2/tree/llvm_v2 > Thanks, I will try it on monday > > > Steven Shi > Intel\SSG\STO\UEFI Firmware > > Tel: +86 021-61166522 > iNet: 821-6522 > > >> -----Original Message----- >> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] >> Sent: Saturday, July 16, 2016 10:48 PM >> To: Shi, Steven <steven.shi@intel.com> >> Cc: Zhu, Yonghong <yonghong.zhu@intel.com>; Gao, Liming >> <liming.gao@intel.com>; Kinney, Michael D <michael.d.kinney@intel.com>; >> Justen, Jordan L <jordan.l.justen@intel.com>; lersek@redhat.com; >> afish@apple.com; edk2-devel@lists.01.org >> Subject: Re: [PATCH 4/5] MdePkg X64: force 'hidden' visibility when building >> with -fpic >> >>> On 16 July 2016 at 15:54, Shi, Steven <steven.shi@intel.com> wrote: >>> Hi Ard, >>> Sorry for my late response. Setting all symbols visibility as hidden will block >> LTO on GCC. GCC LTO need symbols to be public to let linker freely do the >> link time optimization across local to external. Below is the GCC5 build >> failure in my side with your patch. >> >> Do you have any references that support this? Hidden visibility has >> nothing to do with linkage. It only controls the symbols that are >> exported from a shared library. Knowing whether a symbol is ever >> exported from a shared library allows the compiler to generate better >> code. >> >> >>> "gcc" -o /home/jshi19/edk2- >> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain >> /DEBUG/PeiCore.dll -Os -flto -nostdlib -Wl,-n -Wl,-q -Wl,--gc-sections -z >> common-page-size=0x40 -Wl,--entry,_ModuleEntryPoint -Wl,- >> u,_ModuleEntryPoint -Wl,-Map,/home/jshi19/edk2- >> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain >> /DEBUG/PeiCore.map -mcmodel=small -Wl,-melf_x86_64 -Wl,-- >> oformat=elf64-x86-64 -Wl,--start-group,,@/home/jshi19/edk2- >> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain >> /OUTPUT/static_library_files.lst -Wl,--end-group -Wl,-- >> defsym=PECOFF_HEADER_SIZE=0x228 -Wl,--script=/home/jshi19/edk2- >> fork3/BaseTools/Scripts/GccBase.lds >>> "objcopy" /home/jshi19/edk2- >> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain >> /DEBUG/PeiCore.dll >>> objcopy: error: the input file '/home/jshi19/edk2- >> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain >> /DEBUG/PeiCore.dll' has no sections >> >> This looks like an issue with --gc-sections. Could you check the map >> file why everything gets discarded? >> >>> make: *** [/home/jshi19/edk2- >> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain >> /DEBUG/PeiCore.dll] Error 1 >>> GNUmakefile:478: recipe for target '/home/jshi19/edk2- >> fork3/Build/OvmfX64/DEBUG_GCC5/X64/MdeModulePkg/Core/Pei/PeiMain >> /DEBUG/PeiCore.dll' failed >>> >>> >>> Steven Shi >>> Intel\SSG\STO\UEFI Firmware >>> >>> Tel: +86 021-61166522 >>> iNet: 821-6522 >>> >>>> -----Original Message----- >>>> From: Ard Biesheuvel [mailto:ard.biesheuvel@linaro.org] >>>> Sent: Thursday, July 14, 2016 9:16 PM >>>> To: Shi, Steven <steven.shi@intel.com>; Zhu, Yonghong >>>> <yonghong.zhu@intel.com>; Gao, Liming <liming.gao@intel.com>; >> Kinney, >>>> Michael D <michael.d.kinney@intel.com>; Justen, Jordan L >>>> <jordan.l.justen@intel.com>; lersek@redhat.com; afish@apple.com; >> edk2- >>>> devel@lists.01.org >>>> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org> >>>> Subject: [PATCH 4/5] MdePkg X64: force 'hidden' visibility when building >> with >>>> -fpic >>>> >>>> When building position independent (PIC) ELF objects, the GCC compiler >>>> assumes that each symbol with external linkage may potentially end up >>>> being exported from a shared library, which means that each of those >>>> symbols may be subject to symbol preemption, i.e., the executable >>>> linking to the shared library at runtime may override symbols exported >>>> by the shared library, and every internal reference held by the shared >>>> library itself *must* be made to point to the overridden version instead. >>>> >>>> For this reason, PIC code symbol references always go via the Global >>>> Offset Table (GOT), even if the code in question references symbols that >>>> are defined in the same compilation unit. The GOT refers to each symbol >>>> by absolute address, and so each entry is subject to runtime relocation. >>>> >>>> Since not every symbol with external linkage is ultimately exported from >>>> a shared library, the GCC compiler allows control over symbol visibility >>>> using attributes, command line arguments and pragmas, where 'hidden' >>>> means that the symbol is only referenced by the shared library itself. >>>> Due to the poor hygiene in EDK2 regarding the use of the 'static' >>>> modifier, many symbols that are local to their compilation unit end up >>>> being referenced indirectly via the GOT when building PIC code. >>>> >>>> In UEFI, there are no shared libraries and so there is no need to deal >>>> with symbol preemption, and we can mark every symbol reference >> 'hidden'. >>>> The only method that applies to all symbol definitions as well as >>>> declarations is the #pragma. So set the visibility 'hidden' pragma when >>>> building PIC code for X64 using GCC. >>>> >>>> Contributed-under: TianoCore Contribution Agreement 1.0 >>>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> >>>> --- >>>> MdePkg/Include/X64/ProcessorBind.h | 10 ++++++++++ >>>> 1 file changed, 10 insertions(+) >>>> >>>> diff --git a/MdePkg/Include/X64/ProcessorBind.h >>>> b/MdePkg/Include/X64/ProcessorBind.h >>>> index 705104af062a..96df78fca07d 100644 >>>> --- a/MdePkg/Include/X64/ProcessorBind.h >>>> +++ b/MdePkg/Include/X64/ProcessorBind.h >>>> @@ -27,6 +27,16 @@ >>>> #pragma pack() >>>> #endif >>>> >>>> +#if defined(__GNUC__) && defined(__pic__) >>>> +// >>>> +// Mark all symbol declarations and references as hidden, meaning they >> will >>>> not >>>> +// be exported from a shared library, and thus will not be subject to >> symbol >>>> +// preemption. This allows the compiler to refer to symbols directly using >>>> +// relative references rather than via the GOT, which contains absolute >>>> symbol >>>> +// addresses that are subject to runtime relocation. >>>> +// >>>> +#pragma GCC visibility push (hidden) >>>> +#endif >>>> >>>> #if defined(__INTEL_COMPILER) >>>> // >>>> -- >>>> 2.7.4 >>> _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
On 16 July 2016 at 19:25, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > > >> On 16 jul. 2016, at 17:10, Shi, Steven <steven.shi@intel.com> wrote: >> >> You clone and can try the GCC LTO build on my below branch, and welcome directly debug on it. >> https://github.com/shijunjing/edk2/tree/llvm_v2 >> > > Thanks, I will try it on monday > I just tried it, and indeed, it turns out that 'hidden' visibility interacts poorly with LTO, and we should use 'protected' visibility instead, which achieves the same thing, i.e., that the compiler is allowed to assume that symbols are not subject to preemption, and so it can refer to them relatively rather than indirectly via an absolute address in the GOT. I will update this in my v3 series. Thanks, Ard. _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel
diff --git a/MdePkg/Include/X64/ProcessorBind.h b/MdePkg/Include/X64/ProcessorBind.h index 705104af062a..96df78fca07d 100644 --- a/MdePkg/Include/X64/ProcessorBind.h +++ b/MdePkg/Include/X64/ProcessorBind.h @@ -27,6 +27,16 @@ #pragma pack() #endif +#if defined(__GNUC__) && defined(__pic__) +// +// Mark all symbol declarations and references as hidden, meaning they will not +// be exported from a shared library, and thus will not be subject to symbol +// preemption. This allows the compiler to refer to symbols directly using +// relative references rather than via the GOT, which contains absolute symbol +// addresses that are subject to runtime relocation. +// +#pragma GCC visibility push (hidden) +#endif #if defined(__INTEL_COMPILER) //
When building position independent (PIC) ELF objects, the GCC compiler assumes that each symbol with external linkage may potentially end up being exported from a shared library, which means that each of those symbols may be subject to symbol preemption, i.e., the executable linking to the shared library at runtime may override symbols exported by the shared library, and every internal reference held by the shared library itself *must* be made to point to the overridden version instead. For this reason, PIC code symbol references always go via the Global Offset Table (GOT), even if the code in question references symbols that are defined in the same compilation unit. The GOT refers to each symbol by absolute address, and so each entry is subject to runtime relocation. Since not every symbol with external linkage is ultimately exported from a shared library, the GCC compiler allows control over symbol visibility using attributes, command line arguments and pragmas, where 'hidden' means that the symbol is only referenced by the shared library itself. Due to the poor hygiene in EDK2 regarding the use of the 'static' modifier, many symbols that are local to their compilation unit end up being referenced indirectly via the GOT when building PIC code. In UEFI, there are no shared libraries and so there is no need to deal with symbol preemption, and we can mark every symbol reference 'hidden'. The only method that applies to all symbol definitions as well as declarations is the #pragma. So set the visibility 'hidden' pragma when building PIC code for X64 using GCC. Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- MdePkg/Include/X64/ProcessorBind.h | 10 ++++++++++ 1 file changed, 10 insertions(+) -- 2.7.4 _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel