| Message ID | 20241218150316.1583806-7-ardb+git@google.com |
|---|---|
| State | New |
| Headers | show |
| Series | efi/zboot: Encapsulate ELF image for arm64 | expand |
Hi, On Wed, Dec 18, 2024 at 04:03:17PM +0100, Ard Biesheuvel wrote: > From: Ard Biesheuvel <ardb@kernel.org> > > Remove the dependency on the decompression wrappers used by the legacy > decompressor, which do some odd things like providing a barebones > malloc() implementation. Instead, implement GZIP deflate and ZSTD > decompression in terms of the underlying libraries. > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org> With this patch in linux-next, my loongarch boot tests with qemu no longer boot. Log message is: qemu log: EFI stub: Decompressing Linux Kernel... EFI stub: EFI_RNG_PROTOCOL unavailable EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path EFI stub: Exiting boot services ... and then there is nothing until I abort the emulation. Reverting this patch results in a build failure, so I awas not able to test it. Bisect results are atatched for reference. Guenter --- # bad: [37136bf5c3a6f6b686d74f41837a6406bec6b7bc] Add linux-next specific files for 20250113 # good: [9d89551994a430b50c4fffcb1e617a057fa76e20] Linux 6.13-rc6 git bisect start 'next-20250113' 'v6.13-rc6' # good: [25dcaaf9b3bdaa117b8eb722ebde76ec9ed30038] Merge branch 'main' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git git bisect good 25dcaaf9b3bdaa117b8eb722ebde76ec9ed30038 # good: [c6ab5ee56509953c3ee6647ac9f266a7c628f082] Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git git bisect good c6ab5ee56509953c3ee6647ac9f266a7c628f082 # good: [37b72ff92ef30c021dca27cc5673d8c4bad49f8b] Merge branch 'usb-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git git bisect good 37b72ff92ef30c021dca27cc5673d8c4bad49f8b # good: [be5db029f3b05a02234a4b1530c0b3cdaf974718] Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git git bisect good be5db029f3b05a02234a4b1530c0b3cdaf974718 # good: [6071064d2ffec442f437277f7d411839b4898a77] Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/livepatching/livepatching git bisect good 6071064d2ffec442f437277f7d411839b4898a77 # bad: [7a7dd09ee43911c0c1f4cc65bcef2aa6482f0e91] Merge branch 'rust-next' of https://github.com/Rust-for-Linux/linux.git git bisect bad 7a7dd09ee43911c0c1f4cc65bcef2aa6482f0e91 # good: [f97b043bb40f86c715a50ba415d7fcdbed926b78] Merge branch 'zstd-next' of https://github.com/terrelln/linux.git git bisect good f97b043bb40f86c715a50ba415d7fcdbed926b78 # bad: [dd60abee3f85689f8a6b5c817ed7f7dcc56ef822] Merge branch 'slab/for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git git bisect bad dd60abee3f85689f8a6b5c817ed7f7dcc56ef822 # bad: [6f18bb40970790b176a7aede3f1054fa7b9aa718] efi: sysfb_efi: fix W=1 warnings when EFI is not set git bisect bad 6f18bb40970790b176a7aede3f1054fa7b9aa718 # bad: [9e45f9e16bc4d8fa963399808463d0f478e01911] efi/libstub: Simplify GOP handling code git bisect bad 9e45f9e16bc4d8fa963399808463d0f478e01911 # bad: [7f0158f3ee12403c26e815c9df4f00d41ee3bb41] efi/libstub: Avoid legacy decompressor zlib/zstd wrappers git bisect bad 7f0158f3ee12403c26e815c9df4f00d41ee3bb41 # good: [ec4696925da6b9baec38345184403ce9e29a2e48] efi/libstub: Bump up EFI_MMAP_NR_SLACK_SLOTS to 32 git bisect good ec4696925da6b9baec38345184403ce9e29a2e48 # first bad commit: [7f0158f3ee12403c26e815c9df4f00d41ee3bb41] efi/libstub: Avoid legacy decompressor zlib/zstd wrappers
On Tue, 14 Jan 2025 at 01:31, Guenter Roeck <linux@roeck-us.net> wrote: > > Hi, > > On Wed, Dec 18, 2024 at 04:03:17PM +0100, Ard Biesheuvel wrote: > > From: Ard Biesheuvel <ardb@kernel.org> > > > > Remove the dependency on the decompression wrappers used by the legacy > > decompressor, which do some odd things like providing a barebones > > malloc() implementation. Instead, implement GZIP deflate and ZSTD > > decompression in terms of the underlying libraries. > > > > Signed-off-by: Ard Biesheuvel <ardb@kernel.org> > > With this patch in linux-next, my loongarch boot tests with qemu > no longer boot. Log message is: > > qemu log: > EFI stub: Decompressing Linux Kernel... > EFI stub: EFI_RNG_PROTOCOL unavailable > EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path > EFI stub: Exiting boot services > > ... and then there is nothing until I abort the emulation. > > Reverting this patch results in a build failure, so I awas not able to > test it. Bisect results are atatched for reference. > Thanks for the report - I'll drop the patch for now, and try again next cycle.
On 1/14/25 03:40, Ard Biesheuvel wrote: > On Tue, 14 Jan 2025 at 08:33, Ard Biesheuvel <ardb@kernel.org> wrote: >> >> On Tue, 14 Jan 2025 at 01:31, Guenter Roeck <linux@roeck-us.net> wrote: >>> >>> Hi, >>> >>> On Wed, Dec 18, 2024 at 04:03:17PM +0100, Ard Biesheuvel wrote: >>>> From: Ard Biesheuvel <ardb@kernel.org> >>>> >>>> Remove the dependency on the decompression wrappers used by the legacy >>>> decompressor, which do some odd things like providing a barebones >>>> malloc() implementation. Instead, implement GZIP deflate and ZSTD >>>> decompression in terms of the underlying libraries. >>>> >>>> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> >>> >>> With this patch in linux-next, my loongarch boot tests with qemu >>> no longer boot. Log message is: >>> >>> qemu log: >>> EFI stub: Decompressing Linux Kernel... >>> EFI stub: EFI_RNG_PROTOCOL unavailable >>> EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path >>> EFI stub: Exiting boot services >>> >>> ... and then there is nothing until I abort the emulation. >>> >>> Reverting this patch results in a build failure, so I awas not able to >>> test it. Bisect results are atatched for reference. >>> >> >> Thanks for the report - I'll drop the patch for now, and try again next cycle. > > Mind sharing your config/firmware/command line details? I'm failing to > reproduce the issue. qemu-system-loongarch64 -M virt -cpu \ la464 -kernel arch/loongarch/boot/vmlinuz.efi -smp 2 \ -no-reboot -m 4G -initrd rootfs.cpio \ -bios QEMU_EFI-loongarch64.fd \ --append "panic=-1 kunit.stats_enabled=2 kunit.filter=speed>slow rdinit=/sbin/init console=ttyS0,115200 earlycon=uart8250,mmio,0x1fe001e0,115200n8" \ -nographic -serial stdio -monitor none qemu version is 9.1. The EFI image is at https://github.com/groeck/linux-build-test/blob/master/rootfs/firmware/QEMU_EFI-loongarch64.fd and the initrd is at https://github.com/groeck/linux-build-test/blob/master/rootfs/loongarch/rootfs.cpio.gz Configuration is defconfig with various debug options enabled. I'll be happy to make a test directory available with all information needed if that helps. Please let me know. Thanks, Guenter
On Tue, 14 Jan 2025 at 16:34, Ard Biesheuvel <ardb@kernel.org> wrote: > > On Tue, 14 Jan 2025 at 16:08, Guenter Roeck <linux@roeck-us.net> wrote: > > > > On 1/14/25 03:40, Ard Biesheuvel wrote: > > > On Tue, 14 Jan 2025 at 08:33, Ard Biesheuvel <ardb@kernel.org> wrote: > > >> > > >> On Tue, 14 Jan 2025 at 01:31, Guenter Roeck <linux@roeck-us.net> wrote: > > >>> > > >>> Hi, > > >>> > > >>> On Wed, Dec 18, 2024 at 04:03:17PM +0100, Ard Biesheuvel wrote: > > >>>> From: Ard Biesheuvel <ardb@kernel.org> > > >>>> > > >>>> Remove the dependency on the decompression wrappers used by the legacy > > >>>> decompressor, which do some odd things like providing a barebones > > >>>> malloc() implementation. Instead, implement GZIP deflate and ZSTD > > >>>> decompression in terms of the underlying libraries. > > >>>> > > >>>> Signed-off-by: Ard Biesheuvel <ardb@kernel.org> > > >>> > > >>> With this patch in linux-next, my loongarch boot tests with qemu > > >>> no longer boot. Log message is: > > >>> > > >>> qemu log: > > >>> EFI stub: Decompressing Linux Kernel... > > >>> EFI stub: EFI_RNG_PROTOCOL unavailable > > >>> EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path > > >>> EFI stub: Exiting boot services > > >>> > > >>> ... and then there is nothing until I abort the emulation. > > >>> > > >>> Reverting this patch results in a build failure, so I awas not able to > > >>> test it. Bisect results are atatched for reference. > > >>> > > >> > > >> Thanks for the report - I'll drop the patch for now, and try again next cycle. > > > > > > Mind sharing your config/firmware/command line details? I'm failing to > > > reproduce the issue. > > > > qemu-system-loongarch64 -M virt -cpu \ > > la464 -kernel arch/loongarch/boot/vmlinuz.efi -smp 2 \ > > -no-reboot -m 4G -initrd rootfs.cpio \ > > -bios QEMU_EFI-loongarch64.fd \ > > --append "panic=-1 kunit.stats_enabled=2 kunit.filter=speed>slow rdinit=/sbin/init console=ttyS0,115200 earlycon=uart8250,mmio,0x1fe001e0,115200n8" \ > > -nographic -serial stdio -monitor none > > > > qemu version is 9.1. The EFI image is at > > https://github.com/groeck/linux-build-test/blob/master/rootfs/firmware/QEMU_EFI-loongarch64.fd > > and the initrd is at > > https://github.com/groeck/linux-build-test/blob/master/rootfs/loongarch/rootfs.cpio.gz > > > > Configuration is defconfig with various debug options enabled. > > > > I'll be happy to make a test directory available with all information needed > > if that helps. Please let me know. > > > > Everything works fine with my distro QEMU: > > $ qemu-system-loongarch64 --version > QEMU emulator version 8.2.4 (Debian 1:8.2.4+ds-1+build1) > Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers > > and it doesn't matter if I use your firmware or my own (grabbed > randomly from [0]) > > When I build QEMU from source (stable-9.2), things still work happily > if I use that same firmware. With your firmware, my QEMU 9.2 is > completely dead. > > Where did you find that image? I tried rebuilding it myself from a > recent EDK2 base, but that doesn't work at all either. > > [0] https://github.com/AOSC-Dev/LoongArchQemuVirtFirmware ... actually, turns out I built the wrong platform. If I build OvmfPkg/LoongArchVirt/LoongArchVirtQemu.dsc from the EDK2 repository, I can boot the kernel with my change applied, using your command line. Perhaps I should try your .config as well?
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile index ed4e8ddbe76a..e04285a7a6b9 100644 --- a/drivers/firmware/efi/libstub/Makefile +++ b/drivers/firmware/efi/libstub/Makefile @@ -89,7 +89,12 @@ lib-$(CONFIG_LOONGARCH) += loongarch.o loongarch-stub.o CFLAGS_arm32-stub.o := -DTEXT_OFFSET=$(TEXT_OFFSET) -zboot-obj-$(CONFIG_RISCV) := lib-clz_ctz.o lib-ashldi3.o +zboot-obj-y := zboot-decompress-gzip.o +CFLAGS_zboot-decompress-gzip.o += -I$(srctree)/lib/zlib_inflate +zboot-obj-$(CONFIG_KERNEL_ZSTD) := zboot-decompress-zstd.o lib-xxhash.o +CFLAGS_zboot-decompress-zstd.o += -I$(srctree)/lib/zstd + +zboot-obj-$(CONFIG_RISCV) += lib-clz_ctz.o lib-ashldi3.o lib-$(CONFIG_EFI_ZBOOT) += zboot.o $(zboot-obj-y) lib-$(CONFIG_UNACCEPTED_MEMORY) += unaccepted_memory.o bitmap.o find.o diff --git a/drivers/firmware/efi/libstub/efistub.h b/drivers/firmware/efi/libstub/efistub.h index 76e44c185f29..172f4edab30b 100644 --- a/drivers/firmware/efi/libstub/efistub.h +++ b/drivers/firmware/efi/libstub/efistub.h @@ -1232,4 +1232,7 @@ void process_unaccepted_memory(u64 start, u64 end); void accept_memory(phys_addr_t start, unsigned long size); void arch_accept_memory(phys_addr_t start, phys_addr_t end); +efi_status_t efi_zboot_decompress_init(unsigned long *alloc_size); +efi_status_t efi_zboot_decompress(u8 *out, unsigned long outlen); + #endif diff --git a/drivers/firmware/efi/libstub/zboot-decompress-gzip.c b/drivers/firmware/efi/libstub/zboot-decompress-gzip.c new file mode 100644 index 000000000000..79cf8c48b033 --- /dev/null +++ b/drivers/firmware/efi/libstub/zboot-decompress-gzip.c @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/efi.h> +#include <linux/zlib.h> + +#include <asm/efi.h> + +#include "efistub.h" + +#include "inftrees.c" +#include "inffast.c" +#include "inflate.c" + +extern unsigned char _gzdata_start[], _gzdata_end[]; +extern u32 __aligned(1) payload_size; + +static struct z_stream_s stream; + +efi_status_t efi_zboot_decompress_init(unsigned long *alloc_size) +{ + efi_status_t status; + int rc; + + /* skip the 10 byte header, assume no recorded filename */ + stream.next_in = _gzdata_start + 10; + stream.avail_in = _gzdata_end - stream.next_in; + + status = efi_allocate_pages(zlib_inflate_workspacesize(), + (unsigned long *)&stream.workspace, + ULONG_MAX); + if (status != EFI_SUCCESS) + return status; + + rc = zlib_inflateInit2(&stream, -MAX_WBITS); + if (rc != Z_OK) { + efi_err("failed to initialize GZIP decompressor: %d\n", rc); + status = EFI_LOAD_ERROR; + goto out; + } + + *alloc_size = payload_size; + return EFI_SUCCESS; +out: + efi_free(zlib_inflate_workspacesize(), (unsigned long)stream.workspace); + return status; +} + +efi_status_t efi_zboot_decompress(u8 *out, unsigned long outlen) +{ + int rc; + + stream.next_out = out; + stream.avail_out = outlen; + + rc = zlib_inflate(&stream, 0); + zlib_inflateEnd(&stream); + + efi_free(zlib_inflate_workspacesize(), (unsigned long)stream.workspace); + + if (rc != Z_STREAM_END) { + efi_err("GZIP decompression failed with status %d\n", rc); + return EFI_LOAD_ERROR; + } + + return EFI_SUCCESS; +} diff --git a/drivers/firmware/efi/libstub/zboot-decompress-zstd.c b/drivers/firmware/efi/libstub/zboot-decompress-zstd.c new file mode 100644 index 000000000000..268ae53c6fda --- /dev/null +++ b/drivers/firmware/efi/libstub/zboot-decompress-zstd.c @@ -0,0 +1,81 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/efi.h> +#include <linux/zstd.h> + +#include <asm/efi.h> + +#include "decompress_sources.h" +#include "efistub.h" + +extern unsigned char _gzdata_start[], _gzdata_end[]; +extern u32 __aligned(1) payload_size; + +static ZSTD_inBuffer zstd_buf; +static ZSTD_DStream *dstream; +static size_t wksp_size; +static void *wksp; + +efi_status_t efi_zboot_decompress_init(unsigned long *alloc_size) +{ + zstd_frame_header header; + efi_status_t status; + size_t ret; + + zstd_buf.src = _gzdata_start; + zstd_buf.pos = 0; + zstd_buf.size = _gzdata_end - _gzdata_start; + + ret = zstd_get_frame_header(&header, zstd_buf.src, zstd_buf.size); + if (ret != 0) { + efi_err("ZSTD-compressed data has an incomplete frame header\n"); + status = EFI_LOAD_ERROR; + goto out; + } + + if (header.windowSize > (1 << ZSTD_WINDOWLOG_MAX)) { + efi_err("ZSTD-compressed data has too large a window size\n"); + status = EFI_LOAD_ERROR; + goto out; + } + + wksp_size = zstd_dstream_workspace_bound(header.windowSize); + status = efi_allocate_pages(wksp_size, (unsigned long *)&wksp, ULONG_MAX); + if (status != EFI_SUCCESS) + goto out; + + dstream = zstd_init_dstream(header.windowSize, wksp, wksp_size); + if (!dstream) { + efi_err("Can't initialize ZSTD stream\n"); + status = EFI_OUT_OF_RESOURCES; + goto out; + } + + *alloc_size = payload_size; + return EFI_SUCCESS; +out: + efi_free(wksp_size, (unsigned long)wksp); + return status; +} + +efi_status_t efi_zboot_decompress(u8 *out, unsigned long outlen) +{ + ZSTD_outBuffer zstd_dec; + size_t ret; + int retval; + + zstd_dec.dst = out; + zstd_dec.pos = 0; + zstd_dec.size = outlen; + + ret = zstd_decompress_stream(dstream, &zstd_dec, &zstd_buf); + efi_free(wksp_size, (unsigned long)wksp); + + retval = zstd_get_error_code(ret); + if (retval) { + efi_err("ZSTD-decompression failed with status %d\n", retval); + return EFI_LOAD_ERROR; + } + + return EFI_SUCCESS; +} diff --git a/drivers/firmware/efi/libstub/zboot.c b/drivers/firmware/efi/libstub/zboot.c index af23b3c50228..4a885fbe1ccc 100644 --- a/drivers/firmware/efi/libstub/zboot.c +++ b/drivers/firmware/efi/libstub/zboot.c @@ -7,36 +7,6 @@ #include "efistub.h" -static unsigned char zboot_heap[SZ_256K] __aligned(64); -static unsigned long free_mem_ptr, free_mem_end_ptr; - -#define STATIC static -#if defined(CONFIG_KERNEL_GZIP) -#include "../../../../lib/decompress_inflate.c" -#elif defined(CONFIG_KERNEL_LZ4) -#include "../../../../lib/decompress_unlz4.c" -#elif defined(CONFIG_KERNEL_LZMA) -#include "../../../../lib/decompress_unlzma.c" -#elif defined(CONFIG_KERNEL_LZO) -#include "../../../../lib/decompress_unlzo.c" -#elif defined(CONFIG_KERNEL_XZ) -#undef memcpy -#define memcpy memcpy -#undef memmove -#define memmove memmove -#include "../../../../lib/decompress_unxz.c" -#elif defined(CONFIG_KERNEL_ZSTD) -#include "../../../../lib/decompress_unzstd.c" -#endif - -extern char efi_zboot_header[]; -extern char _gzdata_start[], _gzdata_end[]; - -static void error(char *x) -{ - efi_err("EFI decompressor: %s\n", x); -} - static unsigned long alloc_preferred_address(unsigned long alloc_size) { #ifdef EFI_KIMG_PREFERRED_ADDRESS @@ -64,22 +34,17 @@ struct screen_info *alloc_screen_info(void) asmlinkage efi_status_t __efiapi efi_zboot_entry(efi_handle_t handle, efi_system_table_t *systab) { - unsigned long compressed_size = _gzdata_end - _gzdata_start; unsigned long image_base, alloc_size; efi_loaded_image_t *image; efi_status_t status; char *cmdline_ptr; - int ret; WRITE_ONCE(efi_system_table, systab); - free_mem_ptr = (unsigned long)&zboot_heap; - free_mem_end_ptr = free_mem_ptr + sizeof(zboot_heap); - status = efi_bs_call(handle_protocol, handle, &LOADED_IMAGE_PROTOCOL_GUID, (void **)&image); if (status != EFI_SUCCESS) { - error("Failed to locate parent's loaded image protocol"); + efi_err("Failed to locate parent's loaded image protocol\n"); return status; } @@ -89,9 +54,9 @@ efi_zboot_entry(efi_handle_t handle, efi_system_table_t *systab) efi_info("Decompressing Linux Kernel...\n"); - // SizeOfImage from the compressee's PE/COFF header - alloc_size = round_up(get_unaligned_le32(_gzdata_end - 4), - EFI_ALLOC_ALIGN); + status = efi_zboot_decompress_init(&alloc_size); + if (status != EFI_SUCCESS) + return status; // If the architecture has a preferred address for the image, // try that first. @@ -127,13 +92,9 @@ efi_zboot_entry(efi_handle_t handle, efi_system_table_t *systab) } // Decompress the payload into the newly allocated buffer. - ret = __decompress(_gzdata_start, compressed_size, NULL, NULL, - (void *)image_base, alloc_size, NULL, error); - if (ret < 0) { - error("Decompression failed"); - status = EFI_DEVICE_ERROR; + status = efi_zboot_decompress((void *)image_base, alloc_size); + if (status != EFI_SUCCESS) goto free_image; - } efi_cache_sync_image(image_base, alloc_size); diff --git a/drivers/firmware/efi/libstub/zboot.lds b/drivers/firmware/efi/libstub/zboot.lds index af2c82f7bd90..9ecc57ff5b45 100644 --- a/drivers/firmware/efi/libstub/zboot.lds +++ b/drivers/firmware/efi/libstub/zboot.lds @@ -17,6 +17,7 @@ SECTIONS .rodata : ALIGN(8) { __efistub__gzdata_start = .; *(.gzdata) + __efistub_payload_size = . - 4; __efistub__gzdata_end = .; *(.rodata* .init.rodata* .srodata*)