| Message ID | 20220318155304.14916-1-zajec5@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | UBI: support EOF block marking end of ubinized image | expand |
On 18.03.2022 16:53, Rafał Miłecki wrote: > From: Rafał Miłecki <rafal@milecki.pl> > > On home routers UBI is often used with UBI unaware bootloaders. In such > case an ubinized image needs to be created & flashed. > > The problem is many such bootloaders are poorly written and they don't > erase whole flash (partition) before writing an image. Only part of > flash required to fit new image gets erased. It results in UBI > complaining about flash garbage content, e.g.: > > ubi0: attaching mtd2 > ubi0 error: ubi_attach: bad image sequence number 1646161998 in PEB 119, expected 524503983 > Erase counter header dump: > magic 0x55424923 > version 1 > ec 1 > vid_hdr_offset 2048 > data_offset 4096 > image_seq 1646161998 > hdr_crc 0x1eb28994 > erase counter header hexdump: > ubi0 error: ubi_attach_mtd_dev: failed to attach mtd2, error -22 > UBI error: cannot attach mtd2 > > To fix up such flashed images it's required to identify the end of > ubinized image on flash and erase remaining blocks. > > Idea of extending on-disk UBI format to mark end of ubinized image was > rejected, see: > [PATCH RFC 2/2] ubi: add support for UBI_EC_FLAG_ERASE_FROM_HERE > https://patchwork.ozlabs.org/project/linux-mtd/patch/20161230171151.13448-2-zajec5@gmail.com/ > > This patch implements support for a simple "EOF" text content (block) > appended to the ubinized image instead. It doesn't require changing > on-disk format and still allows simple integration into ubi code. > > Signed-off-by: Rafał Miłecki <rafal@milecki.pl> While I personally think that UBI should provide *any* solution for such poor bootloaders it has been stated upstream won't accept this hack. There is no idea for any other / cleaner solution neither. For details see e-mail thread: Handling ubinized image flashed on not erased flash space http://lists.infradead.org/pipermail/linux-mtd/2022-March/091993.html http://lists.infradead.org/pipermail/linux-mtd/2022-March/092081.html This patch is meant for downstream usage in projects like OpenWrt / Buildroot / Yocto / foo.
Hi, Rafał > While I personally think that UBI should provide *any* solution for such > poor bootloaders it has been stated upstream won't accept this hack. > There is no idea for any other / cleaner solution neither. > > For details see e-mail thread: > Handling ubinized image flashed on not erased flash space > http://lists.infradead.org/pipermail/linux-mtd/2022-March/091993.html > http://lists.infradead.org/pipermail/linux-mtd/2022-March/092081.html > > This patch is meant for downstream usage in projects like OpenWrt / > Buildroot / Yocto / foo. Does 'downstream' mean this patch won't be applied to mainline? Eventhough, I don't think this modification is sutiable for common scenarios. The UBI image provided by poor bootloader does not follow the standard UBI format defined in [1], and linux kernel should have recognized bad image and rejected attaching rather than tries fixing bad image. Maybe there are other better solutions? Such as, 1. For 'EOF' ended image scenarios, DIY an userspace tool to convert bad UBI image to valid UBI image. 2. Erasing/Formatting the whole mtd partition. (BTW, why does booloader only format part of mtd partition? What will the left space in mtd partition be used for?) [1] http://www.linux-mtd.infradead.org/doc/ubi.html#L_ubi_headers > > ______________________________________________________ > Linux MTD discussion mailing list > http://lists.infradead.org/mailman/listinfo/linux-mtd/
On 19.03.2022 04:15, Zhihao Cheng wrote: >> While I personally think that UBI should provide *any* solution for such >> poor bootloaders it has been stated upstream won't accept this hack. >> There is no idea for any other / cleaner solution neither. >> >> For details see e-mail thread: >> Handling ubinized image flashed on not erased flash space >> http://lists.infradead.org/pipermail/linux-mtd/2022-March/091993.html >> http://lists.infradead.org/pipermail/linux-mtd/2022-March/092081.html >> >> This patch is meant for downstream usage in projects like OpenWrt / >> Buildroot / Yocto / foo. > Does 'downstream' mean this patch won't be applied to mainline? Eventhough, I don't think this modification is sutiable for common scenarios. The UBI image provided by poor bootloader does not follow the standard UBI format defined in [1], and linux kernel should have recognized bad image and rejected attaching rather than tries fixing bad image. Yes, downstream means carrying on that patch in projects using Linux kernel (I named few). They have to patch kernel with it before compiling it. Upstream means simply: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git I'm not sure what do you mean by "sutiable". Linux doesn't really offer for raw NAND flashes anything except UBI. So that's the only choice. Of course those poor bootloaders don't "follow" UBI on-disk format. We all know that. The problem is they are there. On millions of devices. People out of your utopia still need a way to use UBI with them. All crazy ideas like flashing intermediate firmware just to flash a target one properly (with UBI proper flash setup) just don't work in real world > Maybe there are other better solutions? Such as, > 1. For 'EOF' ended image scenarios, DIY an userspace tool to convert bad UBI image to valid UBI image. How does it make life easier for anyone? 1. Every distro still has to carry on such user-space tool 2. We don't have any intermediate rootfs so it needs to go to initramfs 3. That may confict with other initramfs usages 4. It requires scanning blocks twice (in user-space and then in kernel) I don't see any gain from moving that task from kernel to user-space. > 2. Erasing/Formatting the whole mtd partition. (BTW, why does booloader only format part of mtd partition? What will the left space in mtd partition be used for?) Don't ask me. Ask ASUS, Linksys, Netgear, TP-Link, whoever. I think you can even try Huawei - they seem to have some Broadcom and Ralink based hw that does the same.
diff --git a/drivers/mtd/ubi/attach.c b/drivers/mtd/ubi/attach.c index ae5abe492b52..8bea4bbcfbd9 100644 --- a/drivers/mtd/ubi/attach.c +++ b/drivers/mtd/ubi/attach.c @@ -958,9 +958,28 @@ static int scan_peb(struct ubi_device *ubi, struct ubi_attach_info *ai, return 0; } + if (ai->eof_found) { + ai->empty_peb_count += 1; + return add_to_list(ai, pnum, UBI_UNKNOWN, UBI_UNKNOWN, + UBI_UNKNOWN, 0, &ai->erase); + } + err = ubi_io_read_ec_hdr(ubi, pnum, ech, 0); if (err < 0) return err; + + if (err == UBI_IO_BAD_HDR) { + uint8_t eof[] = { 'E', 'O', 'F' }; + uint8_t *data = (uint8_t *)ech; + + if (!memcmp(data, eof, sizeof(eof))) { + ai->eof_found = true; + ai->empty_peb_count += 1; + return add_to_list(ai, pnum, UBI_UNKNOWN, UBI_UNKNOWN, + UBI_UNKNOWN, 0, &ai->erase); + } + } + switch (err) { case 0: break; diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h index 7c083ad58274..218c7dfedfc0 100644 --- a/drivers/mtd/ubi/ubi.h +++ b/drivers/mtd/ubi/ubi.h @@ -783,6 +783,7 @@ struct ubi_attach_info { struct kmem_cache *aeb_slab_cache; struct ubi_ec_hdr *ech; struct ubi_vid_io_buf *vidb; + bool eof_found; }; /**