[v2,00/15] env: Multiple env support and env transition for sunxi

Message ID cover.fedfb8856bdb3ebff3ce72afb6d0e1e0bcb9675c.1516094113.git-series.maxime.ripard@free-electrons.com
Headers show
Series
  • env: Multiple env support and env transition for sunxi
Related show

Message

Maxime Ripard Jan. 16, 2018, 9:16 a.m.
Hi,

Here is a second attempt at transitioning away from the MMC raw environment
to a FAT-based one for Allwinner SoCs. Since the RFC was quite well
received, I actually tested it and fixed a few rough edges.

You'll find the first RFC here for reference:
https://lists.denx.de/pipermail/u-boot/2017-October/310111.html

And the second that originated in this series:
https://lists.denx.de/pipermail/u-boot/2017-November/311608.html

I gave it a travis run, and one test seems to fail in one test:
https://travis-ci.org/mripard/u-boot/jobs/329229382

I really cannot really make any sense of why a change in the way the
environment loads could affect the operation of a DHCP client. Is there a
way to reproduce locally?

Thanks!
Maxime

Original description of the patches:

The fundamental issue I'm trying to adress is that we've had for a
very long time the assumption that the main U-Boot binary wouldn't
exceed around 500 bytes.

However, we're starting to get real close to that limit, and are
running out of silver bullets to deal with the consequences of having
a bigger U-Boot binary, the main consequence being that we would
have some overlap between the environment and U-Boot.

One way to address this that has been suggested by Tom is to move away
from the raw MMC environment to a FAT-based one. This would allow us
to slowly migrate away, and eventually remove the MMC-raw option
entirely to reclaim that space for the binary.

That cannot be done in a single release however, since we might have
environments in the wild already that people rely on. And since we
always encouraged people to use the raw MMC environment, noone would
expect that.

This is even worse since some platforms are using the U-Boot
environment to deal with implement their upgrade mechanism, such as
mender.io, and force moving the environment would break any further
upgrade.

The suggested implementation is to allow U-Boot to compile multiple
environments backend at once, based on the work done by Simon. The
default behaviour shouldn't change obviously. We can then piggy-back
on this to tweak on a per-board basis the environment lookup algorithm
to always favour the FAT-based environment and then fallback to the
MMC. It will allow us to migrate a raw-MMC user to a FAT based
solution as soon as they update their environment (assuming that there
is a bootable FAT partition in the system).

This has just been compile tested on sunxi so far, and I'd like
general comments on the approach taken. Obviously, this will need to
work properly before being merged.

Changes from v1:
  - Collect tags
  - Rebased on v2018.01
  - Fixed build failures on a couple of boards
  - Added back the message and the error code when an environment is
    failing
  - Added some comments about the printf in environments
  - Added a build time check about the number of our locations array to see
    if we're overflowing the location variable
  - Fixed the drv->init return code being ignored
  - Added helpers to manage the init status
  - Changed the ENVO prefix for the operations to ENVOP
  - Added some comments and documentation

Changes from the RFC:
  - Added more useful messages to see where we're loading / saving
  - Init all the environments no matter what, and the deal with whatever
    env we want to pick at load time
  - Added the various tags collected

Maxime Ripard (15):
  cmd: nvedit: Get rid of the env lookup
  env: Rename env_driver_lookup_default and env_get_default_location
  env: Pass additional parameters to the env lookup function
  env: Make the env save message a bit more explicit
  env: Make it explicit where we're loading our environment from
  env: fat: Make the debug messages play a little nicer
  env: mmc: Make the debug messages play a little nicer
  env: common: Make the debug messages play a little nicer
  env: Support multiple environments
  env: Initialise all the environments
  env: mmc: depends on the MMC framework
  env: Allow to build multiple environments in Kconfig
  env: Mark env_get_location as weak
  sunxi: Transition from the MMC to a FAT-based environment
  env: sunxi: Enable FAT-based environment support by default

 board/sunxi/board.c                   |  16 ++-
 cmd/nvedit.c                          |   4 +-
 configs/MPC8313ERDB_NAND_33_defconfig |   1 +-
 configs/MPC8313ERDB_NAND_66_defconfig |   1 +-
 configs/cl-som-imx7_defconfig         |   1 +-
 configs/microblaze-generic_defconfig  |   1 +-
 env/Kconfig                           |  70 ++++----
 env/common.c                          |   2 +-
 env/env.c                             | 252 +++++++++++++++++++--------
 env/fat.c                             |  25 ++-
 env/mmc.c                             |   1 +-
 include/asm-generic/global_data.h     |   1 +-
 include/environment.h                 |  15 +-
 13 files changed, 267 insertions(+), 123 deletions(-)

base-commit: f3dd87e0b98999a78e500e8c6d2b063ebadf535a

Comments

Tom Rini Jan. 16, 2018, 9:56 p.m. | #1
On Tue, Jan 16, 2018 at 10:16:36AM +0100, Maxime Ripard wrote:
> Hi,

> 

> Here is a second attempt at transitioning away from the MMC raw environment

> to a FAT-based one for Allwinner SoCs. Since the RFC was quite well

> received, I actually tested it and fixed a few rough edges.

> 

> You'll find the first RFC here for reference:

> https://lists.denx.de/pipermail/u-boot/2017-October/310111.html

> 

> And the second that originated in this series:

> https://lists.denx.de/pipermail/u-boot/2017-November/311608.html

> 

> I gave it a travis run, and one test seems to fail in one test:

> https://travis-ci.org/mripard/u-boot/jobs/329229382

> 

> I really cannot really make any sense of why a change in the way the

> environment loads could affect the operation of a DHCP client. Is there a

> way to reproduce locally?


In this case, roughly, clone https://github.com/swarren/uboot-test-hooks
and then:
$ export PATH=${PATH}:/path/to/uboot-test-hooks/bin
$ export PYTHONPATH=/path/to/uboot-test-hooks/py/travis-ci:${PYTHONPATH}
$ export UBOOT_TRAVIS_BUILD_DIR=`cd .. && pwd`/.bm-work/$1
$ ./test/py/test.py --bd vexpress_ca15_tc2 --build-dir "$UBOOT_TRAVIS_BUILD_DIR" -k 'net'

-- 
Tom
Maxime Ripard Jan. 17, 2018, 8:58 a.m. | #2
Hi,

On Tue, Jan 16, 2018 at 04:56:49PM -0500, Tom Rini wrote:
> On Tue, Jan 16, 2018 at 10:16:36AM +0100, Maxime Ripard wrote:

> > Hi,

> > 

> > Here is a second attempt at transitioning away from the MMC raw environment

> > to a FAT-based one for Allwinner SoCs. Since the RFC was quite well

> > received, I actually tested it and fixed a few rough edges.

> > 

> > You'll find the first RFC here for reference:

> > https://lists.denx.de/pipermail/u-boot/2017-October/310111.html

> > 

> > And the second that originated in this series:

> > https://lists.denx.de/pipermail/u-boot/2017-November/311608.html

> > 

> > I gave it a travis run, and one test seems to fail in one test:

> > https://travis-ci.org/mripard/u-boot/jobs/329229382

> > 

> > I really cannot really make any sense of why a change in the way the

> > environment loads could affect the operation of a DHCP client. Is there a

> > way to reproduce locally?

> 

> In this case, roughly, clone https://github.com/swarren/uboot-test-hooks

> and then:

> $ export PATH=${PATH}:/path/to/uboot-test-hooks/bin

> $ export PYTHONPATH=/path/to/uboot-test-hooks/py/travis-ci:${PYTHONPATH}

> $ export UBOOT_TRAVIS_BUILD_DIR=`cd .. && pwd`/.bm-work/$1

> $ ./test/py/test.py --bd vexpress_ca15_tc2 --build-dir "$UBOOT_TRAVIS_BUILD_DIR" -k 'net'


Thanks, I had to tweak it a bit to add --id qemu, and copy the
conf.vexpress_ca15_tc2_qemu file in a directory with my hostname in
the uboot-test-hooks repo.

And this is really getting weird, since the tests passes on my
machine:
http://code.bulix.org/rp6z29-258577

Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com
Tom Rini Jan. 17, 2018, 1:01 p.m. | #3
On Wed, Jan 17, 2018 at 09:58:53AM +0100, Maxime Ripard wrote:
> Hi,

> 

> On Tue, Jan 16, 2018 at 04:56:49PM -0500, Tom Rini wrote:

> > On Tue, Jan 16, 2018 at 10:16:36AM +0100, Maxime Ripard wrote:

> > > Hi,

> > > 

> > > Here is a second attempt at transitioning away from the MMC raw environment

> > > to a FAT-based one for Allwinner SoCs. Since the RFC was quite well

> > > received, I actually tested it and fixed a few rough edges.

> > > 

> > > You'll find the first RFC here for reference:

> > > https://lists.denx.de/pipermail/u-boot/2017-October/310111.html

> > > 

> > > And the second that originated in this series:

> > > https://lists.denx.de/pipermail/u-boot/2017-November/311608.html

> > > 

> > > I gave it a travis run, and one test seems to fail in one test:

> > > https://travis-ci.org/mripard/u-boot/jobs/329229382

> > > 

> > > I really cannot really make any sense of why a change in the way the

> > > environment loads could affect the operation of a DHCP client. Is there a

> > > way to reproduce locally?

> > 

> > In this case, roughly, clone https://github.com/swarren/uboot-test-hooks

> > and then:

> > $ export PATH=${PATH}:/path/to/uboot-test-hooks/bin

> > $ export PYTHONPATH=/path/to/uboot-test-hooks/py/travis-ci:${PYTHONPATH}

> > $ export UBOOT_TRAVIS_BUILD_DIR=`cd .. && pwd`/.bm-work/$1

> > $ ./test/py/test.py --bd vexpress_ca15_tc2 --build-dir "$UBOOT_TRAVIS_BUILD_DIR" -k 'net'

> 

> Thanks, I had to tweak it a bit to add --id qemu, and copy the

> conf.vexpress_ca15_tc2_qemu file in a directory with my hostname in

> the uboot-test-hooks repo.


Ah right, yes, oops.  I quickly grabbed bits out of my local wrapper
script around it.

> And this is really getting weird, since the tests passes on my

> machine:

> http://code.bulix.org/rp6z29-258577


Does it repeatedly fail in travis (which, uh, stopped working for me
yesterday, I haven't seen if it's un-wedged today)?  You can restart a
specific sub-job rather than the whole.

-- 
Tom
Maxime Ripard Jan. 17, 2018, 1:27 p.m. | #4
On Wed, Jan 17, 2018 at 08:01:34AM -0500, Tom Rini wrote:
> On Wed, Jan 17, 2018 at 09:58:53AM +0100, Maxime Ripard wrote:

> > Hi,

> > 

> > On Tue, Jan 16, 2018 at 04:56:49PM -0500, Tom Rini wrote:

> > > On Tue, Jan 16, 2018 at 10:16:36AM +0100, Maxime Ripard wrote:

> > > > Hi,

> > > > 

> > > > Here is a second attempt at transitioning away from the MMC raw environment

> > > > to a FAT-based one for Allwinner SoCs. Since the RFC was quite well

> > > > received, I actually tested it and fixed a few rough edges.

> > > > 

> > > > You'll find the first RFC here for reference:

> > > > https://lists.denx.de/pipermail/u-boot/2017-October/310111.html

> > > > 

> > > > And the second that originated in this series:

> > > > https://lists.denx.de/pipermail/u-boot/2017-November/311608.html

> > > > 

> > > > I gave it a travis run, and one test seems to fail in one test:

> > > > https://travis-ci.org/mripard/u-boot/jobs/329229382

> > > > 

> > > > I really cannot really make any sense of why a change in the way the

> > > > environment loads could affect the operation of a DHCP client. Is there a

> > > > way to reproduce locally?

> > > 

> > > In this case, roughly, clone https://github.com/swarren/uboot-test-hooks

> > > and then:

> > > $ export PATH=${PATH}:/path/to/uboot-test-hooks/bin

> > > $ export PYTHONPATH=/path/to/uboot-test-hooks/py/travis-ci:${PYTHONPATH}

> > > $ export UBOOT_TRAVIS_BUILD_DIR=`cd .. && pwd`/.bm-work/$1

> > > $ ./test/py/test.py --bd vexpress_ca15_tc2 --build-dir "$UBOOT_TRAVIS_BUILD_DIR" -k 'net'

> > 

> > Thanks, I had to tweak it a bit to add --id qemu, and copy the

> > conf.vexpress_ca15_tc2_qemu file in a directory with my hostname in

> > the uboot-test-hooks repo.

> 

> Ah right, yes, oops.  I quickly grabbed bits out of my local wrapper

> script around it.


Don't worry, I was on a good enough path :)

> > And this is really getting weird, since the tests passes on my

> > machine:

> > http://code.bulix.org/rp6z29-258577

> 

> Does it repeatedly fail in travis (which, uh, stopped working for me

> yesterday, I haven't seen if it's un-wedged today)?  You can restart a

> specific sub-job rather than the whole.


It was repeatedly failing for that commit, but I reordered a few
commits since (and I tested the new branch on my machine). I triggered
a new build on travis that is currently running, but is taking a long
time to complete. I'll let you know the results.

Thanks!
Maxime

-- 
Maxime Ripard, Free Electrons
Embedded Linux and Kernel engineering
http://free-electrons.com