mbox series

[00/20] gitlab: Simplify the test script

Message ID 20200307030734.237401-1-sjg@chromium.org
Headers show
Series gitlab: Simplify the test script | expand

Message

Simon Glass March 7, 2020, 3:07 a.m. UTC
At present there are several things in the gitlab script which work around
limitations in buildman. With a few small feature additions these can be
removed.

This series adds some new features to buildman and simplifies the script:
- Option to run a single build in a specified output directory
- Allow ignoring warnings
- Removes a restriction on the build output directory

It also
- moves test.py over to use buildman for the --build option
- makes one change to azure since the same approach should be possible there
- fixes a few minor problems noticed in main and sandbox docs


Simon Glass (20):
  sandbox: Add documentation about required/useful packages
  main: Drop show_boot_progress() prototype
  buildman: Document the members of BuilderJob
  bulidman: Add support for a simple build
  gitlab: Use the -w option for sandbox_spl
  azure: Use the -w option for sandbox_spl
  gitlab: Use the --board buildman flag
  gitlab: Drop the BUILDMAN variable
  buildman: Update help for -d
  gitlab: Drop the buildman -d flag
  gitlab: Drop unnecessary if..fi
  gitlab: Use -w flag for all builds
  gitlab: Use bash to avoid needing a_test_which_does_not_exist
  buildman: Allow ignoring warnings in the return code
  gitlab: Use the buildman -W flag
  gitlab: Enable test_handoff
  buildman: Be more selective about which directories to remove
  buildman: Allow building within a subdir of the current dir
  test/py: Use buildman to build U-Boot
  gitlab: Simplify the exit code for test.py

 .azure-pipelines.yml            |  4 +-
 .gitlab-ci.yml                  | 81 +++++++++------------------------
 common/main.c                   |  5 --
 doc/arch/sandbox.rst            | 10 ++++
 test/py/conftest.py             | 16 +++----
 tools/buildman/README           | 13 +++++-
 tools/buildman/builder.py       | 51 ++++++++++++++++-----
 tools/buildman/builderthread.py | 34 ++++++++++----
 tools/buildman/cmdline.py       |  6 ++-
 tools/buildman/control.py       | 35 ++++----------
 tools/buildman/func_test.py     | 46 ++++++++++++++-----
 tools/buildman/test.py          | 20 ++++++++
 12 files changed, 186 insertions(+), 135 deletions(-)

Comments

Tom Rini March 9, 2020, 5:55 p.m. UTC | #1
On Fri, Mar 06, 2020 at 08:07:14PM -0700, Simon Glass wrote:

> At present there are several things in the gitlab script which work around
> limitations in buildman. With a few small feature additions these can be
> removed.
> 
> This series adds some new features to buildman and simplifies the script:
> - Option to run a single build in a specified output directory
> - Allow ignoring warnings
> - Removes a restriction on the build output directory
> 
> It also
> - moves test.py over to use buildman for the --build option
> - makes one change to azure since the same approach should be possible there
> - fixes a few minor problems noticed in main and sandbox docs

One general comment is that while it's clear from this series that
you're focusing on test.py invocation most of the time, a lot of the
commit messages aren't clear that you're changing the
buildman_and_testpy_template stanza.

Also, while you're keeping Azure in sync (as it's largely based on the
GitLab logic), Travis is being left out.  But the GitLab logic is based
on the Travis logic and as much as I dislike Travis for being slow and
having network induced failures, it's still a valid and supported CI
flow that we can't drop, so please keep it in sync with these
improvements, thanks!
Simon Glass March 15, 2020, 3:10 a.m. UTC | #2
Hi Tom,

On Mon, 9 Mar 2020 at 11:55, Tom Rini <trini at konsulko.com> wrote:
>
> On Fri, Mar 06, 2020 at 08:07:14PM -0700, Simon Glass wrote:
>
> > At present there are several things in the gitlab script which work around
> > limitations in buildman. With a few small feature additions these can be
> > removed.
> >
> > This series adds some new features to buildman and simplifies the script:
> > - Option to run a single build in a specified output directory
> > - Allow ignoring warnings
> > - Removes a restriction on the build output directory
> >
> > It also
> > - moves test.py over to use buildman for the --build option
> > - makes one change to azure since the same approach should be possible there
> > - fixes a few minor problems noticed in main and sandbox docs
>
> One general comment is that while it's clear from this series that
> you're focusing on test.py invocation most of the time, a lot of the
> commit messages aren't clear that you're changing the
> buildman_and_testpy_template stanza.

I'm not sure what you are looking for there. Do you want the commit
message to mention which part of the gitlab file is being changed?

>
> Also, while you're keeping Azure in sync (as it's largely based on the
> GitLab logic), Travis is being left out.  But the GitLab logic is based
> on the Travis logic and as much as I dislike Travis for being slow and
> having network induced failures, it's still a valid and supported CI
> flow that we can't drop, so please keep it in sync with these
> improvements, thanks!

OK.

Regards,
Simon
Tom Rini March 15, 2020, 1:02 p.m. UTC | #3
On Sat, Mar 14, 2020 at 09:10:09PM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Mon, 9 Mar 2020 at 11:55, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Fri, Mar 06, 2020 at 08:07:14PM -0700, Simon Glass wrote:
> >
> > > At present there are several things in the gitlab script which work around
> > > limitations in buildman. With a few small feature additions these can be
> > > removed.
> > >
> > > This series adds some new features to buildman and simplifies the script:
> > > - Option to run a single build in a specified output directory
> > > - Allow ignoring warnings
> > > - Removes a restriction on the build output directory
> > >
> > > It also
> > > - moves test.py over to use buildman for the --build option
> > > - makes one change to azure since the same approach should be possible there
> > > - fixes a few minor problems noticed in main and sandbox docs
> >
> > One general comment is that while it's clear from this series that
> > you're focusing on test.py invocation most of the time, a lot of the
> > commit messages aren't clear that you're changing the
> > buildman_and_testpy_template stanza.
> 
> I'm not sure what you are looking for there. Do you want the commit
> message to mention which part of the gitlab file is being changed?

I mean a lot of places say something like "Change gitlab ..." but aren't
changing any of the world builds, only test.py stuff, so I would prefer
"Change gitlab when using test.py ..."
Thanks!
Simon Glass March 15, 2020, 3:07 p.m. UTC | #4
Hi Tom,

On Sun, 15 Mar 2020 at 07:02, Tom Rini <trini at konsulko.com> wrote:
>
> On Sat, Mar 14, 2020 at 09:10:09PM -0600, Simon Glass wrote:
> > Hi Tom,
> >
> > On Mon, 9 Mar 2020 at 11:55, Tom Rini <trini at konsulko.com> wrote:
> > >
> > > On Fri, Mar 06, 2020 at 08:07:14PM -0700, Simon Glass wrote:
> > >
> > > > At present there are several things in the gitlab script which work around
> > > > limitations in buildman. With a few small feature additions these can be
> > > > removed.
> > > >
> > > > This series adds some new features to buildman and simplifies the script:
> > > > - Option to run a single build in a specified output directory
> > > > - Allow ignoring warnings
> > > > - Removes a restriction on the build output directory
> > > >
> > > > It also
> > > > - moves test.py over to use buildman for the --build option
> > > > - makes one change to azure since the same approach should be possible there
> > > > - fixes a few minor problems noticed in main and sandbox docs
> > >
> > > One general comment is that while it's clear from this series that
> > > you're focusing on test.py invocation most of the time, a lot of the
> > > commit messages aren't clear that you're changing the
> > > buildman_and_testpy_template stanza.
> >
> > I'm not sure what you are looking for there. Do you want the commit
> > message to mention which part of the gitlab file is being changed?
>
> I mean a lot of places say something like "Change gitlab ..." but aren't
> changing any of the world builds, only test.py stuff, so I would prefer
> "Change gitlab when using test.py ..."

OK I will try. I've got the changes to all three environments in a
single commit at present, so we don't end up with double the commis.
Or would you prefer it split out?

One more thing...I notice that gitlab and azure use 'set -ex' to avoid
needing to check errors in the script. Is is possible for travis to do
that do, or is there some restriction?

> Thanks!
>
> --
> Tom

Regards,
Simon
Tom Rini March 15, 2020, 3:17 p.m. UTC | #5
On Sun, Mar 15, 2020 at 09:07:48AM -0600, Simon Glass wrote:
> Hi Tom,
> 
> On Sun, 15 Mar 2020 at 07:02, Tom Rini <trini at konsulko.com> wrote:
> >
> > On Sat, Mar 14, 2020 at 09:10:09PM -0600, Simon Glass wrote:
> > > Hi Tom,
> > >
> > > On Mon, 9 Mar 2020 at 11:55, Tom Rini <trini at konsulko.com> wrote:
> > > >
> > > > On Fri, Mar 06, 2020 at 08:07:14PM -0700, Simon Glass wrote:
> > > >
> > > > > At present there are several things in the gitlab script which work around
> > > > > limitations in buildman. With a few small feature additions these can be
> > > > > removed.
> > > > >
> > > > > This series adds some new features to buildman and simplifies the script:
> > > > > - Option to run a single build in a specified output directory
> > > > > - Allow ignoring warnings
> > > > > - Removes a restriction on the build output directory
> > > > >
> > > > > It also
> > > > > - moves test.py over to use buildman for the --build option
> > > > > - makes one change to azure since the same approach should be possible there
> > > > > - fixes a few minor problems noticed in main and sandbox docs
> > > >
> > > > One general comment is that while it's clear from this series that
> > > > you're focusing on test.py invocation most of the time, a lot of the
> > > > commit messages aren't clear that you're changing the
> > > > buildman_and_testpy_template stanza.
> > >
> > > I'm not sure what you are looking for there. Do you want the commit
> > > message to mention which part of the gitlab file is being changed?
> >
> > I mean a lot of places say something like "Change gitlab ..." but aren't
> > changing any of the world builds, only test.py stuff, so I would prefer
> > "Change gitlab when using test.py ..."
> 
> OK I will try. I've got the changes to all three environments in a
> single commit at present, so we don't end up with double the commis.
> Or would you prefer it split out?

Doing all CIs in step makes sense.  So something like...:
Azure/GitLab/Travis: Change test.py and buildman ...

> One more thing...I notice that gitlab and azure use 'set -ex' to avoid
> needing to check errors in the script. Is is possible for travis to do
> that do, or is there some restriction?

Not sure.  When in doubt I always hack the heck out of .travis.yml to
just be a testcase of what I want to figure out, push and see.  Thanks!