| Message ID | 20200528101039.24600-1-thuth@redhat.com |
|---|---|
| Headers | show |
| Series | Gitlab CI fixes and improvements | expand |
Hi Thomas, Alex, +Daniel On 5/28/20 12:10 PM, Thomas Huth wrote: > We have a dedicated folder for the gitlab-ci - so there is no need > to clutter the top directory with these .yml files. > > Message-Id: <20200525131823.715-5-thuth@redhat.com> > Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> > Reviewed-by: Alex Bennée <alex.bennee@linaro.org> > Signed-off-by: Thomas Huth <thuth@redhat.com> > --- > .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml | 0 > .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml | 0 > .gitlab-ci.yml | 4 ++-- > MAINTAINERS | 2 +- > 4 files changed, 3 insertions(+), 3 deletions(-) > rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%) > rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%) > > diff --git a/.gitlab-ci-edk2.yml b/.gitlab-ci.d/edk2.yml > similarity index 100% > rename from .gitlab-ci-edk2.yml > rename to .gitlab-ci.d/edk2.yml > diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci.d/opensbi.yml > similarity index 100% > rename from .gitlab-ci-opensbi.yml > rename to .gitlab-ci.d/opensbi.yml > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > index bc6aee6aba..5208d93ff8 100644 > --- a/.gitlab-ci.yml > +++ b/.gitlab-ci.yml > @@ -1,6 +1,6 @@ > include: > - - local: '/.gitlab-ci-edk2.yml' > - - local: '/.gitlab-ci-opensbi.yml' > + - local: '/.gitlab-ci.d/edk2.yml' > + - local: '/.gitlab-ci.d/opensbi.yml' It seems these jobs are now always run, the "rules:changes" to restrict them is ignored. I searched a bit and found a plausible explanation here: https://docs.gitlab.com/ee/ci/yaml/#onlychangesexceptchanges "Caution: In pipelines with sources other than the three above changes can’t determine if a given file is new or old and always returns true. This includes pipelines triggered by pushing new tags. Configuring jobs to use only: changes with other only: refs keywords is possible, but not recommended." I don't understand what they mean by "three", is a directory considered a three? Then that would explain the change. Regards, Phil.
On Mon, Oct 12, 2020 at 03:44:00PM +0200, Philippe Mathieu-Daudé wrote: > Hi Thomas, Alex, > > +Daniel > > On 5/28/20 12:10 PM, Thomas Huth wrote: > > We have a dedicated folder for the gitlab-ci - so there is no need > > to clutter the top directory with these .yml files. > > > > Message-Id: <20200525131823.715-5-thuth@redhat.com> > > Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> > > Reviewed-by: Alex Bennée <alex.bennee@linaro.org> > > Signed-off-by: Thomas Huth <thuth@redhat.com> > > --- > > .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml | 0 > > .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml | 0 > > .gitlab-ci.yml | 4 ++-- > > MAINTAINERS | 2 +- > > 4 files changed, 3 insertions(+), 3 deletions(-) > > rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%) > > rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%) > > > > diff --git a/.gitlab-ci-edk2.yml b/.gitlab-ci.d/edk2.yml > > similarity index 100% > > rename from .gitlab-ci-edk2.yml > > rename to .gitlab-ci.d/edk2.yml > > diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci.d/opensbi.yml > > similarity index 100% > > rename from .gitlab-ci-opensbi.yml > > rename to .gitlab-ci.d/opensbi.yml > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > > index bc6aee6aba..5208d93ff8 100644 > > --- a/.gitlab-ci.yml > > +++ b/.gitlab-ci.yml > > @@ -1,6 +1,6 @@ > > include: > > - - local: '/.gitlab-ci-edk2.yml' > > - - local: '/.gitlab-ci-opensbi.yml' > > + - local: '/.gitlab-ci.d/edk2.yml' > > + - local: '/.gitlab-ci.d/opensbi.yml' > > It seems these jobs are now always run, the "rules:changes" > to restrict them is ignored. > > I searched a bit and found a plausible explanation here: > https://docs.gitlab.com/ee/ci/yaml/#onlychangesexceptchanges > > "Caution: In pipelines with sources other than the three above changes can’t > determine if a given file is new or old and always returns true. This > includes pipelines triggered by pushing new tags. Configuring jobs to use > only: changes with other only: refs keywords is possible, but not > recommended." > > I don't understand what they mean by "three", is a directory considered > a three? Then that would explain the change. The edk2.yml file as a "when: always" clause attached to every rule, so surely this negates the point of filtering by making it always run ? Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
On 10/12/20 4:01 PM, Daniel P. Berrangé wrote: > On Mon, Oct 12, 2020 at 03:44:00PM +0200, Philippe Mathieu-Daudé wrote: >> Hi Thomas, Alex, >> >> +Daniel >> >> On 5/28/20 12:10 PM, Thomas Huth wrote: >>> We have a dedicated folder for the gitlab-ci - so there is no need >>> to clutter the top directory with these .yml files. >>> >>> Message-Id: <20200525131823.715-5-thuth@redhat.com> >>> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> >>> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> >>> Signed-off-by: Thomas Huth <thuth@redhat.com> >>> --- >>> .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml | 0 >>> .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml | 0 >>> .gitlab-ci.yml | 4 ++-- >>> MAINTAINERS | 2 +- >>> 4 files changed, 3 insertions(+), 3 deletions(-) >>> rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%) >>> rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%) >>> >>> diff --git a/.gitlab-ci-edk2.yml b/.gitlab-ci.d/edk2.yml >>> similarity index 100% >>> rename from .gitlab-ci-edk2.yml >>> rename to .gitlab-ci.d/edk2.yml >>> diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci.d/opensbi.yml >>> similarity index 100% >>> rename from .gitlab-ci-opensbi.yml >>> rename to .gitlab-ci.d/opensbi.yml >>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >>> index bc6aee6aba..5208d93ff8 100644 >>> --- a/.gitlab-ci.yml >>> +++ b/.gitlab-ci.yml >>> @@ -1,6 +1,6 @@ >>> include: >>> - - local: '/.gitlab-ci-edk2.yml' >>> - - local: '/.gitlab-ci-opensbi.yml' >>> + - local: '/.gitlab-ci.d/edk2.yml' >>> + - local: '/.gitlab-ci.d/opensbi.yml' >> >> It seems these jobs are now always run, the "rules:changes" >> to restrict them is ignored. >> >> I searched a bit and found a plausible explanation here: >> https://docs.gitlab.com/ee/ci/yaml/#onlychangesexceptchanges >> >> "Caution: In pipelines with sources other than the three above changes can’t >> determine if a given file is new or old and always returns true. This >> includes pipelines triggered by pushing new tags. Configuring jobs to use >> only: changes with other only: refs keywords is possible, but not >> recommended." >> >> I don't understand what they mean by "three", is a directory considered >> a three? Then that would explain the change. > > The edk2.yml file as a "when: always" clause attached to every rule, > so surely this negates the point of filtering by making it always > run ? I understand it runs regardless a job in any previous stage failed. From https://docs.gitlab.com/ee/ci/yaml/#when * always - execute job regardless of the status of jobs from prior stages. * on_success - execute job only when all jobs from prior stages succeed (or are considered succeeding because they have allow_failure: true). This is the default. I think I used that because at the time I sent that patch some docker image builds were failing. But we don't really need it, I'll remove. Anyhow this used to work with "when: always", see: https://gitlab.com/qemu-project/qemu/-/pipelines/168158357 > > Regards, > Daniel >
On 10/13/20 4:18 PM, Philippe Mathieu-Daudé wrote: > On 10/12/20 4:01 PM, Daniel P. Berrangé wrote: >> On Mon, Oct 12, 2020 at 03:44:00PM +0200, Philippe Mathieu-Daudé wrote: >>> Hi Thomas, Alex, >>> >>> +Daniel >>> >>> On 5/28/20 12:10 PM, Thomas Huth wrote: >>>> We have a dedicated folder for the gitlab-ci - so there is no need >>>> to clutter the top directory with these .yml files. >>>> >>>> Message-Id: <20200525131823.715-5-thuth@redhat.com> >>>> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> >>>> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> >>>> Signed-off-by: Thomas Huth <thuth@redhat.com> >>>> --- >>>> .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml | 0 >>>> .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml | 0 >>>> .gitlab-ci.yml | 4 ++-- >>>> MAINTAINERS | 2 +- >>>> 4 files changed, 3 insertions(+), 3 deletions(-) >>>> rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%) >>>> rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%) >>>> >>>> diff --git a/.gitlab-ci-edk2.yml b/.gitlab-ci.d/edk2.yml >>>> similarity index 100% >>>> rename from .gitlab-ci-edk2.yml >>>> rename to .gitlab-ci.d/edk2.yml >>>> diff --git a/.gitlab-ci-opensbi.yml b/.gitlab-ci.d/opensbi.yml >>>> similarity index 100% >>>> rename from .gitlab-ci-opensbi.yml >>>> rename to .gitlab-ci.d/opensbi.yml >>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml >>>> index bc6aee6aba..5208d93ff8 100644 >>>> --- a/.gitlab-ci.yml >>>> +++ b/.gitlab-ci.yml >>>> @@ -1,6 +1,6 @@ >>>> include: >>>> - - local: '/.gitlab-ci-edk2.yml' >>>> - - local: '/.gitlab-ci-opensbi.yml' >>>> + - local: '/.gitlab-ci.d/edk2.yml' >>>> + - local: '/.gitlab-ci.d/opensbi.yml' >>> >>> It seems these jobs are now always run, the "rules:changes" >>> to restrict them is ignored. >>> >>> I searched a bit and found a plausible explanation here: >>> https://docs.gitlab.com/ee/ci/yaml/#onlychangesexceptchanges >>> >>> "Caution: In pipelines with sources other than the three above >>> changes can’t >>> determine if a given file is new or old and always returns true. This >>> includes pipelines triggered by pushing new tags. Configuring jobs to >>> use >>> only: changes with other only: refs keywords is possible, but not >>> recommended." >>> >>> I don't understand what they mean by "three", is a directory considered >>> a three? Then that would explain the change. >> >> The edk2.yml file as a "when: always" clause attached to every rule, >> so surely this negates the point of filtering by making it always >> run ? git show 71920809ceab +docker-edk2: + stage: build + rules: # Only run this job when the Dockerfile is modified + - changes: + - .gitlab-ci-edk2.yml + - .gitlab-ci.d/edk2/Dockerfile + when: always Per https://docs.gitlab.com/ee/ci/yaml/#rules-clauses Rules are evaluated in order until a match is found. The job is not added to the pipeline: If no rules match, and there is no standalone when: on_success, when: delayed or when: always. For docker-edk2 the rule is "If .gitlab-ci-edk2.yml or .gitlab-ci.d/edk2/Dockerfile changed, then always add; else do not add". > I understand it runs regardless a job in any previous stage failed. > > From https://docs.gitlab.com/ee/ci/yaml/#when > > * always - execute job regardless of the status of jobs from prior stages. > * on_success - execute job only when all jobs from prior stages succeed > (or are considered succeeding because they have allow_failure: true). > This is the default. > > I think I used that because at the time I sent that patch > some docker image builds were failing. But we don't really > need it, I'll remove. Also because this is a long job burning free tier credits pointlessly. (I'll also do it for the OpenSBI job). > > Anyhow this used to work with "when: always", see: > https://gitlab.com/qemu-project/qemu/-/pipelines/168158357 > >> >> Regards, >> Daniel >> >
Hi Peter, the following changes since commit 06539ebc76b8625587aa78d646a9d8d5fddf84f3: Merge remote-tracking branch 'remotes/philmd-gitlab/tags/mips-hw-next-20200526' into staging (2020-05-26 20:25:06 +0100) are available in the Git repository at: https://gitlab.com/huth/qemu.git tags/pull-request-2020-05-28 for you to fetch changes up to 0016afa250d251a1b3fbc90c6d618e7f00e02f14: gitlab-ci: Determine the number of jobs dynamically (2020-05-28 11:01:38 +0200) ---------------------------------------------------------------- - Build with other images instead of the broken Debian containers - Fix building with the latest version of Clang (at least wrt. to the gitlab-CI pipeline) - Add Philippe, Alex and Wainer to the Gitlab-CI section in MAINTAINERS ---------------------------------------------------------------- Alex Bennée (1): linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS Cleber Rosa (1): GitLab CI: avoid calling before_scripts on unintended jobs Philippe Mathieu-Daudé (1): gitlab-ci: Remove flex/bison packages Thomas Huth (4): MAINTAINERS: Add Philippe, Alex and Wainer to the Gitlab-CI section gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder gitlab-ci: Do not use the standard container images from gitlab gitlab-ci: Determine the number of jobs dynamically .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml | 0 .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml | 0 .gitlab-ci.yml | 67 +++++++++++++++------- MAINTAINERS | 5 +- linux-user/elfload.c | 2 + 5 files changed, 51 insertions(+), 23 deletions(-) rename .gitlab-ci-edk2.yml => .gitlab-ci.d/edk2.yml (100%) rename .gitlab-ci-opensbi.yml => .gitlab-ci.d/opensbi.yml (100%)