diff mbox series

[v3,29/30] gitlab: limit re-builds of the containers

Message ID 20200626181357.26211-30-alex.bennee@linaro.org
State New
Headers show
Series testing/next (gitlab, vm, docker) | expand

Commit Message

Alex Bennée June 26, 2020, 6:13 p.m. UTC
Most of the time we are just rebuilding the same things. We can skip
this although currently there is no mechanism for picking up new
distro releases.

Rather than try to be too fine grained allow any change to trigger all
the images being rebuilt.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
 .gitlab-ci.d/containers.yml | 4 ++++
 1 file changed, 4 insertions(+)

-- 
2.20.1

Comments

Philippe Mathieu-Daudé June 29, 2020, 10:01 a.m. UTC | #1
On 6/26/20 8:13 PM, Alex Bennée wrote:
> Most of the time we are just rebuilding the same things. We can skip

> this although currently there is no mechanism for picking up new

> distro releases.

> 

> Rather than try to be too fine grained allow any change to trigger all

> the images being rebuilt.

> 

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>


Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>


> ---

>  .gitlab-ci.d/containers.yml | 4 ++++

>  1 file changed, 4 insertions(+)

> 

> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml

> index 20b2fb1de5d..f56aa44f711 100644

> --- a/.gitlab-ci.d/containers.yml

> +++ b/.gitlab-ci.d/containers.yml

> @@ -19,6 +19,10 @@

>      - docker push "$TAG"

>    after_script:

>      - docker logout

> +  only:

> +    changes:

> +      - .gitlab-ci.d/containers.yml

> +      - tests/docker/*

>  

>  amd64-centos7-container:

>    <<: *container_job_definition

>
Daniel P. Berrangé June 29, 2020, 10:03 a.m. UTC | #2
On Fri, Jun 26, 2020 at 07:13:56PM +0100, Alex Bennée wrote:
> Most of the time we are just rebuilding the same things. We can skip

> this although currently there is no mechanism for picking up new

> distro releases.

> 

> Rather than try to be too fine grained allow any change to trigger all

> the images being rebuilt.

> 

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

> ---

>  .gitlab-ci.d/containers.yml | 4 ++++

>  1 file changed, 4 insertions(+)

> 

> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml

> index 20b2fb1de5d..f56aa44f711 100644

> --- a/.gitlab-ci.d/containers.yml

> +++ b/.gitlab-ci.d/containers.yml

> @@ -19,6 +19,10 @@

>      - docker push "$TAG"

>    after_script:

>      - docker logout

> +  only:

> +    changes:

> +      - .gitlab-ci.d/containers.yml

> +      - tests/docker/*


How does this work for a person who forks the QEMU git repo and pushes
a change which doesn't touch the containers.yml file ?  AFAICT, all
their jobs will fail due to not having previously built any container
images in their brand new fork.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Alex Bennée June 29, 2020, 10:11 a.m. UTC | #3
Daniel P. Berrangé <berrange@redhat.com> writes:

> On Fri, Jun 26, 2020 at 07:13:56PM +0100, Alex Bennée wrote:

>> Most of the time we are just rebuilding the same things. We can skip

>> this although currently there is no mechanism for picking up new

>> distro releases.

>> 

>> Rather than try to be too fine grained allow any change to trigger all

>> the images being rebuilt.

>> 

>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

>> ---

>>  .gitlab-ci.d/containers.yml | 4 ++++

>>  1 file changed, 4 insertions(+)

>> 

>> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml

>> index 20b2fb1de5d..f56aa44f711 100644

>> --- a/.gitlab-ci.d/containers.yml

>> +++ b/.gitlab-ci.d/containers.yml

>> @@ -19,6 +19,10 @@

>>      - docker push "$TAG"

>>    after_script:

>>      - docker logout

>> +  only:

>> +    changes:

>> +      - .gitlab-ci.d/containers.yml

>> +      - tests/docker/*

>

> How does this work for a person who forks the QEMU git repo and pushes

> a change which doesn't touch the containers.yml file ?  AFAICT, all

> their jobs will fail due to not having previously built any container

> images in their brand new fork.


Hmm what we really need is a condition check to see if there is a local
registry with images in it.

>

> Regards,

> Daniel



-- 
Alex Bennée
Thomas Huth June 29, 2020, 11:42 a.m. UTC | #4
On 29/06/2020 12.11, Alex Bennée wrote:
> 

> Daniel P. Berrangé <berrange@redhat.com> writes:

> 

>> On Fri, Jun 26, 2020 at 07:13:56PM +0100, Alex Bennée wrote:

>>> Most of the time we are just rebuilding the same things. We can skip

>>> this although currently there is no mechanism for picking up new

>>> distro releases.

>>>

>>> Rather than try to be too fine grained allow any change to trigger all

>>> the images being rebuilt.

>>>

>>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

>>> ---

>>>   .gitlab-ci.d/containers.yml | 4 ++++

>>>   1 file changed, 4 insertions(+)

>>>

>>> diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml

>>> index 20b2fb1de5d..f56aa44f711 100644

>>> --- a/.gitlab-ci.d/containers.yml

>>> +++ b/.gitlab-ci.d/containers.yml

>>> @@ -19,6 +19,10 @@

>>>       - docker push "$TAG"

>>>     after_script:

>>>       - docker logout

>>> +  only:

>>> +    changes:

>>> +      - .gitlab-ci.d/containers.yml

>>> +      - tests/docker/*

>>

>> How does this work for a person who forks the QEMU git repo and pushes

>> a change which doesn't touch the containers.yml file ?  AFAICT, all

>> their jobs will fail due to not having previously built any container

>> images in their brand new fork.

> 

> Hmm what we really need is a condition check to see if there is a local

> registry with images in it.


Maybe it would be good enough to do something like this instead:

   rules:
     - if: '$CI_COMMIT_BRANCH == "master"'
     - changes:
       - .gitlab-ci.d/containers.yml
       - tests/docker/*

... so that the pipeline is certainly run the first time when the master 
branch gets initialized? (I did not try it out, just had the idea after 
looking at the gitlab CI docs)

  Thomas
Daniel P. Berrangé June 29, 2020, 11:52 a.m. UTC | #5
On Mon, Jun 29, 2020 at 01:42:19PM +0200, Thomas Huth wrote:
> On 29/06/2020 12.11, Alex Bennée wrote:

> > 

> > Daniel P. Berrangé <berrange@redhat.com> writes:

> > 

> > > On Fri, Jun 26, 2020 at 07:13:56PM +0100, Alex Bennée wrote:

> > > > Most of the time we are just rebuilding the same things. We can skip

> > > > this although currently there is no mechanism for picking up new

> > > > distro releases.

> > > > 

> > > > Rather than try to be too fine grained allow any change to trigger all

> > > > the images being rebuilt.

> > > > 

> > > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

> > > > ---

> > > >   .gitlab-ci.d/containers.yml | 4 ++++

> > > >   1 file changed, 4 insertions(+)

> > > > 

> > > > diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml

> > > > index 20b2fb1de5d..f56aa44f711 100644

> > > > --- a/.gitlab-ci.d/containers.yml

> > > > +++ b/.gitlab-ci.d/containers.yml

> > > > @@ -19,6 +19,10 @@

> > > >       - docker push "$TAG"

> > > >     after_script:

> > > >       - docker logout

> > > > +  only:

> > > > +    changes:

> > > > +      - .gitlab-ci.d/containers.yml

> > > > +      - tests/docker/*

> > > 

> > > How does this work for a person who forks the QEMU git repo and pushes

> > > a change which doesn't touch the containers.yml file ?  AFAICT, all

> > > their jobs will fail due to not having previously built any container

> > > images in their brand new fork.

> > 

> > Hmm what we really need is a condition check to see if there is a local

> > registry with images in it.

> 

> Maybe it would be good enough to do something like this instead:

> 

>   rules:

>     - if: '$CI_COMMIT_BRANCH == "master"'

>     - changes:

>       - .gitlab-ci.d/containers.yml

>       - tests/docker/*

> 

> ... so that the pipeline is certainly run the first time when the master

> branch gets initialized? (I did not try it out, just had the idea after

> looking at the gitlab CI docs)


IIUC, the rules: can only reduce scenarios where a job is run, vs the
baseline config. AFAIK, forking a repo doesn't cause any jobs to be
run, so adding extra rules: can't make that happen.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
diff mbox series

Patch

diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
index 20b2fb1de5d..f56aa44f711 100644
--- a/.gitlab-ci.d/containers.yml
+++ b/.gitlab-ci.d/containers.yml
@@ -19,6 +19,10 @@ 
     - docker push "$TAG"
   after_script:
     - docker logout
+  only:
+    changes:
+      - .gitlab-ci.d/containers.yml
+      - tests/docker/*
 
 amd64-centos7-container:
   <<: *container_job_definition