diff mbox series

[v5,1/2] cirrus: Fixing and speedup the msys2/mingw CI

Message ID 20201007145300.1197-2-luoyonggang@gmail.com
State New
Headers show
Series Improve cirrus msys2 | expand

Commit Message

罗勇刚(Yonggang Luo) Oct. 7, 2020, 2:52 p.m. UTC
Use cache of cirrus caching msys2
The install of msys2 are refer to https://github.com/msys2/setup-msys2
The first time install msys2 would be time consuming, so increase timeout_in to 90m
according to https://cirrus-ci.org/faq/#instance-timed-out

Apply patch of https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg00072.html

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
---
 .cirrus.yml | 110 +++++++++++++++++++++++++++++++---------------------
 1 file changed, 66 insertions(+), 44 deletions(-)

Comments

Thomas Huth Oct. 7, 2020, 3:12 p.m. UTC | #1
On 07/10/2020 16.52, Yonggang Luo wrote:
> Use cache of cirrus caching msys2

> The install of msys2 are refer to https://github.com/msys2/setup-msys2

> The first time install msys2 would be time consuming, so increase timeout_in to 90m

> according to https://cirrus-ci.org/faq/#instance-timed-out

> 

> Apply patch of https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg00072.html

> 

> Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>

> ---

>  .cirrus.yml | 110 +++++++++++++++++++++++++++++++---------------------

>  1 file changed, 66 insertions(+), 44 deletions(-)

> 

> diff --git a/.cirrus.yml b/.cirrus.yml

> index d58782ce67..a1145d0366 100644

> --- a/.cirrus.yml

> +++ b/.cirrus.yml

> @@ -46,64 +46,86 @@ macos_xcode_task:

>      - gmake check

>  

>  windows_msys2_task:

> +  timeout_in: 90m

>    windows_container:

> -    image: cirrusci/windowsservercore:cmake

> +    image: cirrusci/windowsservercore:2019

>      os_version: 2019

>      cpu: 8

>      memory: 8G

>    env:

> +    CIRRUS_SHELL: powershell

>      MSYS: winsymlinks:nativestrict

>      MSYSTEM: MINGW64

>      CHERE_INVOKING: 1

>    printenv_script:

> -    - C:\tools\msys64\usr\bin\bash.exe -lc 'printenv'

> +    - choco install -y --no-progress 7zip


Since you've now removed the 'printenv' line, I think the section should
maybe not be called "printenv_script" anymore, should it?

 Thomas
罗勇刚(Yonggang Luo) Oct. 7, 2020, 3:54 p.m. UTC | #2
On Wed, Oct 7, 2020 at 11:12 PM Thomas Huth <thuth@redhat.com> wrote:
>

> On 07/10/2020 16.52, Yonggang Luo wrote:

> > Use cache of cirrus caching msys2

> > The install of msys2 are refer to https://github.com/msys2/setup-msys2

> > The first time install msys2 would be time consuming, so increase

timeout_in to 90m
> > according to https://cirrus-ci.org/faq/#instance-timed-out

> >

> > Apply patch of

https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg00072.html
> >

> > Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>

> > ---

> >  .cirrus.yml | 110 +++++++++++++++++++++++++++++++---------------------

> >  1 file changed, 66 insertions(+), 44 deletions(-)

> >

> > diff --git a/.cirrus.yml b/.cirrus.yml

> > index d58782ce67..a1145d0366 100644

> > --- a/.cirrus.yml

> > +++ b/.cirrus.yml

> > @@ -46,64 +46,86 @@ macos_xcode_task:

> >      - gmake check

> >

> >  windows_msys2_task:

> > +  timeout_in: 90m

> >    windows_container:

> > -    image: cirrusci/windowsservercore:cmake

> > +    image: cirrusci/windowsservercore:2019

> >      os_version: 2019

> >      cpu: 8

> >      memory: 8G

> >    env:

> > +    CIRRUS_SHELL: powershell

> >      MSYS: winsymlinks:nativestrict

> >      MSYSTEM: MINGW64

> >      CHERE_INVOKING: 1

> >    printenv_script:

> > -    - C:\tools\msys64\usr\bin\bash.exe -lc 'printenv'

> > +    - choco install -y --no-progress 7zip

>

> Since you've now removed the 'printenv' line, I think the section should

> maybe not be called "printenv_script" anymore, should it?

May I name it install_7zip?
Cause that's a deps need installed first.
>

>  Thomas

>



--
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo
<div dir="ltr"><br><br>On Wed, Oct 7, 2020 at 11:12 PM Thomas Huth &lt;<a href="mailto:thuth@redhat.com">thuth@redhat.com</a>&gt; wrote:<br>&gt;<br>&gt; On 07/10/2020 16.52, Yonggang Luo wrote:<br>&gt; &gt; Use cache of cirrus caching msys2<br>&gt; &gt; The install of msys2 are refer to <a href="https://github.com/msys2/setup-msys2">https://github.com/msys2/setup-msys2</a><br>&gt; &gt; The first time install msys2 would be time consuming, so increase timeout_in to 90m<br>&gt; &gt; according to <a href="https://cirrus-ci.org/faq/#instance-timed-out">https://cirrus-ci.org/faq/#instance-timed-out</a><br>&gt; &gt;<br>&gt; &gt; Apply patch of <a href="https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg00072.html">https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg00072.html</a><br>&gt; &gt;<br>&gt; &gt; Signed-off-by: Yonggang Luo &lt;<a href="mailto:luoyonggang@gmail.com">luoyonggang@gmail.com</a>&gt;<br>&gt; &gt; ---<br>&gt; &gt;  .cirrus.yml | 110 +++++++++++++++++++++++++++++++---------------------<br>&gt; &gt;  1 file changed, 66 insertions(+), 44 deletions(-)<br>&gt; &gt;<br>&gt; &gt; diff --git a/.cirrus.yml b/.cirrus.yml<br>&gt; &gt; index d58782ce67..a1145d0366 100644<br>&gt; &gt; --- a/.cirrus.yml<br>&gt; &gt; +++ b/.cirrus.yml<br>&gt; &gt; @@ -46,64 +46,86 @@ macos_xcode_task:<br>&gt; &gt;      - gmake check<br>&gt; &gt; <br>&gt; &gt;  windows_msys2_task:<br>&gt; &gt; +  timeout_in: 90m<br>&gt; &gt;    windows_container:<br>&gt; &gt; -    image: cirrusci/windowsservercore:cmake<br>&gt; &gt; +    image: cirrusci/windowsservercore:2019<br>&gt; &gt;      os_version: 2019<br>&gt; &gt;      cpu: 8<br>&gt; &gt;      memory: 8G<br>&gt; &gt;    env:<br>&gt; &gt; +    CIRRUS_SHELL: powershell<br>&gt; &gt;      MSYS: winsymlinks:nativestrict<br>&gt; &gt;      MSYSTEM: MINGW64<br>&gt; &gt;      CHERE_INVOKING: 1<br>&gt; &gt;    printenv_script:<br>&gt; &gt; -    - C:\tools\msys64\usr\bin\bash.exe -lc &#39;printenv&#39;<br>&gt; &gt; +    - choco install -y --no-progress 7zip<br>&gt;<br>&gt; Since you&#39;ve now removed the &#39;printenv&#39; line, I think the section should<br>&gt; maybe not be called &quot;printenv_script&quot; anymore, should it?<div>May I name it install_7zip?</div><div>Cause that&#39;s a deps need installed first.<br>&gt;<br>&gt;  Thomas<br>&gt;<br><br><br>--<br>         此致<br>礼<br>罗勇刚<br>Yours<br>    sincerely,<br>Yonggang Luo</div></div>
Alex Bennée Oct. 7, 2020, 6:53 p.m. UTC | #3
罗勇刚(Yonggang Luo) <luoyonggang@gmail.com> writes:

> On Wed, Oct 7, 2020 at 11:12 PM Thomas Huth <thuth@redhat.com> wrote:

>>

>> On 07/10/2020 16.52, Yonggang Luo wrote:

>> > Use cache of cirrus caching msys2

>> > The install of msys2 are refer to https://github.com/msys2/setup-msys2

>> > The first time install msys2 would be time consuming, so increase

> timeout_in to 90m

>> > according to https://cirrus-ci.org/faq/#instance-timed-out

>> >

>> > Apply patch of

> https://lists.gnu.org/archive/html/qemu-devel/2020-10/msg00072.html

>> >

>> > Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>

>> > ---

>> >  .cirrus.yml | 110 +++++++++++++++++++++++++++++++---------------------

>> >  1 file changed, 66 insertions(+), 44 deletions(-)

>> >

>> > diff --git a/.cirrus.yml b/.cirrus.yml

>> > index d58782ce67..a1145d0366 100644

>> > --- a/.cirrus.yml

>> > +++ b/.cirrus.yml

>> > @@ -46,64 +46,86 @@ macos_xcode_task:

>> >      - gmake check

>> >

>> >  windows_msys2_task:

>> > +  timeout_in: 90m

>> >    windows_container:

>> > -    image: cirrusci/windowsservercore:cmake

>> > +    image: cirrusci/windowsservercore:2019

>> >      os_version: 2019

>> >      cpu: 8

>> >      memory: 8G

>> >    env:

>> > +    CIRRUS_SHELL: powershell

>> >      MSYS: winsymlinks:nativestrict

>> >      MSYSTEM: MINGW64

>> >      CHERE_INVOKING: 1

>> >    printenv_script:

>> > -    - C:\tools\msys64\usr\bin\bash.exe -lc 'printenv'

>> > +    - choco install -y --no-progress 7zip

>>

>> Since you've now removed the 'printenv' line, I think the section should

>> maybe not be called "printenv_script" anymore, should it?

> May I name it install_7zip?

> Cause that's a deps need installed first.


I renamed it setup_script: when I merged it.

>>

>>  Thomas

>>



-- 
Alex Bennée
diff mbox series

Patch

diff --git a/.cirrus.yml b/.cirrus.yml
index d58782ce67..a1145d0366 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -46,64 +46,86 @@  macos_xcode_task:
     - gmake check
 
 windows_msys2_task:
+  timeout_in: 90m
   windows_container:
-    image: cirrusci/windowsservercore:cmake
+    image: cirrusci/windowsservercore:2019
     os_version: 2019
     cpu: 8
     memory: 8G
   env:
+    CIRRUS_SHELL: powershell
     MSYS: winsymlinks:nativestrict
     MSYSTEM: MINGW64
     CHERE_INVOKING: 1
   printenv_script:
-    - C:\tools\msys64\usr\bin\bash.exe -lc 'printenv'
+    - choco install -y --no-progress 7zip
+    - Write-Output $env:PATH
+  msys2_cache:
+    folder: C:\tools\archive
+    reupload_on_changes: false
+    fingerprint_script: cat .cirrus.yml
+    populate_script:
+      - |
+        md C:\tools
+        md C:\tools\archive
+        $start_time = Get-Date
+        cd C:\tools
+        bitsadmin /transfer msys_download /dynamic /download /priority FOREGROUND https://github.com/msys2/msys2-installer/releases/download/2020-09-03/msys2-base-x86_64-20200903.sfx.exe C:\tools\base.exe
+        Write-Output "Download time taken: $((Get-Date).Subtract($start_time).Seconds) second(s)"
+        C:\tools\base.exe -y
+        ((Get-Content -path C:\tools\msys64\etc\\post-install\\07-pacman-key.post -Raw) -replace '--refresh-keys', '--version') | Set-Content -Path C:\tools\msys64\etc\\post-install\\07-pacman-key.post
+        C:\tools\msys64\usr\bin\bash.exe -lc "sed -i 's/^CheckSpace/#CheckSpace/g' /etc/pacman.conf"
+        C:\tools\msys64\usr\bin\bash.exe -lc "export"
+        C:\tools\msys64\usr\bin\bash.exe -lc "grep -rl 'repo.msys2.org/' /etc/pacman.d/mirrorlist.* | xargs sed -i 's/repo.msys2.org\//mirrors.tuna.tsinghua.edu.cn\/msys2\//g'"
+        C:\tools\msys64\usr\bin\pacman.exe --noconfirm -Sy
+        echo Y | C:\tools\msys64\usr\bin\pacman.exe --noconfirm -Suu --overwrite=*
+        taskkill /F /FI "MODULES eq msys-2.0.dll"
+        tasklist
+        C:\tools\msys64\usr\bin\bash.exe -lc "mv -f /etc/pacman.conf.pacnew /etc/pacman.conf || true"
+        C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -Suu --overwrite=*"
+        C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -S --needed \
+          diffutils git grep make pkg-config sed \
+          mingw-w64-x86_64-python \
+          mingw-w64-x86_64-python-setuptools \
+          mingw-w64-x86_64-toolchain \
+          mingw-w64-x86_64-SDL2 \
+          mingw-w64-x86_64-SDL2_image \
+          mingw-w64-x86_64-gtk3 \
+          mingw-w64-x86_64-glib2 \
+          mingw-w64-x86_64-ninja \
+          mingw-w64-x86_64-jemalloc \
+          mingw-w64-x86_64-lzo2 \
+          mingw-w64-x86_64-zstd \
+          mingw-w64-x86_64-libjpeg-turbo \
+          mingw-w64-x86_64-pixman \
+          mingw-w64-x86_64-libgcrypt \
+          mingw-w64-x86_64-libpng \
+          mingw-w64-x86_64-libssh \
+          mingw-w64-x86_64-libxml2 \
+          mingw-w64-x86_64-snappy \
+          mingw-w64-x86_64-libusb \
+          mingw-w64-x86_64-usbredir \
+          mingw-w64-x86_64-libtasn1 \
+          mingw-w64-x86_64-nettle \
+          mingw-w64-x86_64-cyrus-sasl \
+          mingw-w64-x86_64-curl \
+          mingw-w64-x86_64-gnutls \
+          "
+        C:\tools\msys64\usr\bin\bash.exe -lc "rm -rf /var/cache/pacman/pkg/*"
+        cd C:\tools\msys64
+        echo "Start archive"
+        cmd /C "7z a -ttar . -so | 7z a -txz -simsys2-x86_64.tar C:\tools\archive\msys2-x86_64.tar.xz"
   install_script:
-    - C:\tools\msys64\usr\bin\bash.exe -lc "cd /c/tools &&
-        curl -O http://repo.msys2.org/msys/x86_64/msys2-keyring-r21.b39fb11-1-any.pkg.tar.xz"
-    - C:\tools\msys64\usr\bin\bash.exe -lc "cd /c/tools &&
-        curl -O http://repo.msys2.org/msys/x86_64/msys2-keyring-r21.b39fb11-1-any.pkg.tar.xz.sig"
-    - C:\tools\msys64\usr\bin\bash.exe -lc "cd /c/tools &&
-        pacman -U --noconfirm msys2-keyring-r21.b39fb11-1-any.pkg.tar.xz"
-    - C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -Sy"
-    - C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -S --needed
-        bash pacman pacman-mirrors msys2-runtime"
-    - taskkill /F /IM gpg-agent.exe
-    - C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -Su"
-    - C:\tools\msys64\usr\bin\bash.exe -lc "pacman --noconfirm -S --needed
-        base-devel
-        git
-        mingw-w64-x86_64-python
-        mingw-w64-x86_64-python-setuptools
-        mingw-w64-x86_64-toolchain
-        mingw-w64-x86_64-capstone
-        mingw-w64-x86_64-SDL2
-        mingw-w64-x86_64-SDL2_image
-        mingw-w64-x86_64-gtk3
-        mingw-w64-x86_64-glib2
-        mingw-w64-x86_64-ninja
-        mingw-w64-x86_64-make
-        mingw-w64-x86_64-lzo2
-        mingw-w64-x86_64-zstd
-        mingw-w64-x86_64-libjpeg-turbo
-        mingw-w64-x86_64-pixman
-        mingw-w64-x86_64-libgcrypt
-        mingw-w64-x86_64-libpng
-        mingw-w64-x86_64-libssh
-        mingw-w64-x86_64-libxml2
-        mingw-w64-x86_64-snappy
-        mingw-w64-x86_64-libusb
-        mingw-w64-x86_64-usbredir
-        mingw-w64-x86_64-libtasn1
-        mingw-w64-x86_64-nettle
-        mingw-w64-x86_64-cyrus-sasl
-        mingw-w64-x86_64-curl
-        mingw-w64-x86_64-gnutls
-        mingw-w64-x86_64-zstd"
+    - |
+      cd C:\tools
+      cmd /C "7z x C:\tools\archive\msys2-x86_64.tar.xz -so | 7z x -aoa -simsys2-x86_64.tar -ttar -omsys64"
+      C:\tools\msys64\usr\bin\bash.exe -lc "export"
+
   script:
     - C:\tools\msys64\usr\bin\bash.exe -lc "mkdir build"
     - C:\tools\msys64\usr\bin\bash.exe -lc "cd build && ../configure
       --python=python3 --ninja=ninja
       --target-list-exclude=i386-softmmu,arm-softmmu,ppc-softmmu,mips-softmmu"
-    - C:\tools\msys64\usr\bin\bash.exe -lc "cd build && make -j$NUMBER_OF_PROCESSORS"
+    - C:\tools\msys64\usr\bin\bash.exe -lc "cd build && make -j8"
   test_script:
     - C:\tools\msys64\usr\bin\bash.exe -lc "cd build && make V=1 check"