[PULL,v2,04/15] gitlab-ci.yml: Add jobs to test CFI flags

Message ID 20210310160002.11659-5-alex.bennee@linaro.org
State New
Headers show
Series
  • testing, docs, semihosting move and guest-loader
Related show

Commit Message

Alex Bennée March 10, 2021, 3:59 p.m.
From: Daniele Buono <dbuono@linux.vnet.ibm.com>


QEMU has had options to enable control-flow integrity features
for a few months now. Add two sets of build/check/acceptance
jobs to ensure the binary produced is working fine.

The three sets allow testing of x86_64 binaries for x86_64, s390x,
ppc64 and aarch64 targets

[AJB: tweak job names to avoid brands]

Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com>
Message-Id: <20210305092328.31792-9-alex.bennee@linaro.org>

-- 
2.20.1

Comments

Philippe Mathieu-Daudé March 22, 2021, 1:39 p.m. | #1
On 3/10/21 4:59 PM, Alex Bennée wrote:
> From: Daniele Buono <dbuono@linux.vnet.ibm.com>

> 

> QEMU has had options to enable control-flow integrity features

> for a few months now. Add two sets of build/check/acceptance

> jobs to ensure the binary produced is working fine.

> 

> The three sets allow testing of x86_64 binaries for x86_64, s390x,

> ppc64 and aarch64 targets

> 

> [AJB: tweak job names to avoid brands]

> 

> Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>

> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

> Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com>

> Message-Id: <20210305092328.31792-9-alex.bennee@linaro.org>


> +build-cfi-x86_64:

> +  <<: *native_build_job_definition

> +  needs:

> +  - job: amd64-fedora-container

> +  variables:

> +    LD_JOBS: 1

> +    AR: llvm-ar

> +    IMAGE: fedora

> +    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug

> +      --enable-safe-stack --enable-slirp=git

> +    TARGETS: x86_64-softmmu

> +    MAKE_CHECK_ARGS: check-build

> +  artifacts:

> +    expire_in: 2 days

> +    paths:

> +      - build


FYI this job is timeouting:

ERROR: Job failed: execution took longer than 1h0m0s seconds

https://gitlab.com/qemu-project/qemu/-/jobs/1112829128

> +

> +check-cfi-x86_64:

> +  <<: *native_test_job_definition

> +  needs:

> +    - job: build-cfi-x86_64

> +      artifacts: true

> +  variables:

> +    IMAGE: fedora

> +    MAKE_CHECK_ARGS: check

> +

> +acceptance-cfi-x86_64:

> +  <<: *native_test_job_definition

> +  needs:

> +    - job: build-cfi-x86_64

> +      artifacts: true

> +  variables:

> +    IMAGE: fedora

> +    MAKE_CHECK_ARGS: check-acceptance

> +  <<: *acceptance_definition

> +

>  tsan-build:

>    <<: *native_build_job_definition

>    variables:

>
Alex Bennée March 22, 2021, 2:06 p.m. | #2
Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 3/10/21 4:59 PM, Alex Bennée wrote:

>> From: Daniele Buono <dbuono@linux.vnet.ibm.com>

>> 

>> QEMU has had options to enable control-flow integrity features

>> for a few months now. Add two sets of build/check/acceptance

>> jobs to ensure the binary produced is working fine.

>> 

>> The three sets allow testing of x86_64 binaries for x86_64, s390x,

>> ppc64 and aarch64 targets

>> 

>> [AJB: tweak job names to avoid brands]

>> 

>> Signed-off-by: Daniele Buono <dbuono@linux.vnet.ibm.com>

>> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

>> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

>> Message-Id: <20210304030948.9367-3-dbuono@linux.vnet.ibm.com>

>> Message-Id: <20210305092328.31792-9-alex.bennee@linaro.org>

>

>> +build-cfi-x86_64:

>> +  <<: *native_build_job_definition

>> +  needs:

>> +  - job: amd64-fedora-container

>> +  variables:

>> +    LD_JOBS: 1

>> +    AR: llvm-ar

>> +    IMAGE: fedora

>> +    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug

>> +      --enable-safe-stack --enable-slirp=git

>> +    TARGETS: x86_64-softmmu

>> +    MAKE_CHECK_ARGS: check-build

>> +  artifacts:

>> +    expire_in: 2 days

>> +    paths:

>> +      - build

>

> FYI this job is timeouting:

>

> ERROR: Job failed: execution took longer than 1h0m0s seconds

>

> https://gitlab.com/qemu-project/qemu/-/jobs/1112829128

>


I didn't see it fail in my tests (although visibility is somewhat masked
by the current failing iotest). However I notice the builds are fairly
close to the limit (56-58 mins). As they are already pared down maybe
just increasing "timeout" is enough?

>> +

>> +check-cfi-x86_64:

>> +  <<: *native_test_job_definition

>> +  needs:

>> +    - job: build-cfi-x86_64

>> +      artifacts: true

>> +  variables:

>> +    IMAGE: fedora

>> +    MAKE_CHECK_ARGS: check

>> +

>> +acceptance-cfi-x86_64:

>> +  <<: *native_test_job_definition

>> +  needs:

>> +    - job: build-cfi-x86_64

>> +      artifacts: true

>> +  variables:

>> +    IMAGE: fedora

>> +    MAKE_CHECK_ARGS: check-acceptance

>> +  <<: *acceptance_definition

>> +

>>  tsan-build:

>>    <<: *native_build_job_definition

>>    variables:

>> 



-- 
Alex Bennée
Daniele Buono March 22, 2021, 2:08 p.m. | #3
Hi Philippe,

I'm looking at the public QEMU pipelines and it seems that that job
usually takes between 50 and 55 minutes, but there are higher spikes at
56, 57 and one where it failed at 1h.

We could perhaps set the timeout a bit higher, like 1h 10m, to not
terminate the outliers immediately?
The job you linked was almost over, there were just about 20-ish tests
to be linked, so it was probably next to completion.

On 3/22/2021 9:39 AM, Philippe Mathieu-Daudé wrote:
>> +build-cfi-x86_64:

>> +  <<: *native_build_job_definition

>> +  needs:

>> +  - job: amd64-fedora-container

>> +  variables:

>> +    LD_JOBS: 1

>> +    AR: llvm-ar

>> +    IMAGE: fedora

>> +    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug

>> +      --enable-safe-stack --enable-slirp=git

>> +    TARGETS: x86_64-softmmu

>> +    MAKE_CHECK_ARGS: check-build

>> +  artifacts:

>> +    expire_in: 2 days

>> +    paths:

>> +      - build

> FYI this job is timeouting:

> 

> ERROR: Job failed: execution took longer than 1h0m0s seconds

> 

> https://gitlab.com/qemu-project/qemu/-/jobs/1112829128

>

Patch

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 814f51873f..b23364bf3a 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -483,6 +483,125 @@  clang-user:
       --extra-cflags=-fsanitize=undefined --extra-cflags=-fno-sanitize-recover=undefined
     MAKE_CHECK_ARGS: check-unit check-tcg
 
+# Set LD_JOBS=1 because this requires LTO and ld consumes a large amount of memory.
+# On gitlab runners, default value sometimes end up calling 2 lds concurrently and
+# triggers an Out-Of-Memory error
+#
+# Since slirp callbacks are used in QEMU Timers, slirp needs to be compiled together
+# with QEMU and linked as a static library to avoid false positives in CFI checks.
+# This can be accomplished by using -enable-slirp=git, which avoids the use of
+# a system-wide version of the library
+#
+# Split in three sets of build/check/acceptance to limit the execution time of each
+# job
+build-cfi-aarch64:
+  <<: *native_build_job_definition
+  needs:
+  - job: amd64-fedora-container
+  variables:
+    LD_JOBS: 1
+    AR: llvm-ar
+    IMAGE: fedora
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug
+      --enable-safe-stack --enable-slirp=git
+    TARGETS: aarch64-softmmu
+    MAKE_CHECK_ARGS: check-build
+  artifacts:
+    expire_in: 2 days
+    paths:
+      - build
+
+check-cfi-aarch64:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-aarch64
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check
+
+acceptance-cfi-aarch64:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-aarch64
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check-acceptance
+  <<: *acceptance_definition
+
+build-cfi-ppc64-s390x:
+  <<: *native_build_job_definition
+  needs:
+  - job: amd64-fedora-container
+  variables:
+    LD_JOBS: 1
+    AR: llvm-ar
+    IMAGE: fedora
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug
+      --enable-safe-stack --enable-slirp=git
+    TARGETS: ppc64-softmmu s390x-softmmu
+    MAKE_CHECK_ARGS: check-build
+  artifacts:
+    expire_in: 2 days
+    paths:
+      - build
+
+check-cfi-ppc64-s390x:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-ppc64-s390x
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check
+
+acceptance-cfi-ppc64-s390x:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-ppc64-s390x
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check-acceptance
+  <<: *acceptance_definition
+
+build-cfi-x86_64:
+  <<: *native_build_job_definition
+  needs:
+  - job: amd64-fedora-container
+  variables:
+    LD_JOBS: 1
+    AR: llvm-ar
+    IMAGE: fedora
+    CONFIGURE_ARGS: --cc=clang --cxx=clang++ --enable-cfi --enable-cfi-debug
+      --enable-safe-stack --enable-slirp=git
+    TARGETS: x86_64-softmmu
+    MAKE_CHECK_ARGS: check-build
+  artifacts:
+    expire_in: 2 days
+    paths:
+      - build
+
+check-cfi-x86_64:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-x86_64
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check
+
+acceptance-cfi-x86_64:
+  <<: *native_test_job_definition
+  needs:
+    - job: build-cfi-x86_64
+      artifacts: true
+  variables:
+    IMAGE: fedora
+    MAKE_CHECK_ARGS: check-acceptance
+  <<: *acceptance_definition
+
 tsan-build:
   <<: *native_build_job_definition
   variables: