diff mbox series

[v3,09/13] tests/avocado: introduce alpine virt test for CI

Message ID 20221117172532.538149-10-alex.bennee@linaro.org
State Superseded
Headers show
Series testing and doc updates (pre-PR) | expand

Commit Message

Alex Bennée Nov. 17, 2022, 5:25 p.m. UTC
The boot_linux tests download and run a full cloud image boot and
start a full distro. While the ability to test the full boot chain is
worthwhile it is perhaps a little too heavy weight and causes issues
in CI. Fix this by introducing a new alpine linux ISO boot in
machine_aarch64_virt.

This boots a fully loaded -cpu max with all the bells and whistles in
31s on my machine. A full debug build takes around 180s on my machine
so we set a more generous timeout to cover that.

We don't add a test for lesser GIC versions although there is some
coverage for that already in the boot_xen.py tests. If we want to
introduce more comprehensive testing we can do it with a custom kernel
and initrd rather than a full distro boot.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

---
v1
  - use "virt" image instead (even faster)
  - don't drop boot_linux (it is now disabled for CI)
  - re-phrase commit message
  - add alpine to the test name
---
 tests/avocado/machine_aarch64_virt.py | 46 ++++++++++++++++++++++++++-
 1 file changed, 45 insertions(+), 1 deletion(-)

Comments

Philippe Mathieu-Daudé Nov. 18, 2022, 8:04 a.m. UTC | #1
On 17/11/22 18:25, Alex Bennée wrote:
> The boot_linux tests download and run a full cloud image boot and
> start a full distro. While the ability to test the full boot chain is
> worthwhile it is perhaps a little too heavy weight and causes issues
> in CI. Fix this by introducing a new alpine linux ISO boot in
> machine_aarch64_virt.
> 
> This boots a fully loaded -cpu max with all the bells and whistles in
> 31s on my machine. A full debug build takes around 180s on my machine
> so we set a more generous timeout to cover that.
> 
> We don't add a test for lesser GIC versions although there is some
> coverage for that already in the boot_xen.py tests. If we want to
> introduce more comprehensive testing we can do it with a custom kernel
> and initrd rather than a full distro boot.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> 
> ---
> v1
>    - use "virt" image instead (even faster)
>    - don't drop boot_linux (it is now disabled for CI)
>    - re-phrase commit message
>    - add alpine to the test name
> ---
>   tests/avocado/machine_aarch64_virt.py | 46 ++++++++++++++++++++++++++-
>   1 file changed, 45 insertions(+), 1 deletion(-)


> +    # This tests the whole boot chain from EFI to Userspace
> +    # We only boot a whole OS for the current top level CPU and GIC
> +    # Other test profiles should use more minimal boots
> +    def test_alpine_virt_tcg_gic_max(self):
> +        """
> +        :avocado: tags=arch:aarch64
> +        :avocado: tags=machine:virt
> +        :avocado: tags=accel:tcg
> +        """
> +        iso_url = ('https://dl-cdn.alpinelinux.org/'
> +                   'alpine/v3.16/releases/aarch64/'
> +                   'alpine-virt-3.16.3-aarch64.iso')
> +
> +        # Alpine use sha256 so I recalculated this myself
> +        iso_sha1 = '0683bc089486d55c91bf6607d5ecb93925769bc0'
> +        iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha1)
> +
> +        self.vm.set_console()
> +        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
> +                               'console=ttyAMA0')
> +        self.require_accelerator("tcg")
> +
> +        self.vm.add_args("-accel", "tcg")
> +        self.vm.add_args("-cpu", "max,pauth-impdef=on")
> +        self.vm.add_args("-machine",
> +                         "virt,acpi=on,"
> +                         "virtualization=on,"
> +                         "mte=on,"
> +                         "gic-version=max,iommu=smmuv3")
> +        self.vm.add_args("-smp", "2", "-m", "1024")
> +        self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
> +                                               'edk2-aarch64-code.fd'))

I am not sure about restricting to BUILD_DIR, I'd rather use an
externally prebuilt image, i.e.:
https://snapshots.linaro.org/components/kernel/leg-virt-tianocore-edk2-upstream/4710/QEMU-AARCH64/RELEASE_GCC5/

(I'd like to use these Avocado tests with a distrib provided QEMU
binary).

Anyhow can be fixed later, so:
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
diff mbox series

Patch

diff --git a/tests/avocado/machine_aarch64_virt.py b/tests/avocado/machine_aarch64_virt.py
index 21848cba70..c2b2ba2cf8 100644
--- a/tests/avocado/machine_aarch64_virt.py
+++ b/tests/avocado/machine_aarch64_virt.py
@@ -1,4 +1,5 @@ 
-# Functional test that boots a Linux kernel and checks the console
+# Functional test that boots a various Linux systems and checks the
+# console output.
 #
 # Copyright (c) 2022 Linaro Ltd.
 #
@@ -8,19 +9,62 @@ 
 # SPDX-License-Identifier: GPL-2.0-or-later
 
 import time
+import os
 
 from avocado_qemu import QemuSystemTest
 from avocado_qemu import wait_for_console_pattern
 from avocado_qemu import exec_command
+from avocado_qemu import BUILD_DIR
 
 class Aarch64VirtMachine(QemuSystemTest):
     KERNEL_COMMON_COMMAND_LINE = 'printk.time=0 '
+    timeout = 360
 
     def wait_for_console_pattern(self, success_message, vm=None):
         wait_for_console_pattern(self, success_message,
                                  failure_message='Kernel panic - not syncing',
                                  vm=vm)
 
+    # This tests the whole boot chain from EFI to Userspace
+    # We only boot a whole OS for the current top level CPU and GIC
+    # Other test profiles should use more minimal boots
+    def test_alpine_virt_tcg_gic_max(self):
+        """
+        :avocado: tags=arch:aarch64
+        :avocado: tags=machine:virt
+        :avocado: tags=accel:tcg
+        """
+        iso_url = ('https://dl-cdn.alpinelinux.org/'
+                   'alpine/v3.16/releases/aarch64/'
+                   'alpine-virt-3.16.3-aarch64.iso')
+
+        # Alpine use sha256 so I recalculated this myself
+        iso_sha1 = '0683bc089486d55c91bf6607d5ecb93925769bc0'
+        iso_path = self.fetch_asset(iso_url, asset_hash=iso_sha1)
+
+        self.vm.set_console()
+        kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE +
+                               'console=ttyAMA0')
+        self.require_accelerator("tcg")
+
+        self.vm.add_args("-accel", "tcg")
+        self.vm.add_args("-cpu", "max,pauth-impdef=on")
+        self.vm.add_args("-machine",
+                         "virt,acpi=on,"
+                         "virtualization=on,"
+                         "mte=on,"
+                         "gic-version=max,iommu=smmuv3")
+        self.vm.add_args("-smp", "2", "-m", "1024")
+        self.vm.add_args('-bios', os.path.join(BUILD_DIR, 'pc-bios',
+                                               'edk2-aarch64-code.fd'))
+        self.vm.add_args("-drive", f"file={iso_path},format=raw")
+        self.vm.add_args('-device', 'virtio-rng-pci,rng=rng0')
+        self.vm.add_args('-object', 'rng-random,id=rng0,filename=/dev/urandom')
+
+        self.vm.launch()
+        self.wait_for_console_pattern('Welcome to Alpine Linux 3.16')
+
+
     def test_aarch64_virt(self):
         """
         :avocado: tags=arch:aarch64