From patchwork Thu Nov 5 17:11:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jonathan Cameron X-Patchwork-Id: 320642 Delivered-To: patch@linaro.org Received: by 2002:a92:7b12:0:0:0:0:0 with SMTP id w18csp618284ilc; Thu, 5 Nov 2020 09:14:44 -0800 (PST) X-Google-Smtp-Source: ABdhPJz0rRQ+pLKrc5z70apXkIq9bS1GOqMYBfiLzyv6HSGlWUNIolfr2uxWw6yHjv31Re264V+T X-Received: by 2002:a25:cc43:: with SMTP id l64mr5181466ybf.114.1604596484306; Thu, 05 Nov 2020 09:14:44 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1604596484; cv=none; d=google.com; s=arc-20160816; b=jD3Hu+rA3MijjTuIPyid1kDe06gGMl6s52KDJKLQ8Q9qhG/AgNMztgxGKmEx9QxTr1 7tnIFYCSY/l2CVzc+QevE2JjhGunmjXUY4ct5rElW92PGS7m/q+CcLaYKfJFRVBEBefk 4iaHQIQNdKIuG0OS5agr1jyBkCH72jW1y4lfY43h3O8hUUKa6MClENismm6WABqUj25y cV9il9OEp4uZO4v0H3Pbf4EmzKs5xqa9IXRtMqMo5uoKNc7BNjxoUeenevMsf3S1gRgE xuzXeIDwq8jMaT8iP1EhftnTJJXGZQDJi4zVxoWDsZQTUJbhPmTHHeEKNBjqr7+w+iU9 Ij1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:message-id:date:subject:to:from; bh=UxPoJWR9KyH3MIa39q+wTgFXJzRugHHOixObh8RyHaE=; b=G/pPPZ8ZktNb3PaIWSjXT/Si3xO62P6aDVUuyibPbJSL8bOBWAklsooV4qeFRvMpKE rOH7uxGEYvFJWcrLcU1Ov4HTqKOHWYO1fpSCoErYz5Qrvc0mxKA2b9zFZ9ZgPAs7I4io sPAV2vAHFA3MZjnRmOJ+bi3QkS930RgtzJf0Hifl77slArM5OPkSYrned0cPF6FbcdoA MP13AEa4P5czR0j/YQCEjadCVLrUF/NtUkBGK2XCdYGIMMC7tLxC7hiZSPCvAI36xsJc 0O/Wio6yp56Kj5xXbq0rNSp2emJauHa1Eamn5poVZYZ4c+mQEs0Yz50CtYRSheJ15y16 0LDw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org" Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e79si2010485ybf.297.2020.11.05.09.14.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 05 Nov 2020 09:14:44 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org" Received: from localhost ([::1]:43406 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kaiqR-0001vW-O6 for patch@linaro.org; Thu, 05 Nov 2020 12:14:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:54398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kaipH-0001ro-0B for qemu-devel@nongnu.org; Thu, 05 Nov 2020 12:13:31 -0500 Received: from frasgout.his.huawei.com ([185.176.79.56]:2048) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kaipE-0008Gh-4B for qemu-devel@nongnu.org; Thu, 05 Nov 2020 12:13:30 -0500 Received: from fraeml704-chm.china.huawei.com (unknown [172.18.147.226]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4CRqn54jrnz67Hpt; Fri, 6 Nov 2020 01:11:37 +0800 (CST) Received: from lhreml710-chm.china.huawei.com (10.201.108.61) by fraeml704-chm.china.huawei.com (10.206.15.53) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1913.5; Thu, 5 Nov 2020 18:13:12 +0100 Received: from lhrphicprd00229.huawei.com (10.123.41.22) by lhreml710-chm.china.huawei.com (10.201.108.61) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.1913.5; Thu, 5 Nov 2020 17:13:12 +0000 From: Jonathan Cameron To: Subject: [PATCH] hw/arm/virt enable support for virtio-mem Date: Fri, 6 Nov 2020 01:11:44 +0800 Message-ID: <20201105171144.560612-1-Jonathan.Cameron@huawei.com> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 X-Originating-IP: [10.123.41.22] X-ClientProxiedBy: lhreml705-chm.china.huawei.com (10.201.108.54) To lhreml710-chm.china.huawei.com (10.201.108.61) X-CFilter-Loop: Reflected Received-SPF: pass client-ip=185.176.79.56; envelope-from=jonathan.cameron@huawei.com; helo=frasgout.his.huawei.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/11/05 12:13:13 X-ACL-Warn: Detected OS = Linux 3.1-3.10 [fuzzy] X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , David Hildenbrand , linuxarm@huawei.com, Jonathan Cameron , "Michael S . Tsirkin" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Basically a cut and paste job from the x86 support with the exception of needing a larger block size as the Memory Block Size (MIN_SECTION_SIZE) on ARM64 in Linux is 1G. Tested: * In full emulation and with KVM on an arm64 server. * cold and hotplug for the virtio-mem-pci device. * Wide range of memory sizes, added at creation and later. * Fairly basic memory usage of memory added. Seems to function as normal. * NUMA setup with virtio-mem-pci devices on each node. * Simple migration test. Related kernel patch just enables the Kconfig item for ARM64 as an alternative to x86 in drivers/virtio/Kconfig The original patches from David Hildenbrand stated that he thought it should work for ARM64 but it wasn't enabled in the kernel [1] It appears he was correct and everything 'just works'. The build system related stuff is intended to ensure virtio-mem support is not built for arm32 (build will fail due no defined block size). If there is a more elegant way to do this, please point me in the right direction. [1] https://lore.kernel.org/linux-mm/20191212171137.13872-1-david@redhat.com/ Signed-off-by: Jonathan Cameron --- default-configs/devices/aarch64-softmmu.mak | 1 + hw/arm/Kconfig | 1 + hw/arm/virt.c | 64 ++++++++++++++++++++- hw/virtio/Kconfig | 4 ++ hw/virtio/virtio-mem.c | 2 + 5 files changed, 71 insertions(+), 1 deletion(-) -- 2.19.1 diff --git a/default-configs/devices/aarch64-softmmu.mak b/default-configs/devices/aarch64-softmmu.mak index 958b1e08e4..31d6128a29 100644 --- a/default-configs/devices/aarch64-softmmu.mak +++ b/default-configs/devices/aarch64-softmmu.mak @@ -6,3 +6,4 @@ include arm-softmmu.mak CONFIG_XLNX_ZYNQMP_ARM=y CONFIG_XLNX_VERSAL=y CONFIG_SBSA_REF=y +CONFIG_ARCH_VIRTIO_MEM_SUPPORTED=y diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig index fdf4464b94..eeae77eee9 100644 --- a/hw/arm/Kconfig +++ b/hw/arm/Kconfig @@ -20,6 +20,7 @@ config ARM_VIRT select PLATFORM_BUS select SMBIOS select VIRTIO_MMIO + select VIRTIO_MEM_SUPPORTED if ARCH_VIRTIO_MEM_SUPPORTED select ACPI_PCI select MEM_DEVICE select DIMM diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 8abb385d4e..a9f927ce83 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -73,9 +73,11 @@ #include "hw/acpi/acpi.h" #include "target/arm/internals.h" #include "hw/mem/pc-dimm.h" +#include "hw/mem/memory-device.h" #include "hw/mem/nvdimm.h" #include "hw/acpi/generic_event_device.h" #include "hw/virtio/virtio-iommu.h" +#include "hw/virtio/virtio-mem-pci.h" #include "hw/char/pl011.h" #include "qemu/guest-random.h" @@ -2286,6 +2288,34 @@ static void virt_memory_plug(HotplugHandler *hotplug_dev, dev, &error_abort); } +static void virt_virtio_md_pci_pre_plug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + HotplugHandler *hotplug_dev2 = qdev_get_bus_hotplug_handler(dev); + Error *local_err = NULL; + + if (!hotplug_dev2 && dev->hotplugged) { + /* + * Without a bus hotplug handler, we cannot control the plug/unplug + * order. We should never reach this point when hotplugging, + * however, better add a safety net. + */ + error_setg(errp, "hotplug of virtio-mem based memory devices not supported" + " on this bus."); + return; + } + /* + * First, see if we can plug this memory device at all. If that + * succeeds, branch of to the actual hotplug handler. + */ + memory_device_pre_plug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev), NULL, + &local_err); + if (!local_err && hotplug_dev2) { + hotplug_handler_pre_plug(hotplug_dev2, dev, &local_err); + } + error_propagate(errp, local_err); +} + static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { @@ -2293,6 +2323,8 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { virt_memory_pre_plug(hotplug_dev, dev, errp); + } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + virt_virtio_md_pci_pre_plug(hotplug_dev, dev, errp); } else if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) { hwaddr db_start = 0, db_end = 0; char *resv_prop_str; @@ -2322,6 +2354,27 @@ static void virt_machine_device_pre_plug_cb(HotplugHandler *hotplug_dev, } } +static void virt_virtio_md_pci_plug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + HotplugHandler *hotplug_dev2 = qdev_get_bus_hotplug_handler(dev); + Error *local_err = NULL; + + /* + * Plug the memory device first and then branch off to the actual + * hotplug handler. If that one fails, we can easily undo the memory + * device bits. + */ + memory_device_plug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev)); + if (hotplug_dev2) { + hotplug_handler_plug(hotplug_dev2, dev, &local_err); + if (local_err) { + memory_device_unplug(MEMORY_DEVICE(dev), MACHINE(hotplug_dev)); + } + } + error_propagate(errp, local_err); +} + static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev, DeviceState *dev, Error **errp) { @@ -2336,6 +2389,9 @@ static void virt_machine_device_plug_cb(HotplugHandler *hotplug_dev, if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { virt_memory_plug(hotplug_dev, dev, errp); } + if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + virt_virtio_md_pci_plug(hotplug_dev, dev, errp); + } if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) { PCIDevice *pdev = PCI_DEVICE(dev); @@ -2363,6 +2419,11 @@ static void virt_dimm_unplug_request(HotplugHandler *hotplug_dev, goto out; } + if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)) { + error_setg(&local_err, + "virtio-mem based memory devices cannot be unplugged."); + goto out; + } hotplug_handler_unplug_request(HOTPLUG_HANDLER(vms->acpi_dev), dev, &local_err); out: @@ -2413,7 +2474,8 @@ static HotplugHandler *virt_machine_get_hotplug_handler(MachineState *machine, DeviceState *dev) { if (object_dynamic_cast(OBJECT(dev), TYPE_SYS_BUS_DEVICE) || - (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM))) { + object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM) || + object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_MEM_PCI)){ return HOTPLUG_HANDLER(machine); } if (object_dynamic_cast(OBJECT(dev), TYPE_VIRTIO_IOMMU_PCI)) { diff --git a/hw/virtio/Kconfig b/hw/virtio/Kconfig index 0eda25c4e1..00dbf2939e 100644 --- a/hw/virtio/Kconfig +++ b/hw/virtio/Kconfig @@ -48,6 +48,10 @@ config VIRTIO_PMEM depends on VIRTIO_PMEM_SUPPORTED select MEM_DEVICE +config ARCH_VIRTIO_MEM_SUPPORTED + bool + default n + config VIRTIO_MEM_SUPPORTED bool diff --git a/hw/virtio/virtio-mem.c b/hw/virtio/virtio-mem.c index 7c8ca9f28b..c28f12547a 100644 --- a/hw/virtio/virtio-mem.c +++ b/hw/virtio/virtio-mem.c @@ -53,6 +53,8 @@ */ #if defined(TARGET_X86_64) || defined(TARGET_I386) #define VIRTIO_MEM_USABLE_EXTENT (2 * (128 * MiB)) +#elif defined (TARGET_AARCH64) +#define VIRTIO_MEM_USABLE_EXTENT (2 * (1024 * MiB)) #else #error VIRTIO_MEM_USABLE_EXTENT not defined #endif