From patchwork Wed May 16 15:20:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 136008 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1068180lji; Wed, 16 May 2018 08:26:16 -0700 (PDT) X-Google-Smtp-Source: AB8JxZorQjtYx9SJTVcWklvu8FFsvmMgdj1Bo3Yxs3hHibV0aWQGufXvtTDllYifKfSwYbL2diVr X-Received: by 2002:ac8:364d:: with SMTP id n13-v6mr1337775qtb.174.1526484376883; Wed, 16 May 2018 08:26:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526484376; cv=none; d=google.com; s=arc-20160816; b=FKL7K10W51wO/OUGg/WaA/t7eIf9Ttv4RJtclXT30GiwIAnXZ5nQF8SLekNjvXzbCx 7pVFy0MvTs2oIXUuTveisgPy/X0m/vKh/oTIXRbgQRdWeVJR2h/fNgmB4FqRs+7KHgDB xxf/eoWwRAO+eQawnsj7mQqXIiGoVjvyJvu/mLklTnkuVu8C4zOISYKMV5jskPshzKIw IgpmYNiz908XfTDEEbLGxR0xA6YpZSFonqYgn1j52je2bAfu7q+HdX4qQDmT7bDbs5W4 ZsbtfjvxgPEzsl/IERco6yS2j3YfqERZZx7IbncYG3kQdRT+b1WVs34GJ8DfAt1uKN1U OEPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:mime-version:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=Fb5HikhDWUErmOb6J3sSybUBLTZKRPHbT8OuLVI8e1A=; b=Y8ER5freBr3gE/935VQk66ODLBRgjKbStY/XbZQtCr5Fy1PFpqaaN+/VrFyWxB2EvM Ys+NOhN10BeD8xkWDOfxYQyagqe/0Kb+JGZxJS4rZZkkARJ4CsG8VTFom4nI0AKfyoSy bF2si9z4UDz6NJ5564jigFx2oUIxTs8RoKguiNm11EG1FDpq2TInIMh22WKH1ZpzueOk 8gMcuMBeQrlSSYcIGTEyZVGwRws+E6YVw1YPXi/9UNTOh9dBdZ11Uyigi5pBTv78Hu8n ynHg7QJmcfy309WMo9ascSQbmLYX8wwTPh8nRUasnobBZuu4DnEHZv2lW/GoF+DBtqnH cuqA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id g18-v6si2838487qvd.192.2018.05.16.08.26.16 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 16 May 2018 08:26:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:51325 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyJk-00025f-Cu for patch@linaro.org; Wed, 16 May 2018 11:26:16 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54110) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyFn-0007pN-2k for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:15 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIyFl-0005mn-Ij for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:11 -0400 Received: from [45.249.212.32] (port=54404 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIyFb-0005cu-Q7; Wed, 16 May 2018 11:22:00 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.59]) by Forcepoint Email with ESMTP id 3B1826B686841; Wed, 16 May 2018 23:21:46 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.202.227.237) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.361.1; Wed, 16 May 2018 23:21:40 +0800 From: Shameer Kolothum To: , Date: Wed, 16 May 2018 16:20:21 +0100 Message-ID: <20180516152026.2920-2-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> References: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.237] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 45.249.212.32 Subject: [Qemu-devel] [RFC v2 1/6] hw/vfio: Retrieve valid iova ranges from kernel X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, jonathan.cameron@huawei.com, linuxarm@huawei.com, Shameer Kolothum , eric.auger@redhat.com, alex.williamson@redhat.com, zhaoshenglong@huawei.com, imammedo@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This makes use of the newly introduced iova cap chains added to the type1 VFIO_IOMMU_GET_INFO ioctl. The retrieved iova info is stored in a list for later use. Signed-off-by: Shameer Kolothum --- hw/vfio/common.c | 108 +++++++++++++++++++++++++++++++++++++++--- include/hw/vfio/vfio-common.h | 7 +++ linux-headers/linux/vfio.h | 23 +++++++++ 3 files changed, 132 insertions(+), 6 deletions(-) -- 2.7.4 diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 07ffa0b..94d7b24 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -40,6 +40,8 @@ struct vfio_group_head vfio_group_list = QLIST_HEAD_INITIALIZER(vfio_group_list); struct vfio_as_head vfio_address_spaces = QLIST_HEAD_INITIALIZER(vfio_address_spaces); +struct vfio_iova_head vfio_iova_regions = + QLIST_HEAD_INITIALIZER(vfio_iova_regions); #ifdef CONFIG_KVM /* @@ -1030,6 +1032,85 @@ static void vfio_put_address_space(VFIOAddressSpace *space) } } +static void vfio_iommu_get_iova_ranges(struct vfio_iommu_type1_info *info) +{ + struct vfio_info_cap_header *hdr; + struct vfio_iommu_type1_info_cap_iova_range *cap_iova; + VFIOIovaRange *iova, *tmp, *prev = NULL; + void *ptr = info; + bool found = false; + int i; + + if (!(info->flags & VFIO_IOMMU_INFO_CAPS)) { + return; + } + + for (hdr = ptr + info->cap_offset; hdr != ptr; hdr = ptr + hdr->next) { + if (hdr->id == VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE) { + found = true; + break; + } + } + + if (!found) { + return; + } + + /* purge the current iova list, if any */ + QLIST_FOREACH_SAFE(iova, &vfio_iova_regions, next, tmp) { + QLIST_REMOVE(iova, next); + g_free(iova); + } + + cap_iova = container_of(hdr, struct vfio_iommu_type1_info_cap_iova_range, + header); + + /* populate the list */ + for (i = 0; i < cap_iova->nr_iovas; i++) { + iova = g_malloc0(sizeof(*iova)); + iova->start = cap_iova->iova_ranges[i].start; + iova->end = cap_iova->iova_ranges[i].end; + + if (prev) { + QLIST_INSERT_AFTER(prev, iova, next); + } else { + QLIST_INSERT_HEAD(&vfio_iova_regions, iova, next); + } + prev = iova; + } + + return; +} + +static int vfio_get_iommu_info(VFIOContainer *container, + struct vfio_iommu_type1_info **info) +{ + + size_t argsz = sizeof(struct vfio_iommu_type1_info); + + + *info = g_malloc0(argsz); + +retry: + (*info)->argsz = argsz; + + if (ioctl(container->fd, VFIO_IOMMU_GET_INFO, *info)) { + g_free(*info); + *info = NULL; + return -errno; + } + + if (((*info)->argsz > argsz)) { + argsz = (*info)->argsz; + *info = g_realloc(*info, argsz); + goto retry; + } + + vfio_iommu_get_iova_ranges(*info); + + return 0; +} + static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, Error **errp) { @@ -1044,6 +1125,15 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, group->container = container; QLIST_INSERT_HEAD(&container->group_list, group, container_next); vfio_kvm_device_add_group(group); + + /* New group might change the valid iovas. Get the updated list */ + if ((container->iommu_type == VFIO_TYPE1_IOMMU) || + (container->iommu_type == VFIO_TYPE1v2_IOMMU)) { + struct vfio_iommu_type1_info *info; + + vfio_get_iommu_info(container, &info); + g_free(info); + } return 0; } } @@ -1071,7 +1161,7 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU) || ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) { bool v2 = !!ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU); - struct vfio_iommu_type1_info info; + struct vfio_iommu_type1_info *info; ret = ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &fd); if (ret) { @@ -1095,14 +1185,14 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as, * existing Type1 IOMMUs generally support any IOVA we're * going to actually try in practice. */ - info.argsz = sizeof(info); - ret = ioctl(fd, VFIO_IOMMU_GET_INFO, &info); + ret = vfio_get_iommu_info(container, &info); /* Ignore errors */ - if (ret || !(info.flags & VFIO_IOMMU_INFO_PGSIZES)) { + if (ret || !(info->flags & VFIO_IOMMU_INFO_PGSIZES)) { /* Assume 4k IOVA page size */ - info.iova_pgsizes = 4096; + info->iova_pgsizes = 4096; } - vfio_host_win_add(container, 0, (hwaddr)-1, info.iova_pgsizes); + vfio_host_win_add(container, 0, (hwaddr)-1, info->iova_pgsizes); + g_free(info); } else if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_SPAPR_TCE_IOMMU) || ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_SPAPR_TCE_v2_IOMMU)) { struct vfio_iommu_spapr_tce_info info; @@ -1256,6 +1346,7 @@ static void vfio_disconnect_container(VFIOGroup *group) if (QLIST_EMPTY(&container->group_list)) { VFIOAddressSpace *space = container->space; VFIOGuestIOMMU *giommu, *tmp; + VFIOIovaRange *iova, *next_iova; QLIST_REMOVE(container, next); @@ -1266,6 +1357,11 @@ static void vfio_disconnect_container(VFIOGroup *group) g_free(giommu); } + QLIST_FOREACH_SAFE(iova, &vfio_iova_regions, next, next_iova) { + QLIST_REMOVE(iova, next); + g_free(iova); + } + trace_vfio_disconnect_container(container->fd); close(container->fd); g_free(container); diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index d936014..874fe2c 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -164,6 +164,12 @@ typedef struct VFIODisplay { } dmabuf; } VFIODisplay; +typedef struct VFIOIovaRange { + uint64_t start; + uint64_t end; + QLIST_ENTRY(VFIOIovaRange) next; +} VFIOIovaRange; + void vfio_put_base_device(VFIODevice *vbasedev); void vfio_disable_irqindex(VFIODevice *vbasedev, int index); void vfio_unmask_single_irqindex(VFIODevice *vbasedev, int index); @@ -187,6 +193,7 @@ int vfio_get_device(VFIOGroup *group, const char *name, extern const MemoryRegionOps vfio_region_ops; extern QLIST_HEAD(vfio_group_head, VFIOGroup) vfio_group_list; extern QLIST_HEAD(vfio_as_head, VFIOAddressSpace) vfio_address_spaces; +extern QLIST_HEAD(vfio_iova_head, VFIOIovaRange) vfio_iova_regions; #ifdef CONFIG_LINUX int vfio_get_region_info(VFIODevice *vbasedev, int index, diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h index 3a0a305..117341d 100644 --- a/linux-headers/linux/vfio.h +++ b/linux-headers/linux/vfio.h @@ -589,7 +589,30 @@ struct vfio_iommu_type1_info { __u32 argsz; __u32 flags; #define VFIO_IOMMU_INFO_PGSIZES (1 << 0) /* supported page sizes info */ +#define VFIO_IOMMU_INFO_CAPS (1 << 1) /* Info supports caps */ __u64 iova_pgsizes; /* Bitmap of supported page sizes */ + __u32 cap_offset; /* Offset within info struct of first cap */ +}; + +/* + * The IOVA capability allows to report the valid IOVA range(s) + * excluding any reserved regions associated with dev group. Any dma + * map attempt outside the valid iova range will return error. + * + * The structures below define version 1 of this capability. + */ +#define VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE 1 + +struct vfio_iova_range { + __u64 start; + __u64 end; +}; + +struct vfio_iommu_type1_info_cap_iova_range { + struct vfio_info_cap_header header; + __u32 nr_iovas; + __u32 reserved; + struct vfio_iova_range iova_ranges[]; }; #define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12) From patchwork Wed May 16 15:20:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 136006 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1066862lji; Wed, 16 May 2018 08:25:04 -0700 (PDT) X-Google-Smtp-Source: AB8JxZryrpSecszvTSAxT31prq3tITlRFyljPVy2JjslJ2S9nquVVzsK3t0wmMW0mcEUQlFUwzUJ X-Received: by 2002:a0c:b551:: with SMTP id w17-v6mr1368358qvd.7.1526484303983; Wed, 16 May 2018 08:25:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526484303; cv=none; d=google.com; s=arc-20160816; b=A5cpQlyIYx+klfGcQXgNJXpD1Hk6+6UTeEyD/Hvy630jy2Zm7flr5T1fUHxMn2JX4x VdwmbdCl0fKDCcBYVRhjm2tWnSAIZrdAPuVDf/05a/nQVeCA4KujbWFFqa+pU7NNzw4H bDHYBcMf7QTXpgktBsFo0qu55eubIIwTyPbBj9L6dA+HMan1g9F+hORrEQL/PTIfz3NH mELwwIrV/1q9L9Z5eCtunVlH7JwfajPjhsq2luAGzGNmABuFQHkP1Z+9jrWCmir0VTbh wmweobO9Ak+b0VJfDmS+8bOMjF7qvSFyzvdAmSyOvjt2qkhjx5KE1TyGwoO9yfWhuHL4 TAcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:mime-version:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=wWUWHGcsDFzqG+NQLYWXDcxorLaXMH8nQVdeCrHH4+I=; b=DLonFtHxHi0xsQ9RpeFAge6V3sT7a9dOpsxNj0/LPrdkQxjGaXI2NreV3m5gFybwCZ /kHgPo3IHjwHgVbCFTzgw9tg/KGZ3ktOSFGx60m4c5pl5ibGxYZhBPayk2jkqGhPwI2q GlxHB2fK47+lO55COWAMsdxqZx5jRleo5IXewm4DpmLHWT6ggwb3gn3yt2SuV7bpiwqd wLouf9oZz6KXFFjx7yZzPN2PplJw1B0W+iq+lCUBpAZ/GaJd+Qyy1z6YF/ylGLp0QpDZ vnX1ZwxoEeeM/+o7YNzFEc43AOdzw0to/Zlu2i0clYXxEnGFToS+EQVOXD4PJm6CGMs6 whWw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id p185-v6si2606141qkf.387.2018.05.16.08.25.03 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 16 May 2018 08:25:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:51237 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyIZ-0000jF-Cc for patch@linaro.org; Wed, 16 May 2018 11:25:03 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54185) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyFs-0007uZ-Vp for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:21 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIyFm-0005nk-T1 for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:16 -0400 Received: from szxga04-in.huawei.com ([45.249.212.190]:2617 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIyFj-0005iu-CG; Wed, 16 May 2018 11:22:08 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id C2EEEBB4595AC; Wed, 16 May 2018 23:21:51 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.202.227.237) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.361.1; Wed, 16 May 2018 23:21:44 +0800 From: Shameer Kolothum To: , Date: Wed, 16 May 2018 16:20:22 +0100 Message-ID: <20180516152026.2920-3-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> References: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.237] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 45.249.212.190 Subject: [Qemu-devel] [RFC v2 2/6] hw/arm/virt: Enable dynamic generation of guest RAM memory regions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, jonathan.cameron@huawei.com, linuxarm@huawei.com, Shameer Kolothum , eric.auger@redhat.com, alex.williamson@redhat.com, zhaoshenglong@huawei.com, imammedo@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Register ram_memory_region_init notifier to allocate memory region from system memory. Signed-off-by: Zhu Yijun Signed-off-by: Shameer Kolothum --- hw/arm/virt.c | 28 ++++++++++++++++++++++------ include/hw/arm/virt.h | 1 + 2 files changed, 23 insertions(+), 6 deletions(-) -- 2.7.4 diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 94dcb12..05fcb62 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1171,6 +1171,19 @@ void virt_machine_done(Notifier *notifier, void *data) virt_build_smbios(vms); } +static void virt_ram_memory_region_init(Notifier *notifier, void *data) +{ + MemoryRegion *sysmem = get_system_memory(); + MemoryRegion *ram = g_new(MemoryRegion, 1); + VirtMachineState *vms = container_of(notifier, VirtMachineState, + ram_memory_region_init); + MachineState *machine = MACHINE(vms); + + memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram", + machine->ram_size); + memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram); +} + static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx) { uint8_t clustersz = ARM_DEFAULT_CPUS_PER_CLUSTER; @@ -1204,7 +1217,6 @@ static void machvirt_init(MachineState *machine) MemoryRegion *sysmem = get_system_memory(); MemoryRegion *secure_sysmem = NULL; int n, virt_max_cpus; - MemoryRegion *ram = g_new(MemoryRegion, 1); bool firmware_loaded = bios_name || drive_get(IF_PFLASH, 0, 0); /* We can probe only here because during property set @@ -1361,10 +1373,6 @@ static void machvirt_init(MachineState *machine) fdt_add_timer_nodes(vms); fdt_add_cpu_nodes(vms); - memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram", - machine->ram_size); - memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram); - create_flash(vms, sysmem, secure_sysmem ? secure_sysmem : sysmem); create_gic(vms, pic); @@ -1405,15 +1413,23 @@ static void machvirt_init(MachineState *machine) vms->bootinfo.loader_start = vms->memmap[VIRT_MEM].base; vms->bootinfo.get_dtb = machvirt_dtb; vms->bootinfo.firmware_loaded = firmware_loaded; + + /* Register notifiers. They are executed in registration reverse order */ arm_load_kernel(ARM_CPU(first_cpu), &vms->bootinfo); /* * arm_load_kernel machine init done notifier registration must * happen before the platform_bus_create call. In this latter, * another notifier is registered which adds platform bus nodes. - * Notifiers are executed in registration reverse order. */ create_platform_bus(vms, pic); + + /* + * Register memory region notifier last as this has to be executed + * first. + */ + vms->ram_memory_region_init.notify = virt_ram_memory_region_init; + qemu_add_machine_init_done_notifier(&vms->ram_memory_region_init); } static bool virt_get_secure(Object *obj, Error **errp) diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index ba0c1a4..fc24f3a 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -91,6 +91,7 @@ typedef struct { typedef struct { MachineState parent; Notifier machine_done; + Notifier ram_memory_region_init; FWCfgState *fw_cfg; bool secure; bool highmem; From patchwork Wed May 16 15:20:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 136032 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1126695lji; Wed, 16 May 2018 09:14:55 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqj2/8qyVqZEFMlXAs6KYRcl5Qo3JhcbGsTdQspgYModfYwa0S8CPcRWi1rpLBtzGYNAckj X-Received: by 2002:ac8:2d6e:: with SMTP id o43-v6mr1558411qta.316.1526487295728; Wed, 16 May 2018 09:14:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526487295; cv=none; d=google.com; s=arc-20160816; b=en0y7ESaD+Xs74WkF0YvNo9qGN6xt9hvLDV0N/OtMuWNet23eOetfV8iKAI8dwKnZp 9gnqux36DH4mUzUkG/fyPy556Y+KO1EWW1A+0QJi5CU6RVIuGLrCaGhaF7r6YszNDKJ1 Vp3UECLKMl8IHrG9lpmn9ZVD/TGByQmqvFxpgFyZvenSxQAgLVjEf7aK2md8ZuzvZAwT 7Tw95CIljAGgkj5odmkiEA9cc9T2rK49xzSO7XRkp27PWrvgIdJOk2SjBhex9IWdiG0/ wu0o7hecp1wfmNFBAjxDcN8GxPz6xQoRFy15uVdwVW/3L8d9bveYcLrDsM+loJxj1l0b SjjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:mime-version:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=DZ5ucGknKUl4UPbUqlwbWO++nJgtnqQPoRnewCNgLnQ=; b=eonFcXUHP5qPjr0jp6BLQaXgkrIaNRHgcBkIWUGqDBTYrrmNGF96rFJNUpNFjvBwOQ MJun1b0nSCwPpeGQObSheB1x5T6P3tz0eOo/WLWv2C4lN5g239UB5f8YAP53WyGdlm9e bsp4o1iGEyVoeGnWxrKk1BvOF48pcYtZvXii5tv0/tgnPUbXV0gdMxnnDy3raaoWF3uT 2pk+oOOkI/N4+o4XwNTMV9v+yIZ5maBCynaNsCkljqzdgL8XVAWsEaopAJ7ZtL6yJoAl lVDbIeyCylWlvvzk3kIbvcgPoCRoFK0SInts46ri989qbsBYykzcgwwobV9QQ/o21Sh1 4How== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id l3-v6si2987862qkf.93.2018.05.16.09.14.55 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 16 May 2018 09:14:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:53414 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIz4p-0004Ta-51 for patch@linaro.org; Wed, 16 May 2018 12:14:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51900) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyzQ-0000oH-VU for qemu-devel@nongnu.org; Wed, 16 May 2018 12:09:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIyzP-0004fT-Oe for qemu-devel@nongnu.org; Wed, 16 May 2018 12:09:20 -0400 Received: from [45.249.212.35] (port=58059 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIyzK-0004au-W5; Wed, 16 May 2018 12:09:15 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 6C751BB0E397D; Wed, 16 May 2018 23:21:56 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.202.227.237) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.361.1; Wed, 16 May 2018 23:21:49 +0800 From: Shameer Kolothum To: , Date: Wed, 16 May 2018 16:20:23 +0100 Message-ID: <20180516152026.2920-4-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> References: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.237] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 45.249.212.35 Subject: [Qemu-devel] [RFC v2 3/6] hw/arm/virt: Add pc-dimm mem hotplug framework X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, jonathan.cameron@huawei.com, linuxarm@huawei.com, Shameer Kolothum , eric.auger@redhat.com, alex.williamson@redhat.com, zhaoshenglong@huawei.com, imammedo@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This will be used in subsequent patches to model a chunk of memory as pc-dimm(cold plug) if the valid iova regions are non-contiguous. This is not yet a full hotplug support. Signed-off-by: Shameer Kolothum --- default-configs/aarch64-softmmu.mak | 1 + hw/arm/virt.c | 82 +++++++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 2 + 3 files changed, 85 insertions(+) -- 2.7.4 diff --git a/default-configs/aarch64-softmmu.mak b/default-configs/aarch64-softmmu.mak index 9ddccf8..7a82ed8 100644 --- a/default-configs/aarch64-softmmu.mak +++ b/default-configs/aarch64-softmmu.mak @@ -8,3 +8,4 @@ CONFIG_DDC=y CONFIG_DPCD=y CONFIG_XLNX_ZYNQMP=y CONFIG_XLNX_ZYNQMP_ARM=y +CONFIG_MEM_HOTPLUG=y diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 05fcb62..be3ad14 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1552,9 +1552,82 @@ static const CPUArchIdList *virt_possible_cpu_arch_ids(MachineState *ms) return ms->possible_cpus; } +static void virt_dimm_plug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + VirtMachineState *vms = VIRT_MACHINE(hotplug_dev); + PCDIMMDevice *dimm = PC_DIMM(dev); + PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm); + MemoryRegion *mr; + uint64_t align; + Error *local_err = NULL; + + mr = ddc->get_memory_region(dimm, &local_err); + if (local_err) { + goto out; + } + + align = memory_region_get_alignment(mr); + pc_dimm_memory_plug(dev, &vms->hotplug_memory, mr, align, &local_err); + if (local_err) { + goto out; + } + +out: + error_propagate(errp, local_err); +} + +static void virt_dimm_unplug(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + VirtMachineState *vms = VIRT_MACHINE(hotplug_dev); + PCDIMMDevice *dimm = PC_DIMM(dev); + PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm); + MemoryRegion *mr; + Error *local_err = NULL; + + mr = ddc->get_memory_region(dimm, &local_err); + pc_dimm_memory_unplug(dev, &vms->hotplug_memory, mr); + object_unparent(OBJECT(dev)); + + error_propagate(errp, local_err); +} + +static void virt_machinedevice_plug_cb(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { + virt_dimm_plug(hotplug_dev, dev, errp); + } else { + error_setg(errp, "device plug request for not supported device" + " type: %s", object_get_typename(OBJECT(dev))); + } +} + +static void virt_machinedevice_unplug_cb(HotplugHandler *hotplug_dev, + DeviceState *dev, Error **errp) +{ + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { + virt_dimm_unplug(hotplug_dev, dev, errp); + } else { + error_setg(errp, "device unplug for not supported device" + " type: %s", object_get_typename(OBJECT(dev))); + } +} + +static HotplugHandler *virt_get_hotplug_handler(MachineState *machine, + DeviceState *dev) +{ + if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) { + return HOTPLUG_HANDLER(machine); + } + return NULL; +} + static void virt_machine_class_init(ObjectClass *oc, void *data) { MachineClass *mc = MACHINE_CLASS(oc); + HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc); mc->init = machvirt_init; /* Start max_cpus at the maximum QEMU supports. We'll further restrict @@ -1573,6 +1646,11 @@ static void virt_machine_class_init(ObjectClass *oc, void *data) mc->cpu_index_to_instance_props = virt_cpu_index_to_props; mc->default_cpu_type = ARM_CPU_TYPE_NAME("cortex-a15"); mc->get_default_cpu_node_id = virt_get_default_cpu_node_id; + + mc->get_hotplug_handler = virt_get_hotplug_handler; + hc->plug = virt_machinedevice_plug_cb; + hc->unplug = virt_machinedevice_unplug_cb; + } static const TypeInfo virt_machine_info = { @@ -1582,6 +1660,10 @@ static const TypeInfo virt_machine_info = { .instance_size = sizeof(VirtMachineState), .class_size = sizeof(VirtMachineClass), .class_init = virt_machine_class_init, + .interfaces = (InterfaceInfo[]) { + { TYPE_HOTPLUG_HANDLER }, + { } + }, }; static void machvirt_machine_init(void) diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index fc24f3a..a39f29e 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -35,6 +35,7 @@ #include "qemu/notify.h" #include "hw/boards.h" #include "hw/arm/arm.h" +#include "hw/mem/pc-dimm.h" #define NUM_GICV2M_SPIS 64 #define NUM_VIRTIO_TRANSPORTS 32 @@ -108,6 +109,7 @@ typedef struct { uint32_t gic_phandle; uint32_t msi_phandle; int psci_conduit; + MemoryHotplugState hotplug_memory; } VirtMachineState; #define TYPE_VIRT_MACHINE MACHINE_TYPE_NAME("virt") From patchwork Wed May 16 15:20:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 136009 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1069612lji; Wed, 16 May 2018 08:27:32 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqrpC4awdA7cZBH86Mmigf8wXC83hsNTc0tKMsedhEajVBqPc8fJ3zu/jh+q8AqYELuxVL1 X-Received: by 2002:a37:9d8d:: with SMTP id g135-v6mr1278495qke.197.1526484452449; Wed, 16 May 2018 08:27:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526484452; cv=none; d=google.com; s=arc-20160816; b=kxN+OwaAzFilKF8J80dt5xf+pN5Qv8AdLLDmHyt/LZ4Flgphm167OznB9T9ylElid8 EZlicKTjx2rcuLgzN7qrzFinNDy6cgpx0+XgSVet5/42l18oAwzXmKPeZ0gLS+fXL1ny Zvi99iTK61yf5AdojDjI7uS4zL4wY7VghKuNADnGYwod4w8/JWfAE/E7ufRrtRTDJDI8 W7b0plR3R18UKp16cp+NGNOBU4xfwOWPaAimDEBJ5YKzlCGDtxYY59/IVHS+M52LRvj/ hLqX4ypZCuFH/rkOOHZluM2dsUTvx2OnHQ0cdXuc0DdUd9uK2mnNReVS0r9nEY4oJbum zNVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:mime-version:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=Uvi2Erl9aJDgJiDZ/uKw+pAyIWunAY5jNTAwP+VGb0g=; b=Uq+2rZ1AWIrnB/ipnrFzOOiiq5vbVkqrKUM8UYmu8t96nZEtxPizCFQfWNhd8lquTQ F8nDUBqLvvcIViGK8bRuFRZWjcXMymoX/a4YhEkKfTmUOERlfVh75nk736FPvuySYVud l3AqsHSQweZBiiadwH6wS5ZqzD7RCvnatyAdDazrlm+5H0qR/8Be0SV6RVdvG+QoZ2an hCl8fTnRfJLZ83IU9aa8pnXSIMlmuNotE1DOrp/i1fl0kaWoEjvjT+b/qMOCZ6TVr0ek qlNEElOmXirHuQ7bVlmCKmiRkK2XbDKhDny7B0gu+HyNJmOKTMYDADjvT1Rz/fV83mji 3tJA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id r126-v6si2800927qke.293.2018.05.16.08.27.32 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 16 May 2018 08:27:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:51376 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyKx-0002qe-V8 for patch@linaro.org; Wed, 16 May 2018 11:27:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54199) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyFt-0007vD-Kv for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIyFs-0005rW-C5 for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:17 -0400 Received: from [45.249.212.32] (port=54447 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIyFi-0005iq-Bm; Wed, 16 May 2018 11:22:07 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 6E18C6500791A; Wed, 16 May 2018 23:22:01 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.202.227.237) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.361.1; Wed, 16 May 2018 23:21:53 +0800 From: Shameer Kolothum To: , Date: Wed, 16 May 2018 16:20:24 +0100 Message-ID: <20180516152026.2920-5-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> References: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.237] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 45.249.212.32 Subject: [Qemu-devel] [RFC v2 4/6] hw/arm: Changes required to accommodate non-contiguous DT mem nodes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, jonathan.cameron@huawei.com, linuxarm@huawei.com, Shameer Kolothum , eric.auger@redhat.com, alex.williamson@redhat.com, zhaoshenglong@huawei.com, imammedo@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This makes changes to the DT mem node creation such that its easier to add non-contiguous mem modeled as non-pluggable and a pc-dimm mem later. Signed-off-by: Shameer Kolothum --- hw/arm/boot.c | 91 ++++++++++++++++++++++++++++++++++++---------------- include/hw/arm/arm.h | 12 +++++++ 2 files changed, 75 insertions(+), 28 deletions(-) -- 2.7.4 diff --git a/hw/arm/boot.c b/hw/arm/boot.c index 26184bc..73db0aa 100644 --- a/hw/arm/boot.c +++ b/hw/arm/boot.c @@ -486,6 +486,27 @@ static void fdt_add_psci_node(void *fdt) qemu_fdt_setprop_cell(fdt, "/psci", "migrate", migrate_fn); } +static char *create_memory_fdt(void *fdt, uint32_t acells, hwaddr mem_base, + uint32_t scells, hwaddr mem_len) +{ + char *nodename = NULL; + int rc; + + nodename = g_strdup_printf("/memory@%" PRIx64, mem_base); + qemu_fdt_add_subnode(fdt, nodename); + qemu_fdt_setprop_string(fdt, nodename, "device_type", "memory"); + rc = qemu_fdt_setprop_sized_cells(fdt, nodename, "reg", acells, mem_base, + scells, mem_len); + if (rc < 0) { + fprintf(stderr, "couldn't set %s/reg\n", nodename); + g_free(nodename); + return NULL; + } + + return nodename; +} + + /** * load_dtb() - load a device tree binary image into memory * @addr: the address to load the image at @@ -567,50 +588,64 @@ static int load_dtb(hwaddr addr, const struct arm_boot_info *binfo, goto fail; } + /* + * Turn the /memory node created before into a NOP node, then create + * /memory@addr nodes for all numa nodes respectively. + */ + qemu_fdt_nop_node(fdt, "/memory"); + if (nb_numa_nodes > 0) { - /* - * Turn the /memory node created before into a NOP node, then create - * /memory@addr nodes for all numa nodes respectively. - */ - qemu_fdt_nop_node(fdt, "/memory"); + hwaddr mem_sz; + mem_base = binfo->loader_start; + mem_sz = binfo->ram_size; for (i = 0; i < nb_numa_nodes; i++) { - mem_len = numa_info[i].node_mem; - nodename = g_strdup_printf("/memory@%" PRIx64, mem_base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "memory"); - rc = qemu_fdt_setprop_sized_cells(fdt, nodename, "reg", - acells, mem_base, + mem_len = MIN(numa_info[i].node_mem, mem_sz); + + nodename = create_memory_fdt(fdt, acells, mem_base, scells, mem_len); - if (rc < 0) { - fprintf(stderr, "couldn't set %s/reg for node %d\n", nodename, - i); + if (!nodename) { goto fail; } qemu_fdt_setprop_cell(fdt, nodename, "numa-node-id", i); - mem_base += mem_len; g_free(nodename); + mem_base += mem_len; + mem_sz -= mem_len; + if (!mem_sz) { + break; + } } - } else { - Error *err = NULL; - rc = fdt_path_offset(fdt, "/memory"); - if (rc < 0) { - qemu_fdt_add_subnode(fdt, "/memory"); - } + /* Create the node for initial pc-dimm ram, if any */ + if (binfo->dimm_mem) { - if (!qemu_fdt_getprop(fdt, "/memory", "device_type", NULL, &err)) { - qemu_fdt_setprop_string(fdt, "/memory", "device_type", "memory"); + nodename = create_memory_fdt(fdt, acells, binfo->dimm_mem->base, + scells, binfo->dimm_mem->size); + if (!nodename) { + goto fail; + } + qemu_fdt_setprop_cell(fdt, nodename, "numa-node-id", + binfo->dimm_mem->node); + g_free(nodename); } - rc = qemu_fdt_setprop_sized_cells(fdt, "/memory", "reg", - acells, binfo->loader_start, - scells, binfo->ram_size); - if (rc < 0) { - fprintf(stderr, "couldn't set /memory/reg\n"); + } else { + + nodename = create_memory_fdt(fdt, acells, binfo->loader_start, + scells, binfo->ram_size); + if (!nodename) { goto fail; } + + if (binfo->dimm_mem) { + nodename = create_memory_fdt(fdt, acells, binfo->dimm_mem->base, + scells, binfo->dimm_mem->size); + if (!nodename) { + goto fail; + } + g_free(nodename); + } } rc = fdt_path_offset(fdt, "/chosen"); diff --git a/include/hw/arm/arm.h b/include/hw/arm/arm.h index ce769bd..0ee3b4e 100644 --- a/include/hw/arm/arm.h +++ b/include/hw/arm/arm.h @@ -48,6 +48,12 @@ typedef struct { ARMCPU *cpu; /* handle to the first cpu object */ } ArmLoadKernelNotifier; +struct dimm_mem_info { + int node; + hwaddr base; + hwaddr size; +}; + /* arm_boot.c */ struct arm_boot_info { uint64_t ram_size; @@ -124,6 +130,12 @@ struct arm_boot_info { bool secure_board_setup; arm_endianness endianness; + + /* This is used to model a pc-dimm based mem if the valid iova region + * is non-contiguous. + */ + struct dimm_mem_info *dimm_mem; + }; /** From patchwork Wed May 16 15:20:25 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 136010 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1071179lji; Wed, 16 May 2018 08:28:58 -0700 (PDT) X-Google-Smtp-Source: AB8JxZrfpqpKiJfckG7PAyu5uSqNDtIiZSvXEdvwjwmChQwYFIYU5sbdSA9tuJWg2pehjZnftZ+H X-Received: by 2002:a0c:d5f5:: with SMTP id h50-v6mr1379076qvi.218.1526484538710; Wed, 16 May 2018 08:28:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526484538; cv=none; d=google.com; s=arc-20160816; b=gu91g27xyuoYnrYE+1eCS/5cVvMgUzrpeG2mygitYawTa6ibDlfsnxDNKYM2Z+sozW h7kNEgluo94+5OVKCamwYVnxVYcbh1Wo5C5Q0fVZcX9O/s5jFuE6RusBIa08VmQ3auLB HsGojZ52YuTXL50fs/GZGngdgzbN3eqZVlNc6uyHaDC5b1VuvDhKV7Kf5zGwUWxKXM1X U8aKqDzAHtQwq9Kdu+IO4/0mFCINB5Q7ZwdIlSm0OW0ePk6omg8wcAYJo5zG+J6Ple+7 VnE6aXitsBNKQ4S2XRAoli7cIzaADzlbCoZfTy2fTa3C4INswdsTPVJ1qZU/zfISOhSG BqwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:mime-version:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=jMVykAtftxhHfjtXcd1NgkJxs0XvF2WnZvFkcjKhBU0=; b=WzXMJEx4pRQVUGYajHFeYb20FuaP6pA8HF+KLds/PjjagpD7rgThn8XKd16UEuP8Va 3OgQqV3uiVnBtpvWEoZqciTRKPisxo2NiMSpoZSCRlBARALAwjhCE6iiDm+Ja8ZM6p2a NoViYKvkQjpdhjMM3CJozC7zvwbpvkCAVK/cm0V3xKBa/Z+tGPxQfkTVHMi1DQUStJfw w1WTs1zBnnpUMYKUX7YH8RNc4VXsGXQ+JeTCsdcdM2wFqyvo6UFVey1mCR8U2XGxrLeU gkqoTYwQgSxUwGGvhNGnwDWdjjeX78AFwXOJHiw4pNRpUhIr+hQOnKfmAQ/FNB7Y1Y7L zBrA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id s6-v6si2823719qtg.169.2018.05.16.08.28.58 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 16 May 2018 08:28:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:51441 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyMM-0003oQ-86 for patch@linaro.org; Wed, 16 May 2018 11:28:58 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54292) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyG5-000856-F3 for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:35 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIyG4-0005x1-IE for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:29 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:2097 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIyFy-0005sq-4E; Wed, 16 May 2018 11:22:22 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 274CED4DEFF90; Wed, 16 May 2018 23:22:06 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.202.227.237) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.361.1; Wed, 16 May 2018 23:21:57 +0800 From: Shameer Kolothum To: , Date: Wed, 16 May 2018 16:20:25 +0100 Message-ID: <20180516152026.2920-6-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> References: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.237] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 45.249.212.191 Subject: [Qemu-devel] [RFC v2 5/6] hw/arm: ACPI SRAT changes to accommodate non-contiguous mem X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, jonathan.cameron@huawei.com, linuxarm@huawei.com, Shameer Kolothum , eric.auger@redhat.com, alex.williamson@redhat.com, zhaoshenglong@huawei.com, imammedo@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is in preparation for the next patch where initial ram is split into a non-pluggable chunk and a pc-dimm modeled mem if the vaild iova regions are non-contiguous. Signed-off-by: Shameer Kolothum --- hw/arm/virt-acpi-build.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) -- 2.7.4 diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index c7c6a57..8d17b40 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -488,7 +488,7 @@ build_srat(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) AcpiSratProcessorGiccAffinity *core; AcpiSratMemoryAffinity *numamem; int i, srat_start; - uint64_t mem_base; + uint64_t mem_base, mem_sz, mem_len; MachineClass *mc = MACHINE_GET_CLASS(vms); const CPUArchIdList *cpu_list = mc->possible_cpu_arch_ids(MACHINE(vms)); @@ -505,12 +505,28 @@ build_srat(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) core->flags = cpu_to_le32(1); } - mem_base = vms->memmap[VIRT_MEM].base; + mem_base = vms->bootinfo.loader_start; + mem_sz = vms->bootinfo.loader_start; for (i = 0; i < nb_numa_nodes; ++i) { numamem = acpi_data_push(table_data, sizeof(*numamem)); - build_srat_memory(numamem, mem_base, numa_info[i].node_mem, i, + mem_len = MIN(numa_info[i].node_mem, mem_sz); + build_srat_memory(numamem, mem_base, mem_len, i, MEM_AFFINITY_ENABLED); - mem_base += numa_info[i].node_mem; + mem_base += mem_len; + mem_sz -= mem_len; + if (!mem_sz) { + break; + } + } + + /* Create table for initial pc-dimm ram, if any */ + if (vms->bootinfo.dimm_mem) { + numamem = acpi_data_push(table_data, sizeof(*numamem)); + build_srat_memory(numamem, vms->bootinfo.dimm_mem->base, + vms->bootinfo.dimm_mem->size, + vms->bootinfo.dimm_mem->node, + MEM_AFFINITY_ENABLED); + } build_header(linker, table_data, (void *)(table_data->data + srat_start), From patchwork Wed May 16 15:20:26 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 136007 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp1066943lji; Wed, 16 May 2018 08:25:08 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqrglUiFjSQv6dXszAOrgE6vVxSvGwhjsar795IAlEiEUNI95xFgGRqZ16x1Bmbpl7btXLL X-Received: by 2002:aed:2d07:: with SMTP id h7-v6mr1394005qtd.252.1526484308708; Wed, 16 May 2018 08:25:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526484308; cv=none; d=google.com; s=arc-20160816; b=g1AYR5VBODaC2viXH/t7x4RIDYevcXA/aI3b+H/IiGhYFdCxZ/gdXTu8qDdtlGZbMz 3Rp5tb+2lrPQ3xpzYJXrzw5H8AsPpXar6CcCZPX7weSJcpMhVztnvnhQbvYdHFOx+DE3 ZUiq82zr3RZMSNmjiAWg9xLHHLDJXgtTG9sl650/5adH+HdG+A4weTdg/AdGboEZH59j O/BEKwSC8eKb2WIxcY0h39olxNYW9NonmEEx4vfVuWjK/awJdjNVsKSdtPu5uW7PmayR 40wIia1jTTa+CWHpea87CvHXbIrQGVtWPZx7jPr8f5Pn1q8kY1sqz7ccxUZjGr4M6MAr NHIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:mime-version:references :in-reply-to:message-id:date:to:from:arc-authentication-results; bh=4a3qgemOLQb6j4jk0vDKXTkh2RUh9l8PicCTqW4vKNU=; b=oqfZypwaX09dmTqo+UVrkOT4B5hCRiD30FHaY+TYkggFcFdz9JpIRZL8Hu9dvlCY8j lpQSZpbZtjSS1uvD22eH/FWNqjcjP2nOwbBQe3AgRhroAum35BfTc6eU8bE0Q/+Qb0Cv qneNpZe+hS8W2xdKG567VP/T+kTO8xpX0KpCd0/8wlNAZx4vY97JGnulfaHqvoIh3pPz U+zR37QVMOv/AWyQRQQ+KX1vej1kI5MhUtPyYx8abKXPDN9nWIwg5GiPn8WDlfGxYk6k LzoAUQUh9EHYAk2OBCG+7z1d53cJUlTfXQY+ztOiqh+jte9aHkIk50doBz0M9awTZ7Tm plwA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id m14-v6si2827478qti.49.2018.05.16.08.25.08 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 16 May 2018 08:25:08 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org Received: from localhost ([::1]:51238 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyIe-0000nT-5H for patch@linaro.org; Wed, 16 May 2018 11:25:08 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54319) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fIyGE-0008BU-28 for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:39 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fIyG9-0005yW-7C for qemu-devel@nongnu.org; Wed, 16 May 2018 11:22:38 -0400 Received: from szxga05-in.huawei.com ([45.249.212.191]:2098 helo=huawei.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fIyFz-0005sv-0U; Wed, 16 May 2018 11:22:24 -0400 Received: from DGGEMS404-HUB.china.huawei.com (unknown [172.30.72.58]) by Forcepoint Email with ESMTP id 36BEBE5CA1DC9; Wed, 16 May 2018 23:22:06 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.202.227.237) by DGGEMS404-HUB.china.huawei.com (10.3.19.204) with Microsoft SMTP Server id 14.3.361.1; Wed, 16 May 2018 23:22:01 +0800 From: Shameer Kolothum To: , Date: Wed, 16 May 2018 16:20:26 +0100 Message-ID: <20180516152026.2920-7-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> References: <20180516152026.2920-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.202.227.237] X-CFilter-Loop: Reflected X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 45.249.212.191 Subject: [Qemu-devel] [RFC v2 6/6] hw/arm: Populate non-contiguous memory regions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, drjones@redhat.com, jonathan.cameron@huawei.com, linuxarm@huawei.com, Shameer Kolothum , eric.auger@redhat.com, alex.williamson@redhat.com, zhaoshenglong@huawei.com, imammedo@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" In case valid iova regions are non-contiguous, split the RAM mem into a 1GB non-pluggable dimm and remaining as a single pc-dimm mem. Signed-off-by: Shameer Kolothum --- hw/arm/virt.c | 261 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 256 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/hw/arm/virt.c b/hw/arm/virt.c index be3ad14..562e389 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -58,6 +58,12 @@ #include "hw/smbios/smbios.h" #include "qapi/visitor.h" #include "standard-headers/linux/input.h" +#include "hw/vfio/vfio-common.h" +#include "qemu/config-file.h" +#include "monitor/qdev.h" +#include "qom/object_interfaces.h" +#include "qapi/qmp/qdict.h" +#include "qemu/option.h" #define DEFINE_VIRT_MACHINE_LATEST(major, minor, latest) \ static void virt_##major##_##minor##_class_init(ObjectClass *oc, \ @@ -110,7 +116,10 @@ static ARMPlatformBusSystemParams platform_bus_params; * terabyte of physical address space.) */ #define RAMLIMIT_GB 255 -#define RAMLIMIT_BYTES (RAMLIMIT_GB * 1024ULL * 1024 * 1024) +#define SZ_1G (1024ULL * 1024 * 1024) +#define RAMLIMIT_BYTES (RAMLIMIT_GB * SZ_1G) + +#define ALIGN_1G (1ULL << 30) /* Addresses and sizes of our components. * 0..128MB is space for a flash device so we can run bootrom code such as UEFI. @@ -1171,6 +1180,236 @@ void virt_machine_done(Notifier *notifier, void *data) virt_build_smbios(vms); } +static void free_iova_copy(struct vfio_iova_head *iova_copy) +{ + VFIOIovaRange *iova, *tmp; + + QLIST_FOREACH_SAFE(iova, iova_copy, next, tmp) { + QLIST_REMOVE(iova, next); + g_free(iova); + } +} + +static void get_iova_copy(struct vfio_iova_head *iova_copy) +{ + VFIOIovaRange *iova, *new, *prev_iova = NULL; + + QLIST_FOREACH(iova, &vfio_iova_regions, next) { + new = g_malloc0(sizeof(*iova)); + new->start = iova->start; + new->end = iova->end; + + if (prev_iova) { + QLIST_INSERT_AFTER(prev_iova, new, next); + } else { + QLIST_INSERT_HEAD(iova_copy, new, next); + } + prev_iova = new; + } +} + +static hwaddr find_memory_chunk(VirtMachineState *vms, + struct vfio_iova_head *iova_copy, + hwaddr req_size, bool pcdimm) +{ + VFIOIovaRange *iova, *tmp; + hwaddr new_start, new_size, sz_align; + hwaddr virt_start = vms->memmap[VIRT_MEM].base; + hwaddr addr_align = ALIGN_1G; /* Set to max ARM64 hugepage size */ + + /* Size alignment */ + sz_align = (pcdimm) ? MAX(TARGET_PAGE_SIZE, QEMU_VMALLOC_ALIGN) : + TARGET_PAGE_SIZE; + + QLIST_FOREACH_SAFE(iova, iova_copy, next, tmp) { + if (virt_start >= iova->end) { + continue; + } + + /* Align addr */ + new_start = ROUND_UP(MAX(virt_start, iova->start), addr_align); + if (new_start >= iova->end) { + continue; + } + + if (req_size > iova->end - new_start + 1) { + continue; + } + + /* + * Check the region can hold any size alignment requirement. + */ + new_size = QEMU_ALIGN_UP(req_size, sz_align); + + if ((new_start + new_size - 1 > iova->end) || + (new_start + new_size >= virt_start + RAMLIMIT_BYTES)) { + continue; + } + + /* + * Modify the iova list entry for non pc-dimm case so that it + * is not used again for pc-dimm allocation. + */ + if (!pcdimm) { + if (new_size - req_size) { + iova->start = new_start + new_size; + } else { + QLIST_REMOVE(iova, next); + } + } + return new_start; + } + + return -1; +} + +static void update_memory_regions(VirtMachineState *vms) +{ + hwaddr addr; + VFIOIovaRange *iova; + MachineState *machine = MACHINE(vms); + hwaddr virt_start = vms->memmap[VIRT_MEM].base; + hwaddr req_size, ram_size = machine->ram_size; + struct vfio_iova_head iova_copy = QLIST_HEAD_INITIALIZER(iova_copy); + + /* No valid iova regions, use default */ + if (QLIST_EMPTY(&vfio_iova_regions)) { + vms->bootinfo.loader_start = vms->memmap[VIRT_MEM].base; + vms->bootinfo.ram_size = ram_size; + return; + } + + /* + * If valid iovas has only one entry, check the req size fits in + * and can have the loader start < 4GB. This will make sure platforms + * with no holes in mem will have the same mem model as before. + */ + req_size = ram_size; + iova = QLIST_NEXT(QLIST_FIRST(&vfio_iova_regions), next); + if (!iova) { + iova = QLIST_FIRST(&vfio_iova_regions); + addr = ROUND_UP(MAX(virt_start, iova->start), ALIGN_1G); + if ((addr < 4 * SZ_1G) && (ram_size <= iova->end - addr + 1) && + (addr + ram_size < virt_start + RAMLIMIT_BYTES)) { + vms->bootinfo.loader_start = addr; + vms->bootinfo.ram_size = ram_size; + return; + } + } + + /* Get a copy of valid iovas and work on it */ + get_iova_copy(&iova_copy); + + /* Split the mem as first 1GB non-pluggable and rest as pc-dimm */ + req_size = MIN(ram_size, SZ_1G); + addr = find_memory_chunk(vms, &iova_copy, req_size, false); + if (addr == -1 || addr >= 4 * SZ_1G) { + goto out; + } + + /*Update non-pluggable mem details */ + machine->ram_size = req_size; + vms->bootinfo.loader_start = addr; + vms->bootinfo.ram_size = machine->ram_size; + + req_size = ram_size - req_size; + if (!req_size) { + goto done; + } + + /* Remaining memory is modeled as a pc-dimm. */ + addr = find_memory_chunk(vms, &iova_copy, req_size, true); + if (addr == -1) { + goto out; + } + + /*Update pc-dimm mem details */ + vms->bootinfo.dimm_mem = g_new(struct dimm_mem_info, 1); + vms->bootinfo.dimm_mem->base = addr; + vms->bootinfo.dimm_mem->size = req_size; + machine->maxram_size = machine->ram_size + req_size; + machine->ram_slots += 1; + +done: + free_iova_copy(&iova_copy); + return; + +out: + free_iova_copy(&iova_copy); + error_report("mach-virt: Not enough contiguous memory to model ram"); + exit(1); +} + +static void create_pcdimms(VirtMachineState *vms, + MemoryRegion *sysmem, + MemoryRegion *ram) +{ + hwaddr addr, size; + Error *local_err = NULL; + QDict *qdict; + QemuOpts *opts; + char *tmp; + + if (!vms->bootinfo.dimm_mem) { + return; + } + + addr = vms->bootinfo.dimm_mem->base; + size = vms->bootinfo.dimm_mem->size; + + /*Create hotplug address space */ + vms->hotplug_memory.base = ROUND_UP(addr, ALIGN_1G); + size = ROUND_UP(size, MAX(TARGET_PAGE_SIZE, QEMU_VMALLOC_ALIGN)); + + memory_region_init(&vms->hotplug_memory.mr, OBJECT(vms), + "hotplug-memory", size); + memory_region_add_subregion(sysmem, vms->hotplug_memory.base, + &vms->hotplug_memory.mr); + /* Create backend mem object */ + qdict = qdict_new(); + qdict_put_str(qdict, "qom-type", "memory-backend-ram"); + qdict_put_str(qdict, "id", "mem1"); + tmp = g_strdup_printf("%"PRIu64 "M", size / (1024 * 1024)); + qdict_put_str(qdict, "size", tmp); + g_free(tmp); + + opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &local_err); + if (local_err) { + goto err; + } + + user_creatable_add_opts(opts, &local_err); + qemu_opts_del(opts); + QDECREF(qdict); + if (local_err) { + goto err; + } + + /* Create pc-dimm dev*/ + qdict = qdict_new(); + qdict_put_str(qdict, "driver", "pc-dimm"); + qdict_put_str(qdict, "id", "dimm1"); + qdict_put_str(qdict, "memdev", "mem1"); + + opts = qemu_opts_from_qdict(qemu_find_opts("device"), qdict, &local_err); + if (local_err) { + goto err; + } + + qdev_device_add(opts, &local_err); + qemu_opts_del(opts); + QDECREF(qdict); + if (local_err) { + goto err; + } + + return; + +err: + error_report_err(local_err); + exit(1); +} + static void virt_ram_memory_region_init(Notifier *notifier, void *data) { MemoryRegion *sysmem = get_system_memory(); @@ -1179,9 +1418,14 @@ static void virt_ram_memory_region_init(Notifier *notifier, void *data) ram_memory_region_init); MachineState *machine = MACHINE(vms); + update_memory_regions(vms); memory_region_allocate_system_memory(ram, NULL, "mach-virt.ram", machine->ram_size); - memory_region_add_subregion(sysmem, vms->memmap[VIRT_MEM].base, ram); + memory_region_add_subregion(sysmem, vms->bootinfo.loader_start, ram); + + if (vms->bootinfo.dimm_mem) { + create_pcdimms(vms, sysmem, ram); + } } static uint64_t virt_cpu_mp_affinity(VirtMachineState *vms, int idx) @@ -1404,13 +1648,11 @@ static void machvirt_init(MachineState *machine) vms->machine_done.notify = virt_machine_done; qemu_add_machine_init_done_notifier(&vms->machine_done); - vms->bootinfo.ram_size = machine->ram_size; vms->bootinfo.kernel_filename = machine->kernel_filename; vms->bootinfo.kernel_cmdline = machine->kernel_cmdline; vms->bootinfo.initrd_filename = machine->initrd_filename; vms->bootinfo.nb_cpus = smp_cpus; vms->bootinfo.board_id = -1; - vms->bootinfo.loader_start = vms->memmap[VIRT_MEM].base; vms->bootinfo.get_dtb = machvirt_dtb; vms->bootinfo.firmware_loaded = firmware_loaded; @@ -1559,7 +1801,7 @@ static void virt_dimm_plug(HotplugHandler *hotplug_dev, PCDIMMDevice *dimm = PC_DIMM(dev); PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm); MemoryRegion *mr; - uint64_t align; + uint64_t align, addr; Error *local_err = NULL; mr = ddc->get_memory_region(dimm, &local_err); @@ -1573,6 +1815,15 @@ static void virt_dimm_plug(HotplugHandler *hotplug_dev, goto out; } + addr = object_property_get_uint(OBJECT(dimm), PC_DIMM_ADDR_PROP, + &error_fatal); + /* Assign the node for pc-dimm initial ram */ + if (vms->bootinfo.dimm_mem && (addr == vms->bootinfo.dimm_mem->base) + && (nb_numa_nodes > 0)) { + vms->bootinfo.dimm_mem->node = object_property_get_uint(OBJECT(dev), + PC_DIMM_NODE_PROP, &error_fatal); + } + out: error_propagate(errp, local_err); }