From patchwork Tue Apr 18 16:21:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 674525 Delivered-To: patch@linaro.org Received: by 2002:a5d:4a41:0:0:0:0:0 with SMTP id v1csp59266wrs; Tue, 18 Apr 2023 09:24:15 -0700 (PDT) X-Google-Smtp-Source: AKy350aSId0N9/tu/l28PuN4FHdxZEuZDVdcOJMZKGe43wsG/IyUGwsLm82aLbPj5amz5gpiJb+C X-Received: by 2002:ac8:5bc6:0:b0:3bf:cea5:7004 with SMTP id b6-20020ac85bc6000000b003bfcea57004mr447015qtb.51.1681835055256; Tue, 18 Apr 2023 09:24:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1681835055; cv=none; d=google.com; s=arc-20160816; b=Ea2l5A8JGzXXnrcWtWR833wOqW+kicF73w4+Z1AtXUhQ1n9TujwWz9/Dhr5r6XJZTa XXAnrex2WwEemSWUA6WyijnBnTOPLLIwbxJBO4OsrSiqgM2/ffy0YyWPBfA+Gq8a1JJr StM+vZO2jzLaJ31x+T/9wHo91YCcJLSwgsmHAqi5liaRA2RZputoNIk9XJTIkKWcAp2b Kxqc9QDwT+N44ETWQ+gfzHETxhMXAs8TloOZT6WHNeB3F3Wi2uGDWGcH1ZZNpRn/AvM0 WQYsVa9nUzgdNTQICuV+RwapX4jpN4PtTjJiFJXrS166C69zZ3x9FWYLDzDVNZrj+n6U CxCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=1A+5QxSTbXGCo55UOJ3nazzxUb4+UhfLG866jSa6yr4=; b=WxXyK6O9oG3IR2EHvvpVZOsN2wJs6yszgtSEEpca3H7wnwWeGcOzkBNJYU/carx0Ro Y0fX4BO9NggIQNDBxi+6pq9r32PYbB+Y+ACRayrPQ1qK+1kHnnJnqXqUn3skJD1zgw6i Byhem6I2ZY2tQciPgyNaONDoOvF3OGyv6G0TxZYAxLYYrBLKMHJa8jI453xyOE6E+RQT /UeEemBwBx7kRfZLwLJt4xDhyW307qo/FazbTf87pAHBGun/c4FNLoV9iMROt+ZZeWtb 2RajixTecIfwcbvaxIWFBG7Uu2LSCig4ui2OhrguqkiJWeUEsyn2Aeby3tnNusZYw5mf bgdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=f1mgtZ69; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n62-20020a37bd41000000b0074a4fab58basi7221526qkf.3.2023.04.18.09.24.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 18 Apr 2023 09:24:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=f1mgtZ69; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1poo5W-0006n3-2q; Tue, 18 Apr 2023 12:21:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1poo5U-0006ma-IA for qemu-devel@nongnu.org; Tue, 18 Apr 2023 12:21:48 -0400 Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1poo5S-0001Av-8H for qemu-devel@nongnu.org; Tue, 18 Apr 2023 12:21:48 -0400 Received: by mail-wm1-x32b.google.com with SMTP id l31-20020a05600c1d1f00b003f1718d89b2so168513wms.0 for ; Tue, 18 Apr 2023 09:21:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1681834904; x=1684426904; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1A+5QxSTbXGCo55UOJ3nazzxUb4+UhfLG866jSa6yr4=; b=f1mgtZ69k7Kj4ijAMctLgIExd27VV0GYl8ZoJeqILw5TcDiA6VMDGnpLmzyPnBWUHW f56feqkULdz6T5g838ckwfrD2TDGMvol2vfZzqb30tBheX1yBcz5mqU+UU1pYtqxjpvC b6aCgaP8/bIZlKUp1kGL3126f9Eu9qH83/sgyndWcMiFT715RhaRoUFUFfpDdYsTV3HC 5GvfarJtUDIUfvbVa8f1ZQ+NHuhQBMQQaxoXrATs4+8Ez3Eqn8ZhFlh3g0RRPrnqKuMT qGOX8UsgyU1ika6PwVKKBJQnOLhPU6jTy36jRWdJm/+lk4AzUhAebndGsrqMGIObcCec s6Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681834904; x=1684426904; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1A+5QxSTbXGCo55UOJ3nazzxUb4+UhfLG866jSa6yr4=; b=D66YBpmodvb57xMrK9215YYO38TeIYMWxjD2XLMbSoawG1OseF+sy5SLOMmYb+Hznb OufR1BVDNSsvqNpaqh+IcYUEqblEnwU7kRFKSVpoD1haP+HGPZqE83rCLWgA89ezf551 tmbuUyhtom036crUg0TjE6nEGNb4h1pME2BpDhxzfn47MiHyI0ZyvGUpA4+YBsZqeBNi IlfwEQhkV8g/Jpd6fLlr5xDYMX7Ofy4Qs59W+ed1rsLo1pvWDx5sbT5qXZx6IakPW/fv YzZ5oO5a3mBoW2NhPRKKyQEimOhxB4uuw8NiYULhegteNPWq9cWgViTWomgGNvMpc/Jo m2NQ== X-Gm-Message-State: AAQBX9fhu2wnMuqN4KOj3ARqvjm+MC/TGTnyrOOmVhujqlgpqPcfwCB4 PdHsfQUZm1LBdUs/1SfagxI4cA== X-Received: by 2002:a05:600c:228b:b0:3f1:718d:a21c with SMTP id 11-20020a05600c228b00b003f1718da21cmr6200468wmf.31.1681834904383; Tue, 18 Apr 2023 09:21:44 -0700 (PDT) Received: from zen.linaroharston ([85.9.250.243]) by smtp.gmail.com with ESMTPSA id t15-20020a05600c198f00b003f173956a82sm6563522wmq.21.2023.04.18.09.21.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 09:21:42 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id E41061FFBF; Tue, 18 Apr 2023 17:21:40 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , =?utf-8?q?Daniel_?= =?utf-8?q?P=2E_Berrang=C3=A9?= , Eduardo Habkost , Viresh Kumar , Mathieu Poirier , "Gonglei (Arei)" , Markus Armbruster , Erik Schilling , Jason Wang , Mark Cave-Ayland , Gerd Hoffmann , virtio-fs@redhat.com, =?utf-8?q?Marc-And?= =?utf-8?q?r=C3=A9_Lureau?= , Paolo Bonzini , "Michael S. Tsirkin" , Stefan Hajnoczi , Eric Blake Subject: [PATCH v2 07/13] virtio: add vhost-user-base and a generic vhost-user-device Date: Tue, 18 Apr 2023 17:21:34 +0100 Message-Id: <20230418162140.373219-8-alex.bennee@linaro.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230418162140.373219-1-alex.bennee@linaro.org> References: <20230418162140.373219-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::32b; envelope-from=alex.bennee@linaro.org; helo=mail-wm1-x32b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org In theory we shouldn't need to repeat so much boilerplate to support vhost-user backends. This provides a generic vhost-user-base QOM object and a derived vhost-user-device for which the user needs to provide the few bits of information that aren't currently provided by the vhost-user protocol. This should provide a baseline implementation from which the other vhost-user stub can specialise. Signed-off-by: Alex Bennée --- v2 - split into vub and vud --- include/hw/virtio/vhost-user-device.h | 45 ++++ hw/virtio/vhost-user-device.c | 324 ++++++++++++++++++++++++++ hw/virtio/meson.build | 2 + 3 files changed, 371 insertions(+) create mode 100644 include/hw/virtio/vhost-user-device.h create mode 100644 hw/virtio/vhost-user-device.c diff --git a/include/hw/virtio/vhost-user-device.h b/include/hw/virtio/vhost-user-device.h new file mode 100644 index 0000000000..9105011e25 --- /dev/null +++ b/include/hw/virtio/vhost-user-device.h @@ -0,0 +1,45 @@ +/* + * Vhost-user generic virtio device + * + * Copyright (c) 2023 Linaro Ltd + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef QEMU_VHOST_USER_DEVICE_H +#define QEMU_VHOST_USER_DEVICE_H + +#include "hw/virtio/vhost.h" +#include "hw/virtio/vhost-user.h" + +#define TYPE_VHOST_USER_BASE "vhost-user-base" + +OBJECT_DECLARE_TYPE(VHostUserBase, VHostUserBaseClass, VHOST_USER_BASE) + +struct VHostUserBase { + VirtIODevice parent; + /* Properties */ + CharBackend chardev; + uint16_t virtio_id; + uint32_t num_vqs; + /* State tracking */ + VhostUserState vhost_user; + struct vhost_virtqueue *vhost_vq; + struct vhost_dev vhost_dev; + GPtrArray *vqs; + bool connected; +}; + + /* needed so we can use the base realize after specialisation + tweaks */ +struct VHostUserBaseClass { + /*< private >*/ + VirtioDeviceClass parent_class; + /*< public >*/ + DeviceRealize parent_realize; +}; + +/* shared for the benefit of the derived pci class */ +#define TYPE_VHOST_USER_DEVICE "vhost-user-device" + +#endif /* QEMU_VHOST_USER_DEVICE_H */ diff --git a/hw/virtio/vhost-user-device.c b/hw/virtio/vhost-user-device.c new file mode 100644 index 0000000000..b0239fa033 --- /dev/null +++ b/hw/virtio/vhost-user-device.c @@ -0,0 +1,324 @@ +/* + * Generic vhost-user stub. This can be used to connect to any + * vhost-user backend. All configuration details must be handled by + * the vhost-user daemon itself + * + * Copyright (c) 2023 Linaro Ltd + * Author: Alex Bennée + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "hw/qdev-properties.h" +#include "hw/virtio/virtio-bus.h" +#include "hw/virtio/vhost-user-device.h" +#include "qemu/error-report.h" + +static void vub_start(VirtIODevice *vdev) +{ + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + VHostUserBase *vub = VHOST_USER_BASE(vdev); + int ret, i; + + if (!k->set_guest_notifiers) { + error_report("binding does not support guest notifiers"); + return; + } + + ret = vhost_dev_enable_notifiers(&vub->vhost_dev, vdev); + if (ret < 0) { + error_report("Error enabling host notifiers: %d", -ret); + return; + } + + ret = k->set_guest_notifiers(qbus->parent, vub->vhost_dev.nvqs, true); + if (ret < 0) { + error_report("Error binding guest notifier: %d", -ret); + goto err_host_notifiers; + } + + vub->vhost_dev.acked_features = vdev->guest_features; + + ret = vhost_dev_start(&vub->vhost_dev, vdev, true); + if (ret < 0) { + error_report("Error starting vhost-user-device: %d", -ret); + goto err_guest_notifiers; + } + + /* + * guest_notifier_mask/pending not used yet, so just unmask + * everything here. virtio-pci will do the right thing by + * enabling/disabling irqfd. + */ + for (i = 0; i < vub->vhost_dev.nvqs; i++) { + vhost_virtqueue_mask(&vub->vhost_dev, vdev, i, false); + } + + return; + +err_guest_notifiers: + k->set_guest_notifiers(qbus->parent, vub->vhost_dev.nvqs, false); +err_host_notifiers: + vhost_dev_disable_notifiers(&vub->vhost_dev, vdev); +} + +static void vub_stop(VirtIODevice *vdev) +{ + VHostUserBase *vub = VHOST_USER_BASE(vdev); + BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); + VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus); + int ret; + + if (!k->set_guest_notifiers) { + return; + } + + vhost_dev_stop(&vub->vhost_dev, vdev, true); + + ret = k->set_guest_notifiers(qbus->parent, vub->vhost_dev.nvqs, false); + if (ret < 0) { + error_report("vhost guest notifier cleanup failed: %d", ret); + return; + } + + vhost_dev_disable_notifiers(&vub->vhost_dev, vdev); +} + +static void vub_set_status(VirtIODevice *vdev, uint8_t status) +{ + VHostUserBase *vub = VHOST_USER_BASE(vdev); + bool should_start = virtio_device_should_start(vdev, status); + + if (vhost_dev_is_started(&vub->vhost_dev) == should_start) { + return; + } + + if (should_start) { + vub_start(vdev); + } else { + vub_stop(vdev); + } +} + +/* + * For an implementation where everything is delegated to the backend + * we don't do anything other than return the full feature set offered + * by the daemon (module the reserved feature bit). + */ +static uint64_t vub_get_features(VirtIODevice *vdev, + uint64_t requested_features, Error **errp) +{ + VHostUserBase *vub = VHOST_USER_BASE(vdev); + /* This should be set when the vhost connection initialises */ + g_assert(vub->vhost_dev.features); + return vub->vhost_dev.features & ~(1ULL << VHOST_USER_F_PROTOCOL_FEATURES); +} + +static void vub_handle_output(VirtIODevice *vdev, VirtQueue *vq) +{ + /* + * Not normally called; it's the daemon that handles the queue; + * however virtio's cleanup path can call this. + */ +} + +static void do_vhost_user_cleanup(VirtIODevice *vdev, VHostUserBase *vub) +{ + vhost_user_cleanup(&vub->vhost_user); + + for (int i = 0; i < vub->num_vqs; i++) { + VirtQueue *vq = g_ptr_array_index(vub->vqs, i); + virtio_delete_queue(vq); + } + + virtio_cleanup(vdev); +} + +static int vub_connect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBase *vub = VHOST_USER_BASE(vdev); + + if (vub->connected) { + return 0; + } + vub->connected = true; + + /* restore vhost state */ + if (virtio_device_started(vdev, vdev->status)) { + vub_start(vdev); + } + + return 0; +} + +static void vub_disconnect(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBase *vub = VHOST_USER_BASE(vdev); + + if (!vub->connected) { + return; + } + vub->connected = false; + + if (vhost_dev_is_started(&vub->vhost_dev)) { + vub_stop(vdev); + } +} + +static void vub_event(void *opaque, QEMUChrEvent event) +{ + DeviceState *dev = opaque; + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBase *vub = VHOST_USER_BASE(vdev); + + switch (event) { + case CHR_EVENT_OPENED: + if (vub_connect(dev) < 0) { + qemu_chr_fe_disconnect(&vub->chardev); + return; + } + break; + case CHR_EVENT_CLOSED: + vub_disconnect(dev); + break; + case CHR_EVENT_BREAK: + case CHR_EVENT_MUX_IN: + case CHR_EVENT_MUX_OUT: + /* Ignore */ + break; + } +} + +static void vub_device_realize(DeviceState *dev, Error **errp) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBase *vub = VHOST_USER_BASE(dev); + int ret; + + if (!vub->chardev.chr) { + error_setg(errp, "vhost-user-device: missing chardev"); + return; + } + + if (!vub->virtio_id) { + error_setg(errp, "vhost-user-device: need to define device id"); + return; + } + + if (!vub->num_vqs) { + vub->num_vqs = 1; /* reasonable default? */ + } + + if (!vhost_user_init(&vub->vhost_user, &vub->chardev, errp)) { + return; + } + + virtio_init(vdev, vub->virtio_id, 0); + + /* + * Disable guest notifiers, by default all notifications will be via the + * asynchronous vhost-user socket. + */ + vdev->use_guest_notifier_mask = false; + + /* Allocate queues */ + vub->vqs = g_ptr_array_sized_new(vub->num_vqs); + for (int i = 0; i < vub->num_vqs; i++) { + g_ptr_array_add(vub->vqs, + virtio_add_queue(vdev, 4, vub_handle_output)); + } + + vub->vhost_dev.nvqs = vub->num_vqs; + vub->vhost_dev.vqs = g_new0(struct vhost_virtqueue, vub->vhost_dev.nvqs); + + /* connect to backend */ + ret = vhost_dev_init(&vub->vhost_dev, &vub->vhost_user, + VHOST_BACKEND_TYPE_USER, 0, errp); + + if (ret < 0) { + do_vhost_user_cleanup(vdev, vub); + } + + qemu_chr_fe_set_handlers(&vub->chardev, NULL, NULL, vub_event, NULL, + dev, NULL, true); +} + +static void vub_device_unrealize(DeviceState *dev) +{ + VirtIODevice *vdev = VIRTIO_DEVICE(dev); + VHostUserBase *vub = VHOST_USER_BASE(dev); + struct vhost_virtqueue *vhost_vqs = vub->vhost_dev.vqs; + + /* This will stop vhost backend if appropriate. */ + vub_set_status(vdev, 0); + vhost_dev_cleanup(&vub->vhost_dev); + g_free(vhost_vqs); + do_vhost_user_cleanup(vdev, vub); +} + +static void vub_class_init(ObjectClass *klass, void *data) +{ + VirtioDeviceClass *vdc = VIRTIO_DEVICE_CLASS(klass); + + vdc->realize = vub_device_realize; + vdc->unrealize = vub_device_unrealize; + vdc->get_features = vub_get_features; + vdc->set_status = vub_set_status; +} + +static const TypeInfo vub_info = { + .name = TYPE_VHOST_USER_BASE, + .parent = TYPE_VIRTIO_DEVICE, + .instance_size = sizeof(VHostUserBase), + .class_init = vub_class_init, + .class_size = sizeof(VHostUserBaseClass), + .abstract = true +}; + + +/* + * The following is a concrete implementation of the base class which + * allows the user to define the key parameters via the command line. + */ + +static const VMStateDescription vud_vmstate = { + .name = "vhost-user-device", + .unmigratable = 1, +}; + +static Property vud_properties[] = { + DEFINE_PROP_CHR("chardev", VHostUserBase, chardev), + DEFINE_PROP_UINT16("virtio-id", VHostUserBase, virtio_id, 0), + DEFINE_PROP_UINT32("num_vqs", VHostUserBase, num_vqs, 1), + DEFINE_PROP_END_OF_LIST(), +}; + +static void vud_class_init(ObjectClass *klass, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + + device_class_set_props(dc, vud_properties); + dc->vmsd = &vud_vmstate; + set_bit(DEVICE_CATEGORY_INPUT, dc->categories); +} + +static const TypeInfo vud_info = { + .name = TYPE_VHOST_USER_DEVICE, + .parent = TYPE_VHOST_USER_BASE, + .instance_size = sizeof(VHostUserBase), + .class_init = vud_class_init, + .class_size = sizeof(VHostUserBaseClass), +}; + +static void vu_register_types(void) +{ + type_register_static(&vub_info); + type_register_static(&vud_info); +} + +type_init(vu_register_types) diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build index bdec78bfc6..43e5fa3f7d 100644 --- a/hw/virtio/meson.build +++ b/hw/virtio/meson.build @@ -10,7 +10,9 @@ specific_virtio_ss.add(files('virtio-config-io.c', 'virtio-qmp.c')) if have_vhost specific_virtio_ss.add(files('vhost.c', 'vhost-backend.c', 'vhost-iova-tree.c')) if have_vhost_user + # fixme - this really should be generic specific_virtio_ss.add(files('vhost-user.c')) + softmmu_virtio_ss.add(files('vhost-user-device.c')) endif if have_vhost_vdpa specific_virtio_ss.add(files('vhost-vdpa.c', 'vhost-shadow-virtqueue.c'))