From patchwork Mon May 16 10:39:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 572974 Delivered-To: patch@linaro.org Received: by 2002:a05:7000:1590:0:0:0:0 with SMTP id w16csp989823map; Mon, 16 May 2022 04:55:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz49+dSP+IbPcVjc6X54vauJaoAzE93MQ0/Tq9sswN1NrtisLyYvK1a70xOLgPzY3YcvgMn X-Received: by 2002:ad4:5e8e:0:b0:45b:477:892f with SMTP id jl14-20020ad45e8e000000b0045b0477892fmr14699563qvb.124.1652702125962; Mon, 16 May 2022 04:55:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1652702125; cv=none; d=google.com; s=arc-20160816; b=Zl0f3zF7frBf0Lyn/KstJ5ODeVJx7Rc1aqGpqUcB1zamNHMxIft6ijcg/FzG35IQXT njSor/q9wpR97YayzCvY+5zB1+miKjcmTP/6h0nyqfq7l23+vD7xdXdrI3FZ+YybklXE hAOjmyuCavDechHTT9TKTmZmDzg45FNYucSHR8lWXU04Hhj03O6e/o5o+s4W4rwEF0u7 LllRNGy6+OQIYlYzlEeXjI/qD9q5BeyxcSTY3anNFDZ1EBW+YcIjVG8S+nXXOEHtGtxG dIRD5ix3eXRnH2CuxSEXYudSqM2fY6ZmzvaCGvBiG7AiWUsHCeeTGU5kpFJ9jut120QG YUkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:in-reply-to :content-transfer-encoding:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=c9w3iT2kzWZXwmqWDUx+rSUtwr5cBul7vws3lLdpQ2Y=; b=V70FUX2NrNZsB9YTJc26IRoNQLd3oShNkPZsii2XqnhAklo+R7/GQzsUkyxO4ETfGI tctOzk9C2itrBIZMs80AXyPD36Vogsd1VlwePSCafnWYFnD5Tf1Uyt8vMP9h0795y5yF O3FRuchsrX8iTL6vOICWgqxq+pCYBS1lwn6N3QhTOyJ9tCzFmpigIgRsmvfA2esORM+K GeLdup7yRPviabyxq0+N6d/E8kCsjUQbzbTnwCg+mjEgD+T2slHJn1hSOwfalMI2V/fZ LPaKpEqUvL4TSnfLElEOWKYc+R1pxWn3kGa3n/3uELI2LbmiGTUBkfuSy2KfKUBLUnxW 5uLw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=NaW75Vnu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 14-20020a056214202e00b00456446ea6b3si4084874qvf.367.2022.05.16.04.55.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 16 May 2022 04:55:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=NaW75Vnu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from localhost ([::1]:50846 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nqZJt-0005N4-Gr for patch@linaro.org; Mon, 16 May 2022 07:55:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49278) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqY8V-0001Bg-R9 for qemu-devel@nongnu.org; Mon, 16 May 2022 06:39:35 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:31887) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nqY8T-0005do-H7 for qemu-devel@nongnu.org; Mon, 16 May 2022 06:39:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1652697572; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c9w3iT2kzWZXwmqWDUx+rSUtwr5cBul7vws3lLdpQ2Y=; b=NaW75VnuiKMHg9X3N6ospnQXYEbuCnDpGTZ910Ut2tIdte/DIeEXiYsaatfsexEnDtPExz nWZg/DqqlBSa+2IWkTfR8ZadrHqdxbX89d0Yw4HDUHY4ErKSq2v3FFG/EGn058V5ZLsFuA vBWXbJhQ1n8cn0yxRvR/bza0vdXtKz8= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-281-JKXEs6LsOBS3qv16EF2YHQ-1; Mon, 16 May 2022 06:39:31 -0400 X-MC-Unique: JKXEs6LsOBS3qv16EF2YHQ-1 Received: by mail-wr1-f72.google.com with SMTP id ba21-20020a0560001c1500b0020ca6a45dfcso3800190wrb.9 for ; Mon, 16 May 2022 03:39:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=c9w3iT2kzWZXwmqWDUx+rSUtwr5cBul7vws3lLdpQ2Y=; b=6YBYxiF1wYJ5gm5SwE3enBicMZ2ePesx9NvyQ/5oeF8/G9RkBU+FM8h5CdyX2V2Veu YjtU5VayNoA+/8p9euXlyedB4C1t6d2ubQMdy1CHsuVCtGPjYy5Wj6DNW3yg0bHurfHU nWb4dP5TuMOAUFiz4VsfnsDY4en+j8QhZdYfoM+jRTNOIkz7jXfeMqd06zuXWjzAH1He Ls6u29QKFIbdo+q8N6pR1QhZ4zvmObkCXwpngYjhTPdOaoijn97nIyb0rkm6o/MHhXse fq9rskYWd8q6zv3ESUqJTsAvGWGn45CUUYSmlhpgUD6pGYblrZWeStu8aX76HMlQwH0U X3mA== X-Gm-Message-State: AOAM531iGyq3SBIKALM+1pPPKvniNkWWrSjEDmd+nOCS1dgct06JdQyo t0tBR9nlPzkYbzWyhNHGaNsfv6EI0mWXFkuWBDh3MNfJuLkPE1OtdYUWW8yyGc4GxcGUVhU+2g4 kvKTdu8kFTyjotrhYLn+LeCwedIT/a29S7DDPbKiD+V7FVO7iy3MtZPQXlM46 X-Received: by 2002:a05:600c:4f52:b0:394:63eb:ad27 with SMTP id m18-20020a05600c4f5200b0039463ebad27mr16415575wmq.35.1652697570261; Mon, 16 May 2022 03:39:30 -0700 (PDT) X-Received: by 2002:a05:600c:4f52:b0:394:63eb:ad27 with SMTP id m18-20020a05600c4f5200b0039463ebad27mr16415542wmq.35.1652697569941; Mon, 16 May 2022 03:39:29 -0700 (PDT) Received: from redhat.com ([2.55.141.66]) by smtp.gmail.com with ESMTPSA id o16-20020adf8b90000000b0020c5253d8e0sm9600626wra.44.2022.05.16.03.39.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 May 2022 03:39:29 -0700 (PDT) Date: Mon, 16 May 2022 06:39:26 -0400 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Alex =?utf-8?q?Benn=C3=A9e?= , Stefan Hajnoczi , Gerd Hoffmann , =?utf-8?q?Marc-Andr=C3=A9?= Lureau , Viresh Kumar , Mathieu Poirier , "Dr . David Alan Gilbert" , Kashyap Chamarthy , Richard Henderson Subject: [PULL 66/91] docs/devel: start documenting writing VirtIO devices Message-ID: <20220516095448.507876-67-mst@redhat.com> References: <20220516095448.507876-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220516095448.507876-1-mst@redhat.com> X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -28 X-Spam_score: -2.9 X-Spam_bar: -- X-Spam_report: (-2.9 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Alex Bennée While writing my own VirtIO devices I've gotten confused with how things are structured and what sort of shared infrastructure there is. If we can document how everything is supposed to work we can then maybe start cleaning up inconsistencies in the code. Signed-off-by: Alex Bennée Cc: Stefan Hajnoczi Cc: "Michael S. Tsirkin" Cc: Gerd Hoffmann Cc: Marc-André Lureau Cc: Viresh Kumar Cc: Mathieu Poirier Cc: Dr. David Alan Gilbert Message-Id: <20220309164929.19395-1-alex.bennee@linaro.org> Message-Id: <20220321153037.3622127-10-alex.bennee@linaro.org> Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- docs/devel/index-internals.rst | 1 + docs/devel/virtio-backends.rst | 214 +++++++++++++++++++++++++++++++++ 2 files changed, 215 insertions(+) create mode 100644 docs/devel/virtio-backends.rst diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst index a50889c556..e1a93df263 100644 --- a/docs/devel/index-internals.rst +++ b/docs/devel/index-internals.rst @@ -18,3 +18,4 @@ Details about QEMU's various subsystems including how to add features to them. tracing vfio-migration writing-monitor-commands + virtio-backends diff --git a/docs/devel/virtio-backends.rst b/docs/devel/virtio-backends.rst new file mode 100644 index 0000000000..9ff092e7a0 --- /dev/null +++ b/docs/devel/virtio-backends.rst @@ -0,0 +1,214 @@ +.. + Copyright (c) 2022, Linaro Limited + Written by Alex Bennée + +Writing VirtIO backends for QEMU +================================ + +This document attempts to outline the information a developer needs to +know to write device emulations in QEMU. It is specifically focused on +implementing VirtIO devices. For VirtIO the frontend is the driver +running on the guest. The backend is the everything that QEMU needs to +do to handle the emulation of the VirtIO device. This can be done +entirely in QEMU, divided between QEMU and the kernel (vhost) or +handled by a separate process which is configured by QEMU +(vhost-user). + +VirtIO Transports +----------------- + +VirtIO supports a number of different transports. While the details of +the configuration and operation of the device will generally be the +same QEMU represents them as different devices depending on the +transport they use. For example -device virtio-foo represents the foo +device using mmio and -device virtio-foo-pci is the same class of +device using the PCI transport. + +Using the QEMU Object Model (QOM) +--------------------------------- + +Generally all devices in QEMU are super classes of ``TYPE_DEVICE`` +however VirtIO devices should be based on ``TYPE_VIRTIO_DEVICE`` which +itself is derived from the base class. For example: + +.. code:: c + + static const TypeInfo virtio_blk_info = { + .name = TYPE_VIRTIO_BLK, + .parent = TYPE_VIRTIO_DEVICE, + .instance_size = sizeof(VirtIOBlock), + .instance_init = virtio_blk_instance_init, + .class_init = virtio_blk_class_init, + }; + +The author may decide to have a more expansive class hierarchy to +support multiple device types. For example the Virtio GPU device: + +.. code:: c + + static const TypeInfo virtio_gpu_base_info = { + .name = TYPE_VIRTIO_GPU_BASE, + .parent = TYPE_VIRTIO_DEVICE, + .instance_size = sizeof(VirtIOGPUBase), + .class_size = sizeof(VirtIOGPUBaseClass), + .class_init = virtio_gpu_base_class_init, + .abstract = true + }; + + static const TypeInfo vhost_user_gpu_info = { + .name = TYPE_VHOST_USER_GPU, + .parent = TYPE_VIRTIO_GPU_BASE, + .instance_size = sizeof(VhostUserGPU), + .instance_init = vhost_user_gpu_instance_init, + .instance_finalize = vhost_user_gpu_instance_finalize, + .class_init = vhost_user_gpu_class_init, + }; + + static const TypeInfo virtio_gpu_info = { + .name = TYPE_VIRTIO_GPU, + .parent = TYPE_VIRTIO_GPU_BASE, + .instance_size = sizeof(VirtIOGPU), + .class_size = sizeof(VirtIOGPUClass), + .class_init = virtio_gpu_class_init, + }; + +defines a base class for the VirtIO GPU and then specialises two +versions, one for the internal implementation and the other for the +vhost-user version. + +VirtIOPCIProxy +^^^^^^^^^^^^^^ + +[AJB: the following is supposition and welcomes more informed +opinions] + +Probably due to legacy from the pre-QOM days PCI VirtIO devices don't +follow the normal hierarchy. Instead the a standalone object is based +on the VirtIOPCIProxy class and the specific VirtIO instance is +manually instantiated: + +.. code:: c + + /* + * virtio-blk-pci: This extends VirtioPCIProxy. + */ + #define TYPE_VIRTIO_BLK_PCI "virtio-blk-pci-base" + DECLARE_INSTANCE_CHECKER(VirtIOBlkPCI, VIRTIO_BLK_PCI, + TYPE_VIRTIO_BLK_PCI) + + struct VirtIOBlkPCI { + VirtIOPCIProxy parent_obj; + VirtIOBlock vdev; + }; + + static Property virtio_blk_pci_properties[] = { + DEFINE_PROP_UINT32("class", VirtIOPCIProxy, class_code, 0), + DEFINE_PROP_BIT("ioeventfd", VirtIOPCIProxy, flags, + VIRTIO_PCI_FLAG_USE_IOEVENTFD_BIT, true), + DEFINE_PROP_UINT32("vectors", VirtIOPCIProxy, nvectors, + DEV_NVECTORS_UNSPECIFIED), + DEFINE_PROP_END_OF_LIST(), + }; + + static void virtio_blk_pci_realize(VirtIOPCIProxy *vpci_dev, Error **errp) + { + VirtIOBlkPCI *dev = VIRTIO_BLK_PCI(vpci_dev); + DeviceState *vdev = DEVICE(&dev->vdev); + + ... + + qdev_realize(vdev, BUS(&vpci_dev->bus), errp); + } + + static void virtio_blk_pci_class_init(ObjectClass *klass, void *data) + { + DeviceClass *dc = DEVICE_CLASS(klass); + VirtioPCIClass *k = VIRTIO_PCI_CLASS(klass); + PCIDeviceClass *pcidev_k = PCI_DEVICE_CLASS(klass); + + set_bit(DEVICE_CATEGORY_STORAGE, dc->categories); + device_class_set_props(dc, virtio_blk_pci_properties); + k->realize = virtio_blk_pci_realize; + pcidev_k->vendor_id = PCI_VENDOR_ID_REDHAT_QUMRANET; + pcidev_k->device_id = PCI_DEVICE_ID_VIRTIO_BLOCK; + pcidev_k->revision = VIRTIO_PCI_ABI_VERSION; + pcidev_k->class_id = PCI_CLASS_STORAGE_SCSI; + } + + static void virtio_blk_pci_instance_init(Object *obj) + { + VirtIOBlkPCI *dev = VIRTIO_BLK_PCI(obj); + + virtio_instance_init_common(obj, &dev->vdev, sizeof(dev->vdev), + TYPE_VIRTIO_BLK); + object_property_add_alias(obj, "bootindex", OBJECT(&dev->vdev), + "bootindex"); + } + + static const VirtioPCIDeviceTypeInfo virtio_blk_pci_info = { + .base_name = TYPE_VIRTIO_BLK_PCI, + .generic_name = "virtio-blk-pci", + .transitional_name = "virtio-blk-pci-transitional", + .non_transitional_name = "virtio-blk-pci-non-transitional", + .instance_size = sizeof(VirtIOBlkPCI), + .instance_init = virtio_blk_pci_instance_init, + .class_init = virtio_blk_pci_class_init, + }; + +Here you can see the instance_init has to manually instantiate the +underlying ``TYPE_VIRTIO_BLOCK`` object and link an alias for one of +it's properties to the PCI device. + + +Back End Implementations +------------------------ + +There are a number of places where the implementation of the backend +can be done: + +* in QEMU itself +* in the host kernel (a.k.a vhost) +* in a separate process (a.k.a. vhost-user) + +vhost_ops vs TYPE_VHOST_USER_BACKEND +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +There are two choices to how to implement vhost code. Most of the code +which has to work with either vhost or vhost-user uses +``vhost_dev_init()`` to instantiate the appropriate backend. This +means including a ``struct vhost_dev`` in the main object structure. + +For vhost-user devices you also need to add code to track the +initialisation of the ``chardev`` device used for the control socket +between QEMU and the external vhost-user process. + +If you only need to implement a vhost-user backed the other option is +a use a QOM-ified version of vhost-user. + +.. code:: c + + static void + vhost_user_gpu_instance_init(Object *obj) + { + VhostUserGPU *g = VHOST_USER_GPU(obj); + + g->vhost = VHOST_USER_BACKEND(object_new(TYPE_VHOST_USER_BACKEND)); + object_property_add_alias(obj, "chardev", + OBJECT(g->vhost), "chardev"); + } + + static const TypeInfo vhost_user_gpu_info = { + .name = TYPE_VHOST_USER_GPU, + .parent = TYPE_VIRTIO_GPU_BASE, + .instance_size = sizeof(VhostUserGPU), + .instance_init = vhost_user_gpu_instance_init, + .instance_finalize = vhost_user_gpu_instance_finalize, + .class_init = vhost_user_gpu_class_init, + }; + +Using it this way entails adding a ``struct VhostUserBackend`` to your +core object structure and manually instantiating the backend. This +sub-structure tracks both the ``vhost_dev`` and ``CharDev`` types +needed for the connection. Instead of calling ``vhost_dev_init`` you +would call ``vhost_user_backend_dev_init`` which does what is needed +on your behalf.