From patchwork Fri Jun 19 18:22:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 217520 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AE3FC433E0 for ; Fri, 19 Jun 2020 18:23:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4C6F92100A for ; Fri, 19 Jun 2020 18:23:27 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UYDDyQBz" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2395032AbgFSSXZ (ORCPT ); Fri, 19 Jun 2020 14:23:25 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:30900 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2389061AbgFSSXX (ORCPT ); Fri, 19 Jun 2020 14:23:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592591000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+CtPUIMGWcoYx3GArUVC0wAi/+1VQx1I1wXsVmTNvLE=; b=UYDDyQBznlr5iDVLs+ZcW9PK+wpO9sXrJZWEkPlhld1k3jbuNqqz+Me7V1MVUJON90mxMm 4OAW5QscqRzA3Yoi87YZ7PsPgcDdTLOA3/VcoAcq6zC2NsLvVq/Xkw1W6GOxHPbhU4kMn2 O60M3sMT+JvIcbiCZjC/riUjF8Z9rCk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-425-U4G3TYIWMoqQwdFgwQ0Ivw-1; Fri, 19 Jun 2020 14:23:16 -0400 X-MC-Unique: U4G3TYIWMoqQwdFgwQ0Ivw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3AB27107ACCA; Fri, 19 Jun 2020 18:23:15 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-14.ams2.redhat.com [10.36.113.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 190487C1E2; Fri, 19 Jun 2020 18:23:12 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: mst@redhat.com Cc: kvm list , Stefano Garzarella , virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Hajnoczi , Jason Wang Subject: [RFC v9 01/11] vhost: option to fetch descriptors through an independent struct Date: Fri, 19 Jun 2020 20:22:52 +0200 Message-Id: <20200619182302.850-2-eperezma@redhat.com> In-Reply-To: <20200619182302.850-1-eperezma@redhat.com> References: <20200619182302.850-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Michael S. Tsirkin" The idea is to support multiple ring formats by converting to a format-independent array of descriptors. This costs extra cycles, but we gain in ability to fetch a batch of descriptors in one go, which is good for code cache locality. When used, this causes a minor performance degradation, it's been kept as simple as possible for ease of review. A follow-up patch gets us back the performance by adding batching. To simplify benchmarking, I kept the old code around so one can switch back and forth between old and new code. This will go away in the final submission. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez Link: https://lore.kernel.org/r/20200401183118.8334-2-eperezma@redhat.com Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez --- drivers/vhost/vhost.c | 305 +++++++++++++++++++++++++++++++++++++++++- drivers/vhost/vhost.h | 16 +++ 2 files changed, 320 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 062595ee1f83..2d784681b0fa 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -303,6 +303,7 @@ static void vhost_vq_reset(struct vhost_dev *dev, struct vhost_virtqueue *vq) { vq->num = 1; + vq->ndescs = 0; vq->desc = NULL; vq->avail = NULL; vq->used = NULL; @@ -373,6 +374,9 @@ static int vhost_worker(void *data) static void vhost_vq_free_iovecs(struct vhost_virtqueue *vq) { + kfree(vq->descs); + vq->descs = NULL; + vq->max_descs = 0; kfree(vq->indirect); vq->indirect = NULL; kfree(vq->log); @@ -389,6 +393,10 @@ static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) for (i = 0; i < dev->nvqs; ++i) { vq = dev->vqs[i]; + vq->max_descs = dev->iov_limit; + vq->descs = kmalloc_array(vq->max_descs, + sizeof(*vq->descs), + GFP_KERNEL); vq->indirect = kmalloc_array(UIO_MAXIOV, sizeof(*vq->indirect), GFP_KERNEL); @@ -396,7 +404,7 @@ static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) GFP_KERNEL); vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads), GFP_KERNEL); - if (!vq->indirect || !vq->log || !vq->heads) + if (!vq->indirect || !vq->log || !vq->heads || !vq->descs) goto err_nomem; } return 0; @@ -488,6 +496,8 @@ void vhost_dev_init(struct vhost_dev *dev, for (i = 0; i < dev->nvqs; ++i) { vq = dev->vqs[i]; + vq->descs = NULL; + vq->max_descs = 0; vq->log = NULL; vq->indirect = NULL; vq->heads = NULL; @@ -2314,6 +2324,299 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, } EXPORT_SYMBOL_GPL(vhost_get_vq_desc); +static struct vhost_desc *peek_split_desc(struct vhost_virtqueue *vq) +{ + BUG_ON(!vq->ndescs); + return &vq->descs[vq->ndescs - 1]; +} + +static void pop_split_desc(struct vhost_virtqueue *vq) +{ + BUG_ON(!vq->ndescs); + --vq->ndescs; +} + +#define VHOST_DESC_FLAGS (VRING_DESC_F_INDIRECT | VRING_DESC_F_WRITE | \ + VRING_DESC_F_NEXT) +static int push_split_desc(struct vhost_virtqueue *vq, struct vring_desc *desc, u16 id) +{ + struct vhost_desc *h; + + if (unlikely(vq->ndescs >= vq->max_descs)) + return -EINVAL; + h = &vq->descs[vq->ndescs++]; + h->addr = vhost64_to_cpu(vq, desc->addr); + h->len = vhost32_to_cpu(vq, desc->len); + h->flags = vhost16_to_cpu(vq, desc->flags) & VHOST_DESC_FLAGS; + h->id = id; + + return 0; +} + +static int fetch_indirect_descs(struct vhost_virtqueue *vq, + struct vhost_desc *indirect, + u16 head) +{ + struct vring_desc desc; + unsigned int i = 0, count, found = 0; + u32 len = indirect->len; + struct iov_iter from; + int ret; + + /* Sanity check */ + if (unlikely(len % sizeof desc)) { + vq_err(vq, "Invalid length in indirect descriptor: " + "len 0x%llx not multiple of 0x%zx\n", + (unsigned long long)len, + sizeof desc); + return -EINVAL; + } + + ret = translate_desc(vq, indirect->addr, len, vq->indirect, + UIO_MAXIOV, VHOST_ACCESS_RO); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Translation failure %d in indirect.\n", ret); + return ret; + } + iov_iter_init(&from, READ, vq->indirect, ret, len); + + /* We will use the result as an address to read from, so most + * architectures only need a compiler barrier here. */ + read_barrier_depends(); + + count = len / sizeof desc; + /* Buffers are chained via a 16 bit next field, so + * we can have at most 2^16 of these. */ + if (unlikely(count > USHRT_MAX + 1)) { + vq_err(vq, "Indirect buffer length too big: %d\n", + indirect->len); + return -E2BIG; + } + if (unlikely(vq->ndescs + count > vq->max_descs)) { + vq_err(vq, "Too many indirect + direct descs: %d + %d\n", + vq->ndescs, indirect->len); + return -E2BIG; + } + + do { + if (unlikely(++found > count)) { + vq_err(vq, "Loop detected: last one at %u " + "indirect size %u\n", + i, count); + return -EINVAL; + } + if (unlikely(!copy_from_iter_full(&desc, sizeof(desc), &from))) { + vq_err(vq, "Failed indirect descriptor: idx %d, %zx\n", + i, (size_t)indirect->addr + i * sizeof desc); + return -EINVAL; + } + if (unlikely(desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_INDIRECT))) { + vq_err(vq, "Nested indirect descriptor: idx %d, %zx\n", + i, (size_t)indirect->addr + i * sizeof desc); + return -EINVAL; + } + + /* Note: push_split_desc can't fail here: + * we never fetch unless there's space. */ + ret = push_split_desc(vq, &desc, head); + WARN_ON(ret); + } while ((i = next_desc(vq, &desc)) != -1); + return 0; +} + +/* This function returns a value > 0 if a descriptor was found, or 0 if none were found. + * A negative code is returned on error. */ +static int fetch_descs(struct vhost_virtqueue *vq) +{ + unsigned int i, head, found = 0; + struct vhost_desc *last; + struct vring_desc desc; + __virtio16 avail_idx; + __virtio16 ring_head; + u16 last_avail_idx; + int ret; + + /* Check it isn't doing very strange things with descriptor numbers. */ + last_avail_idx = vq->last_avail_idx; + + if (vq->avail_idx == vq->last_avail_idx) { + if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { + vq_err(vq, "Failed to access avail idx at %p\n", + &vq->avail->idx); + return -EFAULT; + } + vq->avail_idx = vhost16_to_cpu(vq, avail_idx); + + if (unlikely((u16)(vq->avail_idx - last_avail_idx) > vq->num)) { + vq_err(vq, "Guest moved used index from %u to %u", + last_avail_idx, vq->avail_idx); + return -EFAULT; + } + + /* If there's nothing new since last we looked, return + * invalid. + */ + if (vq->avail_idx == last_avail_idx) + return 0; + + /* Only get avail ring entries after they have been + * exposed by guest. + */ + smp_rmb(); + } + + /* Grab the next descriptor number they're advertising */ + if (unlikely(vhost_get_avail_head(vq, &ring_head, last_avail_idx))) { + vq_err(vq, "Failed to read head: idx %d address %p\n", + last_avail_idx, + &vq->avail->ring[last_avail_idx % vq->num]); + return -EFAULT; + } + + head = vhost16_to_cpu(vq, ring_head); + + /* If their number is silly, that's an error. */ + if (unlikely(head >= vq->num)) { + vq_err(vq, "Guest says index %u > %u is available", + head, vq->num); + return -EINVAL; + } + + i = head; + do { + if (unlikely(i >= vq->num)) { + vq_err(vq, "Desc index is %u > %u, head = %u", + i, vq->num, head); + return -EINVAL; + } + if (unlikely(++found > vq->num)) { + vq_err(vq, "Loop detected: last one at %u " + "vq size %u head %u\n", + i, vq->num, head); + return -EINVAL; + } + ret = vhost_get_desc(vq, &desc, i); + if (unlikely(ret)) { + vq_err(vq, "Failed to get descriptor: idx %d addr %p\n", + i, vq->desc + i); + return -EFAULT; + } + ret = push_split_desc(vq, &desc, head); + if (unlikely(ret)) { + vq_err(vq, "Failed to save descriptor: idx %d\n", i); + return -EINVAL; + } + } while ((i = next_desc(vq, &desc)) != -1); + + last = peek_split_desc(vq); + if (unlikely(last->flags & VRING_DESC_F_INDIRECT)) { + pop_split_desc(vq); + ret = fetch_indirect_descs(vq, last, head); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Failure detected " + "in indirect descriptor at idx %d\n", head); + return ret; + } + } + + /* Assume notifications from guest are disabled at this point, + * if they aren't we would need to update avail_event index. */ + BUG_ON(!(vq->used_flags & VRING_USED_F_NO_NOTIFY)); + + /* On success, increment avail index. */ + vq->last_avail_idx++; + + return 1; +} + +/* This looks in the virtqueue and for the first available buffer, and converts + * it to an iovec for convenient access. Since descriptors consist of some + * number of output then some number of input descriptors, it's actually two + * iovecs, but we pack them into one and note how many of each there were. + * + * This function returns the descriptor number found, or vq->num (which is + * never a valid descriptor number) if none was found. A negative code is + * returned on error. */ +int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, + struct iovec iov[], unsigned int iov_size, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num) +{ + int ret = fetch_descs(vq); + int i; + + if (ret <= 0) + goto err_fetch; + + /* Now convert to IOV */ + /* When we start there are none of either input nor output. */ + *out_num = *in_num = 0; + if (unlikely(log)) + *log_num = 0; + + for (i = 0; i < vq->ndescs; ++i) { + unsigned iov_count = *in_num + *out_num; + struct vhost_desc *desc = &vq->descs[i]; + int access; + + if (desc->flags & ~VHOST_DESC_FLAGS) { + vq_err(vq, "Unexpected flags: 0x%x at descriptor id 0x%x\n", + desc->flags, desc->id); + ret = -EINVAL; + goto err; + } + if (desc->flags & VRING_DESC_F_WRITE) + access = VHOST_ACCESS_WO; + else + access = VHOST_ACCESS_RO; + ret = translate_desc(vq, desc->addr, + desc->len, iov + iov_count, + iov_size - iov_count, access); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Translation failure %d descriptor idx %d\n", + ret, i); + goto err; + } + if (access == VHOST_ACCESS_WO) { + /* If this is an input descriptor, + * increment that count. */ + *in_num += ret; + if (unlikely(log && ret)) { + log[*log_num].addr = desc->addr; + log[*log_num].len = desc->len; + ++*log_num; + } + } else { + /* If it's an output descriptor, they're all supposed + * to come before any input descriptors. */ + if (unlikely(*in_num)) { + vq_err(vq, "Descriptor has out after in: " + "idx %d\n", i); + ret = -EINVAL; + goto err; + } + *out_num += ret; + } + + ret = desc->id; + } + + vq->ndescs = 0; + + return ret; + +err: + vhost_discard_vq_desc(vq, 1); +err_fetch: + vq->ndescs = 0; + + return ret ? ret : vq->num; +} +EXPORT_SYMBOL_GPL(vhost_get_vq_desc_batch); + /* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) { diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index c8e96a095d3b..87089d51490d 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -60,6 +60,13 @@ enum vhost_uaddr_type { VHOST_NUM_ADDRS = 3, }; +struct vhost_desc { + u64 addr; + u32 len; + u16 flags; /* VRING_DESC_F_WRITE, VRING_DESC_F_NEXT */ + u16 id; +}; + /* The virtqueue structure describes a queue attached to a device. */ struct vhost_virtqueue { struct vhost_dev *dev; @@ -71,6 +78,11 @@ struct vhost_virtqueue { vring_avail_t __user *avail; vring_used_t __user *used; const struct vhost_iotlb_map *meta_iotlb[VHOST_NUM_ADDRS]; + + struct vhost_desc *descs; + int ndescs; + int max_descs; + struct file *kick; struct eventfd_ctx *call_ctx; struct eventfd_ctx *error_ctx; @@ -177,6 +189,10 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg bool vhost_vq_access_ok(struct vhost_virtqueue *vq); bool vhost_log_access_ok(struct vhost_dev *); +int vhost_get_vq_desc_batch(struct vhost_virtqueue *, + struct iovec iov[], unsigned int iov_count, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num); int vhost_get_vq_desc(struct vhost_virtqueue *, struct iovec iov[], unsigned int iov_count, unsigned int *out_num, unsigned int *in_num, From patchwork Fri Jun 19 18:22:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 217515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 098F6C433E2 for ; Fri, 19 Jun 2020 18:25:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D9F3320757 for ; Fri, 19 Jun 2020 18:25:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="itC13VNr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390290AbgFSSZJ (ORCPT ); Fri, 19 Jun 2020 14:25:09 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:47857 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2392431AbgFSSXZ (ORCPT ); Fri, 19 Jun 2020 14:23:25 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592591003; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jyGq6komk0fYvUmbv+TQR+iV83L9rQMuuJZlKEm5NNc=; b=itC13VNrqgupDPOmR+U2+TE7VbTLyvpTRzM5V1F7frnzHzGYowub1tcdDVwDNdZEbvpjuj e1O+pZYJuZa9oz2I2HwNPMvalifi54gEQYQRMj/Y3rukFLfNZvZ7GObjidGL0jS6cjSHXR 5bleDTrF8DFr6hZwxDufFesqGMvgt1s= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-299-kXtmFMfGNluBxE8Al5UWBw-1; Fri, 19 Jun 2020 14:23:21 -0400 X-MC-Unique: kXtmFMfGNluBxE8Al5UWBw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0DE79835B8B; Fri, 19 Jun 2020 18:23:20 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-14.ams2.redhat.com [10.36.113.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1C2107C1E2; Fri, 19 Jun 2020 18:23:17 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: mst@redhat.com Cc: kvm list , Stefano Garzarella , virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Hajnoczi , Jason Wang Subject: [RFC v9 03/11] vhost/net: pass net specific struct pointer Date: Fri, 19 Jun 2020 20:22:54 +0200 Message-Id: <20200619182302.850-4-eperezma@redhat.com> In-Reply-To: <20200619182302.850-1-eperezma@redhat.com> References: <20200619182302.850-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Michael S. Tsirkin" In preparation for further cleanup, pass net specific pointer to ubuf callbacks so we can move net specific fields out to net structures. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez --- drivers/vhost/net.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index e992decfec53..81900ce673b4 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -94,7 +94,7 @@ struct vhost_net_ubuf_ref { */ atomic_t refcount; wait_queue_head_t wait; - struct vhost_virtqueue *vq; + struct vhost_net_virtqueue *nvq; }; #define VHOST_NET_BATCH 64 @@ -231,7 +231,7 @@ static void vhost_net_enable_zcopy(int vq) } static struct vhost_net_ubuf_ref * -vhost_net_ubuf_alloc(struct vhost_virtqueue *vq, bool zcopy) +vhost_net_ubuf_alloc(struct vhost_net_virtqueue *nvq, bool zcopy) { struct vhost_net_ubuf_ref *ubufs; /* No zero copy backend? Nothing to count. */ @@ -242,7 +242,7 @@ vhost_net_ubuf_alloc(struct vhost_virtqueue *vq, bool zcopy) return ERR_PTR(-ENOMEM); atomic_set(&ubufs->refcount, 1); init_waitqueue_head(&ubufs->wait); - ubufs->vq = vq; + ubufs->nvq = nvq; return ubufs; } @@ -384,13 +384,13 @@ static void vhost_zerocopy_signal_used(struct vhost_net *net, static void vhost_zerocopy_callback(struct ubuf_info *ubuf, bool success) { struct vhost_net_ubuf_ref *ubufs = ubuf->ctx; - struct vhost_virtqueue *vq = ubufs->vq; + struct vhost_net_virtqueue *nvq = ubufs->nvq; int cnt; rcu_read_lock_bh(); /* set len to mark this desc buffers done DMA */ - vq->heads[ubuf->desc].len = success ? + nvq->vq.heads[ubuf->desc].len = success ? VHOST_DMA_DONE_LEN : VHOST_DMA_FAILED_LEN; cnt = vhost_net_ubuf_put(ubufs); @@ -402,7 +402,7 @@ static void vhost_zerocopy_callback(struct ubuf_info *ubuf, bool success) * less than 10% of times). */ if (cnt <= 1 || !(cnt % 16)) - vhost_poll_queue(&vq->poll); + vhost_poll_queue(&nvq->vq.poll); rcu_read_unlock_bh(); } @@ -1526,7 +1526,7 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd) /* start polling new socket */ oldsock = vhost_vq_get_backend(vq); if (sock != oldsock) { - ubufs = vhost_net_ubuf_alloc(vq, + ubufs = vhost_net_ubuf_alloc(nvq, sock && vhost_sock_zcopy(sock)); if (IS_ERR(ubufs)) { r = PTR_ERR(ubufs); From patchwork Fri Jun 19 18:22:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 217519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5EF05C433E0 for ; Fri, 19 Jun 2020 18:23:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 42F7221527 for ; Fri, 19 Jun 2020 18:23:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HZQ86KNE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405284AbgFSSXf (ORCPT ); Fri, 19 Jun 2020 14:23:35 -0400 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:48198 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2395060AbgFSSX0 (ORCPT ); Fri, 19 Jun 2020 14:23:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592591005; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TtSatW4fx1QmInWApX4FeSfqG2/RwS871fY7cs8kwpU=; b=HZQ86KNEc7Tta5DHcbsUA990sn/BXn4wSuMz8kbBI0q+sgG2sfULmYS2cLbESMGu9c8dsR NuN9s26NjoNuNxyFJJnLHGgcMBIIl1pQoCfKgK6TFiwDHl0vAhWJptRbynqR0egDRpPFMc aiqV0M32VMUxwXRKz2GUa7Yypg7X1vc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-48-NC5tv3sAPp2arDDP1jaJYg-1; Fri, 19 Jun 2020 14:23:23 -0400 X-MC-Unique: NC5tv3sAPp2arDDP1jaJYg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5AE79107ACCD; Fri, 19 Jun 2020 18:23:22 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-14.ams2.redhat.com [10.36.113.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 68F651C8; Fri, 19 Jun 2020 18:23:20 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: mst@redhat.com Cc: kvm list , Stefano Garzarella , virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Hajnoczi , Jason Wang Subject: [RFC v9 04/11] vhost: reorder functions Date: Fri, 19 Jun 2020 20:22:55 +0200 Message-Id: <20200619182302.850-5-eperezma@redhat.com> In-Reply-To: <20200619182302.850-1-eperezma@redhat.com> References: <20200619182302.850-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Michael S. Tsirkin" Reorder functions in the file to not rely on forward declarations, in preparation to making them static down the road. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez --- drivers/vhost/vhost.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 13021d6986eb..ba8da8785ede 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2424,19 +2424,6 @@ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) } EXPORT_SYMBOL_GPL(vhost_discard_vq_desc); -/* After we've used one of their buffers, we tell them about it. We'll then - * want to notify the guest, using eventfd. */ -int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) -{ - struct vring_used_elem heads = { - cpu_to_vhost32(vq, head), - cpu_to_vhost32(vq, len) - }; - - return vhost_add_used_n(vq, &heads, 1); -} -EXPORT_SYMBOL_GPL(vhost_add_used); - static int __vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, unsigned count) @@ -2506,6 +2493,19 @@ int vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, } EXPORT_SYMBOL_GPL(vhost_add_used_n); +/* After we've used one of their buffers, we tell them about it. We'll then + * want to notify the guest, using eventfd. */ +int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) +{ + struct vring_used_elem heads = { + cpu_to_vhost32(vq, head), + cpu_to_vhost32(vq, len) + }; + + return vhost_add_used_n(vq, &heads, 1); +} +EXPORT_SYMBOL_GPL(vhost_add_used); + static bool vhost_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq) { __u16 old, new; From patchwork Fri Jun 19 18:22:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 217516 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71B96C433DF for ; Fri, 19 Jun 2020 18:24:30 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4E0102100A for ; Fri, 19 Jun 2020 18:24:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NFFWikqx" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405484AbgFSSXo (ORCPT ); Fri, 19 Jun 2020 14:23:44 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:44668 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405392AbgFSSXj (ORCPT ); Fri, 19 Jun 2020 14:23:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592591018; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hizvuErrfe2IuRCAoJyyo6xCFe+4OpL2MM7w/xo1d5g=; b=NFFWikqxwoVHlXbGzwMMqrj7SIGT/HTLpqlCrMImpHngVLl5I5YfCCGr8sgG2r6oaNQz+x t4q5TeR7sHpLDjyphXp4NoE4YaGjYCbtqRBP8GW4Cex6PSxmtyWvaa/3aXaPo6PxaEvNy2 HSJdW4q/Y5FyJPRsIaqVyvEsn4LMR44= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-193-CYcofGs2NzCziALfYEt4aw-1; Fri, 19 Jun 2020 14:23:36 -0400 X-MC-Unique: CYcofGs2NzCziALfYEt4aw-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D5FD7107ACCA; Fri, 19 Jun 2020 18:23:34 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-14.ams2.redhat.com [10.36.113.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id E414C7C1E4; Fri, 19 Jun 2020 18:23:32 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: mst@redhat.com Cc: kvm list , Stefano Garzarella , virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Hajnoczi , Jason Wang Subject: [RFC v9 07/11] vhost/net: avoid iov length math Date: Fri, 19 Jun 2020 20:22:58 +0200 Message-Id: <20200619182302.850-8-eperezma@redhat.com> In-Reply-To: <20200619182302.850-1-eperezma@redhat.com> References: <20200619182302.850-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Michael S. Tsirkin" Now that API exposes buffer length, we no longer need to scan IOVs to figure it out. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez --- drivers/vhost/net.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c index 871214d8c64d..3356b249bf64 100644 --- a/drivers/vhost/net.c +++ b/drivers/vhost/net.c @@ -607,11 +607,9 @@ static bool vhost_exceeds_maxpend(struct vhost_net *net) } static size_t init_iov_iter(struct vhost_virtqueue *vq, struct iov_iter *iter, - size_t hdr_size, int out) + size_t len, size_t hdr_size, int out) { /* Skip header. TODO: support TSO. */ - size_t len = iov_length(vq->iov, out); - iov_iter_init(iter, WRITE, vq->iov, out, len); iov_iter_advance(iter, hdr_size); @@ -640,7 +638,7 @@ static int get_tx_bufs(struct vhost_net *net, } /* Sanity check */ - *len = init_iov_iter(vq, &msg->msg_iter, nvq->vhost_hlen, *out); + *len = init_iov_iter(vq, &msg->msg_iter, buf->out_len, nvq->vhost_hlen, *out); if (*len == 0) { vq_err(vq, "Unexpected header len for TX: %zd expected %zd\n", *len, nvq->vhost_hlen); @@ -1081,7 +1079,7 @@ static int get_rx_bufs(struct vhost_virtqueue *vq, nlogs += *log_num; log += *log_num; } - len = iov_length(vq->iov + seg, in); + len = bufs[bufcount].in_len; datalen -= len; ++bufcount; seg += in; From patchwork Fri Jun 19 18:22:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 217518 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2017C433E0 for ; Fri, 19 Jun 2020 18:23:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C234B2100A for ; Fri, 19 Jun 2020 18:23:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a51yvuZg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405995AbgFSSXt (ORCPT ); Fri, 19 Jun 2020 14:23:49 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:35045 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405379AbgFSSXm (ORCPT ); Fri, 19 Jun 2020 14:23:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592591021; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MQa85S+Gu9mVyqbk6eUj8fFtN+Dd+Dol2b2QZrFCNIg=; b=a51yvuZgVRwy93UXFg+jrP41K065x62hhcfvcIkb+RV8Yn2Wg/lrwbA0em1Kza1EeUdbFj cLNOdDj+Y3YFERuwQNfe6D4SN2rDhMc2JBOcrR+UukIufgPe6OjJU3HVMlaqZ0mUrjYrUQ wGyr8q8Rbo7//pFM+EOUHOpcM3jiiU4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-127-CiGvTY12PFKdDV5VNrlB1g-1; Fri, 19 Jun 2020 14:23:38 -0400 X-MC-Unique: CiGvTY12PFKdDV5VNrlB1g-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2F4AF8015CE; Fri, 19 Jun 2020 18:23:37 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-14.ams2.redhat.com [10.36.113.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3D11A1C8; Fri, 19 Jun 2020 18:23:35 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: mst@redhat.com Cc: kvm list , Stefano Garzarella , virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Hajnoczi , Jason Wang Subject: [RFC v9 08/11] vhost/test: convert to the buf API Date: Fri, 19 Jun 2020 20:22:59 +0200 Message-Id: <20200619182302.850-9-eperezma@redhat.com> In-Reply-To: <20200619182302.850-1-eperezma@redhat.com> References: <20200619182302.850-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Michael S. Tsirkin" Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez --- drivers/vhost/test.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c index 650e69261557..39e68f797a93 100644 --- a/drivers/vhost/test.c +++ b/drivers/vhost/test.c @@ -44,9 +44,10 @@ static void handle_vq(struct vhost_test *n) { struct vhost_virtqueue *vq = &n->vqs[VHOST_TEST_VQ]; unsigned out, in; - int head; + int ret; size_t len, total_len = 0; void *private; + struct vhost_buf buf; mutex_lock(&vq->mutex); private = vhost_vq_get_backend(vq); @@ -58,15 +59,15 @@ static void handle_vq(struct vhost_test *n) vhost_disable_notify(&n->dev, vq); for (;;) { - head = vhost_get_vq_desc(vq, vq->iov, - ARRAY_SIZE(vq->iov), - &out, &in, - NULL, NULL); + ret = vhost_get_avail_buf(vq, &buf, vq->iov, + ARRAY_SIZE(vq->iov), + &out, &in, + NULL, NULL); /* On error, stop handling until the next kick. */ - if (unlikely(head < 0)) + if (unlikely(ret < 0)) break; /* Nothing new? Wait for eventfd to tell us they refilled. */ - if (head == vq->num) { + if (!ret) { if (unlikely(vhost_enable_notify(&n->dev, vq))) { vhost_disable_notify(&n->dev, vq); continue; @@ -78,13 +79,14 @@ static void handle_vq(struct vhost_test *n) "out %d, int %d\n", out, in); break; } - len = iov_length(vq->iov, out); + len = buf.out_len; /* Sanity check */ if (!len) { vq_err(vq, "Unexpected 0 len for TX\n"); break; } - vhost_add_used_and_signal(&n->dev, vq, head, 0); + vhost_put_used_buf(vq, &buf); + vhost_signal(&n->dev, vq); total_len += len; if (unlikely(vhost_exceeds_weight(vq, 0, total_len))) break; From patchwork Fri Jun 19 18:23:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eugenio Perez Martin X-Patchwork-Id: 217517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D09A6C433E0 for ; Fri, 19 Jun 2020 18:24:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A92EC214D8 for ; Fri, 19 Jun 2020 18:24:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QN/uc4DV" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2436637AbgFSSYV (ORCPT ); Fri, 19 Jun 2020 14:24:21 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:47829 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405381AbgFSSXq (ORCPT ); Fri, 19 Jun 2020 14:23:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1592591024; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MDyMx/QYjv9YV91ubBW7aVwYK/qJjL8pnhsfMBIV3zE=; b=QN/uc4DVGf2ehT7byYjvpes2yNc3cXCx4tqW7n/KdFx/zePWGmoNtVVWJixkUOGEMkQYDh 0E/sX2X9uX79iCwgOCgd71P9ymeJJcann2HTJHFuDnlgkxa2n2Y0pYBEeLAw/5byAxwlWk Qo9CuTXEfA++PSu8w8MSz05cFbzXmPY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-177-nAqeIG35MDOL3vAb7c9CYQ-1; Fri, 19 Jun 2020 14:23:40 -0400 X-MC-Unique: nAqeIG35MDOL3vAb7c9CYQ-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 7D62C8018A5; Fri, 19 Jun 2020 18:23:39 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-113-14.ams2.redhat.com [10.36.113.14]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8A8821C8; Fri, 19 Jun 2020 18:23:37 +0000 (UTC) From: =?utf-8?q?Eugenio_P=C3=A9rez?= To: mst@redhat.com Cc: kvm list , Stefano Garzarella , virtualization@lists.linux-foundation.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Stefan Hajnoczi , Jason Wang Subject: [RFC v9 09/11] vhost/scsi: switch to buf APIs Date: Fri, 19 Jun 2020 20:23:00 +0200 Message-Id: <20200619182302.850-10-eperezma@redhat.com> In-Reply-To: <20200619182302.850-1-eperezma@redhat.com> References: <20200619182302.850-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Michael S. Tsirkin" Switch to buf APIs. Doing this exposes a spec violation in vhost scsi: all used bufs are marked with length 0. Fix that is left for another day. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez --- drivers/vhost/scsi.c | 73 ++++++++++++++++++++++++++------------------ 1 file changed, 44 insertions(+), 29 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 6fb4d7ecfa19..f09ca5244ab9 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -71,8 +71,8 @@ struct vhost_scsi_inflight { }; struct vhost_scsi_cmd { - /* Descriptor from vhost_get_vq_desc() for virt_queue segment */ - int tvc_vq_desc; + /* Descriptor from vhost_get_avail_buf() for virt_queue segment */ + struct vhost_buf tvc_vq_desc; /* virtio-scsi initiator task attribute */ int tvc_task_attr; /* virtio-scsi response incoming iovecs */ @@ -213,7 +213,7 @@ struct vhost_scsi { * Context for processing request and control queue operations. */ struct vhost_scsi_ctx { - int head; + struct vhost_buf buf; unsigned int out, in; size_t req_size, rsp_size; size_t out_size, in_size; @@ -443,6 +443,20 @@ static int vhost_scsi_check_stop_free(struct se_cmd *se_cmd) return target_put_sess_cmd(se_cmd); } +/* Signal to guest that request finished with no input buffer. */ +/* TODO calling this when writing into buffer and most likely a bug */ +static void vhost_scsi_signal_noinput(struct vhost_dev *vdev, + struct vhost_virtqueue *vq, + struct vhost_buf *bufp) +{ + struct vhost_buf buf = *bufp; + + buf.in_len = 0; + vhost_put_used_buf(vq, &buf); + vhost_signal(vdev, vq); +} + + static void vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt) { @@ -450,7 +464,8 @@ vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt) struct virtio_scsi_event *event = &evt->event; struct virtio_scsi_event __user *eventp; unsigned out, in; - int head, ret; + struct vhost_buf buf; + int ret; if (!vhost_vq_get_backend(vq)) { vs->vs_events_missed = true; @@ -459,14 +474,14 @@ vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt) again: vhost_disable_notify(&vs->dev, vq); - head = vhost_get_vq_desc(vq, vq->iov, - ARRAY_SIZE(vq->iov), &out, &in, - NULL, NULL); - if (head < 0) { + ret = vhost_get_avail_buf(vq, &buf, + vq->iov, ARRAY_SIZE(vq->iov), &out, &in, + NULL, NULL); + if (ret < 0) { vs->vs_events_missed = true; return; } - if (head == vq->num) { + if (!ret) { if (vhost_enable_notify(&vs->dev, vq)) goto again; vs->vs_events_missed = true; @@ -488,7 +503,7 @@ vhost_scsi_do_evt_work(struct vhost_scsi *vs, struct vhost_scsi_evt *evt) eventp = vq->iov[out].iov_base; ret = __copy_to_user(eventp, event, sizeof(*event)); if (!ret) - vhost_add_used_and_signal(&vs->dev, vq, head, 0); + vhost_scsi_signal_noinput(&vs->dev, vq, &buf); else vq_err(vq, "Faulted on vhost_scsi_send_event\n"); } @@ -549,7 +564,7 @@ static void vhost_scsi_complete_cmd_work(struct vhost_work *work) ret = copy_to_iter(&v_rsp, sizeof(v_rsp), &iov_iter); if (likely(ret == sizeof(v_rsp))) { struct vhost_scsi_virtqueue *q; - vhost_add_used(cmd->tvc_vq, cmd->tvc_vq_desc, 0); + vhost_put_used_buf(cmd->tvc_vq, &cmd->tvc_vq_desc); q = container_of(cmd->tvc_vq, struct vhost_scsi_virtqueue, vq); vq = q - vs->vqs; __set_bit(vq, signal); @@ -793,7 +808,7 @@ static void vhost_scsi_submission_work(struct work_struct *work) static void vhost_scsi_send_bad_target(struct vhost_scsi *vs, struct vhost_virtqueue *vq, - int head, unsigned out) + struct vhost_buf *buf, unsigned out) { struct virtio_scsi_cmd_resp __user *resp; struct virtio_scsi_cmd_resp rsp; @@ -804,7 +819,7 @@ vhost_scsi_send_bad_target(struct vhost_scsi *vs, resp = vq->iov[out].iov_base; ret = __copy_to_user(resp, &rsp, sizeof(rsp)); if (!ret) - vhost_add_used_and_signal(&vs->dev, vq, head, 0); + vhost_scsi_signal_noinput(&vs->dev, vq, buf); else pr_err("Faulted on virtio_scsi_cmd_resp\n"); } @@ -813,21 +828,21 @@ static int vhost_scsi_get_desc(struct vhost_scsi *vs, struct vhost_virtqueue *vq, struct vhost_scsi_ctx *vc) { - int ret = -ENXIO; + int r, ret = -ENXIO; - vc->head = vhost_get_vq_desc(vq, vq->iov, - ARRAY_SIZE(vq->iov), &vc->out, &vc->in, - NULL, NULL); + r = vhost_get_avail_buf(vq, &vc->buf, + vq->iov, ARRAY_SIZE(vq->iov), &vc->out, &vc->in, + NULL, NULL); - pr_debug("vhost_get_vq_desc: head: %d, out: %u in: %u\n", - vc->head, vc->out, vc->in); + pr_debug("vhost_get_avail_buf: buf: %d, out: %u in: %u\n", + vc->buf.id, vc->out, vc->in); /* On error, stop handling until the next kick. */ - if (unlikely(vc->head < 0)) + if (unlikely(r < 0)) goto done; /* Nothing new? Wait for eventfd to tell us they refilled. */ - if (vc->head == vq->num) { + if (!r) { if (unlikely(vhost_enable_notify(&vs->dev, vq))) { vhost_disable_notify(&vs->dev, vq); ret = -EAGAIN; @@ -1093,11 +1108,11 @@ vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) } } /* - * Save the descriptor from vhost_get_vq_desc() to be used to + * Save the descriptor from vhost_get_avail_buf() to be used to * complete the virtio-scsi request in TCM callback context via * vhost_scsi_queue_data_in() and vhost_scsi_queue_status() */ - cmd->tvc_vq_desc = vc.head; + cmd->tvc_vq_desc = vc.buf; /* * Dispatch cmd descriptor for cmwq execution in process * context provided by vhost_scsi_workqueue. This also ensures @@ -1117,7 +1132,7 @@ vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) if (ret == -ENXIO) break; else if (ret == -EIO) - vhost_scsi_send_bad_target(vs, vq, vc.head, vc.out); + vhost_scsi_send_bad_target(vs, vq, &vc.buf, vc.out); } while (likely(!vhost_exceeds_weight(vq, ++c, 0))); out: mutex_unlock(&vq->mutex); @@ -1139,9 +1154,9 @@ vhost_scsi_send_tmf_reject(struct vhost_scsi *vs, iov_iter_init(&iov_iter, READ, &vq->iov[vc->out], vc->in, sizeof(rsp)); ret = copy_to_iter(&rsp, sizeof(rsp), &iov_iter); - if (likely(ret == sizeof(rsp))) - vhost_add_used_and_signal(&vs->dev, vq, vc->head, 0); - else + if (likely(ret == sizeof(rsp))) { + vhost_scsi_signal_noinput(&vs->dev, vq, &vc->buf); + } else pr_err("Faulted on virtio_scsi_ctrl_tmf_resp\n"); } @@ -1162,7 +1177,7 @@ vhost_scsi_send_an_resp(struct vhost_scsi *vs, ret = copy_to_iter(&rsp, sizeof(rsp), &iov_iter); if (likely(ret == sizeof(rsp))) - vhost_add_used_and_signal(&vs->dev, vq, vc->head, 0); + vhost_scsi_signal_noinput(&vs->dev, vq, &vc->buf); else pr_err("Faulted on virtio_scsi_ctrl_an_resp\n"); } @@ -1269,7 +1284,7 @@ vhost_scsi_ctl_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) if (ret == -ENXIO) break; else if (ret == -EIO) - vhost_scsi_send_bad_target(vs, vq, vc.head, vc.out); + vhost_scsi_send_bad_target(vs, vq, &vc.buf, vc.out); } while (likely(!vhost_exceeds_weight(vq, ++c, 0))); out: mutex_unlock(&vq->mutex);