From patchwork Mon Jun 8 12:52:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 217934 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E96BC433DF for ; Mon, 8 Jun 2020 12:55:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D0A2C206C3 for ; Mon, 8 Jun 2020 12:55:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MIkoL279" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729903AbgFHMz1 (ORCPT ); Mon, 8 Jun 2020 08:55:27 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:33777 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728007AbgFHMxJ (ORCPT ); Mon, 8 Jun 2020 08:53:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591620785; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pyZgoc1IoaMWB9Jefr7e/GoAFJJxo7imj3dgpBB/etA=; b=MIkoL279Rs5lwqGyWsNvXdGFNE1oUgW44V7RSdCSvGbTFXmZxncrySsD1PT9SDbtGnnQx9 iQphV1ppTqzObFRHfFoSBZxvoVYNbu9ExqNA/pojgviCap5/gyGxCDVdEaTxtG6ubYzHoA 8bReyyeAs1sTFEWd82PqNdsqVbfndmM= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-511--D4hnQiePiusV9EVhe4D8w-1; Mon, 08 Jun 2020 08:52:57 -0400 X-MC-Unique: -D4hnQiePiusV9EVhe4D8w-1 Received: by mail-wm1-f70.google.com with SMTP id a7so1503182wmf.1 for ; Mon, 08 Jun 2020 05:52:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=pyZgoc1IoaMWB9Jefr7e/GoAFJJxo7imj3dgpBB/etA=; b=SAeEOTD4rea/ac9SO85hZ4oRMnyovzA0Q2WsJI/NioCCLvtROkvuW5WXdYiB0YxV2z NL/nMfB3AU9LM9SEoVJ7JJvhnKokVc4ih5i2R5TacDyWk/RXK5APf13BvOA3YX/Ko28X XFFbSqbxIcYohM19ld8+4/8+7Lp12vKz0Lq7qwHk5hST7WnXBzFL4vfo2Y7eY5BeMARn exJjDIV/kTAnuHvPOqCybRbfylcIiqaF7Q8VeiaZoLftk89mFUY127+dLgiLkUrsAE41 BfTlacgefrIOfmr7cdoKj/h8+2uIV8oSxz7RcO0dKNPBNi+lBikMg/9CJvIwHv2ZNP+c jDTA== X-Gm-Message-State: AOAM533lWiIRZPtzZlZyT+8c64I/RXajNmDgCRSDn+HUonb6p14B+2BU RRoBCI3cPzl70C73c8U8k3zGQL9hjguXr01ETIWKY2JDq0Na8PhNgOweFFqTfnyUHcjhOEjMKI7 tNYJlQhr8ykw6S6nX X-Received: by 2002:a7b:cb93:: with SMTP id m19mr17016540wmi.165.1591620776335; Mon, 08 Jun 2020 05:52:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzEdknqoJU3HSkUfhKPXzphp7+7B7/1HQpacB61dXsZyraKBYychpqxInl7px+4iWcK/8HI/Q== X-Received: by 2002:a7b:cb93:: with SMTP id m19mr17016499wmi.165.1591620775843; Mon, 08 Jun 2020 05:52:55 -0700 (PDT) Received: from redhat.com (bzq-109-64-41-91.red.bezeqint.net. [109.64.41.91]) by smtp.gmail.com with ESMTPSA id g187sm22919251wma.17.2020.06.08.05.52.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2020 05:52:55 -0700 (PDT) Date: Mon, 8 Jun 2020 08:52:53 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v6 01/11] vhost: option to fetch descriptors through an independent struct Message-ID: <20200608125238.728563-2-mst@redhat.com> References: <20200608125238.728563-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200608125238.728563-1-mst@redhat.com> X-Mailer: git-send-email 2.24.1.751.gd10ce2899c X-Mutt-Fcc: =sent Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The idea is to support multiple ring formats by converting to a format-independent array of descriptors. This costs extra cycles, but we gain in ability to fetch a batch of descriptors in one go, which is good for code cache locality. When used, this causes a minor performance degradation, it's been kept as simple as possible for ease of review. A follow-up patch gets us back the performance by adding batching. To simplify benchmarking, I kept the old code around so one can switch back and forth between old and new code. This will go away in the final submission. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez Link: https://lore.kernel.org/r/20200401183118.8334-2-eperezma@redhat.com Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 305 +++++++++++++++++++++++++++++++++++++++++- drivers/vhost/vhost.h | 16 +++ 2 files changed, 320 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 172da092107e..180b7b58c76b 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -303,6 +303,7 @@ static void vhost_vq_reset(struct vhost_dev *dev, struct vhost_virtqueue *vq) { vq->num = 1; + vq->ndescs = 0; vq->desc = NULL; vq->avail = NULL; vq->used = NULL; @@ -373,6 +374,9 @@ static int vhost_worker(void *data) static void vhost_vq_free_iovecs(struct vhost_virtqueue *vq) { + kfree(vq->descs); + vq->descs = NULL; + vq->max_descs = 0; kfree(vq->indirect); vq->indirect = NULL; kfree(vq->log); @@ -389,6 +393,10 @@ static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) for (i = 0; i < dev->nvqs; ++i) { vq = dev->vqs[i]; + vq->max_descs = dev->iov_limit; + vq->descs = kmalloc_array(vq->max_descs, + sizeof(*vq->descs), + GFP_KERNEL); vq->indirect = kmalloc_array(UIO_MAXIOV, sizeof(*vq->indirect), GFP_KERNEL); @@ -396,7 +404,7 @@ static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) GFP_KERNEL); vq->heads = kmalloc_array(dev->iov_limit, sizeof(*vq->heads), GFP_KERNEL); - if (!vq->indirect || !vq->log || !vq->heads) + if (!vq->indirect || !vq->log || !vq->heads || !vq->descs) goto err_nomem; } return 0; @@ -488,6 +496,8 @@ void vhost_dev_init(struct vhost_dev *dev, for (i = 0; i < dev->nvqs; ++i) { vq = dev->vqs[i]; + vq->descs = NULL; + vq->max_descs = 0; vq->log = NULL; vq->indirect = NULL; vq->heads = NULL; @@ -2315,6 +2325,299 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, } EXPORT_SYMBOL_GPL(vhost_get_vq_desc); +static struct vhost_desc *peek_split_desc(struct vhost_virtqueue *vq) +{ + BUG_ON(!vq->ndescs); + return &vq->descs[vq->ndescs - 1]; +} + +static void pop_split_desc(struct vhost_virtqueue *vq) +{ + BUG_ON(!vq->ndescs); + --vq->ndescs; +} + +#define VHOST_DESC_FLAGS (VRING_DESC_F_INDIRECT | VRING_DESC_F_WRITE | \ + VRING_DESC_F_NEXT) +static int push_split_desc(struct vhost_virtqueue *vq, struct vring_desc *desc, u16 id) +{ + struct vhost_desc *h; + + if (unlikely(vq->ndescs >= vq->max_descs)) + return -EINVAL; + h = &vq->descs[vq->ndescs++]; + h->addr = vhost64_to_cpu(vq, desc->addr); + h->len = vhost32_to_cpu(vq, desc->len); + h->flags = vhost16_to_cpu(vq, desc->flags) & VHOST_DESC_FLAGS; + h->id = id; + + return 0; +} + +static int fetch_indirect_descs(struct vhost_virtqueue *vq, + struct vhost_desc *indirect, + u16 head) +{ + struct vring_desc desc; + unsigned int i = 0, count, found = 0; + u32 len = indirect->len; + struct iov_iter from; + int ret; + + /* Sanity check */ + if (unlikely(len % sizeof desc)) { + vq_err(vq, "Invalid length in indirect descriptor: " + "len 0x%llx not multiple of 0x%zx\n", + (unsigned long long)len, + sizeof desc); + return -EINVAL; + } + + ret = translate_desc(vq, indirect->addr, len, vq->indirect, + UIO_MAXIOV, VHOST_ACCESS_RO); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Translation failure %d in indirect.\n", ret); + return ret; + } + iov_iter_init(&from, READ, vq->indirect, ret, len); + + /* We will use the result as an address to read from, so most + * architectures only need a compiler barrier here. */ + read_barrier_depends(); + + count = len / sizeof desc; + /* Buffers are chained via a 16 bit next field, so + * we can have at most 2^16 of these. */ + if (unlikely(count > USHRT_MAX + 1)) { + vq_err(vq, "Indirect buffer length too big: %d\n", + indirect->len); + return -E2BIG; + } + if (unlikely(vq->ndescs + count > vq->max_descs)) { + vq_err(vq, "Too many indirect + direct descs: %d + %d\n", + vq->ndescs, indirect->len); + return -E2BIG; + } + + do { + if (unlikely(++found > count)) { + vq_err(vq, "Loop detected: last one at %u " + "indirect size %u\n", + i, count); + return -EINVAL; + } + if (unlikely(!copy_from_iter_full(&desc, sizeof(desc), &from))) { + vq_err(vq, "Failed indirect descriptor: idx %d, %zx\n", + i, (size_t)indirect->addr + i * sizeof desc); + return -EINVAL; + } + if (unlikely(desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_INDIRECT))) { + vq_err(vq, "Nested indirect descriptor: idx %d, %zx\n", + i, (size_t)indirect->addr + i * sizeof desc); + return -EINVAL; + } + + /* Note: push_split_desc can't fail here: + * we never fetch unless there's space. */ + ret = push_split_desc(vq, &desc, head); + WARN_ON(ret); + } while ((i = next_desc(vq, &desc)) != -1); + return 0; +} + +/* This function returns a value > 0 if a descriptor was found, or 0 if none were found. + * A negative code is returned on error. */ +static int fetch_descs(struct vhost_virtqueue *vq) +{ + unsigned int i, head, found = 0; + struct vhost_desc *last; + struct vring_desc desc; + __virtio16 avail_idx; + __virtio16 ring_head; + u16 last_avail_idx; + int ret; + + /* Check it isn't doing very strange things with descriptor numbers. */ + last_avail_idx = vq->last_avail_idx; + + if (vq->avail_idx == vq->last_avail_idx) { + if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { + vq_err(vq, "Failed to access avail idx at %p\n", + &vq->avail->idx); + return -EFAULT; + } + vq->avail_idx = vhost16_to_cpu(vq, avail_idx); + + if (unlikely((u16)(vq->avail_idx - last_avail_idx) > vq->num)) { + vq_err(vq, "Guest moved used index from %u to %u", + last_avail_idx, vq->avail_idx); + return -EFAULT; + } + + /* If there's nothing new since last we looked, return + * invalid. + */ + if (vq->avail_idx == last_avail_idx) + return 0; + + /* Only get avail ring entries after they have been + * exposed by guest. + */ + smp_rmb(); + } + + /* Grab the next descriptor number they're advertising */ + if (unlikely(vhost_get_avail_head(vq, &ring_head, last_avail_idx))) { + vq_err(vq, "Failed to read head: idx %d address %p\n", + last_avail_idx, + &vq->avail->ring[last_avail_idx % vq->num]); + return -EFAULT; + } + + head = vhost16_to_cpu(vq, ring_head); + + /* If their number is silly, that's an error. */ + if (unlikely(head >= vq->num)) { + vq_err(vq, "Guest says index %u > %u is available", + head, vq->num); + return -EINVAL; + } + + i = head; + do { + if (unlikely(i >= vq->num)) { + vq_err(vq, "Desc index is %u > %u, head = %u", + i, vq->num, head); + return -EINVAL; + } + if (unlikely(++found > vq->num)) { + vq_err(vq, "Loop detected: last one at %u " + "vq size %u head %u\n", + i, vq->num, head); + return -EINVAL; + } + ret = vhost_get_desc(vq, &desc, i); + if (unlikely(ret)) { + vq_err(vq, "Failed to get descriptor: idx %d addr %p\n", + i, vq->desc + i); + return -EFAULT; + } + ret = push_split_desc(vq, &desc, head); + if (unlikely(ret)) { + vq_err(vq, "Failed to save descriptor: idx %d\n", i); + return -EINVAL; + } + } while ((i = next_desc(vq, &desc)) != -1); + + last = peek_split_desc(vq); + if (unlikely(last->flags & VRING_DESC_F_INDIRECT)) { + pop_split_desc(vq); + ret = fetch_indirect_descs(vq, last, head); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Failure detected " + "in indirect descriptor at idx %d\n", head); + return ret; + } + } + + /* Assume notifications from guest are disabled at this point, + * if they aren't we would need to update avail_event index. */ + BUG_ON(!(vq->used_flags & VRING_USED_F_NO_NOTIFY)); + + /* On success, increment avail index. */ + vq->last_avail_idx++; + + return 1; +} + +/* This looks in the virtqueue and for the first available buffer, and converts + * it to an iovec for convenient access. Since descriptors consist of some + * number of output then some number of input descriptors, it's actually two + * iovecs, but we pack them into one and note how many of each there were. + * + * This function returns the descriptor number found, or vq->num (which is + * never a valid descriptor number) if none was found. A negative code is + * returned on error. */ +int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, + struct iovec iov[], unsigned int iov_size, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num) +{ + int ret = fetch_descs(vq); + int i; + + if (ret <= 0) + goto err_fetch; + + /* Now convert to IOV */ + /* When we start there are none of either input nor output. */ + *out_num = *in_num = 0; + if (unlikely(log)) + *log_num = 0; + + for (i = 0; i < vq->ndescs; ++i) { + unsigned iov_count = *in_num + *out_num; + struct vhost_desc *desc = &vq->descs[i]; + int access; + + if (desc->flags & ~VHOST_DESC_FLAGS) { + vq_err(vq, "Unexpected flags: 0x%x at descriptor id 0x%x\n", + desc->flags, desc->id); + ret = -EINVAL; + goto err; + } + if (desc->flags & VRING_DESC_F_WRITE) + access = VHOST_ACCESS_WO; + else + access = VHOST_ACCESS_RO; + ret = translate_desc(vq, desc->addr, + desc->len, iov + iov_count, + iov_size - iov_count, access); + if (unlikely(ret < 0)) { + if (ret != -EAGAIN) + vq_err(vq, "Translation failure %d descriptor idx %d\n", + ret, i); + goto err; + } + if (access == VHOST_ACCESS_WO) { + /* If this is an input descriptor, + * increment that count. */ + *in_num += ret; + if (unlikely(log && ret)) { + log[*log_num].addr = desc->addr; + log[*log_num].len = desc->len; + ++*log_num; + } + } else { + /* If it's an output descriptor, they're all supposed + * to come before any input descriptors. */ + if (unlikely(*in_num)) { + vq_err(vq, "Descriptor has out after in: " + "idx %d\n", i); + ret = -EINVAL; + goto err; + } + *out_num += ret; + } + + ret = desc->id; + } + + vq->ndescs = 0; + + return ret; + +err: + vhost_discard_vq_desc(vq, 1); +err_fetch: + vq->ndescs = 0; + + return ret; +} +EXPORT_SYMBOL_GPL(vhost_get_vq_desc_batch); + /* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) { diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index c8e96a095d3b..87089d51490d 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -60,6 +60,13 @@ enum vhost_uaddr_type { VHOST_NUM_ADDRS = 3, }; +struct vhost_desc { + u64 addr; + u32 len; + u16 flags; /* VRING_DESC_F_WRITE, VRING_DESC_F_NEXT */ + u16 id; +}; + /* The virtqueue structure describes a queue attached to a device. */ struct vhost_virtqueue { struct vhost_dev *dev; @@ -71,6 +78,11 @@ struct vhost_virtqueue { vring_avail_t __user *avail; vring_used_t __user *used; const struct vhost_iotlb_map *meta_iotlb[VHOST_NUM_ADDRS]; + + struct vhost_desc *descs; + int ndescs; + int max_descs; + struct file *kick; struct eventfd_ctx *call_ctx; struct eventfd_ctx *error_ctx; @@ -177,6 +189,10 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg bool vhost_vq_access_ok(struct vhost_virtqueue *vq); bool vhost_log_access_ok(struct vhost_dev *); +int vhost_get_vq_desc_batch(struct vhost_virtqueue *, + struct iovec iov[], unsigned int iov_count, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num); int vhost_get_vq_desc(struct vhost_virtqueue *, struct iovec iov[], unsigned int iov_count, unsigned int *out_num, unsigned int *in_num, From patchwork Mon Jun 8 12:52:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 217936 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF9F9C433DF for ; Mon, 8 Jun 2020 12:54:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BDDF8206C3 for ; Mon, 8 Jun 2020 12:54:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="W7Pww0y1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729828AbgFHMyY (ORCPT ); Mon, 8 Jun 2020 08:54:24 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:48411 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729618AbgFHMxP (ORCPT ); Mon, 8 Jun 2020 08:53:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591620792; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1DnrUxs6WiaZdAUTZ1DCu3SMBfamowHkG+YOVbhUgOU=; b=W7Pww0y1gc6W6mOlN9UUKQ94VRwrbS5OB5k4+cjyXXehuZvAxrhNMhUD7hQYTCJTtPk1jU Oaie8pHFUVoAC1qDw0Ar/bUhOOtDYakXCjKQPynlQ14Uqvvf0xbQhE86k0qiQWL7XI5JSd 9AmNZMKlr6QOgmPqvdAW5qxAX11hpWo= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-247-jwvdD6XOOJq48d9igBM7sw-1; Mon, 08 Jun 2020 08:52:59 -0400 X-MC-Unique: jwvdD6XOOJq48d9igBM7sw-1 Received: by mail-wr1-f69.google.com with SMTP id t5so7100223wro.20 for ; Mon, 08 Jun 2020 05:52:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to; bh=1DnrUxs6WiaZdAUTZ1DCu3SMBfamowHkG+YOVbhUgOU=; b=CL0WhQRczuvwsnhgTDABZ50UHaIyhPkTKh5Pzh5pgw76aqMEXNsv5dGyKPZn8uv4TY XaOQCJ09+zX0EvjKJzodl58FbozfzPWN74szDdtQvh95NYDimwWX94345XYRo1khdZEc HoYC39+0yUD7dKE5bLZ4M/7Kkf05CO1Ui2CSwyuAQY/etXl2QRqmOfVN5yOyz/1GaiAF C9LAkXRnTB/Ftdnc5Ot1eJJr9fjzebsr8dZjdcl2m0yHQU/JP7hsqtS95QNzZ5WQKRgH 0WCeKY5SkXNX4OE/MBXa4RgvEl6VKsfgUrtUwVSWehPYgj1FTkVUG14AcJ+ezJ7ErSGq L70g== X-Gm-Message-State: AOAM5337xaXWnj4Ui+dQ99kTA8iWg3jCN/v4pifPqebLb8Fuv8MqAz/c XOoqP35eFqQxDKJNnHXAH96IUKH8lhgNLFskauIzNAflX97iB0eXLtlNwfOaCwDeRewqaaZOXta UEmZT4cZKk9EJduAr X-Received: by 2002:adf:fb92:: with SMTP id a18mr23863444wrr.263.1591620778392; Mon, 08 Jun 2020 05:52:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy7zln2kdb1pxmg3n/kEmFfClQ7VDjVlR/0I8gaqlDMVkpw4lmd34/LQrhQscpfdGSeiviTPw== X-Received: by 2002:adf:fb92:: with SMTP id a18mr23863408wrr.263.1591620777892; Mon, 08 Jun 2020 05:52:57 -0700 (PDT) Received: from redhat.com (bzq-109-64-41-91.red.bezeqint.net. [109.64.41.91]) by smtp.gmail.com with ESMTPSA id t8sm22817233wro.56.2020.06.08.05.52.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2020 05:52:57 -0700 (PDT) Date: Mon, 8 Jun 2020 08:52:56 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v6 02/11] vhost: use batched get_vq_desc version Message-ID: <20200608125238.728563-3-mst@redhat.com> References: <20200608125238.728563-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200608125238.728563-1-mst@redhat.com> X-Mailer: git-send-email 2.24.1.751.gd10ce2899c X-Mutt-Fcc: =sent Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org As testing shows no performance change, switch to that now. Signed-off-by: Michael S. Tsirkin Signed-off-by: Eugenio Pérez Link: https://lore.kernel.org/r/20200401183118.8334-3-eperezma@redhat.com Signed-off-by: Michael S. Tsirkin --- drivers/vhost/test.c | 2 +- drivers/vhost/vhost.c | 318 ++++++++---------------------------------- drivers/vhost/vhost.h | 7 +- 3 files changed, 65 insertions(+), 262 deletions(-) diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c index 0466921f4772..7d69778aaa26 100644 --- a/drivers/vhost/test.c +++ b/drivers/vhost/test.c @@ -119,7 +119,7 @@ static int vhost_test_open(struct inode *inode, struct file *f) dev = &n->dev; vqs[VHOST_TEST_VQ] = &n->vqs[VHOST_TEST_VQ]; n->vqs[VHOST_TEST_VQ].handle_kick = handle_vq_kick; - vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV, + vhost_dev_init(dev, vqs, VHOST_TEST_VQ_MAX, UIO_MAXIOV + 64, VHOST_TEST_PKT_WEIGHT, VHOST_TEST_WEIGHT, true, NULL); f->private_data = n; diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 180b7b58c76b..41d6b132c234 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -304,6 +304,7 @@ static void vhost_vq_reset(struct vhost_dev *dev, { vq->num = 1; vq->ndescs = 0; + vq->first_desc = 0; vq->desc = NULL; vq->avail = NULL; vq->used = NULL; @@ -372,6 +373,11 @@ static int vhost_worker(void *data) return 0; } +static int vhost_vq_num_batch_descs(struct vhost_virtqueue *vq) +{ + return vq->max_descs - UIO_MAXIOV; +} + static void vhost_vq_free_iovecs(struct vhost_virtqueue *vq) { kfree(vq->descs); @@ -394,6 +400,9 @@ static long vhost_dev_alloc_iovecs(struct vhost_dev *dev) for (i = 0; i < dev->nvqs; ++i) { vq = dev->vqs[i]; vq->max_descs = dev->iov_limit; + if (vhost_vq_num_batch_descs(vq) < 0) { + return -EINVAL; + } vq->descs = kmalloc_array(vq->max_descs, sizeof(*vq->descs), GFP_KERNEL); @@ -1610,6 +1619,7 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg vq->last_avail_idx = s.num; /* Forget the cached index value. */ vq->avail_idx = vq->last_avail_idx; + vq->ndescs = vq->first_desc = 0; break; case VHOST_GET_VRING_BASE: s.index = idx; @@ -2078,253 +2088,6 @@ static unsigned next_desc(struct vhost_virtqueue *vq, struct vring_desc *desc) return next; } -static int get_indirect(struct vhost_virtqueue *vq, - struct iovec iov[], unsigned int iov_size, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num, - struct vring_desc *indirect) -{ - struct vring_desc desc; - unsigned int i = 0, count, found = 0; - u32 len = vhost32_to_cpu(vq, indirect->len); - struct iov_iter from; - int ret, access; - - /* Sanity check */ - if (unlikely(len % sizeof desc)) { - vq_err(vq, "Invalid length in indirect descriptor: " - "len 0x%llx not multiple of 0x%zx\n", - (unsigned long long)len, - sizeof desc); - return -EINVAL; - } - - ret = translate_desc(vq, vhost64_to_cpu(vq, indirect->addr), len, vq->indirect, - UIO_MAXIOV, VHOST_ACCESS_RO); - if (unlikely(ret < 0)) { - if (ret != -EAGAIN) - vq_err(vq, "Translation failure %d in indirect.\n", ret); - return ret; - } - iov_iter_init(&from, READ, vq->indirect, ret, len); - - /* We will use the result as an address to read from, so most - * architectures only need a compiler barrier here. */ - read_barrier_depends(); - - count = len / sizeof desc; - /* Buffers are chained via a 16 bit next field, so - * we can have at most 2^16 of these. */ - if (unlikely(count > USHRT_MAX + 1)) { - vq_err(vq, "Indirect buffer length too big: %d\n", - indirect->len); - return -E2BIG; - } - - do { - unsigned iov_count = *in_num + *out_num; - if (unlikely(++found > count)) { - vq_err(vq, "Loop detected: last one at %u " - "indirect size %u\n", - i, count); - return -EINVAL; - } - if (unlikely(!copy_from_iter_full(&desc, sizeof(desc), &from))) { - vq_err(vq, "Failed indirect descriptor: idx %d, %zx\n", - i, (size_t)vhost64_to_cpu(vq, indirect->addr) + i * sizeof desc); - return -EINVAL; - } - if (unlikely(desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_INDIRECT))) { - vq_err(vq, "Nested indirect descriptor: idx %d, %zx\n", - i, (size_t)vhost64_to_cpu(vq, indirect->addr) + i * sizeof desc); - return -EINVAL; - } - - if (desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_WRITE)) - access = VHOST_ACCESS_WO; - else - access = VHOST_ACCESS_RO; - - ret = translate_desc(vq, vhost64_to_cpu(vq, desc.addr), - vhost32_to_cpu(vq, desc.len), iov + iov_count, - iov_size - iov_count, access); - if (unlikely(ret < 0)) { - if (ret != -EAGAIN) - vq_err(vq, "Translation failure %d indirect idx %d\n", - ret, i); - return ret; - } - /* If this is an input descriptor, increment that count. */ - if (access == VHOST_ACCESS_WO) { - *in_num += ret; - if (unlikely(log && ret)) { - log[*log_num].addr = vhost64_to_cpu(vq, desc.addr); - log[*log_num].len = vhost32_to_cpu(vq, desc.len); - ++*log_num; - } - } else { - /* If it's an output descriptor, they're all supposed - * to come before any input descriptors. */ - if (unlikely(*in_num)) { - vq_err(vq, "Indirect descriptor " - "has out after in: idx %d\n", i); - return -EINVAL; - } - *out_num += ret; - } - } while ((i = next_desc(vq, &desc)) != -1); - return 0; -} - -/* This looks in the virtqueue and for the first available buffer, and converts - * it to an iovec for convenient access. Since descriptors consist of some - * number of output then some number of input descriptors, it's actually two - * iovecs, but we pack them into one and note how many of each there were. - * - * This function returns the descriptor number found, or vq->num (which is - * never a valid descriptor number) if none was found. A negative code is - * returned on error. */ -int vhost_get_vq_desc(struct vhost_virtqueue *vq, - struct iovec iov[], unsigned int iov_size, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num) -{ - struct vring_desc desc; - unsigned int i, head, found = 0; - u16 last_avail_idx; - __virtio16 avail_idx; - __virtio16 ring_head; - int ret, access; - - /* Check it isn't doing very strange things with descriptor numbers. */ - last_avail_idx = vq->last_avail_idx; - - if (vq->avail_idx == vq->last_avail_idx) { - if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { - vq_err(vq, "Failed to access avail idx at %p\n", - &vq->avail->idx); - return -EFAULT; - } - vq->avail_idx = vhost16_to_cpu(vq, avail_idx); - - if (unlikely((u16)(vq->avail_idx - last_avail_idx) > vq->num)) { - vq_err(vq, "Guest moved used index from %u to %u", - last_avail_idx, vq->avail_idx); - return -EFAULT; - } - - /* If there's nothing new since last we looked, return - * invalid. - */ - if (vq->avail_idx == last_avail_idx) - return vq->num; - - /* Only get avail ring entries after they have been - * exposed by guest. - */ - smp_rmb(); - } - - /* Grab the next descriptor number they're advertising, and increment - * the index we've seen. */ - if (unlikely(vhost_get_avail_head(vq, &ring_head, last_avail_idx))) { - vq_err(vq, "Failed to read head: idx %d address %p\n", - last_avail_idx, - &vq->avail->ring[last_avail_idx % vq->num]); - return -EFAULT; - } - - head = vhost16_to_cpu(vq, ring_head); - - /* If their number is silly, that's an error. */ - if (unlikely(head >= vq->num)) { - vq_err(vq, "Guest says index %u > %u is available", - head, vq->num); - return -EINVAL; - } - - /* When we start there are none of either input nor output. */ - *out_num = *in_num = 0; - if (unlikely(log)) - *log_num = 0; - - i = head; - do { - unsigned iov_count = *in_num + *out_num; - if (unlikely(i >= vq->num)) { - vq_err(vq, "Desc index is %u > %u, head = %u", - i, vq->num, head); - return -EINVAL; - } - if (unlikely(++found > vq->num)) { - vq_err(vq, "Loop detected: last one at %u " - "vq size %u head %u\n", - i, vq->num, head); - return -EINVAL; - } - ret = vhost_get_desc(vq, &desc, i); - if (unlikely(ret)) { - vq_err(vq, "Failed to get descriptor: idx %d addr %p\n", - i, vq->desc + i); - return -EFAULT; - } - if (desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_INDIRECT)) { - ret = get_indirect(vq, iov, iov_size, - out_num, in_num, - log, log_num, &desc); - if (unlikely(ret < 0)) { - if (ret != -EAGAIN) - vq_err(vq, "Failure detected " - "in indirect descriptor at idx %d\n", i); - return ret; - } - continue; - } - - if (desc.flags & cpu_to_vhost16(vq, VRING_DESC_F_WRITE)) - access = VHOST_ACCESS_WO; - else - access = VHOST_ACCESS_RO; - ret = translate_desc(vq, vhost64_to_cpu(vq, desc.addr), - vhost32_to_cpu(vq, desc.len), iov + iov_count, - iov_size - iov_count, access); - if (unlikely(ret < 0)) { - if (ret != -EAGAIN) - vq_err(vq, "Translation failure %d descriptor idx %d\n", - ret, i); - return ret; - } - if (access == VHOST_ACCESS_WO) { - /* If this is an input descriptor, - * increment that count. */ - *in_num += ret; - if (unlikely(log && ret)) { - log[*log_num].addr = vhost64_to_cpu(vq, desc.addr); - log[*log_num].len = vhost32_to_cpu(vq, desc.len); - ++*log_num; - } - } else { - /* If it's an output descriptor, they're all supposed - * to come before any input descriptors. */ - if (unlikely(*in_num)) { - vq_err(vq, "Descriptor has out after in: " - "idx %d\n", i); - return -EINVAL; - } - *out_num += ret; - } - } while ((i = next_desc(vq, &desc)) != -1); - - /* On success, increment avail index. */ - vq->last_avail_idx++; - - /* Assume notifications from guest are disabled at this point, - * if they aren't we would need to update avail_event index. */ - BUG_ON(!(vq->used_flags & VRING_USED_F_NO_NOTIFY)); - return head; -} -EXPORT_SYMBOL_GPL(vhost_get_vq_desc); - static struct vhost_desc *peek_split_desc(struct vhost_virtqueue *vq) { BUG_ON(!vq->ndescs); @@ -2428,7 +2191,7 @@ static int fetch_indirect_descs(struct vhost_virtqueue *vq, /* This function returns a value > 0 if a descriptor was found, or 0 if none were found. * A negative code is returned on error. */ -static int fetch_descs(struct vhost_virtqueue *vq) +static int fetch_buf(struct vhost_virtqueue *vq) { unsigned int i, head, found = 0; struct vhost_desc *last; @@ -2441,7 +2204,11 @@ static int fetch_descs(struct vhost_virtqueue *vq) /* Check it isn't doing very strange things with descriptor numbers. */ last_avail_idx = vq->last_avail_idx; - if (vq->avail_idx == vq->last_avail_idx) { + if (unlikely(vq->avail_idx == vq->last_avail_idx)) { + /* If we already have work to do, don't bother re-checking. */ + if (likely(vq->ndescs)) + return 1; + if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { vq_err(vq, "Failed to access avail idx at %p\n", &vq->avail->idx); @@ -2532,6 +2299,41 @@ static int fetch_descs(struct vhost_virtqueue *vq) return 1; } +/* This function returns a value > 0 if a descriptor was found, or 0 if none were found. + * A negative code is returned on error. */ +static int fetch_descs(struct vhost_virtqueue *vq) +{ + int ret; + + if (unlikely(vq->first_desc >= vq->ndescs)) { + vq->first_desc = 0; + vq->ndescs = 0; + } + + if (vq->ndescs) + return 1; + + for (ret = 1; + ret > 0 && vq->ndescs <= vhost_vq_num_batch_descs(vq); + ret = fetch_buf(vq)) + ; + + /* On success we expect some descs */ + BUG_ON(ret > 0 && !vq->ndescs); + return ret; +} + +/* Reverse the effects of fetch_descs */ +static void unfetch_descs(struct vhost_virtqueue *vq) +{ + int i; + + for (i = vq->first_desc; i < vq->ndescs; ++i) + if (!(vq->descs[i].flags & VRING_DESC_F_NEXT)) + vq->last_avail_idx -= 1; + vq->ndescs = 0; +} + /* This looks in the virtqueue and for the first available buffer, and converts * it to an iovec for convenient access. Since descriptors consist of some * number of output then some number of input descriptors, it's actually two @@ -2540,7 +2342,7 @@ static int fetch_descs(struct vhost_virtqueue *vq) * This function returns the descriptor number found, or vq->num (which is * never a valid descriptor number) if none was found. A negative code is * returned on error. */ -int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, +int vhost_get_vq_desc(struct vhost_virtqueue *vq, struct iovec iov[], unsigned int iov_size, unsigned int *out_num, unsigned int *in_num, struct vhost_log *log, unsigned int *log_num) @@ -2549,7 +2351,7 @@ int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, int i; if (ret <= 0) - goto err_fetch; + goto err; /* Now convert to IOV */ /* When we start there are none of either input nor output. */ @@ -2557,7 +2359,7 @@ int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, if (unlikely(log)) *log_num = 0; - for (i = 0; i < vq->ndescs; ++i) { + for (i = vq->first_desc; i < vq->ndescs; ++i) { unsigned iov_count = *in_num + *out_num; struct vhost_desc *desc = &vq->descs[i]; int access; @@ -2603,24 +2405,26 @@ int vhost_get_vq_desc_batch(struct vhost_virtqueue *vq, } ret = desc->id; + + if (!(desc->flags & VRING_DESC_F_NEXT)) + break; } - vq->ndescs = 0; + vq->first_desc = i + 1; return ret; err: - vhost_discard_vq_desc(vq, 1); -err_fetch: - vq->ndescs = 0; + unfetch_descs(vq); return ret; } -EXPORT_SYMBOL_GPL(vhost_get_vq_desc_batch); +EXPORT_SYMBOL_GPL(vhost_get_vq_desc); /* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) { + unfetch_descs(vq); vq->last_avail_idx -= n; } EXPORT_SYMBOL_GPL(vhost_discard_vq_desc); diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 87089d51490d..fed36af5c444 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -81,6 +81,7 @@ struct vhost_virtqueue { struct vhost_desc *descs; int ndescs; + int first_desc; int max_descs; struct file *kick; @@ -189,10 +190,6 @@ long vhost_vring_ioctl(struct vhost_dev *d, unsigned int ioctl, void __user *arg bool vhost_vq_access_ok(struct vhost_virtqueue *vq); bool vhost_log_access_ok(struct vhost_dev *); -int vhost_get_vq_desc_batch(struct vhost_virtqueue *, - struct iovec iov[], unsigned int iov_count, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num); int vhost_get_vq_desc(struct vhost_virtqueue *, struct iovec iov[], unsigned int iov_count, unsigned int *out_num, unsigned int *in_num, @@ -261,6 +258,8 @@ static inline void vhost_vq_set_backend(struct vhost_virtqueue *vq, void *private_data) { vq->private_data = private_data; + vq->ndescs = 0; + vq->first_desc = 0; } /** From patchwork Mon Jun 8 12:53:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 217939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 170DCC433E0 for ; Mon, 8 Jun 2020 12:53:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E1D2D206C3 for ; Mon, 8 Jun 2020 12:53:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZdOB4Xij" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729580AbgFHMxL (ORCPT ); Mon, 8 Jun 2020 08:53:11 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:52836 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729319AbgFHMxG (ORCPT ); Mon, 8 Jun 2020 08:53:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591620785; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YC9C1bg85UqEARLoLzVVH0wVxPzUMhgdZVeMnCOVW8I=; b=ZdOB4XijOdoagcKqSp1IuyScqjZPiXAQOir8D2sxtrZVsXu/9w5TmnywilpC3cJ1/dPcwb ucf/NhPII+A4cz+kni6Gwi6C7lxhl5+8cH8AqkkSR3GypjRlM6gfFTesMeee5hJiWjCZUM BsPPDYR9ED+av+veP0rexnAl4LdRx7c= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-463-5rv3m6_AP0ylpVf6oM86eQ-1; Mon, 08 Jun 2020 08:53:03 -0400 X-MC-Unique: 5rv3m6_AP0ylpVf6oM86eQ-1 Received: by mail-wm1-f70.google.com with SMTP id u15so3900819wmm.5 for ; Mon, 08 Jun 2020 05:53:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=YC9C1bg85UqEARLoLzVVH0wVxPzUMhgdZVeMnCOVW8I=; b=hnShubdQll3loxBziR/XSaQUPnTtCC+hB5R4hoVa7eFQlBxbogMvwTO+pDca+qmGDr Kd+/ZcDUfEjO4Mfkf+oWYN+1Aiq/5PIMRJCjzgljgZK3/74Ft8jlOJTrWiHkOeg3MxUs TuvVJa21H+PzzejC9Pjb/QUyUCJ6G0dmj9w6+NAwN9kWWCHxvwUajLEDQcR6oWDZZUZg atqfEZm4kVHnk/znGpXAWV0cbq+xDoopYnoQ+yi1mnww+v1Lnbdln8SaUjYGppA/+Cbj b+j0K2b2vJTHD9HSRbn/214IgI1lmr5U1I9vVh+cMAnDi60rHWcWDj9e1Y3BtFn/NCLR XbhQ== X-Gm-Message-State: AOAM531KMCBt1Qx9xbTxOArlPNLEhfVZ/VtRWDgNEw0vhOQ9mlzouKbl tQ6DvTasABOHGzj6w1MrNM1tpHHq224eAac0mraAIOV+wBApFFalyzjepmVxBpMU+mtgQkuKUl5 MBLxMO3vohKUSPDmM X-Received: by 2002:a1c:59c7:: with SMTP id n190mr15634969wmb.61.1591620782175; Mon, 08 Jun 2020 05:53:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJycvWcSZ9pgrB8OOPKZQyruJy8vStQkFo1WrrMhroFInHbRmdrcF8cq2oRf/qnvmhPPXJjLnQ== X-Received: by 2002:a1c:59c7:: with SMTP id n190mr15634952wmb.61.1591620781966; Mon, 08 Jun 2020 05:53:01 -0700 (PDT) Received: from redhat.com (bzq-109-64-41-91.red.bezeqint.net. [109.64.41.91]) by smtp.gmail.com with ESMTPSA id r12sm23319300wrc.22.2020.06.08.05.53.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2020 05:53:01 -0700 (PDT) Date: Mon, 8 Jun 2020 08:53:00 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v6 04/11] vhost: reorder functions Message-ID: <20200608125238.728563-5-mst@redhat.com> References: <20200608125238.728563-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200608125238.728563-1-mst@redhat.com> X-Mailer: git-send-email 2.24.1.751.gd10ce2899c X-Mutt-Fcc: =sent Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Reorder functions in the file to not rely on forward declarations, in preparation to making them static down the road. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 41d6b132c234..334529ebecab 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2429,19 +2429,6 @@ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) } EXPORT_SYMBOL_GPL(vhost_discard_vq_desc); -/* After we've used one of their buffers, we tell them about it. We'll then - * want to notify the guest, using eventfd. */ -int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) -{ - struct vring_used_elem heads = { - cpu_to_vhost32(vq, head), - cpu_to_vhost32(vq, len) - }; - - return vhost_add_used_n(vq, &heads, 1); -} -EXPORT_SYMBOL_GPL(vhost_add_used); - static int __vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, unsigned count) @@ -2511,6 +2498,19 @@ int vhost_add_used_n(struct vhost_virtqueue *vq, struct vring_used_elem *heads, } EXPORT_SYMBOL_GPL(vhost_add_used_n); +/* After we've used one of their buffers, we tell them about it. We'll then + * want to notify the guest, using eventfd. */ +int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) +{ + struct vring_used_elem heads = { + cpu_to_vhost32(vq, head), + cpu_to_vhost32(vq, len) + }; + + return vhost_add_used_n(vq, &heads, 1); +} +EXPORT_SYMBOL_GPL(vhost_add_used); + static bool vhost_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq) { __u16 old, new; From patchwork Mon Jun 8 12:53:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 217937 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EFFAC433DF for ; Mon, 8 Jun 2020 12:54:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 70ED3206C3 for ; Mon, 8 Jun 2020 12:54:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="g1P9o+Fr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729787AbgFHMxy (ORCPT ); Mon, 8 Jun 2020 08:53:54 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:26680 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729719AbgFHMxY (ORCPT ); Mon, 8 Jun 2020 08:53:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591620801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=/BMsXCp8as6XIEKwP+1OuBgzlgZF+LObUXAj5LC6nhc=; b=g1P9o+FrIgEk6EEQjKYoOTMZNbK4xK9vu3jbP6aIe3ACQIhmZghy06Wgvr2uzqvZFQASwr btdbF/8nG8I+4Vm7/0+BCHb+A3fJKFRCR/pe0KJG/kXbF/G9KhoV532GtiL1fdoe7zxoAq Ufvc14cxn0tbrGiVTmE6ZMrT4jrF0iE= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-510-_1eZspQXPK2AEXNWMKV9og-1; Mon, 08 Jun 2020 08:53:05 -0400 X-MC-Unique: _1eZspQXPK2AEXNWMKV9og-1 Received: by mail-wr1-f70.google.com with SMTP id r5so7135776wrt.9 for ; Mon, 08 Jun 2020 05:53:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=/BMsXCp8as6XIEKwP+1OuBgzlgZF+LObUXAj5LC6nhc=; b=n1IU8lew1PSXCkuXyEnhw1OOtnlmwRti18c3UmN0NGpWOSeA6OLcEdfX9K3gych6pn sqmSKqUotvaVp25CtHCy4b5x+GERt9NZ9wL5K+UE6QuWGy2Xfudh71oowWik4Hf/RBFl LWjA0auRKWgKNuOFMRYlBj4OzCboQssm9o4nMQv8j1ImfwmEJJE9L9GaVD9d0ofZ9chd zRv746n+H3BuNQ8RfdQzRxmZ3qjMU4pi+Kx9Qw6ykBnA9GJUA4wyhMtufr+k5t5ybiHi JgrY3uqATX6wzKn5tNiHDTOKT0rAuGyIBkGWCcAuYTYSbQZtp6FhHyO7p1EBTZdTKcsc jZxg== X-Gm-Message-State: AOAM532arwnF2qR92hUcJmlLwIH4CHk7joBSTU5QoIBa8JVrE544UBeN F03hPNxfU+/IU/PRDBHQ/ss11VeVhGXZzjV3VMD1gHpEnl/4AaEtf6zy5sWdTcIFpwJcygvGDAi TXPxIBHW0MtLuZJ+5 X-Received: by 2002:a5d:40d0:: with SMTP id b16mr23153087wrq.218.1591620784268; Mon, 08 Jun 2020 05:53:04 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw2VxXE5VOIOVPo0RMboZTneqAdwSqXaMGZre+l+cehJC8Sptf2UOfLNmzLTRHxVBL8MWleKA== X-Received: by 2002:a5d:40d0:: with SMTP id b16mr23153071wrq.218.1591620784029; Mon, 08 Jun 2020 05:53:04 -0700 (PDT) Received: from redhat.com (bzq-109-64-41-91.red.bezeqint.net. [109.64.41.91]) by smtp.gmail.com with ESMTPSA id q4sm22610888wma.47.2020.06.08.05.53.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2020 05:53:03 -0700 (PDT) Date: Mon, 8 Jun 2020 08:53:02 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v6 05/11] vhost: format-independent API for used buffers Message-ID: <20200608125238.728563-6-mst@redhat.com> References: <20200608125238.728563-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200608125238.728563-1-mst@redhat.com> X-Mailer: git-send-email 2.24.1.751.gd10ce2899c X-Mutt-Fcc: =sent Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add a new API that doesn't assume used ring, heads, etc. For now, we keep the old APIs around to make it easier to convert drivers. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 52 ++++++++++++++++++++++++++++++++++--------- drivers/vhost/vhost.h | 17 +++++++++++++- 2 files changed, 58 insertions(+), 11 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 334529ebecab..f4a6ff9ef77a 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2339,13 +2339,12 @@ static void unfetch_descs(struct vhost_virtqueue *vq) * number of output then some number of input descriptors, it's actually two * iovecs, but we pack them into one and note how many of each there were. * - * This function returns the descriptor number found, or vq->num (which is - * never a valid descriptor number) if none was found. A negative code is - * returned on error. */ -int vhost_get_vq_desc(struct vhost_virtqueue *vq, - struct iovec iov[], unsigned int iov_size, - unsigned int *out_num, unsigned int *in_num, - struct vhost_log *log, unsigned int *log_num) + * This function returns a value > 0 if a descriptor was found, or 0 if none were found. + * A negative code is returned on error. */ +int vhost_get_avail_buf(struct vhost_virtqueue *vq, struct vhost_buf *buf, + struct iovec iov[], unsigned int iov_size, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num) { int ret = fetch_descs(vq); int i; @@ -2358,6 +2357,8 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, *out_num = *in_num = 0; if (unlikely(log)) *log_num = 0; + buf->in_len = buf->out_len = 0; + buf->descs = 0; for (i = vq->first_desc; i < vq->ndescs; ++i) { unsigned iov_count = *in_num + *out_num; @@ -2387,6 +2388,7 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, /* If this is an input descriptor, * increment that count. */ *in_num += ret; + buf->in_len += desc->len; if (unlikely(log && ret)) { log[*log_num].addr = desc->addr; log[*log_num].len = desc->len; @@ -2402,9 +2404,11 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, goto err; } *out_num += ret; + buf->out_len += desc->len; } - ret = desc->id; + buf->id = desc->id; + ++buf->descs; if (!(desc->flags & VRING_DESC_F_NEXT)) break; @@ -2412,14 +2416,22 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, vq->first_desc = i + 1; - return ret; + return 1; err: unfetch_descs(vq); return ret; } -EXPORT_SYMBOL_GPL(vhost_get_vq_desc); +EXPORT_SYMBOL_GPL(vhost_get_avail_buf); + +/* Reverse the effect of vhost_get_avail_buf. Useful for error handling. */ +void vhost_discard_avail_bufs(struct vhost_virtqueue *vq, + struct vhost_buf *buf, unsigned count) +{ + vhost_discard_vq_desc(vq, count); +} +EXPORT_SYMBOL_GPL(vhost_discard_avail_bufs); /* Reverse the effect of vhost_get_vq_desc. Useful for error handling. */ void vhost_discard_vq_desc(struct vhost_virtqueue *vq, int n) @@ -2511,6 +2523,26 @@ int vhost_add_used(struct vhost_virtqueue *vq, unsigned int head, int len) } EXPORT_SYMBOL_GPL(vhost_add_used); +int vhost_put_used_buf(struct vhost_virtqueue *vq, struct vhost_buf *buf) +{ + return vhost_add_used(vq, buf->id, buf->in_len); +} +EXPORT_SYMBOL_GPL(vhost_put_used_buf); + +int vhost_put_used_n_bufs(struct vhost_virtqueue *vq, + struct vhost_buf *bufs, unsigned count) +{ + unsigned i; + + for (i = 0; i < count; ++i) { + vq->heads[i].id = cpu_to_vhost32(vq, bufs[i].id); + vq->heads[i].len = cpu_to_vhost32(vq, bufs[i].in_len); + } + + return vhost_add_used_n(vq, vq->heads, count); +} +EXPORT_SYMBOL_GPL(vhost_put_used_n_bufs); + static bool vhost_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq) { __u16 old, new; diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index fed36af5c444..28eea0155efb 100644 --- a/drivers/vhost/vhost.h +++ b/drivers/vhost/vhost.h @@ -67,6 +67,13 @@ struct vhost_desc { u16 id; }; +struct vhost_buf { + u32 out_len; + u32 in_len; + u16 descs; + u16 id; +}; + /* The virtqueue structure describes a queue attached to a device. */ struct vhost_virtqueue { struct vhost_dev *dev; @@ -195,7 +202,12 @@ int vhost_get_vq_desc(struct vhost_virtqueue *, unsigned int *out_num, unsigned int *in_num, struct vhost_log *log, unsigned int *log_num); void vhost_discard_vq_desc(struct vhost_virtqueue *, int n); - +int vhost_get_avail_buf(struct vhost_virtqueue *, struct vhost_buf *buf, + struct iovec iov[], unsigned int iov_count, + unsigned int *out_num, unsigned int *in_num, + struct vhost_log *log, unsigned int *log_num); +void vhost_discard_avail_bufs(struct vhost_virtqueue *, + struct vhost_buf *, unsigned count); int vhost_vq_init_access(struct vhost_virtqueue *); int vhost_add_used(struct vhost_virtqueue *, unsigned int head, int len); int vhost_add_used_n(struct vhost_virtqueue *, struct vring_used_elem *heads, @@ -204,6 +216,9 @@ void vhost_add_used_and_signal(struct vhost_dev *, struct vhost_virtqueue *, unsigned int id, int len); void vhost_add_used_and_signal_n(struct vhost_dev *, struct vhost_virtqueue *, struct vring_used_elem *heads, unsigned count); +int vhost_put_used_buf(struct vhost_virtqueue *, struct vhost_buf *buf); +int vhost_put_used_n_bufs(struct vhost_virtqueue *, + struct vhost_buf *bufs, unsigned count); void vhost_signal(struct vhost_dev *, struct vhost_virtqueue *); void vhost_disable_notify(struct vhost_dev *, struct vhost_virtqueue *); bool vhost_vq_avail_empty(struct vhost_dev *, struct vhost_virtqueue *); From patchwork Mon Jun 8 12:53:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 217935 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56032C433DF for ; Mon, 8 Jun 2020 12:55:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 319CE2076A for ; Mon, 8 Jun 2020 12:55:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cdK3hBal" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729837AbgFHMyd (ORCPT ); Mon, 8 Jun 2020 08:54:33 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:51326 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729315AbgFHMxP (ORCPT ); Mon, 8 Jun 2020 08:53:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591620793; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nNh1TL7xZUtXOusVSg8tIG35kTb/6beDbMDeGMbkL8c=; b=cdK3hBalYnQTZ2ivoNup29kX+WgRgCaAnDsw5wfyGBdugUnta7zGmGaDV+15nJQns5GPRJ 3GryBGOPIok8qhGcXLqe2hE9foMQtYKpzJzxjWt4OCmt5QWp2ExK7hHLytKKm+6LttVg4K zxoZmC//Jkzvnn47x3BXU0ECh6klymA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-317-T0ZYK9QkPBiWS5TMZm7DpQ-1; Mon, 08 Jun 2020 08:53:11 -0400 X-MC-Unique: T0ZYK9QkPBiWS5TMZm7DpQ-1 Received: by mail-wr1-f72.google.com with SMTP id h6so7166789wrx.4 for ; Mon, 08 Jun 2020 05:53:11 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=nNh1TL7xZUtXOusVSg8tIG35kTb/6beDbMDeGMbkL8c=; b=IcSVvp6KifOuvN8M6XlksBsZ9Eb+qy1ah0gLXLYcDHHOhuE/7ibzyy1OVsYrq7uVTC oqeybvZ/EkaT7BDxscvE93YQBQbelPg24FRnD2d6UEWUfnQnfj6GD/Mg0R41HGSxuGSJ Eg56rWDcyN7ist1PYAHxP3v5DvyjszfuuI2D8CZG3MD17QJcCaUv38Q762YnehdMj4S5 9V9U/PbXh09f1UmZup3zBso9DYZDMHBwWY5RN8MBenkQGk/oh6D48Cae3Wf88oNMzyRF XCJq0xP8hs/eFiqZTM8OpgIGmF0oXH2XuCFPs4U791kEW69fqyDZ5BUe20gwHxh6+LJB TOww== X-Gm-Message-State: AOAM533klTy3o23tufGOFDgpsh+NiDKPGDSwk/6I87M6Js/FAs5txR7y G31WmZykExdDBjfRY1P2WTWQcbM0cMFqsiFQXsWhq7urH6ejoZ9j1Tu6NdI2LoifgJ06TR+Agqz UMMWQxlUc4J79mwHV X-Received: by 2002:a5d:4b47:: with SMTP id w7mr22414162wrs.234.1591620790802; Mon, 08 Jun 2020 05:53:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJweTCKDLbfiVGUFqWREKS9iQ8j4XUwzsDXWV07w/+PxSXiYAJVsAr7kpiUE/TGQ+cExzBaktw== X-Received: by 2002:a5d:4b47:: with SMTP id w7mr22414144wrs.234.1591620790552; Mon, 08 Jun 2020 05:53:10 -0700 (PDT) Received: from redhat.com (bzq-109-64-41-91.red.bezeqint.net. [109.64.41.91]) by smtp.gmail.com with ESMTPSA id h15sm22578283wrt.73.2020.06.08.05.53.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2020 05:53:10 -0700 (PDT) Date: Mon, 8 Jun 2020 08:53:08 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com Subject: [PATCH RFC v6 08/11] vhost/test: convert to the buf API Message-ID: <20200608125238.728563-9-mst@redhat.com> References: <20200608125238.728563-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200608125238.728563-1-mst@redhat.com> X-Mailer: git-send-email 2.24.1.751.gd10ce2899c X-Mutt-Fcc: =sent Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Signed-off-by: Michael S. Tsirkin --- drivers/vhost/test.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/drivers/vhost/test.c b/drivers/vhost/test.c index 7d69778aaa26..12304eb8da15 100644 --- a/drivers/vhost/test.c +++ b/drivers/vhost/test.c @@ -44,9 +44,10 @@ static void handle_vq(struct vhost_test *n) { struct vhost_virtqueue *vq = &n->vqs[VHOST_TEST_VQ]; unsigned out, in; - int head; + int ret; size_t len, total_len = 0; void *private; + struct vhost_buf buf; mutex_lock(&vq->mutex); private = vhost_vq_get_backend(vq); @@ -58,15 +59,15 @@ static void handle_vq(struct vhost_test *n) vhost_disable_notify(&n->dev, vq); for (;;) { - head = vhost_get_vq_desc(vq, vq->iov, - ARRAY_SIZE(vq->iov), - &out, &in, - NULL, NULL); + ret = vhost_get_avail_buf(vq, &buf, vq->iov, + ARRAY_SIZE(vq->iov), + &out, &in, + NULL, NULL); /* On error, stop handling until the next kick. */ - if (unlikely(head < 0)) + if (unlikely(ret < 0)) break; /* Nothing new? Wait for eventfd to tell us they refilled. */ - if (head == vq->num) { + if (!ret) { if (unlikely(vhost_enable_notify(&n->dev, vq))) { vhost_disable_notify(&n->dev, vq); continue; @@ -78,13 +79,14 @@ static void handle_vq(struct vhost_test *n) "out %d, int %d\n", out, in); break; } - len = iov_length(vq->iov, out); + len = buf.out_len; /* Sanity check */ if (!len) { vq_err(vq, "Unexpected 0 len for TX\n"); break; } - vhost_add_used_and_signal(&n->dev, vq, head, 0); + vhost_put_used_buf(vq, &buf); + vhost_signal(&n->dev, vq); total_len += len; if (unlikely(vhost_exceeds_weight(vq, 0, total_len))) break; From patchwork Mon Jun 8 12:53:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 217938 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CC18C433DF for ; Mon, 8 Jun 2020 12:53:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4FEC7206C3 for ; Mon, 8 Jun 2020 12:53:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bvmfN3v+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729754AbgFHMxd (ORCPT ); Mon, 8 Jun 2020 08:53:33 -0400 Received: from us-smtp-1.mimecast.com ([207.211.31.81]:40317 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1729696AbgFHMxV (ORCPT ); Mon, 8 Jun 2020 08:53:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1591620798; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=TEosg1cbEhe9hhKHYKqZg2N/D89Ri/A+CQJLw8Cq4Bg=; b=bvmfN3v+w6ikhXyCNHkoWNLwdTWu9IdmgCnc4ftx6Dz8lhUTnHYT7h5RxWKw2GIUrFJKgy 24Iz8Dm5czNCSVusW8d7fhgEGIZ5tQTtES0SA5/nNQ/ih5MgNwpbtk8NQWmGQAqkMuLBvD WXRPJQMR+bnz022eKkKTfLZiBKj3Rj8= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-141-3fldm4uyM1OTB3qiMHYdeg-1; Mon, 08 Jun 2020 08:53:16 -0400 X-MC-Unique: 3fldm4uyM1OTB3qiMHYdeg-1 Received: by mail-wm1-f70.google.com with SMTP id p24so5217939wmc.1 for ; Mon, 08 Jun 2020 05:53:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=TEosg1cbEhe9hhKHYKqZg2N/D89Ri/A+CQJLw8Cq4Bg=; b=MEg+gB8OlkMNuDY/SaHUez/45S3pUd60CZ6awNprzFiE6TCAkjEXsCHgvYJaLwZ27W XTlxSbyz61WhXToccRp+AkIqX5SmagxHj4XGtOb2Y75DBSKNDIk0LeYUrofVzS34FHXA jkoR8tee8AupJDAYbpYSCPHHIE/o+gQO8wWwohAd4KWEdaknJ0SDvUXA8FVWBqkk3gjs 1jW+MWw15Nmw/gEGOJUDvuYjXB+NlpZA9NYBF0NhsFiNh7T8f01dVFlQL7pVwmLX93ni tt+rcm22Q0td9b11P+B8W07XCCiNPU72mKLAcS1Ow2p8Rjud27ArWu1Fp+WenWhDPOBf nhlA== X-Gm-Message-State: AOAM532EjxusQq4yVnHIjnRqq8KYZdZ7SzGEOP8ooP/n3yNA920EnB2Y iisRdk95U7gOUkc0zkjCGfISi8+WJ7X8BALFhB1nSILebIqsvzVC9Su6hRd8PZFDn7YiwzJcMDH wLslwdILVBrrcxQGZ X-Received: by 2002:adf:9d8e:: with SMTP id p14mr22776196wre.236.1591620795493; Mon, 08 Jun 2020 05:53:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzojN0Rni5a/KRQQtayEgafAMQjL2u7TFZEVEHmdO1lnxFxUgOcYrWuJF226hamluzbsGThDA== X-Received: by 2002:adf:9d8e:: with SMTP id p14mr22776176wre.236.1591620795249; Mon, 08 Jun 2020 05:53:15 -0700 (PDT) Received: from redhat.com (bzq-109-64-41-91.red.bezeqint.net. [109.64.41.91]) by smtp.gmail.com with ESMTPSA id i10sm22891380wrw.51.2020.06.08.05.53.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2020 05:53:14 -0700 (PDT) Date: Mon, 8 Jun 2020 08:53:13 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: kvm@vger.kernel.org, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, Jason Wang , eperezma@redhat.com, Stefan Hajnoczi , Stefano Garzarella Subject: [PATCH RFC v6 10/11] vhost/vsock: switch to the buf API Message-ID: <20200608125238.728563-11-mst@redhat.com> References: <20200608125238.728563-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20200608125238.728563-1-mst@redhat.com> X-Mailer: git-send-email 2.24.1.751.gd10ce2899c X-Mutt-Fcc: =sent Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org A straight-forward conversion. Signed-off-by: Michael S. Tsirkin --- drivers/vhost/vsock.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index a483cec31d5c..61c6d3dd2ae3 100644 --- a/drivers/vhost/vsock.c +++ b/drivers/vhost/vsock.c @@ -103,7 +103,8 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, unsigned out, in; size_t nbytes; size_t iov_len, payload_len; - int head; + struct vhost_buf buf; + int ret; spin_lock_bh(&vsock->send_pkt_list_lock); if (list_empty(&vsock->send_pkt_list)) { @@ -117,16 +118,17 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, list_del_init(&pkt->list); spin_unlock_bh(&vsock->send_pkt_list_lock); - head = vhost_get_vq_desc(vq, vq->iov, ARRAY_SIZE(vq->iov), - &out, &in, NULL, NULL); - if (head < 0) { + ret = vhost_get_avail_buf(vq, &buf, + vq->iov, ARRAY_SIZE(vq->iov), + &out, &in, NULL, NULL); + if (ret < 0) { spin_lock_bh(&vsock->send_pkt_list_lock); list_add(&pkt->list, &vsock->send_pkt_list); spin_unlock_bh(&vsock->send_pkt_list_lock); break; } - if (head == vq->num) { + if (!ret) { spin_lock_bh(&vsock->send_pkt_list_lock); list_add(&pkt->list, &vsock->send_pkt_list); spin_unlock_bh(&vsock->send_pkt_list_lock); @@ -186,7 +188,8 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock, */ virtio_transport_deliver_tap_pkt(pkt); - vhost_add_used(vq, head, sizeof(pkt->hdr) + payload_len); + buf.in_len = sizeof(pkt->hdr) + payload_len; + vhost_put_used_buf(vq, &buf); added = true; pkt->off += payload_len; @@ -440,7 +443,8 @@ static void vhost_vsock_handle_tx_kick(struct vhost_work *work) struct vhost_vsock *vsock = container_of(vq->dev, struct vhost_vsock, dev); struct virtio_vsock_pkt *pkt; - int head, pkts = 0, total_len = 0; + int ret, pkts = 0, total_len = 0; + struct vhost_buf buf; unsigned int out, in; bool added = false; @@ -461,12 +465,13 @@ static void vhost_vsock_handle_tx_kick(struct vhost_work *work) goto no_more_replies; } - head = vhost_get_vq_desc(vq, vq->iov, ARRAY_SIZE(vq->iov), - &out, &in, NULL, NULL); - if (head < 0) + ret = vhost_get_avail_buf(vq, &buf, + vq->iov, ARRAY_SIZE(vq->iov), + &out, &in, NULL, NULL); + if (ret < 0) break; - if (head == vq->num) { + if (!ret) { if (unlikely(vhost_enable_notify(&vsock->dev, vq))) { vhost_disable_notify(&vsock->dev, vq); continue; @@ -494,7 +499,8 @@ static void vhost_vsock_handle_tx_kick(struct vhost_work *work) virtio_transport_free_pkt(pkt); len += sizeof(pkt->hdr); - vhost_add_used(vq, head, len); + buf.in_len = len; + vhost_put_used_buf(vq, &buf); total_len += len; added = true; } while(likely(!vhost_exceeds_weight(vq, ++pkts, total_len)));