diff mbox series

[v2,1/4] hw/virtio: incorporate backend features in features

Message ID 20220728135503.1060062-2-alex.bennee@linaro.org
State Superseded
Headers show
Series virtio fixes (split from virtio-gpio series) | expand

Commit Message

Alex Bennée July 28, 2022, 1:55 p.m. UTC
There are some extra bits used over a vhost-user connection which are
hidden from the device itself. We need to set them here to ensure we
enable things like the protocol extensions.

Currently net/vhost-user.c has it's own inscrutable way of persisting
this data but it really should live in the core vhost_user code.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org>
---
 hw/virtio/vhost-user.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

Comments

Michael S. Tsirkin Aug. 17, 2022, 11 a.m. UTC | #1
On Thu, Jul 28, 2022 at 02:55:00PM +0100, Alex Bennée wrote:
> There are some extra bits used over a vhost-user connection which are
> hidden from the device itself. We need to set them here to ensure we
> enable things like the protocol extensions.
> 
> Currently net/vhost-user.c has it's own inscrutable way of persisting
> this data but it really should live in the core vhost_user code.
> 
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org>

I went back and forth on this, but in the end I feel
it is safer to defer this one until after the release.

> ---
>  hw/virtio/vhost-user.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
> index 75b8df21a4..1936a44e82 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev,
>       */
>      bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
>  
> -    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features,
> +    /*
> +     * We need to include any extra backend only feature bits that
> +     * might be needed by our device. Currently this includes the
> +     * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
> +     * features.
> +     */
> +    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
> +                              features | dev->backend_features,
>                                log_enabled);
>  }
>  
> -- 
> 2.30.2
Yajun Wu Oct. 26, 2022, 6:11 a.m. UTC | #2
Hi Alex,

With this change, VHOST_USER_F_PROTOCOL_FEATURES bit will be set to backend for virtio block device (previously not).

From https://www.qemu.org/docs/master/interop/vhost-user.html spec:
If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, the ring starts directly in the enabled state.
If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is initialized in a disabled state and is enabled by VHOST_USER_SET_VRING_ENABLE with parameter 1.

Vhost-user-blk won't send out VHOST_USER_SET_VRING_ENABLE today. 
Backend gets VHOST_USER_F_PROTOCOL_FEATURES negotiated and can't get VHOST_USER_SET_VRING_ENABLE.
VQs keep in disabled state.

Can you check on this scenario?

Thanks

-----Original Message-----
From: Qemu-devel <qemu-devel-bounces+yajunw=nvidia.com@nongnu.org> On Behalf Of Alex Bennée
Sent: Thursday, July 28, 2022 9:55 PM
To: qemu-devel@nongnu.org
Cc: mst@redhat.com; Alex Bennée <alex.bennee@linaro.org>
Subject: [PATCH v2 1/4] hw/virtio: incorporate backend features in features

External email: Use caution opening links or attachments


There are some extra bits used over a vhost-user connection which are hidden from the device itself. We need to set them here to ensure we enable things like the protocol extensions.

Currently net/vhost-user.c has it's own inscrutable way of persisting this data but it really should live in the core vhost_user code.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org>
---
 hw/virtio/vhost-user.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 75b8df21a4..1936a44e82 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev,
      */
     bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);

-    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features,
+    /*
+     * We need to include any extra backend only feature bits that
+     * might be needed by our device. Currently this includes the
+     * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
+     * features.
+     */
+    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
+                              features | dev->backend_features,
                               log_enabled);  }

--
2.30.2
Alex Bennée Oct. 26, 2022, 7:41 a.m. UTC | #3
Yajun Wu <yajunw@nvidia.com> writes:

> Hi Alex,
>
> With this change, VHOST_USER_F_PROTOCOL_FEATURES bit will be set to
> backend for virtio block device (previously not).
>
> From https://www.qemu.org/docs/master/interop/vhost-user.html spec:
> If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, the ring starts directly in the enabled state.
> If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is
> initialized in a disabled state and is enabled by
> VHOST_USER_SET_VRING_ENABLE with parameter 1.
>
> Vhost-user-blk won't send out VHOST_USER_SET_VRING_ENABLE today. 
> Backend gets VHOST_USER_F_PROTOCOL_FEATURES negotiated and can't get VHOST_USER_SET_VRING_ENABLE.
> VQs keep in disabled state.

If the backend advertises protocol features but the stub doesn't support
it how does it get enabled?

The testing I did was mostly by hand with the gpio backend and using the
qtests. I Think we need to add some acceptance testing into avocado with
some real daemons because I don't think we have enough coverage with the
current qtest approach.

>
> Can you check on this scenario?
>
> Thanks
>
> -----Original Message-----
> From: Qemu-devel <qemu-devel-bounces+yajunw=nvidia.com@nongnu.org> On Behalf Of Alex Bennée
> Sent: Thursday, July 28, 2022 9:55 PM
> To: qemu-devel@nongnu.org
> Cc: mst@redhat.com; Alex Bennée <alex.bennee@linaro.org>
> Subject: [PATCH v2 1/4] hw/virtio: incorporate backend features in features
>
> External email: Use caution opening links or attachments
>
>
> There are some extra bits used over a vhost-user connection which are hidden from the device itself. We need to set them here to ensure we enable things like the protocol extensions.
>
> Currently net/vhost-user.c has it's own inscrutable way of persisting this data but it really should live in the core vhost_user code.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org>
> ---
>  hw/virtio/vhost-user.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 75b8df21a4..1936a44e82 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev,
>       */
>      bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
>
> -    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features,
> +    /*
> +     * We need to include any extra backend only feature bits that
> +     * might be needed by our device. Currently this includes the
> +     * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
> +     * features.
> +     */
> +    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
> +                              features | dev->backend_features,
>                                log_enabled);  }
Yajun Wu Nov. 10, 2022, 10:35 a.m. UTC | #4
Hi Alex,

Sorry for the late response, I missed your mail.
You can test together with dpdk and have reproduce.

Steps:
1. DPDK 
git clone https://github.com/DPDK/dpdk.git
git checkout v22.07
meson build -Dexamples=vhost_blk
ninja -C build
cd /tmp/
sudo dpdk/build/examples/dpdk-vhost_blk # it's a daemon 

2. Add blk device to qemu, then bootup VM.
  <qemu:commandline>
    <qemu:arg value='-chardev'/>
    <qemu:arg value='socket,id=char0,path=/tmp/vhost.socket'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vhost-user-blk-pci,chardev=char0,num-queues=1'/>
  </qemu:commandline>

Without this commit, you can get device ready log from dpdk-vhost_blk:

VHOST_CONFIG: (/tmp/vhost.socket) negotiated Vhost-user protocol features: 0x11ebf
VHOST_CONFIG: (/tmp/vhost.socket) negotiated Virtio features: 0x100000000
VHOST_CONFIG: (/tmp/vhost.socket) virtio is now ready for processing.
New Device /tmp/vhost.socket, Device ID 0
Ctrlr Worker Thread start

With this commit, device won't be ready and VM will hang. 
You can see VHOST_USER_F_PROTOCOL_FEATURES bit is added.

VHOST_CONFIG: (/tmp/vhost.socket) negotiated Virtio features: 0x140000000

Dpdk code related:
./lib/vhost/vhost_user.c

2044     /*                                                                          
2045      * When VHOST_USER_F_PROTOCOL_FEATURES is not negotiated,                   
2046      * the ring starts already enabled. Otherwise, it is enabled via            
2047      * the SET_VRING_ENABLE message.                                            
2048      */                                                                         
2049     if (!(dev->features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES))) {          
2050         vq->enabled = true;                                                     
2051     }


Thanks,
Yajun 




-----Original Message-----
From: Alex Bennée <alex.bennee@linaro.org> 
Sent: Wednesday, October 26, 2022 3:42 PM
To: Yajun Wu <yajunw@nvidia.com>
Cc: qemu-devel@nongnu.org; mst@redhat.com; Parav Pandit <parav@nvidia.com>
Subject: Re: [PATCH v2 1/4] hw/virtio: incorporate backend features in features

External email: Use caution opening links or attachments


Yajun Wu <yajunw@nvidia.com> writes:

> Hi Alex,
>
> With this change, VHOST_USER_F_PROTOCOL_FEATURES bit will be set to 
> backend for virtio block device (previously not).
>
> From https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.qemu.org%2Fdocs%2Fmaster%2Finterop%2Fvhost-user.html&amp;data=05%7C01%7Cyajunw%40nvidia.com%7C2e8901540bf441248ec608dab725ca87%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638023670196631779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=y5g9wwTfh%2BxrkESRypoo4pg3eKYInyDerDmI844PBSE%3D&amp;reserved=0 spec:
> If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, the ring starts directly in the enabled state.
> If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is 
> initialized in a disabled state and is enabled by 
> VHOST_USER_SET_VRING_ENABLE with parameter 1.
>
> Vhost-user-blk won't send out VHOST_USER_SET_VRING_ENABLE today.
> Backend gets VHOST_USER_F_PROTOCOL_FEATURES negotiated and can't get VHOST_USER_SET_VRING_ENABLE.
> VQs keep in disabled state.

If the backend advertises protocol features but the stub doesn't support it how does it get enabled?

The testing I did was mostly by hand with the gpio backend and using the qtests. I Think we need to add some acceptance testing into avocado with some real daemons because I don't think we have enough coverage with the current qtest approach.

>
> Can you check on this scenario?
>
> Thanks
>
> -----Original Message-----
> From: Qemu-devel <qemu-devel-bounces+yajunw=nvidia.com@nongnu.org> On 
> Behalf Of Alex Bennée
> Sent: Thursday, July 28, 2022 9:55 PM
> To: qemu-devel@nongnu.org
> Cc: mst@redhat.com; Alex Bennée <alex.bennee@linaro.org>
> Subject: [PATCH v2 1/4] hw/virtio: incorporate backend features in 
> features
>
> External email: Use caution opening links or attachments
>
>
> There are some extra bits used over a vhost-user connection which are hidden from the device itself. We need to set them here to ensure we enable things like the protocol extensions.
>
> Currently net/vhost-user.c has it's own inscrutable way of persisting this data but it really should live in the core vhost_user code.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org>
> ---
>  hw/virtio/vhost-user.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 
> 75b8df21a4..1936a44e82 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev,
>       */
>      bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
>
> -    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features,
> +    /*
> +     * We need to include any extra backend only feature bits that
> +     * might be needed by our device. Currently this includes the
> +     * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
> +     * features.
> +     */
> +    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
> +                              features | dev->backend_features,
>                                log_enabled);  }


--
Alex Bennée
diff mbox series

Patch

diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
index 75b8df21a4..1936a44e82 100644
--- a/hw/virtio/vhost-user.c
+++ b/hw/virtio/vhost-user.c
@@ -1460,7 +1460,14 @@  static int vhost_user_set_features(struct vhost_dev *dev,
      */
     bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
 
-    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features,
+    /*
+     * We need to include any extra backend only feature bits that
+     * might be needed by our device. Currently this includes the
+     * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
+     * features.
+     */
+    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
+                              features | dev->backend_features,
                               log_enabled);
 }