| Message ID | 20220728135503.1060062-2-alex.bennee@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
| Series | virtio fixes (split from virtio-gpio series) | expand |
On Thu, Jul 28, 2022 at 02:55:00PM +0100, Alex Bennée wrote: > There are some extra bits used over a vhost-user connection which are > hidden from the device itself. We need to set them here to ensure we > enable things like the protocol extensions. > > Currently net/vhost-user.c has it's own inscrutable way of persisting > this data but it really should live in the core vhost_user code. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org> I went back and forth on this, but in the end I feel it is safer to defer this one until after the release. > --- > hw/virtio/vhost-user.c | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c > index 75b8df21a4..1936a44e82 100644 > --- a/hw/virtio/vhost-user.c > +++ b/hw/virtio/vhost-user.c > @@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev, > */ > bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL); > > - return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features, > + /* > + * We need to include any extra backend only feature bits that > + * might be needed by our device. Currently this includes the > + * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol > + * features. > + */ > + return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, > + features | dev->backend_features, > log_enabled); > } > > -- > 2.30.2
Hi Alex, With this change, VHOST_USER_F_PROTOCOL_FEATURES bit will be set to backend for virtio block device (previously not). From https://www.qemu.org/docs/master/interop/vhost-user.html spec: If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, the ring starts directly in the enabled state. If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is initialized in a disabled state and is enabled by VHOST_USER_SET_VRING_ENABLE with parameter 1. Vhost-user-blk won't send out VHOST_USER_SET_VRING_ENABLE today. Backend gets VHOST_USER_F_PROTOCOL_FEATURES negotiated and can't get VHOST_USER_SET_VRING_ENABLE. VQs keep in disabled state. Can you check on this scenario? Thanks -----Original Message----- From: Qemu-devel <qemu-devel-bounces+yajunw=nvidia.com@nongnu.org> On Behalf Of Alex Bennée Sent: Thursday, July 28, 2022 9:55 PM To: qemu-devel@nongnu.org Cc: mst@redhat.com; Alex Bennée <alex.bennee@linaro.org> Subject: [PATCH v2 1/4] hw/virtio: incorporate backend features in features External email: Use caution opening links or attachments There are some extra bits used over a vhost-user connection which are hidden from the device itself. We need to set them here to ensure we enable things like the protocol extensions. Currently net/vhost-user.c has it's own inscrutable way of persisting this data but it really should live in the core vhost_user code. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org> --- hw/virtio/vhost-user.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 75b8df21a4..1936a44e82 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev, */ bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL); - return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features, + /* + * We need to include any extra backend only feature bits that + * might be needed by our device. Currently this includes the + * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol + * features. + */ + return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, + features | dev->backend_features, log_enabled); } -- 2.30.2
Yajun Wu <yajunw@nvidia.com> writes: > Hi Alex, > > With this change, VHOST_USER_F_PROTOCOL_FEATURES bit will be set to > backend for virtio block device (previously not). > > From https://www.qemu.org/docs/master/interop/vhost-user.html spec: > If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, the ring starts directly in the enabled state. > If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is > initialized in a disabled state and is enabled by > VHOST_USER_SET_VRING_ENABLE with parameter 1. > > Vhost-user-blk won't send out VHOST_USER_SET_VRING_ENABLE today. > Backend gets VHOST_USER_F_PROTOCOL_FEATURES negotiated and can't get VHOST_USER_SET_VRING_ENABLE. > VQs keep in disabled state. If the backend advertises protocol features but the stub doesn't support it how does it get enabled? The testing I did was mostly by hand with the gpio backend and using the qtests. I Think we need to add some acceptance testing into avocado with some real daemons because I don't think we have enough coverage with the current qtest approach. > > Can you check on this scenario? > > Thanks > > -----Original Message----- > From: Qemu-devel <qemu-devel-bounces+yajunw=nvidia.com@nongnu.org> On Behalf Of Alex Bennée > Sent: Thursday, July 28, 2022 9:55 PM > To: qemu-devel@nongnu.org > Cc: mst@redhat.com; Alex Bennée <alex.bennee@linaro.org> > Subject: [PATCH v2 1/4] hw/virtio: incorporate backend features in features > > External email: Use caution opening links or attachments > > > There are some extra bits used over a vhost-user connection which are hidden from the device itself. We need to set them here to ensure we enable things like the protocol extensions. > > Currently net/vhost-user.c has it's own inscrutable way of persisting this data but it really should live in the core vhost_user code. > > Signed-off-by: Alex Bennée <alex.bennee@linaro.org> > Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org> > --- > hw/virtio/vhost-user.c | 9 ++++++++- > 1 file changed, 8 insertions(+), 1 deletion(-) > > diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 75b8df21a4..1936a44e82 100644 > --- a/hw/virtio/vhost-user.c > +++ b/hw/virtio/vhost-user.c > @@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev, > */ > bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL); > > - return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features, > + /* > + * We need to include any extra backend only feature bits that > + * might be needed by our device. Currently this includes the > + * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol > + * features. > + */ > + return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, > + features | dev->backend_features, > log_enabled); }
Hi Alex,
Sorry for the late response, I missed your mail.
You can test together with dpdk and have reproduce.
Steps:
1. DPDK
git clone https://github.com/DPDK/dpdk.git
git checkout v22.07
meson build -Dexamples=vhost_blk
ninja -C build
cd /tmp/
sudo dpdk/build/examples/dpdk-vhost_blk # it's a daemon
2. Add blk device to qemu, then bootup VM.
<qemu:commandline>
<qemu:arg value='-chardev'/>
<qemu:arg value='socket,id=char0,path=/tmp/vhost.socket'/>
<qemu:arg value='-device'/>
<qemu:arg value='vhost-user-blk-pci,chardev=char0,num-queues=1'/>
</qemu:commandline>
Without this commit, you can get device ready log from dpdk-vhost_blk:
VHOST_CONFIG: (/tmp/vhost.socket) negotiated Vhost-user protocol features: 0x11ebf
VHOST_CONFIG: (/tmp/vhost.socket) negotiated Virtio features: 0x100000000
VHOST_CONFIG: (/tmp/vhost.socket) virtio is now ready for processing.
New Device /tmp/vhost.socket, Device ID 0
Ctrlr Worker Thread start
With this commit, device won't be ready and VM will hang.
You can see VHOST_USER_F_PROTOCOL_FEATURES bit is added.
VHOST_CONFIG: (/tmp/vhost.socket) negotiated Virtio features: 0x140000000
Dpdk code related:
./lib/vhost/vhost_user.c
2044 /*
2045 * When VHOST_USER_F_PROTOCOL_FEATURES is not negotiated,
2046 * the ring starts already enabled. Otherwise, it is enabled via
2047 * the SET_VRING_ENABLE message.
2048 */
2049 if (!(dev->features & (1ULL << VHOST_USER_F_PROTOCOL_FEATURES))) {
2050 vq->enabled = true;
2051 }
Thanks,
Yajun
-----Original Message-----
From: Alex Bennée <alex.bennee@linaro.org>
Sent: Wednesday, October 26, 2022 3:42 PM
To: Yajun Wu <yajunw@nvidia.com>
Cc: qemu-devel@nongnu.org; mst@redhat.com; Parav Pandit <parav@nvidia.com>
Subject: Re: [PATCH v2 1/4] hw/virtio: incorporate backend features in features
External email: Use caution opening links or attachments
Yajun Wu <yajunw@nvidia.com> writes:
> Hi Alex,
>
> With this change, VHOST_USER_F_PROTOCOL_FEATURES bit will be set to
> backend for virtio block device (previously not).
>
> From https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.qemu.org%2Fdocs%2Fmaster%2Finterop%2Fvhost-user.html&data=05%7C01%7Cyajunw%40nvidia.com%7C2e8901540bf441248ec608dab725ca87%7C43083d15727340c1b7db39efd9ccc17a%7C0%7C0%7C638023670196631779%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=y5g9wwTfh%2BxrkESRypoo4pg3eKYInyDerDmI844PBSE%3D&reserved=0 spec:
> If VHOST_USER_F_PROTOCOL_FEATURES has not been negotiated, the ring starts directly in the enabled state.
> If VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, the ring is
> initialized in a disabled state and is enabled by
> VHOST_USER_SET_VRING_ENABLE with parameter 1.
>
> Vhost-user-blk won't send out VHOST_USER_SET_VRING_ENABLE today.
> Backend gets VHOST_USER_F_PROTOCOL_FEATURES negotiated and can't get VHOST_USER_SET_VRING_ENABLE.
> VQs keep in disabled state.
If the backend advertises protocol features but the stub doesn't support it how does it get enabled?
The testing I did was mostly by hand with the gpio backend and using the qtests. I Think we need to add some acceptance testing into avocado with some real daemons because I don't think we have enough coverage with the current qtest approach.
>
> Can you check on this scenario?
>
> Thanks
>
> -----Original Message-----
> From: Qemu-devel <qemu-devel-bounces+yajunw=nvidia.com@nongnu.org> On
> Behalf Of Alex Bennée
> Sent: Thursday, July 28, 2022 9:55 PM
> To: qemu-devel@nongnu.org
> Cc: mst@redhat.com; Alex Bennée <alex.bennee@linaro.org>
> Subject: [PATCH v2 1/4] hw/virtio: incorporate backend features in
> features
>
> External email: Use caution opening links or attachments
>
>
> There are some extra bits used over a vhost-user connection which are hidden from the device itself. We need to set them here to ensure we enable things like the protocol extensions.
>
> Currently net/vhost-user.c has it's own inscrutable way of persisting this data but it really should live in the core vhost_user code.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org>
> ---
> hw/virtio/vhost-user.c | 9 ++++++++-
> 1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index
> 75b8df21a4..1936a44e82 100644
> --- a/hw/virtio/vhost-user.c
> +++ b/hw/virtio/vhost-user.c
> @@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev,
> */
> bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL);
>
> - return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features,
> + /*
> + * We need to include any extra backend only feature bits that
> + * might be needed by our device. Currently this includes the
> + * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
> + * features.
> + */
> + return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
> + features | dev->backend_features,
> log_enabled); }
--
Alex Bennée
diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 75b8df21a4..1936a44e82 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -1460,7 +1460,14 @@ static int vhost_user_set_features(struct vhost_dev *dev, */ bool log_enabled = features & (0x1ULL << VHOST_F_LOG_ALL); - return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features, + /* + * We need to include any extra backend only feature bits that + * might be needed by our device. Currently this includes the + * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol + * features. + */ + return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, + features | dev->backend_features, log_enabled); }
There are some extra bits used over a vhost-user connection which are hidden from the device itself. We need to set them here to ensure we enable things like the protocol extensions. Currently net/vhost-user.c has it's own inscrutable way of persisting this data but it really should live in the core vhost_user code. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20220726192150.2435175-7-alex.bennee@linaro.org> --- hw/virtio/vhost-user.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)