| Message ID | 1485132904-17632-1-git-send-email-zhaoshenglong@huawei.com |
|---|---|
| State | New |
| Headers | show |
On Mon, Jan 23, 2017 at 08:55:04AM +0800, Shannon Zhao wrote: > From: Shannon Zhao <shannon.zhao@linaro.org> > > For ARM virt machine, if we use virt-2.7 which will not create ITS node, > the virtio-net can not recieve interrupts so it can't get ip address > through dhcp. > This fixes commit 83d768b(virtio: set ISR on dataplane notifications). > > Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org> Paolo, could you comment on this pls? > --- > V2: Factor out a common function instead of duplicating code > --- > hw/virtio/virtio.c | 11 ++++++++--- > 1 file changed, 8 insertions(+), 3 deletions(-) > > diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c > index aa4f38f..00e8f74 100644 > --- a/hw/virtio/virtio.c > +++ b/hw/virtio/virtio.c > @@ -1441,6 +1441,12 @@ void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq) > event_notifier_set(&vq->guest_notifier); > } > > +static void virtio_irq(VirtQueue *vq) > +{ > + virtio_set_isr(vq->vdev, 0x1); > + virtio_notify_vector(vq->vdev, vq->vector); > +} > + > void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) > { > if (!virtio_should_notify(vdev, vq)) { > @@ -1448,8 +1454,7 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) > } > > trace_virtio_notify(vdev, vq); > - virtio_set_isr(vq->vdev, 0x1); > - virtio_notify_vector(vdev, vq->vector); > + virtio_irq(vq); > } > > void virtio_notify_config(VirtIODevice *vdev) > @@ -2082,7 +2087,7 @@ static void virtio_queue_guest_notifier_read(EventNotifier *n) > { > VirtQueue *vq = container_of(n, VirtQueue, guest_notifier); > if (event_notifier_test_and_clear(n)) { > - virtio_notify_vector(vq->vdev, vq->vector); > + virtio_irq(vq); > } > } > > -- > 2.0.4 >
On 23/01/2017 17:07, Michael S. Tsirkin wrote: > On Mon, Jan 23, 2017 at 08:55:04AM +0800, Shannon Zhao wrote: >> From: Shannon Zhao <shannon.zhao@linaro.org> >> >> For ARM virt machine, if we use virt-2.7 which will not create ITS node, >> the virtio-net can not recieve interrupts so it can't get ip address >> through dhcp. >> This fixes commit 83d768b(virtio: set ISR on dataplane notifications). >> >> Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org> > > Paolo, could you comment on this pls? For virtio-blk, my patch moved the setting of ISR from virtio_queue_guest_notifier_read to virtio_notify_irqfd. This is because the irqfd emulation only needs to trigger the interrupt. Setting the ISR should have been done elsewhere. Can vhost set ISR (I thought it couldn't)? If so, it seems like ARM was relying on QEMU's irqfd emulation, as a work around for vhost not setting ISR. But this only works if irqfd is directed to QEMU and not to KVM. So if vhost cannot set ISR, I think vhost should be disabled unless MSI is active. Am I missing something? Paolo > >> --- >> V2: Factor out a common function instead of duplicating code >> --- >> hw/virtio/virtio.c | 11 ++++++++--- >> 1 file changed, 8 insertions(+), 3 deletions(-) >> >> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c >> index aa4f38f..00e8f74 100644 >> --- a/hw/virtio/virtio.c >> +++ b/hw/virtio/virtio.c >> @@ -1441,6 +1441,12 @@ void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq) >> event_notifier_set(&vq->guest_notifier); >> } >> >> +static void virtio_irq(VirtQueue *vq) >> +{ >> + virtio_set_isr(vq->vdev, 0x1); >> + virtio_notify_vector(vq->vdev, vq->vector); >> +} >> + >> void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) >> { >> if (!virtio_should_notify(vdev, vq)) { >> @@ -1448,8 +1454,7 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) >> } >> >> trace_virtio_notify(vdev, vq); >> - virtio_set_isr(vq->vdev, 0x1); >> - virtio_notify_vector(vdev, vq->vector); >> + virtio_irq(vq); >> } >> >> void virtio_notify_config(VirtIODevice *vdev) >> @@ -2082,7 +2087,7 @@ static void virtio_queue_guest_notifier_read(EventNotifier *n) >> { >> VirtQueue *vq = container_of(n, VirtQueue, guest_notifier); >> if (event_notifier_test_and_clear(n)) { >> - virtio_notify_vector(vq->vdev, vq->vector); >> + virtio_irq(vq); >> } >> } >> >> -- >> 2.0.4 >>
On Mon, Jan 23, 2017 at 05:46:47PM +0100, Paolo Bonzini wrote: > > > On 23/01/2017 17:07, Michael S. Tsirkin wrote: > > On Mon, Jan 23, 2017 at 08:55:04AM +0800, Shannon Zhao wrote: > >> From: Shannon Zhao <shannon.zhao@linaro.org> > >> > >> For ARM virt machine, if we use virt-2.7 which will not create ITS node, > >> the virtio-net can not recieve interrupts so it can't get ip address > >> through dhcp. > >> This fixes commit 83d768b(virtio: set ISR on dataplane notifications). > >> > >> Signed-off-by: Shannon Zhao <shannon.zhao@linaro.org> > > > > Paolo, could you comment on this pls? > > For virtio-blk, my patch moved the setting of ISR from > virtio_queue_guest_notifier_read to virtio_notify_irqfd. This is > because the irqfd emulation only needs to trigger the interrupt. > Setting the ISR should have been done elsewhere. > > Can vhost set ISR (I thought it couldn't)? If so, it seems like ARM was > relying on QEMU's irqfd emulation, as a work around for vhost not > setting ISR. But this only works if irqfd is directed to QEMU and not > to KVM. So if vhost cannot set ISR, I think vhost should be disabled > unless MSI is active. > > Am I missing something? > > Paolo vhost doesn't set ISR ATM. Without MSI we are always bouncing interrupts through QEMU in particular in order to set ISR. Disabling vhost when user requested it was what we did at some point but this just confused people. > > > >> --- > >> V2: Factor out a common function instead of duplicating code > >> --- > >> hw/virtio/virtio.c | 11 ++++++++--- > >> 1 file changed, 8 insertions(+), 3 deletions(-) > >> > >> diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c > >> index aa4f38f..00e8f74 100644 > >> --- a/hw/virtio/virtio.c > >> +++ b/hw/virtio/virtio.c > >> @@ -1441,6 +1441,12 @@ void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq) > >> event_notifier_set(&vq->guest_notifier); > >> } > >> > >> +static void virtio_irq(VirtQueue *vq) > >> +{ > >> + virtio_set_isr(vq->vdev, 0x1); > >> + virtio_notify_vector(vq->vdev, vq->vector); > >> +} > >> + > >> void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) > >> { > >> if (!virtio_should_notify(vdev, vq)) { > >> @@ -1448,8 +1454,7 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) > >> } > >> > >> trace_virtio_notify(vdev, vq); > >> - virtio_set_isr(vq->vdev, 0x1); > >> - virtio_notify_vector(vdev, vq->vector); > >> + virtio_irq(vq); > >> } > >> > >> void virtio_notify_config(VirtIODevice *vdev) > >> @@ -2082,7 +2087,7 @@ static void virtio_queue_guest_notifier_read(EventNotifier *n) > >> { > >> VirtQueue *vq = container_of(n, VirtQueue, guest_notifier); > >> if (event_notifier_test_and_clear(n)) { > >> - virtio_notify_vector(vq->vdev, vq->vector); > >> + virtio_irq(vq); > >> } > >> } > >> > >> -- > >> 2.0.4 > >>
On 23/01/2017 18:04, Michael S. Tsirkin wrote: >> For virtio-blk, my patch moved the setting of ISR from >> virtio_queue_guest_notifier_read to virtio_notify_irqfd. This is >> because the irqfd emulation only needs to trigger the interrupt. >> Setting the ISR should have been done elsewhere. >> >> Can vhost set ISR (I thought it couldn't)? If so, it seems like ARM was >> relying on QEMU's irqfd emulation, as a work around for vhost not >> setting ISR. But this only works if irqfd is directed to QEMU and not >> to KVM. So if vhost cannot set ISR, I think vhost should be disabled >> unless MSI is active. >> >> Am I missing something? > > vhost doesn't set ISR ATM. Without MSI we are always bouncing > interrupts through QEMU in particular in order to set ISR. Where is the code that disables KVM irqfd? Paolo > Disabling vhost when user requested it was what we did > at some point but this just confused people. > >
On Tue, Jan 24, 2017 at 10:25:00AM +0100, Paolo Bonzini wrote: > > > On 23/01/2017 18:04, Michael S. Tsirkin wrote: > >> For virtio-blk, my patch moved the setting of ISR from > >> virtio_queue_guest_notifier_read to virtio_notify_irqfd. This is > >> because the irqfd emulation only needs to trigger the interrupt. > >> Setting the ISR should have been done elsewhere. > >> > >> Can vhost set ISR (I thought it couldn't)? If so, it seems like ARM was > >> relying on QEMU's irqfd emulation, as a work around for vhost not > >> setting ISR. But this only works if irqfd is directed to QEMU and not > >> to KVM. So if vhost cannot set ISR, I think vhost should be disabled > >> unless MSI is active. > >> > >> Am I missing something? > > > > vhost doesn't set ISR ATM. Without MSI we are always bouncing > > interrupts through QEMU in particular in order to set ISR. > > Where is the code that disables KVM irqfd? > > Paolo This bit in virtio_pci_set_guest_notifiers does the trick I think: bool with_irqfd = msix_enabled(&proxy->pci_dev) && kvm_msi_via_irqfd_enabled(); I don't think we ever used irqfd for level interrupts. > > Disabling vhost when user requested it was what we did > > at some point but this just confused people. > > > >
On 24/01/2017 15:44, Michael S. Tsirkin wrote: > On Tue, Jan 24, 2017 at 10:25:00AM +0100, Paolo Bonzini wrote: >> >> >> On 23/01/2017 18:04, Michael S. Tsirkin wrote: >>>> For virtio-blk, my patch moved the setting of ISR from >>>> virtio_queue_guest_notifier_read to virtio_notify_irqfd. This is >>>> because the irqfd emulation only needs to trigger the interrupt. >>>> Setting the ISR should have been done elsewhere. >>>> >>>> Can vhost set ISR (I thought it couldn't)? If so, it seems like ARM was >>>> relying on QEMU's irqfd emulation, as a work around for vhost not >>>> setting ISR. But this only works if irqfd is directed to QEMU and not >>>> to KVM. So if vhost cannot set ISR, I think vhost should be disabled >>>> unless MSI is active. >>>> >>>> Am I missing something? >>> >>> vhost doesn't set ISR ATM. Without MSI we are always bouncing >>> interrupts through QEMU in particular in order to set ISR. >> >> Where is the code that disables KVM irqfd? > > This bit in virtio_pci_set_guest_notifiers does the trick I think: > > > bool with_irqfd = msix_enabled(&proxy->pci_dev) && > kvm_msi_via_irqfd_enabled(); > > I don't think we ever used irqfd for level interrupts. Ok, thanks! So we have four cases: - assign with emulated irqfd + ISR - assign with emulated irqfd - assign with irqfd - deassign Currently we don't distinguish the first two, so (which I don't like) Shannon's patch ends up setting ISR twice. The only (ugly) solution that comes to mind is making with_irqfd an enum... Paolo
On Tue, Jan 24, 2017 at 04:00:55PM +0100, Paolo Bonzini wrote: > > > On 24/01/2017 15:44, Michael S. Tsirkin wrote: > > On Tue, Jan 24, 2017 at 10:25:00AM +0100, Paolo Bonzini wrote: > >> > >> > >> On 23/01/2017 18:04, Michael S. Tsirkin wrote: > >>>> For virtio-blk, my patch moved the setting of ISR from > >>>> virtio_queue_guest_notifier_read to virtio_notify_irqfd. This is > >>>> because the irqfd emulation only needs to trigger the interrupt. > >>>> Setting the ISR should have been done elsewhere. > >>>> > >>>> Can vhost set ISR (I thought it couldn't)? If so, it seems like ARM was > >>>> relying on QEMU's irqfd emulation, as a work around for vhost not > >>>> setting ISR. But this only works if irqfd is directed to QEMU and not > >>>> to KVM. So if vhost cannot set ISR, I think vhost should be disabled > >>>> unless MSI is active. > >>>> > >>>> Am I missing something? > >>> > >>> vhost doesn't set ISR ATM. Without MSI we are always bouncing > >>> interrupts through QEMU in particular in order to set ISR. > >> > >> Where is the code that disables KVM irqfd? > > > > This bit in virtio_pci_set_guest_notifiers does the trick I think: > > > > > > bool with_irqfd = msix_enabled(&proxy->pci_dev) && > > kvm_msi_via_irqfd_enabled(); > > > > I don't think we ever used irqfd for level interrupts. > > Ok, thanks! > > So we have four cases: > > - assign with emulated irqfd + ISR > - assign with emulated irqfd > - assign with irqfd > - deassign > > Currently we don't distinguish the first two, so (which I don't like) > Shannon's patch ends up setting ISR twice. I don't see this yet - where is it set twice? > The only (ugly) solution > that comes to mind is making with_irqfd an enum... > > Paolo
On 24/01/2017 16:05, Michael S. Tsirkin wrote: >> >> So we have four cases: >> >> - assign with emulated irqfd + ISR >> - assign with emulated irqfd >> - assign with irqfd >> - deassign >> >> Currently we don't distinguish the first two, so (which I don't like) >> Shannon's patch ends up setting ISR twice. > > I don't see this yet - where is it set twice? Once in virtio_notify_irqfd, once in virtio_queue_guest_notifier_read. Unfortunately, for virtio-blk + MSI + KVM + old Windows drivers we need the one in virtio_notify_irqfd. For virtio-net + vhost + INTx we need the one in virtio_queue_guest_notifier_read.
On Tue, Jan 24, 2017 at 04:50:44PM +0100, Paolo Bonzini wrote: > > > On 24/01/2017 16:05, Michael S. Tsirkin wrote: > >> > >> So we have four cases: > >> > >> - assign with emulated irqfd + ISR > >> - assign with emulated irqfd > >> - assign with irqfd > >> - deassign > >> > >> Currently we don't distinguish the first two, so (which I don't like) > >> Shannon's patch ends up setting ISR twice. > > > > I don't see this yet - where is it set twice? > > Once in virtio_notify_irqfd, once in virtio_queue_guest_notifier_read. > > Unfortunately, for virtio-blk + MSI + KVM + old Windows drivers we need > the one in virtio_notify_irqfd. For virtio-net + vhost + INTx we need > the one in virtio_queue_guest_notifier_read. Well in MSI mode virtio_queue_guest_notifier_read so we only do this once, right? I need to check INTx mode - does virtio_notify_irqfd get called then? -- MST
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index aa4f38f..00e8f74 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -1441,6 +1441,12 @@ void virtio_notify_irqfd(VirtIODevice *vdev, VirtQueue *vq) event_notifier_set(&vq->guest_notifier); } +static void virtio_irq(VirtQueue *vq) +{ + virtio_set_isr(vq->vdev, 0x1); + virtio_notify_vector(vq->vdev, vq->vector); +} + void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) { if (!virtio_should_notify(vdev, vq)) { @@ -1448,8 +1454,7 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq) } trace_virtio_notify(vdev, vq); - virtio_set_isr(vq->vdev, 0x1); - virtio_notify_vector(vdev, vq->vector); + virtio_irq(vq); } void virtio_notify_config(VirtIODevice *vdev) @@ -2082,7 +2087,7 @@ static void virtio_queue_guest_notifier_read(EventNotifier *n) { VirtQueue *vq = container_of(n, VirtQueue, guest_notifier); if (event_notifier_test_and_clear(n)) { - virtio_notify_vector(vq->vdev, vq->vector); + virtio_irq(vq); } }