[dpdk-dev] Question about vhost user interrupt mode

[dpdk-dev] Question about vhost user interrupt mode

[Yifeng Sun]

Hi all,

Right now on OVS, dpdkvhostuser can only run in polling mode (please
correct me if I am wrong). We are trying to enable interrupt mode of
dpdkvhostuser type port on OVS. We found that, with changes below, OVS
can poll&block on exposed kickfd and vhostuser is working under
interrupt mode without consuming 2 CPUs.

My question is, is this the correct direction to do so, or is there a
better way? Thanks!

--- a/lib/librte_vhost/vhost.h
+++ b/lib/librte_vhost/vhost.h
@@ -318,7 +318,6 @@ struct vring_packed_desc_event {
                                (1ULL << VIRTIO_NET_F_GUEST_UFO) | \
                                (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
                                (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | \
-                               (1ULL << VIRTIO_RING_F_EVENT_IDX) | \
                                (1ULL << VIRTIO_NET_F_MTU)  | \
                                (1ULL << VIRTIO_F_IN_ORDER) | \
                                (1ULL << VIRTIO_F_IOMMU_PLATFORM) | \

+int rte_vhost_get_kickfd(int vid, uint16_t queue_id)
+{
+       struct virtio_net *dev;
+       struct vhost_virtqueue *vq;
+
+       dev = get_device(vid);
+       if (!dev)
+               return -1;
+
+       if (unlikely(!(dev->flags & VIRTIO_DEV_BUILTIN_VIRTIO_NET))) {
+               //  vhost net backend is disabled.
+               return -1;
+       }
+
+       if (unlikely(!is_valid_virt_queue_idx(queue_id, 1, dev->nr_vring))) {
+               return -1;
+       }
+
+       vq = dev->virtqueue[queue_id];
+       // XXX lock?
+       return vq->kickfd;
+}

Best,
Yifeng Sun

Re: [dpdk-dev] Question about vhost user interrupt mode

[Yifeng Sun]

+ Tiwei and Maxime

Thanks,
Yifeng

On Fri, Feb 21, 2020 at 2:32 PM Yifeng Sun <pkusunyifeng@gmail.com> wrote:
>
> Hi all,
>
> Right now on OVS, dpdkvhostuser can only run in polling mode (please
> correct me if I am wrong). We are trying to enable interrupt mode of
> dpdkvhostuser type port on OVS. We found that, with changes below, OVS
> can poll&block on exposed kickfd and vhostuser is working under
> interrupt mode without consuming 2 CPUs.
>
> My question is, is this the correct direction to do so, or is there a
> better way? Thanks!
>
> --- a/lib/librte_vhost/vhost.h
> +++ b/lib/librte_vhost/vhost.h
> @@ -318,7 +318,6 @@ struct vring_packed_desc_event {
>                                 (1ULL << VIRTIO_NET_F_GUEST_UFO) | \
>                                 (1ULL << VIRTIO_NET_F_GUEST_ECN) | \
>                                 (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | \
> -                               (1ULL << VIRTIO_RING_F_EVENT_IDX) | \
>                                 (1ULL << VIRTIO_NET_F_MTU)  | \
>                                 (1ULL << VIRTIO_F_IN_ORDER) | \
>                                 (1ULL << VIRTIO_F_IOMMU_PLATFORM) | \
>
> +int rte_vhost_get_kickfd(int vid, uint16_t queue_id)
> +{
> +       struct virtio_net *dev;
> +       struct vhost_virtqueue *vq;
> +
> +       dev = get_device(vid);
> +       if (!dev)
> +               return -1;
> +
> +       if (unlikely(!(dev->flags & VIRTIO_DEV_BUILTIN_VIRTIO_NET))) {
> +               //  vhost net backend is disabled.
> +               return -1;
> +       }
> +
> +       if (unlikely(!is_valid_virt_queue_idx(queue_id, 1, dev->nr_vring))) {
> +               return -1;
> +       }
> +
> +       vq = dev->virtqueue[queue_id];
> +       // XXX lock?
> +       return vq->kickfd;
> +}
>
> Best,
> Yifeng Sun

Re: [dpdk-dev] Question about vhost user interrupt mode

[Tiwei Bie]

On Tue, Feb 25, 2020 at 12:44:48PM -0800, Yifeng Sun wrote:
> >
> > +int rte_vhost_get_kickfd(int vid, uint16_t queue_id)

Introducing rte_vhost_get_kickfd() may not help much, we already
have rte_vhost_get_vhost_vring() to do that.

https://github.com/DPDK/dpdk/blob/d7142fbae16f/drivers/net/vhost/rte_eth_vhost.c#L628-L644

> > +{
> > +       struct virtio_net *dev;
> > +       struct vhost_virtqueue *vq;
> > +
> > +       dev = get_device(vid);
> > +       if (!dev)
> > +               return -1;
> > +
> > +       if (unlikely(!(dev->flags & VIRTIO_DEV_BUILTIN_VIRTIO_NET))) {
> > +               //  vhost net backend is disabled.
> > +               return -1;
> > +       }
> > +
> > +       if (unlikely(!is_valid_virt_queue_idx(queue_id, 1, dev->nr_vring))) {
> > +               return -1;
> > +       }
> > +
> > +       vq = dev->virtqueue[queue_id];
> > +       // XXX lock?
> > +       return vq->kickfd;
> > +}
> >
> > Best,
> > Yifeng Sun

Re: [dpdk-dev] Question about vhost user interrupt mode

[Yifeng Sun]

Hi Tiwei,

Thanks a lot for your comment. I will take a look at rte_vhost_get_vhost_vring.

Best,
Yifeng

On Tue, Feb 25, 2020 at 5:51 PM Tiwei Bie <tiwei.bie@intel.com> wrote:
>
> On Tue, Feb 25, 2020 at 12:44:48PM -0800, Yifeng Sun wrote:
> > >
> > > +int rte_vhost_get_kickfd(int vid, uint16_t queue_id)
>
> Introducing rte_vhost_get_kickfd() may not help much, we already
> have rte_vhost_get_vhost_vring() to do that.
>
> https://github.com/DPDK/dpdk/blob/d7142fbae16f/drivers/net/vhost/rte_eth_vhost.c#L628-L644
>
> > > +{
> > > +       struct virtio_net *dev;
> > > +       struct vhost_virtqueue *vq;
> > > +
> > > +       dev = get_device(vid);
> > > +       if (!dev)
> > > +               return -1;
> > > +
> > > +       if (unlikely(!(dev->flags & VIRTIO_DEV_BUILTIN_VIRTIO_NET))) {
> > > +               //  vhost net backend is disabled.
> > > +               return -1;
> > > +       }
> > > +
> > > +       if (unlikely(!is_valid_virt_queue_idx(queue_id, 1, dev->nr_vring))) {
> > > +               return -1;
> > > +       }
> > > +
> > > +       vq = dev->virtqueue[queue_id];
> > > +       // XXX lock?
> > > +       return vq->kickfd;
> > > +}
> > >
> > > Best,
> > > Yifeng Sun

Re: [dpdk-dev] Question about vhost user interrupt mode

[William Tu]

On Tue, Feb 25, 2020 at 5:51 PM Tiwei Bie <tiwei.bie@intel.com> wrote:
>
> On Tue, Feb 25, 2020 at 12:44:48PM -0800, Yifeng Sun wrote:
> > >
> > > +int rte_vhost_get_kickfd(int vid, uint16_t queue_id)
>
> Introducing rte_vhost_get_kickfd() may not help much, we already
> have rte_vhost_get_vhost_vring() to do that.
>
> https://github.com/DPDK/dpdk/blob/d7142fbae16f/drivers/net/vhost/rte_eth_vhost.c#L628-L644
>

Hi Tiwei,

Thanks for your feedback.
I'd like to learn about the event index feature of vhost.
Is there any document we can read to understand it?

Thanks
William