[dpdk-dev] DPDK's vhost-user logging capability

[dpdk-dev] DPDK's vhost-user logging capability

[shesha Sreenivasamurthy (shesha)]

Hi All,

I was going over vhost-user migration capability in DPDK in lieu of a Cisco's multi-q DPDK vhost-user application. I see that log_base address is implemented as per virtio_net device. However, desc, addr and used is per vhost_virtqueue. Additionally, QEMU sends one VHOST_USER_SET_LOG_BASE per queue-pair (QEMU - hw/virtio/vhost.c::vhost_dev_set_log).

Does it mean we need to log dirty pages of all rings to same location ? If that is the case then why does QEMU sends separate VHOST_USER_SET_LOG_BASE per queue pair ?

--
- Thanks
char * (*shesha) (uint64_t cache, uint8_t F00D)
{ return 0x0000C0DE; }

Re: [dpdk-dev] DPDK's vhost-user logging capability

[Yuanhan Liu]

On Wed, Mar 23, 2016 at 03:34:09PM +0000, shesha Sreenivasamurthy (shesha) wrote:
> Hi All,
> 
> I was going over vhost-user migration capability in DPDK in lieu of a Cisco's
> multi-q DPDK vhost-user application. I see that log_base address is implemented
> as per virtio_net device. However, desc, addr and used is per vhost_virtqueue.
> Additionally, QEMU sends one VHOST_USER_SET_LOG_BASE per queue-pair (QEMU - hw/
> virtio/vhost.c::vhost_dev_set_log).
> 
> Does it mean we need to log dirty pages of all rings to same location ?

Hi,

Yes, and QEMU allocates only one block of memory (see vhost_log_alloc())
after all.

> If that
> is the case then why does QEMU sends separate VHOST_USER_SET_LOG_BASE per queue
> pair ?

That's kind of like a design. One queue pair is associated with one
vhost_dev struct in QEMU, hence, all those requests will go through
vhost_dev structs (aka, all qeueu pairs), including those that one
time request is needed only, such as VHOST_USER_SET_MEM_TABLE. Thus,
we introduced vhost_user_one_time_request() to avoid such case.

So, good question, and we may need add it to the "one time request"
group, Marc?

And FYI, for queue-pair (or vring) request, there should be an index
in the payload, to point to the right vring. If not, it normally
means a global request, that _may_ need be sent once only.

	--yliu

Re: [dpdk-dev] DPDK's vhost-user logging capability

[Marc-André Lureau]

Hi

----- Original Message -----
> On Wed, Mar 23, 2016 at 03:34:09PM +0000, shesha Sreenivasamurthy (shesha)
> wrote:
> > Hi All,
> > 
> > I was going over vhost-user migration capability in DPDK in lieu of a
> > Cisco's
> > multi-q DPDK vhost-user application. I see that log_base address is
> > implemented
> > as per virtio_net device. However, desc, addr and used is per
> > vhost_virtqueue.
> > Additionally, QEMU sends one VHOST_USER_SET_LOG_BASE per queue-pair (QEMU -
> > hw/
> > virtio/vhost.c::vhost_dev_set_log).
> > 
> > Does it mean we need to log dirty pages of all rings to same location ?
> 
> Hi,
> 
> Yes, and QEMU allocates only one block of memory (see vhost_log_alloc())
> after all.

> > If that
> > is the case then why does QEMU sends separate VHOST_USER_SET_LOG_BASE per
> > queue
> > pair ?
> 
> That's kind of like a design. One queue pair is associated with one
> vhost_dev struct in QEMU, hence, all those requests will go through
> vhost_dev structs (aka, all qeueu pairs), including those that one
> time request is needed only, such as VHOST_USER_SET_MEM_TABLE. Thus,
> we introduced vhost_user_one_time_request() to avoid such case.

I am not familiar with multi-queue, but I can see that in vhost_net_start.

> So, good question, and we may need add it to the "one time request"
> group, Marc?

That would probably prevent vhost_dev_log_resize(), so it would need some kind of "force" flag. I am not sure it's worth it at this point.

> 
> And FYI, for queue-pair (or vring) request, there should be an index
> in the payload, to point to the right vring. If not, it normally
> means a global request, that _may_ need be sent once only.
> 
> 	--yliu
>