[dpdk-dev] data copy in vhost-user

[dpdk-dev] data copy in vhost-user

[Nikita Kalyazin]

Hi,


As far as I understand, DPDK vhost-user implementation requires data copy for either RX or TX (rte_vhost_dequeue_burst() and rte_vhost_enqueue_burst()). It means that two data copies are needed to transfer a packet from one VM to another.

Why is not it possible to eliminate one of the copies (e.g., rte_vhost_enqueue_burst() might set up a reference at vring descriptor to mbuf's data rather than copying the data)?

-- 

Best regards,

Nikita Kalyazin,
n.kalyazin@samsung.com

CE OS Group
Samsung R&D Institute Russia
Tel: +7 (495) 797-25-00 #3816
Tel: +7 (495) 797-25-03
Office #1501, 12-1, Dvintsev str.,
Moscow, 127018, Russia

Re: [dpdk-dev] data copy in vhost-user

[Zoltan Kiss]

On 27/04/15 12:54, Nikita Kalyazin wrote:
> Hi,
>
>
> As far as I understand, DPDK vhost-user implementation requires data copy for either RX or TX (rte_vhost_dequeue_burst() and rte_vhost_enqueue_burst()). It means that two data copies are needed to transfer a packet from one VM to another.
>
> Why is not it possible to eliminate one of the copies (e.g., rte_vhost_enqueue_burst() might set up a reference at vring descriptor to mbuf's data rather than copying the data)?

I'm just guessing, but in case of VM-to-VM traffic the receiving one 
could hold onto the buffer indefinitely, preventing the sender to reuse 
the buffer. That could lead to a DoS in some cases, and shutting down 
the sender would be also tricky. At least in case of Xen 
netback/netfront that's the reason. A reasonable solution for this 
problem is to make sure the buffer is swapped out with a copy after a 
finite time.

Regards,

Zoltan

Re: [dpdk-dev] data copy in vhost-user

[Xie, Huawei]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Zoltan Kiss
> Sent: Tuesday, April 28, 2015 12:27 AM
> To: Nikita Kalyazin; dev@dpdk.org
> Subject: Re: [dpdk-dev] data copy in vhost-user
> 
> 
> 
> On 27/04/15 12:54, Nikita Kalyazin wrote:
> > Hi,
> >
> >
> > As far as I understand, DPDK vhost-user implementation requires data copy
> for either RX or TX (rte_vhost_dequeue_burst() and
> rte_vhost_enqueue_burst()). It means that two data copies are needed to
> transfer a packet from one VM to another.
> >
> > Why is not it possible to eliminate one of the copies (e.g.,
> rte_vhost_enqueue_burst() might set up a reference at vring descriptor to
> mbuf's data rather than copying the data)?
This had been added to the to-do list. We could delay the copy until the real copy is needed.
> 
> I'm just guessing, but in case of VM-to-VM traffic the receiving one
> could hold onto the buffer indefinitely, preventing the sender to reuse
> the buffer. That could lead to a DoS in some cases, and shutting down
> the sender would be also tricky. At least in case of Xen
> netback/netfront that's the reason. A reasonable solution for this
> problem is to make sure the buffer is swapped out with a copy after a
> finite time.
Do you mean we associate a timeout for the buffer?
> 
> Regards,
> 
> Zoltan

Re: [dpdk-dev] data copy in vhost-user

[Zoltan Kiss]

On 28/04/15 02:22, Xie, Huawei wrote:
>
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Zoltan Kiss
>> Sent: Tuesday, April 28, 2015 12:27 AM
>> To: Nikita Kalyazin; dev@dpdk.org
>> Subject: Re: [dpdk-dev] data copy in vhost-user
>>
>>
>>
>> On 27/04/15 12:54, Nikita Kalyazin wrote:
>>> Hi,
>>>
>>>
>>> As far as I understand, DPDK vhost-user implementation requires data copy
>> for either RX or TX (rte_vhost_dequeue_burst() and
>> rte_vhost_enqueue_burst()). It means that two data copies are needed to
>> transfer a packet from one VM to another.
>>>
>>> Why is not it possible to eliminate one of the copies (e.g.,
>> rte_vhost_enqueue_burst() might set up a reference at vring descriptor to
>> mbuf's data rather than copying the data)?
> This had been added to the to-do list. We could delay the copy until the real copy is needed.
>>
>> I'm just guessing, but in case of VM-to-VM traffic the receiving one
>> could hold onto the buffer indefinitely, preventing the sender to reuse
>> the buffer. That could lead to a DoS in some cases, and shutting down
>> the sender would be also tricky. At least in case of Xen
>> netback/netfront that's the reason. A reasonable solution for this
>> problem is to make sure the buffer is swapped out with a copy after a
>> finite time.
> Do you mean we associate a timeout for the buffer?
Yes, I think xen-netback had such a version once, but it was removed. As 
far as I know the overhead and complexity of handling these timeouts 
were too severe.
I might be wrong about this, I don't know if this problem applies here 
as well or not.

>>
>> Regards,
>>
>> Zoltan

Re: [dpdk-dev] data copy in vhost-user

[Nikita Kalyazin]

Zoltan, Huawei,

Thank you for your replies.

-- 

Best regards,

Nikita Kalyazin,
n.kalyazin@samsung.com

CE OS Group
Samsung R&D Institute Russia
Tel: +7 (495) 797-25-00 #3816
Tel: +7 (495) 797-25-03
Office #1501, 12-1, Dvintsev str.,
Moscow, 127018, Russia

On Tue, Apr 28, 2015 at 01:24:25PM +0100, Zoltan Kiss wrote:
> 
> 
> On 28/04/15 02:22, Xie, Huawei wrote:
> >
> >
> >> -----Original Message-----
> >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Zoltan Kiss
> >> Sent: Tuesday, April 28, 2015 12:27 AM
> >> To: Nikita Kalyazin; dev@dpdk.org
> >> Subject: Re: [dpdk-dev] data copy in vhost-user
> >>
> >>
> >>
> >> On 27/04/15 12:54, Nikita Kalyazin wrote:
> >>> Hi,
> >>>
> >>>
> >>> As far as I understand, DPDK vhost-user implementation requires data copy
> >> for either RX or TX (rte_vhost_dequeue_burst() and
> >> rte_vhost_enqueue_burst()). It means that two data copies are needed to
> >> transfer a packet from one VM to another.
> >>>
> >>> Why is not it possible to eliminate one of the copies (e.g.,
> >> rte_vhost_enqueue_burst() might set up a reference at vring descriptor to
> >> mbuf's data rather than copying the data)?
> > This had been added to the to-do list. We could delay the copy until the real copy is needed.
> >>
> >> I'm just guessing, but in case of VM-to-VM traffic the receiving one
> >> could hold onto the buffer indefinitely, preventing the sender to reuse
> >> the buffer. That could lead to a DoS in some cases, and shutting down
> >> the sender would be also tricky. At least in case of Xen
> >> netback/netfront that's the reason. A reasonable solution for this
> >> problem is to make sure the buffer is swapped out with a copy after a
> >> finite time.
> > Do you mean we associate a timeout for the buffer?
> Yes, I think xen-netback had such a version once, but it was removed. As 
> far as I know the overhead and complexity of handling these timeouts 
> were too severe.
> I might be wrong about this, I don't know if this problem applies here 
> as well or not.
> 
> >>
> >> Regards,
> >>
> >> Zoltan