[dpdk-dev] Testing memnic for VM to VM transfer

[dpdk-dev] Testing memnic for VM to VM transfer

[John Joyce (joycej)]

Hi everyone:
        We are interested in testing the performance of the memnic driver posted at     http://dpdk.org/browse/memnic/refs/.   We want to compare its performance compared to other techniques to transfer packets between the guest and the kernel,  predominately for VM to VM transfers.

We have downloaded the memnic components and have got it running in a guest VM.

The question we hope this group might be able to help with is what would be the best way to processes the packets in the kernel to get a VM to VM transfer.

A couple options might be possible


1.       Common shared buffer between two VMs.  With some utility/code to switch TX & RX rings between the two VMs.

VM1 application --- memnic  ---  common shared memory buffer on the host --- memnic  ---  VM2 application

2.       Special purpose Kernel switching module

VM1 application --- memnic  ---  shared memory VM1  --- Kernel switching module  --- shared memory VM2  --- memnic  ---  VM2 application

3.       Existing Kernel switching module

VM1 application --- memnic  ---  shared memory VM1  --- existing Kernel switching module (e.g. OVS/linux Bridge/VETh pair)  --- shared memory VM2  --- memnic  ---  VM2 application

Can anyone recommend which approach might be best or easiest?   We would like to avoid writing much (or any) kernel code so if there are already any open source code or test utilities that provide one of these options or would be a good starting point to start from,  a pointer would be much appreciated.

Thanks in advance

                                                                                                                                                                John Joyce

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[Hiroshi Shimamoto]

Hi,

> Subject: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> Hi everyone:
>         We are interested in testing the performance of the memnic driver posted at     http://dpdk.org/browse/memnic/refs/.
> We want to compare its performance compared to other techniques to transfer packets between the guest and the kernel,
> predominately for VM to VM transfers.
> 
> We have downloaded the memnic components and have got it running in a guest VM.
> 
> The question we hope this group might be able to help with is what would be the best way to processes the packets in the
> kernel to get a VM to VM transfer.

I think there is no kernel code work with MEMNIC.
The recommend switching software on the host is Intel DPDK vSwitch hosted on 01.org and github.
https://github.com/01org/dpdk-ovs/tree/development

Intel DPDK vSwitch runs on userspace not kernel.

I introduced this mechanism to DPDK vSwitch and the guest drivers are maintained in dpdk.org.

thanks,
Hiroshi

> 
> A couple options might be possible
> 
> 
> 1.       Common shared buffer between two VMs.  With some utility/code to switch TX & RX rings between the two VMs.
> 
> VM1 application --- memnic  ---  common shared memory buffer on the host --- memnic  ---  VM2 application
> 
> 2.       Special purpose Kernel switching module
> 
> VM1 application --- memnic  ---  shared memory VM1  --- Kernel switching module  --- shared memory VM2  --- memnic  ---
> VM2 application
> 
> 3.       Existing Kernel switching module
> 
> VM1 application --- memnic  ---  shared memory VM1  --- existing Kernel switching module (e.g. OVS/linux Bridge/VETh pair)
> --- shared memory VM2  --- memnic  ---  VM2 application
> 
> Can anyone recommend which approach might be best or easiest?   We would like to avoid writing much (or any) kernel code
> so if there are already any open source code or test utilities that provide one of these options or would be a good starting
> point to start from,  a pointer would be much appreciated.
> 
> Thanks in advance
> 
> 
> John Joyce

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[GongJinrong]

Hi, Hiroshi

   I just start to learn DPDK and memnic, in memnic guide, you said "On
host, the shared memory must be initialized by an application using memnic",
I am not so clear that how to initialize the share memory in host, do you
means use posix API or DPDK API to create the share memory?(it seems memnic
guest side use rte_mbuf to transfer data), do you have any sample code to
demo how to use memnic in host?

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hiroshi Shimamoto
Sent: Wednesday, June 18, 2014 12:02 PM
To: John Joyce (joycej); dev@dpdk.org
Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer

Hi,

> Subject: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> Hi everyone:
>         We are interested in testing the performance of the memnic driver
posted at     http://dpdk.org/browse/memnic/refs/.
> We want to compare its performance compared to other techniques to 
> transfer packets between the guest and the kernel, predominately for VM to
VM transfers.
> 
> We have downloaded the memnic components and have got it running in a
guest VM.
> 
> The question we hope this group might be able to help with is what 
> would be the best way to processes the packets in the kernel to get a VM
to VM transfer.

I think there is no kernel code work with MEMNIC.
The recommend switching software on the host is Intel DPDK vSwitch hosted on
01.org and github.
https://github.com/01org/dpdk-ovs/tree/development

Intel DPDK vSwitch runs on userspace not kernel.

I introduced this mechanism to DPDK vSwitch and the guest drivers are
maintained in dpdk.org.

thanks,
Hiroshi

> 
> A couple options might be possible
> 
> 
> 1.       Common shared buffer between two VMs.  With some utility/code to
switch TX & RX rings between the two VMs.
> 
> VM1 application --- memnic  ---  common shared memory buffer on the 
> host --- memnic  ---  VM2 application
> 
> 2.       Special purpose Kernel switching module
> 
> VM1 application --- memnic  ---  shared memory VM1  --- Kernel 
> switching module  --- shared memory VM2  --- memnic  ---
> VM2 application
> 
> 3.       Existing Kernel switching module
> 
> VM1 application --- memnic  ---  shared memory VM1  --- existing 
> Kernel switching module (e.g. OVS/linux Bridge/VETh pair)
> --- shared memory VM2  --- memnic  ---  VM2 application
> 
> Can anyone recommend which approach might be best or easiest?   We would
like to avoid writing much (or any) kernel code
> so if there are already any open source code or test utilities that 
> provide one of these options or would be a good starting point to start
from,  a pointer would be much appreciated.
> 
> Thanks in advance
> 
> 
> John Joyce

Re: [dpdk-dev] ##freemail## RE: Testing memnic for VM to VM transfer

[Hiroshi Shimamoto]

Hi,

> Subject: ##freemail## RE: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> Hi, Hiroshi
> 
>    I just start to learn DPDK and memnic, in memnic guide, you said "On
> host, the shared memory must be initialized by an application using memnic",
> I am not so clear that how to initialize the share memory in host, do you
> means use posix API or DPDK API to create the share memory?(it seems memnic
> guest side use rte_mbuf to transfer data), do you have any sample code to
> demo how to use memnic in host?

I don't have simple MEMNIC sample to use it on host.
Could you please try DPDK vSwitch and enables MEMNIC vport?
DPDK vSwitch must handle packets between physical NIC port and MEMNIC vport
exposed to guest with dpdk.org memnic driver.

thanks,
Hiroshi

> 
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hiroshi Shimamoto
> Sent: Wednesday, June 18, 2014 12:02 PM
> To: John Joyce (joycej); dev@dpdk.org
> Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> Hi,
> 
> > Subject: [dpdk-dev] Testing memnic for VM to VM transfer
> >
> > Hi everyone:
> >         We are interested in testing the performance of the memnic driver
> posted at     http://dpdk.org/browse/memnic/refs/.
> > We want to compare its performance compared to other techniques to
> > transfer packets between the guest and the kernel, predominately for VM to
> VM transfers.
> >
> > We have downloaded the memnic components and have got it running in a
> guest VM.
> >
> > The question we hope this group might be able to help with is what
> > would be the best way to processes the packets in the kernel to get a VM
> to VM transfer.
> 
> I think there is no kernel code work with MEMNIC.
> The recommend switching software on the host is Intel DPDK vSwitch hosted on
> 01.org and github.
> https://github.com/01org/dpdk-ovs/tree/development
> 
> Intel DPDK vSwitch runs on userspace not kernel.
> 
> I introduced this mechanism to DPDK vSwitch and the guest drivers are
> maintained in dpdk.org.
> 
> thanks,
> Hiroshi
> 
> >
> > A couple options might be possible
> >
> >
> > 1.       Common shared buffer between two VMs.  With some utility/code to
> switch TX & RX rings between the two VMs.
> >
> > VM1 application --- memnic  ---  common shared memory buffer on the
> > host --- memnic  ---  VM2 application
> >
> > 2.       Special purpose Kernel switching module
> >
> > VM1 application --- memnic  ---  shared memory VM1  --- Kernel
> > switching module  --- shared memory VM2  --- memnic  ---
> > VM2 application
> >
> > 3.       Existing Kernel switching module
> >
> > VM1 application --- memnic  ---  shared memory VM1  --- existing
> > Kernel switching module (e.g. OVS/linux Bridge/VETh pair)
> > --- shared memory VM2  --- memnic  ---  VM2 application
> >
> > Can anyone recommend which approach might be best or easiest?   We would
> like to avoid writing much (or any) kernel code
> > so if there are already any open source code or test utilities that
> > provide one of these options or would be a good starting point to start
> from,  a pointer would be much appreciated.
> >
> > Thanks in advance
> >
> >
> > John Joyce

Re: [dpdk-dev] ##freemail## RE: Testing memnic for VM to VM transfer

[GongJinrong]

Hi, Hiroshi

   Do you mean I must use DPDK vSwitch in host when I use MEMNIC PMD in
guest VM? actually, I just want a channel which can put the data from host
to guest quickly. Do you have any idea that how to write a host application
to put the data to guest memnic PMD?

-----Original Message-----
From: Hiroshi Shimamoto [mailto:h-shimamoto@ct.jp.nec.com] 
Sent: Wednesday, June 18, 2014 7:11 PM
To: GongJinrong; 'John Joyce (joycej)'; dev@dpdk.org
Subject: RE: ##freemail## RE: [dpdk-dev] Testing memnic for VM to VM
transfer

Hi,

> Subject: ##freemail## RE: [dpdk-dev] Testing memnic for VM to VM 
> transfer
> 
> Hi, Hiroshi
> 
>    I just start to learn DPDK and memnic, in memnic guide, you said 
> "On host, the shared memory must be initialized by an application 
> using memnic", I am not so clear that how to initialize the share 
> memory in host, do you means use posix API or DPDK API to create the 
> share memory?(it seems memnic guest side use rte_mbuf to transfer 
> data), do you have any sample code to demo how to use memnic in host?

I don't have simple MEMNIC sample to use it on host.
Could you please try DPDK vSwitch and enables MEMNIC vport?
DPDK vSwitch must handle packets between physical NIC port and MEMNIC vport
exposed to guest with dpdk.org memnic driver.

thanks,
Hiroshi

> 
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hiroshi Shimamoto
> Sent: Wednesday, June 18, 2014 12:02 PM
> To: John Joyce (joycej); dev@dpdk.org
> Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> Hi,
> 
> > Subject: [dpdk-dev] Testing memnic for VM to VM transfer
> >
> > Hi everyone:
> >         We are interested in testing the performance of the memnic 
> > driver
> posted at     http://dpdk.org/browse/memnic/refs/.
> > We want to compare its performance compared to other techniques to 
> > transfer packets between the guest and the kernel, predominately for 
> > VM to
> VM transfers.
> >
> > We have downloaded the memnic components and have got it running in 
> > a
> guest VM.
> >
> > The question we hope this group might be able to help with is what 
> > would be the best way to processes the packets in the kernel to get 
> > a VM
> to VM transfer.
> 
> I think there is no kernel code work with MEMNIC.
> The recommend switching software on the host is Intel DPDK vSwitch 
> hosted on 01.org and github.
> https://github.com/01org/dpdk-ovs/tree/development
> 
> Intel DPDK vSwitch runs on userspace not kernel.
> 
> I introduced this mechanism to DPDK vSwitch and the guest drivers are 
> maintained in dpdk.org.
> 
> thanks,
> Hiroshi
> 
> >
> > A couple options might be possible
> >
> >
> > 1.       Common shared buffer between two VMs.  With some utility/code
to
> switch TX & RX rings between the two VMs.
> >
> > VM1 application --- memnic  ---  common shared memory buffer on the 
> > host --- memnic  ---  VM2 application
> >
> > 2.       Special purpose Kernel switching module
> >
> > VM1 application --- memnic  ---  shared memory VM1  --- Kernel 
> > switching module  --- shared memory VM2  --- memnic  ---
> > VM2 application
> >
> > 3.       Existing Kernel switching module
> >
> > VM1 application --- memnic  ---  shared memory VM1  --- existing 
> > Kernel switching module (e.g. OVS/linux Bridge/VETh pair)
> > --- shared memory VM2  --- memnic  ---  VM2 application
> >
> > Can anyone recommend which approach might be best or easiest?   We would
> like to avoid writing much (or any) kernel code
> > so if there are already any open source code or test utilities that 
> > provide one of these options or would be a good starting point to 
> > start
> from,  a pointer would be much appreciated.
> >
> > Thanks in advance
> >
> >
> > John Joyce