Re: [dpdk-dev] Testing memnic for VM to VM transfer

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[Hiroshi Shimamoto]

Hi,

> Subject: ##freemail## RE: ##freemail## RE: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> Hi, Hiroshi
> 
>    Do you mean I must use DPDK vSwitch in host when I use MEMNIC PMD in
> guest VM? actually, I just want a channel which can put the data from host
> to guest quickly. Do you have any idea that how to write a host application
> to put the data to guest memnic PMD?

Yes, basically I made the MEMNIC interface work with DPDK vSwitch.

By the way, you can mmap() the shm which specified as the ivshmem and put
the proper data to send a packet to guest PMD.
I don't have time to make proper code, but can advise you;
please see common/memnic.h and the memory layout.
1) Set magic and version in header on host.
2) Initialize PMD on guest.
3) Check the reset is 1 and set valid to 1, reset to 0 on host.
4) Use uplink area the default block size 4K.
   Set len and fill ether frame data, then set the status to 2 on host.
   Guest PMD may receive the packet.
   Proceed to the next packet block.

thanks,
Hiroshi

> 
> -----Original Message-----
> From: Hiroshi Shimamoto [mailto:h-shimamoto@ct.jp.nec.com]
> Sent: Wednesday, June 18, 2014 7:11 PM
> To: GongJinrong; 'John Joyce (joycej)'; dev@dpdk.org
> Subject: RE: ##freemail## RE: [dpdk-dev] Testing memnic for VM to VM
> transfer
> 
> Hi,
> 
> > Subject: ##freemail## RE: [dpdk-dev] Testing memnic for VM to VM
> > transfer
> >
> > Hi, Hiroshi
> >
> >    I just start to learn DPDK and memnic, in memnic guide, you said
> > "On host, the shared memory must be initialized by an application
> > using memnic", I am not so clear that how to initialize the share
> > memory in host, do you means use posix API or DPDK API to create the
> > share memory?(it seems memnic guest side use rte_mbuf to transfer
> > data), do you have any sample code to demo how to use memnic in host?
> 
> I don't have simple MEMNIC sample to use it on host.
> Could you please try DPDK vSwitch and enables MEMNIC vport?
> DPDK vSwitch must handle packets between physical NIC port and MEMNIC vport
> exposed to guest with dpdk.org memnic driver.
> 
> thanks,
> Hiroshi
> 
> >
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hiroshi Shimamoto
> > Sent: Wednesday, June 18, 2014 12:02 PM
> > To: John Joyce (joycej); dev@dpdk.org
> > Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer
> >
> > Hi,
> >
> > > Subject: [dpdk-dev] Testing memnic for VM to VM transfer
> > >
> > > Hi everyone:
> > >         We are interested in testing the performance of the memnic
> > > driver
> > posted at     http://dpdk.org/browse/memnic/refs/.
> > > We want to compare its performance compared to other techniques to
> > > transfer packets between the guest and the kernel, predominately for
> > > VM to
> > VM transfers.
> > >
> > > We have downloaded the memnic components and have got it running in
> > > a
> > guest VM.
> > >
> > > The question we hope this group might be able to help with is what
> > > would be the best way to processes the packets in the kernel to get
> > > a VM
> > to VM transfer.
> >
> > I think there is no kernel code work with MEMNIC.
> > The recommend switching software on the host is Intel DPDK vSwitch
> > hosted on 01.org and github.
> > https://github.com/01org/dpdk-ovs/tree/development
> >
> > Intel DPDK vSwitch runs on userspace not kernel.
> >
> > I introduced this mechanism to DPDK vSwitch and the guest drivers are
> > maintained in dpdk.org.
> >
> > thanks,
> > Hiroshi
> >
> > >
> > > A couple options might be possible
> > >
> > >
> > > 1.       Common shared buffer between two VMs.  With some utility/code
> to
> > switch TX & RX rings between the two VMs.
> > >
> > > VM1 application --- memnic  ---  common shared memory buffer on the
> > > host --- memnic  ---  VM2 application
> > >
> > > 2.       Special purpose Kernel switching module
> > >
> > > VM1 application --- memnic  ---  shared memory VM1  --- Kernel
> > > switching module  --- shared memory VM2  --- memnic  ---
> > > VM2 application
> > >
> > > 3.       Existing Kernel switching module
> > >
> > > VM1 application --- memnic  ---  shared memory VM1  --- existing
> > > Kernel switching module (e.g. OVS/linux Bridge/VETh pair)
> > > --- shared memory VM2  --- memnic  ---  VM2 application
> > >
> > > Can anyone recommend which approach might be best or easiest?   We would
> > like to avoid writing much (or any) kernel code
> > > so if there are already any open source code or test utilities that
> > > provide one of these options or would be a good starting point to
> > > start
> > from,  a pointer would be much appreciated.
> > >
> > > Thanks in advance
> > >
> > >
> > > John Joyce

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[Thomas Monjalon]

2014-06-18 11:42, Hiroshi Shimamoto:
> 2014-06-18 19:26, GongJinrong:
> > Do you have any idea that how to write a host application
> > to put the data to guest memnic PMD?
> 
> Yes, basically I made the MEMNIC interface work with DPDK vSwitch.
> 
> By the way, you can mmap() the shm which specified as the ivshmem and put
> the proper data to send a packet to guest PMD.
> I don't have time to make proper code, but can advise you;
> please see common/memnic.h and the memory layout.
> 1) Set magic and version in header on host.
> 2) Initialize PMD on guest.
> 3) Check the reset is 1 and set valid to 1, reset to 0 on host.
> 4) Use uplink area the default block size 4K.
>    Set len and fill ether frame data, then set the status to 2 on host.
>    Guest PMD may receive the packet.
>    Proceed to the next packet block.

Such application should be integrated in memnic repository.
I know Olivier wrote one which could be sent on next week.

-- 
Thomas

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[Hiroshi Shimamoto]

Hi,

> Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> 2014-06-18 11:42, Hiroshi Shimamoto:
> > 2014-06-18 19:26, GongJinrong:
> > > Do you have any idea that how to write a host application
> > > to put the data to guest memnic PMD?
> >
> > Yes, basically I made the MEMNIC interface work with DPDK vSwitch.
> >
> > By the way, you can mmap() the shm which specified as the ivshmem and put
> > the proper data to send a packet to guest PMD.
> > I don't have time to make proper code, but can advise you;
> > please see common/memnic.h and the memory layout.
> > 1) Set magic and version in header on host.
> > 2) Initialize PMD on guest.
> > 3) Check the reset is 1 and set valid to 1, reset to 0 on host.
> > 4) Use uplink area the default block size 4K.
> >    Set len and fill ether frame data, then set the status to 2 on host.
> >    Guest PMD may receive the packet.
> >    Proceed to the next packet block.
> 
> Such application should be integrated in memnic repository.
> I know Olivier wrote one which could be sent on next week.

yeah, I just begin to feel to need such a software in the repository.

thanks,
Hiroshi

> 
> --
> Thomas

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[GongJinrong]

Thanks guys, I will try.

-----Original Message-----
From: Hiroshi Shimamoto [mailto:h-shimamoto@ct.jp.nec.com] 
Sent: Wednesday, June 18, 2014 8:06 PM
To: Thomas Monjalon; GongJinrong
Cc: dev@dpdk.org; 'John Joyce (joycej)'; Olivier MATZ
Subject: RE: [dpdk-dev] Testing memnic for VM to VM transfer

Hi,

> Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> 2014-06-18 11:42, Hiroshi Shimamoto:
> > 2014-06-18 19:26, GongJinrong:
> > > Do you have any idea that how to write a host application to put 
> > > the data to guest memnic PMD?
> >
> > Yes, basically I made the MEMNIC interface work with DPDK vSwitch.
> >
> > By the way, you can mmap() the shm which specified as the ivshmem 
> > and put the proper data to send a packet to guest PMD.
> > I don't have time to make proper code, but can advise you; please 
> > see common/memnic.h and the memory layout.
> > 1) Set magic and version in header on host.
> > 2) Initialize PMD on guest.
> > 3) Check the reset is 1 and set valid to 1, reset to 0 on host.
> > 4) Use uplink area the default block size 4K.
> >    Set len and fill ether frame data, then set the status to 2 on host.
> >    Guest PMD may receive the packet.
> >    Proceed to the next packet block.
> 
> Such application should be integrated in memnic repository.
> I know Olivier wrote one which could be sent on next week.

yeah, I just begin to feel to need such a software in the repository.

thanks,
Hiroshi

> 
> --
> Thomas

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[Thomas Monjalon]

2014-06-18 12:06, Hiroshi Shimamoto:
> 2014-06-18 13:49, Thomas Monjalon:
> > 2014-06-18 11:42, Hiroshi Shimamoto:
> > > 2014-06-18 19:26, GongJinrong:
> > > > Do you have any idea that how to write a host application
> > > > to put the data to guest memnic PMD?
> > > 
> > > Yes, basically I made the MEMNIC interface work with DPDK vSwitch.
> > > 
> > > By the way, you can mmap() the shm which specified as the ivshmem and
> > > put
> > > the proper data to send a packet to guest PMD.
> > > I don't have time to make proper code, but can advise you;
> > > please see common/memnic.h and the memory layout.
> > > 1) Set magic and version in header on host.
> > > 2) Initialize PMD on guest.
> > > 3) Check the reset is 1 and set valid to 1, reset to 0 on host.
> > > 4) Use uplink area the default block size 4K.
> > > 
> > >    Set len and fill ether frame data, then set the status to 2 on host.
> > >    Guest PMD may receive the packet.
> > >    Proceed to the next packet block.
> > 
> > Such application should be integrated in memnic repository.
> > I know Olivier wrote one which could be sent on next week.
> 
> yeah, I just begin to feel to need such a software in the repository.

I've pushed the test application (host-sim) that Olivier wrote.
Feel free to improve it.

I think that we could also use the PMD driver in host side with some 
modifications.

-- 
Thomas

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[GongJinrong]

Thanks Thomas，I will try it^^

-----Original Message-----
From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] 
Sent: Wednesday, July 02, 2014 11:59 PM
To: Hiroshi Shimamoto
Cc: GongJinrong; dev@dpdk.org; 'John Joyce (joycej)'; Olivier MATZ
Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer

2014-06-18 12:06, Hiroshi Shimamoto:
> 2014-06-18 13:49, Thomas Monjalon:
> > 2014-06-18 11:42, Hiroshi Shimamoto:
> > > 2014-06-18 19:26, GongJinrong:
> > > > Do you have any idea that how to write a host application to put 
> > > > the data to guest memnic PMD?
> > > 
> > > Yes, basically I made the MEMNIC interface work with DPDK vSwitch.
> > > 
> > > By the way, you can mmap() the shm which specified as the ivshmem 
> > > and put the proper data to send a packet to guest PMD.
> > > I don't have time to make proper code, but can advise you; please 
> > > see common/memnic.h and the memory layout.
> > > 1) Set magic and version in header on host.
> > > 2) Initialize PMD on guest.
> > > 3) Check the reset is 1 and set valid to 1, reset to 0 on host.
> > > 4) Use uplink area the default block size 4K.
> > > 
> > >    Set len and fill ether frame data, then set the status to 2 on
host.
> > >    Guest PMD may receive the packet.
> > >    Proceed to the next packet block.
> > 
> > Such application should be integrated in memnic repository.
> > I know Olivier wrote one which could be sent on next week.
> 
> yeah, I just begin to feel to need such a software in the repository.

I've pushed the test application (host-sim) that Olivier wrote.
Feel free to improve it.

I think that we could also use the PMD driver in host side with some
modifications.

--
Thomas

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[GongJinrong]

Hi, Thomas, I cannot find the host-sim, where can I get it?

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of GongJinrong
Sent: Thursday, July 03, 2014 2:35 PM
To: 'Thomas Monjalon'; 'Hiroshi Shimamoto'
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer

Thanks Thomas，I will try it^^

-----Original Message-----
From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
Sent: Wednesday, July 02, 2014 11:59 PM
To: Hiroshi Shimamoto
Cc: GongJinrong; dev@dpdk.org; 'John Joyce (joycej)'; Olivier MATZ
Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer

2014-06-18 12:06, Hiroshi Shimamoto:
> 2014-06-18 13:49, Thomas Monjalon:
> > 2014-06-18 11:42, Hiroshi Shimamoto:
> > > 2014-06-18 19:26, GongJinrong:
> > > > Do you have any idea that how to write a host application to put 
> > > > the data to guest memnic PMD?
> > > 
> > > Yes, basically I made the MEMNIC interface work with DPDK vSwitch.
> > > 
> > > By the way, you can mmap() the shm which specified as the ivshmem 
> > > and put the proper data to send a packet to guest PMD.
> > > I don't have time to make proper code, but can advise you; please 
> > > see common/memnic.h and the memory layout.
> > > 1) Set magic and version in header on host.
> > > 2) Initialize PMD on guest.
> > > 3) Check the reset is 1 and set valid to 1, reset to 0 on host.
> > > 4) Use uplink area the default block size 4K.
> > > 
> > >    Set len and fill ether frame data, then set the status to 2 on
host.
> > >    Guest PMD may receive the packet.
> > >    Proceed to the next packet block.
> > 
> > Such application should be integrated in memnic repository.
> > I know Olivier wrote one which could be sent on next week.
> 
> yeah, I just begin to feel to need such a software in the repository.

I've pushed the test application (host-sim) that Olivier wrote.
Feel free to improve it.

I think that we could also use the PMD driver in host side with some
modifications.

--
Thomas

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[Thomas Monjalon]

2014-07-03 19:03, GongJinrong:
> Hi, Thomas, I cannot find the host-sim, where can I get it?

In memnic repository:
	http://dpdk.org/browse/memnic/commit/?id=18e78437e75ef3f

-- 
Thomas

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[xie huawei]

Hi Thomas:

Do we have recommendation between virtio-PMD and memnic-PMD?


On Thu, Jul 3, 2014 at 8:05 PM, Thomas Monjalon
<thomas.monjalon@6wind.com> wrote:
> 2014-07-03 19:03, GongJinrong:
>> Hi, Thomas, I cannot find the host-sim, where can I get it?
>
> In memnic repository:
>         http://dpdk.org/browse/memnic/commit/?id=18e78437e75ef3f
>
> --
> Thomas

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[Thomas Monjalon]

Hi,

2014-07-07 17:58, xie huawei:
> Hi Thomas:
> 
> Do we have recommendation between virtio-PMD and memnic-PMD?

I don't understand your question.
I think your question is not related to the test application for memnic.
Please, could you open a new thread for this question (more detailed)?

-- 
Thomas


> On Thu, Jul 3, 2014 at 8:05 PM, Thomas Monjalon
> 
> <thomas.monjalon@6wind.com> wrote:
> > 2014-07-03 19:03, GongJinrong:
> >> Hi, Thomas, I cannot find the host-sim, where can I get it?
> > 
> > In memnic repository:
> >         http://dpdk.org/browse/memnic/commit/?id=18e78437e75ef3f
> > 
> > --
> > Thomas

[dpdk-dev] Testing memnic for VM to VM transfer

[John Joyce (joycej)]

Hi everyone:
        We are interested in testing the performance of the memnic driver posted at     http://dpdk.org/browse/memnic/refs/.   We want to compare its performance compared to other techniques to transfer packets between the guest and the kernel,  predominately for VM to VM transfers.

We have downloaded the memnic components and have got it running in a guest VM.

The question we hope this group might be able to help with is what would be the best way to processes the packets in the kernel to get a VM to VM transfer.

A couple options might be possible


1.       Common shared buffer between two VMs.  With some utility/code to switch TX & RX rings between the two VMs.

VM1 application --- memnic  ---  common shared memory buffer on the host --- memnic  ---  VM2 application

2.       Special purpose Kernel switching module

VM1 application --- memnic  ---  shared memory VM1  --- Kernel switching module  --- shared memory VM2  --- memnic  ---  VM2 application

3.       Existing Kernel switching module

VM1 application --- memnic  ---  shared memory VM1  --- existing Kernel switching module (e.g. OVS/linux Bridge/VETh pair)  --- shared memory VM2  --- memnic  ---  VM2 application

Can anyone recommend which approach might be best or easiest?   We would like to avoid writing much (or any) kernel code so if there are already any open source code or test utilities that provide one of these options or would be a good starting point to start from,  a pointer would be much appreciated.

Thanks in advance

                                                                                                                                                                John Joyce

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[Hiroshi Shimamoto]

Hi,

> Subject: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> Hi everyone:
>         We are interested in testing the performance of the memnic driver posted at     http://dpdk.org/browse/memnic/refs/.
> We want to compare its performance compared to other techniques to transfer packets between the guest and the kernel,
> predominately for VM to VM transfers.
> 
> We have downloaded the memnic components and have got it running in a guest VM.
> 
> The question we hope this group might be able to help with is what would be the best way to processes the packets in the
> kernel to get a VM to VM transfer.

I think there is no kernel code work with MEMNIC.
The recommend switching software on the host is Intel DPDK vSwitch hosted on 01.org and github.
https://github.com/01org/dpdk-ovs/tree/development

Intel DPDK vSwitch runs on userspace not kernel.

I introduced this mechanism to DPDK vSwitch and the guest drivers are maintained in dpdk.org.

thanks,
Hiroshi

> 
> A couple options might be possible
> 
> 
> 1.       Common shared buffer between two VMs.  With some utility/code to switch TX & RX rings between the two VMs.
> 
> VM1 application --- memnic  ---  common shared memory buffer on the host --- memnic  ---  VM2 application
> 
> 2.       Special purpose Kernel switching module
> 
> VM1 application --- memnic  ---  shared memory VM1  --- Kernel switching module  --- shared memory VM2  --- memnic  ---
> VM2 application
> 
> 3.       Existing Kernel switching module
> 
> VM1 application --- memnic  ---  shared memory VM1  --- existing Kernel switching module (e.g. OVS/linux Bridge/VETh pair)
> --- shared memory VM2  --- memnic  ---  VM2 application
> 
> Can anyone recommend which approach might be best or easiest?   We would like to avoid writing much (or any) kernel code
> so if there are already any open source code or test utilities that provide one of these options or would be a good starting
> point to start from,  a pointer would be much appreciated.
> 
> Thanks in advance
> 
> 
> John Joyce

Re: [dpdk-dev] Testing memnic for VM to VM transfer

[GongJinrong]

Hi, Hiroshi

   I just start to learn DPDK and memnic, in memnic guide, you said "On
host, the shared memory must be initialized by an application using memnic",
I am not so clear that how to initialize the share memory in host, do you
means use posix API or DPDK API to create the share memory?(it seems memnic
guest side use rte_mbuf to transfer data), do you have any sample code to
demo how to use memnic in host?

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Hiroshi Shimamoto
Sent: Wednesday, June 18, 2014 12:02 PM
To: John Joyce (joycej); dev@dpdk.org
Subject: Re: [dpdk-dev] Testing memnic for VM to VM transfer

Hi,

> Subject: [dpdk-dev] Testing memnic for VM to VM transfer
> 
> Hi everyone:
>         We are interested in testing the performance of the memnic driver
posted at     http://dpdk.org/browse/memnic/refs/.
> We want to compare its performance compared to other techniques to 
> transfer packets between the guest and the kernel, predominately for VM to
VM transfers.
> 
> We have downloaded the memnic components and have got it running in a
guest VM.
> 
> The question we hope this group might be able to help with is what 
> would be the best way to processes the packets in the kernel to get a VM
to VM transfer.

I think there is no kernel code work with MEMNIC.
The recommend switching software on the host is Intel DPDK vSwitch hosted on
01.org and github.
https://github.com/01org/dpdk-ovs/tree/development

Intel DPDK vSwitch runs on userspace not kernel.

I introduced this mechanism to DPDK vSwitch and the guest drivers are
maintained in dpdk.org.

thanks,
Hiroshi

> 
> A couple options might be possible
> 
> 
> 1.       Common shared buffer between two VMs.  With some utility/code to
switch TX & RX rings between the two VMs.
> 
> VM1 application --- memnic  ---  common shared memory buffer on the 
> host --- memnic  ---  VM2 application
> 
> 2.       Special purpose Kernel switching module
> 
> VM1 application --- memnic  ---  shared memory VM1  --- Kernel 
> switching module  --- shared memory VM2  --- memnic  ---
> VM2 application
> 
> 3.       Existing Kernel switching module
> 
> VM1 application --- memnic  ---  shared memory VM1  --- existing 
> Kernel switching module (e.g. OVS/linux Bridge/VETh pair)
> --- shared memory VM2  --- memnic  ---  VM2 application
> 
> Can anyone recommend which approach might be best or easiest?   We would
like to avoid writing much (or any) kernel code
> so if there are already any open source code or test utilities that 
> provide one of these options or would be a good starting point to start
from,  a pointer would be much appreciated.
> 
> Thanks in advance
> 
> 
> John Joyce