[dpdk-users] OVS-DPDK - Very poor performance when connected to namespace/container

[dpdk-users] OVS-DPDK - Very poor performance when connected to namespace/container

[Avi Cohen (A)]

Hello   All,
I have OVS-DPDK connected to a namespace via veth pair device.

I've got a very poor performance - compared to normal OVS (i.e. no DPDK). 
For example - TCP jumbo pkts throughput: normal OVS  ~ 10Gbps , OVS-DPDK 1.7 Gbps.

This can be explained as follows: 
veth is implemented in kernel - in OVS-DPDK data is transferred from veth to user space while in normal OVS we save this transfer

Is there any other device to connect to namespace ? something like vhost-user ? I understand that vhost-user cannot be used for namespace 

Best Regards
avi

Re: [dpdk-users] OVS-DPDK - Very poor performance when connected to namespace/container

[Avi Cohen (A)]

> -----Original Message-----
> From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
> Sent: Thursday, 15 June, 2017 11:24 AM
> To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org; ovs-
> discuss@openvswitch.org
> Cc: Mooney, Sean K
> Subject: RE: OVS-DPDK - Very poor performance when connected to
> namespace/container
> 
> 
> 
> > -----Original Message-----
> > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf Of Avi
> > Cohen (A)
> > Sent: Thursday, June 15, 2017 8:14 AM
> > To: dpdk-ovs@lists.01.org; users@dpdk.org; ovs-discuss@openvswitch.org
> > Subject: [Dpdk-ovs] OVS-DPDK - Very poor performance when connected to
> > namespace/container
> >
> > Hello   All,
> > I have OVS-DPDK connected to a namespace via veth pair device.
> >
> > I've got a very poor performance - compared to normal OVS (i.e. no
> > DPDK).
> > For example - TCP jumbo pkts throughput: normal OVS  ~ 10Gbps , OVS-
> > DPDK 1.7 Gbps.
> >
> > This can be explained as follows:
> > veth is implemented in kernel - in OVS-DPDK data is transferred from
> > veth to user space while in normal OVS we save this transfer
> [Mooney, Sean K] that is part of the reason, the other reson this is slow and The
> main limiter to scalling adding veth pairs or ovs internal port to ovs with dpdk is
> That these linux kernel ports are not processed by the dpdk pmds. They are
> server by the Ovs-vswitchd main thread via a fall back to the non dpdk
> acclarated netdev implementation.
> >
> > Is there any other device to connect to namespace ? something like
> > vhost-user ? I understand that vhost-user cannot be used for namespace
> [Mooney, Sean K] I have been doing some experiments in this regard.
> You should be able to use the tap, pcap or afpacket pmd to add a vedv that will
> improve Performance. I have seen some strange issue with the tap pmd that
> cause packet to be drop By the kernel on tx on some ports but not others so
> there may be issues with that dirver.
> 
> Previous experiment with libpcap seemed to work well with ovs 2.5 but I have
> not tried it With ovs 2.7/master since the introduction of generic vdev support
> at runtime. Previously vdevs And to be allocated using the dpdk args.
> 
> I would try following the af_packet example here
> https://github.com/openvswitch/ovs/blob/b132189d8456f38f3ee139f126d6809
> 01a9ee9a8/Documentation/howto/dpdk.rst#vdev-support
> 
[Avi Cohen (A)] 
Thank you Mooney, Sean K
I already tried to connect the namespace with a tap device (see 1 & 2 below)  - and got the worst performance 
 for some reason the packet  is cut to default MTU inside the  OVS-DPDK which transmit the packet to its peer. - although all interfaces MTU were set to 9000.

 1. ovs-vsctl add-port $BRIDGE tap1 -- set Interface tap1 type=internal
 
 2. ip link set tap1 netns ns1 // attach it to namespace

I'm looking at your link to create a virtual PMD with vdev support - I see there a creation of a virtual PMD device , but I'm not sure how this is connected to the namespace ?  what device should I assign to the namespace ? 

Best Regards
avi

> if you happen to be investigating this for use with openstack routers we Are
> currently working on a way to remove the use of namespace entirely for dvr
> when using The default neutron agent and sdn controllers such as ovn already
> provide that functionality.
> >
> > Best Regards
> > avi
> > _______________________________________________
> > Dpdk-ovs mailing list
> > Dpdk-ovs@lists.01.org
> > https://lists.01.org/mailman/listinfo/dpdk-ovs

Re: [dpdk-users] OVS-DPDK - Very poor performance when connected to namespace/container

[Avi Cohen (A)]

> -----Original Message-----
> From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
> Sent: Thursday, 15 June, 2017 2:33 PM
> To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org; ovs-
> discuss@openvswitch.org
> Subject: RE: OVS-DPDK - Very poor performance when connected to
> namespace/container
> 
> 
> 
> > -----Original Message-----
> > From: Avi Cohen (A) [mailto:avi.cohen@huawei.com]
> > Sent: Thursday, June 15, 2017 9:50 AM
> > To: Mooney, Sean K <sean.k.mooney@intel.com>; dpdk-ovs@lists.01.org;
> > users@dpdk.org; ovs-discuss@openvswitch.org
> > Subject: RE: OVS-DPDK - Very poor performance when connected to
> > namespace/container
> >
> >
> >
> > > -----Original Message-----
> > > From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
> > > Sent: Thursday, 15 June, 2017 11:24 AM
> > > To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org; ovs-
> > > discuss@openvswitch.org
> > > Cc: Mooney, Sean K
> > > Subject: RE: OVS-DPDK - Very poor performance when connected to
> > > namespace/container
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf Of
> > > > Avi Cohen (A)
> > > > Sent: Thursday, June 15, 2017 8:14 AM
> > > > To: dpdk-ovs@lists.01.org; users@dpdk.org;
> > > > ovs-discuss@openvswitch.org
> > > > Subject: [Dpdk-ovs] OVS-DPDK - Very poor performance when
> > > > connected to namespace/container
> > > >
> > > > Hello   All,
> > > > I have OVS-DPDK connected to a namespace via veth pair device.
> > > >
> > > > I've got a very poor performance - compared to normal OVS (i.e. no
> > > > DPDK).
> > > > For example - TCP jumbo pkts throughput: normal OVS  ~ 10Gbps ,
> > OVS-
> > > > DPDK 1.7 Gbps.
> > > >
> > > > This can be explained as follows:
> > > > veth is implemented in kernel - in OVS-DPDK data is transferred
> > from
> > > > veth to user space while in normal OVS we save this transfer
> > > [Mooney, Sean K] that is part of the reason, the other reson this is
> > > slow and The main limiter to scalling adding veth pairs or ovs
> > > internal port to ovs with dpdk is That these linux kernel ports are
> > > not processed by the dpdk pmds. They are server by the Ovs-vswitchd
> > > main thread via a fall back to the non dpdk acclarated netdev
> > implementation.
> > > >
> > > > Is there any other device to connect to namespace ? something like
> > > > vhost-user ? I understand that vhost-user cannot be used for
> > > > namespace
> > > [Mooney, Sean K] I have been doing some experiments in this regard.
> > > You should be able to use the tap, pcap or afpacket pmd to add a
> > > vedv that will improve Performance. I have seen some strange issue
> > > with
> > the
> > > tap pmd that cause packet to be drop By the kernel on tx on some
> > ports
> > > but not others so there may be issues with that dirver.
> > >
> > > Previous experiment with libpcap seemed to work well with ovs 2.5
> > > but I have not tried it With ovs 2.7/master since the introduction
> > > of generic vdev support at runtime. Previously vdevs And to be
> > > allocated
> > using the dpdk args.
> > >
> > > I would try following the af_packet example here
> > >
> > https://github.com/openvswitch/ovs/blob/b132189d8456f38f3ee139f126d680
> > > 9 01a9ee9a8/Documentation/howto/dpdk.rst#vdev-support
> > >
> > [Avi Cohen (A)]
> > Thank you Mooney, Sean K
> > I already tried to connect the namespace with a tap device (see 1 & 2
> > below)  - and got the worst performance  for some reason the packet
> > is cut to default MTU inside the  OVS-DPDK which transmit the packet
> > to its peer. - although all interfaces MTU were set to 9000.
> >
> >  1. ovs-vsctl add-port $BRIDGE tap1 -- set Interface tap1
> > type=internal
> >
> >  2. ip link set tap1 netns ns1 // attach it to namespace
> [Mooney, Sean K] this is not using the dpdk tap pmd , internal port and veth
> ports If added to ovs will not be accelerated by dpdk unless you use a vdev to
> attach them.
> >
> > I'm looking at your link to create a virtual PMD with vdev support - I
> > see there a creation of a virtual PMD device , but I'm not sure how
> > this is connected to the namespace ?  what device should I assign to
> > the namespace ?
> [Mooney, Sean K]
> You would use it as follows
> 
> ip tuntap add dev tap1 mode tap
> 
> ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
> options:dpdk-devargs=eth_af_packet0,iface=tap1
[Avi Cohen (A)] 
Thanks Sean - are u sure about the syntax - I get an error msg  [could not open network device tap1 - No such device]  - when I add-port
The syntax in your link is different  - note there is myeth0 and eth0 while in your command only tap1 
The command in the link is as follows:
" ovs-vsctl add-port br0 myeth0 -- set Interface myeth0 type=dpdk \
    options:dpdk-devargs=eth_af_packet0,iface=eth0"

> 
> ip link set tap1 netns ns1
> 
> ip netns exec ns1 ifconfig 192.168.1.1/24 up
> 
> in general though if you are using ovs-dpdk you should avoid using network
> namespace and the kernel where possible but the above should improve you
> performance. One caveat, the amount of vdev+phyical interfaces is limited by
> how dpdk is compiled by default to 32 devices but it can be increased to 256 if
> required.
> 
> >
> > Best Regards
> > avi
> >
> > > if you happen to be investigating this for use with openstack
> > > routers we Are currently working on a way to remove the use of
> > > namespace entirely for dvr when using The default neutron agent and
> > > sdn controllers such as ovn already provide that functionality.
> > > >
> > > > Best Regards
> > > > avi
> > > > _______________________________________________
> > > > Dpdk-ovs mailing list
> > > > Dpdk-ovs@lists.01.org
> > > > https://lists.01.org/mailman/listinfo/dpdk-ovs

Re: [dpdk-users] OVS-DPDK - Very poor performance when connected to namespace/container

[Gray, Mark D]

Hi Avi,

The other option is to use virtio-use (http://dpdk.org/doc/guides/howto/virtio_user_for_container_networking.html) which gives dpdk-like performance to a dpdk application running in a container. The configuration for ovs-dpdk has not been documented but it is possible to use (as a vdev).

Also, I have dropped the dpdk-ovs@lists.01.org mail address as this is for the discontinued ovdk project.

Mark

> -----Original Message-----
> From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf Of
> Mooney, Sean K
> Sent: Thursday, June 15, 2017 12:33 PM
> To: Avi Cohen (A) <avi.cohen@huawei.com>; dpdk-ovs@lists.01.org;
> users@dpdk.org; ovs-discuss@openvswitch.org
> Subject: Re: [Dpdk-ovs] OVS-DPDK - Very poor performance when
> connected to namespace/container
> 
> 
> 
> > -----Original Message-----
> > From: Avi Cohen (A) [mailto:avi.cohen@huawei.com]
> > Sent: Thursday, June 15, 2017 9:50 AM
> > To: Mooney, Sean K <sean.k.mooney@intel.com>; dpdk-ovs@lists.01.org;
> > users@dpdk.org; ovs-discuss@openvswitch.org
> > Subject: RE: OVS-DPDK - Very poor performance when connected to
> > namespace/container
> >
> >
> >
> > > -----Original Message-----
> > > From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
> > > Sent: Thursday, 15 June, 2017 11:24 AM
> > > To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org; ovs-
> > > discuss@openvswitch.org
> > > Cc: Mooney, Sean K
> > > Subject: RE: OVS-DPDK - Very poor performance when connected to
> > > namespace/container
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf Of
> > > > Avi Cohen (A)
> > > > Sent: Thursday, June 15, 2017 8:14 AM
> > > > To: dpdk-ovs@lists.01.org; users@dpdk.org;
> > > > ovs-discuss@openvswitch.org
> > > > Subject: [Dpdk-ovs] OVS-DPDK - Very poor performance when
> > > > connected to namespace/container
> > > >
> > > > Hello   All,
> > > > I have OVS-DPDK connected to a namespace via veth pair device.
> > > >
> > > > I've got a very poor performance - compared to normal OVS (i.e. no
> > > > DPDK).
> > > > For example - TCP jumbo pkts throughput: normal OVS  ~ 10Gbps ,
> > OVS-
> > > > DPDK 1.7 Gbps.
> > > >
> > > > This can be explained as follows:
> > > > veth is implemented in kernel - in OVS-DPDK data is transferred
> > from
> > > > veth to user space while in normal OVS we save this transfer
> > > [Mooney, Sean K] that is part of the reason, the other reson this is
> > > slow and The main limiter to scalling adding veth pairs or ovs
> > > internal port to ovs with dpdk is That these linux kernel ports are
> > > not processed by the dpdk pmds. They are server by the Ovs-vswitchd
> > > main thread via a fall back to the non dpdk acclarated netdev
> > implementation.
> > > >
> > > > Is there any other device to connect to namespace ? something like
> > > > vhost-user ? I understand that vhost-user cannot be used for
> > > > namespace
> > > [Mooney, Sean K] I have been doing some experiments in this regard.
> > > You should be able to use the tap, pcap or afpacket pmd to add a
> > > vedv that will improve Performance. I have seen some strange issue
> > > with
> > the
> > > tap pmd that cause packet to be drop By the kernel on tx on some
> > ports
> > > but not others so there may be issues with that dirver.
> > >
> > > Previous experiment with libpcap seemed to work well with ovs 2.5
> > > but I have not tried it With ovs 2.7/master since the introduction
> > > of generic vdev support at runtime. Previously vdevs And to be
> > > allocated
> > using the dpdk args.
> > >
> > > I would try following the af_packet example here
> > >
> >
> https://github.com/openvswitch/ovs/blob/b132189d8456f38f3ee139f126d68
> 0
> > > 9 01a9ee9a8/Documentation/howto/dpdk.rst#vdev-support
> > >
> > [Avi Cohen (A)]
> > Thank you Mooney, Sean K
> > I already tried to connect the namespace with a tap device (see 1 & 2
> > below)  - and got the worst performance  for some reason the packet
> > is cut to default MTU inside the  OVS-DPDK which transmit the packet
> > to its peer. - although all interfaces MTU were set to 9000.
> >
> >  1. ovs-vsctl add-port $BRIDGE tap1 -- set Interface tap1
> > type=internal
> >
> >  2. ip link set tap1 netns ns1 // attach it to namespace
> [Mooney, Sean K] this is not using the dpdk tap pmd , internal port and veth
> ports If added to ovs will not be accelerated by dpdk unless you use a vdev to
> attach them.
> >
> > I'm looking at your link to create a virtual PMD with vdev support - I
> > see there a creation of a virtual PMD device , but I'm not sure how
> > this is connected to the namespace ?  what device should I assign to
> > the namespace ?
> [Mooney, Sean K]
> You would use it as follows
> 
> ip tuntap add dev tap1 mode tap
> 
> ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
> options:dpdk-devargs=eth_af_packet0,iface=tap1
> 
> ip link set tap1 netns ns1
> 
> ip netns exec ns1 ifconfig 192.168.1.1/24 up
> 
> in general though if you are using ovs-dpdk you should avoid using network
> namespace and the kernel where possible but the above should improve
> you performance. One caveat, the amount of vdev+phyical interfaces is
> limited by how dpdk is compiled by default to 32 devices but it can be
> increased to 256 if required.
> 
> >
> > Best Regards
> > avi
> >
> > > if you happen to be investigating this for use with openstack
> > > routers we Are currently working on a way to remove the use of
> > > namespace entirely for dvr when using The default neutron agent and
> > > sdn controllers such as ovn already provide that functionality.
> > > >
> > > > Best Regards
> > > > avi
> > > > _______________________________________________
> > > > Dpdk-ovs mailing list
> > > > Dpdk-ovs@lists.01.org
> > > > https://lists.01.org/mailman/listinfo/dpdk-ovs
> _______________________________________________
> Dpdk-ovs mailing list
> Dpdk-ovs@lists.01.org
> https://lists.01.org/mailman/listinfo/dpdk-ovs

Re: [dpdk-users] [ovs-discuss] OVS-DPDK - Very poor performance when connected to namespace/container

[Darrell Ball]

This should be quite a bit better than the AF_PACKET PMD. 
This becomes available in 17.08 and is important, it would be nice to get into the next OVS release.

Also, any existing data on the performance advantage of AF_PACKET PMD with single queue ?

Darrell


On 6/16/17, 1:56 AM, "ovs-discuss-bounces@openvswitch.org on behalf of Gray, Mark D" <ovs-discuss-bounces@openvswitch.org on behalf of mark.d.gray@intel.com> wrote:

    Hi Avi,
    
    The other option is to use virtio-use (https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides_howto_virtio-5Fuser-5Ffor-5Fcontainer-5Fnetworking.html&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ThmUVGZC2M1LBAIhnvs-5OiSZz7ywpez2Qj70BrQjoM&e= ) which gives dpdk-like performance to a dpdk application running in a container. The configuration for ovs-dpdk has not been documented but it is possible to use (as a vdev).
    
    Also, I have dropped the dpdk-ovs@lists.01.org mail address as this is for the discontinued ovdk project.
    
    Mark
    
    > -----Original Message-----
    > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf Of
    > Mooney, Sean K
    > Sent: Thursday, June 15, 2017 12:33 PM
    > To: Avi Cohen (A) <avi.cohen@huawei.com>; dpdk-ovs@lists.01.org;
    > users@dpdk.org; ovs-discuss@openvswitch.org
    > Subject: Re: [Dpdk-ovs] OVS-DPDK - Very poor performance when
    > connected to namespace/container
    > 
    > 
    > 
    > > -----Original Message-----
    > > From: Avi Cohen (A) [mailto:avi.cohen@huawei.com]
    > > Sent: Thursday, June 15, 2017 9:50 AM
    > > To: Mooney, Sean K <sean.k.mooney@intel.com>; dpdk-ovs@lists.01.org;
    > > users@dpdk.org; ovs-discuss@openvswitch.org
    > > Subject: RE: OVS-DPDK - Very poor performance when connected to
    > > namespace/container
    > >
    > >
    > >
    > > > -----Original Message-----
    > > > From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
    > > > Sent: Thursday, 15 June, 2017 11:24 AM
    > > > To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org; ovs-
    > > > discuss@openvswitch.org
    > > > Cc: Mooney, Sean K
    > > > Subject: RE: OVS-DPDK - Very poor performance when connected to
    > > > namespace/container
    > > >
    > > >
    > > >
    > > > > -----Original Message-----
    > > > > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf Of
    > > > > Avi Cohen (A)
    > > > > Sent: Thursday, June 15, 2017 8:14 AM
    > > > > To: dpdk-ovs@lists.01.org; users@dpdk.org;
    > > > > ovs-discuss@openvswitch.org
    > > > > Subject: [Dpdk-ovs] OVS-DPDK - Very poor performance when
    > > > > connected to namespace/container
    > > > >
    > > > > Hello   All,
    > > > > I have OVS-DPDK connected to a namespace via veth pair device.
    > > > >
    > > > > I've got a very poor performance - compared to normal OVS (i.e. no
    > > > > DPDK).
    > > > > For example - TCP jumbo pkts throughput: normal OVS  ~ 10Gbps ,
    > > OVS-
    > > > > DPDK 1.7 Gbps.
    > > > >
    > > > > This can be explained as follows:
    > > > > veth is implemented in kernel - in OVS-DPDK data is transferred
    > > from
    > > > > veth to user space while in normal OVS we save this transfer
    > > > [Mooney, Sean K] that is part of the reason, the other reson this is
    > > > slow and The main limiter to scalling adding veth pairs or ovs
    > > > internal port to ovs with dpdk is That these linux kernel ports are
    > > > not processed by the dpdk pmds. They are server by the Ovs-vswitchd
    > > > main thread via a fall back to the non dpdk acclarated netdev
    > > implementation.
    > > > >
    > > > > Is there any other device to connect to namespace ? something like
    > > > > vhost-user ? I understand that vhost-user cannot be used for
    > > > > namespace
    > > > [Mooney, Sean K] I have been doing some experiments in this regard.
    > > > You should be able to use the tap, pcap or afpacket pmd to add a
    > > > vedv that will improve Performance. I have seen some strange issue
    > > > with
    > > the
    > > > tap pmd that cause packet to be drop By the kernel on tx on some
    > > ports
    > > > but not others so there may be issues with that dirver.
    > > >
    > > > Previous experiment with libpcap seemed to work well with ovs 2.5
    > > > but I have not tried it With ovs 2.7/master since the introduction
    > > > of generic vdev support at runtime. Previously vdevs And to be
    > > > allocated
    > > using the dpdk args.
    > > >
    > > > I would try following the af_packet example here
    > > >
    > >
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openvswitch_ovs_blob_b132189d8456f38f3ee139f126d68&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=REyqxTB8Gd9BnEtetH_Aul0OgyyGK0DFhKl3tFGzOGI&e= 
    > 0
    > > > 9 01a9ee9a8/Documentation/howto/dpdk.rst#vdev-support
    > > >
    > > [Avi Cohen (A)]
    > > Thank you Mooney, Sean K
    > > I already tried to connect the namespace with a tap device (see 1 & 2
    > > below)  - and got the worst performance  for some reason the packet
    > > is cut to default MTU inside the  OVS-DPDK which transmit the packet
    > > to its peer. - although all interfaces MTU were set to 9000.
    > >
    > >  1. ovs-vsctl add-port $BRIDGE tap1 -- set Interface tap1
    > > type=internal
    > >
    > >  2. ip link set tap1 netns ns1 // attach it to namespace
    > [Mooney, Sean K] this is not using the dpdk tap pmd , internal port and veth
    > ports If added to ovs will not be accelerated by dpdk unless you use a vdev to
    > attach them.
    > >
    > > I'm looking at your link to create a virtual PMD with vdev support - I
    > > see there a creation of a virtual PMD device , but I'm not sure how
    > > this is connected to the namespace ?  what device should I assign to
    > > the namespace ?
    > [Mooney, Sean K]
    > You would use it as follows
    > 
    > ip tuntap add dev tap1 mode tap
    > 
    > ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
    > options:dpdk-devargs=eth_af_packet0,iface=tap1
    > 
    > ip link set tap1 netns ns1
    > 
    > ip netns exec ns1 ifconfig 192.168.1.1/24 up
    > 
    > in general though if you are using ovs-dpdk you should avoid using network
    > namespace and the kernel where possible but the above should improve
    > you performance. One caveat, the amount of vdev+phyical interfaces is
    > limited by how dpdk is compiled by default to 32 devices but it can be
    > increased to 256 if required.
    > 
    > >
    > > Best Regards
    > > avi
    > >
    > > > if you happen to be investigating this for use with openstack
    > > > routers we Are currently working on a way to remove the use of
    > > > namespace entirely for dvr when using The default neutron agent and
    > > > sdn controllers such as ovn already provide that functionality.
    > > > >
    > > > > Best Regards
    > > > > avi
    > > > > _______________________________________________
    > > > > Dpdk-ovs mailing list
    > > > > Dpdk-ovs@lists.01.org
    > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.01.org_mailman_listinfo_dpdk-2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0MKa41bi9F6bBW3Srw-WghpVzyDs-5oSFc&e= 
    > _______________________________________________
    > Dpdk-ovs mailing list
    > Dpdk-ovs@lists.01.org
    > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.01.org_mailman_listinfo_dpdk-2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0MKa41bi9F6bBW3Srw-WghpVzyDs-5oSFc&e= 
    _______________________________________________
    discuss mailing list
    discuss@openvswitch.org
    https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.openvswitch.org_mailman_listinfo_ovs-2Ddiscuss&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=5dbke1-uxXxS5IYomkpMMPVvbXG9uXZthWdONHRQapo&e= 
    

Re: [dpdk-users] [ovs-discuss] OVS-DPDK - Very poor performance when connected to namespace/container

[Mooney, Sean K]

> -----Original Message-----
> From: Darrell Ball [mailto:dball@vmware.com]
> Sent: Friday, June 16, 2017 5:53 PM
> To: Gray, Mark D <mark.d.gray@intel.com>; Mooney, Sean K
> <sean.k.mooney@intel.com>; Avi Cohen (A) <avi.cohen@huawei.com>;
> users@dpdk.org; ovs-discuss@openvswitch.org
> Subject: Re: [ovs-discuss] OVS-DPDK - Very poor performance when
> connected to namespace/container
> 
> This should be quite a bit better than the AF_PACKET PMD.
> This becomes available in 17.08 and is important, it would be nice to
> get into the next OVS release.
[Mooney, Sean K] 
Virtio user will work only if you have dpdk in the container/namespce correct?
The vhsot frontend is being provisioned by isided the dpdk vhost driver in the application.

I agree that is the way to go if your application support dpdk, if not you would need
To use one of the other vdevs.

> 
> Also, any existing data on the performance advantage of AF_PACKET PMD
> with single queue ?
[Mooney, Sean K] I would be interested in seeing this also if anyone has data to share.
> 
> Darrell
> 
> 
> On 6/16/17, 1:56 AM, "ovs-discuss-bounces@openvswitch.org on behalf of
> Gray, Mark D" <ovs-discuss-bounces@openvswitch.org on behalf of
> mark.d.gray@intel.com> wrote:
> 
>     Hi Avi,
> 
>     The other option is to use virtio-use
> (https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__dpdk.org_doc_guides_howto_virtio-5Fuser-5Ffor-5Fcontainer-
> 5Fnetworking.html&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ThmUVGZC2M1LBAIhn
> vs-5OiSZz7ywpez2Qj70BrQjoM&e= ) which gives dpdk-like performance to a
> dpdk application running in a container. The configuration for ovs-dpdk
> has not been documented but it is possible to use (as a vdev).
> 
>     Also, I have dropped the dpdk-ovs@lists.01.org mail address as this
> is for the discontinued ovdk project.
> 
>     Mark
> 
>     > -----Original Message-----
>     > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf
> Of
>     > Mooney, Sean K
>     > Sent: Thursday, June 15, 2017 12:33 PM
>     > To: Avi Cohen (A) <avi.cohen@huawei.com>; dpdk-ovs@lists.01.org;
>     > users@dpdk.org; ovs-discuss@openvswitch.org
>     > Subject: Re: [Dpdk-ovs] OVS-DPDK - Very poor performance when
>     > connected to namespace/container
>     >
>     >
>     >
>     > > -----Original Message-----
>     > > From: Avi Cohen (A) [mailto:avi.cohen@huawei.com]
>     > > Sent: Thursday, June 15, 2017 9:50 AM
>     > > To: Mooney, Sean K <sean.k.mooney@intel.com>; dpdk-
> ovs@lists.01.org;
>     > > users@dpdk.org; ovs-discuss@openvswitch.org
>     > > Subject: RE: OVS-DPDK - Very poor performance when connected to
>     > > namespace/container
>     > >
>     > >
>     > >
>     > > > -----Original Message-----
>     > > > From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
>     > > > Sent: Thursday, 15 June, 2017 11:24 AM
>     > > > To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org;
> ovs-
>     > > > discuss@openvswitch.org
>     > > > Cc: Mooney, Sean K
>     > > > Subject: RE: OVS-DPDK - Very poor performance when connected
> to
>     > > > namespace/container
>     > > >
>     > > >
>     > > >
>     > > > > -----Original Message-----
>     > > > > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On
> Behalf Of
>     > > > > Avi Cohen (A)
>     > > > > Sent: Thursday, June 15, 2017 8:14 AM
>     > > > > To: dpdk-ovs@lists.01.org; users@dpdk.org;
>     > > > > ovs-discuss@openvswitch.org
>     > > > > Subject: [Dpdk-ovs] OVS-DPDK - Very poor performance when
>     > > > > connected to namespace/container
>     > > > >
>     > > > > Hello   All,
>     > > > > I have OVS-DPDK connected to a namespace via veth pair
> device.
>     > > > >
>     > > > > I've got a very poor performance - compared to normal OVS
> (i.e. no
>     > > > > DPDK).
>     > > > > For example - TCP jumbo pkts throughput: normal OVS  ~
> 10Gbps ,
>     > > OVS-
>     > > > > DPDK 1.7 Gbps.
>     > > > >
>     > > > > This can be explained as follows:
>     > > > > veth is implemented in kernel - in OVS-DPDK data is
> transferred
>     > > from
>     > > > > veth to user space while in normal OVS we save this
> transfer
>     > > > [Mooney, Sean K] that is part of the reason, the other reson
> this is
>     > > > slow and The main limiter to scalling adding veth pairs or
> ovs
>     > > > internal port to ovs with dpdk is That these linux kernel
> ports are
>     > > > not processed by the dpdk pmds. They are server by the Ovs-
> vswitchd
>     > > > main thread via a fall back to the non dpdk acclarated netdev
>     > > implementation.
>     > > > >
>     > > > > Is there any other device to connect to namespace ?
> something like
>     > > > > vhost-user ? I understand that vhost-user cannot be used
> for
>     > > > > namespace
>     > > > [Mooney, Sean K] I have been doing some experiments in this
> regard.
>     > > > You should be able to use the tap, pcap or afpacket pmd to
> add a
>     > > > vedv that will improve Performance. I have seen some strange
> issue
>     > > > with
>     > > the
>     > > > tap pmd that cause packet to be drop By the kernel on tx on
> some
>     > > ports
>     > > > but not others so there may be issues with that dirver.
>     > > >
>     > > > Previous experiment with libpcap seemed to work well with ovs
> 2.5
>     > > > but I have not tried it With ovs 2.7/master since the
> introduction
>     > > > of generic vdev support at runtime. Previously vdevs And to
> be
>     > > > allocated
>     > > using the dpdk args.
>     > > >
>     > > > I would try following the af_packet example here
>     > > >
>     > >
>     > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__github.com_openvswitch_ovs_blob_b132189d8456f38f3ee139f126d68&d=DwI
> CAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=REyqxTB8Gd9BnEtet
> H_Aul0OgyyGK0DFhKl3tFGzOGI&e=
>     > 0
>     > > > 9 01a9ee9a8/Documentation/howto/dpdk.rst#vdev-support
>     > > >
>     > > [Avi Cohen (A)]
>     > > Thank you Mooney, Sean K
>     > > I already tried to connect the namespace with a tap device (see
> 1 & 2
>     > > below)  - and got the worst performance  for some reason the
> packet
>     > > is cut to default MTU inside the  OVS-DPDK which transmit the
> packet
>     > > to its peer. - although all interfaces MTU were set to 9000.
>     > >
>     > >  1. ovs-vsctl add-port $BRIDGE tap1 -- set Interface tap1
>     > > type=internal
>     > >
>     > >  2. ip link set tap1 netns ns1 // attach it to namespace
>     > [Mooney, Sean K] this is not using the dpdk tap pmd , internal
> port and veth
>     > ports If added to ovs will not be accelerated by dpdk unless you
> use a vdev to
>     > attach them.
>     > >
>     > > I'm looking at your link to create a virtual PMD with vdev
> support - I
>     > > see there a creation of a virtual PMD device , but I'm not sure
> how
>     > > this is connected to the namespace ?  what device should I
> assign to
>     > > the namespace ?
>     > [Mooney, Sean K]
>     > You would use it as follows
>     >
>     > ip tuntap add dev tap1 mode tap
>     >
>     > ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
>     > options:dpdk-devargs=eth_af_packet0,iface=tap1
>     >
>     > ip link set tap1 netns ns1
>     >
>     > ip netns exec ns1 ifconfig 192.168.1.1/24 up
>     >
>     > in general though if you are using ovs-dpdk you should avoid
> using network
>     > namespace and the kernel where possible but the above should
> improve
>     > you performance. One caveat, the amount of vdev+phyical
> interfaces is
>     > limited by how dpdk is compiled by default to 32 devices but it
> can be
>     > increased to 256 if required.
>     >
>     > >
>     > > Best Regards
>     > > avi
>     > >
>     > > > if you happen to be investigating this for use with openstack
>     > > > routers we Are currently working on a way to remove the use
> of
>     > > > namespace entirely for dvr when using The default neutron
> agent and
>     > > > sdn controllers such as ovn already provide that
> functionality.
>     > > > >
>     > > > > Best Regards
>     > > > > avi
>     > > > > _______________________________________________
>     > > > > Dpdk-ovs mailing list
>     > > > > Dpdk-ovs@lists.01.org
>     > > > > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__lists.01.org_mailman_listinfo_dpdk-
> 2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0MKa41bi
> 9F6bBW3Srw-WghpVzyDs-5oSFc&e=
>     > _______________________________________________
>     > Dpdk-ovs mailing list
>     > Dpdk-ovs@lists.01.org
>     > https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__lists.01.org_mailman_listinfo_dpdk-
> 2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0MKa41bi
> 9F6bBW3Srw-WghpVzyDs-5oSFc&e=
>     _______________________________________________
>     discuss mailing list
>     discuss@openvswitch.org
>     https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__mail.openvswitch.org_mailman_listinfo_ovs-
> 2Ddiscuss&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=5dbke1-
> uxXxS5IYomkpMMPVvbXG9uXZthWdONHRQapo&e=
> 

Re: [dpdk-users] [ovs-discuss] OVS-DPDK - Very poor performance when connected to namespace/container

[Darrell Ball]

On 6/16/17, 10:01 AM, "Mooney, Sean K" <sean.k.mooney@intel.com> wrote:

    
    > -----Original Message-----
    > From: Darrell Ball [mailto:dball@vmware.com]
    > Sent: Friday, June 16, 2017 5:53 PM
    > To: Gray, Mark D <mark.d.gray@intel.com>; Mooney, Sean K
    > <sean.k.mooney@intel.com>; Avi Cohen (A) <avi.cohen@huawei.com>;
    > users@dpdk.org; ovs-discuss@openvswitch.org
    > Subject: Re: [ovs-discuss] OVS-DPDK - Very poor performance when
    > connected to namespace/container
    > 
    > This should be quite a bit better than the AF_PACKET PMD.
    > This becomes available in 17.08 and is important, it would be nice to
    > get into the next OVS release.
    [Mooney, Sean K] 
    Virtio user will work only if you have dpdk in the container/namespce correct?
    The vhsot frontend is being provisioned by isided the dpdk vhost driver in the application.

Yes, it requires more effort on the install/config side, but may be worth it in many cases. 

    
    I agree that is the way to go if your application support dpdk, if not you would need
    To use one of the other vdevs.
    
    > 
    > Also, any existing data on the performance advantage of AF_PACKET PMD
    > with single queue ?
    [Mooney, Sean K] I would be interested in seeing this also if anyone has data to share.
    > 
    > Darrell
    > 
    > 
    > On 6/16/17, 1:56 AM, "ovs-discuss-bounces@openvswitch.org on behalf of
    > Gray, Mark D" <ovs-discuss-bounces@openvswitch.org on behalf of
    > mark.d.gray@intel.com> wrote:
    > 
    >     Hi Avi,
    > 
    >     The other option is to use virtio-use
    > (https://urldefense.proofpoint.com/v2/url?u=http-
    > 3A__dpdk.org_doc_guides_howto_virtio-5Fuser-5Ffor-5Fcontainer-
    > 5Fnetworking.html&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
    > uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ThmUVGZC2M1LBAIhn
    > vs-5OiSZz7ywpez2Qj70BrQjoM&e= ) which gives dpdk-like performance to a
    > dpdk application running in a container. The configuration for ovs-dpdk
    > has not been documented but it is possible to use (as a vdev).
    > 
    >     Also, I have dropped the dpdk-ovs@lists.01.org mail address as this
    > is for the discontinued ovdk project.
    > 
    >     Mark
    > 
    >     > -----Original Message-----
    >     > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf
    > Of
    >     > Mooney, Sean K
    >     > Sent: Thursday, June 15, 2017 12:33 PM
    >     > To: Avi Cohen (A) <avi.cohen@huawei.com>; dpdk-ovs@lists.01.org;
    >     > users@dpdk.org; ovs-discuss@openvswitch.org
    >     > Subject: Re: [Dpdk-ovs] OVS-DPDK - Very poor performance when
    >     > connected to namespace/container
    >     >
    >     >
    >     >
    >     > > -----Original Message-----
    >     > > From: Avi Cohen (A) [mailto:avi.cohen@huawei.com]
    >     > > Sent: Thursday, June 15, 2017 9:50 AM
    >     > > To: Mooney, Sean K <sean.k.mooney@intel.com>; dpdk-
    > ovs@lists.01.org;
    >     > > users@dpdk.org; ovs-discuss@openvswitch.org
    >     > > Subject: RE: OVS-DPDK - Very poor performance when connected to
    >     > > namespace/container
    >     > >
    >     > >
    >     > >
    >     > > > -----Original Message-----
    >     > > > From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
    >     > > > Sent: Thursday, 15 June, 2017 11:24 AM
    >     > > > To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org;
    > ovs-
    >     > > > discuss@openvswitch.org
    >     > > > Cc: Mooney, Sean K
    >     > > > Subject: RE: OVS-DPDK - Very poor performance when connected
    > to
    >     > > > namespace/container
    >     > > >
    >     > > >
    >     > > >
    >     > > > > -----Original Message-----
    >     > > > > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On
    > Behalf Of
    >     > > > > Avi Cohen (A)
    >     > > > > Sent: Thursday, June 15, 2017 8:14 AM
    >     > > > > To: dpdk-ovs@lists.01.org; users@dpdk.org;
    >     > > > > ovs-discuss@openvswitch.org
    >     > > > > Subject: [Dpdk-ovs] OVS-DPDK - Very poor performance when
    >     > > > > connected to namespace/container
    >     > > > >
    >     > > > > Hello   All,
    >     > > > > I have OVS-DPDK connected to a namespace via veth pair
    > device.
    >     > > > >
    >     > > > > I've got a very poor performance - compared to normal OVS
    > (i.e. no
    >     > > > > DPDK).
    >     > > > > For example - TCP jumbo pkts throughput: normal OVS  ~
    > 10Gbps ,
    >     > > OVS-
    >     > > > > DPDK 1.7 Gbps.
    >     > > > >
    >     > > > > This can be explained as follows:
    >     > > > > veth is implemented in kernel - in OVS-DPDK data is
    > transferred
    >     > > from
    >     > > > > veth to user space while in normal OVS we save this
    > transfer
    >     > > > [Mooney, Sean K] that is part of the reason, the other reson
    > this is
    >     > > > slow and The main limiter to scalling adding veth pairs or
    > ovs
    >     > > > internal port to ovs with dpdk is That these linux kernel
    > ports are
    >     > > > not processed by the dpdk pmds. They are server by the Ovs-
    > vswitchd
    >     > > > main thread via a fall back to the non dpdk acclarated netdev
    >     > > implementation.
    >     > > > >
    >     > > > > Is there any other device to connect to namespace ?
    > something like
    >     > > > > vhost-user ? I understand that vhost-user cannot be used
    > for
    >     > > > > namespace
    >     > > > [Mooney, Sean K] I have been doing some experiments in this
    > regard.
    >     > > > You should be able to use the tap, pcap or afpacket pmd to
    > add a
    >     > > > vedv that will improve Performance. I have seen some strange
    > issue
    >     > > > with
    >     > > the
    >     > > > tap pmd that cause packet to be drop By the kernel on tx on
    > some
    >     > > ports
    >     > > > but not others so there may be issues with that dirver.
    >     > > >
    >     > > > Previous experiment with libpcap seemed to work well with ovs
    > 2.5
    >     > > > but I have not tried it With ovs 2.7/master since the
    > introduction
    >     > > > of generic vdev support at runtime. Previously vdevs And to
    > be
    >     > > > allocated
    >     > > using the dpdk args.
    >     > > >
    >     > > > I would try following the af_packet example here
    >     > > >
    >     > >
    >     > https://urldefense.proofpoint.com/v2/url?u=https-
    > 3A__github.com_openvswitch_ovs_blob_b132189d8456f38f3ee139f126d68&d=DwI
    > CAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
    > uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=REyqxTB8Gd9BnEtet
    > H_Aul0OgyyGK0DFhKl3tFGzOGI&e=
    >     > 0
    >     > > > 9 01a9ee9a8/Documentation/howto/dpdk.rst#vdev-support
    >     > > >
    >     > > [Avi Cohen (A)]
    >     > > Thank you Mooney, Sean K
    >     > > I already tried to connect the namespace with a tap device (see
    > 1 & 2
    >     > > below)  - and got the worst performance  for some reason the
    > packet
    >     > > is cut to default MTU inside the  OVS-DPDK which transmit the
    > packet
    >     > > to its peer. - although all interfaces MTU were set to 9000.
    >     > >
    >     > >  1. ovs-vsctl add-port $BRIDGE tap1 -- set Interface tap1
    >     > > type=internal
    >     > >
    >     > >  2. ip link set tap1 netns ns1 // attach it to namespace
    >     > [Mooney, Sean K] this is not using the dpdk tap pmd , internal
    > port and veth
    >     > ports If added to ovs will not be accelerated by dpdk unless you
    > use a vdev to
    >     > attach them.
    >     > >
    >     > > I'm looking at your link to create a virtual PMD with vdev
    > support - I
    >     > > see there a creation of a virtual PMD device , but I'm not sure
    > how
    >     > > this is connected to the namespace ?  what device should I
    > assign to
    >     > > the namespace ?
    >     > [Mooney, Sean K]
    >     > You would use it as follows
    >     >
    >     > ip tuntap add dev tap1 mode tap
    >     >
    >     > ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
    >     > options:dpdk-devargs=eth_af_packet0,iface=tap1
    >     >
    >     > ip link set tap1 netns ns1
    >     >
    >     > ip netns exec ns1 ifconfig 192.168.1.1/24 up
    >     >
    >     > in general though if you are using ovs-dpdk you should avoid
    > using network
    >     > namespace and the kernel where possible but the above should
    > improve
    >     > you performance. One caveat, the amount of vdev+phyical
    > interfaces is
    >     > limited by how dpdk is compiled by default to 32 devices but it
    > can be
    >     > increased to 256 if required.
    >     >
    >     > >
    >     > > Best Regards
    >     > > avi
    >     > >
    >     > > > if you happen to be investigating this for use with openstack
    >     > > > routers we Are currently working on a way to remove the use
    > of
    >     > > > namespace entirely for dvr when using The default neutron
    > agent and
    >     > > > sdn controllers such as ovn already provide that
    > functionality.
    >     > > > >
    >     > > > > Best Regards
    >     > > > > avi
    >     > > > > _______________________________________________
    >     > > > > Dpdk-ovs mailing list
    >     > > > > Dpdk-ovs@lists.01.org
    >     > > > > https://urldefense.proofpoint.com/v2/url?u=https-
    > 3A__lists.01.org_mailman_listinfo_dpdk-
    > 2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
    > uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0MKa41bi
    > 9F6bBW3Srw-WghpVzyDs-5oSFc&e=
    >     > _______________________________________________
    >     > Dpdk-ovs mailing list
    >     > Dpdk-ovs@lists.01.org
    >     > https://urldefense.proofpoint.com/v2/url?u=https-
    > 3A__lists.01.org_mailman_listinfo_dpdk-
    > 2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
    > uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0MKa41bi
    > 9F6bBW3Srw-WghpVzyDs-5oSFc&e=
    >     _______________________________________________
    >     discuss mailing list
    >     discuss@openvswitch.org
    >     https://urldefense.proofpoint.com/v2/url?u=https-
    > 3A__mail.openvswitch.org_mailman_listinfo_ovs-
    > 2Ddiscuss&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
    > uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=5dbke1-
    > uxXxS5IYomkpMMPVvbXG9uXZthWdONHRQapo&e=
    > 
    
    

Re: [dpdk-users] [ovs-discuss] OVS-DPDK - Very poor performance when connected to namespace/container

[Avi Cohen (A)]

Thank you all
1. I cannot run dpdk in the container since I have to run there applications require the linux ip-stack
2. I have ovs 2.6.1 with dpdk-stable-16.07.2 and I get an error message [could not open network device tap1 - No such device ] when try to set an af-packet port:

       ip tuntap add dev tap1 mode tap

      ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
      options:dpdk-devargs=eth_af_packet0,iface=tap1

in other doc I saw a bit different command   with 2 if names tap1 and eth0
    ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
    options:dpdk-devargs=eth_af_packet0, iface=eth0

what is the correct command ?
Best Regards
avi

> -----Original Message-----
> From: Darrell Ball [mailto:dball@vmware.com]
> Sent: Friday, 16 June, 2017 8:25 PM
> To: Mooney, Sean K; Gray, Mark D; Avi Cohen (A); users@dpdk.org; ovs-
> discuss@openvswitch.org
> Subject: Re: [ovs-discuss] OVS-DPDK - Very poor performance when connected
> to namespace/container
> 
> 
> 
> On 6/16/17, 10:01 AM, "Mooney, Sean K" <sean.k.mooney@intel.com> wrote:
> 
> 
>     > -----Original Message-----
>     > From: Darrell Ball [mailto:dball@vmware.com]
>     > Sent: Friday, June 16, 2017 5:53 PM
>     > To: Gray, Mark D <mark.d.gray@intel.com>; Mooney, Sean K
>     > <sean.k.mooney@intel.com>; Avi Cohen (A) <avi.cohen@huawei.com>;
>     > users@dpdk.org; ovs-discuss@openvswitch.org
>     > Subject: Re: [ovs-discuss] OVS-DPDK - Very poor performance when
>     > connected to namespace/container
>     >
>     > This should be quite a bit better than the AF_PACKET PMD.
>     > This becomes available in 17.08 and is important, it would be nice to
>     > get into the next OVS release.
>     [Mooney, Sean K]
>     Virtio user will work only if you have dpdk in the container/namespce
> correct?
>     The vhsot frontend is being provisioned by isided the dpdk vhost driver in the
> application.
> 
> Yes, it requires more effort on the install/config side, but may be worth it in
> many cases.
> 
> 
>     I agree that is the way to go if your application support dpdk, if not you
> would need
>     To use one of the other vdevs.
> 
>     >
>     > Also, any existing data on the performance advantage of AF_PACKET PMD
>     > with single queue ?
>     [Mooney, Sean K] I would be interested in seeing this also if anyone has data
> to share.
>     >
>     > Darrell
>     >
>     >
>     > On 6/16/17, 1:56 AM, "ovs-discuss-bounces@openvswitch.org on behalf of
>     > Gray, Mark D" <ovs-discuss-bounces@openvswitch.org on behalf of
>     > mark.d.gray@intel.com> wrote:
>     >
>     >     Hi Avi,
>     >
>     >     The other option is to use virtio-use
>     > (https://urldefense.proofpoint.com/v2/url?u=http-
>     > 3A__dpdk.org_doc_guides_howto_virtio-5Fuser-5Ffor-5Fcontainer-
>     >
> 5Fnetworking.html&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX
> 7JQ5Ih-
>     >
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ThmUVGZC2
> M1LBAIhn
>     > vs-5OiSZz7ywpez2Qj70BrQjoM&e= ) which gives dpdk-like performance to
> a
>     > dpdk application running in a container. The configuration for ovs-dpdk
>     > has not been documented but it is possible to use (as a vdev).
>     >
>     >     Also, I have dropped the dpdk-ovs@lists.01.org mail address as this
>     > is for the discontinued ovdk project.
>     >
>     >     Mark
>     >
>     >     > -----Original Message-----
>     >     > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf
>     > Of
>     >     > Mooney, Sean K
>     >     > Sent: Thursday, June 15, 2017 12:33 PM
>     >     > To: Avi Cohen (A) <avi.cohen@huawei.com>; dpdk-ovs@lists.01.org;
>     >     > users@dpdk.org; ovs-discuss@openvswitch.org
>     >     > Subject: Re: [Dpdk-ovs] OVS-DPDK - Very poor performance when
>     >     > connected to namespace/container
>     >     >
>     >     >
>     >     >
>     >     > > -----Original Message-----
>     >     > > From: Avi Cohen (A) [mailto:avi.cohen@huawei.com]
>     >     > > Sent: Thursday, June 15, 2017 9:50 AM
>     >     > > To: Mooney, Sean K <sean.k.mooney@intel.com>; dpdk-
>     > ovs@lists.01.org;
>     >     > > users@dpdk.org; ovs-discuss@openvswitch.org
>     >     > > Subject: RE: OVS-DPDK - Very poor performance when connected to
>     >     > > namespace/container
>     >     > >
>     >     > >
>     >     > >
>     >     > > > -----Original Message-----
>     >     > > > From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
>     >     > > > Sent: Thursday, 15 June, 2017 11:24 AM
>     >     > > > To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org;
>     > ovs-
>     >     > > > discuss@openvswitch.org
>     >     > > > Cc: Mooney, Sean K
>     >     > > > Subject: RE: OVS-DPDK - Very poor performance when connected
>     > to
>     >     > > > namespace/container
>     >     > > >
>     >     > > >
>     >     > > >
>     >     > > > > -----Original Message-----
>     >     > > > > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On
>     > Behalf Of
>     >     > > > > Avi Cohen (A)
>     >     > > > > Sent: Thursday, June 15, 2017 8:14 AM
>     >     > > > > To: dpdk-ovs@lists.01.org; users@dpdk.org;
>     >     > > > > ovs-discuss@openvswitch.org
>     >     > > > > Subject: [Dpdk-ovs] OVS-DPDK - Very poor performance when
>     >     > > > > connected to namespace/container
>     >     > > > >
>     >     > > > > Hello   All,
>     >     > > > > I have OVS-DPDK connected to a namespace via veth pair
>     > device.
>     >     > > > >
>     >     > > > > I've got a very poor performance - compared to normal OVS
>     > (i.e. no
>     >     > > > > DPDK).
>     >     > > > > For example - TCP jumbo pkts throughput: normal OVS  ~
>     > 10Gbps ,
>     >     > > OVS-
>     >     > > > > DPDK 1.7 Gbps.
>     >     > > > >
>     >     > > > > This can be explained as follows:
>     >     > > > > veth is implemented in kernel - in OVS-DPDK data is
>     > transferred
>     >     > > from
>     >     > > > > veth to user space while in normal OVS we save this
>     > transfer
>     >     > > > [Mooney, Sean K] that is part of the reason, the other reson
>     > this is
>     >     > > > slow and The main limiter to scalling adding veth pairs or
>     > ovs
>     >     > > > internal port to ovs with dpdk is That these linux kernel
>     > ports are
>     >     > > > not processed by the dpdk pmds. They are server by the Ovs-
>     > vswitchd
>     >     > > > main thread via a fall back to the non dpdk acclarated netdev
>     >     > > implementation.
>     >     > > > >
>     >     > > > > Is there any other device to connect to namespace ?
>     > something like
>     >     > > > > vhost-user ? I understand that vhost-user cannot be used
>     > for
>     >     > > > > namespace
>     >     > > > [Mooney, Sean K] I have been doing some experiments in this
>     > regard.
>     >     > > > You should be able to use the tap, pcap or afpacket pmd to
>     > add a
>     >     > > > vedv that will improve Performance. I have seen some strange
>     > issue
>     >     > > > with
>     >     > > the
>     >     > > > tap pmd that cause packet to be drop By the kernel on tx on
>     > some
>     >     > > ports
>     >     > > > but not others so there may be issues with that dirver.
>     >     > > >
>     >     > > > Previous experiment with libpcap seemed to work well with ovs
>     > 2.5
>     >     > > > but I have not tried it With ovs 2.7/master since the
>     > introduction
>     >     > > > of generic vdev support at runtime. Previously vdevs And to
>     > be
>     >     > > > allocated
>     >     > > using the dpdk args.
>     >     > > >
>     >     > > > I would try following the af_packet example here
>     >     > > >
>     >     > >
>     >     > https://urldefense.proofpoint.com/v2/url?u=https-
>     >
> 3A__github.com_openvswitch_ovs_blob_b132189d8456f38f3ee139f126d68&d=
> DwI
>     > CAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
>     >
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=REyqxTB8Gd
> 9BnEtet
>     > H_Aul0OgyyGK0DFhKl3tFGzOGI&e=
>     >     > 0
>     >     > > > 9 01a9ee9a8/Documentation/howto/dpdk.rst#vdev-support
>     >     > > >
>     >     > > [Avi Cohen (A)]
>     >     > > Thank you Mooney, Sean K
>     >     > > I already tried to connect the namespace with a tap device (see
>     > 1 & 2
>     >     > > below)  - and got the worst performance  for some reason the
>     > packet
>     >     > > is cut to default MTU inside the  OVS-DPDK which transmit the
>     > packet
>     >     > > to its peer. - although all interfaces MTU were set to 9000.
>     >     > >
>     >     > >  1. ovs-vsctl add-port $BRIDGE tap1 -- set Interface tap1
>     >     > > type=internal
>     >     > >
>     >     > >  2. ip link set tap1 netns ns1 // attach it to namespace
>     >     > [Mooney, Sean K] this is not using the dpdk tap pmd , internal
>     > port and veth
>     >     > ports If added to ovs will not be accelerated by dpdk unless you
>     > use a vdev to
>     >     > attach them.
>     >     > >
>     >     > > I'm looking at your link to create a virtual PMD with vdev
>     > support - I
>     >     > > see there a creation of a virtual PMD device , but I'm not sure
>     > how
>     >     > > this is connected to the namespace ?  what device should I
>     > assign to
>     >     > > the namespace ?
>     >     > [Mooney, Sean K]
>     >     > You would use it as follows
>     >     >
>     >     > ip tuntap add dev tap1 mode tap
>     >     >
>     >     > ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
>     >     > options:dpdk-devargs=eth_af_packet0,iface=tap1
>     >     >
>     >     > ip link set tap1 netns ns1
>     >     >
>     >     > ip netns exec ns1 ifconfig 192.168.1.1/24 up
>     >     >
>     >     > in general though if you are using ovs-dpdk you should avoid
>     > using network
>     >     > namespace and the kernel where possible but the above should
>     > improve
>     >     > you performance. One caveat, the amount of vdev+phyical
>     > interfaces is
>     >     > limited by how dpdk is compiled by default to 32 devices but it
>     > can be
>     >     > increased to 256 if required.
>     >     >
>     >     > >
>     >     > > Best Regards
>     >     > > avi
>     >     > >
>     >     > > > if you happen to be investigating this for use with openstack
>     >     > > > routers we Are currently working on a way to remove the use
>     > of
>     >     > > > namespace entirely for dvr when using The default neutron
>     > agent and
>     >     > > > sdn controllers such as ovn already provide that
>     > functionality.
>     >     > > > >
>     >     > > > > Best Regards
>     >     > > > > avi
>     >     > > > > _______________________________________________
>     >     > > > > Dpdk-ovs mailing list
>     >     > > > > Dpdk-ovs@lists.01.org
>     >     > > > > https://urldefense.proofpoint.com/v2/url?u=https-
>     > 3A__lists.01.org_mailman_listinfo_dpdk-
>     > 2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
>     >
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0
> MKa41bi
>     > 9F6bBW3Srw-WghpVzyDs-5oSFc&e=
>     >     > _______________________________________________
>     >     > Dpdk-ovs mailing list
>     >     > Dpdk-ovs@lists.01.org
>     >     > https://urldefense.proofpoint.com/v2/url?u=https-
>     > 3A__lists.01.org_mailman_listinfo_dpdk-
>     > 2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
>     >
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0
> MKa41bi
>     > 9F6bBW3Srw-WghpVzyDs-5oSFc&e=
>     >     _______________________________________________
>     >     discuss mailing list
>     >     discuss@openvswitch.org
>     >     https://urldefense.proofpoint.com/v2/url?u=https-
>     > 3A__mail.openvswitch.org_mailman_listinfo_ovs-
>     >
> 2Ddiscuss&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
>     > uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=5dbke1-
>     > uxXxS5IYomkpMMPVvbXG9uXZthWdONHRQapo&e=
>     >
> 
>