Re: [dpdk-users] [ovs-discuss] OVS-DPDK - Very poor performance when connected to namespace/container

["Avi Cohen (A)" <avi.cohen@huawei.com>]

Thank you all
1. I cannot run dpdk in the container since I have to run there applications require the linux ip-stack
2. I have ovs 2.6.1 with dpdk-stable-16.07.2 and I get an error message [could not open network device tap1 - No such device ] when try to set an af-packet port:

       ip tuntap add dev tap1 mode tap

      ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
      options:dpdk-devargs=eth_af_packet0,iface=tap1

in other doc I saw a bit different command   with 2 if names tap1 and eth0
    ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
    options:dpdk-devargs=eth_af_packet0, iface=eth0

what is the correct command ?
Best Regards
avi

> -----Original Message-----
> From: Darrell Ball [mailto:dball@vmware.com]
> Sent: Friday, 16 June, 2017 8:25 PM
> To: Mooney, Sean K; Gray, Mark D; Avi Cohen (A); users@dpdk.org; ovs-
> discuss@openvswitch.org
> Subject: Re: [ovs-discuss] OVS-DPDK - Very poor performance when connected
> to namespace/container
> 
> 
> 
> On 6/16/17, 10:01 AM, "Mooney, Sean K" <sean.k.mooney@intel.com> wrote:
> 
> 
>     > -----Original Message-----
>     > From: Darrell Ball [mailto:dball@vmware.com]
>     > Sent: Friday, June 16, 2017 5:53 PM
>     > To: Gray, Mark D <mark.d.gray@intel.com>; Mooney, Sean K
>     > <sean.k.mooney@intel.com>; Avi Cohen (A) <avi.cohen@huawei.com>;
>     > users@dpdk.org; ovs-discuss@openvswitch.org
>     > Subject: Re: [ovs-discuss] OVS-DPDK - Very poor performance when
>     > connected to namespace/container
>     >
>     > This should be quite a bit better than the AF_PACKET PMD.
>     > This becomes available in 17.08 and is important, it would be nice to
>     > get into the next OVS release.
>     [Mooney, Sean K]
>     Virtio user will work only if you have dpdk in the container/namespce
> correct?
>     The vhsot frontend is being provisioned by isided the dpdk vhost driver in the
> application.
> 
> Yes, it requires more effort on the install/config side, but may be worth it in
> many cases.
> 
> 
>     I agree that is the way to go if your application support dpdk, if not you
> would need
>     To use one of the other vdevs.
> 
>     >
>     > Also, any existing data on the performance advantage of AF_PACKET PMD
>     > with single queue ?
>     [Mooney, Sean K] I would be interested in seeing this also if anyone has data
> to share.
>     >
>     > Darrell
>     >
>     >
>     > On 6/16/17, 1:56 AM, "ovs-discuss-bounces@openvswitch.org on behalf of
>     > Gray, Mark D" <ovs-discuss-bounces@openvswitch.org on behalf of
>     > mark.d.gray@intel.com> wrote:
>     >
>     >     Hi Avi,
>     >
>     >     The other option is to use virtio-use
>     > (https://urldefense.proofpoint.com/v2/url?u=http-
>     > 3A__dpdk.org_doc_guides_howto_virtio-5Fuser-5Ffor-5Fcontainer-
>     >
> 5Fnetworking.html&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX
> 7JQ5Ih-
>     >
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ThmUVGZC2
> M1LBAIhn
>     > vs-5OiSZz7ywpez2Qj70BrQjoM&e= ) which gives dpdk-like performance to
> a
>     > dpdk application running in a container. The configuration for ovs-dpdk
>     > has not been documented but it is possible to use (as a vdev).
>     >
>     >     Also, I have dropped the dpdk-ovs@lists.01.org mail address as this
>     > is for the discontinued ovdk project.
>     >
>     >     Mark
>     >
>     >     > -----Original Message-----
>     >     > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On Behalf
>     > Of
>     >     > Mooney, Sean K
>     >     > Sent: Thursday, June 15, 2017 12:33 PM
>     >     > To: Avi Cohen (A) <avi.cohen@huawei.com>; dpdk-ovs@lists.01.org;
>     >     > users@dpdk.org; ovs-discuss@openvswitch.org
>     >     > Subject: Re: [Dpdk-ovs] OVS-DPDK - Very poor performance when
>     >     > connected to namespace/container
>     >     >
>     >     >
>     >     >
>     >     > > -----Original Message-----
>     >     > > From: Avi Cohen (A) [mailto:avi.cohen@huawei.com]
>     >     > > Sent: Thursday, June 15, 2017 9:50 AM
>     >     > > To: Mooney, Sean K <sean.k.mooney@intel.com>; dpdk-
>     > ovs@lists.01.org;
>     >     > > users@dpdk.org; ovs-discuss@openvswitch.org
>     >     > > Subject: RE: OVS-DPDK - Very poor performance when connected to
>     >     > > namespace/container
>     >     > >
>     >     > >
>     >     > >
>     >     > > > -----Original Message-----
>     >     > > > From: Mooney, Sean K [mailto:sean.k.mooney@intel.com]
>     >     > > > Sent: Thursday, 15 June, 2017 11:24 AM
>     >     > > > To: Avi Cohen (A); dpdk-ovs@lists.01.org; users@dpdk.org;
>     > ovs-
>     >     > > > discuss@openvswitch.org
>     >     > > > Cc: Mooney, Sean K
>     >     > > > Subject: RE: OVS-DPDK - Very poor performance when connected
>     > to
>     >     > > > namespace/container
>     >     > > >
>     >     > > >
>     >     > > >
>     >     > > > > -----Original Message-----
>     >     > > > > From: Dpdk-ovs [mailto:dpdk-ovs-bounces@lists.01.org] On
>     > Behalf Of
>     >     > > > > Avi Cohen (A)
>     >     > > > > Sent: Thursday, June 15, 2017 8:14 AM
>     >     > > > > To: dpdk-ovs@lists.01.org; users@dpdk.org;
>     >     > > > > ovs-discuss@openvswitch.org
>     >     > > > > Subject: [Dpdk-ovs] OVS-DPDK - Very poor performance when
>     >     > > > > connected to namespace/container
>     >     > > > >
>     >     > > > > Hello   All,
>     >     > > > > I have OVS-DPDK connected to a namespace via veth pair
>     > device.
>     >     > > > >
>     >     > > > > I've got a very poor performance - compared to normal OVS
>     > (i.e. no
>     >     > > > > DPDK).
>     >     > > > > For example - TCP jumbo pkts throughput: normal OVS  ~
>     > 10Gbps ,
>     >     > > OVS-
>     >     > > > > DPDK 1.7 Gbps.
>     >     > > > >
>     >     > > > > This can be explained as follows:
>     >     > > > > veth is implemented in kernel - in OVS-DPDK data is
>     > transferred
>     >     > > from
>     >     > > > > veth to user space while in normal OVS we save this
>     > transfer
>     >     > > > [Mooney, Sean K] that is part of the reason, the other reson
>     > this is
>     >     > > > slow and The main limiter to scalling adding veth pairs or
>     > ovs
>     >     > > > internal port to ovs with dpdk is That these linux kernel
>     > ports are
>     >     > > > not processed by the dpdk pmds. They are server by the Ovs-
>     > vswitchd
>     >     > > > main thread via a fall back to the non dpdk acclarated netdev
>     >     > > implementation.
>     >     > > > >
>     >     > > > > Is there any other device to connect to namespace ?
>     > something like
>     >     > > > > vhost-user ? I understand that vhost-user cannot be used
>     > for
>     >     > > > > namespace
>     >     > > > [Mooney, Sean K] I have been doing some experiments in this
>     > regard.
>     >     > > > You should be able to use the tap, pcap or afpacket pmd to
>     > add a
>     >     > > > vedv that will improve Performance. I have seen some strange
>     > issue
>     >     > > > with
>     >     > > the
>     >     > > > tap pmd that cause packet to be drop By the kernel on tx on
>     > some
>     >     > > ports
>     >     > > > but not others so there may be issues with that dirver.
>     >     > > >
>     >     > > > Previous experiment with libpcap seemed to work well with ovs
>     > 2.5
>     >     > > > but I have not tried it With ovs 2.7/master since the
>     > introduction
>     >     > > > of generic vdev support at runtime. Previously vdevs And to
>     > be
>     >     > > > allocated
>     >     > > using the dpdk args.
>     >     > > >
>     >     > > > I would try following the af_packet example here
>     >     > > >
>     >     > >
>     >     > https://urldefense.proofpoint.com/v2/url?u=https-
>     >
> 3A__github.com_openvswitch_ovs_blob_b132189d8456f38f3ee139f126d68&d=
> DwI
>     > CAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
>     >
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=REyqxTB8Gd
> 9BnEtet
>     > H_Aul0OgyyGK0DFhKl3tFGzOGI&e=
>     >     > 0
>     >     > > > 9 01a9ee9a8/Documentation/howto/dpdk.rst#vdev-support
>     >     > > >
>     >     > > [Avi Cohen (A)]
>     >     > > Thank you Mooney, Sean K
>     >     > > I already tried to connect the namespace with a tap device (see
>     > 1 & 2
>     >     > > below)  - and got the worst performance  for some reason the
>     > packet
>     >     > > is cut to default MTU inside the  OVS-DPDK which transmit the
>     > packet
>     >     > > to its peer. - although all interfaces MTU were set to 9000.
>     >     > >
>     >     > >  1. ovs-vsctl add-port $BRIDGE tap1 -- set Interface tap1
>     >     > > type=internal
>     >     > >
>     >     > >  2. ip link set tap1 netns ns1 // attach it to namespace
>     >     > [Mooney, Sean K] this is not using the dpdk tap pmd , internal
>     > port and veth
>     >     > ports If added to ovs will not be accelerated by dpdk unless you
>     > use a vdev to
>     >     > attach them.
>     >     > >
>     >     > > I'm looking at your link to create a virtual PMD with vdev
>     > support - I
>     >     > > see there a creation of a virtual PMD device , but I'm not sure
>     > how
>     >     > > this is connected to the namespace ?  what device should I
>     > assign to
>     >     > > the namespace ?
>     >     > [Mooney, Sean K]
>     >     > You would use it as follows
>     >     >
>     >     > ip tuntap add dev tap1 mode tap
>     >     >
>     >     > ovs-vsctl add-port br0 tap1 -- set Interface tap1 type=dpdk \
>     >     > options:dpdk-devargs=eth_af_packet0,iface=tap1
>     >     >
>     >     > ip link set tap1 netns ns1
>     >     >
>     >     > ip netns exec ns1 ifconfig 192.168.1.1/24 up
>     >     >
>     >     > in general though if you are using ovs-dpdk you should avoid
>     > using network
>     >     > namespace and the kernel where possible but the above should
>     > improve
>     >     > you performance. One caveat, the amount of vdev+phyical
>     > interfaces is
>     >     > limited by how dpdk is compiled by default to 32 devices but it
>     > can be
>     >     > increased to 256 if required.
>     >     >
>     >     > >
>     >     > > Best Regards
>     >     > > avi
>     >     > >
>     >     > > > if you happen to be investigating this for use with openstack
>     >     > > > routers we Are currently working on a way to remove the use
>     > of
>     >     > > > namespace entirely for dvr when using The default neutron
>     > agent and
>     >     > > > sdn controllers such as ovn already provide that
>     > functionality.
>     >     > > > >
>     >     > > > > Best Regards
>     >     > > > > avi
>     >     > > > > _______________________________________________
>     >     > > > > Dpdk-ovs mailing list
>     >     > > > > Dpdk-ovs@lists.01.org
>     >     > > > > https://urldefense.proofpoint.com/v2/url?u=https-
>     > 3A__lists.01.org_mailman_listinfo_dpdk-
>     > 2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
>     >
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0
> MKa41bi
>     > 9F6bBW3Srw-WghpVzyDs-5oSFc&e=
>     >     > _______________________________________________
>     >     > Dpdk-ovs mailing list
>     >     > Dpdk-ovs@lists.01.org
>     >     > https://urldefense.proofpoint.com/v2/url?u=https-
>     > 3A__lists.01.org_mailman_listinfo_dpdk-
>     > 2Dovs&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
>     >
> uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=ACWRjXvbu0
> MKa41bi
>     > 9F6bBW3Srw-WghpVzyDs-5oSFc&e=
>     >     _______________________________________________
>     >     discuss mailing list
>     >     discuss@openvswitch.org
>     >     https://urldefense.proofpoint.com/v2/url?u=https-
>     > 3A__mail.openvswitch.org_mailman_listinfo_ovs-
>     >
> 2Ddiscuss&d=DwICAg&c=uilaK90D4TOVoH58JNXRgQ&r=BVhFA09CGX7JQ5Ih-
>     > uZnsw&m=_7l_cdKorhow4zRqAu5lnTmzv9Etgn5TX7D6P0pqP8c&s=5dbke1-
>     > uxXxS5IYomkpMMPVvbXG9uXZthWdONHRQapo&e=
>     >
> 
>