From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from BLU004-OMC4S27.hotmail.com (blu004-omc4s27.hotmail.com [65.55.111.166]) by dpdk.org (Postfix) with ESMTP id B8E4FADE4 for ; Wed, 22 Jun 2016 10:57:25 +0200 (CEST) Received: from BLU170-W1 ([65.55.111.135]) by BLU004-OMC4S27.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.23008); Wed, 22 Jun 2016 01:57:25 -0700 X-TMN: [wsMA7he4YDovuxx6iO7RxDZ2yxx+yDRv] X-Originating-Email: [yedurushyam@hotmail.com] Message-ID: From: shyam yeduru To: Sergey Matov CC: "users@dpdk.org" Date: Wed, 22 Jun 2016 14:27:25 +0530 Importance: Normal In-Reply-To: References: , , , Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginalArrivalTime: 22 Jun 2016 08:57:25.0489 (UTC) FILETIME=[18C62610:01D1CC64] Subject: Re: [dpdk-users] OVS Switch with DPDK X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 08:57:26 -0000 yep!! I am testing 4 different scenarios i.e=0A= =0A= 1.SR-IOV with Vyatta vRouter=0A= 2.SR-IOV + DPDK with Vyatta vRouter=0A= 3.OVS with Vyatta vRouter=0A= 4.OVS + DPDK with Vyatta vRouter=0A= =0A= >>From the above 4 =2C we are good with 3 scenarios expect OVS+DPDK . We are = getting errors while adding dpdk0 interface to OVS.=0A= =0A= I have followed these websites to understand and configure it. but no l=0A= insights.ubuntu.com=0A= the-new-simplicity-to-consume-dpdk=0A= =0A= Error mesg:=0A= =0A= 2016-06-22T06:18:28.972Z|00024|bridge|WARN|could not open network =A0device= dpdk0 (No such device)=0A= =0A= Thanks=0A= Shyam Yeduru=0A= =0A= om: smatov@mirantis.com=0A= > Date: Wed=2C 22 Jun 2016 11:11:54 +0300=0A= > Subject: Re: [dpdk-users] OVS Switch with DPDK=0A= > To: yedurushyam@hotmail.com=0A= >=0A= > I can see 82599 Intel chipset here witch is listed in DPDK supported=0A= > NICs http://dpdk.org/doc/nics.=0A= > I am also working mainly with 82599 ixgbe nics.=0A= > Don't forget that you can only bind device either to DPDK driver or to=0A= > kernel.=0A= > However for dual-port NICs it's possible to have 1st port bond to DPDK=0A= > and 2nd port used as PF for several VFs.=0A= >=0A= > For example=2C if you are running OpenStack Mitaka with OVS+DPDK=0A= > deployment it's possible to spawn a VM with both vhostuser and SR-IOV=0A= > ports.=0A= >=0A= > 2016-06-22 9:45 GMT+03:00 shyam yeduru=0A= > >:=0A= > Hi Sergey=0A= >=0A= > No i have not done binding any interfaces. I have used dpdk with SR-IOV= =0A= > VF =2C At that point in time I have binded the VF to dpdk and exposed to= =0A= > VNF.=0A= >=0A= > Which interface you recommend to bind on to dpdk ? Please advise.=0A= >=0A= > root@ovskvm1:~# dpdk_nic_bind -s=0A= >=0A= > Network devices using DPDK-compatible driver=0A= > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= > =0A= >=0A= > Network devices using kernel driver=0A= > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= > 0000:03:00.0 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=3Deno1 drv=3Dtg= 3=0A= > unused=3D=0A= > 0000:03:00.1 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=3Deno2 drv=3Dtg= 3=0A= > unused=3D *Active*=0A= > 0000:03:00.2 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=3Deno3 drv=3Dtg= 3=0A= > unused=3D=0A= > 0000:03:00.3 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=3Deno4 drv=3Dtg= 3=0A= > unused=3D=0A= > 0000:07:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=3Dens2f0= =0A= > drv=3Dixgbe unused=3D=0A= > 0000:07:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=3Dens2f1= =0A= > drv=3Dixgbe unused=3D=0A= > 0000:07:10.0 '82599 Ethernet Controller Virtual Function' if=3Denp7s16=0A= > drv=3Dixgbevf unused=3D=0A= > 0000:07:10.1 '82599 Ethernet Controller Virtual Function' if=3Denp7s16f1= =0A= > drv=3Dixgbevf unused=3D=0A= > 0000:07:10.2 '82599 Ethernet Controller Virtual Function' if=3Denp7s16f2= =0A= > drv=3Dixgbevf unused=3D=0A= > 0000:07:10.3 '82599 Ethernet Controller Virtual Function' if=3Denp7s16f3= =0A= > drv=3Dixgbevf unused=3D=0A= >=0A= > Other network devices=0A= > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= > =0A= > root@ovskvm1:~#=0A= >=0A= > Thanks=0A= > Shyam Yeduru=0A= > ________________________________=0A= >> From: smatov@mirantis.com=0A= >> Date: Wed=2C 22 Jun 2016 09:37:40 +0300=0A= >> Subject: Re: [dpdk-users] OVS Switch with DPDK=0A= >> To: yedurushyam@hotmail.com=0A= >> CC: users@dpdk.org=3B=0A= > sy00454302@techmahindra.com=3B=0A= > yedurushyam@gmail.com=0A= >>=0A= >> Hello.=0A= >>=0A= >> Have you done dpdk-binding before adding DPDK interface to your bridge?= =0A= >> It can be done via (your DPDK dir)/tools/dpdk_nic_bind.py script.=0A= >>=0A= >> 2016-06-22 9:23 GMT+03:00 shyam yeduru=0A= >>=0A= > >>:=0A= >> Hi Team=2C=0A= >>=0A= >> We have a setup at our lab to demonstrate DPDK with OVS and With-Out.=0A= >> Here are my configuration details and error message.=0A= >>=0A= >> Hardware : HP DL 380 G8=0A= >> OS - Ubuntu 16.0.4=0A= >> OVS Package - 2.5=0A= >> DPDK has been installed along with OVS=0A= >>=0A= >> Packages installed : apt-get install openvswitch-switch-dpdk uvtool=0A= > qemu-kvm=0A= >>=0A= >> Procedure is as follows:=0A= >> 1.0 enabled iommu on Kernel=0A= >> 2.0 enalbled hugepages=0A= >> NR_2M_PAGES=3D4096">> /etc/dpdk/dpdk.conf=0A= >> root@ovskvm1:~# mount | column -t | grep huge=0A= >> cgroup on /sys/fs/cgroup/hugetlb=0A= >> type cgroup=0A= >>=0A= > (rw=2Cnosuid=2Cnodev=2Cnoexec=2Crelatime=2Chugetlb=2Crelease_agent=3D/run= /cgmanager/agents/cgm-release-agent.hugetlb=2Cnsroot=3D/)=0A= >> hugetlbfs on /dev/hugepages=0A= >> type hugetlbfs (rw=2Crelatime)=0A= >> hugetlbfs-kvm on /run/hugepages/kvm=0A= >> type hugetlbfs (rw=2Crelatime=2Cmode=3D775=2Cgid=3D111)=0A= >> nodev on /var/opt/huge=0A= >> type hugetlbfs (rw=2Crelatime=2Cpagesize=3D1GB)=0A= >> hugetlb on /run/lxcfs/controllers/hugetlb=0A= >> type cgroup=0A= >> (rw=2Crelatime=2Chugetlb=2Crelease_agent=3D/run/cgmanager/agents/cgm-rel= ease-agent=0A= >> hugetlb=2Cnsroot=3D/)=0A= >>=0A= >> 3.0 OVS conf settings=0A= >>=0A= >> DPDK_OPTS=3D'--dpdk -c 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm= =0A= >> --vhost-perm 0664'=0A= >>=0A= >>=0A= >> 4.0 Connecting dpdk0 interface and two guests. We got issue while=0A= >> adding dpdko interface=0A= >>=0A= >> root@ovskvm1:~# ovs-vsctl show=0A= >> 0ee7d41d-3f51-43ae-9181-bb23aa81ebc7=0A= >> Bridge "ovsdpdkbr0"=0A= >> Port "vhost-user-2"=0A= >> Interface "vhost-user-2"=0A= >> type: dpdkvhostuser=0A= >> Port "vhost-user-1"=0A= >> Interface "vhost-user-1"=0A= >> type: dpdkvhostuser=0A= >> Port "ovsdpdkbr0"=0A= >> Interface "ovsdpdkbr0"=0A= >> type: internal=0A= >> Port "dpdk0"=0A= >> Interface "dpdk0"=0A= >> type: dpdk=0A= >> error: "could not open network device dpdk0 (No such device)"=0A= >> ovs_version: "2.5.0"=0A= >> root@ovskvm1:~#=0A= >>=0A= >> ovs log=0A= >>=0A= >> root@ovskvm1:~# tailf /var/log/openvswitch/ovs-vswitchd.log=0A= >> 2016-06-22T06:10:46.041Z|00015|bridge|INFO|bridge ovsdpdkbr0: added=0A= >> interface ovsdpdkbr0 on port 65534=0A= >> 2016-06-22T06:10:46.041Z|00016|bridge|INFO|bridge ovsdpdkbr0: using=0A= >> datapath ID 00001e82be6ada4d=0A= >> 2016-06-22T06:10:46.041Z|00017|connmgr|INFO|ovsdpdkbr0: added service=0A= >> controller "punix:/var/run/openvswitch/ovsdpdkbr0.mgmt"=0A= >> 2016-06-22T06:10:46.079Z|00018|bridge|INFO|ovs-vswitchd (Open=0A= > vSwitch) 2.5.0=0A= >> 2016-06-22T06:10:53.089Z|00019|memory|INFO|183016 kB peak resident set= =0A= >> size after 10.2 seconds=0A= >> 2016-06-22T06:10:53.089Z|00020|memory|INFO|handlers:11 ports:1=0A= >> revalidators:5 rules:5=0A= >> 2016-06-22T06:18:16.397Z|00021|bridge|INFO|bridge ovsdpdkbr1: added=0A= >> interface ovsdpdkbr1 on port 65534=0A= >> 2016-06-22T06:18:16.397Z|00022|bridge|INFO|bridge ovsdpdkbr1: using=0A= >> datapath ID 00002e9f58d5d440=0A= >> 2016-06-22T06:18:16.397Z|00023|connmgr|INFO|ovsdpdkbr1: added service=0A= >> controller "punix:/var/run/openvswitch/ovsdpdkbr1.mgmt"=0A= >> 2016-06-22T06:18:28.972Z|00024|bridge|WARN|could not open network=0A= >> device dpdk0 (No such device)=0A= >>=0A= >> OVS-DPDK Process:=0A= >>=0A= >> root@ovskvm1:~# ps -ef|grep ovs|grep dpdk=0A= >> root 3568 1 0 12:31 ? 00:00:00 ovs-vswit... --dpdk -c=0A= >> 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664 --=0A= >> unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err=0A= >> -vfile:info --mlockall --no-chdir=0A= >> --log-file=3D/var/log/openvswitch/ovs-vswitchd.log=0A= >> --pidfile=3D/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor=0A= >> root 3569 3568 99 12:31 ? 00:00:07 ovs-vswitchd --dpdk -c=0A= >> 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664 --=0A= >> unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err=0A= >> -vfile:info --mlockall --no-chdir=0A= >> --log-file=3D/var/log/openvswitch/ovs-vswitchd.log=0A= >> --pidfile=3D/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor=0A= >>=0A= >> Can some one guide us =2C Where are we failing and how to over come=0A= > this issue.=0A= >>=0A= >> Thanks=0A= >> Shyam Yeduru=0A= >>=0A= >>=0A= >>=0A= >>=0A= >>=0A= >>=0A= >>=0A= >> --=0A= >> Best Regards=0A= >> Sergey Matov=0A= >> Mirantis Inc=0A= >=0A= >=0A= >=0A= >=0A= > --=0A= > Best Regards=0A= > Sergey Matov=0A= > Mirantis Inc=0A= =