* [dpdk-users] OVS Switch with DPDK
@ 2016-06-22 6:23 shyam yeduru
2016-06-22 6:37 ` Sergey Matov
0 siblings, 1 reply; 3+ messages in thread
From: shyam yeduru @ 2016-06-22 6:23 UTC (permalink / raw)
To: users; +Cc: sy00454302, yeduru shyam
Hi Team,
We have a setup at our lab to demonstrate DPDK with OVS and With-Out. Here are my configuration details and error message.
Hardware : HP DL 380 G8
OS - Ubuntu 16.0.4
OVS Package - 2.5
DPDK has been installed along with OVS
Packages installed : apt-get install openvswitch-switch-dpdk uvtool qemu-kvm
Procedure is as follows:
1.0 enabled iommu on Kernel
2.0 enalbled hugepages
NR_2M_PAGES=4096">> /etc/dpdk/dpdk.conf
root@ovskvm1:~# mount | column -t | grep huge
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime)
hugetlbfs-kvm on /run/hugepages/kvm type hugetlbfs (rw,relatime,mode=775,gid=111)
nodev on /var/opt/huge type hugetlbfs (rw,relatime,pagesize=1GB)
hugetlb on /run/lxcfs/controllers/hugetlb type cgroup (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent hugetlb,nsroot=/)
3.0 OVS conf settings
DPDK_OPTS='--dpdk -c 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664'
4.0 Connecting dpdk0 interface and two guests. We got issue while adding dpdko interface
root@ovskvm1:~# ovs-vsctl show
0ee7d41d-3f51-43ae-9181-bb23aa81ebc7
Bridge "ovsdpdkbr0"
Port "vhost-user-2"
Interface "vhost-user-2"
type: dpdkvhostuser
Port "vhost-user-1"
Interface "vhost-user-1"
type: dpdkvhostuser
Port "ovsdpdkbr0"
Interface "ovsdpdkbr0"
type: internal
Port "dpdk0"
Interface "dpdk0"
type: dpdk
error: "could not open network device dpdk0 (No such device)"
ovs_version: "2.5.0"
root@ovskvm1:~#
ovs log
root@ovskvm1:~# tailf /var/log/openvswitch/ovs-vswitchd.log
2016-06-22T06:10:46.041Z|00015|bridge|INFO|bridge ovsdpdkbr0: added interface ovsdpdkbr0 on port 65534
2016-06-22T06:10:46.041Z|00016|bridge|INFO|bridge ovsdpdkbr0: using datapath ID 00001e82be6ada4d
2016-06-22T06:10:46.041Z|00017|connmgr|INFO|ovsdpdkbr0: added service controller "punix:/var/run/openvswitch/ovsdpdkbr0.mgmt"
2016-06-22T06:10:46.079Z|00018|bridge|INFO|ovs-vswitchd (Open vSwitch) 2.5.0
2016-06-22T06:10:53.089Z|00019|memory|INFO|183016 kB peak resident set size after 10.2 seconds
2016-06-22T06:10:53.089Z|00020|memory|INFO|handlers:11 ports:1 revalidators:5 rules:5
2016-06-22T06:18:16.397Z|00021|bridge|INFO|bridge ovsdpdkbr1: added interface ovsdpdkbr1 on port 65534
2016-06-22T06:18:16.397Z|00022|bridge|INFO|bridge ovsdpdkbr1: using datapath ID 00002e9f58d5d440
2016-06-22T06:18:16.397Z|00023|connmgr|INFO|ovsdpdkbr1: added service controller "punix:/var/run/openvswitch/ovsdpdkbr1.mgmt"
2016-06-22T06:18:28.972Z|00024|bridge|WARN|could not open network device dpdk0 (No such device)
OVS-DPDK Process:
root@ovskvm1:~# ps -ef|grep ovs|grep dpdk
root 3568 1 0 12:31 ? 00:00:00 ovs-vswit... --dpdk -c 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
root 3569 3568 99 12:31 ? 00:00:07 ovs-vswitchd --dpdk -c 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
Can some one guide us , Where are we failing and how to over come this issue.
Thanks
Shyam Yeduru
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-users] OVS Switch with DPDK
2016-06-22 6:23 [dpdk-users] OVS Switch with DPDK shyam yeduru
@ 2016-06-22 6:37 ` Sergey Matov
[not found] ` <BLU170-W123F6B04531DE26D9640B3FAB2C0@phx.gbl>
0 siblings, 1 reply; 3+ messages in thread
From: Sergey Matov @ 2016-06-22 6:37 UTC (permalink / raw)
To: shyam yeduru; +Cc: users, sy00454302, yeduru shyam
Hello.
Have you done dpdk-binding before adding DPDK interface to your bridge? It
can be done via (your DPDK dir)/tools/dpdk_nic_bind.py script.
2016-06-22 9:23 GMT+03:00 shyam yeduru <yedurushyam@hotmail.com>:
> Hi Team,
>
> We have a setup at our lab to demonstrate DPDK with OVS and With-Out. Here
> are my configuration details and error message.
>
> Hardware : HP DL 380 G8
> OS - Ubuntu 16.0.4
> OVS Package - 2.5
> DPDK has been installed along with OVS
>
> Packages installed : apt-get install openvswitch-switch-dpdk uvtool
> qemu-kvm
>
> Procedure is as follows:
> 1.0 enabled iommu on Kernel
> 2.0 enalbled hugepages
> NR_2M_PAGES=4096">> /etc/dpdk/dpdk.conf
> root@ovskvm1:~# mount | column -t | grep huge
> cgroup on /sys/fs/cgroup/hugetlb
> type cgroup
> (rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
> hugetlbfs on /dev/hugepages
> type hugetlbfs (rw,relatime)
> hugetlbfs-kvm on /run/hugepages/kvm
> type hugetlbfs (rw,relatime,mode=775,gid=111)
> nodev on /var/opt/huge
> type hugetlbfs (rw,relatime,pagesize=1GB)
> hugetlb on /run/lxcfs/controllers/hugetlb
> type cgroup
> (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent
> hugetlb,nsroot=/)
>
> 3.0 OVS conf settings
>
> DPDK_OPTS='--dpdk -c 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm
> --vhost-perm 0664'
>
>
> 4.0 Connecting dpdk0 interface and two guests. We got issue while adding
> dpdko interface
>
> root@ovskvm1:~# ovs-vsctl show
> 0ee7d41d-3f51-43ae-9181-bb23aa81ebc7
> Bridge "ovsdpdkbr0"
> Port "vhost-user-2"
> Interface "vhost-user-2"
> type: dpdkvhostuser
> Port "vhost-user-1"
> Interface "vhost-user-1"
> type: dpdkvhostuser
> Port "ovsdpdkbr0"
> Interface "ovsdpdkbr0"
> type: internal
> Port "dpdk0"
> Interface "dpdk0"
> type: dpdk
> error: "could not open network device dpdk0 (No such
> device)"
> ovs_version: "2.5.0"
> root@ovskvm1:~#
>
> ovs log
>
> root@ovskvm1:~# tailf /var/log/openvswitch/ovs-vswitchd.log
> 2016-06-22T06:10:46.041Z|00015|bridge|INFO|bridge ovsdpdkbr0: added
> interface ovsdpdkbr0 on port 65534
> 2016-06-22T06:10:46.041Z|00016|bridge|INFO|bridge ovsdpdkbr0: using
> datapath ID 00001e82be6ada4d
> 2016-06-22T06:10:46.041Z|00017|connmgr|INFO|ovsdpdkbr0: added service
> controller "punix:/var/run/openvswitch/ovsdpdkbr0.mgmt"
> 2016-06-22T06:10:46.079Z|00018|bridge|INFO|ovs-vswitchd (Open vSwitch)
> 2.5.0
> 2016-06-22T06:10:53.089Z|00019|memory|INFO|183016 kB peak resident set
> size after 10.2 seconds
> 2016-06-22T06:10:53.089Z|00020|memory|INFO|handlers:11 ports:1
> revalidators:5 rules:5
> 2016-06-22T06:18:16.397Z|00021|bridge|INFO|bridge ovsdpdkbr1: added
> interface ovsdpdkbr1 on port 65534
> 2016-06-22T06:18:16.397Z|00022|bridge|INFO|bridge ovsdpdkbr1: using
> datapath ID 00002e9f58d5d440
> 2016-06-22T06:18:16.397Z|00023|connmgr|INFO|ovsdpdkbr1: added service
> controller "punix:/var/run/openvswitch/ovsdpdkbr1.mgmt"
> 2016-06-22T06:18:28.972Z|00024|bridge|WARN|could not open network device
> dpdk0 (No such device)
>
> OVS-DPDK Process:
>
> root@ovskvm1:~# ps -ef|grep ovs|grep dpdk
> root 3568 1 0 12:31 ? 00:00:00 ovs-vswit... --dpdk -c 0x1
> -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664 --
> unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info
> --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log
> --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
> root 3569 3568 99 12:31 ? 00:00:07 ovs-vswitchd --dpdk -c 0x1
> -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664 --
> unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info
> --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log
> --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
>
> Can some one guide us , Where are we failing and how to over come this
> issue.
>
> Thanks
> Shyam Yeduru
>
>
>
>
--
*Best Regards*
*Sergey Matov*
*Mirantis Inc*
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [dpdk-users] OVS Switch with DPDK
[not found] ` <CAPykWKEa4DT42ftXdsOmp94G2zcewEUbdTKYo8PgJUbLPKmFgA@mail.gmail.com>
@ 2016-06-22 8:57 ` shyam yeduru
0 siblings, 0 replies; 3+ messages in thread
From: shyam yeduru @ 2016-06-22 8:57 UTC (permalink / raw)
To: Sergey Matov; +Cc: users
yep!! I am testing 4 different scenarios i.e
1.SR-IOV with Vyatta vRouter
2.SR-IOV + DPDK with Vyatta vRouter
3.OVS with Vyatta vRouter
4.OVS + DPDK with Vyatta vRouter
>From the above 4 , we are good with 3 scenarios expect OVS+DPDK . We are getting errors while adding dpdk0 interface to OVS.
I have followed these websites to understand and configure it. but no l
insights.ubuntu.com
the-new-simplicity-to-consume-dpdk
Error mesg:
2016-06-22T06:18:28.972Z|00024|bridge|WARN|could not open network device dpdk0 (No such device)
Thanks
Shyam Yeduru
om: smatov@mirantis.com
> Date: Wed, 22 Jun 2016 11:11:54 +0300
> Subject: Re: [dpdk-users] OVS Switch with DPDK
> To: yedurushyam@hotmail.com
>
> I can see 82599 Intel chipset here witch is listed in DPDK supported
> NICs http://dpdk.org/doc/nics.
> I am also working mainly with 82599 ixgbe nics.
> Don't forget that you can only bind device either to DPDK driver or to
> kernel.
> However for dual-port NICs it's possible to have 1st port bond to DPDK
> and 2nd port used as PF for several VFs.
>
> For example, if you are running OpenStack Mitaka with OVS+DPDK
> deployment it's possible to spawn a VM with both vhostuser and SR-IOV
> ports.
>
> 2016-06-22 9:45 GMT+03:00 shyam yeduru
> <yedurushyam@hotmail.com<mailto:yedurushyam@hotmail.com>>:
> Hi Sergey
>
> No i have not done binding any interfaces. I have used dpdk with SR-IOV
> VF , At that point in time I have binded the VF to dpdk and exposed to
> VNF.
>
> Which interface you recommend to bind on to dpdk ? Please advise.
>
> root@ovskvm1:~# dpdk_nic_bind -s
>
> Network devices using DPDK-compatible driver
> ============================================
> <none>
>
> Network devices using kernel driver
> ===================================
> 0000:03:00.0 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=eno1 drv=tg3
> unused=
> 0000:03:00.1 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=eno2 drv=tg3
> unused= *Active*
> 0000:03:00.2 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=eno3 drv=tg3
> unused=
> 0000:03:00.3 'NetXtreme BCM5719 Gigabit Ethernet PCIe' if=eno4 drv=tg3
> unused=
> 0000:07:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=ens2f0
> drv=ixgbe unused=
> 0000:07:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' if=ens2f1
> drv=ixgbe unused=
> 0000:07:10.0 '82599 Ethernet Controller Virtual Function' if=enp7s16
> drv=ixgbevf unused=
> 0000:07:10.1 '82599 Ethernet Controller Virtual Function' if=enp7s16f1
> drv=ixgbevf unused=
> 0000:07:10.2 '82599 Ethernet Controller Virtual Function' if=enp7s16f2
> drv=ixgbevf unused=
> 0000:07:10.3 '82599 Ethernet Controller Virtual Function' if=enp7s16f3
> drv=ixgbevf unused=
>
> Other network devices
> =====================
> <none>
> root@ovskvm1:~#
>
> Thanks
> Shyam Yeduru
> ________________________________
>> From: smatov@mirantis.com<mailto:smatov@mirantis.com>
>> Date: Wed, 22 Jun 2016 09:37:40 +0300
>> Subject: Re: [dpdk-users] OVS Switch with DPDK
>> To: yedurushyam@hotmail.com<mailto:yedurushyam@hotmail.com>
>> CC: users@dpdk.org<mailto:users@dpdk.org>;
> sy00454302@techmahindra.com<mailto:sy00454302@techmahindra.com>;
> yedurushyam@gmail.com<mailto:yedurushyam@gmail.com>
>>
>> Hello.
>>
>> Have you done dpdk-binding before adding DPDK interface to your bridge?
>> It can be done via (your DPDK dir)/tools/dpdk_nic_bind.py script.
>>
>> 2016-06-22 9:23 GMT+03:00 shyam yeduru
>>
> <yedurushyam@hotmail.com<mailto:yedurushyam@hotmail.com><mailto:yedurushyam@hotmail.com<mailto:yedurushyam@hotmail.com>>>:
>> Hi Team,
>>
>> We have a setup at our lab to demonstrate DPDK with OVS and With-Out.
>> Here are my configuration details and error message.
>>
>> Hardware : HP DL 380 G8
>> OS - Ubuntu 16.0.4
>> OVS Package - 2.5
>> DPDK has been installed along with OVS
>>
>> Packages installed : apt-get install openvswitch-switch-dpdk uvtool
> qemu-kvm
>>
>> Procedure is as follows:
>> 1.0 enabled iommu on Kernel
>> 2.0 enalbled hugepages
>> NR_2M_PAGES=4096">> /etc/dpdk/dpdk.conf
>> root@ovskvm1:~# mount | column -t | grep huge
>> cgroup on /sys/fs/cgroup/hugetlb
>> type cgroup
>>
> (rw,nosuid,nodev,noexec,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb,nsroot=/)
>> hugetlbfs on /dev/hugepages
>> type hugetlbfs (rw,relatime)
>> hugetlbfs-kvm on /run/hugepages/kvm
>> type hugetlbfs (rw,relatime,mode=775,gid=111)
>> nodev on /var/opt/huge
>> type hugetlbfs (rw,relatime,pagesize=1GB)
>> hugetlb on /run/lxcfs/controllers/hugetlb
>> type cgroup
>> (rw,relatime,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent
>> hugetlb,nsroot=/)
>>
>> 3.0 OVS conf settings
>>
>> DPDK_OPTS='--dpdk -c 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm
>> --vhost-perm 0664'
>>
>>
>> 4.0 Connecting dpdk0 interface and two guests. We got issue while
>> adding dpdko interface
>>
>> root@ovskvm1:~# ovs-vsctl show
>> 0ee7d41d-3f51-43ae-9181-bb23aa81ebc7
>> Bridge "ovsdpdkbr0"
>> Port "vhost-user-2"
>> Interface "vhost-user-2"
>> type: dpdkvhostuser
>> Port "vhost-user-1"
>> Interface "vhost-user-1"
>> type: dpdkvhostuser
>> Port "ovsdpdkbr0"
>> Interface "ovsdpdkbr0"
>> type: internal
>> Port "dpdk0"
>> Interface "dpdk0"
>> type: dpdk
>> error: "could not open network device dpdk0 (No such device)"
>> ovs_version: "2.5.0"
>> root@ovskvm1:~#
>>
>> ovs log
>>
>> root@ovskvm1:~# tailf /var/log/openvswitch/ovs-vswitchd.log
>> 2016-06-22T06:10:46.041Z|00015|bridge|INFO|bridge ovsdpdkbr0: added
>> interface ovsdpdkbr0 on port 65534
>> 2016-06-22T06:10:46.041Z|00016|bridge|INFO|bridge ovsdpdkbr0: using
>> datapath ID 00001e82be6ada4d
>> 2016-06-22T06:10:46.041Z|00017|connmgr|INFO|ovsdpdkbr0: added service
>> controller "punix:/var/run/openvswitch/ovsdpdkbr0.mgmt"
>> 2016-06-22T06:10:46.079Z|00018|bridge|INFO|ovs-vswitchd (Open
> vSwitch) 2.5.0
>> 2016-06-22T06:10:53.089Z|00019|memory|INFO|183016 kB peak resident set
>> size after 10.2 seconds
>> 2016-06-22T06:10:53.089Z|00020|memory|INFO|handlers:11 ports:1
>> revalidators:5 rules:5
>> 2016-06-22T06:18:16.397Z|00021|bridge|INFO|bridge ovsdpdkbr1: added
>> interface ovsdpdkbr1 on port 65534
>> 2016-06-22T06:18:16.397Z|00022|bridge|INFO|bridge ovsdpdkbr1: using
>> datapath ID 00002e9f58d5d440
>> 2016-06-22T06:18:16.397Z|00023|connmgr|INFO|ovsdpdkbr1: added service
>> controller "punix:/var/run/openvswitch/ovsdpdkbr1.mgmt"
>> 2016-06-22T06:18:28.972Z|00024|bridge|WARN|could not open network
>> device dpdk0 (No such device)
>>
>> OVS-DPDK Process:
>>
>> root@ovskvm1:~# ps -ef|grep ovs|grep dpdk
>> root 3568 1 0 12:31 ? 00:00:00 ovs-vswit... --dpdk -c
>> 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664 --
>> unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err
>> -vfile:info --mlockall --no-chdir
>> --log-file=/var/log/openvswitch/ovs-vswitchd.log
>> --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
>> root 3569 3568 99 12:31 ? 00:00:07 ovs-vswitchd --dpdk -c
>> 0x1 -n 4 -m 2048 --vhost-owner libvirt-qemu:kvm --vhost-perm 0664 --
>> unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err
>> -vfile:info --mlockall --no-chdir
>> --log-file=/var/log/openvswitch/ovs-vswitchd.log
>> --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
>>
>> Can some one guide us , Where are we failing and how to over come
> this issue.
>>
>> Thanks
>> Shyam Yeduru
>>
>>
>>
>>
>>
>>
>>
>> --
>> Best Regards
>> Sergey Matov
>> Mirantis Inc
>
>
>
>
> --
> Best Regards
> Sergey Matov
> Mirantis Inc
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-06-22 8:57 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-06-22 6:23 [dpdk-users] OVS Switch with DPDK shyam yeduru
2016-06-22 6:37 ` Sergey Matov
[not found] ` <BLU170-W123F6B04531DE26D9640B3FAB2C0@phx.gbl>
[not found] ` <CAPykWKEa4DT42ftXdsOmp94G2zcewEUbdTKYo8PgJUbLPKmFgA@mail.gmail.com>
2016-06-22 8:57 ` shyam yeduru
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).