Re: [dpdk-dev] [ovs-discuss] OVS DPDK|DPDK|ERR|interface dpdk0 start error:operation not permitted.

Re: [dpdk-dev] [ovs-discuss] OVS DPDK|DPDK|ERR|interface dpdk0 start error:operation not permitted.

[sothy shan]

On Tue, Jul 26, 2016 at 6:32 PM, Mauricio Vasquez
<mauricio.vasquez@polito.it> wrote:
> Hello Sothy,
>
> Sorry for the delay in answering, some busy days here.
>
>
> On 07/25/2016 12:03 PM, sothy shan wrote:
>>
>> Hello Mauricio,
>>
>> On Sun, Jul 24, 2016 at 12:17 PM, Mauricio Vasquez
>> <mauricio.vasquez@polito.it> wrote:
>>>
>>> Hi Sothy,
>>>
>>>
>>> On 07/21/2016 11:55 AM, sothy shan wrote:
>>>>
>>>> On Thu, Jul 21, 2016 at 11:43 AM, Mauricio Vasquez
>>>> <mauricio.vasquez@polito.it> wrote:
>>>>>
>>>>>
>>>>> On 07/21/2016 10:51 AM, sothy shan wrote:
>>>>>>
>>>>>> On Thu, Jul 21, 2016 at 10:41 AM, sothy shan <sothy.e98@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> On Wed, Jul 20, 2016 at 9:02 PM, Mauricio Vasquez
>>>>>>> <mauricio.vasquez@polito.it> wrote:
>>>>>>>>
>>>>>>>> Hi Sothy,
>>>>>>>>
>>>>>>>> On 07/20/2016 07:40 PM, Aaron Conole wrote:
>>>>>>>>>
>>>>>>>>> sothy shan <sothy.e98@gmail.com> writes:
>>>>>>>>>
>>>>>>>>>> Hello all,
>>>>>>>>>
>>>>>>>>> Hi Sothy,
>>>>>>>>>
>>>>>>>>>> I followed the installion guide provided in the web site
>>>>>>>>>>
>>>>>>>>>> https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md#ovstc
>>>>>>>>>> to install dpdk OVS.
>>>>>>>>>>
>>>>>>>>>> I am rurnning Ubuntu 14.04 (kernel 3.13). It is a VM running
>>>>>>>>>> VMware.
>>>>>>>>>> One intereface is enabled by VMXNET3.
>>>>>>>>
>>>>>>>>
>>>>>>>> It is not clear for me, are you trying to install OvS within the VM?
>>>>>>>>
>>>>>>>>>> I used dpdk 16.04
>>>>>>>>>> Today I made clone of OVS (committ
>>>>>>>>>> :bf32e3e2c029da18b5d6fd9210cb0ea12a1d0383).
>>>>>>>>>>
>>>>>>>>>> The following are different configuration compared to above link.
>>>>>>>>>>
>>>>>>>>>> sysctl -w vm.nr_hugepages=2048
>>>>>>>>>> Binded interface via igb_uio.
>>>>>>>>>>
>>>>>>>>>> Then followed same configuration to start ovs-vsswitch with dpdk.
>>>>>>>>>
>>>>>>>>> Can you please post the steps exactly that you followed?  Which
>>>>>>>>> user
>>>>>>>>> you
>>>>>>>>> were running as, etc?  Can you also post how you expect packets to
>>>>>>>>> arrive into the vm?
>>>>>>>>>
>>>>>>>>>> I am able to creatre bridge. When I create a port. It gives error
>>>>>>>>>> message.
>>>>>>>>>
>>>>>>>>> Please show the steps for this, too.  The exact commands you used.
>>>>>>>>>
>>>>>>>>>> When I check ovs-vswitchd.log file,
>>>>>>>>>>
>>>>>>>>>> There is warning that dpif_netlink|WARN|genric netlink family,
>>>>>>>>>> ovs_datapath does not exist. The openvswitch kernel module may be
>>>>>>>>>> not
>>>>>>>>>> loaded.
>>>>>>>>>
>>>>>>>>> This is definitely a problem.  How did you start the ovs-vswitchd
>>>>>>>>> daemon?
>>>>>>>>
>>>>>>>>
>>>>>>>> This is not a problem Aaron, the ovs kernel module is not needed
>>>>>>>> when
>>>>>>>> ovs is
>>>>>>>> compiled with DPDK.
>>>>>>>>>>
>>>>>>>>>> Then ERR: ERR|interface dpdk0 start error:operation not permitted.
>>>>>>>>>>
>>>>>>>>>> The corresponding port is loaded by dpdk driver as shown in
>>>>>>>>>> dpdk_nic_bind --status.
>>>>>>>>>>
>>>>>>>>>> After that I did sudo ovs-vsctl show. it show br and port details
>>>>>>>>>> as
>>>>>>>>>> well.
>>>>>>>>>> I am able to add one flow rule as well. But ovs bridge didnt get
>>>>>>>>>> packets from the interface.
>>>>>>>>>>
>>>>>>>>>> Do you know why this error is popup? anyway to solve the problem?
>>>>>>>>>>
>>>>>>>>>> Thank you very much
>>>>>>>>>>
>>>>>>>>>> Best regars
>>>>>>>>>> Sothy
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> discuss mailing list
>>>>>>>>> discuss@openvswitch.org
>>>>>>>>> http://openvswitch.org/mailman/listinfo/discuss
>>>>>>>>
>>>>>>>> As Aaron wrote, having the exact commands is important to help you.
>>>>>>>
>>>>>>> My configuration and log info:
>>>>>>>
>>>>>>> sudo sysctl vm.nr_hugepages=2048
>>>>>>> sudo mkdir -p /dev/hugepages
>>>>>>> sudo  mount -t hugetlbfs hugetlbfs /dev/hugepages
>>>>>>> sudo modprobe uio
>>>>>>> sudo insmod $DPDK_BUILD/kmod/igb_uio.ko
>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py --status
>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py -b igb_uio 0000:0b:00.0
>>>>>>> sudo ./dpdk-16.04/tools/dpdk_nic_bind.py --status
>>>>>>>
>>>>>>>
>>>>>>> Network devices using DPDK-compatible driver
>>>>>>> ============================================
>>>>>>> 0000:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=
>>>>>>> 0000:13:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=
>>>>>>>
>>>>>>> Network devices using kernel driver
>>>>>>> ===================================
>>>>>>> 0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3
>>>>>>> unused=igb_uio *Active*
>>>>>>>
>>>>>>> Other network devices
>>>>>>> =====================
>>>>>>> <none>
>>>>>>>
>>>>>>> .........Then......................
>>>>>>>
>>>>>>>
>>>>>>> sudo ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
>>>>>>> sudo ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
>>>>>
>>>>> These two commands are inverted.
>>>>> The bridge has to be created first and then ports have to be added to
>>>>> it.
>>>>>
>>>> It is my mistake when coping.
>>>>>>>
>>>>>>> Log file/
>>>>>>> 2016-07-21T07:53:35.527Z|00001|vlog|INFO|opened log file
>>>>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>>>>> 2016-07-21T07:53:35.529Z|00002|ovs_numa|INFO|Discovered 2 CPU cores
>>>>>>> on
>>>>>>> NUMA node 0
>>>>>>> 2016-07-21T07:53:35.529Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes
>>>>>>> and 2 CPU cores
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2016-07-21T07:53:35.529Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>>>> connecting...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2016-07-21T07:53:35.529Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>>>> connected
>>>>>>> 2016-07-21T07:53:35.533Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>>>>>> 2016-07-21T07:53:35.533Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>>>>>> defaulting to /usr/local/var/run/openvswitch
>>>>>>> 2016-07-21T07:53:35.533Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>>>>> --socket-mem 1024,0 -c 0x00000001
>>>>>>> 2016-07-21T07:53:36.606Z|00009|timeval|WARN|Unreasonably long 1076ms
>>>>>>> poll interval (80ms user, 875ms system)
>>>>>>> 2016-07-21T07:53:36.606Z|00010|timeval|WARN|faults: 3306 minor, 0
>>>>>>> major
>>>>>>> 2016-07-21T07:53:36.606Z|00011|timeval|WARN|context switches: 3
>>>>>>> voluntary, 101 involuntary
>>>>>>> 2016-07-21T07:53:36.606Z|00012|coverage|INFO|Event coverage, avg rate
>>>>>>> over last: 5 seconds, last minute, last hour,  hash=398361de:
>>>>>>> 2016-07-21T07:53:36.606Z|00013|coverage|INFO|bridge_reconfigure
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>> 2016-07-21T07:53:36.606Z|00014|coverage|INFO|cmap_expand
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 9
>>>>>>> 2016-07-21T07:53:36.606Z|00015|coverage|INFO|miniflow_malloc
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 9
>>>>>>> 2016-07-21T07:53:36.606Z|00016|coverage|INFO|hmap_pathological
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>> 2016-07-21T07:53:36.606Z|00017|coverage|INFO|hmap_expand
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 640
>>>>>>> 2016-07-21T07:53:36.606Z|00018|coverage|INFO|txn_unchanged
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>>>> 2016-07-21T07:53:36.606Z|00019|coverage|INFO|txn_incomplete
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>> 2016-07-21T07:53:36.607Z|00020|coverage|INFO|poll_create_node
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 35
>>>>>>> 2016-07-21T07:53:36.607Z|00021|coverage|INFO|seq_change
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 38
>>>>>>> 2016-07-21T07:53:36.607Z|00022|coverage|INFO|pstream_open
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>> 2016-07-21T07:53:36.607Z|00023|coverage|INFO|stream_open
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>> 2016-07-21T07:53:36.607Z|00024|coverage|INFO|util_xalloc
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 11049
>>>>>>> 2016-07-21T07:53:36.607Z|00025|coverage|INFO|netdev_get_hwaddr
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>> 2016-07-21T07:53:36.607Z|00026|coverage|INFO|netlink_received
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>>>>> 2016-07-21T07:53:36.607Z|00027|coverage|INFO|netlink_sent
>>>>>>>     0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>> 2016-07-21T07:53:36.607Z|00028|coverage|INFO|86 events never hit
>>>>>>> 2016-07-21T07:53:36.607Z|00029|bridge|INFO|ovs-vswitchd (Open
>>>>>>> vSwitch)
>>>>>>> 2.5.90
>>>>>>> 2016-07-21T07:54:50.861Z|00030|memory|INFO|18980 kB peak resident set
>>>>>>> size after 75.3 seconds
>>>>>>> 2016-07-21T07:54:50.866Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> Datapath supports recirculation
>>>>>>> 2016-07-21T07:54:50.866Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> MPLS label stack length probed as 3
>>>>>>> 2016-07-21T07:54:50.866Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> Datapath supports truncate action
>>>>>>> 2016-07-21T07:54:50.866Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> Datapath supports unique flow ids
>>>>>>> 2016-07-21T07:54:50.866Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> Datapath does not support ct_state
>>>>>>> 2016-07-21T07:54:50.866Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> Datapath does not support ct_zone
>>>>>>> 2016-07-21T07:54:50.866Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> Datapath does not support ct_mark
>>>>>>> 2016-07-21T07:54:50.866Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> Datapath does not support ct_label
>>>>>>> 2016-07-21T07:54:50.866Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>> Datapath does not support ct_state_nat
>>>>>>> 2016-07-21T07:54:50.875Z|00040|bridge|INFO|bridge br0: added
>>>>>>> interface
>>>>>>> br0 on port 65534
>>>>>>> 2016-07-21T07:54:50.887Z|00041|dpif_netlink|WARN|Generic Netlink
>>>>>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module
>>>>>>> is probably not loaded.
>>>>>>> 2016-07-21T07:54:50.887Z|00042|bridge|INFO|bridge br0: using datapath
>>>>>>> ID 0000c664fbd86348
>>>>>>> 2016-07-21T07:54:50.887Z|00043|connmgr|INFO|br0: added service
>>>>>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>>>>>> 2016-07-21T07:58:18.796Z|00044|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>>>>> 2016-07-21T07:58:18.810Z|00045|dpdk|ERR|Interface dpdk0 start error:
>>>>>>> Operation not permitted
>>>>>>> 2016-07-21T07:58:18.810Z|00046|dpif|WARN|netdev@ovs-netdev: failed to
>>>>>>> add dpdk0 as port: Operation not permitted
>>>>>>> 2016-07-21T08:10:35.440Z|00047|connmgr|INFO|br0<->unix: 1 flow_mods
>>>>>>> in
>>>>>>> the last 0 s (1 adds)
>>>>>
>>>>> Could you also provide the full terminal output when you run
>>>>> ovs-vswitchd?,
>>>>> I would like to see the DPDK output
>>>>
>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vswitchd unix:$DB_SOCK
>>>> --pidfile --detach --log-file=/var/log/openvswitch/ovs-vswitchd.log
>>>> 2016-07-21T09:33:35Z|00001|vlog|INFO|opened log file
>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>> 2016-07-21T09:33:35Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA
>>>> node 0
>>>> 2016-07-21T09:33:35Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2
>>>> CPU
>>>> cores
>>>>
>>>>
>>>> 2016-07-21T09:33:35Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>> connecting...
>>>>
>>>>
>>>> 2016-07-21T09:33:35Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>> connected
>>>> 2016-07-21T09:33:35Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>>> 2016-07-21T09:33:35Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>>> defaulting to /usr/local/var/run/openvswitch
>>>> 2016-07-21T09:33:35Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>> --socket-mem 1024,0 -c 0x00000001
>>>> EAL: Detected lcore 0 as core 0 on socket 0
>>>> EAL: Detected lcore 1 as core 0 on socket 0
>>>> EAL: Support maximum 128 logical core(s) by configuration.
>>>> EAL: Detected 2 lcore(s)
>>>> EAL: Probing VFIO support...
>>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>>>> directory)
>>>> EAL: VFIO modules not loaded, skipping VFIO support...
>>>> EAL: Setting up physically contiguous memory...
>>>> EAL: Ask a virtual area of 0x5e400000 bytes
>>>> EAL: Virtual area found at 0x7fa618400000 (size = 0x5e400000)
>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>> EAL: Virtual area found at 0x7fa618000000 (size = 0x200000)
>>>> EAL: Ask a virtual area of 0xa1800000 bytes
>>>> EAL: Virtual area found at 0x7fa576600000 (size = 0xa1800000)
>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>> EAL: Virtual area found at 0x7fa576200000 (size = 0x200000)
>>>> EAL: Requesting 512 pages of size 2MB from socket 0
>>>> EAL: TSC frequency is ~2799213 KHz
>>>> EAL: Master lcore 0 is ready (tid=78ad7b80;cpuset=[0])
>>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1
>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>> EAL:   Not managed by a supported kernel driver, skipped
>>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>> EAL:   PCI memory mapped at 0x7fa658400000
>>>> EAL:   PCI memory mapped at 0x7fa658401000
>>>> EAL:   PCI memory mapped at 0x7fa658402000
>>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1
>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>> EAL:   PCI memory mapped at 0x7fa658404000
>>>> EAL:   PCI memory mapped at 0x7fa658405000
>>>> EAL:   PCI memory mapped at 0x7fa658406000
>>>> Zone 0: name:<RG_MP_log_history>, phys:0x82bfdec0, len:0x2080,
>>>> virt:0x7fa6583fdec0, socket_id:0, flags:0
>>>> Zone 1: name:<MP_log_history>, phys:0x82973d40, len:0x28a0c0,
>>>> virt:0x7fa658173d40, socket_id:0, flags:0
>>>> Zone 2: name:<rte_eth_dev_data>, phys:0x82943200, len:0x2f800,
>>>> virt:0x7fa658143200, socket_id:0, flags:0
>>>> 2016-07-21T09:33:36Z|00009|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports recirculation
>>>> 2016-07-21T09:33:36Z|00010|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS
>>>> label stack length probed as 3
>>>> 2016-07-21T09:33:36Z|00011|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports truncate action
>>>> 2016-07-21T09:33:36Z|00012|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath supports unique flow ids
>>>> 2016-07-21T09:33:36Z|00013|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath does not support ct_state
>>>> 2016-07-21T09:33:36Z|00014|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath does not support ct_zone
>>>> 2016-07-21T09:33:36Z|00015|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath does not support ct_mark
>>>> 2016-07-21T09:33:36Z|00016|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath does not support ct_label
>>>> 2016-07-21T09:33:36Z|00017|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>> Datapath does not support ct_state_nat
>>>> 2016-07-21T09:33:36Z|00018|bridge|INFO|bridge br0: added interface br0
>>>> on port 65534
>>>> 2016-07-21T09:33:36Z|00019|dpif_netlink|WARN|Generic Netlink family
>>>> 'ovs_datapath' does not exist. The Open vSwitch kernel module is
>>>> probably not loaded.
>>>> 2016-07-21T09:33:36Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>> 2016-07-21T09:33:36Z|00021|dpdk|ERR|Interface dpdk0 start error:
>>>> Operation not permitted
>>>> 2016-07-21T09:33:36Z|00022|bridge|INFO|bridge br0: using datapath ID
>>>> 0000c664fbd86348
>>>> 2016-07-21T09:33:36Z|00023|connmgr|INFO|br0: added service controller
>>>> "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>>> 2016-07-21T09:33:36Z|00024|timeval|WARN|Unreasonably long 1244ms poll
>>>> interval (233ms user, 880ms system)
>>>> 2016-07-21T09:33:36Z|00025|timeval|WARN|faults: 3902 minor, 0 major
>>>> 2016-07-21T09:33:36Z|00026|timeval|WARN|context switches: 10
>>>> voluntary, 214 involuntary
>>>> 2016-07-21T09:33:36Z|00027|coverage|INFO|Event coverage, avg rate over
>>>> last: 5 seconds, last minute, last hour,  hash=75cfd2fc:
>>>> 2016-07-21T09:33:36Z|00028|coverage|INFO|bridge_reconfigure
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00029|coverage|INFO|ofproto_flush
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00030|coverage|INFO|ofproto_update_port
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00031|coverage|INFO|rev_flow_table
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00032|coverage|INFO|cmap_expand
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 22
>>>> 2016-07-21T09:33:36Z|00033|coverage|INFO|cmap_shrink
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 7
>>>> 2016-07-21T09:33:36Z|00034|coverage|INFO|dpif_port_add
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>> 2016-07-21T09:33:36Z|00035|coverage|INFO|dpif_flow_flush
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>> 2016-07-21T09:33:36Z|00036|coverage|INFO|dpif_flow_get
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>> 2016-07-21T09:33:36Z|00037|coverage|INFO|dpif_flow_put
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 10
>>>> 2016-07-21T09:33:36Z|00038|coverage|INFO|dpif_flow_del
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>> 2016-07-21T09:33:36Z|00039|coverage|INFO|dpif_execute
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>> 2016-07-21T09:33:36Z|00040|coverage|INFO|flow_extract
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 4
>>>> 2016-07-21T09:33:36Z|00041|coverage|INFO|miniflow_malloc
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 16
>>>> 2016-07-21T09:33:36Z|00042|coverage|INFO|hmap_pathological
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00043|coverage|INFO|hmap_expand
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 653
>>>> 2016-07-21T09:33:36Z|00044|coverage|INFO|netdev_get_stats
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00045|coverage|INFO|poll_create_node
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 15
>>>> 2016-07-21T09:33:36Z|00046|coverage|INFO|poll_zero_timeout
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00047|coverage|INFO|seq_change
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 575
>>>> 2016-07-21T09:33:36Z|00048|coverage|INFO|pstream_open
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>> 2016-07-21T09:33:36Z|00049|coverage|INFO|stream_open
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00050|coverage|INFO|util_xalloc
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 11820
>>>> 2016-07-21T09:33:36Z|00051|coverage|INFO|netdev_set_policing
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00052|coverage|INFO|netdev_get_ifindex
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00053|coverage|INFO|netdev_get_hwaddr
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>> 2016-07-21T09:33:36Z|00054|coverage|INFO|netdev_set_hwaddr
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00055|coverage|INFO|netdev_get_ethtool
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>> 2016-07-21T09:33:36Z|00056|coverage|INFO|netlink_received
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 6
>>>> 2016-07-21T09:33:36Z|00057|coverage|INFO|netlink_recv_jumbo
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>> 2016-07-21T09:33:36Z|00058|coverage|INFO|netlink_sent
>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>> 2016-07-21T09:33:36Z|00059|coverage|INFO|70 events never hit
>>>> 2016-07-21T09:33:36Z|00060|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>> 2016-07-21T09:33:36Z|00061|dpdk|ERR|Interface dpdk0 start error:
>>>> Operation not permitted
>>>
>>> The PMD start function is failing, unfortunately that error does not tell
>>> that much information.
>>>
>>> I would suggest you to enable the debug of the vmxnet3 PMD in DPDK, set
>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y in the config/common_base file of
>>> DPDK, then recompile DPDK and OvS.
>>>
>>> Please try once again and send the terminal output, I hope we can get a
>>> more
>>> information of what is going on with that log.
>>
>> Based on your request, I reproduce the log with
>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y
>> sudo ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock
>> --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile --detach
>> 2016-07-25T09:24:17Z|00001|vlog|INFO|opened log file
>> /var/log/openvswitch/ovs-vswitchd.log
>> 2016-07-25T09:24:17Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA
>> node 0
>> 2016-07-25T09:24:17Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU
>> cores
>>
>> 2016-07-25T09:24:17Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>> connecting...
>>
>> 2016-07-25T09:24:17Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>> connected
>> 2016-07-25T09:24:17Z|00006|dpdk|INFO|DPDK Enabled, initializing
>> 2016-07-25T09:24:17Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>> defaulting to /usr/local/var/run/openvswitch
>> 2016-07-25T09:24:17Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>> --socket-mem 1024,0 -c 0x00000001
>> EAL: Detected lcore 0 as core 0 on socket 0
>> EAL: Detected lcore 1 as core 0 on socket 0
>> EAL: Support maximum 128 logical core(s) by configuration.
>> EAL: Detected 2 lcore(s)
>> EAL: Probing VFIO support...
>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>> directory)
>> EAL: VFIO modules not loaded, skipping VFIO support...
>> EAL: Setting up physically contiguous memory...
>> EAL: Ask a virtual area of 0x200000 bytes
>> EAL: Virtual area found at 0x7fdea6000000 (size = 0x200000)
>> EAL: Ask a virtual area of 0x5e000000 bytes
>> EAL: Virtual area found at 0x7fde47e00000 (size = 0x5e000000)
>> EAL: Ask a virtual area of 0x200000 bytes
>> EAL: Virtual area found at 0x7fde47a00000 (size = 0x200000)
>> EAL: Ask a virtual area of 0x200000 bytes
>> EAL: Virtual area found at 0x7fde47600000 (size = 0x200000)
>> EAL: Ask a virtual area of 0xa1800000 bytes
>> EAL: Virtual area found at 0x7fdda5c00000 (size = 0xa1800000)
>> EAL: Ask a virtual area of 0x200000 bytes
>> EAL: Virtual area found at 0x7fdda5800000 (size = 0x200000)
>> EAL: Requesting 512 pages of size 2MB from socket 0
>> EAL: TSC frequency is ~2799209 KHz
>> EAL: Master lcore 0 is ready (tid=a855eb80;cpuset=[0])
>> PMD: rte_vmxnet3_pmd_init():  >>
>> EAL: PCI device 0000:03:00.0 on NUMA socket -1
>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>> EAL:   Not managed by a supported kernel driver, skipped
>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>> EAL:   PCI memory mapped at 0x7fdea6200000
>> EAL:   PCI memory mapped at 0x7fdea6201000
>> EAL:   PCI memory mapped at 0x7fdea6202000
>> PMD: eth_vmxnet3_dev_init():  >>
>> PMD: eth_vmxnet3_dev_init(): Hardware version : 1
>> PMD: eth_vmxnet3_dev_init(): UPT hardware version : 1
>> PMD: eth_vmxnet3_dev_init(): MAC Address : 00:0c:29:b5:9f:5b
>> EAL: PCI device 0000:13:00.0 on NUMA socket -1
>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>> EAL:   Not managed by a supported kernel driver, skipped
>> Zone 0: name:<RG_MP_log_history>, phys:0x555fdec0, len:0x2080,
>> virt:0x7fdea61fdec0, socket_id:0, flags:0
>> Zone 1: name:<MP_log_history>, phys:0x95375f00, len:0x28a0c0,
>> virt:0x7fde87975f00, socket_id:0, flags:0
>> Zone 2: name:<rte_eth_dev_data>, phys:0x555cd300, len:0x2f800,
>> virt:0x7fdea61cd300, socket_id:0, flags:0
>>
>> PMD: vmxnet3_dev_configure():  >>
>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>> PMD: vmxnet3_dev_rx_queue_setup():  >>
>> PMD: vmxnet3_dev_start():  >>
>> PMD: vmxnet3_rss_configure():  >>
>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b
>> PMD: vmxnet3_disable_intr():  >>
>> PMD: vmxnet3_dev_rxtx_init():  >>
>> 2016-07-25T09:40:23Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>> PMD: vmxnet3_dev_stop():  >>
>> PMD: vmxnet3_disable_intr():  >>
>> PMD: vmxnet3_dev_stop(): Device reset.
>> PMD: vmxnet3_dev_clear_queues():  >>
>> PMD: vmxnet3_dev_configure():  >>
>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>> PMD: vmxnet3_dev_rx_queue_setup():  >>
>> PMD: vmxnet3_dev_start():  >>
>> PMD: vmxnet3_rss_configure():  >>
>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b
>> PMD: vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
>
>
> Unfortunately it is the most I can help you, it is strange for me why
> test-pmd is working and why OvS  is not.
> I think it is a DPDK issue, then the next step is to post on the DPDK
> mailing (dev@dpdk.org, add also in CC Yong Wang <yongwang@vmware.com> that
> is the vmxnet3 maintainer), send a link to this thread in order to be clear
> what you have tested.


Hello Yong Wang,

I am running fedora VM in ESXi 5.5.0 environment. I tried to install
OVS and DPDK. OVS couldnt start DPDK device.
You can see the error in the mail below. Do you know why? IF you need
more information. I can provide more information.

Good news, when I tried in Fedora 23 VM in ESXi 5.5.0, it works. It is
possible to bind an dpdk device and get packets on port.

Thanks Mauricio for your help and reply.

Best regards
Sothy

>
>> I deleted exisiting bridge and recreated::::::
>>
>>
>> Then I added bridge. After I try to create bridge. It gives error. The
>> erros is same as shown in log file.
>>
>> 2016-07-25T09:29:43.248Z|00020|bridge|INFO|bridge br0: added interface
>> br0 on port 65534
>> 2016-07-25T09:29:43.253Z|00021|dpif_netlink|WARN|Generic Netlink
>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module
>> is probably not loaded.
>> 2016-07-25T09:29:43.253Z|00022|bridge|INFO|bridge br0: using datapath
>> ID 0000e6aac740584b
>> 2016-07-25T09:29:43.253Z|00023|connmgr|INFO|br0: added service
>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>> 2016-07-25T09:30:54.008Z|00024|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>> 2016-07-25T09:30:54.021Z|00025|dpdk|ERR|Interface dpdk0 start error:
>> Operation not permitted
>> 2016-07-25T09:30:54.021Z|00026|dpif|WARN|netdev@ovs-netdev: failed to
>> add dpdk0 as port: Operation not permitted
>>
>>
>>
>>> Another thing to do would be to bind the NICs to the uio_pci_generic and
>>> VFIO modules, just to be sure that the issue is also present there.
>>
>> I tried in uio and uio_pci_genric also. Both are giving same error.
>> I suspect I cannt activate VFIO in VM since it is BIOS suport also. AM I
>> right?
>>
>> Thanks
>>
>>>
>>>> in command line some information for your verification:
>>>>
>>>>    sudo ovs-vsctl show
>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2
>>>>       Bridge "br0"
>>>>           Port "dpdk0"
>>>>               Interface "dpdk0"
>>>>                   type: dpdk
>>>>           Port "br0"
>>>>               Interface "br0"
>>>>                   type: internal
>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl del-br br0
>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-br br0 --
>>>> set bridge br0 datapath_type=netdev
>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-port br0
>>>> dpdk0 -- set Interface dpdk0 type=dpdk
>>>> ovs-vsctl: Error detected while setting up 'dpdk0'.  See ovs-vswitchd
>>>> log for details.
>>>>
>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>> sudo ovs-vsctl show
>>>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2
>>>>>>>        Bridge "br0"
>>>>>>>            Port "dpdk0"
>>>>>>>                Interface "dpdk0"
>>>>>>>                    type: dpdk
>>>>>>>            Port "br0"
>>>>>>>                Interface "br0"
>>>>>>>                    type: internal
>>>>>>>
>>>>>>> sudo ovs-ofctl dump-flows br0
>>>>>>> NXST_FLOW reply (xid=0x4):
>>>>>>>     cookie=0x0, duration=487.638s, table=0, n_packets=0, n_bytes=0,
>>>>>>> idle_age=487, priority=0 actions=NORMAL
>>>>>>>
>>>>>>>
>>>>>>>> In addition I would recommend to use the test-pmd [1] application
>>>>>>>> from
>>>>>>>> DPDK
>>>>>>>> to check if the issue is also present in that case.
>>>>>>
>>>>>> I run testpmd command.The result is
>>>>>>
>>>>>> Configuring Port 0 (socket 0)
>>>>>> Port 0: 00:0C:29:B5:9F:5B
>>>>>> Configuring Port 1 (socket 0)
>>>>>> Port 1: 00:0C:29:B5:9F:65
>>>>>> Checking link statuses...
>>>>>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>>>>>> Port 1 Link Up - speed 10000 Mbps - full-duplex
>>>>>> Done
>>>>>> No commandline core given, start packet forwarding
>>>>>>      io packet forwarding - CRC stripping disabled - packets/burst=32
>>>>>>      nb forwarding cores=1 - nb forwarding ports=2
>>>>>>      RX queues=1 - RX desc=128 - RX free threshold=0
>>>>>>      RX threshold registers: pthresh=0 hthresh=0 wthresh=0
>>>>>>      TX queues=1 - TX desc=512 - TX free threshold=0
>>>>>>      TX threshold registers: pthresh=0 hthresh=0 wthresh=0
>>>>>>      TX RS bit threshold=0 - TXQ flags=0x200
>>>>>> Press enter to exit
>>>>>>
>>>>>> Telling cores to stop...
>>>>>> Waiting for lcores to finish...
>>>>>>
>>>>>>      ---------------------- Forward statistics for port 0
>>>>>> ----------------------
>>>>>>      RX-packets: 88             RX-dropped: 0             RX-total: 88
>>>>>>      TX-packets: 2              TX-dropped: 0             TX-total: 2
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----------------------------------------------------------------------------
>>>>>>
>>>>>>      ---------------------- Forward statistics for port 1
>>>>>> ----------------------
>>>>>>      RX-packets: 2              RX-dropped: 0             RX-total: 2
>>>>>>      TX-packets: 88             TX-dropped: 0             TX-total: 88
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----------------------------------------------------------------------------
>>>>>>
>>>>>>      +++++++++++++++ Accumulated forward statistics for all
>>>>>> ports+++++++++++++++
>>>>>>      RX-packets: 90             RX-dropped: 0             RX-total: 90
>>>>>>      TX-packets: 90             TX-dropped: 0             TX-total: 90
>>>>>>
>>>>>>
>>>>>>
>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>
>>>>>> make sense?
>>>>>
>>>>>
>>>>> Yes, it makes sense. Ports are being recognized by DPDK and some
>>>>> packets
>>>>> were exchanged.
>>>>>
>>>>>
>>>>>>>> [1] http://dpdk.org/doc/guides-16.04/testpmd_app_ug/index.html
>>>>>>>>
>>>>>>>> Mauricio Vasquez,
>>>>>
>>>>>
>>> Mauricio V,
>
>

Re: [dpdk-dev] [ovs-discuss] OVS DPDK|DPDK|ERR|interface dpdk0 start error:operation not permitted.

[Yong Wang]

> On Jul 27, 2016, at 12:25 AM, sothy shan <sothy.e98@gmail.com> wrote:
> 
> On Tue, Jul 26, 2016 at 6:32 PM, Mauricio Vasquez
> <mauricio.vasquez@polito.it> wrote:
>> Hello Sothy,
>> 
>> Sorry for the delay in answering, some busy days here.
>> 
>> 
>> On 07/25/2016 12:03 PM, sothy shan wrote:
>>> 
>>> Hello Mauricio,
>>> 
>>> On Sun, Jul 24, 2016 at 12:17 PM, Mauricio Vasquez
>>> <mauricio.vasquez@polito.it> wrote:
>>>> 
>>>> Hi Sothy,
>>>> 
>>>> 
>>>> On 07/21/2016 11:55 AM, sothy shan wrote:
>>>>> 
>>>>> On Thu, Jul 21, 2016 at 11:43 AM, Mauricio Vasquez
>>>>> <mauricio.vasquez@polito.it> wrote:
>>>>>> 
>>>>>> 
>>>>>> On 07/21/2016 10:51 AM, sothy shan wrote:
>>>>>>> 
>>>>>>> On Thu, Jul 21, 2016 at 10:41 AM, sothy shan <sothy.e98@gmail.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> On Wed, Jul 20, 2016 at 9:02 PM, Mauricio Vasquez
>>>>>>>> <mauricio.vasquez@polito.it> wrote:
>>>>>>>>> 
>>>>>>>>> Hi Sothy,
>>>>>>>>> 
>>>>>>>>> On 07/20/2016 07:40 PM, Aaron Conole wrote:
>>>>>>>>>> 
>>>>>>>>>> sothy shan <sothy.e98@gmail.com> writes:
>>>>>>>>>> 
>>>>>>>>>>> Hello all,
>>>>>>>>>> 
>>>>>>>>>> Hi Sothy,
>>>>>>>>>> 
>>>>>>>>>>> I followed the installion guide provided in the web site
>>>>>>>>>>> 
>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openvswitch_ovs_blob_master_INSTALL.DPDK.md-23ovstc&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=ojiXK7Tj1ANsHE_AlchVROP4R7LrgLQqVqTx7BjJQas&e= 
>>>>>>>>>>> to install dpdk OVS.
>>>>>>>>>>> 
>>>>>>>>>>> I am rurnning Ubuntu 14.04 (kernel 3.13). It is a VM running
>>>>>>>>>>> VMware.
>>>>>>>>>>> One intereface is enabled by VMXNET3.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> It is not clear for me, are you trying to install OvS within the VM?
>>>>>>>>> 
>>>>>>>>>>> I used dpdk 16.04
>>>>>>>>>>> Today I made clone of OVS (committ
>>>>>>>>>>> :bf32e3e2c029da18b5d6fd9210cb0ea12a1d0383).
>>>>>>>>>>> 
>>>>>>>>>>> The following are different configuration compared to above link.
>>>>>>>>>>> 
>>>>>>>>>>> sysctl -w vm.nr_hugepages=2048
>>>>>>>>>>> Binded interface via igb_uio.
>>>>>>>>>>> 
>>>>>>>>>>> Then followed same configuration to start ovs-vsswitch with dpdk.
>>>>>>>>>> 
>>>>>>>>>> Can you please post the steps exactly that you followed?  Which
>>>>>>>>>> user
>>>>>>>>>> you
>>>>>>>>>> were running as, etc?  Can you also post how you expect packets to
>>>>>>>>>> arrive into the vm?
>>>>>>>>>> 
>>>>>>>>>>> I am able to creatre bridge. When I create a port. It gives error
>>>>>>>>>>> message.
>>>>>>>>>> 
>>>>>>>>>> Please show the steps for this, too.  The exact commands you used.
>>>>>>>>>> 
>>>>>>>>>>> When I check ovs-vswitchd.log file,
>>>>>>>>>>> 
>>>>>>>>>>> There is warning that dpif_netlink|WARN|genric netlink family,
>>>>>>>>>>> ovs_datapath does not exist. The openvswitch kernel module may be
>>>>>>>>>>> not
>>>>>>>>>>> loaded.
>>>>>>>>>> 
>>>>>>>>>> This is definitely a problem.  How did you start the ovs-vswitchd
>>>>>>>>>> daemon?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> This is not a problem Aaron, the ovs kernel module is not needed
>>>>>>>>> when
>>>>>>>>> ovs is
>>>>>>>>> compiled with DPDK.
>>>>>>>>>>> 
>>>>>>>>>>> Then ERR: ERR|interface dpdk0 start error:operation not permitted.
>>>>>>>>>>> 
>>>>>>>>>>> The corresponding port is loaded by dpdk driver as shown in
>>>>>>>>>>> dpdk_nic_bind --status.
>>>>>>>>>>> 
>>>>>>>>>>> After that I did sudo ovs-vsctl show. it show br and port details
>>>>>>>>>>> as
>>>>>>>>>>> well.
>>>>>>>>>>> I am able to add one flow rule as well. But ovs bridge didnt get
>>>>>>>>>>> packets from the interface.
>>>>>>>>>>> 
>>>>>>>>>>> Do you know why this error is popup? anyway to solve the problem?
>>>>>>>>>>> 
>>>>>>>>>>> Thank you very much
>>>>>>>>>>> 
>>>>>>>>>>> Best regars
>>>>>>>>>>> Sothy
>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> discuss mailing list
>>>>>>>>>> discuss@openvswitch.org
>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailman_listinfo_discuss&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=DZpyuDSP-5BYH4bvfNHk68XRFB9-M-M43FxZCCpZKho&e= 
>>>>>>>>> 
>>>>>>>>> As Aaron wrote, having the exact commands is important to help you.
>>>>>>>> 
>>>>>>>> My configuration and log info:
>>>>>>>> 
>>>>>>>> sudo sysctl vm.nr_hugepages=2048
>>>>>>>> sudo mkdir -p /dev/hugepages
>>>>>>>> sudo  mount -t hugetlbfs hugetlbfs /dev/hugepages
>>>>>>>> sudo modprobe uio
>>>>>>>> sudo insmod $DPDK_BUILD/kmod/igb_uio.ko
>>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py --status
>>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py -b igb_uio 0000:0b:00.0
>>>>>>>> sudo ./dpdk-16.04/tools/dpdk_nic_bind.py --status
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Network devices using DPDK-compatible driver
>>>>>>>> ============================================
>>>>>>>> 0000:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=
>>>>>>>> 0000:13:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=
>>>>>>>> 
>>>>>>>> Network devices using kernel driver
>>>>>>>> ===================================
>>>>>>>> 0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3
>>>>>>>> unused=igb_uio *Active*
>>>>>>>> 
>>>>>>>> Other network devices
>>>>>>>> =====================
>>>>>>>> <none>
>>>>>>>> 
>>>>>>>> .........Then......................
>>>>>>>> 
>>>>>>>> 
>>>>>>>> sudo ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
>>>>>>>> sudo ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
>>>>>> 
>>>>>> These two commands are inverted.
>>>>>> The bridge has to be created first and then ports have to be added to
>>>>>> it.
>>>>>> 
>>>>> It is my mistake when coping.
>>>>>>>> 
>>>>>>>> Log file/
>>>>>>>> 2016-07-21T07:53:35.527Z|00001|vlog|INFO|opened log file
>>>>>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>>>>>> 2016-07-21T07:53:35.529Z|00002|ovs_numa|INFO|Discovered 2 CPU cores
>>>>>>>> on
>>>>>>>> NUMA node 0
>>>>>>>> 2016-07-21T07:53:35.529Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes
>>>>>>>> and 2 CPU cores
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 2016-07-21T07:53:35.529Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>>>>> connecting...
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 2016-07-21T07:53:35.529Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>>>>> connected
>>>>>>>> 2016-07-21T07:53:35.533Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>>>>>>> 2016-07-21T07:53:35.533Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>>>>>>> defaulting to /usr/local/var/run/openvswitch
>>>>>>>> 2016-07-21T07:53:35.533Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>>>>>> --socket-mem 1024,0 -c 0x00000001
>>>>>>>> 2016-07-21T07:53:36.606Z|00009|timeval|WARN|Unreasonably long 1076ms
>>>>>>>> poll interval (80ms user, 875ms system)
>>>>>>>> 2016-07-21T07:53:36.606Z|00010|timeval|WARN|faults: 3306 minor, 0
>>>>>>>> major
>>>>>>>> 2016-07-21T07:53:36.606Z|00011|timeval|WARN|context switches: 3
>>>>>>>> voluntary, 101 involuntary
>>>>>>>> 2016-07-21T07:53:36.606Z|00012|coverage|INFO|Event coverage, avg rate
>>>>>>>> over last: 5 seconds, last minute, last hour,  hash=398361de:
>>>>>>>> 2016-07-21T07:53:36.606Z|00013|coverage|INFO|bridge_reconfigure
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>> 2016-07-21T07:53:36.606Z|00014|coverage|INFO|cmap_expand
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 9
>>>>>>>> 2016-07-21T07:53:36.606Z|00015|coverage|INFO|miniflow_malloc
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 9
>>>>>>>> 2016-07-21T07:53:36.606Z|00016|coverage|INFO|hmap_pathological
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>> 2016-07-21T07:53:36.606Z|00017|coverage|INFO|hmap_expand
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 640
>>>>>>>> 2016-07-21T07:53:36.606Z|00018|coverage|INFO|txn_unchanged
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>>>>> 2016-07-21T07:53:36.606Z|00019|coverage|INFO|txn_incomplete
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>> 2016-07-21T07:53:36.607Z|00020|coverage|INFO|poll_create_node
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 35
>>>>>>>> 2016-07-21T07:53:36.607Z|00021|coverage|INFO|seq_change
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 38
>>>>>>>> 2016-07-21T07:53:36.607Z|00022|coverage|INFO|pstream_open
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>> 2016-07-21T07:53:36.607Z|00023|coverage|INFO|stream_open
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>> 2016-07-21T07:53:36.607Z|00024|coverage|INFO|util_xalloc
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 11049
>>>>>>>> 2016-07-21T07:53:36.607Z|00025|coverage|INFO|netdev_get_hwaddr
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>> 2016-07-21T07:53:36.607Z|00026|coverage|INFO|netlink_received
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>>>>>> 2016-07-21T07:53:36.607Z|00027|coverage|INFO|netlink_sent
>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>> 2016-07-21T07:53:36.607Z|00028|coverage|INFO|86 events never hit
>>>>>>>> 2016-07-21T07:53:36.607Z|00029|bridge|INFO|ovs-vswitchd (Open
>>>>>>>> vSwitch)
>>>>>>>> 2.5.90
>>>>>>>> 2016-07-21T07:54:50.861Z|00030|memory|INFO|18980 kB peak resident set
>>>>>>>> size after 75.3 seconds
>>>>>>>> 2016-07-21T07:54:50.866Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> Datapath supports recirculation
>>>>>>>> 2016-07-21T07:54:50.866Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> MPLS label stack length probed as 3
>>>>>>>> 2016-07-21T07:54:50.866Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> Datapath supports truncate action
>>>>>>>> 2016-07-21T07:54:50.866Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> Datapath supports unique flow ids
>>>>>>>> 2016-07-21T07:54:50.866Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> Datapath does not support ct_state
>>>>>>>> 2016-07-21T07:54:50.866Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> Datapath does not support ct_zone
>>>>>>>> 2016-07-21T07:54:50.866Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> Datapath does not support ct_mark
>>>>>>>> 2016-07-21T07:54:50.866Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> Datapath does not support ct_label
>>>>>>>> 2016-07-21T07:54:50.866Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>> Datapath does not support ct_state_nat
>>>>>>>> 2016-07-21T07:54:50.875Z|00040|bridge|INFO|bridge br0: added
>>>>>>>> interface
>>>>>>>> br0 on port 65534
>>>>>>>> 2016-07-21T07:54:50.887Z|00041|dpif_netlink|WARN|Generic Netlink
>>>>>>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module
>>>>>>>> is probably not loaded.
>>>>>>>> 2016-07-21T07:54:50.887Z|00042|bridge|INFO|bridge br0: using datapath
>>>>>>>> ID 0000c664fbd86348
>>>>>>>> 2016-07-21T07:54:50.887Z|00043|connmgr|INFO|br0: added service
>>>>>>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>>>>>>> 2016-07-21T07:58:18.796Z|00044|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>>>>>> 2016-07-21T07:58:18.810Z|00045|dpdk|ERR|Interface dpdk0 start error:
>>>>>>>> Operation not permitted
>>>>>>>> 2016-07-21T07:58:18.810Z|00046|dpif|WARN|netdev@ovs-netdev: failed to
>>>>>>>> add dpdk0 as port: Operation not permitted
>>>>>>>> 2016-07-21T08:10:35.440Z|00047|connmgr|INFO|br0<->unix: 1 flow_mods
>>>>>>>> in
>>>>>>>> the last 0 s (1 adds)
>>>>>> 
>>>>>> Could you also provide the full terminal output when you run
>>>>>> ovs-vswitchd?,
>>>>>> I would like to see the DPDK output
>>>>> 
>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vswitchd unix:$DB_SOCK
>>>>> --pidfile --detach --log-file=/var/log/openvswitch/ovs-vswitchd.log
>>>>> 2016-07-21T09:33:35Z|00001|vlog|INFO|opened log file
>>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>>> 2016-07-21T09:33:35Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA
>>>>> node 0
>>>>> 2016-07-21T09:33:35Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2
>>>>> CPU
>>>>> cores
>>>>> 
>>>>> 
>>>>> 2016-07-21T09:33:35Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>> connecting...
>>>>> 
>>>>> 
>>>>> 2016-07-21T09:33:35Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>> connected
>>>>> 2016-07-21T09:33:35Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>>>> 2016-07-21T09:33:35Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>>>> defaulting to /usr/local/var/run/openvswitch
>>>>> 2016-07-21T09:33:35Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>>> --socket-mem 1024,0 -c 0x00000001
>>>>> EAL: Detected lcore 0 as core 0 on socket 0
>>>>> EAL: Detected lcore 1 as core 0 on socket 0
>>>>> EAL: Support maximum 128 logical core(s) by configuration.
>>>>> EAL: Detected 2 lcore(s)
>>>>> EAL: Probing VFIO support...
>>>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>>>>> directory)
>>>>> EAL: VFIO modules not loaded, skipping VFIO support...
>>>>> EAL: Setting up physically contiguous memory...
>>>>> EAL: Ask a virtual area of 0x5e400000 bytes
>>>>> EAL: Virtual area found at 0x7fa618400000 (size = 0x5e400000)
>>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>>> EAL: Virtual area found at 0x7fa618000000 (size = 0x200000)
>>>>> EAL: Ask a virtual area of 0xa1800000 bytes
>>>>> EAL: Virtual area found at 0x7fa576600000 (size = 0xa1800000)
>>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>>> EAL: Virtual area found at 0x7fa576200000 (size = 0x200000)
>>>>> EAL: Requesting 512 pages of size 2MB from socket 0
>>>>> EAL: TSC frequency is ~2799213 KHz
>>>>> EAL: Master lcore 0 is ready (tid=78ad7b80;cpuset=[0])
>>>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1
>>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>>> EAL:   Not managed by a supported kernel driver, skipped
>>>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
>>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>>> EAL:   PCI memory mapped at 0x7fa658400000
>>>>> EAL:   PCI memory mapped at 0x7fa658401000
>>>>> EAL:   PCI memory mapped at 0x7fa658402000
>>>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1
>>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>>> EAL:   PCI memory mapped at 0x7fa658404000
>>>>> EAL:   PCI memory mapped at 0x7fa658405000
>>>>> EAL:   PCI memory mapped at 0x7fa658406000
>>>>> Zone 0: name:<RG_MP_log_history>, phys:0x82bfdec0, len:0x2080,
>>>>> virt:0x7fa6583fdec0, socket_id:0, flags:0
>>>>> Zone 1: name:<MP_log_history>, phys:0x82973d40, len:0x28a0c0,
>>>>> virt:0x7fa658173d40, socket_id:0, flags:0
>>>>> Zone 2: name:<rte_eth_dev_data>, phys:0x82943200, len:0x2f800,
>>>>> virt:0x7fa658143200, socket_id:0, flags:0
>>>>> 2016-07-21T09:33:36Z|00009|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>> Datapath supports recirculation
>>>>> 2016-07-21T09:33:36Z|00010|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS
>>>>> label stack length probed as 3
>>>>> 2016-07-21T09:33:36Z|00011|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>> Datapath supports truncate action
>>>>> 2016-07-21T09:33:36Z|00012|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>> Datapath supports unique flow ids
>>>>> 2016-07-21T09:33:36Z|00013|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>> Datapath does not support ct_state
>>>>> 2016-07-21T09:33:36Z|00014|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>> Datapath does not support ct_zone
>>>>> 2016-07-21T09:33:36Z|00015|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>> Datapath does not support ct_mark
>>>>> 2016-07-21T09:33:36Z|00016|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>> Datapath does not support ct_label
>>>>> 2016-07-21T09:33:36Z|00017|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>> Datapath does not support ct_state_nat
>>>>> 2016-07-21T09:33:36Z|00018|bridge|INFO|bridge br0: added interface br0
>>>>> on port 65534
>>>>> 2016-07-21T09:33:36Z|00019|dpif_netlink|WARN|Generic Netlink family
>>>>> 'ovs_datapath' does not exist. The Open vSwitch kernel module is
>>>>> probably not loaded.
>>>>> 2016-07-21T09:33:36Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>>> 2016-07-21T09:33:36Z|00021|dpdk|ERR|Interface dpdk0 start error:
>>>>> Operation not permitted
>>>>> 2016-07-21T09:33:36Z|00022|bridge|INFO|bridge br0: using datapath ID
>>>>> 0000c664fbd86348
>>>>> 2016-07-21T09:33:36Z|00023|connmgr|INFO|br0: added service controller
>>>>> "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>>>> 2016-07-21T09:33:36Z|00024|timeval|WARN|Unreasonably long 1244ms poll
>>>>> interval (233ms user, 880ms system)
>>>>> 2016-07-21T09:33:36Z|00025|timeval|WARN|faults: 3902 minor, 0 major
>>>>> 2016-07-21T09:33:36Z|00026|timeval|WARN|context switches: 10
>>>>> voluntary, 214 involuntary
>>>>> 2016-07-21T09:33:36Z|00027|coverage|INFO|Event coverage, avg rate over
>>>>> last: 5 seconds, last minute, last hour,  hash=75cfd2fc:
>>>>> 2016-07-21T09:33:36Z|00028|coverage|INFO|bridge_reconfigure
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00029|coverage|INFO|ofproto_flush
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00030|coverage|INFO|ofproto_update_port
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00031|coverage|INFO|rev_flow_table
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00032|coverage|INFO|cmap_expand
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 22
>>>>> 2016-07-21T09:33:36Z|00033|coverage|INFO|cmap_shrink
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 7
>>>>> 2016-07-21T09:33:36Z|00034|coverage|INFO|dpif_port_add
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>> 2016-07-21T09:33:36Z|00035|coverage|INFO|dpif_flow_flush
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>> 2016-07-21T09:33:36Z|00036|coverage|INFO|dpif_flow_get
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>>> 2016-07-21T09:33:36Z|00037|coverage|INFO|dpif_flow_put
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 10
>>>>> 2016-07-21T09:33:36Z|00038|coverage|INFO|dpif_flow_del
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>>> 2016-07-21T09:33:36Z|00039|coverage|INFO|dpif_execute
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>>> 2016-07-21T09:33:36Z|00040|coverage|INFO|flow_extract
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 4
>>>>> 2016-07-21T09:33:36Z|00041|coverage|INFO|miniflow_malloc
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 16
>>>>> 2016-07-21T09:33:36Z|00042|coverage|INFO|hmap_pathological
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00043|coverage|INFO|hmap_expand
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 653
>>>>> 2016-07-21T09:33:36Z|00044|coverage|INFO|netdev_get_stats
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00045|coverage|INFO|poll_create_node
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 15
>>>>> 2016-07-21T09:33:36Z|00046|coverage|INFO|poll_zero_timeout
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00047|coverage|INFO|seq_change
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 575
>>>>> 2016-07-21T09:33:36Z|00048|coverage|INFO|pstream_open
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>>> 2016-07-21T09:33:36Z|00049|coverage|INFO|stream_open
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00050|coverage|INFO|util_xalloc
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 11820
>>>>> 2016-07-21T09:33:36Z|00051|coverage|INFO|netdev_set_policing
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00052|coverage|INFO|netdev_get_ifindex
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00053|coverage|INFO|netdev_get_hwaddr
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>> 2016-07-21T09:33:36Z|00054|coverage|INFO|netdev_set_hwaddr
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00055|coverage|INFO|netdev_get_ethtool
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>> 2016-07-21T09:33:36Z|00056|coverage|INFO|netlink_received
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 6
>>>>> 2016-07-21T09:33:36Z|00057|coverage|INFO|netlink_recv_jumbo
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>> 2016-07-21T09:33:36Z|00058|coverage|INFO|netlink_sent
>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>>> 2016-07-21T09:33:36Z|00059|coverage|INFO|70 events never hit
>>>>> 2016-07-21T09:33:36Z|00060|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>>> 2016-07-21T09:33:36Z|00061|dpdk|ERR|Interface dpdk0 start error:
>>>>> Operation not permitted
>>>> 
>>>> The PMD start function is failing, unfortunately that error does not tell
>>>> that much information.
>>>> 
>>>> I would suggest you to enable the debug of the vmxnet3 PMD in DPDK, set
>>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y in the config/common_base file of
>>>> DPDK, then recompile DPDK and OvS.
>>>> 
>>>> Please try once again and send the terminal output, I hope we can get a
>>>> more
>>>> information of what is going on with that log.
>>> 
>>> Based on your request, I reproduce the log with
>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y
>>> sudo ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock
>>> --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile --detach
>>> 2016-07-25T09:24:17Z|00001|vlog|INFO|opened log file
>>> /var/log/openvswitch/ovs-vswitchd.log
>>> 2016-07-25T09:24:17Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA
>>> node 0
>>> 2016-07-25T09:24:17Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU
>>> cores
>>> 
>>> 2016-07-25T09:24:17Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>> connecting...
>>> 
>>> 2016-07-25T09:24:17Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>> connected
>>> 2016-07-25T09:24:17Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>> 2016-07-25T09:24:17Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>> defaulting to /usr/local/var/run/openvswitch
>>> 2016-07-25T09:24:17Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>> --socket-mem 1024,0 -c 0x00000001
>>> EAL: Detected lcore 0 as core 0 on socket 0
>>> EAL: Detected lcore 1 as core 0 on socket 0
>>> EAL: Support maximum 128 logical core(s) by configuration.
>>> EAL: Detected 2 lcore(s)
>>> EAL: Probing VFIO support...
>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>>> directory)
>>> EAL: VFIO modules not loaded, skipping VFIO support...
>>> EAL: Setting up physically contiguous memory...
>>> EAL: Ask a virtual area of 0x200000 bytes
>>> EAL: Virtual area found at 0x7fdea6000000 (size = 0x200000)
>>> EAL: Ask a virtual area of 0x5e000000 bytes
>>> EAL: Virtual area found at 0x7fde47e00000 (size = 0x5e000000)
>>> EAL: Ask a virtual area of 0x200000 bytes
>>> EAL: Virtual area found at 0x7fde47a00000 (size = 0x200000)
>>> EAL: Ask a virtual area of 0x200000 bytes
>>> EAL: Virtual area found at 0x7fde47600000 (size = 0x200000)
>>> EAL: Ask a virtual area of 0xa1800000 bytes
>>> EAL: Virtual area found at 0x7fdda5c00000 (size = 0xa1800000)
>>> EAL: Ask a virtual area of 0x200000 bytes
>>> EAL: Virtual area found at 0x7fdda5800000 (size = 0x200000)
>>> EAL: Requesting 512 pages of size 2MB from socket 0
>>> EAL: TSC frequency is ~2799209 KHz
>>> EAL: Master lcore 0 is ready (tid=a855eb80;cpuset=[0])
>>> PMD: rte_vmxnet3_pmd_init():  >>
>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1
>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>> EAL:   Not managed by a supported kernel driver, skipped
>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>> EAL:   PCI memory mapped at 0x7fdea6200000
>>> EAL:   PCI memory mapped at 0x7fdea6201000
>>> EAL:   PCI memory mapped at 0x7fdea6202000
>>> PMD: eth_vmxnet3_dev_init():  >>
>>> PMD: eth_vmxnet3_dev_init(): Hardware version : 1
>>> PMD: eth_vmxnet3_dev_init(): UPT hardware version : 1
>>> PMD: eth_vmxnet3_dev_init(): MAC Address : 00:0c:29:b5:9f:5b
>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1
>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>> EAL:   Not managed by a supported kernel driver, skipped
>>> Zone 0: name:<RG_MP_log_history>, phys:0x555fdec0, len:0x2080,
>>> virt:0x7fdea61fdec0, socket_id:0, flags:0
>>> Zone 1: name:<MP_log_history>, phys:0x95375f00, len:0x28a0c0,
>>> virt:0x7fde87975f00, socket_id:0, flags:0
>>> Zone 2: name:<rte_eth_dev_data>, phys:0x555cd300, len:0x2f800,
>>> virt:0x7fdea61cd300, socket_id:0, flags:0
>>> 
>>> PMD: vmxnet3_dev_configure():  >>
>>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>>> PMD: vmxnet3_dev_rx_queue_setup():  >>
>>> PMD: vmxnet3_dev_start():  >>
>>> PMD: vmxnet3_rss_configure():  >>
>>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b
>>> PMD: vmxnet3_disable_intr():  >>
>>> PMD: vmxnet3_dev_rxtx_init():  >>
>>> 2016-07-25T09:40:23Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>> PMD: vmxnet3_dev_stop():  >>
>>> PMD: vmxnet3_disable_intr():  >>
>>> PMD: vmxnet3_dev_stop(): Device reset.
>>> PMD: vmxnet3_dev_clear_queues():  >>
>>> PMD: vmxnet3_dev_configure():  >>
>>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>>> PMD: vmxnet3_dev_rx_queue_setup():  >>
>>> PMD: vmxnet3_dev_start():  >>
>>> PMD: vmxnet3_rss_configure():  >>
>>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b
>>> PMD: vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
>> 
>> 
>> Unfortunately it is the most I can help you, it is strange for me why
>> test-pmd is working and why OvS  is not.
>> I think it is a DPDK issue, then the next step is to post on the DPDK
>> mailing (dev@dpdk.org, add also in CC Yong Wang <yongwang@vmware.com> that
>> is the vmxnet3 maintainer), send a link to this thread in order to be clear
>> what you have tested.
> 
> 
> Hello Yong Wang,
> 
> I am running fedora VM in ESXi 5.5.0 environment. I tried to install
> OVS and DPDK. OVS couldnt start DPDK device.
> You can see the error in the mail below. Do you know why? IF you need
> more information. I can provide more information.
> 
> Good news, when I tried in Fedora 23 VM in ESXi 5.5.0, it works. It is
> possible to bind an dpdk device and get packets on port.
> 
> Thanks Mauricio for your help and reply.
> 
> Best regards
> Sothy

Sothy,

You mentioned that on Fedora 23, it works.  Then on some version Fedora that it failed?  From your previous post, you mentioned that it’s a Ubuntu 14.04 VM?  Can you clarify?  Other than the changes you mentioned already, are there any other changes (such as ring buffer size, number of rx queues, etc) that have been changed?  Is there any updates installed on your ESX 5.5? If yes, what’s the exact update version?  Can you also send me a copy of your vmkernel.log offline?

Also copying Daniele as he is trying to help reproduce the reported issue.

> 
>> 
>>> I deleted exisiting bridge and recreated::::::
>>> 
>>> 
>>> Then I added bridge. After I try to create bridge. It gives error. The
>>> erros is same as shown in log file.
>>> 
>>> 2016-07-25T09:29:43.248Z|00020|bridge|INFO|bridge br0: added interface
>>> br0 on port 65534
>>> 2016-07-25T09:29:43.253Z|00021|dpif_netlink|WARN|Generic Netlink
>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module
>>> is probably not loaded.
>>> 2016-07-25T09:29:43.253Z|00022|bridge|INFO|bridge br0: using datapath
>>> ID 0000e6aac740584b
>>> 2016-07-25T09:29:43.253Z|00023|connmgr|INFO|br0: added service
>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>> 2016-07-25T09:30:54.008Z|00024|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>> 2016-07-25T09:30:54.021Z|00025|dpdk|ERR|Interface dpdk0 start error:
>>> Operation not permitted
>>> 2016-07-25T09:30:54.021Z|00026|dpif|WARN|netdev@ovs-netdev: failed to
>>> add dpdk0 as port: Operation not permitted
>>> 
>>> 
>>> 
>>>> Another thing to do would be to bind the NICs to the uio_pci_generic and
>>>> VFIO modules, just to be sure that the issue is also present there.
>>> 
>>> I tried in uio and uio_pci_genric also. Both are giving same error.
>>> I suspect I cannt activate VFIO in VM since it is BIOS suport also. AM I
>>> right?
>>> 
>>> Thanks
>>> 
>>>> 
>>>>> in command line some information for your verification:
>>>>> 
>>>>>   sudo ovs-vsctl show
>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2
>>>>>      Bridge "br0"
>>>>>          Port "dpdk0"
>>>>>              Interface "dpdk0"
>>>>>                  type: dpdk
>>>>>          Port "br0"
>>>>>              Interface "br0"
>>>>>                  type: internal
>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl del-br br0
>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-br br0 --
>>>>> set bridge br0 datapath_type=netdev
>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-port br0
>>>>> dpdk0 -- set Interface dpdk0 type=dpdk
>>>>> ovs-vsctl: Error detected while setting up 'dpdk0'.  See ovs-vswitchd
>>>>> log for details.
>>>>> 
>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>> sudo ovs-vsctl show
>>>>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2
>>>>>>>>       Bridge "br0"
>>>>>>>>           Port "dpdk0"
>>>>>>>>               Interface "dpdk0"
>>>>>>>>                   type: dpdk
>>>>>>>>           Port "br0"
>>>>>>>>               Interface "br0"
>>>>>>>>                   type: internal
>>>>>>>> 
>>>>>>>> sudo ovs-ofctl dump-flows br0
>>>>>>>> NXST_FLOW reply (xid=0x4):
>>>>>>>>    cookie=0x0, duration=487.638s, table=0, n_packets=0, n_bytes=0,
>>>>>>>> idle_age=487, priority=0 actions=NORMAL
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> In addition I would recommend to use the test-pmd [1] application
>>>>>>>>> from
>>>>>>>>> DPDK
>>>>>>>>> to check if the issue is also present in that case.
>>>>>>> 
>>>>>>> I run testpmd command.The result is
>>>>>>> 
>>>>>>> Configuring Port 0 (socket 0)
>>>>>>> Port 0: 00:0C:29:B5:9F:5B
>>>>>>> Configuring Port 1 (socket 0)
>>>>>>> Port 1: 00:0C:29:B5:9F:65
>>>>>>> Checking link statuses...
>>>>>>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>>>>>>> Port 1 Link Up - speed 10000 Mbps - full-duplex
>>>>>>> Done
>>>>>>> No commandline core given, start packet forwarding
>>>>>>>     io packet forwarding - CRC stripping disabled - packets/burst=32
>>>>>>>     nb forwarding cores=1 - nb forwarding ports=2
>>>>>>>     RX queues=1 - RX desc=128 - RX free threshold=0
>>>>>>>     RX threshold registers: pthresh=0 hthresh=0 wthresh=0
>>>>>>>     TX queues=1 - TX desc=512 - TX free threshold=0
>>>>>>>     TX threshold registers: pthresh=0 hthresh=0 wthresh=0
>>>>>>>     TX RS bit threshold=0 - TXQ flags=0x200
>>>>>>> Press enter to exit
>>>>>>> 
>>>>>>> Telling cores to stop...
>>>>>>> Waiting for lcores to finish...
>>>>>>> 
>>>>>>>     ---------------------- Forward statistics for port 0
>>>>>>> ----------------------
>>>>>>>     RX-packets: 88             RX-dropped: 0             RX-total: 88
>>>>>>>     TX-packets: 2              TX-dropped: 0             TX-total: 2
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ----------------------------------------------------------------------------
>>>>>>> 
>>>>>>>     ---------------------- Forward statistics for port 1
>>>>>>> ----------------------
>>>>>>>     RX-packets: 2              RX-dropped: 0             RX-total: 2
>>>>>>>     TX-packets: 88             TX-dropped: 0             TX-total: 88
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ----------------------------------------------------------------------------
>>>>>>> 
>>>>>>>     +++++++++++++++ Accumulated forward statistics for all
>>>>>>> ports+++++++++++++++
>>>>>>>     RX-packets: 90             RX-dropped: 0             RX-total: 90
>>>>>>>     TX-packets: 90             TX-dropped: 0             TX-total: 90
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>> 
>>>>>>> make sense?
>>>>>> 
>>>>>> 
>>>>>> Yes, it makes sense. Ports are being recognized by DPDK and some
>>>>>> packets
>>>>>> were exchanged.
>>>>>> 
>>>>>> 
>>>>>>>>> [1] https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides-2D16.04_testpmd-5Fapp-5Fug_index.html&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=JnLeHoyKuiecYI95-fcfFiuev5g2CDweT2MnyUiKWzM&e= 
>>>>>>>>> 
>>>>>>>>> Mauricio Vasquez,
>>>>>> 
>>>>>> 
>>>> Mauricio V,
>> 
>> 

Re: [dpdk-dev] [ovs-discuss] OVS DPDK|DPDK|ERR|interface dpdk0 start error:operation not permitted.

[sothy shan]

On Thu, Jul 28, 2016 at 2:29 AM, Yong Wang <yongwang@vmware.com> wrote:
>
>> On Jul 27, 2016, at 12:25 AM, sothy shan <sothy.e98@gmail.com> wrote:
>>
>> On Tue, Jul 26, 2016 at 6:32 PM, Mauricio Vasquez
>> <mauricio.vasquez@polito.it> wrote:
>>> Hello Sothy,
>>>
>>> Sorry for the delay in answering, some busy days here.
>>>
>>>
>>> On 07/25/2016 12:03 PM, sothy shan wrote:
>>>>
>>>> Hello Mauricio,
>>>>
>>>> On Sun, Jul 24, 2016 at 12:17 PM, Mauricio Vasquez
>>>> <mauricio.vasquez@polito.it> wrote:
>>>>>
>>>>> Hi Sothy,
>>>>>
>>>>>
>>>>> On 07/21/2016 11:55 AM, sothy shan wrote:
>>>>>>
>>>>>> On Thu, Jul 21, 2016 at 11:43 AM, Mauricio Vasquez
>>>>>> <mauricio.vasquez@polito.it> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 07/21/2016 10:51 AM, sothy shan wrote:
>>>>>>>>
>>>>>>>> On Thu, Jul 21, 2016 at 10:41 AM, sothy shan <sothy.e98@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On Wed, Jul 20, 2016 at 9:02 PM, Mauricio Vasquez
>>>>>>>>> <mauricio.vasquez@polito.it> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Sothy,
>>>>>>>>>>
>>>>>>>>>> On 07/20/2016 07:40 PM, Aaron Conole wrote:
>>>>>>>>>>>
>>>>>>>>>>> sothy shan <sothy.e98@gmail.com> writes:
>>>>>>>>>>>
>>>>>>>>>>>> Hello all,
>>>>>>>>>>>
>>>>>>>>>>> Hi Sothy,
>>>>>>>>>>>
>>>>>>>>>>>> I followed the installion guide provided in the web site
>>>>>>>>>>>>
>>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openvswitch_ovs_blob_master_INSTALL.DPDK.md-23ovstc&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=ojiXK7Tj1ANsHE_AlchVROP4R7LrgLQqVqTx7BjJQas&e=
>>>>>>>>>>>> to install dpdk OVS.
>>>>>>>>>>>>
>>>>>>>>>>>> I am rurnning Ubuntu 14.04 (kernel 3.13). It is a VM running
>>>>>>>>>>>> VMware.
>>>>>>>>>>>> One intereface is enabled by VMXNET3.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> It is not clear for me, are you trying to install OvS within the VM?
>>>>>>>>>>
>>>>>>>>>>>> I used dpdk 16.04
>>>>>>>>>>>> Today I made clone of OVS (committ
>>>>>>>>>>>> :bf32e3e2c029da18b5d6fd9210cb0ea12a1d0383).
>>>>>>>>>>>>
>>>>>>>>>>>> The following are different configuration compared to above link.
>>>>>>>>>>>>
>>>>>>>>>>>> sysctl -w vm.nr_hugepages=2048
>>>>>>>>>>>> Binded interface via igb_uio.
>>>>>>>>>>>>
>>>>>>>>>>>> Then followed same configuration to start ovs-vsswitch with dpdk.
>>>>>>>>>>>
>>>>>>>>>>> Can you please post the steps exactly that you followed?  Which
>>>>>>>>>>> user
>>>>>>>>>>> you
>>>>>>>>>>> were running as, etc?  Can you also post how you expect packets to
>>>>>>>>>>> arrive into the vm?
>>>>>>>>>>>
>>>>>>>>>>>> I am able to creatre bridge. When I create a port. It gives error
>>>>>>>>>>>> message.
>>>>>>>>>>>
>>>>>>>>>>> Please show the steps for this, too.  The exact commands you used.
>>>>>>>>>>>
>>>>>>>>>>>> When I check ovs-vswitchd.log file,
>>>>>>>>>>>>
>>>>>>>>>>>> There is warning that dpif_netlink|WARN|genric netlink family,
>>>>>>>>>>>> ovs_datapath does not exist. The openvswitch kernel module may be
>>>>>>>>>>>> not
>>>>>>>>>>>> loaded.
>>>>>>>>>>>
>>>>>>>>>>> This is definitely a problem.  How did you start the ovs-vswitchd
>>>>>>>>>>> daemon?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This is not a problem Aaron, the ovs kernel module is not needed
>>>>>>>>>> when
>>>>>>>>>> ovs is
>>>>>>>>>> compiled with DPDK.
>>>>>>>>>>>>
>>>>>>>>>>>> Then ERR: ERR|interface dpdk0 start error:operation not permitted.
>>>>>>>>>>>>
>>>>>>>>>>>> The corresponding port is loaded by dpdk driver as shown in
>>>>>>>>>>>> dpdk_nic_bind --status.
>>>>>>>>>>>>
>>>>>>>>>>>> After that I did sudo ovs-vsctl show. it show br and port details
>>>>>>>>>>>> as
>>>>>>>>>>>> well.
>>>>>>>>>>>> I am able to add one flow rule as well. But ovs bridge didnt get
>>>>>>>>>>>> packets from the interface.
>>>>>>>>>>>>
>>>>>>>>>>>> Do you know why this error is popup? anyway to solve the problem?
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you very much
>>>>>>>>>>>>
>>>>>>>>>>>> Best regars
>>>>>>>>>>>> Sothy
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> discuss mailing list
>>>>>>>>>>> discuss@openvswitch.org
>>>>>>>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__openvswitch.org_mailman_listinfo_discuss&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=DZpyuDSP-5BYH4bvfNHk68XRFB9-M-M43FxZCCpZKho&e=
>>>>>>>>>>
>>>>>>>>>> As Aaron wrote, having the exact commands is important to help you.
>>>>>>>>>
>>>>>>>>> My configuration and log info:
>>>>>>>>>
>>>>>>>>> sudo sysctl vm.nr_hugepages=2048
>>>>>>>>> sudo mkdir -p /dev/hugepages
>>>>>>>>> sudo  mount -t hugetlbfs hugetlbfs /dev/hugepages
>>>>>>>>> sudo modprobe uio
>>>>>>>>> sudo insmod $DPDK_BUILD/kmod/igb_uio.ko
>>>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py --status
>>>>>>>>> sudo $DPDK_DIR/tools/dpdk_nic_bind.py -b igb_uio 0000:0b:00.0
>>>>>>>>> sudo ./dpdk-16.04/tools/dpdk_nic_bind.py --status
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Network devices using DPDK-compatible driver
>>>>>>>>> ============================================
>>>>>>>>> 0000:0b:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=
>>>>>>>>> 0000:13:00.0 'VMXNET3 Ethernet Controller' drv=igb_uio unused=
>>>>>>>>>
>>>>>>>>> Network devices using kernel driver
>>>>>>>>> ===================================
>>>>>>>>> 0000:03:00.0 'VMXNET3 Ethernet Controller' if=eth0 drv=vmxnet3
>>>>>>>>> unused=igb_uio *Active*
>>>>>>>>>
>>>>>>>>> Other network devices
>>>>>>>>> =====================
>>>>>>>>> <none>
>>>>>>>>>
>>>>>>>>> .........Then......................
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> sudo ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
>>>>>>>>> sudo ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
>>>>>>>
>>>>>>> These two commands are inverted.
>>>>>>> The bridge has to be created first and then ports have to be added to
>>>>>>> it.
>>>>>>>
>>>>>> It is my mistake when coping.
>>>>>>>>>
>>>>>>>>> Log file/
>>>>>>>>> 2016-07-21T07:53:35.527Z|00001|vlog|INFO|opened log file
>>>>>>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>>>>>>> 2016-07-21T07:53:35.529Z|00002|ovs_numa|INFO|Discovered 2 CPU cores
>>>>>>>>> on
>>>>>>>>> NUMA node 0
>>>>>>>>> 2016-07-21T07:53:35.529Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes
>>>>>>>>> and 2 CPU cores
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2016-07-21T07:53:35.529Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>>>>>> connecting...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2016-07-21T07:53:35.529Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>>>>>> connected
>>>>>>>>> 2016-07-21T07:53:35.533Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>>>>>>>> 2016-07-21T07:53:35.533Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>>>>>>>> defaulting to /usr/local/var/run/openvswitch
>>>>>>>>> 2016-07-21T07:53:35.533Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>>>>>>> --socket-mem 1024,0 -c 0x00000001
>>>>>>>>> 2016-07-21T07:53:36.606Z|00009|timeval|WARN|Unreasonably long 1076ms
>>>>>>>>> poll interval (80ms user, 875ms system)
>>>>>>>>> 2016-07-21T07:53:36.606Z|00010|timeval|WARN|faults: 3306 minor, 0
>>>>>>>>> major
>>>>>>>>> 2016-07-21T07:53:36.606Z|00011|timeval|WARN|context switches: 3
>>>>>>>>> voluntary, 101 involuntary
>>>>>>>>> 2016-07-21T07:53:36.606Z|00012|coverage|INFO|Event coverage, avg rate
>>>>>>>>> over last: 5 seconds, last minute, last hour,  hash=398361de:
>>>>>>>>> 2016-07-21T07:53:36.606Z|00013|coverage|INFO|bridge_reconfigure
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>>> 2016-07-21T07:53:36.606Z|00014|coverage|INFO|cmap_expand
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 9
>>>>>>>>> 2016-07-21T07:53:36.606Z|00015|coverage|INFO|miniflow_malloc
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 9
>>>>>>>>> 2016-07-21T07:53:36.606Z|00016|coverage|INFO|hmap_pathological
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>>> 2016-07-21T07:53:36.606Z|00017|coverage|INFO|hmap_expand
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 640
>>>>>>>>> 2016-07-21T07:53:36.606Z|00018|coverage|INFO|txn_unchanged
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>>>>>> 2016-07-21T07:53:36.606Z|00019|coverage|INFO|txn_incomplete
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>>> 2016-07-21T07:53:36.607Z|00020|coverage|INFO|poll_create_node
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 35
>>>>>>>>> 2016-07-21T07:53:36.607Z|00021|coverage|INFO|seq_change
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 38
>>>>>>>>> 2016-07-21T07:53:36.607Z|00022|coverage|INFO|pstream_open
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>>> 2016-07-21T07:53:36.607Z|00023|coverage|INFO|stream_open
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>>> 2016-07-21T07:53:36.607Z|00024|coverage|INFO|util_xalloc
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 11049
>>>>>>>>> 2016-07-21T07:53:36.607Z|00025|coverage|INFO|netdev_get_hwaddr
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>>> 2016-07-21T07:53:36.607Z|00026|coverage|INFO|netlink_received
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>>>>>>> 2016-07-21T07:53:36.607Z|00027|coverage|INFO|netlink_sent
>>>>>>>>>    0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>>>>> 2016-07-21T07:53:36.607Z|00028|coverage|INFO|86 events never hit
>>>>>>>>> 2016-07-21T07:53:36.607Z|00029|bridge|INFO|ovs-vswitchd (Open
>>>>>>>>> vSwitch)
>>>>>>>>> 2.5.90
>>>>>>>>> 2016-07-21T07:54:50.861Z|00030|memory|INFO|18980 kB peak resident set
>>>>>>>>> size after 75.3 seconds
>>>>>>>>> 2016-07-21T07:54:50.866Z|00031|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> Datapath supports recirculation
>>>>>>>>> 2016-07-21T07:54:50.866Z|00032|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> MPLS label stack length probed as 3
>>>>>>>>> 2016-07-21T07:54:50.866Z|00033|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> Datapath supports truncate action
>>>>>>>>> 2016-07-21T07:54:50.866Z|00034|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> Datapath supports unique flow ids
>>>>>>>>> 2016-07-21T07:54:50.866Z|00035|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> Datapath does not support ct_state
>>>>>>>>> 2016-07-21T07:54:50.866Z|00036|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> Datapath does not support ct_zone
>>>>>>>>> 2016-07-21T07:54:50.866Z|00037|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> Datapath does not support ct_mark
>>>>>>>>> 2016-07-21T07:54:50.866Z|00038|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> Datapath does not support ct_label
>>>>>>>>> 2016-07-21T07:54:50.866Z|00039|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>>>>> Datapath does not support ct_state_nat
>>>>>>>>> 2016-07-21T07:54:50.875Z|00040|bridge|INFO|bridge br0: added
>>>>>>>>> interface
>>>>>>>>> br0 on port 65534
>>>>>>>>> 2016-07-21T07:54:50.887Z|00041|dpif_netlink|WARN|Generic Netlink
>>>>>>>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module
>>>>>>>>> is probably not loaded.
>>>>>>>>> 2016-07-21T07:54:50.887Z|00042|bridge|INFO|bridge br0: using datapath
>>>>>>>>> ID 0000c664fbd86348
>>>>>>>>> 2016-07-21T07:54:50.887Z|00043|connmgr|INFO|br0: added service
>>>>>>>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>>>>>>>> 2016-07-21T07:58:18.796Z|00044|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>>>>>>> 2016-07-21T07:58:18.810Z|00045|dpdk|ERR|Interface dpdk0 start error:
>>>>>>>>> Operation not permitted
>>>>>>>>> 2016-07-21T07:58:18.810Z|00046|dpif|WARN|netdev@ovs-netdev: failed to
>>>>>>>>> add dpdk0 as port: Operation not permitted
>>>>>>>>> 2016-07-21T08:10:35.440Z|00047|connmgr|INFO|br0<->unix: 1 flow_mods
>>>>>>>>> in
>>>>>>>>> the last 0 s (1 adds)
>>>>>>>
>>>>>>> Could you also provide the full terminal output when you run
>>>>>>> ovs-vswitchd?,
>>>>>>> I would like to see the DPDK output
>>>>>>
>>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vswitchd unix:$DB_SOCK
>>>>>> --pidfile --detach --log-file=/var/log/openvswitch/ovs-vswitchd.log
>>>>>> 2016-07-21T09:33:35Z|00001|vlog|INFO|opened log file
>>>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>>>> 2016-07-21T09:33:35Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA
>>>>>> node 0
>>>>>> 2016-07-21T09:33:35Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2
>>>>>> CPU
>>>>>> cores
>>>>>>
>>>>>>
>>>>>> 2016-07-21T09:33:35Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>>> connecting...
>>>>>>
>>>>>>
>>>>>> 2016-07-21T09:33:35Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>>>> connected
>>>>>> 2016-07-21T09:33:35Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>>>>> 2016-07-21T09:33:35Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>>>>> defaulting to /usr/local/var/run/openvswitch
>>>>>> 2016-07-21T09:33:35Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>>>> --socket-mem 1024,0 -c 0x00000001
>>>>>> EAL: Detected lcore 0 as core 0 on socket 0
>>>>>> EAL: Detected lcore 1 as core 0 on socket 0
>>>>>> EAL: Support maximum 128 logical core(s) by configuration.
>>>>>> EAL: Detected 2 lcore(s)
>>>>>> EAL: Probing VFIO support...
>>>>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>>>>>> directory)
>>>>>> EAL: VFIO modules not loaded, skipping VFIO support...
>>>>>> EAL: Setting up physically contiguous memory...
>>>>>> EAL: Ask a virtual area of 0x5e400000 bytes
>>>>>> EAL: Virtual area found at 0x7fa618400000 (size = 0x5e400000)
>>>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>>>> EAL: Virtual area found at 0x7fa618000000 (size = 0x200000)
>>>>>> EAL: Ask a virtual area of 0xa1800000 bytes
>>>>>> EAL: Virtual area found at 0x7fa576600000 (size = 0xa1800000)
>>>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>>>> EAL: Virtual area found at 0x7fa576200000 (size = 0x200000)
>>>>>> EAL: Requesting 512 pages of size 2MB from socket 0
>>>>>> EAL: TSC frequency is ~2799213 KHz
>>>>>> EAL: Master lcore 0 is ready (tid=78ad7b80;cpuset=[0])
>>>>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1
>>>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>>>> EAL:   Not managed by a supported kernel driver, skipped
>>>>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
>>>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>>>> EAL:   PCI memory mapped at 0x7fa658400000
>>>>>> EAL:   PCI memory mapped at 0x7fa658401000
>>>>>> EAL:   PCI memory mapped at 0x7fa658402000
>>>>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1
>>>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>>>> EAL:   PCI memory mapped at 0x7fa658404000
>>>>>> EAL:   PCI memory mapped at 0x7fa658405000
>>>>>> EAL:   PCI memory mapped at 0x7fa658406000
>>>>>> Zone 0: name:<RG_MP_log_history>, phys:0x82bfdec0, len:0x2080,
>>>>>> virt:0x7fa6583fdec0, socket_id:0, flags:0
>>>>>> Zone 1: name:<MP_log_history>, phys:0x82973d40, len:0x28a0c0,
>>>>>> virt:0x7fa658173d40, socket_id:0, flags:0
>>>>>> Zone 2: name:<rte_eth_dev_data>, phys:0x82943200, len:0x2f800,
>>>>>> virt:0x7fa658143200, socket_id:0, flags:0
>>>>>> 2016-07-21T09:33:36Z|00009|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>> Datapath supports recirculation
>>>>>> 2016-07-21T09:33:36Z|00010|ofproto_dpif|INFO|netdev@ovs-netdev: MPLS
>>>>>> label stack length probed as 3
>>>>>> 2016-07-21T09:33:36Z|00011|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>> Datapath supports truncate action
>>>>>> 2016-07-21T09:33:36Z|00012|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>> Datapath supports unique flow ids
>>>>>> 2016-07-21T09:33:36Z|00013|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>> Datapath does not support ct_state
>>>>>> 2016-07-21T09:33:36Z|00014|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>> Datapath does not support ct_zone
>>>>>> 2016-07-21T09:33:36Z|00015|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>> Datapath does not support ct_mark
>>>>>> 2016-07-21T09:33:36Z|00016|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>> Datapath does not support ct_label
>>>>>> 2016-07-21T09:33:36Z|00017|ofproto_dpif|INFO|netdev@ovs-netdev:
>>>>>> Datapath does not support ct_state_nat
>>>>>> 2016-07-21T09:33:36Z|00018|bridge|INFO|bridge br0: added interface br0
>>>>>> on port 65534
>>>>>> 2016-07-21T09:33:36Z|00019|dpif_netlink|WARN|Generic Netlink family
>>>>>> 'ovs_datapath' does not exist. The Open vSwitch kernel module is
>>>>>> probably not loaded.
>>>>>> 2016-07-21T09:33:36Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>>>> 2016-07-21T09:33:36Z|00021|dpdk|ERR|Interface dpdk0 start error:
>>>>>> Operation not permitted
>>>>>> 2016-07-21T09:33:36Z|00022|bridge|INFO|bridge br0: using datapath ID
>>>>>> 0000c664fbd86348
>>>>>> 2016-07-21T09:33:36Z|00023|connmgr|INFO|br0: added service controller
>>>>>> "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>>>>> 2016-07-21T09:33:36Z|00024|timeval|WARN|Unreasonably long 1244ms poll
>>>>>> interval (233ms user, 880ms system)
>>>>>> 2016-07-21T09:33:36Z|00025|timeval|WARN|faults: 3902 minor, 0 major
>>>>>> 2016-07-21T09:33:36Z|00026|timeval|WARN|context switches: 10
>>>>>> voluntary, 214 involuntary
>>>>>> 2016-07-21T09:33:36Z|00027|coverage|INFO|Event coverage, avg rate over
>>>>>> last: 5 seconds, last minute, last hour,  hash=75cfd2fc:
>>>>>> 2016-07-21T09:33:36Z|00028|coverage|INFO|bridge_reconfigure
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00029|coverage|INFO|ofproto_flush
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00030|coverage|INFO|ofproto_update_port
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00031|coverage|INFO|rev_flow_table
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00032|coverage|INFO|cmap_expand
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 22
>>>>>> 2016-07-21T09:33:36Z|00033|coverage|INFO|cmap_shrink
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 7
>>>>>> 2016-07-21T09:33:36Z|00034|coverage|INFO|dpif_port_add
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>>> 2016-07-21T09:33:36Z|00035|coverage|INFO|dpif_flow_flush
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>>> 2016-07-21T09:33:36Z|00036|coverage|INFO|dpif_flow_get
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>>>> 2016-07-21T09:33:36Z|00037|coverage|INFO|dpif_flow_put
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 10
>>>>>> 2016-07-21T09:33:36Z|00038|coverage|INFO|dpif_flow_del
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>>>> 2016-07-21T09:33:36Z|00039|coverage|INFO|dpif_execute
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>>>> 2016-07-21T09:33:36Z|00040|coverage|INFO|flow_extract
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 4
>>>>>> 2016-07-21T09:33:36Z|00041|coverage|INFO|miniflow_malloc
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 16
>>>>>> 2016-07-21T09:33:36Z|00042|coverage|INFO|hmap_pathological
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00043|coverage|INFO|hmap_expand
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 653
>>>>>> 2016-07-21T09:33:36Z|00044|coverage|INFO|netdev_get_stats
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00045|coverage|INFO|poll_create_node
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 15
>>>>>> 2016-07-21T09:33:36Z|00046|coverage|INFO|poll_zero_timeout
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00047|coverage|INFO|seq_change
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 575
>>>>>> 2016-07-21T09:33:36Z|00048|coverage|INFO|pstream_open
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 3
>>>>>> 2016-07-21T09:33:36Z|00049|coverage|INFO|stream_open
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00050|coverage|INFO|util_xalloc
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 11820
>>>>>> 2016-07-21T09:33:36Z|00051|coverage|INFO|netdev_set_policing
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00052|coverage|INFO|netdev_get_ifindex
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00053|coverage|INFO|netdev_get_hwaddr
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>>> 2016-07-21T09:33:36Z|00054|coverage|INFO|netdev_set_hwaddr
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00055|coverage|INFO|netdev_get_ethtool
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 2
>>>>>> 2016-07-21T09:33:36Z|00056|coverage|INFO|netlink_received
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 6
>>>>>> 2016-07-21T09:33:36Z|00057|coverage|INFO|netlink_recv_jumbo
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 1
>>>>>> 2016-07-21T09:33:36Z|00058|coverage|INFO|netlink_sent
>>>>>> 0.0/sec     0.000/sec        0.0000/sec   total: 5
>>>>>> 2016-07-21T09:33:36Z|00059|coverage|INFO|70 events never hit
>>>>>> 2016-07-21T09:33:36Z|00060|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>>>> 2016-07-21T09:33:36Z|00061|dpdk|ERR|Interface dpdk0 start error:
>>>>>> Operation not permitted
>>>>>
>>>>> The PMD start function is failing, unfortunately that error does not tell
>>>>> that much information.
>>>>>
>>>>> I would suggest you to enable the debug of the vmxnet3 PMD in DPDK, set
>>>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y in the config/common_base file of
>>>>> DPDK, then recompile DPDK and OvS.
>>>>>
>>>>> Please try once again and send the terminal output, I hope we can get a
>>>>> more
>>>>> information of what is going on with that log.
>>>>
>>>> Based on your request, I reproduce the log with
>>>> CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y
>>>> sudo ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock
>>>> --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile --detach
>>>> 2016-07-25T09:24:17Z|00001|vlog|INFO|opened log file
>>>> /var/log/openvswitch/ovs-vswitchd.log
>>>> 2016-07-25T09:24:17Z|00002|ovs_numa|INFO|Discovered 2 CPU cores on NUMA
>>>> node 0
>>>> 2016-07-25T09:24:17Z|00003|ovs_numa|INFO|Discovered 1 NUMA nodes and 2 CPU
>>>> cores
>>>>
>>>> 2016-07-25T09:24:17Z|00004|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>> connecting...
>>>>
>>>> 2016-07-25T09:24:17Z|00005|reconnect|INFO|unix:/usr/local/var/run/openvswitch/db.sock:
>>>> connected
>>>> 2016-07-25T09:24:17Z|00006|dpdk|INFO|DPDK Enabled, initializing
>>>> 2016-07-25T09:24:17Z|00007|dpdk|INFO|No vhost-sock-dir provided -
>>>> defaulting to /usr/local/var/run/openvswitch
>>>> 2016-07-25T09:24:17Z|00008|dpdk|INFO|EAL ARGS: ovs-vswitchd
>>>> --socket-mem 1024,0 -c 0x00000001
>>>> EAL: Detected lcore 0 as core 0 on socket 0
>>>> EAL: Detected lcore 1 as core 0 on socket 0
>>>> EAL: Support maximum 128 logical core(s) by configuration.
>>>> EAL: Detected 2 lcore(s)
>>>> EAL: Probing VFIO support...
>>>> EAL: Module /sys/module/vfio_pci not found! error 2 (No such file or
>>>> directory)
>>>> EAL: VFIO modules not loaded, skipping VFIO support...
>>>> EAL: Setting up physically contiguous memory...
>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>> EAL: Virtual area found at 0x7fdea6000000 (size = 0x200000)
>>>> EAL: Ask a virtual area of 0x5e000000 bytes
>>>> EAL: Virtual area found at 0x7fde47e00000 (size = 0x5e000000)
>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>> EAL: Virtual area found at 0x7fde47a00000 (size = 0x200000)
>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>> EAL: Virtual area found at 0x7fde47600000 (size = 0x200000)
>>>> EAL: Ask a virtual area of 0xa1800000 bytes
>>>> EAL: Virtual area found at 0x7fdda5c00000 (size = 0xa1800000)
>>>> EAL: Ask a virtual area of 0x200000 bytes
>>>> EAL: Virtual area found at 0x7fdda5800000 (size = 0x200000)
>>>> EAL: Requesting 512 pages of size 2MB from socket 0
>>>> EAL: TSC frequency is ~2799209 KHz
>>>> EAL: Master lcore 0 is ready (tid=a855eb80;cpuset=[0])
>>>> PMD: rte_vmxnet3_pmd_init():  >>
>>>> EAL: PCI device 0000:03:00.0 on NUMA socket -1
>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>> EAL:   Not managed by a supported kernel driver, skipped
>>>> EAL: PCI device 0000:0b:00.0 on NUMA socket -1
>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>> EAL:   PCI memory mapped at 0x7fdea6200000
>>>> EAL:   PCI memory mapped at 0x7fdea6201000
>>>> EAL:   PCI memory mapped at 0x7fdea6202000
>>>> PMD: eth_vmxnet3_dev_init():  >>
>>>> PMD: eth_vmxnet3_dev_init(): Hardware version : 1
>>>> PMD: eth_vmxnet3_dev_init(): UPT hardware version : 1
>>>> PMD: eth_vmxnet3_dev_init(): MAC Address : 00:0c:29:b5:9f:5b
>>>> EAL: PCI device 0000:13:00.0 on NUMA socket -1
>>>> EAL:   probe driver: 15ad:7b0 rte_vmxnet3_pmd
>>>> EAL:   Not managed by a supported kernel driver, skipped
>>>> Zone 0: name:<RG_MP_log_history>, phys:0x555fdec0, len:0x2080,
>>>> virt:0x7fdea61fdec0, socket_id:0, flags:0
>>>> Zone 1: name:<MP_log_history>, phys:0x95375f00, len:0x28a0c0,
>>>> virt:0x7fde87975f00, socket_id:0, flags:0
>>>> Zone 2: name:<rte_eth_dev_data>, phys:0x555cd300, len:0x2f800,
>>>> virt:0x7fdea61cd300, socket_id:0, flags:0
>>>>
>>>> PMD: vmxnet3_dev_configure():  >>
>>>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>>>> PMD: vmxnet3_dev_rx_queue_setup():  >>
>>>> PMD: vmxnet3_dev_start():  >>
>>>> PMD: vmxnet3_rss_configure():  >>
>>>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b
>>>> PMD: vmxnet3_disable_intr():  >>
>>>> PMD: vmxnet3_dev_rxtx_init():  >>
>>>> 2016-07-25T09:40:23Z|00020|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>> PMD: vmxnet3_dev_stop():  >>
>>>> PMD: vmxnet3_disable_intr():  >>
>>>> PMD: vmxnet3_dev_stop(): Device reset.
>>>> PMD: vmxnet3_dev_clear_queues():  >>
>>>> PMD: vmxnet3_dev_configure():  >>
>>>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>>>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>>>> PMD: vmxnet3_dev_tx_queue_setup():  >>
>>>> PMD: vmxnet3_dev_rx_queue_setup():  >>
>>>> PMD: vmxnet3_dev_start():  >>
>>>> PMD: vmxnet3_rss_configure():  >>
>>>> PMD: vmxnet3_write_mac(): Writing MAC Address : 00:0c:29:b5:9f:5b
>>>> PMD: vmxnet3_dev_start(): Device activation: UNSUCCESSFUL
>>>
>>>
>>> Unfortunately it is the most I can help you, it is strange for me why
>>> test-pmd is working and why OvS  is not.
>>> I think it is a DPDK issue, then the next step is to post on the DPDK
>>> mailing (dev@dpdk.org, add also in CC Yong Wang <yongwang@vmware.com> that
>>> is the vmxnet3 maintainer), send a link to this thread in order to be clear
>>> what you have tested.
>>
>>
>> Hello Yong Wang,
>>
>> I am running fedora VM in ESXi 5.5.0 environment. I tried to install
>> OVS and DPDK. OVS couldnt start DPDK device.
>> You can see the error in the mail below. Do you know why? IF you need
>> more information. I can provide more information.
>>
>> Good news, when I tried in Fedora 23 VM in ESXi 5.5.0, it works. It is
>> possible to bind an dpdk device and get packets on port.
>>
>> Thanks Mauricio for your help and reply.
>>
>> Best regards
>> Sothy
>
> Sothy,
>

Hello,
> You mentioned that on Fedora 23, it works.  Then on some version Fedora that it failed?

Initially, I started working on Ubuntu 14.04. It produce error.
Therefore, I changed in Fedora. I worked directly on Fedora 23.
For fedora, I only tried on Fedora 23. It works. I didnt tried on
other version.  All Vms are in ESX 5.5. The problem is in Ubuntu vm
ONLY,

> From your previous post, you mentioned that it’s a Ubuntu 14.04 VM?  Can you clarify?  Other than the changes you >mentioned already, are there any other changes (such as ring buffer size, number of rx queues, etc) that have been >changed?
I didnt do any changes.  I just followed basic commands given in OVDK
install.md  and
 >Is there any updates installed on your ESX 5.5?

no
>If yes, what’s the exact update version?  Can you also send me a copy of your vmkernel.log offline?

I will send you offline.
>
> Also copying Daniele as he is trying to help reproduce the reported issue.
>
>>
>>>
>>>> I deleted exisiting bridge and recreated::::::
>>>>
>>>>
>>>> Then I added bridge. After I try to create bridge. It gives error. The
>>>> erros is same as shown in log file.
>>>>
>>>> 2016-07-25T09:29:43.248Z|00020|bridge|INFO|bridge br0: added interface
>>>> br0 on port 65534
>>>> 2016-07-25T09:29:43.253Z|00021|dpif_netlink|WARN|Generic Netlink
>>>> family 'ovs_datapath' does not exist. The Open vSwitch kernel module
>>>> is probably not loaded.
>>>> 2016-07-25T09:29:43.253Z|00022|bridge|INFO|bridge br0: using datapath
>>>> ID 0000e6aac740584b
>>>> 2016-07-25T09:29:43.253Z|00023|connmgr|INFO|br0: added service
>>>> controller "punix:/usr/local/var/run/openvswitch/br0.mgmt"
>>>> 2016-07-25T09:30:54.008Z|00024|dpdk|INFO|Port 0: 00:0c:29:b5:9f:5b
>>>> 2016-07-25T09:30:54.021Z|00025|dpdk|ERR|Interface dpdk0 start error:
>>>> Operation not permitted
>>>> 2016-07-25T09:30:54.021Z|00026|dpif|WARN|netdev@ovs-netdev: failed to
>>>> add dpdk0 as port: Operation not permitted
>>>>
>>>>
>>>>
>>>>> Another thing to do would be to bind the NICs to the uio_pci_generic and
>>>>> VFIO modules, just to be sure that the issue is also present there.
>>>>
>>>> I tried in uio and uio_pci_genric also. Both are giving same error.
>>>> I suspect I cannt activate VFIO in VM since it is BIOS suport also. AM I
>>>> right?
>>>>
>>>> Thanks
>>>>
>>>>>
>>>>>> in command line some information for your verification:
>>>>>>
>>>>>>   sudo ovs-vsctl show
>>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2
>>>>>>      Bridge "br0"
>>>>>>          Port "dpdk0"
>>>>>>              Interface "dpdk0"
>>>>>>                  type: dpdk
>>>>>>          Port "br0"
>>>>>>              Interface "br0"
>>>>>>                  type: internal
>>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl del-br br0
>>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-br br0 --
>>>>>> set bridge br0 datapath_type=netdev
>>>>>> vrouter@vrouter-virtual-machine:~/ovs$ sudo ovs-vsctl add-port br0
>>>>>> dpdk0 -- set Interface dpdk0 type=dpdk
>>>>>> ovs-vsctl: Error detected while setting up 'dpdk0'.  See ovs-vswitchd
>>>>>> log for details.
>>>>>>
>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>> sudo ovs-vsctl show
>>>>>>>>> d53612e0-5d82-4cc1-af7d-ee887a737ef2
>>>>>>>>>       Bridge "br0"
>>>>>>>>>           Port "dpdk0"
>>>>>>>>>               Interface "dpdk0"
>>>>>>>>>                   type: dpdk
>>>>>>>>>           Port "br0"
>>>>>>>>>               Interface "br0"
>>>>>>>>>                   type: internal
>>>>>>>>>
>>>>>>>>> sudo ovs-ofctl dump-flows br0
>>>>>>>>> NXST_FLOW reply (xid=0x4):
>>>>>>>>>    cookie=0x0, duration=487.638s, table=0, n_packets=0, n_bytes=0,
>>>>>>>>> idle_age=487, priority=0 actions=NORMAL
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> In addition I would recommend to use the test-pmd [1] application
>>>>>>>>>> from
>>>>>>>>>> DPDK
>>>>>>>>>> to check if the issue is also present in that case.
>>>>>>>>
>>>>>>>> I run testpmd command.The result is
>>>>>>>>
>>>>>>>> Configuring Port 0 (socket 0)
>>>>>>>> Port 0: 00:0C:29:B5:9F:5B
>>>>>>>> Configuring Port 1 (socket 0)
>>>>>>>> Port 1: 00:0C:29:B5:9F:65
>>>>>>>> Checking link statuses...
>>>>>>>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>>>>>>>> Port 1 Link Up - speed 10000 Mbps - full-duplex
>>>>>>>> Done
>>>>>>>> No commandline core given, start packet forwarding
>>>>>>>>     io packet forwarding - CRC stripping disabled - packets/burst=32
>>>>>>>>     nb forwarding cores=1 - nb forwarding ports=2
>>>>>>>>     RX queues=1 - RX desc=128 - RX free threshold=0
>>>>>>>>     RX threshold registers: pthresh=0 hthresh=0 wthresh=0
>>>>>>>>     TX queues=1 - TX desc=512 - TX free threshold=0
>>>>>>>>     TX threshold registers: pthresh=0 hthresh=0 wthresh=0
>>>>>>>>     TX RS bit threshold=0 - TXQ flags=0x200
>>>>>>>> Press enter to exit
>>>>>>>>
>>>>>>>> Telling cores to stop...
>>>>>>>> Waiting for lcores to finish...
>>>>>>>>
>>>>>>>>     ---------------------- Forward statistics for port 0
>>>>>>>> ----------------------
>>>>>>>>     RX-packets: 88             RX-dropped: 0             RX-total: 88
>>>>>>>>     TX-packets: 2              TX-dropped: 0             TX-total: 2
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>     ---------------------- Forward statistics for port 1
>>>>>>>> ----------------------
>>>>>>>>     RX-packets: 2              RX-dropped: 0             RX-total: 2
>>>>>>>>     TX-packets: 88             TX-dropped: 0             TX-total: 88
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ----------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>     +++++++++++++++ Accumulated forward statistics for all
>>>>>>>> ports+++++++++++++++
>>>>>>>>     RX-packets: 90             RX-dropped: 0             RX-total: 90
>>>>>>>>     TX-packets: 90             TX-dropped: 0             TX-total: 90
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>>>>>>
>>>>>>>> make sense?
>>>>>>>
>>>>>>>
>>>>>>> Yes, it makes sense. Ports are being recognized by DPDK and some
>>>>>>> packets
>>>>>>> were exchanged.
>>>>>>>
>>>>>>>
>>>>>>>>>> [1] https://urldefense.proofpoint.com/v2/url?u=http-3A__dpdk.org_doc_guides-2D16.04_testpmd-5Fapp-5Fug_index.html&d=CwIBaQ&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=44mSO5N5yEs4CeCdtQE0xt0F7J0p67_mApYVAzyYms0&m=vpaHcL64W3vOTY9P8yRZOdCJzbL_oWzZH6J2u4iT614&s=JnLeHoyKuiecYI95-fcfFiuev5g2CDweT2MnyUiKWzM&e=
>>>>>>>>>>
>>>>>>>>>> Mauricio Vasquez,
>>>>>>>
>>>>>>>
>>>>> Mauricio V,
>>>
>>>
>