Re: [dpdk-dev] DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

Re: [dpdk-dev] DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

[Karmarkar Suyash]

I was looking at older thread here it mentions the kernel vmxnet3 pmd dependency with Vmware tsc map, Is this the reason why the CISCO NIC is coming up fine on ESX 6.0 which will have higher version/updated driver and ESX 5.5 has older version of the driver and because of incompatability the NIC is not coming up and initialization fails? Thanks.
http://dpdk.org/ml/archives/dev/2013-July/000302.html


From: Karmarkar Suyash
Sent: Thursday, November 19, 2015 9:03 AM
To: 'dev@dpdk.org' <dev@dpdk.org>
Subject: DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

Hello All,

I am trying some experiments with DPDK 1.6 and Cisco NIC - (0000:06:00.0 Network controller: Cisco Systems Inc. Cisco VIC Ethernet NIC [vmnic0]), on the guest I am using VMXNET3 as interface/pmd and one weird behavior that is see is with VMWare ESX 5.5 DPDK is not able to initialize the NIC and I get the error rte_eth_dev_start failed with error code as, but same NIC and with DPDK 1.6 version works fine with VMWare ESX 6.0 and with VMXNET3 PMD.

Two questions:

1.     Is there any dependency with VMWare version ESX (5.0/5.1/6.0) with DPDK Release version (1.6/1.7/2.x)? If yes can you please let me know?

2.     If we are not using the NIC in pass-through mode and we are using it via Hypervisor - vmxnet3 or virtio pmd is there still a dependency for the list of NIC's supported by DPDK?

Regards
Suyash

[dpdk-dev] DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

[Karmarkar Suyash]

Hello All,

I am trying some experiments with DPDK 1.6 and Cisco NIC - (0000:06:00.0 Network controller: Cisco Systems Inc. Cisco VIC Ethernet NIC [vmnic0]), on the guest I am using VMXNET3 as interface/pmd and one weird behavior that is see is with VMWare ESX 5.5 DPDK is not able to initialize the NIC and I get the error rte_eth_dev_start failed with error code as, but same NIC and with DPDK 1.6 version works fine with VMWare ESX 6.0 and with VMXNET3 PMD.

Two questions:

1.     Is there any dependency with VMWare version ESX (5.0/5.1/6.0) with DPDK Release version (1.6/1.7/2.x)? If yes can you please let me know?

2.     If we are not using the NIC in pass-through mode and we are using it via Hypervisor - vmxnet3 or virtio pmd is there still a dependency for the list of NIC's supported by DPDK?

Regards
Suyash

Re: [dpdk-dev] DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

[Patel, Rashmin N]

What is the error code it failed with?

There is no dependency between ESXi and DPDK versions as such. But depending your VM settings, it may pop up some issues related to memory assigned to VM.

When you use NIC in Hypervisor and feed packets to VM via Vmxnet3, the guest OS can use Vmxnet3-pmd and that's the sole requirement. Guest OS or DPDK API in VM are unaware and independent of the physical NIC beneath.
When you use NIC in VM via Passthrough (or device direct assignment,) the guest OS can use ixgbe-pmd/igb-pmd/xyz-pmd to control them for Rx/Tx operations. ESXi can't look into such device operations like it can't see packets coming in or going out.
When you enable SRIOV on a NIC, you get a PF and multiple VFs and if you use a NIC-Virtual Function via passthrough, the guest OS can use respective pmd (ixgbevf/igbvf/xyzvf) in DPDK API to control it for Rx/Tx ops. ESXi control the Physical Function through a respective driver (ixgbe/igb/xyz provided with ESXi) DPDK API doesn't have support to run in ESXi to control that PF.

Thanks,
Rashmin

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Karmarkar Suyash
Sent: Wednesday, November 18, 2015 8:34 PM
To: dev@dpdk.org
Subject: [dpdk-dev] DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

Hello All,

I am trying some experiments with DPDK 1.6 and Cisco NIC - (0000:06:00.0 Network controller: Cisco Systems Inc. Cisco VIC Ethernet NIC [vmnic0]), on the guest I am using VMXNET3 as interface/pmd and one weird behavior that is see is with VMWare ESX 5.5 DPDK is not able to initialize the NIC and I get the error rte_eth_dev_start failed with error code as, but same NIC and with DPDK 1.6 version works fine with VMWare ESX 6.0 and with VMXNET3 PMD.

Two questions:

1.     Is there any dependency with VMWare version ESX (5.0/5.1/6.0) with DPDK Release version (1.6/1.7/2.x)? If yes can you please let me know?

2.     If we are not using the NIC in pass-through mode and we are using it via Hypervisor - vmxnet3 or virtio pmd is there still a dependency for the list of NIC's supported by DPDK?

Regards
Suyash

Re: [dpdk-dev] DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

[Karmarkar Suyash]

Thanks Rashmin for the response. The error code is -1 and the box I am doing my experiments is Cisco B200-M3 which would support CISCO VNIC 1240 or 1340 NIC.  The NIC is in hypervisor but what puzzles me is if there is no dependency with ESX the NIC initialization should have passed in both ESX versions 5.5 and 6.0. 

Error is rte_eth_dev_start error = -1 

Regards
Suyash

-----Original Message-----
From: Patel, Rashmin N [mailto:rashmin.n.patel@intel.com] 
Sent: Thursday, November 19, 2015 11:50 PM
To: Karmarkar Suyash <skarmarkar@sonusnet.com>; dev@dpdk.org
Subject: RE: DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

What is the error code it failed with?

There is no dependency between ESXi and DPDK versions as such. But depending your VM settings, it may pop up some issues related to memory assigned to VM.

When you use NIC in Hypervisor and feed packets to VM via Vmxnet3, the guest OS can use Vmxnet3-pmd and that's the sole requirement. Guest OS or DPDK API in VM are unaware and independent of the physical NIC beneath.
When you use NIC in VM via Passthrough (or device direct assignment,) the guest OS can use ixgbe-pmd/igb-pmd/xyz-pmd to control them for Rx/Tx operations. ESXi can't look into such device operations like it can't see packets coming in or going out.
When you enable SRIOV on a NIC, you get a PF and multiple VFs and if you use a NIC-Virtual Function via passthrough, the guest OS can use respective pmd (ixgbevf/igbvf/xyzvf) in DPDK API to control it for Rx/Tx ops. ESXi control the Physical Function through a respective driver (ixgbe/igb/xyz provided with ESXi) DPDK API doesn't have support to run in ESXi to control that PF.

Thanks,
Rashmin

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Karmarkar Suyash
Sent: Wednesday, November 18, 2015 8:34 PM
To: dev@dpdk.org
Subject: [dpdk-dev] DPDK 1.6 and Cisco VIC-NIC pmd fails to initialize with VMWare ESX 5.5

Hello All,

I am trying some experiments with DPDK 1.6 and Cisco NIC - (0000:06:00.0 Network controller: Cisco Systems Inc. Cisco VIC Ethernet NIC [vmnic0]), on the guest I am using VMXNET3 as interface/pmd and one weird behavior that is see is with VMWare ESX 5.5 DPDK is not able to initialize the NIC and I get the error rte_eth_dev_start failed with error code as, but same NIC and with DPDK 1.6 version works fine with VMWare ESX 6.0 and with VMXNET3 PMD.

Two questions:

1.     Is there any dependency with VMWare version ESX (5.0/5.1/6.0) with DPDK Release version (1.6/1.7/2.x)? If yes can you please let me know?

2.     If we are not using the NIC in pass-through mode and we are using it via Hypervisor - vmxnet3 or virtio pmd is there still a dependency for the list of NIC's supported by DPDK?

Regards
Suyash