* Re: [dpdk-dev] rte_eth_dev_count() returns 0
2013-05-10 1:18 ` Henry Wang
@ 2013-05-10 16:05 ` Bhavesh Davda
2013-05-10 16:15 ` Stephen Hemminger
2013-05-18 0:09 ` Rui Vaz (ruvaz)
1 sibling, 1 reply; 11+ messages in thread
From: Bhavesh Davda @ 2013-05-10 16:05 UTC (permalink / raw)
To: Henry Wang; +Cc: dev
[-- Attachment #1: Type: text/plain, Size: 2715 bytes --]
ESXi 5.0 doesn't have SR-IOV support (it's in the code, but is disabled and not usable for customers). VMware introduced SR-IOV support, including out of the box support for Intel Niantic 10GbE NICs, in ESXi 5.1.
--
Bhavesh Davda
----- Original Message -----
> From: "Henry Wang" <wang_hr@neusoft.com>
> To: "Rui Vaz (ruvaz)" <ruvaz@cisco.com>
> Cc: dev@dpdk.org
> Sent: Thursday, May 9, 2013 6:18:04 PM
> Subject: Re: [dpdk-dev] rte_eth_dev_count() returns 0
> As far as I known, DPDK(1.2.3) only supports the following NICs:
> (1) Intel 82576, 82580, I350 chipset based NICs;
> (2) Intel 82599, X540T, X520T chipset based NICs;
> (3) virtual NICs generated by SR-IOV from (2).
> If the NICs on your C220 M3 is one of (2), you can run l2fwdvf OR l3fwdvf
> successfully with SR-IOV enabled in ESXi.
> And I do not know wheather ESXi 5.0 VMM has SR-IOV function or not.
> Further more, please reference to DPDK release notes.
> http://www.intel.com/content/dam/www/public/us/en/documents/release-notes/intel-dpdk-release-notes.pdf
> On Fri, May 10, 2013 at 9:00 AM, Rui Vaz (ruvaz) < ruvaz@cisco.com > wrote:
> > Hi dev-dpdk,
>
> > I am following a Intel DPDK Step by Step Instruction found here:
> > http://www.slideshare.net/hisaki/intel-dpdk-step-by-step-instructions
>
> > My Hello app from the DPDK examples runs fine, but I get the following
> > error
> > when running the l2fwd app:
>
> > “EAL: Error – exiting with code: 1
>
> > Cause: no Ethernet ports – bye”
>
> > The rte_eth_dev_count() call returns 0, I am using a CentOS 6.3 Virtual
> > Machine on VMWare esxi 5.0 on a Cisco UCS C220 M3 which comes with Xeon
> > E5-2600.
>
> > I would really appreciate If you could provide some help to solve this
> > issue.
>
> > Thank you,
>
> > Rui
>
> ---------------------------------------------------------------------------------------------------
> Confidentiality Notice: The information contained in this e-mail and any
> accompanying attachment(s)
> is intended only for the use of the intended recipient and may be
> confidential and/or privileged of
> Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of
> this communication is
> not the intended recipient, unauthorized use, forwarding, printing, storing,
> disclosure or copying
> is strictly prohibited, and may be unlawful.If you have received this
> communication in error,please
> immediately notify the sender by return e-mail, and delete the original
> message and all copies from
> your system. Thank you.
> ---------------------------------------------------------------------------------------------------
[-- Attachment #2: Type: text/html, Size: 7520 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] rte_eth_dev_count() returns 0
2013-05-10 16:05 ` Bhavesh Davda
@ 2013-05-10 16:15 ` Stephen Hemminger
2013-05-10 16:21 ` Bhavesh Davda
0 siblings, 1 reply; 11+ messages in thread
From: Stephen Hemminger @ 2013-05-10 16:15 UTC (permalink / raw)
To: Bhavesh Davda; +Cc: Henry Wang, dev
On Fri, 10 May 2013 09:05:08 -0700 (PDT)
Bhavesh Davda <bhavesh@vmware.com> wrote:
> ESXi 5.0 doesn't have SR-IOV support (it's in the code, but is disabled and not usable for customers). VMware introduced SR-IOV support, including out of the box support for Intel Niantic 10GbE NICs, in ESXi 5.1.
>
You can use PCI passthrough (Directpath) on 5.0 (and 5.1).
I found that SR-IOV on Vmware is even more fussy than SR-IOV on Linux.
It requires a supported platform with VT-d and PCIe ASPM.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] rte_eth_dev_count() returns 0
2013-05-10 16:15 ` Stephen Hemminger
@ 2013-05-10 16:21 ` Bhavesh Davda
2013-05-10 18:44 ` Stephen Hemminger
0 siblings, 1 reply; 11+ messages in thread
From: Bhavesh Davda @ 2013-05-10 16:21 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Henry Wang, dev
Right: VMware has supported VMDirectPath passthrough since ESXi 4.0, on any platform with an IOMMU (Intel VT-d) which is anything Nehalem Xeons or newer.
I know this is a DPDK forum, but Stephen, if you know of specific limitations of ESXi 5.1's SR-IOV support, please let me know and I'll make sure we work on addressing them.
--
Bhavesh Davda
----- Original Message -----
> From: "Stephen Hemminger" <stephen@networkplumber.org>
> To: "Bhavesh Davda" <bhavesh@vmware.com>
> Cc: "Henry Wang" <wang_hr@neusoft.com>, dev@dpdk.org
> Sent: Friday, May 10, 2013 9:15:49 AM
> Subject: Re: [dpdk-dev] rte_eth_dev_count() returns 0
>
> On Fri, 10 May 2013 09:05:08 -0700 (PDT)
> Bhavesh Davda <bhavesh@vmware.com> wrote:
>
> > ESXi 5.0 doesn't have SR-IOV support (it's in the code, but is disabled and
> > not usable for customers). VMware introduced SR-IOV support, including out
> > of the box support for Intel Niantic 10GbE NICs, in ESXi 5.1.
> >
>
> You can use PCI passthrough (Directpath) on 5.0 (and 5.1).
> I found that SR-IOV on Vmware is even more fussy than SR-IOV on Linux.
> It requires a supported platform with VT-d and PCIe ASPM
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] rte_eth_dev_count() returns 0
2013-05-10 16:21 ` Bhavesh Davda
@ 2013-05-10 18:44 ` Stephen Hemminger
2013-05-10 19:10 ` Bhavesh Davda
0 siblings, 1 reply; 11+ messages in thread
From: Stephen Hemminger @ 2013-05-10 18:44 UTC (permalink / raw)
To: Bhavesh Davda; +Cc: Henry Wang, dev
On Fri, 10 May 2013 09:21:36 -0700 (PDT)
Bhavesh Davda <bhavesh@vmware.com> wrote:
> Right: VMware has supported VMDirectPath passthrough since ESXi 4.0, on any platform with an IOMMU (Intel VT-d) which is anything Nehalem Xeons or newer.
>
> I know this is a DPDK forum, but Stephen, if you know of specific limitations of ESXi 5.1's SR-IOV support, please let me know and I'll make sure we work on addressing them.
I had trouble getting it to work on a white box ASUS motherboard. It seems the MB didn't
support PCIe power management ASPM. And Intel confirmed that was required by Vmware.
Linux didn't care if ASPM wasn't available and worked fine with SR-IOV.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] rte_eth_dev_count() returns 0
2013-05-10 18:44 ` Stephen Hemminger
@ 2013-05-10 19:10 ` Bhavesh Davda
2013-05-10 19:29 ` Stephen Hemminger
0 siblings, 1 reply; 11+ messages in thread
From: Bhavesh Davda @ 2013-05-10 19:10 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Henry Wang, dev
Hi Stephen,
I checked both our hypervisor code and with the experts who wrote and maintain the code, and we don't have any such limitation about not supporting SR-IOV if the motherboard or device doesn't support PCIe ASPM.
If you can point me to the Intel person(s) who gave you that information (offline, no need to make this public :)), we can make sure such FUD doesn't continue getting spread too far.
Thanks for making us aware of this FUD though.
--
Bhavesh Davda
----- Original Message -----
> From: "Stephen Hemminger" <stephen@networkplumber.org>
> To: "Bhavesh Davda" <bhavesh@vmware.com>
> Cc: "Henry Wang" <wang_hr@neusoft.com>, dev@dpdk.org
> Sent: Friday, May 10, 2013 11:44:50 AM
> Subject: Re: [dpdk-dev] rte_eth_dev_count() returns 0
>
> On Fri, 10 May 2013 09:21:36 -0700 (PDT)
> Bhavesh Davda <bhavesh@vmware.com> wrote:
>
> > Right: VMware has supported VMDirectPath passthrough since ESXi 4.0, on any
> > platform with an IOMMU (Intel VT-d) which is anything Nehalem Xeons or
> > newer.
> >
> > I know this is a DPDK forum, but Stephen, if you know of specific
> > limitations of ESXi 5.1's SR-IOV support, please let me know and I'll make
> > sure we work on addressing them.
>
> I had trouble getting it to work on a white box ASUS motherboard. It seems
> the MB didn't
> support PCIe power management ASPM. And Intel confirmed that was required by
> Vmware.
> Linux didn't care if ASPM wasn't available and worked fine with SR-IOV
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] rte_eth_dev_count() returns 0
2013-05-10 19:10 ` Bhavesh Davda
@ 2013-05-10 19:29 ` Stephen Hemminger
2013-05-10 21:00 ` Bhavesh Davda
0 siblings, 1 reply; 11+ messages in thread
From: Stephen Hemminger @ 2013-05-10 19:29 UTC (permalink / raw)
To: Bhavesh Davda; +Cc: Henry Wang, dev
On Fri, 10 May 2013 12:10:42 -0700 (PDT)
Bhavesh Davda <bhavesh@vmware.com> wrote:
> Hi Stephen,
>
> I checked both our hypervisor code and with the experts who wrote and maintain the code, and we don't have any such limitation about not supporting SR-IOV if the motherboard or device doesn't support PCIe ASPM.
>
> If you can point me to the Intel person(s) who gave you that information (offline, no need to make this public :)), we can make sure such FUD doesn't continue getting spread too far.
>
> Thanks for making us aware of this FUD though.
>
> --
> Bhavesh Davda
>
According to Intel the problem was that the BIOS did not support _OSC table.
SR-IOV needs VT-d enabled and BIOS with _OSC and DMA ACPI tables.
The _OSC table is an ACPI control method that the kernel uses to interact
with PCIe Active Statep Power Management. Linux just assume no ASPM if table
is missing, VMWare ESX 5.1 seems to need it.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] rte_eth_dev_count() returns 0
2013-05-10 19:29 ` Stephen Hemminger
@ 2013-05-10 21:00 ` Bhavesh Davda
0 siblings, 0 replies; 11+ messages in thread
From: Bhavesh Davda @ 2013-05-10 21:00 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: Henry Wang, dev
> From: "Stephen Hemminger" <stephen@networkplumber.org>
> Sent: Friday, May 10, 2013 12:29:29 PM
>
> On Fri, 10 May 2013 12:10:42 -0700 (PDT)
> Bhavesh Davda <bhavesh@vmware.com> wrote:
>
> > Hi Stephen,
> >
> > I checked both our hypervisor code and with the experts who wrote and
> > maintain the code, and we don't have any such limitation about not
> > supporting SR-IOV if the motherboard or device doesn't support PCIe ASPM.
> >
> > If you can point me to the Intel person(s) who gave you that information
> > (offline, no need to make this public :)), we can make sure such FUD
> > doesn't continue getting spread too far.
> >
> > Thanks for making us aware of this FUD though.
> >
> > --
> > Bhavesh Davda
> >
>
> According to Intel the problem was that the BIOS did not support _OSC table.
> SR-IOV needs VT-d enabled and BIOS with _OSC and DMA ACPI tables.
> The _OSC table is an ACPI control method that the kernel uses to interact
> with PCIe Active Statep Power Management. Linux just assume no ASPM if table
> is missing, VMWare ESX 5.1 seems to need it
>
Once again, both in looking at our ACPI _OSC code in the hypervisor and in consulting with the experts, there is no such limitation for SR-IOV support in ESXi.
Regards
- Bhavesh
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [dpdk-dev] rte_eth_dev_count() returns 0
2013-05-10 1:18 ` Henry Wang
2013-05-10 16:05 ` Bhavesh Davda
@ 2013-05-18 0:09 ` Rui Vaz (ruvaz)
1 sibling, 0 replies; 11+ messages in thread
From: Rui Vaz (ruvaz) @ 2013-05-18 0:09 UTC (permalink / raw)
To: Henry Wang; +Cc: dev
[-- Attachment #1: Type: text/plain, Size: 2481 bytes --]
Hi Henry,
Thank you very much for your help.
I have a I350 chipset based NIC, can you tell me why I will not be able to run l2fwd or l3fwd successfully with SR-IOV enabled in ESXi?
Thank you,
Rui
From: Henry Wang [mailto:wang_hr@neusoft.com]
Sent: Thursday, May 09, 2013 6:18 PM
To: Rui Vaz (ruvaz)
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] rte_eth_dev_count() returns 0
As far as I known, DPDK(1.2.3) only supports the following NICs:
(1) Intel 82576, 82580, I350 chipset based NICs;
(2) Intel 82599, X540T, X520T chipset based NICs;
(3) virtual NICs generated by SR-IOV from (2).
If the NICs on your C220 M3 is one of (2), you can run l2fwdvf OR l3fwdvf successfully with SR-IOV enabled in ESXi.
And I do not know wheather ESXi 5.0 VMM has SR-IOV function or not.
Further more, please reference to DPDK release notes.
http://www.intel.com/content/dam/www/public/us/en/documents/release-notes/intel-dpdk-release-notes.pdf
On Fri, May 10, 2013 at 9:00 AM, Rui Vaz (ruvaz) <ruvaz@cisco.com<mailto:ruvaz@cisco.com>> wrote:
Hi dev-dpdk,
I am following a Intel DPDK Step by Step Instruction found here: http://www.slideshare.net/hisaki/intel-dpdk-step-by-step-instructions
My Hello app from the DPDK examples runs fine, but I get the following error when running the l2fwd app:
"EAL: Error - exiting with code: 1
Cause: no Ethernet ports - bye"
The rte_eth_dev_count() call returns 0, I am using a CentOS 6.3 Virtual Machine on VMWare esxi 5.0 on a Cisco UCS C220 M3 which comes with Xeon E5-2600.
I would really appreciate If you could provide some help to solve this issue.
Thank you,
Rui
---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------
[-- Attachment #2: Type: text/html, Size: 9432 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread