[dpdk-dev] 82599ES NIC support

[dpdk-dev] 82599ES NIC support

[HS]

Hi,

I was trying to test DPDK 1.6 with "82599ES" Intel NIC, but noticed that
DPDK does not seem to work with this NIC.  DPDK app fails because
rte_eth_dev_count() returns 0.

On the other hand, I confirmed DPDK works okay with "82599EB".

So looks like not all 82599 (ixgbe) chipsets are supported by DPDK while it
says so in http://dpdk.org/doc/nics

What is the discrepancy between 82599ES and 82599EB, which causes the
former to fail to work with DPDK?

Is it possible to fix this problem?  I'll be glad to help fix it.

Thanks,
-HS

Re: [dpdk-dev] 82599ES NIC support

[Shaw, Jeffrey B]

Can you check if your PCI device ID is listed in "lib/librte_eal/common/include/rte_pci_dev_ids.h"?
Can you verify that you have bound your device to "igb_uio", perhaps using "tools/pci_unbind.py" (maybe renamed to tools/igb_uio_bind.py)?
You might also try to edit the ".config" (in your build directory, NOT the Linux .config) to enable any of the "CONFIG_RTE_LIBRTE_IXGBE_DEBUG_*" options, in case there are more error messages.

Thanks,
Jeff

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of HS
Sent: Monday, March 31, 2014 10:30 AM
To: dev@dpdk.org
Subject: [dpdk-dev] 82599ES NIC support

Hi,

I was trying to test DPDK 1.6 with "82599ES" Intel NIC, but noticed that DPDK does not seem to work with this NIC.  DPDK app fails because
rte_eth_dev_count() returns 0.

On the other hand, I confirmed DPDK works okay with "82599EB".

So looks like not all 82599 (ixgbe) chipsets are supported by DPDK while it says so in http://dpdk.org/doc/nics

What is the discrepancy between 82599ES and 82599EB, which causes the former to fail to work with DPDK?

Is it possible to fix this problem?  I'll be glad to help fix it.

Thanks,
-HS

Re: [dpdk-dev] 82599ES NIC support

[HS]

> Can you check if your PCI device ID is listed in "lib/librte_eal/common/
include/rte_pci_dev_ids.h"?

82599ES is not listed in lib/librte_eal/common/
include/rte_pci_dev_ids.h

On the other hand, 82599EB (which works okay with DPDK) is not listed there
either.


> Can you verify that you have bound your device to "igb_uio", perhaps
using "tools/pci_unbind.py" (maybe renamed to tools/igb_uio_bind.py)?

Yes:

$ sudo pci_unbind.py --status

Network devices using IGB_UIO driver
====================================
0000:07:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio
unused=


> You might also try to edit the ".config" (in your build directory, NOT
the Linux .config) to enable any of the "CONFIG_RTE_LIBRTE_IXGBE_
DEBUG_*" options, in case there are more error messages.

I enabled debugging as you said, and run testpmd app.

It ends with an error "Cause: No probed ethernet devices - check that
CONFIG_RTE_LIBRTE_IGB_PMD=y and that CONFIG_RTE_LIBRTE_EM_PMD=y and that
CONFIG_RTE_LIBRTE_IXGBE_PMD=y in your configuration file"

Here is the detailed debug output:
http://pastebin.com/tWkUZBKj


Thanks,
-hs




On Mon, Mar 31, 2014 at 1:38 PM, Shaw, Jeffrey B
<jeffrey.b.shaw@intel.com>wrote:

> Can you check if your PCI device ID is listed in
> "lib/librte_eal/common/include/rte_pci_dev_ids.h"?
> Can you verify that you have bound your device to "igb_uio", perhaps using
> "tools/pci_unbind.py" (maybe renamed to tools/igb_uio_bind.py)?
> You might also try to edit the ".config" (in your build directory, NOT the
> Linux .config) to enable any of the "CONFIG_RTE_LIBRTE_IXGBE_DEBUG_*"
> options, in case there are more error messages.
>
> Thanks,
> Jeff
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of HS
> Sent: Monday, March 31, 2014 10:30 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] 82599ES NIC support
>
> Hi,
>
> I was trying to test DPDK 1.6 with "82599ES" Intel NIC, but noticed that
> DPDK does not seem to work with this NIC.  DPDK app fails because
> rte_eth_dev_count() returns 0.
>
> On the other hand, I confirmed DPDK works okay with "82599EB".
>
> So looks like not all 82599 (ixgbe) chipsets are supported by DPDK while
> it says so in http://dpdk.org/doc/nics
>
> What is the discrepancy between 82599ES and 82599EB, which causes the
> former to fail to work with DPDK?
>
> Is it possible to fix this problem?  I'll be glad to help fix it.
>
> Thanks,
> -HS
>

Re: [dpdk-dev] 82599ES NIC support

[Shaw, Jeffrey B]

If you read the output that you posted, you will see the real error: "eth_ixgbe_dev_init(): Hardware Initialization Failure: -30".  The error code is defined in lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h as IXGBE_ERR_SFP_SETUP_NOT_COMPLETE.
This has been reported several times on the mailing list, but you can search "Unusable interfaces although apparently attached to IGB_UIO" to see for yourself.


To clarify your understanding of the internals of the driver code, I have provided some information for your information, but this is irrelevant to your problem.
You should see your PCI device ID (not the branding name associated with the product line) in rte_pci_dev_ids.h.
You can use the command "lspci -s 0000:07:00.0 -x" to dump the config space for your device. The first 4 bytes will be the vendor/device ID.

For one specific instance of the 82599EB NIC, I have vendor ID of 0x8086 and device ID of 0x10FB.
# lspci -s 0000:02:00.0 -x
02:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
00: 86 80 fb 10 <... snipped... >

You should notice that the lines do in fact exist in include/rte_pci_dev_ids.h
#define IXGBE_DEV_ID_82599_SFP                  0x10FB
RTE_PCI_DEV_ID_DECL_IXGBE(PCI_VENDOR_ID_INTEL, IXGBE_DEV_ID_82599_SFP)

This confirms that the specific instantiation of my device (0x8086, 0x10FB) is supported.

Thanks,
Jeff

From: hyunseok.chang@gmail.com [mailto:hyunseok.chang@gmail.com] On Behalf Of HS
Sent: Monday, March 31, 2014 11:16 AM
To: Shaw, Jeffrey B
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] 82599ES NIC support


> Can you check if your PCI device ID is listed in "lib/librte_eal/common/
include/rte_pci_dev_ids.h"?

82599ES is not listed in lib/librte_eal/common/
include/rte_pci_dev_ids.h
On the other hand, 82599EB (which works okay with DPDK) is not listed there either.


> Can you verify that you have bound your device to "igb_uio", perhaps using "tools/pci_unbind.py" (maybe renamed to tools/igb_uio_bind.py)?
Yes:

$ sudo pci_unbind.py --status

Network devices using IGB_UIO driver
====================================
0000:07:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio unused=


> You might also try to edit the ".config" (in your build directory, NOT the Linux .config) to enable any of the "CONFIG_RTE_LIBRTE_IXGBE_
DEBUG_*" options, in case there are more error messages.
I enabled debugging as you said, and run testpmd app.
It ends with an error "Cause: No probed ethernet devices - check that CONFIG_RTE_LIBRTE_IGB_PMD=y and that CONFIG_RTE_LIBRTE_EM_PMD=y and that CONFIG_RTE_LIBRTE_IXGBE_PMD=y in your configuration file"
Here is the detailed debug output:
http://pastebin.com/tWkUZBKj

Thanks,
-hs


On Mon, Mar 31, 2014 at 1:38 PM, Shaw, Jeffrey B <jeffrey.b.shaw@intel.com<mailto:jeffrey.b.shaw@intel.com>> wrote:
Can you check if your PCI device ID is listed in "lib/librte_eal/common/include/rte_pci_dev_ids.h"?
Can you verify that you have bound your device to "igb_uio", perhaps using "tools/pci_unbind.py" (maybe renamed to tools/igb_uio_bind.py)?
You might also try to edit the ".config" (in your build directory, NOT the Linux .config) to enable any of the "CONFIG_RTE_LIBRTE_IXGBE_DEBUG_*" options, in case there are more error messages.

Thanks,
Jeff

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of HS
Sent: Monday, March 31, 2014 10:30 AM
To: dev@dpdk.org<mailto:dev@dpdk.org>
Subject: [dpdk-dev] 82599ES NIC support

Hi,

I was trying to test DPDK 1.6 with "82599ES" Intel NIC, but noticed that DPDK does not seem to work with this NIC.  DPDK app fails because
rte_eth_dev_count() returns 0.

On the other hand, I confirmed DPDK works okay with "82599EB".

So looks like not all 82599 (ixgbe) chipsets are supported by DPDK while it says so in http://dpdk.org/doc/nics

What is the discrepancy between 82599ES and 82599EB, which causes the former to fail to work with DPDK?

Is it possible to fix this problem?  I'll be glad to help fix it.

Thanks,
-HS

Re: [dpdk-dev] 82599ES NIC support

[HS]

Thanks for the information.
Indeed, it was failing during NIC initialization stage.
The problem was due to frequency scaling.
After I disable it from bios, I am now able to run dpdk.

Thanks again!
-hs




On Mon, Mar 31, 2014 at 2:46 PM, Shaw, Jeffrey B
<jeffrey.b.shaw@intel.com>wrote:

>  If you read the output that you posted, you will see the real error:
> "eth_ixgbe_dev_init(): Hardware Initialization Failure: -30".  The error
> code is defined in lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h as
> IXGBE_ERR_SFP_SETUP_NOT_COMPLETE.
>
> This has been reported several times on the mailing list, but you can
> search "Unusable interfaces although apparently attached to IGB_UIO" to see
> for yourself.
>
>
>
>
>
> To clarify your understanding of the internals of the driver code, I have
> provided some information for your information, but this is irrelevant to
> your problem.
>
> You should see your PCI device ID (not the branding name associated with
> the product line) in rte_pci_dev_ids.h.
>
> You can use the command "lspci -s 0000:07:00.0 -x" to dump the config
> space for your device. The first 4 bytes will be the vendor/device ID.
>
>
>
> For one specific instance of the 82599EB NIC, I have vendor ID of 0x8086
> and device ID of 0x10FB.
>
> # lspci -s 0000:02:00.0 -x
>
> 02:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+
> Network Connection (rev 01)
>
> 00: 86 80 fb 10 <... snipped... >
>
>
>
> You should notice that the lines do in fact exist in
> include/rte_pci_dev_ids.h
>
> #define IXGBE_DEV_ID_82599_SFP                  0x10FB
>
> RTE_PCI_DEV_ID_DECL_IXGBE(PCI_VENDOR_ID_INTEL, IXGBE_DEV_ID_82599_SFP)
>
>
>
> This confirms that the specific instantiation of my device (0x8086,
> 0x10FB) is supported.
>
>
>
> Thanks,
>
> Jeff
>
>
>
> *From:* hyunseok.chang@gmail.com [mailto:hyunseok.chang@gmail.com] *On
> Behalf Of *HS
> *Sent:* Monday, March 31, 2014 11:16 AM
> *To:* Shaw, Jeffrey B
> *Cc:* dev@dpdk.org
> *Subject:* Re: [dpdk-dev] 82599ES NIC support
>
>
>
>
> > Can you check if your PCI device ID is listed in "lib/librte_eal/common/
>
> include/rte_pci_dev_ids.h"?
>
> 82599ES is not listed in lib/librte_eal/common/
>
> include/rte_pci_dev_ids.h
>
> On the other hand, 82599EB (which works okay with DPDK) is not listed
> there either.
>
>
> > Can you verify that you have bound your device to "igb_uio", perhaps
> using "tools/pci_unbind.py" (maybe renamed to tools/igb_uio_bind.py)?
>
> Yes:
>
> $ sudo pci_unbind.py --status
>
> Network devices using IGB_UIO driver
> ====================================
> 0000:07:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' drv=igb_uio
> unused=
>
>
> > You might also try to edit the ".config" (in your build directory, NOT
> the Linux .config) to enable any of the "CONFIG_RTE_LIBRTE_IXGBE_
>
> DEBUG_*" options, in case there are more error messages.
>
> I enabled debugging as you said, and run testpmd app.
>
> It ends with an error "Cause: No probed ethernet devices - check that
> CONFIG_RTE_LIBRTE_IGB_PMD=y and that CONFIG_RTE_LIBRTE_EM_PMD=y and that
> CONFIG_RTE_LIBRTE_IXGBE_PMD=y in your configuration file"
>
> Here is the detailed debug output:
> http://pastebin.com/tWkUZBKj
>
>
>
> Thanks,
>
> -hs
>
>
>
>
>
> On Mon, Mar 31, 2014 at 1:38 PM, Shaw, Jeffrey B <jeffrey.b.shaw@intel.com>
> wrote:
>
> Can you check if your PCI device ID is listed in
> "lib/librte_eal/common/include/rte_pci_dev_ids.h"?
> Can you verify that you have bound your device to "igb_uio", perhaps using
> "tools/pci_unbind.py" (maybe renamed to tools/igb_uio_bind.py)?
> You might also try to edit the ".config" (in your build directory, NOT the
> Linux .config) to enable any of the "CONFIG_RTE_LIBRTE_IXGBE_DEBUG_*"
> options, in case there are more error messages.
>
> Thanks,
> Jeff
>
>
> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of HS
> Sent: Monday, March 31, 2014 10:30 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] 82599ES NIC support
>
> Hi,
>
> I was trying to test DPDK 1.6 with "82599ES" Intel NIC, but noticed that
> DPDK does not seem to work with this NIC.  DPDK app fails because
> rte_eth_dev_count() returns 0.
>
> On the other hand, I confirmed DPDK works okay with "82599EB".
>
> So looks like not all 82599 (ixgbe) chipsets are supported by DPDK while
> it says so in http://dpdk.org/doc/nics
>
> What is the discrepancy between 82599ES and 82599EB, which causes the
> former to fail to work with DPDK?
>
> Is it possible to fix this problem?  I'll be glad to help fix it.
>
> Thanks,
> -HS
>
>
>