[dpdk-dev] Link status issue with i40e firmware 7.0x

[dpdk-dev] Link status issue with i40e firmware 7.0x

[David Marchand]

Hello,

We recently upgraded a XXV710 nic on a Dell R740 server (running RHEL
7.7) to firmware version 7.00.

# ethtool -i enp59s0f0
driver: i40e
version: 2.8.10-k
firmware-version: 7.00 0x80004cf8 1.2154.0
expansion-rom-version:
bus-info: 0000:3b:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes


Following this upgrade, using the nic with dpdk only works once.
Restarting testpmd (dpdk 18.11.2) ends up with all ports reporting
that link is down, and indeed, the other side of the cable reports
that there is no light.

I tried with the 19.11-rc1 tag, same behavior.

To restore connectivity, the workaround is to unbind the device from
vfio-pci, bind it again to the i40e kernel driver, and bind it back to
vfio-pci.


We noticed a question on Intel forum:
https://forums.intel.com/s/question/0D50P00004OjvP2SAJ/xxv7102-how-to-rollbackdowngrade-firmware-to-version-601-from-version-700
But no public solution in this forum.


We are trying to downgrade the firmware to version 6.8x or 6.0x but so
far, we did not manage to do this downgrade.


Any help appreciated.

Thanks.

-- 
David Marchand

Re: [dpdk-dev] Link status issue with i40e firmware 7.0x

[David Marchand]

On Wed, Oct 30, 2019 at 3:20 PM David Marchand
<david.marchand@redhat.com> wrote:
>
> Hello,
>
> We recently upgraded a XXV710 nic on a Dell R740 server (running RHEL
> 7.7) to firmware version 7.00.
>
> # ethtool -i enp59s0f0
> driver: i40e
> version: 2.8.10-k
> firmware-version: 7.00 0x80004cf8 1.2154.0
> expansion-rom-version:
> bus-info: 0000:3b:00.0
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: yes
>
>
> Following this upgrade, using the nic with dpdk only works once.
> Restarting testpmd (dpdk 18.11.2) ends up with all ports reporting
> that link is down, and indeed, the other side of the cable reports
> that there is no light.

More info on reproducing the issue.
From a testpmd with links up:

# all good
testpmd> show port summary all
Number of available ports: 2
Port MAC Address       Name         Driver         Status   Link
0    3C:FD:FE:AD:BF:C4 0000:3b:00.0 net_i40e       up       25000Mbps
1    3C:FD:FE:AD:BF:C5 0000:3b:00.1 net_i40e       up       25000Mbps


# setting one link down
testpmd> set link-down port 0
i40e_phy_conf_link():     Current: abilities 1c, link_speed 4c
i40e_phy_conf_link():     Config:  abilities 24, link_speed 4c
testpmd> i40e_dev_alarm_handler(): ICR0: adminq event

Port 0: link state change event
i40e_dev_alarm_handler(): ICR0: adminq event

testpmd> show port summary all
Number of available ports: 2
Port MAC Address       Name         Driver         Status   Link
0    3C:FD:FE:AD:BF:C4 0000:3b:00.0 net_i40e       down     0Mbps
1    3C:FD:FE:AD:BF:C5 0000:3b:00.1 net_i40e       up       25000Mbps


# trying to set back up
testpmd> set link-up port 0
i40e_phy_conf_link():     Current: abilities 4, link_speed 0
i40e_phy_conf_link():     Config:  abilities 3c, link_speed 7e
testpmd> i40e_dev_alarm_handler(): ICR0: adminq event

testpmd> show port summary all
Number of available ports: 2
Port MAC Address       Name         Driver         Status   Link
0    3C:FD:FE:AD:BF:C4 0000:3b:00.0 net_i40e       down     0Mbps
1    3C:FD:FE:AD:BF:C5 0000:3b:00.1 net_i40e       up       25000Mbps

End of the game.
Need to bind back to the i40e kernel module to restore connectivity.


Any idea?

-- 
David Marchand

Re: [dpdk-dev] Link status issue with i40e firmware 7.0x

[Xing, Beilei]

> -----Original Message-----
> From: David Marchand [mailto:david.marchand@redhat.com]
> Sent: Wednesday, October 30, 2019 10:21 PM
> To: Xing, Beilei <beilei.xing@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>
> Cc: dev <dev@dpdk.org>; Jean Hsiao <jhsiao@redhat.com>; Christophe
> Fontaine <cfontain@redhat.com>
> Subject: Link status issue with i40e firmware 7.0x
> 
> Hello,
> 
> We recently upgraded a XXV710 nic on a Dell R740 server (running RHEL
> 7.7) to firmware version 7.00.
> 
> # ethtool -i enp59s0f0
> driver: i40e
> version: 2.8.10-k
> firmware-version: 7.00 0x80004cf8 1.2154.0
> expansion-rom-version:
> bus-info: 0000:3b:00.0
> supports-statistics: yes
> supports-test: yes
> supports-eeprom-access: yes
> supports-register-dump: yes
> supports-priv-flags: yes
> 
> 
> Following this upgrade, using the nic with dpdk only works once.
> Restarting testpmd (dpdk 18.11.2) ends up with all ports reporting that link is
> down, and indeed, the other side of the cable reports that there is no light.

Hi,

Sorry we didn't meet this problem before, you can find the FW support for help.

BR,
Beilei

> 
> I tried with the 19.11-rc1 tag, same behavior.
> 
> To restore connectivity, the workaround is to unbind the device from vfio-pci,
> bind it again to the i40e kernel driver, and bind it back to vfio-pci.
> 
> 
> We noticed a question on Intel forum:
> https://forums.intel.com/s/question/0D50P00004OjvP2SAJ/xxv7102-how-to-
> rollbackdowngrade-firmware-to-version-601-from-version-700
> But no public solution in this forum.
> 
> 
> We are trying to downgrade the firmware to version 6.8x or 6.0x but so far, we
> did not manage to do this downgrade.
> 
> 
> Any help appreciated.
> 
> Thanks.
> 
> --
> David Marchand