From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2CD17A04DD; Wed, 27 Nov 2019 18:07:41 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 421BA1BE99; Wed, 27 Nov 2019 18:07:40 +0100 (CET) Received: from inbox.dpdk.org (xvm-172-178.dc0.ghst.net [95.142.172.178]) by dpdk.org (Postfix) with ESMTP id 827EA1BE91 for ; Wed, 27 Nov 2019 18:07:39 +0100 (CET) Received: by inbox.dpdk.org (Postfix, from userid 33) id 6262CA04E0; Wed, 27 Nov 2019 18:07:39 +0100 (CET) From: bugzilla@dpdk.org To: dev@dpdk.org Date: Wed, 27 Nov 2019 17:07:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: DPDK X-Bugzilla-Component: ethdev X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: major X-Bugzilla-Who: julien.meunier@nokia.com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: dev@dpdk.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter target_milestone attachments.created Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.dpdk.org/ Auto-Submitted: auto-generated X-Auto-Response-Suppress: All MIME-Version: 1.0 Subject: [dpdk-dev] [Bug 372] ixgbevf: cannot probe or start port if PF link is unstable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" https://bugs.dpdk.org/show_bug.cgi?id=3D372 Bug ID: 372 Summary: ixgbevf: cannot probe or start port if PF link is unstable Product: DPDK Version: unspecified Hardware: All OS: All Status: UNCONFIRMED Severity: major Priority: Normal Component: ethdev Assignee: dev@dpdk.org Reporter: julien.meunier@nokia.com Target Milestone: --- Created attachment 76 --> https://bugs.dpdk.org/attachment.cgi?id=3D76&action=3Dedit ixgbevf: debug mbx during init Configuration =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Kernel: 4.19 NIC: Intel Niantic X550 PF driver: Kernel ixgbe v.5.1.0 SRIO-V: enabled DPDK version: v18.05 - v19.11-rc4 Detail Test Steps =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - Set PF up:: ip link set dev eth0 up - Export 16 VF:: echo 16 > /sys/class/net/eth0/device/sriov_numvfs=20 - Configure MAC of the VF:: for i in {0..16}; do ip link set dev eth0 vf $i mac 00:11:22:33:44:${i}; do= ne - Bind it to igb_uio (example: 04:10.0 ...):: dpdk-devbind -b igb_uio 04:10.0 04:10.2 04:10.4 04:10.6 04:11.0 \ 04:11.2 04:11.4 04:11.6 04:12.0 04:12.2 04:12.4 04:12.6 04:13.0 \ 04:13.2 04:13.4 04:13.6=20 - PF is UP, but physical link changes status each seconds (UP / DOWN) - Start a DPDK application (example: testpmd) when link is unstable:: testpmd -w 04:10.0 -w 04:10.2 -w 04:10.4 -w 04:10.6 \ -w 04:11.0 -w 04:11.2 -w 04:11.4 -w 04:11.6 \ -w 04:12.0 -w 04:12.2 -w 04:12.4 -w 04:12.6 \ -w 04:13.0 -w 04:13.2 -w 04:13.4 -w 04:13.6 -- -i --no-mlockall Expected Result =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D No issue during EAL init or during rte_eth_dev_start when link changes its status. Actual Result =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Issue during the EAL init:: EAL: PCI device 0000:04:10.0 on NUMA socket 0 EAL: probe driver: 8086:15a8 net_ixgbe_vf EAL: Requested device 0000:04:10.0 cannot be used Issue during the rte_eth_dev_start:: ixgbevf_dev_start(): Unable to reset vf hardware (-10) No issue if link status is stable during the init Workaround =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - Application has to probe again the same device - Application has to start again the same port Analysis =3D=3D=3D=3D=3D=3D=3D=3D I added some traces in ixgbe_reset_hw_vf in order to dump the content of the mailbox. (see attachment) When VF is correctly initialized:=20 ixgbe_reset_hw_vf: jmeunier - msgbuf[0] =3D 80000001 =3D> (IXGBE_VF_RESET | IXGBE_VT_MSGTYPE_ACK) When VF is NOT correctly initialized: ixgbe_reset_hw_vf: jmeunier - msgbuf[0] =3D 20000100 =3D> (IXGBE_VT_MSGTYPE_CTS | IXGBE_PF_CONTROL_MSG) An invalid message is received in the mailbox during the initialization. This message seems to be sent by the PF to notify that the link has changed= its status. According to the current implementation (on Kernel side), when link status changes, PF notifies all associated VFs through the mailbox:: ixgbe_watchdog_link_is_up =3D> ixgbe_ping_all_vfs =3D> ixgbe_write_= mbx(hw, &ping, 1, i) So, if VF waits for one specific message, if link changes at the same time,= VF will read an invalid message. In this PR, it's IXGBE_VF_RESET, but I can see others cases like that in the current PF/VF implementation. BTW, I suspect that Kernel overwrites the mailbox message (to be confirmed) --=20 You are receiving this mail because: You are the assignee for the bug.=