From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2D5BEA00BE; Mon, 14 Mar 2022 22:50:51 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F2215410E7; Mon, 14 Mar 2022 22:50:50 +0100 (CET) Received: from inbox.dpdk.org (inbox.dpdk.org [95.142.172.178]) by mails.dpdk.org (Postfix) with ESMTP id CFD0040E2D for ; Mon, 14 Mar 2022 22:50:49 +0100 (CET) Received: by inbox.dpdk.org (Postfix, from userid 33) id AE5B1A034D; Mon, 14 Mar 2022 22:50:49 +0100 (CET) From: bugzilla@dpdk.org To: dev@dpdk.org Subject: [Bug 957] IXGBE LSC IRQ configured state is lost on certain link down events Date: Mon, 14 Mar 2022 21:50:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: DPDK X-Bugzilla-Component: ethdev X-Bugzilla-Version: 20.11 X-Bugzilla-Keywords: X-Bugzilla-Severity: normal X-Bugzilla-Who: bly454@gmail.com X-Bugzilla-Status: UNCONFIRMED X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: dev@dpdk.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter target_milestone Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: http://bugs.dpdk.org/ Auto-Submitted: auto-generated X-Auto-Response-Suppress: All MIME-Version: 1.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org https://bugs.dpdk.org/show_bug.cgi?id=3D957 Bug ID: 957 Summary: IXGBE LSC IRQ configured state is lost on certain link down events Product: DPDK Version: 20.11 Hardware: All OS: All Status: UNCONFIRMED Severity: normal Priority: Normal Component: ethdev Assignee: dev@dpdk.org Reporter: bly454@gmail.com Target Milestone: --- Hello, We recently ran into an issue with DPDK 20.11 for the IXGBE driver operatin= g in 10G BASE-T mode. We have been able to replicate this behavior using dpdk-testpmd and do not see any recent/pertinent updates, so we are hopeful someone may be able to advise based on the information provided below. On t= he surface, based on our investigation, it would appear the current link-down transition logic does not correctly preserve IRQ mask configurations, specifically LSC, when a link partner causes some sort of slow or bounced l= ink down event. Background:=20 We recently started using a new 3rd party traffic generator card for testing our application. We found when using this card in 10G BASE-T mode and toggl= ing link up/down, it would correctly cause our application to detect the port t= o be down in our DPDK design. However, the link down event handling by the DPDK IXGBE driver appears to permanently disable its LSC IRQ detection on the fi= rst port down event such that any subsequent link up or down events from the external test card on this port would no longer be detected. The only way to restore link up was to restart the DPDK port in our design (stop/start). Ha= ving looked at this a bit, we switched over to the classic testpmd application a= nd observed the exact same behavior. Here is the data we believe you would find interesting: NIC in question: # lspci -D -nn | grep -F [0200] | grep 552 0000:03:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connect= ion X552/X557-AT 10GBASE-T [8086:15ad] 0000:03:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Connect= ion X552/X557-AT 10GBASE-T [8086:15ad] # dpdk-devbind.py -s | grep 552 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' drv=3Dvfio-p= ci unused=3Duio_pci_generic 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' drv=3Dvfio-p= ci unused=3Duio_pci_generic We made the following debug logging changes to try an capture interesting d= ata to share: diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index 5a30c39593..75a9f9163b 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -4497,7 +4497,7 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *de= v) /* read-on-clear nic registers here */ eicr =3D IXGBE_READ_REG(hw, IXGBE_EICR); - PMD_DRV_LOG(DEBUG, "eicr %x", eicr); + PMD_DRV_LOG(ERR, "eicr %x", eicr); intr->flags =3D 0; @@ -4614,7 +4613,7 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev) } } - PMD_DRV_LOG(DEBUG, "enable intr immediately"); + PMD_DRV_LOG(ERR, "enable intr immediately, mask: 0x%08x, orig: 0x%08x, flags: 0x%08x", intr->mask, intr->mask_original, intr->flags); ixgbe_enable_intr(dev); return 0; @@ -4648,7 +4647,9 @@ ixgbe_dev_interrupt_delayed_handler(void *param) ixgbe_disable_intr(hw); - eicr =3D IXGBE_READ_REG(hw, IXGBE_EICR); + eicr =3D IXGBE_READ_REG(hw, IXGBE_EICR); + PMD_DRV_LOG(ERR, "in delay func: eicr 0x%08x", eicr); + PMD_DRV_LOG(ERR, "enable intr delayed, mask: 0x%08x, orig: 0x%08x, flag= s: 0x%08x", intr->mask, intr->mask_original, intr->flags); if (eicr & IXGBE_EICR_MAILBOX) ixgbe_pf_mbx_process(dev); With the above =E2=80=9Clog-err=E2=80=9D additions, we have provided the fo= llowing results. The first set of data below was generated using an older 3rd party traffic generator card to provide =E2=80=9Cgood=E2=80=9D results that show the IXGB= E driver working correctly. Following that are the non-working (bad) logging results for the= new traffic generator card. Both 3rd party cards correctly transition between d= own and up states. ###################################################################### # good sequence, both down detection and then up detection ###################################################################### # port transition from up to down <27>1 2022-03-05T00:12:11.415436+00:00 - - ixgbe_dev_interrupt_get_status(= ): eicr 100000 <27>1 2022-03-05T00:12:11.415489+00:00 - - ixgbe_dev_interrupt_action(): enable intr immediately, mask: 0x02200000, orig: 0x02300000, flags: 0x00000= 001 <27>1 2022-03-05T00:12:11.425448+00:00 - - ixgbe_dev_interrupt_get_status(= ): eicr 2000000 <27>1 2022-03-05T00:12:11.446191+00:00 - - ixgbe_dev_interrupt_action(): enable intr immediately, mask: 0x02200000, orig: 0x02300000, flags: 0x00000= 000 <27>1 2022-03-05T00:12:15.415600+00:00 - -=20 ixgbe_dev_interrupt_delayed_handler(): in delay func: eicr 0x00000000 <27>1 2022-03-05T00:12:15.415655+00:00 - -=20 ixgbe_dev_interrupt_delayed_handler(): enable intr delayed, mask: 0x0220000= 0, orig: 0x02300000, flags: 0x00000000 # port transition from down to up <27>1 2022-03-05T00:12:33.856734+00:00 - - ixgbe_dev_interrupt_get_status(= ): eicr 2000000 <27>1 2022-03-05T00:12:33.877463+00:00 - - ixgbe_dev_interrupt_action(): enable intr immediately, mask: 0x02300000, orig: 0x00000000, flags: 0x00000= 000 <27>1 2022-03-05T00:12:34.203274+00:00 - - ixgbe_dev_interrupt_get_status(= ): eicr 100000 <27>1 2022-03-05T00:12:34.207905+00:00 - - ixgbe_dev_interrupt_action(): enable intr immediately, mask: 0x02200000, orig: 0x02300000, flags: 0x00000= 001 <27>1 2022-03-05T00:12:35.207994+00:00 - -=20 ixgbe_dev_interrupt_delayed_handler(): in delay func: eicr 0x00100000 <27>1 2022-03-05T00:12:35.208027+00:00 - -=20 ixgbe_dev_interrupt_delayed_handler(): enable intr delayed, mask: 0x0220000= 0, orig: 0x02300000, flags: 0x00000001 ###################################################################### # bad sequence, detects down event, but does not see the up event ###################################################################### # port transition from up to down <27>1 2022-03-05T00:13:00.377072+00:00 - - ixgbe_dev_interrupt_get_status(= ): eicr 100000 <27>1 2022-03-05T00:13:00.377127+00:00 - - ixgbe_dev_interrupt_action(): enable intr immediately, mask: 0x02200000, orig: 0x02300000, flags: 0x00000= 001 <27>1 2022-03-05T00:13:00.643788+00:00 - - ixgbe_dev_interrupt_get_status(= ): eicr 2100000 <27>1 2022-03-05T00:13:00.664603+00:00 - - ixgbe_dev_interrupt_action(): enable intr immediately, mask: 0x02200000, orig: 0x02200000, flags: 0x00000= 001 <27>1 2022-03-05T00:13:01.664703+00:00 - -=20 ixgbe_dev_interrupt_delayed_handler(): in delay func: eicr 0x00000000 <27>1 2022-03-05T00:13:01.664738+00:00 - -=20 ixgbe_dev_interrupt_delayed_handler(): enable intr delayed, mask: 0x0220000= 0, orig: 0x02200000, flags: 0x00000001 <27>1 2022-03-05T00:13:04.377237+00:00 - -=20 ixgbe_dev_interrupt_delayed_handler(): in delay func: eicr 0x00000000 <27>1 2022-03-05T00:13:04.377269+00:00 - -=20 ixgbe_dev_interrupt_delayed_handler(): enable intr delayed, mask: 0x0220000= 0, orig: 0x00000000, flags: 0x00000000 # port transition from down to up --=20 You are receiving this mail because: You are the assignee for the bug.=