From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 11845A00E6 for ; Mon, 8 Jul 2019 10:46:27 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DF080322C; Mon, 8 Jul 2019 10:46:26 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 484D42E81 for ; Mon, 8 Jul 2019 10:46:25 +0200 (CEST) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Jul 2019 01:36:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,466,1557212400"; d="scan'208";a="340388468" Received: from kmsmsx152.gar.corp.intel.com ([172.21.73.87]) by orsmga005.jf.intel.com with ESMTP; 08 Jul 2019 01:36:08 -0700 Received: from pgsmsx103.gar.corp.intel.com ([169.254.2.4]) by KMSMSX152.gar.corp.intel.com ([169.254.11.92]) with mapi id 14.03.0439.000; Mon, 8 Jul 2019 16:35:56 +0800 From: "Zhao1, Wei" To: "Zhang, Xiao" , "dev@dpdk.org" CC: "Zhang, Xiao" Thread-Topic: [dpdk-dev] [v2] net/e1000: i219 unit hang issue fix on reset/close Thread-Index: AQHVNTewShdJJa7o30+PlvglGG5q0abAYYEg Date: Mon, 8 Jul 2019 08:35:56 +0000 Message-ID: References: <1562586229-28201-1-git-send-email-xiao.zhang@intel.com> In-Reply-To: <1562586229-28201-1-git-send-email-xiao.zhang@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-version: 11.0.600.7 dlp-reaction: no-action x-originating-ip: [172.30.20.205] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [v2] net/e1000: i219 unit hang issue fix on reset/close X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, xiao > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Xiao Zhang > Sent: Monday, July 8, 2019 7:44 PM > To: dev@dpdk.org > Cc: Zhang, Xiao > Subject: [dpdk-dev] [v2] net/e1000: i219 unit hang issue fix on reset/clo= se >=20 > Unit hang may occur if multiple descriptors are available in the rings du= ring > reset or close. This state can be detected by configure status by bit 8 i= n register. > If the bit is set and there are pending descriptors in one of the rings, = we must > flush them before reset or close. >=20 > Signed-off-by: Xiao Zhang > --- > drivers/net/e1000/base/e1000_ich8lan.h | 1 + > drivers/net/e1000/e1000_ethdev.h | 1 + > drivers/net/e1000/igb_ethdev.c | 4 ++ > drivers/net/e1000/igb_rxtx.c | 96 > ++++++++++++++++++++++++++++++++++ > 4 files changed, 102 insertions(+) >=20 > diff --git a/drivers/net/e1000/base/e1000_ich8lan.h > b/drivers/net/e1000/base/e1000_ich8lan.h > index bc4ed1d..1f2a3f8 100644 > --- a/drivers/net/e1000/base/e1000_ich8lan.h > +++ b/drivers/net/e1000/base/e1000_ich8lan.h > @@ -120,6 +120,7 @@ POSSIBILITY OF SUCH DAMAGE. > #define E1000_FEXTNVM7_SIDE_CLK_UNGATE 0x00000004 > #if !defined(EXTERNAL_RELEASE) || defined(ULP_SUPPORT) > #define E1000_FEXTNVM7_DISABLE_SMB_PERST 0x00000020 > +#define E1000_FEXTNVM7_NEED_DESCRING_FLUSH 0x00000100 > #endif /* !EXTERNAL_RELEASE || ULP_SUPPORT */ > #define E1000_FEXTNVM9_IOSFSB_CLKGATE_DIS 0x00000800 > #define E1000_FEXTNVM9_IOSFSB_CLKREQ_DIS 0x00001000 > diff --git a/drivers/net/e1000/e1000_ethdev.h > b/drivers/net/e1000/e1000_ethdev.h > index 67acb73..3451979 100644 > --- a/drivers/net/e1000/e1000_ethdev.h > +++ b/drivers/net/e1000/e1000_ethdev.h > @@ -522,5 +522,6 @@ int igb_action_rss_same(const struct > rte_flow_action_rss *comp, int igb_config_rss_filter(struct rte_eth_dev = *dev, > struct igb_rte_flow_rss_conf *conf, > bool add); > +void igb_flush_desc_rings(struct rte_eth_dev *dev); >=20 > #endif /* _E1000_ETHDEV_H_ */ > diff --git a/drivers/net/e1000/igb_ethdev.c b/drivers/net/e1000/igb_ethde= v.c > index 3ee28cf..845101b 100644 > --- a/drivers/net/e1000/igb_ethdev.c > +++ b/drivers/net/e1000/igb_ethdev.c > @@ -1589,6 +1589,10 @@ eth_igb_close(struct rte_eth_dev *dev) > eth_igb_stop(dev); > adapter->stopped =3D 1; >=20 > + /* Flush desc rings for i219 */ > + if (hw->mac.type >=3D e1000_pch_spt) > + igb_flush_desc_rings(dev); > + > e1000_phy_hw_reset(hw); > igb_release_manageability(hw); > igb_hw_control_release(hw); > diff --git a/drivers/net/e1000/igb_rxtx.c b/drivers/net/e1000/igb_rxtx.c = index > c5606de..33eeb4e 100644 > --- a/drivers/net/e1000/igb_rxtx.c > +++ b/drivers/net/e1000/igb_rxtx.c > @@ -18,6 +18,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -63,6 +64,9 @@ > #define IGB_TX_OFFLOAD_NOTSUP_MASK \ > (PKT_TX_OFFLOAD_MASK ^ IGB_TX_OFFLOAD_MASK) >=20 > +/* PCI offset for querying descriptor ring status*/ > +#define PCICFG_DESC_RING_STATUS 0xE4 > + > /** > * Structure associated with each descriptor of the RX ring of a RX queu= e. > */ > @@ -2962,3 +2966,95 @@ igb_config_rss_filter(struct rte_eth_dev *dev, >=20 > return 0; > } > + > +static void e1000_flush_tx_ring(struct rte_eth_dev *dev) { > + struct e1000_hw *hw =3D E1000_DEV_PRIVATE_TO_HW(dev->data- > >dev_private); > + volatile union e1000_adv_tx_desc *tx_desc; > + uint32_t tdt, tctl, txd_lower =3D E1000_TXD_CMD_IFCS; > + uint16_t size =3D 512; > + struct igb_tx_queue *txq; > + > + if (dev->data->tx_queues =3D=3D NULL) > + return; > + txq =3D dev->data->tx_queues[0]; > + > + tctl =3D E1000_READ_REG(hw, E1000_TCTL); > + E1000_WRITE_REG(hw, E1000_TCTL, tctl | E1000_TCTL_EN); > + tdt =3D E1000_READ_REG(hw, E1000_TDT(0)); > + if (tdt !=3D txq->tx_tail) > + return; > + tx_desc =3D txq->tx_ring; > + tx_desc->read.buffer_addr =3D txq->tx_ring_phys_addr; > + tx_desc->read.cmd_type_len =3D rte_cpu_to_le_32(txd_lower | size); > + tx_desc->read.olinfo_status =3D 0; > + > + rte_wmb(); > + txq->tx_tail++; > + if (txq->tx_tail =3D=3D txq->nb_tx_desc) > + txq->tx_tail =3D 0; > + rte_io_wmb(); > + E1000_WRITE_REG(hw, E1000_TDT(0), txq->tx_tail); > + usec_delay(250); > +} > + > +static void e1000_flush_rx_ring(struct rte_eth_dev *dev) { > + uint32_t rctl, rxdctl; > + struct e1000_hw *hw =3D E1000_DEV_PRIVATE_TO_HW(dev->data- > >dev_private); > + > + rctl =3D E1000_READ_REG(hw, E1000_RCTL); > + E1000_WRITE_REG(hw, E1000_TCTL, rctl & ~E1000_RCTL_EN); > + E1000_WRITE_FLUSH(hw); > + usec_delay(150); > + > + rxdctl =3D E1000_READ_REG(hw, E1000_RXDCTL(0)); > + /* zero the lower 14 bits (prefetch and host thresholds) */ > + rxdctl &=3D 0xffffc000; > + > + /* update thresholds: prefetch threshold to 31, host threshold to 1 > + * and make sure the granularity is "descriptors" and not "cache lines" > + */ > + rxdctl |=3D (0x1F | (1UL << 8) | E1000_RXDCTL_THRESH_UNIT_DESC); > + > + E1000_WRITE_REG(hw, E1000_RXDCTL(0), rxdctl); > + /* momentarily enable the RX ring for the changes to take effect */ > + E1000_WRITE_REG(hw, E1000_RCTL, rctl | E1000_RCTL_EN); > + E1000_WRITE_FLUSH(hw); > + usec_delay(150); > + E1000_WRITE_REG(hw, E1000_RCTL, rctl & ~E1000_RCTL_EN); } > + In dpdk driver, it may enable and use many queues, so why do we only=20 Process tx and rx queue 0, ring for other queues not need to handle? > +/** > + * igb_flush_desc_rings - remove all descriptors from the descriptor > +rings > + * > + * In i219, the descriptor rings must be emptied before > +resetting/closing the > + * HW. Failure to do this will cause the HW to enter a unit hang state > +which > + * can only be released by PCI reset on the device > + * > + */ > + > +void igb_flush_desc_rings(struct rte_eth_dev *dev) { > + uint32_t fextnvm11, tdlen; > + struct e1000_hw *hw =3D E1000_DEV_PRIVATE_TO_HW(dev->data- > >dev_private); > + struct rte_pci_device *pci_dev =3D RTE_ETH_DEV_TO_PCI(dev); > + uint32_t hang_state =3D 0; > + > + fextnvm11 =3D E1000_READ_REG(hw, E1000_FEXTNVM11); > + E1000_WRITE_REG(hw, E1000_FEXTNVM11, > + fextnvm11 | E1000_FEXTNVM11_DISABLE_MULR_FIX); > + tdlen =3D E1000_READ_REG(hw, E1000_TDLEN(0)); > + rte_pci_read_config(pci_dev, &hang_state, sizeof(hang_state), > + PCICFG_DESC_RING_STATUS); > + > + /* do nothing if we're not in faulty state, or if the queue is empty */ > + if ((hang_state & E1000_FEXTNVM7_NEED_DESCRING_FLUSH) && > tdlen) { > + /* flush desc ring */ > + e1000_flush_tx_ring(dev); > + rte_pci_read_config(pci_dev, &hang_state, sizeof(hang_state), > + PCICFG_DESC_RING_STATUS); > + if (hang_state & E1000_FEXTNVM7_NEED_DESCRING_FLUSH) > + e1000_flush_rx_ring(dev); > + } > +} > -- > 2.7.4