[dpdk-dev] [PATCH] ixgbe: fix link down issue on x550em_x

[dpdk-dev] [PATCH] ixgbe: fix link down issue on x550em_x

[Wenzhuo Lu]

Normally the auto-negotiation is supported by FW. But on
X550EM_X_10G_T it's not supported by FW. As the port of
X550EM_X_10G_T is 10G. If we connect the port with a peer
which is 1G. The link is always down.
We have to supprted auto-neg by SW to avoid such link down
issue.

Signed-off-by: root <root@localhost.localdomain>
---
 doc/guides/rel_notes/release_2_3.rst |  6 ++++++
 drivers/net/ixgbe/ixgbe_ethdev.c     | 38 ++++++++++++++++++++++++++++++++++++
 drivers/net/ixgbe/ixgbe_ethdev.h     |  1 +
 3 files changed, 45 insertions(+)

diff --git a/doc/guides/rel_notes/release_2_3.rst b/doc/guides/rel_notes/release_2_3.rst
index 99de186..a8d34d1 100644
--- a/doc/guides/rel_notes/release_2_3.rst
+++ b/doc/guides/rel_notes/release_2_3.rst
@@ -15,6 +15,12 @@ EAL
 Drivers
 ~~~~~~~
 
+* **ixgbe: fix link down issue on X550EM_X.**
+  Normally the auto-negotiation is supported by FW. SW need not care about
+  that. But on x550em_x, FW doesn't support auto-neg. As the ports of x550em_x
+  are 10G, if we connect the port will a peer which is 1G, the link will always
+  be donw on x550em_x.
+  We will support auto-neg by SW to avoid this link down issue.
 
 Libraries
 ~~~~~~~~~
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 4c4c6df..a71c49f 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1961,6 +1961,25 @@ ixgbe_dev_configure(struct rte_eth_dev *dev)
 	return 0;
 }
 
+static void
+ixgbe_dev_phy_intr_setup(struct rte_eth_dev *dev)
+{
+	struct ixgbe_hw *hw =
+		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct ixgbe_interrupt *intr =
+		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
+	uint32_t gpie;
+
+	/* only set up it on X550EM_X */
+	if (hw->mac.type == ixgbe_mac_X550EM_x) {
+		gpie = IXGBE_READ_REG(hw, IXGBE_GPIE);
+		gpie |= IXGBE_SDP0_GPIEN_X550EM_x;
+		IXGBE_WRITE_REG(hw, IXGBE_GPIE, gpie);
+		if (hw->phy.type == ixgbe_phy_x550em_ext_t)
+			intr->mask |= IXGBE_EICR_GPI_SDP0_X550EM_x;
+	}
+}
+
 /*
  * Configure device link speed and setup link.
  * It returns 0 on success.
@@ -2009,6 +2028,8 @@ ixgbe_dev_start(struct rte_eth_dev *dev)
 	/* configure PF module if SRIOV enabled */
 	ixgbe_pf_host_configure(dev);
 
+	ixgbe_dev_phy_intr_setup(dev);
+
 	/* check and configure queue intr-vector mapping */
 	if ((rte_intr_cap_multiple(intr_handle) ||
 	     !RTE_ETH_DEV_SRIOV(dev).active) &&
@@ -3082,6 +3103,11 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)
 	if (eicr & IXGBE_EICR_MAILBOX)
 		intr->flags |= IXGBE_FLAG_MAILBOX;
 
+	if (hw->mac.type ==  ixgbe_mac_X550EM_x &&
+	    hw->phy.type == ixgbe_phy_x550em_ext_t &&
+	    (eicr & IXGBE_EICR_GPI_SDP0_X550EM_x))
+		intr->flags |= IXGBE_FLAG_PHY_INTERRUPT;
+
 	return 0;
 }
 
@@ -3137,6 +3163,8 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)
 	int64_t timeout;
 	struct rte_eth_link link;
 	int intr_enable_delay = false;
+	struct ixgbe_hw *hw =
+		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 
 	PMD_DRV_LOG(DEBUG, "intr action type %d", intr->flags);
 
@@ -3145,6 +3173,11 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)
 		intr->flags &= ~IXGBE_FLAG_MAILBOX;
 	}
 
+	if (intr->flags & IXGBE_FLAG_PHY_INTERRUPT) {
+		ixgbe_handle_lasi(hw);
+		intr->flags &= ~IXGBE_FLAG_PHY_INTERRUPT;
+	}
+
 	if (intr->flags & IXGBE_FLAG_NEED_LINK_UPDATE) {
 		/* get the link status before link update, for predicting later */
 		memset(&link, 0, sizeof(link));
@@ -3208,6 +3241,11 @@ ixgbe_dev_interrupt_delayed_handler(void *param)
 	if (eicr & IXGBE_EICR_MAILBOX)
 		ixgbe_pf_mbx_process(dev);
 
+	if (intr->flags & IXGBE_FLAG_PHY_INTERRUPT) {
+		ixgbe_handle_lasi(hw);
+		intr->flags &= ~IXGBE_FLAG_PHY_INTERRUPT;
+	}
+
 	if (intr->flags & IXGBE_FLAG_NEED_LINK_UPDATE) {
 		ixgbe_dev_link_update(dev, 0);
 		intr->flags &= ~IXGBE_FLAG_NEED_LINK_UPDATE;
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.h b/drivers/net/ixgbe/ixgbe_ethdev.h
index d26771a..5c3aa16 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/ixgbe/ixgbe_ethdev.h
@@ -42,6 +42,7 @@
 /* need update link, bit flag */
 #define IXGBE_FLAG_NEED_LINK_UPDATE (uint32_t)(1 << 0)
 #define IXGBE_FLAG_MAILBOX          (uint32_t)(1 << 1)
+#define IXGBE_FLAG_PHY_INTERRUPT    (uint32_t)(1 << 2)
 
 /*
  * Defines that were not part of ixgbe_type.h as they are not used by the
-- 
1.9.3

Re: [dpdk-dev] [PATCH] ixgbe: fix link down issue on x550em_x

[He, Shaopeng]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu
> Sent: Monday, December 21, 2015 3:57 PM
> To: dev@dpdk.org
> Cc: root
> Subject: [dpdk-dev] [PATCH] ixgbe: fix link down issue on x550em_x
> 
> Normally the auto-negotiation is supported by FW. But on
> X550EM_X_10G_T it's not supported by FW. As the port of
> X550EM_X_10G_T is 10G. If we connect the port with a peer
> which is 1G. The link is always down.
> We have to supprted auto-neg by SW to avoid such link down
> issue.
> 
> Signed-off-by: root <root@localhost.localdomain>

Should be "Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>" ?

Re: [dpdk-dev] [PATCH] ixgbe: fix link down issue on x550em_x

[Lu, Wenzhuo]

Hi Shaopeng,

> -----Original Message-----
> From: He, Shaopeng
> Sent: Monday, February 1, 2016 4:23 PM
> To: Lu, Wenzhuo; dev@dpdk.org
> Cc: root
> Subject: RE: [dpdk-dev] [PATCH] ixgbe: fix link down issue on x550em_x
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu
> > Sent: Monday, December 21, 2015 3:57 PM
> > To: dev@dpdk.org
> > Cc: root
> > Subject: [dpdk-dev] [PATCH] ixgbe: fix link down issue on x550em_x
> >
> > Normally the auto-negotiation is supported by FW. But on
> > X550EM_X_10G_T it's not supported by FW. As the port of X550EM_X_10G_T
> > is 10G. If we connect the port with a peer which is 1G. The link is
> > always down.
> > We have to supprted auto-neg by SW to avoid such link down issue.
> >
> > Signed-off-by: root <root@localhost.localdomain>
> 
> Should be "Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>" ?
Yes, thanks. Seems I've used the wrong machine to create this patch. I'll send a V2.

[dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x

[Wenzhuo Lu]

Normally the auto-negotiation is supported by FW. But on
X550EM_X_10G_T it's not supported by FW. As the port of
X550EM_X_10G_T is 10G. If we connect the port with a peer
which is 1G. The link is always down.
We have to supprted auto-neg by SW to avoid such link down
issue.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
---
 doc/guides/rel_notes/release_2_3.rst |  6 ++++++
 drivers/net/ixgbe/ixgbe_ethdev.c     | 38 ++++++++++++++++++++++++++++++++++++
 drivers/net/ixgbe/ixgbe_ethdev.h     |  1 +
 3 files changed, 45 insertions(+)

diff --git a/doc/guides/rel_notes/release_2_3.rst b/doc/guides/rel_notes/release_2_3.rst
index 99de186..a8d34d1 100644
--- a/doc/guides/rel_notes/release_2_3.rst
+++ b/doc/guides/rel_notes/release_2_3.rst
@@ -15,6 +15,12 @@ EAL
 Drivers
 ~~~~~~~
 
+* **ixgbe: fix link down issue on X550EM_X.**
+  Normally the auto-negotiation is supported by FW. SW need not care about
+  that. But on x550em_x, FW doesn't support auto-neg. As the ports of x550em_x
+  are 10G, if we connect the port will a peer which is 1G, the link will always
+  be donw on x550em_x.
+  We will support auto-neg by SW to avoid this link down issue.
 
 Libraries
 ~~~~~~~~~
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 4c4c6df..a71c49f 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1961,6 +1961,25 @@ ixgbe_dev_configure(struct rte_eth_dev *dev)
 	return 0;
 }
 
+static void
+ixgbe_dev_phy_intr_setup(struct rte_eth_dev *dev)
+{
+	struct ixgbe_hw *hw =
+		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct ixgbe_interrupt *intr =
+		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
+	uint32_t gpie;
+
+	/* only set up it on X550EM_X */
+	if (hw->mac.type == ixgbe_mac_X550EM_x) {
+		gpie = IXGBE_READ_REG(hw, IXGBE_GPIE);
+		gpie |= IXGBE_SDP0_GPIEN_X550EM_x;
+		IXGBE_WRITE_REG(hw, IXGBE_GPIE, gpie);
+		if (hw->phy.type == ixgbe_phy_x550em_ext_t)
+			intr->mask |= IXGBE_EICR_GPI_SDP0_X550EM_x;
+	}
+}
+
 /*
  * Configure device link speed and setup link.
  * It returns 0 on success.
@@ -2009,6 +2028,8 @@ ixgbe_dev_start(struct rte_eth_dev *dev)
 	/* configure PF module if SRIOV enabled */
 	ixgbe_pf_host_configure(dev);
 
+	ixgbe_dev_phy_intr_setup(dev);
+
 	/* check and configure queue intr-vector mapping */
 	if ((rte_intr_cap_multiple(intr_handle) ||
 	     !RTE_ETH_DEV_SRIOV(dev).active) &&
@@ -3082,6 +3103,11 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)
 	if (eicr & IXGBE_EICR_MAILBOX)
 		intr->flags |= IXGBE_FLAG_MAILBOX;
 
+	if (hw->mac.type ==  ixgbe_mac_X550EM_x &&
+	    hw->phy.type == ixgbe_phy_x550em_ext_t &&
+	    (eicr & IXGBE_EICR_GPI_SDP0_X550EM_x))
+		intr->flags |= IXGBE_FLAG_PHY_INTERRUPT;
+
 	return 0;
 }
 
@@ -3137,6 +3163,8 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)
 	int64_t timeout;
 	struct rte_eth_link link;
 	int intr_enable_delay = false;
+	struct ixgbe_hw *hw =
+		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 
 	PMD_DRV_LOG(DEBUG, "intr action type %d", intr->flags);
 
@@ -3145,6 +3173,11 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)
 		intr->flags &= ~IXGBE_FLAG_MAILBOX;
 	}
 
+	if (intr->flags & IXGBE_FLAG_PHY_INTERRUPT) {
+		ixgbe_handle_lasi(hw);
+		intr->flags &= ~IXGBE_FLAG_PHY_INTERRUPT;
+	}
+
 	if (intr->flags & IXGBE_FLAG_NEED_LINK_UPDATE) {
 		/* get the link status before link update, for predicting later */
 		memset(&link, 0, sizeof(link));
@@ -3208,6 +3241,11 @@ ixgbe_dev_interrupt_delayed_handler(void *param)
 	if (eicr & IXGBE_EICR_MAILBOX)
 		ixgbe_pf_mbx_process(dev);
 
+	if (intr->flags & IXGBE_FLAG_PHY_INTERRUPT) {
+		ixgbe_handle_lasi(hw);
+		intr->flags &= ~IXGBE_FLAG_PHY_INTERRUPT;
+	}
+
 	if (intr->flags & IXGBE_FLAG_NEED_LINK_UPDATE) {
 		ixgbe_dev_link_update(dev, 0);
 		intr->flags &= ~IXGBE_FLAG_NEED_LINK_UPDATE;
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.h b/drivers/net/ixgbe/ixgbe_ethdev.h
index d26771a..5c3aa16 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/ixgbe/ixgbe_ethdev.h
@@ -42,6 +42,7 @@
 /* need update link, bit flag */
 #define IXGBE_FLAG_NEED_LINK_UPDATE (uint32_t)(1 << 0)
 #define IXGBE_FLAG_MAILBOX          (uint32_t)(1 << 1)
+#define IXGBE_FLAG_PHY_INTERRUPT    (uint32_t)(1 << 2)
 
 /*
  * Defines that were not part of ixgbe_type.h as they are not used by the
-- 
1.9.3

Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x

[He, Shaopeng]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu
> Sent: Monday, February 01, 2016 4:43 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x
> 
> Normally the auto-negotiation is supported by FW. But on
> X550EM_X_10G_T it's not supported by FW. As the port of
> X550EM_X_10G_T is 10G. If we connect the port with a peer
> which is 1G. The link is always down.
> We have to supprted auto-neg by SW to avoid such link down
> issue.
> 
> Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Shaopeng He <shaopeng.he@intel.com>

Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x

[Bruce Richardson]

On Thu, Feb 04, 2016 at 06:21:04AM +0000, He, Shaopeng wrote:
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu
> > Sent: Monday, February 01, 2016 4:43 PM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x
> > 
> > Normally the auto-negotiation is supported by FW. But on
> > X550EM_X_10G_T it's not supported by FW. As the port of
> > X550EM_X_10G_T is 10G. If we connect the port with a peer
> > which is 1G. The link is always down.
> > We have to supprted auto-neg by SW to avoid such link down
> > issue.
> > 
> > Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> Acked-by: Shaopeng He <shaopeng.he@intel.com>
> 
I'm a bit confused regarding the commit message and the code in the patch.
The commit message talks about enabling speed auto-negotiation, while the code
never refers to any such thing. Instead all we have are settings for manipulating
interrupt masks to enable PHY interrupts. I think some additional information is
needed to connect A and B together here.

A second, more minor nit is that the commit title never refers to link
auto-negotiation, but refers to this as a bug fix - which is also correct. If
this is primarily a bug fix, please include a fixes line if possible, but please
also refer to auto-neg in the title if possible too.

/Bruce

Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x

[Bruce Richardson]

On Mon, Feb 01, 2016 at 04:42:39PM +0800, Wenzhuo Lu wrote:
> Normally the auto-negotiation is supported by FW. But on
> X550EM_X_10G_T it's not supported by FW. As the port of
> X550EM_X_10G_T is 10G. If we connect the port with a peer
> which is 1G. The link is always down.
> We have to supprted auto-neg by SW to avoid such link down
> issue.
> 
> Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> ---
>  doc/guides/rel_notes/release_2_3.rst |  6 ++++++
>  drivers/net/ixgbe/ixgbe_ethdev.c     | 38 ++++++++++++++++++++++++++++++++++++
>  drivers/net/ixgbe/ixgbe_ethdev.h     |  1 +
>  3 files changed, 45 insertions(+)
> 
> diff --git a/doc/guides/rel_notes/release_2_3.rst b/doc/guides/rel_notes/release_2_3.rst
> index 99de186..a8d34d1 100644
> --- a/doc/guides/rel_notes/release_2_3.rst
> +++ b/doc/guides/rel_notes/release_2_3.rst
> @@ -15,6 +15,12 @@ EAL
>  Drivers
>  ~~~~~~~
>  
> +* **ixgbe: fix link down issue on X550EM_X.**
> +  Normally the auto-negotiation is supported by FW. SW need not care about
> +  that. But on x550em_x, FW doesn't support auto-neg. As the ports of x550em_x
> +  are 10G, if we connect the port will a peer which is 1G, the link will always
> +  be donw on x550em_x.
> +  We will support auto-neg by SW to avoid this link down issue.
>  

Couple of typos present in the text above.

/Bruce

Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x

[Lu, Wenzhuo]

Hi Bruce,

> -----Original Message-----
> From: Richardson, Bruce
> Sent: Wednesday, February 24, 2016 10:26 PM
> To: He, Shaopeng
> Cc: Lu, Wenzhuo; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x
> 
> On Thu, Feb 04, 2016 at 06:21:04AM +0000, He, Shaopeng wrote:
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu
> > > Sent: Monday, February 01, 2016 4:43 PM
> > > To: dev@dpdk.org
> > > Subject: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on
> > > x550em_x
> > >
> > > Normally the auto-negotiation is supported by FW. But on
> > > X550EM_X_10G_T it's not supported by FW. As the port of
> > > X550EM_X_10G_T is 10G. If we connect the port with a peer which is
> > > 1G. The link is always down.
> > > We have to supprted auto-neg by SW to avoid such link down issue.
> > >
> > > Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > Acked-by: Shaopeng He <shaopeng.he@intel.com>
> >
> I'm a bit confused regarding the commit message and the code in the patch.
> The commit message talks about enabling speed auto-negotiation, while the
> code never refers to any such thing. Instead all we have are settings for
> manipulating interrupt masks to enable PHY interrupts. I think some additional
> information is needed to connect A and B together here.
The reason is that the handler of the link speed auto-neg is already in the base code. It's ixgbe_handle_lasi.
What we need is a trigger. When the advertised link speed changes, a PHY interruption will be triggered. So, we should handle this interruption and call ixgbe_handle_lasi to set the link speed correct.
Let me add more explanation in the commit log.

> 
> A second, more minor nit is that the commit title never refers to link auto-
> negotiation, but refers to this as a bug fix - which is also correct. If this is
> primarily a bug fix, please include a fixes line if possible, but please also refer to
> auto-neg in the title if possible too.
I didn't add a fixes line because it doesn't fix an issue introduced by SW, or even FW, HW. As said in commit log, we hit a link down issue, but the root cause is the link speed auto-neg is not supported on this specific NIC.
On the other NICs, link speed auto-neg is supported by FW, SW need no do anything. So we didn't implement the link speed auto-neg. But we have to implement this feature on this NIC for FW doesn't implement it.
Should I change the tittle to "support link speed auto-neg on x550em_x"?

> 
> /Bruce

Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x

[Lu, Wenzhuo]

Hi Bruce,

> -----Original Message-----
> From: Richardson, Bruce
> Sent: Wednesday, February 24, 2016 10:27 PM
> To: Lu, Wenzhuo
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x
> 
> On Mon, Feb 01, 2016 at 04:42:39PM +0800, Wenzhuo Lu wrote:
> > Normally the auto-negotiation is supported by FW. But on
> > X550EM_X_10G_T it's not supported by FW. As the port of X550EM_X_10G_T
> > is 10G. If we connect the port with a peer which is 1G. The link is
> > always down.
> > We have to supprted auto-neg by SW to avoid such link down issue.
> >
> > Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > ---
> >  doc/guides/rel_notes/release_2_3.rst |  6 ++++++
> >  drivers/net/ixgbe/ixgbe_ethdev.c     | 38
> ++++++++++++++++++++++++++++++++++++
> >  drivers/net/ixgbe/ixgbe_ethdev.h     |  1 +
> >  3 files changed, 45 insertions(+)
> >
> > diff --git a/doc/guides/rel_notes/release_2_3.rst
> > b/doc/guides/rel_notes/release_2_3.rst
> > index 99de186..a8d34d1 100644
> > --- a/doc/guides/rel_notes/release_2_3.rst
> > +++ b/doc/guides/rel_notes/release_2_3.rst
> > @@ -15,6 +15,12 @@ EAL
> >  Drivers
> >  ~~~~~~~
> >
> > +* **ixgbe: fix link down issue on X550EM_X.**
> > +  Normally the auto-negotiation is supported by FW. SW need not care
> > +about
> > +  that. But on x550em_x, FW doesn't support auto-neg. As the ports of
> > +x550em_x
> > +  are 10G, if we connect the port will a peer which is 1G, the link
> > +will always
> > +  be donw on x550em_x.
> > +  We will support auto-neg by SW to avoid this link down issue.
> >
> 
> Couple of typos present in the text above.
Sorry for that. I'll correct them. Thanks.

> 
> /Bruce

Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x

[Bruce Richardson]

On Thu, Feb 25, 2016 at 01:32:33AM +0000, Lu, Wenzhuo wrote:
> Hi Bruce,
> 
> > -----Original Message-----
> > From: Richardson, Bruce
> > Sent: Wednesday, February 24, 2016 10:26 PM
> > To: He, Shaopeng
> > Cc: Lu, Wenzhuo; dev@dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x
> > 
> > On Thu, Feb 04, 2016 at 06:21:04AM +0000, He, Shaopeng wrote:
> > >
> > > > -----Original Message-----
> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu
> > > > Sent: Monday, February 01, 2016 4:43 PM
> > > > To: dev@dpdk.org
> > > > Subject: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on
> > > > x550em_x
> > > >
> > > > Normally the auto-negotiation is supported by FW. But on
> > > > X550EM_X_10G_T it's not supported by FW. As the port of
> > > > X550EM_X_10G_T is 10G. If we connect the port with a peer which is
> > > > 1G. The link is always down.
> > > > We have to supprted auto-neg by SW to avoid such link down issue.
> > > >
> > > > Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > > Acked-by: Shaopeng He <shaopeng.he@intel.com>
> > >
> > I'm a bit confused regarding the commit message and the code in the patch.
> > The commit message talks about enabling speed auto-negotiation, while the
> > code never refers to any such thing. Instead all we have are settings for
> > manipulating interrupt masks to enable PHY interrupts. I think some additional
> > information is needed to connect A and B together here.
> The reason is that the handler of the link speed auto-neg is already in the base code. It's ixgbe_handle_lasi.
> What we need is a trigger. When the advertised link speed changes, a PHY interruption will be triggered. So, we should handle this interruption and call ixgbe_handle_lasi to set the link speed correct.
> Let me add more explanation in the commit log.
> 
Yes, please do. What you have just explained makes much more sense and should
be included in the log.

> > 
> > A second, more minor nit is that the commit title never refers to link auto-
> > negotiation, but refers to this as a bug fix - which is also correct. If this is
> > primarily a bug fix, please include a fixes line if possible, but please also refer to
> > auto-neg in the title if possible too.
> I didn't add a fixes line because it doesn't fix an issue introduced by SW, or even FW, HW. As said in commit log, we hit a link down issue, but the root cause is the link speed auto-neg is not supported on this specific NIC.
> On the other NICs, link speed auto-neg is supported by FW, SW need no do anything. So we didn't implement the link speed auto-neg. But we have to implement this feature on this NIC for FW doesn't implement it.
> Should I change the tittle to "support link speed auto-neg on x550em_x"?
> 

Yes, that is probably better.

Thanks,
/Bruce

Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x

[Lu, Wenzhuo]

Hi Bruce,


> -----Original Message-----
> From: Richardson, Bruce
> Sent: Thursday, February 25, 2016 9:30 PM
> To: Lu, Wenzhuo
> Cc: He, Shaopeng; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on x550em_x
> 
> On Thu, Feb 25, 2016 at 01:32:33AM +0000, Lu, Wenzhuo wrote:
> > Hi Bruce,
> >
> > > -----Original Message-----
> > > From: Richardson, Bruce
> > > Sent: Wednesday, February 24, 2016 10:26 PM
> > > To: He, Shaopeng
> > > Cc: Lu, Wenzhuo; dev@dpdk.org
> > > Subject: Re: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on
> > > x550em_x
> > >
> > > On Thu, Feb 04, 2016 at 06:21:04AM +0000, He, Shaopeng wrote:
> > > >
> > > > > -----Original Message-----
> > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wenzhuo Lu
> > > > > Sent: Monday, February 01, 2016 4:43 PM
> > > > > To: dev@dpdk.org
> > > > > Subject: [dpdk-dev] [PATCH v2] ixgbe: fix link down issue on
> > > > > x550em_x
> > > > >
> > > > > Normally the auto-negotiation is supported by FW. But on
> > > > > X550EM_X_10G_T it's not supported by FW. As the port of
> > > > > X550EM_X_10G_T is 10G. If we connect the port with a peer which
> > > > > is 1G. The link is always down.
> > > > > We have to supprted auto-neg by SW to avoid such link down issue.
> > > > >
> > > > > Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> > > > Acked-by: Shaopeng He <shaopeng.he@intel.com>
> > > >
> > > I'm a bit confused regarding the commit message and the code in the patch.
> > > The commit message talks about enabling speed auto-negotiation,
> > > while the code never refers to any such thing. Instead all we have
> > > are settings for manipulating interrupt masks to enable PHY
> > > interrupts. I think some additional information is needed to connect A and B
> together here.
> > The reason is that the handler of the link speed auto-neg is already in the base
> code. It's ixgbe_handle_lasi.
> > What we need is a trigger. When the advertised link speed changes, a PHY
> interruption will be triggered. So, we should handle this interruption and call
> ixgbe_handle_lasi to set the link speed correct.
> > Let me add more explanation in the commit log.
> >
> Yes, please do. What you have just explained makes much more sense and
> should be included in the log.
> 
> > >
> > > A second, more minor nit is that the commit title never refers to
> > > link auto- negotiation, but refers to this as a bug fix - which is
> > > also correct. If this is primarily a bug fix, please include a fixes
> > > line if possible, but please also refer to auto-neg in the title if possible too.
> > I didn't add a fixes line because it doesn't fix an issue introduced by SW, or
> even FW, HW. As said in commit log, we hit a link down issue, but the root cause
> is the link speed auto-neg is not supported on this specific NIC.
> > On the other NICs, link speed auto-neg is supported by FW, SW need no do
> anything. So we didn't implement the link speed auto-neg. But we have to
> implement this feature on this NIC for FW doesn't implement it.
> > Should I change the tittle to "support link speed auto-neg on x550em_x"?
> >
> 
> Yes, that is probably better.
Thanks for the comments. I'll send a new version.

> 
> Thanks,
> /Bruce

[dpdk-dev] [PATCH v3] ixgbe: support link speed auto-neg on x550em_x

[Wenzhuo Lu]

Normally the auto-negotiation is supported by FW. SW need not care about
that. But on x550em_x, FW doesn't support auto-neg. As the ports of x550em_x
are 10G, if we connect the port will a peer which is 1G, the link will always
be down.
We need support auto-neg by SW to avoid this link down issue. As we already
have the code to handle the link speed setting, what we need is a trigger.
When the advertised link speed changes, a PHY interruption will be triggered.
So, we should handle this interruption and call ixgbe_handle_lasi to set the
link speed correctly.

Please be aware it's working when auto-neg is on. If the auto-neg of the peer
port is turned off and its speed is indicated manually, we should also set
the speed of our own port manually.

V2:
 *Fix the wrong signoff name.

V3:
 *Change the tittle and add the explanation of the implementation in commit log.

Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Acked-by: Shaopeng He <shaopeng.he@intel.com>
---
 doc/guides/rel_notes/release_16_04.rst |  8 +++++++
 drivers/net/ixgbe/ixgbe_ethdev.c       | 38 ++++++++++++++++++++++++++++++++++
 drivers/net/ixgbe/ixgbe_ethdev.h       |  1 +
 3 files changed, 47 insertions(+)

diff --git a/doc/guides/rel_notes/release_16_04.rst b/doc/guides/rel_notes/release_16_04.rst
index 8273817..fbdf286 100644
--- a/doc/guides/rel_notes/release_16_04.rst
+++ b/doc/guides/rel_notes/release_16_04.rst
@@ -46,6 +46,14 @@ This section should contain new features added in this release. Sample format:
 
 * **Added vhost-user live migration support.**
 
+* **Supported link speed auto-negotiation on X550EM_X**
+
+  Normally the auto-negotiation is supported by FW. SW need not care about
+  that. But on x550em_x, FW doesn't support auto-neg. As the ports of x550em_x
+  are 10G, if we connect the port will a peer which is 1G, the link will always
+  be down.
+  We added the support of auto-neg by SW to avoid this link down issue.
+
 
 Resolved Issues
 ---------------
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 3e6fe86..2384ba7 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -1961,6 +1961,25 @@ ixgbe_dev_configure(struct rte_eth_dev *dev)
 	return 0;
 }
 
+static void
+ixgbe_dev_phy_intr_setup(struct rte_eth_dev *dev)
+{
+	struct ixgbe_hw *hw =
+		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct ixgbe_interrupt *intr =
+		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
+	uint32_t gpie;
+
+	/* only set up it on X550EM_X */
+	if (hw->mac.type == ixgbe_mac_X550EM_x) {
+		gpie = IXGBE_READ_REG(hw, IXGBE_GPIE);
+		gpie |= IXGBE_SDP0_GPIEN_X550EM_x;
+		IXGBE_WRITE_REG(hw, IXGBE_GPIE, gpie);
+		if (hw->phy.type == ixgbe_phy_x550em_ext_t)
+			intr->mask |= IXGBE_EICR_GPI_SDP0_X550EM_x;
+	}
+}
+
 /*
  * Configure device link speed and setup link.
  * It returns 0 on success.
@@ -2009,6 +2028,8 @@ ixgbe_dev_start(struct rte_eth_dev *dev)
 	/* configure PF module if SRIOV enabled */
 	ixgbe_pf_host_configure(dev);
 
+	ixgbe_dev_phy_intr_setup(dev);
+
 	/* check and configure queue intr-vector mapping */
 	if ((rte_intr_cap_multiple(intr_handle) ||
 	     !RTE_ETH_DEV_SRIOV(dev).active) &&
@@ -3082,6 +3103,11 @@ ixgbe_dev_interrupt_get_status(struct rte_eth_dev *dev)
 	if (eicr & IXGBE_EICR_MAILBOX)
 		intr->flags |= IXGBE_FLAG_MAILBOX;
 
+	if (hw->mac.type ==  ixgbe_mac_X550EM_x &&
+	    hw->phy.type == ixgbe_phy_x550em_ext_t &&
+	    (eicr & IXGBE_EICR_GPI_SDP0_X550EM_x))
+		intr->flags |= IXGBE_FLAG_PHY_INTERRUPT;
+
 	return 0;
 }
 
@@ -3137,6 +3163,8 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)
 	int64_t timeout;
 	struct rte_eth_link link;
 	int intr_enable_delay = false;
+	struct ixgbe_hw *hw =
+		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 
 	PMD_DRV_LOG(DEBUG, "intr action type %d", intr->flags);
 
@@ -3145,6 +3173,11 @@ ixgbe_dev_interrupt_action(struct rte_eth_dev *dev)
 		intr->flags &= ~IXGBE_FLAG_MAILBOX;
 	}
 
+	if (intr->flags & IXGBE_FLAG_PHY_INTERRUPT) {
+		ixgbe_handle_lasi(hw);
+		intr->flags &= ~IXGBE_FLAG_PHY_INTERRUPT;
+	}
+
 	if (intr->flags & IXGBE_FLAG_NEED_LINK_UPDATE) {
 		/* get the link status before link update, for predicting later */
 		memset(&link, 0, sizeof(link));
@@ -3208,6 +3241,11 @@ ixgbe_dev_interrupt_delayed_handler(void *param)
 	if (eicr & IXGBE_EICR_MAILBOX)
 		ixgbe_pf_mbx_process(dev);
 
+	if (intr->flags & IXGBE_FLAG_PHY_INTERRUPT) {
+		ixgbe_handle_lasi(hw);
+		intr->flags &= ~IXGBE_FLAG_PHY_INTERRUPT;
+	}
+
 	if (intr->flags & IXGBE_FLAG_NEED_LINK_UPDATE) {
 		ixgbe_dev_link_update(dev, 0);
 		intr->flags &= ~IXGBE_FLAG_NEED_LINK_UPDATE;
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.h b/drivers/net/ixgbe/ixgbe_ethdev.h
index d26771a..5c3aa16 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/ixgbe/ixgbe_ethdev.h
@@ -42,6 +42,7 @@
 /* need update link, bit flag */
 #define IXGBE_FLAG_NEED_LINK_UPDATE (uint32_t)(1 << 0)
 #define IXGBE_FLAG_MAILBOX          (uint32_t)(1 << 1)
+#define IXGBE_FLAG_PHY_INTERRUPT    (uint32_t)(1 << 2)
 
 /*
  * Defines that were not part of ixgbe_type.h as they are not used by the
-- 
1.9.3

Re: [dpdk-dev] [PATCH v3] ixgbe: support link speed auto-neg on x550em_x

[Bruce Richardson]

On Fri, Feb 26, 2016 at 11:05:29AM +0800, Wenzhuo Lu wrote:
> Normally the auto-negotiation is supported by FW. SW need not care about
> that. But on x550em_x, FW doesn't support auto-neg. As the ports of x550em_x
> are 10G, if we connect the port will a peer which is 1G, the link will always
> be down.
> We need support auto-neg by SW to avoid this link down issue. As we already
> have the code to handle the link speed setting, what we need is a trigger.
> When the advertised link speed changes, a PHY interruption will be triggered.
> So, we should handle this interruption and call ixgbe_handle_lasi to set the
> link speed correctly.
> 
> Please be aware it's working when auto-neg is on. If the auto-neg of the peer
> port is turned off and its speed is indicated manually, we should also set
> the speed of our own port manually.
> 
> V2:
>  *Fix the wrong signoff name.
> 
> V3:
>  *Change the tittle and add the explanation of the implementation in commit log.
> 
> Signed-off-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
> Acked-by: Shaopeng He <shaopeng.he@intel.com>
> ---
Applied to dpdk-next-net/rel_16_04

/Bruce