[dpdk-dev] [PATCH 0/4] net/fm10k: support switch restart

[dpdk-dev] [PATCH 0/4] net/fm10k: support switch restart

[Xiao Wang]

For fm10k multi host based design, DPDK app running in VM or host needs to
get aware of the switch's state because switch may undergo a quit-restart.
This patch series supports switch restart feature for DPDK app to be able
to resume its packet processing without an app-level restart.

Xiao Wang (4):
  net/fm10k: redefine link status semantics
  net/fm10k: support switch restart on PF
  net/fm10k: support switch restart on VF
  doc: add switch restart support to fm10k guide

 doc/guides/nics/fm10k.rst        |  10 ++++
 drivers/net/fm10k/fm10k.h        |   1 +
 drivers/net/fm10k/fm10k_ethdev.c | 106 ++++++++++++++++++++++++++++++++++++---
 3 files changed, 111 insertions(+), 6 deletions(-)

-- 
1.8.3.1

[dpdk-dev] [PATCH 1/4] net/fm10k: redefine link status semantics

[Xiao Wang]

As fm10k host interface is not directly connected to PHY, marking the link
status as UP doesn't mean a lot to the application. So, this patch
basically redefines the link status as the state of switch manager: when
switch manager is running, it's LINK_UP; when switch manager goes down by
calling the fmTerminate function, status turns to LINK_DOWN.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
 drivers/net/fm10k/fm10k.h        |  1 +
 drivers/net/fm10k/fm10k_ethdev.c | 11 +++++++----
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/net/fm10k/fm10k.h b/drivers/net/fm10k/fm10k.h
index 060982b..594dca4 100644
--- a/drivers/net/fm10k/fm10k.h
+++ b/drivers/net/fm10k/fm10k.h
@@ -155,6 +155,7 @@ struct fm10k_dev_info {
 	struct fm10k_macvlan_filter_info    macvlan;
 	/* Flag to indicate if RX vector conditions satisfied */
 	bool rx_vec_allowed;
+	bool sm_down;
 };
 
 /*
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 586f482..2587696 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1256,14 +1256,17 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
 fm10k_link_update(struct rte_eth_dev *dev,
 	__rte_unused int wait_to_complete)
 {
+	struct fm10k_dev_info *dev_info =
+		FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
 	PMD_INIT_FUNC_TRACE();
 
-	/* The host-interface link is always up.  The speed is ~50Gbps per Gen3
-	 * x8 PCIe interface. For now, we leave the speed undefined since there
-	 * is no 50Gbps Ethernet. */
+	/* The speed is ~50Gbps per Gen3 x8 PCIe interface. For now, we
+	 * leave the speed undefined since there is no 50Gbps Ethernet.
+	 */
 	dev->data->dev_link.link_speed  = 0;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX;
-	dev->data->dev_link.link_status = ETH_LINK_UP;
+	dev->data->dev_link.link_status =
+		dev_info->sm_down ? ETH_LINK_DOWN : ETH_LINK_UP;
 
 	return 0;
 }
-- 
1.8.3.1

[dpdk-dev] [PATCH 2/4] net/fm10k: support switch restart on PF

[Xiao Wang]

For events indicating the change in the state of the switch manager. The
driver will restore the basic port configurations and then pass this event
to application so that the application can restore any additional
configurations if required. In this way, once the switch manager restarts,
the DPDK application can resume its network.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
 drivers/net/fm10k/fm10k_ethdev.c | 54 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 52 insertions(+), 2 deletions(-)

diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 2587696..898bd5d 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2556,6 +2556,10 @@ static void fm10k_MAC_filter_set(struct rte_eth_dev *dev,
 	struct rte_eth_dev *dev = (struct rte_eth_dev *)param;
 	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	uint32_t cause, status;
+	struct fm10k_dev_info *dev_info =
+		FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
+	int status_mbx;
+	s32 err;
 
 	if (hw->mac.type != fm10k_mac_pf)
 		return;
@@ -2572,14 +2576,60 @@ static void fm10k_MAC_filter_set(struct rte_eth_dev *dev,
 	if (cause & FM10K_EICR_SWITCHNOTREADY)
 		PMD_INIT_LOG(ERR, "INT: Switch is not ready");
 
-	if (cause & FM10K_EICR_SWITCHREADY)
+	if (cause & FM10K_EICR_SWITCHREADY) {
 		PMD_INIT_LOG(INFO, "INT: Switch is ready");
+		if (dev_info->sm_down == 1) {
+			fm10k_mbx_lock(hw);
+
+			/* For recreating logical ports */
+			status_mbx = hw->mac.ops.update_lport_state(hw,
+							hw->mac.dglort_map, MAX_LPORT_NUM, 1);
+			if (status_mbx == FM10K_SUCCESS)
+				PMD_INIT_LOG(INFO, "INT: Recreated Logical port");
+			else
+				PMD_INIT_LOG(INFO, "INT: Logical ports weren't recreated");
+
+			status_mbx = hw->mac.ops.update_xcast_mode(hw,
+					hw->mac.dglort_map, FM10K_XCAST_MODE_NONE);
+			if (status_mbx != FM10K_SUCCESS)
+				PMD_INIT_LOG(ERR, "Failed to set XCAST mode");
+
+			fm10k_mbx_unlock(hw);
+
+			/* first clear the internal SW recording structure */
+			if (!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_VMDQ_FLAG))
+				fm10k_vlan_filter_set(dev, hw->mac.default_vid, false);
+
+			fm10k_MAC_filter_set(dev, hw->mac.addr, false,
+					MAIN_VSI_POOL_NUMBER);
+
+			/* Add default mac address and vlan for the logical ports that
+			 * have been created, leave to the application to fully recover
+			 * Rx filtering.
+			 */
+			fm10k_MAC_filter_set(dev, hw->mac.addr, true,
+					MAIN_VSI_POOL_NUMBER);
+
+			if (!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_VMDQ_FLAG))
+				fm10k_vlan_filter_set(dev, hw->mac.default_vid, true);
+
+			dev_info->sm_down = 0;
+			_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC,
+					NULL, NULL);
+		}
+	}
 
 	/* Handle mailbox message */
 	fm10k_mbx_lock(hw);
-	hw->mbx.ops.process(hw, &hw->mbx);
+	err = hw->mbx.ops.process(hw, &hw->mbx);
 	fm10k_mbx_unlock(hw);
 
+	if (err == FM10K_ERR_RESET_REQUESTED) {
+		PMD_INIT_LOG(INFO, "INT: Switch is down");
+		dev_info->sm_down = 1;
+		_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL, NULL);
+	}
+
 	/* Handle SRAM error */
 	if (cause & FM10K_EICR_SRAMERROR) {
 		PMD_INIT_LOG(ERR, "INT: SRAM error on PEP");
-- 
1.8.3.1

[dpdk-dev] [PATCH 3/4] net/fm10k: support switch restart on VF

[Xiao Wang]

For events indicating a change in the state of the switch manager, the
driver will restore the basic port configurations and then pass this event
to application so that the application can restore any additional
configurations if required. In this way, once the switch manager restarts,
the DPDK application can resume its network.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
 drivers/net/fm10k/fm10k_ethdev.c | 41 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 898bd5d..45ba943 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2670,6 +2670,11 @@ static void fm10k_MAC_filter_set(struct rte_eth_dev *dev,
 {
 	struct rte_eth_dev *dev = (struct rte_eth_dev *)param;
 	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct fm10k_mbx_info *mbx = &hw->mbx;
+	struct fm10k_dev_info *dev_info =
+		FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
+	const enum fm10k_mbx_state state = mbx->state;
+	int status_mbx;
 
 	if (hw->mac.type != fm10k_mac_vf)
 		return;
@@ -2679,6 +2684,42 @@ static void fm10k_MAC_filter_set(struct rte_eth_dev *dev,
 	hw->mbx.ops.process(hw, &hw->mbx);
 	fm10k_mbx_unlock(hw);
 
+	if ((state == FM10K_STATE_OPEN) && (mbx->state == FM10K_STATE_CONNECT)) {
+		PMD_INIT_LOG(INFO, "INT: Switch has gone down");
+
+		fm10k_mbx_lock(hw);
+		hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map, MAX_LPORT_NUM, 1);
+		fm10k_mbx_unlock(hw);
+
+		/* Setting reset flag */
+		dev_info->sm_down = 1;
+		_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL, NULL);
+	}
+
+	if ((dev_info->sm_down == 1) && (hw->mac.dglort_map == FM10K_DGLORTMAP_ZERO)) {
+		PMD_INIT_LOG(INFO, "INT: Switch has gone up");
+		fm10k_mbx_lock(hw);
+		status_mbx = hw->mac.ops.update_xcast_mode(hw, hw->mac.dglort_map,
+				FM10K_XCAST_MODE_NONE);
+		if (status_mbx != FM10K_SUCCESS)
+			PMD_INIT_LOG(ERR, "Failed to set XCAST mode");
+		fm10k_mbx_unlock(hw);
+
+		/* first clear the internal SW recording structure */
+		fm10k_vlan_filter_set(dev, hw->mac.default_vid, false);
+		fm10k_MAC_filter_set(dev, hw->mac.addr, false, MAIN_VSI_POOL_NUMBER);
+
+		/* Add default mac address and vlan for the logical ports that
+		 * have been created, leave to the application to fully recover
+		 * Rx filtering.
+		 */
+		fm10k_MAC_filter_set(dev, hw->mac.addr, true, MAIN_VSI_POOL_NUMBER);
+		fm10k_vlan_filter_set(dev, hw->mac.default_vid, true);
+
+		dev_info->sm_down = 0;
+		_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL, NULL);
+	}
+
 	/* Re-enable interrupt from device side */
 	FM10K_WRITE_REG(hw, FM10K_VFITR(0), FM10K_ITR_AUTOMASK |
 					FM10K_ITR_MASK_CLEAR);
-- 
1.8.3.1

[dpdk-dev] [PATCH 4/4] doc: add switch restart support to fm10k guide

[Xiao Wang]

This patch documents how DPDK app should handle the event of switch
quit-restart to resume its network without an app-level restart.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
 doc/guides/nics/fm10k.rst | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/doc/guides/nics/fm10k.rst b/doc/guides/nics/fm10k.rst
index 7fc4862..51b8326 100644
--- a/doc/guides/nics/fm10k.rst
+++ b/doc/guides/nics/fm10k.rst
@@ -161,6 +161,16 @@ FM10000 PMD driver. The switch driver can be acquired from Intel support.
 Only Testpoint is validated with DPDK, the latest version that has been
 validated with DPDK is 4.1.6.
 
+Support Switch Restart
+~~~~~~~~~~~~~~~~~~~~~~
+
+For fm10k multi host based design, DPDK app running in VM or host needs to
+get aware of the switch's state because switch may undergo a quit-restart.
+When switch goes down, DPDK app receives a LSC event indicating link status
+turns down, and app should stop the worker threads that are polling on the
+Rx/Tx queues. When switch gets up, a LSC event indicating LINK_UP is sent to
+the app, then app can restart fm10k port to resume its network processing.
+
 CRC striping
 ~~~~~~~~~~~~
 
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH 4/4] doc: add switch restart support to fm10k guide

[Mcnamara, John]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Xiao Wang
> Sent: Sunday, October 22, 2017 10:37 PM
> To: dev@dpdk.org
> Cc: Wu, Jingjing <jingjing.wu@intel.com>; Chen, Jing D
> <jing.d.chen@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: [dpdk-dev] [PATCH 4/4] doc: add switch restart support to fm10k
> guide
> 
> This patch documents how DPDK app should handle the event of switch quit-
> restart to resume its network without an app-level restart.
> 
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>


I'd suggest some minor changes as follows:


Support for Switch Restart
~~~~~~~~~~~~~~~~~~~~~~~~~~

For FM10000 multi host based design a DPDK app running in the VM or host needs
to be aware of the switch's state since it may undergo a quit-restart. When
the switch goes down the DPDK app will receive a LSC event indicating link
status down, and the app should stop the worker threads that are polling on
the Rx/Tx queues. When switch comes up, a LSC event indicating ``LINK_UP`` is
sent to the app, which can then restart the FM100 port to resume network
processing.

Re: [dpdk-dev] [PATCH 4/4] doc: add switch restart support to fm10k guide

[Wang, Xiao W]

> -----Original Message-----
> From: Mcnamara, John
> Sent: Monday, October 23, 2017 9:07 PM
> To: Wang, Xiao W <xiao.w.wang@intel.com>; dev@dpdk.org
> Cc: Wu, Jingjing <jingjing.wu@intel.com>; Chen, Jing D
> <jing.d.chen@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: RE: [dpdk-dev] [PATCH 4/4] doc: add switch restart support to fm10k
> guide
> 
> 
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Xiao Wang
> > Sent: Sunday, October 22, 2017 10:37 PM
> > To: dev@dpdk.org
> > Cc: Wu, Jingjing <jingjing.wu@intel.com>; Chen, Jing D
> > <jing.d.chen@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> > Subject: [dpdk-dev] [PATCH 4/4] doc: add switch restart support to fm10k
> > guide
> >
> > This patch documents how DPDK app should handle the event of switch quit-
> > restart to resume its network without an app-level restart.
> >
> > Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
> 
> 
> I'd suggest some minor changes as follows:
> 
> 
> Support for Switch Restart
> ~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> For FM10000 multi host based design a DPDK app running in the VM or host
> needs
> to be aware of the switch's state since it may undergo a quit-restart. When
> the switch goes down the DPDK app will receive a LSC event indicating link
> status down, and the app should stop the worker threads that are polling on
> the Rx/Tx queues. When switch comes up, a LSC event indicating ``LINK_UP`` is
> sent to the app, which can then restart the FM100 port to resume network
> processing.

Thanks John, I will make it in v2.

BRs,
Xiao

Re: [dpdk-dev] [PATCH v2 4/4] doc: add switch restart support to fm10k guide

[Mcnamara, John]

> -----Original Message-----
> From: Wang, Xiao W
> Sent: Tuesday, October 24, 2017 2:46 PM
> To: Wu, Jingjing <jingjing.wu@intel.com>
> Cc: dev@dpdk.org; Mcnamara, John <john.mcnamara@intel.com>; Zhang, Qi Z
> <qi.z.zhang@intel.com>; Wang, Xiao W <xiao.w.wang@intel.com>
> Subject: [PATCH v2 4/4] doc: add switch restart support to fm10k guide
> 
> This patch documents how DPDK app should handle the event of switch quit-
> restart to resume its network without an app-level restart.
> 
> Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>

Acked-by: John McNamara <john.mcnamara@intel.com>

[dpdk-dev] [PATCH v2 0/4] net/fm10k: support switch restart

[Xiao Wang]

For fm10k multi host based design, DPDK app running in VM or host needs to
get aware of the switch's state because switch may undergo a quit-restart.
This patch series supports switch restart feature for DPDK app to be able
to resume its packet processing without an app-level restart.

Changes in v2:
- Improve function description wording in fm10k guide.
- Fix checkpatch warnings.

Xiao Wang (4):
  net/fm10k: redefine link status semantics
  net/fm10k: support switch restart on PF
  net/fm10k: support switch restart on VF
  doc: add switch restart support to fm10k guide

 doc/guides/nics/fm10k.rst        |  11 ++++
 drivers/net/fm10k/fm10k.h        |   1 +
 drivers/net/fm10k/fm10k_ethdev.c | 106 ++++++++++++++++++++++++++++++++++++---
 3 files changed, 112 insertions(+), 6 deletions(-)

-- 
1.8.3.1

[dpdk-dev] [PATCH v2 1/4] net/fm10k: redefine link status semantics

[Xiao Wang]

As fm10k host interface is not directly connected to PHY, marking the link
status as UP doesn't mean a lot to the application. So, this patch
basically redefines the link status as the state of switch manager: when
switch manager is running, it's LINK_UP; when switch manager goes down by
calling the fmTerminate function, status turns to LINK_DOWN.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
 drivers/net/fm10k/fm10k.h        |  1 +
 drivers/net/fm10k/fm10k_ethdev.c | 11 +++++++----
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/net/fm10k/fm10k.h b/drivers/net/fm10k/fm10k.h
index 060982b..594dca4 100644
--- a/drivers/net/fm10k/fm10k.h
+++ b/drivers/net/fm10k/fm10k.h
@@ -155,6 +155,7 @@ struct fm10k_dev_info {
 	struct fm10k_macvlan_filter_info    macvlan;
 	/* Flag to indicate if RX vector conditions satisfied */
 	bool rx_vec_allowed;
+	bool sm_down;
 };
 
 /*
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 586f482..2587696 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -1256,14 +1256,17 @@ static inline int fm10k_glort_valid(struct fm10k_hw *hw)
 fm10k_link_update(struct rte_eth_dev *dev,
 	__rte_unused int wait_to_complete)
 {
+	struct fm10k_dev_info *dev_info =
+		FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
 	PMD_INIT_FUNC_TRACE();
 
-	/* The host-interface link is always up.  The speed is ~50Gbps per Gen3
-	 * x8 PCIe interface. For now, we leave the speed undefined since there
-	 * is no 50Gbps Ethernet. */
+	/* The speed is ~50Gbps per Gen3 x8 PCIe interface. For now, we
+	 * leave the speed undefined since there is no 50Gbps Ethernet.
+	 */
 	dev->data->dev_link.link_speed  = 0;
 	dev->data->dev_link.link_duplex = ETH_LINK_FULL_DUPLEX;
-	dev->data->dev_link.link_status = ETH_LINK_UP;
+	dev->data->dev_link.link_status =
+		dev_info->sm_down ? ETH_LINK_DOWN : ETH_LINK_UP;
 
 	return 0;
 }
-- 
1.8.3.1

[dpdk-dev] [PATCH v2 2/4] net/fm10k: support switch restart on PF

[Xiao Wang]

For events indicating the change in the state of the switch manager. The
driver will restore the basic port configurations and then pass this event
to application so that the application can restore any additional
configurations if required. In this way, once the switch manager restarts,
the DPDK application can resume its network.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
v2:
- Fix line over 90 characters.
---
 drivers/net/fm10k/fm10k_ethdev.c | 54 ++++++++++++++++++++++++++++++++++++++--
 1 file changed, 52 insertions(+), 2 deletions(-)

diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index 2587696..c71c0fc 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2556,6 +2556,10 @@ static void fm10k_MAC_filter_set(struct rte_eth_dev *dev,
 	struct rte_eth_dev *dev = (struct rte_eth_dev *)param;
 	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	uint32_t cause, status;
+	struct fm10k_dev_info *dev_info =
+		FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
+	int status_mbx;
+	s32 err;
 
 	if (hw->mac.type != fm10k_mac_pf)
 		return;
@@ -2572,14 +2576,60 @@ static void fm10k_MAC_filter_set(struct rte_eth_dev *dev,
 	if (cause & FM10K_EICR_SWITCHNOTREADY)
 		PMD_INIT_LOG(ERR, "INT: Switch is not ready");
 
-	if (cause & FM10K_EICR_SWITCHREADY)
+	if (cause & FM10K_EICR_SWITCHREADY) {
 		PMD_INIT_LOG(INFO, "INT: Switch is ready");
+		if (dev_info->sm_down == 1) {
+			fm10k_mbx_lock(hw);
+
+			/* For recreating logical ports */
+			status_mbx = hw->mac.ops.update_lport_state(hw,
+					hw->mac.dglort_map, MAX_LPORT_NUM, 1);
+			if (status_mbx == FM10K_SUCCESS)
+				PMD_INIT_LOG(INFO, "INT: Recreated Logical port");
+			else
+				PMD_INIT_LOG(INFO, "INT: Logical ports weren't recreated");
+
+			status_mbx = hw->mac.ops.update_xcast_mode(hw,
+					hw->mac.dglort_map, FM10K_XCAST_MODE_NONE);
+			if (status_mbx != FM10K_SUCCESS)
+				PMD_INIT_LOG(ERR, "Failed to set XCAST mode");
+
+			fm10k_mbx_unlock(hw);
+
+			/* first clear the internal SW recording structure */
+			if (!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_VMDQ_FLAG))
+				fm10k_vlan_filter_set(dev, hw->mac.default_vid, false);
+
+			fm10k_MAC_filter_set(dev, hw->mac.addr, false,
+					MAIN_VSI_POOL_NUMBER);
+
+			/* Add default mac address and vlan for the logical ports that
+			 * have been created, leave to the application to fully recover
+			 * Rx filtering.
+			 */
+			fm10k_MAC_filter_set(dev, hw->mac.addr, true,
+					MAIN_VSI_POOL_NUMBER);
+
+			if (!(dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_VMDQ_FLAG))
+				fm10k_vlan_filter_set(dev, hw->mac.default_vid, true);
+
+			dev_info->sm_down = 0;
+			_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC,
+					NULL, NULL);
+		}
+	}
 
 	/* Handle mailbox message */
 	fm10k_mbx_lock(hw);
-	hw->mbx.ops.process(hw, &hw->mbx);
+	err = hw->mbx.ops.process(hw, &hw->mbx);
 	fm10k_mbx_unlock(hw);
 
+	if (err == FM10K_ERR_RESET_REQUESTED) {
+		PMD_INIT_LOG(INFO, "INT: Switch is down");
+		dev_info->sm_down = 1;
+		_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL, NULL);
+	}
+
 	/* Handle SRAM error */
 	if (cause & FM10K_EICR_SRAMERROR) {
 		PMD_INIT_LOG(ERR, "INT: SRAM error on PEP");
-- 
1.8.3.1

[dpdk-dev] [PATCH v2 3/4] net/fm10k: support switch restart on VF

[Xiao Wang]

For events indicating a change in the state of the switch manager, the
driver will restore the basic port configurations and then pass this event
to application so that the application can restore any additional
configurations if required. In this way, once the switch manager restarts,
the DPDK application can resume its network.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
v2:
- Remove unnecessary parentheses.
---
 drivers/net/fm10k/fm10k_ethdev.c | 41 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c
index c71c0fc..d0dfc14 100644
--- a/drivers/net/fm10k/fm10k_ethdev.c
+++ b/drivers/net/fm10k/fm10k_ethdev.c
@@ -2670,6 +2670,11 @@ static void fm10k_MAC_filter_set(struct rte_eth_dev *dev,
 {
 	struct rte_eth_dev *dev = (struct rte_eth_dev *)param;
 	struct fm10k_hw *hw = FM10K_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+	struct fm10k_mbx_info *mbx = &hw->mbx;
+	struct fm10k_dev_info *dev_info =
+		FM10K_DEV_PRIVATE_TO_INFO(dev->data->dev_private);
+	const enum fm10k_mbx_state state = mbx->state;
+	int status_mbx;
 
 	if (hw->mac.type != fm10k_mac_vf)
 		return;
@@ -2679,6 +2684,42 @@ static void fm10k_MAC_filter_set(struct rte_eth_dev *dev,
 	hw->mbx.ops.process(hw, &hw->mbx);
 	fm10k_mbx_unlock(hw);
 
+	if (state == FM10K_STATE_OPEN && mbx->state == FM10K_STATE_CONNECT) {
+		PMD_INIT_LOG(INFO, "INT: Switch has gone down");
+
+		fm10k_mbx_lock(hw);
+		hw->mac.ops.update_lport_state(hw, hw->mac.dglort_map, MAX_LPORT_NUM, 1);
+		fm10k_mbx_unlock(hw);
+
+		/* Setting reset flag */
+		dev_info->sm_down = 1;
+		_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL, NULL);
+	}
+
+	if (dev_info->sm_down == 1 && hw->mac.dglort_map == FM10K_DGLORTMAP_ZERO) {
+		PMD_INIT_LOG(INFO, "INT: Switch has gone up");
+		fm10k_mbx_lock(hw);
+		status_mbx = hw->mac.ops.update_xcast_mode(hw, hw->mac.dglort_map,
+				FM10K_XCAST_MODE_NONE);
+		if (status_mbx != FM10K_SUCCESS)
+			PMD_INIT_LOG(ERR, "Failed to set XCAST mode");
+		fm10k_mbx_unlock(hw);
+
+		/* first clear the internal SW recording structure */
+		fm10k_vlan_filter_set(dev, hw->mac.default_vid, false);
+		fm10k_MAC_filter_set(dev, hw->mac.addr, false, MAIN_VSI_POOL_NUMBER);
+
+		/* Add default mac address and vlan for the logical ports that
+		 * have been created, leave to the application to fully recover
+		 * Rx filtering.
+		 */
+		fm10k_MAC_filter_set(dev, hw->mac.addr, true, MAIN_VSI_POOL_NUMBER);
+		fm10k_vlan_filter_set(dev, hw->mac.default_vid, true);
+
+		dev_info->sm_down = 0;
+		_rte_eth_dev_callback_process(dev, RTE_ETH_EVENT_INTR_LSC, NULL, NULL);
+	}
+
 	/* Re-enable interrupt from device side */
 	FM10K_WRITE_REG(hw, FM10K_VFITR(0), FM10K_ITR_AUTOMASK |
 					FM10K_ITR_MASK_CLEAR);
-- 
1.8.3.1

[dpdk-dev] [PATCH v2 4/4] doc: add switch restart support to fm10k guide

[Xiao Wang]

This patch documents how DPDK app should handle the event of switch
quit-restart to resume its network without an app-level restart.

Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
---
v2:
- Improve the wording.
---
 doc/guides/nics/fm10k.rst | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/doc/guides/nics/fm10k.rst b/doc/guides/nics/fm10k.rst
index 7fc4862..b47fc0d 100644
--- a/doc/guides/nics/fm10k.rst
+++ b/doc/guides/nics/fm10k.rst
@@ -161,6 +161,17 @@ FM10000 PMD driver. The switch driver can be acquired from Intel support.
 Only Testpoint is validated with DPDK, the latest version that has been
 validated with DPDK is 4.1.6.
 
+Support for Switch Restart
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For FM10000 multi host based design a DPDK app running in the VM or host needs
+to be aware of the switch's state since it may undergo a quit-restart. When
+the switch goes down the DPDK app will receive a LSC event indicating link
+status down, and the app should stop the worker threads that are polling on
+the Rx/Tx queues. When switch comes up, a LSC event indicating ``LINK_UP`` is
+sent to the app, which can then restart the FM10000 port to resume network
+processing.
+
 CRC striping
 ~~~~~~~~~~~~
 
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH v2 0/4] net/fm10k: support switch restart

[Ferruh Yigit]

On 10/24/2017 6:45 AM, Xiao Wang wrote:
> For fm10k multi host based design, DPDK app running in VM or host needs to
> get aware of the switch's state because switch may undergo a quit-restart.
> This patch series supports switch restart feature for DPDK app to be able
> to resume its packet processing without an app-level restart.
> 
> Changes in v2:
> - Improve function description wording in fm10k guide.
> - Fix checkpatch warnings.
> 
> Xiao Wang (4):
>   net/fm10k: redefine link status semantics
>   net/fm10k: support switch restart on PF
>   net/fm10k: support switch restart on VF
>   doc: add switch restart support to fm10k guide

Series Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

Re: [dpdk-dev] [PATCH v2 0/4] net/fm10k: support switch restart

[Ferruh Yigit]

On 10/24/2017 12:29 PM, Ferruh Yigit wrote:
> On 10/24/2017 6:45 AM, Xiao Wang wrote:
>> For fm10k multi host based design, DPDK app running in VM or host needs to
>> get aware of the switch's state because switch may undergo a quit-restart.
>> This patch series supports switch restart feature for DPDK app to be able
>> to resume its packet processing without an app-level restart.
>>
>> Changes in v2:
>> - Improve function description wording in fm10k guide.
>> - Fix checkpatch warnings.
>>
>> Xiao Wang (4):
>>   net/fm10k: redefine link status semantics
>>   net/fm10k: support switch restart on PF
>>   net/fm10k: support switch restart on VF
>>   doc: add switch restart support to fm10k guide
> 
> Series Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

Series applied to dpdk-next-net/master, thanks.