DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure
@ 2021-04-21  5:02 Haiyue Wang
  2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 1/3] bus/pci: enable PCI master in command register Haiyue Wang
                   ` (9 more replies)
  0 siblings, 10 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-21  5:02 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang

By triggerring the VF reset from PF reset,
	echo 1 > /sys/bus/pci/devices/PF-BDF/reset

the PCI bus master bit will cleared on VF, so the VF needs to enable
this bit before restart.

This patch set adds the API to enable PCI bus master.

Haiyue Wang (3):
  bus/pci: enable PCI master in command register
  net/iavf: enable PCI bus master after reset
  net/i40e: enable PCI bus master after reset

 drivers/bus/pci/pci_common.c      | 20 ++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h     | 12 ++++++++++++
 drivers/bus/pci/version.map       |  1 +
 drivers/net/i40e/i40e_ethdev_vf.c |  7 ++++++-
 drivers/net/iavf/iavf_ethdev.c    |  3 +++
 lib/librte_pci/rte_pci.h          |  4 ++++
 6 files changed, 46 insertions(+), 1 deletion(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v1 1/3] bus/pci: enable PCI master in command register
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-04-21  5:02 ` Haiyue Wang
  2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-21  5:02 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Ray Kinsella,
	Neil Horman, Gaetan Rivet

This adds the support to set 'Bus Master Enable' bit in the PCI command
register.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/bus/pci/pci_common.c  | 20 ++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h | 12 ++++++++++++
 drivers/bus/pci/version.map   |  1 +
 lib/librte_pci/rte_pci.h      |  4 ++++
 4 files changed, 37 insertions(+)

diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index ee7f966358..b631cb9c7e 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -746,6 +746,26 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
 	return 0;
 }
 
+int
+rte_pci_enable_bus_master(struct rte_pci_device *dev)
+{
+	uint16_t cmd;
+
+	if (rte_pci_read_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
+		return -1;
+	}
+
+	cmd |= RTE_PCI_COMMAND_MASTER;
+
+	if (rte_pci_write_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 64886b4731..83caf477ba 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -249,6 +249,18 @@ void rte_pci_dump(FILE *f);
 __rte_experimental
 off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
 
+/**
+ * Enables Bus Master for device's PCI command register.
+ *
+ *  @param dev
+ *    A pointer to rte_pci_device structure.
+ *
+ *  @return
+ *  0 on success, -1 on error in PCI config space read/write.
+ */
+__rte_experimental
+int rte_pci_enable_bus_master(struct rte_pci_device *dev);
+
 /**
  * Register a PCI driver.
  *
diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
index f33ed0abd1..b271e48a8f 100644
--- a/drivers/bus/pci/version.map
+++ b/drivers/bus/pci/version.map
@@ -20,5 +20,6 @@ DPDK_21 {
 EXPERIMENTAL {
 	global:
 
+	rte_pci_enable_bus_master;
 	rte_pci_find_ext_capability;
 };
diff --git a/lib/librte_pci/rte_pci.h b/lib/librte_pci/rte_pci.h
index a8f8e404a9..1f33d687f4 100644
--- a/lib/librte_pci/rte_pci.h
+++ b/lib/librte_pci/rte_pci.h
@@ -32,6 +32,10 @@ extern "C" {
 
 #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
 #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
+#define RTE_PCI_COMMAND		0x04	/* 16 bits */
+
+/* PCI Command Register */
+#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
 
 /* PCI Express capability registers */
 #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v1 2/3] net/iavf: enable PCI bus master after reset
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
  2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 1/3] bus/pci: enable PCI master in command register Haiyue Wang
@ 2021-04-21  5:02 ` Haiyue Wang
  2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 3/3] net/i40e: " Haiyue Wang
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-21  5:02 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Jingjing Wu, Beilei Xing

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/iavf/iavf_ethdev.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index d523a0618c..8c924d21b2 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
 
+	if (vf->vf_reset)
+		rte_pci_enable_bus_master(pci_dev);
+
 	vf->vf_reset = false;
 
 	return ret;
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v1 3/3] net/i40e: enable PCI bus master after reset
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
  2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 1/3] bus/pci: enable PCI master in command register Haiyue Wang
  2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-04-21  5:02 ` Haiyue Wang
  2021-04-21 11:59 ` [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Zhang, Qi Z
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-21  5:02 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Beilei Xing, Jeff Guo

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.

And align the VF reset event handling in device close module as the AVF
driver does.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/i40e/i40e_ethdev_vf.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 3c258ba7cf..4f1d04eb2b 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1212,7 +1212,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev)
 	if (i >= MAX_RESET_WAIT_CNT)
 		return -1;
 
-	vf->vf_reset = false;
 	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
 
 	return 0;
@@ -1391,6 +1390,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg,
 	switch (pf_msg->event) {
 	case VIRTCHNL_EVENT_RESET_IMPENDING:
 		PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event");
+		vf->vf_reset = true;
 		rte_eth_dev_callback_process(dev,
 				RTE_ETH_EVENT_INTR_RESET, NULL);
 		break;
@@ -2487,6 +2487,11 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
+	if (vf->vf_reset)
+		rte_pci_enable_bus_master(RTE_ETH_DEV_TO_PCI(dev));
+
+	vf->vf_reset = false;
+
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
                   ` (2 preceding siblings ...)
  2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 3/3] net/i40e: " Haiyue Wang
@ 2021-04-21 11:59 ` Zhang, Qi Z
  2021-04-22  1:18 ` [dpdk-dev] [PATCH v2 0/3] fix " Haiyue Wang
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 46+ messages in thread
From: Zhang, Qi Z @ 2021-04-21 11:59 UTC (permalink / raw)
  To: Wang, Haiyue, dev; +Cc: Wang, Liang-min



> -----Original Message-----
> From: Wang, Haiyue <haiyue.wang@intel.com>
> Sent: Wednesday, April 21, 2021 1:03 PM
> To: dev@dpdk.org
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min
> <liang-min.wang@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>
> Subject: [PATCH v1 0/3] Fix PF reset causes VF memory request failure
> 
> By triggerring the VF reset from PF reset,
> 	echo 1 > /sys/bus/pci/devices/PF-BDF/reset
> 
> the PCI bus master bit will cleared on VF, so the VF needs to enable this bit
> before restart.
> 
> This patch set adds the API to enable PCI bus master.
> 
> Haiyue Wang (3):
>   bus/pci: enable PCI master in command register
>   net/iavf: enable PCI bus master after reset
>   net/i40e: enable PCI bus master after reset
> 
>  drivers/bus/pci/pci_common.c      | 20 ++++++++++++++++++++
>  drivers/bus/pci/rte_bus_pci.h     | 12 ++++++++++++
>  drivers/bus/pci/version.map       |  1 +
>  drivers/net/i40e/i40e_ethdev_vf.c |  7 ++++++-
>  drivers/net/iavf/iavf_ethdev.c    |  3 +++
>  lib/librte_pci/rte_pci.h          |  4 ++++
>  6 files changed, 46 insertions(+), 1 deletion(-)
> 
> --
> 2.31.1

Tested-by: Qi Zhang <qi.z.zhang@intel.com> on iavf with ice kernel PF.

Thanks
Qi


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v2 0/3] fix PF reset causes VF memory request failure
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
                   ` (3 preceding siblings ...)
  2021-04-21 11:59 ` [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Zhang, Qi Z
@ 2021-04-22  1:18 ` Haiyue Wang
  2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 1/3] bus/pci: enable PCI master in command register Haiyue Wang
                     ` (2 more replies)
  2021-04-23 11:39 ` [dpdk-dev] [PATCH v3 0/3] fix PF reset causes VF memory request failure Haiyue Wang
                   ` (4 subsequent siblings)
  9 siblings, 3 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-22  1:18 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang

By triggerring the VF reset from PF reset,

        echo 1 > /sys/bus/pci/devices/PF-BDF/reset

the PCI bus master bit will cleared on VF, so the VF needs to enable
this bit before restart.

This patch set adds the API to enable PCI bus master.

v2: rebase to new librte directory path.

Haiyue Wang (3):
  bus/pci: enable PCI master in command register
  net/iavf: enable PCI bus master after reset
  net/i40e: enable PCI bus master after reset

 drivers/bus/pci/pci_common.c      | 20 ++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h     | 12 ++++++++++++
 drivers/bus/pci/version.map       |  1 +
 drivers/net/i40e/i40e_ethdev_vf.c |  7 ++++++-
 drivers/net/iavf/iavf_ethdev.c    |  3 +++
 lib/pci/rte_pci.h                 |  4 ++++
 6 files changed, 46 insertions(+), 1 deletion(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v2 1/3] bus/pci: enable PCI master in command register
  2021-04-22  1:18 ` [dpdk-dev] [PATCH v2 0/3] fix " Haiyue Wang
@ 2021-04-22  1:18   ` Haiyue Wang
  2021-04-23 10:43     ` Kinsella, Ray
  2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
  2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 3/3] net/i40e: " Haiyue Wang
  2 siblings, 1 reply; 46+ messages in thread
From: Haiyue Wang @ 2021-04-22  1:18 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Ray Kinsella,
	Neil Horman, Gaetan Rivet

This adds the support to set 'Bus Master Enable' bit in the PCI command
register.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Tested-by: Qi Zhang <qi.z.zhang@intel.com>
---
 drivers/bus/pci/pci_common.c  | 20 ++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h | 12 ++++++++++++
 drivers/bus/pci/version.map   |  1 +
 lib/pci/rte_pci.h             |  4 ++++
 4 files changed, 37 insertions(+)

diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index ee7f96635..b631cb9c7 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -746,6 +746,26 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
 	return 0;
 }
 
+int
+rte_pci_enable_bus_master(struct rte_pci_device *dev)
+{
+	uint16_t cmd;
+
+	if (rte_pci_read_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
+		return -1;
+	}
+
+	cmd |= RTE_PCI_COMMAND_MASTER;
+
+	if (rte_pci_write_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 64886b473..83caf477b 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -249,6 +249,18 @@ void rte_pci_dump(FILE *f);
 __rte_experimental
 off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
 
+/**
+ * Enables Bus Master for device's PCI command register.
+ *
+ *  @param dev
+ *    A pointer to rte_pci_device structure.
+ *
+ *  @return
+ *  0 on success, -1 on error in PCI config space read/write.
+ */
+__rte_experimental
+int rte_pci_enable_bus_master(struct rte_pci_device *dev);
+
 /**
  * Register a PCI driver.
  *
diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
index f33ed0abd..b271e48a8 100644
--- a/drivers/bus/pci/version.map
+++ b/drivers/bus/pci/version.map
@@ -20,5 +20,6 @@ DPDK_21 {
 EXPERIMENTAL {
 	global:
 
+	rte_pci_enable_bus_master;
 	rte_pci_find_ext_capability;
 };
diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
index a8f8e404a..1f33d687f 100644
--- a/lib/pci/rte_pci.h
+++ b/lib/pci/rte_pci.h
@@ -32,6 +32,10 @@ extern "C" {
 
 #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
 #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
+#define RTE_PCI_COMMAND		0x04	/* 16 bits */
+
+/* PCI Command Register */
+#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
 
 /* PCI Express capability registers */
 #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v2 2/3] net/iavf: enable PCI bus master after reset
  2021-04-22  1:18 ` [dpdk-dev] [PATCH v2 0/3] fix " Haiyue Wang
  2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 1/3] bus/pci: enable PCI master in command register Haiyue Wang
@ 2021-04-22  1:18   ` Haiyue Wang
  2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 3/3] net/i40e: " Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-22  1:18 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Jingjing Wu, Beilei Xing

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Tested-by: Qi Zhang <qi.z.zhang@intel.com>
---
 drivers/net/iavf/iavf_ethdev.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index d523a0618..8c924d21b 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
 
+	if (vf->vf_reset)
+		rte_pci_enable_bus_master(pci_dev);
+
 	vf->vf_reset = false;
 
 	return ret;
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v2 3/3] net/i40e: enable PCI bus master after reset
  2021-04-22  1:18 ` [dpdk-dev] [PATCH v2 0/3] fix " Haiyue Wang
  2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 1/3] bus/pci: enable PCI master in command register Haiyue Wang
  2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-04-22  1:18   ` Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-22  1:18 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Beilei Xing, Jeff Guo

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.

And align the VF reset event handling in device close module as the AVF
driver does.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Tested-by: Qi Zhang <qi.z.zhang@intel.com>
---
 drivers/net/i40e/i40e_ethdev_vf.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 3c258ba7c..4f1d04eb2 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1212,7 +1212,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev)
 	if (i >= MAX_RESET_WAIT_CNT)
 		return -1;
 
-	vf->vf_reset = false;
 	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
 
 	return 0;
@@ -1391,6 +1390,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg,
 	switch (pf_msg->event) {
 	case VIRTCHNL_EVENT_RESET_IMPENDING:
 		PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event");
+		vf->vf_reset = true;
 		rte_eth_dev_callback_process(dev,
 				RTE_ETH_EVENT_INTR_RESET, NULL);
 		break;
@@ -2487,6 +2487,11 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
+	if (vf->vf_reset)
+		rte_pci_enable_bus_master(RTE_ETH_DEV_TO_PCI(dev));
+
+	vf->vf_reset = false;
+
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v2 1/3] bus/pci: enable PCI master in command register
  2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 1/3] bus/pci: enable PCI master in command register Haiyue Wang
@ 2021-04-23 10:43     ` Kinsella, Ray
  2021-04-23 12:07       ` Wang, Haiyue
  0 siblings, 1 reply; 46+ messages in thread
From: Kinsella, Ray @ 2021-04-23 10:43 UTC (permalink / raw)
  To: Haiyue Wang, dev; +Cc: qi.z.zhang, liang-min.wang, Neil Horman, Gaetan Rivet



On 22/04/2021 02:18, Haiyue Wang wrote:
> This adds the support to set 'Bus Master Enable' bit in the PCI command
> register.
> 
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> Tested-by: Qi Zhang <qi.z.zhang@intel.com>
> ---
>  drivers/bus/pci/pci_common.c  | 20 ++++++++++++++++++++
>  drivers/bus/pci/rte_bus_pci.h | 12 ++++++++++++
>  drivers/bus/pci/version.map   |  1 +
>  lib/pci/rte_pci.h             |  4 ++++
>  4 files changed, 37 insertions(+)
> 
> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
> index ee7f96635..b631cb9c7 100644
> --- a/drivers/bus/pci/pci_common.c
> +++ b/drivers/bus/pci/pci_common.c
> @@ -746,6 +746,26 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
>  	return 0;
>  }
>  
> +int
> +rte_pci_enable_bus_master(struct rte_pci_device *dev)
> +{
> +	uint16_t cmd;
> +
> +	if (rte_pci_read_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
> +		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
> +		return -1;
> +	}
> +
> +	cmd |= RTE_PCI_COMMAND_MASTER;
> +
> +	if (rte_pci_write_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
> +		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
>  struct rte_pci_bus rte_pci_bus = {
>  	.bus = {
>  		.scan = rte_pci_scan,
> diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
> index 64886b473..83caf477b 100644
> --- a/drivers/bus/pci/rte_bus_pci.h
> +++ b/drivers/bus/pci/rte_bus_pci.h
> @@ -249,6 +249,18 @@ void rte_pci_dump(FILE *f);
>  __rte_experimental
>  off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
>  
> +/**
> + * Enables Bus Master for device's PCI command register.
> + *
> + *  @param dev
> + *    A pointer to rte_pci_device structure.
> + *
> + *  @return
> + *  0 on success, -1 on error in PCI config space read/write.
> + */
> +__rte_experimental
> +int rte_pci_enable_bus_master(struct rte_pci_device *dev);
> +
>  /**
>   * Register a PCI driver.
>   *
> diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
> index f33ed0abd..b271e48a8 100644
> --- a/drivers/bus/pci/version.map
> +++ b/drivers/bus/pci/version.map
> @@ -20,5 +20,6 @@ DPDK_21 {
>  EXPERIMENTAL {
>  	global:
>  

Please annotate when the symbol was added.

> +	rte_pci_enable_bus_master;
>  	rte_pci_find_ext_capability;
>  };
> diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
> index a8f8e404a..1f33d687f 100644
> --- a/lib/pci/rte_pci.h
> +++ b/lib/pci/rte_pci.h
> @@ -32,6 +32,10 @@ extern "C" {
>  
>  #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
>  #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
> +#define RTE_PCI_COMMAND		0x04	/* 16 bits */
> +
> +/* PCI Command Register */
> +#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
>  
>  /* PCI Express capability registers */
>  #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
> 

^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v3 0/3] fix PF reset causes VF memory request failure
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
                   ` (4 preceding siblings ...)
  2021-04-22  1:18 ` [dpdk-dev] [PATCH v2 0/3] fix " Haiyue Wang
@ 2021-04-23 11:39 ` Haiyue Wang
  2021-04-23 11:39   ` [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register Haiyue Wang
                     ` (2 more replies)
  2021-04-27 13:39 ` [dpdk-dev] [PATCH v4 0/3] fix PF reset causes VF memory request failure Haiyue Wang
                   ` (3 subsequent siblings)
  9 siblings, 3 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-23 11:39 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang

By triggerring the VF reset from PF reset,

        echo 1 > /sys/bus/pci/devices/PF-BDF/reset

the PCI bus master bit will cleared on VF, so the VF needs to enable
this bit before restart.

This patch set adds the API to enable PCI bus master.

v3: added the missed annotate symbol add time
v2: rebase to new librte directory path.

Haiyue Wang (3):
  bus/pci: enable PCI master in command register
  net/iavf: enable PCI bus master after reset
  net/i40e: enable PCI bus master after reset

 drivers/bus/pci/pci_common.c      | 20 ++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h     | 12 ++++++++++++
 drivers/bus/pci/version.map       |  3 +++
 drivers/net/i40e/i40e_ethdev_vf.c |  7 ++++++-
 drivers/net/iavf/iavf_ethdev.c    |  3 +++
 lib/pci/rte_pci.h                 |  4 ++++
 6 files changed, 48 insertions(+), 1 deletion(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register
  2021-04-23 11:39 ` [dpdk-dev] [PATCH v3 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-04-23 11:39   ` Haiyue Wang
  2021-04-23 12:32     ` Kinsella, Ray
  2021-04-27  9:28     ` David Marchand
  2021-04-23 11:40   ` [dpdk-dev] [PATCH v3 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
  2021-04-23 11:40   ` [dpdk-dev] [PATCH v3 3/3] net/i40e: " Haiyue Wang
  2 siblings, 2 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-23 11:39 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Ray Kinsella,
	Neil Horman, Gaetan Rivet

This adds the support to set 'Bus Master Enable' bit in the PCI command
register.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Tested-by: Qi Zhang <qi.z.zhang@intel.com>
---
 drivers/bus/pci/pci_common.c  | 20 ++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h | 12 ++++++++++++
 drivers/bus/pci/version.map   |  3 +++
 lib/pci/rte_pci.h             |  4 ++++
 4 files changed, 39 insertions(+)

diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index ee7f96635..b631cb9c7 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -746,6 +746,26 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
 	return 0;
 }
 
+int
+rte_pci_enable_bus_master(struct rte_pci_device *dev)
+{
+	uint16_t cmd;
+
+	if (rte_pci_read_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
+		return -1;
+	}
+
+	cmd |= RTE_PCI_COMMAND_MASTER;
+
+	if (rte_pci_write_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 64886b473..83caf477b 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -249,6 +249,18 @@ void rte_pci_dump(FILE *f);
 __rte_experimental
 off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
 
+/**
+ * Enables Bus Master for device's PCI command register.
+ *
+ *  @param dev
+ *    A pointer to rte_pci_device structure.
+ *
+ *  @return
+ *  0 on success, -1 on error in PCI config space read/write.
+ */
+__rte_experimental
+int rte_pci_enable_bus_master(struct rte_pci_device *dev);
+
 /**
  * Register a PCI driver.
  *
diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
index f33ed0abd..9dbec12a0 100644
--- a/drivers/bus/pci/version.map
+++ b/drivers/bus/pci/version.map
@@ -21,4 +21,7 @@ EXPERIMENTAL {
 	global:
 
 	rte_pci_find_ext_capability;
+
+	# added in 21.05
+	rte_pci_enable_bus_master;
 };
diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
index a8f8e404a..1f33d687f 100644
--- a/lib/pci/rte_pci.h
+++ b/lib/pci/rte_pci.h
@@ -32,6 +32,10 @@ extern "C" {
 
 #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
 #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
+#define RTE_PCI_COMMAND		0x04	/* 16 bits */
+
+/* PCI Command Register */
+#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
 
 /* PCI Express capability registers */
 #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v3 2/3] net/iavf: enable PCI bus master after reset
  2021-04-23 11:39 ` [dpdk-dev] [PATCH v3 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-04-23 11:39   ` [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register Haiyue Wang
@ 2021-04-23 11:40   ` Haiyue Wang
  2021-04-23 11:40   ` [dpdk-dev] [PATCH v3 3/3] net/i40e: " Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-23 11:40 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Jingjing Wu, Beilei Xing

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Tested-by: Qi Zhang <qi.z.zhang@intel.com>
---
 drivers/net/iavf/iavf_ethdev.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index d523a0618..8c924d21b 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
 
+	if (vf->vf_reset)
+		rte_pci_enable_bus_master(pci_dev);
+
 	vf->vf_reset = false;
 
 	return ret;
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v3 3/3] net/i40e: enable PCI bus master after reset
  2021-04-23 11:39 ` [dpdk-dev] [PATCH v3 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-04-23 11:39   ` [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register Haiyue Wang
  2021-04-23 11:40   ` [dpdk-dev] [PATCH v3 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-04-23 11:40   ` Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-23 11:40 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, Haiyue Wang, Beilei Xing, Jeff Guo

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.

And align the VF reset event handling in device close module as the AVF
driver does.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Tested-by: Qi Zhang <qi.z.zhang@intel.com>
---
 drivers/net/i40e/i40e_ethdev_vf.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 3c258ba7c..4f1d04eb2 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1212,7 +1212,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev)
 	if (i >= MAX_RESET_WAIT_CNT)
 		return -1;
 
-	vf->vf_reset = false;
 	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
 
 	return 0;
@@ -1391,6 +1390,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg,
 	switch (pf_msg->event) {
 	case VIRTCHNL_EVENT_RESET_IMPENDING:
 		PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event");
+		vf->vf_reset = true;
 		rte_eth_dev_callback_process(dev,
 				RTE_ETH_EVENT_INTR_RESET, NULL);
 		break;
@@ -2487,6 +2487,11 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
+	if (vf->vf_reset)
+		rte_pci_enable_bus_master(RTE_ETH_DEV_TO_PCI(dev));
+
+	vf->vf_reset = false;
+
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v2 1/3] bus/pci: enable PCI master in command register
  2021-04-23 10:43     ` Kinsella, Ray
@ 2021-04-23 12:07       ` Wang, Haiyue
  0 siblings, 0 replies; 46+ messages in thread
From: Wang, Haiyue @ 2021-04-23 12:07 UTC (permalink / raw)
  To: Kinsella, Ray, dev
  Cc: Zhang, Qi Z, Wang, Liang-min, Neil Horman, Gaetan Rivet

> -----Original Message-----
> From: Kinsella, Ray <mdr@ashroe.eu>
> Sent: Friday, April 23, 2021 18:44
> To: Wang, Haiyue <haiyue.wang@intel.com>; dev@dpdk.org
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-min.wang@intel.com>; Neil Horman
> <nhorman@tuxdriver.com>; Gaetan Rivet <grive@u256.net>
> Subject: Re: [PATCH v2 1/3] bus/pci: enable PCI master in command register
> 
> 
> 
> On 22/04/2021 02:18, Haiyue Wang wrote:
> > This adds the support to set 'Bus Master Enable' bit in the PCI command
> > register.
> >
> > Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> > Tested-by: Qi Zhang <qi.z.zhang@intel.com>
> > ---
> >  drivers/bus/pci/pci_common.c  | 20 ++++++++++++++++++++
> >  drivers/bus/pci/rte_bus_pci.h | 12 ++++++++++++
> >  drivers/bus/pci/version.map   |  1 +
> >  lib/pci/rte_pci.h             |  4 ++++
> >  4 files changed, 37 insertions(+)
> >


> >   *
> > diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
> > index f33ed0abd..b271e48a8 100644
> > --- a/drivers/bus/pci/version.map
> > +++ b/drivers/bus/pci/version.map
> > @@ -20,5 +20,6 @@ DPDK_21 {
> >  EXPERIMENTAL {
> >  	global:
> >
> 
> Please annotate when the symbol was added.
> 

Fixed in v3.

> >

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register
  2021-04-23 11:39   ` [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register Haiyue Wang
@ 2021-04-23 12:32     ` Kinsella, Ray
  2021-04-27  9:28     ` David Marchand
  1 sibling, 0 replies; 46+ messages in thread
From: Kinsella, Ray @ 2021-04-23 12:32 UTC (permalink / raw)
  To: Haiyue Wang, dev; +Cc: qi.z.zhang, liang-min.wang, Neil Horman, Gaetan Rivet



On 23/04/2021 12:39, Haiyue Wang wrote:
> This adds the support to set 'Bus Master Enable' bit in the PCI command
> register.
> 
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> Tested-by: Qi Zhang <qi.z.zhang@intel.com>
> ---
>  drivers/bus/pci/pci_common.c  | 20 ++++++++++++++++++++
>  drivers/bus/pci/rte_bus_pci.h | 12 ++++++++++++
>  drivers/bus/pci/version.map   |  3 +++
>  lib/pci/rte_pci.h             |  4 ++++
>  4 files changed, 39 insertions(+)
> 
> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
> index ee7f96635..b631cb9c7 100644
> --- a/drivers/bus/pci/pci_common.c
> +++ b/drivers/bus/pci/pci_common.c
> @@ -746,6 +746,26 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
>  	return 0;
>  }
>  
> +int
> +rte_pci_enable_bus_master(struct rte_pci_device *dev)
> +{
> +	uint16_t cmd;
> +
> +	if (rte_pci_read_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
> +		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
> +		return -1;
> +	}
> +
> +	cmd |= RTE_PCI_COMMAND_MASTER;
> +
> +	if (rte_pci_write_config(dev, &cmd, sizeof(cmd), RTE_PCI_COMMAND) < 0) {
> +		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
>  struct rte_pci_bus rte_pci_bus = {
>  	.bus = {
>  		.scan = rte_pci_scan,
> diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
> index 64886b473..83caf477b 100644
> --- a/drivers/bus/pci/rte_bus_pci.h
> +++ b/drivers/bus/pci/rte_bus_pci.h
> @@ -249,6 +249,18 @@ void rte_pci_dump(FILE *f);
>  __rte_experimental
>  off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
>  
> +/**
> + * Enables Bus Master for device's PCI command register.
> + *
> + *  @param dev
> + *    A pointer to rte_pci_device structure.
> + *
> + *  @return
> + *  0 on success, -1 on error in PCI config space read/write.
> + */
> +__rte_experimental
> +int rte_pci_enable_bus_master(struct rte_pci_device *dev);
> +
>  /**
>   * Register a PCI driver.
>   *
> diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
> index f33ed0abd..9dbec12a0 100644
> --- a/drivers/bus/pci/version.map
> +++ b/drivers/bus/pci/version.map
> @@ -21,4 +21,7 @@ EXPERIMENTAL {
>  	global:
>  
>  	rte_pci_find_ext_capability;
> +
> +	# added in 21.05
> +	rte_pci_enable_bus_master;
>  };
> diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
> index a8f8e404a..1f33d687f 100644
> --- a/lib/pci/rte_pci.h
> +++ b/lib/pci/rte_pci.h
> @@ -32,6 +32,10 @@ extern "C" {
>  
>  #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
>  #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
> +#define RTE_PCI_COMMAND		0x04	/* 16 bits */
> +
> +/* PCI Command Register */
> +#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
>  
>  /* PCI Express capability registers */
>  #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
> 

Acked-by: Ray Kinsella <mdr@ashroe.eu>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register
  2021-04-23 11:39   ` [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register Haiyue Wang
  2021-04-23 12:32     ` Kinsella, Ray
@ 2021-04-27  9:28     ` David Marchand
  2021-04-27 13:34       ` Wang, Haiyue
  1 sibling, 1 reply; 46+ messages in thread
From: David Marchand @ 2021-04-27  9:28 UTC (permalink / raw)
  To: Haiyue Wang, Gaetan Rivet
  Cc: dev, Qi Zhang, liang-min.wang, Ray Kinsella, Neil Horman

On Fri, Apr 23, 2021 at 2:06 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
> index 64886b473..83caf477b 100644
> --- a/drivers/bus/pci/rte_bus_pci.h
> +++ b/drivers/bus/pci/rte_bus_pci.h
> @@ -249,6 +249,18 @@ void rte_pci_dump(FILE *f);
>  __rte_experimental
>  off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
>
> +/**
> + * Enables Bus Master for device's PCI command register.
> + *
> + *  @param dev
> + *    A pointer to rte_pci_device structure.
> + *
> + *  @return
> + *  0 on success, -1 on error in PCI config space read/write.
> + */
> +__rte_experimental
> +int rte_pci_enable_bus_master(struct rte_pci_device *dev);

I can see pci/vfio and net/hns3 has a similar helper to enable *and*
disable bus master.

I'd rather go with a "set" helper, and then we can clean existing
drivers who had their own helper:
drivers/bus/pci/linux/pci_uio.c
drivers/bus/pci/linux/pci_vfio.c
drivers/net/hns3/hns3_ethdev_vf.c


> +
>  /**
>   * Register a PCI driver.
>   *
> diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
> index f33ed0abd..9dbec12a0 100644
> --- a/drivers/bus/pci/version.map
> +++ b/drivers/bus/pci/version.map
> @@ -21,4 +21,7 @@ EXPERIMENTAL {
>         global:
>
>         rte_pci_find_ext_capability;
> +
> +       # added in 21.05
> +       rte_pci_enable_bus_master;
>  };
> diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
> index a8f8e404a..1f33d687f 100644
> --- a/lib/pci/rte_pci.h
> +++ b/lib/pci/rte_pci.h
> @@ -32,6 +32,10 @@ extern "C" {
>
>  #define RTE_PCI_VENDOR_ID      0x00    /* 16 bits */
>  #define RTE_PCI_DEVICE_ID      0x02    /* 16 bits */
> +#define RTE_PCI_COMMAND                0x04    /* 16 bits */

This file uses tab for indent.


> +
> +/* PCI Command Register */
> +#define RTE_PCI_COMMAND_MASTER 0x4     /* Bus Master Enable */
>
>  /* PCI Express capability registers */
>  #define RTE_PCI_EXP_DEVCTL     8       /* Device Control */
> --
> 2.31.1
>


-- 
David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register
  2021-04-27  9:28     ` David Marchand
@ 2021-04-27 13:34       ` Wang, Haiyue
  2021-04-27 13:40         ` David Marchand
  0 siblings, 1 reply; 46+ messages in thread
From: Wang, Haiyue @ 2021-04-27 13:34 UTC (permalink / raw)
  To: David Marchand, Gaetan Rivet
  Cc: dev, Zhang, Qi Z, Wang, Liang-min, Ray Kinsella, Neil Horman

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, April 27, 2021 17:29
> To: Wang, Haiyue <haiyue.wang@intel.com>; Gaetan Rivet <grive@u256.net>
> Cc: dev <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-min.wang@intel.com>;
> Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
> Subject: Re: [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register
> 
> On Fri, Apr 23, 2021 at 2:06 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> > diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
> > index 64886b473..83caf477b 100644
> > --- a/drivers/bus/pci/rte_bus_pci.h
> > +++ b/drivers/bus/pci/rte_bus_pci.h
> > @@ -249,6 +249,18 @@ void rte_pci_dump(FILE *f);
> >  __rte_experimental
> >  off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
> >
> > +/**
> > + * Enables Bus Master for device's PCI command register.
> > + *
> > + *  @param dev
> > + *    A pointer to rte_pci_device structure.
> > + *
> > + *  @return
> > + *  0 on success, -1 on error in PCI config space read/write.
> > + */
> > +__rte_experimental
> > +int rte_pci_enable_bus_master(struct rte_pci_device *dev);
> 
> I can see pci/vfio and net/hns3 has a similar helper to enable *and*
> disable bus master.
> 
> I'd rather go with a "set" helper, and then we can clean existing
> drivers who had their own helper:
> drivers/bus/pci/linux/pci_uio.c
> drivers/bus/pci/linux/pci_vfio.c
> drivers/net/hns3/hns3_ethdev_vf.c
> 
> 

Got it, will change it to 'set'.

> > +
> >  /**
> >   * Register a PCI driver.
> >   *
> > diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
> > index f33ed0abd..9dbec12a0 100644
> > --- a/drivers/bus/pci/version.map
> > +++ b/drivers/bus/pci/version.map
> > @@ -21,4 +21,7 @@ EXPERIMENTAL {
> >         global:
> >
> >         rte_pci_find_ext_capability;
> > +
> > +       # added in 21.05
> > +       rte_pci_enable_bus_master;
> >  };
> > diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
> > index a8f8e404a..1f33d687f 100644
> > --- a/lib/pci/rte_pci.h
> > +++ b/lib/pci/rte_pci.h
> > @@ -32,6 +32,10 @@ extern "C" {
> >
> >  #define RTE_PCI_VENDOR_ID      0x00    /* 16 bits */
> >  #define RTE_PCI_DEVICE_ID      0x02    /* 16 bits */
> > +#define RTE_PCI_COMMAND                0x04    /* 16 bits */
> 
> This file uses tab for indent.
> 

Yes, I use two tabs, then the vi editor shows that they are aligned.
but the patch doesn't show well.

So use one tab ?

> 
> > +
> > +/* PCI Command Register */
> > +#define RTE_PCI_COMMAND_MASTER 0x4     /* Bus Master Enable */
> >
> >  /* PCI Express capability registers */
> >  #define RTE_PCI_EXP_DEVCTL     8       /* Device Control */
> > --
> > 2.31.1
> >
> 
> 
> --
> David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v4 0/3] fix PF reset causes VF memory request failure
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
                   ` (5 preceding siblings ...)
  2021-04-23 11:39 ` [dpdk-dev] [PATCH v3 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-04-27 13:39 ` Haiyue Wang
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 1/3] bus/pci: set PCI master in command register Haiyue Wang
                     ` (2 more replies)
  2021-05-06  3:49 ` [dpdk-dev] [PATCH v5 0/3] fix PF reset causes VF memory request failure Haiyue Wang
                   ` (2 subsequent siblings)
  9 siblings, 3 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-27 13:39 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang

By triggerring the VF reset from PF reset,

        echo 1 > /sys/bus/pci/devices/PF-BDF/reset

the PCI bus master bit will cleared on VF, so the VF needs to enable
this bit before restart.

This patch set adds the API to enable PCI bus master.

v4: change the API to set type, so can enable or disable
v3: added the missed annotate symbol add time
v2: rebase to new librte directory path.

Haiyue Wang (3):
  bus/pci: set PCI master in command register
  net/iavf: enable PCI bus master after reset
  net/i40e: enable PCI bus master after reset

 drivers/bus/pci/pci_common.c      | 28 ++++++++++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h     | 14 ++++++++++++++
 drivers/bus/pci/version.map       |  3 +++
 drivers/net/i40e/i40e_ethdev_vf.c |  7 ++++++-
 drivers/net/iavf/iavf_ethdev.c    |  3 +++
 lib/pci/rte_pci.h                 |  4 ++++
 6 files changed, 58 insertions(+), 1 deletion(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v4 1/3] bus/pci: set PCI master in command register
  2021-04-27 13:39 ` [dpdk-dev] [PATCH v4 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-04-27 13:39   ` Haiyue Wang
  2021-04-27 15:07     ` Kinsella, Ray
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 3/3] net/i40e: " Haiyue Wang
  2 siblings, 1 reply; 46+ messages in thread
From: Haiyue Wang @ 2021-04-27 13:39 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Ray Kinsella, Neil Horman, Gaetan Rivet

Add the API to set 'Bus Master Enable' bit to be enabled or disabled in
the PCI command register.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/bus/pci/pci_common.c  | 28 ++++++++++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h | 14 ++++++++++++++
 drivers/bus/pci/version.map   |  3 +++
 lib/pci/rte_pci.h             |  4 ++++
 4 files changed, 49 insertions(+)

diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index ee7f96635..35d7d092d 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -746,6 +746,34 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
 	return 0;
 }
 
+int
+rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable)
+{
+	uint16_t old_cmd, cmd;
+
+	if (rte_pci_read_config(dev, &old_cmd, sizeof(old_cmd),
+				RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
+		return -1;
+	}
+
+	if (enable)
+		cmd = old_cmd | RTE_PCI_COMMAND_MASTER;
+	else
+		cmd = old_cmd & ~RTE_PCI_COMMAND_MASTER;
+
+	if (cmd == old_cmd)
+		return 0;
+
+	if (rte_pci_write_config(dev, &cmd, sizeof(cmd),
+				 RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 64886b473..976c33c92 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -249,6 +249,20 @@ void rte_pci_dump(FILE *f);
 __rte_experimental
 off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
 
+/**
+ * Enables/Disables Bus Master for device's PCI command register.
+ *
+ *  @param dev
+ *    A pointer to rte_pci_device structure.
+ *  @param enable
+ *    Enable or disable Bus Master.
+ *
+ *  @return
+ *  0 on success, -1 on error in PCI config space read/write.
+ */
+__rte_experimental
+int rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable);
+
 /**
  * Register a PCI driver.
  *
diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
index f33ed0abd..c6e5f797c 100644
--- a/drivers/bus/pci/version.map
+++ b/drivers/bus/pci/version.map
@@ -21,4 +21,7 @@ EXPERIMENTAL {
 	global:
 
 	rte_pci_find_ext_capability;
+
+	# added in 21.05
+	rte_pci_set_bus_master;
 };
diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
index a8f8e404a..1f33d687f 100644
--- a/lib/pci/rte_pci.h
+++ b/lib/pci/rte_pci.h
@@ -32,6 +32,10 @@ extern "C" {
 
 #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
 #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
+#define RTE_PCI_COMMAND		0x04	/* 16 bits */
+
+/* PCI Command Register */
+#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
 
 /* PCI Express capability registers */
 #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
  2021-04-27 13:39 ` [dpdk-dev] [PATCH v4 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 1/3] bus/pci: set PCI master in command register Haiyue Wang
@ 2021-04-27 13:39   ` Haiyue Wang
  2021-04-28  3:34     ` Zhang, Qi Z
  2021-05-04 11:32     ` David Marchand
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 3/3] net/i40e: " Haiyue Wang
  2 siblings, 2 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-04-27 13:39 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Jingjing Wu, Beilei Xing

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/iavf/iavf_ethdev.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index d523a0618..9a0a20a29 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
 
+	if (vf->vf_reset)
+		rte_pci_set_bus_master(pci_dev, true);
+
 	vf->vf_reset = false;
 
 	return ret;
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v4 3/3] net/i40e: enable PCI bus master after reset
  2021-04-27 13:39 ` [dpdk-dev] [PATCH v4 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 1/3] bus/pci: set PCI master in command register Haiyue Wang
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-04-27 13:39   ` Haiyue Wang
  2021-04-28  3:35     ` Zhang, Qi Z
  2 siblings, 1 reply; 46+ messages in thread
From: Haiyue Wang @ 2021-04-27 13:39 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Beilei Xing, Jeff Guo

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.

And align the VF reset event handling in device close module as the AVF
driver does.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/i40e/i40e_ethdev_vf.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index 3c258ba7c..8b041e94c 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1212,7 +1212,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev)
 	if (i >= MAX_RESET_WAIT_CNT)
 		return -1;
 
-	vf->vf_reset = false;
 	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
 
 	return 0;
@@ -1391,6 +1390,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg,
 	switch (pf_msg->event) {
 	case VIRTCHNL_EVENT_RESET_IMPENDING:
 		PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event");
+		vf->vf_reset = true;
 		rte_eth_dev_callback_process(dev,
 				RTE_ETH_EVENT_INTR_RESET, NULL);
 		break;
@@ -2487,6 +2487,11 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
+	if (vf->vf_reset)
+		rte_pci_set_bus_master(RTE_ETH_DEV_TO_PCI(dev), true);
+
+	vf->vf_reset = false;
+
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register
  2021-04-27 13:34       ` Wang, Haiyue
@ 2021-04-27 13:40         ` David Marchand
  0 siblings, 0 replies; 46+ messages in thread
From: David Marchand @ 2021-04-27 13:40 UTC (permalink / raw)
  To: Wang, Haiyue
  Cc: Gaetan Rivet, dev, Zhang, Qi Z, Wang, Liang-min, Ray Kinsella,
	Neil Horman

On Tue, Apr 27, 2021 at 3:35 PM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> > > diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
> > > index a8f8e404a..1f33d687f 100644
> > > --- a/lib/pci/rte_pci.h
> > > +++ b/lib/pci/rte_pci.h
> > > @@ -32,6 +32,10 @@ extern "C" {
> > >
> > >  #define RTE_PCI_VENDOR_ID      0x00    /* 16 bits */
> > >  #define RTE_PCI_DEVICE_ID      0x02    /* 16 bits */
> > > +#define RTE_PCI_COMMAND                0x04    /* 16 bits */
> >
> > This file uses tab for indent.
> >
>
> Yes, I use two tabs, then the vi editor shows that they are aligned.
> but the patch doesn't show well.
>
> So use one tab ?

Sorry, your patch was good.
You can discard this comment.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v4 1/3] bus/pci: set PCI master in command register
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 1/3] bus/pci: set PCI master in command register Haiyue Wang
@ 2021-04-27 15:07     ` Kinsella, Ray
  0 siblings, 0 replies; 46+ messages in thread
From: Kinsella, Ray @ 2021-04-27 15:07 UTC (permalink / raw)
  To: Haiyue Wang, dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Neil Horman, Gaetan Rivet



On 27/04/2021 14:39, Haiyue Wang wrote:
> Add the API to set 'Bus Master Enable' bit to be enabled or disabled in
> the PCI command register.
> 
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
>  drivers/bus/pci/pci_common.c  | 28 ++++++++++++++++++++++++++++
>  drivers/bus/pci/rte_bus_pci.h | 14 ++++++++++++++
>  drivers/bus/pci/version.map   |  3 +++
>  lib/pci/rte_pci.h             |  4 ++++
>  4 files changed, 49 insertions(+)
> 

Acked-by: Ray Kinsella <mdr@ashroe.eu>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-04-28  3:34     ` Zhang, Qi Z
  2021-05-04 11:32     ` David Marchand
  1 sibling, 0 replies; 46+ messages in thread
From: Zhang, Qi Z @ 2021-04-28  3:34 UTC (permalink / raw)
  To: Wang, Haiyue, dev
  Cc: Wang, Liang-min, david.marchand, Wu, Jingjing, Xing, Beilei



> -----Original Message-----
> From: Wang, Haiyue <haiyue.wang@intel.com>
> Sent: Tuesday, April 27, 2021 9:39 PM
> To: dev@dpdk.org
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min
> <liang-min.wang@intel.com>; david.marchand@redhat.com; Wang, Haiyue
> <haiyue.wang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Subject: [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
> 
> The VF reset can be triggerred by the PF reset event, in this case, the PCI bus
> master will be cleared, then the VF is not allowed to issue any Memory or I/O
> Requests.
> 
> So after the reset event is detected, always enable the PCI bus master.
> 
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
>  drivers/net/iavf/iavf_ethdev.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index d523a0618..9a0a20a29 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
>  	rte_free(vf->aq_resp);
>  	vf->aq_resp = NULL;
> 
> +	if (vf->vf_reset)
> +		rte_pci_set_bus_master(pci_dev, true);
> +
>  	vf->vf_reset = false;
> 
>  	return ret;
> --
> 2.31.1

Acked-by: Qi Zhang <qi.z.zhang@intel.com>



^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v4 3/3] net/i40e: enable PCI bus master after reset
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 3/3] net/i40e: " Haiyue Wang
@ 2021-04-28  3:35     ` Zhang, Qi Z
  0 siblings, 0 replies; 46+ messages in thread
From: Zhang, Qi Z @ 2021-04-28  3:35 UTC (permalink / raw)
  To: Wang, Haiyue, dev
  Cc: Wang, Liang-min, david.marchand, Xing, Beilei, Guo,  Jia



> -----Original Message-----
> From: Wang, Haiyue <haiyue.wang@intel.com>
> Sent: Tuesday, April 27, 2021 9:39 PM
> To: dev@dpdk.org
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min
> <liang-min.wang@intel.com>; david.marchand@redhat.com; Wang, Haiyue
> <haiyue.wang@intel.com>; Xing, Beilei <beilei.xing@intel.com>; Guo, Jia
> <jia.guo@intel.com>
> Subject: [PATCH v4 3/3] net/i40e: enable PCI bus master after reset
> 
> The VF reset can be triggerred by the PF reset event, in this case, the PCI bus
> master will be cleared, then the VF is not allowed to issue any Memory or I/O
> Requests.
> 
> So after the reset event is detected, always enable the PCI bus master.
> 
> And align the VF reset event handling in device close module as the AVF driver
> does.
> 
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
>  drivers/net/i40e/i40e_ethdev_vf.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c
> b/drivers/net/i40e/i40e_ethdev_vf.c
> index 3c258ba7c..8b041e94c 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -1212,7 +1212,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev
> *dev)
>  	if (i >= MAX_RESET_WAIT_CNT)
>  		return -1;
> 
> -	vf->vf_reset = false;
>  	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
> 
>  	return 0;
> @@ -1391,6 +1390,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev,
> uint8_t *msg,
>  	switch (pf_msg->event) {
>  	case VIRTCHNL_EVENT_RESET_IMPENDING:
>  		PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING
> event");
> +		vf->vf_reset = true;
>  		rte_eth_dev_callback_process(dev,
>  				RTE_ETH_EVENT_INTR_RESET, NULL);
>  		break;
> @@ -2487,6 +2487,11 @@ i40evf_dev_close(struct rte_eth_dev *dev)
>  	i40e_shutdown_adminq(hw);
>  	i40evf_disable_irq0(hw);
> 
> +	if (vf->vf_reset)
> +		rte_pci_set_bus_master(RTE_ETH_DEV_TO_PCI(dev), true);
> +
> +	vf->vf_reset = false;
> +
>  	rte_free(vf->vf_res);
>  	vf->vf_res = NULL;
>  	rte_free(vf->aq_resp);
> --
> 2.31.1

Acked-by: Qi Zhang <qi.z.zhang@intel.com>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
  2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
  2021-04-28  3:34     ` Zhang, Qi Z
@ 2021-05-04 11:32     ` David Marchand
  2021-05-04 15:07       ` Wang, Haiyue
  2021-05-05  2:56       ` Wang, Haiyue
  1 sibling, 2 replies; 46+ messages in thread
From: David Marchand @ 2021-05-04 11:32 UTC (permalink / raw)
  To: Haiyue Wang; +Cc: dev, Qi Zhang, Wang, Liang-min, Jingjing Wu, Beilei Xing

On Tue, Apr 27, 2021 at 4:05 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
>
> The VF reset can be triggerred by the PF reset event, in this case, the
> PCI bus master will be cleared, then the VF is not allowed to issue any
> Memory or I/O Requests.
>
> So after the reset event is detected, always enable the PCI bus master.
>
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
>  drivers/net/iavf/iavf_ethdev.c | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index d523a0618..9a0a20a29 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
>         rte_free(vf->aq_resp);
>         vf->aq_resp = NULL;
>
> +       if (vf->vf_reset)
> +               rte_pci_set_bus_master(pci_dev, true);
> +
>         vf->vf_reset = false;

Not checking for the return code can leave the device in an invalid state.
Then after this, calling the init code will fail.

I'd rather move rte_pci_set_bus_master() (it is a noop if bus master
is already enabled) in the init path and check for the return code
there?
WDYT?


-- 
David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
  2021-05-04 11:32     ` David Marchand
@ 2021-05-04 15:07       ` Wang, Haiyue
  2021-05-05  2:56       ` Wang, Haiyue
  1 sibling, 0 replies; 46+ messages in thread
From: Wang, Haiyue @ 2021-05-04 15:07 UTC (permalink / raw)
  To: David Marchand
  Cc: dev, Zhang, Qi Z, Wang, Liang-min, Wu, Jingjing, Xing, Beilei

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, May 4, 2021 19:32
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-min.wang@intel.com>;
> Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Subject: Re: [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
> 
> On Tue, Apr 27, 2021 at 4:05 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> >
> > The VF reset can be triggerred by the PF reset event, in this case, the
> > PCI bus master will be cleared, then the VF is not allowed to issue any
> > Memory or I/O Requests.
> >
> > So after the reset event is detected, always enable the PCI bus master.
> >
> > Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> > ---
> >  drivers/net/iavf/iavf_ethdev.c | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> > index d523a0618..9a0a20a29 100644
> > --- a/drivers/net/iavf/iavf_ethdev.c
> > +++ b/drivers/net/iavf/iavf_ethdev.c
> > @@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
> >         rte_free(vf->aq_resp);
> >         vf->aq_resp = NULL;
> >
> > +       if (vf->vf_reset)
> > +               rte_pci_set_bus_master(pci_dev, true);
> > +
> >         vf->vf_reset = false;
> 
> Not checking for the return code can leave the device in an invalid state.
> Then after this, calling the init code will fail.
> 
> I'd rather move rte_pci_set_bus_master() (it is a noop if bus master
> is already enabled) in the init path and check for the return code

Currently, just called when vf_reset happened. And I noticed that the kernel
doesn't handle pci_read/write_config_word return code in most cases. I'm wondering
if the PCI ops really failed, the system can return back ?

> there?
> WDYT?
> 
> 
> --
> David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
  2021-05-04 11:32     ` David Marchand
  2021-05-04 15:07       ` Wang, Haiyue
@ 2021-05-05  2:56       ` Wang, Haiyue
  2021-05-05  8:39         ` David Marchand
  1 sibling, 1 reply; 46+ messages in thread
From: Wang, Haiyue @ 2021-05-05  2:56 UTC (permalink / raw)
  To: David Marchand
  Cc: dev, Zhang, Qi Z, Wang, Liang-min, Wu, Jingjing, Xing, Beilei

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Tuesday, May 4, 2021 19:32
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-min.wang@intel.com>;
> Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Subject: Re: [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
> 
> On Tue, Apr 27, 2021 at 4:05 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> >
> > The VF reset can be triggerred by the PF reset event, in this case, the
> > PCI bus master will be cleared, then the VF is not allowed to issue any
> > Memory or I/O Requests.
> >
> > So after the reset event is detected, always enable the PCI bus master.
> >
> > Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> > ---
> >  drivers/net/iavf/iavf_ethdev.c | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> > index d523a0618..9a0a20a29 100644
> > --- a/drivers/net/iavf/iavf_ethdev.c
> > +++ b/drivers/net/iavf/iavf_ethdev.c
> > @@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
> >         rte_free(vf->aq_resp);
> >         vf->aq_resp = NULL;
> >
> > +       if (vf->vf_reset)
> > +               rte_pci_set_bus_master(pci_dev, true);
> > +
> >         vf->vf_reset = false;
> 
> Not checking for the return code can leave the device in an invalid state.
> Then after this, calling the init code will fail.

From the upper application's view, if this bus master fix can't recover
the device into valid state, then the device hotplug API should be used
to make the device fully recover. So I'd prefer to call bus master "try
best" to fix. If still have error, the system may be in bad state. 

The init code is mostly called from PCI VFIO/UIO device management which
has enabled bus master. If we put the bus master here, people may confuse.
Bind setting the bus master with 'vf->vf_reset' together looks better.

> 
> I'd rather move rte_pci_set_bus_master() (it is a noop if bus master
> is already enabled) in the init path and check for the return code

In fact, not real noop, since the reading PCI ops may fail. ;-)

> there?
> WDYT?
> 
> 
> --
> David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
  2021-05-05  2:56       ` Wang, Haiyue
@ 2021-05-05  8:39         ` David Marchand
  2021-05-06  3:02           ` Wang, Haiyue
  0 siblings, 1 reply; 46+ messages in thread
From: David Marchand @ 2021-05-05  8:39 UTC (permalink / raw)
  To: Wang, Haiyue
  Cc: dev, Zhang, Qi Z, Wang, Liang-min, Wu, Jingjing, Xing, Beilei

On Wed, May 5, 2021 at 4:56 AM Wang, Haiyue <haiyue.wang@intel.com> wrote:
>
> > -----Original Message-----
> > From: David Marchand <david.marchand@redhat.com>
> > Sent: Tuesday, May 4, 2021 19:32
> > To: Wang, Haiyue <haiyue.wang@intel.com>
> > Cc: dev <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-min.wang@intel.com>;
> > Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> > Subject: Re: [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
> >
> > On Tue, Apr 27, 2021 at 4:05 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> > >
> > > The VF reset can be triggerred by the PF reset event, in this case, the
> > > PCI bus master will be cleared, then the VF is not allowed to issue any
> > > Memory or I/O Requests.
> > >
> > > So after the reset event is detected, always enable the PCI bus master.
> > >
> > > Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> > > ---
> > >  drivers/net/iavf/iavf_ethdev.c | 3 +++
> > >  1 file changed, 3 insertions(+)
> > >
> > > diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> > > index d523a0618..9a0a20a29 100644
> > > --- a/drivers/net/iavf/iavf_ethdev.c
> > > +++ b/drivers/net/iavf/iavf_ethdev.c
> > > @@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
> > >         rte_free(vf->aq_resp);
> > >         vf->aq_resp = NULL;
> > >
> > > +       if (vf->vf_reset)
> > > +               rte_pci_set_bus_master(pci_dev, true);
> > > +
> > >         vf->vf_reset = false;
> >
> > Not checking for the return code can leave the device in an invalid state.
> > Then after this, calling the init code will fail.
>
> From the upper application's view, if this bus master fix can't recover
> the device into valid state, then the device hotplug API should be used
> to make the device fully recover. So I'd prefer to call bus master "try
> best" to fix. If still have error, the system may be in bad state.

I find it odd to put something required for (re)initialising in a
.dev_close ops.
Maybe the place is more into .dev_reset if you find it confusing in .dev_init.
But this is your driver, I'll let it to your judgement.


On the other hand, what is the point of not failing early/propagating
the rte_pci_set_bus_master error?
The driver can't work without bus master enabled, correct?


-- 
David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
  2021-05-05  8:39         ` David Marchand
@ 2021-05-06  3:02           ` Wang, Haiyue
  0 siblings, 0 replies; 46+ messages in thread
From: Wang, Haiyue @ 2021-05-06  3:02 UTC (permalink / raw)
  To: David Marchand
  Cc: dev, Zhang, Qi Z, Wang, Liang-min, Wu, Jingjing, Xing, Beilei

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Wednesday, May 5, 2021 16:39
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-min.wang@intel.com>;
> Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Subject: Re: [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
> 
> On Wed, May 5, 2021 at 4:56 AM Wang, Haiyue <haiyue.wang@intel.com> wrote:
> >
> > > -----Original Message-----
> > > From: David Marchand <david.marchand@redhat.com>
> > > Sent: Tuesday, May 4, 2021 19:32
> > > To: Wang, Haiyue <haiyue.wang@intel.com>
> > > Cc: dev <dev@dpdk.org>; Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-
> min.wang@intel.com>;
> > > Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> > > Subject: Re: [PATCH v4 2/3] net/iavf: enable PCI bus master after reset
> > >
> > > On Tue, Apr 27, 2021 at 4:05 PM Haiyue Wang <haiyue.wang@intel.com> wrote:
> > > >
> > > > The VF reset can be triggerred by the PF reset event, in this case, the
> > > > PCI bus master will be cleared, then the VF is not allowed to issue any
> > > > Memory or I/O Requests.
> > > >
> > > > So after the reset event is detected, always enable the PCI bus master.
> > > >
> > > > Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> > > > ---
> > > >  drivers/net/iavf/iavf_ethdev.c | 3 +++
> > > >  1 file changed, 3 insertions(+)
> > > >
> > > > diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> > > > index d523a0618..9a0a20a29 100644
> > > > --- a/drivers/net/iavf/iavf_ethdev.c
> > > > +++ b/drivers/net/iavf/iavf_ethdev.c
> > > > @@ -2255,6 +2255,9 @@ iavf_dev_close(struct rte_eth_dev *dev)
> > > >         rte_free(vf->aq_resp);
> > > >         vf->aq_resp = NULL;
> > > >
> > > > +       if (vf->vf_reset)
> > > > +               rte_pci_set_bus_master(pci_dev, true);
> > > > +
> > > >         vf->vf_reset = false;
> > >
> > > Not checking for the return code can leave the device in an invalid state.
> > > Then after this, calling the init code will fail.
> >
> > From the upper application's view, if this bus master fix can't recover
> > the device into valid state, then the device hotplug API should be used
> > to make the device fully recover. So I'd prefer to call bus master "try
> > best" to fix. If still have error, the system may be in bad state.
> 
> I find it odd to put something required for (re)initialising in a
> .dev_close ops.
> Maybe the place is more into .dev_reset if you find it confusing in .dev_init.
> But this is your driver, I'll let it to your judgement.
> 

In fact, the .dev_close is close to .dev_reset.

static int
iavf_dev_reset(struct rte_eth_dev *dev)
{
	int ret;

	ret = iavf_dev_uninit(dev); --> iavf_dev_close
	if (ret)
		return ret;

	return iavf_dev_init(dev);
}

> 
> On the other hand, what is the point of not failing early/propagating
> the rte_pci_set_bus_master error?
> The driver can't work without bus master enabled, correct?

From this point of view, correct. ;-)

Now I changed the vf_reset handling as bellow, if bus master failed, the
device will still be in VF reset state, so that other modules will return
immediately (the PMD has considered the VF reset state already).

if (vf->vf_reset &&
    !rte_pci_set_bus_master(pci_dev, true))
	vf->vf_reset = false;

> 
> 
> --
> David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v5 0/3] fix PF reset causes VF memory request failure
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
                   ` (6 preceding siblings ...)
  2021-04-27 13:39 ` [dpdk-dev] [PATCH v4 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-05-06  3:49 ` Haiyue Wang
  2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 1/3] bus/pci: set PCI master in command register Haiyue Wang
                     ` (2 more replies)
  2021-05-23 11:46 ` [dpdk-dev] [PATCH v6 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-05-24  1:23 ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  9 siblings, 3 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-06  3:49 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang

By triggerring the VF reset from PF reset,

        echo 1 > /sys/bus/pci/devices/PF-BDF/reset

the PCI bus master bit will cleared on VF, so the VF needs to enable
this bit before restart.

This patch set adds the API to enable PCI bus master.

v5: error handling if bus master enable failed
v4: change the API to set type, so can enable or disable
v3: added the missed annotate symbol add time
v2: rebase to new librte directory path

Haiyue Wang (3):
  bus/pci: set PCI master in command register
  net/iavf: enable PCI bus master after reset
  net/i40e: enable PCI bus master after reset

 drivers/bus/pci/pci_common.c      | 28 ++++++++++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h     | 14 ++++++++++++++
 drivers/bus/pci/version.map       |  3 +++
 drivers/net/i40e/i40e_ethdev_vf.c |  6 +++++-
 drivers/net/iavf/iavf_ethdev.c    |  3 ++-
 lib/pci/rte_pci.h                 |  4 ++++
 6 files changed, 56 insertions(+), 2 deletions(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v5 1/3] bus/pci: set PCI master in command register
  2021-05-06  3:49 ` [dpdk-dev] [PATCH v5 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-05-06  3:49   ` Haiyue Wang
  2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
  2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 3/3] net/i40e: " Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-06  3:49 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Ray Kinsella, Neil Horman, Gaetan Rivet

Add the API to set 'Bus Master Enable' bit to be enabled or disabled in
the PCI command register.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
---
 drivers/bus/pci/pci_common.c  | 28 ++++++++++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h | 14 ++++++++++++++
 drivers/bus/pci/version.map   |  3 +++
 lib/pci/rte_pci.h             |  4 ++++
 4 files changed, 49 insertions(+)

diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index ee7f96635..35d7d092d 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -746,6 +746,34 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
 	return 0;
 }
 
+int
+rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable)
+{
+	uint16_t old_cmd, cmd;
+
+	if (rte_pci_read_config(dev, &old_cmd, sizeof(old_cmd),
+				RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
+		return -1;
+	}
+
+	if (enable)
+		cmd = old_cmd | RTE_PCI_COMMAND_MASTER;
+	else
+		cmd = old_cmd & ~RTE_PCI_COMMAND_MASTER;
+
+	if (cmd == old_cmd)
+		return 0;
+
+	if (rte_pci_write_config(dev, &cmd, sizeof(cmd),
+				 RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 64886b473..976c33c92 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -249,6 +249,20 @@ void rte_pci_dump(FILE *f);
 __rte_experimental
 off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
 
+/**
+ * Enables/Disables Bus Master for device's PCI command register.
+ *
+ *  @param dev
+ *    A pointer to rte_pci_device structure.
+ *  @param enable
+ *    Enable or disable Bus Master.
+ *
+ *  @return
+ *  0 on success, -1 on error in PCI config space read/write.
+ */
+__rte_experimental
+int rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable);
+
 /**
  * Register a PCI driver.
  *
diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
index f33ed0abd..c6e5f797c 100644
--- a/drivers/bus/pci/version.map
+++ b/drivers/bus/pci/version.map
@@ -21,4 +21,7 @@ EXPERIMENTAL {
 	global:
 
 	rte_pci_find_ext_capability;
+
+	# added in 21.05
+	rte_pci_set_bus_master;
 };
diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
index a8f8e404a..1f33d687f 100644
--- a/lib/pci/rte_pci.h
+++ b/lib/pci/rte_pci.h
@@ -32,6 +32,10 @@ extern "C" {
 
 #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
 #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
+#define RTE_PCI_COMMAND		0x04	/* 16 bits */
+
+/* PCI Command Register */
+#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
 
 /* PCI Express capability registers */
 #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v5 2/3] net/iavf: enable PCI bus master after reset
  2021-05-06  3:49 ` [dpdk-dev] [PATCH v5 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 1/3] bus/pci: set PCI master in command register Haiyue Wang
@ 2021-05-06  3:49   ` Haiyue Wang
  2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 3/3] net/i40e: " Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-06  3:49 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Jingjing Wu, Beilei Xing

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.
And if failed, the device or system may be in an invalid state, so keep
the VF reset state to mark it as I/O error.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/iavf/iavf_ethdev.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index c06873d26..0084a083b 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -2356,7 +2356,8 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
 
-	vf->vf_reset = false;
+	if (vf->vf_reset && !rte_pci_set_bus_master(pci_dev, true))
+		vf->vf_reset = false;
 
 	return ret;
 }
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v5 3/3] net/i40e: enable PCI bus master after reset
  2021-05-06  3:49 ` [dpdk-dev] [PATCH v5 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 1/3] bus/pci: set PCI master in command register Haiyue Wang
  2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-05-06  3:49   ` Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-06  3:49 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang, Beilei Xing

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.
And if failed, the device or system may be in an invalid state, so keep
the VF reset state to mark it as I/O error.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/i40e/i40e_ethdev_vf.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index cb898bdb6..bded64842 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1213,7 +1213,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev)
 	if (i >= MAX_RESET_WAIT_CNT)
 		return -1;
 
-	vf->vf_reset = false;
 	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
 
 	return 0;
@@ -1392,6 +1391,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg,
 	switch (pf_msg->event) {
 	case VIRTCHNL_EVENT_RESET_IMPENDING:
 		PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event");
+		vf->vf_reset = true;
 		rte_eth_dev_callback_process(dev,
 				RTE_ETH_EVENT_INTR_RESET, NULL);
 		break;
@@ -2490,6 +2490,10 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
+	if (vf->vf_reset &&
+	    !rte_pci_set_bus_master(RTE_ETH_DEV_TO_PCI(dev), true))
+		vf->vf_reset = false;
+
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v6 0/3] fix PF reset causes VF memory request failure
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
                   ` (7 preceding siblings ...)
  2021-05-06  3:49 ` [dpdk-dev] [PATCH v5 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-05-23 11:46 ` Haiyue Wang
  2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 1/3] bus/pci: set PCI master in command register Haiyue Wang
                     ` (2 more replies)
  2021-05-24  1:23 ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  9 siblings, 3 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-23 11:46 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang

By triggerring the VF reset from PF reset,

        echo 1 > /sys/bus/pci/devices/PF-BDF/reset

the PCI bus master bit will cleared on VF, so the VF needs to enable
this bit before restart.

This patch set adds the API to enable PCI bus master.

v6: update the annotate symbol version, and add some comments in source code
v5: error handling if bus master enable failed
v4: change the API to set type, so can enable or disable
v3: added the missed annotate symbol add time
v2: rebase to new librte directory path

Haiyue Wang (3):
  bus/pci: set PCI master in command register
  net/iavf: enable PCI bus master after reset
  net/i40e: enable PCI bus master after reset

 drivers/bus/pci/pci_common.c      | 28 ++++++++++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h     | 14 ++++++++++++++
 drivers/bus/pci/version.map       |  3 +++
 drivers/net/i40e/i40e_ethdev_vf.c | 13 ++++++++++++-
 drivers/net/iavf/iavf_ethdev.c    | 10 +++++++++-
 lib/pci/rte_pci.h                 |  4 ++++
 6 files changed, 70 insertions(+), 2 deletions(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v6 1/3] bus/pci: set PCI master in command register
  2021-05-23 11:46 ` [dpdk-dev] [PATCH v6 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-05-23 11:46   ` Haiyue Wang
  2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
  2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 3/3] net/i40e: " Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-23 11:46 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Ray Kinsella, Neil Horman, Gaetan Rivet

Add the API to set 'Bus Master Enable' bit to be enabled or disabled in
the PCI command register.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
---
 drivers/bus/pci/pci_common.c  | 28 ++++++++++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h | 14 ++++++++++++++
 drivers/bus/pci/version.map   |  3 +++
 lib/pci/rte_pci.h             |  4 ++++
 4 files changed, 49 insertions(+)

diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index ee7f966358..35d7d092d1 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -746,6 +746,34 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
 	return 0;
 }
 
+int
+rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable)
+{
+	uint16_t old_cmd, cmd;
+
+	if (rte_pci_read_config(dev, &old_cmd, sizeof(old_cmd),
+				RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
+		return -1;
+	}
+
+	if (enable)
+		cmd = old_cmd | RTE_PCI_COMMAND_MASTER;
+	else
+		cmd = old_cmd & ~RTE_PCI_COMMAND_MASTER;
+
+	if (cmd == old_cmd)
+		return 0;
+
+	if (rte_pci_write_config(dev, &cmd, sizeof(cmd),
+				 RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 64886b4731..976c33c921 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -249,6 +249,20 @@ void rte_pci_dump(FILE *f);
 __rte_experimental
 off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
 
+/**
+ * Enables/Disables Bus Master for device's PCI command register.
+ *
+ *  @param dev
+ *    A pointer to rte_pci_device structure.
+ *  @param enable
+ *    Enable or disable Bus Master.
+ *
+ *  @return
+ *  0 on success, -1 on error in PCI config space read/write.
+ */
+__rte_experimental
+int rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable);
+
 /**
  * Register a PCI driver.
  *
diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
index f33ed0abd1..00fac8864c 100644
--- a/drivers/bus/pci/version.map
+++ b/drivers/bus/pci/version.map
@@ -21,4 +21,7 @@ EXPERIMENTAL {
 	global:
 
 	rte_pci_find_ext_capability;
+
+	# added in 21.08
+	rte_pci_set_bus_master;
 };
diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
index a8f8e404a9..1f33d687f4 100644
--- a/lib/pci/rte_pci.h
+++ b/lib/pci/rte_pci.h
@@ -32,6 +32,10 @@ extern "C" {
 
 #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
 #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
+#define RTE_PCI_COMMAND		0x04	/* 16 bits */
+
+/* PCI Command Register */
+#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
 
 /* PCI Express capability registers */
 #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v6 2/3] net/iavf: enable PCI bus master after reset
  2021-05-23 11:46 ` [dpdk-dev] [PATCH v6 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 1/3] bus/pci: set PCI master in command register Haiyue Wang
@ 2021-05-23 11:46   ` Haiyue Wang
  2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 3/3] net/i40e: " Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-23 11:46 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Jingjing Wu, Beilei Xing

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.
And if failed, the device or system may be in an invalid state, so keep
the VF reset state to mark it as I/O error.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/iavf/iavf_ethdev.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index d688c31cfb..a7ef7a6d4d 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -2356,7 +2356,15 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
 
-	vf->vf_reset = false;
+	/*
+	 * If the VF is reset via VFLR, the device will be knocked out of bus
+	 * master mode, and the driver will fail to recover from the reset. Fix
+	 * this by enabling bus mastering after every reset. In a non-VFLR case,
+	 * the bus master bit will not be disabled, and this call will have no
+	 * effect.
+	 */
+	if (vf->vf_reset && !rte_pci_set_bus_master(pci_dev, true))
+		vf->vf_reset = false;
 
 	return ret;
 }
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v6 3/3] net/i40e: enable PCI bus master after reset
  2021-05-23 11:46 ` [dpdk-dev] [PATCH v6 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 1/3] bus/pci: set PCI master in command register Haiyue Wang
  2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-05-23 11:46   ` Haiyue Wang
  2 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-23 11:46 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang, Beilei Xing

The VF reset can be triggerred by the PF reset event, in this case, the
PCI bus master will be cleared, then the VF is not allowed to issue any
Memory or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.
And if failed, the device or system may be in an invalid state, so keep
the VF reset state to mark it as I/O error.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/i40e/i40e_ethdev_vf.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index cb898bdb68..385ebedcd3 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1213,7 +1213,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev)
 	if (i >= MAX_RESET_WAIT_CNT)
 		return -1;
 
-	vf->vf_reset = false;
 	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
 
 	return 0;
@@ -1392,6 +1391,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg,
 	switch (pf_msg->event) {
 	case VIRTCHNL_EVENT_RESET_IMPENDING:
 		PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event");
+		vf->vf_reset = true;
 		rte_eth_dev_callback_process(dev,
 				RTE_ETH_EVENT_INTR_RESET, NULL);
 		break;
@@ -2468,6 +2468,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	int ret;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
@@ -2490,6 +2491,16 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
+	/*
+	 * If the VF is reset via VFLR, the device will be knocked out of bus
+	 * master mode, and the driver will fail to recover from the reset. Fix
+	 * this by enabling bus mastering after every reset. In a non-VFLR case,
+	 * the bus master bit will not be disabled, and this call will have no
+	 * effect.
+	 */
+	if (vf->vf_reset && !rte_pci_set_bus_master(pci_dev, true))
+		vf->vf_reset = false;
+
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure
  2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
                   ` (8 preceding siblings ...)
  2021-05-23 11:46 ` [dpdk-dev] [PATCH v6 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-05-24  1:23 ` Haiyue Wang
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 1/3] bus/pci: set PCI master in command register Haiyue Wang
                     ` (3 more replies)
  9 siblings, 4 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-24  1:23 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang

Trigger the VF reset from PF reset,

        echo 1 > /sys/bus/pci/devices/PF-BDF/reset

the PCI bus master bit will cleared on VF, so the VF needs to enable
this bit before restart.

This patch set adds the API to enable PCI bus master.

v7: fix the commit message typo, and update some description.
v6: update the annotate symbol version, and add some comments in source code
v5: error handling if bus master enable failed
v4: change the API to set type, so can enable or disable
v3: added the missed annotate symbol add time
v2: rebase to new librte directory path

Haiyue Wang (3):
  bus/pci: set PCI master in command register
  net/iavf: enable PCI bus master after reset
  net/i40e: enable PCI bus master after reset

 drivers/bus/pci/pci_common.c      | 28 ++++++++++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h     | 14 ++++++++++++++
 drivers/bus/pci/version.map       |  3 +++
 drivers/net/i40e/i40e_ethdev_vf.c | 13 ++++++++++++-
 drivers/net/iavf/iavf_ethdev.c    | 10 +++++++++-
 lib/pci/rte_pci.h                 |  4 ++++
 6 files changed, 70 insertions(+), 2 deletions(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v7 1/3] bus/pci: set PCI master in command register
  2021-05-24  1:23 ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure Haiyue Wang
@ 2021-05-24  1:23   ` Haiyue Wang
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
                     ` (2 subsequent siblings)
  3 siblings, 0 replies; 46+ messages in thread
From: Haiyue Wang @ 2021-05-24  1:23 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Ray Kinsella, Neil Horman, Gaetan Rivet

Add the API to set 'Bus Master Enable' bit to be enabled or disabled in
the PCI command register.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
---
 drivers/bus/pci/pci_common.c  | 28 ++++++++++++++++++++++++++++
 drivers/bus/pci/rte_bus_pci.h | 14 ++++++++++++++
 drivers/bus/pci/version.map   |  3 +++
 lib/pci/rte_pci.h             |  4 ++++
 4 files changed, 49 insertions(+)

diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
index ee7f966358..35d7d092d1 100644
--- a/drivers/bus/pci/pci_common.c
+++ b/drivers/bus/pci/pci_common.c
@@ -746,6 +746,34 @@ rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap)
 	return 0;
 }
 
+int
+rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable)
+{
+	uint16_t old_cmd, cmd;
+
+	if (rte_pci_read_config(dev, &old_cmd, sizeof(old_cmd),
+				RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in reading PCI command register\n");
+		return -1;
+	}
+
+	if (enable)
+		cmd = old_cmd | RTE_PCI_COMMAND_MASTER;
+	else
+		cmd = old_cmd & ~RTE_PCI_COMMAND_MASTER;
+
+	if (cmd == old_cmd)
+		return 0;
+
+	if (rte_pci_write_config(dev, &cmd, sizeof(cmd),
+				 RTE_PCI_COMMAND) < 0) {
+		RTE_LOG(ERR, EAL, "error in writing PCI command register\n");
+		return -1;
+	}
+
+	return 0;
+}
+
 struct rte_pci_bus rte_pci_bus = {
 	.bus = {
 		.scan = rte_pci_scan,
diff --git a/drivers/bus/pci/rte_bus_pci.h b/drivers/bus/pci/rte_bus_pci.h
index 64886b4731..976c33c921 100644
--- a/drivers/bus/pci/rte_bus_pci.h
+++ b/drivers/bus/pci/rte_bus_pci.h
@@ -249,6 +249,20 @@ void rte_pci_dump(FILE *f);
 __rte_experimental
 off_t rte_pci_find_ext_capability(struct rte_pci_device *dev, uint32_t cap);
 
+/**
+ * Enables/Disables Bus Master for device's PCI command register.
+ *
+ *  @param dev
+ *    A pointer to rte_pci_device structure.
+ *  @param enable
+ *    Enable or disable Bus Master.
+ *
+ *  @return
+ *  0 on success, -1 on error in PCI config space read/write.
+ */
+__rte_experimental
+int rte_pci_set_bus_master(struct rte_pci_device *dev, bool enable);
+
 /**
  * Register a PCI driver.
  *
diff --git a/drivers/bus/pci/version.map b/drivers/bus/pci/version.map
index f33ed0abd1..00fac8864c 100644
--- a/drivers/bus/pci/version.map
+++ b/drivers/bus/pci/version.map
@@ -21,4 +21,7 @@ EXPERIMENTAL {
 	global:
 
 	rte_pci_find_ext_capability;
+
+	# added in 21.08
+	rte_pci_set_bus_master;
 };
diff --git a/lib/pci/rte_pci.h b/lib/pci/rte_pci.h
index a8f8e404a9..1f33d687f4 100644
--- a/lib/pci/rte_pci.h
+++ b/lib/pci/rte_pci.h
@@ -32,6 +32,10 @@ extern "C" {
 
 #define RTE_PCI_VENDOR_ID	0x00	/* 16 bits */
 #define RTE_PCI_DEVICE_ID	0x02	/* 16 bits */
+#define RTE_PCI_COMMAND		0x04	/* 16 bits */
+
+/* PCI Command Register */
+#define RTE_PCI_COMMAND_MASTER	0x4	/* Bus Master Enable */
 
 /* PCI Express capability registers */
 #define RTE_PCI_EXP_DEVCTL	8	/* Device Control */
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v7 2/3] net/iavf: enable PCI bus master after reset
  2021-05-24  1:23 ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 1/3] bus/pci: set PCI master in command register Haiyue Wang
@ 2021-05-24  1:23   ` Haiyue Wang
  2021-06-04  2:07     ` Xing, Beilei
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 3/3] net/i40e: " Haiyue Wang
  2021-06-08  8:31   ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure David Marchand
  3 siblings, 1 reply; 46+ messages in thread
From: Haiyue Wang @ 2021-05-24  1:23 UTC (permalink / raw)
  To: dev
  Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang,
	Jingjing Wu, Beilei Xing

The VF reset can be triggered by the PF reset event, then the PCI bus
master will be cleared, the VF will be not allowed to issue any Memory
or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.
And if failed, the device or system may be in an invalid state, so keep
the VF reset state to mark it as I/O error.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/iavf/iavf_ethdev.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index d688c31cfb..a7ef7a6d4d 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -2356,7 +2356,15 @@ iavf_dev_close(struct rte_eth_dev *dev)
 	rte_free(vf->aq_resp);
 	vf->aq_resp = NULL;
 
-	vf->vf_reset = false;
+	/*
+	 * If the VF is reset via VFLR, the device will be knocked out of bus
+	 * master mode, and the driver will fail to recover from the reset. Fix
+	 * this by enabling bus mastering after every reset. In a non-VFLR case,
+	 * the bus master bit will not be disabled, and this call will have no
+	 * effect.
+	 */
+	if (vf->vf_reset && !rte_pci_set_bus_master(pci_dev, true))
+		vf->vf_reset = false;
 
 	return ret;
 }
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* [dpdk-dev] [PATCH v7 3/3] net/i40e: enable PCI bus master after reset
  2021-05-24  1:23 ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure Haiyue Wang
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 1/3] bus/pci: set PCI master in command register Haiyue Wang
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-05-24  1:23   ` Haiyue Wang
  2021-06-04  1:58     ` Xing, Beilei
  2021-06-08  8:31   ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure David Marchand
  3 siblings, 1 reply; 46+ messages in thread
From: Haiyue Wang @ 2021-05-24  1:23 UTC (permalink / raw)
  To: dev; +Cc: qi.z.zhang, liang-min.wang, david.marchand, Haiyue Wang, Beilei Xing

The VF reset can be triggered by the PF reset event, then the PCI bus
master will be cleared, the VF will be not allowed to issue any Memory
or I/O Requests.

So after the reset event is detected, always enable the PCI bus master.
And if failed, the device or system may be in an invalid state, so keep
the VF reset state to mark it as I/O error.

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
---
 drivers/net/i40e/i40e_ethdev_vf.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c b/drivers/net/i40e/i40e_ethdev_vf.c
index cb898bdb68..385ebedcd3 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1213,7 +1213,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev *dev)
 	if (i >= MAX_RESET_WAIT_CNT)
 		return -1;
 
-	vf->vf_reset = false;
 	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
 
 	return 0;
@@ -1392,6 +1391,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev, uint8_t *msg,
 	switch (pf_msg->event) {
 	case VIRTCHNL_EVENT_RESET_IMPENDING:
 		PMD_DRV_LOG(DEBUG, "VIRTCHNL_EVENT_RESET_IMPENDING event");
+		vf->vf_reset = true;
 		rte_eth_dev_callback_process(dev,
 				RTE_ETH_EVENT_INTR_RESET, NULL);
 		break;
@@ -2468,6 +2468,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 {
 	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
+	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	int ret;
 
 	if (rte_eal_process_type() != RTE_PROC_PRIMARY)
@@ -2490,6 +2491,16 @@ i40evf_dev_close(struct rte_eth_dev *dev)
 	i40e_shutdown_adminq(hw);
 	i40evf_disable_irq0(hw);
 
+	/*
+	 * If the VF is reset via VFLR, the device will be knocked out of bus
+	 * master mode, and the driver will fail to recover from the reset. Fix
+	 * this by enabling bus mastering after every reset. In a non-VFLR case,
+	 * the bus master bit will not be disabled, and this call will have no
+	 * effect.
+	 */
+	if (vf->vf_reset && !rte_pci_set_bus_master(pci_dev, true))
+		vf->vf_reset = false;
+
 	rte_free(vf->vf_res);
 	vf->vf_res = NULL;
 	rte_free(vf->aq_resp);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v7 3/3] net/i40e: enable PCI bus master after reset
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 3/3] net/i40e: " Haiyue Wang
@ 2021-06-04  1:58     ` Xing, Beilei
  0 siblings, 0 replies; 46+ messages in thread
From: Xing, Beilei @ 2021-06-04  1:58 UTC (permalink / raw)
  To: Wang, Haiyue, dev; +Cc: Zhang, Qi Z, Wang, Liang-min, david.marchand



> -----Original Message-----
> From: Wang, Haiyue <haiyue.wang@intel.com>
> Sent: Monday, May 24, 2021 9:24 AM
> To: dev@dpdk.org
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-
> min.wang@intel.com>; david.marchand@redhat.com; Wang, Haiyue
> <haiyue.wang@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Subject: [PATCH v7 3/3] net/i40e: enable PCI bus master after reset
> 
> The VF reset can be triggered by the PF reset event, then the PCI bus master
> will be cleared, the VF will be not allowed to issue any Memory or I/O
> Requests.
> 
> So after the reset event is detected, always enable the PCI bus master.
> And if failed, the device or system may be in an invalid state, so keep the VF
> reset state to mark it as I/O error.
> 
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
>  drivers/net/i40e/i40e_ethdev_vf.c | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/i40e/i40e_ethdev_vf.c
> b/drivers/net/i40e/i40e_ethdev_vf.c
> index cb898bdb68..385ebedcd3 100644
> --- a/drivers/net/i40e/i40e_ethdev_vf.c
> +++ b/drivers/net/i40e/i40e_ethdev_vf.c
> @@ -1213,7 +1213,6 @@ i40evf_check_vf_reset_done(struct rte_eth_dev
> *dev)
>  	if (i >= MAX_RESET_WAIT_CNT)
>  		return -1;
> 
> -	vf->vf_reset = false;
>  	vf->pend_msg &= ~PFMSG_RESET_IMPENDING;
> 
>  	return 0;
> @@ -1392,6 +1391,7 @@ i40evf_handle_pf_event(struct rte_eth_dev *dev,
> uint8_t *msg,
>  	switch (pf_msg->event) {
>  	case VIRTCHNL_EVENT_RESET_IMPENDING:
>  		PMD_DRV_LOG(DEBUG,
> "VIRTCHNL_EVENT_RESET_IMPENDING event");
> +		vf->vf_reset = true;
>  		rte_eth_dev_callback_process(dev,
>  				RTE_ETH_EVENT_INTR_RESET, NULL);
>  		break;
> @@ -2468,6 +2468,7 @@ i40evf_dev_close(struct rte_eth_dev *dev)  {
>  	struct i40e_hw *hw = I40E_DEV_PRIVATE_TO_HW(dev->data-
> >dev_private);
>  	struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data-
> >dev_private);
> +	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
>  	int ret;
> 
>  	if (rte_eal_process_type() != RTE_PROC_PRIMARY) @@ -2490,6
> +2491,16 @@ i40evf_dev_close(struct rte_eth_dev *dev)
>  	i40e_shutdown_adminq(hw);
>  	i40evf_disable_irq0(hw);
> 
> +	/*
> +	 * If the VF is reset via VFLR, the device will be knocked out of bus
> +	 * master mode, and the driver will fail to recover from the reset. Fix
> +	 * this by enabling bus mastering after every reset. In a non-VFLR
> case,
> +	 * the bus master bit will not be disabled, and this call will have no
> +	 * effect.
> +	 */
> +	if (vf->vf_reset && !rte_pci_set_bus_master(pci_dev, true))
> +		vf->vf_reset = false;
> +
>  	rte_free(vf->vf_res);
>  	vf->vf_res = NULL;
>  	rte_free(vf->aq_resp);
> --
> 2.31.1

Acked-by: Beilei Xing <beilei.xing@intel.com>

^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v7 2/3] net/iavf: enable PCI bus master after reset
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
@ 2021-06-04  2:07     ` Xing, Beilei
  0 siblings, 0 replies; 46+ messages in thread
From: Xing, Beilei @ 2021-06-04  2:07 UTC (permalink / raw)
  To: Wang, Haiyue, dev
  Cc: Zhang, Qi Z, Wang, Liang-min, david.marchand, Wu, Jingjing



> -----Original Message-----
> From: Wang, Haiyue <haiyue.wang@intel.com>
> Sent: Monday, May 24, 2021 9:24 AM
> To: dev@dpdk.org
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Wang, Liang-min <liang-
> min.wang@intel.com>; david.marchand@redhat.com; Wang, Haiyue
> <haiyue.wang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> <beilei.xing@intel.com>
> Subject: [PATCH v7 2/3] net/iavf: enable PCI bus master after reset
> 
> The VF reset can be triggered by the PF reset event, then the PCI bus master
> will be cleared, the VF will be not allowed to issue any Memory or I/O
> Requests.
> 
> So after the reset event is detected, always enable the PCI bus master.
> And if failed, the device or system may be in an invalid state, so keep the VF
> reset state to mark it as I/O error.
> 
> Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
> ---
>  drivers/net/iavf/iavf_ethdev.c | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index d688c31cfb..a7ef7a6d4d 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -2356,7 +2356,15 @@ iavf_dev_close(struct rte_eth_dev *dev)
>  	rte_free(vf->aq_resp);
>  	vf->aq_resp = NULL;
> 
> -	vf->vf_reset = false;
> +	/*
> +	 * If the VF is reset via VFLR, the device will be knocked out of bus
> +	 * master mode, and the driver will fail to recover from the reset. Fix
> +	 * this by enabling bus mastering after every reset. In a non-VFLR
> case,
> +	 * the bus master bit will not be disabled, and this call will have no
> +	 * effect.
> +	 */
> +	if (vf->vf_reset && !rte_pci_set_bus_master(pci_dev, true))
> +		vf->vf_reset = false;
> 
>  	return ret;
>  }
> --
> 2.31.1

Acked-by: Beilei Xing <beilei.xing@intel.com>


^ permalink raw reply	[flat|nested] 46+ messages in thread

* Re: [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure
  2021-05-24  1:23 ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure Haiyue Wang
                     ` (2 preceding siblings ...)
  2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 3/3] net/i40e: " Haiyue Wang
@ 2021-06-08  8:31   ` David Marchand
  3 siblings, 0 replies; 46+ messages in thread
From: David Marchand @ 2021-06-08  8:31 UTC (permalink / raw)
  To: Haiyue Wang; +Cc: dev, Qi Zhang, Wang, Liang-min

On Mon, May 24, 2021 at 3:44 AM Haiyue Wang <haiyue.wang@intel.com> wrote:
>
> Trigger the VF reset from PF reset,
>
>         echo 1 > /sys/bus/pci/devices/PF-BDF/reset
>
> the PCI bus master bit will cleared on VF, so the VF needs to enable
> this bit before restart.
>
> This patch set adds the API to enable PCI bus master.
>
> v7: fix the commit message typo, and update some description.
> v6: update the annotate symbol version, and add some comments in source code
> v5: error handling if bus master enable failed
> v4: change the API to set type, so can enable or disable
> v3: added the missed annotate symbol add time
> v2: rebase to new librte directory path
>
> Haiyue Wang (3):
>   bus/pci: set PCI master in command register
>   net/iavf: enable PCI bus master after reset
>   net/i40e: enable PCI bus master after reset

Series applied, thanks Haiyue.
I'll post a followup patch on using this API in other parts of dpdk.


-- 
David Marchand


^ permalink raw reply	[flat|nested] 46+ messages in thread

end of thread, other threads:[~2021-06-08  8:31 UTC | newest]

Thread overview: 46+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-21  5:02 [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Haiyue Wang
2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 1/3] bus/pci: enable PCI master in command register Haiyue Wang
2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
2021-04-21  5:02 ` [dpdk-dev] [PATCH v1 3/3] net/i40e: " Haiyue Wang
2021-04-21 11:59 ` [dpdk-dev] [PATCH v1 0/3] Fix PF reset causes VF memory request failure Zhang, Qi Z
2021-04-22  1:18 ` [dpdk-dev] [PATCH v2 0/3] fix " Haiyue Wang
2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 1/3] bus/pci: enable PCI master in command register Haiyue Wang
2021-04-23 10:43     ` Kinsella, Ray
2021-04-23 12:07       ` Wang, Haiyue
2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
2021-04-22  1:18   ` [dpdk-dev] [PATCH v2 3/3] net/i40e: " Haiyue Wang
2021-04-23 11:39 ` [dpdk-dev] [PATCH v3 0/3] fix PF reset causes VF memory request failure Haiyue Wang
2021-04-23 11:39   ` [dpdk-dev] [PATCH v3 1/3] bus/pci: enable PCI master in command register Haiyue Wang
2021-04-23 12:32     ` Kinsella, Ray
2021-04-27  9:28     ` David Marchand
2021-04-27 13:34       ` Wang, Haiyue
2021-04-27 13:40         ` David Marchand
2021-04-23 11:40   ` [dpdk-dev] [PATCH v3 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
2021-04-23 11:40   ` [dpdk-dev] [PATCH v3 3/3] net/i40e: " Haiyue Wang
2021-04-27 13:39 ` [dpdk-dev] [PATCH v4 0/3] fix PF reset causes VF memory request failure Haiyue Wang
2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 1/3] bus/pci: set PCI master in command register Haiyue Wang
2021-04-27 15:07     ` Kinsella, Ray
2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
2021-04-28  3:34     ` Zhang, Qi Z
2021-05-04 11:32     ` David Marchand
2021-05-04 15:07       ` Wang, Haiyue
2021-05-05  2:56       ` Wang, Haiyue
2021-05-05  8:39         ` David Marchand
2021-05-06  3:02           ` Wang, Haiyue
2021-04-27 13:39   ` [dpdk-dev] [PATCH v4 3/3] net/i40e: " Haiyue Wang
2021-04-28  3:35     ` Zhang, Qi Z
2021-05-06  3:49 ` [dpdk-dev] [PATCH v5 0/3] fix PF reset causes VF memory request failure Haiyue Wang
2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 1/3] bus/pci: set PCI master in command register Haiyue Wang
2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
2021-05-06  3:49   ` [dpdk-dev] [PATCH v5 3/3] net/i40e: " Haiyue Wang
2021-05-23 11:46 ` [dpdk-dev] [PATCH v6 0/3] fix PF reset causes VF memory request failure Haiyue Wang
2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 1/3] bus/pci: set PCI master in command register Haiyue Wang
2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
2021-05-23 11:46   ` [dpdk-dev] [PATCH v6 3/3] net/i40e: " Haiyue Wang
2021-05-24  1:23 ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure Haiyue Wang
2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 1/3] bus/pci: set PCI master in command register Haiyue Wang
2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 2/3] net/iavf: enable PCI bus master after reset Haiyue Wang
2021-06-04  2:07     ` Xing, Beilei
2021-05-24  1:23   ` [dpdk-dev] [PATCH v7 3/3] net/i40e: " Haiyue Wang
2021-06-04  1:58     ` Xing, Beilei
2021-06-08  8:31   ` [dpdk-dev] [PATCH v7 0/3] fix PF reset causes VF memory request failure David Marchand

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).