[dpdk-dev] [PATCH net-next v4 0/4] set ixgbe/ixgbevf EITR for performance

[dpdk-dev] [PATCH net-next v4 0/4] set ixgbe/ixgbevf EITR for performance

[xiangxia.m.yue]

From: Tonghao Zhang <xiangxia.m.yue@gmail.com>

v4: for reducing config options, remove the config option and use the
    default value defined in header file. Remove the patch:
    http://dpdk.org/dev/patchwork/patch/36433

    rebase the net-next.

v3: configure eitr via configuration file and remove the patch:
    http://dpdk.org/dev/patchwork/patch/33698

Tonghao Zhang (4):
  net/ixgbevf: set the interrupt interval for EITR
  net/ixgbe: set the default value for EITR
  net/ixgbe: write disable to EITR counter
  net/ixgbevf: save IXGBE_VTEIMS to intr->mask for performance.

 drivers/net/ixgbe/ixgbe_ethdev.c | 68 ++++++++++++++++++++++++++--------------
 drivers/net/ixgbe/ixgbe_ethdev.h |  1 +
 drivers/net/ixgbe/ixgbe_rxtx.c   |  3 +-
 3 files changed, 47 insertions(+), 25 deletions(-)

-- 
1.8.3.1

[dpdk-dev] [PATCH net-next v4 1/4] net/ixgbevf: set the interrupt interval for EITR

[xiangxia.m.yue]

From: Tonghao Zhang <xiangxia.m.yue@gmail.com>

Set EITR interval as default. This patch can improve the
performance when we enable the rx-interrupt to process the
packets because we hope rx-interrupt reduce CPU. For example,
the 200us value of EITR makes the performance better with
the low CPU. The default value of ITR is 500us, compatible
with RSC of ixgbe PF, and next patch will use the default value.

Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 7 +++++++
 drivers/net/ixgbe/ixgbe_ethdev.h | 1 +
 2 files changed, 8 insertions(+)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 6088c7e..bdede8b 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -5803,6 +5803,13 @@ static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 		if (vector_idx < base + intr_handle->nb_efd - 1)
 			vector_idx++;
 	}
+
+	/* As RX queue setting above show, all queues use the vector 0.
+	 * Set only the ITR value of IXGBE_MISC_VEC_ID.
+	 */
+	IXGBE_WRITE_REG(hw, IXGBE_VTEITR(IXGBE_MISC_VEC_ID),
+			IXGBE_EITR_INTERVAL_US(IXGBE_QUEUE_ITR_INTERVAL_DEFAULT)
+			| IXGBE_EITR_CNT_WDIS);
 }
 
 /**
diff --git a/drivers/net/ixgbe/ixgbe_ethdev.h b/drivers/net/ixgbe/ixgbe_ethdev.h
index cc512d6..e42ec30 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.h
+++ b/drivers/net/ixgbe/ixgbe_ethdev.h
@@ -62,6 +62,7 @@
 	(((us) * 1000 / IXGBE_EITR_INTERVAL_UNIT_NS << IXGBE_EITR_ITR_INT_SHIFT) & \
 		IXGBE_EITR_ITR_INT_MASK)
 
+#define IXGBE_QUEUE_ITR_INTERVAL_DEFAULT	500 /* 500us */
 
 /* Loopback operation modes */
 /* 82599 specific loopback operation types */
-- 
1.8.3.1

[dpdk-dev] [PATCH net-next v4 2/4] net/ixgbe: set the default value for EITR

[xiangxia.m.yue]

From: Tonghao Zhang <xiangxia.m.yue@gmail.com>

The ixgbe PF and VF use the same default value
for EITR defined in header file.

Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 5 +----
 drivers/net/ixgbe/ixgbe_rxtx.c   | 3 ++-
 2 files changed, 3 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index bdede8b..fcc10f6 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -59,9 +59,6 @@
  */
 #define IXGBE_FC_LO    0x40
 
-/* Default minimum inter-interrupt interval for EITR configuration */
-#define IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT    0x79E
-
 /* Timer value included in XOFF frames. */
 #define IXGBE_FC_PAUSE 0x680
 
@@ -5879,7 +5876,7 @@ static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 		break;
 	}
 	IXGBE_WRITE_REG(hw, IXGBE_EITR(IXGBE_MISC_VEC_ID),
-			IXGBE_MIN_INTER_INTERRUPT_INTERVAL_DEFAULT & 0xFFF);
+			IXGBE_EITR_INTERVAL_US(IXGBE_QUEUE_ITR_INTERVAL_DEFAULT));
 
 	/* set up to autoclear timer, and the vectors */
 	mask = IXGBE_EIMS_ENABLE_MASK;
diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c
index 2892436..ce19358 100644
--- a/drivers/net/ixgbe/ixgbe_rxtx.c
+++ b/drivers/net/ixgbe/ixgbe_rxtx.c
@@ -4842,7 +4842,8 @@ void __attribute__((cold))
 		 * at most 500us latency for a single RSC aggregation.
 		 */
 		eitr &= ~IXGBE_EITR_ITR_INT_MASK;
-		eitr |= IXGBE_EITR_INTERVAL_US(500) | IXGBE_EITR_CNT_WDIS;
+		eitr |= IXGBE_EITR_INTERVAL_US(IXGBE_QUEUE_ITR_INTERVAL_DEFAULT);
+		eitr |= IXGBE_EITR_CNT_WDIS;
 
 		IXGBE_WRITE_REG(hw, IXGBE_SRRCTL(rxq->reg_idx), srrctl);
 		IXGBE_WRITE_REG(hw, IXGBE_RSCCTL(rxq->reg_idx), rscctl);
-- 
1.8.3.1

[dpdk-dev] [PATCH net-next v4 3/4] net/ixgbe: write disable to EITR counter

[xiangxia.m.yue]

From: Tonghao Zhang <xiangxia.m.yue@gmail.com>

ixgbe doesn't write the EITR counter, disable it now.

Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index fcc10f6..07b4c1b 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -5876,7 +5876,8 @@ static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 		break;
 	}
 	IXGBE_WRITE_REG(hw, IXGBE_EITR(IXGBE_MISC_VEC_ID),
-			IXGBE_EITR_INTERVAL_US(IXGBE_QUEUE_ITR_INTERVAL_DEFAULT));
+			IXGBE_EITR_INTERVAL_US(IXGBE_QUEUE_ITR_INTERVAL_DEFAULT)
+			| IXGBE_EITR_CNT_WDIS);
 
 	/* set up to autoclear timer, and the vectors */
 	mask = IXGBE_EIMS_ENABLE_MASK;
-- 
1.8.3.1

[dpdk-dev] [PATCH net-next v4 4/4] net/ixgbevf: save IXGBE_VTEIMS to intr->mask for performance.

[xiangxia.m.yue]

From: Tonghao Zhang <xiangxia.m.yue@gmail.com>

If dpdk APPs call the rte_eth_dev_rx_intr_enable or
rte_eth_dev_rx_intr_disable frequently, and ixgbe vf will read
the IXGBE_VTEIMS register everytime. The patch saves the IXGBE_VTEIMS
to mask to avoid read frequently.

Signed-off-by: Tonghao Zhang <xiangxia.m.yue@gmail.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Wei Dai <wei.dai@intel.com>
---
 drivers/net/ixgbe/ixgbe_ethdev.c | 55 +++++++++++++++++++++++++---------------
 1 file changed, 35 insertions(+), 20 deletions(-)

diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c
index 07b4c1b..95c4d28 100644
--- a/drivers/net/ixgbe/ixgbe_ethdev.c
+++ b/drivers/net/ixgbe/ixgbe_ethdev.c
@@ -237,8 +237,8 @@ static int ixgbevf_dev_link_update(struct rte_eth_dev *dev,
 static void ixgbevf_dev_stop(struct rte_eth_dev *dev);
 static void ixgbevf_dev_close(struct rte_eth_dev *dev);
 static int  ixgbevf_dev_reset(struct rte_eth_dev *dev);
-static void ixgbevf_intr_disable(struct ixgbe_hw *hw);
-static void ixgbevf_intr_enable(struct ixgbe_hw *hw);
+static void ixgbevf_intr_disable(struct rte_eth_dev *dev);
+static void ixgbevf_intr_enable(struct rte_eth_dev *dev);
 static int ixgbevf_dev_stats_get(struct rte_eth_dev *dev,
 		struct rte_eth_stats *stats);
 static void ixgbevf_dev_stats_reset(struct rte_eth_dev *dev);
@@ -1603,7 +1603,7 @@ static int ixgbe_l2_tn_filter_init(struct rte_eth_dev *eth_dev)
 	ixgbevf_dev_stats_reset(eth_dev);
 
 	/* Disable the interrupts for VF */
-	ixgbevf_intr_disable(hw);
+	ixgbevf_intr_disable(eth_dev);
 
 	hw->mac.num_rar_entries = 128; /* The MAX of the underlying PF */
 	diag = hw->mac.ops.reset_hw(hw);
@@ -1672,7 +1672,7 @@ static int ixgbe_l2_tn_filter_init(struct rte_eth_dev *eth_dev)
 	rte_intr_callback_register(intr_handle,
 				   ixgbevf_dev_interrupt_handler, eth_dev);
 	rte_intr_enable(intr_handle);
-	ixgbevf_intr_enable(hw);
+	ixgbevf_intr_enable(eth_dev);
 
 	PMD_INIT_LOG(DEBUG, "port %d vendorID=0x%x deviceID=0x%x mac.type=%s",
 		     eth_dev->data->port_id, pci_dev->id.vendor_id,
@@ -1705,7 +1705,7 @@ static int ixgbe_l2_tn_filter_init(struct rte_eth_dev *eth_dev)
 	eth_dev->tx_pkt_burst = NULL;
 
 	/* Disable the interrupts for VF */
-	ixgbevf_intr_disable(hw);
+	ixgbevf_intr_disable(eth_dev);
 
 	rte_free(eth_dev->data->mac_addrs);
 	eth_dev->data->mac_addrs = NULL;
@@ -4933,19 +4933,32 @@ static int ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
  * Virtual Function operations
  */
 static void
-ixgbevf_intr_disable(struct ixgbe_hw *hw)
+ixgbevf_intr_disable(struct rte_eth_dev *dev)
 {
+	struct ixgbe_interrupt *intr =
+		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
+	struct ixgbe_hw *hw =
+		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
 	PMD_INIT_FUNC_TRACE();
 
 	/* Clear interrupt mask to stop from interrupts being generated */
 	IXGBE_WRITE_REG(hw, IXGBE_VTEIMC, IXGBE_VF_IRQ_CLEAR_MASK);
 
 	IXGBE_WRITE_FLUSH(hw);
+
+	/* Clear mask value. */
+	intr->mask = 0;
 }
 
 static void
-ixgbevf_intr_enable(struct ixgbe_hw *hw)
+ixgbevf_intr_enable(struct rte_eth_dev *dev)
 {
+	struct ixgbe_interrupt *intr =
+		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
+	struct ixgbe_hw *hw =
+		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
+
 	PMD_INIT_FUNC_TRACE();
 
 	/* VF enable interrupt autoclean */
@@ -4954,6 +4967,9 @@ static int ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
 	IXGBE_WRITE_REG(hw, IXGBE_VTEIMS, IXGBE_VF_IRQ_ENABLE_MASK);
 
 	IXGBE_WRITE_FLUSH(hw);
+
+	/* Save IXGBE_VTEIMS value to mask. */
+	intr->mask = IXGBE_VF_IRQ_ENABLE_MASK;
 }
 
 static int
@@ -5096,7 +5112,7 @@ static int ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
 	rte_intr_enable(intr_handle);
 
 	/* Re-enable interrupt for VF */
-	ixgbevf_intr_enable(hw);
+	ixgbevf_intr_enable(dev);
 
 	return 0;
 }
@@ -5110,7 +5126,7 @@ static int ixgbevf_dev_xstats_get_names(__rte_unused struct rte_eth_dev *dev,
 
 	PMD_INIT_FUNC_TRACE();
 
-	ixgbevf_intr_disable(hw);
+	ixgbevf_intr_disable(dev);
 
 	hw->adapter_stopped = 1;
 	ixgbe_stop_adapter(hw);
@@ -5602,17 +5618,17 @@ static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 {
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
-	uint32_t mask;
+	struct ixgbe_interrupt *intr =
+		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
 	struct ixgbe_hw *hw =
 		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	uint32_t vec = IXGBE_MISC_VEC_ID;
 
-	mask = IXGBE_READ_REG(hw, IXGBE_VTEIMS);
 	if (rte_intr_allow_others(intr_handle))
 		vec = IXGBE_RX_VEC_START;
-	mask |= (1 << vec);
+	intr->mask |= (1 << vec);
 	RTE_SET_USED(queue_id);
-	IXGBE_WRITE_REG(hw, IXGBE_VTEIMS, mask);
+	IXGBE_WRITE_REG(hw, IXGBE_VTEIMS, intr->mask);
 
 	rte_intr_enable(intr_handle);
 
@@ -5622,19 +5638,19 @@ static void ixgbevf_set_vfta_all(struct rte_eth_dev *dev, bool on)
 static int
 ixgbevf_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id)
 {
-	uint32_t mask;
+	struct ixgbe_interrupt *intr =
+		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
 	struct ixgbe_hw *hw =
 		IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct rte_pci_device *pci_dev = RTE_ETH_DEV_TO_PCI(dev);
 	struct rte_intr_handle *intr_handle = &pci_dev->intr_handle;
 	uint32_t vec = IXGBE_MISC_VEC_ID;
 
-	mask = IXGBE_READ_REG(hw, IXGBE_VTEIMS);
 	if (rte_intr_allow_others(intr_handle))
 		vec = IXGBE_RX_VEC_START;
-	mask &= ~(1 << vec);
+	intr->mask &= ~(1 << vec);
 	RTE_SET_USED(queue_id);
-	IXGBE_WRITE_REG(hw, IXGBE_VTEIMS, mask);
+	IXGBE_WRITE_REG(hw, IXGBE_VTEIMS, intr->mask);
 
 	return 0;
 }
@@ -8273,7 +8289,7 @@ static void ixgbevf_mbx_process(struct rte_eth_dev *dev)
 	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct ixgbe_interrupt *intr =
 		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
-	ixgbevf_intr_disable(hw);
+	ixgbevf_intr_disable(dev);
 
 	/* read-on-clear nic registers here */
 	eicr = IXGBE_READ_REG(hw, IXGBE_VTEICR);
@@ -8290,7 +8306,6 @@ static void ixgbevf_mbx_process(struct rte_eth_dev *dev)
 static int
 ixgbevf_dev_interrupt_action(struct rte_eth_dev *dev)
 {
-	struct ixgbe_hw *hw = IXGBE_DEV_PRIVATE_TO_HW(dev->data->dev_private);
 	struct ixgbe_interrupt *intr =
 		IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
 
@@ -8299,7 +8314,7 @@ static void ixgbevf_mbx_process(struct rte_eth_dev *dev)
 		intr->flags &= ~IXGBE_FLAG_MAILBOX;
 	}
 
-	ixgbevf_intr_enable(hw);
+	ixgbevf_intr_enable(dev);
 
 	return 0;
 }
-- 
1.8.3.1

Re: [dpdk-dev] [PATCH net-next v4 0/4] set ixgbe/ixgbevf EITR for performance

[Tonghao Zhang]

Hi Ferruh, Qi:
This version is ok ?

On Sat, Apr 28, 2018 at 11:03 AM,  <xiangxia.m.yue@gmail.com> wrote:
> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
>
> v4: for reducing config options, remove the config option and use the
>     default value defined in header file. Remove the patch:
>     http://dpdk.org/dev/patchwork/patch/36433
>
>     rebase the net-next.
>
> v3: configure eitr via configuration file and remove the patch:
>     http://dpdk.org/dev/patchwork/patch/33698
>
> Tonghao Zhang (4):
>   net/ixgbevf: set the interrupt interval for EITR
>   net/ixgbe: set the default value for EITR
>   net/ixgbe: write disable to EITR counter
>   net/ixgbevf: save IXGBE_VTEIMS to intr->mask for performance.
>
>  drivers/net/ixgbe/ixgbe_ethdev.c | 68 ++++++++++++++++++++++++++--------------
>  drivers/net/ixgbe/ixgbe_ethdev.h |  1 +
>  drivers/net/ixgbe/ixgbe_rxtx.c   |  3 +-
>  3 files changed, 47 insertions(+), 25 deletions(-)
>
> --
> 1.8.3.1
>

Re: [dpdk-dev] [PATCH net-next v4 0/4] set ixgbe/ixgbevf EITR for performance

[Zhang, Qi Z]

> -----Original Message-----
> From: xiangxia.m.yue@gmail.com [mailto:xiangxia.m.yue@gmail.com]
> Sent: Saturday, April 28, 2018 11:04 AM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>;
> Zhang, Helin <helin.zhang@intel.com>; Xing, Beilei <beilei.xing@intel.com>;
> Dai, Wei <wei.dai@intel.com>
> Cc: dev@dpdk.org; Tonghao Zhang <xiangxia.m.yue@gmail.com>
> Subject: [PATCH net-next v4 0/4] set ixgbe/ixgbevf EITR for performance
> 
> From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> 
> v4: for reducing config options, remove the config option and use the
>     default value defined in header file. Remove the patch:
>     http://dpdk.org/dev/patchwork/patch/36433
> 
>     rebase the net-next.
> 
> v3: configure eitr via configuration file and remove the patch:
>     http://dpdk.org/dev/patchwork/patch/33698
> 
> Tonghao Zhang (4):
>   net/ixgbevf: set the interrupt interval for EITR
>   net/ixgbe: set the default value for EITR
>   net/ixgbe: write disable to EITR counter
>   net/ixgbevf: save IXGBE_VTEIMS to intr->mask for performance.
> 
>  drivers/net/ixgbe/ixgbe_ethdev.c | 68
> ++++++++++++++++++++++++++--------------
>  drivers/net/ixgbe/ixgbe_ethdev.h |  1 +
>  drivers/net/ixgbe/ixgbe_rxtx.c   |  3 +-
>  3 files changed, 47 insertions(+), 25 deletions(-)
> 
> --
> 1.8.3.1

Acked-by: Qi Zhang <qi.z.zhang@intel.com>

Re: [dpdk-dev] [PATCH net-next v4 0/4] set ixgbe/ixgbevf EITR for performance

[Zhang, Helin]

> -----Original Message-----
> From: Zhang, Qi Z
> Sent: Wednesday, May 9, 2018 1:32 PM
> To: xiangxia.m.yue@gmail.com; Yigit, Ferruh; Zhang, Helin; Xing, Beilei; Dai,
> Wei
> Cc: dev@dpdk.org
> Subject: RE: [PATCH net-next v4 0/4] set ixgbe/ixgbevf EITR for performance
> 
> 
> 
> > -----Original Message-----
> > From: xiangxia.m.yue@gmail.com [mailto:xiangxia.m.yue@gmail.com]
> > Sent: Saturday, April 28, 2018 11:04 AM
> > To: Zhang, Qi Z <qi.z.zhang@intel.com>; Yigit, Ferruh
> > <ferruh.yigit@intel.com>; Zhang, Helin <helin.zhang@intel.com>; Xing,
> > Beilei <beilei.xing@intel.com>; Dai, Wei <wei.dai@intel.com>
> > Cc: dev@dpdk.org; Tonghao Zhang <xiangxia.m.yue@gmail.com>
> > Subject: [PATCH net-next v4 0/4] set ixgbe/ixgbevf EITR for
> > performance
> >
> > From: Tonghao Zhang <xiangxia.m.yue@gmail.com>
> >
> > v4: for reducing config options, remove the config option and use the
> >     default value defined in header file. Remove the patch:
> >     http://dpdk.org/dev/patchwork/patch/36433
> >
> >     rebase the net-next.
> >
> > v3: configure eitr via configuration file and remove the patch:
> >     http://dpdk.org/dev/patchwork/patch/33698
> >
> > Tonghao Zhang (4):
> >   net/ixgbevf: set the interrupt interval for EITR
> >   net/ixgbe: set the default value for EITR
> >   net/ixgbe: write disable to EITR counter
> >   net/ixgbevf: save IXGBE_VTEIMS to intr->mask for performance.
> >
> >  drivers/net/ixgbe/ixgbe_ethdev.c | 68
> > ++++++++++++++++++++++++++--------------
> >  drivers/net/ixgbe/ixgbe_ethdev.h |  1 +
> >  drivers/net/ixgbe/ixgbe_rxtx.c   |  3 +-
> >  3 files changed, 47 insertions(+), 25 deletions(-)
> >
> > --
> > 1.8.3.1
> 
> Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Applied to dpdk-next-net-intel, thanks!
BTW, the v3 version has been removed.

/Helin