This patch series is targeted at the v20.11 release. It fixes two separate bugs, on the PF with reducing the latency from 16us to 2us maximum for a single packet, and on the VF with adding an ITR timeout to both allow descriptor aggregation for writeback, and minimize latency, setting the value to the same as the PF for E810 family. v2: subscribe to list and re-send Jesse Brandeburg (2): iavf: Fix performance with writeback policy ice: update writeback policy to reduce latency drivers/net/iavf/iavf_ethdev.c | 13 +++++++++++-- drivers/net/ice/ice_ethdev.c | 2 +- 2 files changed, 12 insertions(+), 3 deletions(-) -- 2.25.4
The iavf driver was trying to use writeback on ITR, but was never setting an ITR, so it didn't work. This caused performance to be limited due to too much PCIe traffic and partial writes during most benchmarking workloads. Set the ITR during queue setup, which can be checked at runtime by reading register 0x2800. Setting the value to 2us allows for generally good streaming packet performance while keeping latency down. Fixes: d6bde6b5eae9 ("net/avf: enable Rx interrupt") Reported-by: Brian Johnson <brian.johnson@intel.com> Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com> --- drivers/net/iavf/iavf_ethdev.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c index 0ef023c0aee4..44372294066f 100644 --- a/drivers/net/iavf/iavf_ethdev.c +++ b/drivers/net/iavf/iavf_ethdev.c @@ -383,10 +383,19 @@ static int iavf_config_rx_queues_irqs(struct rte_eth_dev *dev, VIRTCHNL_VF_OFFLOAD_WB_ON_ITR) { /* If WB_ON_ITR supports, enable it */ vf->msix_base = IAVF_RX_VEC_START; + /* Set the ITR for index zero, to 2us to make sure that + * we leave time for aggregation to occur, but don't + * increase latency dramatically. + */ IAVF_WRITE_REG(hw, IAVF_VFINT_DYN_CTLN1(vf->msix_base - 1), - IAVF_VFINT_DYN_CTLN1_ITR_INDX_MASK | - IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK); + (0 << IAVF_VFINT_DYN_CTLN1_ITR_INDX_SHIFT) | + IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK | + (2UL << IAVF_VFINT_DYN_CTLN1_INTERVAL_SHIFT)); + /* debug - check for success! the return value + * should be 2, offset is 0x2800 + */ + /* IAVF_READ_REG(hw, IAVF_VFINT_ITRN1(0, 0)); */ } else { /* If no WB_ON_ITR offload flags, need to set * interrupt for descriptor write back. -- 2.25.4
Just like iavf, setting the value to 2us allows for generally good streaming packet performance while keeping latency down, and generally keeps the performance of the PF and VF interfaces similar. The previous value of 0x10 was making latency on a single packet receive be as much as 16us. Fixes: 65dfc889d86b ("net/ice: support Rx queue interruption") Reported-by: Brian Johnson <brian.johnson@intel.com> Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com> --- drivers/net/ice/ice_ethdev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ice/ice_ethdev.c b/drivers/net/ice/ice_ethdev.c index c65125ff3260..0df491f93126 100644 --- a/drivers/net/ice/ice_ethdev.c +++ b/drivers/net/ice/ice_ethdev.c @@ -3345,7 +3345,7 @@ __vsi_queues_bind_intr(struct ice_vsi *vsi, uint16_t msix_vect, PMD_DRV_LOG(INFO, "queue %d is binding to vect %d", base_queue + i, msix_vect); /* set ITR0 value */ - ICE_WRITE_REG(hw, GLINT_ITR(0, msix_vect), 0x10); + ICE_WRITE_REG(hw, GLINT_ITR(0, msix_vect), 0x2); ICE_WRITE_REG(hw, QINT_RQCTL(base_queue + i), val); ICE_WRITE_REG(hw, QINT_TQCTL(base_queue + i), val_tx); } -- 2.25.4
> -----Original Message-----
> From: Jesse Brandeburg <jesse.brandeburg@intel.com>
> Sent: Saturday, October 24, 2020 4:22 AM
> To: Johnson, Brian <brian.johnson@intel.com>; thomas@monjalon.net;
> david.marchand@redhat.com
> Cc: Brandeburg, Jesse <jesse.brandeburg@intel.com>; dev@dpdk.org; Zhang,
> Qi Z <qi.z.zhang@intel.com>
> Subject: [PATCH v2 0/2] intel: fix writeback
>
> This patch series is targeted at the v20.11 release.
>
> It fixes two separate bugs, on the PF with reducing the latency from 16us to
> 2us maximum for a single packet, and on the VF with adding an ITR timeout to
> both allow descriptor aggregation for writeback, and minimize latency, setting
> the value to the same as the PF for E810 family.
>
> v2: subscribe to list and re-send
>
> Jesse Brandeburg (2):
> iavf: Fix performance with writeback policy
> ice: update writeback policy to reduce latency
>
> drivers/net/iavf/iavf_ethdev.c | 13 +++++++++++--
> drivers/net/ice/ice_ethdev.c | 2 +-
> 2 files changed, 12 insertions(+), 3 deletions(-)
>
> --
> 2.25.4
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Applied to dpdk-next-net-intel.
Thanks
Qi