[PATCH 0/2] Add logic to IAVF to count continuous DD bits for Arm

[PATCH 0/2] Add logic to IAVF to count continuous DD bits for Arm

[Kathleen Capella]

This patchset introduces a fix for Arm platforms to the IAVF driver that was 
added to the i40e driver in a previous patchset [1]. 

The driver determines which descriptors in the HW ring reference packets
that are ready to be received by counting those descriptors whose DD bit is set
to 1. On Arm, the reading of descriptors can be reordered. The CPU may be
reading descriptors as the NIC is updating them. Tt is possbile that the DD bit 
for a descriptor earlier in the queue is read as not set while the DD bit for a
descriptor later in the queue is read as set. This patchset ensures only
contiguous DD bits set to 1 are counted.

The first patch in this series adds this logic to the bulk Rx path.
The second patch adds this same logic to the function which reads flexible Rx
descriptors.

No performance drop was observed when running l3fwd on N1SDP with a single core.

[1]
https://patches.dpdk.org/project/dpdk/patch/20210706065404.25137-2-joyce.kong@arm.com/

Kathleen Capella (2):
  net/iavf: count continuous DD bits for Arm
  net/iavf: count continuous DD bits for Arm in flex Rx

 drivers/net/iavf/iavf_rxtx.c | 52 ++++++++++++++++++++++++++++++------
 1 file changed, 44 insertions(+), 8 deletions(-)

-- 
2.17.1

[PATCH 1/2] net/iavf: count continuous DD bits for Arm

[Kathleen Capella]

On Arm platforms, reading of descriptors may be re-ordered causing the
status of DD bits to be discontinuous. Add logic to only process
continuous descriptors by checking DD bits.

Fixes: 1060591eada5 ("net/avf: enable bulk allocate Rx")
Cc: stable@dpdk.org

Signed-off-by: Kathleen Capella <kathleen.capella@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
---
 drivers/net/iavf/iavf_rxtx.c | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c
index 59623ac820..4fc1bf5e78 100644
--- a/drivers/net/iavf/iavf_rxtx.c
+++ b/drivers/net/iavf/iavf_rxtx.c
@@ -1898,7 +1898,7 @@ iavf_rx_scan_hw_ring(struct iavf_rx_queue *rxq)
 	uint16_t pkt_len;
 	uint64_t qword1;
 	uint32_t rx_status;
-	int32_t s[IAVF_LOOK_AHEAD], nb_dd;
+	int32_t s[IAVF_LOOK_AHEAD], var, nb_dd;
 	int32_t i, j, nb_rx = 0;
 	uint64_t pkt_flags;
 	const uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
@@ -1929,9 +1929,27 @@ iavf_rx_scan_hw_ring(struct iavf_rx_queue *rxq)
 
 		rte_smp_rmb();
 
-		/* Compute how many status bits were set */
-		for (j = 0, nb_dd = 0; j < IAVF_LOOK_AHEAD; j++)
-			nb_dd += s[j] & (1 << IAVF_RX_DESC_STATUS_DD_SHIFT);
+		/* Compute how many contiguous DD bits were set */
+		for (j = 0, nb_dd = 0; j < IAVF_LOOK_AHEAD; j++) {
+			var = s[j] & (1 << IAVF_RX_DESC_STATUS_DD_SHIFT);
+#ifdef RTE_ARCH_ARM
+			/* For Arm platforms, count only contiguous descriptors
+			 * whose DD bit is set to 1. On Arm platforms, reads of
+			 * descriptors can be reordered. Since the CPU may
+			 * be reading the descriptors as the NIC updates them
+			 * in memory, it is possbile that the DD bit for a
+			 * descriptor earlier in the queue is read as not set
+			 * while the DD bit for a descriptor later in the queue
+			 * is read as set.
+			 */
+			if (var)
+				nb_dd += 1;
+			else
+				break;
+#else
+			nb_dd += var;
+#endif
+		}
 
 		nb_rx += nb_dd;
 
-- 
2.17.1

[PATCH 2/2] net/iavf: count continuous DD bits for Arm in flex Rx

[Kathleen Capella]

On Arm platforms, reading of descriptors may be re-ordered causing the
status of DD bits to be discontinuous. Add logic to only process
continuous descriptors by checking DD bits.

Fixes: b8b4c54ef9b0 ("net/iavf: support flexible Rx descriptor in normal path")
Cc: stable@dpdk.org

Signed-off-by: Kathleen Capella <kathleen.capella@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
---
 drivers/net/iavf/iavf_rxtx.c | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/drivers/net/iavf/iavf_rxtx.c b/drivers/net/iavf/iavf_rxtx.c
index 4fc1bf5e78..ba272bb211 100644
--- a/drivers/net/iavf/iavf_rxtx.c
+++ b/drivers/net/iavf/iavf_rxtx.c
@@ -1819,7 +1819,7 @@ iavf_rx_scan_hw_ring_flex_rxd(struct iavf_rx_queue *rxq)
 	struct rte_mbuf *mb;
 	uint16_t stat_err0;
 	uint16_t pkt_len;
-	int32_t s[IAVF_LOOK_AHEAD], nb_dd;
+	int32_t s[IAVF_LOOK_AHEAD], var, nb_dd;
 	int32_t i, j, nb_rx = 0;
 	uint64_t pkt_flags;
 	const uint32_t *ptype_tbl = rxq->vsi->adapter->ptype_tbl;
@@ -1844,9 +1844,27 @@ iavf_rx_scan_hw_ring_flex_rxd(struct iavf_rx_queue *rxq)
 
 		rte_smp_rmb();
 
-		/* Compute how many status bits were set */
-		for (j = 0, nb_dd = 0; j < IAVF_LOOK_AHEAD; j++)
-			nb_dd += s[j] & (1 << IAVF_RX_FLEX_DESC_STATUS0_DD_S);
+		/* Compute how many contiguous DD bits were set */
+		for (j = 0, nb_dd = 0; j < IAVF_LOOK_AHEAD; j++) {
+			var = s[j] & (1 << IAVF_RX_FLEX_DESC_STATUS0_DD_S);
+#ifdef RTE_ARCH_ARM
+			/* For Arm platforms, count only contiguous descriptors
+			 * whose DD bit is set to 1. On Arm platforms, reads of
+			 * descriptors can be reordered. Since the CPU may
+			 * be reading the descriptors as the NIC updates them
+			 * in memory, it is possbile that the DD bit for a
+			 * descriptor earlier in the queue is read as not set
+			 * while the DD bit for a descriptor later in the queue
+			 * is read as set.
+			 */
+			if (var)
+				nb_dd += 1;
+			else
+				break;
+#else
+			nb_dd += var;
+#endif
+		}
 
 		nb_rx += nb_dd;
 
-- 
2.17.1

RE: [PATCH 0/2] Add logic to IAVF to count continuous DD bits for Arm

[Kathleen Capella]

The failure in ci/iol-x86_64-unit-testing on this patch seems to be unrelated to the patch.

> -----Original Message-----
> From: Kathleen Capella <kathleen.capella@arm.com>
> Sent: Friday, February 4, 2022 6:26 PM
> Cc: dev@dpdk.org; nd <nd@arm.com>; Dharmik Thakkar
> <Dharmik.Thakkar@arm.com>; Honnappa Nagarahalli
> <Honnappa.Nagarahalli@arm.com>; Kathleen Capella
> <Kathleen.Capella@arm.com>
> Subject: [PATCH 0/2] Add logic to IAVF to count continuous DD bits for Arm
> 
> This patchset introduces a fix for Arm platforms to the IAVF driver that was
> added to the i40e driver in a previous patchset [1].
> 
> The driver determines which descriptors in the HW ring reference packets
> that are ready to be received by counting those descriptors whose DD bit is
> set to 1. On Arm, the reading of descriptors can be reordered. The CPU may
> be reading descriptors as the NIC is updating them. Tt is possbile that the DD
> bit for a descriptor earlier in the queue is read as not set while the DD bit for
> a descriptor later in the queue is read as set. This patchset ensures only
> contiguous DD bits set to 1 are counted.
> 
> The first patch in this series adds this logic to the bulk Rx path.
> The second patch adds this same logic to the function which reads flexible Rx
> descriptors.
> 
> No performance drop was observed when running l3fwd on N1SDP with a
> single core.
> 
> [1]
> https://patches.dpdk.org/project/dpdk/patch/20210706065404.25137-2-
> joyce.kong@arm.com/
> 
> Kathleen Capella (2):
>   net/iavf: count continuous DD bits for Arm
>   net/iavf: count continuous DD bits for Arm in flex Rx
> 
>  drivers/net/iavf/iavf_rxtx.c | 52 ++++++++++++++++++++++++++++++------
>  1 file changed, 44 insertions(+), 8 deletions(-)
> 
> --
> 2.17.1

RE: [PATCH 0/2] Add logic to IAVF to count continuous DD bits for Arm

[Zhang, Qi Z]

> -----Original Message-----
> From: Kathleen Capella <kathleen.capella@arm.com>
> Sent: Saturday, February 5, 2022 8:26 AM
> Cc: dev@dpdk.org; nd@arm.com; dharmik.thakkar@arm.com;
> honnappa.nagarahalli@arm.com; Kathleen Capella
> <kathleen.capella@arm.com>
> Subject: [PATCH 0/2] Add logic to IAVF to count continuous DD bits for Arm
> 
> This patchset introduces a fix for Arm platforms to the IAVF driver that was
> added to the i40e driver in a previous patchset [1].
> 
> The driver determines which descriptors in the HW ring reference packets
> that are ready to be received by counting those descriptors whose DD bit is
> set to 1. On Arm, the reading of descriptors can be reordered. The CPU may
> be reading descriptors as the NIC is updating them. Tt is possbile that the DD
> bit for a descriptor earlier in the queue is read as not set while the DD bit for
> a descriptor later in the queue is read as set. This patchset ensures only
> contiguous DD bits set to 1 are counted.
> 
> The first patch in this series adds this logic to the bulk Rx path.
> The second patch adds this same logic to the function which reads flexible Rx
> descriptors.
> 
> No performance drop was observed when running l3fwd on N1SDP with a
> single core.
> 
> [1]
> https://patches.dpdk.org/project/dpdk/patch/20210706065404.25137-2-
> joyce.kong@arm.com/
> 
> Kathleen Capella (2):
>   net/iavf: count continuous DD bits for Arm
>   net/iavf: count continuous DD bits for Arm in flex Rx
> 
>  drivers/net/iavf/iavf_rxtx.c | 52 ++++++++++++++++++++++++++++++------
>  1 file changed, 44 insertions(+), 8 deletions(-)
> 
> --
> 2.17.1

Reviewed-by: Qi Zhang <qi.z.zhang@intel.com>

Applied to dpdk-next-net-intel.

Thanks
Qi