DPDK patches and discussions
 help / color / mirror / Atom feed
From: Xueming Li <xuemingl@nvidia.com>
Cc: ".Xueming Li" <xuemingl@nvidia.com>,
	dev@dpdk.org, "谢华伟 (此时此刻)" <huawei.xhw@alibaba-inc.com>,
	jerin.jacob@caviumnetworks.com, drc@linux.vnet.ibm.com,
	stable@dpdk.org, "Maxime Coquelin" <maxime.coquelin@redhat.com>,
	"Chenbo Xia" <chenbo.xia@intel.com>,
	"Jerin Jacob" <jerinj@marvell.com>,
	"Ruifeng Wang" <ruifeng.wang@arm.com>,
	"Bruce Richardson" <bruce.richardson@intel.com>,
	"Konstantin Ananyev" <konstantin.ananyev@intel.com>,
	"Jianfeng Tan" <jianfeng.tan@intel.com>,
	"Huawei Xie" <huawei.xie@intel.com>,
	"Jianbo Liu" <jianbo.liu@linaro.org>,
	"Yuanhan Liu" <yuanhan.liu@linux.intel.com>
Subject: [dpdk-dev] [PATCH v1] net/virtio: fix vectorized Rx queue stuck
Date: Wed, 14 Apr 2021 22:14:04 +0800	[thread overview]
Message-ID: <20210414141404.9486-1-xuemingl@nvidia.com> (raw)
In-Reply-To: <20210414042631.7041-1-xuemingl@nvidia.com>

From: ".Xueming Li" <xuemingl@nvidia.com>

When Rx queue worked in vectorized mode and rxd <= 512, under traffic of
high PPS rate, testpmd often start and receive packets of rxd without
further growth.

Testpmd started with rxq flush which tried to rx MAX_PKT_BURST(512)
packets and drop. When Rx burst size >= Rx queue size, all descriptors
in used queue consumed without rearm, device can't receive more packets.
The next Rx burst returned at once since no used descriptors found,
rearm logic was skipped, rx vq kept in starving state.

To avoid rx vq starving, this patch always check the available queue,
rearm if needed even no used descriptor reported by device.

Fixes: fc3d66212fed ("virtio: add vector Rx")
Cc: 谢华伟(此时此刻) <huawei.xhw@alibaba-inc.com>
Fixes: 2d7c37194ee4 ("net/virtio: add NEON based Rx handler")
Cc: jerin.jacob@caviumnetworks.com
Fixes: 52b5a707e6ca ("net/virtio: add Altivec Rx")
Cc: drc@linux.vnet.ibm.com
Cc: stable@dpdk.org

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
---
 drivers/net/virtio/virtio_rxtx_simple_altivec.c | 12 ++++++------
 drivers/net/virtio/virtio_rxtx_simple_neon.c    | 12 ++++++------
 drivers/net/virtio/virtio_rxtx_simple_sse.c     | 12 ++++++------
 3 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/drivers/net/virtio/virtio_rxtx_simple_altivec.c b/drivers/net/virtio/virtio_rxtx_simple_altivec.c
index 62e5100a48..7534974ef4 100644
--- a/drivers/net/virtio/virtio_rxtx_simple_altivec.c
+++ b/drivers/net/virtio/virtio_rxtx_simple_altivec.c
@@ -85,6 +85,12 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	if (unlikely(nb_pkts < RTE_VIRTIO_DESC_PER_LOOP))
 		return 0;
 
+	if (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
+		virtio_rxq_rearm_vec(rxvq);
+		if (unlikely(virtqueue_kick_prepare(vq)))
+			virtqueue_notify(vq);
+	}
+
 	nb_used = virtqueue_nused(vq);
 
 	rte_compiler_barrier();
@@ -102,12 +108,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 
 	rte_prefetch0(rused);
 
-	if (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
-		virtio_rxq_rearm_vec(rxvq);
-		if (unlikely(virtqueue_kick_prepare(vq)))
-			virtqueue_notify(vq);
-	}
-
 	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
diff --git a/drivers/net/virtio/virtio_rxtx_simple_neon.c b/drivers/net/virtio/virtio_rxtx_simple_neon.c
index c8e4b13a02..7fd92d1b0c 100644
--- a/drivers/net/virtio/virtio_rxtx_simple_neon.c
+++ b/drivers/net/virtio/virtio_rxtx_simple_neon.c
@@ -84,6 +84,12 @@ virtio_recv_pkts_vec(void *rx_queue,
 	if (unlikely(nb_pkts < RTE_VIRTIO_DESC_PER_LOOP))
 		return 0;
 
+	if (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
+		virtio_rxq_rearm_vec(rxvq);
+		if (unlikely(virtqueue_kick_prepare(vq)))
+			virtqueue_notify(vq);
+	}
+
 	/* virtqueue_nused has a load-acquire or rte_io_rmb inside */
 	nb_used = virtqueue_nused(vq);
 
@@ -100,12 +106,6 @@ virtio_recv_pkts_vec(void *rx_queue,
 
 	rte_prefetch_non_temporal(rused);
 
-	if (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
-		virtio_rxq_rearm_vec(rxvq);
-		if (unlikely(virtqueue_kick_prepare(vq)))
-			virtqueue_notify(vq);
-	}
-
 	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
diff --git a/drivers/net/virtio/virtio_rxtx_simple_sse.c b/drivers/net/virtio/virtio_rxtx_simple_sse.c
index ff4eba33d6..7577f5e86d 100644
--- a/drivers/net/virtio/virtio_rxtx_simple_sse.c
+++ b/drivers/net/virtio/virtio_rxtx_simple_sse.c
@@ -85,6 +85,12 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 	if (unlikely(nb_pkts < RTE_VIRTIO_DESC_PER_LOOP))
 		return 0;
 
+	if (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
+		virtio_rxq_rearm_vec(rxvq);
+		if (unlikely(virtqueue_kick_prepare(vq)))
+			virtqueue_notify(vq);
+	}
+
 	nb_used = virtqueue_nused(vq);
 
 	if (unlikely(nb_used == 0))
@@ -100,12 +106,6 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts,
 
 	rte_prefetch0(rused);
 
-	if (vq->vq_free_cnt >= RTE_VIRTIO_VPMD_RX_REARM_THRESH) {
-		virtio_rxq_rearm_vec(rxvq);
-		if (unlikely(virtqueue_kick_prepare(vq)))
-			virtqueue_notify(vq);
-	}
-
 	nb_total = nb_used;
 	ref_rx_pkts = rx_pkts;
 	for (nb_pkts_received = 0;
-- 
2.25.1


  parent reply	other threads:[~2021-04-14 14:14 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-04-14  4:26 [dpdk-dev] [PATCH] " Xueming Li
2021-04-14  6:11 ` Xueming(Steven) Li
2021-04-14 14:14 ` Xueming Li [this message]
2021-04-16 20:58   ` [dpdk-dev] [PATCH v1] " David Christensen
2021-05-03 14:53   ` Maxime Coquelin
2021-05-04  8:26   ` Maxime Coquelin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210414141404.9486-1-xuemingl@nvidia.com \
    --to=xuemingl@nvidia.com \
    --cc=bruce.richardson@intel.com \
    --cc=chenbo.xia@intel.com \
    --cc=dev@dpdk.org \
    --cc=drc@linux.vnet.ibm.com \
    --cc=huawei.xhw@alibaba-inc.com \
    --cc=huawei.xie@intel.com \
    --cc=jerin.jacob@caviumnetworks.com \
    --cc=jerinj@marvell.com \
    --cc=jianbo.liu@linaro.org \
    --cc=jianfeng.tan@intel.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=ruifeng.wang@arm.com \
    --cc=stable@dpdk.org \
    --cc=yuanhan.liu@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).