[PATCH] net/virtio: Fix check of threshold for Tx freeing

[PATCH] net/virtio: Fix check of threshold for Tx freeing

[Hengqi Chen]

Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
    Start freeing Tx buffers if there are
    less free descriptors than this value.

The rationale behind this change is that:
  * vq->vq_nentries is set during device probe with the queue size specified
    by vhost backend, this value does not reflect the real nb_tx_desc
  * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
    param of rte_eth_tx_queue_setup() API
  * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
    if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
    see bug report 1716 ([1]) for details.

Bugzilla ID: 1716

  [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
  [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716

Signed-off-by: Baoyuan Li <updoing@sina.com>
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
---
 drivers/net/virtio/virtio_rxtx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
index edecd2011f..ab97f03d7d 100644
--- a/drivers/net/virtio/virtio_rxtx.c
+++ b/drivers/net/virtio/virtio_rxtx.c
@@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 	nb_used = virtqueue_nused(vq);
 
-	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
+	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
 		virtio_xmit_cleanup(vq, nb_used);
 
 	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
-- 
2.43.5

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Maxime Coquelin]

On 6/9/25 9:23 AM, Hengqi Chen wrote:
> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
>      Start freeing Tx buffers if there are
>      less free descriptors than this value.
> 
> The rationale behind this change is that:
>    * vq->vq_nentries is set during device probe with the queue size specified
>      by vhost backend, this value does not reflect the real nb_tx_desc
>    * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
>      param of rte_eth_tx_queue_setup() API
>    * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
>      if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
>      see bug report 1716 ([1]) for details.
> 
> Bugzilla ID: 1716
> 
>    [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
>    [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> 
> Signed-off-by: Baoyuan Li <updoing@sina.com>
> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> ---
>   drivers/net/virtio/virtio_rxtx.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> index edecd2011f..ab97f03d7d 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>   
>   	nb_used = virtqueue_nused(vq);
>   
> -	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> +	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
>   		virtio_xmit_cleanup(vq, nb_used);
>   
>   	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Maxime Coquelin]

On 6/9/25 9:23 AM, Hengqi Chen wrote:
> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
>      Start freeing Tx buffers if there are
>      less free descriptors than this value.
> 
> The rationale behind this change is that:
>    * vq->vq_nentries is set during device probe with the queue size specified
>      by vhost backend, this value does not reflect the real nb_tx_desc
>    * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
>      param of rte_eth_tx_queue_setup() API
>    * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
>      if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
>      see bug report 1716 ([1]) for details.
> 
> Bugzilla ID: 1716
> 
>    [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
>    [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> 
> Signed-off-by: Baoyuan Li <updoing@sina.com>
> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> ---
>   drivers/net/virtio/virtio_rxtx.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> index edecd2011f..ab97f03d7d 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>   
>   	nb_used = virtqueue_nused(vq);
>   
> -	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> +	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
>   		virtio_xmit_cleanup(vq, nb_used);
>   
>   	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {

Applied to next-virtio/for-net-next

Thanks,
Maxime

RE: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Jiang, YuX]

Hi Hengqi,

Can you help to check the ticket: https://bugs.dpdk.org/show_bug.cgi?id=1747? The perf drop is caused by your patch. Thanks so much.

Best regards,
Yu Jiang

> -----Original Message-----
> From: Hengqi Chen <hengqi.chen@gmail.com>
> Sent: Monday, June 9, 2025 3:24 PM
> To: dev@dpdk.org
> Cc: Hengqi Chen <hengqi.chen@gmail.com>; Baoyuan Li
> <updoing@sina.com>; Maxime Coquelin <maxime.coquelin@redhat.com>;
> Chenbo Xia <chenbox@nvidia.com>
> Subject: [PATCH] net/virtio: Fix check of threshold for Tx freeing
> 
> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
>     Start freeing Tx buffers if there are
>     less free descriptors than this value.
> 
> The rationale behind this change is that:
>   * vq->vq_nentries is set during device probe with the queue size specified
>     by vhost backend, this value does not reflect the real nb_tx_desc
>   * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
>     param of rte_eth_tx_queue_setup() API
>   * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
>     if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
>     see bug report 1716 ([1]) for details.
> 
> Bugzilla ID: 1716
> 
>   [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
>   [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> 
> Signed-off-by: Baoyuan Li <updoing@sina.com>
> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> ---
>  drivers/net/virtio/virtio_rxtx.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> index edecd2011f..ab97f03d7d 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
> **tx_pkts, uint16_t nb_pkts)
> 
>  	nb_used = virtqueue_nused(vq);
> 
> -	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> +	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
>  		virtio_xmit_cleanup(vq, nb_used);
> 
>  	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
> --
> 2.43.5

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Maxime Coquelin]

Hi Yu, Hengqi,

First, thanks Yu for testing and reporting the performance degradation,
this is much appreciated!

On 7/4/25 1:29 PM, Jiang, YuX wrote:
> Hi Hengqi,
> 
> Can you help to check the ticket: https://bugs.dpdk.org/show_bug.cgi?id=1747? The perf drop is caused by your patch. Thanks so much.

The options I see are:
1. Revert the patch for v25.07, fix it and resubmit for v25.11.
2. Fix it on time for v25.07-rc3
3. Keep the patch in v25.07 and try to fix it in v25.11.

As I will be OoO starting this evening, I would be in favor of option 1.
What do you think?

Thanks,
Maxime

> Best regards,
> Yu Jiang
> 
>> -----Original Message-----
>> From: Hengqi Chen <hengqi.chen@gmail.com>
>> Sent: Monday, June 9, 2025 3:24 PM
>> To: dev@dpdk.org
>> Cc: Hengqi Chen <hengqi.chen@gmail.com>; Baoyuan Li
>> <updoing@sina.com>; Maxime Coquelin <maxime.coquelin@redhat.com>;
>> Chenbo Xia <chenbox@nvidia.com>
>> Subject: [PATCH] net/virtio: Fix check of threshold for Tx freeing
>>
>> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
>> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
>>      Start freeing Tx buffers if there are
>>      less free descriptors than this value.
>>
>> The rationale behind this change is that:
>>    * vq->vq_nentries is set during device probe with the queue size specified
>>      by vhost backend, this value does not reflect the real nb_tx_desc
>>    * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
>>      param of rte_eth_tx_queue_setup() API
>>    * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
>>      if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
>>      see bug report 1716 ([1]) for details.
>>
>> Bugzilla ID: 1716
>>
>>    [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
>>    [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
>>
>> Signed-off-by: Baoyuan Li <updoing@sina.com>
>> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
>> ---
>>   drivers/net/virtio/virtio_rxtx.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
>> index edecd2011f..ab97f03d7d 100644
>> --- a/drivers/net/virtio/virtio_rxtx.c
>> +++ b/drivers/net/virtio/virtio_rxtx.c
>> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
>> **tx_pkts, uint16_t nb_pkts)
>>
>>   	nb_used = virtqueue_nused(vq);
>>
>> -	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
>> +	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
>>   		virtio_xmit_cleanup(vq, nb_used);
>>
>>   	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
>> --
>> 2.43.5
>