[PATCH] net/virtio: Fix check of threshold for Tx freeing

[PATCH] net/virtio: Fix check of threshold for Tx freeing

[Hengqi Chen]

Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
    Start freeing Tx buffers if there are
    less free descriptors than this value.

The rationale behind this change is that:
  * vq->vq_nentries is set during device probe with the queue size specified
    by vhost backend, this value does not reflect the real nb_tx_desc
  * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
    param of rte_eth_tx_queue_setup() API
  * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
    if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
    see bug report 1716 ([1]) for details.

Bugzilla ID: 1716

  [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
  [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716

Signed-off-by: Baoyuan Li <updoing@sina.com>
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
---
 drivers/net/virtio/virtio_rxtx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
index edecd2011f..ab97f03d7d 100644
--- a/drivers/net/virtio/virtio_rxtx.c
+++ b/drivers/net/virtio/virtio_rxtx.c
@@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
 
 	nb_used = virtqueue_nused(vq);
 
-	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
+	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
 		virtio_xmit_cleanup(vq, nb_used);
 
 	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
-- 
2.43.5

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Maxime Coquelin]

On 6/9/25 9:23 AM, Hengqi Chen wrote:
> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
>      Start freeing Tx buffers if there are
>      less free descriptors than this value.
> 
> The rationale behind this change is that:
>    * vq->vq_nentries is set during device probe with the queue size specified
>      by vhost backend, this value does not reflect the real nb_tx_desc
>    * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
>      param of rte_eth_tx_queue_setup() API
>    * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
>      if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
>      see bug report 1716 ([1]) for details.
> 
> Bugzilla ID: 1716
> 
>    [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
>    [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> 
> Signed-off-by: Baoyuan Li <updoing@sina.com>
> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> ---
>   drivers/net/virtio/virtio_rxtx.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> index edecd2011f..ab97f03d7d 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>   
>   	nb_used = virtqueue_nused(vq);
>   
> -	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> +	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
>   		virtio_xmit_cleanup(vq, nb_used);
>   
>   	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {

Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>

Thanks,
Maxime

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Maxime Coquelin]

On 6/9/25 9:23 AM, Hengqi Chen wrote:
> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
>      Start freeing Tx buffers if there are
>      less free descriptors than this value.
> 
> The rationale behind this change is that:
>    * vq->vq_nentries is set during device probe with the queue size specified
>      by vhost backend, this value does not reflect the real nb_tx_desc
>    * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
>      param of rte_eth_tx_queue_setup() API
>    * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
>      if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
>      see bug report 1716 ([1]) for details.
> 
> Bugzilla ID: 1716
> 
>    [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
>    [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> 
> Signed-off-by: Baoyuan Li <updoing@sina.com>
> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> ---
>   drivers/net/virtio/virtio_rxtx.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> index edecd2011f..ab97f03d7d 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>   
>   	nb_used = virtqueue_nused(vq);
>   
> -	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> +	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
>   		virtio_xmit_cleanup(vq, nb_used);
>   
>   	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {

Applied to next-virtio/for-net-next

Thanks,
Maxime

RE: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Jiang, YuX]

Hi Hengqi,

Can you help to check the ticket: https://bugs.dpdk.org/show_bug.cgi?id=1747? The perf drop is caused by your patch. Thanks so much.

Best regards,
Yu Jiang

> -----Original Message-----
> From: Hengqi Chen <hengqi.chen@gmail.com>
> Sent: Monday, June 9, 2025 3:24 PM
> To: dev@dpdk.org
> Cc: Hengqi Chen <hengqi.chen@gmail.com>; Baoyuan Li
> <updoing@sina.com>; Maxime Coquelin <maxime.coquelin@redhat.com>;
> Chenbo Xia <chenbox@nvidia.com>
> Subject: [PATCH] net/virtio: Fix check of threshold for Tx freeing
> 
> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
>     Start freeing Tx buffers if there are
>     less free descriptors than this value.
> 
> The rationale behind this change is that:
>   * vq->vq_nentries is set during device probe with the queue size specified
>     by vhost backend, this value does not reflect the real nb_tx_desc
>   * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
>     param of rte_eth_tx_queue_setup() API
>   * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
>     if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
>     see bug report 1716 ([1]) for details.
> 
> Bugzilla ID: 1716
> 
>   [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
>   [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> 
> Signed-off-by: Baoyuan Li <updoing@sina.com>
> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> ---
>  drivers/net/virtio/virtio_rxtx.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> index edecd2011f..ab97f03d7d 100644
> --- a/drivers/net/virtio/virtio_rxtx.c
> +++ b/drivers/net/virtio/virtio_rxtx.c
> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
> **tx_pkts, uint16_t nb_pkts)
> 
>  	nb_used = virtqueue_nused(vq);
> 
> -	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> +	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
>  		virtio_xmit_cleanup(vq, nb_used);
> 
>  	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
> --
> 2.43.5

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Maxime Coquelin]

Hi Yu, Hengqi,

First, thanks Yu for testing and reporting the performance degradation,
this is much appreciated!

On 7/4/25 1:29 PM, Jiang, YuX wrote:
> Hi Hengqi,
> 
> Can you help to check the ticket: https://bugs.dpdk.org/show_bug.cgi?id=1747? The perf drop is caused by your patch. Thanks so much.

The options I see are:
1. Revert the patch for v25.07, fix it and resubmit for v25.11.
2. Fix it on time for v25.07-rc3
3. Keep the patch in v25.07 and try to fix it in v25.11.

As I will be OoO starting this evening, I would be in favor of option 1.
What do you think?

Thanks,
Maxime

> Best regards,
> Yu Jiang
> 
>> -----Original Message-----
>> From: Hengqi Chen <hengqi.chen@gmail.com>
>> Sent: Monday, June 9, 2025 3:24 PM
>> To: dev@dpdk.org
>> Cc: Hengqi Chen <hengqi.chen@gmail.com>; Baoyuan Li
>> <updoing@sina.com>; Maxime Coquelin <maxime.coquelin@redhat.com>;
>> Chenbo Xia <chenbox@nvidia.com>
>> Subject: [PATCH] net/virtio: Fix check of threshold for Tx freeing
>>
>> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
>> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
>>      Start freeing Tx buffers if there are
>>      less free descriptors than this value.
>>
>> The rationale behind this change is that:
>>    * vq->vq_nentries is set during device probe with the queue size specified
>>      by vhost backend, this value does not reflect the real nb_tx_desc
>>    * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
>>      param of rte_eth_tx_queue_setup() API
>>    * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
>>      if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
>>      see bug report 1716 ([1]) for details.
>>
>> Bugzilla ID: 1716
>>
>>    [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
>>    [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
>>
>> Signed-off-by: Baoyuan Li <updoing@sina.com>
>> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
>> ---
>>   drivers/net/virtio/virtio_rxtx.c | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
>> index edecd2011f..ab97f03d7d 100644
>> --- a/drivers/net/virtio/virtio_rxtx.c
>> +++ b/drivers/net/virtio/virtio_rxtx.c
>> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
>> **tx_pkts, uint16_t nb_pkts)
>>
>>   	nb_used = virtqueue_nused(vq);
>>
>> -	if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
>> +	if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
>>   		virtio_xmit_cleanup(vq, nb_used);
>>
>>   	for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
>> --
>> 2.43.5
> 

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Hengqi Chen]

Hi Maxime and other maintainers,

On Fri, Jul 4, 2025 at 7:57 PM Maxime Coquelin
<maxime.coquelin@redhat.com> wrote:
>
> Hi Yu, Hengqi,
>
> First, thanks Yu for testing and reporting the performance degradation,
> this is much appreciated!
>
> On 7/4/25 1:29 PM, Jiang, YuX wrote:
> > Hi Hengqi,
> >
> > Can you help to check the ticket: https://bugs.dpdk.org/show_bug.cgi?id=1747? The perf drop is caused by your patch. Thanks so much.
>
> The options I see are:
> 1. Revert the patch for v25.07, fix it and resubmit for v25.11.
> 2. Fix it on time for v25.07-rc3
> 3. Keep the patch in v25.07 and try to fix it in v25.11.
>
> As I will be OoO starting this evening, I would be in favor of option 1.
> What do you think?
>

I am OK with both option 1 and 3.
Please handle it according to the DPDK community rules.

Thanks,
Hengqi

> Thanks,
> Maxime
>
> > Best regards,
> > Yu Jiang
> >
> >> -----Original Message-----
> >> From: Hengqi Chen <hengqi.chen@gmail.com>
> >> Sent: Monday, June 9, 2025 3:24 PM
> >> To: dev@dpdk.org
> >> Cc: Hengqi Chen <hengqi.chen@gmail.com>; Baoyuan Li
> >> <updoing@sina.com>; Maxime Coquelin <maxime.coquelin@redhat.com>;
> >> Chenbo Xia <chenbox@nvidia.com>
> >> Subject: [PATCH] net/virtio: Fix check of threshold for Tx freeing
> >>
> >> Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> >> described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
> >>      Start freeing Tx buffers if there are
> >>      less free descriptors than this value.
> >>
> >> The rationale behind this change is that:
> >>    * vq->vq_nentries is set during device probe with the queue size specified
> >>      by vhost backend, this value does not reflect the real nb_tx_desc
> >>    * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
> >>      param of rte_eth_tx_queue_setup() API
> >>    * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
> >>      if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
> >>      see bug report 1716 ([1]) for details.
> >>
> >> Bugzilla ID: 1716
> >>
> >>    [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
> >>    [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> >>
> >> Signed-off-by: Baoyuan Li <updoing@sina.com>
> >> Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> >> ---
> >>   drivers/net/virtio/virtio_rxtx.c | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> >> index edecd2011f..ab97f03d7d 100644
> >> --- a/drivers/net/virtio/virtio_rxtx.c
> >> +++ b/drivers/net/virtio/virtio_rxtx.c
> >> @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
> >> **tx_pkts, uint16_t nb_pkts)
> >>
> >>      nb_used = virtqueue_nused(vq);
> >>
> >> -    if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> >> +    if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
> >>              virtio_xmit_cleanup(vq, nb_used);
> >>
> >>      for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
> >> --
> >> 2.43.5
> >
>

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Hengqi Chen]

Hi Yu,

On Fri, Jul 4, 2025 at 7:29 PM Jiang, YuX <yux.jiang@intel.com> wrote:
>
> Hi Hengqi,
>
> Can you help to check the ticket: https://bugs.dpdk.org/show_bug.cgi?id=1747? The perf drop is caused by your patch. Thanks so much.
>

Thanks for the testing. Will investigate.

> Best regards,
> Yu Jiang
>
> > -----Original Message-----
> > From: Hengqi Chen <hengqi.chen@gmail.com>
> > Sent: Monday, June 9, 2025 3:24 PM
> > To: dev@dpdk.org
> > Cc: Hengqi Chen <hengqi.chen@gmail.com>; Baoyuan Li
> > <updoing@sina.com>; Maxime Coquelin <maxime.coquelin@redhat.com>;
> > Chenbo Xia <chenbox@nvidia.com>
> > Subject: [PATCH] net/virtio: Fix check of threshold for Tx freeing
> >
> > Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> > described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
> >     Start freeing Tx buffers if there are
> >     less free descriptors than this value.
> >
> > The rationale behind this change is that:
> >   * vq->vq_nentries is set during device probe with the queue size specified
> >     by vhost backend, this value does not reflect the real nb_tx_desc
> >   * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
> >     param of rte_eth_tx_queue_setup() API
> >   * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
> >     if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
> >     see bug report 1716 ([1]) for details.
> >
> > Bugzilla ID: 1716
> >
> >   [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
> >   [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> >
> > Signed-off-by: Baoyuan Li <updoing@sina.com>
> > Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> > ---
> >  drivers/net/virtio/virtio_rxtx.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> > index edecd2011f..ab97f03d7d 100644
> > --- a/drivers/net/virtio/virtio_rxtx.c
> > +++ b/drivers/net/virtio/virtio_rxtx.c
> > @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
> > **tx_pkts, uint16_t nb_pkts)
> >
> >       nb_used = virtqueue_nused(vq);
> >
> > -     if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> > +     if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
> >               virtio_xmit_cleanup(vq, nb_used);
> >
> >       for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
> > --
> > 2.43.5
>

Re: [PATCH] net/virtio: Fix check of threshold for Tx freeing

[Hengqi Chen]

Hi Yu,

On Fri, Jul 4, 2025 at 7:29 PM Jiang, YuX <yux.jiang@intel.com> wrote:
>
> Hi Hengqi,
>
> Can you help to check the ticket: https://bugs.dpdk.org/show_bug.cgi?id=1747? The perf drop is caused by your patch. Thanks so much.
>

The `[Expect Result]` section contains a non-zero `TX-errors` counter,
is this expected ?
BTW, the expected result is done by dropping/reverting the bad commit
ONLY, right ?

> Best regards,
> Yu Jiang
>
> > -----Original Message-----
> > From: Hengqi Chen <hengqi.chen@gmail.com>
> > Sent: Monday, June 9, 2025 3:24 PM
> > To: dev@dpdk.org
> > Cc: Hengqi Chen <hengqi.chen@gmail.com>; Baoyuan Li
> > <updoing@sina.com>; Maxime Coquelin <maxime.coquelin@redhat.com>;
> > Chenbo Xia <chenbox@nvidia.com>
> > Subject: [PATCH] net/virtio: Fix check of threshold for Tx freeing
> >
> > Like most dirvers, make the fast path of virtio_xmit_cleanup() behave as
> > described by the comments of rte_eth_txconf::tx_free_thresh ([0]):
> >     Start freeing Tx buffers if there are
> >     less free descriptors than this value.
> >
> > The rationale behind this change is that:
> >   * vq->vq_nentries is set during device probe with the queue size specified
> >     by vhost backend, this value does not reflect the real nb_tx_desc
> >   * the real available tx desc is set to vq->vq_free_cnt via the nb_tx_desc
> >     param of rte_eth_tx_queue_setup() API
> >   * so `nb_used > vq->vq_nentries - vq->vq_free_thresh` could never be true
> >     if say nb_tx_desc=2048, vq->vq_nentries=4096, vq->vq_free_thresh=32,
> >     see bug report 1716 ([1]) for details.
> >
> > Bugzilla ID: 1716
> >
> >   [0]: https://github.com/DPDK/dpdk/commit/72514b5d5543
> >   [1]: https://bugs.dpdk.org/show_bug.cgi?id=1716
> >
> > Signed-off-by: Baoyuan Li <updoing@sina.com>
> > Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
> > ---
> >  drivers/net/virtio/virtio_rxtx.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
> > index edecd2011f..ab97f03d7d 100644
> > --- a/drivers/net/virtio/virtio_rxtx.c
> > +++ b/drivers/net/virtio/virtio_rxtx.c
> > @@ -1873,7 +1873,7 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf
> > **tx_pkts, uint16_t nb_pkts)
> >
> >       nb_used = virtqueue_nused(vq);
> >
> > -     if (likely(nb_used > vq->vq_nentries - vq->vq_free_thresh))
> > +     if (likely(vq->vq_free_cnt < vq->vq_free_thresh))
> >               virtio_xmit_cleanup(vq, nb_used);
> >
> >       for (nb_tx = 0; nb_tx < nb_pkts; nb_tx++) {
> > --
> > 2.43.5
>