[dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Qi Zhang]

vPMD tx does not set sw_ring's mbuf to NULL after free it.
So to prevent same mbuf be free again, we need more carefully
check in i40e_tx_queue_release_mbufs.

Fixes: b4669bb95038 ("i40e: add vector Tx")
Cc: stable@dpdk.org

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
---
v2:
- fix at i40e_tx_queue_release_mbufs so no performance impact.

 drivers/net/i40e/i40e_rxtx.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 3a8d32c..f21c1c5 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -2305,18 +2305,40 @@ i40e_reset_rx_queue(struct i40e_rx_queue *rxq)
 void
 i40e_tx_queue_release_mbufs(struct i40e_tx_queue *txq)
 {
+	struct rte_eth_dev *dev;
 	uint16_t i;
 
+	dev = &rte_eth_devices[txq->port_id];
+
 	if (!txq || !txq->sw_ring) {
 		PMD_DRV_LOG(DEBUG, "Pointer to rxq or sw_ring is NULL");
 		return;
 	}
 
-	for (i = 0; i < txq->nb_tx_desc; i++) {
-		if (txq->sw_ring[i].mbuf) {
+	/**
+	 *  vPMD tx will not set sw_ring's mbuf to NULL after free,
+	 *  so need to free remains more carefully.
+	 */
+	if (dev->tx_pkt_burst == i40e_xmit_pkts_vec) {
+		i = txq->tx_next_dd - txq->tx_rs_thresh + 1;
+		if (txq->tx_tail < i) {
+			for (; i < txq->nb_tx_desc; i++) {
+				rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+				txq->sw_ring[i].mbuf = NULL;
+			}
+			i = 0;
+		}
+		for (; i < txq->tx_tail; i++) {
 			rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
 			txq->sw_ring[i].mbuf = NULL;
 		}
+	} else {
+		for (i = 0; i < txq->nb_tx_desc; i++) {
+			if (txq->sw_ring[i].mbuf) {
+				rte_pktmbuf_free_seg(txq->sw_ring[i].mbuf);
+				txq->sw_ring[i].mbuf = NULL;
+			}
+		}
 	}
 }
 
-- 
2.9.5

Re: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Bruce Richardson]

+Roger Melton

On Tue, Oct 10, 2017 at 09:22:05AM -0400, Qi Zhang wrote:
> vPMD tx does not set sw_ring's mbuf to NULL after free it.
> So to prevent same mbuf be free again, we need more carefully
> check in i40e_tx_queue_release_mbufs.
> 
> Fixes: b4669bb95038 ("i40e: add vector Tx")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> ---
> v2:
> - fix at i40e_tx_queue_release_mbufs so no performance impact.
> 
This patch also supercedes this one, http://dpdk.org/dev/patchwork/patch/29814/
right?
Roger, can you please confirm that this alternative fix works in your
testing and that your patch is no longer necessary too.

Thanks,
/Bruce

Re: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Roger B. Melton]

Hi Bruce,

I can.  It will take a day or 2 to get the results.

Regards,
Roger


On 10/10/17 4:48 AM, Bruce Richardson wrote:
> +Roger Melton
>
> On Tue, Oct 10, 2017 at 09:22:05AM -0400, Qi Zhang wrote:
>> vPMD tx does not set sw_ring's mbuf to NULL after free it.
>> So to prevent same mbuf be free again, we need more carefully
>> check in i40e_tx_queue_release_mbufs.
>>
>> Fixes: b4669bb95038 ("i40e: add vector Tx")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
>> ---
>> v2:
>> - fix at i40e_tx_queue_release_mbufs so no performance impact.
>>
> This patch also supercedes this one, http://dpdk.org/dev/patchwork/patch/29814/
> right?
> Roger, can you please confirm that this alternative fix works in your
> testing and that your patch is no longer necessary too.
>
> Thanks,
> /Bruce
>
> .
>

Re: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Bruce Richardson]

On Tue, Oct 10, 2017 at 07:05:33AM -0400, Roger B. Melton wrote:
> Hi Bruce,
> 
> I can.  It will take a day or 2 to get the results.
> 
> Regards,
> Roger
> 

Thanks. Keep us posted. We want to ensure we have the best fix possible
for the issue in this release.

/Bruce

> 
> On 10/10/17 4:48 AM, Bruce Richardson wrote:
> > +Roger Melton
> > 
> > On Tue, Oct 10, 2017 at 09:22:05AM -0400, Qi Zhang wrote:
> > > vPMD tx does not set sw_ring's mbuf to NULL after free it.
> > > So to prevent same mbuf be free again, we need more carefully
> > > check in i40e_tx_queue_release_mbufs.
> > > 
> > > Fixes: b4669bb95038 ("i40e: add vector Tx")
> > > Cc: stable@dpdk.org
> > > 
> > > Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> > > ---
> > > v2:
> > > - fix at i40e_tx_queue_release_mbufs so no performance impact.
> > > 
> > This patch also supercedes this one, http://dpdk.org/dev/patchwork/patch/29814/
> > right?
> > Roger, can you please confirm that this alternative fix works in your
> > testing and that your patch is no longer necessary too.
> > 
> > Thanks,
> > /Bruce
> > 
> > .
> > 
> 

Re: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Roger B. Melton]

On 10/10/17 7:39 AM, Bruce Richardson wrote:
> On Tue, Oct 10, 2017 at 07:05:33AM -0400, Roger B. Melton wrote:
>> Hi Bruce,
>>
>> I can.  It will take a day or 2 to get the results.
>>
>> Regards,
>> Roger
>>
> Thanks. Keep us posted. We want to ensure we have the best fix possible
> for the issue in this release.

Thanks for finding a better solution ;-).  I'll let you know the results 
when I have them, just wanted you to know it may take a couple of days 
to turn around.

-Roger

>
> /Bruce
>
>> On 10/10/17 4:48 AM, Bruce Richardson wrote:
>>> +Roger Melton
>>>
>>> On Tue, Oct 10, 2017 at 09:22:05AM -0400, Qi Zhang wrote:
>>>> vPMD tx does not set sw_ring's mbuf to NULL after free it.
>>>> So to prevent same mbuf be free again, we need more carefully
>>>> check in i40e_tx_queue_release_mbufs.
>>>>
>>>> Fixes: b4669bb95038 ("i40e: add vector Tx")
>>>> Cc: stable@dpdk.org
>>>>
>>>> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
>>>> ---
>>>> v2:
>>>> - fix at i40e_tx_queue_release_mbufs so no performance impact.
>>>>
>>> This patch also supercedes this one, http://dpdk.org/dev/patchwork/patch/29814/
>>> right?
>>> Roger, can you please confirm that this alternative fix works in your
>>> testing and that your patch is no longer necessary too.
>>>
>>> Thanks,
>>> /Bruce
>>>
>>> .
>>>
> .
>

Re: [dpdk-dev] [dpdk-stable] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Roger B. Melton]

On 10/10/17 7:47 AM, Roger B. Melton wrote:
> On 10/10/17 7:39 AM, Bruce Richardson wrote:
>> On Tue, Oct 10, 2017 at 07:05:33AM -0400, Roger B. Melton wrote:
>>> Hi Bruce,
>>>
>>> I can.  It will take a day or 2 to get the results.
>>>
>>> Regards,
>>> Roger
>>>
>> Thanks. Keep us posted. We want to ensure we have the best fix possible
>> for the issue in this release.
>
> Thanks for finding a better solution ;-).  I'll let you know the 
> results when I have them, just wanted you to know it may take a couple 
> of days to turn around.
>
> -Roger

Confirmed that this patch does address the issue.  Thanks again Bruce/Qi 
for finding a better alternative.

Regards, Roger
>
>>
>> /Bruce
>>
>>> On 10/10/17 4:48 AM, Bruce Richardson wrote:
>>>> +Roger Melton
>>>>
>>>> On Tue, Oct 10, 2017 at 09:22:05AM -0400, Qi Zhang wrote:
>>>>> vPMD tx does not set sw_ring's mbuf to NULL after free it.
>>>>> So to prevent same mbuf be free again, we need more carefully
>>>>> check in i40e_tx_queue_release_mbufs.
>>>>>
>>>>> Fixes: b4669bb95038 ("i40e: add vector Tx")
>>>>> Cc: stable@dpdk.org
>>>>>
>>>>> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
>>>>> ---
>>>>> v2:
>>>>> - fix at i40e_tx_queue_release_mbufs so no performance impact.
>>>>>
>>>> This patch also supercedes this one, 
>>>> http://dpdk.org/dev/patchwork/patch/29814/
>>>> right?
>>>> Roger, can you please confirm that this alternative fix works in your
>>>> testing and that your patch is no longer necessary too.
>>>>
>>>> Thanks,
>>>> /Bruce
>>>>
>>>> .
>>>>
>> .
>>
>
> .
>

Re: [dpdk-dev] [dpdk-stable] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Richardson, Bruce]

> -----Original Message-----
> From: Roger B. Melton [mailto:rmelton@cisco.com]
> Sent: Thursday, October 12, 2017 12:15 PM
> To: Richardson, Bruce <bruce.richardson@intel.com>
> Cc: Zhang, Qi Z <qi.z.zhang@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>;
> Xing, Beilei <beilei.xing@intel.com>; dev@dpdk.org; stable@dpdk.org
> Subject: Re: [dpdk-stable] [dpdk-dev] [PATCH] net/i40e: fix unexpected
> mbuf free in vPMD
> 
> On 10/10/17 7:47 AM, Roger B. Melton wrote:
> > On 10/10/17 7:39 AM, Bruce Richardson wrote:
> >> On Tue, Oct 10, 2017 at 07:05:33AM -0400, Roger B. Melton wrote:
> >>> Hi Bruce,
> >>>
> >>> I can.  It will take a day or 2 to get the results.
> >>>
> >>> Regards,
> >>> Roger
> >>>
> >> Thanks. Keep us posted. We want to ensure we have the best fix
> >> possible for the issue in this release.
> >
> > Thanks for finding a better solution ;-).  I'll let you know the
> > results when I have them, just wanted you to know it may take a couple
> > of days to turn around.
> >
> > -Roger
> 
> Confirmed that this patch does address the issue.  Thanks again Bruce/Qi
> for finding a better alternative.
> 
> Regards, Roger
> >
Great.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>

See also comment below.

> >>
> >> /Bruce
> >>
> >>> On 10/10/17 4:48 AM, Bruce Richardson wrote:
> >>>> +Roger Melton
> >>>>
> >>>> On Tue, Oct 10, 2017 at 09:22:05AM -0400, Qi Zhang wrote:
> >>>>> vPMD tx does not set sw_ring's mbuf to NULL after free it.
> >>>>> So to prevent same mbuf be free again, we need more carefully
> >>>>> check in i40e_tx_queue_release_mbufs.
> >>>>>
Maybe this commit message could be reworded to be a bit more detailed.

"vPMD tx does not set sw_ring's mbuf to NULL after free.
Therefore, in cases where the vector transmit function is in
use, we must use the appropriate index and threshold values
for the queue to only free the unreleased mbufs"

Re: [dpdk-dev] [dpdk-stable] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Ferruh Yigit]

On 10/10/2017 2:22 PM, Qi Zhang wrote:
> vPMD tx does not set sw_ring's mbuf to NULL after free it.
> So to prevent same mbuf be free again, we need more carefully
> check in i40e_tx_queue_release_mbufs.
> 
> Fixes: b4669bb95038 ("i40e: add vector Tx")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>

> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Applied to dpdk-next-net/master, thanks.

(Used suggested commit log, thanks.)

[dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Qi Zhang]

The patch reset tx queue sw_ring's mbuf to NULL after it is free in
i40_tx_free_bufs, this prevent same mbuf be free again in
i40e_dev_tx_queue_release. This fix follow the same implemenation of
non-vPMD.

Fixes: b4669bb95038 ("i40e: add vector Tx")
Cc: stable@dpdk.org

Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
---
 drivers/net/i40e/i40e_rxtx_vec_common.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h b/drivers/net/i40e/i40e_rxtx_vec_common.h
index 39a6da0..ed51b4d 100644
--- a/drivers/net/i40e/i40e_rxtx_vec_common.h
+++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
@@ -124,11 +124,13 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
 	  */
 	txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)];
 	m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
+	txep[0].mbuf = NULL;
 	if (likely(m != NULL)) {
 		free[0] = m;
 		nb_free = 1;
 		for (i = 1; i < n; i++) {
 			m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+			txep[i].mbuf = NULL;
 			if (likely(m != NULL)) {
 				if (likely(m->pool == free[0]->pool)) {
 					free[nb_free++] = m;
@@ -145,6 +147,7 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
 	} else {
 		for (i = 1; i < n; i++) {
 			m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
+			txep[i].mbuf = NULL;
 			if (m != NULL)
 				rte_mempool_put(m->pool, m);
 		}
-- 
2.9.5

Re: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Wu, Jingjing]

Hi, qi

Is your patch duplicated with this http://www.dpdk.org/dev/patchwork/patch/29814/ ?

Thanks
Jingjing

> -----Original Message-----
> From: Zhang, Qi Z
> Sent: Monday, October 9, 2017 4:54 PM
> To: Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; stable@dpdk.org
> Subject: [PATCH] net/i40e: fix unexpected mbuf free in vPMD
> 
> The patch reset tx queue sw_ring's mbuf to NULL after it is free in
> i40_tx_free_bufs, this prevent same mbuf be free again in
> i40e_dev_tx_queue_release. This fix follow the same implemenation of non-
> vPMD.
> 
> Fixes: b4669bb95038 ("i40e: add vector Tx")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> ---
>  drivers/net/i40e/i40e_rxtx_vec_common.h | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h
> b/drivers/net/i40e/i40e_rxtx_vec_common.h
> index 39a6da0..ed51b4d 100644
> --- a/drivers/net/i40e/i40e_rxtx_vec_common.h
> +++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
> @@ -124,11 +124,13 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
>  	  */
>  	txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)];
>  	m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
> +	txep[0].mbuf = NULL;
>  	if (likely(m != NULL)) {
>  		free[0] = m;
>  		nb_free = 1;
>  		for (i = 1; i < n; i++) {
>  			m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
> +			txep[i].mbuf = NULL;
>  			if (likely(m != NULL)) {
>  				if (likely(m->pool == free[0]->pool)) {
>  					free[nb_free++] = m;
> @@ -145,6 +147,7 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
>  	} else {
>  		for (i = 1; i < n; i++) {
>  			m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
> +			txep[i].mbuf = NULL;
>  			if (m != NULL)
>  				rte_mempool_put(m->pool, m);
>  		}
> --
> 2.9.5

Re: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Ananyev, Konstantin]

Hi Zhang,

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Qi Zhang
> Sent: Monday, October 9, 2017 9:54 AM
> To: Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; stable@dpdk.org
> Subject: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD
> 
> The patch reset tx queue sw_ring's mbuf to NULL after it is free in
> i40_tx_free_bufs, this prevent same mbuf be free again in
> i40e_dev_tx_queue_release. This fix follow the same implemenation of
> non-vPMD.

Wonder why we can't change i40e_dev_tx_queue_release() instead, 
so it will only go through the TXDs that were really armed?
Let say from txq->tx_next_dd - tx_rs_thresh - 1 till txq->tx_tail?
Let say _ixgbe_tx_queue_release_mbufs_vec() works that way.
Then we probably can keep our runtime code intact.
Konstantin

> 
> Fixes: b4669bb95038 ("i40e: add vector Tx")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> ---
>  drivers/net/i40e/i40e_rxtx_vec_common.h | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h b/drivers/net/i40e/i40e_rxtx_vec_common.h
> index 39a6da0..ed51b4d 100644
> --- a/drivers/net/i40e/i40e_rxtx_vec_common.h
> +++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
> @@ -124,11 +124,13 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
>  	  */
>  	txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)];
>  	m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
> +	txep[0].mbuf = NULL;
>  	if (likely(m != NULL)) {
>  		free[0] = m;
>  		nb_free = 1;
>  		for (i = 1; i < n; i++) {
>  			m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
> +			txep[i].mbuf = NULL;
>  			if (likely(m != NULL)) {
>  				if (likely(m->pool == free[0]->pool)) {
>  					free[nb_free++] = m;
> @@ -145,6 +147,7 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
>  	} else {
>  		for (i = 1; i < n; i++) {
>  			m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
> +			txep[i].mbuf = NULL;
>  			if (m != NULL)
>  				rte_mempool_put(m->pool, m);
>  		}
> --
> 2.9.5

Re: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Bruce Richardson]

On Mon, Oct 09, 2017 at 12:44:25PM +0000, Ananyev, Konstantin wrote:
> Hi Zhang,
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Qi Zhang
> > Sent: Monday, October 9, 2017 9:54 AM
> > To: Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei <beilei.xing@intel.com>
> > Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; stable@dpdk.org
> > Subject: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD
> > 
> > The patch reset tx queue sw_ring's mbuf to NULL after it is free in
> > i40_tx_free_bufs, this prevent same mbuf be free again in
> > i40e_dev_tx_queue_release. This fix follow the same implemenation of
> > non-vPMD.
> 
> Wonder why we can't change i40e_dev_tx_queue_release() instead, 
> so it will only go through the TXDs that were really armed?
> Let say from txq->tx_next_dd - tx_rs_thresh - 1 till txq->tx_tail?
> Let say _ixgbe_tx_queue_release_mbufs_vec() works that way.
> Then we probably can keep our runtime code intact.
> Konstantin
> 
+1 to this. We should avoid adding writes to the TX path. Instead we
should ensure that other routines properly track the buffers to be freed
using the appropriate indexes.

/Bruce

Re: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD

[Zhang, Qi Z]

Hi Konstantin:

> -----Original Message-----
> From: Ananyev, Konstantin
> Sent: Monday, October 9, 2017 8:44 PM
> To: Zhang, Qi Z <qi.z.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>;
> Xing, Beilei <beilei.xing@intel.com>
> Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; stable@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD
> 
> Hi Zhang,
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Qi Zhang
> > Sent: Monday, October 9, 2017 9:54 AM
> > To: Wu, Jingjing <jingjing.wu@intel.com>; Xing, Beilei
> > <beilei.xing@intel.com>
> > Cc: dev@dpdk.org; Zhang, Qi Z <qi.z.zhang@intel.com>; stable@dpdk.org
> > Subject: [dpdk-dev] [PATCH] net/i40e: fix unexpected mbuf free in vPMD
> >
> > The patch reset tx queue sw_ring's mbuf to NULL after it is free in
> > i40_tx_free_bufs, this prevent same mbuf be free again in
> > i40e_dev_tx_queue_release. This fix follow the same implemenation of
> > non-vPMD.
> 
> Wonder why we can't change i40e_dev_tx_queue_release() instead, so it will
> only go through the TXDs that were really armed?
Yes we can, I've thought about this before the patch, but after I measure the performance impact, I saw less than 0.6% downgrade for single core performance.
So I think maybe a simple fix could be good enough and low risk and an improvement can be delivered later after carefully validate.

> Let say from txq->tx_next_dd - tx_rs_thresh - 1 till txq->tx_tail?
> Let say _ixgbe_tx_queue_release_mbufs_vec() works that way.

Yes, That's the same way I figured out, thanks for giving the detail instruction that confirm my thought :)
Now, I think my v2 can upgrade to this directly.

Thanks
Qi

> Then we probably can keep our runtime code intact.

> Konstantin
> 
> >
> > Fixes: b4669bb95038 ("i40e: add vector Tx")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
> > ---
> >  drivers/net/i40e/i40e_rxtx_vec_common.h | 3 +++
> >  1 file changed, 3 insertions(+)
> >
> > diff --git a/drivers/net/i40e/i40e_rxtx_vec_common.h
> > b/drivers/net/i40e/i40e_rxtx_vec_common.h
> > index 39a6da0..ed51b4d 100644
> > --- a/drivers/net/i40e/i40e_rxtx_vec_common.h
> > +++ b/drivers/net/i40e/i40e_rxtx_vec_common.h
> > @@ -124,11 +124,13 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
> >  	  */
> >  	txep = &txq->sw_ring[txq->tx_next_dd - (n - 1)];
> >  	m = rte_pktmbuf_prefree_seg(txep[0].mbuf);
> > +	txep[0].mbuf = NULL;
> >  	if (likely(m != NULL)) {
> >  		free[0] = m;
> >  		nb_free = 1;
> >  		for (i = 1; i < n; i++) {
> >  			m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
> > +			txep[i].mbuf = NULL;
> >  			if (likely(m != NULL)) {
> >  				if (likely(m->pool == free[0]->pool)) {
> >  					free[nb_free++] = m;
> > @@ -145,6 +147,7 @@ i40e_tx_free_bufs(struct i40e_tx_queue *txq)
> >  	} else {
> >  		for (i = 1; i < n; i++) {
> >  			m = rte_pktmbuf_prefree_seg(txep[i].mbuf);
> > +			txep[i].mbuf = NULL;
> >  			if (m != NULL)
> >  				rte_mempool_put(m->pool, m);
> >  		}
> > --
> > 2.9.5