DPDK patches and discussions
 help / color / mirror / Atom feed
From: Julien Meunier <julien.meunier@nokia.com>
To: "Wang, Haiyue" <haiyue.wang@intel.com>, "dev@dpdk.org" <dev@dpdk.org>
Cc: "stable@dpdk.org" <stable@dpdk.org>,
	"Richardson, Bruce" <bruce.richardson@intel.com>
Subject: Re: [dpdk-dev] [PATCH] net/ixgbe: fix RxQ/TxQ release
Date: Tue, 28 Sep 2021 09:46:12 +0200	[thread overview]
Message-ID: <6e713015-8438-5af0-dc34-5372795b34c7@nokia.com> (raw)
In-Reply-To: <DM4PR11MB5518DE0865D89CD6F2A462B1F7A89@DM4PR11MB5518.namprd11.prod.outlook.com>

Hello,

On 28/09/2021 05:21, Wang, Haiyue wrote:
>> -----Original Message-----
>> From: Wang, Haiyue
>> Sent: Tuesday, September 28, 2021 11:06
>> To: 'Julien Meunier' <julien.meunier@nokia.com>; dev@dpdk.org
>> Cc: stable@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>
>> Subject: RE: [PATCH] net/ixgbe: fix RxQ/TxQ release
>>
>>> -----Original Message-----
>>> From: Julien Meunier <julien.meunier@nokia.com>
>>> Sent: Tuesday, September 28, 2021 01:18
>>> To: dev@dpdk.org
>>> Cc: stable@dpdk.org; Richardson, Bruce <bruce.richardson@intel.com>; Wang, Haiyue
>>> <haiyue.wang@intel.com>
>>> Subject: [PATCH] net/ixgbe: fix RxQ/TxQ release
>>>
>>> On the vector implementation, during the tear-down, the mbufs not
>>> drained in the RxQ and TxQ are freed based on an algorithm which
>>> supposed that the number of descriptors is a power of 2 (max_desc).
>>> Based on this hypothesis, this algorithm uses a bitmask in order to
>>> detect an index overflow during the iteration, and to restart the loop
>>> from 0.
>>>
>>> However, there is no such power of 2 requirement in the ixgbe for the
>>> number of descriptors in the RxQ / TxQ. The only requirement is to have
>>> a number correctly aligned.
>>>
>>> If a user requested to configure a number of descriptors which is not a
>>> power of 2, as a consequence, during the tear-down, it was possible to
>>> be in an infinite loop, and to never reach the exit loop condition.
>>>
>>
>> Are you able to setup not a power of 2 successfully ?
>>
> 
> My fault, yes, possible. ;-)
> 

Yes, we have some usecases where the nb of descriptiors for the TxQ is 
set to 1536.
I modified the test_pmd_perf in order to validate this behavior, as my 
ixgbe X550 supports the loopback mode:
- nb_desc = 2048 => txq is drained and stopped correctly
- nb_desc = 1536 => freeze during the teardown

>> int
>> rte_eth_tx_queue_setup(uint16_t port_id, uint16_t tx_queue_id,
>> 		       uint16_t nb_tx_desc, unsigned int socket_id,
>> 		       const struct rte_eth_txconf *tx_conf)
>> {
>> 	...
>>
>> 	if (nb_tx_desc > dev_info.tx_desc_lim.nb_max ||
>> 	    nb_tx_desc < dev_info.tx_desc_lim.nb_min ||
>> 	    nb_tx_desc % dev_info.tx_desc_lim.nb_align != 0) {
>> 		RTE_ETHDEV_LOG(ERR,
>> 			"Invalid value for nb_tx_desc(=%hu), should be: <= %hu, >= %hu, and a product
>> of %hu\n",
>> 			nb_tx_desc, dev_info.tx_desc_lim.nb_max,
>> 			dev_info.tx_desc_lim.nb_min,
>> 			dev_info.tx_desc_lim.nb_align);
>> 		return -EINVAL;
>> 	}
>>
>> 	...
>>
>> }
>>
>>> By removing the bitmask and changing the loop method, we can avoid this
>>> issue, and allow the user to configure a RxQ / TxQ which is not a power
>>> of 2.
>>>
>>> Fixes: c95584dc2b18 ("ixgbe: new vectorized functions for Rx/Tx")
>>> Cc: bruce.richardson@intel.com
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Julien Meunier <julien.meunier@nokia.com>
>>> ---
>>>   drivers/net/ixgbe/ixgbe_rxtx_vec_common.h | 20 +++++++++++++-------
>>>   1 file changed, 13 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
>>> index adba855ca3..8912558918 100644
>>> --- a/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
>>> +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec_common.h
>>> @@ -150,11 +150,14 @@ _ixgbe_tx_queue_release_mbufs_vec(struct ixgbe_tx_queue *txq)
>>>   		return;
>>>
> 
> Just one line ?
> 
> i = (i + 1) % txq->nb_tx_desc
> 

Ah yes, I was too focused with this bitmask...
The shorter, the better.

I will send a V2 today.

Thanks for this feedback !

>>>   	/* release the used mbufs in sw_ring */
>>> -	for (i = txq->tx_next_dd - (txq->tx_rs_thresh - 1);
>>> -	     i != txq->tx_tail;
>>> -	     i = (i + 1) & max_desc) {
>>> +	i = txq->tx_next_dd - (txq->tx_rs_thresh - 1);
>>> +	while (i != txq->tx_tail) {
>>>   		txe = &txq->sw_ring_v[i];
>>>   		rte_pktmbuf_free_seg(txe->mbuf);
>>> +
>>> +		i = i + 1;
>>> +		if (i > max_desc)
>>> +			i = 0;
>>>   	}
>>>   	txq->nb_tx_free = max_desc;
>>>
>>> @@ -168,7 +171,7 @@ _ixgbe_tx_queue_release_mbufs_vec(struct ixgbe_tx_queue *txq)
>>>   static inline void
>>>   _ixgbe_rx_queue_release_mbufs_vec(struct ixgbe_rx_queue *rxq)
>>>   {
>>> -	const unsigned int mask = rxq->nb_rx_desc - 1;
>>> +	const unsigned int max_desc = rxq->nb_rx_desc - 1;
>>>   	unsigned int i;
>>>
>>>   	if (rxq->sw_ring == NULL || rxq->rxrearm_nb >= rxq->nb_rx_desc)
>>> @@ -181,11 +184,14 @@ _ixgbe_rx_queue_release_mbufs_vec(struct ixgbe_rx_queue *rxq)
>>>   				rte_pktmbuf_free_seg(rxq->sw_ring[i].mbuf);
>>>   		}
>>>   	} else {
>>> -		for (i = rxq->rx_tail;
>>> -		     i != rxq->rxrearm_start;
>>> -		     i = (i + 1) & mask) {
>>> +		i = rxq->rx_tail;
>>> +		while (i != rxq->rxrearm_start) {
>>>   			if (rxq->sw_ring[i].mbuf != NULL)
>>>   				rte_pktmbuf_free_seg(rxq->sw_ring[i].mbuf);
>>> +
>>> +			i = i + 1;
>>> +			if (i > max_desc)
>>> +				i = 0;
>>>   		}
>>>   	}
>>>
>>> --
>>> 2.17.1
> 

-- 
Julien Meunier

  reply	other threads:[~2021-09-28  7:46 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-27 17:18 Julien Meunier
2021-09-28  3:06 ` Wang, Haiyue
2021-09-28  3:21   ` Wang, Haiyue
2021-09-28  7:46     ` Julien Meunier [this message]
2021-09-28  8:12 ` [dpdk-dev] [PATCH v2] " Julien Meunier
2021-09-28 12:18   ` Wang, Haiyue
2021-10-09  4:43     ` Zhang, Qi Z

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6e713015-8438-5af0-dc34-5372795b34c7@nokia.com \
    --to=julien.meunier@nokia.com \
    --cc=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=haiyue.wang@intel.com \
    --cc=stable@dpdk.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).