[dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup

[dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup

[Chengfeng Ye]

The lock sh->txpp.mutex was not correctly released on one path
of cleanup function return, potentially causing the deadlock.

Fixes: d133f4cdb7 ("net/mlx5: create clock queue for packet pacing")
Cc: stable@dpdk.org

Signed-off-by: Chengfeng Ye <cyeaa@connect.ust.hk>
---
 drivers/net/mlx5/mlx5_txpp.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 4f6da9f2d1..0ece788a84 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -961,8 +961,12 @@ mlx5_txpp_stop(struct rte_eth_dev *dev)
 	MLX5_ASSERT(!ret);
 	RTE_SET_USED(ret);
 	MLX5_ASSERT(sh->txpp.refcnt);
-	if (!sh->txpp.refcnt || --sh->txpp.refcnt)
+	if (!sh->txpp.refcnt || --sh->txpp.refcnt) {
+		ret = pthread_mutex_unlock(&sh->txpp.mutex);
+		MLX5_ASSERT(!ret);
+		RTE_SET_USED(ret);
 		return;
+	}
 	/* No references any more, do actual destroy. */
 	mlx5_txpp_destroy(sh);
 	ret = pthread_mutex_unlock(&sh->txpp.mutex);
-- 
2.17.1

Re: [dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup

[Ferruh Yigit]

On 11/11/2021 7:06 AM, Slava Ovsiienko wrote:
> Hi, Ferruh
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Wednesday, November 10, 2021 18:57
>> To: Chengfeng Ye <cyeaa@connect.ust.hk>; david.marchand@redhat.com;
>> Slava Ovsiienko <viacheslavo@nvidia.com>; Shahaf Shuler
>> <shahafs@nvidia.com>; Matan Azrad <matan@nvidia.com>
>> Cc: dev@dpdk.org; stable@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp
>> cleanup
>>
>> On 10/12/2021 11:02 AM, Chengfeng Ye wrote:
>>> The lock sh->txpp.mutex was not correctly released on one path of
>>> cleanup function return, potentially causing the deadlock.
>>>
>>> Fixes: d133f4cdb7 ("net/mlx5: create clock queue for packet pacing")
>>> Cc: stable@dpdk.org
>>>
>>> Signed-off-by: Chengfeng Ye <cyeaa@connect.ust.hk>
>>> ---
>>>    drivers/net/mlx5/mlx5_txpp.c | 6 +++++-
>>>    1 file changed, 5 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/net/mlx5/mlx5_txpp.c
>>> b/drivers/net/mlx5/mlx5_txpp.c index 4f6da9f2d1..0ece788a84 100644
>>> --- a/drivers/net/mlx5/mlx5_txpp.c
>>> +++ b/drivers/net/mlx5/mlx5_txpp.c
>>> @@ -961,8 +961,12 @@ mlx5_txpp_stop(struct rte_eth_dev *dev)
>>>    	MLX5_ASSERT(!ret);
>>>    	RTE_SET_USED(ret);
>>>    	MLX5_ASSERT(sh->txpp.refcnt);
>>> -	if (!sh->txpp.refcnt || --sh->txpp.refcnt)
>>> +	if (!sh->txpp.refcnt || --sh->txpp.refcnt) {
>>> +		ret = pthread_mutex_unlock(&sh->txpp.mutex);
>>> +		MLX5_ASSERT(!ret);
>>> +		RTE_SET_USED(ret);
>>
>> Is this 'RTE_SET_USED()' need to be used multiple times for same variable?
> mmm, It seems "claim_zero()" macro would be better here:
> 
> claim_zero(pthread_mutex_lock(&sh->txpp.mutex));
> 
> I will provide the cleanup patch, thank you for noticing that
> 
>> This usage looks ugly, I can see why it is used but I wonder if this can be
>> solved differently, what about something like following:
>>
>>    #ifdef RTE_LIBRTE_MLX5_DEBUG
>>     #define MLX5_ASSERT(exp) RTE_VERIFY(exp)
>>    #else
>>     #ifdef RTE_ENABLE_ASSERT
>>      #define MLX5_ASSERT(exp) RTE_ASSERT(exp)
>>     #else
>>      #define MLX5_ASSERT(exp) RTE_SET_USED(exp)
>>     #endif
>>    #endif
> It would directly replace MLX5_ASSERT(exp) with RTE_SET_USED(exp)
> if there is neither RTE_ENABLE_ASSERT nor RTE_LIBRTE_MLX5_DEBUG.
> We would not like to drop the "not used" check functionality at all , right?
> 

The suggestion was to prevent following kind of usage:
  	MLX5_ASSERT(!ret);
  	RTE_SET_USED(ret);

I assume you need above usage when a variable is used only in the 'MLX5_ASSERT',
if there is a way to prevent warning in that case without 'RTE_SET_USED' that
may be better.

RE: [dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup

[Slava Ovsiienko]

Hi, Ferruh

> -----Original Message-----
> From: Ferruh Yigit <ferruh.yigit@intel.com>
> Sent: Wednesday, November 10, 2021 18:57
> To: Chengfeng Ye <cyeaa@connect.ust.hk>; david.marchand@redhat.com;
> Slava Ovsiienko <viacheslavo@nvidia.com>; Shahaf Shuler
> <shahafs@nvidia.com>; Matan Azrad <matan@nvidia.com>
> Cc: dev@dpdk.org; stable@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp
> cleanup
> 
> On 10/12/2021 11:02 AM, Chengfeng Ye wrote:
> > The lock sh->txpp.mutex was not correctly released on one path of
> > cleanup function return, potentially causing the deadlock.
> >
> > Fixes: d133f4cdb7 ("net/mlx5: create clock queue for packet pacing")
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Chengfeng Ye <cyeaa@connect.ust.hk>
> > ---
> >   drivers/net/mlx5/mlx5_txpp.c | 6 +++++-
> >   1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/net/mlx5/mlx5_txpp.c
> > b/drivers/net/mlx5/mlx5_txpp.c index 4f6da9f2d1..0ece788a84 100644
> > --- a/drivers/net/mlx5/mlx5_txpp.c
> > +++ b/drivers/net/mlx5/mlx5_txpp.c
> > @@ -961,8 +961,12 @@ mlx5_txpp_stop(struct rte_eth_dev *dev)
> >   	MLX5_ASSERT(!ret);
> >   	RTE_SET_USED(ret);
> >   	MLX5_ASSERT(sh->txpp.refcnt);
> > -	if (!sh->txpp.refcnt || --sh->txpp.refcnt)
> > +	if (!sh->txpp.refcnt || --sh->txpp.refcnt) {
> > +		ret = pthread_mutex_unlock(&sh->txpp.mutex);
> > +		MLX5_ASSERT(!ret);
> > +		RTE_SET_USED(ret);
> 
> Is this 'RTE_SET_USED()' need to be used multiple times for same variable?
mmm, It seems "claim_zero()" macro would be better here:

claim_zero(pthread_mutex_lock(&sh->txpp.mutex));

I will provide the cleanup patch, thank you for noticing that

> This usage looks ugly, I can see why it is used but I wonder if this can be
> solved differently, what about something like following:
> 
>   #ifdef RTE_LIBRTE_MLX5_DEBUG
>    #define MLX5_ASSERT(exp) RTE_VERIFY(exp)
>   #else
>    #ifdef RTE_ENABLE_ASSERT
>     #define MLX5_ASSERT(exp) RTE_ASSERT(exp)
>    #else
>     #define MLX5_ASSERT(exp) RTE_SET_USED(exp)
>    #endif
>   #endif
It would directly replace MLX5_ASSERT(exp) with RTE_SET_USED(exp)
if there is neither RTE_ENABLE_ASSERT nor RTE_LIBRTE_MLX5_DEBUG.
We would not like to drop the "not used" check functionality at all , right?

With best regards,
Slava

Re: [dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup

[Ferruh Yigit]

On 10/12/2021 11:02 AM, Chengfeng Ye wrote:
> The lock sh->txpp.mutex was not correctly released on one path
> of cleanup function return, potentially causing the deadlock.
> 
> Fixes: d133f4cdb7 ("net/mlx5: create clock queue for packet pacing")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Chengfeng Ye <cyeaa@connect.ust.hk>
> ---
>   drivers/net/mlx5/mlx5_txpp.c | 6 +++++-
>   1 file changed, 5 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
> index 4f6da9f2d1..0ece788a84 100644
> --- a/drivers/net/mlx5/mlx5_txpp.c
> +++ b/drivers/net/mlx5/mlx5_txpp.c
> @@ -961,8 +961,12 @@ mlx5_txpp_stop(struct rte_eth_dev *dev)
>   	MLX5_ASSERT(!ret);
>   	RTE_SET_USED(ret);
>   	MLX5_ASSERT(sh->txpp.refcnt);
> -	if (!sh->txpp.refcnt || --sh->txpp.refcnt)
> +	if (!sh->txpp.refcnt || --sh->txpp.refcnt) {
> +		ret = pthread_mutex_unlock(&sh->txpp.mutex);
> +		MLX5_ASSERT(!ret);
> +		RTE_SET_USED(ret);

Is this 'RTE_SET_USED()' need to be used multiple times for same variable?

This usage looks ugly, I can see why it is used but I wonder if this can
be solved differently, what about something like following:

  #ifdef RTE_LIBRTE_MLX5_DEBUG
   #define MLX5_ASSERT(exp) RTE_VERIFY(exp)
  #else
   #ifdef RTE_ENABLE_ASSERT
    #define MLX5_ASSERT(exp) RTE_ASSERT(exp)
   #else
    #define MLX5_ASSERT(exp) RTE_SET_USED(exp)
   #endif
  #endif

>   		return;
> +	}
>   	/* No references any more, do actual destroy. */
>   	mlx5_txpp_destroy(sh);
>   	ret = pthread_mutex_unlock(&sh->txpp.mutex);
> 

Re: [dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup

[Raslan Darawsheh]

Hi,

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Chengfeng Ye
> Sent: Tuesday, October 12, 2021 1:02 PM
> To: david.marchand@redhat.com; Slava Ovsiienko
> <viacheslavo@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Matan
> Azrad <matan@nvidia.com>
> Cc: dev@dpdk.org; Chengfeng Ye <cyeaa@connect.ust.hk>;
> stable@dpdk.org
> Subject: [dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup
> 
> The lock sh->txpp.mutex was not correctly released on one path of cleanup
> function return, potentially causing the deadlock.
> 
> Fixes: d133f4cdb7 ("net/mlx5: create clock queue for packet pacing")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Chengfeng Ye <cyeaa@connect.ust.hk>

Patch applied to next-net-mlx,

Kindest regards
Raslan Darawsheh

Re: [dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup

[Slava Ovsiienko]

> -----Original Message-----
> From: Chengfeng Ye <cyeaa@connect.ust.hk>
> Sent: Tuesday, October 12, 2021 13:02
> To: david.marchand@redhat.com; Slava Ovsiienko
> <viacheslavo@nvidia.com>; Shahaf Shuler <shahafs@nvidia.com>; Matan
> Azrad <matan@nvidia.com>
> Cc: dev@dpdk.org; Chengfeng Ye <cyeaa@connect.ust.hk>;
> stable@dpdk.org
> Subject: [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup
> 
> The lock sh->txpp.mutex was not correctly released on one path of cleanup
> function return, potentially causing the deadlock.
> 
> Fixes: d133f4cdb7 ("net/mlx5: create clock queue for packet pacing")
> Cc: stable@dpdk.org
> 
> Signed-off-by: Chengfeng Ye <cyeaa@connect.ust.hk>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>

[dpdk-dev] [PATCH v5] net/mlx5: fix mutex unlock in txpp cleanup

[Chengfeng Ye]

The lock sh->txpp.mutex was not correctly released on one path
of cleanup function return, potentially causing the deadlock.

Fixes: d133f4cdb7 ("net/mlx5: create clock queue for packet pacing")
Cc: stable@dpdk.org

Signed-off-by: Chengfeng Ye <cyeaa@connect.ust.hk>
---
 drivers/net/mlx5/mlx5_txpp.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/net/mlx5/mlx5_txpp.c b/drivers/net/mlx5/mlx5_txpp.c
index 4f6da9f2d1..0ece788a84 100644
--- a/drivers/net/mlx5/mlx5_txpp.c
+++ b/drivers/net/mlx5/mlx5_txpp.c
@@ -961,8 +961,12 @@ mlx5_txpp_stop(struct rte_eth_dev *dev)
 	MLX5_ASSERT(!ret);
 	RTE_SET_USED(ret);
 	MLX5_ASSERT(sh->txpp.refcnt);
-	if (!sh->txpp.refcnt || --sh->txpp.refcnt)
+	if (!sh->txpp.refcnt || --sh->txpp.refcnt) {
+		ret = pthread_mutex_unlock(&sh->txpp.mutex);
+		MLX5_ASSERT(!ret);
+		RTE_SET_USED(ret);
 		return;
+	}
 	/* No references any more, do actual destroy. */
 	mlx5_txpp_destroy(sh);
 	ret = pthread_mutex_unlock(&sh->txpp.mutex);
-- 
2.17.1