[dpdk-dev] [PATCH] ethdev: check for Rx RSS distribution and RSS hash

[dpdk-dev] [PATCH] ethdev: check for Rx RSS distribution and RSS hash

[Lijun Ou]

When rte api checks the Rx RSS distribution is enable but the RSS
hash is disabled, it will return an error.

Signed-off-by: Lijun Ou <oulijun@huawei.com>
---
 lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 0854ef8..07734c4 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
 		goto rollback;
 	}
 
+	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
+	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
+	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
+		RTE_ETHDEV_LOG(ERR,
+			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
+			port_id,
+			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
+		ret = -EINVAL;
+		goto rollback;
+	}
+
 	/*
 	 * Setup new number of RX/TX queues and reconfigure device.
 	 */
-- 
2.7.4

Re: [dpdk-dev] [PATCH] ethdev: check for Rx RSS distribution and RSS hash

[Andrew Rybchenko]

On 4/27/20 3:34 PM, Lijun Ou wrote:
> When rte api checks the Rx RSS distribution is enable but the RSS
> hash is disabled, it will return an error.
>
> Signed-off-by: Lijun Ou <oulijun@huawei.com>
> ---
>  lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
> index 0854ef8..07734c4 100644
> --- a/lib/librte_ethdev/rte_ethdev.c
> +++ b/lib/librte_ethdev/rte_ethdev.c
> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>  		goto rollback;
>  	}
>  
> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
> +		RTE_ETHDEV_LOG(ERR,
> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
> +			port_id,
> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
> +		ret = -EINVAL;
> +		goto rollback;
> +	}
> +
>  	/*
>  	 * Setup new number of RX/TX queues and reconfigure device.
>  	 */

NACK. It is perfectly fine to do distribution, but do not need RSS hash
information.
 - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
 - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
   itself from HW to SW

Re: [dpdk-dev] [PATCH] ethdev: check for Rx RSS distribution and RSS hash

[Ferruh Yigit]

On 4/27/2020 1:49 PM, Andrew Rybchenko wrote:
> On 4/27/20 3:34 PM, Lijun Ou wrote:
>> When rte api checks the Rx RSS distribution is enable but the RSS
>> hash is disabled, it will return an error.
>>
>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>> ---
>>  lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>  1 file changed, 11 insertions(+)
>>
>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>> index 0854ef8..07734c4 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>  		goto rollback;
>>  	}
>>  
>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>> +		RTE_ETHDEV_LOG(ERR,
>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>> +			port_id,
>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>> +		ret = -EINVAL;
>> +		goto rollback;
>> +	}
>> +
>>  	/*
>>  	 * Setup new number of RX/TX queues and reconfigure device.
>>  	 */
> 
> NACK. It is perfectly fine to do distribution, but do not need RSS hash
> information.
>  - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>  - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>    itself from HW to SW
> 

Hi Lijun,

As Andrew described, now 'DEV_RX_OFFLOAD_RSS_HASH' controls to copy calculated
hash value to 'mbuf::hash::rss' or not. When rss hash copied to mubf,
'PKT_RX_RSS_HASH' flag of 'mbuf::ol_flags' set to notify the application that
hash value is valid. This was a performance optimization.

We know above because we were involved in development of it, if this is not
clean for third party, can you please amend your patch to clarify above behavior?

Thanks,
ferruh

Re: [dpdk-dev] [PATCH] ethdev: check for Rx RSS distribution and RSS hash

[oulijun]

在 2020/4/27 20:49, Andrew Rybchenko 写道:
> On 4/27/20 3:34 PM, Lijun Ou wrote:
>> When rte api checks the Rx RSS distribution is enable but the RSS
>> hash is disabled, it will return an error.
>>
>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>> ---
>>   lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>   1 file changed, 11 insertions(+)
>>
>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>> index 0854ef8..07734c4 100644
>> --- a/lib/librte_ethdev/rte_ethdev.c
>> +++ b/lib/librte_ethdev/rte_ethdev.c
>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>   		goto rollback;
>>   	}
>>   
>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>> +		RTE_ETHDEV_LOG(ERR,
>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>> +			port_id,
>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>> +		ret = -EINVAL;
>> +		goto rollback;
>> +	}
>> +
>>   	/*
>>   	 * Setup new number of RX/TX queues and reconfigure device.
>>   	 */
> 
> NACK. It is perfectly fine to do distribution, but do not need RSS hash
> information.
>   - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>   - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>     itself from HW to SW
> 
Thanks for your detail explanation. I still have a question, what is the 
use of setting DEV_RX_OFFLOAD_HASH, what is the difference between user 
configuration DEV_RX_OFFLOAD_HASH and no configuration?

> .
> 

Re: [dpdk-dev] [PATCH] ethdev: check for Rx RSS distribution and RSS hash

[oulijun]

在 2020/4/27 21:27, Ferruh Yigit 写道:
> On 4/27/2020 1:49 PM, Andrew Rybchenko wrote:
>> On 4/27/20 3:34 PM, Lijun Ou wrote:
>>> When rte api checks the Rx RSS distribution is enable but the RSS
>>> hash is disabled, it will return an error.
>>>
>>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>>> ---
>>>   lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>>   1 file changed, 11 insertions(+)
>>>
>>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>>> index 0854ef8..07734c4 100644
>>> --- a/lib/librte_ethdev/rte_ethdev.c
>>> +++ b/lib/librte_ethdev/rte_ethdev.c
>>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>   		goto rollback;
>>>   	}
>>>   
>>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>>> +		RTE_ETHDEV_LOG(ERR,
>>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>>> +			port_id,
>>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>>> +		ret = -EINVAL;
>>> +		goto rollback;
>>> +	}
>>> +
>>>   	/*
>>>   	 * Setup new number of RX/TX queues and reconfigure device.
>>>   	 */
>>
>> NACK. It is perfectly fine to do distribution, but do not need RSS hash
>> information.
>>   - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>>   - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>>     itself from HW to SW
>>
> 
> Hi Lijun,
> 
> As Andrew described, now 'DEV_RX_OFFLOAD_RSS_HASH' controls to copy calculated
> hash value to 'mbuf::hash::rss' or not. When rss hash copied to mubf,
> 'PKT_RX_RSS_HASH' flag of 'mbuf::ol_flags' set to notify the application that
> hash value is valid. This was a performance optimization.
>
if the user is not configure the DEV_RX_OFFLOAD_RSS_HASH and the users 
add the ETH_MQ_RX_RSS_FLAG， as a result, the hardware has calculated the 
hash result and distributed, the mbuf::o1_flags have set the 
PKT_RX_RSS_HASH and the mbuf::hash::rss have set the hash result. What 
are the advantage of this configuration with DEV_RX_OFFLOAD_RSS_HASH?

Can I understand it this way, If the user does not apply this rss hash, 
does it mean that rss cannot be used to improve performance, even if the 
hardware has calculated the rss hash result?
> We know above because we were involved in development of it, if this is not
> clean for third party, can you please amend your patch to clarify above behavior?
> 
> Thanks,
> ferruh
> 
> .
> 

Re: [dpdk-dev] [PATCH] ethdev: check for Rx RSS distribution and RSS hash

[Ferruh Yigit]

On 4/29/2020 7:41 AM, oulijun wrote:
> 
> 
> 在 2020/4/27 21:27, Ferruh Yigit 写道:
>> On 4/27/2020 1:49 PM, Andrew Rybchenko wrote:
>>> On 4/27/20 3:34 PM, Lijun Ou wrote:
>>>> When rte api checks the Rx RSS distribution is enable but the RSS
>>>> hash is disabled, it will return an error.
>>>>
>>>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>>>> ---
>>>>   lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>>>   1 file changed, 11 insertions(+)
>>>>
>>>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>>>> index 0854ef8..07734c4 100644
>>>> --- a/lib/librte_ethdev/rte_ethdev.c
>>>> +++ b/lib/librte_ethdev/rte_ethdev.c
>>>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>>   		goto rollback;
>>>>   	}
>>>>   
>>>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>>>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>>>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>>>> +		RTE_ETHDEV_LOG(ERR,
>>>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>>>> +			port_id,
>>>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>>>> +		ret = -EINVAL;
>>>> +		goto rollback;
>>>> +	}
>>>> +
>>>>   	/*
>>>>   	 * Setup new number of RX/TX queues and reconfigure device.
>>>>   	 */
>>>
>>> NACK. It is perfectly fine to do distribution, but do not need RSS hash
>>> information.
>>>   - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>>>   - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>>>     itself from HW to SW
>>>
>>
>> Hi Lijun,
>>
>> As Andrew described, now 'DEV_RX_OFFLOAD_RSS_HASH' controls to copy calculated
>> hash value to 'mbuf::hash::rss' or not. When rss hash copied to mubf,
>> 'PKT_RX_RSS_HASH' flag of 'mbuf::ol_flags' set to notify the application that
>> hash value is valid. This was a performance optimization.
>>
> if the user is not configure the DEV_RX_OFFLOAD_RSS_HASH and the users 
> add the ETH_MQ_RX_RSS_FLAG， as a result, the hardware has calculated the 
> hash result and distributed, the mbuf::o1_flags have set the 
> PKT_RX_RSS_HASH and the mbuf::hash::rss have set the hash result. What 
> are the advantage of this configuration with DEV_RX_OFFLOAD_RSS_HASH?

If 'DEV_RX_OFFLOAD_RSS_HASH' NOT set, but 'ETH_MQ_RX_RSS_FLAG' set, RSS should
work as expected but 'mbuf::hash::rss' not filled.

This is optimization for the case user doesn't need the calculated hash value,
and seems this optimization has bigger affect in some NICs that pays some cost
to receive the HW calculated hash values.

User requesting 'DEV_RX_OFFLOAD_RSS_HASH' offload should enable driver to update
'mbuf::hash::rss' and 'PKT_RX_RSS_HASH' in 'mbuf::ol_flags' addition to RSS feature.

> 
> Can I understand it this way, If the user does not apply this rss hash, 
> does it mean that rss cannot be used to improve performance, even if the 
> hardware has calculated the rss hash result?

If 'DEV_RX_OFFLOAD_RSS_HASH' is NOT set, application won't able to receive the
HW calculated hash values.

>> We know above because we were involved in development of it, if this is not
>> clean for third party, can you please amend your patch to clarify above behavior?
>>
>> Thanks,
>> ferruh
>>
>> .
>>
> 

Re: [dpdk-dev] [PATCH] ethdev: check for Rx RSS distribution and RSS hash

[Ferruh Yigit]

On 4/29/2020 1:42 PM, Ferruh Yigit wrote:
> On 4/29/2020 7:41 AM, oulijun wrote:
>>
>>
>> 在 2020/4/27 21:27, Ferruh Yigit 写道:
>>> On 4/27/2020 1:49 PM, Andrew Rybchenko wrote:
>>>> On 4/27/20 3:34 PM, Lijun Ou wrote:
>>>>> When rte api checks the Rx RSS distribution is enable but the RSS
>>>>> hash is disabled, it will return an error.
>>>>>
>>>>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>>>>> ---
>>>>>   lib/librte_ethdev/rte_ethdev.c | 11 +++++++++++
>>>>>   1 file changed, 11 insertions(+)
>>>>>
>>>>> diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
>>>>> index 0854ef8..07734c4 100644
>>>>> --- a/lib/librte_ethdev/rte_ethdev.c
>>>>> +++ b/lib/librte_ethdev/rte_ethdev.c
>>>>> @@ -1411,6 +1411,17 @@ rte_eth_dev_configure(uint16_t port_id, uint16_t nb_rx_q, uint16_t nb_tx_q,
>>>>>   		goto rollback;
>>>>>   	}
>>>>>   
>>>>> +	/* Check if Rx RSS distribution is enable but RSS hash is disabled. */
>>>>> +	if (((dev_conf->rxmode.mq_mode & ETH_MQ_RX_RSS_FLAG) != 0) &&
>>>>> +	    !(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_RSS_HASH)) {
>>>>> +		RTE_ETHDEV_LOG(ERR,
>>>>> +			"Ethdev port_id=%u config valid Rx mq_mode with RSS but %s offload is no-requested\n",
>>>>> +			port_id,
>>>>> +			rte_eth_dev_rx_offload_name(DEV_RX_OFFLOAD_RSS_HASH));
>>>>> +		ret = -EINVAL;
>>>>> +		goto rollback;
>>>>> +	}
>>>>> +
>>>>>   	/*
>>>>>   	 * Setup new number of RX/TX queues and reconfigure device.
>>>>>   	 */
>>>>
>>>> NACK. It is perfectly fine to do distribution, but do not need RSS hash
>>>> information.
>>>>   - ETH_MQ_RX_RSS_FLAG controls RSS hash calculation and distribution
>>>>   - DEV_RX_OFFLOAD_RSS_HASH controls delivery of the hash value
>>>>     itself from HW to SW
>>>>
>>>
>>> Hi Lijun,
>>>
>>> As Andrew described, now 'DEV_RX_OFFLOAD_RSS_HASH' controls to copy calculated
>>> hash value to 'mbuf::hash::rss' or not. When rss hash copied to mubf,
>>> 'PKT_RX_RSS_HASH' flag of 'mbuf::ol_flags' set to notify the application that
>>> hash value is valid. This was a performance optimization.
>>>
>> if the user is not configure the DEV_RX_OFFLOAD_RSS_HASH and the users 
>> add the ETH_MQ_RX_RSS_FLAG， as a result, the hardware has calculated the 
>> hash result and distributed, the mbuf::o1_flags have set the 
>> PKT_RX_RSS_HASH and the mbuf::hash::rss have set the hash result. What 
>> are the advantage of this configuration with DEV_RX_OFFLOAD_RSS_HASH?
> 
> If 'DEV_RX_OFFLOAD_RSS_HASH' NOT set, but 'ETH_MQ_RX_RSS_FLAG' set, RSS should
> work as expected but 'mbuf::hash::rss' not filled.
> 
> This is optimization for the case user doesn't need the calculated hash value,
> and seems this optimization has bigger affect in some NICs that pays some cost
> to receive the HW calculated hash values.
> 
> User requesting 'DEV_RX_OFFLOAD_RSS_HASH' offload should enable driver to update
> 'mbuf::hash::rss' and 'PKT_RX_RSS_HASH' in 'mbuf::ol_flags' addition to RSS feature.

I hope this is clear now, and back to my first point if you think above behavior
is not clear in the doc or API comments, can you please send a patch to clarify
it better?

Thanks,
ferruh

> 
>>
>> Can I understand it this way, If the user does not apply this rss hash, 
>> does it mean that rss cannot be used to improve performance, even if the 
>> hardware has calculated the rss hash result?
> 
> If 'DEV_RX_OFFLOAD_RSS_HASH' is NOT set, application won't able to receive the
> HW calculated hash values.
> 
>>> We know above because we were involved in development of it, if this is not
>>> clean for third party, can you please amend your patch to clarify above behavior?
>>>
>>> Thanks,
>>> ferruh
>>>
>>> .
>>>
>>
>