[dpdk-dev] [PATCH] i40e: Fix eth_i40e_dev_init sequence on ThunderX

[dpdk-dev] [PATCH] i40e: Fix eth_i40e_dev_init sequence on ThunderX

[Satha Rao]

i40e_asq_send_command: rd32 & wr32 under ThunderX gives unpredictable
                       results. To solve this include rte memory barriers

Signed-off-by: Satha Rao <skoteshwar@caviumnetworks.com>
---
 drivers/net/i40e/base/i40e_adminq.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/base/i40e_adminq.c b/drivers/net/i40e/base/i40e_adminq.c
index 0d3a83f..1038a95 100644
--- a/drivers/net/i40e/base/i40e_adminq.c
+++ b/drivers/net/i40e/base/i40e_adminq.c
@@ -832,6 +832,7 @@ enum i40e_status_code i40e_asq_send_command(struct i40e_hw *hw,
 	}
 
 	val = rd32(hw, hw->aq.asq.head);
+	rte_rmb();
 	if (val >= hw->aq.num_asq_entries) {
 		i40e_debug(hw, I40E_DEBUG_AQ_MESSAGE,
 			   "AQTX: head overrun at %d\n", val);
@@ -929,8 +930,10 @@ enum i40e_status_code i40e_asq_send_command(struct i40e_hw *hw,
 	(hw->aq.asq.next_to_use)++;
 	if (hw->aq.asq.next_to_use == hw->aq.asq.count)
 		hw->aq.asq.next_to_use = 0;
-	if (!details->postpone)
+	if (!details->postpone) {
 		wr32(hw, hw->aq.asq.tail, hw->aq.asq.next_to_use);
+		rte_wmb();
+	}
 
 	/* if cmd_details are not defined or async flag is not set,
 	 * we need to wait for desc write back
-- 
2.7.4

Re: [dpdk-dev] [PATCH] i40e: Fix eth_i40e_dev_init sequence on ThunderX

[Jerin Jacob]

On Thu, Nov 10, 2016 at 04:04:27AM -0800, Satha Rao wrote:
> i40e_asq_send_command: rd32 & wr32 under ThunderX gives unpredictable
>                        results. To solve this include rte memory barriers
> 
> Signed-off-by: Satha Rao <skoteshwar@caviumnetworks.com>
> ---
>  drivers/net/i40e/base/i40e_adminq.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/i40e/base/i40e_adminq.c b/drivers/net/i40e/base/i40e_adminq.c
> index 0d3a83f..1038a95 100644
> --- a/drivers/net/i40e/base/i40e_adminq.c
> +++ b/drivers/net/i40e/base/i40e_adminq.c
> @@ -832,6 +832,7 @@ enum i40e_status_code i40e_asq_send_command(struct i40e_hw *hw,
>  	}
>  
>  	val = rd32(hw, hw->aq.asq.head);
> +	rte_rmb();

use rte_smp_rmb() variant to avoid performance regression on x86

>  	if (val >= hw->aq.num_asq_entries) {
>  		i40e_debug(hw, I40E_DEBUG_AQ_MESSAGE,
>  			   "AQTX: head overrun at %d\n", val);
> @@ -929,8 +930,10 @@ enum i40e_status_code i40e_asq_send_command(struct i40e_hw *hw,
>  	(hw->aq.asq.next_to_use)++;
>  	if (hw->aq.asq.next_to_use == hw->aq.asq.count)
>  		hw->aq.asq.next_to_use = 0;
> -	if (!details->postpone)
> +	if (!details->postpone) {
>  		wr32(hw, hw->aq.asq.tail, hw->aq.asq.next_to_use);
> +		rte_wmb();

ditto

> +	}
>  
>  	/* if cmd_details are not defined or async flag is not set,
>  	 * we need to wait for desc write back
> -- 
> 2.7.4
> 

Re: [dpdk-dev] [PATCH] i40e: Fix eth_i40e_dev_init sequence on ThunderX

[Zhang, Helin]

Hi Rao

The existing rule is to try best not modify source files in the folder of base/ for each PMD. The exceptions are *_osdep.h or *_osdep.c.
So for your case, please redefine the rd32/wr32 in i40e_osdep.h. (with condition compiles for ARM?, as other CPU structures may not need that.)

Regards,
Helin

> -----Original Message-----
> From: Satha Rao [mailto:skoteshwar@caviumnetworks.com]
> Sent: Thursday, November 10, 2016 8:04 PM
> To: Zhang, Helin
> Cc: Wu, Jingjing; jerin.jacob@caviumnetworks.com; jianbo.liu@linaro.org;
> dev@dpdk.org; Satha Rao
> Subject: [PATCH] i40e: Fix eth_i40e_dev_init sequence on ThunderX
> 
> i40e_asq_send_command: rd32 & wr32 under ThunderX gives unpredictable
>                        results. To solve this include rte memory barriers
> 
> Signed-off-by: Satha Rao <skoteshwar@caviumnetworks.com>
> ---
>  drivers/net/i40e/base/i40e_adminq.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/i40e/base/i40e_adminq.c
> b/drivers/net/i40e/base/i40e_adminq.c
> index 0d3a83f..1038a95 100644
> --- a/drivers/net/i40e/base/i40e_adminq.c
> +++ b/drivers/net/i40e/base/i40e_adminq.c
> @@ -832,6 +832,7 @@ enum i40e_status_code
> i40e_asq_send_command(struct i40e_hw *hw,
>  	}
> 
>  	val = rd32(hw, hw->aq.asq.head);
> +	rte_rmb();
>  	if (val >= hw->aq.num_asq_entries) {
>  		i40e_debug(hw, I40E_DEBUG_AQ_MESSAGE,
>  			   "AQTX: head overrun at %d\n", val); @@ -929,8
> +930,10 @@ enum i40e_status_code i40e_asq_send_command(struct
> i40e_hw *hw,
>  	(hw->aq.asq.next_to_use)++;
>  	if (hw->aq.asq.next_to_use == hw->aq.asq.count)
>  		hw->aq.asq.next_to_use = 0;
> -	if (!details->postpone)
> +	if (!details->postpone) {
>  		wr32(hw, hw->aq.asq.tail, hw->aq.asq.next_to_use);
> +		rte_wmb();
> +	}
> 
>  	/* if cmd_details are not defined or async flag is not set,
>  	 * we need to wait for desc write back
> --
> 2.7.4