Re: [dpdk-dev] [PATCH] event/octeontx2: use wfe while waiting for head

["Gavin Hu (Arm Technology China)" <Gavin.Hu@arm.com>]

Hi Pavan,

> -----Original Message-----
> From: pbhagavatula@marvell.com <pbhagavatula@marvell.com>
> Sent: Thursday, October 24, 2019 12:13 AM
> To: Gavin Hu (Arm Technology China) <Gavin.Hu@arm.com>;
> jerinj@marvell.com; Pavan Nikhilesh <pbhagavatula@marvell.com>
> Cc: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH] event/octeontx2: use wfe while waiting for
> head
> 
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Use wfe to save power while waiting for tag to become head.
> 
> SSO signals EVENTI to allow cores to exit from wfe when they
> are waiting for specific operations in which one of them is
> setting HEAD bit in GWS_TAG.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> ---
>  drivers/event/octeontx2/otx2_worker.h | 30 ++++++++++++++++++++++++--
> -
>  1 file changed, 27 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/event/octeontx2/otx2_worker.h
> b/drivers/event/octeontx2/otx2_worker.h
> index 4e971f27c..7a55caca5 100644
> --- a/drivers/event/octeontx2/otx2_worker.h
> +++ b/drivers/event/octeontx2/otx2_worker.h
> @@ -226,10 +226,34 @@ otx2_ssogws_swtag_wait(struct otx2_ssogws *ws)
>  }
> 
>  static __rte_always_inline void
> -otx2_ssogws_head_wait(struct otx2_ssogws *ws, const uint8_t wait_flag)
> +otx2_ssogws_head_wait(struct otx2_ssogws *ws)
>  {
> -	while (wait_flag && !(otx2_read64(ws->tag_op) & BIT_ULL(35)))
> +#ifdef RTE_ARCH_ARM64
> +	uint64_t tag;
> +
> +	asm volatile (
> +			"	ldr %[tag], [%[tag_op]]		\n"
"ldxr" should be used, exclusive-load is required to "monitor" the location, then a write to the location will cause clear of the exclusive monitor, thus a wake up event is generated implicitly.
You can find more explanation is here:
http://inbox.dpdk.org/dev/AM0PR08MB5363F9D1BA158B66B803EA068F6B0@AM0PR08MB5363.eurprd08.prod.outlook.com/ 
/Gavin
> +			"	tbnz %[tag], 35, done%=		\n"
> +			"	sevl				\n"
> +			"rty%=:	wfe				\n"
> +			"	ldr %[tag], [%[tag_op]]		\n"
> +			"	tbz %[tag], 35, rty%=		\n"
> +			"done%=:				\n"
> +			: [tag] "=&r" (tag)
> +			: [tag_op] "r" (ws->tag_op)
> +			);
> +#else
> +	/* Wait for the HEAD to be set */
> +	while (!(otx2_read64(ws->tag_op) & BIT_ULL(35)))
>  		;
> +#endif
> +}
> +
> +static __rte_always_inline void
> +otx2_ssogws_order(struct otx2_ssogws *ws, const uint8_t wait_flag)
> +{
> +	if (wait_flag)
> +		otx2_ssogws_head_wait(ws);
> 
>  	rte_cio_wmb();
What ordering does this barrier try to keep?  If there is a write then wait for kind of response, should this barrier move before  otx2_ssogws_head_wait?
/Gavin
>  }
> @@ -258,7 +282,7 @@ otx2_ssogws_event_tx(struct otx2_ssogws *ws,
> struct rte_event ev[],
> 
>  	/* Perform header writes before barrier for TSO */
>  	otx2_nix_xmit_prepare_tso(m, flags);
> -	otx2_ssogws_head_wait(ws, !ev->sched_type);
> +	otx2_ssogws_order(ws, !ev->sched_type);
>  	otx2_ssogws_prepare_pkt(txq, m, cmd, flags);
> 
>  	if (flags & NIX_TX_MULTI_SEG_F) {
> --
> 2.17.1