Re: [dpdk-dev] [RFC] ethdev: change queue release callback

["Xueming(Steven) Li" <xuemingl@nvidia.com>]

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Xueming(Steven) Li
> Sent: Wednesday, August 11, 2021 8:13 PM
> To: Ferruh Yigit <ferruh.yigit@intel.com>; Singh, Aman Deep <aman.deep.singh@intel.com>; Andrew Rybchenko
> <andrew.rybchenko@oktetlabs.ru>
> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>; NBU-Contact-Thomas Monjalon <thomas@monjalon.net>;
> jerinj@marvell.com
> Subject: Re: [dpdk-dev] [RFC] ethdev: change queue release callback
> 
> 
> 
> > -----Original Message-----
> > From: Ferruh Yigit <ferruh.yigit@intel.com>
> > Sent: Wednesday, August 11, 2021 7:58 PM
> > To: Xueming(Steven) Li <xuemingl@nvidia.com>; Singh, Aman Deep
> > <aman.deep.singh@intel.com>; Andrew Rybchenko
> > <andrew.rybchenko@oktetlabs.ru>
> > Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>;
> > NBU-Contact-Thomas Monjalon <thomas@monjalon.net>; jerinj@marvell.com
> > Subject: Re: [dpdk-dev] [RFC] ethdev: change queue release callback
> >
> > On 8/10/2021 10:07 AM, Xueming(Steven) Li wrote:
> > >
> > >
> > >> -----Original Message-----
> > >> From: Ferruh Yigit <ferruh.yigit@intel.com>
> > >> Sent: Tuesday, August 10, 2021 4:54 PM
> > >> To: Xueming(Steven) Li <xuemingl@nvidia.com>; Singh, Aman Deep
> > >> <aman.deep.singh@intel.com>; Andrew Rybchenko
> > >> <andrew.rybchenko@oktetlabs.ru>
> > >> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>;
> > >> NBU-Contact-Thomas Monjalon <thomas@monjalon.net>
> > >> Subject: Re: [dpdk-dev] [RFC] ethdev: change queue release callback
> > >>
> > >> On 8/10/2021 9:03 AM, Xueming(Steven) Li wrote:
> > >>> Hi Singh and Ferruh,
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: Ferruh Yigit <ferruh.yigit@intel.com>
> > >>>> Sent: Monday, August 9, 2021 11:31 PM
> > >>>> To: Singh, Aman Deep <aman.deep.singh@intel.com>; Andrew
> > >>>> Rybchenko <andrew.rybchenko@oktetlabs.ru>; Xueming(Steven) Li
> > >>>> <xuemingl@nvidia.com>
> > >>>> Cc: dev@dpdk.org; Slava Ovsiienko <viacheslavo@nvidia.com>;
> > >>>> NBU-Contact-Thomas Monjalon <thomas@monjalon.net>
> > >>>> Subject: Re: [dpdk-dev] [RFC] ethdev: change queue release
> > >>>> callback
> > >>>>
> > >>>> On 8/9/2021 3:39 PM, Singh, Aman Deep wrote:
> > >>>>> Hi Xueming,
> > >>>>>
> > >>>>> On 7/28/2021 1:10 PM, Andrew Rybchenko wrote:
> > >>>>>> On 7/27/21 6:41 AM, Xueming Li wrote:
> > >>>>>>> To align with other eth device queue configuration callbacks,
> > >>>>>>> change RX and TX queue release callback API parameter from
> > >>>>>>> queue object to device and queue index.
> > >>>>>>>
> > >>>>>>> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
> > >>>>>>
> > >>>>>> In fact, there is no strong reasons to do it, but I think it is
> > >>>>>> a nice cleanup to use (dev + queue index) on control path.
> > >>>>>>
> > >>>>>> Hopefully it will not result in any regressions.
> > >>>>>
> > >>>>> Combined there are 100+ API's for Rx/Tx queue_release that need
> > >>>>> to be modified for it.
> > >>>>>
> > >>>>> I believe all regression possibilities here will be caught, in
> > >>>>> compilation phase itself.
> > >>>>>
> > >>>>
> > >>>> Same here, it is a good cleanup but there is no strong reason for it.
> > >>>>
> > >>>> Since it is all internal, there is no ABI restriction on the
> > >>>> patch, and v21.11 will be full ABI break patches, to not cause conflicts with this change, what would you think to have it on
> v22.02?
> > >>>
> > >>> This patch is required by shared-rxq feature which ABI broken, target to 21.11.
> > >>
> > >> Why it is required?
> > >
> > > In rx burst function, rxq object is used in data path. For best data performance, it's shared-rxq object in case of shared rxq enabled.
> > > I think eth api defined rxq object for performance as well, specific on data plane.
> > > Hardware saves port info received packet descriptor for my case.
> > > Can't tell which device's queue with this shared rxq object, control
> > > path can't use this shared rxq anymore, have to be specific on
> > dev and queue id.
> > >
> >
> > I have seen shared Rx queue patch, but that just introduces the
> > offload and doesn't have the PMD implementation, so hard to see the dependency, can you please put the pseudocode for PMDs
> for shared-rxq?
> 
> The code is almost ready, I'll upload the PMD part soon.

Seems lots of PMD conflicts to rebase, have to hold it due to other urgent tasks. Here is the overall data structure:
Struct mlx5_rxq_ctrl {
	Bool shared;
	LIST_HEAD owners; // owner rxq(s)
	// datapath resources
}
Struct mlx5_rxq_priv { // rx queue 
	U16 queue_index;
	LIST_ENTRY owner_entry; // membership in shared rxq
	Struct mlx5_rxq_ctrl *ctrl; // save to dev->data->rx_queues[]
	// other per queue resources
}
Rxq_ctrl could be 1:1 mapping to rxq_priv in case of standard rxq, 1:N in case of shared
Shared rxq_ctrl will be released till last owner rxq_priv released.

BTW, v1 posted, please check.

> But firstly, I'll upload v1 patch for this RFC, the make PMD patches depends on this v1 patch.
> 
> > How a queue will know if it is shared or not, during release?
> 
> That's why this RFC want to change callback parameter to device and queue id.
> There is an offloading flag during rxq setup, either in device or in queue configuration.
> PMD driver saves the flag and operate accordingly.
> Ethdev api doesn't need to save this, unless a solid reason.
> 
> >
> > Btw, shared Rx doesn't mention from this dependency in the patch.
> 
> Agree, indeed a strong dependency, thanks!
> 
> >
> > >>
> > >>> I'll do it carefully, fortunately, the change is straightforward.
> > >>>
> > >