Re: [dpdk-dev] [PATCH 2/3] app/compress-perf: add performance measurement

["Jozwiak, TomaszX" <tomaszx.jozwiak@intel.com>]

Hi Shally,

I'm about of sending V5 of compression-perf tool.

Our performance testing shows that the number of sgls in a chain can be a factor in the performance.
So we want to keep this on the cmd line for the performance tool.
There are alternatives, like setting the input size and segment size to get the num segments desired, but I prefer
to have the option to specify the num segments explicitly.
We'll document that if the max-num-sgl-segs x seg_sz > input size then segments number in the chain will be lower ( to store all the data)
As regards adding the max_nb_segments_per_sgl into the rte_compressdev_info struct we're investigating
another workaround to this limitation in QAT, so will leave this off the API unless some other PMD needs it.
In the meantime we'll document the limitation in QAT.

Please let me know your thoughts.

--
Tomek

> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> Sent: Wednesday, October 17, 2018 6:48 PM
> To: Trahe, Fiona <fiona.trahe@intel.com>; Daly, Lee <lee.daly@intel.com>
> Cc: Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>; dev@dpdk.org;
> akhil.goyal@nxp.com
> Subject: RE: [dpdk-dev] [PATCH 2/3] app/compress-perf: add performance
> measurement
> 
> 
> 
> >-----Original Message-----
> >From: Trahe, Fiona <fiona.trahe@intel.com>
> >Sent: 17 October 2018 22:15
> >To: Verma, Shally <Shally.Verma@cavium.com>; Daly, Lee
> ><lee.daly@intel.com>
> >Cc: Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>; dev@dpdk.org;
> >akhil.goyal@nxp.com; Trahe, Fiona <fiona.trahe@intel.com>
> >Subject: RE: [dpdk-dev] [PATCH 2/3] app/compress-perf: add performance
> >measurement
> >
> >External Email
> >
> >> -----Original Message-----
> >> From: Verma, Shally [mailto:Shally.Verma@cavium.com]
> >> Sent: Wednesday, October 17, 2018 8:43 AM
> >> To: Trahe, Fiona <fiona.trahe@intel.com>; Daly, Lee
> >> <lee.daly@intel.com>
> >> Cc: Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>; dev@dpdk.org;
> >> akhil.goyal@nxp.com
> >> Subject: RE: [dpdk-dev] [PATCH 2/3] app/compress-perf: add
> >> performance measurement
> >>
> >>
> >>
> >> >-----Original Message-----
> >> >From: Trahe, Fiona <fiona.trahe@intel.com>
> >> >Sent: 17 October 2018 20:04
> >> >To: Daly, Lee <lee.daly@intel.com>; Verma, Shally
> >> ><Shally.Verma@cavium.com>
> >> >Cc: Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>; dev@dpdk.org;
> >> >akhil.goyal@nxp.com; Trahe, Fiona
> >> <fiona.trahe@intel.com>
> >> >Subject: RE: [dpdk-dev] [PATCH 2/3] app/compress-perf: add
> >> >performance measurement
> >> >
> >> >External Email
> >> >
> >> >Hi Shally, Lee,
> >> >
> >> >> -----Original Message-----
> >> >> From: Daly, Lee
> >> >> Sent: Monday, October 15, 2018 8:10 AM
> >> >> To: Verma, Shally <Shally.Verma@cavium.com>
> >> >> Cc: Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>; dev@dpdk.org;
> >> >> Trahe, Fiona <fiona.trahe@intel.com>; akhil.goyal@nxp.com
> >> >> Subject: RE: [dpdk-dev] [PATCH 2/3] app/compress-perf: add
> >> >> performance measurement
> >> >>
> >> >> Thanks for your input Shally see comments below.
> >> >>
> >> >>
> >> >> I will be reviewing these changes while Tomasz is out this week.
> >> >>
> >> >> > -----Original Message-----
> >> >> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Verma,
> >> >> > Shally
> >> >> > Sent: Friday, October 12, 2018 11:16 AM
> >> >> > To: Jozwiak, TomaszX <tomaszx.jozwiak@intel.com>;
> dev@dpdk.org;
> >> >> > Trahe, Fiona <fiona.trahe@intel.com>; akhil.goyal@nxp.com; De
> >> >> > Lara Guarch, Pablo <pablo.de.lara.guarch@intel.com>
> >> >> > Cc: De@dpdk.org; Lara@dpdk.org; Guarch@dpdk.org
> >> >> > Subject: Re: [dpdk-dev] [PATCH 2/3] app/compress-perf: add
> >> >> > performance measurement
> >> >> >
> >> >///
> >> >
> >> >> >Also, why do we need --max-num-
> >> >> > sgl-segs as an input option from user? Shouldn't input_sz and
> >> >> >seg_sz  internally decide on num-segs?
> >> >> > Or is it added to serve some other different purpose?
> >> >> Will have to get back to you on this one, seems illogical to get
> >> >> this input from user, But I will have to do further investigation to find if
> there was a different purpose.
> >> >
> >> >[Fiona] Some PMDs have a limit on how many links can be in an sgl
> >> >chain, e.g. in QAT case the PMD allocates a pool of internal
> >> >structures of a suitable size during device initialisation, this is not a hard
> limit but can be configured in .config to give the user control over the
> memory resources allocated.
> >> >This perf-tool max-num-sgl-segs is so the user can pick a value <=
> whatever the PMD's max is.
> >>
> >> Then also, I believe this could be taken care internally by an app.
> >> App can choose convenient number of sgl segs as per PMD capability
> >> and input sz and chunk sz selected by user.
> >> Just my thoughts.
> >[Fiona] Then we'd need to add this capability to the API, e.g. add
> >uint16_t max_nb_segments_per_sgl into the rte_compressdev_info struct.
> >Special case 0 means no limit.
> >We did consider this before, I can't remember why we didn't do it, I think
> it's needed.
> >I'll push an API patch for this in 19.02 and we can remove the
> >--max-num-sgl-segs param from the performance tool and hardcode it in
> the tool in the meantime.
> >Ok?
> Yea. Sounds better.