RE: [EXT] [PATCH 1/8] app/test-crypto-perf: improve dequeue logic

[Gagandeep Singh <G.Singh@nxp.com>]

Hi,

> -----Original Message-----
> From: Anoob Joseph <anoobj@marvell.com>
> Sent: Monday, May 16, 2022 12:57 PM
> To: Gagandeep Singh <G.Singh@nxp.com>
> Cc: Akhil Goyal <gakhil@marvell.com>; dev@dpdk.org; Hemant Agrawal
> <hemant.agrawal@nxp.com>
> Subject: RE: [EXT] [PATCH 1/8] app/test-crypto-perf: improve dequeue logic
> 
> Hi Gagandeep,
> 
> Please see inline.
> 
> Thanks,
> Anoob
> 
> > -----Original Message-----
> > From: Gagandeep Singh <G.Singh@nxp.com>
> > Sent: Monday, May 16, 2022 12:44 PM
> > To: Akhil Goyal <gakhil@marvell.com>; dev@dpdk.org; Hemant Agrawal
> > <hemant.agrawal@nxp.com>
> > Cc: Anoob Joseph <anoobj@marvell.com>
> > Subject: RE: [EXT] [PATCH 1/8] app/test-crypto-perf: improve dequeue
> > logic
> >
> > Hi
> >
> > > -----Original Message-----
> > > From: Akhil Goyal <gakhil@marvell.com>
> > > Sent: Friday, May 13, 2022 3:17 PM
> > > To: Gagandeep Singh <G.Singh@nxp.com>; dev@dpdk.org; Hemant Agrawal
> > > <hemant.agrawal@nxp.com>
> > > Cc: Anoob Joseph <anoobj@marvell.com>
> > > Subject: RE: [EXT] [PATCH 1/8] app/test-crypto-perf: improve dequeue
> > > logic
> > >
> > > Hi Gagan,
> > > > Issue more dequeue commands if the gap between enqueued and
> > > > dequeued packets is more than burst size *8
> > > >
> > > > Signed-off-by: Gagandeep Singh <g.singh@nxp.com>
> > > > ---
> > > Why is this change required? What gain are we getting?
> > > I see a performance drop due to this patch.
> >
> > Issue is, in case if security engine/driver is slow in processing the
> > Jobs especially for larger packet sizes then in that case application
> > will keep enqueuing packets with higher rate than dequeue which may results
> in buffer pool exhaustion.
> > Application has option to increase pool size but that may not be
> > Helpful for the platforms those with memory constraints.
> 
> [Anoob] Can you elaborate the issue that you are hitting?
> 
> >
> > We can work on limiting the enqueue side instead of keeping the
> > dequeue to avoid any performance drop due to any empty dequeue.
> 
> [Anoob] Shouldn't PMD take care of limiting the enqueue side? Application can
> specify the desired queue depth and when application enqueues beyond that,
> enqueue API can return 0. Wouldn't that good enough?

Agree. Unfortunately, Currently In the NXP platform drivers like dpaa_sec, caam_jr
not using the API's given queue depth. This is what I will discuss internally to support
this in the PMDs itself to limit the enqueue, if we can do so without impacting
the NXP's PMDs performance.

> >
> > Dropping this patch from this series. I will update the logic and will
> > try to send as separate patch.
> >
> > >
> > > >  app/test-crypto-perf/cperf_test_throughput.c | 42
> > > > +++++++++++---------
> > > >  1 file changed, 23 insertions(+), 19 deletions(-)
> > > >
> > > > diff --git a/app/test-crypto-perf/cperf_test_throughput.c
> > > > b/app/test-crypto- perf/cperf_test_throughput.c index
> > > > cecf30e470..5cd8919c91 100644
> > > > --- a/app/test-crypto-perf/cperf_test_throughput.c
> > > > +++ b/app/test-crypto-perf/cperf_test_throughput.c
> > > > @@ -223,26 +223,30 @@ cperf_throughput_test_runner(void *test_ctx)
> > > >  			ops_unused = burst_size - ops_enqd;
> > > >  			ops_enqd_total += ops_enqd;
> > > >
> > > > -
> > > >  			/* Dequeue processed burst of ops from crypto device
> > */
> > > > -			ops_deqd = rte_cryptodev_dequeue_burst(ctx->dev_id,
> > > > ctx->qp_id,
> > > > -					ops_processed, test_burst_size);
> > > > -
> > > > -			if (likely(ops_deqd))  {
> > > > -				/* Free crypto ops so they can be reused. */
> > > > -				rte_mempool_put_bulk(ctx->pool,
> > > > -						(void **)ops_processed,
> > > > ops_deqd);
> > > > -
> > > > -				ops_deqd_total += ops_deqd;
> > > > -			} else {
> > > > -				/**
> > > > -				 * Count dequeue polls which didn't return any
> > > > -				 * processed operations. This statistic is mainly
> > > > -				 * relevant to hw accelerators.
> > > > -				 */
> > > > -				ops_deqd_failed++;
> > > > -			}
> > > > -
> > > > +			do {
> > > > +				ops_deqd = rte_cryptodev_dequeue_burst(
> > > > +						ctx->dev_id, ctx->qp_id,
> > > > +						ops_processed,
> > > > test_burst_size);
> > > > +
> > > > +				if (likely(ops_deqd))  {
> > > > +					/* Free crypto ops for reuse */
> > > > +					rte_mempool_put_bulk(ctx->pool,
> > > > +							(void
> > > > **)ops_processed,
> > > > +							ops_deqd);
> > > > +
> > > > +					ops_deqd_total += ops_deqd;
> > > > +				} else {
> > > > +					/**
> > > > +					 * Count dequeue polls which didn't
> > > > +					 * return any processed operations.
> > > > +					 * This statistic is mainly relevant
> > > > +					 * to hw accelerators.
> > > > +					 */
> > > > +					ops_deqd_failed++;
> > > > +				}
> > > > +			} while (ops_enqd_total - ops_deqd_total >
> > > > +					test_burst_size * 8);
> > > >  		}
> > > >
> > > >  		/* Dequeue any operations still in the crypto device */
> > > > --
> > > > 2.25.1