Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

["Trahe, Fiona" <fiona.trahe@intel.com>]

Hi Alex,

> -----Original Message-----
> From: Changchun Zhang [mailto:changchun.zhang@oracle.com]
> Sent: Friday, January 18, 2019 4:42 PM
> To: Trahe, Fiona <fiona.trahe@intel.com>; Pathak, Pravin <pravin.pathak@intel.com>;
> users@dpdk.org
> Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK
> 
> 
> 
> Thanks!
> Changchun (Alex)
> 
> 
> -----Original Message-----
> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> Sent: Friday, January 18, 2019 11:26 AM
> To: Changchun Zhang <changchun.zhang@oracle.com>; Pathak, Pravin <pravin.pathak@intel.com>;
> users@dpdk.org
> Cc: Trahe, Fiona <fiona.trahe@intel.com>
> Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK
> 
> Hi Alex,
> > [changchun] In the same thread, but how about to dequeuer at the
> > beginning of the thread each time, if data presents then processing
> > them, if no data just do other work, and equeue the packets at some time but does not wait.
> > For example:
> > While(1)
> > {
> > 	Nb_ops = dequeuer();
> > 	If(nb_ops > )
> >              {
> >                  process_dequeued_data();
> >                  continue;
> >              }
> >
> >               Other_work();
> >               If(ipsec)
> >                   Enqueuer();
> > }
> > Does it make sense?
> [Fiona] It can, though on the first loop ro after a queit time youll proably get very few back on first and
> second dequeue as It'll be called immediately after the enqueue. Once it gets busy that could be ok
> though
> [changchun] Thank you Fiona. One more question, as you said, enqueuer/dequeue should be called
> within the same thread. Why? Is it because the other thread(lcore 1) cannot dequeuer the processed
> data from other thread(lcore 2)? But as the cryptograph device lib doc says, "it is howerver possible to
> use a different logical core to dequeuer an operation on a queue pair from the logical core which it
> was enqueued on". Looking forward to more details.
It's because the QAT inflight counter would be incremented and decremented by both threads so would need to be an atomic. It used to be atomic until 17.11 release but we got a good reduction in offload
cycle-count by replacing this with a normal variable and as all the feedback we got was that
applications were not using in pipeline mode we decided to 
trade off this limitation for the added performance. The limitation is documented here:
http://doc.dpdk.org/guides/cryptodevs/qat.html
You can look at code before 17.11 release to see the difference.