[dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Changchun Zhang]

Hi,

 

I have user question on using the QAT device in the DPDK. 

In the real design, after calling enqueuer_burst() on the specified queue pair at one of the lcore, usually which one is usually done?

1.     should we do run-to-completion to call dequeuer_burst() waiting for the device finishing the crypto operation, 

2.     or should we do pipe-line, in which we return right after enqueuer_burst() and release the CPU. And call dequeuer_burst() on other thread function?

Option 1 is more like synchronous and can be seen on all the DPDK crypto examples, while option 2 is asynchronous which I have never seen in any reference design if I missed anything.

 

Thanks,

Alex

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Trahe, Fiona]

Hi Alex,

> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Changchun Zhang
> Sent: Thursday, January 17, 2019 11:01 PM
> To: users@dpdk.org
> Subject: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK
> 
> Hi,
> 
> 
> 
> I have user question on using the QAT device in the DPDK.
> 
> In the real design, after calling enqueuer_burst() on the specified queue pair at one of the lcore,
> usually which one is usually done?
> 
> 1.     should we do run-to-completion to call dequeuer_burst() waiting for the device finishing the
> crypto operation,
> 
> 2.     or should we do pipe-line, in which we return right after enqueuer_burst() and release the CPU.
> And call dequeuer_burst() on other thread function?
> 
> Option 1 is more like synchronous and can be seen on all the DPDK crypto examples, while option 2 is
> asynchronous which I have never seen in any reference design if I missed anything.
[Fiona] 
Option 2 is not possible with QAT - the dequeue must be called in the same thread as the enqueue. This is
optimised without atomics for best performance - if this is a problem let us know. 
However best performance is not quite using option 1 and not a synchronous blocking method. 
If you enqueue and then go straight to dequeue, you're not getting the best advantage from the
cycles freed up by  offloading. 
i.e. best to enqueue a burst, then go do some other work, like maybe collecting more requests for 
next enqueue or other processing, then dequeue. Take and process whatever ops are dequeued - this
will not necessarily match up with the number you've enqueued - depends on how quickly you call the dequeue.
Don't wait until all the enqueued ops are dequeued before enqueuing the next batch.
SO it's asynchronous. But in the same thread.
You'll get best throughput when you keep the input filled up so the device has operations to work on and
regularly dequeue a burst. Dequeuing too often will waste cycles in the overhead calling the API, dequeuing too
slowly will cause the device to back up. Ideally tune for your application to find the sweet spot in
between these 2 extremes.  
 

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Pathak, Pravin]

Hi Alex -

-----Original Message-----
From: users [mailto:users-bounces@dpdk.org] On Behalf Of Trahe, Fiona
Sent: Friday, January 18, 2019 8:14 AM
To: Changchun Zhang <changchun.zhang@oracle.com>; users@dpdk.org
Cc: Trahe, Fiona <fiona.trahe@intel.com>
Subject: Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

Hi Alex,

> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Changchun 
> Zhang
> Sent: Thursday, January 17, 2019 11:01 PM
> To: users@dpdk.org
> Subject: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in 
> DPDK
> 
> Hi,
> 
> 
> 
> I have user question on using the QAT device in the DPDK.
> 
> In the real design, after calling enqueuer_burst() on the specified 
> queue pair at one of the lcore, usually which one is usually done?
> 
> 1.     should we do run-to-completion to call dequeuer_burst() waiting for the device finishing the
> crypto operation,
> 
> 2.     or should we do pipe-line, in which we return right after enqueuer_burst() and release the CPU.
> And call dequeuer_burst() on other thread function?
> 
> Option 1 is more like synchronous and can be seen on all the DPDK 
> crypto examples, while option 2 is asynchronous which I have never seen in any reference design if I missed anything.
[Fiona]
Option 2 is not possible with QAT - the dequeue must be called in the same thread as the enqueue. This is optimised without atomics for best performance - if this is a problem let us know. 
However best performance is not quite using option 1 and not a synchronous blocking method. 
If you enqueue and then go straight to dequeue, you're not getting the best advantage from the cycles freed up by  offloading. 
i.e. best to enqueue a burst, then go do some other work, like maybe collecting more requests for next enqueue or other processing, then dequeue. Take and process whatever ops are dequeued - this will not necessarily match up with the number you've enqueued - depends on how quickly you call the dequeue.
Don't wait until all the enqueued ops are dequeued before enqueuing the next batch.
SO it's asynchronous. But in the same thread.
You'll get best throughput when you keep the input filled up so the device has operations to work on and regularly dequeue a burst. Dequeuing too often will waste cycles in the overhead calling the API, dequeuing too slowly will cause the device to back up. Ideally tune for your application to find the sweet spot in between these 2 extremes.  
 [Pravin]
I faced exact same issue while moving from software crypto to HW. I implemented option Fiona suggested.  
Thread enqueues to crypto engine and goes back to other work. It periodically polls crypto to see if work is finished.
As we have a single thread running, it keeps doing queuing as work arrives and de-queuing as results are ready while in between doing other stuff.
To keep track of packets, I put some ID into crypto operation private data.

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Changchun Zhang]

Hi Fiona and Pathak,

Thanks!
Changchun (Alex)

-----Original Message-----
From: Pathak, Pravin [mailto:pravin.pathak@intel.com] 
Sent: Friday, January 18, 2019 9:29 AM
To: Trahe, Fiona <fiona.trahe@intel.com>; Changchun Zhang <changchun.zhang@oracle.com>; users@dpdk.org
Cc: Trahe, Fiona <fiona.trahe@intel.com>
Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

Hi Alex -

-----Original Message-----
From: users [mailto:users-bounces@dpdk.org] On Behalf Of Trahe, Fiona
Sent: Friday, January 18, 2019 8:14 AM
To: Changchun Zhang <changchun.zhang@oracle.com>; users@dpdk.org
Cc: Trahe, Fiona <fiona.trahe@intel.com>
Subject: Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

Hi Alex,

> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Changchun 
> Zhang
> Sent: Thursday, January 17, 2019 11:01 PM
> To: users@dpdk.org
> Subject: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in 
> DPDK
> 
> Hi,
> 
> 
> 
> I have user question on using the QAT device in the DPDK.
> 
> In the real design, after calling enqueuer_burst() on the specified 
> queue pair at one of the lcore, usually which one is usually done?
> 
> 1.     should we do run-to-completion to call dequeuer_burst() waiting for the device finishing the
> crypto operation,
> 
> 2.     or should we do pipe-line, in which we return right after enqueuer_burst() and release the CPU.
> And call dequeuer_burst() on other thread function?
> 
> Option 1 is more like synchronous and can be seen on all the DPDK 
> crypto examples, while option 2 is asynchronous which I have never seen in any reference design if I missed anything.
[Fiona]
Option 2 is not possible with QAT - the dequeue must be called in the same thread as the enqueue. This is optimised without atomics for best performance - if this is a problem let us know. 
However best performance is not quite using option 1 and not a synchronous blocking method. 
If you enqueue and then go straight to dequeue, you're not getting the best advantage from the cycles freed up by  offloading. 
i.e. best to enqueue a burst, then go do some other work, like maybe collecting more requests for next enqueue or other processing, then dequeue. Take and process whatever ops are dequeued - this will not necessarily match up with the number you've enqueued - depends on how quickly you call the dequeue.
Don't wait until all the enqueued ops are dequeued before enqueuing the next batch.
SO it's asynchronous. But in the same thread.
[changchun] In the same thread, but how about to dequeuer at the beginning of the thread each time, if data presents then processing them, if no data just do other work, and equeue the packets at some time but does not wait.
For example:
While(1)
{
	Nb_ops = dequeuer();
	If(nb_ops > ) 
             {
                 process_dequeued_data();
                 continue;	
             }
              
              Other_work();
              If(ipsec)
                  Enqueuer(); 
}
Does it make sense?

You'll get best throughput when you keep the input filled up so the device has operations to work on and regularly dequeue a burst. Dequeuing too often will waste cycles in the overhead calling the API, dequeuing too slowly will cause the device to back up. Ideally tune for your application to find the sweet spot in between these 2 extremes.  
 [Pravin]
I faced exact same issue while moving from software crypto to HW. I implemented option Fiona suggested.  
Thread enqueues to crypto engine and goes back to other work. It periodically polls crypto to see if work is finished.
As we have a single thread running, it keeps doing queuing as work arrives and de-queuing as results are ready while in between doing other stuff.
To keep track of packets, I put some ID into crypto operation private data.

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Trahe, Fiona]

Hi Alex,
> [changchun] In the same thread, but how about to dequeuer at the beginning of the thread each time,
> if data presents then processing them, if no data just do other work, and equeue the packets at some
> time but does not wait.
> For example:
> While(1)
> {
> 	Nb_ops = dequeuer();
> 	If(nb_ops > )
>              {
>                  process_dequeued_data();
>                  continue;
>              }
> 
>               Other_work();
>               If(ipsec)
>                   Enqueuer();
> }
> Does it make sense?
[Fiona] It can, though on the first loop ro after a queit time youll proably get very few back on first and second dequeue as
It'll be called immediately after the enqueue. Once it gets busy that could be ok though

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Changchun Zhang]

Thanks!
Changchun (Alex)


-----Original Message-----
From: Trahe, Fiona [mailto:fiona.trahe@intel.com] 
Sent: Friday, January 18, 2019 11:26 AM
To: Changchun Zhang <changchun.zhang@oracle.com>; Pathak, Pravin <pravin.pathak@intel.com>; users@dpdk.org
Cc: Trahe, Fiona <fiona.trahe@intel.com>
Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

Hi Alex,
> [changchun] In the same thread, but how about to dequeuer at the 
> beginning of the thread each time, if data presents then processing 
> them, if no data just do other work, and equeue the packets at some time but does not wait.
> For example:
> While(1)
> {
> 	Nb_ops = dequeuer();
> 	If(nb_ops > )
>              {
>                  process_dequeued_data();
>                  continue;
>              }
> 
>               Other_work();
>               If(ipsec)
>                   Enqueuer();
> }
> Does it make sense?
[Fiona] It can, though on the first loop ro after a queit time youll proably get very few back on first and second dequeue as It'll be called immediately after the enqueue. Once it gets busy that could be ok though
[changchun] Thank you Fiona. One more question, as you said, enqueuer/dequeue should be called within the same thread. Why? Is it because the other thread(lcore 1) cannot dequeuer the processed data from other thread(lcore 2)? But as the cryptograph device lib doc says, "it is howerver possible to use a different logical core to dequeuer an operation on a queue pair from the logical core which it was enqueued on". Looking forward to more details.

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Trahe, Fiona]

Hi Alex,

> -----Original Message-----
> From: Changchun Zhang [mailto:changchun.zhang@oracle.com]
> Sent: Friday, January 18, 2019 4:42 PM
> To: Trahe, Fiona <fiona.trahe@intel.com>; Pathak, Pravin <pravin.pathak@intel.com>;
> users@dpdk.org
> Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK
> 
> 
> 
> Thanks!
> Changchun (Alex)
> 
> 
> -----Original Message-----
> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> Sent: Friday, January 18, 2019 11:26 AM
> To: Changchun Zhang <changchun.zhang@oracle.com>; Pathak, Pravin <pravin.pathak@intel.com>;
> users@dpdk.org
> Cc: Trahe, Fiona <fiona.trahe@intel.com>
> Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK
> 
> Hi Alex,
> > [changchun] In the same thread, but how about to dequeuer at the
> > beginning of the thread each time, if data presents then processing
> > them, if no data just do other work, and equeue the packets at some time but does not wait.
> > For example:
> > While(1)
> > {
> > 	Nb_ops = dequeuer();
> > 	If(nb_ops > )
> >              {
> >                  process_dequeued_data();
> >                  continue;
> >              }
> >
> >               Other_work();
> >               If(ipsec)
> >                   Enqueuer();
> > }
> > Does it make sense?
> [Fiona] It can, though on the first loop ro after a queit time youll proably get very few back on first and
> second dequeue as It'll be called immediately after the enqueue. Once it gets busy that could be ok
> though
> [changchun] Thank you Fiona. One more question, as you said, enqueuer/dequeue should be called
> within the same thread. Why? Is it because the other thread(lcore 1) cannot dequeuer the processed
> data from other thread(lcore 2)? But as the cryptograph device lib doc says, "it is howerver possible to
> use a different logical core to dequeuer an operation on a queue pair from the logical core which it
> was enqueued on". Looking forward to more details.
It's because the QAT inflight counter would be incremented and decremented by both threads so would need to be an atomic. It used to be atomic until 17.11 release but we got a good reduction in offload
cycle-count by replacing this with a normal variable and as all the feedback we got was that
applications were not using in pipeline mode we decided to 
trade off this limitation for the added performance. The limitation is documented here:
http://doc.dpdk.org/guides/cryptodevs/qat.html
You can look at code before 17.11 release to see the difference.

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Changchun Zhang]

Thanks!
Changchun (Alex)


-----Original Message-----
From: Trahe, Fiona [mailto:fiona.trahe@intel.com] 
Sent: Friday, January 18, 2019 11:58 AM
To: Changchun Zhang <changchun.zhang@oracle.com>; Pathak, Pravin <pravin.pathak@intel.com>; users@dpdk.org
Cc: Trahe, Fiona <fiona.trahe@intel.com>
Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

Hi Alex,

> -----Original Message-----
> From: Changchun Zhang [mailto:changchun.zhang@oracle.com]
> Sent: Friday, January 18, 2019 4:42 PM
> To: Trahe, Fiona <fiona.trahe@intel.com>; Pathak, Pravin 
> <pravin.pathak@intel.com>; users@dpdk.org
> Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD 
> in DPDK
> 
> 
> 
> Thanks!
> Changchun (Alex)
> 
> 
> -----Original Message-----
> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> Sent: Friday, January 18, 2019 11:26 AM
> To: Changchun Zhang <changchun.zhang@oracle.com>; Pathak, Pravin 
> <pravin.pathak@intel.com>; users@dpdk.org
> Cc: Trahe, Fiona <fiona.trahe@intel.com>
> Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD 
> in DPDK
> 
> Hi Alex,
> > [changchun] In the same thread, but how about to dequeuer at the 
> > beginning of the thread each time, if data presents then processing 
> > them, if no data just do other work, and equeue the packets at some time but does not wait.
> > For example:
> > While(1)
> > {
> > 	Nb_ops = dequeuer();
> > 	If(nb_ops > )
> >              {
> >                  process_dequeued_data();
> >                  continue;
> >              }
> >
> >               Other_work();
> >               If(ipsec)
> >                   Enqueuer();
> > }
> > Does it make sense?
> [Fiona] It can, though on the first loop ro after a queit time youll 
> proably get very few back on first and second dequeue as It'll be 
> called immediately after the enqueue. Once it gets busy that could be 
> ok though [changchun] Thank you Fiona. One more question, as you said, 
> enqueuer/dequeue should be called within the same thread. Why? Is it 
> because the other thread(lcore 1) cannot dequeuer the processed data 
> from other thread(lcore 2)? But as the cryptograph device lib doc 
> says, "it is howerver possible to use a different logical core to dequeuer an operation on a queue pair from the logical core which it was enqueued on". Looking forward to more details.
It's because the QAT inflight counter would be incremented and decremented by both threads so would need to be an atomic. It used to be atomic until 17.11 release but we got a good reduction in offload cycle-count by replacing this with a normal variable and as all the feedback we got was that applications were not using in pipeline mode we decided to trade off this limitation for the added performance. The limitation is documented here:
https://urldefense.proofpoint.com/v2/url?u=http-3A__doc.dpdk.org_guides_cryptodevs_qat.html&d=DwIFAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=VCO7jqQFACS5pfBnqRHRln4bZ3htFERYZ3PkX0ytpns&m=Bp2X1mxAJ_Eg1kB1t91mgFDUfIrr8Vl-2Nxq_9RLxrI&s=2MPxBDAAKTRMsgE6mrw0cbGX8C4oFcFstJWzpEp8opI&e=
You can look at code before 17.11 release to see the difference.

 [changchun] Many thanks! So from this limitation, we can conclude that Lcore can only dequeue the QAT queue which was enqueued by itself, right. If so, then the Crypto device lib doc may be a little misleading, at least some notes should be put there.

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Trahe, Fiona]

Hi Alex,

>  [changchun] Many thanks! So from this limitation, we can conclude that Lcore can only dequeue the
> QAT queue which was enqueued by itself, right. If so, then the Crypto device lib doc may be a little
> misleading, at least some notes should be put there.
[Fiona] Limitations are generally on the device documentation - it wouldn't make sense to pollute the lib
with the limitations of individual devices. (though I understand it's easy to miss the limitations)
As mentioned before, if this is an important use-case for you we would be interested in hearing about it,
and we could investigate performant ways to remove the limitation.

Re: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

[Changchun Zhang]

HI Fiona,

Thanks!
Changchun (Alex)


-----Original Message-----
From: Trahe, Fiona [mailto:fiona.trahe@intel.com] 
Sent: Friday, January 18, 2019 1:21 PM
To: Changchun Zhang <changchun.zhang@oracle.com>; Pathak, Pravin <pravin.pathak@intel.com>; users@dpdk.org
Cc: Trahe, Fiona <fiona.trahe@intel.com>
Subject: RE: [dpdk-users] Run-to-completion or Pipe-line for QAT PMD in DPDK

Hi Alex,

>  [changchun] Many thanks! So from this limitation, we can conclude 
> that Lcore can only dequeue the QAT queue which was enqueued by 
> itself, right. If so, then the Crypto device lib doc may be a little misleading, at least some notes should be put there.
[Fiona] Limitations are generally on the device documentation - it wouldn't make sense to pollute the lib with the limitations of individual devices. (though I understand it's easy to miss the limitations) As mentioned before, if this is an important use-case for you we would be interested in hearing about it, and we could investigate performant ways to remove the limitation.

 [changchun] Currently we don't see if it is necessary to remove this limitation or not. But we do need to confirm what the relationship between logical core, queue pair, and crypto device. As my understanding, no matter pipe line or current limitation, the QAT accepts the request from the RX queue of a queue pair and after the processing, the data will be put the TX queue on the same queue pair, it is right? Say enqueue data to the RX queue of Queue pair 1, the return data would always be put to the TX queue of Queue pair 1, not possible to other Queue pair's TX queue, right? Let me know if you did not get my question.