[dpdk-users] Query for QAT software running in SR-IOV mode at VM

[dpdk-users] Query for QAT software running in SR-IOV mode at VM

[Pankaj Joshi]

Hello Team,

I have followed the steps given in section 10.4 in below link :



http://dpdk.org/doc/guides/cryptodevs/qat.html


<http://dpdk.org/doc/guides/cryptodevs/qat.html>


<http://dpdk.org/doc/guides/cryptodevs/qat.html>

<http://dpdk.org/doc/guides/cryptodevs/qat.html>

1.          Export ICP_WITHOUT_IOMMU=1, and install the QAT software in
host ( SR-IOV mode ).

2.         Checked that there are 32 VF devices are there.

3.        Launched VM and add 1 PCI device to the VM.

4.       1 device map to 1 /etc/dh895xcc_qa_dev0.conf file which is having
2 instance.

5.        Install QAT software in VM m/c with guest mode as explained in
below link.

6.    Running my application, it is able to submit the request to hardware
queue using API,cpaCySymDpEnqueueOp.

7.     But unable to call the registered callback function.

http://www.intel.com/content/dam/www/public/us/en/documents/application-notes/virtualization-technology-with-quickassist-technology-app-note.pdf

How I can debug the issue, or how I enable the logs in QAT software.

Please help here.

Regards,

Pankaj Joshi

Re: [dpdk-users] [dpdk-dev] Query for QAT software running in SR-IOV mode at VM

[Trahe, Fiona]

Hi Pankaj,
See below

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Pankaj Joshi
> Sent: Monday, March 27, 2017 5:20 PM
> To: qat-linux <qat-linux@intel.com>; dev@dpdk.org; users@dpdk.org
> Subject: [dpdk-dev] Query for QAT software running in SR-IOV mode at VM
> 
> Hello Team,
> 
> I have followed the steps given in section 10.4 in below link :
> 
> 
> 
> http://dpdk.org/doc/guides/cryptodevs/qat.html
> 
> 
> <http://dpdk.org/doc/guides/cryptodevs/qat.html>
> 
> 
> <http://dpdk.org/doc/guides/cryptodevs/qat.html>
> 
> <http://dpdk.org/doc/guides/cryptodevs/qat.html>
> 
> 1.          Export ICP_WITHOUT_IOMMU=1, and install the QAT software in
> host ( SR-IOV mode ).
> 
> 2.         Checked that there are 32 VF devices are there.
> 
> 3.        Launched VM and add 1 PCI device to the VM.
> 
> 4.       1 device map to 1 /etc/dh895xcc_qa_dev0.conf file which is having
> 2 instance.
> 
> 5.        Install QAT software in VM m/c with guest mode as explained in
> below link.
> 
> 6.    Running my application, it is able to submit the request to hardware
> queue using API,cpaCySymDpEnqueueOp.
> 
> 7.     But unable to call the registered callback function.
> 
> http://www.intel.com/content/dam/www/public/us/en/documents/applicati
> on-notes/virtualization-technology-with-quickassist-technology-app-note.pdf
> 
> How I can debug the issue, or how I enable the logs in QAT software.
> 
If you're trying to use QAT with DPDK cryptodev API then there's no need for steps 4,5,6,7
above. These are steps for using the QA API, i.e. cpaCyXXX APIs.

Also the ICP_WITHOUT_IOMMU condition is only needed if intending to access the QAT HW from 
the Host. If you intend to access the crypto services on QAT via the VF on a VM, then you can 
use the standard out of the box install for a HOST.

Once the VF device is passed through to the VM, on the VM it can be passed straight through 
to the user-space DPDK QAT PMD without a QAT kernel driver.
i.e. follow the bind commands on http://dpdk.org/doc/guides/cryptodevs/qat.html
to bind the VF to either igb_uio or vfio-pci

Then build your application to use the cryptodev APIs, see 
http://dpdk.org/doc/guides/sample_app_ug/ipsec_secgw.html
for an example of how to do this.
If you just want to get it up and running without a traffic generator, you can use the test app to 
run a sanity check against any of the crypto poll mode drivers, e.g. to run some basic functional tests
against the QAT PMD, e.g.
use dpdk/usertools/dpdk-setup.sh to allocate huge pages
start test app, e.g. test -n1 -l1  
(The test app has moved in a recent release, you should find it at either dpdk/build/app/ or dpdk/build/build/test/test ) 
Run the qat test suite with 
>cryptodev_qat_autotest


> Please help here.
> 
> Regards,
> 
> Pankaj Joshi

Re: [dpdk-users] [dpdk-dev] Query for QAT software running in SR-IOV mode at VM

[Pankaj Joshi]

Hello Fiona,

Thanks for giving response. Below are my some points.

Yes,I am using the cpaCyXXX API's so I need steps 4,5,6,7 here.
Without SR-IOV, with "ICP_WITHOUT_IOMMU" application is running fine.
Here I am trying to run my application on VM with SR-IOV enabled mode.
Also I have pass through VM devices to VM and binded those devices to
vfio-pci as given in section 10.6.

So trying to get response through registered callback function.

Regards,
Pankaj Joshi





On Mon, Mar 27, 2017 at 10:44 PM, Trahe, Fiona <fiona.trahe@intel.com>
wrote:

> Hi Pankaj,
> See below
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Pankaj Joshi
> > Sent: Monday, March 27, 2017 5:20 PM
> > To: qat-linux <qat-linux@intel.com>; dev@dpdk.org; users@dpdk.org
> > Subject: [dpdk-dev] Query for QAT software running in SR-IOV mode at VM
> >
> > Hello Team,
> >
> > I have followed the steps given in section 10.4 in below link :
> >
> >
> >
> > http://dpdk.org/doc/guides/cryptodevs/qat.html
> >
> >
> > <http://dpdk.org/doc/guides/cryptodevs/qat.html>
> >
> >
> > <http://dpdk.org/doc/guides/cryptodevs/qat.html>
> >
> > <http://dpdk.org/doc/guides/cryptodevs/qat.html>
> >
> > 1.          Export ICP_WITHOUT_IOMMU=1, and install the QAT software in
> > host ( SR-IOV mode ).
> >
> > 2.         Checked that there are 32 VF devices are there.
> >
> > 3.        Launched VM and add 1 PCI device to the VM.
> >
> > 4.       1 device map to 1 /etc/dh895xcc_qa_dev0.conf file which is
> having
> > 2 instance.
> >
> > 5.        Install QAT software in VM m/c with guest mode as explained in
> > below link.
> >
> > 6.    Running my application, it is able to submit the request to
> hardware
> > queue using API,cpaCySymDpEnqueueOp.
> >
> > 7.     But unable to call the registered callback function.
> >
> > http://www.intel.com/content/dam/www/public/us/en/documents/applicati
> > on-notes/virtualization-technology-with-quickassist-
> technology-app-note.pdf
> >
> > How I can debug the issue, or how I enable the logs in QAT software.
> >
> If you're trying to use QAT with DPDK cryptodev API then there's no need
> for steps 4,5,6,7
> above. These are steps for using the QA API, i.e. cpaCyXXX APIs.
>
> Also the ICP_WITHOUT_IOMMU condition is only needed if intending to access
> the QAT HW from
> the Host. If you intend to access the crypto services on QAT via the VF on
> a VM, then you can
> use the standard out of the box install for a HOST.
>
> Once the VF device is passed through to the VM, on the VM it can be passed
> straight through
> to the user-space DPDK QAT PMD without a QAT kernel driver.
> i.e. follow the bind commands on http://dpdk.org/doc/guides/
> cryptodevs/qat.html
> to bind the VF to either igb_uio or vfio-pci
>
> Then build your application to use the cryptodev APIs, see
> http://dpdk.org/doc/guides/sample_app_ug/ipsec_secgw.html
> for an example of how to do this.
> If you just want to get it up and running without a traffic generator, you
> can use the test app to
> run a sanity check against any of the crypto poll mode drivers, e.g. to
> run some basic functional tests
> against the QAT PMD, e.g.
> use dpdk/usertools/dpdk-setup.sh to allocate huge pages
> start test app, e.g. test -n1 -l1
> (The test app has moved in a recent release, you should find it at either
> dpdk/build/app/ or dpdk/build/build/test/test )
> Run the qat test suite with
> >cryptodev_qat_autotest
>
>
> > Please help here.
> >
> > Regards,
> >
> > Pankaj Joshi
>

Re: [dpdk-users] [dpdk-dev] Query for QAT software running in SR-IOV mode at VM

[Trahe, Fiona]

Hi Pankaj,


From: Pankaj Joshi [mailto:pankaj.joshi63@gmail.com]
Sent: Tuesday, March 28, 2017 7:02 AM
To: Trahe, Fiona <fiona.trahe@intel.com>
Cc: qat-linux <qat-linux@intel.com>; dev@dpdk.org; users@dpdk.org
Subject: Re: [dpdk-dev] Query for QAT software running in SR-IOV mode at VM

Hello Fiona,
Thanks for giving response. Below are my some points.

Yes,I am using the cpaCyXXX API's so I need steps 4,5,6,7 here.
Without SR-IOV, with "ICP_WITHOUT_IOMMU" application is running fine.
Here I am trying to run my application on VM with SR-IOV enabled mode.
Also I have pass through VM devices to VM and binded those devices to vfio-pci as given in section 10.6.
So trying to get response through registered callback function.
Regards,
Pankaj Joshi

If you want to continue using the cpa APIs, then you don’t need DPDK at all, as you can’t access those crypto APIs through DPDK. You should just follow the instructions in the QAT driver you got from 01.org and don’t use the instructions from dpdk.org.

If on the other hand you want to use DPDK for crypto, then you need to change your application to use the DPDK cryptodev APIs.

The QuickAssist device can be accessed through either method, but they are 2 different user-space drivers with 2 different API sets.

Regards,
Fiona




On Mon, Mar 27, 2017 at 10:44 PM, Trahe, Fiona <fiona.trahe@intel.com<mailto:fiona.trahe@intel.com>> wrote:
Hi Pankaj,
See below

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of Pankaj Joshi
> Sent: Monday, March 27, 2017 5:20 PM
> To: qat-linux <qat-linux@intel.com<mailto:qat-linux@intel.com>>; dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>
> Subject: [dpdk-dev] Query for QAT software running in SR-IOV mode at VM
>
> Hello Team,
>
> I have followed the steps given in section 10.4 in below link :
>
>
>
> http://dpdk.org/doc/guides/cryptodevs/qat.html
>
>
> <http://dpdk.org/doc/guides/cryptodevs/qat.html>
>
>
> <http://dpdk.org/doc/guides/cryptodevs/qat.html>
>
> <http://dpdk.org/doc/guides/cryptodevs/qat.html>
>
> 1.          Export ICP_WITHOUT_IOMMU=1, and install the QAT software in
> host ( SR-IOV mode ).
>
> 2.         Checked that there are 32 VF devices are there.
>
> 3.        Launched VM and add 1 PCI device to the VM.
>
> 4.       1 device map to 1 /etc/dh895xcc_qa_dev0.conf file which is having
> 2 instance.
>
> 5.        Install QAT software in VM m/c with guest mode as explained in
> below link.
>
> 6.    Running my application, it is able to submit the request to hardware
> queue using API,cpaCySymDpEnqueueOp.
>
> 7.     But unable to call the registered callback function.
>
> http://www.intel.com/content/dam/www/public/us/en/documents/applicati
> on-notes/virtualization-technology-with-quickassist-technology-app-note.pdf
>
> How I can debug the issue, or how I enable the logs in QAT software.
>
If you're trying to use QAT with DPDK cryptodev API then there's no need for steps 4,5,6,7
above. These are steps for using the QA API, i.e. cpaCyXXX APIs.

Also the ICP_WITHOUT_IOMMU condition is only needed if intending to access the QAT HW from
the Host. If you intend to access the crypto services on QAT via the VF on a VM, then you can
use the standard out of the box install for a HOST.

Once the VF device is passed through to the VM, on the VM it can be passed straight through
to the user-space DPDK QAT PMD without a QAT kernel driver.
i.e. follow the bind commands on http://dpdk.org/doc/guides/cryptodevs/qat.html
to bind the VF to either igb_uio or vfio-pci

Then build your application to use the cryptodev APIs, see
http://dpdk.org/doc/guides/sample_app_ug/ipsec_secgw.html
for an example of how to do this.
If you just want to get it up and running without a traffic generator, you can use the test app to
run a sanity check against any of the crypto poll mode drivers, e.g. to run some basic functional tests
against the QAT PMD, e.g.
use dpdk/usertools/dpdk-setup.sh to allocate huge pages
start test app, e.g. test -n1 -l1
(The test app has moved in a recent release, you should find it at either dpdk/build/app/ or dpdk/build/build/test/test )
Run the qat test suite with
>cryptodev_qat_autotest


> Please help here.
>
> Regards,
>
> Pankaj Joshi