DPDK usage discussions
 help / color / mirror / Atom feed
From: "Trahe, Fiona" <fiona.trahe@intel.com>
To: "Joshi, Venkatesh" <venkatesh.joshi@hpe.com>,
	"users@dpdk.org" <users@dpdk.org>
Cc: "Trahe, Fiona" <fiona.trahe@intel.com>
Subject: Re: [dpdk-users] DPDK-19.11 : IPSEC-SECGW tests not successful
Date: Wed, 22 Jul 2020 17:42:17 +0000	[thread overview]
Message-ID: <SN6PR11MB28805727BFF97B7B928E5EEDE4790@SN6PR11MB2880.namprd11.prod.outlook.com> (raw)
In-Reply-To: <AT5PR8401MB05452BA989847993E9CFE04BEC790@AT5PR8401MB0545.NAMPRD84.PROD.OUTLOOK.COM>

No, you can use the same bdf on each board - as you say they're not related.
But in your original post you listed only bdf 68:01.0 on board 2 as being bound to vfio.
So whichever bdf you use, you need to make sure it's bound to vfio.


Yes, if the fw counters are not updated, QAT is not being exercised.


> -----Original Message-----
> From: Joshi, Venkatesh <venkatesh.joshi@hpe.com>
> Sent: Wednesday, July 22, 2020 6:24 PM
> To: Trahe, Fiona <fiona.trahe@intel.com>; users@dpdk.org
> Subject: RE: DPDK-19.11 : IPSEC-SECGW tests not successful
> 
> Hi Fiona,
> 
> Could you elaborate on this?
> 
> Here is my understanding of the same - just to run it by you:
> 
> * I have two Intel boards with the C627 PCH (Lewisburg) on each.
> * 67:01.0 - This the VF for the QAT engine with PF 67:00.0.
> * I put the same VF on each board for the whitelist. I assumed that they are two different devices and it
> shouldn't matter what device I chose.
> 
> Do I need to use a different bdf on each board?
> I have no issues in changing to 68:01.0 on board 2. I tried with that too but the results are the same.
> 
> Also, the fw_counters don't get updated while traffic is running. Does this mean that the QAT is not
> exercised?
> 
> # cat /sys/kernel/debug/qat_c6xx_0000\:68\:00.0/fw_counters
> +------------------------------------------------+
> | FW Statistics for Qat Device                   |
> +------------------------------------------------+
> | Firmware Requests [AE  0]:                   1 |
> | Firmware Responses[AE  0]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  1]:                   1 |
> | Firmware Responses[AE  1]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  2]:                   1 |
> | Firmware Responses[AE  2]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  3]:                   1 |
> | Firmware Responses[AE  3]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  4]:                   1 |
> | Firmware Responses[AE  4]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  5]:                   1 |
> | Firmware Responses[AE  5]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  6]:                   1 |
> | Firmware Responses[AE  6]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  7]:                   1 |
> | Firmware Responses[AE  7]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  8]:                   1 |
> | Firmware Responses[AE  8]:                   1 |
> +------------------------------------------------+
> | Firmware Requests [AE  9]:                   1 |
> | Firmware Responses[AE  9]:                   1 |
> +------------------------------------------------+
> 
> 
> Regards,
> Venkatesh
> 
> 
> -----Original Message-----
> From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> Sent: Wednesday, July 22, 2020 9:40 PM
> To: Joshi, Venkatesh <venkatesh.joshi@hpe.com>; users@dpdk.org
> Cc: Trahe, Fiona <fiona.trahe@intel.com>
> Subject: RE: DPDK-19.11 : IPSEC-SECGW tests not successful
> 
> Maybe just a typo - but you're using the same bdf on both boards - should be 68.01.0 on board 2 ?
> 
> > -----Original Message-----
> > From: Joshi, Venkatesh <venkatesh.joshi@hpe.com>
> > Sent: Wednesday, July 22, 2020 5:05 PM
> > To: Trahe, Fiona <fiona.trahe@intel.com>; users@dpdk.org
> > Subject: RE: DPDK-19.11 : IPSEC-SECGW tests not successful
> >
> > Hi Fiona,
> >
> > Here are the command-lines:
> >
> >
> > For the Intel Board 1:
> > -------------------------
> >
> > ./build/ipsec-secgw -l 1 -n 4 -w b3:00.0 -w b3:00.1 -w 67:01.0 -- -P
> > -p 0x3 -u 1 --config="(0,0,1),(1,0,1)" -f ./ep0-intel-board1.cfg
> >
> > ep0-intel-board1.cfg:
> > -------------------------
> > #SP IPv4 rule - for outgoing (to crb-3) sp ipv4 out esp protect 1000
> > pri 5 dst 40.1.1.0/24 sport 0:65535 dport 0:65535
> >
> > #SA rules
> > sa out 1000 cipher_algo aes-128-cbc cipher_key
> > a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:\
> > a0:a0:a0:a0 auth_algo sha1-hmac auth_key
> > a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:\
> > a0:a0:a0:a0:a0:a0:a0:a0 mode ipv4-tunnel src 20.1.1.50 dst 20.1.1.20
> >
> > #SP IPv4 rule - for incoming (to IXIA) sp ipv4 in esp protect 1010 pri
> > 5 dst 20.1.1.50/32 src 20.1.1.20/32 sport 0:65535 dport 0:65535
> >
> > #SA rules
> > sa in 1010 cipher_algo aes-128-cbc cipher_key
> > a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:\
> > a0:a0:a0:a0 auth_algo sha1-hmac auth_key
> > a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:\
> > a0:a0:a0:a0:a0:a0:a0:a0 mode ipv4-tunnel src 20.1.1.20 dst 20.1.1.50
> >
> >
> > #Routing rules
> > rt ipv4 dst 20.1.1.20/32 port 1
> > rt ipv4 dst 30.1.1.10/32 port 0
> >
> >
> > For  the Intel Board 2:
> > -------------------------
> > ./build/ipsec-secgw -l 1 -n 4 -w 17:00.0 -w 17:00.1 -w 67:01.0 -- -P
> > -p 0x3 -u 1 --config="(0,0,1),(1,0,1)" -f ./ep1-intel-board2.cfg
> >
> > ep1-intel-board2.cfg:
> > ----------------------
> > #SP IPv4 rule - for outgoing (from IXIA) sp ipv4 out esp protect 1010
> > pri 1 dst 30.1.1.0/24 sport 0:65535 dport 0:65535
> >
> > #SA rules
> > sa out 1010 cipher_algo aes-128-cbc cipher_key
> > a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:\
> > a0:a0:a0:a0 auth_algo sha1-hmac auth_key
> > a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:a0:\
> > a0:a0:a0:a0:a0:a0:a0:a0 mode ipv4-tunnel src 20.1.1.20 dst 20.1.1.50
> >
> > #Routing rules
> > rt ipv4 dst 20.1.1.50/32 port 1
> >
> >
> >
> >
> >
> >
> > Regards,
> > Venkatesh
> >
> >
> > -----Original Message-----
> > From: Trahe, Fiona [mailto:fiona.trahe@intel.com]
> > Sent: Wednesday, July 22, 2020 5:15 PM
> > To: Joshi, Venkatesh <venkatesh.joshi@hpe.com>; users@dpdk.org
> > Cc: Trahe, Fiona <fiona.trahe@intel.com>
> > Subject: RE: DPDK-19.11 : IPSEC-SECGW tests not successful
> >
> > Have you tried using --log-level=8 (or ="qat,8") on your process commandline?
> > In conjunction with rebuilding with
> > CONFIG_RTE_LOG_DP_LEVEL=RTE_LOG_DEBUG
> > this should show if any cryptodev ops are being sent to QAT PMD.
> >
> > Something else to try would be  - can you run either
> > dpdk-test-crypto-perf or the unit test application - just to validate
> > that the process can run crypto on QAT PMD ok.
> > Your setup for QAT looks ok.
> >
> > Also could you share the command-line you're using for each process please.
> >
> >
> > > -----Original Message-----
> > > From: users <users-bounces@dpdk.org> On Behalf Of Joshi, Venkatesh
> > > Sent: Wednesday, July 22, 2020 8:51 AM
> > > To: users@dpdk.org
> > > Subject: [dpdk-users] DPDK-19.11 : IPSEC-SECGW tests not successful
> > >
> > > Hi,
> > >
> > > I'm not able to successfully run the IPSEC-SECGW tests with
> > > DPDK-19.11. I have followed the guide published at doc.dpdk.org but still not able to get things right.
> > >
> > > Please help me figure out what could be wrong.
> > >
> > > Here are the setup details:
> > >
> > > Network Diagram:
> > > ------------------
> > >                                       XL710                             XL 710
> > >                |---------------------|Port 1        IPSEC Tunnel        Port 1|----------------------|
> > >                |    Intel board 1    |<-------------------------------------->|   Intel board 2      |
> > >                |---------------------|                                        |----------------------|
> > >                  Port 0  ^                                                               ^ Port 0 (XL710)
> > >                  (XL710) |                                                               |
> > >                          |         |--------------------------------|                    |
> > >                           -------->|7          IXIA                4|<-------------------
> > >
> > > |--------------------------------|
> > >
> > > * Linux kernel: 4.14
> > > * DPDK version: 19.11
> > > * All ports are 40G ports (XL710 NICs)
> > > * The intel board is: Intel(R) Xeon(R) Gold 5220 CPU
> > >                       - has a single socket, 18 cores, 2 threads per
> > > core
> > > * The QAT card: C62x
> > >
> > >
> > > Issue:
> > > --------
> > > * Traffic is sent from IXIA Port 4 to IXIA port 7:
> > >    - This is IP/UDP traffic of size 1024 bytes
> > >    - The dst mac is set to the MAC of port 0 of the XL710 NIC of Intel board 2
> > >    - The src ip: 40.1.1.10, dst ip: 30.1.1.10
> > >    - No packets are received on IXIA Port 7
> > >    - On further debugging: Packets are not getting forwarded out of the IPSEC tunnel from Intel board
> 2.
> > >
> > > On Intel board 1:
> > > -------------------
> > > XL710 Port 0: b3:00.0 - bound to vfio-pci
> > > XL710 Port 1: b3:00.1 - bound to vfio-pci QAT VF: 0000:67:01.0
> > > 'Device 37c9' drv=vfio-pci
> > >
> > > Endpoint 0 config file: attached
> > > Command-line and output file: attached
> > >
> > > On Intel board 2:
> > > -------------------
> > > XL710 Port 0: 17:00.0 - bound to vfio-pci
> > > XL710 Port 1: 17:00.1 - bound to vfio-pci QAT VF: 0000:68:01.0
> > > 'Device 37c9' drv=vfio-pci
> > >
> > > Endpoint 1 config file: attached
> > > Command-line and output file: attached
> > >
> > >
> > > DPDK:
> > > -------
> > > - config has the following set:
> > >   CONFIG_RTE_LIBRTE_PMD_QAT=y
> > >   CONFIG_RTE_LIBRTE_PMD_QAT_SYM=y
> > >
> > > QAT driver version: qat1.7.l.4.9.0-00008
> > > -------------------------------------------
> > > Makefile has: ICP_SRIOV_AM=1
> > > ./configure --enable-icp-sriov=host
> > >
> > >
> > > Please let me know if any additional information is required.
> > >
> > >
> > > Regards,
> > > Venkatesh
> > >
> > >
> > > -------------- next part -------------- An embedded and
> > > charset-unspecified text was scrubbed...
> > > Name: intel-board1-cmdline-and-output.txt
> > > URL: <https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__mails.dpdk.org_archives_users_attachments_20200722_3050bab3_attach
> > ment.txt&d=DwIFAg&c=
> > C5b8zRQO1miGmBeVZ2LFWg&r=3IqRcCAftcE8t2M39oG2GFhwi2xpH1_aQ5zAqwEWGSE&m
> > =bVttGFqDb
> > BrrR-0XNypNRdR9q74y5kINxi7lKx2nepg&s=uzWkHTzMJ2sddyvpQCSvFjJ921e7w3ocW
> > Ib_VrRJZxM&e= >
> > > -------------- next part -------------- An embedded and
> > > charset-unspecified text was scrubbed...
> > > Name: intel-board2-cmdline-and-output.txt
> > > URL: <https://urldefense.proofpoint.com/v2/url?u=http-
> > 3A__mails.dpdk.org_archives_users_attachments_20200722_3050bab3_attach
> > ment-
> > 2D0001.txt&d=DwIFAg&c=C5b8zRQO1miGmBeVZ2LFWg&r=3IqRcCAftcE8t2M39oG2GFh
> > wi2xpH1_aQ5z
> > AqwEWGSE&m=bVttGFqDbBrrR-
> > 0XNypNRdR9q74y5kINxi7lKx2nepg&s=i0zTM7t5stYHLY2UVdn9IVYKGZudqptQk6lO_0
> > alxNM&e= >
> 


      reply	other threads:[~2020-07-22 17:42 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-22  7:50 Joshi, Venkatesh
2020-07-22 11:45 ` Trahe, Fiona
2020-07-22 16:05   ` Joshi, Venkatesh
2020-07-22 16:10     ` Trahe, Fiona
2020-07-22 17:23       ` Joshi, Venkatesh
2020-07-22 17:42         ` Trahe, Fiona [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=SN6PR11MB28805727BFF97B7B928E5EEDE4790@SN6PR11MB2880.namprd11.prod.outlook.com \
    --to=fiona.trahe@intel.com \
    --cc=users@dpdk.org \
    --cc=venkatesh.joshi@hpe.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).