From: "Power, Ciara" <ciara.power@intel.com>
To: Volodymyr Fialko <vfialko@marvell.com>,
"dev@dpdk.org" <dev@dpdk.org>, Akhil Goyal <gakhil@marvell.com>,
Fan Zhang <fanzhang.oss@gmail.com>
Cc: "jerinj@marvell.com" <jerinj@marvell.com>,
"anoobj@marvell.com" <anoobj@marvell.com>,
"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
"Ji, Kai" <kai.ji@intel.com>
Subject: RE: [PATCH v2] test: add cryptodev crosscheck suite
Date: Wed, 8 Mar 2023 18:03:37 +0000 [thread overview]
Message-ID: <DS0PR11MB7631122AC625E3D9EAADA00FE6B49@DS0PR11MB7631.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20230209123402.1993179-1-vfialko@marvell.com>
> -----Original Message-----
> From: Power, Ciara
> Sent: Wednesday 8 March 2023 17:31
> To: Volodymyr Fialko <vfialko@marvell.com>; dev@dpdk.org; Akhil Goyal
> <gakhil@marvell.com>; Fan Zhang <fanzhang.oss@gmail.com>
> Cc: jerinj@marvell.com; anoobj@marvell.com; hemant.agrawal@nxp.com; Ji,
> Kai <kai.ji@intel.com>
> Subject: RE: [PATCH v2] test: add cryptodev crosscheck suite
>
> Hi Volodymyr,
>
>
> > -----Original Message-----
> > From: Volodymyr Fialko <vfialko@marvell.com>
> > Sent: Thursday 9 February 2023 12:34
> > To: dev@dpdk.org; Akhil Goyal <gakhil@marvell.com>; Fan Zhang
> > <fanzhang.oss@gmail.com>
> > Cc: jerinj@marvell.com; anoobj@marvell.com; hemant.agrawal@nxp.com;
> > Ji, Kai <kai.ji@intel.com>; Power, Ciara <ciara.power@intel.com>;
> > Volodymyr Fialko <vfialko@marvell.com>
> > Subject: [PATCH v2] test: add cryptodev crosscheck suite
> >
> > Add a validation test suite that helps in verifying that the output
> > generated by two different cryptodevs match for a wide range of input
> > parameter combinations.
> >
> > Crypto autotest performs a comprehensive testing of the cryptodev but
> > since it performs verification by comparing against known vectors, the
> > extend to which various parameters (like packet size) can be tested is
> > limited. This test suite attempts to simulate various cases by running
> > same test case on different cryptodevs and compares the output
> > generated. The test suite relies on capabilities to determine the
> > combinations of tests to be attempted.
> >
> > A typical use case would be to compare outputs generated from a
> > standard driver such as openSSL PMD and a new cryptodev PMD. This test
> > suite is to compliment the testing coverage that crypto autotest provides.
> >
> > Currently supported symmetric xforms(cipher, auth, aead) without
> chaining.
> >
> > Example command:
> > DPDK_TEST=cryptodev_crosscheck ./dpdk-test \
> > -a <cryptodev> --vdev "crypto_openssl"
> >
> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
> > ---
> > V2:
> > - Updated commit message.
> >
> <snip>
> > +
> > +static void
> > +capabilities_inspect(void)
> > +{
> > + struct rte_cryptodev_sym_capability_idx
> > cap_indexes[CRYPTO_ALGOS_LEN], *cap_idx;
> > + struct crypto_testsuite_params *ts_params = &testsuite_params;
> > + const struct rte_cryptodev_symmetric_capability *sym_capa;
> > + struct rte_cryptodev_symmetric_capability *common_capa;
> > + uint32_t algo, i, dev_id, caps_idx;
> > +
> > + caps_idx = 0;
> > + /* Create capability idx for known algorithms*/
> > + for (algo = 1; algo <= CRYPTO_AUTH_MAX_IDX; algo++) {
> > + cap_idx = &cap_indexes[caps_idx++];
> > + cap_idx->type = RTE_CRYPTO_SYM_XFORM_AUTH;
> > + cap_idx->algo.auth = algo;
> > + }
> > + for (algo = 1; algo <= CRYPTO_CIPHER_MAX_IDX; algo++) {
> > + cap_idx = &cap_indexes[caps_idx++];
> > + cap_idx->type = RTE_CRYPTO_SYM_XFORM_CIPHER;
> > + cap_idx->algo.cipher = algo;
> > + }
> > + for (algo = 1; algo <= CRYPTO_AEAD_MAX_IDX; algo++) {
> > + cap_idx = &cap_indexes[caps_idx++];
> > + cap_idx->type = RTE_CRYPTO_SYM_XFORM_AEAD;
> > + cap_idx->algo.aead = algo;
> > + }
> > +
> > + for (caps_idx = 0; caps_idx < CRYPTO_ALGOS_LEN; caps_idx++) {
> > + /* Gather common capabilities */
> > + common_capa = &common_symm_capas[caps_idx];
> > + common_capa->xform_type =
> > RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED;
> > + for (i = 0; i < ts_params->valid_dev_count; i++) {
> > + dev_id = ts_params->valid_devs[i];
> > + sym_capa =
> > rte_cryptodev_sym_capability_get(dev_id,
> > + &cap_indexes[caps_idx]);
> > + if (sym_capa == NULL) {
> > + /* Capability not supported by one of devs,
> > mark and skip */
> > + goto next_algo;
> > + }
> > +
> > + if (common_capa->xform_type ==
> > RTE_CRYPTO_SYM_XFORM_NOT_SPECIFIED) {
> > + /* First time initialization, copy data, go to
> > next device */
> > + *common_capa = *sym_capa;
> > + continue;
> > + }
> [CP]
>
> This function - from what I understand after review, is looping through all
> algorithms in DPDK crypto, and then checking if each is supported by each
> device?
> Could we instead just take the capabilities list from one device as the starting
> point - That will be the max list of capabilities, they will only get knocked out
> if not supported on the 2nd/3rd device etc.
>
>
> <snip>
>
> Some trials I did with this patch showed some issues.
>
> 1. For QAT + AESNI_MB I get all skipped tests. I would have expected some
> common algs here.
> + ------------------------------------------------------- +
> + Test Suite Summary : Crosscheck Unit Test Suite + -----------------------------
> -------------------------- + + Algo AUTH 'null' : 0/0 passed, 0/0 skipped, 0/0
> failed, 0/0 unsupported + Algo AUTH 'aes-cbc-mac' : 0/0 passed, 0/0
> skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'aes-cmac' : 0/0 passed,
> 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'aes-gmac' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'aes-xcbc-
> mac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH
> 'kasumi-f9' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> AUTH 'md5' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> AUTH 'md5-hmac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo AUTH 'sha1' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo AUTH 'sha1-hmac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0
> unsupported + Algo AUTH 'sha2-224' : 0/0 passed, 0/0 skipped, 0/0 failed,
> 0/0 unsupported + Algo AUTH 'sha2-224-hmac' : 0/0 passed, 0/0 skipped, 0/0
> failed, 0/0 unsupported + Algo AUTH 'sha2-256' : 0/0 passed, 0/0 skipped,
> 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-256-hmac' : 0/0 passed, 0/0
> skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-384' : 0/0 passed,
> 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-384-hmac' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-512' :
> 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha2-512-
> hmac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH
> 'snow3g-uia2' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> AUTH 'zuc-eia3' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> AUTH 'sha3-224' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo AUTH 'sha3-224-hmac' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0
> unsupported + Algo AUTH 'sha3-256' : 0/0 passed, 0/0 skipped, 0/0 failed,
> 0/0 unsupported + Algo AUTH 'sha3-256-hmac' : 0/0 passed, 0/0 skipped, 0/0
> failed, 0/0 unsupported + Algo AUTH 'sha3-384' : 0/0 passed, 0/0 skipped,
> 0/0 failed, 0/0 unsupported + Algo AUTH 'sha3-384-hmac' : 0/0 passed, 0/0
> skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha3-512' : 0/0 passed,
> 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AUTH 'sha3-512-hmac' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER 'null' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER '3des-cbc' :
> 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER '3des-ctr'
> : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER '3des-
> ecb' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER
> 'aes-cbc' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> CIPHER 'aes-ctr' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo
> CIPHER 'aes-ecb' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo CIPHER 'aes-f8' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo CIPHER 'aes-xts' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo CIPHER 'arc4' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> Algo CIPHER 'kasumi-f8' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0
> unsupported + Algo CIPHER 'snow3g-uea2' : 0/0 passed, 0/0 skipped, 0/0
> failed, 0/0 unsupported + Algo CIPHER 'zuc-eea3' : 0/0 passed, 0/0 skipped,
> 0/0 failed, 0/0 unsupported + Algo CIPHER 'des-cbc' : 0/0 passed, 0/0
> skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER 'aes-docsisbpi' : 0/0
> passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo CIPHER 'des-
> docsisbpi' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AEAD
> 'aes-gcm' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported + Algo AEAD
> 'chacha20-poly1305' : 0/0 passed, 0/0 skipped, 0/0 failed, 0/0 unsupported +
> ------------------------------------------------------- +
> + Sub Testsuites Total : 46
> + Sub Testsuites Skipped : 46
> + Sub Testsuites Passed : 0
> + Sub Testsuites Failed : 0
> + ------------------------------------------------------- +
> + Tests Total : 0
> + Tests Skipped : 0
> + Tests Executed : 0
> + Tests Unsupported: 0
> + Tests Passed : 0
> + Tests Failed : 0
> + ------------------------------------------------------- + Test Skipped
>
>
> 2. For 2 openssl devices I get 3 failures, which is curious as they are the same.
> Seems to be a space issue:
> USER1: Operation status 2
> USER1: No space for aad in single mbuf
> EAL: Test assert crosscheck_all_devices line 776 failed: Error occurred during
> processing
> EAL: Test assert crosscheck_with_profile_run line 841 failed: Error occurred
> during encryption
>
>
> Thanks,
> Ciara
[CP]
Comments above are based on V3 version of patch ... replied to the wrong email.
-Ciara
prev parent reply other threads:[~2023-03-08 18:04 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-21 11:08 [PATCH] " Volodymyr Fialko
2023-02-09 12:34 ` [PATCH v2] " Volodymyr Fialko
2023-02-09 14:34 ` [PATCH v3] " Volodymyr Fialko
2023-03-14 13:32 ` [PATCH v4] " Volodymyr Fialko
2023-03-14 13:56 ` [PATCH v5] " Volodymyr Fialko
2023-03-16 9:44 ` Akhil Goyal
2023-03-16 15:46 ` Power, Ciara
2023-03-16 18:35 ` Akhil Goyal
2023-03-08 17:30 ` [PATCH v2] " Power, Ciara
2023-03-14 13:33 ` Volodymyr Fialko
2023-03-08 18:03 ` Power, Ciara [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DS0PR11MB7631122AC625E3D9EAADA00FE6B49@DS0PR11MB7631.namprd11.prod.outlook.com \
--to=ciara.power@intel.com \
--cc=anoobj@marvell.com \
--cc=dev@dpdk.org \
--cc=fanzhang.oss@gmail.com \
--cc=gakhil@marvell.com \
--cc=hemant.agrawal@nxp.com \
--cc=jerinj@marvell.com \
--cc=kai.ji@intel.com \
--cc=vfialko@marvell.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).