From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by dpdk.org (Postfix) with ESMTP id 241C7325F for ; Tue, 11 Dec 2018 11:34:08 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 11 Dec 2018 02:34:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,342,1539673200"; d="scan'208";a="117394687" Received: from silpixa00398673.ir.intel.com (HELO silpixa00398673.ger.corp.intel.com) ([10.237.223.54]) by orsmga002.jf.intel.com with ESMTP; 11 Dec 2018 02:34:06 -0800 From: Fan Zhang To: dev@dpdk.org Cc: akhil.goyal@nxp.com, konstantin.ananyev@intel.com Date: Tue, 11 Dec 2018 10:34:02 +0000 Message-Id: <20181211103402.81914-3-roy.fan.zhang@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20181211103402.81914-1-roy.fan.zhang@intel.com> References: <20181115172427.67607-1-roy.fan.zhang@intel.com> <20181211103402.81914-1-roy.fan.zhang@intel.com> Subject: [dpdk-dev] [PATCH v2 2/2] cryptodev: change symmetric session structure X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Dec 2018 10:34:09 -0000 This patch changes the symmetric session structure of cryptodev. The symmetric session now contains extra information for secure access purposes. The patch also includes the updates to the PMDs, test applications, and examples to fit the change. Signed-off-by: Fan Zhang --- app/test-crypto-perf/cperf.h | 1 + app/test-crypto-perf/cperf_ops.c | 11 +- app/test-crypto-perf/cperf_ops.h | 2 +- app/test-crypto-perf/cperf_test_latency.c | 5 +- app/test-crypto-perf/cperf_test_latency.h | 1 + app/test-crypto-perf/cperf_test_pmd_cyclecount.c | 5 +- app/test-crypto-perf/cperf_test_pmd_cyclecount.h | 1 + app/test-crypto-perf/cperf_test_throughput.c | 5 +- app/test-crypto-perf/cperf_test_throughput.h | 1 + app/test-crypto-perf/cperf_test_verify.c | 5 +- app/test-crypto-perf/cperf_test_verify.h | 1 + app/test-crypto-perf/main.c | 103 +++++++---- doc/guides/rel_notes/release_19_02.rst | 14 ++ drivers/crypto/aesni_gcm/Makefile | 1 + drivers/crypto/aesni_gcm/aesni_gcm_pmd.c | 3 +- drivers/crypto/aesni_gcm/meson.build | 1 + drivers/crypto/aesni_mb/Makefile | 1 + drivers/crypto/aesni_mb/meson.build | 1 + drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c | 3 +- drivers/crypto/armv8/Makefile | 1 + drivers/crypto/armv8/rte_armv8_pmd.c | 3 +- drivers/crypto/kasumi/Makefile | 1 + drivers/crypto/kasumi/meson.build | 1 + drivers/crypto/kasumi/rte_kasumi_pmd.c | 3 +- drivers/crypto/openssl/Makefile | 1 + drivers/crypto/openssl/meson.build | 1 + drivers/crypto/openssl/rte_openssl_pmd.c | 3 +- drivers/crypto/snow3g/Makefile | 1 + drivers/crypto/snow3g/rte_snow3g_pmd.c | 3 +- drivers/crypto/zuc/Makefile | 1 + drivers/crypto/zuc/meson.build | 1 + drivers/crypto/zuc/rte_zuc_pmd.c | 3 +- drivers/net/softnic/rte_eth_softnic_cli.c | 53 +++++- drivers/net/softnic/rte_eth_softnic_cryptodev.c | 50 +++++- drivers/net/softnic/rte_eth_softnic_internals.h | 3 + examples/fips_validation/main.c | 43 ++++- examples/ip_pipeline/cli.c | 49 +++-- examples/ip_pipeline/cryptodev.c | 49 ++++- examples/ip_pipeline/cryptodev.h | 3 + examples/ip_pipeline/examples/flow_crypto.cli | 9 +- examples/ipsec-secgw/ipsec-secgw.c | 48 +++-- examples/ipsec-secgw/ipsec.c | 2 +- examples/ipsec-secgw/ipsec.h | 2 + examples/l2fwd-crypto/main.c | 66 ++++--- examples/vhost_crypto/main.c | 13 +- lib/librte_cryptodev/Makefile | 1 + lib/librte_cryptodev/meson.build | 1 + lib/librte_cryptodev/rte_cryptodev.c | 185 +++++++++++++++---- lib/librte_cryptodev/rte_cryptodev.h | 65 ++++++- lib/librte_cryptodev/rte_cryptodev_pmd.h | 13 +- lib/librte_cryptodev/rte_cryptodev_version.map | 2 + lib/librte_vhost/rte_vhost_crypto.h | 9 +- lib/librte_vhost/vhost_crypto.c | 8 +- test/test/test_cryptodev.c | 220 ++++++++++++++--------- test/test/test_cryptodev_blockcipher.c | 7 +- test/test/test_cryptodev_blockcipher.h | 1 + test/test/test_event_crypto_adapter.c | 32 ++-- 57 files changed, 836 insertions(+), 286 deletions(-) diff --git a/app/test-crypto-perf/cperf.h b/app/test-crypto-perf/cperf.h index db58228dc..2b0aad095 100644 --- a/app/test-crypto-perf/cperf.h +++ b/app/test-crypto-perf/cperf.h @@ -15,6 +15,7 @@ struct cperf_op_fns; typedef void *(*cperf_constructor_t)( struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, diff --git a/app/test-crypto-perf/cperf_ops.c b/app/test-crypto-perf/cperf_ops.c index 44808f50a..f59568b80 100644 --- a/app/test-crypto-perf/cperf_ops.c +++ b/app/test-crypto-perf/cperf_ops.c @@ -469,6 +469,7 @@ cperf_set_ops_aead(struct rte_crypto_op **ops, static struct rte_cryptodev_sym_session * cperf_create_session(struct rte_mempool *sess_mp, + struct rte_mempool *priv_mp, uint8_t dev_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector, @@ -505,7 +506,7 @@ cperf_create_session(struct rte_mempool *sess_mp, } /* create crypto session */ rte_cryptodev_sym_session_init(dev_id, sess, &cipher_xform, - sess_mp); + priv_mp); /* * auth only */ @@ -533,7 +534,7 @@ cperf_create_session(struct rte_mempool *sess_mp, } /* create crypto session */ rte_cryptodev_sym_session_init(dev_id, sess, &auth_xform, - sess_mp); + priv_mp); /* * cipher and auth */ @@ -592,12 +593,12 @@ cperf_create_session(struct rte_mempool *sess_mp, cipher_xform.next = &auth_xform; /* create crypto session */ rte_cryptodev_sym_session_init(dev_id, - sess, &cipher_xform, sess_mp); + sess, &cipher_xform, priv_mp); } else { /* auth then cipher */ auth_xform.next = &cipher_xform; /* create crypto session */ rte_cryptodev_sym_session_init(dev_id, - sess, &auth_xform, sess_mp); + sess, &auth_xform, priv_mp); } } else { /* options->op_type == CPERF_AEAD */ aead_xform.type = RTE_CRYPTO_SYM_XFORM_AEAD; @@ -618,7 +619,7 @@ cperf_create_session(struct rte_mempool *sess_mp, /* Create crypto session */ rte_cryptodev_sym_session_init(dev_id, - sess, &aead_xform, sess_mp); + sess, &aead_xform, priv_mp); } return sess; diff --git a/app/test-crypto-perf/cperf_ops.h b/app/test-crypto-perf/cperf_ops.h index 29e109f2a..ff125d12c 100644 --- a/app/test-crypto-perf/cperf_ops.h +++ b/app/test-crypto-perf/cperf_ops.h @@ -13,7 +13,7 @@ typedef struct rte_cryptodev_sym_session *(*cperf_sessions_create_t)( - struct rte_mempool *sess_mp, + struct rte_mempool *sess_mp, struct rte_mempool *sess_priv_mp, uint8_t dev_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector, uint16_t iv_offset); diff --git a/app/test-crypto-perf/cperf_test_latency.c b/app/test-crypto-perf/cperf_test_latency.c index c9c98dc50..0fc3a6680 100644 --- a/app/test-crypto-perf/cperf_test_latency.c +++ b/app/test-crypto-perf/cperf_test_latency.c @@ -62,6 +62,7 @@ cperf_latency_test_free(struct cperf_latency_ctx *ctx) void * cperf_latency_test_constructor(struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector, @@ -86,8 +87,8 @@ cperf_latency_test_constructor(struct rte_mempool *sess_mp, sizeof(struct rte_crypto_sym_op) + sizeof(struct cperf_op_result *); - ctx->sess = op_fns->sess_create(sess_mp, dev_id, options, test_vector, - iv_offset); + ctx->sess = op_fns->sess_create(sess_mp, sess_priv_mp, dev_id, options, + test_vector, iv_offset); if (ctx->sess == NULL) goto err; diff --git a/app/test-crypto-perf/cperf_test_latency.h b/app/test-crypto-perf/cperf_test_latency.h index d3fc3218d..ed5b0a07b 100644 --- a/app/test-crypto-perf/cperf_test_latency.h +++ b/app/test-crypto-perf/cperf_test_latency.h @@ -17,6 +17,7 @@ void * cperf_latency_test_constructor( struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, diff --git a/app/test-crypto-perf/cperf_test_pmd_cyclecount.c b/app/test-crypto-perf/cperf_test_pmd_cyclecount.c index c8d16db6d..92af5ec90 100644 --- a/app/test-crypto-perf/cperf_test_pmd_cyclecount.c +++ b/app/test-crypto-perf/cperf_test_pmd_cyclecount.c @@ -80,6 +80,7 @@ cperf_pmd_cyclecount_test_free(struct cperf_pmd_cyclecount_ctx *ctx) void * cperf_pmd_cyclecount_test_constructor(struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector, @@ -106,8 +107,8 @@ cperf_pmd_cyclecount_test_constructor(struct rte_mempool *sess_mp, uint16_t iv_offset = sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op); - ctx->sess = op_fns->sess_create( - sess_mp, dev_id, options, test_vector, iv_offset); + ctx->sess = op_fns->sess_create(sess_mp, sess_priv_mp, dev_id, options, + test_vector, iv_offset); if (ctx->sess == NULL) goto err; diff --git a/app/test-crypto-perf/cperf_test_pmd_cyclecount.h b/app/test-crypto-perf/cperf_test_pmd_cyclecount.h index beb441991..3084038a1 100644 --- a/app/test-crypto-perf/cperf_test_pmd_cyclecount.h +++ b/app/test-crypto-perf/cperf_test_pmd_cyclecount.h @@ -18,6 +18,7 @@ void * cperf_pmd_cyclecount_test_constructor( struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, diff --git a/app/test-crypto-perf/cperf_test_throughput.c b/app/test-crypto-perf/cperf_test_throughput.c index 8766d6e9b..2767f4ea8 100644 --- a/app/test-crypto-perf/cperf_test_throughput.c +++ b/app/test-crypto-perf/cperf_test_throughput.c @@ -47,6 +47,7 @@ cperf_throughput_test_free(struct cperf_throughput_ctx *ctx) void * cperf_throughput_test_constructor(struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector, @@ -69,8 +70,8 @@ cperf_throughput_test_constructor(struct rte_mempool *sess_mp, uint16_t iv_offset = sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op); - ctx->sess = op_fns->sess_create(sess_mp, dev_id, options, test_vector, - iv_offset); + ctx->sess = op_fns->sess_create(sess_mp, sess_priv_mp, dev_id, options, + test_vector, iv_offset); if (ctx->sess == NULL) goto err; diff --git a/app/test-crypto-perf/cperf_test_throughput.h b/app/test-crypto-perf/cperf_test_throughput.h index 439ec8e55..91e1a4b70 100644 --- a/app/test-crypto-perf/cperf_test_throughput.h +++ b/app/test-crypto-perf/cperf_test_throughput.h @@ -18,6 +18,7 @@ void * cperf_throughput_test_constructor( struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, diff --git a/app/test-crypto-perf/cperf_test_verify.c b/app/test-crypto-perf/cperf_test_verify.c index 9134b921e..0497cf9a1 100644 --- a/app/test-crypto-perf/cperf_test_verify.c +++ b/app/test-crypto-perf/cperf_test_verify.c @@ -51,6 +51,7 @@ cperf_verify_test_free(struct cperf_verify_ctx *ctx) void * cperf_verify_test_constructor(struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, const struct cperf_test_vector *test_vector, @@ -73,8 +74,8 @@ cperf_verify_test_constructor(struct rte_mempool *sess_mp, uint16_t iv_offset = sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op); - ctx->sess = op_fns->sess_create(sess_mp, dev_id, options, test_vector, - iv_offset); + ctx->sess = op_fns->sess_create(sess_mp, sess_priv_mp, dev_id, options, + test_vector, iv_offset); if (ctx->sess == NULL) goto err; diff --git a/app/test-crypto-perf/cperf_test_verify.h b/app/test-crypto-perf/cperf_test_verify.h index 9f70ad87b..ac2192ba9 100644 --- a/app/test-crypto-perf/cperf_test_verify.h +++ b/app/test-crypto-perf/cperf_test_verify.h @@ -18,6 +18,7 @@ void * cperf_verify_test_constructor( struct rte_mempool *sess_mp, + struct rte_mempool *sess_priv_mp, uint8_t dev_id, uint16_t qp_id, const struct cperf_options *options, diff --git a/app/test-crypto-perf/main.c b/app/test-crypto-perf/main.c index 38a2e429f..175c639fb 100644 --- a/app/test-crypto-perf/main.c +++ b/app/test-crypto-perf/main.c @@ -21,6 +21,10 @@ #include "cperf_test_verify.h" #include "cperf_test_pmd_cyclecount.h" +static struct { + struct rte_mempool *sess_mp; + struct rte_mempool *priv_mp; +} session_pool_socket[RTE_MAX_NUMA_NODES]; const char *cperf_test_type_strs[] = { [CPERF_TEST_TYPE_THROUGHPUT] = "throughput", @@ -61,8 +65,58 @@ const struct cperf_test cperf_testmap[] = { }; static int -cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs, - struct rte_mempool *session_pool_socket[]) +fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size, + uint32_t nb_sessions) +{ + char mp_name[RTE_MEMPOOL_NAMESIZE]; + struct rte_mempool *sess_mp; + + if (session_pool_socket[socket_id].priv_mp == NULL) { + snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, + "priv_sess_mp_%u", socket_id); + + sess_mp = rte_mempool_create(mp_name, + nb_sessions, + session_priv_size, + 0, 0, NULL, NULL, NULL, + NULL, socket_id, + 0); + + if (sess_mp == NULL) { + printf("Cannot create pool \"%s\" on socket %d\n", + mp_name, socket_id); + return -ENOMEM; + } + + printf("Allocated pool \"%s\" on socket %d\n", + mp_name, socket_id); + session_pool_socket[socket_id].priv_mp = sess_mp; + } + + if (session_pool_socket[socket_id].sess_mp == NULL) { + + snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, + "sess_mp_%u", socket_id); + + sess_mp = rte_cryptodev_sym_session_pool_create(mp_name, + nb_sessions, 0, 0, 0, socket_id); + + if (sess_mp == NULL) { + printf("Cannot create pool \"%s\" on socket %d\n", + mp_name, socket_id); + return -ENOMEM; + } + + printf("Allocated pool \"%s\" on socket %d\n", + mp_name, socket_id); + session_pool_socket[socket_id].sess_mp = sess_mp; + } + + return 0; +} + +static int +cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs) { uint8_t enabled_cdev_count = 0, nb_lcores, cdev_id; uint32_t sessions_needed = 0; @@ -177,11 +231,11 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs, rte_cryptodev_scheduler_slaves_get(cdev_id, NULL); - sessions_needed = 2 * enabled_cdev_count * + sessions_needed = enabled_cdev_count * opts->nb_qps * nb_slaves; #endif } else - sessions_needed = 2 * enabled_cdev_count * + sessions_needed = enabled_cdev_count * opts->nb_qps; /* @@ -194,32 +248,15 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs, "%u sessions\n", opts->nb_qps); return -ENOTSUP; } - if (session_pool_socket[socket_id] == NULL) { - char mp_name[RTE_MEMPOOL_NAMESIZE]; - struct rte_mempool *sess_mp; - - snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, - "sess_mp_%u", socket_id); - sess_mp = rte_mempool_create(mp_name, - sessions_needed, - max_sess_size, - 0, - 0, NULL, NULL, NULL, - NULL, socket_id, - 0); - - if (sess_mp == NULL) { - printf("Cannot create session pool on socket %d\n", - socket_id); - return -ENOMEM; - } - printf("Allocated session pool on socket %d\n", socket_id); - session_pool_socket[socket_id] = sess_mp; - } + ret = fill_session_pool_socket(socket_id, max_sess_size, + sessions_needed); + if (ret < 0) + return ret; - qp_conf.mp_session = session_pool_socket[socket_id]; - qp_conf.mp_session_private = session_pool_socket[socket_id]; + qp_conf.mp_session = session_pool_socket[socket_id].sess_mp; + qp_conf.mp_session_private = + session_pool_socket[socket_id].priv_mp; ret = rte_cryptodev_configure(cdev_id, &conf); if (ret < 0) { @@ -453,10 +490,7 @@ main(int argc, char **argv) struct cperf_options opts = {0}; struct cperf_test_vector *t_vec = NULL; struct cperf_op_fns op_fns; - void *ctx[RTE_MAX_LCORE] = { }; - struct rte_mempool *session_pool_socket[RTE_MAX_NUMA_NODES] = { 0 }; - int nb_cryptodevs = 0; uint16_t total_nb_qps = 0; uint8_t cdev_id, i; @@ -489,8 +523,7 @@ main(int argc, char **argv) goto err; } - nb_cryptodevs = cperf_initialize_cryptodev(&opts, enabled_cdevs, - session_pool_socket); + nb_cryptodevs = cperf_initialize_cryptodev(&opts, enabled_cdevs); if (!opts.silent) cperf_options_dump(&opts); @@ -558,7 +591,9 @@ main(int argc, char **argv) uint8_t socket_id = rte_cryptodev_socket_id(cdev_id); ctx[i] = cperf_testmap[opts.test].constructor( - session_pool_socket[socket_id], cdev_id, qp_id, + session_pool_socket[socket_id].sess_mp, + session_pool_socket[socket_id].priv_mp, + cdev_id, qp_id, &opts, t_vec, &op_fns); if (ctx[i] == NULL) { RTE_LOG(ERR, USER1, "Test run constructor failed\n"); diff --git a/doc/guides/rel_notes/release_19_02.rst b/doc/guides/rel_notes/release_19_02.rst index 633e06b41..5c8e8b7ac 100644 --- a/doc/guides/rel_notes/release_19_02.rst +++ b/doc/guides/rel_notes/release_19_02.rst @@ -24,6 +24,20 @@ DPDK Release 19.02 mempool and symmetric session private data mempool, and the last paramter of ``rte_cryptodev_queue_pair_setup()`` is removed. +* cryptodev: as shown in the the 18.08 deprecation notice, the structure + ``rte_cryptodev_sym_session`` has been updated to contain more information + to ensure safely accessing driver private data and user data area. The + creation of mempool for ``rte_cryptodev_sym_session`` objects is now enforced + to use new function ``rte_cryptodev_sym_session_pool_create()`` so that + the correct session information is set. Failed to do so will cause + ``rte_cryptodev_sym_session_create()`` function call failed. With the change + cryptodev does no longer enforce using the same mempool for symmetric session + header and the driver specific private data, and prevent the segmentation + faults caused by the incorrect access to both the session's private data and + user data. All cryptodev related applications (test applications and sample + applications) have been updated to demonstrate how to use this new + method. + New Features ------------ diff --git a/drivers/crypto/aesni_gcm/Makefile b/drivers/crypto/aesni_gcm/Makefile index 0a5c1a872..9241ad762 100644 --- a/drivers/crypto/aesni_gcm/Makefile +++ b/drivers/crypto/aesni_gcm/Makefile @@ -8,6 +8,7 @@ LIB = librte_pmd_aesni_gcm.a # build flags CFLAGS += -O3 +CFLAGS += -DALLOW_EXPERIMENTAL_API CFLAGS += $(WERROR_FLAGS) # library version diff --git a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c index abc7a6d5f..948ff0763 100644 --- a/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c +++ b/drivers/crypto/aesni_gcm/aesni_gcm_pmd.c @@ -419,7 +419,8 @@ handle_completed_gcm_crypto_op(struct aesni_gcm_qp *qp, if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sess, 0, sizeof(struct aesni_gcm_session)); memset(op->sym->session, 0, - rte_cryptodev_sym_get_header_session_size()); + rte_cryptodev_sym_get_existing_header_session_size( + op->sym->session)); rte_mempool_put(qp->sess_mp_priv, sess); rte_mempool_put(qp->sess_mp, op->sym->session); op->sym->session = NULL; diff --git a/drivers/crypto/aesni_gcm/meson.build b/drivers/crypto/aesni_gcm/meson.build index a02da1ef5..70f57ad73 100644 --- a/drivers/crypto/aesni_gcm/meson.build +++ b/drivers/crypto/aesni_gcm/meson.build @@ -8,5 +8,6 @@ else ext_deps += lib endif +allow_experimental_apis = true sources = files('aesni_gcm_pmd.c', 'aesni_gcm_pmd_ops.c') deps += ['bus_vdev'] diff --git a/drivers/crypto/aesni_mb/Makefile b/drivers/crypto/aesni_mb/Makefile index 806a95eb8..a6fa071ec 100644 --- a/drivers/crypto/aesni_mb/Makefile +++ b/drivers/crypto/aesni_mb/Makefile @@ -9,6 +9,7 @@ LIB = librte_pmd_aesni_mb.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API # library version LIBABIVER := 1 diff --git a/drivers/crypto/aesni_mb/meson.build b/drivers/crypto/aesni_mb/meson.build index aae0995e5..39a436bc7 100644 --- a/drivers/crypto/aesni_mb/meson.build +++ b/drivers/crypto/aesni_mb/meson.build @@ -8,5 +8,6 @@ else ext_deps += lib endif +allow_experimental_apis = true sources = files('rte_aesni_mb_pmd.c', 'rte_aesni_mb_pmd_ops.c') deps += ['bus_vdev'] diff --git a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c index b0f5c4d67..f3b270d09 100644 --- a/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c +++ b/drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c @@ -951,7 +951,8 @@ post_process_mb_job(struct aesni_mb_qp *qp, JOB_AES_HMAC *job) if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sess, 0, sizeof(struct aesni_mb_session)); memset(op->sym->session, 0, - rte_cryptodev_sym_get_header_session_size()); + rte_cryptodev_sym_get_existing_header_session_size( + op->sym->session)); rte_mempool_put(qp->sess_mp_priv, sess); rte_mempool_put(qp->sess_mp, op->sym->session); op->sym->session = NULL; diff --git a/drivers/crypto/armv8/Makefile b/drivers/crypto/armv8/Makefile index e862af72e..f71f6b14a 100644 --- a/drivers/crypto/armv8/Makefile +++ b/drivers/crypto/armv8/Makefile @@ -28,6 +28,7 @@ EXPORT_MAP := rte_pmd_armv8_version.map # external library dependencies CFLAGS += -I$(ARMV8_CRYPTO_LIB_PATH) CFLAGS += -I$(ARMV8_CRYPTO_LIB_PATH)/asm/include +CFLAGS += -DALLOW_EXPERIMENTAL_API LDLIBS += -L$(ARMV8_CRYPTO_LIB_PATH) -larmv8_crypto LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring LDLIBS += -lrte_cryptodev diff --git a/drivers/crypto/armv8/rte_armv8_pmd.c b/drivers/crypto/armv8/rte_armv8_pmd.c index 3b2d7fb2f..0d4649adc 100644 --- a/drivers/crypto/armv8/rte_armv8_pmd.c +++ b/drivers/crypto/armv8/rte_armv8_pmd.c @@ -655,7 +655,8 @@ process_op(struct armv8_crypto_qp *qp, struct rte_crypto_op *op, if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sess, 0, sizeof(struct armv8_crypto_session)); memset(op->sym->session, 0, - rte_cryptodev_sym_get_header_session_size()); + rte_cryptodev_sym_get_existing_header_session_size( + op->sym->session)); rte_mempool_put(qp->sess_mp, sess); rte_mempool_put(qp->sess_mp_priv, op->sym->session); op->sym->session = NULL; diff --git a/drivers/crypto/kasumi/Makefile b/drivers/crypto/kasumi/Makefile index cafe94986..3de2f97ed 100644 --- a/drivers/crypto/kasumi/Makefile +++ b/drivers/crypto/kasumi/Makefile @@ -15,6 +15,7 @@ LIB = librte_pmd_kasumi.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API # library version LIBABIVER := 1 diff --git a/drivers/crypto/kasumi/meson.build b/drivers/crypto/kasumi/meson.build index a09b0e251..80f13cd24 100644 --- a/drivers/crypto/kasumi/meson.build +++ b/drivers/crypto/kasumi/meson.build @@ -8,5 +8,6 @@ else ext_deps += lib endif +allow_experimental_apis = true sources = files('rte_kasumi_pmd.c', 'rte_kasumi_pmd_ops.c') deps += ['bus_vdev'] diff --git a/drivers/crypto/kasumi/rte_kasumi_pmd.c b/drivers/crypto/kasumi/rte_kasumi_pmd.c index 6df645a23..3abdb01a9 100644 --- a/drivers/crypto/kasumi/rte_kasumi_pmd.c +++ b/drivers/crypto/kasumi/rte_kasumi_pmd.c @@ -325,7 +325,8 @@ process_ops(struct rte_crypto_op **ops, struct kasumi_session *session, if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(session, 0, sizeof(struct kasumi_session)); memset(ops[i]->sym->session, 0, - rte_cryptodev_sym_get_header_session_size()); + rte_cryptodev_sym_get_existing_header_session_size( + ops[i]->sym->session)); rte_mempool_put(qp->sess_mp_priv, session); rte_mempool_put(qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session = NULL; diff --git a/drivers/crypto/openssl/Makefile b/drivers/crypto/openssl/Makefile index 8fe086b90..ae6c3bcac 100644 --- a/drivers/crypto/openssl/Makefile +++ b/drivers/crypto/openssl/Makefile @@ -9,6 +9,7 @@ LIB = librte_pmd_openssl.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API # library version LIBABIVER := 1 diff --git a/drivers/crypto/openssl/meson.build b/drivers/crypto/openssl/meson.build index c2a0dd8ba..77a6596d7 100644 --- a/drivers/crypto/openssl/meson.build +++ b/drivers/crypto/openssl/meson.build @@ -5,6 +5,7 @@ dep = dependency('libcrypto', required: false) if not dep.found() build = false endif +allow_experimental_apis = true deps += 'bus_vdev' sources = files('rte_openssl_pmd.c', 'rte_openssl_pmd_ops.c') ext_deps += dep diff --git a/drivers/crypto/openssl/rte_openssl_pmd.c b/drivers/crypto/openssl/rte_openssl_pmd.c index f985afa93..11f5b5f41 100644 --- a/drivers/crypto/openssl/rte_openssl_pmd.c +++ b/drivers/crypto/openssl/rte_openssl_pmd.c @@ -2027,7 +2027,8 @@ process_op(struct openssl_qp *qp, struct rte_crypto_op *op, openssl_reset_session(sess); memset(sess, 0, sizeof(struct openssl_session)); memset(op->sym->session, 0, - rte_cryptodev_sym_get_header_session_size()); + rte_cryptodev_sym_get_existing_header_session_size( + op->sym->session)); rte_mempool_put(qp->sess_mp_priv, sess); rte_mempool_put(qp->sess_mp, op->sym->session); op->sym->session = NULL; diff --git a/drivers/crypto/snow3g/Makefile b/drivers/crypto/snow3g/Makefile index ee5027d0c..37f77dbf8 100644 --- a/drivers/crypto/snow3g/Makefile +++ b/drivers/crypto/snow3g/Makefile @@ -15,6 +15,7 @@ LIB = librte_pmd_snow3g.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API # library version LIBABIVER := 1 diff --git a/drivers/crypto/snow3g/rte_snow3g_pmd.c b/drivers/crypto/snow3g/rte_snow3g_pmd.c index 7d131f780..5fd94b686 100644 --- a/drivers/crypto/snow3g/rte_snow3g_pmd.c +++ b/drivers/crypto/snow3g/rte_snow3g_pmd.c @@ -340,7 +340,8 @@ process_ops(struct rte_crypto_op **ops, struct snow3g_session *session, if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(session, 0, sizeof(struct snow3g_session)); memset(ops[i]->sym->session, 0, - rte_cryptodev_sym_get_header_session_size()); + rte_cryptodev_sym_get_existing_header_session_size( + ops[i]->sym->session)); rte_mempool_put(qp->sess_mp_priv, session); rte_mempool_put(qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session = NULL; diff --git a/drivers/crypto/zuc/Makefile b/drivers/crypto/zuc/Makefile index 68d84eebc..8d625aa01 100644 --- a/drivers/crypto/zuc/Makefile +++ b/drivers/crypto/zuc/Makefile @@ -15,6 +15,7 @@ LIB = librte_pmd_zuc.a # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API # library version LIBABIVER := 1 diff --git a/drivers/crypto/zuc/meson.build b/drivers/crypto/zuc/meson.build index b8ca7107e..63f2a5298 100644 --- a/drivers/crypto/zuc/meson.build +++ b/drivers/crypto/zuc/meson.build @@ -8,5 +8,6 @@ else ext_deps += lib endif +allow_experimental_apis = true sources = files('rte_zuc_pmd.c', 'rte_zuc_pmd_ops.c') deps += ['bus_vdev'] diff --git a/drivers/crypto/zuc/rte_zuc_pmd.c b/drivers/crypto/zuc/rte_zuc_pmd.c index 997c2092f..637994dfd 100644 --- a/drivers/crypto/zuc/rte_zuc_pmd.c +++ b/drivers/crypto/zuc/rte_zuc_pmd.c @@ -327,7 +327,8 @@ process_ops(struct rte_crypto_op **ops, enum zuc_operation op_type, if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sessions[i], 0, sizeof(struct zuc_session)); memset(ops[i]->sym->session, 0, - rte_cryptodev_sym_get_header_session_size()); + rte_cryptodev_sym_get_existing_header_session_size( + ops[i]->sym->session)); rte_mempool_put(qp->sess_mp_priv, sessions[i]); rte_mempool_put(qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session = NULL; diff --git a/drivers/net/softnic/rte_eth_softnic_cli.c b/drivers/net/softnic/rte_eth_softnic_cli.c index 57b623377..3ff51744f 100644 --- a/drivers/net/softnic/rte_eth_softnic_cli.c +++ b/drivers/net/softnic/rte_eth_softnic_cli.c @@ -1092,7 +1092,7 @@ cmd_tap(struct pmd_internals *softnic, /** * cryptodev dev | dev_id - * queue + * queue max_sessions **/ static void @@ -1106,7 +1106,7 @@ cmd_cryptodev(struct pmd_internals *softnic, char *name; memset(¶ms, 0, sizeof(params)); - if (n_tokens != 7) { + if (n_tokens != 9) { snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); return; } @@ -1145,6 +1145,19 @@ cmd_cryptodev(struct pmd_internals *softnic, return; } + if (strcmp(tokens[7], "max_sessions")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, + "4"); + return; + } + + if (softnic_parser_read_uint32(¶ms.session_pool_size, tokens[8]) + < 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "q"); + return; + } + if (softnic_cryptodev_create(softnic, name, ¶ms) == NULL) { snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); return; @@ -1739,6 +1752,42 @@ cmd_table_action_profile(struct pmd_internals *softnic, t0 += 1; } /* decap */ + if ((t0 < n_tokens) && + (strcmp(tokens[t0], "sym_crypto") == 0)) { + struct softnic_cryptodev *cryptodev; + + if (n_tokens < t0 + 5 || + strcmp(tokens[t0 + 1], "dev") || + strcmp(tokens[t0 + 3], "offset")) { + snprintf(out, out_size, MSG_ARG_MISMATCH, + "table action profile sym_crypto"); + return; + } + + cryptodev = softnic_cryptodev_find(softnic, tokens[t0 + 2]); + if (cryptodev == NULL) { + snprintf(out, out_size, MSG_ARG_INVALID, + "table action profile sym_crypto"); + return; + } + + p.sym_crypto.cryptodev_id = cryptodev->dev_id; + + if (softnic_parser_read_uint32(&p.sym_crypto.op_offset, + tokens[t0 + 4]) != 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "table action profile sym_crypto"); + return; + } + + p.sym_crypto.mp_create = cryptodev->mp_create; + p.sym_crypto.mp_init = cryptodev->mp_init; + + p.action_mask |= 1LLU << RTE_TABLE_ACTION_SYM_CRYPTO; + + t0 += 5; + } /* sym_crypto */ + if (t0 < n_tokens) { snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); return; diff --git a/drivers/net/softnic/rte_eth_softnic_cryptodev.c b/drivers/net/softnic/rte_eth_softnic_cryptodev.c index f031d8803..18a52ed05 100644 --- a/drivers/net/softnic/rte_eth_softnic_cryptodev.c +++ b/drivers/net/softnic/rte_eth_softnic_cryptodev.c @@ -11,6 +11,8 @@ #include "rte_eth_softnic_internals.h" +#define SOFTNIC_CRYPTO_SESSION_CACHE_SIZE 128 + int softnic_cryptodev_init(struct pmd_internals *p) { @@ -61,13 +63,16 @@ softnic_cryptodev_create(struct pmd_internals *p, struct softnic_cryptodev *cryptodev; uint32_t dev_id, i; uint32_t socket_id; + uint32_t cache_size; + char mp_name[NAME_SIZE]; int status; /* Check input params */ if ((name == NULL) || softnic_cryptodev_find(p, name) || (params->n_queues == 0) || - (params->queue_size == 0)) + (params->queue_size == 0) || + (params->session_pool_size == 0)) return NULL; if (params->dev_name) { @@ -83,6 +88,12 @@ softnic_cryptodev_create(struct pmd_internals *p, dev_id = params->dev_id; } + cache_size = (params->session_pool_size / 2 < + SOFTNIC_CRYPTO_SESSION_CACHE_SIZE) ? + (params->session_pool_size / 2) : + SOFTNIC_CRYPTO_SESSION_CACHE_SIZE; + + socket_id = rte_cryptodev_socket_id(dev_id); rte_cryptodev_info_get(dev_id, &dev_info); @@ -119,7 +130,44 @@ softnic_cryptodev_create(struct pmd_internals *p, cryptodev->dev_id = dev_id; cryptodev->n_queues = params->n_queues; + snprintf(mp_name, NAME_SIZE, "%s_mp%u", name, dev_id); + cryptodev->mp_create = rte_cryptodev_sym_session_pool_create( + mp_name, + params->session_pool_size, + 0, + cache_size, + 0, + socket_id); + if (!cryptodev->mp_create) + goto error_exit; + + snprintf(mp_name, NAME_SIZE, "%s_priv_mp%u", name, dev_id); + cryptodev->mp_init = rte_mempool_create( + mp_name, + params->session_pool_size, + rte_cryptodev_sym_get_private_session_size(dev_id), + cache_size, + 0, + NULL, + NULL, + NULL, + NULL, + socket_id, + 0); + if (!cryptodev->mp_init) + goto error_exit; + TAILQ_INSERT_TAIL(&p->cryptodev_list, cryptodev, node); return cryptodev; + +error_exit: + if (cryptodev->mp_create) + rte_mempool_free(cryptodev->mp_create); + if (cryptodev->mp_init) + rte_mempool_free(cryptodev->mp_init); + + free(cryptodev); + + return NULL; } diff --git a/drivers/net/softnic/rte_eth_softnic_internals.h b/drivers/net/softnic/rte_eth_softnic_internals.h index 31698b9f0..685058be6 100644 --- a/drivers/net/softnic/rte_eth_softnic_internals.h +++ b/drivers/net/softnic/rte_eth_softnic_internals.h @@ -286,6 +286,7 @@ struct softnic_cryptodev_params { uint32_t dev_id; /**< Valid only when *dev_name* is NULL. */ uint32_t n_queues; uint32_t queue_size; + uint32_t session_pool_size; }; struct softnic_cryptodev { @@ -293,6 +294,8 @@ struct softnic_cryptodev { char name[NAME_SIZE]; uint16_t dev_id; uint32_t n_queues; + struct rte_mempool *mp_create; + struct rte_mempool *mp_init; }; TAILQ_HEAD(softnic_cryptodev_list, softnic_cryptodev); diff --git a/examples/fips_validation/main.c b/examples/fips_validation/main.c index e7559c633..588ffa16f 100644 --- a/examples/fips_validation/main.c +++ b/examples/fips_validation/main.c @@ -29,6 +29,8 @@ struct cryptodev_fips_validate_env { uint32_t is_path_folder; uint32_t dev_id; struct rte_mempool *mpool; + struct rte_mempool *sess_mpool; + struct rte_mempool *sess_priv_mpool; struct rte_mempool *op_pool; struct rte_mbuf *mbuf; struct rte_crypto_op *op; @@ -39,7 +41,9 @@ static int cryptodev_fips_validate_app_int(void) { struct rte_cryptodev_config conf = {rte_socket_id(), 1}; - struct rte_cryptodev_qp_conf qp_conf = {128}; + struct rte_cryptodev_qp_conf qp_conf = {128, NULL, NULL}; + uint32_t sess_sz = rte_cryptodev_sym_get_private_session_size( + env.dev_id); int ret; ret = rte_cryptodev_configure(env.dev_id, &conf); @@ -49,15 +53,21 @@ cryptodev_fips_validate_app_int(void) env.mpool = rte_pktmbuf_pool_create("FIPS_MEMPOOL", 128, 0, 0, UINT16_MAX, rte_socket_id()); if (!env.mpool) - return ret; - - ret = rte_cryptodev_queue_pair_setup(env.dev_id, 0, &qp_conf, - rte_socket_id(), env.mpool); - if (ret < 0) - return ret; + return -ENOMEM; ret = -ENOMEM; + env.sess_mpool = rte_cryptodev_sym_session_pool_create( + "FIPS_SESS_MEMPOOL", 16, 0, 0, 0, rte_socket_id()); + if (!env.sess_mpool) + goto error_exit; + + env.sess_priv_mpool = rte_mempool_create("FIPS_SESS_PRIV_MEMPOOL", + 16, sess_sz, 0, 0, NULL, NULL, NULL, + NULL, rte_socket_id(), 0); + if (!env.sess_priv_mpool) + goto error_exit; + env.op_pool = rte_crypto_op_pool_create( "FIPS_OP_POOL", RTE_CRYPTO_OP_TYPE_SYMMETRIC, @@ -75,10 +85,23 @@ cryptodev_fips_validate_app_int(void) if (!env.op) goto error_exit; + qp_conf.mp_session = env.sess_mpool; + qp_conf.mp_session_private = env.sess_priv_mpool; + + ret = rte_cryptodev_queue_pair_setup(env.dev_id, 0, &qp_conf, + rte_socket_id()); + if (ret < 0) + goto error_exit; + return 0; error_exit: + rte_mempool_free(env.mpool); + if (env.sess_mpool) + rte_mempool_free(env.sess_mpool); + if (env.sess_priv_mpool) + rte_mempool_free(env.sess_priv_mpool); if (env.op_pool) rte_mempool_free(env.op_pool); @@ -93,6 +116,8 @@ cryptodev_fips_validate_app_uninit(void) rte_cryptodev_sym_session_clear(env.dev_id, env.sess); rte_cryptodev_sym_session_free(env.sess); rte_mempool_free(env.mpool); + rte_mempool_free(env.sess_mpool); + rte_mempool_free(env.sess_priv_mpool); rte_mempool_free(env.op_pool); } @@ -797,12 +822,12 @@ fips_run_test(void) if (ret < 0) return ret; - env.sess = rte_cryptodev_sym_session_create(env.mpool); + env.sess = rte_cryptodev_sym_session_create(env.sess_mpool); if (!env.sess) return -ENOMEM; ret = rte_cryptodev_sym_session_init(env.dev_id, - env.sess, &xform, env.mpool); + env.sess, &xform, env.sess_priv_mpool); if (ret < 0) { RTE_LOG(ERR, USER1, "Error %i: Init session\n", ret); diff --git a/examples/ip_pipeline/cli.c b/examples/ip_pipeline/cli.c index 910386282..a92467e63 100644 --- a/examples/ip_pipeline/cli.c +++ b/examples/ip_pipeline/cli.c @@ -790,7 +790,8 @@ cmd_kni(char **tokens, static const char cmd_cryptodev_help[] = "cryptodev \n" " dev | dev_id \n" -" queue \n"; +" queue \n" +" max_sessions "; static void cmd_cryptodev(char **tokens, @@ -802,7 +803,7 @@ cmd_cryptodev(char **tokens, char *name; memset(¶ms, 0, sizeof(params)); - if (n_tokens != 7) { + if (n_tokens != 9) { snprintf(out, out_size, MSG_ARG_MISMATCH, tokens[0]); return; } @@ -825,7 +826,7 @@ cmd_cryptodev(char **tokens, if (strcmp(tokens[4], "queue")) { snprintf(out, out_size, MSG_ARG_NOT_FOUND, - "4"); + "queue"); return; } @@ -841,6 +842,18 @@ cmd_cryptodev(char **tokens, return; } + if (strcmp(tokens[7], "max_sessions")) { + snprintf(out, out_size, MSG_ARG_NOT_FOUND, + "max_sessions"); + return; + } + + if (parser_read_uint32(¶ms.session_pool_size, tokens[8]) < 0) { + snprintf(out, out_size, MSG_ARG_INVALID, + "queue_size"); + return; + } + if (cryptodev_create(name, ¶ms) == NULL) { snprintf(out, out_size, MSG_CMD_FAIL, tokens[0]); return; @@ -1030,9 +1043,7 @@ static const char cmd_table_action_profile_help[] = " stats none | pkts]\n" " [stats pkts | bytes | both]\n" " [time]\n" -" [sym_crypto dev offset " -" mempool_create \n" -" mempool_init ]\n" +" [sym_crypto dev offset ]\n" " [tag]\n" " [decap]\n"; @@ -1404,13 +1415,10 @@ cmd_table_action_profile(char **tokens, if ((t0 < n_tokens) && (strcmp(tokens[t0], "sym_crypto") == 0)) { struct cryptodev *cryptodev; - struct mempool *mempool; - if (n_tokens < t0 + 9 || + if (n_tokens < t0 + 5 || strcmp(tokens[t0 + 1], "dev") || - strcmp(tokens[t0 + 3], "offset") || - strcmp(tokens[t0 + 5], "mempool_create") || - strcmp(tokens[t0 + 7], "mempool_init")) { + strcmp(tokens[t0 + 3], "offset")) { snprintf(out, out_size, MSG_ARG_MISMATCH, "table action profile sym_crypto"); return; @@ -1432,25 +1440,12 @@ cmd_table_action_profile(char **tokens, return; } - mempool = mempool_find(tokens[t0 + 6]); - if (mempool == NULL) { - snprintf(out, out_size, MSG_ARG_INVALID, - "table action profile sym_crypto"); - return; - } - p.sym_crypto.mp_create = mempool->m; - - mempool = mempool_find(tokens[t0 + 8]); - if (mempool == NULL) { - snprintf(out, out_size, MSG_ARG_INVALID, - "table action profile sym_crypto"); - return; - } - p.sym_crypto.mp_init = mempool->m; + p.sym_crypto.mp_create = cryptodev->mp_create; + p.sym_crypto.mp_init = cryptodev->mp_init; p.action_mask |= 1LLU << RTE_TABLE_ACTION_SYM_CRYPTO; - t0 += 9; + t0 += 5; } /* sym_crypto */ if ((t0 < n_tokens) && (strcmp(tokens[t0], "tag") == 0)) { diff --git a/examples/ip_pipeline/cryptodev.c b/examples/ip_pipeline/cryptodev.c index b365810de..ac1e38d6a 100644 --- a/examples/ip_pipeline/cryptodev.c +++ b/examples/ip_pipeline/cryptodev.c @@ -11,6 +11,8 @@ #include "cryptodev.h" +#define PIPELINE_CRYPTO_SESSION_CACHE_SIZE 128 + static struct cryptodev_list cryptodev_list; int @@ -53,13 +55,16 @@ cryptodev_create(const char *name, struct cryptodev_params *params) struct cryptodev *cryptodev; uint32_t dev_id, i; uint32_t socket_id; + uint32_t cache_size; + char mp_name[NAME_SIZE]; int status; /* Check input params */ if ((name == NULL) || cryptodev_find(name) || (params->n_queues == 0) || - (params->queue_size == 0)) + (params->queue_size == 0) || + (params->session_pool_size == 0)) return NULL; if (params->dev_name) { @@ -75,6 +80,11 @@ cryptodev_create(const char *name, struct cryptodev_params *params) dev_id = params->dev_id; } + cache_size = (params->session_pool_size / 2 < + PIPELINE_CRYPTO_SESSION_CACHE_SIZE) ? + (params->session_pool_size / 2) : + PIPELINE_CRYPTO_SESSION_CACHE_SIZE; + socket_id = rte_cryptodev_socket_id(dev_id); rte_cryptodev_info_get(dev_id, &dev_info); @@ -111,7 +121,44 @@ cryptodev_create(const char *name, struct cryptodev_params *params) cryptodev->dev_id = dev_id; cryptodev->n_queues = params->n_queues; + snprintf(mp_name, NAME_SIZE, "%s_mp%u", name, dev_id); + cryptodev->mp_create = rte_cryptodev_sym_session_pool_create( + mp_name, + params->session_pool_size, + 0, + cache_size, + 0, + socket_id); + if (!cryptodev->mp_create) + goto error_exit; + + snprintf(mp_name, NAME_SIZE, "%s_mp_priv%u", name, dev_id); + cryptodev->mp_init = rte_mempool_create( + NULL, + params->session_pool_size, + rte_cryptodev_sym_get_private_session_size(dev_id), + cache_size, + 0, + NULL, + NULL, + NULL, + NULL, + socket_id, + 0); + if (!cryptodev->mp_init) + goto error_exit; + TAILQ_INSERT_TAIL(&cryptodev_list, cryptodev, node); return cryptodev; + +error_exit: + if (cryptodev->mp_create) + rte_mempool_free(cryptodev->mp_create); + if (cryptodev->mp_init) + rte_mempool_free(cryptodev->mp_init); + + free(cryptodev); + + return NULL; } diff --git a/examples/ip_pipeline/cryptodev.h b/examples/ip_pipeline/cryptodev.h index d06b3f2f1..d00434379 100644 --- a/examples/ip_pipeline/cryptodev.h +++ b/examples/ip_pipeline/cryptodev.h @@ -17,6 +17,8 @@ struct cryptodev { char name[NAME_SIZE]; uint16_t dev_id; uint32_t n_queues; + struct rte_mempool *mp_create; + struct rte_mempool *mp_init; }; TAILQ_HEAD(cryptodev_list, cryptodev); @@ -35,6 +37,7 @@ struct cryptodev_params { uint32_t dev_id; /**< Valid only when *dev_name* is NULL. */ uint32_t n_queues; uint32_t queue_size; + uint32_t session_pool_size; }; struct cryptodev * diff --git a/examples/ip_pipeline/examples/flow_crypto.cli b/examples/ip_pipeline/examples/flow_crypto.cli index 9b639deb7..849f9d5fe 100644 --- a/examples/ip_pipeline/examples/flow_crypto.cli +++ b/examples/ip_pipeline/examples/flow_crypto.cli @@ -21,14 +21,13 @@ ; 5 Crypto Operation 1792 160 mempool MEMPOOL0 buffer 2304 pool 32K cache 256 cpu 1 -mempool MEMPOOL_SESSION0 buffer 1024 pool 1024 cache 128 cpu 1 -link LINK0 dev 0000:81:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on +link LINK0 dev 81:00.0 rxq 1 128 MEMPOOL0 txq 1 512 promiscuous on #Cryptodev -cryptodev CRYPTO0 dev crypto_aesni_gcm0 queue 1 1024 +cryptodev CRYPTO0 dev crypto_aesni_gcm0 queue 1 1024 max_sessions 512 -table action profile AP0 ipv4 offset 270 fwd sym_crypto dev CRYPTO0 offset 1792 mempool_create MEMPOOL_SESSION0 mempool_init MEMPOOL_SESSION0 +table action profile AP0 ipv4 offset 270 fwd sym_crypto dev CRYPTO0 offset 1792 table action profile AP1 ipv4 offset 270 fwd pipeline PIPELINE0 period 10 offset_port_id 0 cpu 1 @@ -46,7 +45,7 @@ pipeline PIPELINE0 table match stub action AP1 pipeline PIPELINE0 port in 0 table 0 pipeline PIPELINE0 port in 1 table 1 -thread 24 pipeline PIPELINE0 enable +thread 2 pipeline PIPELINE0 enable pipeline PIPELINE0 table 0 rule add match default action fwd port 2 diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c index a0ff8f7f7..fc102a396 100644 --- a/examples/ipsec-secgw/ipsec-secgw.c +++ b/examples/ipsec-secgw/ipsec-secgw.c @@ -55,7 +55,7 @@ #define CDEV_QUEUE_DESC 2048 #define CDEV_MAP_ENTRIES 16384 -#define CDEV_MP_NB_OBJS 2048 +#define CDEV_MP_NB_OBJS 1024 #define CDEV_MP_CACHE_SZ 64 #define MAX_QUEUE_PAIRS 1 @@ -820,11 +820,15 @@ main_loop(__attribute__((unused)) void *dummy) qconf->inbound.sa_ctx = socket_ctx[socket_id].sa_in; qconf->inbound.cdev_map = cdev_map_in; qconf->inbound.session_pool = socket_ctx[socket_id].session_pool; + qconf->inbound.session_priv_pool = + socket_ctx[socket_id].session_priv_pool; qconf->outbound.sp4_ctx = socket_ctx[socket_id].sp_ip4_out; qconf->outbound.sp6_ctx = socket_ctx[socket_id].sp_ip6_out; qconf->outbound.sa_ctx = socket_ctx[socket_id].sa_out; qconf->outbound.cdev_map = cdev_map_out; qconf->outbound.session_pool = socket_ctx[socket_id].session_pool; + qconf->outbound.session_priv_pool = + socket_ctx[socket_id].session_priv_pool; if (qconf->nb_rx_queue == 0) { RTE_LOG(INFO, IPSEC, "lcore %u has nothing to do\n", lcore_id); @@ -1460,10 +1464,10 @@ cryptodevs_init(void) dev_conf.nb_queue_pairs = qp; uint32_t dev_max_sess = cdev_info.sym.max_nb_sessions; - if (dev_max_sess != 0 && dev_max_sess < (CDEV_MP_NB_OBJS / 2)) + if (dev_max_sess != 0 && dev_max_sess < CDEV_MP_NB_OBJS) rte_exit(EXIT_FAILURE, "Device does not support at least %u " - "sessions", CDEV_MP_NB_OBJS / 2); + "sessions", CDEV_MP_NB_OBJS); if (!socket_ctx[dev_conf.socket_id].session_pool) { char mp_name[RTE_MEMPOOL_NAMESIZE]; @@ -1471,6 +1475,19 @@ cryptodevs_init(void) snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, "sess_mp_%u", dev_conf.socket_id); + sess_mp = rte_cryptodev_sym_session_pool_create( + mp_name, CDEV_MP_NB_OBJS, + 0, CDEV_MP_CACHE_SZ, 0, + dev_conf.socket_id); + socket_ctx[dev_conf.socket_id].session_pool = sess_mp; + } + + if (!socket_ctx[dev_conf.socket_id].session_priv_pool) { + char mp_name[RTE_MEMPOOL_NAMESIZE]; + struct rte_mempool *sess_mp; + + snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, + "sess_mp_priv_%u", dev_conf.socket_id); sess_mp = rte_mempool_create(mp_name, CDEV_MP_NB_OBJS, max_sess_sz, @@ -1478,25 +1495,28 @@ cryptodevs_init(void) 0, NULL, NULL, NULL, NULL, dev_conf.socket_id, 0); - if (sess_mp == NULL) - rte_exit(EXIT_FAILURE, - "Cannot create session pool on socket %d\n", - dev_conf.socket_id); - else - printf("Allocated session pool on socket %d\n", - dev_conf.socket_id); - socket_ctx[dev_conf.socket_id].session_pool = sess_mp; + socket_ctx[dev_conf.socket_id].session_priv_pool = + sess_mp; } + if (!socket_ctx[dev_conf.socket_id].session_priv_pool || + !socket_ctx[dev_conf.socket_id].session_pool) + rte_exit(EXIT_FAILURE, + "Cannot create session pool on socket %d\n", + dev_conf.socket_id); + else + printf("Allocated session pool on socket %d\n", + dev_conf.socket_id); + if (rte_cryptodev_configure(cdev_id, &dev_conf)) rte_panic("Failed to initialize cryptodev %u\n", cdev_id); qp_conf.nb_descriptors = CDEV_QUEUE_DESC; qp_conf.mp_session = - socket_ctx[dev_conf.socket_id].session_pool; + socket_ctx[dev_conf.socket_id].session_pool; qp_conf.mp_session_private = - socket_ctx[dev_conf.socket_id].session_pool; + socket_ctx[dev_conf.socket_id].session_priv_pool; for (qp = 0; qp < dev_conf.nb_queue_pairs; qp++) if (rte_cryptodev_queue_pair_setup(cdev_id, qp, &qp_conf, dev_conf.socket_id)) @@ -1521,7 +1541,7 @@ cryptodevs_init(void) snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, "sess_mp_%u", socket_id); sess_mp = rte_mempool_create(mp_name, - CDEV_MP_NB_OBJS, + (CDEV_MP_NB_OBJS * 2), max_sess_sz, CDEV_MP_CACHE_SZ, 0, NULL, NULL, NULL, diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c index 3d415f1af..9dc6e173c 100644 --- a/examples/ipsec-secgw/ipsec.c +++ b/examples/ipsec-secgw/ipsec.c @@ -323,7 +323,7 @@ create_session(struct ipsec_ctx *ipsec_ctx, struct ipsec_sa *sa) ipsec_ctx->session_pool); rte_cryptodev_sym_session_init(ipsec_ctx->tbl[cdev_id_qp].id, sa->crypto_session, sa->xforms, - ipsec_ctx->session_pool); + ipsec_ctx->session_priv_pool); rte_cryptodev_info_get(ipsec_ctx->tbl[cdev_id_qp].id, &cdev_info); diff --git a/examples/ipsec-secgw/ipsec.h b/examples/ipsec-secgw/ipsec.h index c998c8076..f35552857 100644 --- a/examples/ipsec-secgw/ipsec.h +++ b/examples/ipsec-secgw/ipsec.h @@ -144,6 +144,7 @@ struct ipsec_ctx { uint16_t last_qp; struct cdev_qp tbl[MAX_QP_PER_LCORE]; struct rte_mempool *session_pool; + struct rte_mempool *session_priv_pool; struct rte_mbuf *ol_pkts[MAX_PKT_BURST] __rte_aligned(sizeof(void *)); uint16_t ol_pkts_cnt; }; @@ -166,6 +167,7 @@ struct socket_ctx { struct rt_ctx *rt_ip6; struct rte_mempool *mbuf_pool; struct rte_mempool *session_pool; + struct rte_mempool *session_priv_pool; }; struct cnt_blk { diff --git a/examples/l2fwd-crypto/main.c b/examples/l2fwd-crypto/main.c index 1df7ba743..9982f07e9 100644 --- a/examples/l2fwd-crypto/main.c +++ b/examples/l2fwd-crypto/main.c @@ -221,7 +221,10 @@ static struct rte_eth_conf port_conf = { struct rte_mempool *l2fwd_pktmbuf_pool; struct rte_mempool *l2fwd_crypto_op_pool; -struct rte_mempool *session_pool_socket[RTE_MAX_NUMA_NODES] = { 0 }; +static struct { + struct rte_mempool *sess_mp; + struct rte_mempool *priv_mp; +} session_pool_socket[RTE_MAX_NUMA_NODES]; /* Per-port statistics struct */ struct l2fwd_port_statistics { @@ -645,7 +648,6 @@ initialize_crypto_session(struct l2fwd_crypto_options *options, uint8_t cdev_id) return NULL; uint8_t socket_id = (uint8_t) retval; - struct rte_mempool *sess_mp = session_pool_socket[socket_id]; if (options->xform_chain == L2FWD_CRYPTO_AEAD) { first_xform = &options->aead_xform; @@ -661,13 +663,14 @@ initialize_crypto_session(struct l2fwd_crypto_options *options, uint8_t cdev_id) first_xform = &options->auth_xform; } - session = rte_cryptodev_sym_session_create(sess_mp); - + session = rte_cryptodev_sym_session_create( + session_pool_socket[socket_id].sess_mp); if (session == NULL) return NULL; if (rte_cryptodev_sym_session_init(cdev_id, session, - first_xform, sess_mp) < 0) + first_xform, + session_pool_socket[socket_id].priv_mp) < 0) return NULL; return session; @@ -2267,38 +2270,54 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, rte_cryptodev_scheduler_slaves_get(cdev_id, NULL); - sessions_needed = 2 * enabled_cdev_count * nb_slaves; + sessions_needed = enabled_cdev_count * nb_slaves; #endif } else - sessions_needed = 2 * enabled_cdev_count; + sessions_needed = enabled_cdev_count; - if (session_pool_socket[socket_id] == NULL) { + if (session_pool_socket[socket_id].priv_mp == NULL) { char mp_name[RTE_MEMPOOL_NAMESIZE]; - struct rte_mempool *sess_mp; snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, - "sess_mp_%u", socket_id); + "priv_sess_mp_%u", socket_id); - /* - * Create enough objects for session headers and - * device private data - */ - sess_mp = rte_mempool_create(mp_name, + session_pool_socket[socket_id].priv_mp = + rte_mempool_create(mp_name, sessions_needed, max_sess_sz, - SESSION_POOL_CACHE_SIZE, - 0, NULL, NULL, NULL, + 0, 0, NULL, NULL, NULL, NULL, socket_id, 0); - if (sess_mp == NULL) { - printf("Cannot create session pool on socket %d\n", + if (session_pool_socket[socket_id].priv_mp == NULL) { + printf("Cannot create pool on socket %d\n", + socket_id); + return -ENOMEM; + } + + printf("Allocated pool \"%s\" on socket %d\n", + mp_name, socket_id); + } + + if (session_pool_socket[socket_id].sess_mp == NULL) { + char mp_name[RTE_MEMPOOL_NAMESIZE]; + snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, + "sess_mp_%u", socket_id); + + session_pool_socket[socket_id].sess_mp = + rte_cryptodev_sym_session_pool_create( + mp_name, + sessions_needed, + 0, 0, 0, socket_id); + + if (session_pool_socket[socket_id].sess_mp == NULL) { + printf("Cannot create pool on socket %d\n", socket_id); return -ENOMEM; } - printf("Allocated session pool on socket %d\n", socket_id); - session_pool_socket[socket_id] = sess_mp; + printf("Allocated pool \"%s\" on socket %d\n", + mp_name, socket_id); } /* Set AEAD parameters */ @@ -2443,8 +2462,9 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, } qp_conf.nb_descriptors = 2048; - qp_conf.mp_session = session_pool_socket[socket_id]; - qp_conf.mp_session_private = session_pool_socket[socket_id]; + qp_conf.mp_session = session_pool_socket[socket_id].sess_mp; + qp_conf.mp_session_private = + session_pool_socket[socket_id].priv_mp; retval = rte_cryptodev_queue_pair_setup(cdev_id, 0, &qp_conf, socket_id); diff --git a/examples/vhost_crypto/main.c b/examples/vhost_crypto/main.c index aa2727dfc..f8eb1f5c6 100644 --- a/examples/vhost_crypto/main.c +++ b/examples/vhost_crypto/main.c @@ -46,6 +46,7 @@ struct vhost_crypto_info { int vids[MAX_NB_SOCKETS]; uint32_t nb_vids; struct rte_mempool *sess_pool; + struct rte_mempool *sess_priv_pool; struct rte_mempool *cop_pool; uint8_t cid; uint32_t qid; @@ -289,6 +290,7 @@ new_device(int vid) } ret = rte_vhost_crypto_create(vid, info->cid, info->sess_pool, + info->sess_priv_pool, rte_lcore_to_socket_id(options.los[i].lcore_id)); if (ret) { RTE_LOG(ERR, USER1, "Cannot create vhost crypto\n"); @@ -448,6 +450,7 @@ free_resource(void) rte_mempool_free(info->cop_pool); rte_mempool_free(info->sess_pool); + rte_mempool_free(info->sess_priv_pool); for (j = 0; j < lo->nb_sockets; j++) { rte_vhost_driver_unregister(lo->socket_files[i]); @@ -527,11 +530,17 @@ main(int argc, char *argv[]) } snprintf(name, 127, "SESS_POOL_%u", lo->lcore_id); - info->sess_pool = rte_mempool_create(name, SESSION_MAP_ENTRIES, + info->sess_pool = rte_cryptodev_sym_session_pool_create(name, + SESSION_MAP_ENTRIES, 0, 0, 0, + rte_lcore_to_socket_id(lo->lcore_id)); + + snprintf(name, 127, "SESS_POOL_PRIV_%u", lo->lcore_id); + info->sess_priv_pool = rte_mempool_create(name, + SESSION_MAP_ENTRIES, rte_cryptodev_sym_get_private_session_size( info->cid), 64, 0, NULL, NULL, NULL, NULL, rte_lcore_to_socket_id(lo->lcore_id), 0); - if (!info->sess_pool) { + if (!info->sess_priv_pool || info->sess_pool) { RTE_LOG(ERR, USER1, "Failed to create mempool"); goto error_exit; } diff --git a/lib/librte_cryptodev/Makefile b/lib/librte_cryptodev/Makefile index a8f94c097..ade108b90 100644 --- a/lib/librte_cryptodev/Makefile +++ b/lib/librte_cryptodev/Makefile @@ -12,6 +12,7 @@ LIBABIVER := 5 # build flags CFLAGS += -O3 CFLAGS += $(WERROR_FLAGS) +CFLAGS += -DALLOW_EXPERIMENTAL_API LDLIBS += -lrte_eal -lrte_mempool -lrte_ring -lrte_mbuf LDLIBS += -lrte_kvargs diff --git a/lib/librte_cryptodev/meson.build b/lib/librte_cryptodev/meson.build index 990dd3d44..40ccce37a 100644 --- a/lib/librte_cryptodev/meson.build +++ b/lib/librte_cryptodev/meson.build @@ -2,6 +2,7 @@ # Copyright(c) 2017 Intel Corporation version = 5 +allow_experimental_apis = true sources = files('rte_cryptodev.c', 'rte_cryptodev_pmd.c') headers = files('rte_cryptodev.h', 'rte_cryptodev_pmd.h', diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c index 11776b6ac..7ad084089 100644 --- a/lib/librte_cryptodev/rte_cryptodev.c +++ b/lib/librte_cryptodev/rte_cryptodev.c @@ -189,6 +189,16 @@ const char *rte_crypto_asym_op_strings[] = { [RTE_CRYPTO_ASYM_OP_SHARED_SECRET_COMPUTE] = "sharedsecret_compute", }; +/** + * The private data structure stored in the session mempool private data. + */ +struct rte_cryptodev_sym_session_pool_private_data { + uint16_t nb_drivers; + /**< number of elements in sess_data array */ + uint16_t user_data_sz; + /**< session user data will be placed after sess_data */ +}; + int rte_cryptodev_get_cipher_algo_enum(enum rte_crypto_cipher_algorithm *algo_enum, const char *algo_string) @@ -951,7 +961,45 @@ rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, return -EINVAL; } + if (!qp_conf) { + CDEV_LOG_ERR("qp_conf cannot be NULL\n"); + return -EINVAL; + } + + if ((qp_conf->mp_session && !qp_conf->mp_session_private) || + (!qp_conf->mp_session && qp_conf->mp_session_private)) { + CDEV_LOG_ERR("Invalid mempools\n"); + return -EINVAL; + } + dev = &rte_crypto_devices[dev_id]; + + if (qp_conf->mp_session) { + struct rte_cryptodev_sym_session_pool_private_data *pool_priv; + uint32_t obj_size = qp_conf->mp_session->elt_size; + uint32_t obj_priv_size = qp_conf->mp_session_private->elt_size; + struct rte_cryptodev_sym_session s = {0}; + + pool_priv = rte_mempool_get_priv(qp_conf->mp_session); + if (!pool_priv || qp_conf->mp_session->private_data_size < + sizeof(*pool_priv)) { + CDEV_LOG_ERR("Invalid mempool\n"); + return -EINVAL; + } + + s.nb_drivers = pool_priv->nb_drivers; + s.user_data_sz = pool_priv->user_data_sz; + + if ((rte_cryptodev_sym_get_existing_header_session_size(&s) > + obj_size) || (s.nb_drivers <= dev->driver_id) || + rte_cryptodev_sym_get_private_session_size(dev_id) > + obj_priv_size) { + CDEV_LOG_ERR("Invalid mempool\n"); + return -EINVAL; + } + + } + if (queue_pair_id >= dev->data->nb_queue_pairs) { CDEV_LOG_ERR("Invalid queue_pair_id=%d", queue_pair_id); return -EINVAL; @@ -969,7 +1017,6 @@ rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, socket_id); } - int rte_cryptodev_stats_get(uint8_t dev_id, struct rte_cryptodev_stats *stats) { @@ -1143,7 +1190,6 @@ rte_cryptodev_pmd_callback_process(struct rte_cryptodev *dev, rte_spinlock_unlock(&rte_cryptodev_cb_lock); } - int rte_cryptodev_sym_session_init(uint8_t dev_id, struct rte_cryptodev_sym_session *sess, @@ -1160,12 +1206,15 @@ rte_cryptodev_sym_session_init(uint8_t dev_id, return -EINVAL; index = dev->driver_id; + if (index > sess->nb_drivers) + return -EINVAL; RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->sym_session_configure, -ENOTSUP); - if (sess->sess_private_data[index] == NULL) { + if (sess->sess_data[index].refcnt == 0) { ret = dev->dev_ops->sym_session_configure(dev, xforms, - sess, mp); + sess, mp); + if (ret < 0) { CDEV_LOG_ERR( "dev_id %d failed to configure session details", @@ -1174,6 +1223,7 @@ rte_cryptodev_sym_session_init(uint8_t dev_id, } } + sess->sess_data[index].refcnt++; return 0; } @@ -1212,10 +1262,70 @@ rte_cryptodev_asym_session_init(uint8_t dev_id, return 0; } +struct rte_mempool * __rte_experimental +rte_cryptodev_sym_session_pool_create(const char *name, uint32_t nb_elts, + uint32_t elt_size, uint32_t cache_size, uint16_t user_data_size, + int socket_id) +{ + struct rte_mempool *mp; + struct rte_cryptodev_sym_session_pool_private_data *pool_priv; + uint32_t obj_sz; + + obj_sz = rte_cryptodev_sym_get_header_session_size() + user_data_size; + if (obj_sz > elt_size) + CDEV_LOG_INFO("elt_size %u is expanded to %u\n", elt_size, + obj_sz); + else + obj_sz = elt_size; + + mp = rte_mempool_create(name, nb_elts, obj_sz, cache_size, + (uint32_t)(sizeof(*pool_priv)), + NULL, NULL, NULL, NULL, + socket_id, 0); + if (mp == NULL) { + CDEV_LOG_ERR("%s(name=%s) failed, rte_errno=%d\n", + __func__, name, rte_errno); + return NULL; + } + + pool_priv = rte_mempool_get_priv(mp); + if (!pool_priv) { + CDEV_LOG_ERR("%s(name=%s) failed to get private data\n", + __func__, name); + rte_mempool_free(mp); + return NULL; + } + + pool_priv->nb_drivers = nb_drivers; + pool_priv->user_data_sz = user_data_size; + + return mp; +} + +static unsigned int +rte_cryptodev_sym_session_data_size(struct rte_cryptodev_sym_session *sess) +{ + return (sizeof(sess->sess_data[0]) * sess->nb_drivers) + + sess->user_data_sz; +} + struct rte_cryptodev_sym_session * rte_cryptodev_sym_session_create(struct rte_mempool *mp) { struct rte_cryptodev_sym_session *sess; + struct rte_cryptodev_sym_session_pool_private_data *pool_priv; + + if (!mp) { + CDEV_LOG_ERR("Invalid mempool\n"); + return NULL; + } + + pool_priv = rte_mempool_get_priv(mp); + + if (!pool_priv || mp->private_data_size < sizeof(*pool_priv)) { + CDEV_LOG_ERR("Invalid mempool\n"); + return NULL; + } /* Allocate a session structure from the session pool */ if (rte_mempool_get(mp, (void **)&sess)) { @@ -1226,7 +1336,12 @@ rte_cryptodev_sym_session_create(struct rte_mempool *mp) /* Clear device session pointer. * Include the flag indicating presence of user data */ - memset(sess, 0, (sizeof(void *) * nb_drivers) + sizeof(uint8_t)); + sess->nb_drivers = pool_priv->nb_drivers; + sess->user_data_sz= pool_priv->user_data_sz; + sess->opaque_data = 0; + + memset(sess->sess_data, 0, + rte_cryptodev_sym_session_data_size(sess)); return sess; } @@ -1255,12 +1370,17 @@ rte_cryptodev_sym_session_clear(uint8_t dev_id, struct rte_cryptodev_sym_session *sess) { struct rte_cryptodev *dev; + uint8_t driver_id; dev = rte_cryptodev_pmd_get_dev(dev_id); if (dev == NULL || sess == NULL) return -EINVAL; + driver_id = dev->driver_id; + if (--sess->sess_data[driver_id].refcnt != 0) + return -EBUSY; + RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->sym_session_clear, -ENOTSUP); dev->dev_ops->sym_session_clear(dev, sess); @@ -1290,16 +1410,15 @@ int rte_cryptodev_sym_session_free(struct rte_cryptodev_sym_session *sess) { uint8_t i; - void *sess_priv; struct rte_mempool *sess_mp; if (sess == NULL) return -EINVAL; /* Check that all device private data has been freed */ - for (i = 0; i < nb_drivers; i++) { - sess_priv = get_sym_session_private_data(sess, i); - if (sess_priv != NULL) + /* Check that all device private data has been freed */ + for (i = 0; i < sess->nb_drivers; i++) { + if (sess->sess_data[i].refcnt != 0) return -EBUSY; } @@ -1334,16 +1453,31 @@ rte_cryptodev_asym_session_free(struct rte_cryptodev_asym_session *sess) return 0; } - unsigned int rte_cryptodev_sym_get_header_session_size(void) { /* - * Header contains pointers to the private data + * Header contains all dpointers to the private data * of all registered drivers, and a flag which * indicates presence of user data */ - return ((sizeof(void *) * nb_drivers) + sizeof(uint8_t)); + struct rte_cryptodev_sym_session s = {0}; + s.nb_drivers = nb_drivers; + s.user_data_sz = 0; + + return (unsigned int)(sizeof(s) + + rte_cryptodev_sym_session_data_size(&s)); +} + +unsigned int __rte_experimental +rte_cryptodev_sym_get_existing_header_session_size( + struct rte_cryptodev_sym_session *sess) +{ + if (!sess) + return 0; + else + return (unsigned int)(sizeof(*sess) + + rte_cryptodev_sym_session_data_size(sess)); } unsigned int __rte_experimental @@ -1361,7 +1495,6 @@ unsigned int rte_cryptodev_sym_get_private_session_size(uint8_t dev_id) { struct rte_cryptodev *dev; - unsigned int header_size = sizeof(void *) * nb_drivers; unsigned int priv_sess_size; if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) @@ -1374,14 +1507,6 @@ rte_cryptodev_sym_get_private_session_size(uint8_t dev_id) priv_sess_size = (*dev->dev_ops->sym_session_get_size)(dev); - /* - * If size is less than session header size, - * return the latter, as this guarantees that - * sessionless operations will work - */ - if (priv_sess_size < header_size) - return header_size; - return priv_sess_size; } @@ -1415,15 +1540,13 @@ rte_cryptodev_sym_session_set_user_data( void *data, uint16_t size) { - uint16_t off_set = sizeof(void *) * nb_drivers; - uint8_t *user_data_present = (uint8_t *)sess + off_set; - if (sess == NULL) return -EINVAL; - *user_data_present = 1; - off_set += sizeof(uint8_t); - rte_memcpy((uint8_t *)sess + off_set, data, size); + if (sess->user_data_sz < size) + return -ENOMEM; + + rte_memcpy(sess->sess_data + sess->nb_drivers, data, size); return 0; } @@ -1431,14 +1554,10 @@ void * __rte_experimental rte_cryptodev_sym_session_get_user_data( struct rte_cryptodev_sym_session *sess) { - uint16_t off_set = sizeof(void *) * nb_drivers; - uint8_t *user_data_present = (uint8_t *)sess + off_set; - - if (sess == NULL || !*user_data_present) + if (sess == NULL || sess->user_data_sz == 0) return NULL; - off_set += sizeof(uint8_t); - return (uint8_t *)sess + off_set; + return (void *)(sess->sess_data + sess->nb_drivers); } /** Initialise rte_crypto_op mempool element */ diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h index f9e7507da..698a859f3 100644 --- a/lib/librte_cryptodev/rte_cryptodev.h +++ b/lib/librte_cryptodev/rte_cryptodev.h @@ -951,14 +951,22 @@ rte_cryptodev_enqueue_burst(uint8_t dev_id, uint16_t qp_id, dev->data->queue_pairs[qp_id], ops, nb_ops); } - /** Cryptodev symmetric crypto session * Each session is derived from a fixed xform chain. Therefore each session * has a fixed algo, key, op-type, digest_len etc. */ struct rte_cryptodev_sym_session { - __extension__ void *sess_private_data[0]; - /**< Private symmetric session material */ + uint64_t opaque_data; + /**< Can be used for external metadata */ + uint16_t nb_drivers; + /**< number of elements in sess_data array */ + uint16_t user_data_sz; + /**< session user data will be placed after sess_data */ + __extension__ struct { + void *data; + uint16_t refcnt; + } sess_data[0]; + /**< Driver specific session material, variable size */ }; /** Cryptodev asymmetric crypto session */ @@ -968,6 +976,37 @@ struct rte_cryptodev_asym_session { }; /** + * Create a symmetric session mempool. + * + * @param name + * The unique mempool name. + * @param nb_elts + * The number of elements in the mempool. + * @param elt_size + * The size of the element. This value will be ignored if it is smaller than + * the minimum session header size required for the system. For the user who + * want to use the same mempool for sym session and session private data it + * can be the maximum value of all existing devices' private data and session + * header sizes. + * @param cache_size + * The number of per-lcore cache elements + * @param priv_size + * The private data size of each session. + * @param socket_id + * The *socket_id* argument is the socket identifier in the case of + * NUMA. The value can be *SOCKET_ID_ANY* if there is no NUMA + * constraint for the reserved zone. + * + * @return + * - On success return size of the session + * - On failure returns 0 + */ +struct rte_mempool * __rte_experimental +rte_cryptodev_sym_session_pool_create(const char *name, uint32_t nb_elts, + uint32_t elt_size, uint32_t cache_size, uint16_t priv_size, + int socket_id); + +/** * Create symmetric crypto session header (generic with no private data) * * @param mempool Symmetric session mempool to allocate session @@ -1097,15 +1136,31 @@ rte_cryptodev_asym_session_clear(uint8_t dev_id, struct rte_cryptodev_asym_session *sess); /** - * Get the size of the header session, for all registered drivers. + * Get the size of the header session, for all registered drivers excluding + * the user data size. * * @return - * Size of the symmetric eader session. + * Size of the symmetric header session. */ unsigned int rte_cryptodev_sym_get_header_session_size(void); /** + * Get the size of the header session from created session. + * + * @param sess + * The sym cryptodev session pointer + * + * @return + * - If sess is not NULL, return the size of the header session including + * the private data size defined within sess. + * - If sess is NULL, return 0. + */ +unsigned int __rte_experimental +rte_cryptodev_sym_get_existing_header_session_size( + struct rte_cryptodev_sym_session *sess); + +/** * Get the size of the asymmetric session header, for all registered drivers. * * @return diff --git a/lib/librte_cryptodev/rte_cryptodev_pmd.h b/lib/librte_cryptodev/rte_cryptodev_pmd.h index f15c9af30..defe05ea0 100644 --- a/lib/librte_cryptodev/rte_cryptodev_pmd.h +++ b/lib/librte_cryptodev/rte_cryptodev_pmd.h @@ -475,14 +475,23 @@ RTE_INIT(init_ ##driver_id)\ static inline void * get_sym_session_private_data(const struct rte_cryptodev_sym_session *sess, uint8_t driver_id) { - return sess->sess_private_data[driver_id]; + if (unlikely(sess->nb_drivers <= driver_id)) + return NULL; + + return sess->sess_data[driver_id].data; } static inline void set_sym_session_private_data(struct rte_cryptodev_sym_session *sess, uint8_t driver_id, void *private_data) { - sess->sess_private_data[driver_id] = private_data; + if (unlikely(sess->nb_drivers <= driver_id)) { + CDEV_LOG_ERR("Set private data for driver %u not allowed\n", + driver_id); + return; + } + + sess->sess_data[driver_id].data = private_data; } static inline void * diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map b/lib/librte_cryptodev/rte_cryptodev_version.map index a695b61dc..0d0f3bc28 100644 --- a/lib/librte_cryptodev/rte_cryptodev_version.map +++ b/lib/librte_cryptodev/rte_cryptodev_version.map @@ -105,4 +105,6 @@ EXPERIMENTAL { rte_cryptodev_sym_session_set_user_data; rte_crypto_asym_op_strings; rte_crypto_asym_xform_strings; + rte_cryptodev_sym_session_pool_create; + rte_cryptodev_sym_get_existing_header_session_size; }; diff --git a/lib/librte_vhost/rte_vhost_crypto.h b/lib/librte_vhost/rte_vhost_crypto.h index f9fbc0548..d08e0ffab 100644 --- a/lib/librte_vhost/rte_vhost_crypto.h +++ b/lib/librte_vhost/rte_vhost_crypto.h @@ -26,8 +26,9 @@ enum rte_vhost_crypto_zero_copy { * The identifier of DPDK Cryptodev, the same cryptodev_id can be assigned to * multiple Vhost-crypto devices. * @param sess_pool - * The pointer to the created cryptodev session pool with the private data size - * matches the target DPDK Cryptodev. + * The pointer to the created cryptodev session pool. + * @param sess_priv_pool + * The pointer to the created cryptodev session private data mempool. * @param socket_id * NUMA Socket ID to allocate resources on. * * @return @@ -36,7 +37,9 @@ enum rte_vhost_crypto_zero_copy { */ int __rte_experimental rte_vhost_crypto_create(int vid, uint8_t cryptodev_id, - struct rte_mempool *sess_pool, int socket_id); + struct rte_mempool *sess_pool, + struct rte_mempool *sess_priv_pool, + int socket_id); /** * Free the Vhost-crypto instance diff --git a/lib/librte_vhost/vhost_crypto.c b/lib/librte_vhost/vhost_crypto.c index dd01afc08..598196fb7 100644 --- a/lib/librte_vhost/vhost_crypto.c +++ b/lib/librte_vhost/vhost_crypto.c @@ -198,6 +198,7 @@ struct vhost_crypto { struct rte_hash *session_map; struct rte_mempool *mbuf_pool; struct rte_mempool *sess_pool; + struct rte_mempool *sess_priv_pool; struct rte_mempool *wb_pool; /** DPDK cryptodev ID */ @@ -369,7 +370,7 @@ vhost_crypto_create_sess(struct vhost_crypto *vcrypto, } if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1, - vcrypto->sess_pool) < 0) { + vcrypto->sess_priv_pool) < 0) { VC_LOG_ERR("Failed to initialize session"); sess_param->session_id = -VIRTIO_CRYPTO_ERR; return; @@ -1293,7 +1294,9 @@ vhost_crypto_complete_one_vm_requests(struct rte_crypto_op **ops, int __rte_experimental rte_vhost_crypto_create(int vid, uint8_t cryptodev_id, - struct rte_mempool *sess_pool, int socket_id) + struct rte_mempool *sess_pool, + struct rte_mempool *sess_priv_pool, + int socket_id) { struct virtio_net *dev = get_device(vid); struct rte_hash_parameters params = {0}; @@ -1321,6 +1324,7 @@ rte_vhost_crypto_create(int vid, uint8_t cryptodev_id, } vcrypto->sess_pool = sess_pool; + vcrypto->sess_priv_pool = sess_priv_pool; vcrypto->cid = cryptodev_id; vcrypto->cache_session_id = UINT64_MAX; vcrypto->last_session_id = 1; diff --git a/test/test/test_cryptodev.c b/test/test/test_cryptodev.c index aac0b1f0b..b66e73486 100644 --- a/test/test/test_cryptodev.c +++ b/test/test/test_cryptodev.c @@ -48,6 +48,7 @@ struct crypto_testsuite_params { struct rte_mempool *large_mbuf_pool; struct rte_mempool *op_mpool; struct rte_mempool *session_mpool; + struct rte_mempool *session_priv_mpool; struct rte_cryptodev_config conf; struct rte_cryptodev_qp_conf qp_conf; @@ -444,17 +445,24 @@ testsuite_setup(void) return TEST_FAILED; } - ts_params->session_mpool = rte_mempool_create( - "test_sess_mp", - MAX_NB_SESSIONS * 2, - session_size, - 0, 0, NULL, NULL, NULL, - NULL, SOCKET_ID_ANY, - 0); - + ts_params->session_mpool = rte_cryptodev_sym_session_pool_create( + "test_sess_mp", MAX_NB_SESSIONS, 0, 0, 0, + SOCKET_ID_ANY); TEST_ASSERT_NOT_NULL(ts_params->session_mpool, "session mempool allocation failed"); + ts_params->session_priv_mpool = rte_mempool_create( + "test_sess_mp_priv", + MAX_NB_SESSIONS, + session_size, + 0, 0, NULL, NULL, NULL, + NULL, SOCKET_ID_ANY, + 0); + TEST_ASSERT_NOT_NULL(ts_params->session_priv_mpool, + "session mempool allocation failed"); + + + TEST_ASSERT_SUCCESS(rte_cryptodev_configure(dev_id, &ts_params->conf), "Failed to configure cryptodev %u with %u qps", @@ -462,7 +470,7 @@ testsuite_setup(void) ts_params->qp_conf.nb_descriptors = DEFAULT_NUM_OPS_INFLIGHT; ts_params->qp_conf.mp_session = ts_params->session_mpool; - ts_params->qp_conf.mp_session_private = ts_params->session_mpool; + ts_params->qp_conf.mp_session_private = ts_params->session_priv_mpool; for (qp_id = 0; qp_id < info.max_nb_queue_pairs; qp_id++) { TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( @@ -491,6 +499,11 @@ testsuite_teardown(void) } /* Free session mempools */ + if (ts_params->session_priv_mpool != NULL) { + rte_mempool_free(ts_params->session_priv_mpool); + ts_params->session_priv_mpool = NULL; + } + if (ts_params->session_mpool != NULL) { rte_mempool_free(ts_params->session_mpool); ts_params->session_mpool = NULL; @@ -710,7 +723,7 @@ test_queue_pair_descriptor_setup(void) */ qp_conf.nb_descriptors = MIN_NUM_OPS_INFLIGHT; /* min size*/ qp_conf.mp_session = ts_params->session_mpool; - qp_conf.mp_session_private = ts_params->session_mpool; + qp_conf.mp_session_private = ts_params->session_priv_mpool; for (qp_id = 0; qp_id < ts_params->conf.nb_queue_pairs; qp_id++) { TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( @@ -1337,7 +1350,7 @@ test_AES_CBC_HMAC_SHA1_encrypt_digest(void) /* Create crypto session*/ rte_cryptodev_sym_session_init(ts_params->valid_devs[0], ut_params->sess, &ut_params->cipher_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); /* Generate crypto op data structure */ @@ -1551,7 +1564,7 @@ test_AES_cipheronly_mb_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)), @@ -1570,7 +1583,7 @@ test_AES_docsis_mb_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)), @@ -1589,7 +1602,7 @@ test_AES_docsis_qat_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD)), @@ -1608,7 +1621,7 @@ test_DES_docsis_qat_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD)), @@ -1627,7 +1640,7 @@ test_authonly_mb_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)), @@ -1646,7 +1659,7 @@ test_authonly_qat_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD)), @@ -1664,7 +1677,7 @@ test_AES_chain_mb_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)), @@ -1685,7 +1698,7 @@ test_AES_cipheronly_scheduler_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD)), @@ -1704,7 +1717,7 @@ test_AES_chain_scheduler_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD)), @@ -1723,7 +1736,7 @@ test_authonly_scheduler_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_SCHEDULER_PMD)), @@ -1744,7 +1757,7 @@ test_AES_chain_openssl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)), @@ -1763,7 +1776,7 @@ test_AES_cipheronly_openssl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)), @@ -1782,7 +1795,7 @@ test_AES_chain_ccp_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CCP_PMD)), @@ -1801,7 +1814,7 @@ test_AES_cipheronly_ccp_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CCP_PMD)), @@ -1820,7 +1833,7 @@ test_AES_chain_qat_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD)), @@ -1839,7 +1852,7 @@ test_AES_cipheronly_qat_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD)), @@ -1858,7 +1871,7 @@ test_AES_cipheronly_virtio_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_VIRTIO_PMD)), @@ -1877,7 +1890,7 @@ test_AES_chain_caam_jr_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CAAM_JR_PMD)), @@ -1896,7 +1909,7 @@ test_AES_cipheronly_caam_jr_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CAAM_JR_PMD)), @@ -1915,7 +1928,7 @@ test_authonly_caam_jr_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CAAM_JR_PMD)), @@ -1935,7 +1948,7 @@ test_AES_chain_dpaa_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA_SEC_PMD)), @@ -1954,7 +1967,7 @@ test_AES_cipheronly_dpaa_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA_SEC_PMD)), @@ -1973,7 +1986,7 @@ test_authonly_dpaa_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA_SEC_PMD)), @@ -1992,7 +2005,7 @@ test_AES_chain_dpaa2_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA2_SEC_PMD)), @@ -2011,7 +2024,7 @@ test_AES_cipheronly_dpaa2_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA2_SEC_PMD)), @@ -2030,7 +2043,7 @@ test_authonly_dpaa2_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA2_SEC_PMD)), @@ -2049,7 +2062,7 @@ test_authonly_openssl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)), @@ -2068,7 +2081,7 @@ test_authonly_ccp_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CCP_PMD)), @@ -2087,7 +2100,7 @@ test_AES_chain_armv8_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_ARMV8_PMD)), @@ -2106,7 +2119,7 @@ test_AES_chain_mrvl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_MVSAM_PMD)), @@ -2125,7 +2138,7 @@ test_AES_cipheronly_mrvl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_MVSAM_PMD)), @@ -2144,7 +2157,7 @@ test_authonly_mrvl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_MVSAM_PMD)), @@ -2163,7 +2176,7 @@ test_3DES_chain_mrvl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_MVSAM_PMD)), @@ -2182,7 +2195,7 @@ test_3DES_cipheronly_mrvl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_MVSAM_PMD)), @@ -2201,6 +2214,7 @@ test_AES_chain_octeontx_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, ts_params->session_mpool, + ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OCTEONTX_SYM_PMD)), @@ -2219,6 +2233,7 @@ test_AES_cipheronly_octeontx_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, ts_params->session_mpool, + ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OCTEONTX_SYM_PMD)), @@ -2237,6 +2252,7 @@ test_3DES_chain_octeontx_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, ts_params->session_mpool, + ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OCTEONTX_SYM_PMD)), @@ -2255,6 +2271,7 @@ test_3DES_cipheronly_octeontx_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, ts_params->session_mpool, + ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OCTEONTX_SYM_PMD)), @@ -2273,6 +2290,7 @@ test_authonly_octeontx_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, ts_params->session_mpool, + ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OCTEONTX_SYM_PMD)), @@ -2315,7 +2333,8 @@ create_wireless_algo_hash_session(uint8_t dev_id, ts_params->session_mpool); rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->auth_xform, ts_params->session_mpool); + &ut_params->auth_xform, + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; } @@ -2352,7 +2371,8 @@ create_wireless_algo_cipher_session(uint8_t dev_id, ts_params->session_mpool); rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->cipher_xform, ts_params->session_mpool); + &ut_params->cipher_xform, + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; } @@ -2468,7 +2488,8 @@ create_wireless_algo_cipher_auth_session(uint8_t dev_id, ts_params->session_mpool); rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->cipher_xform, ts_params->session_mpool); + &ut_params->cipher_xform, + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; @@ -2527,7 +2548,8 @@ create_wireless_cipher_auth_session(uint8_t dev_id, ts_params->session_mpool); rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->cipher_xform, ts_params->session_mpool); + &ut_params->cipher_xform, + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; @@ -2589,7 +2611,8 @@ create_wireless_algo_auth_cipher_session(uint8_t dev_id, ts_params->session_mpool); rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->auth_xform, ts_params->session_mpool); + &ut_params->auth_xform, + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); @@ -5075,7 +5098,7 @@ test_3DES_chain_qat_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD)), @@ -5094,7 +5117,7 @@ test_DES_cipheronly_qat_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD)), @@ -5113,7 +5136,7 @@ test_DES_cipheronly_openssl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)), @@ -5132,7 +5155,7 @@ test_DES_docsis_openssl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)), @@ -5151,7 +5174,7 @@ test_DES_cipheronly_mb_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)), @@ -5169,7 +5192,7 @@ test_3DES_cipheronly_mb_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)), @@ -5188,7 +5211,7 @@ test_DES_docsis_mb_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD)), @@ -5207,7 +5230,7 @@ test_3DES_chain_caam_jr_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CAAM_JR_PMD)), @@ -5226,7 +5249,7 @@ test_3DES_cipheronly_caam_jr_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CAAM_JR_PMD)), @@ -5245,7 +5268,7 @@ test_3DES_chain_dpaa_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA_SEC_PMD)), @@ -5264,7 +5287,7 @@ test_3DES_cipheronly_dpaa_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA_SEC_PMD)), @@ -5283,7 +5306,7 @@ test_3DES_chain_dpaa2_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA2_SEC_PMD)), @@ -5302,7 +5325,7 @@ test_3DES_cipheronly_dpaa2_sec_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_DPAA2_SEC_PMD)), @@ -5321,7 +5344,7 @@ test_3DES_chain_ccp_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CCP_PMD)), @@ -5340,7 +5363,7 @@ test_3DES_cipheronly_ccp_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_CCP_PMD)), @@ -5359,7 +5382,7 @@ test_3DES_cipheronly_qat_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD)), @@ -5378,7 +5401,7 @@ test_3DES_chain_openssl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)), @@ -5397,7 +5420,7 @@ test_3DES_cipheronly_openssl_all(void) status = test_blockcipher_all_tests(ts_params->mbuf_pool, ts_params->op_mpool, - ts_params->session_mpool, + ts_params->session_mpool, ts_params->session_priv_mpool, ts_params->valid_devs[0], rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_OPENSSL_PMD)), @@ -5443,7 +5466,8 @@ create_aead_session(uint8_t dev_id, enum rte_crypto_aead_algorithm algo, ts_params->session_mpool); rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->aead_xform, ts_params->session_mpool); + &ut_params->aead_xform, + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); @@ -6728,7 +6752,7 @@ test_multi_session(void) rte_cryptodev_sym_session_init(ts_params->valid_devs[0], sessions[i], &ut_params->auth_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(sessions[i], "Session creation failed at session number %u", i); @@ -6766,7 +6790,7 @@ test_multi_session(void) /* Next session create should fail */ rte_cryptodev_sym_session_init(ts_params->valid_devs[0], sessions[i], &ut_params->auth_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NULL(sessions[i], "Session creation succeeded unexpectedly!"); @@ -6847,7 +6871,7 @@ test_multi_session_random_usage(void) ts_params->valid_devs[0], sessions[i], &ut_paramz[i].ut_params.auth_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(sessions[i], "Session creation failed at session number %u", @@ -6925,7 +6949,7 @@ test_null_cipher_only_operation(void) rte_cryptodev_sym_session_init(ts_params->valid_devs[0], ut_params->sess, &ut_params->cipher_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); /* Generate Crypto op data structure */ @@ -6998,7 +7022,7 @@ test_null_auth_only_operation(void) /* Create Crypto session*/ rte_cryptodev_sym_session_init(ts_params->valid_devs[0], ut_params->sess, &ut_params->auth_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); /* Generate Crypto op data structure */ @@ -7077,7 +7101,7 @@ test_null_cipher_auth_operation(void) /* Create Crypto session*/ rte_cryptodev_sym_session_init(ts_params->valid_devs[0], ut_params->sess, &ut_params->cipher_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); /* Generate Crypto op data structure */ @@ -7165,7 +7189,7 @@ test_null_auth_cipher_operation(void) /* Create Crypto session*/ rte_cryptodev_sym_session_init(ts_params->valid_devs[0], ut_params->sess, &ut_params->cipher_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); /* Generate Crypto op data structure */ @@ -7235,7 +7259,7 @@ test_null_invalid_operation(void) /* Create Crypto session*/ ret = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], ut_params->sess, &ut_params->cipher_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT(ret < 0, "Session creation succeeded unexpectedly"); @@ -7253,7 +7277,7 @@ test_null_invalid_operation(void) /* Create Crypto session*/ ret = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], ut_params->sess, &ut_params->auth_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT(ret < 0, "Session creation succeeded unexpectedly"); @@ -7294,7 +7318,7 @@ test_null_burst_operation(void) /* Create Crypto session*/ rte_cryptodev_sym_session_init(ts_params->valid_devs[0], ut_params->sess, &ut_params->cipher_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); TEST_ASSERT_EQUAL(rte_crypto_op_bulk_alloc(ts_params->op_mpool, @@ -7429,7 +7453,7 @@ static int create_gmac_session(uint8_t dev_id, rte_cryptodev_sym_session_init(dev_id, ut_params->sess, &ut_params->auth_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); @@ -7809,7 +7833,7 @@ create_auth_session(struct crypto_unittest_params *ut_params, rte_cryptodev_sym_session_init(dev_id, ut_params->sess, &ut_params->auth_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); @@ -7862,7 +7886,7 @@ create_auth_cipher_session(struct crypto_unittest_params *ut_params, rte_cryptodev_sym_session_init(dev_id, ut_params->sess, &ut_params->auth_xform, - ts_params->session_mpool); + ts_params->session_priv_mpool); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); @@ -8740,12 +8764,14 @@ test_scheduler_attach_slave_op(void) for (i = 0; i < rte_cryptodev_count() && nb_devs_attached < 2; i++) { struct rte_cryptodev_info info; + unsigned int session_size; rte_cryptodev_info_get(i, &info); if (info.driver_id != rte_cryptodev_driver_id_get( RTE_STR(CRYPTODEV_NAME_AESNI_MB_PMD))) continue; + session_size = rte_cryptodev_sym_get_private_session_size(i); /* * Create the session mempool again, since now there are new devices * to use the mempool. @@ -8754,8 +8780,10 @@ test_scheduler_attach_slave_op(void) rte_mempool_free(ts_params->session_mpool); ts_params->session_mpool = NULL; } - unsigned int session_size = - rte_cryptodev_sym_get_private_session_size(i); + if (ts_params->session_priv_mpool) { + rte_mempool_free(ts_params->session_priv_mpool); + ts_params->session_priv_mpool = NULL; + } if (info.sym.max_nb_sessions != 0 && info.sym.max_nb_sessions < MAX_NB_SESSIONS) { @@ -8766,19 +8794,33 @@ test_scheduler_attach_slave_op(void) return TEST_FAILED; } /* - * Create mempool with maximum number of sessions * 2, + * Create mempool with maximum number of sessions, * to include the session headers */ if (ts_params->session_mpool == NULL) { - ts_params->session_mpool = rte_mempool_create( - "test_sess_mp", - MAX_NB_SESSIONS * 2, + ts_params->session_mpool = + rte_cryptodev_sym_session_pool_create( + "test_sess_mp", + MAX_NB_SESSIONS, 0, 0, 0, + SOCKET_ID_ANY); + TEST_ASSERT_NOT_NULL(ts_params->session_mpool, + "session mempool allocation failed"); + } + + /* + * Create mempool with maximum number of sessions, + * to include device specific session private data + */ + if (ts_params->session_priv_mpool == NULL) { + ts_params->session_priv_mpool = rte_mempool_create( + "test_sess_mp_priv", + MAX_NB_SESSIONS, session_size, 0, 0, NULL, NULL, NULL, NULL, SOCKET_ID_ANY, 0); - TEST_ASSERT_NOT_NULL(ts_params->session_mpool, + TEST_ASSERT_NOT_NULL(ts_params->session_priv_mpool, "session mempool allocation failed"); } diff --git a/test/test/test_cryptodev_blockcipher.c b/test/test/test_cryptodev_blockcipher.c index 1c3f29f6b..4d6f46a6e 100644 --- a/test/test/test_cryptodev_blockcipher.c +++ b/test/test/test_cryptodev_blockcipher.c @@ -25,6 +25,7 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, struct rte_mempool *mbuf_pool, struct rte_mempool *op_mpool, struct rte_mempool *sess_mpool, + struct rte_mempool *sess_priv_mpool, uint8_t dev_id, int driver_id, char *test_msg) @@ -347,7 +348,7 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, sess = rte_cryptodev_sym_session_create(sess_mpool); rte_cryptodev_sym_session_init(dev_id, sess, init_xform, - sess_mpool); + sess_priv_mpool); if (!sess) { snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, "line %u " "FAILED: %s", __LINE__, @@ -615,6 +616,7 @@ int test_blockcipher_all_tests(struct rte_mempool *mbuf_pool, struct rte_mempool *op_mpool, struct rte_mempool *sess_mpool, + struct rte_mempool *sess_priv_mpool, uint8_t dev_id, int driver_id, enum blockcipher_test_type test_type) @@ -730,7 +732,8 @@ test_blockcipher_all_tests(struct rte_mempool *mbuf_pool, continue; status = test_blockcipher_one_case(tc, mbuf_pool, op_mpool, - sess_mpool, dev_id, driver_id, test_msg); + sess_mpool, sess_priv_mpool, dev_id, driver_id, + test_msg); printf(" %u) TestCase %s %s\n", test_index ++, tc->test_descr, test_msg); diff --git a/test/test/test_cryptodev_blockcipher.h b/test/test/test_cryptodev_blockcipher.h index f8bd85838..5c22d5da6 100644 --- a/test/test/test_cryptodev_blockcipher.h +++ b/test/test/test_cryptodev_blockcipher.h @@ -104,6 +104,7 @@ int test_blockcipher_all_tests(struct rte_mempool *mbuf_pool, struct rte_mempool *op_mpool, struct rte_mempool *sess_mpool, + struct rte_mempool *sess_priv_mpool, uint8_t dev_id, int driver_id, enum blockcipher_test_type test_type); diff --git a/test/test/test_event_crypto_adapter.c b/test/test/test_event_crypto_adapter.c index 54717870e..f750ce3d8 100644 --- a/test/test/test_event_crypto_adapter.c +++ b/test/test/test_event_crypto_adapter.c @@ -61,6 +61,7 @@ struct event_crypto_adapter_test_params { struct rte_mempool *mbuf_pool; struct rte_mempool *op_mpool; struct rte_mempool *session_mpool; + struct rte_mempool *session_priv_mpool; struct rte_cryptodev_config *config; uint8_t crypto_event_port_id; }; @@ -193,12 +194,13 @@ test_op_forward_mode(uint8_t session_less) sym_op = op->sym; if (!session_less) { - sess = rte_cryptodev_sym_session_create(params.session_mpool); + sess = rte_cryptodev_sym_session_create( + params.session_mpool); TEST_ASSERT_NOT_NULL(sess, "Session creation failed\n"); /* Create Crypto session*/ rte_cryptodev_sym_session_init(TEST_CDEV_ID, sess, - &cipher_xform, params.session_mpool); + &cipher_xform, params.session_priv_mpool); ret = rte_event_crypto_adapter_caps_get(TEST_ADAPTER_ID, evdev, &cap); @@ -381,7 +383,8 @@ test_op_new_mode(uint8_t session_less) sym_op = op->sym; if (!session_less) { - sess = rte_cryptodev_sym_session_create(params.session_mpool); + sess = rte_cryptodev_sym_session_create( + params.session_mpool); TEST_ASSERT_NOT_NULL(sess, "Session creation failed\n"); ret = rte_event_crypto_adapter_caps_get(TEST_ADAPTER_ID, @@ -396,7 +399,7 @@ test_op_new_mode(uint8_t session_less) &m_data, sizeof(m_data)); } rte_cryptodev_sym_session_init(TEST_CDEV_ID, sess, - &cipher_xform, params.session_mpool); + &cipher_xform, params.session_priv_mpool); rte_crypto_op_attach_sym_session(op, sess); } else { struct rte_crypto_sym_xform *first_xform; @@ -526,15 +529,20 @@ configure_cryptodev(void) session_size = rte_cryptodev_sym_get_private_session_size(TEST_CDEV_ID); session_size += sizeof(union rte_event_crypto_metadata); - params.session_mpool = rte_mempool_create( - "CRYPTO_ADAPTER_SESSION_MP", - MAX_NB_SESSIONS * 2, + params.session_mpool = rte_cryptodev_sym_session_pool_create( + "CRYPTO_ADAPTER_SESSION_MP", + MAX_NB_SESSIONS, 0, 0, 0, SOCKET_ID_ANY); + TEST_ASSERT_NOT_NULL(params.session_mpool, + "session mempool allocation failed\n"); + + params.session_priv_mpool = rte_mempool_create( + "CRYPTO_ADAPTER_SESSION_MP_PRIV", + MAX_NB_SESSIONS, session_size, 0, 0, NULL, NULL, NULL, NULL, SOCKET_ID_ANY, 0); - - TEST_ASSERT_NOT_NULL(params.session_mpool, + TEST_ASSERT_NOT_NULL(params.session_priv_mpool, "session mempool allocation failed\n"); rte_cryptodev_info_get(TEST_CDEV_ID, &info); @@ -547,7 +555,7 @@ configure_cryptodev(void) qp_conf.nb_descriptors = DEFAULT_NUM_OPS_INFLIGHT; qp_conf.mp_session = params.session_mpool; - qp_conf.mp_session_private = params.session_mpool; + qp_conf.mp_session_private = params.session_priv_mpool; TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( TEST_CDEV_ID, TEST_CDEV_QP_ID, &qp_conf, @@ -859,6 +867,10 @@ crypto_teardown(void) rte_mempool_free(params.session_mpool); params.session_mpool = NULL; } + if (params.session_priv_mpool != NULL) { + rte_mempool_free(params.session_priv_mpool); + params.session_priv_mpool = NULL; + } /* Free ops mempool */ if (params.op_mpool != NULL) { -- 2.13.6