From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 77076A0542; Mon, 29 Aug 2022 18:11:22 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 568984069D; Mon, 29 Aug 2022 18:11:22 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mails.dpdk.org (Postfix) with ESMTP id 7C6B54003C for ; Mon, 29 Aug 2022 18:11:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1661789479; x=1693325479; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TJ+omq9OmPIRrcUfxnQfC4JdvFOnazfbYlSks2xGEbs=; b=jCw5zA87QE+CNLLG7OsQeUXDuqcS3lOFSoSWIKNiZuQMGYKgQZir6cFA zEdCMil4AwhfANIYCb9wBVba1NJh7FVP2mDVoxTYiWz2f4olEprVC7jBn tWxAXpkStXv3VZEwsr7PJvPx3Ksb0C0BYL0RTFmwO6EO1Rbv5uHWtyANu NyX1yLnZBTtTy1eJsyG46KW0aYt0axhTBrmiF5XXJ8tpWkccucn8AA6F3 a3324JLEZbW3p5uArN+wpTixNER+Uuy3y8BCnunNFt4RupklCbzR5W6aH /e+7vRjHqMMZ1THP8XYBsFqLSsSLgwIYNg/pLsV0DIw71ulC5ASJfa7G6 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10454"; a="358896763" X-IronPort-AV: E=Sophos;i="5.93,272,1654585200"; d="scan'208";a="358896763" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Aug 2022 09:06:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,272,1654585200"; d="scan'208";a="588211667" Received: from silpixa00400885.ir.intel.com ([10.243.22.192]) by orsmga006.jf.intel.com with ESMTP; 29 Aug 2022 09:06:54 -0700 From: Fan Zhang To: dev@dpdk.org Cc: gakhil@marvell.com, maxime.coquelin@redhat.com, chandu@amd.com, ruifeng.wang@arm.com, ajit.khaparde@broadcom.com, anoobj@marvell.com, pablo.de.lara.guarch@intel.com, matan@nvidia.com, g.singh@nxp.com, jianjay.zhou@huawei.com, Fan Zhang Subject: [PATCH 1/3] cryptodev: rework session framework Date: Mon, 29 Aug 2022 16:06:43 +0000 Message-Id: <20220829160645.378406-2-roy.fan.zhang@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220829160645.378406-1-roy.fan.zhang@intel.com> References: <20220829160645.378406-1-roy.fan.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org As per current design, rte_cryptodev_sym_session_create() and rte_cryptodev_sym_session_init() use separate mempool objects for a single session. And structure rte_cryptodev_sym_session is not directly used by the application, it may cause ABI breakage if the structure is modified in future. To address these two issues, the rte_cryptodev_sym_session_create will take one mempool object that the session and session private data are virtually/physically contiguous, and initializes both fields. The API rte_cryptodev_sym_session_init is removed. rte_cryptodev_sym_session_create will now return an opaque session pointer which will be used by the app and other APIs. In data path, opaque session pointer is attached to rte_crypto_op and the PMD can call an internal library API to get the session private data pointer based on the driver id. Note: currently single session may be used by different device drivers, given it is initialized by them. After the change the session created by one device driver cannot be used or reinitialized by another driver. Signed-off-by: Fan Zhang Signed-off-by: Akhil Goyal --- app/test-crypto-perf/cperf_ops.c | 21 +- app/test-crypto-perf/cperf_test_latency.c | 6 +- .../cperf_test_pmd_cyclecount.c | 5 +- app/test-crypto-perf/cperf_test_throughput.c | 6 +- app/test-crypto-perf/cperf_test_verify.c | 6 +- app/test-crypto-perf/main.c | 29 +- app/test-eventdev/test_perf_common.c | 35 +- app/test-eventdev/test_perf_common.h | 1 - app/test/test_cryptodev.c | 298 +++++------------- app/test/test_cryptodev_blockcipher.c | 16 +- app/test/test_event_crypto_adapter.c | 35 +- app/test/test_ipsec.c | 42 +-- drivers/crypto/armv8/rte_armv8_pmd.c | 21 +- drivers/crypto/armv8/rte_armv8_pmd_ops.c | 32 +- drivers/crypto/bcmfs/bcmfs_sym_session.c | 38 +-- drivers/crypto/bcmfs/bcmfs_sym_session.h | 3 +- drivers/crypto/caam_jr/caam_jr.c | 28 +- drivers/crypto/ccp/ccp_crypto.c | 58 +--- drivers/crypto/ccp/ccp_pmd_ops.c | 32 +- drivers/crypto/ccp/ccp_pmd_private.h | 2 - drivers/crypto/ccp/rte_ccp_pmd.c | 29 +- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 36 +-- drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 31 +- drivers/crypto/cnxk/cnxk_cryptodev_ops.c | 54 +--- drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 14 +- drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c | 31 +- drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c | 3 +- drivers/crypto/dpaa_sec/dpaa_sec.c | 37 +-- drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c | 4 +- drivers/crypto/ipsec_mb/ipsec_mb_ops.c | 34 +- drivers/crypto/ipsec_mb/ipsec_mb_private.h | 41 ++- drivers/crypto/ipsec_mb/pmd_aesni_gcm.c | 48 +-- drivers/crypto/ipsec_mb/pmd_aesni_mb.c | 26 +- drivers/crypto/ipsec_mb/pmd_chacha_poly.c | 4 - drivers/crypto/ipsec_mb/pmd_kasumi.c | 10 +- drivers/crypto/ipsec_mb/pmd_snow3g.c | 9 +- drivers/crypto/ipsec_mb/pmd_zuc.c | 4 - drivers/crypto/mlx5/mlx5_crypto.c | 26 +- drivers/crypto/mvsam/rte_mrvl_pmd.c | 8 +- drivers/crypto/mvsam/rte_mrvl_pmd_ops.c | 21 +- drivers/crypto/nitrox/nitrox_sym.c | 39 +-- drivers/crypto/null/null_crypto_pmd.c | 19 +- drivers/crypto/null/null_crypto_pmd_ops.c | 33 +- drivers/crypto/null/null_crypto_pmd_private.h | 2 - .../crypto/octeontx/otx_cryptodev_hw_access.h | 1 - drivers/crypto/octeontx/otx_cryptodev_ops.c | 67 +--- drivers/crypto/openssl/openssl_pmd_private.h | 2 - drivers/crypto/openssl/rte_openssl_pmd.c | 24 +- drivers/crypto/openssl/rte_openssl_pmd_ops.c | 29 +- drivers/crypto/qat/qat_sym.c | 10 +- drivers/crypto/qat/qat_sym.h | 4 +- drivers/crypto/qat/qat_sym_session.c | 40 +-- drivers/crypto/qat/qat_sym_session.h | 6 +- drivers/crypto/scheduler/scheduler_pmd_ops.c | 38 +-- drivers/crypto/virtio/virtio_cryptodev.c | 40 +-- drivers/crypto/virtio/virtio_rxtx.c | 3 +- examples/fips_validation/fips_dev_self_test.c | 30 +- examples/fips_validation/main.c | 35 +- examples/ipsec-secgw/ipsec-secgw.c | 10 +- examples/ipsec-secgw/ipsec.c | 7 +- examples/l2fwd-crypto/main.c | 54 +--- examples/vhost_crypto/main.c | 16 +- lib/cryptodev/cryptodev_pmd.h | 28 +- lib/cryptodev/cryptodev_trace_points.c | 6 - lib/cryptodev/rte_cryptodev.c | 272 ++++++---------- lib/cryptodev/rte_cryptodev.h | 123 ++------ lib/cryptodev/rte_cryptodev_trace.h | 33 +- lib/cryptodev/version.map | 6 - lib/pipeline/rte_table_action.c | 10 +- lib/vhost/rte_vhost_crypto.h | 3 - lib/vhost/vhost_crypto.c | 28 +- 71 files changed, 545 insertions(+), 1657 deletions(-) diff --git a/app/test-crypto-perf/cperf_ops.c b/app/test-crypto-perf/cperf_ops.c index d746d51082..c6f5735bb0 100644 --- a/app/test-crypto-perf/cperf_ops.c +++ b/app/test-crypto-perf/cperf_ops.c @@ -912,7 +912,6 @@ cperf_create_session(struct rte_mempool *sess_mp, &sess_conf, sess_mp, priv_mp); } #endif - sess = rte_cryptodev_sym_session_create(sess_mp); /* * cipher only */ @@ -937,8 +936,8 @@ cperf_create_session(struct rte_mempool *sess_mp, cipher_xform.cipher.iv.length = 0; } /* create crypto session */ - rte_cryptodev_sym_session_init(dev_id, sess, &cipher_xform, - priv_mp); + sess = rte_cryptodev_sym_session_create(dev_id, &cipher_xform, + sess_mp); /* * auth only */ @@ -965,8 +964,8 @@ cperf_create_session(struct rte_mempool *sess_mp, auth_xform.auth.iv.length = 0; } /* create crypto session */ - rte_cryptodev_sym_session_init(dev_id, sess, &auth_xform, - priv_mp); + sess = rte_cryptodev_sym_session_create(dev_id, &auth_xform, + sess_mp); /* * cipher and auth */ @@ -1024,13 +1023,13 @@ cperf_create_session(struct rte_mempool *sess_mp, if (options->op_type == CPERF_CIPHER_THEN_AUTH) { cipher_xform.next = &auth_xform; /* create crypto session */ - rte_cryptodev_sym_session_init(dev_id, - sess, &cipher_xform, priv_mp); + sess = rte_cryptodev_sym_session_create(dev_id, + &cipher_xform, sess_mp); } else { /* auth then cipher */ auth_xform.next = &cipher_xform; /* create crypto session */ - rte_cryptodev_sym_session_init(dev_id, - sess, &auth_xform, priv_mp); + sess = rte_cryptodev_sym_session_create(dev_id, + &auth_xform, sess_mp); } } else { /* options->op_type == CPERF_AEAD */ aead_xform.type = RTE_CRYPTO_SYM_XFORM_AEAD; @@ -1050,8 +1049,8 @@ cperf_create_session(struct rte_mempool *sess_mp, options->aead_aad_sz; /* Create crypto session */ - rte_cryptodev_sym_session_init(dev_id, - sess, &aead_xform, priv_mp); + sess = rte_cryptodev_sym_session_create(dev_id, &aead_xform, + sess_mp); } return sess; diff --git a/app/test-crypto-perf/cperf_test_latency.c b/app/test-crypto-perf/cperf_test_latency.c index 6f972cea49..afd8cb209b 100644 --- a/app/test-crypto-perf/cperf_test_latency.c +++ b/app/test-crypto-perf/cperf_test_latency.c @@ -44,10 +44,8 @@ static void cperf_latency_test_free(struct cperf_latency_ctx *ctx) { if (ctx) { - if (ctx->sess) { - rte_cryptodev_sym_session_clear(ctx->dev_id, ctx->sess); - rte_cryptodev_sym_session_free(ctx->sess); - } + if (ctx->sess) + rte_cryptodev_sym_session_free(ctx->dev_id, ctx->sess); rte_mempool_free(ctx->pool); diff --git a/app/test-crypto-perf/cperf_test_pmd_cyclecount.c b/app/test-crypto-perf/cperf_test_pmd_cyclecount.c index 6b4d09e623..7efeecf848 100644 --- a/app/test-crypto-perf/cperf_test_pmd_cyclecount.c +++ b/app/test-crypto-perf/cperf_test_pmd_cyclecount.c @@ -73,10 +73,7 @@ cperf_pmd_cyclecount_test_free(struct cperf_pmd_cyclecount_ctx *ctx) (struct rte_security_session *)ctx->sess); } else #endif - { - rte_cryptodev_sym_session_clear(ctx->dev_id, ctx->sess); - rte_cryptodev_sym_session_free(ctx->sess); - } + rte_cryptodev_sym_session_free(ctx->dev_id, ctx->sess); } rte_mempool_free(ctx->pool); diff --git a/app/test-crypto-perf/cperf_test_throughput.c b/app/test-crypto-perf/cperf_test_throughput.c index cecf30e470..24261a1d10 100644 --- a/app/test-crypto-perf/cperf_test_throughput.c +++ b/app/test-crypto-perf/cperf_test_throughput.c @@ -50,10 +50,8 @@ cperf_throughput_test_free(struct cperf_throughput_ctx *ctx) (struct rte_security_session *)ctx->sess); } #endif - else { - rte_cryptodev_sym_session_clear(ctx->dev_id, ctx->sess); - rte_cryptodev_sym_session_free(ctx->sess); - } + else + rte_cryptodev_sym_session_free(ctx->dev_id, ctx->sess); } rte_mempool_free(ctx->pool); diff --git a/app/test-crypto-perf/cperf_test_verify.c b/app/test-crypto-perf/cperf_test_verify.c index 5c0dc82290..54f7e8ba2f 100644 --- a/app/test-crypto-perf/cperf_test_verify.c +++ b/app/test-crypto-perf/cperf_test_verify.c @@ -37,10 +37,8 @@ static void cperf_verify_test_free(struct cperf_verify_ctx *ctx) { if (ctx) { - if (ctx->sess) { - rte_cryptodev_sym_session_clear(ctx->dev_id, ctx->sess); - rte_cryptodev_sym_session_free(ctx->sess); - } + if (ctx->sess) + rte_cryptodev_sym_session_free(ctx->dev_id, ctx->sess); rte_mempool_free(ctx->pool); diff --git a/app/test-crypto-perf/main.c b/app/test-crypto-perf/main.c index 17e30a8e74..55d7614d84 100644 --- a/app/test-crypto-perf/main.c +++ b/app/test-crypto-perf/main.c @@ -96,35 +96,14 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size, char mp_name[RTE_MEMPOOL_NAMESIZE]; struct rte_mempool *sess_mp; - if (session_pool_socket[socket_id].priv_mp == NULL) { - snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, - "priv_sess_mp_%u", socket_id); - - sess_mp = rte_mempool_create(mp_name, - nb_sessions, - session_priv_size, - 0, 0, NULL, NULL, NULL, - NULL, socket_id, - 0); - - if (sess_mp == NULL) { - printf("Cannot create pool \"%s\" on socket %d\n", - mp_name, socket_id); - return -ENOMEM; - } - - printf("Allocated pool \"%s\" on socket %d\n", - mp_name, socket_id); - session_pool_socket[socket_id].priv_mp = sess_mp; - } - if (session_pool_socket[socket_id].sess_mp == NULL) { snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, "sess_mp_%u", socket_id); sess_mp = rte_cryptodev_sym_session_pool_create(mp_name, - nb_sessions, 0, 0, 0, socket_id); + nb_sessions, session_priv_size, 0, 0, + socket_id); if (sess_mp == NULL) { printf("Cannot create pool \"%s\" on socket %d\n", @@ -135,6 +114,7 @@ fill_session_pool_socket(int32_t socket_id, uint32_t session_priv_size, printf("Allocated pool \"%s\" on socket %d\n", mp_name, socket_id); session_pool_socket[socket_id].sess_mp = sess_mp; + session_pool_socket[socket_id].priv_mp = sess_mp; } return 0; @@ -322,12 +302,9 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs) return ret; qp_conf.mp_session = session_pool_socket[socket_id].sess_mp; - qp_conf.mp_session_private = - session_pool_socket[socket_id].priv_mp; if (opts->op_type == CPERF_ASYM_MODEX) { qp_conf.mp_session = NULL; - qp_conf.mp_session_private = NULL; } ret = rte_cryptodev_configure(cdev_id, &conf); diff --git a/app/test-eventdev/test_perf_common.c b/app/test-eventdev/test_perf_common.c index 81420be73a..30cefcb6dd 100644 --- a/app/test-eventdev/test_perf_common.c +++ b/app/test-eventdev/test_perf_common.c @@ -864,18 +864,13 @@ cryptodev_sym_sess_create(struct prod_data *p, struct test_perf *t) cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; cipher_xform.next = NULL; - sess = rte_cryptodev_sym_session_create(t->ca_sess_pool); + sess = rte_cryptodev_sym_session_create(p->ca.cdev_id, &cipher_xform, + t->ca_sess_pool); if (sess == NULL) { evt_err("Failed to create sym session"); return NULL; } - if (rte_cryptodev_sym_session_init(p->ca.cdev_id, sess, &cipher_xform, - t->ca_sess_priv_pool)) { - evt_err("Failed to init session"); - return NULL; - } - return sess; } @@ -1382,15 +1377,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) goto err; } - t->ca_sess_pool = rte_cryptodev_sym_session_pool_create( - "ca_sess_pool", nb_sessions, 0, 0, - sizeof(union rte_event_crypto_metadata), SOCKET_ID_ANY); - if (t->ca_sess_pool == NULL) { - evt_err("Failed to create sym session pool"); - ret = -ENOMEM; - goto err; - } - max_session_size = 0; for (cdev_id = 0; cdev_id < cdev_count; cdev_id++) { unsigned int session_size; @@ -1401,12 +1387,11 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) max_session_size = session_size; } - max_session_size += sizeof(union rte_event_crypto_metadata); - t->ca_sess_priv_pool = rte_mempool_create( - "ca_sess_priv_pool", nb_sessions, max_session_size, 0, 0, NULL, - NULL, NULL, NULL, SOCKET_ID_ANY, 0); - if (t->ca_sess_priv_pool == NULL) { - evt_err("failed to create sym session private pool"); + t->ca_sess_pool = rte_cryptodev_sym_session_pool_create( + "ca_sess_pool", nb_sessions, max_session_size, 0, + sizeof(union rte_event_crypto_metadata), SOCKET_ID_ANY); + if (t->ca_sess_pool == NULL) { + evt_err("Failed to create sym session pool"); ret = -ENOMEM; goto err; } @@ -1446,7 +1431,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) qp_conf.nb_descriptors = NB_CRYPTODEV_DESCRIPTORS; qp_conf.mp_session = t->ca_sess_pool; - qp_conf.mp_session_private = t->ca_sess_priv_pool; for (qp_id = 0; qp_id < conf.nb_queue_pairs; qp_id++) { ret = rte_cryptodev_queue_pair_setup( @@ -1467,7 +1451,6 @@ perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt) rte_mempool_free(t->ca_op_pool); rte_mempool_free(t->ca_sess_pool); - rte_mempool_free(t->ca_sess_priv_pool); rte_mempool_free(t->ca_asym_sess_pool); return ret; @@ -1492,8 +1475,7 @@ perf_cryptodev_destroy(struct evt_test *test, struct evt_options *opt) for (flow_id = 0; flow_id < t->nb_flows; flow_id++) { sess = p->ca.crypto_sess[flow_id]; cdev_id = p->ca.cdev_id; - rte_cryptodev_sym_session_clear(cdev_id, sess); - rte_cryptodev_sym_session_free(sess); + rte_cryptodev_sym_session_free(cdev_id, sess); } rte_event_crypto_adapter_queue_pair_del( @@ -1509,7 +1491,6 @@ perf_cryptodev_destroy(struct evt_test *test, struct evt_options *opt) rte_mempool_free(t->ca_op_pool); rte_mempool_free(t->ca_sess_pool); - rte_mempool_free(t->ca_sess_priv_pool); rte_mempool_free(t->ca_asym_sess_pool); } diff --git a/app/test-eventdev/test_perf_common.h b/app/test-eventdev/test_perf_common.h index 8cbd06fe42..d06d52cdf8 100644 --- a/app/test-eventdev/test_perf_common.h +++ b/app/test-eventdev/test_perf_common.h @@ -70,7 +70,6 @@ struct test_perf { RTE_EVENT_TIMER_ADAPTER_NUM_MAX] __rte_cache_aligned; struct rte_mempool *ca_op_pool; struct rte_mempool *ca_sess_pool; - struct rte_mempool *ca_sess_priv_pool; struct rte_mempool *ca_asym_sess_pool; } __rte_cache_aligned; diff --git a/app/test/test_cryptodev.c b/app/test/test_cryptodev.c index 6ee4480399..c975c38001 100644 --- a/app/test/test_cryptodev.c +++ b/app/test/test_cryptodev.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -644,23 +645,17 @@ testsuite_setup(void) } ts_params->session_mpool = rte_cryptodev_sym_session_pool_create( - "test_sess_mp", MAX_NB_SESSIONS, 0, 0, 0, + "test_sess_mp", MAX_NB_SESSIONS, session_size, 0, 0, SOCKET_ID_ANY); TEST_ASSERT_NOT_NULL(ts_params->session_mpool, "session mempool allocation failed"); - ts_params->session_priv_mpool = rte_mempool_create( - "test_sess_mp_priv", - MAX_NB_SESSIONS, - session_size, - 0, 0, NULL, NULL, NULL, - NULL, SOCKET_ID_ANY, - 0); + "test_sess_mp_priv", MAX_NB_SESSIONS, session_size, + 0, 0, NULL, NULL, NULL, NULL, SOCKET_ID_ANY, 0); + TEST_ASSERT_NOT_NULL(ts_params->session_priv_mpool, "session mempool allocation failed"); - - TEST_ASSERT_SUCCESS(rte_cryptodev_configure(dev_id, &ts_params->conf), "Failed to configure cryptodev %u with %u qps", @@ -668,7 +663,6 @@ testsuite_setup(void) ts_params->qp_conf.nb_descriptors = MAX_NUM_OPS_INFLIGHT; ts_params->qp_conf.mp_session = ts_params->session_mpool; - ts_params->qp_conf.mp_session_private = ts_params->session_priv_mpool; for (qp_id = 0; qp_id < info.max_nb_queue_pairs; qp_id++) { TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( @@ -697,15 +691,11 @@ testsuite_teardown(void) rte_mempool_avail_count(ts_params->op_mpool)); } - /* Free session mempools */ - if (ts_params->session_priv_mpool != NULL) { - rte_mempool_free(ts_params->session_priv_mpool); - ts_params->session_priv_mpool = NULL; - } - if (ts_params->session_mpool != NULL) { rte_mempool_free(ts_params->session_mpool); ts_params->session_mpool = NULL; + rte_mempool_free(ts_params->session_priv_mpool); + ts_params->session_priv_mpool = NULL; } res = rte_cryptodev_close(ts_params->valid_devs[0]); @@ -1381,7 +1371,6 @@ dev_configure_and_start(uint64_t ff_disable) ts_params->conf.ff_disable = ff_disable; ts_params->qp_conf.nb_descriptors = MAX_NUM_OPS_INFLIGHT; ts_params->qp_conf.mp_session = ts_params->session_mpool; - ts_params->qp_conf.mp_session_private = ts_params->session_priv_mpool; TEST_ASSERT_SUCCESS(rte_cryptodev_configure(ts_params->valid_devs[0], &ts_params->conf), @@ -1441,10 +1430,8 @@ ut_teardown(void) #endif { if (ut_params->sess) { - rte_cryptodev_sym_session_clear( - ts_params->valid_devs[0], + rte_cryptodev_sym_session_free(ts_params->valid_devs[0], ut_params->sess); - rte_cryptodev_sym_session_free(ut_params->sess); ut_params->sess = NULL; } } @@ -1599,7 +1586,6 @@ test_queue_pair_descriptor_setup(void) */ qp_conf.nb_descriptors = MIN_NUM_OPS_INFLIGHT; /* min size*/ qp_conf.mp_session = ts_params->session_mpool; - qp_conf.mp_session_private = ts_params->session_priv_mpool; for (qp_id = 0; qp_id < ts_params->conf.nb_queue_pairs; qp_id++) { TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( @@ -2144,8 +2130,6 @@ test_AES_CBC_HMAC_SHA1_encrypt_digest(void) { struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; - int status; - /* Verify the capabilities */ struct rte_cryptodev_sym_capability_idx cap_idx; cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; @@ -2190,18 +2174,11 @@ test_AES_CBC_HMAC_SHA1_encrypt_digest(void) ut_params->auth_xform.auth.digest_length = DIGEST_BYTE_LENGTH_SHA1; ut_params->sess = rte_cryptodev_sym_session_create( + ts_params->valid_devs[0], &ut_params->cipher_xform, ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - - /* Create crypto session*/ - status = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], - ut_params->sess, &ut_params->cipher_xform, - ts_params->session_priv_mpool); - - if (status == -ENOTSUP) + if (rte_errno == ENOTSUP) return TEST_SKIPPED; - - TEST_ASSERT_EQUAL(status, 0, "Session init failed"); + TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); /* Generate crypto op data structure */ ut_params->op = rte_crypto_op_alloc(ts_params->op_mpool, @@ -2430,7 +2407,6 @@ create_wireless_algo_hash_session(uint8_t dev_id, enum rte_crypto_auth_algorithm algo) { uint8_t hash_key[key_len]; - int status; struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; @@ -2450,16 +2426,11 @@ create_wireless_algo_hash_session(uint8_t dev_id, ut_params->auth_xform.auth.digest_length = auth_len; ut_params->auth_xform.auth.iv.offset = IV_OFFSET; ut_params->auth_xform.auth.iv.length = iv_len; - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); - - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->auth_xform, - ts_params->session_priv_mpool); - if (status == -ENOTSUP) + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, + &ut_params->auth_xform, ts_params->session_mpool); + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - TEST_ASSERT_EQUAL(status, 0, "session init failed"); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; } @@ -2472,7 +2443,6 @@ create_wireless_algo_cipher_session(uint8_t dev_id, uint8_t iv_len) { uint8_t cipher_key[key_len]; - int status; struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; @@ -2492,16 +2462,12 @@ create_wireless_algo_cipher_session(uint8_t dev_id, debug_hexdump(stdout, "key:", key, key_len); /* Create Crypto session */ - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, + &ut_params->cipher_xform, ts_params->session_mpool); - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->cipher_xform, - ts_params->session_priv_mpool); - if (status == -ENOTSUP) + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - TEST_ASSERT_EQUAL(status, 0, "session init failed"); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; } @@ -2579,7 +2545,6 @@ create_wireless_algo_cipher_auth_session(uint8_t dev_id, { uint8_t cipher_auth_key[key_len]; - int status; struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; @@ -2614,17 +2579,12 @@ create_wireless_algo_cipher_auth_session(uint8_t dev_id, debug_hexdump(stdout, "key:", key, key_len); /* Create Crypto session*/ - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->cipher_xform, - ts_params->session_priv_mpool); - if (status == -ENOTSUP) + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, + &ut_params->cipher_xform, ts_params->session_mpool); + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - TEST_ASSERT_EQUAL(status, 0, "session init failed"); + TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; } @@ -2638,7 +2598,6 @@ create_wireless_cipher_auth_session(uint8_t dev_id, { const uint8_t key_len = tdata->key.len; uint8_t cipher_auth_key[key_len]; - int status; struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; @@ -2678,16 +2637,11 @@ create_wireless_cipher_auth_session(uint8_t dev_id, debug_hexdump(stdout, "key:", key, key_len); /* Create Crypto session*/ - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); - - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->cipher_xform, - ts_params->session_priv_mpool); - if (status == -ENOTSUP) + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, + &ut_params->cipher_xform, ts_params->session_mpool); + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - TEST_ASSERT_EQUAL(status, 0, "session init failed"); TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; } @@ -2713,7 +2667,6 @@ create_wireless_algo_auth_cipher_session(uint8_t dev_id, uint8_t cipher_iv_len) { uint8_t auth_cipher_key[key_len]; - int status; struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; @@ -2744,26 +2697,19 @@ create_wireless_algo_auth_cipher_session(uint8_t dev_id, debug_hexdump(stdout, "key:", key, key_len); /* Create Crypto session*/ - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - if (cipher_op == RTE_CRYPTO_CIPHER_OP_DECRYPT) { ut_params->auth_xform.next = NULL; ut_params->cipher_xform.next = &ut_params->auth_xform; - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->cipher_xform, - ts_params->session_priv_mpool); - + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, + &ut_params->cipher_xform, ts_params->session_mpool); } else - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->auth_xform, - ts_params->session_priv_mpool); + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, + &ut_params->auth_xform, ts_params->session_mpool); - if (status == -ENOTSUP) + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - TEST_ASSERT_EQUAL(status, 0, "session init failed"); + TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); return 0; } @@ -8194,7 +8140,6 @@ create_aead_session(uint8_t dev_id, enum rte_crypto_aead_algorithm algo, uint8_t iv_len) { uint8_t aead_key[key_len]; - int status; struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; @@ -8216,15 +8161,12 @@ create_aead_session(uint8_t dev_id, enum rte_crypto_aead_algorithm algo, debug_hexdump(stdout, "key:", key, key_len); /* Create Crypto session*/ - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, + &ut_params->aead_xform, ts_params->session_mpool); + if (ut_params->sess == NULL && rte_errno == ENOTSUP) + return TEST_SKIPPED; TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->aead_xform, - ts_params->session_priv_mpool); - - return status; + return 0; } static int @@ -8668,7 +8610,7 @@ static int test_pdcp_proto(int i, int oop, enum rte_crypto_cipher_operation opc, /* Create security session */ ut_params->sec_session = rte_security_session_create(ctx, &sess_conf, ts_params->session_mpool, - ts_params->session_priv_mpool); + NULL); if (!ut_params->sec_session) { printf("TestCase %s()-%d line %d failed %s: ", @@ -12018,7 +11960,6 @@ static int MD5_HMAC_create_session(struct crypto_testsuite_params *ts_params, const struct HMAC_MD5_vector *test_case) { uint8_t key[64]; - int status; memcpy(key, test_case->key.data, test_case->key.len); @@ -12033,16 +11974,11 @@ static int MD5_HMAC_create_session(struct crypto_testsuite_params *ts_params, ut_params->auth_xform.auth.key.data = key; ut_params->sess = rte_cryptodev_sym_session_create( + ts_params->valid_devs[0], &ut_params->auth_xform, ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - if (ut_params->sess == NULL) - return TEST_FAILED; - - status = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], - ut_params->sess, &ut_params->auth_xform, - ts_params->session_priv_mpool); - if (status == -ENOTSUP) + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; + TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); @@ -12250,12 +12186,9 @@ test_multi_session(void) { struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; - struct rte_cryptodev_info dev_info; struct rte_cryptodev_sym_session **sessions; - uint16_t i; - int status; /* Verify the capabilities */ struct rte_cryptodev_sym_capability_idx cap_idx; @@ -12282,20 +12215,15 @@ test_multi_session(void) /* Create multiple crypto sessions*/ for (i = 0; i < MAX_NB_SESSIONS; i++) { - sessions[i] = rte_cryptodev_sym_session_create( + ts_params->valid_devs[0], &ut_params->auth_xform, ts_params->session_mpool); + if (sessions[i] == NULL && rte_errno == ENOTSUP) + return TEST_SKIPPED; + TEST_ASSERT_NOT_NULL(sessions[i], "Session creation failed at session number %u", i); - - status = rte_cryptodev_sym_session_init( - ts_params->valid_devs[0], - sessions[i], &ut_params->auth_xform, - ts_params->session_priv_mpool); - if (status == -ENOTSUP) - return TEST_SKIPPED; - /* Attempt to send a request on each session */ TEST_ASSERT_SUCCESS( test_AES_CBC_HMAC_SHA512_decrypt_perform( sessions[i], @@ -12325,18 +12253,9 @@ test_multi_session(void) } } - sessions[i] = NULL; - /* Next session create should fail */ - rte_cryptodev_sym_session_init(ts_params->valid_devs[0], - sessions[i], &ut_params->auth_xform, - ts_params->session_priv_mpool); - TEST_ASSERT_NULL(sessions[i], - "Session creation succeeded unexpectedly!"); - for (i = 0; i < MAX_NB_SESSIONS; i++) { - rte_cryptodev_sym_session_clear(ts_params->valid_devs[0], + rte_cryptodev_sym_session_free(ts_params->valid_devs[0], sessions[i]); - rte_cryptodev_sym_session_free(sessions[i]); } rte_free(sessions); @@ -12387,7 +12306,6 @@ test_multi_session_random_usage(void) }, }; - int status; /* Verify the capabilities */ struct rte_cryptodev_sym_capability_idx cap_idx; @@ -12409,11 +12327,6 @@ test_multi_session_random_usage(void) * MAX_NB_SESSIONS) + 1, 0); for (i = 0; i < MB_SESSION_NUMBER; i++) { - sessions[i] = rte_cryptodev_sym_session_create( - ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(sessions[i], - "Session creation failed at session number %u", - i); rte_memcpy(&ut_paramz[i].ut_params, &unittest_params, sizeof(struct crypto_unittest_params)); @@ -12423,16 +12336,16 @@ test_multi_session_random_usage(void) ut_paramz[i].cipher_key, ut_paramz[i].hmac_key); /* Create multiple crypto sessions*/ - status = rte_cryptodev_sym_session_init( + sessions[i] = rte_cryptodev_sym_session_create( ts_params->valid_devs[0], - sessions[i], &ut_paramz[i].ut_params.auth_xform, - ts_params->session_priv_mpool); - - if (status == -ENOTSUP) + ts_params->session_mpool); + if (sessions[i] == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - TEST_ASSERT_EQUAL(status, 0, "Session init failed"); + TEST_ASSERT_NOT_NULL(sessions[i], + "Session creation failed at session number %u", + i); } srand(time(NULL)); @@ -12470,9 +12383,8 @@ test_multi_session_random_usage(void) } for (i = 0; i < MB_SESSION_NUMBER; i++) { - rte_cryptodev_sym_session_clear(ts_params->valid_devs[0], + rte_cryptodev_sym_session_free(ts_params->valid_devs[0], sessions[i]); - rte_cryptodev_sym_session_free(sessions[i]); } rte_free(sessions); @@ -12490,7 +12402,6 @@ test_null_invalid_operation(void) { struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; - int ret; /* This test is for NULL PMD only */ if (gbl_driver_id != rte_cryptodev_driver_id_get( @@ -12504,17 +12415,13 @@ test_null_invalid_operation(void) ut_params->cipher_xform.cipher.algo = RTE_CRYPTO_CIPHER_AES_CBC; ut_params->cipher_xform.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + /* Create Crypto session*/ ut_params->sess = rte_cryptodev_sym_session_create( + ts_params->valid_devs[0], &ut_params->cipher_xform, ts_params->session_mpool); - - /* Create Crypto session*/ - ret = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], - ut_params->sess, &ut_params->cipher_xform, - ts_params->session_priv_mpool); - TEST_ASSERT(ret < 0, + TEST_ASSERT(ut_params->sess == NULL, "Session creation succeeded unexpectedly"); - /* Setup HMAC Parameters */ ut_params->auth_xform.type = RTE_CRYPTO_SYM_XFORM_AUTH; ut_params->auth_xform.next = NULL; @@ -12522,14 +12429,11 @@ test_null_invalid_operation(void) ut_params->auth_xform.auth.algo = RTE_CRYPTO_AUTH_SHA1_HMAC; ut_params->auth_xform.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + /* Create Crypto session*/ ut_params->sess = rte_cryptodev_sym_session_create( + ts_params->valid_devs[0], &ut_params->auth_xform, ts_params->session_mpool); - - /* Create Crypto session*/ - ret = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], - ut_params->sess, &ut_params->auth_xform, - ts_params->session_priv_mpool); - TEST_ASSERT(ret < 0, + TEST_ASSERT(ut_params->sess == NULL, "Session creation succeeded unexpectedly"); return TEST_SUCCESS; @@ -12543,7 +12447,6 @@ test_null_burst_operation(void) { struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; - int status; unsigned i, burst_len = NULL_BURST_LENGTH; @@ -12569,19 +12472,14 @@ test_null_burst_operation(void) ut_params->auth_xform.auth.algo = RTE_CRYPTO_AUTH_NULL; ut_params->auth_xform.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - /* Create Crypto session*/ - status = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], - ut_params->sess, &ut_params->cipher_xform, - ts_params->session_priv_mpool); - - if (status == -ENOTSUP) + ut_params->sess = rte_cryptodev_sym_session_create( + ts_params->valid_devs[0], + &ut_params->auth_xform, + ts_params->session_mpool); + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - - TEST_ASSERT_EQUAL(status, 0, "Session init failed"); + TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); TEST_ASSERT_EQUAL(rte_crypto_op_bulk_alloc(ts_params->op_mpool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, burst, burst_len), @@ -12692,7 +12590,6 @@ test_enq_callback_setup(void) qp_conf.nb_descriptors = MAX_NUM_OPS_INFLIGHT; qp_conf.mp_session = ts_params->session_mpool; - qp_conf.mp_session_private = ts_params->session_priv_mpool; TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( ts_params->valid_devs[0], qp_id, &qp_conf, @@ -12792,7 +12689,6 @@ test_deq_callback_setup(void) qp_conf.nb_descriptors = MAX_NUM_OPS_INFLIGHT; qp_conf.mp_session = ts_params->session_mpool; - qp_conf.mp_session_private = ts_params->session_priv_mpool; TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( ts_params->valid_devs[0], qp_id, &qp_conf, @@ -12979,7 +12875,6 @@ static int create_gmac_session(uint8_t dev_id, enum rte_crypto_auth_operation auth_op) { uint8_t auth_key[tdata->key.len]; - int status; struct crypto_testsuite_params *ts_params = &testsuite_params; struct crypto_unittest_params *ut_params = &unittest_params; @@ -12998,15 +12893,13 @@ static int create_gmac_session(uint8_t dev_id, ut_params->auth_xform.auth.iv.length = tdata->iv.len; - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, + &ut_params->auth_xform, ts_params->session_mpool); + if (ut_params->sess == NULL && rte_errno == ENOTSUP) + return TEST_SKIPPED; TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, - &ut_params->auth_xform, - ts_params->session_priv_mpool); - - return status; + return 0; } static int @@ -13635,7 +13528,6 @@ create_auth_session(struct crypto_unittest_params *ut_params, { struct crypto_testsuite_params *ts_params = &testsuite_params; uint8_t auth_key[reference->auth_key.len + 1]; - int status; memcpy(auth_key, reference->auth_key.data, reference->auth_key.len); @@ -13649,15 +13541,13 @@ create_auth_session(struct crypto_unittest_params *ut_params, ut_params->auth_xform.auth.digest_length = reference->digest.len; /* Create Crypto session*/ - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, &ut_params->auth_xform, - ts_params->session_priv_mpool); + ts_params->session_mpool); + if (ut_params->sess == NULL && rte_errno == ENOTSUP) + return TEST_SKIPPED; - return status; + return 0; } static int @@ -13670,7 +13560,6 @@ create_auth_cipher_session(struct crypto_unittest_params *ut_params, struct crypto_testsuite_params *ts_params = &testsuite_params; uint8_t cipher_key[reference->cipher_key.len + 1]; uint8_t auth_key[reference->auth_key.len + 1]; - int status; memcpy(cipher_key, reference->cipher_key.data, reference->cipher_key.len); @@ -13702,15 +13591,13 @@ create_auth_cipher_session(struct crypto_unittest_params *ut_params, } /* Create Crypto session*/ - ut_params->sess = rte_cryptodev_sym_session_create( - ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - - status = rte_cryptodev_sym_session_init(dev_id, ut_params->sess, + ut_params->sess = rte_cryptodev_sym_session_create(dev_id, &ut_params->auth_xform, - ts_params->session_priv_mpool); + ts_params->session_mpool); + if (ut_params->sess == NULL && rte_errno == ENOTSUP) + return TEST_SKIPPED; - return status; + return 0; } static int @@ -14168,7 +14055,6 @@ test_authenticated_encrypt_with_esn( uint8_t cipher_key[reference->cipher_key.len + 1]; uint8_t auth_key[reference->auth_key.len + 1]; struct rte_cryptodev_info dev_info; - int status; rte_cryptodev_info_get(ts_params->valid_devs[0], &dev_info); uint64_t feat_flags = dev_info.feature_flags; @@ -14219,18 +14105,11 @@ test_authenticated_encrypt_with_esn( /* Create Crypto session*/ ut_params->sess = rte_cryptodev_sym_session_create( + ts_params->valid_devs[0], &ut_params->cipher_xform, ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - - status = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], - ut_params->sess, - &ut_params->cipher_xform, - ts_params->session_priv_mpool); - - if (status == -ENOTSUP) + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - - TEST_ASSERT_EQUAL(status, 0, "Session init failed"); + TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); TEST_ASSERT_NOT_NULL(ut_params->ibuf, @@ -14355,18 +14234,11 @@ test_authenticated_decrypt_with_esn( /* Create Crypto session*/ ut_params->sess = rte_cryptodev_sym_session_create( + ts_params->valid_devs[0], &ut_params->auth_xform, ts_params->session_mpool); - TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); - - retval = rte_cryptodev_sym_session_init(ts_params->valid_devs[0], - ut_params->sess, - &ut_params->auth_xform, - ts_params->session_priv_mpool); - - if (retval == -ENOTSUP) + if (ut_params->sess == NULL && rte_errno == ENOTSUP) return TEST_SKIPPED; - - TEST_ASSERT_EQUAL(retval, 0, "Session init failed"); + TEST_ASSERT_NOT_NULL(ut_params->sess, "Session creation failed"); ut_params->ibuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); TEST_ASSERT_NOT_NULL(ut_params->ibuf, @@ -15138,8 +15010,6 @@ test_scheduler_attach_worker_op(void) } ts_params->qp_conf.mp_session = ts_params->session_mpool; - ts_params->qp_conf.mp_session_private = - ts_params->session_priv_mpool; ret = rte_cryptodev_scheduler_worker_attach(sched_id, (uint8_t)i); diff --git a/app/test/test_cryptodev_blockcipher.c b/app/test/test_cryptodev_blockcipher.c index b5813b956f..4fcdd55660 100644 --- a/app/test/test_cryptodev_blockcipher.c +++ b/app/test/test_cryptodev_blockcipher.c @@ -68,7 +68,6 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, struct rte_mempool *mbuf_pool, struct rte_mempool *op_mpool, struct rte_mempool *sess_mpool, - struct rte_mempool *sess_priv_mpool, uint8_t dev_id, char *test_msg) { @@ -514,11 +513,9 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, */ if (!(t->feature_mask & BLOCKCIPHER_TEST_FEATURE_SESSIONLESS) && nb_iterates == 0) { - sess = rte_cryptodev_sym_session_create(sess_mpool); - - status = rte_cryptodev_sym_session_init(dev_id, sess, - init_xform, sess_priv_mpool); - if (status == -ENOTSUP) { + sess = rte_cryptodev_sym_session_create(dev_id, init_xform, + sess_mpool); + if (sess == NULL) { snprintf(test_msg, BLOCKCIPHER_TEST_MSG_LEN, "UNSUPPORTED"); status = TEST_SKIPPED; goto error_exit; @@ -801,10 +798,8 @@ test_blockcipher_one_case(const struct blockcipher_test_case *t, error_exit: if (!(t->feature_mask & BLOCKCIPHER_TEST_FEATURE_SESSIONLESS)) { - if (sess) { - rte_cryptodev_sym_session_clear(dev_id, sess); - rte_cryptodev_sym_session_free(sess); - } + if (sess) + rte_cryptodev_sym_session_free(dev_id, sess); rte_free(cipher_xform); rte_free(auth_xform); } @@ -829,7 +824,6 @@ blockcipher_test_case_run(const void *data) p_testsuite_params->mbuf_pool, p_testsuite_params->op_mpool, p_testsuite_params->session_mpool, - p_testsuite_params->session_priv_mpool, p_testsuite_params->valid_devs[0], test_msg); return status; diff --git a/app/test/test_event_crypto_adapter.c b/app/test/test_event_crypto_adapter.c index 2ecc7e2cea..c6cf9c5401 100644 --- a/app/test/test_event_crypto_adapter.c +++ b/app/test/test_event_crypto_adapter.c @@ -157,7 +157,6 @@ struct event_crypto_adapter_test_params { struct rte_mempool *op_mpool; struct rte_mempool *asym_op_mpool; struct rte_mempool *session_mpool; - struct rte_mempool *session_priv_mpool; struct rte_mempool *asym_sess_mpool; struct rte_cryptodev_config *config; uint8_t crypto_event_port_id; @@ -307,15 +306,10 @@ test_op_forward_mode(uint8_t session_less) sym_op = op->sym; if (!session_less) { - sess = rte_cryptodev_sym_session_create( - params.session_mpool); + sess = rte_cryptodev_sym_session_create(TEST_CDEV_ID, + &cipher_xform, params.session_mpool); TEST_ASSERT_NOT_NULL(sess, "Session creation failed\n"); - /* Create Crypto session*/ - ret = rte_cryptodev_sym_session_init(TEST_CDEV_ID, sess, - &cipher_xform, params.session_priv_mpool); - TEST_ASSERT_SUCCESS(ret, "Failed to init session\n"); - ret = rte_event_crypto_adapter_caps_get(evdev, TEST_CDEV_ID, &cap); TEST_ASSERT_SUCCESS(ret, "Failed to get adapter capabilities\n"); @@ -683,8 +677,8 @@ test_op_new_mode(uint8_t session_less) sym_op = op->sym; if (!session_less) { - sess = rte_cryptodev_sym_session_create( - params.session_mpool); + sess = rte_cryptodev_sym_session_create(TEST_CDEV_ID, + &cipher_xform, params.session_mpool); TEST_ASSERT_NOT_NULL(sess, "Session creation failed\n"); ret = rte_event_crypto_adapter_caps_get(evdev, TEST_CDEV_ID, @@ -699,9 +693,6 @@ test_op_new_mode(uint8_t session_less) RTE_CRYPTO_OP_WITH_SESSION, &m_data, sizeof(m_data)); } - ret = rte_cryptodev_sym_session_init(TEST_CDEV_ID, sess, - &cipher_xform, params.session_priv_mpool); - TEST_ASSERT_SUCCESS(ret, "Failed to init session\n"); rte_crypto_op_attach_sym_session(op, sess); } else { @@ -994,22 +985,12 @@ configure_cryptodev(void) params.session_mpool = rte_cryptodev_sym_session_pool_create( "CRYPTO_ADAPTER_SESSION_MP", - MAX_NB_SESSIONS, 0, 0, + MAX_NB_SESSIONS, session_size, 0, sizeof(union rte_event_crypto_metadata), SOCKET_ID_ANY); TEST_ASSERT_NOT_NULL(params.session_mpool, "session mempool allocation failed\n"); - params.session_priv_mpool = rte_mempool_create( - "CRYPTO_AD_SESS_MP_PRIV", - MAX_NB_SESSIONS, - session_size, - 0, 0, NULL, NULL, NULL, - NULL, SOCKET_ID_ANY, - 0); - TEST_ASSERT_NOT_NULL(params.session_priv_mpool, - "session mempool allocation failed\n"); - rte_cryptodev_info_get(TEST_CDEV_ID, &info); while ((capability = &info.capabilities[i++])->op != @@ -1048,7 +1029,6 @@ configure_cryptodev(void) qp_conf.nb_descriptors = DEFAULT_NUM_OPS_INFLIGHT; qp_conf.mp_session = params.session_mpool; - qp_conf.mp_session_private = params.session_priv_mpool; TEST_ASSERT_SUCCESS(rte_cryptodev_queue_pair_setup( TEST_CDEV_ID, TEST_CDEV_QP_ID, &qp_conf, @@ -1410,11 +1390,6 @@ crypto_teardown(void) rte_mempool_free(params.session_mpool); params.session_mpool = NULL; } - if (params.session_priv_mpool != NULL) { - rte_mempool_avail_count(params.session_priv_mpool); - rte_mempool_free(params.session_priv_mpool); - params.session_priv_mpool = NULL; - } /* Free asym session mempool */ if (params.asym_sess_mpool != NULL) { diff --git a/app/test/test_ipsec.c b/app/test/test_ipsec.c index aa533483fd..04d231468b 100644 --- a/app/test/test_ipsec.c +++ b/app/test/test_ipsec.c @@ -370,20 +370,9 @@ testsuite_setup(void) return TEST_FAILED; } - ts_params->qp_conf.mp_session_private = rte_mempool_create( - "test_priv_sess_mp", - MAX_NB_SESSIONS, - sess_sz, - 0, 0, NULL, NULL, NULL, - NULL, SOCKET_ID_ANY, - 0); - - TEST_ASSERT_NOT_NULL(ts_params->qp_conf.mp_session_private, - "private session mempool allocation failed"); - ts_params->qp_conf.mp_session = rte_cryptodev_sym_session_pool_create("test_sess_mp", - MAX_NB_SESSIONS, 0, 0, 0, SOCKET_ID_ANY); + MAX_NB_SESSIONS, sess_sz, 0, 0, SOCKET_ID_ANY); TEST_ASSERT_NOT_NULL(ts_params->qp_conf.mp_session, "session mempool allocation failed"); @@ -428,11 +417,6 @@ testsuite_teardown(void) rte_mempool_free(ts_params->qp_conf.mp_session); ts_params->qp_conf.mp_session = NULL; } - - if (ts_params->qp_conf.mp_session_private != NULL) { - rte_mempool_free(ts_params->qp_conf.mp_session_private); - ts_params->qp_conf.mp_session_private = NULL; - } } static int @@ -647,8 +631,7 @@ create_dummy_sec_session(struct ipsec_unitest_params *ut, static struct rte_security_session_conf conf; ut->ss[j].security.ses = rte_security_session_create(&dummy_sec_ctx, - &conf, qp->mp_session, - qp->mp_session_private); + &conf, qp->mp_session, NULL); if (ut->ss[j].security.ses == NULL) return -ENOMEM; @@ -662,25 +645,15 @@ static int create_crypto_session(struct ipsec_unitest_params *ut, struct rte_cryptodev_qp_conf *qp, uint8_t dev_id, uint32_t j) { - int32_t rc; struct rte_cryptodev_sym_session *s; - s = rte_cryptodev_sym_session_create(qp->mp_session); + s = rte_cryptodev_sym_session_create(dev_id, ut->crypto_xforms, + qp->mp_session); if (s == NULL) return -ENOMEM; - /* initialize SA crypto session for device */ - rc = rte_cryptodev_sym_session_init(dev_id, s, - ut->crypto_xforms, qp->mp_session_private); - if (rc == 0) { - ut->ss[j].crypto.ses = s; - return 0; - } else { - /* failure, do cleanup */ - rte_cryptodev_sym_session_clear(dev_id, s); - rte_cryptodev_sym_session_free(s); - return rc; - } + ut->ss[j].crypto.ses = s; + return 0; } static int @@ -1196,8 +1169,7 @@ static void destroy_crypto_session(struct ipsec_unitest_params *ut, uint8_t crypto_dev, uint32_t j) { - rte_cryptodev_sym_session_clear(crypto_dev, ut->ss[j].crypto.ses); - rte_cryptodev_sym_session_free(ut->ss[j].crypto.ses); + rte_cryptodev_sym_session_free(crypto_dev, ut->ss[j].crypto.ses); memset(&ut->ss[j], 0, sizeof(ut->ss[j])); } diff --git a/drivers/crypto/armv8/rte_armv8_pmd.c b/drivers/crypto/armv8/rte_armv8_pmd.c index 32127a874c..75a95e3689 100644 --- a/drivers/crypto/armv8/rte_armv8_pmd.c +++ b/drivers/crypto/armv8/rte_armv8_pmd.c @@ -521,34 +521,23 @@ get_session(struct armv8_crypto_qp *qp, struct rte_crypto_op *op) if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { /* get existing session */ if (likely(op->sym->session != NULL)) { - sess = (struct armv8_crypto_session *) - get_sym_session_private_data( - op->sym->session, - cryptodev_driver_id); + sess = op->sym->session->data; } } else { /* provide internal session */ - void *_sess = NULL; - void *_sess_private_data = NULL; + struct rte_cryptodev_sym_session *_sess = NULL; if (rte_mempool_get(qp->sess_mp, (void **)&_sess)) return NULL; - if (rte_mempool_get(qp->sess_mp_priv, - (void **)&_sess_private_data)) - return NULL; - - sess = (struct armv8_crypto_session *)_sess_private_data; + sess = (struct armv8_crypto_session *)_sess->data; if (unlikely(armv8_crypto_set_session_parameters(sess, op->sym->xform) != 0)) { rte_mempool_put(qp->sess_mp, _sess); - rte_mempool_put(qp->sess_mp_priv, _sess_private_data); sess = NULL; } op->sym->session = (struct rte_cryptodev_sym_session *)_sess; - set_sym_session_private_data(op->sym->session, - cryptodev_driver_id, _sess_private_data); } if (unlikely(sess == NULL)) @@ -674,10 +663,6 @@ process_op(struct armv8_crypto_qp *qp, struct rte_crypto_op *op, /* Free session if a session-less crypto op */ if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sess, 0, sizeof(struct armv8_crypto_session)); - memset(op->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - op->sym->session)); - rte_mempool_put(qp->sess_mp_priv, sess); rte_mempool_put(qp->sess_mp, op->sym->session); op->sym->session = NULL; } diff --git a/drivers/crypto/armv8/rte_armv8_pmd_ops.c b/drivers/crypto/armv8/rte_armv8_pmd_ops.c index c07ac0489e..176bb9e6a0 100644 --- a/drivers/crypto/armv8/rte_armv8_pmd_ops.c +++ b/drivers/crypto/armv8/rte_armv8_pmd_ops.c @@ -267,8 +267,7 @@ armv8_crypto_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) static int armv8_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { void *sess_private_data; int ret; @@ -278,43 +277,22 @@ armv8_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev, return -EINVAL; } - if (rte_mempool_get(mempool, &sess_private_data)) { - CDEV_LOG_ERR( - "Couldn't get object from session mempool"); - return -ENOMEM; - } + sess_private_data = sess->data; ret = armv8_crypto_set_session_parameters(sess_private_data, xform); if (ret != 0) { ARMV8_CRYPTO_LOG_ERR("failed configure session parameters"); - - /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - return 0; } /** Clear the memory of session so it doesn't leave key material behind */ static void -armv8_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess) -{ - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); - - /* Zero out the whole structure */ - if (sess_priv) { - memset(sess_priv, 0, sizeof(struct armv8_crypto_session)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); - } -} +armv8_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{} struct rte_cryptodev_ops armv8_crypto_pmd_ops = { .dev_configure = armv8_crypto_pmd_config, diff --git a/drivers/crypto/bcmfs/bcmfs_sym_session.c b/drivers/crypto/bcmfs/bcmfs_sym_session.c index 675ed0ad55..d3334dc920 100644 --- a/drivers/crypto/bcmfs/bcmfs_sym_session.c +++ b/drivers/crypto/bcmfs/bcmfs_sym_session.c @@ -211,8 +211,7 @@ bcmfs_sym_get_session(struct rte_crypto_op *op) } else if (likely(op->sym->session != NULL)) { /* get existing session */ sess = (struct bcmfs_sym_session *) - get_sym_session_private_data(op->sym->session, - cryptodev_bcmfs_driver_id); + op->sym->session->driver_priv_data; } if (sess == NULL) @@ -222,10 +221,9 @@ bcmfs_sym_get_session(struct rte_crypto_op *op) } int -bcmfs_sym_session_configure(struct rte_cryptodev *dev, +bcmfs_sym_session_configure(struct rte_cryptodev *dev __rte_unused, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { void *sess_private_data; int ret; @@ -235,45 +233,23 @@ bcmfs_sym_session_configure(struct rte_cryptodev *dev, return -EINVAL; } - if (rte_mempool_get(mempool, &sess_private_data)) { - BCMFS_DP_LOG(ERR, - "Couldn't get object from session mempool"); - return -ENOMEM; - } + sess_private_data = (void *)sess->driver_priv_data; ret = crypto_set_session_parameters(sess_private_data, xform); if (ret != 0) { BCMFS_DP_LOG(ERR, "Failed configure session parameters"); - /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - return 0; } /* Clear the memory of session so it doesn't leave key material behind */ void -bcmfs_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess) -{ - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); - - if (sess_priv) { - struct rte_mempool *sess_mp; - - memset(sess_priv, 0, sizeof(struct bcmfs_sym_session)); - sess_mp = rte_mempool_from_obj(sess_priv); - - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); - } -} +bcmfs_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{} unsigned int bcmfs_sym_session_get_private_size(struct rte_cryptodev *dev __rte_unused) diff --git a/drivers/crypto/bcmfs/bcmfs_sym_session.h b/drivers/crypto/bcmfs/bcmfs_sym_session.h index d40595b4bd..4a0a012ae7 100644 --- a/drivers/crypto/bcmfs/bcmfs_sym_session.h +++ b/drivers/crypto/bcmfs/bcmfs_sym_session.h @@ -93,8 +93,7 @@ bcmfs_process_crypto_op(struct rte_crypto_op *op, int bcmfs_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool); + struct rte_cryptodev_sym_session *sess); void bcmfs_sym_session_clear(struct rte_cryptodev *dev, diff --git a/drivers/crypto/caam_jr/caam_jr.c b/drivers/crypto/caam_jr/caam_jr.c index 8e9cfe73d8..a9c93dea13 100644 --- a/drivers/crypto/caam_jr/caam_jr.c +++ b/drivers/crypto/caam_jr/caam_jr.c @@ -1357,8 +1357,7 @@ caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp) switch (op->sess_type) { case RTE_CRYPTO_OP_WITH_SESSION: ses = (struct caam_jr_session *) - get_sym_session_private_data(op->sym->session, - cryptodev_driver_id); + op->sym->session->driver_priv_data; break; case RTE_CRYPTO_OP_SECURITY_SESSION: ses = (struct caam_jr_session *) @@ -1692,54 +1691,39 @@ caam_jr_set_session_parameters(struct rte_cryptodev *dev, } static int -caam_jr_sym_session_configure(struct rte_cryptodev *dev, +caam_jr_sym_session_configure(struct rte_cryptodev *dev __rte_unused, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { void *sess_private_data; int ret; PMD_INIT_FUNC_TRACE(); - - if (rte_mempool_get(mempool, &sess_private_data)) { - CAAM_JR_ERR("Couldn't get object from session mempool"); - return -ENOMEM; - } - + sess_private_data = (void *)sess->driver_priv_data; memset(sess_private_data, 0, sizeof(struct caam_jr_session)); ret = caam_jr_set_session_parameters(dev, xform, sess_private_data); if (ret != 0) { CAAM_JR_ERR("failed to configure session parameters"); /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, sess_private_data); - return 0; } /* Clear the memory of session so it doesn't leave key material behind */ static void -caam_jr_sym_session_clear(struct rte_cryptodev *dev, +caam_jr_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); + void *sess_priv = (void *)sess->driver_priv_data; struct caam_jr_session *s = (struct caam_jr_session *)sess_priv; PMD_INIT_FUNC_TRACE(); if (sess_priv) { - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - rte_free(s->cipher_key.data); rte_free(s->auth_key.data); - memset(s, 0, sizeof(struct caam_jr_session)); - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); } } diff --git a/drivers/crypto/ccp/ccp_crypto.c b/drivers/crypto/ccp/ccp_crypto.c index 4bab18323b..bd999abe61 100644 --- a/drivers/crypto/ccp/ccp_crypto.c +++ b/drivers/crypto/ccp/ccp_crypto.c @@ -1585,9 +1585,7 @@ ccp_perform_hmac(struct rte_crypto_op *op, void *append_ptr; uint8_t *addr; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; addr = session->auth.pre_compute; src_addr = rte_pktmbuf_iova_offset(op->sym->m_src, @@ -1766,9 +1764,7 @@ ccp_perform_sha(struct rte_crypto_op *op, void *append_ptr; uint64_t auth_msg_bits; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; src_addr = rte_pktmbuf_iova_offset(op->sym->m_src, op->sym->auth.data.offset); @@ -1859,9 +1855,7 @@ ccp_perform_sha3_hmac(struct rte_crypto_op *op, uint32_t tail; phys_addr_t src_addr, dest_addr, ctx_paddr, dest_addr_t; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; src_addr = rte_pktmbuf_iova_offset(op->sym->m_src, op->sym->auth.data.offset); @@ -2005,9 +1999,7 @@ ccp_perform_sha3(struct rte_crypto_op *op, uint32_t tail; phys_addr_t src_addr, dest_addr, ctx_paddr; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; src_addr = rte_pktmbuf_iova_offset(op->sym->m_src, op->sym->auth.data.offset); @@ -2079,9 +2071,7 @@ ccp_perform_aes_cmac(struct rte_crypto_op *op, phys_addr_t src_addr, dest_addr, key_addr; int length, non_align_len; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; key_addr = rte_mem_virt2phy(session->auth.key_ccp); src_addr = rte_pktmbuf_iova_offset(op->sym->m_src, @@ -2242,9 +2232,7 @@ ccp_perform_aes(struct rte_crypto_op *op, phys_addr_t src_addr, dest_addr, key_addr; uint8_t *iv; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; function.raw = 0; iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset); @@ -2330,9 +2318,7 @@ ccp_perform_3des(struct rte_crypto_op *op, uint8_t *iv; phys_addr_t src_addr, dest_addr, key_addr; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset); switch (session->cipher.um.des_mode) { @@ -2440,9 +2426,7 @@ ccp_perform_aes_gcm(struct rte_crypto_op *op, struct ccp_queue *cmd_q) phys_addr_t digest_dest_addr; int length, non_align_len; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; iv = rte_crypto_op_ctod_offset(op, uint8_t *, session->iv.offset); key_addr = session->cipher.key_phys; @@ -2607,9 +2591,7 @@ ccp_crypto_cipher(struct rte_crypto_op *op, int result = 0; struct ccp_session *session; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; switch (session->cipher.algo) { case CCP_CIPHER_ALGO_AES_CBC: @@ -2645,9 +2627,7 @@ ccp_crypto_auth(struct rte_crypto_op *op, int result = 0; struct ccp_session *session; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; switch (session->auth.algo) { case CCP_AUTH_ALGO_SHA1: @@ -2715,9 +2695,7 @@ ccp_crypto_aead(struct rte_crypto_op *op, int result = 0; struct ccp_session *session; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; switch (session->auth.algo) { case CCP_AUTH_ALGO_AES_GCM: @@ -2780,9 +2758,8 @@ process_ops_to_enqueue(struct ccp_qp *qp, b_info->head_offset = (uint32_t)(cmd_q->qbase_phys_addr + cmd_q->qidx * Q_DESC_SIZE); for (i = b_idx; i < (nb_ops+b_idx); i++) { - session = (struct ccp_session *)get_sym_session_private_data( - op[i]->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *) + op[i]->sym->session->driver_priv_data; switch (session->cmd_id) { case CCP_CMD_CIPHER: result = ccp_crypto_cipher(op[i], cmd_q, b_info); @@ -2858,9 +2835,7 @@ static inline void ccp_auth_dq_prepare(struct rte_crypto_op *op) int offset, digest_offset; uint8_t digest_le[64]; - session = (struct ccp_session *)get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *)op->sym->session->driver_priv_data; if (session->cmd_id == CCP_CMD_COMBINED) { digest_data = op->sym->aead.digest.data; @@ -2934,9 +2909,8 @@ ccp_prepare_ops(struct ccp_qp *qp, for (i = b_info->b_idx; i < min_ops; i++) { op_d[i] = b_info->op[b_info->b_idx + b_info->op_idx++]; - session = (struct ccp_session *)get_sym_session_private_data( - op_d[i]->sym->session, - ccp_cryptodev_driver_id); + session = (struct ccp_session *) + op_d[i]->sym->session->driver_priv_data; switch (session->cmd_id) { case CCP_CMD_CIPHER: op_d[i]->status = RTE_CRYPTO_OP_STATUS_SUCCESS; diff --git a/drivers/crypto/ccp/ccp_pmd_ops.c b/drivers/crypto/ccp/ccp_pmd_ops.c index 1b600e81ad..e401793a76 100644 --- a/drivers/crypto/ccp/ccp_pmd_ops.c +++ b/drivers/crypto/ccp/ccp_pmd_ops.c @@ -727,7 +727,6 @@ ccp_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; /* mempool for batch info */ qp->batch_mp = rte_mempool_create( @@ -757,8 +756,7 @@ ccp_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) static int ccp_pmd_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { int ret; void *sess_private_data; @@ -769,40 +767,22 @@ ccp_pmd_sym_session_configure(struct rte_cryptodev *dev, return -ENOMEM; } - if (rte_mempool_get(mempool, &sess_private_data)) { - CCP_LOG_ERR("Couldn't get object from session mempool"); - return -ENOMEM; - } + sess_private_data = (void *)sess->driver_priv_data; + internals = (struct ccp_private *)dev->data->dev_private; ret = ccp_set_session_parameters(sess_private_data, xform, internals); if (ret != 0) { CCP_LOG_ERR("failed configure session parameters"); - - /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); return 0; } static void -ccp_pmd_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess) -{ - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); - - if (sess_priv) { - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - - rte_mempool_put(sess_mp, sess_priv); - memset(sess_priv, 0, sizeof(struct ccp_session)); - set_sym_session_private_data(sess, index, NULL); - } -} +ccp_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{} struct rte_cryptodev_ops ccp_ops = { .dev_configure = ccp_pmd_config, diff --git a/drivers/crypto/ccp/ccp_pmd_private.h b/drivers/crypto/ccp/ccp_pmd_private.h index 1c4118ee3c..6704e39ab8 100644 --- a/drivers/crypto/ccp/ccp_pmd_private.h +++ b/drivers/crypto/ccp/ccp_pmd_private.h @@ -78,8 +78,6 @@ struct ccp_qp { /**< Ring for placing process packets */ struct rte_mempool *sess_mp; /**< Session Mempool */ - struct rte_mempool *sess_mp_priv; - /**< Session Private Data Mempool */ struct rte_mempool *batch_mp; /**< Session Mempool for batch info */ struct rte_cryptodev_stats qp_stats; diff --git a/drivers/crypto/ccp/rte_ccp_pmd.c b/drivers/crypto/ccp/rte_ccp_pmd.c index a35a8cd775..7502281ad9 100644 --- a/drivers/crypto/ccp/rte_ccp_pmd.c +++ b/drivers/crypto/ccp/rte_ccp_pmd.c @@ -56,33 +56,23 @@ get_ccp_session(struct ccp_qp *qp, struct rte_crypto_op *op) if (unlikely(op->sym->session == NULL)) return NULL; - sess = (struct ccp_session *) - get_sym_session_private_data( - op->sym->session, - ccp_cryptodev_driver_id); + sess = (void *)op->sym->session->driver_priv_data; } else if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { - void *_sess; - void *_sess_private_data = NULL; + struct rte_cryptodev_sym_session *_sess; struct ccp_private *internals; - if (rte_mempool_get(qp->sess_mp, &_sess)) - return NULL; - if (rte_mempool_get(qp->sess_mp, (void **)&_sess_private_data)) + if (rte_mempool_get(qp->sess_mp, (void **)&_sess)) return NULL; - sess = (struct ccp_session *)_sess_private_data; + sess = (void *)_sess->driver_priv_data; internals = (struct ccp_private *)qp->dev->data->dev_private; if (unlikely(ccp_set_session_parameters(sess, op->sym->xform, internals) != 0)) { rte_mempool_put(qp->sess_mp, _sess); - rte_mempool_put(qp->sess_mp_priv, _sess_private_data); sess = NULL; } - op->sym->session = (struct rte_cryptodev_sym_session *)_sess; - set_sym_session_private_data(op->sym->session, - ccp_cryptodev_driver_id, - _sess_private_data); + op->sym->session = _sess; } return sess; @@ -161,13 +151,10 @@ ccp_pmd_dequeue_burst(void *queue_pair, struct rte_crypto_op **ops, for (i = 0; i < nb_dequeued; i++) if (unlikely(ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { - struct ccp_session *sess = (struct ccp_session *) - get_sym_session_private_data( - ops[i]->sym->session, - ccp_cryptodev_driver_id); + struct ccp_session *sess = + (void *)ops[i]->sym->session->driver_priv_data; - rte_mempool_put(qp->sess_mp_priv, - sess); + memset(sess, 0, sizeof(*sess)); rte_mempool_put(qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session = NULL; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index 586941cd70..f96e132513 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -22,24 +22,23 @@ static inline struct cnxk_se_sess * cn10k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) { - const int driver_id = cn10k_cryptodev_driver_id; struct rte_crypto_sym_op *sym_op = op->sym; struct rte_cryptodev_sym_session *sess; struct cnxk_se_sess *priv; int ret; /* Create temporary session */ - sess = rte_cryptodev_sym_session_create(qp->sess_mp); - if (sess == NULL) + if (rte_mempool_get(qp->sess_mp, (void **)&sess) < 0) return NULL; - ret = sym_session_configure(qp->lf.roc_cpt, driver_id, sym_op->xform, - sess, qp->sess_mp_priv); - if (ret) + ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, + sess); + if (ret) { + rte_mempool_put(qp->sess_mp, (void *)sess); goto sess_put; + } - priv = get_sym_session_private_data(sess, driver_id); - + priv = (void *)sess->driver_priv_data; sym_op->session = sess; return priv; @@ -111,8 +110,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], return 0; w7 = sec_sess->sa.inst.w7; } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { - sess = get_sym_session_private_data( - sym_op->session, cn10k_cryptodev_driver_id); + sess = (void *)sym_op->session->driver_priv_data; ret = cpt_sym_inst_fill(qp, op, sess, infl_req, &inst[0]); if (unlikely(ret)) @@ -128,8 +126,7 @@ cn10k_cpt_fill_inst(struct cnxk_cpt_qp *qp, struct rte_crypto_op *ops[], ret = cpt_sym_inst_fill(qp, op, sess, infl_req, &inst[0]); if (unlikely(ret)) { - sym_session_clear(cn10k_cryptodev_driver_id, - op->sym->session); + sym_session_clear(op->sym->session); rte_mempool_put(qp->sess_mp, op->sym->session); return 0; } @@ -296,8 +293,9 @@ cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, } else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { struct cnxk_se_sess *priv; - priv = get_sym_session_private_data( - sess, cn10k_cryptodev_driver_id); + priv = (void *)( + ((struct rte_cryptodev_sym_session *)sess)-> + driver_priv_data); priv->qp = qp; priv->cpt_inst_w2 = w2; } else @@ -334,8 +332,7 @@ cn10k_ca_meta_info_extract(struct rte_crypto_op *op, } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { struct cnxk_se_sess *priv; - priv = get_sym_session_private_data( - op->sym->session, cn10k_cryptodev_driver_id); + priv = (void *)op->sym->session->driver_priv_data; *qp = priv->qp; *w2 = priv->cpt_inst_w2; } else { @@ -543,7 +540,6 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, { const uint8_t uc_compcode = res->uc_compcode; const uint8_t compcode = res->compcode; - unsigned int sz; cop->status = RTE_CRYPTO_OP_STATUS_SUCCESS; @@ -620,11 +616,7 @@ cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, temp_sess_free: if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { - sym_session_clear(cn10k_cryptodev_driver_id, - cop->sym->session); - sz = rte_cryptodev_sym_get_existing_header_session_size( - cop->sym->session); - memset(cop->sym->session, 0, sz); + sym_session_clear(cop->sym->session); rte_mempool_put(qp->sess_mp, cop->sym->session); cop->sym->session = NULL; } diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index b753c1cb4b..a44f111ba6 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -56,23 +56,20 @@ cn9k_cpt_sec_inst_fill(struct rte_crypto_op *op, static inline struct cnxk_se_sess * cn9k_cpt_sym_temp_sess_create(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op) { - const int driver_id = cn9k_cryptodev_driver_id; struct rte_crypto_sym_op *sym_op = op->sym; struct rte_cryptodev_sym_session *sess; struct cnxk_se_sess *priv; int ret; /* Create temporary session */ - sess = rte_cryptodev_sym_session_create(qp->sess_mp); - if (sess == NULL) + if (rte_mempool_get(qp->sess_mp, (void **)&sess) < 0) return NULL; - ret = sym_session_configure(qp->lf.roc_cpt, driver_id, sym_op->xform, - sess, qp->sess_mp_priv); + ret = sym_session_configure(qp->lf.roc_cpt, sym_op->xform, sess); if (ret) goto sess_put; - priv = get_sym_session_private_data(sess, driver_id); + priv = (void *)sess->driver_priv_data; sym_op->session = sess; @@ -95,8 +92,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { sym_op = op->sym; - sess = get_sym_session_private_data( - sym_op->session, cn9k_cryptodev_driver_id); + sess = (void *)sym_op->session->driver_priv_data; ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst); inst->w7.u64 = sess->cpt_inst_w7; } else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) @@ -110,8 +106,7 @@ cn9k_cpt_inst_prep(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, ret = cpt_sym_inst_fill(qp, op, sess, infl_req, inst); if (unlikely(ret)) { - sym_session_clear(cn9k_cryptodev_driver_id, - op->sym->session); + sym_session_clear(op->sym->session); rte_mempool_put(qp->sess_mp, op->sym->session); } inst->w7.u64 = sess->cpt_inst_w7; @@ -349,8 +344,9 @@ cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, } else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { struct cnxk_se_sess *priv; - priv = get_sym_session_private_data( - sess, cn9k_cryptodev_driver_id); + priv = (void *)(( + (struct rte_cryptodev_sym_session *)sess)-> + driver_priv_data); priv->qp = qp; priv->cpt_inst_w2 = w2; } else @@ -387,8 +383,7 @@ cn9k_ca_meta_info_extract(struct rte_crypto_op *op, } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { struct cnxk_se_sess *priv; - priv = get_sym_session_private_data( - op->sym->session, cn9k_cryptodev_driver_id); + priv = (void *)op->sym->session->driver_priv_data; *qp = priv->qp; inst->w2.u64 = priv->cpt_inst_w2; } else { @@ -583,8 +578,6 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, struct cpt_inflight_req *infl_req, struct cpt_cn9k_res_s *res) { - unsigned int sz; - if (likely(res->compcode == CPT_COMP_GOOD)) { if (unlikely(res->uc_compcode)) { if (res->uc_compcode == ROC_SE_ERR_GC_ICV_MISCOMPARE) @@ -645,11 +638,7 @@ cn9k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, temp_sess_free: if (unlikely(cop->sess_type == RTE_CRYPTO_OP_SESSIONLESS)) { if (cop->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { - sym_session_clear(cn9k_cryptodev_driver_id, - cop->sym->session); - sz = rte_cryptodev_sym_get_existing_header_session_size( - cop->sym->session); - memset(cop->sym->session, 0, sz); + sym_session_clear(cop->sym->session); rte_mempool_put(qp->sess_mp, cop->sym->session); cop->sym->session = NULL; } diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c index cf91b92c2c..0c76dd36a6 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.c @@ -406,7 +406,6 @@ cnxk_cpt_queue_pair_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = conf->mp_session; - qp->sess_mp_priv = conf->mp_session_private; dev->data->queue_pairs[qp_id] = qp; return 0; @@ -620,25 +619,14 @@ cnxk_cpt_inst_w7_get(struct cnxk_se_sess *sess, struct roc_cpt *roc_cpt) } int -sym_session_configure(struct roc_cpt *roc_cpt, int driver_id, +sym_session_configure(struct roc_cpt *roc_cpt, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *pool) + struct rte_cryptodev_sym_session *sess) { enum cpt_dp_thread_type thr_type; - struct cnxk_se_sess *sess_priv; - void *priv; + struct cnxk_se_sess *sess_priv = (void *)sess->driver_priv_data; int ret; - if (unlikely(rte_mempool_get(pool, &priv))) { - plt_dp_err("Could not allocate session private data"); - return -ENOMEM; - } - - memset(priv, 0, sizeof(struct cnxk_se_sess)); - - sess_priv = priv; - ret = cnxk_sess_fill(roc_cpt, xform, sess_priv); if (ret) goto priv_put; @@ -684,61 +672,37 @@ sym_session_configure(struct roc_cpt *roc_cpt, int driver_id, } sess_priv->cpt_inst_w7 = cnxk_cpt_inst_w7_get(sess_priv, roc_cpt); - - set_sym_session_private_data(sess, driver_id, sess_priv); - return 0; priv_put: - rte_mempool_put(pool, priv); - return ret; } int cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *pool) + struct rte_cryptodev_sym_session *sess) { struct cnxk_cpt_vf *vf = dev->data->dev_private; struct roc_cpt *roc_cpt = &vf->cpt; - uint8_t driver_id; - - driver_id = dev->driver_id; - return sym_session_configure(roc_cpt, driver_id, xform, sess, pool); + return sym_session_configure(roc_cpt, xform, sess); } void -sym_session_clear(int driver_id, struct rte_cryptodev_sym_session *sess) +sym_session_clear(struct rte_cryptodev_sym_session *sess) { - void *priv = get_sym_session_private_data(sess, driver_id); - struct cnxk_se_sess *sess_priv; - struct rte_mempool *pool; - - if (priv == NULL) - return; - - sess_priv = priv; + struct cnxk_se_sess *sess_priv = (void *)sess->driver_priv_data; if (sess_priv->roc_se_ctx.auth_key != NULL) plt_free(sess_priv->roc_se_ctx.auth_key); - - memset(priv, 0, cnxk_cpt_sym_session_get_size(NULL)); - - pool = rte_mempool_from_obj(priv); - - set_sym_session_private_data(sess, driver_id, NULL); - - rte_mempool_put(pool, priv); } void -cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev, +cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { - return sym_session_clear(dev->driver_id, sess); + return sym_session_clear(sess); } unsigned int diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h index ffe4ae19aa..ca90567620 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h @@ -78,8 +78,6 @@ struct cnxk_cpt_qp { /**< Crypto adapter related info */ struct rte_mempool *sess_mp; /**< Session mempool */ - struct rte_mempool *sess_mp_priv; - /**< Session private data mempool */ }; int cnxk_cpt_dev_config(struct rte_cryptodev *dev, @@ -104,18 +102,16 @@ unsigned int cnxk_cpt_sym_session_get_size(struct rte_cryptodev *dev); int cnxk_cpt_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *pool); + struct rte_cryptodev_sym_session *sess); -int sym_session_configure(struct roc_cpt *roc_cpt, int driver_id, +int sym_session_configure(struct roc_cpt *roc_cpt, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *pool); + struct rte_cryptodev_sym_session *sess); void cnxk_cpt_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess); + struct rte_cryptodev_sym_session *sess); -void sym_session_clear(int driver_id, struct rte_cryptodev_sym_session *sess); +void sym_session_clear(struct rte_cryptodev_sym_session *sess); unsigned int cnxk_ae_session_size_get(struct rte_cryptodev *dev __rte_unused); diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c index 8444f1a795..5a38d53022 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_dpseci.c @@ -1379,8 +1379,7 @@ build_sec_fd(struct rte_crypto_op *op, dpaa2_sec_session *sess; if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa2_sec_session *)get_sym_session_private_data( - op->sym->session, cryptodev_driver_id); + sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data; #ifdef RTE_LIB_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data( @@ -1678,8 +1677,7 @@ dpaa2_sec_dump(struct rte_crypto_op *op) struct rte_crypto_sym_op *sym_op; if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa2_sec_session *)get_sym_session_private_data( - op->sym->session, cryptodev_driver_id); + sess = (dpaa2_sec_session *)op->sym->session->driver_priv_data; #ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa2_sec_session *)get_sec_session_private_data( @@ -3754,51 +3752,36 @@ dpaa2_sec_security_session_destroy(void *dev __rte_unused, } #endif static int -dpaa2_sec_sym_session_configure(struct rte_cryptodev *dev, +dpaa2_sec_sym_session_configure(struct rte_cryptodev *dev __rte_unused, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { - void *sess_private_data; + void *sess_private_data = (void *)sess->driver_priv_data; int ret; - if (rte_mempool_get(mempool, &sess_private_data)) { - DPAA2_SEC_ERR("Couldn't get object from session mempool"); - return -ENOMEM; - } - ret = dpaa2_sec_set_session_parameters(xform, sess_private_data); if (ret != 0) { DPAA2_SEC_ERR("Failed to configure session parameters"); /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - return 0; } /** Clear the memory of session so it doesn't leave key material behind */ static void -dpaa2_sec_sym_session_clear(struct rte_cryptodev *dev, +dpaa2_sec_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { PMD_INIT_FUNC_TRACE(); - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); + void *sess_priv = (void *)sess->driver_priv_data; dpaa2_sec_session *s = (dpaa2_sec_session *)sess_priv; if (sess_priv) { rte_free(s->ctxt); rte_free(s->cipher_key.data); rte_free(s->auth_key.data); - memset(s, 0, sizeof(dpaa2_sec_session)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); } } diff --git a/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c b/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c index e68a4875dd..795be3acc3 100644 --- a/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c +++ b/drivers/crypto/dpaa2_sec/dpaa2_sec_raw_dp.c @@ -1012,8 +1012,7 @@ dpaa2_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa2_sec_session *)get_sec_session_private_data( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa2_sec_session *)get_sym_session_private_data( - session_ctx.crypto_sess, cryptodev_driver_id); + sess = (void *)session_ctx.crypto_sess->driver_priv_data; else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa2_sec_raw_dequeue_burst; diff --git a/drivers/crypto/dpaa_sec/dpaa_sec.c b/drivers/crypto/dpaa_sec/dpaa_sec.c index 05415dbf3b..0cc773fba6 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec.c +++ b/drivers/crypto/dpaa_sec/dpaa_sec.c @@ -670,10 +670,7 @@ dpaa_sec_dump(struct dpaa_sec_op_ctx *ctx, struct dpaa_sec_qp *qp) struct qm_sg_entry sg[2]; if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa_sec_session *) - get_sym_session_private_data( - op->sym->session, - dpaa_cryptodev_driver_id); + sess = (dpaa_sec_session *)op->sym->session->driver_priv_data; #ifdef RTE_LIBRTE_SECURITY else if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) sess = (dpaa_sec_session *) @@ -1927,10 +1924,8 @@ dpaa_sec_enqueue_burst(void *qp, struct rte_crypto_op **ops, switch (op->sess_type) { case RTE_CRYPTO_OP_WITH_SESSION: - ses = (dpaa_sec_session *) - get_sym_session_private_data( - op->sym->session, - dpaa_cryptodev_driver_id); + ses = (void *) + op->sym->session->driver_priv_data; break; #ifdef RTE_LIB_SECURITY case RTE_CRYPTO_OP_SECURITY_SESSION: @@ -2676,31 +2671,19 @@ dpaa_sec_set_session_parameters(struct rte_cryptodev *dev, static int dpaa_sec_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { - void *sess_private_data; + void *sess_private_data = (void *)sess->driver_priv_data; int ret; PMD_INIT_FUNC_TRACE(); - if (rte_mempool_get(mempool, &sess_private_data)) { - DPAA_SEC_ERR("Couldn't get object from session mempool"); - return -ENOMEM; - } - ret = dpaa_sec_set_session_parameters(dev, xform, sess_private_data); if (ret != 0) { DPAA_SEC_ERR("failed to configure session parameters"); - - /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - ret = dpaa_sec_prep_cdb(sess_private_data); if (ret) { DPAA_SEC_ERR("Unable to prepare sec cdb"); @@ -2714,7 +2697,6 @@ static inline void free_session_memory(struct rte_cryptodev *dev, dpaa_sec_session *s) { struct dpaa_sec_dev_private *qi = dev->data->dev_private; - struct rte_mempool *sess_mp = rte_mempool_from_obj((void *)s); uint8_t i; for (i = 0; i < MAX_DPAA_CORES; i++) { @@ -2724,7 +2706,6 @@ free_session_memory(struct rte_cryptodev *dev, dpaa_sec_session *s) s->qp[i] = NULL; } free_session_data(s); - rte_mempool_put(sess_mp, (void *)s); } /** Clear the memory of session so it doesn't leave key material behind */ @@ -2733,14 +2714,10 @@ dpaa_sec_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) { PMD_INIT_FUNC_TRACE(); - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); + void *sess_priv = (void *)sess->driver_priv_data; dpaa_sec_session *s = (dpaa_sec_session *)sess_priv; - if (sess_priv) { - free_session_memory(dev, s); - set_sym_session_private_data(sess, index, NULL); - } + free_session_memory(dev, s); } #ifdef RTE_LIB_SECURITY diff --git a/drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c b/drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c index d081953e26..d2e4d9d787 100644 --- a/drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c +++ b/drivers/crypto/dpaa_sec/dpaa_sec_raw_dp.c @@ -1017,8 +1017,8 @@ dpaa_sec_configure_raw_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, sess = (dpaa_sec_session *)get_sec_session_private_data( session_ctx.sec_sess); else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) - sess = (dpaa_sec_session *)get_sym_session_private_data( - session_ctx.crypto_sess, dpaa_cryptodev_driver_id); + sess = (dpaa_sec_session *) + session_ctx.crypto_sess->driver_priv_data; else return -ENOTSUP; raw_dp_ctx->dequeue_burst = dpaa_sec_raw_dequeue_burst; diff --git a/drivers/crypto/ipsec_mb/ipsec_mb_ops.c b/drivers/crypto/ipsec_mb/ipsec_mb_ops.c index 7e8396b4a3..90ce5bc965 100644 --- a/drivers/crypto/ipsec_mb/ipsec_mb_ops.c +++ b/drivers/crypto/ipsec_mb/ipsec_mb_ops.c @@ -264,7 +264,6 @@ ipsec_mb_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, qp->pmd_type = internals->pmd_type; qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; qp->ingress_queue = ipsec_mb_qp_create_processed_ops_ring(qp, qp_conf->nb_descriptors, socket_id); @@ -312,9 +311,8 @@ ipsec_mb_sym_session_get_size(struct rte_cryptodev *dev) int ipsec_mb_sym_session_configure( struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { - void *sess_private_data; struct ipsec_mb_dev_private *internals = dev->data->dev_private; struct ipsec_mb_internals *pmd_data = &ipsec_mb_pmds[internals->pmd_type]; @@ -330,42 +328,22 @@ ipsec_mb_sym_session_configure( return -EINVAL; } - if (rte_mempool_get(mempool, &sess_private_data)) { - IPSEC_MB_LOG(ERR, "Couldn't get object from session mempool"); - free_mb_mgr(mb_mgr); - return -ENOMEM; - } - - ret = (*pmd_data->session_configure)(mb_mgr, sess_private_data, xform); + ret = (*pmd_data->session_configure)(mb_mgr, + (void *)sess->driver_priv_data, xform); if (ret != 0) { IPSEC_MB_LOG(ERR, "failed configure session parameters"); /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); free_mb_mgr(mb_mgr); return ret; } - set_sym_session_private_data(sess, dev->driver_id, sess_private_data); - free_mb_mgr(mb_mgr); return 0; } /** Clear the session memory */ void -ipsec_mb_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess) -{ - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); - - /* Zero out the whole structure */ - if (sess_priv) { - memset(sess_priv, 0, ipsec_mb_sym_session_get_size(dev)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); - } -} +ipsec_mb_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{} diff --git a/drivers/crypto/ipsec_mb/ipsec_mb_private.h b/drivers/crypto/ipsec_mb/ipsec_mb_private.h index d074b33133..14f962ef28 100644 --- a/drivers/crypto/ipsec_mb/ipsec_mb_private.h +++ b/drivers/crypto/ipsec_mb/ipsec_mb_private.h @@ -136,8 +136,6 @@ struct ipsec_mb_qp { struct rte_ring *ingress_queue; /**< Ring for placing operations ready for processing */ struct rte_mempool *sess_mp; - /**< Session Mempool */ - struct rte_mempool *sess_mp_priv; /**< Session Private Data Mempool */ struct rte_cryptodev_stats stats; /**< Queue pair statistics */ @@ -399,8 +397,7 @@ ipsec_mb_sym_session_get_size(struct rte_cryptodev *dev); int ipsec_mb_sym_session_configure( struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool); + struct rte_cryptodev_sym_session *sess); /** Clear the memory of session so it does not leave key material behind */ void @@ -411,50 +408,50 @@ ipsec_mb_sym_session_clear(struct rte_cryptodev *dev, static __rte_always_inline void * ipsec_mb_get_session_private(struct ipsec_mb_qp *qp, struct rte_crypto_op *op) { - void *sess = NULL; + struct rte_cryptodev_sym_session *sess = NULL; uint32_t driver_id = ipsec_mb_get_driver_id(qp->pmd_type); struct rte_crypto_sym_op *sym_op = op->sym; uint8_t sess_type = op->sess_type; void *_sess; - void *_sess_private_data = NULL; struct ipsec_mb_internals *pmd_data = &ipsec_mb_pmds[qp->pmd_type]; switch (sess_type) { case RTE_CRYPTO_OP_WITH_SESSION: if (likely(sym_op->session != NULL)) - sess = get_sym_session_private_data(sym_op->session, - driver_id); + sess = sym_op->session; + else + goto error_exit; break; case RTE_CRYPTO_OP_SESSIONLESS: if (!qp->sess_mp || rte_mempool_get(qp->sess_mp, (void **)&_sess)) return NULL; - if (!qp->sess_mp_priv || - rte_mempool_get(qp->sess_mp_priv, - (void **)&_sess_private_data)) - return NULL; + sess = _sess; + if (sess->sess_data_sz < pmd_data->session_priv_size) { + rte_mempool_put(qp->sess_mp, _sess); + goto error_exit; + } - sess = _sess_private_data; if (unlikely(pmd_data->session_configure(qp->mb_mgr, - sess, sym_op->xform) != 0)) { + (void *)sess->driver_priv_data, sym_op->xform) != 0)) { rte_mempool_put(qp->sess_mp, _sess); - rte_mempool_put(qp->sess_mp_priv, _sess_private_data); - sess = NULL; + goto error_exit; } - sym_op->session = (struct rte_cryptodev_sym_session *)_sess; - set_sym_session_private_data(sym_op->session, driver_id, - _sess_private_data); + sess->driver_id = driver_id; + sym_op->session = sess; + break; default: IPSEC_MB_LOG(ERR, "Unrecognized session type %u", sess_type); } - if (unlikely(sess == NULL)) - op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION; + return (void *)sess->driver_priv_data; - return sess; +error_exit: + op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION; + return NULL; } #endif /* _IPSEC_MB_PRIVATE_H_ */ diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c index 2c033c6f28..e4f274b608 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_gcm.c @@ -241,10 +241,6 @@ handle_completed_gcm_crypto_op(struct ipsec_mb_qp *qp, /* Free session if a session-less crypto op */ if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sess, 0, sizeof(struct aesni_gcm_session)); - memset(op->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - op->sym->session)); - rte_mempool_put(qp->sess_mp_priv, sess); rte_mempool_put(qp->sess_mp, op->sym->session); op->sym->session = NULL; } @@ -455,44 +451,35 @@ static inline struct aesni_gcm_session * aesni_gcm_get_session(struct ipsec_mb_qp *qp, struct rte_crypto_op *op) { - struct aesni_gcm_session *sess = NULL; - uint32_t driver_id = - ipsec_mb_get_driver_id(IPSEC_MB_PMD_TYPE_AESNI_GCM); + struct rte_cryptodev_sym_session *sess = NULL; struct rte_crypto_sym_op *sym_op = op->sym; if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { if (likely(sym_op->session != NULL)) - sess = (struct aesni_gcm_session *) - get_sym_session_private_data(sym_op->session, - driver_id); + sess = sym_op->session; } else { - void *_sess; - void *_sess_private_data = NULL; - - if (rte_mempool_get(qp->sess_mp, (void **)&_sess)) + if (rte_mempool_get(qp->sess_mp, (void **)&sess)) return NULL; - if (rte_mempool_get(qp->sess_mp_priv, - (void **)&_sess_private_data)) + if (unlikely(sess->sess_data_sz < + sizeof(struct aesni_gcm_session))) { + rte_mempool_put(qp->sess_mp, sess); return NULL; - - sess = (struct aesni_gcm_session *)_sess_private_data; + } if (unlikely(aesni_gcm_session_configure(qp->mb_mgr, - _sess_private_data, sym_op->xform) != 0)) { - rte_mempool_put(qp->sess_mp, _sess); - rte_mempool_put(qp->sess_mp_priv, _sess_private_data); + (void *)sess->driver_priv_data, + sym_op->xform) != 0)) { + rte_mempool_put(qp->sess_mp, sess); sess = NULL; } - sym_op->session = (struct rte_cryptodev_sym_session *)_sess; - set_sym_session_private_data(sym_op->session, driver_id, - _sess_private_data); + sym_op->session = sess; } if (unlikely(sess == NULL)) op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION; - return sess; + return (void *)sess->driver_priv_data; } static uint16_t @@ -712,22 +699,15 @@ aesni_gmac_sgl_verify(struct aesni_gcm_session *s, /** Process CPU crypto bulk operations */ static uint32_t -aesni_gcm_process_bulk(struct rte_cryptodev *dev, +aesni_gcm_process_bulk(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess, __rte_unused union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec) { - struct aesni_gcm_session *s; + struct aesni_gcm_session *s = (void *)sess->driver_priv_data; struct gcm_context_data gdata_ctx; IMB_MGR *mb_mgr; - s = (struct aesni_gcm_session *) get_sym_session_private_data(sess, - dev->driver_id); - if (unlikely(s == NULL)) { - aesni_gcm_fill_error_code(vec, EINVAL); - return 0; - } - /* get per-thread MB MGR, create one if needed */ mb_mgr = get_per_thread_mb_mgr(); if (unlikely(mb_mgr == NULL)) diff --git a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c index 6d5d3ce8eb..fe5a7005f3 100644 --- a/drivers/crypto/ipsec_mb/pmd_aesni_mb.c +++ b/drivers/crypto/ipsec_mb/pmd_aesni_mb.c @@ -1710,8 +1710,6 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) { struct rte_crypto_op *op = (struct rte_crypto_op *)job->user_data; struct aesni_mb_session *sess = NULL; - uint32_t driver_id = ipsec_mb_get_driver_id( - IPSEC_MB_PMD_TYPE_AESNI_MB); #ifdef AESNI_MB_DOCSIS_SEC_ENABLED uint8_t is_docsis_sec = 0; @@ -1725,15 +1723,7 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) sess = get_sec_session_private_data(op->sym->sec_session); } else #endif - { - sess = get_sym_session_private_data(op->sym->session, - driver_id); - } - - if (unlikely(sess == NULL)) { - op->status = RTE_CRYPTO_OP_STATUS_INVALID_SESSION; - return op; - } + sess = (void *)op->sym->session->driver_priv_data; if (likely(op->status == RTE_CRYPTO_OP_STATUS_NOT_PROCESSED)) { switch (job->status) { @@ -1771,10 +1761,6 @@ post_process_mb_job(struct ipsec_mb_qp *qp, IMB_JOB *job) /* Free session if a session-less crypto op */ if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sess, 0, sizeof(struct aesni_mb_session)); - memset(op->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - op->sym->session)); - rte_mempool_put(qp->sess_mp_priv, sess); rte_mempool_put(qp->sess_mp, op->sym->session); op->sym->session = NULL; } @@ -2028,7 +2014,7 @@ verify_sync_dgst(struct rte_crypto_sym_vec *vec, } static uint32_t -aesni_mb_process_bulk(struct rte_cryptodev *dev, +aesni_mb_process_bulk(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs sofs, struct rte_crypto_sym_vec *vec) { @@ -2037,15 +2023,9 @@ aesni_mb_process_bulk(struct rte_cryptodev *dev, void *buf; IMB_JOB *job; IMB_MGR *mb_mgr; - struct aesni_mb_session *s; + struct aesni_mb_session *s = (void *)sess->driver_priv_data; uint8_t tmp_dgst[vec->num][DIGEST_LENGTH_MAX]; - s = get_sym_session_private_data(sess, dev->driver_id); - if (s == NULL) { - ipsec_mb_fill_error_code(vec, EINVAL); - return 0; - } - /* get per-thread MB MGR, create one if needed */ mb_mgr = get_per_thread_mb_mgr(); if (unlikely(mb_mgr == NULL)) diff --git a/drivers/crypto/ipsec_mb/pmd_chacha_poly.c b/drivers/crypto/ipsec_mb/pmd_chacha_poly.c index d953d6e5f5..97e7cef233 100644 --- a/drivers/crypto/ipsec_mb/pmd_chacha_poly.c +++ b/drivers/crypto/ipsec_mb/pmd_chacha_poly.c @@ -290,10 +290,6 @@ handle_completed_chacha20_poly1305_crypto_op(struct ipsec_mb_qp *qp, /* Free session if a session-less crypto op */ if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sess, 0, sizeof(struct chacha20_poly1305_session)); - memset(op->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - op->sym->session)); - rte_mempool_put(qp->sess_mp_priv, sess); rte_mempool_put(qp->sess_mp, op->sym->session); op->sym->session = NULL; } diff --git a/drivers/crypto/ipsec_mb/pmd_kasumi.c b/drivers/crypto/ipsec_mb/pmd_kasumi.c index c9d4f9d0ae..b2fb168154 100644 --- a/drivers/crypto/ipsec_mb/pmd_kasumi.c +++ b/drivers/crypto/ipsec_mb/pmd_kasumi.c @@ -231,11 +231,6 @@ process_ops(struct rte_crypto_op **ops, struct kasumi_session *session, /* Free session if a session-less crypto op. */ if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(session, 0, sizeof(struct kasumi_session)); - memset( - ops[i]->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - ops[i]->sym->session)); - rte_mempool_put(qp->sess_mp_priv, session); rte_mempool_put(qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session = NULL; } @@ -287,8 +282,9 @@ process_op_bit(struct rte_crypto_op *op, struct kasumi_session *session, /* Free session if a session-less crypto op. */ if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { - memset(op->sym->session, 0, sizeof(struct kasumi_session)); - rte_cryptodev_sym_session_free(op->sym->session); + memset(op->sym->session->driver_priv_data, 0, + sizeof(struct kasumi_session)); + rte_mempool_put(qp->sess_mp, (void *)op->sym->session); op->sym->session = NULL; } return processed_op; diff --git a/drivers/crypto/ipsec_mb/pmd_snow3g.c b/drivers/crypto/ipsec_mb/pmd_snow3g.c index 9a85f46721..f052d6d847 100644 --- a/drivers/crypto/ipsec_mb/pmd_snow3g.c +++ b/drivers/crypto/ipsec_mb/pmd_snow3g.c @@ -362,10 +362,6 @@ process_ops(struct rte_crypto_op **ops, struct snow3g_session *session, /* Free session if a session-less crypto op. */ if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(session, 0, sizeof(struct snow3g_session)); - memset(ops[i]->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - ops[i]->sym->session)); - rte_mempool_put(qp->sess_mp_priv, session); rte_mempool_put(qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session = NULL; } @@ -417,8 +413,9 @@ process_op_bit(struct rte_crypto_op *op, struct snow3g_session *session, /* Free session if a session-less crypto op. */ if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { - memset(op->sym->session, 0, sizeof(struct snow3g_session)); - rte_cryptodev_sym_session_free(op->sym->session); + memset(op->sym->session->driver_priv_data, 0, + sizeof(struct snow3g_session)); + rte_mempool_put(qp->sess_mp, (void *)op->sym->session); op->sym->session = NULL; } diff --git a/drivers/crypto/ipsec_mb/pmd_zuc.c b/drivers/crypto/ipsec_mb/pmd_zuc.c index e36c7092d6..92fd9d1808 100644 --- a/drivers/crypto/ipsec_mb/pmd_zuc.c +++ b/drivers/crypto/ipsec_mb/pmd_zuc.c @@ -239,10 +239,6 @@ process_ops(struct rte_crypto_op **ops, enum ipsec_mb_operation op_type, /* Free session if a session-less crypto op. */ if (ops[i]->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(sessions[i], 0, sizeof(struct zuc_session)); - memset(ops[i]->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - ops[i]->sym->session)); - rte_mempool_put(qp->sess_mp_priv, sessions[i]); rte_mempool_put(qp->sess_mp, ops[i]->sym->session); ops[i]->sym->session = NULL; } diff --git a/drivers/crypto/mlx5/mlx5_crypto.c b/drivers/crypto/mlx5/mlx5_crypto.c index eb7616257e..46ea507cf5 100644 --- a/drivers/crypto/mlx5/mlx5_crypto.c +++ b/drivers/crypto/mlx5/mlx5_crypto.c @@ -171,14 +171,13 @@ mlx5_crypto_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) static int mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *session, - struct rte_mempool *mp) + struct rte_cryptodev_sym_session *session) { struct mlx5_crypto_priv *priv = dev->data->dev_private; - struct mlx5_crypto_session *sess_private_data; + struct mlx5_crypto_session *sess_private_data = + (void *)session->driver_priv_data; struct rte_crypto_cipher_xform *cipher; uint8_t encryption_order; - int ret; if (unlikely(xform->next != NULL)) { DRV_LOG(ERR, "Xform next is not supported."); @@ -189,17 +188,9 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, DRV_LOG(ERR, "Only AES-XTS algorithm is supported."); return -ENOTSUP; } - ret = rte_mempool_get(mp, (void *)&sess_private_data); - if (ret != 0) { - DRV_LOG(ERR, - "Failed to get session %p private data from mempool.", - sess_private_data); - return -ENOMEM; - } cipher = &xform->cipher; sess_private_data->dek = mlx5_crypto_dek_prepare(priv, cipher); if (sess_private_data->dek == NULL) { - rte_mempool_put(mp, sess_private_data); DRV_LOG(ERR, "Failed to prepare dek."); return -ENOMEM; } @@ -239,8 +230,6 @@ mlx5_crypto_sym_session_configure(struct rte_cryptodev *dev, sess_private_data->dek_id = rte_cpu_to_be_32(sess_private_data->dek->obj->id & 0xffffff); - set_sym_session_private_data(session, dev->driver_id, - sess_private_data); DRV_LOG(DEBUG, "Session %p was configured.", sess_private_data); return 0; } @@ -250,16 +239,13 @@ mlx5_crypto_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) { struct mlx5_crypto_priv *priv = dev->data->dev_private; - struct mlx5_crypto_session *spriv = get_sym_session_private_data(sess, - dev->driver_id); + struct mlx5_crypto_session *spriv = (void *)sess->driver_priv_data; if (unlikely(spriv == NULL)) { DRV_LOG(ERR, "Failed to get session %p private data.", spriv); return; } mlx5_crypto_dek_destroy(priv, spriv->dek); - set_sym_session_private_data(sess, dev->driver_id, NULL); - rte_mempool_put(rte_mempool_from_obj(spriv), spriv); DRV_LOG(DEBUG, "Session %p was cleared.", spriv); } @@ -369,8 +355,8 @@ mlx5_crypto_wqe_set(struct mlx5_crypto_priv *priv, struct rte_crypto_op *op, struct mlx5_umr_wqe *umr) { - struct mlx5_crypto_session *sess = get_sym_session_private_data - (op->sym->session, mlx5_crypto_driver_id); + struct mlx5_crypto_session *sess = + (void *)op->sym->session->driver_priv_data; struct mlx5_wqe_cseg *cseg = &umr->ctr; struct mlx5_wqe_mkey_cseg *mkc = &umr->mkc; struct mlx5_wqe_dseg *klms = &umr->kseg[0]; diff --git a/drivers/crypto/mvsam/rte_mrvl_pmd.c b/drivers/crypto/mvsam/rte_mrvl_pmd.c index 04efd9aaa8..bb86f671a4 100644 --- a/drivers/crypto/mvsam/rte_mrvl_pmd.c +++ b/drivers/crypto/mvsam/rte_mrvl_pmd.c @@ -597,13 +597,7 @@ mrvl_request_prepare_crp(struct sam_cio_op_params *request, return -EINVAL; } - sess = (struct mrvl_crypto_session *)get_sym_session_private_data( - op->sym->session, - cryptodev_driver_id); - if (unlikely(sess == NULL)) { - MRVL_LOG(ERR, "Session was not created for this device!"); - return -EINVAL; - } + sess = (struct mrvl_crypto_session *)op->sym->session->data; request->sa = sess->sam_sess; request->cookie = op; diff --git a/drivers/crypto/mvsam/rte_mrvl_pmd_ops.c b/drivers/crypto/mvsam/rte_mrvl_pmd_ops.c index f828dc9db5..214c0fcb08 100644 --- a/drivers/crypto/mvsam/rte_mrvl_pmd_ops.c +++ b/drivers/crypto/mvsam/rte_mrvl_pmd_ops.c @@ -736,8 +736,7 @@ mrvl_crypto_pmd_sym_session_get_size(__rte_unused struct rte_cryptodev *dev) static int mrvl_crypto_pmd_sym_session_configure(__rte_unused struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mp) + struct rte_cryptodev_sym_session *sess) { struct mrvl_crypto_session *mrvl_sess; void *sess_private_data; @@ -748,23 +747,15 @@ mrvl_crypto_pmd_sym_session_configure(__rte_unused struct rte_cryptodev *dev, return -EINVAL; } - if (rte_mempool_get(mp, &sess_private_data)) { - CDEV_LOG_ERR("Couldn't get object from session mempool."); - return -ENOMEM; - } - + sess_private_data = sess->data; memset(sess_private_data, 0, sizeof(struct mrvl_crypto_session)); ret = mrvl_crypto_set_session_parameters(sess_private_data, xform); if (ret != 0) { MRVL_LOG(ERR, "Failed to configure session parameters!"); - - /* Return session to mempool */ - rte_mempool_put(mp, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, sess_private_data); mrvl_sess = (struct mrvl_crypto_session *)sess_private_data; if (sam_session_create(&mrvl_sess->sam_sess_params, @@ -791,8 +782,7 @@ mrvl_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess) { - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); + void *sess_priv = sess->data; /* Zero out the whole structure */ if (sess_priv) { @@ -803,11 +793,6 @@ mrvl_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, sam_session_destroy(mrvl_sess->sam_sess) < 0) { MRVL_LOG(ERR, "Error while destroying session!"); } - - memset(mrvl_sess, 0, sizeof(struct mrvl_crypto_session)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); } } diff --git a/drivers/crypto/nitrox/nitrox_sym.c b/drivers/crypto/nitrox/nitrox_sym.c index cb5393d2f1..505024a810 100644 --- a/drivers/crypto/nitrox/nitrox_sym.c +++ b/drivers/crypto/nitrox/nitrox_sym.c @@ -530,24 +530,16 @@ configure_aead_ctx(struct rte_crypto_aead_xform *xform, } static int -nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev, +nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev __rte_unused, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { - void *mp_obj; - struct nitrox_crypto_ctx *ctx; + struct nitrox_crypto_ctx *ctx = (void *)sess->driver_priv_data; struct rte_crypto_cipher_xform *cipher_xform = NULL; struct rte_crypto_auth_xform *auth_xform = NULL; struct rte_crypto_aead_xform *aead_xform = NULL; int ret = -EINVAL; - if (rte_mempool_get(mempool, &mp_obj)) { - NITROX_LOG(ERR, "Couldn't allocate context\n"); - return -ENOMEM; - } - - ctx = mp_obj; ctx->nitrox_chain = get_crypto_chain_order(xform); switch (ctx->nitrox_chain) { case NITROX_CHAIN_CIPHER_ONLY: @@ -585,38 +577,23 @@ nitrox_sym_dev_sess_configure(struct rte_cryptodev *cdev, goto err; } - ctx->iova = rte_mempool_virt2iova(ctx); - set_sym_session_private_data(sess, cdev->driver_id, ctx); + ctx->iova = sess->driver_priv_data_iova; return 0; err: - rte_mempool_put(mempool, mp_obj); return ret; } static void -nitrox_sym_dev_sess_clear(struct rte_cryptodev *cdev, - struct rte_cryptodev_sym_session *sess) -{ - struct nitrox_crypto_ctx *ctx = get_sym_session_private_data(sess, - cdev->driver_id); - struct rte_mempool *sess_mp; - - if (!ctx) - return; - - memset(ctx, 0, sizeof(*ctx)); - sess_mp = rte_mempool_from_obj(ctx); - set_sym_session_private_data(sess, cdev->driver_id, NULL); - rte_mempool_put(sess_mp, ctx); -} +nitrox_sym_dev_sess_clear(struct rte_cryptodev *cdev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{} static struct nitrox_crypto_ctx * get_crypto_ctx(struct rte_crypto_op *op) { if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { if (likely(op->sym->session)) - return get_sym_session_private_data(op->sym->session, - nitrox_sym_drv_id); + return (void *)op->sym->session->driver_priv_data; } return NULL; diff --git a/drivers/crypto/null/null_crypto_pmd.c b/drivers/crypto/null/null_crypto_pmd.c index 9ecb434fd0..09aac3065e 100644 --- a/drivers/crypto/null/null_crypto_pmd.c +++ b/drivers/crypto/null/null_crypto_pmd.c @@ -58,7 +58,7 @@ process_op(const struct null_crypto_qp *qp, struct rte_crypto_op *op, if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { memset(op->sym->session, 0, sizeof(struct null_crypto_session)); - rte_cryptodev_sym_session_free(op->sym->session); + rte_mempool_put(qp->sess_mp, (void *)op->sym->session); op->sym->session = NULL; } @@ -78,30 +78,21 @@ get_session(struct null_crypto_qp *qp, struct rte_crypto_op *op) if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { if (likely(sym_op->session != NULL)) sess = (struct null_crypto_session *) - get_sym_session_private_data( - sym_op->session, cryptodev_driver_id); + sym_op->session->driver_priv_data; } else { - void *_sess = NULL; - void *_sess_private_data = NULL; + struct rte_cryptodev_sym_session *_sess = NULL; if (rte_mempool_get(qp->sess_mp, (void **)&_sess)) return NULL; - if (rte_mempool_get(qp->sess_mp_priv, - (void **)&_sess_private_data)) - return NULL; - - sess = (struct null_crypto_session *)_sess_private_data; + sess = (struct null_crypto_session *)_sess->driver_priv_data; if (unlikely(null_crypto_set_session_parameters(sess, sym_op->xform) != 0)) { rte_mempool_put(qp->sess_mp, _sess); - rte_mempool_put(qp->sess_mp_priv, _sess_private_data); sess = NULL; } - sym_op->session = (struct rte_cryptodev_sym_session *)_sess; - set_sym_session_private_data(op->sym->session, - cryptodev_driver_id, _sess_private_data); + sym_op->session = _sess; } return sess; diff --git a/drivers/crypto/null/null_crypto_pmd_ops.c b/drivers/crypto/null/null_crypto_pmd_ops.c index 90a675dfff..fb43d3f7b5 100644 --- a/drivers/crypto/null/null_crypto_pmd_ops.c +++ b/drivers/crypto/null/null_crypto_pmd_ops.c @@ -233,7 +233,6 @@ null_crypto_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, } qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; memset(&qp->qp_stats, 0, sizeof(qp->qp_stats)); @@ -256,8 +255,7 @@ null_crypto_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) static int null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mp) + struct rte_cryptodev_sym_session *sess) { void *sess_private_data; int ret; @@ -267,43 +265,22 @@ null_crypto_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused, return -EINVAL; } - if (rte_mempool_get(mp, &sess_private_data)) { - NULL_LOG(ERR, - "Couldn't get object from session mempool"); - return -ENOMEM; - } + sess_private_data = (void *)sess->driver_priv_data; ret = null_crypto_set_session_parameters(sess_private_data, xform); if (ret != 0) { NULL_LOG(ERR, "failed configure session parameters"); - - /* Return session to mempool */ - rte_mempool_put(mp, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - return 0; } /** Clear the memory of session so it doesn't leave key material behind */ static void -null_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess) -{ - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); - - /* Zero out the whole structure */ - if (sess_priv) { - memset(sess_priv, 0, sizeof(struct null_crypto_session)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); - } -} +null_crypto_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{} static struct rte_cryptodev_ops pmd_ops = { .dev_configure = null_crypto_pmd_config, diff --git a/drivers/crypto/null/null_crypto_pmd_private.h b/drivers/crypto/null/null_crypto_pmd_private.h index 89c4345b6f..ae34ce6671 100644 --- a/drivers/crypto/null/null_crypto_pmd_private.h +++ b/drivers/crypto/null/null_crypto_pmd_private.h @@ -31,8 +31,6 @@ struct null_crypto_qp { /**< Ring for placing process packets */ struct rte_mempool *sess_mp; /**< Session Mempool */ - struct rte_mempool *sess_mp_priv; - /**< Session Mempool */ struct rte_cryptodev_stats qp_stats; /**< Queue pair statistics */ } __rte_cache_aligned; diff --git a/drivers/crypto/octeontx/otx_cryptodev_hw_access.h b/drivers/crypto/octeontx/otx_cryptodev_hw_access.h index e48805fb09..4647d568de 100644 --- a/drivers/crypto/octeontx/otx_cryptodev_hw_access.h +++ b/drivers/crypto/octeontx/otx_cryptodev_hw_access.h @@ -49,7 +49,6 @@ struct cpt_instance { uint32_t queue_id; uintptr_t rsvd; struct rte_mempool *sess_mp; - struct rte_mempool *sess_mp_priv; struct cpt_qp_meta_info meta_info; uint8_t ca_enabled; }; diff --git a/drivers/crypto/octeontx/otx_cryptodev_ops.c b/drivers/crypto/octeontx/otx_cryptodev_ops.c index 7aee67ab09..cd5ebc2af6 100644 --- a/drivers/crypto/octeontx/otx_cryptodev_ops.c +++ b/drivers/crypto/octeontx/otx_cryptodev_ops.c @@ -171,7 +171,6 @@ otx_cpt_que_pair_setup(struct rte_cryptodev *dev, instance->queue_id = que_pair_id; instance->sess_mp = qp_conf->mp_session; - instance->sess_mp_priv = qp_conf->mp_session_private; dev->data->queue_pairs[que_pair_id] = instance; return 0; @@ -243,25 +242,19 @@ sym_xform_verify(struct rte_crypto_sym_xform *xform) } static int -sym_session_configure(int driver_id, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *pool) +sym_session_configure(struct rte_crypto_sym_xform *xform, + struct rte_cryptodev_sym_session *sess) { struct rte_crypto_sym_xform *temp_xform = xform; struct cpt_sess_misc *misc; vq_cmd_word3_t vq_cmd_w3; - void *priv; + void *priv = (void *)sess->driver_priv_data; int ret; ret = sym_xform_verify(xform); if (unlikely(ret)) return ret; - if (unlikely(rte_mempool_get(pool, &priv))) { - CPT_LOG_ERR("Could not allocate session private data"); - return -ENOMEM; - } - memset(priv, 0, sizeof(struct cpt_sess_misc) + offsetof(struct cpt_ctx, mc_ctx)); @@ -301,9 +294,7 @@ sym_session_configure(int driver_id, struct rte_crypto_sym_xform *xform, goto priv_put; } - set_sym_session_private_data(sess, driver_id, priv); - - misc->ctx_dma_addr = rte_mempool_virt2iova(misc) + + misc->ctx_dma_addr = sess->driver_priv_data_iova + sizeof(struct cpt_sess_misc); vq_cmd_w3.u64 = 0; @@ -316,17 +307,14 @@ sym_session_configure(int driver_id, struct rte_crypto_sym_xform *xform, return 0; priv_put: - if (priv) - rte_mempool_put(pool, priv); return -ENOTSUP; } static void -sym_session_clear(int driver_id, struct rte_cryptodev_sym_session *sess) +sym_session_clear(struct rte_cryptodev_sym_session *sess) { - void *priv = get_sym_session_private_data(sess, driver_id); + void *priv = (void *)sess->driver_priv_data; struct cpt_sess_misc *misc; - struct rte_mempool *pool; struct cpt_ctx *ctx; if (priv == NULL) @@ -336,35 +324,26 @@ sym_session_clear(int driver_id, struct rte_cryptodev_sym_session *sess) ctx = SESS_PRIV(misc); rte_free(ctx->auth_key); - - memset(priv, 0, cpt_get_session_size()); - - pool = rte_mempool_from_obj(priv); - - set_sym_session_private_data(sess, driver_id, NULL); - - rte_mempool_put(pool, priv); } static int -otx_cpt_session_cfg(struct rte_cryptodev *dev, +otx_cpt_session_cfg(struct rte_cryptodev *dev __rte_unused, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *pool) + struct rte_cryptodev_sym_session *sess) { CPT_PMD_INIT_FUNC_TRACE(); - return sym_session_configure(dev->driver_id, xform, sess, pool); + return sym_session_configure(xform, sess); } static void -otx_cpt_session_clear(struct rte_cryptodev *dev, +otx_cpt_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { CPT_PMD_INIT_FUNC_TRACE(); - return sym_session_clear(dev->driver_id, sess); + return sym_session_clear(sess); } static unsigned int @@ -528,10 +507,7 @@ otx_cpt_enq_single_sym(struct cpt_instance *instance, void *req; uint64_t cpt_op; - sess = (struct cpt_sess_misc *) - get_sym_session_private_data(sym_op->session, - otx_cryptodev_driver_id); - + sess = (struct cpt_sess_misc *)sym_op->session->driver_priv_data; cpt_op = sess->cpt_op; if (likely(cpt_op & CPT_OP_CIPHER_MASK)) @@ -560,21 +536,18 @@ static __rte_always_inline void * __rte_hot otx_cpt_enq_single_sym_sessless(struct cpt_instance *instance, struct rte_crypto_op *op) { - const int driver_id = otx_cryptodev_driver_id; struct rte_crypto_sym_op *sym_op = op->sym; struct rte_cryptodev_sym_session *sess; void *req; int ret; /* Create temporary session */ - sess = rte_cryptodev_sym_session_create(instance->sess_mp); - if (sess == NULL) { + if (rte_mempool_get(instance->sess_mp, (void **)&sess) < 0) { rte_errno = ENOMEM; return NULL; } - ret = sym_session_configure(driver_id, sym_op->xform, sess, - instance->sess_mp_priv); + ret = sym_session_configure(sym_op->xform, sess); if (ret) goto sess_put; @@ -583,12 +556,10 @@ otx_cpt_enq_single_sym_sessless(struct cpt_instance *instance, /* Enqueue op with the tmp session set */ req = otx_cpt_enq_single_sym(instance, op); if (unlikely(req == NULL)) - goto priv_put; + goto sess_put; return req; -priv_put: - sym_session_clear(driver_id, sess); sess_put: rte_mempool_put(instance->sess_mp, sess); return NULL; @@ -873,13 +844,9 @@ static inline void free_sym_session_data(const struct cpt_instance *instance, struct rte_crypto_op *cop) { - void *sess_private_data_t = get_sym_session_private_data( - cop->sym->session, otx_cryptodev_driver_id); + void *sess_private_data_t = (void *)cop->sym->session->driver_priv_data; + memset(sess_private_data_t, 0, cpt_get_session_size()); - memset(cop->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - cop->sym->session)); - rte_mempool_put(instance->sess_mp_priv, sess_private_data_t); rte_mempool_put(instance->sess_mp, cop->sym->session); cop->sym->session = NULL; } diff --git a/drivers/crypto/openssl/openssl_pmd_private.h b/drivers/crypto/openssl/openssl_pmd_private.h index c34fd9a546..ed6841e460 100644 --- a/drivers/crypto/openssl/openssl_pmd_private.h +++ b/drivers/crypto/openssl/openssl_pmd_private.h @@ -70,8 +70,6 @@ struct openssl_qp { /**< Ring for placing process packets */ struct rte_mempool *sess_mp; /**< Session Mempool */ - struct rte_mempool *sess_mp_priv; - /**< Session Private Data Mempool */ struct rte_cryptodev_stats stats; /**< Queue pair statistics */ uint8_t temp_digest[DIGEST_LENGTH_MAX]; diff --git a/drivers/crypto/openssl/rte_openssl_pmd.c b/drivers/crypto/openssl/rte_openssl_pmd.c index 392b546a63..48e8ff107b 100644 --- a/drivers/crypto/openssl/rte_openssl_pmd.c +++ b/drivers/crypto/openssl/rte_openssl_pmd.c @@ -887,10 +887,8 @@ get_session(struct openssl_qp *qp, struct rte_crypto_op *op) if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { /* get existing session */ if (likely(op->sym->session != NULL)) - sess = (struct openssl_session *) - get_sym_session_private_data( - op->sym->session, - cryptodev_driver_id); + sess = (void *) + op->sym->session->driver_priv_data; } else { if (likely(op->asym->session != NULL)) asym_sess = (struct openssl_asym_session *) @@ -901,32 +899,26 @@ get_session(struct openssl_qp *qp, struct rte_crypto_op *op) return asym_sess; } } else { + struct rte_cryptodev_sym_session *_sess; /* sessionless asymmetric not supported */ if (op->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) return NULL; /* provide internal session */ - void *_sess = rte_cryptodev_sym_session_create(qp->sess_mp); - void *_sess_private_data = NULL; + rte_mempool_get(qp->sess_mp, (void **)&_sess); if (_sess == NULL) return NULL; - if (rte_mempool_get(qp->sess_mp_priv, - (void **)&_sess_private_data)) - return NULL; - - sess = (struct openssl_session *)_sess_private_data; + sess = (struct openssl_session *)_sess->driver_priv_data; if (unlikely(openssl_set_session_parameters(sess, op->sym->xform) != 0)) { rte_mempool_put(qp->sess_mp, _sess); - rte_mempool_put(qp->sess_mp_priv, _sess_private_data); sess = NULL; } op->sym->session = (struct rte_cryptodev_sym_session *)_sess; - set_sym_session_private_data(op->sym->session, - cryptodev_driver_id, _sess_private_data); + } if (sess == NULL) @@ -2900,10 +2892,6 @@ process_op(struct openssl_qp *qp, struct rte_crypto_op *op, if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS) { openssl_reset_session(sess); memset(sess, 0, sizeof(struct openssl_session)); - memset(op->sym->session, 0, - rte_cryptodev_sym_get_existing_header_session_size( - op->sym->session)); - rte_mempool_put(qp->sess_mp_priv, sess); rte_mempool_put(qp->sess_mp, op->sym->session); op->sym->session = NULL; } diff --git a/drivers/crypto/openssl/rte_openssl_pmd_ops.c b/drivers/crypto/openssl/rte_openssl_pmd_ops.c index f7ddbf9c73..2a3662ee5a 100644 --- a/drivers/crypto/openssl/rte_openssl_pmd_ops.c +++ b/drivers/crypto/openssl/rte_openssl_pmd_ops.c @@ -764,7 +764,6 @@ openssl_pmd_qp_setup(struct rte_cryptodev *dev, uint16_t qp_id, goto qp_setup_cleanup; qp->sess_mp = qp_conf->mp_session; - qp->sess_mp_priv = qp_conf->mp_session_private; memset(&qp->stats, 0, sizeof(qp->stats)); @@ -794,10 +793,9 @@ openssl_pmd_asym_session_get_size(struct rte_cryptodev *dev __rte_unused) static int openssl_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { - void *sess_private_data; + void *sess_private_data = (void *)sess->driver_priv_data; int ret; if (unlikely(sess == NULL)) { @@ -805,24 +803,14 @@ openssl_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused, return -EINVAL; } - if (rte_mempool_get(mempool, &sess_private_data)) { - OPENSSL_LOG(ERR, - "Couldn't get object from session mempool"); - return -ENOMEM; - } - ret = openssl_set_session_parameters(sess_private_data, xform); if (ret != 0) { OPENSSL_LOG(ERR, "failed configure session parameters"); /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - return 0; } @@ -1328,20 +1316,13 @@ openssl_pmd_asym_session_configure(struct rte_cryptodev *dev __rte_unused, /** Clear the memory of session so it doesn't leave key material behind */ static void -openssl_pmd_sym_session_clear(struct rte_cryptodev *dev, +openssl_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); + void *sess_priv = (void *)sess->driver_priv_data; /* Zero out the whole structure */ - if (sess_priv) { - openssl_reset_session(sess_priv); - memset(sess_priv, 0, sizeof(struct openssl_session)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); - } + openssl_reset_session(sess_priv); } static void openssl_reset_asym_session(struct openssl_asym_session *sess) diff --git a/drivers/crypto/qat/qat_sym.c b/drivers/crypto/qat/qat_sym.c index db6316fbe9..408ae9e42b 100644 --- a/drivers/crypto/qat/qat_sym.c +++ b/drivers/crypto/qat/qat_sym.c @@ -67,12 +67,7 @@ qat_sym_build_request(void *in_op, uint8_t *out_msg, return -EINVAL; if (likely(op->sess_type == RTE_CRYPTO_OP_WITH_SESSION)) { - ctx = get_sym_session_private_data(op->sym->session, - qat_sym_driver_id); - if (unlikely(!ctx)) { - QAT_DP_LOG(ERR, "No session for this device"); - return -EINVAL; - } + ctx = (void *)op->sym->session->driver_priv_data; if (sess != (uintptr_t)ctx) { struct rte_cryptodev *cdev; struct qat_cryptodev_private *internals; @@ -391,8 +386,7 @@ qat_sym_configure_dp_ctx(struct rte_cryptodev *dev, uint16_t qp_id, if (sess_type != RTE_CRYPTO_OP_WITH_SESSION) return -EINVAL; - ctx = (struct qat_sym_session *)get_sym_session_private_data( - session_ctx.crypto_sess, qat_sym_driver_id); + ctx = (void *)session_ctx.crypto_sess->driver_priv_data; dp_ctx->session = ctx; diff --git a/drivers/crypto/qat/qat_sym.h b/drivers/crypto/qat/qat_sym.h index 074612c11b..2853ac5b88 100644 --- a/drivers/crypto/qat/qat_sym.h +++ b/drivers/crypto/qat/qat_sym.h @@ -317,9 +317,7 @@ qat_sym_process_response(void **op, uint8_t *resp, void *op_cookie, #endif { sess = (struct qat_sym_session *) - get_sym_session_private_data( - rx_op->sym->session, - qat_sym_driver_id); + rx_op->sym->session->driver_priv_data; is_docsis_sec = 0; } diff --git a/drivers/crypto/qat/qat_sym_session.c b/drivers/crypto/qat/qat_sym_session.c index 52b3455cf0..8dd2357702 100644 --- a/drivers/crypto/qat/qat_sym_session.c +++ b/drivers/crypto/qat/qat_sym_session.c @@ -219,22 +219,13 @@ qat_is_auth_alg_supported(enum rte_crypto_auth_algorithm algo, } void -qat_sym_session_clear(struct rte_cryptodev *dev, +qat_sym_session_clear(struct rte_cryptodev *dev __rte_unused, struct rte_cryptodev_sym_session *sess) { - uint8_t index = dev->driver_id; - void *sess_priv = get_sym_session_private_data(sess, index); - struct qat_sym_session *s = (struct qat_sym_session *)sess_priv; - - if (sess_priv) { - if (s->bpi_ctx) - bpi_cipher_ctx_free(s->bpi_ctx); - memset(s, 0, qat_sym_session_get_private_size(dev)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(sess_priv); + struct qat_sym_session *s = (void *)sess->driver_priv_data; - set_sym_session_private_data(sess, index, NULL); - rte_mempool_put(sess_mp, sess_priv); - } + if (s->bpi_ctx) + bpi_cipher_ctx_free(s->bpi_ctx); } static int @@ -505,35 +496,24 @@ qat_sym_session_configure_cipher(struct rte_cryptodev *dev, int qat_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { - void *sess_private_data; int ret; - if (rte_mempool_get(mempool, &sess_private_data)) { - CDEV_LOG_ERR( - "Couldn't get object from session mempool"); - return -ENOMEM; - } - #if (OPENSSL_VERSION_NUMBER >= 0x30000000L) if (ossl_legacy_provider_load()) return -EINVAL; #endif - ret = qat_sym_session_set_parameters(dev, xform, sess_private_data); + ret = qat_sym_session_set_parameters(dev, xform, + (void *)sess->driver_priv_data, + sess->driver_priv_data_iova); if (ret != 0) { QAT_LOG(ERR, "Crypto QAT PMD: failed to configure session parameters"); - /* Return session to mempool */ - rte_mempool_put(mempool, sess_private_data); return ret; } - set_sym_session_private_data(sess, dev->driver_id, - sess_private_data); - # if (OPENSSL_VERSION_NUMBER >= 0x30000000L) ossl_legacy_provider_unload(); # endif @@ -542,7 +522,8 @@ qat_sym_session_configure(struct rte_cryptodev *dev, int qat_sym_session_set_parameters(struct rte_cryptodev *dev, - struct rte_crypto_sym_xform *xform, void *session_private) + struct rte_crypto_sym_xform *xform, void *session_private, + rte_iova_t session_paddr) { struct qat_sym_session *session = session_private; struct qat_cryptodev_private *internals = dev->data->dev_private; @@ -551,7 +532,6 @@ qat_sym_session_set_parameters(struct rte_cryptodev *dev, int qat_cmd_id; /* Verify the session physical address is known */ - rte_iova_t session_paddr = rte_mempool_virt2iova(session); if (session_paddr == 0 || session_paddr == RTE_BAD_IOVA) { QAT_LOG(ERR, "Session physical address unknown. Bad memory pool."); diff --git a/drivers/crypto/qat/qat_sym_session.h b/drivers/crypto/qat/qat_sym_session.h index 01908abd9e..9e4aab06a6 100644 --- a/drivers/crypto/qat/qat_sym_session.h +++ b/drivers/crypto/qat/qat_sym_session.h @@ -123,12 +123,12 @@ struct qat_sym_session { int qat_sym_session_configure(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool); + struct rte_cryptodev_sym_session *sess); int qat_sym_session_set_parameters(struct rte_cryptodev *dev, - struct rte_crypto_sym_xform *xform, void *session_private); + struct rte_crypto_sym_xform *xform, void *session_private, + rte_iova_t session_private_iova); int qat_sym_session_configure_aead(struct rte_cryptodev *dev, diff --git a/drivers/crypto/scheduler/scheduler_pmd_ops.c b/drivers/crypto/scheduler/scheduler_pmd_ops.c index f3a1bd626c..b93821783b 100644 --- a/drivers/crypto/scheduler/scheduler_pmd_ops.c +++ b/drivers/crypto/scheduler/scheduler_pmd_ops.c @@ -468,44 +468,18 @@ scheduler_pmd_sym_session_get_size(struct rte_cryptodev *dev __rte_unused) } static int -scheduler_pmd_sym_session_configure(struct rte_cryptodev *dev, - struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) +scheduler_pmd_sym_session_configure(struct rte_cryptodev *dev __rte_unused, + struct rte_crypto_sym_xform *xform __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) { - struct scheduler_ctx *sched_ctx = dev->data->dev_private; - uint32_t i; - int ret; - - for (i = 0; i < sched_ctx->nb_workers; i++) { - struct scheduler_worker *worker = &sched_ctx->workers[i]; - - ret = rte_cryptodev_sym_session_init(worker->dev_id, sess, - xform, mempool); - if (ret < 0) { - CR_SCHED_LOG(ERR, "unable to config sym session"); - return ret; - } - } - return 0; } /** Clear the memory of session so it doesn't leave key material behind */ static void -scheduler_pmd_sym_session_clear(struct rte_cryptodev *dev, - struct rte_cryptodev_sym_session *sess) -{ - struct scheduler_ctx *sched_ctx = dev->data->dev_private; - uint32_t i; - - /* Clear private data of workers */ - for (i = 0; i < sched_ctx->nb_workers; i++) { - struct scheduler_worker *worker = &sched_ctx->workers[i]; - - rte_cryptodev_sym_session_clear(worker->dev_id, sess); - } -} +scheduler_pmd_sym_session_clear(struct rte_cryptodev *dev __rte_unused, + struct rte_cryptodev_sym_session *sess __rte_unused) +{} static struct rte_cryptodev_ops scheduler_pmd_ops = { .dev_configure = scheduler_pmd_config, diff --git a/drivers/crypto/virtio/virtio_cryptodev.c b/drivers/crypto/virtio/virtio_cryptodev.c index ed64866758..6b9e57ee66 100644 --- a/drivers/crypto/virtio/virtio_cryptodev.c +++ b/drivers/crypto/virtio/virtio_cryptodev.c @@ -40,8 +40,7 @@ static void virtio_crypto_sym_clear_session(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess); static int virtio_crypto_sym_configure_session(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *session, - struct rte_mempool *mp); + struct rte_cryptodev_sym_session *session); /* * The set of PCI devices this driver supports @@ -952,12 +951,7 @@ virtio_crypto_sym_clear_session( hw = dev->data->dev_private; vq = hw->cvq; - session = (struct virtio_crypto_session *)get_sym_session_private_data( - sess, cryptodev_virtio_driver_id); - if (session == NULL) { - VIRTIO_CRYPTO_SESSION_LOG_ERR("Invalid session parameter"); - return; - } + session = (struct virtio_crypto_session *)sess->driver_priv_data; VIRTIO_CRYPTO_SESSION_LOG_INFO("vq->vq_desc_head_idx = %d, " "vq = %p", vq->vq_desc_head_idx, vq); @@ -1070,10 +1064,6 @@ virtio_crypto_sym_clear_session( VIRTIO_CRYPTO_SESSION_LOG_INFO("Close session %"PRIu64" successfully ", session->session_id); - memset(session, 0, sizeof(struct virtio_crypto_session)); - struct rte_mempool *sess_mp = rte_mempool_from_obj(session); - set_sym_session_private_data(sess, cryptodev_virtio_driver_id, NULL); - rte_mempool_put(sess_mp, session); rte_free(malloc_virt_addr); } @@ -1292,11 +1282,9 @@ static int virtio_crypto_check_sym_configure_session_paras( struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sym_sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sym_sess) { - if (unlikely(xform == NULL) || unlikely(sym_sess == NULL) || - unlikely(mempool == NULL)) { + if (unlikely(xform == NULL) || unlikely(sym_sess == NULL)) { VIRTIO_CRYPTO_SESSION_LOG_ERR("NULL pointer"); return -1; } @@ -1311,12 +1299,9 @@ static int virtio_crypto_sym_configure_session( struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *sess, - struct rte_mempool *mempool) + struct rte_cryptodev_sym_session *sess) { int ret; - struct virtio_crypto_session crypto_sess; - void *session_private = &crypto_sess; struct virtio_crypto_session *session; struct virtio_crypto_op_ctrl_req *ctrl_req; enum virtio_crypto_cmd_id cmd_id; @@ -1328,19 +1313,12 @@ virtio_crypto_sym_configure_session( PMD_INIT_FUNC_TRACE(); ret = virtio_crypto_check_sym_configure_session_paras(dev, xform, - sess, mempool); + sess); if (ret < 0) { VIRTIO_CRYPTO_SESSION_LOG_ERR("Invalid parameters"); return ret; } - - if (rte_mempool_get(mempool, &session_private)) { - VIRTIO_CRYPTO_SESSION_LOG_ERR( - "Couldn't get object from session mempool"); - return -ENOMEM; - } - - session = (struct virtio_crypto_session *)session_private; + session = (struct virtio_crypto_session *)sess->driver_priv_data; memset(session, 0, sizeof(struct virtio_crypto_session)); ctrl_req = &session->ctrl; ctrl_req->header.opcode = VIRTIO_CRYPTO_CIPHER_CREATE_SESSION; @@ -1402,10 +1380,6 @@ virtio_crypto_sym_configure_session( "Unsupported operation chain order parameter"); goto error_out; } - - set_sym_session_private_data(sess, dev->driver_id, - session_private); - return 0; error_out: diff --git a/drivers/crypto/virtio/virtio_rxtx.c b/drivers/crypto/virtio/virtio_rxtx.c index 08359b3a39..b7f492a7f2 100644 --- a/drivers/crypto/virtio/virtio_rxtx.c +++ b/drivers/crypto/virtio/virtio_rxtx.c @@ -207,8 +207,7 @@ virtqueue_crypto_sym_enqueue_xmit( offsetof(struct virtio_crypto_op_cookie, iv); struct rte_crypto_sym_op *sym_op = cop->sym; struct virtio_crypto_session *session = - (struct virtio_crypto_session *)get_sym_session_private_data( - cop->sym->session, cryptodev_virtio_driver_id); + (void *)cop->sym->session->driver_priv_data; struct virtio_crypto_op_data_req *op_data_req; uint32_t hash_result_len = 0; struct virtio_crypto_op_cookie *crypto_op_cookie; diff --git a/examples/fips_validation/fips_dev_self_test.c b/examples/fips_validation/fips_dev_self_test.c index 19af134bbe..bce903e007 100644 --- a/examples/fips_validation/fips_dev_self_test.c +++ b/examples/fips_validation/fips_dev_self_test.c @@ -969,7 +969,6 @@ struct fips_dev_auto_test_env { struct rte_mempool *mpool; struct rte_mempool *op_pool; struct rte_mempool *sess_pool; - struct rte_mempool *sess_priv_pool; struct rte_mbuf *mbuf; struct rte_crypto_op *op; }; @@ -1479,13 +1478,8 @@ run_single_test(uint8_t dev_id, return ret; } - sess = rte_cryptodev_sym_session_create(env->sess_pool); - if (!sess) - return -ENOMEM; - - ret = rte_cryptodev_sym_session_init(dev_id, - sess, &xform, env->sess_priv_pool); - if (ret < 0) { + sess = rte_cryptodev_sym_session_create(dev_id, &xform, env->sess_pool); + if (!sess) { RTE_LOG(ERR, PMD, "Error %i: Init session\n", ret); return ret; } @@ -1508,8 +1502,7 @@ run_single_test(uint8_t dev_id, 1); } while (n_deqd == 0); - rte_cryptodev_sym_session_clear(dev_id, sess); - rte_cryptodev_sym_session_free(sess); + rte_cryptodev_sym_session_free(dev_id, sess); if (env->op->status != RTE_CRYPTO_OP_STATUS_SUCCESS) return -1; @@ -1527,7 +1520,6 @@ fips_dev_auto_test_uninit(uint8_t dev_id, rte_mempool_free(env->mpool); rte_mempool_free(env->op_pool); rte_mempool_free(env->sess_pool); - rte_mempool_free(env->sess_priv_pool); rte_cryptodev_stop(dev_id); } @@ -1535,7 +1527,7 @@ fips_dev_auto_test_uninit(uint8_t dev_id, static int fips_dev_auto_test_init(uint8_t dev_id, struct fips_dev_auto_test_env *env) { - struct rte_cryptodev_qp_conf qp_conf = {128, NULL, NULL}; + struct rte_cryptodev_qp_conf qp_conf = {128, NULL}; uint32_t sess_sz = rte_cryptodev_sym_get_private_session_size(dev_id); struct rte_cryptodev_config conf; char name[128]; @@ -1579,25 +1571,13 @@ fips_dev_auto_test_init(uint8_t dev_id, struct fips_dev_auto_test_env *env) snprintf(name, 128, "%s%u", "SELF_TEST_SESS_POOL", dev_id); env->sess_pool = rte_cryptodev_sym_session_pool_create(name, - 128, 0, 0, 0, rte_cryptodev_socket_id(dev_id)); + 128, sess_sz, 0, 0, rte_cryptodev_socket_id(dev_id)); if (!env->sess_pool) { ret = -ENOMEM; goto error_exit; } - memset(name, 0, 128); - snprintf(name, 128, "%s%u", "SELF_TEST_SESS_PRIV_POOL", dev_id); - - env->sess_priv_pool = rte_mempool_create(name, - 128, sess_sz, 0, 0, NULL, NULL, NULL, - NULL, rte_cryptodev_socket_id(dev_id), 0); - if (!env->sess_priv_pool) { - ret = -ENOMEM; - goto error_exit; - } - qp_conf.mp_session = env->sess_pool; - qp_conf.mp_session_private = env->sess_priv_pool; ret = rte_cryptodev_queue_pair_setup(dev_id, 0, &qp_conf, rte_cryptodev_socket_id(dev_id)); diff --git a/examples/fips_validation/main.c b/examples/fips_validation/main.c index 8bd5a66889..23878f17cd 100644 --- a/examples/fips_validation/main.c +++ b/examples/fips_validation/main.c @@ -54,7 +54,6 @@ struct cryptodev_fips_validate_env { uint16_t mbuf_data_room; struct rte_mempool *mpool; struct rte_mempool *sess_mpool; - struct rte_mempool *sess_priv_mpool; struct rte_mempool *op_pool; struct rte_mbuf *mbuf; uint8_t *digest; @@ -69,7 +68,7 @@ static int cryptodev_fips_validate_app_int(void) { struct rte_cryptodev_config conf = {rte_socket_id(), 1, 0}; - struct rte_cryptodev_qp_conf qp_conf = {128, NULL, NULL}; + struct rte_cryptodev_qp_conf qp_conf = {128, NULL}; struct rte_cryptodev_info dev_info; uint32_t sess_sz = rte_cryptodev_sym_get_private_session_size( env.dev_id); @@ -109,16 +108,11 @@ cryptodev_fips_validate_app_int(void) ret = -ENOMEM; env.sess_mpool = rte_cryptodev_sym_session_pool_create( - "FIPS_SESS_MEMPOOL", 16, 0, 0, 0, rte_socket_id()); + "FIPS_SESS_MEMPOOL", 16, sess_sz, 0, 0, + rte_socket_id()); if (!env.sess_mpool) goto error_exit; - env.sess_priv_mpool = rte_mempool_create("FIPS_SESS_PRIV_MEMPOOL", - 16, sess_sz, 0, 0, NULL, NULL, NULL, - NULL, rte_socket_id(), 0); - if (!env.sess_priv_mpool) - goto error_exit; - env.op_pool = rte_crypto_op_pool_create( "FIPS_OP_POOL", RTE_CRYPTO_OP_TYPE_SYMMETRIC, @@ -133,7 +127,6 @@ cryptodev_fips_validate_app_int(void) goto error_exit; qp_conf.mp_session = env.sess_mpool; - qp_conf.mp_session_private = env.sess_priv_mpool; ret = rte_cryptodev_queue_pair_setup(env.dev_id, 0, &qp_conf, rte_socket_id()); @@ -150,7 +143,6 @@ cryptodev_fips_validate_app_int(void) rte_mempool_free(env.mpool); rte_mempool_free(env.sess_mpool); - rte_mempool_free(env.sess_priv_mpool); rte_mempool_free(env.op_pool); return ret; @@ -161,11 +153,9 @@ cryptodev_fips_validate_app_uninit(void) { rte_pktmbuf_free(env.mbuf); rte_crypto_op_free(env.op); - rte_cryptodev_sym_session_clear(env.dev_id, env.sess); - rte_cryptodev_sym_session_free(env.sess); + rte_cryptodev_sym_session_free(env.dev_id, env.sess); rte_mempool_free(env.mpool); rte_mempool_free(env.sess_mpool); - rte_mempool_free(env.sess_priv_mpool); rte_mempool_free(env.op_pool); } @@ -1201,13 +1191,9 @@ fips_run_test(void) if (ret < 0) return ret; - env.sess = rte_cryptodev_sym_session_create(env.sess_mpool); - if (!env.sess) - return -ENOMEM; - - ret = rte_cryptodev_sym_session_init(env.dev_id, - env.sess, &xform, env.sess_priv_mpool); - if (ret < 0) { + env.sess = rte_cryptodev_sym_session_create(env.dev_id, &xform, + env.sess_mpool); + if (!env.sess) { RTE_LOG(ERR, USER1, "Error %i: Init session\n", ret); goto exit; @@ -1236,9 +1222,10 @@ fips_run_test(void) vec.status = env.op->status; exit: - rte_cryptodev_sym_session_clear(env.dev_id, env.sess); - rte_cryptodev_sym_session_free(env.sess); - env.sess = NULL; + if (env.sess) { + rte_cryptodev_sym_session_free(env.dev_id, env.sess); + env.sess = NULL; + } return ret; } diff --git a/examples/ipsec-secgw/ipsec-secgw.c b/examples/ipsec-secgw/ipsec-secgw.c index 8a25b83535..ce4b4ba806 100644 --- a/examples/ipsec-secgw/ipsec-secgw.c +++ b/examples/ipsec-secgw/ipsec-secgw.c @@ -1693,8 +1693,6 @@ cryptodevs_init(uint16_t req_queue_num) qp_conf.nb_descriptors = qp_desc_nb; qp_conf.mp_session = socket_ctx[dev_conf.socket_id].session_pool; - qp_conf.mp_session_private = - socket_ctx[dev_conf.socket_id].session_priv_pool; for (qp = 0; qp < dev_conf.nb_queue_pairs; qp++) if (rte_cryptodev_queue_pair_setup(cdev_id, qp, &qp_conf, dev_conf.socket_id)) @@ -2523,12 +2521,8 @@ one_session_free(struct rte_ipsec_session *ips) if (ips->crypto.ses == NULL) return 0; - ret = rte_cryptodev_sym_session_clear(ips->crypto.dev_id, - ips->crypto.ses); - if (ret) - return ret; - - ret = rte_cryptodev_sym_session_free(ips->crypto.ses); + ret = rte_cryptodev_sym_session_free(ips->crypto.dev_id, + ips->crypto.ses); } else { /* Session has not been created */ if (ips->security.ctx == NULL || ips->security.ses == NULL) diff --git a/examples/ipsec-secgw/ipsec.c b/examples/ipsec-secgw/ipsec.c index 7b7bfff696..bb84dcec7e 100644 --- a/examples/ipsec-secgw/ipsec.c +++ b/examples/ipsec-secgw/ipsec.c @@ -174,11 +174,8 @@ create_lookaside_session(struct ipsec_ctx *ipsec_ctx_lcore[], } ips->crypto.dev_id = cdev_id; - ips->crypto.ses = rte_cryptodev_sym_session_create( - skt_ctx->session_pool); - rte_cryptodev_sym_session_init(cdev_id, - ips->crypto.ses, sa->xforms, - skt_ctx->session_priv_pool); + ips->crypto.ses = rte_cryptodev_sym_session_create(cdev_id, + sa->xforms, skt_ctx->session_pool); rte_cryptodev_info_get(cdev_id, &cdev_info); } diff --git a/examples/l2fwd-crypto/main.c b/examples/l2fwd-crypto/main.c index bf4b862379..b555e63ff6 100644 --- a/examples/l2fwd-crypto/main.c +++ b/examples/l2fwd-crypto/main.c @@ -228,7 +228,6 @@ struct rte_mempool *l2fwd_pktmbuf_pool; struct rte_mempool *l2fwd_crypto_op_pool; static struct { struct rte_mempool *sess_mp; - struct rte_mempool *priv_mp; } session_pool_socket[RTE_MAX_NUMA_NODES]; /* Per-port statistics struct */ @@ -675,7 +674,6 @@ static struct rte_cryptodev_sym_session * initialize_crypto_session(struct l2fwd_crypto_options *options, uint8_t cdev_id) { struct rte_crypto_sym_xform *first_xform; - struct rte_cryptodev_sym_session *session; int retval = rte_cryptodev_socket_id(cdev_id); if (retval < 0) @@ -697,17 +695,8 @@ initialize_crypto_session(struct l2fwd_crypto_options *options, uint8_t cdev_id) first_xform = &options->auth_xform; } - session = rte_cryptodev_sym_session_create( + return rte_cryptodev_sym_session_create(cdev_id, first_xform, session_pool_socket[socket_id].sess_mp); - if (session == NULL) - return NULL; - - if (rte_cryptodev_sym_session_init(cdev_id, session, - first_xform, - session_pool_socket[socket_id].priv_mp) < 0) - return NULL; - - return session; } /* >8 End of creation of session. */ @@ -2380,13 +2369,10 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, rte_cryptodev_info_get(cdev_id, &dev_info); - /* - * Two sessions objects are required for each session - * (one for the header, one for the private data) - */ if (!strcmp(dev_info.driver_name, "crypto_scheduler")) { #ifdef RTE_CRYPTO_SCHEDULER - uint32_t nb_workers = + /* scheduler session header + 1 session per worker */ + uint32_t nb_workers = 1 + rte_cryptodev_scheduler_workers_get(cdev_id, NULL); @@ -2395,41 +2381,15 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, } else sessions_needed = enabled_cdev_count; - if (session_pool_socket[socket_id].priv_mp == NULL) { - char mp_name[RTE_MEMPOOL_NAMESIZE]; - - snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, - "priv_sess_mp_%u", socket_id); - - session_pool_socket[socket_id].priv_mp = - rte_mempool_create(mp_name, - sessions_needed, - max_sess_sz, - 0, 0, NULL, NULL, NULL, - NULL, socket_id, - 0); - - if (session_pool_socket[socket_id].priv_mp == NULL) { - printf("Cannot create pool on socket %d\n", - socket_id); - return -ENOMEM; - } - - printf("Allocated pool \"%s\" on socket %d\n", - mp_name, socket_id); - } - if (session_pool_socket[socket_id].sess_mp == NULL) { char mp_name[RTE_MEMPOOL_NAMESIZE]; snprintf(mp_name, RTE_MEMPOOL_NAMESIZE, "sess_mp_%u", socket_id); session_pool_socket[socket_id].sess_mp = - rte_cryptodev_sym_session_pool_create( - mp_name, - sessions_needed, - 0, 0, 0, socket_id); - + rte_cryptodev_sym_session_pool_create( + mp_name, sessions_needed, max_sess_sz, + 0, 0, socket_id); if (session_pool_socket[socket_id].sess_mp == NULL) { printf("Cannot create pool on socket %d\n", socket_id); @@ -2580,8 +2540,6 @@ initialize_cryptodevs(struct l2fwd_crypto_options *options, unsigned nb_ports, qp_conf.nb_descriptors = 2048; qp_conf.mp_session = session_pool_socket[socket_id].sess_mp; - qp_conf.mp_session_private = - session_pool_socket[socket_id].priv_mp; retval = rte_cryptodev_queue_pair_setup(cdev_id, 0, &qp_conf, socket_id); diff --git a/examples/vhost_crypto/main.c b/examples/vhost_crypto/main.c index 7d75623a5e..02987ebd76 100644 --- a/examples/vhost_crypto/main.c +++ b/examples/vhost_crypto/main.c @@ -46,7 +46,6 @@ struct vhost_crypto_info { int vids[MAX_NB_SOCKETS]; uint32_t nb_vids; struct rte_mempool *sess_pool; - struct rte_mempool *sess_priv_pool; struct rte_mempool *cop_pool; uint8_t cid; uint32_t qid; @@ -304,7 +303,6 @@ new_device(int vid) } ret = rte_vhost_crypto_create(vid, info->cid, info->sess_pool, - info->sess_priv_pool, rte_lcore_to_socket_id(options.los[i].lcore_id)); if (ret) { RTE_LOG(ERR, USER1, "Cannot create vhost crypto\n"); @@ -458,7 +456,6 @@ free_resource(void) rte_mempool_free(info->cop_pool); rte_mempool_free(info->sess_pool); - rte_mempool_free(info->sess_priv_pool); for (j = 0; j < lo->nb_sockets; j++) { rte_vhost_driver_unregister(lo->socket_files[i]); @@ -544,16 +541,12 @@ main(int argc, char *argv[]) snprintf(name, 127, "SESS_POOL_%u", lo->lcore_id); info->sess_pool = rte_cryptodev_sym_session_pool_create(name, - SESSION_MAP_ENTRIES, 0, 0, 0, - rte_lcore_to_socket_id(lo->lcore_id)); - - snprintf(name, 127, "SESS_POOL_PRIV_%u", lo->lcore_id); - info->sess_priv_pool = rte_mempool_create(name, SESSION_MAP_ENTRIES, rte_cryptodev_sym_get_private_session_size( - info->cid), 64, 0, NULL, NULL, NULL, NULL, - rte_lcore_to_socket_id(lo->lcore_id), 0); - if (!info->sess_priv_pool || !info->sess_pool) { + info->cid), 0, 0, + rte_lcore_to_socket_id(lo->lcore_id)); + + if (!info->sess_pool) { RTE_LOG(ERR, USER1, "Failed to create mempool"); goto error_exit; } @@ -574,7 +567,6 @@ main(int argc, char *argv[]) qp_conf.nb_descriptors = NB_CRYPTO_DESCRIPTORS; qp_conf.mp_session = info->sess_pool; - qp_conf.mp_session_private = info->sess_priv_pool; for (j = 0; j < dev_info.max_nb_queue_pairs; j++) { ret = rte_cryptodev_queue_pair_setup(info->cid, j, diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index 3dcc3cb7ed..f518a0f89b 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -301,7 +301,6 @@ typedef unsigned int (*cryptodev_asym_get_session_private_size_t)( * @param dev Crypto device pointer * @param xform Single or chain of crypto xforms * @param session Pointer to cryptodev's private session structure - * @param mp Mempool where the private session is allocated * * @return * - Returns 0 if private session structure have been created successfully. @@ -311,8 +310,8 @@ typedef unsigned int (*cryptodev_asym_get_session_private_size_t)( */ typedef int (*cryptodev_sym_configure_session_t)(struct rte_cryptodev *dev, struct rte_crypto_sym_xform *xform, - struct rte_cryptodev_sym_session *session, - struct rte_mempool *mp); + struct rte_cryptodev_sym_session *session); + /** * Configure a Crypto asymmetric session on a device. * @@ -337,6 +336,7 @@ typedef int (*cryptodev_asym_configure_session_t)(struct rte_cryptodev *dev, */ typedef void (*cryptodev_sym_free_session_t)(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess); + /** * Clear asymmetric session private data. * @@ -637,28 +637,6 @@ __rte_internal void * rte_cryptodev_session_event_mdata_get(struct rte_crypto_op *op); -static inline void * -get_sym_session_private_data(const struct rte_cryptodev_sym_session *sess, - uint8_t driver_id) { - if (unlikely(sess->nb_drivers <= driver_id)) - return NULL; - - return sess->sess_data[driver_id].data; -} - -static inline void -set_sym_session_private_data(struct rte_cryptodev_sym_session *sess, - uint8_t driver_id, void *private_data) -{ - if (unlikely(sess->nb_drivers <= driver_id)) { - CDEV_LOG_ERR("Set private data for driver %u not allowed\n", - driver_id); - return; - } - - sess->sess_data[driver_id].data = private_data; -} - /** * @internal * Cryptodev asymmetric crypto session. diff --git a/lib/cryptodev/cryptodev_trace_points.c b/lib/cryptodev/cryptodev_trace_points.c index c5bfe08b79..019fc0c800 100644 --- a/lib/cryptodev/cryptodev_trace_points.c +++ b/lib/cryptodev/cryptodev_trace_points.c @@ -39,12 +39,6 @@ RTE_TRACE_POINT_REGISTER(rte_cryptodev_trace_sym_session_free, RTE_TRACE_POINT_REGISTER(rte_cryptodev_trace_asym_session_free, lib.cryptodev.asym.free) -RTE_TRACE_POINT_REGISTER(rte_cryptodev_trace_sym_session_init, - lib.cryptodev.sym.init) - -RTE_TRACE_POINT_REGISTER(rte_cryptodev_trace_sym_session_clear, - lib.cryptodev.sym.clear) - RTE_TRACE_POINT_REGISTER(rte_cryptodev_trace_enqueue_burst, lib.cryptodev.enq.burst) diff --git a/lib/cryptodev/rte_cryptodev.c b/lib/cryptodev/rte_cryptodev.c index 42f3221052..63b8255a45 100644 --- a/lib/cryptodev/rte_cryptodev.c +++ b/lib/cryptodev/rte_cryptodev.c @@ -190,12 +190,9 @@ const char *rte_crypto_asym_ke_strings[] = { [RTE_CRYPTO_ASYM_KE_PUB_KEY_VERIFY] = "pub_ec_key_verify" }; -/** - * The private data structure stored in the sym session mempool private data. - */ struct rte_cryptodev_sym_session_pool_private_data { - uint16_t nb_drivers; - /**< number of elements in sess_data array */ + uint16_t sess_data_sz; + /**< driver session data size */ uint16_t user_data_sz; /**< session user data will be placed after sess_data */ }; @@ -1202,6 +1199,24 @@ rte_cryptodev_get_qp_status(uint8_t dev_id, uint16_t queue_pair_id) return 0; } +static uint8_t +rte_cryptodev_sym_is_valid_session_pool(struct rte_mempool *mp, + uint32_t sess_priv_size) +{ + struct rte_cryptodev_sym_session_pool_private_data *pool_priv; + + if (!mp) + return 0; + + pool_priv = rte_mempool_get_priv(mp); + + if (!pool_priv || mp->private_data_size < sizeof(*pool_priv) || + pool_priv->sess_data_sz < sess_priv_size) + return 0; + + return 1; +} + int rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, const struct rte_cryptodev_qp_conf *qp_conf, int socket_id) @@ -1225,17 +1240,8 @@ rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, return -EINVAL; } - if ((qp_conf->mp_session && !qp_conf->mp_session_private) || - (!qp_conf->mp_session && qp_conf->mp_session_private)) { - CDEV_LOG_ERR("Invalid mempools\n"); - return -EINVAL; - } - if (qp_conf->mp_session) { struct rte_cryptodev_sym_session_pool_private_data *pool_priv; - uint32_t obj_size = qp_conf->mp_session->elt_size; - uint32_t obj_priv_size = qp_conf->mp_session_private->elt_size; - struct rte_cryptodev_sym_session s = {0}; pool_priv = rte_mempool_get_priv(qp_conf->mp_session); if (!pool_priv || qp_conf->mp_session->private_data_size < @@ -1244,13 +1250,9 @@ rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, return -EINVAL; } - s.nb_drivers = pool_priv->nb_drivers; - s.user_data_sz = pool_priv->user_data_sz; - - if ((rte_cryptodev_sym_get_existing_header_session_size(&s) > - obj_size) || (s.nb_drivers <= dev->driver_id) || - rte_cryptodev_sym_get_private_session_size(dev_id) > - obj_priv_size) { + if (!rte_cryptodev_sym_is_valid_session_pool( + qp_conf->mp_session, + rte_cryptodev_sym_get_private_session_size(dev_id))) { CDEV_LOG_ERR("Invalid mempool\n"); return -EINVAL; } @@ -1710,53 +1712,6 @@ rte_cryptodev_pmd_callback_process(struct rte_cryptodev *dev, rte_spinlock_unlock(&rte_cryptodev_cb_lock); } -int -rte_cryptodev_sym_session_init(uint8_t dev_id, - struct rte_cryptodev_sym_session *sess, - struct rte_crypto_sym_xform *xforms, - struct rte_mempool *mp) -{ - struct rte_cryptodev *dev; - uint32_t sess_priv_sz = rte_cryptodev_sym_get_private_session_size( - dev_id); - uint8_t index; - int ret; - - if (!rte_cryptodev_is_valid_dev(dev_id)) { - CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id); - return -EINVAL; - } - - dev = rte_cryptodev_pmd_get_dev(dev_id); - - if (sess == NULL || xforms == NULL || dev == NULL || mp == NULL) - return -EINVAL; - - if (mp->elt_size < sess_priv_sz) - return -EINVAL; - - index = dev->driver_id; - if (index >= sess->nb_drivers) - return -EINVAL; - - RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->sym_session_configure, -ENOTSUP); - - if (sess->sess_data[index].refcnt == 0) { - ret = dev->dev_ops->sym_session_configure(dev, xforms, - sess, mp); - if (ret < 0) { - CDEV_LOG_ERR( - "dev_id %d failed to configure session details", - dev_id); - return ret; - } - } - - rte_cryptodev_trace_sym_session_init(dev_id, sess, xforms, mp); - sess->sess_data[index].refcnt++; - return 0; -} - struct rte_mempool * rte_cryptodev_sym_session_pool_create(const char *name, uint32_t nb_elts, uint32_t elt_size, uint32_t cache_size, uint16_t user_data_size, @@ -1766,16 +1721,13 @@ rte_cryptodev_sym_session_pool_create(const char *name, uint32_t nb_elts, struct rte_cryptodev_sym_session_pool_private_data *pool_priv; uint32_t obj_sz; - obj_sz = rte_cryptodev_sym_get_header_session_size() + user_data_size; - if (obj_sz > elt_size) - CDEV_LOG_INFO("elt_size %u is expanded to %u\n", elt_size, - obj_sz); - else - obj_sz = elt_size; + obj_sz = sizeof(struct rte_cryptodev_sym_session) + elt_size + + user_data_size; + obj_sz = RTE_ALIGN_CEIL(obj_sz, RTE_CACHE_LINE_SIZE); mp = rte_mempool_create(name, nb_elts, obj_sz, cache_size, - (uint32_t)(sizeof(*pool_priv)), - NULL, NULL, NULL, NULL, + (uint32_t)(sizeof(*pool_priv)), NULL, NULL, + NULL, NULL, socket_id, 0); if (mp == NULL) { CDEV_LOG_ERR("%s(name=%s) failed, rte_errno=%d\n", @@ -1791,7 +1743,7 @@ rte_cryptodev_sym_session_pool_create(const char *name, uint32_t nb_elts, return NULL; } - pool_priv->nb_drivers = nb_drivers; + pool_priv->sess_data_sz = elt_size; pool_priv->user_data_sz = user_data_size; rte_cryptodev_trace_sym_session_pool_create(name, nb_elts, @@ -1849,64 +1801,68 @@ rte_cryptodev_asym_session_pool_create(const char *name, uint32_t nb_elts, return mp; } -static unsigned int -rte_cryptodev_sym_session_data_size(struct rte_cryptodev_sym_session *sess) -{ - return (sizeof(sess->sess_data[0]) * sess->nb_drivers) + - sess->user_data_sz; -} - -static uint8_t -rte_cryptodev_sym_is_valid_session_pool(struct rte_mempool *mp) +void * +rte_cryptodev_sym_session_create(uint8_t dev_id, + struct rte_crypto_sym_xform *xforms, + struct rte_mempool *mp) { + struct rte_cryptodev *dev; + struct rte_cryptodev_sym_session *sess; struct rte_cryptodev_sym_session_pool_private_data *pool_priv; - if (!mp) - return 0; - - pool_priv = rte_mempool_get_priv(mp); - - if (!pool_priv || mp->private_data_size < sizeof(*pool_priv) || - pool_priv->nb_drivers != nb_drivers || - mp->elt_size < - rte_cryptodev_sym_get_header_session_size() - + pool_priv->user_data_sz) - return 0; + uint32_t sess_priv_sz; + int ret; - return 1; -} + if (!rte_cryptodev_is_valid_dev(dev_id)) { + CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id); + return NULL; + } -struct rte_cryptodev_sym_session * -rte_cryptodev_sym_session_create(struct rte_mempool *mp) -{ - struct rte_cryptodev_sym_session *sess; - struct rte_cryptodev_sym_session_pool_private_data *pool_priv; + if (xforms == NULL) { + CDEV_LOG_ERR("Invalid xform\n"); + return NULL; + } - if (!rte_cryptodev_sym_is_valid_session_pool(mp)) { + sess_priv_sz = rte_cryptodev_sym_get_private_session_size(dev_id); + if (!rte_cryptodev_sym_is_valid_session_pool(mp, sess_priv_sz)) { CDEV_LOG_ERR("Invalid mempool\n"); return NULL; } - pool_priv = rte_mempool_get_priv(mp); + dev = rte_cryptodev_pmd_get_dev(dev_id); /* Allocate a session structure from the session pool */ if (rte_mempool_get(mp, (void **)&sess)) { CDEV_LOG_ERR("couldn't get object from session mempool"); + rte_errno = ENOMEM; return NULL; } - sess->nb_drivers = pool_priv->nb_drivers; + pool_priv = rte_mempool_get_priv(mp); + sess->driver_id = dev->driver_id; + sess->sess_data_sz = pool_priv->sess_data_sz; sess->user_data_sz = pool_priv->user_data_sz; - sess->opaque_data = 0; + sess->driver_priv_data_iova = rte_mempool_virt2iova(sess) + + offsetof(struct rte_cryptodev_sym_session, driver_priv_data); - /* Clear device session pointer. - * Include the flag indicating presence of user data - */ - memset(sess->sess_data, 0, - rte_cryptodev_sym_session_data_size(sess)); + if (dev->dev_ops->sym_session_configure == NULL) { + rte_errno = ENOTSUP; + goto error_exit; + } - rte_cryptodev_trace_sym_session_create(mp, sess); - return sess; + ret = dev->dev_ops->sym_session_configure(dev, xforms, sess); + if (ret < 0) { + rte_errno = -ret; + goto error_exit; + } + sess->driver_id = dev->driver_id; + + rte_cryptodev_trace_sym_session_create(dev_id, sess, xforms, mp); + + return (void *)sess; +error_exit: + rte_mempool_put(mp, (void *)sess); + return NULL; } int @@ -1985,11 +1941,15 @@ rte_cryptodev_asym_session_create(uint8_t dev_id, } int -rte_cryptodev_sym_session_clear(uint8_t dev_id, - struct rte_cryptodev_sym_session *sess) +rte_cryptodev_sym_session_free(uint8_t dev_id, + struct rte_cryptodev_sym_session *sess) { struct rte_cryptodev *dev; - uint8_t driver_id; + struct rte_mempool *sess_mp; + struct rte_cryptodev_sym_session_pool_private_data *pool_priv; + + if (sess == NULL) + return -EINVAL; if (!rte_cryptodev_is_valid_dev(dev_id)) { CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id); @@ -2001,37 +1961,24 @@ rte_cryptodev_sym_session_clear(uint8_t dev_id, if (dev == NULL || sess == NULL) return -EINVAL; - driver_id = dev->driver_id; - if (sess->sess_data[driver_id].refcnt == 0) - return 0; - if (--sess->sess_data[driver_id].refcnt != 0) - return -EBUSY; + sess_mp = rte_mempool_from_obj(sess); + if (!sess_mp) + return -EINVAL; + pool_priv = rte_mempool_get_priv(sess_mp); + + if (sess->driver_id != dev->driver_id) { + CDEV_LOG_ERR("Session created by driver %u but freed by %u", + sess->driver_id, dev->driver_id); + return -EINVAL; + } RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->sym_session_clear, -ENOTSUP); dev->dev_ops->sym_session_clear(dev, sess); - - rte_cryptodev_trace_sym_session_clear(dev_id, sess); - return 0; -} - -int -rte_cryptodev_sym_session_free(struct rte_cryptodev_sym_session *sess) -{ - uint8_t i; - struct rte_mempool *sess_mp; - - if (sess == NULL) - return -EINVAL; - - /* Check that all device private data has been freed */ - for (i = 0; i < sess->nb_drivers; i++) { - if (sess->sess_data[i].refcnt != 0) - return -EBUSY; - } + memset(sess->driver_priv_data, 0, pool_priv->sess_data_sz + + pool_priv->user_data_sz); /* Return session to mempool */ - sess_mp = rte_mempool_from_obj(sess); rte_mempool_put(sess_mp, sess); rte_cryptodev_trace_sym_session_free(sess); @@ -2068,33 +2015,6 @@ rte_cryptodev_asym_session_free(uint8_t dev_id, void *sess) return 0; } -unsigned int -rte_cryptodev_sym_get_header_session_size(void) -{ - /* - * Header contains pointers to the private data of all registered - * drivers and all necessary information to ensure safely clear - * or free al session. - */ - struct rte_cryptodev_sym_session s = {0}; - - s.nb_drivers = nb_drivers; - - return (unsigned int)(sizeof(s) + - rte_cryptodev_sym_session_data_size(&s)); -} - -unsigned int -rte_cryptodev_sym_get_existing_header_session_size( - struct rte_cryptodev_sym_session *sess) -{ - if (!sess) - return 0; - else - return (unsigned int)(sizeof(*sess) + - rte_cryptodev_sym_session_data_size(sess)); -} - unsigned int rte_cryptodev_asym_get_header_session_size(void) { @@ -2141,9 +2061,8 @@ rte_cryptodev_asym_get_private_session_size(uint8_t dev_id) int rte_cryptodev_sym_session_set_user_data( - struct rte_cryptodev_sym_session *sess, - void *data, - uint16_t size) + struct rte_cryptodev_sym_session *sess, void *data, + uint16_t size) { if (sess == NULL) return -EINVAL; @@ -2151,18 +2070,17 @@ rte_cryptodev_sym_session_set_user_data( if (sess->user_data_sz < size) return -ENOMEM; - rte_memcpy(sess->sess_data + sess->nb_drivers, data, size); + rte_memcpy(sess->driver_priv_data + sess->sess_data_sz, data, size); return 0; } void * -rte_cryptodev_sym_session_get_user_data( - struct rte_cryptodev_sym_session *sess) +rte_cryptodev_sym_session_get_user_data(struct rte_cryptodev_sym_session *sess) { if (sess == NULL || sess->user_data_sz == 0) return NULL; - return (void *)(sess->sess_data + sess->nb_drivers); + return (void *)(sess->driver_priv_data + sess->sess_data_sz); } int diff --git a/lib/cryptodev/rte_cryptodev.h b/lib/cryptodev/rte_cryptodev.h index 56f459c6a0..0c65958f25 100644 --- a/lib/cryptodev/rte_cryptodev.h +++ b/lib/cryptodev/rte_cryptodev.h @@ -544,8 +544,6 @@ struct rte_cryptodev_qp_conf { uint32_t nb_descriptors; /**< Number of descriptors per queue pair */ struct rte_mempool *mp_session; /**< The mempool for creating session in sessionless mode */ - struct rte_mempool *mp_session_private; - /**< The mempool for creating sess private data in sessionless mode */ }; /** @@ -909,17 +907,21 @@ rte_cryptodev_get_sec_ctx(uint8_t dev_id); * has a fixed algo, key, op-type, digest_len etc. */ struct rte_cryptodev_sym_session { + RTE_MARKER cacheline0; + uint8_t driver_id; uint64_t opaque_data; /**< Can be used for external metadata */ - uint16_t nb_drivers; - /**< number of elements in sess_data array */ + uint32_t sess_data_sz; + /**< Pointer to the user data stored after sess data */ uint16_t user_data_sz; - /**< session user data will be placed after sess_data */ - __extension__ struct { - void *data; - uint16_t refcnt; - } sess_data[]; - /**< Driver specific session material, variable size */ + /**< session user data will be placed after sess data */ + rte_iova_t driver_priv_data_iova; + /**< session driver data IOVA address */ + + RTE_MARKER cacheline1 __rte_cache_min_aligned; + /**< second cache line - start of the driver session data */ + uint8_t driver_priv_data[0]; + /**< Driver specific session data, variable size */ }; /** @@ -954,6 +956,7 @@ rte_cryptodev_sym_session_pool_create(const char *name, uint32_t nb_elts, uint32_t elt_size, uint32_t cache_size, uint16_t priv_size, int socket_id); + /** * Create an asymmetric session mempool. * @@ -980,17 +983,22 @@ rte_cryptodev_asym_session_pool_create(const char *name, uint32_t nb_elts, uint32_t cache_size, uint16_t user_data_size, int socket_id); /** - * Create symmetric crypto session header (generic with no private data) + * Create symmetric crypto session and fill out private data for the device id, + * based on its device type. + * + * @param dev_id ID of device that we want the session to be used on + * @param xforms Symmetric crypto transform operations to apply on flow + * processed with this session + * @param mempool Mempool where the private data is allocated. * - * @param mempool Symmetric session mempool to allocate session - * objects from * @return - * - On success return pointer to sym-session - * - On failure returns NULL + * - On success return pointer to sym-session. + * - On failure returns NULL. */ -struct rte_cryptodev_sym_session * -rte_cryptodev_sym_session_create(struct rte_mempool *mempool); - +void * +rte_cryptodev_sym_session_create(uint8_t dev_id, + struct rte_crypto_sym_xform *xforms, + struct rte_mempool *mp); /** * Create and initialise an asymmetric crypto session structure. * Calls the PMD to configure the private session data. @@ -1015,19 +1023,20 @@ rte_cryptodev_asym_session_create(uint8_t dev_id, void **session); /** - * Frees symmetric crypto session header, after checking that all - * the device private data has been freed, returning it - * to its original mempool. + * Frees session for the device id and returning it to its mempool. + * It is the application's responsibility to ensure that the session + * is not still in-flight operations using it. * + * @param dev_id ID of device that uses the session. * @param sess Session header to be freed. * * @return * - 0 if successful. - * - -EINVAL if session is NULL. - * - -EBUSY if not all device private data has been freed. + * - -EINVAL if session is NULL or the mismatched device ids. */ int -rte_cryptodev_sym_session_free(struct rte_cryptodev_sym_session *sess); +rte_cryptodev_sym_session_free(uint8_t dev_id, + struct rte_cryptodev_sym_session *sess); /** * Clears and frees asymmetric crypto session header and private data, @@ -1044,72 +1053,6 @@ __rte_experimental int rte_cryptodev_asym_session_free(uint8_t dev_id, void *sess); -/** - * Fill out private data for the device id, based on its device type. - * - * @param dev_id ID of device that we want the session to be used on - * @param sess Session where the private data will be attached to - * @param xforms Symmetric crypto transform operations to apply on flow - * processed with this session - * @param mempool Mempool where the private data is allocated. - * - * @return - * - On success, zero. - * - -EINVAL if input parameters are invalid. - * - -ENOTSUP if crypto device does not support the crypto transform or - * does not support symmetric operations. - * - -ENOMEM if the private session could not be allocated. - */ -int -rte_cryptodev_sym_session_init(uint8_t dev_id, - struct rte_cryptodev_sym_session *sess, - struct rte_crypto_sym_xform *xforms, - struct rte_mempool *mempool); - -/** - * Frees private data for the device id, based on its device type, - * returning it to its mempool. It is the application's responsibility - * to ensure that private session data is not cleared while there are - * still in-flight operations using it. - * - * @param dev_id ID of device that uses the session. - * @param sess Session containing the reference to the private data - * - * @return - * - 0 if successful. - * - -EINVAL if device is invalid or session is NULL. - * - -ENOTSUP if crypto device does not support symmetric operations. - */ -int -rte_cryptodev_sym_session_clear(uint8_t dev_id, - struct rte_cryptodev_sym_session *sess); - -/** - * Get the size of the header session, for all registered drivers excluding - * the user data size. - * - * @return - * Size of the symmetric header session. - */ -unsigned int -rte_cryptodev_sym_get_header_session_size(void); - -/** - * Get the size of the header session from created session. - * - * @param sess - * The sym cryptodev session pointer - * - * @return - * - If sess is not NULL, return the size of the header session including - * the private data size defined within sess. - * - If sess is NULL, return 0. - */ -__rte_experimental -unsigned int -rte_cryptodev_sym_get_existing_header_session_size( - struct rte_cryptodev_sym_session *sess); - /** * Get the size of the asymmetric session header. * diff --git a/lib/cryptodev/rte_cryptodev_trace.h b/lib/cryptodev/rte_cryptodev_trace.h index a3f6048e7d..055c44fb22 100644 --- a/lib/cryptodev/rte_cryptodev_trace.h +++ b/lib/cryptodev/rte_cryptodev_trace.h @@ -56,7 +56,6 @@ RTE_TRACE_POINT( rte_trace_point_emit_u16(queue_pair_id); rte_trace_point_emit_u32(conf->nb_descriptors); rte_trace_point_emit_ptr(conf->mp_session); - rte_trace_point_emit_ptr(conf->mp_session_private); ) RTE_TRACE_POINT( @@ -74,13 +73,16 @@ RTE_TRACE_POINT( RTE_TRACE_POINT( rte_cryptodev_trace_sym_session_create, - RTE_TRACE_POINT_ARGS(void *mempool, - struct rte_cryptodev_sym_session *sess), - rte_trace_point_emit_ptr(mempool); + RTE_TRACE_POINT_ARGS(uint8_t dev_id, + struct rte_cryptodev_sym_session *sess, void *xforms, + void *mempool), + rte_trace_point_emit_u8(dev_id); rte_trace_point_emit_ptr(sess); rte_trace_point_emit_u64(sess->opaque_data); - rte_trace_point_emit_u16(sess->nb_drivers); + rte_trace_point_emit_u8(sess->driver_id); rte_trace_point_emit_u16(sess->user_data_sz); + rte_trace_point_emit_ptr(xforms); + rte_trace_point_emit_ptr(mempool); ) RTE_TRACE_POINT( @@ -117,27 +119,6 @@ RTE_TRACE_POINT( rte_trace_point_emit_ptr(sess); ) -RTE_TRACE_POINT( - rte_cryptodev_trace_sym_session_init, - RTE_TRACE_POINT_ARGS(uint8_t dev_id, - struct rte_cryptodev_sym_session *sess, void *xforms, - void *mempool), - rte_trace_point_emit_u8(dev_id); - rte_trace_point_emit_ptr(sess); - rte_trace_point_emit_u64(sess->opaque_data); - rte_trace_point_emit_u16(sess->nb_drivers); - rte_trace_point_emit_u16(sess->user_data_sz); - rte_trace_point_emit_ptr(xforms); - rte_trace_point_emit_ptr(mempool); -) - -RTE_TRACE_POINT( - rte_cryptodev_trace_sym_session_clear, - RTE_TRACE_POINT_ARGS(uint8_t dev_id, void *sess), - rte_trace_point_emit_u8(dev_id); - rte_trace_point_emit_ptr(sess); -) - #ifdef __cplusplus } #endif diff --git a/lib/cryptodev/version.map b/lib/cryptodev/version.map index 5aee87c6f7..6fcf6ae985 100644 --- a/lib/cryptodev/version.map +++ b/lib/cryptodev/version.map @@ -38,12 +38,9 @@ DPDK_23 { rte_cryptodev_sym_capability_check_auth; rte_cryptodev_sym_capability_check_cipher; rte_cryptodev_sym_capability_get; - rte_cryptodev_sym_get_header_session_size; rte_cryptodev_sym_get_private_session_size; - rte_cryptodev_sym_session_clear; rte_cryptodev_sym_session_create; rte_cryptodev_sym_session_free; - rte_cryptodev_sym_session_init; local: *; }; @@ -60,7 +57,6 @@ EXPERIMENTAL { rte_cryptodev_asym_xform_capability_check_modlen; rte_cryptodev_asym_xform_capability_check_optype; rte_cryptodev_sym_cpu_crypto_process; - rte_cryptodev_sym_get_existing_header_session_size; rte_cryptodev_sym_session_get_user_data; rte_cryptodev_sym_session_pool_create; rte_cryptodev_sym_session_set_user_data; @@ -78,8 +74,6 @@ EXPERIMENTAL { __rte_cryptodev_trace_asym_session_create; __rte_cryptodev_trace_sym_session_free; __rte_cryptodev_trace_asym_session_free; - __rte_cryptodev_trace_sym_session_init; - __rte_cryptodev_trace_sym_session_clear; __rte_cryptodev_trace_dequeue_burst; __rte_cryptodev_trace_enqueue_burst; diff --git a/lib/pipeline/rte_table_action.c b/lib/pipeline/rte_table_action.c index b1310be565..cb792bbe0d 100644 --- a/lib/pipeline/rte_table_action.c +++ b/lib/pipeline/rte_table_action.c @@ -1898,17 +1898,11 @@ sym_crypto_apply(struct sym_crypto_data *data, } } - session = rte_cryptodev_sym_session_create(cfg->mp_create); + session = rte_cryptodev_sym_session_create(cfg->cryptodev_id, + p->xform, cfg->mp_create); if (!session) return -ENOMEM; - ret = rte_cryptodev_sym_session_init(cfg->cryptodev_id, session, - p->xform, cfg->mp_init); - if (ret < 0) { - rte_cryptodev_sym_session_free(session); - return ret; - } - data->data_offset = (uint16_t)p->data_offset; data->session = session; diff --git a/lib/vhost/rte_vhost_crypto.h b/lib/vhost/rte_vhost_crypto.h index b49e389579..2b01ecda08 100644 --- a/lib/vhost/rte_vhost_crypto.h +++ b/lib/vhost/rte_vhost_crypto.h @@ -54,8 +54,6 @@ rte_vhost_crypto_driver_start(const char *path); * multiple Vhost-crypto devices. * @param sess_pool * The pointer to the created cryptodev session pool. - * @param sess_priv_pool - * The pointer to the created cryptodev session private data mempool. * @param socket_id * NUMA Socket ID to allocate resources on. * * @return @@ -65,7 +63,6 @@ rte_vhost_crypto_driver_start(const char *path); int rte_vhost_crypto_create(int vid, uint8_t cryptodev_id, struct rte_mempool *sess_pool, - struct rte_mempool *sess_priv_pool, int socket_id); /** diff --git a/lib/vhost/vhost_crypto.c b/lib/vhost/vhost_crypto.c index 54946f46d9..7321da21b7 100644 --- a/lib/vhost/vhost_crypto.c +++ b/lib/vhost/vhost_crypto.c @@ -197,7 +197,6 @@ struct vhost_crypto { struct rte_hash *session_map; struct rte_mempool *mbuf_pool; struct rte_mempool *sess_pool; - struct rte_mempool *sess_priv_pool; struct rte_mempool *wb_pool; /** DPDK cryptodev ID */ @@ -376,31 +375,21 @@ vhost_crypto_create_sess(struct vhost_crypto *vcrypto, return; } - session = rte_cryptodev_sym_session_create(vcrypto->sess_pool); + session = rte_cryptodev_sym_session_create(vcrypto->cid, &xform1, + vcrypto->sess_pool); if (!session) { VC_LOG_ERR("Failed to create session"); sess_param->session_id = -VIRTIO_CRYPTO_ERR; return; } - if (rte_cryptodev_sym_session_init(vcrypto->cid, session, &xform1, - vcrypto->sess_priv_pool) < 0) { - VC_LOG_ERR("Failed to initialize session"); - sess_param->session_id = -VIRTIO_CRYPTO_ERR; - return; - } - /* insert hash to map */ if (rte_hash_add_key_data(vcrypto->session_map, &vcrypto->last_session_id, session) < 0) { VC_LOG_ERR("Failed to insert session to hash table"); - if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) - VC_LOG_ERR("Failed to clear session"); - else { - if (rte_cryptodev_sym_session_free(session) < 0) - VC_LOG_ERR("Failed to free session"); - } + if (rte_cryptodev_sym_session_free(vcrypto->cid, session) < 0) + VC_LOG_ERR("Failed to free session"); sess_param->session_id = -VIRTIO_CRYPTO_ERR; return; } @@ -427,12 +416,7 @@ vhost_crypto_close_sess(struct vhost_crypto *vcrypto, uint64_t session_id) return -VIRTIO_CRYPTO_INVSESS; } - if (rte_cryptodev_sym_session_clear(vcrypto->cid, session) < 0) { - VC_LOG_DBG("Failed to clear session"); - return -VIRTIO_CRYPTO_ERR; - } - - if (rte_cryptodev_sym_session_free(session) < 0) { + if (rte_cryptodev_sym_session_free(vcrypto->cid, session) < 0) { VC_LOG_DBG("Failed to free session"); return -VIRTIO_CRYPTO_ERR; } @@ -1393,7 +1377,6 @@ rte_vhost_crypto_driver_start(const char *path) int rte_vhost_crypto_create(int vid, uint8_t cryptodev_id, struct rte_mempool *sess_pool, - struct rte_mempool *sess_priv_pool, int socket_id) { struct virtio_net *dev = get_device(vid); @@ -1415,7 +1398,6 @@ rte_vhost_crypto_create(int vid, uint8_t cryptodev_id, } vcrypto->sess_pool = sess_pool; - vcrypto->sess_priv_pool = sess_priv_pool; vcrypto->cid = cryptodev_id; vcrypto->cache_session_id = UINT64_MAX; vcrypto->last_session_id = 1; -- 2.34.1