From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5B2AEA00C2; Thu, 6 Oct 2022 10:16:55 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0B33942B88; Thu, 6 Oct 2022 10:16:55 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id 7139741153 for ; Thu, 6 Oct 2022 10:16:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665044213; x=1696580213; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=cUjusCB7LihFgBPyS6LurclPlm+9uic3BlHNuz2pYU0=; b=d1f5Yq3yYZ/nLNuvGG4Z1XbGcRBMCweHwGEsuSyTnz88tiRFtmshvs44 7h9QutqX6GR9slcs6E4jaZ04V2zHDNFLf3DZe0sN08Ent8WvXTuYSjfI4 5U+hKjKP9eyymUizjKIPtb9UT/wE/a3VtpAg9ReeIU4EhDQNj8OVyrCFK H8TbPaCTkoCoKYkutD2mkfZZSElcOm2RgXh57LDZpX/CkXJ6ikGP2oBlx aRG8iEuCN83Af6ao+p8EoQY0EL4CVrXWlaoSnQvr+6MvEwFRJOwCkZtaD MUTbY8kx0RSUbyfXJfz14Y6Edmh6ci4XXYIB44qv84q7Pan0NJyadotIj A==; X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="290639915" X-IronPort-AV: E=Sophos;i="5.95,163,1661842800"; d="scan'208";a="290639915" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Oct 2022 01:16:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="767060444" X-IronPort-AV: E=Sophos;i="5.95,163,1661842800"; d="scan'208";a="767060444" Received: from silpixa00400465.ir.intel.com ([10.55.128.22]) by fmsmga001.fm.intel.com with ESMTP; 06 Oct 2022 01:16:48 -0700 From: Kai Ji To: dev@dpdk.org Cc: Kai Ji , Akhil Goyal , Fan Zhang , Ray Kinsella , Anatoly Burakov Subject: [dpdk-dev v4] lib/cryptodev: multi-process IPC request handler Date: Thu, 6 Oct 2022 16:16:46 +0800 Message-Id: <20221006081646.81901-1-kai.ji@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221002224436.39571-1-kai.ji@intel.com> References: <20221002224436.39571-1-kai.ji@intel.com> X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org As some cryptode PMDs have multiprocess support, the secondary process needs queue-pair to be configured by the primary process before to use. This patch adds an IPC register function to help the primary process to register IPC action that allow secondary process to configure cryptodev queue-pair via IPC messages during the runtime. After setup, a new "qp_in_used_pid" param stores the PID to provide the ownership of the queue-pair so that only the PID matched queue-pair free request is allowed in the future. Signed-off-by: Kai Ji --- v4: - release note update - Doxygen comments update v3: - addin missing free function for qp_in_use_by_pid v2: - code rework --- doc/guides/rel_notes/release_22_11.rst | 8 ++- lib/cryptodev/cryptodev_pmd.h | 3 +- lib/cryptodev/rte_cryptodev.c | 91 ++++++++++++++++++++++++++ lib/cryptodev/rte_cryptodev.h | 46 +++++++++++++ lib/cryptodev/version.map | 2 + 5 files changed, 148 insertions(+), 2 deletions(-) diff --git a/doc/guides/rel_notes/release_22_11.rst b/doc/guides/rel_notes/release_22_11.rst index 4e64710a69..fdcd479916 100644 --- a/doc/guides/rel_notes/release_22_11.rst +++ b/doc/guides/rel_notes/release_22_11.rst @@ -87,6 +87,13 @@ New Features Added MACsec transform for rte_security session and added new API to configure security associations (SA) and secure channels (SC). +* **Added MP IPC request register function in cryptodev ** + + Added new functions ``rte_cryptodev_mp_request_register`` and + ``rte_cryptodev_mp_request_unregister`` + The function helps the primary process to register IPC action that allow + secondary process to request cryptodev queue pairs setups via IPC messages. + * **Added new algorithms to cryptodev.** * Added symmetric hash algorithm ShangMi 3 (SM3). @@ -123,7 +130,6 @@ New Features into single event containing ``rte_event_vector`` whose event type is ``RTE_EVENT_TYPE_CRYPTODEV_VECTOR``. - Removed Items ------------- diff --git a/lib/cryptodev/cryptodev_pmd.h b/lib/cryptodev/cryptodev_pmd.h index f27b3249ea..574aebe279 100644 --- a/lib/cryptodev/cryptodev_pmd.h +++ b/lib/cryptodev/cryptodev_pmd.h @@ -78,7 +78,8 @@ struct rte_cryptodev_data { void **queue_pairs; /** Number of device queue pairs. */ uint16_t nb_queue_pairs; - + /** Array of process id used for queue pairs **/ + uint16_t *qp_in_use_by_pid; /** PMD-specific private data */ void *dev_private; } __rte_cache_aligned; diff --git a/lib/cryptodev/rte_cryptodev.c b/lib/cryptodev/rte_cryptodev.c index 2165a0688c..1313b9071b 100644 --- a/lib/cryptodev/rte_cryptodev.c +++ b/lib/cryptodev/rte_cryptodev.c @@ -49,6 +49,9 @@ struct rte_crypto_fp_ops rte_crypto_fp_ops[RTE_CRYPTO_MAX_DEVS]; /* spinlock for crypto device callbacks */ static rte_spinlock_t rte_cryptodev_cb_lock = RTE_SPINLOCK_INITIALIZER; +/* crypto queue pair config */ +#define CRYPTODEV_MP_REQ "cryptodev_mp_request" + /** * The user application callback description. * @@ -1047,6 +1050,9 @@ rte_cryptodev_pmd_release_device(struct rte_cryptodev *cryptodev) return ret; } + if (cryptodev->data->qp_in_use_by_pid) + rte_free(cryptodev->data->qp_in_use_by_pid); + ret = rte_cryptodev_data_free(dev_id, &cryptodev_globals.data[dev_id]); if (ret < 0) return ret; @@ -1135,6 +1141,21 @@ rte_cryptodev_queue_pairs_config(struct rte_cryptodev *dev, uint16_t nb_qpairs, } dev->data->nb_queue_pairs = nb_qpairs; + + if (dev->data->qp_in_use_by_pid == NULL) { + dev->data->qp_in_use_by_pid = rte_zmalloc_socket( + "cryptodev->qp_in_use_by_pid", + sizeof(dev->data->qp_in_use_by_pid[0]) * + dev_info.max_nb_queue_pairs, + RTE_CACHE_LINE_SIZE, socket_id); + if (dev->data->qp_in_use_by_pid == NULL) { + CDEV_LOG_ERR("failed to get memory for qp meta data, " + "nb_queues %u", + nb_qpairs); + return -(ENOMEM); + } + } + return 0; } @@ -1401,6 +1422,76 @@ rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, socket_id); } +static int +rte_cryptodev_ipc_request(const struct rte_mp_msg *mp_msg, const void *peer) +{ + struct rte_mp_msg mp_res; + struct rte_cryptodev_mp_param *resp_param = + (struct rte_cryptodev_mp_param *)mp_res.param; + const struct rte_cryptodev_mp_param *req_param = + (const struct rte_cryptodev_mp_param *)mp_msg->param; + + int ret; + struct rte_cryptodev *dev; + uint16_t *qp_in_used_by_pid; + int dev_id = req_param->dev_id; + int qp_id = req_param->qp_id; + struct rte_cryptodev_qp_conf *queue_conf = req_param->queue_conf; + + resp_param->result = -EINVAL; + if (!rte_cryptodev_is_valid_dev(dev_id)) { + CDEV_LOG_ERR("Invalid dev_id=%" PRIu8, dev_id); + goto out; + } + + if (!rte_cryptodev_get_qp_status(dev_id, qp_id)) + goto out; + + dev = &rte_crypto_devices[dev_id]; + qp_in_used_by_pid = dev->data->qp_in_use_by_pid; + + switch (req_param->type) { + case RTE_CRYPTODEV_MP_REQ_QP_SET: + ret = rte_cryptodev_queue_pair_setup(dev_id, qp_id, + queue_conf, req_param->socket_id); + if (!ret) + qp_in_used_by_pid[qp_id] = req_param->process_id; + resp_param->result = ret; + break; + case RTE_CRYPTODEV_MP_REQ_QP_FREE: + if (qp_in_used_by_pid[qp_id] != req_param->process_id) { + CDEV_LOG_ERR("Unable to release qp_id=%" PRIu8, qp_id); + goto out; + } + + ret = (*dev->dev_ops->queue_pair_release)(dev, qp_id); + if (!ret) + qp_in_used_by_pid[qp_id] = 0; + + resp_param->result = ret; + break; + default: + CDEV_LOG_ERR("invalid mp request type\n"); + } + +out: + ret = rte_mp_reply(&mp_res, peer); + return ret; +} + +int rte_cryptodev_mp_request_register(void) +{ + RTE_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + return rte_mp_action_register(CRYPTODEV_MP_REQ, + rte_cryptodev_ipc_request); +} + +void rte_cryptodev_mp_request_unregister(void) +{ + RTE_ASSERT(rte_eal_process_type() == RTE_PROC_PRIMARY); + rte_mp_action_unregister(CRYPTODEV_MP_REQ); +} + struct rte_cryptodev_cb * rte_cryptodev_add_enq_callback(uint8_t dev_id, uint16_t qp_id, diff --git a/lib/cryptodev/rte_cryptodev.h b/lib/cryptodev/rte_cryptodev.h index ece7157970..80076f487c 100644 --- a/lib/cryptodev/rte_cryptodev.h +++ b/lib/cryptodev/rte_cryptodev.h @@ -539,6 +539,30 @@ enum rte_cryptodev_event_type { RTE_CRYPTODEV_EVENT_MAX /**< max value of this enum */ }; +/** Request types for IPC. */ +enum rte_cryptodev_mp_req_type { + RTE_CRYPTODEV_MP_REQ_NONE, /**< unknown event type */ + RTE_CRYPTODEV_MP_REQ_QP_SET, /**< Queue pair setup request */ + RTE_CRYPTODEV_MP_REQ_QP_FREE /**< Queue pair free request */ +}; + +/** Parameters for IPC. */ +struct rte_cryptodev_mp_param { + enum rte_cryptodev_mp_req_type type; /**< IPC request type */ + int dev_id; + /**< The identifier of the device */ + int qp_id; + /**< The index of the queue pair to be configured */ + int socket_id; + /**< Socket to allocate resources on */ + uint16_t process_id; + /**< The pid who send out the requested */ + struct rte_cryptodev_qp_conf *queue_conf; + /**< A pointer of Crypto device queue pair configuration structure */ + int result; + /**< The request result for response message */ +}; + /** Crypto device queue pair configuration structure. */ struct rte_cryptodev_qp_conf { uint32_t nb_descriptors; /**< Number of descriptors per queue pair */ @@ -767,6 +791,28 @@ extern int rte_cryptodev_queue_pair_setup(uint8_t dev_id, uint16_t queue_pair_id, const struct rte_cryptodev_qp_conf *qp_conf, int socket_id); +/** + * Register multi process request IPC handler + * + * Allow secondary process to send IPC request to setup queue pairs + * once register function called in primary process. + * + * @return + * - 0: Success registered + * - 1: Failed registration failed + * - EINVAL: device was not configured + */ +__rte_experimental +int +rte_cryptodev_mp_request_register(void); + +/** + * Unregister multi process unrequest IPC handler + */ +__rte_experimental +void +rte_cryptodev_mp_request_unregister(void); + /** * Get the status of queue pairs setup on a specific crypto device * diff --git a/lib/cryptodev/version.map b/lib/cryptodev/version.map index 00c99fb45c..e964a3d5ab 100644 --- a/lib/cryptodev/version.map +++ b/lib/cryptodev/version.map @@ -150,6 +150,8 @@ EXPERIMENTAL { __rte_cryptodev_trace_sym_session_get_user_data; __rte_cryptodev_trace_sym_session_set_user_data; __rte_cryptodev_trace_count; + rte_cryptodev_mp_request_register; + rte_cryptodev_mp_request_unregister; }; INTERNAL { -- 2.17.1