From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 23D98A0093; Thu, 21 Apr 2022 16:37:43 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8EC07427EE; Thu, 21 Apr 2022 16:37:40 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 2D967427EE for ; Thu, 21 Apr 2022 16:37:39 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 23LDCRcC027250; Thu, 21 Apr 2022 07:37:38 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=Jm3GwwjftevhBwO1vSqQUMdYM2Ak6QxHsB7/c9dX5BQ=; b=UeaMSbKxc6II6E8p1Mn6v1P6v5WJ9nw+VNUSHClLR1Efcih2jRh1l5+QzjBkUOFvLArB ZVn/jHyCxzVZznbJkwphCJJvFgCtmw0pT0PJ0Yy4ULyuz3tmY+M6O7OWvpiguIegLKLz TFxYFg3kiHnNHtVR+E4xZG95e5tkBtIGnNTZdGoAMQNBveJwDpMIG9EURPH3QsN0yG/8 1yEGgTUIOzVmqOo/20QwiiZype7TEqNIZ897v8R7cPaJjEbZ12dBizapr8QF9ZwzJgOa kwC33M5SAzZQPCzEdQW+LmzWOt5CYXcOKKzjZ9jFbaf/goCuCwD0bOCs3U5KQGj3+C21 CA== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3fhtapjcac-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 21 Apr 2022 07:37:38 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Thu, 21 Apr 2022 07:37:36 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.18 via Frontend Transport; Thu, 21 Apr 2022 07:37:36 -0700 Received: from localhost.localdomain (unknown [10.28.48.55]) by maili.marvell.com (Postfix) with ESMTP id A262D3F7061; Thu, 21 Apr 2022 07:37:33 -0700 (PDT) From: Akhil Goyal To: CC: , , , , , , Akhil Goyal Subject: [PATCH v3 2/7] crypto/cnxk: add event metadata set operation Date: Thu, 21 Apr 2022 20:07:15 +0530 Message-ID: <20220421143720.1583062-3-gakhil@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220421143720.1583062-1-gakhil@marvell.com> References: <20220418193325.368313-1-gakhil@marvell.com> <20220421143720.1583062-1-gakhil@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: UJNcpkfIa_G2hzwSvAFB_9YtuFBwzGqH X-Proofpoint-ORIG-GUID: UJNcpkfIa_G2hzwSvAFB_9YtuFBwzGqH X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514 definitions=2022-04-21_01,2022-04-21_01,2022-02-23_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko Added cryptodev operation for setting event crypto metadata for all supported sessions - sym/asym/security. Signed-off-by: Volodymyr Fialko Signed-off-by: Akhil Goyal --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 144 +++++++++++++++++++--- drivers/crypto/cnxk/cn10k_ipsec.h | 2 + drivers/crypto/cnxk/cn9k_cryptodev_ops.c | 138 ++++++++++++++++++--- drivers/crypto/cnxk/cn9k_ipsec.h | 2 + drivers/crypto/cnxk/cnxk_ae.h | 2 + drivers/crypto/cnxk/cnxk_cryptodev_ops.h | 18 --- drivers/crypto/cnxk/cnxk_se.h | 2 + 7 files changed, 255 insertions(+), 53 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index c4d5d039ec..01aa0d6870 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -264,30 +264,136 @@ cn10k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return count + i; } -uint16_t -cn10k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) +static int +cn10k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, + void *sess, + enum rte_crypto_op_type op_type, + enum rte_crypto_op_sess_type sess_type, + void *mdata) { - union rte_event_crypto_metadata *ec_mdata; - struct cpt_inflight_req *infl_req; + union rte_event_crypto_metadata *ec_mdata = mdata; struct rte_event *rsp_info; - uint64_t lmt_base, lmt_arg; - struct cpt_inst_s *inst; struct cnxk_cpt_qp *qp; uint8_t cdev_id; - uint16_t lmt_id; - uint16_t qp_id; - int ret; - - ec_mdata = cnxk_event_crypto_mdata_get(op); - if (!ec_mdata) { - rte_errno = EINVAL; - return 0; - } + int16_t qp_id; + uint64_t w2; + /* Get queue pair */ cdev_id = ec_mdata->request_info.cdev_id; qp_id = ec_mdata->request_info.queue_pair_id; qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + + /* Prepare w2 */ rsp_info = &ec_mdata->response_info; + w2 = CNXK_CPT_INST_W2( + (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, + rsp_info->sched_type, rsp_info->queue_id, 0); + + /* Set meta according to session type */ + if (op_type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct cn10k_sec_session *priv; + struct cn10k_ipsec_sa *sa; + + priv = get_sec_session_private_data(sess); + sa = &priv->sa; + sa->qp = qp; + sa->inst.w2 = w2; + } else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct cnxk_se_sess *priv; + + priv = get_sym_session_private_data( + sess, cn10k_cryptodev_driver_id); + priv->qp = qp; + priv->cpt_inst_w2 = w2; + } else + return -EINVAL; + } else if (op_type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct rte_cryptodev_asym_session *asym_sess = sess; + struct cnxk_ae_sess *priv; + + priv = (struct cnxk_ae_sess *)asym_sess->sess_private_data; + priv->qp = qp; + priv->cpt_inst_w2 = w2; + } else + return -EINVAL; + } else + return -EINVAL; + + return 0; +} + +static inline int +cn10k_ca_meta_info_extract(struct rte_crypto_op *op, + struct cnxk_cpt_qp **qp, uint64_t *w2) +{ + if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct cn10k_sec_session *priv; + struct cn10k_ipsec_sa *sa; + + priv = get_sec_session_private_data(op->sym->sec_session); + sa = &priv->sa; + *qp = sa->qp; + *w2 = sa->inst.w2; + } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct cnxk_se_sess *priv; + + priv = get_sym_session_private_data( + op->sym->session, cn10k_cryptodev_driver_id); + *qp = priv->qp; + *w2 = priv->cpt_inst_w2; + } else { + union rte_event_crypto_metadata *ec_mdata; + struct rte_event *rsp_info; + uint8_t cdev_id; + uint16_t qp_id; + + ec_mdata = (union rte_event_crypto_metadata *) + ((uint8_t *)op + op->private_data_offset); + if (!ec_mdata) + return -EINVAL; + rsp_info = &ec_mdata->response_info; + cdev_id = ec_mdata->request_info.cdev_id; + qp_id = ec_mdata->request_info.queue_pair_id; + *qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + *w2 = CNXK_CPT_INST_W2( + (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, + rsp_info->sched_type, rsp_info->queue_id, 0); + } + } else if (op->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct rte_cryptodev_asym_session *asym_sess; + struct cnxk_ae_sess *priv; + + asym_sess = op->asym->session; + priv = (struct cnxk_ae_sess *)asym_sess->sess_private_data; + *qp = priv->qp; + *w2 = priv->cpt_inst_w2; + } else + return -EINVAL; + } else + return -EINVAL; + + return 0; +} + +uint16_t +cn10k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) +{ + struct cpt_inflight_req *infl_req; + uint64_t lmt_base, lmt_arg, w2; + struct cpt_inst_s *inst; + struct cnxk_cpt_qp *qp; + uint16_t lmt_id; + int ret; + + ret = cn10k_ca_meta_info_extract(op, &qp, &w2); + if (unlikely(ret)) { + rte_errno = EINVAL; + return 0; + } if (unlikely(!qp->ca.enabled)) { rte_errno = EINVAL; @@ -316,9 +422,7 @@ cn10k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) infl_req->qp = qp; inst->w0.u64 = 0; inst->res_addr = (uint64_t)&infl_req->res; - inst->w2.u64 = CNXK_CPT_INST_W2( - (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, - rsp_info->sched_type, rsp_info->queue_id, 0); + inst->w2.u64 = w2; inst->w3.u64 = CNXK_CPT_INST_W3(1, infl_req); if (roc_cpt_is_iq_full(&qp->lf)) { @@ -327,7 +431,7 @@ cn10k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) return 0; } - if (!rsp_info->sched_type) + if (inst->w2.s.tt == RTE_SCHED_TYPE_ORDERED) roc_sso_hws_head_wait(tag_op); lmt_arg = ROC_CN10K_CPT_LMT_ARG | (uint64_t)lmt_id; @@ -592,4 +696,6 @@ struct rte_cryptodev_ops cn10k_cpt_ops = { .asym_session_configure = cnxk_ae_session_cfg, .asym_session_clear = cnxk_ae_session_clear, + /* Event crypto ops */ + .session_ev_mdata_set = cn10k_cpt_crypto_adapter_ev_mdata_set, }; diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h index 647a71cdd5..1c1d904799 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.h +++ b/drivers/crypto/cnxk/cn10k_ipsec.h @@ -20,6 +20,8 @@ struct cn10k_ipsec_sa { uint16_t iv_offset; uint8_t iv_length; bool is_outbound; + /** Queue pair */ + struct cnxk_cpt_qp *qp; /** * End of SW mutable area diff --git a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c index d3d441cb24..98fa97ef01 100644 --- a/drivers/crypto/cnxk/cn9k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn9k_cryptodev_ops.c @@ -316,28 +316,134 @@ cn9k_cpt_enqueue_burst(void *qptr, struct rte_crypto_op **ops, uint16_t nb_ops) return count; } -uint16_t -cn9k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) +static int +cn9k_cpt_crypto_adapter_ev_mdata_set(struct rte_cryptodev *dev __rte_unused, + void *sess, + enum rte_crypto_op_type op_type, + enum rte_crypto_op_sess_type sess_type, + void *mdata) { - union rte_event_crypto_metadata *ec_mdata; - struct cpt_inflight_req *infl_req; + union rte_event_crypto_metadata *ec_mdata = mdata; struct rte_event *rsp_info; struct cnxk_cpt_qp *qp; - struct cpt_inst_s inst; uint8_t cdev_id; uint16_t qp_id; - int ret; - - ec_mdata = cnxk_event_crypto_mdata_get(op); - if (!ec_mdata) { - rte_errno = EINVAL; - return 0; - } + uint64_t w2; + /* Get queue pair */ cdev_id = ec_mdata->request_info.cdev_id; qp_id = ec_mdata->request_info.queue_pair_id; qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + + /* Prepare w2 */ rsp_info = &ec_mdata->response_info; + w2 = CNXK_CPT_INST_W2( + (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, + rsp_info->sched_type, rsp_info->queue_id, 0); + + /* Set meta according to session type */ + if (op_type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct cn9k_sec_session *priv; + struct cn9k_ipsec_sa *sa; + + priv = get_sec_session_private_data(sess); + sa = &priv->sa; + sa->qp = qp; + sa->inst.w2 = w2; + } else if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct cnxk_se_sess *priv; + + priv = get_sym_session_private_data( + sess, cn9k_cryptodev_driver_id); + priv->qp = qp; + priv->cpt_inst_w2 = w2; + } else + return -EINVAL; + } else if (op_type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + if (sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct rte_cryptodev_asym_session *asym_sess = sess; + struct cnxk_ae_sess *priv; + + priv = (struct cnxk_ae_sess *)asym_sess->sess_private_data; + priv->qp = qp; + priv->cpt_inst_w2 = w2; + } else + return -EINVAL; + } else + return -EINVAL; + + return 0; +} + +static inline int +cn9k_ca_meta_info_extract(struct rte_crypto_op *op, + struct cnxk_cpt_qp **qp, struct cpt_inst_s *inst) +{ + if (op->type == RTE_CRYPTO_OP_TYPE_SYMMETRIC) { + if (op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct cn9k_sec_session *priv; + struct cn9k_ipsec_sa *sa; + + priv = get_sec_session_private_data(op->sym->sec_session); + sa = &priv->sa; + *qp = sa->qp; + inst->w2.u64 = sa->inst.w2; + } else if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct cnxk_se_sess *priv; + + priv = get_sym_session_private_data( + op->sym->session, cn9k_cryptodev_driver_id); + *qp = priv->qp; + inst->w2.u64 = priv->cpt_inst_w2; + } else { + union rte_event_crypto_metadata *ec_mdata; + struct rte_event *rsp_info; + uint8_t cdev_id; + uint16_t qp_id; + + ec_mdata = (union rte_event_crypto_metadata *) + ((uint8_t *)op + op->private_data_offset); + if (!ec_mdata) + return -EINVAL; + rsp_info = &ec_mdata->response_info; + cdev_id = ec_mdata->request_info.cdev_id; + qp_id = ec_mdata->request_info.queue_pair_id; + *qp = rte_cryptodevs[cdev_id].data->queue_pairs[qp_id]; + inst->w2.u64 = CNXK_CPT_INST_W2( + (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, + rsp_info->sched_type, rsp_info->queue_id, 0); + } + } else if (op->type == RTE_CRYPTO_OP_TYPE_ASYMMETRIC) { + if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + struct rte_cryptodev_asym_session *asym_sess; + struct cnxk_ae_sess *priv; + + asym_sess = op->asym->session; + priv = (struct cnxk_ae_sess *)asym_sess->sess_private_data; + *qp = priv->qp; + inst->w2.u64 = priv->cpt_inst_w2; + } else + return -EINVAL; + } else + return -EINVAL; + + return 0; +} + +uint16_t +cn9k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) +{ + struct cpt_inflight_req *infl_req; + struct cnxk_cpt_qp *qp; + struct cpt_inst_s inst; + int ret; + + ret = cn9k_ca_meta_info_extract(op, &qp, &inst); + if (unlikely(ret)) { + rte_errno = EINVAL; + return 0; + } if (unlikely(!qp->ca.enabled)) { rte_errno = EINVAL; @@ -362,9 +468,6 @@ cn9k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) infl_req->qp = qp; inst.w0.u64 = 0; inst.res_addr = (uint64_t)&infl_req->res; - inst.w2.u64 = CNXK_CPT_INST_W2( - (RTE_EVENT_TYPE_CRYPTODEV << 28) | rsp_info->flow_id, - rsp_info->sched_type, rsp_info->queue_id, 0); inst.w3.u64 = CNXK_CPT_INST_W3(1, infl_req); if (roc_cpt_is_iq_full(&qp->lf)) { @@ -373,7 +476,7 @@ cn9k_cpt_crypto_adapter_enqueue(uintptr_t tag_op, struct rte_crypto_op *op) return 0; } - if (!rsp_info->sched_type) + if (inst.w2.s.tt == RTE_SCHED_TYPE_ORDERED) roc_sso_hws_head_wait(tag_op); cn9k_cpt_inst_submit(&inst, qp->lmtline.lmt_base, qp->lmtline.io_addr); @@ -613,4 +716,7 @@ struct rte_cryptodev_ops cn9k_cpt_ops = { .asym_session_configure = cnxk_ae_session_cfg, .asym_session_clear = cnxk_ae_session_clear, + /* Event crypto ops */ + .session_ev_mdata_set = cn9k_cpt_crypto_adapter_ev_mdata_set, + }; diff --git a/drivers/crypto/cnxk/cn9k_ipsec.h b/drivers/crypto/cnxk/cn9k_ipsec.h index f3acad561b..499dbc2782 100644 --- a/drivers/crypto/cnxk/cn9k_ipsec.h +++ b/drivers/crypto/cnxk/cn9k_ipsec.h @@ -42,6 +42,8 @@ struct cn9k_ipsec_sa { struct cnxk_on_ipsec_ar ar; /** Anti replay window size */ uint32_t replay_win_sz; + /** Queue pair */ + struct cnxk_cpt_qp *qp; }; struct cn9k_sec_session { diff --git a/drivers/crypto/cnxk/cnxk_ae.h b/drivers/crypto/cnxk/cnxk_ae.h index 01ccfcd334..10854c79c8 100644 --- a/drivers/crypto/cnxk/cnxk_ae.h +++ b/drivers/crypto/cnxk/cnxk_ae.h @@ -22,6 +22,8 @@ struct cnxk_ae_sess { uint64_t *cnxk_fpm_iova; struct roc_ae_ec_group **ec_grp; uint64_t cpt_inst_w7; + uint64_t cpt_inst_w2; + struct cnxk_cpt_qp *qp; }; static __rte_always_inline void diff --git a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h index ab0f00ee7c..7ece0214dc 100644 --- a/drivers/crypto/cnxk/cnxk_cryptodev_ops.h +++ b/drivers/crypto/cnxk/cnxk_cryptodev_ops.h @@ -125,24 +125,6 @@ int cnxk_ae_session_cfg(struct rte_cryptodev *dev, struct rte_cryptodev_asym_session *sess); void cnxk_cpt_dump_on_err(struct cnxk_cpt_qp *qp); -static inline union rte_event_crypto_metadata * -cnxk_event_crypto_mdata_get(struct rte_crypto_op *op) -{ - union rte_event_crypto_metadata *ec_mdata; - - if (op->sess_type == RTE_CRYPTO_OP_WITH_SESSION) - ec_mdata = rte_cryptodev_sym_session_get_user_data( - op->sym->session); - else if (op->sess_type == RTE_CRYPTO_OP_SESSIONLESS && - op->private_data_offset) - ec_mdata = (union rte_event_crypto_metadata - *)((uint8_t *)op + op->private_data_offset); - else - return NULL; - - return ec_mdata; -} - static __rte_always_inline void pending_queue_advance(uint64_t *index, const uint64_t mask) { diff --git a/drivers/crypto/cnxk/cnxk_se.h b/drivers/crypto/cnxk/cnxk_se.h index e988d57b94..b11c2bb3a2 100644 --- a/drivers/crypto/cnxk/cnxk_se.h +++ b/drivers/crypto/cnxk/cnxk_se.h @@ -33,6 +33,8 @@ struct cnxk_se_sess { uint16_t auth_iv_offset; uint32_t salt; uint64_t cpt_inst_w7; + uint64_t cpt_inst_w2; + struct cnxk_cpt_qp *qp; struct roc_se_ctx roc_se_ctx; } __rte_cache_aligned; -- 2.25.1