From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 32776A034F; Tue, 7 Dec 2021 07:52:06 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 2172342712; Tue, 7 Dec 2021 07:51:53 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 56115411CB for ; Tue, 7 Dec 2021 07:51:51 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 1B75a0oN014933 for ; Mon, 6 Dec 2021 22:51:50 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=lUupCfPhoh7ZLpolk4eOqSXxgsd7E/63pLqp6YctdJc=; b=Z/zoCbu1EAeH+SsJIvDLL+TaFsxZl2yxgBj/Ez4Ys92h4xVzvT8cFI1PtsD9JAo4DrAj 2CffUBZ5gi7tT755KVLCS2Yy0PF55HW5KnibvnAQ0EwtqJh9723a67E0jik0IqNe9O6V Q0CW4BzuibsvJImb06KAUDUJWxq1f8paq2Sqgx7GkVjJKxxwP2ol6UkZvpz7MSB+RpwI 09zgAPbmx9HewiLLMn0ECMp48YYS/PLzvosHnem5yADfJKqQ5KwqQQmtdtIgUyYmw0+p TtdNYrvf+Y83rLP7ug0Lc1OHdRSqkJef9AH1zMTLRJbWt+wwJRgJ1UXpzQK4DHnAGwGa KQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3ct1hyg7eb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Mon, 06 Dec 2021 22:51:50 -0800 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 6 Dec 2021 22:51:48 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 6 Dec 2021 22:51:48 -0800 Received: from HY-LT1002.marvell.com (HY-LT1002.marvell.com [10.28.176.218]) by maili.marvell.com (Postfix) with ESMTP id 7C52F3F7071; Mon, 6 Dec 2021 22:51:46 -0800 (PST) From: Anoob Joseph To: Akhil Goyal , Jerin Jacob CC: Anoob Joseph , Archana Muniganti , Tejasree Kondoj , Subject: [PATCH 12/25] crypto/cnxk: account for CPT CTX updates and flush delays Date: Tue, 7 Dec 2021 12:20:45 +0530 Message-ID: <1638859858-734-13-git-send-email-anoobj@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1638859858-734-1-git-send-email-anoobj@marvell.com> References: <1638859858-734-1-git-send-email-anoobj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: LXxq_SU2-zd8uMtCwcf-DEfJvnuA1Uvn X-Proofpoint-GUID: LXxq_SU2-zd8uMtCwcf-DEfJvnuA1Uvn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2021-12-07_02,2021-12-06_02,2021-12-02_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org CPT CTX write with microcode would require CPT flush to complete to have DRAM updated with the SA. Since datapath requires SA direction field, introduce a new flag for the same. Session destroy path is also updated to clear sa.valid bit using CTX reload operation. Session is updated with marker to differentiate s/w immutable and s/w mutable portions. Signed-off-by: Anoob Joseph --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 4 +-- drivers/crypto/cnxk/cn10k_ipsec.c | 60 ++++++++++++++++++++++++------- drivers/crypto/cnxk/cn10k_ipsec.h | 25 ++++++++----- drivers/crypto/cnxk/cn10k_ipsec_la_ops.h | 18 +++++----- 4 files changed, 76 insertions(+), 31 deletions(-) diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index d25a17c..7617bdc 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -53,7 +53,6 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess, struct cpt_inflight_req *infl_req, struct cpt_inst_s *inst) { struct rte_crypto_sym_op *sym_op = op->sym; - union roc_ot_ipsec_sa_word2 *w2; struct cn10k_ipsec_sa *sa; int ret; @@ -68,9 +67,8 @@ cpt_sec_inst_fill(struct rte_crypto_op *op, struct cn10k_sec_session *sess, } sa = &sess->sa; - w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2; - if (w2->s.dir == ROC_IE_SA_DIR_OUTBOUND) + if (sa->is_outbound) ret = process_outb_sa(op, sa, inst); else { infl_req->op_flags |= CPT_OP_FLAGS_IPSEC_DIR_INBOUND; diff --git a/drivers/crypto/cnxk/cn10k_ipsec.c b/drivers/crypto/cnxk/cn10k_ipsec.c index a11a6b7..b4acbac 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.c +++ b/drivers/crypto/cnxk/cn10k_ipsec.c @@ -67,7 +67,7 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, goto sa_dptr_free; } - sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa); + sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, out_sa); #ifdef LA_IPSEC_DEBUG /* Use IV from application in debug mode */ @@ -89,6 +89,8 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, } #endif + sa->is_outbound = true; + /* Get Rlen calculation data */ ret = cnxk_ipsec_outb_rlens_get(&rlens, ipsec_xfrm, crypto_xfrm); if (ret) @@ -127,6 +129,8 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, /* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */ memcpy(out_sa, sa_dptr, 8); + plt_atomic_thread_fence(__ATOMIC_SEQ_CST); + /* Write session using microcode opcode */ ret = roc_cpt_ctx_write(lf, sa_dptr, out_sa, ROC_NIX_INL_OT_IPSEC_OUTB_HW_SZ); @@ -135,9 +139,11 @@ cn10k_ipsec_outb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, goto sa_dptr_free; } - /* Trigger CTX flush to write dirty data back to DRAM */ + /* Trigger CTX flush so that data is written back to DRAM */ roc_cpt_lf_ctx_flush(lf, out_sa, false); + plt_atomic_thread_fence(__ATOMIC_SEQ_CST); + sa_dptr_free: plt_free(sa_dptr); @@ -178,7 +184,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, goto sa_dptr_free; } - sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, sa); + sa->is_outbound = false; + sa->inst.w7 = ipsec_cpt_inst_w7_get(roc_cpt, in_sa); /* pre-populate CPT INST word 4 */ inst_w4.u64 = 0; @@ -214,6 +221,8 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, /* Copy word0 from sa_dptr to populate ctx_push_sz ctx_size fields */ memcpy(in_sa, sa_dptr, 8); + plt_atomic_thread_fence(__ATOMIC_SEQ_CST); + /* Write session using microcode opcode */ ret = roc_cpt_ctx_write(lf, sa_dptr, in_sa, ROC_NIX_INL_OT_IPSEC_INB_HW_SZ); @@ -222,9 +231,11 @@ cn10k_ipsec_inb_sa_create(struct roc_cpt *roc_cpt, struct roc_cpt_lf *lf, goto sa_dptr_free; } - /* Trigger CTX flush to write dirty data back to DRAM */ + /* Trigger CTX flush so that data is written back to DRAM */ roc_cpt_lf_ctx_flush(lf, in_sa, false); + plt_atomic_thread_fence(__ATOMIC_SEQ_CST); + sa_dptr_free: plt_free(sa_dptr); @@ -300,21 +311,46 @@ cn10k_sec_session_create(void *device, struct rte_security_session_conf *conf, } static int -cn10k_sec_session_destroy(void *device __rte_unused, - struct rte_security_session *sess) +cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) { - struct cn10k_sec_session *priv; + struct rte_cryptodev *crypto_dev = dev; + union roc_ot_ipsec_sa_word2 *w2; + struct cn10k_sec_session *sess; struct rte_mempool *sess_mp; + struct cn10k_ipsec_sa *sa; + struct cnxk_cpt_qp *qp; + struct roc_cpt_lf *lf; - priv = get_sec_session_private_data(sess); + sess = get_sec_session_private_data(sec_sess); + if (sess == NULL) + return 0; - if (priv == NULL) + qp = crypto_dev->data->queue_pairs[0]; + if (qp == NULL) return 0; - sess_mp = rte_mempool_from_obj(priv); + lf = &qp->lf; - set_sec_session_private_data(sess, NULL); - rte_mempool_put(sess_mp, priv); + sa = &sess->sa; + + /* Trigger CTX flush to write dirty data back to DRAM */ + roc_cpt_lf_ctx_flush(lf, &sa->in_sa, false); + + /* Wait for 1 ms so that flush is complete */ + rte_delay_ms(1); + + w2 = (union roc_ot_ipsec_sa_word2 *)&sa->in_sa.w2; + w2->s.valid = 0; + + plt_atomic_thread_fence(__ATOMIC_SEQ_CST); + + /* Trigger CTX reload to fetch new data from DRAM */ + roc_cpt_lf_ctx_reload(lf, &sa->in_sa); + + sess_mp = rte_mempool_from_obj(sess); + + set_sec_session_private_data(sec_sess, NULL); + rte_mempool_put(sess_mp, sess); return 0; } diff --git a/drivers/crypto/cnxk/cn10k_ipsec.h b/drivers/crypto/cnxk/cn10k_ipsec.h index 86cd248..8be1fee 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec.h +++ b/drivers/crypto/cnxk/cn10k_ipsec.h @@ -7,24 +7,33 @@ #include +#include "roc_api.h" + #include "cnxk_ipsec.h" -#define CN10K_IPSEC_SA_CTX_HDR_SIZE 1 +typedef void *CN10K_SA_CONTEXT_MARKER[0]; struct cn10k_ipsec_sa { - union { - /** Inbound SA */ - struct roc_ot_ipsec_inb_sa in_sa; - /** Outbound SA */ - struct roc_ot_ipsec_outb_sa out_sa; - }; /** Pre-populated CPT inst words */ struct cnxk_cpt_inst_tmpl inst; uint16_t max_extended_len; uint16_t iv_offset; uint8_t iv_length; bool ip_csum_enable; -}; + bool is_outbound; + + /** + * End of SW mutable area + */ + CN10K_SA_CONTEXT_MARKER sw_area_end __rte_aligned(ROC_ALIGN); + + union { + /** Inbound SA */ + struct roc_ot_ipsec_inb_sa in_sa; + /** Outbound SA */ + struct roc_ot_ipsec_outb_sa out_sa; + }; +} __rte_aligned(ROC_ALIGN); struct cn10k_sec_session { struct cn10k_ipsec_sa sa; diff --git a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h index 881fbd1..cab6a50 100644 --- a/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h +++ b/drivers/crypto/cnxk/cn10k_ipsec_la_ops.h @@ -54,6 +54,7 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess, struct rte_crypto_sym_op *sym_op = cop->sym; struct rte_mbuf *m_src = sym_op->m_src; uint64_t inst_w4_u64 = sess->inst.w4; + uint64_t dptr; if (unlikely(rte_pktmbuf_tailroom(m_src) < sess->max_extended_len)) { plt_dp_err("Not enough tail room"); @@ -76,10 +77,10 @@ process_outb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sess, inst_w4_u64 &= ~BIT_ULL(32); /* Prepare CPT instruction */ - inst->w4.u64 = inst_w4_u64; - inst->w4.s.dlen = rte_pktmbuf_pkt_len(m_src); - inst->dptr = rte_pktmbuf_iova(m_src); - inst->rptr = inst->dptr; + inst->w4.u64 = inst_w4_u64 | rte_pktmbuf_pkt_len(m_src); + dptr = rte_pktmbuf_iova(m_src); + inst->dptr = dptr; + inst->rptr = dptr; return 0; } @@ -90,12 +91,13 @@ process_inb_sa(struct rte_crypto_op *cop, struct cn10k_ipsec_sa *sa, { struct rte_crypto_sym_op *sym_op = cop->sym; struct rte_mbuf *m_src = sym_op->m_src; + uint64_t dptr; /* Prepare CPT instruction */ - inst->w4.u64 = sa->inst.w4; - inst->w4.s.dlen = rte_pktmbuf_pkt_len(m_src); - inst->dptr = rte_pktmbuf_iova(m_src); - inst->rptr = inst->dptr; + inst->w4.u64 = sa->inst.w4 | rte_pktmbuf_pkt_len(m_src); + dptr = rte_pktmbuf_iova(m_src); + inst->dptr = dptr; + inst->rptr = dptr; return 0; } -- 2.7.4