From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5128F43750; Thu, 21 Dec 2023 13:37:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CB10640A81; Thu, 21 Dec 2023 13:36:52 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id C78CB40A7A for ; Thu, 21 Dec 2023 13:36:50 +0100 (CET) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BLCVS30019305 for ; Thu, 21 Dec 2023 04:36:50 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s= pfpt0220; bh=ooczj5tBdgwBBToxYaYPBuy7UD8WFOqDWIKaa0dCMYY=; b=ZvR wPXnuO+ZMquXZ8b63wZDmkiiBlU2M26WUmf8p5sYzu9fXBX0Q4r0vAUFnQucw2aO Z3+X20ZBgiW5c2RC6AwvvFsu4I1BjMpBp2InAVwbU/oV6V6RaqRGBDqcS52L2loa YSm8vLN3URAccGGZDfVoMvfaHI0tX61fS6sSSFPfsgT2h/28HarW6F4ekJG2Io2v ZFbO7mIVAkQRXSRafIUTMKOvDTPExDxG50xc/rPgb6kgLvtek4gNd2qyj8BIQ+3T Sv+7y9QrNc7+W+xzlQQWbKyhhWILqwyC60NrGq/qvgOtDgoFwqdpjMRzqnGEU5pj XWI++JBAHBKnOirW2oQ== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3v4nekg0jq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT) for ; Thu, 21 Dec 2023 04:36:49 -0800 (PST) Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Thu, 21 Dec 2023 04:36:32 -0800 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Thu, 21 Dec 2023 04:36:32 -0800 Received: from BG-LT92004.corp.innovium.com (unknown [10.193.71.152]) by maili.marvell.com (Postfix) with ESMTP id 949BA3F7073; Thu, 21 Dec 2023 04:36:29 -0800 (PST) From: Anoob Joseph To: Akhil Goyal CC: Vidya Sagar Velumuri , Jerin Jacob , Tejasree Kondoj , Subject: [PATCH 16/24] crypto/cnxk: add TLS record datapath handling Date: Thu, 21 Dec 2023 18:05:37 +0530 Message-ID: <20231221123545.510-17-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20231221123545.510-1-anoobj@marvell.com> References: <20231221123545.510-1-anoobj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: TrESxIed-PYKHlacPyLLb1-_Y5NMnJgW X-Proofpoint-ORIG-GUID: TrESxIed-PYKHlacPyLLb1-_Y5NMnJgW X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Vidya Sagar Velumuri Add support for TLS record handling in datapath. Signed-off-by: Anoob Joseph Signed-off-by: Vidya Sagar Velumuri --- drivers/crypto/cnxk/cn10k_cryptodev_ops.c | 57 +++- drivers/crypto/cnxk/cn10k_cryptodev_sec.c | 7 + drivers/crypto/cnxk/cn10k_tls_ops.h | 322 ++++++++++++++++++++++ 3 files changed, 380 insertions(+), 6 deletions(-) create mode 100644 drivers/crypto/cnxk/cn10k_tls_ops.h diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c index f105a431f8..c87a8bae1a 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_ops.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_ops.c @@ -20,11 +20,14 @@ #include "roc_sso_dp.h" #include "cn10k_cryptodev.h" -#include "cn10k_cryptodev_ops.h" #include "cn10k_cryptodev_event_dp.h" +#include "cn10k_cryptodev_ops.h" +#include "cn10k_cryptodev_sec.h" #include "cn10k_eventdev.h" #include "cn10k_ipsec.h" #include "cn10k_ipsec_la_ops.h" +#include "cn10k_tls.h" +#include "cn10k_tls_ops.h" #include "cnxk_ae.h" #include "cnxk_cryptodev.h" #include "cnxk_cryptodev_ops.h" @@ -101,6 +104,18 @@ cpt_sec_ipsec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, return ret; } +static __rte_always_inline int __rte_hot +cpt_sec_tls_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, + struct cn10k_sec_session *sess, struct cpt_inst_s *inst, + struct cpt_inflight_req *infl_req, const bool is_sg_ver2) +{ + if (sess->tls.is_write) + return process_tls_write(&qp->lf, op, sess, &qp->meta_info, infl_req, inst, + is_sg_ver2); + else + return process_tls_read(op, sess, &qp->meta_info, infl_req, inst, is_sg_ver2); +} + static __rte_always_inline int __rte_hot cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k_sec_session *sess, struct cpt_inst_s *inst, struct cpt_inflight_req *infl_req, const bool is_sg_ver2) @@ -108,6 +123,8 @@ cpt_sec_inst_fill(struct cnxk_cpt_qp *qp, struct rte_crypto_op *op, struct cn10k if (sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) return cpt_sec_ipsec_inst_fill(qp, op, sess, &inst[0], infl_req, is_sg_ver2); + else if (sess->proto == RTE_SECURITY_PROTOCOL_TLS_RECORD) + return cpt_sec_tls_inst_fill(qp, op, sess, &inst[0], infl_req, is_sg_ver2); return 0; } @@ -812,7 +829,7 @@ cn10k_cpt_sg_ver2_crypto_adapter_enqueue(void *ws, struct rte_event ev[], uint16 } static inline void -cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) +cn10k_cpt_ipsec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) { struct rte_mbuf *mbuf = cop->sym->m_src; const uint16_t m_len = res->rlen; @@ -849,10 +866,38 @@ cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *re } static inline void -cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, - struct rte_crypto_op *cop, - struct cpt_inflight_req *infl_req, - struct cpt_cn10k_res_s *res) +cn10k_cpt_tls_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) +{ + struct rte_mbuf *mbuf = cop->sym->m_src; + const uint16_t m_len = res->rlen; + + if (!res->uc_compcode) { + if (mbuf->next == NULL) + mbuf->data_len = m_len; + mbuf->pkt_len = m_len; + } else { + cop->status = RTE_CRYPTO_OP_STATUS_ERROR; + cop->aux_flags = res->uc_compcode; + plt_err("crypto op failed with UC compcode: 0x%x", res->uc_compcode); + } +} + +static inline void +cn10k_cpt_sec_post_process(struct rte_crypto_op *cop, struct cpt_cn10k_res_s *res) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; + struct cn10k_sec_session *sess; + + sess = sym_op->session; + if (sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) + cn10k_cpt_ipsec_post_process(cop, res); + else if (sess->proto == RTE_SECURITY_PROTOCOL_TLS_RECORD) + cn10k_cpt_tls_post_process(cop, res); +} + +static inline void +cn10k_cpt_dequeue_post_process(struct cnxk_cpt_qp *qp, struct rte_crypto_op *cop, + struct cpt_inflight_req *infl_req, struct cpt_cn10k_res_s *res) { const uint8_t uc_compcode = res->uc_compcode; const uint8_t compcode = res->compcode; diff --git a/drivers/crypto/cnxk/cn10k_cryptodev_sec.c b/drivers/crypto/cnxk/cn10k_cryptodev_sec.c index 0fd0a5b03c..300a8e4f94 100644 --- a/drivers/crypto/cnxk/cn10k_cryptodev_sec.c +++ b/drivers/crypto/cnxk/cn10k_cryptodev_sec.c @@ -32,6 +32,10 @@ cn10k_sec_session_create(void *dev, struct rte_security_session_conf *conf, return cn10k_ipsec_session_create(vf, qp, &conf->ipsec, conf->crypto_xform, sess); } + if (conf->protocol == RTE_SECURITY_PROTOCOL_TLS_RECORD) + return cn10k_tls_record_session_create(vf, qp, &conf->tls_record, + conf->crypto_xform, sess); + return -ENOTSUP; } @@ -54,6 +58,9 @@ cn10k_sec_session_destroy(void *dev, struct rte_security_session *sec_sess) if (cn10k_sec_sess->proto == RTE_SECURITY_PROTOCOL_IPSEC) return cn10k_sec_ipsec_session_destroy(qp, cn10k_sec_sess); + if (cn10k_sec_sess->proto == RTE_SECURITY_PROTOCOL_TLS_RECORD) + return cn10k_sec_tls_session_destroy(qp, cn10k_sec_sess); + return -EINVAL; } diff --git a/drivers/crypto/cnxk/cn10k_tls_ops.h b/drivers/crypto/cnxk/cn10k_tls_ops.h new file mode 100644 index 0000000000..a5d38bacbb --- /dev/null +++ b/drivers/crypto/cnxk/cn10k_tls_ops.h @@ -0,0 +1,322 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef __CN10K_TLS_OPS_H__ +#define __CN10K_TLS_OPS_H__ + +#include +#include + +#include "roc_ie.h" + +#include "cn10k_cryptodev.h" +#include "cn10k_cryptodev_sec.h" +#include "cnxk_cryptodev.h" +#include "cnxk_cryptodev_ops.h" +#include "cnxk_sg.h" + +static __rte_always_inline int +process_tls_write(struct roc_cpt_lf *lf, struct rte_crypto_op *cop, struct cn10k_sec_session *sess, + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst, const bool is_sg_ver2) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; +#ifdef LA_IPSEC_DEBUG + struct roc_ie_ot_tls_write_sa *write_sa; +#endif + struct rte_mbuf *m_src = sym_op->m_src; + struct rte_mbuf *last_seg; + union cpt_inst_w4 w4; + void *m_data = NULL; + uint8_t *in_buffer; + +#ifdef LA_IPSEC_DEBUG + write_sa = &sess->tls_rec.write_sa; + if (write_sa->w2.s.iv_at_cptr == ROC_IE_OT_TLS_IV_SRC_FROM_SA) { + + uint8_t *iv = PLT_PTR_ADD(write_sa->cipher_key, 32); + + if (write_sa->w2.s.cipher_select == ROC_IE_OT_TLS_CIPHER_AES_GCM) { + uint32_t *tmp; + + /* For GCM, the IV and salt format will be like below: + * iv[0-3]: lower bytes of IV in BE format. + * iv[4-7]: salt / nonce. + * iv[12-15]: upper bytes of IV in BE format. + */ + memcpy(iv, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), 4); + tmp = (uint32_t *)iv; + *tmp = rte_be_to_cpu_32(*tmp); + + memcpy(iv + 12, + rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset + 4), 4); + tmp = (uint32_t *)(iv + 12); + *tmp = rte_be_to_cpu_32(*tmp); + } else if (write_sa->w2.s.cipher_select == ROC_IE_OT_TLS_CIPHER_AES_CBC) { + uint64_t *tmp; + + memcpy(iv, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), 16); + tmp = (uint64_t *)iv; + *tmp = rte_be_to_cpu_64(*tmp); + tmp = (uint64_t *)(iv + 8); + *tmp = rte_be_to_cpu_64(*tmp); + } else if (write_sa->w2.s.cipher_select == ROC_IE_OT_TLS_CIPHER_3DES) { + uint64_t *tmp; + + memcpy(iv, rte_crypto_op_ctod_offset(cop, uint8_t *, sess->iv_offset), 8); + tmp = (uint64_t *)iv; + *tmp = rte_be_to_cpu_64(*tmp); + } + + /* Trigger CTX reload to fetch new data from DRAM */ + roc_cpt_lf_ctx_reload(lf, write_sa); + rte_delay_ms(1); + } +#else + RTE_SET_USED(lf); +#endif + /* Single buffer direct mode */ + if (likely(m_src->next == NULL)) { + void *vaddr; + + if (unlikely(rte_pktmbuf_tailroom(m_src) < sess->max_extended_len)) { + plt_dp_err("Not enough tail room"); + return -ENOMEM; + } + + vaddr = rte_pktmbuf_mtod(m_src, void *); + inst->dptr = (uint64_t)vaddr; + inst->rptr = (uint64_t)vaddr; + + w4.u64 = sess->inst.w4; + w4.s.param1 = m_src->data_len; + w4.s.dlen = m_src->data_len; + + w4.s.param2 = cop->param1.tls_record.content_type; + w4.s.opcode_minor = sess->tls.enable_padding * cop->aux_flags * 8; + + inst->w4.u64 = w4.u64; + } else if (is_sg_ver2 == false) { + struct roc_sglist_comp *scatter_comp, *gather_comp; + uint32_t g_size_bytes, s_size_bytes; + uint32_t dlen; + int i; + + last_seg = rte_pktmbuf_lastseg(m_src); + + if (unlikely(rte_pktmbuf_tailroom(last_seg) < sess->max_extended_len)) { + plt_dp_err("Not enough tail room (required: %d, available: %d)", + sess->max_extended_len, rte_pktmbuf_tailroom(last_seg)); + return -ENOMEM; + } + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + in_buffer = (uint8_t *)m_data; + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* Input Gather List */ + i = 0; + gather_comp = (struct roc_sglist_comp *)((uint8_t *)in_buffer + 8); + + i = fill_sg_comp_from_pkt(gather_comp, i, m_src); + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + i = 0; + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg_comp_from_pkt(scatter_comp, i, m_src); + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + dlen = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; + + inst->dptr = (uint64_t)in_buffer; + inst->rptr = (uint64_t)in_buffer; + + w4.u64 = sess->inst.w4; + w4.s.dlen = dlen; + w4.s.param1 = rte_pktmbuf_pkt_len(m_src); + w4.s.param2 = cop->param1.tls_record.content_type; + w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; + w4.s.opcode_minor = sess->tls.enable_padding * cop->aux_flags * 8; + + /* Output Scatter List */ + last_seg->data_len += sess->max_extended_len; + inst->w4.u64 = w4.u64; + } else { + struct roc_sg2list_comp *scatter_comp, *gather_comp; + union cpt_inst_w5 cpt_inst_w5; + union cpt_inst_w6 cpt_inst_w6; + uint32_t g_size_bytes; + int i; + + last_seg = rte_pktmbuf_lastseg(m_src); + + if (unlikely(rte_pktmbuf_tailroom(last_seg) < sess->max_extended_len)) { + plt_dp_err("Not enough tail room (required: %d, available: %d)", + sess->max_extended_len, rte_pktmbuf_tailroom(last_seg)); + return -ENOMEM; + } + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + in_buffer = (uint8_t *)m_data; + /* Input Gather List */ + i = 0; + gather_comp = (struct roc_sg2list_comp *)((uint8_t *)in_buffer); + i = fill_sg2_comp_from_pkt(gather_comp, i, m_src); + + cpt_inst_w5.s.gather_sz = ((i + 2) / 3); + g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_sg2list_comp); + + i = 0; + scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg2_comp_from_pkt(scatter_comp, i, m_src); + + cpt_inst_w6.s.scatter_sz = ((i + 2) / 3); + + cpt_inst_w5.s.dptr = (uint64_t)gather_comp; + cpt_inst_w6.s.rptr = (uint64_t)scatter_comp; + + inst->w5.u64 = cpt_inst_w5.u64; + inst->w6.u64 = cpt_inst_w6.u64; + w4.u64 = sess->inst.w4; + w4.s.dlen = rte_pktmbuf_pkt_len(m_src); + w4.s.opcode_major &= (~(ROC_IE_OT_INPLACE_BIT)); + w4.s.opcode_minor = sess->tls.enable_padding * cop->aux_flags * 8; + w4.s.param1 = w4.s.dlen; + w4.s.param2 = cop->param1.tls_record.content_type; + /* Output Scatter List */ + last_seg->data_len += sess->max_extended_len; + inst->w4.u64 = w4.u64; + } + + return 0; +} + +static __rte_always_inline int +process_tls_read(struct rte_crypto_op *cop, struct cn10k_sec_session *sess, + struct cpt_qp_meta_info *m_info, struct cpt_inflight_req *infl_req, + struct cpt_inst_s *inst, const bool is_sg_ver2) +{ + struct rte_crypto_sym_op *sym_op = cop->sym; + struct rte_mbuf *m_src = sym_op->m_src; + union cpt_inst_w4 w4; + uint8_t *in_buffer; + void *m_data; + + if (likely(m_src->next == NULL)) { + void *vaddr; + + vaddr = rte_pktmbuf_mtod(m_src, void *); + + inst->dptr = (uint64_t)vaddr; + inst->rptr = (uint64_t)vaddr; + + w4.u64 = sess->inst.w4; + w4.s.dlen = m_src->data_len; + w4.s.param1 = m_src->data_len; + inst->w4.u64 = w4.u64; + } else if (is_sg_ver2 == false) { + struct roc_sglist_comp *scatter_comp, *gather_comp; + uint32_t g_size_bytes, s_size_bytes; + uint32_t dlen; + int i; + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + in_buffer = (uint8_t *)m_data; + ((uint16_t *)in_buffer)[0] = 0; + ((uint16_t *)in_buffer)[1] = 0; + + /* Input Gather List */ + i = 0; + gather_comp = (struct roc_sglist_comp *)((uint8_t *)in_buffer + 8); + + i = fill_sg_comp_from_pkt(gather_comp, i, m_src); + ((uint16_t *)in_buffer)[2] = rte_cpu_to_be_16(i); + + g_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + i = 0; + scatter_comp = (struct roc_sglist_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg_comp_from_pkt(scatter_comp, i, m_src); + ((uint16_t *)in_buffer)[3] = rte_cpu_to_be_16(i); + + s_size_bytes = ((i + 3) / 4) * sizeof(struct roc_sglist_comp); + + dlen = g_size_bytes + s_size_bytes + ROC_SG_LIST_HDR_SIZE; + + inst->dptr = (uint64_t)in_buffer; + inst->rptr = (uint64_t)in_buffer; + + w4.u64 = sess->inst.w4; + w4.s.dlen = dlen; + w4.s.opcode_major |= (uint64_t)ROC_DMA_MODE_SG; + w4.s.param1 = rte_pktmbuf_pkt_len(m_src); + inst->w4.u64 = w4.u64; + } else { + struct roc_sg2list_comp *scatter_comp, *gather_comp; + union cpt_inst_w5 cpt_inst_w5; + union cpt_inst_w6 cpt_inst_w6; + uint32_t g_size_bytes; + int i; + + m_data = alloc_op_meta(NULL, m_info->mlen, m_info->pool, infl_req); + if (unlikely(m_data == NULL)) { + plt_dp_err("Error allocating meta buffer for request"); + return -ENOMEM; + } + + in_buffer = (uint8_t *)m_data; + /* Input Gather List */ + i = 0; + + gather_comp = (struct roc_sg2list_comp *)((uint8_t *)in_buffer); + i = fill_sg2_comp_from_pkt(gather_comp, i, m_src); + + cpt_inst_w5.s.gather_sz = ((i + 2) / 3); + g_size_bytes = ((i + 2) / 3) * sizeof(struct roc_sg2list_comp); + + i = 0; + scatter_comp = (struct roc_sg2list_comp *)((uint8_t *)gather_comp + g_size_bytes); + + i = fill_sg2_comp_from_pkt(scatter_comp, i, m_src); + + cpt_inst_w6.s.scatter_sz = ((i + 2) / 3); + + cpt_inst_w5.s.dptr = (uint64_t)gather_comp; + cpt_inst_w6.s.rptr = (uint64_t)scatter_comp; + + inst->w5.u64 = cpt_inst_w5.u64; + inst->w6.u64 = cpt_inst_w6.u64; + w4.u64 = sess->inst.w4; + w4.s.dlen = rte_pktmbuf_pkt_len(m_src); + w4.s.param1 = w4.s.dlen; + w4.s.opcode_major &= (~(ROC_IE_OT_INPLACE_BIT)); + inst->w4.u64 = w4.u64; + } + + return 0; +} +#endif /* __CN10K_TLS_OPS_H__ */ -- 2.25.1