From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0F2DB42943; Fri, 14 Apr 2023 19:46:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3456341153; Fri, 14 Apr 2023 19:45:58 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9482E41153 for ; Fri, 14 Apr 2023 19:45:56 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 33EHJ0L4013323; Fri, 14 Apr 2023 10:45:55 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=yJH/D3Ngnbpyo9uu9omO5jdcCK/8KWuSOiL3Ff13G8k=; b=ad0RDAoondoKIBb78dmYZUWMO4sv3BDu1wwAmhWnHYIzEKFUwpcN3aNx3qzZkfJXvATP BNQ/KzPVHjYe2ARV2qFS+MKdDpyB+sPcAqKQF9BnGs6dEemA+QCZXZX3VGfTtXNfTvQJ O4D0l1qzrEBfZ2KmyCID6oJDM6ozGPUgvHnHak1YicXM42G8zSlQW/MoQE/bgikTuQna 5j7L11vuwjFqmP7eYhwZ7Lcw1kzudWqWbtv8fx5BXrRJfdoH6wkGpI5lMcIT3PqFPoKx /Tl61OS9MmVH138OmlXMNYkUNhhdhsLNo7+Jt4J/y6Ww3Rz+DJBGrn0oNBstuugeY2v7 ew== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3py646s6je-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 14 Apr 2023 10:45:55 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Fri, 14 Apr 2023 10:45:53 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Fri, 14 Apr 2023 10:45:53 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.161.183]) by maili.marvell.com (Postfix) with ESMTP id 3CFCE3F7080; Fri, 14 Apr 2023 10:45:47 -0700 (PDT) From: Anoob Joseph To: Thomas Monjalon , Akhil Goyal , Jerin Jacob , Konstantin Ananyev , Bernard Iremonger CC: Hemant Agrawal , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= , "Kiran Kumar K" , Volodymyr Fialko , , Olivier Matz Subject: [PATCH v2 06/22] pdcp: add pre and post process for UL Date: Fri, 14 Apr 2023 23:14:56 +0530 Message-ID: <20230414174512.642-7-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230414174512.642-1-anoobj@marvell.com> References: <20221222092522.1628-1-anoobj@marvell.com> <20230414174512.642-1-anoobj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: eal_7lGJfHfhhoFVRuoajNcya2RFVsrZ X-Proofpoint-ORIG-GUID: eal_7lGJfHfhhoFVRuoajNcya2RFVsrZ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-04-14_10,2023-04-14_01,2023-02-09_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Add routines to perform pre & post processing based on the type of entity. To avoid checks in datapath, there are different function pointers registered based on the following, 1. Control plane v/s user plane 2. 12 bit v/s 18 bit SN For control plane only 12 bit SN need to be supported (as per PDCP specification). Signed-off-by: Anoob Joseph Signed-off-by: Kiran Kumar K Signed-off-by: Volodymyr Fialko --- lib/pdcp/pdcp_entity.h | 42 +++++ lib/pdcp/pdcp_process.c | 330 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 372 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index ca1d56b516..46cdaead09 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -92,4 +92,46 @@ pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static inline uint32_t +pdcp_window_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return 1 << (sn_size - 1); +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + +static inline uint32_t +pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) +{ + return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); +} + +static inline uint32_t +pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << (32 - sn_size)) - 1; +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index d4b158536d..7c1fc85fcb 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -36,6 +36,332 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static inline void +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, + uint8_t data_offset, uint32_t count, const bool is_auth) +{ + const struct rte_crypto_op cop_init = { + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, + }; + struct rte_crypto_sym_op *op; + uint32_t pkt_len; + + const uint8_t ciph_shift = 3 * en_priv->flags.is_ciph_in_bits; + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; + + op = cop->sym; + cop->raw = cop_init.raw; + op->m_src = mb; + op->m_dst = mb; + + /* Set IV */ + en_priv->iv_gen(cop, en_priv, count); + + /* Prepare op */ + pkt_len = rte_pktmbuf_pkt_len(mb); + op->cipher.data.offset = data_offset << ciph_shift; + op->cipher.data.length = (pkt_len - data_offset) << ciph_shift; + + if (is_auth) { + op->auth.data.offset = 0; + op->auth.data.length = (pkt_len - PDCP_MAC_I_LEN) << auth_shift; + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t *, + (pkt_len - PDCP_MAC_I_LEN)); + } + + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); +} + +static inline bool +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + return true; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline bool +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + return true; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count, sn; + uint8_t *mac_i; + int i; + + const uint8_t hdr_sz = en_priv->hdr_sz; + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + mac_i = (uint8_t *)rte_pktmbuf_append(mb, PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, PDCP_MAC_I_LEN); + + /* Update sequence number in the PDU header */ + count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static int +pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + entity->pre_process = NULL; + entity->post_process = NULL; + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + entity->post_process = pdcp_post_process_ul; + } + + if (entity->pre_process == NULL || entity->post_process == NULL) + return -ENOTSUP; + + return 0; +} + static int pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) { @@ -134,5 +460,9 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti if (ret) return ret; + ret = pdcp_pre_post_func_set(entity, conf); + if (ret) + return ret; + return 0; } -- 2.25.1