* [RFC 0/1] lib: add pdcp protocol
@ 2022-10-27 5:21 Anoob Joseph
2022-10-27 5:21 ` [RFC 1/1] " Anoob Joseph
` (2 more replies)
0 siblings, 3 replies; 192+ messages in thread
From: Anoob Joseph @ 2022-10-27 5:21 UTC (permalink / raw)
To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev,
Bernard Iremonger
Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K,
Volodymyr Fialko, dev
Add Packet Data Convergence Protocol (PDCP) processing library.
The library is similar to lib_ipsec which provides IPsec processing
capabilities in DPDK.
PDCP would involve roughly the following options,
1. Transfer of user plane data
2. Transfer of control plane data
3. Header compression
4. Uplink data compression
5. Ciphering and integrity protection
PDCP library provides following control path APIs that is used to
configure various PDCP entities,
1. rte_pdcp_establish()
2. rte_pdcp_suspend()
3. rte_pdcp_release()
PDCP process is split into 2 parts. One before crypto processing
(rte_pdcp_pkt_pre_process()) and one after crypto processing
(rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto
operations belonging to multiple entities, rte_pdcp_pkt_crypto_group()
is added to help grouping crypto operations belonging to same entity.
Similar to lib IPsec, lib PDCP would allow application to use same API
sequence while leveraging protocol offload features enabled by rte_security
library. Lib PDCP would internally change the handles registered for
*pre_process* and *post_process* based on features enabled in the entity.
Lib PDCP would create the required sessions on the device provided in entity to
minimize the application requirements. Also, the crypto_op allocation and free
would also be done internally by lib PDCP to allow the library to create
crypto ops as required for the input packets. For example, when control PDUs are
received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP
is expected to handle it differently.
Changes planned for next version
- Add APIs for rte_pdcp_entity_re_establish, rte_pdcp_entity_reconfigure
- Implementation for all APIs covering basic control plane & user plane packets
- Framework to handle control PDUs
- Unit test leveraging existing PDCP vectors available in test_cryptodev
- Eventdev based timer integration for t-Reordering & discardTimer
- Integration with reorder library to handle in-order delivery
Sample application sequence:
struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE];
struct rte_crypto_op *cop[MAX_BURST_SIZE];
struct rte_pdcp_group grp[MAX_BURST_SIZE];
struct rte_pdcp_entity *pdcp_entity;
int nb_max_out_mb, ret, nb_grp;
/* Create PDCP entity */
pdcp_entity = rte_pdcp_entity_establish(&conf);
/**
* Allocate buffer for holding mbufs returned during PDCP suspend,
* release & post-process APIs.
*/
/* Max packets that can be cached in entity + burst size */
nb_max_out_mb = pdcp_entity->max_pkt_cache + 1;
out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0);
if (out_mb == NULL) {
/* Handle error */
}
while (1) {
/* Receive packet and form mbuf */
/**
* Prepare packets for crypto operation. Following operations
* would be done,
*
* Transmiting entity/UL (only data PDUs):
* - Perform compression
* - Assign sequence number
* - Add PDCP header
* - Create & prepare crypto_op
* - Prepare IV for crypto operation (auth_gen, encrypt)
* - Save original PDCP SDU (during PDCP re-establishment,
* unconfirmed PDCP SDUs need to crypto processed again and
* transmitted/re-transmitted)
*
* Receiving entity/DL:
* - Any control PDUs received would be processed and
* appropriate actions taken. If data PDU, continue.
* - Determine sequence number (based on HFN & per packet SN)
* - Prepare crypto_op
* - Prepare IV for crypto operation (decrypt, auth_verify)
*/
nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop,
nb_rx, &nb_err);
if (nb_err != 0) {
/* Handle error packets */
}
if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success)
!= nb_success) {
/* Retry for enqueue failure packets */
}
...
ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop,
MAX_BURST_SIZE);
if (ret == 0)
continue;
/**
* Received a burst of completed crypto ops from cryptodev. It
* may belong to various entities. Group similar ones together
* for entity specific post-processing.
*/
/**
* Groups similar entities together. Frees crypto op and based
* on crypto_op status, set mbuf->ol_flags which would be
* checked in rte_pdcp_pkt_post_process().
*/
nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret);
for (i = 0; i != nb_grp; i++) {
/**
* Post process packets after crypto completion.
* Following operations would be done,
*
* Transmiting entity/UL:
* - Check crypto result
*
* Receiving entity/DL:
* - Check crypto operation status
* - Check for duplication (if yes, drop duplicate)
* - Perform decompression
* - Trim PDCP header
* - Hold packet (SDU) for in-order delivery (return
* completed packets as and when sequence is
* completed)
* - If not in sequence, cache the packet and start
* t-Reordering timer. When timer expires, the
* packets need to delivered to upper layers (not
* treated as error packets).
*/
nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr,
grp[i].m, out_mb,
grp[i].cnt,
&nb_err);
if (nb_err != 0) {
/* Handle error packets */
}
/* Perform additional operations */
/**
* Tranmitting entity/UL
* - If duplication is enabled, duplicate PDCP PDUs
* - When lower layers confirm reception of a PDCP PDU,
* it should be communicated to PDCP layer so that
* PDCP can drop the corresponding SDU
*/
}
}
Anoob Joseph (1):
lib: add pdcp protocol
doc/api/doxy-api-index.md | 3 +-
doc/api/doxy-api.conf.in | 1 +
lib/meson.build | 1 +
lib/pdcp/meson.build | 7 ++
lib/pdcp/rte_pdcp.h | 244 ++++++++++++++++++++++++++++++++++++++
lib/pdcp/rte_pdcp_group.h | 134 +++++++++++++++++++++
lib/pdcp/version.map | 13 ++
7 files changed, 402 insertions(+), 1 deletion(-)
create mode 100644 lib/pdcp/meson.build
create mode 100644 lib/pdcp/rte_pdcp.h
create mode 100644 lib/pdcp/rte_pdcp_group.h
create mode 100644 lib/pdcp/version.map
--
2.28.0
^ permalink raw reply [flat|nested] 192+ messages in thread* [RFC 1/1] lib: add pdcp protocol 2022-10-27 5:21 [RFC 0/1] lib: add pdcp protocol Anoob Joseph @ 2022-10-27 5:21 ` Anoob Joseph 2022-12-13 7:01 ` [RFC 0/1] " Akhil Goyal 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph 2 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2022-10-27 5:21 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_establish() 2. rte_pdcp_suspend() 3. rte_pdcp_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations beloning to same entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + lib/meson.build | 1 + lib/pdcp/meson.build | 7 ++ lib/pdcp/rte_pdcp.h | 244 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp_group.h | 134 +++++++++++++++++++++ lib/pdcp/version.map | 13 ++ 7 files changed, 402 insertions(+), 1 deletion(-) create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index de488c7abf..42bcef9a17 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -125,7 +125,8 @@ The public API headers are grouped by topics: [Geneve](@ref rte_geneve.h), [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), - [PPP](@ref rte_ppp.h) + [PPP](@ref rte_ppp.h), + [PDCP](@ref rte_pdcp.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index f0886c3bd1..01314b087e 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -61,6 +61,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/net \ @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ + @TOPDIR@/lib/pdcp \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ @TOPDIR@/lib/port \ diff --git a/lib/meson.build b/lib/meson.build index c51cdc24fa..15b7f77c68 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -63,6 +63,7 @@ libraries = [ 'flow_classify', # flow_classify lib depends on pkt framework table lib 'graph', 'node', + 'pdcp', # pdcp lib depends on crypto and security ] optional_libs = [ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode 100644 index 0000000000..4c329cf57d --- /dev/null +++ b/lib/pdcp/meson.build @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2022 Marvell. +# + +headers = files('rte_pdcp.h') + +deps += ['security'] diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode 100644 index 0000000000..3d2640240a --- /dev/null +++ b/lib/pdcp/rte_pdcp.h @@ -0,0 +1,244 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _RTE_PDCP_H_ +#define _RTE_PDCP_H_ + +/** + * @file rte_pdcp.h + * + * RTE PDCP support. + * + * librte_pdcp provides a framework for PDCP protocol processing. + */ + +#include <rte_compat.h> +#include <rte_common.h> +#include <rte_errno.h> +#include <rte_mempool.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/* Forward declarations */ + +struct rte_pdcp_entity; + +/* PDCP pre-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err); + +/* PDCP post-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err); + +/** + * PDCP entity. + */ +struct rte_pdcp_entity { + /** Entity specific pre-process handle */ + rte_pdcp_pre_p_t pre_process; + /** Entity specific post-process handle */ + rte_pdcp_post_p_t post_process; + /** + * PDCP entities may hold packets for purposes of in-order delivery (in + * case of receiving PDCP entity) and re-transmission (in case of + * transmitting PDCP entity). + * + * For receiving PDCP entity, it may hold packets when in-order + * delivery is enabled. The packets would be cached until either a + * packet that completes the sequence arrives or when discard timer + * expires. + * + * When post-processing of PDCP packet which completes a sequence is + * done, the API may return more packets than enqueued. Application is + * expected to provide @see rte_pdcp_pkt_post_process() with *out_mb* + * which can hold maximum number of packets which may be returned. + * + * For transmitting PDCP entity, during re-establishment (5.1.2), + * entity may be required to perform re-transmission of the buffers + * after applying new ciphering & integrity algorithms. For performing + * crypto operation, @see rte_pdcp_entity_re_establish() would return as + * many crypto_ops as the ones cached. + * + */ + uint16_t max_pkt_cache; + /** User area for saving application data */ + uint64_t user_area[2]; +} __rte_cache_aligned; + +struct rte_pdcp_entity_conf { + /** PDCP transform for the entity */ + struct rte_security_pdcp_xform pdcp_xfrm; + /** Crypto transform applicable for the entity */ + struct rte_crypto_sym_xform *crypto_xfrm; + /** Mempool for crypto symmetric session */ + struct rte_mempool *sess_mpool; + /** Mempool for crypto symmetric session private part */ + struct rte_mempool *sess_priv_mpool; + /** Crypto op pool*/ + struct rte_mempool *cop_pool; + /** + * 32 bit count value (HFN + SN) to be used for the first packet. + * pdcp_xfrm.hfn would be ignored as the HFN would be derived from this value. + */ + uint32_t count; + /** Indicate whether the PDCP entity belongs to Side Link Radio Bearer */ + bool is_slrb; + /** Enable security offload on the device specified */ + bool en_sec_offload; + /** Enable non-atomic usage of entity */ + bool en_non_atomic; + /** Device on which security/crypto session need to be created */ + uint8_t dev_id; +}; + +/** + * 5.1.1 PDCP entity establishment + * + * Establish PDCP entity based on provided input configuration. + * + * @param conf + * Parameters to be used for initializing PDCP entity object. + * @return + * - Valid handle if success + * - NULL in case of failure. rte_errno will be set to error code + */ +__rte_experimental +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); + +/** + * 5.1.3 PDCP entity release + * + * Release PDCP entity. + * + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be released. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * 5.1.4 PDCP entity suspend + * + * Suspend PDCP entity. + * + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be suspended. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ + +__rte_experimental +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * For input mbufs and given PDCP entity pre-process the mbufs and prepare + * crypto ops that can be enqueued to the cryptodev associated with given + * session. Only error packets would be moved returned in the input buffer, + * *mb*, and it is the responsibility of the application to free the same. + * + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param[out] mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * which contain the input packets. Any error packets would be returned in the + * same buffer. + * @param[out] cop + * The address of an array that can hold up to *num* pointers to + * *rte_crypto_op* structures. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * Pointer to return the number of error packets returned in *mb* + * @return + * Count of crypto_ops prepared + */ +static inline uint16_t +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err) +{ + return entity->pre_process(entity, mb, cop, num, nb_err); +} + +/** + * For input mbufs and given PDCP entity, perform PDCP post-processing of the + * mbufs. + * + * Input mbufs are the ones retrieved from crypto_ops dequeued from cryptodev + * and grouped by @see rte_pdcp_pkt_crypto_group(). + * + * The post-processed packets would be returned in the *out_mb* buffer. + * The resultant mbufs would be grouped into success packets and error packets. + * Error packets would be grouped in the end of the array and it is the + * responsibility of the application to handle the same. + * + * When in-order delivery is enabled, PDCP entity may buffer packets and would + * deliver packets only when all prior packets have been post-processed. That + * would result in returning more/less packets than enqueued. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param in_mb + * The address of an array of *num* pointers to *rte_mbuf* structures. + * @param[out] out_mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * to output packets after PDCP post-processing. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * The number of error packets returned in *out_mb* buffer. + * @return + * Count of packets returned in *out_mb* buffer. + */ +static inline uint16_t +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return entity->post_process(entity, in_mb, out_mb, num, nb_err); +} + +#include <rte_pdcp_group.h> + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PDCP_H_ */ diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new file mode 100644 index 0000000000..234d41e0a3 --- /dev/null +++ b/lib/pdcp/rte_pdcp_group.h @@ -0,0 +1,134 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _RTE_PDCP_GROUP_H_ +#define _RTE_PDCP_GROUP_H_ + +/** + * @file rte_pdcp_group.h + * + * RTE PDCP grouping support. + * It is not recommended to include this file directly, include <rte_pdcp.h> + * instead. + * Provides helper functions to process completed crypto-ops and group related + * packets by sessions they belong to. + */ + +#include <rte_common.h> +#include <rte_crypto.h> +#include <rte_cryptodev.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Group packets belonging to same PDCP entity. + */ +struct rte_pdcp_group { + union { + uint64_t val; + void *ptr; + } id; /**< Grouped by value */ + struct rte_mbuf **m; /**< Start of the group */ + uint32_t cnt; /**< Number of entries in the group */ + int32_t rc; /**< Status code associated with the group */ +}; + +/** + * Take crypto-op as an input and extract pointer to related PDCP entity. + * @param cop + * The address of an input *rte_crypto_op* structure. + * @return + * The pointer to the related *rte_pdcp_entity* structure. + */ +static inline struct rte_pdcp_entity * +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) +{ + void *ses; + + if (cop->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + ses = cop->sym[0].session; + return (struct rte_pdcp_entity *)(uintptr_t) + rte_security_session_opaque_data_get(ses); + } else if (cop->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + ses = cop->sym[0].session; + return (struct rte_pdcp_entity *)(uintptr_t) + rte_cryptodev_sym_session_opaque_data_get(ses); + } + return NULL; +} + +/** + * Take as input completed crypto ops, extract related mbufs and group them by + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* flag + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. + * + * Note that application must ensure only crypto-ops prepared by lib_pdcp is + * provided back to *rte_pdcp_pkt_crypto_group()*. + * + * @param cop + * The address of an array of *num* pointers to the input *rte_crypto_op* + * structures. + * @param[out] mb + * The address of an array of *num* pointers to output *rte_mbuf* structures. + * @param[out] grp + * The address of an array of *num* to output *rte_pdcp_group* structures. + * @param num + * The maximum number of crypto-ops to process. + * @return + * Number of filled elements in *grp* array. + * + */ +static inline uint16_t +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_pdcp_group grp[], uint16_t num) +{ + uint32_t i, j = 0, n = 0; + void *ns, *ps = NULL; + struct rte_mbuf *m; + + for (i = 0; i != num; i++) { + m = cop[i]->sym[0].m_src; + ns = cop[i]->sym[0].session; + + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; + + /* Different entity */ + if (ps != ns) { + + /* Finalize open group and start a new one */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + /* Start new group */ + grp[n].m = mb + j; + ps = ns; + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); + } + + mb[j++] = m; + rte_crypto_op_free(cop[i]); + } + + /* Finalize last group */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + return n; +} + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PDCP_GROUP_H_ */ \ No newline at end of file diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode 100644 index 0000000000..b62361b47e --- /dev/null +++ b/lib/pdcp/version.map @@ -0,0 +1,13 @@ +EXPERIMENTAL { + global: + + # added in 23.03 + rte_pdcp_entity_establish; + rte_pdcp_entity_release; + rte_pdcp_entity_suspend; + + rte_pdcp_pkt_post_process; + rte_pdcp_pkt_pre_process; + + local: *; +}; -- 2.28.0 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [RFC 0/1] lib: add pdcp protocol 2022-10-27 5:21 [RFC 0/1] lib: add pdcp protocol Anoob Joseph 2022-10-27 5:21 ` [RFC 1/1] " Anoob Joseph @ 2022-12-13 7:01 ` Akhil Goyal 2022-12-20 12:15 ` Anoob Joseph 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph 2 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2022-12-13 7:01 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev > Subject: [RFC 0/1] lib: add pdcp protocol > > Add Packet Data Convergence Protocol (PDCP) processing library. > > The library is similar to lib_ipsec which provides IPsec processing > capabilities in DPDK. > > PDCP would involve roughly the following options, > 1. Transfer of user plane data > 2. Transfer of control plane data > 3. Header compression > 4. Uplink data compression > 5. Ciphering and integrity protection > > PDCP library provides following control path APIs that is used to > configure various PDCP entities, > 1. rte_pdcp_establish() > 2. rte_pdcp_suspend() > 3. rte_pdcp_release() > > PDCP process is split into 2 parts. One before crypto processing > (rte_pdcp_pkt_pre_process()) and one after crypto processing > (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto > operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() > is added to help grouping crypto operations belonging to same entity. > > Similar to lib IPsec, lib PDCP would allow application to use same API > sequence while leveraging protocol offload features enabled by rte_security > library. Lib PDCP would internally change the handles registered for > *pre_process* and *post_process* based on features enabled in the entity. > > Lib PDCP would create the required sessions on the device provided in entity to > minimize the application requirements. Also, the crypto_op allocation and free > would also be done internally by lib PDCP to allow the library to create > crypto ops as required for the input packets. For example, when control PDUs > are > received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP > is expected to handle it differently. > > Changes planned for next version > - Add APIs for rte_pdcp_entity_re_establish, rte_pdcp_entity_reconfigure > - Implementation for all APIs covering basic control plane & user plane packets > - Framework to handle control PDUs > - Unit test leveraging existing PDCP vectors available in test_cryptodev > - Eventdev based timer integration for t-Reordering & discardTimer > - Integration with reorder library to handle in-order delivery > Can you send the next version rebased over v22.11? Also add documentation (pdcp.rst) file explaining about the new library. Add sample sequence also in that. Do you also plan to add, header compression? I see that it is mentioned above, But there is no API as of now? > Sample application sequence: > > struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; > struct rte_crypto_op *cop[MAX_BURST_SIZE]; > struct rte_pdcp_group grp[MAX_BURST_SIZE]; > struct rte_pdcp_entity *pdcp_entity; > int nb_max_out_mb, ret, nb_grp; > > /* Create PDCP entity */ > pdcp_entity = rte_pdcp_entity_establish(&conf); > > /** > * Allocate buffer for holding mbufs returned during PDCP suspend, > * release & post-process APIs. > */ > > /* Max packets that can be cached in entity + burst size */ > nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; > out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); > if (out_mb == NULL) { > /* Handle error */ > } > > while (1) { > /* Receive packet and form mbuf */ > > /** > * Prepare packets for crypto operation. Following operations > * would be done, > * > * Transmiting entity/UL (only data PDUs): > * - Perform compression > * - Assign sequence number > * - Add PDCP header > * - Create & prepare crypto_op > * - Prepare IV for crypto operation (auth_gen, encrypt) > * - Save original PDCP SDU (during PDCP re-establishment, > * unconfirmed PDCP SDUs need to crypto processed again > and > * transmitted/re-transmitted) > * > * Receiving entity/DL: > * - Any control PDUs received would be processed and > * appropriate actions taken. If data PDU, continue. > * - Determine sequence number (based on HFN & per packet > SN) > * - Prepare crypto_op > * - Prepare IV for crypto operation (decrypt, auth_verify) > */ > nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, > nb_rx, &nb_err); > if (nb_err != 0) { > /* Handle error packets */ > } > > if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, > nb_success) > != nb_success) { > /* Retry for enqueue failure packets */ > } > > ... > > ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, > MAX_BURST_SIZE); > if (ret == 0) > continue; > > /** > * Received a burst of completed crypto ops from cryptodev. It > * may belong to various entities. Group similar ones together > * for entity specific post-processing. > */ > > /** > * Groups similar entities together. Frees crypto op and based > * on crypto_op status, set mbuf->ol_flags which would be > * checked in rte_pdcp_pkt_post_process(). > */ > nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); > > for (i = 0; i != nb_grp; i++) { > > /** > * Post process packets after crypto completion. > * Following operations would be done, > * > * Transmiting entity/UL: > * - Check crypto result > * > * Receiving entity/DL: > * - Check crypto operation status > * - Check for duplication (if yes, drop duplicate) > * - Perform decompression > * - Trim PDCP header > * - Hold packet (SDU) for in-order delivery (return > * completed packets as and when sequence is > * completed) > * - If not in sequence, cache the packet and start > * t-Reordering timer. When timer expires, the > * packets need to delivered to upper layers (not > * treated as error packets). > */ > nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, > grp[i].m, out_mb, > grp[i].cnt, > &nb_err); > if (nb_err != 0) { > /* Handle error packets */ > } > > /* Perform additional operations */ > > /** > * Tranmitting entity/UL > * - If duplication is enabled, duplicate PDCP PDUs > * - When lower layers confirm reception of a PDCP > PDU, > * it should be communicated to PDCP layer so that > * PDCP can drop the corresponding SDU > */ > } > } > > Anoob Joseph (1): > lib: add pdcp protocol > > doc/api/doxy-api-index.md | 3 +- > doc/api/doxy-api.conf.in | 1 + > lib/meson.build | 1 + > lib/pdcp/meson.build | 7 ++ > lib/pdcp/rte_pdcp.h | 244 ++++++++++++++++++++++++++++++++++++++ > lib/pdcp/rte_pdcp_group.h | 134 +++++++++++++++++++++ > lib/pdcp/version.map | 13 ++ > 7 files changed, 402 insertions(+), 1 deletion(-) > create mode 100644 lib/pdcp/meson.build > create mode 100644 lib/pdcp/rte_pdcp.h > create mode 100644 lib/pdcp/rte_pdcp_group.h > create mode 100644 lib/pdcp/version.map > > -- > 2.28.0 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [RFC 0/1] lib: add pdcp protocol 2022-12-13 7:01 ` [RFC 0/1] " Akhil Goyal @ 2022-12-20 12:15 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2022-12-20 12:15 UTC (permalink / raw) To: Akhil Goyal Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Hi Akhil, Thanks for the review. Please see inline. Thanks Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Tuesday, December 13, 2022 12:32 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org > Subject: RE: [RFC 0/1] lib: add pdcp protocol > > > Subject: [RFC 0/1] lib: add pdcp protocol > > > > Add Packet Data Convergence Protocol (PDCP) processing library. > > > > The library is similar to lib_ipsec which provides IPsec processing > > capabilities in DPDK. > > > > PDCP would involve roughly the following options, 1. Transfer of user > > plane data 2. Transfer of control plane data 3. Header compression 4. > > Uplink data compression 5. Ciphering and integrity protection > > > > PDCP library provides following control path APIs that is used to > > configure various PDCP entities, 1. rte_pdcp_establish() 2. > > rte_pdcp_suspend() 3. rte_pdcp_release() > > > > PDCP process is split into 2 parts. One before crypto processing > > (rte_pdcp_pkt_pre_process()) and one after crypto processing > > (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return > > crypto operations belonging to multiple entities, > > rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations > belonging to same entity. > > > > Similar to lib IPsec, lib PDCP would allow application to use same API > > sequence while leveraging protocol offload features enabled by > > rte_security library. Lib PDCP would internally change the handles > > registered for > > *pre_process* and *post_process* based on features enabled in the > entity. > > > > Lib PDCP would create the required sessions on the device provided in > > entity to minimize the application requirements. Also, the crypto_op > > allocation and free would also be done internally by lib PDCP to allow > > the library to create crypto ops as required for the input packets. > > For example, when control PDUs are received, no cryptodev > > enqueue-dequeue is expected for the same and lib PDCP is expected to > > handle it differently. > > > > Changes planned for next version > > - Add APIs for rte_pdcp_entity_re_establish, > > rte_pdcp_entity_reconfigure > > - Implementation for all APIs covering basic control plane & user > > plane packets > > - Framework to handle control PDUs > > - Unit test leveraging existing PDCP vectors available in > > test_cryptodev > > - Eventdev based timer integration for t-Reordering & discardTimer > > - Integration with reorder library to handle in-order delivery > > > > Can you send the next version rebased over v22.11? > Also add documentation (pdcp.rst) file explaining about the new library. > Add sample sequence also in that. [Anoob] Sure. Will send v1 with test application in another couple of days. > > Do you also plan to add, header compression? I see that it is mentioned > above, But there is no API as of now? There are 2 kinds of compression when dealing with PDCP. 1. Header compression (which is more like a dictionary lookup) 2. UDC compression (which uses standard compression methods). [Anoob] We have plans to support Item 1, ie, header compression. For item 2, we are evaluating the practical usage requirements to decide on whether PDCP lib would need to natively support it. Adding an interface to compressdev would increase the API requirements, but since it would come outside (& before) any of the proposed APIs, the feature can be added as a standalone item. > > > > Sample application sequence: > > > > struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; > > struct rte_crypto_op *cop[MAX_BURST_SIZE]; > > struct rte_pdcp_group grp[MAX_BURST_SIZE]; > > struct rte_pdcp_entity *pdcp_entity; > > int nb_max_out_mb, ret, nb_grp; > > > > /* Create PDCP entity */ > > pdcp_entity = rte_pdcp_entity_establish(&conf); > > > > /** > > * Allocate buffer for holding mbufs returned during PDCP suspend, > > * release & post-process APIs. > > */ > > > > /* Max packets that can be cached in entity + burst size */ > > nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; > > out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); > > if (out_mb == NULL) { > > /* Handle error */ > > } > > > > while (1) { > > /* Receive packet and form mbuf */ > > > > /** > > * Prepare packets for crypto operation. Following operations > > * would be done, > > * > > * Transmiting entity/UL (only data PDUs): > > * - Perform compression > > * - Assign sequence number > > * - Add PDCP header > > * - Create & prepare crypto_op > > * - Prepare IV for crypto operation (auth_gen, encrypt) > > * - Save original PDCP SDU (during PDCP re-establishment, > > * unconfirmed PDCP SDUs need to crypto processed again > > and > > * transmitted/re-transmitted) > > * > > * Receiving entity/DL: > > * - Any control PDUs received would be processed and > > * appropriate actions taken. If data PDU, continue. > > * - Determine sequence number (based on HFN & per > packet > > SN) > > * - Prepare crypto_op > > * - Prepare IV for crypto operation (decrypt, auth_verify) > > */ > > nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, > cop, > > nb_rx, &nb_err); > > if (nb_err != 0) { > > /* Handle error packets */ > > } > > > > if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, > > nb_success) > > != nb_success) { > > /* Retry for enqueue failure packets */ > > } > > > > ... > > > > ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, > > MAX_BURST_SIZE); > > if (ret == 0) > > continue; > > > > /** > > * Received a burst of completed crypto ops from cryptodev. > It > > * may belong to various entities. Group similar ones > together > > * for entity specific post-processing. > > */ > > > > /** > > * Groups similar entities together. Frees crypto op and > based > > * on crypto_op status, set mbuf->ol_flags which would be > > * checked in rte_pdcp_pkt_post_process(). > > */ > > nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); > > > > for (i = 0; i != nb_grp; i++) { > > > > /** > > * Post process packets after crypto completion. > > * Following operations would be done, > > * > > * Transmiting entity/UL: > > * - Check crypto result > > * > > * Receiving entity/DL: > > * - Check crypto operation status > > * - Check for duplication (if yes, drop duplicate) > > * - Perform decompression > > * - Trim PDCP header > > * - Hold packet (SDU) for in-order delivery (return > > * completed packets as and when sequence is > > * completed) > > * - If not in sequence, cache the packet and start > > * t-Reordering timer. When timer expires, the > > * packets need to delivered to upper layers (not > > * treated as error packets). > > */ > > nb_success = > rte_pdcp_pkt_post_process(grp[i].id.ptr, > > grp[i].m, out_mb, > > grp[i].cnt, > > &nb_err); > > if (nb_err != 0) { > > /* Handle error packets */ > > } > > > > /* Perform additional operations */ > > > > /** > > * Tranmitting entity/UL > > * - If duplication is enabled, duplicate PDCP PDUs > > * - When lower layers confirm reception of a PDCP > PDU, > > * it should be communicated to PDCP layer so that > > * PDCP can drop the corresponding SDU > > */ > > } > > } > > > > Anoob Joseph (1): > > lib: add pdcp protocol > > > > doc/api/doxy-api-index.md | 3 +- > > doc/api/doxy-api.conf.in | 1 + > > lib/meson.build | 1 + > > lib/pdcp/meson.build | 7 ++ > > lib/pdcp/rte_pdcp.h | 244 > ++++++++++++++++++++++++++++++++++++++ > > lib/pdcp/rte_pdcp_group.h | 134 +++++++++++++++++++++ > > lib/pdcp/version.map | 13 ++ > > 7 files changed, 402 insertions(+), 1 deletion(-) create mode 100644 > > lib/pdcp/meson.build create mode 100644 lib/pdcp/rte_pdcp.h create > > mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 > > lib/pdcp/version.map > > > > -- > > 2.28.0 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH 0/5] lib: add pdcp protocol 2022-10-27 5:21 [RFC 0/1] lib: add pdcp protocol Anoob Joseph 2022-10-27 5:21 ` [RFC 1/1] " Anoob Joseph 2022-12-13 7:01 ` [RFC 0/1] " Akhil Goyal @ 2022-12-22 9:25 ` Anoob Joseph 2022-12-22 9:25 ` [PATCH 1/5] net: add PDCP header Anoob Joseph ` (6 more replies) 2 siblings, 7 replies; 192+ messages in thread From: Anoob Joseph @ 2022-12-22 9:25 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Similar to lib IPsec, lib PDCP would allow application to use same API sequence while leveraging protocol offload features enabled by rte_security library. Lib PDCP would internally change the handles registered for *pre_process* and *post_process* based on features enabled in the entity. Lib PDCP would create the required sessions on the device provided in entity to minimize the application requirements. Also, the crypto_op allocation and free would also be done internally by lib PDCP to allow the library to create crypto ops as required for the input packets. For example, when control PDUs are received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP is expected to handle it differently. Changes from RFC - Implementation for all APIs covering basic control plane & user plane packets - Unit test leveraging existing PDCP vectors available in test_cryptodev - Unit test performing both UL & DL operations to verify various protocol features - Updated documentation Changes planned for next version - Add APIs for rte_pdcp_entity_re_establish, rte_pdcp_entity_reconfigure - Framework to handle control PDUs - Eventdev based timer integration for t-Reordering & discardTimer - Integration with reorder library to handle in-order delivery Sample application sequence: struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; struct rte_crypto_op *cop[MAX_BURST_SIZE]; struct rte_pdcp_group grp[MAX_BURST_SIZE]; struct rte_pdcp_entity *pdcp_entity; int nb_max_out_mb, ret, nb_grp; /* Create PDCP entity */ pdcp_entity = rte_pdcp_entity_establish(&conf); /** * Allocate buffer for holding mbufs returned during PDCP suspend, * release & post-process APIs. */ /* Max packets that can be cached in entity + burst size */ nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); if (out_mb == NULL) { /* Handle error */ } while (1) { /* Receive packet and form mbuf */ /** * Prepare packets for crypto operation. Following operations * would be done, * * Transmitting entity/UL (only data PDUs): * - Perform compression * - Assign sequence number * - Add PDCP header * - Create & prepare crypto_op * - Prepare IV for crypto operation (auth_gen, encrypt) * - Save original PDCP SDU (during PDCP re-establishment, * unconfirmed PDCP SDUs need to crypto processed again and * transmitted/re-transmitted) * * Receiving entity/DL: * - Any control PDUs received would be processed and * appropriate actions taken. If data PDU, continue. * - Determine sequence number (based on HFN & per packet SN) * - Prepare crypto_op * - Prepare IV for crypto operation (decrypt, auth_verify) */ nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, nb_rx, &nb_err); if (nb_err != 0) { /* Handle error packets */ } if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) != nb_success) { /* Retry for enqueue failure packets */ } ... ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, MAX_BURST_SIZE); if (ret == 0) continue; /** * Received a burst of completed crypto ops from cryptodev. It * may belong to various entities. Group similar ones together * for entity specific post-processing. */ /** * Groups similar entities together. Frees crypto op and based * on crypto_op status, set mbuf->ol_flags which would be * checked in rte_pdcp_pkt_post_process(). */ nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); for (i = 0; i != nb_grp; i++) { /** * Post process packets after crypto completion. * Following operations would be done, * * Transmitting entity/UL: * - Check crypto result * * Receiving entity/DL: * - Check crypto operation status * - Check for duplication (if yes, drop duplicate) * - Perform decompression * - Trim PDCP header * - Hold packet (SDU) for in-order delivery (return * completed packets as and when sequence is * completed) * - If not in sequence, cache the packet and start * t-Reordering timer. When timer expires, the * packets need to delivered to upper layers (not * treated as error packets). */ nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, grp[i].m, out_mb, grp[i].cnt, &nb_err); if (nb_err != 0) { /* Handle error packets */ } /* Perform additional operations */ /** * Tranmitting entity/UL * - If duplication is enabled, duplicate PDCP PDUs * - When lower layers confirm reception of a PDCP PDU, * it should be communicated to PDCP layer so that * PDCP can drop the corresponding SDU */ } } Anoob Joseph (3): lib: add pdcp protocol app/test: add lib pdcp tests doc: add PDCP library guide Volodymyr Fialko (2): net: add PDCP header app/test: pdcp HFN tests in combined mode app/test/meson.build | 1 + app/test/test_cryptodev.h | 3 + app/test/test_pdcp.c | 999 ++++++++++++++ doc/api/doxy-api-index.md | 4 +- doc/api/doxy-api.conf.in | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 211 +++ lib/meson.build | 1 + lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 93 ++ lib/pdcp/meson.build | 8 + lib/pdcp/pdcp_crypto.c | 240 ++++ lib/pdcp/pdcp_crypto.h | 20 + lib/pdcp/pdcp_entity.h | 218 +++ lib/pdcp/pdcp_process.c | 1195 +++++++++++++++++ lib/pdcp/pdcp_process.h | 13 + lib/pdcp/rte_pdcp.c | 136 ++ lib/pdcp/rte_pdcp.h | 263 ++++ lib/pdcp/rte_pdcp_group.h | 133 ++ lib/pdcp/version.map | 13 + 21 files changed, 3554 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pdcp.c create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst create mode 100644 lib/net/rte_pdcp_hdr.h create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH 1/5] net: add PDCP header 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph @ 2022-12-22 9:25 ` Anoob Joseph 2023-01-18 16:36 ` Thomas Monjalon 2022-12-22 9:25 ` [PATCH 2/5] lib: add pdcp protocol Anoob Joseph ` (5 subsequent siblings) 6 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2022-12-22 9:25 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger, Olivier Matz Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev From: Volodymyr Fialko <vfialko@marvell.com> Added PDCP protocol header to be used for supporting PDCP protocol processing. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/api/doxy-api-index.md | 3 +- lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 93 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 96 insertions(+), 1 deletion(-) create mode 100644 lib/net/rte_pdcp_hdr.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index de488c7abf..ae4b107240 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -125,7 +125,8 @@ The public API headers are grouped by topics: [Geneve](@ref rte_geneve.h), [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), - [PPP](@ref rte_ppp.h) + [PPP](@ref rte_ppp.h), + [PDCP hdr](@ref rte_pdcp_hdr.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/lib/net/meson.build b/lib/net/meson.build index 379d161ee0..bd56f91c22 100644 --- a/lib/net/meson.build +++ b/lib/net/meson.build @@ -22,6 +22,7 @@ headers = files( 'rte_geneve.h', 'rte_l2tpv2.h', 'rte_ppp.h', + 'rte_pdcp_hdr.h', ) sources = files( diff --git a/lib/net/rte_pdcp_hdr.h b/lib/net/rte_pdcp_hdr.h new file mode 100644 index 0000000000..f9b8258949 --- /dev/null +++ b/lib/net/rte_pdcp_hdr.h @@ -0,0 +1,93 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _RTE_PDCP_HDR_H_ +#define _RTE_PDCP_HDR_H_ + +/** + * @file + * + * PDCP-related defines + * + * Based on - ETSI TS 138 323 V17.1.0 (2022-08) + * https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + */ + +#include <rte_byteorder.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * 6.2.2.1 Data PDU for SRBs + */ +__extension__ +struct rte_pdcp_cp_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 4; /**< Reserved */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.2.2 Data PDU for DRBs and MRBs with 12 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 3; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 3; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.2.3 Data PDU for DRBs and MRBs with 18 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_18_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ + uint8_t r : 5; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 5; /**< Reserved */ + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ +#endif + uint8_t sn_15_8; /**< Sequence number bits 8-15 */ + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.3.1 Control PDU for PDCP status report + */ +__extension__ +struct rte_pdcp_up_ctrl_pdu_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t r : 4; /**< Reserved */ +#endif +}; + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PDCP_HDR_H_ */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [PATCH 1/5] net: add PDCP header 2022-12-22 9:25 ` [PATCH 1/5] net: add PDCP header Anoob Joseph @ 2023-01-18 16:36 ` Thomas Monjalon 2023-01-18 17:39 ` [EXT] " Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Thomas Monjalon @ 2023-01-18 16:36 UTC (permalink / raw) To: Anoob Joseph Cc: Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Olivier Matz, dev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev 22/12/2022 10:25, Anoob Joseph: > --- /dev/null > +++ b/lib/net/rte_pdcp_hdr.h > @@ -0,0 +1,93 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#ifndef _RTE_PDCP_HDR_H_ > +#define _RTE_PDCP_HDR_H_ No need of extra underscores before and after. Sorry I cannot review the rest of this, it is out of my knowledge :) ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [EXT] Re: [PATCH 1/5] net: add PDCP header 2023-01-18 16:36 ` Thomas Monjalon @ 2023-01-18 17:39 ` Anoob Joseph 2023-01-19 8:05 ` Thomas Monjalon 0 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-01-18 17:39 UTC (permalink / raw) To: Thomas Monjalon Cc: Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Olivier Matz, dev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev Hi Thomas, Please see inline. Thanks, Anoob > Subject: [EXT] Re: [PATCH 1/5] net: add PDCP header > > External Email > > ---------------------------------------------------------------------- > 22/12/2022 10:25, Anoob Joseph: > > --- /dev/null > > +++ b/lib/net/rte_pdcp_hdr.h > > @@ -0,0 +1,93 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#ifndef _RTE_PDCP_HDR_H_ > > +#define _RTE_PDCP_HDR_H_ > > No need of extra underscores before and after. [Anoob] I was following the example quoted in https://doc.dpdk.org/guides/contributing/coding_style.html. Also, other files such as rte_esp.h, rte_udp.h, etc. follow the same. Isn't it better to have a uniform coding style? ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [EXT] Re: [PATCH 1/5] net: add PDCP header 2023-01-18 17:39 ` [EXT] " Anoob Joseph @ 2023-01-19 8:05 ` Thomas Monjalon 2023-01-23 9:21 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Thomas Monjalon @ 2023-01-19 8:05 UTC (permalink / raw) To: Anoob Joseph Cc: Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Olivier Matz, dev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev 18/01/2023 18:39, Anoob Joseph: > > 22/12/2022 10:25, Anoob Joseph: > > > --- /dev/null > > > +++ b/lib/net/rte_pdcp_hdr.h > > > @@ -0,0 +1,93 @@ > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > + * Copyright(C) 2022 Marvell. > > > + */ > > > + > > > +#ifndef _RTE_PDCP_HDR_H_ > > > +#define _RTE_PDCP_HDR_H_ > > > > No need of extra underscores before and after. > > [Anoob] I was following the example quoted in https://doc.dpdk.org/guides/contributing/coding_style.html. Oh thanks for the reference, I will fix it. > Also, other files such as rte_esp.h, rte_udp.h, etc. follow the same. Some other files don't have underscores. > Isn't it better to have a uniform coding style? No really I prefer no underscores, they are supposed to be used for standard libraries. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [EXT] Re: [PATCH 1/5] net: add PDCP header 2023-01-19 8:05 ` Thomas Monjalon @ 2023-01-23 9:21 ` Anoob Joseph 2023-01-23 15:31 ` Thomas Monjalon 0 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-01-23 9:21 UTC (permalink / raw) To: Thomas Monjalon Cc: Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Olivier Matz, dev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev Hi Thomas, Please see inline. Thanks, Anoob > Subject: Re: [EXT] Re: [PATCH 1/5] net: add PDCP header > > 18/01/2023 18:39, Anoob Joseph: > > > 22/12/2022 10:25, Anoob Joseph: > > > > --- /dev/null > > > > +++ b/lib/net/rte_pdcp_hdr.h > > > > @@ -0,0 +1,93 @@ > > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > > + * Copyright(C) 2022 Marvell. > > > > + */ > > > > + > > > > +#ifndef _RTE_PDCP_HDR_H_ > > > > +#define _RTE_PDCP_HDR_H_ > > > > > > No need of extra underscores before and after. > > > > [Anoob] I was following the example quoted in > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__doc.dpdk.org_guides_contributing_coding- > 5Fstyle.html&d=DwICAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=jPfB8rwwviRSxyL > Ws2n6B-WYLn1v9SyTMrT5EQqh2TU&m=Btr0wWqHo_dHGchyRWTnjO6xY- > 7p33pUSKSBLIXtL4DOAv-GsZmL2lZ2OHyJWoiZ&s=- > AQhSWF0bWFXTyxL0rPCW6fz6I7GYhwyQ9qjYG3FFn0&e= . > > Oh thanks for the reference, I will fix it. > > > Also, other files such as rte_esp.h, rte_udp.h, etc. follow the same. > > Some other files don't have underscores. > > > Isn't it better to have a uniform coding style? > > No really I prefer no underscores, > they are supposed to be used for standard libraries. [Anoob] I see that most of the files do have underscores. While I do not have any personal preference, I would really prefer new code to not stand out from the rest. If you have considered this already, then I'll make the change in next version. Please confirm. ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [EXT] Re: [PATCH 1/5] net: add PDCP header 2023-01-23 9:21 ` Anoob Joseph @ 2023-01-23 15:31 ` Thomas Monjalon 0 siblings, 0 replies; 192+ messages in thread From: Thomas Monjalon @ 2023-01-23 15:31 UTC (permalink / raw) To: Anoob Joseph Cc: Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Olivier Matz, dev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev 23/01/2023 10:21, Anoob Joseph: > Hi Thomas, > > Please see inline. > > Thanks, > Anoob > > > Subject: Re: [EXT] Re: [PATCH 1/5] net: add PDCP header > > > > 18/01/2023 18:39, Anoob Joseph: > > > > 22/12/2022 10:25, Anoob Joseph: > > > > > --- /dev/null > > > > > +++ b/lib/net/rte_pdcp_hdr.h > > > > > @@ -0,0 +1,93 @@ > > > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > > > + * Copyright(C) 2022 Marvell. > > > > > + */ > > > > > + > > > > > +#ifndef _RTE_PDCP_HDR_H_ > > > > > +#define _RTE_PDCP_HDR_H_ > > > > > > > > No need of extra underscores before and after. > > > > > > [Anoob] I was following the example quoted in > > https://urldefense.proofpoint.com/v2/url?u=https- > > 3A__doc.dpdk.org_guides_contributing_coding- > > 5Fstyle.html&d=DwICAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=jPfB8rwwviRSxyL > > Ws2n6B-WYLn1v9SyTMrT5EQqh2TU&m=Btr0wWqHo_dHGchyRWTnjO6xY- > > 7p33pUSKSBLIXtL4DOAv-GsZmL2lZ2OHyJWoiZ&s=- > > AQhSWF0bWFXTyxL0rPCW6fz6I7GYhwyQ9qjYG3FFn0&e= . > > > > Oh thanks for the reference, I will fix it. > > > > > Also, other files such as rte_esp.h, rte_udp.h, etc. follow the same. > > > > Some other files don't have underscores. > > > > > Isn't it better to have a uniform coding style? > > > > No really I prefer no underscores, > > they are supposed to be used for standard libraries. > > [Anoob] I see that most of the files do have underscores. While I do not have any personal preference, I would really prefer new code to not stand out from the rest. If you have considered this already, then I'll make the change in next version. Please confirm. I prefer no underscore, and I will probably propose to change all files while updating the contributing guide. For new patch, I gave my opinion, now you can choose it is not a big deal. ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH 2/5] lib: add pdcp protocol 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph 2022-12-22 9:25 ` [PATCH 1/5] net: add PDCP header Anoob Joseph @ 2022-12-22 9:25 ` Anoob Joseph 2023-01-18 16:26 ` Akhil Goyal 2022-12-22 9:25 ` [PATCH 3/5] app/test: add lib pdcp tests Anoob Joseph ` (4 subsequent siblings) 6 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2022-12-22 9:25 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + lib/meson.build | 1 + lib/pdcp/meson.build | 8 + lib/pdcp/pdcp_crypto.c | 240 ++++++++ lib/pdcp/pdcp_crypto.h | 20 + lib/pdcp/pdcp_entity.h | 218 +++++++ lib/pdcp/pdcp_process.c | 1195 +++++++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 13 + lib/pdcp/rte_pdcp.c | 136 +++++ lib/pdcp/rte_pdcp.h | 263 ++++++++ lib/pdcp/rte_pdcp_group.h | 133 +++++ lib/pdcp/version.map | 13 + 13 files changed, 2243 insertions(+), 1 deletion(-) create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index ae4b107240..6014bee079 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -126,7 +126,8 @@ The public API headers are grouped by topics: [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), [PPP](@ref rte_ppp.h), - [PDCP hdr](@ref rte_pdcp_hdr.h) + [PDCP hdr](@ref rte_pdcp_hdr.h), + [PDCP](@ref rte_pdcp.h), - **QoS**: [metering](@ref rte_meter.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index f0886c3bd1..01314b087e 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -61,6 +61,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/net \ @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ + @TOPDIR@/lib/pdcp \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ @TOPDIR@/lib/port \ diff --git a/lib/meson.build b/lib/meson.build index fd55925340..a827006d29 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -63,6 +63,7 @@ libraries = [ 'flow_classify', # flow_classify lib depends on pkt framework table lib 'graph', 'node', + 'pdcp', # pdcp lib depends on crypto and security ] optional_libs = [ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode 100644 index 0000000000..a7f5a408cf --- /dev/null +++ b/lib/pdcp/meson.build @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2022 Marvell. +# + +sources = files('pdcp_crypto.c', 'pdcp_process.c', 'rte_pdcp.c') +headers = files('rte_pdcp.h') + +deps += ['security'] diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c new file mode 100644 index 0000000000..7ffb8a07a7 --- /dev/null +++ b/lib/pdcp/pdcp_crypto.c @@ -0,0 +1,240 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_pdcp.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" + +static int +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, + const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm, + bool is_auth_then_cipher) +{ + uint16_t ciph_iv_len, auth_digest_len, auth_iv_len; + int ret; + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protection is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (c_xfrm == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + + /* With UPLINK, if auth is enabled, it should be before cipher */ + if (a_xfrm != NULL && !is_auth_then_cipher) + return -EINVAL; + + /* With UPLINK, cipher operation must be encrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return -EINVAL; + + /* With UPLINK, auth operation (if present) must be generate */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_GENERATE) + return -EINVAL; + + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + + /* With DOWNLINK, if auth is enabled, it should be after cipher */ + if (a_xfrm != NULL && is_auth_then_cipher) + return -EINVAL; + + /* With DOWNLINK, cipher operation must be decrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) + return -EINVAL; + + /* With DOWNLINK, auth operation (if present) must be verify */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_VERIFY) + return -EINVAL; + + } else { + return -EINVAL; + } + + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) + return -EINVAL; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + ciph_iv_len = 0; + else + ciph_iv_len = PDCP_IV_LENGTH; + + if (ciph_iv_len != c_xfrm->cipher.iv.length) + return -EINVAL; + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + return -EINVAL; + + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + auth_digest_len = 0; + else + auth_digest_len = 4; + + if (auth_digest_len != a_xfrm->auth.digest_length) + return -EINVAL; + + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + auth_iv_len = PDCP_IV_LENGTH; + else + auth_iv_len = 0; + + if (a_xfrm->auth.iv.length != auth_iv_len) + return -EINVAL; + } + + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) + return -EINVAL; + + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); + if (ret) + return -ENOTSUP; + + if (a_xfrm != NULL) { + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); + if (ret) + return -ENOTSUP; + } + + return 0; +} + +int +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + struct entity_priv *en_priv; + bool is_auth_then_cipher; + int ret; + + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + en_priv->dev_id = conf->dev_id; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + c_xfrm = conf->crypto_xfrm; + a_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = false; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + a_xfrm = conf->crypto_xfrm; + c_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = true; + } else { + return -EINVAL; + } + + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, is_auth_then_cipher); + if (ret) + return ret; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + c_xfrm->cipher.iv.offset = 0; + else + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; + + if (a_xfrm != NULL) { + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm->auth.iv.offset = 0; + else + if (c_xfrm->cipher.iv.offset) + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + PDCP_IV_LENGTH; + else + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; + } + + if (conf->sess_mpool == NULL) + return -EINVAL; + + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf->dev_id, conf->crypto_xfrm, + conf->sess_mpool); + if (en_priv->crypto_sess == NULL) { + /* API returns positive values as error codes */ + return -rte_errno; + } + + rte_cryptodev_sym_session_opaque_data_set(en_priv->crypto_sess, (uint64_t)entity); + + return 0; +} + +int +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) +{ + struct entity_priv *en_priv; + + if (entity == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + if (en_priv->crypto_sess != NULL) { + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv->crypto_sess); + en_priv->crypto_sess = NULL; + } + + return 0; +} diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h new file mode 100644 index 0000000000..dc625b35d0 --- /dev/null +++ b/lib/pdcp/pdcp_crypto.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _PDCP_CRYPTO_H_ +#define _PDCP_CRYPTO_H_ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_pdcp.h> + +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op)) +#define PDCP_IV_LENGTH 16 + +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, + const struct rte_pdcp_entity_conf *conf); + +int pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); + +#endif /* _PDCP_CRYPTO_H_ */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h new file mode 100644 index 0000000000..e312fd4a8c --- /dev/null +++ b/lib/pdcp/pdcp_entity.h @@ -0,0 +1,218 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _PDCP_ENTITY_H_ +#define _PDCP_ENTITY_H_ + +#include <rte_common.h> +#include <rte_crypto_sym.h> +#include <rte_mempool.h> +#include <rte_pdcp.h> +#include <rte_security.h> + +struct entity_priv; + +#define PDCP_PDU_HDR_SIZE_SN_12 (RTE_ALIGN_MUL_CEIL(12, 8) / 8) +#define PDCP_PDU_HDR_SIZE_SN_18 (RTE_ALIGN_MUL_CEIL(18, 8) / 8) + +#define PDCP_GET_SN_12_FROM_COUNT(c) ((c) & 0xfff) +#define PDCP_GET_SN_18_FROM_COUNT(c) ((c) & 0x3ffff) + +#define PDCP_GET_HFN_SN_12_FROM_COUNT(c) (((c) >> 12) & 0xfffff) +#define PDCP_GET_HFN_SN_18_FROM_COUNT(c) (((c) >> 18) & 0x3fff) + +#define PDCP_SET_COUNT_FROM_HFN_SN_12(h, s) ((((h) & 0xfffff) << 12) | ((s) & 0xfff)) +#define PDCP_SET_COUNT_FROM_HFN_SN_18(h, s) ((((h) & 0x3fff) << 18) | ((s) & 0x3ffff)) + +#define PDCP_SN_12_WINDOW_SZ 0x800 +#define PDCP_SN_18_WINDOW_SZ 0x20000 + +#define PDCP_SN_12_HFN_MAX ((1 << (32 - 12)) - 1) +#define PDCP_SN_12_HFN_MIN 0 +#define PDCP_SN_18_HFN_MAX ((1 << (32 - 18)) - 1) +#define PDCP_SN_18_HFN_MIN 0 + +/* IV generation function based on the entity configuration */ +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count); + +enum pdcp_pdu_type { + PDCP_PDU_TYPE_CTRL = 0, + PDCP_PDU_TYPE_DATA = 1, +}; + +enum pdcp_up_ctrl_pdu_type { + PDCP_UP_CTRL_PDU_TYPE_STATUS_REPORT, + PDCP_UP_CTRL_PDU_TYPE_ROHC_FEEDBACK, + PDCP_UP_CTRL_PDU_TYPE_EHC_FEEDBACK, + PDCP_UP_CRTL_PDU_TYPE_UDC_FEEDBACK +}; + +struct entity_state { + uint32_t rx_next; + uint32_t tx_next; + uint32_t rx_deliv; + uint32_t rx_reord; +}; + +union auth_iv_partial { + /* For AES-CMAC, there is no IV, but message gets prepended */ + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + } aes_cmac; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_37_39 : 3; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_65_71 : 7; + uint64_t direction_64 : 1; + uint64_t rsvd_72_111 : 40; + uint64_t rsvd_113_119 : 7; + uint64_t direction_112 : 1; + uint64_t rsvd_120_127 : 8; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t zero_37_39 : 3; + uint64_t zero_40_63 : 24; + + uint64_t direction_64 : 1; + uint64_t rsvd_65_71 : 7; + uint64_t rsvd_72_111 : 40; + uint64_t direction_112 : 1; + uint64_t rsvd_113_119 : 7; + uint64_t rsvd_120_127 : 8; +#endif + } zs; + uint64_t u64[2]; +}; + +union cipher_iv_partial { + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t zero_64_127; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; + + uint64_t zero_64_127; +#endif + } aes_ctr; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_64_127; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_64_127; +#endif + } zs; + uint64_t u64[2]; +}; + +/* + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + */ + +struct entity_priv { + /** Crypto sym session. */ + struct rte_cryptodev_sym_session *crypto_sess; + /** Entity specific IV generation function. */ + iv_gen_t iv_gen; + /** Pre-prepared auth IV. */ + union auth_iv_partial auth_iv_part; + /** Pre-prepared cipher IV. */ + union cipher_iv_partial cipher_iv_part; + /** Entity state variables. */ + struct entity_state state; + /** Flags. */ + struct { + /** PDCP PDU has 4 byte MAC-I. */ + uint64_t is_authenticated : 1; + /** Cipher offset & length in bits. */ + uint64_t is_ciph_in_bits : 1; + /** Auth offset & length in bits. */ + uint64_t is_auth_in_bits : 1; + /** Is UL/transmitting PDCP entity */ + uint64_t is_ul_entity : 1; + } flags; + /** Crypto op pool. */ + struct rte_mempool *cop_pool; + /** PDCP header size. */ + uint8_t hdr_sz; + /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ + uint8_t aad_sz; + /** Device ID of the device to be used for offload. */ + uint8_t dev_id; +}; + +struct entity_priv_dl_part { + /* TODO - when in-order-delivery is supported, post PDCP packets would need to cached. */ + uint8_t dummy; +}; + +struct entity_priv_ul_part { + /* + * TODO - when re-establish is supported, both plain & post PDCP packets would need to be + * cached. + */ + uint8_t dummy; +}; + +static inline struct entity_priv * +entity_priv_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); +} + +static inline struct entity_priv_dl_part * +entity_dl_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline struct entity_priv_ul_part * +entity_ul_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +#endif /* _PDCP_ENTITY_H_ */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c new file mode 100644 index 0000000000..282cf38ec4 --- /dev/null +++ b/lib/pdcp/pdcp_process.c @@ -0,0 +1,1195 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_memcpy.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +#define PDCP_MAC_I_LEN 4 + +/* Enum of supported algorithms for ciphering */ +enum pdcp_cipher_algo { + PDCP_CIPHER_ALGO_NULL, + PDCP_CIPHER_ALGO_AES, + PDCP_CIPHER_ALGO_ZUC, + PDCP_CIPHER_ALGO_SNOW3G, + PDCP_CIPHER_ALGO_MAX +}; + +/* Enum of supported algorithms for integrity */ +enum pdcp_auth_algo { + PDCP_AUTH_ALGO_NULL, + PDCP_AUTH_ALGO_AES, + PDCP_AUTH_ALGO_ZUC, + PDCP_AUTH_ALGO_SNOW3G, + PDCP_AUTH_ALGO_MAX +}; + +/* IV generation functions based on type of operation (cipher - auth) */ + +static void +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + /* No IV required for NULL cipher + NULL auth */ + RTE_SET_USED(cop); + RTE_SET_USED(en_priv); + RTE_SET_USED(count); +} + +static void +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr; + uint64_t m; + + /* AES-CMAC requires message to be prepended with info on count etc */ + + /* Prepend by 8 bytes to add custom message */ + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, 16); +} + +static void +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64 = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64, 8); + rte_memcpy(iv + 8, &iv_u64, 8); +} + +static void +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + /* Generating cipher IV */ + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + iv += PDCP_IV_LENGTH; + + /* Generating auth IV */ + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LENGTH); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LENGTH); + + iv += PDCP_IV_LENGTH; + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static int +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, + struct rte_crypto_sym_xform **a_xfrm) +{ + *c_xfrm = NULL; + *a_xfrm = NULL; + + if (conf->crypto_xfrm == NULL) + return -EINVAL; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + *c_xfrm = conf->crypto_xfrm; + *a_xfrm = conf->crypto_xfrm->next; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + *a_xfrm = conf->crypto_xfrm; + *c_xfrm = conf->crypto_xfrm->next; + } else { + return -EINVAL; + } + + return 0; +} + +static int +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + enum rte_security_pdcp_direction direction; + enum pdcp_cipher_algo ciph_algo; + enum pdcp_auth_algo auth_algo; + struct entity_priv *en_priv; + int ret; + + en_priv = entity_priv_get(entity); + + direction = conf->pdcp_xfrm.pkt_dir; + if (conf->reverse_iv_direction) + direction = !direction; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (c_xfrm == NULL) + return -EINVAL; + + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); + + switch (c_xfrm->cipher.algo) { + case RTE_CRYPTO_CIPHER_NULL: + ciph_algo = PDCP_CIPHER_ALGO_NULL; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + ciph_algo = PDCP_CIPHER_ALGO_AES; + en_priv->cipher_iv_part.aes_ctr.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.aes_ctr.direction = direction; + break; + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + ciph_algo = PDCP_CIPHER_ALGO_SNOW3G; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + ciph_algo = PDCP_CIPHER_ALGO_ZUC; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + default: + return -ENOTSUP; + } + + if (a_xfrm != NULL) { + switch (a_xfrm->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + auth_algo = PDCP_AUTH_ALGO_NULL; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + auth_algo = PDCP_AUTH_ALGO_AES; + en_priv->auth_iv_part.aes_cmac.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.aes_cmac.direction = direction; + break; + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + auth_algo = PDCP_AUTH_ALGO_SNOW3G; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + case RTE_CRYPTO_AUTH_ZUC_EIA3: + auth_algo = PDCP_AUTH_ALGO_ZUC; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + default: + return -ENOTSUP; + } + } else { + auth_algo = PDCP_AUTH_ALGO_NULL; + } + + static const iv_gen_t iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_null_null, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_null_aes_cmac, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_null_zs, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_null_zs, + + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_aes_ctr_null, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_aes_ctr_aes_cmac, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_aes_ctr_zs, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_aes_ctr_zs, + + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + }; + + en_priv->iv_gen = iv_gen_map[ciph_algo][auth_algo]; + + return 0; +} + +static inline void +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, + uint8_t data_offset, uint32_t count, const bool is_auth) +{ + const struct rte_crypto_op cop_init = { + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, + }; + struct rte_crypto_sym_op *op; + uint32_t pkt_len; + + const uint8_t ciph_shift = 3 * en_priv->flags.is_ciph_in_bits; + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; + + op = cop->sym; + cop->raw = cop_init.raw; + op->m_src = mb; + op->m_dst = mb; + + /* Set IV */ + en_priv->iv_gen(cop, en_priv, count); + + /* Prepare op */ + pkt_len = rte_pktmbuf_pkt_len(mb); + op->cipher.data.offset = data_offset << ciph_shift; + op->cipher.data.length = (pkt_len - data_offset) << ciph_shift; + + if (is_auth) { + op->auth.data.offset = 0; + op->auth.data.length = (pkt_len - PDCP_MAC_I_LEN) << auth_shift; + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t *, + (pkt_len - PDCP_MAC_I_LEN)); + } + + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); +} + +static inline bool +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = __atomic_fetch_add(&en_priv->state.tx_next, 1, __ATOMIC_RELAXED); + sn = PDCP_GET_SN_12_FROM_COUNT(*count); + + pdu_hdr->d_c = PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + return true; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + if (unlikely(rte_pktmbuf_append(mb[i], PDCP_MAC_I_LEN) == NULL)) + goto cop_free; + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb[i], + &count))) + goto cop_free; + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb[i], + &count))) + goto cop_free; + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, false); + } + } + + *nb_err = num - nb_cop; + return nb_cop; +cop_free: + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop - i); + *nb_err = num - i; + return i; +} + +static inline bool +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = __atomic_fetch_add(&en_priv->state.tx_next, 1, __ATOMIC_RELAXED); + sn = PDCP_GET_SN_18_FROM_COUNT(*count); + + pdu_hdr->d_c = PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + return true; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + if (unlikely(rte_pktmbuf_append(mb[i], PDCP_MAC_I_LEN) == NULL)) + goto cop_free; + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb[i], + &count))) + goto cop_free; + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb[i], + &count))) + goto cop_free; + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, false); + } + } + + *nb_err = num - nb_cop; + return nb_cop; + +cop_free: + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop - i); + *nb_err = num - i; + return i; +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint32_t count, sn; + uint16_t nb_cop; + int i; + + const uint8_t hdr_sz = en_priv->hdr_sz; + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb[i], + hdr_sz); + if (unlikely(pdu_hdr == NULL)) + goto cop_free; + if (unlikely(rte_pktmbuf_append(mb[i], PDCP_MAC_I_LEN) == NULL)) + goto cop_free; + + /* Update sequence number in the PDU header */ + count = __atomic_fetch_add(&en_priv->state.tx_next, 1, __ATOMIC_RELAXED); + sn = PDCP_GET_SN_12_FROM_COUNT(count); + + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, true); + } + + *nb_err = num - nb_cop; + return nb_cop; + +cop_free: + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop - i); + *nb_err = num - i; + return i; +} + +static uint16_t +pdcp_post_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static inline int +pdcp_sn_18_count_get(const struct rte_pdcp_entity *entity, int32_t rsn, uint32_t *count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint32_t rhfn, rx_deliv; + + rx_deliv = __atomic_load_n(&en_priv->state.rx_deliv, __ATOMIC_RELAXED); + rhfn = PDCP_GET_HFN_SN_18_FROM_COUNT(rx_deliv); + + if (rsn < (int32_t)(PDCP_GET_SN_18_FROM_COUNT(rx_deliv) - PDCP_SN_18_WINDOW_SZ)) { + if (unlikely(rhfn == PDCP_SN_18_HFN_MAX)) + return -ERANGE; + rhfn += 1; + } else if ((uint32_t)rsn >= (PDCP_GET_SN_18_FROM_COUNT(rx_deliv) + PDCP_SN_18_WINDOW_SZ)) { + if (unlikely(rhfn == PDCP_SN_18_HFN_MIN)) + return -ERANGE; + rhfn -= 1; + } + + *count = PDCP_SET_COUNT_FROM_HFN_SN_18(rhfn, rsn); + + return 0; +} + +static inline int +pdcp_sn_12_count_get(const struct rte_pdcp_entity *entity, int32_t rsn, uint32_t *count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint32_t rhfn, rx_deliv; + + rx_deliv = __atomic_load_n(&en_priv->state.rx_deliv, __ATOMIC_RELAXED); + rhfn = PDCP_GET_HFN_SN_12_FROM_COUNT(rx_deliv); + + if (rsn < (int32_t)(PDCP_GET_SN_12_FROM_COUNT(rx_deliv) - PDCP_SN_12_WINDOW_SZ)) { + if (unlikely(rhfn == PDCP_SN_12_HFN_MAX)) + return -ERANGE; + rhfn += 1; + } else if ((uint32_t)rsn >= (PDCP_GET_SN_12_FROM_COUNT(rx_deliv) + PDCP_SN_12_WINDOW_SZ)) { + if (unlikely(rhfn == PDCP_SN_12_HFN_MIN)) + return -ERANGE; + rhfn -= 1; + } + + *count = PDCP_SET_COUNT_FROM_HFN_SN_12(rhfn, rsn); + + return 0; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct rte_pdcp_up_data_pdu_sn_12_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + else + rte_panic("TODO: Control PDU not handled"); + + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) + break; + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, is_integ_protected); + } + + *nb_err = num - nb_cop; + + return nb_cop; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + uint16_t nb_cop; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct rte_pdcp_up_data_pdu_sn_18_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + else + rte_panic("TODO: Control PDU not handled"); + + if (unlikely(pdcp_sn_18_count_get(entity, rsn, &count))) + break; + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, is_integ_protected); + } + + *nb_err = num - nb_cop; + + return nb_cop; +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop; + uint32_t count; + int32_t rsn; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct rte_pdcp_cp_data_pdu_sn_12_hdr *); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) + break; + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, true); + } + + *nb_err = num - nb_cop; + return nb_cop; +} + +static inline bool +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, + const uint32_t count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + + if (count < __atomic_load_n(&en_priv->state.rx_deliv, __ATOMIC_RELAXED)) + return false; + + /* t-Reordering timer is not supported - SDU will be delivered immediately. + * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not + * been delivered to upper layers + */ + __atomic_store_n(&en_priv->state.rx_deliv, (count + 1), __ATOMIC_RELAXED); + + if (count >= __atomic_load_n(&en_priv->state.rx_next, __ATOMIC_RELAXED)) + __atomic_store_n(&en_priv->state.rx_next, (count + 1), __ATOMIC_RELAXED); + + return true; +} + +static inline uint16_t +pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + int i, nb_success = 0, nb_err = 0, rsn = 0; + const uint32_t aad_sz = en_priv->aad_sz; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *, + aad_sz); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + else + rte_panic("Control PDU should not be received"); + + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + rte_pktmbuf_adj(mb, hdr_trim_sz); + if (is_integ_protected) + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static inline uint16_t +pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + int32_t rsn = 0; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *, + aad_sz); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + else + rte_panic("Control PDU should not be received"); + + if (unlikely(pdcp_sn_18_count_get(entity, rsn, &count))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + rte_pktmbuf_adj(mb, hdr_trim_sz); + if (is_integ_protected) + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static uint16_t +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *, + aad_sz); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + rte_pktmbuf_adj(mb, hdr_trim_sz); + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static int +pdcp_pre_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + + entity->pre_process = NULL; + entity->post_process = NULL; + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + entity->post_process = pdcp_post_process_cplane_sn_12_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; + entity->post_process = pdcp_post_process_cplane_sn_12_dl; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + entity->post_process = pdcp_post_process_uplane_sn_12_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + entity->post_process = pdcp_post_process_uplane_sn_18_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; + entity->post_process = pdcp_post_process_uplane_sn_12_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; + entity->post_process = pdcp_post_process_uplane_sn_12_dl; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; + entity->post_process = pdcp_post_process_uplane_sn_18_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; + entity->post_process = pdcp_post_process_uplane_sn_18_dl; + } + + if (entity->pre_process == NULL || entity->post_process == NULL) + return -ENOTSUP; + + return 0; +} + +static int +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + int ret; + + /** + * flags.is_authenticated + * + * MAC-I would be added in case of control plane packets and when authentication + * transform is not NULL. + */ + + if (conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) + en_priv->flags.is_authenticated = 1; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (a_xfrm != NULL) + en_priv->flags.is_authenticated = 1; + + /** + * flags.is_ciph_in_bits + * + * For ZUC & SNOW3G cipher algos, offset & length need to be provided in bits. + */ + + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) + en_priv->flags.is_ciph_in_bits = 1; + + /** + * flags.is_auth_in_bits + * + * For ZUC & SNOW3G authentication algos, offset & length need to be provided in bits. + */ + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) + en_priv->flags.is_auth_in_bits = 1; + } + + /** + * flags.is_ul_entity + * + * Indicate whether the entity is UL/transmitting PDCP entity. + */ + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + en_priv->flags.is_ul_entity = 1; + + /** + * hdr_sz + * + * PDCP header size of the entity + */ + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); + + /** + * aad_sz + * + * For AES-CMAC, additional message is prepended for processing. Need to be trimmed after + * crypto processing is done. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) + en_priv->aad_sz = 8; + else + en_priv->aad_sz = 0; + + return 0; +} + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv *en_priv; + int ret; + + if (entity == NULL || conf == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + ret = pdcp_iv_gen_func_set(entity, conf); + if (ret) + return ret; + + ret = pdcp_entity_priv_populate(en_priv, conf); + if (ret) + return ret; + + ret = pdcp_pre_process_func_set(entity, conf); + if (ret) + return ret; + + return 0; +} diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h new file mode 100644 index 0000000000..c92ab34c40 --- /dev/null +++ b/lib/pdcp/pdcp_process.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _PDCP_PROCESS_H_ +#define _PDCP_PROCESS_H_ + +#include <rte_pdcp.h> + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); + +#endif /* _PDCP_PROCESS_H_ */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode 100644 index 0000000000..b1533971c2 --- /dev/null +++ b/lib/pdcp/rte_pdcp.c @@ -0,0 +1,136 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include <rte_pdcp.h> +#include <rte_malloc.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +{ + int size; + + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + size += sizeof(struct entity_priv_dl_part); + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size += sizeof(struct entity_priv_ul_part); + else + return -EINVAL; + + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); +} + +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) +{ + struct rte_pdcp_entity *entity = NULL; + struct entity_priv *en_priv; + int ret; + + if (conf == NULL || conf->cop_pool == NULL) { + rte_errno = -EINVAL; + return NULL; + } + + if (conf->pdcp_xfrm.en_ordering || conf->pdcp_xfrm.remove_duplicates || conf->is_slrb || + conf->en_sec_offload) { + rte_errno = -ENOTSUP; + return NULL; + } + + /* + * 6.3.2 PDCP SN + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the PDCP SN is + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl-PDCP-SN-Size in + * TS 38.331 [3]) + */ + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { + rte_errno = -ENOTSUP; + return NULL; + } + + if (conf->pdcp_xfrm.hfn || conf->pdcp_xfrm.hfn_threshold) { + rte_errno = -EINVAL; + return NULL; + } + + entity = rte_zmalloc_socket("pdcp_entity", pdcp_entity_size_get(conf), + RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (entity == NULL) { + rte_errno = -ENOMEM; + return NULL; + } + + en_priv = entity_priv_get(entity); + + en_priv->state.rx_deliv = conf->count; + en_priv->state.tx_next = conf->count; + en_priv->cop_pool = conf->cop_pool; + + /* Setup crypto session */ + ret = pdcp_crypto_sess_create(entity, conf); + if (ret) + goto entity_free; + + ret = pdcp_process_func_set(entity, conf); + if (ret) + goto crypto_sess_destroy; + + return entity; + +crypto_sess_destroy: + pdcp_crypto_sess_destroy(entity); +entity_free: + rte_free(entity); + rte_errno = ret; + return NULL; +} + +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) +{ + int ret; + + if (pdcp_entity == NULL) + return -EINVAL; + + /* Teardown crypto sessions */ + ret = pdcp_crypto_sess_destroy(pdcp_entity); + if (ret) + return ret; + + rte_free(pdcp_entity); + + RTE_SET_USED(out_mb); + return 0; +} + +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]) +{ + struct entity_priv *en_priv; + + if (pdcp_entity == NULL) + return -EINVAL; + + en_priv = entity_priv_get(pdcp_entity); + + if (en_priv->flags.is_ul_entity) { + en_priv->state.tx_next = 0; + } else { + en_priv->state.rx_next = 0; + en_priv->state.rx_deliv = 0; + } + + RTE_SET_USED(out_mb); + + return 0; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode 100644 index 0000000000..b6c7f32c05 --- /dev/null +++ b/lib/pdcp/rte_pdcp.h @@ -0,0 +1,263 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _RTE_PDCP_H_ +#define _RTE_PDCP_H_ + +/** + * @file rte_pdcp.h + * + * RTE PDCP support. + * + * librte_pdcp provides a framework for PDCP protocol processing. + */ + +#include <rte_compat.h> +#include <rte_common.h> +#include <rte_errno.h> +#include <rte_mempool.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/* Forward declarations */ +struct rte_pdcp_entity; + +/* PDCP pre-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err); + +/* PDCP post-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err); + +/** + * PDCP entity. + */ +struct rte_pdcp_entity { + /** Entity specific pre-process handle. */ + rte_pdcp_pre_p_t pre_process; + /** Entity specific post-process handle. */ + rte_pdcp_post_p_t post_process; + /** + * PDCP entities may hold packets for purposes of in-order delivery (in + * case of receiving PDCP entity) and re-transmission (in case of + * transmitting PDCP entity). + * + * For receiving PDCP entity, it may hold packets when in-order + * delivery is enabled. The packets would be cached until either a + * packet that completes the sequence arrives or when discard timer + * expires. + * + * When post-processing of PDCP packet which completes a sequence is + * done, the API may return more packets than enqueued. Application is + * expected to provide *rte_pdcp_pkt_post_process()* with *out_mb* + * which can hold maximum number of packets which may be returned. + * + * For transmitting PDCP entity, during re-establishment (5.1.2), + * entity may be required to perform re-transmission of the buffers + * after applying new ciphering & integrity algorithms. For performing + * crypto operation, *rte_pdcp_entity_re_establish()* would return as + * many crypto_ops as the ones cached. + */ + uint16_t max_pkt_cache; + /** User area for saving application data. */ + uint64_t user_area[2]; +} __rte_cache_aligned; + +/** + * PDCP entity configuration to be used for establishing an entity. + */ +struct rte_pdcp_entity_conf { + /** PDCP transform for the entity. */ + struct rte_security_pdcp_xform pdcp_xfrm; + /** Crypto transform applicable for the entity. */ + struct rte_crypto_sym_xform *crypto_xfrm; + /** Mempool for crypto symmetric session. */ + struct rte_mempool *sess_mpool; + /** Crypto op pool.*/ + struct rte_mempool *cop_pool; + /** + * 32 bit count value (HFN + SN) to be used for the first packet. + * pdcp_xfrm.hfn would be ignored as the HFN would be derived from this value. + */ + uint32_t count; + /** Indicate whether the PDCP entity belongs to Side Link Radio Bearer. */ + bool is_slrb; + /** Enable security offload on the device specified. */ + bool en_sec_offload; + /** Enable non-atomic usage of entity. */ + bool en_non_atomic; + /** Device on which security/crypto session need to be created. */ + uint8_t dev_id; + /** Reverse direction during IV generation. Can be used to simulate UE crypto processing.*/ + bool reverse_iv_direction; +}; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.1 PDCP entity establishment + * + * Establish PDCP entity based on provided input configuration. + * + * @param conf + * Parameters to be used for initializing PDCP entity object. + * @return + * - Valid handle if success + * - NULL in case of failure. rte_errno will be set to error code + */ +__rte_experimental +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.3 PDCP entity release + * + * Release PDCP entity. + * + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be released. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.4 PDCP entity suspend + * + * Suspend PDCP entity. + * + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be suspended. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity pre-process the mbufs and prepare + * crypto ops that can be enqueued to the cryptodev associated with given + * session. Only error packets would be moved returned in the input buffer, + * *mb*, and it is the responsibility of the application to free the same. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param[in, out] mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * which contain the input packets. Any error packets would be returned in the + * same buffer. + * @param[out] cop + * The address of an array that can hold up to *num* pointers to + * *rte_crypto_op* structures. Crypto ops would be allocated by + * ``rte_pdcp_pkt_pre_process`` API. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * Pointer to return the number of error packets returned in *mb* + * @return + * Count of crypto_ops prepared + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err) +{ + return entity->pre_process(entity, mb, cop, num, nb_err); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity, perform PDCP post-processing of the + * mbufs. + * + * Input mbufs are the ones retrieved from crypto_ops dequeued from cryptodev + * and grouped by *rte_pdcp_pkt_crypto_group()*. + * + * The post-processed packets would be returned in the *out_mb* buffer. + * The resultant mbufs would be grouped into success packets and error packets. + * Error packets would be grouped in the end of the array and it is the + * responsibility of the application to handle the same. + * + * When in-order delivery is enabled, PDCP entity may buffer packets and would + * deliver packets only when all prior packets have been post-processed. That + * would result in returning more/less packets than enqueued. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param in_mb + * The address of an array of *num* pointers to *rte_mbuf* structures. + * @param[out] out_mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * to output packets after PDCP post-processing. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * The number of error packets returned in *out_mb* buffer. + * @return + * Count of packets returned in *out_mb* buffer. + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return entity->post_process(entity, in_mb, out_mb, num, nb_err); +} + +#include <rte_pdcp_group.h> + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PDCP_H_ */ diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new file mode 100644 index 0000000000..2c01c19d4e --- /dev/null +++ b/lib/pdcp/rte_pdcp_group.h @@ -0,0 +1,133 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#ifndef _RTE_PDCP_GROUP_H_ +#define _RTE_PDCP_GROUP_H_ + +/** + * @file rte_pdcp_group.h + * + * RTE PDCP grouping support. + * It is not recommended to include this file directly, include <rte_pdcp.h> + * instead. + * Provides helper functions to process completed crypto-ops and group related + * packets by sessions they belong to. + */ + +#include <rte_common.h> +#include <rte_crypto.h> +#include <rte_cryptodev.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Group packets belonging to same PDCP entity. + */ +struct rte_pdcp_group { + union { + uint64_t val; + void *ptr; + } id; /**< Grouped by value */ + struct rte_mbuf **m; /**< Start of the group */ + uint32_t cnt; /**< Number of entries in the group */ + int32_t rc; /**< Status code associated with the group */ +}; + +/** + * Take crypto-op as an input and extract pointer to related PDCP entity. + * @param cop + * The address of an input *rte_crypto_op* structure. + * @return + * The pointer to the related *rte_pdcp_entity* structure. + */ +static inline struct rte_pdcp_entity * +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) +{ + void *sess = cop->sym[0].session; + + if (cop->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + return (struct rte_pdcp_entity *)(uintptr_t) + rte_security_session_opaque_data_get(sess); + } else if (cop->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { + return (struct rte_pdcp_entity *)(uintptr_t) + rte_cryptodev_sym_session_opaque_data_get(sess); + } + + return NULL; +} + +/** + * Take as input completed crypto ops, extract related mbufs and group them by + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* flag + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. + * + * Note that application must ensure only crypto-ops prepared by lib_pdcp is + * provided back to @see rte_pdcp_pkt_crypto_group(). + * + * @param cop + * The address of an array of *num* pointers to the input *rte_crypto_op* + * structures. + * @param[out] mb + * The address of an array of *num* pointers to output *rte_mbuf* structures. + * @param[out] grp + * The address of an array of *num* to output *rte_pdcp_group* structures. + * @param num + * The maximum number of crypto-ops to process. + * @return + * Number of filled elements in *grp* array. + * + */ +static inline uint16_t +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_pdcp_group grp[], uint16_t num) +{ + uint32_t i, j = 0, n = 0; + void *ns, *ps = NULL; + struct rte_mbuf *m; + + for (i = 0; i != num; i++) { + m = cop[i]->sym[0].m_src; + ns = cop[i]->sym[0].session; + + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; + + /* Different entity */ + if (ps != ns) { + + /* Finalize open group and start a new one */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + /* Start new group */ + grp[n].m = mb + j; + ps = ns; + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); + } + + mb[j++] = m; + rte_crypto_op_free(cop[i]); + } + + /* Finalize last group */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + return n; +} + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PDCP_GROUP_H_ */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode 100644 index 0000000000..8fa9d5d7cc --- /dev/null +++ b/lib/pdcp/version.map @@ -0,0 +1,13 @@ +EXPERIMENTAL { + global: + + # added in 22.11 + rte_pdcp_entity_establish; + rte_pdcp_entity_release; + rte_pdcp_entity_suspend; + + rte_pdcp_pkt_post_process; + rte_pdcp_pkt_pre_process; + + local: *; +}; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH 2/5] lib: add pdcp protocol 2022-12-22 9:25 ` [PATCH 2/5] lib: add pdcp protocol Anoob Joseph @ 2023-01-18 16:26 ` Akhil Goyal 2023-02-13 10:59 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-01-18 16:26 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Anoob, Please see inline comments. > Subject: [PATCH 2/5] lib: add pdcp protocol > > Add Packet Data Convergence Protocol (PDCP) processing library. > > The library is similar to lib_ipsec which provides IPsec processing > capabilities in DPDK. > > PDCP would involve roughly the following options, > 1. Transfer of user plane data > 2. Transfer of control plane data > 3. Header compression > 4. Uplink data compression > 5. Ciphering and integrity protection > > PDCP library provides following control path APIs that is used to > configure various PDCP entities, > 1. rte_pdcp_entity_establish() > 2. rte_pdcp_entity_suspend() > 3. rte_pdcp_entity_release() > > PDCP process is split into 2 parts. One before crypto processing > (rte_pdcp_pkt_pre_process()) and one after crypto processing > (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto > operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() > is added to help grouping crypto operations belonging to same entity. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > --- > doc/api/doxy-api-index.md | 3 +- > doc/api/doxy-api.conf.in | 1 + > lib/meson.build | 1 + > lib/pdcp/meson.build | 8 + > lib/pdcp/pdcp_crypto.c | 240 ++++++++ > lib/pdcp/pdcp_crypto.h | 20 + > lib/pdcp/pdcp_entity.h | 218 +++++++ > lib/pdcp/pdcp_process.c | 1195 +++++++++++++++++++++++++++++++++++++ > lib/pdcp/pdcp_process.h | 13 + > lib/pdcp/rte_pdcp.c | 136 +++++ > lib/pdcp/rte_pdcp.h | 263 ++++++++ > lib/pdcp/rte_pdcp_group.h | 133 +++++ > lib/pdcp/version.map | 13 + > 13 files changed, 2243 insertions(+), 1 deletion(-) > create mode 100644 lib/pdcp/meson.build > create mode 100644 lib/pdcp/pdcp_crypto.c > create mode 100644 lib/pdcp/pdcp_crypto.h > create mode 100644 lib/pdcp/pdcp_entity.h > create mode 100644 lib/pdcp/pdcp_process.c > create mode 100644 lib/pdcp/pdcp_process.h > create mode 100644 lib/pdcp/rte_pdcp.c > create mode 100644 lib/pdcp/rte_pdcp.h > create mode 100644 lib/pdcp/rte_pdcp_group.h > create mode 100644 lib/pdcp/version.map > > diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md > index ae4b107240..6014bee079 100644 > --- a/doc/api/doxy-api-index.md > +++ b/doc/api/doxy-api-index.md > @@ -126,7 +126,8 @@ The public API headers are grouped by topics: > [eCPRI](@ref rte_ecpri.h), > [L2TPv2](@ref rte_l2tpv2.h), > [PPP](@ref rte_ppp.h), > - [PDCP hdr](@ref rte_pdcp_hdr.h) > + [PDCP hdr](@ref rte_pdcp_hdr.h), > + [PDCP](@ref rte_pdcp.h), > > - **QoS**: > [metering](@ref rte_meter.h), > diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in > index f0886c3bd1..01314b087e 100644 > --- a/doc/api/doxy-api.conf.in > +++ b/doc/api/doxy-api.conf.in > @@ -61,6 +61,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api- > index.md \ > @TOPDIR@/lib/net \ > @TOPDIR@/lib/pcapng \ > @TOPDIR@/lib/pci \ > + @TOPDIR@/lib/pdcp \ > @TOPDIR@/lib/pdump \ > @TOPDIR@/lib/pipeline \ > @TOPDIR@/lib/port \ > diff --git a/lib/meson.build b/lib/meson.build > index fd55925340..a827006d29 100644 > --- a/lib/meson.build > +++ b/lib/meson.build > @@ -63,6 +63,7 @@ libraries = [ > 'flow_classify', # flow_classify lib depends on pkt framework table lib > 'graph', > 'node', > + 'pdcp', # pdcp lib depends on crypto and security > ] > > optional_libs = [ > diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build > new file mode 100644 > index 0000000000..a7f5a408cf > --- /dev/null > +++ b/lib/pdcp/meson.build > @@ -0,0 +1,8 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(C) 2022 Marvell. > +# Extra # here. Do we support compilation on Windows as well? Check missing here. > + > +sources = files('pdcp_crypto.c', 'pdcp_process.c', 'rte_pdcp.c') > +headers = files('rte_pdcp.h') Do we need to add the indirect header as well for lib/pdcp/rte_pdcp_group.h? > + > +deps += ['security'] Crypto not needed as dependency? > diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c > new file mode 100644 > index 0000000000..7ffb8a07a7 > --- /dev/null > +++ b/lib/pdcp/pdcp_crypto.c > @@ -0,0 +1,240 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#include <rte_crypto.h> > +#include <rte_crypto_sym.h> > +#include <rte_cryptodev.h> > +#include <rte_pdcp.h> > + > +#include "pdcp_crypto.h" > +#include "pdcp_entity.h" > + > +static int > +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct > rte_crypto_sym_xform *c_xfrm) > +{ > + const struct rte_cryptodev_symmetric_capability *cap; > + struct rte_cryptodev_sym_capability_idx cap_idx; > + int ret; > + > + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; > + cap_idx.algo.cipher = c_xfrm->cipher.algo; > + > + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); > + if (cap == NULL) > + return -1; > + > + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm- > >cipher.key.length, > + c_xfrm- > >cipher.iv.length); > + > + return ret; > +} > + > +static int > +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct > rte_crypto_sym_xform *a_xfrm) > +{ > + const struct rte_cryptodev_symmetric_capability *cap; > + struct rte_cryptodev_sym_capability_idx cap_idx; > + int ret; > + > + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; > + cap_idx.algo.auth = a_xfrm->auth.algo; > + > + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); > + if (cap == NULL) > + return -1; > + > + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm- > >auth.key.length, > + a_xfrm- > >auth.digest_length, > + a_xfrm->auth.iv.length); > + > + return ret; > +} > + > +static int > +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, > + const struct rte_crypto_sym_xform *c_xfrm, > + const struct rte_crypto_sym_xform *a_xfrm, > + bool is_auth_then_cipher) > +{ > + uint16_t ciph_iv_len, auth_digest_len, auth_iv_len; > + int ret; > + > + /* > + * Uplink means PDCP entity is configured for transmit. Downlink means > PDCP entity is > + * configured for receive. When integrity protection is enabled, PDCP > always performs > + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth- > verify for downlink). > + * So for uplink, crypto chain would be auth-cipher while for downlink it > would be > + * cipher-auth. > + * > + * When integrity protection is not required, xform would be cipher only. > + */ > + > + if (c_xfrm == NULL) > + return -EINVAL; > + > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { > + > + /* With UPLINK, if auth is enabled, it should be before cipher */ > + if (a_xfrm != NULL && !is_auth_then_cipher) > + return -EINVAL; > + > + /* With UPLINK, cipher operation must be encrypt */ > + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) > + return -EINVAL; > + > + /* With UPLINK, auth operation (if present) must be generate */ > + if (a_xfrm != NULL && a_xfrm->auth.op != > RTE_CRYPTO_AUTH_OP_GENERATE) > + return -EINVAL; > + > + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > { > + > + /* With DOWNLINK, if auth is enabled, it should be after cipher > */ > + if (a_xfrm != NULL && is_auth_then_cipher) > + return -EINVAL; > + > + /* With DOWNLINK, cipher operation must be decrypt */ > + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) > + return -EINVAL; > + > + /* With DOWNLINK, auth operation (if present) must be verify > */ > + if (a_xfrm != NULL && a_xfrm->auth.op != > RTE_CRYPTO_AUTH_OP_VERIFY) > + return -EINVAL; > + > + } else { > + return -EINVAL; > + } > + > + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && > + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && > + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && > + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) > + return -EINVAL; > + > + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) > + ciph_iv_len = 0; > + else > + ciph_iv_len = PDCP_IV_LENGTH; > + > + if (ciph_iv_len != c_xfrm->cipher.iv.length) > + return -EINVAL; > + > + if (a_xfrm != NULL) { > + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && > + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) && > + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && > + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_SNOW3G_UIA2)) > + return -EINVAL; > + > + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) > + auth_digest_len = 0; > + else > + auth_digest_len = 4; If we have a macro for IV length, why not for digest also? Moreover, for NULL integrity, digest length is also 4 with all 0s. Refer Annex D.1 in https://www.etsi.org/deliver/etsi_ts/133500_133599/133501/15.04.00_60/ts_133501v150400p.pdf Digest len would be 0 only in case of a_xfrm == NULL > + > + if (auth_digest_len != a_xfrm->auth.digest_length) > + return -EINVAL; > + > + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || > + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) > + auth_iv_len = PDCP_IV_LENGTH; > + else > + auth_iv_len = 0; > + > + if (a_xfrm->auth.iv.length != auth_iv_len) > + return -EINVAL; > + } > + > + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) > + return -EINVAL; > + > + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); > + if (ret) > + return -ENOTSUP; > + > + if (a_xfrm != NULL) { > + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); > + if (ret) > + return -ENOTSUP; > + } > + > + return 0; > +} > + > +int > +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > + struct entity_priv *en_priv; > + bool is_auth_then_cipher; > + int ret; > + > + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) > + return -EINVAL; > + > + en_priv = entity_priv_get(entity); > + > + en_priv->dev_id = conf->dev_id; > + > + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { > + c_xfrm = conf->crypto_xfrm; > + a_xfrm = conf->crypto_xfrm->next; > + is_auth_then_cipher = false; > + } else if (conf->crypto_xfrm->type == > RTE_CRYPTO_SYM_XFORM_AUTH) { > + a_xfrm = conf->crypto_xfrm; > + c_xfrm = conf->crypto_xfrm->next; > + is_auth_then_cipher = true; > + } else { > + return -EINVAL; > + } > + > + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, > is_auth_then_cipher); > + if (ret) > + return ret; > + > + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) > + c_xfrm->cipher.iv.offset = 0; > + else > + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; > + > + if (a_xfrm != NULL) { > + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) > + a_xfrm->auth.iv.offset = 0; > + else > + if (c_xfrm->cipher.iv.offset) > + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + > PDCP_IV_LENGTH; > + else > + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; > + } > + > + if (conf->sess_mpool == NULL) > + return -EINVAL; > + > + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf- > >dev_id, conf->crypto_xfrm, > + conf- > >sess_mpool); > + if (en_priv->crypto_sess == NULL) { > + /* API returns positive values as error codes */ > + return -rte_errno; > + } > + > + rte_cryptodev_sym_session_opaque_data_set(en_priv->crypto_sess, > (uint64_t)entity); > + > + return 0; > +} > + > +int > +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) > +{ > + struct entity_priv *en_priv; > + > + if (entity == NULL) > + return -EINVAL; > + > + en_priv = entity_priv_get(entity); > + > + if (en_priv->crypto_sess != NULL) { > + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv- > >crypto_sess); > + en_priv->crypto_sess = NULL; > + } > + > + return 0; > +} > diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h > new file mode 100644 > index 0000000000..dc625b35d0 > --- /dev/null > +++ b/lib/pdcp/pdcp_crypto.h > @@ -0,0 +1,20 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#ifndef _PDCP_CRYPTO_H_ > +#define _PDCP_CRYPTO_H_ > + > +#include <rte_crypto.h> > +#include <rte_crypto_sym.h> > +#include <rte_pdcp.h> > + > +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct > rte_crypto_sym_op)) > +#define PDCP_IV_LENGTH 16 > + > +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, > + const struct rte_pdcp_entity_conf *conf); > + > +int pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); > + > +#endif /* _PDCP_CRYPTO_H_ */ > diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h > new file mode 100644 > index 0000000000..e312fd4a8c > --- /dev/null > +++ b/lib/pdcp/pdcp_entity.h > @@ -0,0 +1,218 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#ifndef _PDCP_ENTITY_H_ > +#define _PDCP_ENTITY_H_ > + > +#include <rte_common.h> > +#include <rte_crypto_sym.h> > +#include <rte_mempool.h> > +#include <rte_pdcp.h> > +#include <rte_security.h> > + > +struct entity_priv; > + > +#define PDCP_PDU_HDR_SIZE_SN_12 (RTE_ALIGN_MUL_CEIL(12, 8) / 8) > +#define PDCP_PDU_HDR_SIZE_SN_18 (RTE_ALIGN_MUL_CEIL(18, 8) / 8) > + > +#define PDCP_GET_SN_12_FROM_COUNT(c) ((c) & 0xfff) > +#define PDCP_GET_SN_18_FROM_COUNT(c) ((c) & 0x3ffff) > + > +#define PDCP_GET_HFN_SN_12_FROM_COUNT(c) (((c) >> 12) & 0xfffff) > +#define PDCP_GET_HFN_SN_18_FROM_COUNT(c) (((c) >> 18) & 0x3fff) > + > +#define PDCP_SET_COUNT_FROM_HFN_SN_12(h, s) ((((h) & 0xfffff) << 12) | ((s) > & 0xfff)) > +#define PDCP_SET_COUNT_FROM_HFN_SN_18(h, s) ((((h) & 0x3fff) << 18) | ((s) > & 0x3ffff)) > + > +#define PDCP_SN_12_WINDOW_SZ 0x800 > +#define PDCP_SN_18_WINDOW_SZ 0x20000 > + > +#define PDCP_SN_12_HFN_MAX ((1 << (32 - 12)) - 1) > +#define PDCP_SN_12_HFN_MIN 0 > +#define PDCP_SN_18_HFN_MAX ((1 << (32 - 18)) - 1) > +#define PDCP_SN_18_HFN_MIN 0 > + Can we have common defines for SN-12 and SN-18 and take SN as parameter? We can have something like this. #define PDCP_PDU_HDR_SIZE(sn_size) (RTE_ALIGN_MUL_CEIL((sn_size), 8) / 8) #define PDCP_GET_SN_FROM_COUNT(c, sn_size) ((c) & ((1<<sn_size)-1)) #define PDCP_GET_HFN_FROM_COUNT(c, sn_size) (((c) >> sn_size) & ((1 << (32 - sn_size)) - 1)) #define PDCP_SET_COUNT_FROM_HFN_SN(h, s, sn_size) ((((h) & ((1 << (32 - sn_size)) - 1)) << sn_size) | ((s) & ((1<<sn_size)-1))) #define PDCP_HFN_MAX(sn_size) ((1 << (32 - (sn_size))) - 1) #define PDCP_HFN_MIN 0 > +/* IV generation function based on the entity configuration */ > +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, > + uint32_t count); > + > +enum pdcp_pdu_type { > + PDCP_PDU_TYPE_CTRL = 0, > + PDCP_PDU_TYPE_DATA = 1, > +}; > + > +enum pdcp_up_ctrl_pdu_type { > + PDCP_UP_CTRL_PDU_TYPE_STATUS_REPORT, > + PDCP_UP_CTRL_PDU_TYPE_ROHC_FEEDBACK, > + PDCP_UP_CTRL_PDU_TYPE_EHC_FEEDBACK, > + PDCP_UP_CRTL_PDU_TYPE_UDC_FEEDBACK > +}; > + > +struct entity_state { > + uint32_t rx_next; > + uint32_t tx_next; > + uint32_t rx_deliv; > + uint32_t rx_reord; > +}; > + > +union auth_iv_partial { > + /* For AES-CMAC, there is no IV, but message gets prepended */ > + struct { > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > + uint64_t count : 32; > + uint64_t zero_38_39 : 2; > + uint64_t direction : 1; > + uint64_t bearer : 5; > + uint64_t zero_40_63 : 24; > +#else > + uint64_t count : 32; > + uint64_t bearer : 5; > + uint64_t direction : 1; > + uint64_t zero_38_39 : 2; > + uint64_t zero_40_63 : 24; > +#endif > + } aes_cmac; > + struct { > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > + uint64_t count : 32; > + uint64_t zero_37_39 : 3; > + uint64_t bearer : 5; > + uint64_t zero_40_63 : 24; > + > + uint64_t rsvd_65_71 : 7; > + uint64_t direction_64 : 1; > + uint64_t rsvd_72_111 : 40; > + uint64_t rsvd_113_119 : 7; > + uint64_t direction_112 : 1; > + uint64_t rsvd_120_127 : 8; > +#else > + uint64_t count : 32; > + uint64_t bearer : 5; > + uint64_t zero_37_39 : 3; > + uint64_t zero_40_63 : 24; > + > + uint64_t direction_64 : 1; > + uint64_t rsvd_65_71 : 7; > + uint64_t rsvd_72_111 : 40; > + uint64_t direction_112 : 1; > + uint64_t rsvd_113_119 : 7; > + uint64_t rsvd_120_127 : 8; > +#endif > + } zs; > + uint64_t u64[2]; > +}; > + > +union cipher_iv_partial { > + struct { > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > + uint64_t count : 32; > + uint64_t zero_38_39 : 2; > + uint64_t direction : 1; > + uint64_t bearer : 5; > + uint64_t zero_40_63 : 24; > + > + uint64_t zero_64_127; > +#else > + uint64_t count : 32; > + uint64_t bearer : 5; > + uint64_t direction : 1; > + uint64_t zero_38_39 : 2; > + uint64_t zero_40_63 : 24; > + > + uint64_t zero_64_127; Can we take zero_64_127 out of #if-else > +#endif > + } aes_ctr; > + struct { > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > + uint64_t count : 32; > + uint64_t zero_38_39 : 2; > + uint64_t direction : 1; > + uint64_t bearer : 5; > + uint64_t zero_40_63 : 24; > + > + uint64_t rsvd_64_127; > +#else > + uint64_t count : 32; > + uint64_t bearer : 5; > + uint64_t direction : 1; > + uint64_t zero_38_39 : 2; > + uint64_t zero_40_63 : 24; > + > + uint64_t rsvd_64_127; > +#endif rsvd_64_127 can also be out of #if-else > + } zs; > + uint64_t u64[2]; > +}; > + > +/* > + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] If the layout is fixed, can we have 0 length array in rte_pdcp_entity for entity_priv and in entity_priv for entity_dl/ul. I see that entity_dl/ul are not completely defined. You can define them later when they are supported. > + */ > + Extra line > +struct entity_priv { > + /** Crypto sym session. */ > + struct rte_cryptodev_sym_session *crypto_sess; > + /** Entity specific IV generation function. */ > + iv_gen_t iv_gen; > + /** Pre-prepared auth IV. */ > + union auth_iv_partial auth_iv_part; > + /** Pre-prepared cipher IV. */ > + union cipher_iv_partial cipher_iv_part; > + /** Entity state variables. */ > + struct entity_state state; > + /** Flags. */ > + struct { > + /** PDCP PDU has 4 byte MAC-I. */ > + uint64_t is_authenticated : 1; > + /** Cipher offset & length in bits. */ > + uint64_t is_ciph_in_bits : 1; > + /** Auth offset & length in bits. */ > + uint64_t is_auth_in_bits : 1; > + /** Is UL/transmitting PDCP entity */ > + uint64_t is_ul_entity : 1; > + } flags; > + /** Crypto op pool. */ > + struct rte_mempool *cop_pool; > + /** PDCP header size. */ > + uint8_t hdr_sz; > + /** PDCP AAD size. For AES-CMAC, additional message is prepended for > the operation. */ > + uint8_t aad_sz; > + /** Device ID of the device to be used for offload. */ > + uint8_t dev_id; > +}; > + > +struct entity_priv_dl_part { > + /* TODO - when in-order-delivery is supported, post PDCP packets > would need to cached. */ > + uint8_t dummy; > +}; > + > +struct entity_priv_ul_part { > + /* > + * TODO - when re-establish is supported, both plain & post PDCP > packets would need to be > + * cached. > + */ > + uint8_t dummy; > +}; > + > +static inline struct entity_priv * > +entity_priv_get(const struct rte_pdcp_entity *entity) { > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); > +} > + > +static inline struct entity_priv_dl_part * > +entity_dl_part_get(const struct rte_pdcp_entity *entity) { > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + > sizeof(struct entity_priv)); > +} > + > +static inline struct entity_priv_ul_part * > +entity_ul_part_get(const struct rte_pdcp_entity *entity) { > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + > sizeof(struct entity_priv)); > +} Above inline functions may not be needed also if we have 0 len arrays. > + > +static inline int > +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) > +{ > + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; > +} PDCP_PDU_HDR_SIZE is same as this inline function. Can we get away with this one? > + > +#endif /* _PDCP_ENTITY_H_ */ > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c > new file mode 100644 > index 0000000000..282cf38ec4 > --- /dev/null > +++ b/lib/pdcp/pdcp_process.c > @@ -0,0 +1,1195 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#include <rte_crypto.h> > +#include <rte_crypto_sym.h> > +#include <rte_cryptodev.h> > +#include <rte_memcpy.h> > +#include <rte_pdcp.h> > +#include <rte_pdcp_hdr.h> > + > +#include "pdcp_crypto.h" > +#include "pdcp_entity.h" > +#include "pdcp_process.h" > + > +#define PDCP_MAC_I_LEN 4 Can you define it at same place where PDCP_IV_LENGTH is defined and use this in xform validate. > + > +/* Enum of supported algorithms for ciphering */ > +enum pdcp_cipher_algo { > + PDCP_CIPHER_ALGO_NULL, > + PDCP_CIPHER_ALGO_AES, > + PDCP_CIPHER_ALGO_ZUC, > + PDCP_CIPHER_ALGO_SNOW3G, > + PDCP_CIPHER_ALGO_MAX > +}; > + > +/* Enum of supported algorithms for integrity */ > +enum pdcp_auth_algo { > + PDCP_AUTH_ALGO_NULL, > + PDCP_AUTH_ALGO_AES, > + PDCP_AUTH_ALGO_ZUC, > + PDCP_AUTH_ALGO_SNOW3G, > + PDCP_AUTH_ALGO_MAX > +}; > + > +/* IV generation functions based on type of operation (cipher - auth) */ > + > +static void > +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + /* No IV required for NULL cipher + NULL auth */ > + RTE_SET_USED(cop); > + RTE_SET_USED(en_priv); > + RTE_SET_USED(count); > +} > + > +static void > +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct > entity_priv *en_priv, > + uint32_t count) > +{ > + struct rte_crypto_sym_op *op = cop->sym; > + struct rte_mbuf *mb = op->m_src; > + uint8_t *m_ptr; > + uint64_t m; > + > + /* AES-CMAC requires message to be prepended with info on count etc > */ > + > + /* Prepend by 8 bytes to add custom message */ > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > + > + m = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + > + rte_memcpy(m_ptr, &m, 8); > +} > + > +static void > +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + uint64_t iv_u64[2]; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > + rte_memcpy(iv + 8, &iv_u64[1], 8); > +} > + > +static void > +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, > + uint32_t count) > +{ > + uint64_t iv_u64[2]; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + iv_u64[1] = 0; > + rte_memcpy(iv, iv_u64, 16); > +} > + > +static void > +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + uint64_t iv_u64; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64 = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64, 8); > + rte_memcpy(iv + 8, &iv_u64, 8); > +} > + > +static void > +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + uint64_t iv_u64[2]; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + /* Generating cipher IV */ > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + rte_memcpy(iv + 8, &iv_u64[0], 8); > + > + iv += PDCP_IV_LENGTH; > + > + /* Generating auth IV */ > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > + rte_memcpy(iv + 8, &iv_u64[1], 8); > +} > + > +static void > +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, > + uint32_t count) > +{ > + struct rte_crypto_sym_op *op = cop->sym; > + struct rte_mbuf *mb = op->m_src; > + uint8_t *m_ptr, *iv; > + uint64_t iv_u64[2]; > + uint64_t m; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + rte_memcpy(iv + 8, &iv_u64[0], 8); > + > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > + m = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(m_ptr, &m, 8); > +} > + > +static void > +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct > entity_priv *en_priv, > + uint32_t count) > +{ > + struct rte_crypto_sym_op *op = cop->sym; > + struct rte_mbuf *mb = op->m_src; > + uint8_t *m_ptr, *iv; > + uint64_t iv_u64[2]; > + uint64_t m; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + iv_u64[1] = 0; > + rte_memcpy(iv, iv_u64, PDCP_IV_LENGTH); > + > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > + m = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(m_ptr, &m, 8); > +} > + > +static void > +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + uint64_t iv_u64[2]; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + iv_u64[1] = 0; > + rte_memcpy(iv, iv_u64, PDCP_IV_LENGTH); > + > + iv += PDCP_IV_LENGTH; > + > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > + rte_memcpy(iv + 8, &iv_u64[1], 8); > +} > + > +static int > +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct > rte_crypto_sym_xform **c_xfrm, > + struct rte_crypto_sym_xform **a_xfrm) > +{ > + *c_xfrm = NULL; > + *a_xfrm = NULL; > + > + if (conf->crypto_xfrm == NULL) > + return -EINVAL; > + > + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { > + *c_xfrm = conf->crypto_xfrm; > + *a_xfrm = conf->crypto_xfrm->next; > + } else if (conf->crypto_xfrm->type == > RTE_CRYPTO_SYM_XFORM_AUTH) { > + *a_xfrm = conf->crypto_xfrm; > + *c_xfrm = conf->crypto_xfrm->next; > + } else { > + return -EINVAL; > + } > + > + return 0; > +} > + > +static int > +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > + enum rte_security_pdcp_direction direction; > + enum pdcp_cipher_algo ciph_algo; > + enum pdcp_auth_algo auth_algo; > + struct entity_priv *en_priv; > + int ret; > + > + en_priv = entity_priv_get(entity); > + > + direction = conf->pdcp_xfrm.pkt_dir; > + if (conf->reverse_iv_direction) > + direction = !direction; > + > + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); > + if (ret) > + return ret; > + > + if (c_xfrm == NULL) > + return -EINVAL; > + > + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); > + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); > + > + switch (c_xfrm->cipher.algo) { > + case RTE_CRYPTO_CIPHER_NULL: > + ciph_algo = PDCP_CIPHER_ALGO_NULL; > + break; > + case RTE_CRYPTO_CIPHER_AES_CTR: > + ciph_algo = PDCP_CIPHER_ALGO_AES; > + en_priv->cipher_iv_part.aes_ctr.bearer = conf- > >pdcp_xfrm.bearer; > + en_priv->cipher_iv_part.aes_ctr.direction = direction; > + break; > + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: > + ciph_algo = PDCP_CIPHER_ALGO_SNOW3G; > + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; > + en_priv->cipher_iv_part.zs.direction = direction; > + break; > + case RTE_CRYPTO_CIPHER_ZUC_EEA3: > + ciph_algo = PDCP_CIPHER_ALGO_ZUC; > + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; > + en_priv->cipher_iv_part.zs.direction = direction; > + break; > + default: > + return -ENOTSUP; > + } > + > + if (a_xfrm != NULL) { > + switch (a_xfrm->auth.algo) { > + case RTE_CRYPTO_AUTH_NULL: > + auth_algo = PDCP_AUTH_ALGO_NULL; > + break; > + case RTE_CRYPTO_AUTH_AES_CMAC: > + auth_algo = PDCP_AUTH_ALGO_AES; > + en_priv->auth_iv_part.aes_cmac.bearer = conf- > >pdcp_xfrm.bearer; > + en_priv->auth_iv_part.aes_cmac.direction = direction; > + break; > + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: > + auth_algo = PDCP_AUTH_ALGO_SNOW3G; > + en_priv->auth_iv_part.zs.bearer = conf- > >pdcp_xfrm.bearer; > + en_priv->auth_iv_part.zs.direction_64 = direction; > + en_priv->auth_iv_part.zs.direction_112 = direction; > + break; > + case RTE_CRYPTO_AUTH_ZUC_EIA3: > + auth_algo = PDCP_AUTH_ALGO_ZUC; > + en_priv->auth_iv_part.zs.bearer = conf- > >pdcp_xfrm.bearer; > + en_priv->auth_iv_part.zs.direction_64 = direction; > + en_priv->auth_iv_part.zs.direction_112 = direction; > + break; > + default: > + return -ENOTSUP; > + } > + } else { > + auth_algo = PDCP_AUTH_ALGO_NULL; > + } > + > + static const iv_gen_t > iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = > pdcp_iv_gen_null_null, > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = > pdcp_iv_gen_null_aes_cmac, > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = > pdcp_iv_gen_null_zs, > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = > pdcp_iv_gen_null_zs, > + > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = > pdcp_iv_gen_aes_ctr_null, > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = > pdcp_iv_gen_aes_ctr_aes_cmac, > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = > pdcp_iv_gen_aes_ctr_zs, > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = > pdcp_iv_gen_aes_ctr_zs, > + > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = > pdcp_iv_gen_zs_null, > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = > pdcp_iv_gen_zs_aes_cmac, > + > [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = > pdcp_iv_gen_zs_zs, > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = > pdcp_iv_gen_zs_zs, > + > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = > pdcp_iv_gen_zs_null, > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = > pdcp_iv_gen_zs_aes_cmac, > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = > pdcp_iv_gen_zs_zs, > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = > pdcp_iv_gen_zs_zs, > + }; > + > + en_priv->iv_gen = iv_gen_map[ciph_algo][auth_algo]; > + > + return 0; > +} > + > +static inline void > +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct > rte_crypto_op *cop, > + uint8_t data_offset, uint32_t count, const bool is_auth) > +{ > + const struct rte_crypto_op cop_init = { > + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, > + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, > + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, > + }; > + struct rte_crypto_sym_op *op; > + uint32_t pkt_len; > + > + const uint8_t ciph_shift = 3 * en_priv->flags.is_ciph_in_bits; > + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; > + > + op = cop->sym; > + cop->raw = cop_init.raw; > + op->m_src = mb; > + op->m_dst = mb; > + > + /* Set IV */ > + en_priv->iv_gen(cop, en_priv, count); > + > + /* Prepare op */ > + pkt_len = rte_pktmbuf_pkt_len(mb); > + op->cipher.data.offset = data_offset << ciph_shift; > + op->cipher.data.length = (pkt_len - data_offset) << ciph_shift; > + > + if (is_auth) { > + op->auth.data.offset = 0; > + op->auth.data.length = (pkt_len - PDCP_MAC_I_LEN) << > auth_shift; > + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t > *, > + (pkt_len - > PDCP_MAC_I_LEN)); > + } > + > + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); > +} > + > +static inline bool > +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct > rte_mbuf *mb, > + uint32_t *count) > +{ > + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; > + const uint8_t hdr_sz = en_priv->hdr_sz; > + uint32_t sn; > + > + /* Prepend PDU header */ > + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr > *)rte_pktmbuf_prepend(mb, hdr_sz); > + if (unlikely(pdu_hdr == NULL)) > + return false; > + > + /* Update sequence num in the PDU header */ > + *count = __atomic_fetch_add(&en_priv->state.tx_next, 1, > __ATOMIC_RELAXED); > + sn = PDCP_GET_SN_12_FROM_COUNT(*count); > + > + pdu_hdr->d_c = PDCP_PDU_TYPE_DATA; > + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); > + pdu_hdr->sn_7_0 = (sn & 0xff); > + pdu_hdr->r = 0; > + return true; > +} > + > +static uint16_t > +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, > struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + uint16_t nb_cop; > + uint32_t count; > + int i; > + > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > + > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > + num); > + > + if (en_priv->flags.is_authenticated) { > + for (i = 0; i < nb_cop; i++) { > + if (unlikely(rte_pktmbuf_append(mb[i], > PDCP_MAC_I_LEN) == NULL)) > + goto cop_free; > + if > (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb[i], > + > &count))) > + goto cop_free; > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > true); > + } > + } else { > + for (i = 0; i < nb_cop; i++) { > + if > (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb[i], > + > &count))) > + goto cop_free; > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > false); > + } > + } > + > + *nb_err = num - nb_cop; > + return nb_cop; > +cop_free: > + /* Using mempool API since crypto API is not providing bulk free */ > + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop - i); > + *nb_err = num - i; > + return i; > +} > + > +static inline bool > +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct > rte_mbuf *mb, > + uint32_t *count) > +{ > + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; > + const uint8_t hdr_sz = en_priv->hdr_sz; > + uint32_t sn; > + > + /* Prepend PDU header */ > + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr > *)rte_pktmbuf_prepend(mb, hdr_sz); > + if (unlikely(pdu_hdr == NULL)) > + return false; > + > + /* Update sequence num in the PDU header */ > + *count = __atomic_fetch_add(&en_priv->state.tx_next, 1, > __ATOMIC_RELAXED); > + sn = PDCP_GET_SN_18_FROM_COUNT(*count); > + > + pdu_hdr->d_c = PDCP_PDU_TYPE_DATA; > + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); > + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); > + pdu_hdr->sn_7_0 = (sn & 0xff); > + pdu_hdr->r = 0; > + > + return true; > +} > + > +static inline uint16_t > +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, > struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + uint16_t nb_cop; > + uint32_t count; > + int i; > + > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > + > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > + num); > + > + if (en_priv->flags.is_authenticated) { > + for (i = 0; i < nb_cop; i++) { > + if (unlikely(rte_pktmbuf_append(mb[i], > PDCP_MAC_I_LEN) == NULL)) > + goto cop_free; > + if > (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb[i], > + > &count))) > + goto cop_free; > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > true); > + } > + } else { > + for (i = 0; i < nb_cop; i++) { > + if > (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb[i], > + > &count))) > + goto cop_free; > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > false); > + } > + } > + > + *nb_err = num - nb_cop; > + return nb_cop; > + > +cop_free: > + /* Using mempool API since crypto API is not providing bulk free */ > + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop - i); > + *nb_err = num - i; > + return i; > +} > + > +static uint16_t > +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, > struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; > + uint32_t count, sn; > + uint16_t nb_cop; > + int i; > + > + const uint8_t hdr_sz = en_priv->hdr_sz; > + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; > + > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > + num); > + > + for (i = 0; i < nb_cop; i++) { > + /* Prepend PDU header */ > + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr > *)rte_pktmbuf_prepend(mb[i], > + > hdr_sz); > + if (unlikely(pdu_hdr == NULL)) > + goto cop_free; > + if (unlikely(rte_pktmbuf_append(mb[i], PDCP_MAC_I_LEN) == > NULL)) > + goto cop_free; > + > + /* Update sequence number in the PDU header */ > + count = __atomic_fetch_add(&en_priv->state.tx_next, 1, > __ATOMIC_RELAXED); > + sn = PDCP_GET_SN_12_FROM_COUNT(count); > + > + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); > + pdu_hdr->sn_7_0 = (sn & 0xff); > + pdu_hdr->r = 0; > + > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, true); > + } > + > + *nb_err = num - nb_cop; > + return nb_cop; > + > +cop_free: > + /* Using mempool API since crypto API is not providing bulk free */ > + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop - i); > + *nb_err = num - i; > + return i; > +} > + > +static uint16_t > +pdcp_post_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err_ret) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + const uint32_t hdr_trim_sz = en_priv->aad_sz; > + int i, nb_success = 0, nb_err = 0; > + struct rte_mbuf *err_mb[num]; > + struct rte_mbuf *mb; > + > + for (i = 0; i < num; i++) { > + mb = in_mb[i]; > + if (unlikely(mb->ol_flags & > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { > + err_mb[nb_err++] = mb; > + continue; > + } > + > + if (hdr_trim_sz) > + rte_pktmbuf_adj(mb, hdr_trim_sz); > + > + out_mb[nb_success++] = mb; > + } > + > + if (unlikely(nb_err != 0)) > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > sizeof(struct rte_mbuf *)); > + > + *nb_err_ret = nb_err; > + return nb_success; > +} > + > +static uint16_t > +pdcp_post_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err_ret) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + const uint32_t hdr_trim_sz = en_priv->aad_sz; > + int i, nb_success = 0, nb_err = 0; > + struct rte_mbuf *err_mb[num]; > + struct rte_mbuf *mb; > + > + for (i = 0; i < num; i++) { > + mb = in_mb[i]; > + if (unlikely(mb->ol_flags & > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { > + err_mb[nb_err++] = mb; > + continue; > + } > + > + if (hdr_trim_sz) > + rte_pktmbuf_adj(mb, hdr_trim_sz); > + > + out_mb[nb_success++] = mb; > + } > + > + if (unlikely(nb_err != 0)) > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > sizeof(struct rte_mbuf *)); > + > + *nb_err_ret = nb_err; > + return nb_success; > +} > + > +static uint16_t > +pdcp_post_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err_ret) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + const uint32_t hdr_trim_sz = en_priv->aad_sz; > + int i, nb_success = 0, nb_err = 0; > + struct rte_mbuf *mb, *err_mb[num]; > + > + for (i = 0; i < num; i++) { > + mb = in_mb[i]; > + if (unlikely(mb->ol_flags & > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { > + err_mb[nb_err++] = mb; > + continue; > + } > + > + if (hdr_trim_sz) > + rte_pktmbuf_adj(mb, hdr_trim_sz); > + > + out_mb[nb_success++] = mb; > + } > + > + if (unlikely(nb_err != 0)) > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > sizeof(struct rte_mbuf *)); > + > + *nb_err_ret = nb_err; > + return nb_success; > +} > + > +static inline int > +pdcp_sn_18_count_get(const struct rte_pdcp_entity *entity, int32_t rsn, > uint32_t *count) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + uint32_t rhfn, rx_deliv; > + > + rx_deliv = __atomic_load_n(&en_priv->state.rx_deliv, > __ATOMIC_RELAXED); > + rhfn = PDCP_GET_HFN_SN_18_FROM_COUNT(rx_deliv); > + > + if (rsn < (int32_t)(PDCP_GET_SN_18_FROM_COUNT(rx_deliv) - > PDCP_SN_18_WINDOW_SZ)) { > + if (unlikely(rhfn == PDCP_SN_18_HFN_MAX)) > + return -ERANGE; > + rhfn += 1; > + } else if ((uint32_t)rsn >= (PDCP_GET_SN_18_FROM_COUNT(rx_deliv) + > PDCP_SN_18_WINDOW_SZ)) { > + if (unlikely(rhfn == PDCP_SN_18_HFN_MIN)) > + return -ERANGE; > + rhfn -= 1; > + } > + > + *count = PDCP_SET_COUNT_FROM_HFN_SN_18(rhfn, rsn); > + > + return 0; > +} > + > +static inline int > +pdcp_sn_12_count_get(const struct rte_pdcp_entity *entity, int32_t rsn, > uint32_t *count) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + uint32_t rhfn, rx_deliv; > + > + rx_deliv = __atomic_load_n(&en_priv->state.rx_deliv, > __ATOMIC_RELAXED); > + rhfn = PDCP_GET_HFN_SN_12_FROM_COUNT(rx_deliv); > + > + if (rsn < (int32_t)(PDCP_GET_SN_12_FROM_COUNT(rx_deliv) - > PDCP_SN_12_WINDOW_SZ)) { > + if (unlikely(rhfn == PDCP_SN_12_HFN_MAX)) > + return -ERANGE; > + rhfn += 1; > + } else if ((uint32_t)rsn >= (PDCP_GET_SN_12_FROM_COUNT(rx_deliv) + > PDCP_SN_12_WINDOW_SZ)) { > + if (unlikely(rhfn == PDCP_SN_12_HFN_MIN)) > + return -ERANGE; > + rhfn -= 1; > + } > + > + *count = PDCP_SET_COUNT_FROM_HFN_SN_12(rhfn, rsn); > + > + return 0; > +} > + > +static inline uint16_t > +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity > *entity, struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err, > + const bool is_integ_protected) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; > + uint16_t nb_cop; > + int32_t rsn = 0; > + uint32_t count; > + int i; > + > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > + > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > + num); > + > + for (i = 0; i < nb_cop; i++) { > + > + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct > rte_pdcp_up_data_pdu_sn_12_hdr *); > + > + /* Check for PDU type */ > + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) > + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); > + else > + rte_panic("TODO: Control PDU not handled"); > + > + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) > + break; > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > is_integ_protected); > + } > + > + *nb_err = num - nb_cop; > + > + return nb_cop; > +} > + > +static uint16_t > +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, > struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err) > +{ > + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, > nb_err, true); > +} > + > +static uint16_t > +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, > struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err) > +{ > + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, > nb_err, false); > +} > + > +static inline uint16_t > +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity > *entity, struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err, > + const bool is_integ_protected) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; > + uint16_t nb_cop; > + int32_t rsn = 0; > + uint32_t count; > + int i; > + > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > + num); > + > + for (i = 0; i < nb_cop; i++) { > + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct > rte_pdcp_up_data_pdu_sn_18_hdr *); > + > + /* Check for PDU type */ > + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) > + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 > << 8) | > + (pdu_hdr->sn_7_0)); > + else > + rte_panic("TODO: Control PDU not handled"); > + > + if (unlikely(pdcp_sn_18_count_get(entity, rsn, &count))) > + break; > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > is_integ_protected); > + } > + > + *nb_err = num - nb_cop; > + > + return nb_cop; > +} > + > +static uint16_t > +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, > struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err) > +{ > + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, > nb_err, true); > +} > + > +static uint16_t > +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, > struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err) > +{ > + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, > nb_err, false); > +} > + > +static uint16_t > +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, > struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], uint16_t num, > uint16_t *nb_err) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; > + uint16_t nb_cop; > + uint32_t count; > + int32_t rsn; > + int i; > + > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > + > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > + num); > + > + for (i = 0; i < nb_cop; i++) { > + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct > rte_pdcp_cp_data_pdu_sn_12_hdr *); > + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); > + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) > + break; > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, true); > + } > + > + *nb_err = num - nb_cop; > + return nb_cop; > +} > + > +static inline bool > +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, > + const uint32_t count) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + > + if (count < __atomic_load_n(&en_priv->state.rx_deliv, > __ATOMIC_RELAXED)) > + return false; > + > + /* t-Reordering timer is not supported - SDU will be delivered > immediately. > + * Update RX_DELIV to the COUNT value of the first PDCP SDU which > has not > + * been delivered to upper layers > + */ > + __atomic_store_n(&en_priv->state.rx_deliv, (count + 1), > __ATOMIC_RELAXED); > + > + if (count >= __atomic_load_n(&en_priv->state.rx_next, > __ATOMIC_RELAXED)) > + __atomic_store_n(&en_priv->state.rx_next, (count + 1), > __ATOMIC_RELAXED); > + > + return true; > +} > + > +static inline uint16_t > +pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity > *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err_ret, > + const bool is_integ_protected) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; > + int i, nb_success = 0, nb_err = 0, rsn = 0; > + const uint32_t aad_sz = en_priv->aad_sz; > + struct rte_mbuf *err_mb[num]; > + struct rte_mbuf *mb; > + uint32_t count; > + > + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; > + > + for (i = 0; i < num; i++) { > + mb = in_mb[i]; > + if (unlikely(mb->ol_flags & > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) > + goto error; > + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct > rte_pdcp_up_data_pdu_sn_12_hdr *, > + aad_sz); > + > + /* Check for PDU type */ > + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) > + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); > + else > + rte_panic("Control PDU should not be received"); > + > + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) > + goto error; > + > + if (unlikely(!pdcp_post_process_update_entity_state(entity, > count))) > + goto error; > + > + rte_pktmbuf_adj(mb, hdr_trim_sz); > + if (is_integ_protected) > + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); > + out_mb[nb_success++] = mb; > + continue; > + > +error: > + err_mb[nb_err++] = mb; > + } > + > + if (unlikely(nb_err != 0)) > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > sizeof(struct rte_mbuf *)); > + > + *nb_err_ret = nb_err; > + return nb_success; > +} > + > +static uint16_t > +pdcp_post_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err) > +{ > + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, > out_mb, num, nb_err, true); > +} > + > +static uint16_t > +pdcp_post_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err) > +{ > + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, > out_mb, num, nb_err, false); > +} > + > +static inline uint16_t > +pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity > *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err_ret, > + const bool is_integ_protected) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; > + const uint32_t aad_sz = en_priv->aad_sz; > + int i, nb_success = 0, nb_err = 0; > + struct rte_mbuf *mb, *err_mb[num]; > + int32_t rsn = 0; > + uint32_t count; > + > + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; > + > + for (i = 0; i < num; i++) { > + mb = in_mb[i]; > + if (unlikely(mb->ol_flags & > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) > + goto error; > + > + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct > rte_pdcp_up_data_pdu_sn_18_hdr *, > + aad_sz); > + > + /* Check for PDU type */ > + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) > + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 > << 8) | > + (pdu_hdr->sn_7_0)); > + else > + rte_panic("Control PDU should not be received"); > + > + if (unlikely(pdcp_sn_18_count_get(entity, rsn, &count))) > + goto error; > + > + if (unlikely(!pdcp_post_process_update_entity_state(entity, > count))) > + goto error; > + > + rte_pktmbuf_adj(mb, hdr_trim_sz); > + if (is_integ_protected) > + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); > + out_mb[nb_success++] = mb; > + continue; > + > +error: > + err_mb[nb_err++] = mb; > + } > + > + if (unlikely(nb_err != 0)) > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > sizeof(struct rte_mbuf *)); > + > + *nb_err_ret = nb_err; > + return nb_success; > +} > + > +static uint16_t > +pdcp_post_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err) > +{ > + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, > out_mb, num, nb_err, true); > +} > + > +static uint16_t > +pdcp_post_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err) > +{ > + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, > out_mb, num, nb_err, false); > +} > + > +static uint16_t > +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err_ret) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; > + const uint32_t aad_sz = en_priv->aad_sz; > + int i, nb_success = 0, nb_err = 0; > + struct rte_mbuf *err_mb[num]; > + struct rte_mbuf *mb; > + uint32_t count; > + int32_t rsn; > + > + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; > + > + for (i = 0; i < num; i++) { > + mb = in_mb[i]; > + if (unlikely(mb->ol_flags & > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) > + goto error; > + > + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct > rte_pdcp_cp_data_pdu_sn_12_hdr *, > + aad_sz); > + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); > + > + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) > + goto error; > + > + if (unlikely(!pdcp_post_process_update_entity_state(entity, > count))) > + goto error; > + > + rte_pktmbuf_adj(mb, hdr_trim_sz); > + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); > + out_mb[nb_success++] = mb; > + continue; > + > +error: > + err_mb[nb_err++] = mb; > + } > + > + if (unlikely(nb_err != 0)) > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > sizeof(struct rte_mbuf *)); > + > + *nb_err_ret = nb_err; > + return nb_success; > +} > + > +static int > +pdcp_pre_process_func_set(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + > + entity->pre_process = NULL; > + entity->post_process = NULL; > + > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_CONTROL) && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { > + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; > + entity->post_process = pdcp_post_process_cplane_sn_12_ul; > + } > + > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_CONTROL) && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { > + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; > + entity->post_process = pdcp_post_process_cplane_sn_12_dl; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; > + entity->post_process = pdcp_post_process_uplane_sn_12_ul; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; > + entity->post_process = pdcp_post_process_uplane_sn_18_ul; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && > + (en_priv->flags.is_authenticated)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; > + entity->post_process = > pdcp_post_process_uplane_sn_12_dl_ip; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && > + (!en_priv->flags.is_authenticated)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; > + entity->post_process = pdcp_post_process_uplane_sn_12_dl; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && > + (en_priv->flags.is_authenticated)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; > + entity->post_process = > pdcp_post_process_uplane_sn_18_dl_ip; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && > + (!en_priv->flags.is_authenticated)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; > + entity->post_process = pdcp_post_process_uplane_sn_18_dl; > + } > + > + if (entity->pre_process == NULL || entity->post_process == NULL) > + return -ENOTSUP; > + > + return 0; > +} > + > +static int > +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > + int ret; > + > + /** > + * flags.is_authenticated > + * > + * MAC-I would be added in case of control plane packets and when > authentication > + * transform is not NULL. > + */ > + > + if (conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_CONTROL) > + en_priv->flags.is_authenticated = 1; > + > + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); > + if (ret) > + return ret; > + > + if (a_xfrm != NULL) > + en_priv->flags.is_authenticated = 1; > + > + /** > + * flags.is_ciph_in_bits > + * > + * For ZUC & SNOW3G cipher algos, offset & length need to be provided > in bits. > + */ > + > + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || > + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) > + en_priv->flags.is_ciph_in_bits = 1; > + > + /** > + * flags.is_auth_in_bits > + * > + * For ZUC & SNOW3G authentication algos, offset & length need to be > provided in bits. > + */ > + > + if (a_xfrm != NULL) { > + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) > || > + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) > + en_priv->flags.is_auth_in_bits = 1; > + } > + > + /** > + * flags.is_ul_entity > + * > + * Indicate whether the entity is UL/transmitting PDCP entity. > + */ > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > + en_priv->flags.is_ul_entity = 1; > + > + /** > + * hdr_sz > + * > + * PDCP header size of the entity > + */ > + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); > + > + /** > + * aad_sz > + * > + * For AES-CMAC, additional message is prepended for processing. Need > to be trimmed after > + * crypto processing is done. > + */ > + if (a_xfrm != NULL && a_xfrm->auth.algo == > RTE_CRYPTO_AUTH_AES_CMAC) > + en_priv->aad_sz = 8; > + else > + en_priv->aad_sz = 0; > + > + return 0; > +} > + > +int > +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct entity_priv *en_priv; > + int ret; > + > + if (entity == NULL || conf == NULL) > + return -EINVAL; > + > + en_priv = entity_priv_get(entity); > + > + ret = pdcp_iv_gen_func_set(entity, conf); > + if (ret) > + return ret; > + > + ret = pdcp_entity_priv_populate(en_priv, conf); > + if (ret) > + return ret; > + > + ret = pdcp_pre_process_func_set(entity, conf); > + if (ret) > + return ret; > + > + return 0; > +} > diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h > new file mode 100644 > index 0000000000..c92ab34c40 > --- /dev/null > +++ b/lib/pdcp/pdcp_process.h > @@ -0,0 +1,13 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#ifndef _PDCP_PROCESS_H_ > +#define _PDCP_PROCESS_H_ > + > +#include <rte_pdcp.h> > + > +int > +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf); > + > +#endif /* _PDCP_PROCESS_H_ */ > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c > new file mode 100644 > index 0000000000..b1533971c2 > --- /dev/null > +++ b/lib/pdcp/rte_pdcp.c > @@ -0,0 +1,136 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#include <rte_pdcp.h> > +#include <rte_malloc.h> > + > +#include "pdcp_crypto.h" > +#include "pdcp_entity.h" > +#include "pdcp_process.h" > + > +static int > +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) > +{ > + int size; > + > + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); > + > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > + size += sizeof(struct entity_priv_dl_part); > + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > + size += sizeof(struct entity_priv_ul_part); > + else > + return -EINVAL; > + > + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); > +} > + > +struct rte_pdcp_entity * > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) > +{ > + struct rte_pdcp_entity *entity = NULL; > + struct entity_priv *en_priv; > + int ret; > + > + if (conf == NULL || conf->cop_pool == NULL) { > + rte_errno = -EINVAL; > + return NULL; > + } > + > + if (conf->pdcp_xfrm.en_ordering || conf- > >pdcp_xfrm.remove_duplicates || conf->is_slrb || > + conf->en_sec_offload) { > + rte_errno = -ENOTSUP; > + return NULL; > + } > + > + /* > + * 6.3.2 PDCP SN > + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the > PDCP SN is > + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl- > PDCP-SN-Size in > + * TS 38.331 [3]) > + */ > + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { > + rte_errno = -ENOTSUP; > + return NULL; > + } > + > + if (conf->pdcp_xfrm.hfn || conf->pdcp_xfrm.hfn_threshold) { > + rte_errno = -EINVAL; > + return NULL; > + } > + > + entity = rte_zmalloc_socket("pdcp_entity", pdcp_entity_size_get(conf), > + RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); > + if (entity == NULL) { > + rte_errno = -ENOMEM; > + return NULL; > + } > + > + en_priv = entity_priv_get(entity); > + > + en_priv->state.rx_deliv = conf->count; > + en_priv->state.tx_next = conf->count; > + en_priv->cop_pool = conf->cop_pool; > + > + /* Setup crypto session */ > + ret = pdcp_crypto_sess_create(entity, conf); > + if (ret) > + goto entity_free; > + > + ret = pdcp_process_func_set(entity, conf); > + if (ret) > + goto crypto_sess_destroy; > + > + return entity; > + > +crypto_sess_destroy: > + pdcp_crypto_sess_destroy(entity); > +entity_free: > + rte_free(entity); > + rte_errno = ret; > + return NULL; > +} > + > +int > +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf > *out_mb[]) > +{ > + int ret; > + > + if (pdcp_entity == NULL) > + return -EINVAL; > + > + /* Teardown crypto sessions */ > + ret = pdcp_crypto_sess_destroy(pdcp_entity); > + if (ret) > + return ret; > + > + rte_free(pdcp_entity); > + > + RTE_SET_USED(out_mb); > + return 0; > +} > + > +int > +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > + struct rte_mbuf *out_mb[]) > +{ > + struct entity_priv *en_priv; > + > + if (pdcp_entity == NULL) > + return -EINVAL; > + > + en_priv = entity_priv_get(pdcp_entity); > + > + if (en_priv->flags.is_ul_entity) { > + en_priv->state.tx_next = 0; > + } else { > + en_priv->state.rx_next = 0; > + en_priv->state.rx_deliv = 0; > + } > + > + RTE_SET_USED(out_mb); > + > + return 0; > +} > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h > new file mode 100644 > index 0000000000..b6c7f32c05 > --- /dev/null > +++ b/lib/pdcp/rte_pdcp.h > @@ -0,0 +1,263 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#ifndef _RTE_PDCP_H_ > +#define _RTE_PDCP_H_ > + > +/** > + * @file rte_pdcp.h > + * > + * RTE PDCP support. > + * > + * librte_pdcp provides a framework for PDCP protocol processing. > + */ > + > +#include <rte_compat.h> > +#include <rte_common.h> > +#include <rte_errno.h> > +#include <rte_mempool.h> > +#include <rte_security.h> Remove header file which is not needed. I do not see use of rte_errno.h I believe rte_common.h and rte_compat.h are also not needed. > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +/* Forward declarations */ > +struct rte_pdcp_entity; > + > +/* PDCP pre-process function based on entity configuration */ > +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *mb[], > + struct rte_crypto_op *cop[], > + uint16_t num, uint16_t *nb_err); > + > +/* PDCP post-process function based on entity configuration */ > +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err); > + > +/** > + * PDCP entity. > + */ > +struct rte_pdcp_entity { > + /** Entity specific pre-process handle. */ > + rte_pdcp_pre_p_t pre_process; > + /** Entity specific post-process handle. */ > + rte_pdcp_post_p_t post_process; > + /** > + * PDCP entities may hold packets for purposes of in-order delivery (in > + * case of receiving PDCP entity) and re-transmission (in case of > + * transmitting PDCP entity). > + * > + * For receiving PDCP entity, it may hold packets when in-order > + * delivery is enabled. The packets would be cached until either a > + * packet that completes the sequence arrives or when discard timer > + * expires. > + * > + * When post-processing of PDCP packet which completes a sequence is > + * done, the API may return more packets than enqueued. Application is > + * expected to provide *rte_pdcp_pkt_post_process()* with *out_mb* > + * which can hold maximum number of packets which may be returned. > + * > + * For transmitting PDCP entity, during re-establishment (5.1.2), > + * entity may be required to perform re-transmission of the buffers > + * after applying new ciphering & integrity algorithms. For performing > + * crypto operation, *rte_pdcp_entity_re_establish()* would return as > + * many crypto_ops as the ones cached. > + */ > + uint16_t max_pkt_cache; > + /** User area for saving application data. */ > + uint64_t user_area[2]; > +} __rte_cache_aligned; > + > +/** > + * PDCP entity configuration to be used for establishing an entity. > + */ > +struct rte_pdcp_entity_conf { > + /** PDCP transform for the entity. */ > + struct rte_security_pdcp_xform pdcp_xfrm; > + /** Crypto transform applicable for the entity. */ > + struct rte_crypto_sym_xform *crypto_xfrm; > + /** Mempool for crypto symmetric session. */ > + struct rte_mempool *sess_mpool; > + /** Crypto op pool.*/ > + struct rte_mempool *cop_pool; > + /** > + * 32 bit count value (HFN + SN) to be used for the first packet. > + * pdcp_xfrm.hfn would be ignored as the HFN would be derived from > this value. > + */ > + uint32_t count; > + /** Indicate whether the PDCP entity belongs to Side Link Radio Bearer. > */ > + bool is_slrb; > + /** Enable security offload on the device specified. */ > + bool en_sec_offload; > + /** Enable non-atomic usage of entity. */ > + bool en_non_atomic; > + /** Device on which security/crypto session need to be created. */ > + uint8_t dev_id; > + /** Reverse direction during IV generation. Can be used to simulate UE > crypto processing.*/ > + bool reverse_iv_direction; > +}; > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * 5.1.1 PDCP entity establishment > + * > + * Establish PDCP entity based on provided input configuration. > + * > + * @param conf > + * Parameters to be used for initializing PDCP entity object. > + * @return > + * - Valid handle if success > + * - NULL in case of failure. rte_errno will be set to error code > + */ > +__rte_experimental > +struct rte_pdcp_entity * > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * 5.1.3 PDCP entity release > + * > + * Release PDCP entity. > + * > + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. > + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in > + * *out_mb* buffer. The buffer should be large enough to hold all cached > + * packets in the entity. > + * > + * @param pdcp_entity > + * Pointer to the PDCP entity to be released. > + * @param[out] out_mb > + * The address of an array that can hold up to > *rte_pdcp_entity.max_pkt_cache* > + * pointers to *rte_mbuf* structures. > + * @return > + * - 0: Success and no cached packets to return > + * - >0: Success and the number of packets returned in out_mb > + * - <0: Error code in case of failures > + */ > +__rte_experimental > +int > +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, > + struct rte_mbuf *out_mb[]); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * 5.1.4 PDCP entity suspend > + * > + * Suspend PDCP entity. > + * > + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in > + * *out_mb* buffer. The buffer should be large enough to hold all cached > + * packets in the entity. > + * > + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. > + * > + * @param pdcp_entity > + * Pointer to the PDCP entity to be suspended. > + * @param[out] out_mb > + * The address of an array that can hold up to > *rte_pdcp_entity.max_pkt_cache* > + * pointers to *rte_mbuf* structures. > + * @return > + * - 0: Success and no cached packets to return > + * - >0: Success and the number of packets returned in out_mb > + * - <0: Error code in case of failures > + */ > +__rte_experimental > +int > +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > + struct rte_mbuf *out_mb[]); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * For input mbufs and given PDCP entity pre-process the mbufs and prepare > + * crypto ops that can be enqueued to the cryptodev associated with given > + * session. Only error packets would be moved returned in the input buffer, > + * *mb*, and it is the responsibility of the application to free the same. > + * > + * @param entity > + * Pointer to the *rte_pdcp_entity* object the packets belong to. > + * @param[in, out] mb > + * The address of an array of *num* pointers to *rte_mbuf* structures > + * which contain the input packets. Any error packets would be returned in the > + * same buffer. > + * @param[out] cop > + * The address of an array that can hold up to *num* pointers to > + * *rte_crypto_op* structures. Crypto ops would be allocated by > + * ``rte_pdcp_pkt_pre_process`` API. > + * @param num > + * The maximum number of packets to process. > + * @param[out] nb_err > + * Pointer to return the number of error packets returned in *mb* > + * @return > + * Count of crypto_ops prepared > + */ > +__rte_experimental > +static inline uint16_t > +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], > + uint16_t num, uint16_t *nb_err) > +{ > + return entity->pre_process(entity, mb, cop, num, nb_err); > +} > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * For input mbufs and given PDCP entity, perform PDCP post-processing of the > + * mbufs. > + * > + * Input mbufs are the ones retrieved from crypto_ops dequeued from > cryptodev > + * and grouped by *rte_pdcp_pkt_crypto_group()*. > + * > + * The post-processed packets would be returned in the *out_mb* buffer. > + * The resultant mbufs would be grouped into success packets and error > packets. > + * Error packets would be grouped in the end of the array and it is the > + * responsibility of the application to handle the same. > + * > + * When in-order delivery is enabled, PDCP entity may buffer packets and would > + * deliver packets only when all prior packets have been post-processed. That > + * would result in returning more/less packets than enqueued. > + * > + * @param entity > + * Pointer to the *rte_pdcp_entity* object the packets belong to. > + * @param in_mb > + * The address of an array of *num* pointers to *rte_mbuf* structures. > + * @param[out] out_mb > + * The address of an array of *num* pointers to *rte_mbuf* structures > + * to output packets after PDCP post-processing. > + * @param num > + * The maximum number of packets to process. > + * @param[out] nb_err > + * The number of error packets returned in *out_mb* buffer. > + * @return > + * Count of packets returned in *out_mb* buffer. > + */ > +__rte_experimental > +static inline uint16_t > +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], > + struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err) > +{ > + return entity->post_process(entity, in_mb, out_mb, num, nb_err); > +} > + > +#include <rte_pdcp_group.h> > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif /* _RTE_PDCP_H_ */ > diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h > new file mode 100644 > index 0000000000..2c01c19d4e > --- /dev/null > +++ b/lib/pdcp/rte_pdcp_group.h > @@ -0,0 +1,133 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2022 Marvell. > + */ > + > +#ifndef _RTE_PDCP_GROUP_H_ > +#define _RTE_PDCP_GROUP_H_ > + > +/** > + * @file rte_pdcp_group.h > + * > + * RTE PDCP grouping support. > + * It is not recommended to include this file directly, include <rte_pdcp.h> > + * instead. > + * Provides helper functions to process completed crypto-ops and group > related > + * packets by sessions they belong to. > + */ > + > +#include <rte_common.h> > +#include <rte_crypto.h> > +#include <rte_cryptodev.h> > +#include <rte_security.h> Remove header files which are not needed. > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +/** > + * Group packets belonging to same PDCP entity. > + */ > +struct rte_pdcp_group { > + union { > + uint64_t val; > + void *ptr; > + } id; /**< Grouped by value */ > + struct rte_mbuf **m; /**< Start of the group */ > + uint32_t cnt; /**< Number of entries in the group */ > + int32_t rc; /**< Status code associated with the group */ > +}; > + > +/** > + * Take crypto-op as an input and extract pointer to related PDCP entity. > + * @param cop > + * The address of an input *rte_crypto_op* structure. > + * @return > + * The pointer to the related *rte_pdcp_entity* structure. > + */ > +static inline struct rte_pdcp_entity * > +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) > +{ > + void *sess = cop->sym[0].session; > + > + if (cop->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { > + return (struct rte_pdcp_entity *)(uintptr_t) > + rte_security_session_opaque_data_get(sess); > + } else if (cop->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { > + return (struct rte_pdcp_entity *)(uintptr_t) > + rte_cryptodev_sym_session_opaque_data_get(sess); > + } This patchset is not supporting security sessions, so it would be better to return NULL for that. Moreover, we can directly call rte_cryptodev_sym_session_opaque_data_get(cop->sym[0].session) From rte_pdcp_pkt_crypto_group. No need to have a wrapper. > + > + return NULL; > +} > + > +/** > + * Take as input completed crypto ops, extract related mbufs and group them > by > + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has > + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* > flag > + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. > + * > + * Note that application must ensure only crypto-ops prepared by lib_pdcp is > + * provided back to @see rte_pdcp_pkt_crypto_group(). > + * > + * @param cop > + * The address of an array of *num* pointers to the input *rte_crypto_op* > + * structures. > + * @param[out] mb > + * The address of an array of *num* pointers to output *rte_mbuf* structures. > + * @param[out] grp > + * The address of an array of *num* to output *rte_pdcp_group* structures. > + * @param num > + * The maximum number of crypto-ops to process. > + * @return > + * Number of filled elements in *grp* array. > + * > + */ > +static inline uint16_t > +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf > *mb[], > + struct rte_pdcp_group grp[], uint16_t num) > +{ > + uint32_t i, j = 0, n = 0; > + void *ns, *ps = NULL; > + struct rte_mbuf *m; > + > + for (i = 0; i != num; i++) { > + m = cop[i]->sym[0].m_src; > + ns = cop[i]->sym[0].session; > + > + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; > + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) > + m->ol_flags |= > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; > + > + /* Different entity */ > + if (ps != ns) { > + > + /* Finalize open group and start a new one */ > + if (ps != NULL) { > + grp[n].cnt = mb + j - grp[n].m; > + n++; > + } > + > + /* Start new group */ > + grp[n].m = mb + j; > + ps = ns; > + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); > + } > + > + mb[j++] = m; > + rte_crypto_op_free(cop[i]); > + } > + > + /* Finalize last group */ > + if (ps != NULL) { > + grp[n].cnt = mb + j - grp[n].m; > + n++; > + } > + > + return n; > +} > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif /* _RTE_PDCP_GROUP_H_ */ > diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map > new file mode 100644 > index 0000000000..8fa9d5d7cc > --- /dev/null > +++ b/lib/pdcp/version.map > @@ -0,0 +1,13 @@ > +EXPERIMENTAL { > + global: > + > + # added in 22.11 Change to 23.03 > + rte_pdcp_entity_establish; > + rte_pdcp_entity_release; > + rte_pdcp_entity_suspend; > + > + rte_pdcp_pkt_post_process; > + rte_pdcp_pkt_pre_process; > + > + local: *; > +}; > -- > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH 2/5] lib: add pdcp protocol 2023-01-18 16:26 ` Akhil Goyal @ 2023-02-13 10:59 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-02-13 10:59 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Thanks for the review. Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Wednesday, January 18, 2023 9:57 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH 2/5] lib: add pdcp protocol > > Hi Anoob, > > Please see inline comments. > > Subject: [PATCH 2/5] lib: add pdcp protocol > > > > Add Packet Data Convergence Protocol (PDCP) processing library. > > > > The library is similar to lib_ipsec which provides IPsec processing > > capabilities in DPDK. > > > > PDCP would involve roughly the following options, > > 1. Transfer of user plane data > > 2. Transfer of control plane data > > 3. Header compression > > 4. Uplink data compression > > 5. Ciphering and integrity protection > > > > PDCP library provides following control path APIs that is used to > > configure various PDCP entities, > > 1. rte_pdcp_entity_establish() > > 2. rte_pdcp_entity_suspend() > > 3. rte_pdcp_entity_release() > > > > PDCP process is split into 2 parts. One before crypto processing > > (rte_pdcp_pkt_pre_process()) and one after crypto processing > > (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return > crypto > > operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() > > is added to help grouping crypto operations belonging to same entity. > > > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > > --- > > doc/api/doxy-api-index.md | 3 +- > > doc/api/doxy-api.conf.in | 1 + > > lib/meson.build | 1 + > > lib/pdcp/meson.build | 8 + > > lib/pdcp/pdcp_crypto.c | 240 ++++++++ > > lib/pdcp/pdcp_crypto.h | 20 + > > lib/pdcp/pdcp_entity.h | 218 +++++++ > > lib/pdcp/pdcp_process.c | 1195 > +++++++++++++++++++++++++++++++++++++ > > lib/pdcp/pdcp_process.h | 13 + > > lib/pdcp/rte_pdcp.c | 136 +++++ > > lib/pdcp/rte_pdcp.h | 263 ++++++++ > > lib/pdcp/rte_pdcp_group.h | 133 +++++ > > lib/pdcp/version.map | 13 + > > 13 files changed, 2243 insertions(+), 1 deletion(-) > > create mode 100644 lib/pdcp/meson.build > > create mode 100644 lib/pdcp/pdcp_crypto.c > > create mode 100644 lib/pdcp/pdcp_crypto.h > > create mode 100644 lib/pdcp/pdcp_entity.h > > create mode 100644 lib/pdcp/pdcp_process.c > > create mode 100644 lib/pdcp/pdcp_process.h > > create mode 100644 lib/pdcp/rte_pdcp.c > > create mode 100644 lib/pdcp/rte_pdcp.h > > create mode 100644 lib/pdcp/rte_pdcp_group.h > > create mode 100644 lib/pdcp/version.map > > > > diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md > > index ae4b107240..6014bee079 100644 > > --- a/doc/api/doxy-api-index.md > > +++ b/doc/api/doxy-api-index.md > > @@ -126,7 +126,8 @@ The public API headers are grouped by topics: > > [eCPRI](@ref rte_ecpri.h), > > [L2TPv2](@ref rte_l2tpv2.h), > > [PPP](@ref rte_ppp.h), > > - [PDCP hdr](@ref rte_pdcp_hdr.h) > > + [PDCP hdr](@ref rte_pdcp_hdr.h), > > + [PDCP](@ref rte_pdcp.h), > > > > - **QoS**: > > [metering](@ref rte_meter.h), > > diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in > > index f0886c3bd1..01314b087e 100644 > > --- a/doc/api/doxy-api.conf.in > > +++ b/doc/api/doxy-api.conf.in > > @@ -61,6 +61,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api- > > index.md \ > > @TOPDIR@/lib/net \ > > @TOPDIR@/lib/pcapng \ > > @TOPDIR@/lib/pci \ > > + @TOPDIR@/lib/pdcp \ > > @TOPDIR@/lib/pdump \ > > @TOPDIR@/lib/pipeline \ > > @TOPDIR@/lib/port \ > > diff --git a/lib/meson.build b/lib/meson.build > > index fd55925340..a827006d29 100644 > > --- a/lib/meson.build > > +++ b/lib/meson.build > > @@ -63,6 +63,7 @@ libraries = [ > > 'flow_classify', # flow_classify lib depends on pkt framework table lib > > 'graph', > > 'node', > > + 'pdcp', # pdcp lib depends on crypto and security > > ] > > > > optional_libs = [ > > diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build > > new file mode 100644 > > index 0000000000..a7f5a408cf > > --- /dev/null > > +++ b/lib/pdcp/meson.build > > @@ -0,0 +1,8 @@ > > +# SPDX-License-Identifier: BSD-3-Clause > > +# Copyright(C) 2022 Marvell. > > +# > Extra # here. > > Do we support compilation on Windows as well? > Check missing here. > [Anoob] Will add checks in next version. > > + > > +sources = files('pdcp_crypto.c', 'pdcp_process.c', 'rte_pdcp.c') > > +headers = files('rte_pdcp.h') > > Do we need to add the indirect header as well for > lib/pdcp/rte_pdcp_group.h? [Anoob] Yes. Will update in next version. > > > + > > +deps += ['security'] > > Crypto not needed as dependency? [Anoob] Yes. Will include other dependencies as well. > > > diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c > > new file mode 100644 > > index 0000000000..7ffb8a07a7 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_crypto.c > > @@ -0,0 +1,240 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#include <rte_crypto.h> > > +#include <rte_crypto_sym.h> > > +#include <rte_cryptodev.h> > > +#include <rte_pdcp.h> > > + > > +#include "pdcp_crypto.h" > > +#include "pdcp_entity.h" > > + > > +static int > > +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct > > rte_crypto_sym_xform *c_xfrm) > > +{ > > + const struct rte_cryptodev_symmetric_capability *cap; > > + struct rte_cryptodev_sym_capability_idx cap_idx; > > + int ret; > > + > > + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; > > + cap_idx.algo.cipher = c_xfrm->cipher.algo; > > + > > + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); > > + if (cap == NULL) > > + return -1; > > + > > + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm- > > >cipher.key.length, > > + c_xfrm- > > >cipher.iv.length); > > + > > + return ret; > > +} > > + > > +static int > > +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct > > rte_crypto_sym_xform *a_xfrm) > > +{ > > + const struct rte_cryptodev_symmetric_capability *cap; > > + struct rte_cryptodev_sym_capability_idx cap_idx; > > + int ret; > > + > > + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; > > + cap_idx.algo.auth = a_xfrm->auth.algo; > > + > > + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); > > + if (cap == NULL) > > + return -1; > > + > > + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm- > > >auth.key.length, > > + a_xfrm- > > >auth.digest_length, > > + a_xfrm->auth.iv.length); > > + > > + return ret; > > +} > > + > > +static int > > +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, > > + const struct rte_crypto_sym_xform *c_xfrm, > > + const struct rte_crypto_sym_xform > *a_xfrm, > > + bool is_auth_then_cipher) > > +{ > > + uint16_t ciph_iv_len, auth_digest_len, auth_iv_len; > > + int ret; > > + > > + /* > > + * Uplink means PDCP entity is configured for transmit. Downlink > means > > PDCP entity is > > + * configured for receive. When integrity protection is enabled, PDCP > > always performs > > + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt- > auth- > > verify for downlink). > > + * So for uplink, crypto chain would be auth-cipher while for downlink > it > > would be > > + * cipher-auth. > > + * > > + * When integrity protection is not required, xform would be cipher > only. > > + */ > > + > > + if (c_xfrm == NULL) > > + return -EINVAL; > > + > > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { > > + > > + /* With UPLINK, if auth is enabled, it should be before cipher > */ > > + if (a_xfrm != NULL && !is_auth_then_cipher) > > + return -EINVAL; > > + > > + /* With UPLINK, cipher operation must be encrypt */ > > + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) > > + return -EINVAL; > > + > > + /* With UPLINK, auth operation (if present) must be > generate */ > > + if (a_xfrm != NULL && a_xfrm->auth.op != > > RTE_CRYPTO_AUTH_OP_GENERATE) > > + return -EINVAL; > > + > > + } else if (conf->pdcp_xfrm.pkt_dir == > RTE_SECURITY_PDCP_DOWNLINK) > > { > > + > > + /* With DOWNLINK, if auth is enabled, it should be after > cipher > > */ > > + if (a_xfrm != NULL && is_auth_then_cipher) > > + return -EINVAL; > > + > > + /* With DOWNLINK, cipher operation must be decrypt */ > > + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) > > + return -EINVAL; > > + > > + /* With DOWNLINK, auth operation (if present) must be > verify > > */ > > + if (a_xfrm != NULL && a_xfrm->auth.op != > > RTE_CRYPTO_AUTH_OP_VERIFY) > > + return -EINVAL; > > + > > + } else { > > + return -EINVAL; > > + } > > + > > + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && > > + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && > > + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && > > + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) > > + return -EINVAL; > > + > > + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) > > + ciph_iv_len = 0; > > + else > > + ciph_iv_len = PDCP_IV_LENGTH; > > + > > + if (ciph_iv_len != c_xfrm->cipher.iv.length) > > + return -EINVAL; > > + > > + if (a_xfrm != NULL) { > > + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && > > + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) > && > > + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && > > + (a_xfrm->auth.algo != > RTE_CRYPTO_AUTH_SNOW3G_UIA2)) > > + return -EINVAL; > > + > > + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) > > + auth_digest_len = 0; > > + else > > + auth_digest_len = 4; > > If we have a macro for IV length, why not for digest also? [Anoob] Will add macro for digest length. > Moreover, for NULL integrity, digest length is also 4 with all 0s. > Refer Annex D.1 in > https://www.etsi.org/deliver/etsi_ts/133500_133599/133501/15.04.00_60/ts > _133501v150400p.pdf > > Digest len would be 0 only in case of a_xfrm == NULL [Anoob] I agree that PDCP requires AUTH NULL to add all 0s. And we satisfy the same by having lib PDCP add zeros. Most of the crypto drivers in DPDK doesn't allow non-zero digest length for NULL auth. The above change was added to make sure the library works well with existing drivers. > > > + > > + if (auth_digest_len != a_xfrm->auth.digest_length) > > + return -EINVAL; > > + > > + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || > > + (a_xfrm->auth.algo == > RTE_CRYPTO_AUTH_SNOW3G_UIA2)) > > + auth_iv_len = PDCP_IV_LENGTH; > > + else > > + auth_iv_len = 0; > > + > > + if (a_xfrm->auth.iv.length != auth_iv_len) > > + return -EINVAL; > > + } > > + > > + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) > > + return -EINVAL; > > + > > + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); > > + if (ret) > > + return -ENOTSUP; > > + > > + if (a_xfrm != NULL) { > > + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); > > + if (ret) > > + return -ENOTSUP; > > + } > > + > > + return 0; > > +} > > + > > +int > > +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > > + struct entity_priv *en_priv; > > + bool is_auth_then_cipher; > > + int ret; > > + > > + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) > > + return -EINVAL; > > + > > + en_priv = entity_priv_get(entity); > > + > > + en_priv->dev_id = conf->dev_id; > > + > > + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) > { > > + c_xfrm = conf->crypto_xfrm; > > + a_xfrm = conf->crypto_xfrm->next; > > + is_auth_then_cipher = false; > > + } else if (conf->crypto_xfrm->type == > > RTE_CRYPTO_SYM_XFORM_AUTH) { > > + a_xfrm = conf->crypto_xfrm; > > + c_xfrm = conf->crypto_xfrm->next; > > + is_auth_then_cipher = true; > > + } else { > > + return -EINVAL; > > + } > > + > > + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, > > is_auth_then_cipher); > > + if (ret) > > + return ret; > > + > > + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) > > + c_xfrm->cipher.iv.offset = 0; > > + else > > + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; > > + > > + if (a_xfrm != NULL) { > > + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) > > + a_xfrm->auth.iv.offset = 0; > > + else > > + if (c_xfrm->cipher.iv.offset) > > + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + > > PDCP_IV_LENGTH; > > + else > > + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; > > + } > > + > > + if (conf->sess_mpool == NULL) > > + return -EINVAL; > > + > > + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf- > > >dev_id, conf->crypto_xfrm, > > + conf- > > >sess_mpool); > > + if (en_priv->crypto_sess == NULL) { > > + /* API returns positive values as error codes */ > > + return -rte_errno; > > + } > > + > > + rte_cryptodev_sym_session_opaque_data_set(en_priv- > >crypto_sess, > > (uint64_t)entity); > > + > > + return 0; > > +} > > + > > +int > > +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) > > +{ > > + struct entity_priv *en_priv; > > + > > + if (entity == NULL) > > + return -EINVAL; > > + > > + en_priv = entity_priv_get(entity); > > + > > + if (en_priv->crypto_sess != NULL) { > > + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv- > > >crypto_sess); > > + en_priv->crypto_sess = NULL; > > + } > > + > > + return 0; > > +} > > diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h > > new file mode 100644 > > index 0000000000..dc625b35d0 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_crypto.h > > @@ -0,0 +1,20 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#ifndef _PDCP_CRYPTO_H_ > > +#define _PDCP_CRYPTO_H_ > > + > > +#include <rte_crypto.h> > > +#include <rte_crypto_sym.h> > > +#include <rte_pdcp.h> > > + > > +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct > > rte_crypto_sym_op)) > > +#define PDCP_IV_LENGTH 16 > > + > > +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, > > + const struct rte_pdcp_entity_conf *conf); > > + > > +int pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); > > + > > +#endif /* _PDCP_CRYPTO_H_ */ > > diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h > > new file mode 100644 > > index 0000000000..e312fd4a8c > > --- /dev/null > > +++ b/lib/pdcp/pdcp_entity.h > > @@ -0,0 +1,218 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#ifndef _PDCP_ENTITY_H_ > > +#define _PDCP_ENTITY_H_ > > + > > +#include <rte_common.h> > > +#include <rte_crypto_sym.h> > > +#include <rte_mempool.h> > > +#include <rte_pdcp.h> > > +#include <rte_security.h> > > + > > +struct entity_priv; > > + > > +#define PDCP_PDU_HDR_SIZE_SN_12 (RTE_ALIGN_MUL_CEIL(12, 8) / 8) > > +#define PDCP_PDU_HDR_SIZE_SN_18 (RTE_ALIGN_MUL_CEIL(18, 8) / 8) > > + > > +#define PDCP_GET_SN_12_FROM_COUNT(c) ((c) & 0xfff) > > +#define PDCP_GET_SN_18_FROM_COUNT(c) ((c) & 0x3ffff) > > + > > +#define PDCP_GET_HFN_SN_12_FROM_COUNT(c) (((c) >> 12) & 0xfffff) > > +#define PDCP_GET_HFN_SN_18_FROM_COUNT(c) (((c) >> 18) & 0x3fff) > > + > > +#define PDCP_SET_COUNT_FROM_HFN_SN_12(h, s) ((((h) & 0xfffff) << > 12) | ((s) > > & 0xfff)) > > +#define PDCP_SET_COUNT_FROM_HFN_SN_18(h, s) ((((h) & 0x3fff) << > 18) | ((s) > > & 0x3ffff)) > > + > > +#define PDCP_SN_12_WINDOW_SZ 0x800 > > +#define PDCP_SN_18_WINDOW_SZ 0x20000 > > + > > +#define PDCP_SN_12_HFN_MAX ((1 << (32 - 12)) - 1) > > +#define PDCP_SN_12_HFN_MIN 0 > > +#define PDCP_SN_18_HFN_MAX ((1 << (32 - 18)) - 1) > > +#define PDCP_SN_18_HFN_MIN 0 > > + > > Can we have common defines for SN-12 and SN-18 and take SN as > parameter? > We can have something like this. > > #define PDCP_PDU_HDR_SIZE(sn_size) (RTE_ALIGN_MUL_CEIL((sn_size), 8) > / 8) > #define PDCP_GET_SN_FROM_COUNT(c, sn_size) ((c) & ((1<<sn_size)-1)) > #define PDCP_GET_HFN_FROM_COUNT(c, sn_size) (((c) >> sn_size) & ((1 << > (32 - sn_size)) - 1)) > #define PDCP_SET_COUNT_FROM_HFN_SN(h, s, sn_size) ((((h) & ((1 << (32 > - sn_size)) - 1)) << sn_size) | ((s) & ((1<<sn_size)-1))) > #define PDCP_HFN_MAX(sn_size) ((1 << (32 - (sn_size))) - 1) > #define PDCP_HFN_MIN 0 > [Anoob] Agreed. Will replace these macros with generic static inlines. > > +/* IV generation function based on the entity configuration */ > > +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct > entity_priv > > *en_priv, > > + uint32_t count); > > + > > +enum pdcp_pdu_type { > > + PDCP_PDU_TYPE_CTRL = 0, > > + PDCP_PDU_TYPE_DATA = 1, > > +}; > > + > > +enum pdcp_up_ctrl_pdu_type { > > + PDCP_UP_CTRL_PDU_TYPE_STATUS_REPORT, > > + PDCP_UP_CTRL_PDU_TYPE_ROHC_FEEDBACK, > > + PDCP_UP_CTRL_PDU_TYPE_EHC_FEEDBACK, > > + PDCP_UP_CRTL_PDU_TYPE_UDC_FEEDBACK > > +}; > > + > > +struct entity_state { > > + uint32_t rx_next; > > + uint32_t tx_next; > > + uint32_t rx_deliv; > > + uint32_t rx_reord; > > +}; > > + > > +union auth_iv_partial { > > + /* For AES-CMAC, there is no IV, but message gets prepended */ > > + struct { > > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > > + uint64_t count : 32; > > + uint64_t zero_38_39 : 2; > > + uint64_t direction : 1; > > + uint64_t bearer : 5; > > + uint64_t zero_40_63 : 24; > > +#else > > + uint64_t count : 32; > > + uint64_t bearer : 5; > > + uint64_t direction : 1; > > + uint64_t zero_38_39 : 2; > > + uint64_t zero_40_63 : 24; > > +#endif > > + } aes_cmac; > > + struct { > > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > > + uint64_t count : 32; > > + uint64_t zero_37_39 : 3; > > + uint64_t bearer : 5; > > + uint64_t zero_40_63 : 24; > > + > > + uint64_t rsvd_65_71 : 7; > > + uint64_t direction_64 : 1; > > + uint64_t rsvd_72_111 : 40; > > + uint64_t rsvd_113_119 : 7; > > + uint64_t direction_112 : 1; > > + uint64_t rsvd_120_127 : 8; > > +#else > > + uint64_t count : 32; > > + uint64_t bearer : 5; > > + uint64_t zero_37_39 : 3; > > + uint64_t zero_40_63 : 24; > > + > > + uint64_t direction_64 : 1; > > + uint64_t rsvd_65_71 : 7; > > + uint64_t rsvd_72_111 : 40; > > + uint64_t direction_112 : 1; > > + uint64_t rsvd_113_119 : 7; > > + uint64_t rsvd_120_127 : 8; > > +#endif > > + } zs; > > + uint64_t u64[2]; > > +}; > > + > > +union cipher_iv_partial { > > + struct { > > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > > + uint64_t count : 32; > > + uint64_t zero_38_39 : 2; > > + uint64_t direction : 1; > > + uint64_t bearer : 5; > > + uint64_t zero_40_63 : 24; > > + > > + uint64_t zero_64_127; > > +#else > > + uint64_t count : 32; > > + uint64_t bearer : 5; > > + uint64_t direction : 1; > > + uint64_t zero_38_39 : 2; > > + uint64_t zero_40_63 : 24; > > + > > + uint64_t zero_64_127; > > Can we take zero_64_127 out of #if-else [Anoob] Agreed. Will address in next version. > > > +#endif > > + } aes_ctr; > > + struct { > > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > > + uint64_t count : 32; > > + uint64_t zero_38_39 : 2; > > + uint64_t direction : 1; > > + uint64_t bearer : 5; > > + uint64_t zero_40_63 : 24; > > + > > + uint64_t rsvd_64_127; > > +#else > > + uint64_t count : 32; > > + uint64_t bearer : 5; > > + uint64_t direction : 1; > > + uint64_t zero_38_39 : 2; > > + uint64_t zero_40_63 : 24; > > + > > + uint64_t rsvd_64_127; > > +#endif > > rsvd_64_127 can also be out of #if-else > > > + } zs; > > + uint64_t u64[2]; > > +}; > > + > > +/* > > + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] > > If the layout is fixed, can we have 0 length array in rte_pdcp_entity for > entity_priv and in entity_priv for entity_dl/ul. > I see that entity_dl/ul are not completely defined. You can define them later > when they are supported. [Anoob] The implementation will be part of next version. > > > + */ > > + > > Extra line [Anoob] The description was not just for the following struct, but the section (3 structures basically). Hence kept a blank line in between. > > > +struct entity_priv { > > + /** Crypto sym session. */ > > + struct rte_cryptodev_sym_session *crypto_sess; > > + /** Entity specific IV generation function. */ > > + iv_gen_t iv_gen; > > + /** Pre-prepared auth IV. */ > > + union auth_iv_partial auth_iv_part; > > + /** Pre-prepared cipher IV. */ > > + union cipher_iv_partial cipher_iv_part; > > + /** Entity state variables. */ > > + struct entity_state state; > > + /** Flags. */ > > + struct { > > + /** PDCP PDU has 4 byte MAC-I. */ > > + uint64_t is_authenticated : 1; > > + /** Cipher offset & length in bits. */ > > + uint64_t is_ciph_in_bits : 1; > > + /** Auth offset & length in bits. */ > > + uint64_t is_auth_in_bits : 1; > > + /** Is UL/transmitting PDCP entity */ > > + uint64_t is_ul_entity : 1; > > + } flags; > > + /** Crypto op pool. */ > > + struct rte_mempool *cop_pool; > > + /** PDCP header size. */ > > + uint8_t hdr_sz; > > + /** PDCP AAD size. For AES-CMAC, additional message is prepended > for > > the operation. */ > > + uint8_t aad_sz; > > + /** Device ID of the device to be used for offload. */ > > + uint8_t dev_id; > > +}; > > + > > +struct entity_priv_dl_part { > > + /* TODO - when in-order-delivery is supported, post PDCP packets > > would need to cached. */ > > + uint8_t dummy; > > +}; > > + > > +struct entity_priv_ul_part { > > + /* > > + * TODO - when re-establish is supported, both plain & post PDCP > > packets would need to be > > + * cached. > > + */ > > + uint8_t dummy; > > +}; > > + > > +static inline struct entity_priv * > > +entity_priv_get(const struct rte_pdcp_entity *entity) { > > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); > > +} > > + > > +static inline struct entity_priv_dl_part * > > +entity_dl_part_get(const struct rte_pdcp_entity *entity) { > > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + > > sizeof(struct entity_priv)); > > +} > > + > > +static inline struct entity_priv_ul_part * > > +entity_ul_part_get(const struct rte_pdcp_entity *entity) { > > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + > > sizeof(struct entity_priv)); > > +} > > Above inline functions may not be needed also if we have 0 len arrays. > > > + > > +static inline int > > +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) > > +{ > > + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; > > +} > > PDCP_PDU_HDR_SIZE is same as this inline function. > Can we get away with this one? [Anoob] Removed the macro. > > > + > > +#endif /* _PDCP_ENTITY_H_ */ > > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c > > new file mode 100644 > > index 0000000000..282cf38ec4 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_process.c > > @@ -0,0 +1,1195 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#include <rte_crypto.h> > > +#include <rte_crypto_sym.h> > > +#include <rte_cryptodev.h> > > +#include <rte_memcpy.h> > > +#include <rte_pdcp.h> > > +#include <rte_pdcp_hdr.h> > > + > > +#include "pdcp_crypto.h" > > +#include "pdcp_entity.h" > > +#include "pdcp_process.h" > > + > > +#define PDCP_MAC_I_LEN 4 > > Can you define it at same place where PDCP_IV_LENGTH is defined and use > this in xform validate. [Anoob] Moved to pdcp_crypto.h > > > + > > +/* Enum of supported algorithms for ciphering */ > > +enum pdcp_cipher_algo { > > + PDCP_CIPHER_ALGO_NULL, > > + PDCP_CIPHER_ALGO_AES, > > + PDCP_CIPHER_ALGO_ZUC, > > + PDCP_CIPHER_ALGO_SNOW3G, > > + PDCP_CIPHER_ALGO_MAX > > +}; > > + > > +/* Enum of supported algorithms for integrity */ > > +enum pdcp_auth_algo { > > + PDCP_AUTH_ALGO_NULL, > > + PDCP_AUTH_ALGO_AES, > > + PDCP_AUTH_ALGO_ZUC, > > + PDCP_AUTH_ALGO_SNOW3G, > > + PDCP_AUTH_ALGO_MAX > > +}; > > + > > +/* IV generation functions based on type of operation (cipher - auth) */ > > + > > +static void > > +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct > entity_priv > > *en_priv, uint32_t count) > > +{ > > + /* No IV required for NULL cipher + NULL auth */ > > + RTE_SET_USED(cop); > > + RTE_SET_USED(en_priv); > > + RTE_SET_USED(count); > > +} > > + > > +static void > > +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct > > entity_priv *en_priv, > > + uint32_t count) > > +{ > > + struct rte_crypto_sym_op *op = cop->sym; > > + struct rte_mbuf *mb = op->m_src; > > + uint8_t *m_ptr; > > + uint64_t m; > > + > > + /* AES-CMAC requires message to be prepended with info on count > etc > > */ > > + > > + /* Prepend by 8 bytes to add custom message */ > > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > > + > > + m = en_priv->auth_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + > > + rte_memcpy(m_ptr, &m, 8); > > +} > > + > > +static void > > +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv > > *en_priv, uint32_t count) > > +{ > > + uint64_t iv_u64[2]; > > + uint8_t *iv; > > + > > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > > + > > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + rte_memcpy(iv, &iv_u64[0], 8); > > + > > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > > + rte_memcpy(iv + 8, &iv_u64[1], 8); > > +} > > + > > +static void > > +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct > entity_priv > > *en_priv, > > + uint32_t count) > > +{ > > + uint64_t iv_u64[2]; > > + uint8_t *iv; > > + > > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > > + > > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + iv_u64[1] = 0; > > + rte_memcpy(iv, iv_u64, 16); > > +} > > + > > +static void > > +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv > > *en_priv, uint32_t count) > > +{ > > + uint64_t iv_u64; > > + uint8_t *iv; > > + > > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > > + > > + iv_u64 = en_priv->cipher_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + rte_memcpy(iv, &iv_u64, 8); > > + rte_memcpy(iv + 8, &iv_u64, 8); > > +} > > + > > +static void > > +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv > > *en_priv, uint32_t count) > > +{ > > + uint64_t iv_u64[2]; > > + uint8_t *iv; > > + > > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > > + > > + /* Generating cipher IV */ > > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + rte_memcpy(iv, &iv_u64[0], 8); > > + rte_memcpy(iv + 8, &iv_u64[0], 8); > > + > > + iv += PDCP_IV_LENGTH; > > + > > + /* Generating auth IV */ > > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + rte_memcpy(iv, &iv_u64[0], 8); > > + > > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > > + rte_memcpy(iv + 8, &iv_u64[1], 8); > > +} > > + > > +static void > > +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct > entity_priv > > *en_priv, > > + uint32_t count) > > +{ > > + struct rte_crypto_sym_op *op = cop->sym; > > + struct rte_mbuf *mb = op->m_src; > > + uint8_t *m_ptr, *iv; > > + uint64_t iv_u64[2]; > > + uint64_t m; > > + > > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + rte_memcpy(iv, &iv_u64[0], 8); > > + rte_memcpy(iv + 8, &iv_u64[0], 8); > > + > > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > > + m = en_priv->auth_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + rte_memcpy(m_ptr, &m, 8); > > +} > > + > > +static void > > +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct > > entity_priv *en_priv, > > + uint32_t count) > > +{ > > + struct rte_crypto_sym_op *op = cop->sym; > > + struct rte_mbuf *mb = op->m_src; > > + uint8_t *m_ptr, *iv; > > + uint64_t iv_u64[2]; > > + uint64_t m; > > + > > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > > + > > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + iv_u64[1] = 0; > > + rte_memcpy(iv, iv_u64, PDCP_IV_LENGTH); > > + > > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > > + m = en_priv->auth_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + rte_memcpy(m_ptr, &m, 8); > > +} > > + > > +static void > > +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct > entity_priv > > *en_priv, uint32_t count) > > +{ > > + uint64_t iv_u64[2]; > > + uint8_t *iv; > > + > > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > > + > > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + iv_u64[1] = 0; > > + rte_memcpy(iv, iv_u64, PDCP_IV_LENGTH); > > + > > + iv += PDCP_IV_LENGTH; > > + > > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > > ((uint64_t)(rte_cpu_to_be_32(count))); > > + rte_memcpy(iv, &iv_u64[0], 8); > > + > > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > > + rte_memcpy(iv + 8, &iv_u64[1], 8); > > +} > > + > > +static int > > +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct > > rte_crypto_sym_xform **c_xfrm, > > + struct rte_crypto_sym_xform **a_xfrm) > > +{ > > + *c_xfrm = NULL; > > + *a_xfrm = NULL; > > + > > + if (conf->crypto_xfrm == NULL) > > + return -EINVAL; > > + > > + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) > { > > + *c_xfrm = conf->crypto_xfrm; > > + *a_xfrm = conf->crypto_xfrm->next; > > + } else if (conf->crypto_xfrm->type == > > RTE_CRYPTO_SYM_XFORM_AUTH) { > > + *a_xfrm = conf->crypto_xfrm; > > + *c_xfrm = conf->crypto_xfrm->next; > > + } else { > > + return -EINVAL; > > + } > > + > > + return 0; > > +} > > + > > +static int > > +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > > + enum rte_security_pdcp_direction direction; > > + enum pdcp_cipher_algo ciph_algo; > > + enum pdcp_auth_algo auth_algo; > > + struct entity_priv *en_priv; > > + int ret; > > + > > + en_priv = entity_priv_get(entity); > > + > > + direction = conf->pdcp_xfrm.pkt_dir; > > + if (conf->reverse_iv_direction) > > + direction = !direction; > > + > > + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); > > + if (ret) > > + return ret; > > + > > + if (c_xfrm == NULL) > > + return -EINVAL; > > + > > + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); > > + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv- > >cipher_iv_part)); > > + > > + switch (c_xfrm->cipher.algo) { > > + case RTE_CRYPTO_CIPHER_NULL: > > + ciph_algo = PDCP_CIPHER_ALGO_NULL; > > + break; > > + case RTE_CRYPTO_CIPHER_AES_CTR: > > + ciph_algo = PDCP_CIPHER_ALGO_AES; > > + en_priv->cipher_iv_part.aes_ctr.bearer = conf- > > >pdcp_xfrm.bearer; > > + en_priv->cipher_iv_part.aes_ctr.direction = direction; > > + break; > > + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: > > + ciph_algo = PDCP_CIPHER_ALGO_SNOW3G; > > + en_priv->cipher_iv_part.zs.bearer = conf- > >pdcp_xfrm.bearer; > > + en_priv->cipher_iv_part.zs.direction = direction; > > + break; > > + case RTE_CRYPTO_CIPHER_ZUC_EEA3: > > + ciph_algo = PDCP_CIPHER_ALGO_ZUC; > > + en_priv->cipher_iv_part.zs.bearer = conf- > >pdcp_xfrm.bearer; > > + en_priv->cipher_iv_part.zs.direction = direction; > > + break; > > + default: > > + return -ENOTSUP; > > + } > > + > > + if (a_xfrm != NULL) { > > + switch (a_xfrm->auth.algo) { > > + case RTE_CRYPTO_AUTH_NULL: > > + auth_algo = PDCP_AUTH_ALGO_NULL; > > + break; > > + case RTE_CRYPTO_AUTH_AES_CMAC: > > + auth_algo = PDCP_AUTH_ALGO_AES; > > + en_priv->auth_iv_part.aes_cmac.bearer = conf- > > >pdcp_xfrm.bearer; > > + en_priv->auth_iv_part.aes_cmac.direction = > direction; > > + break; > > + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: > > + auth_algo = PDCP_AUTH_ALGO_SNOW3G; > > + en_priv->auth_iv_part.zs.bearer = conf- > > >pdcp_xfrm.bearer; > > + en_priv->auth_iv_part.zs.direction_64 = direction; > > + en_priv->auth_iv_part.zs.direction_112 = direction; > > + break; > > + case RTE_CRYPTO_AUTH_ZUC_EIA3: > > + auth_algo = PDCP_AUTH_ALGO_ZUC; > > + en_priv->auth_iv_part.zs.bearer = conf- > > >pdcp_xfrm.bearer; > > + en_priv->auth_iv_part.zs.direction_64 = direction; > > + en_priv->auth_iv_part.zs.direction_112 = direction; > > + break; > > + default: > > + return -ENOTSUP; > > + } > > + } else { > > + auth_algo = PDCP_AUTH_ALGO_NULL; > > + } > > + > > + static const iv_gen_t > > iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { > > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = > > pdcp_iv_gen_null_null, > > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = > > pdcp_iv_gen_null_aes_cmac, > > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] > = > > pdcp_iv_gen_null_zs, > > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = > > pdcp_iv_gen_null_zs, > > + > > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = > > pdcp_iv_gen_aes_ctr_null, > > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = > > pdcp_iv_gen_aes_ctr_aes_cmac, > > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = > > pdcp_iv_gen_aes_ctr_zs, > > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = > > pdcp_iv_gen_aes_ctr_zs, > > + > > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] > = > > pdcp_iv_gen_zs_null, > > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = > > pdcp_iv_gen_zs_aes_cmac, > > + > > [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = > > pdcp_iv_gen_zs_zs, > > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] > = > > pdcp_iv_gen_zs_zs, > > + > > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = > > pdcp_iv_gen_zs_null, > > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = > > pdcp_iv_gen_zs_aes_cmac, > > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] > = > > pdcp_iv_gen_zs_zs, > > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = > > pdcp_iv_gen_zs_zs, > > + }; > > + > > + en_priv->iv_gen = iv_gen_map[ciph_algo][auth_algo]; > > + > > + return 0; > > +} > > + > > +static inline void > > +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, > struct > > rte_crypto_op *cop, > > + uint8_t data_offset, uint32_t count, const bool is_auth) > > +{ > > + const struct rte_crypto_op cop_init = { > > + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, > > + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, > > + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, > > + }; > > + struct rte_crypto_sym_op *op; > > + uint32_t pkt_len; > > + > > + const uint8_t ciph_shift = 3 * en_priv->flags.is_ciph_in_bits; > > + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; > > + > > + op = cop->sym; > > + cop->raw = cop_init.raw; > > + op->m_src = mb; > > + op->m_dst = mb; > > + > > + /* Set IV */ > > + en_priv->iv_gen(cop, en_priv, count); > > + > > + /* Prepare op */ > > + pkt_len = rte_pktmbuf_pkt_len(mb); > > + op->cipher.data.offset = data_offset << ciph_shift; > > + op->cipher.data.length = (pkt_len - data_offset) << ciph_shift; > > + > > + if (is_auth) { > > + op->auth.data.offset = 0; > > + op->auth.data.length = (pkt_len - PDCP_MAC_I_LEN) << > > auth_shift; > > + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, > uint8_t > > *, > > + (pkt_len - > > PDCP_MAC_I_LEN)); > > + } > > + > > + __rte_crypto_sym_op_attach_sym_session(op, en_priv- > >crypto_sess); > > +} > > + > > +static inline bool > > +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, > struct > > rte_mbuf *mb, > > + uint32_t *count) > > +{ > > + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; > > + const uint8_t hdr_sz = en_priv->hdr_sz; > > + uint32_t sn; > > + > > + /* Prepend PDU header */ > > + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr > > *)rte_pktmbuf_prepend(mb, hdr_sz); > > + if (unlikely(pdu_hdr == NULL)) > > + return false; > > + > > + /* Update sequence num in the PDU header */ > > + *count = __atomic_fetch_add(&en_priv->state.tx_next, 1, > > __ATOMIC_RELAXED); > > + sn = PDCP_GET_SN_12_FROM_COUNT(*count); > > + > > + pdu_hdr->d_c = PDCP_PDU_TYPE_DATA; > > + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); > > + pdu_hdr->sn_7_0 = (sn & 0xff); > > + pdu_hdr->r = 0; > > + return true; > > +} > > + > > +static uint16_t > > +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity > *entity, > > struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t num, > > uint16_t *nb_err) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + uint16_t nb_cop; > > + uint32_t count; > > + int i; > > + > > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > > + > > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > > + num); > > + > > + if (en_priv->flags.is_authenticated) { > > + for (i = 0; i < nb_cop; i++) { > > + if (unlikely(rte_pktmbuf_append(mb[i], > > PDCP_MAC_I_LEN) == NULL)) > > + goto cop_free; > > + if > > (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb[i], > > + > > &count))) > > + goto cop_free; > > + cop_prepare(en_priv, mb[i], cop[i], data_offset, > count, > > true); > > + } > > + } else { > > + for (i = 0; i < nb_cop; i++) { > > + if > > (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb[i], > > + > > &count))) > > + goto cop_free; > > + cop_prepare(en_priv, mb[i], cop[i], data_offset, > count, > > false); > > + } > > + } > > + > > + *nb_err = num - nb_cop; > > + return nb_cop; > > +cop_free: > > + /* Using mempool API since crypto API is not providing bulk free */ > > + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop > - i); > > + *nb_err = num - i; > > + return i; > > +} > > + > > +static inline bool > > +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, > struct > > rte_mbuf *mb, > > + uint32_t *count) > > +{ > > + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; > > + const uint8_t hdr_sz = en_priv->hdr_sz; > > + uint32_t sn; > > + > > + /* Prepend PDU header */ > > + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr > > *)rte_pktmbuf_prepend(mb, hdr_sz); > > + if (unlikely(pdu_hdr == NULL)) > > + return false; > > + > > + /* Update sequence num in the PDU header */ > > + *count = __atomic_fetch_add(&en_priv->state.tx_next, 1, > > __ATOMIC_RELAXED); > > + sn = PDCP_GET_SN_18_FROM_COUNT(*count); > > + > > + pdu_hdr->d_c = PDCP_PDU_TYPE_DATA; > > + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); > > + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); > > + pdu_hdr->sn_7_0 = (sn & 0xff); > > + pdu_hdr->r = 0; > > + > > + return true; > > +} > > + > > +static inline uint16_t > > +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity > *entity, > > struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t num, > > uint16_t *nb_err) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + uint16_t nb_cop; > > + uint32_t count; > > + int i; > > + > > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > > + > > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > > + num); > > + > > + if (en_priv->flags.is_authenticated) { > > + for (i = 0; i < nb_cop; i++) { > > + if (unlikely(rte_pktmbuf_append(mb[i], > > PDCP_MAC_I_LEN) == NULL)) > > + goto cop_free; > > + if > > (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb[i], > > + > > &count))) > > + goto cop_free; > > + cop_prepare(en_priv, mb[i], cop[i], data_offset, > count, > > true); > > + } > > + } else { > > + for (i = 0; i < nb_cop; i++) { > > + if > > (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb[i], > > + > > &count))) > > + goto cop_free; > > + cop_prepare(en_priv, mb[i], cop[i], data_offset, > count, > > false); > > + } > > + } > > + > > + *nb_err = num - nb_cop; > > + return nb_cop; > > + > > +cop_free: > > + /* Using mempool API since crypto API is not providing bulk free */ > > + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop > - i); > > + *nb_err = num - i; > > + return i; > > +} > > + > > +static uint16_t > > +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity > *entity, > > struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t num, > > uint16_t *nb_err) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; > > + uint32_t count, sn; > > + uint16_t nb_cop; > > + int i; > > + > > + const uint8_t hdr_sz = en_priv->hdr_sz; > > + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; > > + > > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > > + num); > > + > > + for (i = 0; i < nb_cop; i++) { > > + /* Prepend PDU header */ > > + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr > > *)rte_pktmbuf_prepend(mb[i], > > + > > hdr_sz); > > + if (unlikely(pdu_hdr == NULL)) > > + goto cop_free; > > + if (unlikely(rte_pktmbuf_append(mb[i], PDCP_MAC_I_LEN) > == > > NULL)) > > + goto cop_free; > > + > > + /* Update sequence number in the PDU header */ > > + count = __atomic_fetch_add(&en_priv->state.tx_next, 1, > > __ATOMIC_RELAXED); > > + sn = PDCP_GET_SN_12_FROM_COUNT(count); > > + > > + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); > > + pdu_hdr->sn_7_0 = (sn & 0xff); > > + pdu_hdr->r = 0; > > + > > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > true); > > + } > > + > > + *nb_err = num - nb_cop; > > + return nb_cop; > > + > > +cop_free: > > + /* Using mempool API since crypto API is not providing bulk free */ > > + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[i], nb_cop > - i); > > + *nb_err = num - i; > > + return i; > > +} > > + > > +static uint16_t > > +pdcp_post_process_uplane_sn_12_ul(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err_ret) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + const uint32_t hdr_trim_sz = en_priv->aad_sz; > > + int i, nb_success = 0, nb_err = 0; > > + struct rte_mbuf *err_mb[num]; > > + struct rte_mbuf *mb; > > + > > + for (i = 0; i < num; i++) { > > + mb = in_mb[i]; > > + if (unlikely(mb->ol_flags & > > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { > > + err_mb[nb_err++] = mb; > > + continue; > > + } > > + > > + if (hdr_trim_sz) > > + rte_pktmbuf_adj(mb, hdr_trim_sz); > > + > > + out_mb[nb_success++] = mb; > > + } > > + > > + if (unlikely(nb_err != 0)) > > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > > sizeof(struct rte_mbuf *)); > > + > > + *nb_err_ret = nb_err; > > + return nb_success; > > +} > > + > > +static uint16_t > > +pdcp_post_process_uplane_sn_18_ul(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err_ret) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + const uint32_t hdr_trim_sz = en_priv->aad_sz; > > + int i, nb_success = 0, nb_err = 0; > > + struct rte_mbuf *err_mb[num]; > > + struct rte_mbuf *mb; > > + > > + for (i = 0; i < num; i++) { > > + mb = in_mb[i]; > > + if (unlikely(mb->ol_flags & > > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { > > + err_mb[nb_err++] = mb; > > + continue; > > + } > > + > > + if (hdr_trim_sz) > > + rte_pktmbuf_adj(mb, hdr_trim_sz); > > + > > + out_mb[nb_success++] = mb; > > + } > > + > > + if (unlikely(nb_err != 0)) > > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > > sizeof(struct rte_mbuf *)); > > + > > + *nb_err_ret = nb_err; > > + return nb_success; > > +} > > + > > +static uint16_t > > +pdcp_post_process_cplane_sn_12_ul(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err_ret) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + const uint32_t hdr_trim_sz = en_priv->aad_sz; > > + int i, nb_success = 0, nb_err = 0; > > + struct rte_mbuf *mb, *err_mb[num]; > > + > > + for (i = 0; i < num; i++) { > > + mb = in_mb[i]; > > + if (unlikely(mb->ol_flags & > > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { > > + err_mb[nb_err++] = mb; > > + continue; > > + } > > + > > + if (hdr_trim_sz) > > + rte_pktmbuf_adj(mb, hdr_trim_sz); > > + > > + out_mb[nb_success++] = mb; > > + } > > + > > + if (unlikely(nb_err != 0)) > > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > > sizeof(struct rte_mbuf *)); > > + > > + *nb_err_ret = nb_err; > > + return nb_success; > > +} > > + > > +static inline int > > +pdcp_sn_18_count_get(const struct rte_pdcp_entity *entity, int32_t rsn, > > uint32_t *count) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + uint32_t rhfn, rx_deliv; > > + > > + rx_deliv = __atomic_load_n(&en_priv->state.rx_deliv, > > __ATOMIC_RELAXED); > > + rhfn = PDCP_GET_HFN_SN_18_FROM_COUNT(rx_deliv); > > + > > + if (rsn < (int32_t)(PDCP_GET_SN_18_FROM_COUNT(rx_deliv) - > > PDCP_SN_18_WINDOW_SZ)) { > > + if (unlikely(rhfn == PDCP_SN_18_HFN_MAX)) > > + return -ERANGE; > > + rhfn += 1; > > + } else if ((uint32_t)rsn >= > (PDCP_GET_SN_18_FROM_COUNT(rx_deliv) + > > PDCP_SN_18_WINDOW_SZ)) { > > + if (unlikely(rhfn == PDCP_SN_18_HFN_MIN)) > > + return -ERANGE; > > + rhfn -= 1; > > + } > > + > > + *count = PDCP_SET_COUNT_FROM_HFN_SN_18(rhfn, rsn); > > + > > + return 0; > > +} > > + > > +static inline int > > +pdcp_sn_12_count_get(const struct rte_pdcp_entity *entity, int32_t rsn, > > uint32_t *count) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + uint32_t rhfn, rx_deliv; > > + > > + rx_deliv = __atomic_load_n(&en_priv->state.rx_deliv, > > __ATOMIC_RELAXED); > > + rhfn = PDCP_GET_HFN_SN_12_FROM_COUNT(rx_deliv); > > + > > + if (rsn < (int32_t)(PDCP_GET_SN_12_FROM_COUNT(rx_deliv) - > > PDCP_SN_12_WINDOW_SZ)) { > > + if (unlikely(rhfn == PDCP_SN_12_HFN_MAX)) > > + return -ERANGE; > > + rhfn += 1; > > + } else if ((uint32_t)rsn >= > (PDCP_GET_SN_12_FROM_COUNT(rx_deliv) + > > PDCP_SN_12_WINDOW_SZ)) { > > + if (unlikely(rhfn == PDCP_SN_12_HFN_MIN)) > > + return -ERANGE; > > + rhfn -= 1; > > + } > > + > > + *count = PDCP_SET_COUNT_FROM_HFN_SN_12(rhfn, rsn); > > + > > + return 0; > > +} > > + > > +static inline uint16_t > > +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity > > *entity, struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t > num, > > uint16_t *nb_err, > > + const bool is_integ_protected) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; > > + uint16_t nb_cop; > > + int32_t rsn = 0; > > + uint32_t count; > > + int i; > > + > > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > > + > > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > > + num); > > + > > + for (i = 0; i < nb_cop; i++) { > > + > > + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct > > rte_pdcp_up_data_pdu_sn_12_hdr *); > > + > > + /* Check for PDU type */ > > + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) > > + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr- > >sn_7_0)); > > + else > > + rte_panic("TODO: Control PDU not handled"); > > + > > + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) > > + break; > > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > > is_integ_protected); > > + } > > + > > + *nb_err = num - nb_cop; > > + > > + return nb_cop; > > +} > > + > > +static uint16_t > > +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity > *entity, > > struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t num, > > uint16_t *nb_err) > > +{ > > + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, > num, > > nb_err, true); > > +} > > + > > +static uint16_t > > +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity > *entity, > > struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t num, > > uint16_t *nb_err) > > +{ > > + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, > num, > > nb_err, false); > > +} > > + > > +static inline uint16_t > > +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity > > *entity, struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t > num, > > uint16_t *nb_err, > > + const bool is_integ_protected) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; > > + uint16_t nb_cop; > > + int32_t rsn = 0; > > + uint32_t count; > > + int i; > > + > > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > > + num); > > + > > + for (i = 0; i < nb_cop; i++) { > > + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct > > rte_pdcp_up_data_pdu_sn_18_hdr *); > > + > > + /* Check for PDU type */ > > + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) > > + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr- > >sn_15_8 > > << 8) | > > + (pdu_hdr->sn_7_0)); > > + else > > + rte_panic("TODO: Control PDU not handled"); > > + > > + if (unlikely(pdcp_sn_18_count_get(entity, rsn, &count))) > > + break; > > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > > is_integ_protected); > > + } > > + > > + *nb_err = num - nb_cop; > > + > > + return nb_cop; > > +} > > + > > +static uint16_t > > +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity > *entity, > > struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t num, > > uint16_t *nb_err) > > +{ > > + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, > num, > > nb_err, true); > > +} > > + > > +static uint16_t > > +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity > *entity, > > struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t num, > > uint16_t *nb_err) > > +{ > > + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, > num, > > nb_err, false); > > +} > > + > > +static uint16_t > > +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity > *entity, > > struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], uint16_t num, > > uint16_t *nb_err) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; > > + uint16_t nb_cop; > > + uint32_t count; > > + int32_t rsn; > > + int i; > > + > > + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; > > + > > + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, > > RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, > > + num); > > + > > + for (i = 0; i < nb_cop; i++) { > > + pdu_hdr = rte_pktmbuf_mtod(mb[i], struct > > rte_pdcp_cp_data_pdu_sn_12_hdr *); > > + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); > > + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) > > + break; > > + cop_prepare(en_priv, mb[i], cop[i], data_offset, count, > true); > > + } > > + > > + *nb_err = num - nb_cop; > > + return nb_cop; > > +} > > + > > +static inline bool > > +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity > *entity, > > + const uint32_t count) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + > > + if (count < __atomic_load_n(&en_priv->state.rx_deliv, > > __ATOMIC_RELAXED)) > > + return false; > > + > > + /* t-Reordering timer is not supported - SDU will be delivered > > immediately. > > + * Update RX_DELIV to the COUNT value of the first PDCP SDU which > > has not > > + * been delivered to upper layers > > + */ > > + __atomic_store_n(&en_priv->state.rx_deliv, (count + 1), > > __ATOMIC_RELAXED); > > + > > + if (count >= __atomic_load_n(&en_priv->state.rx_next, > > __ATOMIC_RELAXED)) > > + __atomic_store_n(&en_priv->state.rx_next, (count + 1), > > __ATOMIC_RELAXED); > > + > > + return true; > > +} > > + > > +static inline uint16_t > > +pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity > > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err_ret, > > + const bool is_integ_protected) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; > > + int i, nb_success = 0, nb_err = 0, rsn = 0; > > + const uint32_t aad_sz = en_priv->aad_sz; > > + struct rte_mbuf *err_mb[num]; > > + struct rte_mbuf *mb; > > + uint32_t count; > > + > > + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; > > + > > + for (i = 0; i < num; i++) { > > + mb = in_mb[i]; > > + if (unlikely(mb->ol_flags & > > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) > > + goto error; > > + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct > > rte_pdcp_up_data_pdu_sn_12_hdr *, > > + aad_sz); > > + > > + /* Check for PDU type */ > > + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) > > + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr- > >sn_7_0)); > > + else > > + rte_panic("Control PDU should not be received"); > > + > > + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) > > + goto error; > > + > > + if (unlikely(!pdcp_post_process_update_entity_state(entity, > > count))) > > + goto error; > > + > > + rte_pktmbuf_adj(mb, hdr_trim_sz); > > + if (is_integ_protected) > > + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); > > + out_mb[nb_success++] = mb; > > + continue; > > + > > +error: > > + err_mb[nb_err++] = mb; > > + } > > + > > + if (unlikely(nb_err != 0)) > > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > > sizeof(struct rte_mbuf *)); > > + > > + *nb_err_ret = nb_err; > > + return nb_success; > > +} > > + > > +static uint16_t > > +pdcp_post_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err) > > +{ > > + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, > > out_mb, num, nb_err, true); > > +} > > + > > +static uint16_t > > +pdcp_post_process_uplane_sn_12_dl(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err) > > +{ > > + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, > > out_mb, num, nb_err, false); > > +} > > + > > +static inline uint16_t > > +pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity > > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err_ret, > > + const bool is_integ_protected) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; > > + const uint32_t aad_sz = en_priv->aad_sz; > > + int i, nb_success = 0, nb_err = 0; > > + struct rte_mbuf *mb, *err_mb[num]; > > + int32_t rsn = 0; > > + uint32_t count; > > + > > + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; > > + > > + for (i = 0; i < num; i++) { > > + mb = in_mb[i]; > > + if (unlikely(mb->ol_flags & > > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) > > + goto error; > > + > > + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct > > rte_pdcp_up_data_pdu_sn_18_hdr *, > > + aad_sz); > > + > > + /* Check for PDU type */ > > + if (likely(pdu_hdr->d_c == PDCP_PDU_TYPE_DATA)) > > + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr- > >sn_15_8 > > << 8) | > > + (pdu_hdr->sn_7_0)); > > + else > > + rte_panic("Control PDU should not be received"); > > + > > + if (unlikely(pdcp_sn_18_count_get(entity, rsn, &count))) > > + goto error; > > + > > + if (unlikely(!pdcp_post_process_update_entity_state(entity, > > count))) > > + goto error; > > + > > + rte_pktmbuf_adj(mb, hdr_trim_sz); > > + if (is_integ_protected) > > + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); > > + out_mb[nb_success++] = mb; > > + continue; > > + > > +error: > > + err_mb[nb_err++] = mb; > > + } > > + > > + if (unlikely(nb_err != 0)) > > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > > sizeof(struct rte_mbuf *)); > > + > > + *nb_err_ret = nb_err; > > + return nb_success; > > +} > > + > > +static uint16_t > > +pdcp_post_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err) > > +{ > > + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, > > out_mb, num, nb_err, true); > > +} > > + > > +static uint16_t > > +pdcp_post_process_uplane_sn_18_dl(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err) > > +{ > > + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, > > out_mb, num, nb_err, false); > > +} > > + > > +static uint16_t > > +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err_ret) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; > > + const uint32_t aad_sz = en_priv->aad_sz; > > + int i, nb_success = 0, nb_err = 0; > > + struct rte_mbuf *err_mb[num]; > > + struct rte_mbuf *mb; > > + uint32_t count; > > + int32_t rsn; > > + > > + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; > > + > > + for (i = 0; i < num; i++) { > > + mb = in_mb[i]; > > + if (unlikely(mb->ol_flags & > > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) > > + goto error; > > + > > + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct > > rte_pdcp_cp_data_pdu_sn_12_hdr *, > > + aad_sz); > > + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); > > + > > + if (unlikely(pdcp_sn_12_count_get(entity, rsn, &count))) > > + goto error; > > + > > + if (unlikely(!pdcp_post_process_update_entity_state(entity, > > count))) > > + goto error; > > + > > + rte_pktmbuf_adj(mb, hdr_trim_sz); > > + rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); > > + out_mb[nb_success++] = mb; > > + continue; > > + > > +error: > > + err_mb[nb_err++] = mb; > > + } > > + > > + if (unlikely(nb_err != 0)) > > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > > sizeof(struct rte_mbuf *)); > > + > > + *nb_err_ret = nb_err; > > + return nb_success; > > +} > > + > > +static int > > +pdcp_pre_process_func_set(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + > > + entity->pre_process = NULL; > > + entity->post_process = NULL; > > + > > + if ((conf->pdcp_xfrm.domain == > > RTE_SECURITY_PDCP_MODE_CONTROL) && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { > > + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; > > + entity->post_process = > pdcp_post_process_cplane_sn_12_ul; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > > RTE_SECURITY_PDCP_MODE_CONTROL) && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { > > + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; > > + entity->post_process = > pdcp_post_process_cplane_sn_12_dl; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { > > + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; > > + entity->post_process = > pdcp_post_process_uplane_sn_12_ul; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { > > + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; > > + entity->post_process = > pdcp_post_process_uplane_sn_18_ul; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > && > > + (en_priv->flags.is_authenticated)) { > > + entity->pre_process = > pdcp_pre_process_uplane_sn_12_dl_ip; > > + entity->post_process = > > pdcp_post_process_uplane_sn_12_dl_ip; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > && > > + (!en_priv->flags.is_authenticated)) { > > + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; > > + entity->post_process = > pdcp_post_process_uplane_sn_12_dl; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > && > > + (en_priv->flags.is_authenticated)) { > > + entity->pre_process = > pdcp_pre_process_uplane_sn_18_dl_ip; > > + entity->post_process = > > pdcp_post_process_uplane_sn_18_dl_ip; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > && > > + (!en_priv->flags.is_authenticated)) { > > + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; > > + entity->post_process = > pdcp_post_process_uplane_sn_18_dl; > > + } > > + > > + if (entity->pre_process == NULL || entity->post_process == NULL) > > + return -ENOTSUP; > > + > > + return 0; > > +} > > + > > +static int > > +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > > + int ret; > > + > > + /** > > + * flags.is_authenticated > > + * > > + * MAC-I would be added in case of control plane packets and when > > authentication > > + * transform is not NULL. > > + */ > > + > > + if (conf->pdcp_xfrm.domain == > > RTE_SECURITY_PDCP_MODE_CONTROL) > > + en_priv->flags.is_authenticated = 1; > > + > > + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); > > + if (ret) > > + return ret; > > + > > + if (a_xfrm != NULL) > > + en_priv->flags.is_authenticated = 1; > > + > > + /** > > + * flags.is_ciph_in_bits > > + * > > + * For ZUC & SNOW3G cipher algos, offset & length need to be > provided > > in bits. > > + */ > > + > > + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) > || > > + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) > > + en_priv->flags.is_ciph_in_bits = 1; > > + > > + /** > > + * flags.is_auth_in_bits > > + * > > + * For ZUC & SNOW3G authentication algos, offset & length need to > be > > provided in bits. > > + */ > > + > > + if (a_xfrm != NULL) { > > + if ((a_xfrm->auth.algo == > RTE_CRYPTO_AUTH_SNOW3G_UIA2) > > || > > + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) > > + en_priv->flags.is_auth_in_bits = 1; > > + } > > + > > + /** > > + * flags.is_ul_entity > > + * > > + * Indicate whether the entity is UL/transmitting PDCP entity. > > + */ > > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > > + en_priv->flags.is_ul_entity = 1; > > + > > + /** > > + * hdr_sz > > + * > > + * PDCP header size of the entity > > + */ > > + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); > > + > > + /** > > + * aad_sz > > + * > > + * For AES-CMAC, additional message is prepended for processing. > Need > > to be trimmed after > > + * crypto processing is done. > > + */ > > + if (a_xfrm != NULL && a_xfrm->auth.algo == > > RTE_CRYPTO_AUTH_AES_CMAC) > > + en_priv->aad_sz = 8; > > + else > > + en_priv->aad_sz = 0; > > + > > + return 0; > > +} > > + > > +int > > +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + struct entity_priv *en_priv; > > + int ret; > > + > > + if (entity == NULL || conf == NULL) > > + return -EINVAL; > > + > > + en_priv = entity_priv_get(entity); > > + > > + ret = pdcp_iv_gen_func_set(entity, conf); > > + if (ret) > > + return ret; > > + > > + ret = pdcp_entity_priv_populate(en_priv, conf); > > + if (ret) > > + return ret; > > + > > + ret = pdcp_pre_process_func_set(entity, conf); > > + if (ret) > > + return ret; > > + > > + return 0; > > +} > > diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h > > new file mode 100644 > > index 0000000000..c92ab34c40 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_process.h > > @@ -0,0 +1,13 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#ifndef _PDCP_PROCESS_H_ > > +#define _PDCP_PROCESS_H_ > > + > > +#include <rte_pdcp.h> > > + > > +int > > +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf); > > + > > +#endif /* _PDCP_PROCESS_H_ */ > > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c > > new file mode 100644 > > index 0000000000..b1533971c2 > > --- /dev/null > > +++ b/lib/pdcp/rte_pdcp.c > > @@ -0,0 +1,136 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#include <rte_pdcp.h> > > +#include <rte_malloc.h> > > + > > +#include "pdcp_crypto.h" > > +#include "pdcp_entity.h" > > +#include "pdcp_process.h" > > + > > +static int > > +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) > > +{ > > + int size; > > + > > + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); > > + > > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > > + size += sizeof(struct entity_priv_dl_part); > > + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > > + size += sizeof(struct entity_priv_ul_part); > > + else > > + return -EINVAL; > > + > > + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); > > +} > > + > > +struct rte_pdcp_entity * > > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) > > +{ > > + struct rte_pdcp_entity *entity = NULL; > > + struct entity_priv *en_priv; > > + int ret; > > + > > + if (conf == NULL || conf->cop_pool == NULL) { > > + rte_errno = -EINVAL; > > + return NULL; > > + } > > + > > + if (conf->pdcp_xfrm.en_ordering || conf- > > >pdcp_xfrm.remove_duplicates || conf->is_slrb || > > + conf->en_sec_offload) { > > + rte_errno = -ENOTSUP; > > + return NULL; > > + } > > + > > + /* > > + * 6.3.2 PDCP SN > > + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the > > PDCP SN is > > + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl- > > PDCP-SN-Size in > > + * TS 38.331 [3]) > > + */ > > + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { > > + rte_errno = -ENOTSUP; > > + return NULL; > > + } > > + > > + if (conf->pdcp_xfrm.hfn || conf->pdcp_xfrm.hfn_threshold) { > > + rte_errno = -EINVAL; > > + return NULL; > > + } > > + > > + entity = rte_zmalloc_socket("pdcp_entity", > pdcp_entity_size_get(conf), > > + RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); > > + if (entity == NULL) { > > + rte_errno = -ENOMEM; > > + return NULL; > > + } > > + > > + en_priv = entity_priv_get(entity); > > + > > + en_priv->state.rx_deliv = conf->count; > > + en_priv->state.tx_next = conf->count; > > + en_priv->cop_pool = conf->cop_pool; > > + > > + /* Setup crypto session */ > > + ret = pdcp_crypto_sess_create(entity, conf); > > + if (ret) > > + goto entity_free; > > + > > + ret = pdcp_process_func_set(entity, conf); > > + if (ret) > > + goto crypto_sess_destroy; > > + > > + return entity; > > + > > +crypto_sess_destroy: > > + pdcp_crypto_sess_destroy(entity); > > +entity_free: > > + rte_free(entity); > > + rte_errno = ret; > > + return NULL; > > +} > > + > > +int > > +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct > rte_mbuf > > *out_mb[]) > > +{ > > + int ret; > > + > > + if (pdcp_entity == NULL) > > + return -EINVAL; > > + > > + /* Teardown crypto sessions */ > > + ret = pdcp_crypto_sess_destroy(pdcp_entity); > > + if (ret) > > + return ret; > > + > > + rte_free(pdcp_entity); > > + > > + RTE_SET_USED(out_mb); > > + return 0; > > +} > > + > > +int > > +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > > + struct rte_mbuf *out_mb[]) > > +{ > > + struct entity_priv *en_priv; > > + > > + if (pdcp_entity == NULL) > > + return -EINVAL; > > + > > + en_priv = entity_priv_get(pdcp_entity); > > + > > + if (en_priv->flags.is_ul_entity) { > > + en_priv->state.tx_next = 0; > > + } else { > > + en_priv->state.rx_next = 0; > > + en_priv->state.rx_deliv = 0; > > + } > > + > > + RTE_SET_USED(out_mb); > > + > > + return 0; > > +} > > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h > > new file mode 100644 > > index 0000000000..b6c7f32c05 > > --- /dev/null > > +++ b/lib/pdcp/rte_pdcp.h > > @@ -0,0 +1,263 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#ifndef _RTE_PDCP_H_ > > +#define _RTE_PDCP_H_ > > + > > +/** > > + * @file rte_pdcp.h > > + * > > + * RTE PDCP support. > > + * > > + * librte_pdcp provides a framework for PDCP protocol processing. > > + */ > > + > > +#include <rte_compat.h> > > +#include <rte_common.h> > > +#include <rte_errno.h> > > +#include <rte_mempool.h> > > +#include <rte_security.h> > > Remove header file which is not needed. > I do not see use of rte_errno.h > I believe rte_common.h and rte_compat.h are also not needed. [Anoob] Generally following the philosophy of "include-what-you-own". rte_errno.h - The API would set rte_errno in certain cases. Also, certain APIs return error numbers in case of failures. Since this is grey area, I'll follow what you suggest. Please suggest. rte_common.h - For generic & basic defines. Included it only to address build failure that we had early on. There are other similar rte_ files which include rte_common.h rte_compat.h - For 'experimental' etc. Also please check patch commit 1094dd940ec0cc4e3ce2c5cd94807350855a17f9 Author: David Marchand <david.marchand@redhat.com> Date: Fri Oct 28 14:13:39 2022 +0200 cleanup compat header inclusions With symbols going though experimental/stable stages, we accumulated a lot of discrepancies about inclusion of the rte_compat.h header. Some headers are including it where unneeded, while others rely on implicit inclusion. > > > + > > +#ifdef __cplusplus > > +extern "C" { > > +#endif > > + > > +/* Forward declarations */ > > +struct rte_pdcp_entity; > > + > > +/* PDCP pre-process function based on entity configuration */ > > +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *mb[], > > + struct rte_crypto_op *cop[], > > + uint16_t num, uint16_t *nb_err); > > + > > +/* PDCP post-process function based on entity configuration */ > > +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err); > > + > > +/** > > + * PDCP entity. > > + */ > > +struct rte_pdcp_entity { > > + /** Entity specific pre-process handle. */ > > + rte_pdcp_pre_p_t pre_process; > > + /** Entity specific post-process handle. */ > > + rte_pdcp_post_p_t post_process; > > + /** > > + * PDCP entities may hold packets for purposes of in-order delivery > (in > > + * case of receiving PDCP entity) and re-transmission (in case of > > + * transmitting PDCP entity). > > + * > > + * For receiving PDCP entity, it may hold packets when in-order > > + * delivery is enabled. The packets would be cached until either a > > + * packet that completes the sequence arrives or when discard timer > > + * expires. > > + * > > + * When post-processing of PDCP packet which completes a > sequence is > > + * done, the API may return more packets than enqueued. > Application is > > + * expected to provide *rte_pdcp_pkt_post_process()* with > *out_mb* > > + * which can hold maximum number of packets which may be > returned. > > + * > > + * For transmitting PDCP entity, during re-establishment (5.1.2), > > + * entity may be required to perform re-transmission of the buffers > > + * after applying new ciphering & integrity algorithms. For performing > > + * crypto operation, *rte_pdcp_entity_re_establish()* would return > as > > + * many crypto_ops as the ones cached. > > + */ > > + uint16_t max_pkt_cache; > > + /** User area for saving application data. */ > > + uint64_t user_area[2]; > > +} __rte_cache_aligned; > > + > > +/** > > + * PDCP entity configuration to be used for establishing an entity. > > + */ > > +struct rte_pdcp_entity_conf { > > + /** PDCP transform for the entity. */ > > + struct rte_security_pdcp_xform pdcp_xfrm; > > + /** Crypto transform applicable for the entity. */ > > + struct rte_crypto_sym_xform *crypto_xfrm; > > + /** Mempool for crypto symmetric session. */ > > + struct rte_mempool *sess_mpool; > > + /** Crypto op pool.*/ > > + struct rte_mempool *cop_pool; > > + /** > > + * 32 bit count value (HFN + SN) to be used for the first packet. > > + * pdcp_xfrm.hfn would be ignored as the HFN would be derived > from > > this value. > > + */ > > + uint32_t count; > > + /** Indicate whether the PDCP entity belongs to Side Link Radio > Bearer. > > */ > > + bool is_slrb; > > + /** Enable security offload on the device specified. */ > > + bool en_sec_offload; > > + /** Enable non-atomic usage of entity. */ > > + bool en_non_atomic; > > + /** Device on which security/crypto session need to be created. */ > > + uint8_t dev_id; > > + /** Reverse direction during IV generation. Can be used to simulate > UE > > crypto processing.*/ > > + bool reverse_iv_direction; > > +}; > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * 5.1.1 PDCP entity establishment > > + * > > + * Establish PDCP entity based on provided input configuration. > > + * > > + * @param conf > > + * Parameters to be used for initializing PDCP entity object. > > + * @return > > + * - Valid handle if success > > + * - NULL in case of failure. rte_errno will be set to error code > > + */ > > +__rte_experimental > > +struct rte_pdcp_entity * > > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * 5.1.3 PDCP entity release > > + * > > + * Release PDCP entity. > > + * > > + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be > dropped. > > + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned > in > > + * *out_mb* buffer. The buffer should be large enough to hold all cached > > + * packets in the entity. > > + * > > + * @param pdcp_entity > > + * Pointer to the PDCP entity to be released. > > + * @param[out] out_mb > > + * The address of an array that can hold up to > > *rte_pdcp_entity.max_pkt_cache* > > + * pointers to *rte_mbuf* structures. > > + * @return > > + * - 0: Success and no cached packets to return > > + * - >0: Success and the number of packets returned in out_mb > > + * - <0: Error code in case of failures > > + */ > > +__rte_experimental > > +int > > +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, > > + struct rte_mbuf *out_mb[]); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * 5.1.4 PDCP entity suspend > > + * > > + * Suspend PDCP entity. > > + * > > + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned > in > > + * *out_mb* buffer. The buffer should be large enough to hold all cached > > + * packets in the entity. > > + * > > + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. > > + * > > + * @param pdcp_entity > > + * Pointer to the PDCP entity to be suspended. > > + * @param[out] out_mb > > + * The address of an array that can hold up to > > *rte_pdcp_entity.max_pkt_cache* > > + * pointers to *rte_mbuf* structures. > > + * @return > > + * - 0: Success and no cached packets to return > > + * - >0: Success and the number of packets returned in out_mb > > + * - <0: Error code in case of failures > > + */ > > +__rte_experimental > > +int > > +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > > + struct rte_mbuf *out_mb[]); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * For input mbufs and given PDCP entity pre-process the mbufs and > prepare > > + * crypto ops that can be enqueued to the cryptodev associated with > given > > + * session. Only error packets would be moved returned in the input > buffer, > > + * *mb*, and it is the responsibility of the application to free the same. > > + * > > + * @param entity > > + * Pointer to the *rte_pdcp_entity* object the packets belong to. > > + * @param[in, out] mb > > + * The address of an array of *num* pointers to *rte_mbuf* structures > > + * which contain the input packets. Any error packets would be returned > in the > > + * same buffer. > > + * @param[out] cop > > + * The address of an array that can hold up to *num* pointers to > > + * *rte_crypto_op* structures. Crypto ops would be allocated by > > + * ``rte_pdcp_pkt_pre_process`` API. > > + * @param num > > + * The maximum number of packets to process. > > + * @param[out] nb_err > > + * Pointer to return the number of error packets returned in *mb* > > + * @return > > + * Count of crypto_ops prepared > > + */ > > +__rte_experimental > > +static inline uint16_t > > +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, > > + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], > > + uint16_t num, uint16_t *nb_err) > > +{ > > + return entity->pre_process(entity, mb, cop, num, nb_err); > > +} > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * For input mbufs and given PDCP entity, perform PDCP post-processing > of the > > + * mbufs. > > + * > > + * Input mbufs are the ones retrieved from crypto_ops dequeued from > > cryptodev > > + * and grouped by *rte_pdcp_pkt_crypto_group()*. > > + * > > + * The post-processed packets would be returned in the *out_mb* > buffer. > > + * The resultant mbufs would be grouped into success packets and error > > packets. > > + * Error packets would be grouped in the end of the array and it is the > > + * responsibility of the application to handle the same. > > + * > > + * When in-order delivery is enabled, PDCP entity may buffer packets and > would > > + * deliver packets only when all prior packets have been post-processed. > That > > + * would result in returning more/less packets than enqueued. > > + * > > + * @param entity > > + * Pointer to the *rte_pdcp_entity* object the packets belong to. > > + * @param in_mb > > + * The address of an array of *num* pointers to *rte_mbuf* structures. > > + * @param[out] out_mb > > + * The address of an array of *num* pointers to *rte_mbuf* structures > > + * to output packets after PDCP post-processing. > > + * @param num > > + * The maximum number of packets to process. > > + * @param[out] nb_err > > + * The number of error packets returned in *out_mb* buffer. > > + * @return > > + * Count of packets returned in *out_mb* buffer. > > + */ > > +__rte_experimental > > +static inline uint16_t > > +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, > > + struct rte_mbuf *in_mb[], > > + struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err) > > +{ > > + return entity->post_process(entity, in_mb, out_mb, num, nb_err); > > +} > > + > > +#include <rte_pdcp_group.h> > > + > > +#ifdef __cplusplus > > +} > > +#endif > > + > > +#endif /* _RTE_PDCP_H_ */ > > diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h > > new file mode 100644 > > index 0000000000..2c01c19d4e > > --- /dev/null > > +++ b/lib/pdcp/rte_pdcp_group.h > > @@ -0,0 +1,133 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2022 Marvell. > > + */ > > + > > +#ifndef _RTE_PDCP_GROUP_H_ > > +#define _RTE_PDCP_GROUP_H_ > > + > > +/** > > + * @file rte_pdcp_group.h > > + * > > + * RTE PDCP grouping support. > > + * It is not recommended to include this file directly, include <rte_pdcp.h> > > + * instead. > > + * Provides helper functions to process completed crypto-ops and group > > related > > + * packets by sessions they belong to. > > + */ > > + > > +#include <rte_common.h> > > +#include <rte_crypto.h> > > +#include <rte_cryptodev.h> > > +#include <rte_security.h> > > Remove header files which are not needed. > > > + > > +#ifdef __cplusplus > > +extern "C" { > > +#endif > > + > > +/** > > + * Group packets belonging to same PDCP entity. > > + */ > > +struct rte_pdcp_group { > > + union { > > + uint64_t val; > > + void *ptr; > > + } id; /**< Grouped by value */ > > + struct rte_mbuf **m; /**< Start of the group */ > > + uint32_t cnt; /**< Number of entries in the group */ > > + int32_t rc; /**< Status code associated with the group */ > > +}; > > + > > +/** > > + * Take crypto-op as an input and extract pointer to related PDCP entity. > > + * @param cop > > + * The address of an input *rte_crypto_op* structure. > > + * @return > > + * The pointer to the related *rte_pdcp_entity* structure. > > + */ > > +static inline struct rte_pdcp_entity * > > +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) > > +{ > > + void *sess = cop->sym[0].session; > > + > > + if (cop->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { > > + return (struct rte_pdcp_entity *)(uintptr_t) > > + rte_security_session_opaque_data_get(sess); > > + } else if (cop->sess_type == RTE_CRYPTO_OP_WITH_SESSION) { > > + return (struct rte_pdcp_entity *)(uintptr_t) > > + > rte_cryptodev_sym_session_opaque_data_get(sess); > > + } > > This patchset is not supporting security sessions, so it would be better to > return NULL for that. > Moreover, we can directly call > rte_cryptodev_sym_session_opaque_data_get(cop->sym[0].session) > From rte_pdcp_pkt_crypto_group. No need to have a wrapper. [Anoob] Agreed. Will address in next version. > > > + > > + return NULL; > > +} > > + > > +/** > > + * Take as input completed crypto ops, extract related mbufs and group > them > > by > > + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto > operation has > > + * failed would be flagged using > *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* > > flag > > + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the > grouping. > > + * > > + * Note that application must ensure only crypto-ops prepared by > lib_pdcp is > > + * provided back to @see rte_pdcp_pkt_crypto_group(). > > + * > > + * @param cop > > + * The address of an array of *num* pointers to the input > *rte_crypto_op* > > + * structures. > > + * @param[out] mb > > + * The address of an array of *num* pointers to output *rte_mbuf* > structures. > > + * @param[out] grp > > + * The address of an array of *num* to output *rte_pdcp_group* > structures. > > + * @param num > > + * The maximum number of crypto-ops to process. > > + * @return > > + * Number of filled elements in *grp* array. > > + * > > + */ > > +static inline uint16_t > > +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct > rte_mbuf > > *mb[], > > + struct rte_pdcp_group grp[], uint16_t num) > > +{ > > + uint32_t i, j = 0, n = 0; > > + void *ns, *ps = NULL; > > + struct rte_mbuf *m; > > + > > + for (i = 0; i != num; i++) { > > + m = cop[i]->sym[0].m_src; > > + ns = cop[i]->sym[0].session; > > + > > + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; > > + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) > > + m->ol_flags |= > > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; > > + > > + /* Different entity */ > > + if (ps != ns) { > > + > > + /* Finalize open group and start a new one */ > > + if (ps != NULL) { > > + grp[n].cnt = mb + j - grp[n].m; > > + n++; > > + } > > + > > + /* Start new group */ > > + grp[n].m = mb + j; > > + ps = ns; > > + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); > > + } > > + > > + mb[j++] = m; > > + rte_crypto_op_free(cop[i]); > > + } > > + > > + /* Finalize last group */ > > + if (ps != NULL) { > > + grp[n].cnt = mb + j - grp[n].m; > > + n++; > > + } > > + > > + return n; > > +} > > + > > +#ifdef __cplusplus > > +} > > +#endif > > + > > +#endif /* _RTE_PDCP_GROUP_H_ */ > > diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map > > new file mode 100644 > > index 0000000000..8fa9d5d7cc > > --- /dev/null > > +++ b/lib/pdcp/version.map > > @@ -0,0 +1,13 @@ > > +EXPERIMENTAL { > > + global: > > + > > + # added in 22.11 > > Change to 23.03 > > > + rte_pdcp_entity_establish; > > + rte_pdcp_entity_release; > > + rte_pdcp_entity_suspend; > > + > > + rte_pdcp_pkt_post_process; > > + rte_pdcp_pkt_pre_process; > > + > > + local: *; > > +}; > > -- > > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH 3/5] app/test: add lib pdcp tests 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph 2022-12-22 9:25 ` [PATCH 1/5] net: add PDCP header Anoob Joseph 2022-12-22 9:25 ` [PATCH 2/5] lib: add pdcp protocol Anoob Joseph @ 2022-12-22 9:25 ` Anoob Joseph 2022-12-22 9:25 ` [PATCH 4/5] app/test: pdcp HFN tests in combined mode Anoob Joseph ` (3 subsequent siblings) 6 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2022-12-22 9:25 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add tests to verify lib PDCP operations. Tests leverage existing PDCP test vectors. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/meson.build | 1 + app/test/test_cryptodev.h | 3 + app/test/test_pdcp.c | 663 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 667 insertions(+) create mode 100644 app/test/test_pdcp.c diff --git a/app/test/meson.build b/app/test/meson.build index 1b9ce355f8..7e1f3f6543 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -95,6 +95,7 @@ test_sources = files( 'test_meter.c', 'test_mcslock.c', 'test_mp_secondary.c', + 'test_pdcp.c', 'test_per_lcore.c', 'test_pflock.c', 'test_pmd_perf.c', diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h index abd795f54a..89057dba22 100644 --- a/app/test/test_cryptodev.h +++ b/app/test/test_cryptodev.h @@ -4,6 +4,9 @@ #ifndef TEST_CRYPTODEV_H_ #define TEST_CRYPTODEV_H_ +#include <rte_crypto.h> +#include <rte_cryptodev.h> + #define HEX_DUMP 0 #define FALSE 0 diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c new file mode 100644 index 0000000000..10a383ddd7 --- /dev/null +++ b/app/test/test_pdcp.c @@ -0,0 +1,663 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2022 Marvell. + */ + +#include <rte_malloc.h> +#include <rte_pdcp.h> + +#include "test.h" +#include "test_cryptodev.h" +#include "test_cryptodev_security_pdcp_test_vectors.h" + +#define NB_DESC 1024 + +struct pdcp_testsuite_params { + struct rte_mempool *mbuf_pool; + struct rte_mempool *cop_pool; + struct rte_mempool *sess_pool; +}; + +static struct pdcp_testsuite_params testsuite_params; + +#define PDCP_MAX_TEST_INPUT_LEN 2048 + +struct pdcp_test_conf { + struct rte_pdcp_entity_conf entity; + struct rte_crypto_sym_xform c_xfrm; + struct rte_crypto_sym_xform a_xfrm; + bool is_integrity_protected; + uint8_t input[PDCP_MAX_TEST_INPUT_LEN]; + uint32_t input_len; + const uint8_t *expected; + uint32_t expected_len; +}; + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static int +cryptodev_init(int dev_id) +{ + struct rte_cryptodev_qp_conf qp_conf; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_config config; + int ret, socket_id; + + rte_cryptodev_info_get(dev_id, &dev_info); + + if (dev_info.max_nb_queue_pairs < 1) { + RTE_LOG(ERR, USER1, "Cryptodev doesn't have sufficient queue pairs available\n"); + return -ENODEV; + } + + socket_id = rte_socket_id(); + + memset(&config, 0, sizeof(config)); + config.nb_queue_pairs = 1; + config.socket_id = socket_id; + + ret = rte_cryptodev_configure(dev_id, &config); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure cryptodev - %d\n", dev_id); + return -ENODEV; + } + + memset(&qp_conf, 0, sizeof(qp_conf)); + qp_conf.nb_descriptors = NB_DESC; + + ret = rte_cryptodev_queue_pair_setup(dev_id, 0, &qp_conf, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure queue pair\n"); + return -ENODEV; + } + + ret = rte_cryptodev_start(dev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not start cryptodev\n"); + return -ENODEV; + } + + return 0; +} + +static void +cryptodev_fini(int dev_id) +{ + rte_cryptodev_stop(dev_id); +} + +static unsigned int +cryptodev_sess_priv_max_req_get(void) +{ + struct rte_cryptodev_info info; + unsigned int sess_priv_sz; + int i, nb_dev; + void *sec_ctx; + + nb_dev = rte_cryptodev_count(); + + sess_priv_sz = 0; + + for (i = 0; i < nb_dev; i++) { + rte_cryptodev_info_get(i, &info); + sess_priv_sz = RTE_MAX(sess_priv_sz, rte_cryptodev_sym_get_private_session_size(i)); + if (info.feature_flags & RTE_CRYPTODEV_FF_SECURITY) { + sec_ctx = rte_cryptodev_get_sec_ctx(i); + sess_priv_sz = RTE_MAX(sess_priv_sz, + rte_security_session_get_size(sec_ctx)); + } + } + + return sess_priv_sz; +} + +static int +testsuite_setup(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + int nb_cdev, sess_priv_size, nb_sess = 1024; + + RTE_SET_USED(pdcp_test_auth_key); + RTE_SET_USED(pdcp_test_crypto_key); + RTE_SET_USED(pdcp_test_data_out); + RTE_SET_USED(pdcp_test_hfn_threshold); + + nb_cdev = rte_cryptodev_count(); + if (nb_cdev < 1) { + RTE_LOG(ERR, USER1, "No crypto devices found.\n"); + return TEST_SKIPPED; + } + + memset(ts_params, 0, sizeof(*ts_params)); + + ts_params->mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NUM_MBUFS, MBUF_CACHE_SIZE, 0, + MBUF_SIZE, SOCKET_ID_ANY); + if (ts_params->mbuf_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf pool\n"); + return TEST_FAILED; + } + + ts_params->cop_pool = rte_crypto_op_pool_create("cop_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, MBUF_CACHE_SIZE, + 2 * MAXIMUM_IV_LENGTH, SOCKET_ID_ANY); + if (ts_params->cop_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create crypto_op pool\n"); + goto mbuf_pool_free; + } + + /* Get max session priv size required */ + sess_priv_size = cryptodev_sess_priv_max_req_get(); + + ts_params->sess_pool = rte_cryptodev_sym_session_pool_create("sess_pool", nb_sess, + sess_priv_size, + RTE_MEMPOOL_CACHE_MAX_SIZE, + 0, SOCKET_ID_ANY); + if (ts_params->sess_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create session pool\n"); + goto cop_pool_free; + } + + printf("TEST SUITE SETUP\n"); + return 0; + +cop_pool_free: + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; +mbuf_pool_free: + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; + return TEST_FAILED; +} + +static void +testsuite_teardown(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + + rte_mempool_free(ts_params->sess_pool); + ts_params->sess_pool = NULL; + + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; + + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; + + printf("TEST SUITE TEARDOWN\n"); +} + +static int +ut_setup_pdcp(void) +{ + printf("SETUP PDCP\n"); + return 0; +} + +static void +ut_teardown_pdcp(void) +{ + printf("TEARDOWN PDCP\n"); +} + +static int +crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +cryptodev_id_get(bool is_integrity_protected, const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm) +{ + int i, nb_devs; + + nb_devs = rte_cryptodev_count(); + + /* Check capabilities */ + + for (i = 0; i < nb_devs; i++) { + if ((crypto_caps_cipher_verify(i, c_xfrm) == 0) && + (!is_integrity_protected || crypto_caps_auth_verify(i, a_xfrm) == 0)) + break; + } + + if (i == nb_devs) + return -1; + + return i; +} + +static int +pdcp_known_vec_verify(struct rte_mbuf *m, const uint8_t *expected, uint32_t expected_pkt_len) +{ + uint8_t *actual = rte_pktmbuf_mtod(m, uint8_t *); + uint32_t actual_pkt_len = rte_pktmbuf_pkt_len(m); + + debug_hexdump(stdout, "Received:", actual, actual_pkt_len); + debug_hexdump(stdout, "Expected:", expected, expected_pkt_len); + + TEST_ASSERT_EQUAL(actual_pkt_len, expected_pkt_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_pkt_len, actual_pkt_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(actual, expected, expected_pkt_len, + "Generated packet not as expected"); + + return 0; +} + +static struct rte_crypto_op * +process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op) +{ + if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) { + RTE_LOG(ERR, USER1, "Error sending packet to cryptodev\n"); + return NULL; + } + + op = NULL; + + while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0) + rte_pause(); + + return op; +} + +static int +create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_crypto_sym_xform c_xfrm, a_xfrm; + uint32_t hfn, sn, count = 0; + int pdcp_hdr_sz; + uint8_t *data; + + memset(conf, 0, sizeof(*conf)); + memset(&c_xfrm, 0, sizeof(c_xfrm)); + memset(&a_xfrm, 0, sizeof(a_xfrm)); + + conf->entity.sess_mpool = ts_params->sess_pool; + conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; + conf->entity.pdcp_xfrm.en_ordering = 0; + conf->entity.pdcp_xfrm.remove_duplicates = 0; + conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + + if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; + else + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + + conf->entity.pdcp_xfrm.sn_size = pdcp_test_data_sn_size[index]; + conf->entity.pdcp_xfrm.hfn_threshold = 0; + conf->entity.pdcp_xfrm.hfn_ovrd = 0; + conf->entity.pdcp_xfrm.sdap_enabled = 0; + + c_xfrm.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + c_xfrm.cipher.algo = pdcp_test_params[index].cipher_alg; + c_xfrm.cipher.key.length = pdcp_test_params[index].cipher_key_len; + c_xfrm.cipher.key.data = pdcp_test_crypto_key[index]; + + a_xfrm.type = RTE_CRYPTO_SYM_XFORM_AUTH; + + if (pdcp_test_params[index].auth_alg == 0) { + conf->is_integrity_protected = false; + } else { + a_xfrm.auth.algo = pdcp_test_params[index].auth_alg; + a_xfrm.auth.key.data = pdcp_test_auth_key[index]; + a_xfrm.auth.key.length = pdcp_test_params[index].auth_key_len; + conf->is_integrity_protected = true; + } + + pdcp_hdr_sz = pdcp_hdr_size_get(pdcp_test_data_sn_size[index]); + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protecting is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (conf->is_integrity_protected) { + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + conf->entity.crypto_xfrm = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + a_xfrm.next = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + c_xfrm.next = NULL; + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + c_xfrm.next = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + a_xfrm.next = NULL; + } + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + c_xfrm.next = NULL; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + else + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + /* Update xforms to match PDCP requirements */ + + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) + c_xfrm.cipher.iv.length = 16; + else + c_xfrm.cipher.iv.length = 0; + + if (conf->is_integrity_protected) { + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm.auth.digest_length = 0; + else + a_xfrm.auth.digest_length = 4; + + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + a_xfrm.auth.iv.length = 16; + else + a_xfrm.auth.iv.length = 0; + } + + conf->c_xfrm = c_xfrm; + conf->a_xfrm = a_xfrm; + + if (pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_CONTROL || + pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_DATA) { + data = pdcp_test_data_in[index]; + hfn = pdcp_test_hfn[index] << pdcp_test_data_sn_size[index]; + sn = 0; + if (pdcp_test_data_sn_size[index] == RTE_SECURITY_PDCP_SN_SIZE_12) { + sn = rte_cpu_to_be_16(*(uint16_t *)data); + sn = sn & 0xfff; + } else if (pdcp_test_data_sn_size[index] == RTE_SECURITY_PDCP_SN_SIZE_18) { + sn = rte_cpu_to_be_32(*(uint32_t *)data); + sn = (sn & 0x3ffff00) >> 8; + } + count = hfn | sn; + } + conf->entity.count = count; + + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", pdcp_test_data_in[index], + pdcp_test_data_in_len[index]); +#endif + /* Since the vectors available already have PDCP header, trim the same */ + conf->input_len = pdcp_test_data_in_len[index] - pdcp_hdr_sz; + memcpy(conf->input, pdcp_test_data_in[index] + pdcp_hdr_sz, conf->input_len); + } else { + conf->input_len = pdcp_test_data_in_len[index]; + + if (conf->is_integrity_protected) + conf->input_len += 4; + + memcpy(conf->input, pdcp_test_data_out[index], conf->input_len); +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", conf->input, conf->input_len); +#endif + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + conf->expected = pdcp_test_data_out[index]; + else + conf->expected = pdcp_test_data_in[index]; + + /* Calculate expected packet length */ + conf->expected_len = pdcp_test_data_in_len[index]; + + /* In DL processing, PDCP header would be stripped */ + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + conf->expected += pdcp_hdr_sz; + conf->expected_len -= pdcp_hdr_sz; + } + + /* In UL processing with integrity protection, MAC would be added */ + if (conf->is_integrity_protected && + conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + conf->expected_len += 4; + + return 0; +} + +static int +test_attempt_single(const struct pdcp_test_conf *t_conf) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_mbuf *mbuf, *mb, **out_mb = NULL; + uint16_t nb_success, nb_err, nb_grp; + struct rte_pdcp_entity *pdcp_entity; + struct rte_crypto_op *cop, *cop_out; + int cdev_id, ret = 0, nb_max_out_mb; + struct rte_pdcp_entity_conf conf; + struct rte_pdcp_group grp[1]; + uint8_t *input_text; + + if (t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12 && + t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18) { + ret = -ENOTSUP; + goto exit; + } + + cdev_id = cryptodev_id_get(t_conf->is_integrity_protected, + &t_conf->c_xfrm, &t_conf->a_xfrm); + if (cdev_id == -1) { + RTE_LOG(DEBUG, USER1, "Could not find device with required capabilities\n"); + ret = -ENOTSUP; + goto exit; + } + + ret = cryptodev_init(cdev_id); + if (ret) { + RTE_LOG(DEBUG, USER1, "Could not initialize cryptode\nv"); + goto exit; + } + + conf = t_conf->entity; + + pdcp_entity = rte_pdcp_entity_establish(&conf); + if (pdcp_entity == NULL) { + RTE_LOG(DEBUG, USER1, "Could not establish PDCP entity\n"); + ret = rte_errno; + goto cdev_fini; + } + + /* Allocate buffer for holding mbufs returned */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate buffer for holding out_mb buffers\n"); + ret = -ENOMEM; + goto entity_release; + } + + mbuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (mbuf == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf\n"); + ret = -ENOMEM; + goto entity_release; + } + + memset(rte_pktmbuf_mtod(mbuf, uint8_t *), 0, + rte_pktmbuf_tailroom(mbuf)); + input_text = (uint8_t *)rte_pktmbuf_append(mbuf, t_conf->input_len); + memcpy(input_text, t_conf->input, t_conf->input_len); + +#ifdef VEC_DUMP + printf("Adjusted vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, t_conf->input_len); +#endif + + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, &mbuf, &cop_out, 1, &nb_err); + if (nb_success != 1 || nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not pre process PDCP packet\n"); + ret = -ENOSYS; + goto mbuf_free; + } + +#ifdef VEC_DUMP + printf("Pre-processed vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, rte_pktmbuf_pkt_len(mbuf)); +#endif + + cop = process_crypto_request(cdev_id, cop_out); + if (cop == NULL) { + RTE_LOG(ERR, USER1, "Could not process crypto request\n"); + ret = -EIO; + goto mbuf_free; + } + + nb_grp = rte_pdcp_pkt_crypto_group(&cop_out, &mb, grp, 1); + if (nb_grp != 1 || grp[0].cnt != 1) { + RTE_LOG(ERR, USER1, "Could not group PDCP crypto results\n"); + ret = -ENOTRECOVERABLE; + goto mbuf_free; + } + + if ((uintptr_t)pdcp_entity != grp[0].id.val) { + RTE_LOG(ERR, USER1, "PDCP entity not matching the one from crypto_op\n"); + ret = -ENOTRECOVERABLE; + goto mbuf_free; + } + +#ifdef VEC_DUMP + printf("Crypto processed vector:\n"); + rte_pktmbuf_dump(stdout, cop->sym->m_dst, rte_pktmbuf_pkt_len(mbuf)); +#endif + + nb_success = rte_pdcp_pkt_post_process(grp[0].id.ptr, grp[0].m, out_mb, grp[0].cnt, + &nb_err); + if (nb_success != 1 || nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not post process PDCP packet\n"); + ret = -ENOSYS; + goto mbuf_free; + } + + ret = pdcp_known_vec_verify(mbuf, t_conf->expected, t_conf->expected_len); + if (ret) + goto mbuf_free; + + ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); + if (ret) { + RTE_LOG(DEBUG, USER1, "Could not suspend PDCP entity\n"); + goto mbuf_free; + } + +mbuf_free: + rte_pktmbuf_free(mbuf); +entity_release: + rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_free(out_mb); +cdev_fini: + cryptodev_fini(cdev_id); +exit: + if (ret == -ENOTSUP) + return TEST_SKIPPED; + if (ret == 0) + return TEST_SUCCESS; + return TEST_FAILED; +} + +static int +test_iterate_all(void) +{ + int ret, nb_test, i, passed = 0; + struct pdcp_test_conf t_conf; + + nb_test = RTE_DIM(pdcp_test_params); + + for (i = 0; i < nb_test; i++) { + printf("[%03i] - %s", i, pdcp_test_params[i].name); + create_test_conf_from_index(i, &t_conf); + ret = test_attempt_single(&t_conf); + if (ret == TEST_FAILED) { + printf(" - failed\n"); + return ret; + } else if (ret == TEST_SKIPPED) { + printf(" - skipped\n"); + continue; + } + + printf(" - passed\n"); + passed += 1; + } + printf("Passed: %i\n", passed); + + return TEST_SUCCESS; +} + +static int +test_sample(void) +{ + return test_iterate_all(); +} + +static struct unit_test_suite pdcp_testsuite = { + .suite_name = "PDCP Unit Test Suite", + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_ST(ut_setup_pdcp, ut_teardown_pdcp, + test_sample), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static int +test_pdcp(void) +{ + return unit_test_suite_runner(&pdcp_testsuite); +} + +REGISTER_TEST_COMMAND(pdcp_autotest, test_pdcp); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH 4/5] app/test: pdcp HFN tests in combined mode 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph ` (2 preceding siblings ...) 2022-12-22 9:25 ` [PATCH 3/5] app/test: add lib pdcp tests Anoob Joseph @ 2022-12-22 9:25 ` Anoob Joseph 2022-12-22 9:25 ` [PATCH 5/5] doc: add PDCP library guide Anoob Joseph ` (2 subsequent siblings) 6 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2022-12-22 9:25 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add tests to verify HFN/SN behaviour. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 390 ++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 363 insertions(+), 27 deletions(-) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 10a383ddd7..1e74147579 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -4,6 +4,7 @@ #include <rte_malloc.h> #include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> #include "test.h" #include "test_cryptodev.h" @@ -19,17 +20,17 @@ struct pdcp_testsuite_params { static struct pdcp_testsuite_params testsuite_params; -#define PDCP_MAX_TEST_INPUT_LEN 2048 +#define PDCP_MAX_TEST_DATA_LEN 2048 struct pdcp_test_conf { struct rte_pdcp_entity_conf entity; struct rte_crypto_sym_xform c_xfrm; struct rte_crypto_sym_xform a_xfrm; bool is_integrity_protected; - uint8_t input[PDCP_MAX_TEST_INPUT_LEN]; + uint8_t input[PDCP_MAX_TEST_DATA_LEN]; uint32_t input_len; - const uint8_t *expected; - uint32_t expected_len; + uint8_t output[PDCP_MAX_TEST_DATA_LEN]; + uint32_t output_len; }; static inline int @@ -38,6 +39,19 @@ pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static int +pktmbuf_read_into(const struct rte_mbuf *m, void *buf, size_t buf_len) +{ + if (m->pkt_len > buf_len) + return -ENOMEM; + + const void *read = rte_pktmbuf_read(m, 0, m->pkt_len, buf); + if (read != NULL && read != buf) + memcpy(buf, read, m->pkt_len); + + return 0; +} + static int cryptodev_init(int dev_id) { @@ -300,14 +314,45 @@ process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op) return op; } +static uint32_t +pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) +{ + uint32_t sn = 0; + + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + sn = rte_cpu_to_be_16(*(const uint16_t *)data); + sn = sn & 0xfff; + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + sn = rte_cpu_to_be_32(*(const uint32_t *)data); + sn = (sn & 0x3ffff00) >> 8; + } + + return sn; +} + +static void +pdcp_sn_to_raw_set(void *data, uint32_t sn, int size) +{ + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr = data; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr = data; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } +} + static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) { const struct pdcp_testsuite_params *ts_params = &testsuite_params; struct rte_crypto_sym_xform c_xfrm, a_xfrm; - uint32_t hfn, sn, count = 0; + uint32_t hfn, sn, expected_len, count = 0; + uint8_t *data, *expected; int pdcp_hdr_sz; - uint8_t *data; memset(conf, 0, sizeof(*conf)); memset(&c_xfrm, 0, sizeof(c_xfrm)); @@ -326,6 +371,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; conf->entity.pdcp_xfrm.sn_size = pdcp_test_data_sn_size[index]; + /* Zero initialize unsupported flags */ conf->entity.pdcp_xfrm.hfn_threshold = 0; conf->entity.pdcp_xfrm.hfn_ovrd = 0; conf->entity.pdcp_xfrm.sdap_enabled = 0; @@ -414,14 +460,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_DATA) { data = pdcp_test_data_in[index]; hfn = pdcp_test_hfn[index] << pdcp_test_data_sn_size[index]; - sn = 0; - if (pdcp_test_data_sn_size[index] == RTE_SECURITY_PDCP_SN_SIZE_12) { - sn = rte_cpu_to_be_16(*(uint16_t *)data); - sn = sn & 0xfff; - } else if (pdcp_test_data_sn_size[index] == RTE_SECURITY_PDCP_SN_SIZE_18) { - sn = rte_cpu_to_be_32(*(uint32_t *)data); - sn = (sn & 0x3ffff00) >> 8; - } + sn = pdcp_sn_from_raw_get(data, pdcp_test_data_sn_size[index]); count = hfn | sn; } conf->entity.count = count; @@ -448,29 +487,32 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) } if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) - conf->expected = pdcp_test_data_out[index]; + expected = pdcp_test_data_out[index]; else - conf->expected = pdcp_test_data_in[index]; + expected = pdcp_test_data_in[index]; /* Calculate expected packet length */ - conf->expected_len = pdcp_test_data_in_len[index]; + expected_len = pdcp_test_data_in_len[index]; /* In DL processing, PDCP header would be stripped */ if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { - conf->expected += pdcp_hdr_sz; - conf->expected_len -= pdcp_hdr_sz; + expected += pdcp_hdr_sz; + expected_len -= pdcp_hdr_sz; } /* In UL processing with integrity protection, MAC would be added */ if (conf->is_integrity_protected && conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) - conf->expected_len += 4; + expected_len += 4; + + memcpy(conf->output, expected, expected_len); + conf->output_len = expected_len; return 0; } static int -test_attempt_single(const struct pdcp_test_conf *t_conf) +test_attempt_single(struct pdcp_test_conf *t_conf) { const struct pdcp_testsuite_params *ts_params = &testsuite_params; struct rte_mbuf *mbuf, *mb, **out_mb = NULL; @@ -584,9 +626,17 @@ test_attempt_single(const struct pdcp_test_conf *t_conf) goto mbuf_free; } - ret = pdcp_known_vec_verify(mbuf, t_conf->expected, t_conf->expected_len); - if (ret) - goto mbuf_free; + /* If expected output provided - verify, else - store for future use */ + if (t_conf->output_len) { + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + } else { + ret = pktmbuf_read_into(mbuf, t_conf->output, PDCP_MAX_TEST_DATA_LEN); + if (ret) + goto mbuf_free; + t_conf->output_len = mbuf->pkt_len; + } ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); if (ret) { @@ -609,6 +659,195 @@ test_attempt_single(const struct pdcp_test_conf *t_conf) return TEST_FAILED; } +static void +uplink_to_downlink_convert(const struct pdcp_test_conf *ul_cfg, + struct pdcp_test_conf *dl_cfg) +{ + assert(ul_cfg->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK); + + memcpy(dl_cfg, ul_cfg, sizeof(*dl_cfg)); + dl_cfg->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + dl_cfg->entity.reverse_iv_direction = false; + + if (dl_cfg->is_integrity_protected) { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + dl_cfg->c_xfrm.next = &dl_cfg->a_xfrm; + + dl_cfg->a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + dl_cfg->a_xfrm.next = NULL; + } else { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + dl_cfg->c_xfrm.next = NULL; + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + memcpy(dl_cfg->input, ul_cfg->output, ul_cfg->output_len); + dl_cfg->input_len = ul_cfg->output_len; + + memcpy(dl_cfg->output, ul_cfg->input, ul_cfg->input_len); + dl_cfg->output_len = ul_cfg->input_len; +} + +/* + * According to ETSI TS 138 323 V17.1.0, Section 5.2.2.1, + * SN could be divided into following ranges, + * relatively to current value of RX_DELIV state: + * +-------------+-------------+-------------+-------------+ + * | -Outside | -Window | +Window | +Outside | + * | (valid) | (Invalid) | (Valid) | (Invalid) | + * +-------------+-------------^-------------+-------------+ + * | + * v + * SN(RX_DELIV) + */ +enum sn_range_type { + SN_RANGE_MINUS_OUTSIDE, + SN_RANGE_MINUS_WINDOW, + SN_RANGE_PLUS_WINDOW, + SN_RANGE_PLUS_OUTSIDE, +}; + +#define PDCP_SET_COUNT(hfn, sn, size) ((hfn << size) | (sn & ((1 << size) - 1))) + +/* + * Take uplink test case as base, modify RX_DELIV in state and SN in input + */ +static int +test_sn_range_type_with_config(enum sn_range_type type, struct pdcp_test_conf *conf) +{ + uint32_t rx_deliv_hfn, rx_deliv_sn, rx_deliv, new_hfn, new_sn; + const int domain = conf->entity.pdcp_xfrm.domain; + struct pdcp_test_conf dl_conf; + int ret, expected_ret; + + if (domain != RTE_SECURITY_PDCP_MODE_CONTROL && domain != RTE_SECURITY_PDCP_MODE_DATA) + return TEST_SKIPPED; + + const uint32_t sn_size = conf->entity.pdcp_xfrm.sn_size; + /* According to formula(7.2.a Window_Size) */ + const uint32_t window_size = 1 << (sn_size - 1); + /* Max value of SN that could fit in `sn_size` bits */ + const uint32_t max_sn = (1 << sn_size) - 1; + const uint32_t shift = (max_sn - window_size) / 2; + /* Could be any number up to `shift` value */ + const uint32_t default_sn = RTE_MIN(2u, shift); + + /* Initialize HFN as non zero value, to be able check values before */ + rx_deliv_hfn = 0xa; + + switch (type) { + case SN_RANGE_PLUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + 1; + expected_ret = TEST_SUCCESS; + break; + case SN_RANGE_MINUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn - 1; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_PLUS_OUTSIDE: + /* RCVD_SN >= SN(RX_DELIV) + Window_Size */ + new_hfn = rx_deliv_hfn - 1; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + window_size; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_MINUS_OUTSIDE: + /* RCVD_SN < SN(RX_DELIV) - Window_Size */ + new_hfn = rx_deliv_hfn + 1; + rx_deliv_sn = window_size + default_sn; + new_sn = rx_deliv_sn - window_size - 1; + expected_ret = TEST_SUCCESS; + break; + default: + return TEST_FAILED; + } + + rx_deliv = PDCP_SET_COUNT(rx_deliv_hfn, rx_deliv_sn, sn_size); + + /* Configure Uplink to generate expected, encrypted packet */ + pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.reverse_iv_direction = true; + conf->entity.count = PDCP_SET_COUNT(new_hfn, new_sn, sn_size); + conf->output_len = 0; + ret = test_attempt_single(conf); + if (ret != TEST_SUCCESS) + return ret; + + /* Flip configuration to downlink */ + uplink_to_downlink_convert(conf, &dl_conf); + /* Modify the rx_deliv to verify the expected behaviour */ + dl_conf.entity.count = rx_deliv; + ret = test_attempt_single(&dl_conf); + if (ret == TEST_SKIPPED) + return TEST_SKIPPED; + TEST_ASSERT_EQUAL(ret, expected_ret, "Unexpected result"); + + return TEST_SUCCESS; +} + +static int +test_sn_range_type(enum sn_range_type type) +{ + int i, ret, passed = 0; + struct pdcp_test_conf t_conf; + + int nb_test = RTE_DIM(pdcp_test_params); + + for (i = 0; i < nb_test; i++) { + create_test_conf_from_index(i, &t_conf); + if (t_conf.entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + continue; + printf("[%03i] - %s", i, pdcp_test_params[i].name); + ret = test_sn_range_type_with_config(type, &t_conf); + if (ret == TEST_FAILED) { + printf(" - failed\n"); + return ret; + } else if (ret == TEST_SKIPPED) { + printf(" - skipped\n"); + continue; + } + printf(" - passed\n"); + passed += 1; + } + + printf("Passed: %i\n", passed); + + return TEST_SUCCESS; +} + +static int +test_sn_plus_window(void) +{ + return test_sn_range_type(SN_RANGE_PLUS_WINDOW); +} + +static int +test_sn_minus_window(void) +{ + return test_sn_range_type(SN_RANGE_MINUS_WINDOW); +} + + +static int +test_sn_plus_outside(void) +{ + return test_sn_range_type(SN_RANGE_PLUS_OUTSIDE); +} + +static int +test_sn_minus_outside(void) +{ + return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE); +} + static int test_iterate_all(void) { @@ -632,6 +871,7 @@ test_iterate_all(void) printf(" - passed\n"); passed += 1; } + printf("Passed: %i\n", passed); return TEST_SUCCESS; @@ -643,8 +883,52 @@ test_sample(void) return test_iterate_all(); } -static struct unit_test_suite pdcp_testsuite = { - .suite_name = "PDCP Unit Test Suite", +static int +test_combined(void) +{ + struct pdcp_test_conf ul_conf, dl_conf; + int ret, nb_test, i, passed = 0; + + nb_test = RTE_DIM(pdcp_test_params); + + for (i = 0; i < nb_test; i++) { + create_test_conf_from_index(i, &ul_conf); + if (ul_conf.entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + continue; + + ul_conf.entity.reverse_iv_direction = true; + ul_conf.output_len = 0; + printf("[%03i] - %s", i, pdcp_test_params[i].name); + + ret = test_attempt_single(&ul_conf); + if (ret == TEST_FAILED) { + printf(" - failed\n"); + return ret; + } else if (ret == TEST_SKIPPED) { + printf(" - skipped\n"); + continue; + } + + uplink_to_downlink_convert(&ul_conf, &dl_conf); + ret = test_attempt_single(&dl_conf); + if (ret == TEST_FAILED) { + printf(" - failed\n"); + return ret; + } else if (ret == TEST_SKIPPED) { + printf(" - skipped\n"); + continue; + } + + printf(" - passed\n"); + passed += 1; + } + printf("Passed: %i\n", passed); + + return TEST_SUCCESS; +} + +static struct unit_test_suite known_vector_cases = { + .suite_name = "PDCP known vectors", .setup = testsuite_setup, .teardown = testsuite_teardown, .unit_test_cases = { @@ -654,10 +938,62 @@ static struct unit_test_suite pdcp_testsuite = { } }; +static struct unit_test_suite combined_mode_cases = { + .suite_name = "PDCP combined mode", + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_ST(ut_setup_pdcp, ut_teardown_pdcp, + test_combined), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static struct unit_test_suite hfn_sn_test_cases = { + .suite_name = "PDCP HFN/SN", + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_cases = { + TEST_CASE_ST(ut_setup_pdcp, ut_teardown_pdcp, + test_sn_plus_window), + TEST_CASE_ST(ut_setup_pdcp, ut_teardown_pdcp, + test_sn_minus_window), + TEST_CASE_ST(ut_setup_pdcp, ut_teardown_pdcp, + test_sn_plus_outside), + TEST_CASE_ST(ut_setup_pdcp, ut_teardown_pdcp, + test_sn_minus_outside), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + static int test_pdcp(void) { - return unit_test_suite_runner(&pdcp_testsuite); + int ret; + + static struct unit_test_suite pdcp_testsuite = { + .suite_name = "PDCP Unit Test Suite", + .unit_test_cases = {TEST_CASES_END()}, + }; + + struct unit_test_suite *static_suites[] = { + &known_vector_cases, + &combined_mode_cases, + &hfn_sn_test_cases, + NULL /* End of suites list */ + }; + + pdcp_testsuite.unit_test_suites = rte_zmalloc(NULL, sizeof(static_suites), 0); + if (pdcp_testsuite.unit_test_suites == NULL) { + RTE_LOG(ERR, USER1, "No memory for: '%s'\n", pdcp_testsuite.suite_name); + return TEST_FAILED; + } + + memcpy(pdcp_testsuite.unit_test_suites, static_suites, sizeof(static_suites)); + + ret = unit_test_suite_runner(&pdcp_testsuite); + rte_free(pdcp_testsuite.unit_test_suites); + return ret; } REGISTER_TEST_COMMAND(pdcp_autotest, test_pdcp); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH 5/5] doc: add PDCP library guide 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph ` (3 preceding siblings ...) 2022-12-22 9:25 ` [PATCH 4/5] app/test: pdcp HFN tests in combined mode Anoob Joseph @ 2022-12-22 9:25 ` Anoob Joseph 2023-01-18 16:39 ` [PATCH 0/5] lib: add pdcp protocol Thomas Monjalon 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph 6 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2022-12-22 9:25 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add guide for PDCP library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 211 ++++++++++++++++++ 3 files changed, 213 insertions(+) create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst diff --git a/doc/guides/prog_guide/img/pdcp_functional_overview.svg b/doc/guides/prog_guide/img/pdcp_functional_overview.svg new file mode 100644 index 0000000000..287daafc21 --- /dev/null +++ b/doc/guides/prog_guide/img/pdcp_functional_overview.svg @@ -0,0 +1 @@ +<svg width="1280" height="720" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><clipPath id="clip0"><rect x="0" y="0" width="1280" height="720"/></clipPath></defs><g clip-path="url(#clip0)"><rect x="0" y="0" width="1280" height="720" fill="#FFFFFF"/><rect x="202" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><rect x="640" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><path d="M605.5 73.5001 605.5 590.633" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M380.5 634 803.25 634 803.25 585.907 790.5 585.907 816 557.5 841.5 585.907 828.75 585.907 828.75 659.5 380.5 659.5Z" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5" fill-rule="evenodd"/><rect x="362.5" y="557.5" width="28" height="102" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5"/><rect x="412.5" y="671.5" width="370" height="32" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(492.364 694)">Radio Interface (<tspan font-size="19" x="137.5" y="0">Uu</tspan><tspan font-size="19" x="161.333" y="0">/PC5)</tspan><tspan font-size="19" x="-282.121" y="-653">UE/NG</tspan><tspan font-size="19" x="-222.955" y="-653">-</tspan><tspan font-size="19" x="-216.788" y="-653">RAN/UE A</tspan><tspan font-size="19" x="375.54" y="-653">NG</tspan>-<tspan font-size="19" x="409.706" y="-653">RAN/UE/UE B</tspan><tspan font-size="14" x="0.896851" y="-647">Transmitting </tspan><tspan font-size="14" x="1.31018" y="-631">PDCP entity</tspan><tspan font-size="14" x="167.401" y="-647">Receiving </tspan><tspan font-size="14" x="160.148" y="-631">PDCP entity</tspan></text><path d="M314.5 71.5001 431.364 71.6549" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M331.5 71.5001C331.5 65.9772 349.633 61.5001 372 61.5001 394.368 61.5001 412.5 65.9772 412.5 71.5001 412.5 77.0229 394.368 81.5001 372 81.5001 349.633 81.5001 331.5 77.0229 331.5 71.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M353.5 90.5001 363.5 90.5001 363.5 48.5001 383.5 48.5001 383.5 90.5001 393.5 90.5001 373.5 110.5Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M877.364 82.6549 760.5 82.5001" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M860.5 83.5001C860.5 89.0229 842.368 93.5001 820 93.5001 797.633 93.5001 779.5 89.0229 779.5 83.5001 779.5 77.9772 797.633 73.5001 820 73.5001 842.368 73.5001 860.5 77.9772 860.5 83.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M838.5 64.5 828.5 64.5 828.5 106.5 808.5 106.5 808.5 64.5 798.5 64.5 818.5 44.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><rect x="244.5" y="128.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(302.494 150)">Transmission buffer:<tspan font-size="19" x="-4.67999" y="23">Sequence</tspan><tspan font-size="19" x="84.32" y="23">numberin</tspan>g</text><rect x="244.5" y="199.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(297.314 222)">Header or uplink data<tspan font-size="19" x="34.08" y="23">Compressio</tspan>n</text><rect x="682.5" y="141.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(735.782 164)">Header or uplink data<tspan font-size="19" x="24.2466" y="23">Decompressio</tspan>n</text><rect x="244.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(300.314 505)">Routing / Duplication</text><rect x="244.5" y="437.5" width="285" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(309.734 460)">Add PDCP header</text><rect x="244.5" y="383.5" width="189" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(298.62 406)">Ciphering</text><rect x="244.5" y="333.5" width="189" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(261.953 356)">Integrity protection</text><path d="M472.167 267.5 472.167 409.802 470.833 409.802 470.833 267.5ZM475.5 408.468 471.5 416.468 467.5 408.468Z"/><path d="M332.167 267.5 332.167 319.552 330.833 319.552 330.833 267.5ZM335.5 318.218 331.5 326.218 327.5 318.219Z"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(253.644 291)">Packets associated <tspan font-size="19" x="14.0067" y="23">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 499.312 299)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="-2.75546e-15" y="45">PDCP SDU</tspan></text><rect x="682.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(728.535 505)">Remove PDCP Header</text><rect x="682.5" y="437.5" width="203" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(733.729 460)">Deciphering</text><rect x="682.5" y="389.5" width="203" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(702.159 412)">Integrity Verification</text><rect x="682.5" y="303.5" width="203" height="77" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(712.729 325)">Reception buffer:<tspan font-size="19" x="24.6667" y="23">Reordering</tspan><tspan font-size="19" x="-13.1667" y="45">Duplicate discardin</tspan><tspan font-size="19" x="144.167" y="45">g</tspan><tspan font-size="19" x="-10.0989" y="-84">Packets associated </tspan><tspan font-size="19" x="3.90784" y="-61">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 960.34 294)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="1" y="45">PDCP SDU</tspan></text><path d="M0.666667-8.2074e-07 0.666763 78.6116-0.66657 78.6116-0.666667 8.2074e-07ZM4.00009 77.2782 0.000104987 85.2782-3.9999 77.2782Z" transform="matrix(1 0 0 -1 779.5 296.778)"/><path d="M0.666667-2.88742e-07 0.666769 235.734-0.666565 235.734-0.666667 2.88742e-07ZM4.0001 234.401 0.000104987 242.401-3.9999 234.401Z" transform="matrix(1 0 0 -1 931.5 453.901)"/></g></svg> \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 8564883018..d062cd53ac 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -76,4 +76,5 @@ Programmer's Guide lto profile_app asan + pdcp_lib glossary diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst new file mode 100644 index 0000000000..8e89f0e3a1 --- /dev/null +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -0,0 +1,211 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2022 Marvell. + +PDCP Protocol Processing Library +================================ + +DPDK provides a library for PDCP protocol processing. The library utilizes +other DPDK libraries such as cryptodev, reorder etc to provide the application +with a transparent and high performant PDCP protocol processing library. + +The library abstracts complete PDCP protocol processing conforming to +``ETSI TS 138 323 V17.1.0 (2022-08)``. +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + +PDCP would involve the following operations, + +1. Transfer of user plane data +2. Transfer of control plane data +3. Header compression +4. Uplink data compression +5. Ciphering and integrity protection + +.. _figure_pdcp_functional_overview: + +.. figure:: img/pdcp_functional_overview.* + + PDCP functional overview new + +PDCP library would abstract the protocol offload features of the cryptodev and +would provide a uniform interface and consistent API usage to work with +cryptodev irrespective of the protocol offload features supported. + +PDCP entity API +--------------- + +PDCP library provides following control path APIs that is used to +configure various PDCP entities, + +1. rte_pdcp_entity_establish() +2. rte_pdcp_entity_suspend() +3. rte_pdcp_entity_release() + +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or +``rte_security_session`` based on the config. The sessions would be created/ +destroyed while corresponding PDCP entity operations are performed. + +PDCP packet processing API +-------------------------- + +PDCP processing is split into 2 parts. One before cryptodev processing +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return crypto +operations belonging to multiple entities, ``rte_pdcp_pkt_crypto_group()`` +is added to help grouping crypto operations belonging to same PDCP entity. + +Lib PDCP would allow application to use same API sequence while leveraging +protocol offload features enabled by ``rte_security`` library. Lib PDCP would +internally change the handles registered for ``pre_process`` and +``post_process`` based on features enabled in the entity. + +Lib PDCP would create the required sessions on the device provided in entity to +minimize the application requirements. Also, the crypto_op allocation and free +would also be done internally by lib PDCP to allow the library to create +crypto ops as required for the input packets. For example, when control PDUs are +received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP +is expected to handle it differently. + +.. code-block:: c + + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; + struct rte_crypto_op *cop[MAX_BURST_SIZE]; + struct rte_pdcp_group grp[MAX_BURST_SIZE]; + struct rte_pdcp_entity *pdcp_entity; + int nb_max_out_mb, ret, nb_grp; + + /* Create PDCP entity */ + pdcp_entity = rte_pdcp_entity_establish(&conf); + + /** + * Allocate buffer for holding mbufs returned during PDCP suspend, + * release & post-process APIs. + */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + /* Handle error */ + } + + while (1) { + /* Receive packet and form mbuf */ + + /** + * Prepare packets for crypto operation. Following operations + * would be done, + * + * Transmitting entity/UL (only data PDUs): + * - Perform compression + * - Assign sequence number + * - Add PDCP header + * - Create & prepare crypto_op + * - Prepare IV for crypto operation (auth_gen, encrypt) + * - Save original PDCP SDU (during PDCP re-establishment, + * unconfirmed PDCP SDUs need to crypto processed again and + * transmitted/re-transmitted) + * + * Receiving entity/DL: + * - Any control PDUs received would be processed and + * appropriate actions taken. If data PDU, continue. + * - Determine sequence number (based on HFN & per packet SN) + * - Prepare crypto_op + * - Prepare IV for crypto operation (decrypt, auth_verify) + */ + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, + nb_rx, &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) + != nb_success) { + /* Retry for enqueue failure packets */ + } + + ... + + ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, + MAX_BURST_SIZE); + if (ret == 0) + continue; + + /** + * Received a burst of completed crypto ops from cryptodev. It + * may belong to various entities. Group similar ones together + * for entity specific post-processing. + */ + + /** + * Groups similar entities together. Frees crypto op and based + * on crypto_op status, set mbuf->ol_flags which would be + * checked in rte_pdcp_pkt_post_process(). + */ + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); + + for (i = 0; i != nb_grp; i++) { + + /** + * Post process packets after crypto completion. + * Following operations would be done, + * + * Transmitting entity/UL: + * - Check crypto result + * + * Receiving entity/DL: + * - Check crypto operation status + * - Check for duplication (if yes, drop duplicate) + * - Perform decompression + * - Trim PDCP header + * - Hold packet (SDU) for in-order delivery (return + * completed packets as and when sequence is + * completed) + * - If not in sequence, cache the packet and start + * t-Reordering timer. When timer expires, the + * packets need to delivered to upper layers (not + * treated as error packets). + */ + nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, + grp[i].m, out_mb, + grp[i].cnt, + &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + /* Perform additional operations */ + + /** + * Transmitting entity/UL + * - If duplication is enabled, duplicate PDCP PDUs + * - When lower layers confirm reception of a PDCP PDU, + * it should be communicated to PDCP layer so that + * PDCP can drop the corresponding SDU + */ + } + } + + +Supported features +------------------ + +- 12 bit & 18 bit sequence numbers +- Uplink & downlink traffic +- HFN increment +- IV generation as required per algorithm + +Supported ciphering algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CTR +- SNOW3G-CIPHER +- ZUC-CIPHER + +Supported integrity protection algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CMAC +- SNOW3G-AUTH +- ZUC-AUTH -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [PATCH 0/5] lib: add pdcp protocol 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph ` (4 preceding siblings ...) 2022-12-22 9:25 ` [PATCH 5/5] doc: add PDCP library guide Anoob Joseph @ 2023-01-18 16:39 ` Thomas Monjalon 2023-01-23 17:36 ` Jerin Jacob 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph 6 siblings, 1 reply; 192+ messages in thread From: Thomas Monjalon @ 2023-01-18 16:39 UTC (permalink / raw) To: Anoob Joseph Cc: Akhil Goyal, Jerin Jacob, Konstantin Ananyev, dev, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, Olivier Matz, techboard For such new library, I think we need to vote in techboard to accept it. That's a process we had in recent years to keep the project scope in control. 22/12/2022 10:25, Anoob Joseph: > Add Packet Data Convergence Protocol (PDCP) processing library. > > The library is similar to lib_ipsec which provides IPsec processing > capabilities in DPDK. > > PDCP would involve roughly the following options, > 1. Transfer of user plane data > 2. Transfer of control plane data > 3. Header compression > 4. Uplink data compression > 5. Ciphering and integrity protection > > PDCP library provides following control path APIs that is used to > configure various PDCP entities, > 1. rte_pdcp_entity_establish() > 2. rte_pdcp_entity_suspend() > 3. rte_pdcp_entity_release() > > PDCP process is split into 2 parts. One before crypto processing > (rte_pdcp_pkt_pre_process()) and one after crypto processing > (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto > operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() > is added to help grouping crypto operations belonging to same entity. > > Similar to lib IPsec, lib PDCP would allow application to use same API > sequence while leveraging protocol offload features enabled by rte_security > library. Lib PDCP would internally change the handles registered for > *pre_process* and *post_process* based on features enabled in the entity. > > Lib PDCP would create the required sessions on the device provided in entity to > minimize the application requirements. Also, the crypto_op allocation and free > would also be done internally by lib PDCP to allow the library to create > crypto ops as required for the input packets. For example, when control PDUs are > received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP > is expected to handle it differently. ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [PATCH 0/5] lib: add pdcp protocol 2023-01-18 16:39 ` [PATCH 0/5] lib: add pdcp protocol Thomas Monjalon @ 2023-01-23 17:36 ` Jerin Jacob 0 siblings, 0 replies; 192+ messages in thread From: Jerin Jacob @ 2023-01-23 17:36 UTC (permalink / raw) To: Thomas Monjalon Cc: Anoob Joseph, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, dev, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, Olivier Matz, techboard, Nathan Southern, Srikanth Yalavarthi, fengchengwen On Wed, Jan 18, 2023 at 10:09 PM Thomas Monjalon <thomas@monjalon.net> wrote: > > For such new library, I think we need to vote in techboard to accept it. > That's a process we had in recent years to keep the project scope in control. Looks like this process not defined nor documented. In order to bring clarity for new library acceptance and make life easy for contributors (also, less forking the DPDK repo), IMO, we should document the process as there is a lot of effort involved and send multiple version, and finally it gets rejected due to various reasons. I would suggest, some guideline to get started for discussion. I would suggest taking up this for TB meeting to define it. 1)For new library(new directory under lib/ directory) needs to get approval from DPDK tech board. 2)Author needs to send any of the following to start the approval process -Send a write-up on the scope and purpose library. TB may ask for sending API specification RFC is the scope or purpose is not clear. OR -Send RFC with API specification. If it's above, following new libraries needs approval. So I suggest we take approval in coming TB meeting to conclude one way or another. 1) pdcp library (https://patches.dpdk.org/project/dpdk/list/?series=26246) 2) mldev library or device class(https://patches.dpdk.org/project/dpdk/list/?series=25753) 3) memara (https://patches.dpdk.org/project/dpdk/list/?series=26546) 4) pmu ( started as EAL file and moving to separate library at https://patches.dpdk.org/project/dpdk/list/?series=26616) Please add the libraries if I missed something. ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 00/22] lib: add pdcp protocol 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph ` (5 preceding siblings ...) 2023-01-18 16:39 ` [PATCH 0/5] lib: add pdcp protocol Thomas Monjalon @ 2023-04-14 17:44 ` Anoob Joseph 2023-04-14 17:44 ` [PATCH v2 01/22] net: add PDCP header Anoob Joseph ` (22 more replies) 6 siblings, 23 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following operations, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Similar to lib IPsec, lib PDCP would allow application to use same API sequence while leveraging protocol offload features enabled by rte_security library. Lib PDCP would internally change the handles registered for *pre_process* and *post_process* based on features enabled in the entity. Lib PDCP would create the required sessions on the device provided in entity to minimize the application requirements. Also, the crypto_op allocation and free would also be done internally by lib PDCP to allow the library to create crypto ops as required for the input packets. For example, when control PDUs are received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP is expected to handle it differently. Lib PDCP utilizes reorder library for implementing in-order delivery. It utilizes bitmap library for implementing status reports and track the COUNT value of the packets received. To allow application to choose timer implementation of choice, lib PDCP allows application to configure handles that can be used for starting & stopping timers. Upon expiry, application can call corresponding PDCP API(``rte_pdcp_t_reordering_expiry_handle``) for handling the event. Unit tests are added to verify both rte_timer based timers as well as rte_eventdev based timers. PDCP tracks the sequence number of the received packets and during events such as re-establishment, it is required to generate reports and transmit to the peer. This series introduces ``rte_pdcp_control_pdu_create`` for handling control PDU generation. Changes in v2: - Added control PDU handling - Added t-Reordering timer - Added in-order delivery - Added status PDU generation - Rebased on top of new features added in reorder library - Split base patch - Increased test coverage - Improved thread safety Changes from RFC - Implementation for all APIs covering basic control plane & user plane packets - Unit test leveraging existing PDCP vectors available in test_cryptodev - Unit test performing both UL & DL operations to verify various protocol features - Updated documentation Sample application sequence: struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; struct rte_crypto_op *cop[MAX_BURST_SIZE]; struct rte_pdcp_group grp[MAX_BURST_SIZE]; struct rte_pdcp_entity *pdcp_entity; int nb_max_out_mb, ret, nb_grp; /* Create PDCP entity */ pdcp_entity = rte_pdcp_entity_establish(&conf); /** * Allocate buffer for holding mbufs returned during PDCP suspend, * release & post-process APIs. */ /* Max packets that can be cached in entity + burst size */ nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); if (out_mb == NULL) { /* Handle error */ } while (1) { /* Receive packet and form mbuf */ /** * Prepare packets for crypto operation. Following operations * would be done, * * Transmitting entity/UL (only data PDUs): * - Perform compression * - Assign sequence number * - Add PDCP header * - Create & prepare crypto_op * - Prepare IV for crypto operation (auth_gen, encrypt) * - Save original PDCP SDU (during PDCP re-establishment, * unconfirmed PDCP SDUs need to crypto processed again and * transmitted/re-transmitted) * * Receiving entity/DL: * - Any control PDUs received would be processed and * appropriate actions taken. If data PDU, continue. * - Determine sequence number (based on HFN & per packet SN) * - Prepare crypto_op * - Prepare IV for crypto operation (decrypt, auth_verify) */ nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, nb_rx, &nb_err); if (nb_err != 0) { /* Handle error packets */ } if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) != nb_success) { /* Retry for enqueue failure packets */ } ... ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, MAX_BURST_SIZE); if (ret == 0) continue; /** * Received a burst of completed crypto ops from cryptodev. It * may belong to various entities. Group similar ones together * for entity specific post-processing. */ /** * Groups similar entities together. Frees crypto op and based * on crypto_op status, set mbuf->ol_flags which would be * checked in rte_pdcp_pkt_post_process(). */ nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); for (i = 0; i != nb_grp; i++) { /** * Post process packets after crypto completion. * Following operations would be done, * * Transmitting entity/UL: * - Check crypto result * * Receiving entity/DL: * - Check crypto operation status * - Check for duplication (if yes, drop duplicate) * - Perform decompression * - Trim PDCP header * - Hold packet (SDU) for in-order delivery (return * completed packets as and when sequence is * completed) * - If not in sequence, cache the packet and start * t-Reordering timer. When timer expires, the * packets need to delivered to upper layers (not * treated as error packets). */ nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, grp[i].m, out_mb, grp[i].cnt, &nb_err); if (nb_err != 0) { /* Handle error packets */ } /* Perform additional operations */ /** * Tranmitting entity/UL * - If duplication is enabled, duplicate PDCP PDUs * - When lower layers confirm reception of a PDCP PDU, * it should be communicated to PDCP layer so that * PDCP can drop the corresponding SDU */ } } Sample output from unit test executed on crypto_cn10k PMD(summary): ~# DPDK_TEST=pdcp_autotest ./dpdk-test --log-level=7 <snip> + ------------------------------------------------------- + + Test Suite Summary : PDCP Unit Test Suite + ------------------------------------------------------- + + Known vector cases : 109/160 passed, 0/160 skipped, 0/160 failed, 51/160 unsupported + PDCP combined mode : 1/1 passed, 0/1 skipped, 0/1 failed, 0/1 unsupported + PDCP HFN/SN : 4/4 passed, 0/4 skipped, 0/4 failed, 0/4 unsupported + PDCP reorder : 5/5 passed, 0/5 skipped, 0/5 failed, 0/5 unsupported + PDCP status report : 9/9 passed, 0/9 skipped, 0/9 failed, 0/9 unsupported + ------------------------------------------------------- + + Sub Testsuites Total : 5 + Sub Testsuites Skipped : 0 + Sub Testsuites Passed : 5 + Sub Testsuites Failed : 0 + ------------------------------------------------------- + + Tests Total : 179 + Tests Skipped : 0 + Tests Executed : 179 + Tests Unsupported: 51 + Tests Passed : 128 + Tests Failed : 0 + ------------------------------------------------------- + Test OK Anoob Joseph (10): lib: add pdcp protocol pdcp: add pre and post-process pdcp: add packet group pdcp: add crypto session create and destroy pdcp: add pre and post process for UL pdcp: add pre and post process for DL pdcp: add IV generation routines app/test: add lib pdcp tests doc: add PDCP library guide pdcp: add control PDU handling Volodymyr Fialko (12): net: add PDCP header test/pdcp: pdcp HFN tests in combined mode pdcp: implement t-Reordering and packet buffering test/pdcp: add in-order delivery cases pdcp: add timer callback handlers pdcp: add timer expiry handle test/pdcp: add timer expiry cases test/pdcp: add timer restart case pdcp: add support for status report pdcp: allocate reorder buffer alongside with entity pdcp: add thread safe processing test/pdcp: add PDCP status report cases app/test/meson.build | 1 + app/test/test_cryptodev.h | 3 + app/test/test_pdcp.c | 1923 +++++++++++++++++ doc/api/doxy-api-index.md | 4 +- doc/api/doxy-api.conf.in | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 285 +++ lib/meson.build | 1 + lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 140 ++ lib/pdcp/meson.build | 21 + lib/pdcp/pdcp_cnt.c | 164 ++ lib/pdcp/pdcp_cnt.h | 24 + lib/pdcp/pdcp_crypto.c | 237 ++ lib/pdcp/pdcp_crypto.h | 21 + lib/pdcp/pdcp_ctrl_pdu.c | 72 + lib/pdcp/pdcp_ctrl_pdu.h | 16 + lib/pdcp/pdcp_entity.h | 306 +++ lib/pdcp/pdcp_process.c | 1379 ++++++++++++ lib/pdcp/pdcp_process.h | 13 + lib/pdcp/pdcp_reorder.c | 20 + lib/pdcp/pdcp_reorder.h | 68 + lib/pdcp/rte_pdcp.c | 261 +++ lib/pdcp/rte_pdcp.h | 369 ++++ lib/pdcp/rte_pdcp_group.h | 125 ++ lib/pdcp/version.map | 17 + 27 files changed, 5473 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pdcp.c create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst create mode 100644 lib/net/rte_pdcp_hdr.h create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 01/22] net: add PDCP header 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-16 14:02 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 02/22] lib: add pdcp protocol Anoob Joseph ` (21 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add PDCP protocol header to be used for supporting PDCP protocol processing. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/api/doxy-api-index.md | 3 +- lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 140 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 143 insertions(+), 1 deletion(-) create mode 100644 lib/net/rte_pdcp_hdr.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index c709fd48ad..debbe4134f 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -127,7 +127,8 @@ The public API headers are grouped by topics: [Geneve](@ref rte_geneve.h), [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), - [PPP](@ref rte_ppp.h) + [PPP](@ref rte_ppp.h), + [PDCP hdr](@ref rte_pdcp_hdr.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/lib/net/meson.build b/lib/net/meson.build index 379d161ee0..bd56f91c22 100644 --- a/lib/net/meson.build +++ b/lib/net/meson.build @@ -22,6 +22,7 @@ headers = files( 'rte_geneve.h', 'rte_l2tpv2.h', 'rte_ppp.h', + 'rte_pdcp_hdr.h', ) sources = files( diff --git a/lib/net/rte_pdcp_hdr.h b/lib/net/rte_pdcp_hdr.h new file mode 100644 index 0000000000..87ecd379c4 --- /dev/null +++ b/lib/net/rte_pdcp_hdr.h @@ -0,0 +1,140 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_HDR_H +#define RTE_PDCP_HDR_H + +/** + * @file + * + * PDCP-related defines + * + * Based on - ETSI TS 138 323 V17.1.0 (2022-08) + * https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + */ + +#include <rte_byteorder.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * 4.3.1 + * + * Indicate the maximum supported size of a PDCP Control PDU. + */ +#define RTE_PDCP_CTRL_PDU_SIZE_MAX 9000u + +/** + * Indicate type of control information included in the corresponding PDCP + * Control PDU. + */ +enum rte_pdcp_ctrl_pdu_type { + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT = 0, + RTE_PDCP_CTRL_PDU_TYPE_ROHC_FEEDBACK = 1, + RTE_PDCP_CTRL_PDU_TYPE_EHC_FEEDBACK = 2, + RTE_PDCP_CRTL_PDU_TYPE_UDC_FEEDBACK = 3, +}; + +/** + * 6.3.7 D/C + * + * This field indicates whether the corresponding PDCP PDU is a + * PDCP Data PDU or a PDCP Control PDU. + */ +enum rte_pdcp_pdu_type { + RTE_PDCP_PDU_TYPE_CTRL = 0, + RTE_PDCP_PDU_TYPE_DATA = 1, +}; + +/** + * 6.2.2.1 Data PDU for SRBs + */ +__extension__ +struct rte_pdcp_cp_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 4; /**< Reserved */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.2.2 Data PDU for DRBs and MRBs with 12 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 3; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 3; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.2.3 Data PDU for DRBs and MRBs with 18 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_18_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ + uint8_t r : 5; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 5; /**< Reserved */ + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ +#endif + uint8_t sn_15_8; /**< Sequence number bits 8-15 */ + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.3.1 Control PDU for PDCP status report + */ +__extension__ +struct rte_pdcp_up_ctrl_pdu_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t r : 4; /**< Reserved */ +#endif + /** + * 6.3.9 FMC + * + * First Missing COUNT. This field indicates the COUNT value of the + * first missing PDCP SDU within the reordering window, i.e. RX_DELIV. + */ + rte_be32_t fmc; + /** + * 6.3.10 Bitmap + * + * Length: Variable. The length of the bitmap field can be 0. + * + * This field indicates which SDUs are missing and which SDUs are + * correctly received in the receiving PDCP entity. The bit position of + * Nth bit in the Bitmap is N, i.e., the bit position of the first bit + * in the Bitmap is 1. + */ + uint8_t bitmap[]; +} __rte_packed; + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_HDR_H */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 01/22] net: add PDCP header 2023-04-14 17:44 ` [PATCH v2 01/22] net: add PDCP header Anoob Joseph @ 2023-05-16 14:02 ` Akhil Goyal 0 siblings, 0 replies; 192+ messages in thread From: Akhil Goyal @ 2023-05-16 14:02 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz > Subject: [PATCH v2 01/22] net: add PDCP header > > From: Volodymyr Fialko <vfialko@marvell.com> > > Add PDCP protocol header to be used for supporting PDCP protocol > processing. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > --- Acked-by: Akhil Goyal <gakhil@marvell.com> ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 02/22] lib: add pdcp protocol 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph 2023-04-14 17:44 ` [PATCH v2 01/22] net: add PDCP header Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-16 15:30 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 03/22] pdcp: add pre and post-process Anoob Joseph ` (20 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + lib/meson.build | 1 + lib/pdcp/meson.build | 17 +++++ lib/pdcp/pdcp_crypto.c | 21 +++++ lib/pdcp/pdcp_crypto.h | 15 ++++ lib/pdcp/pdcp_entity.h | 95 +++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 138 +++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 13 ++++ lib/pdcp/rte_pdcp.c | 138 +++++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 157 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 10 +++ 12 files changed, 608 insertions(+), 1 deletion(-) create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/version.map diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index debbe4134f..cd7a6cae44 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -128,7 +128,8 @@ The public API headers are grouped by topics: [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), [PPP](@ref rte_ppp.h), - [PDCP hdr](@ref rte_pdcp_hdr.h) + [PDCP hdr](@ref rte_pdcp_hdr.h), + [PDCP](@ref rte_pdcp.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index d230a19e1f..58789308a9 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -62,6 +62,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/net \ @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ + @TOPDIR@/lib/pdcp \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ @TOPDIR@/lib/port \ diff --git a/lib/meson.build b/lib/meson.build index 0812ce6026..d217c04ea9 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -64,6 +64,7 @@ libraries = [ 'flow_classify', # flow_classify lib depends on pkt framework table lib 'graph', 'node', + 'pdcp', # pdcp lib depends on crypto and security ] optional_libs = [ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode 100644 index 0000000000..ccaf426240 --- /dev/null +++ b/lib/pdcp/meson.build @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +sources = files( + 'pdcp_crypto.c', + 'pdcp_process.c', + 'rte_pdcp.c', + ) +headers = files('rte_pdcp.h') + +deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c new file mode 100644 index 0000000000..755e27ec9e --- /dev/null +++ b/lib/pdcp/pdcp_crypto.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_crypto.h" + +int +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + RTE_SET_USED(entity); + RTE_SET_USED(conf); + return 0; +} + +void +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) +{ + RTE_SET_USED(entity); +} diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h new file mode 100644 index 0000000000..6563331d37 --- /dev/null +++ b/lib/pdcp/pdcp_crypto.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CRYPTO_H +#define PDCP_CRYPTO_H + +#include <rte_pdcp.h> + +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, + const struct rte_pdcp_entity_conf *conf); + +void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); + +#endif /* PDCP_CRYPTO_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h new file mode 100644 index 0000000000..ca1d56b516 --- /dev/null +++ b/lib/pdcp/pdcp_entity.h @@ -0,0 +1,95 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_ENTITY_H +#define PDCP_ENTITY_H + +#include <rte_common.h> +#include <rte_crypto_sym.h> +#include <rte_mempool.h> +#include <rte_pdcp.h> +#include <rte_security.h> + +struct entity_priv; + +/* IV generation function based on the entity configuration */ +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count); + +struct entity_state { + uint32_t rx_next; + uint32_t tx_next; + uint32_t rx_deliv; + uint32_t rx_reord; +}; + +/* + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + */ + +struct entity_priv { + /** Crypto sym session. */ + struct rte_cryptodev_sym_session *crypto_sess; + /** Entity specific IV generation function. */ + iv_gen_t iv_gen; + /** Entity state variables. */ + struct entity_state state; + /** Flags. */ + struct { + /** PDCP PDU has 4 byte MAC-I. */ + uint64_t is_authenticated : 1; + /** Cipher offset & length in bits. */ + uint64_t is_ciph_in_bits : 1; + /** Auth offset & length in bits. */ + uint64_t is_auth_in_bits : 1; + /** Is UL/transmitting PDCP entity. */ + uint64_t is_ul_entity : 1; + /** Is NULL auth. */ + uint64_t is_null_auth : 1; + } flags; + /** Crypto op pool. */ + struct rte_mempool *cop_pool; + /** PDCP header size. */ + uint8_t hdr_sz; + /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ + uint8_t aad_sz; + /** Device ID of the device to be used for offload. */ + uint8_t dev_id; +}; + +struct entity_priv_dl_part { + /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ + uint8_t dummy; +}; + +struct entity_priv_ul_part { + /* + * NOTE: when re-establish is supported, plain PDCP packets & COUNT values need to be + * cached. + */ + uint8_t dummy; +}; + +static inline struct entity_priv * +entity_priv_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); +} + +static inline struct entity_priv_dl_part * +entity_dl_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline struct entity_priv_ul_part * +entity_ul_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +#endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c new file mode 100644 index 0000000000..d4b158536d --- /dev/null +++ b/lib/pdcp/pdcp_process.c @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_memcpy.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, + struct rte_crypto_sym_xform **a_xfrm) +{ + *c_xfrm = NULL; + *a_xfrm = NULL; + + if (conf->crypto_xfrm == NULL) + return -EINVAL; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + *c_xfrm = conf->crypto_xfrm; + *a_xfrm = conf->crypto_xfrm->next; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + *a_xfrm = conf->crypto_xfrm; + *c_xfrm = conf->crypto_xfrm->next; + } else { + return -EINVAL; + } + + return 0; +} + +static int +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + int ret; + + /** + * flags.is_authenticated + * + * MAC-I would be added in case of control plane packets and when authentication + * transform is not NULL. + */ + + if (conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) + en_priv->flags.is_authenticated = 1; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (a_xfrm != NULL) + en_priv->flags.is_authenticated = 1; + + /** + * flags.is_ciph_in_bits + * + * For ZUC & SNOW3G cipher algos, offset & length need to be provided in bits. + */ + + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) + en_priv->flags.is_ciph_in_bits = 1; + + /** + * flags.is_auth_in_bits + * + * For ZUC & SNOW3G authentication algos, offset & length need to be provided in bits. + */ + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) + en_priv->flags.is_auth_in_bits = 1; + } + + /** + * flags.is_ul_entity + * + * Indicate whether the entity is UL/transmitting PDCP entity. + */ + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + en_priv->flags.is_ul_entity = 1; + + /** + * flags.is_null_auth + * + * For NULL auth, 4B zeros need to be added by lib PDCP. Indicate that + * algo is NULL auth to perform the same. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + en_priv->flags.is_null_auth = 1; + + /** + * hdr_sz + * + * PDCP header size of the entity + */ + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); + + /** + * aad_sz + * + * For AES-CMAC, additional message is prepended for processing. Need to be trimmed after + * crypto processing is done. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) + en_priv->aad_sz = 8; + else + en_priv->aad_sz = 0; + + return 0; +} + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv *en_priv; + int ret; + + if (entity == NULL || conf == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + ret = pdcp_entity_priv_populate(en_priv, conf); + if (ret) + return ret; + + return 0; +} diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h new file mode 100644 index 0000000000..fd53fff0aa --- /dev/null +++ b/lib/pdcp/pdcp_process.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_PROCESS_H +#define PDCP_PROCESS_H + +#include <rte_pdcp.h> + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode 100644 index 0000000000..8914548dbd --- /dev/null +++ b/lib/pdcp/rte_pdcp.c @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_pdcp.h> +#include <rte_malloc.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +{ + int size; + + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + size += sizeof(struct entity_priv_dl_part); + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size += sizeof(struct entity_priv_ul_part); + else + return -EINVAL; + + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); +} + +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) +{ + struct rte_pdcp_entity *entity = NULL; + struct entity_priv *en_priv; + int ret, entity_size; + + if (conf == NULL || conf->cop_pool == NULL) { + rte_errno = -EINVAL; + return NULL; + } + + if (conf->pdcp_xfrm.en_ordering || conf->pdcp_xfrm.remove_duplicates || conf->is_slrb || + conf->en_sec_offload) { + rte_errno = -ENOTSUP; + return NULL; + } + + /* + * 6.3.2 PDCP SN + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the PDCP SN is + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl-PDCP-SN-Size in + * TS 38.331 [3]) + */ + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { + rte_errno = -ENOTSUP; + return NULL; + } + + if (conf->pdcp_xfrm.hfn || conf->pdcp_xfrm.hfn_threshold) { + rte_errno = -EINVAL; + return NULL; + } + + entity_size = pdcp_entity_size_get(conf); + if (entity_size < 0) { + rte_errno = -EINVAL; + return NULL; + } + + entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (entity == NULL) { + rte_errno = -ENOMEM; + return NULL; + } + + en_priv = entity_priv_get(entity); + + en_priv->state.rx_deliv = conf->count; + en_priv->state.tx_next = conf->count; + en_priv->cop_pool = conf->cop_pool; + + /* Setup crypto session */ + ret = pdcp_crypto_sess_create(entity, conf); + if (ret) + goto entity_free; + + ret = pdcp_process_func_set(entity, conf); + if (ret) + goto crypto_sess_destroy; + + return entity; + +crypto_sess_destroy: + pdcp_crypto_sess_destroy(entity); +entity_free: + rte_free(entity); + rte_errno = ret; + return NULL; +} + +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) +{ + if (pdcp_entity == NULL) + return -EINVAL; + + /* Teardown crypto sessions */ + pdcp_crypto_sess_destroy(pdcp_entity); + + rte_free(pdcp_entity); + + RTE_SET_USED(out_mb); + return 0; +} + +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]) +{ + struct entity_priv *en_priv; + + if (pdcp_entity == NULL) + return -EINVAL; + + en_priv = entity_priv_get(pdcp_entity); + + if (en_priv->flags.is_ul_entity) { + en_priv->state.tx_next = 0; + } else { + en_priv->state.rx_next = 0; + en_priv->state.rx_deliv = 0; + } + + RTE_SET_USED(out_mb); + + return 0; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode 100644 index 0000000000..33c355b05a --- /dev/null +++ b/lib/pdcp/rte_pdcp.h @@ -0,0 +1,157 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_H +#define RTE_PDCP_H + +/** + * @file rte_pdcp.h + * + * RTE PDCP support. + * + * librte_pdcp provides a framework for PDCP protocol processing. + */ + +#include <rte_compat.h> +#include <rte_common.h> +#include <rte_mempool.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * PDCP entity. + */ +struct rte_pdcp_entity { + /** + * PDCP entities may hold packets for purposes of in-order delivery (in + * case of receiving PDCP entity) and re-transmission (in case of + * transmitting PDCP entity). + * + * For receiving PDCP entity, it may hold packets when in-order + * delivery is enabled. The packets would be cached until either a + * packet that completes the sequence arrives or when t-Reordering timer + * expires. + * + * When post-processing of PDCP packet which completes a sequence is + * done, the API may return more packets than enqueued. Application is + * expected to provide *rte_pdcp_pkt_post_process()* with *out_mb* + * which can hold maximum number of packets which may be returned. + */ + uint32_t max_pkt_cache; + /** User area for saving application data. */ + uint64_t user_area[2]; +} __rte_cache_aligned; + +/** + * PDCP entity configuration to be used for establishing an entity. + */ +/* Structure rte_pdcp_entity_conf 8< */ +struct rte_pdcp_entity_conf { + /** PDCP transform for the entity. */ + struct rte_security_pdcp_xform pdcp_xfrm; + /** Crypto transform applicable for the entity. */ + struct rte_crypto_sym_xform *crypto_xfrm; + /** Mempool for crypto symmetric session. */ + struct rte_mempool *sess_mpool; + /** Crypto op pool.*/ + struct rte_mempool *cop_pool; + /** + * 32 bit count value (HFN + SN) to be used for the first packet. + * pdcp_xfrm.hfn would be ignored as the HFN would be derived from this value. + */ + uint32_t count; + /** Indicate whether the PDCP entity belongs to Side Link Radio Bearer. */ + bool is_slrb; + /** Enable security offload on the device specified. */ + bool en_sec_offload; + /** Device on which security/crypto session need to be created. */ + uint8_t dev_id; + /** Reverse direction during IV generation. Can be used to simulate UE crypto processing.*/ + bool reverse_iv_direction; +}; +/* >8 End of structure rte_pdcp_entity_conf. */ + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.1 PDCP entity establishment + * + * Establish PDCP entity based on provided input configuration. + * + * @param conf + * Parameters to be used for initializing PDCP entity object. + * @return + * - Valid handle if success + * - NULL in case of failure. rte_errno will be set to error code + */ +__rte_experimental +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.3 PDCP entity release + * + * Release PDCP entity. + * + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be released. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.4 PDCP entity suspend + * + * Suspend PDCP entity. + * + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be suspended. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode 100644 index 0000000000..923e165f3f --- /dev/null +++ b/lib/pdcp/version.map @@ -0,0 +1,10 @@ +EXPERIMENTAL { + global: + + # added in 23.07 + rte_pdcp_entity_establish; + rte_pdcp_entity_release; + rte_pdcp_entity_suspend; + + local: *; +}; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 02/22] lib: add pdcp protocol 2023-04-14 17:44 ` [PATCH v2 02/22] lib: add pdcp protocol Anoob Joseph @ 2023-05-16 15:30 ` Akhil Goyal 2023-05-18 6:53 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-16 15:30 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Anoob, Fix check patch issues and please see some inline comments. > Subject: [PATCH v2 02/22] lib: add pdcp protocol > > Add Packet Data Convergence Protocol (PDCP) processing library. > > The library is similar to lib_ipsec which provides IPsec processing > capabilities in DPDK. > > PDCP would involve roughly the following options, > 1. Transfer of user plane data > 2. Transfer of control plane data > 3. Header compression > 4. Uplink data compression > 5. Ciphering and integrity protection > > PDCP library provides following control path APIs that is used to > configure various PDCP entities, > 1. rte_pdcp_entity_establish() > 2. rte_pdcp_entity_suspend() > 3. rte_pdcp_entity_release() > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > --- > doc/api/doxy-api-index.md | 3 +- > doc/api/doxy-api.conf.in | 1 + > lib/meson.build | 1 + > lib/pdcp/meson.build | 17 +++++ > lib/pdcp/pdcp_crypto.c | 21 +++++ > lib/pdcp/pdcp_crypto.h | 15 ++++ > lib/pdcp/pdcp_entity.h | 95 +++++++++++++++++++++++ > lib/pdcp/pdcp_process.c | 138 +++++++++++++++++++++++++++++++++ > lib/pdcp/pdcp_process.h | 13 ++++ > lib/pdcp/rte_pdcp.c | 138 +++++++++++++++++++++++++++++++++ > lib/pdcp/rte_pdcp.h | 157 ++++++++++++++++++++++++++++++++++++++ > lib/pdcp/version.map | 10 +++ > 12 files changed, 608 insertions(+), 1 deletion(-) > create mode 100644 lib/pdcp/meson.build > create mode 100644 lib/pdcp/pdcp_crypto.c > create mode 100644 lib/pdcp/pdcp_crypto.h > create mode 100644 lib/pdcp/pdcp_entity.h > create mode 100644 lib/pdcp/pdcp_process.c > create mode 100644 lib/pdcp/pdcp_process.h > create mode 100644 lib/pdcp/rte_pdcp.c > create mode 100644 lib/pdcp/rte_pdcp.h > create mode 100644 lib/pdcp/version.map > > diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md > index debbe4134f..cd7a6cae44 100644 > --- a/doc/api/doxy-api-index.md > +++ b/doc/api/doxy-api-index.md > @@ -128,7 +128,8 @@ The public API headers are grouped by topics: > [eCPRI](@ref rte_ecpri.h), > [L2TPv2](@ref rte_l2tpv2.h), > [PPP](@ref rte_ppp.h), > - [PDCP hdr](@ref rte_pdcp_hdr.h) > + [PDCP hdr](@ref rte_pdcp_hdr.h), > + [PDCP](@ref rte_pdcp.h) > > - **QoS**: > [metering](@ref rte_meter.h), > diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in > index d230a19e1f..58789308a9 100644 > --- a/doc/api/doxy-api.conf.in > +++ b/doc/api/doxy-api.conf.in > @@ -62,6 +62,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api- > index.md \ > @TOPDIR@/lib/net \ > @TOPDIR@/lib/pcapng \ > @TOPDIR@/lib/pci \ > + @TOPDIR@/lib/pdcp \ > @TOPDIR@/lib/pdump \ > @TOPDIR@/lib/pipeline \ > @TOPDIR@/lib/port \ > diff --git a/lib/meson.build b/lib/meson.build > index 0812ce6026..d217c04ea9 100644 > --- a/lib/meson.build > +++ b/lib/meson.build > @@ -64,6 +64,7 @@ libraries = [ > 'flow_classify', # flow_classify lib depends on pkt framework table lib > 'graph', > 'node', > + 'pdcp', # pdcp lib depends on crypto and security > ] > > optional_libs = [ > diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build > new file mode 100644 > index 0000000000..ccaf426240 > --- /dev/null > +++ b/lib/pdcp/meson.build > @@ -0,0 +1,17 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(C) 2023 Marvell. > + > +if is_windows > + build = false > + reason = 'not supported on Windows' > + subdir_done() > +endif > + > +sources = files( > + 'pdcp_crypto.c', > + 'pdcp_process.c', > + 'rte_pdcp.c', > + ) > +headers = files('rte_pdcp.h') > + > +deps += ['mbuf', 'net', 'cryptodev', 'security'] > diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c > new file mode 100644 > index 0000000000..755e27ec9e > --- /dev/null > +++ b/lib/pdcp/pdcp_crypto.c > @@ -0,0 +1,21 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#include <rte_pdcp.h> > + > +#include "pdcp_crypto.h" > + > +int > +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf) > +{ > + RTE_SET_USED(entity); > + RTE_SET_USED(conf); > + return 0; > +} > + > +void > +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) > +{ > + RTE_SET_USED(entity); > +} > diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h > new file mode 100644 > index 0000000000..6563331d37 > --- /dev/null > +++ b/lib/pdcp/pdcp_crypto.h > @@ -0,0 +1,15 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#ifndef PDCP_CRYPTO_H > +#define PDCP_CRYPTO_H > + > +#include <rte_pdcp.h> > + > +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, > + const struct rte_pdcp_entity_conf *conf); > + > +void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); > + > +#endif /* PDCP_CRYPTO_H */ > diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h > new file mode 100644 > index 0000000000..ca1d56b516 > --- /dev/null > +++ b/lib/pdcp/pdcp_entity.h > @@ -0,0 +1,95 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#ifndef PDCP_ENTITY_H > +#define PDCP_ENTITY_H > + > +#include <rte_common.h> > +#include <rte_crypto_sym.h> > +#include <rte_mempool.h> > +#include <rte_pdcp.h> > +#include <rte_security.h> > + > +struct entity_priv; > + > +/* IV generation function based on the entity configuration */ > +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, > + uint32_t count); > + > +struct entity_state { > + uint32_t rx_next; > + uint32_t tx_next; > + uint32_t rx_deliv; > + uint32_t rx_reord; > +}; > + > +/* > + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] > + */ > + > +struct entity_priv { > + /** Crypto sym session. */ > + struct rte_cryptodev_sym_session *crypto_sess; > + /** Entity specific IV generation function. */ > + iv_gen_t iv_gen; > + /** Entity state variables. */ > + struct entity_state state; > + /** Flags. */ > + struct { > + /** PDCP PDU has 4 byte MAC-I. */ > + uint64_t is_authenticated : 1; > + /** Cipher offset & length in bits. */ > + uint64_t is_ciph_in_bits : 1; > + /** Auth offset & length in bits. */ > + uint64_t is_auth_in_bits : 1; > + /** Is UL/transmitting PDCP entity. */ > + uint64_t is_ul_entity : 1; > + /** Is NULL auth. */ > + uint64_t is_null_auth : 1; > + } flags; > + /** Crypto op pool. */ > + struct rte_mempool *cop_pool; > + /** PDCP header size. */ > + uint8_t hdr_sz; > + /** PDCP AAD size. For AES-CMAC, additional message is prepended for > the operation. */ > + uint8_t aad_sz; > + /** Device ID of the device to be used for offload. */ > + uint8_t dev_id; > +}; > + > +struct entity_priv_dl_part { > + /* NOTE: when in-order-delivery is supported, post PDCP packets would > need to cached. */ > + uint8_t dummy; > +}; > + > +struct entity_priv_ul_part { > + /* > + * NOTE: when re-establish is supported, plain PDCP packets & COUNT > values need to be > + * cached. > + */ > + uint8_t dummy; > +}; > + > +static inline struct entity_priv * > +entity_priv_get(const struct rte_pdcp_entity *entity) { > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); > +} > + > +static inline struct entity_priv_dl_part * > +entity_dl_part_get(const struct rte_pdcp_entity *entity) { > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + > sizeof(struct entity_priv)); > +} > + > +static inline struct entity_priv_ul_part * > +entity_ul_part_get(const struct rte_pdcp_entity *entity) { > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + > sizeof(struct entity_priv)); > +} > + > +static inline int > +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) > +{ > + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; > +} > + > +#endif /* PDCP_ENTITY_H */ > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c > new file mode 100644 > index 0000000000..d4b158536d > --- /dev/null > +++ b/lib/pdcp/pdcp_process.c > @@ -0,0 +1,138 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#include <rte_crypto.h> > +#include <rte_crypto_sym.h> > +#include <rte_cryptodev.h> > +#include <rte_memcpy.h> > +#include <rte_pdcp.h> > +#include <rte_pdcp_hdr.h> > + > +#include "pdcp_crypto.h" > +#include "pdcp_entity.h" > +#include "pdcp_process.h" > + > +static int > +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct > rte_crypto_sym_xform **c_xfrm, > + struct rte_crypto_sym_xform **a_xfrm) > +{ > + *c_xfrm = NULL; > + *a_xfrm = NULL; > + > + if (conf->crypto_xfrm == NULL) > + return -EINVAL; > + > + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { > + *c_xfrm = conf->crypto_xfrm; > + *a_xfrm = conf->crypto_xfrm->next; > + } else if (conf->crypto_xfrm->type == > RTE_CRYPTO_SYM_XFORM_AUTH) { > + *a_xfrm = conf->crypto_xfrm; > + *c_xfrm = conf->crypto_xfrm->next; > + } else { > + return -EINVAL; > + } > + > + return 0; > +} > + > +static int > +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > + int ret; > + > + /** > + * flags.is_authenticated > + * > + * MAC-I would be added in case of control plane packets and when > authentication > + * transform is not NULL. > + */ > + > + if (conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_CONTROL) > + en_priv->flags.is_authenticated = 1; This check should be added after getting the xfrm. If domain == control and a_xfrm is NULL, then it should be error, right? > + > + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); > + if (ret) > + return ret; > + > + if (a_xfrm != NULL) > + en_priv->flags.is_authenticated = 1; > + > + /** > + * flags.is_ciph_in_bits > + * > + * For ZUC & SNOW3G cipher algos, offset & length need to be provided > in bits. > + */ > + > + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || > + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) > + en_priv->flags.is_ciph_in_bits = 1; > + > + /** > + * flags.is_auth_in_bits > + * > + * For ZUC & SNOW3G authentication algos, offset & length need to be > provided in bits. > + */ > + > + if (a_xfrm != NULL) { > + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) > || > + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) > + en_priv->flags.is_auth_in_bits = 1; > + } > + > + /** > + * flags.is_ul_entity > + * > + * Indicate whether the entity is UL/transmitting PDCP entity. > + */ > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > + en_priv->flags.is_ul_entity = 1; > + > + /** > + * flags.is_null_auth > + * > + * For NULL auth, 4B zeros need to be added by lib PDCP. Indicate that > + * algo is NULL auth to perform the same. > + */ > + if (a_xfrm != NULL && a_xfrm->auth.algo == > RTE_CRYPTO_AUTH_NULL) > + en_priv->flags.is_null_auth = 1; > + > + /** > + * hdr_sz > + * > + * PDCP header size of the entity > + */ > + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); > + > + /** > + * aad_sz > + * > + * For AES-CMAC, additional message is prepended for processing. Need > to be trimmed after > + * crypto processing is done. > + */ > + if (a_xfrm != NULL && a_xfrm->auth.algo == > RTE_CRYPTO_AUTH_AES_CMAC) > + en_priv->aad_sz = 8; > + else > + en_priv->aad_sz = 0; > + > + return 0; > +} > + > +int > +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct entity_priv *en_priv; > + int ret; > + > + if (entity == NULL || conf == NULL) > + return -EINVAL; > + > + en_priv = entity_priv_get(entity); > + > + ret = pdcp_entity_priv_populate(en_priv, conf); > + if (ret) > + return ret; > + > + return 0; > +} > diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h > new file mode 100644 > index 0000000000..fd53fff0aa > --- /dev/null > +++ b/lib/pdcp/pdcp_process.h > @@ -0,0 +1,13 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#ifndef PDCP_PROCESS_H > +#define PDCP_PROCESS_H > + > +#include <rte_pdcp.h> > + > +int > +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf); > + > +#endif /* PDCP_PROCESS_H */ > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c > new file mode 100644 > index 0000000000..8914548dbd > --- /dev/null > +++ b/lib/pdcp/rte_pdcp.c > @@ -0,0 +1,138 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#include <rte_errno.h> > +#include <rte_pdcp.h> > +#include <rte_malloc.h> > + > +#include "pdcp_crypto.h" > +#include "pdcp_entity.h" > +#include "pdcp_process.h" > + > +static int > +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) > +{ > + int size; > + > + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); > + > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > + size += sizeof(struct entity_priv_dl_part); > + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > + size += sizeof(struct entity_priv_ul_part); > + else > + return -EINVAL; > + > + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); > +} > + > +struct rte_pdcp_entity * > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) > +{ > + struct rte_pdcp_entity *entity = NULL; > + struct entity_priv *en_priv; > + int ret, entity_size; > + > + if (conf == NULL || conf->cop_pool == NULL) { > + rte_errno = -EINVAL; > + return NULL; > + } errnos are normally set as positive values. > + > + if (conf->pdcp_xfrm.en_ordering || conf- > >pdcp_xfrm.remove_duplicates || conf->is_slrb || > + conf->en_sec_offload) { > + rte_errno = -ENOTSUP; > + return NULL; > + } > + > + /* > + * 6.3.2 PDCP SN > + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the > PDCP SN is > + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl- > PDCP-SN-Size in > + * TS 38.331 [3]) > + */ > + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { > + rte_errno = -ENOTSUP; > + return NULL; > + } Check for PDCP crypto algos may also be added. As only 4 cipher and 4 auth algos are supported in case of PDCP. > + > + if (conf->pdcp_xfrm.hfn || conf->pdcp_xfrm.hfn_threshold) { > + rte_errno = -EINVAL; > + return NULL; > + } What is the reason to set errno as EINVAL when HFN is set? > + > + entity_size = pdcp_entity_size_get(conf); > + if (entity_size < 0) { > + rte_errno = -EINVAL; > + return NULL; > + } > + > + entity = rte_zmalloc_socket("pdcp_entity", entity_size, > RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); > + if (entity == NULL) { > + rte_errno = -ENOMEM; > + return NULL; > + } > + > + en_priv = entity_priv_get(entity); > + > + en_priv->state.rx_deliv = conf->count; > + en_priv->state.tx_next = conf->count; > + en_priv->cop_pool = conf->cop_pool; > + > + /* Setup crypto session */ > + ret = pdcp_crypto_sess_create(entity, conf); > + if (ret) > + goto entity_free; > + > + ret = pdcp_process_func_set(entity, conf); > + if (ret) > + goto crypto_sess_destroy; > + > + return entity; > + > +crypto_sess_destroy: > + pdcp_crypto_sess_destroy(entity); > +entity_free: > + rte_free(entity); > + rte_errno = ret; > + return NULL; > +} > + > +int > +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf > *out_mb[]) > +{ > + if (pdcp_entity == NULL) > + return -EINVAL; > + > + /* Teardown crypto sessions */ > + pdcp_crypto_sess_destroy(pdcp_entity); > + > + rte_free(pdcp_entity); > + > + RTE_SET_USED(out_mb); > + return 0; > +} > + > +int > +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > + struct rte_mbuf *out_mb[]) > +{ > + struct entity_priv *en_priv; > + > + if (pdcp_entity == NULL) > + return -EINVAL; > + > + en_priv = entity_priv_get(pdcp_entity); > + > + if (en_priv->flags.is_ul_entity) { > + en_priv->state.tx_next = 0; > + } else { > + en_priv->state.rx_next = 0; > + en_priv->state.rx_deliv = 0; > + } > + > + RTE_SET_USED(out_mb); > + > + return 0; > +} > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h > new file mode 100644 > index 0000000000..33c355b05a > --- /dev/null > +++ b/lib/pdcp/rte_pdcp.h > @@ -0,0 +1,157 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#ifndef RTE_PDCP_H > +#define RTE_PDCP_H > + > +/** > + * @file rte_pdcp.h > + * > + * RTE PDCP support. > + * > + * librte_pdcp provides a framework for PDCP protocol processing. A framework for PDCP protocol processing. > + */ > + > +#include <rte_compat.h> > +#include <rte_common.h> > +#include <rte_mempool.h> > +#include <rte_security.h> > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +/** > + * PDCP entity. You can probably explain more on what a PDCP entity is. > + */ > +struct rte_pdcp_entity { > + /** > + * PDCP entities may hold packets for purposes of in-order delivery (in > + * case of receiving PDCP entity) and re-transmission (in case of > + * transmitting PDCP entity). > + * > + * For receiving PDCP entity, it may hold packets when in-order > + * delivery is enabled. The packets would be cached until either a > + * packet that completes the sequence arrives or when t-Reordering > timer > + * expires. > + * > + * When post-processing of PDCP packet which completes a sequence is > + * done, the API may return more packets than enqueued. Application is > + * expected to provide *rte_pdcp_pkt_post_process()* with *out_mb* > + * which can hold maximum number of packets which may be returned. > + */ The above comment explains the need for holding the packets. But it does not talk about the parameter it is explaining. This explanation should be part of programmer's guide and not the API guide. > + uint32_t max_pkt_cache; > + /** User area for saving application data. */ > + uint64_t user_area[2]; Is it being used right now in the patches? If not, can we add it later? And if really needed now, can we rename to user_data > +} __rte_cache_aligned; > + > +/** > + * PDCP entity configuration to be used for establishing an entity. > + */ > +/* Structure rte_pdcp_entity_conf 8< */ > +struct rte_pdcp_entity_conf { > + /** PDCP transform for the entity. */ > + struct rte_security_pdcp_xform pdcp_xfrm; > + /** Crypto transform applicable for the entity. */ > + struct rte_crypto_sym_xform *crypto_xfrm; > + /** Mempool for crypto symmetric session. */ > + struct rte_mempool *sess_mpool; > + /** Crypto op pool.*/ > + struct rte_mempool *cop_pool; > + /** > + * 32 bit count value (HFN + SN) to be used for the first packet. > + * pdcp_xfrm.hfn would be ignored as the HFN would be derived from > this value. > + */ If the HFN is to be ignored, then why to add a check in entity establish and return EINVAL? It should be silently ignored in that case with a debug print at max. > + uint32_t count; > + /** Indicate whether the PDCP entity belongs to Side Link Radio Bearer. > */ > + bool is_slrb; > + /** Enable security offload on the device specified. */ > + bool en_sec_offload; > + /** Device on which security/crypto session need to be created. */ > + uint8_t dev_id; > + /** Reverse direction during IV generation. Can be used to simulate UE > crypto processing.*/ > + bool reverse_iv_direction; > +}; > +/* >8 End of structure rte_pdcp_entity_conf. */ > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * 5.1.1 PDCP entity establishment > + * > + * Establish PDCP entity based on provided input configuration. > + * > + * @param conf > + * Parameters to be used for initializing PDCP entity object. > + * @return > + * - Valid handle if success > + * - NULL in case of failure. rte_errno will be set to error code > + */ > +__rte_experimental > +struct rte_pdcp_entity * > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * 5.1.3 PDCP entity release > + * > + * Release PDCP entity. > + * > + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. > + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in > + * *out_mb* buffer. The buffer should be large enough to hold all cached > + * packets in the entity. > + * > + * @param pdcp_entity > + * Pointer to the PDCP entity to be released. > + * @param[out] out_mb > + * The address of an array that can hold up to > *rte_pdcp_entity.max_pkt_cache* > + * pointers to *rte_mbuf* structures. > + * @return > + * - 0: Success and no cached packets to return > + * - >0: Success and the number of packets returned in out_mb > + * - <0: Error code in case of failures > + */ > +__rte_experimental > +int > +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, > + struct rte_mbuf *out_mb[]); > + > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * 5.1.4 PDCP entity suspend > + * > + * Suspend PDCP entity. > + * > + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in > + * *out_mb* buffer. The buffer should be large enough to hold all cached > + * packets in the entity. > + * > + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. > + * > + * @param pdcp_entity > + * Pointer to the PDCP entity to be suspended. > + * @param[out] out_mb > + * The address of an array that can hold up to > *rte_pdcp_entity.max_pkt_cache* > + * pointers to *rte_mbuf* structures. > + * @return > + * - 0: Success and no cached packets to return > + * - >0: Success and the number of packets returned in out_mb > + * - <0: Error code in case of failures > + */ > +__rte_experimental > +int > +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > + struct rte_mbuf *out_mb[]); > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif /* RTE_PDCP_H */ > diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map > new file mode 100644 > index 0000000000..923e165f3f > --- /dev/null > +++ b/lib/pdcp/version.map > @@ -0,0 +1,10 @@ > +EXPERIMENTAL { > + global: > + > + # added in 23.07 > + rte_pdcp_entity_establish; > + rte_pdcp_entity_release; > + rte_pdcp_entity_suspend; > + > + local: *; > +}; > -- > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 02/22] lib: add pdcp protocol 2023-05-16 15:30 ` Akhil Goyal @ 2023-05-18 6:53 ` Anoob Joseph 2023-05-18 7:40 ` Akhil Goyal 0 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-05-18 6:53 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Thanks for the review. Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Tuesday, May 16, 2023 9:01 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 02/22] lib: add pdcp protocol > > Hi Anoob, > > Fix check patch issues and please see some inline comments. [Anoob] Checkpatch warnings are false positives. I'll try to work around couple of them. Rest we may need to ignore. > > > Subject: [PATCH v2 02/22] lib: add pdcp protocol > > > > Add Packet Data Convergence Protocol (PDCP) processing library. > > > > The library is similar to lib_ipsec which provides IPsec processing > > capabilities in DPDK. > > > > PDCP would involve roughly the following options, 1. Transfer of user > > plane data 2. Transfer of control plane data 3. Header compression 4. > > Uplink data compression 5. Ciphering and integrity protection > > > > PDCP library provides following control path APIs that is used to > > configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. > > rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() > > > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > > --- > > doc/api/doxy-api-index.md | 3 +- > > doc/api/doxy-api.conf.in | 1 + > > lib/meson.build | 1 + > > lib/pdcp/meson.build | 17 +++++ > > lib/pdcp/pdcp_crypto.c | 21 +++++ > > lib/pdcp/pdcp_crypto.h | 15 ++++ > > lib/pdcp/pdcp_entity.h | 95 +++++++++++++++++++++++ > > lib/pdcp/pdcp_process.c | 138 > +++++++++++++++++++++++++++++++++ > > lib/pdcp/pdcp_process.h | 13 ++++ > > lib/pdcp/rte_pdcp.c | 138 +++++++++++++++++++++++++++++++++ > > lib/pdcp/rte_pdcp.h | 157 > ++++++++++++++++++++++++++++++++++++++ > > lib/pdcp/version.map | 10 +++ > > 12 files changed, 608 insertions(+), 1 deletion(-) create mode > > 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c > > create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 > > lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c > > create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 > > lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create > > mode 100644 lib/pdcp/version.map > > > > diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md > > index debbe4134f..cd7a6cae44 100644 > > --- a/doc/api/doxy-api-index.md > > +++ b/doc/api/doxy-api-index.md > > @@ -128,7 +128,8 @@ The public API headers are grouped by topics: > > [eCPRI](@ref rte_ecpri.h), > > [L2TPv2](@ref rte_l2tpv2.h), > > [PPP](@ref rte_ppp.h), > > - [PDCP hdr](@ref rte_pdcp_hdr.h) > > + [PDCP hdr](@ref rte_pdcp_hdr.h), > > + [PDCP](@ref rte_pdcp.h) > > > > - **QoS**: > > [metering](@ref rte_meter.h), > > diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index > > d230a19e1f..58789308a9 100644 > > --- a/doc/api/doxy-api.conf.in > > +++ b/doc/api/doxy-api.conf.in > > @@ -62,6 +62,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api- > > index.md \ > > @TOPDIR@/lib/net \ > > @TOPDIR@/lib/pcapng \ > > @TOPDIR@/lib/pci \ > > + @TOPDIR@/lib/pdcp \ > > @TOPDIR@/lib/pdump \ > > @TOPDIR@/lib/pipeline \ > > @TOPDIR@/lib/port \ diff --git > > a/lib/meson.build b/lib/meson.build index 0812ce6026..d217c04ea9 > > 100644 > > --- a/lib/meson.build > > +++ b/lib/meson.build > > @@ -64,6 +64,7 @@ libraries = [ > > 'flow_classify', # flow_classify lib depends on pkt framework table lib > > 'graph', > > 'node', > > + 'pdcp', # pdcp lib depends on crypto and security > > ] > > > > optional_libs = [ > > diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode > > 100644 index 0000000000..ccaf426240 > > --- /dev/null > > +++ b/lib/pdcp/meson.build > > @@ -0,0 +1,17 @@ > > +# SPDX-License-Identifier: BSD-3-Clause # Copyright(C) 2023 Marvell. > > + > > +if is_windows > > + build = false > > + reason = 'not supported on Windows' > > + subdir_done() > > +endif > > + > > +sources = files( > > + 'pdcp_crypto.c', > > + 'pdcp_process.c', > > + 'rte_pdcp.c', > > + ) > > +headers = files('rte_pdcp.h') > > + > > +deps += ['mbuf', 'net', 'cryptodev', 'security'] > > diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c new file > > mode 100644 index 0000000000..755e27ec9e > > --- /dev/null > > +++ b/lib/pdcp/pdcp_crypto.c > > @@ -0,0 +1,21 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#include <rte_pdcp.h> > > + > > +#include "pdcp_crypto.h" > > + > > +int > > +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + RTE_SET_USED(entity); > > + RTE_SET_USED(conf); > > + return 0; > > +} > > + > > +void > > +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) { > > + RTE_SET_USED(entity); > > +} > > diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h new file > > mode 100644 index 0000000000..6563331d37 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_crypto.h > > @@ -0,0 +1,15 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#ifndef PDCP_CRYPTO_H > > +#define PDCP_CRYPTO_H > > + > > +#include <rte_pdcp.h> > > + > > +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, > > + const struct rte_pdcp_entity_conf *conf); > > + > > +void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); > > + > > +#endif /* PDCP_CRYPTO_H */ > > diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h new file > > mode 100644 index 0000000000..ca1d56b516 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_entity.h > > @@ -0,0 +1,95 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#ifndef PDCP_ENTITY_H > > +#define PDCP_ENTITY_H > > + > > +#include <rte_common.h> > > +#include <rte_crypto_sym.h> > > +#include <rte_mempool.h> > > +#include <rte_pdcp.h> > > +#include <rte_security.h> > > + > > +struct entity_priv; > > + > > +/* IV generation function based on the entity configuration */ > > +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct > > +entity_priv > > *en_priv, > > + uint32_t count); > > + > > +struct entity_state { > > + uint32_t rx_next; > > + uint32_t tx_next; > > + uint32_t rx_deliv; > > + uint32_t rx_reord; > > +}; > > + > > +/* > > + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] > > +[entity_dl/ul] */ > > + > > +struct entity_priv { > > + /** Crypto sym session. */ > > + struct rte_cryptodev_sym_session *crypto_sess; > > + /** Entity specific IV generation function. */ > > + iv_gen_t iv_gen; > > + /** Entity state variables. */ > > + struct entity_state state; > > + /** Flags. */ > > + struct { > > + /** PDCP PDU has 4 byte MAC-I. */ > > + uint64_t is_authenticated : 1; > > + /** Cipher offset & length in bits. */ > > + uint64_t is_ciph_in_bits : 1; > > + /** Auth offset & length in bits. */ > > + uint64_t is_auth_in_bits : 1; > > + /** Is UL/transmitting PDCP entity. */ > > + uint64_t is_ul_entity : 1; > > + /** Is NULL auth. */ > > + uint64_t is_null_auth : 1; > > + } flags; > > + /** Crypto op pool. */ > > + struct rte_mempool *cop_pool; > > + /** PDCP header size. */ > > + uint8_t hdr_sz; > > + /** PDCP AAD size. For AES-CMAC, additional message is prepended > for > > the operation. */ > > + uint8_t aad_sz; > > + /** Device ID of the device to be used for offload. */ > > + uint8_t dev_id; > > +}; > > + > > +struct entity_priv_dl_part { > > + /* NOTE: when in-order-delivery is supported, post PDCP packets > > +would > > need to cached. */ > > + uint8_t dummy; > > +}; > > + > > +struct entity_priv_ul_part { > > + /* > > + * NOTE: when re-establish is supported, plain PDCP packets & > COUNT > > values need to be > > + * cached. > > + */ > > + uint8_t dummy; > > +}; > > + > > +static inline struct entity_priv * > > +entity_priv_get(const struct rte_pdcp_entity *entity) { > > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); } > > + > > +static inline struct entity_priv_dl_part * entity_dl_part_get(const > > +struct rte_pdcp_entity *entity) { > > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + > > sizeof(struct entity_priv)); > > +} > > + > > +static inline struct entity_priv_ul_part * entity_ul_part_get(const > > +struct rte_pdcp_entity *entity) { > > + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + > > sizeof(struct entity_priv)); > > +} > > + > > +static inline int > > +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { > > + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } > > + > > +#endif /* PDCP_ENTITY_H */ > > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c new > > file mode 100644 index 0000000000..d4b158536d > > --- /dev/null > > +++ b/lib/pdcp/pdcp_process.c > > @@ -0,0 +1,138 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#include <rte_crypto.h> > > +#include <rte_crypto_sym.h> > > +#include <rte_cryptodev.h> > > +#include <rte_memcpy.h> > > +#include <rte_pdcp.h> > > +#include <rte_pdcp_hdr.h> > > + > > +#include "pdcp_crypto.h" > > +#include "pdcp_entity.h" > > +#include "pdcp_process.h" > > + > > +static int > > +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct > > rte_crypto_sym_xform **c_xfrm, > > + struct rte_crypto_sym_xform **a_xfrm) { > > + *c_xfrm = NULL; > > + *a_xfrm = NULL; > > + > > + if (conf->crypto_xfrm == NULL) > > + return -EINVAL; > > + > > + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) > { > > + *c_xfrm = conf->crypto_xfrm; > > + *a_xfrm = conf->crypto_xfrm->next; > > + } else if (conf->crypto_xfrm->type == > > RTE_CRYPTO_SYM_XFORM_AUTH) { > > + *a_xfrm = conf->crypto_xfrm; > > + *c_xfrm = conf->crypto_xfrm->next; > > + } else { > > + return -EINVAL; > > + } > > + > > + return 0; > > +} > > + > > +static int > > +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > > + int ret; > > + > > + /** > > + * flags.is_authenticated > > + * > > + * MAC-I would be added in case of control plane packets and when > > authentication > > + * transform is not NULL. > > + */ > > + > > + if (conf->pdcp_xfrm.domain == > > RTE_SECURITY_PDCP_MODE_CONTROL) > > + en_priv->flags.is_authenticated = 1; > > This check should be added after getting the xfrm. > If domain == control and a_xfrm is NULL, then it should be error, right? [Anoob] Lib PDCP would handle such cases. Even if a_xfrm is non NULL but is NULL auth, it is lib PDCP which would add zeroized digest. And a_xfrm == NULL is also treated as NULL auth generally. The comment above this explains the same. Idea is to have lib PDCP handle all possible cases rather than putting too much restrictions on both app & PMD. > > > + > > + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); > > + if (ret) > > + return ret; > > + > > + if (a_xfrm != NULL) > > + en_priv->flags.is_authenticated = 1; > > + > > + /** > > + * flags.is_ciph_in_bits > > + * > > + * For ZUC & SNOW3G cipher algos, offset & length need to be > > +provided > > in bits. > > + */ > > + > > + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) > || > > + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) > > + en_priv->flags.is_ciph_in_bits = 1; > > + > > + /** > > + * flags.is_auth_in_bits > > + * > > + * For ZUC & SNOW3G authentication algos, offset & length need to > be > > provided in bits. > > + */ > > + > > + if (a_xfrm != NULL) { > > + if ((a_xfrm->auth.algo == > RTE_CRYPTO_AUTH_SNOW3G_UIA2) > > || > > + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) > > + en_priv->flags.is_auth_in_bits = 1; > > + } > > + > > + /** > > + * flags.is_ul_entity > > + * > > + * Indicate whether the entity is UL/transmitting PDCP entity. > > + */ > > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > > + en_priv->flags.is_ul_entity = 1; > > + > > + /** > > + * flags.is_null_auth > > + * > > + * For NULL auth, 4B zeros need to be added by lib PDCP. Indicate > that > > + * algo is NULL auth to perform the same. > > + */ > > + if (a_xfrm != NULL && a_xfrm->auth.algo == > > RTE_CRYPTO_AUTH_NULL) > > + en_priv->flags.is_null_auth = 1; > > + > > + /** > > + * hdr_sz > > + * > > + * PDCP header size of the entity > > + */ > > + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); > > + > > + /** > > + * aad_sz > > + * > > + * For AES-CMAC, additional message is prepended for processing. > > +Need > > to be trimmed after > > + * crypto processing is done. > > + */ > > + if (a_xfrm != NULL && a_xfrm->auth.algo == > > RTE_CRYPTO_AUTH_AES_CMAC) > > + en_priv->aad_sz = 8; > > + else > > + en_priv->aad_sz = 0; > > + > > + return 0; > > +} > > + > > +int > > +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + struct entity_priv *en_priv; > > + int ret; > > + > > + if (entity == NULL || conf == NULL) > > + return -EINVAL; > > + > > + en_priv = entity_priv_get(entity); > > + > > + ret = pdcp_entity_priv_populate(en_priv, conf); > > + if (ret) > > + return ret; > > + > > + return 0; > > +} > > diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h new > > file mode 100644 index 0000000000..fd53fff0aa > > --- /dev/null > > +++ b/lib/pdcp/pdcp_process.h > > @@ -0,0 +1,13 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#ifndef PDCP_PROCESS_H > > +#define PDCP_PROCESS_H > > + > > +#include <rte_pdcp.h> > > + > > +int > > +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf); > > + > > +#endif /* PDCP_PROCESS_H */ > > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode > > 100644 index 0000000000..8914548dbd > > --- /dev/null > > +++ b/lib/pdcp/rte_pdcp.c > > @@ -0,0 +1,138 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#include <rte_errno.h> > > +#include <rte_pdcp.h> > > +#include <rte_malloc.h> > > + > > +#include "pdcp_crypto.h" > > +#include "pdcp_entity.h" > > +#include "pdcp_process.h" > > + > > +static int > > +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { > > + int size; > > + > > + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); > > + > > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > > + size += sizeof(struct entity_priv_dl_part); > > + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > > + size += sizeof(struct entity_priv_ul_part); > > + else > > + return -EINVAL; > > + > > + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } > > + > > +struct rte_pdcp_entity * > > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { > > + struct rte_pdcp_entity *entity = NULL; > > + struct entity_priv *en_priv; > > + int ret, entity_size; > > + > > + if (conf == NULL || conf->cop_pool == NULL) { > > + rte_errno = -EINVAL; > > + return NULL; > > + } > > errnos are normally set as positive values. [Anoob] I do not think so. I checked rte_ethdev.h, rte_flow.h etc and all APIs are returning negative values in case of errors. > > > > + > > + if (conf->pdcp_xfrm.en_ordering || conf- > > >pdcp_xfrm.remove_duplicates || conf->is_slrb || > > + conf->en_sec_offload) { > > + rte_errno = -ENOTSUP; > > + return NULL; > > + } > > + > > + /* > > + * 6.3.2 PDCP SN > > + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length > > +of the > > PDCP SN is > > + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or > > +sl- > > PDCP-SN-Size in > > + * TS 38.331 [3]) > > + */ > > + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { > > + rte_errno = -ENOTSUP; > > + return NULL; > > + } > > Check for PDCP crypto algos may also be added. > As only 4 cipher and 4 auth algos are supported in case of PDCP. [Anoob] Validation happens when we create session. Please check, pdcp: add crypto session create and destroy > > > + > > + if (conf->pdcp_xfrm.hfn || conf->pdcp_xfrm.hfn_threshold) { > > + rte_errno = -EINVAL; > > + return NULL; > > + } > > What is the reason to set errno as EINVAL when HFN is set? [Anoob] HFN is part of pdcp_xfrm which is defined in rte_security. Lib PDCP allows user to specify complete 32 bit count value using rte_pdcp_entity_conf.count. Since HFN is also used to construct 32 bit count value, having two ways to set count would be misleading. Hence lib PDCP would enforce that application does not set this value. > > > + > > + entity_size = pdcp_entity_size_get(conf); > > + if (entity_size < 0) { > > + rte_errno = -EINVAL; > > + return NULL; > > + } > > + > > + entity = rte_zmalloc_socket("pdcp_entity", entity_size, > > RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); > > + if (entity == NULL) { > > + rte_errno = -ENOMEM; > > + return NULL; > > + } > > + > > + en_priv = entity_priv_get(entity); > > + > > + en_priv->state.rx_deliv = conf->count; > > + en_priv->state.tx_next = conf->count; > > + en_priv->cop_pool = conf->cop_pool; > > + > > + /* Setup crypto session */ > > + ret = pdcp_crypto_sess_create(entity, conf); > > + if (ret) > > + goto entity_free; > > + > > + ret = pdcp_process_func_set(entity, conf); > > + if (ret) > > + goto crypto_sess_destroy; > > + > > + return entity; > > + > > +crypto_sess_destroy: > > + pdcp_crypto_sess_destroy(entity); > > +entity_free: > > + rte_free(entity); > > + rte_errno = ret; > > + return NULL; > > +} > > + > > +int > > +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct > > +rte_mbuf > > *out_mb[]) > > +{ > > + if (pdcp_entity == NULL) > > + return -EINVAL; > > + > > + /* Teardown crypto sessions */ > > + pdcp_crypto_sess_destroy(pdcp_entity); > > + > > + rte_free(pdcp_entity); > > + > > + RTE_SET_USED(out_mb); > > + return 0; > > +} > > + > > +int > > +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > > + struct rte_mbuf *out_mb[]) > > +{ > > + struct entity_priv *en_priv; > > + > > + if (pdcp_entity == NULL) > > + return -EINVAL; > > + > > + en_priv = entity_priv_get(pdcp_entity); > > + > > + if (en_priv->flags.is_ul_entity) { > > + en_priv->state.tx_next = 0; > > + } else { > > + en_priv->state.rx_next = 0; > > + en_priv->state.rx_deliv = 0; > > + } > > + > > + RTE_SET_USED(out_mb); > > + > > + return 0; > > +} > > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode > > 100644 index 0000000000..33c355b05a > > --- /dev/null > > +++ b/lib/pdcp/rte_pdcp.h > > @@ -0,0 +1,157 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#ifndef RTE_PDCP_H > > +#define RTE_PDCP_H > > + > > +/** > > + * @file rte_pdcp.h > > + * > > + * RTE PDCP support. > > + * > > + * librte_pdcp provides a framework for PDCP protocol processing. > > A framework for PDCP protocol processing. [Anoob] Will do next version. > > > + */ > > + > > +#include <rte_compat.h> > > +#include <rte_common.h> > > +#include <rte_mempool.h> > > +#include <rte_security.h> > > + > > +#ifdef __cplusplus > > +extern "C" { > > +#endif > > + > > +/** > > + * PDCP entity. > > You can probably explain more on what a PDCP entity is. [Anoob] Will do in next version. > > > + */ > > +struct rte_pdcp_entity { > > + /** > > + * PDCP entities may hold packets for purposes of in-order delivery > (in > > + * case of receiving PDCP entity) and re-transmission (in case of > > + * transmitting PDCP entity). > > + * > > + * For receiving PDCP entity, it may hold packets when in-order > > + * delivery is enabled. The packets would be cached until either a > > + * packet that completes the sequence arrives or when t-Reordering > > timer > > + * expires. > > + * > > + * When post-processing of PDCP packet which completes a > sequence is > > + * done, the API may return more packets than enqueued. > Application is > > + * expected to provide *rte_pdcp_pkt_post_process()* with > *out_mb* > > + * which can hold maximum number of packets which may be > returned. > > + */ > > The above comment explains the need for holding the packets. > But it does not talk about the parameter it is explaining. > This explanation should be part of programmer's guide and not the API > guide. [Anoob] Agreed. Will make the required changes. > > > + uint32_t max_pkt_cache; > > + /** User area for saving application data. */ > > + uint64_t user_area[2]; > > Is it being used right now in the patches? > If not, can we add it later? > And if really needed now, can we rename to user_data [Anoob] Agreed. Will remove. > > > +} __rte_cache_aligned; > > + > > +/** > > + * PDCP entity configuration to be used for establishing an entity. > > + */ > > +/* Structure rte_pdcp_entity_conf 8< */ struct rte_pdcp_entity_conf { > > + /** PDCP transform for the entity. */ > > + struct rte_security_pdcp_xform pdcp_xfrm; > > + /** Crypto transform applicable for the entity. */ > > + struct rte_crypto_sym_xform *crypto_xfrm; > > + /** Mempool for crypto symmetric session. */ > > + struct rte_mempool *sess_mpool; > > + /** Crypto op pool.*/ > > + struct rte_mempool *cop_pool; > > + /** > > + * 32 bit count value (HFN + SN) to be used for the first packet. > > + * pdcp_xfrm.hfn would be ignored as the HFN would be derived > from > > this value. > > + */ > > If the HFN is to be ignored, then why to add a check in entity establish and > return EINVAL? > It should be silently ignored in that case with a debug print at max. [Anoob] Explained above. Please check. > > > > + uint32_t count; > > + /** Indicate whether the PDCP entity belongs to Side Link Radio > Bearer. > > */ > > + bool is_slrb; > > + /** Enable security offload on the device specified. */ > > + bool en_sec_offload; > > + /** Device on which security/crypto session need to be created. */ > > + uint8_t dev_id; > > + /** Reverse direction during IV generation. Can be used to simulate > > +UE > > crypto processing.*/ > > + bool reverse_iv_direction; > > +}; > > +/* >8 End of structure rte_pdcp_entity_conf. */ > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * 5.1.1 PDCP entity establishment > > + * > > + * Establish PDCP entity based on provided input configuration. > > + * > > + * @param conf > > + * Parameters to be used for initializing PDCP entity object. > > + * @return > > + * - Valid handle if success > > + * - NULL in case of failure. rte_errno will be set to error code > > + */ > > +__rte_experimental > > +struct rte_pdcp_entity * > > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * 5.1.3 PDCP entity release > > + * > > + * Release PDCP entity. > > + * > > + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be > dropped. > > + * For DL/receiving PDCP entity, the stored PDCP SDUs would be > > +returned in > > + * *out_mb* buffer. The buffer should be large enough to hold all > > +cached > > + * packets in the entity. > > + * > > + * @param pdcp_entity > > + * Pointer to the PDCP entity to be released. > > + * @param[out] out_mb > > + * The address of an array that can hold up to > > *rte_pdcp_entity.max_pkt_cache* > > + * pointers to *rte_mbuf* structures. > > + * @return > > + * - 0: Success and no cached packets to return > > + * - >0: Success and the number of packets returned in out_mb > > + * - <0: Error code in case of failures > > + */ > > +__rte_experimental > > +int > > +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, > > + struct rte_mbuf *out_mb[]); > > + > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * 5.1.4 PDCP entity suspend > > + * > > + * Suspend PDCP entity. > > + * > > + * For DL/receiving PDCP entity, the stored PDCP SDUs would be > > +returned in > > + * *out_mb* buffer. The buffer should be large enough to hold all > > +cached > > + * packets in the entity. > > + * > > + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. > > + * > > + * @param pdcp_entity > > + * Pointer to the PDCP entity to be suspended. > > + * @param[out] out_mb > > + * The address of an array that can hold up to > > *rte_pdcp_entity.max_pkt_cache* > > + * pointers to *rte_mbuf* structures. > > + * @return > > + * - 0: Success and no cached packets to return > > + * - >0: Success and the number of packets returned in out_mb > > + * - <0: Error code in case of failures > > + */ > > +__rte_experimental > > +int > > +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > > + struct rte_mbuf *out_mb[]); > > + > > +#ifdef __cplusplus > > +} > > +#endif > > + > > +#endif /* RTE_PDCP_H */ > > diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode > > 100644 index 0000000000..923e165f3f > > --- /dev/null > > +++ b/lib/pdcp/version.map > > @@ -0,0 +1,10 @@ > > +EXPERIMENTAL { > > + global: > > + > > + # added in 23.07 > > + rte_pdcp_entity_establish; > > + rte_pdcp_entity_release; > > + rte_pdcp_entity_suspend; > > + > > + local: *; > > +}; > > -- > > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 02/22] lib: add pdcp protocol 2023-05-18 6:53 ` Anoob Joseph @ 2023-05-18 7:40 ` Akhil Goyal 2023-05-18 8:32 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 7:40 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Anoob, > > > +static int > > > +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct > > > rte_pdcp_entity_conf *conf) > > > +{ > > > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > > > + int ret; > > > + > > > + /** > > > + * flags.is_authenticated > > > + * > > > + * MAC-I would be added in case of control plane packets and when > > > authentication > > > + * transform is not NULL. > > > + */ > > > + > > > + if (conf->pdcp_xfrm.domain == > > > RTE_SECURITY_PDCP_MODE_CONTROL) > > > + en_priv->flags.is_authenticated = 1; > > > > This check should be added after getting the xfrm. > > If domain == control and a_xfrm is NULL, then it should be error, right? > > [Anoob] Lib PDCP would handle such cases. Even if a_xfrm is non NULL but is > NULL auth, it is lib PDCP which would add zeroized digest. And a_xfrm == NULL > is also treated as NULL auth generally. The comment above this explains the > same. Idea is to have lib PDCP handle all possible cases rather than putting too > much restrictions on both app & PMD. I believe there is a case in PDCP which allows no integrity as well.(LTE Uplane) In that case, a_xfrm is NULL is not equivalent to NULL auth. No integrity would mean no MAC-I. > > > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode > > > 100644 index 0000000000..8914548dbd > > > --- /dev/null > > > +++ b/lib/pdcp/rte_pdcp.c > > > @@ -0,0 +1,138 @@ > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > + * Copyright(C) 2023 Marvell. > > > + */ > > > + > > > +#include <rte_errno.h> > > > +#include <rte_pdcp.h> > > > +#include <rte_malloc.h> > > > + > > > +#include "pdcp_crypto.h" > > > +#include "pdcp_entity.h" > > > +#include "pdcp_process.h" > > > + > > > +static int > > > +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { > > > + int size; > > > + > > > + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); > > > + > > > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > > > + size += sizeof(struct entity_priv_dl_part); > > > + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > > > + size += sizeof(struct entity_priv_ul_part); > > > + else > > > + return -EINVAL; > > > + > > > + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } > > > + > > > +struct rte_pdcp_entity * > > > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { > > > + struct rte_pdcp_entity *entity = NULL; > > > + struct entity_priv *en_priv; > > > + int ret, entity_size; > > > + > > > + if (conf == NULL || conf->cop_pool == NULL) { > > > + rte_errno = -EINVAL; > > > + return NULL; > > > + } > > > > errnos are normally set as positive values. > > [Anoob] I do not think so. I checked rte_ethdev.h, rte_flow.h etc and all APIs are > returning negative values in case of errors. Check again lib/ethdev/rte_ethdev.c rte_errno are set as positive values only and APIs return error numbers as negative values. > > > > > > > > + > > > + if (conf->pdcp_xfrm.en_ordering || conf- > > > >pdcp_xfrm.remove_duplicates || conf->is_slrb || > > > + conf->en_sec_offload) { > > > + rte_errno = -ENOTSUP; > > > + return NULL; > > > + } > > > + > > > + /* > > > + * 6.3.2 PDCP SN > > > + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length > > > +of the > > > PDCP SN is > > > + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or > > > +sl- > > > PDCP-SN-Size in > > > + * TS 38.331 [3]) > > > + */ > > > + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) > > && > > > + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { > > > + rte_errno = -ENOTSUP; > > > + return NULL; > > > + } > > > > Check for PDCP crypto algos may also be added. > > As only 4 cipher and 4 auth algos are supported in case of PDCP. > > [Anoob] Validation happens when we create session. Please check, > pdcp: add crypto session create and destroy Agreed, already acked that. > > > > > > + > > > + if (conf->pdcp_xfrm.hfn || conf->pdcp_xfrm.hfn_threshold) { > > > + rte_errno = -EINVAL; > > > + return NULL; > > > + } > > > > What is the reason to set errno as EINVAL when HFN is set? > > [Anoob] HFN is part of pdcp_xfrm which is defined in rte_security. Lib PDCP > allows user to specify complete 32 bit count value using > rte_pdcp_entity_conf.count. Since HFN is also used to construct 32 bit count > value, having two ways to set count would be misleading. Hence lib PDCP would > enforce that application does not set this value. > > > > > > +/** > > > + * PDCP entity configuration to be used for establishing an entity. > > > + */ > > > +/* Structure rte_pdcp_entity_conf 8< */ struct rte_pdcp_entity_conf { > > > + /** PDCP transform for the entity. */ > > > + struct rte_security_pdcp_xform pdcp_xfrm; > > > + /** Crypto transform applicable for the entity. */ > > > + struct rte_crypto_sym_xform *crypto_xfrm; > > > + /** Mempool for crypto symmetric session. */ > > > + struct rte_mempool *sess_mpool; > > > + /** Crypto op pool.*/ > > > + struct rte_mempool *cop_pool; > > > + /** > > > + * 32 bit count value (HFN + SN) to be used for the first packet. > > > + * pdcp_xfrm.hfn would be ignored as the HFN would be derived > > from > > > this value. > > > + */ > > > > If the HFN is to be ignored, then why to add a check in entity establish and > > return EINVAL? > > It should be silently ignored in that case with a debug print at max. > > [Anoob] Explained above. Please check. > No it is not explained. Above comments says " pdcp_xfrm.hfn would be ignored as the HFN would be derived from this value" Here you are saying the field will be ignored. But while creating entity, it is returning EINVAL if hfn is provided. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 02/22] lib: add pdcp protocol 2023-05-18 7:40 ` Akhil Goyal @ 2023-05-18 8:32 ` Anoob Joseph 2023-05-18 8:46 ` Akhil Goyal 0 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-05-18 8:32 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Thursday, May 18, 2023 1:10 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 02/22] lib: add pdcp protocol > > Hi Anoob, > > > > +static int > > > > +pdcp_entity_priv_populate(struct entity_priv *en_priv, const > > > > +struct > > > > rte_pdcp_entity_conf *conf) > > > > +{ > > > > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > > > > + int ret; > > > > + > > > > + /** > > > > + * flags.is_authenticated > > > > + * > > > > + * MAC-I would be added in case of control plane packets and > > > > +when > > > > authentication > > > > + * transform is not NULL. > > > > + */ > > > > + > > > > + if (conf->pdcp_xfrm.domain == > > > > RTE_SECURITY_PDCP_MODE_CONTROL) > > > > + en_priv->flags.is_authenticated = 1; > > > > > > This check should be added after getting the xfrm. > > > If domain == control and a_xfrm is NULL, then it should be error, right? > > > > [Anoob] Lib PDCP would handle such cases. Even if a_xfrm is non NULL > > but is NULL auth, it is lib PDCP which would add zeroized digest. And > > a_xfrm == NULL is also treated as NULL auth generally. The comment > > above this explains the same. Idea is to have lib PDCP handle all > > possible cases rather than putting too much restrictions on both app & > PMD. > > I believe there is a case in PDCP which allows no integrity as well.(LTE Uplane) > In that case, a_xfrm is NULL is not equivalent to NULL auth. > No integrity would mean no MAC-I. [Anoob] The comment explains the logic. If it is control plane, then digest is must and that case lib PDCP is trying to handle. Anyway, adding an extra error check and returning error in that case is not a big deal. Will add that in the appropriate place. > > > > > > > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file > > > > mode > > > > 100644 index 0000000000..8914548dbd > > > > --- /dev/null > > > > +++ b/lib/pdcp/rte_pdcp.c > > > > @@ -0,0 +1,138 @@ > > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > > + * Copyright(C) 2023 Marvell. > > > > + */ > > > > + > > > > +#include <rte_errno.h> > > > > +#include <rte_pdcp.h> > > > > +#include <rte_malloc.h> > > > > + > > > > +#include "pdcp_crypto.h" > > > > +#include "pdcp_entity.h" > > > > +#include "pdcp_process.h" > > > > + > > > > +static int > > > > +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { > > > > + int size; > > > > + > > > > + size = sizeof(struct rte_pdcp_entity) + sizeof(struct > > > > +entity_priv); > > > > + > > > > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > > > > + size += sizeof(struct entity_priv_dl_part); > > > > + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > > > > + size += sizeof(struct entity_priv_ul_part); > > > > + else > > > > + return -EINVAL; > > > > + > > > > + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } > > > > + > > > > +struct rte_pdcp_entity * > > > > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { > > > > + struct rte_pdcp_entity *entity = NULL; > > > > + struct entity_priv *en_priv; > > > > + int ret, entity_size; > > > > + > > > > + if (conf == NULL || conf->cop_pool == NULL) { > > > > + rte_errno = -EINVAL; > > > > + return NULL; > > > > + } > > > > > > errnos are normally set as positive values. > > > > [Anoob] I do not think so. I checked rte_ethdev.h, rte_flow.h etc and > > all APIs are returning negative values in case of errors. > > Check again lib/ethdev/rte_ethdev.c > rte_errno are set as positive values only and APIs return error numbers as > negative values. [Anoob] Okay. There are many APIs were this is not done correctly (check rte_flow.h) . For lib PDCP additions, I'll have this handled. Some of these conventions should be documented to avoid confusion. > > > > > > > > > > > > > > + > > > > + if (conf->pdcp_xfrm.en_ordering || conf- > > > > >pdcp_xfrm.remove_duplicates || conf->is_slrb || > > > > + conf->en_sec_offload) { > > > > + rte_errno = -ENOTSUP; > > > > + return NULL; > > > > + } > > > > + > > > > + /* > > > > + * 6.3.2 PDCP SN > > > > + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length > > > > +of the > > > > PDCP SN is > > > > + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or > > > > +sl- > > > > PDCP-SN-Size in > > > > + * TS 38.331 [3]) > > > > + */ > > > > + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) > > > && > > > > + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { > > > > + rte_errno = -ENOTSUP; > > > > + return NULL; > > > > + } > > > > > > Check for PDCP crypto algos may also be added. > > > As only 4 cipher and 4 auth algos are supported in case of PDCP. > > > > [Anoob] Validation happens when we create session. Please check, > > pdcp: add crypto session create and destroy > > Agreed, already acked that. > > > > > > > > > > + > > > > + if (conf->pdcp_xfrm.hfn || conf->pdcp_xfrm.hfn_threshold) { > > > > + rte_errno = -EINVAL; > > > > + return NULL; > > > > + } > > > > > > What is the reason to set errno as EINVAL when HFN is set? > > > > [Anoob] HFN is part of pdcp_xfrm which is defined in rte_security. Lib PDCP > > allows user to specify complete 32 bit count value using > > rte_pdcp_entity_conf.count. Since HFN is also used to construct 32 bit > count > > value, having two ways to set count would be misleading. Hence lib PDCP > would > > enforce that application does not set this value. > > > > > > > > > +/** > > > > + * PDCP entity configuration to be used for establishing an entity. > > > > + */ > > > > +/* Structure rte_pdcp_entity_conf 8< */ struct rte_pdcp_entity_conf > { > > > > + /** PDCP transform for the entity. */ > > > > + struct rte_security_pdcp_xform pdcp_xfrm; > > > > + /** Crypto transform applicable for the entity. */ > > > > + struct rte_crypto_sym_xform *crypto_xfrm; > > > > + /** Mempool for crypto symmetric session. */ > > > > + struct rte_mempool *sess_mpool; > > > > + /** Crypto op pool.*/ > > > > + struct rte_mempool *cop_pool; > > > > + /** > > > > + * 32 bit count value (HFN + SN) to be used for the first packet. > > > > + * pdcp_xfrm.hfn would be ignored as the HFN would be derived > > > from > > > > this value. > > > > + */ > > > > > > If the HFN is to be ignored, then why to add a check in entity establish > and > > > return EINVAL? > > > It should be silently ignored in that case with a debug print at max. > > > > [Anoob] Explained above. Please check. > > > No it is not explained. Above comments says > " pdcp_xfrm.hfn would be ignored as the HFN would be derived from this > value" > > Here you are saying the field will be ignored. > But while creating entity, it is returning EINVAL if hfn is provided. [Anoob] Agreed. Will address in next version. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 02/22] lib: add pdcp protocol 2023-05-18 8:32 ` Anoob Joseph @ 2023-05-18 8:46 ` Akhil Goyal 2023-05-22 7:03 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 8:46 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > > > > > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file > > > > > mode > > > > > 100644 index 0000000000..8914548dbd > > > > > --- /dev/null > > > > > +++ b/lib/pdcp/rte_pdcp.c > > > > > @@ -0,0 +1,138 @@ > > > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > > > + * Copyright(C) 2023 Marvell. > > > > > + */ > > > > > + > > > > > +#include <rte_errno.h> > > > > > +#include <rte_pdcp.h> > > > > > +#include <rte_malloc.h> > > > > > + > > > > > +#include "pdcp_crypto.h" > > > > > +#include "pdcp_entity.h" > > > > > +#include "pdcp_process.h" > > > > > + > > > > > +static int > > > > > +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { > > > > > + int size; > > > > > + > > > > > + size = sizeof(struct rte_pdcp_entity) + sizeof(struct > > > > > +entity_priv); > > > > > + > > > > > + if (conf->pdcp_xfrm.pkt_dir == > RTE_SECURITY_PDCP_DOWNLINK) > > > > > + size += sizeof(struct entity_priv_dl_part); > > > > > + else if (conf->pdcp_xfrm.pkt_dir == > RTE_SECURITY_PDCP_UPLINK) > > > > > + size += sizeof(struct entity_priv_ul_part); > > > > > + else > > > > > + return -EINVAL; > > > > > + > > > > > + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } > > > > > + > > > > > +struct rte_pdcp_entity * > > > > > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { > > > > > + struct rte_pdcp_entity *entity = NULL; > > > > > + struct entity_priv *en_priv; > > > > > + int ret, entity_size; > > > > > + > > > > > + if (conf == NULL || conf->cop_pool == NULL) { > > > > > + rte_errno = -EINVAL; > > > > > + return NULL; > > > > > + } > > > > > > > > errnos are normally set as positive values. > > > > > > [Anoob] I do not think so. I checked rte_ethdev.h, rte_flow.h etc and > > > all APIs are returning negative values in case of errors. > > > > Check again lib/ethdev/rte_ethdev.c > > rte_errno are set as positive values only and APIs return error numbers as > > negative values. > > [Anoob] Okay. There are many APIs were this is not done correctly (check > rte_flow.h) . For lib PDCP additions, I'll have this handled. Some of these > conventions should be documented to avoid confusion. I am not sure what you are referring to. I cannot find any discrepancy in rte_flow.c as well. Can you give an example where it is wrong. We can ask specific people to fix that. Regarding documentation. It is mentioned in rte_errno.h /** * Error number value, stored per-thread, which can be queried after * calls to certain functions to determine why those functions failed. * * Uses standard values from errno.h wherever possible, with a small number * of additional possible values for RTE-specific conditions. */ #define rte_errno RTE_PER_LCORE(_rte_errno) And errno.h has all positive values defined. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 02/22] lib: add pdcp protocol 2023-05-18 8:46 ` Akhil Goyal @ 2023-05-22 7:03 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-22 7:03 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Thursday, May 18, 2023 2:16 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 02/22] lib: add pdcp protocol > > > > > > > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new > > > > > > file mode > > > > > > 100644 index 0000000000..8914548dbd > > > > > > --- /dev/null > > > > > > +++ b/lib/pdcp/rte_pdcp.c > > > > > > @@ -0,0 +1,138 @@ > > > > > > +/* SPDX-License-Identifier: BSD-3-Clause > > > > > > + * Copyright(C) 2023 Marvell. > > > > > > + */ > > > > > > + > > > > > > +#include <rte_errno.h> > > > > > > +#include <rte_pdcp.h> > > > > > > +#include <rte_malloc.h> > > > > > > + > > > > > > +#include "pdcp_crypto.h" > > > > > > +#include "pdcp_entity.h" > > > > > > +#include "pdcp_process.h" > > > > > > + > > > > > > +static int > > > > > > +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { > > > > > > + int size; > > > > > > + > > > > > > + size = sizeof(struct rte_pdcp_entity) + sizeof(struct > > > > > > +entity_priv); > > > > > > + > > > > > > + if (conf->pdcp_xfrm.pkt_dir == > > RTE_SECURITY_PDCP_DOWNLINK) > > > > > > + size += sizeof(struct entity_priv_dl_part); > > > > > > + else if (conf->pdcp_xfrm.pkt_dir == > > RTE_SECURITY_PDCP_UPLINK) > > > > > > + size += sizeof(struct entity_priv_ul_part); > > > > > > + else > > > > > > + return -EINVAL; > > > > > > + > > > > > > + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } > > > > > > + > > > > > > +struct rte_pdcp_entity * > > > > > > +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf > *conf) { > > > > > > + struct rte_pdcp_entity *entity = NULL; > > > > > > + struct entity_priv *en_priv; > > > > > > + int ret, entity_size; > > > > > > + > > > > > > + if (conf == NULL || conf->cop_pool == NULL) { > > > > > > + rte_errno = -EINVAL; > > > > > > + return NULL; > > > > > > + } > > > > > > > > > > errnos are normally set as positive values. > > > > > > > > [Anoob] I do not think so. I checked rte_ethdev.h, rte_flow.h etc > > > > and all APIs are returning negative values in case of errors. > > > > > > Check again lib/ethdev/rte_ethdev.c > > > rte_errno are set as positive values only and APIs return error > > > numbers as negative values. > > > > [Anoob] Okay. There are many APIs were this is not done correctly > > (check > > rte_flow.h) . For lib PDCP additions, I'll have this handled. Some of > > these conventions should be documented to avoid confusion. > > I am not sure what you are referring to. > I cannot find any discrepancy in rte_flow.c as well. > > Can you give an example where it is wrong. We can ask specific people to fix > that. > > Regarding documentation. > It is mentioned in rte_errno.h > /** > * Error number value, stored per-thread, which can be queried after > * calls to certain functions to determine why those functions failed. > * > * Uses standard values from errno.h wherever possible, with a small > number > * of additional possible values for RTE-specific conditions. > */ > #define rte_errno RTE_PER_LCORE(_rte_errno) > > And errno.h has all positive values defined. [Anoob] Agreed. There are descriptions in rte_flow.h which got us confused. Thanks for the clarification. Changes in PDCP will come in next version. ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 03/22] pdcp: add pre and post-process 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph 2023-04-14 17:44 ` [PATCH v2 01/22] net: add PDCP header Anoob Joseph 2023-04-14 17:44 ` [PATCH v2 02/22] lib: add pdcp protocol Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-16 15:43 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 04/22] pdcp: add packet group Anoob Joseph ` (19 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Functionality of pre-process & post-process varies based on the type of entity. Registration of entity specific function pointer allows skipping multiple checks that would come in datapath otherwise. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/rte_pdcp.h | 97 ++++++++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 ++ 2 files changed, 100 insertions(+) diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 33c355b05a..75dc569f66 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -22,10 +22,29 @@ extern "C" { #endif +/* Forward declarations */ +struct rte_pdcp_entity; + +/* PDCP pre-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err); + +/* PDCP post-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err); + /** * PDCP entity. */ struct rte_pdcp_entity { + /** Entity specific pre-process handle. */ + rte_pdcp_pre_p_t pre_process; + /** Entity specific post-process handle. */ + rte_pdcp_post_p_t post_process; /** * PDCP entities may hold packets for purposes of in-order delivery (in * case of receiving PDCP entity) and re-transmission (in case of @@ -150,6 +169,84 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity pre-process the mbufs and prepare + * crypto ops that can be enqueued to the cryptodev associated with given + * session. Only error packets would be moved returned in the input buffer, + * *mb*, and it is the responsibility of the application to free the same. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param[in, out] mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * which contain the input packets. Any error packets would be returned in the + * same buffer. + * @param[out] cop + * The address of an array that can hold up to *num* pointers to + * *rte_crypto_op* structures. Crypto ops would be allocated by + * ``rte_pdcp_pkt_pre_process`` API. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * Pointer to return the number of error packets returned in *mb* + * @return + * Count of crypto_ops prepared + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err) +{ + return entity->pre_process(entity, mb, cop, num, nb_err); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity, perform PDCP post-processing of the + * mbufs. + * + * Input mbufs are the ones retrieved from crypto_ops dequeued from cryptodev + * and grouped by *rte_pdcp_pkt_crypto_group()*. + * + * The post-processed packets would be returned in the *out_mb* buffer. + * The resultant mbufs would be grouped into success packets and error packets. + * Error packets would be grouped in the end of the array and it is the + * responsibility of the application to handle the same. + * + * When in-order delivery is enabled, PDCP entity may buffer packets and would + * deliver packets only when all prior packets have been post-processed. That + * would result in returning more/less packets than enqueued. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param in_mb + * The address of an array of *num* pointers to *rte_mbuf* structures. + * @param[out] out_mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * to output packets after PDCP post-processing. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * The number of error packets returned in *out_mb* buffer. + * @return + * Count of packets returned in *out_mb* buffer. + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return entity->post_process(entity, in_mb, out_mb, num, nb_err); +} + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 923e165f3f..f9ff30600a 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -6,5 +6,8 @@ EXPERIMENTAL { rte_pdcp_entity_release; rte_pdcp_entity_suspend; + rte_pdcp_pkt_post_process; + rte_pdcp_pkt_pre_process; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 03/22] pdcp: add pre and post-process 2023-04-14 17:44 ` [PATCH v2 03/22] pdcp: add pre and post-process Anoob Joseph @ 2023-05-16 15:43 ` Akhil Goyal 0 siblings, 0 replies; 192+ messages in thread From: Akhil Goyal @ 2023-05-16 15:43 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > Subject: [PATCH v2 03/22] pdcp: add pre and post-process > > PDCP process is split into 2 parts. One before crypto processing > (rte_pdcp_pkt_pre_process()) and one after crypto processing > (rte_pdcp_pkt_post_process()). Functionality of pre-process & > post-process varies based on the type of entity. Registration of entity > specific function pointer allows skipping multiple checks that would > come in datapath otherwise. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> Haven't compiled it yet. Check for doxygen build issues if any. ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 04/22] pdcp: add packet group 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (2 preceding siblings ...) 2023-04-14 17:44 ` [PATCH v2 03/22] pdcp: add pre and post-process Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-16 15:56 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 05/22] pdcp: add crypto session create and destroy Anoob Joseph ` (18 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Crypto processing in PDCP is performed asynchronously by rte_cryptodev_enqueue_burst() and rte_cryptodev_dequeue_burst(). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 1 + lib/pdcp/rte_pdcp.h | 2 + lib/pdcp/rte_pdcp_group.h | 125 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 128 insertions(+) create mode 100644 lib/pdcp/rte_pdcp_group.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index ccaf426240..08679b743a 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -13,5 +13,6 @@ sources = files( 'rte_pdcp.c', ) headers = files('rte_pdcp.h') +indirect_headers += files('rte_pdcp_group.h') deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 75dc569f66..54f88e3fd3 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -247,6 +247,8 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +#include <rte_pdcp_group.h> + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new file mode 100644 index 0000000000..cb322f55c7 --- /dev/null +++ b/lib/pdcp/rte_pdcp_group.h @@ -0,0 +1,125 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_GROUP_H +#define RTE_PDCP_GROUP_H + +/** + * @file rte_pdcp_group.h + * + * RTE PDCP grouping support. + * It is not recommended to include this file directly, include <rte_pdcp.h> + * instead. + * Provides helper functions to process completed crypto-ops and group related + * packets by sessions they belong to. + */ + +#include <rte_common.h> +#include <rte_crypto.h> +#include <rte_cryptodev.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Group packets belonging to same PDCP entity. + */ +struct rte_pdcp_group { + union { + uint64_t val; + void *ptr; + } id; /**< Grouped by value */ + struct rte_mbuf **m; /**< Start of the group */ + uint32_t cnt; /**< Number of entries in the group */ + int32_t rc; /**< Status code associated with the group */ +}; + +/** + * Take crypto-op as an input and extract pointer to related PDCP entity. + * @param cop + * The address of an input *rte_crypto_op* structure. + * @return + * The pointer to the related *rte_pdcp_entity* structure. + */ +static inline struct rte_pdcp_entity * +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) +{ + void *sess = cop->sym[0].session; + + return (struct rte_pdcp_entity *) + rte_cryptodev_sym_session_opaque_data_get(sess); +} + +/** + * Take as input completed crypto ops, extract related mbufs and group them by + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* flag + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. + * + * Note that application must ensure only crypto-ops prepared by lib_pdcp is + * provided back to @see rte_pdcp_pkt_crypto_group(). + * + * @param cop + * The address of an array of *num* pointers to the input *rte_crypto_op* + * structures. + * @param[out] mb + * The address of an array of *num* pointers to output *rte_mbuf* structures. + * @param[out] grp + * The address of an array of *num* to output *rte_pdcp_group* structures. + * @param num + * The maximum number of crypto-ops to process. + * @return + * Number of filled elements in *grp* array. + * + */ +static inline uint16_t +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_pdcp_group grp[], uint16_t num) +{ + uint32_t i, j = 0, n = 0; + void *ns, *ps = NULL; + struct rte_mbuf *m; + + for (i = 0; i != num; i++) { + m = cop[i]->sym[0].m_src; + ns = cop[i]->sym[0].session; + + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; + + /* Different entity */ + if (ps != ns) { + + /* Finalize open group and start a new one */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + /* Start new group */ + grp[n].m = mb + j; + ps = ns; + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); + } + + mb[j++] = m; + rte_crypto_op_free(cop[i]); + } + + /* Finalize last group */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + return n; +} + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_GROUP_H */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 04/22] pdcp: add packet group 2023-04-14 17:44 ` [PATCH v2 04/22] pdcp: add packet group Anoob Joseph @ 2023-05-16 15:56 ` Akhil Goyal 2023-05-18 8:12 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-16 15:56 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > Subject: [PATCH v2 04/22] pdcp: add packet group > > Crypto processing in PDCP is performed asynchronously by > rte_cryptodev_enqueue_burst() and rte_cryptodev_dequeue_burst(). Since > cryptodev dequeue can return crypto operations belonging to multiple > entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto > operations belonging to same entity. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > --- > lib/pdcp/meson.build | 1 + > lib/pdcp/rte_pdcp.h | 2 + > lib/pdcp/rte_pdcp_group.h | 125 > ++++++++++++++++++++++++++++++++++++++ > 3 files changed, 128 insertions(+) > create mode 100644 lib/pdcp/rte_pdcp_group.h > > diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build > index ccaf426240..08679b743a 100644 > --- a/lib/pdcp/meson.build > +++ b/lib/pdcp/meson.build > @@ -13,5 +13,6 @@ sources = files( > 'rte_pdcp.c', > ) > headers = files('rte_pdcp.h') > +indirect_headers += files('rte_pdcp_group.h') > > deps += ['mbuf', 'net', 'cryptodev', 'security'] > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h > index 75dc569f66..54f88e3fd3 100644 > --- a/lib/pdcp/rte_pdcp.h > +++ b/lib/pdcp/rte_pdcp.h > @@ -247,6 +247,8 @@ rte_pdcp_pkt_post_process(const struct > rte_pdcp_entity *entity, > return entity->post_process(entity, in_mb, out_mb, num, nb_err); > } > > +#include <rte_pdcp_group.h> > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h > new file mode 100644 > index 0000000000..cb322f55c7 > --- /dev/null > +++ b/lib/pdcp/rte_pdcp_group.h > @@ -0,0 +1,125 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#ifndef RTE_PDCP_GROUP_H > +#define RTE_PDCP_GROUP_H > + > +/** > + * @file rte_pdcp_group.h > + * > + * RTE PDCP grouping support. > + * It is not recommended to include this file directly, include <rte_pdcp.h> > + * instead. > + * Provides helper functions to process completed crypto-ops and group > related > + * packets by sessions they belong to. > + */ Why do we need to have a separate public header file which we do not wish user to use directly for just a single API? Can it not be added into rte_pdcp.h? > + > +#include <rte_common.h> > +#include <rte_crypto.h> > +#include <rte_cryptodev.h> > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +/** > + * Group packets belonging to same PDCP entity. > + */ > +struct rte_pdcp_group { > + union { > + uint64_t val; > + void *ptr; > + } id; /**< Grouped by value */ > + struct rte_mbuf **m; /**< Start of the group */ > + uint32_t cnt; /**< Number of entries in the group */ > + int32_t rc; /**< Status code associated with the group */ > +}; > + > +/** > + * Take crypto-op as an input and extract pointer to related PDCP entity. > + * @param cop > + * The address of an input *rte_crypto_op* structure. > + * @return > + * The pointer to the related *rte_pdcp_entity* structure. > + */ > +static inline struct rte_pdcp_entity * > +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) > +{ > + void *sess = cop->sym[0].session; > + > + return (struct rte_pdcp_entity *) > + rte_cryptodev_sym_session_opaque_data_get(sess); > +} > + > +/** > + * Take as input completed crypto ops, extract related mbufs and group them > by > + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has > + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* > flag > + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. > + * > + * Note that application must ensure only crypto-ops prepared by lib_pdcp is > + * provided back to @see rte_pdcp_pkt_crypto_group(). > + * > + * @param cop > + * The address of an array of *num* pointers to the input *rte_crypto_op* > + * structures. > + * @param[out] mb > + * The address of an array of *num* pointers to output *rte_mbuf* structures. > + * @param[out] grp > + * The address of an array of *num* to output *rte_pdcp_group* structures. > + * @param num > + * The maximum number of crypto-ops to process. > + * @return > + * Number of filled elements in *grp* array. > + * > + */ > +static inline uint16_t > +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf > *mb[], > + struct rte_pdcp_group grp[], uint16_t num) > +{ > + uint32_t i, j = 0, n = 0; > + void *ns, *ps = NULL; > + struct rte_mbuf *m; > + > + for (i = 0; i != num; i++) { > + m = cop[i]->sym[0].m_src; > + ns = cop[i]->sym[0].session; > + > + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; > + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) > + m->ol_flags |= > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; > + > + /* Different entity */ > + if (ps != ns) { > + > + /* Finalize open group and start a new one */ > + if (ps != NULL) { > + grp[n].cnt = mb + j - grp[n].m; > + n++; > + } > + > + /* Start new group */ > + grp[n].m = mb + j; > + ps = ns; > + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); > + } > + > + mb[j++] = m; > + rte_crypto_op_free(cop[i]); > + } > + > + /* Finalize last group */ > + if (ps != NULL) { > + grp[n].cnt = mb + j - grp[n].m; > + n++; > + } > + > + return n; > +} These APIs are being called from application directly (as per the cover letter). Should be marked as experimental and also add them in version.map > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif /* RTE_PDCP_GROUP_H */ > -- > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 04/22] pdcp: add packet group 2023-05-16 15:56 ` Akhil Goyal @ 2023-05-18 8:12 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-18 8:12 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Konstantin, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Tuesday, May 16, 2023 9:27 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 04/22] pdcp: add packet group > > > Subject: [PATCH v2 04/22] pdcp: add packet group > > > > Crypto processing in PDCP is performed asynchronously by > > rte_cryptodev_enqueue_burst() and rte_cryptodev_dequeue_burst(). > Since > > cryptodev dequeue can return crypto operations belonging to multiple > > entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto > > operations belonging to same entity. > > > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > > --- > > lib/pdcp/meson.build | 1 + > > lib/pdcp/rte_pdcp.h | 2 + > > lib/pdcp/rte_pdcp_group.h | 125 > > ++++++++++++++++++++++++++++++++++++++ > > 3 files changed, 128 insertions(+) > > create mode 100644 lib/pdcp/rte_pdcp_group.h > > > > diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index > > ccaf426240..08679b743a 100644 > > --- a/lib/pdcp/meson.build > > +++ b/lib/pdcp/meson.build > > @@ -13,5 +13,6 @@ sources = files( > > 'rte_pdcp.c', > > ) > > headers = files('rte_pdcp.h') > > +indirect_headers += files('rte_pdcp_group.h') > > > > deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git > > a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index > > 75dc569f66..54f88e3fd3 100644 > > --- a/lib/pdcp/rte_pdcp.h > > +++ b/lib/pdcp/rte_pdcp.h > > @@ -247,6 +247,8 @@ rte_pdcp_pkt_post_process(const struct > > rte_pdcp_entity *entity, > > return entity->post_process(entity, in_mb, out_mb, num, nb_err); } > > > > +#include <rte_pdcp_group.h> > > + > > #ifdef __cplusplus > > } > > #endif > > diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new > > file mode 100644 index 0000000000..cb322f55c7 > > --- /dev/null > > +++ b/lib/pdcp/rte_pdcp_group.h > > @@ -0,0 +1,125 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#ifndef RTE_PDCP_GROUP_H > > +#define RTE_PDCP_GROUP_H > > + > > +/** > > + * @file rte_pdcp_group.h > > + * > > + * RTE PDCP grouping support. > > + * It is not recommended to include this file directly, include > > +<rte_pdcp.h> > > + * instead. > > + * Provides helper functions to process completed crypto-ops and > > +group > > related > > + * packets by sessions they belong to. > > + */ > > Why do we need to have a separate public header file which we do not wish > user to use directly for just a single API? > > Can it not be added into rte_pdcp.h? [Anoob] This follows how the code organization is done in lib IPsec. My understanding is, it is done for 2 reasons, 1. Separate out helper inline functions into a separate header 2. Main header focus on core design of the protocol interface, ie, structs, macros & APIs. Also, lib_pdcp.h is bound to grow. So rather than splitting it later, why don't we start with better organized code layout which is already followed in similar use case, ie, lib IPsec. @Konstantin, did you have any other thoughts when you made the split this way? > > > + > > +#include <rte_common.h> > > +#include <rte_crypto.h> > > +#include <rte_cryptodev.h> > > + > > +#ifdef __cplusplus > > +extern "C" { > > +#endif > > + > > +/** > > + * Group packets belonging to same PDCP entity. > > + */ > > +struct rte_pdcp_group { > > + union { > > + uint64_t val; > > + void *ptr; > > + } id; /**< Grouped by value */ > > + struct rte_mbuf **m; /**< Start of the group */ > > + uint32_t cnt; /**< Number of entries in the group */ > > + int32_t rc; /**< Status code associated with the group */ > > +}; > > + > > +/** > > + * Take crypto-op as an input and extract pointer to related PDCP entity. > > + * @param cop > > + * The address of an input *rte_crypto_op* structure. > > + * @return > > + * The pointer to the related *rte_pdcp_entity* structure. > > + */ > > +static inline struct rte_pdcp_entity * rte_pdcp_en_from_cop(const > > +struct rte_crypto_op *cop) { > > + void *sess = cop->sym[0].session; > > + > > + return (struct rte_pdcp_entity *) > > + rte_cryptodev_sym_session_opaque_data_get(sess); > > +} > > + > > +/** > > + * Take as input completed crypto ops, extract related mbufs and > > +group them > > by > > + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto > > + operation has > > + * failed would be flagged using > *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* > > flag > > + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the > grouping. > > + * > > + * Note that application must ensure only crypto-ops prepared by > > +lib_pdcp is > > + * provided back to @see rte_pdcp_pkt_crypto_group(). > > + * > > + * @param cop > > + * The address of an array of *num* pointers to the input > *rte_crypto_op* > > + * structures. > > + * @param[out] mb > > + * The address of an array of *num* pointers to output *rte_mbuf* > structures. > > + * @param[out] grp > > + * The address of an array of *num* to output *rte_pdcp_group* > structures. > > + * @param num > > + * The maximum number of crypto-ops to process. > > + * @return > > + * Number of filled elements in *grp* array. > > + * > > + */ > > +static inline uint16_t > > +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct > > +rte_mbuf > > *mb[], > > + struct rte_pdcp_group grp[], uint16_t num) { > > + uint32_t i, j = 0, n = 0; > > + void *ns, *ps = NULL; > > + struct rte_mbuf *m; > > + > > + for (i = 0; i != num; i++) { > > + m = cop[i]->sym[0].m_src; > > + ns = cop[i]->sym[0].session; > > + > > + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; > > + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) > > + m->ol_flags |= > > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; > > + > > + /* Different entity */ > > + if (ps != ns) { > > + > > + /* Finalize open group and start a new one */ > > + if (ps != NULL) { > > + grp[n].cnt = mb + j - grp[n].m; > > + n++; > > + } > > + > > + /* Start new group */ > > + grp[n].m = mb + j; > > + ps = ns; > > + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); > > + } > > + > > + mb[j++] = m; > > + rte_crypto_op_free(cop[i]); > > + } > > + > > + /* Finalize last group */ > > + if (ps != NULL) { > > + grp[n].cnt = mb + j - grp[n].m; > > + n++; > > + } > > + > > + return n; > > +} > > These APIs are being called from application directly (as per the cover letter). > Should be marked as experimental and also add them in version.map [Anoob] Agreed. Will do. > > > > + > > +#ifdef __cplusplus > > +} > > +#endif > > + > > +#endif /* RTE_PDCP_GROUP_H */ > > -- > > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 05/22] pdcp: add crypto session create and destroy 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (3 preceding siblings ...) 2023-04-14 17:44 ` [PATCH v2 04/22] pdcp: add packet group Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-16 16:21 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 06/22] pdcp: add pre and post process for UL Anoob Joseph ` (17 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add routines to create & destroy sessions. PDCP lib would take crypto transforms as input and creates the session on the corresponding device after verifying capabilities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_crypto.c | 222 ++++++++++++++++++++++++++++++++++++++++- lib/pdcp/pdcp_crypto.h | 6 ++ 2 files changed, 225 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c index 755e27ec9e..0e6e7d99d8 100644 --- a/lib/pdcp/pdcp_crypto.c +++ b/lib/pdcp/pdcp_crypto.c @@ -2,20 +2,236 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_errno.h> #include <rte_pdcp.h> #include "pdcp_crypto.h" +#include "pdcp_entity.h" + +static int +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, + const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm, + bool is_auth_then_cipher) +{ + uint16_t ciph_iv_len, auth_digest_len, auth_iv_len; + int ret; + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protection is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (c_xfrm == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + + /* With UPLINK, if auth is enabled, it should be before cipher */ + if (a_xfrm != NULL && !is_auth_then_cipher) + return -EINVAL; + + /* With UPLINK, cipher operation must be encrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return -EINVAL; + + /* With UPLINK, auth operation (if present) must be generate */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_GENERATE) + return -EINVAL; + + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + + /* With DOWNLINK, if auth is enabled, it should be after cipher */ + if (a_xfrm != NULL && is_auth_then_cipher) + return -EINVAL; + + /* With DOWNLINK, cipher operation must be decrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) + return -EINVAL; + + /* With DOWNLINK, auth operation (if present) must be verify */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_VERIFY) + return -EINVAL; + + } else { + return -EINVAL; + } + + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) + return -EINVAL; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + ciph_iv_len = 0; + else + ciph_iv_len = PDCP_IV_LEN; + + if (ciph_iv_len != c_xfrm->cipher.iv.length) + return -EINVAL; + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + return -EINVAL; + + /* For AUTH NULL, lib PDCP would add 4 byte 0s */ + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + auth_digest_len = 0; + else + auth_digest_len = PDCP_MAC_I_LEN; + + if (auth_digest_len != a_xfrm->auth.digest_length) + return -EINVAL; + + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + auth_iv_len = PDCP_IV_LEN; + else + auth_iv_len = 0; + + if (a_xfrm->auth.iv.length != auth_iv_len) + return -EINVAL; + } + + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) + return -EINVAL; + + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); + if (ret) + return -ENOTSUP; + + if (a_xfrm != NULL) { + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); + if (ret) + return -ENOTSUP; + } + + return 0; +} int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { - RTE_SET_USED(entity); - RTE_SET_USED(conf); + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + struct entity_priv *en_priv; + bool is_auth_then_cipher; + int ret; + + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + en_priv->dev_id = conf->dev_id; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + c_xfrm = conf->crypto_xfrm; + a_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = false; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + a_xfrm = conf->crypto_xfrm; + c_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = true; + } else { + return -EINVAL; + } + + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, is_auth_then_cipher); + if (ret) + return ret; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + c_xfrm->cipher.iv.offset = 0; + else + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; + + if (a_xfrm != NULL) { + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm->auth.iv.offset = 0; + else + if (c_xfrm->cipher.iv.offset) + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + PDCP_IV_LEN; + else + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; + } + + if (conf->sess_mpool == NULL) + return -EINVAL; + + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf->dev_id, conf->crypto_xfrm, + conf->sess_mpool); + if (en_priv->crypto_sess == NULL) { + /* API returns positive values as error codes */ + return -rte_errno; + } + + rte_cryptodev_sym_session_opaque_data_set(en_priv->crypto_sess, (uint64_t)entity); + return 0; } void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) { - RTE_SET_USED(entity); + struct entity_priv *en_priv; + + en_priv = entity_priv_get(entity); + + if (en_priv->crypto_sess != NULL) { + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv->crypto_sess); + en_priv->crypto_sess = NULL; + } } diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h index 6563331d37..e178ddd725 100644 --- a/lib/pdcp/pdcp_crypto.h +++ b/lib/pdcp/pdcp_crypto.h @@ -5,8 +5,14 @@ #ifndef PDCP_CRYPTO_H #define PDCP_CRYPTO_H +#include <rte_crypto.h> +#include <rte_crypto_sym.h> #include <rte_pdcp.h> +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op)) +#define PDCP_IV_LEN 16 +#define PDCP_MAC_I_LEN 4 + int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 05/22] pdcp: add crypto session create and destroy 2023-04-14 17:44 ` [PATCH v2 05/22] pdcp: add crypto session create and destroy Anoob Joseph @ 2023-05-16 16:21 ` Akhil Goyal 0 siblings, 0 replies; 192+ messages in thread From: Akhil Goyal @ 2023-05-16 16:21 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > Subject: [PATCH v2 05/22] pdcp: add crypto session create and destroy > > Add routines to create & destroy sessions. PDCP lib would take > crypto transforms as input and creates the session on the corresponding > device after verifying capabilities. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 06/22] pdcp: add pre and post process for UL 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (4 preceding siblings ...) 2023-04-14 17:44 ` [PATCH v2 05/22] pdcp: add crypto session create and destroy Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-18 6:38 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 07/22] pdcp: add pre and post process for DL Anoob Joseph ` (16 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add routines to perform pre & post processing based on the type of entity. To avoid checks in datapath, there are different function pointers registered based on the following, 1. Control plane v/s user plane 2. 12 bit v/s 18 bit SN For control plane only 12 bit SN need to be supported (as per PDCP specification). Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 42 +++++ lib/pdcp/pdcp_process.c | 330 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 372 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index ca1d56b516..46cdaead09 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -92,4 +92,46 @@ pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static inline uint32_t +pdcp_window_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return 1 << (sn_size - 1); +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + +static inline uint32_t +pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) +{ + return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); +} + +static inline uint32_t +pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << (32 - sn_size)) - 1; +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index d4b158536d..7c1fc85fcb 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -36,6 +36,332 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static inline void +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, + uint8_t data_offset, uint32_t count, const bool is_auth) +{ + const struct rte_crypto_op cop_init = { + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, + }; + struct rte_crypto_sym_op *op; + uint32_t pkt_len; + + const uint8_t ciph_shift = 3 * en_priv->flags.is_ciph_in_bits; + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; + + op = cop->sym; + cop->raw = cop_init.raw; + op->m_src = mb; + op->m_dst = mb; + + /* Set IV */ + en_priv->iv_gen(cop, en_priv, count); + + /* Prepare op */ + pkt_len = rte_pktmbuf_pkt_len(mb); + op->cipher.data.offset = data_offset << ciph_shift; + op->cipher.data.length = (pkt_len - data_offset) << ciph_shift; + + if (is_auth) { + op->auth.data.offset = 0; + op->auth.data.length = (pkt_len - PDCP_MAC_I_LEN) << auth_shift; + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t *, + (pkt_len - PDCP_MAC_I_LEN)); + } + + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); +} + +static inline bool +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + return true; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline bool +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + return true; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count, sn; + uint8_t *mac_i; + int i; + + const uint8_t hdr_sz = en_priv->hdr_sz; + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + mac_i = (uint8_t *)rte_pktmbuf_append(mb, PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, PDCP_MAC_I_LEN); + + /* Update sequence number in the PDU header */ + count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static int +pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + entity->pre_process = NULL; + entity->post_process = NULL; + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + entity->post_process = pdcp_post_process_ul; + } + + if (entity->pre_process == NULL || entity->post_process == NULL) + return -ENOTSUP; + + return 0; +} + static int pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) { @@ -134,5 +460,9 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti if (ret) return ret; + ret = pdcp_pre_post_func_set(entity, conf); + if (ret) + return ret; + return 0; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 06/22] pdcp: add pre and post process for UL 2023-04-14 17:44 ` [PATCH v2 06/22] pdcp: add pre and post process for UL Anoob Joseph @ 2023-05-18 6:38 ` Akhil Goyal 0 siblings, 0 replies; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 6:38 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > Subject: [PATCH v2 06/22] pdcp: add pre and post process for UL > > Add routines to perform pre & post processing based on the type of > entity. To avoid checks in datapath, there are different function > pointers registered based on the following, > 1. Control plane v/s user plane > 2. 12 bit v/s 18 bit SN > > For control plane only 12 bit SN need to be supported (as per PDCP > specification). > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > --- Acked-by: Akhil Goyal <gakhil@marvell.com> ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 07/22] pdcp: add pre and post process for DL 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (5 preceding siblings ...) 2023-04-14 17:44 ` [PATCH v2 06/22] pdcp: add pre and post process for UL Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-18 6:47 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 08/22] pdcp: add IV generation routines Anoob Joseph ` (15 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add routines to perform pre & post processing for down link entities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 453 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 455 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 46cdaead09..d2d9bbe149 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -13,6 +13,8 @@ struct entity_priv; +#define PDCP_HFN_MIN 0 + /* IV generation function based on the entity configuration */ typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count); diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 7c1fc85fcb..79d6ca352a 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -329,9 +329,423 @@ pdcp_post_process_ul(const struct rte_pdcp_entity *entity, return nb_success; } +static inline int +pdcp_sn_count_get(const uint32_t rx_deliv, int32_t rsn, uint32_t *count, + const enum rte_security_pdcp_sn_size sn_size) +{ + const uint32_t rx_deliv_sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + const uint32_t window_sz = pdcp_window_size_get(sn_size); + uint32_t rhfn; + + rhfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + + if (rsn < (int32_t)(rx_deliv_sn - window_sz)) { + if (unlikely(rhfn == pdcp_hfn_max(sn_size))) + return -ERANGE; + rhfn += 1; + } else if ((uint32_t)rsn >= (rx_deliv_sn + window_sz)) { + if (unlikely(rhfn == PDCP_HFN_MIN)) + return -ERANGE; + rhfn -= 1; + } + + *count = pdcp_count_from_hfn_sn_get(rhfn, rsn, sn_size); + + return 0; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + else + rte_panic("TODO: Control PDU not handled"); + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + } + + if (unlikely(nb_err)) + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + else + rte_panic("TODO: Control PDU not handled"); + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_18))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline void +pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool trim_mac) +{ + char *p = rte_pktmbuf_adj(mb, hdr_trim_sz); + RTE_VERIFY(p != NULL); + + if (trim_mac) { + int ret = rte_pktmbuf_trim(mb, PDCP_MAC_I_LEN); + RTE_VERIFY(ret == 0); + } +} + +static inline bool +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, + const uint32_t count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + + if (count < en_priv->state.rx_deliv) + return false; + + /* t-Reordering timer is not supported - SDU will be delivered immediately. + * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not + * been delivered to upper layers + */ + en_priv->state.rx_next = count + 1; + + if (count >= en_priv->state.rx_next) + en_priv->state.rx_next = count + 1; + + return true; +} + +static inline uint16_t +pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + int i, nb_success = 0, nb_err = 0, rsn = 0; + const uint32_t aad_sz = en_priv->aad_sz; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *, + aad_sz); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + else + rte_panic("Control PDU should not be received"); + + if (unlikely(pdcp_sn_count_get(en_priv->state.rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static inline uint16_t +pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + int32_t rsn = 0; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *, + aad_sz); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + else + rte_panic("Control PDU should not be received"); + + if (unlikely(pdcp_sn_count_get(en_priv->state.rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_18))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static uint16_t +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *, + aad_sz); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + + if (unlikely(pdcp_sn_count_get(en_priv->state.rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, true); + + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + static int pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { + struct entity_priv *en_priv = entity_priv_get(entity); + entity->pre_process = NULL; entity->post_process = NULL; @@ -342,6 +756,13 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; + entity->post_process = pdcp_post_process_cplane_sn_12_dl; + } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { @@ -356,6 +777,38 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; + entity->post_process = pdcp_post_process_uplane_sn_12_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; + entity->post_process = pdcp_post_process_uplane_sn_12_dl; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; + entity->post_process = pdcp_post_process_uplane_sn_18_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; + entity->post_process = pdcp_post_process_uplane_sn_18_dl; + } + if (entity->pre_process == NULL || entity->post_process == NULL) return -ENOTSUP; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 07/22] pdcp: add pre and post process for DL 2023-04-14 17:44 ` [PATCH v2 07/22] pdcp: add pre and post process for DL Anoob Joseph @ 2023-05-18 6:47 ` Akhil Goyal 2023-05-18 7:33 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 6:47 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > static int > pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf) > { > + struct entity_priv *en_priv = entity_priv_get(entity); > + > entity->pre_process = NULL; > entity->post_process = NULL; > > @@ -342,6 +756,13 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity > *entity, const struct rte_pdcp_ent > entity->post_process = pdcp_post_process_ul; > } > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_CONTROL) && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { > + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; > + entity->post_process = pdcp_post_process_cplane_sn_12_dl; > + } > + > if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { > @@ -356,6 +777,38 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity > *entity, const struct rte_pdcp_ent > entity->post_process = pdcp_post_process_ul; > } > > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && > + (en_priv->flags.is_authenticated)) { is_authenticated is checked only in case of DL. For UL, the check is there in the same function. Can we have it consistent with UL? > + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; > + entity->post_process = > pdcp_post_process_uplane_sn_12_dl_ip; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && > + (!en_priv->flags.is_authenticated)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; > + entity->post_process = pdcp_post_process_uplane_sn_12_dl; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && > + (en_priv->flags.is_authenticated)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; > + entity->post_process = > pdcp_post_process_uplane_sn_18_dl_ip; > + } > + > + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) > && > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && > + (!en_priv->flags.is_authenticated)) { > + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; > + entity->post_process = pdcp_post_process_uplane_sn_18_dl; > + } > + > if (entity->pre_process == NULL || entity->post_process == NULL) > return -ENOTSUP; > > -- > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 07/22] pdcp: add pre and post process for DL 2023-05-18 6:47 ` Akhil Goyal @ 2023-05-18 7:33 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-18 7:33 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz HI Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Thursday, May 18, 2023 12:17 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 07/22] pdcp: add pre and post process for DL > > > static int > > pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct > > rte_pdcp_entity_conf *conf) { > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + > > entity->pre_process = NULL; > > entity->post_process = NULL; > > > > @@ -342,6 +756,13 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity > > *entity, const struct rte_pdcp_ent > > entity->post_process = pdcp_post_process_ul; > > } > > > > + if ((conf->pdcp_xfrm.domain == > > RTE_SECURITY_PDCP_MODE_CONTROL) && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { > > + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; > > + entity->post_process = > pdcp_post_process_cplane_sn_12_dl; > > + } > > + > > if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) && > > (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { > @@ > > -356,6 +777,38 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity > > *entity, const struct rte_pdcp_ent > > entity->post_process = pdcp_post_process_ul; > > } > > > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > && > > + (en_priv->flags.is_authenticated)) { > > is_authenticated is checked only in case of DL. > For UL, the check is there in the same function. > Can we have it consistent with UL? [Anoob] The check here allows registering different function pointers so that datapath checks are avoided. The level of optimization vs level of code complexity is the reason for delta. For DL, most of the code is the same and so we added const flags & allowed different functions to call with flags. For UL, the code had significant changes in both paths. Hence an extra top level check was okay. > > > + entity->pre_process = > pdcp_pre_process_uplane_sn_12_dl_ip; > > + entity->post_process = > > pdcp_post_process_uplane_sn_12_dl_ip; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > && > > + (!en_priv->flags.is_authenticated)) { > > + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; > > + entity->post_process = > pdcp_post_process_uplane_sn_12_dl; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > && > > + (en_priv->flags.is_authenticated)) { > > + entity->pre_process = > pdcp_pre_process_uplane_sn_18_dl_ip; > > + entity->post_process = > > pdcp_post_process_uplane_sn_18_dl_ip; > > + } > > + > > + if ((conf->pdcp_xfrm.domain == > RTE_SECURITY_PDCP_MODE_DATA) > > && > > + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) > && > > + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) > && > > + (!en_priv->flags.is_authenticated)) { > > + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; > > + entity->post_process = > pdcp_post_process_uplane_sn_18_dl; > > + } > > + > > if (entity->pre_process == NULL || entity->post_process == NULL) > > return -ENOTSUP; > > > > -- > > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 08/22] pdcp: add IV generation routines 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (6 preceding siblings ...) 2023-04-14 17:44 ` [PATCH v2 07/22] pdcp: add pre and post process for DL Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-18 6:51 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 09/22] app/test: add lib pdcp tests Anoob Joseph ` (14 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz For PDCP, IV generated has varying formats depending on the ciphering and authentication algorithm used. Add routines to populate IV accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 87 ++++++++++++ lib/pdcp/pdcp_process.c | 284 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 371 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index d2d9bbe149..3108795977 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -26,6 +26,89 @@ struct entity_state { uint32_t rx_reord; }; +union auth_iv_partial { + /* For AES-CMAC, there is no IV, but message gets prepended */ + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + } aes_cmac; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_37_39 : 3; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_65_71 : 7; + uint64_t direction_64 : 1; + uint64_t rsvd_72_111 : 40; + uint64_t rsvd_113_119 : 7; + uint64_t direction_112 : 1; + uint64_t rsvd_120_127 : 8; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t zero_37_39 : 3; + uint64_t zero_40_63 : 24; + + uint64_t direction_64 : 1; + uint64_t rsvd_65_71 : 7; + uint64_t rsvd_72_111 : 40; + uint64_t direction_112 : 1; + uint64_t rsvd_113_119 : 7; + uint64_t rsvd_120_127 : 8; +#endif + } zs; + uint64_t u64[2]; +}; + +union cipher_iv_partial { + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t zero_64_127; + } aes_ctr; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t rsvd_64_127; + } zs; + uint64_t u64[2]; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -35,6 +118,10 @@ struct entity_priv { struct rte_cryptodev_sym_session *crypto_sess; /** Entity specific IV generation function. */ iv_gen_t iv_gen; + /** Pre-prepared auth IV. */ + union auth_iv_partial auth_iv_part; + /** Pre-prepared cipher IV. */ + union cipher_iv_partial cipher_iv_part; /** Entity state variables. */ struct entity_state state; /** Flags. */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 79d6ca352a..9c1a5e0669 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -13,6 +13,181 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +/* Enum of supported algorithms for ciphering */ +enum pdcp_cipher_algo { + PDCP_CIPHER_ALGO_NULL, + PDCP_CIPHER_ALGO_AES, + PDCP_CIPHER_ALGO_ZUC, + PDCP_CIPHER_ALGO_SNOW3G, + PDCP_CIPHER_ALGO_MAX +}; + +/* Enum of supported algorithms for integrity */ +enum pdcp_auth_algo { + PDCP_AUTH_ALGO_NULL, + PDCP_AUTH_ALGO_AES, + PDCP_AUTH_ALGO_ZUC, + PDCP_AUTH_ALGO_SNOW3G, + PDCP_AUTH_ALGO_MAX +}; + +/* IV generation functions based on type of operation (cipher - auth) */ + +static void +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + /* No IV required for NULL cipher + NULL auth */ + RTE_SET_USED(cop); + RTE_SET_USED(en_priv); + RTE_SET_USED(count); +} + +static void +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr; + uint64_t m; + + /* AES-CMAC requires message to be prepended with info on count etc */ + + /* Prepend by 8 bytes to add custom message */ + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, 16); +} + +static void +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64 = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64, 8); + rte_memcpy(iv + 8, &iv_u64, 8); +} + +static void +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + /* Generating cipher IV */ + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + iv += PDCP_IV_LEN; + + /* Generating auth IV */ + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + iv += PDCP_IV_LEN; + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + static int pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, struct rte_crypto_sym_xform **a_xfrm) @@ -36,6 +211,111 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static int +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + enum rte_security_pdcp_direction direction; + enum pdcp_cipher_algo ciph_algo; + enum pdcp_auth_algo auth_algo; + struct entity_priv *en_priv; + int ret; + + en_priv = entity_priv_get(entity); + + direction = conf->pdcp_xfrm.pkt_dir; + if (conf->reverse_iv_direction) + direction = !direction; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (c_xfrm == NULL) + return -EINVAL; + + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); + + switch (c_xfrm->cipher.algo) { + case RTE_CRYPTO_CIPHER_NULL: + ciph_algo = PDCP_CIPHER_ALGO_NULL; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + ciph_algo = PDCP_CIPHER_ALGO_AES; + en_priv->cipher_iv_part.aes_ctr.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.aes_ctr.direction = direction; + break; + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + ciph_algo = PDCP_CIPHER_ALGO_SNOW3G; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + ciph_algo = PDCP_CIPHER_ALGO_ZUC; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + default: + return -ENOTSUP; + } + + if (a_xfrm != NULL) { + switch (a_xfrm->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + auth_algo = PDCP_AUTH_ALGO_NULL; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + auth_algo = PDCP_AUTH_ALGO_AES; + en_priv->auth_iv_part.aes_cmac.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.aes_cmac.direction = direction; + break; + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + auth_algo = PDCP_AUTH_ALGO_SNOW3G; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + case RTE_CRYPTO_AUTH_ZUC_EIA3: + auth_algo = PDCP_AUTH_ALGO_ZUC; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + default: + return -ENOTSUP; + } + } else { + auth_algo = PDCP_AUTH_ALGO_NULL; + } + + static const iv_gen_t iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_null_null, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_null_aes_cmac, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_null_zs, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_null_zs, + + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_aes_ctr_null, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_aes_ctr_aes_cmac, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_aes_ctr_zs, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_aes_ctr_zs, + + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + }; + + en_priv->iv_gen = iv_gen_map[ciph_algo][auth_algo]; + + return 0; +} + static inline void cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, uint8_t data_offset, uint32_t count, const bool is_auth) @@ -909,6 +1189,10 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti en_priv = entity_priv_get(entity); + ret = pdcp_iv_gen_func_set(entity, conf); + if (ret) + return ret; + ret = pdcp_entity_priv_populate(en_priv, conf); if (ret) return ret; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 08/22] pdcp: add IV generation routines 2023-04-14 17:44 ` [PATCH v2 08/22] pdcp: add IV generation routines Anoob Joseph @ 2023-05-18 6:51 ` Akhil Goyal 0 siblings, 0 replies; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 6:51 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > Subject: [PATCH v2 08/22] pdcp: add IV generation routines > > For PDCP, IV generated has varying formats depending on the ciphering and > authentication algorithm used. Add routines to populate IV accordingly. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > --- > lib/pdcp/pdcp_entity.h | 87 ++++++++++++ > lib/pdcp/pdcp_process.c | 284 > ++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 371 insertions(+) > > diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h > index d2d9bbe149..3108795977 100644 > --- a/lib/pdcp/pdcp_entity.h > +++ b/lib/pdcp/pdcp_entity.h > @@ -26,6 +26,89 @@ struct entity_state { > uint32_t rx_reord; > }; > > +union auth_iv_partial { > + /* For AES-CMAC, there is no IV, but message gets prepended */ > + struct { > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > + uint64_t count : 32; > + uint64_t zero_38_39 : 2; > + uint64_t direction : 1; > + uint64_t bearer : 5; > + uint64_t zero_40_63 : 24; > +#else > + uint64_t count : 32; > + uint64_t bearer : 5; > + uint64_t direction : 1; > + uint64_t zero_38_39 : 2; > + uint64_t zero_40_63 : 24; > +#endif > + } aes_cmac; > + struct { > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > + uint64_t count : 32; > + uint64_t zero_37_39 : 3; > + uint64_t bearer : 5; > + uint64_t zero_40_63 : 24; > + > + uint64_t rsvd_65_71 : 7; > + uint64_t direction_64 : 1; > + uint64_t rsvd_72_111 : 40; > + uint64_t rsvd_113_119 : 7; > + uint64_t direction_112 : 1; > + uint64_t rsvd_120_127 : 8; > +#else > + uint64_t count : 32; > + uint64_t bearer : 5; > + uint64_t zero_37_39 : 3; > + uint64_t zero_40_63 : 24; > + > + uint64_t direction_64 : 1; > + uint64_t rsvd_65_71 : 7; > + uint64_t rsvd_72_111 : 40; > + uint64_t direction_112 : 1; > + uint64_t rsvd_113_119 : 7; > + uint64_t rsvd_120_127 : 8; > +#endif > + } zs; > + uint64_t u64[2]; > +}; Can we add pictorial comments for each of the IV formats? So that we may not need to refer the specifications again and again. > + > +union cipher_iv_partial { > + struct { > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > + uint64_t count : 32; > + uint64_t zero_38_39 : 2; > + uint64_t direction : 1; > + uint64_t bearer : 5; > + uint64_t zero_40_63 : 24; > +#else > + uint64_t count : 32; > + uint64_t bearer : 5; > + uint64_t direction : 1; > + uint64_t zero_38_39 : 2; > + uint64_t zero_40_63 : 24; > +#endif > + uint64_t zero_64_127; > + } aes_ctr; > + struct { > +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN > + uint64_t count : 32; > + uint64_t zero_38_39 : 2; > + uint64_t direction : 1; > + uint64_t bearer : 5; > + uint64_t zero_40_63 : 24; > +#else > + uint64_t count : 32; > + uint64_t bearer : 5; > + uint64_t direction : 1; > + uint64_t zero_38_39 : 2; > + uint64_t zero_40_63 : 24; > +#endif > + uint64_t rsvd_64_127; > + } zs; > + uint64_t u64[2]; > +}; > + > /* > * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] > */ > @@ -35,6 +118,10 @@ struct entity_priv { > struct rte_cryptodev_sym_session *crypto_sess; > /** Entity specific IV generation function. */ > iv_gen_t iv_gen; > + /** Pre-prepared auth IV. */ > + union auth_iv_partial auth_iv_part; > + /** Pre-prepared cipher IV. */ > + union cipher_iv_partial cipher_iv_part; > /** Entity state variables. */ > struct entity_state state; > /** Flags. */ > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c > index 79d6ca352a..9c1a5e0669 100644 > --- a/lib/pdcp/pdcp_process.c > +++ b/lib/pdcp/pdcp_process.c > @@ -13,6 +13,181 @@ > #include "pdcp_entity.h" > #include "pdcp_process.h" > > +/* Enum of supported algorithms for ciphering */ > +enum pdcp_cipher_algo { > + PDCP_CIPHER_ALGO_NULL, > + PDCP_CIPHER_ALGO_AES, > + PDCP_CIPHER_ALGO_ZUC, > + PDCP_CIPHER_ALGO_SNOW3G, > + PDCP_CIPHER_ALGO_MAX > +}; > + > +/* Enum of supported algorithms for integrity */ > +enum pdcp_auth_algo { > + PDCP_AUTH_ALGO_NULL, > + PDCP_AUTH_ALGO_AES, > + PDCP_AUTH_ALGO_ZUC, > + PDCP_AUTH_ALGO_SNOW3G, > + PDCP_AUTH_ALGO_MAX > +}; > + > +/* IV generation functions based on type of operation (cipher - auth) */ > + > +static void > +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + /* No IV required for NULL cipher + NULL auth */ > + RTE_SET_USED(cop); > + RTE_SET_USED(en_priv); > + RTE_SET_USED(count); > +} > + > +static void > +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct > entity_priv *en_priv, > + uint32_t count) > +{ > + struct rte_crypto_sym_op *op = cop->sym; > + struct rte_mbuf *mb = op->m_src; > + uint8_t *m_ptr; > + uint64_t m; > + > + /* AES-CMAC requires message to be prepended with info on count etc > */ > + > + /* Prepend by 8 bytes to add custom message */ > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > + > + m = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + > + rte_memcpy(m_ptr, &m, 8); > +} > + > +static void > +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + uint64_t iv_u64[2]; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > + rte_memcpy(iv + 8, &iv_u64[1], 8); > +} > + > +static void > +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, > + uint32_t count) > +{ > + uint64_t iv_u64[2]; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + iv_u64[1] = 0; > + rte_memcpy(iv, iv_u64, 16); > +} > + > +static void > +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + uint64_t iv_u64; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64 = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64, 8); > + rte_memcpy(iv + 8, &iv_u64, 8); > +} > + > +static void > +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + uint64_t iv_u64[2]; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + /* Generating cipher IV */ > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + rte_memcpy(iv + 8, &iv_u64[0], 8); > + > + iv += PDCP_IV_LEN; > + > + /* Generating auth IV */ > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > + rte_memcpy(iv + 8, &iv_u64[1], 8); > +} > + > +static void > +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, > + uint32_t count) > +{ > + struct rte_crypto_sym_op *op = cop->sym; > + struct rte_mbuf *mb = op->m_src; > + uint8_t *m_ptr, *iv; > + uint64_t iv_u64[2]; > + uint64_t m; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + rte_memcpy(iv + 8, &iv_u64[0], 8); > + > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > + m = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(m_ptr, &m, 8); > +} > + > +static void > +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct > entity_priv *en_priv, > + uint32_t count) > +{ > + struct rte_crypto_sym_op *op = cop->sym; > + struct rte_mbuf *mb = op->m_src; > + uint8_t *m_ptr, *iv; > + uint64_t iv_u64[2]; > + uint64_t m; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + iv_u64[1] = 0; > + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); > + > + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); > + m = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(m_ptr, &m, 8); > +} > + > +static void > +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv > *en_priv, uint32_t count) > +{ > + uint64_t iv_u64[2]; > + uint8_t *iv; > + > + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); > + > + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + iv_u64[1] = 0; > + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); > + > + iv += PDCP_IV_LEN; > + > + iv_u64[0] = en_priv->auth_iv_part.u64[0] | > ((uint64_t)(rte_cpu_to_be_32(count))); > + rte_memcpy(iv, &iv_u64[0], 8); > + > + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; > + rte_memcpy(iv + 8, &iv_u64[1], 8); > +} > + > static int > pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct > rte_crypto_sym_xform **c_xfrm, > struct rte_crypto_sym_xform **a_xfrm) > @@ -36,6 +211,111 @@ pdcp_crypto_xfrm_get(const struct > rte_pdcp_entity_conf *conf, struct rte_crypto_ > return 0; > } > > +static int > +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; > + enum rte_security_pdcp_direction direction; > + enum pdcp_cipher_algo ciph_algo; > + enum pdcp_auth_algo auth_algo; > + struct entity_priv *en_priv; > + int ret; > + > + en_priv = entity_priv_get(entity); > + > + direction = conf->pdcp_xfrm.pkt_dir; > + if (conf->reverse_iv_direction) > + direction = !direction; > + > + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); > + if (ret) > + return ret; > + > + if (c_xfrm == NULL) > + return -EINVAL; > + > + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); > + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); > + > + switch (c_xfrm->cipher.algo) { > + case RTE_CRYPTO_CIPHER_NULL: > + ciph_algo = PDCP_CIPHER_ALGO_NULL; > + break; > + case RTE_CRYPTO_CIPHER_AES_CTR: > + ciph_algo = PDCP_CIPHER_ALGO_AES; > + en_priv->cipher_iv_part.aes_ctr.bearer = conf- > >pdcp_xfrm.bearer; > + en_priv->cipher_iv_part.aes_ctr.direction = direction; > + break; > + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: > + ciph_algo = PDCP_CIPHER_ALGO_SNOW3G; > + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; > + en_priv->cipher_iv_part.zs.direction = direction; > + break; > + case RTE_CRYPTO_CIPHER_ZUC_EEA3: > + ciph_algo = PDCP_CIPHER_ALGO_ZUC; > + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; > + en_priv->cipher_iv_part.zs.direction = direction; > + break; > + default: > + return -ENOTSUP; > + } > + > + if (a_xfrm != NULL) { > + switch (a_xfrm->auth.algo) { > + case RTE_CRYPTO_AUTH_NULL: > + auth_algo = PDCP_AUTH_ALGO_NULL; > + break; > + case RTE_CRYPTO_AUTH_AES_CMAC: > + auth_algo = PDCP_AUTH_ALGO_AES; > + en_priv->auth_iv_part.aes_cmac.bearer = conf- > >pdcp_xfrm.bearer; > + en_priv->auth_iv_part.aes_cmac.direction = direction; > + break; > + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: > + auth_algo = PDCP_AUTH_ALGO_SNOW3G; > + en_priv->auth_iv_part.zs.bearer = conf- > >pdcp_xfrm.bearer; > + en_priv->auth_iv_part.zs.direction_64 = direction; > + en_priv->auth_iv_part.zs.direction_112 = direction; > + break; > + case RTE_CRYPTO_AUTH_ZUC_EIA3: > + auth_algo = PDCP_AUTH_ALGO_ZUC; > + en_priv->auth_iv_part.zs.bearer = conf- > >pdcp_xfrm.bearer; > + en_priv->auth_iv_part.zs.direction_64 = direction; > + en_priv->auth_iv_part.zs.direction_112 = direction; > + break; > + default: > + return -ENOTSUP; > + } > + } else { > + auth_algo = PDCP_AUTH_ALGO_NULL; > + } > + > + static const iv_gen_t > iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = > pdcp_iv_gen_null_null, > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = > pdcp_iv_gen_null_aes_cmac, > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = > pdcp_iv_gen_null_zs, > + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = > pdcp_iv_gen_null_zs, > + > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = > pdcp_iv_gen_aes_ctr_null, > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = > pdcp_iv_gen_aes_ctr_aes_cmac, > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = > pdcp_iv_gen_aes_ctr_zs, > + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = > pdcp_iv_gen_aes_ctr_zs, > + > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = > pdcp_iv_gen_zs_null, > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = > pdcp_iv_gen_zs_aes_cmac, > + > [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = > pdcp_iv_gen_zs_zs, > + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = > pdcp_iv_gen_zs_zs, > + > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = > pdcp_iv_gen_zs_null, > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = > pdcp_iv_gen_zs_aes_cmac, > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = > pdcp_iv_gen_zs_zs, > + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = > pdcp_iv_gen_zs_zs, > + }; > + > + en_priv->iv_gen = iv_gen_map[ciph_algo][auth_algo]; > + > + return 0; > +} > + > static inline void > cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct > rte_crypto_op *cop, > uint8_t data_offset, uint32_t count, const bool is_auth) > @@ -909,6 +1189,10 @@ pdcp_process_func_set(struct rte_pdcp_entity > *entity, const struct rte_pdcp_enti > > en_priv = entity_priv_get(entity); > > + ret = pdcp_iv_gen_func_set(entity, conf); > + if (ret) > + return ret; > + > ret = pdcp_entity_priv_populate(en_priv, conf); > if (ret) > return ret; > -- > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 09/22] app/test: add lib pdcp tests 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (7 preceding siblings ...) 2023-04-14 17:44 ` [PATCH v2 08/22] pdcp: add IV generation routines Anoob Joseph @ 2023-04-14 17:44 ` Anoob Joseph 2023-05-18 8:03 ` Akhil Goyal 2023-04-14 17:45 ` [PATCH v2 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph ` (13 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:44 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add tests to verify lib PDCP operations. Tests leverage existing PDCP test vectors. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/meson.build | 1 + app/test/test_cryptodev.h | 3 + app/test/test_pdcp.c | 729 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 733 insertions(+) create mode 100644 app/test/test_pdcp.c diff --git a/app/test/meson.build b/app/test/meson.build index 52d9088578..0f658aa2ab 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -96,6 +96,7 @@ test_sources = files( 'test_meter.c', 'test_mcslock.c', 'test_mp_secondary.c', + 'test_pdcp.c', 'test_per_lcore.c', 'test_pflock.c', 'test_pmd_perf.c', diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h index abd795f54a..89057dba22 100644 --- a/app/test/test_cryptodev.h +++ b/app/test/test_cryptodev.h @@ -4,6 +4,9 @@ #ifndef TEST_CRYPTODEV_H_ #define TEST_CRYPTODEV_H_ +#include <rte_crypto.h> +#include <rte_cryptodev.h> + #define HEX_DUMP 0 #define FALSE 0 diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c new file mode 100644 index 0000000000..cb88817004 --- /dev/null +++ b/app/test/test_pdcp.c @@ -0,0 +1,729 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_malloc.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "test.h" +#include "test_cryptodev.h" +#include "test_cryptodev_security_pdcp_test_vectors.h" + +#define NB_DESC 1024 +#define CDEV_INVALID_ID UINT8_MAX +#define NB_TESTS RTE_DIM(pdcp_test_params) + +struct pdcp_testsuite_params { + struct rte_mempool *mbuf_pool; + struct rte_mempool *cop_pool; + struct rte_mempool *sess_pool; + bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; +}; + +static struct pdcp_testsuite_params testsuite_params; + +struct pdcp_test_conf { + struct rte_pdcp_entity_conf entity; + struct rte_crypto_sym_xform c_xfrm; + struct rte_crypto_sym_xform a_xfrm; + bool is_integrity_protected; + uint8_t input[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t input_len; + uint8_t output[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t output_len; +}; + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static int +cryptodev_init(int dev_id) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_cryptodev_qp_conf qp_conf; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_config config; + int ret, socket_id; + + /* Check if device was already initialized */ + if (ts_params->cdevs_used[dev_id]) + return 0; + + rte_cryptodev_info_get(dev_id, &dev_info); + + if (dev_info.max_nb_queue_pairs < 1) { + RTE_LOG(ERR, USER1, "Cryptodev doesn't have sufficient queue pairs available\n"); + return -ENODEV; + } + + socket_id = rte_socket_id(); + + memset(&config, 0, sizeof(config)); + config.nb_queue_pairs = 1; + config.socket_id = socket_id; + + ret = rte_cryptodev_configure(dev_id, &config); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure cryptodev - %d\n", dev_id); + return -ENODEV; + } + + memset(&qp_conf, 0, sizeof(qp_conf)); + qp_conf.nb_descriptors = NB_DESC; + + ret = rte_cryptodev_queue_pair_setup(dev_id, 0, &qp_conf, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure queue pair\n"); + return -ENODEV; + } + + ret = rte_cryptodev_start(dev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not start cryptodev\n"); + return -ENODEV; + } + + /* Mark device as initialized */ + ts_params->cdevs_used[dev_id] = true; + + return 0; +} + +static void +cryptodev_fini(int dev_id) +{ + rte_cryptodev_stop(dev_id); +} + +static unsigned int +cryptodev_sess_priv_max_req_get(void) +{ + struct rte_cryptodev_info info; + unsigned int sess_priv_sz; + int i, nb_dev; + void *sec_ctx; + + nb_dev = rte_cryptodev_count(); + + sess_priv_sz = 0; + + for (i = 0; i < nb_dev; i++) { + rte_cryptodev_info_get(i, &info); + sess_priv_sz = RTE_MAX(sess_priv_sz, rte_cryptodev_sym_get_private_session_size(i)); + if (info.feature_flags & RTE_CRYPTODEV_FF_SECURITY) { + sec_ctx = rte_cryptodev_get_sec_ctx(i); + sess_priv_sz = RTE_MAX(sess_priv_sz, + rte_security_session_get_size(sec_ctx)); + } + } + + return sess_priv_sz; +} + +static int +testsuite_setup(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + int nb_cdev, sess_priv_size, nb_sess = 1024; + + RTE_SET_USED(pdcp_test_hfn_threshold); + + nb_cdev = rte_cryptodev_count(); + if (nb_cdev < 1) { + RTE_LOG(ERR, USER1, "No crypto devices found.\n"); + return TEST_SKIPPED; + } + + memset(ts_params, 0, sizeof(*ts_params)); + + ts_params->mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NUM_MBUFS, MBUF_CACHE_SIZE, 0, + MBUF_SIZE, SOCKET_ID_ANY); + if (ts_params->mbuf_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf pool\n"); + return TEST_FAILED; + } + + ts_params->cop_pool = rte_crypto_op_pool_create("cop_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, MBUF_CACHE_SIZE, + 2 * MAXIMUM_IV_LENGTH, SOCKET_ID_ANY); + if (ts_params->cop_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create crypto_op pool\n"); + goto mbuf_pool_free; + } + + /* Get max session priv size required */ + sess_priv_size = cryptodev_sess_priv_max_req_get(); + + ts_params->sess_pool = rte_cryptodev_sym_session_pool_create("sess_pool", nb_sess, + sess_priv_size, + RTE_MEMPOOL_CACHE_MAX_SIZE, + 0, SOCKET_ID_ANY); + if (ts_params->sess_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create session pool\n"); + goto cop_pool_free; + } + + return 0; + +cop_pool_free: + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; +mbuf_pool_free: + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; + return TEST_FAILED; +} + +static void +testsuite_teardown(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + uint8_t dev_id; + + for (dev_id = 0; dev_id < RTE_CRYPTO_MAX_DEVS; dev_id++) { + if (ts_params->cdevs_used[dev_id]) + cryptodev_fini(dev_id); + } + + rte_mempool_free(ts_params->sess_pool); + ts_params->sess_pool = NULL; + + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; + + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; +} + +static int +ut_setup_pdcp(void) +{ + return 0; +} + +static void +ut_teardown_pdcp(void) +{ +} + +static int +crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +cryptodev_id_get(bool is_integrity_protected, const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm) +{ + int i, nb_devs; + + nb_devs = rte_cryptodev_count(); + + /* Check capabilities */ + + for (i = 0; i < nb_devs; i++) { + if ((crypto_caps_cipher_verify(i, c_xfrm) == 0) && + (!is_integrity_protected || crypto_caps_auth_verify(i, a_xfrm) == 0)) + break; + } + + if (i == nb_devs) + return -1; + + return i; +} + +static int +pdcp_known_vec_verify(struct rte_mbuf *m, const uint8_t *expected, uint32_t expected_pkt_len) +{ + uint8_t *actual = rte_pktmbuf_mtod(m, uint8_t *); + uint32_t actual_pkt_len = rte_pktmbuf_pkt_len(m); + + debug_hexdump(stdout, "Received:", actual, actual_pkt_len); + debug_hexdump(stdout, "Expected:", expected, expected_pkt_len); + + TEST_ASSERT_EQUAL(actual_pkt_len, expected_pkt_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_pkt_len, actual_pkt_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(actual, expected, expected_pkt_len, + "Generated packet not as expected"); + + return 0; +} + +static struct rte_crypto_op * +process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op) +{ + if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) { + RTE_LOG(ERR, USER1, "Error sending packet to cryptodev\n"); + return NULL; + } + + op = NULL; + + while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0) + rte_pause(); + + return op; +} + +static uint32_t +pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) +{ + uint32_t sn = 0; + + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + sn = rte_cpu_to_be_16(*(const uint16_t *)data); + sn = sn & 0xfff; + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + sn = rte_cpu_to_be_32(*(const uint32_t *)data); + sn = (sn & 0x3ffff00) >> 8; + } + + return sn; +} + +static int +create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_crypto_sym_xform c_xfrm, a_xfrm; + uint32_t hfn, sn, expected_len, count = 0; + uint8_t *data, *expected; + int pdcp_hdr_sz; + + memset(conf, 0, sizeof(*conf)); + memset(&c_xfrm, 0, sizeof(c_xfrm)); + memset(&a_xfrm, 0, sizeof(a_xfrm)); + + conf->entity.sess_mpool = ts_params->sess_pool; + conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; + conf->entity.pdcp_xfrm.en_ordering = 0; + conf->entity.pdcp_xfrm.remove_duplicates = 0; + conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + + if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; + else + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + + conf->entity.pdcp_xfrm.sn_size = pdcp_test_data_sn_size[index]; + + /* Zero initialize unsupported flags */ + conf->entity.pdcp_xfrm.hfn_threshold = 0; + conf->entity.pdcp_xfrm.hfn_ovrd = 0; + conf->entity.pdcp_xfrm.sdap_enabled = 0; + + c_xfrm.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + c_xfrm.cipher.algo = pdcp_test_params[index].cipher_alg; + c_xfrm.cipher.key.length = pdcp_test_params[index].cipher_key_len; + c_xfrm.cipher.key.data = pdcp_test_crypto_key[index]; + + a_xfrm.type = RTE_CRYPTO_SYM_XFORM_AUTH; + + if (pdcp_test_params[index].auth_alg == 0) { + conf->is_integrity_protected = false; + } else { + a_xfrm.auth.algo = pdcp_test_params[index].auth_alg; + a_xfrm.auth.key.data = pdcp_test_auth_key[index]; + a_xfrm.auth.key.length = pdcp_test_params[index].auth_key_len; + conf->is_integrity_protected = true; + } + + pdcp_hdr_sz = pdcp_hdr_size_get(pdcp_test_data_sn_size[index]); + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protecting is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (conf->is_integrity_protected) { + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + conf->entity.crypto_xfrm = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + a_xfrm.next = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + c_xfrm.next = NULL; + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + c_xfrm.next = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + a_xfrm.next = NULL; + } + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + c_xfrm.next = NULL; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + else + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + /* Update xforms to match PDCP requirements */ + + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) + c_xfrm.cipher.iv.length = 16; + else + c_xfrm.cipher.iv.length = 0; + + if (conf->is_integrity_protected) { + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm.auth.digest_length = 0; + else + a_xfrm.auth.digest_length = 4; + + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + a_xfrm.auth.iv.length = 16; + else + a_xfrm.auth.iv.length = 0; + } + + conf->c_xfrm = c_xfrm; + conf->a_xfrm = a_xfrm; + + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf->is_integrity_protected, + &conf->c_xfrm, &conf->a_xfrm); + + if (pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_CONTROL || + pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_DATA) { + data = pdcp_test_data_in[index]; + hfn = pdcp_test_hfn[index] << pdcp_test_data_sn_size[index]; + sn = pdcp_sn_from_raw_get(data, pdcp_test_data_sn_size[index]); + count = hfn | sn; + } + conf->entity.count = count; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", pdcp_test_data_in[index], + pdcp_test_data_in_len[index]); +#endif + /* Since the vectors available already have PDCP header, trim the same */ + conf->input_len = pdcp_test_data_in_len[index] - pdcp_hdr_sz; + memcpy(conf->input, pdcp_test_data_in[index] + pdcp_hdr_sz, conf->input_len); + } else { + conf->input_len = pdcp_test_data_in_len[index]; + + if (conf->is_integrity_protected) + conf->input_len += 4; + + memcpy(conf->input, pdcp_test_data_out[index], conf->input_len); +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", conf->input, conf->input_len); +#endif + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected = pdcp_test_data_out[index]; + else + expected = pdcp_test_data_in[index]; + + /* Calculate expected packet length */ + expected_len = pdcp_test_data_in_len[index]; + + /* In DL processing, PDCP header would be stripped */ + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + expected += pdcp_hdr_sz; + expected_len -= pdcp_hdr_sz; + } + + /* In UL processing with integrity protection, MAC would be added */ + if (conf->is_integrity_protected && + conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected_len += 4; + + memcpy(conf->output, expected, expected_len); + conf->output_len = expected_len; + + return 0; +} + +static struct rte_pdcp_entity* +test_entity_create(const struct pdcp_test_conf *t_conf, int *rc) +{ + struct rte_pdcp_entity *pdcp_entity; + int ret; + + if (t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12 && + t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18) { + *rc = -ENOTSUP; + return NULL; + } + + if (t_conf->entity.dev_id == CDEV_INVALID_ID) { + RTE_LOG(DEBUG, USER1, "Could not find device with required capabilities\n"); + *rc = -ENOTSUP; + return NULL; + } + + ret = cryptodev_init(t_conf->entity.dev_id); + if (ret) { + *rc = ret; + RTE_LOG(DEBUG, USER1, "Could not initialize cryptodev\n"); + return NULL; + } + + rte_errno = 0; + + pdcp_entity = rte_pdcp_entity_establish(&t_conf->entity); + if (pdcp_entity == NULL) { + *rc = rte_errno; + RTE_LOG(DEBUG, USER1, "Could not establish PDCP entity\n"); + return NULL; + } + + return pdcp_entity; +} + +static uint16_t +test_process_packets(const struct rte_pdcp_entity *pdcp_entity, uint8_t cdev_id, + struct rte_mbuf *in_mb[], uint16_t nb_in, + struct rte_mbuf *out_mb[], uint16_t *nb_err) +{ + struct rte_crypto_op *cop, *cop_out; + struct rte_pdcp_group grp[1]; + uint16_t nb_success, nb_grp; + struct rte_mbuf *mbuf, *mb; + + if (nb_in != 1) + return -ENOTSUP; + + mbuf = in_mb[0]; + + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, &mbuf, &cop_out, 1, nb_err); + if (nb_success != 1 || *nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not pre process PDCP packet\n"); + return TEST_FAILED; + } + +#ifdef VEC_DUMP + printf("Pre-processed vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, rte_pktmbuf_pkt_len(mbuf)); +#endif + + cop = process_crypto_request(cdev_id, cop_out); + if (cop == NULL) { + RTE_LOG(ERR, USER1, "Could not process crypto request\n"); + return -EIO; + } + + nb_grp = rte_pdcp_pkt_crypto_group(&cop_out, &mb, grp, 1); + if (nb_grp != 1 || grp[0].cnt != 1) { + RTE_LOG(ERR, USER1, "Could not group PDCP crypto results\n"); + return -ENOTRECOVERABLE; + } + + if ((uintptr_t)pdcp_entity != grp[0].id.val) { + RTE_LOG(ERR, USER1, "PDCP entity not matching the one from crypto_op\n"); + return -ENOTRECOVERABLE; + } + +#ifdef VEC_DUMP + printf("Crypto processed vector:\n"); + rte_pktmbuf_dump(stdout, cop->sym->m_dst, rte_pktmbuf_pkt_len(mbuf)); +#endif + + return rte_pdcp_pkt_post_process(grp[0].id.ptr, grp[0].m, out_mb, grp[0].cnt, nb_err); +} + +static struct rte_mbuf* +mbuf_from_data_create(uint8_t *data, uint16_t data_len) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_mbuf *mbuf; + uint8_t *input_text; + + mbuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (mbuf == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf\n"); + return NULL; + } + + memset(rte_pktmbuf_mtod(mbuf, uint8_t *), 0, rte_pktmbuf_tailroom(mbuf)); + + input_text = (uint8_t *)rte_pktmbuf_append(mbuf, data_len); + memcpy(input_text, data, data_len); + + return mbuf; +} + +static int +test_attempt_single(struct pdcp_test_conf *t_conf) +{ + struct rte_mbuf *mbuf, **out_mb = NULL; + struct rte_pdcp_entity *pdcp_entity; + uint16_t nb_success, nb_err; + int ret = 0, nb_max_out_mb; + + pdcp_entity = test_entity_create(t_conf, &ret); + if (pdcp_entity == NULL) + goto exit; + + /* Allocate buffer for holding mbufs returned */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate buffer for holding out_mb buffers\n"); + ret = -ENOMEM; + goto entity_release; + } + + mbuf = mbuf_from_data_create(t_conf->input, t_conf->input_len); + if (mbuf == NULL) { + ret = -ENOMEM; + goto entity_release; + } + +#ifdef VEC_DUMP + printf("Adjusted vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, t_conf->input_len); +#endif + + nb_success = test_process_packets(pdcp_entity, t_conf->entity.dev_id, &mbuf, 1, out_mb, + &nb_err); + if (nb_success != 1 || nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not process PDCP packet\n"); + ret = TEST_FAILED; + goto mbuf_free; + } + + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + + ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); + if (ret) { + RTE_LOG(DEBUG, USER1, "Could not suspend PDCP entity\n"); + goto mbuf_free; + } + +mbuf_free: + rte_pktmbuf_free(mbuf); +entity_release: + rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_free(out_mb); +exit: + return ret; +} + +static int +run_test_for_one_known_vec(const void *arg) +{ + struct pdcp_test_conf test_conf; + int i = *(const uint32_t *)arg; + + create_test_conf_from_index(i, &test_conf); + return test_attempt_single(&test_conf); +} + +struct unit_test_suite *test_suites[] = { + NULL, /* Place holder for known_vector_cases */ + NULL /* End of suites list */ +}; + +static struct unit_test_suite pdcp_testsuite = { + .suite_name = "PDCP Unit Test Suite", + .unit_test_cases = {TEST_CASES_END()}, + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_suites = test_suites, +}; + +static int +test_pdcp(void) +{ + struct unit_test_suite *known_vector_cases; + int ret, index[NB_TESTS]; + uint32_t i, size; + + size = sizeof(struct unit_test_suite); + size += (NB_TESTS + 1) * sizeof(struct unit_test_case); + + known_vector_cases = rte_zmalloc(NULL, size, 0); + if (known_vector_cases == NULL) + return TEST_FAILED; + + known_vector_cases->suite_name = "Known vector cases"; + + for (i = 0; i < NB_TESTS; i++) { + index[i] = i; + known_vector_cases->unit_test_cases[i].name = pdcp_test_params[i].name; + known_vector_cases->unit_test_cases[i].data = (void *)&index[i]; + known_vector_cases->unit_test_cases[i].enabled = 1; + known_vector_cases->unit_test_cases[i].setup = ut_setup_pdcp; + known_vector_cases->unit_test_cases[i].teardown = ut_teardown_pdcp; + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data + = run_test_for_one_known_vec; + } + + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data = NULL; + + test_suites[0] = known_vector_cases; + + ret = unit_test_suite_runner(&pdcp_testsuite); + + rte_free(known_vector_cases); + return ret; +} + +REGISTER_TEST_COMMAND(pdcp_autotest, test_pdcp); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 09/22] app/test: add lib pdcp tests 2023-04-14 17:44 ` [PATCH v2 09/22] app/test: add lib pdcp tests Anoob Joseph @ 2023-05-18 8:03 ` Akhil Goyal 2023-05-18 11:31 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 8:03 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > diff --git a/app/test/meson.build b/app/test/meson.build > index 52d9088578..0f658aa2ab 100644 > --- a/app/test/meson.build > +++ b/app/test/meson.build > @@ -96,6 +96,7 @@ test_sources = files( > 'test_meter.c', > 'test_mcslock.c', > 'test_mp_secondary.c', > + 'test_pdcp.c', > 'test_per_lcore.c', > 'test_pflock.c', > 'test_pmd_perf.c', > diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h > index abd795f54a..89057dba22 100644 > --- a/app/test/test_cryptodev.h > +++ b/app/test/test_cryptodev.h > @@ -4,6 +4,9 @@ > #ifndef TEST_CRYPTODEV_H_ > #define TEST_CRYPTODEV_H_ > > +#include <rte_crypto.h> > +#include <rte_cryptodev.h> > + Can we remove these includes from here and add in test_pdcp.c directly? > + if (conf->is_integrity_protected) { > + if (conf->entity.pdcp_xfrm.pkt_dir == > RTE_SECURITY_PDCP_UPLINK) { > + conf->entity.crypto_xfrm = &conf->a_xfrm; > + > + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; > + a_xfrm.next = &conf->c_xfrm; > + > + c_xfrm.cipher.op = > RTE_CRYPTO_CIPHER_OP_ENCRYPT; > + c_xfrm.next = NULL; > + } else { > + conf->entity.crypto_xfrm = &conf->c_xfrm; > + > + c_xfrm.cipher.op = > RTE_CRYPTO_CIPHER_OP_DECRYPT; > + c_xfrm.next = &conf->a_xfrm; > + > + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; > + a_xfrm.next = NULL; > + } > + } else { > + conf->entity.crypto_xfrm = &conf->c_xfrm; > + c_xfrm.next = NULL; > + > + if (conf->entity.pdcp_xfrm.pkt_dir == > RTE_SECURITY_PDCP_UPLINK) > + c_xfrm.cipher.op = > RTE_CRYPTO_CIPHER_OP_ENCRYPT; > + else > + c_xfrm.cipher.op = > RTE_CRYPTO_CIPHER_OP_DECRYPT; > + } > + /* Update xforms to match PDCP requirements */ > + > + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || > + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || > + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) > + c_xfrm.cipher.iv.length = 16; > + else > + c_xfrm.cipher.iv.length = 0; > + > + if (conf->is_integrity_protected) { > + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) > + a_xfrm.auth.digest_length = 0; > + else > + a_xfrm.auth.digest_length = 4; This if-else is not needed. If is_integrity_protected, digest_length should always be 4. Also define a macro for MAC-I len. It is being used at multiple places. Similarly for IV length macro can be defined. > + > + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || > + (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) > + a_xfrm.auth.iv.length = 16; > + else > + a_xfrm.auth.iv.length = 0; > + } > + > + conf->c_xfrm = c_xfrm; > + conf->a_xfrm = a_xfrm; > + > + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf- > >is_integrity_protected, > + &conf->c_xfrm, &conf->a_xfrm); > + > + if (pdcp_test_params[index].domain == > RTE_SECURITY_PDCP_MODE_CONTROL || > + pdcp_test_params[index].domain == > RTE_SECURITY_PDCP_MODE_DATA) { > + data = pdcp_test_data_in[index]; > + hfn = pdcp_test_hfn[index] << pdcp_test_data_sn_size[index]; > + sn = pdcp_sn_from_raw_get(data, > pdcp_test_data_sn_size[index]); > + count = hfn | sn; > + } The above logic can go inside lib PDCP as well. This is specific to PDCP and not user dependent. You can reuse the pdcp_xfrm.hfn as well. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 09/22] app/test: add lib pdcp tests 2023-05-18 8:03 ` Akhil Goyal @ 2023-05-18 11:31 ` Anoob Joseph 2023-05-18 12:06 ` Akhil Goyal 0 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-05-18 11:31 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Thursday, May 18, 2023 1:34 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 09/22] app/test: add lib pdcp tests > > > diff --git a/app/test/meson.build b/app/test/meson.build index > > 52d9088578..0f658aa2ab 100644 > > --- a/app/test/meson.build > > +++ b/app/test/meson.build > > @@ -96,6 +96,7 @@ test_sources = files( > > 'test_meter.c', > > 'test_mcslock.c', > > 'test_mp_secondary.c', > > + 'test_pdcp.c', > > 'test_per_lcore.c', > > 'test_pflock.c', > > 'test_pmd_perf.c', > > diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h > > index abd795f54a..89057dba22 100644 > > --- a/app/test/test_cryptodev.h > > +++ b/app/test/test_cryptodev.h > > @@ -4,6 +4,9 @@ > > #ifndef TEST_CRYPTODEV_H_ > > #define TEST_CRYPTODEV_H_ > > > > +#include <rte_crypto.h> > > +#include <rte_cryptodev.h> > > + > > Can we remove these includes from here and add in test_pdcp.c directly? [Anoob] Why? 'test_cryptodev.h' already has many references to rte_cryptodev symbols. Not including the dependencies is not correct. > > > > + if (conf->is_integrity_protected) { > > + if (conf->entity.pdcp_xfrm.pkt_dir == > > RTE_SECURITY_PDCP_UPLINK) { > > + conf->entity.crypto_xfrm = &conf->a_xfrm; > > + > > + a_xfrm.auth.op = > RTE_CRYPTO_AUTH_OP_GENERATE; > > + a_xfrm.next = &conf->c_xfrm; > > + > > + c_xfrm.cipher.op = > > RTE_CRYPTO_CIPHER_OP_ENCRYPT; > > + c_xfrm.next = NULL; > > + } else { > > + conf->entity.crypto_xfrm = &conf->c_xfrm; > > + > > + c_xfrm.cipher.op = > > RTE_CRYPTO_CIPHER_OP_DECRYPT; > > + c_xfrm.next = &conf->a_xfrm; > > + > > + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; > > + a_xfrm.next = NULL; > > + } > > + } else { > > + conf->entity.crypto_xfrm = &conf->c_xfrm; > > + c_xfrm.next = NULL; > > + > > + if (conf->entity.pdcp_xfrm.pkt_dir == > > RTE_SECURITY_PDCP_UPLINK) > > + c_xfrm.cipher.op = > > RTE_CRYPTO_CIPHER_OP_ENCRYPT; > > + else > > + c_xfrm.cipher.op = > > RTE_CRYPTO_CIPHER_OP_DECRYPT; > > + } > > + /* Update xforms to match PDCP requirements */ > > + > > + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || > > + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || > > + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) > > + c_xfrm.cipher.iv.length = 16; > > + else > > + c_xfrm.cipher.iv.length = 0; > > + > > + if (conf->is_integrity_protected) { > > + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) > > + a_xfrm.auth.digest_length = 0; > > + else > > + a_xfrm.auth.digest_length = 4; > > This if-else is not needed. If is_integrity_protected, digest_length should > always be 4. [Anoob] I had explained this in v1 patch set. Will try again. In PDCP, with AUTH_NULL it is expected to have 4 bytes of zeroized digest. With AUTH_NULL, it is lib PDCP which would add zeroized digest. No PMD currently supported in DPDK supports non-zero digest with AUTH-NULL. Also, it is not specified what is the digest added in case of AUTH-NULL. > Also define a macro for MAC-I len. It is being used at multiple places. > Similarly for IV length macro can be defined. > [Anoob] Agreed. You want me to introduce RTE_PDCP_MAC_I_LEN or something local would do? > > + > > + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || > > + (a_xfrm.auth.algo == > RTE_CRYPTO_AUTH_SNOW3G_UIA2)) > > + a_xfrm.auth.iv.length = 16; > > + else > > + a_xfrm.auth.iv.length = 0; > > + } > > + > > + conf->c_xfrm = c_xfrm; > > + conf->a_xfrm = a_xfrm; > > + > > + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf- > > >is_integrity_protected, > > + &conf->c_xfrm, &conf->a_xfrm); > > + > > + if (pdcp_test_params[index].domain == > > RTE_SECURITY_PDCP_MODE_CONTROL || > > + pdcp_test_params[index].domain == > > RTE_SECURITY_PDCP_MODE_DATA) { > > + data = pdcp_test_data_in[index]; > > + hfn = pdcp_test_hfn[index] << > pdcp_test_data_sn_size[index]; > > + sn = pdcp_sn_from_raw_get(data, > > pdcp_test_data_sn_size[index]); > > + count = hfn | sn; > > + } > > The above logic can go inside lib PDCP as well. > This is specific to PDCP and not user dependent. > You can reuse the pdcp_xfrm.hfn as well. > [Anoob] Sorry, what exactly can go into lib PDCP? This snippet is reading SN used in a test vector and constructs the count based on SN & HFN value from vector. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 09/22] app/test: add lib pdcp tests 2023-05-18 11:31 ` Anoob Joseph @ 2023-05-18 12:06 ` Akhil Goyal 2023-05-19 10:31 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 12:06 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > > > diff --git a/app/test/meson.build b/app/test/meson.build index > > > 52d9088578..0f658aa2ab 100644 > > > --- a/app/test/meson.build > > > +++ b/app/test/meson.build > > > @@ -96,6 +96,7 @@ test_sources = files( > > > 'test_meter.c', > > > 'test_mcslock.c', > > > 'test_mp_secondary.c', > > > + 'test_pdcp.c', > > > 'test_per_lcore.c', > > > 'test_pflock.c', > > > 'test_pmd_perf.c', > > > diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h > > > index abd795f54a..89057dba22 100644 > > > --- a/app/test/test_cryptodev.h > > > +++ b/app/test/test_cryptodev.h > > > @@ -4,6 +4,9 @@ > > > #ifndef TEST_CRYPTODEV_H_ > > > #define TEST_CRYPTODEV_H_ > > > > > > +#include <rte_crypto.h> > > > +#include <rte_cryptodev.h> > > > + > > > > Can we remove these includes from here and add in test_pdcp.c directly? > > [Anoob] Why? 'test_cryptodev.h' already has many references to rte_cryptodev > symbols. Not including the dependencies is not correct. > In that case, it can be a separate patch. But not in this patch. > > > > > > > + if (conf->is_integrity_protected) { > > > + if (conf->entity.pdcp_xfrm.pkt_dir == > > > RTE_SECURITY_PDCP_UPLINK) { > > > + conf->entity.crypto_xfrm = &conf->a_xfrm; > > > + > > > + a_xfrm.auth.op = > > RTE_CRYPTO_AUTH_OP_GENERATE; > > > + a_xfrm.next = &conf->c_xfrm; > > > + > > > + c_xfrm.cipher.op = > > > RTE_CRYPTO_CIPHER_OP_ENCRYPT; > > > + c_xfrm.next = NULL; > > > + } else { > > > + conf->entity.crypto_xfrm = &conf->c_xfrm; > > > + > > > + c_xfrm.cipher.op = > > > RTE_CRYPTO_CIPHER_OP_DECRYPT; > > > + c_xfrm.next = &conf->a_xfrm; > > > + > > > + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; > > > + a_xfrm.next = NULL; > > > + } > > > + } else { > > > + conf->entity.crypto_xfrm = &conf->c_xfrm; > > > + c_xfrm.next = NULL; > > > + > > > + if (conf->entity.pdcp_xfrm.pkt_dir == > > > RTE_SECURITY_PDCP_UPLINK) > > > + c_xfrm.cipher.op = > > > RTE_CRYPTO_CIPHER_OP_ENCRYPT; > > > + else > > > + c_xfrm.cipher.op = > > > RTE_CRYPTO_CIPHER_OP_DECRYPT; > > > + } > > > + /* Update xforms to match PDCP requirements */ > > > + > > > + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || > > > + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || > > > + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) > > > + c_xfrm.cipher.iv.length = 16; > > > + else > > > + c_xfrm.cipher.iv.length = 0; > > > + > > > + if (conf->is_integrity_protected) { > > > + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) > > > + a_xfrm.auth.digest_length = 0; > > > + else > > > + a_xfrm.auth.digest_length = 4; > > > > This if-else is not needed. If is_integrity_protected, digest_length should > > always be 4. > > [Anoob] I had explained this in v1 patch set. Will try again. > > In PDCP, with AUTH_NULL it is expected to have 4 bytes of zeroized digest. > > With AUTH_NULL, it is lib PDCP which would add zeroized digest. No PMD > currently supported in DPDK supports non-zero digest with AUTH-NULL. Also, it > is not specified what is the digest added in case of AUTH-NULL. In auth_xform, digest_length is the expected length of digest coming out of crypto device. Now if the device is expected to support PDCP, and we are reusing the crypto APIs, We can specify the digest length required for NULL auth. The PMDs capability can be updated accordingly. You can add a comment in the rte_crypto_auth_xform specifically for NULL auth for PDCP case. The reason, I am insisting on this is, for the user, while configuring auth_xform, it is setting digest length as 0 and when the packet is received the packet length is increased by digest. This will create confusion. And it will also help in identifying the case of no integrity and null integrity. > > > Also define a macro for MAC-I len. It is being used at multiple places. > > Similarly for IV length macro can be defined. > > > > [Anoob] Agreed. You want me to introduce RTE_PDCP_MAC_I_LEN or > something local would do? I am ok either way. Having defined in library would be better, to be used in lib and PMD as well. > > > > + > > > + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || > > > + (a_xfrm.auth.algo == > > RTE_CRYPTO_AUTH_SNOW3G_UIA2)) > > > + a_xfrm.auth.iv.length = 16; > > > + else > > > + a_xfrm.auth.iv.length = 0; > > > + } > > > + > > > + conf->c_xfrm = c_xfrm; > > > + conf->a_xfrm = a_xfrm; > > > + > > > + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf- > > > >is_integrity_protected, > > > + &conf->c_xfrm, &conf->a_xfrm); > > > + > > > + if (pdcp_test_params[index].domain == > > > RTE_SECURITY_PDCP_MODE_CONTROL || > > > + pdcp_test_params[index].domain == > > > RTE_SECURITY_PDCP_MODE_DATA) { > > > + data = pdcp_test_data_in[index]; > > > + hfn = pdcp_test_hfn[index] << > > pdcp_test_data_sn_size[index]; > > > + sn = pdcp_sn_from_raw_get(data, > > > pdcp_test_data_sn_size[index]); > > > + count = hfn | sn; > > > + } > > > > The above logic can go inside lib PDCP as well. > > This is specific to PDCP and not user dependent. > > You can reuse the pdcp_xfrm.hfn as well. > > > > [Anoob] Sorry, what exactly can go into lib PDCP? This snippet is reading SN used > in a test vector and constructs the count based on SN & HFN value from vector. This count value is being used to establish entity. I am saying, instead of taking Count, take sn as input and in the libpdcp combine pdcp_xfrm.hfn and sn as needed to create count and store in entity_priv. Just wanted to reduce the application headache to make bitshifting and ORing to SN. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 09/22] app/test: add lib pdcp tests 2023-05-18 12:06 ` Akhil Goyal @ 2023-05-19 10:31 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-19 10:31 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Thursday, May 18, 2023 5:36 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 09/22] app/test: add lib pdcp tests > > > > > diff --git a/app/test/meson.build b/app/test/meson.build index > > > > 52d9088578..0f658aa2ab 100644 > > > > --- a/app/test/meson.build > > > > +++ b/app/test/meson.build > > > > @@ -96,6 +96,7 @@ test_sources = files( > > > > 'test_meter.c', > > > > 'test_mcslock.c', > > > > 'test_mp_secondary.c', > > > > + 'test_pdcp.c', > > > > 'test_per_lcore.c', > > > > 'test_pflock.c', > > > > 'test_pmd_perf.c', > > > > diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h > > > > index abd795f54a..89057dba22 100644 > > > > --- a/app/test/test_cryptodev.h > > > > +++ b/app/test/test_cryptodev.h > > > > @@ -4,6 +4,9 @@ > > > > #ifndef TEST_CRYPTODEV_H_ > > > > #define TEST_CRYPTODEV_H_ > > > > > > > > +#include <rte_crypto.h> > > > > +#include <rte_cryptodev.h> > > > > + > > > > > > Can we remove these includes from here and add in test_pdcp.c directly? > > > > [Anoob] Why? 'test_cryptodev.h' already has many references to > > rte_cryptodev symbols. Not including the dependencies is not correct. > > > In that case, it can be a separate patch. But not in this patch. [Anoob] Is your suggestion to push this specific change as a separate patch? I can do that if you suggest so. > > > > > > > > > > > + if (conf->is_integrity_protected) { > > > > + if (conf->entity.pdcp_xfrm.pkt_dir == > > > > RTE_SECURITY_PDCP_UPLINK) { > > > > + conf->entity.crypto_xfrm = &conf->a_xfrm; > > > > + > > > > + a_xfrm.auth.op = > > > RTE_CRYPTO_AUTH_OP_GENERATE; > > > > + a_xfrm.next = &conf->c_xfrm; > > > > + > > > > + c_xfrm.cipher.op = > > > > RTE_CRYPTO_CIPHER_OP_ENCRYPT; > > > > + c_xfrm.next = NULL; > > > > + } else { > > > > + conf->entity.crypto_xfrm = &conf->c_xfrm; > > > > + > > > > + c_xfrm.cipher.op = > > > > RTE_CRYPTO_CIPHER_OP_DECRYPT; > > > > + c_xfrm.next = &conf->a_xfrm; > > > > + > > > > + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; > > > > + a_xfrm.next = NULL; > > > > + } > > > > + } else { > > > > + conf->entity.crypto_xfrm = &conf->c_xfrm; > > > > + c_xfrm.next = NULL; > > > > + > > > > + if (conf->entity.pdcp_xfrm.pkt_dir == > > > > RTE_SECURITY_PDCP_UPLINK) > > > > + c_xfrm.cipher.op = > > > > RTE_CRYPTO_CIPHER_OP_ENCRYPT; > > > > + else > > > > + c_xfrm.cipher.op = > > > > RTE_CRYPTO_CIPHER_OP_DECRYPT; > > > > + } > > > > + /* Update xforms to match PDCP requirements */ > > > > + > > > > + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || > > > > + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || > > > > + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) > > > > + c_xfrm.cipher.iv.length = 16; > > > > + else > > > > + c_xfrm.cipher.iv.length = 0; > > > > + > > > > + if (conf->is_integrity_protected) { > > > > + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) > > > > + a_xfrm.auth.digest_length = 0; > > > > + else > > > > + a_xfrm.auth.digest_length = 4; > > > > > > This if-else is not needed. If is_integrity_protected, digest_length > > > should always be 4. > > > > [Anoob] I had explained this in v1 patch set. Will try again. > > > > In PDCP, with AUTH_NULL it is expected to have 4 bytes of zeroized digest. > > > > With AUTH_NULL, it is lib PDCP which would add zeroized digest. No PMD > > currently supported in DPDK supports non-zero digest with AUTH-NULL. > > Also, it is not specified what is the digest added in case of AUTH-NULL. > > In auth_xform, digest_length is the expected length of digest coming out of > crypto device. [Anoob] Yes. But the packet append is the duty lib PDCP. Crypto PMD is expected to just write the digest at a specific location. > Now if the device is expected to support PDCP, and we are reusing the > crypto APIs, We can specify the digest length required for NULL auth. > The PMDs capability can be updated accordingly. [Anoob] Again, none of the current PMDs do a zeroized digest with NULL auth. And it is not a requirement as well. Here, we are doing PDCP offload and the crypto_xforms provided here are for specific crypto transformations. > You can add a comment in the rte_crypto_auth_xform specifically for NULL > auth for PDCP case. > > The reason, I am insisting on this is, for the user, while configuring > auth_xform, it is setting digest length as 0 and when the packet is received > the packet length is increased by digest. This will create confusion. > And it will also help in identifying the case of no integrity and null integrity. [Anoob] When working with protocol, the packet would change both at header and trailer. It is just that for PDCP, the trailer is only digest. For IPsec, the change is more than digest. > > > > > > Also define a macro for MAC-I len. It is being used at multiple places. > > > Similarly for IV length macro can be defined. > > > > > > > [Anoob] Agreed. You want me to introduce RTE_PDCP_MAC_I_LEN or > > something local would do? > I am ok either way. Having defined in library would be better, to be used in > lib and PMD as well. > > > > > > > + > > > > + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || > > > > + (a_xfrm.auth.algo == > > > RTE_CRYPTO_AUTH_SNOW3G_UIA2)) > > > > + a_xfrm.auth.iv.length = 16; > > > > + else > > > > + a_xfrm.auth.iv.length = 0; > > > > + } > > > > + > > > > + conf->c_xfrm = c_xfrm; > > > > + conf->a_xfrm = a_xfrm; > > > > + > > > > + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf- > > > > >is_integrity_protected, > > > > + &conf->c_xfrm, &conf->a_xfrm); > > > > + > > > > + if (pdcp_test_params[index].domain == > > > > RTE_SECURITY_PDCP_MODE_CONTROL || > > > > + pdcp_test_params[index].domain == > > > > RTE_SECURITY_PDCP_MODE_DATA) { > > > > + data = pdcp_test_data_in[index]; > > > > + hfn = pdcp_test_hfn[index] << > > > pdcp_test_data_sn_size[index]; > > > > + sn = pdcp_sn_from_raw_get(data, > > > > pdcp_test_data_sn_size[index]); > > > > + count = hfn | sn; > > > > + } > > > > > > The above logic can go inside lib PDCP as well. > > > This is specific to PDCP and not user dependent. > > > You can reuse the pdcp_xfrm.hfn as well. > > > > > > > [Anoob] Sorry, what exactly can go into lib PDCP? This snippet is > > reading SN used in a test vector and constructs the count based on SN & > HFN value from vector. > > This count value is being used to establish entity. I am saying, instead of > taking Count, take sn as input and in the libpdcp combine pdcp_xfrm.hfn and > sn as needed to create count and store in entity_priv. > Just wanted to reduce the application headache to make bitshifting and > ORing to SN. [Anoob] I was not able to identify a use case where having two 32 bit values for HFN & SN would make sense. But then, it would align better with current rte_security specification. Will make the change in next version. ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 10/22] test/pdcp: pdcp HFN tests in combined mode 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (8 preceding siblings ...) 2023-04-14 17:44 ` [PATCH v2 09/22] app/test: add lib pdcp tests Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 11/22] doc: add PDCP library guide Anoob Joseph ` (12 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add tests to verify HFN/SN behaviour. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 302 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 299 insertions(+), 3 deletions(-) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index cb88817004..fc49947ba2 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -15,6 +15,9 @@ #define CDEV_INVALID_ID UINT8_MAX #define NB_TESTS RTE_DIM(pdcp_test_params) +/* According to formula(7.2.a Window_Size) */ +#define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) + struct pdcp_testsuite_params { struct rte_mempool *mbuf_pool; struct rte_mempool *cop_pool; @@ -35,12 +38,69 @@ struct pdcp_test_conf { uint32_t output_len; }; +static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf); + +typedef int (*test_with_conf_t)(struct pdcp_test_conf *conf); + +static int +run_test_foreach_known_vec(test_with_conf_t test, bool stop_on_first_pass) +{ + struct pdcp_test_conf test_conf; + bool all_tests_skipped = true; + uint32_t i; + int ret; + + for (i = 0; i < NB_TESTS; i++) { + create_test_conf_from_index(i, &test_conf); + ret = test(&test_conf); + + if (ret == TEST_FAILED) { + printf("[%03i] - %s - failed\n", i, pdcp_test_params[i].name); + return TEST_FAILED; + } + + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + continue; + + if (stop_on_first_pass) + return TEST_SUCCESS; + + all_tests_skipped = false; + } + + if (all_tests_skipped) + return TEST_SKIPPED; + + return TEST_SUCCESS; +} + +static int +run_test_with_all_known_vec(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, false); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static int +pktmbuf_read_into(const struct rte_mbuf *m, void *buf, size_t buf_len) +{ + if (m->pkt_len > buf_len) + return -ENOMEM; + + const void *read = rte_pktmbuf_read(m, 0, m->pkt_len, buf); + if (read != NULL && read != buf) + memcpy(buf, read, m->pkt_len); + + return 0; +} + static int cryptodev_init(int dev_id) { @@ -325,6 +385,21 @@ pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) return sn; } +static void +pdcp_sn_to_raw_set(void *data, uint32_t sn, int size) +{ + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr = data; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr = data; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } +} + static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) { @@ -645,9 +720,17 @@ test_attempt_single(struct pdcp_test_conf *t_conf) goto mbuf_free; } - ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); - if (ret) - goto mbuf_free; + /* If expected output provided - verify, else - store for future use */ + if (t_conf->output_len) { + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + } else { + ret = pktmbuf_read_into(mbuf, t_conf->output, RTE_PDCP_CTRL_PDU_SIZE_MAX); + if (ret) + goto mbuf_free; + t_conf->output_len = mbuf->pkt_len; + } ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); if (ret) { @@ -664,6 +747,193 @@ test_attempt_single(struct pdcp_test_conf *t_conf) return ret; } +static void +uplink_to_downlink_convert(const struct pdcp_test_conf *ul_cfg, + struct pdcp_test_conf *dl_cfg) +{ + assert(ul_cfg->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK); + + memcpy(dl_cfg, ul_cfg, sizeof(*dl_cfg)); + dl_cfg->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + dl_cfg->entity.reverse_iv_direction = false; + + if (dl_cfg->is_integrity_protected) { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + dl_cfg->c_xfrm.next = &dl_cfg->a_xfrm; + + dl_cfg->a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + dl_cfg->a_xfrm.next = NULL; + } else { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + dl_cfg->c_xfrm.next = NULL; + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + dl_cfg->entity.dev_id = (uint8_t)cryptodev_id_get(dl_cfg->is_integrity_protected, + &dl_cfg->c_xfrm, &dl_cfg->a_xfrm); + + memcpy(dl_cfg->input, ul_cfg->output, ul_cfg->output_len); + dl_cfg->input_len = ul_cfg->output_len; + + memcpy(dl_cfg->output, ul_cfg->input, ul_cfg->input_len); + dl_cfg->output_len = ul_cfg->input_len; +} + +/* + * According to ETSI TS 138 323 V17.1.0, Section 5.2.2.1, + * SN could be divided into following ranges, + * relatively to current value of RX_DELIV state: + * +-------------+-------------+-------------+-------------+ + * | -Outside | -Window | +Window | +Outside | + * | (valid) | (Invalid) | (Valid) | (Invalid) | + * +-------------+-------------^-------------+-------------+ + * | + * v + * SN(RX_DELIV) + */ +enum sn_range_type { + SN_RANGE_MINUS_OUTSIDE, + SN_RANGE_MINUS_WINDOW, + SN_RANGE_PLUS_WINDOW, + SN_RANGE_PLUS_OUTSIDE, +}; + +#define PDCP_SET_COUNT(hfn, sn, size) ((hfn << size) | (sn & ((1 << size) - 1))) + +/* + * Take uplink test case as base, modify RX_DELIV in state and SN in input + */ +static int +test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) +{ + uint32_t rx_deliv_hfn, rx_deliv_sn, rx_deliv, new_hfn, new_sn; + const int domain = conf->entity.pdcp_xfrm.domain; + struct pdcp_test_conf dl_conf; + int ret, expected_ret; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + if (domain != RTE_SECURITY_PDCP_MODE_CONTROL && domain != RTE_SECURITY_PDCP_MODE_DATA) + return TEST_SKIPPED; + + const uint32_t sn_size = conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + /* Max value of SN that could fit in `sn_size` bits */ + const uint32_t max_sn = (1 << sn_size) - 1; + const uint32_t shift = (max_sn - window_size) / 2; + /* Could be any number up to `shift` value */ + const uint32_t default_sn = RTE_MIN(2u, shift); + + /* Initialize HFN as non zero value, to be able check values before */ + rx_deliv_hfn = 0xa; + + switch (type) { + case SN_RANGE_PLUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + 1; + expected_ret = TEST_SUCCESS; + break; + case SN_RANGE_MINUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn - 1; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_PLUS_OUTSIDE: + /* RCVD_SN >= SN(RX_DELIV) + Window_Size */ + new_hfn = rx_deliv_hfn - 1; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + window_size; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_MINUS_OUTSIDE: + /* RCVD_SN < SN(RX_DELIV) - Window_Size */ + new_hfn = rx_deliv_hfn + 1; + rx_deliv_sn = window_size + default_sn; + new_sn = rx_deliv_sn - window_size - 1; + expected_ret = TEST_SUCCESS; + break; + default: + return TEST_FAILED; + } + + rx_deliv = PDCP_SET_COUNT(rx_deliv_hfn, rx_deliv_sn, sn_size); + + /* Configure Uplink to generate expected, encrypted packet */ + pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.reverse_iv_direction = true; + conf->entity.count = PDCP_SET_COUNT(new_hfn, new_sn, sn_size); + conf->output_len = 0; + ret = test_attempt_single(conf); + if (ret != TEST_SUCCESS) + return ret; + + /* Flip configuration to downlink */ + uplink_to_downlink_convert(conf, &dl_conf); + + /* Modify the rx_deliv to verify the expected behaviour */ + dl_conf.entity.count = rx_deliv; + ret = test_attempt_single(&dl_conf); + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + return ret; + + TEST_ASSERT_EQUAL(ret, expected_ret, "Unexpected result"); + + return TEST_SUCCESS; +} + +static int +test_sn_plus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_WINDOW, t_conf); +} + +static int +test_sn_minus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_WINDOW, t_conf); +} + +static int +test_sn_plus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_OUTSIDE, t_conf); +} + +static int +test_sn_minus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); +} + +static int +test_combined(struct pdcp_test_conf *ul_conf) +{ + struct pdcp_test_conf dl_conf; + int ret; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + ul_conf->entity.reverse_iv_direction = true; + ul_conf->output_len = 0; + + ret = test_attempt_single(ul_conf); + if (ret != TEST_SUCCESS) + return ret; + + uplink_to_downlink_convert(ul_conf, &dl_conf); + ret = test_attempt_single(&dl_conf); + + return ret; +} + static int run_test_for_one_known_vec(const void *arg) { @@ -674,8 +944,34 @@ run_test_for_one_known_vec(const void *arg) return test_attempt_single(&test_conf); } +static struct unit_test_suite combined_mode_cases = { + .suite_name = "PDCP combined mode", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("combined mode", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_combined), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static struct unit_test_suite hfn_sn_test_cases = { + .suite_name = "PDCP HFN/SN", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("SN plus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_window), + TEST_CASE_NAMED_WITH_DATA("SN minus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_window), + TEST_CASE_NAMED_WITH_DATA("SN plus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_outside), + TEST_CASE_NAMED_WITH_DATA("SN minus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_outside), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ + &combined_mode_cases, + &hfn_sn_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 11/22] doc: add PDCP library guide 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (9 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-05-18 8:26 ` Akhil Goyal 2023-04-14 17:45 ` [PATCH v2 12/22] pdcp: add control PDU handling Anoob Joseph ` (11 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add guide for PDCP library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 246 ++++++++++++++++++ 3 files changed, 248 insertions(+) create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst diff --git a/doc/guides/prog_guide/img/pdcp_functional_overview.svg b/doc/guides/prog_guide/img/pdcp_functional_overview.svg new file mode 100644 index 0000000000..287daafc21 --- /dev/null +++ b/doc/guides/prog_guide/img/pdcp_functional_overview.svg @@ -0,0 +1 @@ +<svg width="1280" height="720" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><clipPath id="clip0"><rect x="0" y="0" width="1280" height="720"/></clipPath></defs><g clip-path="url(#clip0)"><rect x="0" y="0" width="1280" height="720" fill="#FFFFFF"/><rect x="202" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><rect x="640" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><path d="M605.5 73.5001 605.5 590.633" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M380.5 634 803.25 634 803.25 585.907 790.5 585.907 816 557.5 841.5 585.907 828.75 585.907 828.75 659.5 380.5 659.5Z" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5" fill-rule="evenodd"/><rect x="362.5" y="557.5" width="28" height="102" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5"/><rect x="412.5" y="671.5" width="370" height="32" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(492.364 694)">Radio Interface (<tspan font-size="19" x="137.5" y="0">Uu</tspan><tspan font-size="19" x="161.333" y="0">/PC5)</tspan><tspan font-size="19" x="-282.121" y="-653">UE/NG</tspan><tspan font-size="19" x="-222.955" y="-653">-</tspan><tspan font-size="19" x="-216.788" y="-653">RAN/UE A</tspan><tspan font-size="19" x="375.54" y="-653">NG</tspan>-<tspan font-size="19" x="409.706" y="-653">RAN/UE/UE B</tspan><tspan font-size="14" x="0.896851" y="-647">Transmitting </tspan><tspan font-size="14" x="1.31018" y="-631">PDCP entity</tspan><tspan font-size="14" x="167.401" y="-647">Receiving </tspan><tspan font-size="14" x="160.148" y="-631">PDCP entity</tspan></text><path d="M314.5 71.5001 431.364 71.6549" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M331.5 71.5001C331.5 65.9772 349.633 61.5001 372 61.5001 394.368 61.5001 412.5 65.9772 412.5 71.5001 412.5 77.0229 394.368 81.5001 372 81.5001 349.633 81.5001 331.5 77.0229 331.5 71.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M353.5 90.5001 363.5 90.5001 363.5 48.5001 383.5 48.5001 383.5 90.5001 393.5 90.5001 373.5 110.5Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M877.364 82.6549 760.5 82.5001" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M860.5 83.5001C860.5 89.0229 842.368 93.5001 820 93.5001 797.633 93.5001 779.5 89.0229 779.5 83.5001 779.5 77.9772 797.633 73.5001 820 73.5001 842.368 73.5001 860.5 77.9772 860.5 83.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M838.5 64.5 828.5 64.5 828.5 106.5 808.5 106.5 808.5 64.5 798.5 64.5 818.5 44.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><rect x="244.5" y="128.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(302.494 150)">Transmission buffer:<tspan font-size="19" x="-4.67999" y="23">Sequence</tspan><tspan font-size="19" x="84.32" y="23">numberin</tspan>g</text><rect x="244.5" y="199.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(297.314 222)">Header or uplink data<tspan font-size="19" x="34.08" y="23">Compressio</tspan>n</text><rect x="682.5" y="141.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(735.782 164)">Header or uplink data<tspan font-size="19" x="24.2466" y="23">Decompressio</tspan>n</text><rect x="244.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(300.314 505)">Routing / Duplication</text><rect x="244.5" y="437.5" width="285" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(309.734 460)">Add PDCP header</text><rect x="244.5" y="383.5" width="189" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(298.62 406)">Ciphering</text><rect x="244.5" y="333.5" width="189" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(261.953 356)">Integrity protection</text><path d="M472.167 267.5 472.167 409.802 470.833 409.802 470.833 267.5ZM475.5 408.468 471.5 416.468 467.5 408.468Z"/><path d="M332.167 267.5 332.167 319.552 330.833 319.552 330.833 267.5ZM335.5 318.218 331.5 326.218 327.5 318.219Z"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(253.644 291)">Packets associated <tspan font-size="19" x="14.0067" y="23">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 499.312 299)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="-2.75546e-15" y="45">PDCP SDU</tspan></text><rect x="682.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(728.535 505)">Remove PDCP Header</text><rect x="682.5" y="437.5" width="203" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(733.729 460)">Deciphering</text><rect x="682.5" y="389.5" width="203" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(702.159 412)">Integrity Verification</text><rect x="682.5" y="303.5" width="203" height="77" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(712.729 325)">Reception buffer:<tspan font-size="19" x="24.6667" y="23">Reordering</tspan><tspan font-size="19" x="-13.1667" y="45">Duplicate discardin</tspan><tspan font-size="19" x="144.167" y="45">g</tspan><tspan font-size="19" x="-10.0989" y="-84">Packets associated </tspan><tspan font-size="19" x="3.90784" y="-61">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 960.34 294)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="1" y="45">PDCP SDU</tspan></text><path d="M0.666667-8.2074e-07 0.666763 78.6116-0.66657 78.6116-0.666667 8.2074e-07ZM4.00009 77.2782 0.000104987 85.2782-3.9999 77.2782Z" transform="matrix(1 0 0 -1 779.5 296.778)"/><path d="M0.666667-2.88742e-07 0.666769 235.734-0.666565 235.734-0.666667 2.88742e-07ZM4.0001 234.401 0.000104987 242.401-3.9999 234.401Z" transform="matrix(1 0 0 -1 931.5 453.901)"/></g></svg> \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 87333ee84a..6099ff63cd 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -77,4 +77,5 @@ Programmer's Guide lto profile_app asan + pdcp_lib glossary diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst new file mode 100644 index 0000000000..abd874f2cc --- /dev/null +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -0,0 +1,246 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2023 Marvell. + +PDCP Protocol Processing Library +================================ + +DPDK provides a library for PDCP protocol processing. The library utilizes +other DPDK libraries such as cryptodev, reorder, etc., to provide the +application with a transparent and high performant PDCP protocol processing +library. + +The library abstracts complete PDCP protocol processing conforming to +``ETSI TS 138 323 V17.1.0 (2022-08)``. +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + +PDCP would involve the following operations, + +1. Transfer of user plane data +2. Transfer of control plane data +3. Header compression +4. Uplink data compression +5. Ciphering and integrity protection + +.. _figure_pdcp_functional_overview: + +.. figure:: img/pdcp_functional_overview.* + + PDCP functional overview new + +PDCP library would abstract the protocol offload features of the cryptodev and +would provide a uniform interface and consistent API usage to work with +cryptodev irrespective of the protocol offload features supported. + +PDCP entity API +--------------- + +PDCP library provides following control path APIs that is used to +configure various PDCP entities, + +1. ``rte_pdcp_entity_establish()`` +2. ``rte_pdcp_entity_suspend()`` +3. ``rte_pdcp_entity_release()`` + +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or +``rte_security_session`` based on the config. The sessions would be created/ +destroyed while corresponding PDCP entity operations are performed. + +PDCP PDU (Protocol Data Unit) +----------------------------- + +PDCP PDUs can be categorized as, + +1. Control PDU +2. Data PDU + +Control PDUs are used for signalling between entities on either end and can be +one of the following, + +1. PDCP status report +2. ROHC feedback +3. EHC feedback + +Control PDUs are not ciphered or authenticated, and so such packets are not +submitted to cryptodev for processing. + +Data PDUs are regular packets submitted by upper layers for transmission to +other end. Such packets would need to be ciphered and authenticated based on +the entity configuration. + +PDCP packet processing API for data PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PDCP processing is split into 2 parts. One before cryptodev processing +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return crypto +operations belonging to multiple entities, ``rte_pdcp_pkt_crypto_group()`` +is added to help grouping crypto operations belonging to same PDCP entity. + +Lib PDCP would allow application to use same API sequence while leveraging +protocol offload features enabled by ``rte_security`` library. Lib PDCP would +internally change the handles registered for ``pre_process`` and +``post_process`` based on features enabled in the entity. + +Lib PDCP would create the required sessions on the device provided in entity to +minimize the application requirements. Also, the crypto_op allocation and free +would also be done internally by lib PDCP to allow the library to create +crypto ops as required for the input packets. For example, when control PDUs are +received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP +is expected to handle it differently. + +Sample API usage +---------------- + +The ``rte_pdcp_entity_conf`` structure is used to pass the configuration +parameters for entity creation. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_entity_conf 8< + :end-before: >8 End of structure rte_pdcp_entity_conf. + +.. code-block:: c + + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; + struct rte_crypto_op *cop[MAX_BURST_SIZE]; + struct rte_pdcp_group grp[MAX_BURST_SIZE]; + struct rte_pdcp_entity *pdcp_entity; + int nb_max_out_mb, ret, nb_grp; + uint16_t nb_ops; + + /* Create PDCP entity */ + pdcp_entity = rte_pdcp_entity_establish(&conf); + + /** + * Allocate buffer for holding mbufs returned during PDCP suspend, + * release & post-process APIs. + */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + MAX_BURST_SIZE; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + /* Handle error */ + } + + while (1) { + /* Receive packet and form mbuf */ + + /** + * Prepare packets for crypto operation. Following operations + * would be done, + * + * Transmitting entity/UL (only data PDUs): + * - Perform compression + * - Assign sequence number + * - Add PDCP header + * - Create & prepare crypto_op + * - Prepare IV for crypto operation (auth_gen, encrypt) + * - Save original PDCP SDU (during PDCP re-establishment, + * unconfirmed PDCP SDUs need to crypto processed again and + * transmitted/re-transmitted) + * + * Receiving entity/DL: + * - Any control PDUs received would be processed and + * appropriate actions taken. If data PDU, continue. + * - Determine sequence number (based on HFN & per packet SN) + * - Prepare crypto_op + * - Prepare IV for crypto operation (decrypt, auth_verify) + */ + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, + nb_rx, &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) + != nb_success) { + /* Retry for enqueue failure packets */ + } + + ... + + nb_ops = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, + MAX_BURST_SIZE); + if (nb_ops == 0) + continue; + + /** + * Received a burst of completed crypto ops from cryptodev. It + * may belong to various entities. Group similar ones together + * for entity specific post-processing. + */ + + /** + * Groups similar entities together. Frees crypto op and based + * on crypto_op status, set mbuf->ol_flags which would be + * checked in rte_pdcp_pkt_post_process(). + */ + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); + + for (i = 0; i != nb_grp; i++) { + + /** + * Post process packets after crypto completion. + * Following operations would be done, + * + * Transmitting entity/UL: + * - Check crypto result + * + * Receiving entity/DL: + * - Check crypto operation status + * - Check for duplication (if yes, drop duplicate) + * - Perform decompression + * - Trim PDCP header + * - Hold packet (SDU) for in-order delivery (return + * completed packets as and when sequence is + * completed) + * - If not in sequence, cache the packet and start + * t-Reordering timer. When timer expires, the + * packets need to delivered to upper layers (not + * treated as error packets). + */ + nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, + grp[i].m, out_mb, + grp[i].cnt, + &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + /* Perform additional operations */ + + /** + * Transmitting entity/UL + * - If duplication is enabled, duplicate PDCP PDUs + * - When lower layers confirm reception of a PDCP PDU, + * it should be communicated to PDCP layer so that + * PDCP can drop the corresponding SDU + */ + } + } + + +Supported features +------------------ + +- 12 bit & 18 bit sequence numbers +- Uplink & downlink traffic +- HFN increment +- IV generation as required per algorithm + +Supported ciphering algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CTR +- SNOW3G-CIPHER +- ZUC-CIPHER + +Supported integrity protection algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CMAC +- SNOW3G-AUTH +- ZUC-AUTH -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 11/22] doc: add PDCP library guide 2023-04-14 17:45 ` [PATCH v2 11/22] doc: add PDCP library guide Anoob Joseph @ 2023-05-18 8:26 ` Akhil Goyal 2023-05-22 10:22 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 8:26 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > diff --git a/doc/guides/prog_guide/pdcp_lib.rst > b/doc/guides/prog_guide/pdcp_lib.rst > new file mode 100644 > index 0000000000..abd874f2cc > --- /dev/null > +++ b/doc/guides/prog_guide/pdcp_lib.rst > @@ -0,0 +1,246 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(C) 2023 Marvell. > + > +PDCP Protocol Processing Library > +================================ > + > +DPDK provides a library for PDCP protocol processing. The library utilizes > +other DPDK libraries such as cryptodev, reorder, etc., to provide the > +application with a transparent and high performant PDCP protocol processing > +library. > + > +The library abstracts complete PDCP protocol processing conforming to > +``ETSI TS 138 323 V17.1.0 (2022-08)``. > +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_ > 138323v170100p.pdf > + > +PDCP would involve the following operations, > + > +1. Transfer of user plane data > +2. Transfer of control plane data > +3. Header compression > +4. Uplink data compression > +5. Ciphering and integrity protection > + > +.. _figure_pdcp_functional_overview: > + > +.. figure:: img/pdcp_functional_overview.* > + > + PDCP functional overview new > + > +PDCP library would abstract the protocol offload features of the cryptodev and > +would provide a uniform interface and consistent API usage to work with > +cryptodev irrespective of the protocol offload features supported. > + > +PDCP entity API > +--------------- > + > +PDCP library provides following control path APIs that is used to > +configure various PDCP entities, > + > +1. ``rte_pdcp_entity_establish()`` > +2. ``rte_pdcp_entity_suspend()`` > +3. ``rte_pdcp_entity_release()`` > + > +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or > +``rte_security_session`` based on the config. The sessions would be created/ > +destroyed while corresponding PDCP entity operations are performed. Please explain the difference between suspend and release here. > + > +PDCP PDU (Protocol Data Unit) > +----------------------------- > + > +PDCP PDUs can be categorized as, > + > +1. Control PDU > +2. Data PDU > + > +Control PDUs are used for signalling between entities on either end and can be > +one of the following, > + > +1. PDCP status report > +2. ROHC feedback > +3. EHC feedback > + > +Control PDUs are not ciphered or authenticated, and so such packets are not > +submitted to cryptodev for processing. > + > +Data PDUs are regular packets submitted by upper layers for transmission to > +other end. Such packets would need to be ciphered and authenticated based on > +the entity configuration. > + Please move the PDCP PDU section above PDCP entity API section. So that all APIs are together. > +PDCP packet processing API for data PDU > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +PDCP processing is split into 2 parts. One before cryptodev processing > +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing > +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return crypto > +operations belonging to multiple entities, ``rte_pdcp_pkt_crypto_group()`` > +is added to help grouping crypto operations belonging to same PDCP entity. > + > +Lib PDCP would allow application to use same API sequence while leveraging > +protocol offload features enabled by ``rte_security`` library. Lib PDCP would > +internally change the handles registered for ``pre_process`` and > +``post_process`` based on features enabled in the entity. > + > +Lib PDCP would create the required sessions on the device provided in entity to > +minimize the application requirements. Also, the crypto_op allocation and free > +would also be done internally by lib PDCP to allow the library to create > +crypto ops as required for the input packets. For example, when control PDUs > are > +received, no cryptodev enqueue-dequeue is expected for the same and lib > PDCP > +is expected to handle it differently. > + > +Sample API usage > +---------------- > + > +The ``rte_pdcp_entity_conf`` structure is used to pass the configuration > +parameters for entity creation. > + > +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h > + :language: c > + :start-after: Structure rte_pdcp_entity_conf 8< > + :end-before: >8 End of structure rte_pdcp_entity_conf. > + > +.. code-block:: c > + > + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; > + struct rte_crypto_op *cop[MAX_BURST_SIZE]; > + struct rte_pdcp_group grp[MAX_BURST_SIZE]; > + struct rte_pdcp_entity *pdcp_entity; > + int nb_max_out_mb, ret, nb_grp; > + uint16_t nb_ops; > + > + /* Create PDCP entity */ > + pdcp_entity = rte_pdcp_entity_establish(&conf); > + > + /** > + * Allocate buffer for holding mbufs returned during PDCP suspend, > + * release & post-process APIs. > + */ > + > + /* Max packets that can be cached in entity + burst size */ > + nb_max_out_mb = pdcp_entity->max_pkt_cache + MAX_BURST_SIZE; > + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); > + if (out_mb == NULL) { > + /* Handle error */ > + } > + > + while (1) { > + /* Receive packet and form mbuf */ > + > + /** > + * Prepare packets for crypto operation. Following operations > + * would be done, > + * > + * Transmitting entity/UL (only data PDUs): > + * - Perform compression > + * - Assign sequence number > + * - Add PDCP header > + * - Create & prepare crypto_op > + * - Prepare IV for crypto operation (auth_gen, encrypt) > + * - Save original PDCP SDU (during PDCP re-establishment, > + * unconfirmed PDCP SDUs need to crypto processed again > and > + * transmitted/re-transmitted) > + * > + * Receiving entity/DL: > + * - Any control PDUs received would be processed and > + * appropriate actions taken. If data PDU, continue. > + * - Determine sequence number (based on HFN & per packet > SN) > + * - Prepare crypto_op > + * - Prepare IV for crypto operation (decrypt, auth_verify) > + */ > + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, > + nb_rx, &nb_err); > + if (nb_err != 0) { > + /* Handle error packets */ > + } > + > + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, > nb_success) > + != nb_success) { > + /* Retry for enqueue failure packets */ > + } > + > + ... > + > + nb_ops = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, > + MAX_BURST_SIZE); > + if (nb_ops == 0) > + continue; > + > + /** > + * Received a burst of completed crypto ops from cryptodev. It > + * may belong to various entities. Group similar ones together > + * for entity specific post-processing. > + */ > + > + /** > + * Groups similar entities together. Frees crypto op and based > + * on crypto_op status, set mbuf->ol_flags which would be > + * checked in rte_pdcp_pkt_post_process(). > + */ > + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); > + > + for (i = 0; i != nb_grp; i++) { > + > + /** > + * Post process packets after crypto completion. > + * Following operations would be done, > + * > + * Transmitting entity/UL: > + * - Check crypto result > + * > + * Receiving entity/DL: > + * - Check crypto operation status > + * - Check for duplication (if yes, drop duplicate) > + * - Perform decompression > + * - Trim PDCP header > + * - Hold packet (SDU) for in-order delivery (return > + * completed packets as and when sequence is > + * completed) > + * - If not in sequence, cache the packet and start > + * t-Reordering timer. When timer expires, the > + * packets need to delivered to upper layers (not > + * treated as error packets). > + */ > + nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, > + grp[i].m, out_mb, > + grp[i].cnt, > + &nb_err); > + if (nb_err != 0) { > + /* Handle error packets */ > + } > + > + /* Perform additional operations */ > + > + /** > + * Transmitting entity/UL > + * - If duplication is enabled, duplicate PDCP PDUs > + * - When lower layers confirm reception of a PDCP > PDU, > + * it should be communicated to PDCP layer so that > + * PDCP can drop the corresponding SDU > + */ > + } > + } > + > + > +Supported features > +------------------ > + > +- 12 bit & 18 bit sequence numbers > +- Uplink & downlink traffic > +- HFN increment > +- IV generation as required per algorithm > + > +Supported ciphering algorithms > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +- NULL > +- AES-CTR > +- SNOW3G-CIPHER > +- ZUC-CIPHER > + > +Supported integrity protection algorithms > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +- NULL > +- AES-CMAC > +- SNOW3G-AUTH > +- ZUC-AUTH > -- Move Supported features and algos after PDCP PDU explanation. Sample sequence should be in the end. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 11/22] doc: add PDCP library guide 2023-05-18 8:26 ` Akhil Goyal @ 2023-05-22 10:22 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-22 10:22 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Thursday, May 18, 2023 1:56 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 11/22] doc: add PDCP library guide > > > diff --git a/doc/guides/prog_guide/pdcp_lib.rst > > b/doc/guides/prog_guide/pdcp_lib.rst > > new file mode 100644 > > index 0000000000..abd874f2cc > > --- /dev/null > > +++ b/doc/guides/prog_guide/pdcp_lib.rst > > @@ -0,0 +1,246 @@ > > +.. SPDX-License-Identifier: BSD-3-Clause > > + Copyright(C) 2023 Marvell. > > + > > +PDCP Protocol Processing Library > > +================================ > > + > > +DPDK provides a library for PDCP protocol processing. The library > > +utilizes other DPDK libraries such as cryptodev, reorder, etc., to > > +provide the application with a transparent and high performant PDCP > > +protocol processing library. > > + > > +The library abstracts complete PDCP protocol processing conforming to > > +``ETSI TS 138 323 V17.1.0 (2022-08)``. > > +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60 > > +/ts_ > > 138323v170100p.pdf > > + > > +PDCP would involve the following operations, > > + > > +1. Transfer of user plane data > > +2. Transfer of control plane data > > +3. Header compression > > +4. Uplink data compression > > +5. Ciphering and integrity protection > > + > > +.. _figure_pdcp_functional_overview: > > + > > +.. figure:: img/pdcp_functional_overview.* > > + > > + PDCP functional overview new > > + > > +PDCP library would abstract the protocol offload features of the > > +cryptodev and would provide a uniform interface and consistent API > > +usage to work with cryptodev irrespective of the protocol offload > features supported. > > + > > +PDCP entity API > > +--------------- > > + > > +PDCP library provides following control path APIs that is used to > > +configure various PDCP entities, > > + > > +1. ``rte_pdcp_entity_establish()`` > > +2. ``rte_pdcp_entity_suspend()`` > > +3. ``rte_pdcp_entity_release()`` > > + > > +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or > > +``rte_security_session`` based on the config. The sessions would be > > +created/ destroyed while corresponding PDCP entity operations are > performed. > > Please explain the difference between suspend and release here. [Anoob] Sure. Will do in next version. > > > + > > +PDCP PDU (Protocol Data Unit) > > +----------------------------- > > + > > +PDCP PDUs can be categorized as, > > + > > +1. Control PDU > > +2. Data PDU > > + > > +Control PDUs are used for signalling between entities on either end > > +and can be one of the following, > > + > > +1. PDCP status report > > +2. ROHC feedback > > +3. EHC feedback > > + > > +Control PDUs are not ciphered or authenticated, and so such packets > > +are not submitted to cryptodev for processing. > > + > > +Data PDUs are regular packets submitted by upper layers for > > +transmission to other end. Such packets would need to be ciphered and > > +authenticated based on the entity configuration. > > + > Please move the PDCP PDU section above PDCP entity API section. > So that all APIs are together. [Anoob] PDCP PDU section is also talking about APIs. I'll rename above title from 'PDCP PDU(Protocol Data Unit)' to 'PDCP PDU(Protocol Data Unit) API' to make it more uniform. > > > > +PDCP packet processing API for data PDU > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > > +PDCP processing is split into 2 parts. One before cryptodev > > +processing > > +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing > > +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return > > +crypto operations belonging to multiple entities, > > +``rte_pdcp_pkt_crypto_group()`` is added to help grouping crypto > operations belonging to same PDCP entity. > > + > > +Lib PDCP would allow application to use same API sequence while > > +leveraging protocol offload features enabled by ``rte_security`` > > +library. Lib PDCP would internally change the handles registered for > > +``pre_process`` and ``post_process`` based on features enabled in the > entity. > > + > > +Lib PDCP would create the required sessions on the device provided in > > +entity to minimize the application requirements. Also, the crypto_op > > +allocation and free would also be done internally by lib PDCP to > > +allow the library to create crypto ops as required for the input > > +packets. For example, when control PDUs > > are > > +received, no cryptodev enqueue-dequeue is expected for the same and > > +lib > > PDCP > > +is expected to handle it differently. > > + > > +Sample API usage > > +---------------- > > + > > +The ``rte_pdcp_entity_conf`` structure is used to pass the > > +configuration parameters for entity creation. > > + > > +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h > > + :language: c > > + :start-after: Structure rte_pdcp_entity_conf 8< > > + :end-before: >8 End of structure rte_pdcp_entity_conf. > > + > > +.. code-block:: c > > + > > + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; > > + struct rte_crypto_op *cop[MAX_BURST_SIZE]; > > + struct rte_pdcp_group grp[MAX_BURST_SIZE]; > > + struct rte_pdcp_entity *pdcp_entity; > > + int nb_max_out_mb, ret, nb_grp; > > + uint16_t nb_ops; > > + > > + /* Create PDCP entity */ > > + pdcp_entity = rte_pdcp_entity_establish(&conf); > > + > > + /** > > + * Allocate buffer for holding mbufs returned during PDCP suspend, > > + * release & post-process APIs. > > + */ > > + > > + /* Max packets that can be cached in entity + burst size */ > > + nb_max_out_mb = pdcp_entity->max_pkt_cache + > MAX_BURST_SIZE; > > + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); > > + if (out_mb == NULL) { > > + /* Handle error */ > > + } > > + > > + while (1) { > > + /* Receive packet and form mbuf */ > > + > > + /** > > + * Prepare packets for crypto operation. Following operations > > + * would be done, > > + * > > + * Transmitting entity/UL (only data PDUs): > > + * - Perform compression > > + * - Assign sequence number > > + * - Add PDCP header > > + * - Create & prepare crypto_op > > + * - Prepare IV for crypto operation (auth_gen, encrypt) > > + * - Save original PDCP SDU (during PDCP re-establishment, > > + * unconfirmed PDCP SDUs need to crypto processed again > > and > > + * transmitted/re-transmitted) > > + * > > + * Receiving entity/DL: > > + * - Any control PDUs received would be processed and > > + * appropriate actions taken. If data PDU, continue. > > + * - Determine sequence number (based on HFN & per > packet > > SN) > > + * - Prepare crypto_op > > + * - Prepare IV for crypto operation (decrypt, auth_verify) > > + */ > > + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, > cop, > > + nb_rx, &nb_err); > > + if (nb_err != 0) { > > + /* Handle error packets */ > > + } > > + > > + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, > > nb_success) > > + != nb_success) { > > + /* Retry for enqueue failure packets */ > > + } > > + > > + ... > > + > > + nb_ops = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, > > + MAX_BURST_SIZE); > > + if (nb_ops == 0) > > + continue; > > + > > + /** > > + * Received a burst of completed crypto ops from cryptodev. > It > > + * may belong to various entities. Group similar ones > together > > + * for entity specific post-processing. > > + */ > > + > > + /** > > + * Groups similar entities together. Frees crypto op and > based > > + * on crypto_op status, set mbuf->ol_flags which would be > > + * checked in rte_pdcp_pkt_post_process(). > > + */ > > + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); > > + > > + for (i = 0; i != nb_grp; i++) { > > + > > + /** > > + * Post process packets after crypto completion. > > + * Following operations would be done, > > + * > > + * Transmitting entity/UL: > > + * - Check crypto result > > + * > > + * Receiving entity/DL: > > + * - Check crypto operation status > > + * - Check for duplication (if yes, drop duplicate) > > + * - Perform decompression > > + * - Trim PDCP header > > + * - Hold packet (SDU) for in-order delivery (return > > + * completed packets as and when sequence is > > + * completed) > > + * - If not in sequence, cache the packet and start > > + * t-Reordering timer. When timer expires, the > > + * packets need to delivered to upper layers (not > > + * treated as error packets). > > + */ > > + nb_success = > rte_pdcp_pkt_post_process(grp[i].id.ptr, > > + grp[i].m, out_mb, > > + grp[i].cnt, > > + &nb_err); > > + if (nb_err != 0) { > > + /* Handle error packets */ > > + } > > + > > + /* Perform additional operations */ > > + > > + /** > > + * Transmitting entity/UL > > + * - If duplication is enabled, duplicate PDCP PDUs > > + * - When lower layers confirm reception of a PDCP > > PDU, > > + * it should be communicated to PDCP layer so that > > + * PDCP can drop the corresponding SDU > > + */ > > + } > > + } > > + > > + > > +Supported features > > +------------------ > > + > > +- 12 bit & 18 bit sequence numbers > > +- Uplink & downlink traffic > > +- HFN increment > > +- IV generation as required per algorithm > > + > > +Supported ciphering algorithms > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > > +- NULL > > +- AES-CTR > > +- SNOW3G-CIPHER > > +- ZUC-CIPHER > > + > > +Supported integrity protection algorithms > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > > +- NULL > > +- AES-CMAC > > +- SNOW3G-AUTH > > +- ZUC-AUTH > > -- > Move Supported features and algos after PDCP PDU explanation. > Sample sequence should be in the end. [Anoob] Agreed. Will make this change in next version. ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 12/22] pdcp: add control PDU handling 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (10 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 11/22] doc: add PDCP library guide Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-05-18 9:15 ` Akhil Goyal 2023-04-14 17:45 ` [PATCH v2 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph ` (10 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add control PDU handling and implement status report generation. Status report generation works only when RX_DELIV = RX_NEXT. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 1 + doc/guides/prog_guide/pdcp_lib.rst | 10 +++++++ lib/pdcp/meson.build | 2 ++ lib/pdcp/pdcp_cnt.c | 29 ++++++++++++++++++ lib/pdcp/pdcp_cnt.h | 14 +++++++++ lib/pdcp/pdcp_ctrl_pdu.c | 46 +++++++++++++++++++++++++++++ lib/pdcp/pdcp_ctrl_pdu.h | 15 ++++++++++ lib/pdcp/pdcp_entity.h | 15 ++++++++-- lib/pdcp/pdcp_process.c | 13 +++++++++ lib/pdcp/rte_pdcp.c | 47 +++++++++++++++++++++++++++++- lib/pdcp/rte_pdcp.h | 31 ++++++++++++++++++++ lib/pdcp/version.map | 2 ++ 12 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index fc49947ba2..4ecb4d9572 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -415,6 +415,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.sess_mpool = ts_params->sess_pool; conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.ctr_pdu_pool = ts_params->mbuf_pool; conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index abd874f2cc..f23360dfc3 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -67,6 +67,15 @@ Data PDUs are regular packets submitted by upper layers for transmission to other end. Such packets would need to be ciphered and authenticated based on the entity configuration. +PDCP packet processing API for control PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Control PDUs are used in PDCP as a communication channel between transmitting +and receiving entities. When upper layer request for operations such +re-establishment, receiving PDCP entity need to prepare a status report and +send it to the other end. The API ``pdcp_ctrl_pdu_status_gen`` allows +application to request the same. + PDCP packet processing API for data PDU ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -228,6 +237,7 @@ Supported features - Uplink & downlink traffic - HFN increment - IV generation as required per algorithm +- Control PDU generation Supported ciphering algorithms ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 08679b743a..75d476bf6d 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -8,7 +8,9 @@ if is_windows endif sources = files( + 'pdcp_cnt.c', 'pdcp_crypto.c', + 'pdcp_ctrl_pdu.c', 'pdcp_process.c', 'rte_pdcp.c', ) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c new file mode 100644 index 0000000000..c9b952184b --- /dev/null +++ b/lib/pdcp/pdcp_cnt.c @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_cnt.h" +#include "pdcp_entity.h" + +int +pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv_dl_part *en_priv_dl; + uint32_t window_sz; + + if (en == NULL || conf == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + return 0; + + en_priv_dl = entity_dl_part_get(en); + window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + RTE_SET_USED(window_sz); + RTE_SET_USED(en_priv_dl); + + return 0; +} diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h new file mode 100644 index 0000000000..bbda478b55 --- /dev/null +++ b/lib/pdcp/pdcp_cnt.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CNT_H +#define PDCP_CNT_H + +#include <rte_common.h> + +#include "pdcp_entity.h" + +int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c new file mode 100644 index 0000000000..feb05fd863 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_byteorder.h> +#include <rte_mbuf.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_ctrl_pdu.h" +#include "pdcp_entity.h" + +static __rte_always_inline void +pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) +{ + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + pdu_hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + pdu_hdr->r = 0; + pdu_hdr->fmc = rte_cpu_to_be_32(rx_deliv); +} + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; + uint32_t rx_deliv; + int pdu_sz; + + if (!en_priv->flags.is_status_report_required) + return -EINVAL; + + pdu_sz = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + + rx_deliv = en_priv->state.rx_deliv; + + /* Zero missing PDUs - status report contains only FMC */ + if (rx_deliv >= en_priv->state.rx_next) { + pdu_hdr = (struct rte_pdcp_up_ctrl_pdu_hdr *)rte_pktmbuf_append(m, pdu_sz); + if (pdu_hdr == NULL) + return -ENOMEM; + pdcp_hdr_fill(pdu_hdr, rx_deliv); + + return 0; + } + + return -ENOTSUP; +} diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h new file mode 100644 index 0000000000..a2424fbd10 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CTRL_PDU_H +#define PDCP_CTRL_PDU_H + +#include <rte_mbuf.h> + +#include "pdcp_entity.h" + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); + +#endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 3108795977..7b7e7f69dd 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -109,6 +109,13 @@ union cipher_iv_partial { uint64_t u64[2]; }; +struct pdcp_cnt_bitmap { + /** Number of entries that can be stored. */ + uint32_t size; + /** Bitmap of the count values already received.*/ + struct rte_bitmap *bmp; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -136,9 +143,13 @@ struct entity_priv { uint64_t is_ul_entity : 1; /** Is NULL auth. */ uint64_t is_null_auth : 1; + /** Is status report required.*/ + uint64_t is_status_report_required : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; + /** Control PDU pool. */ + struct rte_mempool *ctr_pdu_pool; /** PDCP header size. */ uint8_t hdr_sz; /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ @@ -148,8 +159,8 @@ struct entity_priv { }; struct entity_priv_dl_part { - /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ - uint8_t dummy; + /** PDCP would need to track the count values that are already received.*/ + struct pdcp_cnt_bitmap bitmap; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 9c1a5e0669..267b3b7723 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -1157,6 +1157,19 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) en_priv->flags.is_null_auth = 1; + /** + * flags.is_status_report_required + * + * Indicate whether status report is required. + */ + if (conf->status_report_required) { + /** Status report is required only for DL entities. */ + if (conf->pdcp_xfrm.pkt_dir != RTE_SECURITY_PDCP_DOWNLINK) + return -EINVAL; + + en_priv->flags.is_status_report_required = 1; + } + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 8914548dbd..5cd3f5ca31 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -6,7 +6,9 @@ #include <rte_pdcp.h> #include <rte_malloc.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -34,7 +36,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) struct entity_priv *en_priv; int ret, entity_size; - if (conf == NULL || conf->cop_pool == NULL) { + if (conf == NULL || conf->cop_pool == NULL || conf->ctr_pdu_pool == NULL) { rte_errno = -EINVAL; return NULL; } @@ -79,6 +81,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) en_priv->state.rx_deliv = conf->count; en_priv->state.tx_next = conf->count; en_priv->cop_pool = conf->cop_pool; + en_priv->ctr_pdu_pool = conf->ctr_pdu_pool; /* Setup crypto session */ ret = pdcp_crypto_sess_create(entity, conf); @@ -89,6 +92,10 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + ret = pdcp_cnt_ring_create(entity, conf); + if (ret) + goto crypto_sess_destroy; + return entity; crypto_sess_destroy: @@ -136,3 +143,41 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, return 0; } + +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type) +{ + struct entity_priv *en_priv; + struct rte_mbuf *m; + int ret; + + if (pdcp_entity == NULL) { + rte_errno = -EINVAL; + return NULL; + } + + en_priv = entity_priv_get(pdcp_entity); + + m = rte_pktmbuf_alloc(en_priv->ctr_pdu_pool); + if (m == NULL) { + rte_errno = -ENOMEM; + return NULL; + } + + switch (type) { + case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: + ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + break; + default: + ret = -ENOTSUP; + } + + if (ret) { + rte_pktmbuf_free(m); + rte_errno = ret; + return NULL; + } + + return m; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 54f88e3fd3..d2db25d7d9 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -16,6 +16,7 @@ #include <rte_compat.h> #include <rte_common.h> #include <rte_mempool.h> +#include <rte_pdcp_hdr.h> #include <rte_security.h> #ifdef __cplusplus @@ -78,6 +79,8 @@ struct rte_pdcp_entity_conf { struct rte_mempool *sess_mpool; /** Crypto op pool.*/ struct rte_mempool *cop_pool; + /** Mbuf pool to be used for allocating control PDUs.*/ + struct rte_mempool *ctr_pdu_pool; /** * 32 bit count value (HFN + SN) to be used for the first packet. * pdcp_xfrm.hfn would be ignored as the HFN would be derived from this value. @@ -91,6 +94,15 @@ struct rte_pdcp_entity_conf { uint8_t dev_id; /** Reverse direction during IV generation. Can be used to simulate UE crypto processing.*/ bool reverse_iv_direction; + /** + * Status report required (specified in TS 38.331). + * + * If PDCP entity is configured to send a PDCP status report, the upper layer application + * may request a receiving PDCP entity to generate a PDCP status report using + * ``rte_pdcp_ctrl_pdu_create``. In addition, PDCP status reports may be generated during + * operations such as entity re-establishment. + */ + bool status_report_required; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -169,6 +181,25 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Create control PDU packet of the `type` specified. + * + * @param pdcp_entity + * Pointer to the PDCP entity for which the control PDU need to be generated. + * @param type + * Type of control PDU to be generated. + * @return + * - Control PDU generated, in case of success. + * - NULL in case of failure. rte_errno will be set to error code. + */ +__rte_experimental +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index f9ff30600a..d0cf338e1f 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -6,6 +6,8 @@ EXPERIMENTAL { rte_pdcp_entity_release; rte_pdcp_entity_suspend; + rte_pdcp_control_pdu_create; + rte_pdcp_pkt_post_process; rte_pdcp_pkt_pre_process; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 12/22] pdcp: add control PDU handling 2023-04-14 17:45 ` [PATCH v2 12/22] pdcp: add control PDU handling Anoob Joseph @ 2023-05-18 9:15 ` Akhil Goyal 2023-05-22 11:09 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 9:15 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz > Subject: [PATCH v2 12/22] pdcp: add control PDU handling > > Add control PDU handling and implement status report generation. Status > report generation works only when RX_DELIV = RX_NEXT. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > --- > app/test/test_pdcp.c | 1 + Separate out test app changes from library changes. > doc/guides/prog_guide/pdcp_lib.rst | 10 +++++++ > lib/pdcp/meson.build | 2 ++ > lib/pdcp/pdcp_cnt.c | 29 ++++++++++++++++++ > lib/pdcp/pdcp_cnt.h | 14 +++++++++ > lib/pdcp/pdcp_ctrl_pdu.c | 46 +++++++++++++++++++++++++++++ > lib/pdcp/pdcp_ctrl_pdu.h | 15 ++++++++++ > lib/pdcp/pdcp_entity.h | 15 ++++++++-- > lib/pdcp/pdcp_process.c | 13 +++++++++ > lib/pdcp/rte_pdcp.c | 47 +++++++++++++++++++++++++++++- > lib/pdcp/rte_pdcp.h | 31 ++++++++++++++++++++ > lib/pdcp/version.map | 2 ++ > 12 files changed, 222 insertions(+), 3 deletions(-) > create mode 100644 lib/pdcp/pdcp_cnt.c > create mode 100644 lib/pdcp/pdcp_cnt.h > create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c > create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h > > diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c > index fc49947ba2..4ecb4d9572 100644 > --- a/app/test/test_pdcp.c > +++ b/app/test/test_pdcp.c > @@ -415,6 +415,7 @@ create_test_conf_from_index(const int index, struct > pdcp_test_conf *conf) > > conf->entity.sess_mpool = ts_params->sess_pool; > conf->entity.cop_pool = ts_params->cop_pool; > + conf->entity.ctr_pdu_pool = ts_params->mbuf_pool; > conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; > conf->entity.pdcp_xfrm.en_ordering = 0; > conf->entity.pdcp_xfrm.remove_duplicates = 0; > diff --git a/doc/guides/prog_guide/pdcp_lib.rst > b/doc/guides/prog_guide/pdcp_lib.rst > index abd874f2cc..f23360dfc3 100644 > --- a/doc/guides/prog_guide/pdcp_lib.rst > +++ b/doc/guides/prog_guide/pdcp_lib.rst > @@ -67,6 +67,15 @@ Data PDUs are regular packets submitted by upper layers > for transmission to > other end. Such packets would need to be ciphered and authenticated based on > the entity configuration. > > +PDCP packet processing API for control PDU > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +Control PDUs are used in PDCP as a communication channel between > transmitting > +and receiving entities. When upper layer request for operations such > +re-establishment, receiving PDCP entity need to prepare a status report and > +send it to the other end. The API ``pdcp_ctrl_pdu_status_gen`` allows > +application to request the same. pdcp_ctrl_pdu_status_gen() - Is this a user visible API? I believe it is not exposed. How can application request that? > + > PDCP packet processing API for data PDU > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > @@ -228,6 +237,7 @@ Supported features > - Uplink & downlink traffic > - HFN increment > - IV generation as required per algorithm > +- Control PDU generation > > Supported ciphering algorithms > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build > index 08679b743a..75d476bf6d 100644 > --- a/lib/pdcp/meson.build > +++ b/lib/pdcp/meson.build > @@ -8,7 +8,9 @@ if is_windows > endif > > sources = files( > + 'pdcp_cnt.c', pdcp_cnt seems to be something related to count. And it is not related to this patch probably. > 'pdcp_crypto.c', > + 'pdcp_ctrl_pdu.c', > 'pdcp_process.c', > 'rte_pdcp.c', > ) > diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c > new file mode 100644 > index 0000000000..c9b952184b > --- /dev/null > +++ b/lib/pdcp/pdcp_cnt.c > @@ -0,0 +1,29 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#include <rte_pdcp.h> > + > +#include "pdcp_cnt.h" > +#include "pdcp_entity.h" > + > +int > +pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct > rte_pdcp_entity_conf *conf) > +{ > + struct entity_priv_dl_part *en_priv_dl; > + uint32_t window_sz; > + > + if (en == NULL || conf == NULL) > + return -EINVAL; > + > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > + return 0; > + > + en_priv_dl = entity_dl_part_get(en); > + window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); > + > + RTE_SET_USED(window_sz); > + RTE_SET_USED(en_priv_dl); > + > + return 0; > +} > diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h > new file mode 100644 > index 0000000000..bbda478b55 > --- /dev/null > +++ b/lib/pdcp/pdcp_cnt.h > @@ -0,0 +1,14 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#ifndef PDCP_CNT_H > +#define PDCP_CNT_H > + > +#include <rte_common.h> > + > +#include "pdcp_entity.h" > + > +int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct > rte_pdcp_entity_conf *conf); > + > +#endif /* PDCP_CNT_H */ > diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c > new file mode 100644 > index 0000000000..feb05fd863 > --- /dev/null > +++ b/lib/pdcp/pdcp_ctrl_pdu.c > @@ -0,0 +1,46 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#include <rte_byteorder.h> > +#include <rte_mbuf.h> > +#include <rte_pdcp_hdr.h> > + > +#include "pdcp_ctrl_pdu.h" > +#include "pdcp_entity.h" > + > +static __rte_always_inline void > +pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) > +{ > + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; > + pdu_hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; > + pdu_hdr->r = 0; > + pdu_hdr->fmc = rte_cpu_to_be_32(rx_deliv); > +} > + > +int > +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) > +{ > + struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; > + uint32_t rx_deliv; > + int pdu_sz; > + > + if (!en_priv->flags.is_status_report_required) > + return -EINVAL; > + > + pdu_sz = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); > + > + rx_deliv = en_priv->state.rx_deliv; > + > + /* Zero missing PDUs - status report contains only FMC */ > + if (rx_deliv >= en_priv->state.rx_next) { > + pdu_hdr = (struct rte_pdcp_up_ctrl_pdu_hdr > *)rte_pktmbuf_append(m, pdu_sz); > + if (pdu_hdr == NULL) > + return -ENOMEM; > + pdcp_hdr_fill(pdu_hdr, rx_deliv); > + > + return 0; > + } > + > + return -ENOTSUP; > +} > diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h > new file mode 100644 > index 0000000000..a2424fbd10 > --- /dev/null > +++ b/lib/pdcp/pdcp_ctrl_pdu.h > @@ -0,0 +1,15 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(C) 2023 Marvell. > + */ > + > +#ifndef PDCP_CTRL_PDU_H > +#define PDCP_CTRL_PDU_H > + > +#include <rte_mbuf.h> > + > +#include "pdcp_entity.h" > + > +int > +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); > + > +#endif /* PDCP_CTRL_PDU_H */ > diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h > index 3108795977..7b7e7f69dd 100644 > --- a/lib/pdcp/pdcp_entity.h > +++ b/lib/pdcp/pdcp_entity.h > @@ -109,6 +109,13 @@ union cipher_iv_partial { > uint64_t u64[2]; > }; > > +struct pdcp_cnt_bitmap { > + /** Number of entries that can be stored. */ > + uint32_t size; > + /** Bitmap of the count values already received.*/ > + struct rte_bitmap *bmp; > +}; This pdcp_cnt_bitmap is not related to control PDU. Right? I believe the patch split is not proper here. > + > /* > * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] > */ > @@ -136,9 +143,13 @@ struct entity_priv { > uint64_t is_ul_entity : 1; > /** Is NULL auth. */ > uint64_t is_null_auth : 1; > + /** Is status report required.*/ > + uint64_t is_status_report_required : 1; > } flags; > /** Crypto op pool. */ > struct rte_mempool *cop_pool; > + /** Control PDU pool. */ > + struct rte_mempool *ctr_pdu_pool; > /** PDCP header size. */ > uint8_t hdr_sz; > /** PDCP AAD size. For AES-CMAC, additional message is prepended for > the operation. */ > @@ -148,8 +159,8 @@ struct entity_priv { > }; > > struct entity_priv_dl_part { > - /* NOTE: when in-order-delivery is supported, post PDCP packets would > need to cached. */ > - uint8_t dummy; > + /** PDCP would need to track the count values that are already > received.*/ > + struct pdcp_cnt_bitmap bitmap; > }; > > struct entity_priv_ul_part { > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c > index 9c1a5e0669..267b3b7723 100644 > --- a/lib/pdcp/pdcp_process.c > +++ b/lib/pdcp/pdcp_process.c > @@ -1157,6 +1157,19 @@ pdcp_entity_priv_populate(struct entity_priv > *en_priv, const struct rte_pdcp_ent > if (a_xfrm != NULL && a_xfrm->auth.algo == > RTE_CRYPTO_AUTH_NULL) > en_priv->flags.is_null_auth = 1; > > + /** > + * flags.is_status_report_required > + * > + * Indicate whether status report is required. > + */ > + if (conf->status_report_required) { > + /** Status report is required only for DL entities. */ > + if (conf->pdcp_xfrm.pkt_dir != > RTE_SECURITY_PDCP_DOWNLINK) > + return -EINVAL; > + > + en_priv->flags.is_status_report_required = 1; > + } > + > /** > * hdr_sz > * > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c > index 8914548dbd..5cd3f5ca31 100644 > --- a/lib/pdcp/rte_pdcp.c > +++ b/lib/pdcp/rte_pdcp.c > @@ -6,7 +6,9 @@ > #include <rte_pdcp.h> > #include <rte_malloc.h> > > +#include "pdcp_cnt.h" > #include "pdcp_crypto.h" > +#include "pdcp_ctrl_pdu.h" > #include "pdcp_entity.h" > #include "pdcp_process.h" > > @@ -34,7 +36,7 @@ rte_pdcp_entity_establish(const struct > rte_pdcp_entity_conf *conf) > struct entity_priv *en_priv; > int ret, entity_size; > > - if (conf == NULL || conf->cop_pool == NULL) { > + if (conf == NULL || conf->cop_pool == NULL || conf->ctr_pdu_pool == > NULL) { > rte_errno = -EINVAL; > return NULL; > } > @@ -79,6 +81,7 @@ rte_pdcp_entity_establish(const struct > rte_pdcp_entity_conf *conf) > en_priv->state.rx_deliv = conf->count; > en_priv->state.tx_next = conf->count; > en_priv->cop_pool = conf->cop_pool; > + en_priv->ctr_pdu_pool = conf->ctr_pdu_pool; > > /* Setup crypto session */ > ret = pdcp_crypto_sess_create(entity, conf); > @@ -89,6 +92,10 @@ rte_pdcp_entity_establish(const struct > rte_pdcp_entity_conf *conf) > if (ret) > goto crypto_sess_destroy; > > + ret = pdcp_cnt_ring_create(entity, conf); > + if (ret) > + goto crypto_sess_destroy; > + > return entity; > > crypto_sess_destroy: > @@ -136,3 +143,41 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity > *pdcp_entity, > > return 0; > } > + > +struct rte_mbuf * > +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, > + enum rte_pdcp_ctrl_pdu_type type) > +{ > + struct entity_priv *en_priv; > + struct rte_mbuf *m; > + int ret; > + > + if (pdcp_entity == NULL) { > + rte_errno = -EINVAL; rte_errno should be positive values. > + return NULL; > + } > + > + en_priv = entity_priv_get(pdcp_entity); > + > + m = rte_pktmbuf_alloc(en_priv->ctr_pdu_pool); > + if (m == NULL) { > + rte_errno = -ENOMEM; > + return NULL; > + } > + > + switch (type) { > + case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: > + ret = pdcp_ctrl_pdu_status_gen(en_priv, m); > + break; > + default: > + ret = -ENOTSUP; > + } > + > + if (ret) { > + rte_pktmbuf_free(m); > + rte_errno = ret; > + return NULL; > + } > + > + return m; > +} > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h > index 54f88e3fd3..d2db25d7d9 100644 > --- a/lib/pdcp/rte_pdcp.h > +++ b/lib/pdcp/rte_pdcp.h > @@ -16,6 +16,7 @@ > #include <rte_compat.h> > #include <rte_common.h> > #include <rte_mempool.h> > +#include <rte_pdcp_hdr.h> > #include <rte_security.h> > > #ifdef __cplusplus > @@ -78,6 +79,8 @@ struct rte_pdcp_entity_conf { > struct rte_mempool *sess_mpool; > /** Crypto op pool.*/ > struct rte_mempool *cop_pool; > + /** Mbuf pool to be used for allocating control PDUs.*/ > + struct rte_mempool *ctr_pdu_pool; Please use same prefix for all control pdu stuff. I see cnt, ctr, ctrl, control is being used. I think "control" for top level API and "ctrl" for parameters should be fine. > /** > * 32 bit count value (HFN + SN) to be used for the first packet. > * pdcp_xfrm.hfn would be ignored as the HFN would be derived from > this value. > @@ -91,6 +94,15 @@ struct rte_pdcp_entity_conf { > uint8_t dev_id; > /** Reverse direction during IV generation. Can be used to simulate UE > crypto processing.*/ > bool reverse_iv_direction; > + /** > + * Status report required (specified in TS 38.331). > + * > + * If PDCP entity is configured to send a PDCP status report, the upper > layer application > + * may request a receiving PDCP entity to generate a PDCP status report > using > + * ``rte_pdcp_ctrl_pdu_create``. In addition, PDCP status reports may be > generated during > + * operations such as entity re-establishment. > + */ > + bool status_report_required; rte_pdcp_ctrl_pdu_create -> rte_pdcp_control_pdu_create Please specify that a PDU will be generated for status report. Otherwise, it seems some structure would be returned for status report But there is no mention of that. This will avoid confusion. > }; > /* >8 End of structure rte_pdcp_entity_conf. */ > > @@ -169,6 +181,25 @@ int > rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > struct rte_mbuf *out_mb[]); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * Create control PDU packet of the `type` specified. > + * > + * @param pdcp_entity > + * Pointer to the PDCP entity for which the control PDU need to be generated. > + * @param type > + * Type of control PDU to be generated. > + * @return > + * - Control PDU generated, in case of success. > + * - NULL in case of failure. rte_errno will be set to error code. > + */ > +__rte_experimental > +struct rte_mbuf * > +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, > + enum rte_pdcp_ctrl_pdu_type type); > + > /** > * @warning > * @b EXPERIMENTAL: this API may change without prior notice > diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map > index f9ff30600a..d0cf338e1f 100644 > --- a/lib/pdcp/version.map > +++ b/lib/pdcp/version.map > @@ -6,6 +6,8 @@ EXPERIMENTAL { > rte_pdcp_entity_release; > rte_pdcp_entity_suspend; > > + rte_pdcp_control_pdu_create; > + > rte_pdcp_pkt_post_process; > rte_pdcp_pkt_pre_process; > > -- > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 12/22] pdcp: add control PDU handling 2023-05-18 9:15 ` Akhil Goyal @ 2023-05-22 11:09 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-22 11:09 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz Hi Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Thursday, May 18, 2023 2:45 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Hemant Agrawal <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > dev@dpdk.org; Olivier Matz <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 12/22] pdcp: add control PDU handling > > > Subject: [PATCH v2 12/22] pdcp: add control PDU handling > > > > Add control PDU handling and implement status report generation. > > Status report generation works only when RX_DELIV = RX_NEXT. > > > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > > --- > > app/test/test_pdcp.c | 1 + > > Separate out test app changes from library changes. [Anoob] Agreed. Will make the change in next version. > > > doc/guides/prog_guide/pdcp_lib.rst | 10 +++++++ > > lib/pdcp/meson.build | 2 ++ > > lib/pdcp/pdcp_cnt.c | 29 ++++++++++++++++++ > > lib/pdcp/pdcp_cnt.h | 14 +++++++++ > > lib/pdcp/pdcp_ctrl_pdu.c | 46 +++++++++++++++++++++++++++++ > > lib/pdcp/pdcp_ctrl_pdu.h | 15 ++++++++++ > > lib/pdcp/pdcp_entity.h | 15 ++++++++-- > > lib/pdcp/pdcp_process.c | 13 +++++++++ > > lib/pdcp/rte_pdcp.c | 47 +++++++++++++++++++++++++++++- > > lib/pdcp/rte_pdcp.h | 31 ++++++++++++++++++++ > > lib/pdcp/version.map | 2 ++ > > 12 files changed, 222 insertions(+), 3 deletions(-) create mode > > 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h > > create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 > > lib/pdcp/pdcp_ctrl_pdu.h > > > > diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index > > fc49947ba2..4ecb4d9572 100644 > > --- a/app/test/test_pdcp.c > > +++ b/app/test/test_pdcp.c > > @@ -415,6 +415,7 @@ create_test_conf_from_index(const int index, > > struct pdcp_test_conf *conf) > > > > conf->entity.sess_mpool = ts_params->sess_pool; > > conf->entity.cop_pool = ts_params->cop_pool; > > + conf->entity.ctr_pdu_pool = ts_params->mbuf_pool; > > conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; > > conf->entity.pdcp_xfrm.en_ordering = 0; > > conf->entity.pdcp_xfrm.remove_duplicates = 0; diff --git > > a/doc/guides/prog_guide/pdcp_lib.rst > > b/doc/guides/prog_guide/pdcp_lib.rst > > index abd874f2cc..f23360dfc3 100644 > > --- a/doc/guides/prog_guide/pdcp_lib.rst > > +++ b/doc/guides/prog_guide/pdcp_lib.rst > > @@ -67,6 +67,15 @@ Data PDUs are regular packets submitted by upper > > layers for transmission to other end. Such packets would need to be > > ciphered and authenticated based on the entity configuration. > > > > +PDCP packet processing API for control PDU > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > > +Control PDUs are used in PDCP as a communication channel between > > transmitting > > +and receiving entities. When upper layer request for operations such > > +re-establishment, receiving PDCP entity need to prepare a status > > +report and send it to the other end. The API > > +``pdcp_ctrl_pdu_status_gen`` allows application to request the same. > > pdcp_ctrl_pdu_status_gen() - Is this a user visible API? I believe it is not > exposed. > How can application request that? [Anoob] Quoted wrong API. It should be 'rte_pdcp_control_pdu_create()'. Will have this corrected in next version. > > > > + > > PDCP packet processing API for data PDU > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > > @@ -228,6 +237,7 @@ Supported features > > - Uplink & downlink traffic > > - HFN increment > > - IV generation as required per algorithm > > +- Control PDU generation > > > > Supported ciphering algorithms > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index > > 08679b743a..75d476bf6d 100644 > > --- a/lib/pdcp/meson.build > > +++ b/lib/pdcp/meson.build > > @@ -8,7 +8,9 @@ if is_windows > > endif > > > > sources = files( > > + 'pdcp_cnt.c', > > pdcp_cnt seems to be something related to count. > And it is not related to this patch probably. [Anoob] 'pdcp_cnt' tracks the sequence number of the received packets etc. This info is required for generating status report which is one of the control PDUs supported. > > > 'pdcp_crypto.c', > > + 'pdcp_ctrl_pdu.c', > > 'pdcp_process.c', > > 'rte_pdcp.c', > > ) > > diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c new file mode > > 100644 index 0000000000..c9b952184b > > --- /dev/null > > +++ b/lib/pdcp/pdcp_cnt.c > > @@ -0,0 +1,29 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#include <rte_pdcp.h> > > + > > +#include "pdcp_cnt.h" > > +#include "pdcp_entity.h" > > + > > +int > > +pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct > > rte_pdcp_entity_conf *conf) > > +{ > > + struct entity_priv_dl_part *en_priv_dl; > > + uint32_t window_sz; > > + > > + if (en == NULL || conf == NULL) > > + return -EINVAL; > > + > > + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) > > + return 0; > > + > > + en_priv_dl = entity_dl_part_get(en); > > + window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); > > + > > + RTE_SET_USED(window_sz); > > + RTE_SET_USED(en_priv_dl); > > + > > + return 0; > > +} > > diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h new file mode > > 100644 index 0000000000..bbda478b55 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_cnt.h > > @@ -0,0 +1,14 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#ifndef PDCP_CNT_H > > +#define PDCP_CNT_H > > + > > +#include <rte_common.h> > > + > > +#include "pdcp_entity.h" > > + > > +int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct > > rte_pdcp_entity_conf *conf); > > + > > +#endif /* PDCP_CNT_H */ > > diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c new > > file mode 100644 index 0000000000..feb05fd863 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_ctrl_pdu.c > > @@ -0,0 +1,46 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#include <rte_byteorder.h> > > +#include <rte_mbuf.h> > > +#include <rte_pdcp_hdr.h> > > + > > +#include "pdcp_ctrl_pdu.h" > > +#include "pdcp_entity.h" > > + > > +static __rte_always_inline void > > +pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t > > +rx_deliv) { > > + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; > > + pdu_hdr->pdu_type = > RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; > > + pdu_hdr->r = 0; > > + pdu_hdr->fmc = rte_cpu_to_be_32(rx_deliv); } > > + > > +int > > +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf > > +*m) { > > + struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; > > + uint32_t rx_deliv; > > + int pdu_sz; > > + > > + if (!en_priv->flags.is_status_report_required) > > + return -EINVAL; > > + > > + pdu_sz = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); > > + > > + rx_deliv = en_priv->state.rx_deliv; > > + > > + /* Zero missing PDUs - status report contains only FMC */ > > + if (rx_deliv >= en_priv->state.rx_next) { > > + pdu_hdr = (struct rte_pdcp_up_ctrl_pdu_hdr > > *)rte_pktmbuf_append(m, pdu_sz); > > + if (pdu_hdr == NULL) > > + return -ENOMEM; > > + pdcp_hdr_fill(pdu_hdr, rx_deliv); > > + > > + return 0; > > + } > > + > > + return -ENOTSUP; > > +} > > diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h new > > file mode 100644 index 0000000000..a2424fbd10 > > --- /dev/null > > +++ b/lib/pdcp/pdcp_ctrl_pdu.h > > @@ -0,0 +1,15 @@ > > +/* SPDX-License-Identifier: BSD-3-Clause > > + * Copyright(C) 2023 Marvell. > > + */ > > + > > +#ifndef PDCP_CTRL_PDU_H > > +#define PDCP_CTRL_PDU_H > > + > > +#include <rte_mbuf.h> > > + > > +#include "pdcp_entity.h" > > + > > +int > > +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf > > +*m); > > + > > +#endif /* PDCP_CTRL_PDU_H */ > > diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index > > 3108795977..7b7e7f69dd 100644 > > --- a/lib/pdcp/pdcp_entity.h > > +++ b/lib/pdcp/pdcp_entity.h > > @@ -109,6 +109,13 @@ union cipher_iv_partial { > > uint64_t u64[2]; > > }; > > > > +struct pdcp_cnt_bitmap { > > + /** Number of entries that can be stored. */ > > + uint32_t size; > > + /** Bitmap of the count values already received.*/ > > + struct rte_bitmap *bmp; > > +}; > > This pdcp_cnt_bitmap is not related to control PDU. Right? > I believe the patch split is not proper here. [Anoob] Bitmap is required for generating status report. With this patch, we are adding support for control PDU handling and adding basic support for status report generation. > > > + > > /* > > * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] > > */ > > @@ -136,9 +143,13 @@ struct entity_priv { > > uint64_t is_ul_entity : 1; > > /** Is NULL auth. */ > > uint64_t is_null_auth : 1; > > + /** Is status report required.*/ > > + uint64_t is_status_report_required : 1; > > } flags; > > /** Crypto op pool. */ > > struct rte_mempool *cop_pool; > > + /** Control PDU pool. */ > > + struct rte_mempool *ctr_pdu_pool; > > /** PDCP header size. */ > > uint8_t hdr_sz; > > /** PDCP AAD size. For AES-CMAC, additional message is prepended > for > > the operation. */ @@ -148,8 +159,8 @@ struct entity_priv { }; > > > > struct entity_priv_dl_part { > > - /* NOTE: when in-order-delivery is supported, post PDCP packets > would > > need to cached. */ > > - uint8_t dummy; > > + /** PDCP would need to track the count values that are already > > received.*/ > > + struct pdcp_cnt_bitmap bitmap; > > }; > > > > struct entity_priv_ul_part { > > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index > > 9c1a5e0669..267b3b7723 100644 > > --- a/lib/pdcp/pdcp_process.c > > +++ b/lib/pdcp/pdcp_process.c > > @@ -1157,6 +1157,19 @@ pdcp_entity_priv_populate(struct entity_priv > > *en_priv, const struct rte_pdcp_ent > > if (a_xfrm != NULL && a_xfrm->auth.algo == > > RTE_CRYPTO_AUTH_NULL) > > en_priv->flags.is_null_auth = 1; > > > > + /** > > + * flags.is_status_report_required > > + * > > + * Indicate whether status report is required. > > + */ > > + if (conf->status_report_required) { > > + /** Status report is required only for DL entities. */ > > + if (conf->pdcp_xfrm.pkt_dir != > > RTE_SECURITY_PDCP_DOWNLINK) > > + return -EINVAL; > > + > > + en_priv->flags.is_status_report_required = 1; > > + } > > + > > /** > > * hdr_sz > > * > > diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index > > 8914548dbd..5cd3f5ca31 100644 > > --- a/lib/pdcp/rte_pdcp.c > > +++ b/lib/pdcp/rte_pdcp.c > > @@ -6,7 +6,9 @@ > > #include <rte_pdcp.h> > > #include <rte_malloc.h> > > > > +#include "pdcp_cnt.h" > > #include "pdcp_crypto.h" > > +#include "pdcp_ctrl_pdu.h" > > #include "pdcp_entity.h" > > #include "pdcp_process.h" > > > > @@ -34,7 +36,7 @@ rte_pdcp_entity_establish(const struct > > rte_pdcp_entity_conf *conf) > > struct entity_priv *en_priv; > > int ret, entity_size; > > > > - if (conf == NULL || conf->cop_pool == NULL) { > > + if (conf == NULL || conf->cop_pool == NULL || conf->ctr_pdu_pool > == > > NULL) { > > rte_errno = -EINVAL; > > return NULL; > > } > > @@ -79,6 +81,7 @@ rte_pdcp_entity_establish(const struct > > rte_pdcp_entity_conf *conf) > > en_priv->state.rx_deliv = conf->count; > > en_priv->state.tx_next = conf->count; > > en_priv->cop_pool = conf->cop_pool; > > + en_priv->ctr_pdu_pool = conf->ctr_pdu_pool; > > > > /* Setup crypto session */ > > ret = pdcp_crypto_sess_create(entity, conf); @@ -89,6 +92,10 @@ > > rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) > > if (ret) > > goto crypto_sess_destroy; > > > > + ret = pdcp_cnt_ring_create(entity, conf); > > + if (ret) > > + goto crypto_sess_destroy; > > + > > return entity; > > > > crypto_sess_destroy: > > @@ -136,3 +143,41 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity > > *pdcp_entity, > > > > return 0; > > } > > + > > +struct rte_mbuf * > > +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, > > + enum rte_pdcp_ctrl_pdu_type type) { > > + struct entity_priv *en_priv; > > + struct rte_mbuf *m; > > + int ret; > > + > > + if (pdcp_entity == NULL) { > > + rte_errno = -EINVAL; > > rte_errno should be positive values. > > > + return NULL; > > + } > > + > > + en_priv = entity_priv_get(pdcp_entity); > > + > > + m = rte_pktmbuf_alloc(en_priv->ctr_pdu_pool); > > + if (m == NULL) { > > + rte_errno = -ENOMEM; > > + return NULL; > > + } > > + > > + switch (type) { > > + case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: > > + ret = pdcp_ctrl_pdu_status_gen(en_priv, m); > > + break; > > + default: > > + ret = -ENOTSUP; > > + } > > + > > + if (ret) { > > + rte_pktmbuf_free(m); > > + rte_errno = ret; > > + return NULL; > > + } > > + > > + return m; > > +} > > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index > > 54f88e3fd3..d2db25d7d9 100644 > > --- a/lib/pdcp/rte_pdcp.h > > +++ b/lib/pdcp/rte_pdcp.h > > @@ -16,6 +16,7 @@ > > #include <rte_compat.h> > > #include <rte_common.h> > > #include <rte_mempool.h> > > +#include <rte_pdcp_hdr.h> > > #include <rte_security.h> > > > > #ifdef __cplusplus > > @@ -78,6 +79,8 @@ struct rte_pdcp_entity_conf { > > struct rte_mempool *sess_mpool; > > /** Crypto op pool.*/ > > struct rte_mempool *cop_pool; > > + /** Mbuf pool to be used for allocating control PDUs.*/ > > + struct rte_mempool *ctr_pdu_pool; > > Please use same prefix for all control pdu stuff. > I see cnt, ctr, ctrl, control is being used. > I think "control" for top level API and "ctrl" for parameters should be fine. [Anoob] Agreed. Will make this change in next version. > > > /** > > * 32 bit count value (HFN + SN) to be used for the first packet. > > * pdcp_xfrm.hfn would be ignored as the HFN would be derived > from > > this value. > > @@ -91,6 +94,15 @@ struct rte_pdcp_entity_conf { > > uint8_t dev_id; > > /** Reverse direction during IV generation. Can be used to simulate > > UE crypto processing.*/ > > bool reverse_iv_direction; > > + /** > > + * Status report required (specified in TS 38.331). > > + * > > + * If PDCP entity is configured to send a PDCP status report, the > > +upper > > layer application > > + * may request a receiving PDCP entity to generate a PDCP status > > +report > > using > > + * ``rte_pdcp_ctrl_pdu_create``. In addition, PDCP status reports > > +may be > > generated during > > + * operations such as entity re-establishment. > > + */ > > + bool status_report_required; > > rte_pdcp_ctrl_pdu_create -> rte_pdcp_control_pdu_create > > Please specify that a PDU will be generated for status report. > Otherwise, it seems some structure would be returned for status report But > there is no mention of that. This will avoid confusion. [Anoob] Agreed. Will update documentation of rte_pdcp_control_pdu_create() to denote that it is lib PDCP which would allocate the packet. > > > > }; > > /* >8 End of structure rte_pdcp_entity_conf. */ > > > > @@ -169,6 +181,25 @@ int > > rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, > > struct rte_mbuf *out_mb[]); > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * Create control PDU packet of the `type` specified. > > + * > > + * @param pdcp_entity > > + * Pointer to the PDCP entity for which the control PDU need to be > generated. > > + * @param type > > + * Type of control PDU to be generated. > > + * @return > > + * - Control PDU generated, in case of success. > > + * - NULL in case of failure. rte_errno will be set to error code. > > + */ > > +__rte_experimental > > +struct rte_mbuf * > > +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, > > + enum rte_pdcp_ctrl_pdu_type type); > > + > > /** > > * @warning > > * @b EXPERIMENTAL: this API may change without prior notice diff > > --git a/lib/pdcp/version.map b/lib/pdcp/version.map index > > f9ff30600a..d0cf338e1f 100644 > > --- a/lib/pdcp/version.map > > +++ b/lib/pdcp/version.map > > @@ -6,6 +6,8 @@ EXPERIMENTAL { > > rte_pdcp_entity_release; > > rte_pdcp_entity_suspend; > > > > + rte_pdcp_control_pdu_create; > > + > > rte_pdcp_pkt_post_process; > > rte_pdcp_pkt_pre_process; > > > > -- > > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 13/22] pdcp: implement t-Reordering and packet buffering 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (11 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 12/22] pdcp: add control PDU handling Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 14/22] test/pdcp: add in-order delivery cases Anoob Joseph ` (9 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add in-order delivery of packets in PDCP. Delivery of packets in-order relies on t-Reordering timer. When 'out-of-order delivery' is disabled, PDCP will buffer all received packets that are out of order. The t-Reordering timer determines the time period these packets would be held in the buffer, waiting for any missing packets to arrive. Introduce packet buffering and state variables which indicate status of the timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 3 +- lib/pdcp/pdcp_entity.h | 19 +++++++ lib/pdcp/pdcp_process.c | 123 +++++++++++++++++++++++++++++++--------- lib/pdcp/pdcp_reorder.c | 27 +++++++++ lib/pdcp/pdcp_reorder.h | 62 ++++++++++++++++++++ lib/pdcp/rte_pdcp.c | 53 +++++++++++++++-- lib/pdcp/rte_pdcp.h | 6 +- 7 files changed, 257 insertions(+), 36 deletions(-) create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 75d476bf6d..f4f9246bcb 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -12,9 +12,10 @@ sources = files( 'pdcp_crypto.c', 'pdcp_ctrl_pdu.c', 'pdcp_process.c', + 'pdcp_reorder.c', 'rte_pdcp.c', ) headers = files('rte_pdcp.h') indirect_headers += files('rte_pdcp_group.h') -deps += ['mbuf', 'net', 'cryptodev', 'security'] +deps += ['mbuf', 'net', 'cryptodev', 'security', 'reorder'] diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 7b7e7f69dd..71962d7279 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -11,6 +11,8 @@ #include <rte_pdcp.h> #include <rte_security.h> +#include "pdcp_reorder.h" + struct entity_priv; #define PDCP_HFN_MIN 0 @@ -109,6 +111,17 @@ union cipher_iv_partial { uint64_t u64[2]; }; +enum timer_state { + TIMER_STOP, + TIMER_RUNNING, + TIMER_EXPIRED, +}; + +struct pdcp_t_reordering { + /** Represent timer state */ + enum timer_state state; +}; + struct pdcp_cnt_bitmap { /** Number of entries that can be stored. */ uint32_t size; @@ -145,6 +158,8 @@ struct entity_priv { uint64_t is_null_auth : 1; /** Is status report required.*/ uint64_t is_status_report_required : 1; + /** Is out-of-order delivery enabled */ + uint64_t is_out_of_order_delivery : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -161,6 +176,10 @@ struct entity_priv { struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; + /** t-Reordering handles */ + struct pdcp_t_reordering t_reorder; + /** Reorder packet buffer */ + struct pdcp_reorder reorder; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 267b3b7723..16d22cbe14 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -809,25 +809,88 @@ pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool tr } } -static inline bool +static inline int pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, - const uint32_t count) + const uint32_t count, struct rte_mbuf *mb, + struct rte_mbuf *out_mb[], + const bool trim_mac) { struct entity_priv *en_priv = entity_priv_get(entity); + struct pdcp_t_reordering *t_reorder; + struct pdcp_reorder *reorder; + uint16_t processed = 0; - if (count < en_priv->state.rx_deliv) - return false; + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + const uint32_t hdr_trim_sz = en_priv->hdr_sz + en_priv->aad_sz; - /* t-Reordering timer is not supported - SDU will be delivered immediately. - * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not - * been delivered to upper layers - */ - en_priv->state.rx_next = count + 1; + if (count < en_priv->state.rx_deliv) + return -EINVAL; if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; - return true; + pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); + + if (en_priv->flags.is_out_of_order_delivery) { + out_mb[0] = mb; + en_priv->state.rx_deliv = count + 1; + + return 1; + } + + reorder = &dl->reorder; + t_reorder = &dl->t_reorder; + + if (count == en_priv->state.rx_deliv) { + if (reorder->is_active) { + /* + * This insert used only to increment reorder->min_seqn + * To remove it - min_seqn_set() has to work with non-empty buffer + */ + pdcp_reorder_insert(reorder, mb, count); + + /* Get buffered packets */ + struct rte_mbuf **cached_mbufs = &out_mb[processed]; + uint32_t nb_cached = pdcp_reorder_get_sequential(reorder, + cached_mbufs, entity->max_pkt_cache - processed); + + processed += nb_cached; + } else { + out_mb[processed++] = mb; + } + + /* Processed should never exceed the window size */ + en_priv->state.rx_deliv = count + processed; + + } else { + if (!reorder->is_active) + /* Initialize reordering buffer with RX_DELIV */ + pdcp_reorder_start(reorder, en_priv->state.rx_deliv); + /* Buffer the packet */ + pdcp_reorder_insert(reorder, mb, count); + } + + /* Stop & reset current timer if rx_reord is received */ + if (t_reorder->state == TIMER_RUNNING && + en_priv->state.rx_deliv >= en_priv->state.rx_reord) { + t_reorder->state = TIMER_STOP; + /* Stop reorder buffer, only if it's empty */ + if (en_priv->state.rx_deliv == en_priv->state.rx_next) + pdcp_reorder_stop(reorder); + } + + /* + * If t-Reordering is not running (includes the case when t-Reordering is stopped due to + * actions above). + */ + if (t_reorder->state == TIMER_STOP && en_priv->state.rx_deliv < en_priv->state.rx_next) { + /* Update RX_REORD to RX_NEXT */ + en_priv->state.rx_reord = en_priv->state.rx_next; + /* Start t-Reordering */ + t_reorder->state = TIMER_RUNNING; + } + + return processed; } static inline uint16_t @@ -837,16 +900,14 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, uint16_t num, uint16_t *nb_err_ret, const bool is_integ_protected) { + int i, nb_processed, nb_success = 0, nb_err = 0, rsn = 0; struct entity_priv *en_priv = entity_priv_get(entity); struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; - int i, nb_success = 0, nb_err = 0, rsn = 0; const uint32_t aad_sz = en_priv->aad_sz; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -864,11 +925,12 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, RTE_SECURITY_PDCP_SN_SIZE_12))) goto error; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], is_integ_protected); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -908,14 +970,13 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, const bool is_integ_protected) { struct entity_priv *en_priv = entity_priv_get(entity); + int i, nb_processed, nb_success = 0, nb_err = 0; struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; struct rte_mbuf *mb, *err_mb[num]; int32_t rsn = 0; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; for (i = 0; i < num; i++) { mb = in_mb[i]; @@ -936,11 +997,12 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, RTE_SECURITY_PDCP_SN_SIZE_18))) goto error; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], is_integ_protected); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -979,16 +1041,14 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, uint16_t num, uint16_t *nb_err_ret) { struct entity_priv *en_priv = entity_priv_get(entity); + int i, nb_processed, nb_success = 0, nb_err = 0; struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; int32_t rsn; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -1002,12 +1062,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, RTE_SECURITY_PDCP_SN_SIZE_12))) goto error; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], true); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, true); - - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -1170,6 +1230,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent en_priv->flags.is_status_report_required = 1; } + /** + * flags.is_out_of_order_delivery + * + * Indicate whether the outoforder delivery is enabled for PDCP entity. + */ + en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** * hdr_sz * diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c new file mode 100644 index 0000000000..5399f0dc28 --- /dev/null +++ b/lib/pdcp/pdcp_reorder.c @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_reorder.h> + +#include "pdcp_reorder.h" + +int +pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +{ + reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + if (reorder->buf == NULL) + return -rte_errno; + + reorder->window_size = window_size; + reorder->is_active = false; + + return 0; +} + +void +pdcp_reorder_destroy(const struct pdcp_reorder *reorder) +{ + rte_reorder_free(reorder->buf); +} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h new file mode 100644 index 0000000000..6a2f61d6ae --- /dev/null +++ b/lib/pdcp/pdcp_reorder.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_REORDER_H +#define PDCP_REORDER_H + +#include <rte_reorder.h> + +struct pdcp_reorder { + struct rte_reorder_buffer *buf; + uint32_t window_size; + bool is_active; +}; + +int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); +void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); + +static inline uint32_t +pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs) +{ + return rte_reorder_drain(reorder->buf, mbufs, max_mbufs); +} + +static inline uint32_t +pdcp_reorder_up_to_get(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs, uint32_t seqn) +{ + return rte_reorder_drain_up_to_seqn(reorder->buf, mbufs, max_mbufs, seqn); +} + +static inline void +pdcp_reorder_start(struct pdcp_reorder *reorder, uint32_t min_seqn) +{ + int ret; + + reorder->is_active = true; + + ret = rte_reorder_min_seqn_set(reorder->buf, min_seqn); + RTE_VERIFY(ret == 0); +} + +static inline void +pdcp_reorder_stop(struct pdcp_reorder *reorder) +{ + reorder->is_active = false; +} + +static inline void +pdcp_reorder_insert(struct pdcp_reorder *reorder, struct rte_mbuf *mbuf, + rte_reorder_seqn_t pkt_count) +{ + int ret; + + *rte_reorder_seqn(mbuf) = pkt_count; + + ret = rte_reorder_insert(reorder->buf, mbuf); + RTE_VERIFY(ret == 0); +} + +#endif /* PDCP_REORDER_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 5cd3f5ca31..c9d44ca539 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -29,6 +29,17 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } +static int +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + + entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + + return pdcp_reorder_create(&dl->reorder, window_size); +} + struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { @@ -92,6 +103,12 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + ret = pdcp_dl_establish(entity, conf); + if (ret) + goto crypto_sess_destroy; + } + ret = pdcp_cnt_ring_create(entity, conf); if (ret) goto crypto_sess_destroy; @@ -106,26 +123,50 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } +static int +pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + int nb_out; + + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, + en_priv->state.rx_next); + + pdcp_reorder_destroy(&dl->reorder); + + return nb_out; +} + int rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv *en_priv; + int nb_out = 0; + if (pdcp_entity == NULL) return -EINVAL; + en_priv = entity_priv_get(pdcp_entity); + + if (!en_priv->flags.is_ul_entity) + nb_out = pdcp_dl_release(pdcp_entity, out_mb); + /* Teardown crypto sessions */ pdcp_crypto_sess_destroy(pdcp_entity); rte_free(pdcp_entity); - RTE_SET_USED(out_mb); - return 0; + return nb_out; } int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; + int nb_out = 0; if (pdcp_entity == NULL) return -EINVAL; @@ -135,13 +176,15 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, if (en_priv->flags.is_ul_entity) { en_priv->state.tx_next = 0; } else { + dl = entity_dl_part_get(pdcp_entity); + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, pdcp_entity->max_pkt_cache, + en_priv->state.rx_next); + pdcp_reorder_stop(&dl->reorder); en_priv->state.rx_next = 0; en_priv->state.rx_deliv = 0; } - RTE_SET_USED(out_mb); - - return 0; + return nb_out; } struct rte_mbuf * diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index d2db25d7d9..0d2f4fe7c1 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -103,6 +103,8 @@ struct rte_pdcp_entity_conf { * operations such as entity re-establishment. */ bool status_report_required; + /** Enable out of order delivery. */ + bool out_of_order_delivery; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -259,8 +261,8 @@ rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, * @param in_mb * The address of an array of *num* pointers to *rte_mbuf* structures. * @param[out] out_mb - * The address of an array of *num* pointers to *rte_mbuf* structures - * to output packets after PDCP post-processing. + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures to output packets after PDCP post-processing. * @param num * The maximum number of packets to process. * @param[out] nb_err -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 14/22] test/pdcp: add in-order delivery cases 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (12 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 15/22] pdcp: add timer callback handlers Anoob Joseph ` (8 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add test cases to verify behaviour when in-order delivery is enabled and packets arrive in out-of-order. PDCP library is expected to buffer the packets and return packets in-order when the missing packet arrives. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 195 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 195 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 4ecb4d9572..c96ddc4f53 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -15,6 +15,15 @@ #define CDEV_INVALID_ID UINT8_MAX #define NB_TESTS RTE_DIM(pdcp_test_params) +/* Assert that condition is true, or goto the mark */ +#define ASSERT_TRUE_OR_GOTO(cond, mark, ...) do {\ + if (!(cond)) { \ + RTE_LOG(ERR, USER1, "Error at: %s:%d\n", __func__, __LINE__); \ + RTE_LOG(ERR, USER1, __VA_ARGS__); \ + goto mark; \ + } \ +} while (0) + /* According to formula(7.2.a Window_Size) */ #define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) @@ -82,6 +91,14 @@ run_test_with_all_known_vec(const void *args) return run_test_foreach_known_vec(test, false); } +static int +run_test_with_all_known_vec_until_first_pass(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, true); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -868,6 +885,7 @@ test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) /* Configure Uplink to generate expected, encrypted packet */ pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.out_of_order_delivery = true; conf->entity.reverse_iv_direction = true; conf->entity.count = PDCP_SET_COUNT(new_hfn, new_sn, sn_size); conf->output_len = 0; @@ -913,6 +931,168 @@ test_sn_minus_outside(struct pdcp_test_conf *t_conf) return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); } +static struct rte_mbuf * +generate_packet_for_dl_with_sn(struct pdcp_test_conf ul_conf, uint32_t sn) +{ + int ret; + + ul_conf.entity.count = sn; + ul_conf.entity.out_of_order_delivery = true; + ul_conf.entity.reverse_iv_direction = true; + ul_conf.output_len = 0; + + ret = test_attempt_single(&ul_conf); + if (ret != TEST_SUCCESS) + return NULL; + + return mbuf_from_data_create(ul_conf.output, ul_conf.output_len); +} + +static bool +array_asc_sorted_check(struct rte_mbuf *m[], uint32_t len, enum rte_security_pdcp_sn_size sn_size) +{ + uint32_t i; + + if (len < 2) + return true; + + for (i = 0; i < (len - 1); i++) { + if (pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i], void *), sn_size) > + pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i + 1], void *), sn_size)) + return false; + } + + return true; +} + +static int +test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) +{ + struct rte_mbuf *m0 = NULL, *m1 = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.count = start_count; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + const uint32_t sn_size = dl_conf.entity.pdcp_xfrm.sn_size; + cdev_id = dl_conf.entity.dev_id; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Generate packet to fill the existing gap */ + m0 = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m0 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1] + * Gap filled, all packets should be returned + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m0, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m0 = NULL; /* Packet was moved to out_mb */ + + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, + "Error occurred during packet drain\n"); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m0); + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + +static int +test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) +{ + struct rte_mbuf *m1 = NULL, **out_mb = NULL; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + const int rx_deliv = 0; + int ret = TEST_FAILED; + size_t i, nb_out; + uint8_t cdev_id; + + const uint32_t sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.count = rx_deliv; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + ASSERT_TRUE_OR_GOTO(pdcp_entity->max_pkt_cache >= window_size, exit, + "PDCP max packet cache is too small"); + cdev_id = dl_conf.entity.dev_id; + out_mb = rte_zmalloc(NULL, pdcp_entity->max_pkt_cache * sizeof(uintptr_t), 0); + ASSERT_TRUE_OR_GOTO(out_mb != NULL, exit, + "Could not allocate buffer for holding out_mb buffers\n"); + + /* Send packets with SN > RX_DELIV to create a gap */ + for (i = rx_deliv + 1; i < window_size; i++) { + m1 = generate_packet_for_dl_with_sn(*ul_conf, i); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + } + + m1 = generate_packet_for_dl_with_sn(*ul_conf, rx_deliv); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Insert missing packet */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == window_size, exit, + "Packet count mismatch (received: %i, expected: %i)\n", + nb_success, window_size); + m1 = NULL; + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + rte_free(out_mb); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -969,10 +1149,25 @@ static struct unit_test_suite hfn_sn_test_cases = { } }; +static struct unit_test_suite reorder_test_cases = { + .suite_name = "PDCP reorder", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_reorder_buffer_full_window_size_sn_12), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, + &reorder_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 15/22] pdcp: add timer callback handlers 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (13 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 14/22] test/pdcp: add in-order delivery cases Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-05-18 9:37 ` Akhil Goyal 2023-04-14 17:45 ` [PATCH v2 16/22] pdcp: add timer expiry handle Anoob Joseph ` (7 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> PDCP has a windowing mechanism which allows only packets that fall in a reception window. The pivot point for this window is RX_REORD which happens to be the first missing or next expected packet. If the missing packet is not received after a specified time, then the RX_REORD state variable needs to be moved up to slide the reception window. PDCP relies on timers for such operations. The timer needs to be armed when PDCP library doesn't receive all packets in-order and starts buffering packets that arrived after a missing packet. The timer needs to be cancelled when a missing packet is received. To avoid dependency on particular timer implementation, PDCP library allows application to register two callbacks, timer_start() and timer_stop() that will be called later by library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 ++ lib/pdcp/pdcp_process.c | 2 ++ lib/pdcp/rte_pdcp.c | 1 + lib/pdcp/rte_pdcp.h | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 52 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 71962d7279..ca98a1d0f7 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -120,6 +120,8 @@ enum timer_state { struct pdcp_t_reordering { /** Represent timer state */ enum timer_state state; + /** User defined callback handles */ + struct rte_pdcp_t_reordering handle; }; struct pdcp_cnt_bitmap { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 16d22cbe14..9ba07d5255 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -874,6 +874,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (t_reorder->state == TIMER_RUNNING && en_priv->state.rx_deliv >= en_priv->state.rx_reord) { t_reorder->state = TIMER_STOP; + t_reorder->handle.stop(t_reorder->handle.timer, t_reorder->handle.args); /* Stop reorder buffer, only if it's empty */ if (en_priv->state.rx_deliv == en_priv->state.rx_next) pdcp_reorder_stop(reorder); @@ -888,6 +889,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, en_priv->state.rx_reord = en_priv->state.rx_next; /* Start t-Reordering */ t_reorder->state = TIMER_RUNNING; + t_reorder->handle.start(t_reorder->handle.timer, t_reorder->handle.args); } return processed; diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index c9d44ca539..755d592578 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -36,6 +36,7 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct entity_priv_dl_part *dl = entity_dl_part_get(entity); entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + dl->t_reorder.handle = conf->t_reordering; return pdcp_reorder_create(&dl->reorder, window_size); } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 0d2f4fe7c1..55e57c3b9b 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -66,6 +66,51 @@ struct rte_pdcp_entity { uint64_t user_area[2]; } __rte_cache_aligned; +/** + * Callback function type for t-Reordering timer start, set during PDCP entity establish. + * This callback is invoked by PDCP library, during t-Reordering timer start event. + * Only one t-Reordering per receiving PDCP entity would be running at a given time. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_start_cb_t)(void *timer, void *args); + +/** + * Callback function type for t-Reordering timer stop, set during PDCP entity establish. + * This callback will be invoked by PDCP library, during t-Reordering timer stop event. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); + +/** + * PDCP t-Reordering timer interface + * + * Configuration provided by user, that PDCP library will invoke according to timer behaviour. + */ +struct rte_pdcp_t_reordering { + /** Timer pointer, stored for later use in callback functions */ + void *timer; + /** Timer arguments, stored for later use in callback functions */ + void *args; + /** Timer start callback handle */ + rte_pdcp_t_reordering_start_cb_t start; + /** Timer start callback handle */ + rte_pdcp_t_reordering_stop_cb_t stop; +}; + /** * PDCP entity configuration to be used for establishing an entity. */ @@ -105,6 +150,8 @@ struct rte_pdcp_entity_conf { bool status_report_required; /** Enable out of order delivery. */ bool out_of_order_delivery; + /** t-Reordering timer configuration */ + struct rte_pdcp_t_reordering t_reordering; }; /* >8 End of structure rte_pdcp_entity_conf. */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 15/22] pdcp: add timer callback handlers 2023-04-14 17:45 ` [PATCH v2 15/22] pdcp: add timer callback handlers Anoob Joseph @ 2023-05-18 9:37 ` Akhil Goyal 0 siblings, 0 replies; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 9:37 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz > +struct rte_pdcp_t_reordering { > + /** Timer pointer, stored for later use in callback functions */ > + void *timer; > + /** Timer arguments, stored for later use in callback functions */ > + void *args; > + /** Timer start callback handle */ > + rte_pdcp_t_reordering_start_cb_t start; > + /** Timer start callback handle */ start->stop > + rte_pdcp_t_reordering_stop_cb_t stop; > +}; > + ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 16/22] pdcp: add timer expiry handle 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (14 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 15/22] pdcp: add timer callback handlers Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-05-18 9:43 ` Akhil Goyal 2023-04-14 17:45 ` [PATCH v2 17/22] test/pdcp: add timer expiry cases Anoob Joseph ` (6 subsequent siblings) 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> The PDCP protocol requires usage of timers to keep track of how long an out-of-order packet should be buffered while waiting for missing packets. Applications can register a desired timer implementation with the PDCP library. Once the timer expires, the application will be notified, and further handling of the event will be performed in the PDCP library. When the timer expires, the PDCP library will return the cached packets, and PDCP internal state variables (like RX_REORD, RX_DELIV etc) will be updated accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 29 ++++++++++++++++++ lib/pdcp/pdcp_process.c | 49 ++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 31 +++++++++++++++++++ lib/pdcp/version.map | 2 ++ 4 files changed, 111 insertions(+) diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index f23360dfc3..32370633e5 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -229,6 +229,35 @@ parameters for entity creation. } } +Timers +------ + +PDCP utilizes a reception window mechanism to limit the bits of COUNT value +transmitted in the packet. It utilizes state variables such as RX_REORD, +RX_DELIV to define the window and uses RX_DELIV as the lower pivot point of the +window. + +RX_DELIV would be updated only when packets are received in-order. Any missing +packet would mean RX_DELIV won't be updated. A timer, t-Reordering, helps PDCP +to slide the window if the missing packet is not received in a specified time +duration. + +While starting and stopping the timer need to be done by lib PDCP, application +could register its own timer implementation. This is to make sure application +can choose between timers such as rte_timer and rte_event based timers. Starting +and stopping of timer would happen during pre & post process API. + +When the t-Reordering timer expires, application would receive the expiry event. +To perform the PDCP handling of the expiry event, +``rte_pdcp_t_reordering_expiry_handle`` can be used. Expiry handling would +involve sliding the window by updating state variables and passing the expired +packets to the application. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_t_reordering 8< + :end-before: >8 End of structure rte_pdcp_t_reordering. + Supported features ------------------ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 9ba07d5255..91b87a2a81 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -1285,3 +1285,52 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti return 0; } + +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t capacity = entity->max_pkt_cache; + uint16_t nb_out, nb_seq; + + /* 5.2.2.2 Actions when a t-Reordering expires */ + + /* + * - deliver to upper layers in ascending order of the associated COUNT value after + * performing header decompression, if not decompressed before: + */ + + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); + capacity -= nb_out; + out_mb = &out_mb[nb_out]; + + /* + * - all stored PDCP SDU(s) with consecutively associated COUNT value(s) starting from + * RX_REORD; + */ + nb_seq = pdcp_reorder_get_sequential(&dl->reorder, out_mb, capacity); + nb_out += nb_seq; + + /* + * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered + * to upper layers, with COUNT value >= RX_REORD; + */ + en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + + /* + * - if RX_DELIV < RX_NEXT: + * - update RX_REORD to RX_NEXT; + * - start t-Reordering. + */ + if (en_priv->state.rx_deliv < en_priv->state.rx_next) { + en_priv->state.rx_reord = en_priv->state.rx_next; + dl->t_reorder.state = TIMER_RUNNING; + dl->t_reorder.handle.start(dl->t_reorder.handle.timer, dl->t_reorder.handle.args); + } else { + dl->t_reorder.state = TIMER_EXPIRED; + } + + return nb_out; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 55e57c3b9b..c077acce63 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -100,6 +100,7 @@ typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); * * Configuration provided by user, that PDCP library will invoke according to timer behaviour. */ +/* Structure rte_pdcp_t_reordering 8< */ struct rte_pdcp_t_reordering { /** Timer pointer, stored for later use in callback functions */ void *timer; @@ -110,6 +111,7 @@ struct rte_pdcp_t_reordering { /** Timer start callback handle */ rte_pdcp_t_reordering_stop_cb_t stop; }; +/* >8 End of structure rte_pdcp_t_reordering. */ /** * PDCP entity configuration to be used for establishing an entity. @@ -327,6 +329,35 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.2.2.2 Actions when a t-Reordering expires + * + * When t-Reordering timer expires, PDCP is required to slide the reception + * window by updating state variables such as RX_REORD & RX_DELIV. PDCP would + * need to deliver some of the buffered packets based on the state variables and + * conditions described. + * + * The expiry handle need to be invoked by the application when t-Reordering + * timer expires. In addition to returning buffered packets, it may also restart + * timer based on the state variables. + * + * @param entity + * Pointer to the *rte_pdcp_entity* for which the timer expired. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. Used to return buffered packets that are + * expired. + * @return + * Number of packets returned in *out_mb* buffer. + */ +__rte_experimental +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, + struct rte_mbuf *out_mb[]); + #include <rte_pdcp_group.h> #ifdef __cplusplus diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index d0cf338e1f..89b0463be6 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -11,5 +11,7 @@ EXPERIMENTAL { rte_pdcp_pkt_post_process; rte_pdcp_pkt_pre_process; + rte_pdcp_t_reordering_expiry_handle; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 16/22] pdcp: add timer expiry handle 2023-04-14 17:45 ` [PATCH v2 16/22] pdcp: add timer expiry handle Anoob Joseph @ 2023-05-18 9:43 ` Akhil Goyal 2023-05-22 11:34 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Akhil Goyal @ 2023-05-18 9:43 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz > Subject: [PATCH v2 16/22] pdcp: add timer expiry handle > > From: Volodymyr Fialko <vfialko@marvell.com> > > The PDCP protocol requires usage of timers to keep track of how long > an out-of-order packet should be buffered while waiting for missing > packets. Applications can register a desired timer implementation with the > PDCP library. Once the timer expires, the application will be notified, and > further handling of the event will be performed in the PDCP library. > > When the timer expires, the PDCP library will return the cached packets, > and PDCP internal state variables (like RX_REORD, RX_DELIV etc) will be > updated accordingly. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > --- > doc/guides/prog_guide/pdcp_lib.rst | 29 ++++++++++++++++++ > lib/pdcp/pdcp_process.c | 49 ++++++++++++++++++++++++++++++ > lib/pdcp/rte_pdcp.h | 31 +++++++++++++++++++ > lib/pdcp/version.map | 2 ++ > 4 files changed, 111 insertions(+) > > diff --git a/doc/guides/prog_guide/pdcp_lib.rst > b/doc/guides/prog_guide/pdcp_lib.rst > index f23360dfc3..32370633e5 100644 > --- a/doc/guides/prog_guide/pdcp_lib.rst > +++ b/doc/guides/prog_guide/pdcp_lib.rst > @@ -229,6 +229,35 @@ parameters for entity creation. > } > } > > +Timers > +------ > + > +PDCP utilizes a reception window mechanism to limit the bits of COUNT value > +transmitted in the packet. It utilizes state variables such as RX_REORD, > +RX_DELIV to define the window and uses RX_DELIV as the lower pivot point of > the > +window. > + > +RX_DELIV would be updated only when packets are received in-order. Any > missing > +packet would mean RX_DELIV won't be updated. A timer, t-Reordering, helps > PDCP > +to slide the window if the missing packet is not received in a specified time > +duration. > + > +While starting and stopping the timer need to be done by lib PDCP, application > +could register its own timer implementation. This is to make sure application > +can choose between timers such as rte_timer and rte_event based timers. > Starting > +and stopping of timer would happen during pre & post process API. > + > +When the t-Reordering timer expires, application would receive the expiry > event. > +To perform the PDCP handling of the expiry event, > +``rte_pdcp_t_reordering_expiry_handle`` can be used. Expiry handling would > +involve sliding the window by updating state variables and passing the expired > +packets to the application. > + > +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h > + :language: c > + :start-after: Structure rte_pdcp_t_reordering 8< > + :end-before: >8 End of structure rte_pdcp_t_reordering. > + > > Supported features > ------------------ > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c > index 9ba07d5255..91b87a2a81 100644 > --- a/lib/pdcp/pdcp_process.c > +++ b/lib/pdcp/pdcp_process.c > @@ -1285,3 +1285,52 @@ pdcp_process_func_set(struct rte_pdcp_entity > *entity, const struct rte_pdcp_enti > > return 0; > } > + > +uint16_t > +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, > struct rte_mbuf *out_mb[]) Move this public API implementation to rte_pdcp.c > +{ > + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); > + struct entity_priv *en_priv = entity_priv_get(entity); > + uint16_t capacity = entity->max_pkt_cache; > + uint16_t nb_out, nb_seq; > + > + /* 5.2.2.2 Actions when a t-Reordering expires */ > + > + /* > + * - deliver to upper layers in ascending order of the associated COUNT > value after > + * performing header decompression, if not decompressed before: > + */ > + > + /* - all stored PDCP SDU(s) with associated COUNT value(s) < > RX_REORD; */ > + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, > en_priv->state.rx_reord); > + capacity -= nb_out; > + out_mb = &out_mb[nb_out]; > + > + /* > + * - all stored PDCP SDU(s) with consecutively associated COUNT > value(s) starting from > + * RX_REORD; > + */ > + nb_seq = pdcp_reorder_get_sequential(&dl->reorder, out_mb, > capacity); > + nb_out += nb_seq; > + > + /* > + * - update RX_DELIV to the COUNT value of the first PDCP SDU which > has not been delivered > + * to upper layers, with COUNT value >= RX_REORD; > + */ > + en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; > + > + /* > + * - if RX_DELIV < RX_NEXT: > + * - update RX_REORD to RX_NEXT; > + * - start t-Reordering. > + */ > + if (en_priv->state.rx_deliv < en_priv->state.rx_next) { > + en_priv->state.rx_reord = en_priv->state.rx_next; > + dl->t_reorder.state = TIMER_RUNNING; > + dl->t_reorder.handle.start(dl->t_reorder.handle.timer, dl- > >t_reorder.handle.args); > + } else { > + dl->t_reorder.state = TIMER_EXPIRED; > + } > + > + return nb_out; > +} > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h > index 55e57c3b9b..c077acce63 100644 > --- a/lib/pdcp/rte_pdcp.h > +++ b/lib/pdcp/rte_pdcp.h > @@ -100,6 +100,7 @@ typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void > *timer, void *args); > * > * Configuration provided by user, that PDCP library will invoke according to > timer behaviour. > */ > +/* Structure rte_pdcp_t_reordering 8< */ > struct rte_pdcp_t_reordering { > /** Timer pointer, stored for later use in callback functions */ > void *timer; > @@ -110,6 +111,7 @@ struct rte_pdcp_t_reordering { > /** Timer start callback handle */ > rte_pdcp_t_reordering_stop_cb_t stop; > }; > +/* >8 End of structure rte_pdcp_t_reordering. */ > > /** > * PDCP entity configuration to be used for establishing an entity. > @@ -327,6 +329,35 @@ rte_pdcp_pkt_post_process(const struct > rte_pdcp_entity *entity, > return entity->post_process(entity, in_mb, out_mb, num, nb_err); > } > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * 5.2.2.2 Actions when a t-Reordering expires > + * > + * When t-Reordering timer expires, PDCP is required to slide the reception > + * window by updating state variables such as RX_REORD & RX_DELIV. PDCP > would > + * need to deliver some of the buffered packets based on the state variables > and > + * conditions described. > + * > + * The expiry handle need to be invoked by the application when t-Reordering > + * timer expires. In addition to returning buffered packets, it may also restart > + * timer based on the state variables. > + * > + * @param entity > + * Pointer to the *rte_pdcp_entity* for which the timer expired. > + * @param[out] out_mb > + * The address of an array that can hold up to > *rte_pdcp_entity.max_pkt_cache* > + * pointers to *rte_mbuf* structures. Used to return buffered packets that are > + * expired. > + * @return > + * Number of packets returned in *out_mb* buffer. > + */ > +__rte_experimental > +uint16_t > +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *out_mb[]); > + > #include <rte_pdcp_group.h> > > #ifdef __cplusplus > diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map > index d0cf338e1f..89b0463be6 100644 > --- a/lib/pdcp/version.map > +++ b/lib/pdcp/version.map > @@ -11,5 +11,7 @@ EXPERIMENTAL { > rte_pdcp_pkt_post_process; > rte_pdcp_pkt_pre_process; > > + rte_pdcp_t_reordering_expiry_handle; > + > local: *; > }; > -- > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v2 16/22] pdcp: add timer expiry handle 2023-05-18 9:43 ` Akhil Goyal @ 2023-05-22 11:34 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-22 11:34 UTC (permalink / raw) To: Akhil Goyal, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz Hi Akhil, Please see inline. Thanks, Anoob > -----Original Message----- > From: Akhil Goyal <gakhil@marvell.com> > Sent: Thursday, May 18, 2023 3:13 PM > To: Anoob Joseph <anoobj@marvell.com>; Thomas Monjalon > <thomas@monjalon.net>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com> > Cc: Volodymyr Fialko <vfialko@marvell.com>; Hemant Agrawal > <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; dev@dpdk.org; Olivier Matz > <olivier.matz@6wind.com> > Subject: RE: [PATCH v2 16/22] pdcp: add timer expiry handle > > > Subject: [PATCH v2 16/22] pdcp: add timer expiry handle > > > > From: Volodymyr Fialko <vfialko@marvell.com> > > > > The PDCP protocol requires usage of timers to keep track of how long > > an out-of-order packet should be buffered while waiting for missing > > packets. Applications can register a desired timer implementation with > > the PDCP library. Once the timer expires, the application will be > > notified, and further handling of the event will be performed in the PDCP > library. > > > > When the timer expires, the PDCP library will return the cached > > packets, and PDCP internal state variables (like RX_REORD, RX_DELIV > > etc) will be updated accordingly. > > > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > > --- > > doc/guides/prog_guide/pdcp_lib.rst | 29 ++++++++++++++++++ > > lib/pdcp/pdcp_process.c | 49 ++++++++++++++++++++++++++++++ > > lib/pdcp/rte_pdcp.h | 31 +++++++++++++++++++ > > lib/pdcp/version.map | 2 ++ > > 4 files changed, 111 insertions(+) > > > > diff --git a/doc/guides/prog_guide/pdcp_lib.rst > > b/doc/guides/prog_guide/pdcp_lib.rst > > index f23360dfc3..32370633e5 100644 > > --- a/doc/guides/prog_guide/pdcp_lib.rst > > +++ b/doc/guides/prog_guide/pdcp_lib.rst > > @@ -229,6 +229,35 @@ parameters for entity creation. > > } > > } > > > > +Timers > > +------ > > + > > +PDCP utilizes a reception window mechanism to limit the bits of COUNT > > +value transmitted in the packet. It utilizes state variables such as > > +RX_REORD, RX_DELIV to define the window and uses RX_DELIV as the > > +lower pivot point of > > the > > +window. > > + > > +RX_DELIV would be updated only when packets are received in-order. > > +Any > > missing > > +packet would mean RX_DELIV won't be updated. A timer, t-Reordering, > > +helps > > PDCP > > +to slide the window if the missing packet is not received in a > > +specified time duration. > > + > > +While starting and stopping the timer need to be done by lib PDCP, > > +application could register its own timer implementation. This is to > > +make sure application can choose between timers such as rte_timer and > rte_event based timers. > > Starting > > +and stopping of timer would happen during pre & post process API. > > + > > +When the t-Reordering timer expires, application would receive the > > +expiry > > event. > > +To perform the PDCP handling of the expiry event, > > +``rte_pdcp_t_reordering_expiry_handle`` can be used. Expiry handling > > +would involve sliding the window by updating state variables and > > +passing the expired packets to the application. > > + > > +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h > > + :language: c > > + :start-after: Structure rte_pdcp_t_reordering 8< > > + :end-before: >8 End of structure rte_pdcp_t_reordering. > > + > > > > Supported features > > ------------------ > > diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index > > 9ba07d5255..91b87a2a81 100644 > > --- a/lib/pdcp/pdcp_process.c > > +++ b/lib/pdcp/pdcp_process.c > > @@ -1285,3 +1285,52 @@ pdcp_process_func_set(struct rte_pdcp_entity > > *entity, const struct rte_pdcp_enti > > > > return 0; > > } > > + > > +uint16_t > > +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity > > +*entity, > > struct rte_mbuf *out_mb[]) > > Move this public API implementation to rte_pdcp.c [Anoob] Agreed. Will make this change in next version. > > > > +{ > > + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + uint16_t capacity = entity->max_pkt_cache; > > + uint16_t nb_out, nb_seq; > > + > > + /* 5.2.2.2 Actions when a t-Reordering expires */ > > + > > + /* > > + * - deliver to upper layers in ascending order of the associated > > +COUNT > > value after > > + * performing header decompression, if not decompressed before: > > + */ > > + > > + /* - all stored PDCP SDU(s) with associated COUNT value(s) < > > RX_REORD; */ > > + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, > > en_priv->state.rx_reord); > > + capacity -= nb_out; > > + out_mb = &out_mb[nb_out]; > > + > > + /* > > + * - all stored PDCP SDU(s) with consecutively associated COUNT > > value(s) starting from > > + * RX_REORD; > > + */ > > + nb_seq = pdcp_reorder_get_sequential(&dl->reorder, out_mb, > > capacity); > > + nb_out += nb_seq; > > + > > + /* > > + * - update RX_DELIV to the COUNT value of the first PDCP SDU > which > > has not been delivered > > + * to upper layers, with COUNT value >= RX_REORD; > > + */ > > + en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; > > + > > + /* > > + * - if RX_DELIV < RX_NEXT: > > + * - update RX_REORD to RX_NEXT; > > + * - start t-Reordering. > > + */ > > + if (en_priv->state.rx_deliv < en_priv->state.rx_next) { > > + en_priv->state.rx_reord = en_priv->state.rx_next; > > + dl->t_reorder.state = TIMER_RUNNING; > > + dl->t_reorder.handle.start(dl->t_reorder.handle.timer, dl- > > >t_reorder.handle.args); > > + } else { > > + dl->t_reorder.state = TIMER_EXPIRED; > > + } > > + > > + return nb_out; > > +} > > diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index > > 55e57c3b9b..c077acce63 100644 > > --- a/lib/pdcp/rte_pdcp.h > > +++ b/lib/pdcp/rte_pdcp.h > > @@ -100,6 +100,7 @@ typedef void > > (*rte_pdcp_t_reordering_stop_cb_t)(void > > *timer, void *args); > > * > > * Configuration provided by user, that PDCP library will invoke > > according to timer behaviour. > > */ > > +/* Structure rte_pdcp_t_reordering 8< */ > > struct rte_pdcp_t_reordering { > > /** Timer pointer, stored for later use in callback functions */ > > void *timer; > > @@ -110,6 +111,7 @@ struct rte_pdcp_t_reordering { > > /** Timer start callback handle */ > > rte_pdcp_t_reordering_stop_cb_t stop; }; > > +/* >8 End of structure rte_pdcp_t_reordering. */ > > > > /** > > * PDCP entity configuration to be used for establishing an entity. > > @@ -327,6 +329,35 @@ rte_pdcp_pkt_post_process(const struct > > rte_pdcp_entity *entity, > > return entity->post_process(entity, in_mb, out_mb, num, nb_err); } > > > > +/** > > + * @warning > > + * @b EXPERIMENTAL: this API may change without prior notice > > + * > > + * 5.2.2.2 Actions when a t-Reordering expires > > + * > > + * When t-Reordering timer expires, PDCP is required to slide the > > +reception > > + * window by updating state variables such as RX_REORD & RX_DELIV. > > +PDCP > > would > > + * need to deliver some of the buffered packets based on the state > > + variables > > and > > + * conditions described. > > + * > > + * The expiry handle need to be invoked by the application when > > + t-Reordering > > + * timer expires. In addition to returning buffered packets, it may > > + also restart > > + * timer based on the state variables. > > + * > > + * @param entity > > + * Pointer to the *rte_pdcp_entity* for which the timer expired. > > + * @param[out] out_mb > > + * The address of an array that can hold up to > > *rte_pdcp_entity.max_pkt_cache* > > + * pointers to *rte_mbuf* structures. Used to return buffered packets > that are > > + * expired. > > + * @return > > + * Number of packets returned in *out_mb* buffer. > > + */ > > +__rte_experimental > > +uint16_t > > +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity > *entity, > > + struct rte_mbuf *out_mb[]); > > + > > #include <rte_pdcp_group.h> > > > > #ifdef __cplusplus > > diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index > > d0cf338e1f..89b0463be6 100644 > > --- a/lib/pdcp/version.map > > +++ b/lib/pdcp/version.map > > @@ -11,5 +11,7 @@ EXPERIMENTAL { > > rte_pdcp_pkt_post_process; > > rte_pdcp_pkt_pre_process; > > > > + rte_pdcp_t_reordering_expiry_handle; > > + > > local: *; > > }; > > -- > > 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 17/22] test/pdcp: add timer expiry cases 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (15 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 16/22] pdcp: add timer expiry handle Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 18/22] test/pdcp: add timer restart case Anoob Joseph ` (5 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add test cases for handling the expiry with rte_timer and rte_event_timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 325 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 325 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index c96ddc4f53..a0e982777d 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -3,15 +3,22 @@ */ #include <rte_errno.h> +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> #include <rte_malloc.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include <rte_timer.h> #include "test.h" #include "test_cryptodev.h" #include "test_cryptodev_security_pdcp_test_vectors.h" +#define NSECPERSEC 1E9 #define NB_DESC 1024 +#define TIMER_ADAPTER_ID 0 +#define TEST_EV_QUEUE_ID 0 +#define TEST_EV_PORT_ID 0 #define CDEV_INVALID_ID UINT8_MAX #define NB_TESTS RTE_DIM(pdcp_test_params) @@ -32,10 +39,19 @@ struct pdcp_testsuite_params { struct rte_mempool *cop_pool; struct rte_mempool *sess_pool; bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; + int evdev; + struct rte_event_timer_adapter *timdev; + bool timer_is_running; + uint64_t min_resolution_ns; }; static struct pdcp_testsuite_params testsuite_params; +struct test_rte_timer_args { + int status; + struct rte_pdcp_entity *pdcp_entity; +}; + struct pdcp_test_conf { struct rte_pdcp_entity_conf entity; struct rte_crypto_sym_xform c_xfrm; @@ -99,6 +115,30 @@ run_test_with_all_known_vec_until_first_pass(const void *args) return run_test_foreach_known_vec(test, true); } +static void +pdcp_timer_start_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = true; +} + +static void +pdcp_timer_stop_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = false; +} + +static struct rte_pdcp_t_reordering t_reorder_timer = { + .timer = &testsuite_params.timer_is_running, + .start = pdcp_timer_start_cb, + .stop = pdcp_timer_stop_cb, +}; + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -437,6 +477,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + conf->entity.t_reordering = t_reorder_timer; if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; @@ -1018,6 +1059,8 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) /* Check that packets in correct order */ ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == false, exit, + "Timer should be stopped after full drain\n"); ret = TEST_SUCCESS; exit: @@ -1093,6 +1136,170 @@ test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) return ret; } +static void +event_timer_start_cb(void *timer, void *args) +{ + struct rte_event_timer *evtims = args; + int ret = 0; + + ret = rte_event_timer_arm_burst(timer, &evtims, 1); + assert(ret == 1); +} + +static int +test_expiry_with_event_timer(const struct pdcp_test_conf *ul_conf) +{ + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_event event; + + const int start_count = 0; + struct rte_event_timer evtim = { + .ev.op = RTE_EVENT_OP_NEW, + .ev.queue_id = TEST_EV_QUEUE_ID, + .ev.sched_type = RTE_SCHED_TYPE_ATOMIC, + .ev.priority = RTE_EVENT_DEV_PRIORITY_NORMAL, + .ev.event_type = RTE_EVENT_TYPE_TIMER, + .state = RTE_EVENT_TIMER_NOT_ARMED, + .timeout_ticks = 1, + }; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.count = start_count; + dl_conf.entity.t_reordering.args = &evtim; + dl_conf.entity.t_reordering.timer = testsuite_params.timdev; + dl_conf.entity.t_reordering.start = event_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + evtim.ev.event_ptr = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + while (n != 1) { + rte_delay_us(testsuite_params.min_resolution_ns / 1000); + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, + "Dequeued unexpected timer expiry event: %i\n", n); + } + + ASSERT_TRUE_OR_GOTO(event.event_type == RTE_EVENT_TYPE_TIMER, exit, "Unexpected event type\n"); + + /* Handle expiry event */ + n = rte_pdcp_t_reordering_expiry_handle(event.event_ptr, out_mb); + ASSERT_TRUE_OR_GOTO(n == 1, exit, "Unexpected number of expired packets :%i\n", n); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + +static void +test_rte_timer_expiry_handle(struct rte_timer *tim, void *arg) +{ + struct test_rte_timer_args *tim_data = arg; + struct rte_mbuf *out_mb[1] = {0}; + uint16_t n; + + RTE_SET_USED(tim); + + n = rte_pdcp_t_reordering_expiry_handle(tim_data->pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, n); + + tim_data->status = n == 1 ? n : -1; +} + +static void +test_rte_timer_start_cb(void *timer, void *args) +{ + rte_timer_reset_sync(timer, 1, SINGLE, rte_lcore_id(), test_rte_timer_expiry_handle, args); +} + +static int +test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) +{ + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct test_rte_timer_args timer_args; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_timer timer = {0}; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Set up a timer */ + rte_timer_init(&timer); + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.count = start_count; + dl_conf.entity.t_reordering.args = &timer_args; + dl_conf.entity.t_reordering.timer = &timer; + dl_conf.entity.t_reordering.start = test_rte_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + timer_args.status = 0; + timer_args.pdcp_entity = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Verify that expire was handled correctly */ + rte_timer_manage(); + while (timer_args.status != 1) { + rte_delay_us(1); + rte_timer_manage(); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, "Bad expire handle status %i\n", + timer_args.status); + } + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1115,6 +1322,116 @@ test_combined(struct pdcp_test_conf *ul_conf) return ret; } +static inline void +eventdev_conf_default_set(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info) +{ + memset(dev_conf, 0, sizeof(struct rte_event_dev_config)); + dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns; + dev_conf->nb_event_ports = 1; + dev_conf->nb_event_queues = 1; + dev_conf->nb_event_queue_flows = info->max_event_queue_flows; + dev_conf->nb_event_port_dequeue_depth = info->max_event_port_dequeue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_events_limit = info->max_num_events; +} + +static inline int +eventdev_setup(void) +{ + struct rte_event_dev_config dev_conf; + struct rte_event_dev_info info; + int ret, evdev = 0; + + if (!rte_event_dev_count()) + return TEST_SKIPPED; + + ret = rte_event_dev_info_get(evdev, &info); + TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info"); + TEST_ASSERT(info.max_num_events < 0 || info.max_num_events >= 1, + "ERROR max_num_events=%d < max_events=%d", info.max_num_events, 1); + + eventdev_conf_default_set(&dev_conf, &info); + ret = rte_event_dev_configure(evdev, &dev_conf); + TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev"); + + ret = rte_event_queue_setup(evdev, TEST_EV_QUEUE_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d", TEST_EV_QUEUE_ID); + + /* Configure event port */ + ret = rte_event_port_setup(evdev, TEST_EV_PORT_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d", TEST_EV_PORT_ID); + ret = rte_event_port_link(evdev, TEST_EV_PORT_ID, NULL, NULL, 0); + TEST_ASSERT(ret >= 0, "Failed to link all queues port=%d", TEST_EV_PORT_ID); + + ret = rte_event_dev_start(evdev); + TEST_ASSERT_SUCCESS(ret, "Failed to start device"); + + testsuite_params.evdev = evdev; + + return TEST_SUCCESS; +} + +static int +event_timer_setup(void) +{ + struct rte_event_timer_adapter_info info; + struct rte_event_timer_adapter *timdev; + uint32_t caps = 0; + + struct rte_event_timer_adapter_conf config = { + .event_dev_id = testsuite_params.evdev, + .timer_adapter_id = TIMER_ADAPTER_ID, + .timer_tick_ns = NSECPERSEC, + .max_tmo_ns = 10 * NSECPERSEC, + .nb_timers = 10, + .flags = 0, + }; + + TEST_ASSERT_SUCCESS(rte_event_timer_adapter_caps_get(testsuite_params.evdev, &caps), + "Failed to get adapter capabilities"); + + if (!(caps & RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) + return TEST_SKIPPED; + + timdev = rte_event_timer_adapter_create(&config); + + TEST_ASSERT_NOT_NULL(timdev, "Failed to create event timer ring"); + + testsuite_params.timdev = timdev; + + TEST_ASSERT_EQUAL(rte_event_timer_adapter_start(timdev), 0, + "Failed to start event timer adapter"); + + rte_event_timer_adapter_get_info(timdev, &info); + testsuite_params.min_resolution_ns = info.min_resolution_ns; + + return TEST_SUCCESS; +} + +static int +ut_setup_pdcp_event_timer(void) +{ + int ret; + ret = eventdev_setup(); + if (ret) + return ret; + return event_timer_setup(); +} + +static void +ut_teardown_pdcp_event_timer(void) +{ + struct rte_event_timer_adapter *timdev = testsuite_params.timdev; + int evdev = testsuite_params.evdev; + + rte_event_dev_stop(evdev); + rte_event_dev_close(evdev); + + rte_event_timer_adapter_stop(timdev); + rte_event_timer_adapter_free(timdev); +} + static int run_test_for_one_known_vec(const void *arg) { @@ -1159,6 +1476,14 @@ static struct unit_test_suite reorder_test_cases = { ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, test_reorder_buffer_full_window_size_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_event_timer", + ut_setup_pdcp_event_timer, ut_teardown_pdcp_event_timer, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_event_timer), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_rte_timer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_rte_timer), TEST_CASES_END() /**< NULL terminate unit test array */ } }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 18/22] test/pdcp: add timer restart case 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (16 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 17/22] test/pdcp: add timer expiry cases Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 19/22] pdcp: add support for status report Anoob Joseph ` (4 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add test to cover the case when t-reordering timer should be restarted on the same packet. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 67 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index a0e982777d..de3375bb22 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -1072,6 +1072,70 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) return ret; } +static int +test_reorder_gap_in_reorder_buffer(const struct pdcp_test_conf *ul_conf) +{ + struct rte_mbuf *m = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + int ret = TEST_FAILED, nb_out, i; + struct pdcp_test_conf dl_conf; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.count = start_count; + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + const uint32_t sn_size = dl_conf.entity.pdcp_xfrm.sn_size; + cdev_id = dl_conf.entity.dev_id; + + /* Create two gaps [NULL, m1, NULL, m3]*/ + for (i = 0; i < 2; i++) { + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + 2 * i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m = NULL; /* Packet was moved to PDCP lib */ + } + + /* Generate packet to fill the first gap */ + m = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1, NULL, m3] + * Only first gap should be filled, timer should be restarted for second gap + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m = NULL; + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), + exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == true, exit, + "Timer should be restarted after partial drain"); + + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) { @@ -1472,6 +1536,9 @@ static struct unit_test_suite reorder_test_cases = { TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_in_reorder_buffer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_in_reorder_buffer), TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 19/22] pdcp: add support for status report 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (17 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 18/22] test/pdcp: add timer restart case Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph ` (3 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Implement status report generation for PDCP entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 158 ++++++++++++++++++++++++++++++++++++--- lib/pdcp/pdcp_cnt.h | 11 ++- lib/pdcp/pdcp_ctrl_pdu.c | 34 ++++++++- lib/pdcp/pdcp_ctrl_pdu.h | 3 +- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 21 +++++- lib/pdcp/rte_pdcp.c | 32 ++++++-- 7 files changed, 233 insertions(+), 28 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index c9b952184b..af027b00d3 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -2,28 +2,164 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_pdcp.h> #include "pdcp_cnt.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#define SLAB_BYTE_SIZE (RTE_BITMAP_SLAB_BIT_SIZE / 8) + +uint32_t +pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) +{ + uint32_t n_bits = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + return rte_bitmap_get_memory_footprint(n_bits); +} + int -pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) { - struct entity_priv_dl_part *en_priv_dl; - uint32_t window_sz; + uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - if (en == NULL || conf == NULL) + dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + if (dl->bitmap.bmp == NULL) return -EINVAL; - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) - return 0; + dl->bitmap.size = window_size; - en_priv_dl = entity_dl_part_get(en); - window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + return 0; +} - RTE_SET_USED(window_sz); - RTE_SET_USED(en_priv_dl); +void +pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + rte_bitmap_set(bitmap.bmp, count % bitmap.size); +} - return 0; +bool +pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + return rte_bitmap_get(bitmap.bmp, count % bitmap.size); +} + +void +pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop) +{ + uint32_t i; + + for (i = start; i < stop; i++) + rte_bitmap_clear(bitmap.bmp, i % bitmap.size); +} + +uint16_t +pdcp_cnt_get_bitmap_size(uint32_t pending_bytes) +{ + /* + * Round up bitmap size to slab size to operate only on slabs sizes, instead of individual + * bytes + */ + return RTE_ALIGN_MUL_CEIL(pending_bytes, SLAB_BYTE_SIZE); +} + +static __rte_always_inline uint64_t +leftover_get(uint64_t slab, uint32_t shift, uint64_t mask) +{ + return (slab & mask) << shift; +} + +void +pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len) +{ + uint64_t slab = 0, next_slab = 0, leftover; + uint32_t zeros, report_len, diff; + uint32_t slab_id, next_slab_id; + uint32_t pos = 0, next_pos = 0; + + const uint32_t start_count = state.rx_deliv + 1; + const uint32_t nb_slabs = bitmap.size / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t nb_data_slabs = data_len / SLAB_BYTE_SIZE; + const uint32_t start_slab_id = start_count / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t stop_slab_id = (start_slab_id + nb_data_slabs) % nb_slabs; + const uint32_t shift = start_count % RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t leftover_shift = shift ? RTE_BITMAP_SLAB_BIT_SIZE - shift : 0; + const uint8_t *data_end = RTE_PTR_ADD(data, data_len + SLAB_BYTE_SIZE); + + /* NOTE: Mask required to workaround case - when shift is not needed */ + const uint64_t leftover_mask = shift ? ~0 : 0; + + /* NOTE: implement scan init at to set custom position */ + __rte_bitmap_scan_init(bitmap.bmp); + while (true) { + assert(rte_bitmap_scan(bitmap.bmp, &pos, &slab) == 1); + slab_id = pos / RTE_BITMAP_SLAB_BIT_SIZE; + if (slab_id >= start_slab_id) + break; + } + + report_len = nb_data_slabs; + + if (slab_id > start_slab_id) { + /* Zero slabs at beginning */ + zeros = (slab_id - start_slab_id - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + leftover = leftover_get(slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + report_len -= (slab_id - start_slab_id); + } + + while (report_len) { + rte_bitmap_scan(bitmap.bmp, &next_pos, &next_slab); + next_slab_id = next_pos / RTE_BITMAP_SLAB_BIT_SIZE; + diff = (next_slab_id + nb_slabs - slab_id) % nb_slabs; + + /* If next_slab_id == slab_id - overlap */ + diff += !(next_slab_id ^ slab_id) * nb_slabs; + + /* Size check - next slab is outsize of size range */ + if (diff > report_len) { + next_slab = 0; + next_slab_id = stop_slab_id; + diff = report_len; + } + + report_len -= diff; + + /* Calculate gap between slabs, taking wrap around into account */ + zeros = (next_slab_id + nb_slabs - slab_id - 1) % nb_slabs; + if (zeros) { + /* Non continues slabs, align them individually */ + slab >>= shift; + memcpy(data, &slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + + /* Fill zeros between slabs */ + zeros = (zeros - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + + /* Align beginning of next slab */ + leftover = leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } else { + /* Continues slabs, combine them */ + uint64_t new_slab = (slab >> shift) | + leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &new_slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } + + slab = next_slab; + pos = next_pos; + slab_id = next_slab_id; + + }; + + assert(data < data_end); } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index bbda478b55..5941b7a406 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -9,6 +9,15 @@ #include "pdcp_entity.h" -int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); +uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); + +void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +void pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop); + +uint16_t pdcp_cnt_get_bitmap_size(uint32_t pending_bytes); +void pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len); #endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c index feb05fd863..e0ac2d3720 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.c +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -8,6 +8,14 @@ #include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#include "pdcp_cnt.h" + +static inline uint16_t +round_up_bits(uint32_t bits) +{ + /* round up to the next multiple of 8 */ + return RTE_ALIGN_MUL_CEIL(bits, 8) / 8; +} static __rte_always_inline void pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) @@ -19,11 +27,13 @@ pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) } int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m) { struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; - uint32_t rx_deliv; - int pdu_sz; + uint32_t rx_deliv, actual_sz; + uint16_t pdu_sz, bitmap_sz; + uint8_t *data; if (!en_priv->flags.is_status_report_required) return -EINVAL; @@ -42,5 +52,21 @@ pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) return 0; } - return -ENOTSUP; + actual_sz = RTE_MIN(round_up_bits(en_priv->state.rx_next - rx_deliv - 1), + RTE_PDCP_CTRL_PDU_SIZE_MAX - pdu_sz); + bitmap_sz = pdcp_cnt_get_bitmap_size(actual_sz); + + data = (uint8_t *)rte_pktmbuf_append(m, pdu_sz + bitmap_sz); + if (data == NULL) + return -ENOMEM; + + m->pkt_len = pdu_sz + actual_sz; + m->data_len = pdu_sz + actual_sz; + + pdcp_hdr_fill((struct rte_pdcp_up_ctrl_pdu_hdr *)data, rx_deliv); + + data = RTE_PTR_ADD(data, pdu_sz); + pdcp_cnt_report_fill(dl->bitmap, en_priv->state, data, bitmap_sz); + + return 0; } diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h index a2424fbd10..2a87928b88 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.h +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -10,6 +10,7 @@ #include "pdcp_entity.h" int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m); #endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index ca98a1d0f7..8e1d6254dd 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -182,6 +182,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /** Bitmap memory region */ + uint8_t bitmap_mem[0]; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 91b87a2a81..cdadc9b6b8 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -9,6 +9,7 @@ #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -809,6 +810,16 @@ pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool tr } } +static inline void +pdcp_rx_deliv_set(const struct rte_pdcp_entity *entity, uint32_t rx_deliv) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + pdcp_cnt_bitmap_range_clear(dl->bitmap, en_priv->state.rx_deliv, rx_deliv); + en_priv->state.rx_deliv = rx_deliv; +} + static inline int pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, const uint32_t count, struct rte_mbuf *mb, @@ -829,11 +840,15 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; + if (unlikely(pdcp_cnt_bitmap_is_set(dl->bitmap, count))) + return -EEXIST; + + pdcp_cnt_bitmap_set(dl->bitmap, count); pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); if (en_priv->flags.is_out_of_order_delivery) { out_mb[0] = mb; - en_priv->state.rx_deliv = count + 1; + pdcp_rx_deliv_set(entity, count + 1); return 1; } @@ -860,7 +875,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, } /* Processed should never exceed the window size */ - en_priv->state.rx_deliv = count + processed; + pdcp_rx_deliv_set(entity, count + processed); } else { if (!reorder->is_active) @@ -1317,7 +1332,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered * to upper layers, with COUNT value >= RX_REORD; */ - en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + pdcp_rx_deliv_set(entity, en_priv->state.rx_reord + nb_seq); /* * - if RX_DELIV < RX_NEXT: diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 755d592578..ce846d687e 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -12,6 +12,8 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +static int bitmap_mem_offset; + static int pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { @@ -19,9 +21,12 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); - else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size = RTE_CACHE_LINE_ROUNDUP(size); + bitmap_mem_offset = size; + size += pdcp_cnt_bitmap_get_memory_footprint(conf); + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; @@ -34,11 +39,24 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + void *bitmap_mem; + int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - return pdcp_reorder_create(&dl->reorder, window_size); + ret = pdcp_reorder_create(&dl->reorder, window_size); + if (ret) + return ret; + + bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); + ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); + if (ret) { + pdcp_reorder_destroy(&dl->reorder); + return ret; + } + + return 0; } struct rte_pdcp_entity * @@ -110,10 +128,6 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; } - ret = pdcp_cnt_ring_create(entity, conf); - if (ret) - goto crypto_sess_destroy; - return entity; crypto_sess_destroy: @@ -192,6 +206,7 @@ struct rte_mbuf * rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, enum rte_pdcp_ctrl_pdu_type type) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; struct rte_mbuf *m; int ret; @@ -202,6 +217,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, } en_priv = entity_priv_get(pdcp_entity); + dl = entity_dl_part_get(pdcp_entity); m = rte_pktmbuf_alloc(en_priv->ctr_pdu_pool); if (m == NULL) { @@ -211,7 +227,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, switch (type) { case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: - ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + ret = pdcp_ctrl_pdu_status_gen(en_priv, dl, m); break; default: ret = -ENOTSUP; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 20/22] pdcp: allocate reorder buffer alongside with entity 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (18 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 19/22] pdcp: add support for status report Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 21/22] pdcp: add thread safe processing Anoob Joseph ` (2 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Instead of allocating reorder buffer separately on heap, allocate memory for it together with rest of entity, and then only initialize buffer via `rte_reorder_init()`. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 9 +++---- lib/pdcp/pdcp_cnt.h | 3 ++- lib/pdcp/pdcp_entity.h | 2 +- lib/pdcp/pdcp_reorder.c | 11 ++------ lib/pdcp/pdcp_reorder.h | 12 ++++++--- lib/pdcp/rte_pdcp.c | 58 ++++++++++++++++++++++++++--------------- 6 files changed, 55 insertions(+), 40 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index af027b00d3..e1d0634b4d 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -20,15 +20,14 @@ pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) } int -pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size) { - uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - - dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + dl->bitmap.bmp = rte_bitmap_init(nb_elem, bitmap_mem, mem_size); if (dl->bitmap.bmp == NULL) return -EINVAL; - dl->bitmap.size = window_size; + dl->bitmap.size = nb_elem; return 0; } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index 5941b7a406..87b011f9dc 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -10,7 +10,8 @@ #include "pdcp_entity.h" uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); -int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size); void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 8e1d6254dd..38fa71acef 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -132,7 +132,7 @@ struct pdcp_cnt_bitmap { }; /* - * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] [reorder/bitmap] */ struct entity_priv { diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c index 5399f0dc28..bc45f2e19b 100644 --- a/lib/pdcp/pdcp_reorder.c +++ b/lib/pdcp/pdcp_reorder.c @@ -8,20 +8,13 @@ #include "pdcp_reorder.h" int -pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size) { - reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + reorder->buf = rte_reorder_init(mem, mem_size, "reorder_buffer", nb_elem); if (reorder->buf == NULL) return -rte_errno; - reorder->window_size = window_size; reorder->is_active = false; return 0; } - -void -pdcp_reorder_destroy(const struct pdcp_reorder *reorder) -{ - rte_reorder_free(reorder->buf); -} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h index 6a2f61d6ae..7e4f079d4b 100644 --- a/lib/pdcp/pdcp_reorder.h +++ b/lib/pdcp/pdcp_reorder.h @@ -9,12 +9,18 @@ struct pdcp_reorder { struct rte_reorder_buffer *buf; - uint32_t window_size; bool is_active; }; -int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); -void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); +int pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size); + +/* NOTE: replace with `rte_reorder_memory_footprint_get` after DPDK 23.07 */ +#define SIZE_OF_REORDER_BUFFER (4 * RTE_CACHE_LINE_SIZE) +static inline size_t +pdcp_reorder_memory_footprint_get(size_t nb_elem) +{ + return SIZE_OF_REORDER_BUFFER + (2 * nb_elem * sizeof(struct rte_mbuf *)); +} static inline uint32_t pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index ce846d687e..95d2283cef 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -12,49 +12,65 @@ #include "pdcp_entity.h" #include "pdcp_process.h" -static int bitmap_mem_offset; +struct entity_layout { + size_t bitmap_offset; + size_t bitmap_size; + + size_t reorder_buf_offset; + size_t reorder_buf_size; + + size_t total_size; +}; static int -pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +pdcp_entity_layout_get(const struct rte_pdcp_entity_conf *conf, struct entity_layout *layout) { - int size; + size_t size; + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); + /* Bitmap require memory to be cache aligned */ size = RTE_CACHE_LINE_ROUNDUP(size); - bitmap_mem_offset = size; - size += pdcp_cnt_bitmap_get_memory_footprint(conf); + layout->bitmap_offset = size; + layout->bitmap_size = pdcp_cnt_bitmap_get_memory_footprint(conf); + size += layout->bitmap_size; + layout->reorder_buf_offset = size; + layout->reorder_buf_size = pdcp_reorder_memory_footprint_get(window_size); + size += layout->reorder_buf_size; } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; - return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); + layout->total_size = size; + + return 0; } static int -pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf, + const struct entity_layout *layout) { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); - void *bitmap_mem; + void *memory; int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - ret = pdcp_reorder_create(&dl->reorder, window_size); + memory = RTE_PTR_ADD(entity, layout->reorder_buf_offset); + ret = pdcp_reorder_create(&dl->reorder, window_size, memory, layout->reorder_buf_size); if (ret) return ret; - bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); - ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); - if (ret) { - pdcp_reorder_destroy(&dl->reorder); + memory = RTE_PTR_ADD(entity, layout->bitmap_offset); + ret = pdcp_cnt_bitmap_create(dl, window_size, memory, layout->bitmap_size); + if (ret) return ret; - } return 0; } @@ -62,9 +78,10 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { + struct entity_layout entity_layout = { 0 }; struct rte_pdcp_entity *entity = NULL; struct entity_priv *en_priv; - int ret, entity_size; + int ret; if (conf == NULL || conf->cop_pool == NULL || conf->ctr_pdu_pool == NULL) { rte_errno = -EINVAL; @@ -94,13 +111,14 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } - entity_size = pdcp_entity_size_get(conf); - if (entity_size < 0) { + ret = pdcp_entity_layout_get(conf, &entity_layout); + if (ret < 0) { rte_errno = -EINVAL; return NULL; } - entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + entity = rte_zmalloc_socket("pdcp_entity", entity_layout.total_size, RTE_CACHE_LINE_SIZE, + SOCKET_ID_ANY); if (entity == NULL) { rte_errno = -ENOMEM; return NULL; @@ -123,7 +141,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { - ret = pdcp_dl_establish(entity, conf); + ret = pdcp_dl_establish(entity, conf, &entity_layout); if (ret) goto crypto_sess_destroy; } @@ -148,8 +166,6 @@ pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, en_priv->state.rx_next); - pdcp_reorder_destroy(&dl->reorder); - return nb_out; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 21/22] pdcp: add thread safe processing 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (19 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 22/22] test/pdcp: add PDCP status report cases Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> PDCP state has to be guarded for: - Uplink pre_process: - tx_next atomic increment - Downlink pre_process: - rx_deliv - read - Downlink post_process: - rx_deliv, rx_reorder, rx_next - read/write - bitmask/reorder buffer - read/write When application requires thread safe processing, the state variables need to be updated atomically. Add config option to select this option per entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 46 +++++++++++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 34 +++++++++++++++++++++++++++--- lib/pdcp/rte_pdcp.c | 2 ++ lib/pdcp/rte_pdcp.h | 2 ++ 4 files changed, 81 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 38fa71acef..2dd6d2417d 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -10,6 +10,7 @@ #include <rte_mempool.h> #include <rte_pdcp.h> #include <rte_security.h> +#include <rte_spinlock.h> #include "pdcp_reorder.h" @@ -162,6 +163,8 @@ struct entity_priv { uint64_t is_status_report_required : 1; /** Is out-of-order delivery enabled */ uint64_t is_out_of_order_delivery : 1; + /** Is thread safety disabled */ + uint64_t is_thread_safety_disabled : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -175,6 +178,8 @@ struct entity_priv { uint8_t dev_id; }; +typedef rte_spinlock_t pdcp_lock_t; + struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; @@ -182,6 +187,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /* Lock to protect concurrent updates */ + pdcp_lock_t lock; /** Bitmap memory region */ uint8_t bitmap_mem[0]; }; @@ -257,4 +264,43 @@ pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) return (1 << (32 - sn_size)) - 1; } +static inline uint32_t +pdcp_atomic_inc(const struct entity_priv *en_priv, uint32_t *val) +{ + if (en_priv->flags.is_thread_safety_disabled) + return (*val)++; + else + return __atomic_fetch_add(val, 1, __ATOMIC_RELAXED); +} + +static inline void +pdcp_lock_init(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_init(&dl->lock); +} + +static inline void +pdcp_lock_lock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_lock(&dl->lock); +} + +static inline void +pdcp_lock_unlock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_unlock(&dl->lock); +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index cdadc9b6b8..0bafa3447a 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -369,7 +369,7 @@ pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_ return false; /* Update sequence num in the PDU header */ - *count = en_priv->state.tx_next++; + *count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; @@ -451,7 +451,7 @@ pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_ return false; /* Update sequence num in the PDU header */ - *count = en_priv->state.tx_next++; + *count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; @@ -561,7 +561,7 @@ pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rt memset(mac_i, 0, PDCP_MAC_I_LEN); /* Update sequence number in the PDU header */ - count = en_priv->state.tx_next++; + count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); @@ -654,7 +654,9 @@ pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -714,7 +716,9 @@ pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -775,7 +779,9 @@ pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rt nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -925,6 +931,8 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb; uint32_t count; + pdcp_lock_lock(entity); + for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -954,6 +962,8 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -994,6 +1004,7 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, int32_t rsn = 0; uint32_t count; + pdcp_lock_lock(entity); for (i = 0; i < num; i++) { mb = in_mb[i]; @@ -1026,6 +1037,8 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -1066,6 +1079,8 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, uint32_t count; int32_t rsn; + pdcp_lock_lock(entity); + for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -1091,6 +1106,8 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -1254,6 +1271,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent */ en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** + * flags.disable_thread_safety + * + * Indicate whether the thread safety is disabled for PDCP entity. + */ + en_priv->flags.is_thread_safety_disabled = conf->disable_thread_safety; + /** * hdr_sz * @@ -1316,6 +1340,8 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * performing header decompression, if not decompressed before: */ + pdcp_lock_lock(entity); + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); capacity -= nb_out; @@ -1347,5 +1373,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct dl->t_reorder.state = TIMER_EXPIRED; } + pdcp_lock_unlock(entity); + return nb_out; } diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 95d2283cef..06b86c274e 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -72,6 +72,8 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c if (ret) return ret; + pdcp_lock_init(entity); + return 0; } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index c077acce63..a8b824a7ee 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -137,6 +137,8 @@ struct rte_pdcp_entity_conf { bool is_slrb; /** Enable security offload on the device specified. */ bool en_sec_offload; + /** Disable usage of synchronization primitives for entity. */ + bool disable_thread_safety; /** Device on which security/crypto session need to be created. */ uint8_t dev_id; /** Reverse direction during IV generation. Can be used to simulate UE crypto processing.*/ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v2 22/22] test/pdcp: add PDCP status report cases 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (20 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 21/22] pdcp: add thread safe processing Anoob Joseph @ 2023-04-14 17:45 ` Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-04-14 17:45 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Test PDCP status report generation. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 310 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 310 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index de3375bb22..c9e4b894ac 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -2,6 +2,7 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_errno.h> #include <rte_eventdev.h> #include <rte_event_timer_adapter.h> @@ -43,6 +44,9 @@ struct pdcp_testsuite_params { struct rte_event_timer_adapter *timdev; bool timer_is_running; uint64_t min_resolution_ns; + struct rte_pdcp_up_ctrl_pdu_hdr *status_report; + uint32_t status_report_bitmask_capacity; + uint8_t *ctrl_pdu_buf; }; static struct pdcp_testsuite_params testsuite_params; @@ -139,6 +143,18 @@ static struct rte_pdcp_t_reordering t_reorder_timer = { .stop = pdcp_timer_stop_cb, }; +static inline void +bitmask_set_bit(uint8_t *mask, uint32_t bit) +{ + mask[bit / 8] |= (1 << bit % 8); +} + +static inline bool +bitmask_is_bit_set(const uint8_t *mask, uint32_t bit) +{ + return mask[bit / 8] & (1 << (bit % 8)); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -285,6 +301,21 @@ testsuite_setup(void) goto cop_pool_free; } + /* Allocate memory for longest possible status report */ + ts_params->status_report_bitmask_capacity = RTE_PDCP_CTRL_PDU_SIZE_MAX - + sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + ts_params->status_report = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->status_report == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report\n"); + goto cop_pool_free; + } + + ts_params->ctrl_pdu_buf = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->ctrl_pdu_buf == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report data\n"); + goto cop_pool_free; + } + return 0; cop_pool_free: @@ -293,6 +324,8 @@ testsuite_setup(void) mbuf_pool_free: rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); return TEST_FAILED; } @@ -315,6 +348,9 @@ testsuite_teardown(void) rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); } static int @@ -1364,6 +1400,244 @@ test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) return ret; } +static struct rte_pdcp_up_ctrl_pdu_hdr * +pdcp_status_report_init(uint32_t fmc) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + + hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + hdr->fmc = rte_cpu_to_be_32(fmc); + hdr->r = 0; + memset(hdr->bitmap, 0, testsuite_params.status_report_bitmask_capacity); + + return hdr; +} + +static uint32_t +pdcp_status_report_len(void) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + uint32_t i; + + for (i = testsuite_params.status_report_bitmask_capacity; i != 0; i--) { + if (hdr->bitmap[i - 1]) + return i; + } + + return 0; +} + +static int +pdcp_status_report_verify(struct rte_mbuf *status_report, + const struct rte_pdcp_up_ctrl_pdu_hdr *expected_hdr, uint32_t expected_len) +{ + uint32_t received_len = rte_pktmbuf_pkt_len(status_report); + uint8_t *received_buf = testsuite_params.ctrl_pdu_buf; + int ret; + + ret = pktmbuf_read_into(status_report, received_buf, RTE_PDCP_CTRL_PDU_SIZE_MAX); + TEST_ASSERT_SUCCESS(ret, "Failed to copy status report pkt into continuous buffer"); + + debug_hexdump(stdout, "Received:", received_buf, received_len); + debug_hexdump(stdout, "Expected:", expected_hdr, expected_len); + + TEST_ASSERT_EQUAL(expected_len, received_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_len, received_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(received_buf, expected_hdr, expected_len, + "Generated packet not as expected"); + + return 0; +} + +static int +test_status_report_gen(const struct pdcp_test_conf *ul_conf, + const struct rte_pdcp_up_ctrl_pdu_hdr *hdr, + uint32_t bitmap_len) +{ + struct rte_mbuf *status_report = NULL, **out_mb, *m; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint32_t nb_pkts = 0, i; + uint8_t cdev_id; + + const uint32_t start_count = rte_be_to_cpu_32(hdr->fmc); + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.count = start_count; + dl_conf.entity.status_report_required = true; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + out_mb = calloc(pdcp_entity->max_pkt_cache, sizeof(uintptr_t)); + + for (i = 0; i < bitmap_len * 8; i++) { + if (!bitmask_is_bit_set(hdr->bitmap, i)) + continue; + + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + + } + + m = NULL; + + /* Check status report */ + status_report = rte_pdcp_control_pdu_create(pdcp_entity, + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT); + ASSERT_TRUE_OR_GOTO(status_report != NULL, exit, "Could not generate status report\n"); + + const uint32_t expected_len = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr) + bitmap_len; + + ASSERT_TRUE_OR_GOTO(pdcp_status_report_verify(status_report, hdr, expected_len) == 0, exit, + "Report verification failure\n"); + + ret = TEST_SUCCESS; +exit: + rte_free(m); + rte_pktmbuf_free(status_report); + rte_pktmbuf_free_bulk(out_mb, nb_pkts); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + free(out_mb); + return ret; +} + +static void +ctrl_pdu_hdr_packet_set(struct rte_pdcp_up_ctrl_pdu_hdr *hdr, uint32_t pkt_count) +{ + bitmask_set_bit(hdr->bitmap, pkt_count - rte_be_to_cpu_32(hdr->fmc) - 1); +} + +static int +test_status_report_fmc_only(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(42); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_first_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_second_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_full_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE + 1; + int i; + + for (i = 0; i < RTE_BITMAP_SLAB_BIT_SIZE; i++) + ctrl_pdu_hdr_packet_set(hdr, start_offset + i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_non_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + ctrl_pdu_hdr_packet_set(hdr, 3 * RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_max_length_sn_12(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr; + const uint32_t fmc = 0; + uint32_t i; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + ul_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + hdr = pdcp_status_report_init(fmc); + + const uint32_t max_count = RTE_MIN((RTE_PDCP_CTRL_PDU_SIZE_MAX - sizeof(hdr)) * 8, + (uint32_t)PDCP_WINDOW_SIZE(RTE_SECURITY_PDCP_SN_SIZE_12)); + + i = fmc + 2; /* set first count to have a gap, to enable packet buffering */ + + for (; i < max_count; i++) + ctrl_pdu_hdr_packet_set(hdr, i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_different_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(63); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 64 + 1); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_same_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(2); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 4); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1555,11 +1829,47 @@ static struct unit_test_suite reorder_test_cases = { } }; +static struct unit_test_suite status_report_test_cases = { + .suite_name = "PDCP status report", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_status_report_fmc_only", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_fmc_only), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_first_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_first_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_second_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_second_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_full_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_full_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_non_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_non_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_max_length_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_status_report_max_length_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_different_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_different_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_same_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_same_slab), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, &reorder_test_cases, + &status_report_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 00/22] lib: add pdcp protocol 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph ` (21 preceding siblings ...) 2023-04-14 17:45 ` [PATCH v2 22/22] test/pdcp: add PDCP status report cases Anoob Joseph @ 2023-05-24 16:00 ` Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 01/22] net: add PDCP header Anoob Joseph ` (22 more replies) 22 siblings, 23 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:00 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following operations, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Similar to lib IPsec, lib PDCP would allow application to use same API sequence while leveraging protocol offload features enabled by rte_security library. Lib PDCP would internally change the handles registered for *pre_process* and *post_process* based on features enabled in the entity. Lib PDCP would create the required sessions on the device provided in entity to minimize the application requirements. Also, the crypto_op allocation and free would also be done internally by lib PDCP to allow the library to create crypto ops as required for the input packets. For example, when control PDUs are received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP is expected to handle it differently. Lib PDCP utilizes reorder library for implementing in-order delivery. It utilizes bitmap library for implementing status reports and track the COUNT value of the packets received. To allow application to choose timer implementation of choice, lib PDCP allows application to configure handles that can be used for starting & stopping timers. Upon expiry, application can call corresponding PDCP API(``rte_pdcp_t_reordering_expiry_handle``) for handling the event. Unit tests are added to verify both rte_timer based timers as well as rte_eventdev based timers. PDCP tracks the sequence number of the received packets and during events such as re-establishment, it is required to generate reports and transmit to the peer. This series introduces ``rte_pdcp_control_pdu_create`` for handling control PDU generation. Changes in v3: - Addressed review comments (Akhil) - Addressed build failure in CI (tests with lib eventdev disabled) - Addressed checkpatch issues - Set only positive values to rte_errno (Akhil) Changes in v2: - Added control PDU handling - Added t-Reordering timer - Added in-order delivery - Added status PDU generation - Rebased on top of new features added in reorder library - Split base patch - Increased test coverage - Improved thread safety Changes from RFC - Implementation for all APIs covering basic control plane & user plane packets - Unit test leveraging existing PDCP vectors available in test_cryptodev - Unit test performing both UL & DL operations to verify various protocol features - Updated documentation Sample application sequence: struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; struct rte_crypto_op *cop[MAX_BURST_SIZE]; struct rte_pdcp_group grp[MAX_BURST_SIZE]; struct rte_pdcp_entity *pdcp_entity; int nb_max_out_mb, ret, nb_grp; /* Create PDCP entity */ pdcp_entity = rte_pdcp_entity_establish(&conf); /** * Allocate buffer for holding mbufs returned during PDCP suspend, * release & post-process APIs. */ /* Max packets that can be cached in entity + burst size */ nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); if (out_mb == NULL) { /* Handle error */ } while (1) { /* Receive packet and form mbuf */ /** * Prepare packets for crypto operation. Following operations * would be done, * * Transmitting entity/UL (only data PDUs): * - Perform compression * - Assign sequence number * - Add PDCP header * - Create & prepare crypto_op * - Prepare IV for crypto operation (auth_gen, encrypt) * - Save original PDCP SDU (during PDCP re-establishment, * unconfirmed PDCP SDUs need to crypto processed again and * transmitted/re-transmitted) * * Receiving entity/DL: * - Any control PDUs received would be processed and * appropriate actions taken. If data PDU, continue. * - Determine sequence number (based on HFN & per packet SN) * - Prepare crypto_op * - Prepare IV for crypto operation (decrypt, auth_verify) */ nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, nb_rx, &nb_err); if (nb_err != 0) { /* Handle error packets */ } if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) != nb_success) { /* Retry for enqueue failure packets */ } ... ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, MAX_BURST_SIZE); if (ret == 0) continue; /** * Received a burst of completed crypto ops from cryptodev. It * may belong to various entities. Group similar ones together * for entity specific post-processing. */ /** * Groups similar entities together. Frees crypto op and based * on crypto_op status, set mbuf->ol_flags which would be * checked in rte_pdcp_pkt_post_process(). */ nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); for (i = 0; i != nb_grp; i++) { /** * Post process packets after crypto completion. * Following operations would be done, * * Transmitting entity/UL: * - Check crypto result * * Receiving entity/DL: * - Check crypto operation status * - Check for duplication (if yes, drop duplicate) * - Perform decompression * - Trim PDCP header * - Hold packet (SDU) for in-order delivery (return * completed packets as and when sequence is * completed) * - If not in sequence, cache the packet and start * t-Reordering timer. When timer expires, the * packets need to delivered to upper layers (not * treated as error packets). */ nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, grp[i].m, out_mb, grp[i].cnt, &nb_err); if (nb_err != 0) { /* Handle error packets */ } /* Perform additional operations */ /** * Tranmitting entity/UL * - If duplication is enabled, duplicate PDCP PDUs * - When lower layers confirm reception of a PDCP PDU, * it should be communicated to PDCP layer so that * PDCP can drop the corresponding SDU */ } } Sample output from unit test executed on crypto_cn10k PMD(summary): ~# DPDK_TEST=pdcp_autotest ./dpdk-test --log-level=7 <snip> + ------------------------------------------------------- + + Test Suite Summary : PDCP Unit Test Suite + ------------------------------------------------------- + + Known vector cases : 112/160 passed, 0/160 skipped, 0/160 failed, 48/160 unsupported + PDCP combined mode : 1/1 passed, 0/1 skipped, 0/1 failed, 0/1 unsupported + PDCP HFN/SN : 4/4 passed, 0/4 skipped, 0/4 failed, 0/4 unsupported + PDCP reorder : 5/5 passed, 0/5 skipped, 0/5 failed, 0/5 unsupported + PDCP status report : 9/9 passed, 0/9 skipped, 0/9 failed, 0/9 unsupported + ------------------------------------------------------- + + Sub Testsuites Total : 5 + Sub Testsuites Skipped : 0 + Sub Testsuites Passed : 5 + Sub Testsuites Failed : 0 + ------------------------------------------------------- + + Tests Total : 179 + Tests Skipped : 0 + Tests Executed : 179 + Tests Unsupported: 48 + Tests Passed : 131 + Tests Failed : 0 + ------------------------------------------------------- + Test OK Anoob Joseph (10): lib: add pdcp protocol pdcp: add pre and post-process pdcp: add packet group pdcp: add crypto session create and destroy pdcp: add pre and post process for UL pdcp: add pre and post process for DL pdcp: add IV generation routines app/test: add lib pdcp tests doc: add PDCP library guide pdcp: add control PDU handling for status report Volodymyr Fialko (12): net: add PDCP header test/pdcp: pdcp HFN tests in combined mode pdcp: implement t-Reordering and packet buffering test/pdcp: add in-order delivery cases pdcp: add timer callback handlers pdcp: add timer expiry handle test/pdcp: add timer expiry cases test/pdcp: add timer restart case pdcp: add support for status report pdcp: allocate reorder buffer alongside with entity pdcp: add thread safe processing test/pdcp: add PDCP status report cases app/test/meson.build | 1 + app/test/test_cryptodev.h | 3 + app/test/test_pdcp.c | 1979 +++++++++++++++++ doc/api/doxy-api-index.md | 4 +- doc/api/doxy-api.conf.in | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 293 +++ lib/meson.build | 1 + lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++ lib/pdcp/meson.build | 21 + lib/pdcp/pdcp_cnt.c | 164 ++ lib/pdcp/pdcp_cnt.h | 24 + lib/pdcp/pdcp_crypto.c | 238 ++ lib/pdcp/pdcp_crypto.h | 20 + lib/pdcp/pdcp_ctrl_pdu.c | 72 + lib/pdcp/pdcp_ctrl_pdu.h | 16 + lib/pdcp/pdcp_entity.h | 306 +++ lib/pdcp/pdcp_process.c | 1317 +++++++++++ lib/pdcp/pdcp_process.h | 26 + lib/pdcp/pdcp_reorder.c | 20 + lib/pdcp/pdcp_reorder.h | 68 + lib/pdcp/rte_pdcp.c | 317 +++ lib/pdcp/rte_pdcp.h | 381 ++++ lib/pdcp/rte_pdcp_group.h | 131 ++ lib/pdcp/version.map | 20 + 27 files changed, 5572 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pdcp.c create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst create mode 100644 lib/net/rte_pdcp_hdr.h create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 01/22] net: add PDCP header 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph @ 2023-05-24 16:00 ` Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 02/22] lib: add pdcp protocol Anoob Joseph ` (21 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:00 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add PDCP protocol header to be used for supporting PDCP protocol processing. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- doc/api/doxy-api-index.md | 3 +- lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 150 insertions(+), 1 deletion(-) create mode 100644 lib/net/rte_pdcp_hdr.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index c709fd48ad..debbe4134f 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -127,7 +127,8 @@ The public API headers are grouped by topics: [Geneve](@ref rte_geneve.h), [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), - [PPP](@ref rte_ppp.h) + [PPP](@ref rte_ppp.h), + [PDCP hdr](@ref rte_pdcp_hdr.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/lib/net/meson.build b/lib/net/meson.build index 379d161ee0..bd56f91c22 100644 --- a/lib/net/meson.build +++ b/lib/net/meson.build @@ -22,6 +22,7 @@ headers = files( 'rte_geneve.h', 'rte_l2tpv2.h', 'rte_ppp.h', + 'rte_pdcp_hdr.h', ) sources = files( diff --git a/lib/net/rte_pdcp_hdr.h b/lib/net/rte_pdcp_hdr.h new file mode 100644 index 0000000000..a37cf683b7 --- /dev/null +++ b/lib/net/rte_pdcp_hdr.h @@ -0,0 +1,147 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_HDR_H +#define RTE_PDCP_HDR_H + +/** + * @file + * + * PDCP-related defines + * + * Based on - ETSI TS 138 323 V17.1.0 (2022-08) + * https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + */ + +#include <rte_byteorder.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * 4.3.1 + * + * Indicate the maximum supported size of a PDCP Control PDU. + */ +#define RTE_PDCP_CTRL_PDU_SIZE_MAX 9000u + +/** + * 6.3.4 MAC-I + * + * Indicate the size of MAC-I in PDCP PDU. + */ +#define RTE_PDCP_MAC_I_LEN 4 + +/** + * Indicate type of control information included in the corresponding PDCP + * Control PDU. + */ +enum rte_pdcp_ctrl_pdu_type { + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT = 0, + RTE_PDCP_CTRL_PDU_TYPE_ROHC_FEEDBACK = 1, + RTE_PDCP_CTRL_PDU_TYPE_EHC_FEEDBACK = 2, + RTE_PDCP_CRTL_PDU_TYPE_UDC_FEEDBACK = 3, +}; + +/** + * 6.3.7 D/C + * + * This field indicates whether the corresponding PDCP PDU is a + * PDCP Data PDU or a PDCP Control PDU. + */ +enum rte_pdcp_pdu_type { + RTE_PDCP_PDU_TYPE_CTRL = 0, + RTE_PDCP_PDU_TYPE_DATA = 1, +}; + +/** + * 6.2.2.1 Data PDU for SRBs + */ +__extension__ +struct rte_pdcp_cp_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 4; /**< Reserved */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.2.2 Data PDU for DRBs and MRBs with 12 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 3; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 3; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.2.3 Data PDU for DRBs and MRBs with 18 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_18_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ + uint8_t r : 5; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 5; /**< Reserved */ + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ +#endif + uint8_t sn_15_8; /**< Sequence number bits 8-15 */ + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +}; + +/** + * 6.2.3.1 Control PDU for PDCP status report + */ +__extension__ +struct rte_pdcp_up_ctrl_pdu_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t r : 4; /**< Reserved */ +#endif + /** + * 6.3.9 FMC + * + * First Missing COUNT. This field indicates the COUNT value of the + * first missing PDCP SDU within the reordering window, i.e. RX_DELIV. + */ + rte_be32_t fmc; + /** + * 6.3.10 Bitmap + * + * Length: Variable. The length of the bitmap field can be 0. + * + * This field indicates which SDUs are missing and which SDUs are + * correctly received in the receiving PDCP entity. The bit position of + * Nth bit in the Bitmap is N, i.e., the bit position of the first bit + * in the Bitmap is 1. + */ + uint8_t bitmap[]; +} __rte_packed; + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_HDR_H */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 02/22] lib: add pdcp protocol 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 01/22] net: add PDCP header Anoob Joseph @ 2023-05-24 16:00 ` Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 03/22] pdcp: add pre and post-process Anoob Joseph ` (20 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:00 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + lib/meson.build | 1 + lib/pdcp/meson.build | 17 ++++ lib/pdcp/pdcp_crypto.c | 21 +++++ lib/pdcp/pdcp_crypto.h | 15 ++++ lib/pdcp/pdcp_entity.h | 113 ++++++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 138 +++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 13 +++ lib/pdcp/rte_pdcp.c | 141 ++++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 167 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 10 +++ 12 files changed, 639 insertions(+), 1 deletion(-) create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/version.map diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index debbe4134f..cd7a6cae44 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -128,7 +128,8 @@ The public API headers are grouped by topics: [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), [PPP](@ref rte_ppp.h), - [PDCP hdr](@ref rte_pdcp_hdr.h) + [PDCP hdr](@ref rte_pdcp_hdr.h), + [PDCP](@ref rte_pdcp.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index d230a19e1f..58789308a9 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -62,6 +62,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/net \ @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ + @TOPDIR@/lib/pdcp \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ @TOPDIR@/lib/port \ diff --git a/lib/meson.build b/lib/meson.build index dc8aa4ac84..a6a54c196c 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -64,6 +64,7 @@ libraries = [ 'flow_classify', # flow_classify lib depends on pkt framework table lib 'graph', 'node', + 'pdcp', # pdcp lib depends on crypto and security ] optional_libs = [ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode 100644 index 0000000000..ccaf426240 --- /dev/null +++ b/lib/pdcp/meson.build @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +sources = files( + 'pdcp_crypto.c', + 'pdcp_process.c', + 'rte_pdcp.c', + ) +headers = files('rte_pdcp.h') + +deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c new file mode 100644 index 0000000000..755e27ec9e --- /dev/null +++ b/lib/pdcp/pdcp_crypto.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_crypto.h" + +int +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + RTE_SET_USED(entity); + RTE_SET_USED(conf); + return 0; +} + +void +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) +{ + RTE_SET_USED(entity); +} diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h new file mode 100644 index 0000000000..6563331d37 --- /dev/null +++ b/lib/pdcp/pdcp_crypto.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CRYPTO_H +#define PDCP_CRYPTO_H + +#include <rte_pdcp.h> + +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, + const struct rte_pdcp_entity_conf *conf); + +void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); + +#endif /* PDCP_CRYPTO_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h new file mode 100644 index 0000000000..000297588f --- /dev/null +++ b/lib/pdcp/pdcp_entity.h @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_ENTITY_H +#define PDCP_ENTITY_H + +#include <rte_common.h> +#include <rte_crypto_sym.h> +#include <rte_mempool.h> +#include <rte_pdcp.h> +#include <rte_security.h> + +struct entity_priv; + +/* IV generation function based on the entity configuration */ +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count); + +struct entity_state { + uint32_t rx_next; + uint32_t tx_next; + uint32_t rx_deliv; + uint32_t rx_reord; +}; + +/* + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + */ + +struct entity_priv { + /** Crypto sym session. */ + struct rte_cryptodev_sym_session *crypto_sess; + /** Entity specific IV generation function. */ + iv_gen_t iv_gen; + /** Entity state variables. */ + struct entity_state state; + /** Flags. */ + struct { + /** PDCP PDU has 4 byte MAC-I. */ + uint64_t is_authenticated : 1; + /** Cipher offset & length in bits. */ + uint64_t is_cipher_in_bits : 1; + /** Auth offset & length in bits. */ + uint64_t is_auth_in_bits : 1; + /** Is UL/transmitting PDCP entity. */ + uint64_t is_ul_entity : 1; + /** Is NULL auth. */ + uint64_t is_null_auth : 1; + } flags; + /** Crypto op pool. */ + struct rte_mempool *cop_pool; + /** PDCP header size. */ + uint8_t hdr_sz; + /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ + uint8_t aad_sz; + /** Device ID of the device to be used for offload. */ + uint8_t dev_id; +}; + +struct entity_priv_dl_part { + /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ + uint8_t dummy; +}; + +struct entity_priv_ul_part { + /* + * NOTE: when re-establish is supported, plain PDCP packets & COUNT values need to be + * cached. + */ + uint8_t dummy; +}; + +static inline struct entity_priv * +entity_priv_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); +} + +static inline struct entity_priv_dl_part * +entity_dl_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline struct entity_priv_ul_part * +entity_ul_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) +{ + return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); +} + +#endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c new file mode 100644 index 0000000000..79f5dce5db --- /dev/null +++ b/lib/pdcp/pdcp_process.c @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_memcpy.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, + struct rte_crypto_sym_xform **a_xfrm) +{ + *c_xfrm = NULL; + *a_xfrm = NULL; + + if (conf->crypto_xfrm == NULL) + return -EINVAL; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + *c_xfrm = conf->crypto_xfrm; + *a_xfrm = conf->crypto_xfrm->next; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + *a_xfrm = conf->crypto_xfrm; + *c_xfrm = conf->crypto_xfrm->next; + } else { + return -EINVAL; + } + + return 0; +} + +static int +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + int ret; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + /** + * flags.is_authenticated + * + * MAC-I would be added in case of control plane packets and when authentication + * transform is not NULL. + */ + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && (a_xfrm == NULL)) + return -EINVAL; + + if (a_xfrm != NULL) + en_priv->flags.is_authenticated = 1; + + /** + * flags.is_cipher_in_bits + * + * For ZUC & SNOW3G cipher algos, offset & length need to be provided in bits. + */ + + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) + en_priv->flags.is_cipher_in_bits = 1; + + /** + * flags.is_auth_in_bits + * + * For ZUC & SNOW3G authentication algos, offset & length need to be provided in bits. + */ + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) + en_priv->flags.is_auth_in_bits = 1; + } + + /** + * flags.is_ul_entity + * + * Indicate whether the entity is UL/transmitting PDCP entity. + */ + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + en_priv->flags.is_ul_entity = 1; + + /** + * flags.is_null_auth + * + * For NULL auth, 4B zeros need to be added by lib PDCP. Indicate that + * algo is NULL auth to perform the same. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + en_priv->flags.is_null_auth = 1; + + /** + * hdr_sz + * + * PDCP header size of the entity + */ + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); + + /** + * aad_sz + * + * For AES-CMAC, additional message is prepended for processing. Need to be trimmed after + * crypto processing is done. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) + en_priv->aad_sz = 8; + else + en_priv->aad_sz = 0; + + return 0; +} + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv *en_priv; + int ret; + + if (entity == NULL || conf == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + ret = pdcp_entity_priv_populate(en_priv, conf); + if (ret) + return ret; + + return 0; +} diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h new file mode 100644 index 0000000000..fd53fff0aa --- /dev/null +++ b/lib/pdcp/pdcp_process.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_PROCESS_H +#define PDCP_PROCESS_H + +#include <rte_pdcp.h> + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode 100644 index 0000000000..adcad5dd25 --- /dev/null +++ b/lib/pdcp/rte_pdcp.c @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_pdcp.h> +#include <rte_malloc.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +{ + int size; + + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + size += sizeof(struct entity_priv_dl_part); + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size += sizeof(struct entity_priv_ul_part); + else + return -EINVAL; + + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); +} + +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) +{ + struct rte_pdcp_entity *entity = NULL; + struct entity_priv *en_priv; + int ret, entity_size; + uint32_t count; + + if (conf == NULL || conf->cop_pool == NULL) { + rte_errno = EINVAL; + return NULL; + } + + if (conf->pdcp_xfrm.en_ordering || conf->pdcp_xfrm.remove_duplicates || conf->is_slrb || + conf->en_sec_offload) { + rte_errno = ENOTSUP; + return NULL; + } + + /* + * 6.3.2 PDCP SN + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the PDCP SN is + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl-PDCP-SN-Size in + * TS 38.331 [3]) + */ + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { + rte_errno = ENOTSUP; + return NULL; + } + + if (conf->pdcp_xfrm.hfn_threshold) { + rte_errno = EINVAL; + return NULL; + } + + entity_size = pdcp_entity_size_get(conf); + if (entity_size < 0) { + rte_errno = EINVAL; + return NULL; + } + + entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (entity == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + en_priv = entity_priv_get(entity); + + count = pdcp_count_from_hfn_sn_get(conf->pdcp_xfrm.hfn, conf->sn, conf->pdcp_xfrm.sn_size); + + en_priv->state.rx_deliv = count; + en_priv->state.tx_next = count; + en_priv->cop_pool = conf->cop_pool; + + /* Setup crypto session */ + ret = pdcp_crypto_sess_create(entity, conf); + if (ret) + goto entity_free; + + ret = pdcp_process_func_set(entity, conf); + if (ret) + goto crypto_sess_destroy; + + return entity; + +crypto_sess_destroy: + pdcp_crypto_sess_destroy(entity); +entity_free: + rte_free(entity); + rte_errno = -ret; + return NULL; +} + +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) +{ + if (pdcp_entity == NULL) + return -EINVAL; + + /* Teardown crypto sessions */ + pdcp_crypto_sess_destroy(pdcp_entity); + + rte_free(pdcp_entity); + + RTE_SET_USED(out_mb); + return 0; +} + +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]) +{ + struct entity_priv *en_priv; + + if (pdcp_entity == NULL) + return -EINVAL; + + en_priv = entity_priv_get(pdcp_entity); + + if (en_priv->flags.is_ul_entity) { + en_priv->state.tx_next = 0; + } else { + en_priv->state.rx_next = 0; + en_priv->state.rx_deliv = 0; + } + + RTE_SET_USED(out_mb); + + return 0; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode 100644 index 0000000000..1f96fdc9a1 --- /dev/null +++ b/lib/pdcp/rte_pdcp.h @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_H +#define RTE_PDCP_H + +/** + * @file rte_pdcp.h + * + * RTE PDCP support. + * + * A framework for PDCP protocol processing. + */ + +#include <rte_compat.h> +#include <rte_common.h> +#include <rte_mempool.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * PDCP entity. + * + * 4.2.2 PDCP entities + * + * The PDCP entities are located in the PDCP sublayer. Several PDCP entities may + * be defined for a UE. Each PDCP entity is carrying the data of one radio + * bearer. A PDCP entity is associated either to the control plane or the user + * plane depending on which radio bearer it is carrying data for. + */ +struct rte_pdcp_entity { + /** + * PDCP entities may hold packets for purposes of in-order delivery (in + * case of receiving PDCP entity) and re-transmission (in case of + * transmitting PDCP entity). + * + * The field 'max_pkt_cache' would be used to indicate the maximum + * number of packets that may be cached in an entity at any point of + * time. When application provides buffers to receive packets from + * PDCP entity, the size of the buffer should be such that it can + * hold additionally 'max_pkt_cache' number of packets. + */ + uint32_t max_pkt_cache; +} __rte_cache_aligned; + +/** + * PDCP entity configuration to be used for establishing an entity. + */ +/* Structure rte_pdcp_entity_conf 8< */ +struct rte_pdcp_entity_conf { + /** PDCP transform for the entity. */ + struct rte_security_pdcp_xform pdcp_xfrm; + /** Crypto transform applicable for the entity. */ + struct rte_crypto_sym_xform *crypto_xfrm; + /** Mempool for crypto symmetric session. */ + struct rte_mempool *sess_mpool; + /** Crypto op pool.*/ + struct rte_mempool *cop_pool; + /** + * SN value to be used. 32 bit count value to be used for the first + * packet would be derived based on HFN (`rte_security_pdcp_xform.hfn`) + * and SN. + */ + uint32_t sn; + /** + * Indicate whether the PDCP entity belongs to Side Link Radio Bearer. + */ + bool is_slrb; + /** Enable security offload on the device specified. */ + bool en_sec_offload; + /** Device on which security/crypto session need to be created. */ + uint8_t dev_id; + /** + * Reverse direction during IV generation. Can be used to simulate UE + * crypto processing. + */ + bool reverse_iv_direction; +}; +/* >8 End of structure rte_pdcp_entity_conf. */ + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.1 PDCP entity establishment + * + * Establish PDCP entity based on provided input configuration. + * + * @param conf + * Parameters to be used for initializing PDCP entity object. + * @return + * - Valid handle if success + * - NULL in case of failure. rte_errno will be set to error code + */ +__rte_experimental +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.3 PDCP entity release + * + * Release PDCP entity. + * + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * Entity release would result in freeing all memory associated with the PDCP + * entity as well as any crypto/security sessions created. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be released. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.4 PDCP entity suspend + * + * Suspend PDCP entity. + * + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be suspended. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode 100644 index 0000000000..923e165f3f --- /dev/null +++ b/lib/pdcp/version.map @@ -0,0 +1,10 @@ +EXPERIMENTAL { + global: + + # added in 23.07 + rte_pdcp_entity_establish; + rte_pdcp_entity_release; + rte_pdcp_entity_suspend; + + local: *; +}; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 03/22] pdcp: add pre and post-process 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 01/22] net: add PDCP header Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 02/22] lib: add pdcp protocol Anoob Joseph @ 2023-05-24 16:00 ` Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 04/22] pdcp: add packet group Anoob Joseph ` (19 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:00 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Functionality of pre-process & post-process varies based on the type of entity. Registration of entity specific function pointer allows skipping multiple checks that would come in datapath otherwise. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/rte_pdcp.h | 97 ++++++++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 ++ 2 files changed, 100 insertions(+) diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 1f96fdc9a1..46c3c2a416 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -22,6 +22,21 @@ extern "C" { #endif +/* Forward declarations */ +struct rte_pdcp_entity; + +/* PDCP pre-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err); + +/* PDCP post-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err); + /** * PDCP entity. * @@ -33,6 +48,10 @@ extern "C" { * plane depending on which radio bearer it is carrying data for. */ struct rte_pdcp_entity { + /** Entity specific pre-process handle. */ + rte_pdcp_pre_p_t pre_process; + /** Entity specific post-process handle. */ + rte_pdcp_post_p_t post_process; /** * PDCP entities may hold packets for purposes of in-order delivery (in * case of receiving PDCP entity) and re-transmission (in case of @@ -160,6 +179,84 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity pre-process the mbufs and prepare + * crypto ops that can be enqueued to the cryptodev associated with given + * session. Only error packets would be moved returned in the input buffer, + * *mb*, and it is the responsibility of the application to free the same. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param[in, out] mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * which contain the input packets. Any error packets would be returned in the + * same buffer. + * @param[out] cop + * The address of an array that can hold up to *num* pointers to + * *rte_crypto_op* structures. Crypto ops would be allocated by + * ``rte_pdcp_pkt_pre_process`` API. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * Pointer to return the number of error packets returned in *mb* + * @return + * Count of crypto_ops prepared + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err) +{ + return entity->pre_process(entity, mb, cop, num, nb_err); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity, perform PDCP post-processing of the + * mbufs. + * + * Input mbufs are the ones retrieved from crypto_ops dequeued from cryptodev + * and grouped by *rte_pdcp_pkt_crypto_group()*. + * + * The post-processed packets would be returned in the *out_mb* buffer. + * The resultant mbufs would be grouped into success packets and error packets. + * Error packets would be grouped in the end of the array and it is the + * responsibility of the application to handle the same. + * + * When in-order delivery is enabled, PDCP entity may buffer packets and would + * deliver packets only when all prior packets have been post-processed. That + * would result in returning more/less packets than enqueued. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param in_mb + * The address of an array of *num* pointers to *rte_mbuf* structures. + * @param[out] out_mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * to output packets after PDCP post-processing. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * The number of error packets returned in *out_mb* buffer. + * @return + * Count of packets returned in *out_mb* buffer. + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return entity->post_process(entity, in_mb, out_mb, num, nb_err); +} + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 923e165f3f..f9ff30600a 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -6,5 +6,8 @@ EXPERIMENTAL { rte_pdcp_entity_release; rte_pdcp_entity_suspend; + rte_pdcp_pkt_post_process; + rte_pdcp_pkt_pre_process; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 04/22] pdcp: add packet group 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (2 preceding siblings ...) 2023-05-24 16:00 ` [PATCH v3 03/22] pdcp: add pre and post-process Anoob Joseph @ 2023-05-24 16:00 ` Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 05/22] pdcp: add crypto session create and destroy Anoob Joseph ` (18 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:00 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Crypto processing in PDCP is performed asynchronously by rte_cryptodev_enqueue_burst() and rte_cryptodev_dequeue_burst(). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 1 + lib/pdcp/rte_pdcp.h | 2 + lib/pdcp/rte_pdcp_group.h | 131 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 + 4 files changed, 137 insertions(+) create mode 100644 lib/pdcp/rte_pdcp_group.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index ccaf426240..08679b743a 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -13,5 +13,6 @@ sources = files( 'rte_pdcp.c', ) headers = files('rte_pdcp.h') +indirect_headers += files('rte_pdcp_group.h') deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 46c3c2a416..4143fae0b2 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -257,6 +257,8 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +#include <rte_pdcp_group.h> + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new file mode 100644 index 0000000000..ece3e8c0ff --- /dev/null +++ b/lib/pdcp/rte_pdcp_group.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_GROUP_H +#define RTE_PDCP_GROUP_H + +/** + * @file rte_pdcp_group.h + * + * RTE PDCP grouping support. + * It is not recommended to include this file directly, include <rte_pdcp.h> + * instead. + * Provides helper functions to process completed crypto-ops and group related + * packets by sessions they belong to. + */ + +#include <rte_common.h> +#include <rte_crypto.h> +#include <rte_cryptodev.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Group packets belonging to same PDCP entity. + */ +struct rte_pdcp_group { + union { + uint64_t val; + void *ptr; + } id; /**< Grouped by value */ + struct rte_mbuf **m; /**< Start of the group */ + uint32_t cnt; /**< Number of entries in the group */ + int32_t rc; /**< Status code associated with the group */ +}; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take crypto-op as an input and extract pointer to related PDCP entity. + * @param cop + * The address of an input *rte_crypto_op* structure. + * @return + * The pointer to the related *rte_pdcp_entity* structure. + */ +static inline struct rte_pdcp_entity * +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) +{ + void *sess = cop->sym[0].session; + + return (struct rte_pdcp_entity *)(uintptr_t) + rte_cryptodev_sym_session_opaque_data_get(sess); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take as input completed crypto ops, extract related mbufs and group them by + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* flag + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. + * + * Note that application must ensure only crypto-ops prepared by lib_pdcp is + * provided back to @see rte_pdcp_pkt_crypto_group(). + * + * @param cop + * The address of an array of *num* pointers to the input *rte_crypto_op* + * structures. + * @param[out] mb + * The address of an array of *num* pointers to output *rte_mbuf* structures. + * @param[out] grp + * The address of an array of *num* to output *rte_pdcp_group* structures. + * @param num + * The maximum number of crypto-ops to process. + * @return + * Number of filled elements in *grp* array. + * + */ +static inline uint16_t +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_pdcp_group grp[], uint16_t num) +{ + uint32_t i, j = 0, n = 0; + void *ns, *ps = NULL; + struct rte_mbuf *m; + + for (i = 0; i != num; i++) { + m = cop[i]->sym[0].m_src; + ns = cop[i]->sym[0].session; + + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; + + /* Different entity */ + if (ps != ns) { + + /* Finalize open group and start a new one */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + /* Start new group */ + grp[n].m = mb + j; + ps = ns; + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); + } + + mb[j++] = m; + rte_crypto_op_free(cop[i]); + } + + /* Finalize last group */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + return n; +} + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_GROUP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index f9ff30600a..d564f155e0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,12 +2,15 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_en_from_cop; + rte_pdcp_entity_establish; rte_pdcp_entity_release; rte_pdcp_entity_suspend; rte_pdcp_pkt_post_process; rte_pdcp_pkt_pre_process; + rte_pdcp_pkt_crypto_group; local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 05/22] pdcp: add crypto session create and destroy 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (3 preceding siblings ...) 2023-05-24 16:00 ` [PATCH v3 04/22] pdcp: add packet group Anoob Joseph @ 2023-05-24 16:00 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 06/22] pdcp: add pre and post process for UL Anoob Joseph ` (17 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:00 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add routines to create & destroy sessions. PDCP lib would take crypto transforms as input and creates the session on the corresponding device after verifying capabilities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_crypto.c | 223 ++++++++++++++++++++++++++++++++++++++++- lib/pdcp/pdcp_crypto.h | 5 + 2 files changed, 225 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c index 755e27ec9e..17feef43df 100644 --- a/lib/pdcp/pdcp_crypto.c +++ b/lib/pdcp/pdcp_crypto.c @@ -2,20 +2,237 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_errno.h> #include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> #include "pdcp_crypto.h" +#include "pdcp_entity.h" + +static int +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, + const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm, + bool is_auth_then_cipher) +{ + uint16_t ciph_iv_len, auth_digest_len, auth_iv_len; + int ret; + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protection is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (c_xfrm == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + + /* With UPLINK, if auth is enabled, it should be before cipher */ + if (a_xfrm != NULL && !is_auth_then_cipher) + return -EINVAL; + + /* With UPLINK, cipher operation must be encrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return -EINVAL; + + /* With UPLINK, auth operation (if present) must be generate */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_GENERATE) + return -EINVAL; + + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + + /* With DOWNLINK, if auth is enabled, it should be after cipher */ + if (a_xfrm != NULL && is_auth_then_cipher) + return -EINVAL; + + /* With DOWNLINK, cipher operation must be decrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) + return -EINVAL; + + /* With DOWNLINK, auth operation (if present) must be verify */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_VERIFY) + return -EINVAL; + + } else { + return -EINVAL; + } + + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) + return -EINVAL; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + ciph_iv_len = 0; + else + ciph_iv_len = PDCP_IV_LEN; + + if (ciph_iv_len != c_xfrm->cipher.iv.length) + return -EINVAL; + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + return -EINVAL; + + /* For AUTH NULL, lib PDCP would add 4 byte 0s */ + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + auth_digest_len = 0; + else + auth_digest_len = RTE_PDCP_MAC_I_LEN; + + if (auth_digest_len != a_xfrm->auth.digest_length) + return -EINVAL; + + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + auth_iv_len = PDCP_IV_LEN; + else + auth_iv_len = 0; + + if (a_xfrm->auth.iv.length != auth_iv_len) + return -EINVAL; + } + + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) + return -EINVAL; + + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); + if (ret) + return -ENOTSUP; + + if (a_xfrm != NULL) { + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); + if (ret) + return -ENOTSUP; + } + + return 0; +} int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { - RTE_SET_USED(entity); - RTE_SET_USED(conf); + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + struct entity_priv *en_priv; + bool is_auth_then_cipher; + int ret; + + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + en_priv->dev_id = conf->dev_id; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + c_xfrm = conf->crypto_xfrm; + a_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = false; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + a_xfrm = conf->crypto_xfrm; + c_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = true; + } else { + return -EINVAL; + } + + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, is_auth_then_cipher); + if (ret) + return ret; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + c_xfrm->cipher.iv.offset = 0; + else + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; + + if (a_xfrm != NULL) { + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm->auth.iv.offset = 0; + else + if (c_xfrm->cipher.iv.offset) + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + PDCP_IV_LEN; + else + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; + } + + if (conf->sess_mpool == NULL) + return -EINVAL; + + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf->dev_id, conf->crypto_xfrm, + conf->sess_mpool); + if (en_priv->crypto_sess == NULL) { + /* rte_errno is set as positive values of error codes */ + return -rte_errno; + } + + rte_cryptodev_sym_session_opaque_data_set(en_priv->crypto_sess, (uint64_t)entity); + return 0; } void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) { - RTE_SET_USED(entity); + struct entity_priv *en_priv; + + en_priv = entity_priv_get(entity); + + if (en_priv->crypto_sess != NULL) { + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv->crypto_sess); + en_priv->crypto_sess = NULL; + } } diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h index 6563331d37..f694818713 100644 --- a/lib/pdcp/pdcp_crypto.h +++ b/lib/pdcp/pdcp_crypto.h @@ -5,8 +5,13 @@ #ifndef PDCP_CRYPTO_H #define PDCP_CRYPTO_H +#include <rte_crypto.h> +#include <rte_crypto_sym.h> #include <rte_pdcp.h> +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op)) +#define PDCP_IV_LEN 16 + int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 06/22] pdcp: add pre and post process for UL 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (4 preceding siblings ...) 2023-05-24 16:00 ` [PATCH v3 05/22] pdcp: add crypto session create and destroy Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 07/22] pdcp: add pre and post process for DL Anoob Joseph ` (16 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add routines to perform pre & post processing based on the type of entity. To avoid checks in datapath, there are different function pointers registered based on the following, 1. Control plane v/s user plane 2. 12 bit v/s 18 bit SN For control plane only 12 bit SN need to be supported (as per PDCP specification). Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_entity.h | 24 +++ lib/pdcp/pdcp_process.c | 330 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 354 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 000297588f..23628ebad4 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -92,22 +92,46 @@ pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static inline uint32_t +pdcp_window_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return 1 << (sn_size - 1); +} + static inline uint32_t pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return (1 << sn_size) - 1; } +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + static inline uint32_t pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return ~pdcp_sn_mask_get(sn_size); } +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline uint32_t pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) { return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); } +static inline uint32_t +pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << (32 - sn_size)) - 1; +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 79f5dce5db..2132a136d9 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -36,6 +36,332 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static inline void +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, + uint8_t data_offset, uint32_t count, const bool is_auth) +{ + const struct rte_crypto_op cop_init = { + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, + }; + struct rte_crypto_sym_op *op; + uint32_t pkt_len; + + const uint8_t ciph_shift = 3 * en_priv->flags.is_cipher_in_bits; + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; + + op = cop->sym; + cop->raw = cop_init.raw; + op->m_src = mb; + op->m_dst = mb; + + /* Set IV */ + en_priv->iv_gen(cop, en_priv, count); + + /* Prepare op */ + pkt_len = rte_pktmbuf_pkt_len(mb); + op->cipher.data.offset = data_offset << ciph_shift; + op->cipher.data.length = (pkt_len - data_offset) << ciph_shift; + + if (is_auth) { + op->auth.data.offset = 0; + op->auth.data.length = (pkt_len - RTE_PDCP_MAC_I_LEN) << auth_shift; + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t *, + (pkt_len - RTE_PDCP_MAC_I_LEN)); + } + + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); +} + +static inline bool +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + return true; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline bool +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + return true; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count, sn; + uint8_t *mac_i; + int i; + + const uint8_t hdr_sz = en_priv->hdr_sz; + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + /* Update sequence number in the PDU header */ + count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static int +pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + entity->pre_process = NULL; + entity->post_process = NULL; + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + entity->post_process = pdcp_post_process_ul; + } + + if (entity->pre_process == NULL || entity->post_process == NULL) + return -ENOTSUP; + + return 0; +} + static int pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) { @@ -134,5 +460,9 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti if (ret) return ret; + ret = pdcp_pre_post_func_set(entity, conf); + if (ret) + return ret; + return 0; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 07/22] pdcp: add pre and post process for DL 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (5 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 06/22] pdcp: add pre and post process for UL Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 08/22] pdcp: add IV generation routines Anoob Joseph ` (15 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add routines to perform pre & post processing for down link entities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 454 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 456 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 23628ebad4..1d4a43a3bc 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -13,6 +13,8 @@ struct entity_priv; +#define PDCP_HFN_MIN 0 + /* IV generation function based on the entity configuration */ typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count); diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 2132a136d9..3af0df75ef 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -329,9 +329,424 @@ pdcp_post_process_ul(const struct rte_pdcp_entity *entity, return nb_success; } +static inline int +pdcp_sn_count_get(const uint32_t rx_deliv, int32_t rsn, uint32_t *count, + const enum rte_security_pdcp_sn_size sn_size) +{ + const uint32_t rx_deliv_sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + const uint32_t window_sz = pdcp_window_size_get(sn_size); + uint32_t rhfn; + + rhfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + + if (rsn < (int32_t)(rx_deliv_sn - window_sz)) { + if (unlikely(rhfn == pdcp_hfn_max(sn_size))) + return -ERANGE; + rhfn += 1; + } else if ((uint32_t)rsn >= (rx_deliv_sn + window_sz)) { + if (unlikely(rhfn == PDCP_HFN_MIN)) + return -ERANGE; + rhfn -= 1; + } + + *count = pdcp_count_from_hfn_sn_get(rhfn, rsn, sn_size); + + return 0; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + } + + if (unlikely(nb_err)) + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_18))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline void +pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool trim_mac) +{ + char *p = rte_pktmbuf_adj(mb, hdr_trim_sz); + RTE_ASSERT(p != NULL); + RTE_SET_USED(p); + + if (trim_mac) { + int ret = rte_pktmbuf_trim(mb, RTE_PDCP_MAC_I_LEN); + RTE_ASSERT(ret == 0); + RTE_SET_USED(ret); + } +} + +static inline bool +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, + const uint32_t count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + + if (count < en_priv->state.rx_deliv) + return false; + + /* t-Reordering timer is not supported - SDU will be delivered immediately. + * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not + * been delivered to upper layers + */ + en_priv->state.rx_next = count + 1; + + if (count >= en_priv->state.rx_next) + en_priv->state.rx_next = count + 1; + + return true; +} + +static inline uint16_t +pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + int i, nb_success = 0, nb_err = 0, rsn = 0; + const uint32_t aad_sz = en_priv->aad_sz; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *, + aad_sz); + + RTE_ASSERT(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + + if (unlikely(pdcp_sn_count_get(en_priv->state.rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_12_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static inline uint16_t +pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + int32_t rsn = 0; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *, + aad_sz); + + RTE_ASSERT(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA); + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | (pdu_hdr->sn_7_0)); + + if (unlikely(pdcp_sn_count_get(en_priv->state.rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_18))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_sn_18_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static uint16_t +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + pdu_hdr = rte_pktmbuf_mtod_offset(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *, + aad_sz); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + + if (unlikely(pdcp_sn_count_get(en_priv->state.rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) + goto error; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, true); + + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + static int pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { + struct entity_priv *en_priv = entity_priv_get(entity); + entity->pre_process = NULL; entity->post_process = NULL; @@ -342,6 +757,13 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; + entity->post_process = pdcp_post_process_cplane_sn_12_dl; + } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { @@ -356,6 +778,38 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; + entity->post_process = pdcp_post_process_uplane_sn_12_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; + entity->post_process = pdcp_post_process_uplane_sn_12_dl; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; + entity->post_process = pdcp_post_process_uplane_sn_18_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; + entity->post_process = pdcp_post_process_uplane_sn_18_dl; + } + if (entity->pre_process == NULL || entity->post_process == NULL) return -ENOTSUP; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 08/22] pdcp: add IV generation routines 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (6 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 07/22] pdcp: add pre and post process for DL Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 09/22] app/test: add lib pdcp tests Anoob Joseph ` (14 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz For PDCP, IV generated has varying formats depending on the ciphering and authentication algorithm used. Add routines to populate IV accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 87 ++++++++++++ lib/pdcp/pdcp_process.c | 284 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 371 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 1d4a43a3bc..10a72faae1 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -26,6 +26,89 @@ struct entity_state { uint32_t rx_reord; }; +union auth_iv_partial { + /* For AES-CMAC, there is no IV, but message gets prepended */ + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + } aes_cmac; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_37_39 : 3; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_65_71 : 7; + uint64_t direction_64 : 1; + uint64_t rsvd_72_111 : 40; + uint64_t rsvd_113_119 : 7; + uint64_t direction_112 : 1; + uint64_t rsvd_120_127 : 8; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t zero_37_39 : 3; + uint64_t zero_40_63 : 24; + + uint64_t direction_64 : 1; + uint64_t rsvd_65_71 : 7; + uint64_t rsvd_72_111 : 40; + uint64_t direction_112 : 1; + uint64_t rsvd_113_119 : 7; + uint64_t rsvd_120_127 : 8; +#endif + } zs; + uint64_t u64[2]; +}; + +union cipher_iv_partial { + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t zero_64_127; + } aes_ctr; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t rsvd_64_127; + } zs; + uint64_t u64[2]; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -35,6 +118,10 @@ struct entity_priv { struct rte_cryptodev_sym_session *crypto_sess; /** Entity specific IV generation function. */ iv_gen_t iv_gen; + /** Pre-prepared auth IV. */ + union auth_iv_partial auth_iv_part; + /** Pre-prepared cipher IV. */ + union cipher_iv_partial cipher_iv_part; /** Entity state variables. */ struct entity_state state; /** Flags. */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 3af0df75ef..e8112adfb8 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -13,6 +13,181 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +/* Enum of supported algorithms for ciphering */ +enum pdcp_cipher_algo { + PDCP_CIPHER_ALGO_NULL, + PDCP_CIPHER_ALGO_AES, + PDCP_CIPHER_ALGO_ZUC, + PDCP_CIPHER_ALGO_SNOW3G, + PDCP_CIPHER_ALGO_MAX +}; + +/* Enum of supported algorithms for integrity */ +enum pdcp_auth_algo { + PDCP_AUTH_ALGO_NULL, + PDCP_AUTH_ALGO_AES, + PDCP_AUTH_ALGO_ZUC, + PDCP_AUTH_ALGO_SNOW3G, + PDCP_AUTH_ALGO_MAX +}; + +/* IV generation functions based on type of operation (cipher - auth) */ + +static void +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + /* No IV required for NULL cipher + NULL auth */ + RTE_SET_USED(cop); + RTE_SET_USED(en_priv); + RTE_SET_USED(count); +} + +static void +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr; + uint64_t m; + + /* AES-CMAC requires message to be prepended with info on count etc */ + + /* Prepend by 8 bytes to add custom message */ + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, 16); +} + +static void +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64 = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64, 8); + rte_memcpy(iv + 8, &iv_u64, 8); +} + +static void +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + /* Generating cipher IV */ + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + iv += PDCP_IV_LEN; + + /* Generating auth IV */ + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + iv += PDCP_IV_LEN; + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + static int pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, struct rte_crypto_sym_xform **a_xfrm) @@ -36,6 +211,111 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static int +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + enum rte_security_pdcp_direction direction; + enum pdcp_cipher_algo ciph_algo; + enum pdcp_auth_algo auth_algo; + struct entity_priv *en_priv; + int ret; + + en_priv = entity_priv_get(entity); + + direction = conf->pdcp_xfrm.pkt_dir; + if (conf->reverse_iv_direction) + direction = !direction; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (c_xfrm == NULL) + return -EINVAL; + + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); + + switch (c_xfrm->cipher.algo) { + case RTE_CRYPTO_CIPHER_NULL: + ciph_algo = PDCP_CIPHER_ALGO_NULL; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + ciph_algo = PDCP_CIPHER_ALGO_AES; + en_priv->cipher_iv_part.aes_ctr.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.aes_ctr.direction = direction; + break; + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + ciph_algo = PDCP_CIPHER_ALGO_SNOW3G; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + ciph_algo = PDCP_CIPHER_ALGO_ZUC; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + default: + return -ENOTSUP; + } + + if (a_xfrm != NULL) { + switch (a_xfrm->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + auth_algo = PDCP_AUTH_ALGO_NULL; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + auth_algo = PDCP_AUTH_ALGO_AES; + en_priv->auth_iv_part.aes_cmac.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.aes_cmac.direction = direction; + break; + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + auth_algo = PDCP_AUTH_ALGO_SNOW3G; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + case RTE_CRYPTO_AUTH_ZUC_EIA3: + auth_algo = PDCP_AUTH_ALGO_ZUC; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + default: + return -ENOTSUP; + } + } else { + auth_algo = PDCP_AUTH_ALGO_NULL; + } + + static const iv_gen_t iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_null_null, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_null_aes_cmac, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_null_zs, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_null_zs, + + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_aes_ctr_null, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_aes_ctr_aes_cmac, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_aes_ctr_zs, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_aes_ctr_zs, + + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + }; + + en_priv->iv_gen = iv_gen_map[ciph_algo][auth_algo]; + + return 0; +} + static inline void cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, uint8_t data_offset, uint32_t count, const bool is_auth) @@ -910,6 +1190,10 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti en_priv = entity_priv_get(entity); + ret = pdcp_iv_gen_func_set(entity, conf); + if (ret) + return ret; + ret = pdcp_entity_priv_populate(en_priv, conf); if (ret) return ret; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 09/22] app/test: add lib pdcp tests 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (7 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 08/22] pdcp: add IV generation routines Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph ` (13 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add tests to verify lib PDCP operations. Tests leverage existing PDCP test vectors. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/meson.build | 1 + app/test/test_cryptodev.h | 3 + app/test/test_pdcp.c | 730 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 734 insertions(+) create mode 100644 app/test/test_pdcp.c diff --git a/app/test/meson.build b/app/test/meson.build index b9b5432496..24e9a124fc 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -96,6 +96,7 @@ test_sources = files( 'test_meter.c', 'test_mcslock.c', 'test_mp_secondary.c', + 'test_pdcp.c', 'test_per_lcore.c', 'test_pflock.c', 'test_pmd_perf.c', diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h index abd795f54a..89057dba22 100644 --- a/app/test/test_cryptodev.h +++ b/app/test/test_cryptodev.h @@ -4,6 +4,9 @@ #ifndef TEST_CRYPTODEV_H_ #define TEST_CRYPTODEV_H_ +#include <rte_crypto.h> +#include <rte_cryptodev.h> + #define HEX_DUMP 0 #define FALSE 0 diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c new file mode 100644 index 0000000000..22db7020c4 --- /dev/null +++ b/app/test/test_pdcp.c @@ -0,0 +1,730 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_malloc.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "test.h" +#include "test_cryptodev.h" +#include "test_cryptodev_security_pdcp_test_vectors.h" + +#define NB_DESC 1024 +#define CDEV_INVALID_ID UINT8_MAX +#define NB_TESTS RTE_DIM(pdcp_test_params) +#define PDCP_IV_LEN 16 + +struct pdcp_testsuite_params { + struct rte_mempool *mbuf_pool; + struct rte_mempool *cop_pool; + struct rte_mempool *sess_pool; + bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; +}; + +static struct pdcp_testsuite_params testsuite_params; + +struct pdcp_test_conf { + struct rte_pdcp_entity_conf entity; + struct rte_crypto_sym_xform c_xfrm; + struct rte_crypto_sym_xform a_xfrm; + bool is_integrity_protected; + uint8_t input[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t input_len; + uint8_t output[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t output_len; +}; + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static int +cryptodev_init(int dev_id) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_cryptodev_qp_conf qp_conf; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_config config; + int ret, socket_id; + + /* Check if device was already initialized */ + if (ts_params->cdevs_used[dev_id]) + return 0; + + rte_cryptodev_info_get(dev_id, &dev_info); + + if (dev_info.max_nb_queue_pairs < 1) { + RTE_LOG(ERR, USER1, "Cryptodev doesn't have sufficient queue pairs available\n"); + return -ENODEV; + } + + socket_id = rte_socket_id(); + + memset(&config, 0, sizeof(config)); + config.nb_queue_pairs = 1; + config.socket_id = socket_id; + + ret = rte_cryptodev_configure(dev_id, &config); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure cryptodev - %d\n", dev_id); + return -ENODEV; + } + + memset(&qp_conf, 0, sizeof(qp_conf)); + qp_conf.nb_descriptors = NB_DESC; + + ret = rte_cryptodev_queue_pair_setup(dev_id, 0, &qp_conf, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure queue pair\n"); + return -ENODEV; + } + + ret = rte_cryptodev_start(dev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not start cryptodev\n"); + return -ENODEV; + } + + /* Mark device as initialized */ + ts_params->cdevs_used[dev_id] = true; + + return 0; +} + +static void +cryptodev_fini(int dev_id) +{ + rte_cryptodev_stop(dev_id); +} + +static unsigned int +cryptodev_sess_priv_max_req_get(void) +{ + struct rte_cryptodev_info info; + unsigned int sess_priv_sz; + int i, nb_dev; + void *sec_ctx; + + nb_dev = rte_cryptodev_count(); + + sess_priv_sz = 0; + + for (i = 0; i < nb_dev; i++) { + rte_cryptodev_info_get(i, &info); + sess_priv_sz = RTE_MAX(sess_priv_sz, rte_cryptodev_sym_get_private_session_size(i)); + if (info.feature_flags & RTE_CRYPTODEV_FF_SECURITY) { + sec_ctx = rte_cryptodev_get_sec_ctx(i); + sess_priv_sz = RTE_MAX(sess_priv_sz, + rte_security_session_get_size(sec_ctx)); + } + } + + return sess_priv_sz; +} + +static int +testsuite_setup(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + int nb_cdev, sess_priv_size, nb_sess = 1024; + + RTE_SET_USED(pdcp_test_hfn_threshold); + + nb_cdev = rte_cryptodev_count(); + if (nb_cdev < 1) { + RTE_LOG(ERR, USER1, "No crypto devices found.\n"); + return TEST_SKIPPED; + } + + memset(ts_params, 0, sizeof(*ts_params)); + + ts_params->mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NUM_MBUFS, MBUF_CACHE_SIZE, 0, + MBUF_SIZE, SOCKET_ID_ANY); + if (ts_params->mbuf_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf pool\n"); + return TEST_FAILED; + } + + ts_params->cop_pool = rte_crypto_op_pool_create("cop_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, MBUF_CACHE_SIZE, + 2 * MAXIMUM_IV_LENGTH, SOCKET_ID_ANY); + if (ts_params->cop_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create crypto_op pool\n"); + goto mbuf_pool_free; + } + + /* Get max session priv size required */ + sess_priv_size = cryptodev_sess_priv_max_req_get(); + + ts_params->sess_pool = rte_cryptodev_sym_session_pool_create("sess_pool", nb_sess, + sess_priv_size, + RTE_MEMPOOL_CACHE_MAX_SIZE, + 0, SOCKET_ID_ANY); + if (ts_params->sess_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create session pool\n"); + goto cop_pool_free; + } + + return 0; + +cop_pool_free: + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; +mbuf_pool_free: + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; + return TEST_FAILED; +} + +static void +testsuite_teardown(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + uint8_t dev_id; + + for (dev_id = 0; dev_id < RTE_CRYPTO_MAX_DEVS; dev_id++) { + if (ts_params->cdevs_used[dev_id]) + cryptodev_fini(dev_id); + } + + rte_mempool_free(ts_params->sess_pool); + ts_params->sess_pool = NULL; + + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; + + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; +} + +static int +ut_setup_pdcp(void) +{ + return 0; +} + +static void +ut_teardown_pdcp(void) +{ +} + +static int +crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +cryptodev_id_get(bool is_integrity_protected, const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm) +{ + int i, nb_devs; + + nb_devs = rte_cryptodev_count(); + + /* Check capabilities */ + + for (i = 0; i < nb_devs; i++) { + if ((crypto_caps_cipher_verify(i, c_xfrm) == 0) && + (!is_integrity_protected || crypto_caps_auth_verify(i, a_xfrm) == 0)) + break; + } + + if (i == nb_devs) + return -1; + + return i; +} + +static int +pdcp_known_vec_verify(struct rte_mbuf *m, const uint8_t *expected, uint32_t expected_pkt_len) +{ + uint8_t *actual = rte_pktmbuf_mtod(m, uint8_t *); + uint32_t actual_pkt_len = rte_pktmbuf_pkt_len(m); + + debug_hexdump(stdout, "Received:", actual, actual_pkt_len); + debug_hexdump(stdout, "Expected:", expected, expected_pkt_len); + + TEST_ASSERT_EQUAL(actual_pkt_len, expected_pkt_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_pkt_len, actual_pkt_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(actual, expected, expected_pkt_len, + "Generated packet not as expected"); + + return 0; +} + +static struct rte_crypto_op * +process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op) +{ + if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) { + RTE_LOG(ERR, USER1, "Error sending packet to cryptodev\n"); + return NULL; + } + + op = NULL; + + while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0) + rte_pause(); + + return op; +} + +static uint32_t +pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) +{ + uint32_t sn = 0; + + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + sn = rte_cpu_to_be_16(*(const uint16_t *)data); + sn = sn & 0xfff; + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + sn = rte_cpu_to_be_32(*(const uint32_t *)data); + sn = (sn & 0x3ffff00) >> 8; + } + + return sn; +} + +static int +create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_crypto_sym_xform c_xfrm, a_xfrm; + uint32_t sn, expected_len; + uint8_t *data, *expected; + int pdcp_hdr_sz; + + memset(conf, 0, sizeof(*conf)); + memset(&c_xfrm, 0, sizeof(c_xfrm)); + memset(&a_xfrm, 0, sizeof(a_xfrm)); + + conf->entity.sess_mpool = ts_params->sess_pool; + conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; + conf->entity.pdcp_xfrm.en_ordering = 0; + conf->entity.pdcp_xfrm.remove_duplicates = 0; + conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + + if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; + else + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + + conf->entity.pdcp_xfrm.sn_size = pdcp_test_data_sn_size[index]; + + /* Zero initialize unsupported flags */ + conf->entity.pdcp_xfrm.hfn_threshold = 0; + conf->entity.pdcp_xfrm.hfn_ovrd = 0; + conf->entity.pdcp_xfrm.sdap_enabled = 0; + + c_xfrm.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + c_xfrm.cipher.algo = pdcp_test_params[index].cipher_alg; + c_xfrm.cipher.key.length = pdcp_test_params[index].cipher_key_len; + c_xfrm.cipher.key.data = pdcp_test_crypto_key[index]; + + a_xfrm.type = RTE_CRYPTO_SYM_XFORM_AUTH; + + if (pdcp_test_params[index].auth_alg == 0) { + conf->is_integrity_protected = false; + } else { + a_xfrm.auth.algo = pdcp_test_params[index].auth_alg; + a_xfrm.auth.key.data = pdcp_test_auth_key[index]; + a_xfrm.auth.key.length = pdcp_test_params[index].auth_key_len; + conf->is_integrity_protected = true; + } + + pdcp_hdr_sz = pdcp_hdr_size_get(pdcp_test_data_sn_size[index]); + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protecting is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (conf->is_integrity_protected) { + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + conf->entity.crypto_xfrm = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + a_xfrm.next = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + c_xfrm.next = NULL; + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + c_xfrm.next = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + a_xfrm.next = NULL; + } + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + c_xfrm.next = NULL; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + else + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + /* Update xforms to match PDCP requirements */ + + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) + c_xfrm.cipher.iv.length = PDCP_IV_LEN; + else + c_xfrm.cipher.iv.length = 0; + + if (conf->is_integrity_protected) { + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm.auth.digest_length = 0; + else + a_xfrm.auth.digest_length = RTE_PDCP_MAC_I_LEN; + + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + a_xfrm.auth.iv.length = PDCP_IV_LEN; + else + a_xfrm.auth.iv.length = 0; + } + + conf->c_xfrm = c_xfrm; + conf->a_xfrm = a_xfrm; + + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf->is_integrity_protected, + &conf->c_xfrm, &conf->a_xfrm); + + if (pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_CONTROL || + pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_DATA) { + data = pdcp_test_data_in[index]; + sn = pdcp_sn_from_raw_get(data, pdcp_test_data_sn_size[index]); + conf->entity.pdcp_xfrm.hfn = pdcp_test_hfn[index]; + conf->entity.sn = sn; + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", pdcp_test_data_in[index], + pdcp_test_data_in_len[index]); +#endif + /* Since the vectors available already have PDCP header, trim the same */ + conf->input_len = pdcp_test_data_in_len[index] - pdcp_hdr_sz; + memcpy(conf->input, pdcp_test_data_in[index] + pdcp_hdr_sz, conf->input_len); + } else { + conf->input_len = pdcp_test_data_in_len[index]; + + if (conf->is_integrity_protected) + conf->input_len += RTE_PDCP_MAC_I_LEN; + + memcpy(conf->input, pdcp_test_data_out[index], conf->input_len); +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", conf->input, conf->input_len); +#endif + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected = pdcp_test_data_out[index]; + else + expected = pdcp_test_data_in[index]; + + /* Calculate expected packet length */ + expected_len = pdcp_test_data_in_len[index]; + + /* In DL processing, PDCP header would be stripped */ + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + expected += pdcp_hdr_sz; + expected_len -= pdcp_hdr_sz; + } + + /* In UL processing with integrity protection, MAC would be added */ + if (conf->is_integrity_protected && + conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected_len += 4; + + memcpy(conf->output, expected, expected_len); + conf->output_len = expected_len; + + return 0; +} + +static struct rte_pdcp_entity* +test_entity_create(const struct pdcp_test_conf *t_conf, int *rc) +{ + struct rte_pdcp_entity *pdcp_entity; + int ret; + + if (t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12 && + t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18) { + *rc = -ENOTSUP; + return NULL; + } + + if (t_conf->entity.dev_id == CDEV_INVALID_ID) { + RTE_LOG(DEBUG, USER1, "Could not find device with required capabilities\n"); + *rc = -ENOTSUP; + return NULL; + } + + ret = cryptodev_init(t_conf->entity.dev_id); + if (ret) { + *rc = ret; + RTE_LOG(DEBUG, USER1, "Could not initialize cryptodev\n"); + return NULL; + } + + rte_errno = 0; + + pdcp_entity = rte_pdcp_entity_establish(&t_conf->entity); + if (pdcp_entity == NULL) { + *rc = -rte_errno; + RTE_LOG(DEBUG, USER1, "Could not establish PDCP entity\n"); + return NULL; + } + + return pdcp_entity; +} + +static uint16_t +test_process_packets(const struct rte_pdcp_entity *pdcp_entity, uint8_t cdev_id, + struct rte_mbuf *in_mb[], uint16_t nb_in, + struct rte_mbuf *out_mb[], uint16_t *nb_err) +{ + struct rte_crypto_op *cop, *cop_out; + struct rte_pdcp_group grp[1]; + uint16_t nb_success, nb_grp; + struct rte_mbuf *mbuf, *mb; + + if (nb_in != 1) + return -ENOTSUP; + + mbuf = in_mb[0]; + + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, &mbuf, &cop_out, 1, nb_err); + if (nb_success != 1 || *nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not pre process PDCP packet\n"); + return TEST_FAILED; + } + +#ifdef VEC_DUMP + printf("Pre-processed vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, rte_pktmbuf_pkt_len(mbuf)); +#endif + + cop = process_crypto_request(cdev_id, cop_out); + if (cop == NULL) { + RTE_LOG(ERR, USER1, "Could not process crypto request\n"); + return -EIO; + } + + nb_grp = rte_pdcp_pkt_crypto_group(&cop_out, &mb, grp, 1); + if (nb_grp != 1 || grp[0].cnt != 1) { + RTE_LOG(ERR, USER1, "Could not group PDCP crypto results\n"); + return -ENOTRECOVERABLE; + } + + if ((uintptr_t)pdcp_entity != grp[0].id.val) { + RTE_LOG(ERR, USER1, "PDCP entity not matching the one from crypto_op\n"); + return -ENOTRECOVERABLE; + } + +#ifdef VEC_DUMP + printf("Crypto processed vector:\n"); + rte_pktmbuf_dump(stdout, cop->sym->m_dst, rte_pktmbuf_pkt_len(mbuf)); +#endif + + return rte_pdcp_pkt_post_process(grp[0].id.ptr, grp[0].m, out_mb, grp[0].cnt, nb_err); +} + +static struct rte_mbuf* +mbuf_from_data_create(uint8_t *data, uint16_t data_len) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_mbuf *mbuf; + uint8_t *input_text; + + mbuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (mbuf == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf\n"); + return NULL; + } + + memset(rte_pktmbuf_mtod(mbuf, uint8_t *), 0, rte_pktmbuf_tailroom(mbuf)); + + input_text = (uint8_t *)rte_pktmbuf_append(mbuf, data_len); + memcpy(input_text, data, data_len); + + return mbuf; +} + +static int +test_attempt_single(struct pdcp_test_conf *t_conf) +{ + struct rte_mbuf *mbuf, **out_mb = NULL; + struct rte_pdcp_entity *pdcp_entity; + uint16_t nb_success, nb_err; + int ret = 0, nb_max_out_mb; + + pdcp_entity = test_entity_create(t_conf, &ret); + if (pdcp_entity == NULL) + goto exit; + + /* Allocate buffer for holding mbufs returned */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate buffer for holding out_mb buffers\n"); + ret = -ENOMEM; + goto entity_release; + } + + mbuf = mbuf_from_data_create(t_conf->input, t_conf->input_len); + if (mbuf == NULL) { + ret = -ENOMEM; + goto entity_release; + } + +#ifdef VEC_DUMP + printf("Adjusted vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, t_conf->input_len); +#endif + + nb_success = test_process_packets(pdcp_entity, t_conf->entity.dev_id, &mbuf, 1, out_mb, + &nb_err); + if (nb_success != 1 || nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not process PDCP packet\n"); + ret = TEST_FAILED; + goto mbuf_free; + } + + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + + ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); + if (ret) { + RTE_LOG(DEBUG, USER1, "Could not suspend PDCP entity\n"); + goto mbuf_free; + } + +mbuf_free: + rte_pktmbuf_free(mbuf); +entity_release: + rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_free(out_mb); +exit: + return ret; +} + +static int +run_test_for_one_known_vec(const void *arg) +{ + struct pdcp_test_conf test_conf; + int i = *(const uint32_t *)arg; + + create_test_conf_from_index(i, &test_conf); + return test_attempt_single(&test_conf); +} + +struct unit_test_suite *test_suites[] = { + NULL, /* Place holder for known_vector_cases */ + NULL /* End of suites list */ +}; + +static struct unit_test_suite pdcp_testsuite = { + .suite_name = "PDCP Unit Test Suite", + .unit_test_cases = {TEST_CASES_END()}, + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_suites = test_suites, +}; + +static int +test_pdcp(void) +{ + struct unit_test_suite *known_vector_cases; + int ret, index[NB_TESTS]; + uint32_t i, size; + + size = sizeof(struct unit_test_suite); + size += (NB_TESTS + 1) * sizeof(struct unit_test_case); + + known_vector_cases = rte_zmalloc(NULL, size, 0); + if (known_vector_cases == NULL) + return TEST_FAILED; + + known_vector_cases->suite_name = "Known vector cases"; + + for (i = 0; i < NB_TESTS; i++) { + index[i] = i; + known_vector_cases->unit_test_cases[i].name = pdcp_test_params[i].name; + known_vector_cases->unit_test_cases[i].data = (void *)&index[i]; + known_vector_cases->unit_test_cases[i].enabled = 1; + known_vector_cases->unit_test_cases[i].setup = ut_setup_pdcp; + known_vector_cases->unit_test_cases[i].teardown = ut_teardown_pdcp; + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data + = run_test_for_one_known_vec; + } + + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data = NULL; + + test_suites[0] = known_vector_cases; + + ret = unit_test_suite_runner(&pdcp_testsuite); + + rte_free(known_vector_cases); + return ret; +} + +REGISTER_TEST_COMMAND(pdcp_autotest, test_pdcp); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 10/22] test/pdcp: pdcp HFN tests in combined mode 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (8 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 09/22] app/test: add lib pdcp tests Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 11/22] doc: add PDCP library guide Anoob Joseph ` (12 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add tests to verify HFN/SN behaviour. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 302 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 299 insertions(+), 3 deletions(-) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 22db7020c4..ed2dfe921c 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,9 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* According to formula(7.2.a Window_Size) */ +#define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) + struct pdcp_testsuite_params { struct rte_mempool *mbuf_pool; struct rte_mempool *cop_pool; @@ -36,12 +39,69 @@ struct pdcp_test_conf { uint32_t output_len; }; +static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf); + +typedef int (*test_with_conf_t)(struct pdcp_test_conf *conf); + +static int +run_test_foreach_known_vec(test_with_conf_t test, bool stop_on_first_pass) +{ + struct pdcp_test_conf test_conf; + bool all_tests_skipped = true; + uint32_t i; + int ret; + + for (i = 0; i < NB_TESTS; i++) { + create_test_conf_from_index(i, &test_conf); + ret = test(&test_conf); + + if (ret == TEST_FAILED) { + printf("[%03i] - %s - failed\n", i, pdcp_test_params[i].name); + return TEST_FAILED; + } + + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + continue; + + if (stop_on_first_pass) + return TEST_SUCCESS; + + all_tests_skipped = false; + } + + if (all_tests_skipped) + return TEST_SKIPPED; + + return TEST_SUCCESS; +} + +static int +run_test_with_all_known_vec(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, false); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static int +pktmbuf_read_into(const struct rte_mbuf *m, void *buf, size_t buf_len) +{ + if (m->pkt_len > buf_len) + return -ENOMEM; + + const void *read = rte_pktmbuf_read(m, 0, m->pkt_len, buf); + if (read != NULL && read != buf) + memcpy(buf, read, m->pkt_len); + + return 0; +} + static int cryptodev_init(int dev_id) { @@ -326,6 +386,21 @@ pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) return sn; } +static void +pdcp_sn_to_raw_set(void *data, uint32_t sn, int size) +{ + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr = data; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr = data; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } +} + static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) { @@ -646,9 +721,17 @@ test_attempt_single(struct pdcp_test_conf *t_conf) goto mbuf_free; } - ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); - if (ret) - goto mbuf_free; + /* If expected output provided - verify, else - store for future use */ + if (t_conf->output_len) { + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + } else { + ret = pktmbuf_read_into(mbuf, t_conf->output, RTE_PDCP_CTRL_PDU_SIZE_MAX); + if (ret) + goto mbuf_free; + t_conf->output_len = mbuf->pkt_len; + } ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); if (ret) { @@ -665,6 +748,193 @@ test_attempt_single(struct pdcp_test_conf *t_conf) return ret; } +static void +uplink_to_downlink_convert(const struct pdcp_test_conf *ul_cfg, + struct pdcp_test_conf *dl_cfg) +{ + assert(ul_cfg->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK); + + memcpy(dl_cfg, ul_cfg, sizeof(*dl_cfg)); + dl_cfg->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + dl_cfg->entity.reverse_iv_direction = false; + + if (dl_cfg->is_integrity_protected) { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + dl_cfg->c_xfrm.next = &dl_cfg->a_xfrm; + + dl_cfg->a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + dl_cfg->a_xfrm.next = NULL; + } else { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + dl_cfg->c_xfrm.next = NULL; + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + dl_cfg->entity.dev_id = (uint8_t)cryptodev_id_get(dl_cfg->is_integrity_protected, + &dl_cfg->c_xfrm, &dl_cfg->a_xfrm); + + memcpy(dl_cfg->input, ul_cfg->output, ul_cfg->output_len); + dl_cfg->input_len = ul_cfg->output_len; + + memcpy(dl_cfg->output, ul_cfg->input, ul_cfg->input_len); + dl_cfg->output_len = ul_cfg->input_len; +} + +/* + * According to ETSI TS 138 323 V17.1.0, Section 5.2.2.1, + * SN could be divided into following ranges, + * relatively to current value of RX_DELIV state: + * +-------------+-------------+-------------+-------------+ + * | -Outside | -Window | +Window | +Outside | + * | (valid) | (Invalid) | (Valid) | (Invalid) | + * +-------------+-------------^-------------+-------------+ + * | + * v + * SN(RX_DELIV) + */ +enum sn_range_type { + SN_RANGE_MINUS_OUTSIDE, + SN_RANGE_MINUS_WINDOW, + SN_RANGE_PLUS_WINDOW, + SN_RANGE_PLUS_OUTSIDE, +}; + +#define PDCP_SET_COUNT(hfn, sn, size) ((hfn << size) | (sn & ((1 << size) - 1))) + +/* + * Take uplink test case as base, modify RX_DELIV in state and SN in input + */ +static int +test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) +{ + uint32_t rx_deliv_hfn, rx_deliv_sn, new_hfn, new_sn; + const int domain = conf->entity.pdcp_xfrm.domain; + struct pdcp_test_conf dl_conf; + int ret, expected_ret; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + if (domain != RTE_SECURITY_PDCP_MODE_CONTROL && domain != RTE_SECURITY_PDCP_MODE_DATA) + return TEST_SKIPPED; + + const uint32_t sn_size = conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + /* Max value of SN that could fit in `sn_size` bits */ + const uint32_t max_sn = (1 << sn_size) - 1; + const uint32_t shift = (max_sn - window_size) / 2; + /* Could be any number up to `shift` value */ + const uint32_t default_sn = RTE_MIN(2u, shift); + + /* Initialize HFN as non zero value, to be able check values before */ + rx_deliv_hfn = 0xa; + + switch (type) { + case SN_RANGE_PLUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + 1; + expected_ret = TEST_SUCCESS; + break; + case SN_RANGE_MINUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn - 1; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_PLUS_OUTSIDE: + /* RCVD_SN >= SN(RX_DELIV) + Window_Size */ + new_hfn = rx_deliv_hfn - 1; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + window_size; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_MINUS_OUTSIDE: + /* RCVD_SN < SN(RX_DELIV) - Window_Size */ + new_hfn = rx_deliv_hfn + 1; + rx_deliv_sn = window_size + default_sn; + new_sn = rx_deliv_sn - window_size - 1; + expected_ret = TEST_SUCCESS; + break; + default: + return TEST_FAILED; + } + + /* Configure Uplink to generate expected, encrypted packet */ + pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.reverse_iv_direction = true; + conf->entity.pdcp_xfrm.hfn = new_hfn; + conf->entity.sn = new_sn; + conf->output_len = 0; + ret = test_attempt_single(conf); + if (ret != TEST_SUCCESS) + return ret; + + /* Flip configuration to downlink */ + uplink_to_downlink_convert(conf, &dl_conf); + + /* Modify the rx_deliv to verify the expected behaviour */ + dl_conf.entity.pdcp_xfrm.hfn = rx_deliv_hfn; + dl_conf.entity.sn = rx_deliv_sn; + ret = test_attempt_single(&dl_conf); + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + return ret; + + TEST_ASSERT_EQUAL(ret, expected_ret, "Unexpected result"); + + return TEST_SUCCESS; +} + +static int +test_sn_plus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_WINDOW, t_conf); +} + +static int +test_sn_minus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_WINDOW, t_conf); +} + +static int +test_sn_plus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_OUTSIDE, t_conf); +} + +static int +test_sn_minus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); +} + +static int +test_combined(struct pdcp_test_conf *ul_conf) +{ + struct pdcp_test_conf dl_conf; + int ret; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + ul_conf->entity.reverse_iv_direction = true; + ul_conf->output_len = 0; + + ret = test_attempt_single(ul_conf); + if (ret != TEST_SUCCESS) + return ret; + + uplink_to_downlink_convert(ul_conf, &dl_conf); + ret = test_attempt_single(&dl_conf); + + return ret; +} + static int run_test_for_one_known_vec(const void *arg) { @@ -675,8 +945,34 @@ run_test_for_one_known_vec(const void *arg) return test_attempt_single(&test_conf); } +static struct unit_test_suite combined_mode_cases = { + .suite_name = "PDCP combined mode", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("combined mode", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_combined), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static struct unit_test_suite hfn_sn_test_cases = { + .suite_name = "PDCP HFN/SN", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("SN plus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_window), + TEST_CASE_NAMED_WITH_DATA("SN minus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_window), + TEST_CASE_NAMED_WITH_DATA("SN plus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_outside), + TEST_CASE_NAMED_WITH_DATA("SN minus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_outside), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ + &combined_mode_cases, + &hfn_sn_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 11/22] doc: add PDCP library guide 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (9 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 12/22] pdcp: add control PDU handling for status report Anoob Joseph ` (11 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add guide for PDCP library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 254 ++++++++++++++++++ 3 files changed, 256 insertions(+) create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst diff --git a/doc/guides/prog_guide/img/pdcp_functional_overview.svg b/doc/guides/prog_guide/img/pdcp_functional_overview.svg new file mode 100644 index 0000000000..287daafc21 --- /dev/null +++ b/doc/guides/prog_guide/img/pdcp_functional_overview.svg @@ -0,0 +1 @@ +<svg width="1280" height="720" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><clipPath id="clip0"><rect x="0" y="0" width="1280" height="720"/></clipPath></defs><g clip-path="url(#clip0)"><rect x="0" y="0" width="1280" height="720" fill="#FFFFFF"/><rect x="202" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><rect x="640" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><path d="M605.5 73.5001 605.5 590.633" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M380.5 634 803.25 634 803.25 585.907 790.5 585.907 816 557.5 841.5 585.907 828.75 585.907 828.75 659.5 380.5 659.5Z" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5" fill-rule="evenodd"/><rect x="362.5" y="557.5" width="28" height="102" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5"/><rect x="412.5" y="671.5" width="370" height="32" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(492.364 694)">Radio Interface (<tspan font-size="19" x="137.5" y="0">Uu</tspan><tspan font-size="19" x="161.333" y="0">/PC5)</tspan><tspan font-size="19" x="-282.121" y="-653">UE/NG</tspan><tspan font-size="19" x="-222.955" y="-653">-</tspan><tspan font-size="19" x="-216.788" y="-653">RAN/UE A</tspan><tspan font-size="19" x="375.54" y="-653">NG</tspan>-<tspan font-size="19" x="409.706" y="-653">RAN/UE/UE B</tspan><tspan font-size="14" x="0.896851" y="-647">Transmitting </tspan><tspan font-size="14" x="1.31018" y="-631">PDCP entity</tspan><tspan font-size="14" x="167.401" y="-647">Receiving </tspan><tspan font-size="14" x="160.148" y="-631">PDCP entity</tspan></text><path d="M314.5 71.5001 431.364 71.6549" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M331.5 71.5001C331.5 65.9772 349.633 61.5001 372 61.5001 394.368 61.5001 412.5 65.9772 412.5 71.5001 412.5 77.0229 394.368 81.5001 372 81.5001 349.633 81.5001 331.5 77.0229 331.5 71.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M353.5 90.5001 363.5 90.5001 363.5 48.5001 383.5 48.5001 383.5 90.5001 393.5 90.5001 373.5 110.5Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M877.364 82.6549 760.5 82.5001" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M860.5 83.5001C860.5 89.0229 842.368 93.5001 820 93.5001 797.633 93.5001 779.5 89.0229 779.5 83.5001 779.5 77.9772 797.633 73.5001 820 73.5001 842.368 73.5001 860.5 77.9772 860.5 83.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M838.5 64.5 828.5 64.5 828.5 106.5 808.5 106.5 808.5 64.5 798.5 64.5 818.5 44.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><rect x="244.5" y="128.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(302.494 150)">Transmission buffer:<tspan font-size="19" x="-4.67999" y="23">Sequence</tspan><tspan font-size="19" x="84.32" y="23">numberin</tspan>g</text><rect x="244.5" y="199.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(297.314 222)">Header or uplink data<tspan font-size="19" x="34.08" y="23">Compressio</tspan>n</text><rect x="682.5" y="141.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(735.782 164)">Header or uplink data<tspan font-size="19" x="24.2466" y="23">Decompressio</tspan>n</text><rect x="244.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(300.314 505)">Routing / Duplication</text><rect x="244.5" y="437.5" width="285" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(309.734 460)">Add PDCP header</text><rect x="244.5" y="383.5" width="189" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(298.62 406)">Ciphering</text><rect x="244.5" y="333.5" width="189" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(261.953 356)">Integrity protection</text><path d="M472.167 267.5 472.167 409.802 470.833 409.802 470.833 267.5ZM475.5 408.468 471.5 416.468 467.5 408.468Z"/><path d="M332.167 267.5 332.167 319.552 330.833 319.552 330.833 267.5ZM335.5 318.218 331.5 326.218 327.5 318.219Z"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(253.644 291)">Packets associated <tspan font-size="19" x="14.0067" y="23">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 499.312 299)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="-2.75546e-15" y="45">PDCP SDU</tspan></text><rect x="682.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(728.535 505)">Remove PDCP Header</text><rect x="682.5" y="437.5" width="203" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(733.729 460)">Deciphering</text><rect x="682.5" y="389.5" width="203" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(702.159 412)">Integrity Verification</text><rect x="682.5" y="303.5" width="203" height="77" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(712.729 325)">Reception buffer:<tspan font-size="19" x="24.6667" y="23">Reordering</tspan><tspan font-size="19" x="-13.1667" y="45">Duplicate discardin</tspan><tspan font-size="19" x="144.167" y="45">g</tspan><tspan font-size="19" x="-10.0989" y="-84">Packets associated </tspan><tspan font-size="19" x="3.90784" y="-61">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 960.34 294)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="1" y="45">PDCP SDU</tspan></text><path d="M0.666667-8.2074e-07 0.666763 78.6116-0.66657 78.6116-0.666667 8.2074e-07ZM4.00009 77.2782 0.000104987 85.2782-3.9999 77.2782Z" transform="matrix(1 0 0 -1 779.5 296.778)"/><path d="M0.666667-2.88742e-07 0.666769 235.734-0.666565 235.734-0.666667 2.88742e-07ZM4.0001 234.401 0.000104987 242.401-3.9999 234.401Z" transform="matrix(1 0 0 -1 931.5 453.901)"/></g></svg> \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 87333ee84a..6099ff63cd 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -77,4 +77,5 @@ Programmer's Guide lto profile_app asan + pdcp_lib glossary diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst new file mode 100644 index 0000000000..8369c71600 --- /dev/null +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -0,0 +1,254 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2023 Marvell. + +PDCP Protocol Processing Library +================================ + +DPDK provides a library for PDCP protocol processing. The library utilizes +other DPDK libraries such as cryptodev, reorder, etc., to provide the +application with a transparent and high performant PDCP protocol processing +library. + +The library abstracts complete PDCP protocol processing conforming to +``ETSI TS 138 323 V17.1.0 (2022-08)``. +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + +PDCP would involve the following operations, + +1. Transfer of user plane data +2. Transfer of control plane data +3. Header compression +4. Uplink data compression +5. Ciphering and integrity protection + +.. _figure_pdcp_functional_overview: + +.. figure:: img/pdcp_functional_overview.* + + PDCP functional overview new + +PDCP library would abstract the protocol offload features of the cryptodev and +would provide a uniform interface and consistent API usage to work with +cryptodev irrespective of the protocol offload features supported. + +PDCP entity API +--------------- + +PDCP library provides following control path APIs that is used to +configure various PDCP entities, + +1. ``rte_pdcp_entity_establish()`` +2. ``rte_pdcp_entity_suspend()`` +3. ``rte_pdcp_entity_release()`` + +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or +``rte_security_session`` based on the config. The sessions would be created/ +destroyed while corresponding PDCP entity operations are performed. + +When upper layers request a PDCP entity suspend (``rte_pdcp_entity_suspend()``), +it would result in flushing out of all cached packets. Internal state variables +are updated as described in 5.1.4. + +When upper layers request a PDCP entity release (``rte_pdcp_entity_release()``), +it would result in flushing out of all cached packets and releasing of all +memory associated with the entity. It would internally free any crypto/security +sessions created. All procedures mentioned in 5.1.3 would be performed. + +PDCP PDU (Protocol Data Unit) API +--------------------------------- + +PDCP PDUs can be categorized as, + +1. Control PDU +2. Data PDU + +Control PDUs are used for signalling between entities on either end and can be +one of the following, + +1. PDCP status report +2. ROHC feedback +3. EHC feedback + +Control PDUs are not ciphered or authenticated, and so such packets are not +submitted to cryptodev for processing. + +Data PDUs are regular packets submitted by upper layers for transmission to +other end. Such packets would need to be ciphered and authenticated based on +the entity configuration. + +PDCP packet processing API for data PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PDCP processing is split into 2 parts. One before cryptodev processing +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return crypto +operations belonging to multiple entities, ``rte_pdcp_pkt_crypto_group()`` +is added to help grouping crypto operations belonging to same PDCP entity. + +Lib PDCP would allow application to use same API sequence while leveraging +protocol offload features enabled by ``rte_security`` library. Lib PDCP would +internally change the handles registered for ``pre_process`` and +``post_process`` based on features enabled in the entity. + +Lib PDCP would create the required sessions on the device provided in entity to +minimize the application requirements. Also, the crypto_op allocation and free +would also be done internally by lib PDCP to allow the library to create +crypto ops as required for the input packets. For example, when control PDUs are +received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP +is expected to handle it differently. + +Supported features +------------------ + +- 12 bit & 18 bit sequence numbers +- Uplink & downlink traffic +- HFN increment +- IV generation as required per algorithm + +Supported ciphering algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CTR +- SNOW3G-CIPHER +- ZUC-CIPHER + +Supported integrity protection algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CMAC +- SNOW3G-AUTH +- ZUC-AUTH + +Sample API usage +---------------- + +The ``rte_pdcp_entity_conf`` structure is used to pass the configuration +parameters for entity creation. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_entity_conf 8< + :end-before: >8 End of structure rte_pdcp_entity_conf. + +.. code-block:: c + + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; + struct rte_crypto_op *cop[MAX_BURST_SIZE]; + struct rte_pdcp_group grp[MAX_BURST_SIZE]; + struct rte_pdcp_entity *pdcp_entity; + int nb_max_out_mb, ret, nb_grp; + uint16_t nb_ops; + + /* Create PDCP entity */ + pdcp_entity = rte_pdcp_entity_establish(&conf); + + /** + * Allocate buffer for holding mbufs returned during PDCP suspend, + * release & post-process APIs. + */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + MAX_BURST_SIZE; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + /* Handle error */ + } + + while (1) { + /* Receive packet and form mbuf */ + + /** + * Prepare packets for crypto operation. Following operations + * would be done, + * + * Transmitting entity/UL (only data PDUs): + * - Perform compression + * - Assign sequence number + * - Add PDCP header + * - Create & prepare crypto_op + * - Prepare IV for crypto operation (auth_gen, encrypt) + * - Save original PDCP SDU (during PDCP re-establishment, + * unconfirmed PDCP SDUs need to crypto processed again and + * transmitted/re-transmitted) + * + * Receiving entity/DL: + * - Any control PDUs received would be processed and + * appropriate actions taken. If data PDU, continue. + * - Determine sequence number (based on HFN & per packet SN) + * - Prepare crypto_op + * - Prepare IV for crypto operation (decrypt, auth_verify) + */ + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, + nb_rx, &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) + != nb_success) { + /* Retry for enqueue failure packets */ + } + + ... + + nb_ops = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, + MAX_BURST_SIZE); + if (nb_ops == 0) + continue; + + /** + * Received a burst of completed crypto ops from cryptodev. It + * may belong to various entities. Group similar ones together + * for entity specific post-processing. + */ + + /** + * Groups similar entities together. Frees crypto op and based + * on crypto_op status, set mbuf->ol_flags which would be + * checked in rte_pdcp_pkt_post_process(). + */ + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); + + for (i = 0; i != nb_grp; i++) { + + /** + * Post process packets after crypto completion. + * Following operations would be done, + * + * Transmitting entity/UL: + * - Check crypto result + * + * Receiving entity/DL: + * - Check crypto operation status + * - Check for duplication (if yes, drop duplicate) + * - Perform decompression + * - Trim PDCP header + * - Hold packet (SDU) for in-order delivery (return + * completed packets as and when sequence is + * completed) + * - If not in sequence, cache the packet and start + * t-Reordering timer. When timer expires, the + * packets need to delivered to upper layers (not + * treated as error packets). + */ + nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, + grp[i].m, out_mb, + grp[i].cnt, + &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + /* Perform additional operations */ + + /** + * Transmitting entity/UL + * - If duplication is enabled, duplicate PDCP PDUs + * - When lower layers confirm reception of a PDCP PDU, + * it should be communicated to PDCP layer so that + * PDCP can drop the corresponding SDU + */ + } + } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 12/22] pdcp: add control PDU handling for status report 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (10 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 11/22] doc: add PDCP library guide Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph ` (10 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz Add control PDU handling and implement status report generation. Status report generation works only when RX_DELIV = RX_NEXT. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 9 ++++++ lib/pdcp/meson.build | 2 ++ lib/pdcp/pdcp_cnt.c | 29 ++++++++++++++++++ lib/pdcp/pdcp_cnt.h | 14 +++++++++ lib/pdcp/pdcp_ctrl_pdu.c | 46 +++++++++++++++++++++++++++++ lib/pdcp/pdcp_ctrl_pdu.h | 15 ++++++++++ lib/pdcp/pdcp_entity.h | 15 ++++++++-- lib/pdcp/pdcp_process.c | 13 +++++++++ lib/pdcp/rte_pdcp.c | 47 +++++++++++++++++++++++++++++- lib/pdcp/rte_pdcp.h | 33 +++++++++++++++++++++ lib/pdcp/version.map | 2 ++ 11 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index 8369c71600..dcb424bb1d 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -76,6 +76,15 @@ Data PDUs are regular packets submitted by upper layers for transmission to other end. Such packets would need to be ciphered and authenticated based on the entity configuration. +PDCP packet processing API for control PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Control PDUs are used in PDCP as a communication channel between transmitting +and receiving entities. When upper layer request for operations such +re-establishment, receiving PDCP entity need to prepare a status report and +send it to the other end. The API ``rte_pdcp_control_pdu_create`` allows +application to request the same. + PDCP packet processing API for data PDU ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 08679b743a..75d476bf6d 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -8,7 +8,9 @@ if is_windows endif sources = files( + 'pdcp_cnt.c', 'pdcp_crypto.c', + 'pdcp_ctrl_pdu.c', 'pdcp_process.c', 'rte_pdcp.c', ) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c new file mode 100644 index 0000000000..c9b952184b --- /dev/null +++ b/lib/pdcp/pdcp_cnt.c @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_cnt.h" +#include "pdcp_entity.h" + +int +pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv_dl_part *en_priv_dl; + uint32_t window_sz; + + if (en == NULL || conf == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + return 0; + + en_priv_dl = entity_dl_part_get(en); + window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + RTE_SET_USED(window_sz); + RTE_SET_USED(en_priv_dl); + + return 0; +} diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h new file mode 100644 index 0000000000..bbda478b55 --- /dev/null +++ b/lib/pdcp/pdcp_cnt.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CNT_H +#define PDCP_CNT_H + +#include <rte_common.h> + +#include "pdcp_entity.h" + +int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c new file mode 100644 index 0000000000..feb05fd863 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_byteorder.h> +#include <rte_mbuf.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_ctrl_pdu.h" +#include "pdcp_entity.h" + +static __rte_always_inline void +pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) +{ + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + pdu_hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + pdu_hdr->r = 0; + pdu_hdr->fmc = rte_cpu_to_be_32(rx_deliv); +} + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; + uint32_t rx_deliv; + int pdu_sz; + + if (!en_priv->flags.is_status_report_required) + return -EINVAL; + + pdu_sz = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + + rx_deliv = en_priv->state.rx_deliv; + + /* Zero missing PDUs - status report contains only FMC */ + if (rx_deliv >= en_priv->state.rx_next) { + pdu_hdr = (struct rte_pdcp_up_ctrl_pdu_hdr *)rte_pktmbuf_append(m, pdu_sz); + if (pdu_hdr == NULL) + return -ENOMEM; + pdcp_hdr_fill(pdu_hdr, rx_deliv); + + return 0; + } + + return -ENOTSUP; +} diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h new file mode 100644 index 0000000000..a2424fbd10 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CTRL_PDU_H +#define PDCP_CTRL_PDU_H + +#include <rte_mbuf.h> + +#include "pdcp_entity.h" + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); + +#endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 10a72faae1..28691a504b 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -109,6 +109,13 @@ union cipher_iv_partial { uint64_t u64[2]; }; +struct pdcp_cnt_bitmap { + /** Number of entries that can be stored. */ + uint32_t size; + /** Bitmap of the count values already received.*/ + struct rte_bitmap *bmp; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -136,9 +143,13 @@ struct entity_priv { uint64_t is_ul_entity : 1; /** Is NULL auth. */ uint64_t is_null_auth : 1; + /** Is status report required.*/ + uint64_t is_status_report_required : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; + /** Control PDU pool. */ + struct rte_mempool *ctrl_pdu_pool; /** PDCP header size. */ uint8_t hdr_sz; /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ @@ -148,8 +159,8 @@ struct entity_priv { }; struct entity_priv_dl_part { - /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ - uint8_t dummy; + /** PDCP would need to track the count values that are already received.*/ + struct pdcp_cnt_bitmap bitmap; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index e8112adfb8..2e4fc7c664 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -1158,6 +1158,19 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) en_priv->flags.is_null_auth = 1; + /** + * flags.is_status_report_required + * + * Indicate whether status report is required. + */ + if (conf->status_report_required) { + /** Status report is required only for DL entities. */ + if (conf->pdcp_xfrm.pkt_dir != RTE_SECURITY_PDCP_DOWNLINK) + return -EINVAL; + + en_priv->flags.is_status_report_required = 1; + } + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index adcad5dd25..cf1a5f8eeb 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -6,7 +6,9 @@ #include <rte_pdcp.h> #include <rte_malloc.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -35,7 +37,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) int ret, entity_size; uint32_t count; - if (conf == NULL || conf->cop_pool == NULL) { + if (conf == NULL || conf->cop_pool == NULL || conf->ctrl_pdu_pool == NULL) { rte_errno = EINVAL; return NULL; } @@ -82,6 +84,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) en_priv->state.rx_deliv = count; en_priv->state.tx_next = count; en_priv->cop_pool = conf->cop_pool; + en_priv->ctrl_pdu_pool = conf->ctrl_pdu_pool; /* Setup crypto session */ ret = pdcp_crypto_sess_create(entity, conf); @@ -92,6 +95,10 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + ret = pdcp_cnt_ring_create(entity, conf); + if (ret) + goto crypto_sess_destroy; + return entity; crypto_sess_destroy: @@ -139,3 +146,41 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, return 0; } + +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type) +{ + struct entity_priv *en_priv; + struct rte_mbuf *m; + int ret; + + if (pdcp_entity == NULL) { + rte_errno = EINVAL; + return NULL; + } + + en_priv = entity_priv_get(pdcp_entity); + + m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); + if (m == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + switch (type) { + case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: + ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + break; + default: + ret = -ENOTSUP; + } + + if (ret) { + rte_pktmbuf_free(m); + rte_errno = -ret; + return NULL; + } + + return m; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 4143fae0b2..553518328e 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -16,6 +16,7 @@ #include <rte_compat.h> #include <rte_common.h> #include <rte_mempool.h> +#include <rte_pdcp_hdr.h> #include <rte_security.h> #ifdef __cplusplus @@ -79,6 +80,8 @@ struct rte_pdcp_entity_conf { struct rte_mempool *sess_mpool; /** Crypto op pool.*/ struct rte_mempool *cop_pool; + /** Mbuf pool to be used for allocating control PDUs.*/ + struct rte_mempool *ctrl_pdu_pool; /** * SN value to be used. 32 bit count value to be used for the first * packet would be derived based on HFN (`rte_security_pdcp_xform.hfn`) @@ -98,6 +101,16 @@ struct rte_pdcp_entity_conf { * crypto processing. */ bool reverse_iv_direction; + /** + * Status report required (specified in TS 38.331). + * + * If PDCP entity is configured to send a PDCP status report, the upper + * layer application may request a receiving PDCP entity to generate a + * PDCP status report using ``rte_pdcp_control_pdu_create``. In + * addition, PDCP status reports may be generated during operations such + * as entity re-establishment. + */ + bool status_report_required; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -179,6 +192,26 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Create control PDU packet of the `type` specified. The control PDU packet + * would be allocated from *rte_pdcp_entity_conf.ctrl_pdu_pool* by lib PDCP. + * + * @param pdcp_entity + * Pointer to the PDCP entity for which the control PDU need to be generated. + * @param type + * Type of control PDU to be generated. + * @return + * - Control PDU generated, in case of success. + * - NULL in case of failure. rte_errno will be set to error code. + */ +__rte_experimental +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index d564f155e0..97171f902e 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,6 +2,8 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_control_pdu_create; + rte_pdcp_en_from_cop; rte_pdcp_entity_establish; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 13/22] pdcp: implement t-Reordering and packet buffering 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (11 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 12/22] pdcp: add control PDU handling for status report Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 14/22] test/pdcp: add in-order delivery cases Anoob Joseph ` (9 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add in-order delivery of packets in PDCP. Delivery of packets in-order relies on t-Reordering timer. When 'out-of-order delivery' is disabled, PDCP will buffer all received packets that are out of order. The t-Reordering timer determines the time period these packets would be held in the buffer, waiting for any missing packets to arrive. Introduce packet buffering and state variables which indicate status of the timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 3 +- lib/pdcp/pdcp_entity.h | 19 +++++++ lib/pdcp/pdcp_process.c | 123 +++++++++++++++++++++++++++++++--------- lib/pdcp/pdcp_reorder.c | 27 +++++++++ lib/pdcp/pdcp_reorder.h | 62 ++++++++++++++++++++ lib/pdcp/rte_pdcp.c | 53 +++++++++++++++-- lib/pdcp/rte_pdcp.h | 6 +- 7 files changed, 257 insertions(+), 36 deletions(-) create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 75d476bf6d..f4f9246bcb 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -12,9 +12,10 @@ sources = files( 'pdcp_crypto.c', 'pdcp_ctrl_pdu.c', 'pdcp_process.c', + 'pdcp_reorder.c', 'rte_pdcp.c', ) headers = files('rte_pdcp.h') indirect_headers += files('rte_pdcp_group.h') -deps += ['mbuf', 'net', 'cryptodev', 'security'] +deps += ['mbuf', 'net', 'cryptodev', 'security', 'reorder'] diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 28691a504b..34341cdc11 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -11,6 +11,8 @@ #include <rte_pdcp.h> #include <rte_security.h> +#include "pdcp_reorder.h" + struct entity_priv; #define PDCP_HFN_MIN 0 @@ -109,6 +111,17 @@ union cipher_iv_partial { uint64_t u64[2]; }; +enum timer_state { + TIMER_STOP, + TIMER_RUNNING, + TIMER_EXPIRED, +}; + +struct pdcp_t_reordering { + /** Represent timer state */ + enum timer_state state; +}; + struct pdcp_cnt_bitmap { /** Number of entries that can be stored. */ uint32_t size; @@ -145,6 +158,8 @@ struct entity_priv { uint64_t is_null_auth : 1; /** Is status report required.*/ uint64_t is_status_report_required : 1; + /** Is out-of-order delivery enabled */ + uint64_t is_out_of_order_delivery : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -161,6 +176,10 @@ struct entity_priv { struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; + /** t-Reordering handles */ + struct pdcp_t_reordering t_reorder; + /** Reorder packet buffer */ + struct pdcp_reorder reorder; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 2e4fc7c664..d5f2a68b02 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -817,25 +817,88 @@ pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool tr } } -static inline bool +static inline int pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, - const uint32_t count) + const uint32_t count, struct rte_mbuf *mb, + struct rte_mbuf *out_mb[], + const bool trim_mac) { struct entity_priv *en_priv = entity_priv_get(entity); + struct pdcp_t_reordering *t_reorder; + struct pdcp_reorder *reorder; + uint16_t processed = 0; - if (count < en_priv->state.rx_deliv) - return false; + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + const uint32_t hdr_trim_sz = en_priv->hdr_sz + en_priv->aad_sz; - /* t-Reordering timer is not supported - SDU will be delivered immediately. - * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not - * been delivered to upper layers - */ - en_priv->state.rx_next = count + 1; + if (count < en_priv->state.rx_deliv) + return -EINVAL; if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; - return true; + pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); + + if (en_priv->flags.is_out_of_order_delivery) { + out_mb[0] = mb; + en_priv->state.rx_deliv = count + 1; + + return 1; + } + + reorder = &dl->reorder; + t_reorder = &dl->t_reorder; + + if (count == en_priv->state.rx_deliv) { + if (reorder->is_active) { + /* + * This insert used only to increment reorder->min_seqn + * To remove it - min_seqn_set() has to work with non-empty buffer + */ + pdcp_reorder_insert(reorder, mb, count); + + /* Get buffered packets */ + struct rte_mbuf **cached_mbufs = &out_mb[processed]; + uint32_t nb_cached = pdcp_reorder_get_sequential(reorder, + cached_mbufs, entity->max_pkt_cache - processed); + + processed += nb_cached; + } else { + out_mb[processed++] = mb; + } + + /* Processed should never exceed the window size */ + en_priv->state.rx_deliv = count + processed; + + } else { + if (!reorder->is_active) + /* Initialize reordering buffer with RX_DELIV */ + pdcp_reorder_start(reorder, en_priv->state.rx_deliv); + /* Buffer the packet */ + pdcp_reorder_insert(reorder, mb, count); + } + + /* Stop & reset current timer if rx_reord is received */ + if (t_reorder->state == TIMER_RUNNING && + en_priv->state.rx_deliv >= en_priv->state.rx_reord) { + t_reorder->state = TIMER_STOP; + /* Stop reorder buffer, only if it's empty */ + if (en_priv->state.rx_deliv == en_priv->state.rx_next) + pdcp_reorder_stop(reorder); + } + + /* + * If t-Reordering is not running (includes the case when t-Reordering is stopped due to + * actions above). + */ + if (t_reorder->state == TIMER_STOP && en_priv->state.rx_deliv < en_priv->state.rx_next) { + /* Update RX_REORD to RX_NEXT */ + en_priv->state.rx_reord = en_priv->state.rx_next; + /* Start t-Reordering */ + t_reorder->state = TIMER_RUNNING; + } + + return processed; } static inline uint16_t @@ -845,16 +908,14 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, uint16_t num, uint16_t *nb_err_ret, const bool is_integ_protected) { + int i, nb_processed, nb_success = 0, nb_err = 0, rsn = 0; struct entity_priv *en_priv = entity_priv_get(entity); struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; - int i, nb_success = 0, nb_err = 0, rsn = 0; const uint32_t aad_sz = en_priv->aad_sz; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -869,11 +930,12 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, RTE_SECURITY_PDCP_SN_SIZE_12))) goto error; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], is_integ_protected); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -913,14 +975,13 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, const bool is_integ_protected) { struct entity_priv *en_priv = entity_priv_get(entity); + int i, nb_processed, nb_success = 0, nb_err = 0; struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; struct rte_mbuf *mb, *err_mb[num]; int32_t rsn = 0; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; for (i = 0; i < num; i++) { mb = in_mb[i]; @@ -937,11 +998,12 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, RTE_SECURITY_PDCP_SN_SIZE_18))) goto error; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], is_integ_protected); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -980,16 +1042,14 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, uint16_t num, uint16_t *nb_err_ret) { struct entity_priv *en_priv = entity_priv_get(entity); + int i, nb_processed, nb_success = 0, nb_err = 0; struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; int32_t rsn; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -1003,12 +1063,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, RTE_SECURITY_PDCP_SN_SIZE_12))) goto error; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], true); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, true); - - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -1171,6 +1231,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent en_priv->flags.is_status_report_required = 1; } + /** + * flags.is_out_of_order_delivery + * + * Indicate whether the outoforder delivery is enabled for PDCP entity. + */ + en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** * hdr_sz * diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c new file mode 100644 index 0000000000..5399f0dc28 --- /dev/null +++ b/lib/pdcp/pdcp_reorder.c @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_reorder.h> + +#include "pdcp_reorder.h" + +int +pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +{ + reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + if (reorder->buf == NULL) + return -rte_errno; + + reorder->window_size = window_size; + reorder->is_active = false; + + return 0; +} + +void +pdcp_reorder_destroy(const struct pdcp_reorder *reorder) +{ + rte_reorder_free(reorder->buf); +} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h new file mode 100644 index 0000000000..6a2f61d6ae --- /dev/null +++ b/lib/pdcp/pdcp_reorder.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_REORDER_H +#define PDCP_REORDER_H + +#include <rte_reorder.h> + +struct pdcp_reorder { + struct rte_reorder_buffer *buf; + uint32_t window_size; + bool is_active; +}; + +int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); +void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); + +static inline uint32_t +pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs) +{ + return rte_reorder_drain(reorder->buf, mbufs, max_mbufs); +} + +static inline uint32_t +pdcp_reorder_up_to_get(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs, uint32_t seqn) +{ + return rte_reorder_drain_up_to_seqn(reorder->buf, mbufs, max_mbufs, seqn); +} + +static inline void +pdcp_reorder_start(struct pdcp_reorder *reorder, uint32_t min_seqn) +{ + int ret; + + reorder->is_active = true; + + ret = rte_reorder_min_seqn_set(reorder->buf, min_seqn); + RTE_VERIFY(ret == 0); +} + +static inline void +pdcp_reorder_stop(struct pdcp_reorder *reorder) +{ + reorder->is_active = false; +} + +static inline void +pdcp_reorder_insert(struct pdcp_reorder *reorder, struct rte_mbuf *mbuf, + rte_reorder_seqn_t pkt_count) +{ + int ret; + + *rte_reorder_seqn(mbuf) = pkt_count; + + ret = rte_reorder_insert(reorder->buf, mbuf); + RTE_VERIFY(ret == 0); +} + +#endif /* PDCP_REORDER_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index cf1a5f8eeb..8237baa53c 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -29,6 +29,17 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } +static int +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + + entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + + return pdcp_reorder_create(&dl->reorder, window_size); +} + struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { @@ -95,6 +106,12 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + ret = pdcp_dl_establish(entity, conf); + if (ret) + goto crypto_sess_destroy; + } + ret = pdcp_cnt_ring_create(entity, conf); if (ret) goto crypto_sess_destroy; @@ -109,26 +126,50 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } +static int +pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + int nb_out; + + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, + en_priv->state.rx_next); + + pdcp_reorder_destroy(&dl->reorder); + + return nb_out; +} + int rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv *en_priv; + int nb_out = 0; + if (pdcp_entity == NULL) return -EINVAL; + en_priv = entity_priv_get(pdcp_entity); + + if (!en_priv->flags.is_ul_entity) + nb_out = pdcp_dl_release(pdcp_entity, out_mb); + /* Teardown crypto sessions */ pdcp_crypto_sess_destroy(pdcp_entity); rte_free(pdcp_entity); - RTE_SET_USED(out_mb); - return 0; + return nb_out; } int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; + int nb_out = 0; if (pdcp_entity == NULL) return -EINVAL; @@ -138,13 +179,15 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, if (en_priv->flags.is_ul_entity) { en_priv->state.tx_next = 0; } else { + dl = entity_dl_part_get(pdcp_entity); + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, pdcp_entity->max_pkt_cache, + en_priv->state.rx_next); + pdcp_reorder_stop(&dl->reorder); en_priv->state.rx_next = 0; en_priv->state.rx_deliv = 0; } - RTE_SET_USED(out_mb); - - return 0; + return nb_out; } struct rte_mbuf * diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 553518328e..8c55968be8 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -111,6 +111,8 @@ struct rte_pdcp_entity_conf { * as entity re-establishment. */ bool status_report_required; + /** Enable out of order delivery. */ + bool out_of_order_delivery; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -271,8 +273,8 @@ rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, * @param in_mb * The address of an array of *num* pointers to *rte_mbuf* structures. * @param[out] out_mb - * The address of an array of *num* pointers to *rte_mbuf* structures - * to output packets after PDCP post-processing. + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures to output packets after PDCP post-processing. * @param num * The maximum number of packets to process. * @param[out] nb_err -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 14/22] test/pdcp: add in-order delivery cases 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (12 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 15/22] pdcp: add timer callback handlers Anoob Joseph ` (8 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add test cases to verify behaviour when in-order delivery is enabled and packets arrive in out-of-order. PDCP library is expected to buffer the packets and return packets in-order when the missing packet arrives. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 223 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index ed2dfe921c..b2563f0cc0 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,15 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* Assert that condition is true, or goto the mark */ +#define ASSERT_TRUE_OR_GOTO(cond, mark, ...) do {\ + if (!(cond)) { \ + RTE_LOG(ERR, USER1, "Error at: %s:%d\n", __func__, __LINE__); \ + RTE_LOG(ERR, USER1, __VA_ARGS__); \ + goto mark; \ + } \ +} while (0) + /* According to formula(7.2.a Window_Size) */ #define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) @@ -83,6 +92,38 @@ run_test_with_all_known_vec(const void *args) return run_test_foreach_known_vec(test, false); } +static int +run_test_with_all_known_vec_until_first_pass(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, true); +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -416,6 +457,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.sess_mpool = ts_params->sess_pool; conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.ctrl_pdu_pool = ts_params->mbuf_pool; conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; @@ -866,6 +908,7 @@ test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) /* Configure Uplink to generate expected, encrypted packet */ pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.out_of_order_delivery = true; conf->entity.reverse_iv_direction = true; conf->entity.pdcp_xfrm.hfn = new_hfn; conf->entity.sn = new_sn; @@ -913,6 +956,171 @@ test_sn_minus_outside(struct pdcp_test_conf *t_conf) return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); } +static struct rte_mbuf * +generate_packet_for_dl_with_sn(struct pdcp_test_conf ul_conf, uint32_t count) +{ + enum rte_security_pdcp_sn_size sn_size = ul_conf.entity.pdcp_xfrm.sn_size; + int ret; + + ul_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(count, sn_size); + ul_conf.entity.sn = pdcp_sn_from_count_get(count, sn_size); + ul_conf.entity.out_of_order_delivery = true; + ul_conf.entity.reverse_iv_direction = true; + ul_conf.output_len = 0; + + ret = test_attempt_single(&ul_conf); + if (ret != TEST_SUCCESS) + return NULL; + + return mbuf_from_data_create(ul_conf.output, ul_conf.output_len); +} + +static bool +array_asc_sorted_check(struct rte_mbuf *m[], uint32_t len, enum rte_security_pdcp_sn_size sn_size) +{ + uint32_t i; + + if (len < 2) + return true; + + for (i = 0; i < (len - 1); i++) { + if (pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i], void *), sn_size) > + pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i + 1], void *), sn_size)) + return false; + } + + return true; +} + +static int +test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m0 = NULL, *m1 = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Generate packet to fill the existing gap */ + m0 = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m0 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1] + * Gap filled, all packets should be returned + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m0, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m0 = NULL; /* Packet was moved to out_mb */ + + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, + "Error occurred during packet drain\n"); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m0); + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + +static int +test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + struct rte_mbuf *m1 = NULL, **out_mb = NULL; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + const int rx_deliv = 0; + int ret = TEST_FAILED; + size_t i, nb_out; + uint8_t cdev_id; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + ASSERT_TRUE_OR_GOTO(pdcp_entity->max_pkt_cache >= window_size, exit, + "PDCP max packet cache is too small"); + cdev_id = dl_conf.entity.dev_id; + out_mb = rte_zmalloc(NULL, pdcp_entity->max_pkt_cache * sizeof(uintptr_t), 0); + ASSERT_TRUE_OR_GOTO(out_mb != NULL, exit, + "Could not allocate buffer for holding out_mb buffers\n"); + + /* Send packets with SN > RX_DELIV to create a gap */ + for (i = rx_deliv + 1; i < window_size; i++) { + m1 = generate_packet_for_dl_with_sn(*ul_conf, i); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + } + + m1 = generate_packet_for_dl_with_sn(*ul_conf, rx_deliv); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Insert missing packet */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == window_size, exit, + "Packet count mismatch (received: %i, expected: %i)\n", + nb_success, window_size); + m1 = NULL; + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + rte_free(out_mb); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -969,10 +1177,25 @@ static struct unit_test_suite hfn_sn_test_cases = { } }; +static struct unit_test_suite reorder_test_cases = { + .suite_name = "PDCP reorder", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_reorder_buffer_full_window_size_sn_12), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, + &reorder_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 15/22] pdcp: add timer callback handlers 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (13 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 14/22] test/pdcp: add in-order delivery cases Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 16/22] pdcp: add timer expiry handle Anoob Joseph ` (7 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> PDCP has a windowing mechanism which allows only packets that fall in a reception window. The pivot point for this window is RX_REORD which happens to be the first missing or next expected packet. If the missing packet is not received after a specified time, then the RX_REORD state variable needs to be moved up to slide the reception window. PDCP relies on timers for such operations. The timer needs to be armed when PDCP library doesn't receive all packets in-order and starts buffering packets that arrived after a missing packet. The timer needs to be cancelled when a missing packet is received. To avoid dependency on particular timer implementation, PDCP library allows application to register two callbacks, timer_start() and timer_stop() that will be called later by library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 ++ lib/pdcp/pdcp_process.c | 2 ++ lib/pdcp/rte_pdcp.c | 1 + lib/pdcp/rte_pdcp.h | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 52 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 34341cdc11..efc74ba9b9 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -120,6 +120,8 @@ enum timer_state { struct pdcp_t_reordering { /** Represent timer state */ enum timer_state state; + /** User defined callback handles */ + struct rte_pdcp_t_reordering handle; }; struct pdcp_cnt_bitmap { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index d5f2a68b02..e273dd4658 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -882,6 +882,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (t_reorder->state == TIMER_RUNNING && en_priv->state.rx_deliv >= en_priv->state.rx_reord) { t_reorder->state = TIMER_STOP; + t_reorder->handle.stop(t_reorder->handle.timer, t_reorder->handle.args); /* Stop reorder buffer, only if it's empty */ if (en_priv->state.rx_deliv == en_priv->state.rx_next) pdcp_reorder_stop(reorder); @@ -896,6 +897,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, en_priv->state.rx_reord = en_priv->state.rx_next; /* Start t-Reordering */ t_reorder->state = TIMER_RUNNING; + t_reorder->handle.start(t_reorder->handle.timer, t_reorder->handle.args); } return processed; diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 8237baa53c..1ee9511cd4 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -36,6 +36,7 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct entity_priv_dl_part *dl = entity_dl_part_get(entity); entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + dl->t_reorder.handle = conf->t_reordering; return pdcp_reorder_create(&dl->reorder, window_size); } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 8c55968be8..5d3e38a668 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -67,6 +67,51 @@ struct rte_pdcp_entity { uint32_t max_pkt_cache; } __rte_cache_aligned; +/** + * Callback function type for t-Reordering timer start, set during PDCP entity establish. + * This callback is invoked by PDCP library, during t-Reordering timer start event. + * Only one t-Reordering per receiving PDCP entity would be running at a given time. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_start_cb_t)(void *timer, void *args); + +/** + * Callback function type for t-Reordering timer stop, set during PDCP entity establish. + * This callback will be invoked by PDCP library, during t-Reordering timer stop event. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); + +/** + * PDCP t-Reordering timer interface + * + * Configuration provided by user, that PDCP library will invoke according to timer behaviour. + */ +struct rte_pdcp_t_reordering { + /** Timer pointer, stored for later use in callback functions */ + void *timer; + /** Timer arguments, stored for later use in callback functions */ + void *args; + /** Timer start callback handle */ + rte_pdcp_t_reordering_start_cb_t start; + /** Timer stop callback handle */ + rte_pdcp_t_reordering_stop_cb_t stop; +}; + /** * PDCP entity configuration to be used for establishing an entity. */ @@ -113,6 +158,8 @@ struct rte_pdcp_entity_conf { bool status_report_required; /** Enable out of order delivery. */ bool out_of_order_delivery; + /** t-Reordering timer configuration */ + struct rte_pdcp_t_reordering t_reordering; }; /* >8 End of structure rte_pdcp_entity_conf. */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 16/22] pdcp: add timer expiry handle 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (14 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 15/22] pdcp: add timer callback handlers Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 17/22] test/pdcp: add timer expiry cases Anoob Joseph ` (6 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> The PDCP protocol requires usage of timers to keep track of how long an out-of-order packet should be buffered while waiting for missing packets. Applications can register a desired timer implementation with the PDCP library. Once the timer expires, the application will be notified, and further handling of the event will be performed in the PDCP library. When the timer expires, the PDCP library will return the cached packets, and PDCP internal state variables (like RX_REORD, RX_DELIV etc) will be updated accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 30 ++++++++++++++++++ lib/pdcp/rte_pdcp.c | 49 ++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 31 +++++++++++++++++++ lib/pdcp/version.map | 2 ++ 4 files changed, 112 insertions(+) diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index dcb424bb1d..16deaead15 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -130,6 +130,36 @@ Supported integrity protection algorithms - SNOW3G-AUTH - ZUC-AUTH +Timers +------ + +PDCP utilizes a reception window mechanism to limit the bits of COUNT value +transmitted in the packet. It utilizes state variables such as RX_REORD, +RX_DELIV to define the window and uses RX_DELIV as the lower pivot point of the +window. + +RX_DELIV would be updated only when packets are received in-order. Any missing +packet would mean RX_DELIV won't be updated. A timer, t-Reordering, helps PDCP +to slide the window if the missing packet is not received in a specified time +duration. + +While starting and stopping the timer need to be done by lib PDCP, application +could register its own timer implementation. This is to make sure application +can choose between timers such as rte_timer and rte_event based timers. Starting +and stopping of timer would happen during pre & post process API. + +When the t-Reordering timer expires, application would receive the expiry event. +To perform the PDCP handling of the expiry event, +``rte_pdcp_t_reordering_expiry_handle`` can be used. Expiry handling would +involve sliding the window by updating state variables and passing the expired +packets to the application. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_t_reordering 8< + :end-before: >8 End of structure rte_pdcp_t_reordering. + + Sample API usage ---------------- diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 1ee9511cd4..32ec8bc8f1 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -228,3 +228,52 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, return m; } + +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t capacity = entity->max_pkt_cache; + uint16_t nb_out, nb_seq; + + /* 5.2.2.2 Actions when a t-Reordering expires */ + + /* + * - deliver to upper layers in ascending order of the associated COUNT value after + * performing header decompression, if not decompressed before: + */ + + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); + capacity -= nb_out; + out_mb = &out_mb[nb_out]; + + /* + * - all stored PDCP SDU(s) with consecutively associated COUNT value(s) starting from + * RX_REORD; + */ + nb_seq = pdcp_reorder_get_sequential(&dl->reorder, out_mb, capacity); + nb_out += nb_seq; + + /* + * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered + * to upper layers, with COUNT value >= RX_REORD; + */ + en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + + /* + * - if RX_DELIV < RX_NEXT: + * - update RX_REORD to RX_NEXT; + * - start t-Reordering. + */ + if (en_priv->state.rx_deliv < en_priv->state.rx_next) { + en_priv->state.rx_reord = en_priv->state.rx_next; + dl->t_reorder.state = TIMER_RUNNING; + dl->t_reorder.handle.start(dl->t_reorder.handle.timer, dl->t_reorder.handle.args); + } else { + dl->t_reorder.state = TIMER_EXPIRED; + } + + return nb_out; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 5d3e38a668..76ec569577 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -101,6 +101,7 @@ typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); * * Configuration provided by user, that PDCP library will invoke according to timer behaviour. */ +/* Structure rte_pdcp_t_reordering 8< */ struct rte_pdcp_t_reordering { /** Timer pointer, stored for later use in callback functions */ void *timer; @@ -111,6 +112,7 @@ struct rte_pdcp_t_reordering { /** Timer stop callback handle */ rte_pdcp_t_reordering_stop_cb_t stop; }; +/* >8 End of structure rte_pdcp_t_reordering. */ /** * PDCP entity configuration to be used for establishing an entity. @@ -339,6 +341,35 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.2.2.2 Actions when a t-Reordering expires + * + * When t-Reordering timer expires, PDCP is required to slide the reception + * window by updating state variables such as RX_REORD & RX_DELIV. PDCP would + * need to deliver some of the buffered packets based on the state variables and + * conditions described. + * + * The expiry handle need to be invoked by the application when t-Reordering + * timer expires. In addition to returning buffered packets, it may also restart + * timer based on the state variables. + * + * @param entity + * Pointer to the *rte_pdcp_entity* for which the timer expired. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. Used to return buffered packets that are + * expired. + * @return + * Number of packets returned in *out_mb* buffer. + */ +__rte_experimental +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, + struct rte_mbuf *out_mb[]); + #include <rte_pdcp_group.h> #ifdef __cplusplus diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 97171f902e..4fd912fac0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -14,5 +14,7 @@ EXPERIMENTAL { rte_pdcp_pkt_pre_process; rte_pdcp_pkt_crypto_group; + rte_pdcp_t_reordering_expiry_handle; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 17/22] test/pdcp: add timer expiry cases 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (15 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 16/22] pdcp: add timer expiry handle Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 18/22] test/pdcp: add timer restart case Anoob Joseph ` (5 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add test cases for handling the expiry with rte_timer and rte_event_timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 350 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 350 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index b2563f0cc0..b4677e5b67 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -3,15 +3,24 @@ */ #include <rte_errno.h> +#ifdef RTE_LIB_EVENTDEV +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#endif /* RTE_LIB_EVENTDEV */ #include <rte_malloc.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include <rte_timer.h> #include "test.h" #include "test_cryptodev.h" #include "test_cryptodev_security_pdcp_test_vectors.h" +#define NSECPERSEC 1E9 #define NB_DESC 1024 +#define TIMER_ADAPTER_ID 0 +#define TEST_EV_QUEUE_ID 0 +#define TEST_EV_PORT_ID 0 #define CDEV_INVALID_ID UINT8_MAX #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 @@ -33,10 +42,21 @@ struct pdcp_testsuite_params { struct rte_mempool *cop_pool; struct rte_mempool *sess_pool; bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; + int evdev; +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev; +#endif /* RTE_LIB_EVENTDEV */ + bool timer_is_running; + uint64_t min_resolution_ns; }; static struct pdcp_testsuite_params testsuite_params; +struct test_rte_timer_args { + int status; + struct rte_pdcp_entity *pdcp_entity; +}; + struct pdcp_test_conf { struct rte_pdcp_entity_conf entity; struct rte_crypto_sym_xform c_xfrm; @@ -124,6 +144,30 @@ pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; } +static void +pdcp_timer_start_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = true; +} + +static void +pdcp_timer_stop_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = false; +} + +static struct rte_pdcp_t_reordering t_reorder_timer = { + .timer = &testsuite_params.timer_is_running, + .start = pdcp_timer_start_cb, + .stop = pdcp_timer_stop_cb, +}; + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -462,6 +506,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + conf->entity.t_reordering = t_reorder_timer; if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; @@ -1046,6 +1091,8 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) /* Check that packets in correct order */ ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == false, exit, + "Timer should be stopped after full drain\n"); ret = TEST_SUCCESS; exit: @@ -1121,6 +1168,181 @@ test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static void +event_timer_start_cb(void *timer, void *args) +{ + struct rte_event_timer *evtims = args; + int ret = 0; + + ret = rte_event_timer_arm_burst(timer, &evtims, 1); + assert(ret == 1); +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +test_expiry_with_event_timer(const struct pdcp_test_conf *ul_conf) +{ +#ifdef RTE_LIB_EVENTDEV + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_event event; + + const int start_count = 0; + struct rte_event_timer evtim = { + .ev.op = RTE_EVENT_OP_NEW, + .ev.queue_id = TEST_EV_QUEUE_ID, + .ev.sched_type = RTE_SCHED_TYPE_ATOMIC, + .ev.priority = RTE_EVENT_DEV_PRIORITY_NORMAL, + .ev.event_type = RTE_EVENT_TYPE_TIMER, + .state = RTE_EVENT_TIMER_NOT_ARMED, + .timeout_ticks = 1, + }; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &evtim; + dl_conf.entity.t_reordering.timer = testsuite_params.timdev; + dl_conf.entity.t_reordering.start = event_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + evtim.ev.event_ptr = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + while (n != 1) { + rte_delay_us(testsuite_params.min_resolution_ns / 1000); + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, + "Dequeued unexpected timer expiry event: %i\n", n); + } + + ASSERT_TRUE_OR_GOTO(event.event_type == RTE_EVENT_TYPE_TIMER, exit, "Unexpected event type\n"); + + /* Handle expiry event */ + n = rte_pdcp_t_reordering_expiry_handle(event.event_ptr, out_mb); + ASSERT_TRUE_OR_GOTO(n == 1, exit, "Unexpected number of expired packets :%i\n", n); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +#else + RTE_SET_USED(ul_conf); + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +test_rte_timer_expiry_handle(struct rte_timer *tim, void *arg) +{ + struct test_rte_timer_args *tim_data = arg; + struct rte_mbuf *out_mb[1] = {0}; + uint16_t n; + + RTE_SET_USED(tim); + + n = rte_pdcp_t_reordering_expiry_handle(tim_data->pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, n); + + tim_data->status = n == 1 ? n : -1; +} + +static void +test_rte_timer_start_cb(void *timer, void *args) +{ + rte_timer_reset_sync(timer, 1, SINGLE, rte_lcore_id(), test_rte_timer_expiry_handle, args); +} + +static int +test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct test_rte_timer_args timer_args; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_timer timer = {0}; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Set up a timer */ + rte_timer_init(&timer); + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &timer_args; + dl_conf.entity.t_reordering.timer = &timer; + dl_conf.entity.t_reordering.start = test_rte_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + timer_args.status = 0; + timer_args.pdcp_entity = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Verify that expire was handled correctly */ + rte_timer_manage(); + while (timer_args.status != 1) { + rte_delay_us(1); + rte_timer_manage(); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, "Bad expire handle status %i\n", + timer_args.status); + } + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1143,6 +1365,126 @@ test_combined(struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static inline void +eventdev_conf_default_set(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info) +{ + memset(dev_conf, 0, sizeof(struct rte_event_dev_config)); + dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns; + dev_conf->nb_event_ports = 1; + dev_conf->nb_event_queues = 1; + dev_conf->nb_event_queue_flows = info->max_event_queue_flows; + dev_conf->nb_event_port_dequeue_depth = info->max_event_port_dequeue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_events_limit = info->max_num_events; +} + +static inline int +eventdev_setup(void) +{ + struct rte_event_dev_config dev_conf; + struct rte_event_dev_info info; + int ret, evdev = 0; + + if (!rte_event_dev_count()) + return TEST_SKIPPED; + + ret = rte_event_dev_info_get(evdev, &info); + TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info"); + TEST_ASSERT(info.max_num_events < 0 || info.max_num_events >= 1, + "ERROR max_num_events=%d < max_events=%d", info.max_num_events, 1); + + eventdev_conf_default_set(&dev_conf, &info); + ret = rte_event_dev_configure(evdev, &dev_conf); + TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev"); + + ret = rte_event_queue_setup(evdev, TEST_EV_QUEUE_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d", TEST_EV_QUEUE_ID); + + /* Configure event port */ + ret = rte_event_port_setup(evdev, TEST_EV_PORT_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d", TEST_EV_PORT_ID); + ret = rte_event_port_link(evdev, TEST_EV_PORT_ID, NULL, NULL, 0); + TEST_ASSERT(ret >= 0, "Failed to link all queues port=%d", TEST_EV_PORT_ID); + + ret = rte_event_dev_start(evdev); + TEST_ASSERT_SUCCESS(ret, "Failed to start device"); + + testsuite_params.evdev = evdev; + + return TEST_SUCCESS; +} + +static int +event_timer_setup(void) +{ + struct rte_event_timer_adapter_info info; + struct rte_event_timer_adapter *timdev; + uint32_t caps = 0; + + struct rte_event_timer_adapter_conf config = { + .event_dev_id = testsuite_params.evdev, + .timer_adapter_id = TIMER_ADAPTER_ID, + .timer_tick_ns = NSECPERSEC, + .max_tmo_ns = 10 * NSECPERSEC, + .nb_timers = 10, + .flags = 0, + }; + + TEST_ASSERT_SUCCESS(rte_event_timer_adapter_caps_get(testsuite_params.evdev, &caps), + "Failed to get adapter capabilities"); + + if (!(caps & RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) + return TEST_SKIPPED; + + timdev = rte_event_timer_adapter_create(&config); + + TEST_ASSERT_NOT_NULL(timdev, "Failed to create event timer ring"); + + testsuite_params.timdev = timdev; + + TEST_ASSERT_EQUAL(rte_event_timer_adapter_start(timdev), 0, + "Failed to start event timer adapter"); + + rte_event_timer_adapter_get_info(timdev, &info); + testsuite_params.min_resolution_ns = info.min_resolution_ns; + + return TEST_SUCCESS; +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +ut_setup_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + int ret; + + ret = eventdev_setup(); + if (ret) + return ret; + + return event_timer_setup(); +#else + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +ut_teardown_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev = testsuite_params.timdev; + int evdev = testsuite_params.evdev; + + rte_event_dev_stop(evdev); + rte_event_dev_close(evdev); + + rte_event_timer_adapter_stop(timdev); + rte_event_timer_adapter_free(timdev); +#endif /* RTE_LIB_EVENTDEV */ +} + static int run_test_for_one_known_vec(const void *arg) { @@ -1187,6 +1529,14 @@ static struct unit_test_suite reorder_test_cases = { ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, test_reorder_buffer_full_window_size_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_event_timer", + ut_setup_pdcp_event_timer, ut_teardown_pdcp_event_timer, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_event_timer), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_rte_timer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_rte_timer), TEST_CASES_END() /**< NULL terminate unit test array */ } }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 18/22] test/pdcp: add timer restart case 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (16 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 17/22] test/pdcp: add timer expiry cases Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 19/22] pdcp: add support for status report Anoob Joseph ` (4 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Add test to cover the case when t-reordering timer should be restarted on the same packet. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 68 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index b4677e5b67..21c27916cb 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -1104,6 +1104,71 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) return ret; } +static int +test_reorder_gap_in_reorder_buffer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + int ret = TEST_FAILED, nb_out, i; + struct pdcp_test_conf dl_conf; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Create two gaps [NULL, m1, NULL, m3]*/ + for (i = 0; i < 2; i++) { + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + 2 * i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m = NULL; /* Packet was moved to PDCP lib */ + } + + /* Generate packet to fill the first gap */ + m = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1, NULL, m3] + * Only first gap should be filled, timer should be restarted for second gap + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m = NULL; + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), + exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == true, exit, + "Timer should be restarted after partial drain"); + + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) { @@ -1525,6 +1590,9 @@ static struct unit_test_suite reorder_test_cases = { TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_in_reorder_buffer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_in_reorder_buffer), TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 19/22] pdcp: add support for status report 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (17 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 18/22] test/pdcp: add timer restart case Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph ` (3 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Implement status report generation for PDCP entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 158 ++++++++++++++++++++++++++++++++++++--- lib/pdcp/pdcp_cnt.h | 11 ++- lib/pdcp/pdcp_ctrl_pdu.c | 34 ++++++++- lib/pdcp/pdcp_ctrl_pdu.h | 3 +- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 9 ++- lib/pdcp/pdcp_process.h | 13 ++++ lib/pdcp/rte_pdcp.c | 34 ++++++--- 8 files changed, 236 insertions(+), 28 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index c9b952184b..af027b00d3 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -2,28 +2,164 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_pdcp.h> #include "pdcp_cnt.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#define SLAB_BYTE_SIZE (RTE_BITMAP_SLAB_BIT_SIZE / 8) + +uint32_t +pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) +{ + uint32_t n_bits = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + return rte_bitmap_get_memory_footprint(n_bits); +} + int -pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) { - struct entity_priv_dl_part *en_priv_dl; - uint32_t window_sz; + uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - if (en == NULL || conf == NULL) + dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + if (dl->bitmap.bmp == NULL) return -EINVAL; - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) - return 0; + dl->bitmap.size = window_size; - en_priv_dl = entity_dl_part_get(en); - window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + return 0; +} - RTE_SET_USED(window_sz); - RTE_SET_USED(en_priv_dl); +void +pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + rte_bitmap_set(bitmap.bmp, count % bitmap.size); +} - return 0; +bool +pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + return rte_bitmap_get(bitmap.bmp, count % bitmap.size); +} + +void +pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop) +{ + uint32_t i; + + for (i = start; i < stop; i++) + rte_bitmap_clear(bitmap.bmp, i % bitmap.size); +} + +uint16_t +pdcp_cnt_get_bitmap_size(uint32_t pending_bytes) +{ + /* + * Round up bitmap size to slab size to operate only on slabs sizes, instead of individual + * bytes + */ + return RTE_ALIGN_MUL_CEIL(pending_bytes, SLAB_BYTE_SIZE); +} + +static __rte_always_inline uint64_t +leftover_get(uint64_t slab, uint32_t shift, uint64_t mask) +{ + return (slab & mask) << shift; +} + +void +pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len) +{ + uint64_t slab = 0, next_slab = 0, leftover; + uint32_t zeros, report_len, diff; + uint32_t slab_id, next_slab_id; + uint32_t pos = 0, next_pos = 0; + + const uint32_t start_count = state.rx_deliv + 1; + const uint32_t nb_slabs = bitmap.size / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t nb_data_slabs = data_len / SLAB_BYTE_SIZE; + const uint32_t start_slab_id = start_count / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t stop_slab_id = (start_slab_id + nb_data_slabs) % nb_slabs; + const uint32_t shift = start_count % RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t leftover_shift = shift ? RTE_BITMAP_SLAB_BIT_SIZE - shift : 0; + const uint8_t *data_end = RTE_PTR_ADD(data, data_len + SLAB_BYTE_SIZE); + + /* NOTE: Mask required to workaround case - when shift is not needed */ + const uint64_t leftover_mask = shift ? ~0 : 0; + + /* NOTE: implement scan init at to set custom position */ + __rte_bitmap_scan_init(bitmap.bmp); + while (true) { + assert(rte_bitmap_scan(bitmap.bmp, &pos, &slab) == 1); + slab_id = pos / RTE_BITMAP_SLAB_BIT_SIZE; + if (slab_id >= start_slab_id) + break; + } + + report_len = nb_data_slabs; + + if (slab_id > start_slab_id) { + /* Zero slabs at beginning */ + zeros = (slab_id - start_slab_id - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + leftover = leftover_get(slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + report_len -= (slab_id - start_slab_id); + } + + while (report_len) { + rte_bitmap_scan(bitmap.bmp, &next_pos, &next_slab); + next_slab_id = next_pos / RTE_BITMAP_SLAB_BIT_SIZE; + diff = (next_slab_id + nb_slabs - slab_id) % nb_slabs; + + /* If next_slab_id == slab_id - overlap */ + diff += !(next_slab_id ^ slab_id) * nb_slabs; + + /* Size check - next slab is outsize of size range */ + if (diff > report_len) { + next_slab = 0; + next_slab_id = stop_slab_id; + diff = report_len; + } + + report_len -= diff; + + /* Calculate gap between slabs, taking wrap around into account */ + zeros = (next_slab_id + nb_slabs - slab_id - 1) % nb_slabs; + if (zeros) { + /* Non continues slabs, align them individually */ + slab >>= shift; + memcpy(data, &slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + + /* Fill zeros between slabs */ + zeros = (zeros - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + + /* Align beginning of next slab */ + leftover = leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } else { + /* Continues slabs, combine them */ + uint64_t new_slab = (slab >> shift) | + leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &new_slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } + + slab = next_slab; + pos = next_pos; + slab_id = next_slab_id; + + }; + + assert(data < data_end); } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index bbda478b55..5941b7a406 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -9,6 +9,15 @@ #include "pdcp_entity.h" -int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); +uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); + +void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +void pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop); + +uint16_t pdcp_cnt_get_bitmap_size(uint32_t pending_bytes); +void pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len); #endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c index feb05fd863..e0ac2d3720 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.c +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -8,6 +8,14 @@ #include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#include "pdcp_cnt.h" + +static inline uint16_t +round_up_bits(uint32_t bits) +{ + /* round up to the next multiple of 8 */ + return RTE_ALIGN_MUL_CEIL(bits, 8) / 8; +} static __rte_always_inline void pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) @@ -19,11 +27,13 @@ pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) } int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m) { struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; - uint32_t rx_deliv; - int pdu_sz; + uint32_t rx_deliv, actual_sz; + uint16_t pdu_sz, bitmap_sz; + uint8_t *data; if (!en_priv->flags.is_status_report_required) return -EINVAL; @@ -42,5 +52,21 @@ pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) return 0; } - return -ENOTSUP; + actual_sz = RTE_MIN(round_up_bits(en_priv->state.rx_next - rx_deliv - 1), + RTE_PDCP_CTRL_PDU_SIZE_MAX - pdu_sz); + bitmap_sz = pdcp_cnt_get_bitmap_size(actual_sz); + + data = (uint8_t *)rte_pktmbuf_append(m, pdu_sz + bitmap_sz); + if (data == NULL) + return -ENOMEM; + + m->pkt_len = pdu_sz + actual_sz; + m->data_len = pdu_sz + actual_sz; + + pdcp_hdr_fill((struct rte_pdcp_up_ctrl_pdu_hdr *)data, rx_deliv); + + data = RTE_PTR_ADD(data, pdu_sz); + pdcp_cnt_report_fill(dl->bitmap, en_priv->state, data, bitmap_sz); + + return 0; } diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h index a2424fbd10..2a87928b88 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.h +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -10,6 +10,7 @@ #include "pdcp_entity.h" int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m); #endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index efc74ba9b9..a9b1428c7a 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -182,6 +182,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /** Bitmap memory region */ + uint8_t bitmap_mem[0]; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index e273dd4658..8d42615270 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -9,6 +9,7 @@ #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -837,11 +838,15 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; + if (unlikely(pdcp_cnt_bitmap_is_set(dl->bitmap, count))) + return -EEXIST; + + pdcp_cnt_bitmap_set(dl->bitmap, count); pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); if (en_priv->flags.is_out_of_order_delivery) { out_mb[0] = mb; - en_priv->state.rx_deliv = count + 1; + pdcp_rx_deliv_set(entity, count + 1); return 1; } @@ -868,7 +873,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, } /* Processed should never exceed the window size */ - en_priv->state.rx_deliv = count + processed; + pdcp_rx_deliv_set(entity, count + processed); } else { if (!reorder->is_active) diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index fd53fff0aa..95bb9d1119 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -7,7 +7,20 @@ #include <rte_pdcp.h> +#include <pdcp_entity.h> +#include <pdcp_cnt.h> + int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); +static inline void +pdcp_rx_deliv_set(const struct rte_pdcp_entity *entity, uint32_t rx_deliv) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + pdcp_cnt_bitmap_range_clear(dl->bitmap, en_priv->state.rx_deliv, rx_deliv); + en_priv->state.rx_deliv = rx_deliv; +} + #endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 32ec8bc8f1..cd79633134 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -12,6 +12,8 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +static int bitmap_mem_offset; + static int pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { @@ -19,9 +21,12 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); - else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size = RTE_CACHE_LINE_ROUNDUP(size); + bitmap_mem_offset = size; + size += pdcp_cnt_bitmap_get_memory_footprint(conf); + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; @@ -34,11 +39,24 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + void *bitmap_mem; + int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - return pdcp_reorder_create(&dl->reorder, window_size); + ret = pdcp_reorder_create(&dl->reorder, window_size); + if (ret) + return ret; + + bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); + ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); + if (ret) { + pdcp_reorder_destroy(&dl->reorder); + return ret; + } + + return 0; } struct rte_pdcp_entity * @@ -113,10 +131,6 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; } - ret = pdcp_cnt_ring_create(entity, conf); - if (ret) - goto crypto_sess_destroy; - return entity; crypto_sess_destroy: @@ -195,6 +209,7 @@ struct rte_mbuf * rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, enum rte_pdcp_ctrl_pdu_type type) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; struct rte_mbuf *m; int ret; @@ -205,6 +220,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, } en_priv = entity_priv_get(pdcp_entity); + dl = entity_dl_part_get(pdcp_entity); m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); if (m == NULL) { @@ -214,7 +230,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, switch (type) { case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: - ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + ret = pdcp_ctrl_pdu_status_gen(en_priv, dl, m); break; default: ret = -ENOTSUP; @@ -260,7 +276,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered * to upper layers, with COUNT value >= RX_REORD; */ - en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + pdcp_rx_deliv_set(entity, en_priv->state.rx_reord + nb_seq); /* * - if RX_DELIV < RX_NEXT: -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 20/22] pdcp: allocate reorder buffer alongside with entity 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (18 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 19/22] pdcp: add support for status report Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 21/22] pdcp: add thread safe processing Anoob Joseph ` (2 subsequent siblings) 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Instead of allocating reorder buffer separately on heap, allocate memory for it together with rest of entity, and then only initialize buffer via `rte_reorder_init()`. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 9 +++---- lib/pdcp/pdcp_cnt.h | 3 ++- lib/pdcp/pdcp_entity.h | 2 +- lib/pdcp/pdcp_reorder.c | 11 ++------ lib/pdcp/pdcp_reorder.h | 12 ++++++--- lib/pdcp/rte_pdcp.c | 58 ++++++++++++++++++++++++++--------------- 6 files changed, 55 insertions(+), 40 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index af027b00d3..e1d0634b4d 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -20,15 +20,14 @@ pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) } int -pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size) { - uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - - dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + dl->bitmap.bmp = rte_bitmap_init(nb_elem, bitmap_mem, mem_size); if (dl->bitmap.bmp == NULL) return -EINVAL; - dl->bitmap.size = window_size; + dl->bitmap.size = nb_elem; return 0; } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index 5941b7a406..87b011f9dc 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -10,7 +10,8 @@ #include "pdcp_entity.h" uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); -int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size); void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index a9b1428c7a..9f74b5d0e5 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -132,7 +132,7 @@ struct pdcp_cnt_bitmap { }; /* - * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] [reorder/bitmap] */ struct entity_priv { diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c index 5399f0dc28..bc45f2e19b 100644 --- a/lib/pdcp/pdcp_reorder.c +++ b/lib/pdcp/pdcp_reorder.c @@ -8,20 +8,13 @@ #include "pdcp_reorder.h" int -pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size) { - reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + reorder->buf = rte_reorder_init(mem, mem_size, "reorder_buffer", nb_elem); if (reorder->buf == NULL) return -rte_errno; - reorder->window_size = window_size; reorder->is_active = false; return 0; } - -void -pdcp_reorder_destroy(const struct pdcp_reorder *reorder) -{ - rte_reorder_free(reorder->buf); -} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h index 6a2f61d6ae..7e4f079d4b 100644 --- a/lib/pdcp/pdcp_reorder.h +++ b/lib/pdcp/pdcp_reorder.h @@ -9,12 +9,18 @@ struct pdcp_reorder { struct rte_reorder_buffer *buf; - uint32_t window_size; bool is_active; }; -int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); -void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); +int pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size); + +/* NOTE: replace with `rte_reorder_memory_footprint_get` after DPDK 23.07 */ +#define SIZE_OF_REORDER_BUFFER (4 * RTE_CACHE_LINE_SIZE) +static inline size_t +pdcp_reorder_memory_footprint_get(size_t nb_elem) +{ + return SIZE_OF_REORDER_BUFFER + (2 * nb_elem * sizeof(struct rte_mbuf *)); +} static inline uint32_t pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index cd79633134..9155669d69 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -12,49 +12,65 @@ #include "pdcp_entity.h" #include "pdcp_process.h" -static int bitmap_mem_offset; +struct entity_layout { + size_t bitmap_offset; + size_t bitmap_size; + + size_t reorder_buf_offset; + size_t reorder_buf_size; + + size_t total_size; +}; static int -pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +pdcp_entity_layout_get(const struct rte_pdcp_entity_conf *conf, struct entity_layout *layout) { - int size; + size_t size; + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); + /* Bitmap require memory to be cache aligned */ size = RTE_CACHE_LINE_ROUNDUP(size); - bitmap_mem_offset = size; - size += pdcp_cnt_bitmap_get_memory_footprint(conf); + layout->bitmap_offset = size; + layout->bitmap_size = pdcp_cnt_bitmap_get_memory_footprint(conf); + size += layout->bitmap_size; + layout->reorder_buf_offset = size; + layout->reorder_buf_size = pdcp_reorder_memory_footprint_get(window_size); + size += layout->reorder_buf_size; } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; - return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); + layout->total_size = size; + + return 0; } static int -pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf, + const struct entity_layout *layout) { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); - void *bitmap_mem; + void *memory; int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - ret = pdcp_reorder_create(&dl->reorder, window_size); + memory = RTE_PTR_ADD(entity, layout->reorder_buf_offset); + ret = pdcp_reorder_create(&dl->reorder, window_size, memory, layout->reorder_buf_size); if (ret) return ret; - bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); - ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); - if (ret) { - pdcp_reorder_destroy(&dl->reorder); + memory = RTE_PTR_ADD(entity, layout->bitmap_offset); + ret = pdcp_cnt_bitmap_create(dl, window_size, memory, layout->bitmap_size); + if (ret) return ret; - } return 0; } @@ -62,10 +78,11 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { + struct entity_layout entity_layout = { 0 }; struct rte_pdcp_entity *entity = NULL; struct entity_priv *en_priv; - int ret, entity_size; uint32_t count; + int ret; if (conf == NULL || conf->cop_pool == NULL || conf->ctrl_pdu_pool == NULL) { rte_errno = EINVAL; @@ -95,13 +112,14 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } - entity_size = pdcp_entity_size_get(conf); - if (entity_size < 0) { + ret = pdcp_entity_layout_get(conf, &entity_layout); + if (ret < 0) { rte_errno = EINVAL; return NULL; } - entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + entity = rte_zmalloc_socket("pdcp_entity", entity_layout.total_size, RTE_CACHE_LINE_SIZE, + SOCKET_ID_ANY); if (entity == NULL) { rte_errno = ENOMEM; return NULL; @@ -126,7 +144,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { - ret = pdcp_dl_establish(entity, conf); + ret = pdcp_dl_establish(entity, conf, &entity_layout); if (ret) goto crypto_sess_destroy; } @@ -151,8 +169,6 @@ pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, en_priv->state.rx_next); - pdcp_reorder_destroy(&dl->reorder); - return nb_out; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 21/22] pdcp: add thread safe processing 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (19 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-24 18:31 ` Stephen Hemminger 2023-05-24 16:01 ` [PATCH v3 22/22] test/pdcp: add PDCP status report cases Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph 22 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> PDCP state has to be guarded for: - Uplink pre_process: - tx_next atomic increment - Downlink pre_process: - rx_deliv - read - Downlink post_process: - rx_deliv, rx_reorder, rx_next - read/write - bitmask/reorder buffer - read/write When application requires thread safe processing, the state variables need to be updated atomically. Add config option to select this option per entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 46 +++++++++++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 30 ++++++++++++++++++++++++--- lib/pdcp/rte_pdcp.c | 6 ++++++ lib/pdcp/rte_pdcp.h | 2 ++ 4 files changed, 81 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 9f74b5d0e5..dc05c19bc0 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -10,6 +10,7 @@ #include <rte_mempool.h> #include <rte_pdcp.h> #include <rte_security.h> +#include <rte_spinlock.h> #include "pdcp_reorder.h" @@ -162,6 +163,8 @@ struct entity_priv { uint64_t is_status_report_required : 1; /** Is out-of-order delivery enabled */ uint64_t is_out_of_order_delivery : 1; + /** Is thread safety disabled */ + uint64_t is_thread_safety_disabled : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -175,6 +178,8 @@ struct entity_priv { uint8_t dev_id; }; +typedef rte_spinlock_t pdcp_lock_t; + struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; @@ -182,6 +187,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /* Lock to protect concurrent updates */ + pdcp_lock_t lock; /** Bitmap memory region */ uint8_t bitmap_mem[0]; }; @@ -257,4 +264,43 @@ pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) return (1 << (32 - sn_size)) - 1; } +static inline uint32_t +pdcp_atomic_inc(const struct entity_priv *en_priv, uint32_t *val) +{ + if (en_priv->flags.is_thread_safety_disabled) + return (*val)++; + else + return __atomic_fetch_add(val, 1, __ATOMIC_RELAXED); +} + +static inline void +pdcp_lock_init(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_init(&dl->lock); +} + +static inline void +pdcp_lock_lock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_lock(&dl->lock); +} + +static inline void +pdcp_lock_unlock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_unlock(&dl->lock); +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 8d42615270..2f253ad76c 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -369,7 +369,7 @@ pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_ return false; /* Update sequence num in the PDU header */ - *count = en_priv->state.tx_next++; + *count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; @@ -451,7 +451,7 @@ pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_ return false; /* Update sequence num in the PDU header */ - *count = en_priv->state.tx_next++; + *count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; @@ -561,7 +561,7 @@ pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rt memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); /* Update sequence number in the PDU header */ - count = en_priv->state.tx_next++; + count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); @@ -654,7 +654,9 @@ pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -717,7 +719,9 @@ pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -781,7 +785,9 @@ pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rt nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -923,6 +929,8 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb; uint32_t count; + pdcp_lock_lock(entity); + for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -949,6 +957,8 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -989,6 +999,7 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, int32_t rsn = 0; uint32_t count; + pdcp_lock_lock(entity); for (i = 0; i < num; i++) { mb = in_mb[i]; @@ -1017,6 +1028,8 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -1057,6 +1070,8 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, uint32_t count; int32_t rsn; + pdcp_lock_lock(entity); + for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -1082,6 +1097,8 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -1245,6 +1262,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent */ en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** + * flags.disable_thread_safety + * + * Indicate whether the thread safety is disabled for PDCP entity. + */ + en_priv->flags.is_thread_safety_disabled = conf->disable_thread_safety; + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 9155669d69..2a1eb20724 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -72,6 +72,8 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c if (ret) return ret; + pdcp_lock_init(entity); + return 0; } @@ -276,6 +278,8 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * performing header decompression, if not decompressed before: */ + pdcp_lock_lock(entity); + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); capacity -= nb_out; @@ -307,5 +311,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct dl->t_reorder.state = TIMER_EXPIRED; } + pdcp_lock_unlock(entity); + return nb_out; } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 76ec569577..cbbedc851f 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -141,6 +141,8 @@ struct rte_pdcp_entity_conf { bool is_slrb; /** Enable security offload on the device specified. */ bool en_sec_offload; + /** Disable usage of synchronization primitives for entity. */ + bool disable_thread_safety; /** Device on which security/crypto session need to be created. */ uint8_t dev_id; /** -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [PATCH v3 21/22] pdcp: add thread safe processing 2023-05-24 16:01 ` [PATCH v3 21/22] pdcp: add thread safe processing Anoob Joseph @ 2023-05-24 18:31 ` Stephen Hemminger 2023-05-25 8:15 ` [EXT] " Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Stephen Hemminger @ 2023-05-24 18:31 UTC (permalink / raw) To: Anoob Joseph Cc: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz On Wed, 24 May 2023 21:31:15 +0530 Anoob Joseph <anoobj@marvell.com> wrote: > From: Volodymyr Fialko <vfialko@marvell.com> > > PDCP state has to be guarded for: > > - Uplink pre_process: > - tx_next atomic increment > > - Downlink pre_process: > - rx_deliv - read > > - Downlink post_process: > - rx_deliv, rx_reorder, rx_next - read/write > - bitmask/reorder buffer - read/write > > When application requires thread safe processing, the state variables > need to be updated atomically. Add config option to select this option > per entity. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> NAK Conditional locking is a bad design pattern. It leads to lots of problems, and makes it almost impossible for analysis tools. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [EXT] Re: [PATCH v3 21/22] pdcp: add thread safe processing 2023-05-24 18:31 ` Stephen Hemminger @ 2023-05-25 8:15 ` Anoob Joseph 2023-05-25 15:25 ` Stephen Hemminger 0 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-05-25 8:15 UTC (permalink / raw) To: Stephen Hemminger Cc: Thomas Monjalon, Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz Hi Stephen, Please see inline. Thanks, Anoob > -----Original Message----- > From: Stephen Hemminger <stephen@networkplumber.org> > Sent: Thursday, May 25, 2023 12:02 AM > To: Anoob Joseph <anoobj@marvell.com> > Cc: Thomas Monjalon <thomas@monjalon.net>; Akhil Goyal > <gakhil@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com>; Volodymyr Fialko > <vfialko@marvell.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; > Mattias Rönnblom <mattias.ronnblom@ericsson.com>; Kiran Kumar > Kokkilagadda <kirankumark@marvell.com>; dev@dpdk.org; Olivier Matz > <olivier.matz@6wind.com> > Subject: [EXT] Re: [PATCH v3 21/22] pdcp: add thread safe processing > > External Email > > ---------------------------------------------------------------------- > On Wed, 24 May 2023 21:31:15 +0530 > Anoob Joseph <anoobj@marvell.com> wrote: > > > From: Volodymyr Fialko <vfialko@marvell.com> > > > > PDCP state has to be guarded for: > > > > - Uplink pre_process: > > - tx_next atomic increment > > > > - Downlink pre_process: > > - rx_deliv - read > > > > - Downlink post_process: > > - rx_deliv, rx_reorder, rx_next - read/write > > - bitmask/reorder buffer - read/write > > > > When application requires thread safe processing, the state variables > > need to be updated atomically. Add config option to select this option > > per entity. > > > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > > NAK > Conditional locking is a bad design pattern. > It leads to lots of problems, and makes it almost impossible for analysis tools. [Anoob] With PDCP (& most other protocols), we have to update the states atomically. Application designers would have a choice of either use single thread or do multi-thread processing. If the library is designed for multi-thread and if application uses only single thread, then there would be unnecessary overheads from library. If library sticks to single-thread and if application needs more threads for scaling, then again it would become a library issue. Is your issue with providing such an option or is it about how it is implemented? IPsec also has a similar challenge and similar per SA configuration is provided in lib IPsec as well. ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [EXT] Re: [PATCH v3 21/22] pdcp: add thread safe processing 2023-05-25 8:15 ` [EXT] " Anoob Joseph @ 2023-05-25 15:25 ` Stephen Hemminger 2023-05-25 15:37 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Stephen Hemminger @ 2023-05-25 15:25 UTC (permalink / raw) To: Anoob Joseph Cc: Thomas Monjalon, Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz On Thu, 25 May 2023 08:15:07 +0000 Anoob Joseph <anoobj@marvell.com> wrote: > [Anoob] With PDCP (& most other protocols), we have to update the states atomically. Application designers would have a choice of either use single thread or do multi-thread processing. If the library is designed for multi-thread and if application uses only single thread, then there would be unnecessary overheads from library. If library sticks to single-thread and if application needs more threads for scaling, then again it would become a library issue. > > Is your issue with providing such an option or is it about how it is implemented? IPsec also has a similar challenge and similar per SA configuration is provided in lib IPsec as well. If you want to provide unlocked access, then it should be done with another set of API's. The cost of conditional branch will be higher than atomic some times. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [EXT] Re: [PATCH v3 21/22] pdcp: add thread safe processing 2023-05-25 15:25 ` Stephen Hemminger @ 2023-05-25 15:37 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-25 15:37 UTC (permalink / raw) To: Stephen Hemminger Cc: Thomas Monjalon, Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Bernard Iremonger, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz Hi Stephen, Please see inline. Thanks, Anoob > -----Original Message----- > From: Stephen Hemminger <stephen@networkplumber.org> > Sent: Thursday, May 25, 2023 8:56 PM > To: Anoob Joseph <anoobj@marvell.com> > Cc: Thomas Monjalon <thomas@monjalon.net>; Akhil Goyal > <gakhil@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Bernard > Iremonger <bernard.iremonger@intel.com>; Volodymyr Fialko > <vfialko@marvell.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; > Mattias Rönnblom <mattias.ronnblom@ericsson.com>; Kiran Kumar > Kokkilagadda <kirankumark@marvell.com>; dev@dpdk.org; Olivier Matz > <olivier.matz@6wind.com> > Subject: Re: [EXT] Re: [PATCH v3 21/22] pdcp: add thread safe processing > > On Thu, 25 May 2023 08:15:07 +0000 > Anoob Joseph <anoobj@marvell.com> wrote: > > > [Anoob] With PDCP (& most other protocols), we have to update the states > atomically. Application designers would have a choice of either use single > thread or do multi-thread processing. If the library is designed for multi- > thread and if application uses only single thread, then there would be > unnecessary overheads from library. If library sticks to single-thread and if > application needs more threads for scaling, then again it would become a > library issue. > > > > Is your issue with providing such an option or is it about how it is > implemented? IPsec also has a similar challenge and similar per SA > configuration is provided in lib IPsec as well. > > If you want to provide unlocked access, then it should be done with another > set of API's. > > The cost of conditional branch will be higher than atomic some times. [Anoob] Understood. I'll try to introduce some const flags so that compiler optimized threads can be registered. That way the conditional branch could be avoided. ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v3 22/22] test/pdcp: add PDCP status report cases 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (20 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 21/22] pdcp: add thread safe processing Anoob Joseph @ 2023-05-24 16:01 ` Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph 22 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-24 16:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Bernard Iremonger Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz From: Volodymyr Fialko <vfialko@marvell.com> Test PDCP status report generation. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 312 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 312 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 21c27916cb..39463c6360 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -2,6 +2,7 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_errno.h> #ifdef RTE_LIB_EVENTDEV #include <rte_eventdev.h> @@ -48,6 +49,9 @@ struct pdcp_testsuite_params { #endif /* RTE_LIB_EVENTDEV */ bool timer_is_running; uint64_t min_resolution_ns; + struct rte_pdcp_up_ctrl_pdu_hdr *status_report; + uint32_t status_report_bitmask_capacity; + uint8_t *ctrl_pdu_buf; }; static struct pdcp_testsuite_params testsuite_params; @@ -168,6 +172,18 @@ static struct rte_pdcp_t_reordering t_reorder_timer = { .stop = pdcp_timer_stop_cb, }; +static inline void +bitmask_set_bit(uint8_t *mask, uint32_t bit) +{ + mask[bit / 8] |= (1 << bit % 8); +} + +static inline bool +bitmask_is_bit_set(const uint8_t *mask, uint32_t bit) +{ + return mask[bit / 8] & (1 << (bit % 8)); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -314,6 +330,21 @@ testsuite_setup(void) goto cop_pool_free; } + /* Allocate memory for longest possible status report */ + ts_params->status_report_bitmask_capacity = RTE_PDCP_CTRL_PDU_SIZE_MAX - + sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + ts_params->status_report = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->status_report == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report\n"); + goto cop_pool_free; + } + + ts_params->ctrl_pdu_buf = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->ctrl_pdu_buf == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report data\n"); + goto cop_pool_free; + } + return 0; cop_pool_free: @@ -322,6 +353,8 @@ testsuite_setup(void) mbuf_pool_free: rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); return TEST_FAILED; } @@ -344,6 +377,9 @@ testsuite_teardown(void) rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); } static int @@ -1408,6 +1444,246 @@ test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) return ret; } +static struct rte_pdcp_up_ctrl_pdu_hdr * +pdcp_status_report_init(uint32_t fmc) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + + hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + hdr->fmc = rte_cpu_to_be_32(fmc); + hdr->r = 0; + memset(hdr->bitmap, 0, testsuite_params.status_report_bitmask_capacity); + + return hdr; +} + +static uint32_t +pdcp_status_report_len(void) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + uint32_t i; + + for (i = testsuite_params.status_report_bitmask_capacity; i != 0; i--) { + if (hdr->bitmap[i - 1]) + return i; + } + + return 0; +} + +static int +pdcp_status_report_verify(struct rte_mbuf *status_report, + const struct rte_pdcp_up_ctrl_pdu_hdr *expected_hdr, uint32_t expected_len) +{ + uint32_t received_len = rte_pktmbuf_pkt_len(status_report); + uint8_t *received_buf = testsuite_params.ctrl_pdu_buf; + int ret; + + ret = pktmbuf_read_into(status_report, received_buf, RTE_PDCP_CTRL_PDU_SIZE_MAX); + TEST_ASSERT_SUCCESS(ret, "Failed to copy status report pkt into continuous buffer"); + + debug_hexdump(stdout, "Received:", received_buf, received_len); + debug_hexdump(stdout, "Expected:", expected_hdr, expected_len); + + TEST_ASSERT_EQUAL(expected_len, received_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_len, received_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(received_buf, expected_hdr, expected_len, + "Generated packet not as expected"); + + return 0; +} + +static int +test_status_report_gen(const struct pdcp_test_conf *ul_conf, + const struct rte_pdcp_up_ctrl_pdu_hdr *hdr, + uint32_t bitmap_len) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *status_report = NULL, **out_mb, *m; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint32_t nb_pkts = 0, i; + uint8_t cdev_id; + + const uint32_t start_count = rte_be_to_cpu_32(hdr->fmc); + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.status_report_required = true; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + out_mb = calloc(pdcp_entity->max_pkt_cache, sizeof(uintptr_t)); + + for (i = 0; i < bitmap_len * 8; i++) { + if (!bitmask_is_bit_set(hdr->bitmap, i)) + continue; + + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + + } + + m = NULL; + + /* Check status report */ + status_report = rte_pdcp_control_pdu_create(pdcp_entity, + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT); + ASSERT_TRUE_OR_GOTO(status_report != NULL, exit, "Could not generate status report\n"); + + const uint32_t expected_len = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr) + bitmap_len; + + ASSERT_TRUE_OR_GOTO(pdcp_status_report_verify(status_report, hdr, expected_len) == 0, exit, + "Report verification failure\n"); + + ret = TEST_SUCCESS; +exit: + rte_free(m); + rte_pktmbuf_free(status_report); + rte_pktmbuf_free_bulk(out_mb, nb_pkts); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + free(out_mb); + return ret; +} + +static void +ctrl_pdu_hdr_packet_set(struct rte_pdcp_up_ctrl_pdu_hdr *hdr, uint32_t pkt_count) +{ + bitmask_set_bit(hdr->bitmap, pkt_count - rte_be_to_cpu_32(hdr->fmc) - 1); +} + +static int +test_status_report_fmc_only(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(42); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_first_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_second_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_full_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE + 1; + int i; + + for (i = 0; i < RTE_BITMAP_SLAB_BIT_SIZE; i++) + ctrl_pdu_hdr_packet_set(hdr, start_offset + i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_non_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + ctrl_pdu_hdr_packet_set(hdr, 3 * RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_max_length_sn_12(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr; + const uint32_t fmc = 0; + uint32_t i; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + ul_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + hdr = pdcp_status_report_init(fmc); + + const uint32_t max_count = RTE_MIN((RTE_PDCP_CTRL_PDU_SIZE_MAX - sizeof(hdr)) * 8, + (uint32_t)PDCP_WINDOW_SIZE(RTE_SECURITY_PDCP_SN_SIZE_12)); + + i = fmc + 2; /* set first count to have a gap, to enable packet buffering */ + + for (; i < max_count; i++) + ctrl_pdu_hdr_packet_set(hdr, i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_different_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(63); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 64 + 1); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_same_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(2); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 4); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1609,11 +1885,47 @@ static struct unit_test_suite reorder_test_cases = { } }; +static struct unit_test_suite status_report_test_cases = { + .suite_name = "PDCP status report", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_status_report_fmc_only", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_fmc_only), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_first_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_first_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_second_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_second_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_full_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_full_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_non_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_non_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_max_length_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_status_report_max_length_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_different_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_different_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_same_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_same_slab), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, &reorder_test_cases, + &status_report_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 00/22] lib: add pdcp protocol 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph ` (21 preceding siblings ...) 2023-05-24 16:01 ` [PATCH v3 22/22] test/pdcp: add PDCP status report cases Anoob Joseph @ 2023-05-26 21:01 ` Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 01/22] net: add PDCP header Anoob Joseph ` (23 more replies) 22 siblings, 24 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following operations, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Similar to lib IPsec, lib PDCP would allow application to use same API sequence while leveraging protocol offload features enabled by rte_security library. Lib PDCP would internally change the handles registered for *pre_process* and *post_process* based on features enabled in the entity. Lib PDCP would create the required sessions on the device provided in entity to minimize the application requirements. Also, the crypto_op allocation and free would also be done internally by lib PDCP to allow the library to create crypto ops as required for the input packets. For example, when control PDUs are received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP is expected to handle it differently. Lib PDCP utilizes reorder library for implementing in-order delivery. It utilizes bitmap library for implementing status reports and track the COUNT value of the packets received. To allow application to choose timer implementation of choice, lib PDCP allows application to configure handles that can be used for starting & stopping timers. Upon expiry, application can call corresponding PDCP API(``rte_pdcp_t_reordering_expiry_handle``) for handling the event. Unit tests are added to verify both rte_timer based timers as well as rte_eventdev based timers. PDCP tracks the sequence number of the received packets and during events such as re-establishment, it is required to generate reports and transmit to the peer. This series introduces ``rte_pdcp_control_pdu_create`` for handling control PDU generation. Changes in v4: - Disabled 'annotate locks' with lib PDCP - Enable PDCP autotest only when lib is enabled - Use rwlock instead of spinlock - Avoid per packet checks for thread safety (Stephen) - In DL path, save count determined during pre-process in mbuf and use the same in post-process. Determining count again may lead To errors - Simplified DL path threads to allow more common code between SN 12 & 18 Changes in v3: - Addressed review comments (Akhil) - Addressed build failure in CI (tests with lib eventdev disabled) - Addressed checkpatch issues - Set only positive values to rte_errno (Akhil) Changes in v2: - Added control PDU handling - Added t-Reordering timer - Added in-order delivery - Added status PDU generation - Rebased on top of new features added in reorder library - Split base patch - Increased test coverage - Improved thread safety Changes from RFC - Implementation for all APIs covering basic control plane & user plane packets - Unit test leveraging existing PDCP vectors available in test_cryptodev - Unit test performing both UL & DL operations to verify various protocol features - Updated documentation Sample application sequence: struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; struct rte_crypto_op *cop[MAX_BURST_SIZE]; struct rte_pdcp_group grp[MAX_BURST_SIZE]; struct rte_pdcp_entity *pdcp_entity; int nb_max_out_mb, ret, nb_grp; /* Create PDCP entity */ pdcp_entity = rte_pdcp_entity_establish(&conf); /** * Allocate buffer for holding mbufs returned during PDCP suspend, * release & post-process APIs. */ /* Max packets that can be cached in entity + burst size */ nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); if (out_mb == NULL) { /* Handle error */ } while (1) { /* Receive packet and form mbuf */ /** * Prepare packets for crypto operation. Following operations * would be done, * * Transmitting entity/UL (only data PDUs): * - Perform compression * - Assign sequence number * - Add PDCP header * - Create & prepare crypto_op * - Prepare IV for crypto operation (auth_gen, encrypt) * - Save original PDCP SDU (during PDCP re-establishment, * unconfirmed PDCP SDUs need to crypto processed again and * transmitted/re-transmitted) * * Receiving entity/DL: * - Any control PDUs received would be processed and * appropriate actions taken. If data PDU, continue. * - Determine sequence number (based on HFN & per packet SN) * - Prepare crypto_op * - Prepare IV for crypto operation (decrypt, auth_verify) */ nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, nb_rx, &nb_err); if (nb_err != 0) { /* Handle error packets */ } if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) != nb_success) { /* Retry for enqueue failure packets */ } ... ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, MAX_BURST_SIZE); if (ret == 0) continue; /** * Received a burst of completed crypto ops from cryptodev. It * may belong to various entities. Group similar ones together * for entity specific post-processing. */ /** * Groups similar entities together. Frees crypto op and based * on crypto_op status, set mbuf->ol_flags which would be * checked in rte_pdcp_pkt_post_process(). */ nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); for (i = 0; i != nb_grp; i++) { /** * Post process packets after crypto completion. * Following operations would be done, * * Transmitting entity/UL: * - Check crypto result * * Receiving entity/DL: * - Check crypto operation status * - Check for duplication (if yes, drop duplicate) * - Perform decompression * - Trim PDCP header * - Hold packet (SDU) for in-order delivery (return * completed packets as and when sequence is * completed) * - If not in sequence, cache the packet and start * t-Reordering timer. When timer expires, the * packets need to delivered to upper layers (not * treated as error packets). */ nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, grp[i].m, out_mb, grp[i].cnt, &nb_err); if (nb_err != 0) { /* Handle error packets */ } /* Perform additional operations */ /** * Tranmitting entity/UL * - If duplication is enabled, duplicate PDCP PDUs * - When lower layers confirm reception of a PDCP PDU, * it should be communicated to PDCP layer so that * PDCP can drop the corresponding SDU */ } } Sample output from unit test executed on crypto_cn10k PMD(summary): ~# DPDK_TEST=pdcp_autotest ./dpdk-test --log-level=7 <snip> + ------------------------------------------------------- + + Test Suite Summary : PDCP Unit Test Suite + ------------------------------------------------------- + + Known vector cases : 112/160 passed, 0/160 skipped, 0/160 failed, 48/160 unsupported + PDCP combined mode : 1/1 passed, 0/1 skipped, 0/1 failed, 0/1 unsupported + PDCP HFN/SN : 4/4 passed, 0/4 skipped, 0/4 failed, 0/4 unsupported + PDCP reorder : 5/5 passed, 0/5 skipped, 0/5 failed, 0/5 unsupported + PDCP status report : 9/9 passed, 0/9 skipped, 0/9 failed, 0/9 unsupported + ------------------------------------------------------- + + Sub Testsuites Total : 5 + Sub Testsuites Skipped : 0 + Sub Testsuites Passed : 5 + Sub Testsuites Failed : 0 + ------------------------------------------------------- + + Tests Total : 179 + Tests Skipped : 0 + Tests Executed : 179 + Tests Unsupported: 48 + Tests Passed : 131 + Tests Failed : 0 + ------------------------------------------------------- + Test OK Anoob Joseph (10): lib: add pdcp protocol pdcp: add pre and post-process pdcp: add packet group pdcp: add crypto session create and destroy pdcp: add pre and post process for UL pdcp: add pre and post process for DL pdcp: add IV generation routines app/test: add lib pdcp tests doc: add PDCP library guide pdcp: add control PDU handling for status report Volodymyr Fialko (12): net: add PDCP header test/pdcp: pdcp HFN tests in combined mode pdcp: implement t-Reordering and packet buffering test/pdcp: add in-order delivery cases pdcp: add timer callback handlers pdcp: add timer expiry handle test/pdcp: add timer expiry cases test/pdcp: add timer restart case pdcp: add support for status report pdcp: allocate reorder buffer alongside with entity pdcp: add thread safe processing test/pdcp: add PDCP status report cases app/test/meson.build | 5 + app/test/test_cryptodev.h | 3 + app/test/test_pdcp.c | 1981 +++++++++++++++++ doc/api/doxy-api-index.md | 4 +- doc/api/doxy-api.conf.in | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 293 +++ lib/meson.build | 1 + lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++ lib/pdcp/meson.build | 23 + lib/pdcp/pdcp_cnt.c | 164 ++ lib/pdcp/pdcp_cnt.h | 24 + lib/pdcp/pdcp_crypto.c | 238 ++ lib/pdcp/pdcp_crypto.h | 20 + lib/pdcp/pdcp_ctrl_pdu.c | 72 + lib/pdcp/pdcp_ctrl_pdu.h | 16 + lib/pdcp/pdcp_entity.h | 324 +++ lib/pdcp/pdcp_process.c | 1301 +++++++++++ lib/pdcp/pdcp_process.h | 36 + lib/pdcp/pdcp_reorder.c | 20 + lib/pdcp/pdcp_reorder.h | 68 + lib/pdcp/rte_pdcp.c | 343 +++ lib/pdcp/rte_pdcp.h | 385 ++++ lib/pdcp/rte_pdcp_group.h | 131 ++ lib/pdcp/version.map | 20 + 27 files changed, 5622 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pdcp.c create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst create mode 100644 lib/net/rte_pdcp_hdr.h create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 01/22] net: add PDCP header 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph @ 2023-05-26 21:01 ` Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 02/22] lib: add pdcp protocol Anoob Joseph ` (22 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add PDCP protocol header to be used for supporting PDCP protocol processing. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- doc/api/doxy-api-index.md | 3 +- lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 150 insertions(+), 1 deletion(-) create mode 100644 lib/net/rte_pdcp_hdr.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index c709fd48ad..debbe4134f 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -127,7 +127,8 @@ The public API headers are grouped by topics: [Geneve](@ref rte_geneve.h), [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), - [PPP](@ref rte_ppp.h) + [PPP](@ref rte_ppp.h), + [PDCP hdr](@ref rte_pdcp_hdr.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/lib/net/meson.build b/lib/net/meson.build index 379d161ee0..bd56f91c22 100644 --- a/lib/net/meson.build +++ b/lib/net/meson.build @@ -22,6 +22,7 @@ headers = files( 'rte_geneve.h', 'rte_l2tpv2.h', 'rte_ppp.h', + 'rte_pdcp_hdr.h', ) sources = files( diff --git a/lib/net/rte_pdcp_hdr.h b/lib/net/rte_pdcp_hdr.h new file mode 100644 index 0000000000..72ae9a66cb --- /dev/null +++ b/lib/net/rte_pdcp_hdr.h @@ -0,0 +1,147 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_HDR_H +#define RTE_PDCP_HDR_H + +/** + * @file + * + * PDCP-related defines + * + * Based on - ETSI TS 138 323 V17.1.0 (2022-08) + * https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + */ + +#include <rte_byteorder.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * 4.3.1 + * + * Indicate the maximum supported size of a PDCP Control PDU. + */ +#define RTE_PDCP_CTRL_PDU_SIZE_MAX 9000u + +/** + * 6.3.4 MAC-I + * + * Indicate the size of MAC-I in PDCP PDU. + */ +#define RTE_PDCP_MAC_I_LEN 4 + +/** + * Indicate type of control information included in the corresponding PDCP + * Control PDU. + */ +enum rte_pdcp_ctrl_pdu_type { + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT = 0, + RTE_PDCP_CTRL_PDU_TYPE_ROHC_FEEDBACK = 1, + RTE_PDCP_CTRL_PDU_TYPE_EHC_FEEDBACK = 2, + RTE_PDCP_CRTL_PDU_TYPE_UDC_FEEDBACK = 3, +}; + +/** + * 6.3.7 D/C + * + * This field indicates whether the corresponding PDCP PDU is a + * PDCP Data PDU or a PDCP Control PDU. + */ +enum rte_pdcp_pdu_type { + RTE_PDCP_PDU_TYPE_CTRL = 0, + RTE_PDCP_PDU_TYPE_DATA = 1, +}; + +/** + * 6.2.2.1 Data PDU for SRBs + */ +__extension__ +struct rte_pdcp_cp_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 4; /**< Reserved */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.2.2 Data PDU for DRBs and MRBs with 12 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 3; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 3; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.2.3 Data PDU for DRBs and MRBs with 18 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_18_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ + uint8_t r : 5; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 5; /**< Reserved */ + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ +#endif + uint8_t sn_15_8; /**< Sequence number bits 8-15 */ + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.3.1 Control PDU for PDCP status report + */ +__extension__ +struct rte_pdcp_up_ctrl_pdu_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t r : 4; /**< Reserved */ +#endif + /** + * 6.3.9 FMC + * + * First Missing COUNT. This field indicates the COUNT value of the + * first missing PDCP SDU within the reordering window, i.e. RX_DELIV. + */ + rte_be32_t fmc; + /** + * 6.3.10 Bitmap + * + * Length: Variable. The length of the bitmap field can be 0. + * + * This field indicates which SDUs are missing and which SDUs are + * correctly received in the receiving PDCP entity. The bit position of + * Nth bit in the Bitmap is N, i.e., the bit position of the first bit + * in the Bitmap is 1. + */ + uint8_t bitmap[]; +} __rte_packed; + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_HDR_H */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 02/22] lib: add pdcp protocol 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 01/22] net: add PDCP header Anoob Joseph @ 2023-05-26 21:01 ` Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 03/22] pdcp: add pre and post-process Anoob Joseph ` (21 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + lib/meson.build | 1 + lib/pdcp/meson.build | 17 ++++ lib/pdcp/pdcp_crypto.c | 21 +++++ lib/pdcp/pdcp_crypto.h | 15 ++++ lib/pdcp/pdcp_entity.h | 113 ++++++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 138 +++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 13 +++ lib/pdcp/rte_pdcp.c | 141 ++++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 167 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 10 +++ 12 files changed, 639 insertions(+), 1 deletion(-) create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/version.map diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index debbe4134f..cd7a6cae44 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -128,7 +128,8 @@ The public API headers are grouped by topics: [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), [PPP](@ref rte_ppp.h), - [PDCP hdr](@ref rte_pdcp_hdr.h) + [PDCP hdr](@ref rte_pdcp_hdr.h), + [PDCP](@ref rte_pdcp.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index d230a19e1f..58789308a9 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -62,6 +62,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/net \ @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ + @TOPDIR@/lib/pdcp \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ @TOPDIR@/lib/port \ diff --git a/lib/meson.build b/lib/meson.build index dc8aa4ac84..a6a54c196c 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -64,6 +64,7 @@ libraries = [ 'flow_classify', # flow_classify lib depends on pkt framework table lib 'graph', 'node', + 'pdcp', # pdcp lib depends on crypto and security ] optional_libs = [ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode 100644 index 0000000000..ccaf426240 --- /dev/null +++ b/lib/pdcp/meson.build @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +sources = files( + 'pdcp_crypto.c', + 'pdcp_process.c', + 'rte_pdcp.c', + ) +headers = files('rte_pdcp.h') + +deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c new file mode 100644 index 0000000000..755e27ec9e --- /dev/null +++ b/lib/pdcp/pdcp_crypto.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_crypto.h" + +int +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + RTE_SET_USED(entity); + RTE_SET_USED(conf); + return 0; +} + +void +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) +{ + RTE_SET_USED(entity); +} diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h new file mode 100644 index 0000000000..6563331d37 --- /dev/null +++ b/lib/pdcp/pdcp_crypto.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CRYPTO_H +#define PDCP_CRYPTO_H + +#include <rte_pdcp.h> + +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, + const struct rte_pdcp_entity_conf *conf); + +void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); + +#endif /* PDCP_CRYPTO_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h new file mode 100644 index 0000000000..000297588f --- /dev/null +++ b/lib/pdcp/pdcp_entity.h @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_ENTITY_H +#define PDCP_ENTITY_H + +#include <rte_common.h> +#include <rte_crypto_sym.h> +#include <rte_mempool.h> +#include <rte_pdcp.h> +#include <rte_security.h> + +struct entity_priv; + +/* IV generation function based on the entity configuration */ +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count); + +struct entity_state { + uint32_t rx_next; + uint32_t tx_next; + uint32_t rx_deliv; + uint32_t rx_reord; +}; + +/* + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + */ + +struct entity_priv { + /** Crypto sym session. */ + struct rte_cryptodev_sym_session *crypto_sess; + /** Entity specific IV generation function. */ + iv_gen_t iv_gen; + /** Entity state variables. */ + struct entity_state state; + /** Flags. */ + struct { + /** PDCP PDU has 4 byte MAC-I. */ + uint64_t is_authenticated : 1; + /** Cipher offset & length in bits. */ + uint64_t is_cipher_in_bits : 1; + /** Auth offset & length in bits. */ + uint64_t is_auth_in_bits : 1; + /** Is UL/transmitting PDCP entity. */ + uint64_t is_ul_entity : 1; + /** Is NULL auth. */ + uint64_t is_null_auth : 1; + } flags; + /** Crypto op pool. */ + struct rte_mempool *cop_pool; + /** PDCP header size. */ + uint8_t hdr_sz; + /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ + uint8_t aad_sz; + /** Device ID of the device to be used for offload. */ + uint8_t dev_id; +}; + +struct entity_priv_dl_part { + /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ + uint8_t dummy; +}; + +struct entity_priv_ul_part { + /* + * NOTE: when re-establish is supported, plain PDCP packets & COUNT values need to be + * cached. + */ + uint8_t dummy; +}; + +static inline struct entity_priv * +entity_priv_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); +} + +static inline struct entity_priv_dl_part * +entity_dl_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline struct entity_priv_ul_part * +entity_ul_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) +{ + return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); +} + +#endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c new file mode 100644 index 0000000000..79f5dce5db --- /dev/null +++ b/lib/pdcp/pdcp_process.c @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_memcpy.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, + struct rte_crypto_sym_xform **a_xfrm) +{ + *c_xfrm = NULL; + *a_xfrm = NULL; + + if (conf->crypto_xfrm == NULL) + return -EINVAL; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + *c_xfrm = conf->crypto_xfrm; + *a_xfrm = conf->crypto_xfrm->next; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + *a_xfrm = conf->crypto_xfrm; + *c_xfrm = conf->crypto_xfrm->next; + } else { + return -EINVAL; + } + + return 0; +} + +static int +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + int ret; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + /** + * flags.is_authenticated + * + * MAC-I would be added in case of control plane packets and when authentication + * transform is not NULL. + */ + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && (a_xfrm == NULL)) + return -EINVAL; + + if (a_xfrm != NULL) + en_priv->flags.is_authenticated = 1; + + /** + * flags.is_cipher_in_bits + * + * For ZUC & SNOW3G cipher algos, offset & length need to be provided in bits. + */ + + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) + en_priv->flags.is_cipher_in_bits = 1; + + /** + * flags.is_auth_in_bits + * + * For ZUC & SNOW3G authentication algos, offset & length need to be provided in bits. + */ + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) + en_priv->flags.is_auth_in_bits = 1; + } + + /** + * flags.is_ul_entity + * + * Indicate whether the entity is UL/transmitting PDCP entity. + */ + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + en_priv->flags.is_ul_entity = 1; + + /** + * flags.is_null_auth + * + * For NULL auth, 4B zeros need to be added by lib PDCP. Indicate that + * algo is NULL auth to perform the same. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + en_priv->flags.is_null_auth = 1; + + /** + * hdr_sz + * + * PDCP header size of the entity + */ + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); + + /** + * aad_sz + * + * For AES-CMAC, additional message is prepended for processing. Need to be trimmed after + * crypto processing is done. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) + en_priv->aad_sz = 8; + else + en_priv->aad_sz = 0; + + return 0; +} + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv *en_priv; + int ret; + + if (entity == NULL || conf == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + ret = pdcp_entity_priv_populate(en_priv, conf); + if (ret) + return ret; + + return 0; +} diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h new file mode 100644 index 0000000000..fd53fff0aa --- /dev/null +++ b/lib/pdcp/pdcp_process.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_PROCESS_H +#define PDCP_PROCESS_H + +#include <rte_pdcp.h> + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode 100644 index 0000000000..adcad5dd25 --- /dev/null +++ b/lib/pdcp/rte_pdcp.c @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_pdcp.h> +#include <rte_malloc.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +{ + int size; + + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + size += sizeof(struct entity_priv_dl_part); + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size += sizeof(struct entity_priv_ul_part); + else + return -EINVAL; + + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); +} + +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) +{ + struct rte_pdcp_entity *entity = NULL; + struct entity_priv *en_priv; + int ret, entity_size; + uint32_t count; + + if (conf == NULL || conf->cop_pool == NULL) { + rte_errno = EINVAL; + return NULL; + } + + if (conf->pdcp_xfrm.en_ordering || conf->pdcp_xfrm.remove_duplicates || conf->is_slrb || + conf->en_sec_offload) { + rte_errno = ENOTSUP; + return NULL; + } + + /* + * 6.3.2 PDCP SN + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the PDCP SN is + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl-PDCP-SN-Size in + * TS 38.331 [3]) + */ + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { + rte_errno = ENOTSUP; + return NULL; + } + + if (conf->pdcp_xfrm.hfn_threshold) { + rte_errno = EINVAL; + return NULL; + } + + entity_size = pdcp_entity_size_get(conf); + if (entity_size < 0) { + rte_errno = EINVAL; + return NULL; + } + + entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (entity == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + en_priv = entity_priv_get(entity); + + count = pdcp_count_from_hfn_sn_get(conf->pdcp_xfrm.hfn, conf->sn, conf->pdcp_xfrm.sn_size); + + en_priv->state.rx_deliv = count; + en_priv->state.tx_next = count; + en_priv->cop_pool = conf->cop_pool; + + /* Setup crypto session */ + ret = pdcp_crypto_sess_create(entity, conf); + if (ret) + goto entity_free; + + ret = pdcp_process_func_set(entity, conf); + if (ret) + goto crypto_sess_destroy; + + return entity; + +crypto_sess_destroy: + pdcp_crypto_sess_destroy(entity); +entity_free: + rte_free(entity); + rte_errno = -ret; + return NULL; +} + +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) +{ + if (pdcp_entity == NULL) + return -EINVAL; + + /* Teardown crypto sessions */ + pdcp_crypto_sess_destroy(pdcp_entity); + + rte_free(pdcp_entity); + + RTE_SET_USED(out_mb); + return 0; +} + +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]) +{ + struct entity_priv *en_priv; + + if (pdcp_entity == NULL) + return -EINVAL; + + en_priv = entity_priv_get(pdcp_entity); + + if (en_priv->flags.is_ul_entity) { + en_priv->state.tx_next = 0; + } else { + en_priv->state.rx_next = 0; + en_priv->state.rx_deliv = 0; + } + + RTE_SET_USED(out_mb); + + return 0; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode 100644 index 0000000000..1f96fdc9a1 --- /dev/null +++ b/lib/pdcp/rte_pdcp.h @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_H +#define RTE_PDCP_H + +/** + * @file rte_pdcp.h + * + * RTE PDCP support. + * + * A framework for PDCP protocol processing. + */ + +#include <rte_compat.h> +#include <rte_common.h> +#include <rte_mempool.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * PDCP entity. + * + * 4.2.2 PDCP entities + * + * The PDCP entities are located in the PDCP sublayer. Several PDCP entities may + * be defined for a UE. Each PDCP entity is carrying the data of one radio + * bearer. A PDCP entity is associated either to the control plane or the user + * plane depending on which radio bearer it is carrying data for. + */ +struct rte_pdcp_entity { + /** + * PDCP entities may hold packets for purposes of in-order delivery (in + * case of receiving PDCP entity) and re-transmission (in case of + * transmitting PDCP entity). + * + * The field 'max_pkt_cache' would be used to indicate the maximum + * number of packets that may be cached in an entity at any point of + * time. When application provides buffers to receive packets from + * PDCP entity, the size of the buffer should be such that it can + * hold additionally 'max_pkt_cache' number of packets. + */ + uint32_t max_pkt_cache; +} __rte_cache_aligned; + +/** + * PDCP entity configuration to be used for establishing an entity. + */ +/* Structure rte_pdcp_entity_conf 8< */ +struct rte_pdcp_entity_conf { + /** PDCP transform for the entity. */ + struct rte_security_pdcp_xform pdcp_xfrm; + /** Crypto transform applicable for the entity. */ + struct rte_crypto_sym_xform *crypto_xfrm; + /** Mempool for crypto symmetric session. */ + struct rte_mempool *sess_mpool; + /** Crypto op pool.*/ + struct rte_mempool *cop_pool; + /** + * SN value to be used. 32 bit count value to be used for the first + * packet would be derived based on HFN (`rte_security_pdcp_xform.hfn`) + * and SN. + */ + uint32_t sn; + /** + * Indicate whether the PDCP entity belongs to Side Link Radio Bearer. + */ + bool is_slrb; + /** Enable security offload on the device specified. */ + bool en_sec_offload; + /** Device on which security/crypto session need to be created. */ + uint8_t dev_id; + /** + * Reverse direction during IV generation. Can be used to simulate UE + * crypto processing. + */ + bool reverse_iv_direction; +}; +/* >8 End of structure rte_pdcp_entity_conf. */ + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.1 PDCP entity establishment + * + * Establish PDCP entity based on provided input configuration. + * + * @param conf + * Parameters to be used for initializing PDCP entity object. + * @return + * - Valid handle if success + * - NULL in case of failure. rte_errno will be set to error code + */ +__rte_experimental +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.3 PDCP entity release + * + * Release PDCP entity. + * + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * Entity release would result in freeing all memory associated with the PDCP + * entity as well as any crypto/security sessions created. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be released. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.4 PDCP entity suspend + * + * Suspend PDCP entity. + * + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be suspended. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode 100644 index 0000000000..923e165f3f --- /dev/null +++ b/lib/pdcp/version.map @@ -0,0 +1,10 @@ +EXPERIMENTAL { + global: + + # added in 23.07 + rte_pdcp_entity_establish; + rte_pdcp_entity_release; + rte_pdcp_entity_suspend; + + local: *; +}; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 03/22] pdcp: add pre and post-process 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 01/22] net: add PDCP header Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 02/22] lib: add pdcp protocol Anoob Joseph @ 2023-05-26 21:01 ` Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 04/22] pdcp: add packet group Anoob Joseph ` (20 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Functionality of pre-process & post-process varies based on the type of entity. Registration of entity specific function pointer allows skipping multiple checks that would come in datapath otherwise. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/rte_pdcp.h | 97 ++++++++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 ++ 2 files changed, 100 insertions(+) diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 1f96fdc9a1..46c3c2a416 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -22,6 +22,21 @@ extern "C" { #endif +/* Forward declarations */ +struct rte_pdcp_entity; + +/* PDCP pre-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err); + +/* PDCP post-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err); + /** * PDCP entity. * @@ -33,6 +48,10 @@ extern "C" { * plane depending on which radio bearer it is carrying data for. */ struct rte_pdcp_entity { + /** Entity specific pre-process handle. */ + rte_pdcp_pre_p_t pre_process; + /** Entity specific post-process handle. */ + rte_pdcp_post_p_t post_process; /** * PDCP entities may hold packets for purposes of in-order delivery (in * case of receiving PDCP entity) and re-transmission (in case of @@ -160,6 +179,84 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity pre-process the mbufs and prepare + * crypto ops that can be enqueued to the cryptodev associated with given + * session. Only error packets would be moved returned in the input buffer, + * *mb*, and it is the responsibility of the application to free the same. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param[in, out] mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * which contain the input packets. Any error packets would be returned in the + * same buffer. + * @param[out] cop + * The address of an array that can hold up to *num* pointers to + * *rte_crypto_op* structures. Crypto ops would be allocated by + * ``rte_pdcp_pkt_pre_process`` API. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * Pointer to return the number of error packets returned in *mb* + * @return + * Count of crypto_ops prepared + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err) +{ + return entity->pre_process(entity, mb, cop, num, nb_err); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity, perform PDCP post-processing of the + * mbufs. + * + * Input mbufs are the ones retrieved from crypto_ops dequeued from cryptodev + * and grouped by *rte_pdcp_pkt_crypto_group()*. + * + * The post-processed packets would be returned in the *out_mb* buffer. + * The resultant mbufs would be grouped into success packets and error packets. + * Error packets would be grouped in the end of the array and it is the + * responsibility of the application to handle the same. + * + * When in-order delivery is enabled, PDCP entity may buffer packets and would + * deliver packets only when all prior packets have been post-processed. That + * would result in returning more/less packets than enqueued. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param in_mb + * The address of an array of *num* pointers to *rte_mbuf* structures. + * @param[out] out_mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * to output packets after PDCP post-processing. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * The number of error packets returned in *out_mb* buffer. + * @return + * Count of packets returned in *out_mb* buffer. + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return entity->post_process(entity, in_mb, out_mb, num, nb_err); +} + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 923e165f3f..f9ff30600a 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -6,5 +6,8 @@ EXPERIMENTAL { rte_pdcp_entity_release; rte_pdcp_entity_suspend; + rte_pdcp_pkt_post_process; + rte_pdcp_pkt_pre_process; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 04/22] pdcp: add packet group 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (2 preceding siblings ...) 2023-05-26 21:01 ` [PATCH v4 03/22] pdcp: add pre and post-process Anoob Joseph @ 2023-05-26 21:01 ` Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 05/22] pdcp: add crypto session create and destroy Anoob Joseph ` (19 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Crypto processing in PDCP is performed asynchronously by rte_cryptodev_enqueue_burst() and rte_cryptodev_dequeue_burst(). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 1 + lib/pdcp/rte_pdcp.h | 6 ++ lib/pdcp/rte_pdcp_group.h | 131 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 + 4 files changed, 141 insertions(+) create mode 100644 lib/pdcp/rte_pdcp_group.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index ccaf426240..08679b743a 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -13,5 +13,6 @@ sources = files( 'rte_pdcp.c', ) headers = files('rte_pdcp.h') +indirect_headers += files('rte_pdcp_group.h') deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 46c3c2a416..b88cad4f64 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -257,6 +257,12 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. So include + * in the end. + */ +#include <rte_pdcp_group.h> + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new file mode 100644 index 0000000000..ece3e8c0ff --- /dev/null +++ b/lib/pdcp/rte_pdcp_group.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_GROUP_H +#define RTE_PDCP_GROUP_H + +/** + * @file rte_pdcp_group.h + * + * RTE PDCP grouping support. + * It is not recommended to include this file directly, include <rte_pdcp.h> + * instead. + * Provides helper functions to process completed crypto-ops and group related + * packets by sessions they belong to. + */ + +#include <rte_common.h> +#include <rte_crypto.h> +#include <rte_cryptodev.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Group packets belonging to same PDCP entity. + */ +struct rte_pdcp_group { + union { + uint64_t val; + void *ptr; + } id; /**< Grouped by value */ + struct rte_mbuf **m; /**< Start of the group */ + uint32_t cnt; /**< Number of entries in the group */ + int32_t rc; /**< Status code associated with the group */ +}; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take crypto-op as an input and extract pointer to related PDCP entity. + * @param cop + * The address of an input *rte_crypto_op* structure. + * @return + * The pointer to the related *rte_pdcp_entity* structure. + */ +static inline struct rte_pdcp_entity * +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) +{ + void *sess = cop->sym[0].session; + + return (struct rte_pdcp_entity *)(uintptr_t) + rte_cryptodev_sym_session_opaque_data_get(sess); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take as input completed crypto ops, extract related mbufs and group them by + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* flag + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. + * + * Note that application must ensure only crypto-ops prepared by lib_pdcp is + * provided back to @see rte_pdcp_pkt_crypto_group(). + * + * @param cop + * The address of an array of *num* pointers to the input *rte_crypto_op* + * structures. + * @param[out] mb + * The address of an array of *num* pointers to output *rte_mbuf* structures. + * @param[out] grp + * The address of an array of *num* to output *rte_pdcp_group* structures. + * @param num + * The maximum number of crypto-ops to process. + * @return + * Number of filled elements in *grp* array. + * + */ +static inline uint16_t +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_pdcp_group grp[], uint16_t num) +{ + uint32_t i, j = 0, n = 0; + void *ns, *ps = NULL; + struct rte_mbuf *m; + + for (i = 0; i != num; i++) { + m = cop[i]->sym[0].m_src; + ns = cop[i]->sym[0].session; + + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; + + /* Different entity */ + if (ps != ns) { + + /* Finalize open group and start a new one */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + /* Start new group */ + grp[n].m = mb + j; + ps = ns; + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); + } + + mb[j++] = m; + rte_crypto_op_free(cop[i]); + } + + /* Finalize last group */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + return n; +} + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_GROUP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index f9ff30600a..d564f155e0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,12 +2,15 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_en_from_cop; + rte_pdcp_entity_establish; rte_pdcp_entity_release; rte_pdcp_entity_suspend; rte_pdcp_pkt_post_process; rte_pdcp_pkt_pre_process; + rte_pdcp_pkt_crypto_group; local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 05/22] pdcp: add crypto session create and destroy 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (3 preceding siblings ...) 2023-05-26 21:01 ` [PATCH v4 04/22] pdcp: add packet group Anoob Joseph @ 2023-05-26 21:01 ` Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 06/22] pdcp: add pre and post process for UL Anoob Joseph ` (18 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to create & destroy sessions. PDCP lib would take crypto transforms as input and creates the session on the corresponding device after verifying capabilities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_crypto.c | 223 ++++++++++++++++++++++++++++++++++++++++- lib/pdcp/pdcp_crypto.h | 5 + 2 files changed, 225 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c index 755e27ec9e..6d2a85dc7d 100644 --- a/lib/pdcp/pdcp_crypto.c +++ b/lib/pdcp/pdcp_crypto.c @@ -2,20 +2,237 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_errno.h> #include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> #include "pdcp_crypto.h" +#include "pdcp_entity.h" + +static int +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, + const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm, + bool is_auth_then_cipher) +{ + uint16_t cipher_iv_len, auth_digest_len, auth_iv_len; + int ret; + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protection is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (c_xfrm == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + + /* With UPLINK, if auth is enabled, it should be before cipher */ + if (a_xfrm != NULL && !is_auth_then_cipher) + return -EINVAL; + + /* With UPLINK, cipher operation must be encrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return -EINVAL; + + /* With UPLINK, auth operation (if present) must be generate */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_GENERATE) + return -EINVAL; + + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + + /* With DOWNLINK, if auth is enabled, it should be after cipher */ + if (a_xfrm != NULL && is_auth_then_cipher) + return -EINVAL; + + /* With DOWNLINK, cipher operation must be decrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) + return -EINVAL; + + /* With DOWNLINK, auth operation (if present) must be verify */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_VERIFY) + return -EINVAL; + + } else { + return -EINVAL; + } + + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) + return -EINVAL; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + cipher_iv_len = 0; + else + cipher_iv_len = PDCP_IV_LEN; + + if (cipher_iv_len != c_xfrm->cipher.iv.length) + return -EINVAL; + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + return -EINVAL; + + /* For AUTH NULL, lib PDCP would add 4 byte 0s */ + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + auth_digest_len = 0; + else + auth_digest_len = RTE_PDCP_MAC_I_LEN; + + if (auth_digest_len != a_xfrm->auth.digest_length) + return -EINVAL; + + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + auth_iv_len = PDCP_IV_LEN; + else + auth_iv_len = 0; + + if (a_xfrm->auth.iv.length != auth_iv_len) + return -EINVAL; + } + + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) + return -EINVAL; + + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); + if (ret) + return -ENOTSUP; + + if (a_xfrm != NULL) { + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); + if (ret) + return -ENOTSUP; + } + + return 0; +} int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { - RTE_SET_USED(entity); - RTE_SET_USED(conf); + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + struct entity_priv *en_priv; + bool is_auth_then_cipher; + int ret; + + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + en_priv->dev_id = conf->dev_id; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + c_xfrm = conf->crypto_xfrm; + a_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = false; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + a_xfrm = conf->crypto_xfrm; + c_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = true; + } else { + return -EINVAL; + } + + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, is_auth_then_cipher); + if (ret) + return ret; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + c_xfrm->cipher.iv.offset = 0; + else + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; + + if (a_xfrm != NULL) { + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm->auth.iv.offset = 0; + else + if (c_xfrm->cipher.iv.offset) + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + PDCP_IV_LEN; + else + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; + } + + if (conf->sess_mpool == NULL) + return -EINVAL; + + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf->dev_id, conf->crypto_xfrm, + conf->sess_mpool); + if (en_priv->crypto_sess == NULL) { + /* rte_errno is set as positive values of error codes */ + return -rte_errno; + } + + rte_cryptodev_sym_session_opaque_data_set(en_priv->crypto_sess, (uint64_t)entity); + return 0; } void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) { - RTE_SET_USED(entity); + struct entity_priv *en_priv; + + en_priv = entity_priv_get(entity); + + if (en_priv->crypto_sess != NULL) { + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv->crypto_sess); + en_priv->crypto_sess = NULL; + } } diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h index 6563331d37..f694818713 100644 --- a/lib/pdcp/pdcp_crypto.h +++ b/lib/pdcp/pdcp_crypto.h @@ -5,8 +5,13 @@ #ifndef PDCP_CRYPTO_H #define PDCP_CRYPTO_H +#include <rte_crypto.h> +#include <rte_crypto_sym.h> #include <rte_pdcp.h> +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op)) +#define PDCP_IV_LEN 16 + int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 06/22] pdcp: add pre and post process for UL 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (4 preceding siblings ...) 2023-05-26 21:01 ` [PATCH v4 05/22] pdcp: add crypto session create and destroy Anoob Joseph @ 2023-05-26 21:01 ` Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 07/22] pdcp: add pre and post process for DL Anoob Joseph ` (17 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to perform pre & post processing based on the type of entity. To avoid checks in datapath, there are different function pointers registered based on the following, 1. Control plane v/s user plane 2. 12 bit v/s 18 bit SN For control plane only 12 bit SN need to be supported (as per PDCP specification). Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_entity.h | 24 +++ lib/pdcp/pdcp_process.c | 334 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 358 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 000297588f..23628ebad4 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -92,22 +92,46 @@ pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static inline uint32_t +pdcp_window_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return 1 << (sn_size - 1); +} + static inline uint32_t pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return (1 << sn_size) - 1; } +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + static inline uint32_t pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return ~pdcp_sn_mask_get(sn_size); } +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline uint32_t pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) { return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); } +static inline uint32_t +pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << (32 - sn_size)) - 1; +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 79f5dce5db..9b7de39db6 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -36,6 +36,336 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static inline void +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, + uint8_t data_offset, uint32_t count, const bool is_auth) +{ + const struct rte_crypto_op cop_init = { + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, + }; + struct rte_crypto_sym_op *op; + uint32_t pkt_len; + + const uint8_t cipher_shift = 3 * en_priv->flags.is_cipher_in_bits; + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; + + op = cop->sym; + cop->raw = cop_init.raw; + op->m_src = mb; + op->m_dst = mb; + + /* Set IV */ + en_priv->iv_gen(cop, en_priv, count); + + /* Prepare op */ + pkt_len = rte_pktmbuf_pkt_len(mb); + op->cipher.data.offset = data_offset << cipher_shift; + op->cipher.data.length = (pkt_len - data_offset) << cipher_shift; + + if (is_auth) { + op->auth.data.offset = 0; + op->auth.data.length = (pkt_len - RTE_PDCP_MAC_I_LEN) << auth_shift; + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t *, + (pkt_len - RTE_PDCP_MAC_I_LEN)); + } + + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); +} + +static inline bool +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + return true; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline bool +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + return true; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count, sn; + uint8_t *mac_i; + int i; + + const uint8_t hdr_sz = en_priv->hdr_sz; + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + /* Update sequence number in the PDU header */ + count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static int +pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + entity->pre_process = NULL; + entity->post_process = NULL; + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + entity->post_process = pdcp_post_process_ul; + } + + if (entity->pre_process == NULL || entity->post_process == NULL) + return -ENOTSUP; + + return 0; +} + static int pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) { @@ -134,5 +464,9 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti if (ret) return ret; + ret = pdcp_pre_post_func_set(entity, conf); + if (ret) + return ret; + return 0; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 07/22] pdcp: add pre and post process for DL 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (5 preceding siblings ...) 2023-05-26 21:01 ` [PATCH v4 06/22] pdcp: add pre and post process for UL Anoob Joseph @ 2023-05-26 21:01 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 08/22] pdcp: add IV generation routines Anoob Joseph ` (16 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to perform pre & post processing for down link entities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 383 ++++++++++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 10 ++ lib/pdcp/rte_pdcp.c | 23 +++ 4 files changed, 418 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 23628ebad4..1d4a43a3bc 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -13,6 +13,8 @@ struct entity_priv; +#define PDCP_HFN_MIN 0 + /* IV generation function based on the entity configuration */ typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count); diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 9b7de39db6..0737a60df2 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -333,9 +333,353 @@ pdcp_post_process_ul(const struct rte_pdcp_entity *entity, return nb_success; } +static inline int +pdcp_sn_count_get(const uint32_t rx_deliv, int32_t rsn, uint32_t *count, + const enum rte_security_pdcp_sn_size sn_size) +{ + const uint32_t rx_deliv_sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + const uint32_t window_sz = pdcp_window_size_get(sn_size); + uint32_t rhfn; + + rhfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + + if (rsn < (int32_t)(rx_deliv_sn - window_sz)) { + if (unlikely(rhfn == pdcp_hfn_max(sn_size))) + return -ERANGE; + rhfn += 1; + } else if ((uint32_t)rsn >= (rx_deliv_sn + window_sz)) { + if (unlikely(rhfn == PDCP_HFN_MIN)) + return -ERANGE; + rhfn -= 1; + } + + *count = pdcp_count_from_hfn_sn_get(rhfn, rsn, sn_size); + + return 0; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_18))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline void +pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool trim_mac) +{ + char *p = rte_pktmbuf_adj(mb, hdr_trim_sz); + RTE_ASSERT(p != NULL); + RTE_SET_USED(p); + + if (trim_mac) { + int ret = rte_pktmbuf_trim(mb, RTE_PDCP_MAC_I_LEN); + RTE_ASSERT(ret == 0); + RTE_SET_USED(ret); + } +} + +static inline bool +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, + const uint32_t count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + + if (count < en_priv->state.rx_deliv) + return false; + + /* t-Reordering timer is not supported - SDU will be delivered immediately. + * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not + * been delivered to upper layers + */ + en_priv->state.rx_next = count + 1; + + if (count >= en_priv->state.rx_next) + en_priv->state.rx_next = count + 1; + + return true; +} + +static inline uint16_t +pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + mb_dynfield = pdcp_dynfield(mb); + count = *mb_dynfield; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static uint16_t +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + mb_dynfield = pdcp_dynfield(mb); + count = *mb_dynfield; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, true); + + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + static int pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { + struct entity_priv *en_priv = entity_priv_get(entity); + entity->pre_process = NULL; entity->post_process = NULL; @@ -346,6 +690,13 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; + entity->post_process = pdcp_post_process_cplane_sn_12_dl; + } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { @@ -360,6 +711,38 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; + entity->post_process = pdcp_post_process_uplane_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; + entity->post_process = pdcp_post_process_uplane_dl; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; + entity->post_process = pdcp_post_process_uplane_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; + entity->post_process = pdcp_post_process_uplane_dl; + } + if (entity->pre_process == NULL || entity->post_process == NULL) return -ENOTSUP; diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index fd53fff0aa..c270f15caf 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -7,6 +7,16 @@ #include <rte_pdcp.h> +typedef uint32_t rte_pdcp_dynfield_t; + +extern int rte_pdcp_dynfield_offset; + +static inline rte_pdcp_dynfield_t * +pdcp_dynfield(struct rte_mbuf *mbuf) +{ + return RTE_MBUF_DYNFIELD(mbuf, rte_pdcp_dynfield_offset, rte_pdcp_dynfield_t *); +} + int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index adcad5dd25..91dab91f73 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -10,6 +10,26 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +#define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" + +int rte_pdcp_dynfield_offset = -1; + +static int +pdcp_dynfield_register(void) +{ + const struct rte_mbuf_dynfield dynfield_desc = { + .name = RTE_PDCP_DYNFIELD_NAME, + .size = sizeof(rte_pdcp_dynfield_t), + .align = __alignof__(rte_pdcp_dynfield_t), + }; + + if (rte_pdcp_dynfield_offset != -1) + return rte_pdcp_dynfield_offset; + + rte_pdcp_dynfield_offset = rte_mbuf_dynfield_register(&dynfield_desc); + return rte_pdcp_dynfield_offset; +} + static int pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { @@ -35,6 +55,9 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) int ret, entity_size; uint32_t count; + if (pdcp_dynfield_register() < 0) + return NULL; + if (conf == NULL || conf->cop_pool == NULL) { rte_errno = EINVAL; return NULL; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 08/22] pdcp: add IV generation routines 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (6 preceding siblings ...) 2023-05-26 21:01 ` [PATCH v4 07/22] pdcp: add pre and post process for DL Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 09/22] app/test: add lib pdcp tests Anoob Joseph ` (15 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger For PDCP, IV generated has varying formats depending on the ciphering and authentication algorithm used. Add routines to populate IV accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 87 ++++++++++++ lib/pdcp/pdcp_process.c | 284 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 371 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 1d4a43a3bc..10a72faae1 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -26,6 +26,89 @@ struct entity_state { uint32_t rx_reord; }; +union auth_iv_partial { + /* For AES-CMAC, there is no IV, but message gets prepended */ + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + } aes_cmac; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_37_39 : 3; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_65_71 : 7; + uint64_t direction_64 : 1; + uint64_t rsvd_72_111 : 40; + uint64_t rsvd_113_119 : 7; + uint64_t direction_112 : 1; + uint64_t rsvd_120_127 : 8; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t zero_37_39 : 3; + uint64_t zero_40_63 : 24; + + uint64_t direction_64 : 1; + uint64_t rsvd_65_71 : 7; + uint64_t rsvd_72_111 : 40; + uint64_t direction_112 : 1; + uint64_t rsvd_113_119 : 7; + uint64_t rsvd_120_127 : 8; +#endif + } zs; + uint64_t u64[2]; +}; + +union cipher_iv_partial { + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t zero_64_127; + } aes_ctr; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t rsvd_64_127; + } zs; + uint64_t u64[2]; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -35,6 +118,10 @@ struct entity_priv { struct rte_cryptodev_sym_session *crypto_sess; /** Entity specific IV generation function. */ iv_gen_t iv_gen; + /** Pre-prepared auth IV. */ + union auth_iv_partial auth_iv_part; + /** Pre-prepared cipher IV. */ + union cipher_iv_partial cipher_iv_part; /** Entity state variables. */ struct entity_state state; /** Flags. */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 0737a60df2..8c802e735b 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -13,6 +13,181 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +/* Enum of supported algorithms for ciphering */ +enum pdcp_cipher_algo { + PDCP_CIPHER_ALGO_NULL, + PDCP_CIPHER_ALGO_AES, + PDCP_CIPHER_ALGO_ZUC, + PDCP_CIPHER_ALGO_SNOW3G, + PDCP_CIPHER_ALGO_MAX +}; + +/* Enum of supported algorithms for integrity */ +enum pdcp_auth_algo { + PDCP_AUTH_ALGO_NULL, + PDCP_AUTH_ALGO_AES, + PDCP_AUTH_ALGO_ZUC, + PDCP_AUTH_ALGO_SNOW3G, + PDCP_AUTH_ALGO_MAX +}; + +/* IV generation functions based on type of operation (cipher - auth) */ + +static void +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + /* No IV required for NULL cipher + NULL auth */ + RTE_SET_USED(cop); + RTE_SET_USED(en_priv); + RTE_SET_USED(count); +} + +static void +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr; + uint64_t m; + + /* AES-CMAC requires message to be prepended with info on count etc */ + + /* Prepend by 8 bytes to add custom message */ + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, 16); +} + +static void +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64 = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64, 8); + rte_memcpy(iv + 8, &iv_u64, 8); +} + +static void +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + /* Generating cipher IV */ + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + iv += PDCP_IV_LEN; + + /* Generating auth IV */ + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + iv += PDCP_IV_LEN; + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + static int pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, struct rte_crypto_sym_xform **a_xfrm) @@ -36,6 +211,111 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static int +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + enum rte_security_pdcp_direction direction; + enum pdcp_cipher_algo cipher_algo; + enum pdcp_auth_algo auth_algo; + struct entity_priv *en_priv; + int ret; + + en_priv = entity_priv_get(entity); + + direction = conf->pdcp_xfrm.pkt_dir; + if (conf->reverse_iv_direction) + direction = !direction; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (c_xfrm == NULL) + return -EINVAL; + + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); + + switch (c_xfrm->cipher.algo) { + case RTE_CRYPTO_CIPHER_NULL: + cipher_algo = PDCP_CIPHER_ALGO_NULL; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + cipher_algo = PDCP_CIPHER_ALGO_AES; + en_priv->cipher_iv_part.aes_ctr.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.aes_ctr.direction = direction; + break; + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + cipher_algo = PDCP_CIPHER_ALGO_SNOW3G; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + cipher_algo = PDCP_CIPHER_ALGO_ZUC; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + default: + return -ENOTSUP; + } + + if (a_xfrm != NULL) { + switch (a_xfrm->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + auth_algo = PDCP_AUTH_ALGO_NULL; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + auth_algo = PDCP_AUTH_ALGO_AES; + en_priv->auth_iv_part.aes_cmac.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.aes_cmac.direction = direction; + break; + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + auth_algo = PDCP_AUTH_ALGO_SNOW3G; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + case RTE_CRYPTO_AUTH_ZUC_EIA3: + auth_algo = PDCP_AUTH_ALGO_ZUC; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + default: + return -ENOTSUP; + } + } else { + auth_algo = PDCP_AUTH_ALGO_NULL; + } + + static const iv_gen_t iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_null_null, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_null_aes_cmac, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_null_zs, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_null_zs, + + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_aes_ctr_null, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_aes_ctr_aes_cmac, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_aes_ctr_zs, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_aes_ctr_zs, + + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + }; + + en_priv->iv_gen = iv_gen_map[cipher_algo][auth_algo]; + + return 0; +} + static inline void cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, uint8_t data_offset, uint32_t count, const bool is_auth) @@ -843,6 +1123,10 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti en_priv = entity_priv_get(entity); + ret = pdcp_iv_gen_func_set(entity, conf); + if (ret) + return ret; + ret = pdcp_entity_priv_populate(en_priv, conf); if (ret) return ret; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 09/22] app/test: add lib pdcp tests 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (7 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 08/22] pdcp: add IV generation routines Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph ` (14 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add tests to verify lib PDCP operations. Tests leverage existing PDCP test vectors. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/meson.build | 5 + app/test/test_cryptodev.h | 3 + app/test/test_pdcp.c | 732 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 740 insertions(+) create mode 100644 app/test/test_pdcp.c diff --git a/app/test/meson.build b/app/test/meson.build index b9b5432496..2894793347 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -435,6 +435,11 @@ if dpdk_conf.has('RTE_HAS_LIBPCAP') endif endif +if dpdk_conf.has('RTE_LIB_PDCP') + test_sources += 'test_pdcp.c' + fast_tests += [['pdcp_autotest', false, true]] +endif + if cc.has_argument('-Wno-format-truncation') cflags += '-Wno-format-truncation' endif diff --git a/app/test/test_cryptodev.h b/app/test/test_cryptodev.h index abd795f54a..89057dba22 100644 --- a/app/test/test_cryptodev.h +++ b/app/test/test_cryptodev.h @@ -4,6 +4,9 @@ #ifndef TEST_CRYPTODEV_H_ #define TEST_CRYPTODEV_H_ +#include <rte_crypto.h> +#include <rte_cryptodev.h> + #define HEX_DUMP 0 #define FALSE 0 diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c new file mode 100644 index 0000000000..34b759eaef --- /dev/null +++ b/app/test/test_pdcp.c @@ -0,0 +1,732 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_malloc.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "test.h" +#include "test_cryptodev.h" +#include "test_cryptodev_security_pdcp_test_vectors.h" + +#define NB_DESC 1024 +#define CDEV_INVALID_ID UINT8_MAX +#define NB_TESTS RTE_DIM(pdcp_test_params) +#define PDCP_IV_LEN 16 + +struct pdcp_testsuite_params { + struct rte_mempool *mbuf_pool; + struct rte_mempool *cop_pool; + struct rte_mempool *sess_pool; + bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; +}; + +static struct pdcp_testsuite_params testsuite_params; + +struct pdcp_test_conf { + struct rte_pdcp_entity_conf entity; + struct rte_crypto_sym_xform c_xfrm; + struct rte_crypto_sym_xform a_xfrm; + bool is_integrity_protected; + uint8_t input[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t input_len; + uint8_t output[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t output_len; +}; + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static int +cryptodev_init(int dev_id) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_cryptodev_qp_conf qp_conf; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_config config; + int ret, socket_id; + + /* Check if device was already initialized */ + if (ts_params->cdevs_used[dev_id]) + return 0; + + rte_cryptodev_info_get(dev_id, &dev_info); + + if (dev_info.max_nb_queue_pairs < 1) { + RTE_LOG(ERR, USER1, "Cryptodev doesn't have sufficient queue pairs available\n"); + return -ENODEV; + } + + socket_id = rte_socket_id(); + + memset(&config, 0, sizeof(config)); + config.nb_queue_pairs = 1; + config.socket_id = socket_id; + + ret = rte_cryptodev_configure(dev_id, &config); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure cryptodev - %d\n", dev_id); + return -ENODEV; + } + + memset(&qp_conf, 0, sizeof(qp_conf)); + qp_conf.nb_descriptors = NB_DESC; + + ret = rte_cryptodev_queue_pair_setup(dev_id, 0, &qp_conf, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure queue pair\n"); + return -ENODEV; + } + + ret = rte_cryptodev_start(dev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not start cryptodev\n"); + return -ENODEV; + } + + /* Mark device as initialized */ + ts_params->cdevs_used[dev_id] = true; + + return 0; +} + +static void +cryptodev_fini(int dev_id) +{ + rte_cryptodev_stop(dev_id); +} + +static unsigned int +cryptodev_sess_priv_max_req_get(void) +{ + struct rte_cryptodev_info info; + unsigned int sess_priv_sz; + int i, nb_dev; + void *sec_ctx; + + nb_dev = rte_cryptodev_count(); + + sess_priv_sz = 0; + + for (i = 0; i < nb_dev; i++) { + rte_cryptodev_info_get(i, &info); + sess_priv_sz = RTE_MAX(sess_priv_sz, rte_cryptodev_sym_get_private_session_size(i)); + if (info.feature_flags & RTE_CRYPTODEV_FF_SECURITY) { + sec_ctx = rte_cryptodev_get_sec_ctx(i); + sess_priv_sz = RTE_MAX(sess_priv_sz, + rte_security_session_get_size(sec_ctx)); + } + } + + return sess_priv_sz; +} + +static int +testsuite_setup(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + int nb_cdev, sess_priv_size, nb_sess = 1024; + + RTE_SET_USED(pdcp_test_hfn_threshold); + + nb_cdev = rte_cryptodev_count(); + if (nb_cdev < 1) { + RTE_LOG(ERR, USER1, "No crypto devices found.\n"); + return TEST_SKIPPED; + } + + memset(ts_params, 0, sizeof(*ts_params)); + + ts_params->mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NUM_MBUFS, MBUF_CACHE_SIZE, 0, + MBUF_SIZE, SOCKET_ID_ANY); + if (ts_params->mbuf_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf pool\n"); + return TEST_FAILED; + } + + ts_params->cop_pool = rte_crypto_op_pool_create("cop_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, MBUF_CACHE_SIZE, + 2 * MAXIMUM_IV_LENGTH, SOCKET_ID_ANY); + if (ts_params->cop_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create crypto_op pool\n"); + goto mbuf_pool_free; + } + + /* Get max session priv size required */ + sess_priv_size = cryptodev_sess_priv_max_req_get(); + + ts_params->sess_pool = rte_cryptodev_sym_session_pool_create("sess_pool", nb_sess, + sess_priv_size, + RTE_MEMPOOL_CACHE_MAX_SIZE, + 0, SOCKET_ID_ANY); + if (ts_params->sess_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create session pool\n"); + goto cop_pool_free; + } + + return 0; + +cop_pool_free: + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; +mbuf_pool_free: + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; + return TEST_FAILED; +} + +static void +testsuite_teardown(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + uint8_t dev_id; + + for (dev_id = 0; dev_id < RTE_CRYPTO_MAX_DEVS; dev_id++) { + if (ts_params->cdevs_used[dev_id]) + cryptodev_fini(dev_id); + } + + rte_mempool_free(ts_params->sess_pool); + ts_params->sess_pool = NULL; + + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; + + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; +} + +static int +ut_setup_pdcp(void) +{ + return 0; +} + +static void +ut_teardown_pdcp(void) +{ +} + +static int +crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +cryptodev_id_get(bool is_integrity_protected, const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm) +{ + int i, nb_devs; + + nb_devs = rte_cryptodev_count(); + + /* Check capabilities */ + + for (i = 0; i < nb_devs; i++) { + if ((crypto_caps_cipher_verify(i, c_xfrm) == 0) && + (!is_integrity_protected || crypto_caps_auth_verify(i, a_xfrm) == 0)) + break; + } + + if (i == nb_devs) + return -1; + + return i; +} + +static int +pdcp_known_vec_verify(struct rte_mbuf *m, const uint8_t *expected, uint32_t expected_pkt_len) +{ + uint8_t *actual = rte_pktmbuf_mtod(m, uint8_t *); + uint32_t actual_pkt_len = rte_pktmbuf_pkt_len(m); + + debug_hexdump(stdout, "Received:", actual, actual_pkt_len); + debug_hexdump(stdout, "Expected:", expected, expected_pkt_len); + + TEST_ASSERT_EQUAL(actual_pkt_len, expected_pkt_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_pkt_len, actual_pkt_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(actual, expected, expected_pkt_len, + "Generated packet not as expected"); + + return 0; +} + +static struct rte_crypto_op * +process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op) +{ + if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) { + RTE_LOG(ERR, USER1, "Error sending packet to cryptodev\n"); + return NULL; + } + + op = NULL; + + while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0) + rte_pause(); + + return op; +} + +static uint32_t +pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) +{ + uint32_t sn = 0; + + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + sn = rte_cpu_to_be_16(*(const uint16_t *)data); + sn = sn & 0xfff; + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + sn = rte_cpu_to_be_32(*(const uint32_t *)data); + sn = (sn & 0x3ffff00) >> 8; + } + + return sn; +} + +static int +create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_crypto_sym_xform c_xfrm, a_xfrm; + uint32_t sn, expected_len; + uint8_t *data, *expected; + int pdcp_hdr_sz; + + memset(conf, 0, sizeof(*conf)); + memset(&c_xfrm, 0, sizeof(c_xfrm)); + memset(&a_xfrm, 0, sizeof(a_xfrm)); + + conf->entity.sess_mpool = ts_params->sess_pool; + conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; + conf->entity.pdcp_xfrm.en_ordering = 0; + conf->entity.pdcp_xfrm.remove_duplicates = 0; + conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + + if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; + else + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + + conf->entity.pdcp_xfrm.sn_size = pdcp_test_data_sn_size[index]; + + /* Zero initialize unsupported flags */ + conf->entity.pdcp_xfrm.hfn_threshold = 0; + conf->entity.pdcp_xfrm.hfn_ovrd = 0; + conf->entity.pdcp_xfrm.sdap_enabled = 0; + + c_xfrm.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + c_xfrm.cipher.algo = pdcp_test_params[index].cipher_alg; + c_xfrm.cipher.key.length = pdcp_test_params[index].cipher_key_len; + c_xfrm.cipher.key.data = pdcp_test_crypto_key[index]; + + a_xfrm.type = RTE_CRYPTO_SYM_XFORM_AUTH; + + if (pdcp_test_params[index].auth_alg == 0) { + conf->is_integrity_protected = false; + } else { + a_xfrm.auth.algo = pdcp_test_params[index].auth_alg; + a_xfrm.auth.key.data = pdcp_test_auth_key[index]; + a_xfrm.auth.key.length = pdcp_test_params[index].auth_key_len; + conf->is_integrity_protected = true; + } + + pdcp_hdr_sz = pdcp_hdr_size_get(pdcp_test_data_sn_size[index]); + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protecting is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (conf->is_integrity_protected) { + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + conf->entity.crypto_xfrm = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + a_xfrm.next = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + c_xfrm.next = NULL; + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + c_xfrm.next = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + a_xfrm.next = NULL; + } + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + c_xfrm.next = NULL; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + else + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + /* Update xforms to match PDCP requirements */ + + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) + c_xfrm.cipher.iv.length = PDCP_IV_LEN; + else + c_xfrm.cipher.iv.length = 0; + + if (conf->is_integrity_protected) { + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm.auth.digest_length = 0; + else + a_xfrm.auth.digest_length = RTE_PDCP_MAC_I_LEN; + + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + a_xfrm.auth.iv.length = PDCP_IV_LEN; + else + a_xfrm.auth.iv.length = 0; + } + + conf->c_xfrm = c_xfrm; + conf->a_xfrm = a_xfrm; + + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf->is_integrity_protected, + &conf->c_xfrm, &conf->a_xfrm); + + if (pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_CONTROL || + pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_DATA) { + data = pdcp_test_data_in[index]; + sn = pdcp_sn_from_raw_get(data, pdcp_test_data_sn_size[index]); + conf->entity.pdcp_xfrm.hfn = pdcp_test_hfn[index]; + conf->entity.sn = sn; + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", pdcp_test_data_in[index], + pdcp_test_data_in_len[index]); +#endif + /* Since the vectors available already have PDCP header, trim the same */ + conf->input_len = pdcp_test_data_in_len[index] - pdcp_hdr_sz; + memcpy(conf->input, pdcp_test_data_in[index] + pdcp_hdr_sz, conf->input_len); + } else { + conf->input_len = pdcp_test_data_in_len[index]; + + if (conf->is_integrity_protected) + conf->input_len += RTE_PDCP_MAC_I_LEN; + + memcpy(conf->input, pdcp_test_data_out[index], conf->input_len); +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", conf->input, conf->input_len); +#endif + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected = pdcp_test_data_out[index]; + else + expected = pdcp_test_data_in[index]; + + /* Calculate expected packet length */ + expected_len = pdcp_test_data_in_len[index]; + + /* In DL processing, PDCP header would be stripped */ + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + expected += pdcp_hdr_sz; + expected_len -= pdcp_hdr_sz; + } + + /* In UL processing with integrity protection, MAC would be added */ + if (conf->is_integrity_protected && + conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected_len += 4; + + memcpy(conf->output, expected, expected_len); + conf->output_len = expected_len; + + return 0; +} + +static struct rte_pdcp_entity* +test_entity_create(const struct pdcp_test_conf *t_conf, int *rc) +{ + struct rte_pdcp_entity *pdcp_entity; + int ret; + + if (t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12 && + t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18) { + *rc = -ENOTSUP; + return NULL; + } + + if (t_conf->entity.dev_id == CDEV_INVALID_ID) { + RTE_LOG(DEBUG, USER1, "Could not find device with required capabilities\n"); + *rc = -ENOTSUP; + return NULL; + } + + ret = cryptodev_init(t_conf->entity.dev_id); + if (ret) { + *rc = ret; + RTE_LOG(DEBUG, USER1, "Could not initialize cryptodev\n"); + return NULL; + } + + rte_errno = 0; + + pdcp_entity = rte_pdcp_entity_establish(&t_conf->entity); + if (pdcp_entity == NULL) { + *rc = -rte_errno; + RTE_LOG(DEBUG, USER1, "Could not establish PDCP entity\n"); + return NULL; + } + + return pdcp_entity; +} + +static uint16_t +test_process_packets(const struct rte_pdcp_entity *pdcp_entity, uint8_t cdev_id, + struct rte_mbuf *in_mb[], uint16_t nb_in, + struct rte_mbuf *out_mb[], uint16_t *nb_err) +{ + struct rte_crypto_op *cop, *cop_out; + struct rte_pdcp_group grp[1]; + uint16_t nb_success, nb_grp; + struct rte_mbuf *mbuf, *mb; + + if (nb_in != 1) + return -ENOTSUP; + + mbuf = in_mb[0]; + + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, &mbuf, &cop_out, 1, nb_err); + if (nb_success != 1 || *nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not pre process PDCP packet\n"); + return TEST_FAILED; + } + +#ifdef VEC_DUMP + printf("Pre-processed vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, rte_pktmbuf_pkt_len(mbuf)); +#endif + + cop = process_crypto_request(cdev_id, cop_out); + if (cop == NULL) { + RTE_LOG(ERR, USER1, "Could not process crypto request\n"); + return -EIO; + } + + grp[0].id.val = 0; + + nb_grp = rte_pdcp_pkt_crypto_group(&cop_out, &mb, grp, 1); + if (nb_grp != 1 || grp[0].cnt != 1) { + RTE_LOG(ERR, USER1, "Could not group PDCP crypto results\n"); + return -ENOTRECOVERABLE; + } + + if ((uintptr_t)pdcp_entity != grp[0].id.val) { + RTE_LOG(ERR, USER1, "PDCP entity not matching the one from crypto_op\n"); + return -ENOTRECOVERABLE; + } + +#ifdef VEC_DUMP + printf("Crypto processed vector:\n"); + rte_pktmbuf_dump(stdout, cop->sym->m_dst, rte_pktmbuf_pkt_len(mbuf)); +#endif + + return rte_pdcp_pkt_post_process(grp[0].id.ptr, grp[0].m, out_mb, grp[0].cnt, nb_err); +} + +static struct rte_mbuf* +mbuf_from_data_create(uint8_t *data, uint16_t data_len) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_mbuf *mbuf; + uint8_t *input_text; + + mbuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (mbuf == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf\n"); + return NULL; + } + + memset(rte_pktmbuf_mtod(mbuf, uint8_t *), 0, rte_pktmbuf_tailroom(mbuf)); + + input_text = (uint8_t *)rte_pktmbuf_append(mbuf, data_len); + memcpy(input_text, data, data_len); + + return mbuf; +} + +static int +test_attempt_single(struct pdcp_test_conf *t_conf) +{ + struct rte_mbuf *mbuf, **out_mb = NULL; + struct rte_pdcp_entity *pdcp_entity; + uint16_t nb_success, nb_err; + int ret = 0, nb_max_out_mb; + + pdcp_entity = test_entity_create(t_conf, &ret); + if (pdcp_entity == NULL) + goto exit; + + /* Allocate buffer for holding mbufs returned */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate buffer for holding out_mb buffers\n"); + ret = -ENOMEM; + goto entity_release; + } + + mbuf = mbuf_from_data_create(t_conf->input, t_conf->input_len); + if (mbuf == NULL) { + ret = -ENOMEM; + goto entity_release; + } + +#ifdef VEC_DUMP + printf("Adjusted vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, t_conf->input_len); +#endif + + nb_success = test_process_packets(pdcp_entity, t_conf->entity.dev_id, &mbuf, 1, out_mb, + &nb_err); + if (nb_success != 1 || nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not process PDCP packet\n"); + ret = TEST_FAILED; + goto mbuf_free; + } + + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + + ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); + if (ret) { + RTE_LOG(DEBUG, USER1, "Could not suspend PDCP entity\n"); + goto mbuf_free; + } + +mbuf_free: + rte_pktmbuf_free(mbuf); +entity_release: + rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_free(out_mb); +exit: + return ret; +} + +static int +run_test_for_one_known_vec(const void *arg) +{ + struct pdcp_test_conf test_conf; + int i = *(const uint32_t *)arg; + + create_test_conf_from_index(i, &test_conf); + return test_attempt_single(&test_conf); +} + +struct unit_test_suite *test_suites[] = { + NULL, /* Place holder for known_vector_cases */ + NULL /* End of suites list */ +}; + +static struct unit_test_suite pdcp_testsuite = { + .suite_name = "PDCP Unit Test Suite", + .unit_test_cases = {TEST_CASES_END()}, + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_suites = test_suites, +}; + +static int +test_pdcp(void) +{ + struct unit_test_suite *known_vector_cases; + int ret, index[NB_TESTS]; + uint32_t i, size; + + size = sizeof(struct unit_test_suite); + size += (NB_TESTS + 1) * sizeof(struct unit_test_case); + + known_vector_cases = rte_zmalloc(NULL, size, 0); + if (known_vector_cases == NULL) + return TEST_FAILED; + + known_vector_cases->suite_name = "Known vector cases"; + + for (i = 0; i < NB_TESTS; i++) { + index[i] = i; + known_vector_cases->unit_test_cases[i].name = pdcp_test_params[i].name; + known_vector_cases->unit_test_cases[i].data = (void *)&index[i]; + known_vector_cases->unit_test_cases[i].enabled = 1; + known_vector_cases->unit_test_cases[i].setup = ut_setup_pdcp; + known_vector_cases->unit_test_cases[i].teardown = ut_teardown_pdcp; + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data + = run_test_for_one_known_vec; + } + + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data = NULL; + + test_suites[0] = known_vector_cases; + + ret = unit_test_suite_runner(&pdcp_testsuite); + + rte_free(known_vector_cases); + return ret; +} + +REGISTER_TEST_COMMAND(pdcp_autotest, test_pdcp); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 10/22] test/pdcp: pdcp HFN tests in combined mode 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (8 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 09/22] app/test: add lib pdcp tests Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 11/22] doc: add PDCP library guide Anoob Joseph ` (13 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add tests to verify HFN/SN behaviour. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 302 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 299 insertions(+), 3 deletions(-) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 34b759eaef..cfe2ec6aa9 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,9 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* According to formula(7.2.a Window_Size) */ +#define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) + struct pdcp_testsuite_params { struct rte_mempool *mbuf_pool; struct rte_mempool *cop_pool; @@ -36,12 +39,69 @@ struct pdcp_test_conf { uint32_t output_len; }; +static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf); + +typedef int (*test_with_conf_t)(struct pdcp_test_conf *conf); + +static int +run_test_foreach_known_vec(test_with_conf_t test, bool stop_on_first_pass) +{ + struct pdcp_test_conf test_conf; + bool all_tests_skipped = true; + uint32_t i; + int ret; + + for (i = 0; i < NB_TESTS; i++) { + create_test_conf_from_index(i, &test_conf); + ret = test(&test_conf); + + if (ret == TEST_FAILED) { + printf("[%03i] - %s - failed\n", i, pdcp_test_params[i].name); + return TEST_FAILED; + } + + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + continue; + + if (stop_on_first_pass) + return TEST_SUCCESS; + + all_tests_skipped = false; + } + + if (all_tests_skipped) + return TEST_SKIPPED; + + return TEST_SUCCESS; +} + +static int +run_test_with_all_known_vec(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, false); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static int +pktmbuf_read_into(const struct rte_mbuf *m, void *buf, size_t buf_len) +{ + if (m->pkt_len > buf_len) + return -ENOMEM; + + const void *read = rte_pktmbuf_read(m, 0, m->pkt_len, buf); + if (read != NULL && read != buf) + memcpy(buf, read, m->pkt_len); + + return 0; +} + static int cryptodev_init(int dev_id) { @@ -326,6 +386,21 @@ pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) return sn; } +static void +pdcp_sn_to_raw_set(void *data, uint32_t sn, int size) +{ + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr = data; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr = data; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } +} + static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) { @@ -648,9 +723,17 @@ test_attempt_single(struct pdcp_test_conf *t_conf) goto mbuf_free; } - ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); - if (ret) - goto mbuf_free; + /* If expected output provided - verify, else - store for future use */ + if (t_conf->output_len) { + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + } else { + ret = pktmbuf_read_into(mbuf, t_conf->output, RTE_PDCP_CTRL_PDU_SIZE_MAX); + if (ret) + goto mbuf_free; + t_conf->output_len = mbuf->pkt_len; + } ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); if (ret) { @@ -667,6 +750,193 @@ test_attempt_single(struct pdcp_test_conf *t_conf) return ret; } +static void +uplink_to_downlink_convert(const struct pdcp_test_conf *ul_cfg, + struct pdcp_test_conf *dl_cfg) +{ + assert(ul_cfg->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK); + + memcpy(dl_cfg, ul_cfg, sizeof(*dl_cfg)); + dl_cfg->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + dl_cfg->entity.reverse_iv_direction = false; + + if (dl_cfg->is_integrity_protected) { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + dl_cfg->c_xfrm.next = &dl_cfg->a_xfrm; + + dl_cfg->a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + dl_cfg->a_xfrm.next = NULL; + } else { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + dl_cfg->c_xfrm.next = NULL; + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + dl_cfg->entity.dev_id = (uint8_t)cryptodev_id_get(dl_cfg->is_integrity_protected, + &dl_cfg->c_xfrm, &dl_cfg->a_xfrm); + + memcpy(dl_cfg->input, ul_cfg->output, ul_cfg->output_len); + dl_cfg->input_len = ul_cfg->output_len; + + memcpy(dl_cfg->output, ul_cfg->input, ul_cfg->input_len); + dl_cfg->output_len = ul_cfg->input_len; +} + +/* + * According to ETSI TS 138 323 V17.1.0, Section 5.2.2.1, + * SN could be divided into following ranges, + * relatively to current value of RX_DELIV state: + * +-------------+-------------+-------------+-------------+ + * | -Outside | -Window | +Window | +Outside | + * | (valid) | (Invalid) | (Valid) | (Invalid) | + * +-------------+-------------^-------------+-------------+ + * | + * v + * SN(RX_DELIV) + */ +enum sn_range_type { + SN_RANGE_MINUS_OUTSIDE, + SN_RANGE_MINUS_WINDOW, + SN_RANGE_PLUS_WINDOW, + SN_RANGE_PLUS_OUTSIDE, +}; + +#define PDCP_SET_COUNT(hfn, sn, size) ((hfn << size) | (sn & ((1 << size) - 1))) + +/* + * Take uplink test case as base, modify RX_DELIV in state and SN in input + */ +static int +test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) +{ + uint32_t rx_deliv_hfn, rx_deliv_sn, new_hfn, new_sn; + const int domain = conf->entity.pdcp_xfrm.domain; + struct pdcp_test_conf dl_conf; + int ret, expected_ret; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + if (domain != RTE_SECURITY_PDCP_MODE_CONTROL && domain != RTE_SECURITY_PDCP_MODE_DATA) + return TEST_SKIPPED; + + const uint32_t sn_size = conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + /* Max value of SN that could fit in `sn_size` bits */ + const uint32_t max_sn = (1 << sn_size) - 1; + const uint32_t shift = (max_sn - window_size) / 2; + /* Could be any number up to `shift` value */ + const uint32_t default_sn = RTE_MIN(2u, shift); + + /* Initialize HFN as non zero value, to be able check values before */ + rx_deliv_hfn = 0xa; + + switch (type) { + case SN_RANGE_PLUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + 1; + expected_ret = TEST_SUCCESS; + break; + case SN_RANGE_MINUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn - 1; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_PLUS_OUTSIDE: + /* RCVD_SN >= SN(RX_DELIV) + Window_Size */ + new_hfn = rx_deliv_hfn - 1; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + window_size; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_MINUS_OUTSIDE: + /* RCVD_SN < SN(RX_DELIV) - Window_Size */ + new_hfn = rx_deliv_hfn + 1; + rx_deliv_sn = window_size + default_sn; + new_sn = rx_deliv_sn - window_size - 1; + expected_ret = TEST_SUCCESS; + break; + default: + return TEST_FAILED; + } + + /* Configure Uplink to generate expected, encrypted packet */ + pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.reverse_iv_direction = true; + conf->entity.pdcp_xfrm.hfn = new_hfn; + conf->entity.sn = new_sn; + conf->output_len = 0; + ret = test_attempt_single(conf); + if (ret != TEST_SUCCESS) + return ret; + + /* Flip configuration to downlink */ + uplink_to_downlink_convert(conf, &dl_conf); + + /* Modify the rx_deliv to verify the expected behaviour */ + dl_conf.entity.pdcp_xfrm.hfn = rx_deliv_hfn; + dl_conf.entity.sn = rx_deliv_sn; + ret = test_attempt_single(&dl_conf); + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + return ret; + + TEST_ASSERT_EQUAL(ret, expected_ret, "Unexpected result"); + + return TEST_SUCCESS; +} + +static int +test_sn_plus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_WINDOW, t_conf); +} + +static int +test_sn_minus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_WINDOW, t_conf); +} + +static int +test_sn_plus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_OUTSIDE, t_conf); +} + +static int +test_sn_minus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); +} + +static int +test_combined(struct pdcp_test_conf *ul_conf) +{ + struct pdcp_test_conf dl_conf; + int ret; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + ul_conf->entity.reverse_iv_direction = true; + ul_conf->output_len = 0; + + ret = test_attempt_single(ul_conf); + if (ret != TEST_SUCCESS) + return ret; + + uplink_to_downlink_convert(ul_conf, &dl_conf); + ret = test_attempt_single(&dl_conf); + + return ret; +} + static int run_test_for_one_known_vec(const void *arg) { @@ -677,8 +947,34 @@ run_test_for_one_known_vec(const void *arg) return test_attempt_single(&test_conf); } +static struct unit_test_suite combined_mode_cases = { + .suite_name = "PDCP combined mode", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("combined mode", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_combined), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static struct unit_test_suite hfn_sn_test_cases = { + .suite_name = "PDCP HFN/SN", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("SN plus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_window), + TEST_CASE_NAMED_WITH_DATA("SN minus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_window), + TEST_CASE_NAMED_WITH_DATA("SN plus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_outside), + TEST_CASE_NAMED_WITH_DATA("SN minus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_outside), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ + &combined_mode_cases, + &hfn_sn_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 11/22] doc: add PDCP library guide 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (9 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 12/22] pdcp: add control PDU handling for status report Anoob Joseph ` (12 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add guide for PDCP library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 254 ++++++++++++++++++ 3 files changed, 256 insertions(+) create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst diff --git a/doc/guides/prog_guide/img/pdcp_functional_overview.svg b/doc/guides/prog_guide/img/pdcp_functional_overview.svg new file mode 100644 index 0000000000..287daafc21 --- /dev/null +++ b/doc/guides/prog_guide/img/pdcp_functional_overview.svg @@ -0,0 +1 @@ +<svg width="1280" height="720" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><clipPath id="clip0"><rect x="0" y="0" width="1280" height="720"/></clipPath></defs><g clip-path="url(#clip0)"><rect x="0" y="0" width="1280" height="720" fill="#FFFFFF"/><rect x="202" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><rect x="640" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><path d="M605.5 73.5001 605.5 590.633" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M380.5 634 803.25 634 803.25 585.907 790.5 585.907 816 557.5 841.5 585.907 828.75 585.907 828.75 659.5 380.5 659.5Z" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5" fill-rule="evenodd"/><rect x="362.5" y="557.5" width="28" height="102" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5"/><rect x="412.5" y="671.5" width="370" height="32" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(492.364 694)">Radio Interface (<tspan font-size="19" x="137.5" y="0">Uu</tspan><tspan font-size="19" x="161.333" y="0">/PC5)</tspan><tspan font-size="19" x="-282.121" y="-653">UE/NG</tspan><tspan font-size="19" x="-222.955" y="-653">-</tspan><tspan font-size="19" x="-216.788" y="-653">RAN/UE A</tspan><tspan font-size="19" x="375.54" y="-653">NG</tspan>-<tspan font-size="19" x="409.706" y="-653">RAN/UE/UE B</tspan><tspan font-size="14" x="0.896851" y="-647">Transmitting </tspan><tspan font-size="14" x="1.31018" y="-631">PDCP entity</tspan><tspan font-size="14" x="167.401" y="-647">Receiving </tspan><tspan font-size="14" x="160.148" y="-631">PDCP entity</tspan></text><path d="M314.5 71.5001 431.364 71.6549" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M331.5 71.5001C331.5 65.9772 349.633 61.5001 372 61.5001 394.368 61.5001 412.5 65.9772 412.5 71.5001 412.5 77.0229 394.368 81.5001 372 81.5001 349.633 81.5001 331.5 77.0229 331.5 71.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M353.5 90.5001 363.5 90.5001 363.5 48.5001 383.5 48.5001 383.5 90.5001 393.5 90.5001 373.5 110.5Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M877.364 82.6549 760.5 82.5001" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M860.5 83.5001C860.5 89.0229 842.368 93.5001 820 93.5001 797.633 93.5001 779.5 89.0229 779.5 83.5001 779.5 77.9772 797.633 73.5001 820 73.5001 842.368 73.5001 860.5 77.9772 860.5 83.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M838.5 64.5 828.5 64.5 828.5 106.5 808.5 106.5 808.5 64.5 798.5 64.5 818.5 44.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><rect x="244.5" y="128.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(302.494 150)">Transmission buffer:<tspan font-size="19" x="-4.67999" y="23">Sequence</tspan><tspan font-size="19" x="84.32" y="23">numberin</tspan>g</text><rect x="244.5" y="199.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(297.314 222)">Header or uplink data<tspan font-size="19" x="34.08" y="23">Compressio</tspan>n</text><rect x="682.5" y="141.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(735.782 164)">Header or uplink data<tspan font-size="19" x="24.2466" y="23">Decompressio</tspan>n</text><rect x="244.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(300.314 505)">Routing / Duplication</text><rect x="244.5" y="437.5" width="285" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(309.734 460)">Add PDCP header</text><rect x="244.5" y="383.5" width="189" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(298.62 406)">Ciphering</text><rect x="244.5" y="333.5" width="189" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(261.953 356)">Integrity protection</text><path d="M472.167 267.5 472.167 409.802 470.833 409.802 470.833 267.5ZM475.5 408.468 471.5 416.468 467.5 408.468Z"/><path d="M332.167 267.5 332.167 319.552 330.833 319.552 330.833 267.5ZM335.5 318.218 331.5 326.218 327.5 318.219Z"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(253.644 291)">Packets associated <tspan font-size="19" x="14.0067" y="23">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 499.312 299)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="-2.75546e-15" y="45">PDCP SDU</tspan></text><rect x="682.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(728.535 505)">Remove PDCP Header</text><rect x="682.5" y="437.5" width="203" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(733.729 460)">Deciphering</text><rect x="682.5" y="389.5" width="203" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(702.159 412)">Integrity Verification</text><rect x="682.5" y="303.5" width="203" height="77" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(712.729 325)">Reception buffer:<tspan font-size="19" x="24.6667" y="23">Reordering</tspan><tspan font-size="19" x="-13.1667" y="45">Duplicate discardin</tspan><tspan font-size="19" x="144.167" y="45">g</tspan><tspan font-size="19" x="-10.0989" y="-84">Packets associated </tspan><tspan font-size="19" x="3.90784" y="-61">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 960.34 294)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="1" y="45">PDCP SDU</tspan></text><path d="M0.666667-8.2074e-07 0.666763 78.6116-0.66657 78.6116-0.666667 8.2074e-07ZM4.00009 77.2782 0.000104987 85.2782-3.9999 77.2782Z" transform="matrix(1 0 0 -1 779.5 296.778)"/><path d="M0.666667-2.88742e-07 0.666769 235.734-0.666565 235.734-0.666667 2.88742e-07ZM4.0001 234.401 0.000104987 242.401-3.9999 234.401Z" transform="matrix(1 0 0 -1 931.5 453.901)"/></g></svg> \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 87333ee84a..6099ff63cd 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -77,4 +77,5 @@ Programmer's Guide lto profile_app asan + pdcp_lib glossary diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst new file mode 100644 index 0000000000..8369c71600 --- /dev/null +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -0,0 +1,254 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2023 Marvell. + +PDCP Protocol Processing Library +================================ + +DPDK provides a library for PDCP protocol processing. The library utilizes +other DPDK libraries such as cryptodev, reorder, etc., to provide the +application with a transparent and high performant PDCP protocol processing +library. + +The library abstracts complete PDCP protocol processing conforming to +``ETSI TS 138 323 V17.1.0 (2022-08)``. +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + +PDCP would involve the following operations, + +1. Transfer of user plane data +2. Transfer of control plane data +3. Header compression +4. Uplink data compression +5. Ciphering and integrity protection + +.. _figure_pdcp_functional_overview: + +.. figure:: img/pdcp_functional_overview.* + + PDCP functional overview new + +PDCP library would abstract the protocol offload features of the cryptodev and +would provide a uniform interface and consistent API usage to work with +cryptodev irrespective of the protocol offload features supported. + +PDCP entity API +--------------- + +PDCP library provides following control path APIs that is used to +configure various PDCP entities, + +1. ``rte_pdcp_entity_establish()`` +2. ``rte_pdcp_entity_suspend()`` +3. ``rte_pdcp_entity_release()`` + +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or +``rte_security_session`` based on the config. The sessions would be created/ +destroyed while corresponding PDCP entity operations are performed. + +When upper layers request a PDCP entity suspend (``rte_pdcp_entity_suspend()``), +it would result in flushing out of all cached packets. Internal state variables +are updated as described in 5.1.4. + +When upper layers request a PDCP entity release (``rte_pdcp_entity_release()``), +it would result in flushing out of all cached packets and releasing of all +memory associated with the entity. It would internally free any crypto/security +sessions created. All procedures mentioned in 5.1.3 would be performed. + +PDCP PDU (Protocol Data Unit) API +--------------------------------- + +PDCP PDUs can be categorized as, + +1. Control PDU +2. Data PDU + +Control PDUs are used for signalling between entities on either end and can be +one of the following, + +1. PDCP status report +2. ROHC feedback +3. EHC feedback + +Control PDUs are not ciphered or authenticated, and so such packets are not +submitted to cryptodev for processing. + +Data PDUs are regular packets submitted by upper layers for transmission to +other end. Such packets would need to be ciphered and authenticated based on +the entity configuration. + +PDCP packet processing API for data PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PDCP processing is split into 2 parts. One before cryptodev processing +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return crypto +operations belonging to multiple entities, ``rte_pdcp_pkt_crypto_group()`` +is added to help grouping crypto operations belonging to same PDCP entity. + +Lib PDCP would allow application to use same API sequence while leveraging +protocol offload features enabled by ``rte_security`` library. Lib PDCP would +internally change the handles registered for ``pre_process`` and +``post_process`` based on features enabled in the entity. + +Lib PDCP would create the required sessions on the device provided in entity to +minimize the application requirements. Also, the crypto_op allocation and free +would also be done internally by lib PDCP to allow the library to create +crypto ops as required for the input packets. For example, when control PDUs are +received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP +is expected to handle it differently. + +Supported features +------------------ + +- 12 bit & 18 bit sequence numbers +- Uplink & downlink traffic +- HFN increment +- IV generation as required per algorithm + +Supported ciphering algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CTR +- SNOW3G-CIPHER +- ZUC-CIPHER + +Supported integrity protection algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CMAC +- SNOW3G-AUTH +- ZUC-AUTH + +Sample API usage +---------------- + +The ``rte_pdcp_entity_conf`` structure is used to pass the configuration +parameters for entity creation. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_entity_conf 8< + :end-before: >8 End of structure rte_pdcp_entity_conf. + +.. code-block:: c + + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; + struct rte_crypto_op *cop[MAX_BURST_SIZE]; + struct rte_pdcp_group grp[MAX_BURST_SIZE]; + struct rte_pdcp_entity *pdcp_entity; + int nb_max_out_mb, ret, nb_grp; + uint16_t nb_ops; + + /* Create PDCP entity */ + pdcp_entity = rte_pdcp_entity_establish(&conf); + + /** + * Allocate buffer for holding mbufs returned during PDCP suspend, + * release & post-process APIs. + */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + MAX_BURST_SIZE; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + /* Handle error */ + } + + while (1) { + /* Receive packet and form mbuf */ + + /** + * Prepare packets for crypto operation. Following operations + * would be done, + * + * Transmitting entity/UL (only data PDUs): + * - Perform compression + * - Assign sequence number + * - Add PDCP header + * - Create & prepare crypto_op + * - Prepare IV for crypto operation (auth_gen, encrypt) + * - Save original PDCP SDU (during PDCP re-establishment, + * unconfirmed PDCP SDUs need to crypto processed again and + * transmitted/re-transmitted) + * + * Receiving entity/DL: + * - Any control PDUs received would be processed and + * appropriate actions taken. If data PDU, continue. + * - Determine sequence number (based on HFN & per packet SN) + * - Prepare crypto_op + * - Prepare IV for crypto operation (decrypt, auth_verify) + */ + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, + nb_rx, &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) + != nb_success) { + /* Retry for enqueue failure packets */ + } + + ... + + nb_ops = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, + MAX_BURST_SIZE); + if (nb_ops == 0) + continue; + + /** + * Received a burst of completed crypto ops from cryptodev. It + * may belong to various entities. Group similar ones together + * for entity specific post-processing. + */ + + /** + * Groups similar entities together. Frees crypto op and based + * on crypto_op status, set mbuf->ol_flags which would be + * checked in rte_pdcp_pkt_post_process(). + */ + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); + + for (i = 0; i != nb_grp; i++) { + + /** + * Post process packets after crypto completion. + * Following operations would be done, + * + * Transmitting entity/UL: + * - Check crypto result + * + * Receiving entity/DL: + * - Check crypto operation status + * - Check for duplication (if yes, drop duplicate) + * - Perform decompression + * - Trim PDCP header + * - Hold packet (SDU) for in-order delivery (return + * completed packets as and when sequence is + * completed) + * - If not in sequence, cache the packet and start + * t-Reordering timer. When timer expires, the + * packets need to delivered to upper layers (not + * treated as error packets). + */ + nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, + grp[i].m, out_mb, + grp[i].cnt, + &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + /* Perform additional operations */ + + /** + * Transmitting entity/UL + * - If duplication is enabled, duplicate PDCP PDUs + * - When lower layers confirm reception of a PDCP PDU, + * it should be communicated to PDCP layer so that + * PDCP can drop the corresponding SDU + */ + } + } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 12/22] pdcp: add control PDU handling for status report 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (10 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 11/22] doc: add PDCP library guide Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph ` (11 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add control PDU handling and implement status report generation. Status report generation works only when RX_DELIV = RX_NEXT. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 9 ++++++ lib/pdcp/meson.build | 2 ++ lib/pdcp/pdcp_cnt.c | 29 ++++++++++++++++++ lib/pdcp/pdcp_cnt.h | 14 +++++++++ lib/pdcp/pdcp_ctrl_pdu.c | 46 +++++++++++++++++++++++++++++ lib/pdcp/pdcp_ctrl_pdu.h | 15 ++++++++++ lib/pdcp/pdcp_entity.h | 15 ++++++++-- lib/pdcp/pdcp_process.c | 13 +++++++++ lib/pdcp/rte_pdcp.c | 47 +++++++++++++++++++++++++++++- lib/pdcp/rte_pdcp.h | 33 +++++++++++++++++++++ lib/pdcp/version.map | 2 ++ 11 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index 8369c71600..dcb424bb1d 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -76,6 +76,15 @@ Data PDUs are regular packets submitted by upper layers for transmission to other end. Such packets would need to be ciphered and authenticated based on the entity configuration. +PDCP packet processing API for control PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Control PDUs are used in PDCP as a communication channel between transmitting +and receiving entities. When upper layer request for operations such +re-establishment, receiving PDCP entity need to prepare a status report and +send it to the other end. The API ``rte_pdcp_control_pdu_create`` allows +application to request the same. + PDCP packet processing API for data PDU ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 08679b743a..75d476bf6d 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -8,7 +8,9 @@ if is_windows endif sources = files( + 'pdcp_cnt.c', 'pdcp_crypto.c', + 'pdcp_ctrl_pdu.c', 'pdcp_process.c', 'rte_pdcp.c', ) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c new file mode 100644 index 0000000000..c9b952184b --- /dev/null +++ b/lib/pdcp/pdcp_cnt.c @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_cnt.h" +#include "pdcp_entity.h" + +int +pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv_dl_part *en_priv_dl; + uint32_t window_sz; + + if (en == NULL || conf == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + return 0; + + en_priv_dl = entity_dl_part_get(en); + window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + RTE_SET_USED(window_sz); + RTE_SET_USED(en_priv_dl); + + return 0; +} diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h new file mode 100644 index 0000000000..bbda478b55 --- /dev/null +++ b/lib/pdcp/pdcp_cnt.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CNT_H +#define PDCP_CNT_H + +#include <rte_common.h> + +#include "pdcp_entity.h" + +int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c new file mode 100644 index 0000000000..feb05fd863 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_byteorder.h> +#include <rte_mbuf.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_ctrl_pdu.h" +#include "pdcp_entity.h" + +static __rte_always_inline void +pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) +{ + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + pdu_hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + pdu_hdr->r = 0; + pdu_hdr->fmc = rte_cpu_to_be_32(rx_deliv); +} + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; + uint32_t rx_deliv; + int pdu_sz; + + if (!en_priv->flags.is_status_report_required) + return -EINVAL; + + pdu_sz = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + + rx_deliv = en_priv->state.rx_deliv; + + /* Zero missing PDUs - status report contains only FMC */ + if (rx_deliv >= en_priv->state.rx_next) { + pdu_hdr = (struct rte_pdcp_up_ctrl_pdu_hdr *)rte_pktmbuf_append(m, pdu_sz); + if (pdu_hdr == NULL) + return -ENOMEM; + pdcp_hdr_fill(pdu_hdr, rx_deliv); + + return 0; + } + + return -ENOTSUP; +} diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h new file mode 100644 index 0000000000..a2424fbd10 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CTRL_PDU_H +#define PDCP_CTRL_PDU_H + +#include <rte_mbuf.h> + +#include "pdcp_entity.h" + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); + +#endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 10a72faae1..28691a504b 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -109,6 +109,13 @@ union cipher_iv_partial { uint64_t u64[2]; }; +struct pdcp_cnt_bitmap { + /** Number of entries that can be stored. */ + uint32_t size; + /** Bitmap of the count values already received.*/ + struct rte_bitmap *bmp; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -136,9 +143,13 @@ struct entity_priv { uint64_t is_ul_entity : 1; /** Is NULL auth. */ uint64_t is_null_auth : 1; + /** Is status report required.*/ + uint64_t is_status_report_required : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; + /** Control PDU pool. */ + struct rte_mempool *ctrl_pdu_pool; /** PDCP header size. */ uint8_t hdr_sz; /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ @@ -148,8 +159,8 @@ struct entity_priv { }; struct entity_priv_dl_part { - /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ - uint8_t dummy; + /** PDCP would need to track the count values that are already received.*/ + struct pdcp_cnt_bitmap bitmap; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 8c802e735b..860ca16596 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -1091,6 +1091,19 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) en_priv->flags.is_null_auth = 1; + /** + * flags.is_status_report_required + * + * Indicate whether status report is required. + */ + if (conf->status_report_required) { + /** Status report is required only for DL entities. */ + if (conf->pdcp_xfrm.pkt_dir != RTE_SECURITY_PDCP_DOWNLINK) + return -EINVAL; + + en_priv->flags.is_status_report_required = 1; + } + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 91dab91f73..96ad397667 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -6,7 +6,9 @@ #include <rte_pdcp.h> #include <rte_malloc.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -58,7 +60,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (pdcp_dynfield_register() < 0) return NULL; - if (conf == NULL || conf->cop_pool == NULL) { + if (conf == NULL || conf->cop_pool == NULL || conf->ctrl_pdu_pool == NULL) { rte_errno = EINVAL; return NULL; } @@ -105,6 +107,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) en_priv->state.rx_deliv = count; en_priv->state.tx_next = count; en_priv->cop_pool = conf->cop_pool; + en_priv->ctrl_pdu_pool = conf->ctrl_pdu_pool; /* Setup crypto session */ ret = pdcp_crypto_sess_create(entity, conf); @@ -115,6 +118,10 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + ret = pdcp_cnt_ring_create(entity, conf); + if (ret) + goto crypto_sess_destroy; + return entity; crypto_sess_destroy: @@ -162,3 +169,41 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, return 0; } + +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type) +{ + struct entity_priv *en_priv; + struct rte_mbuf *m; + int ret; + + if (pdcp_entity == NULL) { + rte_errno = EINVAL; + return NULL; + } + + en_priv = entity_priv_get(pdcp_entity); + + m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); + if (m == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + switch (type) { + case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: + ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + break; + default: + ret = -ENOTSUP; + } + + if (ret) { + rte_pktmbuf_free(m); + rte_errno = -ret; + return NULL; + } + + return m; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index b88cad4f64..2a2bae1cd1 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -16,6 +16,7 @@ #include <rte_compat.h> #include <rte_common.h> #include <rte_mempool.h> +#include <rte_pdcp_hdr.h> #include <rte_security.h> #ifdef __cplusplus @@ -79,6 +80,8 @@ struct rte_pdcp_entity_conf { struct rte_mempool *sess_mpool; /** Crypto op pool.*/ struct rte_mempool *cop_pool; + /** Mbuf pool to be used for allocating control PDUs.*/ + struct rte_mempool *ctrl_pdu_pool; /** * SN value to be used. 32 bit count value to be used for the first * packet would be derived based on HFN (`rte_security_pdcp_xform.hfn`) @@ -98,6 +101,16 @@ struct rte_pdcp_entity_conf { * crypto processing. */ bool reverse_iv_direction; + /** + * Status report required (specified in TS 38.331). + * + * If PDCP entity is configured to send a PDCP status report, the upper + * layer application may request a receiving PDCP entity to generate a + * PDCP status report using ``rte_pdcp_control_pdu_create``. In + * addition, PDCP status reports may be generated during operations such + * as entity re-establishment. + */ + bool status_report_required; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -179,6 +192,26 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Create control PDU packet of the `type` specified. The control PDU packet + * would be allocated from *rte_pdcp_entity_conf.ctrl_pdu_pool* by lib PDCP. + * + * @param pdcp_entity + * Pointer to the PDCP entity for which the control PDU need to be generated. + * @param type + * Type of control PDU to be generated. + * @return + * - Control PDU generated, in case of success. + * - NULL in case of failure. rte_errno will be set to error code. + */ +__rte_experimental +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index d564f155e0..97171f902e 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,6 +2,8 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_control_pdu_create; + rte_pdcp_en_from_cop; rte_pdcp_entity_establish; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 13/22] pdcp: implement t-Reordering and packet buffering 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (11 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 12/22] pdcp: add control PDU handling for status report Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 14/22] test/pdcp: add in-order delivery cases Anoob Joseph ` (10 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add in-order delivery of packets in PDCP. Delivery of packets in-order relies on t-Reordering timer. When 'out-of-order delivery' is disabled, PDCP will buffer all received packets that are out of order. The t-Reordering timer determines the time period these packets would be held in the buffer, waiting for any missing packets to arrive. Introduce packet buffering and state variables which indicate status of the timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 3 +- lib/pdcp/pdcp_entity.h | 19 +++++++ lib/pdcp/pdcp_process.c | 117 ++++++++++++++++++++++++++++++---------- lib/pdcp/pdcp_reorder.c | 27 ++++++++++ lib/pdcp/pdcp_reorder.h | 62 +++++++++++++++++++++ lib/pdcp/rte_pdcp.c | 53 ++++++++++++++++-- lib/pdcp/rte_pdcp.h | 6 ++- 7 files changed, 252 insertions(+), 35 deletions(-) create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 75d476bf6d..f4f9246bcb 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -12,9 +12,10 @@ sources = files( 'pdcp_crypto.c', 'pdcp_ctrl_pdu.c', 'pdcp_process.c', + 'pdcp_reorder.c', 'rte_pdcp.c', ) headers = files('rte_pdcp.h') indirect_headers += files('rte_pdcp_group.h') -deps += ['mbuf', 'net', 'cryptodev', 'security'] +deps += ['mbuf', 'net', 'cryptodev', 'security', 'reorder'] diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 28691a504b..34341cdc11 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -11,6 +11,8 @@ #include <rte_pdcp.h> #include <rte_security.h> +#include "pdcp_reorder.h" + struct entity_priv; #define PDCP_HFN_MIN 0 @@ -109,6 +111,17 @@ union cipher_iv_partial { uint64_t u64[2]; }; +enum timer_state { + TIMER_STOP, + TIMER_RUNNING, + TIMER_EXPIRED, +}; + +struct pdcp_t_reordering { + /** Represent timer state */ + enum timer_state state; +}; + struct pdcp_cnt_bitmap { /** Number of entries that can be stored. */ uint32_t size; @@ -145,6 +158,8 @@ struct entity_priv { uint64_t is_null_auth : 1; /** Is status report required.*/ uint64_t is_status_report_required : 1; + /** Is out-of-order delivery enabled */ + uint64_t is_out_of_order_delivery : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -161,6 +176,10 @@ struct entity_priv { struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; + /** t-Reordering handles */ + struct pdcp_t_reordering t_reorder; + /** Reorder packet buffer */ + struct pdcp_reorder reorder; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 860ca16596..d910cb7c03 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -836,25 +836,88 @@ pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool tr } } -static inline bool +static inline int pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, - const uint32_t count) + const uint32_t count, struct rte_mbuf *mb, + struct rte_mbuf *out_mb[], + const bool trim_mac) { struct entity_priv *en_priv = entity_priv_get(entity); + struct pdcp_t_reordering *t_reorder; + struct pdcp_reorder *reorder; + uint16_t processed = 0; - if (count < en_priv->state.rx_deliv) - return false; + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + const uint32_t hdr_trim_sz = en_priv->hdr_sz + en_priv->aad_sz; - /* t-Reordering timer is not supported - SDU will be delivered immediately. - * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not - * been delivered to upper layers - */ - en_priv->state.rx_next = count + 1; + if (count < en_priv->state.rx_deliv) + return -EINVAL; if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; - return true; + pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); + + if (en_priv->flags.is_out_of_order_delivery) { + out_mb[0] = mb; + en_priv->state.rx_deliv = count + 1; + + return 1; + } + + reorder = &dl->reorder; + t_reorder = &dl->t_reorder; + + if (count == en_priv->state.rx_deliv) { + if (reorder->is_active) { + /* + * This insert used only to increment reorder->min_seqn + * To remove it - min_seqn_set() has to work with non-empty buffer + */ + pdcp_reorder_insert(reorder, mb, count); + + /* Get buffered packets */ + struct rte_mbuf **cached_mbufs = &out_mb[processed]; + uint32_t nb_cached = pdcp_reorder_get_sequential(reorder, + cached_mbufs, entity->max_pkt_cache - processed); + + processed += nb_cached; + } else { + out_mb[processed++] = mb; + } + + /* Processed should never exceed the window size */ + en_priv->state.rx_deliv = count + processed; + + } else { + if (!reorder->is_active) + /* Initialize reordering buffer with RX_DELIV */ + pdcp_reorder_start(reorder, en_priv->state.rx_deliv); + /* Buffer the packet */ + pdcp_reorder_insert(reorder, mb, count); + } + + /* Stop & reset current timer if rx_reord is received */ + if (t_reorder->state == TIMER_RUNNING && + en_priv->state.rx_deliv >= en_priv->state.rx_reord) { + t_reorder->state = TIMER_STOP; + /* Stop reorder buffer, only if it's empty */ + if (en_priv->state.rx_deliv == en_priv->state.rx_next) + pdcp_reorder_stop(reorder); + } + + /* + * If t-Reordering is not running (includes the case when t-Reordering is stopped due to + * actions above). + */ + if (t_reorder->state == TIMER_STOP && en_priv->state.rx_deliv < en_priv->state.rx_next) { + /* Update RX_REORD to RX_NEXT */ + en_priv->state.rx_reord = en_priv->state.rx_next; + /* Start t-Reordering */ + t_reorder->state = TIMER_RUNNING; + } + + return processed; } static inline uint16_t @@ -862,16 +925,12 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret, const bool is_integ_protected) { - struct entity_priv *en_priv = entity_priv_get(entity); - const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; + int i, nb_processed, nb_success = 0, nb_err = 0; rte_pdcp_dynfield_t *mb_dynfield; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -880,11 +939,12 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r mb_dynfield = pdcp_dynfield(mb); count = *mb_dynfield; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], is_integ_protected); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -918,16 +978,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret) { - struct entity_priv *en_priv = entity_priv_get(entity); - const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; + int i, nb_processed, nb_success = 0, nb_err = 0; rte_pdcp_dynfield_t *mb_dynfield; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -936,12 +992,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, mb_dynfield = pdcp_dynfield(mb); count = *mb_dynfield; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], true); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, true); - - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -1104,6 +1160,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent en_priv->flags.is_status_report_required = 1; } + /** + * flags.is_out_of_order_delivery + * + * Indicate whether the outoforder delivery is enabled for PDCP entity. + */ + en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** * hdr_sz * diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c new file mode 100644 index 0000000000..5399f0dc28 --- /dev/null +++ b/lib/pdcp/pdcp_reorder.c @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_reorder.h> + +#include "pdcp_reorder.h" + +int +pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +{ + reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + if (reorder->buf == NULL) + return -rte_errno; + + reorder->window_size = window_size; + reorder->is_active = false; + + return 0; +} + +void +pdcp_reorder_destroy(const struct pdcp_reorder *reorder) +{ + rte_reorder_free(reorder->buf); +} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h new file mode 100644 index 0000000000..6a2f61d6ae --- /dev/null +++ b/lib/pdcp/pdcp_reorder.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_REORDER_H +#define PDCP_REORDER_H + +#include <rte_reorder.h> + +struct pdcp_reorder { + struct rte_reorder_buffer *buf; + uint32_t window_size; + bool is_active; +}; + +int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); +void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); + +static inline uint32_t +pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs) +{ + return rte_reorder_drain(reorder->buf, mbufs, max_mbufs); +} + +static inline uint32_t +pdcp_reorder_up_to_get(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs, uint32_t seqn) +{ + return rte_reorder_drain_up_to_seqn(reorder->buf, mbufs, max_mbufs, seqn); +} + +static inline void +pdcp_reorder_start(struct pdcp_reorder *reorder, uint32_t min_seqn) +{ + int ret; + + reorder->is_active = true; + + ret = rte_reorder_min_seqn_set(reorder->buf, min_seqn); + RTE_VERIFY(ret == 0); +} + +static inline void +pdcp_reorder_stop(struct pdcp_reorder *reorder) +{ + reorder->is_active = false; +} + +static inline void +pdcp_reorder_insert(struct pdcp_reorder *reorder, struct rte_mbuf *mbuf, + rte_reorder_seqn_t pkt_count) +{ + int ret; + + *rte_reorder_seqn(mbuf) = pkt_count; + + ret = rte_reorder_insert(reorder->buf, mbuf); + RTE_VERIFY(ret == 0); +} + +#endif /* PDCP_REORDER_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 96ad397667..be37ff392c 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -49,6 +49,17 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } +static int +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + + entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + + return pdcp_reorder_create(&dl->reorder, window_size); +} + struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { @@ -118,6 +129,12 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + ret = pdcp_dl_establish(entity, conf); + if (ret) + goto crypto_sess_destroy; + } + ret = pdcp_cnt_ring_create(entity, conf); if (ret) goto crypto_sess_destroy; @@ -132,26 +149,50 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } +static int +pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + int nb_out; + + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, + en_priv->state.rx_next); + + pdcp_reorder_destroy(&dl->reorder); + + return nb_out; +} + int rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv *en_priv; + int nb_out = 0; + if (pdcp_entity == NULL) return -EINVAL; + en_priv = entity_priv_get(pdcp_entity); + + if (!en_priv->flags.is_ul_entity) + nb_out = pdcp_dl_release(pdcp_entity, out_mb); + /* Teardown crypto sessions */ pdcp_crypto_sess_destroy(pdcp_entity); rte_free(pdcp_entity); - RTE_SET_USED(out_mb); - return 0; + return nb_out; } int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; + int nb_out = 0; if (pdcp_entity == NULL) return -EINVAL; @@ -161,13 +202,15 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, if (en_priv->flags.is_ul_entity) { en_priv->state.tx_next = 0; } else { + dl = entity_dl_part_get(pdcp_entity); + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, pdcp_entity->max_pkt_cache, + en_priv->state.rx_next); + pdcp_reorder_stop(&dl->reorder); en_priv->state.rx_next = 0; en_priv->state.rx_deliv = 0; } - RTE_SET_USED(out_mb); - - return 0; + return nb_out; } struct rte_mbuf * diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 2a2bae1cd1..980086a93a 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -111,6 +111,8 @@ struct rte_pdcp_entity_conf { * as entity re-establishment. */ bool status_report_required; + /** Enable out of order delivery. */ + bool out_of_order_delivery; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -271,8 +273,8 @@ rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, * @param in_mb * The address of an array of *num* pointers to *rte_mbuf* structures. * @param[out] out_mb - * The address of an array of *num* pointers to *rte_mbuf* structures - * to output packets after PDCP post-processing. + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures to output packets after PDCP post-processing. * @param num * The maximum number of packets to process. * @param[out] nb_err -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 14/22] test/pdcp: add in-order delivery cases 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (12 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 15/22] pdcp: add timer callback handlers Anoob Joseph ` (9 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test cases to verify behaviour when in-order delivery is enabled and packets arrive in out-of-order. PDCP library is expected to buffer the packets and return packets in-order when the missing packet arrives. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 223 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index cfe2ec6aa9..24d7826bc2 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,15 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* Assert that condition is true, or goto the mark */ +#define ASSERT_TRUE_OR_GOTO(cond, mark, ...) do {\ + if (!(cond)) { \ + RTE_LOG(ERR, USER1, "Error at: %s:%d\n", __func__, __LINE__); \ + RTE_LOG(ERR, USER1, __VA_ARGS__); \ + goto mark; \ + } \ +} while (0) + /* According to formula(7.2.a Window_Size) */ #define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) @@ -83,6 +92,38 @@ run_test_with_all_known_vec(const void *args) return run_test_foreach_known_vec(test, false); } +static int +run_test_with_all_known_vec_until_first_pass(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, true); +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -416,6 +457,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.sess_mpool = ts_params->sess_pool; conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.ctrl_pdu_pool = ts_params->mbuf_pool; conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; @@ -868,6 +910,7 @@ test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) /* Configure Uplink to generate expected, encrypted packet */ pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.out_of_order_delivery = true; conf->entity.reverse_iv_direction = true; conf->entity.pdcp_xfrm.hfn = new_hfn; conf->entity.sn = new_sn; @@ -915,6 +958,171 @@ test_sn_minus_outside(struct pdcp_test_conf *t_conf) return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); } +static struct rte_mbuf * +generate_packet_for_dl_with_sn(struct pdcp_test_conf ul_conf, uint32_t count) +{ + enum rte_security_pdcp_sn_size sn_size = ul_conf.entity.pdcp_xfrm.sn_size; + int ret; + + ul_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(count, sn_size); + ul_conf.entity.sn = pdcp_sn_from_count_get(count, sn_size); + ul_conf.entity.out_of_order_delivery = true; + ul_conf.entity.reverse_iv_direction = true; + ul_conf.output_len = 0; + + ret = test_attempt_single(&ul_conf); + if (ret != TEST_SUCCESS) + return NULL; + + return mbuf_from_data_create(ul_conf.output, ul_conf.output_len); +} + +static bool +array_asc_sorted_check(struct rte_mbuf *m[], uint32_t len, enum rte_security_pdcp_sn_size sn_size) +{ + uint32_t i; + + if (len < 2) + return true; + + for (i = 0; i < (len - 1); i++) { + if (pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i], void *), sn_size) > + pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i + 1], void *), sn_size)) + return false; + } + + return true; +} + +static int +test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m0 = NULL, *m1 = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Generate packet to fill the existing gap */ + m0 = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m0 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1] + * Gap filled, all packets should be returned + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m0, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m0 = NULL; /* Packet was moved to out_mb */ + + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, + "Error occurred during packet drain\n"); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m0); + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + +static int +test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + struct rte_mbuf *m1 = NULL, **out_mb = NULL; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + const int rx_deliv = 0; + int ret = TEST_FAILED; + size_t i, nb_out; + uint8_t cdev_id; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + ASSERT_TRUE_OR_GOTO(pdcp_entity->max_pkt_cache >= window_size, exit, + "PDCP max packet cache is too small"); + cdev_id = dl_conf.entity.dev_id; + out_mb = rte_zmalloc(NULL, pdcp_entity->max_pkt_cache * sizeof(uintptr_t), 0); + ASSERT_TRUE_OR_GOTO(out_mb != NULL, exit, + "Could not allocate buffer for holding out_mb buffers\n"); + + /* Send packets with SN > RX_DELIV to create a gap */ + for (i = rx_deliv + 1; i < window_size; i++) { + m1 = generate_packet_for_dl_with_sn(*ul_conf, i); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + } + + m1 = generate_packet_for_dl_with_sn(*ul_conf, rx_deliv); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Insert missing packet */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == window_size, exit, + "Packet count mismatch (received: %i, expected: %i)\n", + nb_success, window_size); + m1 = NULL; + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + rte_free(out_mb); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -971,10 +1179,25 @@ static struct unit_test_suite hfn_sn_test_cases = { } }; +static struct unit_test_suite reorder_test_cases = { + .suite_name = "PDCP reorder", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_reorder_buffer_full_window_size_sn_12), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, + &reorder_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 15/22] pdcp: add timer callback handlers 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (13 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 14/22] test/pdcp: add in-order delivery cases Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 16/22] pdcp: add timer expiry handle Anoob Joseph ` (8 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> PDCP has a windowing mechanism which allows only packets that fall in a reception window. The pivot point for this window is RX_REORD which happens to be the first missing or next expected packet. If the missing packet is not received after a specified time, then the RX_REORD state variable needs to be moved up to slide the reception window. PDCP relies on timers for such operations. The timer needs to be armed when PDCP library doesn't receive all packets in-order and starts buffering packets that arrived after a missing packet. The timer needs to be cancelled when a missing packet is received. To avoid dependency on particular timer implementation, PDCP library allows application to register two callbacks, timer_start() and timer_stop() that will be called later by library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 ++ lib/pdcp/pdcp_process.c | 2 ++ lib/pdcp/rte_pdcp.c | 1 + lib/pdcp/rte_pdcp.h | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 52 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 34341cdc11..efc74ba9b9 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -120,6 +120,8 @@ enum timer_state { struct pdcp_t_reordering { /** Represent timer state */ enum timer_state state; + /** User defined callback handles */ + struct rte_pdcp_t_reordering handle; }; struct pdcp_cnt_bitmap { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index d910cb7c03..4fbfc1cfb5 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -901,6 +901,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (t_reorder->state == TIMER_RUNNING && en_priv->state.rx_deliv >= en_priv->state.rx_reord) { t_reorder->state = TIMER_STOP; + t_reorder->handle.stop(t_reorder->handle.timer, t_reorder->handle.args); /* Stop reorder buffer, only if it's empty */ if (en_priv->state.rx_deliv == en_priv->state.rx_next) pdcp_reorder_stop(reorder); @@ -915,6 +916,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, en_priv->state.rx_reord = en_priv->state.rx_next; /* Start t-Reordering */ t_reorder->state = TIMER_RUNNING; + t_reorder->handle.start(t_reorder->handle.timer, t_reorder->handle.args); } return processed; diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index be37ff392c..a0558b99ae 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -56,6 +56,7 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct entity_priv_dl_part *dl = entity_dl_part_get(entity); entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + dl->t_reorder.handle = conf->t_reordering; return pdcp_reorder_create(&dl->reorder, window_size); } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 980086a93a..05c922819e 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -67,6 +67,51 @@ struct rte_pdcp_entity { uint32_t max_pkt_cache; } __rte_cache_aligned; +/** + * Callback function type for t-Reordering timer start, set during PDCP entity establish. + * This callback is invoked by PDCP library, during t-Reordering timer start event. + * Only one t-Reordering per receiving PDCP entity would be running at a given time. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_start_cb_t)(void *timer, void *args); + +/** + * Callback function type for t-Reordering timer stop, set during PDCP entity establish. + * This callback will be invoked by PDCP library, during t-Reordering timer stop event. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); + +/** + * PDCP t-Reordering timer interface + * + * Configuration provided by user, that PDCP library will invoke according to timer behaviour. + */ +struct rte_pdcp_t_reordering { + /** Timer pointer, stored for later use in callback functions */ + void *timer; + /** Timer arguments, stored for later use in callback functions */ + void *args; + /** Timer start callback handle */ + rte_pdcp_t_reordering_start_cb_t start; + /** Timer stop callback handle */ + rte_pdcp_t_reordering_stop_cb_t stop; +}; + /** * PDCP entity configuration to be used for establishing an entity. */ @@ -113,6 +158,8 @@ struct rte_pdcp_entity_conf { bool status_report_required; /** Enable out of order delivery. */ bool out_of_order_delivery; + /** t-Reordering timer configuration */ + struct rte_pdcp_t_reordering t_reordering; }; /* >8 End of structure rte_pdcp_entity_conf. */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 16/22] pdcp: add timer expiry handle 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (14 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 15/22] pdcp: add timer callback handlers Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 17/22] test/pdcp: add timer expiry cases Anoob Joseph ` (7 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> The PDCP protocol requires usage of timers to keep track of how long an out-of-order packet should be buffered while waiting for missing packets. Applications can register a desired timer implementation with the PDCP library. Once the timer expires, the application will be notified, and further handling of the event will be performed in the PDCP library. When the timer expires, the PDCP library will return the cached packets, and PDCP internal state variables (like RX_REORD, RX_DELIV etc) will be updated accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 30 ++++++++++++++++++ lib/pdcp/rte_pdcp.c | 49 ++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 31 +++++++++++++++++++ lib/pdcp/version.map | 2 ++ 4 files changed, 112 insertions(+) diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index dcb424bb1d..16deaead15 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -130,6 +130,36 @@ Supported integrity protection algorithms - SNOW3G-AUTH - ZUC-AUTH +Timers +------ + +PDCP utilizes a reception window mechanism to limit the bits of COUNT value +transmitted in the packet. It utilizes state variables such as RX_REORD, +RX_DELIV to define the window and uses RX_DELIV as the lower pivot point of the +window. + +RX_DELIV would be updated only when packets are received in-order. Any missing +packet would mean RX_DELIV won't be updated. A timer, t-Reordering, helps PDCP +to slide the window if the missing packet is not received in a specified time +duration. + +While starting and stopping the timer need to be done by lib PDCP, application +could register its own timer implementation. This is to make sure application +can choose between timers such as rte_timer and rte_event based timers. Starting +and stopping of timer would happen during pre & post process API. + +When the t-Reordering timer expires, application would receive the expiry event. +To perform the PDCP handling of the expiry event, +``rte_pdcp_t_reordering_expiry_handle`` can be used. Expiry handling would +involve sliding the window by updating state variables and passing the expired +packets to the application. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_t_reordering 8< + :end-before: >8 End of structure rte_pdcp_t_reordering. + + Sample API usage ---------------- diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index a0558b99ae..819c66bd08 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -251,3 +251,52 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, return m; } + +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t capacity = entity->max_pkt_cache; + uint16_t nb_out, nb_seq; + + /* 5.2.2.2 Actions when a t-Reordering expires */ + + /* + * - deliver to upper layers in ascending order of the associated COUNT value after + * performing header decompression, if not decompressed before: + */ + + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); + capacity -= nb_out; + out_mb = &out_mb[nb_out]; + + /* + * - all stored PDCP SDU(s) with consecutively associated COUNT value(s) starting from + * RX_REORD; + */ + nb_seq = pdcp_reorder_get_sequential(&dl->reorder, out_mb, capacity); + nb_out += nb_seq; + + /* + * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered + * to upper layers, with COUNT value >= RX_REORD; + */ + en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + + /* + * - if RX_DELIV < RX_NEXT: + * - update RX_REORD to RX_NEXT; + * - start t-Reordering. + */ + if (en_priv->state.rx_deliv < en_priv->state.rx_next) { + en_priv->state.rx_reord = en_priv->state.rx_next; + dl->t_reorder.state = TIMER_RUNNING; + dl->t_reorder.handle.start(dl->t_reorder.handle.timer, dl->t_reorder.handle.args); + } else { + dl->t_reorder.state = TIMER_EXPIRED; + } + + return nb_out; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 05c922819e..b926b0df29 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -101,6 +101,7 @@ typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); * * Configuration provided by user, that PDCP library will invoke according to timer behaviour. */ +/* Structure rte_pdcp_t_reordering 8< */ struct rte_pdcp_t_reordering { /** Timer pointer, stored for later use in callback functions */ void *timer; @@ -111,6 +112,7 @@ struct rte_pdcp_t_reordering { /** Timer stop callback handle */ rte_pdcp_t_reordering_stop_cb_t stop; }; +/* >8 End of structure rte_pdcp_t_reordering. */ /** * PDCP entity configuration to be used for establishing an entity. @@ -339,6 +341,35 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.2.2.2 Actions when a t-Reordering expires + * + * When t-Reordering timer expires, PDCP is required to slide the reception + * window by updating state variables such as RX_REORD & RX_DELIV. PDCP would + * need to deliver some of the buffered packets based on the state variables and + * conditions described. + * + * The expiry handle need to be invoked by the application when t-Reordering + * timer expires. In addition to returning buffered packets, it may also restart + * timer based on the state variables. + * + * @param entity + * Pointer to the *rte_pdcp_entity* for which the timer expired. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. Used to return buffered packets that are + * expired. + * @return + * Number of packets returned in *out_mb* buffer. + */ +__rte_experimental +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, + struct rte_mbuf *out_mb[]); + /** * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. So include * in the end. diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 97171f902e..4fd912fac0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -14,5 +14,7 @@ EXPERIMENTAL { rte_pdcp_pkt_pre_process; rte_pdcp_pkt_crypto_group; + rte_pdcp_t_reordering_expiry_handle; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 17/22] test/pdcp: add timer expiry cases 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (15 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 16/22] pdcp: add timer expiry handle Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 18/22] test/pdcp: add timer restart case Anoob Joseph ` (6 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test cases for handling the expiry with rte_timer and rte_event_timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 350 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 350 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 24d7826bc2..25729b2bdd 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -3,15 +3,24 @@ */ #include <rte_errno.h> +#ifdef RTE_LIB_EVENTDEV +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#endif /* RTE_LIB_EVENTDEV */ #include <rte_malloc.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include <rte_timer.h> #include "test.h" #include "test_cryptodev.h" #include "test_cryptodev_security_pdcp_test_vectors.h" +#define NSECPERSEC 1E9 #define NB_DESC 1024 +#define TIMER_ADAPTER_ID 0 +#define TEST_EV_QUEUE_ID 0 +#define TEST_EV_PORT_ID 0 #define CDEV_INVALID_ID UINT8_MAX #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 @@ -33,10 +42,21 @@ struct pdcp_testsuite_params { struct rte_mempool *cop_pool; struct rte_mempool *sess_pool; bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; + int evdev; +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev; +#endif /* RTE_LIB_EVENTDEV */ + bool timer_is_running; + uint64_t min_resolution_ns; }; static struct pdcp_testsuite_params testsuite_params; +struct test_rte_timer_args { + int status; + struct rte_pdcp_entity *pdcp_entity; +}; + struct pdcp_test_conf { struct rte_pdcp_entity_conf entity; struct rte_crypto_sym_xform c_xfrm; @@ -124,6 +144,30 @@ pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; } +static void +pdcp_timer_start_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = true; +} + +static void +pdcp_timer_stop_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = false; +} + +static struct rte_pdcp_t_reordering t_reorder_timer = { + .timer = &testsuite_params.timer_is_running, + .start = pdcp_timer_start_cb, + .stop = pdcp_timer_stop_cb, +}; + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -462,6 +506,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + conf->entity.t_reordering = t_reorder_timer; if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; @@ -1048,6 +1093,8 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) /* Check that packets in correct order */ ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == false, exit, + "Timer should be stopped after full drain\n"); ret = TEST_SUCCESS; exit: @@ -1123,6 +1170,181 @@ test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static void +event_timer_start_cb(void *timer, void *args) +{ + struct rte_event_timer *evtims = args; + int ret = 0; + + ret = rte_event_timer_arm_burst(timer, &evtims, 1); + assert(ret == 1); +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +test_expiry_with_event_timer(const struct pdcp_test_conf *ul_conf) +{ +#ifdef RTE_LIB_EVENTDEV + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_event event; + + const int start_count = 0; + struct rte_event_timer evtim = { + .ev.op = RTE_EVENT_OP_NEW, + .ev.queue_id = TEST_EV_QUEUE_ID, + .ev.sched_type = RTE_SCHED_TYPE_ATOMIC, + .ev.priority = RTE_EVENT_DEV_PRIORITY_NORMAL, + .ev.event_type = RTE_EVENT_TYPE_TIMER, + .state = RTE_EVENT_TIMER_NOT_ARMED, + .timeout_ticks = 1, + }; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &evtim; + dl_conf.entity.t_reordering.timer = testsuite_params.timdev; + dl_conf.entity.t_reordering.start = event_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + evtim.ev.event_ptr = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + while (n != 1) { + rte_delay_us(testsuite_params.min_resolution_ns / 1000); + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, + "Dequeued unexpected timer expiry event: %i\n", n); + } + + ASSERT_TRUE_OR_GOTO(event.event_type == RTE_EVENT_TYPE_TIMER, exit, "Unexpected event type\n"); + + /* Handle expiry event */ + n = rte_pdcp_t_reordering_expiry_handle(event.event_ptr, out_mb); + ASSERT_TRUE_OR_GOTO(n == 1, exit, "Unexpected number of expired packets :%i\n", n); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +#else + RTE_SET_USED(ul_conf); + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +test_rte_timer_expiry_handle(struct rte_timer *timer_handle, void *arg) +{ + struct test_rte_timer_args *timer_data = arg; + struct rte_mbuf *out_mb[1] = {0}; + uint16_t n; + + RTE_SET_USED(timer_handle); + + n = rte_pdcp_t_reordering_expiry_handle(timer_data->pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, n); + + timer_data->status = n == 1 ? n : -1; +} + +static void +test_rte_timer_start_cb(void *timer, void *args) +{ + rte_timer_reset_sync(timer, 1, SINGLE, rte_lcore_id(), test_rte_timer_expiry_handle, args); +} + +static int +test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct test_rte_timer_args timer_args; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_timer timer = {0}; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Set up a timer */ + rte_timer_init(&timer); + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &timer_args; + dl_conf.entity.t_reordering.timer = &timer; + dl_conf.entity.t_reordering.start = test_rte_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + timer_args.status = 0; + timer_args.pdcp_entity = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Verify that expire was handled correctly */ + rte_timer_manage(); + while (timer_args.status != 1) { + rte_delay_us(1); + rte_timer_manage(); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, "Bad expire handle status %i\n", + timer_args.status); + } + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1145,6 +1367,126 @@ test_combined(struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static inline void +eventdev_conf_default_set(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info) +{ + memset(dev_conf, 0, sizeof(struct rte_event_dev_config)); + dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns; + dev_conf->nb_event_ports = 1; + dev_conf->nb_event_queues = 1; + dev_conf->nb_event_queue_flows = info->max_event_queue_flows; + dev_conf->nb_event_port_dequeue_depth = info->max_event_port_dequeue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_events_limit = info->max_num_events; +} + +static inline int +eventdev_setup(void) +{ + struct rte_event_dev_config dev_conf; + struct rte_event_dev_info info; + int ret, evdev = 0; + + if (!rte_event_dev_count()) + return TEST_SKIPPED; + + ret = rte_event_dev_info_get(evdev, &info); + TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info"); + TEST_ASSERT(info.max_num_events < 0 || info.max_num_events >= 1, + "ERROR max_num_events=%d < max_events=%d", info.max_num_events, 1); + + eventdev_conf_default_set(&dev_conf, &info); + ret = rte_event_dev_configure(evdev, &dev_conf); + TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev"); + + ret = rte_event_queue_setup(evdev, TEST_EV_QUEUE_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d", TEST_EV_QUEUE_ID); + + /* Configure event port */ + ret = rte_event_port_setup(evdev, TEST_EV_PORT_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d", TEST_EV_PORT_ID); + ret = rte_event_port_link(evdev, TEST_EV_PORT_ID, NULL, NULL, 0); + TEST_ASSERT(ret >= 0, "Failed to link all queues port=%d", TEST_EV_PORT_ID); + + ret = rte_event_dev_start(evdev); + TEST_ASSERT_SUCCESS(ret, "Failed to start device"); + + testsuite_params.evdev = evdev; + + return TEST_SUCCESS; +} + +static int +event_timer_setup(void) +{ + struct rte_event_timer_adapter_info info; + struct rte_event_timer_adapter *timdev; + uint32_t caps = 0; + + struct rte_event_timer_adapter_conf config = { + .event_dev_id = testsuite_params.evdev, + .timer_adapter_id = TIMER_ADAPTER_ID, + .timer_tick_ns = NSECPERSEC, + .max_tmo_ns = 10 * NSECPERSEC, + .nb_timers = 10, + .flags = 0, + }; + + TEST_ASSERT_SUCCESS(rte_event_timer_adapter_caps_get(testsuite_params.evdev, &caps), + "Failed to get adapter capabilities"); + + if (!(caps & RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) + return TEST_SKIPPED; + + timdev = rte_event_timer_adapter_create(&config); + + TEST_ASSERT_NOT_NULL(timdev, "Failed to create event timer ring"); + + testsuite_params.timdev = timdev; + + TEST_ASSERT_EQUAL(rte_event_timer_adapter_start(timdev), 0, + "Failed to start event timer adapter"); + + rte_event_timer_adapter_get_info(timdev, &info); + testsuite_params.min_resolution_ns = info.min_resolution_ns; + + return TEST_SUCCESS; +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +ut_setup_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + int ret; + + ret = eventdev_setup(); + if (ret) + return ret; + + return event_timer_setup(); +#else + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +ut_teardown_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev = testsuite_params.timdev; + int evdev = testsuite_params.evdev; + + rte_event_dev_stop(evdev); + rte_event_dev_close(evdev); + + rte_event_timer_adapter_stop(timdev); + rte_event_timer_adapter_free(timdev); +#endif /* RTE_LIB_EVENTDEV */ +} + static int run_test_for_one_known_vec(const void *arg) { @@ -1189,6 +1531,14 @@ static struct unit_test_suite reorder_test_cases = { ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, test_reorder_buffer_full_window_size_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_event_timer", + ut_setup_pdcp_event_timer, ut_teardown_pdcp_event_timer, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_event_timer), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_rte_timer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_rte_timer), TEST_CASES_END() /**< NULL terminate unit test array */ } }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 18/22] test/pdcp: add timer restart case 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (16 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 17/22] test/pdcp: add timer expiry cases Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 19/22] pdcp: add support for status report Anoob Joseph ` (5 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test to cover the case when t-reordering timer should be restarted on the same packet. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 68 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 25729b2bdd..82cc25ec7a 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -1106,6 +1106,71 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) return ret; } +static int +test_reorder_gap_in_reorder_buffer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + int ret = TEST_FAILED, nb_out, i; + struct pdcp_test_conf dl_conf; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Create two gaps [NULL, m1, NULL, m3]*/ + for (i = 0; i < 2; i++) { + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + 2 * i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m = NULL; /* Packet was moved to PDCP lib */ + } + + /* Generate packet to fill the first gap */ + m = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1, NULL, m3] + * Only first gap should be filled, timer should be restarted for second gap + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m = NULL; + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), + exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == true, exit, + "Timer should be restarted after partial drain"); + + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) { @@ -1527,6 +1592,9 @@ static struct unit_test_suite reorder_test_cases = { TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_in_reorder_buffer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_in_reorder_buffer), TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 19/22] pdcp: add support for status report 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (17 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 18/22] test/pdcp: add timer restart case Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph ` (4 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Implement status report generation for PDCP entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 158 ++++++++++++++++++++++++++++++++++++--- lib/pdcp/pdcp_cnt.h | 11 ++- lib/pdcp/pdcp_ctrl_pdu.c | 34 ++++++++- lib/pdcp/pdcp_ctrl_pdu.h | 3 +- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 9 ++- lib/pdcp/pdcp_process.h | 13 ++++ lib/pdcp/rte_pdcp.c | 34 ++++++--- 8 files changed, 236 insertions(+), 28 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index c9b952184b..af027b00d3 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -2,28 +2,164 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_pdcp.h> #include "pdcp_cnt.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#define SLAB_BYTE_SIZE (RTE_BITMAP_SLAB_BIT_SIZE / 8) + +uint32_t +pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) +{ + uint32_t n_bits = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + return rte_bitmap_get_memory_footprint(n_bits); +} + int -pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) { - struct entity_priv_dl_part *en_priv_dl; - uint32_t window_sz; + uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - if (en == NULL || conf == NULL) + dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + if (dl->bitmap.bmp == NULL) return -EINVAL; - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) - return 0; + dl->bitmap.size = window_size; - en_priv_dl = entity_dl_part_get(en); - window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + return 0; +} - RTE_SET_USED(window_sz); - RTE_SET_USED(en_priv_dl); +void +pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + rte_bitmap_set(bitmap.bmp, count % bitmap.size); +} - return 0; +bool +pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + return rte_bitmap_get(bitmap.bmp, count % bitmap.size); +} + +void +pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop) +{ + uint32_t i; + + for (i = start; i < stop; i++) + rte_bitmap_clear(bitmap.bmp, i % bitmap.size); +} + +uint16_t +pdcp_cnt_get_bitmap_size(uint32_t pending_bytes) +{ + /* + * Round up bitmap size to slab size to operate only on slabs sizes, instead of individual + * bytes + */ + return RTE_ALIGN_MUL_CEIL(pending_bytes, SLAB_BYTE_SIZE); +} + +static __rte_always_inline uint64_t +leftover_get(uint64_t slab, uint32_t shift, uint64_t mask) +{ + return (slab & mask) << shift; +} + +void +pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len) +{ + uint64_t slab = 0, next_slab = 0, leftover; + uint32_t zeros, report_len, diff; + uint32_t slab_id, next_slab_id; + uint32_t pos = 0, next_pos = 0; + + const uint32_t start_count = state.rx_deliv + 1; + const uint32_t nb_slabs = bitmap.size / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t nb_data_slabs = data_len / SLAB_BYTE_SIZE; + const uint32_t start_slab_id = start_count / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t stop_slab_id = (start_slab_id + nb_data_slabs) % nb_slabs; + const uint32_t shift = start_count % RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t leftover_shift = shift ? RTE_BITMAP_SLAB_BIT_SIZE - shift : 0; + const uint8_t *data_end = RTE_PTR_ADD(data, data_len + SLAB_BYTE_SIZE); + + /* NOTE: Mask required to workaround case - when shift is not needed */ + const uint64_t leftover_mask = shift ? ~0 : 0; + + /* NOTE: implement scan init at to set custom position */ + __rte_bitmap_scan_init(bitmap.bmp); + while (true) { + assert(rte_bitmap_scan(bitmap.bmp, &pos, &slab) == 1); + slab_id = pos / RTE_BITMAP_SLAB_BIT_SIZE; + if (slab_id >= start_slab_id) + break; + } + + report_len = nb_data_slabs; + + if (slab_id > start_slab_id) { + /* Zero slabs at beginning */ + zeros = (slab_id - start_slab_id - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + leftover = leftover_get(slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + report_len -= (slab_id - start_slab_id); + } + + while (report_len) { + rte_bitmap_scan(bitmap.bmp, &next_pos, &next_slab); + next_slab_id = next_pos / RTE_BITMAP_SLAB_BIT_SIZE; + diff = (next_slab_id + nb_slabs - slab_id) % nb_slabs; + + /* If next_slab_id == slab_id - overlap */ + diff += !(next_slab_id ^ slab_id) * nb_slabs; + + /* Size check - next slab is outsize of size range */ + if (diff > report_len) { + next_slab = 0; + next_slab_id = stop_slab_id; + diff = report_len; + } + + report_len -= diff; + + /* Calculate gap between slabs, taking wrap around into account */ + zeros = (next_slab_id + nb_slabs - slab_id - 1) % nb_slabs; + if (zeros) { + /* Non continues slabs, align them individually */ + slab >>= shift; + memcpy(data, &slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + + /* Fill zeros between slabs */ + zeros = (zeros - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + + /* Align beginning of next slab */ + leftover = leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } else { + /* Continues slabs, combine them */ + uint64_t new_slab = (slab >> shift) | + leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &new_slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } + + slab = next_slab; + pos = next_pos; + slab_id = next_slab_id; + + }; + + assert(data < data_end); } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index bbda478b55..5941b7a406 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -9,6 +9,15 @@ #include "pdcp_entity.h" -int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); +uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); + +void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +void pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop); + +uint16_t pdcp_cnt_get_bitmap_size(uint32_t pending_bytes); +void pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len); #endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c index feb05fd863..e0ac2d3720 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.c +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -8,6 +8,14 @@ #include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#include "pdcp_cnt.h" + +static inline uint16_t +round_up_bits(uint32_t bits) +{ + /* round up to the next multiple of 8 */ + return RTE_ALIGN_MUL_CEIL(bits, 8) / 8; +} static __rte_always_inline void pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) @@ -19,11 +27,13 @@ pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) } int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m) { struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; - uint32_t rx_deliv; - int pdu_sz; + uint32_t rx_deliv, actual_sz; + uint16_t pdu_sz, bitmap_sz; + uint8_t *data; if (!en_priv->flags.is_status_report_required) return -EINVAL; @@ -42,5 +52,21 @@ pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) return 0; } - return -ENOTSUP; + actual_sz = RTE_MIN(round_up_bits(en_priv->state.rx_next - rx_deliv - 1), + RTE_PDCP_CTRL_PDU_SIZE_MAX - pdu_sz); + bitmap_sz = pdcp_cnt_get_bitmap_size(actual_sz); + + data = (uint8_t *)rte_pktmbuf_append(m, pdu_sz + bitmap_sz); + if (data == NULL) + return -ENOMEM; + + m->pkt_len = pdu_sz + actual_sz; + m->data_len = pdu_sz + actual_sz; + + pdcp_hdr_fill((struct rte_pdcp_up_ctrl_pdu_hdr *)data, rx_deliv); + + data = RTE_PTR_ADD(data, pdu_sz); + pdcp_cnt_report_fill(dl->bitmap, en_priv->state, data, bitmap_sz); + + return 0; } diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h index a2424fbd10..2a87928b88 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.h +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -10,6 +10,7 @@ #include "pdcp_entity.h" int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m); #endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index efc74ba9b9..a9b1428c7a 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -182,6 +182,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /** Bitmap memory region */ + uint8_t bitmap_mem[0]; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 4fbfc1cfb5..c093b78e10 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -9,6 +9,7 @@ #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -856,11 +857,15 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; + if (unlikely(pdcp_cnt_bitmap_is_set(dl->bitmap, count))) + return -EEXIST; + + pdcp_cnt_bitmap_set(dl->bitmap, count); pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); if (en_priv->flags.is_out_of_order_delivery) { out_mb[0] = mb; - en_priv->state.rx_deliv = count + 1; + pdcp_rx_deliv_set(entity, count + 1); return 1; } @@ -887,7 +892,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, } /* Processed should never exceed the window size */ - en_priv->state.rx_deliv = count + processed; + pdcp_rx_deliv_set(entity, count + processed); } else { if (!reorder->is_active) diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index c270f15caf..3ed904b2df 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -7,6 +7,9 @@ #include <rte_pdcp.h> +#include <pdcp_entity.h> +#include <pdcp_cnt.h> + typedef uint32_t rte_pdcp_dynfield_t; extern int rte_pdcp_dynfield_offset; @@ -20,4 +23,14 @@ pdcp_dynfield(struct rte_mbuf *mbuf) int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); +static inline void +pdcp_rx_deliv_set(const struct rte_pdcp_entity *entity, uint32_t rx_deliv) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + pdcp_cnt_bitmap_range_clear(dl->bitmap, en_priv->state.rx_deliv, rx_deliv); + en_priv->state.rx_deliv = rx_deliv; +} + #endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 819c66bd08..9865c620b7 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -14,6 +14,8 @@ #define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" +static int bitmap_mem_offset; + int rte_pdcp_dynfield_offset = -1; static int @@ -39,9 +41,12 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); - else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size = RTE_CACHE_LINE_ROUNDUP(size); + bitmap_mem_offset = size; + size += pdcp_cnt_bitmap_get_memory_footprint(conf); + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; @@ -54,11 +59,24 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + void *bitmap_mem; + int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - return pdcp_reorder_create(&dl->reorder, window_size); + ret = pdcp_reorder_create(&dl->reorder, window_size); + if (ret) + return ret; + + bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); + ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); + if (ret) { + pdcp_reorder_destroy(&dl->reorder); + return ret; + } + + return 0; } struct rte_pdcp_entity * @@ -136,10 +154,6 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; } - ret = pdcp_cnt_ring_create(entity, conf); - if (ret) - goto crypto_sess_destroy; - return entity; crypto_sess_destroy: @@ -218,6 +232,7 @@ struct rte_mbuf * rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, enum rte_pdcp_ctrl_pdu_type type) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; struct rte_mbuf *m; int ret; @@ -228,6 +243,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, } en_priv = entity_priv_get(pdcp_entity); + dl = entity_dl_part_get(pdcp_entity); m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); if (m == NULL) { @@ -237,7 +253,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, switch (type) { case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: - ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + ret = pdcp_ctrl_pdu_status_gen(en_priv, dl, m); break; default: ret = -ENOTSUP; @@ -283,7 +299,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered * to upper layers, with COUNT value >= RX_REORD; */ - en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + pdcp_rx_deliv_set(entity, en_priv->state.rx_reord + nb_seq); /* * - if RX_DELIV < RX_NEXT: -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 20/22] pdcp: allocate reorder buffer alongside with entity 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (18 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 19/22] pdcp: add support for status report Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 21/22] pdcp: add thread safe processing Anoob Joseph ` (3 subsequent siblings) 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Instead of allocating reorder buffer separately on heap, allocate memory for it together with rest of entity, and then only initialize buffer via `rte_reorder_init()`. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 9 +++---- lib/pdcp/pdcp_cnt.h | 3 ++- lib/pdcp/pdcp_entity.h | 2 +- lib/pdcp/pdcp_reorder.c | 11 ++------ lib/pdcp/pdcp_reorder.h | 12 ++++++--- lib/pdcp/rte_pdcp.c | 58 ++++++++++++++++++++++++++--------------- 6 files changed, 55 insertions(+), 40 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index af027b00d3..e1d0634b4d 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -20,15 +20,14 @@ pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) } int -pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size) { - uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - - dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + dl->bitmap.bmp = rte_bitmap_init(nb_elem, bitmap_mem, mem_size); if (dl->bitmap.bmp == NULL) return -EINVAL; - dl->bitmap.size = window_size; + dl->bitmap.size = nb_elem; return 0; } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index 5941b7a406..87b011f9dc 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -10,7 +10,8 @@ #include "pdcp_entity.h" uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); -int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size); void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index a9b1428c7a..9f74b5d0e5 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -132,7 +132,7 @@ struct pdcp_cnt_bitmap { }; /* - * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] [reorder/bitmap] */ struct entity_priv { diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c index 5399f0dc28..bc45f2e19b 100644 --- a/lib/pdcp/pdcp_reorder.c +++ b/lib/pdcp/pdcp_reorder.c @@ -8,20 +8,13 @@ #include "pdcp_reorder.h" int -pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size) { - reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + reorder->buf = rte_reorder_init(mem, mem_size, "reorder_buffer", nb_elem); if (reorder->buf == NULL) return -rte_errno; - reorder->window_size = window_size; reorder->is_active = false; return 0; } - -void -pdcp_reorder_destroy(const struct pdcp_reorder *reorder) -{ - rte_reorder_free(reorder->buf); -} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h index 6a2f61d6ae..7e4f079d4b 100644 --- a/lib/pdcp/pdcp_reorder.h +++ b/lib/pdcp/pdcp_reorder.h @@ -9,12 +9,18 @@ struct pdcp_reorder { struct rte_reorder_buffer *buf; - uint32_t window_size; bool is_active; }; -int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); -void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); +int pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size); + +/* NOTE: replace with `rte_reorder_memory_footprint_get` after DPDK 23.07 */ +#define SIZE_OF_REORDER_BUFFER (4 * RTE_CACHE_LINE_SIZE) +static inline size_t +pdcp_reorder_memory_footprint_get(size_t nb_elem) +{ + return SIZE_OF_REORDER_BUFFER + (2 * nb_elem * sizeof(struct rte_mbuf *)); +} static inline uint32_t pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 9865c620b7..1c6d2466b2 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -14,7 +14,15 @@ #define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" -static int bitmap_mem_offset; +struct entity_layout { + size_t bitmap_offset; + size_t bitmap_size; + + size_t reorder_buf_offset; + size_t reorder_buf_size; + + size_t total_size; +}; int rte_pdcp_dynfield_offset = -1; @@ -35,46 +43,54 @@ pdcp_dynfield_register(void) } static int -pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +pdcp_entity_layout_get(const struct rte_pdcp_entity_conf *conf, struct entity_layout *layout) { - int size; + size_t size; + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); + /* Bitmap require memory to be cache aligned */ size = RTE_CACHE_LINE_ROUNDUP(size); - bitmap_mem_offset = size; - size += pdcp_cnt_bitmap_get_memory_footprint(conf); + layout->bitmap_offset = size; + layout->bitmap_size = pdcp_cnt_bitmap_get_memory_footprint(conf); + size += layout->bitmap_size; + layout->reorder_buf_offset = size; + layout->reorder_buf_size = pdcp_reorder_memory_footprint_get(window_size); + size += layout->reorder_buf_size; } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; - return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); + layout->total_size = size; + + return 0; } static int -pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf, + const struct entity_layout *layout) { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); - void *bitmap_mem; + void *memory; int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - ret = pdcp_reorder_create(&dl->reorder, window_size); + memory = RTE_PTR_ADD(entity, layout->reorder_buf_offset); + ret = pdcp_reorder_create(&dl->reorder, window_size, memory, layout->reorder_buf_size); if (ret) return ret; - bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); - ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); - if (ret) { - pdcp_reorder_destroy(&dl->reorder); + memory = RTE_PTR_ADD(entity, layout->bitmap_offset); + ret = pdcp_cnt_bitmap_create(dl, window_size, memory, layout->bitmap_size); + if (ret) return ret; - } return 0; } @@ -82,10 +98,11 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { + struct entity_layout entity_layout = { 0 }; struct rte_pdcp_entity *entity = NULL; struct entity_priv *en_priv; - int ret, entity_size; uint32_t count; + int ret; if (pdcp_dynfield_register() < 0) return NULL; @@ -118,13 +135,14 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } - entity_size = pdcp_entity_size_get(conf); - if (entity_size < 0) { + ret = pdcp_entity_layout_get(conf, &entity_layout); + if (ret < 0) { rte_errno = EINVAL; return NULL; } - entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + entity = rte_zmalloc_socket("pdcp_entity", entity_layout.total_size, RTE_CACHE_LINE_SIZE, + SOCKET_ID_ANY); if (entity == NULL) { rte_errno = ENOMEM; return NULL; @@ -149,7 +167,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { - ret = pdcp_dl_establish(entity, conf); + ret = pdcp_dl_establish(entity, conf, &entity_layout); if (ret) goto crypto_sess_destroy; } @@ -174,8 +192,6 @@ pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, en_priv->state.rx_next); - pdcp_reorder_destroy(&dl->reorder); - return nb_out; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 21/22] pdcp: add thread safe processing 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (19 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-26 22:11 ` Stephen Hemminger 2023-05-26 22:15 ` Stephen Hemminger 2023-05-26 21:02 ` [PATCH v4 22/22] test/pdcp: add PDCP status report cases Anoob Joseph ` (2 subsequent siblings) 23 siblings, 2 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> PDCP state has to be guarded for: - Uplink pre_process: - tx_next atomic increment - Downlink pre_process: - rx_deliv - read - Downlink post_process: - rx_deliv, rx_reorder, rx_next - read/write - bitmask/reorder buffer - read/write When application requires thread safe processing, the state variables need to be updated atomically. Add config option to select this option per entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 2 + lib/pdcp/pdcp_entity.h | 64 ++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 117 +++++++++++++++++++++++++++++++++------- lib/pdcp/rte_pdcp.c | 9 ++++ lib/pdcp/rte_pdcp.h | 2 + 5 files changed, 175 insertions(+), 19 deletions(-) diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index f4f9246bcb..c4f135778e 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -19,3 +19,5 @@ headers = files('rte_pdcp.h') indirect_headers += files('rte_pdcp_group.h') deps += ['mbuf', 'net', 'cryptodev', 'security', 'reorder'] + +annotate_locks = false diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 9f74b5d0e5..e88af4c3b3 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -9,6 +9,7 @@ #include <rte_crypto_sym.h> #include <rte_mempool.h> #include <rte_pdcp.h> +#include <rte_rwlock.h> #include <rte_security.h> #include "pdcp_reorder.h" @@ -162,6 +163,8 @@ struct entity_priv { uint64_t is_status_report_required : 1; /** Is out-of-order delivery enabled */ uint64_t is_out_of_order_delivery : 1; + /** Is thread safety enabled. */ + uint64_t is_thread_safety_enabled : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -182,6 +185,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /* Lock to protect concurrent updates */ + rte_rwlock_t rwl; /** Bitmap memory region */ uint8_t bitmap_mem[0]; }; @@ -257,4 +262,63 @@ pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) return (1 << (32 - sn_size)) - 1; } +static inline uint32_t +pdcp_atomic_inc(uint32_t *val, const bool mt_safe) +{ + if (mt_safe) + return __atomic_fetch_add(val, 1, __ATOMIC_RELAXED); + else + return (*val)++; +} + +static inline void +pdcp_lock_init(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (en_priv->flags.is_thread_safety_enabled) + rte_rwlock_init(&dl->rwl); +} + +static inline void +pdcp_read_lock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (en_priv->flags.is_thread_safety_enabled) + rte_rwlock_read_lock(&dl->rwl); +} + +static inline void +pdcp_read_unlock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (en_priv->flags.is_thread_safety_enabled) + rte_rwlock_read_unlock(&dl->rwl); +} + +static inline void +pdcp_write_lock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (en_priv->flags.is_thread_safety_enabled) + rte_rwlock_write_lock(&dl->rwl); +} + +static inline void +pdcp_write_unlock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (en_priv->flags.is_thread_safety_enabled) + rte_rwlock_write_unlock(&dl->rwl); +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index c093b78e10..aa13801d5d 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -357,7 +357,7 @@ cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_c static inline bool pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, - uint32_t *count) + uint32_t *count, const bool mt_safe) { struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; const uint8_t hdr_sz = en_priv->hdr_sz; @@ -369,7 +369,7 @@ pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_ return false; /* Update sequence num in the PDU header */ - *count = en_priv->state.tx_next++; + *count = pdcp_atomic_inc(&en_priv->state.tx_next, mt_safe); sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; @@ -379,9 +379,11 @@ pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_ return true; } -static uint16_t -pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], - struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +static inline uint16_t +pdcp_pre_process_uplane_sn_12_ul_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool mt_safe) { struct entity_priv *en_priv = entity_priv_get(entity); uint16_t nb_cop, nb_prep = 0, nb_err = 0; @@ -410,7 +412,7 @@ pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rt memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, - &count))) { + &count, mt_safe))) { in_mb[nb_err++] = mb; continue; } @@ -421,7 +423,7 @@ pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rt for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, - &count))) { + &count, mt_safe))) { in_mb[nb_err++] = mb; continue; } @@ -439,9 +441,25 @@ pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rt return nb_prep; } +static uint16_t +pdcp_pre_process_uplane_sn_12_ul_st(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret) +{ + return pdcp_pre_process_uplane_sn_12_ul_flags(entity, in_mb, cop, num, nb_err_ret, false); +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul_mt(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret) +{ + return pdcp_pre_process_uplane_sn_12_ul_flags(entity, in_mb, cop, num, nb_err_ret, true); +} + static inline bool pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, - uint32_t *count) + uint32_t *count, const bool mt_safe) { struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; const uint8_t hdr_sz = en_priv->hdr_sz; @@ -453,7 +471,7 @@ pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_ return false; /* Update sequence num in the PDU header */ - *count = en_priv->state.tx_next++; + *count = pdcp_atomic_inc(&en_priv->state.tx_next, mt_safe); sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; @@ -466,8 +484,9 @@ pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_ } static inline uint16_t -pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], - struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +pdcp_pre_process_uplane_sn_18_ul_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, const bool mt_safe) { struct entity_priv *en_priv = entity_priv_get(entity); uint16_t nb_cop, nb_prep = 0, nb_err = 0; @@ -496,7 +515,7 @@ pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rt memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, - &count))) { + &count, mt_safe))) { in_mb[nb_err++] = mb; continue; } @@ -507,7 +526,7 @@ pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rt for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, - &count))) { + &count, mt_safe))) { in_mb[nb_err++] = mb; continue; @@ -527,8 +546,23 @@ pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rt } static uint16_t -pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], - struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +pdcp_pre_process_uplane_sn_18_ul_st(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + return pdcp_pre_process_uplane_sn_18_ul_flags(entity, in_mb, cop, num, nb_err_ret, false); +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_ul_mt(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + return pdcp_pre_process_uplane_sn_18_ul_flags(entity, in_mb, cop, num, nb_err_ret, true); +} + +static inline uint16_t +pdcp_pre_process_cplane_sn_12_ul_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, const bool mt_safe) { struct entity_priv *en_priv = entity_priv_get(entity); struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; @@ -565,7 +599,7 @@ pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rt memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); /* Update sequence number in the PDU header */ - count = en_priv->state.tx_next++; + count = pdcp_atomic_inc(&en_priv->state.tx_next, mt_safe); sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); @@ -584,6 +618,21 @@ pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rt return nb_prep; } +static uint16_t +pdcp_pre_process_cplane_sn_12_ul_st(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + return pdcp_pre_process_cplane_sn_12_ul_flags(entity, in_mb, cop, num, nb_err_ret, false); +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul_mt(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + return pdcp_pre_process_cplane_sn_12_ul_flags(entity, in_mb, cop, num, nb_err_ret, true); +} + + static uint16_t pdcp_post_process_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], @@ -659,7 +708,9 @@ pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_read_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_read_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -727,7 +778,9 @@ pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_read_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_read_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -796,7 +849,9 @@ pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rt nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_read_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_read_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -938,6 +993,8 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r struct rte_mbuf *mb; uint32_t count; + pdcp_write_lock(entity); + for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -958,6 +1015,8 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r err_mb[nb_err++] = mb; } + pdcp_write_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -991,6 +1050,8 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb; uint32_t count; + pdcp_write_lock(entity); + for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -1011,6 +1072,8 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_write_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -1029,7 +1092,10 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { - entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + if (conf->enable_thread_safety) + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul_mt; + else + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul_st; entity->post_process = pdcp_post_process_ul; } @@ -1043,14 +1109,20 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { - entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + if (conf->enable_thread_safety) + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul_mt; + else + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul_st; entity->post_process = pdcp_post_process_ul; } if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { - entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + if (conf->enable_thread_safety) + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul_mt; + else + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul_st; entity->post_process = pdcp_post_process_ul; } @@ -1174,6 +1246,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent */ en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** + * flags.enable_thread_safety + * + * Indicate whether the thread safety is enabled for PDCP entity. + */ + en_priv->flags.is_thread_safety_enabled = conf->enable_thread_safety; + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 1c6d2466b2..7431345d3f 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -92,6 +92,8 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c if (ret) return ret; + pdcp_lock_init(entity); + return 0; } @@ -237,8 +239,11 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, pdcp_entity->max_pkt_cache, en_priv->state.rx_next); pdcp_reorder_stop(&dl->reorder); + + pdcp_write_lock(pdcp_entity); en_priv->state.rx_next = 0; en_priv->state.rx_deliv = 0; + pdcp_write_unlock(pdcp_entity); } return nb_out; @@ -299,6 +304,8 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * performing header decompression, if not decompressed before: */ + pdcp_write_lock(entity); + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); capacity -= nb_out; @@ -330,5 +337,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct dl->t_reorder.state = TIMER_EXPIRED; } + pdcp_write_unlock(entity); + return nb_out; } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index b926b0df29..88700e3737 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -141,6 +141,8 @@ struct rte_pdcp_entity_conf { bool is_slrb; /** Enable security offload on the device specified. */ bool en_sec_offload; + /** Enable usage of synchronization primitives for entity. */ + bool enable_thread_safety; /** Device on which security/crypto session need to be created. */ uint8_t dev_id; /** -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [PATCH v4 21/22] pdcp: add thread safe processing 2023-05-26 21:02 ` [PATCH v4 21/22] pdcp: add thread safe processing Anoob Joseph @ 2023-05-26 22:11 ` Stephen Hemminger 2023-05-27 5:24 ` [EXT] " Anoob Joseph 2023-05-26 22:15 ` Stephen Hemminger 1 sibling, 1 reply; 192+ messages in thread From: Stephen Hemminger @ 2023-05-26 22:11 UTC (permalink / raw) To: Anoob Joseph Cc: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz On Sat, 27 May 2023 02:32:13 +0530 Anoob Joseph <anoobj@marvell.com> wrote: > +static inline uint32_t > +pdcp_atomic_inc(uint32_t *val, const bool mt_safe) > +{ > + if (mt_safe) > + return __atomic_fetch_add(val, 1, __ATOMIC_RELAXED); > + else > + return (*val)++; > +} This is a bad pattern. None of the rest of DPDK does this. Either be thread safe or not. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [EXT] Re: [PATCH v4 21/22] pdcp: add thread safe processing 2023-05-26 22:11 ` Stephen Hemminger @ 2023-05-27 5:24 ` Anoob Joseph 2023-05-27 7:17 ` Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 5:24 UTC (permalink / raw) To: Stephen Hemminger Cc: Thomas Monjalon, Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz Hi Stephen, Please see inline. Thanks, Anoob > -----Original Message----- > From: Stephen Hemminger <stephen@networkplumber.org> > Sent: Saturday, May 27, 2023 3:42 AM > To: Anoob Joseph <anoobj@marvell.com> > Cc: Thomas Monjalon <thomas@monjalon.net>; Akhil Goyal > <gakhil@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Volodymyr Fialko > <vfialko@marvell.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; > Mattias Rönnblom <mattias.ronnblom@ericsson.com>; Kiran Kumar > Kokkilagadda <kirankumark@marvell.com>; dev@dpdk.org; Olivier Matz > <olivier.matz@6wind.com> > Subject: [EXT] Re: [PATCH v4 21/22] pdcp: add thread safe processing > > External Email > > ---------------------------------------------------------------------- > On Sat, 27 May 2023 02:32:13 +0530 > Anoob Joseph <anoobj@marvell.com> wrote: > > > +static inline uint32_t > > +pdcp_atomic_inc(uint32_t *val, const bool mt_safe) { > > + if (mt_safe) > > + return __atomic_fetch_add(val, 1, __ATOMIC_RELAXED); > > + else > > + return (*val)++; > > +} > > This is a bad pattern. None of the rest of DPDK does this. > Either be thread safe or not. [Anoob] Most protocol implementation would have a similar issue. I've tried to follow the approach taken in lib IPsec (please check lib/ipsec/ipsec_sqn.h). From the discussion we had on v3, I've tried to make it compile time constant to remove any conditional checks in datapath. If you still think, this is an issue, I could drop this patch for now and re-introduce it later once we have test applications that can work with traffic. Please share your thoughts. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [EXT] Re: [PATCH v4 21/22] pdcp: add thread safe processing 2023-05-27 5:24 ` [EXT] " Anoob Joseph @ 2023-05-27 7:17 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:17 UTC (permalink / raw) To: Anoob Joseph, Stephen Hemminger Cc: Thomas Monjalon, Akhil Goyal, Jerin Jacob Kollanukkaran, Konstantin Ananyev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Olivier Matz Hi Stephen, I've dropped this patch for now. Based on what we conclude on this thread, will post this as a separate patch as required. Thanks, Anoob > -----Original Message----- > From: Anoob Joseph <anoobj@marvell.com> > Sent: Saturday, May 27, 2023 10:55 AM > To: Stephen Hemminger <stephen@networkplumber.org> > Cc: Thomas Monjalon <thomas@monjalon.net>; Akhil Goyal > <gakhil@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Volodymyr Fialko > <vfialko@marvell.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; > Mattias Rönnblom <mattias.ronnblom@ericsson.com>; Kiran Kumar > Kokkilagadda <kirankumark@marvell.com>; dev@dpdk.org; Olivier Matz > <olivier.matz@6wind.com> > Subject: RE: [EXT] Re: [PATCH v4 21/22] pdcp: add thread safe processing > > Hi Stephen, > > Please see inline. > > Thanks, > Anoob > > > -----Original Message----- > > From: Stephen Hemminger <stephen@networkplumber.org> > > Sent: Saturday, May 27, 2023 3:42 AM > > To: Anoob Joseph <anoobj@marvell.com> > > Cc: Thomas Monjalon <thomas@monjalon.net>; Akhil Goyal > > <gakhil@marvell.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; > > Konstantin Ananyev <konstantin.v.ananyev@yandex.ru>; Volodymyr > Fialko > > <vfialko@marvell.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; > > Mattias Rönnblom <mattias.ronnblom@ericsson.com>; Kiran Kumar > > Kokkilagadda <kirankumark@marvell.com>; dev@dpdk.org; Olivier Matz > > <olivier.matz@6wind.com> > > Subject: [EXT] Re: [PATCH v4 21/22] pdcp: add thread safe processing > > > > External Email > > > > ---------------------------------------------------------------------- > > On Sat, 27 May 2023 02:32:13 +0530 > > Anoob Joseph <anoobj@marvell.com> wrote: > > > > > +static inline uint32_t > > > +pdcp_atomic_inc(uint32_t *val, const bool mt_safe) { > > > + if (mt_safe) > > > + return __atomic_fetch_add(val, 1, __ATOMIC_RELAXED); > > > + else > > > + return (*val)++; > > > +} > > > > This is a bad pattern. None of the rest of DPDK does this. > > Either be thread safe or not. > > [Anoob] Most protocol implementation would have a similar issue. I've tried > to follow the approach taken in lib IPsec (please check lib/ipsec/ipsec_sqn.h). > From the discussion we had on v3, I've tried to make it compile time constant > to remove any conditional checks in datapath. > > If you still think, this is an issue, I could drop this patch for now and re- > introduce it later once we have test applications that can work with traffic. > Please share your thoughts. ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [PATCH v4 21/22] pdcp: add thread safe processing 2023-05-26 21:02 ` [PATCH v4 21/22] pdcp: add thread safe processing Anoob Joseph 2023-05-26 22:11 ` Stephen Hemminger @ 2023-05-26 22:15 ` Stephen Hemminger 1 sibling, 0 replies; 192+ messages in thread From: Stephen Hemminger @ 2023-05-26 22:15 UTC (permalink / raw) To: Anoob Joseph Cc: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev, Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz On Sat, 27 May 2023 02:32:13 +0530 Anoob Joseph <anoobj@marvell.com> wrote: > + /* Lock to protect concurrent updates */ > + rte_rwlock_t rwl; > /** Bitmap memory region Reader/writer locks are significantly more expensive than simple spin lock. Even uncontended access requires additional cache update. Unless there is case with a long region where lock is held, just use a spin lock. See some Paul McKenney's locking talks (LCA video is good) about lock overheads in Linux. Since DPDK has same set of instructions to use; the same issues apply to userspace. ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v4 22/22] test/pdcp: add PDCP status report cases 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (20 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 21/22] pdcp: add thread safe processing Anoob Joseph @ 2023-05-26 21:02 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 23 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-26 21:02 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Test PDCP status report generation. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 312 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 312 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 82cc25ec7a..423526380f 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -2,6 +2,7 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_errno.h> #ifdef RTE_LIB_EVENTDEV #include <rte_eventdev.h> @@ -48,6 +49,9 @@ struct pdcp_testsuite_params { #endif /* RTE_LIB_EVENTDEV */ bool timer_is_running; uint64_t min_resolution_ns; + struct rte_pdcp_up_ctrl_pdu_hdr *status_report; + uint32_t status_report_bitmask_capacity; + uint8_t *ctrl_pdu_buf; }; static struct pdcp_testsuite_params testsuite_params; @@ -168,6 +172,18 @@ static struct rte_pdcp_t_reordering t_reorder_timer = { .stop = pdcp_timer_stop_cb, }; +static inline void +bitmask_set_bit(uint8_t *mask, uint32_t bit) +{ + mask[bit / 8] |= (1 << bit % 8); +} + +static inline bool +bitmask_is_bit_set(const uint8_t *mask, uint32_t bit) +{ + return mask[bit / 8] & (1 << (bit % 8)); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -314,6 +330,21 @@ testsuite_setup(void) goto cop_pool_free; } + /* Allocate memory for longest possible status report */ + ts_params->status_report_bitmask_capacity = RTE_PDCP_CTRL_PDU_SIZE_MAX - + sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + ts_params->status_report = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->status_report == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report\n"); + goto cop_pool_free; + } + + ts_params->ctrl_pdu_buf = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->ctrl_pdu_buf == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report data\n"); + goto cop_pool_free; + } + return 0; cop_pool_free: @@ -322,6 +353,8 @@ testsuite_setup(void) mbuf_pool_free: rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); return TEST_FAILED; } @@ -344,6 +377,9 @@ testsuite_teardown(void) rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); } static int @@ -1410,6 +1446,246 @@ test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) return ret; } +static struct rte_pdcp_up_ctrl_pdu_hdr * +pdcp_status_report_init(uint32_t fmc) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + + hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + hdr->fmc = rte_cpu_to_be_32(fmc); + hdr->r = 0; + memset(hdr->bitmap, 0, testsuite_params.status_report_bitmask_capacity); + + return hdr; +} + +static uint32_t +pdcp_status_report_len(void) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + uint32_t i; + + for (i = testsuite_params.status_report_bitmask_capacity; i != 0; i--) { + if (hdr->bitmap[i - 1]) + return i; + } + + return 0; +} + +static int +pdcp_status_report_verify(struct rte_mbuf *status_report, + const struct rte_pdcp_up_ctrl_pdu_hdr *expected_hdr, uint32_t expected_len) +{ + uint32_t received_len = rte_pktmbuf_pkt_len(status_report); + uint8_t *received_buf = testsuite_params.ctrl_pdu_buf; + int ret; + + ret = pktmbuf_read_into(status_report, received_buf, RTE_PDCP_CTRL_PDU_SIZE_MAX); + TEST_ASSERT_SUCCESS(ret, "Failed to copy status report pkt into continuous buffer"); + + debug_hexdump(stdout, "Received:", received_buf, received_len); + debug_hexdump(stdout, "Expected:", expected_hdr, expected_len); + + TEST_ASSERT_EQUAL(expected_len, received_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_len, received_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(received_buf, expected_hdr, expected_len, + "Generated packet not as expected"); + + return 0; +} + +static int +test_status_report_gen(const struct pdcp_test_conf *ul_conf, + const struct rte_pdcp_up_ctrl_pdu_hdr *hdr, + uint32_t bitmap_len) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *status_report = NULL, **out_mb, *m; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint32_t nb_pkts = 0, i; + uint8_t cdev_id; + + const uint32_t start_count = rte_be_to_cpu_32(hdr->fmc); + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.status_report_required = true; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + out_mb = calloc(pdcp_entity->max_pkt_cache, sizeof(uintptr_t)); + + for (i = 0; i < bitmap_len * 8; i++) { + if (!bitmask_is_bit_set(hdr->bitmap, i)) + continue; + + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + + } + + m = NULL; + + /* Check status report */ + status_report = rte_pdcp_control_pdu_create(pdcp_entity, + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT); + ASSERT_TRUE_OR_GOTO(status_report != NULL, exit, "Could not generate status report\n"); + + const uint32_t expected_len = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr) + bitmap_len; + + ASSERT_TRUE_OR_GOTO(pdcp_status_report_verify(status_report, hdr, expected_len) == 0, exit, + "Report verification failure\n"); + + ret = TEST_SUCCESS; +exit: + rte_free(m); + rte_pktmbuf_free(status_report); + rte_pktmbuf_free_bulk(out_mb, nb_pkts); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + free(out_mb); + return ret; +} + +static void +ctrl_pdu_hdr_packet_set(struct rte_pdcp_up_ctrl_pdu_hdr *hdr, uint32_t pkt_count) +{ + bitmask_set_bit(hdr->bitmap, pkt_count - rte_be_to_cpu_32(hdr->fmc) - 1); +} + +static int +test_status_report_fmc_only(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(42); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_first_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_second_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_full_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE + 1; + int i; + + for (i = 0; i < RTE_BITMAP_SLAB_BIT_SIZE; i++) + ctrl_pdu_hdr_packet_set(hdr, start_offset + i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_non_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + ctrl_pdu_hdr_packet_set(hdr, 3 * RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_max_length_sn_12(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr; + const uint32_t fmc = 0; + uint32_t i; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + ul_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + hdr = pdcp_status_report_init(fmc); + + const uint32_t max_count = RTE_MIN((RTE_PDCP_CTRL_PDU_SIZE_MAX - sizeof(hdr)) * 8, + (uint32_t)PDCP_WINDOW_SIZE(RTE_SECURITY_PDCP_SN_SIZE_12)); + + i = fmc + 2; /* set first count to have a gap, to enable packet buffering */ + + for (; i < max_count; i++) + ctrl_pdu_hdr_packet_set(hdr, i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_different_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(63); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 64 + 1); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_same_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(2); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 4); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1611,11 +1887,47 @@ static struct unit_test_suite reorder_test_cases = { } }; +static struct unit_test_suite status_report_test_cases = { + .suite_name = "PDCP status report", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_status_report_fmc_only", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_fmc_only), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_first_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_first_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_second_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_second_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_full_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_full_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_non_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_non_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_max_length_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_status_report_max_length_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_different_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_different_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_same_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_same_slab), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, &reorder_test_cases, + &status_report_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 00/21] lib: add pdcp protocol 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (21 preceding siblings ...) 2023-05-26 21:02 ` [PATCH v4 22/22] test/pdcp: add PDCP status report cases Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph ` (20 more replies) 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 23 siblings, 21 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following operations, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Similar to lib IPsec, lib PDCP would allow application to use same API sequence while leveraging protocol offload features enabled by rte_security library. Lib PDCP would internally change the handles registered for *pre_process* and *post_process* based on features enabled in the entity. Lib PDCP would create the required sessions on the device provided in entity to minimize the application requirements. Also, the crypto_op allocation and free would also be done internally by lib PDCP to allow the library to create crypto ops as required for the input packets. For example, when control PDUs are received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP is expected to handle it differently. Lib PDCP utilizes reorder library for implementing in-order delivery. It utilizes bitmap library for implementing status reports and track the COUNT value of the packets received. To allow application to choose timer implementation of choice, lib PDCP allows application to configure handles that can be used for starting & stopping timers. Upon expiry, application can call corresponding PDCP API(``rte_pdcp_t_reordering_expiry_handle``) for handling the event. Unit tests are added to verify both rte_timer based timers as well as rte_eventdev based timers. PDCP tracks the sequence number of the received packets and during events such as re-establishment, it is required to generate reports and transmit to the peer. This series introduces ``rte_pdcp_control_pdu_create`` for handling control PDU generation. Changes in v5: - Deferred patch adding thread safe processing. - Updated release notes & MAINTAINERS file. Changes in v4: - Disabled 'annotate locks' with lib PDCP - Enable PDCP autotest only when lib is enabled - Use rwlock instead of spinlock - Avoid per packet checks for thread safety (Stephen) - In DL path, save count determined during pre-process in mbuf and use the same in post-process. Determining count again may lead To errors - Simplified DL path threads to allow more common code between SN 12 & 18 Changes in v3: - Addressed review comments (Akhil) - Addressed build failure in CI (tests with lib eventdev disabled) - Addressed checkpatch issues - Set only positive values to rte_errno (Akhil) Changes in v2: - Added control PDU handling - Added t-Reordering timer - Added in-order delivery - Added status PDU generation - Rebased on top of new features added in reorder library - Split base patch - Increased test coverage - Improved thread safety Changes from RFC - Implementation for all APIs covering basic control plane & user plane packets - Unit test leveraging existing PDCP vectors available in test_cryptodev - Unit test performing both UL & DL operations to verify various protocol features - Updated documentation Sample application sequence: struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; struct rte_crypto_op *cop[MAX_BURST_SIZE]; struct rte_pdcp_group grp[MAX_BURST_SIZE]; struct rte_pdcp_entity *pdcp_entity; int nb_max_out_mb, ret, nb_grp; /* Create PDCP entity */ pdcp_entity = rte_pdcp_entity_establish(&conf); /** * Allocate buffer for holding mbufs returned during PDCP suspend, * release & post-process APIs. */ /* Max packets that can be cached in entity + burst size */ nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); if (out_mb == NULL) { /* Handle error */ } while (1) { /* Receive packet and form mbuf */ /** * Prepare packets for crypto operation. Following operations * would be done, * * Transmitting entity/UL (only data PDUs): * - Perform compression * - Assign sequence number * - Add PDCP header * - Create & prepare crypto_op * - Prepare IV for crypto operation (auth_gen, encrypt) * - Save original PDCP SDU (during PDCP re-establishment, * unconfirmed PDCP SDUs need to crypto processed again and * transmitted/re-transmitted) * * Receiving entity/DL: * - Any control PDUs received would be processed and * appropriate actions taken. If data PDU, continue. * - Determine sequence number (based on HFN & per packet SN) * - Prepare crypto_op * - Prepare IV for crypto operation (decrypt, auth_verify) */ nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, nb_rx, &nb_err); if (nb_err != 0) { /* Handle error packets */ } if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) != nb_success) { /* Retry for enqueue failure packets */ } ... ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, MAX_BURST_SIZE); if (ret == 0) continue; /** * Received a burst of completed crypto ops from cryptodev. It * may belong to various entities. Group similar ones together * for entity specific post-processing. */ /** * Groups similar entities together. Frees crypto op and based * on crypto_op status, set mbuf->ol_flags which would be * checked in rte_pdcp_pkt_post_process(). */ nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); for (i = 0; i != nb_grp; i++) { /** * Post process packets after crypto completion. * Following operations would be done, * * Transmitting entity/UL: * - Check crypto result * * Receiving entity/DL: * - Check crypto operation status * - Check for duplication (if yes, drop duplicate) * - Perform decompression * - Trim PDCP header * - Hold packet (SDU) for in-order delivery (return * completed packets as and when sequence is * completed) * - If not in sequence, cache the packet and start * t-Reordering timer. When timer expires, the * packets need to delivered to upper layers (not * treated as error packets). */ nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, grp[i].m, out_mb, grp[i].cnt, &nb_err); if (nb_err != 0) { /* Handle error packets */ } /* Perform additional operations */ /** * Tranmitting entity/UL * - If duplication is enabled, duplicate PDCP PDUs * - When lower layers confirm reception of a PDCP PDU, * it should be communicated to PDCP layer so that * PDCP can drop the corresponding SDU */ } } Sample output from unit test executed on crypto_cn10k PMD(summary): ~# DPDK_TEST=pdcp_autotest ./dpdk-test --log-level=7 <snip> + ------------------------------------------------------- + + Test Suite Summary : PDCP Unit Test Suite + ------------------------------------------------------- + + Known vector cases : 112/160 passed, 0/160 skipped, 0/160 failed, 48/160 unsupported + PDCP combined mode : 1/1 passed, 0/1 skipped, 0/1 failed, 0/1 unsupported + PDCP HFN/SN : 4/4 passed, 0/4 skipped, 0/4 failed, 0/4 unsupported + PDCP reorder : 5/5 passed, 0/5 skipped, 0/5 failed, 0/5 unsupported + PDCP status report : 9/9 passed, 0/9 skipped, 0/9 failed, 0/9 unsupported + ------------------------------------------------------- + + Sub Testsuites Total : 5 + Sub Testsuites Skipped : 0 + Sub Testsuites Passed : 5 + Sub Testsuites Failed : 0 + ------------------------------------------------------- + + Tests Total : 179 + Tests Skipped : 0 + Tests Executed : 179 + Tests Unsupported: 48 + Tests Passed : 131 + Tests Failed : 0 + ------------------------------------------------------- + Test OK Anoob Joseph (10): lib: add pdcp protocol pdcp: add pre and post-process pdcp: add packet group pdcp: add crypto session create and destroy pdcp: add pre and post process for UL pdcp: add pre and post process for DL pdcp: add IV generation routines app/test: add lib pdcp tests doc: add PDCP library guide pdcp: add control PDU handling for status report Volodymyr Fialko (11): net: add PDCP header test/pdcp: pdcp HFN tests in combined mode pdcp: implement t-Reordering and packet buffering test/pdcp: add in-order delivery cases pdcp: add timer callback handlers pdcp: add timer expiry handle test/pdcp: add timer expiry cases test/pdcp: add timer restart case pdcp: add support for status report pdcp: allocate reorder buffer alongside with entity test/pdcp: add PDCP status report cases MAINTAINERS | 8 + app/test/meson.build | 5 + app/test/test_pdcp.c | 1981 +++++++++++++++++ doc/api/doxy-api-index.md | 4 +- doc/api/doxy-api.conf.in | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 293 +++ doc/guides/rel_notes/release_23_07.rst | 12 + lib/meson.build | 1 + lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++ lib/pdcp/meson.build | 21 + lib/pdcp/pdcp_cnt.c | 164 ++ lib/pdcp/pdcp_cnt.h | 24 + lib/pdcp/pdcp_crypto.c | 238 ++ lib/pdcp/pdcp_crypto.h | 20 + lib/pdcp/pdcp_ctrl_pdu.c | 72 + lib/pdcp/pdcp_ctrl_pdu.h | 16 + lib/pdcp/pdcp_entity.h | 260 +++ lib/pdcp/pdcp_process.c | 1223 ++++++++++ lib/pdcp/pdcp_process.h | 37 + lib/pdcp/pdcp_reorder.c | 20 + lib/pdcp/pdcp_reorder.h | 68 + lib/pdcp/rte_pdcp.c | 334 +++ lib/pdcp/rte_pdcp.h | 383 ++++ lib/pdcp/rte_pdcp_group.h | 131 ++ lib/pdcp/version.map | 20 + 28 files changed, 5485 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pdcp.c create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst create mode 100644 lib/net/rte_pdcp_hdr.h create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 01/21] net: add PDCP header 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-30 8:51 ` Akhil Goyal 2023-05-27 7:15 ` [PATCH v5 02/21] lib: add pdcp protocol Anoob Joseph ` (19 subsequent siblings) 20 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add PDCP protocol header to be used for supporting PDCP protocol processing. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- doc/api/doxy-api-index.md | 3 +- lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 150 insertions(+), 1 deletion(-) create mode 100644 lib/net/rte_pdcp_hdr.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index c709fd48ad..debbe4134f 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -127,7 +127,8 @@ The public API headers are grouped by topics: [Geneve](@ref rte_geneve.h), [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), - [PPP](@ref rte_ppp.h) + [PPP](@ref rte_ppp.h), + [PDCP hdr](@ref rte_pdcp_hdr.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/lib/net/meson.build b/lib/net/meson.build index 379d161ee0..bd56f91c22 100644 --- a/lib/net/meson.build +++ b/lib/net/meson.build @@ -22,6 +22,7 @@ headers = files( 'rte_geneve.h', 'rte_l2tpv2.h', 'rte_ppp.h', + 'rte_pdcp_hdr.h', ) sources = files( diff --git a/lib/net/rte_pdcp_hdr.h b/lib/net/rte_pdcp_hdr.h new file mode 100644 index 0000000000..72ae9a66cb --- /dev/null +++ b/lib/net/rte_pdcp_hdr.h @@ -0,0 +1,147 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_HDR_H +#define RTE_PDCP_HDR_H + +/** + * @file + * + * PDCP-related defines + * + * Based on - ETSI TS 138 323 V17.1.0 (2022-08) + * https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + */ + +#include <rte_byteorder.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * 4.3.1 + * + * Indicate the maximum supported size of a PDCP Control PDU. + */ +#define RTE_PDCP_CTRL_PDU_SIZE_MAX 9000u + +/** + * 6.3.4 MAC-I + * + * Indicate the size of MAC-I in PDCP PDU. + */ +#define RTE_PDCP_MAC_I_LEN 4 + +/** + * Indicate type of control information included in the corresponding PDCP + * Control PDU. + */ +enum rte_pdcp_ctrl_pdu_type { + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT = 0, + RTE_PDCP_CTRL_PDU_TYPE_ROHC_FEEDBACK = 1, + RTE_PDCP_CTRL_PDU_TYPE_EHC_FEEDBACK = 2, + RTE_PDCP_CRTL_PDU_TYPE_UDC_FEEDBACK = 3, +}; + +/** + * 6.3.7 D/C + * + * This field indicates whether the corresponding PDCP PDU is a + * PDCP Data PDU or a PDCP Control PDU. + */ +enum rte_pdcp_pdu_type { + RTE_PDCP_PDU_TYPE_CTRL = 0, + RTE_PDCP_PDU_TYPE_DATA = 1, +}; + +/** + * 6.2.2.1 Data PDU for SRBs + */ +__extension__ +struct rte_pdcp_cp_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 4; /**< Reserved */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.2.2 Data PDU for DRBs and MRBs with 12 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 3; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 3; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.2.3 Data PDU for DRBs and MRBs with 18 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_18_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ + uint8_t r : 5; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 5; /**< Reserved */ + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ +#endif + uint8_t sn_15_8; /**< Sequence number bits 8-15 */ + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.3.1 Control PDU for PDCP status report + */ +__extension__ +struct rte_pdcp_up_ctrl_pdu_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t r : 4; /**< Reserved */ +#endif + /** + * 6.3.9 FMC + * + * First Missing COUNT. This field indicates the COUNT value of the + * first missing PDCP SDU within the reordering window, i.e. RX_DELIV. + */ + rte_be32_t fmc; + /** + * 6.3.10 Bitmap + * + * Length: Variable. The length of the bitmap field can be 0. + * + * This field indicates which SDUs are missing and which SDUs are + * correctly received in the receiving PDCP entity. The bit position of + * Nth bit in the Bitmap is N, i.e., the bit position of the first bit + * in the Bitmap is 1. + */ + uint8_t bitmap[]; +} __rte_packed; + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_HDR_H */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v5 01/21] net: add PDCP header 2023-05-27 7:15 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph @ 2023-05-30 8:51 ` Akhil Goyal 0 siblings, 0 replies; 192+ messages in thread From: Akhil Goyal @ 2023-05-30 8:51 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Olivier Matz Cc: Volodymyr Fialko, Hemant Agrawal, Konstantin Ananyev, Mattias Rönnblom, Kiran Kumar Kokkilagadda, dev, Stephen Hemminger, Jerin Jacob Kollanukkaran > Subject: [PATCH v5 01/21] net: add PDCP header > > From: Volodymyr Fialko <vfialko@marvell.com> > > Add PDCP protocol header to be used for supporting PDCP protocol > processing. > > Signed-off-by: Anoob Joseph <anoobj@marvell.com> > Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> > Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> > Acked-by: Akhil Goyal <gakhil@marvell.com> > --- Olivier, Could you please review and give ack on this patch. Thomas, Can this patch be applied to next-crypto? ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 02/21] lib: add pdcp protocol 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 03/21] pdcp: add pre and post-process Anoob Joseph ` (18 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 6 ++ doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + lib/meson.build | 1 + lib/pdcp/meson.build | 17 ++++ lib/pdcp/pdcp_crypto.c | 21 +++++ lib/pdcp/pdcp_crypto.h | 15 ++++ lib/pdcp/pdcp_entity.h | 113 ++++++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 138 +++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 13 +++ lib/pdcp/rte_pdcp.c | 141 ++++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 167 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 10 +++ 13 files changed, 645 insertions(+), 1 deletion(-) create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 8df23e5099..85a3b94644 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1549,6 +1549,12 @@ F: doc/guides/tools/pdump.rst F: app/dumpcap/ F: doc/guides/tools/dumpcap.rst +PDCP - EXPERIMENTAL +M: Anoob Joseph <anoobj@marvell.com> +M: Volodymyr Fialko <vfialko@marvell.com> +T: git://dpdk.org/next/dpdk-next-crypto +F: lib/pdcp/ + Packet Framework ---------------- diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index debbe4134f..cd7a6cae44 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -128,7 +128,8 @@ The public API headers are grouped by topics: [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), [PPP](@ref rte_ppp.h), - [PDCP hdr](@ref rte_pdcp_hdr.h) + [PDCP hdr](@ref rte_pdcp_hdr.h), + [PDCP](@ref rte_pdcp.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index d230a19e1f..58789308a9 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -62,6 +62,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/net \ @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ + @TOPDIR@/lib/pdcp \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ @TOPDIR@/lib/port \ diff --git a/lib/meson.build b/lib/meson.build index dc8aa4ac84..a6a54c196c 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -64,6 +64,7 @@ libraries = [ 'flow_classify', # flow_classify lib depends on pkt framework table lib 'graph', 'node', + 'pdcp', # pdcp lib depends on crypto and security ] optional_libs = [ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode 100644 index 0000000000..ccaf426240 --- /dev/null +++ b/lib/pdcp/meson.build @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +sources = files( + 'pdcp_crypto.c', + 'pdcp_process.c', + 'rte_pdcp.c', + ) +headers = files('rte_pdcp.h') + +deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c new file mode 100644 index 0000000000..755e27ec9e --- /dev/null +++ b/lib/pdcp/pdcp_crypto.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_crypto.h" + +int +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + RTE_SET_USED(entity); + RTE_SET_USED(conf); + return 0; +} + +void +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) +{ + RTE_SET_USED(entity); +} diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h new file mode 100644 index 0000000000..6563331d37 --- /dev/null +++ b/lib/pdcp/pdcp_crypto.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CRYPTO_H +#define PDCP_CRYPTO_H + +#include <rte_pdcp.h> + +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, + const struct rte_pdcp_entity_conf *conf); + +void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); + +#endif /* PDCP_CRYPTO_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h new file mode 100644 index 0000000000..000297588f --- /dev/null +++ b/lib/pdcp/pdcp_entity.h @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_ENTITY_H +#define PDCP_ENTITY_H + +#include <rte_common.h> +#include <rte_crypto_sym.h> +#include <rte_mempool.h> +#include <rte_pdcp.h> +#include <rte_security.h> + +struct entity_priv; + +/* IV generation function based on the entity configuration */ +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count); + +struct entity_state { + uint32_t rx_next; + uint32_t tx_next; + uint32_t rx_deliv; + uint32_t rx_reord; +}; + +/* + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + */ + +struct entity_priv { + /** Crypto sym session. */ + struct rte_cryptodev_sym_session *crypto_sess; + /** Entity specific IV generation function. */ + iv_gen_t iv_gen; + /** Entity state variables. */ + struct entity_state state; + /** Flags. */ + struct { + /** PDCP PDU has 4 byte MAC-I. */ + uint64_t is_authenticated : 1; + /** Cipher offset & length in bits. */ + uint64_t is_cipher_in_bits : 1; + /** Auth offset & length in bits. */ + uint64_t is_auth_in_bits : 1; + /** Is UL/transmitting PDCP entity. */ + uint64_t is_ul_entity : 1; + /** Is NULL auth. */ + uint64_t is_null_auth : 1; + } flags; + /** Crypto op pool. */ + struct rte_mempool *cop_pool; + /** PDCP header size. */ + uint8_t hdr_sz; + /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ + uint8_t aad_sz; + /** Device ID of the device to be used for offload. */ + uint8_t dev_id; +}; + +struct entity_priv_dl_part { + /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ + uint8_t dummy; +}; + +struct entity_priv_ul_part { + /* + * NOTE: when re-establish is supported, plain PDCP packets & COUNT values need to be + * cached. + */ + uint8_t dummy; +}; + +static inline struct entity_priv * +entity_priv_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); +} + +static inline struct entity_priv_dl_part * +entity_dl_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline struct entity_priv_ul_part * +entity_ul_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) +{ + return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); +} + +#endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c new file mode 100644 index 0000000000..79f5dce5db --- /dev/null +++ b/lib/pdcp/pdcp_process.c @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_memcpy.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, + struct rte_crypto_sym_xform **a_xfrm) +{ + *c_xfrm = NULL; + *a_xfrm = NULL; + + if (conf->crypto_xfrm == NULL) + return -EINVAL; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + *c_xfrm = conf->crypto_xfrm; + *a_xfrm = conf->crypto_xfrm->next; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + *a_xfrm = conf->crypto_xfrm; + *c_xfrm = conf->crypto_xfrm->next; + } else { + return -EINVAL; + } + + return 0; +} + +static int +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + int ret; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + /** + * flags.is_authenticated + * + * MAC-I would be added in case of control plane packets and when authentication + * transform is not NULL. + */ + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && (a_xfrm == NULL)) + return -EINVAL; + + if (a_xfrm != NULL) + en_priv->flags.is_authenticated = 1; + + /** + * flags.is_cipher_in_bits + * + * For ZUC & SNOW3G cipher algos, offset & length need to be provided in bits. + */ + + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) + en_priv->flags.is_cipher_in_bits = 1; + + /** + * flags.is_auth_in_bits + * + * For ZUC & SNOW3G authentication algos, offset & length need to be provided in bits. + */ + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) + en_priv->flags.is_auth_in_bits = 1; + } + + /** + * flags.is_ul_entity + * + * Indicate whether the entity is UL/transmitting PDCP entity. + */ + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + en_priv->flags.is_ul_entity = 1; + + /** + * flags.is_null_auth + * + * For NULL auth, 4B zeros need to be added by lib PDCP. Indicate that + * algo is NULL auth to perform the same. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + en_priv->flags.is_null_auth = 1; + + /** + * hdr_sz + * + * PDCP header size of the entity + */ + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); + + /** + * aad_sz + * + * For AES-CMAC, additional message is prepended for processing. Need to be trimmed after + * crypto processing is done. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) + en_priv->aad_sz = 8; + else + en_priv->aad_sz = 0; + + return 0; +} + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv *en_priv; + int ret; + + if (entity == NULL || conf == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + ret = pdcp_entity_priv_populate(en_priv, conf); + if (ret) + return ret; + + return 0; +} diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h new file mode 100644 index 0000000000..fd53fff0aa --- /dev/null +++ b/lib/pdcp/pdcp_process.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_PROCESS_H +#define PDCP_PROCESS_H + +#include <rte_pdcp.h> + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode 100644 index 0000000000..adcad5dd25 --- /dev/null +++ b/lib/pdcp/rte_pdcp.c @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_pdcp.h> +#include <rte_malloc.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +{ + int size; + + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + size += sizeof(struct entity_priv_dl_part); + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size += sizeof(struct entity_priv_ul_part); + else + return -EINVAL; + + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); +} + +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) +{ + struct rte_pdcp_entity *entity = NULL; + struct entity_priv *en_priv; + int ret, entity_size; + uint32_t count; + + if (conf == NULL || conf->cop_pool == NULL) { + rte_errno = EINVAL; + return NULL; + } + + if (conf->pdcp_xfrm.en_ordering || conf->pdcp_xfrm.remove_duplicates || conf->is_slrb || + conf->en_sec_offload) { + rte_errno = ENOTSUP; + return NULL; + } + + /* + * 6.3.2 PDCP SN + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the PDCP SN is + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl-PDCP-SN-Size in + * TS 38.331 [3]) + */ + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { + rte_errno = ENOTSUP; + return NULL; + } + + if (conf->pdcp_xfrm.hfn_threshold) { + rte_errno = EINVAL; + return NULL; + } + + entity_size = pdcp_entity_size_get(conf); + if (entity_size < 0) { + rte_errno = EINVAL; + return NULL; + } + + entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (entity == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + en_priv = entity_priv_get(entity); + + count = pdcp_count_from_hfn_sn_get(conf->pdcp_xfrm.hfn, conf->sn, conf->pdcp_xfrm.sn_size); + + en_priv->state.rx_deliv = count; + en_priv->state.tx_next = count; + en_priv->cop_pool = conf->cop_pool; + + /* Setup crypto session */ + ret = pdcp_crypto_sess_create(entity, conf); + if (ret) + goto entity_free; + + ret = pdcp_process_func_set(entity, conf); + if (ret) + goto crypto_sess_destroy; + + return entity; + +crypto_sess_destroy: + pdcp_crypto_sess_destroy(entity); +entity_free: + rte_free(entity); + rte_errno = -ret; + return NULL; +} + +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) +{ + if (pdcp_entity == NULL) + return -EINVAL; + + /* Teardown crypto sessions */ + pdcp_crypto_sess_destroy(pdcp_entity); + + rte_free(pdcp_entity); + + RTE_SET_USED(out_mb); + return 0; +} + +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]) +{ + struct entity_priv *en_priv; + + if (pdcp_entity == NULL) + return -EINVAL; + + en_priv = entity_priv_get(pdcp_entity); + + if (en_priv->flags.is_ul_entity) { + en_priv->state.tx_next = 0; + } else { + en_priv->state.rx_next = 0; + en_priv->state.rx_deliv = 0; + } + + RTE_SET_USED(out_mb); + + return 0; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode 100644 index 0000000000..1f96fdc9a1 --- /dev/null +++ b/lib/pdcp/rte_pdcp.h @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_H +#define RTE_PDCP_H + +/** + * @file rte_pdcp.h + * + * RTE PDCP support. + * + * A framework for PDCP protocol processing. + */ + +#include <rte_compat.h> +#include <rte_common.h> +#include <rte_mempool.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * PDCP entity. + * + * 4.2.2 PDCP entities + * + * The PDCP entities are located in the PDCP sublayer. Several PDCP entities may + * be defined for a UE. Each PDCP entity is carrying the data of one radio + * bearer. A PDCP entity is associated either to the control plane or the user + * plane depending on which radio bearer it is carrying data for. + */ +struct rte_pdcp_entity { + /** + * PDCP entities may hold packets for purposes of in-order delivery (in + * case of receiving PDCP entity) and re-transmission (in case of + * transmitting PDCP entity). + * + * The field 'max_pkt_cache' would be used to indicate the maximum + * number of packets that may be cached in an entity at any point of + * time. When application provides buffers to receive packets from + * PDCP entity, the size of the buffer should be such that it can + * hold additionally 'max_pkt_cache' number of packets. + */ + uint32_t max_pkt_cache; +} __rte_cache_aligned; + +/** + * PDCP entity configuration to be used for establishing an entity. + */ +/* Structure rte_pdcp_entity_conf 8< */ +struct rte_pdcp_entity_conf { + /** PDCP transform for the entity. */ + struct rte_security_pdcp_xform pdcp_xfrm; + /** Crypto transform applicable for the entity. */ + struct rte_crypto_sym_xform *crypto_xfrm; + /** Mempool for crypto symmetric session. */ + struct rte_mempool *sess_mpool; + /** Crypto op pool.*/ + struct rte_mempool *cop_pool; + /** + * SN value to be used. 32 bit count value to be used for the first + * packet would be derived based on HFN (`rte_security_pdcp_xform.hfn`) + * and SN. + */ + uint32_t sn; + /** + * Indicate whether the PDCP entity belongs to Side Link Radio Bearer. + */ + bool is_slrb; + /** Enable security offload on the device specified. */ + bool en_sec_offload; + /** Device on which security/crypto session need to be created. */ + uint8_t dev_id; + /** + * Reverse direction during IV generation. Can be used to simulate UE + * crypto processing. + */ + bool reverse_iv_direction; +}; +/* >8 End of structure rte_pdcp_entity_conf. */ + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.1 PDCP entity establishment + * + * Establish PDCP entity based on provided input configuration. + * + * @param conf + * Parameters to be used for initializing PDCP entity object. + * @return + * - Valid handle if success + * - NULL in case of failure. rte_errno will be set to error code + */ +__rte_experimental +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.3 PDCP entity release + * + * Release PDCP entity. + * + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * Entity release would result in freeing all memory associated with the PDCP + * entity as well as any crypto/security sessions created. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be released. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.4 PDCP entity suspend + * + * Suspend PDCP entity. + * + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be suspended. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode 100644 index 0000000000..923e165f3f --- /dev/null +++ b/lib/pdcp/version.map @@ -0,0 +1,10 @@ +EXPERIMENTAL { + global: + + # added in 23.07 + rte_pdcp_entity_establish; + rte_pdcp_entity_release; + rte_pdcp_entity_suspend; + + local: *; +}; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 03/21] pdcp: add pre and post-process 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 02/21] lib: add pdcp protocol Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 04/21] pdcp: add packet group Anoob Joseph ` (17 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Functionality of pre-process & post-process varies based on the type of entity. Registration of entity specific function pointer allows skipping multiple checks that would come in datapath otherwise. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/rte_pdcp.h | 97 ++++++++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 ++ 2 files changed, 100 insertions(+) diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 1f96fdc9a1..46c3c2a416 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -22,6 +22,21 @@ extern "C" { #endif +/* Forward declarations */ +struct rte_pdcp_entity; + +/* PDCP pre-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err); + +/* PDCP post-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err); + /** * PDCP entity. * @@ -33,6 +48,10 @@ extern "C" { * plane depending on which radio bearer it is carrying data for. */ struct rte_pdcp_entity { + /** Entity specific pre-process handle. */ + rte_pdcp_pre_p_t pre_process; + /** Entity specific post-process handle. */ + rte_pdcp_post_p_t post_process; /** * PDCP entities may hold packets for purposes of in-order delivery (in * case of receiving PDCP entity) and re-transmission (in case of @@ -160,6 +179,84 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity pre-process the mbufs and prepare + * crypto ops that can be enqueued to the cryptodev associated with given + * session. Only error packets would be moved returned in the input buffer, + * *mb*, and it is the responsibility of the application to free the same. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param[in, out] mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * which contain the input packets. Any error packets would be returned in the + * same buffer. + * @param[out] cop + * The address of an array that can hold up to *num* pointers to + * *rte_crypto_op* structures. Crypto ops would be allocated by + * ``rte_pdcp_pkt_pre_process`` API. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * Pointer to return the number of error packets returned in *mb* + * @return + * Count of crypto_ops prepared + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err) +{ + return entity->pre_process(entity, mb, cop, num, nb_err); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity, perform PDCP post-processing of the + * mbufs. + * + * Input mbufs are the ones retrieved from crypto_ops dequeued from cryptodev + * and grouped by *rte_pdcp_pkt_crypto_group()*. + * + * The post-processed packets would be returned in the *out_mb* buffer. + * The resultant mbufs would be grouped into success packets and error packets. + * Error packets would be grouped in the end of the array and it is the + * responsibility of the application to handle the same. + * + * When in-order delivery is enabled, PDCP entity may buffer packets and would + * deliver packets only when all prior packets have been post-processed. That + * would result in returning more/less packets than enqueued. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param in_mb + * The address of an array of *num* pointers to *rte_mbuf* structures. + * @param[out] out_mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * to output packets after PDCP post-processing. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * The number of error packets returned in *out_mb* buffer. + * @return + * Count of packets returned in *out_mb* buffer. + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return entity->post_process(entity, in_mb, out_mb, num, nb_err); +} + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 923e165f3f..f9ff30600a 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -6,5 +6,8 @@ EXPERIMENTAL { rte_pdcp_entity_release; rte_pdcp_entity_suspend; + rte_pdcp_pkt_post_process; + rte_pdcp_pkt_pre_process; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 04/21] pdcp: add packet group 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (2 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 03/21] pdcp: add pre and post-process Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 05/21] pdcp: add crypto session create and destroy Anoob Joseph ` (16 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Crypto processing in PDCP is performed asynchronously by rte_cryptodev_enqueue_burst() and rte_cryptodev_dequeue_burst(). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 1 + lib/pdcp/rte_pdcp.h | 6 ++ lib/pdcp/rte_pdcp_group.h | 131 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 + 4 files changed, 141 insertions(+) create mode 100644 lib/pdcp/rte_pdcp_group.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index ccaf426240..08679b743a 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -13,5 +13,6 @@ sources = files( 'rte_pdcp.c', ) headers = files('rte_pdcp.h') +indirect_headers += files('rte_pdcp_group.h') deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 46c3c2a416..b88cad4f64 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -257,6 +257,12 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. So include + * in the end. + */ +#include <rte_pdcp_group.h> + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new file mode 100644 index 0000000000..ece3e8c0ff --- /dev/null +++ b/lib/pdcp/rte_pdcp_group.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_GROUP_H +#define RTE_PDCP_GROUP_H + +/** + * @file rte_pdcp_group.h + * + * RTE PDCP grouping support. + * It is not recommended to include this file directly, include <rte_pdcp.h> + * instead. + * Provides helper functions to process completed crypto-ops and group related + * packets by sessions they belong to. + */ + +#include <rte_common.h> +#include <rte_crypto.h> +#include <rte_cryptodev.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Group packets belonging to same PDCP entity. + */ +struct rte_pdcp_group { + union { + uint64_t val; + void *ptr; + } id; /**< Grouped by value */ + struct rte_mbuf **m; /**< Start of the group */ + uint32_t cnt; /**< Number of entries in the group */ + int32_t rc; /**< Status code associated with the group */ +}; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take crypto-op as an input and extract pointer to related PDCP entity. + * @param cop + * The address of an input *rte_crypto_op* structure. + * @return + * The pointer to the related *rte_pdcp_entity* structure. + */ +static inline struct rte_pdcp_entity * +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) +{ + void *sess = cop->sym[0].session; + + return (struct rte_pdcp_entity *)(uintptr_t) + rte_cryptodev_sym_session_opaque_data_get(sess); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take as input completed crypto ops, extract related mbufs and group them by + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* flag + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. + * + * Note that application must ensure only crypto-ops prepared by lib_pdcp is + * provided back to @see rte_pdcp_pkt_crypto_group(). + * + * @param cop + * The address of an array of *num* pointers to the input *rte_crypto_op* + * structures. + * @param[out] mb + * The address of an array of *num* pointers to output *rte_mbuf* structures. + * @param[out] grp + * The address of an array of *num* to output *rte_pdcp_group* structures. + * @param num + * The maximum number of crypto-ops to process. + * @return + * Number of filled elements in *grp* array. + * + */ +static inline uint16_t +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_pdcp_group grp[], uint16_t num) +{ + uint32_t i, j = 0, n = 0; + void *ns, *ps = NULL; + struct rte_mbuf *m; + + for (i = 0; i != num; i++) { + m = cop[i]->sym[0].m_src; + ns = cop[i]->sym[0].session; + + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; + + /* Different entity */ + if (ps != ns) { + + /* Finalize open group and start a new one */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + /* Start new group */ + grp[n].m = mb + j; + ps = ns; + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); + } + + mb[j++] = m; + rte_crypto_op_free(cop[i]); + } + + /* Finalize last group */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + return n; +} + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_GROUP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index f9ff30600a..d564f155e0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,12 +2,15 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_en_from_cop; + rte_pdcp_entity_establish; rte_pdcp_entity_release; rte_pdcp_entity_suspend; rte_pdcp_pkt_post_process; rte_pdcp_pkt_pre_process; + rte_pdcp_pkt_crypto_group; local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 05/21] pdcp: add crypto session create and destroy 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (3 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 04/21] pdcp: add packet group Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 06/21] pdcp: add pre and post process for UL Anoob Joseph ` (15 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to create & destroy sessions. PDCP lib would take crypto transforms as input and creates the session on the corresponding device after verifying capabilities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_crypto.c | 223 ++++++++++++++++++++++++++++++++++++++++- lib/pdcp/pdcp_crypto.h | 5 + 2 files changed, 225 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c index 755e27ec9e..6d2a85dc7d 100644 --- a/lib/pdcp/pdcp_crypto.c +++ b/lib/pdcp/pdcp_crypto.c @@ -2,20 +2,237 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_errno.h> #include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> #include "pdcp_crypto.h" +#include "pdcp_entity.h" + +static int +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, + const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm, + bool is_auth_then_cipher) +{ + uint16_t cipher_iv_len, auth_digest_len, auth_iv_len; + int ret; + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protection is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (c_xfrm == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + + /* With UPLINK, if auth is enabled, it should be before cipher */ + if (a_xfrm != NULL && !is_auth_then_cipher) + return -EINVAL; + + /* With UPLINK, cipher operation must be encrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return -EINVAL; + + /* With UPLINK, auth operation (if present) must be generate */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_GENERATE) + return -EINVAL; + + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + + /* With DOWNLINK, if auth is enabled, it should be after cipher */ + if (a_xfrm != NULL && is_auth_then_cipher) + return -EINVAL; + + /* With DOWNLINK, cipher operation must be decrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) + return -EINVAL; + + /* With DOWNLINK, auth operation (if present) must be verify */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_VERIFY) + return -EINVAL; + + } else { + return -EINVAL; + } + + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) + return -EINVAL; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + cipher_iv_len = 0; + else + cipher_iv_len = PDCP_IV_LEN; + + if (cipher_iv_len != c_xfrm->cipher.iv.length) + return -EINVAL; + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + return -EINVAL; + + /* For AUTH NULL, lib PDCP would add 4 byte 0s */ + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + auth_digest_len = 0; + else + auth_digest_len = RTE_PDCP_MAC_I_LEN; + + if (auth_digest_len != a_xfrm->auth.digest_length) + return -EINVAL; + + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + auth_iv_len = PDCP_IV_LEN; + else + auth_iv_len = 0; + + if (a_xfrm->auth.iv.length != auth_iv_len) + return -EINVAL; + } + + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) + return -EINVAL; + + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); + if (ret) + return -ENOTSUP; + + if (a_xfrm != NULL) { + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); + if (ret) + return -ENOTSUP; + } + + return 0; +} int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { - RTE_SET_USED(entity); - RTE_SET_USED(conf); + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + struct entity_priv *en_priv; + bool is_auth_then_cipher; + int ret; + + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + en_priv->dev_id = conf->dev_id; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + c_xfrm = conf->crypto_xfrm; + a_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = false; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + a_xfrm = conf->crypto_xfrm; + c_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = true; + } else { + return -EINVAL; + } + + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, is_auth_then_cipher); + if (ret) + return ret; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + c_xfrm->cipher.iv.offset = 0; + else + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; + + if (a_xfrm != NULL) { + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm->auth.iv.offset = 0; + else + if (c_xfrm->cipher.iv.offset) + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + PDCP_IV_LEN; + else + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; + } + + if (conf->sess_mpool == NULL) + return -EINVAL; + + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf->dev_id, conf->crypto_xfrm, + conf->sess_mpool); + if (en_priv->crypto_sess == NULL) { + /* rte_errno is set as positive values of error codes */ + return -rte_errno; + } + + rte_cryptodev_sym_session_opaque_data_set(en_priv->crypto_sess, (uint64_t)entity); + return 0; } void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) { - RTE_SET_USED(entity); + struct entity_priv *en_priv; + + en_priv = entity_priv_get(entity); + + if (en_priv->crypto_sess != NULL) { + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv->crypto_sess); + en_priv->crypto_sess = NULL; + } } diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h index 6563331d37..f694818713 100644 --- a/lib/pdcp/pdcp_crypto.h +++ b/lib/pdcp/pdcp_crypto.h @@ -5,8 +5,13 @@ #ifndef PDCP_CRYPTO_H #define PDCP_CRYPTO_H +#include <rte_crypto.h> +#include <rte_crypto_sym.h> #include <rte_pdcp.h> +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op)) +#define PDCP_IV_LEN 16 + int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 06/21] pdcp: add pre and post process for UL 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (4 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 05/21] pdcp: add crypto session create and destroy Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 07/21] pdcp: add pre and post process for DL Anoob Joseph ` (14 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to perform pre & post processing based on the type of entity. To avoid checks in datapath, there are different function pointers registered based on the following, 1. Control plane v/s user plane 2. 12 bit v/s 18 bit SN For control plane only 12 bit SN need to be supported (as per PDCP specification). Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_entity.h | 24 +++ lib/pdcp/pdcp_process.c | 334 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 358 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 000297588f..23628ebad4 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -92,22 +92,46 @@ pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static inline uint32_t +pdcp_window_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return 1 << (sn_size - 1); +} + static inline uint32_t pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return (1 << sn_size) - 1; } +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + static inline uint32_t pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return ~pdcp_sn_mask_get(sn_size); } +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline uint32_t pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) { return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); } +static inline uint32_t +pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << (32 - sn_size)) - 1; +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 79f5dce5db..9b7de39db6 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -36,6 +36,336 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static inline void +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, + uint8_t data_offset, uint32_t count, const bool is_auth) +{ + const struct rte_crypto_op cop_init = { + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, + }; + struct rte_crypto_sym_op *op; + uint32_t pkt_len; + + const uint8_t cipher_shift = 3 * en_priv->flags.is_cipher_in_bits; + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; + + op = cop->sym; + cop->raw = cop_init.raw; + op->m_src = mb; + op->m_dst = mb; + + /* Set IV */ + en_priv->iv_gen(cop, en_priv, count); + + /* Prepare op */ + pkt_len = rte_pktmbuf_pkt_len(mb); + op->cipher.data.offset = data_offset << cipher_shift; + op->cipher.data.length = (pkt_len - data_offset) << cipher_shift; + + if (is_auth) { + op->auth.data.offset = 0; + op->auth.data.length = (pkt_len - RTE_PDCP_MAC_I_LEN) << auth_shift; + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t *, + (pkt_len - RTE_PDCP_MAC_I_LEN)); + } + + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); +} + +static inline bool +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + return true; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline bool +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + return true; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count, sn; + uint8_t *mac_i; + int i; + + const uint8_t hdr_sz = en_priv->hdr_sz; + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + /* Update sequence number in the PDU header */ + count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static int +pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + entity->pre_process = NULL; + entity->post_process = NULL; + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + entity->post_process = pdcp_post_process_ul; + } + + if (entity->pre_process == NULL || entity->post_process == NULL) + return -ENOTSUP; + + return 0; +} + static int pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) { @@ -134,5 +464,9 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti if (ret) return ret; + ret = pdcp_pre_post_func_set(entity, conf); + if (ret) + return ret; + return 0; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 07/21] pdcp: add pre and post process for DL 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (5 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 06/21] pdcp: add pre and post process for UL Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 08/21] pdcp: add IV generation routines Anoob Joseph ` (13 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to perform pre & post processing for down link entities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 384 ++++++++++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 11 ++ lib/pdcp/rte_pdcp.c | 23 +++ 4 files changed, 420 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 23628ebad4..1d4a43a3bc 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -13,6 +13,8 @@ struct entity_priv; +#define PDCP_HFN_MIN 0 + /* IV generation function based on the entity configuration */ typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count); diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 9b7de39db6..bd75e6f802 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -6,6 +6,7 @@ #include <rte_crypto_sym.h> #include <rte_cryptodev.h> #include <rte_memcpy.h> +#include <rte_mbuf_dyn.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> @@ -333,9 +334,353 @@ pdcp_post_process_ul(const struct rte_pdcp_entity *entity, return nb_success; } +static inline int +pdcp_sn_count_get(const uint32_t rx_deliv, int32_t rsn, uint32_t *count, + const enum rte_security_pdcp_sn_size sn_size) +{ + const uint32_t rx_deliv_sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + const uint32_t window_sz = pdcp_window_size_get(sn_size); + uint32_t rhfn; + + rhfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + + if (rsn < (int32_t)(rx_deliv_sn - window_sz)) { + if (unlikely(rhfn == pdcp_hfn_max(sn_size))) + return -ERANGE; + rhfn += 1; + } else if ((uint32_t)rsn >= (rx_deliv_sn + window_sz)) { + if (unlikely(rhfn == PDCP_HFN_MIN)) + return -ERANGE; + rhfn -= 1; + } + + *count = pdcp_count_from_hfn_sn_get(rhfn, rsn, sn_size); + + return 0; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_18))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline void +pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool trim_mac) +{ + char *p = rte_pktmbuf_adj(mb, hdr_trim_sz); + RTE_ASSERT(p != NULL); + RTE_SET_USED(p); + + if (trim_mac) { + int ret = rte_pktmbuf_trim(mb, RTE_PDCP_MAC_I_LEN); + RTE_ASSERT(ret == 0); + RTE_SET_USED(ret); + } +} + +static inline bool +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, + const uint32_t count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + + if (count < en_priv->state.rx_deliv) + return false; + + /* t-Reordering timer is not supported - SDU will be delivered immediately. + * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not + * been delivered to upper layers + */ + en_priv->state.rx_next = count + 1; + + if (count >= en_priv->state.rx_next) + en_priv->state.rx_next = count + 1; + + return true; +} + +static inline uint16_t +pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + mb_dynfield = pdcp_dynfield(mb); + count = *mb_dynfield; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static uint16_t +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + mb_dynfield = pdcp_dynfield(mb); + count = *mb_dynfield; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, true); + + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + static int pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { + struct entity_priv *en_priv = entity_priv_get(entity); + entity->pre_process = NULL; entity->post_process = NULL; @@ -346,6 +691,13 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; + entity->post_process = pdcp_post_process_cplane_sn_12_dl; + } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { @@ -360,6 +712,38 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; + entity->post_process = pdcp_post_process_uplane_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; + entity->post_process = pdcp_post_process_uplane_dl; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; + entity->post_process = pdcp_post_process_uplane_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; + entity->post_process = pdcp_post_process_uplane_dl; + } + if (entity->pre_process == NULL || entity->post_process == NULL) return -ENOTSUP; diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index fd53fff0aa..a52f769b82 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -5,8 +5,19 @@ #ifndef PDCP_PROCESS_H #define PDCP_PROCESS_H +#include <rte_mbuf_dyn.h> #include <rte_pdcp.h> +typedef uint32_t rte_pdcp_dynfield_t; + +extern int rte_pdcp_dynfield_offset; + +static inline rte_pdcp_dynfield_t * +pdcp_dynfield(struct rte_mbuf *mbuf) +{ + return RTE_MBUF_DYNFIELD(mbuf, rte_pdcp_dynfield_offset, rte_pdcp_dynfield_t *); +} + int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index adcad5dd25..91dab91f73 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -10,6 +10,26 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +#define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" + +int rte_pdcp_dynfield_offset = -1; + +static int +pdcp_dynfield_register(void) +{ + const struct rte_mbuf_dynfield dynfield_desc = { + .name = RTE_PDCP_DYNFIELD_NAME, + .size = sizeof(rte_pdcp_dynfield_t), + .align = __alignof__(rte_pdcp_dynfield_t), + }; + + if (rte_pdcp_dynfield_offset != -1) + return rte_pdcp_dynfield_offset; + + rte_pdcp_dynfield_offset = rte_mbuf_dynfield_register(&dynfield_desc); + return rte_pdcp_dynfield_offset; +} + static int pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { @@ -35,6 +55,9 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) int ret, entity_size; uint32_t count; + if (pdcp_dynfield_register() < 0) + return NULL; + if (conf == NULL || conf->cop_pool == NULL) { rte_errno = EINVAL; return NULL; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 08/21] pdcp: add IV generation routines 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (6 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 07/21] pdcp: add pre and post process for DL Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 09/21] app/test: add lib pdcp tests Anoob Joseph ` (12 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger For PDCP, IV generated has varying formats depending on the ciphering and authentication algorithm used. Add routines to populate IV accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 87 ++++++++++++ lib/pdcp/pdcp_process.c | 284 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 371 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 1d4a43a3bc..10a72faae1 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -26,6 +26,89 @@ struct entity_state { uint32_t rx_reord; }; +union auth_iv_partial { + /* For AES-CMAC, there is no IV, but message gets prepended */ + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + } aes_cmac; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_37_39 : 3; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_65_71 : 7; + uint64_t direction_64 : 1; + uint64_t rsvd_72_111 : 40; + uint64_t rsvd_113_119 : 7; + uint64_t direction_112 : 1; + uint64_t rsvd_120_127 : 8; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t zero_37_39 : 3; + uint64_t zero_40_63 : 24; + + uint64_t direction_64 : 1; + uint64_t rsvd_65_71 : 7; + uint64_t rsvd_72_111 : 40; + uint64_t direction_112 : 1; + uint64_t rsvd_113_119 : 7; + uint64_t rsvd_120_127 : 8; +#endif + } zs; + uint64_t u64[2]; +}; + +union cipher_iv_partial { + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t zero_64_127; + } aes_ctr; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t rsvd_64_127; + } zs; + uint64_t u64[2]; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -35,6 +118,10 @@ struct entity_priv { struct rte_cryptodev_sym_session *crypto_sess; /** Entity specific IV generation function. */ iv_gen_t iv_gen; + /** Pre-prepared auth IV. */ + union auth_iv_partial auth_iv_part; + /** Pre-prepared cipher IV. */ + union cipher_iv_partial cipher_iv_part; /** Entity state variables. */ struct entity_state state; /** Flags. */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index bd75e6f802..28ac4102da 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -14,6 +14,181 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +/* Enum of supported algorithms for ciphering */ +enum pdcp_cipher_algo { + PDCP_CIPHER_ALGO_NULL, + PDCP_CIPHER_ALGO_AES, + PDCP_CIPHER_ALGO_ZUC, + PDCP_CIPHER_ALGO_SNOW3G, + PDCP_CIPHER_ALGO_MAX +}; + +/* Enum of supported algorithms for integrity */ +enum pdcp_auth_algo { + PDCP_AUTH_ALGO_NULL, + PDCP_AUTH_ALGO_AES, + PDCP_AUTH_ALGO_ZUC, + PDCP_AUTH_ALGO_SNOW3G, + PDCP_AUTH_ALGO_MAX +}; + +/* IV generation functions based on type of operation (cipher - auth) */ + +static void +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + /* No IV required for NULL cipher + NULL auth */ + RTE_SET_USED(cop); + RTE_SET_USED(en_priv); + RTE_SET_USED(count); +} + +static void +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr; + uint64_t m; + + /* AES-CMAC requires message to be prepended with info on count etc */ + + /* Prepend by 8 bytes to add custom message */ + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, 16); +} + +static void +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64 = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64, 8); + rte_memcpy(iv + 8, &iv_u64, 8); +} + +static void +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + /* Generating cipher IV */ + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + iv += PDCP_IV_LEN; + + /* Generating auth IV */ + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + iv += PDCP_IV_LEN; + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + static int pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, struct rte_crypto_sym_xform **a_xfrm) @@ -37,6 +212,111 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static int +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + enum rte_security_pdcp_direction direction; + enum pdcp_cipher_algo cipher_algo; + enum pdcp_auth_algo auth_algo; + struct entity_priv *en_priv; + int ret; + + en_priv = entity_priv_get(entity); + + direction = conf->pdcp_xfrm.pkt_dir; + if (conf->reverse_iv_direction) + direction = !direction; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (c_xfrm == NULL) + return -EINVAL; + + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); + + switch (c_xfrm->cipher.algo) { + case RTE_CRYPTO_CIPHER_NULL: + cipher_algo = PDCP_CIPHER_ALGO_NULL; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + cipher_algo = PDCP_CIPHER_ALGO_AES; + en_priv->cipher_iv_part.aes_ctr.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.aes_ctr.direction = direction; + break; + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + cipher_algo = PDCP_CIPHER_ALGO_SNOW3G; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + cipher_algo = PDCP_CIPHER_ALGO_ZUC; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + default: + return -ENOTSUP; + } + + if (a_xfrm != NULL) { + switch (a_xfrm->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + auth_algo = PDCP_AUTH_ALGO_NULL; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + auth_algo = PDCP_AUTH_ALGO_AES; + en_priv->auth_iv_part.aes_cmac.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.aes_cmac.direction = direction; + break; + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + auth_algo = PDCP_AUTH_ALGO_SNOW3G; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + case RTE_CRYPTO_AUTH_ZUC_EIA3: + auth_algo = PDCP_AUTH_ALGO_ZUC; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + default: + return -ENOTSUP; + } + } else { + auth_algo = PDCP_AUTH_ALGO_NULL; + } + + static const iv_gen_t iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_null_null, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_null_aes_cmac, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_null_zs, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_null_zs, + + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_aes_ctr_null, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_aes_ctr_aes_cmac, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_aes_ctr_zs, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_aes_ctr_zs, + + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + }; + + en_priv->iv_gen = iv_gen_map[cipher_algo][auth_algo]; + + return 0; +} + static inline void cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, uint8_t data_offset, uint32_t count, const bool is_auth) @@ -844,6 +1124,10 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti en_priv = entity_priv_get(entity); + ret = pdcp_iv_gen_func_set(entity, conf); + if (ret) + return ret; + ret = pdcp_entity_priv_populate(en_priv, conf); if (ret) return ret; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 09/21] app/test: add lib pdcp tests 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (7 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 08/21] pdcp: add IV generation routines Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph ` (11 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add tests to verify lib PDCP operations. Tests leverage existing PDCP test vectors. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 1 + app/test/meson.build | 5 + app/test/test_pdcp.c | 732 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 738 insertions(+) create mode 100644 app/test/test_pdcp.c diff --git a/MAINTAINERS b/MAINTAINERS index 85a3b94644..15dd88af3a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1554,6 +1554,7 @@ M: Anoob Joseph <anoobj@marvell.com> M: Volodymyr Fialko <vfialko@marvell.com> T: git://dpdk.org/next/dpdk-next-crypto F: lib/pdcp/ +F: app/test/test_pdcp* Packet Framework diff --git a/app/test/meson.build b/app/test/meson.build index b9b5432496..2894793347 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -435,6 +435,11 @@ if dpdk_conf.has('RTE_HAS_LIBPCAP') endif endif +if dpdk_conf.has('RTE_LIB_PDCP') + test_sources += 'test_pdcp.c' + fast_tests += [['pdcp_autotest', false, true]] +endif + if cc.has_argument('-Wno-format-truncation') cflags += '-Wno-format-truncation' endif diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c new file mode 100644 index 0000000000..34b759eaef --- /dev/null +++ b/app/test/test_pdcp.c @@ -0,0 +1,732 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_malloc.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "test.h" +#include "test_cryptodev.h" +#include "test_cryptodev_security_pdcp_test_vectors.h" + +#define NB_DESC 1024 +#define CDEV_INVALID_ID UINT8_MAX +#define NB_TESTS RTE_DIM(pdcp_test_params) +#define PDCP_IV_LEN 16 + +struct pdcp_testsuite_params { + struct rte_mempool *mbuf_pool; + struct rte_mempool *cop_pool; + struct rte_mempool *sess_pool; + bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; +}; + +static struct pdcp_testsuite_params testsuite_params; + +struct pdcp_test_conf { + struct rte_pdcp_entity_conf entity; + struct rte_crypto_sym_xform c_xfrm; + struct rte_crypto_sym_xform a_xfrm; + bool is_integrity_protected; + uint8_t input[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t input_len; + uint8_t output[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t output_len; +}; + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static int +cryptodev_init(int dev_id) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_cryptodev_qp_conf qp_conf; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_config config; + int ret, socket_id; + + /* Check if device was already initialized */ + if (ts_params->cdevs_used[dev_id]) + return 0; + + rte_cryptodev_info_get(dev_id, &dev_info); + + if (dev_info.max_nb_queue_pairs < 1) { + RTE_LOG(ERR, USER1, "Cryptodev doesn't have sufficient queue pairs available\n"); + return -ENODEV; + } + + socket_id = rte_socket_id(); + + memset(&config, 0, sizeof(config)); + config.nb_queue_pairs = 1; + config.socket_id = socket_id; + + ret = rte_cryptodev_configure(dev_id, &config); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure cryptodev - %d\n", dev_id); + return -ENODEV; + } + + memset(&qp_conf, 0, sizeof(qp_conf)); + qp_conf.nb_descriptors = NB_DESC; + + ret = rte_cryptodev_queue_pair_setup(dev_id, 0, &qp_conf, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure queue pair\n"); + return -ENODEV; + } + + ret = rte_cryptodev_start(dev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not start cryptodev\n"); + return -ENODEV; + } + + /* Mark device as initialized */ + ts_params->cdevs_used[dev_id] = true; + + return 0; +} + +static void +cryptodev_fini(int dev_id) +{ + rte_cryptodev_stop(dev_id); +} + +static unsigned int +cryptodev_sess_priv_max_req_get(void) +{ + struct rte_cryptodev_info info; + unsigned int sess_priv_sz; + int i, nb_dev; + void *sec_ctx; + + nb_dev = rte_cryptodev_count(); + + sess_priv_sz = 0; + + for (i = 0; i < nb_dev; i++) { + rte_cryptodev_info_get(i, &info); + sess_priv_sz = RTE_MAX(sess_priv_sz, rte_cryptodev_sym_get_private_session_size(i)); + if (info.feature_flags & RTE_CRYPTODEV_FF_SECURITY) { + sec_ctx = rte_cryptodev_get_sec_ctx(i); + sess_priv_sz = RTE_MAX(sess_priv_sz, + rte_security_session_get_size(sec_ctx)); + } + } + + return sess_priv_sz; +} + +static int +testsuite_setup(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + int nb_cdev, sess_priv_size, nb_sess = 1024; + + RTE_SET_USED(pdcp_test_hfn_threshold); + + nb_cdev = rte_cryptodev_count(); + if (nb_cdev < 1) { + RTE_LOG(ERR, USER1, "No crypto devices found.\n"); + return TEST_SKIPPED; + } + + memset(ts_params, 0, sizeof(*ts_params)); + + ts_params->mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NUM_MBUFS, MBUF_CACHE_SIZE, 0, + MBUF_SIZE, SOCKET_ID_ANY); + if (ts_params->mbuf_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf pool\n"); + return TEST_FAILED; + } + + ts_params->cop_pool = rte_crypto_op_pool_create("cop_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, MBUF_CACHE_SIZE, + 2 * MAXIMUM_IV_LENGTH, SOCKET_ID_ANY); + if (ts_params->cop_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create crypto_op pool\n"); + goto mbuf_pool_free; + } + + /* Get max session priv size required */ + sess_priv_size = cryptodev_sess_priv_max_req_get(); + + ts_params->sess_pool = rte_cryptodev_sym_session_pool_create("sess_pool", nb_sess, + sess_priv_size, + RTE_MEMPOOL_CACHE_MAX_SIZE, + 0, SOCKET_ID_ANY); + if (ts_params->sess_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create session pool\n"); + goto cop_pool_free; + } + + return 0; + +cop_pool_free: + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; +mbuf_pool_free: + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; + return TEST_FAILED; +} + +static void +testsuite_teardown(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + uint8_t dev_id; + + for (dev_id = 0; dev_id < RTE_CRYPTO_MAX_DEVS; dev_id++) { + if (ts_params->cdevs_used[dev_id]) + cryptodev_fini(dev_id); + } + + rte_mempool_free(ts_params->sess_pool); + ts_params->sess_pool = NULL; + + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; + + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; +} + +static int +ut_setup_pdcp(void) +{ + return 0; +} + +static void +ut_teardown_pdcp(void) +{ +} + +static int +crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +cryptodev_id_get(bool is_integrity_protected, const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm) +{ + int i, nb_devs; + + nb_devs = rte_cryptodev_count(); + + /* Check capabilities */ + + for (i = 0; i < nb_devs; i++) { + if ((crypto_caps_cipher_verify(i, c_xfrm) == 0) && + (!is_integrity_protected || crypto_caps_auth_verify(i, a_xfrm) == 0)) + break; + } + + if (i == nb_devs) + return -1; + + return i; +} + +static int +pdcp_known_vec_verify(struct rte_mbuf *m, const uint8_t *expected, uint32_t expected_pkt_len) +{ + uint8_t *actual = rte_pktmbuf_mtod(m, uint8_t *); + uint32_t actual_pkt_len = rte_pktmbuf_pkt_len(m); + + debug_hexdump(stdout, "Received:", actual, actual_pkt_len); + debug_hexdump(stdout, "Expected:", expected, expected_pkt_len); + + TEST_ASSERT_EQUAL(actual_pkt_len, expected_pkt_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_pkt_len, actual_pkt_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(actual, expected, expected_pkt_len, + "Generated packet not as expected"); + + return 0; +} + +static struct rte_crypto_op * +process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op) +{ + if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) { + RTE_LOG(ERR, USER1, "Error sending packet to cryptodev\n"); + return NULL; + } + + op = NULL; + + while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0) + rte_pause(); + + return op; +} + +static uint32_t +pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) +{ + uint32_t sn = 0; + + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + sn = rte_cpu_to_be_16(*(const uint16_t *)data); + sn = sn & 0xfff; + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + sn = rte_cpu_to_be_32(*(const uint32_t *)data); + sn = (sn & 0x3ffff00) >> 8; + } + + return sn; +} + +static int +create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_crypto_sym_xform c_xfrm, a_xfrm; + uint32_t sn, expected_len; + uint8_t *data, *expected; + int pdcp_hdr_sz; + + memset(conf, 0, sizeof(*conf)); + memset(&c_xfrm, 0, sizeof(c_xfrm)); + memset(&a_xfrm, 0, sizeof(a_xfrm)); + + conf->entity.sess_mpool = ts_params->sess_pool; + conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; + conf->entity.pdcp_xfrm.en_ordering = 0; + conf->entity.pdcp_xfrm.remove_duplicates = 0; + conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + + if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; + else + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + + conf->entity.pdcp_xfrm.sn_size = pdcp_test_data_sn_size[index]; + + /* Zero initialize unsupported flags */ + conf->entity.pdcp_xfrm.hfn_threshold = 0; + conf->entity.pdcp_xfrm.hfn_ovrd = 0; + conf->entity.pdcp_xfrm.sdap_enabled = 0; + + c_xfrm.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + c_xfrm.cipher.algo = pdcp_test_params[index].cipher_alg; + c_xfrm.cipher.key.length = pdcp_test_params[index].cipher_key_len; + c_xfrm.cipher.key.data = pdcp_test_crypto_key[index]; + + a_xfrm.type = RTE_CRYPTO_SYM_XFORM_AUTH; + + if (pdcp_test_params[index].auth_alg == 0) { + conf->is_integrity_protected = false; + } else { + a_xfrm.auth.algo = pdcp_test_params[index].auth_alg; + a_xfrm.auth.key.data = pdcp_test_auth_key[index]; + a_xfrm.auth.key.length = pdcp_test_params[index].auth_key_len; + conf->is_integrity_protected = true; + } + + pdcp_hdr_sz = pdcp_hdr_size_get(pdcp_test_data_sn_size[index]); + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protecting is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (conf->is_integrity_protected) { + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + conf->entity.crypto_xfrm = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + a_xfrm.next = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + c_xfrm.next = NULL; + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + c_xfrm.next = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + a_xfrm.next = NULL; + } + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + c_xfrm.next = NULL; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + else + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + /* Update xforms to match PDCP requirements */ + + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) + c_xfrm.cipher.iv.length = PDCP_IV_LEN; + else + c_xfrm.cipher.iv.length = 0; + + if (conf->is_integrity_protected) { + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm.auth.digest_length = 0; + else + a_xfrm.auth.digest_length = RTE_PDCP_MAC_I_LEN; + + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + a_xfrm.auth.iv.length = PDCP_IV_LEN; + else + a_xfrm.auth.iv.length = 0; + } + + conf->c_xfrm = c_xfrm; + conf->a_xfrm = a_xfrm; + + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf->is_integrity_protected, + &conf->c_xfrm, &conf->a_xfrm); + + if (pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_CONTROL || + pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_DATA) { + data = pdcp_test_data_in[index]; + sn = pdcp_sn_from_raw_get(data, pdcp_test_data_sn_size[index]); + conf->entity.pdcp_xfrm.hfn = pdcp_test_hfn[index]; + conf->entity.sn = sn; + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", pdcp_test_data_in[index], + pdcp_test_data_in_len[index]); +#endif + /* Since the vectors available already have PDCP header, trim the same */ + conf->input_len = pdcp_test_data_in_len[index] - pdcp_hdr_sz; + memcpy(conf->input, pdcp_test_data_in[index] + pdcp_hdr_sz, conf->input_len); + } else { + conf->input_len = pdcp_test_data_in_len[index]; + + if (conf->is_integrity_protected) + conf->input_len += RTE_PDCP_MAC_I_LEN; + + memcpy(conf->input, pdcp_test_data_out[index], conf->input_len); +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", conf->input, conf->input_len); +#endif + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected = pdcp_test_data_out[index]; + else + expected = pdcp_test_data_in[index]; + + /* Calculate expected packet length */ + expected_len = pdcp_test_data_in_len[index]; + + /* In DL processing, PDCP header would be stripped */ + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + expected += pdcp_hdr_sz; + expected_len -= pdcp_hdr_sz; + } + + /* In UL processing with integrity protection, MAC would be added */ + if (conf->is_integrity_protected && + conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected_len += 4; + + memcpy(conf->output, expected, expected_len); + conf->output_len = expected_len; + + return 0; +} + +static struct rte_pdcp_entity* +test_entity_create(const struct pdcp_test_conf *t_conf, int *rc) +{ + struct rte_pdcp_entity *pdcp_entity; + int ret; + + if (t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12 && + t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18) { + *rc = -ENOTSUP; + return NULL; + } + + if (t_conf->entity.dev_id == CDEV_INVALID_ID) { + RTE_LOG(DEBUG, USER1, "Could not find device with required capabilities\n"); + *rc = -ENOTSUP; + return NULL; + } + + ret = cryptodev_init(t_conf->entity.dev_id); + if (ret) { + *rc = ret; + RTE_LOG(DEBUG, USER1, "Could not initialize cryptodev\n"); + return NULL; + } + + rte_errno = 0; + + pdcp_entity = rte_pdcp_entity_establish(&t_conf->entity); + if (pdcp_entity == NULL) { + *rc = -rte_errno; + RTE_LOG(DEBUG, USER1, "Could not establish PDCP entity\n"); + return NULL; + } + + return pdcp_entity; +} + +static uint16_t +test_process_packets(const struct rte_pdcp_entity *pdcp_entity, uint8_t cdev_id, + struct rte_mbuf *in_mb[], uint16_t nb_in, + struct rte_mbuf *out_mb[], uint16_t *nb_err) +{ + struct rte_crypto_op *cop, *cop_out; + struct rte_pdcp_group grp[1]; + uint16_t nb_success, nb_grp; + struct rte_mbuf *mbuf, *mb; + + if (nb_in != 1) + return -ENOTSUP; + + mbuf = in_mb[0]; + + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, &mbuf, &cop_out, 1, nb_err); + if (nb_success != 1 || *nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not pre process PDCP packet\n"); + return TEST_FAILED; + } + +#ifdef VEC_DUMP + printf("Pre-processed vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, rte_pktmbuf_pkt_len(mbuf)); +#endif + + cop = process_crypto_request(cdev_id, cop_out); + if (cop == NULL) { + RTE_LOG(ERR, USER1, "Could not process crypto request\n"); + return -EIO; + } + + grp[0].id.val = 0; + + nb_grp = rte_pdcp_pkt_crypto_group(&cop_out, &mb, grp, 1); + if (nb_grp != 1 || grp[0].cnt != 1) { + RTE_LOG(ERR, USER1, "Could not group PDCP crypto results\n"); + return -ENOTRECOVERABLE; + } + + if ((uintptr_t)pdcp_entity != grp[0].id.val) { + RTE_LOG(ERR, USER1, "PDCP entity not matching the one from crypto_op\n"); + return -ENOTRECOVERABLE; + } + +#ifdef VEC_DUMP + printf("Crypto processed vector:\n"); + rte_pktmbuf_dump(stdout, cop->sym->m_dst, rte_pktmbuf_pkt_len(mbuf)); +#endif + + return rte_pdcp_pkt_post_process(grp[0].id.ptr, grp[0].m, out_mb, grp[0].cnt, nb_err); +} + +static struct rte_mbuf* +mbuf_from_data_create(uint8_t *data, uint16_t data_len) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_mbuf *mbuf; + uint8_t *input_text; + + mbuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (mbuf == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf\n"); + return NULL; + } + + memset(rte_pktmbuf_mtod(mbuf, uint8_t *), 0, rte_pktmbuf_tailroom(mbuf)); + + input_text = (uint8_t *)rte_pktmbuf_append(mbuf, data_len); + memcpy(input_text, data, data_len); + + return mbuf; +} + +static int +test_attempt_single(struct pdcp_test_conf *t_conf) +{ + struct rte_mbuf *mbuf, **out_mb = NULL; + struct rte_pdcp_entity *pdcp_entity; + uint16_t nb_success, nb_err; + int ret = 0, nb_max_out_mb; + + pdcp_entity = test_entity_create(t_conf, &ret); + if (pdcp_entity == NULL) + goto exit; + + /* Allocate buffer for holding mbufs returned */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate buffer for holding out_mb buffers\n"); + ret = -ENOMEM; + goto entity_release; + } + + mbuf = mbuf_from_data_create(t_conf->input, t_conf->input_len); + if (mbuf == NULL) { + ret = -ENOMEM; + goto entity_release; + } + +#ifdef VEC_DUMP + printf("Adjusted vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, t_conf->input_len); +#endif + + nb_success = test_process_packets(pdcp_entity, t_conf->entity.dev_id, &mbuf, 1, out_mb, + &nb_err); + if (nb_success != 1 || nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not process PDCP packet\n"); + ret = TEST_FAILED; + goto mbuf_free; + } + + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + + ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); + if (ret) { + RTE_LOG(DEBUG, USER1, "Could not suspend PDCP entity\n"); + goto mbuf_free; + } + +mbuf_free: + rte_pktmbuf_free(mbuf); +entity_release: + rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_free(out_mb); +exit: + return ret; +} + +static int +run_test_for_one_known_vec(const void *arg) +{ + struct pdcp_test_conf test_conf; + int i = *(const uint32_t *)arg; + + create_test_conf_from_index(i, &test_conf); + return test_attempt_single(&test_conf); +} + +struct unit_test_suite *test_suites[] = { + NULL, /* Place holder for known_vector_cases */ + NULL /* End of suites list */ +}; + +static struct unit_test_suite pdcp_testsuite = { + .suite_name = "PDCP Unit Test Suite", + .unit_test_cases = {TEST_CASES_END()}, + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_suites = test_suites, +}; + +static int +test_pdcp(void) +{ + struct unit_test_suite *known_vector_cases; + int ret, index[NB_TESTS]; + uint32_t i, size; + + size = sizeof(struct unit_test_suite); + size += (NB_TESTS + 1) * sizeof(struct unit_test_case); + + known_vector_cases = rte_zmalloc(NULL, size, 0); + if (known_vector_cases == NULL) + return TEST_FAILED; + + known_vector_cases->suite_name = "Known vector cases"; + + for (i = 0; i < NB_TESTS; i++) { + index[i] = i; + known_vector_cases->unit_test_cases[i].name = pdcp_test_params[i].name; + known_vector_cases->unit_test_cases[i].data = (void *)&index[i]; + known_vector_cases->unit_test_cases[i].enabled = 1; + known_vector_cases->unit_test_cases[i].setup = ut_setup_pdcp; + known_vector_cases->unit_test_cases[i].teardown = ut_teardown_pdcp; + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data + = run_test_for_one_known_vec; + } + + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data = NULL; + + test_suites[0] = known_vector_cases; + + ret = unit_test_suite_runner(&pdcp_testsuite); + + rte_free(known_vector_cases); + return ret; +} + +REGISTER_TEST_COMMAND(pdcp_autotest, test_pdcp); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 10/21] test/pdcp: pdcp HFN tests in combined mode 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (8 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 09/21] app/test: add lib pdcp tests Anoob Joseph @ 2023-05-27 7:15 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 11/21] doc: add PDCP library guide Anoob Joseph ` (10 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:15 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add tests to verify HFN/SN behaviour. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 302 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 299 insertions(+), 3 deletions(-) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 34b759eaef..cfe2ec6aa9 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,9 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* According to formula(7.2.a Window_Size) */ +#define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) + struct pdcp_testsuite_params { struct rte_mempool *mbuf_pool; struct rte_mempool *cop_pool; @@ -36,12 +39,69 @@ struct pdcp_test_conf { uint32_t output_len; }; +static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf); + +typedef int (*test_with_conf_t)(struct pdcp_test_conf *conf); + +static int +run_test_foreach_known_vec(test_with_conf_t test, bool stop_on_first_pass) +{ + struct pdcp_test_conf test_conf; + bool all_tests_skipped = true; + uint32_t i; + int ret; + + for (i = 0; i < NB_TESTS; i++) { + create_test_conf_from_index(i, &test_conf); + ret = test(&test_conf); + + if (ret == TEST_FAILED) { + printf("[%03i] - %s - failed\n", i, pdcp_test_params[i].name); + return TEST_FAILED; + } + + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + continue; + + if (stop_on_first_pass) + return TEST_SUCCESS; + + all_tests_skipped = false; + } + + if (all_tests_skipped) + return TEST_SKIPPED; + + return TEST_SUCCESS; +} + +static int +run_test_with_all_known_vec(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, false); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static int +pktmbuf_read_into(const struct rte_mbuf *m, void *buf, size_t buf_len) +{ + if (m->pkt_len > buf_len) + return -ENOMEM; + + const void *read = rte_pktmbuf_read(m, 0, m->pkt_len, buf); + if (read != NULL && read != buf) + memcpy(buf, read, m->pkt_len); + + return 0; +} + static int cryptodev_init(int dev_id) { @@ -326,6 +386,21 @@ pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) return sn; } +static void +pdcp_sn_to_raw_set(void *data, uint32_t sn, int size) +{ + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr = data; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr = data; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } +} + static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) { @@ -648,9 +723,17 @@ test_attempt_single(struct pdcp_test_conf *t_conf) goto mbuf_free; } - ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); - if (ret) - goto mbuf_free; + /* If expected output provided - verify, else - store for future use */ + if (t_conf->output_len) { + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + } else { + ret = pktmbuf_read_into(mbuf, t_conf->output, RTE_PDCP_CTRL_PDU_SIZE_MAX); + if (ret) + goto mbuf_free; + t_conf->output_len = mbuf->pkt_len; + } ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); if (ret) { @@ -667,6 +750,193 @@ test_attempt_single(struct pdcp_test_conf *t_conf) return ret; } +static void +uplink_to_downlink_convert(const struct pdcp_test_conf *ul_cfg, + struct pdcp_test_conf *dl_cfg) +{ + assert(ul_cfg->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK); + + memcpy(dl_cfg, ul_cfg, sizeof(*dl_cfg)); + dl_cfg->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + dl_cfg->entity.reverse_iv_direction = false; + + if (dl_cfg->is_integrity_protected) { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + dl_cfg->c_xfrm.next = &dl_cfg->a_xfrm; + + dl_cfg->a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + dl_cfg->a_xfrm.next = NULL; + } else { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + dl_cfg->c_xfrm.next = NULL; + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + dl_cfg->entity.dev_id = (uint8_t)cryptodev_id_get(dl_cfg->is_integrity_protected, + &dl_cfg->c_xfrm, &dl_cfg->a_xfrm); + + memcpy(dl_cfg->input, ul_cfg->output, ul_cfg->output_len); + dl_cfg->input_len = ul_cfg->output_len; + + memcpy(dl_cfg->output, ul_cfg->input, ul_cfg->input_len); + dl_cfg->output_len = ul_cfg->input_len; +} + +/* + * According to ETSI TS 138 323 V17.1.0, Section 5.2.2.1, + * SN could be divided into following ranges, + * relatively to current value of RX_DELIV state: + * +-------------+-------------+-------------+-------------+ + * | -Outside | -Window | +Window | +Outside | + * | (valid) | (Invalid) | (Valid) | (Invalid) | + * +-------------+-------------^-------------+-------------+ + * | + * v + * SN(RX_DELIV) + */ +enum sn_range_type { + SN_RANGE_MINUS_OUTSIDE, + SN_RANGE_MINUS_WINDOW, + SN_RANGE_PLUS_WINDOW, + SN_RANGE_PLUS_OUTSIDE, +}; + +#define PDCP_SET_COUNT(hfn, sn, size) ((hfn << size) | (sn & ((1 << size) - 1))) + +/* + * Take uplink test case as base, modify RX_DELIV in state and SN in input + */ +static int +test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) +{ + uint32_t rx_deliv_hfn, rx_deliv_sn, new_hfn, new_sn; + const int domain = conf->entity.pdcp_xfrm.domain; + struct pdcp_test_conf dl_conf; + int ret, expected_ret; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + if (domain != RTE_SECURITY_PDCP_MODE_CONTROL && domain != RTE_SECURITY_PDCP_MODE_DATA) + return TEST_SKIPPED; + + const uint32_t sn_size = conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + /* Max value of SN that could fit in `sn_size` bits */ + const uint32_t max_sn = (1 << sn_size) - 1; + const uint32_t shift = (max_sn - window_size) / 2; + /* Could be any number up to `shift` value */ + const uint32_t default_sn = RTE_MIN(2u, shift); + + /* Initialize HFN as non zero value, to be able check values before */ + rx_deliv_hfn = 0xa; + + switch (type) { + case SN_RANGE_PLUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + 1; + expected_ret = TEST_SUCCESS; + break; + case SN_RANGE_MINUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn - 1; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_PLUS_OUTSIDE: + /* RCVD_SN >= SN(RX_DELIV) + Window_Size */ + new_hfn = rx_deliv_hfn - 1; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + window_size; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_MINUS_OUTSIDE: + /* RCVD_SN < SN(RX_DELIV) - Window_Size */ + new_hfn = rx_deliv_hfn + 1; + rx_deliv_sn = window_size + default_sn; + new_sn = rx_deliv_sn - window_size - 1; + expected_ret = TEST_SUCCESS; + break; + default: + return TEST_FAILED; + } + + /* Configure Uplink to generate expected, encrypted packet */ + pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.reverse_iv_direction = true; + conf->entity.pdcp_xfrm.hfn = new_hfn; + conf->entity.sn = new_sn; + conf->output_len = 0; + ret = test_attempt_single(conf); + if (ret != TEST_SUCCESS) + return ret; + + /* Flip configuration to downlink */ + uplink_to_downlink_convert(conf, &dl_conf); + + /* Modify the rx_deliv to verify the expected behaviour */ + dl_conf.entity.pdcp_xfrm.hfn = rx_deliv_hfn; + dl_conf.entity.sn = rx_deliv_sn; + ret = test_attempt_single(&dl_conf); + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + return ret; + + TEST_ASSERT_EQUAL(ret, expected_ret, "Unexpected result"); + + return TEST_SUCCESS; +} + +static int +test_sn_plus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_WINDOW, t_conf); +} + +static int +test_sn_minus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_WINDOW, t_conf); +} + +static int +test_sn_plus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_OUTSIDE, t_conf); +} + +static int +test_sn_minus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); +} + +static int +test_combined(struct pdcp_test_conf *ul_conf) +{ + struct pdcp_test_conf dl_conf; + int ret; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + ul_conf->entity.reverse_iv_direction = true; + ul_conf->output_len = 0; + + ret = test_attempt_single(ul_conf); + if (ret != TEST_SUCCESS) + return ret; + + uplink_to_downlink_convert(ul_conf, &dl_conf); + ret = test_attempt_single(&dl_conf); + + return ret; +} + static int run_test_for_one_known_vec(const void *arg) { @@ -677,8 +947,34 @@ run_test_for_one_known_vec(const void *arg) return test_attempt_single(&test_conf); } +static struct unit_test_suite combined_mode_cases = { + .suite_name = "PDCP combined mode", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("combined mode", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_combined), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static struct unit_test_suite hfn_sn_test_cases = { + .suite_name = "PDCP HFN/SN", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("SN plus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_window), + TEST_CASE_NAMED_WITH_DATA("SN minus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_window), + TEST_CASE_NAMED_WITH_DATA("SN plus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_outside), + TEST_CASE_NAMED_WITH_DATA("SN minus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_outside), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ + &combined_mode_cases, + &hfn_sn_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 11/21] doc: add PDCP library guide 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (9 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 12/21] pdcp: add control PDU handling for status report Anoob Joseph ` (9 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add guide for PDCP library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 254 ++++++++++++++++++ doc/guides/rel_notes/release_23_07.rst | 12 + 5 files changed, 269 insertions(+) create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst diff --git a/MAINTAINERS b/MAINTAINERS index 15dd88af3a..4345ae21b6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1555,6 +1555,7 @@ M: Volodymyr Fialko <vfialko@marvell.com> T: git://dpdk.org/next/dpdk-next-crypto F: lib/pdcp/ F: app/test/test_pdcp* +F: doc/guides/prog_guide/pdcp_lib.rst Packet Framework diff --git a/doc/guides/prog_guide/img/pdcp_functional_overview.svg b/doc/guides/prog_guide/img/pdcp_functional_overview.svg new file mode 100644 index 0000000000..287daafc21 --- /dev/null +++ b/doc/guides/prog_guide/img/pdcp_functional_overview.svg @@ -0,0 +1 @@ +<svg width="1280" height="720" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><clipPath id="clip0"><rect x="0" y="0" width="1280" height="720"/></clipPath></defs><g clip-path="url(#clip0)"><rect x="0" y="0" width="1280" height="720" fill="#FFFFFF"/><rect x="202" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><rect x="640" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><path d="M605.5 73.5001 605.5 590.633" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M380.5 634 803.25 634 803.25 585.907 790.5 585.907 816 557.5 841.5 585.907 828.75 585.907 828.75 659.5 380.5 659.5Z" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5" fill-rule="evenodd"/><rect x="362.5" y="557.5" width="28" height="102" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5"/><rect x="412.5" y="671.5" width="370" height="32" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(492.364 694)">Radio Interface (<tspan font-size="19" x="137.5" y="0">Uu</tspan><tspan font-size="19" x="161.333" y="0">/PC5)</tspan><tspan font-size="19" x="-282.121" y="-653">UE/NG</tspan><tspan font-size="19" x="-222.955" y="-653">-</tspan><tspan font-size="19" x="-216.788" y="-653">RAN/UE A</tspan><tspan font-size="19" x="375.54" y="-653">NG</tspan>-<tspan font-size="19" x="409.706" y="-653">RAN/UE/UE B</tspan><tspan font-size="14" x="0.896851" y="-647">Transmitting </tspan><tspan font-size="14" x="1.31018" y="-631">PDCP entity</tspan><tspan font-size="14" x="167.401" y="-647">Receiving </tspan><tspan font-size="14" x="160.148" y="-631">PDCP entity</tspan></text><path d="M314.5 71.5001 431.364 71.6549" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M331.5 71.5001C331.5 65.9772 349.633 61.5001 372 61.5001 394.368 61.5001 412.5 65.9772 412.5 71.5001 412.5 77.0229 394.368 81.5001 372 81.5001 349.633 81.5001 331.5 77.0229 331.5 71.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M353.5 90.5001 363.5 90.5001 363.5 48.5001 383.5 48.5001 383.5 90.5001 393.5 90.5001 373.5 110.5Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M877.364 82.6549 760.5 82.5001" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M860.5 83.5001C860.5 89.0229 842.368 93.5001 820 93.5001 797.633 93.5001 779.5 89.0229 779.5 83.5001 779.5 77.9772 797.633 73.5001 820 73.5001 842.368 73.5001 860.5 77.9772 860.5 83.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M838.5 64.5 828.5 64.5 828.5 106.5 808.5 106.5 808.5 64.5 798.5 64.5 818.5 44.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><rect x="244.5" y="128.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(302.494 150)">Transmission buffer:<tspan font-size="19" x="-4.67999" y="23">Sequence</tspan><tspan font-size="19" x="84.32" y="23">numberin</tspan>g</text><rect x="244.5" y="199.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(297.314 222)">Header or uplink data<tspan font-size="19" x="34.08" y="23">Compressio</tspan>n</text><rect x="682.5" y="141.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(735.782 164)">Header or uplink data<tspan font-size="19" x="24.2466" y="23">Decompressio</tspan>n</text><rect x="244.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(300.314 505)">Routing / Duplication</text><rect x="244.5" y="437.5" width="285" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(309.734 460)">Add PDCP header</text><rect x="244.5" y="383.5" width="189" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(298.62 406)">Ciphering</text><rect x="244.5" y="333.5" width="189" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(261.953 356)">Integrity protection</text><path d="M472.167 267.5 472.167 409.802 470.833 409.802 470.833 267.5ZM475.5 408.468 471.5 416.468 467.5 408.468Z"/><path d="M332.167 267.5 332.167 319.552 330.833 319.552 330.833 267.5ZM335.5 318.218 331.5 326.218 327.5 318.219Z"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(253.644 291)">Packets associated <tspan font-size="19" x="14.0067" y="23">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 499.312 299)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="-2.75546e-15" y="45">PDCP SDU</tspan></text><rect x="682.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(728.535 505)">Remove PDCP Header</text><rect x="682.5" y="437.5" width="203" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(733.729 460)">Deciphering</text><rect x="682.5" y="389.5" width="203" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(702.159 412)">Integrity Verification</text><rect x="682.5" y="303.5" width="203" height="77" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(712.729 325)">Reception buffer:<tspan font-size="19" x="24.6667" y="23">Reordering</tspan><tspan font-size="19" x="-13.1667" y="45">Duplicate discardin</tspan><tspan font-size="19" x="144.167" y="45">g</tspan><tspan font-size="19" x="-10.0989" y="-84">Packets associated </tspan><tspan font-size="19" x="3.90784" y="-61">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 960.34 294)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="1" y="45">PDCP SDU</tspan></text><path d="M0.666667-8.2074e-07 0.666763 78.6116-0.66657 78.6116-0.666667 8.2074e-07ZM4.00009 77.2782 0.000104987 85.2782-3.9999 77.2782Z" transform="matrix(1 0 0 -1 779.5 296.778)"/><path d="M0.666667-2.88742e-07 0.666769 235.734-0.666565 235.734-0.666667 2.88742e-07ZM4.0001 234.401 0.000104987 242.401-3.9999 234.401Z" transform="matrix(1 0 0 -1 931.5 453.901)"/></g></svg> \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 87333ee84a..6099ff63cd 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -77,4 +77,5 @@ Programmer's Guide lto profile_app asan + pdcp_lib glossary diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst new file mode 100644 index 0000000000..8369c71600 --- /dev/null +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -0,0 +1,254 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2023 Marvell. + +PDCP Protocol Processing Library +================================ + +DPDK provides a library for PDCP protocol processing. The library utilizes +other DPDK libraries such as cryptodev, reorder, etc., to provide the +application with a transparent and high performant PDCP protocol processing +library. + +The library abstracts complete PDCP protocol processing conforming to +``ETSI TS 138 323 V17.1.0 (2022-08)``. +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + +PDCP would involve the following operations, + +1. Transfer of user plane data +2. Transfer of control plane data +3. Header compression +4. Uplink data compression +5. Ciphering and integrity protection + +.. _figure_pdcp_functional_overview: + +.. figure:: img/pdcp_functional_overview.* + + PDCP functional overview new + +PDCP library would abstract the protocol offload features of the cryptodev and +would provide a uniform interface and consistent API usage to work with +cryptodev irrespective of the protocol offload features supported. + +PDCP entity API +--------------- + +PDCP library provides following control path APIs that is used to +configure various PDCP entities, + +1. ``rte_pdcp_entity_establish()`` +2. ``rte_pdcp_entity_suspend()`` +3. ``rte_pdcp_entity_release()`` + +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or +``rte_security_session`` based on the config. The sessions would be created/ +destroyed while corresponding PDCP entity operations are performed. + +When upper layers request a PDCP entity suspend (``rte_pdcp_entity_suspend()``), +it would result in flushing out of all cached packets. Internal state variables +are updated as described in 5.1.4. + +When upper layers request a PDCP entity release (``rte_pdcp_entity_release()``), +it would result in flushing out of all cached packets and releasing of all +memory associated with the entity. It would internally free any crypto/security +sessions created. All procedures mentioned in 5.1.3 would be performed. + +PDCP PDU (Protocol Data Unit) API +--------------------------------- + +PDCP PDUs can be categorized as, + +1. Control PDU +2. Data PDU + +Control PDUs are used for signalling between entities on either end and can be +one of the following, + +1. PDCP status report +2. ROHC feedback +3. EHC feedback + +Control PDUs are not ciphered or authenticated, and so such packets are not +submitted to cryptodev for processing. + +Data PDUs are regular packets submitted by upper layers for transmission to +other end. Such packets would need to be ciphered and authenticated based on +the entity configuration. + +PDCP packet processing API for data PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PDCP processing is split into 2 parts. One before cryptodev processing +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return crypto +operations belonging to multiple entities, ``rte_pdcp_pkt_crypto_group()`` +is added to help grouping crypto operations belonging to same PDCP entity. + +Lib PDCP would allow application to use same API sequence while leveraging +protocol offload features enabled by ``rte_security`` library. Lib PDCP would +internally change the handles registered for ``pre_process`` and +``post_process`` based on features enabled in the entity. + +Lib PDCP would create the required sessions on the device provided in entity to +minimize the application requirements. Also, the crypto_op allocation and free +would also be done internally by lib PDCP to allow the library to create +crypto ops as required for the input packets. For example, when control PDUs are +received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP +is expected to handle it differently. + +Supported features +------------------ + +- 12 bit & 18 bit sequence numbers +- Uplink & downlink traffic +- HFN increment +- IV generation as required per algorithm + +Supported ciphering algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CTR +- SNOW3G-CIPHER +- ZUC-CIPHER + +Supported integrity protection algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CMAC +- SNOW3G-AUTH +- ZUC-AUTH + +Sample API usage +---------------- + +The ``rte_pdcp_entity_conf`` structure is used to pass the configuration +parameters for entity creation. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_entity_conf 8< + :end-before: >8 End of structure rte_pdcp_entity_conf. + +.. code-block:: c + + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; + struct rte_crypto_op *cop[MAX_BURST_SIZE]; + struct rte_pdcp_group grp[MAX_BURST_SIZE]; + struct rte_pdcp_entity *pdcp_entity; + int nb_max_out_mb, ret, nb_grp; + uint16_t nb_ops; + + /* Create PDCP entity */ + pdcp_entity = rte_pdcp_entity_establish(&conf); + + /** + * Allocate buffer for holding mbufs returned during PDCP suspend, + * release & post-process APIs. + */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + MAX_BURST_SIZE; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + /* Handle error */ + } + + while (1) { + /* Receive packet and form mbuf */ + + /** + * Prepare packets for crypto operation. Following operations + * would be done, + * + * Transmitting entity/UL (only data PDUs): + * - Perform compression + * - Assign sequence number + * - Add PDCP header + * - Create & prepare crypto_op + * - Prepare IV for crypto operation (auth_gen, encrypt) + * - Save original PDCP SDU (during PDCP re-establishment, + * unconfirmed PDCP SDUs need to crypto processed again and + * transmitted/re-transmitted) + * + * Receiving entity/DL: + * - Any control PDUs received would be processed and + * appropriate actions taken. If data PDU, continue. + * - Determine sequence number (based on HFN & per packet SN) + * - Prepare crypto_op + * - Prepare IV for crypto operation (decrypt, auth_verify) + */ + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, + nb_rx, &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) + != nb_success) { + /* Retry for enqueue failure packets */ + } + + ... + + nb_ops = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, + MAX_BURST_SIZE); + if (nb_ops == 0) + continue; + + /** + * Received a burst of completed crypto ops from cryptodev. It + * may belong to various entities. Group similar ones together + * for entity specific post-processing. + */ + + /** + * Groups similar entities together. Frees crypto op and based + * on crypto_op status, set mbuf->ol_flags which would be + * checked in rte_pdcp_pkt_post_process(). + */ + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); + + for (i = 0; i != nb_grp; i++) { + + /** + * Post process packets after crypto completion. + * Following operations would be done, + * + * Transmitting entity/UL: + * - Check crypto result + * + * Receiving entity/DL: + * - Check crypto operation status + * - Check for duplication (if yes, drop duplicate) + * - Perform decompression + * - Trim PDCP header + * - Hold packet (SDU) for in-order delivery (return + * completed packets as and when sequence is + * completed) + * - If not in sequence, cache the packet and start + * t-Reordering timer. When timer expires, the + * packets need to delivered to upper layers (not + * treated as error packets). + */ + nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, + grp[i].m, out_mb, + grp[i].cnt, + &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + /* Perform additional operations */ + + /** + * Transmitting entity/UL + * - If duplication is enabled, duplicate PDCP PDUs + * - When lower layers confirm reception of a PDCP PDU, + * it should be communicated to PDCP layer so that + * PDCP can drop the corresponding SDU + */ + } + } diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 75079ca7d6..bb48514637 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -65,6 +65,18 @@ New Features * Added support for SM3 hash operations. * Added support for AES-CCM in cn9k and cn10k drivers. +* **Added PDCP Library.** + + Added an experimental library ``lib_pdcp`` to provide PDCP UL and DL + processing of packets. + + The library supports all PDCP algorithms and leverages lookaside crypto + offloads to cryptodevs for crypto processing. PDCP features such as IV + generation, sequence number handling etc are supported. It is planned to add + more features such as packet caching in future releases. + + See :doc:`../prog_guide/pdcp_lib` for more information. + Removed Items ------------- -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 12/21] pdcp: add control PDU handling for status report 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (10 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 11/21] doc: add PDCP library guide Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph ` (8 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add control PDU handling and implement status report generation. Status report generation works only when RX_DELIV = RX_NEXT. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 9 ++++++ lib/pdcp/meson.build | 2 ++ lib/pdcp/pdcp_cnt.c | 29 ++++++++++++++++++ lib/pdcp/pdcp_cnt.h | 14 +++++++++ lib/pdcp/pdcp_ctrl_pdu.c | 46 +++++++++++++++++++++++++++++ lib/pdcp/pdcp_ctrl_pdu.h | 15 ++++++++++ lib/pdcp/pdcp_entity.h | 15 ++++++++-- lib/pdcp/pdcp_process.c | 13 +++++++++ lib/pdcp/rte_pdcp.c | 47 +++++++++++++++++++++++++++++- lib/pdcp/rte_pdcp.h | 33 +++++++++++++++++++++ lib/pdcp/version.map | 2 ++ 11 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index 8369c71600..dcb424bb1d 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -76,6 +76,15 @@ Data PDUs are regular packets submitted by upper layers for transmission to other end. Such packets would need to be ciphered and authenticated based on the entity configuration. +PDCP packet processing API for control PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Control PDUs are used in PDCP as a communication channel between transmitting +and receiving entities. When upper layer request for operations such +re-establishment, receiving PDCP entity need to prepare a status report and +send it to the other end. The API ``rte_pdcp_control_pdu_create`` allows +application to request the same. + PDCP packet processing API for data PDU ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 08679b743a..75d476bf6d 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -8,7 +8,9 @@ if is_windows endif sources = files( + 'pdcp_cnt.c', 'pdcp_crypto.c', + 'pdcp_ctrl_pdu.c', 'pdcp_process.c', 'rte_pdcp.c', ) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c new file mode 100644 index 0000000000..c9b952184b --- /dev/null +++ b/lib/pdcp/pdcp_cnt.c @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_cnt.h" +#include "pdcp_entity.h" + +int +pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv_dl_part *en_priv_dl; + uint32_t window_sz; + + if (en == NULL || conf == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + return 0; + + en_priv_dl = entity_dl_part_get(en); + window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + RTE_SET_USED(window_sz); + RTE_SET_USED(en_priv_dl); + + return 0; +} diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h new file mode 100644 index 0000000000..bbda478b55 --- /dev/null +++ b/lib/pdcp/pdcp_cnt.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CNT_H +#define PDCP_CNT_H + +#include <rte_common.h> + +#include "pdcp_entity.h" + +int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c new file mode 100644 index 0000000000..feb05fd863 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_byteorder.h> +#include <rte_mbuf.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_ctrl_pdu.h" +#include "pdcp_entity.h" + +static __rte_always_inline void +pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) +{ + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + pdu_hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + pdu_hdr->r = 0; + pdu_hdr->fmc = rte_cpu_to_be_32(rx_deliv); +} + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; + uint32_t rx_deliv; + int pdu_sz; + + if (!en_priv->flags.is_status_report_required) + return -EINVAL; + + pdu_sz = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + + rx_deliv = en_priv->state.rx_deliv; + + /* Zero missing PDUs - status report contains only FMC */ + if (rx_deliv >= en_priv->state.rx_next) { + pdu_hdr = (struct rte_pdcp_up_ctrl_pdu_hdr *)rte_pktmbuf_append(m, pdu_sz); + if (pdu_hdr == NULL) + return -ENOMEM; + pdcp_hdr_fill(pdu_hdr, rx_deliv); + + return 0; + } + + return -ENOTSUP; +} diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h new file mode 100644 index 0000000000..a2424fbd10 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CTRL_PDU_H +#define PDCP_CTRL_PDU_H + +#include <rte_mbuf.h> + +#include "pdcp_entity.h" + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); + +#endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 10a72faae1..28691a504b 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -109,6 +109,13 @@ union cipher_iv_partial { uint64_t u64[2]; }; +struct pdcp_cnt_bitmap { + /** Number of entries that can be stored. */ + uint32_t size; + /** Bitmap of the count values already received.*/ + struct rte_bitmap *bmp; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -136,9 +143,13 @@ struct entity_priv { uint64_t is_ul_entity : 1; /** Is NULL auth. */ uint64_t is_null_auth : 1; + /** Is status report required.*/ + uint64_t is_status_report_required : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; + /** Control PDU pool. */ + struct rte_mempool *ctrl_pdu_pool; /** PDCP header size. */ uint8_t hdr_sz; /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ @@ -148,8 +159,8 @@ struct entity_priv { }; struct entity_priv_dl_part { - /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ - uint8_t dummy; + /** PDCP would need to track the count values that are already received.*/ + struct pdcp_cnt_bitmap bitmap; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 28ac4102da..ed1413db6d 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -1092,6 +1092,19 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) en_priv->flags.is_null_auth = 1; + /** + * flags.is_status_report_required + * + * Indicate whether status report is required. + */ + if (conf->status_report_required) { + /** Status report is required only for DL entities. */ + if (conf->pdcp_xfrm.pkt_dir != RTE_SECURITY_PDCP_DOWNLINK) + return -EINVAL; + + en_priv->flags.is_status_report_required = 1; + } + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 91dab91f73..96ad397667 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -6,7 +6,9 @@ #include <rte_pdcp.h> #include <rte_malloc.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -58,7 +60,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (pdcp_dynfield_register() < 0) return NULL; - if (conf == NULL || conf->cop_pool == NULL) { + if (conf == NULL || conf->cop_pool == NULL || conf->ctrl_pdu_pool == NULL) { rte_errno = EINVAL; return NULL; } @@ -105,6 +107,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) en_priv->state.rx_deliv = count; en_priv->state.tx_next = count; en_priv->cop_pool = conf->cop_pool; + en_priv->ctrl_pdu_pool = conf->ctrl_pdu_pool; /* Setup crypto session */ ret = pdcp_crypto_sess_create(entity, conf); @@ -115,6 +118,10 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + ret = pdcp_cnt_ring_create(entity, conf); + if (ret) + goto crypto_sess_destroy; + return entity; crypto_sess_destroy: @@ -162,3 +169,41 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, return 0; } + +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type) +{ + struct entity_priv *en_priv; + struct rte_mbuf *m; + int ret; + + if (pdcp_entity == NULL) { + rte_errno = EINVAL; + return NULL; + } + + en_priv = entity_priv_get(pdcp_entity); + + m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); + if (m == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + switch (type) { + case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: + ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + break; + default: + ret = -ENOTSUP; + } + + if (ret) { + rte_pktmbuf_free(m); + rte_errno = -ret; + return NULL; + } + + return m; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index b88cad4f64..2a2bae1cd1 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -16,6 +16,7 @@ #include <rte_compat.h> #include <rte_common.h> #include <rte_mempool.h> +#include <rte_pdcp_hdr.h> #include <rte_security.h> #ifdef __cplusplus @@ -79,6 +80,8 @@ struct rte_pdcp_entity_conf { struct rte_mempool *sess_mpool; /** Crypto op pool.*/ struct rte_mempool *cop_pool; + /** Mbuf pool to be used for allocating control PDUs.*/ + struct rte_mempool *ctrl_pdu_pool; /** * SN value to be used. 32 bit count value to be used for the first * packet would be derived based on HFN (`rte_security_pdcp_xform.hfn`) @@ -98,6 +101,16 @@ struct rte_pdcp_entity_conf { * crypto processing. */ bool reverse_iv_direction; + /** + * Status report required (specified in TS 38.331). + * + * If PDCP entity is configured to send a PDCP status report, the upper + * layer application may request a receiving PDCP entity to generate a + * PDCP status report using ``rte_pdcp_control_pdu_create``. In + * addition, PDCP status reports may be generated during operations such + * as entity re-establishment. + */ + bool status_report_required; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -179,6 +192,26 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Create control PDU packet of the `type` specified. The control PDU packet + * would be allocated from *rte_pdcp_entity_conf.ctrl_pdu_pool* by lib PDCP. + * + * @param pdcp_entity + * Pointer to the PDCP entity for which the control PDU need to be generated. + * @param type + * Type of control PDU to be generated. + * @return + * - Control PDU generated, in case of success. + * - NULL in case of failure. rte_errno will be set to error code. + */ +__rte_experimental +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index d564f155e0..97171f902e 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,6 +2,8 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_control_pdu_create; + rte_pdcp_en_from_cop; rte_pdcp_entity_establish; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 13/21] pdcp: implement t-Reordering and packet buffering 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (11 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 12/21] pdcp: add control PDU handling for status report Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 14/21] test/pdcp: add in-order delivery cases Anoob Joseph ` (7 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add in-order delivery of packets in PDCP. Delivery of packets in-order relies on t-Reordering timer. When 'out-of-order delivery' is disabled, PDCP will buffer all received packets that are out of order. The t-Reordering timer determines the time period these packets would be held in the buffer, waiting for any missing packets to arrive. Introduce packet buffering and state variables which indicate status of the timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 3 +- lib/pdcp/pdcp_entity.h | 19 +++++++ lib/pdcp/pdcp_process.c | 117 ++++++++++++++++++++++++++++++---------- lib/pdcp/pdcp_reorder.c | 27 ++++++++++ lib/pdcp/pdcp_reorder.h | 62 +++++++++++++++++++++ lib/pdcp/rte_pdcp.c | 53 ++++++++++++++++-- lib/pdcp/rte_pdcp.h | 6 ++- 7 files changed, 252 insertions(+), 35 deletions(-) create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 75d476bf6d..f4f9246bcb 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -12,9 +12,10 @@ sources = files( 'pdcp_crypto.c', 'pdcp_ctrl_pdu.c', 'pdcp_process.c', + 'pdcp_reorder.c', 'rte_pdcp.c', ) headers = files('rte_pdcp.h') indirect_headers += files('rte_pdcp_group.h') -deps += ['mbuf', 'net', 'cryptodev', 'security'] +deps += ['mbuf', 'net', 'cryptodev', 'security', 'reorder'] diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 28691a504b..34341cdc11 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -11,6 +11,8 @@ #include <rte_pdcp.h> #include <rte_security.h> +#include "pdcp_reorder.h" + struct entity_priv; #define PDCP_HFN_MIN 0 @@ -109,6 +111,17 @@ union cipher_iv_partial { uint64_t u64[2]; }; +enum timer_state { + TIMER_STOP, + TIMER_RUNNING, + TIMER_EXPIRED, +}; + +struct pdcp_t_reordering { + /** Represent timer state */ + enum timer_state state; +}; + struct pdcp_cnt_bitmap { /** Number of entries that can be stored. */ uint32_t size; @@ -145,6 +158,8 @@ struct entity_priv { uint64_t is_null_auth : 1; /** Is status report required.*/ uint64_t is_status_report_required : 1; + /** Is out-of-order delivery enabled */ + uint64_t is_out_of_order_delivery : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -161,6 +176,10 @@ struct entity_priv { struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; + /** t-Reordering handles */ + struct pdcp_t_reordering t_reorder; + /** Reorder packet buffer */ + struct pdcp_reorder reorder; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index ed1413db6d..84a0f3a43f 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -837,25 +837,88 @@ pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool tr } } -static inline bool +static inline int pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, - const uint32_t count) + const uint32_t count, struct rte_mbuf *mb, + struct rte_mbuf *out_mb[], + const bool trim_mac) { struct entity_priv *en_priv = entity_priv_get(entity); + struct pdcp_t_reordering *t_reorder; + struct pdcp_reorder *reorder; + uint16_t processed = 0; - if (count < en_priv->state.rx_deliv) - return false; + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + const uint32_t hdr_trim_sz = en_priv->hdr_sz + en_priv->aad_sz; - /* t-Reordering timer is not supported - SDU will be delivered immediately. - * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not - * been delivered to upper layers - */ - en_priv->state.rx_next = count + 1; + if (count < en_priv->state.rx_deliv) + return -EINVAL; if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; - return true; + pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); + + if (en_priv->flags.is_out_of_order_delivery) { + out_mb[0] = mb; + en_priv->state.rx_deliv = count + 1; + + return 1; + } + + reorder = &dl->reorder; + t_reorder = &dl->t_reorder; + + if (count == en_priv->state.rx_deliv) { + if (reorder->is_active) { + /* + * This insert used only to increment reorder->min_seqn + * To remove it - min_seqn_set() has to work with non-empty buffer + */ + pdcp_reorder_insert(reorder, mb, count); + + /* Get buffered packets */ + struct rte_mbuf **cached_mbufs = &out_mb[processed]; + uint32_t nb_cached = pdcp_reorder_get_sequential(reorder, + cached_mbufs, entity->max_pkt_cache - processed); + + processed += nb_cached; + } else { + out_mb[processed++] = mb; + } + + /* Processed should never exceed the window size */ + en_priv->state.rx_deliv = count + processed; + + } else { + if (!reorder->is_active) + /* Initialize reordering buffer with RX_DELIV */ + pdcp_reorder_start(reorder, en_priv->state.rx_deliv); + /* Buffer the packet */ + pdcp_reorder_insert(reorder, mb, count); + } + + /* Stop & reset current timer if rx_reord is received */ + if (t_reorder->state == TIMER_RUNNING && + en_priv->state.rx_deliv >= en_priv->state.rx_reord) { + t_reorder->state = TIMER_STOP; + /* Stop reorder buffer, only if it's empty */ + if (en_priv->state.rx_deliv == en_priv->state.rx_next) + pdcp_reorder_stop(reorder); + } + + /* + * If t-Reordering is not running (includes the case when t-Reordering is stopped due to + * actions above). + */ + if (t_reorder->state == TIMER_STOP && en_priv->state.rx_deliv < en_priv->state.rx_next) { + /* Update RX_REORD to RX_NEXT */ + en_priv->state.rx_reord = en_priv->state.rx_next; + /* Start t-Reordering */ + t_reorder->state = TIMER_RUNNING; + } + + return processed; } static inline uint16_t @@ -863,16 +926,12 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret, const bool is_integ_protected) { - struct entity_priv *en_priv = entity_priv_get(entity); - const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; + int i, nb_processed, nb_success = 0, nb_err = 0; rte_pdcp_dynfield_t *mb_dynfield; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -881,11 +940,12 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r mb_dynfield = pdcp_dynfield(mb); count = *mb_dynfield; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], is_integ_protected); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -919,16 +979,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret) { - struct entity_priv *en_priv = entity_priv_get(entity); - const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; + int i, nb_processed, nb_success = 0, nb_err = 0; rte_pdcp_dynfield_t *mb_dynfield; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -937,12 +993,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, mb_dynfield = pdcp_dynfield(mb); count = *mb_dynfield; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], true); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, true); - - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -1105,6 +1161,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent en_priv->flags.is_status_report_required = 1; } + /** + * flags.is_out_of_order_delivery + * + * Indicate whether the outoforder delivery is enabled for PDCP entity. + */ + en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** * hdr_sz * diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c new file mode 100644 index 0000000000..5399f0dc28 --- /dev/null +++ b/lib/pdcp/pdcp_reorder.c @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_reorder.h> + +#include "pdcp_reorder.h" + +int +pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +{ + reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + if (reorder->buf == NULL) + return -rte_errno; + + reorder->window_size = window_size; + reorder->is_active = false; + + return 0; +} + +void +pdcp_reorder_destroy(const struct pdcp_reorder *reorder) +{ + rte_reorder_free(reorder->buf); +} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h new file mode 100644 index 0000000000..6a2f61d6ae --- /dev/null +++ b/lib/pdcp/pdcp_reorder.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_REORDER_H +#define PDCP_REORDER_H + +#include <rte_reorder.h> + +struct pdcp_reorder { + struct rte_reorder_buffer *buf; + uint32_t window_size; + bool is_active; +}; + +int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); +void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); + +static inline uint32_t +pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs) +{ + return rte_reorder_drain(reorder->buf, mbufs, max_mbufs); +} + +static inline uint32_t +pdcp_reorder_up_to_get(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs, uint32_t seqn) +{ + return rte_reorder_drain_up_to_seqn(reorder->buf, mbufs, max_mbufs, seqn); +} + +static inline void +pdcp_reorder_start(struct pdcp_reorder *reorder, uint32_t min_seqn) +{ + int ret; + + reorder->is_active = true; + + ret = rte_reorder_min_seqn_set(reorder->buf, min_seqn); + RTE_VERIFY(ret == 0); +} + +static inline void +pdcp_reorder_stop(struct pdcp_reorder *reorder) +{ + reorder->is_active = false; +} + +static inline void +pdcp_reorder_insert(struct pdcp_reorder *reorder, struct rte_mbuf *mbuf, + rte_reorder_seqn_t pkt_count) +{ + int ret; + + *rte_reorder_seqn(mbuf) = pkt_count; + + ret = rte_reorder_insert(reorder->buf, mbuf); + RTE_VERIFY(ret == 0); +} + +#endif /* PDCP_REORDER_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 96ad397667..be37ff392c 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -49,6 +49,17 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } +static int +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + + entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + + return pdcp_reorder_create(&dl->reorder, window_size); +} + struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { @@ -118,6 +129,12 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + ret = pdcp_dl_establish(entity, conf); + if (ret) + goto crypto_sess_destroy; + } + ret = pdcp_cnt_ring_create(entity, conf); if (ret) goto crypto_sess_destroy; @@ -132,26 +149,50 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } +static int +pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + int nb_out; + + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, + en_priv->state.rx_next); + + pdcp_reorder_destroy(&dl->reorder); + + return nb_out; +} + int rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv *en_priv; + int nb_out = 0; + if (pdcp_entity == NULL) return -EINVAL; + en_priv = entity_priv_get(pdcp_entity); + + if (!en_priv->flags.is_ul_entity) + nb_out = pdcp_dl_release(pdcp_entity, out_mb); + /* Teardown crypto sessions */ pdcp_crypto_sess_destroy(pdcp_entity); rte_free(pdcp_entity); - RTE_SET_USED(out_mb); - return 0; + return nb_out; } int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; + int nb_out = 0; if (pdcp_entity == NULL) return -EINVAL; @@ -161,13 +202,15 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, if (en_priv->flags.is_ul_entity) { en_priv->state.tx_next = 0; } else { + dl = entity_dl_part_get(pdcp_entity); + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, pdcp_entity->max_pkt_cache, + en_priv->state.rx_next); + pdcp_reorder_stop(&dl->reorder); en_priv->state.rx_next = 0; en_priv->state.rx_deliv = 0; } - RTE_SET_USED(out_mb); - - return 0; + return nb_out; } struct rte_mbuf * diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 2a2bae1cd1..980086a93a 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -111,6 +111,8 @@ struct rte_pdcp_entity_conf { * as entity re-establishment. */ bool status_report_required; + /** Enable out of order delivery. */ + bool out_of_order_delivery; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -271,8 +273,8 @@ rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, * @param in_mb * The address of an array of *num* pointers to *rte_mbuf* structures. * @param[out] out_mb - * The address of an array of *num* pointers to *rte_mbuf* structures - * to output packets after PDCP post-processing. + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures to output packets after PDCP post-processing. * @param num * The maximum number of packets to process. * @param[out] nb_err -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 14/21] test/pdcp: add in-order delivery cases 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (12 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 15/21] pdcp: add timer callback handlers Anoob Joseph ` (6 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test cases to verify behaviour when in-order delivery is enabled and packets arrive in out-of-order. PDCP library is expected to buffer the packets and return packets in-order when the missing packet arrives. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 223 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index cfe2ec6aa9..24d7826bc2 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,15 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* Assert that condition is true, or goto the mark */ +#define ASSERT_TRUE_OR_GOTO(cond, mark, ...) do {\ + if (!(cond)) { \ + RTE_LOG(ERR, USER1, "Error at: %s:%d\n", __func__, __LINE__); \ + RTE_LOG(ERR, USER1, __VA_ARGS__); \ + goto mark; \ + } \ +} while (0) + /* According to formula(7.2.a Window_Size) */ #define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) @@ -83,6 +92,38 @@ run_test_with_all_known_vec(const void *args) return run_test_foreach_known_vec(test, false); } +static int +run_test_with_all_known_vec_until_first_pass(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, true); +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -416,6 +457,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.sess_mpool = ts_params->sess_pool; conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.ctrl_pdu_pool = ts_params->mbuf_pool; conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; @@ -868,6 +910,7 @@ test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) /* Configure Uplink to generate expected, encrypted packet */ pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.out_of_order_delivery = true; conf->entity.reverse_iv_direction = true; conf->entity.pdcp_xfrm.hfn = new_hfn; conf->entity.sn = new_sn; @@ -915,6 +958,171 @@ test_sn_minus_outside(struct pdcp_test_conf *t_conf) return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); } +static struct rte_mbuf * +generate_packet_for_dl_with_sn(struct pdcp_test_conf ul_conf, uint32_t count) +{ + enum rte_security_pdcp_sn_size sn_size = ul_conf.entity.pdcp_xfrm.sn_size; + int ret; + + ul_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(count, sn_size); + ul_conf.entity.sn = pdcp_sn_from_count_get(count, sn_size); + ul_conf.entity.out_of_order_delivery = true; + ul_conf.entity.reverse_iv_direction = true; + ul_conf.output_len = 0; + + ret = test_attempt_single(&ul_conf); + if (ret != TEST_SUCCESS) + return NULL; + + return mbuf_from_data_create(ul_conf.output, ul_conf.output_len); +} + +static bool +array_asc_sorted_check(struct rte_mbuf *m[], uint32_t len, enum rte_security_pdcp_sn_size sn_size) +{ + uint32_t i; + + if (len < 2) + return true; + + for (i = 0; i < (len - 1); i++) { + if (pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i], void *), sn_size) > + pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i + 1], void *), sn_size)) + return false; + } + + return true; +} + +static int +test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m0 = NULL, *m1 = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Generate packet to fill the existing gap */ + m0 = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m0 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1] + * Gap filled, all packets should be returned + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m0, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m0 = NULL; /* Packet was moved to out_mb */ + + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, + "Error occurred during packet drain\n"); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m0); + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + +static int +test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + struct rte_mbuf *m1 = NULL, **out_mb = NULL; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + const int rx_deliv = 0; + int ret = TEST_FAILED; + size_t i, nb_out; + uint8_t cdev_id; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + ASSERT_TRUE_OR_GOTO(pdcp_entity->max_pkt_cache >= window_size, exit, + "PDCP max packet cache is too small"); + cdev_id = dl_conf.entity.dev_id; + out_mb = rte_zmalloc(NULL, pdcp_entity->max_pkt_cache * sizeof(uintptr_t), 0); + ASSERT_TRUE_OR_GOTO(out_mb != NULL, exit, + "Could not allocate buffer for holding out_mb buffers\n"); + + /* Send packets with SN > RX_DELIV to create a gap */ + for (i = rx_deliv + 1; i < window_size; i++) { + m1 = generate_packet_for_dl_with_sn(*ul_conf, i); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + } + + m1 = generate_packet_for_dl_with_sn(*ul_conf, rx_deliv); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Insert missing packet */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == window_size, exit, + "Packet count mismatch (received: %i, expected: %i)\n", + nb_success, window_size); + m1 = NULL; + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + rte_free(out_mb); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -971,10 +1179,25 @@ static struct unit_test_suite hfn_sn_test_cases = { } }; +static struct unit_test_suite reorder_test_cases = { + .suite_name = "PDCP reorder", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_reorder_buffer_full_window_size_sn_12), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, + &reorder_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 15/21] pdcp: add timer callback handlers 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (13 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 14/21] test/pdcp: add in-order delivery cases Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 16/21] pdcp: add timer expiry handle Anoob Joseph ` (5 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> PDCP has a windowing mechanism which allows only packets that fall in a reception window. The pivot point for this window is RX_REORD which happens to be the first missing or next expected packet. If the missing packet is not received after a specified time, then the RX_REORD state variable needs to be moved up to slide the reception window. PDCP relies on timers for such operations. The timer needs to be armed when PDCP library doesn't receive all packets in-order and starts buffering packets that arrived after a missing packet. The timer needs to be cancelled when a missing packet is received. To avoid dependency on particular timer implementation, PDCP library allows application to register two callbacks, timer_start() and timer_stop() that will be called later by library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 ++ lib/pdcp/pdcp_process.c | 2 ++ lib/pdcp/rte_pdcp.c | 1 + lib/pdcp/rte_pdcp.h | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 52 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 34341cdc11..efc74ba9b9 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -120,6 +120,8 @@ enum timer_state { struct pdcp_t_reordering { /** Represent timer state */ enum timer_state state; + /** User defined callback handles */ + struct rte_pdcp_t_reordering handle; }; struct pdcp_cnt_bitmap { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 84a0f3a43f..daf2c27363 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -902,6 +902,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (t_reorder->state == TIMER_RUNNING && en_priv->state.rx_deliv >= en_priv->state.rx_reord) { t_reorder->state = TIMER_STOP; + t_reorder->handle.stop(t_reorder->handle.timer, t_reorder->handle.args); /* Stop reorder buffer, only if it's empty */ if (en_priv->state.rx_deliv == en_priv->state.rx_next) pdcp_reorder_stop(reorder); @@ -916,6 +917,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, en_priv->state.rx_reord = en_priv->state.rx_next; /* Start t-Reordering */ t_reorder->state = TIMER_RUNNING; + t_reorder->handle.start(t_reorder->handle.timer, t_reorder->handle.args); } return processed; diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index be37ff392c..a0558b99ae 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -56,6 +56,7 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct entity_priv_dl_part *dl = entity_dl_part_get(entity); entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + dl->t_reorder.handle = conf->t_reordering; return pdcp_reorder_create(&dl->reorder, window_size); } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 980086a93a..05c922819e 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -67,6 +67,51 @@ struct rte_pdcp_entity { uint32_t max_pkt_cache; } __rte_cache_aligned; +/** + * Callback function type for t-Reordering timer start, set during PDCP entity establish. + * This callback is invoked by PDCP library, during t-Reordering timer start event. + * Only one t-Reordering per receiving PDCP entity would be running at a given time. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_start_cb_t)(void *timer, void *args); + +/** + * Callback function type for t-Reordering timer stop, set during PDCP entity establish. + * This callback will be invoked by PDCP library, during t-Reordering timer stop event. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); + +/** + * PDCP t-Reordering timer interface + * + * Configuration provided by user, that PDCP library will invoke according to timer behaviour. + */ +struct rte_pdcp_t_reordering { + /** Timer pointer, stored for later use in callback functions */ + void *timer; + /** Timer arguments, stored for later use in callback functions */ + void *args; + /** Timer start callback handle */ + rte_pdcp_t_reordering_start_cb_t start; + /** Timer stop callback handle */ + rte_pdcp_t_reordering_stop_cb_t stop; +}; + /** * PDCP entity configuration to be used for establishing an entity. */ @@ -113,6 +158,8 @@ struct rte_pdcp_entity_conf { bool status_report_required; /** Enable out of order delivery. */ bool out_of_order_delivery; + /** t-Reordering timer configuration */ + struct rte_pdcp_t_reordering t_reordering; }; /* >8 End of structure rte_pdcp_entity_conf. */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 16/21] pdcp: add timer expiry handle 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (14 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 15/21] pdcp: add timer callback handlers Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 17/21] test/pdcp: add timer expiry cases Anoob Joseph ` (4 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> The PDCP protocol requires usage of timers to keep track of how long an out-of-order packet should be buffered while waiting for missing packets. Applications can register a desired timer implementation with the PDCP library. Once the timer expires, the application will be notified, and further handling of the event will be performed in the PDCP library. When the timer expires, the PDCP library will return the cached packets, and PDCP internal state variables (like RX_REORD, RX_DELIV etc) will be updated accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 30 ++++++++++++++++++ lib/pdcp/rte_pdcp.c | 49 ++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 31 +++++++++++++++++++ lib/pdcp/version.map | 2 ++ 4 files changed, 112 insertions(+) diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index dcb424bb1d..16deaead15 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -130,6 +130,36 @@ Supported integrity protection algorithms - SNOW3G-AUTH - ZUC-AUTH +Timers +------ + +PDCP utilizes a reception window mechanism to limit the bits of COUNT value +transmitted in the packet. It utilizes state variables such as RX_REORD, +RX_DELIV to define the window and uses RX_DELIV as the lower pivot point of the +window. + +RX_DELIV would be updated only when packets are received in-order. Any missing +packet would mean RX_DELIV won't be updated. A timer, t-Reordering, helps PDCP +to slide the window if the missing packet is not received in a specified time +duration. + +While starting and stopping the timer need to be done by lib PDCP, application +could register its own timer implementation. This is to make sure application +can choose between timers such as rte_timer and rte_event based timers. Starting +and stopping of timer would happen during pre & post process API. + +When the t-Reordering timer expires, application would receive the expiry event. +To perform the PDCP handling of the expiry event, +``rte_pdcp_t_reordering_expiry_handle`` can be used. Expiry handling would +involve sliding the window by updating state variables and passing the expired +packets to the application. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_t_reordering 8< + :end-before: >8 End of structure rte_pdcp_t_reordering. + + Sample API usage ---------------- diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index a0558b99ae..819c66bd08 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -251,3 +251,52 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, return m; } + +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t capacity = entity->max_pkt_cache; + uint16_t nb_out, nb_seq; + + /* 5.2.2.2 Actions when a t-Reordering expires */ + + /* + * - deliver to upper layers in ascending order of the associated COUNT value after + * performing header decompression, if not decompressed before: + */ + + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); + capacity -= nb_out; + out_mb = &out_mb[nb_out]; + + /* + * - all stored PDCP SDU(s) with consecutively associated COUNT value(s) starting from + * RX_REORD; + */ + nb_seq = pdcp_reorder_get_sequential(&dl->reorder, out_mb, capacity); + nb_out += nb_seq; + + /* + * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered + * to upper layers, with COUNT value >= RX_REORD; + */ + en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + + /* + * - if RX_DELIV < RX_NEXT: + * - update RX_REORD to RX_NEXT; + * - start t-Reordering. + */ + if (en_priv->state.rx_deliv < en_priv->state.rx_next) { + en_priv->state.rx_reord = en_priv->state.rx_next; + dl->t_reorder.state = TIMER_RUNNING; + dl->t_reorder.handle.start(dl->t_reorder.handle.timer, dl->t_reorder.handle.args); + } else { + dl->t_reorder.state = TIMER_EXPIRED; + } + + return nb_out; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 05c922819e..b926b0df29 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -101,6 +101,7 @@ typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); * * Configuration provided by user, that PDCP library will invoke according to timer behaviour. */ +/* Structure rte_pdcp_t_reordering 8< */ struct rte_pdcp_t_reordering { /** Timer pointer, stored for later use in callback functions */ void *timer; @@ -111,6 +112,7 @@ struct rte_pdcp_t_reordering { /** Timer stop callback handle */ rte_pdcp_t_reordering_stop_cb_t stop; }; +/* >8 End of structure rte_pdcp_t_reordering. */ /** * PDCP entity configuration to be used for establishing an entity. @@ -339,6 +341,35 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.2.2.2 Actions when a t-Reordering expires + * + * When t-Reordering timer expires, PDCP is required to slide the reception + * window by updating state variables such as RX_REORD & RX_DELIV. PDCP would + * need to deliver some of the buffered packets based on the state variables and + * conditions described. + * + * The expiry handle need to be invoked by the application when t-Reordering + * timer expires. In addition to returning buffered packets, it may also restart + * timer based on the state variables. + * + * @param entity + * Pointer to the *rte_pdcp_entity* for which the timer expired. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. Used to return buffered packets that are + * expired. + * @return + * Number of packets returned in *out_mb* buffer. + */ +__rte_experimental +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, + struct rte_mbuf *out_mb[]); + /** * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. So include * in the end. diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 97171f902e..4fd912fac0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -14,5 +14,7 @@ EXPERIMENTAL { rte_pdcp_pkt_pre_process; rte_pdcp_pkt_crypto_group; + rte_pdcp_t_reordering_expiry_handle; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 17/21] test/pdcp: add timer expiry cases 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (15 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 16/21] pdcp: add timer expiry handle Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 18/21] test/pdcp: add timer restart case Anoob Joseph ` (3 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test cases for handling the expiry with rte_timer and rte_event_timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 350 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 350 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 24d7826bc2..25729b2bdd 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -3,15 +3,24 @@ */ #include <rte_errno.h> +#ifdef RTE_LIB_EVENTDEV +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#endif /* RTE_LIB_EVENTDEV */ #include <rte_malloc.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include <rte_timer.h> #include "test.h" #include "test_cryptodev.h" #include "test_cryptodev_security_pdcp_test_vectors.h" +#define NSECPERSEC 1E9 #define NB_DESC 1024 +#define TIMER_ADAPTER_ID 0 +#define TEST_EV_QUEUE_ID 0 +#define TEST_EV_PORT_ID 0 #define CDEV_INVALID_ID UINT8_MAX #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 @@ -33,10 +42,21 @@ struct pdcp_testsuite_params { struct rte_mempool *cop_pool; struct rte_mempool *sess_pool; bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; + int evdev; +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev; +#endif /* RTE_LIB_EVENTDEV */ + bool timer_is_running; + uint64_t min_resolution_ns; }; static struct pdcp_testsuite_params testsuite_params; +struct test_rte_timer_args { + int status; + struct rte_pdcp_entity *pdcp_entity; +}; + struct pdcp_test_conf { struct rte_pdcp_entity_conf entity; struct rte_crypto_sym_xform c_xfrm; @@ -124,6 +144,30 @@ pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; } +static void +pdcp_timer_start_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = true; +} + +static void +pdcp_timer_stop_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = false; +} + +static struct rte_pdcp_t_reordering t_reorder_timer = { + .timer = &testsuite_params.timer_is_running, + .start = pdcp_timer_start_cb, + .stop = pdcp_timer_stop_cb, +}; + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -462,6 +506,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + conf->entity.t_reordering = t_reorder_timer; if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; @@ -1048,6 +1093,8 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) /* Check that packets in correct order */ ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == false, exit, + "Timer should be stopped after full drain\n"); ret = TEST_SUCCESS; exit: @@ -1123,6 +1170,181 @@ test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static void +event_timer_start_cb(void *timer, void *args) +{ + struct rte_event_timer *evtims = args; + int ret = 0; + + ret = rte_event_timer_arm_burst(timer, &evtims, 1); + assert(ret == 1); +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +test_expiry_with_event_timer(const struct pdcp_test_conf *ul_conf) +{ +#ifdef RTE_LIB_EVENTDEV + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_event event; + + const int start_count = 0; + struct rte_event_timer evtim = { + .ev.op = RTE_EVENT_OP_NEW, + .ev.queue_id = TEST_EV_QUEUE_ID, + .ev.sched_type = RTE_SCHED_TYPE_ATOMIC, + .ev.priority = RTE_EVENT_DEV_PRIORITY_NORMAL, + .ev.event_type = RTE_EVENT_TYPE_TIMER, + .state = RTE_EVENT_TIMER_NOT_ARMED, + .timeout_ticks = 1, + }; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &evtim; + dl_conf.entity.t_reordering.timer = testsuite_params.timdev; + dl_conf.entity.t_reordering.start = event_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + evtim.ev.event_ptr = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + while (n != 1) { + rte_delay_us(testsuite_params.min_resolution_ns / 1000); + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, + "Dequeued unexpected timer expiry event: %i\n", n); + } + + ASSERT_TRUE_OR_GOTO(event.event_type == RTE_EVENT_TYPE_TIMER, exit, "Unexpected event type\n"); + + /* Handle expiry event */ + n = rte_pdcp_t_reordering_expiry_handle(event.event_ptr, out_mb); + ASSERT_TRUE_OR_GOTO(n == 1, exit, "Unexpected number of expired packets :%i\n", n); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +#else + RTE_SET_USED(ul_conf); + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +test_rte_timer_expiry_handle(struct rte_timer *timer_handle, void *arg) +{ + struct test_rte_timer_args *timer_data = arg; + struct rte_mbuf *out_mb[1] = {0}; + uint16_t n; + + RTE_SET_USED(timer_handle); + + n = rte_pdcp_t_reordering_expiry_handle(timer_data->pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, n); + + timer_data->status = n == 1 ? n : -1; +} + +static void +test_rte_timer_start_cb(void *timer, void *args) +{ + rte_timer_reset_sync(timer, 1, SINGLE, rte_lcore_id(), test_rte_timer_expiry_handle, args); +} + +static int +test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct test_rte_timer_args timer_args; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_timer timer = {0}; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Set up a timer */ + rte_timer_init(&timer); + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &timer_args; + dl_conf.entity.t_reordering.timer = &timer; + dl_conf.entity.t_reordering.start = test_rte_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + timer_args.status = 0; + timer_args.pdcp_entity = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Verify that expire was handled correctly */ + rte_timer_manage(); + while (timer_args.status != 1) { + rte_delay_us(1); + rte_timer_manage(); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, "Bad expire handle status %i\n", + timer_args.status); + } + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1145,6 +1367,126 @@ test_combined(struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static inline void +eventdev_conf_default_set(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info) +{ + memset(dev_conf, 0, sizeof(struct rte_event_dev_config)); + dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns; + dev_conf->nb_event_ports = 1; + dev_conf->nb_event_queues = 1; + dev_conf->nb_event_queue_flows = info->max_event_queue_flows; + dev_conf->nb_event_port_dequeue_depth = info->max_event_port_dequeue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_events_limit = info->max_num_events; +} + +static inline int +eventdev_setup(void) +{ + struct rte_event_dev_config dev_conf; + struct rte_event_dev_info info; + int ret, evdev = 0; + + if (!rte_event_dev_count()) + return TEST_SKIPPED; + + ret = rte_event_dev_info_get(evdev, &info); + TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info"); + TEST_ASSERT(info.max_num_events < 0 || info.max_num_events >= 1, + "ERROR max_num_events=%d < max_events=%d", info.max_num_events, 1); + + eventdev_conf_default_set(&dev_conf, &info); + ret = rte_event_dev_configure(evdev, &dev_conf); + TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev"); + + ret = rte_event_queue_setup(evdev, TEST_EV_QUEUE_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d", TEST_EV_QUEUE_ID); + + /* Configure event port */ + ret = rte_event_port_setup(evdev, TEST_EV_PORT_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d", TEST_EV_PORT_ID); + ret = rte_event_port_link(evdev, TEST_EV_PORT_ID, NULL, NULL, 0); + TEST_ASSERT(ret >= 0, "Failed to link all queues port=%d", TEST_EV_PORT_ID); + + ret = rte_event_dev_start(evdev); + TEST_ASSERT_SUCCESS(ret, "Failed to start device"); + + testsuite_params.evdev = evdev; + + return TEST_SUCCESS; +} + +static int +event_timer_setup(void) +{ + struct rte_event_timer_adapter_info info; + struct rte_event_timer_adapter *timdev; + uint32_t caps = 0; + + struct rte_event_timer_adapter_conf config = { + .event_dev_id = testsuite_params.evdev, + .timer_adapter_id = TIMER_ADAPTER_ID, + .timer_tick_ns = NSECPERSEC, + .max_tmo_ns = 10 * NSECPERSEC, + .nb_timers = 10, + .flags = 0, + }; + + TEST_ASSERT_SUCCESS(rte_event_timer_adapter_caps_get(testsuite_params.evdev, &caps), + "Failed to get adapter capabilities"); + + if (!(caps & RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) + return TEST_SKIPPED; + + timdev = rte_event_timer_adapter_create(&config); + + TEST_ASSERT_NOT_NULL(timdev, "Failed to create event timer ring"); + + testsuite_params.timdev = timdev; + + TEST_ASSERT_EQUAL(rte_event_timer_adapter_start(timdev), 0, + "Failed to start event timer adapter"); + + rte_event_timer_adapter_get_info(timdev, &info); + testsuite_params.min_resolution_ns = info.min_resolution_ns; + + return TEST_SUCCESS; +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +ut_setup_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + int ret; + + ret = eventdev_setup(); + if (ret) + return ret; + + return event_timer_setup(); +#else + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +ut_teardown_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev = testsuite_params.timdev; + int evdev = testsuite_params.evdev; + + rte_event_dev_stop(evdev); + rte_event_dev_close(evdev); + + rte_event_timer_adapter_stop(timdev); + rte_event_timer_adapter_free(timdev); +#endif /* RTE_LIB_EVENTDEV */ +} + static int run_test_for_one_known_vec(const void *arg) { @@ -1189,6 +1531,14 @@ static struct unit_test_suite reorder_test_cases = { ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, test_reorder_buffer_full_window_size_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_event_timer", + ut_setup_pdcp_event_timer, ut_teardown_pdcp_event_timer, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_event_timer), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_rte_timer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_rte_timer), TEST_CASES_END() /**< NULL terminate unit test array */ } }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 18/21] test/pdcp: add timer restart case 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (16 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 17/21] test/pdcp: add timer expiry cases Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 19/21] pdcp: add support for status report Anoob Joseph ` (2 subsequent siblings) 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test to cover the case when t-reordering timer should be restarted on the same packet. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 68 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 25729b2bdd..82cc25ec7a 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -1106,6 +1106,71 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) return ret; } +static int +test_reorder_gap_in_reorder_buffer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + int ret = TEST_FAILED, nb_out, i; + struct pdcp_test_conf dl_conf; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Create two gaps [NULL, m1, NULL, m3]*/ + for (i = 0; i < 2; i++) { + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + 2 * i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m = NULL; /* Packet was moved to PDCP lib */ + } + + /* Generate packet to fill the first gap */ + m = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1, NULL, m3] + * Only first gap should be filled, timer should be restarted for second gap + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m = NULL; + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), + exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == true, exit, + "Timer should be restarted after partial drain"); + + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) { @@ -1527,6 +1592,9 @@ static struct unit_test_suite reorder_test_cases = { TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_in_reorder_buffer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_in_reorder_buffer), TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 19/21] pdcp: add support for status report 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (17 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 18/21] test/pdcp: add timer restart case Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 21/21] test/pdcp: add PDCP status report cases Anoob Joseph 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Implement status report generation for PDCP entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 158 ++++++++++++++++++++++++++++++++++++--- lib/pdcp/pdcp_cnt.h | 11 ++- lib/pdcp/pdcp_ctrl_pdu.c | 34 ++++++++- lib/pdcp/pdcp_ctrl_pdu.h | 3 +- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 9 ++- lib/pdcp/pdcp_process.h | 13 ++++ lib/pdcp/rte_pdcp.c | 34 ++++++--- 8 files changed, 236 insertions(+), 28 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index c9b952184b..af027b00d3 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -2,28 +2,164 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_pdcp.h> #include "pdcp_cnt.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#define SLAB_BYTE_SIZE (RTE_BITMAP_SLAB_BIT_SIZE / 8) + +uint32_t +pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) +{ + uint32_t n_bits = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + return rte_bitmap_get_memory_footprint(n_bits); +} + int -pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) { - struct entity_priv_dl_part *en_priv_dl; - uint32_t window_sz; + uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - if (en == NULL || conf == NULL) + dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + if (dl->bitmap.bmp == NULL) return -EINVAL; - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) - return 0; + dl->bitmap.size = window_size; - en_priv_dl = entity_dl_part_get(en); - window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + return 0; +} - RTE_SET_USED(window_sz); - RTE_SET_USED(en_priv_dl); +void +pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + rte_bitmap_set(bitmap.bmp, count % bitmap.size); +} - return 0; +bool +pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + return rte_bitmap_get(bitmap.bmp, count % bitmap.size); +} + +void +pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop) +{ + uint32_t i; + + for (i = start; i < stop; i++) + rte_bitmap_clear(bitmap.bmp, i % bitmap.size); +} + +uint16_t +pdcp_cnt_get_bitmap_size(uint32_t pending_bytes) +{ + /* + * Round up bitmap size to slab size to operate only on slabs sizes, instead of individual + * bytes + */ + return RTE_ALIGN_MUL_CEIL(pending_bytes, SLAB_BYTE_SIZE); +} + +static __rte_always_inline uint64_t +leftover_get(uint64_t slab, uint32_t shift, uint64_t mask) +{ + return (slab & mask) << shift; +} + +void +pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len) +{ + uint64_t slab = 0, next_slab = 0, leftover; + uint32_t zeros, report_len, diff; + uint32_t slab_id, next_slab_id; + uint32_t pos = 0, next_pos = 0; + + const uint32_t start_count = state.rx_deliv + 1; + const uint32_t nb_slabs = bitmap.size / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t nb_data_slabs = data_len / SLAB_BYTE_SIZE; + const uint32_t start_slab_id = start_count / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t stop_slab_id = (start_slab_id + nb_data_slabs) % nb_slabs; + const uint32_t shift = start_count % RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t leftover_shift = shift ? RTE_BITMAP_SLAB_BIT_SIZE - shift : 0; + const uint8_t *data_end = RTE_PTR_ADD(data, data_len + SLAB_BYTE_SIZE); + + /* NOTE: Mask required to workaround case - when shift is not needed */ + const uint64_t leftover_mask = shift ? ~0 : 0; + + /* NOTE: implement scan init at to set custom position */ + __rte_bitmap_scan_init(bitmap.bmp); + while (true) { + assert(rte_bitmap_scan(bitmap.bmp, &pos, &slab) == 1); + slab_id = pos / RTE_BITMAP_SLAB_BIT_SIZE; + if (slab_id >= start_slab_id) + break; + } + + report_len = nb_data_slabs; + + if (slab_id > start_slab_id) { + /* Zero slabs at beginning */ + zeros = (slab_id - start_slab_id - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + leftover = leftover_get(slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + report_len -= (slab_id - start_slab_id); + } + + while (report_len) { + rte_bitmap_scan(bitmap.bmp, &next_pos, &next_slab); + next_slab_id = next_pos / RTE_BITMAP_SLAB_BIT_SIZE; + diff = (next_slab_id + nb_slabs - slab_id) % nb_slabs; + + /* If next_slab_id == slab_id - overlap */ + diff += !(next_slab_id ^ slab_id) * nb_slabs; + + /* Size check - next slab is outsize of size range */ + if (diff > report_len) { + next_slab = 0; + next_slab_id = stop_slab_id; + diff = report_len; + } + + report_len -= diff; + + /* Calculate gap between slabs, taking wrap around into account */ + zeros = (next_slab_id + nb_slabs - slab_id - 1) % nb_slabs; + if (zeros) { + /* Non continues slabs, align them individually */ + slab >>= shift; + memcpy(data, &slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + + /* Fill zeros between slabs */ + zeros = (zeros - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + + /* Align beginning of next slab */ + leftover = leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } else { + /* Continues slabs, combine them */ + uint64_t new_slab = (slab >> shift) | + leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &new_slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } + + slab = next_slab; + pos = next_pos; + slab_id = next_slab_id; + + }; + + assert(data < data_end); } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index bbda478b55..5941b7a406 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -9,6 +9,15 @@ #include "pdcp_entity.h" -int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); +uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); + +void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +void pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop); + +uint16_t pdcp_cnt_get_bitmap_size(uint32_t pending_bytes); +void pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len); #endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c index feb05fd863..e0ac2d3720 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.c +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -8,6 +8,14 @@ #include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#include "pdcp_cnt.h" + +static inline uint16_t +round_up_bits(uint32_t bits) +{ + /* round up to the next multiple of 8 */ + return RTE_ALIGN_MUL_CEIL(bits, 8) / 8; +} static __rte_always_inline void pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) @@ -19,11 +27,13 @@ pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) } int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m) { struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; - uint32_t rx_deliv; - int pdu_sz; + uint32_t rx_deliv, actual_sz; + uint16_t pdu_sz, bitmap_sz; + uint8_t *data; if (!en_priv->flags.is_status_report_required) return -EINVAL; @@ -42,5 +52,21 @@ pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) return 0; } - return -ENOTSUP; + actual_sz = RTE_MIN(round_up_bits(en_priv->state.rx_next - rx_deliv - 1), + RTE_PDCP_CTRL_PDU_SIZE_MAX - pdu_sz); + bitmap_sz = pdcp_cnt_get_bitmap_size(actual_sz); + + data = (uint8_t *)rte_pktmbuf_append(m, pdu_sz + bitmap_sz); + if (data == NULL) + return -ENOMEM; + + m->pkt_len = pdu_sz + actual_sz; + m->data_len = pdu_sz + actual_sz; + + pdcp_hdr_fill((struct rte_pdcp_up_ctrl_pdu_hdr *)data, rx_deliv); + + data = RTE_PTR_ADD(data, pdu_sz); + pdcp_cnt_report_fill(dl->bitmap, en_priv->state, data, bitmap_sz); + + return 0; } diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h index a2424fbd10..2a87928b88 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.h +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -10,6 +10,7 @@ #include "pdcp_entity.h" int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m); #endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index efc74ba9b9..a9b1428c7a 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -182,6 +182,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /** Bitmap memory region */ + uint8_t bitmap_mem[0]; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index daf2c27363..774f5646d8 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -10,6 +10,7 @@ #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -857,11 +858,15 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; + if (unlikely(pdcp_cnt_bitmap_is_set(dl->bitmap, count))) + return -EEXIST; + + pdcp_cnt_bitmap_set(dl->bitmap, count); pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); if (en_priv->flags.is_out_of_order_delivery) { out_mb[0] = mb; - en_priv->state.rx_deliv = count + 1; + pdcp_rx_deliv_set(entity, count + 1); return 1; } @@ -888,7 +893,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, } /* Processed should never exceed the window size */ - en_priv->state.rx_deliv = count + processed; + pdcp_rx_deliv_set(entity, count + processed); } else { if (!reorder->is_active) diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index a52f769b82..fa3d93b405 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -8,6 +8,9 @@ #include <rte_mbuf_dyn.h> #include <rte_pdcp.h> +#include <pdcp_entity.h> +#include <pdcp_cnt.h> + typedef uint32_t rte_pdcp_dynfield_t; extern int rte_pdcp_dynfield_offset; @@ -21,4 +24,14 @@ pdcp_dynfield(struct rte_mbuf *mbuf) int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); +static inline void +pdcp_rx_deliv_set(const struct rte_pdcp_entity *entity, uint32_t rx_deliv) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + pdcp_cnt_bitmap_range_clear(dl->bitmap, en_priv->state.rx_deliv, rx_deliv); + en_priv->state.rx_deliv = rx_deliv; +} + #endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 819c66bd08..9865c620b7 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -14,6 +14,8 @@ #define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" +static int bitmap_mem_offset; + int rte_pdcp_dynfield_offset = -1; static int @@ -39,9 +41,12 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); - else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size = RTE_CACHE_LINE_ROUNDUP(size); + bitmap_mem_offset = size; + size += pdcp_cnt_bitmap_get_memory_footprint(conf); + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; @@ -54,11 +59,24 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + void *bitmap_mem; + int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - return pdcp_reorder_create(&dl->reorder, window_size); + ret = pdcp_reorder_create(&dl->reorder, window_size); + if (ret) + return ret; + + bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); + ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); + if (ret) { + pdcp_reorder_destroy(&dl->reorder); + return ret; + } + + return 0; } struct rte_pdcp_entity * @@ -136,10 +154,6 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; } - ret = pdcp_cnt_ring_create(entity, conf); - if (ret) - goto crypto_sess_destroy; - return entity; crypto_sess_destroy: @@ -218,6 +232,7 @@ struct rte_mbuf * rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, enum rte_pdcp_ctrl_pdu_type type) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; struct rte_mbuf *m; int ret; @@ -228,6 +243,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, } en_priv = entity_priv_get(pdcp_entity); + dl = entity_dl_part_get(pdcp_entity); m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); if (m == NULL) { @@ -237,7 +253,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, switch (type) { case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: - ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + ret = pdcp_ctrl_pdu_status_gen(en_priv, dl, m); break; default: ret = -ENOTSUP; @@ -283,7 +299,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered * to upper layers, with COUNT value >= RX_REORD; */ - en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + pdcp_rx_deliv_set(entity, en_priv->state.rx_reord + nb_seq); /* * - if RX_DELIV < RX_NEXT: -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 20/21] pdcp: allocate reorder buffer alongside with entity 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (18 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 19/21] pdcp: add support for status report Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 21/21] test/pdcp: add PDCP status report cases Anoob Joseph 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Instead of allocating reorder buffer separately on heap, allocate memory for it together with rest of entity, and then only initialize buffer via `rte_reorder_init()`. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 9 +++---- lib/pdcp/pdcp_cnt.h | 3 ++- lib/pdcp/pdcp_entity.h | 2 +- lib/pdcp/pdcp_reorder.c | 11 ++------ lib/pdcp/pdcp_reorder.h | 12 ++++++--- lib/pdcp/rte_pdcp.c | 58 ++++++++++++++++++++++++++--------------- 6 files changed, 55 insertions(+), 40 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index af027b00d3..e1d0634b4d 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -20,15 +20,14 @@ pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) } int -pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size) { - uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - - dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + dl->bitmap.bmp = rte_bitmap_init(nb_elem, bitmap_mem, mem_size); if (dl->bitmap.bmp == NULL) return -EINVAL; - dl->bitmap.size = window_size; + dl->bitmap.size = nb_elem; return 0; } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index 5941b7a406..87b011f9dc 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -10,7 +10,8 @@ #include "pdcp_entity.h" uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); -int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size); void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index a9b1428c7a..9f74b5d0e5 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -132,7 +132,7 @@ struct pdcp_cnt_bitmap { }; /* - * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] [reorder/bitmap] */ struct entity_priv { diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c index 5399f0dc28..bc45f2e19b 100644 --- a/lib/pdcp/pdcp_reorder.c +++ b/lib/pdcp/pdcp_reorder.c @@ -8,20 +8,13 @@ #include "pdcp_reorder.h" int -pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size) { - reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + reorder->buf = rte_reorder_init(mem, mem_size, "reorder_buffer", nb_elem); if (reorder->buf == NULL) return -rte_errno; - reorder->window_size = window_size; reorder->is_active = false; return 0; } - -void -pdcp_reorder_destroy(const struct pdcp_reorder *reorder) -{ - rte_reorder_free(reorder->buf); -} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h index 6a2f61d6ae..7e4f079d4b 100644 --- a/lib/pdcp/pdcp_reorder.h +++ b/lib/pdcp/pdcp_reorder.h @@ -9,12 +9,18 @@ struct pdcp_reorder { struct rte_reorder_buffer *buf; - uint32_t window_size; bool is_active; }; -int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); -void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); +int pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size); + +/* NOTE: replace with `rte_reorder_memory_footprint_get` after DPDK 23.07 */ +#define SIZE_OF_REORDER_BUFFER (4 * RTE_CACHE_LINE_SIZE) +static inline size_t +pdcp_reorder_memory_footprint_get(size_t nb_elem) +{ + return SIZE_OF_REORDER_BUFFER + (2 * nb_elem * sizeof(struct rte_mbuf *)); +} static inline uint32_t pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 9865c620b7..1c6d2466b2 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -14,7 +14,15 @@ #define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" -static int bitmap_mem_offset; +struct entity_layout { + size_t bitmap_offset; + size_t bitmap_size; + + size_t reorder_buf_offset; + size_t reorder_buf_size; + + size_t total_size; +}; int rte_pdcp_dynfield_offset = -1; @@ -35,46 +43,54 @@ pdcp_dynfield_register(void) } static int -pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +pdcp_entity_layout_get(const struct rte_pdcp_entity_conf *conf, struct entity_layout *layout) { - int size; + size_t size; + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); + /* Bitmap require memory to be cache aligned */ size = RTE_CACHE_LINE_ROUNDUP(size); - bitmap_mem_offset = size; - size += pdcp_cnt_bitmap_get_memory_footprint(conf); + layout->bitmap_offset = size; + layout->bitmap_size = pdcp_cnt_bitmap_get_memory_footprint(conf); + size += layout->bitmap_size; + layout->reorder_buf_offset = size; + layout->reorder_buf_size = pdcp_reorder_memory_footprint_get(window_size); + size += layout->reorder_buf_size; } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; - return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); + layout->total_size = size; + + return 0; } static int -pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf, + const struct entity_layout *layout) { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); - void *bitmap_mem; + void *memory; int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - ret = pdcp_reorder_create(&dl->reorder, window_size); + memory = RTE_PTR_ADD(entity, layout->reorder_buf_offset); + ret = pdcp_reorder_create(&dl->reorder, window_size, memory, layout->reorder_buf_size); if (ret) return ret; - bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); - ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); - if (ret) { - pdcp_reorder_destroy(&dl->reorder); + memory = RTE_PTR_ADD(entity, layout->bitmap_offset); + ret = pdcp_cnt_bitmap_create(dl, window_size, memory, layout->bitmap_size); + if (ret) return ret; - } return 0; } @@ -82,10 +98,11 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { + struct entity_layout entity_layout = { 0 }; struct rte_pdcp_entity *entity = NULL; struct entity_priv *en_priv; - int ret, entity_size; uint32_t count; + int ret; if (pdcp_dynfield_register() < 0) return NULL; @@ -118,13 +135,14 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } - entity_size = pdcp_entity_size_get(conf); - if (entity_size < 0) { + ret = pdcp_entity_layout_get(conf, &entity_layout); + if (ret < 0) { rte_errno = EINVAL; return NULL; } - entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + entity = rte_zmalloc_socket("pdcp_entity", entity_layout.total_size, RTE_CACHE_LINE_SIZE, + SOCKET_ID_ANY); if (entity == NULL) { rte_errno = ENOMEM; return NULL; @@ -149,7 +167,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { - ret = pdcp_dl_establish(entity, conf); + ret = pdcp_dl_establish(entity, conf, &entity_layout); if (ret) goto crypto_sess_destroy; } @@ -174,8 +192,6 @@ pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, en_priv->state.rx_next); - pdcp_reorder_destroy(&dl->reorder); - return nb_out; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 21/21] test/pdcp: add PDCP status report cases 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (19 preceding siblings ...) 2023-05-27 7:16 ` [PATCH v5 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph @ 2023-05-27 7:16 ` Anoob Joseph 20 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 7:16 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Test PDCP status report generation. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 312 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 312 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 82cc25ec7a..423526380f 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -2,6 +2,7 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_errno.h> #ifdef RTE_LIB_EVENTDEV #include <rte_eventdev.h> @@ -48,6 +49,9 @@ struct pdcp_testsuite_params { #endif /* RTE_LIB_EVENTDEV */ bool timer_is_running; uint64_t min_resolution_ns; + struct rte_pdcp_up_ctrl_pdu_hdr *status_report; + uint32_t status_report_bitmask_capacity; + uint8_t *ctrl_pdu_buf; }; static struct pdcp_testsuite_params testsuite_params; @@ -168,6 +172,18 @@ static struct rte_pdcp_t_reordering t_reorder_timer = { .stop = pdcp_timer_stop_cb, }; +static inline void +bitmask_set_bit(uint8_t *mask, uint32_t bit) +{ + mask[bit / 8] |= (1 << bit % 8); +} + +static inline bool +bitmask_is_bit_set(const uint8_t *mask, uint32_t bit) +{ + return mask[bit / 8] & (1 << (bit % 8)); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -314,6 +330,21 @@ testsuite_setup(void) goto cop_pool_free; } + /* Allocate memory for longest possible status report */ + ts_params->status_report_bitmask_capacity = RTE_PDCP_CTRL_PDU_SIZE_MAX - + sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + ts_params->status_report = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->status_report == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report\n"); + goto cop_pool_free; + } + + ts_params->ctrl_pdu_buf = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->ctrl_pdu_buf == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report data\n"); + goto cop_pool_free; + } + return 0; cop_pool_free: @@ -322,6 +353,8 @@ testsuite_setup(void) mbuf_pool_free: rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); return TEST_FAILED; } @@ -344,6 +377,9 @@ testsuite_teardown(void) rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); } static int @@ -1410,6 +1446,246 @@ test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) return ret; } +static struct rte_pdcp_up_ctrl_pdu_hdr * +pdcp_status_report_init(uint32_t fmc) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + + hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + hdr->fmc = rte_cpu_to_be_32(fmc); + hdr->r = 0; + memset(hdr->bitmap, 0, testsuite_params.status_report_bitmask_capacity); + + return hdr; +} + +static uint32_t +pdcp_status_report_len(void) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + uint32_t i; + + for (i = testsuite_params.status_report_bitmask_capacity; i != 0; i--) { + if (hdr->bitmap[i - 1]) + return i; + } + + return 0; +} + +static int +pdcp_status_report_verify(struct rte_mbuf *status_report, + const struct rte_pdcp_up_ctrl_pdu_hdr *expected_hdr, uint32_t expected_len) +{ + uint32_t received_len = rte_pktmbuf_pkt_len(status_report); + uint8_t *received_buf = testsuite_params.ctrl_pdu_buf; + int ret; + + ret = pktmbuf_read_into(status_report, received_buf, RTE_PDCP_CTRL_PDU_SIZE_MAX); + TEST_ASSERT_SUCCESS(ret, "Failed to copy status report pkt into continuous buffer"); + + debug_hexdump(stdout, "Received:", received_buf, received_len); + debug_hexdump(stdout, "Expected:", expected_hdr, expected_len); + + TEST_ASSERT_EQUAL(expected_len, received_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_len, received_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(received_buf, expected_hdr, expected_len, + "Generated packet not as expected"); + + return 0; +} + +static int +test_status_report_gen(const struct pdcp_test_conf *ul_conf, + const struct rte_pdcp_up_ctrl_pdu_hdr *hdr, + uint32_t bitmap_len) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *status_report = NULL, **out_mb, *m; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint32_t nb_pkts = 0, i; + uint8_t cdev_id; + + const uint32_t start_count = rte_be_to_cpu_32(hdr->fmc); + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.status_report_required = true; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + out_mb = calloc(pdcp_entity->max_pkt_cache, sizeof(uintptr_t)); + + for (i = 0; i < bitmap_len * 8; i++) { + if (!bitmask_is_bit_set(hdr->bitmap, i)) + continue; + + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + + } + + m = NULL; + + /* Check status report */ + status_report = rte_pdcp_control_pdu_create(pdcp_entity, + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT); + ASSERT_TRUE_OR_GOTO(status_report != NULL, exit, "Could not generate status report\n"); + + const uint32_t expected_len = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr) + bitmap_len; + + ASSERT_TRUE_OR_GOTO(pdcp_status_report_verify(status_report, hdr, expected_len) == 0, exit, + "Report verification failure\n"); + + ret = TEST_SUCCESS; +exit: + rte_free(m); + rte_pktmbuf_free(status_report); + rte_pktmbuf_free_bulk(out_mb, nb_pkts); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + free(out_mb); + return ret; +} + +static void +ctrl_pdu_hdr_packet_set(struct rte_pdcp_up_ctrl_pdu_hdr *hdr, uint32_t pkt_count) +{ + bitmask_set_bit(hdr->bitmap, pkt_count - rte_be_to_cpu_32(hdr->fmc) - 1); +} + +static int +test_status_report_fmc_only(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(42); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_first_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_second_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_full_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE + 1; + int i; + + for (i = 0; i < RTE_BITMAP_SLAB_BIT_SIZE; i++) + ctrl_pdu_hdr_packet_set(hdr, start_offset + i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_non_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + ctrl_pdu_hdr_packet_set(hdr, 3 * RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_max_length_sn_12(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr; + const uint32_t fmc = 0; + uint32_t i; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + ul_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + hdr = pdcp_status_report_init(fmc); + + const uint32_t max_count = RTE_MIN((RTE_PDCP_CTRL_PDU_SIZE_MAX - sizeof(hdr)) * 8, + (uint32_t)PDCP_WINDOW_SIZE(RTE_SECURITY_PDCP_SN_SIZE_12)); + + i = fmc + 2; /* set first count to have a gap, to enable packet buffering */ + + for (; i < max_count; i++) + ctrl_pdu_hdr_packet_set(hdr, i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_different_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(63); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 64 + 1); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_same_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(2); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 4); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1611,11 +1887,47 @@ static struct unit_test_suite reorder_test_cases = { } }; +static struct unit_test_suite status_report_test_cases = { + .suite_name = "PDCP status report", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_status_report_fmc_only", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_fmc_only), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_first_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_first_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_second_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_second_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_full_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_full_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_non_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_non_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_max_length_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_status_report_max_length_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_different_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_different_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_same_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_same_slab), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, &reorder_test_cases, + &status_report_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 00/21] lib: add pdcp protocol 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph ` (22 preceding siblings ...) 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph ` (21 more replies) 23 siblings, 22 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following operations, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Similar to lib IPsec, lib PDCP would allow application to use same API sequence while leveraging protocol offload features enabled by rte_security library. Lib PDCP would internally change the handles registered for *pre_process* and *post_process* based on features enabled in the entity. Lib PDCP would create the required sessions on the device provided in entity to minimize the application requirements. Also, the crypto_op allocation and free would also be done internally by lib PDCP to allow the library to create crypto ops as required for the input packets. For example, when control PDUs are received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP is expected to handle it differently. Lib PDCP utilizes reorder library for implementing in-order delivery. It utilizes bitmap library for implementing status reports and track the COUNT value of the packets received. To allow application to choose timer implementation of choice, lib PDCP allows application to configure handles that can be used for starting & stopping timers. Upon expiry, application can call corresponding PDCP API(``rte_pdcp_t_reordering_expiry_handle``) for handling the event. Unit tests are added to verify both rte_timer based timers as well as rte_eventdev based timers. PDCP tracks the sequence number of the received packets and during events such as re-establishment, it is required to generate reports and transmit to the peer. This series introduces ``rte_pdcp_control_pdu_create`` for handling control PDU generation. Changes in v5: - Deferred patch adding thread safe processing. - Updated release notes & MAINTAINERS file. Changes in v4: - Disabled 'annotate locks' with lib PDCP - Enable PDCP autotest only when lib is enabled - Use rwlock instead of spinlock - Avoid per packet checks for thread safety (Stephen) - In DL path, save count determined during pre-process in mbuf and use the same in post-process. Determining count again may lead To errors - Simplified DL path threads to allow more common code between SN 12 & 18 Changes in v3: - Addressed review comments (Akhil) - Addressed build failure in CI (tests with lib eventdev disabled) - Addressed checkpatch issues - Set only positive values to rte_errno (Akhil) Changes in v2: - Added control PDU handling - Added t-Reordering timer - Added in-order delivery - Added status PDU generation - Rebased on top of new features added in reorder library - Split base patch - Increased test coverage - Improved thread safety Changes from RFC - Implementation for all APIs covering basic control plane & user plane packets - Unit test leveraging existing PDCP vectors available in test_cryptodev - Unit test performing both UL & DL operations to verify various protocol features - Updated documentation Sample application sequence: struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; struct rte_crypto_op *cop[MAX_BURST_SIZE]; struct rte_pdcp_group grp[MAX_BURST_SIZE]; struct rte_pdcp_entity *pdcp_entity; int nb_max_out_mb, ret, nb_grp; /* Create PDCP entity */ pdcp_entity = rte_pdcp_entity_establish(&conf); /** * Allocate buffer for holding mbufs returned during PDCP suspend, * release & post-process APIs. */ /* Max packets that can be cached in entity + burst size */ nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); if (out_mb == NULL) { /* Handle error */ } while (1) { /* Receive packet and form mbuf */ /** * Prepare packets for crypto operation. Following operations * would be done, * * Transmitting entity/UL (only data PDUs): * - Perform compression * - Assign sequence number * - Add PDCP header * - Create & prepare crypto_op * - Prepare IV for crypto operation (auth_gen, encrypt) * - Save original PDCP SDU (during PDCP re-establishment, * unconfirmed PDCP SDUs need to crypto processed again and * transmitted/re-transmitted) * * Receiving entity/DL: * - Any control PDUs received would be processed and * appropriate actions taken. If data PDU, continue. * - Determine sequence number (based on HFN & per packet SN) * - Prepare crypto_op * - Prepare IV for crypto operation (decrypt, auth_verify) */ nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, nb_rx, &nb_err); if (nb_err != 0) { /* Handle error packets */ } if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) != nb_success) { /* Retry for enqueue failure packets */ } ... ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, MAX_BURST_SIZE); if (ret == 0) continue; /** * Received a burst of completed crypto ops from cryptodev. It * may belong to various entities. Group similar ones together * for entity specific post-processing. */ /** * Groups similar entities together. Frees crypto op and based * on crypto_op status, set mbuf->ol_flags which would be * checked in rte_pdcp_pkt_post_process(). */ nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); for (i = 0; i != nb_grp; i++) { /** * Post process packets after crypto completion. * Following operations would be done, * * Transmitting entity/UL: * - Check crypto result * * Receiving entity/DL: * - Check crypto operation status * - Check for duplication (if yes, drop duplicate) * - Perform decompression * - Trim PDCP header * - Hold packet (SDU) for in-order delivery (return * completed packets as and when sequence is * completed) * - If not in sequence, cache the packet and start * t-Reordering timer. When timer expires, the * packets need to delivered to upper layers (not * treated as error packets). */ nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, grp[i].m, out_mb, grp[i].cnt, &nb_err); if (nb_err != 0) { /* Handle error packets */ } /* Perform additional operations */ /** * Tranmitting entity/UL * - If duplication is enabled, duplicate PDCP PDUs * - When lower layers confirm reception of a PDCP PDU, * it should be communicated to PDCP layer so that * PDCP can drop the corresponding SDU */ } } Sample output from unit test executed on crypto_cn10k PMD(summary): ~# DPDK_TEST=pdcp_autotest ./dpdk-test --log-level=7 <snip> + ------------------------------------------------------- + + Test Suite Summary : PDCP Unit Test Suite + ------------------------------------------------------- + + Known vector cases : 112/160 passed, 0/160 skipped, 0/160 failed, 48/160 unsupported + PDCP combined mode : 1/1 passed, 0/1 skipped, 0/1 failed, 0/1 unsupported + PDCP HFN/SN : 4/4 passed, 0/4 skipped, 0/4 failed, 0/4 unsupported + PDCP reorder : 5/5 passed, 0/5 skipped, 0/5 failed, 0/5 unsupported + PDCP status report : 9/9 passed, 0/9 skipped, 0/9 failed, 0/9 unsupported + ------------------------------------------------------- + + Sub Testsuites Total : 5 + Sub Testsuites Skipped : 0 + Sub Testsuites Passed : 5 + Sub Testsuites Failed : 0 + ------------------------------------------------------- + + Tests Total : 179 + Tests Skipped : 0 + Tests Executed : 179 + Tests Unsupported: 48 + Tests Passed : 131 + Tests Failed : 0 + ------------------------------------------------------- + Test OK Anoob Joseph (10): lib: add pdcp protocol pdcp: add pre and post-process pdcp: add packet group pdcp: add crypto session create and destroy pdcp: add pre and post process for UL pdcp: add pre and post process for DL pdcp: add IV generation routines app/test: add lib pdcp tests doc: add PDCP library guide pdcp: add control PDU handling for status report Volodymyr Fialko (11): net: add PDCP header test/pdcp: pdcp HFN tests in combined mode pdcp: implement t-Reordering and packet buffering test/pdcp: add in-order delivery cases pdcp: add timer callback handlers pdcp: add timer expiry handle test/pdcp: add timer expiry cases test/pdcp: add timer restart case pdcp: add support for status report pdcp: allocate reorder buffer alongside with entity test/pdcp: add PDCP status report cases MAINTAINERS | 8 + app/test/meson.build | 5 + app/test/test_pdcp.c | 1981 +++++++++++++++++ doc/api/doxy-api-index.md | 4 +- doc/api/doxy-api.conf.in | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 293 +++ doc/guides/rel_notes/release_23_07.rst | 12 + lib/meson.build | 1 + lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++ lib/pdcp/meson.build | 21 + lib/pdcp/pdcp_cnt.c | 164 ++ lib/pdcp/pdcp_cnt.h | 24 + lib/pdcp/pdcp_crypto.c | 238 ++ lib/pdcp/pdcp_crypto.h | 20 + lib/pdcp/pdcp_ctrl_pdu.c | 72 + lib/pdcp/pdcp_ctrl_pdu.h | 16 + lib/pdcp/pdcp_entity.h | 260 +++ lib/pdcp/pdcp_process.c | 1223 ++++++++++ lib/pdcp/pdcp_process.h | 37 + lib/pdcp/pdcp_reorder.c | 20 + lib/pdcp/pdcp_reorder.h | 68 + lib/pdcp/rte_pdcp.c | 334 +++ lib/pdcp/rte_pdcp.h | 383 ++++ lib/pdcp/rte_pdcp_group.h | 131 ++ lib/pdcp/version.map | 20 + 28 files changed, 5485 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pdcp.c create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst create mode 100644 lib/net/rte_pdcp_hdr.h create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 01/21] net: add PDCP header 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 02/21] lib: add pdcp protocol Anoob Joseph ` (20 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add PDCP protocol header to be used for supporting PDCP protocol processing. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- doc/api/doxy-api-index.md | 3 +- lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 150 insertions(+), 1 deletion(-) create mode 100644 lib/net/rte_pdcp_hdr.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index c709fd48ad..debbe4134f 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -127,7 +127,8 @@ The public API headers are grouped by topics: [Geneve](@ref rte_geneve.h), [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), - [PPP](@ref rte_ppp.h) + [PPP](@ref rte_ppp.h), + [PDCP hdr](@ref rte_pdcp_hdr.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/lib/net/meson.build b/lib/net/meson.build index 379d161ee0..bd56f91c22 100644 --- a/lib/net/meson.build +++ b/lib/net/meson.build @@ -22,6 +22,7 @@ headers = files( 'rte_geneve.h', 'rte_l2tpv2.h', 'rte_ppp.h', + 'rte_pdcp_hdr.h', ) sources = files( diff --git a/lib/net/rte_pdcp_hdr.h b/lib/net/rte_pdcp_hdr.h new file mode 100644 index 0000000000..72ae9a66cb --- /dev/null +++ b/lib/net/rte_pdcp_hdr.h @@ -0,0 +1,147 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_HDR_H +#define RTE_PDCP_HDR_H + +/** + * @file + * + * PDCP-related defines + * + * Based on - ETSI TS 138 323 V17.1.0 (2022-08) + * https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + */ + +#include <rte_byteorder.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * 4.3.1 + * + * Indicate the maximum supported size of a PDCP Control PDU. + */ +#define RTE_PDCP_CTRL_PDU_SIZE_MAX 9000u + +/** + * 6.3.4 MAC-I + * + * Indicate the size of MAC-I in PDCP PDU. + */ +#define RTE_PDCP_MAC_I_LEN 4 + +/** + * Indicate type of control information included in the corresponding PDCP + * Control PDU. + */ +enum rte_pdcp_ctrl_pdu_type { + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT = 0, + RTE_PDCP_CTRL_PDU_TYPE_ROHC_FEEDBACK = 1, + RTE_PDCP_CTRL_PDU_TYPE_EHC_FEEDBACK = 2, + RTE_PDCP_CRTL_PDU_TYPE_UDC_FEEDBACK = 3, +}; + +/** + * 6.3.7 D/C + * + * This field indicates whether the corresponding PDCP PDU is a + * PDCP Data PDU or a PDCP Control PDU. + */ +enum rte_pdcp_pdu_type { + RTE_PDCP_PDU_TYPE_CTRL = 0, + RTE_PDCP_PDU_TYPE_DATA = 1, +}; + +/** + * 6.2.2.1 Data PDU for SRBs + */ +__extension__ +struct rte_pdcp_cp_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 4; /**< Reserved */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.2.2 Data PDU for DRBs and MRBs with 12 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 3; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 3; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.2.3 Data PDU for DRBs and MRBs with 18 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_18_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ + uint8_t r : 5; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 5; /**< Reserved */ + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ +#endif + uint8_t sn_15_8; /**< Sequence number bits 8-15 */ + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.3.1 Control PDU for PDCP status report + */ +__extension__ +struct rte_pdcp_up_ctrl_pdu_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t r : 4; /**< Reserved */ +#endif + /** + * 6.3.9 FMC + * + * First Missing COUNT. This field indicates the COUNT value of the + * first missing PDCP SDU within the reordering window, i.e. RX_DELIV. + */ + rte_be32_t fmc; + /** + * 6.3.10 Bitmap + * + * Length: Variable. The length of the bitmap field can be 0. + * + * This field indicates which SDUs are missing and which SDUs are + * correctly received in the receiving PDCP entity. The bit position of + * Nth bit in the Bitmap is N, i.e., the bit position of the first bit + * in the Bitmap is 1. + */ + uint8_t bitmap[]; +} __rte_packed; + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_HDR_H */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 02/21] lib: add pdcp protocol 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 03/21] pdcp: add pre and post-process Anoob Joseph ` (19 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 6 ++ doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + lib/meson.build | 1 + lib/pdcp/meson.build | 17 ++++ lib/pdcp/pdcp_crypto.c | 21 +++++ lib/pdcp/pdcp_crypto.h | 15 ++++ lib/pdcp/pdcp_entity.h | 113 ++++++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 138 +++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 13 +++ lib/pdcp/rte_pdcp.c | 141 ++++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 167 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 10 +++ 13 files changed, 645 insertions(+), 1 deletion(-) create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/version.map diff --git a/MAINTAINERS b/MAINTAINERS index 8df23e5099..85a3b94644 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1549,6 +1549,12 @@ F: doc/guides/tools/pdump.rst F: app/dumpcap/ F: doc/guides/tools/dumpcap.rst +PDCP - EXPERIMENTAL +M: Anoob Joseph <anoobj@marvell.com> +M: Volodymyr Fialko <vfialko@marvell.com> +T: git://dpdk.org/next/dpdk-next-crypto +F: lib/pdcp/ + Packet Framework ---------------- diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index debbe4134f..cd7a6cae44 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -128,7 +128,8 @@ The public API headers are grouped by topics: [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), [PPP](@ref rte_ppp.h), - [PDCP hdr](@ref rte_pdcp_hdr.h) + [PDCP hdr](@ref rte_pdcp_hdr.h), + [PDCP](@ref rte_pdcp.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index d230a19e1f..58789308a9 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -62,6 +62,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/net \ @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ + @TOPDIR@/lib/pdcp \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ @TOPDIR@/lib/port \ diff --git a/lib/meson.build b/lib/meson.build index dc8aa4ac84..a6a54c196c 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -64,6 +64,7 @@ libraries = [ 'flow_classify', # flow_classify lib depends on pkt framework table lib 'graph', 'node', + 'pdcp', # pdcp lib depends on crypto and security ] optional_libs = [ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode 100644 index 0000000000..ccaf426240 --- /dev/null +++ b/lib/pdcp/meson.build @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +sources = files( + 'pdcp_crypto.c', + 'pdcp_process.c', + 'rte_pdcp.c', + ) +headers = files('rte_pdcp.h') + +deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c new file mode 100644 index 0000000000..755e27ec9e --- /dev/null +++ b/lib/pdcp/pdcp_crypto.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_crypto.h" + +int +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + RTE_SET_USED(entity); + RTE_SET_USED(conf); + return 0; +} + +void +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) +{ + RTE_SET_USED(entity); +} diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h new file mode 100644 index 0000000000..6563331d37 --- /dev/null +++ b/lib/pdcp/pdcp_crypto.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CRYPTO_H +#define PDCP_CRYPTO_H + +#include <rte_pdcp.h> + +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, + const struct rte_pdcp_entity_conf *conf); + +void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); + +#endif /* PDCP_CRYPTO_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h new file mode 100644 index 0000000000..000297588f --- /dev/null +++ b/lib/pdcp/pdcp_entity.h @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_ENTITY_H +#define PDCP_ENTITY_H + +#include <rte_common.h> +#include <rte_crypto_sym.h> +#include <rte_mempool.h> +#include <rte_pdcp.h> +#include <rte_security.h> + +struct entity_priv; + +/* IV generation function based on the entity configuration */ +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count); + +struct entity_state { + uint32_t rx_next; + uint32_t tx_next; + uint32_t rx_deliv; + uint32_t rx_reord; +}; + +/* + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + */ + +struct entity_priv { + /** Crypto sym session. */ + struct rte_cryptodev_sym_session *crypto_sess; + /** Entity specific IV generation function. */ + iv_gen_t iv_gen; + /** Entity state variables. */ + struct entity_state state; + /** Flags. */ + struct { + /** PDCP PDU has 4 byte MAC-I. */ + uint64_t is_authenticated : 1; + /** Cipher offset & length in bits. */ + uint64_t is_cipher_in_bits : 1; + /** Auth offset & length in bits. */ + uint64_t is_auth_in_bits : 1; + /** Is UL/transmitting PDCP entity. */ + uint64_t is_ul_entity : 1; + /** Is NULL auth. */ + uint64_t is_null_auth : 1; + } flags; + /** Crypto op pool. */ + struct rte_mempool *cop_pool; + /** PDCP header size. */ + uint8_t hdr_sz; + /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ + uint8_t aad_sz; + /** Device ID of the device to be used for offload. */ + uint8_t dev_id; +}; + +struct entity_priv_dl_part { + /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ + uint8_t dummy; +}; + +struct entity_priv_ul_part { + /* + * NOTE: when re-establish is supported, plain PDCP packets & COUNT values need to be + * cached. + */ + uint8_t dummy; +}; + +static inline struct entity_priv * +entity_priv_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); +} + +static inline struct entity_priv_dl_part * +entity_dl_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline struct entity_priv_ul_part * +entity_ul_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) +{ + return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); +} + +#endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c new file mode 100644 index 0000000000..79f5dce5db --- /dev/null +++ b/lib/pdcp/pdcp_process.c @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_memcpy.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, + struct rte_crypto_sym_xform **a_xfrm) +{ + *c_xfrm = NULL; + *a_xfrm = NULL; + + if (conf->crypto_xfrm == NULL) + return -EINVAL; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + *c_xfrm = conf->crypto_xfrm; + *a_xfrm = conf->crypto_xfrm->next; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + *a_xfrm = conf->crypto_xfrm; + *c_xfrm = conf->crypto_xfrm->next; + } else { + return -EINVAL; + } + + return 0; +} + +static int +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + int ret; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + /** + * flags.is_authenticated + * + * MAC-I would be added in case of control plane packets and when authentication + * transform is not NULL. + */ + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && (a_xfrm == NULL)) + return -EINVAL; + + if (a_xfrm != NULL) + en_priv->flags.is_authenticated = 1; + + /** + * flags.is_cipher_in_bits + * + * For ZUC & SNOW3G cipher algos, offset & length need to be provided in bits. + */ + + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) + en_priv->flags.is_cipher_in_bits = 1; + + /** + * flags.is_auth_in_bits + * + * For ZUC & SNOW3G authentication algos, offset & length need to be provided in bits. + */ + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) + en_priv->flags.is_auth_in_bits = 1; + } + + /** + * flags.is_ul_entity + * + * Indicate whether the entity is UL/transmitting PDCP entity. + */ + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + en_priv->flags.is_ul_entity = 1; + + /** + * flags.is_null_auth + * + * For NULL auth, 4B zeros need to be added by lib PDCP. Indicate that + * algo is NULL auth to perform the same. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + en_priv->flags.is_null_auth = 1; + + /** + * hdr_sz + * + * PDCP header size of the entity + */ + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); + + /** + * aad_sz + * + * For AES-CMAC, additional message is prepended for processing. Need to be trimmed after + * crypto processing is done. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) + en_priv->aad_sz = 8; + else + en_priv->aad_sz = 0; + + return 0; +} + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv *en_priv; + int ret; + + if (entity == NULL || conf == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + ret = pdcp_entity_priv_populate(en_priv, conf); + if (ret) + return ret; + + return 0; +} diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h new file mode 100644 index 0000000000..fd53fff0aa --- /dev/null +++ b/lib/pdcp/pdcp_process.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_PROCESS_H +#define PDCP_PROCESS_H + +#include <rte_pdcp.h> + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode 100644 index 0000000000..adcad5dd25 --- /dev/null +++ b/lib/pdcp/rte_pdcp.c @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_pdcp.h> +#include <rte_malloc.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +{ + int size; + + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + size += sizeof(struct entity_priv_dl_part); + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size += sizeof(struct entity_priv_ul_part); + else + return -EINVAL; + + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); +} + +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) +{ + struct rte_pdcp_entity *entity = NULL; + struct entity_priv *en_priv; + int ret, entity_size; + uint32_t count; + + if (conf == NULL || conf->cop_pool == NULL) { + rte_errno = EINVAL; + return NULL; + } + + if (conf->pdcp_xfrm.en_ordering || conf->pdcp_xfrm.remove_duplicates || conf->is_slrb || + conf->en_sec_offload) { + rte_errno = ENOTSUP; + return NULL; + } + + /* + * 6.3.2 PDCP SN + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the PDCP SN is + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl-PDCP-SN-Size in + * TS 38.331 [3]) + */ + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { + rte_errno = ENOTSUP; + return NULL; + } + + if (conf->pdcp_xfrm.hfn_threshold) { + rte_errno = EINVAL; + return NULL; + } + + entity_size = pdcp_entity_size_get(conf); + if (entity_size < 0) { + rte_errno = EINVAL; + return NULL; + } + + entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (entity == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + en_priv = entity_priv_get(entity); + + count = pdcp_count_from_hfn_sn_get(conf->pdcp_xfrm.hfn, conf->sn, conf->pdcp_xfrm.sn_size); + + en_priv->state.rx_deliv = count; + en_priv->state.tx_next = count; + en_priv->cop_pool = conf->cop_pool; + + /* Setup crypto session */ + ret = pdcp_crypto_sess_create(entity, conf); + if (ret) + goto entity_free; + + ret = pdcp_process_func_set(entity, conf); + if (ret) + goto crypto_sess_destroy; + + return entity; + +crypto_sess_destroy: + pdcp_crypto_sess_destroy(entity); +entity_free: + rte_free(entity); + rte_errno = -ret; + return NULL; +} + +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) +{ + if (pdcp_entity == NULL) + return -EINVAL; + + /* Teardown crypto sessions */ + pdcp_crypto_sess_destroy(pdcp_entity); + + rte_free(pdcp_entity); + + RTE_SET_USED(out_mb); + return 0; +} + +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]) +{ + struct entity_priv *en_priv; + + if (pdcp_entity == NULL) + return -EINVAL; + + en_priv = entity_priv_get(pdcp_entity); + + if (en_priv->flags.is_ul_entity) { + en_priv->state.tx_next = 0; + } else { + en_priv->state.rx_next = 0; + en_priv->state.rx_deliv = 0; + } + + RTE_SET_USED(out_mb); + + return 0; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode 100644 index 0000000000..1f96fdc9a1 --- /dev/null +++ b/lib/pdcp/rte_pdcp.h @@ -0,0 +1,167 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_H +#define RTE_PDCP_H + +/** + * @file rte_pdcp.h + * + * RTE PDCP support. + * + * A framework for PDCP protocol processing. + */ + +#include <rte_compat.h> +#include <rte_common.h> +#include <rte_mempool.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * PDCP entity. + * + * 4.2.2 PDCP entities + * + * The PDCP entities are located in the PDCP sublayer. Several PDCP entities may + * be defined for a UE. Each PDCP entity is carrying the data of one radio + * bearer. A PDCP entity is associated either to the control plane or the user + * plane depending on which radio bearer it is carrying data for. + */ +struct rte_pdcp_entity { + /** + * PDCP entities may hold packets for purposes of in-order delivery (in + * case of receiving PDCP entity) and re-transmission (in case of + * transmitting PDCP entity). + * + * The field 'max_pkt_cache' would be used to indicate the maximum + * number of packets that may be cached in an entity at any point of + * time. When application provides buffers to receive packets from + * PDCP entity, the size of the buffer should be such that it can + * hold additionally 'max_pkt_cache' number of packets. + */ + uint32_t max_pkt_cache; +} __rte_cache_aligned; + +/** + * PDCP entity configuration to be used for establishing an entity. + */ +/* Structure rte_pdcp_entity_conf 8< */ +struct rte_pdcp_entity_conf { + /** PDCP transform for the entity. */ + struct rte_security_pdcp_xform pdcp_xfrm; + /** Crypto transform applicable for the entity. */ + struct rte_crypto_sym_xform *crypto_xfrm; + /** Mempool for crypto symmetric session. */ + struct rte_mempool *sess_mpool; + /** Crypto op pool.*/ + struct rte_mempool *cop_pool; + /** + * SN value to be used. 32 bit count value to be used for the first + * packet would be derived based on HFN (`rte_security_pdcp_xform.hfn`) + * and SN. + */ + uint32_t sn; + /** + * Indicate whether the PDCP entity belongs to Side Link Radio Bearer. + */ + bool is_slrb; + /** Enable security offload on the device specified. */ + bool en_sec_offload; + /** Device on which security/crypto session need to be created. */ + uint8_t dev_id; + /** + * Reverse direction during IV generation. Can be used to simulate UE + * crypto processing. + */ + bool reverse_iv_direction; +}; +/* >8 End of structure rte_pdcp_entity_conf. */ + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.1 PDCP entity establishment + * + * Establish PDCP entity based on provided input configuration. + * + * @param conf + * Parameters to be used for initializing PDCP entity object. + * @return + * - Valid handle if success + * - NULL in case of failure. rte_errno will be set to error code + */ +__rte_experimental +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.3 PDCP entity release + * + * Release PDCP entity. + * + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * Entity release would result in freeing all memory associated with the PDCP + * entity as well as any crypto/security sessions created. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be released. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.4 PDCP entity suspend + * + * Suspend PDCP entity. + * + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be suspended. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode 100644 index 0000000000..923e165f3f --- /dev/null +++ b/lib/pdcp/version.map @@ -0,0 +1,10 @@ +EXPERIMENTAL { + global: + + # added in 23.07 + rte_pdcp_entity_establish; + rte_pdcp_entity_release; + rte_pdcp_entity_suspend; + + local: *; +}; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 03/21] pdcp: add pre and post-process 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 02/21] lib: add pdcp protocol Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 04/21] pdcp: add packet group Anoob Joseph ` (18 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Functionality of pre-process & post-process varies based on the type of entity. Registration of entity specific function pointer allows skipping multiple checks that would come in datapath otherwise. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/rte_pdcp.h | 97 ++++++++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 ++ 2 files changed, 100 insertions(+) diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 1f96fdc9a1..46c3c2a416 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -22,6 +22,21 @@ extern "C" { #endif +/* Forward declarations */ +struct rte_pdcp_entity; + +/* PDCP pre-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err); + +/* PDCP post-process function based on entity configuration */ +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err); + /** * PDCP entity. * @@ -33,6 +48,10 @@ extern "C" { * plane depending on which radio bearer it is carrying data for. */ struct rte_pdcp_entity { + /** Entity specific pre-process handle. */ + rte_pdcp_pre_p_t pre_process; + /** Entity specific post-process handle. */ + rte_pdcp_post_p_t post_process; /** * PDCP entities may hold packets for purposes of in-order delivery (in * case of receiving PDCP entity) and re-transmission (in case of @@ -160,6 +179,84 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity pre-process the mbufs and prepare + * crypto ops that can be enqueued to the cryptodev associated with given + * session. Only error packets would be moved returned in the input buffer, + * *mb*, and it is the responsibility of the application to free the same. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param[in, out] mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * which contain the input packets. Any error packets would be returned in the + * same buffer. + * @param[out] cop + * The address of an array that can hold up to *num* pointers to + * *rte_crypto_op* structures. Crypto ops would be allocated by + * ``rte_pdcp_pkt_pre_process`` API. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * Pointer to return the number of error packets returned in *mb* + * @return + * Count of crypto_ops prepared + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err) +{ + return entity->pre_process(entity, mb, cop, num, nb_err); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity, perform PDCP post-processing of the + * mbufs. + * + * Input mbufs are the ones retrieved from crypto_ops dequeued from cryptodev + * and grouped by *rte_pdcp_pkt_crypto_group()*. + * + * The post-processed packets would be returned in the *out_mb* buffer. + * The resultant mbufs would be grouped into success packets and error packets. + * Error packets would be grouped in the end of the array and it is the + * responsibility of the application to handle the same. + * + * When in-order delivery is enabled, PDCP entity may buffer packets and would + * deliver packets only when all prior packets have been post-processed. That + * would result in returning more/less packets than enqueued. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param in_mb + * The address of an array of *num* pointers to *rte_mbuf* structures. + * @param[out] out_mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * to output packets after PDCP post-processing. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * The number of error packets returned in *out_mb* buffer. + * @return + * Count of packets returned in *out_mb* buffer. + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return entity->post_process(entity, in_mb, out_mb, num, nb_err); +} + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 923e165f3f..f9ff30600a 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -6,5 +6,8 @@ EXPERIMENTAL { rte_pdcp_entity_release; rte_pdcp_entity_suspend; + rte_pdcp_pkt_post_process; + rte_pdcp_pkt_pre_process; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 04/21] pdcp: add packet group 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (2 preceding siblings ...) 2023-05-27 8:58 ` [PATCH v5 03/21] pdcp: add pre and post-process Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 05/21] pdcp: add crypto session create and destroy Anoob Joseph ` (17 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Crypto processing in PDCP is performed asynchronously by rte_cryptodev_enqueue_burst() and rte_cryptodev_dequeue_burst(). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 1 + lib/pdcp/rte_pdcp.h | 6 ++ lib/pdcp/rte_pdcp_group.h | 131 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 + 4 files changed, 141 insertions(+) create mode 100644 lib/pdcp/rte_pdcp_group.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index ccaf426240..08679b743a 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -13,5 +13,6 @@ sources = files( 'rte_pdcp.c', ) headers = files('rte_pdcp.h') +indirect_headers += files('rte_pdcp_group.h') deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 46c3c2a416..b88cad4f64 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -257,6 +257,12 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. So include + * in the end. + */ +#include <rte_pdcp_group.h> + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new file mode 100644 index 0000000000..ece3e8c0ff --- /dev/null +++ b/lib/pdcp/rte_pdcp_group.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_GROUP_H +#define RTE_PDCP_GROUP_H + +/** + * @file rte_pdcp_group.h + * + * RTE PDCP grouping support. + * It is not recommended to include this file directly, include <rte_pdcp.h> + * instead. + * Provides helper functions to process completed crypto-ops and group related + * packets by sessions they belong to. + */ + +#include <rte_common.h> +#include <rte_crypto.h> +#include <rte_cryptodev.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Group packets belonging to same PDCP entity. + */ +struct rte_pdcp_group { + union { + uint64_t val; + void *ptr; + } id; /**< Grouped by value */ + struct rte_mbuf **m; /**< Start of the group */ + uint32_t cnt; /**< Number of entries in the group */ + int32_t rc; /**< Status code associated with the group */ +}; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take crypto-op as an input and extract pointer to related PDCP entity. + * @param cop + * The address of an input *rte_crypto_op* structure. + * @return + * The pointer to the related *rte_pdcp_entity* structure. + */ +static inline struct rte_pdcp_entity * +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) +{ + void *sess = cop->sym[0].session; + + return (struct rte_pdcp_entity *)(uintptr_t) + rte_cryptodev_sym_session_opaque_data_get(sess); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take as input completed crypto ops, extract related mbufs and group them by + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* flag + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. + * + * Note that application must ensure only crypto-ops prepared by lib_pdcp is + * provided back to @see rte_pdcp_pkt_crypto_group(). + * + * @param cop + * The address of an array of *num* pointers to the input *rte_crypto_op* + * structures. + * @param[out] mb + * The address of an array of *num* pointers to output *rte_mbuf* structures. + * @param[out] grp + * The address of an array of *num* to output *rte_pdcp_group* structures. + * @param num + * The maximum number of crypto-ops to process. + * @return + * Number of filled elements in *grp* array. + * + */ +static inline uint16_t +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_pdcp_group grp[], uint16_t num) +{ + uint32_t i, j = 0, n = 0; + void *ns, *ps = NULL; + struct rte_mbuf *m; + + for (i = 0; i != num; i++) { + m = cop[i]->sym[0].m_src; + ns = cop[i]->sym[0].session; + + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; + + /* Different entity */ + if (ps != ns) { + + /* Finalize open group and start a new one */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + /* Start new group */ + grp[n].m = mb + j; + ps = ns; + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); + } + + mb[j++] = m; + rte_crypto_op_free(cop[i]); + } + + /* Finalize last group */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + return n; +} + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_GROUP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index f9ff30600a..d564f155e0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,12 +2,15 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_en_from_cop; + rte_pdcp_entity_establish; rte_pdcp_entity_release; rte_pdcp_entity_suspend; rte_pdcp_pkt_post_process; rte_pdcp_pkt_pre_process; + rte_pdcp_pkt_crypto_group; local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 05/21] pdcp: add crypto session create and destroy 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (3 preceding siblings ...) 2023-05-27 8:58 ` [PATCH v5 04/21] pdcp: add packet group Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 06/21] pdcp: add pre and post process for UL Anoob Joseph ` (16 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to create & destroy sessions. PDCP lib would take crypto transforms as input and creates the session on the corresponding device after verifying capabilities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_crypto.c | 223 ++++++++++++++++++++++++++++++++++++++++- lib/pdcp/pdcp_crypto.h | 5 + 2 files changed, 225 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c index 755e27ec9e..6d2a85dc7d 100644 --- a/lib/pdcp/pdcp_crypto.c +++ b/lib/pdcp/pdcp_crypto.c @@ -2,20 +2,237 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_errno.h> #include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> #include "pdcp_crypto.h" +#include "pdcp_entity.h" + +static int +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, + const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm, + bool is_auth_then_cipher) +{ + uint16_t cipher_iv_len, auth_digest_len, auth_iv_len; + int ret; + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protection is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (c_xfrm == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + + /* With UPLINK, if auth is enabled, it should be before cipher */ + if (a_xfrm != NULL && !is_auth_then_cipher) + return -EINVAL; + + /* With UPLINK, cipher operation must be encrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return -EINVAL; + + /* With UPLINK, auth operation (if present) must be generate */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_GENERATE) + return -EINVAL; + + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + + /* With DOWNLINK, if auth is enabled, it should be after cipher */ + if (a_xfrm != NULL && is_auth_then_cipher) + return -EINVAL; + + /* With DOWNLINK, cipher operation must be decrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) + return -EINVAL; + + /* With DOWNLINK, auth operation (if present) must be verify */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_VERIFY) + return -EINVAL; + + } else { + return -EINVAL; + } + + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) + return -EINVAL; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + cipher_iv_len = 0; + else + cipher_iv_len = PDCP_IV_LEN; + + if (cipher_iv_len != c_xfrm->cipher.iv.length) + return -EINVAL; + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + return -EINVAL; + + /* For AUTH NULL, lib PDCP would add 4 byte 0s */ + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + auth_digest_len = 0; + else + auth_digest_len = RTE_PDCP_MAC_I_LEN; + + if (auth_digest_len != a_xfrm->auth.digest_length) + return -EINVAL; + + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + auth_iv_len = PDCP_IV_LEN; + else + auth_iv_len = 0; + + if (a_xfrm->auth.iv.length != auth_iv_len) + return -EINVAL; + } + + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) + return -EINVAL; + + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); + if (ret) + return -ENOTSUP; + + if (a_xfrm != NULL) { + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); + if (ret) + return -ENOTSUP; + } + + return 0; +} int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { - RTE_SET_USED(entity); - RTE_SET_USED(conf); + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + struct entity_priv *en_priv; + bool is_auth_then_cipher; + int ret; + + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + en_priv->dev_id = conf->dev_id; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + c_xfrm = conf->crypto_xfrm; + a_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = false; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + a_xfrm = conf->crypto_xfrm; + c_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = true; + } else { + return -EINVAL; + } + + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, is_auth_then_cipher); + if (ret) + return ret; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + c_xfrm->cipher.iv.offset = 0; + else + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; + + if (a_xfrm != NULL) { + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm->auth.iv.offset = 0; + else + if (c_xfrm->cipher.iv.offset) + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + PDCP_IV_LEN; + else + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; + } + + if (conf->sess_mpool == NULL) + return -EINVAL; + + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf->dev_id, conf->crypto_xfrm, + conf->sess_mpool); + if (en_priv->crypto_sess == NULL) { + /* rte_errno is set as positive values of error codes */ + return -rte_errno; + } + + rte_cryptodev_sym_session_opaque_data_set(en_priv->crypto_sess, (uint64_t)entity); + return 0; } void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) { - RTE_SET_USED(entity); + struct entity_priv *en_priv; + + en_priv = entity_priv_get(entity); + + if (en_priv->crypto_sess != NULL) { + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv->crypto_sess); + en_priv->crypto_sess = NULL; + } } diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h index 6563331d37..f694818713 100644 --- a/lib/pdcp/pdcp_crypto.h +++ b/lib/pdcp/pdcp_crypto.h @@ -5,8 +5,13 @@ #ifndef PDCP_CRYPTO_H #define PDCP_CRYPTO_H +#include <rte_crypto.h> +#include <rte_crypto_sym.h> #include <rte_pdcp.h> +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op)) +#define PDCP_IV_LEN 16 + int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 06/21] pdcp: add pre and post process for UL 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (4 preceding siblings ...) 2023-05-27 8:58 ` [PATCH v5 05/21] pdcp: add crypto session create and destroy Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 07/21] pdcp: add pre and post process for DL Anoob Joseph ` (15 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to perform pre & post processing based on the type of entity. To avoid checks in datapath, there are different function pointers registered based on the following, 1. Control plane v/s user plane 2. 12 bit v/s 18 bit SN For control plane only 12 bit SN need to be supported (as per PDCP specification). Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_entity.h | 24 +++ lib/pdcp/pdcp_process.c | 334 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 358 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 000297588f..23628ebad4 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -92,22 +92,46 @@ pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static inline uint32_t +pdcp_window_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return 1 << (sn_size - 1); +} + static inline uint32_t pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return (1 << sn_size) - 1; } +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + static inline uint32_t pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return ~pdcp_sn_mask_get(sn_size); } +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline uint32_t pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) { return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); } +static inline uint32_t +pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << (32 - sn_size)) - 1; +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 79f5dce5db..9b7de39db6 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -36,6 +36,336 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static inline void +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, + uint8_t data_offset, uint32_t count, const bool is_auth) +{ + const struct rte_crypto_op cop_init = { + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, + }; + struct rte_crypto_sym_op *op; + uint32_t pkt_len; + + const uint8_t cipher_shift = 3 * en_priv->flags.is_cipher_in_bits; + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; + + op = cop->sym; + cop->raw = cop_init.raw; + op->m_src = mb; + op->m_dst = mb; + + /* Set IV */ + en_priv->iv_gen(cop, en_priv, count); + + /* Prepare op */ + pkt_len = rte_pktmbuf_pkt_len(mb); + op->cipher.data.offset = data_offset << cipher_shift; + op->cipher.data.length = (pkt_len - data_offset) << cipher_shift; + + if (is_auth) { + op->auth.data.offset = 0; + op->auth.data.length = (pkt_len - RTE_PDCP_MAC_I_LEN) << auth_shift; + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t *, + (pkt_len - RTE_PDCP_MAC_I_LEN)); + } + + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); +} + +static inline bool +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + return true; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline bool +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + return true; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count, sn; + uint8_t *mac_i; + int i; + + const uint8_t hdr_sz = en_priv->hdr_sz; + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + /* Update sequence number in the PDU header */ + count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static int +pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + entity->pre_process = NULL; + entity->post_process = NULL; + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + entity->post_process = pdcp_post_process_ul; + } + + if (entity->pre_process == NULL || entity->post_process == NULL) + return -ENOTSUP; + + return 0; +} + static int pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) { @@ -134,5 +464,9 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti if (ret) return ret; + ret = pdcp_pre_post_func_set(entity, conf); + if (ret) + return ret; + return 0; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 07/21] pdcp: add pre and post process for DL 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (5 preceding siblings ...) 2023-05-27 8:58 ` [PATCH v5 06/21] pdcp: add pre and post process for UL Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 08/21] pdcp: add IV generation routines Anoob Joseph ` (14 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to perform pre & post processing for down link entities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 384 ++++++++++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 11 ++ lib/pdcp/rte_pdcp.c | 23 +++ 4 files changed, 420 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 23628ebad4..1d4a43a3bc 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -13,6 +13,8 @@ struct entity_priv; +#define PDCP_HFN_MIN 0 + /* IV generation function based on the entity configuration */ typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count); diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 9b7de39db6..bd75e6f802 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -6,6 +6,7 @@ #include <rte_crypto_sym.h> #include <rte_cryptodev.h> #include <rte_memcpy.h> +#include <rte_mbuf_dyn.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> @@ -333,9 +334,353 @@ pdcp_post_process_ul(const struct rte_pdcp_entity *entity, return nb_success; } +static inline int +pdcp_sn_count_get(const uint32_t rx_deliv, int32_t rsn, uint32_t *count, + const enum rte_security_pdcp_sn_size sn_size) +{ + const uint32_t rx_deliv_sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + const uint32_t window_sz = pdcp_window_size_get(sn_size); + uint32_t rhfn; + + rhfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + + if (rsn < (int32_t)(rx_deliv_sn - window_sz)) { + if (unlikely(rhfn == pdcp_hfn_max(sn_size))) + return -ERANGE; + rhfn += 1; + } else if ((uint32_t)rsn >= (rx_deliv_sn + window_sz)) { + if (unlikely(rhfn == PDCP_HFN_MIN)) + return -ERANGE; + rhfn -= 1; + } + + *count = pdcp_count_from_hfn_sn_get(rhfn, rsn, sn_size); + + return 0; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_18))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline void +pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool trim_mac) +{ + char *p = rte_pktmbuf_adj(mb, hdr_trim_sz); + RTE_ASSERT(p != NULL); + RTE_SET_USED(p); + + if (trim_mac) { + int ret = rte_pktmbuf_trim(mb, RTE_PDCP_MAC_I_LEN); + RTE_ASSERT(ret == 0); + RTE_SET_USED(ret); + } +} + +static inline bool +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, + const uint32_t count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + + if (count < en_priv->state.rx_deliv) + return false; + + /* t-Reordering timer is not supported - SDU will be delivered immediately. + * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not + * been delivered to upper layers + */ + en_priv->state.rx_next = count + 1; + + if (count >= en_priv->state.rx_next) + en_priv->state.rx_next = count + 1; + + return true; +} + +static inline uint16_t +pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + mb_dynfield = pdcp_dynfield(mb); + count = *mb_dynfield; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static uint16_t +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + mb_dynfield = pdcp_dynfield(mb); + count = *mb_dynfield; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, true); + + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + static int pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { + struct entity_priv *en_priv = entity_priv_get(entity); + entity->pre_process = NULL; entity->post_process = NULL; @@ -346,6 +691,13 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; + entity->post_process = pdcp_post_process_cplane_sn_12_dl; + } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { @@ -360,6 +712,38 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; + entity->post_process = pdcp_post_process_uplane_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; + entity->post_process = pdcp_post_process_uplane_dl; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; + entity->post_process = pdcp_post_process_uplane_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; + entity->post_process = pdcp_post_process_uplane_dl; + } + if (entity->pre_process == NULL || entity->post_process == NULL) return -ENOTSUP; diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index fd53fff0aa..a52f769b82 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -5,8 +5,19 @@ #ifndef PDCP_PROCESS_H #define PDCP_PROCESS_H +#include <rte_mbuf_dyn.h> #include <rte_pdcp.h> +typedef uint32_t rte_pdcp_dynfield_t; + +extern int rte_pdcp_dynfield_offset; + +static inline rte_pdcp_dynfield_t * +pdcp_dynfield(struct rte_mbuf *mbuf) +{ + return RTE_MBUF_DYNFIELD(mbuf, rte_pdcp_dynfield_offset, rte_pdcp_dynfield_t *); +} + int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index adcad5dd25..91dab91f73 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -10,6 +10,26 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +#define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" + +int rte_pdcp_dynfield_offset = -1; + +static int +pdcp_dynfield_register(void) +{ + const struct rte_mbuf_dynfield dynfield_desc = { + .name = RTE_PDCP_DYNFIELD_NAME, + .size = sizeof(rte_pdcp_dynfield_t), + .align = __alignof__(rte_pdcp_dynfield_t), + }; + + if (rte_pdcp_dynfield_offset != -1) + return rte_pdcp_dynfield_offset; + + rte_pdcp_dynfield_offset = rte_mbuf_dynfield_register(&dynfield_desc); + return rte_pdcp_dynfield_offset; +} + static int pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { @@ -35,6 +55,9 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) int ret, entity_size; uint32_t count; + if (pdcp_dynfield_register() < 0) + return NULL; + if (conf == NULL || conf->cop_pool == NULL) { rte_errno = EINVAL; return NULL; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 08/21] pdcp: add IV generation routines 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (6 preceding siblings ...) 2023-05-27 8:58 ` [PATCH v5 07/21] pdcp: add pre and post process for DL Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 09/21] app/test: add lib pdcp tests Anoob Joseph ` (13 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger For PDCP, IV generated has varying formats depending on the ciphering and authentication algorithm used. Add routines to populate IV accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 87 ++++++++++++ lib/pdcp/pdcp_process.c | 284 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 371 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 1d4a43a3bc..10a72faae1 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -26,6 +26,89 @@ struct entity_state { uint32_t rx_reord; }; +union auth_iv_partial { + /* For AES-CMAC, there is no IV, but message gets prepended */ + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + } aes_cmac; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_37_39 : 3; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_65_71 : 7; + uint64_t direction_64 : 1; + uint64_t rsvd_72_111 : 40; + uint64_t rsvd_113_119 : 7; + uint64_t direction_112 : 1; + uint64_t rsvd_120_127 : 8; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t zero_37_39 : 3; + uint64_t zero_40_63 : 24; + + uint64_t direction_64 : 1; + uint64_t rsvd_65_71 : 7; + uint64_t rsvd_72_111 : 40; + uint64_t direction_112 : 1; + uint64_t rsvd_113_119 : 7; + uint64_t rsvd_120_127 : 8; +#endif + } zs; + uint64_t u64[2]; +}; + +union cipher_iv_partial { + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t zero_64_127; + } aes_ctr; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t rsvd_64_127; + } zs; + uint64_t u64[2]; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -35,6 +118,10 @@ struct entity_priv { struct rte_cryptodev_sym_session *crypto_sess; /** Entity specific IV generation function. */ iv_gen_t iv_gen; + /** Pre-prepared auth IV. */ + union auth_iv_partial auth_iv_part; + /** Pre-prepared cipher IV. */ + union cipher_iv_partial cipher_iv_part; /** Entity state variables. */ struct entity_state state; /** Flags. */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index bd75e6f802..28ac4102da 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -14,6 +14,181 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +/* Enum of supported algorithms for ciphering */ +enum pdcp_cipher_algo { + PDCP_CIPHER_ALGO_NULL, + PDCP_CIPHER_ALGO_AES, + PDCP_CIPHER_ALGO_ZUC, + PDCP_CIPHER_ALGO_SNOW3G, + PDCP_CIPHER_ALGO_MAX +}; + +/* Enum of supported algorithms for integrity */ +enum pdcp_auth_algo { + PDCP_AUTH_ALGO_NULL, + PDCP_AUTH_ALGO_AES, + PDCP_AUTH_ALGO_ZUC, + PDCP_AUTH_ALGO_SNOW3G, + PDCP_AUTH_ALGO_MAX +}; + +/* IV generation functions based on type of operation (cipher - auth) */ + +static void +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + /* No IV required for NULL cipher + NULL auth */ + RTE_SET_USED(cop); + RTE_SET_USED(en_priv); + RTE_SET_USED(count); +} + +static void +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr; + uint64_t m; + + /* AES-CMAC requires message to be prepended with info on count etc */ + + /* Prepend by 8 bytes to add custom message */ + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, 16); +} + +static void +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64 = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64, 8); + rte_memcpy(iv + 8, &iv_u64, 8); +} + +static void +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + /* Generating cipher IV */ + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + iv += PDCP_IV_LEN; + + /* Generating auth IV */ + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + iv += PDCP_IV_LEN; + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + static int pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, struct rte_crypto_sym_xform **a_xfrm) @@ -37,6 +212,111 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static int +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + enum rte_security_pdcp_direction direction; + enum pdcp_cipher_algo cipher_algo; + enum pdcp_auth_algo auth_algo; + struct entity_priv *en_priv; + int ret; + + en_priv = entity_priv_get(entity); + + direction = conf->pdcp_xfrm.pkt_dir; + if (conf->reverse_iv_direction) + direction = !direction; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (c_xfrm == NULL) + return -EINVAL; + + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); + + switch (c_xfrm->cipher.algo) { + case RTE_CRYPTO_CIPHER_NULL: + cipher_algo = PDCP_CIPHER_ALGO_NULL; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + cipher_algo = PDCP_CIPHER_ALGO_AES; + en_priv->cipher_iv_part.aes_ctr.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.aes_ctr.direction = direction; + break; + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + cipher_algo = PDCP_CIPHER_ALGO_SNOW3G; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + cipher_algo = PDCP_CIPHER_ALGO_ZUC; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + default: + return -ENOTSUP; + } + + if (a_xfrm != NULL) { + switch (a_xfrm->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + auth_algo = PDCP_AUTH_ALGO_NULL; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + auth_algo = PDCP_AUTH_ALGO_AES; + en_priv->auth_iv_part.aes_cmac.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.aes_cmac.direction = direction; + break; + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + auth_algo = PDCP_AUTH_ALGO_SNOW3G; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + case RTE_CRYPTO_AUTH_ZUC_EIA3: + auth_algo = PDCP_AUTH_ALGO_ZUC; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + default: + return -ENOTSUP; + } + } else { + auth_algo = PDCP_AUTH_ALGO_NULL; + } + + static const iv_gen_t iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_null_null, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_null_aes_cmac, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_null_zs, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_null_zs, + + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_aes_ctr_null, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_aes_ctr_aes_cmac, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_aes_ctr_zs, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_aes_ctr_zs, + + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + }; + + en_priv->iv_gen = iv_gen_map[cipher_algo][auth_algo]; + + return 0; +} + static inline void cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, uint8_t data_offset, uint32_t count, const bool is_auth) @@ -844,6 +1124,10 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti en_priv = entity_priv_get(entity); + ret = pdcp_iv_gen_func_set(entity, conf); + if (ret) + return ret; + ret = pdcp_entity_priv_populate(en_priv, conf); if (ret) return ret; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 09/21] app/test: add lib pdcp tests 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (7 preceding siblings ...) 2023-05-27 8:58 ` [PATCH v5 08/21] pdcp: add IV generation routines Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph ` (12 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add tests to verify lib PDCP operations. Tests leverage existing PDCP test vectors. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 1 + app/test/meson.build | 5 + app/test/test_pdcp.c | 732 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 738 insertions(+) create mode 100644 app/test/test_pdcp.c diff --git a/MAINTAINERS b/MAINTAINERS index 85a3b94644..15dd88af3a 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1554,6 +1554,7 @@ M: Anoob Joseph <anoobj@marvell.com> M: Volodymyr Fialko <vfialko@marvell.com> T: git://dpdk.org/next/dpdk-next-crypto F: lib/pdcp/ +F: app/test/test_pdcp* Packet Framework diff --git a/app/test/meson.build b/app/test/meson.build index b9b5432496..2894793347 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -435,6 +435,11 @@ if dpdk_conf.has('RTE_HAS_LIBPCAP') endif endif +if dpdk_conf.has('RTE_LIB_PDCP') + test_sources += 'test_pdcp.c' + fast_tests += [['pdcp_autotest', false, true]] +endif + if cc.has_argument('-Wno-format-truncation') cflags += '-Wno-format-truncation' endif diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c new file mode 100644 index 0000000000..34b759eaef --- /dev/null +++ b/app/test/test_pdcp.c @@ -0,0 +1,732 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_malloc.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "test.h" +#include "test_cryptodev.h" +#include "test_cryptodev_security_pdcp_test_vectors.h" + +#define NB_DESC 1024 +#define CDEV_INVALID_ID UINT8_MAX +#define NB_TESTS RTE_DIM(pdcp_test_params) +#define PDCP_IV_LEN 16 + +struct pdcp_testsuite_params { + struct rte_mempool *mbuf_pool; + struct rte_mempool *cop_pool; + struct rte_mempool *sess_pool; + bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; +}; + +static struct pdcp_testsuite_params testsuite_params; + +struct pdcp_test_conf { + struct rte_pdcp_entity_conf entity; + struct rte_crypto_sym_xform c_xfrm; + struct rte_crypto_sym_xform a_xfrm; + bool is_integrity_protected; + uint8_t input[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t input_len; + uint8_t output[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t output_len; +}; + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static int +cryptodev_init(int dev_id) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_cryptodev_qp_conf qp_conf; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_config config; + int ret, socket_id; + + /* Check if device was already initialized */ + if (ts_params->cdevs_used[dev_id]) + return 0; + + rte_cryptodev_info_get(dev_id, &dev_info); + + if (dev_info.max_nb_queue_pairs < 1) { + RTE_LOG(ERR, USER1, "Cryptodev doesn't have sufficient queue pairs available\n"); + return -ENODEV; + } + + socket_id = rte_socket_id(); + + memset(&config, 0, sizeof(config)); + config.nb_queue_pairs = 1; + config.socket_id = socket_id; + + ret = rte_cryptodev_configure(dev_id, &config); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure cryptodev - %d\n", dev_id); + return -ENODEV; + } + + memset(&qp_conf, 0, sizeof(qp_conf)); + qp_conf.nb_descriptors = NB_DESC; + + ret = rte_cryptodev_queue_pair_setup(dev_id, 0, &qp_conf, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure queue pair\n"); + return -ENODEV; + } + + ret = rte_cryptodev_start(dev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not start cryptodev\n"); + return -ENODEV; + } + + /* Mark device as initialized */ + ts_params->cdevs_used[dev_id] = true; + + return 0; +} + +static void +cryptodev_fini(int dev_id) +{ + rte_cryptodev_stop(dev_id); +} + +static unsigned int +cryptodev_sess_priv_max_req_get(void) +{ + struct rte_cryptodev_info info; + unsigned int sess_priv_sz; + int i, nb_dev; + void *sec_ctx; + + nb_dev = rte_cryptodev_count(); + + sess_priv_sz = 0; + + for (i = 0; i < nb_dev; i++) { + rte_cryptodev_info_get(i, &info); + sess_priv_sz = RTE_MAX(sess_priv_sz, rte_cryptodev_sym_get_private_session_size(i)); + if (info.feature_flags & RTE_CRYPTODEV_FF_SECURITY) { + sec_ctx = rte_cryptodev_get_sec_ctx(i); + sess_priv_sz = RTE_MAX(sess_priv_sz, + rte_security_session_get_size(sec_ctx)); + } + } + + return sess_priv_sz; +} + +static int +testsuite_setup(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + int nb_cdev, sess_priv_size, nb_sess = 1024; + + RTE_SET_USED(pdcp_test_hfn_threshold); + + nb_cdev = rte_cryptodev_count(); + if (nb_cdev < 1) { + RTE_LOG(ERR, USER1, "No crypto devices found.\n"); + return TEST_SKIPPED; + } + + memset(ts_params, 0, sizeof(*ts_params)); + + ts_params->mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NUM_MBUFS, MBUF_CACHE_SIZE, 0, + MBUF_SIZE, SOCKET_ID_ANY); + if (ts_params->mbuf_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf pool\n"); + return TEST_FAILED; + } + + ts_params->cop_pool = rte_crypto_op_pool_create("cop_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, MBUF_CACHE_SIZE, + 2 * MAXIMUM_IV_LENGTH, SOCKET_ID_ANY); + if (ts_params->cop_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create crypto_op pool\n"); + goto mbuf_pool_free; + } + + /* Get max session priv size required */ + sess_priv_size = cryptodev_sess_priv_max_req_get(); + + ts_params->sess_pool = rte_cryptodev_sym_session_pool_create("sess_pool", nb_sess, + sess_priv_size, + RTE_MEMPOOL_CACHE_MAX_SIZE, + 0, SOCKET_ID_ANY); + if (ts_params->sess_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create session pool\n"); + goto cop_pool_free; + } + + return 0; + +cop_pool_free: + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; +mbuf_pool_free: + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; + return TEST_FAILED; +} + +static void +testsuite_teardown(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + uint8_t dev_id; + + for (dev_id = 0; dev_id < RTE_CRYPTO_MAX_DEVS; dev_id++) { + if (ts_params->cdevs_used[dev_id]) + cryptodev_fini(dev_id); + } + + rte_mempool_free(ts_params->sess_pool); + ts_params->sess_pool = NULL; + + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; + + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; +} + +static int +ut_setup_pdcp(void) +{ + return 0; +} + +static void +ut_teardown_pdcp(void) +{ +} + +static int +crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +cryptodev_id_get(bool is_integrity_protected, const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm) +{ + int i, nb_devs; + + nb_devs = rte_cryptodev_count(); + + /* Check capabilities */ + + for (i = 0; i < nb_devs; i++) { + if ((crypto_caps_cipher_verify(i, c_xfrm) == 0) && + (!is_integrity_protected || crypto_caps_auth_verify(i, a_xfrm) == 0)) + break; + } + + if (i == nb_devs) + return -1; + + return i; +} + +static int +pdcp_known_vec_verify(struct rte_mbuf *m, const uint8_t *expected, uint32_t expected_pkt_len) +{ + uint8_t *actual = rte_pktmbuf_mtod(m, uint8_t *); + uint32_t actual_pkt_len = rte_pktmbuf_pkt_len(m); + + debug_hexdump(stdout, "Received:", actual, actual_pkt_len); + debug_hexdump(stdout, "Expected:", expected, expected_pkt_len); + + TEST_ASSERT_EQUAL(actual_pkt_len, expected_pkt_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_pkt_len, actual_pkt_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(actual, expected, expected_pkt_len, + "Generated packet not as expected"); + + return 0; +} + +static struct rte_crypto_op * +process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op) +{ + if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) { + RTE_LOG(ERR, USER1, "Error sending packet to cryptodev\n"); + return NULL; + } + + op = NULL; + + while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0) + rte_pause(); + + return op; +} + +static uint32_t +pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) +{ + uint32_t sn = 0; + + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + sn = rte_cpu_to_be_16(*(const uint16_t *)data); + sn = sn & 0xfff; + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + sn = rte_cpu_to_be_32(*(const uint32_t *)data); + sn = (sn & 0x3ffff00) >> 8; + } + + return sn; +} + +static int +create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_crypto_sym_xform c_xfrm, a_xfrm; + uint32_t sn, expected_len; + uint8_t *data, *expected; + int pdcp_hdr_sz; + + memset(conf, 0, sizeof(*conf)); + memset(&c_xfrm, 0, sizeof(c_xfrm)); + memset(&a_xfrm, 0, sizeof(a_xfrm)); + + conf->entity.sess_mpool = ts_params->sess_pool; + conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; + conf->entity.pdcp_xfrm.en_ordering = 0; + conf->entity.pdcp_xfrm.remove_duplicates = 0; + conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + + if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; + else + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + + conf->entity.pdcp_xfrm.sn_size = pdcp_test_data_sn_size[index]; + + /* Zero initialize unsupported flags */ + conf->entity.pdcp_xfrm.hfn_threshold = 0; + conf->entity.pdcp_xfrm.hfn_ovrd = 0; + conf->entity.pdcp_xfrm.sdap_enabled = 0; + + c_xfrm.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + c_xfrm.cipher.algo = pdcp_test_params[index].cipher_alg; + c_xfrm.cipher.key.length = pdcp_test_params[index].cipher_key_len; + c_xfrm.cipher.key.data = pdcp_test_crypto_key[index]; + + a_xfrm.type = RTE_CRYPTO_SYM_XFORM_AUTH; + + if (pdcp_test_params[index].auth_alg == 0) { + conf->is_integrity_protected = false; + } else { + a_xfrm.auth.algo = pdcp_test_params[index].auth_alg; + a_xfrm.auth.key.data = pdcp_test_auth_key[index]; + a_xfrm.auth.key.length = pdcp_test_params[index].auth_key_len; + conf->is_integrity_protected = true; + } + + pdcp_hdr_sz = pdcp_hdr_size_get(pdcp_test_data_sn_size[index]); + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protecting is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (conf->is_integrity_protected) { + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + conf->entity.crypto_xfrm = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + a_xfrm.next = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + c_xfrm.next = NULL; + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + c_xfrm.next = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + a_xfrm.next = NULL; + } + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + c_xfrm.next = NULL; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + else + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + /* Update xforms to match PDCP requirements */ + + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) + c_xfrm.cipher.iv.length = PDCP_IV_LEN; + else + c_xfrm.cipher.iv.length = 0; + + if (conf->is_integrity_protected) { + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm.auth.digest_length = 0; + else + a_xfrm.auth.digest_length = RTE_PDCP_MAC_I_LEN; + + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + a_xfrm.auth.iv.length = PDCP_IV_LEN; + else + a_xfrm.auth.iv.length = 0; + } + + conf->c_xfrm = c_xfrm; + conf->a_xfrm = a_xfrm; + + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf->is_integrity_protected, + &conf->c_xfrm, &conf->a_xfrm); + + if (pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_CONTROL || + pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_DATA) { + data = pdcp_test_data_in[index]; + sn = pdcp_sn_from_raw_get(data, pdcp_test_data_sn_size[index]); + conf->entity.pdcp_xfrm.hfn = pdcp_test_hfn[index]; + conf->entity.sn = sn; + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", pdcp_test_data_in[index], + pdcp_test_data_in_len[index]); +#endif + /* Since the vectors available already have PDCP header, trim the same */ + conf->input_len = pdcp_test_data_in_len[index] - pdcp_hdr_sz; + memcpy(conf->input, pdcp_test_data_in[index] + pdcp_hdr_sz, conf->input_len); + } else { + conf->input_len = pdcp_test_data_in_len[index]; + + if (conf->is_integrity_protected) + conf->input_len += RTE_PDCP_MAC_I_LEN; + + memcpy(conf->input, pdcp_test_data_out[index], conf->input_len); +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", conf->input, conf->input_len); +#endif + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected = pdcp_test_data_out[index]; + else + expected = pdcp_test_data_in[index]; + + /* Calculate expected packet length */ + expected_len = pdcp_test_data_in_len[index]; + + /* In DL processing, PDCP header would be stripped */ + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + expected += pdcp_hdr_sz; + expected_len -= pdcp_hdr_sz; + } + + /* In UL processing with integrity protection, MAC would be added */ + if (conf->is_integrity_protected && + conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected_len += 4; + + memcpy(conf->output, expected, expected_len); + conf->output_len = expected_len; + + return 0; +} + +static struct rte_pdcp_entity* +test_entity_create(const struct pdcp_test_conf *t_conf, int *rc) +{ + struct rte_pdcp_entity *pdcp_entity; + int ret; + + if (t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12 && + t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18) { + *rc = -ENOTSUP; + return NULL; + } + + if (t_conf->entity.dev_id == CDEV_INVALID_ID) { + RTE_LOG(DEBUG, USER1, "Could not find device with required capabilities\n"); + *rc = -ENOTSUP; + return NULL; + } + + ret = cryptodev_init(t_conf->entity.dev_id); + if (ret) { + *rc = ret; + RTE_LOG(DEBUG, USER1, "Could not initialize cryptodev\n"); + return NULL; + } + + rte_errno = 0; + + pdcp_entity = rte_pdcp_entity_establish(&t_conf->entity); + if (pdcp_entity == NULL) { + *rc = -rte_errno; + RTE_LOG(DEBUG, USER1, "Could not establish PDCP entity\n"); + return NULL; + } + + return pdcp_entity; +} + +static uint16_t +test_process_packets(const struct rte_pdcp_entity *pdcp_entity, uint8_t cdev_id, + struct rte_mbuf *in_mb[], uint16_t nb_in, + struct rte_mbuf *out_mb[], uint16_t *nb_err) +{ + struct rte_crypto_op *cop, *cop_out; + struct rte_pdcp_group grp[1]; + uint16_t nb_success, nb_grp; + struct rte_mbuf *mbuf, *mb; + + if (nb_in != 1) + return -ENOTSUP; + + mbuf = in_mb[0]; + + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, &mbuf, &cop_out, 1, nb_err); + if (nb_success != 1 || *nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not pre process PDCP packet\n"); + return TEST_FAILED; + } + +#ifdef VEC_DUMP + printf("Pre-processed vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, rte_pktmbuf_pkt_len(mbuf)); +#endif + + cop = process_crypto_request(cdev_id, cop_out); + if (cop == NULL) { + RTE_LOG(ERR, USER1, "Could not process crypto request\n"); + return -EIO; + } + + grp[0].id.val = 0; + + nb_grp = rte_pdcp_pkt_crypto_group(&cop_out, &mb, grp, 1); + if (nb_grp != 1 || grp[0].cnt != 1) { + RTE_LOG(ERR, USER1, "Could not group PDCP crypto results\n"); + return -ENOTRECOVERABLE; + } + + if ((uintptr_t)pdcp_entity != grp[0].id.val) { + RTE_LOG(ERR, USER1, "PDCP entity not matching the one from crypto_op\n"); + return -ENOTRECOVERABLE; + } + +#ifdef VEC_DUMP + printf("Crypto processed vector:\n"); + rte_pktmbuf_dump(stdout, cop->sym->m_dst, rte_pktmbuf_pkt_len(mbuf)); +#endif + + return rte_pdcp_pkt_post_process(grp[0].id.ptr, grp[0].m, out_mb, grp[0].cnt, nb_err); +} + +static struct rte_mbuf* +mbuf_from_data_create(uint8_t *data, uint16_t data_len) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_mbuf *mbuf; + uint8_t *input_text; + + mbuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (mbuf == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf\n"); + return NULL; + } + + memset(rte_pktmbuf_mtod(mbuf, uint8_t *), 0, rte_pktmbuf_tailroom(mbuf)); + + input_text = (uint8_t *)rte_pktmbuf_append(mbuf, data_len); + memcpy(input_text, data, data_len); + + return mbuf; +} + +static int +test_attempt_single(struct pdcp_test_conf *t_conf) +{ + struct rte_mbuf *mbuf, **out_mb = NULL; + struct rte_pdcp_entity *pdcp_entity; + uint16_t nb_success, nb_err; + int ret = 0, nb_max_out_mb; + + pdcp_entity = test_entity_create(t_conf, &ret); + if (pdcp_entity == NULL) + goto exit; + + /* Allocate buffer for holding mbufs returned */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate buffer for holding out_mb buffers\n"); + ret = -ENOMEM; + goto entity_release; + } + + mbuf = mbuf_from_data_create(t_conf->input, t_conf->input_len); + if (mbuf == NULL) { + ret = -ENOMEM; + goto entity_release; + } + +#ifdef VEC_DUMP + printf("Adjusted vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, t_conf->input_len); +#endif + + nb_success = test_process_packets(pdcp_entity, t_conf->entity.dev_id, &mbuf, 1, out_mb, + &nb_err); + if (nb_success != 1 || nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not process PDCP packet\n"); + ret = TEST_FAILED; + goto mbuf_free; + } + + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + + ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); + if (ret) { + RTE_LOG(DEBUG, USER1, "Could not suspend PDCP entity\n"); + goto mbuf_free; + } + +mbuf_free: + rte_pktmbuf_free(mbuf); +entity_release: + rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_free(out_mb); +exit: + return ret; +} + +static int +run_test_for_one_known_vec(const void *arg) +{ + struct pdcp_test_conf test_conf; + int i = *(const uint32_t *)arg; + + create_test_conf_from_index(i, &test_conf); + return test_attempt_single(&test_conf); +} + +struct unit_test_suite *test_suites[] = { + NULL, /* Place holder for known_vector_cases */ + NULL /* End of suites list */ +}; + +static struct unit_test_suite pdcp_testsuite = { + .suite_name = "PDCP Unit Test Suite", + .unit_test_cases = {TEST_CASES_END()}, + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_suites = test_suites, +}; + +static int +test_pdcp(void) +{ + struct unit_test_suite *known_vector_cases; + int ret, index[NB_TESTS]; + uint32_t i, size; + + size = sizeof(struct unit_test_suite); + size += (NB_TESTS + 1) * sizeof(struct unit_test_case); + + known_vector_cases = rte_zmalloc(NULL, size, 0); + if (known_vector_cases == NULL) + return TEST_FAILED; + + known_vector_cases->suite_name = "Known vector cases"; + + for (i = 0; i < NB_TESTS; i++) { + index[i] = i; + known_vector_cases->unit_test_cases[i].name = pdcp_test_params[i].name; + known_vector_cases->unit_test_cases[i].data = (void *)&index[i]; + known_vector_cases->unit_test_cases[i].enabled = 1; + known_vector_cases->unit_test_cases[i].setup = ut_setup_pdcp; + known_vector_cases->unit_test_cases[i].teardown = ut_teardown_pdcp; + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data + = run_test_for_one_known_vec; + } + + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data = NULL; + + test_suites[0] = known_vector_cases; + + ret = unit_test_suite_runner(&pdcp_testsuite); + + rte_free(known_vector_cases); + return ret; +} + +REGISTER_TEST_COMMAND(pdcp_autotest, test_pdcp); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 10/21] test/pdcp: pdcp HFN tests in combined mode 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (8 preceding siblings ...) 2023-05-27 8:58 ` [PATCH v5 09/21] app/test: add lib pdcp tests Anoob Joseph @ 2023-05-27 8:58 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 11/21] doc: add PDCP library guide Anoob Joseph ` (11 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:58 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add tests to verify HFN/SN behaviour. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 302 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 299 insertions(+), 3 deletions(-) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 34b759eaef..cfe2ec6aa9 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,9 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* According to formula(7.2.a Window_Size) */ +#define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) + struct pdcp_testsuite_params { struct rte_mempool *mbuf_pool; struct rte_mempool *cop_pool; @@ -36,12 +39,69 @@ struct pdcp_test_conf { uint32_t output_len; }; +static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf); + +typedef int (*test_with_conf_t)(struct pdcp_test_conf *conf); + +static int +run_test_foreach_known_vec(test_with_conf_t test, bool stop_on_first_pass) +{ + struct pdcp_test_conf test_conf; + bool all_tests_skipped = true; + uint32_t i; + int ret; + + for (i = 0; i < NB_TESTS; i++) { + create_test_conf_from_index(i, &test_conf); + ret = test(&test_conf); + + if (ret == TEST_FAILED) { + printf("[%03i] - %s - failed\n", i, pdcp_test_params[i].name); + return TEST_FAILED; + } + + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + continue; + + if (stop_on_first_pass) + return TEST_SUCCESS; + + all_tests_skipped = false; + } + + if (all_tests_skipped) + return TEST_SKIPPED; + + return TEST_SUCCESS; +} + +static int +run_test_with_all_known_vec(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, false); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static int +pktmbuf_read_into(const struct rte_mbuf *m, void *buf, size_t buf_len) +{ + if (m->pkt_len > buf_len) + return -ENOMEM; + + const void *read = rte_pktmbuf_read(m, 0, m->pkt_len, buf); + if (read != NULL && read != buf) + memcpy(buf, read, m->pkt_len); + + return 0; +} + static int cryptodev_init(int dev_id) { @@ -326,6 +386,21 @@ pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) return sn; } +static void +pdcp_sn_to_raw_set(void *data, uint32_t sn, int size) +{ + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr = data; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr = data; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } +} + static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) { @@ -648,9 +723,17 @@ test_attempt_single(struct pdcp_test_conf *t_conf) goto mbuf_free; } - ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); - if (ret) - goto mbuf_free; + /* If expected output provided - verify, else - store for future use */ + if (t_conf->output_len) { + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + } else { + ret = pktmbuf_read_into(mbuf, t_conf->output, RTE_PDCP_CTRL_PDU_SIZE_MAX); + if (ret) + goto mbuf_free; + t_conf->output_len = mbuf->pkt_len; + } ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); if (ret) { @@ -667,6 +750,193 @@ test_attempt_single(struct pdcp_test_conf *t_conf) return ret; } +static void +uplink_to_downlink_convert(const struct pdcp_test_conf *ul_cfg, + struct pdcp_test_conf *dl_cfg) +{ + assert(ul_cfg->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK); + + memcpy(dl_cfg, ul_cfg, sizeof(*dl_cfg)); + dl_cfg->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + dl_cfg->entity.reverse_iv_direction = false; + + if (dl_cfg->is_integrity_protected) { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + dl_cfg->c_xfrm.next = &dl_cfg->a_xfrm; + + dl_cfg->a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + dl_cfg->a_xfrm.next = NULL; + } else { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + dl_cfg->c_xfrm.next = NULL; + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + dl_cfg->entity.dev_id = (uint8_t)cryptodev_id_get(dl_cfg->is_integrity_protected, + &dl_cfg->c_xfrm, &dl_cfg->a_xfrm); + + memcpy(dl_cfg->input, ul_cfg->output, ul_cfg->output_len); + dl_cfg->input_len = ul_cfg->output_len; + + memcpy(dl_cfg->output, ul_cfg->input, ul_cfg->input_len); + dl_cfg->output_len = ul_cfg->input_len; +} + +/* + * According to ETSI TS 138 323 V17.1.0, Section 5.2.2.1, + * SN could be divided into following ranges, + * relatively to current value of RX_DELIV state: + * +-------------+-------------+-------------+-------------+ + * | -Outside | -Window | +Window | +Outside | + * | (valid) | (Invalid) | (Valid) | (Invalid) | + * +-------------+-------------^-------------+-------------+ + * | + * v + * SN(RX_DELIV) + */ +enum sn_range_type { + SN_RANGE_MINUS_OUTSIDE, + SN_RANGE_MINUS_WINDOW, + SN_RANGE_PLUS_WINDOW, + SN_RANGE_PLUS_OUTSIDE, +}; + +#define PDCP_SET_COUNT(hfn, sn, size) ((hfn << size) | (sn & ((1 << size) - 1))) + +/* + * Take uplink test case as base, modify RX_DELIV in state and SN in input + */ +static int +test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) +{ + uint32_t rx_deliv_hfn, rx_deliv_sn, new_hfn, new_sn; + const int domain = conf->entity.pdcp_xfrm.domain; + struct pdcp_test_conf dl_conf; + int ret, expected_ret; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + if (domain != RTE_SECURITY_PDCP_MODE_CONTROL && domain != RTE_SECURITY_PDCP_MODE_DATA) + return TEST_SKIPPED; + + const uint32_t sn_size = conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + /* Max value of SN that could fit in `sn_size` bits */ + const uint32_t max_sn = (1 << sn_size) - 1; + const uint32_t shift = (max_sn - window_size) / 2; + /* Could be any number up to `shift` value */ + const uint32_t default_sn = RTE_MIN(2u, shift); + + /* Initialize HFN as non zero value, to be able check values before */ + rx_deliv_hfn = 0xa; + + switch (type) { + case SN_RANGE_PLUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + 1; + expected_ret = TEST_SUCCESS; + break; + case SN_RANGE_MINUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn - 1; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_PLUS_OUTSIDE: + /* RCVD_SN >= SN(RX_DELIV) + Window_Size */ + new_hfn = rx_deliv_hfn - 1; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + window_size; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_MINUS_OUTSIDE: + /* RCVD_SN < SN(RX_DELIV) - Window_Size */ + new_hfn = rx_deliv_hfn + 1; + rx_deliv_sn = window_size + default_sn; + new_sn = rx_deliv_sn - window_size - 1; + expected_ret = TEST_SUCCESS; + break; + default: + return TEST_FAILED; + } + + /* Configure Uplink to generate expected, encrypted packet */ + pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.reverse_iv_direction = true; + conf->entity.pdcp_xfrm.hfn = new_hfn; + conf->entity.sn = new_sn; + conf->output_len = 0; + ret = test_attempt_single(conf); + if (ret != TEST_SUCCESS) + return ret; + + /* Flip configuration to downlink */ + uplink_to_downlink_convert(conf, &dl_conf); + + /* Modify the rx_deliv to verify the expected behaviour */ + dl_conf.entity.pdcp_xfrm.hfn = rx_deliv_hfn; + dl_conf.entity.sn = rx_deliv_sn; + ret = test_attempt_single(&dl_conf); + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + return ret; + + TEST_ASSERT_EQUAL(ret, expected_ret, "Unexpected result"); + + return TEST_SUCCESS; +} + +static int +test_sn_plus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_WINDOW, t_conf); +} + +static int +test_sn_minus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_WINDOW, t_conf); +} + +static int +test_sn_plus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_OUTSIDE, t_conf); +} + +static int +test_sn_minus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); +} + +static int +test_combined(struct pdcp_test_conf *ul_conf) +{ + struct pdcp_test_conf dl_conf; + int ret; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + ul_conf->entity.reverse_iv_direction = true; + ul_conf->output_len = 0; + + ret = test_attempt_single(ul_conf); + if (ret != TEST_SUCCESS) + return ret; + + uplink_to_downlink_convert(ul_conf, &dl_conf); + ret = test_attempt_single(&dl_conf); + + return ret; +} + static int run_test_for_one_known_vec(const void *arg) { @@ -677,8 +947,34 @@ run_test_for_one_known_vec(const void *arg) return test_attempt_single(&test_conf); } +static struct unit_test_suite combined_mode_cases = { + .suite_name = "PDCP combined mode", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("combined mode", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_combined), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static struct unit_test_suite hfn_sn_test_cases = { + .suite_name = "PDCP HFN/SN", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("SN plus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_window), + TEST_CASE_NAMED_WITH_DATA("SN minus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_window), + TEST_CASE_NAMED_WITH_DATA("SN plus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_outside), + TEST_CASE_NAMED_WITH_DATA("SN minus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_outside), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ + &combined_mode_cases, + &hfn_sn_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 11/21] doc: add PDCP library guide 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (9 preceding siblings ...) 2023-05-27 8:58 ` [PATCH v5 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 12/21] pdcp: add control PDU handling for status report Anoob Joseph ` (10 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add guide for PDCP library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 254 ++++++++++++++++++ doc/guides/rel_notes/release_23_07.rst | 12 + 5 files changed, 269 insertions(+) create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst diff --git a/MAINTAINERS b/MAINTAINERS index 15dd88af3a..4345ae21b6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1555,6 +1555,7 @@ M: Volodymyr Fialko <vfialko@marvell.com> T: git://dpdk.org/next/dpdk-next-crypto F: lib/pdcp/ F: app/test/test_pdcp* +F: doc/guides/prog_guide/pdcp_lib.rst Packet Framework diff --git a/doc/guides/prog_guide/img/pdcp_functional_overview.svg b/doc/guides/prog_guide/img/pdcp_functional_overview.svg new file mode 100644 index 0000000000..287daafc21 --- /dev/null +++ b/doc/guides/prog_guide/img/pdcp_functional_overview.svg @@ -0,0 +1 @@ +<svg width="1280" height="720" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><clipPath id="clip0"><rect x="0" y="0" width="1280" height="720"/></clipPath></defs><g clip-path="url(#clip0)"><rect x="0" y="0" width="1280" height="720" fill="#FFFFFF"/><rect x="202" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><rect x="640" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><path d="M605.5 73.5001 605.5 590.633" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M380.5 634 803.25 634 803.25 585.907 790.5 585.907 816 557.5 841.5 585.907 828.75 585.907 828.75 659.5 380.5 659.5Z" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5" fill-rule="evenodd"/><rect x="362.5" y="557.5" width="28" height="102" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5"/><rect x="412.5" y="671.5" width="370" height="32" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(492.364 694)">Radio Interface (<tspan font-size="19" x="137.5" y="0">Uu</tspan><tspan font-size="19" x="161.333" y="0">/PC5)</tspan><tspan font-size="19" x="-282.121" y="-653">UE/NG</tspan><tspan font-size="19" x="-222.955" y="-653">-</tspan><tspan font-size="19" x="-216.788" y="-653">RAN/UE A</tspan><tspan font-size="19" x="375.54" y="-653">NG</tspan>-<tspan font-size="19" x="409.706" y="-653">RAN/UE/UE B</tspan><tspan font-size="14" x="0.896851" y="-647">Transmitting </tspan><tspan font-size="14" x="1.31018" y="-631">PDCP entity</tspan><tspan font-size="14" x="167.401" y="-647">Receiving </tspan><tspan font-size="14" x="160.148" y="-631">PDCP entity</tspan></text><path d="M314.5 71.5001 431.364 71.6549" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M331.5 71.5001C331.5 65.9772 349.633 61.5001 372 61.5001 394.368 61.5001 412.5 65.9772 412.5 71.5001 412.5 77.0229 394.368 81.5001 372 81.5001 349.633 81.5001 331.5 77.0229 331.5 71.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M353.5 90.5001 363.5 90.5001 363.5 48.5001 383.5 48.5001 383.5 90.5001 393.5 90.5001 373.5 110.5Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M877.364 82.6549 760.5 82.5001" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M860.5 83.5001C860.5 89.0229 842.368 93.5001 820 93.5001 797.633 93.5001 779.5 89.0229 779.5 83.5001 779.5 77.9772 797.633 73.5001 820 73.5001 842.368 73.5001 860.5 77.9772 860.5 83.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M838.5 64.5 828.5 64.5 828.5 106.5 808.5 106.5 808.5 64.5 798.5 64.5 818.5 44.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><rect x="244.5" y="128.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(302.494 150)">Transmission buffer:<tspan font-size="19" x="-4.67999" y="23">Sequence</tspan><tspan font-size="19" x="84.32" y="23">numberin</tspan>g</text><rect x="244.5" y="199.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(297.314 222)">Header or uplink data<tspan font-size="19" x="34.08" y="23">Compressio</tspan>n</text><rect x="682.5" y="141.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(735.782 164)">Header or uplink data<tspan font-size="19" x="24.2466" y="23">Decompressio</tspan>n</text><rect x="244.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(300.314 505)">Routing / Duplication</text><rect x="244.5" y="437.5" width="285" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(309.734 460)">Add PDCP header</text><rect x="244.5" y="383.5" width="189" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(298.62 406)">Ciphering</text><rect x="244.5" y="333.5" width="189" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(261.953 356)">Integrity protection</text><path d="M472.167 267.5 472.167 409.802 470.833 409.802 470.833 267.5ZM475.5 408.468 471.5 416.468 467.5 408.468Z"/><path d="M332.167 267.5 332.167 319.552 330.833 319.552 330.833 267.5ZM335.5 318.218 331.5 326.218 327.5 318.219Z"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(253.644 291)">Packets associated <tspan font-size="19" x="14.0067" y="23">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 499.312 299)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="-2.75546e-15" y="45">PDCP SDU</tspan></text><rect x="682.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(728.535 505)">Remove PDCP Header</text><rect x="682.5" y="437.5" width="203" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(733.729 460)">Deciphering</text><rect x="682.5" y="389.5" width="203" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(702.159 412)">Integrity Verification</text><rect x="682.5" y="303.5" width="203" height="77" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(712.729 325)">Reception buffer:<tspan font-size="19" x="24.6667" y="23">Reordering</tspan><tspan font-size="19" x="-13.1667" y="45">Duplicate discardin</tspan><tspan font-size="19" x="144.167" y="45">g</tspan><tspan font-size="19" x="-10.0989" y="-84">Packets associated </tspan><tspan font-size="19" x="3.90784" y="-61">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 960.34 294)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="1" y="45">PDCP SDU</tspan></text><path d="M0.666667-8.2074e-07 0.666763 78.6116-0.66657 78.6116-0.666667 8.2074e-07ZM4.00009 77.2782 0.000104987 85.2782-3.9999 77.2782Z" transform="matrix(1 0 0 -1 779.5 296.778)"/><path d="M0.666667-2.88742e-07 0.666769 235.734-0.666565 235.734-0.666667 2.88742e-07ZM4.0001 234.401 0.000104987 242.401-3.9999 234.401Z" transform="matrix(1 0 0 -1 931.5 453.901)"/></g></svg> \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 87333ee84a..6099ff63cd 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -77,4 +77,5 @@ Programmer's Guide lto profile_app asan + pdcp_lib glossary diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst new file mode 100644 index 0000000000..8369c71600 --- /dev/null +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -0,0 +1,254 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2023 Marvell. + +PDCP Protocol Processing Library +================================ + +DPDK provides a library for PDCP protocol processing. The library utilizes +other DPDK libraries such as cryptodev, reorder, etc., to provide the +application with a transparent and high performant PDCP protocol processing +library. + +The library abstracts complete PDCP protocol processing conforming to +``ETSI TS 138 323 V17.1.0 (2022-08)``. +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + +PDCP would involve the following operations, + +1. Transfer of user plane data +2. Transfer of control plane data +3. Header compression +4. Uplink data compression +5. Ciphering and integrity protection + +.. _figure_pdcp_functional_overview: + +.. figure:: img/pdcp_functional_overview.* + + PDCP functional overview new + +PDCP library would abstract the protocol offload features of the cryptodev and +would provide a uniform interface and consistent API usage to work with +cryptodev irrespective of the protocol offload features supported. + +PDCP entity API +--------------- + +PDCP library provides following control path APIs that is used to +configure various PDCP entities, + +1. ``rte_pdcp_entity_establish()`` +2. ``rte_pdcp_entity_suspend()`` +3. ``rte_pdcp_entity_release()`` + +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or +``rte_security_session`` based on the config. The sessions would be created/ +destroyed while corresponding PDCP entity operations are performed. + +When upper layers request a PDCP entity suspend (``rte_pdcp_entity_suspend()``), +it would result in flushing out of all cached packets. Internal state variables +are updated as described in 5.1.4. + +When upper layers request a PDCP entity release (``rte_pdcp_entity_release()``), +it would result in flushing out of all cached packets and releasing of all +memory associated with the entity. It would internally free any crypto/security +sessions created. All procedures mentioned in 5.1.3 would be performed. + +PDCP PDU (Protocol Data Unit) API +--------------------------------- + +PDCP PDUs can be categorized as, + +1. Control PDU +2. Data PDU + +Control PDUs are used for signalling between entities on either end and can be +one of the following, + +1. PDCP status report +2. ROHC feedback +3. EHC feedback + +Control PDUs are not ciphered or authenticated, and so such packets are not +submitted to cryptodev for processing. + +Data PDUs are regular packets submitted by upper layers for transmission to +other end. Such packets would need to be ciphered and authenticated based on +the entity configuration. + +PDCP packet processing API for data PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PDCP processing is split into 2 parts. One before cryptodev processing +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return crypto +operations belonging to multiple entities, ``rte_pdcp_pkt_crypto_group()`` +is added to help grouping crypto operations belonging to same PDCP entity. + +Lib PDCP would allow application to use same API sequence while leveraging +protocol offload features enabled by ``rte_security`` library. Lib PDCP would +internally change the handles registered for ``pre_process`` and +``post_process`` based on features enabled in the entity. + +Lib PDCP would create the required sessions on the device provided in entity to +minimize the application requirements. Also, the crypto_op allocation and free +would also be done internally by lib PDCP to allow the library to create +crypto ops as required for the input packets. For example, when control PDUs are +received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP +is expected to handle it differently. + +Supported features +------------------ + +- 12 bit & 18 bit sequence numbers +- Uplink & downlink traffic +- HFN increment +- IV generation as required per algorithm + +Supported ciphering algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CTR +- SNOW3G-CIPHER +- ZUC-CIPHER + +Supported integrity protection algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- NULL +- AES-CMAC +- SNOW3G-AUTH +- ZUC-AUTH + +Sample API usage +---------------- + +The ``rte_pdcp_entity_conf`` structure is used to pass the configuration +parameters for entity creation. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_entity_conf 8< + :end-before: >8 End of structure rte_pdcp_entity_conf. + +.. code-block:: c + + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; + struct rte_crypto_op *cop[MAX_BURST_SIZE]; + struct rte_pdcp_group grp[MAX_BURST_SIZE]; + struct rte_pdcp_entity *pdcp_entity; + int nb_max_out_mb, ret, nb_grp; + uint16_t nb_ops; + + /* Create PDCP entity */ + pdcp_entity = rte_pdcp_entity_establish(&conf); + + /** + * Allocate buffer for holding mbufs returned during PDCP suspend, + * release & post-process APIs. + */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + MAX_BURST_SIZE; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + /* Handle error */ + } + + while (1) { + /* Receive packet and form mbuf */ + + /** + * Prepare packets for crypto operation. Following operations + * would be done, + * + * Transmitting entity/UL (only data PDUs): + * - Perform compression + * - Assign sequence number + * - Add PDCP header + * - Create & prepare crypto_op + * - Prepare IV for crypto operation (auth_gen, encrypt) + * - Save original PDCP SDU (during PDCP re-establishment, + * unconfirmed PDCP SDUs need to crypto processed again and + * transmitted/re-transmitted) + * + * Receiving entity/DL: + * - Any control PDUs received would be processed and + * appropriate actions taken. If data PDU, continue. + * - Determine sequence number (based on HFN & per packet SN) + * - Prepare crypto_op + * - Prepare IV for crypto operation (decrypt, auth_verify) + */ + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, + nb_rx, &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) + != nb_success) { + /* Retry for enqueue failure packets */ + } + + ... + + nb_ops = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, + MAX_BURST_SIZE); + if (nb_ops == 0) + continue; + + /** + * Received a burst of completed crypto ops from cryptodev. It + * may belong to various entities. Group similar ones together + * for entity specific post-processing. + */ + + /** + * Groups similar entities together. Frees crypto op and based + * on crypto_op status, set mbuf->ol_flags which would be + * checked in rte_pdcp_pkt_post_process(). + */ + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); + + for (i = 0; i != nb_grp; i++) { + + /** + * Post process packets after crypto completion. + * Following operations would be done, + * + * Transmitting entity/UL: + * - Check crypto result + * + * Receiving entity/DL: + * - Check crypto operation status + * - Check for duplication (if yes, drop duplicate) + * - Perform decompression + * - Trim PDCP header + * - Hold packet (SDU) for in-order delivery (return + * completed packets as and when sequence is + * completed) + * - If not in sequence, cache the packet and start + * t-Reordering timer. When timer expires, the + * packets need to delivered to upper layers (not + * treated as error packets). + */ + nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, + grp[i].m, out_mb, + grp[i].cnt, + &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + /* Perform additional operations */ + + /** + * Transmitting entity/UL + * - If duplication is enabled, duplicate PDCP PDUs + * - When lower layers confirm reception of a PDCP PDU, + * it should be communicated to PDCP layer so that + * PDCP can drop the corresponding SDU + */ + } + } diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 75079ca7d6..bb48514637 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -65,6 +65,18 @@ New Features * Added support for SM3 hash operations. * Added support for AES-CCM in cn9k and cn10k drivers. +* **Added PDCP Library.** + + Added an experimental library ``lib_pdcp`` to provide PDCP UL and DL + processing of packets. + + The library supports all PDCP algorithms and leverages lookaside crypto + offloads to cryptodevs for crypto processing. PDCP features such as IV + generation, sequence number handling etc are supported. It is planned to add + more features such as packet caching in future releases. + + See :doc:`../prog_guide/pdcp_lib` for more information. + Removed Items ------------- -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 12/21] pdcp: add control PDU handling for status report 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (10 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 11/21] doc: add PDCP library guide Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph ` (9 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add control PDU handling and implement status report generation. Status report generation works only when RX_DELIV = RX_NEXT. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 9 ++++++ lib/pdcp/meson.build | 2 ++ lib/pdcp/pdcp_cnt.c | 29 ++++++++++++++++++ lib/pdcp/pdcp_cnt.h | 14 +++++++++ lib/pdcp/pdcp_ctrl_pdu.c | 46 +++++++++++++++++++++++++++++ lib/pdcp/pdcp_ctrl_pdu.h | 15 ++++++++++ lib/pdcp/pdcp_entity.h | 15 ++++++++-- lib/pdcp/pdcp_process.c | 13 +++++++++ lib/pdcp/rte_pdcp.c | 47 +++++++++++++++++++++++++++++- lib/pdcp/rte_pdcp.h | 33 +++++++++++++++++++++ lib/pdcp/version.map | 2 ++ 11 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index 8369c71600..dcb424bb1d 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -76,6 +76,15 @@ Data PDUs are regular packets submitted by upper layers for transmission to other end. Such packets would need to be ciphered and authenticated based on the entity configuration. +PDCP packet processing API for control PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Control PDUs are used in PDCP as a communication channel between transmitting +and receiving entities. When upper layer request for operations such +re-establishment, receiving PDCP entity need to prepare a status report and +send it to the other end. The API ``rte_pdcp_control_pdu_create`` allows +application to request the same. + PDCP packet processing API for data PDU ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 08679b743a..75d476bf6d 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -8,7 +8,9 @@ if is_windows endif sources = files( + 'pdcp_cnt.c', 'pdcp_crypto.c', + 'pdcp_ctrl_pdu.c', 'pdcp_process.c', 'rte_pdcp.c', ) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c new file mode 100644 index 0000000000..c9b952184b --- /dev/null +++ b/lib/pdcp/pdcp_cnt.c @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_cnt.h" +#include "pdcp_entity.h" + +int +pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv_dl_part *en_priv_dl; + uint32_t window_sz; + + if (en == NULL || conf == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + return 0; + + en_priv_dl = entity_dl_part_get(en); + window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + RTE_SET_USED(window_sz); + RTE_SET_USED(en_priv_dl); + + return 0; +} diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h new file mode 100644 index 0000000000..bbda478b55 --- /dev/null +++ b/lib/pdcp/pdcp_cnt.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CNT_H +#define PDCP_CNT_H + +#include <rte_common.h> + +#include "pdcp_entity.h" + +int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c new file mode 100644 index 0000000000..feb05fd863 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_byteorder.h> +#include <rte_mbuf.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_ctrl_pdu.h" +#include "pdcp_entity.h" + +static __rte_always_inline void +pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) +{ + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + pdu_hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + pdu_hdr->r = 0; + pdu_hdr->fmc = rte_cpu_to_be_32(rx_deliv); +} + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; + uint32_t rx_deliv; + int pdu_sz; + + if (!en_priv->flags.is_status_report_required) + return -EINVAL; + + pdu_sz = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + + rx_deliv = en_priv->state.rx_deliv; + + /* Zero missing PDUs - status report contains only FMC */ + if (rx_deliv >= en_priv->state.rx_next) { + pdu_hdr = (struct rte_pdcp_up_ctrl_pdu_hdr *)rte_pktmbuf_append(m, pdu_sz); + if (pdu_hdr == NULL) + return -ENOMEM; + pdcp_hdr_fill(pdu_hdr, rx_deliv); + + return 0; + } + + return -ENOTSUP; +} diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h new file mode 100644 index 0000000000..a2424fbd10 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CTRL_PDU_H +#define PDCP_CTRL_PDU_H + +#include <rte_mbuf.h> + +#include "pdcp_entity.h" + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); + +#endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 10a72faae1..28691a504b 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -109,6 +109,13 @@ union cipher_iv_partial { uint64_t u64[2]; }; +struct pdcp_cnt_bitmap { + /** Number of entries that can be stored. */ + uint32_t size; + /** Bitmap of the count values already received.*/ + struct rte_bitmap *bmp; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -136,9 +143,13 @@ struct entity_priv { uint64_t is_ul_entity : 1; /** Is NULL auth. */ uint64_t is_null_auth : 1; + /** Is status report required.*/ + uint64_t is_status_report_required : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; + /** Control PDU pool. */ + struct rte_mempool *ctrl_pdu_pool; /** PDCP header size. */ uint8_t hdr_sz; /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ @@ -148,8 +159,8 @@ struct entity_priv { }; struct entity_priv_dl_part { - /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ - uint8_t dummy; + /** PDCP would need to track the count values that are already received.*/ + struct pdcp_cnt_bitmap bitmap; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 28ac4102da..ed1413db6d 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -1092,6 +1092,19 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) en_priv->flags.is_null_auth = 1; + /** + * flags.is_status_report_required + * + * Indicate whether status report is required. + */ + if (conf->status_report_required) { + /** Status report is required only for DL entities. */ + if (conf->pdcp_xfrm.pkt_dir != RTE_SECURITY_PDCP_DOWNLINK) + return -EINVAL; + + en_priv->flags.is_status_report_required = 1; + } + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 91dab91f73..96ad397667 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -6,7 +6,9 @@ #include <rte_pdcp.h> #include <rte_malloc.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -58,7 +60,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (pdcp_dynfield_register() < 0) return NULL; - if (conf == NULL || conf->cop_pool == NULL) { + if (conf == NULL || conf->cop_pool == NULL || conf->ctrl_pdu_pool == NULL) { rte_errno = EINVAL; return NULL; } @@ -105,6 +107,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) en_priv->state.rx_deliv = count; en_priv->state.tx_next = count; en_priv->cop_pool = conf->cop_pool; + en_priv->ctrl_pdu_pool = conf->ctrl_pdu_pool; /* Setup crypto session */ ret = pdcp_crypto_sess_create(entity, conf); @@ -115,6 +118,10 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + ret = pdcp_cnt_ring_create(entity, conf); + if (ret) + goto crypto_sess_destroy; + return entity; crypto_sess_destroy: @@ -162,3 +169,41 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, return 0; } + +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type) +{ + struct entity_priv *en_priv; + struct rte_mbuf *m; + int ret; + + if (pdcp_entity == NULL) { + rte_errno = EINVAL; + return NULL; + } + + en_priv = entity_priv_get(pdcp_entity); + + m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); + if (m == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + switch (type) { + case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: + ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + break; + default: + ret = -ENOTSUP; + } + + if (ret) { + rte_pktmbuf_free(m); + rte_errno = -ret; + return NULL; + } + + return m; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index b88cad4f64..2a2bae1cd1 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -16,6 +16,7 @@ #include <rte_compat.h> #include <rte_common.h> #include <rte_mempool.h> +#include <rte_pdcp_hdr.h> #include <rte_security.h> #ifdef __cplusplus @@ -79,6 +80,8 @@ struct rte_pdcp_entity_conf { struct rte_mempool *sess_mpool; /** Crypto op pool.*/ struct rte_mempool *cop_pool; + /** Mbuf pool to be used for allocating control PDUs.*/ + struct rte_mempool *ctrl_pdu_pool; /** * SN value to be used. 32 bit count value to be used for the first * packet would be derived based on HFN (`rte_security_pdcp_xform.hfn`) @@ -98,6 +101,16 @@ struct rte_pdcp_entity_conf { * crypto processing. */ bool reverse_iv_direction; + /** + * Status report required (specified in TS 38.331). + * + * If PDCP entity is configured to send a PDCP status report, the upper + * layer application may request a receiving PDCP entity to generate a + * PDCP status report using ``rte_pdcp_control_pdu_create``. In + * addition, PDCP status reports may be generated during operations such + * as entity re-establishment. + */ + bool status_report_required; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -179,6 +192,26 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Create control PDU packet of the `type` specified. The control PDU packet + * would be allocated from *rte_pdcp_entity_conf.ctrl_pdu_pool* by lib PDCP. + * + * @param pdcp_entity + * Pointer to the PDCP entity for which the control PDU need to be generated. + * @param type + * Type of control PDU to be generated. + * @return + * - Control PDU generated, in case of success. + * - NULL in case of failure. rte_errno will be set to error code. + */ +__rte_experimental +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index d564f155e0..97171f902e 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,6 +2,8 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_control_pdu_create; + rte_pdcp_en_from_cop; rte_pdcp_entity_establish; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 13/21] pdcp: implement t-Reordering and packet buffering 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (11 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 12/21] pdcp: add control PDU handling for status report Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 14/21] test/pdcp: add in-order delivery cases Anoob Joseph ` (8 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add in-order delivery of packets in PDCP. Delivery of packets in-order relies on t-Reordering timer. When 'out-of-order delivery' is disabled, PDCP will buffer all received packets that are out of order. The t-Reordering timer determines the time period these packets would be held in the buffer, waiting for any missing packets to arrive. Introduce packet buffering and state variables which indicate status of the timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 3 +- lib/pdcp/pdcp_entity.h | 19 +++++++ lib/pdcp/pdcp_process.c | 117 ++++++++++++++++++++++++++++++---------- lib/pdcp/pdcp_reorder.c | 27 ++++++++++ lib/pdcp/pdcp_reorder.h | 62 +++++++++++++++++++++ lib/pdcp/rte_pdcp.c | 53 ++++++++++++++++-- lib/pdcp/rte_pdcp.h | 6 ++- 7 files changed, 252 insertions(+), 35 deletions(-) create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 75d476bf6d..f4f9246bcb 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -12,9 +12,10 @@ sources = files( 'pdcp_crypto.c', 'pdcp_ctrl_pdu.c', 'pdcp_process.c', + 'pdcp_reorder.c', 'rte_pdcp.c', ) headers = files('rte_pdcp.h') indirect_headers += files('rte_pdcp_group.h') -deps += ['mbuf', 'net', 'cryptodev', 'security'] +deps += ['mbuf', 'net', 'cryptodev', 'security', 'reorder'] diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 28691a504b..34341cdc11 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -11,6 +11,8 @@ #include <rte_pdcp.h> #include <rte_security.h> +#include "pdcp_reorder.h" + struct entity_priv; #define PDCP_HFN_MIN 0 @@ -109,6 +111,17 @@ union cipher_iv_partial { uint64_t u64[2]; }; +enum timer_state { + TIMER_STOP, + TIMER_RUNNING, + TIMER_EXPIRED, +}; + +struct pdcp_t_reordering { + /** Represent timer state */ + enum timer_state state; +}; + struct pdcp_cnt_bitmap { /** Number of entries that can be stored. */ uint32_t size; @@ -145,6 +158,8 @@ struct entity_priv { uint64_t is_null_auth : 1; /** Is status report required.*/ uint64_t is_status_report_required : 1; + /** Is out-of-order delivery enabled */ + uint64_t is_out_of_order_delivery : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -161,6 +176,10 @@ struct entity_priv { struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; + /** t-Reordering handles */ + struct pdcp_t_reordering t_reorder; + /** Reorder packet buffer */ + struct pdcp_reorder reorder; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index ed1413db6d..84a0f3a43f 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -837,25 +837,88 @@ pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool tr } } -static inline bool +static inline int pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, - const uint32_t count) + const uint32_t count, struct rte_mbuf *mb, + struct rte_mbuf *out_mb[], + const bool trim_mac) { struct entity_priv *en_priv = entity_priv_get(entity); + struct pdcp_t_reordering *t_reorder; + struct pdcp_reorder *reorder; + uint16_t processed = 0; - if (count < en_priv->state.rx_deliv) - return false; + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + const uint32_t hdr_trim_sz = en_priv->hdr_sz + en_priv->aad_sz; - /* t-Reordering timer is not supported - SDU will be delivered immediately. - * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not - * been delivered to upper layers - */ - en_priv->state.rx_next = count + 1; + if (count < en_priv->state.rx_deliv) + return -EINVAL; if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; - return true; + pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); + + if (en_priv->flags.is_out_of_order_delivery) { + out_mb[0] = mb; + en_priv->state.rx_deliv = count + 1; + + return 1; + } + + reorder = &dl->reorder; + t_reorder = &dl->t_reorder; + + if (count == en_priv->state.rx_deliv) { + if (reorder->is_active) { + /* + * This insert used only to increment reorder->min_seqn + * To remove it - min_seqn_set() has to work with non-empty buffer + */ + pdcp_reorder_insert(reorder, mb, count); + + /* Get buffered packets */ + struct rte_mbuf **cached_mbufs = &out_mb[processed]; + uint32_t nb_cached = pdcp_reorder_get_sequential(reorder, + cached_mbufs, entity->max_pkt_cache - processed); + + processed += nb_cached; + } else { + out_mb[processed++] = mb; + } + + /* Processed should never exceed the window size */ + en_priv->state.rx_deliv = count + processed; + + } else { + if (!reorder->is_active) + /* Initialize reordering buffer with RX_DELIV */ + pdcp_reorder_start(reorder, en_priv->state.rx_deliv); + /* Buffer the packet */ + pdcp_reorder_insert(reorder, mb, count); + } + + /* Stop & reset current timer if rx_reord is received */ + if (t_reorder->state == TIMER_RUNNING && + en_priv->state.rx_deliv >= en_priv->state.rx_reord) { + t_reorder->state = TIMER_STOP; + /* Stop reorder buffer, only if it's empty */ + if (en_priv->state.rx_deliv == en_priv->state.rx_next) + pdcp_reorder_stop(reorder); + } + + /* + * If t-Reordering is not running (includes the case when t-Reordering is stopped due to + * actions above). + */ + if (t_reorder->state == TIMER_STOP && en_priv->state.rx_deliv < en_priv->state.rx_next) { + /* Update RX_REORD to RX_NEXT */ + en_priv->state.rx_reord = en_priv->state.rx_next; + /* Start t-Reordering */ + t_reorder->state = TIMER_RUNNING; + } + + return processed; } static inline uint16_t @@ -863,16 +926,12 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret, const bool is_integ_protected) { - struct entity_priv *en_priv = entity_priv_get(entity); - const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; + int i, nb_processed, nb_success = 0, nb_err = 0; rte_pdcp_dynfield_t *mb_dynfield; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -881,11 +940,12 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r mb_dynfield = pdcp_dynfield(mb); count = *mb_dynfield; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], is_integ_protected); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -919,16 +979,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret) { - struct entity_priv *en_priv = entity_priv_get(entity); - const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; + int i, nb_processed, nb_success = 0, nb_err = 0; rte_pdcp_dynfield_t *mb_dynfield; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -937,12 +993,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, mb_dynfield = pdcp_dynfield(mb); count = *mb_dynfield; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], true); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, true); - - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -1105,6 +1161,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent en_priv->flags.is_status_report_required = 1; } + /** + * flags.is_out_of_order_delivery + * + * Indicate whether the outoforder delivery is enabled for PDCP entity. + */ + en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** * hdr_sz * diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c new file mode 100644 index 0000000000..5399f0dc28 --- /dev/null +++ b/lib/pdcp/pdcp_reorder.c @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_reorder.h> + +#include "pdcp_reorder.h" + +int +pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +{ + reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + if (reorder->buf == NULL) + return -rte_errno; + + reorder->window_size = window_size; + reorder->is_active = false; + + return 0; +} + +void +pdcp_reorder_destroy(const struct pdcp_reorder *reorder) +{ + rte_reorder_free(reorder->buf); +} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h new file mode 100644 index 0000000000..6a2f61d6ae --- /dev/null +++ b/lib/pdcp/pdcp_reorder.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_REORDER_H +#define PDCP_REORDER_H + +#include <rte_reorder.h> + +struct pdcp_reorder { + struct rte_reorder_buffer *buf; + uint32_t window_size; + bool is_active; +}; + +int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); +void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); + +static inline uint32_t +pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs) +{ + return rte_reorder_drain(reorder->buf, mbufs, max_mbufs); +} + +static inline uint32_t +pdcp_reorder_up_to_get(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs, uint32_t seqn) +{ + return rte_reorder_drain_up_to_seqn(reorder->buf, mbufs, max_mbufs, seqn); +} + +static inline void +pdcp_reorder_start(struct pdcp_reorder *reorder, uint32_t min_seqn) +{ + int ret; + + reorder->is_active = true; + + ret = rte_reorder_min_seqn_set(reorder->buf, min_seqn); + RTE_VERIFY(ret == 0); +} + +static inline void +pdcp_reorder_stop(struct pdcp_reorder *reorder) +{ + reorder->is_active = false; +} + +static inline void +pdcp_reorder_insert(struct pdcp_reorder *reorder, struct rte_mbuf *mbuf, + rte_reorder_seqn_t pkt_count) +{ + int ret; + + *rte_reorder_seqn(mbuf) = pkt_count; + + ret = rte_reorder_insert(reorder->buf, mbuf); + RTE_VERIFY(ret == 0); +} + +#endif /* PDCP_REORDER_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 96ad397667..be37ff392c 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -49,6 +49,17 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } +static int +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + + entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + + return pdcp_reorder_create(&dl->reorder, window_size); +} + struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { @@ -118,6 +129,12 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + ret = pdcp_dl_establish(entity, conf); + if (ret) + goto crypto_sess_destroy; + } + ret = pdcp_cnt_ring_create(entity, conf); if (ret) goto crypto_sess_destroy; @@ -132,26 +149,50 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } +static int +pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + int nb_out; + + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, + en_priv->state.rx_next); + + pdcp_reorder_destroy(&dl->reorder); + + return nb_out; +} + int rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv *en_priv; + int nb_out = 0; + if (pdcp_entity == NULL) return -EINVAL; + en_priv = entity_priv_get(pdcp_entity); + + if (!en_priv->flags.is_ul_entity) + nb_out = pdcp_dl_release(pdcp_entity, out_mb); + /* Teardown crypto sessions */ pdcp_crypto_sess_destroy(pdcp_entity); rte_free(pdcp_entity); - RTE_SET_USED(out_mb); - return 0; + return nb_out; } int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; + int nb_out = 0; if (pdcp_entity == NULL) return -EINVAL; @@ -161,13 +202,15 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, if (en_priv->flags.is_ul_entity) { en_priv->state.tx_next = 0; } else { + dl = entity_dl_part_get(pdcp_entity); + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, pdcp_entity->max_pkt_cache, + en_priv->state.rx_next); + pdcp_reorder_stop(&dl->reorder); en_priv->state.rx_next = 0; en_priv->state.rx_deliv = 0; } - RTE_SET_USED(out_mb); - - return 0; + return nb_out; } struct rte_mbuf * diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 2a2bae1cd1..980086a93a 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -111,6 +111,8 @@ struct rte_pdcp_entity_conf { * as entity re-establishment. */ bool status_report_required; + /** Enable out of order delivery. */ + bool out_of_order_delivery; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -271,8 +273,8 @@ rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, * @param in_mb * The address of an array of *num* pointers to *rte_mbuf* structures. * @param[out] out_mb - * The address of an array of *num* pointers to *rte_mbuf* structures - * to output packets after PDCP post-processing. + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures to output packets after PDCP post-processing. * @param num * The maximum number of packets to process. * @param[out] nb_err -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 14/21] test/pdcp: add in-order delivery cases 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (12 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 15/21] pdcp: add timer callback handlers Anoob Joseph ` (7 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test cases to verify behaviour when in-order delivery is enabled and packets arrive in out-of-order. PDCP library is expected to buffer the packets and return packets in-order when the missing packet arrives. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 223 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index cfe2ec6aa9..24d7826bc2 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,15 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* Assert that condition is true, or goto the mark */ +#define ASSERT_TRUE_OR_GOTO(cond, mark, ...) do {\ + if (!(cond)) { \ + RTE_LOG(ERR, USER1, "Error at: %s:%d\n", __func__, __LINE__); \ + RTE_LOG(ERR, USER1, __VA_ARGS__); \ + goto mark; \ + } \ +} while (0) + /* According to formula(7.2.a Window_Size) */ #define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) @@ -83,6 +92,38 @@ run_test_with_all_known_vec(const void *args) return run_test_foreach_known_vec(test, false); } +static int +run_test_with_all_known_vec_until_first_pass(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, true); +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -416,6 +457,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.sess_mpool = ts_params->sess_pool; conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.ctrl_pdu_pool = ts_params->mbuf_pool; conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; @@ -868,6 +910,7 @@ test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) /* Configure Uplink to generate expected, encrypted packet */ pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.out_of_order_delivery = true; conf->entity.reverse_iv_direction = true; conf->entity.pdcp_xfrm.hfn = new_hfn; conf->entity.sn = new_sn; @@ -915,6 +958,171 @@ test_sn_minus_outside(struct pdcp_test_conf *t_conf) return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); } +static struct rte_mbuf * +generate_packet_for_dl_with_sn(struct pdcp_test_conf ul_conf, uint32_t count) +{ + enum rte_security_pdcp_sn_size sn_size = ul_conf.entity.pdcp_xfrm.sn_size; + int ret; + + ul_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(count, sn_size); + ul_conf.entity.sn = pdcp_sn_from_count_get(count, sn_size); + ul_conf.entity.out_of_order_delivery = true; + ul_conf.entity.reverse_iv_direction = true; + ul_conf.output_len = 0; + + ret = test_attempt_single(&ul_conf); + if (ret != TEST_SUCCESS) + return NULL; + + return mbuf_from_data_create(ul_conf.output, ul_conf.output_len); +} + +static bool +array_asc_sorted_check(struct rte_mbuf *m[], uint32_t len, enum rte_security_pdcp_sn_size sn_size) +{ + uint32_t i; + + if (len < 2) + return true; + + for (i = 0; i < (len - 1); i++) { + if (pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i], void *), sn_size) > + pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i + 1], void *), sn_size)) + return false; + } + + return true; +} + +static int +test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m0 = NULL, *m1 = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Generate packet to fill the existing gap */ + m0 = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m0 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1] + * Gap filled, all packets should be returned + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m0, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m0 = NULL; /* Packet was moved to out_mb */ + + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, + "Error occurred during packet drain\n"); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m0); + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + +static int +test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + struct rte_mbuf *m1 = NULL, **out_mb = NULL; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + const int rx_deliv = 0; + int ret = TEST_FAILED; + size_t i, nb_out; + uint8_t cdev_id; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + ASSERT_TRUE_OR_GOTO(pdcp_entity->max_pkt_cache >= window_size, exit, + "PDCP max packet cache is too small"); + cdev_id = dl_conf.entity.dev_id; + out_mb = rte_zmalloc(NULL, pdcp_entity->max_pkt_cache * sizeof(uintptr_t), 0); + ASSERT_TRUE_OR_GOTO(out_mb != NULL, exit, + "Could not allocate buffer for holding out_mb buffers\n"); + + /* Send packets with SN > RX_DELIV to create a gap */ + for (i = rx_deliv + 1; i < window_size; i++) { + m1 = generate_packet_for_dl_with_sn(*ul_conf, i); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + } + + m1 = generate_packet_for_dl_with_sn(*ul_conf, rx_deliv); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Insert missing packet */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == window_size, exit, + "Packet count mismatch (received: %i, expected: %i)\n", + nb_success, window_size); + m1 = NULL; + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + rte_free(out_mb); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -971,10 +1179,25 @@ static struct unit_test_suite hfn_sn_test_cases = { } }; +static struct unit_test_suite reorder_test_cases = { + .suite_name = "PDCP reorder", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_reorder_buffer_full_window_size_sn_12), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, + &reorder_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 15/21] pdcp: add timer callback handlers 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (13 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 14/21] test/pdcp: add in-order delivery cases Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 16/21] pdcp: add timer expiry handle Anoob Joseph ` (6 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> PDCP has a windowing mechanism which allows only packets that fall in a reception window. The pivot point for this window is RX_REORD which happens to be the first missing or next expected packet. If the missing packet is not received after a specified time, then the RX_REORD state variable needs to be moved up to slide the reception window. PDCP relies on timers for such operations. The timer needs to be armed when PDCP library doesn't receive all packets in-order and starts buffering packets that arrived after a missing packet. The timer needs to be cancelled when a missing packet is received. To avoid dependency on particular timer implementation, PDCP library allows application to register two callbacks, timer_start() and timer_stop() that will be called later by library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 ++ lib/pdcp/pdcp_process.c | 2 ++ lib/pdcp/rte_pdcp.c | 1 + lib/pdcp/rte_pdcp.h | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 52 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 34341cdc11..efc74ba9b9 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -120,6 +120,8 @@ enum timer_state { struct pdcp_t_reordering { /** Represent timer state */ enum timer_state state; + /** User defined callback handles */ + struct rte_pdcp_t_reordering handle; }; struct pdcp_cnt_bitmap { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 84a0f3a43f..daf2c27363 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -902,6 +902,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (t_reorder->state == TIMER_RUNNING && en_priv->state.rx_deliv >= en_priv->state.rx_reord) { t_reorder->state = TIMER_STOP; + t_reorder->handle.stop(t_reorder->handle.timer, t_reorder->handle.args); /* Stop reorder buffer, only if it's empty */ if (en_priv->state.rx_deliv == en_priv->state.rx_next) pdcp_reorder_stop(reorder); @@ -916,6 +917,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, en_priv->state.rx_reord = en_priv->state.rx_next; /* Start t-Reordering */ t_reorder->state = TIMER_RUNNING; + t_reorder->handle.start(t_reorder->handle.timer, t_reorder->handle.args); } return processed; diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index be37ff392c..a0558b99ae 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -56,6 +56,7 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct entity_priv_dl_part *dl = entity_dl_part_get(entity); entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + dl->t_reorder.handle = conf->t_reordering; return pdcp_reorder_create(&dl->reorder, window_size); } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 980086a93a..05c922819e 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -67,6 +67,51 @@ struct rte_pdcp_entity { uint32_t max_pkt_cache; } __rte_cache_aligned; +/** + * Callback function type for t-Reordering timer start, set during PDCP entity establish. + * This callback is invoked by PDCP library, during t-Reordering timer start event. + * Only one t-Reordering per receiving PDCP entity would be running at a given time. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_start_cb_t)(void *timer, void *args); + +/** + * Callback function type for t-Reordering timer stop, set during PDCP entity establish. + * This callback will be invoked by PDCP library, during t-Reordering timer stop event. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); + +/** + * PDCP t-Reordering timer interface + * + * Configuration provided by user, that PDCP library will invoke according to timer behaviour. + */ +struct rte_pdcp_t_reordering { + /** Timer pointer, stored for later use in callback functions */ + void *timer; + /** Timer arguments, stored for later use in callback functions */ + void *args; + /** Timer start callback handle */ + rte_pdcp_t_reordering_start_cb_t start; + /** Timer stop callback handle */ + rte_pdcp_t_reordering_stop_cb_t stop; +}; + /** * PDCP entity configuration to be used for establishing an entity. */ @@ -113,6 +158,8 @@ struct rte_pdcp_entity_conf { bool status_report_required; /** Enable out of order delivery. */ bool out_of_order_delivery; + /** t-Reordering timer configuration */ + struct rte_pdcp_t_reordering t_reordering; }; /* >8 End of structure rte_pdcp_entity_conf. */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 16/21] pdcp: add timer expiry handle 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (14 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 15/21] pdcp: add timer callback handlers Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 17/21] test/pdcp: add timer expiry cases Anoob Joseph ` (5 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> The PDCP protocol requires usage of timers to keep track of how long an out-of-order packet should be buffered while waiting for missing packets. Applications can register a desired timer implementation with the PDCP library. Once the timer expires, the application will be notified, and further handling of the event will be performed in the PDCP library. When the timer expires, the PDCP library will return the cached packets, and PDCP internal state variables (like RX_REORD, RX_DELIV etc) will be updated accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 30 ++++++++++++++++++ lib/pdcp/rte_pdcp.c | 49 ++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 31 +++++++++++++++++++ lib/pdcp/version.map | 2 ++ 4 files changed, 112 insertions(+) diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index dcb424bb1d..16deaead15 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -130,6 +130,36 @@ Supported integrity protection algorithms - SNOW3G-AUTH - ZUC-AUTH +Timers +------ + +PDCP utilizes a reception window mechanism to limit the bits of COUNT value +transmitted in the packet. It utilizes state variables such as RX_REORD, +RX_DELIV to define the window and uses RX_DELIV as the lower pivot point of the +window. + +RX_DELIV would be updated only when packets are received in-order. Any missing +packet would mean RX_DELIV won't be updated. A timer, t-Reordering, helps PDCP +to slide the window if the missing packet is not received in a specified time +duration. + +While starting and stopping the timer need to be done by lib PDCP, application +could register its own timer implementation. This is to make sure application +can choose between timers such as rte_timer and rte_event based timers. Starting +and stopping of timer would happen during pre & post process API. + +When the t-Reordering timer expires, application would receive the expiry event. +To perform the PDCP handling of the expiry event, +``rte_pdcp_t_reordering_expiry_handle`` can be used. Expiry handling would +involve sliding the window by updating state variables and passing the expired +packets to the application. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_t_reordering 8< + :end-before: >8 End of structure rte_pdcp_t_reordering. + + Sample API usage ---------------- diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index a0558b99ae..819c66bd08 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -251,3 +251,52 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, return m; } + +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t capacity = entity->max_pkt_cache; + uint16_t nb_out, nb_seq; + + /* 5.2.2.2 Actions when a t-Reordering expires */ + + /* + * - deliver to upper layers in ascending order of the associated COUNT value after + * performing header decompression, if not decompressed before: + */ + + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); + capacity -= nb_out; + out_mb = &out_mb[nb_out]; + + /* + * - all stored PDCP SDU(s) with consecutively associated COUNT value(s) starting from + * RX_REORD; + */ + nb_seq = pdcp_reorder_get_sequential(&dl->reorder, out_mb, capacity); + nb_out += nb_seq; + + /* + * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered + * to upper layers, with COUNT value >= RX_REORD; + */ + en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + + /* + * - if RX_DELIV < RX_NEXT: + * - update RX_REORD to RX_NEXT; + * - start t-Reordering. + */ + if (en_priv->state.rx_deliv < en_priv->state.rx_next) { + en_priv->state.rx_reord = en_priv->state.rx_next; + dl->t_reorder.state = TIMER_RUNNING; + dl->t_reorder.handle.start(dl->t_reorder.handle.timer, dl->t_reorder.handle.args); + } else { + dl->t_reorder.state = TIMER_EXPIRED; + } + + return nb_out; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 05c922819e..b926b0df29 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -101,6 +101,7 @@ typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); * * Configuration provided by user, that PDCP library will invoke according to timer behaviour. */ +/* Structure rte_pdcp_t_reordering 8< */ struct rte_pdcp_t_reordering { /** Timer pointer, stored for later use in callback functions */ void *timer; @@ -111,6 +112,7 @@ struct rte_pdcp_t_reordering { /** Timer stop callback handle */ rte_pdcp_t_reordering_stop_cb_t stop; }; +/* >8 End of structure rte_pdcp_t_reordering. */ /** * PDCP entity configuration to be used for establishing an entity. @@ -339,6 +341,35 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.2.2.2 Actions when a t-Reordering expires + * + * When t-Reordering timer expires, PDCP is required to slide the reception + * window by updating state variables such as RX_REORD & RX_DELIV. PDCP would + * need to deliver some of the buffered packets based on the state variables and + * conditions described. + * + * The expiry handle need to be invoked by the application when t-Reordering + * timer expires. In addition to returning buffered packets, it may also restart + * timer based on the state variables. + * + * @param entity + * Pointer to the *rte_pdcp_entity* for which the timer expired. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. Used to return buffered packets that are + * expired. + * @return + * Number of packets returned in *out_mb* buffer. + */ +__rte_experimental +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, + struct rte_mbuf *out_mb[]); + /** * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. So include * in the end. diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 97171f902e..4fd912fac0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -14,5 +14,7 @@ EXPERIMENTAL { rte_pdcp_pkt_pre_process; rte_pdcp_pkt_crypto_group; + rte_pdcp_t_reordering_expiry_handle; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 17/21] test/pdcp: add timer expiry cases 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (15 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 16/21] pdcp: add timer expiry handle Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 18/21] test/pdcp: add timer restart case Anoob Joseph ` (4 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test cases for handling the expiry with rte_timer and rte_event_timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 350 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 350 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 24d7826bc2..25729b2bdd 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -3,15 +3,24 @@ */ #include <rte_errno.h> +#ifdef RTE_LIB_EVENTDEV +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#endif /* RTE_LIB_EVENTDEV */ #include <rte_malloc.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include <rte_timer.h> #include "test.h" #include "test_cryptodev.h" #include "test_cryptodev_security_pdcp_test_vectors.h" +#define NSECPERSEC 1E9 #define NB_DESC 1024 +#define TIMER_ADAPTER_ID 0 +#define TEST_EV_QUEUE_ID 0 +#define TEST_EV_PORT_ID 0 #define CDEV_INVALID_ID UINT8_MAX #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 @@ -33,10 +42,21 @@ struct pdcp_testsuite_params { struct rte_mempool *cop_pool; struct rte_mempool *sess_pool; bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; + int evdev; +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev; +#endif /* RTE_LIB_EVENTDEV */ + bool timer_is_running; + uint64_t min_resolution_ns; }; static struct pdcp_testsuite_params testsuite_params; +struct test_rte_timer_args { + int status; + struct rte_pdcp_entity *pdcp_entity; +}; + struct pdcp_test_conf { struct rte_pdcp_entity_conf entity; struct rte_crypto_sym_xform c_xfrm; @@ -124,6 +144,30 @@ pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; } +static void +pdcp_timer_start_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = true; +} + +static void +pdcp_timer_stop_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = false; +} + +static struct rte_pdcp_t_reordering t_reorder_timer = { + .timer = &testsuite_params.timer_is_running, + .start = pdcp_timer_start_cb, + .stop = pdcp_timer_stop_cb, +}; + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -462,6 +506,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + conf->entity.t_reordering = t_reorder_timer; if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; @@ -1048,6 +1093,8 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) /* Check that packets in correct order */ ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == false, exit, + "Timer should be stopped after full drain\n"); ret = TEST_SUCCESS; exit: @@ -1123,6 +1170,181 @@ test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static void +event_timer_start_cb(void *timer, void *args) +{ + struct rte_event_timer *evtims = args; + int ret = 0; + + ret = rte_event_timer_arm_burst(timer, &evtims, 1); + assert(ret == 1); +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +test_expiry_with_event_timer(const struct pdcp_test_conf *ul_conf) +{ +#ifdef RTE_LIB_EVENTDEV + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_event event; + + const int start_count = 0; + struct rte_event_timer evtim = { + .ev.op = RTE_EVENT_OP_NEW, + .ev.queue_id = TEST_EV_QUEUE_ID, + .ev.sched_type = RTE_SCHED_TYPE_ATOMIC, + .ev.priority = RTE_EVENT_DEV_PRIORITY_NORMAL, + .ev.event_type = RTE_EVENT_TYPE_TIMER, + .state = RTE_EVENT_TIMER_NOT_ARMED, + .timeout_ticks = 1, + }; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &evtim; + dl_conf.entity.t_reordering.timer = testsuite_params.timdev; + dl_conf.entity.t_reordering.start = event_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + evtim.ev.event_ptr = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + while (n != 1) { + rte_delay_us(testsuite_params.min_resolution_ns / 1000); + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, + "Dequeued unexpected timer expiry event: %i\n", n); + } + + ASSERT_TRUE_OR_GOTO(event.event_type == RTE_EVENT_TYPE_TIMER, exit, "Unexpected event type\n"); + + /* Handle expiry event */ + n = rte_pdcp_t_reordering_expiry_handle(event.event_ptr, out_mb); + ASSERT_TRUE_OR_GOTO(n == 1, exit, "Unexpected number of expired packets :%i\n", n); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +#else + RTE_SET_USED(ul_conf); + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +test_rte_timer_expiry_handle(struct rte_timer *timer_handle, void *arg) +{ + struct test_rte_timer_args *timer_data = arg; + struct rte_mbuf *out_mb[1] = {0}; + uint16_t n; + + RTE_SET_USED(timer_handle); + + n = rte_pdcp_t_reordering_expiry_handle(timer_data->pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, n); + + timer_data->status = n == 1 ? n : -1; +} + +static void +test_rte_timer_start_cb(void *timer, void *args) +{ + rte_timer_reset_sync(timer, 1, SINGLE, rte_lcore_id(), test_rte_timer_expiry_handle, args); +} + +static int +test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct test_rte_timer_args timer_args; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_timer timer = {0}; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Set up a timer */ + rte_timer_init(&timer); + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &timer_args; + dl_conf.entity.t_reordering.timer = &timer; + dl_conf.entity.t_reordering.start = test_rte_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + timer_args.status = 0; + timer_args.pdcp_entity = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Verify that expire was handled correctly */ + rte_timer_manage(); + while (timer_args.status != 1) { + rte_delay_us(1); + rte_timer_manage(); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, "Bad expire handle status %i\n", + timer_args.status); + } + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1145,6 +1367,126 @@ test_combined(struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static inline void +eventdev_conf_default_set(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info) +{ + memset(dev_conf, 0, sizeof(struct rte_event_dev_config)); + dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns; + dev_conf->nb_event_ports = 1; + dev_conf->nb_event_queues = 1; + dev_conf->nb_event_queue_flows = info->max_event_queue_flows; + dev_conf->nb_event_port_dequeue_depth = info->max_event_port_dequeue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_events_limit = info->max_num_events; +} + +static inline int +eventdev_setup(void) +{ + struct rte_event_dev_config dev_conf; + struct rte_event_dev_info info; + int ret, evdev = 0; + + if (!rte_event_dev_count()) + return TEST_SKIPPED; + + ret = rte_event_dev_info_get(evdev, &info); + TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info"); + TEST_ASSERT(info.max_num_events < 0 || info.max_num_events >= 1, + "ERROR max_num_events=%d < max_events=%d", info.max_num_events, 1); + + eventdev_conf_default_set(&dev_conf, &info); + ret = rte_event_dev_configure(evdev, &dev_conf); + TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev"); + + ret = rte_event_queue_setup(evdev, TEST_EV_QUEUE_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d", TEST_EV_QUEUE_ID); + + /* Configure event port */ + ret = rte_event_port_setup(evdev, TEST_EV_PORT_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d", TEST_EV_PORT_ID); + ret = rte_event_port_link(evdev, TEST_EV_PORT_ID, NULL, NULL, 0); + TEST_ASSERT(ret >= 0, "Failed to link all queues port=%d", TEST_EV_PORT_ID); + + ret = rte_event_dev_start(evdev); + TEST_ASSERT_SUCCESS(ret, "Failed to start device"); + + testsuite_params.evdev = evdev; + + return TEST_SUCCESS; +} + +static int +event_timer_setup(void) +{ + struct rte_event_timer_adapter_info info; + struct rte_event_timer_adapter *timdev; + uint32_t caps = 0; + + struct rte_event_timer_adapter_conf config = { + .event_dev_id = testsuite_params.evdev, + .timer_adapter_id = TIMER_ADAPTER_ID, + .timer_tick_ns = NSECPERSEC, + .max_tmo_ns = 10 * NSECPERSEC, + .nb_timers = 10, + .flags = 0, + }; + + TEST_ASSERT_SUCCESS(rte_event_timer_adapter_caps_get(testsuite_params.evdev, &caps), + "Failed to get adapter capabilities"); + + if (!(caps & RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) + return TEST_SKIPPED; + + timdev = rte_event_timer_adapter_create(&config); + + TEST_ASSERT_NOT_NULL(timdev, "Failed to create event timer ring"); + + testsuite_params.timdev = timdev; + + TEST_ASSERT_EQUAL(rte_event_timer_adapter_start(timdev), 0, + "Failed to start event timer adapter"); + + rte_event_timer_adapter_get_info(timdev, &info); + testsuite_params.min_resolution_ns = info.min_resolution_ns; + + return TEST_SUCCESS; +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +ut_setup_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + int ret; + + ret = eventdev_setup(); + if (ret) + return ret; + + return event_timer_setup(); +#else + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +ut_teardown_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev = testsuite_params.timdev; + int evdev = testsuite_params.evdev; + + rte_event_dev_stop(evdev); + rte_event_dev_close(evdev); + + rte_event_timer_adapter_stop(timdev); + rte_event_timer_adapter_free(timdev); +#endif /* RTE_LIB_EVENTDEV */ +} + static int run_test_for_one_known_vec(const void *arg) { @@ -1189,6 +1531,14 @@ static struct unit_test_suite reorder_test_cases = { ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, test_reorder_buffer_full_window_size_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_event_timer", + ut_setup_pdcp_event_timer, ut_teardown_pdcp_event_timer, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_event_timer), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_rte_timer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_rte_timer), TEST_CASES_END() /**< NULL terminate unit test array */ } }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 18/21] test/pdcp: add timer restart case 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (16 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 17/21] test/pdcp: add timer expiry cases Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 19/21] pdcp: add support for status report Anoob Joseph ` (3 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test to cover the case when t-reordering timer should be restarted on the same packet. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 68 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 25729b2bdd..82cc25ec7a 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -1106,6 +1106,71 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) return ret; } +static int +test_reorder_gap_in_reorder_buffer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + int ret = TEST_FAILED, nb_out, i; + struct pdcp_test_conf dl_conf; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Create two gaps [NULL, m1, NULL, m3]*/ + for (i = 0; i < 2; i++) { + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + 2 * i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m = NULL; /* Packet was moved to PDCP lib */ + } + + /* Generate packet to fill the first gap */ + m = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1, NULL, m3] + * Only first gap should be filled, timer should be restarted for second gap + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m = NULL; + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), + exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == true, exit, + "Timer should be restarted after partial drain"); + + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) { @@ -1527,6 +1592,9 @@ static struct unit_test_suite reorder_test_cases = { TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_in_reorder_buffer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_in_reorder_buffer), TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 19/21] pdcp: add support for status report 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (17 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 18/21] test/pdcp: add timer restart case Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph ` (2 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Implement status report generation for PDCP entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 158 ++++++++++++++++++++++++++++++++++++--- lib/pdcp/pdcp_cnt.h | 11 ++- lib/pdcp/pdcp_ctrl_pdu.c | 34 ++++++++- lib/pdcp/pdcp_ctrl_pdu.h | 3 +- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 9 ++- lib/pdcp/pdcp_process.h | 13 ++++ lib/pdcp/rte_pdcp.c | 34 ++++++--- 8 files changed, 236 insertions(+), 28 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index c9b952184b..af027b00d3 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -2,28 +2,164 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_pdcp.h> #include "pdcp_cnt.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#define SLAB_BYTE_SIZE (RTE_BITMAP_SLAB_BIT_SIZE / 8) + +uint32_t +pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) +{ + uint32_t n_bits = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + return rte_bitmap_get_memory_footprint(n_bits); +} + int -pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) { - struct entity_priv_dl_part *en_priv_dl; - uint32_t window_sz; + uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - if (en == NULL || conf == NULL) + dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + if (dl->bitmap.bmp == NULL) return -EINVAL; - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) - return 0; + dl->bitmap.size = window_size; - en_priv_dl = entity_dl_part_get(en); - window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + return 0; +} - RTE_SET_USED(window_sz); - RTE_SET_USED(en_priv_dl); +void +pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + rte_bitmap_set(bitmap.bmp, count % bitmap.size); +} - return 0; +bool +pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + return rte_bitmap_get(bitmap.bmp, count % bitmap.size); +} + +void +pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop) +{ + uint32_t i; + + for (i = start; i < stop; i++) + rte_bitmap_clear(bitmap.bmp, i % bitmap.size); +} + +uint16_t +pdcp_cnt_get_bitmap_size(uint32_t pending_bytes) +{ + /* + * Round up bitmap size to slab size to operate only on slabs sizes, instead of individual + * bytes + */ + return RTE_ALIGN_MUL_CEIL(pending_bytes, SLAB_BYTE_SIZE); +} + +static __rte_always_inline uint64_t +leftover_get(uint64_t slab, uint32_t shift, uint64_t mask) +{ + return (slab & mask) << shift; +} + +void +pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len) +{ + uint64_t slab = 0, next_slab = 0, leftover; + uint32_t zeros, report_len, diff; + uint32_t slab_id, next_slab_id; + uint32_t pos = 0, next_pos = 0; + + const uint32_t start_count = state.rx_deliv + 1; + const uint32_t nb_slabs = bitmap.size / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t nb_data_slabs = data_len / SLAB_BYTE_SIZE; + const uint32_t start_slab_id = start_count / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t stop_slab_id = (start_slab_id + nb_data_slabs) % nb_slabs; + const uint32_t shift = start_count % RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t leftover_shift = shift ? RTE_BITMAP_SLAB_BIT_SIZE - shift : 0; + const uint8_t *data_end = RTE_PTR_ADD(data, data_len + SLAB_BYTE_SIZE); + + /* NOTE: Mask required to workaround case - when shift is not needed */ + const uint64_t leftover_mask = shift ? ~0 : 0; + + /* NOTE: implement scan init at to set custom position */ + __rte_bitmap_scan_init(bitmap.bmp); + while (true) { + assert(rte_bitmap_scan(bitmap.bmp, &pos, &slab) == 1); + slab_id = pos / RTE_BITMAP_SLAB_BIT_SIZE; + if (slab_id >= start_slab_id) + break; + } + + report_len = nb_data_slabs; + + if (slab_id > start_slab_id) { + /* Zero slabs at beginning */ + zeros = (slab_id - start_slab_id - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + leftover = leftover_get(slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + report_len -= (slab_id - start_slab_id); + } + + while (report_len) { + rte_bitmap_scan(bitmap.bmp, &next_pos, &next_slab); + next_slab_id = next_pos / RTE_BITMAP_SLAB_BIT_SIZE; + diff = (next_slab_id + nb_slabs - slab_id) % nb_slabs; + + /* If next_slab_id == slab_id - overlap */ + diff += !(next_slab_id ^ slab_id) * nb_slabs; + + /* Size check - next slab is outsize of size range */ + if (diff > report_len) { + next_slab = 0; + next_slab_id = stop_slab_id; + diff = report_len; + } + + report_len -= diff; + + /* Calculate gap between slabs, taking wrap around into account */ + zeros = (next_slab_id + nb_slabs - slab_id - 1) % nb_slabs; + if (zeros) { + /* Non continues slabs, align them individually */ + slab >>= shift; + memcpy(data, &slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + + /* Fill zeros between slabs */ + zeros = (zeros - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + + /* Align beginning of next slab */ + leftover = leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } else { + /* Continues slabs, combine them */ + uint64_t new_slab = (slab >> shift) | + leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &new_slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } + + slab = next_slab; + pos = next_pos; + slab_id = next_slab_id; + + }; + + assert(data < data_end); } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index bbda478b55..5941b7a406 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -9,6 +9,15 @@ #include "pdcp_entity.h" -int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); +uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); + +void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +void pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop); + +uint16_t pdcp_cnt_get_bitmap_size(uint32_t pending_bytes); +void pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len); #endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c index feb05fd863..e0ac2d3720 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.c +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -8,6 +8,14 @@ #include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#include "pdcp_cnt.h" + +static inline uint16_t +round_up_bits(uint32_t bits) +{ + /* round up to the next multiple of 8 */ + return RTE_ALIGN_MUL_CEIL(bits, 8) / 8; +} static __rte_always_inline void pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) @@ -19,11 +27,13 @@ pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) } int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m) { struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; - uint32_t rx_deliv; - int pdu_sz; + uint32_t rx_deliv, actual_sz; + uint16_t pdu_sz, bitmap_sz; + uint8_t *data; if (!en_priv->flags.is_status_report_required) return -EINVAL; @@ -42,5 +52,21 @@ pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) return 0; } - return -ENOTSUP; + actual_sz = RTE_MIN(round_up_bits(en_priv->state.rx_next - rx_deliv - 1), + RTE_PDCP_CTRL_PDU_SIZE_MAX - pdu_sz); + bitmap_sz = pdcp_cnt_get_bitmap_size(actual_sz); + + data = (uint8_t *)rte_pktmbuf_append(m, pdu_sz + bitmap_sz); + if (data == NULL) + return -ENOMEM; + + m->pkt_len = pdu_sz + actual_sz; + m->data_len = pdu_sz + actual_sz; + + pdcp_hdr_fill((struct rte_pdcp_up_ctrl_pdu_hdr *)data, rx_deliv); + + data = RTE_PTR_ADD(data, pdu_sz); + pdcp_cnt_report_fill(dl->bitmap, en_priv->state, data, bitmap_sz); + + return 0; } diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h index a2424fbd10..2a87928b88 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.h +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -10,6 +10,7 @@ #include "pdcp_entity.h" int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m); #endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index efc74ba9b9..a9b1428c7a 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -182,6 +182,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /** Bitmap memory region */ + uint8_t bitmap_mem[0]; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index daf2c27363..774f5646d8 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -10,6 +10,7 @@ #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -857,11 +858,15 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; + if (unlikely(pdcp_cnt_bitmap_is_set(dl->bitmap, count))) + return -EEXIST; + + pdcp_cnt_bitmap_set(dl->bitmap, count); pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); if (en_priv->flags.is_out_of_order_delivery) { out_mb[0] = mb; - en_priv->state.rx_deliv = count + 1; + pdcp_rx_deliv_set(entity, count + 1); return 1; } @@ -888,7 +893,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, } /* Processed should never exceed the window size */ - en_priv->state.rx_deliv = count + processed; + pdcp_rx_deliv_set(entity, count + processed); } else { if (!reorder->is_active) diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index a52f769b82..fa3d93b405 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -8,6 +8,9 @@ #include <rte_mbuf_dyn.h> #include <rte_pdcp.h> +#include <pdcp_entity.h> +#include <pdcp_cnt.h> + typedef uint32_t rte_pdcp_dynfield_t; extern int rte_pdcp_dynfield_offset; @@ -21,4 +24,14 @@ pdcp_dynfield(struct rte_mbuf *mbuf) int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); +static inline void +pdcp_rx_deliv_set(const struct rte_pdcp_entity *entity, uint32_t rx_deliv) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + pdcp_cnt_bitmap_range_clear(dl->bitmap, en_priv->state.rx_deliv, rx_deliv); + en_priv->state.rx_deliv = rx_deliv; +} + #endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 819c66bd08..9865c620b7 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -14,6 +14,8 @@ #define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" +static int bitmap_mem_offset; + int rte_pdcp_dynfield_offset = -1; static int @@ -39,9 +41,12 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); - else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size = RTE_CACHE_LINE_ROUNDUP(size); + bitmap_mem_offset = size; + size += pdcp_cnt_bitmap_get_memory_footprint(conf); + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; @@ -54,11 +59,24 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + void *bitmap_mem; + int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - return pdcp_reorder_create(&dl->reorder, window_size); + ret = pdcp_reorder_create(&dl->reorder, window_size); + if (ret) + return ret; + + bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); + ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); + if (ret) { + pdcp_reorder_destroy(&dl->reorder); + return ret; + } + + return 0; } struct rte_pdcp_entity * @@ -136,10 +154,6 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; } - ret = pdcp_cnt_ring_create(entity, conf); - if (ret) - goto crypto_sess_destroy; - return entity; crypto_sess_destroy: @@ -218,6 +232,7 @@ struct rte_mbuf * rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, enum rte_pdcp_ctrl_pdu_type type) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; struct rte_mbuf *m; int ret; @@ -228,6 +243,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, } en_priv = entity_priv_get(pdcp_entity); + dl = entity_dl_part_get(pdcp_entity); m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); if (m == NULL) { @@ -237,7 +253,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, switch (type) { case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: - ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + ret = pdcp_ctrl_pdu_status_gen(en_priv, dl, m); break; default: ret = -ENOTSUP; @@ -283,7 +299,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered * to upper layers, with COUNT value >= RX_REORD; */ - en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + pdcp_rx_deliv_set(entity, en_priv->state.rx_reord + nb_seq); /* * - if RX_DELIV < RX_NEXT: -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 20/21] pdcp: allocate reorder buffer alongside with entity 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (18 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 19/21] pdcp: add support for status report Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 21/21] test/pdcp: add PDCP status report cases Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Instead of allocating reorder buffer separately on heap, allocate memory for it together with rest of entity, and then only initialize buffer via `rte_reorder_init()`. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 9 +++---- lib/pdcp/pdcp_cnt.h | 3 ++- lib/pdcp/pdcp_entity.h | 2 +- lib/pdcp/pdcp_reorder.c | 11 ++------ lib/pdcp/pdcp_reorder.h | 12 ++++++--- lib/pdcp/rte_pdcp.c | 58 ++++++++++++++++++++++++++--------------- 6 files changed, 55 insertions(+), 40 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index af027b00d3..e1d0634b4d 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -20,15 +20,14 @@ pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) } int -pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size) { - uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - - dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + dl->bitmap.bmp = rte_bitmap_init(nb_elem, bitmap_mem, mem_size); if (dl->bitmap.bmp == NULL) return -EINVAL; - dl->bitmap.size = window_size; + dl->bitmap.size = nb_elem; return 0; } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index 5941b7a406..87b011f9dc 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -10,7 +10,8 @@ #include "pdcp_entity.h" uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); -int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size); void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index a9b1428c7a..9f74b5d0e5 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -132,7 +132,7 @@ struct pdcp_cnt_bitmap { }; /* - * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] [reorder/bitmap] */ struct entity_priv { diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c index 5399f0dc28..bc45f2e19b 100644 --- a/lib/pdcp/pdcp_reorder.c +++ b/lib/pdcp/pdcp_reorder.c @@ -8,20 +8,13 @@ #include "pdcp_reorder.h" int -pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size) { - reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + reorder->buf = rte_reorder_init(mem, mem_size, "reorder_buffer", nb_elem); if (reorder->buf == NULL) return -rte_errno; - reorder->window_size = window_size; reorder->is_active = false; return 0; } - -void -pdcp_reorder_destroy(const struct pdcp_reorder *reorder) -{ - rte_reorder_free(reorder->buf); -} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h index 6a2f61d6ae..7e4f079d4b 100644 --- a/lib/pdcp/pdcp_reorder.h +++ b/lib/pdcp/pdcp_reorder.h @@ -9,12 +9,18 @@ struct pdcp_reorder { struct rte_reorder_buffer *buf; - uint32_t window_size; bool is_active; }; -int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); -void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); +int pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size); + +/* NOTE: replace with `rte_reorder_memory_footprint_get` after DPDK 23.07 */ +#define SIZE_OF_REORDER_BUFFER (4 * RTE_CACHE_LINE_SIZE) +static inline size_t +pdcp_reorder_memory_footprint_get(size_t nb_elem) +{ + return SIZE_OF_REORDER_BUFFER + (2 * nb_elem * sizeof(struct rte_mbuf *)); +} static inline uint32_t pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 9865c620b7..1c6d2466b2 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -14,7 +14,15 @@ #define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" -static int bitmap_mem_offset; +struct entity_layout { + size_t bitmap_offset; + size_t bitmap_size; + + size_t reorder_buf_offset; + size_t reorder_buf_size; + + size_t total_size; +}; int rte_pdcp_dynfield_offset = -1; @@ -35,46 +43,54 @@ pdcp_dynfield_register(void) } static int -pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +pdcp_entity_layout_get(const struct rte_pdcp_entity_conf *conf, struct entity_layout *layout) { - int size; + size_t size; + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); + /* Bitmap require memory to be cache aligned */ size = RTE_CACHE_LINE_ROUNDUP(size); - bitmap_mem_offset = size; - size += pdcp_cnt_bitmap_get_memory_footprint(conf); + layout->bitmap_offset = size; + layout->bitmap_size = pdcp_cnt_bitmap_get_memory_footprint(conf); + size += layout->bitmap_size; + layout->reorder_buf_offset = size; + layout->reorder_buf_size = pdcp_reorder_memory_footprint_get(window_size); + size += layout->reorder_buf_size; } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; - return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); + layout->total_size = size; + + return 0; } static int -pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf, + const struct entity_layout *layout) { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); - void *bitmap_mem; + void *memory; int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - ret = pdcp_reorder_create(&dl->reorder, window_size); + memory = RTE_PTR_ADD(entity, layout->reorder_buf_offset); + ret = pdcp_reorder_create(&dl->reorder, window_size, memory, layout->reorder_buf_size); if (ret) return ret; - bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); - ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); - if (ret) { - pdcp_reorder_destroy(&dl->reorder); + memory = RTE_PTR_ADD(entity, layout->bitmap_offset); + ret = pdcp_cnt_bitmap_create(dl, window_size, memory, layout->bitmap_size); + if (ret) return ret; - } return 0; } @@ -82,10 +98,11 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { + struct entity_layout entity_layout = { 0 }; struct rte_pdcp_entity *entity = NULL; struct entity_priv *en_priv; - int ret, entity_size; uint32_t count; + int ret; if (pdcp_dynfield_register() < 0) return NULL; @@ -118,13 +135,14 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } - entity_size = pdcp_entity_size_get(conf); - if (entity_size < 0) { + ret = pdcp_entity_layout_get(conf, &entity_layout); + if (ret < 0) { rte_errno = EINVAL; return NULL; } - entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + entity = rte_zmalloc_socket("pdcp_entity", entity_layout.total_size, RTE_CACHE_LINE_SIZE, + SOCKET_ID_ANY); if (entity == NULL) { rte_errno = ENOMEM; return NULL; @@ -149,7 +167,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { - ret = pdcp_dl_establish(entity, conf); + ret = pdcp_dl_establish(entity, conf, &entity_layout); if (ret) goto crypto_sess_destroy; } @@ -174,8 +192,6 @@ pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, en_priv->state.rx_next); - pdcp_reorder_destroy(&dl->reorder); - return nb_out; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v5 21/21] test/pdcp: add PDCP status report cases 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (19 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph @ 2023-05-27 8:59 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-27 8:59 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Test PDCP status report generation. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 312 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 312 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 82cc25ec7a..423526380f 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -2,6 +2,7 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_errno.h> #ifdef RTE_LIB_EVENTDEV #include <rte_eventdev.h> @@ -48,6 +49,9 @@ struct pdcp_testsuite_params { #endif /* RTE_LIB_EVENTDEV */ bool timer_is_running; uint64_t min_resolution_ns; + struct rte_pdcp_up_ctrl_pdu_hdr *status_report; + uint32_t status_report_bitmask_capacity; + uint8_t *ctrl_pdu_buf; }; static struct pdcp_testsuite_params testsuite_params; @@ -168,6 +172,18 @@ static struct rte_pdcp_t_reordering t_reorder_timer = { .stop = pdcp_timer_stop_cb, }; +static inline void +bitmask_set_bit(uint8_t *mask, uint32_t bit) +{ + mask[bit / 8] |= (1 << bit % 8); +} + +static inline bool +bitmask_is_bit_set(const uint8_t *mask, uint32_t bit) +{ + return mask[bit / 8] & (1 << (bit % 8)); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -314,6 +330,21 @@ testsuite_setup(void) goto cop_pool_free; } + /* Allocate memory for longest possible status report */ + ts_params->status_report_bitmask_capacity = RTE_PDCP_CTRL_PDU_SIZE_MAX - + sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + ts_params->status_report = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->status_report == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report\n"); + goto cop_pool_free; + } + + ts_params->ctrl_pdu_buf = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->ctrl_pdu_buf == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report data\n"); + goto cop_pool_free; + } + return 0; cop_pool_free: @@ -322,6 +353,8 @@ testsuite_setup(void) mbuf_pool_free: rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); return TEST_FAILED; } @@ -344,6 +377,9 @@ testsuite_teardown(void) rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); } static int @@ -1410,6 +1446,246 @@ test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) return ret; } +static struct rte_pdcp_up_ctrl_pdu_hdr * +pdcp_status_report_init(uint32_t fmc) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + + hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + hdr->fmc = rte_cpu_to_be_32(fmc); + hdr->r = 0; + memset(hdr->bitmap, 0, testsuite_params.status_report_bitmask_capacity); + + return hdr; +} + +static uint32_t +pdcp_status_report_len(void) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + uint32_t i; + + for (i = testsuite_params.status_report_bitmask_capacity; i != 0; i--) { + if (hdr->bitmap[i - 1]) + return i; + } + + return 0; +} + +static int +pdcp_status_report_verify(struct rte_mbuf *status_report, + const struct rte_pdcp_up_ctrl_pdu_hdr *expected_hdr, uint32_t expected_len) +{ + uint32_t received_len = rte_pktmbuf_pkt_len(status_report); + uint8_t *received_buf = testsuite_params.ctrl_pdu_buf; + int ret; + + ret = pktmbuf_read_into(status_report, received_buf, RTE_PDCP_CTRL_PDU_SIZE_MAX); + TEST_ASSERT_SUCCESS(ret, "Failed to copy status report pkt into continuous buffer"); + + debug_hexdump(stdout, "Received:", received_buf, received_len); + debug_hexdump(stdout, "Expected:", expected_hdr, expected_len); + + TEST_ASSERT_EQUAL(expected_len, received_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_len, received_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(received_buf, expected_hdr, expected_len, + "Generated packet not as expected"); + + return 0; +} + +static int +test_status_report_gen(const struct pdcp_test_conf *ul_conf, + const struct rte_pdcp_up_ctrl_pdu_hdr *hdr, + uint32_t bitmap_len) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *status_report = NULL, **out_mb, *m; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint32_t nb_pkts = 0, i; + uint8_t cdev_id; + + const uint32_t start_count = rte_be_to_cpu_32(hdr->fmc); + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.status_report_required = true; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + out_mb = calloc(pdcp_entity->max_pkt_cache, sizeof(uintptr_t)); + + for (i = 0; i < bitmap_len * 8; i++) { + if (!bitmask_is_bit_set(hdr->bitmap, i)) + continue; + + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + + } + + m = NULL; + + /* Check status report */ + status_report = rte_pdcp_control_pdu_create(pdcp_entity, + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT); + ASSERT_TRUE_OR_GOTO(status_report != NULL, exit, "Could not generate status report\n"); + + const uint32_t expected_len = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr) + bitmap_len; + + ASSERT_TRUE_OR_GOTO(pdcp_status_report_verify(status_report, hdr, expected_len) == 0, exit, + "Report verification failure\n"); + + ret = TEST_SUCCESS; +exit: + rte_free(m); + rte_pktmbuf_free(status_report); + rte_pktmbuf_free_bulk(out_mb, nb_pkts); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + free(out_mb); + return ret; +} + +static void +ctrl_pdu_hdr_packet_set(struct rte_pdcp_up_ctrl_pdu_hdr *hdr, uint32_t pkt_count) +{ + bitmask_set_bit(hdr->bitmap, pkt_count - rte_be_to_cpu_32(hdr->fmc) - 1); +} + +static int +test_status_report_fmc_only(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(42); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_first_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_second_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_full_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE + 1; + int i; + + for (i = 0; i < RTE_BITMAP_SLAB_BIT_SIZE; i++) + ctrl_pdu_hdr_packet_set(hdr, start_offset + i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_non_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + ctrl_pdu_hdr_packet_set(hdr, 3 * RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_max_length_sn_12(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr; + const uint32_t fmc = 0; + uint32_t i; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + ul_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + hdr = pdcp_status_report_init(fmc); + + const uint32_t max_count = RTE_MIN((RTE_PDCP_CTRL_PDU_SIZE_MAX - sizeof(hdr)) * 8, + (uint32_t)PDCP_WINDOW_SIZE(RTE_SECURITY_PDCP_SN_SIZE_12)); + + i = fmc + 2; /* set first count to have a gap, to enable packet buffering */ + + for (; i < max_count; i++) + ctrl_pdu_hdr_packet_set(hdr, i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_different_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(63); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 64 + 1); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_same_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(2); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 4); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1611,11 +1887,47 @@ static struct unit_test_suite reorder_test_cases = { } }; +static struct unit_test_suite status_report_test_cases = { + .suite_name = "PDCP status report", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_status_report_fmc_only", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_fmc_only), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_first_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_first_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_second_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_second_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_full_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_full_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_non_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_non_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_max_length_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_status_report_max_length_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_different_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_different_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_same_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_same_slab), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, &reorder_test_cases, + &status_report_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 00/21] lib: add pdcp protocol 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph ` (20 preceding siblings ...) 2023-05-27 8:59 ` [PATCH v5 21/21] test/pdcp: add PDCP status report cases Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 01/21] net: add PDCP header Anoob Joseph ` (21 more replies) 21 siblings, 22 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following operations, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Similar to lib IPsec, lib PDCP would allow application to use same API sequence while leveraging protocol offload features enabled by rte_security library. Lib PDCP would internally change the handles registered for *pre_process* and *post_process* based on features enabled in the entity. Lib PDCP would create the required sessions on the device provided in entity to minimize the application requirements. Also, the crypto_op allocation and free would also be done internally by lib PDCP to allow the library to create crypto ops as required for the input packets. For example, when control PDUs are received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP is expected to handle it differently. Lib PDCP utilizes reorder library for implementing in-order delivery. It utilizes bitmap library for implementing status reports and track the COUNT value of the packets received. To allow application to choose timer implementation of choice, lib PDCP allows application to configure handles that can be used for starting & stopping timers. Upon expiry, application can call corresponding PDCP API(``rte_pdcp_t_reordering_expiry_handle``) for handling the event. Unit tests are added to verify both rte_timer based timers as well as rte_eventdev based timers. PDCP tracks the sequence number of the received packets and during events such as re-establishment, it is required to generate reports and transmit to the peer. This series introduces ``rte_pdcp_control_pdu_create`` for handling control PDU generation. Changes in v6: - Rebased - Minor udpates to documentation (Akhil) Changes in v5: - Deferred patch adding thread safe processing. - Updated release notes & MAINTAINERS file. Changes in v4: - Disabled 'annotate locks' with lib PDCP - Enable PDCP autotest only when lib is enabled - Use rwlock instead of spinlock - Avoid per packet checks for thread safety (Stephen) - In DL path, save count determined during pre-process in mbuf and use the same in post-process. Determining count again may lead To errors - Simplified DL path threads to allow more common code between SN 12 & 18 Changes in v3: - Addressed review comments (Akhil) - Addressed build failure in CI (tests with lib eventdev disabled) - Addressed checkpatch issues - Set only positive values to rte_errno (Akhil) Changes in v2: - Added control PDU handling - Added t-Reordering timer - Added in-order delivery - Added status PDU generation - Rebased on top of new features added in reorder library - Split base patch - Increased test coverage - Improved thread safety Changes from RFC - Implementation for all APIs covering basic control plane & user plane packets - Unit test leveraging existing PDCP vectors available in test_cryptodev - Unit test performing both UL & DL operations to verify various protocol features - Updated documentation Sample application sequence: struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; struct rte_crypto_op *cop[MAX_BURST_SIZE]; struct rte_pdcp_group grp[MAX_BURST_SIZE]; struct rte_pdcp_entity *pdcp_entity; int nb_max_out_mb, ret, nb_grp; /* Create PDCP entity */ pdcp_entity = rte_pdcp_entity_establish(&conf); /** * Allocate buffer for holding mbufs returned during PDCP suspend, * release & post-process APIs. */ /* Max packets that can be cached in entity + burst size */ nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); if (out_mb == NULL) { /* Handle error */ } while (1) { /* Receive packet and form mbuf */ /** * Prepare packets for crypto operation. Following operations * would be done, * * Transmitting entity/UL (only data PDUs): * - Perform compression * - Assign sequence number * - Add PDCP header * - Create & prepare crypto_op * - Prepare IV for crypto operation (auth_gen, encrypt) * - Save original PDCP SDU (during PDCP re-establishment, * unconfirmed PDCP SDUs need to crypto processed again and * transmitted/re-transmitted) * * Receiving entity/DL: * - Any control PDUs received would be processed and * appropriate actions taken. If data PDU, continue. * - Determine sequence number (based on HFN & per packet SN) * - Prepare crypto_op * - Prepare IV for crypto operation (decrypt, auth_verify) */ nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, nb_rx, &nb_err); if (nb_err != 0) { /* Handle error packets */ } if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) != nb_success) { /* Retry for enqueue failure packets */ } ... ret = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, MAX_BURST_SIZE); if (ret == 0) continue; /** * Received a burst of completed crypto ops from cryptodev. It * may belong to various entities. Group similar ones together * for entity specific post-processing. */ /** * Groups similar entities together. Frees crypto op and based * on crypto_op status, set mbuf->ol_flags which would be * checked in rte_pdcp_pkt_post_process(). */ nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); for (i = 0; i != nb_grp; i++) { /** * Post process packets after crypto completion. * Following operations would be done, * * Transmitting entity/UL: * - Check crypto result * * Receiving entity/DL: * - Check crypto operation status * - Check for duplication (if yes, drop duplicate) * - Perform decompression * - Trim PDCP header * - Hold packet (SDU) for in-order delivery (return * completed packets as and when sequence is * completed) * - If not in sequence, cache the packet and start * t-Reordering timer. When timer expires, the * packets need to delivered to upper layers (not * treated as error packets). */ nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, grp[i].m, out_mb, grp[i].cnt, &nb_err); if (nb_err != 0) { /* Handle error packets */ } /* Perform additional operations */ /** * Tranmitting entity/UL * - If duplication is enabled, duplicate PDCP PDUs * - When lower layers confirm reception of a PDCP PDU, * it should be communicated to PDCP layer so that * PDCP can drop the corresponding SDU */ } } Sample output from unit test executed on crypto_cn10k PMD(summary): ~# DPDK_TEST=pdcp_autotest ./dpdk-test --log-level=7 <snip> + ------------------------------------------------------- + + Test Suite Summary : PDCP Unit Test Suite + ------------------------------------------------------- + + Known vector cases : 112/160 passed, 0/160 skipped, 0/160 failed, 48/160 unsupported + PDCP combined mode : 1/1 passed, 0/1 skipped, 0/1 failed, 0/1 unsupported + PDCP HFN/SN : 4/4 passed, 0/4 skipped, 0/4 failed, 0/4 unsupported + PDCP reorder : 5/5 passed, 0/5 skipped, 0/5 failed, 0/5 unsupported + PDCP status report : 9/9 passed, 0/9 skipped, 0/9 failed, 0/9 unsupported + ------------------------------------------------------- + + Sub Testsuites Total : 5 + Sub Testsuites Skipped : 0 + Sub Testsuites Passed : 5 + Sub Testsuites Failed : 0 + ------------------------------------------------------- + + Tests Total : 179 + Tests Skipped : 0 + Tests Executed : 179 + Tests Unsupported: 48 + Tests Passed : 131 + Tests Failed : 0 + ------------------------------------------------------- + Test OK Anoob Joseph (10): lib: add pdcp protocol pdcp: add pre and post-process pdcp: add packet group pdcp: add crypto session create and destroy pdcp: add pre and post process for UL pdcp: add pre and post process for DL pdcp: add IV generation routines app/test: add lib pdcp tests doc: add PDCP library guide pdcp: add control PDU handling for status report Volodymyr Fialko (11): net: add PDCP header test/pdcp: pdcp HFN tests in combined mode pdcp: implement t-Reordering and packet buffering test/pdcp: add in-order delivery cases pdcp: add timer callback handlers pdcp: add timer expiry handle test/pdcp: add timer expiry cases test/pdcp: add timer restart case pdcp: add support for status report pdcp: allocate reorder buffer alongside with entity test/pdcp: add PDCP status report cases MAINTAINERS | 8 + app/test/meson.build | 5 + app/test/test_pdcp.c | 1981 +++++++++++++++++ doc/api/doxy-api-index.md | 4 +- doc/api/doxy-api.conf.in | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 294 +++ doc/guides/rel_notes/release_23_07.rst | 12 + lib/meson.build | 1 + lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++ lib/pdcp/meson.build | 21 + lib/pdcp/pdcp_cnt.c | 164 ++ lib/pdcp/pdcp_cnt.h | 24 + lib/pdcp/pdcp_crypto.c | 238 ++ lib/pdcp/pdcp_crypto.h | 20 + lib/pdcp/pdcp_ctrl_pdu.c | 72 + lib/pdcp/pdcp_ctrl_pdu.h | 16 + lib/pdcp/pdcp_entity.h | 260 +++ lib/pdcp/pdcp_process.c | 1223 ++++++++++ lib/pdcp/pdcp_process.h | 37 + lib/pdcp/pdcp_reorder.c | 20 + lib/pdcp/pdcp_reorder.h | 68 + lib/pdcp/rte_pdcp.c | 334 +++ lib/pdcp/rte_pdcp.h | 380 ++++ lib/pdcp/rte_pdcp_group.h | 131 ++ lib/pdcp/version.map | 20 + 28 files changed, 5483 insertions(+), 1 deletion(-) create mode 100644 app/test/test_pdcp.c create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst create mode 100644 lib/net/rte_pdcp_hdr.h create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/rte_pdcp_group.h create mode 100644 lib/pdcp/version.map -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 01/21] net: add PDCP header 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 02/21] lib: add pdcp protocol Anoob Joseph ` (20 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add PDCP protocol header to be used for supporting PDCP protocol processing. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- doc/api/doxy-api-index.md | 3 +- lib/net/meson.build | 1 + lib/net/rte_pdcp_hdr.h | 147 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 150 insertions(+), 1 deletion(-) create mode 100644 lib/net/rte_pdcp_hdr.h diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index c709fd48ad..debbe4134f 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -127,7 +127,8 @@ The public API headers are grouped by topics: [Geneve](@ref rte_geneve.h), [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), - [PPP](@ref rte_ppp.h) + [PPP](@ref rte_ppp.h), + [PDCP hdr](@ref rte_pdcp_hdr.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/lib/net/meson.build b/lib/net/meson.build index 379d161ee0..bd56f91c22 100644 --- a/lib/net/meson.build +++ b/lib/net/meson.build @@ -22,6 +22,7 @@ headers = files( 'rte_geneve.h', 'rte_l2tpv2.h', 'rte_ppp.h', + 'rte_pdcp_hdr.h', ) sources = files( diff --git a/lib/net/rte_pdcp_hdr.h b/lib/net/rte_pdcp_hdr.h new file mode 100644 index 0000000000..72ae9a66cb --- /dev/null +++ b/lib/net/rte_pdcp_hdr.h @@ -0,0 +1,147 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_HDR_H +#define RTE_PDCP_HDR_H + +/** + * @file + * + * PDCP-related defines + * + * Based on - ETSI TS 138 323 V17.1.0 (2022-08) + * https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + */ + +#include <rte_byteorder.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * 4.3.1 + * + * Indicate the maximum supported size of a PDCP Control PDU. + */ +#define RTE_PDCP_CTRL_PDU_SIZE_MAX 9000u + +/** + * 6.3.4 MAC-I + * + * Indicate the size of MAC-I in PDCP PDU. + */ +#define RTE_PDCP_MAC_I_LEN 4 + +/** + * Indicate type of control information included in the corresponding PDCP + * Control PDU. + */ +enum rte_pdcp_ctrl_pdu_type { + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT = 0, + RTE_PDCP_CTRL_PDU_TYPE_ROHC_FEEDBACK = 1, + RTE_PDCP_CTRL_PDU_TYPE_EHC_FEEDBACK = 2, + RTE_PDCP_CRTL_PDU_TYPE_UDC_FEEDBACK = 3, +}; + +/** + * 6.3.7 D/C + * + * This field indicates whether the corresponding PDCP PDU is a + * PDCP Data PDU or a PDCP Control PDU. + */ +enum rte_pdcp_pdu_type { + RTE_PDCP_PDU_TYPE_CTRL = 0, + RTE_PDCP_PDU_TYPE_DATA = 1, +}; + +/** + * 6.2.2.1 Data PDU for SRBs + */ +__extension__ +struct rte_pdcp_cp_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 4; /**< Reserved */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.2.2 Data PDU for DRBs and MRBs with 12 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_12_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ + uint8_t r : 3; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 3; /**< Reserved */ + uint8_t sn_11_8 : 4; /**< Sequence number bits 8-11 */ +#endif + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.2.3 Data PDU for DRBs and MRBs with 18 bits PDCP SN + */ +__extension__ +struct rte_pdcp_up_data_pdu_sn_18_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ + uint8_t r : 5; /**< Reserved */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t r : 5; /**< Reserved */ + uint8_t sn_17_16 : 2; /**< Sequence number bits 16-17 */ +#endif + uint8_t sn_15_8; /**< Sequence number bits 8-15 */ + uint8_t sn_7_0; /**< Sequence number bits 0-7 */ +} __rte_packed; + +/** + * 6.2.3.1 Control PDU for PDCP status report + */ +__extension__ +struct rte_pdcp_up_ctrl_pdu_hdr { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint8_t r : 4; /**< Reserved */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t d_c : 1; /**< D/C bit */ +#elif RTE_BYTE_ORDER == RTE_BIG_ENDIAN + uint8_t d_c : 1; /**< D/C bit */ + uint8_t pdu_type : 3; /**< Control PDU type */ + uint8_t r : 4; /**< Reserved */ +#endif + /** + * 6.3.9 FMC + * + * First Missing COUNT. This field indicates the COUNT value of the + * first missing PDCP SDU within the reordering window, i.e. RX_DELIV. + */ + rte_be32_t fmc; + /** + * 6.3.10 Bitmap + * + * Length: Variable. The length of the bitmap field can be 0. + * + * This field indicates which SDUs are missing and which SDUs are + * correctly received in the receiving PDCP entity. The bit position of + * Nth bit in the Bitmap is N, i.e., the bit position of the first bit + * in the Bitmap is 1. + */ + uint8_t bitmap[]; +} __rte_packed; + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_HDR_H */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 02/21] lib: add pdcp protocol 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 01/21] net: add PDCP header Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 03/21] pdcp: add pre and post-process Anoob Joseph ` (19 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add Packet Data Convergence Protocol (PDCP) processing library. The library is similar to lib_ipsec which provides IPsec processing capabilities in DPDK. PDCP would involve roughly the following options, 1. Transfer of user plane data 2. Transfer of control plane data 3. Header compression 4. Uplink data compression 5. Ciphering and integrity protection PDCP library provides following control path APIs that is used to configure various PDCP entities, 1. rte_pdcp_entity_establish() 2. rte_pdcp_entity_suspend() 3. rte_pdcp_entity_release() Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 6 ++ doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + lib/meson.build | 1 + lib/pdcp/meson.build | 17 ++++ lib/pdcp/pdcp_crypto.c | 21 +++++ lib/pdcp/pdcp_crypto.h | 15 ++++ lib/pdcp/pdcp_entity.h | 113 ++++++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 138 +++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 13 +++ lib/pdcp/rte_pdcp.c | 141 ++++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 166 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 10 +++ 13 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 lib/pdcp/meson.build create mode 100644 lib/pdcp/pdcp_crypto.c create mode 100644 lib/pdcp/pdcp_crypto.h create mode 100644 lib/pdcp/pdcp_entity.h create mode 100644 lib/pdcp/pdcp_process.c create mode 100644 lib/pdcp/pdcp_process.h create mode 100644 lib/pdcp/rte_pdcp.c create mode 100644 lib/pdcp/rte_pdcp.h create mode 100644 lib/pdcp/version.map diff --git a/MAINTAINERS b/MAINTAINERS index a5219926ab..82f490c5c0 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1551,6 +1551,12 @@ F: doc/guides/tools/pdump.rst F: app/dumpcap/ F: doc/guides/tools/dumpcap.rst +PDCP - EXPERIMENTAL +M: Anoob Joseph <anoobj@marvell.com> +M: Volodymyr Fialko <vfialko@marvell.com> +T: git://dpdk.org/next/dpdk-next-crypto +F: lib/pdcp/ + Packet Framework ---------------- diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index debbe4134f..cd7a6cae44 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -128,7 +128,8 @@ The public API headers are grouped by topics: [eCPRI](@ref rte_ecpri.h), [L2TPv2](@ref rte_l2tpv2.h), [PPP](@ref rte_ppp.h), - [PDCP hdr](@ref rte_pdcp_hdr.h) + [PDCP hdr](@ref rte_pdcp_hdr.h), + [PDCP](@ref rte_pdcp.h) - **QoS**: [metering](@ref rte_meter.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index d230a19e1f..58789308a9 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -62,6 +62,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/net \ @TOPDIR@/lib/pcapng \ @TOPDIR@/lib/pci \ + @TOPDIR@/lib/pdcp \ @TOPDIR@/lib/pdump \ @TOPDIR@/lib/pipeline \ @TOPDIR@/lib/port \ diff --git a/lib/meson.build b/lib/meson.build index dc8aa4ac84..a6a54c196c 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -64,6 +64,7 @@ libraries = [ 'flow_classify', # flow_classify lib depends on pkt framework table lib 'graph', 'node', + 'pdcp', # pdcp lib depends on crypto and security ] optional_libs = [ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build new file mode 100644 index 0000000000..ccaf426240 --- /dev/null +++ b/lib/pdcp/meson.build @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(C) 2023 Marvell. + +if is_windows + build = false + reason = 'not supported on Windows' + subdir_done() +endif + +sources = files( + 'pdcp_crypto.c', + 'pdcp_process.c', + 'rte_pdcp.c', + ) +headers = files('rte_pdcp.h') + +deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c new file mode 100644 index 0000000000..755e27ec9e --- /dev/null +++ b/lib/pdcp/pdcp_crypto.c @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_crypto.h" + +int +pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + RTE_SET_USED(entity); + RTE_SET_USED(conf); + return 0; +} + +void +pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) +{ + RTE_SET_USED(entity); +} diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h new file mode 100644 index 0000000000..6563331d37 --- /dev/null +++ b/lib/pdcp/pdcp_crypto.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CRYPTO_H +#define PDCP_CRYPTO_H + +#include <rte_pdcp.h> + +int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, + const struct rte_pdcp_entity_conf *conf); + +void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity); + +#endif /* PDCP_CRYPTO_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h new file mode 100644 index 0000000000..000297588f --- /dev/null +++ b/lib/pdcp/pdcp_entity.h @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_ENTITY_H +#define PDCP_ENTITY_H + +#include <rte_common.h> +#include <rte_crypto_sym.h> +#include <rte_mempool.h> +#include <rte_pdcp.h> +#include <rte_security.h> + +struct entity_priv; + +/* IV generation function based on the entity configuration */ +typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count); + +struct entity_state { + uint32_t rx_next; + uint32_t tx_next; + uint32_t rx_deliv; + uint32_t rx_reord; +}; + +/* + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + */ + +struct entity_priv { + /** Crypto sym session. */ + struct rte_cryptodev_sym_session *crypto_sess; + /** Entity specific IV generation function. */ + iv_gen_t iv_gen; + /** Entity state variables. */ + struct entity_state state; + /** Flags. */ + struct { + /** PDCP PDU has 4 byte MAC-I. */ + uint64_t is_authenticated : 1; + /** Cipher offset & length in bits. */ + uint64_t is_cipher_in_bits : 1; + /** Auth offset & length in bits. */ + uint64_t is_auth_in_bits : 1; + /** Is UL/transmitting PDCP entity. */ + uint64_t is_ul_entity : 1; + /** Is NULL auth. */ + uint64_t is_null_auth : 1; + } flags; + /** Crypto op pool. */ + struct rte_mempool *cop_pool; + /** PDCP header size. */ + uint8_t hdr_sz; + /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ + uint8_t aad_sz; + /** Device ID of the device to be used for offload. */ + uint8_t dev_id; +}; + +struct entity_priv_dl_part { + /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ + uint8_t dummy; +}; + +struct entity_priv_ul_part { + /* + * NOTE: when re-establish is supported, plain PDCP packets & COUNT values need to be + * cached. + */ + uint8_t dummy; +}; + +static inline struct entity_priv * +entity_priv_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity)); +} + +static inline struct entity_priv_dl_part * +entity_dl_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline struct entity_priv_ul_part * +entity_ul_part_get(const struct rte_pdcp_entity *entity) { + return RTE_PTR_ADD(entity, sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv)); +} + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) +{ + return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); +} + +#endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c new file mode 100644 index 0000000000..79f5dce5db --- /dev/null +++ b/lib/pdcp/pdcp_process.c @@ -0,0 +1,138 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_memcpy.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, + struct rte_crypto_sym_xform **a_xfrm) +{ + *c_xfrm = NULL; + *a_xfrm = NULL; + + if (conf->crypto_xfrm == NULL) + return -EINVAL; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + *c_xfrm = conf->crypto_xfrm; + *a_xfrm = conf->crypto_xfrm->next; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + *a_xfrm = conf->crypto_xfrm; + *c_xfrm = conf->crypto_xfrm->next; + } else { + return -EINVAL; + } + + return 0; +} + +static int +pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + int ret; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + /** + * flags.is_authenticated + * + * MAC-I would be added in case of control plane packets and when authentication + * transform is not NULL. + */ + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && (a_xfrm == NULL)) + return -EINVAL; + + if (a_xfrm != NULL) + en_priv->flags.is_authenticated = 1; + + /** + * flags.is_cipher_in_bits + * + * For ZUC & SNOW3G cipher algos, offset & length need to be provided in bits. + */ + + if ((c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2) || + (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3)) + en_priv->flags.is_cipher_in_bits = 1; + + /** + * flags.is_auth_in_bits + * + * For ZUC & SNOW3G authentication algos, offset & length need to be provided in bits. + */ + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3)) + en_priv->flags.is_auth_in_bits = 1; + } + + /** + * flags.is_ul_entity + * + * Indicate whether the entity is UL/transmitting PDCP entity. + */ + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + en_priv->flags.is_ul_entity = 1; + + /** + * flags.is_null_auth + * + * For NULL auth, 4B zeros need to be added by lib PDCP. Indicate that + * algo is NULL auth to perform the same. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + en_priv->flags.is_null_auth = 1; + + /** + * hdr_sz + * + * PDCP header size of the entity + */ + en_priv->hdr_sz = pdcp_hdr_size_get(conf->pdcp_xfrm.sn_size); + + /** + * aad_sz + * + * For AES-CMAC, additional message is prepended for processing. Need to be trimmed after + * crypto processing is done. + */ + if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_AES_CMAC) + en_priv->aad_sz = 8; + else + en_priv->aad_sz = 0; + + return 0; +} + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv *en_priv; + int ret; + + if (entity == NULL || conf == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + ret = pdcp_entity_priv_populate(en_priv, conf); + if (ret) + return ret; + + return 0; +} diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h new file mode 100644 index 0000000000..fd53fff0aa --- /dev/null +++ b/lib/pdcp/pdcp_process.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_PROCESS_H +#define PDCP_PROCESS_H + +#include <rte_pdcp.h> + +int +pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c new file mode 100644 index 0000000000..adcad5dd25 --- /dev/null +++ b/lib/pdcp/rte_pdcp.c @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_pdcp.h> +#include <rte_malloc.h> + +#include "pdcp_crypto.h" +#include "pdcp_entity.h" +#include "pdcp_process.h" + +static int +pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +{ + int size; + + size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + size += sizeof(struct entity_priv_dl_part); + else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size += sizeof(struct entity_priv_ul_part); + else + return -EINVAL; + + return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); +} + +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) +{ + struct rte_pdcp_entity *entity = NULL; + struct entity_priv *en_priv; + int ret, entity_size; + uint32_t count; + + if (conf == NULL || conf->cop_pool == NULL) { + rte_errno = EINVAL; + return NULL; + } + + if (conf->pdcp_xfrm.en_ordering || conf->pdcp_xfrm.remove_duplicates || conf->is_slrb || + conf->en_sec_offload) { + rte_errno = ENOTSUP; + return NULL; + } + + /* + * 6.3.2 PDCP SN + * Length: 12 or 18 bits as indicated in table 6.3.2-1. The length of the PDCP SN is + * configured by upper layers (pdcp-SN-SizeUL, pdcp-SN-SizeDL, or sl-PDCP-SN-Size in + * TS 38.331 [3]) + */ + if ((conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18)) { + rte_errno = ENOTSUP; + return NULL; + } + + if (conf->pdcp_xfrm.hfn_threshold) { + rte_errno = EINVAL; + return NULL; + } + + entity_size = pdcp_entity_size_get(conf); + if (entity_size < 0) { + rte_errno = EINVAL; + return NULL; + } + + entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + if (entity == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + en_priv = entity_priv_get(entity); + + count = pdcp_count_from_hfn_sn_get(conf->pdcp_xfrm.hfn, conf->sn, conf->pdcp_xfrm.sn_size); + + en_priv->state.rx_deliv = count; + en_priv->state.tx_next = count; + en_priv->cop_pool = conf->cop_pool; + + /* Setup crypto session */ + ret = pdcp_crypto_sess_create(entity, conf); + if (ret) + goto entity_free; + + ret = pdcp_process_func_set(entity, conf); + if (ret) + goto crypto_sess_destroy; + + return entity; + +crypto_sess_destroy: + pdcp_crypto_sess_destroy(entity); +entity_free: + rte_free(entity); + rte_errno = -ret; + return NULL; +} + +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) +{ + if (pdcp_entity == NULL) + return -EINVAL; + + /* Teardown crypto sessions */ + pdcp_crypto_sess_destroy(pdcp_entity); + + rte_free(pdcp_entity); + + RTE_SET_USED(out_mb); + return 0; +} + +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]) +{ + struct entity_priv *en_priv; + + if (pdcp_entity == NULL) + return -EINVAL; + + en_priv = entity_priv_get(pdcp_entity); + + if (en_priv->flags.is_ul_entity) { + en_priv->state.tx_next = 0; + } else { + en_priv->state.rx_next = 0; + en_priv->state.rx_deliv = 0; + } + + RTE_SET_USED(out_mb); + + return 0; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h new file mode 100644 index 0000000000..4489b9526b --- /dev/null +++ b/lib/pdcp/rte_pdcp.h @@ -0,0 +1,166 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_H +#define RTE_PDCP_H + +/** + * @file rte_pdcp.h + * + * RTE PDCP support. + * + * A framework for PDCP protocol processing. + */ + +#include <rte_compat.h> +#include <rte_common.h> +#include <rte_mempool.h> +#include <rte_security.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * PDCP entity. + * + * 4.2.2 PDCP entities + * + * The PDCP entities are located in the PDCP sublayer. + * Several PDCP entities may be defined for a UE. + * Each PDCP entity is carrying the data of one radio bearer. + * A PDCP entity is associated either to the control plane or the user plane + * depending on which radio bearer it is carrying data for. + */ +struct rte_pdcp_entity { + /** + * PDCP entities may hold packets for purposes of in-order delivery + * (in case of receiving PDCP entity) and re-transmission + * (in case of transmitting PDCP entity). + * + * The field 'max_pkt_cache' would be used to indicate the maximum + * number of packets that may be cached in an entity at any point of time. + * When application provides buffers to receive packets from PDCP entity, + * the size of the buffer should be such that it can + * hold additionally 'max_pkt_cache' number of packets. + */ + uint32_t max_pkt_cache; +} __rte_cache_aligned; + +/** + * PDCP entity configuration to be used for establishing an entity. + */ +/* Structure rte_pdcp_entity_conf 8< */ +struct rte_pdcp_entity_conf { + /** PDCP transform for the entity. */ + struct rte_security_pdcp_xform pdcp_xfrm; + /** Crypto transform applicable for the entity. */ + struct rte_crypto_sym_xform *crypto_xfrm; + /** Mempool for crypto symmetric session. */ + struct rte_mempool *sess_mpool; + /** Crypto op pool. */ + struct rte_mempool *cop_pool; + /** + * Sequence number value to be used. + * 32 bit count value to be used for the first packet + * would be derived based on HFN (`rte_security_pdcp_xform.hfn`) and SN. + */ + uint32_t sn; + /** Indicate whether the PDCP entity belongs to Side Link Radio Bearer. */ + bool is_slrb; + /** Enable security offload on the device specified. */ + bool en_sec_offload; + /** Device on which security/crypto session need to be created. */ + uint8_t dev_id; + /** + * Reverse direction during IV generation. + * Can be used to simulate UE crypto processing. + */ + bool reverse_iv_direction; +}; +/* >8 End of structure rte_pdcp_entity_conf. */ + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.1 PDCP entity establishment + * + * Establish PDCP entity based on provided input configuration. + * + * @param conf + * Parameters to be used for initializing PDCP entity object. + * @return + * - Valid handle if success + * - NULL in case of failure. rte_errno will be set to error code. + */ +__rte_experimental +struct rte_pdcp_entity * +rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.3 PDCP entity release + * + * Release PDCP entity. + * + * For UL/transmitting PDCP entity, all stored PDCP SDUs would be dropped. + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * Entity release would result in freeing all memory associated with the PDCP + * entity as well as any crypto/security sessions created. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be released. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return + * - >0: Success and the number of packets returned in out_mb + * - <0: Error code in case of failures + */ +__rte_experimental +int +rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.1.4 PDCP entity suspend + * + * Suspend PDCP entity. + * + * For DL/receiving PDCP entity, the stored PDCP SDUs would be returned in + * *out_mb* buffer. The buffer should be large enough to hold all cached + * packets in the entity. + * + * For UL/transmitting PDCP entity, *out_mb* buffer would be unused. + * + * @param pdcp_entity + * Pointer to the PDCP entity to be suspended. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. + * @return + * - 0: Success and no cached packets to return. + * - >0: Success and the number of packets returned in out_mb. + * - <0: Error code in case of failures. + */ +__rte_experimental +int +rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, + struct rte_mbuf *out_mb[]); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map new file mode 100644 index 0000000000..923e165f3f --- /dev/null +++ b/lib/pdcp/version.map @@ -0,0 +1,10 @@ +EXPERIMENTAL { + global: + + # added in 23.07 + rte_pdcp_entity_establish; + rte_pdcp_entity_release; + rte_pdcp_entity_suspend; + + local: *; +}; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 03/21] pdcp: add pre and post-process 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 01/21] net: add PDCP header Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 02/21] lib: add pdcp protocol Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 04/21] pdcp: add packet group Anoob Joseph ` (18 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger PDCP process is split into 2 parts. One before crypto processing (rte_pdcp_pkt_pre_process()) and one after crypto processing (rte_pdcp_pkt_post_process()). Functionality of pre-process & post-process varies based on the type of entity. Registration of entity specific function pointer allows skipping multiple checks that would come in datapath otherwise. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/rte_pdcp.h | 96 ++++++++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 ++ 2 files changed, 99 insertions(+) diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 4489b9526b..d225a6dc5c 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -22,6 +22,21 @@ extern "C" { #endif +/* Forward declarations. */ +struct rte_pdcp_entity; + +/* PDCP pre-process function based on entity configuration. */ +typedef uint16_t (*rte_pdcp_pre_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err); + +/* PDCP post-process function based on entity configuration. */ +typedef uint16_t (*rte_pdcp_post_p_t)(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err); + /** * PDCP entity. * @@ -34,6 +49,10 @@ extern "C" { * depending on which radio bearer it is carrying data for. */ struct rte_pdcp_entity { + /** Entity specific pre-process handle. */ + rte_pdcp_pre_p_t pre_process; + /** Entity specific post-process handle. */ + rte_pdcp_post_p_t post_process; /** * PDCP entities may hold packets for purposes of in-order delivery * (in case of receiving PDCP entity) and re-transmission @@ -159,6 +178,83 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity pre-process the mbufs and prepare + * crypto ops that can be enqueued to the cryptodev associated with given + * session. Only error packets would be moved returned in the input buffer, + * *mb*, and it is the responsibility of the application to free the same. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param[in, out] mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * which contain the input packets. + * Any error packets would be returned in the same buffer. + * @param[out] cop + * The address of an array that can hold up to *num* pointers to + * *rte_crypto_op* structures. Crypto ops would be allocated by + * ``rte_pdcp_pkt_pre_process`` API. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * Pointer to return the number of error packets returned in *mb*. + * @return + * Count of crypto_ops prepared. + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err) +{ + return entity->pre_process(entity, mb, cop, num, nb_err); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * For input mbufs and given PDCP entity, perform PDCP post-processing of the mbufs. + * + * Input mbufs are the ones retrieved from rte_crypto_ops dequeued from cryptodev + * and grouped by *rte_pdcp_pkt_crypto_group()*. + * + * The post-processed packets would be returned in the *out_mb* buffer. + * The resultant mbufs would be grouped into success packets and error packets. + * Error packets would be grouped in the end of the array and + * it is the responsibility of the application to handle the same. + * + * When in-order delivery is enabled, PDCP entity may buffer packets and would + * deliver packets only when all prior packets have been post-processed. + * That would result in returning more/less packets than enqueued. + * + * @param entity + * Pointer to the *rte_pdcp_entity* object the packets belong to. + * @param in_mb + * The address of an array of *num* pointers to *rte_mbuf* structures. + * @param[out] out_mb + * The address of an array of *num* pointers to *rte_mbuf* structures + * to output packets after PDCP post-processing. + * @param num + * The maximum number of packets to process. + * @param[out] nb_err + * The number of error packets returned in *out_mb* buffer. + * @return + * Count of packets returned in *out_mb* buffer. + */ +__rte_experimental +static inline uint16_t +rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err) +{ + return entity->post_process(entity, in_mb, out_mb, num, nb_err); +} + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 923e165f3f..f9ff30600a 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -6,5 +6,8 @@ EXPERIMENTAL { rte_pdcp_entity_release; rte_pdcp_entity_suspend; + rte_pdcp_pkt_post_process; + rte_pdcp_pkt_pre_process; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 04/21] pdcp: add packet group 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (2 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 03/21] pdcp: add pre and post-process Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 05/21] pdcp: add crypto session create and destroy Anoob Joseph ` (17 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Crypto processing in PDCP is performed asynchronously by rte_cryptodev_enqueue_burst() and rte_cryptodev_dequeue_burst(). Since cryptodev dequeue can return crypto operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() is added to help grouping crypto operations belonging to same entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 1 + lib/pdcp/rte_pdcp.h | 6 ++ lib/pdcp/rte_pdcp_group.h | 131 ++++++++++++++++++++++++++++++++++++++ lib/pdcp/version.map | 3 + 4 files changed, 141 insertions(+) create mode 100644 lib/pdcp/rte_pdcp_group.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index ccaf426240..08679b743a 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -13,5 +13,6 @@ sources = files( 'rte_pdcp.c', ) headers = files('rte_pdcp.h') +indirect_headers += files('rte_pdcp_group.h') deps += ['mbuf', 'net', 'cryptodev', 'security'] diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index d225a6dc5c..e63946aa08 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -255,6 +255,12 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, return entity->post_process(entity, in_mb, out_mb, num, nb_err); } +/** + * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. So include + * in the end. + */ +#include <rte_pdcp_group.h> + #ifdef __cplusplus } #endif diff --git a/lib/pdcp/rte_pdcp_group.h b/lib/pdcp/rte_pdcp_group.h new file mode 100644 index 0000000000..ece3e8c0ff --- /dev/null +++ b/lib/pdcp/rte_pdcp_group.h @@ -0,0 +1,131 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef RTE_PDCP_GROUP_H +#define RTE_PDCP_GROUP_H + +/** + * @file rte_pdcp_group.h + * + * RTE PDCP grouping support. + * It is not recommended to include this file directly, include <rte_pdcp.h> + * instead. + * Provides helper functions to process completed crypto-ops and group related + * packets by sessions they belong to. + */ + +#include <rte_common.h> +#include <rte_crypto.h> +#include <rte_cryptodev.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Group packets belonging to same PDCP entity. + */ +struct rte_pdcp_group { + union { + uint64_t val; + void *ptr; + } id; /**< Grouped by value */ + struct rte_mbuf **m; /**< Start of the group */ + uint32_t cnt; /**< Number of entries in the group */ + int32_t rc; /**< Status code associated with the group */ +}; + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take crypto-op as an input and extract pointer to related PDCP entity. + * @param cop + * The address of an input *rte_crypto_op* structure. + * @return + * The pointer to the related *rte_pdcp_entity* structure. + */ +static inline struct rte_pdcp_entity * +rte_pdcp_en_from_cop(const struct rte_crypto_op *cop) +{ + void *sess = cop->sym[0].session; + + return (struct rte_pdcp_entity *)(uintptr_t) + rte_cryptodev_sym_session_opaque_data_get(sess); +} + +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Take as input completed crypto ops, extract related mbufs and group them by + * *rte_pdcp_entity* they belong to. Mbuf for which the crypto operation has + * failed would be flagged using *RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED* flag + * in rte_mbuf.ol_flags. The crypto_ops would be freed after the grouping. + * + * Note that application must ensure only crypto-ops prepared by lib_pdcp is + * provided back to @see rte_pdcp_pkt_crypto_group(). + * + * @param cop + * The address of an array of *num* pointers to the input *rte_crypto_op* + * structures. + * @param[out] mb + * The address of an array of *num* pointers to output *rte_mbuf* structures. + * @param[out] grp + * The address of an array of *num* to output *rte_pdcp_group* structures. + * @param num + * The maximum number of crypto-ops to process. + * @return + * Number of filled elements in *grp* array. + * + */ +static inline uint16_t +rte_pdcp_pkt_crypto_group(struct rte_crypto_op *cop[], struct rte_mbuf *mb[], + struct rte_pdcp_group grp[], uint16_t num) +{ + uint32_t i, j = 0, n = 0; + void *ns, *ps = NULL; + struct rte_mbuf *m; + + for (i = 0; i != num; i++) { + m = cop[i]->sym[0].m_src; + ns = cop[i]->sym[0].session; + + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD; + if (cop[i]->status != RTE_CRYPTO_OP_STATUS_SUCCESS) + m->ol_flags |= RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED; + + /* Different entity */ + if (ps != ns) { + + /* Finalize open group and start a new one */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + /* Start new group */ + grp[n].m = mb + j; + ps = ns; + grp[n].id.ptr = rte_pdcp_en_from_cop(cop[i]); + } + + mb[j++] = m; + rte_crypto_op_free(cop[i]); + } + + /* Finalize last group */ + if (ps != NULL) { + grp[n].cnt = mb + j - grp[n].m; + n++; + } + + return n; +} + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_PDCP_GROUP_H */ diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index f9ff30600a..d564f155e0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,12 +2,15 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_en_from_cop; + rte_pdcp_entity_establish; rte_pdcp_entity_release; rte_pdcp_entity_suspend; rte_pdcp_pkt_post_process; rte_pdcp_pkt_pre_process; + rte_pdcp_pkt_crypto_group; local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 05/21] pdcp: add crypto session create and destroy 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (3 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 04/21] pdcp: add packet group Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 06/21] pdcp: add pre and post process for UL Anoob Joseph ` (16 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to create & destroy sessions. PDCP lib would take crypto transforms as input and creates the session on the corresponding device after verifying capabilities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_crypto.c | 223 ++++++++++++++++++++++++++++++++++++++++- lib/pdcp/pdcp_crypto.h | 5 + 2 files changed, 225 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_crypto.c b/lib/pdcp/pdcp_crypto.c index 755e27ec9e..6d2a85dc7d 100644 --- a/lib/pdcp/pdcp_crypto.c +++ b/lib/pdcp/pdcp_crypto.c @@ -2,20 +2,237 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_crypto.h> +#include <rte_crypto_sym.h> +#include <rte_cryptodev.h> +#include <rte_errno.h> #include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> #include "pdcp_crypto.h" +#include "pdcp_entity.h" + +static int +pdcp_crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +pdcp_crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +pdcp_crypto_xfrm_validate(const struct rte_pdcp_entity_conf *conf, + const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm, + bool is_auth_then_cipher) +{ + uint16_t cipher_iv_len, auth_digest_len, auth_iv_len; + int ret; + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protection is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (c_xfrm == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + + /* With UPLINK, if auth is enabled, it should be before cipher */ + if (a_xfrm != NULL && !is_auth_then_cipher) + return -EINVAL; + + /* With UPLINK, cipher operation must be encrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return -EINVAL; + + /* With UPLINK, auth operation (if present) must be generate */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_GENERATE) + return -EINVAL; + + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + + /* With DOWNLINK, if auth is enabled, it should be after cipher */ + if (a_xfrm != NULL && is_auth_then_cipher) + return -EINVAL; + + /* With DOWNLINK, cipher operation must be decrypt */ + if (c_xfrm->cipher.op != RTE_CRYPTO_CIPHER_OP_DECRYPT) + return -EINVAL; + + /* With DOWNLINK, auth operation (if present) must be verify */ + if (a_xfrm != NULL && a_xfrm->auth.op != RTE_CRYPTO_AUTH_OP_VERIFY) + return -EINVAL; + + } else { + return -EINVAL; + } + + if ((c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_NULL) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_AES_CTR) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_ZUC_EEA3) && + (c_xfrm->cipher.algo != RTE_CRYPTO_CIPHER_SNOW3G_UEA2)) + return -EINVAL; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + cipher_iv_len = 0; + else + cipher_iv_len = PDCP_IV_LEN; + + if (cipher_iv_len != c_xfrm->cipher.iv.length) + return -EINVAL; + + if (a_xfrm != NULL) { + if ((a_xfrm->auth.algo != RTE_CRYPTO_AUTH_NULL) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_AES_CMAC) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_ZUC_EIA3) && + (a_xfrm->auth.algo != RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + return -EINVAL; + + /* For AUTH NULL, lib PDCP would add 4 byte 0s */ + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + auth_digest_len = 0; + else + auth_digest_len = RTE_PDCP_MAC_I_LEN; + + if (auth_digest_len != a_xfrm->auth.digest_length) + return -EINVAL; + + if ((a_xfrm->auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + auth_iv_len = PDCP_IV_LEN; + else + auth_iv_len = 0; + + if (a_xfrm->auth.iv.length != auth_iv_len) + return -EINVAL; + } + + if (!rte_cryptodev_is_valid_dev(conf->dev_id)) + return -EINVAL; + + ret = pdcp_crypto_caps_cipher_verify(conf->dev_id, c_xfrm); + if (ret) + return -ENOTSUP; + + if (a_xfrm != NULL) { + ret = pdcp_crypto_caps_auth_verify(conf->dev_id, a_xfrm); + if (ret) + return -ENOTSUP; + } + + return 0; +} int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { - RTE_SET_USED(entity); - RTE_SET_USED(conf); + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + struct entity_priv *en_priv; + bool is_auth_then_cipher; + int ret; + + if (entity == NULL || conf == NULL || conf->crypto_xfrm == NULL) + return -EINVAL; + + en_priv = entity_priv_get(entity); + + en_priv->dev_id = conf->dev_id; + + if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_CIPHER) { + c_xfrm = conf->crypto_xfrm; + a_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = false; + } else if (conf->crypto_xfrm->type == RTE_CRYPTO_SYM_XFORM_AUTH) { + a_xfrm = conf->crypto_xfrm; + c_xfrm = conf->crypto_xfrm->next; + is_auth_then_cipher = true; + } else { + return -EINVAL; + } + + ret = pdcp_crypto_xfrm_validate(conf, c_xfrm, a_xfrm, is_auth_then_cipher); + if (ret) + return ret; + + if (c_xfrm->cipher.algo == RTE_CRYPTO_CIPHER_NULL) + c_xfrm->cipher.iv.offset = 0; + else + c_xfrm->cipher.iv.offset = PDCP_IV_OFFSET; + + if (a_xfrm != NULL) { + if (a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm->auth.iv.offset = 0; + else + if (c_xfrm->cipher.iv.offset) + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET + PDCP_IV_LEN; + else + a_xfrm->auth.iv.offset = PDCP_IV_OFFSET; + } + + if (conf->sess_mpool == NULL) + return -EINVAL; + + en_priv->crypto_sess = rte_cryptodev_sym_session_create(conf->dev_id, conf->crypto_xfrm, + conf->sess_mpool); + if (en_priv->crypto_sess == NULL) { + /* rte_errno is set as positive values of error codes */ + return -rte_errno; + } + + rte_cryptodev_sym_session_opaque_data_set(en_priv->crypto_sess, (uint64_t)entity); + return 0; } void pdcp_crypto_sess_destroy(struct rte_pdcp_entity *entity) { - RTE_SET_USED(entity); + struct entity_priv *en_priv; + + en_priv = entity_priv_get(entity); + + if (en_priv->crypto_sess != NULL) { + rte_cryptodev_sym_session_free(en_priv->dev_id, en_priv->crypto_sess); + en_priv->crypto_sess = NULL; + } } diff --git a/lib/pdcp/pdcp_crypto.h b/lib/pdcp/pdcp_crypto.h index 6563331d37..f694818713 100644 --- a/lib/pdcp/pdcp_crypto.h +++ b/lib/pdcp/pdcp_crypto.h @@ -5,8 +5,13 @@ #ifndef PDCP_CRYPTO_H #define PDCP_CRYPTO_H +#include <rte_crypto.h> +#include <rte_crypto_sym.h> #include <rte_pdcp.h> +#define PDCP_IV_OFFSET (sizeof(struct rte_crypto_op) + sizeof(struct rte_crypto_sym_op)) +#define PDCP_IV_LEN 16 + int pdcp_crypto_sess_create(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 06/21] pdcp: add pre and post process for UL 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (4 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 05/21] pdcp: add crypto session create and destroy Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-06-10 22:50 ` Thomas Monjalon 2023-05-30 10:01 ` [PATCH v6 07/21] pdcp: add pre and post process for DL Anoob Joseph ` (15 subsequent siblings) 21 siblings, 1 reply; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to perform pre & post processing based on the type of entity. To avoid checks in datapath, there are different function pointers registered based on the following, 1. Control plane v/s user plane 2. 12 bit v/s 18 bit SN For control plane only 12 bit SN need to be supported (as per PDCP specification). Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> Acked-by: Akhil Goyal <gakhil@marvell.com> --- lib/pdcp/pdcp_entity.h | 24 +++ lib/pdcp/pdcp_process.c | 334 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 358 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 000297588f..23628ebad4 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -92,22 +92,46 @@ pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static inline uint32_t +pdcp_window_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return 1 << (sn_size - 1); +} + static inline uint32_t pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return (1 << sn_size) - 1; } +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + static inline uint32_t pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) { return ~pdcp_sn_mask_get(sn_size); } +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline uint32_t pdcp_count_from_hfn_sn_get(uint32_t hfn, uint32_t sn, enum rte_security_pdcp_sn_size sn_size) { return (((hfn << sn_size) & pdcp_hfn_mask_get(sn_size)) | (sn & pdcp_sn_mask_get(sn_size))); } +static inline uint32_t +pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << (32 - sn_size)) - 1; +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 79f5dce5db..9b7de39db6 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -36,6 +36,336 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static inline void +cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, + uint8_t data_offset, uint32_t count, const bool is_auth) +{ + const struct rte_crypto_op cop_init = { + .type = RTE_CRYPTO_OP_TYPE_SYMMETRIC, + .status = RTE_CRYPTO_OP_STATUS_NOT_PROCESSED, + .sess_type = RTE_CRYPTO_OP_WITH_SESSION, + }; + struct rte_crypto_sym_op *op; + uint32_t pkt_len; + + const uint8_t cipher_shift = 3 * en_priv->flags.is_cipher_in_bits; + const uint8_t auth_shift = 3 * en_priv->flags.is_auth_in_bits; + + op = cop->sym; + cop->raw = cop_init.raw; + op->m_src = mb; + op->m_dst = mb; + + /* Set IV */ + en_priv->iv_gen(cop, en_priv, count); + + /* Prepare op */ + pkt_len = rte_pktmbuf_pkt_len(mb); + op->cipher.data.offset = data_offset << cipher_shift; + op->cipher.data.length = (pkt_len - data_offset) << cipher_shift; + + if (is_auth) { + op->auth.data.offset = 0; + op->auth.data.length = (pkt_len - RTE_PDCP_MAC_I_LEN) << auth_shift; + op->auth.digest.data = rte_pktmbuf_mtod_offset(mb, uint8_t *, + (pkt_len - RTE_PDCP_MAC_I_LEN)); + } + + __rte_crypto_sym_op_attach_sym_session(op, en_priv->crypto_sess); +} + +static inline bool +pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + return true; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_12_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline bool +pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_mbuf *mb, + uint32_t *count) +{ + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + const uint8_t hdr_sz = en_priv->hdr_sz; + uint32_t sn; + + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_up_data_pdu_sn_18_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) + return false; + + /* Update sequence num in the PDU header */ + *count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); + + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + return true; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count; + uint8_t *mac_i; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + if (en_priv->flags.is_authenticated) { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + } else { + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + if (unlikely(!pdcp_pre_process_uplane_sn_18_ul_set_sn(en_priv, mb, + &count))) { + + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, false); + } + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + struct rte_mbuf *mb; + uint32_t count, sn; + uint8_t *mac_i; + int i; + + const uint8_t hdr_sz = en_priv->hdr_sz; + const uint8_t data_offset = hdr_sz + en_priv->aad_sz; + const int is_null_auth = en_priv->flags.is_null_auth; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + /* Prepend PDU header */ + pdu_hdr = (struct rte_pdcp_cp_data_pdu_sn_12_hdr *)rte_pktmbuf_prepend(mb, hdr_sz); + if (unlikely(pdu_hdr == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + mac_i = (uint8_t *)rte_pktmbuf_append(mb, RTE_PDCP_MAC_I_LEN); + if (unlikely(mac_i == NULL)) { + in_mb[nb_err++] = mb; + continue; + } + + /* Clear MAC-I field for NULL auth */ + if (is_null_auth) + memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); + + /* Update sequence number in the PDU header */ + count = en_priv->state.tx_next++; + sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); + + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + pdu_hdr->r = 0; + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t hdr_trim_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + struct rte_mbuf *mb, *err_mb[num]; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { + err_mb[nb_err++] = mb; + continue; + } + + if (hdr_trim_sz) + rte_pktmbuf_adj(mb, hdr_trim_sz); + + out_mb[nb_success++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static int +pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + entity->pre_process = NULL; + entity->post_process = NULL; + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_ul; + entity->post_process = pdcp_post_process_ul; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_ul; + entity->post_process = pdcp_post_process_ul; + } + + if (entity->pre_process == NULL || entity->post_process == NULL) + return -ENOTSUP; + + return 0; +} + static int pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_entity_conf *conf) { @@ -134,5 +464,9 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti if (ret) return ret; + ret = pdcp_pre_post_func_set(entity, conf); + if (ret) + return ret; + return 0; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* Re: [PATCH v6 06/21] pdcp: add pre and post process for UL 2023-05-30 10:01 ` [PATCH v6 06/21] pdcp: add pre and post process for UL Anoob Joseph @ 2023-06-10 22:50 ` Thomas Monjalon 2023-06-12 5:19 ` [EXT] " Anoob Joseph 0 siblings, 1 reply; 192+ messages in thread From: Thomas Monjalon @ 2023-06-10 22:50 UTC (permalink / raw) To: Akhil Goyal, Kiran Kumar K, Volodymyr Fialko, Anoob Joseph Cc: Jerin Jacob, Konstantin Ananyev, dev, Hemant Agrawal, Mattias Rönnblom, dev, Olivier Matz, Stephen Hemminger, david.marchand Hello, I'm sorry to inform you that I see a compilation failure when cross-compiling for PPC64: err_mb is seen as potentially unitialized when calling rte_memcpy(). 30/05/2023 12:01, Anoob Joseph: > +static uint16_t > +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, > + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], > + uint16_t num, uint16_t *nb_err_ret) > +{ > + struct entity_priv *en_priv = entity_priv_get(entity); > + const uint32_t hdr_trim_sz = en_priv->aad_sz; > + int i, nb_success = 0, nb_err = 0; > + struct rte_mbuf *mb, *err_mb[num]; > + I need to add a workaround here to make GCC happy: #ifdef RTE_ARCH_PPC_64 err_mb[0] = NULL; /* workaround PPC-GCC bug */ #endif > + for (i = 0; i < num; i++) { > + mb = in_mb[i]; > + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { > + err_mb[nb_err++] = mb; > + continue; > + } > + > + if (hdr_trim_sz) > + rte_pktmbuf_adj(mb, hdr_trim_sz); > + > + out_mb[nb_success++] = mb; > + } > + > + if (unlikely(nb_err != 0)) > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); > + > + *nb_err_ret = nb_err; > + return nb_success; > +} I've added the workaround in 3 places while pulling next-crypto tree. Feel free to improve it with a new patch, thanks. ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [EXT] Re: [PATCH v6 06/21] pdcp: add pre and post process for UL 2023-06-10 22:50 ` Thomas Monjalon @ 2023-06-12 5:19 ` Anoob Joseph 0 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-06-12 5:19 UTC (permalink / raw) To: Thomas Monjalon Cc: Jerin Jacob Kollanukkaran, Konstantin Ananyev, dev, Hemant Agrawal, Mattias Rönnblom, dev, Olivier Matz, Stephen Hemminger, david.marchand, Akhil Goyal, Kiran Kumar Kokkilagadda, Volodymyr Fialko Hi Thomas, Thanks for the fix. Will check if we can have it addressed properly. Thanks, Anoob > -----Original Message----- > From: Thomas Monjalon <thomas@monjalon.net> > Sent: Sunday, June 11, 2023 4:21 AM > To: Akhil Goyal <gakhil@marvell.com>; Kiran Kumar Kokkilagadda > <kirankumark@marvell.com>; Volodymyr Fialko <vfialko@marvell.com>; > Anoob Joseph <anoobj@marvell.com> > Cc: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Konstantin Ananyev > <konstantin.v.ananyev@yandex.ru>; dev@dpdk.org; Hemant Agrawal > <hemant.agrawal@nxp.com>; Mattias Rönnblom > <mattias.ronnblom@ericsson.com>; dev@dpdk.org; Olivier Matz > <olivier.matz@6wind.com>; Stephen Hemminger > <stephen@networkplumber.org>; david.marchand@redhat.com > Subject: [EXT] Re: [PATCH v6 06/21] pdcp: add pre and post process for UL > > External Email > > ---------------------------------------------------------------------- > Hello, > > I'm sorry to inform you that I see a compilation failure when cross-compiling > for PPC64: > err_mb is seen as potentially unitialized when calling rte_memcpy(). > > > 30/05/2023 12:01, Anoob Joseph: > > +static uint16_t > > +pdcp_post_process_ul(const struct rte_pdcp_entity *entity, > > + struct rte_mbuf *in_mb[], struct rte_mbuf *out_mb[], > > + uint16_t num, uint16_t *nb_err_ret) { > > + struct entity_priv *en_priv = entity_priv_get(entity); > > + const uint32_t hdr_trim_sz = en_priv->aad_sz; > > + int i, nb_success = 0, nb_err = 0; > > + struct rte_mbuf *mb, *err_mb[num]; > > + > > I need to add a workaround here to make GCC happy: > #ifdef RTE_ARCH_PPC_64 > err_mb[0] = NULL; /* workaround PPC-GCC bug */ #endif > > > + for (i = 0; i < num; i++) { > > + mb = in_mb[i]; > > + if (unlikely(mb->ol_flags & > RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) { > > + err_mb[nb_err++] = mb; > > + continue; > > + } > > + > > + if (hdr_trim_sz) > > + rte_pktmbuf_adj(mb, hdr_trim_sz); > > + > > + out_mb[nb_success++] = mb; > > + } > > + > > + if (unlikely(nb_err != 0)) > > + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * > sizeof(struct > > +rte_mbuf *)); > > + > > + *nb_err_ret = nb_err; > > + return nb_success; > > +} > > I've added the workaround in 3 places while pulling next-crypto tree. > Feel free to improve it with a new patch, thanks. > ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 07/21] pdcp: add pre and post process for DL 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (5 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 06/21] pdcp: add pre and post process for UL Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 08/21] pdcp: add IV generation routines Anoob Joseph ` (14 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add routines to perform pre & post processing for down link entities. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 384 ++++++++++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.h | 11 ++ lib/pdcp/rte_pdcp.c | 23 +++ 4 files changed, 420 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 23628ebad4..1d4a43a3bc 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -13,6 +13,8 @@ struct entity_priv; +#define PDCP_HFN_MIN 0 + /* IV generation function based on the entity configuration */ typedef void (*iv_gen_t)(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count); diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 9b7de39db6..bd75e6f802 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -6,6 +6,7 @@ #include <rte_crypto_sym.h> #include <rte_cryptodev.h> #include <rte_memcpy.h> +#include <rte_mbuf_dyn.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> @@ -333,9 +334,353 @@ pdcp_post_process_ul(const struct rte_pdcp_entity *entity, return nb_success; } +static inline int +pdcp_sn_count_get(const uint32_t rx_deliv, int32_t rsn, uint32_t *count, + const enum rte_security_pdcp_sn_size sn_size) +{ + const uint32_t rx_deliv_sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + const uint32_t window_sz = pdcp_window_size_get(sn_size); + uint32_t rhfn; + + rhfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + + if (rsn < (int32_t)(rx_deliv_sn - window_sz)) { + if (unlikely(rhfn == pdcp_hfn_max(sn_size))) + return -ERANGE; + rhfn += 1; + } else if ((uint32_t)rsn >= (rx_deliv_sn + window_sz)) { + if (unlikely(rhfn == PDCP_HFN_MIN)) + return -ERANGE; + rhfn -= 1; + } + + *count = pdcp_count_from_hfn_sn_get(rhfn, rsn, sn_size); + + return 0; +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_12_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_12_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static inline uint16_t +pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], struct rte_crypto_op *cop[], + uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + int32_t rsn = 0; + uint32_t count; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_up_data_pdu_sn_18_hdr *); + + /* Check for PDU type */ + if (likely(pdu_hdr->d_c == RTE_PDCP_PDU_TYPE_DATA)) { + rsn = ((pdu_hdr->sn_17_16 << 16) | (pdu_hdr->sn_15_8 << 8) | + (pdu_hdr->sn_7_0)); + } else { + /** NOTE: Control PDU not handled.*/ + in_mb[nb_err++] = mb; + continue; + } + + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_18))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, is_integ_protected); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, true); +} + +static uint16_t +pdcp_pre_process_uplane_sn_18_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_pre_process_uplane_sn_18_dl_flags(entity, mb, cop, num, nb_err, false); +} + +static uint16_t +pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_crypto_op *cop[], uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + struct rte_pdcp_cp_data_pdu_sn_12_hdr *pdu_hdr; + uint16_t nb_cop, nb_prep = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *mb; + uint32_t count; + int32_t rsn; + int i; + + const uint8_t data_offset = en_priv->hdr_sz + en_priv->aad_sz; + + nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, + num); + + const uint32_t rx_deliv = en_priv->state.rx_deliv; + + for (i = 0; i < nb_cop; i++) { + mb = in_mb[i]; + pdu_hdr = rte_pktmbuf_mtod(mb, struct rte_pdcp_cp_data_pdu_sn_12_hdr *); + rsn = ((pdu_hdr->sn_11_8 << 8) | (pdu_hdr->sn_7_0)); + if (unlikely(pdcp_sn_count_get(rx_deliv, rsn, &count, + RTE_SECURITY_PDCP_SN_SIZE_12))) { + in_mb[nb_err++] = mb; + continue; + } + + cop_prepare(en_priv, mb, cop[nb_prep++], data_offset, count, true); + + mb_dynfield = pdcp_dynfield(mb); + *mb_dynfield = count; + } + + if (unlikely(nb_err)) + /* Using mempool API since crypto API is not providing bulk free */ + rte_mempool_put_bulk(en_priv->cop_pool, (void *)&cop[nb_prep], nb_cop - nb_prep); + + *nb_err_ret = num - nb_prep; + + return nb_prep; +} + +static inline void +pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool trim_mac) +{ + char *p = rte_pktmbuf_adj(mb, hdr_trim_sz); + RTE_ASSERT(p != NULL); + RTE_SET_USED(p); + + if (trim_mac) { + int ret = rte_pktmbuf_trim(mb, RTE_PDCP_MAC_I_LEN); + RTE_ASSERT(ret == 0); + RTE_SET_USED(ret); + } +} + +static inline bool +pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, + const uint32_t count) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + + if (count < en_priv->state.rx_deliv) + return false; + + /* t-Reordering timer is not supported - SDU will be delivered immediately. + * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not + * been delivered to upper layers + */ + en_priv->state.rx_next = count + 1; + + if (count >= en_priv->state.rx_next) + en_priv->state.rx_next = count + 1; + + return true; +} + +static inline uint16_t +pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret, + const bool is_integ_protected) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + mb_dynfield = pdcp_dynfield(mb); + count = *mb_dynfield; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + +static uint16_t +pdcp_post_process_uplane_dl_ip(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_dl_flags(entity, in_mb, out_mb, num, nb_err, true); +} + +static uint16_t +pdcp_post_process_uplane_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err) +{ + return pdcp_post_process_uplane_dl_flags(entity, in_mb, out_mb, num, nb_err, false); +} + +static uint16_t +pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, + struct rte_mbuf *in_mb[], + struct rte_mbuf *out_mb[], + uint16_t num, uint16_t *nb_err_ret) +{ + struct entity_priv *en_priv = entity_priv_get(entity); + const uint32_t aad_sz = en_priv->aad_sz; + int i, nb_success = 0, nb_err = 0; + rte_pdcp_dynfield_t *mb_dynfield; + struct rte_mbuf *err_mb[num]; + struct rte_mbuf *mb; + uint32_t count; + + const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; + + for (i = 0; i < num; i++) { + mb = in_mb[i]; + if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) + goto error; + + mb_dynfield = pdcp_dynfield(mb); + count = *mb_dynfield; + + if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + goto error; + + pdcp_packet_strip(mb, hdr_trim_sz, true); + + out_mb[nb_success++] = mb; + continue; + +error: + err_mb[nb_err++] = mb; + } + + if (unlikely(nb_err != 0)) + rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); + + *nb_err_ret = nb_err; + return nb_success; +} + static int pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) { + struct entity_priv *en_priv = entity_priv_get(entity); + entity->pre_process = NULL; entity->post_process = NULL; @@ -346,6 +691,13 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_CONTROL) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK)) { + entity->pre_process = pdcp_pre_process_cplane_sn_12_dl; + entity->post_process = pdcp_post_process_cplane_sn_12_dl; + } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK)) { @@ -360,6 +712,38 @@ pdcp_pre_post_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_ent entity->post_process = pdcp_post_process_ul; } + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl_ip; + entity->post_process = pdcp_post_process_uplane_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_12) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_12_dl; + entity->post_process = pdcp_post_process_uplane_dl; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl_ip; + entity->post_process = pdcp_post_process_uplane_dl_ip; + } + + if ((conf->pdcp_xfrm.domain == RTE_SECURITY_PDCP_MODE_DATA) && + (conf->pdcp_xfrm.sn_size == RTE_SECURITY_PDCP_SN_SIZE_18) && + (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) && + (!en_priv->flags.is_authenticated)) { + entity->pre_process = pdcp_pre_process_uplane_sn_18_dl; + entity->post_process = pdcp_post_process_uplane_dl; + } + if (entity->pre_process == NULL || entity->post_process == NULL) return -ENOTSUP; diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index fd53fff0aa..a52f769b82 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -5,8 +5,19 @@ #ifndef PDCP_PROCESS_H #define PDCP_PROCESS_H +#include <rte_mbuf_dyn.h> #include <rte_pdcp.h> +typedef uint32_t rte_pdcp_dynfield_t; + +extern int rte_pdcp_dynfield_offset; + +static inline rte_pdcp_dynfield_t * +pdcp_dynfield(struct rte_mbuf *mbuf) +{ + return RTE_MBUF_DYNFIELD(mbuf, rte_pdcp_dynfield_offset, rte_pdcp_dynfield_t *); +} + int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index adcad5dd25..91dab91f73 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -10,6 +10,26 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +#define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" + +int rte_pdcp_dynfield_offset = -1; + +static int +pdcp_dynfield_register(void) +{ + const struct rte_mbuf_dynfield dynfield_desc = { + .name = RTE_PDCP_DYNFIELD_NAME, + .size = sizeof(rte_pdcp_dynfield_t), + .align = __alignof__(rte_pdcp_dynfield_t), + }; + + if (rte_pdcp_dynfield_offset != -1) + return rte_pdcp_dynfield_offset; + + rte_pdcp_dynfield_offset = rte_mbuf_dynfield_register(&dynfield_desc); + return rte_pdcp_dynfield_offset; +} + static int pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) { @@ -35,6 +55,9 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) int ret, entity_size; uint32_t count; + if (pdcp_dynfield_register() < 0) + return NULL; + if (conf == NULL || conf->cop_pool == NULL) { rte_errno = EINVAL; return NULL; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 08/21] pdcp: add IV generation routines 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (6 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 07/21] pdcp: add pre and post process for DL Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 09/21] app/test: add lib pdcp tests Anoob Joseph ` (13 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger For PDCP, IV generated has varying formats depending on the ciphering and authentication algorithm used. Add routines to populate IV accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 87 ++++++++++++ lib/pdcp/pdcp_process.c | 284 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 371 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 1d4a43a3bc..10a72faae1 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -26,6 +26,89 @@ struct entity_state { uint32_t rx_reord; }; +union auth_iv_partial { + /* For AES-CMAC, there is no IV, but message gets prepended */ + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + } aes_cmac; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_37_39 : 3; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; + + uint64_t rsvd_65_71 : 7; + uint64_t direction_64 : 1; + uint64_t rsvd_72_111 : 40; + uint64_t rsvd_113_119 : 7; + uint64_t direction_112 : 1; + uint64_t rsvd_120_127 : 8; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t zero_37_39 : 3; + uint64_t zero_40_63 : 24; + + uint64_t direction_64 : 1; + uint64_t rsvd_65_71 : 7; + uint64_t rsvd_72_111 : 40; + uint64_t direction_112 : 1; + uint64_t rsvd_113_119 : 7; + uint64_t rsvd_120_127 : 8; +#endif + } zs; + uint64_t u64[2]; +}; + +union cipher_iv_partial { + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t zero_64_127; + } aes_ctr; + struct { +#if RTE_BYTE_ORDER == RTE_LITTLE_ENDIAN + uint64_t count : 32; + uint64_t zero_38_39 : 2; + uint64_t direction : 1; + uint64_t bearer : 5; + uint64_t zero_40_63 : 24; +#else + uint64_t count : 32; + uint64_t bearer : 5; + uint64_t direction : 1; + uint64_t zero_38_39 : 2; + uint64_t zero_40_63 : 24; +#endif + uint64_t rsvd_64_127; + } zs; + uint64_t u64[2]; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -35,6 +118,10 @@ struct entity_priv { struct rte_cryptodev_sym_session *crypto_sess; /** Entity specific IV generation function. */ iv_gen_t iv_gen; + /** Pre-prepared auth IV. */ + union auth_iv_partial auth_iv_part; + /** Pre-prepared cipher IV. */ + union cipher_iv_partial cipher_iv_part; /** Entity state variables. */ struct entity_state state; /** Flags. */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index bd75e6f802..28ac4102da 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -14,6 +14,181 @@ #include "pdcp_entity.h" #include "pdcp_process.h" +/* Enum of supported algorithms for ciphering */ +enum pdcp_cipher_algo { + PDCP_CIPHER_ALGO_NULL, + PDCP_CIPHER_ALGO_AES, + PDCP_CIPHER_ALGO_ZUC, + PDCP_CIPHER_ALGO_SNOW3G, + PDCP_CIPHER_ALGO_MAX +}; + +/* Enum of supported algorithms for integrity */ +enum pdcp_auth_algo { + PDCP_AUTH_ALGO_NULL, + PDCP_AUTH_ALGO_AES, + PDCP_AUTH_ALGO_ZUC, + PDCP_AUTH_ALGO_SNOW3G, + PDCP_AUTH_ALGO_MAX +}; + +/* IV generation functions based on type of operation (cipher - auth) */ + +static void +pdcp_iv_gen_null_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + /* No IV required for NULL cipher + NULL auth */ + RTE_SET_USED(cop); + RTE_SET_USED(en_priv); + RTE_SET_USED(count); +} + +static void +pdcp_iv_gen_null_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr; + uint64_t m; + + /* AES-CMAC requires message to be prepended with info on count etc */ + + /* Prepend by 8 bytes to add custom message */ + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_null_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_aes_ctr_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, 16); +} + +static void +pdcp_iv_gen_zs_null(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64 = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64, 8); + rte_memcpy(iv + 8, &iv_u64, 8); +} + +static void +pdcp_iv_gen_zs_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + /* Generating cipher IV */ + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + iv += PDCP_IV_LEN; + + /* Generating auth IV */ + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + +static void +pdcp_iv_gen_zs_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + rte_memcpy(iv + 8, &iv_u64[0], 8); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_aes_cmac(struct rte_crypto_op *cop, const struct entity_priv *en_priv, + uint32_t count) +{ + struct rte_crypto_sym_op *op = cop->sym; + struct rte_mbuf *mb = op->m_src; + uint8_t *m_ptr, *iv; + uint64_t iv_u64[2]; + uint64_t m; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + m_ptr = (uint8_t *)rte_pktmbuf_prepend(mb, 8); + m = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(m_ptr, &m, 8); +} + +static void +pdcp_iv_gen_aes_ctr_zs(struct rte_crypto_op *cop, const struct entity_priv *en_priv, uint32_t count) +{ + uint64_t iv_u64[2]; + uint8_t *iv; + + iv = rte_crypto_op_ctod_offset(cop, uint8_t *, PDCP_IV_OFFSET); + + iv_u64[0] = en_priv->cipher_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + iv_u64[1] = 0; + rte_memcpy(iv, iv_u64, PDCP_IV_LEN); + + iv += PDCP_IV_LEN; + + iv_u64[0] = en_priv->auth_iv_part.u64[0] | ((uint64_t)(rte_cpu_to_be_32(count))); + rte_memcpy(iv, &iv_u64[0], 8); + + iv_u64[1] = iv_u64[0] ^ en_priv->auth_iv_part.u64[1]; + rte_memcpy(iv + 8, &iv_u64[1], 8); +} + static int pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_sym_xform **c_xfrm, struct rte_crypto_sym_xform **a_xfrm) @@ -37,6 +212,111 @@ pdcp_crypto_xfrm_get(const struct rte_pdcp_entity_conf *conf, struct rte_crypto_ return 0; } +static int +pdcp_iv_gen_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + struct rte_crypto_sym_xform *c_xfrm, *a_xfrm; + enum rte_security_pdcp_direction direction; + enum pdcp_cipher_algo cipher_algo; + enum pdcp_auth_algo auth_algo; + struct entity_priv *en_priv; + int ret; + + en_priv = entity_priv_get(entity); + + direction = conf->pdcp_xfrm.pkt_dir; + if (conf->reverse_iv_direction) + direction = !direction; + + ret = pdcp_crypto_xfrm_get(conf, &c_xfrm, &a_xfrm); + if (ret) + return ret; + + if (c_xfrm == NULL) + return -EINVAL; + + memset(&en_priv->auth_iv_part, 0, sizeof(en_priv->auth_iv_part)); + memset(&en_priv->cipher_iv_part, 0, sizeof(en_priv->cipher_iv_part)); + + switch (c_xfrm->cipher.algo) { + case RTE_CRYPTO_CIPHER_NULL: + cipher_algo = PDCP_CIPHER_ALGO_NULL; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + cipher_algo = PDCP_CIPHER_ALGO_AES; + en_priv->cipher_iv_part.aes_ctr.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.aes_ctr.direction = direction; + break; + case RTE_CRYPTO_CIPHER_SNOW3G_UEA2: + cipher_algo = PDCP_CIPHER_ALGO_SNOW3G; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + case RTE_CRYPTO_CIPHER_ZUC_EEA3: + cipher_algo = PDCP_CIPHER_ALGO_ZUC; + en_priv->cipher_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->cipher_iv_part.zs.direction = direction; + break; + default: + return -ENOTSUP; + } + + if (a_xfrm != NULL) { + switch (a_xfrm->auth.algo) { + case RTE_CRYPTO_AUTH_NULL: + auth_algo = PDCP_AUTH_ALGO_NULL; + break; + case RTE_CRYPTO_AUTH_AES_CMAC: + auth_algo = PDCP_AUTH_ALGO_AES; + en_priv->auth_iv_part.aes_cmac.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.aes_cmac.direction = direction; + break; + case RTE_CRYPTO_AUTH_SNOW3G_UIA2: + auth_algo = PDCP_AUTH_ALGO_SNOW3G; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + case RTE_CRYPTO_AUTH_ZUC_EIA3: + auth_algo = PDCP_AUTH_ALGO_ZUC; + en_priv->auth_iv_part.zs.bearer = conf->pdcp_xfrm.bearer; + en_priv->auth_iv_part.zs.direction_64 = direction; + en_priv->auth_iv_part.zs.direction_112 = direction; + break; + default: + return -ENOTSUP; + } + } else { + auth_algo = PDCP_AUTH_ALGO_NULL; + } + + static const iv_gen_t iv_gen_map[PDCP_CIPHER_ALGO_MAX][PDCP_AUTH_ALGO_MAX] = { + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_null_null, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_null_aes_cmac, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_null_zs, + [PDCP_CIPHER_ALGO_NULL][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_null_zs, + + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_aes_ctr_null, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_aes_ctr_aes_cmac, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_aes_ctr_zs, + [PDCP_CIPHER_ALGO_AES][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_aes_ctr_zs, + + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_SNOW3G][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_NULL] = pdcp_iv_gen_zs_null, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_AES] = pdcp_iv_gen_zs_aes_cmac, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_SNOW3G] = pdcp_iv_gen_zs_zs, + [PDCP_CIPHER_ALGO_ZUC][PDCP_AUTH_ALGO_ZUC] = pdcp_iv_gen_zs_zs, + }; + + en_priv->iv_gen = iv_gen_map[cipher_algo][auth_algo]; + + return 0; +} + static inline void cop_prepare(const struct entity_priv *en_priv, struct rte_mbuf *mb, struct rte_crypto_op *cop, uint8_t data_offset, uint32_t count, const bool is_auth) @@ -844,6 +1124,10 @@ pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_enti en_priv = entity_priv_get(entity); + ret = pdcp_iv_gen_func_set(entity, conf); + if (ret) + return ret; + ret = pdcp_entity_priv_populate(en_priv, conf); if (ret) return ret; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 09/21] app/test: add lib pdcp tests 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (7 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 08/21] pdcp: add IV generation routines Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph ` (12 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add tests to verify lib PDCP operations. Tests leverage existing PDCP test vectors. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 1 + app/test/meson.build | 5 + app/test/test_pdcp.c | 732 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 738 insertions(+) create mode 100644 app/test/test_pdcp.c diff --git a/MAINTAINERS b/MAINTAINERS index 82f490c5c0..ca684dde83 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1556,6 +1556,7 @@ M: Anoob Joseph <anoobj@marvell.com> M: Volodymyr Fialko <vfialko@marvell.com> T: git://dpdk.org/next/dpdk-next-crypto F: lib/pdcp/ +F: app/test/test_pdcp* Packet Framework diff --git a/app/test/meson.build b/app/test/meson.build index d96ae7a961..8eab3ea8b2 100644 --- a/app/test/meson.build +++ b/app/test/meson.build @@ -435,6 +435,11 @@ if dpdk_conf.has('RTE_HAS_LIBPCAP') endif endif +if dpdk_conf.has('RTE_LIB_PDCP') + test_sources += 'test_pdcp.c' + fast_tests += [['pdcp_autotest', false, true]] +endif + if cc.has_argument('-Wno-format-truncation') cflags += '-Wno-format-truncation' endif diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c new file mode 100644 index 0000000000..34b759eaef --- /dev/null +++ b/app/test/test_pdcp.c @@ -0,0 +1,732 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_malloc.h> +#include <rte_pdcp.h> +#include <rte_pdcp_hdr.h> + +#include "test.h" +#include "test_cryptodev.h" +#include "test_cryptodev_security_pdcp_test_vectors.h" + +#define NB_DESC 1024 +#define CDEV_INVALID_ID UINT8_MAX +#define NB_TESTS RTE_DIM(pdcp_test_params) +#define PDCP_IV_LEN 16 + +struct pdcp_testsuite_params { + struct rte_mempool *mbuf_pool; + struct rte_mempool *cop_pool; + struct rte_mempool *sess_pool; + bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; +}; + +static struct pdcp_testsuite_params testsuite_params; + +struct pdcp_test_conf { + struct rte_pdcp_entity_conf entity; + struct rte_crypto_sym_xform c_xfrm; + struct rte_crypto_sym_xform a_xfrm; + bool is_integrity_protected; + uint8_t input[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t input_len; + uint8_t output[RTE_PDCP_CTRL_PDU_SIZE_MAX]; + uint32_t output_len; +}; + +static inline int +pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) +{ + return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; +} + +static int +cryptodev_init(int dev_id) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_cryptodev_qp_conf qp_conf; + struct rte_cryptodev_info dev_info; + struct rte_cryptodev_config config; + int ret, socket_id; + + /* Check if device was already initialized */ + if (ts_params->cdevs_used[dev_id]) + return 0; + + rte_cryptodev_info_get(dev_id, &dev_info); + + if (dev_info.max_nb_queue_pairs < 1) { + RTE_LOG(ERR, USER1, "Cryptodev doesn't have sufficient queue pairs available\n"); + return -ENODEV; + } + + socket_id = rte_socket_id(); + + memset(&config, 0, sizeof(config)); + config.nb_queue_pairs = 1; + config.socket_id = socket_id; + + ret = rte_cryptodev_configure(dev_id, &config); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure cryptodev - %d\n", dev_id); + return -ENODEV; + } + + memset(&qp_conf, 0, sizeof(qp_conf)); + qp_conf.nb_descriptors = NB_DESC; + + ret = rte_cryptodev_queue_pair_setup(dev_id, 0, &qp_conf, socket_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not configure queue pair\n"); + return -ENODEV; + } + + ret = rte_cryptodev_start(dev_id); + if (ret < 0) { + RTE_LOG(ERR, USER1, "Could not start cryptodev\n"); + return -ENODEV; + } + + /* Mark device as initialized */ + ts_params->cdevs_used[dev_id] = true; + + return 0; +} + +static void +cryptodev_fini(int dev_id) +{ + rte_cryptodev_stop(dev_id); +} + +static unsigned int +cryptodev_sess_priv_max_req_get(void) +{ + struct rte_cryptodev_info info; + unsigned int sess_priv_sz; + int i, nb_dev; + void *sec_ctx; + + nb_dev = rte_cryptodev_count(); + + sess_priv_sz = 0; + + for (i = 0; i < nb_dev; i++) { + rte_cryptodev_info_get(i, &info); + sess_priv_sz = RTE_MAX(sess_priv_sz, rte_cryptodev_sym_get_private_session_size(i)); + if (info.feature_flags & RTE_CRYPTODEV_FF_SECURITY) { + sec_ctx = rte_cryptodev_get_sec_ctx(i); + sess_priv_sz = RTE_MAX(sess_priv_sz, + rte_security_session_get_size(sec_ctx)); + } + } + + return sess_priv_sz; +} + +static int +testsuite_setup(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + int nb_cdev, sess_priv_size, nb_sess = 1024; + + RTE_SET_USED(pdcp_test_hfn_threshold); + + nb_cdev = rte_cryptodev_count(); + if (nb_cdev < 1) { + RTE_LOG(ERR, USER1, "No crypto devices found.\n"); + return TEST_SKIPPED; + } + + memset(ts_params, 0, sizeof(*ts_params)); + + ts_params->mbuf_pool = rte_pktmbuf_pool_create("mbuf_pool", NUM_MBUFS, MBUF_CACHE_SIZE, 0, + MBUF_SIZE, SOCKET_ID_ANY); + if (ts_params->mbuf_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf pool\n"); + return TEST_FAILED; + } + + ts_params->cop_pool = rte_crypto_op_pool_create("cop_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, + NUM_MBUFS, MBUF_CACHE_SIZE, + 2 * MAXIMUM_IV_LENGTH, SOCKET_ID_ANY); + if (ts_params->cop_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create crypto_op pool\n"); + goto mbuf_pool_free; + } + + /* Get max session priv size required */ + sess_priv_size = cryptodev_sess_priv_max_req_get(); + + ts_params->sess_pool = rte_cryptodev_sym_session_pool_create("sess_pool", nb_sess, + sess_priv_size, + RTE_MEMPOOL_CACHE_MAX_SIZE, + 0, SOCKET_ID_ANY); + if (ts_params->sess_pool == NULL) { + RTE_LOG(ERR, USER1, "Could not create session pool\n"); + goto cop_pool_free; + } + + return 0; + +cop_pool_free: + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; +mbuf_pool_free: + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; + return TEST_FAILED; +} + +static void +testsuite_teardown(void) +{ + struct pdcp_testsuite_params *ts_params = &testsuite_params; + uint8_t dev_id; + + for (dev_id = 0; dev_id < RTE_CRYPTO_MAX_DEVS; dev_id++) { + if (ts_params->cdevs_used[dev_id]) + cryptodev_fini(dev_id); + } + + rte_mempool_free(ts_params->sess_pool); + ts_params->sess_pool = NULL; + + rte_mempool_free(ts_params->cop_pool); + ts_params->cop_pool = NULL; + + rte_mempool_free(ts_params->mbuf_pool); + ts_params->mbuf_pool = NULL; +} + +static int +ut_setup_pdcp(void) +{ + return 0; +} + +static void +ut_teardown_pdcp(void) +{ +} + +static int +crypto_caps_cipher_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *c_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + cap_idx.algo.cipher = c_xfrm->cipher.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_cipher(cap, c_xfrm->cipher.key.length, + c_xfrm->cipher.iv.length); + + return ret; +} + +static int +crypto_caps_auth_verify(uint8_t dev_id, const struct rte_crypto_sym_xform *a_xfrm) +{ + const struct rte_cryptodev_symmetric_capability *cap; + struct rte_cryptodev_sym_capability_idx cap_idx; + int ret; + + cap_idx.type = RTE_CRYPTO_SYM_XFORM_AUTH; + cap_idx.algo.auth = a_xfrm->auth.algo; + + cap = rte_cryptodev_sym_capability_get(dev_id, &cap_idx); + if (cap == NULL) + return -1; + + ret = rte_cryptodev_sym_capability_check_auth(cap, a_xfrm->auth.key.length, + a_xfrm->auth.digest_length, + a_xfrm->auth.iv.length); + + return ret; +} + +static int +cryptodev_id_get(bool is_integrity_protected, const struct rte_crypto_sym_xform *c_xfrm, + const struct rte_crypto_sym_xform *a_xfrm) +{ + int i, nb_devs; + + nb_devs = rte_cryptodev_count(); + + /* Check capabilities */ + + for (i = 0; i < nb_devs; i++) { + if ((crypto_caps_cipher_verify(i, c_xfrm) == 0) && + (!is_integrity_protected || crypto_caps_auth_verify(i, a_xfrm) == 0)) + break; + } + + if (i == nb_devs) + return -1; + + return i; +} + +static int +pdcp_known_vec_verify(struct rte_mbuf *m, const uint8_t *expected, uint32_t expected_pkt_len) +{ + uint8_t *actual = rte_pktmbuf_mtod(m, uint8_t *); + uint32_t actual_pkt_len = rte_pktmbuf_pkt_len(m); + + debug_hexdump(stdout, "Received:", actual, actual_pkt_len); + debug_hexdump(stdout, "Expected:", expected, expected_pkt_len); + + TEST_ASSERT_EQUAL(actual_pkt_len, expected_pkt_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_pkt_len, actual_pkt_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(actual, expected, expected_pkt_len, + "Generated packet not as expected"); + + return 0; +} + +static struct rte_crypto_op * +process_crypto_request(uint8_t dev_id, struct rte_crypto_op *op) +{ + if (rte_cryptodev_enqueue_burst(dev_id, 0, &op, 1) != 1) { + RTE_LOG(ERR, USER1, "Error sending packet to cryptodev\n"); + return NULL; + } + + op = NULL; + + while (rte_cryptodev_dequeue_burst(dev_id, 0, &op, 1) == 0) + rte_pause(); + + return op; +} + +static uint32_t +pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) +{ + uint32_t sn = 0; + + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + sn = rte_cpu_to_be_16(*(const uint16_t *)data); + sn = sn & 0xfff; + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + sn = rte_cpu_to_be_32(*(const uint32_t *)data); + sn = (sn & 0x3ffff00) >> 8; + } + + return sn; +} + +static int +create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_crypto_sym_xform c_xfrm, a_xfrm; + uint32_t sn, expected_len; + uint8_t *data, *expected; + int pdcp_hdr_sz; + + memset(conf, 0, sizeof(*conf)); + memset(&c_xfrm, 0, sizeof(c_xfrm)); + memset(&a_xfrm, 0, sizeof(a_xfrm)); + + conf->entity.sess_mpool = ts_params->sess_pool; + conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; + conf->entity.pdcp_xfrm.en_ordering = 0; + conf->entity.pdcp_xfrm.remove_duplicates = 0; + conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + + if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; + else + conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + + conf->entity.pdcp_xfrm.sn_size = pdcp_test_data_sn_size[index]; + + /* Zero initialize unsupported flags */ + conf->entity.pdcp_xfrm.hfn_threshold = 0; + conf->entity.pdcp_xfrm.hfn_ovrd = 0; + conf->entity.pdcp_xfrm.sdap_enabled = 0; + + c_xfrm.type = RTE_CRYPTO_SYM_XFORM_CIPHER; + c_xfrm.cipher.algo = pdcp_test_params[index].cipher_alg; + c_xfrm.cipher.key.length = pdcp_test_params[index].cipher_key_len; + c_xfrm.cipher.key.data = pdcp_test_crypto_key[index]; + + a_xfrm.type = RTE_CRYPTO_SYM_XFORM_AUTH; + + if (pdcp_test_params[index].auth_alg == 0) { + conf->is_integrity_protected = false; + } else { + a_xfrm.auth.algo = pdcp_test_params[index].auth_alg; + a_xfrm.auth.key.data = pdcp_test_auth_key[index]; + a_xfrm.auth.key.length = pdcp_test_params[index].auth_key_len; + conf->is_integrity_protected = true; + } + + pdcp_hdr_sz = pdcp_hdr_size_get(pdcp_test_data_sn_size[index]); + + /* + * Uplink means PDCP entity is configured for transmit. Downlink means PDCP entity is + * configured for receive. When integrity protecting is enabled, PDCP always performs + * digest-encrypted or auth-gen-encrypt for uplink (and decrypt-auth-verify for downlink). + * So for uplink, crypto chain would be auth-cipher while for downlink it would be + * cipher-auth. + * + * When integrity protection is not required, xform would be cipher only. + */ + + if (conf->is_integrity_protected) { + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { + conf->entity.crypto_xfrm = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_GENERATE; + a_xfrm.next = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + c_xfrm.next = NULL; + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + c_xfrm.next = &conf->a_xfrm; + + a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + a_xfrm.next = NULL; + } + } else { + conf->entity.crypto_xfrm = &conf->c_xfrm; + c_xfrm.next = NULL; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_ENCRYPT; + else + c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + /* Update xforms to match PDCP requirements */ + + if ((c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_AES_CTR) || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_ZUC_EEA3 || + (c_xfrm.cipher.algo == RTE_CRYPTO_CIPHER_SNOW3G_UEA2))) + c_xfrm.cipher.iv.length = PDCP_IV_LEN; + else + c_xfrm.cipher.iv.length = 0; + + if (conf->is_integrity_protected) { + if (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_NULL) + a_xfrm.auth.digest_length = 0; + else + a_xfrm.auth.digest_length = RTE_PDCP_MAC_I_LEN; + + if ((a_xfrm.auth.algo == RTE_CRYPTO_AUTH_ZUC_EIA3) || + (a_xfrm.auth.algo == RTE_CRYPTO_AUTH_SNOW3G_UIA2)) + a_xfrm.auth.iv.length = PDCP_IV_LEN; + else + a_xfrm.auth.iv.length = 0; + } + + conf->c_xfrm = c_xfrm; + conf->a_xfrm = a_xfrm; + + conf->entity.dev_id = (uint8_t)cryptodev_id_get(conf->is_integrity_protected, + &conf->c_xfrm, &conf->a_xfrm); + + if (pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_CONTROL || + pdcp_test_params[index].domain == RTE_SECURITY_PDCP_MODE_DATA) { + data = pdcp_test_data_in[index]; + sn = pdcp_sn_from_raw_get(data, pdcp_test_data_sn_size[index]); + conf->entity.pdcp_xfrm.hfn = pdcp_test_hfn[index]; + conf->entity.sn = sn; + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) { +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", pdcp_test_data_in[index], + pdcp_test_data_in_len[index]); +#endif + /* Since the vectors available already have PDCP header, trim the same */ + conf->input_len = pdcp_test_data_in_len[index] - pdcp_hdr_sz; + memcpy(conf->input, pdcp_test_data_in[index] + pdcp_hdr_sz, conf->input_len); + } else { + conf->input_len = pdcp_test_data_in_len[index]; + + if (conf->is_integrity_protected) + conf->input_len += RTE_PDCP_MAC_I_LEN; + + memcpy(conf->input, pdcp_test_data_out[index], conf->input_len); +#ifdef VEC_DUMP + debug_hexdump(stdout, "Original vector:", conf->input, conf->input_len); +#endif + } + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected = pdcp_test_data_out[index]; + else + expected = pdcp_test_data_in[index]; + + /* Calculate expected packet length */ + expected_len = pdcp_test_data_in_len[index]; + + /* In DL processing, PDCP header would be stripped */ + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + expected += pdcp_hdr_sz; + expected_len -= pdcp_hdr_sz; + } + + /* In UL processing with integrity protection, MAC would be added */ + if (conf->is_integrity_protected && + conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + expected_len += 4; + + memcpy(conf->output, expected, expected_len); + conf->output_len = expected_len; + + return 0; +} + +static struct rte_pdcp_entity* +test_entity_create(const struct pdcp_test_conf *t_conf, int *rc) +{ + struct rte_pdcp_entity *pdcp_entity; + int ret; + + if (t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12 && + t_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_18) { + *rc = -ENOTSUP; + return NULL; + } + + if (t_conf->entity.dev_id == CDEV_INVALID_ID) { + RTE_LOG(DEBUG, USER1, "Could not find device with required capabilities\n"); + *rc = -ENOTSUP; + return NULL; + } + + ret = cryptodev_init(t_conf->entity.dev_id); + if (ret) { + *rc = ret; + RTE_LOG(DEBUG, USER1, "Could not initialize cryptodev\n"); + return NULL; + } + + rte_errno = 0; + + pdcp_entity = rte_pdcp_entity_establish(&t_conf->entity); + if (pdcp_entity == NULL) { + *rc = -rte_errno; + RTE_LOG(DEBUG, USER1, "Could not establish PDCP entity\n"); + return NULL; + } + + return pdcp_entity; +} + +static uint16_t +test_process_packets(const struct rte_pdcp_entity *pdcp_entity, uint8_t cdev_id, + struct rte_mbuf *in_mb[], uint16_t nb_in, + struct rte_mbuf *out_mb[], uint16_t *nb_err) +{ + struct rte_crypto_op *cop, *cop_out; + struct rte_pdcp_group grp[1]; + uint16_t nb_success, nb_grp; + struct rte_mbuf *mbuf, *mb; + + if (nb_in != 1) + return -ENOTSUP; + + mbuf = in_mb[0]; + + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, &mbuf, &cop_out, 1, nb_err); + if (nb_success != 1 || *nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not pre process PDCP packet\n"); + return TEST_FAILED; + } + +#ifdef VEC_DUMP + printf("Pre-processed vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, rte_pktmbuf_pkt_len(mbuf)); +#endif + + cop = process_crypto_request(cdev_id, cop_out); + if (cop == NULL) { + RTE_LOG(ERR, USER1, "Could not process crypto request\n"); + return -EIO; + } + + grp[0].id.val = 0; + + nb_grp = rte_pdcp_pkt_crypto_group(&cop_out, &mb, grp, 1); + if (nb_grp != 1 || grp[0].cnt != 1) { + RTE_LOG(ERR, USER1, "Could not group PDCP crypto results\n"); + return -ENOTRECOVERABLE; + } + + if ((uintptr_t)pdcp_entity != grp[0].id.val) { + RTE_LOG(ERR, USER1, "PDCP entity not matching the one from crypto_op\n"); + return -ENOTRECOVERABLE; + } + +#ifdef VEC_DUMP + printf("Crypto processed vector:\n"); + rte_pktmbuf_dump(stdout, cop->sym->m_dst, rte_pktmbuf_pkt_len(mbuf)); +#endif + + return rte_pdcp_pkt_post_process(grp[0].id.ptr, grp[0].m, out_mb, grp[0].cnt, nb_err); +} + +static struct rte_mbuf* +mbuf_from_data_create(uint8_t *data, uint16_t data_len) +{ + const struct pdcp_testsuite_params *ts_params = &testsuite_params; + struct rte_mbuf *mbuf; + uint8_t *input_text; + + mbuf = rte_pktmbuf_alloc(ts_params->mbuf_pool); + if (mbuf == NULL) { + RTE_LOG(ERR, USER1, "Could not create mbuf\n"); + return NULL; + } + + memset(rte_pktmbuf_mtod(mbuf, uint8_t *), 0, rte_pktmbuf_tailroom(mbuf)); + + input_text = (uint8_t *)rte_pktmbuf_append(mbuf, data_len); + memcpy(input_text, data, data_len); + + return mbuf; +} + +static int +test_attempt_single(struct pdcp_test_conf *t_conf) +{ + struct rte_mbuf *mbuf, **out_mb = NULL; + struct rte_pdcp_entity *pdcp_entity; + uint16_t nb_success, nb_err; + int ret = 0, nb_max_out_mb; + + pdcp_entity = test_entity_create(t_conf, &ret); + if (pdcp_entity == NULL) + goto exit; + + /* Allocate buffer for holding mbufs returned */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + 1; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate buffer for holding out_mb buffers\n"); + ret = -ENOMEM; + goto entity_release; + } + + mbuf = mbuf_from_data_create(t_conf->input, t_conf->input_len); + if (mbuf == NULL) { + ret = -ENOMEM; + goto entity_release; + } + +#ifdef VEC_DUMP + printf("Adjusted vector:\n"); + rte_pktmbuf_dump(stdout, mbuf, t_conf->input_len); +#endif + + nb_success = test_process_packets(pdcp_entity, t_conf->entity.dev_id, &mbuf, 1, out_mb, + &nb_err); + if (nb_success != 1 || nb_err != 0) { + RTE_LOG(ERR, USER1, "Could not process PDCP packet\n"); + ret = TEST_FAILED; + goto mbuf_free; + } + + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + + ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); + if (ret) { + RTE_LOG(DEBUG, USER1, "Could not suspend PDCP entity\n"); + goto mbuf_free; + } + +mbuf_free: + rte_pktmbuf_free(mbuf); +entity_release: + rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_free(out_mb); +exit: + return ret; +} + +static int +run_test_for_one_known_vec(const void *arg) +{ + struct pdcp_test_conf test_conf; + int i = *(const uint32_t *)arg; + + create_test_conf_from_index(i, &test_conf); + return test_attempt_single(&test_conf); +} + +struct unit_test_suite *test_suites[] = { + NULL, /* Place holder for known_vector_cases */ + NULL /* End of suites list */ +}; + +static struct unit_test_suite pdcp_testsuite = { + .suite_name = "PDCP Unit Test Suite", + .unit_test_cases = {TEST_CASES_END()}, + .setup = testsuite_setup, + .teardown = testsuite_teardown, + .unit_test_suites = test_suites, +}; + +static int +test_pdcp(void) +{ + struct unit_test_suite *known_vector_cases; + int ret, index[NB_TESTS]; + uint32_t i, size; + + size = sizeof(struct unit_test_suite); + size += (NB_TESTS + 1) * sizeof(struct unit_test_case); + + known_vector_cases = rte_zmalloc(NULL, size, 0); + if (known_vector_cases == NULL) + return TEST_FAILED; + + known_vector_cases->suite_name = "Known vector cases"; + + for (i = 0; i < NB_TESTS; i++) { + index[i] = i; + known_vector_cases->unit_test_cases[i].name = pdcp_test_params[i].name; + known_vector_cases->unit_test_cases[i].data = (void *)&index[i]; + known_vector_cases->unit_test_cases[i].enabled = 1; + known_vector_cases->unit_test_cases[i].setup = ut_setup_pdcp; + known_vector_cases->unit_test_cases[i].teardown = ut_teardown_pdcp; + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data + = run_test_for_one_known_vec; + } + + known_vector_cases->unit_test_cases[i].testcase = NULL; + known_vector_cases->unit_test_cases[i].testcase_with_data = NULL; + + test_suites[0] = known_vector_cases; + + ret = unit_test_suite_runner(&pdcp_testsuite); + + rte_free(known_vector_cases); + return ret; +} + +REGISTER_TEST_COMMAND(pdcp_autotest, test_pdcp); -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 10/21] test/pdcp: pdcp HFN tests in combined mode 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (8 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 09/21] app/test: add lib pdcp tests Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 11/21] doc: add PDCP library guide Anoob Joseph ` (11 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add tests to verify HFN/SN behaviour. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 302 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 299 insertions(+), 3 deletions(-) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 34b759eaef..cfe2ec6aa9 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,9 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* According to formula(7.2.a Window_Size) */ +#define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) + struct pdcp_testsuite_params { struct rte_mempool *mbuf_pool; struct rte_mempool *cop_pool; @@ -36,12 +39,69 @@ struct pdcp_test_conf { uint32_t output_len; }; +static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf); + +typedef int (*test_with_conf_t)(struct pdcp_test_conf *conf); + +static int +run_test_foreach_known_vec(test_with_conf_t test, bool stop_on_first_pass) +{ + struct pdcp_test_conf test_conf; + bool all_tests_skipped = true; + uint32_t i; + int ret; + + for (i = 0; i < NB_TESTS; i++) { + create_test_conf_from_index(i, &test_conf); + ret = test(&test_conf); + + if (ret == TEST_FAILED) { + printf("[%03i] - %s - failed\n", i, pdcp_test_params[i].name); + return TEST_FAILED; + } + + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + continue; + + if (stop_on_first_pass) + return TEST_SUCCESS; + + all_tests_skipped = false; + } + + if (all_tests_skipped) + return TEST_SKIPPED; + + return TEST_SUCCESS; +} + +static int +run_test_with_all_known_vec(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, false); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { return RTE_ALIGN_MUL_CEIL(sn_size, 8) / 8; } +static int +pktmbuf_read_into(const struct rte_mbuf *m, void *buf, size_t buf_len) +{ + if (m->pkt_len > buf_len) + return -ENOMEM; + + const void *read = rte_pktmbuf_read(m, 0, m->pkt_len, buf); + if (read != NULL && read != buf) + memcpy(buf, read, m->pkt_len); + + return 0; +} + static int cryptodev_init(int dev_id) { @@ -326,6 +386,21 @@ pdcp_sn_from_raw_get(const void *data, enum rte_security_pdcp_sn_size size) return sn; } +static void +pdcp_sn_to_raw_set(void *data, uint32_t sn, int size) +{ + if (size == RTE_SECURITY_PDCP_SN_SIZE_12) { + struct rte_pdcp_up_data_pdu_sn_12_hdr *pdu_hdr = data; + pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } else if (size == RTE_SECURITY_PDCP_SN_SIZE_18) { + struct rte_pdcp_up_data_pdu_sn_18_hdr *pdu_hdr = data; + pdu_hdr->sn_17_16 = ((sn & 0x30000) >> 16); + pdu_hdr->sn_15_8 = ((sn & 0xff00) >> 8); + pdu_hdr->sn_7_0 = (sn & 0xff); + } +} + static int create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) { @@ -648,9 +723,17 @@ test_attempt_single(struct pdcp_test_conf *t_conf) goto mbuf_free; } - ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); - if (ret) - goto mbuf_free; + /* If expected output provided - verify, else - store for future use */ + if (t_conf->output_len) { + ret = pdcp_known_vec_verify(mbuf, t_conf->output, t_conf->output_len); + if (ret) + goto mbuf_free; + } else { + ret = pktmbuf_read_into(mbuf, t_conf->output, RTE_PDCP_CTRL_PDU_SIZE_MAX); + if (ret) + goto mbuf_free; + t_conf->output_len = mbuf->pkt_len; + } ret = rte_pdcp_entity_suspend(pdcp_entity, out_mb); if (ret) { @@ -667,6 +750,193 @@ test_attempt_single(struct pdcp_test_conf *t_conf) return ret; } +static void +uplink_to_downlink_convert(const struct pdcp_test_conf *ul_cfg, + struct pdcp_test_conf *dl_cfg) +{ + assert(ul_cfg->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK); + + memcpy(dl_cfg, ul_cfg, sizeof(*dl_cfg)); + dl_cfg->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_DOWNLINK; + dl_cfg->entity.reverse_iv_direction = false; + + if (dl_cfg->is_integrity_protected) { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + dl_cfg->c_xfrm.next = &dl_cfg->a_xfrm; + + dl_cfg->a_xfrm.auth.op = RTE_CRYPTO_AUTH_OP_VERIFY; + dl_cfg->a_xfrm.next = NULL; + } else { + dl_cfg->entity.crypto_xfrm = &dl_cfg->c_xfrm; + dl_cfg->c_xfrm.next = NULL; + dl_cfg->c_xfrm.cipher.op = RTE_CRYPTO_CIPHER_OP_DECRYPT; + } + + dl_cfg->entity.dev_id = (uint8_t)cryptodev_id_get(dl_cfg->is_integrity_protected, + &dl_cfg->c_xfrm, &dl_cfg->a_xfrm); + + memcpy(dl_cfg->input, ul_cfg->output, ul_cfg->output_len); + dl_cfg->input_len = ul_cfg->output_len; + + memcpy(dl_cfg->output, ul_cfg->input, ul_cfg->input_len); + dl_cfg->output_len = ul_cfg->input_len; +} + +/* + * According to ETSI TS 138 323 V17.1.0, Section 5.2.2.1, + * SN could be divided into following ranges, + * relatively to current value of RX_DELIV state: + * +-------------+-------------+-------------+-------------+ + * | -Outside | -Window | +Window | +Outside | + * | (valid) | (Invalid) | (Valid) | (Invalid) | + * +-------------+-------------^-------------+-------------+ + * | + * v + * SN(RX_DELIV) + */ +enum sn_range_type { + SN_RANGE_MINUS_OUTSIDE, + SN_RANGE_MINUS_WINDOW, + SN_RANGE_PLUS_WINDOW, + SN_RANGE_PLUS_OUTSIDE, +}; + +#define PDCP_SET_COUNT(hfn, sn, size) ((hfn << size) | (sn & ((1 << size) - 1))) + +/* + * Take uplink test case as base, modify RX_DELIV in state and SN in input + */ +static int +test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) +{ + uint32_t rx_deliv_hfn, rx_deliv_sn, new_hfn, new_sn; + const int domain = conf->entity.pdcp_xfrm.domain; + struct pdcp_test_conf dl_conf; + int ret, expected_ret; + + if (conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + if (domain != RTE_SECURITY_PDCP_MODE_CONTROL && domain != RTE_SECURITY_PDCP_MODE_DATA) + return TEST_SKIPPED; + + const uint32_t sn_size = conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + /* Max value of SN that could fit in `sn_size` bits */ + const uint32_t max_sn = (1 << sn_size) - 1; + const uint32_t shift = (max_sn - window_size) / 2; + /* Could be any number up to `shift` value */ + const uint32_t default_sn = RTE_MIN(2u, shift); + + /* Initialize HFN as non zero value, to be able check values before */ + rx_deliv_hfn = 0xa; + + switch (type) { + case SN_RANGE_PLUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + 1; + expected_ret = TEST_SUCCESS; + break; + case SN_RANGE_MINUS_WINDOW: + /* Within window size, HFN stay same */ + new_hfn = rx_deliv_hfn; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn - 1; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_PLUS_OUTSIDE: + /* RCVD_SN >= SN(RX_DELIV) + Window_Size */ + new_hfn = rx_deliv_hfn - 1; + rx_deliv_sn = default_sn; + new_sn = rx_deliv_sn + window_size; + expected_ret = TEST_FAILED; + break; + case SN_RANGE_MINUS_OUTSIDE: + /* RCVD_SN < SN(RX_DELIV) - Window_Size */ + new_hfn = rx_deliv_hfn + 1; + rx_deliv_sn = window_size + default_sn; + new_sn = rx_deliv_sn - window_size - 1; + expected_ret = TEST_SUCCESS; + break; + default: + return TEST_FAILED; + } + + /* Configure Uplink to generate expected, encrypted packet */ + pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.reverse_iv_direction = true; + conf->entity.pdcp_xfrm.hfn = new_hfn; + conf->entity.sn = new_sn; + conf->output_len = 0; + ret = test_attempt_single(conf); + if (ret != TEST_SUCCESS) + return ret; + + /* Flip configuration to downlink */ + uplink_to_downlink_convert(conf, &dl_conf); + + /* Modify the rx_deliv to verify the expected behaviour */ + dl_conf.entity.pdcp_xfrm.hfn = rx_deliv_hfn; + dl_conf.entity.sn = rx_deliv_sn; + ret = test_attempt_single(&dl_conf); + if ((ret == TEST_SKIPPED) || (ret == -ENOTSUP)) + return ret; + + TEST_ASSERT_EQUAL(ret, expected_ret, "Unexpected result"); + + return TEST_SUCCESS; +} + +static int +test_sn_plus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_WINDOW, t_conf); +} + +static int +test_sn_minus_window(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_WINDOW, t_conf); +} + +static int +test_sn_plus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_PLUS_OUTSIDE, t_conf); +} + +static int +test_sn_minus_outside(struct pdcp_test_conf *t_conf) +{ + return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); +} + +static int +test_combined(struct pdcp_test_conf *ul_conf) +{ + struct pdcp_test_conf dl_conf; + int ret; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + ul_conf->entity.reverse_iv_direction = true; + ul_conf->output_len = 0; + + ret = test_attempt_single(ul_conf); + if (ret != TEST_SUCCESS) + return ret; + + uplink_to_downlink_convert(ul_conf, &dl_conf); + ret = test_attempt_single(&dl_conf); + + return ret; +} + static int run_test_for_one_known_vec(const void *arg) { @@ -677,8 +947,34 @@ run_test_for_one_known_vec(const void *arg) return test_attempt_single(&test_conf); } +static struct unit_test_suite combined_mode_cases = { + .suite_name = "PDCP combined mode", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("combined mode", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_combined), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + +static struct unit_test_suite hfn_sn_test_cases = { + .suite_name = "PDCP HFN/SN", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("SN plus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_window), + TEST_CASE_NAMED_WITH_DATA("SN minus window", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_window), + TEST_CASE_NAMED_WITH_DATA("SN plus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_plus_outside), + TEST_CASE_NAMED_WITH_DATA("SN minus outside", ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_sn_minus_outside), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ + &combined_mode_cases, + &hfn_sn_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 11/21] doc: add PDCP library guide 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (9 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 12/21] pdcp: add control PDU handling for status report Anoob Joseph ` (10 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add guide for PDCP library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Kiran Kumar K <kirankumark@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- MAINTAINERS | 1 + .../img/pdcp_functional_overview.svg | 1 + doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/pdcp_lib.rst | 254 ++++++++++++++++++ doc/guides/rel_notes/release_23_07.rst | 12 + 5 files changed, 269 insertions(+) create mode 100644 doc/guides/prog_guide/img/pdcp_functional_overview.svg create mode 100644 doc/guides/prog_guide/pdcp_lib.rst diff --git a/MAINTAINERS b/MAINTAINERS index ca684dde83..11ecb153bc 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1557,6 +1557,7 @@ M: Volodymyr Fialko <vfialko@marvell.com> T: git://dpdk.org/next/dpdk-next-crypto F: lib/pdcp/ F: app/test/test_pdcp* +F: doc/guides/prog_guide/pdcp_lib.rst Packet Framework diff --git a/doc/guides/prog_guide/img/pdcp_functional_overview.svg b/doc/guides/prog_guide/img/pdcp_functional_overview.svg new file mode 100644 index 0000000000..287daafc21 --- /dev/null +++ b/doc/guides/prog_guide/img/pdcp_functional_overview.svg @@ -0,0 +1 @@ +<svg width="1280" height="720" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" overflow="hidden"><defs><clipPath id="clip0"><rect x="0" y="0" width="1280" height="720"/></clipPath></defs><g clip-path="url(#clip0)"><rect x="0" y="0" width="1280" height="720" fill="#FFFFFF"/><rect x="202" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><rect x="640" y="100" width="369" height="457" fill="#A5A5A5" fill-opacity="0.501961"/><path d="M605.5 73.5001 605.5 590.633" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M380.5 634 803.25 634 803.25 585.907 790.5 585.907 816 557.5 841.5 585.907 828.75 585.907 828.75 659.5 380.5 659.5Z" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5" fill-rule="evenodd"/><rect x="362.5" y="557.5" width="28" height="102" stroke="#787878" stroke-width="1.33333" stroke-miterlimit="8" fill="#A5A5A5"/><rect x="412.5" y="671.5" width="370" height="32" stroke="#000000" stroke-linejoin="round" stroke-miterlimit="10" fill="none"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(492.364 694)">Radio Interface (<tspan font-size="19" x="137.5" y="0">Uu</tspan><tspan font-size="19" x="161.333" y="0">/PC5)</tspan><tspan font-size="19" x="-282.121" y="-653">UE/NG</tspan><tspan font-size="19" x="-222.955" y="-653">-</tspan><tspan font-size="19" x="-216.788" y="-653">RAN/UE A</tspan><tspan font-size="19" x="375.54" y="-653">NG</tspan>-<tspan font-size="19" x="409.706" y="-653">RAN/UE/UE B</tspan><tspan font-size="14" x="0.896851" y="-647">Transmitting </tspan><tspan font-size="14" x="1.31018" y="-631">PDCP entity</tspan><tspan font-size="14" x="167.401" y="-647">Receiving </tspan><tspan font-size="14" x="160.148" y="-631">PDCP entity</tspan></text><path d="M314.5 71.5001 431.364 71.6549" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M331.5 71.5001C331.5 65.9772 349.633 61.5001 372 61.5001 394.368 61.5001 412.5 65.9772 412.5 71.5001 412.5 77.0229 394.368 81.5001 372 81.5001 349.633 81.5001 331.5 77.0229 331.5 71.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M353.5 90.5001 363.5 90.5001 363.5 48.5001 383.5 48.5001 383.5 90.5001 393.5 90.5001 373.5 110.5Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M877.364 82.6549 760.5 82.5001" stroke="#000000" stroke-width="0.666667" stroke-miterlimit="8" fill="none" fill-rule="evenodd"/><path d="M860.5 83.5001C860.5 89.0229 842.368 93.5001 820 93.5001 797.633 93.5001 779.5 89.0229 779.5 83.5001 779.5 77.9772 797.633 73.5001 820 73.5001 842.368 73.5001 860.5 77.9772 860.5 83.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><path d="M838.5 64.5 828.5 64.5 828.5 106.5 808.5 106.5 808.5 64.5 798.5 64.5 818.5 44.5001Z" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF" fill-rule="evenodd"/><rect x="244.5" y="128.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(302.494 150)">Transmission buffer:<tspan font-size="19" x="-4.67999" y="23">Sequence</tspan><tspan font-size="19" x="84.32" y="23">numberin</tspan>g</text><rect x="244.5" y="199.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(297.314 222)">Header or uplink data<tspan font-size="19" x="34.08" y="23">Compressio</tspan>n</text><rect x="682.5" y="141.5" width="285" height="55" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(735.782 164)">Header or uplink data<tspan font-size="19" x="24.2466" y="23">Decompressio</tspan>n</text><rect x="244.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(300.314 505)">Routing / Duplication</text><rect x="244.5" y="437.5" width="285" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(309.734 460)">Add PDCP header</text><rect x="244.5" y="383.5" width="189" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(298.62 406)">Ciphering</text><rect x="244.5" y="333.5" width="189" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(261.953 356)">Integrity protection</text><path d="M472.167 267.5 472.167 409.802 470.833 409.802 470.833 267.5ZM475.5 408.468 471.5 416.468 467.5 408.468Z"/><path d="M332.167 267.5 332.167 319.552 330.833 319.552 330.833 267.5ZM335.5 318.218 331.5 326.218 327.5 318.219Z"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(253.644 291)">Packets associated <tspan font-size="19" x="14.0067" y="23">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 499.312 299)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="-2.75546e-15" y="45">PDCP SDU</tspan></text><rect x="682.5" y="482.5" width="285" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(728.535 505)">Remove PDCP Header</text><rect x="682.5" y="437.5" width="203" height="32" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(733.729 460)">Deciphering</text><rect x="682.5" y="389.5" width="203" height="33" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(702.159 412)">Integrity Verification</text><rect x="682.5" y="303.5" width="203" height="77" stroke="#000000" stroke-width="1.33333" stroke-miterlimit="8" fill="#FFFFFF"/><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="translate(712.729 325)">Reception buffer:<tspan font-size="19" x="24.6667" y="23">Reordering</tspan><tspan font-size="19" x="-13.1667" y="45">Duplicate discardin</tspan><tspan font-size="19" x="144.167" y="45">g</tspan><tspan font-size="19" x="-10.0989" y="-84">Packets associated </tspan><tspan font-size="19" x="3.90784" y="-61">to a PDCP SDU</tspan></text><text font-family="Arial,Arial_MSFontService,sans-serif" font-weight="400" font-size="19" transform="matrix(6.12323e-17 1 -1 6.12323e-17 960.34 294)">Packets not <tspan font-size="19" x="-14" y="23">associated to a </tspan><tspan font-size="19" x="1" y="45">PDCP SDU</tspan></text><path d="M0.666667-8.2074e-07 0.666763 78.6116-0.66657 78.6116-0.666667 8.2074e-07ZM4.00009 77.2782 0.000104987 85.2782-3.9999 77.2782Z" transform="matrix(1 0 0 -1 779.5 296.778)"/><path d="M0.666667-2.88742e-07 0.666769 235.734-0.666565 235.734-0.666667 2.88742e-07ZM4.0001 234.401 0.000104987 242.401-3.9999 234.401Z" transform="matrix(1 0 0 -1 931.5 453.901)"/></g></svg> \ No newline at end of file diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 87333ee84a..6099ff63cd 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -77,4 +77,5 @@ Programmer's Guide lto profile_app asan + pdcp_lib glossary diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst new file mode 100644 index 0000000000..2eefabf45c --- /dev/null +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -0,0 +1,254 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(C) 2023 Marvell. + +PDCP Protocol Processing Library +================================ + +DPDK provides a library for PDCP protocol processing. +The library utilizes other DPDK libraries such as cryptodev, reorder, etc., +to provide the application with a transparent and +high performant PDCP protocol processing library. + +The library abstracts complete PDCP protocol processing conforming to +``ETSI TS 138 323 V17.1.0 (2022-08)``. +https://www.etsi.org/deliver/etsi_ts/138300_138399/138323/17.01.00_60/ts_138323v170100p.pdf + +PDCP would involve the following operations, + +1. Transfer of user plane data +2. Transfer of control plane data +3. Header compression +4. Uplink data compression +5. Ciphering and integrity protection + +.. _figure_pdcp_functional_overview: + +.. figure:: img/pdcp_functional_overview.* + + PDCP functional overview new + +PDCP library would abstract the protocol offload features of the cryptodev and +would provide a uniform interface and consistent API usage to work with +cryptodev irrespective of the protocol offload features supported. + +PDCP entity API +--------------- + +PDCP library provides following control path APIs that is used to +configure various PDCP entities, + +1. ``rte_pdcp_entity_establish()`` +2. ``rte_pdcp_entity_suspend()`` +3. ``rte_pdcp_entity_release()`` + +A PDCP entity would translate to one ``rte_cryptodev_sym_session`` or +``rte_security_session`` based on the config. The sessions would be created/ +destroyed while corresponding PDCP entity operations are performed. + +When upper layers request a PDCP entity suspend (``rte_pdcp_entity_suspend()``), +it would result in flushing out of all cached packets and +internal state variables are updated as described in 5.1.4. + +When upper layers request a PDCP entity release (``rte_pdcp_entity_release()``), +it would result in flushing out of all cached packets and releasing of all +memory associated with the entity. It would internally free any crypto/security +sessions created. All procedures mentioned in 5.1.3 would be performed. + +PDCP PDU (Protocol Data Unit) API +--------------------------------- + +PDCP PDUs can be categorized as, + +1. Control PDU +2. Data PDU + +Control PDUs are used for signalling between entities on either end and +can be one of the following, + +1. PDCP status report +2. ROHC feedback +3. EHC feedback + +Control PDUs are not ciphered or authenticated, and so such packets are not +submitted to cryptodev for processing. + +Data PDUs are regular packets submitted by upper layers for transmission to +other end. Such packets would need to be ciphered and authenticated based on +the entity configuration. + +PDCP packet processing API for data PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +PDCP processing is split into 2 parts. One before cryptodev processing +(``rte_pdcp_pkt_pre_process()``) and one after cryptodev processing +(``rte_pdcp_pkt_post_process()``). Since cryptodev dequeue can return crypto +operations belonging to multiple entities, ``rte_pdcp_pkt_crypto_group()`` +is added to help grouping crypto operations belonging to same PDCP entity. + +Lib PDCP would allow application to use same API sequence while leveraging +protocol offload features enabled by ``rte_security`` library. +Lib PDCP would internally change the handles registered for ``pre_process`` +and ``post_process`` based on features enabled in the entity. + +Lib PDCP would create the required sessions on the device provided in entity to +minimize the application requirements. Also, the ``rte_crypto_op`` allocation +and free would also be done internally by lib PDCP to allow the library to create +crypto ops as required for the input packets. +For example, when control PDUs are received, no cryptodev enqueue-dequeue is +expected for the same and lib PDCP is expected to handle it differently. + +Supported features +------------------ + +- 12 bit & 18 bit sequence numbers +- Uplink & downlink traffic +- HFN increment +- IV generation as required per algorithm + +Supported ciphering algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- RTE_CRYPTO_CIPHER_NULL +- RTE_CRYPTO_CIPHER_AES_CTR +- RTE_CRYPTO_CIPHER_SNOW3G_UEA2 +- RTE_CRYPTO_CIPHER_ZUC_EEA3 + +Supported integrity protection algorithms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- RTE_CRYPTO_AUTH_NULL +- RTE_CRYPTO_AUTH_AES_CMAC +- RTE_CRYPTO_AUTH_SNOW3G_UIA2 +- RTE_CRYPTO_AUTH_ZUC_EIA3 + +Sample API usage +---------------- + +The ``rte_pdcp_entity_conf`` structure is used to pass the configuration +parameters for entity creation. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_entity_conf 8< + :end-before: >8 End of structure rte_pdcp_entity_conf. + +.. code-block:: c + + struct rte_mbuf **out_mb, *pkts[MAX_BURST_SIZE]; + struct rte_crypto_op *cop[MAX_BURST_SIZE]; + struct rte_pdcp_group grp[MAX_BURST_SIZE]; + struct rte_pdcp_entity *pdcp_entity; + int nb_max_out_mb, ret, nb_grp; + uint16_t nb_ops; + + /* Create PDCP entity */ + pdcp_entity = rte_pdcp_entity_establish(&conf); + + /** + * Allocate buffer for holding mbufs returned during PDCP suspend, + * release & post-process APIs. + */ + + /* Max packets that can be cached in entity + burst size */ + nb_max_out_mb = pdcp_entity->max_pkt_cache + MAX_BURST_SIZE; + out_mb = rte_malloc(NULL, nb_max_out_mb * sizeof(uintptr_t), 0); + if (out_mb == NULL) { + /* Handle error */ + } + + while (1) { + /* Receive packet and form mbuf */ + + /** + * Prepare packets for crypto operation. + * Following operations would be done, + * + * Transmitting entity/UL (only data PDUs): + * - Perform compression + * - Assign sequence number + * - Add PDCP header + * - Create & prepare crypto_op + * - Prepare IV for crypto operation (auth_gen, encrypt) + * - Save original PDCP SDU (during PDCP re-establishment, + * unconfirmed PDCP SDUs need to be crypto processed again and + * transmitted/re-transmitted) + * + * Receiving entity/DL: + * - Any control PDUs received would be processed and + * appropriate actions taken. If data PDU, continue. + * - Determine sequence number (based on HFN & per packet SN) + * - Prepare crypto_op + * - Prepare IV for crypto operation (decrypt, auth_verify) + */ + nb_success = rte_pdcp_pkt_pre_process(pdcp_entity, pkts, cop, + nb_rx, &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + if ((rte_cryptodev_enqueue_burst(dev_id, qp_id, cop, nb_success) + != nb_success) { + /* Retry for enqueue failure packets */ + } + + ... + + nb_ops = rte_cryptodev_dequeue_burst(dev_id, qp_id, cop, + MAX_BURST_SIZE); + if (nb_ops == 0) + continue; + + /** + * Received a burst of completed crypto ops from cryptodev. It + * may belong to various entities. Group similar ones together + * for entity specific post-processing. + */ + + /** + * Groups similar entities together. Frees crypto op and based + * on crypto_op status, set mbuf->ol_flags which would be + * checked in rte_pdcp_pkt_post_process(). + */ + nb_grp = rte_pdcp_pkt_crypto_group(cop, pkts, grp, ret); + + for (i = 0; i != nb_grp; i++) { + + /** + * Post process packets after crypto completion. + * Following operations would be done, + * + * Transmitting entity/UL: + * - Check crypto result + * + * Receiving entity/DL: + * - Check crypto operation status + * - Check for duplication (if yes, drop duplicate) + * - Perform decompression + * - Trim PDCP header + * - Hold packet (SDU) for in-order delivery (return + * completed packets as and when sequence is + * completed) + * - If not in sequence, cache the packet and start + * t-Reordering timer. When timer expires, the + * packets need to delivered to upper layers (not + * treated as error packets). + */ + nb_success = rte_pdcp_pkt_post_process(grp[i].id.ptr, + grp[i].m, out_mb, + grp[i].cnt, + &nb_err); + if (nb_err != 0) { + /* Handle error packets */ + } + + /* Perform additional operations */ + + /** + * Transmitting entity/UL + * - If duplication is enabled, duplicate PDCP PDUs + * - When lower layers confirm reception of a PDCP PDU, + * it should be communicated to PDCP layer so that + * PDCP can drop the corresponding SDU + */ + } + } diff --git a/doc/guides/rel_notes/release_23_07.rst b/doc/guides/rel_notes/release_23_07.rst index 75079ca7d6..bb48514637 100644 --- a/doc/guides/rel_notes/release_23_07.rst +++ b/doc/guides/rel_notes/release_23_07.rst @@ -65,6 +65,18 @@ New Features * Added support for SM3 hash operations. * Added support for AES-CCM in cn9k and cn10k drivers. +* **Added PDCP Library.** + + Added an experimental library ``lib_pdcp`` to provide PDCP UL and DL + processing of packets. + + The library supports all PDCP algorithms and leverages lookaside crypto + offloads to cryptodevs for crypto processing. PDCP features such as IV + generation, sequence number handling etc are supported. It is planned to add + more features such as packet caching in future releases. + + See :doc:`../prog_guide/pdcp_lib` for more information. + Removed Items ------------- -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 12/21] pdcp: add control PDU handling for status report 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (10 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 11/21] doc: add PDCP library guide Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph ` (9 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger Add control PDU handling and implement status report generation. Status report generation works only when RX_DELIV = RX_NEXT. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 9 ++++++ lib/pdcp/meson.build | 2 ++ lib/pdcp/pdcp_cnt.c | 29 ++++++++++++++++++ lib/pdcp/pdcp_cnt.h | 14 +++++++++ lib/pdcp/pdcp_ctrl_pdu.c | 46 +++++++++++++++++++++++++++++ lib/pdcp/pdcp_ctrl_pdu.h | 15 ++++++++++ lib/pdcp/pdcp_entity.h | 15 ++++++++-- lib/pdcp/pdcp_process.c | 13 +++++++++ lib/pdcp/rte_pdcp.c | 47 +++++++++++++++++++++++++++++- lib/pdcp/rte_pdcp.h | 33 +++++++++++++++++++++ lib/pdcp/version.map | 2 ++ 11 files changed, 222 insertions(+), 3 deletions(-) create mode 100644 lib/pdcp/pdcp_cnt.c create mode 100644 lib/pdcp/pdcp_cnt.h create mode 100644 lib/pdcp/pdcp_ctrl_pdu.c create mode 100644 lib/pdcp/pdcp_ctrl_pdu.h diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index 2eefabf45c..a925aa7f14 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -76,6 +76,15 @@ Data PDUs are regular packets submitted by upper layers for transmission to other end. Such packets would need to be ciphered and authenticated based on the entity configuration. +PDCP packet processing API for control PDU +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Control PDUs are used in PDCP as a communication channel between transmitting +and receiving entities. When upper layer request for operations such as +re-establishment, receiving PDCP entity need to prepare a status report and +send it to the other end. The API ``rte_pdcp_control_pdu_create()`` allows +application to request the same. + PDCP packet processing API for data PDU ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 08679b743a..75d476bf6d 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -8,7 +8,9 @@ if is_windows endif sources = files( + 'pdcp_cnt.c', 'pdcp_crypto.c', + 'pdcp_ctrl_pdu.c', 'pdcp_process.c', 'rte_pdcp.c', ) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c new file mode 100644 index 0000000000..c9b952184b --- /dev/null +++ b/lib/pdcp/pdcp_cnt.c @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_pdcp.h> + +#include "pdcp_cnt.h" +#include "pdcp_entity.h" + +int +pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +{ + struct entity_priv_dl_part *en_priv_dl; + uint32_t window_sz; + + if (en == NULL || conf == NULL) + return -EINVAL; + + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + return 0; + + en_priv_dl = entity_dl_part_get(en); + window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + RTE_SET_USED(window_sz); + RTE_SET_USED(en_priv_dl); + + return 0; +} diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h new file mode 100644 index 0000000000..bbda478b55 --- /dev/null +++ b/lib/pdcp/pdcp_cnt.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CNT_H +#define PDCP_CNT_H + +#include <rte_common.h> + +#include "pdcp_entity.h" + +int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); + +#endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c new file mode 100644 index 0000000000..feb05fd863 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_byteorder.h> +#include <rte_mbuf.h> +#include <rte_pdcp_hdr.h> + +#include "pdcp_ctrl_pdu.h" +#include "pdcp_entity.h" + +static __rte_always_inline void +pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) +{ + pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + pdu_hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + pdu_hdr->r = 0; + pdu_hdr->fmc = rte_cpu_to_be_32(rx_deliv); +} + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; + uint32_t rx_deliv; + int pdu_sz; + + if (!en_priv->flags.is_status_report_required) + return -EINVAL; + + pdu_sz = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + + rx_deliv = en_priv->state.rx_deliv; + + /* Zero missing PDUs - status report contains only FMC */ + if (rx_deliv >= en_priv->state.rx_next) { + pdu_hdr = (struct rte_pdcp_up_ctrl_pdu_hdr *)rte_pktmbuf_append(m, pdu_sz); + if (pdu_hdr == NULL) + return -ENOMEM; + pdcp_hdr_fill(pdu_hdr, rx_deliv); + + return 0; + } + + return -ENOTSUP; +} diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h new file mode 100644 index 0000000000..a2424fbd10 --- /dev/null +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_CTRL_PDU_H +#define PDCP_CTRL_PDU_H + +#include <rte_mbuf.h> + +#include "pdcp_entity.h" + +int +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); + +#endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 10a72faae1..28691a504b 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -109,6 +109,13 @@ union cipher_iv_partial { uint64_t u64[2]; }; +struct pdcp_cnt_bitmap { + /** Number of entries that can be stored. */ + uint32_t size; + /** Bitmap of the count values already received.*/ + struct rte_bitmap *bmp; +}; + /* * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] */ @@ -136,9 +143,13 @@ struct entity_priv { uint64_t is_ul_entity : 1; /** Is NULL auth. */ uint64_t is_null_auth : 1; + /** Is status report required.*/ + uint64_t is_status_report_required : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; + /** Control PDU pool. */ + struct rte_mempool *ctrl_pdu_pool; /** PDCP header size. */ uint8_t hdr_sz; /** PDCP AAD size. For AES-CMAC, additional message is prepended for the operation. */ @@ -148,8 +159,8 @@ struct entity_priv { }; struct entity_priv_dl_part { - /* NOTE: when in-order-delivery is supported, post PDCP packets would need to cached. */ - uint8_t dummy; + /** PDCP would need to track the count values that are already received.*/ + struct pdcp_cnt_bitmap bitmap; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 28ac4102da..ed1413db6d 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -1092,6 +1092,19 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent if (a_xfrm != NULL && a_xfrm->auth.algo == RTE_CRYPTO_AUTH_NULL) en_priv->flags.is_null_auth = 1; + /** + * flags.is_status_report_required + * + * Indicate whether status report is required. + */ + if (conf->status_report_required) { + /** Status report is required only for DL entities. */ + if (conf->pdcp_xfrm.pkt_dir != RTE_SECURITY_PDCP_DOWNLINK) + return -EINVAL; + + en_priv->flags.is_status_report_required = 1; + } + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 91dab91f73..96ad397667 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -6,7 +6,9 @@ #include <rte_pdcp.h> #include <rte_malloc.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -58,7 +60,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (pdcp_dynfield_register() < 0) return NULL; - if (conf == NULL || conf->cop_pool == NULL) { + if (conf == NULL || conf->cop_pool == NULL || conf->ctrl_pdu_pool == NULL) { rte_errno = EINVAL; return NULL; } @@ -105,6 +107,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) en_priv->state.rx_deliv = count; en_priv->state.tx_next = count; en_priv->cop_pool = conf->cop_pool; + en_priv->ctrl_pdu_pool = conf->ctrl_pdu_pool; /* Setup crypto session */ ret = pdcp_crypto_sess_create(entity, conf); @@ -115,6 +118,10 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + ret = pdcp_cnt_ring_create(entity, conf); + if (ret) + goto crypto_sess_destroy; + return entity; crypto_sess_destroy: @@ -162,3 +169,41 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, return 0; } + +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type) +{ + struct entity_priv *en_priv; + struct rte_mbuf *m; + int ret; + + if (pdcp_entity == NULL) { + rte_errno = EINVAL; + return NULL; + } + + en_priv = entity_priv_get(pdcp_entity); + + m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); + if (m == NULL) { + rte_errno = ENOMEM; + return NULL; + } + + switch (type) { + case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: + ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + break; + default: + ret = -ENOTSUP; + } + + if (ret) { + rte_pktmbuf_free(m); + rte_errno = -ret; + return NULL; + } + + return m; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index e63946aa08..d7c2080358 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -16,6 +16,7 @@ #include <rte_compat.h> #include <rte_common.h> #include <rte_mempool.h> +#include <rte_pdcp_hdr.h> #include <rte_security.h> #ifdef __cplusplus @@ -80,6 +81,8 @@ struct rte_pdcp_entity_conf { struct rte_mempool *sess_mpool; /** Crypto op pool. */ struct rte_mempool *cop_pool; + /** Mbuf pool to be used for allocating control PDUs.*/ + struct rte_mempool *ctrl_pdu_pool; /** * Sequence number value to be used. * 32 bit count value to be used for the first packet @@ -97,6 +100,16 @@ struct rte_pdcp_entity_conf { * Can be used to simulate UE crypto processing. */ bool reverse_iv_direction; + /** + * Status report required (specified in TS 38.331). + * + * If PDCP entity is configured to send a PDCP status report, + * the upper layer application may request a receiving PDCP entity + * to generate a PDCP status report using ``rte_pdcp_control_pdu_create``. + * In addition, PDCP status reports may be generated during operations + * such as entity re-establishment. + */ + bool status_report_required; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -178,6 +191,26 @@ int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]); +/** + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * Create control PDU packet of the `type` specified. The control PDU packet + * would be allocated from *rte_pdcp_entity_conf.ctrl_pdu_pool* by lib PDCP. + * + * @param pdcp_entity + * Pointer to the PDCP entity for which the control PDU need to be generated. + * @param type + * Type of control PDU to be generated. + * @return + * - Control PDU generated, in case of success. + * - NULL in case of failure. rte_errno will be set to error code. + */ +__rte_experimental +struct rte_mbuf * +rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, + enum rte_pdcp_ctrl_pdu_type type); + /** * @warning * @b EXPERIMENTAL: this API may change without prior notice diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index d564f155e0..97171f902e 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -2,6 +2,8 @@ EXPERIMENTAL { global: # added in 23.07 + rte_pdcp_control_pdu_create; + rte_pdcp_en_from_cop; rte_pdcp_entity_establish; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 13/21] pdcp: implement t-Reordering and packet buffering 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (11 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 12/21] pdcp: add control PDU handling for status report Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 14/21] test/pdcp: add in-order delivery cases Anoob Joseph ` (8 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add in-order delivery of packets in PDCP. Delivery of packets in-order relies on t-Reordering timer. When 'out-of-order delivery' is disabled, PDCP will buffer all received packets that are out of order. The t-Reordering timer determines the time period these packets would be held in the buffer, waiting for any missing packets to arrive. Introduce packet buffering and state variables which indicate status of the timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/meson.build | 3 +- lib/pdcp/pdcp_entity.h | 19 +++++++ lib/pdcp/pdcp_process.c | 117 ++++++++++++++++++++++++++++++---------- lib/pdcp/pdcp_reorder.c | 27 ++++++++++ lib/pdcp/pdcp_reorder.h | 62 +++++++++++++++++++++ lib/pdcp/rte_pdcp.c | 53 ++++++++++++++++-- lib/pdcp/rte_pdcp.h | 6 ++- 7 files changed, 252 insertions(+), 35 deletions(-) create mode 100644 lib/pdcp/pdcp_reorder.c create mode 100644 lib/pdcp/pdcp_reorder.h diff --git a/lib/pdcp/meson.build b/lib/pdcp/meson.build index 75d476bf6d..f4f9246bcb 100644 --- a/lib/pdcp/meson.build +++ b/lib/pdcp/meson.build @@ -12,9 +12,10 @@ sources = files( 'pdcp_crypto.c', 'pdcp_ctrl_pdu.c', 'pdcp_process.c', + 'pdcp_reorder.c', 'rte_pdcp.c', ) headers = files('rte_pdcp.h') indirect_headers += files('rte_pdcp_group.h') -deps += ['mbuf', 'net', 'cryptodev', 'security'] +deps += ['mbuf', 'net', 'cryptodev', 'security', 'reorder'] diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 28691a504b..34341cdc11 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -11,6 +11,8 @@ #include <rte_pdcp.h> #include <rte_security.h> +#include "pdcp_reorder.h" + struct entity_priv; #define PDCP_HFN_MIN 0 @@ -109,6 +111,17 @@ union cipher_iv_partial { uint64_t u64[2]; }; +enum timer_state { + TIMER_STOP, + TIMER_RUNNING, + TIMER_EXPIRED, +}; + +struct pdcp_t_reordering { + /** Represent timer state */ + enum timer_state state; +}; + struct pdcp_cnt_bitmap { /** Number of entries that can be stored. */ uint32_t size; @@ -145,6 +158,8 @@ struct entity_priv { uint64_t is_null_auth : 1; /** Is status report required.*/ uint64_t is_status_report_required : 1; + /** Is out-of-order delivery enabled */ + uint64_t is_out_of_order_delivery : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -161,6 +176,10 @@ struct entity_priv { struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; + /** t-Reordering handles */ + struct pdcp_t_reordering t_reorder; + /** Reorder packet buffer */ + struct pdcp_reorder reorder; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index ed1413db6d..84a0f3a43f 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -837,25 +837,88 @@ pdcp_packet_strip(struct rte_mbuf *mb, const uint32_t hdr_trim_sz, const bool tr } } -static inline bool +static inline int pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, - const uint32_t count) + const uint32_t count, struct rte_mbuf *mb, + struct rte_mbuf *out_mb[], + const bool trim_mac) { struct entity_priv *en_priv = entity_priv_get(entity); + struct pdcp_t_reordering *t_reorder; + struct pdcp_reorder *reorder; + uint16_t processed = 0; - if (count < en_priv->state.rx_deliv) - return false; + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + const uint32_t hdr_trim_sz = en_priv->hdr_sz + en_priv->aad_sz; - /* t-Reordering timer is not supported - SDU will be delivered immediately. - * Update RX_DELIV to the COUNT value of the first PDCP SDU which has not - * been delivered to upper layers - */ - en_priv->state.rx_next = count + 1; + if (count < en_priv->state.rx_deliv) + return -EINVAL; if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; - return true; + pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); + + if (en_priv->flags.is_out_of_order_delivery) { + out_mb[0] = mb; + en_priv->state.rx_deliv = count + 1; + + return 1; + } + + reorder = &dl->reorder; + t_reorder = &dl->t_reorder; + + if (count == en_priv->state.rx_deliv) { + if (reorder->is_active) { + /* + * This insert used only to increment reorder->min_seqn + * To remove it - min_seqn_set() has to work with non-empty buffer + */ + pdcp_reorder_insert(reorder, mb, count); + + /* Get buffered packets */ + struct rte_mbuf **cached_mbufs = &out_mb[processed]; + uint32_t nb_cached = pdcp_reorder_get_sequential(reorder, + cached_mbufs, entity->max_pkt_cache - processed); + + processed += nb_cached; + } else { + out_mb[processed++] = mb; + } + + /* Processed should never exceed the window size */ + en_priv->state.rx_deliv = count + processed; + + } else { + if (!reorder->is_active) + /* Initialize reordering buffer with RX_DELIV */ + pdcp_reorder_start(reorder, en_priv->state.rx_deliv); + /* Buffer the packet */ + pdcp_reorder_insert(reorder, mb, count); + } + + /* Stop & reset current timer if rx_reord is received */ + if (t_reorder->state == TIMER_RUNNING && + en_priv->state.rx_deliv >= en_priv->state.rx_reord) { + t_reorder->state = TIMER_STOP; + /* Stop reorder buffer, only if it's empty */ + if (en_priv->state.rx_deliv == en_priv->state.rx_next) + pdcp_reorder_stop(reorder); + } + + /* + * If t-Reordering is not running (includes the case when t-Reordering is stopped due to + * actions above). + */ + if (t_reorder->state == TIMER_STOP && en_priv->state.rx_deliv < en_priv->state.rx_next) { + /* Update RX_REORD to RX_NEXT */ + en_priv->state.rx_reord = en_priv->state.rx_next; + /* Start t-Reordering */ + t_reorder->state = TIMER_RUNNING; + } + + return processed; } static inline uint16_t @@ -863,16 +926,12 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret, const bool is_integ_protected) { - struct entity_priv *en_priv = entity_priv_get(entity); - const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; + int i, nb_processed, nb_success = 0, nb_err = 0; rte_pdcp_dynfield_t *mb_dynfield; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -881,11 +940,12 @@ pdcp_post_process_uplane_dl_flags(const struct rte_pdcp_entity *entity, struct r mb_dynfield = pdcp_dynfield(mb); count = *mb_dynfield; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], is_integ_protected); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, is_integ_protected); - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -919,16 +979,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[], uint16_t num, uint16_t *nb_err_ret) { - struct entity_priv *en_priv = entity_priv_get(entity); - const uint32_t aad_sz = en_priv->aad_sz; - int i, nb_success = 0, nb_err = 0; + int i, nb_processed, nb_success = 0, nb_err = 0; rte_pdcp_dynfield_t *mb_dynfield; struct rte_mbuf *err_mb[num]; struct rte_mbuf *mb; uint32_t count; - const uint32_t hdr_trim_sz = en_priv->hdr_sz + aad_sz; - for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -937,12 +993,12 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, mb_dynfield = pdcp_dynfield(mb); count = *mb_dynfield; - if (unlikely(!pdcp_post_process_update_entity_state(entity, count))) + nb_processed = pdcp_post_process_update_entity_state( + entity, count, mb, &out_mb[nb_success], true); + if (nb_processed < 0) goto error; - pdcp_packet_strip(mb, hdr_trim_sz, true); - - out_mb[nb_success++] = mb; + nb_success += nb_processed; continue; error: @@ -1105,6 +1161,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent en_priv->flags.is_status_report_required = 1; } + /** + * flags.is_out_of_order_delivery + * + * Indicate whether the outoforder delivery is enabled for PDCP entity. + */ + en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** * hdr_sz * diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c new file mode 100644 index 0000000000..5399f0dc28 --- /dev/null +++ b/lib/pdcp/pdcp_reorder.c @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#include <rte_errno.h> +#include <rte_reorder.h> + +#include "pdcp_reorder.h" + +int +pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +{ + reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + if (reorder->buf == NULL) + return -rte_errno; + + reorder->window_size = window_size; + reorder->is_active = false; + + return 0; +} + +void +pdcp_reorder_destroy(const struct pdcp_reorder *reorder) +{ + rte_reorder_free(reorder->buf); +} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h new file mode 100644 index 0000000000..6a2f61d6ae --- /dev/null +++ b/lib/pdcp/pdcp_reorder.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(C) 2023 Marvell. + */ + +#ifndef PDCP_REORDER_H +#define PDCP_REORDER_H + +#include <rte_reorder.h> + +struct pdcp_reorder { + struct rte_reorder_buffer *buf; + uint32_t window_size; + bool is_active; +}; + +int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); +void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); + +static inline uint32_t +pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs) +{ + return rte_reorder_drain(reorder->buf, mbufs, max_mbufs); +} + +static inline uint32_t +pdcp_reorder_up_to_get(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, + uint32_t max_mbufs, uint32_t seqn) +{ + return rte_reorder_drain_up_to_seqn(reorder->buf, mbufs, max_mbufs, seqn); +} + +static inline void +pdcp_reorder_start(struct pdcp_reorder *reorder, uint32_t min_seqn) +{ + int ret; + + reorder->is_active = true; + + ret = rte_reorder_min_seqn_set(reorder->buf, min_seqn); + RTE_VERIFY(ret == 0); +} + +static inline void +pdcp_reorder_stop(struct pdcp_reorder *reorder) +{ + reorder->is_active = false; +} + +static inline void +pdcp_reorder_insert(struct pdcp_reorder *reorder, struct rte_mbuf *mbuf, + rte_reorder_seqn_t pkt_count) +{ + int ret; + + *rte_reorder_seqn(mbuf) = pkt_count; + + ret = rte_reorder_insert(reorder->buf, mbuf); + RTE_VERIFY(ret == 0); +} + +#endif /* PDCP_REORDER_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 96ad397667..be37ff392c 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -49,6 +49,17 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); } +static int +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +{ + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + + entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + + return pdcp_reorder_create(&dl->reorder, window_size); +} + struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { @@ -118,6 +129,12 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) if (ret) goto crypto_sess_destroy; + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { + ret = pdcp_dl_establish(entity, conf); + if (ret) + goto crypto_sess_destroy; + } + ret = pdcp_cnt_ring_create(entity, conf); if (ret) goto crypto_sess_destroy; @@ -132,26 +149,50 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } +static int +pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + int nb_out; + + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, + en_priv->state.rx_next); + + pdcp_reorder_destroy(&dl->reorder); + + return nb_out; +} + int rte_pdcp_entity_release(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv *en_priv; + int nb_out = 0; + if (pdcp_entity == NULL) return -EINVAL; + en_priv = entity_priv_get(pdcp_entity); + + if (!en_priv->flags.is_ul_entity) + nb_out = pdcp_dl_release(pdcp_entity, out_mb); + /* Teardown crypto sessions */ pdcp_crypto_sess_destroy(pdcp_entity); rte_free(pdcp_entity); - RTE_SET_USED(out_mb); - return 0; + return nb_out; } int rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, struct rte_mbuf *out_mb[]) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; + int nb_out = 0; if (pdcp_entity == NULL) return -EINVAL; @@ -161,13 +202,15 @@ rte_pdcp_entity_suspend(struct rte_pdcp_entity *pdcp_entity, if (en_priv->flags.is_ul_entity) { en_priv->state.tx_next = 0; } else { + dl = entity_dl_part_get(pdcp_entity); + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, pdcp_entity->max_pkt_cache, + en_priv->state.rx_next); + pdcp_reorder_stop(&dl->reorder); en_priv->state.rx_next = 0; en_priv->state.rx_deliv = 0; } - RTE_SET_USED(out_mb); - - return 0; + return nb_out; } struct rte_mbuf * diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index d7c2080358..9c4d06962a 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -110,6 +110,8 @@ struct rte_pdcp_entity_conf { * such as entity re-establishment. */ bool status_report_required; + /** Enable out of order delivery. */ + bool out_of_order_delivery; }; /* >8 End of structure rte_pdcp_entity_conf. */ @@ -269,8 +271,8 @@ rte_pdcp_pkt_pre_process(const struct rte_pdcp_entity *entity, * @param in_mb * The address of an array of *num* pointers to *rte_mbuf* structures. * @param[out] out_mb - * The address of an array of *num* pointers to *rte_mbuf* structures - * to output packets after PDCP post-processing. + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures to output packets after PDCP post-processing. * @param num * The maximum number of packets to process. * @param[out] nb_err -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 14/21] test/pdcp: add in-order delivery cases 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (12 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 15/21] pdcp: add timer callback handlers Anoob Joseph ` (7 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test cases to verify behaviour when in-order delivery is enabled and packets arrive in out-of-order. PDCP library is expected to buffer the packets and return packets in-order when the missing packet arrives. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 223 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index cfe2ec6aa9..24d7826bc2 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -16,6 +16,15 @@ #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 +/* Assert that condition is true, or goto the mark */ +#define ASSERT_TRUE_OR_GOTO(cond, mark, ...) do {\ + if (!(cond)) { \ + RTE_LOG(ERR, USER1, "Error at: %s:%d\n", __func__, __LINE__); \ + RTE_LOG(ERR, USER1, __VA_ARGS__); \ + goto mark; \ + } \ +} while (0) + /* According to formula(7.2.a Window_Size) */ #define PDCP_WINDOW_SIZE(sn_size) (1 << (sn_size - 1)) @@ -83,6 +92,38 @@ run_test_with_all_known_vec(const void *args) return run_test_foreach_known_vec(test, false); } +static int +run_test_with_all_known_vec_until_first_pass(const void *args) +{ + test_with_conf_t test = args; + + return run_test_foreach_known_vec(test, true); +} + +static inline uint32_t +pdcp_sn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return (1 << sn_size) - 1; +} + +static inline uint32_t +pdcp_sn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_sn_mask_get(sn_size)); +} + +static inline uint32_t +pdcp_hfn_mask_get(enum rte_security_pdcp_sn_size sn_size) +{ + return ~pdcp_sn_mask_get(sn_size); +} + +static inline uint32_t +pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) +{ + return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -416,6 +457,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.sess_mpool = ts_params->sess_pool; conf->entity.cop_pool = ts_params->cop_pool; + conf->entity.ctrl_pdu_pool = ts_params->mbuf_pool; conf->entity.pdcp_xfrm.bearer = pdcp_test_bearer[index]; conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; @@ -868,6 +910,7 @@ test_sn_range_type(enum sn_range_type type, struct pdcp_test_conf *conf) /* Configure Uplink to generate expected, encrypted packet */ pdcp_sn_to_raw_set(conf->input, new_sn, conf->entity.pdcp_xfrm.sn_size); + conf->entity.out_of_order_delivery = true; conf->entity.reverse_iv_direction = true; conf->entity.pdcp_xfrm.hfn = new_hfn; conf->entity.sn = new_sn; @@ -915,6 +958,171 @@ test_sn_minus_outside(struct pdcp_test_conf *t_conf) return test_sn_range_type(SN_RANGE_MINUS_OUTSIDE, t_conf); } +static struct rte_mbuf * +generate_packet_for_dl_with_sn(struct pdcp_test_conf ul_conf, uint32_t count) +{ + enum rte_security_pdcp_sn_size sn_size = ul_conf.entity.pdcp_xfrm.sn_size; + int ret; + + ul_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(count, sn_size); + ul_conf.entity.sn = pdcp_sn_from_count_get(count, sn_size); + ul_conf.entity.out_of_order_delivery = true; + ul_conf.entity.reverse_iv_direction = true; + ul_conf.output_len = 0; + + ret = test_attempt_single(&ul_conf); + if (ret != TEST_SUCCESS) + return NULL; + + return mbuf_from_data_create(ul_conf.output, ul_conf.output_len); +} + +static bool +array_asc_sorted_check(struct rte_mbuf *m[], uint32_t len, enum rte_security_pdcp_sn_size sn_size) +{ + uint32_t i; + + if (len < 2) + return true; + + for (i = 0; i < (len - 1); i++) { + if (pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i], void *), sn_size) > + pdcp_sn_from_raw_get(rte_pktmbuf_mtod(m[i + 1], void *), sn_size)) + return false; + } + + return true; +} + +static int +test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m0 = NULL, *m1 = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Generate packet to fill the existing gap */ + m0 = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m0 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1] + * Gap filled, all packets should be returned + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m0, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m0 = NULL; /* Packet was moved to out_mb */ + + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, + "Error occurred during packet drain\n"); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m0); + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + +static int +test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + const uint32_t window_size = PDCP_WINDOW_SIZE(sn_size); + struct rte_mbuf *m1 = NULL, **out_mb = NULL; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + const int rx_deliv = 0; + int ret = TEST_FAILED; + size_t i, nb_out; + uint8_t cdev_id; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(rx_deliv, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(rx_deliv, sn_size); + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + ASSERT_TRUE_OR_GOTO(pdcp_entity->max_pkt_cache >= window_size, exit, + "PDCP max packet cache is too small"); + cdev_id = dl_conf.entity.dev_id; + out_mb = rte_zmalloc(NULL, pdcp_entity->max_pkt_cache * sizeof(uintptr_t), 0); + ASSERT_TRUE_OR_GOTO(out_mb != NULL, exit, + "Could not allocate buffer for holding out_mb buffers\n"); + + /* Send packets with SN > RX_DELIV to create a gap */ + for (i = rx_deliv + 1; i < window_size; i++) { + m1 = generate_packet_for_dl_with_sn(*ul_conf, i); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Buffered packets after insert [NULL, m1] */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + } + + m1 = generate_packet_for_dl_with_sn(*ul_conf, rx_deliv); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + /* Insert missing packet */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == window_size, exit, + "Packet count mismatch (received: %i, expected: %i)\n", + nb_success, window_size); + m1 = NULL; + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + rte_free(out_mb); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -971,10 +1179,25 @@ static struct unit_test_suite hfn_sn_test_cases = { } }; +static struct unit_test_suite reorder_test_cases = { + .suite_name = "PDCP reorder", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_reorder_buffer_full_window_size_sn_12), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, + &reorder_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 15/21] pdcp: add timer callback handlers 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (13 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 14/21] test/pdcp: add in-order delivery cases Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 16/21] pdcp: add timer expiry handle Anoob Joseph ` (6 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> PDCP has a windowing mechanism which allows only packets that fall in a reception window. The pivot point for this window is RX_REORD which happens to be the first missing or next expected packet. If the missing packet is not received after a specified time, then the RX_REORD state variable needs to be moved up to slide the reception window. PDCP relies on timers for such operations. The timer needs to be armed when PDCP library doesn't receive all packets in-order and starts buffering packets that arrived after a missing packet. The timer needs to be cancelled when a missing packet is received. To avoid dependency on particular timer implementation, PDCP library allows application to register two callbacks, timer_start() and timer_stop() that will be called later by library. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_entity.h | 2 ++ lib/pdcp/pdcp_process.c | 2 ++ lib/pdcp/rte_pdcp.c | 1 + lib/pdcp/rte_pdcp.h | 47 +++++++++++++++++++++++++++++++++++++++++ 4 files changed, 52 insertions(+) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 34341cdc11..efc74ba9b9 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -120,6 +120,8 @@ enum timer_state { struct pdcp_t_reordering { /** Represent timer state */ enum timer_state state; + /** User defined callback handles */ + struct rte_pdcp_t_reordering handle; }; struct pdcp_cnt_bitmap { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 84a0f3a43f..daf2c27363 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -902,6 +902,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (t_reorder->state == TIMER_RUNNING && en_priv->state.rx_deliv >= en_priv->state.rx_reord) { t_reorder->state = TIMER_STOP; + t_reorder->handle.stop(t_reorder->handle.timer, t_reorder->handle.args); /* Stop reorder buffer, only if it's empty */ if (en_priv->state.rx_deliv == en_priv->state.rx_next) pdcp_reorder_stop(reorder); @@ -916,6 +917,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, en_priv->state.rx_reord = en_priv->state.rx_next; /* Start t-Reordering */ t_reorder->state = TIMER_RUNNING; + t_reorder->handle.start(t_reorder->handle.timer, t_reorder->handle.args); } return processed; diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index be37ff392c..a0558b99ae 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -56,6 +56,7 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct entity_priv_dl_part *dl = entity_dl_part_get(entity); entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); + dl->t_reorder.handle = conf->t_reordering; return pdcp_reorder_create(&dl->reorder, window_size); } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 9c4d06962a..9cdce7d3a4 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -68,6 +68,51 @@ struct rte_pdcp_entity { uint32_t max_pkt_cache; } __rte_cache_aligned; +/** + * Callback function type for t-Reordering timer start, set during PDCP entity establish. + * This callback is invoked by PDCP library, during t-Reordering timer start event. + * Only one t-Reordering per receiving PDCP entity would be running at a given time. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_start_cb_t)(void *timer, void *args); + +/** + * Callback function type for t-Reordering timer stop, set during PDCP entity establish. + * This callback will be invoked by PDCP library, during t-Reordering timer stop event. + * + * @see struct rte_pdcp_timer + * @see rte_pdcp_entity_establish() + * + * @param timer + * Pointer to timer. + * @param args + * Pointer to timer arguments. + */ +typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); + +/** + * PDCP t-Reordering timer interface + * + * Configuration provided by user, that PDCP library will invoke according to timer behaviour. + */ +struct rte_pdcp_t_reordering { + /** Timer pointer, to be used in callback functions. */ + void *timer; + /** Timer arguments, to be used in callback functions. */ + void *args; + /** Timer start callback handle. */ + rte_pdcp_t_reordering_start_cb_t start; + /** Timer stop callback handle. */ + rte_pdcp_t_reordering_stop_cb_t stop; +}; + /** * PDCP entity configuration to be used for establishing an entity. */ @@ -112,6 +157,8 @@ struct rte_pdcp_entity_conf { bool status_report_required; /** Enable out of order delivery. */ bool out_of_order_delivery; + /** t-Reordering timer configuration. */ + struct rte_pdcp_t_reordering t_reordering; }; /* >8 End of structure rte_pdcp_entity_conf. */ -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 16/21] pdcp: add timer expiry handle 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (14 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 15/21] pdcp: add timer callback handlers Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 17/21] test/pdcp: add timer expiry cases Anoob Joseph ` (5 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> The PDCP protocol requires usage of timers to keep track of how long an out-of-order packet should be buffered while waiting for missing packets. Applications can register a desired timer implementation with the PDCP library. Once the timer expires, the application will be notified, and further handling of the event will be performed in the PDCP library. When the timer expires, the PDCP library will return the cached packets, and PDCP internal state variables (like RX_REORD, RX_DELIV etc) will be updated accordingly. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- doc/guides/prog_guide/pdcp_lib.rst | 31 +++++++++++++++++++ lib/pdcp/rte_pdcp.c | 49 ++++++++++++++++++++++++++++++ lib/pdcp/rte_pdcp.h | 34 +++++++++++++++++++-- lib/pdcp/version.map | 2 ++ 4 files changed, 114 insertions(+), 2 deletions(-) diff --git a/doc/guides/prog_guide/pdcp_lib.rst b/doc/guides/prog_guide/pdcp_lib.rst index a925aa7f14..61242edf92 100644 --- a/doc/guides/prog_guide/pdcp_lib.rst +++ b/doc/guides/prog_guide/pdcp_lib.rst @@ -130,6 +130,37 @@ Supported integrity protection algorithms - RTE_CRYPTO_AUTH_SNOW3G_UIA2 - RTE_CRYPTO_AUTH_ZUC_EIA3 +Timers +------ + +PDCP utilizes a reception window mechanism to limit the bits of COUNT value +transmitted in the packet. It utilizes state variables such as RX_REORD, +RX_DELIV to define the window and uses RX_DELIV as the lower pivot point of the +window. + +RX_DELIV would be updated only when packets are received in-order. +Any missing packet would mean RX_DELIV won't be updated. +A timer, t-Reordering, helps PDCP to slide the window +if the missing packet is not received in a specified time duration. + +While starting and stopping the timer will be done by lib PDCP, +application could register its own timer implementation. +This is to make sure application can choose between timers +such as rte_timer and rte_event based timers. +Starting and stopping of timer would happen during pre & post process API. + +When the t-Reordering timer expires, application would receive the expiry event. +To perform the PDCP handling of the expiry event, +``rte_pdcp_t_reordering_expiry_handle`` can be used. +Expiry handling would involve sliding the window by updating state variables and +passing the expired packets to the application. + +.. literalinclude:: ../../../lib/pdcp/rte_pdcp.h + :language: c + :start-after: Structure rte_pdcp_t_reordering 8< + :end-before: >8 End of structure rte_pdcp_t_reordering. + + Sample API usage ---------------- diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index a0558b99ae..819c66bd08 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -251,3 +251,52 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, return m; } + +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + uint16_t capacity = entity->max_pkt_cache; + uint16_t nb_out, nb_seq; + + /* 5.2.2.2 Actions when a t-Reordering expires */ + + /* + * - deliver to upper layers in ascending order of the associated COUNT value after + * performing header decompression, if not decompressed before: + */ + + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ + nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); + capacity -= nb_out; + out_mb = &out_mb[nb_out]; + + /* + * - all stored PDCP SDU(s) with consecutively associated COUNT value(s) starting from + * RX_REORD; + */ + nb_seq = pdcp_reorder_get_sequential(&dl->reorder, out_mb, capacity); + nb_out += nb_seq; + + /* + * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered + * to upper layers, with COUNT value >= RX_REORD; + */ + en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + + /* + * - if RX_DELIV < RX_NEXT: + * - update RX_REORD to RX_NEXT; + * - start t-Reordering. + */ + if (en_priv->state.rx_deliv < en_priv->state.rx_next) { + en_priv->state.rx_reord = en_priv->state.rx_next; + dl->t_reorder.state = TIMER_RUNNING; + dl->t_reorder.handle.start(dl->t_reorder.handle.timer, dl->t_reorder.handle.args); + } else { + dl->t_reorder.state = TIMER_EXPIRED; + } + + return nb_out; +} diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 9cdce7d3a4..c5b99dfa2b 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -102,6 +102,7 @@ typedef void (*rte_pdcp_t_reordering_stop_cb_t)(void *timer, void *args); * * Configuration provided by user, that PDCP library will invoke according to timer behaviour. */ +/* Structure rte_pdcp_t_reordering 8< */ struct rte_pdcp_t_reordering { /** Timer pointer, to be used in callback functions. */ void *timer; @@ -112,6 +113,7 @@ struct rte_pdcp_t_reordering { /** Timer stop callback handle. */ rte_pdcp_t_reordering_stop_cb_t stop; }; +/* >8 End of structure rte_pdcp_t_reordering. */ /** * PDCP entity configuration to be used for establishing an entity. @@ -338,8 +340,36 @@ rte_pdcp_pkt_post_process(const struct rte_pdcp_entity *entity, } /** - * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. So include - * in the end. + * @warning + * @b EXPERIMENTAL: this API may change without prior notice + * + * 5.2.2.2 Actions when a t-Reordering expires + * + * When t-Reordering timer expires, PDCP is required to slide the reception + * window by updating state variables such as RX_REORD & RX_DELIV. + * PDCP would need to deliver some of the buffered packets + * based on the state variables and conditions described. + * + * The expiry handle need to be invoked by the application when t-Reordering + * timer expires. In addition to returning buffered packets, it may also restart + * timer based on the state variables. + * + * @param entity + * Pointer to the *rte_pdcp_entity* for which the timer expired. + * @param[out] out_mb + * The address of an array that can hold up to *rte_pdcp_entity.max_pkt_cache* + * pointers to *rte_mbuf* structures. Used to return buffered packets that are expired. + * @return + * Number of packets returned in *out_mb* buffer. + */ +__rte_experimental +uint16_t +rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, + struct rte_mbuf *out_mb[]); + +/** + * The header 'rte_pdcp_group.h' depends on defines in 'rte_pdcp.h'. + * So include in the end. */ #include <rte_pdcp_group.h> diff --git a/lib/pdcp/version.map b/lib/pdcp/version.map index 97171f902e..4fd912fac0 100644 --- a/lib/pdcp/version.map +++ b/lib/pdcp/version.map @@ -14,5 +14,7 @@ EXPERIMENTAL { rte_pdcp_pkt_pre_process; rte_pdcp_pkt_crypto_group; + rte_pdcp_t_reordering_expiry_handle; + local: *; }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 17/21] test/pdcp: add timer expiry cases 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (15 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 16/21] pdcp: add timer expiry handle Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 18/21] test/pdcp: add timer restart case Anoob Joseph ` (4 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test cases for handling the expiry with rte_timer and rte_event_timer. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 350 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 350 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 24d7826bc2..25729b2bdd 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -3,15 +3,24 @@ */ #include <rte_errno.h> +#ifdef RTE_LIB_EVENTDEV +#include <rte_eventdev.h> +#include <rte_event_timer_adapter.h> +#endif /* RTE_LIB_EVENTDEV */ #include <rte_malloc.h> #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include <rte_timer.h> #include "test.h" #include "test_cryptodev.h" #include "test_cryptodev_security_pdcp_test_vectors.h" +#define NSECPERSEC 1E9 #define NB_DESC 1024 +#define TIMER_ADAPTER_ID 0 +#define TEST_EV_QUEUE_ID 0 +#define TEST_EV_PORT_ID 0 #define CDEV_INVALID_ID UINT8_MAX #define NB_TESTS RTE_DIM(pdcp_test_params) #define PDCP_IV_LEN 16 @@ -33,10 +42,21 @@ struct pdcp_testsuite_params { struct rte_mempool *cop_pool; struct rte_mempool *sess_pool; bool cdevs_used[RTE_CRYPTO_MAX_DEVS]; + int evdev; +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev; +#endif /* RTE_LIB_EVENTDEV */ + bool timer_is_running; + uint64_t min_resolution_ns; }; static struct pdcp_testsuite_params testsuite_params; +struct test_rte_timer_args { + int status; + struct rte_pdcp_entity *pdcp_entity; +}; + struct pdcp_test_conf { struct rte_pdcp_entity_conf entity; struct rte_crypto_sym_xform c_xfrm; @@ -124,6 +144,30 @@ pdcp_hfn_from_count_get(uint32_t count, enum rte_security_pdcp_sn_size sn_size) return (count & pdcp_hfn_mask_get(sn_size)) >> sn_size; } +static void +pdcp_timer_start_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = true; +} + +static void +pdcp_timer_stop_cb(void *timer, void *args) +{ + bool *is_timer_running = timer; + + RTE_SET_USED(args); + *is_timer_running = false; +} + +static struct rte_pdcp_t_reordering t_reorder_timer = { + .timer = &testsuite_params.timer_is_running, + .start = pdcp_timer_start_cb, + .stop = pdcp_timer_stop_cb, +}; + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -462,6 +506,7 @@ create_test_conf_from_index(const int index, struct pdcp_test_conf *conf) conf->entity.pdcp_xfrm.en_ordering = 0; conf->entity.pdcp_xfrm.remove_duplicates = 0; conf->entity.pdcp_xfrm.domain = pdcp_test_params[index].domain; + conf->entity.t_reordering = t_reorder_timer; if (pdcp_test_packet_direction[index] == PDCP_DIR_UPLINK) conf->entity.pdcp_xfrm.pkt_dir = RTE_SECURITY_PDCP_UPLINK; @@ -1048,6 +1093,8 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) /* Check that packets in correct order */ ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == false, exit, + "Timer should be stopped after full drain\n"); ret = TEST_SUCCESS; exit: @@ -1123,6 +1170,181 @@ test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static void +event_timer_start_cb(void *timer, void *args) +{ + struct rte_event_timer *evtims = args; + int ret = 0; + + ret = rte_event_timer_arm_burst(timer, &evtims, 1); + assert(ret == 1); +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +test_expiry_with_event_timer(const struct pdcp_test_conf *ul_conf) +{ +#ifdef RTE_LIB_EVENTDEV + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_event event; + + const int start_count = 0; + struct rte_event_timer evtim = { + .ev.op = RTE_EVENT_OP_NEW, + .ev.queue_id = TEST_EV_QUEUE_ID, + .ev.sched_type = RTE_SCHED_TYPE_ATOMIC, + .ev.priority = RTE_EVENT_DEV_PRIORITY_NORMAL, + .ev.event_type = RTE_EVENT_TYPE_TIMER, + .state = RTE_EVENT_TIMER_NOT_ARMED, + .timeout_ticks = 1, + }; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &evtim; + dl_conf.entity.t_reordering.timer = testsuite_params.timdev; + dl_conf.entity.t_reordering.start = event_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + evtim.ev.event_ptr = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + while (n != 1) { + rte_delay_us(testsuite_params.min_resolution_ns / 1000); + n = rte_event_dequeue_burst(testsuite_params.evdev, TEST_EV_PORT_ID, &event, 1, 0); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, + "Dequeued unexpected timer expiry event: %i\n", n); + } + + ASSERT_TRUE_OR_GOTO(event.event_type == RTE_EVENT_TYPE_TIMER, exit, "Unexpected event type\n"); + + /* Handle expiry event */ + n = rte_pdcp_t_reordering_expiry_handle(event.event_ptr, out_mb); + ASSERT_TRUE_OR_GOTO(n == 1, exit, "Unexpected number of expired packets :%i\n", n); + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +#else + RTE_SET_USED(ul_conf); + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +test_rte_timer_expiry_handle(struct rte_timer *timer_handle, void *arg) +{ + struct test_rte_timer_args *timer_data = arg; + struct rte_mbuf *out_mb[1] = {0}; + uint16_t n; + + RTE_SET_USED(timer_handle); + + n = rte_pdcp_t_reordering_expiry_handle(timer_data->pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, n); + + timer_data->status = n == 1 ? n : -1; +} + +static void +test_rte_timer_start_cb(void *timer, void *args) +{ + rte_timer_reset_sync(timer, 1, SINGLE, rte_lcore_id(), test_rte_timer_expiry_handle, args); +} + +static int +test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m1 = NULL, *out_mb[1] = {0}; + uint16_t n = 0, nb_err = 0, nb_try = 5; + struct test_rte_timer_args timer_args; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + struct rte_timer timer = {0}; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Set up a timer */ + rte_timer_init(&timer); + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.t_reordering.args = &timer_args; + dl_conf.entity.t_reordering.timer = &timer; + dl_conf.entity.t_reordering.start = test_rte_timer_start_cb; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + timer_args.status = 0; + timer_args.pdcp_entity = pdcp_entity; + + /* Send packet with SN > RX_DELIV to create a gap */ + m1 = generate_packet_for_dl_with_sn(*ul_conf, start_count + 1); + ASSERT_TRUE_OR_GOTO(m1 != NULL, exit, "Could not allocate buffer for packet\n"); + + /* Buffered packets after insert [NULL, m1] */ + n = test_process_packets(pdcp_entity, dl_conf.entity.dev_id, &m1, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(n == 0, exit, "Packet was not buffered as expected\n"); + + m1 = NULL; /* Packet was moved to PDCP lib */ + + /* Verify that expire was handled correctly */ + rte_timer_manage(); + while (timer_args.status != 1) { + rte_delay_us(1); + rte_timer_manage(); + ASSERT_TRUE_OR_GOTO(nb_try-- > 0, exit, "Bad expire handle status %i\n", + timer_args.status); + } + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m1); + rte_pktmbuf_free_bulk(out_mb, n); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1145,6 +1367,126 @@ test_combined(struct pdcp_test_conf *ul_conf) return ret; } +#ifdef RTE_LIB_EVENTDEV +static inline void +eventdev_conf_default_set(struct rte_event_dev_config *dev_conf, struct rte_event_dev_info *info) +{ + memset(dev_conf, 0, sizeof(struct rte_event_dev_config)); + dev_conf->dequeue_timeout_ns = info->min_dequeue_timeout_ns; + dev_conf->nb_event_ports = 1; + dev_conf->nb_event_queues = 1; + dev_conf->nb_event_queue_flows = info->max_event_queue_flows; + dev_conf->nb_event_port_dequeue_depth = info->max_event_port_dequeue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_event_port_enqueue_depth = info->max_event_port_enqueue_depth; + dev_conf->nb_events_limit = info->max_num_events; +} + +static inline int +eventdev_setup(void) +{ + struct rte_event_dev_config dev_conf; + struct rte_event_dev_info info; + int ret, evdev = 0; + + if (!rte_event_dev_count()) + return TEST_SKIPPED; + + ret = rte_event_dev_info_get(evdev, &info); + TEST_ASSERT_SUCCESS(ret, "Failed to get event dev info"); + TEST_ASSERT(info.max_num_events < 0 || info.max_num_events >= 1, + "ERROR max_num_events=%d < max_events=%d", info.max_num_events, 1); + + eventdev_conf_default_set(&dev_conf, &info); + ret = rte_event_dev_configure(evdev, &dev_conf); + TEST_ASSERT_SUCCESS(ret, "Failed to configure eventdev"); + + ret = rte_event_queue_setup(evdev, TEST_EV_QUEUE_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup queue=%d", TEST_EV_QUEUE_ID); + + /* Configure event port */ + ret = rte_event_port_setup(evdev, TEST_EV_PORT_ID, NULL); + TEST_ASSERT_SUCCESS(ret, "Failed to setup port=%d", TEST_EV_PORT_ID); + ret = rte_event_port_link(evdev, TEST_EV_PORT_ID, NULL, NULL, 0); + TEST_ASSERT(ret >= 0, "Failed to link all queues port=%d", TEST_EV_PORT_ID); + + ret = rte_event_dev_start(evdev); + TEST_ASSERT_SUCCESS(ret, "Failed to start device"); + + testsuite_params.evdev = evdev; + + return TEST_SUCCESS; +} + +static int +event_timer_setup(void) +{ + struct rte_event_timer_adapter_info info; + struct rte_event_timer_adapter *timdev; + uint32_t caps = 0; + + struct rte_event_timer_adapter_conf config = { + .event_dev_id = testsuite_params.evdev, + .timer_adapter_id = TIMER_ADAPTER_ID, + .timer_tick_ns = NSECPERSEC, + .max_tmo_ns = 10 * NSECPERSEC, + .nb_timers = 10, + .flags = 0, + }; + + TEST_ASSERT_SUCCESS(rte_event_timer_adapter_caps_get(testsuite_params.evdev, &caps), + "Failed to get adapter capabilities"); + + if (!(caps & RTE_EVENT_TIMER_ADAPTER_CAP_INTERNAL_PORT)) + return TEST_SKIPPED; + + timdev = rte_event_timer_adapter_create(&config); + + TEST_ASSERT_NOT_NULL(timdev, "Failed to create event timer ring"); + + testsuite_params.timdev = timdev; + + TEST_ASSERT_EQUAL(rte_event_timer_adapter_start(timdev), 0, + "Failed to start event timer adapter"); + + rte_event_timer_adapter_get_info(timdev, &info); + testsuite_params.min_resolution_ns = info.min_resolution_ns; + + return TEST_SUCCESS; +} +#endif /* RTE_LIB_EVENTDEV */ + +static int +ut_setup_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + int ret; + + ret = eventdev_setup(); + if (ret) + return ret; + + return event_timer_setup(); +#else + return TEST_SKIPPED; +#endif /* RTE_LIB_EVENTDEV */ +} + +static void +ut_teardown_pdcp_event_timer(void) +{ +#ifdef RTE_LIB_EVENTDEV + struct rte_event_timer_adapter *timdev = testsuite_params.timdev; + int evdev = testsuite_params.evdev; + + rte_event_dev_stop(evdev); + rte_event_dev_close(evdev); + + rte_event_timer_adapter_stop(timdev); + rte_event_timer_adapter_free(timdev); +#endif /* RTE_LIB_EVENTDEV */ +} + static int run_test_for_one_known_vec(const void *arg) { @@ -1189,6 +1531,14 @@ static struct unit_test_suite reorder_test_cases = { ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, test_reorder_buffer_full_window_size_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_event_timer", + ut_setup_pdcp_event_timer, ut_teardown_pdcp_event_timer, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_event_timer), + TEST_CASE_NAMED_WITH_DATA("test_expire_with_rte_timer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_expiry_with_rte_timer), TEST_CASES_END() /**< NULL terminate unit test array */ } }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 18/21] test/pdcp: add timer restart case 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (16 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 17/21] test/pdcp: add timer expiry cases Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 19/21] pdcp: add support for status report Anoob Joseph ` (3 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Add test to cover the case when t-reordering timer should be restarted on the same packet. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 68 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 25729b2bdd..82cc25ec7a 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -1106,6 +1106,71 @@ test_reorder_gap_fill(struct pdcp_test_conf *ul_conf) return ret; } +static int +test_reorder_gap_in_reorder_buffer(const struct pdcp_test_conf *ul_conf) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *m = NULL, *out_mb[2] = {0}; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + int ret = TEST_FAILED, nb_out, i; + struct pdcp_test_conf dl_conf; + uint8_t cdev_id; + + const int start_count = 0; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + + /* Create two gaps [NULL, m1, NULL, m3]*/ + for (i = 0; i < 2; i++) { + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + 2 * i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + m = NULL; /* Packet was moved to PDCP lib */ + } + + /* Generate packet to fill the first gap */ + m = generate_packet_for_dl_with_sn(*ul_conf, start_count); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + /* + * Buffered packets after insert [m0, m1, NULL, m3] + * Only first gap should be filled, timer should be restarted for second gap + */ + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet process\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 2, exit, + "Packet count mismatch (received: %i, expected: 2)\n", nb_success); + m = NULL; + /* Check that packets in correct order */ + ASSERT_TRUE_OR_GOTO(array_asc_sorted_check(out_mb, nb_success, sn_size), + exit, "Error occurred during packet drain\n"); + ASSERT_TRUE_OR_GOTO(testsuite_params.timer_is_running == true, exit, + "Timer should be restarted after partial drain"); + + + ret = TEST_SUCCESS; +exit: + rte_pktmbuf_free(m); + rte_pktmbuf_free_bulk(out_mb, nb_success); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + return ret; +} + static int test_reorder_buffer_full_window_size_sn_12(const struct pdcp_test_conf *ul_conf) { @@ -1527,6 +1592,9 @@ static struct unit_test_suite reorder_test_cases = { TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_fill", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec, test_reorder_gap_fill), + TEST_CASE_NAMED_WITH_DATA("test_reorder_gap_in_reorder_buffer", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_reorder_gap_in_reorder_buffer), TEST_CASE_NAMED_WITH_DATA("test_reorder_buffer_full_window_size_sn_12", ut_setup_pdcp, ut_teardown_pdcp, run_test_with_all_known_vec_until_first_pass, -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 19/21] pdcp: add support for status report 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (17 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 18/21] test/pdcp: add timer restart case Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph ` (2 subsequent siblings) 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Implement status report generation for PDCP entity. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 158 ++++++++++++++++++++++++++++++++++++--- lib/pdcp/pdcp_cnt.h | 11 ++- lib/pdcp/pdcp_ctrl_pdu.c | 34 ++++++++- lib/pdcp/pdcp_ctrl_pdu.h | 3 +- lib/pdcp/pdcp_entity.h | 2 + lib/pdcp/pdcp_process.c | 9 ++- lib/pdcp/pdcp_process.h | 13 ++++ lib/pdcp/rte_pdcp.c | 34 ++++++--- 8 files changed, 236 insertions(+), 28 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index c9b952184b..af027b00d3 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -2,28 +2,164 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_pdcp.h> #include "pdcp_cnt.h" +#include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#define SLAB_BYTE_SIZE (RTE_BITMAP_SLAB_BIT_SIZE / 8) + +uint32_t +pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) +{ + uint32_t n_bits = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + + return rte_bitmap_get_memory_footprint(n_bits); +} + int -pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) { - struct entity_priv_dl_part *en_priv_dl; - uint32_t window_sz; + uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - if (en == NULL || conf == NULL) + dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + if (dl->bitmap.bmp == NULL) return -EINVAL; - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) - return 0; + dl->bitmap.size = window_size; - en_priv_dl = entity_dl_part_get(en); - window_sz = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); + return 0; +} - RTE_SET_USED(window_sz); - RTE_SET_USED(en_priv_dl); +void +pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + rte_bitmap_set(bitmap.bmp, count % bitmap.size); +} - return 0; +bool +pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count) +{ + return rte_bitmap_get(bitmap.bmp, count % bitmap.size); +} + +void +pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop) +{ + uint32_t i; + + for (i = start; i < stop; i++) + rte_bitmap_clear(bitmap.bmp, i % bitmap.size); +} + +uint16_t +pdcp_cnt_get_bitmap_size(uint32_t pending_bytes) +{ + /* + * Round up bitmap size to slab size to operate only on slabs sizes, instead of individual + * bytes + */ + return RTE_ALIGN_MUL_CEIL(pending_bytes, SLAB_BYTE_SIZE); +} + +static __rte_always_inline uint64_t +leftover_get(uint64_t slab, uint32_t shift, uint64_t mask) +{ + return (slab & mask) << shift; +} + +void +pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len) +{ + uint64_t slab = 0, next_slab = 0, leftover; + uint32_t zeros, report_len, diff; + uint32_t slab_id, next_slab_id; + uint32_t pos = 0, next_pos = 0; + + const uint32_t start_count = state.rx_deliv + 1; + const uint32_t nb_slabs = bitmap.size / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t nb_data_slabs = data_len / SLAB_BYTE_SIZE; + const uint32_t start_slab_id = start_count / RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t stop_slab_id = (start_slab_id + nb_data_slabs) % nb_slabs; + const uint32_t shift = start_count % RTE_BITMAP_SLAB_BIT_SIZE; + const uint32_t leftover_shift = shift ? RTE_BITMAP_SLAB_BIT_SIZE - shift : 0; + const uint8_t *data_end = RTE_PTR_ADD(data, data_len + SLAB_BYTE_SIZE); + + /* NOTE: Mask required to workaround case - when shift is not needed */ + const uint64_t leftover_mask = shift ? ~0 : 0; + + /* NOTE: implement scan init at to set custom position */ + __rte_bitmap_scan_init(bitmap.bmp); + while (true) { + assert(rte_bitmap_scan(bitmap.bmp, &pos, &slab) == 1); + slab_id = pos / RTE_BITMAP_SLAB_BIT_SIZE; + if (slab_id >= start_slab_id) + break; + } + + report_len = nb_data_slabs; + + if (slab_id > start_slab_id) { + /* Zero slabs at beginning */ + zeros = (slab_id - start_slab_id - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + leftover = leftover_get(slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + report_len -= (slab_id - start_slab_id); + } + + while (report_len) { + rte_bitmap_scan(bitmap.bmp, &next_pos, &next_slab); + next_slab_id = next_pos / RTE_BITMAP_SLAB_BIT_SIZE; + diff = (next_slab_id + nb_slabs - slab_id) % nb_slabs; + + /* If next_slab_id == slab_id - overlap */ + diff += !(next_slab_id ^ slab_id) * nb_slabs; + + /* Size check - next slab is outsize of size range */ + if (diff > report_len) { + next_slab = 0; + next_slab_id = stop_slab_id; + diff = report_len; + } + + report_len -= diff; + + /* Calculate gap between slabs, taking wrap around into account */ + zeros = (next_slab_id + nb_slabs - slab_id - 1) % nb_slabs; + if (zeros) { + /* Non continues slabs, align them individually */ + slab >>= shift; + memcpy(data, &slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + + /* Fill zeros between slabs */ + zeros = (zeros - 1) * SLAB_BYTE_SIZE; + memset(data, 0, zeros); + data = RTE_PTR_ADD(data, zeros); + + /* Align beginning of next slab */ + leftover = leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &leftover, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } else { + /* Continues slabs, combine them */ + uint64_t new_slab = (slab >> shift) | + leftover_get(next_slab, leftover_shift, leftover_mask); + memcpy(data, &new_slab, SLAB_BYTE_SIZE); + data = RTE_PTR_ADD(data, SLAB_BYTE_SIZE); + } + + slab = next_slab; + pos = next_pos; + slab_id = next_slab_id; + + }; + + assert(data < data_end); } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index bbda478b55..5941b7a406 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -9,6 +9,15 @@ #include "pdcp_entity.h" -int pdcp_cnt_ring_create(struct rte_pdcp_entity *en, const struct rte_pdcp_entity_conf *conf); +uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); + +void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); +void pdcp_cnt_bitmap_range_clear(struct pdcp_cnt_bitmap bitmap, uint32_t start, uint32_t stop); + +uint16_t pdcp_cnt_get_bitmap_size(uint32_t pending_bytes); +void pdcp_cnt_report_fill(struct pdcp_cnt_bitmap bitmap, struct entity_state state, + uint8_t *data, uint16_t data_len); #endif /* PDCP_CNT_H */ diff --git a/lib/pdcp/pdcp_ctrl_pdu.c b/lib/pdcp/pdcp_ctrl_pdu.c index feb05fd863..e0ac2d3720 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.c +++ b/lib/pdcp/pdcp_ctrl_pdu.c @@ -8,6 +8,14 @@ #include "pdcp_ctrl_pdu.h" #include "pdcp_entity.h" +#include "pdcp_cnt.h" + +static inline uint16_t +round_up_bits(uint32_t bits) +{ + /* round up to the next multiple of 8 */ + return RTE_ALIGN_MUL_CEIL(bits, 8) / 8; +} static __rte_always_inline void pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) @@ -19,11 +27,13 @@ pdcp_hdr_fill(struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr, uint32_t rx_deliv) } int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m) { struct rte_pdcp_up_ctrl_pdu_hdr *pdu_hdr; - uint32_t rx_deliv; - int pdu_sz; + uint32_t rx_deliv, actual_sz; + uint16_t pdu_sz, bitmap_sz; + uint8_t *data; if (!en_priv->flags.is_status_report_required) return -EINVAL; @@ -42,5 +52,21 @@ pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m) return 0; } - return -ENOTSUP; + actual_sz = RTE_MIN(round_up_bits(en_priv->state.rx_next - rx_deliv - 1), + RTE_PDCP_CTRL_PDU_SIZE_MAX - pdu_sz); + bitmap_sz = pdcp_cnt_get_bitmap_size(actual_sz); + + data = (uint8_t *)rte_pktmbuf_append(m, pdu_sz + bitmap_sz); + if (data == NULL) + return -ENOMEM; + + m->pkt_len = pdu_sz + actual_sz; + m->data_len = pdu_sz + actual_sz; + + pdcp_hdr_fill((struct rte_pdcp_up_ctrl_pdu_hdr *)data, rx_deliv); + + data = RTE_PTR_ADD(data, pdu_sz); + pdcp_cnt_report_fill(dl->bitmap, en_priv->state, data, bitmap_sz); + + return 0; } diff --git a/lib/pdcp/pdcp_ctrl_pdu.h b/lib/pdcp/pdcp_ctrl_pdu.h index a2424fbd10..2a87928b88 100644 --- a/lib/pdcp/pdcp_ctrl_pdu.h +++ b/lib/pdcp/pdcp_ctrl_pdu.h @@ -10,6 +10,7 @@ #include "pdcp_entity.h" int -pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct rte_mbuf *m); +pdcp_ctrl_pdu_status_gen(struct entity_priv *en_priv, struct entity_priv_dl_part *dl, + struct rte_mbuf *m); #endif /* PDCP_CTRL_PDU_H */ diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index efc74ba9b9..a9b1428c7a 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -182,6 +182,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /** Bitmap memory region */ + uint8_t bitmap_mem[0]; }; struct entity_priv_ul_part { diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index daf2c27363..774f5646d8 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -10,6 +10,7 @@ #include <rte_pdcp.h> #include <rte_pdcp_hdr.h> +#include "pdcp_cnt.h" #include "pdcp_crypto.h" #include "pdcp_entity.h" #include "pdcp_process.h" @@ -857,11 +858,15 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, if (count >= en_priv->state.rx_next) en_priv->state.rx_next = count + 1; + if (unlikely(pdcp_cnt_bitmap_is_set(dl->bitmap, count))) + return -EEXIST; + + pdcp_cnt_bitmap_set(dl->bitmap, count); pdcp_packet_strip(mb, hdr_trim_sz, trim_mac); if (en_priv->flags.is_out_of_order_delivery) { out_mb[0] = mb; - en_priv->state.rx_deliv = count + 1; + pdcp_rx_deliv_set(entity, count + 1); return 1; } @@ -888,7 +893,7 @@ pdcp_post_process_update_entity_state(const struct rte_pdcp_entity *entity, } /* Processed should never exceed the window size */ - en_priv->state.rx_deliv = count + processed; + pdcp_rx_deliv_set(entity, count + processed); } else { if (!reorder->is_active) diff --git a/lib/pdcp/pdcp_process.h b/lib/pdcp/pdcp_process.h index a52f769b82..fa3d93b405 100644 --- a/lib/pdcp/pdcp_process.h +++ b/lib/pdcp/pdcp_process.h @@ -8,6 +8,9 @@ #include <rte_mbuf_dyn.h> #include <rte_pdcp.h> +#include <pdcp_entity.h> +#include <pdcp_cnt.h> + typedef uint32_t rte_pdcp_dynfield_t; extern int rte_pdcp_dynfield_offset; @@ -21,4 +24,14 @@ pdcp_dynfield(struct rte_mbuf *mbuf) int pdcp_process_func_set(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf); +static inline void +pdcp_rx_deliv_set(const struct rte_pdcp_entity *entity, uint32_t rx_deliv) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + pdcp_cnt_bitmap_range_clear(dl->bitmap, en_priv->state.rx_deliv, rx_deliv); + en_priv->state.rx_deliv = rx_deliv; +} + #endif /* PDCP_PROCESS_H */ diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 819c66bd08..9865c620b7 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -14,6 +14,8 @@ #define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" +static int bitmap_mem_offset; + int rte_pdcp_dynfield_offset = -1; static int @@ -39,9 +41,12 @@ pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); - if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); - else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) + size = RTE_CACHE_LINE_ROUNDUP(size); + bitmap_mem_offset = size; + size += pdcp_cnt_bitmap_get_memory_footprint(conf); + } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; @@ -54,11 +59,24 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + void *bitmap_mem; + int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - return pdcp_reorder_create(&dl->reorder, window_size); + ret = pdcp_reorder_create(&dl->reorder, window_size); + if (ret) + return ret; + + bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); + ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); + if (ret) { + pdcp_reorder_destroy(&dl->reorder); + return ret; + } + + return 0; } struct rte_pdcp_entity * @@ -136,10 +154,6 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; } - ret = pdcp_cnt_ring_create(entity, conf); - if (ret) - goto crypto_sess_destroy; - return entity; crypto_sess_destroy: @@ -218,6 +232,7 @@ struct rte_mbuf * rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, enum rte_pdcp_ctrl_pdu_type type) { + struct entity_priv_dl_part *dl; struct entity_priv *en_priv; struct rte_mbuf *m; int ret; @@ -228,6 +243,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, } en_priv = entity_priv_get(pdcp_entity); + dl = entity_dl_part_get(pdcp_entity); m = rte_pktmbuf_alloc(en_priv->ctrl_pdu_pool); if (m == NULL) { @@ -237,7 +253,7 @@ rte_pdcp_control_pdu_create(struct rte_pdcp_entity *pdcp_entity, switch (type) { case RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT: - ret = pdcp_ctrl_pdu_status_gen(en_priv, m); + ret = pdcp_ctrl_pdu_status_gen(en_priv, dl, m); break; default: ret = -ENOTSUP; @@ -283,7 +299,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * - update RX_DELIV to the COUNT value of the first PDCP SDU which has not been delivered * to upper layers, with COUNT value >= RX_REORD; */ - en_priv->state.rx_deliv = en_priv->state.rx_reord + nb_seq; + pdcp_rx_deliv_set(entity, en_priv->state.rx_reord + nb_seq); /* * - if RX_DELIV < RX_NEXT: -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 20/21] pdcp: allocate reorder buffer alongside with entity 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (18 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 19/21] pdcp: add support for status report Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 21/21] test/pdcp: add PDCP status report cases Anoob Joseph 2023-06-01 8:47 ` [PATCH v6 00/21] lib: add pdcp protocol Akhil Goyal 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Instead of allocating reorder buffer separately on heap, allocate memory for it together with rest of entity, and then only initialize buffer via `rte_reorder_init()`. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- lib/pdcp/pdcp_cnt.c | 9 +++---- lib/pdcp/pdcp_cnt.h | 3 ++- lib/pdcp/pdcp_entity.h | 2 +- lib/pdcp/pdcp_reorder.c | 11 ++------ lib/pdcp/pdcp_reorder.h | 12 ++++++--- lib/pdcp/rte_pdcp.c | 58 ++++++++++++++++++++++++++--------------- 6 files changed, 55 insertions(+), 40 deletions(-) diff --git a/lib/pdcp/pdcp_cnt.c b/lib/pdcp/pdcp_cnt.c index af027b00d3..e1d0634b4d 100644 --- a/lib/pdcp/pdcp_cnt.c +++ b/lib/pdcp/pdcp_cnt.c @@ -20,15 +20,14 @@ pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf) } int -pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size) +pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size) { - uint32_t mem_size = rte_bitmap_get_memory_footprint(window_size); - - dl->bitmap.bmp = rte_bitmap_init(window_size, bitmap_mem, mem_size); + dl->bitmap.bmp = rte_bitmap_init(nb_elem, bitmap_mem, mem_size); if (dl->bitmap.bmp == NULL) return -EINVAL; - dl->bitmap.size = window_size; + dl->bitmap.size = nb_elem; return 0; } diff --git a/lib/pdcp/pdcp_cnt.h b/lib/pdcp/pdcp_cnt.h index 5941b7a406..87b011f9dc 100644 --- a/lib/pdcp/pdcp_cnt.h +++ b/lib/pdcp/pdcp_cnt.h @@ -10,7 +10,8 @@ #include "pdcp_entity.h" uint32_t pdcp_cnt_bitmap_get_memory_footprint(const struct rte_pdcp_entity_conf *conf); -int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, void *bitmap_mem, uint32_t window_size); +int pdcp_cnt_bitmap_create(struct entity_priv_dl_part *dl, uint32_t nb_elem, + void *bitmap_mem, uint32_t mem_size); void pdcp_cnt_bitmap_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); bool pdcp_cnt_bitmap_is_set(struct pdcp_cnt_bitmap bitmap, uint32_t count); diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index a9b1428c7a..9f74b5d0e5 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -132,7 +132,7 @@ struct pdcp_cnt_bitmap { }; /* - * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] + * Layout of PDCP entity: [rte_pdcp_entity] [entity_priv] [entity_dl/ul] [reorder/bitmap] */ struct entity_priv { diff --git a/lib/pdcp/pdcp_reorder.c b/lib/pdcp/pdcp_reorder.c index 5399f0dc28..bc45f2e19b 100644 --- a/lib/pdcp/pdcp_reorder.c +++ b/lib/pdcp/pdcp_reorder.c @@ -8,20 +8,13 @@ #include "pdcp_reorder.h" int -pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size) +pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size) { - reorder->buf = rte_reorder_create("reorder_buffer", SOCKET_ID_ANY, window_size); + reorder->buf = rte_reorder_init(mem, mem_size, "reorder_buffer", nb_elem); if (reorder->buf == NULL) return -rte_errno; - reorder->window_size = window_size; reorder->is_active = false; return 0; } - -void -pdcp_reorder_destroy(const struct pdcp_reorder *reorder) -{ - rte_reorder_free(reorder->buf); -} diff --git a/lib/pdcp/pdcp_reorder.h b/lib/pdcp/pdcp_reorder.h index 6a2f61d6ae..7e4f079d4b 100644 --- a/lib/pdcp/pdcp_reorder.h +++ b/lib/pdcp/pdcp_reorder.h @@ -9,12 +9,18 @@ struct pdcp_reorder { struct rte_reorder_buffer *buf; - uint32_t window_size; bool is_active; }; -int pdcp_reorder_create(struct pdcp_reorder *reorder, uint32_t window_size); -void pdcp_reorder_destroy(const struct pdcp_reorder *reorder); +int pdcp_reorder_create(struct pdcp_reorder *reorder, size_t nb_elem, void *mem, size_t mem_size); + +/* NOTE: replace with `rte_reorder_memory_footprint_get` after DPDK 23.07 */ +#define SIZE_OF_REORDER_BUFFER (4 * RTE_CACHE_LINE_SIZE) +static inline size_t +pdcp_reorder_memory_footprint_get(size_t nb_elem) +{ + return SIZE_OF_REORDER_BUFFER + (2 * nb_elem * sizeof(struct rte_mbuf *)); +} static inline uint32_t pdcp_reorder_get_sequential(struct pdcp_reorder *reorder, struct rte_mbuf **mbufs, diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 9865c620b7..1c6d2466b2 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -14,7 +14,15 @@ #define RTE_PDCP_DYNFIELD_NAME "rte_pdcp_dynfield" -static int bitmap_mem_offset; +struct entity_layout { + size_t bitmap_offset; + size_t bitmap_size; + + size_t reorder_buf_offset; + size_t reorder_buf_size; + + size_t total_size; +}; int rte_pdcp_dynfield_offset = -1; @@ -35,46 +43,54 @@ pdcp_dynfield_register(void) } static int -pdcp_entity_size_get(const struct rte_pdcp_entity_conf *conf) +pdcp_entity_layout_get(const struct rte_pdcp_entity_conf *conf, struct entity_layout *layout) { - int size; + size_t size; + const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); size = sizeof(struct rte_pdcp_entity) + sizeof(struct entity_priv); if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { size += sizeof(struct entity_priv_dl_part); + /* Bitmap require memory to be cache aligned */ size = RTE_CACHE_LINE_ROUNDUP(size); - bitmap_mem_offset = size; - size += pdcp_cnt_bitmap_get_memory_footprint(conf); + layout->bitmap_offset = size; + layout->bitmap_size = pdcp_cnt_bitmap_get_memory_footprint(conf); + size += layout->bitmap_size; + layout->reorder_buf_offset = size; + layout->reorder_buf_size = pdcp_reorder_memory_footprint_get(window_size); + size += layout->reorder_buf_size; } else if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_UPLINK) size += sizeof(struct entity_priv_ul_part); else return -EINVAL; - return RTE_ALIGN_CEIL(size, RTE_CACHE_LINE_SIZE); + layout->total_size = size; + + return 0; } static int -pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf) +pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_conf *conf, + const struct entity_layout *layout) { const uint32_t window_size = pdcp_window_size_get(conf->pdcp_xfrm.sn_size); struct entity_priv_dl_part *dl = entity_dl_part_get(entity); - void *bitmap_mem; + void *memory; int ret; entity->max_pkt_cache = RTE_MAX(entity->max_pkt_cache, window_size); dl->t_reorder.handle = conf->t_reordering; - ret = pdcp_reorder_create(&dl->reorder, window_size); + memory = RTE_PTR_ADD(entity, layout->reorder_buf_offset); + ret = pdcp_reorder_create(&dl->reorder, window_size, memory, layout->reorder_buf_size); if (ret) return ret; - bitmap_mem = RTE_PTR_ADD(entity, bitmap_mem_offset); - ret = pdcp_cnt_bitmap_create(dl, bitmap_mem, window_size); - if (ret) { - pdcp_reorder_destroy(&dl->reorder); + memory = RTE_PTR_ADD(entity, layout->bitmap_offset); + ret = pdcp_cnt_bitmap_create(dl, window_size, memory, layout->bitmap_size); + if (ret) return ret; - } return 0; } @@ -82,10 +98,11 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c struct rte_pdcp_entity * rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) { + struct entity_layout entity_layout = { 0 }; struct rte_pdcp_entity *entity = NULL; struct entity_priv *en_priv; - int ret, entity_size; uint32_t count; + int ret; if (pdcp_dynfield_register() < 0) return NULL; @@ -118,13 +135,14 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) return NULL; } - entity_size = pdcp_entity_size_get(conf); - if (entity_size < 0) { + ret = pdcp_entity_layout_get(conf, &entity_layout); + if (ret < 0) { rte_errno = EINVAL; return NULL; } - entity = rte_zmalloc_socket("pdcp_entity", entity_size, RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY); + entity = rte_zmalloc_socket("pdcp_entity", entity_layout.total_size, RTE_CACHE_LINE_SIZE, + SOCKET_ID_ANY); if (entity == NULL) { rte_errno = ENOMEM; return NULL; @@ -149,7 +167,7 @@ rte_pdcp_entity_establish(const struct rte_pdcp_entity_conf *conf) goto crypto_sess_destroy; if (conf->pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) { - ret = pdcp_dl_establish(entity, conf); + ret = pdcp_dl_establish(entity, conf, &entity_layout); if (ret) goto crypto_sess_destroy; } @@ -174,8 +192,6 @@ pdcp_dl_release(struct rte_pdcp_entity *entity, struct rte_mbuf *out_mb[]) nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, entity->max_pkt_cache, en_priv->state.rx_next); - pdcp_reorder_destroy(&dl->reorder); - return nb_out; } -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* [PATCH v6 21/21] test/pdcp: add PDCP status report cases 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (19 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph @ 2023-05-30 10:01 ` Anoob Joseph 2023-06-01 8:47 ` [PATCH v6 00/21] lib: add pdcp protocol Akhil Goyal 21 siblings, 0 replies; 192+ messages in thread From: Anoob Joseph @ 2023-05-30 10:01 UTC (permalink / raw) To: Thomas Monjalon, Akhil Goyal, Jerin Jacob, Konstantin Ananyev Cc: Volodymyr Fialko, Hemant Agrawal, Mattias Rönnblom, Kiran Kumar K, dev, Olivier Matz, Stephen Hemminger From: Volodymyr Fialko <vfialko@marvell.com> Test PDCP status report generation. Signed-off-by: Anoob Joseph <anoobj@marvell.com> Signed-off-by: Volodymyr Fialko <vfialko@marvell.com> --- app/test/test_pdcp.c | 312 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 312 insertions(+) diff --git a/app/test/test_pdcp.c b/app/test/test_pdcp.c index 82cc25ec7a..423526380f 100644 --- a/app/test/test_pdcp.c +++ b/app/test/test_pdcp.c @@ -2,6 +2,7 @@ * Copyright(C) 2023 Marvell. */ +#include <rte_bitmap.h> #include <rte_errno.h> #ifdef RTE_LIB_EVENTDEV #include <rte_eventdev.h> @@ -48,6 +49,9 @@ struct pdcp_testsuite_params { #endif /* RTE_LIB_EVENTDEV */ bool timer_is_running; uint64_t min_resolution_ns; + struct rte_pdcp_up_ctrl_pdu_hdr *status_report; + uint32_t status_report_bitmask_capacity; + uint8_t *ctrl_pdu_buf; }; static struct pdcp_testsuite_params testsuite_params; @@ -168,6 +172,18 @@ static struct rte_pdcp_t_reordering t_reorder_timer = { .stop = pdcp_timer_stop_cb, }; +static inline void +bitmask_set_bit(uint8_t *mask, uint32_t bit) +{ + mask[bit / 8] |= (1 << bit % 8); +} + +static inline bool +bitmask_is_bit_set(const uint8_t *mask, uint32_t bit) +{ + return mask[bit / 8] & (1 << (bit % 8)); +} + static inline int pdcp_hdr_size_get(enum rte_security_pdcp_sn_size sn_size) { @@ -314,6 +330,21 @@ testsuite_setup(void) goto cop_pool_free; } + /* Allocate memory for longest possible status report */ + ts_params->status_report_bitmask_capacity = RTE_PDCP_CTRL_PDU_SIZE_MAX - + sizeof(struct rte_pdcp_up_ctrl_pdu_hdr); + ts_params->status_report = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->status_report == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report\n"); + goto cop_pool_free; + } + + ts_params->ctrl_pdu_buf = rte_zmalloc(NULL, RTE_PDCP_CTRL_PDU_SIZE_MAX, 0); + if (ts_params->ctrl_pdu_buf == NULL) { + RTE_LOG(ERR, USER1, "Could not allocate status report data\n"); + goto cop_pool_free; + } + return 0; cop_pool_free: @@ -322,6 +353,8 @@ testsuite_setup(void) mbuf_pool_free: rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); return TEST_FAILED; } @@ -344,6 +377,9 @@ testsuite_teardown(void) rte_mempool_free(ts_params->mbuf_pool); ts_params->mbuf_pool = NULL; + + rte_free(ts_params->status_report); + rte_free(ts_params->ctrl_pdu_buf); } static int @@ -1410,6 +1446,246 @@ test_expiry_with_rte_timer(const struct pdcp_test_conf *ul_conf) return ret; } +static struct rte_pdcp_up_ctrl_pdu_hdr * +pdcp_status_report_init(uint32_t fmc) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + + hdr->d_c = RTE_PDCP_PDU_TYPE_CTRL; + hdr->pdu_type = RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT; + hdr->fmc = rte_cpu_to_be_32(fmc); + hdr->r = 0; + memset(hdr->bitmap, 0, testsuite_params.status_report_bitmask_capacity); + + return hdr; +} + +static uint32_t +pdcp_status_report_len(void) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = testsuite_params.status_report; + uint32_t i; + + for (i = testsuite_params.status_report_bitmask_capacity; i != 0; i--) { + if (hdr->bitmap[i - 1]) + return i; + } + + return 0; +} + +static int +pdcp_status_report_verify(struct rte_mbuf *status_report, + const struct rte_pdcp_up_ctrl_pdu_hdr *expected_hdr, uint32_t expected_len) +{ + uint32_t received_len = rte_pktmbuf_pkt_len(status_report); + uint8_t *received_buf = testsuite_params.ctrl_pdu_buf; + int ret; + + ret = pktmbuf_read_into(status_report, received_buf, RTE_PDCP_CTRL_PDU_SIZE_MAX); + TEST_ASSERT_SUCCESS(ret, "Failed to copy status report pkt into continuous buffer"); + + debug_hexdump(stdout, "Received:", received_buf, received_len); + debug_hexdump(stdout, "Expected:", expected_hdr, expected_len); + + TEST_ASSERT_EQUAL(expected_len, received_len, + "Mismatch in packet lengths [expected: %d, received: %d]", + expected_len, received_len); + + TEST_ASSERT_BUFFERS_ARE_EQUAL(received_buf, expected_hdr, expected_len, + "Generated packet not as expected"); + + return 0; +} + +static int +test_status_report_gen(const struct pdcp_test_conf *ul_conf, + const struct rte_pdcp_up_ctrl_pdu_hdr *hdr, + uint32_t bitmap_len) +{ + const enum rte_security_pdcp_sn_size sn_size = ul_conf->entity.pdcp_xfrm.sn_size; + struct rte_mbuf *status_report = NULL, **out_mb, *m; + uint16_t nb_success = 0, nb_err = 0; + struct rte_pdcp_entity *pdcp_entity; + struct pdcp_test_conf dl_conf; + int ret = TEST_FAILED, nb_out; + uint32_t nb_pkts = 0, i; + uint8_t cdev_id; + + const uint32_t start_count = rte_be_to_cpu_32(hdr->fmc); + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK) + return TEST_SKIPPED; + + /* Create configuration for actual testing */ + uplink_to_downlink_convert(ul_conf, &dl_conf); + dl_conf.entity.pdcp_xfrm.hfn = pdcp_hfn_from_count_get(start_count, sn_size); + dl_conf.entity.sn = pdcp_sn_from_count_get(start_count, sn_size); + dl_conf.entity.status_report_required = true; + + pdcp_entity = test_entity_create(&dl_conf, &ret); + if (pdcp_entity == NULL) + return ret; + + cdev_id = dl_conf.entity.dev_id; + out_mb = calloc(pdcp_entity->max_pkt_cache, sizeof(uintptr_t)); + + for (i = 0; i < bitmap_len * 8; i++) { + if (!bitmask_is_bit_set(hdr->bitmap, i)) + continue; + + m = generate_packet_for_dl_with_sn(*ul_conf, start_count + i + 1); + ASSERT_TRUE_OR_GOTO(m != NULL, exit, "Could not allocate buffer for packet\n"); + + nb_success = test_process_packets(pdcp_entity, cdev_id, &m, 1, out_mb, &nb_err); + ASSERT_TRUE_OR_GOTO(nb_err == 0, exit, "Error occurred during packet buffering\n"); + ASSERT_TRUE_OR_GOTO(nb_success == 0, exit, "Packet was not buffered as expected\n"); + + } + + m = NULL; + + /* Check status report */ + status_report = rte_pdcp_control_pdu_create(pdcp_entity, + RTE_PDCP_CTRL_PDU_TYPE_STATUS_REPORT); + ASSERT_TRUE_OR_GOTO(status_report != NULL, exit, "Could not generate status report\n"); + + const uint32_t expected_len = sizeof(struct rte_pdcp_up_ctrl_pdu_hdr) + bitmap_len; + + ASSERT_TRUE_OR_GOTO(pdcp_status_report_verify(status_report, hdr, expected_len) == 0, exit, + "Report verification failure\n"); + + ret = TEST_SUCCESS; +exit: + rte_free(m); + rte_pktmbuf_free(status_report); + rte_pktmbuf_free_bulk(out_mb, nb_pkts); + nb_out = rte_pdcp_entity_release(pdcp_entity, out_mb); + rte_pktmbuf_free_bulk(out_mb, nb_out); + free(out_mb); + return ret; +} + +static void +ctrl_pdu_hdr_packet_set(struct rte_pdcp_up_ctrl_pdu_hdr *hdr, uint32_t pkt_count) +{ + bitmask_set_bit(hdr->bitmap, pkt_count - rte_be_to_cpu_32(hdr->fmc) - 1); +} + +static int +test_status_report_fmc_only(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(42); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_first_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_one_pkt_second_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + + ctrl_pdu_hdr_packet_set(hdr, RTE_BITMAP_SLAB_BIT_SIZE + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_full_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(1); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE + 1; + int i; + + for (i = 0; i < RTE_BITMAP_SLAB_BIT_SIZE; i++) + ctrl_pdu_hdr_packet_set(hdr, start_offset + i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_two_non_sequential_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(0); + const uint32_t start_offset = RTE_BITMAP_SLAB_BIT_SIZE / 2 + 1; + + ctrl_pdu_hdr_packet_set(hdr, start_offset); + ctrl_pdu_hdr_packet_set(hdr, start_offset + RTE_BITMAP_SLAB_BIT_SIZE); + ctrl_pdu_hdr_packet_set(hdr, 3 * RTE_BITMAP_SLAB_BIT_SIZE); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_max_length_sn_12(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr; + const uint32_t fmc = 0; + uint32_t i; + + if (ul_conf->entity.pdcp_xfrm.pkt_dir == RTE_SECURITY_PDCP_DOWNLINK || + ul_conf->entity.pdcp_xfrm.sn_size != RTE_SECURITY_PDCP_SN_SIZE_12) + return TEST_SKIPPED; + + hdr = pdcp_status_report_init(fmc); + + const uint32_t max_count = RTE_MIN((RTE_PDCP_CTRL_PDU_SIZE_MAX - sizeof(hdr)) * 8, + (uint32_t)PDCP_WINDOW_SIZE(RTE_SECURITY_PDCP_SN_SIZE_12)); + + i = fmc + 2; /* set first count to have a gap, to enable packet buffering */ + + for (; i < max_count; i++) + ctrl_pdu_hdr_packet_set(hdr, i); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_different_slabs(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(63); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 64 + 1); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + +static int +test_status_report_overlap_same_slab(const struct pdcp_test_conf *ul_conf) +{ + struct rte_pdcp_up_ctrl_pdu_hdr *hdr = pdcp_status_report_init(2); + const uint32_t sn_size = 12; + + ctrl_pdu_hdr_packet_set(hdr, 4); + ctrl_pdu_hdr_packet_set(hdr, PDCP_WINDOW_SIZE(sn_size) + 1); + + return test_status_report_gen(ul_conf, hdr, pdcp_status_report_len()); +} + static int test_combined(struct pdcp_test_conf *ul_conf) { @@ -1611,11 +1887,47 @@ static struct unit_test_suite reorder_test_cases = { } }; +static struct unit_test_suite status_report_test_cases = { + .suite_name = "PDCP status report", + .unit_test_cases = { + TEST_CASE_NAMED_WITH_DATA("test_status_report_fmc_only", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_fmc_only), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_first_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_first_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_one_pkt_second_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_one_pkt_second_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_full_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_full_slab), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_two_non_sequential_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_two_non_sequential_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_max_length_sn_12", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec_until_first_pass, + test_status_report_max_length_sn_12), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_different_slabs", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_different_slabs), + TEST_CASE_NAMED_WITH_DATA("test_status_report_overlap_same_slab", + ut_setup_pdcp, ut_teardown_pdcp, + run_test_with_all_known_vec, test_status_report_overlap_same_slab), + TEST_CASES_END() /**< NULL terminate unit test array */ + } +}; + struct unit_test_suite *test_suites[] = { NULL, /* Place holder for known_vector_cases */ &combined_mode_cases, &hfn_sn_test_cases, &reorder_test_cases, + &status_report_test_cases, NULL /* End of suites list */ }; -- 2.25.1 ^ permalink raw reply [flat|nested] 192+ messages in thread
* RE: [PATCH v6 00/21] lib: add pdcp protocol 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph ` (20 preceding siblings ...) 2023-05-30 10:01 ` [PATCH v6 21/21] test/pdcp: add PDCP status report cases Anoob Joseph @ 2023-06-01 8:47 ` Akhil Goyal 21 siblings, 0 replies; 192+ messages in thread From: Akhil Goyal @ 2023-06-01 8:47 UTC (permalink / raw) To: Anoob Joseph, Thomas Monjalon, Jerin Jacob Kollanukkaran, Konstantin Ananyev Cc: Hemant Agrawal, Mattias Rönnblom, Kiran Kumar Kokkilagadda, Volodymyr Fialko, dev, Olivier Matz, Stephen Hemminger > Subject: [PATCH v6 00/21] lib: add pdcp protocol > > Add Packet Data Convergence Protocol (PDCP) processing library. > > The library is similar to lib_ipsec which provides IPsec processing > capabilities in DPDK. > > PDCP would involve roughly the following operations, > 1. Transfer of user plane data > 2. Transfer of control plane data > 3. Header compression > 4. Uplink data compression > 5. Ciphering and integrity protection > > PDCP library provides following control path APIs that is used to > configure various PDCP entities, > 1. rte_pdcp_entity_establish() > 2. rte_pdcp_entity_suspend() > 3. rte_pdcp_entity_release() > > PDCP process is split into 2 parts. One before crypto processing > (rte_pdcp_pkt_pre_process()) and one after crypto processing > (rte_pdcp_pkt_post_process()). Since cryptodev dequeue can return crypto > operations belonging to multiple entities, rte_pdcp_pkt_crypto_group() > is added to help grouping crypto operations belonging to same entity. > > Similar to lib IPsec, lib PDCP would allow application to use same API > sequence while leveraging protocol offload features enabled by rte_security > library. Lib PDCP would internally change the handles registered for > *pre_process* and *post_process* based on features enabled in the entity. > > Lib PDCP would create the required sessions on the device provided in entity to > minimize the application requirements. Also, the crypto_op allocation and free > would also be done internally by lib PDCP to allow the library to create > crypto ops as required for the input packets. For example, when control PDUs > are > received, no cryptodev enqueue-dequeue is expected for the same and lib PDCP > is expected to handle it differently. > > Lib PDCP utilizes reorder library for implementing in-order delivery. It > utilizes bitmap library for implementing status reports and track the COUNT > value of the packets received. To allow application to choose timer > implementation of choice, lib PDCP allows application to configure handles that > can be used for starting & stopping timers. Upon expiry, application can call > corresponding PDCP API(``rte_pdcp_t_reordering_expiry_handle``) for handling > the > event. Unit tests are added to verify both rte_timer based timers as well as > rte_eventdev based timers. > > PDCP tracks the sequence number of the received packets and during events > such > as re-establishment, it is required to generate reports and transmit to the > peer. This series introduces ``rte_pdcp_control_pdu_create`` for handling > control PDU generation. > > Changes in v6: > - Rebased > - Minor udpates to documentation (Akhil) Series Acked-by: Akhil Goyal <gakhil@marvell.com> Applied to dpdk-next-crypto Thanks. ^ permalink raw reply [flat|nested] 192+ messages in thread
end of thread, other threads:[~2023-06-12 5:19 UTC | newest] Thread overview: 192+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-10-27 5:21 [RFC 0/1] lib: add pdcp protocol Anoob Joseph 2022-10-27 5:21 ` [RFC 1/1] " Anoob Joseph 2022-12-13 7:01 ` [RFC 0/1] " Akhil Goyal 2022-12-20 12:15 ` Anoob Joseph 2022-12-22 9:25 ` [PATCH 0/5] " Anoob Joseph 2022-12-22 9:25 ` [PATCH 1/5] net: add PDCP header Anoob Joseph 2023-01-18 16:36 ` Thomas Monjalon 2023-01-18 17:39 ` [EXT] " Anoob Joseph 2023-01-19 8:05 ` Thomas Monjalon 2023-01-23 9:21 ` Anoob Joseph 2023-01-23 15:31 ` Thomas Monjalon 2022-12-22 9:25 ` [PATCH 2/5] lib: add pdcp protocol Anoob Joseph 2023-01-18 16:26 ` Akhil Goyal 2023-02-13 10:59 ` Anoob Joseph 2022-12-22 9:25 ` [PATCH 3/5] app/test: add lib pdcp tests Anoob Joseph 2022-12-22 9:25 ` [PATCH 4/5] app/test: pdcp HFN tests in combined mode Anoob Joseph 2022-12-22 9:25 ` [PATCH 5/5] doc: add PDCP library guide Anoob Joseph 2023-01-18 16:39 ` [PATCH 0/5] lib: add pdcp protocol Thomas Monjalon 2023-01-23 17:36 ` Jerin Jacob 2023-04-14 17:44 ` [PATCH v2 00/22] " Anoob Joseph 2023-04-14 17:44 ` [PATCH v2 01/22] net: add PDCP header Anoob Joseph 2023-05-16 14:02 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 02/22] lib: add pdcp protocol Anoob Joseph 2023-05-16 15:30 ` Akhil Goyal 2023-05-18 6:53 ` Anoob Joseph 2023-05-18 7:40 ` Akhil Goyal 2023-05-18 8:32 ` Anoob Joseph 2023-05-18 8:46 ` Akhil Goyal 2023-05-22 7:03 ` Anoob Joseph 2023-04-14 17:44 ` [PATCH v2 03/22] pdcp: add pre and post-process Anoob Joseph 2023-05-16 15:43 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 04/22] pdcp: add packet group Anoob Joseph 2023-05-16 15:56 ` Akhil Goyal 2023-05-18 8:12 ` Anoob Joseph 2023-04-14 17:44 ` [PATCH v2 05/22] pdcp: add crypto session create and destroy Anoob Joseph 2023-05-16 16:21 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 06/22] pdcp: add pre and post process for UL Anoob Joseph 2023-05-18 6:38 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 07/22] pdcp: add pre and post process for DL Anoob Joseph 2023-05-18 6:47 ` Akhil Goyal 2023-05-18 7:33 ` Anoob Joseph 2023-04-14 17:44 ` [PATCH v2 08/22] pdcp: add IV generation routines Anoob Joseph 2023-05-18 6:51 ` Akhil Goyal 2023-04-14 17:44 ` [PATCH v2 09/22] app/test: add lib pdcp tests Anoob Joseph 2023-05-18 8:03 ` Akhil Goyal 2023-05-18 11:31 ` Anoob Joseph 2023-05-18 12:06 ` Akhil Goyal 2023-05-19 10:31 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 11/22] doc: add PDCP library guide Anoob Joseph 2023-05-18 8:26 ` Akhil Goyal 2023-05-22 10:22 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 12/22] pdcp: add control PDU handling Anoob Joseph 2023-05-18 9:15 ` Akhil Goyal 2023-05-22 11:09 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 14/22] test/pdcp: add in-order delivery cases Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 15/22] pdcp: add timer callback handlers Anoob Joseph 2023-05-18 9:37 ` Akhil Goyal 2023-04-14 17:45 ` [PATCH v2 16/22] pdcp: add timer expiry handle Anoob Joseph 2023-05-18 9:43 ` Akhil Goyal 2023-05-22 11:34 ` Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 17/22] test/pdcp: add timer expiry cases Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 18/22] test/pdcp: add timer restart case Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 19/22] pdcp: add support for status report Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 21/22] pdcp: add thread safe processing Anoob Joseph 2023-04-14 17:45 ` [PATCH v2 22/22] test/pdcp: add PDCP status report cases Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 00/22] lib: add pdcp protocol Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 01/22] net: add PDCP header Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 02/22] lib: add pdcp protocol Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 03/22] pdcp: add pre and post-process Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 04/22] pdcp: add packet group Anoob Joseph 2023-05-24 16:00 ` [PATCH v3 05/22] pdcp: add crypto session create and destroy Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 06/22] pdcp: add pre and post process for UL Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 07/22] pdcp: add pre and post process for DL Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 08/22] pdcp: add IV generation routines Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 09/22] app/test: add lib pdcp tests Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 11/22] doc: add PDCP library guide Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 12/22] pdcp: add control PDU handling for status report Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 14/22] test/pdcp: add in-order delivery cases Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 15/22] pdcp: add timer callback handlers Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 16/22] pdcp: add timer expiry handle Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 17/22] test/pdcp: add timer expiry cases Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 18/22] test/pdcp: add timer restart case Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 19/22] pdcp: add support for status report Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 21/22] pdcp: add thread safe processing Anoob Joseph 2023-05-24 18:31 ` Stephen Hemminger 2023-05-25 8:15 ` [EXT] " Anoob Joseph 2023-05-25 15:25 ` Stephen Hemminger 2023-05-25 15:37 ` Anoob Joseph 2023-05-24 16:01 ` [PATCH v3 22/22] test/pdcp: add PDCP status report cases Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 00/22] lib: add pdcp protocol Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 01/22] net: add PDCP header Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 02/22] lib: add pdcp protocol Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 03/22] pdcp: add pre and post-process Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 04/22] pdcp: add packet group Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 05/22] pdcp: add crypto session create and destroy Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 06/22] pdcp: add pre and post process for UL Anoob Joseph 2023-05-26 21:01 ` [PATCH v4 07/22] pdcp: add pre and post process for DL Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 08/22] pdcp: add IV generation routines Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 09/22] app/test: add lib pdcp tests Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 10/22] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 11/22] doc: add PDCP library guide Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 12/22] pdcp: add control PDU handling for status report Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 13/22] pdcp: implement t-Reordering and packet buffering Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 14/22] test/pdcp: add in-order delivery cases Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 15/22] pdcp: add timer callback handlers Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 16/22] pdcp: add timer expiry handle Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 17/22] test/pdcp: add timer expiry cases Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 18/22] test/pdcp: add timer restart case Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 19/22] pdcp: add support for status report Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 20/22] pdcp: allocate reorder buffer alongside with entity Anoob Joseph 2023-05-26 21:02 ` [PATCH v4 21/22] pdcp: add thread safe processing Anoob Joseph 2023-05-26 22:11 ` Stephen Hemminger 2023-05-27 5:24 ` [EXT] " Anoob Joseph 2023-05-27 7:17 ` Anoob Joseph 2023-05-26 22:15 ` Stephen Hemminger 2023-05-26 21:02 ` [PATCH v4 22/22] test/pdcp: add PDCP status report cases Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph 2023-05-30 8:51 ` Akhil Goyal 2023-05-27 7:15 ` [PATCH v5 02/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 03/21] pdcp: add pre and post-process Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 04/21] pdcp: add packet group Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 05/21] pdcp: add crypto session create and destroy Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 06/21] pdcp: add pre and post process for UL Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 07/21] pdcp: add pre and post process for DL Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 08/21] pdcp: add IV generation routines Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 09/21] app/test: add lib pdcp tests Anoob Joseph 2023-05-27 7:15 ` [PATCH v5 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 11/21] doc: add PDCP library guide Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 12/21] pdcp: add control PDU handling for status report Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 14/21] test/pdcp: add in-order delivery cases Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 15/21] pdcp: add timer callback handlers Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 16/21] pdcp: add timer expiry handle Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 17/21] test/pdcp: add timer expiry cases Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 18/21] test/pdcp: add timer restart case Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 19/21] pdcp: add support for status report Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph 2023-05-27 7:16 ` [PATCH v5 21/21] test/pdcp: add PDCP status report cases Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 01/21] net: add PDCP header Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 02/21] lib: add pdcp protocol Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 03/21] pdcp: add pre and post-process Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 04/21] pdcp: add packet group Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 05/21] pdcp: add crypto session create and destroy Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 06/21] pdcp: add pre and post process for UL Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 07/21] pdcp: add pre and post process for DL Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 08/21] pdcp: add IV generation routines Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 09/21] app/test: add lib pdcp tests Anoob Joseph 2023-05-27 8:58 ` [PATCH v5 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 11/21] doc: add PDCP library guide Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 12/21] pdcp: add control PDU handling for status report Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 14/21] test/pdcp: add in-order delivery cases Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 15/21] pdcp: add timer callback handlers Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 16/21] pdcp: add timer expiry handle Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 17/21] test/pdcp: add timer expiry cases Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 18/21] test/pdcp: add timer restart case Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 19/21] pdcp: add support for status report Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph 2023-05-27 8:59 ` [PATCH v5 21/21] test/pdcp: add PDCP status report cases Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 00/21] lib: add pdcp protocol Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 01/21] net: add PDCP header Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 02/21] lib: add pdcp protocol Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 03/21] pdcp: add pre and post-process Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 04/21] pdcp: add packet group Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 05/21] pdcp: add crypto session create and destroy Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 06/21] pdcp: add pre and post process for UL Anoob Joseph 2023-06-10 22:50 ` Thomas Monjalon 2023-06-12 5:19 ` [EXT] " Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 07/21] pdcp: add pre and post process for DL Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 08/21] pdcp: add IV generation routines Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 09/21] app/test: add lib pdcp tests Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 10/21] test/pdcp: pdcp HFN tests in combined mode Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 11/21] doc: add PDCP library guide Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 12/21] pdcp: add control PDU handling for status report Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 13/21] pdcp: implement t-Reordering and packet buffering Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 14/21] test/pdcp: add in-order delivery cases Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 15/21] pdcp: add timer callback handlers Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 16/21] pdcp: add timer expiry handle Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 17/21] test/pdcp: add timer expiry cases Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 18/21] test/pdcp: add timer restart case Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 19/21] pdcp: add support for status report Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 20/21] pdcp: allocate reorder buffer alongside with entity Anoob Joseph 2023-05-30 10:01 ` [PATCH v6 21/21] test/pdcp: add PDCP status report cases Anoob Joseph 2023-06-01 8:47 ` [PATCH v6 00/21] lib: add pdcp protocol Akhil Goyal
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).
