From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 4A81E42B8F; Wed, 24 May 2023 18:05:31 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 715EA42D64; Wed, 24 May 2023 18:03:04 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A97AB42D48 for ; Wed, 24 May 2023 18:03:02 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34OFM3vl013829; Wed, 24 May 2023 09:03:01 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=CyZhNij4ZWNWAO/qq/u4XH9dBBzad94tLJdD2hxiBPk=; b=LdSyYuOeJXb3E235vQkB6LOlNKVuXj9HXcXxOjXz1TpWQGXF+JCDyD5E/ZVVdsyqi5Rw PoYzTTk8D5EIlBH9ZsNNK7DkyiNlvmx20rHAoA2MhTN85Z4U1T6BEpRPxrluKKNcmOCP gGk4XEiPwh7PH86YDQlGvarPy6yYb0NQy4NeAVeIVcSifV64vvybGlfmtc/l2nzo6zZf Th0/Syc6Ssmjg/69dc9tRCyMpDgqcX8q8zFkCccNOAnuXJn23YMPtda+DlzuviswNLes gL6GZuBLvL/suYFRpD+AKlk34MElqAVZnWivZWJHAyL0TEbe3MrCikDehXC+itbqsmWz zg== Received: from dc5-exch01.marvell.com ([199.233.59.181]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3qsh90h1ae-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 24 May 2023 09:03:01 -0700 Received: from DC5-EXCH01.marvell.com (10.69.176.38) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Wed, 24 May 2023 09:02:59 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH01.marvell.com (10.69.176.38) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Wed, 24 May 2023 09:02:59 -0700 Received: from BG-LT92004.corp.innovium.com (unknown [10.28.161.183]) by maili.marvell.com (Postfix) with ESMTP id 445CB3F7043; Wed, 24 May 2023 09:02:54 -0700 (PDT) From: Anoob Joseph To: Thomas Monjalon , Akhil Goyal , Jerin Jacob , Konstantin Ananyev , Bernard Iremonger CC: Volodymyr Fialko , Hemant Agrawal , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= , Kiran Kumar K , , Olivier Matz Subject: [PATCH v3 21/22] pdcp: add thread safe processing Date: Wed, 24 May 2023 21:31:15 +0530 Message-ID: <20230524160116.304-22-anoobj@marvell.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230524160116.304-1-anoobj@marvell.com> References: <20230414174512.642-1-anoobj@marvell.com> <20230524160116.304-1-anoobj@marvell.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-GUID: 7MS9QVRS0oDljW4vQ_4T5tMhA00XQN43 X-Proofpoint-ORIG-GUID: 7MS9QVRS0oDljW4vQ_4T5tMhA00XQN43 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-05-24_11,2023-05-24_01,2023-05-22_02 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Volodymyr Fialko PDCP state has to be guarded for: - Uplink pre_process: - tx_next atomic increment - Downlink pre_process: - rx_deliv - read - Downlink post_process: - rx_deliv, rx_reorder, rx_next - read/write - bitmask/reorder buffer - read/write When application requires thread safe processing, the state variables need to be updated atomically. Add config option to select this option per entity. Signed-off-by: Anoob Joseph Signed-off-by: Volodymyr Fialko --- lib/pdcp/pdcp_entity.h | 46 +++++++++++++++++++++++++++++++++++++++++ lib/pdcp/pdcp_process.c | 30 ++++++++++++++++++++++++--- lib/pdcp/rte_pdcp.c | 6 ++++++ lib/pdcp/rte_pdcp.h | 2 ++ 4 files changed, 81 insertions(+), 3 deletions(-) diff --git a/lib/pdcp/pdcp_entity.h b/lib/pdcp/pdcp_entity.h index 9f74b5d0e5..dc05c19bc0 100644 --- a/lib/pdcp/pdcp_entity.h +++ b/lib/pdcp/pdcp_entity.h @@ -10,6 +10,7 @@ #include #include #include +#include #include "pdcp_reorder.h" @@ -162,6 +163,8 @@ struct entity_priv { uint64_t is_status_report_required : 1; /** Is out-of-order delivery enabled */ uint64_t is_out_of_order_delivery : 1; + /** Is thread safety disabled */ + uint64_t is_thread_safety_disabled : 1; } flags; /** Crypto op pool. */ struct rte_mempool *cop_pool; @@ -175,6 +178,8 @@ struct entity_priv { uint8_t dev_id; }; +typedef rte_spinlock_t pdcp_lock_t; + struct entity_priv_dl_part { /** PDCP would need to track the count values that are already received.*/ struct pdcp_cnt_bitmap bitmap; @@ -182,6 +187,8 @@ struct entity_priv_dl_part { struct pdcp_t_reordering t_reorder; /** Reorder packet buffer */ struct pdcp_reorder reorder; + /* Lock to protect concurrent updates */ + pdcp_lock_t lock; /** Bitmap memory region */ uint8_t bitmap_mem[0]; }; @@ -257,4 +264,43 @@ pdcp_hfn_max(enum rte_security_pdcp_sn_size sn_size) return (1 << (32 - sn_size)) - 1; } +static inline uint32_t +pdcp_atomic_inc(const struct entity_priv *en_priv, uint32_t *val) +{ + if (en_priv->flags.is_thread_safety_disabled) + return (*val)++; + else + return __atomic_fetch_add(val, 1, __ATOMIC_RELAXED); +} + +static inline void +pdcp_lock_init(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_init(&dl->lock); +} + +static inline void +pdcp_lock_lock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_lock(&dl->lock); +} + +static inline void +pdcp_lock_unlock(const struct rte_pdcp_entity *entity) +{ + struct entity_priv_dl_part *dl = entity_dl_part_get(entity); + struct entity_priv *en_priv = entity_priv_get(entity); + + if (!en_priv->flags.is_thread_safety_disabled) + rte_spinlock_unlock(&dl->lock); +} + #endif /* PDCP_ENTITY_H */ diff --git a/lib/pdcp/pdcp_process.c b/lib/pdcp/pdcp_process.c index 8d42615270..2f253ad76c 100644 --- a/lib/pdcp/pdcp_process.c +++ b/lib/pdcp/pdcp_process.c @@ -369,7 +369,7 @@ pdcp_pre_process_uplane_sn_12_ul_set_sn(struct entity_priv *en_priv, struct rte_ return false; /* Update sequence num in the PDU header */ - *count = en_priv->state.tx_next++; + *count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_12); pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; @@ -451,7 +451,7 @@ pdcp_pre_process_uplane_sn_18_ul_set_sn(struct entity_priv *en_priv, struct rte_ return false; /* Update sequence num in the PDU header */ - *count = en_priv->state.tx_next++; + *count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(*count, RTE_SECURITY_PDCP_SN_SIZE_18); pdu_hdr->d_c = RTE_PDCP_PDU_TYPE_DATA; @@ -561,7 +561,7 @@ pdcp_pre_process_cplane_sn_12_ul(const struct rte_pdcp_entity *entity, struct rt memset(mac_i, 0, RTE_PDCP_MAC_I_LEN); /* Update sequence number in the PDU header */ - count = en_priv->state.tx_next++; + count = pdcp_atomic_inc(en_priv, &en_priv->state.tx_next); sn = pdcp_sn_from_count_get(count, RTE_SECURITY_PDCP_SN_SIZE_12); pdu_hdr->sn_11_8 = ((sn & 0xf00) >> 8); @@ -654,7 +654,9 @@ pdcp_pre_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -717,7 +719,9 @@ pdcp_pre_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -781,7 +785,9 @@ pdcp_pre_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, struct rt nb_cop = rte_crypto_op_bulk_alloc(en_priv->cop_pool, RTE_CRYPTO_OP_TYPE_SYMMETRIC, cop, num); + pdcp_lock_lock(entity); const uint32_t rx_deliv = en_priv->state.rx_deliv; + pdcp_lock_unlock(entity); for (i = 0; i < nb_cop; i++) { mb = in_mb[i]; @@ -923,6 +929,8 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, struct rte_mbuf *mb; uint32_t count; + pdcp_lock_lock(entity); + for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -949,6 +957,8 @@ pdcp_post_process_uplane_sn_12_dl_flags(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -989,6 +999,7 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, int32_t rsn = 0; uint32_t count; + pdcp_lock_lock(entity); for (i = 0; i < num; i++) { mb = in_mb[i]; @@ -1017,6 +1028,8 @@ pdcp_post_process_uplane_sn_18_dl_flags(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -1057,6 +1070,8 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, uint32_t count; int32_t rsn; + pdcp_lock_lock(entity); + for (i = 0; i < num; i++) { mb = in_mb[i]; if (unlikely(mb->ol_flags & RTE_MBUF_F_RX_SEC_OFFLOAD_FAILED)) @@ -1082,6 +1097,8 @@ pdcp_post_process_cplane_sn_12_dl(const struct rte_pdcp_entity *entity, err_mb[nb_err++] = mb; } + pdcp_lock_unlock(entity); + if (unlikely(nb_err != 0)) rte_memcpy(&out_mb[nb_success], err_mb, nb_err * sizeof(struct rte_mbuf *)); @@ -1245,6 +1262,13 @@ pdcp_entity_priv_populate(struct entity_priv *en_priv, const struct rte_pdcp_ent */ en_priv->flags.is_out_of_order_delivery = conf->out_of_order_delivery; + /** + * flags.disable_thread_safety + * + * Indicate whether the thread safety is disabled for PDCP entity. + */ + en_priv->flags.is_thread_safety_disabled = conf->disable_thread_safety; + /** * hdr_sz * diff --git a/lib/pdcp/rte_pdcp.c b/lib/pdcp/rte_pdcp.c index 9155669d69..2a1eb20724 100644 --- a/lib/pdcp/rte_pdcp.c +++ b/lib/pdcp/rte_pdcp.c @@ -72,6 +72,8 @@ pdcp_dl_establish(struct rte_pdcp_entity *entity, const struct rte_pdcp_entity_c if (ret) return ret; + pdcp_lock_init(entity); + return 0; } @@ -276,6 +278,8 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct * performing header decompression, if not decompressed before: */ + pdcp_lock_lock(entity); + /* - all stored PDCP SDU(s) with associated COUNT value(s) < RX_REORD; */ nb_out = pdcp_reorder_up_to_get(&dl->reorder, out_mb, capacity, en_priv->state.rx_reord); capacity -= nb_out; @@ -307,5 +311,7 @@ rte_pdcp_t_reordering_expiry_handle(const struct rte_pdcp_entity *entity, struct dl->t_reorder.state = TIMER_EXPIRED; } + pdcp_lock_unlock(entity); + return nb_out; } diff --git a/lib/pdcp/rte_pdcp.h b/lib/pdcp/rte_pdcp.h index 76ec569577..cbbedc851f 100644 --- a/lib/pdcp/rte_pdcp.h +++ b/lib/pdcp/rte_pdcp.h @@ -141,6 +141,8 @@ struct rte_pdcp_entity_conf { bool is_slrb; /** Enable security offload on the device specified. */ bool en_sec_offload; + /** Disable usage of synchronization primitives for entity. */ + bool disable_thread_safety; /** Device on which security/crypto session need to be created. */ uint8_t dev_id; /** -- 2.25.1