From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 31753A0C55; Wed, 13 Oct 2021 14:26:10 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1F75B4115A; Wed, 13 Oct 2021 14:26:10 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id AAC0E40E64 for ; Wed, 13 Oct 2021 14:26:07 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10135"; a="214564526" X-IronPort-AV: E=Sophos;i="5.85,370,1624345200"; d="scan'208";a="214564526" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Oct 2021 05:25:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,370,1624345200"; d="scan'208";a="524607769" Received: from silpixa00400884.ir.intel.com ([10.243.22.82]) by orsmga001.jf.intel.com with ESMTP; 13 Oct 2021 05:25:48 -0700 From: Radu Nicolau To: Konstantin Ananyev , Bernard Iremonger , Vladimir Medvedkin , Ray Kinsella Cc: dev@dpdk.org, bruce.richardson@intel.com, roy.fan.zhang@intel.com, hemant.agrawal@nxp.com, gakhil@marvell.com, anoobj@marvell.com, declan.doherty@intel.com, abhijit.sinha@intel.com, daniel.m.buckley@intel.com, marchana@marvell.com, ktejasree@marvell.com, matan@nvidia.com, Radu Nicolau Date: Wed, 13 Oct 2021 13:13:28 +0100 Message-Id: <20211013121331.300245-8-radu.nicolau@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211013121331.300245-1-radu.nicolau@intel.com> References: <20210713133542.3550525-1-radu.nicolau@intel.com> <20211013121331.300245-1-radu.nicolau@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [PATCH v9 07/10] ipsec: add support for SA telemetry X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add telemetry support for ipsec SAs Signed-off-by: Declan Doherty Signed-off-by: Radu Nicolau Signed-off-by: Abhijit Sinha Signed-off-by: Daniel Martin Buckley Acked-by: Fan Zhang Acked-by: Konstantin Ananyev --- doc/guides/prog_guide/ipsec_lib.rst | 7 + doc/guides/rel_notes/release_21_11.rst | 1 + lib/ipsec/esp_inb.c | 18 +- lib/ipsec/esp_outb.c | 12 +- lib/ipsec/ipsec_telemetry.c | 244 +++++++++++++++++++++++++ lib/ipsec/meson.build | 6 +- lib/ipsec/rte_ipsec.h | 23 +++ lib/ipsec/sa.c | 10 +- lib/ipsec/sa.h | 9 + lib/ipsec/version.map | 9 + 10 files changed, 328 insertions(+), 11 deletions(-) create mode 100644 lib/ipsec/ipsec_telemetry.c diff --git a/doc/guides/prog_guide/ipsec_lib.rst b/doc/guides/prog_guide/ipsec_lib.rst index fc0af5eadb..2a262f8c51 100644 --- a/doc/guides/prog_guide/ipsec_lib.rst +++ b/doc/guides/prog_guide/ipsec_lib.rst @@ -321,6 +321,13 @@ Supported features AES_GMAC, HMAC-SHA1, NULL. +Telemetry support +------------------ +Telemetry support implements SA details and IPsec packet add data counters +statistics. Per SA telemetry statistics can be enabled using +``rte_ipsec_telemetry_sa_add`` and disabled using +``rte_ipsec_telemetry_sa_del``. Note that these calls are not thread safe. + Limitations ----------- diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst index 0a9c71d92e..70932fc8a9 100644 --- a/doc/guides/rel_notes/release_21_11.rst +++ b/doc/guides/rel_notes/release_21_11.rst @@ -159,6 +159,7 @@ New Features * Added support for AEAD algorithms AES_CCM, CHACHA20_POLY1305 and AES_GMAC. * Added support for NAT-T / UDP encapsulated ESP * Added support TSO offload support; only supported for inline crypto mode. + * Added support for SA telemetry. Removed Items diff --git a/lib/ipsec/esp_inb.c b/lib/ipsec/esp_inb.c index d66c88f05d..6fbe468a61 100644 --- a/lib/ipsec/esp_inb.c +++ b/lib/ipsec/esp_inb.c @@ -15,7 +15,7 @@ #include "misc.h" #include "pad.h" -typedef uint16_t (*esp_inb_process_t)(const struct rte_ipsec_sa *sa, +typedef uint16_t (*esp_inb_process_t)(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], uint32_t sqn[], uint32_t dr[], uint16_t num, uint8_t sqh_len); @@ -573,10 +573,10 @@ tun_process_step3(struct rte_mbuf *mb, uint64_t txof_msk, uint64_t txof_val) * *process* function for tunnel packets */ static inline uint16_t -tun_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], +tun_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], uint32_t sqn[], uint32_t dr[], uint16_t num, uint8_t sqh_len) { - uint32_t adj, i, k, tl; + uint32_t adj, i, k, tl, bytes; uint32_t hl[num], to[num]; struct rte_esp_tail espt[num]; struct rte_mbuf *ml[num]; @@ -598,6 +598,7 @@ tun_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], process_step1(mb[i], tlen, &ml[i], &espt[i], &hl[i], &to[i]); k = 0; + bytes = 0; for (i = 0; i != num; i++) { adj = hl[i] + cofs; @@ -621,10 +622,13 @@ tun_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], tun_process_step3(mb[i], sa->tx_offload.msk, sa->tx_offload.val); k++; + bytes += mb[i]->pkt_len; } else dr[i - k] = i; } + sa->statistics.count += k; + sa->statistics.bytes += bytes; return k; } @@ -632,11 +636,11 @@ tun_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], * *process* function for tunnel packets */ static inline uint16_t -trs_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], +trs_process(struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], uint32_t sqn[], uint32_t dr[], uint16_t num, uint8_t sqh_len) { char *np; - uint32_t i, k, l2, tl; + uint32_t i, k, l2, tl, bytes; uint32_t hl[num], to[num]; struct rte_esp_tail espt[num]; struct rte_mbuf *ml[num]; @@ -656,6 +660,7 @@ trs_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], process_step1(mb[i], tlen, &ml[i], &espt[i], &hl[i], &to[i]); k = 0; + bytes = 0; for (i = 0; i != num; i++) { tl = tlen + espt[i].pad_len; @@ -674,10 +679,13 @@ trs_process(const struct rte_ipsec_sa *sa, struct rte_mbuf *mb[], /* update mbuf's metadata */ trs_process_step3(mb[i]); k++; + bytes += mb[i]->pkt_len; } else dr[i - k] = i; } + sa->statistics.count += k; + sa->statistics.bytes += bytes; return k; } diff --git a/lib/ipsec/esp_outb.c b/lib/ipsec/esp_outb.c index d327c32a38..812ba1e5ec 100644 --- a/lib/ipsec/esp_outb.c +++ b/lib/ipsec/esp_outb.c @@ -623,7 +623,7 @@ uint16_t esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], uint16_t num) { - uint32_t i, k, icv_len, *icv; + uint32_t i, k, icv_len, *icv, bytes; struct rte_mbuf *ml; struct rte_ipsec_sa *sa; uint32_t dr[num]; @@ -632,6 +632,7 @@ esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], k = 0; icv_len = sa->icv_len; + bytes = 0; for (i = 0; i != num; i++) { if ((mb[i]->ol_flags & PKT_RX_SEC_OFFLOAD_FAILED) == 0) { @@ -642,10 +643,13 @@ esp_outb_sqh_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], icv = rte_pktmbuf_mtod_offset(ml, void *, ml->data_len - icv_len); remove_sqh(icv, icv_len); + bytes += mb[i]->pkt_len; k++; } else dr[i - k] = i; } + sa->statistics.count += k; + sa->statistics.bytes += bytes; /* handle unprocessed mbufs */ if (k != num) { @@ -665,16 +669,20 @@ static inline void inline_outb_mbuf_prepare(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], uint16_t num) { - uint32_t i, ol_flags; + uint32_t i, ol_flags, bytes; ol_flags = ss->security.ol_flags & RTE_SECURITY_TX_OLOAD_NEED_MDATA; + bytes = 0; for (i = 0; i != num; i++) { mb[i]->ol_flags |= PKT_TX_SEC_OFFLOAD; + bytes += mb[i]->pkt_len; if (ol_flags != 0) rte_security_set_pkt_metadata(ss->security.ctx, ss->security.ses, mb[i], NULL); } + ss->sa->statistics.count += num; + ss->sa->statistics.bytes += bytes; } diff --git a/lib/ipsec/ipsec_telemetry.c b/lib/ipsec/ipsec_telemetry.c new file mode 100644 index 0000000000..713da75f38 --- /dev/null +++ b/lib/ipsec/ipsec_telemetry.c @@ -0,0 +1,244 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2021 Intel Corporation + */ + +#include +#include +#include +#include "sa.h" + + +struct ipsec_telemetry_entry { + LIST_ENTRY(ipsec_telemetry_entry) next; + const struct rte_ipsec_sa *sa; +}; +static LIST_HEAD(ipsec_telemetry_head, ipsec_telemetry_entry) + ipsec_telemetry_list = LIST_HEAD_INITIALIZER(); + +static int +handle_telemetry_cmd_ipsec_sa_list(const char *cmd __rte_unused, + const char *params __rte_unused, + struct rte_tel_data *data) +{ + struct ipsec_telemetry_entry *entry; + rte_tel_data_start_array(data, RTE_TEL_U64_VAL); + + LIST_FOREACH(entry, &ipsec_telemetry_list, next) { + const struct rte_ipsec_sa *sa = entry->sa; + rte_tel_data_add_array_u64(data, rte_be_to_cpu_32(sa->spi)); + } + + return 0; +} + +/** + * Handle IPsec SA statistics telemetry request + * + * Return dict of SA's with dict of key/value counters + * + * { + * "SA_SPI_XX": {"count": 0, "bytes": 0, "errors": 0}, + * "SA_SPI_YY": {"count": 0, "bytes": 0, "errors": 0} + * } + * + */ +static int +handle_telemetry_cmd_ipsec_sa_stats(const char *cmd __rte_unused, + const char *params, + struct rte_tel_data *data) +{ + struct ipsec_telemetry_entry *entry; + const struct rte_ipsec_sa *sa; + uint32_t sa_spi = 0; + + if (params) { + sa_spi = rte_cpu_to_be_32((uint32_t)strtoul(params, NULL, 0)); + if (sa_spi == 0) + return -EINVAL; + } + + rte_tel_data_start_dict(data); + + LIST_FOREACH(entry, &ipsec_telemetry_list, next) { + char sa_name[64]; + sa = entry->sa; + static const char *name_pkt_cnt = "count"; + static const char *name_byte_cnt = "bytes"; + static const char *name_error_cnt = "errors"; + struct rte_tel_data *sa_data; + + /* If user provided SPI only get telemetry for that SA */ + if (sa_spi && (sa_spi != sa->spi)) + continue; + + /* allocate telemetry data struct for SA telemetry */ + sa_data = rte_tel_data_alloc(); + if (!sa_data) + return -ENOMEM; + + rte_tel_data_start_dict(sa_data); + + /* add telemetry key/values pairs */ + rte_tel_data_add_dict_u64(sa_data, name_pkt_cnt, + sa->statistics.count); + + rte_tel_data_add_dict_u64(sa_data, name_byte_cnt, + sa->statistics.bytes - + (sa->statistics.count * sa->hdr_len)); + + rte_tel_data_add_dict_u64(sa_data, name_error_cnt, + sa->statistics.errors.count); + + /* generate telemetry label */ + snprintf(sa_name, sizeof(sa_name), "SA_SPI_%i", + rte_be_to_cpu_32(sa->spi)); + + /* add SA telemetry to dictionary container */ + rte_tel_data_add_dict_container(data, sa_name, sa_data, 0); + } + + return 0; +} + +static int +handle_telemetry_cmd_ipsec_sa_details(const char *cmd __rte_unused, + const char *params, + struct rte_tel_data *data) +{ + struct ipsec_telemetry_entry *entry; + const struct rte_ipsec_sa *sa; + uint32_t sa_spi = 0; + + if (params) + sa_spi = rte_cpu_to_be_32((uint32_t)strtoul(params, NULL, 0)); + /* valid SPI needed */ + if (sa_spi == 0) + return -EINVAL; + + + rte_tel_data_start_dict(data); + + LIST_FOREACH(entry, &ipsec_telemetry_list, next) { + uint64_t mode; + sa = entry->sa; + if (sa_spi != sa->spi) + continue; + + /* add SA configuration key/values pairs */ + rte_tel_data_add_dict_string(data, "Type", + (sa->type & RTE_IPSEC_SATP_PROTO_MASK) == + RTE_IPSEC_SATP_PROTO_AH ? "AH" : "ESP"); + + rte_tel_data_add_dict_string(data, "Direction", + (sa->type & RTE_IPSEC_SATP_DIR_MASK) == + RTE_IPSEC_SATP_DIR_IB ? "Inbound" : "Outbound"); + + mode = sa->type & RTE_IPSEC_SATP_MODE_MASK; + + if (mode == RTE_IPSEC_SATP_MODE_TRANS) { + rte_tel_data_add_dict_string(data, "Mode", "Transport"); + } else { + rte_tel_data_add_dict_string(data, "Mode", "Tunnel"); + + if ((sa->type & RTE_IPSEC_SATP_NATT_MASK) == + RTE_IPSEC_SATP_NATT_ENABLE) { + if (sa->type & RTE_IPSEC_SATP_MODE_TUNLV4) { + rte_tel_data_add_dict_string(data, + "Tunnel-Type", + "IPv4-UDP"); + } else if (sa->type & + RTE_IPSEC_SATP_MODE_TUNLV6) { + rte_tel_data_add_dict_string(data, + "Tunnel-Type", + "IPv4-UDP"); + } + } else { + if (sa->type & RTE_IPSEC_SATP_MODE_TUNLV4) { + rte_tel_data_add_dict_string(data, + "Tunnel-Type", + "IPv4-UDP"); + } else if (sa->type & + RTE_IPSEC_SATP_MODE_TUNLV6) { + rte_tel_data_add_dict_string(data, + "Tunnel-Type", + "IPv4-UDP"); + } + } + } + + rte_tel_data_add_dict_string(data, + "extended-sequence-number", + (sa->type & RTE_IPSEC_SATP_ESN_MASK) == + RTE_IPSEC_SATP_ESN_ENABLE ? + "enabled" : "disabled"); + + if ((sa->type & RTE_IPSEC_SATP_DIR_MASK) == + RTE_IPSEC_SATP_DIR_IB) + + if (sa->sqn.inb.rsn[sa->sqn.inb.rdidx]) + rte_tel_data_add_dict_u64(data, + "sequence-number", + sa->sqn.inb.rsn[sa->sqn.inb.rdidx]->sqn); + else + rte_tel_data_add_dict_u64(data, + "sequence-number", 0); + else + rte_tel_data_add_dict_u64(data, "sequence-number", + sa->sqn.outb); + + rte_tel_data_add_dict_string(data, + "explicit-congestion-notification", + (sa->type & RTE_IPSEC_SATP_ECN_MASK) == + RTE_IPSEC_SATP_ECN_ENABLE ? + "enabled" : "disabled"); + + rte_tel_data_add_dict_string(data, + "copy-DSCP", + (sa->type & RTE_IPSEC_SATP_DSCP_MASK) == + RTE_IPSEC_SATP_DSCP_ENABLE ? + "enabled" : "disabled"); + } + + return 0; +} + + +int +rte_ipsec_telemetry_sa_add(const struct rte_ipsec_sa *sa) +{ + struct ipsec_telemetry_entry *entry = rte_zmalloc(NULL, + sizeof(struct ipsec_telemetry_entry), 0); + if (entry == NULL) + return -ENOMEM; + entry->sa = sa; + LIST_INSERT_HEAD(&ipsec_telemetry_list, entry, next); + return 0; +} + +void +rte_ipsec_telemetry_sa_del(const struct rte_ipsec_sa *sa) +{ + struct ipsec_telemetry_entry *entry; + LIST_FOREACH(entry, &ipsec_telemetry_list, next) { + if (sa == entry->sa) { + LIST_REMOVE(entry, next); + rte_free(entry); + return; + } + } +} + + +RTE_INIT(rte_ipsec_telemetry_init) +{ + rte_telemetry_register_cmd("/ipsec/sa/list", + handle_telemetry_cmd_ipsec_sa_list, + "Return list of IPsec SAs with telemetry enabled."); + rte_telemetry_register_cmd("/ipsec/sa/stats", + handle_telemetry_cmd_ipsec_sa_stats, + "Returns IPsec SA stastistics. Parameters: int sa_spi"); + rte_telemetry_register_cmd("/ipsec/sa/details", + handle_telemetry_cmd_ipsec_sa_details, + "Returns IPsec SA configuration. Parameters: int sa_spi"); +} + diff --git a/lib/ipsec/meson.build b/lib/ipsec/meson.build index 1497f573bb..ddb9ea1767 100644 --- a/lib/ipsec/meson.build +++ b/lib/ipsec/meson.build @@ -1,9 +1,11 @@ # SPDX-License-Identifier: BSD-3-Clause # Copyright(c) 2018 Intel Corporation -sources = files('esp_inb.c', 'esp_outb.c', 'sa.c', 'ses.c', 'ipsec_sad.c') +sources = files('esp_inb.c', 'esp_outb.c', + 'sa.c', 'ses.c', 'ipsec_sad.c', + 'ipsec_telemetry.c') headers = files('rte_ipsec.h', 'rte_ipsec_sa.h', 'rte_ipsec_sad.h') indirect_headers += files('rte_ipsec_group.h') -deps += ['mbuf', 'net', 'cryptodev', 'security', 'hash'] +deps += ['mbuf', 'net', 'cryptodev', 'security', 'hash', 'telemetry'] diff --git a/lib/ipsec/rte_ipsec.h b/lib/ipsec/rte_ipsec.h index dd60d95915..5308f250a7 100644 --- a/lib/ipsec/rte_ipsec.h +++ b/lib/ipsec/rte_ipsec.h @@ -158,6 +158,29 @@ rte_ipsec_pkt_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], return ss->pkt_func.process(ss, mb, num); } + +/** + * Enable per SA telemetry for a specific SA. + * Note that this function is not thread safe + * @param sa + * Pointer to the *rte_ipsec_sa* object that will have telemetry enabled. + * @return + * 0 on success, negative value otherwise. + */ +__rte_experimental +int +rte_ipsec_telemetry_sa_add(const struct rte_ipsec_sa *sa); + +/** + * Disable per SA telemetry for a specific SA. + * Note that this function is not thread safe + * @param sa + * Pointer to the *rte_ipsec_sa* object that will have telemetry disabled. + */ +__rte_experimental +void +rte_ipsec_telemetry_sa_del(const struct rte_ipsec_sa *sa); + #include #ifdef __cplusplus diff --git a/lib/ipsec/sa.c b/lib/ipsec/sa.c index 2830506385..d767b2036a 100644 --- a/lib/ipsec/sa.c +++ b/lib/ipsec/sa.c @@ -656,19 +656,25 @@ uint16_t pkt_flag_process(const struct rte_ipsec_session *ss, struct rte_mbuf *mb[], uint16_t num) { - uint32_t i, k; + uint32_t i, k, bytes; uint32_t dr[num]; RTE_SET_USED(ss); k = 0; + bytes = 0; for (i = 0; i != num; i++) { - if ((mb[i]->ol_flags & PKT_RX_SEC_OFFLOAD_FAILED) == 0) + if ((mb[i]->ol_flags & PKT_RX_SEC_OFFLOAD_FAILED) == 0) { k++; + bytes += mb[i]->pkt_len; + } else dr[i - k] = i; } + ss->sa->statistics.count += k; + ss->sa->statistics.bytes += bytes; + /* handle unprocessed mbufs */ if (k != num) { rte_errno = EBADMSG; diff --git a/lib/ipsec/sa.h b/lib/ipsec/sa.h index 107ebd1519..6e59f18e16 100644 --- a/lib/ipsec/sa.h +++ b/lib/ipsec/sa.h @@ -132,6 +132,15 @@ struct rte_ipsec_sa { struct replay_sqn *rsn[REPLAY_SQN_NUM]; } inb; } sqn; + /* Statistics */ + struct { + uint64_t count; + uint64_t bytes; + struct { + uint64_t count; + uint64_t authentication_failed; + } errors; + } statistics; } __rte_cache_aligned; diff --git a/lib/ipsec/version.map b/lib/ipsec/version.map index ba8753eac4..0af27ffd60 100644 --- a/lib/ipsec/version.map +++ b/lib/ipsec/version.map @@ -19,3 +19,12 @@ DPDK_22 { local: *; }; + +EXPERIMENTAL { + global: + + # added in 21.11 + rte_ipsec_telemetry_sa_add; + rte_ipsec_telemetry_sa_del; + +}; -- 2.25.1