From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6598CA04B1; Wed, 9 Sep 2020 02:28:34 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 22E791BF8A; Wed, 9 Sep 2020 02:28:31 +0200 (CEST) Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by dpdk.org (Postfix) with ESMTP id B400E1BEE1 for ; Wed, 9 Sep 2020 02:28:28 +0200 (CEST) IronPort-SDR: wVOK2nWt+fpPbjLVTQil7GdJ67O7zJe+jRPaNtbGCdVv8J38U5ZNajzM/Qp/4D+rGzWVIY6oae o3nskyH/fHUw== X-IronPort-AV: E=McAfee;i="6000,8403,9738"; a="222453007" X-IronPort-AV: E=Sophos;i="5.76,407,1592895600"; d="scan'208";a="222453007" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Sep 2020 17:28:27 -0700 IronPort-SDR: pK/cBdiKSOscJV+RxpKLIBTdP2ecudC+I3f8qrSfSXtbbIU7NEU8O4q3m5MZQYfxvIyPtiZFMN TZ3LQlGv9png== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.76,407,1592895600"; d="scan'208";a="286051334" Received: from npg-dpdk-patrickfu-casc2.sh.intel.com ([10.67.119.92]) by fmsmga008.fm.intel.com with ESMTP; 08 Sep 2020 17:28:23 -0700 From: Patrick Fu To: dev@dpdk.org Cc: thomas@monjalon.net, ferruh.yigit@intel.com, maxime.coquelin@redhat.com, bruce.richardson@intel.com, mm6021@att.com, zhihong.wang@intel.com, liang-min.wang@intel.com, konstantin.ananyev@intel.com, timothy.miskell@intel.com, cunming.liang@intel.com, patrick.fu@intel.com Date: Wed, 9 Sep 2020 08:22:45 +0800 Message-Id: <20200909002247.864844-2-patrick.fu@intel.com> X-Mailer: git-send-email 2.18.4 In-Reply-To: <20200909002247.864844-1-patrick.fu@intel.com> References: <20200909002247.864844-1-patrick.fu@intel.com> Subject: [dpdk-dev] [PATCH v1 1/3] lib/mirror: introduce traffic mirror API X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Network Test Access Point (TAP) is the network monitoring service commonly adotpted in SDN-based network infrastructures. When VMs are inter-connected over virtual switches, TAP requires vSwitch to mirror out network traffics from specific workload VM ports to the TAP device/VM ports. This patch introduce 2 a new APIs to support high-throughput packet mirroring: - rte_mirror_register() - rte_mirror_unregister() Applications use the new API to setup the source traffic port and the mirror traffic port. Packets flowing over the registered source ports will be transmited to the mirror port when registration succeeds. Signed-off-by: Liang-min Wang Signed-off-by: Patrick Fu Signed-off-by: Timothy Miskell --- config/common_base | 5 + lib/Makefile | 2 + lib/librte_mirror/Makefile | 20 ++ lib/librte_mirror/meson.build | 6 + lib/librte_mirror/rte_mirror.c | 297 +++++++++++++++++++++++ lib/librte_mirror/rte_mirror.h | 111 +++++++++ lib/librte_mirror/rte_mirror_version.map | 7 + lib/meson.build | 2 +- 8 files changed, 449 insertions(+), 1 deletion(-) create mode 100644 lib/librte_mirror/Makefile create mode 100644 lib/librte_mirror/meson.build create mode 100644 lib/librte_mirror/rte_mirror.c create mode 100644 lib/librte_mirror/rte_mirror.h create mode 100644 lib/librte_mirror/rte_mirror_version.map diff --git a/config/common_base b/config/common_base index fbf0ee70c..60217cc93 100644 --- a/config/common_base +++ b/config/common_base @@ -1110,6 +1110,11 @@ CONFIG_RTE_LIBRTE_GRAPH_STATS=y # CONFIG_RTE_LIBRTE_NODE=y +# +# Compile librte_mirror +# +CONFIG_RTE_LIBRTE_MIRROR=y + # # Compile the test application # diff --git a/lib/Makefile b/lib/Makefile index 8f5b68a2d..24175dd98 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -112,6 +112,8 @@ DIRS-$(CONFIG_RTE_LIBRTE_REORDER) += librte_reorder DEPDIRS-librte_reorder := librte_eal librte_mempool librte_mbuf DIRS-$(CONFIG_RTE_LIBRTE_PDUMP) += librte_pdump DEPDIRS-librte_pdump := librte_eal librte_mempool librte_mbuf librte_ethdev +DIRS-$(CONFIG_RTE_LIBRTE_MIRROR) += librte_mirror +DEPDIRS-librte_mirror := librte_eal librte_mempool librte_mbuf librte_ethdev DIRS-$(CONFIG_RTE_LIBRTE_GSO) += librte_gso DEPDIRS-librte_gso := librte_eal librte_mbuf librte_ethdev librte_net DEPDIRS-librte_gso += librte_mempool diff --git a/lib/librte_mirror/Makefile b/lib/librte_mirror/Makefile new file mode 100644 index 000000000..3ca1db734 --- /dev/null +++ b/lib/librte_mirror/Makefile @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020- Intel Corporation + +include $(RTE_SDK)/mk/rte.vars.mk + +# library name +LIB = librte_mirror.a + +CFLAGS += $(WERROR_FLAGS) -I$(SRCDIR) -O3 +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ethdev + +EXPORT_MAP := rte_mirror_version.map + +# all source are stored in SRCS-y +SRCS-$(CONFIG_RTE_LIBRTE_MIRROR) := rte_mirror.c + +# install this header file +SYMLINK-$(CONFIG_RTE_LIBRTE_MIRROR)-include := rte_mirror.h + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/lib/librte_mirror/meson.build b/lib/librte_mirror/meson.build new file mode 100644 index 000000000..2d0ba0ed2 --- /dev/null +++ b/lib/librte_mirror/meson.build @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: BSD-3-Clause +# Copyright(c) 2020 Intel Corporation + +sources = files('rte_mirror.c') +headers = files('rte_mirror.h') +deps += ['ethdev'] diff --git a/lib/librte_mirror/rte_mirror.c b/lib/librte_mirror/rte_mirror.c new file mode 100644 index 000000000..f2a85976a --- /dev/null +++ b/lib/librte_mirror/rte_mirror.c @@ -0,0 +1,297 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020- Intel Corporation + */ + +#include +#include +#include +#include +#include + +#include "rte_mirror.h" + +/* Macro for printing using RTE_LOG */ +extern int mirror_logtype; +#define MIRROR_LOG(level, fmt, args...) \ + rte_log(RTE_LOG_ ## level, mirror_logtype, "%s(): " fmt, \ + __func__, ## args) + +#define MAC_ADDR_MAP 0x0000FFFFFFFFFFFFULL +#define VLAN_INSERT_FLAG (PKT_TX_VLAN_PKT) +#define is_mac_addr_match(a, b) (((a^b)&MAC_ADDR_MAP) == 0) +#define INIT_MIRROR_DB_SIZE 8 +#define INVALID_PORT_ID 0xFFFF +#define SRC_MAC_OFFSET 6 +#define DST_MAC_OFFSET 0 + +static int mirror_port_db_size; +static int mirror_port_used; +static struct rte_mirror_offload_port *mirror_port_db; + +static int +rte_mirror_param_copy(struct rte_mirror_param *dst, + struct rte_mirror_param *src) +{ + void *extra_data; + + memcpy(dst, src, sizeof(struct rte_mirror_param)); + + if (src->extra_data_size) { + extra_data = malloc(src->extra_data_size); + if (!extra_data) { + MIRROR_LOG(ERR, "Out of memory !!!\n"); + return -1; + } + memcpy(extra_data, src->extra_data, src->extra_data_size); + + dst->extra_data = extra_data; + } else + dst->extra_data = NULL; + return 0; +} + +static struct rte_mirror_offload_port* +mirror_data_find(uint16_t port_id) +{ + int i; + + if (mirror_port_db == NULL) + return NULL; + for (i = 0; i < mirror_port_db_size; i++) { + if (port_id == mirror_port_db[i].port_id) + return &mirror_port_db[i]; + } + return NULL; +} + +static void +mirror_db_init(struct rte_mirror_offload_port *db, int size) +{ + int i; + + for (i = 0; i < size; i++) { + memset(&db[i], 0, sizeof(struct rte_mirror_offload_port)); + db[i].port_id = INVALID_PORT_ID; + } +} + +/* Double the db size when it runs out of space */ +static int +mirror_db_resize(void) +{ + int new_size = 2*mirror_port_db_size; + struct rte_mirror_offload_port *new_db = malloc( + sizeof(struct rte_mirror_offload_port)*new_size); + + if (new_db == NULL) { + MIRROR_LOG(ERR, "Out of memory!!!\n"); + return -1; + } + + memcpy(new_db, mirror_port_db, sizeof(struct rte_mirror_offload_port) + *mirror_port_db_size); + mirror_db_init(&new_db[mirror_port_db_size], mirror_port_db_size); + mirror_port_db_size = new_size; + mirror_port_db = new_db; + + return 0; +} + +/* release db when used count drop to 0 */ +static void +mirror_data_remove(uint16_t port_id, int tx) +{ + struct rte_mirror_offload_port *target = mirror_data_find(port_id); + + if (!target) + MIRROR_LOG(ERR, "Attempt to remove unsaved port, " + "%d, %s callback\n", port_id, tx?"tx" : "rx"); + + if (tx) + memset(&target->tx, 0, sizeof(struct rte_mirror_param)); + else + memset(&target->rx, 0, sizeof(struct rte_mirror_param)); + + if ((target->rx.mirror_cb == NULL) && + (target->tx.mirror_cb == NULL)) { + target->port_id = INVALID_PORT_ID; + mirror_port_used--; + if (mirror_port_used == 0) { + free(mirror_port_db); + mirror_port_db = NULL; + mirror_port_db_size = 0; + } + } +} + +static struct rte_mirror_offload_port* +mirror_data_add(uint16_t port_id, int tx, + struct rte_mirror_param *new_param) +{ + struct rte_mirror_offload_port *target = NULL; + struct rte_mirror_param *param; + int i; + + if (!mirror_port_db) { + mirror_port_db_size = INIT_MIRROR_DB_SIZE; + mirror_port_db = malloc(sizeof(struct rte_mirror_offload_port)* + mirror_port_db_size); + if (!mirror_port_db) { + MIRROR_LOG(ERR, "Out of memory!!!\n"); + return NULL; + } + mirror_db_init(mirror_port_db, mirror_port_db_size); + } + target = mirror_data_find(port_id); + if (target) { + if (tx) { + if (target->tx.mirror_cb) { + MIRROR_LOG(ERR, "Attempt to add ingress mirror " + "offloading on port, %d, " + "while one is outstanding\n", port_id); + return target; + } + + if (rte_mirror_param_copy(&target->tx, new_param)) + return NULL; + + } else { + if (target->rx.mirror_cb) { + MIRROR_LOG(ERR, "Attempt to add egress mirror " + "offloading on port, %d, " + "while one is outstanding\n", port_id); + return target; + } + + if (rte_mirror_param_copy(&target->rx, new_param)) + return NULL; + } + } else { + /* find an unused spot on db */ + for (i = 0; i < mirror_port_db_size; i++) { + if (mirror_port_db[i].port_id == INVALID_PORT_ID) + break; + } + if (i == mirror_port_db_size) { + if (mirror_db_resize()) + return NULL; + } + + param = tx ? &mirror_port_db[i].tx : &mirror_port_db[i].rx; + if (rte_mirror_param_copy(param, new_param)) + return NULL; + + target = &mirror_port_db[i]; + target->port_id = port_id; + mirror_port_used++; + } + return target; +} + +static inline void +mirror_pkt_update(struct rte_mbuf *pkt, uint16_t dst_vlan_id) +{ + pkt->ol_flags |= VLAN_INSERT_FLAG; + pkt->vlan_tci = dst_vlan_id; + rte_mbuf_refcnt_update(pkt, 1); +} + +int +rte_mirror_offload_register(uint16_t src_port, + struct rte_mirror_param *param, int tx_cb) +{ + int i; + struct rte_mirror_offload_port *port_info; + struct rte_mirror_param *data; + rte_rx_callback_fn rx_fn = NULL; + rte_tx_callback_fn tx_fn = NULL; + + + port_info = mirror_data_add(src_port, tx_cb, param); + if (!port_info) + return -EINVAL; + + data = tx_cb ? &port_info->tx : &port_info->rx; + data->pkt_buf = NULL; + data->mirror_cb = malloc(sizeof(struct rte_eth_rxtx_callbac *) * + data->n_src_queue); + if (!data->mirror_cb) { + MIRROR_LOG(ERR, "Out of memory !!!\n"); + return -EINVAL; + } + + switch (data->mirror_type) { + default: + MIRROR_LOG(ERR, "Un-supported mirror offloading type!!!\n"); + return -ENOTSUP; + } + + if (data->mirror_type != rte_mirror_type_port) { + data->pkt_buf = malloc(sizeof(struct rte_mbuf *) * + data->max_burst_size * data->n_src_queue); + if (!data->pkt_buf) { + RTE_ETHDEV_LOG(ERR, "Out of memory !!!"); + return -EINVAL; + } + } + + if (!tx_cb) { + for (i = 0; i < data->n_src_queue; i++) { + data->mirror_cb[i] = rte_eth_add_rx_callback(src_port, + i, rx_fn, data); + } + } else { + for (i = 0; i < data->n_src_queue; i++) { + data->mirror_cb[i] = rte_eth_add_tx_callback(src_port, + i, tx_fn, data); + } + } + + return 0; +} + +int +rte_mirror_offload_unregister(uint16_t src_port, int tx_cb) +{ + /* release both cb and pkt_buf */ + int i; + struct rte_mirror_offload_port *port_info; + struct rte_mirror_param *data; + + port_info = mirror_data_find(src_port); + if (port_info == NULL) { + MIRROR_LOG(ERR, "Source port %d is not on outstanding " + "port mirror db\n", src_port); + return -1; + } + data = tx_cb ? &port_info->tx : &port_info->rx; + + for (i = 0; i < data->n_src_queue; i++) { + if (data->mirror_cb[i]) { + if (tx_cb) + rte_eth_remove_tx_callback(src_port, + i, data->mirror_cb[i]); + else + rte_eth_remove_rx_callback(src_port, + i, data->mirror_cb[i]); + } + data->mirror_cb[i] = 0; + } + free(data->mirror_cb); + + if (data->pkt_buf) { + free(data->pkt_buf); + data->pkt_buf = NULL; + } + + if (data->extra_data) { + free(data->extra_data); + data->extra_data = NULL; + data->extra_data_size = 0; + } + + mirror_data_remove(src_port, tx_cb); + return 0; +} + +RTE_LOG_REGISTER(mirror_logtype, lib.mirror, NOTICE); diff --git a/lib/librte_mirror/rte_mirror.h b/lib/librte_mirror/rte_mirror.h new file mode 100644 index 000000000..71f3e1ea1 --- /dev/null +++ b/lib/librte_mirror/rte_mirror.h @@ -0,0 +1,111 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright(c) 2020- Intel Corporation + */ + +#ifndef _RTE_MIRROR_H_ +#define _RTE_MIRROR_H_ + +/** + * @file + * RTE mirror + * + * packet mirror offloading library to provide packet mirroring support on dpdk. + */ + +#include +#include +#include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * Function type used for mirrored packet header scan. + * + * The callback function is called on each mirrored packet + * to determinate if mirror condition is match. + * + * @param pkt + * Pointer to one packet data. + * @param user_param + * The arbitrary user parameter passed in by the application when the callback + * was originally configured. + * @return + * One if the condition matches, else 0. + */ +typedef int (*rte_mirror_scan_fn)(struct rte_mbuf *pkt, void *user_param); + +typedef enum { + rte_mirror_type_port = 0, /* port mirror */ + rte_mirror_type_flow_mac, /* mac based flow mirror */ + rte_mirror_type_flow_custom, /* custom flow mirror */ + rte_mirror_type_invalid = 0xFF, /* invalid mirror_type */ +} rte_mirror_type; + +struct rte_mirror_param { + uint16_t dst_port_id; + uint16_t dst_vlan_id; + rte_spinlock_t *locks; + int n_src_queue; + int n_dst_queue; + struct rte_mbuf **pkt_buf; + const struct rte_eth_rxtx_callback **mirror_cb; + unsigned int max_burst_size; + rte_mirror_scan_fn custom_scan; + rte_mirror_type mirror_type; + unsigned int extra_data_size; + void *extra_data; /* extra mirror parameter */ +}; + +struct rte_mirror_offload_port { + uint16_t port_id; + struct rte_mirror_param rx; + struct rte_mirror_param tx; +}; + +/** + * Add a port mirroring via port offloading + * + * This function adds traffic (ingress or egress) mirroring from + * the specified port + * + * @param src_port + * The source port where port traffic is to be captured and sent to + * the offload port. + * @param param + * The port mirror offloading parameters + * @param tx_cb + * 1: register tx callback; 0: register rx callback + * @return + * 0 = success + * N > 0 any error encouter + */ +__rte_experimental +int rte_mirror_offload_register(uint16_t src_port, + struct rte_mirror_param *param, int tx_cb); + +/** + * Remove a port mirroring via port offloading + * + * This function removes traffic (ingress or egress) mirroring from + * the specified port + * + * @param src_port + * The port-id of source port where port traffic is to be captured + * and sent to the offload port. + * @param tx_cb + * 1: tx callback; 0: rx callback + * @return + * 0 = success + * N > 0 any error encouter + */ +__rte_experimental +int rte_mirror_offload_unregister(uint16_t src_port, int tx_cb); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_PDUMP_H_ */ diff --git a/lib/librte_mirror/rte_mirror_version.map b/lib/librte_mirror/rte_mirror_version.map new file mode 100644 index 000000000..e0a4d69d1 --- /dev/null +++ b/lib/librte_mirror/rte_mirror_version.map @@ -0,0 +1,7 @@ +EXPERIMENTAL { + global: + + rte_mirror_offload_register; + rte_mirror_offload_unregister; +}; + diff --git a/lib/meson.build b/lib/meson.build index 3852c0156..6674749dd 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -24,7 +24,7 @@ libraries = [ 'distributor', 'efd', 'eventdev', 'gro', 'gso', 'ip_frag', 'jobstats', 'kni', 'latencystats', 'lpm', 'member', - 'power', 'pdump', 'rawdev', 'regexdev', + 'power', 'pdump', 'rawdev', 'regexdev', 'mirror', 'rib', 'reorder', 'sched', 'security', 'stack', 'vhost', # ipsec lib depends on net, crypto and security 'ipsec', -- 2.18.4