From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by dpdk.org (Postfix) with ESMTP id 3B38BC46E for ; Fri, 19 Feb 2016 06:06:33 +0100 (CET) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP; 18 Feb 2016 21:06:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.22,469,1449561600"; d="scan'208";a="906558247" Received: from irvmail001.ir.intel.com ([163.33.26.43]) by fmsmga001.fm.intel.com with ESMTP; 18 Feb 2016 21:06:30 -0800 Received: from sivswdev02.ir.intel.com (sivswdev02.ir.intel.com [10.237.217.46]) by irvmail001.ir.intel.com (8.14.3/8.13.6/MailSET/Hub) with ESMTP id u1J56TK5031069; Fri, 19 Feb 2016 05:06:29 GMT Received: from sivswdev02.ir.intel.com (localhost [127.0.0.1]) by sivswdev02.ir.intel.com with ESMTP id u1J56TIR014925; Fri, 19 Feb 2016 05:06:29 GMT Received: (from fyigit@localhost) by sivswdev02.ir.intel.com with id u1J56TOe014921; Fri, 19 Feb 2016 05:06:29 GMT From: Ferruh Yigit To: dev@dpdk.org Date: Fri, 19 Feb 2016 05:05:49 +0000 Message-Id: <1455858349-14639-3-git-send-email-ferruh.yigit@intel.com> X-Mailer: git-send-email 1.7.4.1 In-Reply-To: <1455858349-14639-1-git-send-email-ferruh.yigit@intel.com> References: <1453912360-18179-1-git-send-email-ferruh.yigit@intel.com> <1455858349-14639-1-git-send-email-ferruh.yigit@intel.com> Subject: [dpdk-dev] [PATCH v2 2/2] kdp: add virtual PMD for kernel slow data path communication X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Feb 2016 05:06:34 -0000 This patch provides slow data path communication to the Linux kernel. Patch is based on librte_kni, and heavily re-uses it. The main difference is librte_kni library converted into a PMD, to provide ease of use for applications. Now any application can use slow path communication without any update in application, because of existing eal support for virtual PMD. Also this PMD supports two methods to send packets to the Linux, first one is custom FIFO implementation with help of KDP kernel module, second one is Linux in-kernel tun/tap support. PMD first checks for KDP kernel module, if fails it tries to create and use a tap interface. With FIFO method: PMD's rx_pkt_burst() get packets from FIFO, and tx_pkt_burst() puts packet to the FIFO. The corresponding Linux virtual network device driver code also gets/puts packets from FIFO as they are coming from hardware. With tun/tap method: no external kernel module required, PMD reads from and writes packets to the tap interface file descriptor. Tap interface has performance penalty against FIFO implementation. Signed-off-by: Ferruh Yigit --- v2: * Use rtnetlink to create interfaces --- MAINTAINERS | 1 + config/common_linuxapp | 1 + doc/guides/nics/pcap_ring.rst | 125 ++++++- doc/guides/rel_notes/release_16_04.rst | 6 + drivers/net/Makefile | 3 +- drivers/net/kdp/Makefile | 61 +++ drivers/net/kdp/rte_eth_kdp.c | 501 +++++++++++++++++++++++++ drivers/net/kdp/rte_kdp.c | 633 ++++++++++++++++++++++++++++++++ drivers/net/kdp/rte_kdp.h | 116 ++++++ drivers/net/kdp/rte_kdp_fifo.h | 91 +++++ drivers/net/kdp/rte_kdp_tap.c | 101 +++++ drivers/net/kdp/rte_pmd_kdp_version.map | 4 + lib/librte_eal/common/include/rte_log.h | 3 +- mk/rte.app.mk | 3 +- 14 files changed, 1643 insertions(+), 6 deletions(-) create mode 100644 drivers/net/kdp/Makefile create mode 100644 drivers/net/kdp/rte_eth_kdp.c create mode 100644 drivers/net/kdp/rte_kdp.c create mode 100644 drivers/net/kdp/rte_kdp.h create mode 100644 drivers/net/kdp/rte_kdp_fifo.h create mode 100644 drivers/net/kdp/rte_kdp_tap.c create mode 100644 drivers/net/kdp/rte_pmd_kdp_version.map diff --git a/MAINTAINERS b/MAINTAINERS index 05ffe26..deaeea3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -260,6 +260,7 @@ F: doc/guides/sample_app_ug/kernel_nic_interface.rst Linux KDP M: Ferruh Yigit F: lib/librte_eal/linuxapp/kdp/ +F: drivers/net/kdp/ Linux AF_PACKET M: John W. Linville diff --git a/config/common_linuxapp b/config/common_linuxapp index e1b5032..aa13719 100644 --- a/config/common_linuxapp +++ b/config/common_linuxapp @@ -316,6 +316,7 @@ CONFIG_RTE_LIBRTE_PMD_NULL=y # # Compile KDP PMD # +CONFIG_RTE_LIBRTE_PMD_KDP=y CONFIG_RTE_KDP_KMOD=y CONFIG_RTE_KDP_PREEMPT_DEFAULT=y diff --git a/doc/guides/nics/pcap_ring.rst b/doc/guides/nics/pcap_ring.rst index aa48d33..b602e65 100644 --- a/doc/guides/nics/pcap_ring.rst +++ b/doc/guides/nics/pcap_ring.rst @@ -28,11 +28,11 @@ (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -Libpcap and Ring Based Poll Mode Drivers -======================================== +Software Poll Mode Drivers +========================== In addition to Poll Mode Drivers (PMDs) for physical and virtual hardware, -the DPDK also includes two pure-software PMDs. These two drivers are: +the DPDK also includes pure-software PMDs. These drivers are: * A libpcap -based PMD (librte_pmd_pcap) that reads and writes packets using libpcap, - both from files on disk, as well as from physical NIC devices using standard Linux kernel drivers. @@ -40,6 +40,10 @@ the DPDK also includes two pure-software PMDs. These two drivers are: * A ring-based PMD (librte_pmd_ring) that allows a set of software FIFOs (that is, rte_ring) to be accessed using the PMD APIs, as though they were physical NICs. +* A slow data path PMD (librte_pmd_kdp) that allows send/get packets to/from OS network + stack as it is a physical NIC. + + .. note:: The libpcap -based PMD is disabled by default in the build configuration files, @@ -211,6 +215,121 @@ Multiple devices may be specified, separated by commas. Done. +Kernel Data Path PMD +~~~~~~~~~~~~~~~~~~~~ + +Kernel Data Path (KDP) PMD is to communicate with OS network stack easily by application. + +.. code-block:: console + + ./testpmd --vdev eth_kdp0 --vdev eth_kdp1 -- -i + ... + Configuring Port 0 (socket 0) + Port 0: 00:00:00:00:00:00 + Configuring Port 1 (socket 0) + Port 1: 00:00:00:00:00:00 + Checking link statuses... + Port 0 Link Up - speed 10000 Mbps - full-duplex + Port 1 Link Up - speed 10000 Mbps - full-duplex + Done + +KDP PMD supports two type of communication: + +* Custom FIFO implementation +* tun/tap implementation + +Custom FIFO implementation gives more performance but requires KDP kernel module (rte_kdp.ko) inserted. + +By default FIFO communication has priority, if KDP kernel module is not inserted, tun/tap communication used. + +If KDP kernel module inserted, above testpmd command will create following virtual interfaces, these can be used as any interface. + +.. code-block:: console + + # ifconfig kdp0; ifconfig kdp1 + kdp0: flags=4098 mtu 1500 + ether 00:00:00:00:00:00 txqueuelen 1000 (Ethernet) + RX packets 0 bytes 0 (0.0 B) + RX errors 0 dropped 0 overruns 0 frame 0 + TX packets 0 bytes 0 (0.0 B) + TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 + + kdp1: flags=4098 mtu 1500 + ether 00:00:00:00:00:00 txqueuelen 1000 (Ethernet) + RX packets 0 bytes 0 (0.0 B) + RX errors 0 dropped 0 overruns 0 frame 0 + TX packets 0 bytes 0 (0.0 B) + TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 + + +With tun/tap communication method, following interfaces are created: + +.. code-block:: console + + # ifconfig tap_kdp0; ifconfig tap_kdp1 + tap_kdp0: flags=4163 mtu 1500 + inet6 fe80::341f:afff:feb7:23db prefixlen 64 scopeid 0x20 + ether 36:1f:af:b7:23:db txqueuelen 500 (Ethernet) + RX packets 126624864 bytes 6184828655 (5.7 GiB) + RX errors 0 dropped 0 overruns 0 frame 0 + TX packets 126236898 bytes 6150306636 (5.7 GiB) + TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 + + tap_kdp1: flags=4163 mtu 1500 + inet6 fe80::f030:b4ff:fe94:b720 prefixlen 64 scopeid 0x20 + ether f2:30:b4:94:b7:20 txqueuelen 500 (Ethernet) + RX packets 126237370 bytes 6150329717 (5.7 GiB) + RX errors 0 dropped 9 overruns 0 frame 0 + TX packets 126624896 bytes 6184826874 (5.7 GiB) + TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 + +DPDK application can be used to forward packets between these interfaces: + +.. code-block:: console + + In Linux: + ip l add br0 type bridge + ip l set tap_kdp0 master br0 + ip l set tap_kdp1 master br0 + ip l set br0 up + ip l set tap_kdp0 up + ip l set tap_kdp1 up + + + In testpmd: + testpmd> start + io packet forwarding - CRC stripping disabled - packets/burst=32 + nb forwarding cores=1 - nb forwarding ports=2 + RX queues=1 - RX desc=128 - RX free threshold=0 + RX threshold registers: pthresh=0 hthresh=0 wthresh=0 + TX queues=1 - TX desc=512 - TX free threshold=0 + TX threshold registers: pthresh=0 hthresh=0 wthresh=0 + TX RS bit threshold=0 - TXQ flags=0x0 + testpmd> stop + Telling cores to stop... + Waiting for lcores to finish... + + ---------------------- Forward statistics for port 0 ---------------------- + RX-packets: 973900 RX-dropped: 0 RX-total: 973900 + TX-packets: 973903 TX-dropped: 0 TX-total: 973903 + ---------------------------------------------------------------------------- + + ---------------------- Forward statistics for port 1 ---------------------- + RX-packets: 973903 RX-dropped: 0 RX-total: 973903 + TX-packets: 973900 TX-dropped: 0 TX-total: 973900 + ---------------------------------------------------------------------------- + + +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++ + RX-packets: 1947803 RX-dropped: 0 RX-total: 1947803 + TX-packets: 1947803 TX-dropped: 0 TX-total: 1947803 + ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + + Done. + + + + + Using the Poll Mode Driver from an Application ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/guides/rel_notes/release_16_04.rst b/doc/guides/rel_notes/release_16_04.rst index eb1b3b2..d17778c 100644 --- a/doc/guides/rel_notes/release_16_04.rst +++ b/doc/guides/rel_notes/release_16_04.rst @@ -44,6 +44,12 @@ This section should contain new features added in this release. Sample format: Add the offload and negotiation of checksum and TSO between vhost-user and vanilla Linux virtio guest. +* **Added Slow Data Path support.** + + * This is based on KNI work and in long term intends to replace it. + * Added Kernel Data Path (KDP) kernel module. + * Added KDP virtual PMD. + Resolved Issues --------------- diff --git a/drivers/net/Makefile b/drivers/net/Makefile index 6e4497e..0be06f5 100644 --- a/drivers/net/Makefile +++ b/drivers/net/Makefile @@ -1,6 +1,6 @@ # BSD LICENSE # -# Copyright(c) 2010-2015 Intel Corporation. All rights reserved. +# Copyright(c) 2010-2016 Intel Corporation. All rights reserved. # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -51,6 +51,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += szedata2 DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3 DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt +DIRS-$(CONFIG_RTE_LIBRTE_PMD_KDP) += kdp include $(RTE_SDK)/mk/rte.sharelib.mk include $(RTE_SDK)/mk/rte.subdir.mk diff --git a/drivers/net/kdp/Makefile b/drivers/net/kdp/Makefile new file mode 100644 index 0000000..035056e --- /dev/null +++ b/drivers/net/kdp/Makefile @@ -0,0 +1,61 @@ +# BSD LICENSE +# +# Copyright(c) 2016 Intel Corporation. All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_kdp.a + +CFLAGS += -O3 +CFLAGS += $(WERROR_FLAGS) + +EXPORT_MAP := rte_pmd_kdp_version.map + +LIBABIVER := 1 + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_KDP) += rte_eth_kdp.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_KDP) += rte_kdp.c +SRCS-$(CONFIG_RTE_LIBRTE_PMD_KDP) += rte_kdp_tap.c + +# +# Export include files +# +SYMLINK-y-include += + +# this lib depends upon: +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_KDP) += lib/librte_mbuf +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_KDP) += lib/librte_ether + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/kdp/rte_eth_kdp.c b/drivers/net/kdp/rte_eth_kdp.c new file mode 100644 index 0000000..68dd734 --- /dev/null +++ b/drivers/net/kdp/rte_eth_kdp.c @@ -0,0 +1,501 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include + +#include "rte_kdp.h" + +#define MAX_PACKET_SZ 2048 + +struct pmd_queue_stats { + uint64_t pkts; + uint64_t bytes; + uint64_t err_pkts; +}; + +struct pmd_queue { + struct pmd_internals *internals; + struct rte_mempool *mb_pool; + + struct pmd_queue_stats rx; + struct pmd_queue_stats tx; +}; + +struct pmd_internals { + struct kdp_data *kdp; + struct kdp_tap_data *kdp_tap; + + struct pmd_queue rx_queues[RTE_MAX_QUEUES_PER_PORT]; + struct pmd_queue tx_queues[RTE_MAX_QUEUES_PER_PORT]; +}; + +static struct ether_addr eth_addr = { .addr_bytes = {0} }; +static const char *drivername = "KDP PMD"; +static struct rte_eth_link pmd_link = { + .link_speed = 10000, + .link_duplex = ETH_LINK_FULL_DUPLEX, + .link_status = 0 +}; + +static uint16_t +eth_kdp_rx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) +{ + struct pmd_queue *kdp_q = q; + struct kdp_data *kdp = kdp_q->internals->kdp; + uint16_t nb_pkts; + + nb_pkts = kdp_rx_burst(kdp, bufs, nb_bufs); + + kdp_q->rx.pkts += nb_pkts; + kdp_q->rx.err_pkts += nb_bufs - nb_pkts; + + return nb_pkts; +} + +static uint16_t +eth_kdp_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) +{ + struct pmd_queue *kdp_q = q; + struct kdp_data *kdp = kdp_q->internals->kdp; + uint16_t nb_pkts; + + nb_pkts = kdp_tx_burst(kdp, bufs, nb_bufs); + + kdp_q->tx.pkts += nb_pkts; + kdp_q->tx.err_pkts += nb_bufs - nb_pkts; + + return nb_pkts; +} + +static uint16_t +eth_kdp_tap_rx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) +{ + struct pmd_queue *kdp_q = q; + struct pmd_internals *internals = kdp_q->internals; + struct kdp_tap_data *kdp_tap = internals->kdp_tap; + struct rte_mbuf *m; + int ret; + unsigned i; + + for (i = 0; i < nb_bufs; i++) { + m = rte_pktmbuf_alloc(kdp_q->mb_pool); + bufs[i] = m; + ret = read(kdp_tap->tap_fd, rte_pktmbuf_mtod(m, void *), + MAX_PACKET_SZ); + if (ret < 0) { + rte_pktmbuf_free(m); + break; + } + + m->nb_segs = 1; + m->next = NULL; + m->pkt_len = (uint16_t)ret; + m->data_len = (uint16_t)ret; + } + + kdp_q->rx.pkts += i; + kdp_q->rx.err_pkts += nb_bufs - i; + + return i; +} + +static uint16_t +eth_kdp_tap_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs) +{ + struct pmd_queue *kdp_q = q; + struct pmd_internals *internals = kdp_q->internals; + struct kdp_tap_data *kdp_tap = internals->kdp_tap; + struct rte_mbuf *m; + unsigned i; + + for (i = 0; i < nb_bufs; i++) { + m = bufs[i]; + write(kdp_tap->tap_fd, rte_pktmbuf_mtod(m, void*), + rte_pktmbuf_data_len(m)); + rte_pktmbuf_free(m); + } + + kdp_q->tx.pkts += i; + kdp_q->tx.err_pkts += nb_bufs - i; + + return i; +} + +static int +eth_kdp_start(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct kdp_conf conf; + uint16_t port_id = dev->data->port_id; + int ret = 0; + + snprintf(conf.name, RTE_KDP_NAMESIZE, KDP_DEVICE "%u", + port_id); + conf.force_bind = 0; + conf.port_id = port_id; + conf.mbuf_size = MAX_PACKET_SZ; + + ret = kdp_start(internals->kdp, + internals->rx_queues[0].mb_pool, + &conf); + if (ret) + RTE_LOG(ERR, KDP, "Fail to create kdp for port: %d\n", + port_id); + + return ret; +} + +static int +eth_kdp_dev_start(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + int ret; + + if (internals->kdp) { + ret = eth_kdp_start(dev); + if (ret) + return -1; + } + + dev->data->dev_link.link_status = 1; + return 0; +} + +static void +eth_kdp_dev_stop(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + + if (internals->kdp) + kdp_stop(internals->kdp); + + dev->data->dev_link.link_status = 0; +} + +static void +eth_kdp_dev_close(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct kdp_data *kdp = internals->kdp; + struct kdp_tap_data *kdp_tap = internals->kdp_tap; + + if (kdp) { + kdp_close(kdp); + + rte_free(kdp); + internals->kdp = NULL; + } + + if (kdp_tap) { + kdp_tap_close(kdp_tap); + + rte_free(kdp_tap); + internals->kdp_tap = NULL; + } + + rte_free(dev->data->dev_private); + dev->data->dev_private = NULL; +} + +static int +eth_kdp_dev_configure(struct rte_eth_dev *dev __rte_unused) +{ + return 0; +} + +static void +eth_kdp_dev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) +{ + struct rte_eth_dev_data *data = dev->data; + + dev_info->driver_name = data->drv_name; + dev_info->max_mac_addrs = 1; + dev_info->max_rx_pktlen = (uint32_t)-1; + dev_info->max_rx_queues = data->nb_rx_queues; + dev_info->max_tx_queues = data->nb_tx_queues; + dev_info->min_rx_bufsize = 0; + dev_info->pci_dev = NULL; +} + +static int +eth_kdp_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t rx_queue_id __rte_unused, + uint16_t nb_rx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, + struct rte_mempool *mb_pool) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct pmd_queue *q; + + q = &internals->rx_queues[rx_queue_id]; + q->internals = internals; + q->mb_pool = mb_pool; + + dev->data->rx_queues[rx_queue_id] = q; + + return 0; +} + +static int +eth_kdp_tx_queue_setup(struct rte_eth_dev *dev, + uint16_t tx_queue_id, + uint16_t nb_tx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct pmd_queue *q; + + q = &internals->tx_queues[tx_queue_id]; + q->internals = internals; + + dev->data->tx_queues[tx_queue_id] = q; + + return 0; +} + +static void +eth_kdp_queue_release(void *q __rte_unused) +{ +} + +static int +eth_kdp_link_update(struct rte_eth_dev *dev __rte_unused, + int wait_to_complete __rte_unused) +{ + return 0; +} + +static void +eth_kdp_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats) +{ + unsigned i, num_stats; + unsigned long rx_packets_total = 0, rx_bytes_total = 0; + unsigned long tx_packets_total = 0, tx_bytes_total = 0; + unsigned long tx_packets_err_total = 0; + struct rte_eth_dev_data *data = dev->data; + struct pmd_queue *q; + + num_stats = RTE_MIN((unsigned)RTE_ETHDEV_QUEUE_STAT_CNTRS, + data->nb_rx_queues); + for (i = 0; i < num_stats; i++) { + q = data->rx_queues[i]; + stats->q_ipackets[i] = q->rx.pkts; + stats->q_ibytes[i] = q->rx.bytes; + rx_packets_total += stats->q_ipackets[i]; + rx_bytes_total += stats->q_ibytes[i]; + } + + num_stats = RTE_MIN((unsigned)RTE_ETHDEV_QUEUE_STAT_CNTRS, + data->nb_tx_queues); + for (i = 0; i < num_stats; i++) { + q = data->tx_queues[i]; + stats->q_opackets[i] = q->tx.pkts; + stats->q_obytes[i] = q->tx.bytes; + stats->q_errors[i] = q->tx.err_pkts; + tx_packets_total += stats->q_opackets[i]; + tx_bytes_total += stats->q_obytes[i]; + tx_packets_err_total += stats->q_errors[i]; + } + + stats->ipackets = rx_packets_total; + stats->ibytes = rx_bytes_total; + stats->opackets = tx_packets_total; + stats->obytes = tx_bytes_total; + stats->oerrors = tx_packets_err_total; +} + +static void +eth_kdp_stats_reset(struct rte_eth_dev *dev) +{ + unsigned i; + struct rte_eth_dev_data *data = dev->data; + struct pmd_queue *q; + + for (i = 0; i < data->nb_rx_queues; i++) { + q = data->rx_queues[i]; + q->rx.pkts = 0; + q->rx.bytes = 0; + } + for (i = 0; i < data->nb_tx_queues; i++) { + q = data->tx_queues[i]; + q->tx.pkts = 0; + q->tx.bytes = 0; + q->tx.err_pkts = 0; + } +} + +static const struct eth_dev_ops eth_kdp_ops = { + .dev_start = eth_kdp_dev_start, + .dev_stop = eth_kdp_dev_stop, + .dev_close = eth_kdp_dev_close, + .dev_configure = eth_kdp_dev_configure, + .dev_infos_get = eth_kdp_dev_info, + .rx_queue_setup = eth_kdp_rx_queue_setup, + .tx_queue_setup = eth_kdp_tx_queue_setup, + .rx_queue_release = eth_kdp_queue_release, + .tx_queue_release = eth_kdp_queue_release, + .link_update = eth_kdp_link_update, + .stats_get = eth_kdp_stats_get, + .stats_reset = eth_kdp_stats_reset, +}; + +static struct rte_eth_dev * +eth_kdp_create(const char *name, unsigned numa_node) +{ + uint16_t nb_rx_queues = 1; + uint16_t nb_tx_queues = 1; + struct rte_eth_dev_data *data = NULL; + struct pmd_internals *internals = NULL; + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Creating kdp ethdev on numa socket %u\n", + numa_node); + + data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node); + if (data == NULL) + goto error; + + internals = rte_zmalloc_socket(name, sizeof(*internals), 0, numa_node); + if (internals == NULL) + goto error; + + /* reserve an ethdev entry */ + eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); + if (eth_dev == NULL) + goto error; + + data->dev_private = internals; + data->port_id = eth_dev->data->port_id; + memmove(data->name, eth_dev->data->name, sizeof(data->name)); + data->nb_rx_queues = nb_rx_queues; + data->nb_tx_queues = nb_tx_queues; + data->dev_link = pmd_link; + data->mac_addrs = ð_addr; + + eth_dev->data = data; + eth_dev->dev_ops = ð_kdp_ops; + eth_dev->driver = NULL; + + data->dev_flags = RTE_ETH_DEV_DETACHABLE; + data->kdrv = RTE_KDRV_NONE; + data->drv_name = drivername; + data->numa_node = numa_node; + + return eth_dev; + +error: + rte_free(data); + rte_free(internals); + + return NULL; +} + +static int +eth_kdp_devinit(const char *name, const char *params __rte_unused) +{ + struct rte_eth_dev *eth_dev = NULL; + struct pmd_internals *internals; + struct kdp_data *kdp; + struct kdp_tap_data *kdp_tap = NULL; + uint16_t port_id; + + RTE_LOG(INFO, PMD, "Initializing eth_kdp for %s\n", name); + + eth_dev = eth_kdp_create(name, rte_socket_id()); + if (eth_dev == NULL) + return -1; + + internals = eth_dev->data->dev_private; + port_id = eth_dev->data->port_id; + + kdp = kdp_init(port_id); + if (kdp == NULL) + kdp_tap = kdp_tap_init(port_id); + + if (kdp == NULL && kdp_tap == NULL) { + rte_eth_dev_release_port(eth_dev); + rte_free(internals); + + /* Not return error to prevent panic in rte_eal_init() */ + return 0; + } + + internals->kdp = kdp; + internals->kdp_tap = kdp_tap; + + if (kdp == NULL) { + eth_dev->rx_pkt_burst = eth_kdp_tap_rx; + eth_dev->tx_pkt_burst = eth_kdp_tap_tx; + } else { + eth_dev->rx_pkt_burst = eth_kdp_rx; + eth_dev->tx_pkt_burst = eth_kdp_tx; + } + + return 0; +} + +static int +eth_kdp_devuninit(const char *name) +{ + struct rte_eth_dev *eth_dev = NULL; + + RTE_LOG(INFO, PMD, "Un-Initializing eth_kdp for %s\n", name); + + /* find the ethdev entry */ + eth_dev = rte_eth_dev_allocated(name); + if (eth_dev == NULL) + return -1; + + eth_kdp_dev_stop(eth_dev); + + if (eth_dev->data) + rte_free(eth_dev->data->dev_private); + rte_free(eth_dev->data); + + rte_eth_dev_release_port(eth_dev); + + kdp_uninit(); + + return 0; +} + +static struct rte_driver eth_kdp_drv = { + .name = "eth_kdp", + .type = PMD_VDEV, + .init = eth_kdp_devinit, + .uninit = eth_kdp_devuninit, +}; + +PMD_REGISTER_DRIVER(eth_kdp_drv); diff --git a/drivers/net/kdp/rte_kdp.c b/drivers/net/kdp/rte_kdp.c new file mode 100644 index 0000000..ed50a0f --- /dev/null +++ b/drivers/net/kdp/rte_kdp.c @@ -0,0 +1,633 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef RTE_EXEC_ENV_LINUXAPP +#error "KDP is not supported" +#endif + +#include +#include +#include + +#include +#include +#include + +#include "rte_kdp.h" +#include "rte_kdp_fifo.h" + +#define KDP_MODULE_NAME "rte_kdp" +#define MAX_MBUF_BURST_NUM 32 + +/* Maximum number of ring entries */ +#define KDP_FIFO_COUNT_MAX 1024 +#define KDP_FIFO_SIZE (KDP_FIFO_COUNT_MAX * sizeof(void *) + \ + sizeof(struct rte_kdp_fifo)) + +#define BUFSZ 1024 +struct kdp_request { + struct nlmsghdr nlmsg; + char buf[BUFSZ]; +}; + +static int kdp_fd = -1; +static int kdp_ref_count; + +static const struct rte_memzone * +kdp_memzone_reserve(const char *name, size_t len, int socket_id, + unsigned flags) +{ + const struct rte_memzone *mz = rte_memzone_lookup(name); + + if (mz == NULL) + mz = rte_memzone_reserve(name, len, socket_id, flags); + + return mz; +} + +static int +kdp_slot_init(struct kdp_memzone_slot *slot) +{ +#define OBJNAMSIZ 32 + char obj_name[OBJNAMSIZ]; + const struct rte_memzone *mz; + + /* TX RING */ + snprintf(obj_name, OBJNAMSIZ, "kdp_tx_%d", slot->id); + mz = kdp_memzone_reserve(obj_name, KDP_FIFO_SIZE, SOCKET_ID_ANY, 0); + if (mz == NULL) + goto kdp_fail; + slot->m_tx_q = mz; + + /* RX RING */ + snprintf(obj_name, OBJNAMSIZ, "kdp_rx_%d", slot->id); + mz = kdp_memzone_reserve(obj_name, KDP_FIFO_SIZE, SOCKET_ID_ANY, 0); + if (mz == NULL) + goto kdp_fail; + slot->m_rx_q = mz; + + /* ALLOC RING */ + snprintf(obj_name, OBJNAMSIZ, "kdp_alloc_%d", slot->id); + mz = kdp_memzone_reserve(obj_name, KDP_FIFO_SIZE, SOCKET_ID_ANY, 0); + if (mz == NULL) + goto kdp_fail; + slot->m_alloc_q = mz; + + /* FREE RING */ + snprintf(obj_name, OBJNAMSIZ, "kdp_free_%d", slot->id); + mz = kdp_memzone_reserve(obj_name, KDP_FIFO_SIZE, SOCKET_ID_ANY, 0); + if (mz == NULL) + goto kdp_fail; + slot->m_free_q = mz; + + return 0; + +kdp_fail: + return -1; +} + +static void +kdp_ring_init(struct kdp_data *kdp) +{ + struct kdp_memzone_slot *slot = kdp->slot; + const struct rte_memzone *mz; + + /* TX RING */ + mz = slot->m_tx_q; + kdp->tx_q = mz->addr; + kdp_fifo_init(kdp->tx_q, KDP_FIFO_COUNT_MAX); + + /* RX RING */ + mz = slot->m_rx_q; + kdp->rx_q = mz->addr; + kdp_fifo_init(kdp->rx_q, KDP_FIFO_COUNT_MAX); + + /* ALLOC RING */ + mz = slot->m_alloc_q; + kdp->alloc_q = mz->addr; + kdp_fifo_init(kdp->alloc_q, KDP_FIFO_COUNT_MAX); + + /* FREE RING */ + mz = slot->m_free_q; + kdp->free_q = mz->addr; + kdp_fifo_init(kdp->free_q, KDP_FIFO_COUNT_MAX); +} + +static int +kdp_module_check(void) +{ + int fd; + + fd = open("/sys/module/" KDP_MODULE_NAME "/initstate", O_RDONLY); + if (fd < 0) + return -1; + close(fd); + + return 0; +} + +static int +rtnl_socket_open(void) +{ + struct sockaddr_nl src; + int ret; + + /* Check FD and open */ + if (kdp_fd < 0) { + kdp_fd = socket(AF_NETLINK, SOCK_RAW, NETLINK_ROUTE); + if (kdp_fd < 0) { + RTE_LOG(ERR, KDP, "socket for create failed.\n"); + return -1; + } + + memset(&src, 0, sizeof(struct sockaddr_nl)); + + src.nl_family = AF_NETLINK; + src.nl_pid = getpid(); + + ret = bind(kdp_fd, (struct sockaddr *)&src, + sizeof(struct sockaddr_nl)); + if (ret < 0) { + RTE_LOG(ERR, KDP, "Bind for create failed.\n"); + close(kdp_fd); + kdp_fd = -1; + return -1; + } + } + + kdp_ref_count++; + + return 0; +} + +static void +kdp_ref_put(void) +{ + /* not initialized? */ + if (!kdp_ref_count) + return; + + kdp_ref_count--; + + /* not last one? */ + if (kdp_ref_count) + return; + + if (kdp_fd < 0) + return; + + close(kdp_fd); + kdp_fd = -1; +} + +struct kdp_data * +kdp_init(uint16_t port_id) +{ + struct kdp_memzone_slot *slot = NULL; + struct kdp_data *kdp = NULL; + int ret; + + ret = kdp_module_check(); + if (ret) + return NULL; + + ret = rtnl_socket_open(); + if (ret) + return NULL; + + slot = rte_malloc(NULL, sizeof(struct kdp_memzone_slot), 0); + if (slot == NULL) + goto kdp_fail; + slot->id = port_id; + + kdp = rte_malloc(NULL, sizeof(struct kdp_data), 0); + if (kdp == NULL) + goto kdp_fail; + kdp->slot = slot; + + ret = kdp_slot_init(slot); + if (ret < 0) + goto kdp_fail; + + kdp_ring_init(kdp); + + return kdp; + +kdp_fail: + kdp_ref_put(); + rte_free(slot); + rte_free(kdp); + RTE_LOG(ERR, KDP, "Unable to allocate memory\n"); + return NULL; +} + +static void +kdp_mbufs_allocate(struct kdp_data *kdp) +{ + int i, ret; + struct rte_mbuf *pkts[MAX_MBUF_BURST_NUM]; + + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pool) != + offsetof(struct rte_kdp_mbuf, pool)); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, buf_addr) != + offsetof(struct rte_kdp_mbuf, buf_addr)); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, next) != + offsetof(struct rte_kdp_mbuf, next)); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_off) != + offsetof(struct rte_kdp_mbuf, data_off)); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, data_len) != + offsetof(struct rte_kdp_mbuf, data_len)); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, pkt_len) != + offsetof(struct rte_kdp_mbuf, pkt_len)); + RTE_BUILD_BUG_ON(offsetof(struct rte_mbuf, ol_flags) != + offsetof(struct rte_kdp_mbuf, ol_flags)); + + /* Check if pktmbuf pool has been configured */ + if (kdp->pktmbuf_pool == NULL) { + RTE_LOG(ERR, KDP, "No valid mempool for allocating mbufs\n"); + return; + } + + for (i = 0; i < MAX_MBUF_BURST_NUM; i++) { + pkts[i] = rte_pktmbuf_alloc(kdp->pktmbuf_pool); + if (unlikely(pkts[i] == NULL)) { + /* Out of memory */ + RTE_LOG(ERR, KDP, "Out of memory\n"); + break; + } + } + + /* No pkt mbuf alocated */ + if (i <= 0) + return; + + ret = kdp_fifo_put(kdp->alloc_q, (void **)pkts, i); + + /* Check if any mbufs not put into alloc_q, and then free them */ + if (ret >= 0 && ret < i && ret < MAX_MBUF_BURST_NUM) { + int j; + + for (j = ret; j < i; j++) + rte_pktmbuf_free(pkts[j]); + } +} + +static int +attr_add(struct kdp_request *req, unsigned short type, void *buf, size_t len) +{ + struct rtattr *rta; + int nlmsg_len; + + nlmsg_len = NLMSG_ALIGN(req->nlmsg.nlmsg_len); + rta = (struct rtattr *)((char *)&req->nlmsg + nlmsg_len); + if (nlmsg_len + RTA_LENGTH(len) > sizeof(struct kdp_request)) + return -1; + rta->rta_type = type; + rta->rta_len = RTA_LENGTH(len); + memcpy(RTA_DATA(rta), buf, len); + req->nlmsg.nlmsg_len = nlmsg_len + RTA_LENGTH(len); + + return 0; +} + +static struct +rtattr *attr_nested_add(struct kdp_request *req, unsigned short type) +{ + struct rtattr *rta; + int nlmsg_len; + + nlmsg_len = NLMSG_ALIGN(req->nlmsg.nlmsg_len); + rta = (struct rtattr *)((char *)&req->nlmsg + nlmsg_len); + if (nlmsg_len + RTA_LENGTH(0) > sizeof(struct kdp_request)) + return NULL; + rta->rta_type = type; + rta->rta_len = nlmsg_len; + req->nlmsg.nlmsg_len = nlmsg_len + RTA_LENGTH(0); + + return rta; +} + +static void +attr_nested_end(struct kdp_request *req, struct rtattr *rta) +{ + rta->rta_len = req->nlmsg.nlmsg_len - rta->rta_len; +} + +static int +rtnl_create(struct rte_kdp_device_info *dev_info) +{ + struct kdp_request req; + struct ifinfomsg *info; + struct rtattr *rta1; + struct rtattr *rta2; + char name[RTE_KDP_NAMESIZE]; + char type[RTE_KDP_NAMESIZE]; + struct iovec iov; + struct msghdr msg; + struct sockaddr_nl nladdr; + int ret; + char buf[BUFSZ]; + + memset(&req, 0, sizeof(struct kdp_request)); + + req.nlmsg.nlmsg_len = NLMSG_LENGTH(sizeof(struct ifinfomsg)); + req.nlmsg.nlmsg_flags = NLM_F_REQUEST | NLM_F_CREATE | NLM_F_EXCL; + req.nlmsg.nlmsg_flags |= NLM_F_ACK; + req.nlmsg.nlmsg_type = RTM_NEWLINK; + + info = NLMSG_DATA(&req.nlmsg); + + info->ifi_family = AF_UNSPEC; + info->ifi_index = 0; + + snprintf(name, RTE_KDP_NAMESIZE, "%s", dev_info->name); + ret = attr_add(&req, IFLA_IFNAME, name, strlen(name) + 1); + if (ret < 0) + return -1; + + rta1 = attr_nested_add(&req, IFLA_LINKINFO); + if (rta1 == NULL) + return -1; + + snprintf(type, RTE_KDP_NAMESIZE, KDP_DEVICE); + ret = attr_add(&req, IFLA_INFO_KIND, type, strlen(type) + 1); + if (ret < 0) + return -1; + + rta2 = attr_nested_add(&req, IFLA_INFO_DATA); + if (rta2 == NULL) + return -1; + + ret = attr_add(&req, IFLA_KDP_PORTID, &dev_info->port_id, + sizeof(uint8_t)); + if (ret < 0) + return -1; + + ret = attr_add(&req, IFLA_KDP_DEVINFO, dev_info, + sizeof(struct rte_kdp_device_info)); + if (ret < 0) + return -1; + + attr_nested_end(&req, rta2); + attr_nested_end(&req, rta1); + + memset(&nladdr, 0, sizeof(nladdr)); + nladdr.nl_family = AF_NETLINK; + + iov.iov_base = (void *)&req.nlmsg; + iov.iov_len = req.nlmsg.nlmsg_len; + + memset(&msg, 0, sizeof(struct msghdr)); + msg.msg_name = &nladdr; + msg.msg_namelen = sizeof(nladdr); + msg.msg_iov = &iov; + msg.msg_iovlen = 1; + + ret = sendmsg(kdp_fd, &msg, 0); + if (ret < 0) { + RTE_LOG(ERR, KDP, "Send for create failed %d.\n", errno); + return -1; + } + + memset(buf, 0, sizeof(buf)); + iov.iov_base = buf; + iov.iov_len = sizeof(buf); + + ret = recvmsg(kdp_fd, &msg, 0); + if (ret < 0) { + RTE_LOG(ERR, KDP, "Recv for create failed.\n"); + return -1; + } + + return 0; +} + +int +kdp_start(struct kdp_data *kdp, struct rte_mempool *pktmbuf_pool, + const struct kdp_conf *conf) +{ + struct kdp_memzone_slot *slot = kdp->slot; + struct rte_kdp_device_info dev_info; + char mz_name[RTE_MEMZONE_NAMESIZE]; + const struct rte_memzone *mz; + int ret; + + if (!kdp || !pktmbuf_pool || !conf || !conf->name[0]) + return -1; + + snprintf(kdp->name, RTE_KDP_NAMESIZE, "%s", conf->name); + kdp->pktmbuf_pool = pktmbuf_pool; + kdp->port_id = conf->port_id; + + memset(&dev_info, 0, sizeof(dev_info)); + dev_info.core_id = conf->core_id; + dev_info.force_bind = conf->force_bind; + dev_info.port_id = conf->port_id; + dev_info.mbuf_size = conf->mbuf_size; + snprintf(dev_info.name, RTE_KDP_NAMESIZE, "%s", conf->name); + + dev_info.tx_phys = slot->m_tx_q->phys_addr; + dev_info.rx_phys = slot->m_rx_q->phys_addr; + dev_info.alloc_phys = slot->m_alloc_q->phys_addr; + dev_info.free_phys = slot->m_free_q->phys_addr; + + /* MBUF mempool */ + snprintf(mz_name, sizeof(mz_name), RTE_MEMPOOL_OBJ_NAME, + pktmbuf_pool->name); + mz = rte_memzone_lookup(mz_name); + if (mz == NULL) + goto kdp_fail; + dev_info.mbuf_va = mz->addr; + dev_info.mbuf_phys = mz->phys_addr; + + ret = rtnl_create(&dev_info); + if (ret < 0) + goto kdp_fail; + + kdp->in_use = 1; + + /* Allocate mbufs and then put them into alloc_q */ + kdp_mbufs_allocate(kdp); + + return 0; + +kdp_fail: + return -1; +} + +static void +kdp_mbufs_free(struct kdp_data *kdp) +{ + int i, ret; + struct rte_mbuf *pkts[MAX_MBUF_BURST_NUM]; + + ret = kdp_fifo_get(kdp->free_q, (void **)pkts, MAX_MBUF_BURST_NUM); + if (likely(ret > 0)) { + for (i = 0; i < ret; i++) + rte_pktmbuf_free(pkts[i]); + } +} + +unsigned +kdp_tx_burst(struct kdp_data *kdp, struct rte_mbuf **mbufs, unsigned num) +{ + unsigned ret = kdp_fifo_put(kdp->rx_q, (void **)mbufs, num); + + /* Get mbufs from free_q and then free them */ + kdp_mbufs_free(kdp); + + return ret; +} + +unsigned +kdp_rx_burst(struct kdp_data *kdp, struct rte_mbuf **mbufs, unsigned num) +{ + unsigned ret = kdp_fifo_get(kdp->tx_q, (void **)mbufs, num); + + /* If buffers removed, allocate mbufs and then put them into alloc_q */ + if (ret) + kdp_mbufs_allocate(kdp); + + return ret; +} + +static void +kdp_fifo_free(struct rte_kdp_fifo *fifo) +{ + int ret; + struct rte_mbuf *pkt; + + do { + ret = kdp_fifo_get(fifo, (void **)&pkt, 1); + if (ret) + rte_pktmbuf_free(pkt); + } while (ret); +} + +static int +rtnl_destroy(struct kdp_data *kdp) +{ + struct kdp_request req; + struct ifinfomsg *info; + struct iovec iov; + struct msghdr msg; + struct sockaddr_nl nladdr; + int ret; + + memset(&req, 0, sizeof(struct kdp_request)); + + req.nlmsg.nlmsg_len = NLMSG_LENGTH(sizeof(struct ifinfomsg)); + req.nlmsg.nlmsg_flags = NLM_F_REQUEST; + req.nlmsg.nlmsg_type = RTM_DELLINK; + + info = NLMSG_DATA(&req.nlmsg); + + info->ifi_family = AF_UNSPEC; + info->ifi_index = 0; + + ret = attr_add(&req, IFLA_IFNAME, kdp->name, strlen(kdp->name) + 1); + if (ret < 0) + return -1; + + memset(&nladdr, 0, sizeof(nladdr)); + nladdr.nl_family = AF_NETLINK; + + iov.iov_base = (void *)&req.nlmsg; + iov.iov_len = req.nlmsg.nlmsg_len; + + memset(&msg, 0, sizeof(struct msghdr)); + msg.msg_name = &nladdr; + msg.msg_namelen = sizeof(nladdr); + msg.msg_iov = &iov; + msg.msg_iovlen = 1; + + ret = sendmsg(kdp_fd, &msg, 0); + if (ret < 0) { + RTE_LOG(ERR, KDP, "Send for destroy failed.\n"); + return -1; + } + return 0; +} + +int +kdp_stop(struct kdp_data *kdp) +{ + struct rte_mbuf *pkts[MAX_MBUF_BURST_NUM]; + int ret; + int i; + + if (!kdp || !kdp->in_use) + return -1; + + rtnl_destroy(kdp); + + do { + ret = kdp_fifo_get(kdp->free_q, (void **)pkts, + MAX_MBUF_BURST_NUM); + if (ret > 0) { + for (i = 0; i < ret; i++) + rte_pktmbuf_free(pkts[i]); + } + } while (ret > 0); + + do { + ret = kdp_fifo_get(kdp->alloc_q, (void **)pkts, + MAX_MBUF_BURST_NUM); + if (ret > 0) { + for (i = 0; i < ret; i++) + rte_pktmbuf_free(pkts[i]); + } + } while (ret > 0); + return 0; +} + +void +kdp_close(struct kdp_data *kdp) +{ + /* mbufs in all fifo should be released, except request/response */ + kdp_fifo_free(kdp->tx_q); + kdp_fifo_free(kdp->rx_q); + kdp_fifo_free(kdp->alloc_q); + kdp_fifo_free(kdp->free_q); + + rte_free(kdp->slot); + + /* Memset the KDP struct */ + memset(kdp, 0, sizeof(struct kdp_data)); +} + +void +kdp_uninit(void) +{ + kdp_ref_put(); +} diff --git a/drivers/net/kdp/rte_kdp.h b/drivers/net/kdp/rte_kdp.h new file mode 100644 index 0000000..20ad93d --- /dev/null +++ b/drivers/net/kdp/rte_kdp.h @@ -0,0 +1,116 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#ifndef _RTE_KDP_H_ +#define _RTE_KDP_H_ + +#include +#include + +#include + +#include +#include + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * KDP memzone pool slot + */ +struct kdp_memzone_slot { + uint32_t id; + + /* Memzones */ + const struct rte_memzone *m_tx_q; /**< TX queue */ + const struct rte_memzone *m_rx_q; /**< RX queue */ + const struct rte_memzone *m_alloc_q; /**< Allocated mbufs queue */ + const struct rte_memzone *m_free_q; /**< To be freed mbufs queue */ +}; + +/** + * KDP context + */ +struct kdp_data { + char name[RTE_KDP_NAMESIZE]; /**< KDP interface name */ + struct rte_mempool *pktmbuf_pool; /**< pkt mbuf mempool */ + struct kdp_memzone_slot *slot; + uint16_t port_id; /**< Group ID of KDP devices */ + + struct rte_kdp_fifo *tx_q; /**< TX queue */ + struct rte_kdp_fifo *rx_q; /**< RX queue */ + struct rte_kdp_fifo *alloc_q; /**< Allocated mbufs queue */ + struct rte_kdp_fifo *free_q; /**< To be freed mbufs queue */ + + uint8_t in_use; /**< kdp in use */ +}; + +struct kdp_tap_data { + char name[RTE_KDP_NAMESIZE]; + int tap_fd; +}; + +/** + * Structure for configuring KDP device. + */ +struct kdp_conf { + char name[RTE_KDP_NAMESIZE]; + uint32_t core_id; /* Core ID to bind kernel thread on */ + uint16_t port_id; + unsigned mbuf_size; + + uint8_t force_bind; /* Flag to bind kernel thread */ +}; + +struct kdp_data *kdp_init(uint16_t port_id); +int kdp_start(struct kdp_data *kdp, struct rte_mempool *pktmbuf_pool, + const struct kdp_conf *conf); +unsigned kdp_rx_burst(struct kdp_data *kdp, + struct rte_mbuf **mbufs, unsigned num); +unsigned kdp_tx_burst(struct kdp_data *kdp, + struct rte_mbuf **mbufs, unsigned num); +int kdp_stop(struct kdp_data *kdp); +void kdp_close(struct kdp_data *kdp); +void kdp_uninit(void); + +struct kdp_tap_data *kdp_tap_init(uint16_t port_id); +void kdp_tap_close(struct kdp_tap_data *kdp_tap); + +#ifdef __cplusplus +} +#endif + +#endif /* _RTE_KDP_H_ */ diff --git a/drivers/net/kdp/rte_kdp_fifo.h b/drivers/net/kdp/rte_kdp_fifo.h new file mode 100644 index 0000000..1a7e063 --- /dev/null +++ b/drivers/net/kdp/rte_kdp_fifo.h @@ -0,0 +1,91 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +/** + * Initializes the kdp fifo structure + */ +static void +kdp_fifo_init(struct rte_kdp_fifo *fifo, unsigned size) +{ + /* Ensure size is power of 2 */ + if (size & (size - 1)) + rte_panic("KDP fifo size must be power of 2\n"); + + fifo->write = 0; + fifo->read = 0; + fifo->len = size; + fifo->elem_size = sizeof(void *); +} + +/** + * Adds num elements into the fifo. Return the number actually written + */ +static inline unsigned +kdp_fifo_put(struct rte_kdp_fifo *fifo, void **data, unsigned num) +{ + unsigned i = 0; + unsigned fifo_write = fifo->write; + unsigned fifo_read = fifo->read; + unsigned new_write = fifo_write; + + for (i = 0; i < num; i++) { + new_write = (new_write + 1) & (fifo->len - 1); + + if (new_write == fifo_read) + break; + fifo->buffer[fifo_write] = data[i]; + fifo_write = new_write; + } + fifo->write = fifo_write; + return i; +} + +/** + * Get up to num elements from the fifo. Return the number actully read + */ +static inline unsigned +kdp_fifo_get(struct rte_kdp_fifo *fifo, void **data, unsigned num) +{ + unsigned i = 0; + unsigned new_read = fifo->read; + unsigned fifo_write = fifo->write; + for (i = 0; i < num; i++) { + if (new_read == fifo_write) + break; + + data[i] = fifo->buffer[new_read]; + new_read = (new_read + 1) & (fifo->len - 1); + } + fifo->read = new_read; + return i; +} diff --git a/drivers/net/kdp/rte_kdp_tap.c b/drivers/net/kdp/rte_kdp_tap.c new file mode 100644 index 0000000..12f3ad2 --- /dev/null +++ b/drivers/net/kdp/rte_kdp_tap.c @@ -0,0 +1,101 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include + +#include +#include +#include + +#include "rte_kdp.h" + +static int +tap_create(char *name) +{ + struct ifreq ifr; + int fd, ret; + + fd = open("/dev/net/tun", O_RDWR); + if (fd < 0) + return fd; + + memset(&ifr, 0, sizeof(ifr)); + + /* TAP device without packet information */ + ifr.ifr_flags = IFF_TAP | IFF_NO_PI; + + if (name && *name) + snprintf(ifr.ifr_name, IFNAMSIZ, "%s", name); + + ret = ioctl(fd, TUNSETIFF, (void *)&ifr); + if (ret < 0) { + close(fd); + return ret; + } + + if (name) + snprintf(name, IFNAMSIZ, "%s", ifr.ifr_name); + + return fd; +} + +struct kdp_tap_data * +kdp_tap_init(uint16_t port_id) +{ + struct kdp_tap_data *kdp_tap = NULL; + int flags; + + kdp_tap = rte_malloc(NULL, sizeof(struct kdp_tap_data), 0); + if (kdp_tap == NULL) + goto error; + + snprintf(kdp_tap->name, IFNAMSIZ, "tap_kdp%u", port_id); + kdp_tap->tap_fd = tap_create(kdp_tap->name); + if (kdp_tap->tap_fd < 0) + goto error; + + flags = fcntl(kdp_tap->tap_fd, F_GETFL, 0); + fcntl(kdp_tap->tap_fd, F_SETFL, flags | O_NONBLOCK); + + return kdp_tap; + +error: + rte_free(kdp_tap); + return NULL; +} + +void +kdp_tap_close(struct kdp_tap_data *kdp_tap) +{ + close(kdp_tap->tap_fd); +} diff --git a/drivers/net/kdp/rte_pmd_kdp_version.map b/drivers/net/kdp/rte_pmd_kdp_version.map new file mode 100644 index 0000000..0812bb1 --- /dev/null +++ b/drivers/net/kdp/rte_pmd_kdp_version.map @@ -0,0 +1,4 @@ +DPDK_2.3 { + + local: *; +}; diff --git a/lib/librte_eal/common/include/rte_log.h b/lib/librte_eal/common/include/rte_log.h index 2e47e7f..5a0048b 100644 --- a/lib/librte_eal/common/include/rte_log.h +++ b/lib/librte_eal/common/include/rte_log.h @@ -1,7 +1,7 @@ /*- * BSD LICENSE * - * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * Copyright(c) 2010-2016 Intel Corporation. All rights reserved. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -79,6 +79,7 @@ extern struct rte_logs rte_logs; #define RTE_LOGTYPE_PIPELINE 0x00008000 /**< Log related to pipeline. */ #define RTE_LOGTYPE_MBUF 0x00010000 /**< Log related to mbuf. */ #define RTE_LOGTYPE_CRYPTODEV 0x00020000 /**< Log related to cryptodev. */ +#define RTE_LOGTYPE_KDP 0x00080000 /**< Log related to KDP. */ /* these log types can be used in an application */ #define RTE_LOGTYPE_USER1 0x01000000 /**< User-defined log type 1. */ diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 8ecab41..eb18972 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -1,6 +1,6 @@ # BSD LICENSE # -# Copyright(c) 2010-2015 Intel Corporation. All rights reserved. +# Copyright(c) 2010-2016 Intel Corporation. All rights reserved. # Copyright(c) 2014-2015 6WIND S.A. # All rights reserved. # @@ -154,6 +154,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += -lrte_pmd_pcap _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AF_PACKET) += -lrte_pmd_af_packet _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += -lrte_pmd_null _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_QAT) += -lrte_pmd_qat +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KDP) += -lrte_pmd_kdp # AESNI MULTI BUFFER is dependent on the IPSec_MB library _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_AESNI_MB) += -lrte_pmd_aesni_mb -- 2.5.0