* [dpdk-dev] [PATCH] ntnic: add PMD driver @ 2016-08-26 13:44 Finn Christensen 2016-08-26 14:44 ` Thomas Monjalon ` (2 more replies) 0 siblings, 3 replies; 22+ messages in thread From: Finn Christensen @ 2016-08-26 13:44 UTC (permalink / raw) To: dev; +Cc: Finn Christensen This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. This patch adds support for Napatech NICs to DPDK. This is the inital implementation. Signed-off-by: Finn Christensen <fc@napatech.com> --- MAINTAINERS | 5 + config/common_base | 6 + doc/guides/nics/features/ntnic.ini | 15 + doc/guides/nics/index.rst | 1 + doc/guides/nics/ntnic.rst | 138 +++++ drivers/net/Makefile | 1 + drivers/net/ntnic/Makefile | 65 ++ drivers/net/ntnic/rte_eth_ntnic.c | 912 ++++++++++++++++++++++++++++ drivers/net/ntnic/rte_pmd_ntnic_version.map | 4 + mk/rte.app.mk | 7 + 10 files changed, 1154 insertions(+) create mode 100644 doc/guides/nics/features/ntnic.ini create mode 100644 doc/guides/nics/ntnic.rst create mode 100644 drivers/net/ntnic/Makefile create mode 100644 drivers/net/ntnic/rte_eth_ntnic.c create mode 100644 drivers/net/ntnic/rte_pmd_ntnic_version.map diff --git a/MAINTAINERS b/MAINTAINERS index bc9aa02..d3e5f56 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -356,6 +356,11 @@ M: Sony Chacko <sony.chacko@qlogic.com> F: drivers/net/qede/ F: doc/guides/nics/qede.rst +Napatech ntnic +M: Finn Christensen <fc@napatech.com> +F: drivers/net/ntnic/ +F: doc/guides/nics/ntnic.rst + RedHat virtio M: Huawei Xie <huawei.xie@intel.com> M: Yuanhan Liu <yuanhan.liu@linux.intel.com> diff --git a/config/common_base b/config/common_base index 7830535..4a3b2b8 100644 --- a/config/common_base +++ b/config/common_base @@ -309,6 +309,12 @@ CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16 CONFIG_RTE_LIBRTE_PMD_PCAP=n # +# Compile software PMD backed by NTNIC files +# +CONFIG_RTE_LIBRTE_PMD_NTNIC=n + + +# # Compile link bonding PMD library # CONFIG_RTE_LIBRTE_PMD_BOND=y diff --git a/doc/guides/nics/features/ntnic.ini b/doc/guides/nics/features/ntnic.ini new file mode 100644 index 0000000..b6cfc5a --- /dev/null +++ b/doc/guides/nics/features/ntnic.ini @@ -0,0 +1,15 @@ +; +; Supported features of the 'ntnic' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +Queue start/stop = Y +Promiscuous mode = Y +Jumbo frame = Y +Basic stats = Y +Multiprocess aware = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 92d56a5..5c4205c 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -52,6 +52,7 @@ Network Interface Controller Drivers qede szedata2 thunderx + ntnic virtio vhost vmxnet3 diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst new file mode 100644 index 0000000..3f2497c --- /dev/null +++ b/doc/guides/nics/ntnic.rst @@ -0,0 +1,138 @@ +.. BSD LICENSE + Copyright (c) 2016 Napatech A/S + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Napatech nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +NTNIC Poll Mode Driver +====================== + +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements support +for **Napatech NIC** 40/50 Gbps adapters. +This PMD is implemented as a pure software virtual device and must be created +by using the EAL --vdev=parameter (parameters are explained i detail later). +It runs on top of the Napatech NIC Driver Suite that must be installed and +started. + +Supported Features +------------------ + +- RSS (Receive Side Scaling) +- TSS (Transmit Side Scaling) +- Promiscuous mode +- Basic statistics + + +Prerequisites +------------- + +Requires Napatech NIC and Napatech NIC Driver Suite installed and +running in version **0.3.0** or higher. +This includes external libraries and kernel driver for resources +allocations and initialization. + +Pre-Installation Configuration +------------------------------ + +Environment variables +~~~~~~~~~~~~~~~~~~~~~ + +In order to compile the Napatech NIC PMD, user must: + +* Export the environmental variable NAPATECH3_PATH with the path where + the Napatech Driver suite was installed (default location is + /opt/napatech3). + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +- ``CONFIG_RTE_LIBRTE_PMD_NTNIC`` (default **n**) + +Using the NTNIC PMD from a DPDK application using EAL vdev parameter +-------------------------------------------------------------------- + +Napatech NIC PMD VDEV parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Napatech NIC PMD vdev has the following command line parameters to enable +its features. + +* port + Configures which NT port the vdev should use. + + --vdev eth_ntnic0,port=4,rxqs=1,txqs=1 + + This will create a DPDK port0 using NT port 4 + +* rxqs + Control how many receive queues a vdev should have. The traffic from a vdev + will be load balanced to this amount of queues and each queue can be handled + by its own lcore. + Note: The ntservice.ini HostBuffersRx must be configured to the sum of all + receive queues across all vdevs. + + --vdev eth_ntnic0,port=0,rxqs=8,txqs=1 + + Traffic from NT port 0 will be split across + 8 queues meaning that 8 lcores can be + opened on DPDK port 0 and each will be + their individual traffic. + +* hash + Control which load balancing scheme should be used when running with multiple + receive queues. + The possible values are: + 1 = HashRoundRobin + 2 = Hash2TupleSorted + 3 = Hash5TupleSorted + + --vdev eth_ntnic0,port=0,rxqs=4,hash=3,txqs=1 + + Create 4 queues from NT port 0 and + load balance the traffic using the + 5-tuple sorted algorithm. Each of the + 4 queues can be opened via DPDK port 0 + +* txqs + Control how many transmit queues a vdev should have. Each queue can be used + by a lcore and each queue is independant for other queues. + Note: The ntservice.ini HostBuffersTx must be configured to the sum of all + transmit queues across all vdevs. + + --vdev eth_ntnic0,port=0,txqs=32,rxqs=1 + + Enable 32 transmit queues on NT port 0 + + +Changes to Napatech driver configuration file ntservice.ini +----------------------------------------------------------- +Depending on the number of queues that is created, more hostbuffers may be +needed pre-allocated by the Napatech NIC Driver. This is controlled in the +ntservice.ini file (default locations is /opt/napatech3/config). +The Napatech NIC Driver must be re-started when this configuration file is +changed. + diff --git a/drivers/net/Makefile b/drivers/net/Makefile index bc93230..aff29e9 100644 --- a/drivers/net/Makefile +++ b/drivers/net/Makefile @@ -55,6 +55,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3 DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt +DIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += ntnic ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) DIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += vhost diff --git a/drivers/net/ntnic/Makefile b/drivers/net/ntnic/Makefile new file mode 100644 index 0000000..9467464 --- /dev/null +++ b/drivers/net/ntnic/Makefile @@ -0,0 +1,65 @@ +# BSD LICENSE +# +# Copyright(c) 2016 Napatech A/S. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Napatech nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_ntnic.a + +CFLAGS += -O3 -g +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -I/opt/napatech3/include +LDLIBS += -L/opt/napatech3/lib -lntapi + +EXPORT_MAP := rte_pmd_ntnic_version.map + +LIBABIVER := 1 + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += rte_eth_ntnic.c + +# +# Export include files +# +SYMLINK-y-include += + +# this lib depends upon: +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_eal +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mbuf +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mempool +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_ether +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_kvargs + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/ntnic/rte_eth_ntnic.c b/drivers/net/ntnic/rte_eth_ntnic.c new file mode 100644 index 0000000..8094a1d --- /dev/null +++ b/drivers/net/ntnic/rte_eth_ntnic.c @@ -0,0 +1,912 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 Napatech A/S. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Napatech nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <time.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_memcpy.h> +#include <rte_string_fns.h> +#include <rte_cycles.h> +#include <rte_kvargs.h> +#include <rte_dev.h> +#include <net/if.h> +#include <nt.h> + +#define ETH_NTNIC_PORT_ARG "port" +#define ETH_NTNIC_PORTEND_ARG "portend" +#define ETH_NTNIC_RXQUEUES_ARG "rxqs" +#define ETH_NTNIC_TXQUEUES_ARG "txqs" +#define ETH_NTNIC_STREAMID_ARG "streamid" +#define ETH_NTNIC_HASH_ARG "hash" + +#define HW_MAX_PKT_LEN 10000 +#define HW_MTU (HW_MAX_PKT_LEN - ETHER_HDR_LEN - ETHER_CRC_LEN) +/* + * hash = <value> (1..3) + * 1 - RoundRobin + * 2 - 2tupleSorted + * 3 - 5tupleSorted + */ + + +#define MAX_RX_QUEUES 64 +#define MAX_TX_QUEUES 64 +#define MAX_NTNIC_PORTS 32 + +static char errorBuffer[1024]; + +struct array_s { + uint32_t value[MAX_RX_QUEUES]; + int count; +}; + +static volatile uint16_t port_locks[MAX_NTNIC_PORTS]; + +struct ntnic_rx_queue { + NtNetStreamRx_t pNetRx[MAX_RX_QUEUES]; /* NT Rx streams */ + int curNetRx_idx; /* Current Rx stream index */ + struct rte_mempool *mb_pool; /* mbuf memory pool */ + uint16_t buf_size; /* Size of data area in mbuf */ + NtNetBuf_t pSeg; /* The current NT data segment we are working with */ + struct NtNetBuf_s pkt; /* The current packet */ + volatile unsigned long rx_pkts; /* Rx packet statistics */ + volatile unsigned long err_pkts; /* Rx error packet statistics */ + struct array_s astreamids; /* Array of NT streamids to read data from */ + int enabled; /* Enabling/disabling of this queue */ +}; + +struct ntnic_tx_queue { + NtNetStreamTx_t pNetTx; /* NT Tx stream */ + volatile unsigned long tx_pkts; /* Tx packet statistics */ + volatile unsigned long err_pkts; /* Tx error packet statistics */ + volatile uint16_t *plock; /* Per port transmit atomic lock */ + uint32_t port; /* Tx port for this queue */ + int enabled; /* Enabling/disabling of this queue */ + int fcs_add; /* If port HW needs added room for FCS */ +}; + +struct pmd_internals { + struct ntnic_rx_queue rxq[MAX_RX_QUEUES]; /* Array of Rx queues configured */ + struct ntnic_tx_queue txq[MAX_TX_QUEUES]; /* Array of Tx queues configured */ + int if_index; /* Interface index always 0 - no bonding */ + unsigned int nb_rx_queues; /* Number of Rx queues configured */ + unsigned int nb_tx_queues; /* Number of Tx queues configured */ + int MAC_50G; /* True if 50G ports */ +}; + +static const char *valid_arguments[] = { + ETH_NTNIC_PORT_ARG, + ETH_NTNIC_PORTEND_ARG, + ETH_NTNIC_RXQUEUES_ARG, + ETH_NTNIC_TXQUEUES_ARG, + ETH_NTNIC_STREAMID_ARG, + ETH_NTNIC_HASH_ARG, + NULL +}; + +static struct ether_addr eth_addr[MAX_NTNIC_PORTS]; +static const char *drivername = "NTNIC PMD"; + +static int +eth_ntnic_rx_jumbo(struct rte_mempool *mb_pool, + struct rte_mbuf *mbuf, + const u_char *data, + uint16_t data_len) +{ + struct rte_mbuf *m = mbuf; + + /* Copy the first segment. */ + uint16_t len = rte_pktmbuf_tailroom(mbuf); + + rte_memcpy(rte_pktmbuf_append(mbuf, len), data, len); + data_len -= len; + data += len; + + while (data_len > 0) { + /* Allocate next mbuf and point to that. */ + m->next = rte_pktmbuf_alloc(mb_pool); + + if (unlikely(!m->next)) + return -1; + + m = m->next; + + /* Headroom is not needed in chained mbufs. */ + rte_pktmbuf_prepend(m, rte_pktmbuf_headroom(m)); + m->pkt_len = 0; + m->data_len = 0; + + /* Copy next segment. */ + len = RTE_MIN(rte_pktmbuf_tailroom(m), data_len); + rte_memcpy(rte_pktmbuf_append(m, len), data, len); + + mbuf->nb_segs++; + data_len -= len; + data += len; + } + + return mbuf->nb_segs; +} + + + +static uint16_t +eth_ntnic_rx(void *queue, + struct rte_mbuf **bufs, + uint16_t nb_pkts) +{ + unsigned i; + struct rte_mbuf *mbuf; + struct ntnic_rx_queue *rx_q = queue; + uint16_t num_rx = 0; + uint16_t data_len; + + if (unlikely(rx_q->pNetRx[rx_q->curNetRx_idx] == NULL || nb_pkts == 0)) + return 0; + + /* Do we have any data segment */ + if (rx_q->pSeg == NULL) { + /* Next stream - if multiple streams are combined */ + rx_q->curNetRx_idx = (rx_q->curNetRx_idx < (rx_q->astreamids.count - 1)) ? rx_q->curNetRx_idx + 1 : 0; + + if (NT_NetRxGet(rx_q->pNetRx[rx_q->curNetRx_idx], &rx_q->pSeg, 0) != NT_SUCCESS) { + if (rx_q->pSeg != NULL) { + NT_NetRxRelease(rx_q->pNetRx[rx_q->curNetRx_idx], rx_q->pSeg); + rx_q->pSeg = NULL; + } + return 0; + + } + if (NT_NET_GET_SEGMENT_LENGTH(rx_q->pSeg)) { + /* Build a packet structure */ + _nt_net_build_pkt_netbuf(rx_q->pSeg, &rx_q->pkt); + } else { + NT_NetRxRelease(rx_q->pNetRx[rx_q->curNetRx_idx], rx_q->pSeg); + rx_q->pSeg = NULL; + return 0; + } + } + + if (rte_mempool_get_bulk(rx_q->mb_pool, (void **)bufs, nb_pkts) != 0) + return 0; + + for (i = 0; i < nb_pkts; i++) { + mbuf = bufs[i]; + rte_mbuf_refcnt_set(mbuf, 1); + rte_pktmbuf_reset(mbuf); + + /* HW slicing not supported */ + data_len = (uint16_t)NT_NET_GET_PKT_WIRE_LENGTH((&rx_q->pkt)) - 4; + if (data_len <= rx_q->buf_size) { + /* Packet will fit in the mbuf, go ahead and copy */ + mbuf->pkt_len = mbuf->data_len = data_len; + rte_memcpy((u_char *)mbuf->buf_addr + RTE_PKTMBUF_HEADROOM, NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), mbuf->data_len); + } else { + /* Try read jumbo frame into multi mbufs. */ + if (unlikely(eth_ntnic_rx_jumbo(rx_q->mb_pool, + mbuf, + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), + data_len) == -1)) + break; + } + + mbuf->port = NT_NET_GET_PKT_RXPORT((&rx_q->pkt)); + num_rx++; + + /* Get the next packet if any */ + if (_nt_net_get_next_packet(rx_q->pSeg, + NT_NET_GET_SEGMENT_LENGTH(rx_q->pSeg), + &rx_q->pkt) == 0 ) { + NT_NetRxRelease(rx_q->pNetRx[rx_q->curNetRx_idx], rx_q->pSeg); + rx_q->pSeg = NULL; + break; + } + + rx_q->rx_pkts++; + } + + if (num_rx < nb_pkts) { + rte_mempool_put_bulk(rx_q->mb_pool, (void * const *)(bufs + num_rx), nb_pkts-num_rx); + } + return num_rx; +} + + + + + +/* + * Callback to handle sending packets through a NT NIC. + */ +static uint16_t +eth_ntnic_tx_ringbuffer(void *queue, + struct rte_mbuf **bufs, + uint16_t nb_pkts) +{ + unsigned i; + struct ntnic_tx_queue *tx_q = queue; + int retval; + uint16_t old; + uint16_t new_flag = 0; + + + if (unlikely(tx_q == NULL || tx_q->pNetTx == NULL || nb_pkts == 0)) { + return 0; + } + + do { + old = 0; + new_flag = 1; + retval = rte_atomic16_cmpset(tx_q->plock, old, new_flag); + } while (unlikely(retval == 0)); + + for (i = 0; i < nb_pkts; i++) { + /* Not allowed to TX less than 64 byte packets */ + if (bufs[i]->pkt_len < 60) { + bufs[i]->pkt_len = 60; + } + /* transmit a single packet */ + NT_NetTxRingbufferTransmitPacket(tx_q->pNetTx, rte_pktmbuf_mtod(bufs[i], uint8_t *), bufs[i]->pkt_len + 4 - tx_q->fcs_add); + rte_pktmbuf_free(bufs[i]); + } + tx_q->tx_pkts += nb_pkts; + + *tx_q->plock = 0; + return nb_pkts; +} + + + +static int +eth_dev_start(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = internals->rxq; + struct ntnic_tx_queue *tx_q = internals->txq; + uint queue; + int status, idx; + + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + + if ((internals->nb_tx_queues | internals->nb_rx_queues) == 0) { + return -1; + } + + for (queue = 0; queue < internals->nb_rx_queues; queue++) { + if (rx_q[queue].enabled) { + char str[128], val[10]; + str[0] = 0; + /* build list of streamids to log */ + for (idx = 0; idx < rx_q[queue].astreamids.count; idx++) { + if (idx) sprintf(val,",%d",rx_q[queue].astreamids.value[idx]); + else sprintf(val, "%i",rx_q[queue].astreamids.value[idx]); + strcat(str, val); + } + + for (idx = 0; idx < rx_q[queue].astreamids.count; idx++) { + if ((status = NT_NetRxOpen(&rx_q[queue].pNetRx[idx], "DPDK", NT_NET_INTERFACE_SEGMENT, + rx_q[queue].astreamids.value[idx], -1)) != NT_SUCCESS) { + /* try packet interface instead */ + NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); + RTE_LOG(ERR, PMD, "NT_NetRxOpen() failed: %s\n", errorBuffer); + return -1; + } + } + } + } + + for (queue = 0; queue < internals->nb_tx_queues; queue++) { + if (tx_q[queue].enabled) { + RTE_LOG(INFO, PMD, "NTNIC: NT_NetTxOpen(%d)\n", tx_q[queue].port); + if ((status = NT_NetTxOpen(&tx_q[queue].pNetTx, "DPDK", 1 << tx_q[queue].port, + dev->pci_dev->numa_node, 0)) != NT_SUCCESS) { + NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); + RTE_LOG(INFO, PMD, "NT_NetTxOpen(0x%X, %d, 0) failed: %s\n", 1 << tx_q[queue].port, + dev->pci_dev->numa_node, errorBuffer); + return -1; + } + + /* Initialize and associate RTD Tx Ring buffer to Tx handler */ + if ((status = NT_NetTxRingbufferInit(tx_q[queue].pNetTx, tx_q[queue].port)) != NT_SUCCESS) { + NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)); + RTE_LOG(ERR, PMD, "NT_NetRxOpen() failed: %s\n", errorBuffer); + return -1; + } + } + tx_q[queue].plock = &port_locks[tx_q[queue].port]; + tx_q[queue].fcs_add = (internals->MAC_50G)?4:0; + } + + dev->data->dev_link.link_status = 1; + return 0; +} + +/* + * This function gets called when the current port gets stopped. + * Is the only place for us to close all the tx streams dumpers. + * If not called the dumpers will be flushed within each tx burst. + */ +static void +eth_dev_stop(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = internals->rxq; + struct ntnic_tx_queue *tx_q = internals->txq; + uint queue; + int idx; + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + + for (queue = 0; queue < internals->nb_rx_queues; queue++) { + if (rx_q[queue].pSeg) { + NT_NetRxRelease(rx_q[queue].pNetRx[rx_q->curNetRx_idx], rx_q[queue].pSeg); + rx_q[queue].pSeg = NULL; + } + for (idx = 0; idx < rx_q[queue].astreamids.count; idx++) { + if (rx_q[queue].pNetRx[idx]) { + (void)NT_NetRxClose(rx_q[queue].pNetRx[idx]); + } + } + } + for (queue = 0; queue < internals->nb_tx_queues; queue++) { + if (tx_q[queue].pNetTx) { + NT_NetTxRingbufferDone(tx_q[queue].pNetTx); + (void)NT_NetTxClose(tx_q[queue].pNetTx); + } + } + dev->data->dev_link.link_status = 0; +} + +static int +eth_dev_configure(struct rte_eth_dev *dev __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + return 0; +} + +static void +eth_dev_info(struct rte_eth_dev *dev, + struct rte_eth_dev_info *dev_info) +{ + struct pmd_internals *internals = dev->data->dev_private; + dev_info->if_index = internals->if_index; + dev_info->driver_name = drivername; + dev_info->max_mac_addrs = 1; + dev_info->max_rx_pktlen = HW_MTU; + dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; + dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; + dev_info->min_rx_bufsize = 0; + dev_info->pci_dev = NULL; +} + +static void +eth_stats_get(struct rte_eth_dev *dev, + struct rte_eth_stats *igb_stats) +{ + unsigned i; + unsigned long rx_total = 0; + unsigned long tx_total = 0; + unsigned long tx_err_total = 0; + const struct pmd_internals *internal = dev->data->dev_private; + + memset(igb_stats, 0, sizeof(*igb_stats)); + for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal->nb_rx_queues; i++) { + igb_stats->q_ipackets[i] = internal->rxq[i].rx_pkts; + rx_total += igb_stats->q_ipackets[i]; + } + for (i = 0; i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal->nb_tx_queues; i++) { + igb_stats->q_opackets[i] = internal->txq[i].tx_pkts; + igb_stats->q_errors[i] = internal->txq[i].err_pkts; + tx_total += igb_stats->q_opackets[i]; + tx_err_total += igb_stats->q_errors[i]; + } + + igb_stats->ipackets = rx_total; + igb_stats->opackets = tx_total; + igb_stats->oerrors = tx_err_total; +} + +static void +eth_stats_reset(struct rte_eth_dev *dev) +{ + unsigned i; + struct pmd_internals *internal = dev->data->dev_private; + + for (i = 0; i < internal->nb_rx_queues; i++) { + internal->rxq[i].rx_pkts = 0; + } + for (i = 0; i < internal->nb_tx_queues; i++) { + internal->txq[i].tx_pkts = 0; + internal->txq[i].err_pkts = 0; + } +} + +static void +eth_dev_close(struct rte_eth_dev *dev __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); +} + +static void +eth_queue_release(void *q __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); +} + +static int +eth_link_update(struct rte_eth_dev *dev __rte_unused, + int wait_to_complete __rte_unused) +{ + return 0; +} + +static int +eth_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t rx_queue_id, + uint16_t nb_rx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, + struct rte_mempool *mb_pool) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = &internals->rxq[rx_queue_id]; + + RTE_LOG(INFO, PMD, "NTNIC RX queue setup\n"); + rx_q->mb_pool = mb_pool; + dev->data->rx_queues[rx_queue_id] = rx_q; + + mbp_priv = rte_mempool_get_priv(rx_q->mb_pool); + rx_q->buf_size = (uint16_t) (mbp_priv->mbuf_data_room_size - RTE_PKTMBUF_HEADROOM); + rx_q->enabled = 1; + return 0; +} + +static int +eth_tx_queue_setup(struct rte_eth_dev *dev __rte_unused, + uint16_t tx_queue_id __rte_unused, + uint16_t nb_tx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ + struct pmd_internals *internals = dev->data->dev_private; + RTE_LOG(INFO, PMD, "NTNIC TX queue setup\n"); + dev->data->tx_queues[tx_queue_id] = &internals->txq[tx_queue_id]; + internals->txq[tx_queue_id].enabled = 1; + return 0; +} + +static int _dev_set_mtu(struct rte_eth_dev *dev __rte_unused, uint16_t mtu) +{ + if (mtu < 46 || mtu > HW_MTU) + return -EINVAL; + + return 0; +} + +static struct eth_dev_ops ops = { + .dev_start = eth_dev_start, + .dev_stop = eth_dev_stop, + .dev_close = eth_dev_close, + .mtu_set = _dev_set_mtu, + .dev_configure = eth_dev_configure, + .dev_infos_get = eth_dev_info, + .rx_queue_setup = eth_rx_queue_setup, + .tx_queue_setup = eth_tx_queue_setup, + .rx_queue_release = eth_queue_release, + .tx_queue_release = eth_queue_release, + .link_update = eth_link_update, + .stats_get = eth_stats_get, + .stats_reset = eth_stats_reset, +}; + +static int +rte_pmd_init_internals(const char *name, + const unsigned nb_rx_queues, + const unsigned nb_tx_queues, + const unsigned numa_node, + struct array_s *pastreamids, + const uint32_t port, + struct rte_eth_dev **eth_dev) +{ + struct pmd_internals *internals = NULL; + struct rte_eth_dev_data *data = NULL; + struct rte_pci_device *pci_dev = NULL; + uint i, status; + char errBuf[NT_ERRBUF_SIZE]; + NtInfoStream_t hInfo; + NtInfo_t infoPort; + struct rte_eth_link pmd_link; + assert(nb_rx_queues < MAX_RX_QUEUES); + assert(nb_tx_queues < MAX_TX_QUEUES); + assert(port < MAX_NTNIC_PORTS); + + RTE_LOG(INFO, PMD, + "Creating ntnic-backend ethdev on numa socket %u\n", numa_node); + + /* now do all data allocation - for eth_dev structure, dummy pci driver + * and internal (private) data + */ + data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node); + if (data == NULL) + goto error; + + pci_dev = rte_zmalloc_socket(name, sizeof(*pci_dev), 0, numa_node); + if (pci_dev == NULL) + goto error; + + internals = rte_zmalloc_socket(name, sizeof(*internals), 0, numa_node); + if (internals == NULL) + goto error; + + /* reserve an ethdev entry */ + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); + if (*eth_dev == NULL) + goto error; + + /* Open the information stream */ + if ((status = NT_InfoOpen(&hInfo, "DPDK Info stream")) != NT_SUCCESS) { + NT_ExplainError(status, errBuf, sizeof(errBuf)); + RTE_LOG(ERR, PMD, ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", status, errBuf); + return status; + } + + /* Find local port offset */ + infoPort.cmd = NT_INFO_CMD_READ_PORT_V7; + infoPort.u.port_v7.portNo = (uint8_t)(port); + if ((status = NT_InfoRead(hInfo, &infoPort)) != 0) { + NT_ExplainError(status, errBuf, sizeof(errBuf)); + RTE_LOG(ERR, PMD, "ERROR: NT_InfoRead failed. Code 0x%x = %s\n", status, errBuf); + return status; + } + + /* check for 50G MAC */ + if (infoPort.u.port_v7.data.adapterInfo.fpgaid.s.product == 9506 || + infoPort.u.port_v7.data.adapterInfo.fpgaid.s.product == 9509) { + internals->MAC_50G = 1; + } else { + internals->MAC_50G = 0; + /* Check for valid FPGAs (NFV NICs) */ + if (infoPort.u.port_v7.data.adapterInfo.fpgaid.s.product != 9507 && + infoPort.u.port_v7.data.adapterInfo.fpgaid.s.product != 9510) { + RTE_LOG(ERR, PMD, "ERROR: NTNIC PMD does not support the NT adapter - port %i\n", port); + return -1; + } + } + + internals->nb_rx_queues = nb_rx_queues; + internals->nb_tx_queues = nb_tx_queues; + for (i=0; i < nb_rx_queues; i++) { + int idx; + for (idx = 0; idx < pastreamids->count; idx++) { + internals->rxq[i].astreamids.value[idx] = pastreamids->value[idx] + i; + } + + internals->rxq[i].astreamids.count = pastreamids->count; + + internals->rxq[i].pSeg = NULL; + internals->rxq[i].enabled = 0; + } + + for (i = 0; i < nb_tx_queues; i++) { + if (port < infoPort.u.port_v7.data.adapterInfo.portOffset) { + RTE_LOG(ERR, PMD, "Error wrong port specified\n"); + return -1; + } + internals->txq[i].port = port; + internals->txq[i].enabled = 0; + } + + switch (infoPort.u.port_v7.data.speed) { + case NT_LINK_SPEED_UNKNOWN: + pmd_link.link_speed = ETH_SPEED_NUM_1G; + break; + case NT_LINK_SPEED_10M: + pmd_link.link_speed = ETH_SPEED_NUM_10M; + break; + case NT_LINK_SPEED_100M: + pmd_link.link_speed = ETH_SPEED_NUM_100M; + break; + case NT_LINK_SPEED_1G: + pmd_link.link_speed = ETH_SPEED_NUM_1G; + break; + case NT_LINK_SPEED_10G: + pmd_link.link_speed = ETH_SPEED_NUM_10G; + break; + case NT_LINK_SPEED_40G: + pmd_link.link_speed = ETH_SPEED_NUM_40G; + break; + case NT_LINK_SPEED_50G: + pmd_link.link_speed = ETH_SPEED_NUM_50G; + break; + case NT_LINK_SPEED_100G: + pmd_link.link_speed = ETH_SPEED_NUM_100G; + break; + } + + memcpy(ð_addr[port].addr_bytes, &infoPort.u.port_v7.data.macAddress, sizeof(eth_addr[port].addr_bytes)); + + if ((status = NT_InfoClose(hInfo)) != NT_SUCCESS) { + NT_ExplainError(status, errBuf, sizeof(errBuf)); + RTE_LOG(ERR, PMD, ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", status, errBuf); + return status; + } + + pmd_link.link_duplex = ETH_LINK_FULL_DUPLEX; + pmd_link.link_status = 0; + + internals->if_index = 0; + + pci_dev->numa_node = numa_node; + + data->dev_private = internals; + data->port_id = (*eth_dev)->data->port_id; + data->nb_rx_queues = (uint16_t)nb_rx_queues; + data->nb_tx_queues = (uint16_t)nb_tx_queues; + data->dev_link = pmd_link; + data->mac_addrs = ð_addr[port]; + data->numa_node = numa_node; + data->drv_name = drivername; + + (*eth_dev)->data = data; + (*eth_dev)->dev_ops = &ops; + (*eth_dev)->pci_dev = pci_dev; + + return 0; + +error: + if (data) + rte_free(data); + if (pci_dev) + rte_free(pci_dev); + if (internals) + rte_free(internals); + return -1; +} + +/* + * convert ascii to int + */ +static inline int +ascii_to_u32(const char *key __rte_unused, const char *value, void *extra_args __rte_unused) +{ + *(uint32_t*)extra_args = atoi(value); + return 0; +} + +static inline int +ascii_to_u32_array(const char *key __rte_unused, const char *value, void *extra_args) +{ + struct array_s *pastreams = (struct array_s *)extra_args; + int sval, eval; + if (sscanf(value, "%d..%d", &sval, &eval) == 2) { + int cnt; + for (cnt = sval; cnt <= eval; cnt++) + pastreams->value[pastreams->count++] = cnt; + } else { + pastreams->value[pastreams->count++] = atoi(value); + } + return 0; +} + + + + + +static int DoNtpl(const char *ntplStr) +{ + NtConfigStream_t hCfgStream; + NtNtplInfo_t ntplInfo; + int status; + + if((status = NT_ConfigOpen(&hCfgStream, "capture")) != NT_SUCCESS) { + /* Get the status code as text */ + NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)-1); + fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer); + return -1; + } + + RTE_LOG(INFO, PMD, "NTPL : %s\n", ntplStr); + if((status = NT_NTPL(hCfgStream, ntplStr, &ntplInfo, NT_NTPL_PARSER_VALIDATE_NORMAL)) != NT_SUCCESS) { + /* Get the status code as text */ + NT_ExplainError(status, errorBuffer, sizeof(errorBuffer)-1); + fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer); + fprintf(stderr, ">>> NTPL errorcode: %X\n", ntplInfo.u.errorData.errCode); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); + NT_ConfigClose(hCfgStream); + return -1; + } + NT_ConfigClose(hCfgStream); + return 0; +} + + + + +static int +rte_pmd_ntnic_devinit(const char *name, const char *params) +{ + unsigned numa_node; + int ret = 0; + struct rte_kvargs *kvlist; + struct rte_eth_dev *eth_dev; + unsigned int i; + uint32_t rxqueues = 0; + uint32_t txqueues = 0; + uint32_t port=0; + uint32_t portend = (uint32_t)-1; + uint32_t hash = (uint32_t)-1; + struct array_s astreamids; + + static int first = 1; + static int stream_id = 0; + char ntplStr[512]; + + astreamids.count = 0; + + RTE_LOG(INFO, PMD, "Initializing pmd_ntnic for %s\n", name); + + numa_node = rte_socket_id(); + + kvlist = rte_kvargs_parse(params, valid_arguments); + if (kvlist == NULL) { + return -1; + } + + /* Get port to use for Rx/Tx */ + if ((i = rte_kvargs_count(kvlist, ETH_NTNIC_PORT_ARG))) { + assert (i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORT_ARG, + &ascii_to_u32, &port); + } + /* If Rx port merge is need, her the portend is specified */ + if ((i = rte_kvargs_count(kvlist, ETH_NTNIC_PORTEND_ARG))) { + assert (i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORTEND_ARG, + &ascii_to_u32, &portend); + } + + /* Get # RX queues */ + if ((i = rte_kvargs_count(kvlist, ETH_NTNIC_RXQUEUES_ARG))) { + assert (i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_RXQUEUES_ARG, + &ascii_to_u32, &rxqueues); + } + /* Get # TX queues */ + if ((i = rte_kvargs_count(kvlist, ETH_NTNIC_TXQUEUES_ARG))) { + assert (i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_TXQUEUES_ARG, + &ascii_to_u32, &txqueues); + } + + /* Get list of streamIds - if used */ + if ((i = rte_kvargs_count(kvlist, ETH_NTNIC_STREAMID_ARG))) { + ret = rte_kvargs_process(kvlist, ETH_NTNIC_STREAMID_ARG, + &ascii_to_u32_array, &astreamids); + } + + /* Get an alternative hash algorithm */ + if ((i = rte_kvargs_count(kvlist, ETH_NTNIC_HASH_ARG))) { + assert (i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_HASH_ARG, + &ascii_to_u32, &hash); + } + + /* check portend and streamids */ + if (portend != (uint32_t)-1 && astreamids.count) { + RTE_LOG(ERR, PMD, "Cannot specify portend when one or more streamid's are specified\n"); + return -1; + } + rte_kvargs_free(kvlist); + + if (ret < 0) + return -1; + + if (first) + NT_Init(NTAPI_VERSION); + + if (astreamids.count) { + first = 0; + /* specific streamid specified, use that - port defaults to Tx port only */ + if (rte_pmd_init_internals(name, rxqueues, txqueues, numa_node, &astreamids, port, ð_dev) < 0) + return -1; + } else { + struct array_s astrids; + if (first) { + /* Delete all NTPL */ + sprintf(ntplStr, "Delete=All"); + if (DoNtpl(ntplStr) != 0) { + return -1; + } + first = 0; + } + astrids.count=1; + astrids.value[0] = stream_id; + if (rte_pmd_init_internals(name, rxqueues, txqueues, numa_node, &astrids, port, ð_dev) < 0) + return -1; + + /* Assign the traffic */ + if (portend != (uint32_t)-1) { + sprintf(ntplStr, "Assign[streamid=(%d..%d);Descriptor=NT]=port==(%d..%d)", stream_id, + (stream_id+rxqueues-1), port, portend); + + } else { + sprintf(ntplStr, "Assign[streamid=(%d..%d);Descriptor=NT]=port==%d", stream_id, + (stream_id+rxqueues-1), port); + } + if (DoNtpl(ntplStr) != 0) { + return -1; + } + + if (hash != (uint32_t)-1) { + switch (hash) { + default: + case 1: + DoNtpl("HashMode=HashRoundRobin"); + break; + case 2: + DoNtpl("HashMode=Hash2TupleSorted"); + break; + case 3: + DoNtpl("HashMode=Hash5TupleSorted"); + break; + } + } + + stream_id += rxqueues; + } + + eth_dev->rx_pkt_burst = eth_ntnic_rx; + eth_dev->tx_pkt_burst = eth_ntnic_tx_ringbuffer; + + return 0; +} + + + +static struct rte_driver pmd_ntnic_drv = { + .type = PMD_VDEV, + .init = rte_pmd_ntnic_devinit, +}; + +PMD_REGISTER_DRIVER(pmd_ntnic_drv, eth_ntnic); +DRIVER_REGISTER_PARAM_STRING(eth_ntnic, + "port=<int> " + "rxqs=<int>" + "txqs=<int>" + "hash=<int>" + "streamids=<int..int>"); + + diff --git a/drivers/net/ntnic/rte_pmd_ntnic_version.map b/drivers/net/ntnic/rte_pmd_ntnic_version.map new file mode 100644 index 0000000..ef35398 --- /dev/null +++ b/drivers/net/ntnic/rte_pmd_ntnic_version.map @@ -0,0 +1,4 @@ +DPDK_2.0 { + + local: *; +}; diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 1a0095b..c39785f 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -120,6 +120,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += -lrte_pmd_mpipe -lgxio _LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += -lrte_pmd_nfp -lm _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += -lrte_pmd_null _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += -lrte_pmd_pcap -lpcap + _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD) += -lrte_pmd_qede -lz _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING) += -lrte_pmd_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 @@ -143,6 +144,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -lrte_pmd_kasumi _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi endif # CONFIG_RTE_LIBRTE_CRYPTODEV +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += -lrte_pmd_ntnic -L$(NAPATECH3_PATH)/lib -lntapi endif # !CONFIG_RTE_BUILD_SHARED_LIBS _LDLIBS-y += --no-whole-archive @@ -163,6 +165,11 @@ endif _LDLIBS-$(CONFIG_RTE_PORT_PCAP) += -lpcap endif # !CONFIG_RTE_BUILD_SHARED_LIBS +ifeq ($(CONFIG_RTE_LIBRTE_PMD_NTNIC),y) +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += -L$(NAPATECH3_PATH)/lib -lntapi +endif + + _LDLIBS-y += $(EXECENV_LDLIBS) LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS) -- 2.9.0 Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH] ntnic: add PMD driver 2016-08-26 13:44 [dpdk-dev] [PATCH] ntnic: add PMD driver Finn Christensen @ 2016-08-26 14:44 ` Thomas Monjalon 2016-08-26 16:32 ` Finn Christensen 2016-08-26 16:54 ` Stephen Hemminger 2016-09-08 11:14 ` [dpdk-dev] [PATCH v2] " Finn Christensen 2 siblings, 1 reply; 22+ messages in thread From: Thomas Monjalon @ 2016-08-26 14:44 UTC (permalink / raw) To: Finn Christensen; +Cc: dev Welcome, 2016-08-26 13:44, Finn Christensen: > +NTNIC Poll Mode Driver > +====================== > + > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements support > +for **Napatech NIC** 40/50 Gbps adapters. > +This PMD is implemented as a pure software virtual device and must be created > +by using the EAL --vdev=parameter (parameters are explained i detail later). > +It runs on top of the Napatech NIC Driver Suite that must be installed and > +started. It would help a lot to describe what is the Napatech NIC Driver Suite, and why it is a virtual driver. What is the hardware it runs on? Where can we get it? Where can we download the Driver Suite? > +Supported Features > +------------------ > + > +- RSS (Receive Side Scaling) > +- TSS (Transmit Side Scaling) > +- Promiscuous mode > +- Basic statistics > + > + > +Prerequisites > +------------- > + > +Requires Napatech NIC and Napatech NIC Driver Suite installed and > +running in version **0.3.0** or higher. > +This includes external libraries and kernel driver for resources > +allocations and initialization. So the Driver Suite is just a library to help implementing the driver for the Napatech NIC? Thanks for explaining your solution ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH] ntnic: add PMD driver 2016-08-26 14:44 ` Thomas Monjalon @ 2016-08-26 16:32 ` Finn Christensen 2016-08-27 9:07 ` Thomas Monjalon 0 siblings, 1 reply; 22+ messages in thread From: Finn Christensen @ 2016-08-26 16:32 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: 26. august 2016 16:44 > To: Finn Christensen <fc@napatech.com> > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH] ntnic: add PMD driver > > Welcome, Thanks! > > 2016-08-26 13:44, Finn Christensen: > > +NTNIC Poll Mode Driver > > +====================== > > + > > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements > > +support for **Napatech NIC** 40/50 Gbps adapters. > > +This PMD is implemented as a pure software virtual device and must be > > +created by using the EAL --vdev=parameter (parameters are explained i > detail later). > > +It runs on top of the Napatech NIC Driver Suite that must be > > +installed and started. > > It would help a lot to describe what is the Napatech NIC Driver Suite, and why > it is a virtual driver. I will elaborate on the description to make it more clear and send an update patch. However, the driver is a virtual driver because we already bypasses the kernel stack in our NIC driver and does that in an effective manner. > What is the hardware it runs on? Where can we get it? It is running on a newly developed hardware by Napatech. It is not yet available, but it will soon be. > Where can we download the Driver Suite? When the Napatech NIC is released, the driver suite will then be available for download. If you need something sooner, we will need to make special arrangements. > > > +Supported Features > > +------------------ > > + > > +- RSS (Receive Side Scaling) > > +- TSS (Transmit Side Scaling) > > +- Promiscuous mode > > +- Basic statistics > > + > > + > > +Prerequisites > > +------------- > > + > > +Requires Napatech NIC and Napatech NIC Driver Suite installed and > > +running in version **0.3.0** or higher. > > +This includes external libraries and kernel driver for resources > > +allocations and initialization. > > So the Driver Suite is just a library to help implementing the driver for the > Napatech NIC? Yes and no. The driver suite contains additional monitoring and diagnostics tools, but the DPDK PMD only needs a few libraries and the kernel driver. > > Thanks for explaining your solution Thank you for the fast reply. This is really appreciated. Thanks again! Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH] ntnic: add PMD driver 2016-08-26 16:32 ` Finn Christensen @ 2016-08-27 9:07 ` Thomas Monjalon 0 siblings, 0 replies; 22+ messages in thread From: Thomas Monjalon @ 2016-08-27 9:07 UTC (permalink / raw) To: Finn Christensen; +Cc: dev 2016-08-26 16:32, Finn Christensen: > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > It would help a lot to describe what is the Napatech NIC Driver Suite, and > > why it is a virtual driver. > > I will elaborate on the description to make it more clear and send an update > patch. However, the driver is a virtual driver because we already bypasses > the kernel stack in our NIC driver and does that in an effective manner. Is there some plan to upstream your driver in Linux? We need to think about the limitations we could have when implementing a PCI driver as a virtual driver. Any thoughts? [...] > > Where can we download the Driver Suite? > > When the Napatech NIC is released, the driver suite will then be available for download. > If you need something sooner, we will need to make special arrangements. We cannot accept a driver until we can freely compile it. What is the expected release date? ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH] ntnic: add PMD driver 2016-08-26 13:44 [dpdk-dev] [PATCH] ntnic: add PMD driver Finn Christensen 2016-08-26 14:44 ` Thomas Monjalon @ 2016-08-26 16:54 ` Stephen Hemminger 2016-08-29 6:22 ` Finn Christensen 2016-09-08 11:14 ` [dpdk-dev] [PATCH v2] " Finn Christensen 2 siblings, 1 reply; 22+ messages in thread From: Stephen Hemminger @ 2016-08-26 16:54 UTC (permalink / raw) To: Finn Christensen; +Cc: dev On Fri, 26 Aug 2016 13:44:01 +0000 Finn Christensen <fc@napatech.com> wrote: > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements support > +for **Napatech NIC** 40/50 Gbps adapters. > +This PMD is implemented as a pure software virtual device and must be created > +by using the EAL --vdev=parameter (parameters are explained i detail later). > +It runs on top of the Napatech NIC Driver Suite that must be installed and > +started. What is the license of this driver suite? IMHO the upstream DPDK shouldn't be a platform for non-free driver suites. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH] ntnic: add PMD driver 2016-08-26 16:54 ` Stephen Hemminger @ 2016-08-29 6:22 ` Finn Christensen 2016-08-29 10:04 ` Thomas Monjalon 0 siblings, 1 reply; 22+ messages in thread From: Finn Christensen @ 2016-08-29 6:22 UTC (permalink / raw) To: Stephen Hemminger; +Cc: dev > From: Stephen Hemminger [mailto:stephen@networkplumber.org] > Sent: 26. august 2016 18:54 > To: Finn Christensen <fc@napatech.com> > Cc: dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH] ntnic: add PMD driver > > On Fri, 26 Aug 2016 13:44:01 +0000 > Finn Christensen <fc@napatech.com> wrote: > > > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements > > +support for **Napatech NIC** 40/50 Gbps adapters. > > +This PMD is implemented as a pure software virtual device and must be > > +created by using the EAL --vdev=parameter (parameters are explained i > detail later). > > +It runs on top of the Napatech NIC Driver Suite that must be > > +installed and started. > > What is the license of this driver suite? The driver suite is a closed-source driver, which is not free downloadable. > IMHO the upstream DPDK shouldn't be a platform for non-free driver suites. This is our first steps towards opensource, and our upcoming NIC is partly build on HW and SW from our Accelerator products. And since we already bypasses the kernel effectively in our driver, this first solution for a DPDK PMD driver, has been built on top of that software suite. We like the idea of opensource, but we will need to do the transition stepwise, considering our NIC product. We have seen large performance improvements (x4-x7 times with 64 byte packets compared to a std NIC in a phy-ovs-vm-ovs-phy setup utilizing a modified DPDK), and this is the main motivation to go forward and try to push our contribution to DPDK upstream. This is the first step of contributions that we want to make. This DPDK PMD solution is not compileable unless you have our driver. We may need to make that possible, so that a free downloadable driver can be retrieved. Once this is said, we thought that the DPDK was BSD licensed and I must admit that we have failed to see this limitation that you are mentioning. Is there another license or agreement text that we need to read? Thanks for your repy! Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH] ntnic: add PMD driver 2016-08-29 6:22 ` Finn Christensen @ 2016-08-29 10:04 ` Thomas Monjalon 2016-08-29 12:00 ` Finn Christensen 0 siblings, 1 reply; 22+ messages in thread From: Thomas Monjalon @ 2016-08-29 10:04 UTC (permalink / raw) To: Finn Christensen; +Cc: dev, Stephen Hemminger 2016-08-29 06:22, Finn Christensen: > From: Stephen Hemminger [mailto:stephen@networkplumber.org] > > What is the license of this driver suite? > > The driver suite is a closed-source driver, which is not free downloadable. > > > IMHO the upstream DPDK shouldn't be a platform for non-free driver suites. > > This is our first steps towards opensource, and our upcoming NIC is > partly build on HW and SW from our Accelerator products. > And since we already bypasses the kernel effectively in our driver, > this first solution for a DPDK PMD driver, has been built on top of > that software suite. > We like the idea of opensource, but we will need to do the transition > stepwise, considering our NIC product. I think the first step should be to free the lowest level, here the code you build your drivers on. > We have seen large performance improvements (x4-x7 times with 64 byte > packets compared to a std NIC in a phy-ovs-vm-ovs-phy setup utilizing > a modified DPDK), and this is the main motivation to go forward and > try to push our contribution to DPDK upstream. > This is the first step of contributions that we want to make. > This DPDK PMD solution is not compileable unless you have our driver. Not being able to compile the PMD is a real problem for maintenance. The PMD would be considered as dead code, so is forbidden. > We may need to make that possible, so that a free downloadable driver > can be retrieved. > > Once this is said, we thought that the DPDK was BSD licensed and I must > admit that we have failed to see this limitation that you are mentioning. > Is there another license or agreement text that we need to read? Maybe you'll find some interesting parts in the contributing guide. But honestly, I think you have already done the right thing in your case: you sent some code and open the discussion :) Now you just need to enable a free compilation environment for your patch. Thanks ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH] ntnic: add PMD driver 2016-08-29 10:04 ` Thomas Monjalon @ 2016-08-29 12:00 ` Finn Christensen 0 siblings, 0 replies; 22+ messages in thread From: Finn Christensen @ 2016-08-29 12:00 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev, Stephen Hemminger > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: 29. august 2016 12:04 > To: Finn Christensen <fc@napatech.com> > Cc: dev@dpdk.org; Stephen Hemminger <stephen@networkplumber.org> > Subject: Re: [dpdk-dev] [PATCH] ntnic: add PMD driver > > 2016-08-29 06:22, Finn Christensen: > > From: Stephen Hemminger [mailto:stephen@networkplumber.org] > > > What is the license of this driver suite? > > > > The driver suite is a closed-source driver, which is not free downloadable. > > > > > IMHO the upstream DPDK shouldn't be a platform for non-free driver > suites. > > > > This is our first steps towards opensource, and our upcoming NIC is > > partly build on HW and SW from our Accelerator products. > > And since we already bypasses the kernel effectively in our driver, > > this first solution for a DPDK PMD driver, has been built on top of > > that software suite. > > We like the idea of opensource, but we will need to do the transition > > stepwise, considering our NIC product. > > I think the first step should be to free the lowest level, here the code you > build your drivers on. I think that will not become a problem. > > We have seen large performance improvements (x4-x7 times with 64 byte > > packets compared to a std NIC in a phy-ovs-vm-ovs-phy setup utilizing > > a modified DPDK), and this is the main motivation to go forward and > > try to push our contribution to DPDK upstream. > > This is the first step of contributions that we want to make. > > This DPDK PMD solution is not compileable unless you have our driver. > > Not being able to compile the PMD is a real problem for maintenance. > The PMD would be considered as dead code, so is forbidden. Yes, that does fully make sense. We will come up with a way to freely download a driver with header files and libraries to make it possible for anyone to build the PMD. > > > We may need to make that possible, so that a free downloadable driver > > can be retrieved. > > > > Once this is said, we thought that the DPDK was BSD licensed and I > > must admit that we have failed to see this limitation that you are > mentioning. > > Is there another license or agreement text that we need to read? > > Maybe you'll find some interesting parts in the contributing guide. > But honestly, I think you have already done the right thing in your case: > you sent some code and open the discussion :) Now you just need to enable > a free compilation environment for your patch. > Thanks We will do that. I'll get back with an updated patch once this has been established. Thank you both for your help on this. Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* [dpdk-dev] [PATCH v2] ntnic: add PMD driver 2016-08-26 13:44 [dpdk-dev] [PATCH] ntnic: add PMD driver Finn Christensen 2016-08-26 14:44 ` Thomas Monjalon 2016-08-26 16:54 ` Stephen Hemminger @ 2016-09-08 11:14 ` Finn Christensen 2016-09-08 13:49 ` Neil Horman 2016-09-09 12:48 ` [dpdk-dev] [PATCH v3] " Finn Christensen 2 siblings, 2 replies; 22+ messages in thread From: Finn Christensen @ 2016-09-08 11:14 UTC (permalink / raw) To: dev, thomas.monjalon; +Cc: stephen, Finn Christensen This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. This patch adds support for Napatech NICs to DPDK. This is the initial implementation. Signed-off-by: Finn Christensen <fc@napatech.com> --- v2: * Added information how to build the PMD without NIC Board Support Package * Fixed some formatting issues --- MAINTAINERS | 5 + config/common_base | 6 + doc/guides/nics/features/ntnic.ini | 15 + doc/guides/nics/index.rst | 1 + doc/guides/nics/ntnic.rst | 145 +++++ drivers/net/Makefile | 1 + drivers/net/ntnic/Makefile | 65 ++ drivers/net/ntnic/rte_eth_ntnic.c | 975 ++++++++++++++++++++++++++++ drivers/net/ntnic/rte_pmd_ntnic_version.map | 4 + mk/rte.app.mk | 7 + 10 files changed, 1224 insertions(+) create mode 100644 doc/guides/nics/features/ntnic.ini create mode 100644 doc/guides/nics/ntnic.rst create mode 100644 drivers/net/ntnic/Makefile create mode 100644 drivers/net/ntnic/rte_eth_ntnic.c create mode 100644 drivers/net/ntnic/rte_pmd_ntnic_version.map diff --git a/MAINTAINERS b/MAINTAINERS index bc9aa02..d3e5f56 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -356,6 +356,11 @@ M: Sony Chacko <sony.chacko@qlogic.com> F: drivers/net/qede/ F: doc/guides/nics/qede.rst +Napatech ntnic +M: Finn Christensen <fc@napatech.com> +F: drivers/net/ntnic/ +F: doc/guides/nics/ntnic.rst + RedHat virtio M: Huawei Xie <huawei.xie@intel.com> M: Yuanhan Liu <yuanhan.liu@linux.intel.com> diff --git a/config/common_base b/config/common_base index 7830535..4a3b2b8 100644 --- a/config/common_base +++ b/config/common_base @@ -309,6 +309,12 @@ CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16 CONFIG_RTE_LIBRTE_PMD_PCAP=n # +# Compile software PMD backed by NTNIC files +# +CONFIG_RTE_LIBRTE_PMD_NTNIC=n + + +# # Compile link bonding PMD library # CONFIG_RTE_LIBRTE_PMD_BOND=y diff --git a/doc/guides/nics/features/ntnic.ini b/doc/guides/nics/features/ntnic.ini new file mode 100644 index 0000000..b6cfc5a --- /dev/null +++ b/doc/guides/nics/features/ntnic.ini @@ -0,0 +1,15 @@ +; +; Supported features of the 'ntnic' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +Queue start/stop = Y +Promiscuous mode = Y +Jumbo frame = Y +Basic stats = Y +Multiprocess aware = Y +x86-32 = Y +x86-64 = Y +Usage doc = Y diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 92d56a5..5c4205c 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -52,6 +52,7 @@ Network Interface Controller Drivers qede szedata2 thunderx + ntnic virtio vhost vmxnet3 diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst new file mode 100644 index 0000000..8512d02 --- /dev/null +++ b/doc/guides/nics/ntnic.rst @@ -0,0 +1,145 @@ +.. BSD LICENSE + Copyright (c) 2016 Napatech A/S + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Napatech nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +NTNIC Poll Mode Driver +====================== + +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements support +for **Napatech NIC** 40/50 Gbps adapters. +This PMD is implemented as a pure software virtual device and must be created +by using the EAL --vdev=parameter (parameters are explained i detail later). +It runs on top of the Napatech NFV NIC Board Support Package that must be +installed and started. +If no Napatech NIC is available, you can download the Napatech NTAPI library to +build against. + +Supported Features +------------------ + +- RSS (Receive Side Scaling) +- TSS (Transmit Side Scaling) +- Promiscuous mode +- Basic statistics + + +Prerequisites +------------- + +Requires Napatech NIC and Napatech NIC Board Support Package installed and +running in version **0.3.0** or higher. +This includes external libraries and kernel driver for resources +allocations and initialization. +If build only is required, download the Napatech NTAPI to build against. + +Pre-Installation Configuration +------------------------------ + +Environment variables +~~~~~~~~~~~~~~~~~~~~~ + +In order to compile the Napatech NIC PMD, user must: + +* Export the environment variable NAPATECH3_PATH with the path where + the Napatech Board Support Package libs was installed (default location is + /opt/napatech3). + Or if no Napatech NIC available: + Download the Napatech NTAPI libs from Github: + `Napatech NTAPI <https://github.com/napa-tech/ntapi>`_. + + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +- ``CONFIG_RTE_LIBRTE_PMD_NTNIC`` (default **n**) + +Using the NTNIC PMD from a DPDK application using EAL vdev parameter +-------------------------------------------------------------------- + +Napatech NIC PMD VDEV parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Napatech NIC PMD vdev has the following command line parameters to enable +its features. + +* port + Configures which NT port the vdev should use. + + --vdev eth_ntnic0,port=4,rxqs=1,txqs=1 + + This will create a DPDK port0 using NT port 4 + +* rxqs + Control how many receive queues a vdev should have. The traffic from a vdev + will be load balanced to this amount of queues and each queue can be handled + by its own lcore. + Note: The ntservice.ini HostBuffersRx must be configured to the sum of all + receive queues across all vdevs. + + --vdev eth_ntnic0,port=0,rxqs=8,txqs=1 + + Traffic from NT port 0 will be split across + 8 queues meaning that 8 lcores can be + opened on DPDK port 0 and each will be + their individual traffic. + +* hash + Control which load balancing scheme should be used when running with multiple + receive queues. + The possible values are: + 1 = HashRoundRobin + 2 = Hash2TupleSorted + 3 = Hash5TupleSorted + + --vdev eth_ntnic0,port=0,rxqs=4,hash=3,txqs=1 + + Create 4 queues from NT port 0 and + load balance the traffic using the + 5-tuple sorted algorithm. Each of the + 4 queues can be opened via DPDK port 0 + +* txqs + Control how many transmit queues a vdev should have. Each queue can be used + by a lcore and each queue is independent for other queues. + Note: The ntservice.ini HostBuffersTx must be configured to the sum of all + transmit queues across all vdevs. + + --vdev eth_ntnic0,port=0,txqs=32,rxqs=1 + + Enable 32 transmit queues on NT port 0 + + +Changes to Napatech driver configuration file ntservice.ini +----------------------------------------------------------- +Depending on the number of queues that is created, more hostbuffers may be +needed pre-allocated by the Napatech NIC Driver. This is controlled in the +ntservice.ini file (default locations is /opt/napatech3/config). +The Napatech NIC Driver must be re-started when this configuration file is +changed. + diff --git a/drivers/net/Makefile b/drivers/net/Makefile index bc93230..aff29e9 100644 --- a/drivers/net/Makefile +++ b/drivers/net/Makefile @@ -55,6 +55,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3 DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt +DIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += ntnic ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) DIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += vhost diff --git a/drivers/net/ntnic/Makefile b/drivers/net/ntnic/Makefile new file mode 100644 index 0000000..2ed24b3 --- /dev/null +++ b/drivers/net/ntnic/Makefile @@ -0,0 +1,65 @@ +# BSD LICENSE +# +# Copyright(c) 2016 Napatech A/S. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Napatech nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_ntnic.a + +CFLAGS += -O3 -g +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -I$(NAPATECH3_PATH)/include +LDLIBS += -L$(NAPATECH3_PATH)/lib -lntapi -lntos + +EXPORT_MAP := rte_pmd_ntnic_version.map + +LIBABIVER := 1 + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += rte_eth_ntnic.c + +# +# Export include files +# +SYMLINK-y-include += + +# this lib depends upon: +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_eal +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mbuf +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mempool +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_ether +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_kvargs + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/ntnic/rte_eth_ntnic.c b/drivers/net/ntnic/rte_eth_ntnic.c new file mode 100644 index 0000000..41e7707 --- /dev/null +++ b/drivers/net/ntnic/rte_eth_ntnic.c @@ -0,0 +1,975 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 Napatech A/S. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Napatech nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <time.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_memcpy.h> +#include <rte_string_fns.h> +#include <rte_cycles.h> +#include <rte_kvargs.h> +#include <rte_dev.h> +#include <net/if.h> +#include <nt.h> + +#define ETH_NTNIC_PORT_ARG "port" +#define ETH_NTNIC_PORTEND_ARG "portend" +#define ETH_NTNIC_RXQUEUES_ARG "rxqs" +#define ETH_NTNIC_TXQUEUES_ARG "txqs" +#define ETH_NTNIC_STREAMID_ARG "streamid" +#define ETH_NTNIC_HASH_ARG "hash" + +#define HW_MAX_PKT_LEN 10000 +#define HW_MTU (HW_MAX_PKT_LEN - ETHER_HDR_LEN - ETHER_CRC_LEN) + +#define MAX_RX_QUEUES 64 +#define MAX_TX_QUEUES 64 +#define MAX_NTNIC_PORTS 32 + +static char errorBuffer[1024]; + +struct array_s { + uint32_t value[MAX_RX_QUEUES]; + int count; +}; + +static volatile uint16_t port_locks[MAX_NTNIC_PORTS]; + +struct ntnic_rx_queue { + NtNetStreamRx_t net_rx[MAX_RX_QUEUES]; /* NT Rx streams */ + int net_rx_idx; /* Current Rx stream index */ + struct rte_mempool *mb_pool; /* mbuf memory pool */ + uint16_t buf_size; /* Size of data area in mbuf */ + NtNetBuf_t seg; /* The current NT data segment */ + struct NtNetBuf_s pkt; /* The current packet */ + volatile unsigned long rx_pkts; /* Rx packet statistics */ + volatile unsigned long err_pkts; /* Rx error packet statistics */ + struct array_s astreamids; /* NT streams to read from */ + int enabled; /* Enabling/disabling of this queue */ +}; + +struct ntnic_tx_queue { + NtNetStreamTx_t net_tx; /* NT Tx stream */ + volatile unsigned long tx_pkts; /* Tx packet statistics */ + volatile unsigned long err_pkts; /* Tx error packet stat */ + volatile uint16_t *plock; /* Per port transmit atomic lock */ + uint32_t port; /* Tx port for this queue */ + int enabled; /* Enabling/disabling of this queue */ + int fcs_add; /* If needs added room for FCS */ +}; + +struct pmd_internals { + struct ntnic_rx_queue rxq[MAX_RX_QUEUES]; /* Array of Rx queues */ + struct ntnic_tx_queue txq[MAX_TX_QUEUES]; /* Array of Tx queues */ + int if_index; /* Itf index always 0 - no bonding */ + unsigned int nb_rx_queues; /* Number of Rx queues configured */ + unsigned int nb_tx_queues; /* Number of Tx queues configured */ + int MAC_50G; /* True if 50G ports */ +}; + +static const char *valid_arguments[] = { + ETH_NTNIC_PORT_ARG, + ETH_NTNIC_PORTEND_ARG, + ETH_NTNIC_RXQUEUES_ARG, + ETH_NTNIC_TXQUEUES_ARG, + ETH_NTNIC_STREAMID_ARG, + ETH_NTNIC_HASH_ARG, + NULL +}; + +static struct ether_addr eth_addr[MAX_NTNIC_PORTS]; +static const char *drivername = "NTNIC PMD"; + +static int +eth_ntnic_rx_jumbo(struct rte_mempool *mb_pool, + struct rte_mbuf *mbuf, + const u_char *data, + uint16_t data_len) +{ + struct rte_mbuf *m = mbuf; + + /* Copy the first segment. */ + uint16_t len = rte_pktmbuf_tailroom(mbuf); + + rte_memcpy(rte_pktmbuf_append(mbuf, len), data, len); + data_len -= len; + data += len; + + while (data_len > 0) { + /* Allocate next mbuf and point to that. */ + m->next = rte_pktmbuf_alloc(mb_pool); + + if (unlikely(!m->next)) + return -1; + + m = m->next; + + /* Headroom is not needed in chained mbufs. */ + rte_pktmbuf_prepend(m, rte_pktmbuf_headroom(m)); + m->pkt_len = 0; + m->data_len = 0; + + /* Copy next segment. */ + len = RTE_MIN(rte_pktmbuf_tailroom(m), data_len); + rte_memcpy(rte_pktmbuf_append(m, len), data, len); + + mbuf->nb_segs++; + data_len -= len; + data += len; + } + + return mbuf->nb_segs; +} + + + +static uint16_t +eth_ntnic_rx(void *queue, + struct rte_mbuf **bufs, + uint16_t nb_pkts) +{ + unsigned int i; + struct rte_mbuf *mbuf; + struct ntnic_rx_queue *rx_q = queue; + uint16_t num_rx = 0; + uint16_t data_len; + + if (unlikely(rx_q->net_rx[rx_q->net_rx_idx] == NULL || nb_pkts == 0)) + return 0; + + /* Do we have any data segment */ + if (rx_q->seg == NULL) { + /* Next stream - if multiple streams are combined */ + rx_q->net_rx_idx = + (rx_q->net_rx_idx < + (rx_q->astreamids.count - 1)) ? rx_q->net_rx_idx + 1 + : 0; + + if (NT_NetRxGet(rx_q->net_rx[rx_q->net_rx_idx], + &rx_q->seg, 0) != NT_SUCCESS) { + if (rx_q->seg != NULL) { + NT_NetRxRelease(rx_q->net_rx[rx_q->net_rx_idx], + rx_q->seg); + rx_q->seg = NULL; + } + return 0; + } + if (NT_NET_GET_SEGMENT_LENGTH(rx_q->seg)) { + /* Build a packet structure */ + _nt_net_build_pkt_netbuf(rx_q->seg, &rx_q->pkt); + } else { + NT_NetRxRelease(rx_q->net_rx[rx_q->net_rx_idx], + rx_q->seg); + rx_q->seg = NULL; + return 0; + } + } + + if (rte_mempool_get_bulk(rx_q->mb_pool, (void **)bufs, nb_pkts) != 0) + return 0; + + for (i = 0; i < nb_pkts; i++) { + mbuf = bufs[i]; + rte_mbuf_refcnt_set(mbuf, 1); + rte_pktmbuf_reset(mbuf); + + /* HW slicing not supported */ + data_len = (uint16_t)NT_NET_GET_PKT_WIRE_LENGTH((&rx_q->pkt)) + - 4; + if (data_len <= rx_q->buf_size) { + /* Packet will fit in the mbuf, go ahead and copy */ + mbuf->pkt_len = mbuf->data_len; + data_len = mbuf->data_len; + rte_memcpy((u_char *)mbuf->buf_addr + + RTE_PKTMBUF_HEADROOM, + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), + mbuf->data_len); + } else { + /* Try read jumbo frame into multi mbufs. */ + if (unlikely(eth_ntnic_rx_jumbo(rx_q->mb_pool, + mbuf, + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), + data_len) == -1)) + break; + } + + mbuf->port = NT_NET_GET_PKT_RXPORT((&rx_q->pkt)); + num_rx++; + + /* Get the next packet if any */ + if (_nt_net_get_next_packet(rx_q->seg, + NT_NET_GET_SEGMENT_LENGTH(rx_q->seg), + &rx_q->pkt) == 0) { + NT_NetRxRelease(rx_q->net_rx[rx_q->net_rx_idx], + rx_q->seg); + rx_q->seg = NULL; + break; + } + + rx_q->rx_pkts++; + } + + if (num_rx < nb_pkts) { + rte_mempool_put_bulk(rx_q->mb_pool, + (void * const *)(bufs + num_rx), + nb_pkts - num_rx); + } + return num_rx; +} + + + + + +/* Callback to handle sending packets through a NT NIC. */ +static uint16_t +eth_ntnic_tx_ringbuffer(void *queue, + struct rte_mbuf **bufs, + uint16_t nb_pkts) +{ + unsigned int i; + struct ntnic_tx_queue *tx_q = queue; + int retval; + uint16_t old; + uint16_t new_flag = 0; + + + if (unlikely(tx_q == NULL || tx_q->net_tx == NULL || nb_pkts == 0)) + return 0; + + + do { + old = 0; + new_flag = 1; + retval = rte_atomic16_cmpset(tx_q->plock, old, new_flag); + } while (unlikely(retval == 0)); + + for (i = 0; i < nb_pkts; i++) { + /* Not allowed to TX less than 64 byte packets */ + if (bufs[i]->pkt_len < 60) + bufs[i]->pkt_len = 60; + + /* transmit a single packet */ + NT_NetTxRingbufferTransmitPacket(tx_q->net_tx, + rte_pktmbuf_mtod(bufs[i], uint8_t *), + bufs[i]->pkt_len + 4 - tx_q->fcs_add); + rte_pktmbuf_free(bufs[i]); + } + tx_q->tx_pkts += nb_pkts; + + *tx_q->plock = 0; + return nb_pkts; +} + + + +static int +eth_dev_start(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = internals->rxq; + struct ntnic_tx_queue *tx_q = internals->txq; + uint queue; + int status, idx; + + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + + if ((internals->nb_tx_queues | internals->nb_rx_queues) == 0) + return -1; + + for (queue = 0; queue < internals->nb_rx_queues; queue++) { + if (rx_q[queue].enabled) { + char str[128], val[10]; + str[0] = 0; + /* build list of streamids to log */ + for (idx = 0; idx < + rx_q[queue].astreamids.count; idx++) { + if (idx) + sprintf(val, ",%d", + rx_q[queue].astreamids.value[idx]); + else + sprintf(val, "%i", + rx_q[queue].astreamids.value[idx]); + strcat(str, val); + } + + for (idx = 0; idx < rx_q[queue].astreamids.count; + idx++) { + status = NT_NetRxOpen(&rx_q[queue].net_rx[idx], + "DPDK", + NT_NET_INTERFACE_SEGMENT, + rx_q[queue].astreamids.value[idx], -1); + if (status != NT_SUCCESS) { + /* try packet interface instead */ + NT_ExplainError(status, + errorBuffer, + sizeof(errorBuffer)); + RTE_LOG(ERR, PMD, + "NT_NetRxOpen() failed: %s\n", + errorBuffer); + return -1; + } + } + } + } + + for (queue = 0; queue < internals->nb_tx_queues; queue++) { + if (tx_q[queue].enabled) { + RTE_LOG(INFO, PMD, "NTNIC: NT_NetTxOpen(%d)\n", + tx_q[queue].port); + status = NT_NetTxOpen(&tx_q[queue].net_tx, "DPDK", + 1 << tx_q[queue].port, + dev->pci_dev->numa_node, 0); + if (status != NT_SUCCESS) { + NT_ExplainError(status, errorBuffer, + sizeof(errorBuffer)); + RTE_LOG(INFO, PMD, + "NT_NetTxOpen(0x%X, %d, 0) failed: %s\n", + 1 << tx_q[queue].port, + dev->pci_dev->numa_node, errorBuffer); + return -1; + } + + /* Init and associate RTD Tx Ring buf to Tx handler */ + status = NT_NetTxRingbufferInit( + tx_q[queue].net_tx, + tx_q[queue].port); + if (status != NT_SUCCESS) { + NT_ExplainError(status, errorBuffer, + sizeof(errorBuffer)); + RTE_LOG(ERR, PMD, + "NT_NetRxOpen() failed: %s\n", + errorBuffer); + return -1; + } + } + tx_q[queue].plock = &port_locks[tx_q[queue].port]; + tx_q[queue].fcs_add = (internals->MAC_50G) ? 4 : 0; + } + + dev->data->dev_link.link_status = 1; + return 0; +} + + +static void +eth_dev_stop(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = internals->rxq; + struct ntnic_tx_queue *tx_q = internals->txq; + uint queue; + int idx; + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + + for (queue = 0; queue < internals->nb_rx_queues; queue++) { + if (rx_q[queue].seg) { + NT_NetRxRelease(rx_q[queue].net_rx[rx_q->net_rx_idx], + rx_q[queue].seg); + rx_q[queue].seg = NULL; + } + for (idx = 0; idx < rx_q[queue].astreamids.count; idx++) { + if (rx_q[queue].net_rx[idx]) + (void)NT_NetRxClose(rx_q[queue].net_rx[idx]); + } + } + for (queue = 0; queue < internals->nb_tx_queues; queue++) { + if (tx_q[queue].net_tx) { + NT_NetTxRingbufferDone(tx_q[queue].net_tx); + (void)NT_NetTxClose(tx_q[queue].net_tx); + } + } + dev->data->dev_link.link_status = 0; +} + +static int +eth_dev_configure(struct rte_eth_dev *dev __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + return 0; +} + +static void +eth_dev_info(struct rte_eth_dev *dev, + struct rte_eth_dev_info *dev_info) +{ + struct pmd_internals *internals = dev->data->dev_private; + dev_info->if_index = internals->if_index; + dev_info->driver_name = drivername; + dev_info->max_mac_addrs = 1; + dev_info->max_rx_pktlen = HW_MTU; + dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; + dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; + dev_info->min_rx_bufsize = 0; + dev_info->pci_dev = NULL; +} + +static void +eth_stats_get(struct rte_eth_dev *dev, + struct rte_eth_stats *igb_stats) +{ + unsigned int i; + unsigned long rx_total = 0; + unsigned long tx_total = 0; + unsigned long tx_err_total = 0; + const struct pmd_internals *internal = dev->data->dev_private; + + memset(igb_stats, 0, sizeof(*igb_stats)); + for (i = 0; + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal->nb_rx_queues; + i++) { + igb_stats->q_ipackets[i] = internal->rxq[i].rx_pkts; + rx_total += igb_stats->q_ipackets[i]; + } + for (i = 0; + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal->nb_tx_queues; + i++) { + igb_stats->q_opackets[i] = internal->txq[i].tx_pkts; + igb_stats->q_errors[i] = internal->txq[i].err_pkts; + tx_total += igb_stats->q_opackets[i]; + tx_err_total += igb_stats->q_errors[i]; + } + + igb_stats->ipackets = rx_total; + igb_stats->opackets = tx_total; + igb_stats->oerrors = tx_err_total; +} + +static void +eth_stats_reset(struct rte_eth_dev *dev) +{ + unsigned int i; + struct pmd_internals *internal = dev->data->dev_private; + + for (i = 0; i < internal->nb_rx_queues; i++) + internal->rxq[i].rx_pkts = 0; + for (i = 0; i < internal->nb_tx_queues; i++) { + internal->txq[i].tx_pkts = 0; + internal->txq[i].err_pkts = 0; + } +} + +static void +eth_dev_close(struct rte_eth_dev *dev __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); +} + +static void +eth_queue_release(void *q __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); +} + +static int +eth_link_update(struct rte_eth_dev *dev __rte_unused, + int wait_to_complete __rte_unused) +{ + return 0; +} + +static int +eth_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t rx_queue_id, + uint16_t nb_rx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, + struct rte_mempool *mb_pool) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = &internals->rxq[rx_queue_id]; + + RTE_LOG(INFO, PMD, "NTNIC RX queue setup\n"); + rx_q->mb_pool = mb_pool; + dev->data->rx_queues[rx_queue_id] = rx_q; + + mbp_priv = rte_mempool_get_priv(rx_q->mb_pool); + rx_q->buf_size = (uint16_t)(mbp_priv->mbuf_data_room_size - + RTE_PKTMBUF_HEADROOM); + rx_q->enabled = 1; + return 0; +} + +static int +eth_tx_queue_setup(struct rte_eth_dev *dev __rte_unused, + uint16_t tx_queue_id __rte_unused, + uint16_t nb_tx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ + struct pmd_internals *internals = dev->data->dev_private; + RTE_LOG(INFO, PMD, "NTNIC TX queue setup\n"); + dev->data->tx_queues[tx_queue_id] = &internals->txq[tx_queue_id]; + internals->txq[tx_queue_id].enabled = 1; + return 0; +} + +static int _dev_set_mtu(struct rte_eth_dev *dev __rte_unused, uint16_t mtu) +{ + if (mtu < 46 || mtu > HW_MTU) + return -EINVAL; + + return 0; +} + +static struct eth_dev_ops ops = { + .dev_start = eth_dev_start, + .dev_stop = eth_dev_stop, + .dev_close = eth_dev_close, + .mtu_set = _dev_set_mtu, + .dev_configure = eth_dev_configure, + .dev_infos_get = eth_dev_info, + .rx_queue_setup = eth_rx_queue_setup, + .tx_queue_setup = eth_tx_queue_setup, + .rx_queue_release = eth_queue_release, + .tx_queue_release = eth_queue_release, + .link_update = eth_link_update, + .stats_get = eth_stats_get, + .stats_reset = eth_stats_reset, +}; + +static int +rte_pmd_init_internals(const char *name, + const unsigned int nb_rx_queues, + const unsigned int nb_tx_queues, + const unsigned int numa_node, + struct array_s *pastreamids, + const uint32_t port, + struct rte_eth_dev **eth_dev) +{ + struct pmd_internals *internals = NULL; + struct rte_eth_dev_data *data = NULL; + struct rte_pci_device *pci_dev = NULL; + uint i, status; + char err_buf[NT_ERRBUF_SIZE]; + NtInfoStream_t info; + NtInfo_t info_port; + struct rte_eth_link pmd_link; + assert(nb_rx_queues < MAX_RX_QUEUES); + assert(nb_tx_queues < MAX_TX_QUEUES); + assert(port < MAX_NTNIC_PORTS); + + RTE_LOG(INFO, PMD, + "Creating ntnic-backend ethdev on numa socket %u\n", + numa_node); + + /* now do all data allocation - for eth_dev structure, dummy pci driver + * and internal (private) data + */ + data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node); + if (data == NULL) + goto error; + + pci_dev = rte_zmalloc_socket(name, sizeof(*pci_dev), 0, numa_node); + if (pci_dev == NULL) + goto error; + + internals = rte_zmalloc_socket(name, sizeof(*internals), 0, numa_node); + if (internals == NULL) + goto error; + + /* reserve an ethdev entry */ + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); + if (*eth_dev == NULL) + goto error; + + /* Open the information stream */ + status = NT_InfoOpen(&info, "DPDK Info stream"); + if (status != NT_SUCCESS) { + NT_ExplainError(status, err_buf, sizeof(err_buf)); + RTE_LOG(ERR, PMD, + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", + status, err_buf); + return status; + } + + /* Find local port offset */ + info_port.cmd = NT_INFO_CMD_READ_PORT_V7; + info_port.u.port_v7.portNo = (uint8_t)(port); + status = NT_InfoRead(info, &info_port); + if (status != 0) { + NT_ExplainError(status, err_buf, sizeof(err_buf)); + RTE_LOG(ERR, PMD, + "ERROR: NT_InfoRead failed. Code 0x%x = %s\n", + status, err_buf); + return status; + } + + /* check for 50G MAC */ + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9506 || + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9509) { + internals->MAC_50G = 1; + } else { + internals->MAC_50G = 0; + /* Check for valid FPGAs (NFV NICs) */ + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != + 9507 && + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != + 9510) { + RTE_LOG(ERR, PMD, + "ERROR: NTNIC PMD does not support the NT adapter" + "- port %i\n", + port); + return -1; + } + } + + internals->nb_rx_queues = nb_rx_queues; + internals->nb_tx_queues = nb_tx_queues; + for (i = 0; i < nb_rx_queues; i++) { + int idx; + for (idx = 0; idx < pastreamids->count; idx++) { + internals->rxq[i].astreamids.value[idx] = + pastreamids->value[idx] + i; + } + + internals->rxq[i].astreamids.count = pastreamids->count; + + internals->rxq[i].seg = NULL; + internals->rxq[i].enabled = 0; + } + + for (i = 0; i < nb_tx_queues; i++) { + if (port < info_port.u.port_v7.data.adapterInfo.portOffset) { + RTE_LOG(ERR, PMD, "Error wrong port specified\n"); + return -1; + } + internals->txq[i].port = port; + internals->txq[i].enabled = 0; + } + + switch (info_port.u.port_v7.data.speed) { + case NT_LINK_SPEED_UNKNOWN: + pmd_link.link_speed = ETH_SPEED_NUM_1G; + break; + case NT_LINK_SPEED_10M: + pmd_link.link_speed = ETH_SPEED_NUM_10M; + break; + case NT_LINK_SPEED_100M: + pmd_link.link_speed = ETH_SPEED_NUM_100M; + break; + case NT_LINK_SPEED_1G: + pmd_link.link_speed = ETH_SPEED_NUM_1G; + break; + case NT_LINK_SPEED_10G: + pmd_link.link_speed = ETH_SPEED_NUM_10G; + break; + case NT_LINK_SPEED_40G: + pmd_link.link_speed = ETH_SPEED_NUM_40G; + break; + case NT_LINK_SPEED_50G: + pmd_link.link_speed = ETH_SPEED_NUM_50G; + break; + case NT_LINK_SPEED_100G: + pmd_link.link_speed = ETH_SPEED_NUM_100G; + break; + } + + memcpy(ð_addr[port].addr_bytes, &info_port.u.port_v7.data.macAddress, + sizeof(eth_addr[port].addr_bytes)); + status = NT_InfoClose(info); + if (status != NT_SUCCESS) { + NT_ExplainError(status, err_buf, sizeof(err_buf)); + RTE_LOG(ERR, PMD, + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", + status, err_buf); + return status; + } + + pmd_link.link_duplex = ETH_LINK_FULL_DUPLEX; + pmd_link.link_status = 0; + + internals->if_index = 0; + + pci_dev->numa_node = numa_node; + + data->dev_private = internals; + data->port_id = (*eth_dev)->data->port_id; + data->nb_rx_queues = (uint16_t)nb_rx_queues; + data->nb_tx_queues = (uint16_t)nb_tx_queues; + data->dev_link = pmd_link; + data->mac_addrs = ð_addr[port]; + data->numa_node = numa_node; + data->drv_name = drivername; + + (*eth_dev)->data = data; + (*eth_dev)->dev_ops = &ops; + (*eth_dev)->pci_dev = pci_dev; + + return 0; + +error: + if (data) + rte_free(data); + if (pci_dev) + rte_free(pci_dev); + if (internals) + rte_free(internals); + return -1; +} + +/* + * convert ascii to int + */ +static inline int +ascii_to_u32(const char *key __rte_unused, + const char *value, void *extra_args __rte_unused) +{ + *(uint32_t *)extra_args = atoi(value); + return 0; +} + +static inline int +ascii_to_u32_array(const char *key __rte_unused, + const char *value, void *extra_args) +{ + struct array_s *pastreams = (struct array_s *)extra_args; + int sval, eval; + if (sscanf(value, "%d..%d", &sval, &eval) == 2) { + int cnt; + for (cnt = sval; cnt <= eval; cnt++) + pastreams->value[pastreams->count++] = cnt; + } else { + pastreams->value[pastreams->count++] = atoi(value); + } + return 0; +} + + +static int DoNtpl(const char *ntplStr) +{ + NtConfigStream_t hCfgStream; + NtNtplInfo_t ntplInfo; + int status; + + status = NT_ConfigOpen(&hCfgStream, "capture"); + if (status != NT_SUCCESS) { + /* Get the status code as text */ + NT_ExplainError(status, errorBuffer, sizeof(errorBuffer) - 1); + fprintf(stderr, "NT_ConfigOpen() failed: %s\n", errorBuffer); + return -1; + } + + RTE_LOG(INFO, PMD, "NTPL : %s\n", ntplStr); + status = NT_NTPL(hCfgStream, ntplStr, &ntplInfo, + NT_NTPL_PARSER_VALIDATE_NORMAL); + if (status != NT_SUCCESS) { + /* Get the status code as text */ + NT_ExplainError(status, errorBuffer, sizeof(errorBuffer) - 1); + fprintf(stderr, "NT_NTPL() failed: %s\n", errorBuffer); + fprintf(stderr, ">>> NTPL errorcode: %X\n", + ntplInfo.u.errorData.errCode); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); + NT_ConfigClose(hCfgStream); + return -1; + } + NT_ConfigClose(hCfgStream); + return 0; +} + + + + +static int +rte_pmd_ntnic_devinit(const char *name, const char *params) +{ + unsigned int numa_node; + int ret = 0; + struct rte_kvargs *kvlist; + struct rte_eth_dev *eth_dev; + unsigned int i; + uint32_t rxqueues = 0; + uint32_t txqueues = 0; + uint32_t port = 0; + uint32_t portend = (uint32_t)-1; + uint32_t hash = (uint32_t)-1; + struct array_s astreamids; + + static int first = 1; + static int stream_id; + char ntplStr[512]; + + astreamids.count = 0; + + RTE_LOG(INFO, PMD, "Initializing pmd_ntnic for %s\n", name); + + numa_node = rte_socket_id(); + + kvlist = rte_kvargs_parse(params, valid_arguments); + if (kvlist == NULL) + return -1; + + /* Get port to use for Rx/Tx */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORT_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORT_ARG, + &ascii_to_u32, &port); + } + /* If Rx port merge is need, her the portend is specified */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORTEND_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORTEND_ARG, + &ascii_to_u32, &portend); + } + + /* Get # RX queues */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_RXQUEUES_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_RXQUEUES_ARG, + &ascii_to_u32, &rxqueues); + } + /* Get # TX queues */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_TXQUEUES_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_TXQUEUES_ARG, + &ascii_to_u32, &txqueues); + } + + /* Get list of streamIds - if used */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_STREAMID_ARG); + if (i) { + ret = rte_kvargs_process(kvlist, ETH_NTNIC_STREAMID_ARG, + &ascii_to_u32_array, &astreamids); + } + + /* Get an alternative hash algorithm */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_HASH_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_HASH_ARG, + &ascii_to_u32, &hash); + } + + /* check portend and streamids */ + if (portend != (uint32_t)-1 && astreamids.count) { + RTE_LOG(ERR, PMD, "Cannot specify portend when one or more" + " streamid's are specified\n"); + return -1; + } + rte_kvargs_free(kvlist); + + if (ret < 0) + return -1; + + if (first) + NT_Init(NTAPI_VERSION); + + if (astreamids.count) { + first = 0; + /* + * if a specific streamid specified then use that otherwise, + * port defaults to Tx port only + */ + if (rte_pmd_init_internals(name, rxqueues, txqueues, numa_node, + &astreamids, port, ð_dev) < 0) + return -1; + } else { + struct array_s astrids; + if (first) { + /* Delete all NTPL */ + sprintf(ntplStr, "Delete=All"); + if (DoNtpl(ntplStr) != 0) + return -1; + first = 0; + } + astrids.count = 1; + astrids.value[0] = stream_id; + if (rte_pmd_init_internals(name, rxqueues, txqueues, numa_node, + &astrids, port, ð_dev) < 0) + return -1; + + /* Assign the traffic */ + if (portend != (uint32_t)-1) { + sprintf(ntplStr, + "Assign[streamid=(%d..%d);Descriptor=NT]" + "=port==(%d..%d)", + stream_id, + (stream_id + rxqueues - 1), port, portend); + + } else { + sprintf(ntplStr, + "Assign[streamid=(%d..%d);Descriptor=NT]" + "=port==%d", + stream_id, + (stream_id + rxqueues - 1), port); + } + if (DoNtpl(ntplStr) != 0) + return -1; + + if (hash != (uint32_t)-1) { + switch (hash) { + default: + case 1: + DoNtpl("HashMode=HashRoundRobin"); + break; + case 2: + DoNtpl("HashMode=Hash2TupleSorted"); + break; + case 3: + DoNtpl("HashMode=Hash5TupleSorted"); + break; + } + } + + stream_id += rxqueues; + } + + eth_dev->rx_pkt_burst = eth_ntnic_rx; + eth_dev->tx_pkt_burst = eth_ntnic_tx_ringbuffer; + + return 0; +} + + + +static struct rte_driver pmd_ntnic_drv = { + .type = PMD_VDEV, + .init = rte_pmd_ntnic_devinit, +}; + +PMD_REGISTER_DRIVER(pmd_ntnic_drv, eth_ntnic); +DRIVER_REGISTER_PARAM_STRING(eth_ntnic, + "port=<int> " + "rxqs=<int>" + "txqs=<int>" + "hash=<int>" + "streamids=<int..int>"); + diff --git a/drivers/net/ntnic/rte_pmd_ntnic_version.map b/drivers/net/ntnic/rte_pmd_ntnic_version.map new file mode 100644 index 0000000..ef35398 --- /dev/null +++ b/drivers/net/ntnic/rte_pmd_ntnic_version.map @@ -0,0 +1,4 @@ +DPDK_2.0 { + + local: *; +}; diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 1a0095b..1fa614e 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -120,6 +120,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += -lrte_pmd_mpipe -lgxio _LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += -lrte_pmd_nfp -lm _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += -lrte_pmd_null _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += -lrte_pmd_pcap -lpcap + _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD) += -lrte_pmd_qede -lz _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING) += -lrte_pmd_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 @@ -143,6 +144,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -lrte_pmd_kasumi _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi endif # CONFIG_RTE_LIBRTE_CRYPTODEV +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += -lrte_pmd_ntnic -L$(NAPATECH3_PATH)/lib -lntapi -lntos endif # !CONFIG_RTE_BUILD_SHARED_LIBS _LDLIBS-y += --no-whole-archive @@ -163,6 +165,11 @@ endif _LDLIBS-$(CONFIG_RTE_PORT_PCAP) += -lpcap endif # !CONFIG_RTE_BUILD_SHARED_LIBS +ifeq ($(CONFIG_RTE_LIBRTE_PMD_NTNIC),y) +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += -L$(NAPATECH3_PATH)/lib -lntapi -lntos +endif + + _LDLIBS-y += $(EXECENV_LDLIBS) LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS) -- 2.9.0 Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v2] ntnic: add PMD driver 2016-09-08 11:14 ` [dpdk-dev] [PATCH v2] " Finn Christensen @ 2016-09-08 13:49 ` Neil Horman 2016-09-08 14:22 ` Finn Christensen 2016-09-09 12:48 ` [dpdk-dev] [PATCH v3] " Finn Christensen 1 sibling, 1 reply; 22+ messages in thread From: Neil Horman @ 2016-09-08 13:49 UTC (permalink / raw) To: Finn Christensen; +Cc: dev, thomas.monjalon, stephen On Thu, Sep 08, 2016 at 11:14:24AM +0000, Finn Christensen wrote: > This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. > > This patch adds support for Napatech NICs to DPDK. This is the > initial implementation. > > Signed-off-by: Finn Christensen <fc@napatech.com> > --- > v2: > * Added information how to build the PMD without NIC > Board Support Package > * Fixed some formatting issues > --- > MAINTAINERS | 5 + > config/common_base | 6 + > doc/guides/nics/features/ntnic.ini | 15 + > doc/guides/nics/index.rst | 1 + > doc/guides/nics/ntnic.rst | 145 +++++ > drivers/net/Makefile | 1 + > drivers/net/ntnic/Makefile | 65 ++ > drivers/net/ntnic/rte_eth_ntnic.c | 975 ++++++++++++++++++++++++++++ > drivers/net/ntnic/rte_pmd_ntnic_version.map | 4 + > mk/rte.app.mk | 7 + > 10 files changed, 1224 insertions(+) > create mode 100644 doc/guides/nics/features/ntnic.ini > create mode 100644 doc/guides/nics/ntnic.rst > create mode 100644 drivers/net/ntnic/Makefile > create mode 100644 drivers/net/ntnic/rte_eth_ntnic.c > create mode 100644 drivers/net/ntnic/rte_pmd_ntnic_version.map > > diff --git a/MAINTAINERS b/MAINTAINERS > index bc9aa02..d3e5f56 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -356,6 +356,11 @@ M: Sony Chacko <sony.chacko@qlogic.com> > F: drivers/net/qede/ > F: doc/guides/nics/qede.rst > > +Napatech ntnic > +M: Finn Christensen <fc@napatech.com> > +F: drivers/net/ntnic/ > +F: doc/guides/nics/ntnic.rst > + > RedHat virtio > M: Huawei Xie <huawei.xie@intel.com> > M: Yuanhan Liu <yuanhan.liu@linux.intel.com> > diff --git a/config/common_base b/config/common_base > index 7830535..4a3b2b8 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -309,6 +309,12 @@ CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16 > CONFIG_RTE_LIBRTE_PMD_PCAP=n > > # > +# Compile software PMD backed by NTNIC files > +# > +CONFIG_RTE_LIBRTE_PMD_NTNIC=n > + > + > +# > # Compile link bonding PMD library > # > CONFIG_RTE_LIBRTE_PMD_BOND=y > diff --git a/doc/guides/nics/features/ntnic.ini b/doc/guides/nics/features/ntnic.ini > new file mode 100644 > index 0000000..b6cfc5a > --- /dev/null > +++ b/doc/guides/nics/features/ntnic.ini > @@ -0,0 +1,15 @@ > +; > +; Supported features of the 'ntnic' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Link status = Y > +Queue start/stop = Y > +Promiscuous mode = Y > +Jumbo frame = Y > +Basic stats = Y > +Multiprocess aware = Y > +x86-32 = Y > +x86-64 = Y > +Usage doc = Y > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > index 92d56a5..5c4205c 100644 > --- a/doc/guides/nics/index.rst > +++ b/doc/guides/nics/index.rst > @@ -52,6 +52,7 @@ Network Interface Controller Drivers > qede > szedata2 > thunderx > + ntnic > virtio > vhost > vmxnet3 > diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst > new file mode 100644 > index 0000000..8512d02 > --- /dev/null > +++ b/doc/guides/nics/ntnic.rst > @@ -0,0 +1,145 @@ > +.. BSD LICENSE > + Copyright (c) 2016 Napatech A/S > + All rights reserved. > + > + Redistribution and use in source and binary forms, with or without > + modification, are permitted provided that the following conditions > + are met: > + > + * Redistributions of source code must retain the above copyright > + notice, this list of conditions and the following disclaimer. > + * Redistributions in binary form must reproduce the above copyright > + notice, this list of conditions and the following disclaimer in > + the documentation and/or other materials provided with the > + distribution. > + * Neither the name of Napatech nor the names of its > + contributors may be used to endorse or promote products derived > + from this software without specific prior written permission. > + > + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +NTNIC Poll Mode Driver > +====================== > + > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements support > +for **Napatech NIC** 40/50 Gbps adapters. > +This PMD is implemented as a pure software virtual device and must be created > +by using the EAL --vdev=parameter (parameters are explained i detail later). > +It runs on top of the Napatech NFV NIC Board Support Package that must be > +installed and started. > +If no Napatech NIC is available, you can download the Napatech NTAPI library to > +build against. > + > +Supported Features > +------------------ > + > +- RSS (Receive Side Scaling) > +- TSS (Transmit Side Scaling) > +- Promiscuous mode > +- Basic statistics > + > + > +Prerequisites > +------------- > + > +Requires Napatech NIC and Napatech NIC Board Support Package installed and > +running in version **0.3.0** or higher. > +This includes external libraries and kernel driver for resources > +allocations and initialization. > +If build only is required, download the Napatech NTAPI to build against. > + > +Pre-Installation Configuration > +------------------------------ > + > +Environment variables > +~~~~~~~~~~~~~~~~~~~~~ > + > +In order to compile the Napatech NIC PMD, user must: > + > +* Export the environment variable NAPATECH3_PATH with the path where > + the Napatech Board Support Package libs was installed (default location is > + /opt/napatech3). > + Or if no Napatech NIC available: > + Download the Napatech NTAPI libs from Github: > + `Napatech NTAPI <https://github.com/napa-tech/ntapi>`_. > + NAK Most of the code you provide in this patch looks fine, but you require that it be linked with the ntapi and ntos libraries. You provide those above in your external git tree...in binary only form. Thats not ok. If you want this to be in any way supportable, or usable on any platforms DPDK supports besides the x86_64 build you've provided, then you need to make the libraries in that git tree available in source form Neil ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v2] ntnic: add PMD driver 2016-09-08 13:49 ` Neil Horman @ 2016-09-08 14:22 ` Finn Christensen 0 siblings, 0 replies; 22+ messages in thread From: Finn Christensen @ 2016-09-08 14:22 UTC (permalink / raw) To: Neil Horman; +Cc: dev, thomas.monjalon, stephen > -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: 8. september 2016 15:50 > To: Finn Christensen <fc@napatech.com> > Cc: dev@dpdk.org; thomas.monjalon@6wind.com; > stephen@networkplumber.org > Subject: Re: [dpdk-dev] [PATCH v2] ntnic: add PMD driver > > On Thu, Sep 08, 2016 at 11:14:24AM +0000, Finn Christensen wrote: > > This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. > > > > This patch adds support for Napatech NICs to DPDK. This is the initial > > implementation. > > > > Signed-off-by: Finn Christensen <fc@napatech.com> > > --- > > v2: > > * Added information how to build the PMD without NIC > > Board Support Package > > * Fixed some formatting issues > > --- > > MAINTAINERS | 5 + > > config/common_base | 6 + > > doc/guides/nics/features/ntnic.ini | 15 + > > doc/guides/nics/index.rst | 1 + > > doc/guides/nics/ntnic.rst | 145 +++++ > > drivers/net/Makefile | 1 + > > drivers/net/ntnic/Makefile | 65 ++ > > drivers/net/ntnic/rte_eth_ntnic.c | 975 > ++++++++++++++++++++++++++++ > > drivers/net/ntnic/rte_pmd_ntnic_version.map | 4 + > > mk/rte.app.mk | 7 + > > 10 files changed, 1224 insertions(+) > > create mode 100644 doc/guides/nics/features/ntnic.ini > > create mode 100644 doc/guides/nics/ntnic.rst create mode 100644 > > drivers/net/ntnic/Makefile create mode 100644 > > drivers/net/ntnic/rte_eth_ntnic.c create mode 100644 > > drivers/net/ntnic/rte_pmd_ntnic_version.map > > > > diff --git a/MAINTAINERS b/MAINTAINERS index bc9aa02..d3e5f56 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -356,6 +356,11 @@ M: Sony Chacko <sony.chacko@qlogic.com> > > F: drivers/net/qede/ > > F: doc/guides/nics/qede.rst > > > > +Napatech ntnic > > +M: Finn Christensen <fc@napatech.com> > > +F: drivers/net/ntnic/ > > +F: doc/guides/nics/ntnic.rst > > + > > RedHat virtio > > M: Huawei Xie <huawei.xie@intel.com> > > M: Yuanhan Liu <yuanhan.liu@linux.intel.com> diff --git > > a/config/common_base b/config/common_base index 7830535..4a3b2b8 > > 100644 > > --- a/config/common_base > > +++ b/config/common_base > > @@ -309,6 +309,12 @@ CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16 > > CONFIG_RTE_LIBRTE_PMD_PCAP=n > > > > # > > +# Compile software PMD backed by NTNIC files # > > +CONFIG_RTE_LIBRTE_PMD_NTNIC=n > > + > > + > > +# > > # Compile link bonding PMD library > > # > > CONFIG_RTE_LIBRTE_PMD_BOND=y > > diff --git a/doc/guides/nics/features/ntnic.ini > > b/doc/guides/nics/features/ntnic.ini > > new file mode 100644 > > index 0000000..b6cfc5a > > --- /dev/null > > +++ b/doc/guides/nics/features/ntnic.ini > > @@ -0,0 +1,15 @@ > > +; > > +; Supported features of the 'ntnic' network poll mode driver. > > +; > > +; Refer to default.ini for the full list of available PMD features. > > +; > > +[Features] > > +Link status = Y > > +Queue start/stop = Y > > +Promiscuous mode = Y > > +Jumbo frame = Y > > +Basic stats = Y > > +Multiprocess aware = Y > > +x86-32 = Y > > +x86-64 = Y > > +Usage doc = Y > > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > > index 92d56a5..5c4205c 100644 > > --- a/doc/guides/nics/index.rst > > +++ b/doc/guides/nics/index.rst > > @@ -52,6 +52,7 @@ Network Interface Controller Drivers > > qede > > szedata2 > > thunderx > > + ntnic > > virtio > > vhost > > vmxnet3 > > diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst new > > file mode 100644 index 0000000..8512d02 > > --- /dev/null > > +++ b/doc/guides/nics/ntnic.rst > > @@ -0,0 +1,145 @@ > > +.. BSD LICENSE > > + Copyright (c) 2016 Napatech A/S > > + All rights reserved. > > + > > + Redistribution and use in source and binary forms, with or without > > + modification, are permitted provided that the following conditions > > + are met: > > + > > + * Redistributions of source code must retain the above copyright > > + notice, this list of conditions and the following disclaimer. > > + * Redistributions in binary form must reproduce the above copyright > > + notice, this list of conditions and the following disclaimer in > > + the documentation and/or other materials provided with the > > + distribution. > > + * Neither the name of Napatech nor the names of its > > + contributors may be used to endorse or promote products derived > > + from this software without specific prior written permission. > > + > > + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > > + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > NOT > > + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > FITNESS FOR > > + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT > > + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, > > + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, > BUT NOT > > + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS > OF USE, > > + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED > AND ON ANY > > + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > TORT > > + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE > > + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > > + > > +NTNIC Poll Mode Driver > > +====================== > > + > > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements > > +support for **Napatech NIC** 40/50 Gbps adapters. > > +This PMD is implemented as a pure software virtual device and must be > > +created by using the EAL --vdev=parameter (parameters are explained i > detail later). > > +It runs on top of the Napatech NFV NIC Board Support Package that > > +must be installed and started. > > +If no Napatech NIC is available, you can download the Napatech NTAPI > > +library to build against. > > + > > +Supported Features > > +------------------ > > + > > +- RSS (Receive Side Scaling) > > +- TSS (Transmit Side Scaling) > > +- Promiscuous mode > > +- Basic statistics > > + > > + > > +Prerequisites > > +------------- > > + > > +Requires Napatech NIC and Napatech NIC Board Support Package > > +installed and running in version **0.3.0** or higher. > > +This includes external libraries and kernel driver for resources > > +allocations and initialization. > > +If build only is required, download the Napatech NTAPI to build against. > > + > > +Pre-Installation Configuration > > +------------------------------ > > + > > +Environment variables > > +~~~~~~~~~~~~~~~~~~~~~ > > + > > +In order to compile the Napatech NIC PMD, user must: > > + > > +* Export the environment variable NAPATECH3_PATH with the path > where > > + the Napatech Board Support Package libs was installed (default > > +location is > > + /opt/napatech3). > > + Or if no Napatech NIC available: > > + Download the Napatech NTAPI libs from Github: > > + `Napatech NTAPI <https://github.com/napa-tech/ntapi>`_. > > + > NAK > > Most of the code you provide in this patch looks fine, but you require that it > be linked with the ntapi and ntos libraries. You provide those above in your > external git tree...in binary only form. Thats not ok. If you want this to be in > any way supportable, or usable on any platforms DPDK supports besides the > x86_64 build you've provided, then you need to make the libraries in that git > tree available in source form > > Neil No, our NIC is not supported on anything else than x86_64 platform. It would not seem to make sense to compile it against anything else than what we support. We assumed that the supported platforms was to be decribed in the doc/guides/nics/features/<nic>.ini file, and that is what we have done. Many of the other supported nics in the DPDK does also only support either x86 platform, or only a few others. Even though we do not support other platforms, do we need to be able to compile The ntnic PMD on other than x86_64 platforms? Would a dynamic linking in the PMD module be preferred or a possible solution? Thanks! Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-08 11:14 ` [dpdk-dev] [PATCH v2] " Finn Christensen 2016-09-08 13:49 ` Neil Horman @ 2016-09-09 12:48 ` Finn Christensen 2016-09-09 13:51 ` Neil Horman 1 sibling, 1 reply; 22+ messages in thread From: Finn Christensen @ 2016-09-09 12:48 UTC (permalink / raw) To: dev, nhorman; +Cc: thomas.monjalon, stephen, Finn Christensen This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. This patch adds support for Napatech NICs to DPDK. This is the initial implementation. Signed-off-by: Finn Christensen <fc@napatech.com> --- v3: * Removed the need for binary libraries on build v2: * Added information how to build the PMD without NIC Board Support Package * Fixed some formatting issues --- MAINTAINERS | 5 + config/common_base | 6 + doc/guides/nics/features/ntnic.ini | 14 + doc/guides/nics/index.rst | 1 + doc/guides/nics/ntnic.rst | 145 ++++ drivers/net/Makefile | 1 + drivers/net/ntnic/Makefile | 66 ++ drivers/net/ntnic/rte_eth_ntnic.c | 1150 +++++++++++++++++++++++++++ drivers/net/ntnic/rte_pmd_ntnic_version.map | 4 + mk/rte.app.mk | 3 + 10 files changed, 1395 insertions(+) create mode 100644 doc/guides/nics/features/ntnic.ini create mode 100644 doc/guides/nics/ntnic.rst create mode 100644 drivers/net/ntnic/Makefile create mode 100644 drivers/net/ntnic/rte_eth_ntnic.c create mode 100644 drivers/net/ntnic/rte_pmd_ntnic_version.map diff --git a/MAINTAINERS b/MAINTAINERS index bc9aa02..d3e5f56 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -356,6 +356,11 @@ M: Sony Chacko <sony.chacko@qlogic.com> F: drivers/net/qede/ F: doc/guides/nics/qede.rst +Napatech ntnic +M: Finn Christensen <fc@napatech.com> +F: drivers/net/ntnic/ +F: doc/guides/nics/ntnic.rst + RedHat virtio M: Huawei Xie <huawei.xie@intel.com> M: Yuanhan Liu <yuanhan.liu@linux.intel.com> diff --git a/config/common_base b/config/common_base index 7830535..4a3b2b8 100644 --- a/config/common_base +++ b/config/common_base @@ -309,6 +309,12 @@ CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16 CONFIG_RTE_LIBRTE_PMD_PCAP=n # +# Compile software PMD backed by NTNIC files +# +CONFIG_RTE_LIBRTE_PMD_NTNIC=n + + +# # Compile link bonding PMD library # CONFIG_RTE_LIBRTE_PMD_BOND=y diff --git a/doc/guides/nics/features/ntnic.ini b/doc/guides/nics/features/ntnic.ini new file mode 100644 index 0000000..4237e6e --- /dev/null +++ b/doc/guides/nics/features/ntnic.ini @@ -0,0 +1,14 @@ +; +; Supported features of the 'ntnic' network poll mode driver. +; +; Refer to default.ini for the full list of available PMD features. +; +[Features] +Link status = Y +Queue start/stop = Y +Promiscuous mode = Y +Jumbo frame = Y +Basic stats = Y +Multiprocess aware = Y +x86-64 = Y +Usage doc = Y diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst index 92d56a5..5c4205c 100644 --- a/doc/guides/nics/index.rst +++ b/doc/guides/nics/index.rst @@ -52,6 +52,7 @@ Network Interface Controller Drivers qede szedata2 thunderx + ntnic virtio vhost vmxnet3 diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst new file mode 100644 index 0000000..f9398db --- /dev/null +++ b/doc/guides/nics/ntnic.rst @@ -0,0 +1,145 @@ +.. BSD LICENSE + Copyright (c) 2016 Napatech A/S + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Napatech nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +NTNIC Poll Mode Driver +====================== + +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements support +for **Napatech NIC** 40/50 Gbps adapters. +This PMD is implemented as a pure software virtual device and must be created +by using the EAL --vdev=parameter (parameters are explained i detail later). +It runs on top of the Napatech NFV NIC Board Support Package that must be +installed and started. +If no Napatech NIC is available, you can download the Napatech NTAPI includes +to build with. + +Supported Features +------------------ + +- RSS (Receive Side Scaling) +- TSS (Transmit Side Scaling) +- Promiscuous mode +- Basic statistics + + +Prerequisites +------------- + +Requires Napatech NIC and Napatech NIC Board Support Package installed and +running in version **0.3.0** or higher. +This includes external libraries and kernel driver for resources +allocations and initialization. +If build only is required, download the Napatech NTAPI to build against. + +Pre-Installation Configuration +------------------------------ + +Environment variables +~~~~~~~~~~~~~~~~~~~~~ + +In order to compile the Napatech NIC PMD, user must: + +* Export the environment variable NAPATECH3_PATH with the path where + the Napatech Board Support Package was installed (default location is + /opt/napatech3). + Or if no Napatech NIC available: + Download the Napatech NTAPI include files from Github: + `Napatech NTAPI <https://github.com/napa-tech/ntapi>`_. + + +Config File Options +~~~~~~~~~~~~~~~~~~~ + +- ``CONFIG_RTE_LIBRTE_PMD_NTNIC`` (default **n**) + +Using the NTNIC PMD from a DPDK application using EAL vdev parameter +-------------------------------------------------------------------- + +Napatech NIC PMD VDEV parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Napatech NIC PMD vdev has the following command line parameters to enable +its features. + +* port + Configures which NT port the vdev should use. + + --vdev eth_ntnic0,port=4,rxqs=1,txqs=1 + + This will create a DPDK port0 using NT port 4 + +* rxqs + Control how many receive queues a vdev should have. The traffic from a vdev + will be load balanced to this amount of queues and each queue can be handled + by its own lcore. + Note: The ntservice.ini HostBuffersRx must be configured to the sum of all + receive queues across all vdevs. + + --vdev eth_ntnic0,port=0,rxqs=8,txqs=1 + + Traffic from NT port 0 will be split across + 8 queues meaning that 8 lcores can be + opened on DPDK port 0 and each will be + their individual traffic. + +* hash + Control which load balancing scheme should be used when running with multiple + receive queues. + The possible values are: + 1 = HashRoundRobin + 2 = Hash2TupleSorted + 3 = Hash5TupleSorted + + --vdev eth_ntnic0,port=0,rxqs=4,hash=3,txqs=1 + + Create 4 queues from NT port 0 and + load balance the traffic using the + 5-tuple sorted algorithm. Each of the + 4 queues can be opened via DPDK port 0 + +* txqs + Control how many transmit queues a vdev should have. Each queue can be used + by a lcore and each queue is independent for other queues. + Note: The ntservice.ini HostBuffersTx must be configured to the sum of all + transmit queues across all vdevs. + + --vdev eth_ntnic0,port=0,txqs=32,rxqs=1 + + Enable 32 transmit queues on NT port 0 + + +Changes to Napatech driver configuration file ntservice.ini +----------------------------------------------------------- +Depending on the number of queues that is created, more hostbuffers may be +needed pre-allocated by the Napatech NIC Driver. This is controlled in the +ntservice.ini file (default locations is /opt/napatech3/config). +The Napatech NIC Driver must be re-started when this configuration file is +changed. + diff --git a/drivers/net/Makefile b/drivers/net/Makefile index bc93230..aff29e9 100644 --- a/drivers/net/Makefile +++ b/drivers/net/Makefile @@ -55,6 +55,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3 DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt +DIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += ntnic ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) DIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += vhost diff --git a/drivers/net/ntnic/Makefile b/drivers/net/ntnic/Makefile new file mode 100644 index 0000000..d816c56 --- /dev/null +++ b/drivers/net/ntnic/Makefile @@ -0,0 +1,66 @@ +# BSD LICENSE +# +# Copyright(c) 2016 Napatech A/S. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Napatech nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +include $(RTE_SDK)/mk/rte.vars.mk + +# +# library name +# +LIB = librte_pmd_ntnic.a + +CFLAGS += -O3 -g +CFLAGS += $(WERROR_FLAGS) +CFLAGS += -I$(NAPATECH3_PATH)/include +CFLAGS += -DNAPATECH3_LIB_PATH=\"$(NAPATECH3_PATH)/lib\" +LDLIBS += -ldl + +EXPORT_MAP := rte_pmd_ntnic_version.map + +LIBABIVER := 1 + +# +# all source are stored in SRCS-y +# +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += rte_eth_ntnic.c + +# +# Export include files +# +SYMLINK-y-include += + +# this lib depends upon: +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_eal +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mbuf +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mempool +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_ether +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_kvargs + +include $(RTE_SDK)/mk/rte.lib.mk diff --git a/drivers/net/ntnic/rte_eth_ntnic.c b/drivers/net/ntnic/rte_eth_ntnic.c new file mode 100644 index 0000000..f83e94d --- /dev/null +++ b/drivers/net/ntnic/rte_eth_ntnic.c @@ -0,0 +1,1150 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2016 Napatech A/S. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Napatech nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + */ + +#include <time.h> +#include <dlfcn.h> +#include <rte_mbuf.h> +#include <rte_ethdev.h> +#include <rte_malloc.h> +#include <rte_memcpy.h> +#include <rte_string_fns.h> +#include <rte_cycles.h> +#include <rte_kvargs.h> +#include <rte_dev.h> +#include <net/if.h> +#include <nt.h> + +#define ETH_NTNIC_PORT_ARG "port" +#define ETH_NTNIC_PORTEND_ARG "portend" +#define ETH_NTNIC_RXQUEUES_ARG "rxqs" +#define ETH_NTNIC_TXQUEUES_ARG "txqs" +#define ETH_NTNIC_STREAMID_ARG "streamid" +#define ETH_NTNIC_HASH_ARG "hash" + +#define HW_MAX_PKT_LEN 10000 +#define HW_MTU (HW_MAX_PKT_LEN - ETHER_HDR_LEN - ETHER_CRC_LEN) + +#define MAX_RX_QUEUES 64 +#define MAX_TX_QUEUES 64 +#define MAX_NTNIC_PORTS 32 + +static void *_libnt; + +/* NTAPI library functions */ +int (*_NT_Init)(uint32_t); +int (*_NT_NetRxOpen)(NtNetStreamRx_t *, + const char *, enum NtNetInterface_e, uint32_t, int); +int (*_NT_NetRxGet)(NtNetStreamRx_t, NtNetBuf_t *, int); +int (*_NT_NetRxRelease)(NtNetStreamRx_t, NtNetBuf_t); +int (*_NT_NetRxClose)(NtNetStreamRx_t); +char *(*_NT_ExplainError)(int, char *, uint32_t); +int (*_NT_NetTxOpen)(NtNetStreamTx_t *, + const char *, uint64_t, uint32_t, uint32_t); +int (*_NT_NetTxRingbufferInit)(NtNetStreamTx_t, uint32_t); +int (*_NT_NetTxRingbufferTransmitPacket)(NtNetStreamTx_t, + uint8_t *, uint16_t); +int (*_NT_NetTxRingbufferFlush)(NtNetStreamTx_t); +void (*_NT_NetTxRingbufferDone)(NtNetStreamTx_t); +int (*_NT_NetTxClose)(NtNetStreamTx_t); +int (*_NT_InfoOpen)(NtInfoStream_t *, const char *); +int (*_NT_InfoRead)(NtInfoStream_t, NtInfo_t *); +int (*_NT_InfoClose)(NtInfoStream_t); +int (*_NT_ConfigOpen)(NtConfigStream_t *, const char *); +int (*_NT_ConfigClose)(NtConfigStream_t); +int (*_NT_NTPL)(NtConfigStream_t, const char *, + NtNtplInfo_t *, uint32_t); + +static char ebuf[1024]; + +struct array_s { + uint32_t value[MAX_RX_QUEUES]; + int count; +}; + +static volatile uint16_t port_locks[MAX_NTNIC_PORTS]; + +struct ntnic_rx_queue { + NtNetStreamRx_t net_rx[MAX_RX_QUEUES]; /* NT Rx streams */ + int rx_idx; /* Current Rx stream index */ + struct rte_mempool *mb_pool; /* mbuf memory pool */ + uint16_t buf_size; /* Size of data area in mbuf */ + NtNetBuf_t seg; /* The current NT data segment */ + struct NtNetBuf_s pkt; /* The current packet */ + volatile unsigned long rx_pkts; /* Rx packet statistics */ + volatile unsigned long err_pkts; /* Rx error packet statistics */ + struct array_s astreamids; /* NT streams to read from */ + int enabled; /* Enabling/disabling of this queue */ +}; + +struct ntnic_tx_queue { + NtNetStreamTx_t net_tx; /* NT Tx stream */ + volatile unsigned long tx_pkts; /* Tx packet statistics */ + volatile unsigned long err_pkts; /* Tx error packet stat */ + volatile uint16_t *plock; /* Per port transmit atomic lock */ + uint32_t port; /* Tx port for this queue */ + int enabled; /* Enabling/disabling of this queue */ + int fcs_add; /* If needs added room for FCS */ +}; + +struct pmd_internals { + struct ntnic_rx_queue rxq[MAX_RX_QUEUES]; /* Array of Rx queues */ + struct ntnic_tx_queue txq[MAX_TX_QUEUES]; /* Array of Tx queues */ + int if_index; /* Itf index always 0 - no bonding */ + unsigned int nb_rx_queues; /* Number of Rx queues configured */ + unsigned int nb_tx_queues; /* Number of Tx queues configured */ + int MAC_50G; /* True if 50G ports */ +}; + +static const char *valid_arguments[] = { + ETH_NTNIC_PORT_ARG, + ETH_NTNIC_PORTEND_ARG, + ETH_NTNIC_RXQUEUES_ARG, + ETH_NTNIC_TXQUEUES_ARG, + ETH_NTNIC_STREAMID_ARG, + ETH_NTNIC_HASH_ARG, + NULL +}; + +static struct ether_addr eth_addr[MAX_NTNIC_PORTS]; +static const char *drivername = "NTNIC PMD"; + +static int +eth_ntnic_rx_jumbo(struct rte_mempool *mb_pool, + struct rte_mbuf *mbuf, + const u_char *data, + uint16_t data_len) +{ + struct rte_mbuf *m = mbuf; + + /* Copy the first segment. */ + uint16_t len = rte_pktmbuf_tailroom(mbuf); + + rte_memcpy(rte_pktmbuf_append(mbuf, len), data, len); + data_len -= len; + data += len; + + while (data_len > 0) { + /* Allocate next mbuf and point to that. */ + m->next = rte_pktmbuf_alloc(mb_pool); + + if (unlikely(!m->next)) + return -1; + + m = m->next; + + /* Headroom is not needed in chained mbufs. */ + rte_pktmbuf_prepend(m, rte_pktmbuf_headroom(m)); + m->pkt_len = 0; + m->data_len = 0; + + /* Copy next segment. */ + len = RTE_MIN(rte_pktmbuf_tailroom(m), data_len); + rte_memcpy(rte_pktmbuf_append(m, len), data, len); + + mbuf->nb_segs++; + data_len -= len; + data += len; + } + + return mbuf->nb_segs; +} + + + +static uint16_t +eth_ntnic_rx(void *queue, + struct rte_mbuf **bufs, + uint16_t nb_pkts) +{ + unsigned int i; + struct rte_mbuf *mbuf; + struct ntnic_rx_queue *rx_q = queue; + uint16_t num_rx = 0; + uint16_t data_len; + + if (unlikely(rx_q->net_rx[rx_q->rx_idx] == NULL || nb_pkts == 0)) + return 0; + + /* Do we have any data segment */ + if (rx_q->seg == NULL) { + /* Next stream - if multiple streams are combined */ + rx_q->rx_idx = + (rx_q->rx_idx < + (rx_q->astreamids.count - 1)) ? rx_q->rx_idx + 1 + : 0; + + if ((*_NT_NetRxGet)(rx_q->net_rx[rx_q->rx_idx], + &rx_q->seg, 0) != NT_SUCCESS) { + if (rx_q->seg != NULL) { + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], + rx_q->seg); + rx_q->seg = NULL; + } + return 0; + } + if (NT_NET_GET_SEGMENT_LENGTH(rx_q->seg)) { + /* Build a packet structure */ + _nt_net_build_pkt_netbuf(rx_q->seg, &rx_q->pkt); + } else { + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], + rx_q->seg); + rx_q->seg = NULL; + return 0; + } + } + + if (rte_mempool_get_bulk(rx_q->mb_pool, (void **)bufs, nb_pkts) != 0) + return 0; + + for (i = 0; i < nb_pkts; i++) { + mbuf = bufs[i]; + rte_mbuf_refcnt_set(mbuf, 1); + rte_pktmbuf_reset(mbuf); + + /* HW slicing not supported */ + data_len = (uint16_t)NT_NET_GET_PKT_WIRE_LENGTH((&rx_q->pkt)) + - 4; + if (data_len <= rx_q->buf_size) { + /* Packet will fit in the mbuf, go ahead and copy */ + mbuf->pkt_len = mbuf->data_len; + data_len = mbuf->data_len; + rte_memcpy((u_char *)mbuf->buf_addr + + RTE_PKTMBUF_HEADROOM, + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), + mbuf->data_len); + } else { + /* Try read jumbo frame into multi mbufs. */ + if (unlikely(eth_ntnic_rx_jumbo(rx_q->mb_pool, + mbuf, + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), + data_len) == -1)) + break; + } + + mbuf->port = NT_NET_GET_PKT_RXPORT((&rx_q->pkt)); + num_rx++; + + /* Get the next packet if any */ + if (_nt_net_get_next_packet(rx_q->seg, + NT_NET_GET_SEGMENT_LENGTH(rx_q->seg), + &rx_q->pkt) == 0) { + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], + rx_q->seg); + rx_q->seg = NULL; + break; + } + + rx_q->rx_pkts++; + } + + if (num_rx < nb_pkts) { + rte_mempool_put_bulk(rx_q->mb_pool, + (void * const *)(bufs + num_rx), + nb_pkts - num_rx); + } + return num_rx; +} + + + + + +/* Callback to handle sending packets through a NT NIC. */ +static uint16_t +eth_ntnic_tx_ringbuffer(void *queue, + struct rte_mbuf **bufs, + uint16_t nb_pkts) +{ + unsigned int i; + struct ntnic_tx_queue *tx_q = queue; + int retval; + uint16_t old; + uint16_t new_flag = 0; + + + if (unlikely(tx_q == NULL || tx_q->net_tx == NULL || nb_pkts == 0)) + return 0; + + + do { + old = 0; + new_flag = 1; + retval = rte_atomic16_cmpset(tx_q->plock, old, new_flag); + } while (unlikely(retval == 0)); + + for (i = 0; i < nb_pkts; i++) { + /* Not allowed to TX less than 64 byte packets */ + if (bufs[i]->pkt_len < 60) + bufs[i]->pkt_len = 60; + + /* transmit a single packet */ + (*_NT_NetTxRingbufferTransmitPacket)(tx_q->net_tx, + rte_pktmbuf_mtod(bufs[i], uint8_t *), + bufs[i]->pkt_len + 4 - tx_q->fcs_add); + rte_pktmbuf_free(bufs[i]); + } + tx_q->tx_pkts += nb_pkts; + + *tx_q->plock = 0; + return nb_pkts; +} + + + +static int +eth_dev_start(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = internals->rxq; + struct ntnic_tx_queue *tx_q = internals->txq; + uint queue; + int status, idx; + + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + + if ((internals->nb_tx_queues | internals->nb_rx_queues) == 0) + return -1; + + for (queue = 0; queue < internals->nb_rx_queues; queue++) { + if (rx_q[queue].enabled) { + char str[128], val[10]; + str[0] = 0; + /* build list of streamids to log */ + for (idx = 0; idx < + rx_q[queue].astreamids.count; idx++) { + if (idx) + sprintf(val, ",%d", + rx_q[queue].astreamids.value[idx]); + else + sprintf(val, "%i", + rx_q[queue].astreamids.value[idx]); + strcat(str, val); + } + + for (idx = 0; idx < rx_q[queue].astreamids.count; + idx++) { + status = + (*_NT_NetRxOpen)(&rx_q[queue].net_rx[idx], + "DPDK", + NT_NET_INTERFACE_SEGMENT, + rx_q[queue].astreamids.value[idx], -1); + if (status != NT_SUCCESS) { + /* try packet interface instead */ + (*_NT_ExplainError)(status, + ebuf, + sizeof(ebuf)); + RTE_LOG(ERR, PMD, + "NT_NetRxOpen() failed: %s\n", + ebuf); + return -1; + } + } + } + } + + for (queue = 0; queue < internals->nb_tx_queues; queue++) { + if (tx_q[queue].enabled) { + RTE_LOG(INFO, PMD, "NTNIC: NT_NetTxOpen(%d)\n", + tx_q[queue].port); + status = (*_NT_NetTxOpen)(&tx_q[queue].net_tx, "DPDK", + 1 << tx_q[queue].port, + dev->pci_dev->numa_node, 0); + if (status != NT_SUCCESS) { + (*_NT_ExplainError)(status, ebuf, + sizeof(ebuf)); + RTE_LOG(INFO, PMD, + "NT_NetTxOpen(0x%X, %d, 0) failed: %s\n", + 1 << tx_q[queue].port, + dev->pci_dev->numa_node, ebuf); + return -1; + } + + /* Init and associate RTD Tx Ring buf to Tx handler */ + status = (*_NT_NetTxRingbufferInit)( + tx_q[queue].net_tx, + tx_q[queue].port); + if (status != NT_SUCCESS) { + (*_NT_ExplainError)(status, ebuf, + sizeof(ebuf)); + RTE_LOG(ERR, PMD, + "NT_NetRxOpen() failed: %s\n", + ebuf); + return -1; + } + } + tx_q[queue].plock = &port_locks[tx_q[queue].port]; + tx_q[queue].fcs_add = (internals->MAC_50G) ? 4 : 0; + } + + dev->data->dev_link.link_status = 1; + return 0; +} + + +static void +eth_dev_stop(struct rte_eth_dev *dev) +{ + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = internals->rxq; + struct ntnic_tx_queue *tx_q = internals->txq; + uint q; + int idx; + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + + for (q = 0; q < internals->nb_rx_queues; q++) { + if (rx_q[q].seg) { + (*_NT_NetRxRelease)(rx_q[q].net_rx[rx_q->rx_idx], + rx_q[q].seg); + rx_q[q].seg = NULL; + } + for (idx = 0; idx < rx_q[q].astreamids.count; idx++) { + if (rx_q[q].net_rx[idx]) + (void)(*_NT_NetRxClose)(rx_q[q].net_rx[idx]); + } + } + for (q = 0; q < internals->nb_tx_queues; q++) { + if (tx_q[q].net_tx) { + (*_NT_NetTxRingbufferDone)(tx_q[q].net_tx); + (void)(*_NT_NetTxClose)(tx_q[q].net_tx); + } + } + dev->data->dev_link.link_status = 0; +} + +static int +eth_dev_configure(struct rte_eth_dev *dev __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); + return 0; +} + +static void +eth_dev_info(struct rte_eth_dev *dev, + struct rte_eth_dev_info *dev_info) +{ + struct pmd_internals *internals = dev->data->dev_private; + dev_info->if_index = internals->if_index; + dev_info->driver_name = drivername; + dev_info->max_mac_addrs = 1; + dev_info->max_rx_pktlen = HW_MTU; + dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; + dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; + dev_info->min_rx_bufsize = 0; + dev_info->pci_dev = NULL; +} + +static void +eth_stats_get(struct rte_eth_dev *dev, + struct rte_eth_stats *igb_stats) +{ + unsigned int i; + unsigned long rx_total = 0; + unsigned long tx_total = 0; + unsigned long tx_err_total = 0; + const struct pmd_internals *internal = dev->data->dev_private; + + memset(igb_stats, 0, sizeof(*igb_stats)); + for (i = 0; + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal->nb_rx_queues; + i++) { + igb_stats->q_ipackets[i] = internal->rxq[i].rx_pkts; + rx_total += igb_stats->q_ipackets[i]; + } + for (i = 0; + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal->nb_tx_queues; + i++) { + igb_stats->q_opackets[i] = internal->txq[i].tx_pkts; + igb_stats->q_errors[i] = internal->txq[i].err_pkts; + tx_total += igb_stats->q_opackets[i]; + tx_err_total += igb_stats->q_errors[i]; + } + + igb_stats->ipackets = rx_total; + igb_stats->opackets = tx_total; + igb_stats->oerrors = tx_err_total; +} + +static void +eth_stats_reset(struct rte_eth_dev *dev) +{ + unsigned int i; + struct pmd_internals *internal = dev->data->dev_private; + + for (i = 0; i < internal->nb_rx_queues; i++) + internal->rxq[i].rx_pkts = 0; + for (i = 0; i < internal->nb_tx_queues; i++) { + internal->txq[i].tx_pkts = 0; + internal->txq[i].err_pkts = 0; + } +} + +static void +eth_dev_close(struct rte_eth_dev *dev __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); +} + +static void +eth_queue_release(void *q __rte_unused) +{ + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); +} + +static int +eth_link_update(struct rte_eth_dev *dev __rte_unused, + int wait_to_complete __rte_unused) +{ + return 0; +} + +static int +eth_rx_queue_setup(struct rte_eth_dev *dev, + uint16_t rx_queue_id, + uint16_t nb_rx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_rxconf *rx_conf __rte_unused, + struct rte_mempool *mb_pool) +{ + struct rte_pktmbuf_pool_private *mbp_priv; + struct pmd_internals *internals = dev->data->dev_private; + struct ntnic_rx_queue *rx_q = &internals->rxq[rx_queue_id]; + + RTE_LOG(INFO, PMD, "NTNIC RX queue setup\n"); + rx_q->mb_pool = mb_pool; + dev->data->rx_queues[rx_queue_id] = rx_q; + + mbp_priv = rte_mempool_get_priv(rx_q->mb_pool); + rx_q->buf_size = (uint16_t)(mbp_priv->mbuf_data_room_size - + RTE_PKTMBUF_HEADROOM); + rx_q->enabled = 1; + return 0; +} + +static int +eth_tx_queue_setup(struct rte_eth_dev *dev __rte_unused, + uint16_t tx_queue_id __rte_unused, + uint16_t nb_tx_desc __rte_unused, + unsigned int socket_id __rte_unused, + const struct rte_eth_txconf *tx_conf __rte_unused) +{ + struct pmd_internals *internals = dev->data->dev_private; + RTE_LOG(INFO, PMD, "NTNIC TX queue setup\n"); + dev->data->tx_queues[tx_queue_id] = &internals->txq[tx_queue_id]; + internals->txq[tx_queue_id].enabled = 1; + return 0; +} + +static int _dev_set_mtu(struct rte_eth_dev *dev __rte_unused, uint16_t mtu) +{ + if (mtu < 46 || mtu > HW_MTU) + return -EINVAL; + + return 0; +} + +static struct eth_dev_ops ops = { + .dev_start = eth_dev_start, + .dev_stop = eth_dev_stop, + .dev_close = eth_dev_close, + .mtu_set = _dev_set_mtu, + .dev_configure = eth_dev_configure, + .dev_infos_get = eth_dev_info, + .rx_queue_setup = eth_rx_queue_setup, + .tx_queue_setup = eth_tx_queue_setup, + .rx_queue_release = eth_queue_release, + .tx_queue_release = eth_queue_release, + .link_update = eth_link_update, + .stats_get = eth_stats_get, + .stats_reset = eth_stats_reset, +}; + +static int +rte_pmd_init_internals(const char *name, + const unsigned int nb_rx_queues, + const unsigned int nb_tx_queues, + const unsigned int numa_node, + struct array_s *pastreamids, + const uint32_t port, + struct rte_eth_dev **eth_dev) +{ + struct pmd_internals *internals = NULL; + struct rte_eth_dev_data *data = NULL; + struct rte_pci_device *pci_dev = NULL; + uint i, status; + char err_buf[NT_ERRBUF_SIZE]; + NtInfoStream_t info; + NtInfo_t info_port; + struct rte_eth_link pmd_link; + assert(nb_rx_queues < MAX_RX_QUEUES); + assert(nb_tx_queues < MAX_TX_QUEUES); + assert(port < MAX_NTNIC_PORTS); + + RTE_LOG(INFO, PMD, + "Creating ntnic-backend ethdev on numa socket %u\n", + numa_node); + + /* now do all data allocation - for eth_dev structure, dummy pci driver + * and internal (private) data + */ + data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node); + if (data == NULL) + goto error; + + pci_dev = rte_zmalloc_socket(name, sizeof(*pci_dev), 0, numa_node); + if (pci_dev == NULL) + goto error; + + internals = rte_zmalloc_socket(name, sizeof(*internals), 0, numa_node); + if (internals == NULL) + goto error; + + /* reserve an ethdev entry */ + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); + if (*eth_dev == NULL) + goto error; + + /* Open the information stream */ + status = (*_NT_InfoOpen)(&info, "DPDK Info stream"); + if (status != NT_SUCCESS) { + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); + RTE_LOG(ERR, PMD, + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", + status, err_buf); + return status; + } + + /* Find local port offset */ + info_port.cmd = NT_INFO_CMD_READ_PORT_V7; + info_port.u.port_v7.portNo = (uint8_t)(port); + status = (*_NT_InfoRead)(info, &info_port); + if (status != 0) { + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); + RTE_LOG(ERR, PMD, + "ERROR: NT_InfoRead failed. Code 0x%x = %s\n", + status, err_buf); + return status; + } + + /* check for 50G MAC */ + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9506 || + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9509) { + internals->MAC_50G = 1; + } else { + internals->MAC_50G = 0; + /* Check for valid FPGAs (NFV NICs) */ + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != + 9507 && + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != + 9510) { + RTE_LOG(ERR, PMD, + "ERROR: NTNIC PMD does not support the NT adapter" + "- port %i\n", + port); + return -1; + } + } + + internals->nb_rx_queues = nb_rx_queues; + internals->nb_tx_queues = nb_tx_queues; + for (i = 0; i < nb_rx_queues; i++) { + int idx; + for (idx = 0; idx < pastreamids->count; idx++) { + internals->rxq[i].astreamids.value[idx] = + pastreamids->value[idx] + i; + } + + internals->rxq[i].astreamids.count = pastreamids->count; + + internals->rxq[i].seg = NULL; + internals->rxq[i].enabled = 0; + } + + for (i = 0; i < nb_tx_queues; i++) { + if (port < info_port.u.port_v7.data.adapterInfo.portOffset) { + RTE_LOG(ERR, PMD, "Error wrong port specified\n"); + return -1; + } + internals->txq[i].port = port; + internals->txq[i].enabled = 0; + } + + switch (info_port.u.port_v7.data.speed) { + case NT_LINK_SPEED_UNKNOWN: + pmd_link.link_speed = ETH_SPEED_NUM_1G; + break; + case NT_LINK_SPEED_10M: + pmd_link.link_speed = ETH_SPEED_NUM_10M; + break; + case NT_LINK_SPEED_100M: + pmd_link.link_speed = ETH_SPEED_NUM_100M; + break; + case NT_LINK_SPEED_1G: + pmd_link.link_speed = ETH_SPEED_NUM_1G; + break; + case NT_LINK_SPEED_10G: + pmd_link.link_speed = ETH_SPEED_NUM_10G; + break; + case NT_LINK_SPEED_40G: + pmd_link.link_speed = ETH_SPEED_NUM_40G; + break; + case NT_LINK_SPEED_50G: + pmd_link.link_speed = ETH_SPEED_NUM_50G; + break; + case NT_LINK_SPEED_100G: + pmd_link.link_speed = ETH_SPEED_NUM_100G; + break; + } + + memcpy(ð_addr[port].addr_bytes, &info_port.u.port_v7.data.macAddress, + sizeof(eth_addr[port].addr_bytes)); + status = (*_NT_InfoClose)(info); + if (status != NT_SUCCESS) { + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); + RTE_LOG(ERR, PMD, + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", + status, err_buf); + return status; + } + + pmd_link.link_duplex = ETH_LINK_FULL_DUPLEX; + pmd_link.link_status = 0; + + internals->if_index = 0; + + pci_dev->numa_node = numa_node; + + data->dev_private = internals; + data->port_id = (*eth_dev)->data->port_id; + data->nb_rx_queues = (uint16_t)nb_rx_queues; + data->nb_tx_queues = (uint16_t)nb_tx_queues; + data->dev_link = pmd_link; + data->mac_addrs = ð_addr[port]; + data->numa_node = numa_node; + data->drv_name = drivername; + + (*eth_dev)->data = data; + (*eth_dev)->dev_ops = &ops; + (*eth_dev)->pci_dev = pci_dev; + + return 0; + +error: + if (data) + rte_free(data); + if (pci_dev) + rte_free(pci_dev); + if (internals) + rte_free(internals); + return -1; +} + +/* + * convert ascii to int + */ +static inline int +ascii_to_u32(const char *key __rte_unused, + const char *value, void *extra_args __rte_unused) +{ + *(uint32_t *)extra_args = atoi(value); + return 0; +} + +static inline int +ascii_to_u32_array(const char *key __rte_unused, + const char *value, void *extra_args) +{ + struct array_s *pastreams = (struct array_s *)extra_args; + int sval, eval; + if (sscanf(value, "%d..%d", &sval, &eval) == 2) { + int cnt; + for (cnt = sval; cnt <= eval; cnt++) + pastreams->value[pastreams->count++] = cnt; + } else { + pastreams->value[pastreams->count++] = atoi(value); + } + return 0; +} + + +static int DoNtpl(const char *ntplStr) +{ + NtConfigStream_t hCfgStream; + NtNtplInfo_t ntplInfo; + int status; + + status = (*_NT_ConfigOpen)(&hCfgStream, "capture"); + if (status != NT_SUCCESS) { + /* Get the status code as text */ + (*_NT_ExplainError)(status, ebuf, sizeof(ebuf) - 1); + fprintf(stderr, "NT_ConfigOpen() failed: %s\n", ebuf); + return -1; + } + + RTE_LOG(INFO, PMD, "NTPL : %s\n", ntplStr); + status = (*_NT_NTPL)(hCfgStream, ntplStr, &ntplInfo, + NT_NTPL_PARSER_VALIDATE_NORMAL); + if (status != NT_SUCCESS) { + /* Get the status code as text */ + (*_NT_ExplainError)(status, ebuf, sizeof(ebuf) - 1); + fprintf(stderr, "NT_NTPL() failed: %s\n", ebuf); + fprintf(stderr, ">>> NTPL errorcode: %X\n", + ntplInfo.u.errorData.errCode); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); + (*_NT_ConfigClose)(hCfgStream); + return -1; + } + (*_NT_ConfigClose)(hCfgStream); + return 0; +} + + +static int +_nt_lib_open(void) +{ + char path[128]; + strcpy(path, NAPATECH3_LIB_PATH); + strcat(path, "/libntapi.so"); + + /* Load the library */ + _libnt = dlopen(path, RTLD_NOW); + if (_libnt == NULL) { + /* Library does not exist. */ + fprintf(stderr, "Failed to find needed library : %s\n", path); + return -1; + } + _NT_Init = dlsym(_libnt, "NT_Init"); + if (_NT_Init == NULL) { + fprintf(stderr, "Failed to find \"NT_Init\" in %s\n", path); + return -1; + } + + _NT_ConfigOpen = dlsym(_libnt, "NT_ConfigOpen"); + if (_NT_ConfigOpen == NULL) { + fprintf(stderr, "Failed to find \"NT_ConfigOpen\" in %s\n", + path); + return -1; + } + _NT_ConfigClose = dlsym(_libnt, "NT_ConfigClose"); + if (_NT_ConfigClose == NULL) { + fprintf(stderr, "Failed to find \"NT_ConfigClose\" in %s\n", + path); + return -1; + } + _NT_NTPL = dlsym(_libnt, "NT_NTPL"); + if (_NT_NTPL == NULL) { + fprintf(stderr, "Failed to find \"NT_NTPL\" in %s\n", + path); + return -1; + } + + _NT_InfoOpen = dlsym(_libnt, "NT_InfoOpen"); + if (_NT_InfoOpen == NULL) { + fprintf(stderr, "Failed to find \"NT_InfoOpen\" in %s\n", + path); + return -1; + } + _NT_InfoRead = dlsym(_libnt, "NT_InfoRead"); + if (_NT_InfoRead == NULL) { + fprintf(stderr, "Failed to find \"NT_InfoRead\" in %s\n", + path); + return -1; + } + _NT_InfoClose = dlsym(_libnt, "NT_InfoClose"); + if (_NT_InfoClose == NULL) { + fprintf(stderr, "Failed to find \"NT_InfoClose\" in %s\n", + path); + return -1; + } + + _NT_ExplainError = dlsym(_libnt, "NT_ExplainError"); + if (_NT_ExplainError == NULL) { + fprintf(stderr, "Failed to find \"NT_ExplainError\" in %s\n", + path); + return -1; + } + _NT_NetTxOpen = dlsym(_libnt, "NT_NetTxOpen"); + if (_NT_NetTxOpen == NULL) { + fprintf(stderr, "Failed to find \"NT_NetTxOpen\" in %s\n", + path); + return -1; + } + _NT_NetTxRingbufferInit = dlsym(_libnt, "NT_NetTxRingbufferInit"); + if (_NT_NetTxRingbufferInit == NULL) { + fprintf(stderr, + "Failed to find \"NT_NetTxRingbufferInit\" in %s\n", + path); + return -1; + } + _NT_NetTxRingbufferFlush = dlsym(_libnt, "NT_NetTxRingbufferFlush"); + if (_NT_NetTxRingbufferFlush == NULL) { + fprintf(stderr, + "Failed to find \"NT_NetTxRingbufferFlush\" in %s\n", + path); + return -1; + } + _NT_NetTxRingbufferTransmitPacket = + dlsym(_libnt, "NT_NetTxRingbufferTransmitPacket"); + if (_NT_NetTxRingbufferTransmitPacket == NULL) { + fprintf(stderr, + "Failed to find \"NT_NetTxRingbufferTransmitPacket\"" + " in %s\n", path); + return -1; + } + _NT_NetTxRingbufferDone = dlsym(_libnt, "NT_NetTxRingbufferDone"); + if (_NT_NetTxRingbufferDone == NULL) { + fprintf(stderr, + "Failed to find \"NT_NetTxRingbufferDone\" in %s\n", + path); + return -1; + } + _NT_NetTxClose = dlsym(_libnt, "NT_NetTxClose"); + if (_NT_NetTxClose == NULL) { + fprintf(stderr, "Failed to find \"NT_NetTxClose\" in %s\n", + path); + return -1; + } + + _NT_NetRxOpen = dlsym(_libnt, "NT_NetRxOpen"); + if (_NT_NetRxOpen == NULL) { + fprintf(stderr, "Failed to find \"NT_NetRxOpen\" in %s\n", + path); + return -1; + } + _NT_NetRxGet = dlsym(_libnt, "NT_NetRxGet"); + if (_NT_NetRxGet == NULL) { + fprintf(stderr, "Failed to find \"NT_NetRxGet\" in %s\n", + path); + return -1; + } + _NT_NetRxRelease = dlsym(_libnt, "NT_NetRxRelease"); + if (_NT_NetRxRelease == NULL) { + fprintf(stderr, "Failed to find \"NT_NetRxRelease\" in %s\n", + path); + return -1; + } + _NT_NetRxClose = dlsym(_libnt, "NT_NetRxClose"); + if (_NT_NetRxClose == NULL) { + fprintf(stderr, "Failed to find \"NT_NetRxClose\" in %s\n", + path); + return -1; + } + + return 0; +} + + + +static int +rte_pmd_ntnic_devinit(const char *name, const char *params) +{ + unsigned int numa_node; + int ret = 0; + struct rte_kvargs *kvlist; + struct rte_eth_dev *eth_dev; + unsigned int i; + uint32_t rxqueues = 0; + uint32_t txqueues = 0; + uint32_t port = 0; + uint32_t portend = (uint32_t)-1; + uint32_t hash = (uint32_t)-1; + struct array_s astreamids; + + static int first = 1; + static int stream_id; + char ntplStr[512]; + + astreamids.count = 0; + + RTE_LOG(INFO, PMD, "Initializing pmd_ntnic for %s\n", name); + + numa_node = rte_socket_id(); + + kvlist = rte_kvargs_parse(params, valid_arguments); + if (kvlist == NULL) + return -1; + + /* Get port to use for Rx/Tx */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORT_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORT_ARG, + &ascii_to_u32, &port); + } + /* If Rx port merge is need, her the portend is specified */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORTEND_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORTEND_ARG, + &ascii_to_u32, &portend); + } + + /* Get # RX queues */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_RXQUEUES_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_RXQUEUES_ARG, + &ascii_to_u32, &rxqueues); + } + /* Get # TX queues */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_TXQUEUES_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_TXQUEUES_ARG, + &ascii_to_u32, &txqueues); + } + + /* Get list of streamIds - if used */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_STREAMID_ARG); + if (i) { + ret = rte_kvargs_process(kvlist, ETH_NTNIC_STREAMID_ARG, + &ascii_to_u32_array, &astreamids); + } + + /* Get an alternative hash algorithm */ + i = rte_kvargs_count(kvlist, ETH_NTNIC_HASH_ARG); + if (i) { + assert(i == 1); + ret = rte_kvargs_process(kvlist, ETH_NTNIC_HASH_ARG, + &ascii_to_u32, &hash); + } + + /* check portend and streamids */ + if (portend != (uint32_t)-1 && astreamids.count) { + RTE_LOG(ERR, PMD, "Cannot specify portend when one or more" + " streamid's are specified\n"); + return -1; + } + rte_kvargs_free(kvlist); + + if (ret < 0) + return -1; + + ret = _nt_lib_open(); + if (ret < 0) + return -1; + + if (first) + (*_NT_Init)(NTAPI_VERSION); + + if (astreamids.count) { + first = 0; + /* + * if a specific streamid specified then use that otherwise, + * port defaults to Tx port only + */ + if (rte_pmd_init_internals(name, rxqueues, txqueues, numa_node, + &astreamids, port, ð_dev) < 0) + return -1; + } else { + struct array_s astrids; + if (first) { + /* Delete all NTPL */ + sprintf(ntplStr, "Delete=All"); + if (DoNtpl(ntplStr) != 0) + return -1; + first = 0; + } + astrids.count = 1; + astrids.value[0] = stream_id; + if (rte_pmd_init_internals(name, rxqueues, txqueues, numa_node, + &astrids, port, ð_dev) < 0) + return -1; + + /* Assign the traffic */ + if (portend != (uint32_t)-1) { + sprintf(ntplStr, + "Assign[streamid=(%d..%d);Descriptor=NT]" + "=port==(%d..%d)", + stream_id, + (stream_id + rxqueues - 1), port, portend); + + } else { + sprintf(ntplStr, + "Assign[streamid=(%d..%d);Descriptor=NT]" + "=port==%d", + stream_id, + (stream_id + rxqueues - 1), port); + } + if (DoNtpl(ntplStr) != 0) + return -1; + + if (hash != (uint32_t)-1) { + switch (hash) { + default: + case 1: + DoNtpl("HashMode=HashRoundRobin"); + break; + case 2: + DoNtpl("HashMode=Hash2TupleSorted"); + break; + case 3: + DoNtpl("HashMode=Hash5TupleSorted"); + break; + } + } + + stream_id += rxqueues; + } + + eth_dev->rx_pkt_burst = eth_ntnic_rx; + eth_dev->tx_pkt_burst = eth_ntnic_tx_ringbuffer; + + return 0; +} + +static int +rte_pmd_ntnic_devuninit(const char *name) +{ + (void)name; + if (_libnt != NULL) + dlclose(_libnt); + return 0; +} + + +static struct rte_driver pmd_ntnic_drv = { + .type = PMD_VDEV, + .init = rte_pmd_ntnic_devinit, + .uninit = rte_pmd_ntnic_devuninit, +}; + +PMD_REGISTER_DRIVER(pmd_ntnic_drv, eth_ntnic); +DRIVER_REGISTER_PARAM_STRING(eth_ntnic, + "port=<int> " + "rxqs=<int>" + "txqs=<int>" + "hash=<int>" + "streamids=<int..int>"); + diff --git a/drivers/net/ntnic/rte_pmd_ntnic_version.map b/drivers/net/ntnic/rte_pmd_ntnic_version.map new file mode 100644 index 0000000..ef35398 --- /dev/null +++ b/drivers/net/ntnic/rte_pmd_ntnic_version.map @@ -0,0 +1,4 @@ +DPDK_2.0 { + + local: *; +}; diff --git a/mk/rte.app.mk b/mk/rte.app.mk index 1a0095b..9ef357f 100644 --- a/mk/rte.app.mk +++ b/mk/rte.app.mk @@ -120,6 +120,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += -lrte_pmd_mpipe -lgxio _LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += -lrte_pmd_nfp -lm _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += -lrte_pmd_null _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += -lrte_pmd_pcap -lpcap + _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD) += -lrte_pmd_qede -lz _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING) += -lrte_pmd_ring _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 @@ -143,6 +144,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -lrte_pmd_kasumi _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi endif # CONFIG_RTE_LIBRTE_CRYPTODEV +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += -lrte_pmd_ntnic endif # !CONFIG_RTE_BUILD_SHARED_LIBS _LDLIBS-y += --no-whole-archive @@ -163,6 +165,7 @@ endif _LDLIBS-$(CONFIG_RTE_PORT_PCAP) += -lpcap endif # !CONFIG_RTE_BUILD_SHARED_LIBS + _LDLIBS-y += $(EXECENV_LDLIBS) LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS) -- 2.9.0 Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-09 12:48 ` [dpdk-dev] [PATCH v3] " Finn Christensen @ 2016-09-09 13:51 ` Neil Horman 2016-09-10 7:58 ` Finn Christensen 0 siblings, 1 reply; 22+ messages in thread From: Neil Horman @ 2016-09-09 13:51 UTC (permalink / raw) To: Finn Christensen; +Cc: dev, thomas.monjalon, stephen On Fri, Sep 09, 2016 at 12:48:38PM +0000, Finn Christensen wrote: > This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. > > This patch adds support for Napatech NICs to DPDK. This is the > initial implementation. > > Signed-off-by: Finn Christensen <fc@napatech.com> > --- > v3: > * Removed the need for binary libraries on build > v2: > * Added information how to build the PMD without NIC > Board Support Package > * Fixed some formatting issues So, this is a step in the right direction, but I think its solving the wrong problem. If you have a dependency on an external library, thats ok, and accessing it via dlopen makes it possible to build the library without having that library present, but it not really in keeping with the spirit of what I meant. This driver is still effectively dependent on a binary blob that we have no visibility into. The better solution is releasing the source for the ntnic and ntos libraries. The license file in the referenced git tree indicates its BSD licensed, so I don't think there should be a problem in doing that. Neil > --- > MAINTAINERS | 5 + > config/common_base | 6 + > doc/guides/nics/features/ntnic.ini | 14 + > doc/guides/nics/index.rst | 1 + > doc/guides/nics/ntnic.rst | 145 ++++ > drivers/net/Makefile | 1 + > drivers/net/ntnic/Makefile | 66 ++ > drivers/net/ntnic/rte_eth_ntnic.c | 1150 +++++++++++++++++++++++++++ > drivers/net/ntnic/rte_pmd_ntnic_version.map | 4 + > mk/rte.app.mk | 3 + > 10 files changed, 1395 insertions(+) > create mode 100644 doc/guides/nics/features/ntnic.ini > create mode 100644 doc/guides/nics/ntnic.rst > create mode 100644 drivers/net/ntnic/Makefile > create mode 100644 drivers/net/ntnic/rte_eth_ntnic.c > create mode 100644 drivers/net/ntnic/rte_pmd_ntnic_version.map > > diff --git a/MAINTAINERS b/MAINTAINERS > index bc9aa02..d3e5f56 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -356,6 +356,11 @@ M: Sony Chacko <sony.chacko@qlogic.com> > F: drivers/net/qede/ > F: doc/guides/nics/qede.rst > > +Napatech ntnic > +M: Finn Christensen <fc@napatech.com> > +F: drivers/net/ntnic/ > +F: doc/guides/nics/ntnic.rst > + > RedHat virtio > M: Huawei Xie <huawei.xie@intel.com> > M: Yuanhan Liu <yuanhan.liu@linux.intel.com> > diff --git a/config/common_base b/config/common_base > index 7830535..4a3b2b8 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -309,6 +309,12 @@ CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16 > CONFIG_RTE_LIBRTE_PMD_PCAP=n > > # > +# Compile software PMD backed by NTNIC files > +# > +CONFIG_RTE_LIBRTE_PMD_NTNIC=n > + > + > +# > # Compile link bonding PMD library > # > CONFIG_RTE_LIBRTE_PMD_BOND=y > diff --git a/doc/guides/nics/features/ntnic.ini b/doc/guides/nics/features/ntnic.ini > new file mode 100644 > index 0000000..4237e6e > --- /dev/null > +++ b/doc/guides/nics/features/ntnic.ini > @@ -0,0 +1,14 @@ > +; > +; Supported features of the 'ntnic' network poll mode driver. > +; > +; Refer to default.ini for the full list of available PMD features. > +; > +[Features] > +Link status = Y > +Queue start/stop = Y > +Promiscuous mode = Y > +Jumbo frame = Y > +Basic stats = Y > +Multiprocess aware = Y > +x86-64 = Y > +Usage doc = Y > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > index 92d56a5..5c4205c 100644 > --- a/doc/guides/nics/index.rst > +++ b/doc/guides/nics/index.rst > @@ -52,6 +52,7 @@ Network Interface Controller Drivers > qede > szedata2 > thunderx > + ntnic > virtio > vhost > vmxnet3 > diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst > new file mode 100644 > index 0000000..f9398db > --- /dev/null > +++ b/doc/guides/nics/ntnic.rst > @@ -0,0 +1,145 @@ > +.. BSD LICENSE > + Copyright (c) 2016 Napatech A/S > + All rights reserved. > + > + Redistribution and use in source and binary forms, with or without > + modification, are permitted provided that the following conditions > + are met: > + > + * Redistributions of source code must retain the above copyright > + notice, this list of conditions and the following disclaimer. > + * Redistributions in binary form must reproduce the above copyright > + notice, this list of conditions and the following disclaimer in > + the documentation and/or other materials provided with the > + distribution. > + * Neither the name of Napatech nor the names of its > + contributors may be used to endorse or promote products derived > + from this software without specific prior written permission. > + > + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +NTNIC Poll Mode Driver > +====================== > + > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements support > +for **Napatech NIC** 40/50 Gbps adapters. > +This PMD is implemented as a pure software virtual device and must be created > +by using the EAL --vdev=parameter (parameters are explained i detail later). > +It runs on top of the Napatech NFV NIC Board Support Package that must be > +installed and started. > +If no Napatech NIC is available, you can download the Napatech NTAPI includes > +to build with. > + > +Supported Features > +------------------ > + > +- RSS (Receive Side Scaling) > +- TSS (Transmit Side Scaling) > +- Promiscuous mode > +- Basic statistics > + > + > +Prerequisites > +------------- > + > +Requires Napatech NIC and Napatech NIC Board Support Package installed and > +running in version **0.3.0** or higher. > +This includes external libraries and kernel driver for resources > +allocations and initialization. > +If build only is required, download the Napatech NTAPI to build against. > + > +Pre-Installation Configuration > +------------------------------ > + > +Environment variables > +~~~~~~~~~~~~~~~~~~~~~ > + > +In order to compile the Napatech NIC PMD, user must: > + > +* Export the environment variable NAPATECH3_PATH with the path where > + the Napatech Board Support Package was installed (default location is > + /opt/napatech3). > + Or if no Napatech NIC available: > + Download the Napatech NTAPI include files from Github: > + `Napatech NTAPI <https://github.com/napa-tech/ntapi>`_. > + > + > +Config File Options > +~~~~~~~~~~~~~~~~~~~ > + > +- ``CONFIG_RTE_LIBRTE_PMD_NTNIC`` (default **n**) > + > +Using the NTNIC PMD from a DPDK application using EAL vdev parameter > +-------------------------------------------------------------------- > + > +Napatech NIC PMD VDEV parameters > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The Napatech NIC PMD vdev has the following command line parameters to enable > +its features. > + > +* port > + Configures which NT port the vdev should use. > + > + --vdev eth_ntnic0,port=4,rxqs=1,txqs=1 > + > + This will create a DPDK port0 using NT port 4 > + > +* rxqs > + Control how many receive queues a vdev should have. The traffic from a vdev > + will be load balanced to this amount of queues and each queue can be handled > + by its own lcore. > + Note: The ntservice.ini HostBuffersRx must be configured to the sum of all > + receive queues across all vdevs. > + > + --vdev eth_ntnic0,port=0,rxqs=8,txqs=1 > + > + Traffic from NT port 0 will be split across > + 8 queues meaning that 8 lcores can be > + opened on DPDK port 0 and each will be > + their individual traffic. > + > +* hash > + Control which load balancing scheme should be used when running with multiple > + receive queues. > + The possible values are: > + 1 = HashRoundRobin > + 2 = Hash2TupleSorted > + 3 = Hash5TupleSorted > + > + --vdev eth_ntnic0,port=0,rxqs=4,hash=3,txqs=1 > + > + Create 4 queues from NT port 0 and > + load balance the traffic using the > + 5-tuple sorted algorithm. Each of the > + 4 queues can be opened via DPDK port 0 > + > +* txqs > + Control how many transmit queues a vdev should have. Each queue can be used > + by a lcore and each queue is independent for other queues. > + Note: The ntservice.ini HostBuffersTx must be configured to the sum of all > + transmit queues across all vdevs. > + > + --vdev eth_ntnic0,port=0,txqs=32,rxqs=1 > + > + Enable 32 transmit queues on NT port 0 > + > + > +Changes to Napatech driver configuration file ntservice.ini > +----------------------------------------------------------- > +Depending on the number of queues that is created, more hostbuffers may be > +needed pre-allocated by the Napatech NIC Driver. This is controlled in the > +ntservice.ini file (default locations is /opt/napatech3/config). > +The Napatech NIC Driver must be re-started when this configuration file is > +changed. > + > diff --git a/drivers/net/Makefile b/drivers/net/Makefile > index bc93230..aff29e9 100644 > --- a/drivers/net/Makefile > +++ b/drivers/net/Makefile > @@ -55,6 +55,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx > DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio > DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3 > DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt > +DIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += ntnic > > ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) > DIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += vhost > diff --git a/drivers/net/ntnic/Makefile b/drivers/net/ntnic/Makefile > new file mode 100644 > index 0000000..d816c56 > --- /dev/null > +++ b/drivers/net/ntnic/Makefile > @@ -0,0 +1,66 @@ > +# BSD LICENSE > +# > +# Copyright(c) 2016 Napatech A/S. All rights reserved. > +# All rights reserved. > +# > +# Redistribution and use in source and binary forms, with or without > +# modification, are permitted provided that the following conditions > +# are met: > +# > +# * Redistributions of source code must retain the above copyright > +# notice, this list of conditions and the following disclaimer. > +# * Redistributions in binary form must reproduce the above copyright > +# notice, this list of conditions and the following disclaimer in > +# the documentation and/or other materials provided with the > +# distribution. > +# * Neither the name of Napatech nor the names of its > +# contributors may be used to endorse or promote products derived > +# from this software without specific prior written permission. > +# > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +# > +# library name > +# > +LIB = librte_pmd_ntnic.a > + > +CFLAGS += -O3 -g > +CFLAGS += $(WERROR_FLAGS) > +CFLAGS += -I$(NAPATECH3_PATH)/include > +CFLAGS += -DNAPATECH3_LIB_PATH=\"$(NAPATECH3_PATH)/lib\" > +LDLIBS += -ldl > + > +EXPORT_MAP := rte_pmd_ntnic_version.map > + > +LIBABIVER := 1 > + > +# > +# all source are stored in SRCS-y > +# > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += rte_eth_ntnic.c > + > +# > +# Export include files > +# > +SYMLINK-y-include += > + > +# this lib depends upon: > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_eal > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mbuf > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mempool > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_ether > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_kvargs > + > +include $(RTE_SDK)/mk/rte.lib.mk > diff --git a/drivers/net/ntnic/rte_eth_ntnic.c b/drivers/net/ntnic/rte_eth_ntnic.c > new file mode 100644 > index 0000000..f83e94d > --- /dev/null > +++ b/drivers/net/ntnic/rte_eth_ntnic.c > @@ -0,0 +1,1150 @@ > +/*- > + * BSD LICENSE > + * > + * Copyright(c) 2016 Napatech A/S. All rights reserved. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyright > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of Napatech nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. > + */ > + > +#include <time.h> > +#include <dlfcn.h> > +#include <rte_mbuf.h> > +#include <rte_ethdev.h> > +#include <rte_malloc.h> > +#include <rte_memcpy.h> > +#include <rte_string_fns.h> > +#include <rte_cycles.h> > +#include <rte_kvargs.h> > +#include <rte_dev.h> > +#include <net/if.h> > +#include <nt.h> > + > +#define ETH_NTNIC_PORT_ARG "port" > +#define ETH_NTNIC_PORTEND_ARG "portend" > +#define ETH_NTNIC_RXQUEUES_ARG "rxqs" > +#define ETH_NTNIC_TXQUEUES_ARG "txqs" > +#define ETH_NTNIC_STREAMID_ARG "streamid" > +#define ETH_NTNIC_HASH_ARG "hash" > + > +#define HW_MAX_PKT_LEN 10000 > +#define HW_MTU (HW_MAX_PKT_LEN - ETHER_HDR_LEN - ETHER_CRC_LEN) > + > +#define MAX_RX_QUEUES 64 > +#define MAX_TX_QUEUES 64 > +#define MAX_NTNIC_PORTS 32 > + > +static void *_libnt; > + > +/* NTAPI library functions */ > +int (*_NT_Init)(uint32_t); > +int (*_NT_NetRxOpen)(NtNetStreamRx_t *, > + const char *, enum NtNetInterface_e, uint32_t, int); > +int (*_NT_NetRxGet)(NtNetStreamRx_t, NtNetBuf_t *, int); > +int (*_NT_NetRxRelease)(NtNetStreamRx_t, NtNetBuf_t); > +int (*_NT_NetRxClose)(NtNetStreamRx_t); > +char *(*_NT_ExplainError)(int, char *, uint32_t); > +int (*_NT_NetTxOpen)(NtNetStreamTx_t *, > + const char *, uint64_t, uint32_t, uint32_t); > +int (*_NT_NetTxRingbufferInit)(NtNetStreamTx_t, uint32_t); > +int (*_NT_NetTxRingbufferTransmitPacket)(NtNetStreamTx_t, > + uint8_t *, uint16_t); > +int (*_NT_NetTxRingbufferFlush)(NtNetStreamTx_t); > +void (*_NT_NetTxRingbufferDone)(NtNetStreamTx_t); > +int (*_NT_NetTxClose)(NtNetStreamTx_t); > +int (*_NT_InfoOpen)(NtInfoStream_t *, const char *); > +int (*_NT_InfoRead)(NtInfoStream_t, NtInfo_t *); > +int (*_NT_InfoClose)(NtInfoStream_t); > +int (*_NT_ConfigOpen)(NtConfigStream_t *, const char *); > +int (*_NT_ConfigClose)(NtConfigStream_t); > +int (*_NT_NTPL)(NtConfigStream_t, const char *, > + NtNtplInfo_t *, uint32_t); > + > +static char ebuf[1024]; > + > +struct array_s { > + uint32_t value[MAX_RX_QUEUES]; > + int count; > +}; > + > +static volatile uint16_t port_locks[MAX_NTNIC_PORTS]; > + > +struct ntnic_rx_queue { > + NtNetStreamRx_t net_rx[MAX_RX_QUEUES]; /* NT Rx streams */ > + int rx_idx; /* Current Rx stream index */ > + struct rte_mempool *mb_pool; /* mbuf memory pool */ > + uint16_t buf_size; /* Size of data area in mbuf */ > + NtNetBuf_t seg; /* The current NT data segment */ > + struct NtNetBuf_s pkt; /* The current packet */ > + volatile unsigned long rx_pkts; /* Rx packet statistics */ > + volatile unsigned long err_pkts; /* Rx error packet statistics */ > + struct array_s astreamids; /* NT streams to read from */ > + int enabled; /* Enabling/disabling of this queue */ > +}; > + > +struct ntnic_tx_queue { > + NtNetStreamTx_t net_tx; /* NT Tx stream */ > + volatile unsigned long tx_pkts; /* Tx packet statistics */ > + volatile unsigned long err_pkts; /* Tx error packet stat */ > + volatile uint16_t *plock; /* Per port transmit atomic lock */ > + uint32_t port; /* Tx port for this queue */ > + int enabled; /* Enabling/disabling of this queue */ > + int fcs_add; /* If needs added room for FCS */ > +}; > + > +struct pmd_internals { > + struct ntnic_rx_queue rxq[MAX_RX_QUEUES]; /* Array of Rx queues */ > + struct ntnic_tx_queue txq[MAX_TX_QUEUES]; /* Array of Tx queues */ > + int if_index; /* Itf index always 0 - no bonding */ > + unsigned int nb_rx_queues; /* Number of Rx queues configured */ > + unsigned int nb_tx_queues; /* Number of Tx queues configured */ > + int MAC_50G; /* True if 50G ports */ > +}; > + > +static const char *valid_arguments[] = { > + ETH_NTNIC_PORT_ARG, > + ETH_NTNIC_PORTEND_ARG, > + ETH_NTNIC_RXQUEUES_ARG, > + ETH_NTNIC_TXQUEUES_ARG, > + ETH_NTNIC_STREAMID_ARG, > + ETH_NTNIC_HASH_ARG, > + NULL > +}; > + > +static struct ether_addr eth_addr[MAX_NTNIC_PORTS]; > +static const char *drivername = "NTNIC PMD"; > + > +static int > +eth_ntnic_rx_jumbo(struct rte_mempool *mb_pool, > + struct rte_mbuf *mbuf, > + const u_char *data, > + uint16_t data_len) > +{ > + struct rte_mbuf *m = mbuf; > + > + /* Copy the first segment. */ > + uint16_t len = rte_pktmbuf_tailroom(mbuf); > + > + rte_memcpy(rte_pktmbuf_append(mbuf, len), data, len); > + data_len -= len; > + data += len; > + > + while (data_len > 0) { > + /* Allocate next mbuf and point to that. */ > + m->next = rte_pktmbuf_alloc(mb_pool); > + > + if (unlikely(!m->next)) > + return -1; > + > + m = m->next; > + > + /* Headroom is not needed in chained mbufs. */ > + rte_pktmbuf_prepend(m, rte_pktmbuf_headroom(m)); > + m->pkt_len = 0; > + m->data_len = 0; > + > + /* Copy next segment. */ > + len = RTE_MIN(rte_pktmbuf_tailroom(m), data_len); > + rte_memcpy(rte_pktmbuf_append(m, len), data, len); > + > + mbuf->nb_segs++; > + data_len -= len; > + data += len; > + } > + > + return mbuf->nb_segs; > +} > + > + > + > +static uint16_t > +eth_ntnic_rx(void *queue, > + struct rte_mbuf **bufs, > + uint16_t nb_pkts) > +{ > + unsigned int i; > + struct rte_mbuf *mbuf; > + struct ntnic_rx_queue *rx_q = queue; > + uint16_t num_rx = 0; > + uint16_t data_len; > + > + if (unlikely(rx_q->net_rx[rx_q->rx_idx] == NULL || nb_pkts == 0)) > + return 0; > + > + /* Do we have any data segment */ > + if (rx_q->seg == NULL) { > + /* Next stream - if multiple streams are combined */ > + rx_q->rx_idx = > + (rx_q->rx_idx < > + (rx_q->astreamids.count - 1)) ? rx_q->rx_idx + 1 > + : 0; > + > + if ((*_NT_NetRxGet)(rx_q->net_rx[rx_q->rx_idx], > + &rx_q->seg, 0) != NT_SUCCESS) { > + if (rx_q->seg != NULL) { > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > + rx_q->seg); > + rx_q->seg = NULL; > + } > + return 0; > + } > + if (NT_NET_GET_SEGMENT_LENGTH(rx_q->seg)) { > + /* Build a packet structure */ > + _nt_net_build_pkt_netbuf(rx_q->seg, &rx_q->pkt); > + } else { > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > + rx_q->seg); > + rx_q->seg = NULL; > + return 0; > + } > + } > + > + if (rte_mempool_get_bulk(rx_q->mb_pool, (void **)bufs, nb_pkts) != 0) > + return 0; > + > + for (i = 0; i < nb_pkts; i++) { > + mbuf = bufs[i]; > + rte_mbuf_refcnt_set(mbuf, 1); > + rte_pktmbuf_reset(mbuf); > + > + /* HW slicing not supported */ > + data_len = (uint16_t)NT_NET_GET_PKT_WIRE_LENGTH((&rx_q->pkt)) > + - 4; > + if (data_len <= rx_q->buf_size) { > + /* Packet will fit in the mbuf, go ahead and copy */ > + mbuf->pkt_len = mbuf->data_len; > + data_len = mbuf->data_len; > + rte_memcpy((u_char *)mbuf->buf_addr + > + RTE_PKTMBUF_HEADROOM, > + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), > + mbuf->data_len); > + } else { > + /* Try read jumbo frame into multi mbufs. */ > + if (unlikely(eth_ntnic_rx_jumbo(rx_q->mb_pool, > + mbuf, > + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), > + data_len) == -1)) > + break; > + } > + > + mbuf->port = NT_NET_GET_PKT_RXPORT((&rx_q->pkt)); > + num_rx++; > + > + /* Get the next packet if any */ > + if (_nt_net_get_next_packet(rx_q->seg, > + NT_NET_GET_SEGMENT_LENGTH(rx_q->seg), > + &rx_q->pkt) == 0) { > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > + rx_q->seg); > + rx_q->seg = NULL; > + break; > + } > + > + rx_q->rx_pkts++; > + } > + > + if (num_rx < nb_pkts) { > + rte_mempool_put_bulk(rx_q->mb_pool, > + (void * const *)(bufs + num_rx), > + nb_pkts - num_rx); > + } > + return num_rx; > +} > + > + > + > + > + > +/* Callback to handle sending packets through a NT NIC. */ > +static uint16_t > +eth_ntnic_tx_ringbuffer(void *queue, > + struct rte_mbuf **bufs, > + uint16_t nb_pkts) > +{ > + unsigned int i; > + struct ntnic_tx_queue *tx_q = queue; > + int retval; > + uint16_t old; > + uint16_t new_flag = 0; > + > + > + if (unlikely(tx_q == NULL || tx_q->net_tx == NULL || nb_pkts == 0)) > + return 0; > + > + > + do { > + old = 0; > + new_flag = 1; > + retval = rte_atomic16_cmpset(tx_q->plock, old, new_flag); > + } while (unlikely(retval == 0)); > + > + for (i = 0; i < nb_pkts; i++) { > + /* Not allowed to TX less than 64 byte packets */ > + if (bufs[i]->pkt_len < 60) > + bufs[i]->pkt_len = 60; > + > + /* transmit a single packet */ > + (*_NT_NetTxRingbufferTransmitPacket)(tx_q->net_tx, > + rte_pktmbuf_mtod(bufs[i], uint8_t *), > + bufs[i]->pkt_len + 4 - tx_q->fcs_add); > + rte_pktmbuf_free(bufs[i]); > + } > + tx_q->tx_pkts += nb_pkts; > + > + *tx_q->plock = 0; > + return nb_pkts; > +} > + > + > + > +static int > +eth_dev_start(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *internals = dev->data->dev_private; > + struct ntnic_rx_queue *rx_q = internals->rxq; > + struct ntnic_tx_queue *tx_q = internals->txq; > + uint queue; > + int status, idx; > + > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > + > + if ((internals->nb_tx_queues | internals->nb_rx_queues) == 0) > + return -1; > + > + for (queue = 0; queue < internals->nb_rx_queues; queue++) { > + if (rx_q[queue].enabled) { > + char str[128], val[10]; > + str[0] = 0; > + /* build list of streamids to log */ > + for (idx = 0; idx < > + rx_q[queue].astreamids.count; idx++) { > + if (idx) > + sprintf(val, ",%d", > + rx_q[queue].astreamids.value[idx]); > + else > + sprintf(val, "%i", > + rx_q[queue].astreamids.value[idx]); > + strcat(str, val); > + } > + > + for (idx = 0; idx < rx_q[queue].astreamids.count; > + idx++) { > + status = > + (*_NT_NetRxOpen)(&rx_q[queue].net_rx[idx], > + "DPDK", > + NT_NET_INTERFACE_SEGMENT, > + rx_q[queue].astreamids.value[idx], -1); > + if (status != NT_SUCCESS) { > + /* try packet interface instead */ > + (*_NT_ExplainError)(status, > + ebuf, > + sizeof(ebuf)); > + RTE_LOG(ERR, PMD, > + "NT_NetRxOpen() failed: %s\n", > + ebuf); > + return -1; > + } > + } > + } > + } > + > + for (queue = 0; queue < internals->nb_tx_queues; queue++) { > + if (tx_q[queue].enabled) { > + RTE_LOG(INFO, PMD, "NTNIC: NT_NetTxOpen(%d)\n", > + tx_q[queue].port); > + status = (*_NT_NetTxOpen)(&tx_q[queue].net_tx, "DPDK", > + 1 << tx_q[queue].port, > + dev->pci_dev->numa_node, 0); > + if (status != NT_SUCCESS) { > + (*_NT_ExplainError)(status, ebuf, > + sizeof(ebuf)); > + RTE_LOG(INFO, PMD, > + "NT_NetTxOpen(0x%X, %d, 0) failed: %s\n", > + 1 << tx_q[queue].port, > + dev->pci_dev->numa_node, ebuf); > + return -1; > + } > + > + /* Init and associate RTD Tx Ring buf to Tx handler */ > + status = (*_NT_NetTxRingbufferInit)( > + tx_q[queue].net_tx, > + tx_q[queue].port); > + if (status != NT_SUCCESS) { > + (*_NT_ExplainError)(status, ebuf, > + sizeof(ebuf)); > + RTE_LOG(ERR, PMD, > + "NT_NetRxOpen() failed: %s\n", > + ebuf); > + return -1; > + } > + } > + tx_q[queue].plock = &port_locks[tx_q[queue].port]; > + tx_q[queue].fcs_add = (internals->MAC_50G) ? 4 : 0; > + } > + > + dev->data->dev_link.link_status = 1; > + return 0; > +} > + > + > +static void > +eth_dev_stop(struct rte_eth_dev *dev) > +{ > + struct pmd_internals *internals = dev->data->dev_private; > + struct ntnic_rx_queue *rx_q = internals->rxq; > + struct ntnic_tx_queue *tx_q = internals->txq; > + uint q; > + int idx; > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > + > + for (q = 0; q < internals->nb_rx_queues; q++) { > + if (rx_q[q].seg) { > + (*_NT_NetRxRelease)(rx_q[q].net_rx[rx_q->rx_idx], > + rx_q[q].seg); > + rx_q[q].seg = NULL; > + } > + for (idx = 0; idx < rx_q[q].astreamids.count; idx++) { > + if (rx_q[q].net_rx[idx]) > + (void)(*_NT_NetRxClose)(rx_q[q].net_rx[idx]); > + } > + } > + for (q = 0; q < internals->nb_tx_queues; q++) { > + if (tx_q[q].net_tx) { > + (*_NT_NetTxRingbufferDone)(tx_q[q].net_tx); > + (void)(*_NT_NetTxClose)(tx_q[q].net_tx); > + } > + } > + dev->data->dev_link.link_status = 0; > +} > + > +static int > +eth_dev_configure(struct rte_eth_dev *dev __rte_unused) > +{ > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > + return 0; > +} > + > +static void > +eth_dev_info(struct rte_eth_dev *dev, > + struct rte_eth_dev_info *dev_info) > +{ > + struct pmd_internals *internals = dev->data->dev_private; > + dev_info->if_index = internals->if_index; > + dev_info->driver_name = drivername; > + dev_info->max_mac_addrs = 1; > + dev_info->max_rx_pktlen = HW_MTU; > + dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; > + dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; > + dev_info->min_rx_bufsize = 0; > + dev_info->pci_dev = NULL; > +} > + > +static void > +eth_stats_get(struct rte_eth_dev *dev, > + struct rte_eth_stats *igb_stats) > +{ > + unsigned int i; > + unsigned long rx_total = 0; > + unsigned long tx_total = 0; > + unsigned long tx_err_total = 0; > + const struct pmd_internals *internal = dev->data->dev_private; > + > + memset(igb_stats, 0, sizeof(*igb_stats)); > + for (i = 0; > + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal->nb_rx_queues; > + i++) { > + igb_stats->q_ipackets[i] = internal->rxq[i].rx_pkts; > + rx_total += igb_stats->q_ipackets[i]; > + } > + for (i = 0; > + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal->nb_tx_queues; > + i++) { > + igb_stats->q_opackets[i] = internal->txq[i].tx_pkts; > + igb_stats->q_errors[i] = internal->txq[i].err_pkts; > + tx_total += igb_stats->q_opackets[i]; > + tx_err_total += igb_stats->q_errors[i]; > + } > + > + igb_stats->ipackets = rx_total; > + igb_stats->opackets = tx_total; > + igb_stats->oerrors = tx_err_total; > +} > + > +static void > +eth_stats_reset(struct rte_eth_dev *dev) > +{ > + unsigned int i; > + struct pmd_internals *internal = dev->data->dev_private; > + > + for (i = 0; i < internal->nb_rx_queues; i++) > + internal->rxq[i].rx_pkts = 0; > + for (i = 0; i < internal->nb_tx_queues; i++) { > + internal->txq[i].tx_pkts = 0; > + internal->txq[i].err_pkts = 0; > + } > +} > + > +static void > +eth_dev_close(struct rte_eth_dev *dev __rte_unused) > +{ > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > +} > + > +static void > +eth_queue_release(void *q __rte_unused) > +{ > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > +} > + > +static int > +eth_link_update(struct rte_eth_dev *dev __rte_unused, > + int wait_to_complete __rte_unused) > +{ > + return 0; > +} > + > +static int > +eth_rx_queue_setup(struct rte_eth_dev *dev, > + uint16_t rx_queue_id, > + uint16_t nb_rx_desc __rte_unused, > + unsigned int socket_id __rte_unused, > + const struct rte_eth_rxconf *rx_conf __rte_unused, > + struct rte_mempool *mb_pool) > +{ > + struct rte_pktmbuf_pool_private *mbp_priv; > + struct pmd_internals *internals = dev->data->dev_private; > + struct ntnic_rx_queue *rx_q = &internals->rxq[rx_queue_id]; > + > + RTE_LOG(INFO, PMD, "NTNIC RX queue setup\n"); > + rx_q->mb_pool = mb_pool; > + dev->data->rx_queues[rx_queue_id] = rx_q; > + > + mbp_priv = rte_mempool_get_priv(rx_q->mb_pool); > + rx_q->buf_size = (uint16_t)(mbp_priv->mbuf_data_room_size - > + RTE_PKTMBUF_HEADROOM); > + rx_q->enabled = 1; > + return 0; > +} > + > +static int > +eth_tx_queue_setup(struct rte_eth_dev *dev __rte_unused, > + uint16_t tx_queue_id __rte_unused, > + uint16_t nb_tx_desc __rte_unused, > + unsigned int socket_id __rte_unused, > + const struct rte_eth_txconf *tx_conf __rte_unused) > +{ > + struct pmd_internals *internals = dev->data->dev_private; > + RTE_LOG(INFO, PMD, "NTNIC TX queue setup\n"); > + dev->data->tx_queues[tx_queue_id] = &internals->txq[tx_queue_id]; > + internals->txq[tx_queue_id].enabled = 1; > + return 0; > +} > + > +static int _dev_set_mtu(struct rte_eth_dev *dev __rte_unused, uint16_t mtu) > +{ > + if (mtu < 46 || mtu > HW_MTU) > + return -EINVAL; > + > + return 0; > +} > + > +static struct eth_dev_ops ops = { > + .dev_start = eth_dev_start, > + .dev_stop = eth_dev_stop, > + .dev_close = eth_dev_close, > + .mtu_set = _dev_set_mtu, > + .dev_configure = eth_dev_configure, > + .dev_infos_get = eth_dev_info, > + .rx_queue_setup = eth_rx_queue_setup, > + .tx_queue_setup = eth_tx_queue_setup, > + .rx_queue_release = eth_queue_release, > + .tx_queue_release = eth_queue_release, > + .link_update = eth_link_update, > + .stats_get = eth_stats_get, > + .stats_reset = eth_stats_reset, > +}; > + > +static int > +rte_pmd_init_internals(const char *name, > + const unsigned int nb_rx_queues, > + const unsigned int nb_tx_queues, > + const unsigned int numa_node, > + struct array_s *pastreamids, > + const uint32_t port, > + struct rte_eth_dev **eth_dev) > +{ > + struct pmd_internals *internals = NULL; > + struct rte_eth_dev_data *data = NULL; > + struct rte_pci_device *pci_dev = NULL; > + uint i, status; > + char err_buf[NT_ERRBUF_SIZE]; > + NtInfoStream_t info; > + NtInfo_t info_port; > + struct rte_eth_link pmd_link; > + assert(nb_rx_queues < MAX_RX_QUEUES); > + assert(nb_tx_queues < MAX_TX_QUEUES); > + assert(port < MAX_NTNIC_PORTS); > + > + RTE_LOG(INFO, PMD, > + "Creating ntnic-backend ethdev on numa socket %u\n", > + numa_node); > + > + /* now do all data allocation - for eth_dev structure, dummy pci driver > + * and internal (private) data > + */ > + data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node); > + if (data == NULL) > + goto error; > + > + pci_dev = rte_zmalloc_socket(name, sizeof(*pci_dev), 0, numa_node); > + if (pci_dev == NULL) > + goto error; > + > + internals = rte_zmalloc_socket(name, sizeof(*internals), 0, numa_node); > + if (internals == NULL) > + goto error; > + > + /* reserve an ethdev entry */ > + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > + if (*eth_dev == NULL) > + goto error; > + > + /* Open the information stream */ > + status = (*_NT_InfoOpen)(&info, "DPDK Info stream"); > + if (status != NT_SUCCESS) { > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > + RTE_LOG(ERR, PMD, > + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", > + status, err_buf); > + return status; > + } > + > + /* Find local port offset */ > + info_port.cmd = NT_INFO_CMD_READ_PORT_V7; > + info_port.u.port_v7.portNo = (uint8_t)(port); > + status = (*_NT_InfoRead)(info, &info_port); > + if (status != 0) { > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > + RTE_LOG(ERR, PMD, > + "ERROR: NT_InfoRead failed. Code 0x%x = %s\n", > + status, err_buf); > + return status; > + } > + > + /* check for 50G MAC */ > + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9506 || > + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9509) { > + internals->MAC_50G = 1; > + } else { > + internals->MAC_50G = 0; > + /* Check for valid FPGAs (NFV NICs) */ > + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != > + 9507 && > + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != > + 9510) { > + RTE_LOG(ERR, PMD, > + "ERROR: NTNIC PMD does not support the NT adapter" > + "- port %i\n", > + port); > + return -1; > + } > + } > + > + internals->nb_rx_queues = nb_rx_queues; > + internals->nb_tx_queues = nb_tx_queues; > + for (i = 0; i < nb_rx_queues; i++) { > + int idx; > + for (idx = 0; idx < pastreamids->count; idx++) { > + internals->rxq[i].astreamids.value[idx] = > + pastreamids->value[idx] + i; > + } > + > + internals->rxq[i].astreamids.count = pastreamids->count; > + > + internals->rxq[i].seg = NULL; > + internals->rxq[i].enabled = 0; > + } > + > + for (i = 0; i < nb_tx_queues; i++) { > + if (port < info_port.u.port_v7.data.adapterInfo.portOffset) { > + RTE_LOG(ERR, PMD, "Error wrong port specified\n"); > + return -1; > + } > + internals->txq[i].port = port; > + internals->txq[i].enabled = 0; > + } > + > + switch (info_port.u.port_v7.data.speed) { > + case NT_LINK_SPEED_UNKNOWN: > + pmd_link.link_speed = ETH_SPEED_NUM_1G; > + break; > + case NT_LINK_SPEED_10M: > + pmd_link.link_speed = ETH_SPEED_NUM_10M; > + break; > + case NT_LINK_SPEED_100M: > + pmd_link.link_speed = ETH_SPEED_NUM_100M; > + break; > + case NT_LINK_SPEED_1G: > + pmd_link.link_speed = ETH_SPEED_NUM_1G; > + break; > + case NT_LINK_SPEED_10G: > + pmd_link.link_speed = ETH_SPEED_NUM_10G; > + break; > + case NT_LINK_SPEED_40G: > + pmd_link.link_speed = ETH_SPEED_NUM_40G; > + break; > + case NT_LINK_SPEED_50G: > + pmd_link.link_speed = ETH_SPEED_NUM_50G; > + break; > + case NT_LINK_SPEED_100G: > + pmd_link.link_speed = ETH_SPEED_NUM_100G; > + break; > + } > + > + memcpy(ð_addr[port].addr_bytes, &info_port.u.port_v7.data.macAddress, > + sizeof(eth_addr[port].addr_bytes)); > + status = (*_NT_InfoClose)(info); > + if (status != NT_SUCCESS) { > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > + RTE_LOG(ERR, PMD, > + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", > + status, err_buf); > + return status; > + } > + > + pmd_link.link_duplex = ETH_LINK_FULL_DUPLEX; > + pmd_link.link_status = 0; > + > + internals->if_index = 0; > + > + pci_dev->numa_node = numa_node; > + > + data->dev_private = internals; > + data->port_id = (*eth_dev)->data->port_id; > + data->nb_rx_queues = (uint16_t)nb_rx_queues; > + data->nb_tx_queues = (uint16_t)nb_tx_queues; > + data->dev_link = pmd_link; > + data->mac_addrs = ð_addr[port]; > + data->numa_node = numa_node; > + data->drv_name = drivername; > + > + (*eth_dev)->data = data; > + (*eth_dev)->dev_ops = &ops; > + (*eth_dev)->pci_dev = pci_dev; > + > + return 0; > + > +error: > + if (data) > + rte_free(data); > + if (pci_dev) > + rte_free(pci_dev); > + if (internals) > + rte_free(internals); > + return -1; > +} > + > +/* > + * convert ascii to int > + */ > +static inline int > +ascii_to_u32(const char *key __rte_unused, > + const char *value, void *extra_args __rte_unused) > +{ > + *(uint32_t *)extra_args = atoi(value); > + return 0; > +} > + > +static inline int > +ascii_to_u32_array(const char *key __rte_unused, > + const char *value, void *extra_args) > +{ > + struct array_s *pastreams = (struct array_s *)extra_args; > + int sval, eval; > + if (sscanf(value, "%d..%d", &sval, &eval) == 2) { > + int cnt; > + for (cnt = sval; cnt <= eval; cnt++) > + pastreams->value[pastreams->count++] = cnt; > + } else { > + pastreams->value[pastreams->count++] = atoi(value); > + } > + return 0; > +} > + > + > +static int DoNtpl(const char *ntplStr) > +{ > + NtConfigStream_t hCfgStream; > + NtNtplInfo_t ntplInfo; > + int status; > + > + status = (*_NT_ConfigOpen)(&hCfgStream, "capture"); > + if (status != NT_SUCCESS) { > + /* Get the status code as text */ > + (*_NT_ExplainError)(status, ebuf, sizeof(ebuf) - 1); > + fprintf(stderr, "NT_ConfigOpen() failed: %s\n", ebuf); > + return -1; > + } > + > + RTE_LOG(INFO, PMD, "NTPL : %s\n", ntplStr); > + status = (*_NT_NTPL)(hCfgStream, ntplStr, &ntplInfo, > + NT_NTPL_PARSER_VALIDATE_NORMAL); > + if (status != NT_SUCCESS) { > + /* Get the status code as text */ > + (*_NT_ExplainError)(status, ebuf, sizeof(ebuf) - 1); > + fprintf(stderr, "NT_NTPL() failed: %s\n", ebuf); > + fprintf(stderr, ">>> NTPL errorcode: %X\n", > + ntplInfo.u.errorData.errCode); > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); > + (*_NT_ConfigClose)(hCfgStream); > + return -1; > + } > + (*_NT_ConfigClose)(hCfgStream); > + return 0; > +} > + > + > +static int > +_nt_lib_open(void) > +{ > + char path[128]; > + strcpy(path, NAPATECH3_LIB_PATH); > + strcat(path, "/libntapi.so"); > + > + /* Load the library */ > + _libnt = dlopen(path, RTLD_NOW); > + if (_libnt == NULL) { > + /* Library does not exist. */ > + fprintf(stderr, "Failed to find needed library : %s\n", path); > + return -1; > + } > + _NT_Init = dlsym(_libnt, "NT_Init"); > + if (_NT_Init == NULL) { > + fprintf(stderr, "Failed to find \"NT_Init\" in %s\n", path); > + return -1; > + } > + > + _NT_ConfigOpen = dlsym(_libnt, "NT_ConfigOpen"); > + if (_NT_ConfigOpen == NULL) { > + fprintf(stderr, "Failed to find \"NT_ConfigOpen\" in %s\n", > + path); > + return -1; > + } > + _NT_ConfigClose = dlsym(_libnt, "NT_ConfigClose"); > + if (_NT_ConfigClose == NULL) { > + fprintf(stderr, "Failed to find \"NT_ConfigClose\" in %s\n", > + path); > + return -1; > + } > + _NT_NTPL = dlsym(_libnt, "NT_NTPL"); > + if (_NT_NTPL == NULL) { > + fprintf(stderr, "Failed to find \"NT_NTPL\" in %s\n", > + path); > + return -1; > + } > + > + _NT_InfoOpen = dlsym(_libnt, "NT_InfoOpen"); > + if (_NT_InfoOpen == NULL) { > + fprintf(stderr, "Failed to find \"NT_InfoOpen\" in %s\n", > + path); > + return -1; > + } > + _NT_InfoRead = dlsym(_libnt, "NT_InfoRead"); > + if (_NT_InfoRead == NULL) { > + fprintf(stderr, "Failed to find \"NT_InfoRead\" in %s\n", > + path); > + return -1; > + } > + _NT_InfoClose = dlsym(_libnt, "NT_InfoClose"); > + if (_NT_InfoClose == NULL) { > + fprintf(stderr, "Failed to find \"NT_InfoClose\" in %s\n", > + path); > + return -1; > + } > + > + _NT_ExplainError = dlsym(_libnt, "NT_ExplainError"); > + if (_NT_ExplainError == NULL) { > + fprintf(stderr, "Failed to find \"NT_ExplainError\" in %s\n", > + path); > + return -1; > + } > + _NT_NetTxOpen = dlsym(_libnt, "NT_NetTxOpen"); > + if (_NT_NetTxOpen == NULL) { > + fprintf(stderr, "Failed to find \"NT_NetTxOpen\" in %s\n", > + path); > + return -1; > + } > + _NT_NetTxRingbufferInit = dlsym(_libnt, "NT_NetTxRingbufferInit"); > + if (_NT_NetTxRingbufferInit == NULL) { > + fprintf(stderr, > + "Failed to find \"NT_NetTxRingbufferInit\" in %s\n", > + path); > + return -1; > + } > + _NT_NetTxRingbufferFlush = dlsym(_libnt, "NT_NetTxRingbufferFlush"); > + if (_NT_NetTxRingbufferFlush == NULL) { > + fprintf(stderr, > + "Failed to find \"NT_NetTxRingbufferFlush\" in %s\n", > + path); > + return -1; > + } > + _NT_NetTxRingbufferTransmitPacket = > + dlsym(_libnt, "NT_NetTxRingbufferTransmitPacket"); > + if (_NT_NetTxRingbufferTransmitPacket == NULL) { > + fprintf(stderr, > + "Failed to find \"NT_NetTxRingbufferTransmitPacket\"" > + " in %s\n", path); > + return -1; > + } > + _NT_NetTxRingbufferDone = dlsym(_libnt, "NT_NetTxRingbufferDone"); > + if (_NT_NetTxRingbufferDone == NULL) { > + fprintf(stderr, > + "Failed to find \"NT_NetTxRingbufferDone\" in %s\n", > + path); > + return -1; > + } > + _NT_NetTxClose = dlsym(_libnt, "NT_NetTxClose"); > + if (_NT_NetTxClose == NULL) { > + fprintf(stderr, "Failed to find \"NT_NetTxClose\" in %s\n", > + path); > + return -1; > + } > + > + _NT_NetRxOpen = dlsym(_libnt, "NT_NetRxOpen"); > + if (_NT_NetRxOpen == NULL) { > + fprintf(stderr, "Failed to find \"NT_NetRxOpen\" in %s\n", > + path); > + return -1; > + } > + _NT_NetRxGet = dlsym(_libnt, "NT_NetRxGet"); > + if (_NT_NetRxGet == NULL) { > + fprintf(stderr, "Failed to find \"NT_NetRxGet\" in %s\n", > + path); > + return -1; > + } > + _NT_NetRxRelease = dlsym(_libnt, "NT_NetRxRelease"); > + if (_NT_NetRxRelease == NULL) { > + fprintf(stderr, "Failed to find \"NT_NetRxRelease\" in %s\n", > + path); > + return -1; > + } > + _NT_NetRxClose = dlsym(_libnt, "NT_NetRxClose"); > + if (_NT_NetRxClose == NULL) { > + fprintf(stderr, "Failed to find \"NT_NetRxClose\" in %s\n", > + path); > + return -1; > + } > + > + return 0; > +} > + > + > + > +static int > +rte_pmd_ntnic_devinit(const char *name, const char *params) > +{ > + unsigned int numa_node; > + int ret = 0; > + struct rte_kvargs *kvlist; > + struct rte_eth_dev *eth_dev; > + unsigned int i; > + uint32_t rxqueues = 0; > + uint32_t txqueues = 0; > + uint32_t port = 0; > + uint32_t portend = (uint32_t)-1; > + uint32_t hash = (uint32_t)-1; > + struct array_s astreamids; > + > + static int first = 1; > + static int stream_id; > + char ntplStr[512]; > + > + astreamids.count = 0; > + > + RTE_LOG(INFO, PMD, "Initializing pmd_ntnic for %s\n", name); > + > + numa_node = rte_socket_id(); > + > + kvlist = rte_kvargs_parse(params, valid_arguments); > + if (kvlist == NULL) > + return -1; > + > + /* Get port to use for Rx/Tx */ > + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORT_ARG); > + if (i) { > + assert(i == 1); > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORT_ARG, > + &ascii_to_u32, &port); > + } > + /* If Rx port merge is need, her the portend is specified */ > + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORTEND_ARG); > + if (i) { > + assert(i == 1); > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORTEND_ARG, > + &ascii_to_u32, &portend); > + } > + > + /* Get # RX queues */ > + i = rte_kvargs_count(kvlist, ETH_NTNIC_RXQUEUES_ARG); > + if (i) { > + assert(i == 1); > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_RXQUEUES_ARG, > + &ascii_to_u32, &rxqueues); > + } > + /* Get # TX queues */ > + i = rte_kvargs_count(kvlist, ETH_NTNIC_TXQUEUES_ARG); > + if (i) { > + assert(i == 1); > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_TXQUEUES_ARG, > + &ascii_to_u32, &txqueues); > + } > + > + /* Get list of streamIds - if used */ > + i = rte_kvargs_count(kvlist, ETH_NTNIC_STREAMID_ARG); > + if (i) { > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_STREAMID_ARG, > + &ascii_to_u32_array, &astreamids); > + } > + > + /* Get an alternative hash algorithm */ > + i = rte_kvargs_count(kvlist, ETH_NTNIC_HASH_ARG); > + if (i) { > + assert(i == 1); > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_HASH_ARG, > + &ascii_to_u32, &hash); > + } > + > + /* check portend and streamids */ > + if (portend != (uint32_t)-1 && astreamids.count) { > + RTE_LOG(ERR, PMD, "Cannot specify portend when one or more" > + " streamid's are specified\n"); > + return -1; > + } > + rte_kvargs_free(kvlist); > + > + if (ret < 0) > + return -1; > + > + ret = _nt_lib_open(); > + if (ret < 0) > + return -1; > + > + if (first) > + (*_NT_Init)(NTAPI_VERSION); > + > + if (astreamids.count) { > + first = 0; > + /* > + * if a specific streamid specified then use that otherwise, > + * port defaults to Tx port only > + */ > + if (rte_pmd_init_internals(name, rxqueues, txqueues, numa_node, > + &astreamids, port, ð_dev) < 0) > + return -1; > + } else { > + struct array_s astrids; > + if (first) { > + /* Delete all NTPL */ > + sprintf(ntplStr, "Delete=All"); > + if (DoNtpl(ntplStr) != 0) > + return -1; > + first = 0; > + } > + astrids.count = 1; > + astrids.value[0] = stream_id; > + if (rte_pmd_init_internals(name, rxqueues, txqueues, numa_node, > + &astrids, port, ð_dev) < 0) > + return -1; > + > + /* Assign the traffic */ > + if (portend != (uint32_t)-1) { > + sprintf(ntplStr, > + "Assign[streamid=(%d..%d);Descriptor=NT]" > + "=port==(%d..%d)", > + stream_id, > + (stream_id + rxqueues - 1), port, portend); > + > + } else { > + sprintf(ntplStr, > + "Assign[streamid=(%d..%d);Descriptor=NT]" > + "=port==%d", > + stream_id, > + (stream_id + rxqueues - 1), port); > + } > + if (DoNtpl(ntplStr) != 0) > + return -1; > + > + if (hash != (uint32_t)-1) { > + switch (hash) { > + default: > + case 1: > + DoNtpl("HashMode=HashRoundRobin"); > + break; > + case 2: > + DoNtpl("HashMode=Hash2TupleSorted"); > + break; > + case 3: > + DoNtpl("HashMode=Hash5TupleSorted"); > + break; > + } > + } > + > + stream_id += rxqueues; > + } > + > + eth_dev->rx_pkt_burst = eth_ntnic_rx; > + eth_dev->tx_pkt_burst = eth_ntnic_tx_ringbuffer; > + > + return 0; > +} > + > +static int > +rte_pmd_ntnic_devuninit(const char *name) > +{ > + (void)name; > + if (_libnt != NULL) > + dlclose(_libnt); > + return 0; > +} > + > + > +static struct rte_driver pmd_ntnic_drv = { > + .type = PMD_VDEV, > + .init = rte_pmd_ntnic_devinit, > + .uninit = rte_pmd_ntnic_devuninit, > +}; > + > +PMD_REGISTER_DRIVER(pmd_ntnic_drv, eth_ntnic); > +DRIVER_REGISTER_PARAM_STRING(eth_ntnic, > + "port=<int> " > + "rxqs=<int>" > + "txqs=<int>" > + "hash=<int>" > + "streamids=<int..int>"); > + > diff --git a/drivers/net/ntnic/rte_pmd_ntnic_version.map b/drivers/net/ntnic/rte_pmd_ntnic_version.map > new file mode 100644 > index 0000000..ef35398 > --- /dev/null > +++ b/drivers/net/ntnic/rte_pmd_ntnic_version.map > @@ -0,0 +1,4 @@ > +DPDK_2.0 { > + > + local: *; > +}; > diff --git a/mk/rte.app.mk b/mk/rte.app.mk > index 1a0095b..9ef357f 100644 > --- a/mk/rte.app.mk > +++ b/mk/rte.app.mk > @@ -120,6 +120,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += -lrte_pmd_mpipe -lgxio > _LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += -lrte_pmd_nfp -lm > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += -lrte_pmd_null > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += -lrte_pmd_pcap -lpcap > + > _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD) += -lrte_pmd_qede -lz > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING) += -lrte_pmd_ring > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += -lrte_pmd_szedata2 -lsze2 > @@ -143,6 +144,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -lrte_pmd_kasumi > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += -L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi > endif # CONFIG_RTE_LIBRTE_CRYPTODEV > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += -lrte_pmd_ntnic > endif # !CONFIG_RTE_BUILD_SHARED_LIBS > > _LDLIBS-y += --no-whole-archive > @@ -163,6 +165,7 @@ endif > _LDLIBS-$(CONFIG_RTE_PORT_PCAP) += -lpcap > endif # !CONFIG_RTE_BUILD_SHARED_LIBS > > + > _LDLIBS-y += $(EXECENV_LDLIBS) > > LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS) > -- > 2.9.0 > > Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. > ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-09 13:51 ` Neil Horman @ 2016-09-10 7:58 ` Finn Christensen 2016-09-10 8:20 ` Thomas Monjalon 2016-09-12 12:32 ` Neil Horman 0 siblings, 2 replies; 22+ messages in thread From: Finn Christensen @ 2016-09-10 7:58 UTC (permalink / raw) To: Neil Horman; +Cc: dev, thomas.monjalon, stephen > -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: 9. september 2016 15:51 > To: Finn Christensen <fc@napatech.com> > Cc: dev@dpdk.org; thomas.monjalon@6wind.com; > stephen@networkplumber.org > Subject: Re: [PATCH v3] ntnic: add PMD driver > > On Fri, Sep 09, 2016 at 12:48:38PM +0000, Finn Christensen wrote: > > This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. > > > > This patch adds support for Napatech NICs to DPDK. This is the > > initial implementation. > > > > Signed-off-by: Finn Christensen <fc@napatech.com> > > --- > > v3: > > * Removed the need for binary libraries on build > > v2: > > * Added information how to build the PMD without NIC > > Board Support Package > > * Fixed some formatting issues > So, this is a step in the right direction, but I think its solving the wrong > problem. If you have a dependency on an external library, thats ok, and > accessing it via dlopen makes it possible to build the library without having > that library present, but it not really in keeping with the spirit of what I > meant. This driver is still effectively dependent on a binary blob that we > have > no visibility into. The better solution is releasing the source for the ntnic > and ntos libraries. The license file in the referenced git tree indicates its > BSD licensed, so I don't think there should be a problem in doing that. > > Neil > No, unfortunately the ntapi is not BSD licensed, only the header files that you can freely download are. We are building this NT NIC by using parts or our technology from our capture adapters and that is using closed source software. We are new to opensource and we want to go that way, but we haven't yet a complete stand-alone driver ready that we can put into the DPDK PMD to have a complete self contained and open sourced DPDK PMD, that only needs the actual HW NIC plugged in to run. Therefore this version is implemented as a virtual device, exactly like the PCAP PMD driver is, and it runs on top of a driver that follows the NIC itself. In regards to the DPDK functionality we do not see that anything is missing. I cannot either see where we should add source code, because it is not part of the DPDK package and it should not be either. One of the things I really liked about the DPDK open source project is that it uses BSD licensing not GPL. Therefore, I must admit, we completely failed to see that the "spirit" of the DPDK community is not really BSD. Our view of this community was that the main driving force of it was to be able to make DPDK run on everything anywhere effectively, in a global contributing community, without any legally constrains prohibiting us to do so. However, this is our standing, and I don't know what else to do. Please advise or NAK this PMD. Finn > > --- > > MAINTAINERS | 5 + > > config/common_base | 6 + > > doc/guides/nics/features/ntnic.ini | 14 + > > doc/guides/nics/index.rst | 1 + > > doc/guides/nics/ntnic.rst | 145 ++++ > > drivers/net/Makefile | 1 + > > drivers/net/ntnic/Makefile | 66 ++ > > drivers/net/ntnic/rte_eth_ntnic.c | 1150 > +++++++++++++++++++++++++++ > > drivers/net/ntnic/rte_pmd_ntnic_version.map | 4 + > > mk/rte.app.mk | 3 + > > 10 files changed, 1395 insertions(+) > > create mode 100644 doc/guides/nics/features/ntnic.ini > > create mode 100644 doc/guides/nics/ntnic.rst > > create mode 100644 drivers/net/ntnic/Makefile > > create mode 100644 drivers/net/ntnic/rte_eth_ntnic.c > > create mode 100644 drivers/net/ntnic/rte_pmd_ntnic_version.map > > > > diff --git a/MAINTAINERS b/MAINTAINERS > > index bc9aa02..d3e5f56 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -356,6 +356,11 @@ M: Sony Chacko <sony.chacko@qlogic.com> > > F: drivers/net/qede/ > > F: doc/guides/nics/qede.rst > > > > +Napatech ntnic > > +M: Finn Christensen <fc@napatech.com> > > +F: drivers/net/ntnic/ > > +F: doc/guides/nics/ntnic.rst > > + > > RedHat virtio > > M: Huawei Xie <huawei.xie@intel.com> > > M: Yuanhan Liu <yuanhan.liu@linux.intel.com> > > diff --git a/config/common_base b/config/common_base > > index 7830535..4a3b2b8 100644 > > --- a/config/common_base > > +++ b/config/common_base > > @@ -309,6 +309,12 @@ CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16 > > CONFIG_RTE_LIBRTE_PMD_PCAP=n > > > > # > > +# Compile software PMD backed by NTNIC files > > +# > > +CONFIG_RTE_LIBRTE_PMD_NTNIC=n > > + > > + > > +# > > # Compile link bonding PMD library > > # > > CONFIG_RTE_LIBRTE_PMD_BOND=y > > diff --git a/doc/guides/nics/features/ntnic.ini > b/doc/guides/nics/features/ntnic.ini > > new file mode 100644 > > index 0000000..4237e6e > > --- /dev/null > > +++ b/doc/guides/nics/features/ntnic.ini > > @@ -0,0 +1,14 @@ > > +; > > +; Supported features of the 'ntnic' network poll mode driver. > > +; > > +; Refer to default.ini for the full list of available PMD features. > > +; > > +[Features] > > +Link status = Y > > +Queue start/stop = Y > > +Promiscuous mode = Y > > +Jumbo frame = Y > > +Basic stats = Y > > +Multiprocess aware = Y > > +x86-64 = Y > > +Usage doc = Y > > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > > index 92d56a5..5c4205c 100644 > > --- a/doc/guides/nics/index.rst > > +++ b/doc/guides/nics/index.rst > > @@ -52,6 +52,7 @@ Network Interface Controller Drivers > > qede > > szedata2 > > thunderx > > + ntnic > > virtio > > vhost > > vmxnet3 > > diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst > > new file mode 100644 > > index 0000000..f9398db > > --- /dev/null > > +++ b/doc/guides/nics/ntnic.rst > > @@ -0,0 +1,145 @@ > > +.. BSD LICENSE > > + Copyright (c) 2016 Napatech A/S > > + All rights reserved. > > + > > + Redistribution and use in source and binary forms, with or without > > + modification, are permitted provided that the following conditions > > + are met: > > + > > + * Redistributions of source code must retain the above copyright > > + notice, this list of conditions and the following disclaimer. > > + * Redistributions in binary form must reproduce the above copyright > > + notice, this list of conditions and the following disclaimer in > > + the documentation and/or other materials provided with the > > + distribution. > > + * Neither the name of Napatech nor the names of its > > + contributors may be used to endorse or promote products derived > > + from this software without specific prior written permission. > > + > > + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > > + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > NOT > > + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > FITNESS FOR > > + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT > > + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, > > + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, > BUT NOT > > + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS > OF USE, > > + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED > AND ON ANY > > + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > TORT > > + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE > > + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > > + > > +NTNIC Poll Mode Driver > > +====================== > > + > > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements > support > > +for **Napatech NIC** 40/50 Gbps adapters. > > +This PMD is implemented as a pure software virtual device and must be > created > > +by using the EAL --vdev=parameter (parameters are explained i detail > later). > > +It runs on top of the Napatech NFV NIC Board Support Package that must > be > > +installed and started. > > +If no Napatech NIC is available, you can download the Napatech NTAPI > includes > > +to build with. > > + > > +Supported Features > > +------------------ > > + > > +- RSS (Receive Side Scaling) > > +- TSS (Transmit Side Scaling) > > +- Promiscuous mode > > +- Basic statistics > > + > > + > > +Prerequisites > > +------------- > > + > > +Requires Napatech NIC and Napatech NIC Board Support Package > installed and > > +running in version **0.3.0** or higher. > > +This includes external libraries and kernel driver for resources > > +allocations and initialization. > > +If build only is required, download the Napatech NTAPI to build against. > > + > > +Pre-Installation Configuration > > +------------------------------ > > + > > +Environment variables > > +~~~~~~~~~~~~~~~~~~~~~ > > + > > +In order to compile the Napatech NIC PMD, user must: > > + > > +* Export the environment variable NAPATECH3_PATH with the path > where > > + the Napatech Board Support Package was installed (default location is > > + /opt/napatech3). > > + Or if no Napatech NIC available: > > + Download the Napatech NTAPI include files from Github: > > + `Napatech NTAPI <https://github.com/napa-tech/ntapi>`_. > > + > > + > > +Config File Options > > +~~~~~~~~~~~~~~~~~~~ > > + > > +- ``CONFIG_RTE_LIBRTE_PMD_NTNIC`` (default **n**) > > + > > +Using the NTNIC PMD from a DPDK application using EAL vdev parameter > > +-------------------------------------------------------------------- > > + > > +Napatech NIC PMD VDEV parameters > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > + > > +The Napatech NIC PMD vdev has the following command line parameters > to enable > > +its features. > > + > > +* port > > + Configures which NT port the vdev should use. > > + > > + --vdev eth_ntnic0,port=4,rxqs=1,txqs=1 > > + > > + This will create a DPDK port0 using NT port 4 > > + > > +* rxqs > > + Control how many receive queues a vdev should have. The traffic from a > vdev > > + will be load balanced to this amount of queues and each queue can be > handled > > + by its own lcore. > > + Note: The ntservice.ini HostBuffersRx must be configured to the sum of > all > > + receive queues across all vdevs. > > + > > + --vdev eth_ntnic0,port=0,rxqs=8,txqs=1 > > + > > + Traffic from NT port 0 will be split across > > + 8 queues meaning that 8 lcores can be > > + opened on DPDK port 0 and each will be > > + their individual traffic. > > + > > +* hash > > + Control which load balancing scheme should be used when running with > multiple > > + receive queues. > > + The possible values are: > > + 1 = HashRoundRobin > > + 2 = Hash2TupleSorted > > + 3 = Hash5TupleSorted > > + > > + --vdev eth_ntnic0,port=0,rxqs=4,hash=3,txqs=1 > > + > > + Create 4 queues from NT port 0 and > > + load balance the traffic using the > > + 5-tuple sorted algorithm. Each of the > > + 4 queues can be opened via DPDK port 0 > > + > > +* txqs > > + Control how many transmit queues a vdev should have. Each queue can > be used > > + by a lcore and each queue is independent for other queues. > > + Note: The ntservice.ini HostBuffersTx must be configured to the sum of > all > > + transmit queues across all vdevs. > > + > > + --vdev eth_ntnic0,port=0,txqs=32,rxqs=1 > > + > > + Enable 32 transmit queues on NT port 0 > > + > > + > > +Changes to Napatech driver configuration file ntservice.ini > > +----------------------------------------------------------- > > +Depending on the number of queues that is created, more hostbuffers > may be > > +needed pre-allocated by the Napatech NIC Driver. This is controlled in the > > +ntservice.ini file (default locations is /opt/napatech3/config). > > +The Napatech NIC Driver must be re-started when this configuration file is > > +changed. > > + > > diff --git a/drivers/net/Makefile b/drivers/net/Makefile > > index bc93230..aff29e9 100644 > > --- a/drivers/net/Makefile > > +++ b/drivers/net/Makefile > > @@ -55,6 +55,7 @@ DIRS- > $(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx > > DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio > > DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3 > > DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt > > +DIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += ntnic > > > > ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) > > DIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += vhost > > diff --git a/drivers/net/ntnic/Makefile b/drivers/net/ntnic/Makefile > > new file mode 100644 > > index 0000000..d816c56 > > --- /dev/null > > +++ b/drivers/net/ntnic/Makefile > > @@ -0,0 +1,66 @@ > > +# BSD LICENSE > > +# > > +# Copyright(c) 2016 Napatech A/S. All rights reserved. > > +# All rights reserved. > > +# > > +# Redistribution and use in source and binary forms, with or without > > +# modification, are permitted provided that the following conditions > > +# are met: > > +# > > +# * Redistributions of source code must retain the above copyright > > +# notice, this list of conditions and the following disclaimer. > > +# * Redistributions in binary form must reproduce the above copyright > > +# notice, this list of conditions and the following disclaimer in > > +# the documentation and/or other materials provided with the > > +# distribution. > > +# * Neither the name of Napatech nor the names of its > > +# contributors may be used to endorse or promote products derived > > +# from this software without specific prior written permission. > > +# > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > NOT > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > FITNESS FOR > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, > BUT NOT > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; > LOSS OF USE, > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED > AND ON ANY > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > TORT > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > > + > > +include $(RTE_SDK)/mk/rte.vars.mk > > + > > +# > > +# library name > > +# > > +LIB = librte_pmd_ntnic.a > > + > > +CFLAGS += -O3 -g > > +CFLAGS += $(WERROR_FLAGS) > > +CFLAGS += -I$(NAPATECH3_PATH)/include > > +CFLAGS += -DNAPATECH3_LIB_PATH=\"$(NAPATECH3_PATH)/lib\" > > +LDLIBS += -ldl > > + > > +EXPORT_MAP := rte_pmd_ntnic_version.map > > + > > +LIBABIVER := 1 > > + > > +# > > +# all source are stored in SRCS-y > > +# > > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += rte_eth_ntnic.c > > + > > +# > > +# Export include files > > +# > > +SYMLINK-y-include += > > + > > +# this lib depends upon: > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_eal > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mbuf > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mempool > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_ether > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_kvargs > > + > > +include $(RTE_SDK)/mk/rte.lib.mk > > diff --git a/drivers/net/ntnic/rte_eth_ntnic.c > b/drivers/net/ntnic/rte_eth_ntnic.c > > new file mode 100644 > > index 0000000..f83e94d > > --- /dev/null > > +++ b/drivers/net/ntnic/rte_eth_ntnic.c > > @@ -0,0 +1,1150 @@ > > +/*- > > + * BSD LICENSE > > + * > > + * Copyright(c) 2016 Napatech A/S. All rights reserved. > > + * All rights reserved. > > + * > > + * Redistribution and use in source and binary forms, with or without > > + * modification, are permitted provided that the following conditions > > + * are met: > > + * > > + * * Redistributions of source code must retain the above copyright > > + * notice, this list of conditions and the following disclaimer. > > + * * Redistributions in binary form must reproduce the above copyright > > + * notice, this list of conditions and the following disclaimer in > > + * the documentation and/or other materials provided with the > > + * distribution. > > + * * Neither the name of Napatech nor the names of its > > + * contributors may be used to endorse or promote products derived > > + * from this software without specific prior written permission. > > + * > > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > NOT > > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > FITNESS FOR > > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT > > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, > > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, > BUT NOT > > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; > LOSS OF USE, > > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED > AND ON ANY > > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > TORT > > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > OF THE USE > > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > > + */ > > + > > +#include <time.h> > > +#include <dlfcn.h> > > +#include <rte_mbuf.h> > > +#include <rte_ethdev.h> > > +#include <rte_malloc.h> > > +#include <rte_memcpy.h> > > +#include <rte_string_fns.h> > > +#include <rte_cycles.h> > > +#include <rte_kvargs.h> > > +#include <rte_dev.h> > > +#include <net/if.h> > > +#include <nt.h> > > + > > +#define ETH_NTNIC_PORT_ARG "port" > > +#define ETH_NTNIC_PORTEND_ARG "portend" > > +#define ETH_NTNIC_RXQUEUES_ARG "rxqs" > > +#define ETH_NTNIC_TXQUEUES_ARG "txqs" > > +#define ETH_NTNIC_STREAMID_ARG "streamid" > > +#define ETH_NTNIC_HASH_ARG "hash" > > + > > +#define HW_MAX_PKT_LEN 10000 > > +#define HW_MTU (HW_MAX_PKT_LEN - ETHER_HDR_LEN - > ETHER_CRC_LEN) > > + > > +#define MAX_RX_QUEUES 64 > > +#define MAX_TX_QUEUES 64 > > +#define MAX_NTNIC_PORTS 32 > > + > > +static void *_libnt; > > + > > +/* NTAPI library functions */ > > +int (*_NT_Init)(uint32_t); > > +int (*_NT_NetRxOpen)(NtNetStreamRx_t *, > > + const char *, enum NtNetInterface_e, uint32_t, int); > > +int (*_NT_NetRxGet)(NtNetStreamRx_t, NtNetBuf_t *, int); > > +int (*_NT_NetRxRelease)(NtNetStreamRx_t, NtNetBuf_t); > > +int (*_NT_NetRxClose)(NtNetStreamRx_t); > > +char *(*_NT_ExplainError)(int, char *, uint32_t); > > +int (*_NT_NetTxOpen)(NtNetStreamTx_t *, > > + const char *, uint64_t, uint32_t, uint32_t); > > +int (*_NT_NetTxRingbufferInit)(NtNetStreamTx_t, uint32_t); > > +int (*_NT_NetTxRingbufferTransmitPacket)(NtNetStreamTx_t, > > + uint8_t *, uint16_t); > > +int (*_NT_NetTxRingbufferFlush)(NtNetStreamTx_t); > > +void (*_NT_NetTxRingbufferDone)(NtNetStreamTx_t); > > +int (*_NT_NetTxClose)(NtNetStreamTx_t); > > +int (*_NT_InfoOpen)(NtInfoStream_t *, const char *); > > +int (*_NT_InfoRead)(NtInfoStream_t, NtInfo_t *); > > +int (*_NT_InfoClose)(NtInfoStream_t); > > +int (*_NT_ConfigOpen)(NtConfigStream_t *, const char *); > > +int (*_NT_ConfigClose)(NtConfigStream_t); > > +int (*_NT_NTPL)(NtConfigStream_t, const char *, > > + NtNtplInfo_t *, uint32_t); > > + > > +static char ebuf[1024]; > > + > > +struct array_s { > > + uint32_t value[MAX_RX_QUEUES]; > > + int count; > > +}; > > + > > +static volatile uint16_t port_locks[MAX_NTNIC_PORTS]; > > + > > +struct ntnic_rx_queue { > > + NtNetStreamRx_t net_rx[MAX_RX_QUEUES]; /* NT Rx streams */ > > + int rx_idx; /* Current Rx stream index */ > > + struct rte_mempool *mb_pool; /* mbuf memory pool */ > > + uint16_t buf_size; /* Size of data area in mbuf */ > > + NtNetBuf_t seg; /* The current NT data segment */ > > + struct NtNetBuf_s pkt; /* The current packet */ > > + volatile unsigned long rx_pkts; /* Rx packet statistics */ > > + volatile unsigned long err_pkts; /* Rx error packet statistics */ > > + struct array_s astreamids; /* NT streams to read from */ > > + int enabled; /* Enabling/disabling of this queue */ > > +}; > > + > > +struct ntnic_tx_queue { > > + NtNetStreamTx_t net_tx; /* NT Tx stream */ > > + volatile unsigned long tx_pkts; /* Tx packet statistics */ > > + volatile unsigned long err_pkts; /* Tx error packet stat */ > > + volatile uint16_t *plock; /* Per port transmit atomic lock */ > > + uint32_t port; /* Tx port for this queue */ > > + int enabled; /* Enabling/disabling of this queue */ > > + int fcs_add; /* If needs added room for FCS */ > > +}; > > + > > +struct pmd_internals { > > + struct ntnic_rx_queue rxq[MAX_RX_QUEUES]; /* Array of Rx queues > */ > > + struct ntnic_tx_queue txq[MAX_TX_QUEUES]; /* Array of Tx queues > */ > > + int if_index; /* Itf index always 0 - no bonding */ > > + unsigned int nb_rx_queues; /* Number of Rx queues configured > */ > > + unsigned int nb_tx_queues; /* Number of Tx queues configured > */ > > + int MAC_50G; /* True if 50G ports */ > > +}; > > + > > +static const char *valid_arguments[] = { > > + ETH_NTNIC_PORT_ARG, > > + ETH_NTNIC_PORTEND_ARG, > > + ETH_NTNIC_RXQUEUES_ARG, > > + ETH_NTNIC_TXQUEUES_ARG, > > + ETH_NTNIC_STREAMID_ARG, > > + ETH_NTNIC_HASH_ARG, > > + NULL > > +}; > > + > > +static struct ether_addr eth_addr[MAX_NTNIC_PORTS]; > > +static const char *drivername = "NTNIC PMD"; > > + > > +static int > > +eth_ntnic_rx_jumbo(struct rte_mempool *mb_pool, > > + struct rte_mbuf *mbuf, > > + const u_char *data, > > + uint16_t data_len) > > +{ > > + struct rte_mbuf *m = mbuf; > > + > > + /* Copy the first segment. */ > > + uint16_t len = rte_pktmbuf_tailroom(mbuf); > > + > > + rte_memcpy(rte_pktmbuf_append(mbuf, len), data, len); > > + data_len -= len; > > + data += len; > > + > > + while (data_len > 0) { > > + /* Allocate next mbuf and point to that. */ > > + m->next = rte_pktmbuf_alloc(mb_pool); > > + > > + if (unlikely(!m->next)) > > + return -1; > > + > > + m = m->next; > > + > > + /* Headroom is not needed in chained mbufs. */ > > + rte_pktmbuf_prepend(m, rte_pktmbuf_headroom(m)); > > + m->pkt_len = 0; > > + m->data_len = 0; > > + > > + /* Copy next segment. */ > > + len = RTE_MIN(rte_pktmbuf_tailroom(m), data_len); > > + rte_memcpy(rte_pktmbuf_append(m, len), data, len); > > + > > + mbuf->nb_segs++; > > + data_len -= len; > > + data += len; > > + } > > + > > + return mbuf->nb_segs; > > +} > > + > > + > > + > > +static uint16_t > > +eth_ntnic_rx(void *queue, > > + struct rte_mbuf **bufs, > > + uint16_t nb_pkts) > > +{ > > + unsigned int i; > > + struct rte_mbuf *mbuf; > > + struct ntnic_rx_queue *rx_q = queue; > > + uint16_t num_rx = 0; > > + uint16_t data_len; > > + > > + if (unlikely(rx_q->net_rx[rx_q->rx_idx] == NULL || nb_pkts == 0)) > > + return 0; > > + > > + /* Do we have any data segment */ > > + if (rx_q->seg == NULL) { > > + /* Next stream - if multiple streams are combined */ > > + rx_q->rx_idx = > > + (rx_q->rx_idx < > > + (rx_q->astreamids.count - 1)) ? rx_q->rx_idx + 1 > > + : 0; > > + > > + if ((*_NT_NetRxGet)(rx_q->net_rx[rx_q->rx_idx], > > + &rx_q->seg, 0) != NT_SUCCESS) { > > + if (rx_q->seg != NULL) { > > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > > + rx_q->seg); > > + rx_q->seg = NULL; > > + } > > + return 0; > > + } > > + if (NT_NET_GET_SEGMENT_LENGTH(rx_q->seg)) { > > + /* Build a packet structure */ > > + _nt_net_build_pkt_netbuf(rx_q->seg, &rx_q->pkt); > > + } else { > > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > > + rx_q->seg); > > + rx_q->seg = NULL; > > + return 0; > > + } > > + } > > + > > + if (rte_mempool_get_bulk(rx_q->mb_pool, (void **)bufs, nb_pkts) > != 0) > > + return 0; > > + > > + for (i = 0; i < nb_pkts; i++) { > > + mbuf = bufs[i]; > > + rte_mbuf_refcnt_set(mbuf, 1); > > + rte_pktmbuf_reset(mbuf); > > + > > + /* HW slicing not supported */ > > + data_len = (uint16_t)NT_NET_GET_PKT_WIRE_LENGTH((&rx_q- > >pkt)) > > + - 4; > > + if (data_len <= rx_q->buf_size) { > > + /* Packet will fit in the mbuf, go ahead and copy */ > > + mbuf->pkt_len = mbuf->data_len; > > + data_len = mbuf->data_len; > > + rte_memcpy((u_char *)mbuf->buf_addr + > > + RTE_PKTMBUF_HEADROOM, > > + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), > > + mbuf->data_len); > > + } else { > > + /* Try read jumbo frame into multi mbufs. */ > > + if (unlikely(eth_ntnic_rx_jumbo(rx_q->mb_pool, > > + mbuf, > > + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), > > + data_len) == -1)) > > + break; > > + } > > + > > + mbuf->port = NT_NET_GET_PKT_RXPORT((&rx_q->pkt)); > > + num_rx++; > > + > > + /* Get the next packet if any */ > > + if (_nt_net_get_next_packet(rx_q->seg, > > + NT_NET_GET_SEGMENT_LENGTH(rx_q->seg), > > + &rx_q->pkt) == 0) { > > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > > + rx_q->seg); > > + rx_q->seg = NULL; > > + break; > > + } > > + > > + rx_q->rx_pkts++; > > + } > > + > > + if (num_rx < nb_pkts) { > > + rte_mempool_put_bulk(rx_q->mb_pool, > > + (void * const *)(bufs + num_rx), > > + nb_pkts - num_rx); > > + } > > + return num_rx; > > +} > > + > > + > > + > > + > > + > > +/* Callback to handle sending packets through a NT NIC. */ > > +static uint16_t > > +eth_ntnic_tx_ringbuffer(void *queue, > > + struct rte_mbuf **bufs, > > + uint16_t nb_pkts) > > +{ > > + unsigned int i; > > + struct ntnic_tx_queue *tx_q = queue; > > + int retval; > > + uint16_t old; > > + uint16_t new_flag = 0; > > + > > + > > + if (unlikely(tx_q == NULL || tx_q->net_tx == NULL || nb_pkts == 0)) > > + return 0; > > + > > + > > + do { > > + old = 0; > > + new_flag = 1; > > + retval = rte_atomic16_cmpset(tx_q->plock, old, new_flag); > > + } while (unlikely(retval == 0)); > > + > > + for (i = 0; i < nb_pkts; i++) { > > + /* Not allowed to TX less than 64 byte packets */ > > + if (bufs[i]->pkt_len < 60) > > + bufs[i]->pkt_len = 60; > > + > > + /* transmit a single packet */ > > + (*_NT_NetTxRingbufferTransmitPacket)(tx_q->net_tx, > > + rte_pktmbuf_mtod(bufs[i], uint8_t *), > > + bufs[i]->pkt_len + 4 - tx_q->fcs_add); > > + rte_pktmbuf_free(bufs[i]); > > + } > > + tx_q->tx_pkts += nb_pkts; > > + > > + *tx_q->plock = 0; > > + return nb_pkts; > > +} > > + > > + > > + > > +static int > > +eth_dev_start(struct rte_eth_dev *dev) > > +{ > > + struct pmd_internals *internals = dev->data->dev_private; > > + struct ntnic_rx_queue *rx_q = internals->rxq; > > + struct ntnic_tx_queue *tx_q = internals->txq; > > + uint queue; > > + int status, idx; > > + > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > + > > + if ((internals->nb_tx_queues | internals->nb_rx_queues) == 0) > > + return -1; > > + > > + for (queue = 0; queue < internals->nb_rx_queues; queue++) { > > + if (rx_q[queue].enabled) { > > + char str[128], val[10]; > > + str[0] = 0; > > + /* build list of streamids to log */ > > + for (idx = 0; idx < > > + rx_q[queue].astreamids.count; idx++) { > > + if (idx) > > + sprintf(val, ",%d", > > + rx_q[queue].astreamids.value[idx]); > > + else > > + sprintf(val, "%i", > > + rx_q[queue].astreamids.value[idx]); > > + strcat(str, val); > > + } > > + > > + for (idx = 0; idx < rx_q[queue].astreamids.count; > > + idx++) { > > + status = > > + (*_NT_NetRxOpen)(&rx_q[queue].net_rx[idx], > > + "DPDK", > > + NT_NET_INTERFACE_SEGMENT, > > + rx_q[queue].astreamids.value[idx], -1); > > + if (status != NT_SUCCESS) { > > + /* try packet interface instead */ > > + (*_NT_ExplainError)(status, > > + ebuf, > > + sizeof(ebuf)); > > + RTE_LOG(ERR, PMD, > > + "NT_NetRxOpen() failed: %s\n", > > + ebuf); > > + return -1; > > + } > > + } > > + } > > + } > > + > > + for (queue = 0; queue < internals->nb_tx_queues; queue++) { > > + if (tx_q[queue].enabled) { > > + RTE_LOG(INFO, PMD, "NTNIC: NT_NetTxOpen(%d)\n", > > + tx_q[queue].port); > > + status = (*_NT_NetTxOpen)(&tx_q[queue].net_tx, "DPDK", > > + 1 << tx_q[queue].port, > > + dev->pci_dev->numa_node, 0); > > + if (status != NT_SUCCESS) { > > + (*_NT_ExplainError)(status, ebuf, > > + sizeof(ebuf)); > > + RTE_LOG(INFO, PMD, > > + "NT_NetTxOpen(0x%X, %d, 0) failed: %s\n", > > + 1 << tx_q[queue].port, > > + dev->pci_dev->numa_node, ebuf); > > + return -1; > > + } > > + > > + /* Init and associate RTD Tx Ring buf to Tx handler */ > > + status = (*_NT_NetTxRingbufferInit)( > > + tx_q[queue].net_tx, > > + tx_q[queue].port); > > + if (status != NT_SUCCESS) { > > + (*_NT_ExplainError)(status, ebuf, > > + sizeof(ebuf)); > > + RTE_LOG(ERR, PMD, > > + "NT_NetRxOpen() failed: %s\n", > > + ebuf); > > + return -1; > > + } > > + } > > + tx_q[queue].plock = &port_locks[tx_q[queue].port]; > > + tx_q[queue].fcs_add = (internals->MAC_50G) ? 4 : 0; > > + } > > + > > + dev->data->dev_link.link_status = 1; > > + return 0; > > +} > > + > > + > > +static void > > +eth_dev_stop(struct rte_eth_dev *dev) > > +{ > > + struct pmd_internals *internals = dev->data->dev_private; > > + struct ntnic_rx_queue *rx_q = internals->rxq; > > + struct ntnic_tx_queue *tx_q = internals->txq; > > + uint q; > > + int idx; > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > + > > + for (q = 0; q < internals->nb_rx_queues; q++) { > > + if (rx_q[q].seg) { > > + (*_NT_NetRxRelease)(rx_q[q].net_rx[rx_q->rx_idx], > > + rx_q[q].seg); > > + rx_q[q].seg = NULL; > > + } > > + for (idx = 0; idx < rx_q[q].astreamids.count; idx++) { > > + if (rx_q[q].net_rx[idx]) > > + (void)(*_NT_NetRxClose)(rx_q[q].net_rx[idx]); > > + } > > + } > > + for (q = 0; q < internals->nb_tx_queues; q++) { > > + if (tx_q[q].net_tx) { > > + (*_NT_NetTxRingbufferDone)(tx_q[q].net_tx); > > + (void)(*_NT_NetTxClose)(tx_q[q].net_tx); > > + } > > + } > > + dev->data->dev_link.link_status = 0; > > +} > > + > > +static int > > +eth_dev_configure(struct rte_eth_dev *dev __rte_unused) > > +{ > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > + return 0; > > +} > > + > > +static void > > +eth_dev_info(struct rte_eth_dev *dev, > > + struct rte_eth_dev_info *dev_info) > > +{ > > + struct pmd_internals *internals = dev->data->dev_private; > > + dev_info->if_index = internals->if_index; > > + dev_info->driver_name = drivername; > > + dev_info->max_mac_addrs = 1; > > + dev_info->max_rx_pktlen = HW_MTU; > > + dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; > > + dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; > > + dev_info->min_rx_bufsize = 0; > > + dev_info->pci_dev = NULL; > > +} > > + > > +static void > > +eth_stats_get(struct rte_eth_dev *dev, > > + struct rte_eth_stats *igb_stats) > > +{ > > + unsigned int i; > > + unsigned long rx_total = 0; > > + unsigned long tx_total = 0; > > + unsigned long tx_err_total = 0; > > + const struct pmd_internals *internal = dev->data->dev_private; > > + > > + memset(igb_stats, 0, sizeof(*igb_stats)); > > + for (i = 0; > > + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal- > >nb_rx_queues; > > + i++) { > > + igb_stats->q_ipackets[i] = internal->rxq[i].rx_pkts; > > + rx_total += igb_stats->q_ipackets[i]; > > + } > > + for (i = 0; > > + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal- > >nb_tx_queues; > > + i++) { > > + igb_stats->q_opackets[i] = internal->txq[i].tx_pkts; > > + igb_stats->q_errors[i] = internal->txq[i].err_pkts; > > + tx_total += igb_stats->q_opackets[i]; > > + tx_err_total += igb_stats->q_errors[i]; > > + } > > + > > + igb_stats->ipackets = rx_total; > > + igb_stats->opackets = tx_total; > > + igb_stats->oerrors = tx_err_total; > > +} > > + > > +static void > > +eth_stats_reset(struct rte_eth_dev *dev) > > +{ > > + unsigned int i; > > + struct pmd_internals *internal = dev->data->dev_private; > > + > > + for (i = 0; i < internal->nb_rx_queues; i++) > > + internal->rxq[i].rx_pkts = 0; > > + for (i = 0; i < internal->nb_tx_queues; i++) { > > + internal->txq[i].tx_pkts = 0; > > + internal->txq[i].err_pkts = 0; > > + } > > +} > > + > > +static void > > +eth_dev_close(struct rte_eth_dev *dev __rte_unused) > > +{ > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > +} > > + > > +static void > > +eth_queue_release(void *q __rte_unused) > > +{ > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > +} > > + > > +static int > > +eth_link_update(struct rte_eth_dev *dev __rte_unused, > > + int wait_to_complete __rte_unused) > > +{ > > + return 0; > > +} > > + > > +static int > > +eth_rx_queue_setup(struct rte_eth_dev *dev, > > + uint16_t rx_queue_id, > > + uint16_t nb_rx_desc __rte_unused, > > + unsigned int socket_id __rte_unused, > > + const struct rte_eth_rxconf *rx_conf __rte_unused, > > + struct rte_mempool *mb_pool) > > +{ > > + struct rte_pktmbuf_pool_private *mbp_priv; > > + struct pmd_internals *internals = dev->data->dev_private; > > + struct ntnic_rx_queue *rx_q = &internals->rxq[rx_queue_id]; > > + > > + RTE_LOG(INFO, PMD, "NTNIC RX queue setup\n"); > > + rx_q->mb_pool = mb_pool; > > + dev->data->rx_queues[rx_queue_id] = rx_q; > > + > > + mbp_priv = rte_mempool_get_priv(rx_q->mb_pool); > > + rx_q->buf_size = (uint16_t)(mbp_priv->mbuf_data_room_size - > > + RTE_PKTMBUF_HEADROOM); > > + rx_q->enabled = 1; > > + return 0; > > +} > > + > > +static int > > +eth_tx_queue_setup(struct rte_eth_dev *dev __rte_unused, > > + uint16_t tx_queue_id __rte_unused, > > + uint16_t nb_tx_desc __rte_unused, > > + unsigned int socket_id __rte_unused, > > + const struct rte_eth_txconf *tx_conf __rte_unused) > > +{ > > + struct pmd_internals *internals = dev->data->dev_private; > > + RTE_LOG(INFO, PMD, "NTNIC TX queue setup\n"); > > + dev->data->tx_queues[tx_queue_id] = &internals- > >txq[tx_queue_id]; > > + internals->txq[tx_queue_id].enabled = 1; > > + return 0; > > +} > > + > > +static int _dev_set_mtu(struct rte_eth_dev *dev __rte_unused, uint16_t > mtu) > > +{ > > + if (mtu < 46 || mtu > HW_MTU) > > + return -EINVAL; > > + > > + return 0; > > +} > > + > > +static struct eth_dev_ops ops = { > > + .dev_start = eth_dev_start, > > + .dev_stop = eth_dev_stop, > > + .dev_close = eth_dev_close, > > + .mtu_set = _dev_set_mtu, > > + .dev_configure = eth_dev_configure, > > + .dev_infos_get = eth_dev_info, > > + .rx_queue_setup = eth_rx_queue_setup, > > + .tx_queue_setup = eth_tx_queue_setup, > > + .rx_queue_release = eth_queue_release, > > + .tx_queue_release = eth_queue_release, > > + .link_update = eth_link_update, > > + .stats_get = eth_stats_get, > > + .stats_reset = eth_stats_reset, > > +}; > > + > > +static int > > +rte_pmd_init_internals(const char *name, > > + const unsigned int nb_rx_queues, > > + const unsigned int nb_tx_queues, > > + const unsigned int numa_node, > > + struct array_s *pastreamids, > > + const uint32_t port, > > + struct rte_eth_dev **eth_dev) > > +{ > > + struct pmd_internals *internals = NULL; > > + struct rte_eth_dev_data *data = NULL; > > + struct rte_pci_device *pci_dev = NULL; > > + uint i, status; > > + char err_buf[NT_ERRBUF_SIZE]; > > + NtInfoStream_t info; > > + NtInfo_t info_port; > > + struct rte_eth_link pmd_link; > > + assert(nb_rx_queues < MAX_RX_QUEUES); > > + assert(nb_tx_queues < MAX_TX_QUEUES); > > + assert(port < MAX_NTNIC_PORTS); > > + > > + RTE_LOG(INFO, PMD, > > + "Creating ntnic-backend ethdev on numa socket %u\n", > > + numa_node); > > + > > + /* now do all data allocation - for eth_dev structure, dummy pci driver > > + * and internal (private) data > > + */ > > + data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node); > > + if (data == NULL) > > + goto error; > > + > > + pci_dev = rte_zmalloc_socket(name, sizeof(*pci_dev), 0, > numa_node); > > + if (pci_dev == NULL) > > + goto error; > > + > > + internals = rte_zmalloc_socket(name, sizeof(*internals), 0, > numa_node); > > + if (internals == NULL) > > + goto error; > > + > > + /* reserve an ethdev entry */ > > + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > > + if (*eth_dev == NULL) > > + goto error; > > + > > + /* Open the information stream */ > > + status = (*_NT_InfoOpen)(&info, "DPDK Info stream"); > > + if (status != NT_SUCCESS) { > > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > > + RTE_LOG(ERR, PMD, > > + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", > > + status, err_buf); > > + return status; > > + } > > + > > + /* Find local port offset */ > > + info_port.cmd = NT_INFO_CMD_READ_PORT_V7; > > + info_port.u.port_v7.portNo = (uint8_t)(port); > > + status = (*_NT_InfoRead)(info, &info_port); > > + if (status != 0) { > > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > > + RTE_LOG(ERR, PMD, > > + "ERROR: NT_InfoRead failed. Code 0x%x = %s\n", > > + status, err_buf); > > + return status; > > + } > > + > > + /* check for 50G MAC */ > > + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9506 || > > + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9509) { > > + internals->MAC_50G = 1; > > + } else { > > + internals->MAC_50G = 0; > > + /* Check for valid FPGAs (NFV NICs) */ > > + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != > > + 9507 && > > + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != > > + 9510) { > > + RTE_LOG(ERR, PMD, > > + "ERROR: NTNIC PMD does not support the NT adapter" > > + "- port %i\n", > > + port); > > + return -1; > > + } > > + } > > + > > + internals->nb_rx_queues = nb_rx_queues; > > + internals->nb_tx_queues = nb_tx_queues; > > + for (i = 0; i < nb_rx_queues; i++) { > > + int idx; > > + for (idx = 0; idx < pastreamids->count; idx++) { > > + internals->rxq[i].astreamids.value[idx] = > > + pastreamids->value[idx] + i; > > + } > > + > > + internals->rxq[i].astreamids.count = pastreamids->count; > > + > > + internals->rxq[i].seg = NULL; > > + internals->rxq[i].enabled = 0; > > + } > > + > > + for (i = 0; i < nb_tx_queues; i++) { > > + if (port < info_port.u.port_v7.data.adapterInfo.portOffset) { > > + RTE_LOG(ERR, PMD, "Error wrong port specified\n"); > > + return -1; > > + } > > + internals->txq[i].port = port; > > + internals->txq[i].enabled = 0; > > + } > > + > > + switch (info_port.u.port_v7.data.speed) { > > + case NT_LINK_SPEED_UNKNOWN: > > + pmd_link.link_speed = ETH_SPEED_NUM_1G; > > + break; > > + case NT_LINK_SPEED_10M: > > + pmd_link.link_speed = ETH_SPEED_NUM_10M; > > + break; > > + case NT_LINK_SPEED_100M: > > + pmd_link.link_speed = ETH_SPEED_NUM_100M; > > + break; > > + case NT_LINK_SPEED_1G: > > + pmd_link.link_speed = ETH_SPEED_NUM_1G; > > + break; > > + case NT_LINK_SPEED_10G: > > + pmd_link.link_speed = ETH_SPEED_NUM_10G; > > + break; > > + case NT_LINK_SPEED_40G: > > + pmd_link.link_speed = ETH_SPEED_NUM_40G; > > + break; > > + case NT_LINK_SPEED_50G: > > + pmd_link.link_speed = ETH_SPEED_NUM_50G; > > + break; > > + case NT_LINK_SPEED_100G: > > + pmd_link.link_speed = ETH_SPEED_NUM_100G; > > + break; > > + } > > + > > + memcpy(ð_addr[port].addr_bytes, > &info_port.u.port_v7.data.macAddress, > > + sizeof(eth_addr[port].addr_bytes)); > > + status = (*_NT_InfoClose)(info); > > + if (status != NT_SUCCESS) { > > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > > + RTE_LOG(ERR, PMD, > > + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", > > + status, err_buf); > > + return status; > > + } > > + > > + pmd_link.link_duplex = ETH_LINK_FULL_DUPLEX; > > + pmd_link.link_status = 0; > > + > > + internals->if_index = 0; > > + > > + pci_dev->numa_node = numa_node; > > + > > + data->dev_private = internals; > > + data->port_id = (*eth_dev)->data->port_id; > > + data->nb_rx_queues = (uint16_t)nb_rx_queues; > > + data->nb_tx_queues = (uint16_t)nb_tx_queues; > > + data->dev_link = pmd_link; > > + data->mac_addrs = ð_addr[port]; > > + data->numa_node = numa_node; > > + data->drv_name = drivername; > > + > > + (*eth_dev)->data = data; > > + (*eth_dev)->dev_ops = &ops; > > + (*eth_dev)->pci_dev = pci_dev; > > + > > + return 0; > > + > > +error: > > + if (data) > > + rte_free(data); > > + if (pci_dev) > > + rte_free(pci_dev); > > + if (internals) > > + rte_free(internals); > > + return -1; > > +} > > + > > +/* > > + * convert ascii to int > > + */ > > +static inline int > > +ascii_to_u32(const char *key __rte_unused, > > + const char *value, void *extra_args __rte_unused) > > +{ > > + *(uint32_t *)extra_args = atoi(value); > > + return 0; > > +} > > + > > +static inline int > > +ascii_to_u32_array(const char *key __rte_unused, > > + const char *value, void *extra_args) > > +{ > > + struct array_s *pastreams = (struct array_s *)extra_args; > > + int sval, eval; > > + if (sscanf(value, "%d..%d", &sval, &eval) == 2) { > > + int cnt; > > + for (cnt = sval; cnt <= eval; cnt++) > > + pastreams->value[pastreams->count++] = cnt; > > + } else { > > + pastreams->value[pastreams->count++] = atoi(value); > > + } > > + return 0; > > +} > > + > > + > > +static int DoNtpl(const char *ntplStr) > > +{ > > + NtConfigStream_t hCfgStream; > > + NtNtplInfo_t ntplInfo; > > + int status; > > + > > + status = (*_NT_ConfigOpen)(&hCfgStream, "capture"); > > + if (status != NT_SUCCESS) { > > + /* Get the status code as text */ > > + (*_NT_ExplainError)(status, ebuf, sizeof(ebuf) - 1); > > + fprintf(stderr, "NT_ConfigOpen() failed: %s\n", ebuf); > > + return -1; > > + } > > + > > + RTE_LOG(INFO, PMD, "NTPL : %s\n", ntplStr); > > + status = (*_NT_NTPL)(hCfgStream, ntplStr, &ntplInfo, > > + NT_NTPL_PARSER_VALIDATE_NORMAL); > > + if (status != NT_SUCCESS) { > > + /* Get the status code as text */ > > + (*_NT_ExplainError)(status, ebuf, sizeof(ebuf) - 1); > > + fprintf(stderr, "NT_NTPL() failed: %s\n", ebuf); > > + fprintf(stderr, ">>> NTPL errorcode: %X\n", > > + ntplInfo.u.errorData.errCode); > > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); > > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); > > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); > > + (*_NT_ConfigClose)(hCfgStream); > > + return -1; > > + } > > + (*_NT_ConfigClose)(hCfgStream); > > + return 0; > > +} > > + > > + > > +static int > > +_nt_lib_open(void) > > +{ > > + char path[128]; > > + strcpy(path, NAPATECH3_LIB_PATH); > > + strcat(path, "/libntapi.so"); > > + > > + /* Load the library */ > > + _libnt = dlopen(path, RTLD_NOW); > > + if (_libnt == NULL) { > > + /* Library does not exist. */ > > + fprintf(stderr, "Failed to find needed library : %s\n", path); > > + return -1; > > + } > > + _NT_Init = dlsym(_libnt, "NT_Init"); > > + if (_NT_Init == NULL) { > > + fprintf(stderr, "Failed to find \"NT_Init\" in %s\n", path); > > + return -1; > > + } > > + > > + _NT_ConfigOpen = dlsym(_libnt, "NT_ConfigOpen"); > > + if (_NT_ConfigOpen == NULL) { > > + fprintf(stderr, "Failed to find \"NT_ConfigOpen\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_ConfigClose = dlsym(_libnt, "NT_ConfigClose"); > > + if (_NT_ConfigClose == NULL) { > > + fprintf(stderr, "Failed to find \"NT_ConfigClose\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NTPL = dlsym(_libnt, "NT_NTPL"); > > + if (_NT_NTPL == NULL) { > > + fprintf(stderr, "Failed to find \"NT_NTPL\" in %s\n", > > + path); > > + return -1; > > + } > > + > > + _NT_InfoOpen = dlsym(_libnt, "NT_InfoOpen"); > > + if (_NT_InfoOpen == NULL) { > > + fprintf(stderr, "Failed to find \"NT_InfoOpen\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_InfoRead = dlsym(_libnt, "NT_InfoRead"); > > + if (_NT_InfoRead == NULL) { > > + fprintf(stderr, "Failed to find \"NT_InfoRead\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_InfoClose = dlsym(_libnt, "NT_InfoClose"); > > + if (_NT_InfoClose == NULL) { > > + fprintf(stderr, "Failed to find \"NT_InfoClose\" in %s\n", > > + path); > > + return -1; > > + } > > + > > + _NT_ExplainError = dlsym(_libnt, "NT_ExplainError"); > > + if (_NT_ExplainError == NULL) { > > + fprintf(stderr, "Failed to find \"NT_ExplainError\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NetTxOpen = dlsym(_libnt, "NT_NetTxOpen"); > > + if (_NT_NetTxOpen == NULL) { > > + fprintf(stderr, "Failed to find \"NT_NetTxOpen\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NetTxRingbufferInit = dlsym(_libnt, "NT_NetTxRingbufferInit"); > > + if (_NT_NetTxRingbufferInit == NULL) { > > + fprintf(stderr, > > + "Failed to find \"NT_NetTxRingbufferInit\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NetTxRingbufferFlush = dlsym(_libnt, > "NT_NetTxRingbufferFlush"); > > + if (_NT_NetTxRingbufferFlush == NULL) { > > + fprintf(stderr, > > + "Failed to find \"NT_NetTxRingbufferFlush\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NetTxRingbufferTransmitPacket = > > + dlsym(_libnt, "NT_NetTxRingbufferTransmitPacket"); > > + if (_NT_NetTxRingbufferTransmitPacket == NULL) { > > + fprintf(stderr, > > + "Failed to find \"NT_NetTxRingbufferTransmitPacket\"" > > + " in %s\n", path); > > + return -1; > > + } > > + _NT_NetTxRingbufferDone = dlsym(_libnt, > "NT_NetTxRingbufferDone"); > > + if (_NT_NetTxRingbufferDone == NULL) { > > + fprintf(stderr, > > + "Failed to find \"NT_NetTxRingbufferDone\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NetTxClose = dlsym(_libnt, "NT_NetTxClose"); > > + if (_NT_NetTxClose == NULL) { > > + fprintf(stderr, "Failed to find \"NT_NetTxClose\" in %s\n", > > + path); > > + return -1; > > + } > > + > > + _NT_NetRxOpen = dlsym(_libnt, "NT_NetRxOpen"); > > + if (_NT_NetRxOpen == NULL) { > > + fprintf(stderr, "Failed to find \"NT_NetRxOpen\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NetRxGet = dlsym(_libnt, "NT_NetRxGet"); > > + if (_NT_NetRxGet == NULL) { > > + fprintf(stderr, "Failed to find \"NT_NetRxGet\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NetRxRelease = dlsym(_libnt, "NT_NetRxRelease"); > > + if (_NT_NetRxRelease == NULL) { > > + fprintf(stderr, "Failed to find \"NT_NetRxRelease\" in %s\n", > > + path); > > + return -1; > > + } > > + _NT_NetRxClose = dlsym(_libnt, "NT_NetRxClose"); > > + if (_NT_NetRxClose == NULL) { > > + fprintf(stderr, "Failed to find \"NT_NetRxClose\" in %s\n", > > + path); > > + return -1; > > + } > > + > > + return 0; > > +} > > + > > + > > + > > +static int > > +rte_pmd_ntnic_devinit(const char *name, const char *params) > > +{ > > + unsigned int numa_node; > > + int ret = 0; > > + struct rte_kvargs *kvlist; > > + struct rte_eth_dev *eth_dev; > > + unsigned int i; > > + uint32_t rxqueues = 0; > > + uint32_t txqueues = 0; > > + uint32_t port = 0; > > + uint32_t portend = (uint32_t)-1; > > + uint32_t hash = (uint32_t)-1; > > + struct array_s astreamids; > > + > > + static int first = 1; > > + static int stream_id; > > + char ntplStr[512]; > > + > > + astreamids.count = 0; > > + > > + RTE_LOG(INFO, PMD, "Initializing pmd_ntnic for %s\n", name); > > + > > + numa_node = rte_socket_id(); > > + > > + kvlist = rte_kvargs_parse(params, valid_arguments); > > + if (kvlist == NULL) > > + return -1; > > + > > + /* Get port to use for Rx/Tx */ > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORT_ARG); > > + if (i) { > > + assert(i == 1); > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORT_ARG, > > + &ascii_to_u32, &port); > > + } > > + /* If Rx port merge is need, her the portend is specified */ > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORTEND_ARG); > > + if (i) { > > + assert(i == 1); > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORTEND_ARG, > > + &ascii_to_u32, &portend); > > + } > > + > > + /* Get # RX queues */ > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_RXQUEUES_ARG); > > + if (i) { > > + assert(i == 1); > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_RXQUEUES_ARG, > > + &ascii_to_u32, &rxqueues); > > + } > > + /* Get # TX queues */ > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_TXQUEUES_ARG); > > + if (i) { > > + assert(i == 1); > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_TXQUEUES_ARG, > > + &ascii_to_u32, &txqueues); > > + } > > + > > + /* Get list of streamIds - if used */ > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_STREAMID_ARG); > > + if (i) { > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_STREAMID_ARG, > > + &ascii_to_u32_array, &astreamids); > > + } > > + > > + /* Get an alternative hash algorithm */ > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_HASH_ARG); > > + if (i) { > > + assert(i == 1); > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_HASH_ARG, > > + &ascii_to_u32, &hash); > > + } > > + > > + /* check portend and streamids */ > > + if (portend != (uint32_t)-1 && astreamids.count) { > > + RTE_LOG(ERR, PMD, "Cannot specify portend when one or more" > > + " streamid's are specified\n"); > > + return -1; > > + } > > + rte_kvargs_free(kvlist); > > + > > + if (ret < 0) > > + return -1; > > + > > + ret = _nt_lib_open(); > > + if (ret < 0) > > + return -1; > > + > > + if (first) > > + (*_NT_Init)(NTAPI_VERSION); > > + > > + if (astreamids.count) { > > + first = 0; > > + /* > > + * if a specific streamid specified then use that otherwise, > > + * port defaults to Tx port only > > + */ > > + if (rte_pmd_init_internals(name, rxqueues, txqueues, > numa_node, > > + &astreamids, port, ð_dev) < 0) > > + return -1; > > + } else { > > + struct array_s astrids; > > + if (first) { > > + /* Delete all NTPL */ > > + sprintf(ntplStr, "Delete=All"); > > + if (DoNtpl(ntplStr) != 0) > > + return -1; > > + first = 0; > > + } > > + astrids.count = 1; > > + astrids.value[0] = stream_id; > > + if (rte_pmd_init_internals(name, rxqueues, txqueues, > numa_node, > > + &astrids, port, ð_dev) < 0) > > + return -1; > > + > > + /* Assign the traffic */ > > + if (portend != (uint32_t)-1) { > > + sprintf(ntplStr, > > + "Assign[streamid=(%d..%d);Descriptor=NT]" > > + "=port==(%d..%d)", > > + stream_id, > > + (stream_id + rxqueues - 1), port, portend); > > + > > + } else { > > + sprintf(ntplStr, > > + "Assign[streamid=(%d..%d);Descriptor=NT]" > > + "=port==%d", > > + stream_id, > > + (stream_id + rxqueues - 1), port); > > + } > > + if (DoNtpl(ntplStr) != 0) > > + return -1; > > + > > + if (hash != (uint32_t)-1) { > > + switch (hash) { > > + default: > > + case 1: > > + DoNtpl("HashMode=HashRoundRobin"); > > + break; > > + case 2: > > + DoNtpl("HashMode=Hash2TupleSorted"); > > + break; > > + case 3: > > + DoNtpl("HashMode=Hash5TupleSorted"); > > + break; > > + } > > + } > > + > > + stream_id += rxqueues; > > + } > > + > > + eth_dev->rx_pkt_burst = eth_ntnic_rx; > > + eth_dev->tx_pkt_burst = eth_ntnic_tx_ringbuffer; > > + > > + return 0; > > +} > > + > > +static int > > +rte_pmd_ntnic_devuninit(const char *name) > > +{ > > + (void)name; > > + if (_libnt != NULL) > > + dlclose(_libnt); > > + return 0; > > +} > > + > > + > > +static struct rte_driver pmd_ntnic_drv = { > > + .type = PMD_VDEV, > > + .init = rte_pmd_ntnic_devinit, > > + .uninit = rte_pmd_ntnic_devuninit, > > +}; > > + > > +PMD_REGISTER_DRIVER(pmd_ntnic_drv, eth_ntnic); > > +DRIVER_REGISTER_PARAM_STRING(eth_ntnic, > > + "port=<int> " > > + "rxqs=<int>" > > + "txqs=<int>" > > + "hash=<int>" > > + "streamids=<int..int>"); > > + > > diff --git a/drivers/net/ntnic/rte_pmd_ntnic_version.map > b/drivers/net/ntnic/rte_pmd_ntnic_version.map > > new file mode 100644 > > index 0000000..ef35398 > > --- /dev/null > > +++ b/drivers/net/ntnic/rte_pmd_ntnic_version.map > > @@ -0,0 +1,4 @@ > > +DPDK_2.0 { > > + > > + local: *; > > +}; > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk > > index 1a0095b..9ef357f 100644 > > --- a/mk/rte.app.mk > > +++ b/mk/rte.app.mk > > @@ -120,6 +120,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += > -lrte_pmd_mpipe -lgxio > > _LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += -lrte_pmd_nfp -lm > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += -lrte_pmd_null > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += -lrte_pmd_pcap -lpcap > > + > > _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD) += -lrte_pmd_qede -lz > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING) += -lrte_pmd_ring > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += - > lrte_pmd_szedata2 -lsze2 > > @@ -143,6 +144,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) > += -lrte_pmd_kasumi > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += - > L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi > > endif # CONFIG_RTE_LIBRTE_CRYPTODEV > > > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += -lrte_pmd_ntnic > > endif # !CONFIG_RTE_BUILD_SHARED_LIBS > > > > _LDLIBS-y += --no-whole-archive > > @@ -163,6 +165,7 @@ endif > > _LDLIBS-$(CONFIG_RTE_PORT_PCAP) += -lpcap > > endif # !CONFIG_RTE_BUILD_SHARED_LIBS > > > > + > > _LDLIBS-y += $(EXECENV_LDLIBS) > > > > LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS) > > -- > > 2.9.0 > > > > Disclaimer: This email and any files transmitted with it may contain > confidential information intended for the addressee(s) only. The information > is not to be surrendered or copied to unauthorized persons. If you have > received this communication in error, please notify the sender immediately > and delete this e-mail from your system. > > Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-10 7:58 ` Finn Christensen @ 2016-09-10 8:20 ` Thomas Monjalon 2016-09-10 18:31 ` Stephen Hemminger 2016-09-12 7:34 ` Finn Christensen 2016-09-12 12:32 ` Neil Horman 1 sibling, 2 replies; 22+ messages in thread From: Thomas Monjalon @ 2016-09-10 8:20 UTC (permalink / raw) To: Finn Christensen; +Cc: Neil Horman, dev, stephen 2016-09-10 07:58, Finn Christensen: > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > On Fri, Sep 09, 2016 at 12:48:38PM +0000, Finn Christensen wrote: > > > This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. > > > > > > This patch adds support for Napatech NICs to DPDK. This is the > > > initial implementation. > > > > > > Signed-off-by: Finn Christensen <fc@napatech.com> > > > --- > > > v3: > > > * Removed the need for binary libraries on build > > > v2: > > > * Added information how to build the PMD without NIC > > > Board Support Package > > > * Fixed some formatting issues > > > > So, this is a step in the right direction, but I think its solving the wrong > > problem. If you have a dependency on an external library, thats ok, and > > accessing it via dlopen makes it possible to build the library without having > > that library present, but it not really in keeping with the spirit of what I > > meant. This driver is still effectively dependent on a binary blob that we > > have > > no visibility into. The better solution is releasing the source for the ntnic > > and ntos libraries. The license file in the referenced git tree indicates its > > BSD licensed, so I don't think there should be a problem in doing that. > > > > Neil > > > No, unfortunately the ntapi is not BSD licensed, only the header files that > you can freely download are. > We are building this NT NIC by using parts or our technology from our > capture adapters and that is using closed source software. > > We are new to opensource and we want to go that way, but we haven't > yet a complete stand-alone driver ready that we can put into the DPDK > PMD to have a complete self contained and open sourced DPDK PMD, that > only needs the actual HW NIC plugged in to run. > Therefore this version is implemented as a virtual device, exactly like the > PCAP PMD driver is, and it runs on top of a driver that follows the NIC itself. > > In regards to the DPDK functionality we do not see that anything is missing. > I cannot either see where we should add source code, because it is not part > of the DPDK package and it should not be either. > > One of the things I really liked about the DPDK open source project is that it > uses BSD licensing not GPL. Therefore, I must admit, we completely failed > to see that the "spirit" of the DPDK community is not really BSD. Our view > of this community was that the main driving force of it was to be able to > make DPDK run on everything anywhere effectively, in a global contributing > community, without any legally constrains prohibiting us to do so. It is difficult to define what is the spirit of a community, especially only after few mail exchanges. I agree that running on everything anywhere is a nice goal. Here Neil, as a RedHat developer, is probably concerned about enabling your driver in a distribution. It seems your model is not compatible with the "anywhere goal" and will be disabled in that case, until it is fully open. > However, this is our standing, and I don't know what else to do. > Please advise or NAK this PMD. I do not remember having already seen such model in DPDK. So we need to think about the implications a bit more. (Comments/discussions are welcome) Thanks for your patience. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-10 8:20 ` Thomas Monjalon @ 2016-09-10 18:31 ` Stephen Hemminger 2016-09-12 8:08 ` Finn Christensen 2016-09-12 12:33 ` Neil Horman 2016-09-12 7:34 ` Finn Christensen 1 sibling, 2 replies; 22+ messages in thread From: Stephen Hemminger @ 2016-09-10 18:31 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dev, Finn Christensen, Neil Horman I think that if the driver is just a shim for proprietary code, then the vendor should just maintain it out of tree. 6wind and windriver already do this. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-10 18:31 ` Stephen Hemminger @ 2016-09-12 8:08 ` Finn Christensen 2016-09-12 12:33 ` Neil Horman 1 sibling, 0 replies; 22+ messages in thread From: Finn Christensen @ 2016-09-12 8:08 UTC (permalink / raw) To: Stephen Hemminger, Thomas Monjalon; +Cc: dev, Neil Horman Yes, this we can do without asking you guys, and we already do so. But that is not what this ntnic PMD is about. We have built a NT NIC out of our existing accelerator products. This NIC is built using an FPGA and is targeted towards NFV acceleration in virtualized environments. Now, we have seen good acceleration results in our tests by using DPDK (with small additions to DPDK). What we want to do is to make this part of open source DPDK and in general run this project in an open source spirit. We want to make further RFCs to the DPDK community about extensions to the DPDK that can accelerate packet processing, and this ntnic PMD is the first step in our participation/contribution into the DPDK project. Furthermore, we do not think in proprietary code with this project, and no further libraries or other external closed source dependencies will be added in upcoming contributions from our side (at least we have no plan in doing so). Finn From: Stephen Hemminger [mailto:stephen@networkplumber.org] Sent: 10. september 2016 20:31 To: Thomas Monjalon <thomas.monjalon@6wind.com> Cc: dev@dpdk.org; Finn Christensen <fc@napatech.com>; Neil Horman <nhorman@tuxdriver.com> Subject: Re: [PATCH v3] ntnic: add PMD driver I think that if the driver is just a shim for proprietary code, then the vendor should just maintain it out of tree. 6wind and windriver already do this. Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-10 18:31 ` Stephen Hemminger 2016-09-12 8:08 ` Finn Christensen @ 2016-09-12 12:33 ` Neil Horman 1 sibling, 0 replies; 22+ messages in thread From: Neil Horman @ 2016-09-12 12:33 UTC (permalink / raw) To: Stephen Hemminger; +Cc: Thomas Monjalon, dev, Finn Christensen On Sat, Sep 10, 2016 at 11:31:04AM -0700, Stephen Hemminger wrote: > I think that if the driver is just a shim for proprietary code, then the > vendor should just maintain it out of tree. 6wind and windriver already do > this. Agreed, this was my previous comment. Someone probably wants this driver, but I dont' want to maintain a shim. Until its ready to be all open source, just maintain it out of tree Neil ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-10 8:20 ` Thomas Monjalon 2016-09-10 18:31 ` Stephen Hemminger @ 2016-09-12 7:34 ` Finn Christensen 2016-11-21 13:47 ` Ferruh Yigit 1 sibling, 1 reply; 22+ messages in thread From: Finn Christensen @ 2016-09-12 7:34 UTC (permalink / raw) To: Thomas Monjalon; +Cc: Neil Horman, dev, stephen > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: 10. september 2016 10:20 > To: Finn Christensen <fc@napatech.com> > Cc: Neil Horman <nhorman@tuxdriver.com>; dev@dpdk.org; > stephen@networkplumber.org > Subject: Re: [PATCH v3] ntnic: add PMD driver > > 2016-09-10 07:58, Finn Christensen: > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > On Fri, Sep 09, 2016 at 12:48:38PM +0000, Finn Christensen wrote: > > > > This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. > > > > > > > > This patch adds support for Napatech NICs to DPDK. This is the > > > > initial implementation. > > > > > > > > Signed-off-by: Finn Christensen <fc@napatech.com> > > > > --- > > > > v3: > > > > * Removed the need for binary libraries on build > > > > v2: > > > > * Added information how to build the PMD without NIC > > > > Board Support Package > > > > * Fixed some formatting issues > > > > > > So, this is a step in the right direction, but I think its solving > > > the wrong problem. If you have a dependency on an external library, > > > thats ok, and accessing it via dlopen makes it possible to build the > > > library without having that library present, but it not really in > > > keeping with the spirit of what I meant. This driver is still > > > effectively dependent on a binary blob that we have no visibility > > > into. The better solution is releasing the source for the ntnic and > > > ntos libraries. The license file in the referenced git tree > > > indicates its BSD licensed, so I don't think there should be a problem in > doing that. > > > > > > Neil > > > > > No, unfortunately the ntapi is not BSD licensed, only the header files > > that you can freely download are. > > We are building this NT NIC by using parts or our technology from our > > capture adapters and that is using closed source software. > > > > We are new to opensource and we want to go that way, but we haven't > > yet a complete stand-alone driver ready that we can put into the DPDK > > PMD to have a complete self contained and open sourced DPDK PMD, that > > only needs the actual HW NIC plugged in to run. > > Therefore this version is implemented as a virtual device, exactly > > like the PCAP PMD driver is, and it runs on top of a driver that follows the > NIC itself. > > > > In regards to the DPDK functionality we do not see that anything is missing. > > I cannot either see where we should add source code, because it is not > > part of the DPDK package and it should not be either. > > > > One of the things I really liked about the DPDK open source project is > > that it uses BSD licensing not GPL. Therefore, I must admit, we > > completely failed to see that the "spirit" of the DPDK community is > > not really BSD. Our view of this community was that the main driving > > force of it was to be able to make DPDK run on everything anywhere > > effectively, in a global contributing community, without any legally > constrains prohibiting us to do so. > > It is difficult to define what is the spirit of a community, especially only after > few mail exchanges. > I agree that running on everything anywhere is a nice goal. > Here Neil, as a RedHat developer, is probably concerned about enabling your > driver in a distribution. It seems your model is not compatible with the > "anywhere goal" and will be disabled in that case, until it is fully open. The ntnic PMD is not enabled by default and I think it should not be either. To enable it in a distribution for general purposes seems wrong. In that respect we see no difference between the PCAP PMD and this ntnic PMD. > > However, this is our standing, and I don't know what else to do. > > Please advise or NAK this PMD. > > I do not remember having already seen such model in DPDK. > So we need to think about the implications a bit more. > (Comments/discussions are welcome) > Thanks for your patience. Thanks. I will be happy to discuss this further, so that we can get to a conclusion. If the outcome is that the majority of the community does not like the idea that upstream supported PMDs has external linking dependencies to closed source libraries, then it is ok with us(a pity though). But then it might be a good idea to make that decision clear to everybody else by putting in a clause into the contribution section of the DPDK guide, or somewhere else in the guide. In our opinion, the inclusion of the ntnic PMD into upstream DPDK, does not seem to be any different than that of the PCAP PMD, since that is also dependent on external header files and externally built libraries. Of course we see the difference in open source vs close source library. But we cannot see that is has any influence in the usage or functionality of the DPDK. Thanks for this discussion! Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-12 7:34 ` Finn Christensen @ 2016-11-21 13:47 ` Ferruh Yigit 2016-11-21 13:55 ` Finn Christensen 0 siblings, 1 reply; 22+ messages in thread From: Ferruh Yigit @ 2016-11-21 13:47 UTC (permalink / raw) To: Finn Christensen, Thomas Monjalon; +Cc: Neil Horman, dev, stephen On 9/12/2016 8:34 AM, fc at napatech.com (Finn Christensen) wrote: >> -----Original Message----- >> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com] >> Sent: 10. september 2016 10:20 >> To: Finn Christensen <fc at napatech.com> >> Cc: Neil Horman <nhorman at tuxdriver.com>; dev at dpdk.org; >> stephen at networkplumber.org >> Subject: Re: [PATCH v3] ntnic: add PMD driver >> >> 2016-09-10 07:58, Finn Christensen: >>> From: Neil Horman [mailto:nhorman at tuxdriver.com] >>>> On Fri, Sep 09, 2016 at 12:48:38PM +0000, Finn Christensen wrote: >>>>> This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. >>>>> >>>>> This patch adds support for Napatech NICs to DPDK. This is the >>>>> initial implementation. >>>>> >>>>> Signed-off-by: Finn Christensen <fc at napatech.com> >>>>> --- >>>>> v3: >>>>> * Removed the need for binary libraries on build >>>>> v2: >>>>> * Added information how to build the PMD without NIC >>>>> Board Support Package >>>>> * Fixed some formatting issues >>>> >>>> So, this is a step in the right direction, but I think its solving >>>> the wrong problem. If you have a dependency on an external library, >>>> thats ok, and accessing it via dlopen makes it possible to build the >>>> library without having that library present, but it not really in >>>> keeping with the spirit of what I meant. This driver is still >>>> effectively dependent on a binary blob that we have no visibility >>>> into. The better solution is releasing the source for the ntnic and >>>> ntos libraries. The license file in the referenced git tree >>>> indicates its BSD licensed, so I don't think there should be a problem in >> doing that. >>>> >>>> Neil >>>> >>> No, unfortunately the ntapi is not BSD licensed, only the header files >>> that you can freely download are. >>> We are building this NT NIC by using parts or our technology from our >>> capture adapters and that is using closed source software. >>> >>> We are new to opensource and we want to go that way, but we haven't >>> yet a complete stand-alone driver ready that we can put into the DPDK >>> PMD to have a complete self contained and open sourced DPDK PMD, that >>> only needs the actual HW NIC plugged in to run. >>> Therefore this version is implemented as a virtual device, exactly >>> like the PCAP PMD driver is, and it runs on top of a driver that follows the >> NIC itself. >>> >>> In regards to the DPDK functionality we do not see that anything is missing. >>> I cannot either see where we should add source code, because it is not >>> part of the DPDK package and it should not be either. >>> >>> One of the things I really liked about the DPDK open source project is >>> that it uses BSD licensing not GPL. Therefore, I must admit, we >>> completely failed to see that the "spirit" of the DPDK community is >>> not really BSD. Our view of this community was that the main driving >>> force of it was to be able to make DPDK run on everything anywhere >>> effectively, in a global contributing community, without any legally >> constrains prohibiting us to do so. >> >> It is difficult to define what is the spirit of a community, especially only after >> few mail exchanges. >> I agree that running on everything anywhere is a nice goal. >> Here Neil, as a RedHat developer, is probably concerned about enabling your >> driver in a distribution. It seems your model is not compatible with the >> "anywhere goal" and will be disabled in that case, until it is fully open. > > The ntnic PMD is not enabled by default and I think it should not be either. To > enable it in a distribution for general purposes seems wrong. In that respect > we see no difference between the PCAP PMD and this ntnic PMD. > >>> However, this is our standing, and I don't know what else to do. >>> Please advise or NAK this PMD. >> >> I do not remember having already seen such model in DPDK. >> So we need to think about the implications a bit more. >> (Comments/discussions are welcome) >> Thanks for your patience. > > Thanks. I will be happy to discuss this further, so that we can get to a conclusion. > If the outcome is that the majority of the community does not like the idea that > upstream supported PMDs has external linking dependencies to closed source > libraries, then it is ok with us(a pity though). But then it might be a good idea to > make that decision clear to everybody else by putting in a clause into the > contribution section of the DPDK guide, or somewhere else in the guide. > > In our opinion, the inclusion of the ntnic PMD into upstream DPDK, does not > seem to be any different than that of the PCAP PMD, since that is also > dependent on external header files and externally built libraries. > Of course we see the difference in open source vs close source library. But we > cannot see that is has any influence in the usage or functionality of the DPDK. > > Thanks for this discussion! > The patch is still waiting in the patchwork. This requires a high level discussion. Any suggestion on how to proceed? ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-11-21 13:47 ` Ferruh Yigit @ 2016-11-21 13:55 ` Finn Christensen 0 siblings, 0 replies; 22+ messages in thread From: Finn Christensen @ 2016-11-21 13:55 UTC (permalink / raw) To: Ferruh Yigit, Thomas Monjalon; +Cc: Neil Horman, dev, stephen I have changed the state to rejected. Sorry for not doing this earlier. Finn Christensen -----Original Message----- From: Ferruh Yigit [mailto:ferruh.yigit@intel.com] Sent: 21. november 2016 14:48 To: Finn Christensen <fc@napatech.com>; Thomas Monjalon <thomas.monjalon@6wind.com> Cc: Neil Horman <nhorman@tuxdriver.com>; dev@dpdk.org; stephen@networkplumber.org Subject: Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver On 9/12/2016 8:34 AM, fc at napatech.com (Finn Christensen) wrote: >> -----Original Message----- >> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com] >> Sent: 10. september 2016 10:20 >> To: Finn Christensen <fc at napatech.com> >> Cc: Neil Horman <nhorman at tuxdriver.com>; dev at dpdk.org; stephen >> at networkplumber.org >> Subject: Re: [PATCH v3] ntnic: add PMD driver >> >> 2016-09-10 07:58, Finn Christensen: >>> From: Neil Horman [mailto:nhorman at tuxdriver.com] >>>> On Fri, Sep 09, 2016 at 12:48:38PM +0000, Finn Christensen wrote: >>>>> This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. >>>>> >>>>> This patch adds support for Napatech NICs to DPDK. This is the >>>>> initial implementation. >>>>> >>>>> Signed-off-by: Finn Christensen <fc at napatech.com> >>>>> --- >>>>> v3: >>>>> * Removed the need for binary libraries on build >>>>> v2: >>>>> * Added information how to build the PMD without NIC >>>>> Board Support Package >>>>> * Fixed some formatting issues >>>> >>>> So, this is a step in the right direction, but I think its solving >>>> the wrong problem. If you have a dependency on an external >>>> library, thats ok, and accessing it via dlopen makes it possible to >>>> build the library without having that library present, but it not >>>> really in keeping with the spirit of what I meant. This driver is >>>> still effectively dependent on a binary blob that we have no >>>> visibility into. The better solution is releasing the source for >>>> the ntnic and ntos libraries. The license file in the referenced >>>> git tree indicates its BSD licensed, so I don't think there should >>>> be a problem in >> doing that. >>>> >>>> Neil >>>> >>> No, unfortunately the ntapi is not BSD licensed, only the header >>> files that you can freely download are. >>> We are building this NT NIC by using parts or our technology from >>> our capture adapters and that is using closed source software. >>> >>> We are new to opensource and we want to go that way, but we haven't >>> yet a complete stand-alone driver ready that we can put into the >>> DPDK PMD to have a complete self contained and open sourced DPDK >>> PMD, that only needs the actual HW NIC plugged in to run. >>> Therefore this version is implemented as a virtual device, exactly >>> like the PCAP PMD driver is, and it runs on top of a driver that >>> follows the >> NIC itself. >>> >>> In regards to the DPDK functionality we do not see that anything is missing. >>> I cannot either see where we should add source code, because it is >>> not part of the DPDK package and it should not be either. >>> >>> One of the things I really liked about the DPDK open source project >>> is that it uses BSD licensing not GPL. Therefore, I must admit, we >>> completely failed to see that the "spirit" of the DPDK community is >>> not really BSD. Our view of this community was that the main driving >>> force of it was to be able to make DPDK run on everything anywhere >>> effectively, in a global contributing community, without any >>> legally >> constrains prohibiting us to do so. >> >> It is difficult to define what is the spirit of a community, >> especially only after few mail exchanges. >> I agree that running on everything anywhere is a nice goal. >> Here Neil, as a RedHat developer, is probably concerned about >> enabling your driver in a distribution. It seems your model is not >> compatible with the "anywhere goal" and will be disabled in that case, until it is fully open. > > The ntnic PMD is not enabled by default and I think it should not be > either. To enable it in a distribution for general purposes seems > wrong. In that respect we see no difference between the PCAP PMD and this ntnic PMD. > >>> However, this is our standing, and I don't know what else to do. >>> Please advise or NAK this PMD. >> >> I do not remember having already seen such model in DPDK. >> So we need to think about the implications a bit more. >> (Comments/discussions are welcome) >> Thanks for your patience. > > Thanks. I will be happy to discuss this further, so that we can get to a conclusion. > If the outcome is that the majority of the community does not like the > idea that upstream supported PMDs has external linking dependencies to > closed source libraries, then it is ok with us(a pity though). But > then it might be a good idea to make that decision clear to everybody > else by putting in a clause into the contribution section of the DPDK guide, or somewhere else in the guide. > > In our opinion, the inclusion of the ntnic PMD into upstream DPDK, > does not seem to be any different than that of the PCAP PMD, since > that is also dependent on external header files and externally built libraries. > Of course we see the difference in open source vs close source > library. But we cannot see that is has any influence in the usage or functionality of the DPDK. > > Thanks for this discussion! > The patch is still waiting in the patchwork. This requires a high level discussion. Any suggestion on how to proceed? Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. ^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [dpdk-dev] [PATCH v3] ntnic: add PMD driver 2016-09-10 7:58 ` Finn Christensen 2016-09-10 8:20 ` Thomas Monjalon @ 2016-09-12 12:32 ` Neil Horman 1 sibling, 0 replies; 22+ messages in thread From: Neil Horman @ 2016-09-12 12:32 UTC (permalink / raw) To: Finn Christensen; +Cc: dev, thomas.monjalon, stephen On Sat, Sep 10, 2016 at 07:58:38AM +0000, Finn Christensen wrote: > > -----Original Message----- > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > Sent: 9. september 2016 15:51 > > To: Finn Christensen <fc@napatech.com> > > Cc: dev@dpdk.org; thomas.monjalon@6wind.com; > > stephen@networkplumber.org > > Subject: Re: [PATCH v3] ntnic: add PMD driver > > > > On Fri, Sep 09, 2016 at 12:48:38PM +0000, Finn Christensen wrote: > > > This is the Napatech NTNIC Poll Mode Driver (PMD) for DPDK. > > > > > > This patch adds support for Napatech NICs to DPDK. This is the > > > initial implementation. > > > > > > Signed-off-by: Finn Christensen <fc@napatech.com> > > > --- > > > v3: > > > * Removed the need for binary libraries on build > > > v2: > > > * Added information how to build the PMD without NIC > > > Board Support Package > > > * Fixed some formatting issues > > So, this is a step in the right direction, but I think its solving the wrong > > problem. If you have a dependency on an external library, thats ok, and > > accessing it via dlopen makes it possible to build the library without having > > that library present, but it not really in keeping with the spirit of what I > > meant. This driver is still effectively dependent on a binary blob that we > > have > > no visibility into. The better solution is releasing the source for the ntnic > > and ntos libraries. The license file in the referenced git tree indicates its > > BSD licensed, so I don't think there should be a problem in doing that. > > > > Neil > > > No, unfortunately the ntapi is not BSD licensed, only the header files that > you can freely download are. > We are building this NT NIC by using parts or our technology from our > capture adapters and that is using closed source software. > That seems to be different than what your github tree indicates, since the binaries were included in a tree with the BSD license in it. Though I see you've removed those now: https://github.com/napa-tech/ntapi/commits/master > We are new to opensource and we want to go that way, but we haven't > yet a complete stand-alone driver ready that we can put into the DPDK > PMD to have a complete self contained and open sourced DPDK PMD, that > only needs the actual HW NIC plugged in to run. > Therefore this version is implemented as a virtual device, exactly like the > PCAP PMD driver is, and it runs on top of a driver that follows the NIC itself. A stand alone driver that doesn't require additional proprietary software is really the goal, yes, but its also the minimum requirement. Providing a driver that can only be used with additional proprietary sofware doesn't help anyone, as it just means more maintenence for other developers, while not getting the ability to work with new hardware in any meaningful way. Thats not to say that the driver you are providing isn't useful, some end user might find it very useful. But its not fair to the developers working on the DPDK, to ask them to provide maintenence on your driver, when they can't use it without aquiescing to whatever proprietary license you may have. Given the situation you're in, I think the right solution is for you to maintain this driver out of tree, until such time as it can be used without any proprietary software, at which point it can be integrated. > > In regards to the DPDK functionality we do not see that anything is missing. > I cannot either see where we should add source code, because it is not part > of the DPDK package and it should not be either. > Whats missing is the ability to use your driver without having to agree to whatever demands your closed source license impose upon other developers. When you contribute to an open source project there an implied agreement between the development community and the contributor 1) When other developers make changes to the core project, your driver will get updated to comply with any needed corresponding changes 2) When other developers find a bug that affects your NIC (via testing or visual inspection), they'll fix it 3) Other community developers get the ability to modify your driver to create new features or other enhancements By accepting this, you get the advantage of 1 and 2, but prevent any ability for other developers to take advantage of (3), because you're code is behind some additional license that renders the driver useless unless you agree to it. > One of the things I really liked about the DPDK open source project is that it > uses BSD licensing not GPL. Therefore, I must admit, we completely failed > to see that the "spirit" of the DPDK community is not really BSD. Our view > of this community was that the main driving force of it was to be able to > make DPDK run on everything anywhere effectively, in a global contributing > community, without any legally constrains prohibiting us to do so. > This is a pretty telling comment. The BSD license basically says, keep the license in place, and do what you want with the code, but you seem to have taken that to mean you can give us whatever bits you want, and hold back others. While you are certainly within your rights to do that, The spirit of the DPDK, and all open source projects is to provide an environment that allows developers to expand the project in whatever way they see fit. By providing a driver that is useless without agreement to your additional license terms, you hamper developers in that goal, because now they have to be very careful about how they use, and more importantly, contribute back, new code. > However, this is our standing, and I don't know what else to do. > Please advise or NAK this PMD. > I would love to see this driver come into the DPDK, but its got to be usable using only open source code. Until then, I'm sorry NAK. Neil > Finn > > > > --- > > > MAINTAINERS | 5 + > > > config/common_base | 6 + > > > doc/guides/nics/features/ntnic.ini | 14 + > > > doc/guides/nics/index.rst | 1 + > > > doc/guides/nics/ntnic.rst | 145 ++++ > > > drivers/net/Makefile | 1 + > > > drivers/net/ntnic/Makefile | 66 ++ > > > drivers/net/ntnic/rte_eth_ntnic.c | 1150 > > +++++++++++++++++++++++++++ > > > drivers/net/ntnic/rte_pmd_ntnic_version.map | 4 + > > > mk/rte.app.mk | 3 + > > > 10 files changed, 1395 insertions(+) > > > create mode 100644 doc/guides/nics/features/ntnic.ini > > > create mode 100644 doc/guides/nics/ntnic.rst > > > create mode 100644 drivers/net/ntnic/Makefile > > > create mode 100644 drivers/net/ntnic/rte_eth_ntnic.c > > > create mode 100644 drivers/net/ntnic/rte_pmd_ntnic_version.map > > > > > > diff --git a/MAINTAINERS b/MAINTAINERS > > > index bc9aa02..d3e5f56 100644 > > > --- a/MAINTAINERS > > > +++ b/MAINTAINERS > > > @@ -356,6 +356,11 @@ M: Sony Chacko <sony.chacko@qlogic.com> > > > F: drivers/net/qede/ > > > F: doc/guides/nics/qede.rst > > > > > > +Napatech ntnic > > > +M: Finn Christensen <fc@napatech.com> > > > +F: drivers/net/ntnic/ > > > +F: doc/guides/nics/ntnic.rst > > > + > > > RedHat virtio > > > M: Huawei Xie <huawei.xie@intel.com> > > > M: Yuanhan Liu <yuanhan.liu@linux.intel.com> > > > diff --git a/config/common_base b/config/common_base > > > index 7830535..4a3b2b8 100644 > > > --- a/config/common_base > > > +++ b/config/common_base > > > @@ -309,6 +309,12 @@ CONFIG_RTE_PMD_RING_MAX_TX_RINGS=16 > > > CONFIG_RTE_LIBRTE_PMD_PCAP=n > > > > > > # > > > +# Compile software PMD backed by NTNIC files > > > +# > > > +CONFIG_RTE_LIBRTE_PMD_NTNIC=n > > > + > > > + > > > +# > > > # Compile link bonding PMD library > > > # > > > CONFIG_RTE_LIBRTE_PMD_BOND=y > > > diff --git a/doc/guides/nics/features/ntnic.ini > > b/doc/guides/nics/features/ntnic.ini > > > new file mode 100644 > > > index 0000000..4237e6e > > > --- /dev/null > > > +++ b/doc/guides/nics/features/ntnic.ini > > > @@ -0,0 +1,14 @@ > > > +; > > > +; Supported features of the 'ntnic' network poll mode driver. > > > +; > > > +; Refer to default.ini for the full list of available PMD features. > > > +; > > > +[Features] > > > +Link status = Y > > > +Queue start/stop = Y > > > +Promiscuous mode = Y > > > +Jumbo frame = Y > > > +Basic stats = Y > > > +Multiprocess aware = Y > > > +x86-64 = Y > > > +Usage doc = Y > > > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst > > > index 92d56a5..5c4205c 100644 > > > --- a/doc/guides/nics/index.rst > > > +++ b/doc/guides/nics/index.rst > > > @@ -52,6 +52,7 @@ Network Interface Controller Drivers > > > qede > > > szedata2 > > > thunderx > > > + ntnic > > > virtio > > > vhost > > > vmxnet3 > > > diff --git a/doc/guides/nics/ntnic.rst b/doc/guides/nics/ntnic.rst > > > new file mode 100644 > > > index 0000000..f9398db > > > --- /dev/null > > > +++ b/doc/guides/nics/ntnic.rst > > > @@ -0,0 +1,145 @@ > > > +.. BSD LICENSE > > > + Copyright (c) 2016 Napatech A/S > > > + All rights reserved. > > > + > > > + Redistribution and use in source and binary forms, with or without > > > + modification, are permitted provided that the following conditions > > > + are met: > > > + > > > + * Redistributions of source code must retain the above copyright > > > + notice, this list of conditions and the following disclaimer. > > > + * Redistributions in binary form must reproduce the above copyright > > > + notice, this list of conditions and the following disclaimer in > > > + the documentation and/or other materials provided with the > > > + distribution. > > > + * Neither the name of Napatech nor the names of its > > > + contributors may be used to endorse or promote products derived > > > + from this software without specific prior written permission. > > > + > > > + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > > CONTRIBUTORS > > > + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > > NOT > > > + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > > FITNESS FOR > > > + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > > COPYRIGHT > > > + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > > INCIDENTAL, > > > + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, > > BUT NOT > > > + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS > > OF USE, > > > + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED > > AND ON ANY > > > + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > > TORT > > > + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > > OF THE USE > > > + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > > DAMAGE. > > > + > > > +NTNIC Poll Mode Driver > > > +====================== > > > + > > > +The NTNIC poll mode driver library (**librte_pmd_ntnic**) implements > > support > > > +for **Napatech NIC** 40/50 Gbps adapters. > > > +This PMD is implemented as a pure software virtual device and must be > > created > > > +by using the EAL --vdev=parameter (parameters are explained i detail > > later). > > > +It runs on top of the Napatech NFV NIC Board Support Package that must > > be > > > +installed and started. > > > +If no Napatech NIC is available, you can download the Napatech NTAPI > > includes > > > +to build with. > > > + > > > +Supported Features > > > +------------------ > > > + > > > +- RSS (Receive Side Scaling) > > > +- TSS (Transmit Side Scaling) > > > +- Promiscuous mode > > > +- Basic statistics > > > + > > > + > > > +Prerequisites > > > +------------- > > > + > > > +Requires Napatech NIC and Napatech NIC Board Support Package > > installed and > > > +running in version **0.3.0** or higher. > > > +This includes external libraries and kernel driver for resources > > > +allocations and initialization. > > > +If build only is required, download the Napatech NTAPI to build against. > > > + > > > +Pre-Installation Configuration > > > +------------------------------ > > > + > > > +Environment variables > > > +~~~~~~~~~~~~~~~~~~~~~ > > > + > > > +In order to compile the Napatech NIC PMD, user must: > > > + > > > +* Export the environment variable NAPATECH3_PATH with the path > > where > > > + the Napatech Board Support Package was installed (default location is > > > + /opt/napatech3). > > > + Or if no Napatech NIC available: > > > + Download the Napatech NTAPI include files from Github: > > > + `Napatech NTAPI <https://github.com/napa-tech/ntapi>`_. > > > + > > > + > > > +Config File Options > > > +~~~~~~~~~~~~~~~~~~~ > > > + > > > +- ``CONFIG_RTE_LIBRTE_PMD_NTNIC`` (default **n**) > > > + > > > +Using the NTNIC PMD from a DPDK application using EAL vdev parameter > > > +-------------------------------------------------------------------- > > > + > > > +Napatech NIC PMD VDEV parameters > > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > > + > > > +The Napatech NIC PMD vdev has the following command line parameters > > to enable > > > +its features. > > > + > > > +* port > > > + Configures which NT port the vdev should use. > > > + > > > + --vdev eth_ntnic0,port=4,rxqs=1,txqs=1 > > > + > > > + This will create a DPDK port0 using NT port 4 > > > + > > > +* rxqs > > > + Control how many receive queues a vdev should have. The traffic from a > > vdev > > > + will be load balanced to this amount of queues and each queue can be > > handled > > > + by its own lcore. > > > + Note: The ntservice.ini HostBuffersRx must be configured to the sum of > > all > > > + receive queues across all vdevs. > > > + > > > + --vdev eth_ntnic0,port=0,rxqs=8,txqs=1 > > > + > > > + Traffic from NT port 0 will be split across > > > + 8 queues meaning that 8 lcores can be > > > + opened on DPDK port 0 and each will be > > > + their individual traffic. > > > + > > > +* hash > > > + Control which load balancing scheme should be used when running with > > multiple > > > + receive queues. > > > + The possible values are: > > > + 1 = HashRoundRobin > > > + 2 = Hash2TupleSorted > > > + 3 = Hash5TupleSorted > > > + > > > + --vdev eth_ntnic0,port=0,rxqs=4,hash=3,txqs=1 > > > + > > > + Create 4 queues from NT port 0 and > > > + load balance the traffic using the > > > + 5-tuple sorted algorithm. Each of the > > > + 4 queues can be opened via DPDK port 0 > > > + > > > +* txqs > > > + Control how many transmit queues a vdev should have. Each queue can > > be used > > > + by a lcore and each queue is independent for other queues. > > > + Note: The ntservice.ini HostBuffersTx must be configured to the sum of > > all > > > + transmit queues across all vdevs. > > > + > > > + --vdev eth_ntnic0,port=0,txqs=32,rxqs=1 > > > + > > > + Enable 32 transmit queues on NT port 0 > > > + > > > + > > > +Changes to Napatech driver configuration file ntservice.ini > > > +----------------------------------------------------------- > > > +Depending on the number of queues that is created, more hostbuffers > > may be > > > +needed pre-allocated by the Napatech NIC Driver. This is controlled in the > > > +ntservice.ini file (default locations is /opt/napatech3/config). > > > +The Napatech NIC Driver must be re-started when this configuration file is > > > +changed. > > > + > > > diff --git a/drivers/net/Makefile b/drivers/net/Makefile > > > index bc93230..aff29e9 100644 > > > --- a/drivers/net/Makefile > > > +++ b/drivers/net/Makefile > > > @@ -55,6 +55,7 @@ DIRS- > > $(CONFIG_RTE_LIBRTE_THUNDERX_NICVF_PMD) += thunderx > > > DIRS-$(CONFIG_RTE_LIBRTE_VIRTIO_PMD) += virtio > > > DIRS-$(CONFIG_RTE_LIBRTE_VMXNET3_PMD) += vmxnet3 > > > DIRS-$(CONFIG_RTE_LIBRTE_PMD_XENVIRT) += xenvirt > > > +DIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += ntnic > > > > > > ifeq ($(CONFIG_RTE_LIBRTE_VHOST),y) > > > DIRS-$(CONFIG_RTE_LIBRTE_PMD_VHOST) += vhost > > > diff --git a/drivers/net/ntnic/Makefile b/drivers/net/ntnic/Makefile > > > new file mode 100644 > > > index 0000000..d816c56 > > > --- /dev/null > > > +++ b/drivers/net/ntnic/Makefile > > > @@ -0,0 +1,66 @@ > > > +# BSD LICENSE > > > +# > > > +# Copyright(c) 2016 Napatech A/S. All rights reserved. > > > +# All rights reserved. > > > +# > > > +# Redistribution and use in source and binary forms, with or without > > > +# modification, are permitted provided that the following conditions > > > +# are met: > > > +# > > > +# * Redistributions of source code must retain the above copyright > > > +# notice, this list of conditions and the following disclaimer. > > > +# * Redistributions in binary form must reproduce the above copyright > > > +# notice, this list of conditions and the following disclaimer in > > > +# the documentation and/or other materials provided with the > > > +# distribution. > > > +# * Neither the name of Napatech nor the names of its > > > +# contributors may be used to endorse or promote products derived > > > +# from this software without specific prior written permission. > > > +# > > > +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > > CONTRIBUTORS > > > +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > > NOT > > > +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > > FITNESS FOR > > > +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > > COPYRIGHT > > > +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > > INCIDENTAL, > > > +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, > > BUT NOT > > > +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; > > LOSS OF USE, > > > +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED > > AND ON ANY > > > +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > > TORT > > > +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > > OF THE USE > > > +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > > DAMAGE. > > > + > > > +include $(RTE_SDK)/mk/rte.vars.mk > > > + > > > +# > > > +# library name > > > +# > > > +LIB = librte_pmd_ntnic.a > > > + > > > +CFLAGS += -O3 -g > > > +CFLAGS += $(WERROR_FLAGS) > > > +CFLAGS += -I$(NAPATECH3_PATH)/include > > > +CFLAGS += -DNAPATECH3_LIB_PATH=\"$(NAPATECH3_PATH)/lib\" > > > +LDLIBS += -ldl > > > + > > > +EXPORT_MAP := rte_pmd_ntnic_version.map > > > + > > > +LIBABIVER := 1 > > > + > > > +# > > > +# all source are stored in SRCS-y > > > +# > > > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += rte_eth_ntnic.c > > > + > > > +# > > > +# Export include files > > > +# > > > +SYMLINK-y-include += > > > + > > > +# this lib depends upon: > > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_eal > > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mbuf > > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_mempool > > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_ether > > > +DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += lib/librte_kvargs > > > + > > > +include $(RTE_SDK)/mk/rte.lib.mk > > > diff --git a/drivers/net/ntnic/rte_eth_ntnic.c > > b/drivers/net/ntnic/rte_eth_ntnic.c > > > new file mode 100644 > > > index 0000000..f83e94d > > > --- /dev/null > > > +++ b/drivers/net/ntnic/rte_eth_ntnic.c > > > @@ -0,0 +1,1150 @@ > > > +/*- > > > + * BSD LICENSE > > > + * > > > + * Copyright(c) 2016 Napatech A/S. All rights reserved. > > > + * All rights reserved. > > > + * > > > + * Redistribution and use in source and binary forms, with or without > > > + * modification, are permitted provided that the following conditions > > > + * are met: > > > + * > > > + * * Redistributions of source code must retain the above copyright > > > + * notice, this list of conditions and the following disclaimer. > > > + * * Redistributions in binary form must reproduce the above copyright > > > + * notice, this list of conditions and the following disclaimer in > > > + * the documentation and/or other materials provided with the > > > + * distribution. > > > + * * Neither the name of Napatech nor the names of its > > > + * contributors may be used to endorse or promote products derived > > > + * from this software without specific prior written permission. > > > + * > > > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > > CONTRIBUTORS > > > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT > > NOT > > > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > > FITNESS FOR > > > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > > COPYRIGHT > > > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > > INCIDENTAL, > > > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, > > BUT NOT > > > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; > > LOSS OF USE, > > > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED > > AND ON ANY > > > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > > TORT > > > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT > > OF THE USE > > > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > > DAMAGE. > > > + */ > > > + > > > +#include <time.h> > > > +#include <dlfcn.h> > > > +#include <rte_mbuf.h> > > > +#include <rte_ethdev.h> > > > +#include <rte_malloc.h> > > > +#include <rte_memcpy.h> > > > +#include <rte_string_fns.h> > > > +#include <rte_cycles.h> > > > +#include <rte_kvargs.h> > > > +#include <rte_dev.h> > > > +#include <net/if.h> > > > +#include <nt.h> > > > + > > > +#define ETH_NTNIC_PORT_ARG "port" > > > +#define ETH_NTNIC_PORTEND_ARG "portend" > > > +#define ETH_NTNIC_RXQUEUES_ARG "rxqs" > > > +#define ETH_NTNIC_TXQUEUES_ARG "txqs" > > > +#define ETH_NTNIC_STREAMID_ARG "streamid" > > > +#define ETH_NTNIC_HASH_ARG "hash" > > > + > > > +#define HW_MAX_PKT_LEN 10000 > > > +#define HW_MTU (HW_MAX_PKT_LEN - ETHER_HDR_LEN - > > ETHER_CRC_LEN) > > > + > > > +#define MAX_RX_QUEUES 64 > > > +#define MAX_TX_QUEUES 64 > > > +#define MAX_NTNIC_PORTS 32 > > > + > > > +static void *_libnt; > > > + > > > +/* NTAPI library functions */ > > > +int (*_NT_Init)(uint32_t); > > > +int (*_NT_NetRxOpen)(NtNetStreamRx_t *, > > > + const char *, enum NtNetInterface_e, uint32_t, int); > > > +int (*_NT_NetRxGet)(NtNetStreamRx_t, NtNetBuf_t *, int); > > > +int (*_NT_NetRxRelease)(NtNetStreamRx_t, NtNetBuf_t); > > > +int (*_NT_NetRxClose)(NtNetStreamRx_t); > > > +char *(*_NT_ExplainError)(int, char *, uint32_t); > > > +int (*_NT_NetTxOpen)(NtNetStreamTx_t *, > > > + const char *, uint64_t, uint32_t, uint32_t); > > > +int (*_NT_NetTxRingbufferInit)(NtNetStreamTx_t, uint32_t); > > > +int (*_NT_NetTxRingbufferTransmitPacket)(NtNetStreamTx_t, > > > + uint8_t *, uint16_t); > > > +int (*_NT_NetTxRingbufferFlush)(NtNetStreamTx_t); > > > +void (*_NT_NetTxRingbufferDone)(NtNetStreamTx_t); > > > +int (*_NT_NetTxClose)(NtNetStreamTx_t); > > > +int (*_NT_InfoOpen)(NtInfoStream_t *, const char *); > > > +int (*_NT_InfoRead)(NtInfoStream_t, NtInfo_t *); > > > +int (*_NT_InfoClose)(NtInfoStream_t); > > > +int (*_NT_ConfigOpen)(NtConfigStream_t *, const char *); > > > +int (*_NT_ConfigClose)(NtConfigStream_t); > > > +int (*_NT_NTPL)(NtConfigStream_t, const char *, > > > + NtNtplInfo_t *, uint32_t); > > > + > > > +static char ebuf[1024]; > > > + > > > +struct array_s { > > > + uint32_t value[MAX_RX_QUEUES]; > > > + int count; > > > +}; > > > + > > > +static volatile uint16_t port_locks[MAX_NTNIC_PORTS]; > > > + > > > +struct ntnic_rx_queue { > > > + NtNetStreamRx_t net_rx[MAX_RX_QUEUES]; /* NT Rx streams */ > > > + int rx_idx; /* Current Rx stream index */ > > > + struct rte_mempool *mb_pool; /* mbuf memory pool */ > > > + uint16_t buf_size; /* Size of data area in mbuf */ > > > + NtNetBuf_t seg; /* The current NT data segment */ > > > + struct NtNetBuf_s pkt; /* The current packet */ > > > + volatile unsigned long rx_pkts; /* Rx packet statistics */ > > > + volatile unsigned long err_pkts; /* Rx error packet statistics */ > > > + struct array_s astreamids; /* NT streams to read from */ > > > + int enabled; /* Enabling/disabling of this queue */ > > > +}; > > > + > > > +struct ntnic_tx_queue { > > > + NtNetStreamTx_t net_tx; /* NT Tx stream */ > > > + volatile unsigned long tx_pkts; /* Tx packet statistics */ > > > + volatile unsigned long err_pkts; /* Tx error packet stat */ > > > + volatile uint16_t *plock; /* Per port transmit atomic lock */ > > > + uint32_t port; /* Tx port for this queue */ > > > + int enabled; /* Enabling/disabling of this queue */ > > > + int fcs_add; /* If needs added room for FCS */ > > > +}; > > > + > > > +struct pmd_internals { > > > + struct ntnic_rx_queue rxq[MAX_RX_QUEUES]; /* Array of Rx queues > > */ > > > + struct ntnic_tx_queue txq[MAX_TX_QUEUES]; /* Array of Tx queues > > */ > > > + int if_index; /* Itf index always 0 - no bonding */ > > > + unsigned int nb_rx_queues; /* Number of Rx queues configured > > */ > > > + unsigned int nb_tx_queues; /* Number of Tx queues configured > > */ > > > + int MAC_50G; /* True if 50G ports */ > > > +}; > > > + > > > +static const char *valid_arguments[] = { > > > + ETH_NTNIC_PORT_ARG, > > > + ETH_NTNIC_PORTEND_ARG, > > > + ETH_NTNIC_RXQUEUES_ARG, > > > + ETH_NTNIC_TXQUEUES_ARG, > > > + ETH_NTNIC_STREAMID_ARG, > > > + ETH_NTNIC_HASH_ARG, > > > + NULL > > > +}; > > > + > > > +static struct ether_addr eth_addr[MAX_NTNIC_PORTS]; > > > +static const char *drivername = "NTNIC PMD"; > > > + > > > +static int > > > +eth_ntnic_rx_jumbo(struct rte_mempool *mb_pool, > > > + struct rte_mbuf *mbuf, > > > + const u_char *data, > > > + uint16_t data_len) > > > +{ > > > + struct rte_mbuf *m = mbuf; > > > + > > > + /* Copy the first segment. */ > > > + uint16_t len = rte_pktmbuf_tailroom(mbuf); > > > + > > > + rte_memcpy(rte_pktmbuf_append(mbuf, len), data, len); > > > + data_len -= len; > > > + data += len; > > > + > > > + while (data_len > 0) { > > > + /* Allocate next mbuf and point to that. */ > > > + m->next = rte_pktmbuf_alloc(mb_pool); > > > + > > > + if (unlikely(!m->next)) > > > + return -1; > > > + > > > + m = m->next; > > > + > > > + /* Headroom is not needed in chained mbufs. */ > > > + rte_pktmbuf_prepend(m, rte_pktmbuf_headroom(m)); > > > + m->pkt_len = 0; > > > + m->data_len = 0; > > > + > > > + /* Copy next segment. */ > > > + len = RTE_MIN(rte_pktmbuf_tailroom(m), data_len); > > > + rte_memcpy(rte_pktmbuf_append(m, len), data, len); > > > + > > > + mbuf->nb_segs++; > > > + data_len -= len; > > > + data += len; > > > + } > > > + > > > + return mbuf->nb_segs; > > > +} > > > + > > > + > > > + > > > +static uint16_t > > > +eth_ntnic_rx(void *queue, > > > + struct rte_mbuf **bufs, > > > + uint16_t nb_pkts) > > > +{ > > > + unsigned int i; > > > + struct rte_mbuf *mbuf; > > > + struct ntnic_rx_queue *rx_q = queue; > > > + uint16_t num_rx = 0; > > > + uint16_t data_len; > > > + > > > + if (unlikely(rx_q->net_rx[rx_q->rx_idx] == NULL || nb_pkts == 0)) > > > + return 0; > > > + > > > + /* Do we have any data segment */ > > > + if (rx_q->seg == NULL) { > > > + /* Next stream - if multiple streams are combined */ > > > + rx_q->rx_idx = > > > + (rx_q->rx_idx < > > > + (rx_q->astreamids.count - 1)) ? rx_q->rx_idx + 1 > > > + : 0; > > > + > > > + if ((*_NT_NetRxGet)(rx_q->net_rx[rx_q->rx_idx], > > > + &rx_q->seg, 0) != NT_SUCCESS) { > > > + if (rx_q->seg != NULL) { > > > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > > > + rx_q->seg); > > > + rx_q->seg = NULL; > > > + } > > > + return 0; > > > + } > > > + if (NT_NET_GET_SEGMENT_LENGTH(rx_q->seg)) { > > > + /* Build a packet structure */ > > > + _nt_net_build_pkt_netbuf(rx_q->seg, &rx_q->pkt); > > > + } else { > > > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > > > + rx_q->seg); > > > + rx_q->seg = NULL; > > > + return 0; > > > + } > > > + } > > > + > > > + if (rte_mempool_get_bulk(rx_q->mb_pool, (void **)bufs, nb_pkts) > > != 0) > > > + return 0; > > > + > > > + for (i = 0; i < nb_pkts; i++) { > > > + mbuf = bufs[i]; > > > + rte_mbuf_refcnt_set(mbuf, 1); > > > + rte_pktmbuf_reset(mbuf); > > > + > > > + /* HW slicing not supported */ > > > + data_len = (uint16_t)NT_NET_GET_PKT_WIRE_LENGTH((&rx_q- > > >pkt)) > > > + - 4; > > > + if (data_len <= rx_q->buf_size) { > > > + /* Packet will fit in the mbuf, go ahead and copy */ > > > + mbuf->pkt_len = mbuf->data_len; > > > + data_len = mbuf->data_len; > > > + rte_memcpy((u_char *)mbuf->buf_addr + > > > + RTE_PKTMBUF_HEADROOM, > > > + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), > > > + mbuf->data_len); > > > + } else { > > > + /* Try read jumbo frame into multi mbufs. */ > > > + if (unlikely(eth_ntnic_rx_jumbo(rx_q->mb_pool, > > > + mbuf, > > > + NT_NET_GET_PKT_L2_PTR((&rx_q->pkt)), > > > + data_len) == -1)) > > > + break; > > > + } > > > + > > > + mbuf->port = NT_NET_GET_PKT_RXPORT((&rx_q->pkt)); > > > + num_rx++; > > > + > > > + /* Get the next packet if any */ > > > + if (_nt_net_get_next_packet(rx_q->seg, > > > + NT_NET_GET_SEGMENT_LENGTH(rx_q->seg), > > > + &rx_q->pkt) == 0) { > > > + (*_NT_NetRxRelease)(rx_q->net_rx[rx_q->rx_idx], > > > + rx_q->seg); > > > + rx_q->seg = NULL; > > > + break; > > > + } > > > + > > > + rx_q->rx_pkts++; > > > + } > > > + > > > + if (num_rx < nb_pkts) { > > > + rte_mempool_put_bulk(rx_q->mb_pool, > > > + (void * const *)(bufs + num_rx), > > > + nb_pkts - num_rx); > > > + } > > > + return num_rx; > > > +} > > > + > > > + > > > + > > > + > > > + > > > +/* Callback to handle sending packets through a NT NIC. */ > > > +static uint16_t > > > +eth_ntnic_tx_ringbuffer(void *queue, > > > + struct rte_mbuf **bufs, > > > + uint16_t nb_pkts) > > > +{ > > > + unsigned int i; > > > + struct ntnic_tx_queue *tx_q = queue; > > > + int retval; > > > + uint16_t old; > > > + uint16_t new_flag = 0; > > > + > > > + > > > + if (unlikely(tx_q == NULL || tx_q->net_tx == NULL || nb_pkts == 0)) > > > + return 0; > > > + > > > + > > > + do { > > > + old = 0; > > > + new_flag = 1; > > > + retval = rte_atomic16_cmpset(tx_q->plock, old, new_flag); > > > + } while (unlikely(retval == 0)); > > > + > > > + for (i = 0; i < nb_pkts; i++) { > > > + /* Not allowed to TX less than 64 byte packets */ > > > + if (bufs[i]->pkt_len < 60) > > > + bufs[i]->pkt_len = 60; > > > + > > > + /* transmit a single packet */ > > > + (*_NT_NetTxRingbufferTransmitPacket)(tx_q->net_tx, > > > + rte_pktmbuf_mtod(bufs[i], uint8_t *), > > > + bufs[i]->pkt_len + 4 - tx_q->fcs_add); > > > + rte_pktmbuf_free(bufs[i]); > > > + } > > > + tx_q->tx_pkts += nb_pkts; > > > + > > > + *tx_q->plock = 0; > > > + return nb_pkts; > > > +} > > > + > > > + > > > + > > > +static int > > > +eth_dev_start(struct rte_eth_dev *dev) > > > +{ > > > + struct pmd_internals *internals = dev->data->dev_private; > > > + struct ntnic_rx_queue *rx_q = internals->rxq; > > > + struct ntnic_tx_queue *tx_q = internals->txq; > > > + uint queue; > > > + int status, idx; > > > + > > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > > + > > > + if ((internals->nb_tx_queues | internals->nb_rx_queues) == 0) > > > + return -1; > > > + > > > + for (queue = 0; queue < internals->nb_rx_queues; queue++) { > > > + if (rx_q[queue].enabled) { > > > + char str[128], val[10]; > > > + str[0] = 0; > > > + /* build list of streamids to log */ > > > + for (idx = 0; idx < > > > + rx_q[queue].astreamids.count; idx++) { > > > + if (idx) > > > + sprintf(val, ",%d", > > > + rx_q[queue].astreamids.value[idx]); > > > + else > > > + sprintf(val, "%i", > > > + rx_q[queue].astreamids.value[idx]); > > > + strcat(str, val); > > > + } > > > + > > > + for (idx = 0; idx < rx_q[queue].astreamids.count; > > > + idx++) { > > > + status = > > > + (*_NT_NetRxOpen)(&rx_q[queue].net_rx[idx], > > > + "DPDK", > > > + NT_NET_INTERFACE_SEGMENT, > > > + rx_q[queue].astreamids.value[idx], -1); > > > + if (status != NT_SUCCESS) { > > > + /* try packet interface instead */ > > > + (*_NT_ExplainError)(status, > > > + ebuf, > > > + sizeof(ebuf)); > > > + RTE_LOG(ERR, PMD, > > > + "NT_NetRxOpen() failed: %s\n", > > > + ebuf); > > > + return -1; > > > + } > > > + } > > > + } > > > + } > > > + > > > + for (queue = 0; queue < internals->nb_tx_queues; queue++) { > > > + if (tx_q[queue].enabled) { > > > + RTE_LOG(INFO, PMD, "NTNIC: NT_NetTxOpen(%d)\n", > > > + tx_q[queue].port); > > > + status = (*_NT_NetTxOpen)(&tx_q[queue].net_tx, "DPDK", > > > + 1 << tx_q[queue].port, > > > + dev->pci_dev->numa_node, 0); > > > + if (status != NT_SUCCESS) { > > > + (*_NT_ExplainError)(status, ebuf, > > > + sizeof(ebuf)); > > > + RTE_LOG(INFO, PMD, > > > + "NT_NetTxOpen(0x%X, %d, 0) failed: %s\n", > > > + 1 << tx_q[queue].port, > > > + dev->pci_dev->numa_node, ebuf); > > > + return -1; > > > + } > > > + > > > + /* Init and associate RTD Tx Ring buf to Tx handler */ > > > + status = (*_NT_NetTxRingbufferInit)( > > > + tx_q[queue].net_tx, > > > + tx_q[queue].port); > > > + if (status != NT_SUCCESS) { > > > + (*_NT_ExplainError)(status, ebuf, > > > + sizeof(ebuf)); > > > + RTE_LOG(ERR, PMD, > > > + "NT_NetRxOpen() failed: %s\n", > > > + ebuf); > > > + return -1; > > > + } > > > + } > > > + tx_q[queue].plock = &port_locks[tx_q[queue].port]; > > > + tx_q[queue].fcs_add = (internals->MAC_50G) ? 4 : 0; > > > + } > > > + > > > + dev->data->dev_link.link_status = 1; > > > + return 0; > > > +} > > > + > > > + > > > +static void > > > +eth_dev_stop(struct rte_eth_dev *dev) > > > +{ > > > + struct pmd_internals *internals = dev->data->dev_private; > > > + struct ntnic_rx_queue *rx_q = internals->rxq; > > > + struct ntnic_tx_queue *tx_q = internals->txq; > > > + uint q; > > > + int idx; > > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > > + > > > + for (q = 0; q < internals->nb_rx_queues; q++) { > > > + if (rx_q[q].seg) { > > > + (*_NT_NetRxRelease)(rx_q[q].net_rx[rx_q->rx_idx], > > > + rx_q[q].seg); > > > + rx_q[q].seg = NULL; > > > + } > > > + for (idx = 0; idx < rx_q[q].astreamids.count; idx++) { > > > + if (rx_q[q].net_rx[idx]) > > > + (void)(*_NT_NetRxClose)(rx_q[q].net_rx[idx]); > > > + } > > > + } > > > + for (q = 0; q < internals->nb_tx_queues; q++) { > > > + if (tx_q[q].net_tx) { > > > + (*_NT_NetTxRingbufferDone)(tx_q[q].net_tx); > > > + (void)(*_NT_NetTxClose)(tx_q[q].net_tx); > > > + } > > > + } > > > + dev->data->dev_link.link_status = 0; > > > +} > > > + > > > +static int > > > +eth_dev_configure(struct rte_eth_dev *dev __rte_unused) > > > +{ > > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > > + return 0; > > > +} > > > + > > > +static void > > > +eth_dev_info(struct rte_eth_dev *dev, > > > + struct rte_eth_dev_info *dev_info) > > > +{ > > > + struct pmd_internals *internals = dev->data->dev_private; > > > + dev_info->if_index = internals->if_index; > > > + dev_info->driver_name = drivername; > > > + dev_info->max_mac_addrs = 1; > > > + dev_info->max_rx_pktlen = HW_MTU; > > > + dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues; > > > + dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues; > > > + dev_info->min_rx_bufsize = 0; > > > + dev_info->pci_dev = NULL; > > > +} > > > + > > > +static void > > > +eth_stats_get(struct rte_eth_dev *dev, > > > + struct rte_eth_stats *igb_stats) > > > +{ > > > + unsigned int i; > > > + unsigned long rx_total = 0; > > > + unsigned long tx_total = 0; > > > + unsigned long tx_err_total = 0; > > > + const struct pmd_internals *internal = dev->data->dev_private; > > > + > > > + memset(igb_stats, 0, sizeof(*igb_stats)); > > > + for (i = 0; > > > + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal- > > >nb_rx_queues; > > > + i++) { > > > + igb_stats->q_ipackets[i] = internal->rxq[i].rx_pkts; > > > + rx_total += igb_stats->q_ipackets[i]; > > > + } > > > + for (i = 0; > > > + i < RTE_ETHDEV_QUEUE_STAT_CNTRS && i < internal- > > >nb_tx_queues; > > > + i++) { > > > + igb_stats->q_opackets[i] = internal->txq[i].tx_pkts; > > > + igb_stats->q_errors[i] = internal->txq[i].err_pkts; > > > + tx_total += igb_stats->q_opackets[i]; > > > + tx_err_total += igb_stats->q_errors[i]; > > > + } > > > + > > > + igb_stats->ipackets = rx_total; > > > + igb_stats->opackets = tx_total; > > > + igb_stats->oerrors = tx_err_total; > > > +} > > > + > > > +static void > > > +eth_stats_reset(struct rte_eth_dev *dev) > > > +{ > > > + unsigned int i; > > > + struct pmd_internals *internal = dev->data->dev_private; > > > + > > > + for (i = 0; i < internal->nb_rx_queues; i++) > > > + internal->rxq[i].rx_pkts = 0; > > > + for (i = 0; i < internal->nb_tx_queues; i++) { > > > + internal->txq[i].tx_pkts = 0; > > > + internal->txq[i].err_pkts = 0; > > > + } > > > +} > > > + > > > +static void > > > +eth_dev_close(struct rte_eth_dev *dev __rte_unused) > > > +{ > > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > > +} > > > + > > > +static void > > > +eth_queue_release(void *q __rte_unused) > > > +{ > > > + RTE_LOG(INFO, PMD, "NTNIC: %s\n", __func__); > > > +} > > > + > > > +static int > > > +eth_link_update(struct rte_eth_dev *dev __rte_unused, > > > + int wait_to_complete __rte_unused) > > > +{ > > > + return 0; > > > +} > > > + > > > +static int > > > +eth_rx_queue_setup(struct rte_eth_dev *dev, > > > + uint16_t rx_queue_id, > > > + uint16_t nb_rx_desc __rte_unused, > > > + unsigned int socket_id __rte_unused, > > > + const struct rte_eth_rxconf *rx_conf __rte_unused, > > > + struct rte_mempool *mb_pool) > > > +{ > > > + struct rte_pktmbuf_pool_private *mbp_priv; > > > + struct pmd_internals *internals = dev->data->dev_private; > > > + struct ntnic_rx_queue *rx_q = &internals->rxq[rx_queue_id]; > > > + > > > + RTE_LOG(INFO, PMD, "NTNIC RX queue setup\n"); > > > + rx_q->mb_pool = mb_pool; > > > + dev->data->rx_queues[rx_queue_id] = rx_q; > > > + > > > + mbp_priv = rte_mempool_get_priv(rx_q->mb_pool); > > > + rx_q->buf_size = (uint16_t)(mbp_priv->mbuf_data_room_size - > > > + RTE_PKTMBUF_HEADROOM); > > > + rx_q->enabled = 1; > > > + return 0; > > > +} > > > + > > > +static int > > > +eth_tx_queue_setup(struct rte_eth_dev *dev __rte_unused, > > > + uint16_t tx_queue_id __rte_unused, > > > + uint16_t nb_tx_desc __rte_unused, > > > + unsigned int socket_id __rte_unused, > > > + const struct rte_eth_txconf *tx_conf __rte_unused) > > > +{ > > > + struct pmd_internals *internals = dev->data->dev_private; > > > + RTE_LOG(INFO, PMD, "NTNIC TX queue setup\n"); > > > + dev->data->tx_queues[tx_queue_id] = &internals- > > >txq[tx_queue_id]; > > > + internals->txq[tx_queue_id].enabled = 1; > > > + return 0; > > > +} > > > + > > > +static int _dev_set_mtu(struct rte_eth_dev *dev __rte_unused, uint16_t > > mtu) > > > +{ > > > + if (mtu < 46 || mtu > HW_MTU) > > > + return -EINVAL; > > > + > > > + return 0; > > > +} > > > + > > > +static struct eth_dev_ops ops = { > > > + .dev_start = eth_dev_start, > > > + .dev_stop = eth_dev_stop, > > > + .dev_close = eth_dev_close, > > > + .mtu_set = _dev_set_mtu, > > > + .dev_configure = eth_dev_configure, > > > + .dev_infos_get = eth_dev_info, > > > + .rx_queue_setup = eth_rx_queue_setup, > > > + .tx_queue_setup = eth_tx_queue_setup, > > > + .rx_queue_release = eth_queue_release, > > > + .tx_queue_release = eth_queue_release, > > > + .link_update = eth_link_update, > > > + .stats_get = eth_stats_get, > > > + .stats_reset = eth_stats_reset, > > > +}; > > > + > > > +static int > > > +rte_pmd_init_internals(const char *name, > > > + const unsigned int nb_rx_queues, > > > + const unsigned int nb_tx_queues, > > > + const unsigned int numa_node, > > > + struct array_s *pastreamids, > > > + const uint32_t port, > > > + struct rte_eth_dev **eth_dev) > > > +{ > > > + struct pmd_internals *internals = NULL; > > > + struct rte_eth_dev_data *data = NULL; > > > + struct rte_pci_device *pci_dev = NULL; > > > + uint i, status; > > > + char err_buf[NT_ERRBUF_SIZE]; > > > + NtInfoStream_t info; > > > + NtInfo_t info_port; > > > + struct rte_eth_link pmd_link; > > > + assert(nb_rx_queues < MAX_RX_QUEUES); > > > + assert(nb_tx_queues < MAX_TX_QUEUES); > > > + assert(port < MAX_NTNIC_PORTS); > > > + > > > + RTE_LOG(INFO, PMD, > > > + "Creating ntnic-backend ethdev on numa socket %u\n", > > > + numa_node); > > > + > > > + /* now do all data allocation - for eth_dev structure, dummy pci driver > > > + * and internal (private) data > > > + */ > > > + data = rte_zmalloc_socket(name, sizeof(*data), 0, numa_node); > > > + if (data == NULL) > > > + goto error; > > > + > > > + pci_dev = rte_zmalloc_socket(name, sizeof(*pci_dev), 0, > > numa_node); > > > + if (pci_dev == NULL) > > > + goto error; > > > + > > > + internals = rte_zmalloc_socket(name, sizeof(*internals), 0, > > numa_node); > > > + if (internals == NULL) > > > + goto error; > > > + > > > + /* reserve an ethdev entry */ > > > + *eth_dev = rte_eth_dev_allocate(name, RTE_ETH_DEV_VIRTUAL); > > > + if (*eth_dev == NULL) > > > + goto error; > > > + > > > + /* Open the information stream */ > > > + status = (*_NT_InfoOpen)(&info, "DPDK Info stream"); > > > + if (status != NT_SUCCESS) { > > > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > > > + RTE_LOG(ERR, PMD, > > > + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", > > > + status, err_buf); > > > + return status; > > > + } > > > + > > > + /* Find local port offset */ > > > + info_port.cmd = NT_INFO_CMD_READ_PORT_V7; > > > + info_port.u.port_v7.portNo = (uint8_t)(port); > > > + status = (*_NT_InfoRead)(info, &info_port); > > > + if (status != 0) { > > > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > > > + RTE_LOG(ERR, PMD, > > > + "ERROR: NT_InfoRead failed. Code 0x%x = %s\n", > > > + status, err_buf); > > > + return status; > > > + } > > > + > > > + /* check for 50G MAC */ > > > + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9506 || > > > + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product == 9509) { > > > + internals->MAC_50G = 1; > > > + } else { > > > + internals->MAC_50G = 0; > > > + /* Check for valid FPGAs (NFV NICs) */ > > > + if (info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != > > > + 9507 && > > > + info_port.u.port_v7.data.adapterInfo.fpgaid.s.product != > > > + 9510) { > > > + RTE_LOG(ERR, PMD, > > > + "ERROR: NTNIC PMD does not support the NT adapter" > > > + "- port %i\n", > > > + port); > > > + return -1; > > > + } > > > + } > > > + > > > + internals->nb_rx_queues = nb_rx_queues; > > > + internals->nb_tx_queues = nb_tx_queues; > > > + for (i = 0; i < nb_rx_queues; i++) { > > > + int idx; > > > + for (idx = 0; idx < pastreamids->count; idx++) { > > > + internals->rxq[i].astreamids.value[idx] = > > > + pastreamids->value[idx] + i; > > > + } > > > + > > > + internals->rxq[i].astreamids.count = pastreamids->count; > > > + > > > + internals->rxq[i].seg = NULL; > > > + internals->rxq[i].enabled = 0; > > > + } > > > + > > > + for (i = 0; i < nb_tx_queues; i++) { > > > + if (port < info_port.u.port_v7.data.adapterInfo.portOffset) { > > > + RTE_LOG(ERR, PMD, "Error wrong port specified\n"); > > > + return -1; > > > + } > > > + internals->txq[i].port = port; > > > + internals->txq[i].enabled = 0; > > > + } > > > + > > > + switch (info_port.u.port_v7.data.speed) { > > > + case NT_LINK_SPEED_UNKNOWN: > > > + pmd_link.link_speed = ETH_SPEED_NUM_1G; > > > + break; > > > + case NT_LINK_SPEED_10M: > > > + pmd_link.link_speed = ETH_SPEED_NUM_10M; > > > + break; > > > + case NT_LINK_SPEED_100M: > > > + pmd_link.link_speed = ETH_SPEED_NUM_100M; > > > + break; > > > + case NT_LINK_SPEED_1G: > > > + pmd_link.link_speed = ETH_SPEED_NUM_1G; > > > + break; > > > + case NT_LINK_SPEED_10G: > > > + pmd_link.link_speed = ETH_SPEED_NUM_10G; > > > + break; > > > + case NT_LINK_SPEED_40G: > > > + pmd_link.link_speed = ETH_SPEED_NUM_40G; > > > + break; > > > + case NT_LINK_SPEED_50G: > > > + pmd_link.link_speed = ETH_SPEED_NUM_50G; > > > + break; > > > + case NT_LINK_SPEED_100G: > > > + pmd_link.link_speed = ETH_SPEED_NUM_100G; > > > + break; > > > + } > > > + > > > + memcpy(ð_addr[port].addr_bytes, > > &info_port.u.port_v7.data.macAddress, > > > + sizeof(eth_addr[port].addr_bytes)); > > > + status = (*_NT_InfoClose)(info); > > > + if (status != NT_SUCCESS) { > > > + (*_NT_ExplainError)(status, err_buf, sizeof(err_buf)); > > > + RTE_LOG(ERR, PMD, > > > + ">>> Error: NT_InfoOpen failed. Code 0x%x = %s\n", > > > + status, err_buf); > > > + return status; > > > + } > > > + > > > + pmd_link.link_duplex = ETH_LINK_FULL_DUPLEX; > > > + pmd_link.link_status = 0; > > > + > > > + internals->if_index = 0; > > > + > > > + pci_dev->numa_node = numa_node; > > > + > > > + data->dev_private = internals; > > > + data->port_id = (*eth_dev)->data->port_id; > > > + data->nb_rx_queues = (uint16_t)nb_rx_queues; > > > + data->nb_tx_queues = (uint16_t)nb_tx_queues; > > > + data->dev_link = pmd_link; > > > + data->mac_addrs = ð_addr[port]; > > > + data->numa_node = numa_node; > > > + data->drv_name = drivername; > > > + > > > + (*eth_dev)->data = data; > > > + (*eth_dev)->dev_ops = &ops; > > > + (*eth_dev)->pci_dev = pci_dev; > > > + > > > + return 0; > > > + > > > +error: > > > + if (data) > > > + rte_free(data); > > > + if (pci_dev) > > > + rte_free(pci_dev); > > > + if (internals) > > > + rte_free(internals); > > > + return -1; > > > +} > > > + > > > +/* > > > + * convert ascii to int > > > + */ > > > +static inline int > > > +ascii_to_u32(const char *key __rte_unused, > > > + const char *value, void *extra_args __rte_unused) > > > +{ > > > + *(uint32_t *)extra_args = atoi(value); > > > + return 0; > > > +} > > > + > > > +static inline int > > > +ascii_to_u32_array(const char *key __rte_unused, > > > + const char *value, void *extra_args) > > > +{ > > > + struct array_s *pastreams = (struct array_s *)extra_args; > > > + int sval, eval; > > > + if (sscanf(value, "%d..%d", &sval, &eval) == 2) { > > > + int cnt; > > > + for (cnt = sval; cnt <= eval; cnt++) > > > + pastreams->value[pastreams->count++] = cnt; > > > + } else { > > > + pastreams->value[pastreams->count++] = atoi(value); > > > + } > > > + return 0; > > > +} > > > + > > > + > > > +static int DoNtpl(const char *ntplStr) > > > +{ > > > + NtConfigStream_t hCfgStream; > > > + NtNtplInfo_t ntplInfo; > > > + int status; > > > + > > > + status = (*_NT_ConfigOpen)(&hCfgStream, "capture"); > > > + if (status != NT_SUCCESS) { > > > + /* Get the status code as text */ > > > + (*_NT_ExplainError)(status, ebuf, sizeof(ebuf) - 1); > > > + fprintf(stderr, "NT_ConfigOpen() failed: %s\n", ebuf); > > > + return -1; > > > + } > > > + > > > + RTE_LOG(INFO, PMD, "NTPL : %s\n", ntplStr); > > > + status = (*_NT_NTPL)(hCfgStream, ntplStr, &ntplInfo, > > > + NT_NTPL_PARSER_VALIDATE_NORMAL); > > > + if (status != NT_SUCCESS) { > > > + /* Get the status code as text */ > > > + (*_NT_ExplainError)(status, ebuf, sizeof(ebuf) - 1); > > > + fprintf(stderr, "NT_NTPL() failed: %s\n", ebuf); > > > + fprintf(stderr, ">>> NTPL errorcode: %X\n", > > > + ntplInfo.u.errorData.errCode); > > > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[0]); > > > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[1]); > > > + fprintf(stderr, ">>> %s\n", ntplInfo.u.errorData.errBuffer[2]); > > > + (*_NT_ConfigClose)(hCfgStream); > > > + return -1; > > > + } > > > + (*_NT_ConfigClose)(hCfgStream); > > > + return 0; > > > +} > > > + > > > + > > > +static int > > > +_nt_lib_open(void) > > > +{ > > > + char path[128]; > > > + strcpy(path, NAPATECH3_LIB_PATH); > > > + strcat(path, "/libntapi.so"); > > > + > > > + /* Load the library */ > > > + _libnt = dlopen(path, RTLD_NOW); > > > + if (_libnt == NULL) { > > > + /* Library does not exist. */ > > > + fprintf(stderr, "Failed to find needed library : %s\n", path); > > > + return -1; > > > + } > > > + _NT_Init = dlsym(_libnt, "NT_Init"); > > > + if (_NT_Init == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_Init\" in %s\n", path); > > > + return -1; > > > + } > > > + > > > + _NT_ConfigOpen = dlsym(_libnt, "NT_ConfigOpen"); > > > + if (_NT_ConfigOpen == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_ConfigOpen\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_ConfigClose = dlsym(_libnt, "NT_ConfigClose"); > > > + if (_NT_ConfigClose == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_ConfigClose\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NTPL = dlsym(_libnt, "NT_NTPL"); > > > + if (_NT_NTPL == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_NTPL\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + > > > + _NT_InfoOpen = dlsym(_libnt, "NT_InfoOpen"); > > > + if (_NT_InfoOpen == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_InfoOpen\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_InfoRead = dlsym(_libnt, "NT_InfoRead"); > > > + if (_NT_InfoRead == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_InfoRead\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_InfoClose = dlsym(_libnt, "NT_InfoClose"); > > > + if (_NT_InfoClose == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_InfoClose\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + > > > + _NT_ExplainError = dlsym(_libnt, "NT_ExplainError"); > > > + if (_NT_ExplainError == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_ExplainError\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NetTxOpen = dlsym(_libnt, "NT_NetTxOpen"); > > > + if (_NT_NetTxOpen == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_NetTxOpen\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NetTxRingbufferInit = dlsym(_libnt, "NT_NetTxRingbufferInit"); > > > + if (_NT_NetTxRingbufferInit == NULL) { > > > + fprintf(stderr, > > > + "Failed to find \"NT_NetTxRingbufferInit\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NetTxRingbufferFlush = dlsym(_libnt, > > "NT_NetTxRingbufferFlush"); > > > + if (_NT_NetTxRingbufferFlush == NULL) { > > > + fprintf(stderr, > > > + "Failed to find \"NT_NetTxRingbufferFlush\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NetTxRingbufferTransmitPacket = > > > + dlsym(_libnt, "NT_NetTxRingbufferTransmitPacket"); > > > + if (_NT_NetTxRingbufferTransmitPacket == NULL) { > > > + fprintf(stderr, > > > + "Failed to find \"NT_NetTxRingbufferTransmitPacket\"" > > > + " in %s\n", path); > > > + return -1; > > > + } > > > + _NT_NetTxRingbufferDone = dlsym(_libnt, > > "NT_NetTxRingbufferDone"); > > > + if (_NT_NetTxRingbufferDone == NULL) { > > > + fprintf(stderr, > > > + "Failed to find \"NT_NetTxRingbufferDone\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NetTxClose = dlsym(_libnt, "NT_NetTxClose"); > > > + if (_NT_NetTxClose == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_NetTxClose\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + > > > + _NT_NetRxOpen = dlsym(_libnt, "NT_NetRxOpen"); > > > + if (_NT_NetRxOpen == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_NetRxOpen\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NetRxGet = dlsym(_libnt, "NT_NetRxGet"); > > > + if (_NT_NetRxGet == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_NetRxGet\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NetRxRelease = dlsym(_libnt, "NT_NetRxRelease"); > > > + if (_NT_NetRxRelease == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_NetRxRelease\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + _NT_NetRxClose = dlsym(_libnt, "NT_NetRxClose"); > > > + if (_NT_NetRxClose == NULL) { > > > + fprintf(stderr, "Failed to find \"NT_NetRxClose\" in %s\n", > > > + path); > > > + return -1; > > > + } > > > + > > > + return 0; > > > +} > > > + > > > + > > > + > > > +static int > > > +rte_pmd_ntnic_devinit(const char *name, const char *params) > > > +{ > > > + unsigned int numa_node; > > > + int ret = 0; > > > + struct rte_kvargs *kvlist; > > > + struct rte_eth_dev *eth_dev; > > > + unsigned int i; > > > + uint32_t rxqueues = 0; > > > + uint32_t txqueues = 0; > > > + uint32_t port = 0; > > > + uint32_t portend = (uint32_t)-1; > > > + uint32_t hash = (uint32_t)-1; > > > + struct array_s astreamids; > > > + > > > + static int first = 1; > > > + static int stream_id; > > > + char ntplStr[512]; > > > + > > > + astreamids.count = 0; > > > + > > > + RTE_LOG(INFO, PMD, "Initializing pmd_ntnic for %s\n", name); > > > + > > > + numa_node = rte_socket_id(); > > > + > > > + kvlist = rte_kvargs_parse(params, valid_arguments); > > > + if (kvlist == NULL) > > > + return -1; > > > + > > > + /* Get port to use for Rx/Tx */ > > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORT_ARG); > > > + if (i) { > > > + assert(i == 1); > > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORT_ARG, > > > + &ascii_to_u32, &port); > > > + } > > > + /* If Rx port merge is need, her the portend is specified */ > > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_PORTEND_ARG); > > > + if (i) { > > > + assert(i == 1); > > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_PORTEND_ARG, > > > + &ascii_to_u32, &portend); > > > + } > > > + > > > + /* Get # RX queues */ > > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_RXQUEUES_ARG); > > > + if (i) { > > > + assert(i == 1); > > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_RXQUEUES_ARG, > > > + &ascii_to_u32, &rxqueues); > > > + } > > > + /* Get # TX queues */ > > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_TXQUEUES_ARG); > > > + if (i) { > > > + assert(i == 1); > > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_TXQUEUES_ARG, > > > + &ascii_to_u32, &txqueues); > > > + } > > > + > > > + /* Get list of streamIds - if used */ > > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_STREAMID_ARG); > > > + if (i) { > > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_STREAMID_ARG, > > > + &ascii_to_u32_array, &astreamids); > > > + } > > > + > > > + /* Get an alternative hash algorithm */ > > > + i = rte_kvargs_count(kvlist, ETH_NTNIC_HASH_ARG); > > > + if (i) { > > > + assert(i == 1); > > > + ret = rte_kvargs_process(kvlist, ETH_NTNIC_HASH_ARG, > > > + &ascii_to_u32, &hash); > > > + } > > > + > > > + /* check portend and streamids */ > > > + if (portend != (uint32_t)-1 && astreamids.count) { > > > + RTE_LOG(ERR, PMD, "Cannot specify portend when one or more" > > > + " streamid's are specified\n"); > > > + return -1; > > > + } > > > + rte_kvargs_free(kvlist); > > > + > > > + if (ret < 0) > > > + return -1; > > > + > > > + ret = _nt_lib_open(); > > > + if (ret < 0) > > > + return -1; > > > + > > > + if (first) > > > + (*_NT_Init)(NTAPI_VERSION); > > > + > > > + if (astreamids.count) { > > > + first = 0; > > > + /* > > > + * if a specific streamid specified then use that otherwise, > > > + * port defaults to Tx port only > > > + */ > > > + if (rte_pmd_init_internals(name, rxqueues, txqueues, > > numa_node, > > > + &astreamids, port, ð_dev) < 0) > > > + return -1; > > > + } else { > > > + struct array_s astrids; > > > + if (first) { > > > + /* Delete all NTPL */ > > > + sprintf(ntplStr, "Delete=All"); > > > + if (DoNtpl(ntplStr) != 0) > > > + return -1; > > > + first = 0; > > > + } > > > + astrids.count = 1; > > > + astrids.value[0] = stream_id; > > > + if (rte_pmd_init_internals(name, rxqueues, txqueues, > > numa_node, > > > + &astrids, port, ð_dev) < 0) > > > + return -1; > > > + > > > + /* Assign the traffic */ > > > + if (portend != (uint32_t)-1) { > > > + sprintf(ntplStr, > > > + "Assign[streamid=(%d..%d);Descriptor=NT]" > > > + "=port==(%d..%d)", > > > + stream_id, > > > + (stream_id + rxqueues - 1), port, portend); > > > + > > > + } else { > > > + sprintf(ntplStr, > > > + "Assign[streamid=(%d..%d);Descriptor=NT]" > > > + "=port==%d", > > > + stream_id, > > > + (stream_id + rxqueues - 1), port); > > > + } > > > + if (DoNtpl(ntplStr) != 0) > > > + return -1; > > > + > > > + if (hash != (uint32_t)-1) { > > > + switch (hash) { > > > + default: > > > + case 1: > > > + DoNtpl("HashMode=HashRoundRobin"); > > > + break; > > > + case 2: > > > + DoNtpl("HashMode=Hash2TupleSorted"); > > > + break; > > > + case 3: > > > + DoNtpl("HashMode=Hash5TupleSorted"); > > > + break; > > > + } > > > + } > > > + > > > + stream_id += rxqueues; > > > + } > > > + > > > + eth_dev->rx_pkt_burst = eth_ntnic_rx; > > > + eth_dev->tx_pkt_burst = eth_ntnic_tx_ringbuffer; > > > + > > > + return 0; > > > +} > > > + > > > +static int > > > +rte_pmd_ntnic_devuninit(const char *name) > > > +{ > > > + (void)name; > > > + if (_libnt != NULL) > > > + dlclose(_libnt); > > > + return 0; > > > +} > > > + > > > + > > > +static struct rte_driver pmd_ntnic_drv = { > > > + .type = PMD_VDEV, > > > + .init = rte_pmd_ntnic_devinit, > > > + .uninit = rte_pmd_ntnic_devuninit, > > > +}; > > > + > > > +PMD_REGISTER_DRIVER(pmd_ntnic_drv, eth_ntnic); > > > +DRIVER_REGISTER_PARAM_STRING(eth_ntnic, > > > + "port=<int> " > > > + "rxqs=<int>" > > > + "txqs=<int>" > > > + "hash=<int>" > > > + "streamids=<int..int>"); > > > + > > > diff --git a/drivers/net/ntnic/rte_pmd_ntnic_version.map > > b/drivers/net/ntnic/rte_pmd_ntnic_version.map > > > new file mode 100644 > > > index 0000000..ef35398 > > > --- /dev/null > > > +++ b/drivers/net/ntnic/rte_pmd_ntnic_version.map > > > @@ -0,0 +1,4 @@ > > > +DPDK_2.0 { > > > + > > > + local: *; > > > +}; > > > diff --git a/mk/rte.app.mk b/mk/rte.app.mk > > > index 1a0095b..9ef357f 100644 > > > --- a/mk/rte.app.mk > > > +++ b/mk/rte.app.mk > > > @@ -120,6 +120,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_MPIPE_PMD) += > > -lrte_pmd_mpipe -lgxio > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += -lrte_pmd_nfp -lm > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += -lrte_pmd_null > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_PCAP) += -lrte_pmd_pcap -lpcap > > > + > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_QEDE_PMD) += -lrte_pmd_qede -lz > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_RING) += -lrte_pmd_ring > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_SZEDATA2) += - > > lrte_pmd_szedata2 -lsze2 > > > @@ -143,6 +144,7 @@ _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) > > += -lrte_pmd_kasumi > > > _LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_KASUMI) += - > > L$(LIBSSO_KASUMI_PATH)/build -lsso_kasumi > > > endif # CONFIG_RTE_LIBRTE_CRYPTODEV > > > > > > +_LDLIBS-$(CONFIG_RTE_LIBRTE_PMD_NTNIC) += -lrte_pmd_ntnic > > > endif # !CONFIG_RTE_BUILD_SHARED_LIBS > > > > > > _LDLIBS-y += --no-whole-archive > > > @@ -163,6 +165,7 @@ endif > > > _LDLIBS-$(CONFIG_RTE_PORT_PCAP) += -lpcap > > > endif # !CONFIG_RTE_BUILD_SHARED_LIBS > > > > > > + > > > _LDLIBS-y += $(EXECENV_LDLIBS) > > > > > > LDLIBS += $(_LDLIBS-y) $(CPU_LDLIBS) $(EXTRA_LDLIBS) > > > -- > > > 2.9.0 > > > > > > Disclaimer: This email and any files transmitted with it may contain > > confidential information intended for the addressee(s) only. The information > > is not to be surrendered or copied to unauthorized persons. If you have > > received this communication in error, please notify the sender immediately > > and delete this e-mail from your system. > > > > Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system. > ^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2016-11-21 13:55 UTC | newest] Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-08-26 13:44 [dpdk-dev] [PATCH] ntnic: add PMD driver Finn Christensen 2016-08-26 14:44 ` Thomas Monjalon 2016-08-26 16:32 ` Finn Christensen 2016-08-27 9:07 ` Thomas Monjalon 2016-08-26 16:54 ` Stephen Hemminger 2016-08-29 6:22 ` Finn Christensen 2016-08-29 10:04 ` Thomas Monjalon 2016-08-29 12:00 ` Finn Christensen 2016-09-08 11:14 ` [dpdk-dev] [PATCH v2] " Finn Christensen 2016-09-08 13:49 ` Neil Horman 2016-09-08 14:22 ` Finn Christensen 2016-09-09 12:48 ` [dpdk-dev] [PATCH v3] " Finn Christensen 2016-09-09 13:51 ` Neil Horman 2016-09-10 7:58 ` Finn Christensen 2016-09-10 8:20 ` Thomas Monjalon 2016-09-10 18:31 ` Stephen Hemminger 2016-09-12 8:08 ` Finn Christensen 2016-09-12 12:33 ` Neil Horman 2016-09-12 7:34 ` Finn Christensen 2016-11-21 13:47 ` Ferruh Yigit 2016-11-21 13:55 ` Finn Christensen 2016-09-12 12:32 ` Neil Horman
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).