From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm1-f66.google.com (mail-wm1-f66.google.com [209.85.128.66]) by dpdk.org (Postfix) with ESMTP id 5F3FF6CC1 for ; Thu, 7 Mar 2019 14:47:01 +0100 (CET) Received: by mail-wm1-f66.google.com with SMTP id n19so9308471wmi.1 for ; Thu, 07 Mar 2019 05:47:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:content-transfer-encoding:user-agent:mime-version; bh=QGz/cXCzQCGXESPrzLOtYt/To8UpX6boYudyuwbOss0=; b=Q1D4S4g2c/A5WGUMZO1I3H772Xmxdr5HERdwe74tZ9fwZMOLzJTJOLBdwIYdl/Liv7 cUrp1OiouxD5KrkNSZEA5op5oxCOS324Usj8PoQwsF5zHuGQF9ywnUW9OYgrmU+hcQEe KBLcgFhtllgmBSsUq9K1R8myLPQAtj821w+VEu7Lk9hrx0gc8a7CydcbknLXmOtuK53j o+lWfdLSXsFmsyLR/9Q+MVKNSuTyFl8PotUxiRqm3JxmNOkMGX24zE+KVgG+0RAWAGgX Sk/IDsH3gN2Q/nkIbkK3Ro1l7Cgp/BADCkGefjtL3X1NsUPi69CXUPN4zl0dHSPWwMmt Gfyg== X-Gm-Message-State: APjAAAVmF0O9+OI9R1WnK7NuQdEw35Y0hVisfKbVONsXLrrLUhc+hLcQ Z01cPd0Y+F1H5QXLOf6pHkc= X-Google-Smtp-Source: APXvYqy6h32ly7r/ReriU60EmP8eCGDmIj6NB7rg9dwJjelKckiDFLMVhf11EweAL1FT9KfefL93OQ== X-Received: by 2002:a1c:751a:: with SMTP id o26mr5867336wmc.10.1551966420730; Thu, 07 Mar 2019 05:47:00 -0800 (PST) Received: from localhost ([2a01:4b00:f419:6f00:b00c:66c8:99df:336]) by smtp.gmail.com with ESMTPSA id f68sm10377989wmg.5.2019.03.07.05.46.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 07 Mar 2019 05:46:59 -0800 (PST) Message-ID: From: Luca Boccassi To: Rastislav Cernay , dev@dpdk.org Date: Thu, 07 Mar 2019 13:46:59 +0000 In-Reply-To: <1551965093-273264-1-git-send-email-cernay@netcope.com> References: <1551185824-5501-2-git-send-email-cernay@netcope.com> <1551965093-273264-1-git-send-email-cernay@netcope.com> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.30.5-1 MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v4] net/nfb: new netcope driver X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2019 13:47:01 -0000 On Thu, 2019-03-07 at 14:24 +0100, Rastislav Cernay wrote: > From: Rastislav Cernay >=20 > Note: New netcope-commong pcg on which is support for pkg-config, > will be released soon and uploaded on official site. >=20 > Added new net driver for Netcope nfb cards >=20 > Signed-off-by: Rastislav Cernay > --- > v2: remove unnecessary cast > remove unnecessary zeroing > move declaration to not mix with code > restore skeleton example > v3: add release notes > add doc to doc index > add architecture limits to doc > edit features list > add .map file > change link to dependecies to official vendor site > move declarations out of code > remove false comments (rte_errno is set) > comments to c89 style > remove log from main rx loop > remove redundant code > v4: add random initial MAC > fix format > move declarations out of code > fix error log in TX > q_errors is for reception errors > pmd is experimental > dependency check fixed > support for pkg-config > MAINTAINERS | 7 + > config/common_base | 4 + > devtools/test-build.sh | 1 + > doc/guides/nics/features/nfb.ini | 18 + > doc/guides/nics/index.rst | 1 + > doc/guides/nics/nfb.rst | 143 +++++++ > doc/guides/rel_notes/release_19_02.rst | 5 + > drivers/net/Makefile | 1 + > drivers/net/meson.build | 1 + > drivers/net/nfb/Makefile | 44 +++ > drivers/net/nfb/meson.build | 15 + > drivers/net/nfb/nfb.h | 50 +++ > drivers/net/nfb/nfb_ethdev.c | 647 > ++++++++++++++++++++++++++++++++ > drivers/net/nfb/nfb_rx.c | 127 +++++++ > drivers/net/nfb/nfb_rx.h | 231 ++++++++++++ > drivers/net/nfb/nfb_rxmode.c | 100 +++++ > drivers/net/nfb/nfb_rxmode.h | 96 +++++ > drivers/net/nfb/nfb_stats.c | 77 ++++ > drivers/net/nfb/nfb_stats.h | 56 +++ > drivers/net/nfb/nfb_tx.c | 113 ++++++ > drivers/net/nfb/nfb_tx.h | 222 +++++++++++ > drivers/net/nfb/rte_nfb_pmd_version.map | 4 + > mk/rte.app.mk | 2 + > 23 files changed, 1965 insertions(+) > create mode 100644 doc/guides/nics/features/nfb.ini > create mode 100644 doc/guides/nics/nfb.rst > create mode 100644 drivers/net/nfb/Makefile > create mode 100644 drivers/net/nfb/meson.build > create mode 100644 drivers/net/nfb/nfb.h > create mode 100644 drivers/net/nfb/nfb_ethdev.c > create mode 100644 drivers/net/nfb/nfb_rx.c > create mode 100644 drivers/net/nfb/nfb_rx.h > create mode 100644 drivers/net/nfb/nfb_rxmode.c > create mode 100644 drivers/net/nfb/nfb_rxmode.h > create mode 100644 drivers/net/nfb/nfb_stats.c > create mode 100644 drivers/net/nfb/nfb_stats.h > create mode 100644 drivers/net/nfb/nfb_tx.c > create mode 100644 drivers/net/nfb/nfb_tx.h > create mode 100644 drivers/net/nfb/rte_nfb_pmd_version.map <...> > + > +.. note:: > + > + This driver has external dependencies. > + Therefore it is disabled in default configuration files. > + It can be enabled by setting ``CONFIG_RTE_LIBRTE_PMD_NFB=3Dy`` > + and recompiling. > + > +.. note:: > + > + Currently the driver is supported only on x86_64 architectures. > + Only x86_64 versions of the external libraries are provided. > + > +Prerequisites > +------------- > + > +This PMD requires kernel modules which are responsible for > initialization and > +allocation of resources needed for nfb layer function. > +Communication between PMD and kernel modules is mediated by libnfb > library. > +These kernel modules and library are not part of DPDK and must be > installed > +separately: > + > +* **libnfb library** > + > + The library provides API for initialization of nfb transfers, > receiving and > + transmitting data segments. > + > +* **Kernel modules** > + > + * nfb > + > + Kernel modules manage initialization of hardware, allocation and > + sharing of resources for user space applications. > + > +Dependencies can be found here: > +`Netcope common < > https://www.netcope.com/en/company/community-support/dpdk-libsze2#NFB>`_ > . I see netcope-common provides a pkg-config file, that's great. However it's a bit broken at the moment: Name: netcope-common Version: 6.3.0 Description: Common software package for NetCOPE platform Libs: -L${libdir} Cflags: -I${includedir} There is no linker flag, and Libs.private is missing. It should have: Libs: -L${libdir} -lnfb Libs.private: -lfdt Could you please get it fixed? > +Versions of the packages > +~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The minimum version of the provided packages: > + > +* for DPDK from 19.02 > + > +Configuration > +------------- > + > +These configuration options can be modified before compilation in > the > +``.config`` file: > + > +* ``CONFIG_RTE_LIBRTE_PMD_NFB`` default value: **n** > + > + Value **y** enables compilation of nfb PMD. > + > +Using the NFB PMD > +---------------------- > + > +Kernel modules have to be loaded before running the DPDK > application. > + > +NFB card architecture > +--------------------- > + > +The NFB cards are multi-port multi-queue cards, where (generally) > data from any > +Ethernet port may be sent to any queue. > +They are represented in DPDK as a single port. > + > +NFB-200G2QL card employs an addon cable which allows to connect it > to two > +physical PCI-E slots at the same time (see the diagram below). > +This is done to allow 200 Gbps of traffic to be transferred through > the PCI-E > +bus (note that a single PCI-E 3.0 x16 slot provides only 125 Gbps > theoretical > +throughput). > + > +Although each slot may be connected to a different CPU and therefore > to a different > +NUMA node, the card is represented as a single port in DPDK. To work > with data > +from the individual queues on the right NUMA node, connection of > NUMA nodes on > +first and last queue (each NUMA node has half of the queues) need to > be checked. > + > +.. figure:: img/szedata2_nfb200g_architecture.svg > + :align: center > + > + NFB-200G2QL high-level diagram > + > +Limitations > +----------- > + > +Driver is usable only on Linux architecture. > + > +Since a card is always represented as a single port, but can be > connected to two > +NUMA nodes, there is need for manual check where master/slave is > connected. > + > +Example of usage > +---------------- > + > +Read packets from 0. and 1. receive queue and write them to 0. and > 1. > +transmit queue: > + > +.. code-block:: console > + > + $RTE_TARGET/app/testpmd -l 0-3 -n 2 \ > + -- --port-topology=3Dchained --rxq=3D2 --txq=3D2 --nb-cores=3D2 -i -a > + > +Example output: > + > +.. code-block:: console > + > + [...] > + EAL: PCI device 0000:06:00.0 on NUMA socket -1 > + EAL: probe driver: 1b26:c1c1 net_nfb > + PMD: Initializing NFB device (0000:06:00.0) > + PMD: Available DMA queues RX: 8 TX: 8 > + PMD: NFB device (0000:06:00.0) successfully initialized > + Interactive-mode selected > + Auto-start selected > + Configuring Port 0 (socket 0) > + Port 0: 00:11:17:00:00:00 > + Checking link statuses... > + Port 0 Link Up - speed 10000 Mbps - full-duplex > + Done > + Start automatic packet forwarding > + io packet forwarding - CRC stripping disabled - > packets/burst=3D32 > + nb forwarding cores=3D2 - nb forwarding ports=3D1 > + RX queues=3D2 - RX desc=3D128 - RX free threshold=3D0 > + RX threshold registers: pthresh=3D0 hthresh=3D0 wthresh=3D0 > + TX queues=3D2 - TX desc=3D512 - TX free threshold=3D0 > + TX threshold registers: pthresh=3D0 hthresh=3D0 wthresh=3D0 > + TX RS bit threshold=3D0 - TXQ flags=3D0x0 > + testpmd> > diff --git a/doc/guides/rel_notes/release_19_02.rst > b/doc/guides/rel_notes/release_19_02.rst > index 3372c4d..9955b6b 100644 > --- a/doc/guides/rel_notes/release_19_02.rst > +++ b/doc/guides/rel_notes/release_19_02.rst > @@ -54,6 +54,11 @@ New Features > Also, make sure to start the actual text at the margin. > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > =20 > +* **Added an experimental net PMD NFB.** > + > + Added the new ``nfb`` net driver for Netcope NFB cards. See > + the :doc:`../nics/nfb` NIC guide for more details on this new > driver. > + > * **Added support to free hugepages exactly as originally > allocated.** > =20 > Some applications using memory event callbacks (especially for > managing > diff --git a/drivers/net/Makefile b/drivers/net/Makefile > index 670d7f7..013c551 100644 > --- a/drivers/net/Makefile > +++ b/drivers/net/Makefile > @@ -38,6 +38,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) +=3D mlx5 > DIRS-$(CONFIG_RTE_LIBRTE_MVNETA_PMD) +=3D mvneta > DIRS-$(CONFIG_RTE_LIBRTE_MVPP2_PMD) +=3D mvpp2 > DIRS-$(CONFIG_RTE_LIBRTE_NETVSC_PMD) +=3D netvsc > +DIRS-$(CONFIG_RTE_LIBRTE_PMD_NFB) +=3D nfb > DIRS-$(CONFIG_RTE_LIBRTE_NFP_PMD) +=3D nfp > DIRS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) +=3D bnxt > DIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) +=3D null > diff --git a/drivers/net/meson.build b/drivers/net/meson.build > index 45da3bb..6733a19 100644 > --- a/drivers/net/meson.build > +++ b/drivers/net/meson.build > @@ -27,6 +27,7 @@ drivers =3D ['af_packet', > 'mvneta', > 'mvpp2', > 'netvsc', > + 'nfb', > 'nfp', > 'null', 'octeontx', 'pcap', 'qede', 'ring', > 'sfc', > diff --git a/drivers/net/nfb/Makefile b/drivers/net/nfb/Makefile > new file mode 100644 > index 0000000..968184e > --- /dev/null > +++ b/drivers/net/nfb/Makefile > @@ -0,0 +1,44 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Cesnet > +# Copyright(c) 2018 Netcope Technologies, a.s. > +# All rights reserved. > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +# > +# library name > +# > +LIB =3D librte_pmd_nfb.a > + > +INCLUDES :=3D-I$(SRCDIR) > + > + > +CFLAGS +=3D -O3 > +CFLAGS +=3D $(WERROR_FLAGS) > +CFLAGS +=3D -DALLOW_EXPERIMENTAL_API > +CFLAGS +=3D $(shell command -v pkg-config > /dev/null 2>&1 && pkg- > config --cflags libnfb) This will not work - the pkg-config filename is netcope-common.pc, not libnfb.pc. The convention would be to rename the pkg-config file to libnfb.pc, but if you want to keep the current name then all pkg-config=20 calls in this patch need to be updated. > +LDLIBS +=3D -lnfb Do not hard-code this, pkg-config is used below. > +LDLIBS +=3D -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring > +LDLIBS +=3D -lrte_ethdev -lrte_net -lrte_kvargs > +LDLIBS +=3D -lrte_bus_pci > +LDLIBS +=3D $(shell command -v pkg-config > /dev/null 2>&1 && pkg- > config --libs libnfb) Use: LDLIBS +=3D $(shell command -v pkg-config > /dev/null 2>&1 && pkg-config --= libs libnfb || echo "-lnfb") This way you have a fallback if pkg-config is not installed. > +EXPORT_MAP :=3D rte_pmd_nfb_version.map > + > +LIBABIVER :=3D 1 > + > +# > +# all source are stored in SRCS-y > +# > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) +=3D nfb_ethdev.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) +=3D nfb_rx.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) +=3D nfb_tx.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) +=3D nfb_stats.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) +=3D nfb_rxmode.c > + > +# > +# Export include files > +# > +SYMLINK-y-include +=3D > + > +include $(RTE_SDK)/mk/rte.lib.mk > diff --git a/drivers/net/nfb/meson.build > b/drivers/net/nfb/meson.build > new file mode 100644 > index 0000000..16df146 > --- /dev/null > +++ b/drivers/net/nfb/meson.build > @@ -0,0 +1,15 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Cesnet > +# Copyright(c) 2018 Netcope Technologies, a.s. > +# All rights reserved. > + > +dep =3D cc.find_library('nfb', required: false) > + > +build =3D dep.found() and cc.has_header('nfb/nfb.h', dependencies: > dep) > + > +allow_experimental_apis =3D true > + > +ext_deps +=3D dep > + > +sources =3D files('nfb_rx.c', 'nfb_tx.c', 'nfb_stats.c', > 'nfb_ethdev.c', 'nfb_rxmode.c') > +pkgconfig_extra_libs +=3D '-lnfb' libnfb ships a pkg-config file, so please do this instead so that static compilation can work correctly: dep =3D dependency('netcope-common', required: false) build =3D dep.found() allow_experimental_apis =3D true ext_deps +=3D dep sources =3D files('nfb_rx.c', 'nfb_tx.c', 'nfb_stats.c', 'nfb_ethdev.c', 'n= fb_rxmode.c') --=20 Kind regards, Luca Boccassi