Re: [dpdk-dev] [PATCH v5] net/nfb: new netcope driver

[Ferruh Yigit <ferruh.yigit@intel.com>]

On 4/1/2019 3:55 PM, Rastislav Černay wrote:
> Hi Ferruh,
> 
> I fixed most of issues you wrote, but before I send v6, I would like to make
> sure everything is in order, mainly the experimental issue.
> 
> I have a problem here, this requires 'netcope-common-6.4.0-1.x86_64.rpm' rpm to
> be installed [1], meanwhile szedata requires
> 'netcope-common-light-3.0.5-1.x86_64.rpm' to be installed, and these two
> packages conflicts with eachother.
> 
> What is the way of having an environment that dependencies of two netcope HW can
> be provided at same time? This is really time consuming each time update system
> for it.
> 
> <...>
> Sadly, szedata2 and nfb dirvers can not coexist in same enviroment. There is no
> way to provide dependecies at same time.

Isn't this also problem for you too? What is your solution when testing both?

> 
> Is rpm only option, how people using Ubuntu for example will use your devices?
> <...>
> For now, yes, only option is rpm and support for Ubuntu will be added later.

Is this limitation in driver document? If not can you please update it.

> 
> 
> Why allow_experimental only in meson, but not Makefile?
> <...>
> It is in Makefile too.
> CFLAGS += -DALLOW_EXPERIMENTAL_API
> 
> Can you please add list of experimental APIs called just below this as comment?
> <...>
> No experimental API is called in driver.

OK, so please clean the meson file.

> 
> btw, why do you prefer to have driver as experimental?
> What does it mean it being experimental from your point of view exactly?
> <...>
> I do not prefer to have driver experimental, only reason it is there is
> this mail:
> 
> Luca Boccassi bluca@debian.org <mailto:bluca@debian.org>
> Tue, Mar 5, 11:41 PM
> to me, dev
> <...>
>> diff --git a/drivers/net/nfb/rte_nfb_pmd_version.map
>> b/drivers/net/nfb/rte_nfb_pmd_version.map
>> new file mode 100644
>> index 0000000..97fd251
>> --- /dev/null
>> +++ b/drivers/net/nfb/rte_nfb_pmd_version.map
>> @@ -0,0 +1,4 @@
>> +DPDK_19.02 {
>> +
>> +        local: *;
>> +};
> 
> These are all new symbols so they should be marked as experimental,
> please see doc/guides/contributing/versioning.rst
> 
> So after reading doc/guides/contributing/versioning.rst I thought that
> all new drivers should be experimental as they are new before they stabilize,
> and during this time changes can be done to public functions without much hassle.
> Should I keep driver experimental?

No please don't keep it experimental. Luca is right new symbols should be marked
as experimental, here news symbols are APIs and this PMD doesn't have any, all
it has is 'local' symbols.
cc'ed Luca if I am missing anything.

And if this is the only reason you documented the PMD as experimental, you can
clean MAINTAINERS file, release notes etc...

> 
> *
> 
> *Rastislav Černay I /Software Developer/*
> Netcope Technologies, a.s.
> 
> *T:* +420 530 510 680 <tel:+420%20530%20510%20680>
> *A:* Sochorova 3232/34, Brno, 616 00, Czech Republic
> <https://maps.google.com/?q=Sochorova+3232/34,+Brno,+616+00,%C2%A0Czech+Republic&entry=gmail&source=g>
> *W: www.netcope.com <https://www.netcope.com/>
> 
>  <https://www.netcope.com/>*
> 
> *
> **
> 
> 
> 
> On Thu, Mar 28, 2019 at 5:01 PM Ferruh Yigit <ferruh.yigit@intel.com
> <mailto:ferruh.yigit@intel.com>> wrote:
> 
>     On 3/22/2019 12:12 PM, Rastislav Cernay wrote:
>     > From: Rastislav Cernay <cernay@netcope.com <mailto:cernay@netcope.com>>
>     >
>     > Added new net driver for Netcope nfb cards
>     >
>     > Signed-off-by: Rastislav Cernay <cernay@netcope.com
>     <mailto:cernay@netcope.com>>
>     > ---
>     > v2: remove unnecessary cast
>     >     remove unnecessary zeroing
>     >     move declaration to not mix with code
>     >     restore skeleton example
>     > v3: add release notes
>     >     add doc to doc index
>     >     add architecture limits to doc
>     >     edit features list
>     >     add .map file
>     >     change link to dependecies to official vendor site
>     >     move declarations out of code
>     >     remove false comments (rte_errno is set)
>     >     comments to c89 style
>     >     remove log from main rx loop
>     >     remove redundant code
>     > v4: API is experimental
>     >     fixed meson build dependency
>     >     random initial MAC
>     >     stats->q_errors is for intput err only
>     >     move more declarations to a beginning
>     >     fixed err log in TX
>     >     use of pkg-config
>     > v5: fixed pkg-config for new version of netcope-common
>     >  MAINTAINERS                             |   7 +
>     >  config/common_base                      |   4 +
>     >  devtools/test-build.sh                  |   1 +
>     >  doc/guides/nics/features/nfb.ini        |  18 +
>     >  doc/guides/nics/index.rst               |   1 +
>     >  doc/guides/nics/nfb.rst                 | 143 +++++++
>     >  doc/guides/rel_notes/release_19_02.rst  |   5 +
> 
>     Can you update the 19.05 release notes please?
> 
>     >  drivers/net/Makefile                    |   1 +
>     >  drivers/net/meson.build                 |   1 +
>     >  drivers/net/nfb/Makefile                |  44 +++
>     >  drivers/net/nfb/meson.build             |  15 +
>     >  drivers/net/nfb/nfb.h                   |  50 +++
>     >  drivers/net/nfb/nfb_ethdev.c            | 647
>     ++++++++++++++++++++++++++++++++
>     >  drivers/net/nfb/nfb_rx.c                | 127 +++++++
>     >  drivers/net/nfb/nfb_rx.h                | 231 ++++++++++++
>     >  drivers/net/nfb/nfb_rxmode.c            | 100 +++++
>     >  drivers/net/nfb/nfb_rxmode.h            |  96 +++++
>     >  drivers/net/nfb/nfb_stats.c             |  77 ++++
>     >  drivers/net/nfb/nfb_stats.h             |  56 +++
>     >  drivers/net/nfb/nfb_tx.c                | 113 ++++++
>     >  drivers/net/nfb/nfb_tx.h                | 222 +++++++++++
>     >  drivers/net/nfb/rte_nfb_pmd_version.map |   4 +
> 
>     The filename should be "rte_pmd_nfb_version.map" as stated in Makefile.
>     This brings the question, did you ever build dpdk as shared library or with
>     meson build system with this PMD enabled :)
>     Because it fails because of this wrong naming..
> 
> 
>     Even after renaming the .map file, meson build fails, I haven't dig the problem,
>     can you please check it?
> 
>     <...>
> 
>     > @@ -360,6 +360,10 @@ CONFIG_RTE_LIBRTE_SFC_EFX_DEBUG=n
>     >  # Compile software PMD backed by SZEDATA2 device
>     >  #
>     >  CONFIG_RTE_LIBRTE_PMD_SZEDATA2=n
>     > +#
>     > +# Compile software PMD backed by NFB device
>     > +#
>     > +CONFIG_RTE_LIBRTE_PMD_NFB=n
> 
>     Can you please rename the config option to "CONFIG_RTE_LIBRTE_NFB_PMD"?
>     I guess unintentionally, there is an convension,
>     Physical PMDs have the config name: CONFIG_RTE_LIBRTE_.*_PMD
>     Virtual PMDs have the config name:  CONFIG_RTE_LIBRTE_PMD_.*=y
>     I perfer same syntax for all, but it is what it is now, and I believe it doesn't
>     worth the hassle of changing them.
> 
>     I think only 'SZEDATA2' breaks current login, and I guess you copied from it,
>     but let's start NFB according convention: CONFIG_RTE_LIBRTE_NFB_PMD
> 
> 
>     Also please leave a blank line between previous config block.
> 
>     <...>
> 
>     > +Prerequisites
>     > +-------------
>     > +
>     > +This PMD requires kernel modules which are responsible for initialization and
>     > +allocation of resources needed for nfb layer function.
>     > +Communication between PMD and kernel modules is mediated by libnfb library.
>     > +These kernel modules and library are not part of DPDK and must be installed
>     > +separately:
>     > +
>     > +*  **libnfb library**
>     > +
>     > +   The library provides API for initialization of nfb transfers,
>     receiving and
>     > +   transmitting data segments.
>     > +
>     > +*  **Kernel modules**
>     > +
>     > +   * nfb
>     > +
>     > +   Kernel modules manage initialization of hardware, allocation and
>     > +   sharing of resources for user space applications.
>     > +
>     > +Dependencies can be found here:
>     > +`Netcope common
>     <https://www.netcope.com/en/company/community-support/dpdk-libsze2#NFB>`_.
> 
>     I have a problem here, this requires 'netcope-common-6.4.0-1.x86_64.rpm' rpm to
>     be installed [1], meanwhile szedata requires
>     'netcope-common-light-3.0.5-1.x86_64.rpm' to be installed, and these two
>     packages conflicts with eachother.
> 
>     What is the way of having an environment that dependencies of two netcope HW can
>     be provided at same time? This is really time consuming each time update system
>     for it.
> 
>     [1]
>     Is rpm only option, how people using Ubuntu for example will use your devices?
> 
>     > +
>     > +Versions of the packages
>     > +~~~~~~~~~~~~~~~~~~~~~~~~
>     > +
>     > +The minimum version of the provided packages:
>     > +
>     > +* for DPDK from 19.02
> 
>     Is it 19.05 now?
> 
>     <...>
> 
>     > +Example output:
>     > +
>     > +.. code-block:: console
>     > +
>     > +   [...]
>     > +   EAL: PCI device 0000:06:00.0 on NUMA socket -1
>     > +   EAL:   probe driver: 1b26:c1c1 net_nfb
>     > +   PMD: Initializing NFB device (0000:06:00.0)
>     > +   PMD: Available DMA queues RX: 8 TX: 8
>     > +   PMD: NFB device (0000:06:00.0) successfully initialized
>     > +   Interactive-mode selected
>     > +   Auto-start selected
>     > +   Configuring Port 0 (socket 0)
>     > +   Port 0: 00:11:17:00:00:00
>     > +   Checking link statuses...
>     > +   Port 0 Link Up - speed 10000 Mbps - full-duplex
>     > +   Done
>     > +   Start automatic packet forwarding
>     > +     io packet forwarding - CRC stripping disabled - packets/burst=32
>     > +     nb forwarding cores=2 - nb forwarding ports=1
>     > +     RX queues=2 - RX desc=128 - RX free threshold=0
>     > +     RX threshold registers: pthresh=0 hthresh=0 wthresh=0
>     > +     TX queues=2 - TX desc=512 - TX free threshold=0
>     > +     TX threshold registers: pthresh=0 hthresh=0 wthresh=0
>     > +     TX RS bit threshold=0 - TXQ flags=0x0
>     > +   testpmd>
> 
>     I think testpmd logging is not like this any more, can you please update the app
>     log with latest code.
> 
>     <...>
> 
>     > @@ -0,0 +1,44 @@
>     > +# SPDX-License-Identifier: BSD-3-Clause
>     > +# Copyright(c) 2018 Cesnet
>     > +# Copyright(c) 2018 Netcope Technologies, a.s. <info@netcope.com
>     <mailto:info@netcope.com>>
> 
>     Should year be 2019, or 2018-2019?
> 
> 
>     <...>
> 
>     > @@ -0,0 +1,15 @@
>     > +# SPDX-License-Identifier: BSD-3-Clause
>     > +# Copyright(c) 2018 Cesnet
>     > +# Copyright(c) 2018 Netcope Technologies, a.s. <info@netcope.com
>     <mailto:info@netcope.com>>
>     > +# All rights reserved.
>     > +
>     > +dep = cc.find_library('nfb', required: false)
>     > +
>     > +build = dep.found() and cc.has_header('nfb/nfb.h', dependencies: dep)
>     > +
>     > +allow_experimental_apis = true
> 
>     Why allow_experimental only in meson, but not Makefile?
>     Can you please add list of experimental APIs called just below this as comment?
> 
>     <...>
> 
>     > +/**
>     > + * Default MAC addr
>     > + */
>     > +static struct ether_addr eth_addr = {
>     > +     .addr_bytes = { 0x00, 0x11, 0x17, 0x00, 0x00, 0x00 }
>     > +};
> 
>     it looks like this can be static const.
> 
>     > +
>     > +/**
>     > + * @warning
>     > + * @b EXPERIMENTAL: this API may change, or be removed, without prior notice
>     > + *
> 
>     Why these tags are added, to say PMD is experimental?
>     These only makes sense for public APIs which PMD doesn't have at all, these are
>     doxygen comments for API documentation. Please remove them.
> 
>     Also I recognized '__rte_experimental' tags, again which are for internal
>     checking and warning the applications if they are using experimental public API,
>     it doesn't apply here. Please remove them.
>     Last thing, the "EXPERIMENTAL" tag in .map file, that also doesn't apply here,
>     please make it release version, 'DPDK_19.05"
> 
>     I think your note about being experimental in MAINTAINERS file and the note in
>     release notes is good,
>     Can you please also update the driver documentation too, nfb.rst, to say driver
>     is experimental? I think those notes are good enough overall.
> 
> 
>     btw, why do you prefer to have driver as experimental?
>     What does it mean it being experimental from your point of view exactly?
> 
> 
>     <...>
> 
>     > +static int __rte_experimental
>     > +nfb_eth_mac_addr_set(struct rte_eth_dev *dev,
>     > +     struct ether_addr *mac_addr)
>     > +{
>     > +     unsigned int i;
>     > +     unsigned long long mac = 0;
> 
>     Would you prefer to use a fixed length deceleration, uint64_t?
> 
>     > +     struct rte_eth_dev_data *data = dev->data;
>     > +     struct pmd_internals *internals = (struct pmd_internals *)
>     > +             data->dev_private;
>     > +
>     > +     if (!is_valid_assigned_ether_addr(mac_addr))
>     > +             return -EINVAL;
>     > +
>     > +     for (i = 0; i < 6; i++) {
> 
>     Can you please use "ETHER_ADDR_LEN" instead of 6?
> 
>     <...>
> 
>     > +     for (i = 0; i < nb_rx; i++) {
>     > +             if (i < RTE_ETHDEV_QUEUE_STAT_CNTRS) {
>     > +                     stats->q_ipackets[i] = rx_queue[i].rx_pkts;
>     > +                     stats->q_ibytes[i] = rx_queue[i].rx_bytes;
>     > +             }
>     > +             rx_total += stats->q_ipackets[i];
>     > +             rx_total_bytes += stats->q_ibytes[i];
> 
>     Why these are not protected with "RTE_ETHDEV_QUEUE_STAT_CNTRS" check,
>     if "i >= RTE_ETHDEV_QUEUE_STAT_CNTRS" you shouldn't access to 'stats->q_.*[i]'
>