Re: [dpdk-dev] [PATCH v1 10/11] doc: add IPN3KE document

["Xu, Rosen" <rosen.xu@intel.com>]

> -----Original Message-----
> From: Yigit, Ferruh
> Sent: Wednesday, March 06, 2019 20:45
> To: Xu, Rosen <rosen.xu@intel.com>; dev@dpdk.org
> Cc: Zhang, Tianfei <tianfei.zhang@intel.com>; Wei, Dan
> <dan.wei@intel.com>; Pei, Andy <andy.pei@intel.com>; Yang, Qiming
> <qiming.yang@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>; Chen,
> Santos <santos.chen@intel.com>; Zhang, Zhang <zhang.zhang@intel.com>
> Subject: Re: [PATCH v1 10/11] doc: add IPN3KE document
> 
> On 2/28/2019 7:13 AM, Rosen Xu wrote:
> > Add Intel FPGA Acceleration NIC IPN3KE document.
> 
> No need to separate the doc patch, can merge into the one that adds the
> driver.

Okay, I will apply it in patch v2.
 
> >
> > Signed-off-by: Rosen Xu <rosen.xu@intel.com>
> > Signed-off-by: Dan Wei <dan.wei@intel.com>
> > ---
> >  doc/guides/nics/features/ipn3ke.ini | 57 ++++++++++++++++++++++
> >  doc/guides/nics/index.rst           |  1 +
> >  doc/guides/nics/ipn3ke.rst          | 97
> +++++++++++++++++++++++++++++++++++++
> >  3 files changed, 155 insertions(+)
> >  create mode 100644 doc/guides/nics/features/ipn3ke.ini
> >  create mode 100644 doc/guides/nics/ipn3ke.rst
> >
> > diff --git a/doc/guides/nics/features/ipn3ke.ini
> > b/doc/guides/nics/features/ipn3ke.ini
> > new file mode 100644
> > index 0000000..06cfaf5
> > --- /dev/null
> > +++ b/doc/guides/nics/features/ipn3ke.ini
> > @@ -0,0 +1,57 @@
> > +;
> > +; Supported features of the 'ipn3ke' network poll mode driver.
> > +;
> > +; Refer to default.ini for the full list of available PMD features.
> > +;
> > +[Features]
> > +Speed capabilities   = Y
> > +Link status          = Y
> > +Link status event    = Y
> > +Rx interrupt         = Y
> > +Queue start/stop     = Y
> > +Runtime Rx queue setup = Y
> > +Runtime Tx queue setup = Y
> > +Jumbo frame          = Y
> > +Scattered Rx         = Y
> > +TSO                  = Y
> > +Promiscuous mode     = Y
> > +Allmulticast mode    = Y
> > +Unicast MAC filter   = Y
> > +Multicast MAC filter = Y
> > +RSS hash             = Y
> > +RSS key update       = Y
> > +RSS reta update      = Y
> > +VMDq                 = Y
> > +SR-IOV               = Y
> > +DCB                  = Y
> > +VLAN filter          = Y
> > +Ethertype filter     = Y
> > +Tunnel filter        = Y
> > +Hash filter          = Y
> > +Flow director        = Y
> > +Flow control         = Y
> > +Flow API             = Y
> > +Traffic mirroring    = Y
> > +CRC offload          = Y
> > +VLAN offload         = Y
> > +QinQ offload         = Y
> > +L3 checksum offload  = Y
> > +L4 checksum offload  = Y
> > +Inner L3 checksum    = Y
> > +Inner L4 checksum    = Y
> > +Packet type parsing  = Y
> > +Timesync             = Y
> > +Rx descriptor status = Y
> > +Tx descriptor status = Y
> > +Basic stats          = Y
> > +Extended stats       = Y
> > +FW version           = Y
> > +Module EEPROM dump   = Y
> > +Multiprocess aware   = Y
> > +BSD nic_uio          = Y
> > +Linux UIO            = Y
> > +Linux VFIO           = Y
> > +x86-32               = Y
> > +x86-64               = Y
> > +ARMv8                = Y
> > +Power8               = Y
> 
> 
> Are these all marked to 'Y' because the underlying PMD is i40e?

Yes, because there are 2xFVL in Vista Creek, and currently all the CPU
packets TX/RX are handled by FVL.

> Is ipn3ke really verified with Power8, arm etc?

Just copy from FVL, because FVL has this feature and there are 2xFVL in
Vista Creek.

> > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
> > index 5c80e3b..6671481 100644
> > --- a/doc/guides/nics/index.rst
> > +++ b/doc/guides/nics/index.rst
> > @@ -28,6 +28,7 @@ Network Interface Controller Drivers
> >      fm10k
> >      i40e
> >      ice
> > +    ipn3ke
> 
> Please add alphabetically sorted.

Okay
 
> >      ifc
> >      igb
> >      ixgbe
> > diff --git a/doc/guides/nics/ipn3ke.rst b/doc/guides/nics/ipn3ke.rst
> > new file mode 100644 index 0000000..7386075
> > --- /dev/null
> > +++ b/doc/guides/nics/ipn3ke.rst
> > @@ -0,0 +1,97 @@
> > +..  SPDX-License-Identifier: BSD-3-Clause
> > +    Copyright(c) 2019 Intel Corporation.
> > +
> > +IPN3KE Poll Mode Driver
> > +=======================
> > +
> > +The ipn3ke PMD (librte_pmd_ipn3ke) provides poll mode driver support
> > +for Intel® FPGA PAC(Programmable Acceleration Card) N3000 based on
> > +the Intel Ethernet Controller X710/XXV710 and Intel Arria 10 FPGA.
> > +
> > +In this card, FPGA is an acceleration bridge between network
> > +interface and the Intel Ethernet Controller. Although both FPGA and
> > +Ethernet Controllers are connected to CPU with PCIe Gen3x16 Switch,
> > +all the packet RX/TX is handled by Intel Ethernet Controller. So from
> > +application point of view the data path is still the legacy Intel
> > +Ethernet Controller
> > +X710/XXV710 PMD. Besides this, users can enable more acceleration
> > +features by FPGA IP.
> > +
> > +Prerequisites
> > +-------------
> > +
> > +- Identifying your adapter using `Intel Support
> > +  <http://www.intel.com/support>`_ and get the latest NVM/FW images.
> > +
> > +- Follow the DPDK :ref:`Getting Started Guide for Linux <linux_gsg>` to
> setup the basic DPDK environment.
> > +
> > +- To get better performance on Intel platforms, please follow the "How to
> get best performance with NICs on Intel platforms"
> > +  section of the :ref:`Getting Started Guide for Linux <linux_gsg>`.
> > +
> > +
> > +Pre-Installation Configuration
> > +------------------------------
> > +
> > +Config File Options
> > +~~~~~~~~~~~~~~~~~~~
> > +
> > +The following options can be modified in the ``config`` file.
> > +Please note that enabling debugging options may affect system
> performance.
> > +
> > +- ``CONFIG_RTE_LIBRTE_IPN3KE_PMD`` (default ``n``)
> > +
> > +  Toggle compilation of the ``librte_pmd_ipn3ke`` driver.
> > +
> > +- ``CONFIG_RTE_LIBRTE_IPN3KE_DEBUG_*`` (default ``n``)
> 
> The name is wrong, there is no DEBUG_*, also it seems not used, if so please
> remove.

Fix in patch v2.

> > +
> > +  Toggle display of generic debugging messages.
> > +
> > +Runtime Config Options
> > +~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +- ``Maximum Number of Queue Pairs``
> > +
> > +  The maximum number of queue pairs is decided by HW. If not
> > + configured, APP  uses the number from HW. Users can check the number
> > + by calling the API  ``rte_eth_dev_info_get``.
> > +  If users want to limit the number of queues, they can set a smaller
> > + number  using EAL parameter like ``max_queue_pair_num=n``.
> 
> Is this copy/paste error?
> There is no 'max_queue_pair_num' device argument in this driver.
> 
> And there are other devargs, seems not documented here.

Fix in patch v2.

> > +
> > +
> > +Driver compilation and testing
> > +------------------------------
> > +
> > +Refer to the document :ref:`compiling and testing a PMD for a NIC
> > +<pmd_build_and_test>` for details.
> > +
> > +Sample Application Notes
> > +------------------------
> > +
> > +Packet TX/RX with FPGA Pass-through image
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +FPGA Pass-through bitstream is original FPGA Image.
> > +
> > +To start ``testpmd``, and add I40e PF to FPGA network port:
> > +
> > +.. code-block:: console
> > +
> > +    ./app/testpmd -l 0-15 -n 4 --vdev
> > + 'ifpga_rawdev_cfg0,ifpga=b3:00.0,port=0' --vdev
This is IFPGA BUS AFU configuration, same as before.
> > + 'ipn3ke_cfg0,afu=0|b3:00.0,i40e_pf={0000:b1:00.0|0000:b1:00.1|0000:b
> > + 1:00.2|0000:b1:00.3|0000:b5:00.0|0000:b5:00.1|0000:b5:00.2|0000:b5:0
In IPN3KE, there are one2one port mapping relationship between network side(port representor)
and FVL PF, this example is to create mapping of 8 ports.

> > + 0.3}' -- -i --no-numa --port-topology=loop
> 
> Can you please help me understand?

Pls see my reply.

> raw device ifpga_rawdev_cfg0, used to get devargs and to create the AFU
> device.
Yes 
> AFU driver is already in this patch, net_ipn3ke_afu. It should be probed after
> rawdev created the device.
Yes
> vdev ipn3ke_cfg0, used to get devargs and link afu device and i40e devices,
> so i40e devices should be already probed in this stage, and port representors
> created for AFU device.
Yes
> Is above correct?
Yes
> Overall there is a IFPGA bus and AFU device work going on, but they are
> behind rawdev and vdev.
Yes
> > +
> > +HQoS and flow acceleration
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +HQoS and flow acceleration bitstream is used to offloading HQoS and flow
> classifier.
> > +
> > +To start ``testpmd``, and add I40e PF to FPGA network port, enable FPGA
> HQoS and Flow Acceleration:
> > +
> > +.. code-block:: console
> > +
> > +    ./app/testpmd -l 0-15 -n 4 --vdev
> > + 'ifpga_rawdev_cfg0,ifpga=b3:00.0,port=0' --vdev
> > + 'ipn3ke_cfg0,afu=0|b3:00.0,fpga_acc={tm|flow},i40e_pf={0000:b1:00.0|
> > + 0000:b1:00.1|0000:b1:00.2|0000:b1:00.3|0000:b5:00.0|0000:b5:00.1|000
> > + 0:b5:00.2|0000:b5:00.3}' -- -i --no-numa --forward-mode=macswap
> > +
> > +Limitations or Known issues
> > +---------------------------
> > +
> > +19.05 limitation
> > +~~~~~~~~~~~~~~~~
> > +
> > +Ipn3ke code released in 19.05 is for evaluation only.
> >