Re: [dpdk-dev] [PATCH v6 1/8] net/dpaa: add support for fmlib in dpdk

[Ferruh Yigit <ferruh.yigit@intel.com>]

On 9/2/2020 6:15 AM, Hemant Agrawal wrote:
> Hi Ferruh,
> 
>> -----Original Message-----
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> Sent: Tuesday, September 1, 2020 9:19 PM
>> To: Hemant Agrawal <hemant.agrawal@nxp.com>; dev@dpdk.org
>> Subject: Re: [PATCH v6 1/8] net/dpaa: add support for fmlib in dpdk
>>
>> On 9/1/2020 1:36 PM, Hemant Agrawal wrote:
>>> DPAA platorm MAC interface is known as FMAN i.e. Frame Manager.
>>> There are two ways to control it.
>>> 1. Statically configure the queues and classification rules before the
>>> start of the application using FMC tool.
>>> 2. Dynamically configure it within application by making API calls of
>>> fmlib.
>>>
>>> The fmlib or Frame Manager library provides an API on top of the Frame
>>> Manager driver ioctl calls, that provides a user space application
>>> with a simple way to configure driver parameters and PCD (parse -
>>> classify - distribute) rules.
>>>
>>> This patch integrates the base fmlib so that various queue config, RSS
>>> and classification related features can be supported on DPAA platform.
>>>
>>> Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
>>> Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
>>> ---
>>>  doc/guides/nics/dpaa.rst                  |   52 +-
>>>  doc/guides/platform/dpaa.rst              |   21 +-
>>>  drivers/net/dpaa/fmlib/dpaa_integration.h |   50 +
>>>  drivers/net/dpaa/fmlib/fm_ext.h           |  463 ++
>>>  drivers/net/dpaa/fmlib/fm_lib.c           |  561 ++
>>>  drivers/net/dpaa/fmlib/fm_pcd_ext.h       | 5787
>> +++++++++++++++++++++
>>>  drivers/net/dpaa/fmlib/fm_port_ext.h      | 3350 ++++++++++++
>>>  drivers/net/dpaa/fmlib/ncsw_ext.h         |  158 +
>>>  drivers/net/dpaa/fmlib/net_ext.h          |  411 ++
>>>  drivers/net/dpaa/meson.build              |    3 +-
>>>  10 files changed, 10849 insertions(+), 7 deletions(-)  create mode
>>> 100644 drivers/net/dpaa/fmlib/dpaa_integration.h
>>>  create mode 100644 drivers/net/dpaa/fmlib/fm_ext.h  create mode
>>> 100644 drivers/net/dpaa/fmlib/fm_lib.c  create mode 100644
>>> drivers/net/dpaa/fmlib/fm_pcd_ext.h
>>>  create mode 100644 drivers/net/dpaa/fmlib/fm_port_ext.h
>>>  create mode 100644 drivers/net/dpaa/fmlib/ncsw_ext.h  create mode
>>> 100644 drivers/net/dpaa/fmlib/net_ext.h
>>>
>>> diff --git a/doc/guides/nics/dpaa.rst b/doc/guides/nics/dpaa.rst index
>>> 17839a920..7e6010471 100644
>>> --- a/doc/guides/nics/dpaa.rst
>>> +++ b/doc/guides/nics/dpaa.rst
>>> @@ -1,5 +1,5 @@
>>>  ..  SPDX-License-Identifier: BSD-3-Clause
>>> -    Copyright 2017 NXP
>>> +    Copyright 2017,2020 NXP
>>>
>>>
>>>  DPAA Poll Mode Driver
>>> @@ -21,6 +21,7 @@ Contents summary
>>>
>>>  - DPAA overview
>>>  - DPAA driver architecture overview
>>> +- FMAN configuration tools and library
>>>
>>>  .. _dpaa_overview:
>>>
>>> @@ -285,6 +286,55 @@ for details.
>>>        Done
>>>        testpmd>
>>>
>>> +FMAN Config
>>> +-----------
>>> +
>>> +Frame Manager is also responsible for parser, classify and distribute
>>> +functionality in the DPAA.
>>> +
>>> +   FMAN supports:
>>> +   Packet parsing at wire speed. It supports standard protocols parsing and
>>> +   identification by HW
>> (VLAN/IP/UDP/TCP/SCTP/PPPoE/PPP/MPLS/GRE/IPSec).
>>> +   It supports non-standard UDF header parsing for custom protocols.
>>> +   Classification / Distribution: Coarse classification based on Key
>> generation
>>> +   Hash and exact match lookup
>>> +
>>> +FMC - FMAN Configuration Tool
>>> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>> +   This tool is available in User Space. The tool is used to configure FMAN
>>> +   Physical (MAC) or Ephemeral (OH)ports for Parse/Classify/distribute.
>>> +   The PCDs can be hash based where a set of fields are key input for hash
>>> +   generation within FMAN keygen. The hash value is used to generate a
>> FQID for
>>> +   frame. There is a provision to setup exact match lookup too where field
>>> +   values within a packet drives corresponding FQID.
>>> +   Currently it works on XML file inputs.
>>> +
>>> +   Limitations:
>>> +   1.For Dynamic Configuration change, currently no support is available.
>>> +   E.g. enable/disable a port, a operator (set of VLANs and associate rules).
>>> +
>>> +   2.During FMC configuration, port for which policy is being configured is
>>> +   brought down and the policy is flushed on port before new policy is
>> updated
>>> +   for the port. Support is required to add/append/delete etc.
>>> +
>>> +   3.FMC, being a separate user-space application, needs to be invoked
>> from
>>> +   Shell.
>>> +
>>> +
>>> +   The details can be found in FMC Doc at:
>>> +   `Frame Mnager Configuration Tool
>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fww
>> w.nxp.com%2Fdocs%2Fen%2Fapplication-
>> note%2FAN4760.pdf&amp;data=02%7C01%7Chemant.agrawal%40nxp.com%
>> 7Cdb7eda614491479976c508d84e8e8c42%7C686ea1d3bc2b4c6fa92cd99c5c30
>> 1635%7C0%7C1%7C637345721434145756&amp;sdata=ynJeLWqw0hPTya5IRt
>> GviovNA7mAGIZ8k3XSFcZyLP4%3D&amp;reserved=0>`_.
>>> +
>>> +FMLIB
>>> +~~~~~
>>> +   The Frame Manager library provides an API on top of the Frame
>> Manager driver
>>> +   ioctl calls, that provides a user space application with a simple way to
>>> +   configure driver parameters and PCD (parse - classify - distribute) rules.
>>> +
>>> +   This is an alternate to the FMC based configuration. This libray provides
>>> +   direct ioctl based interfaces for FMAN configuration as used by the FMC
>> tool
>>> +   as well. This helps in overcoming the main limitaiton of FMC - i.e. lack
>>> +   of dynamic configuration.
>>> +
>>>  Limitations
>>>  -----------
>>>
>>> diff --git a/doc/guides/platform/dpaa.rst
>>> b/doc/guides/platform/dpaa.rst index 6005f2221..20a0e3932 100644
>>> --- a/doc/guides/platform/dpaa.rst
>>> +++ b/doc/guides/platform/dpaa.rst
>>> @@ -58,17 +58,28 @@ compatible board:
>>>
>>>  4. **FMC Tool**
>>>
>>> -   Before any DPDK application can be executed, the Frame Manager
>> Configuration
>>> -   Tool (FMC) need to be executed to set the configurations of the queues.
>> This
>>> +   If one is planning to use more than 1 Recv queue and hardware
>> capability to
>>> +   parse, classify and distribute the packets, the Frame Manager
>> Configuration
>>> +   Tool (FMC) need to be executed to set the configurations of the queues
>> before
>>> +   running the DPAA based DPDK application. This setting is persistent, the
>>> +   configuration will remain in the hardware till it is
>>> + re-configured. This
>>>     includes the queue state, RSS and other policies.
>>>     This tool can be obtained from `NXP (Freescale) Public Git Repository
>> <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsour
>> ce.codeaurora.org%2Fexternal%2Fqoriq%2Fqoriq-
>> components%2Ffmc&amp;data=02%7C01%7Chemant.agrawal%40nxp.com%
>> 7Cdb7eda614491479976c508d84e8e8c42%7C686ea1d3bc2b4c6fa92cd99c5c30
>> 1635%7C0%7C1%7C637345721434145756&amp;sdata=pwOJGxY%2BGTkcpdk
>> O%2BKwoVejywmOeGsWbudo9OPSDqPU%3D&amp;reserved=0>`_.
>>>
>>>     This tool needs configuration files which are available in the
>>>     :ref:`DPDK Extra Scripts <extra_scripts>`, described below for DPDK
>> usages.
>>>
>>> -As an alternative method, DPAA PMD can also be executed using images
>>> provided -as part of SDK from NXP. The SDK includes all the above
>>> prerequisites necessary -to bring up a DPAA board.
>>> +   Note that DPAA PMD can also be executed using images provided
>>> +   as part of SDK from NXP. The SDK includes all the above prerequisites
>>> +   necessary (i.e. fmc tool) to bring up a DPAA board.
>>> +
>>> +   As an alternate method, DPAA PMDs starting from DPDK 20.11 also
>> support the
>>> +   fmlib library integration. The driver will detect about any existing FMC
>>> +   based config (if /tmp/fmc.bin is present). DPAA FMD will be used only if
>> no
>>> +   previous fmc config is existing.
>>> +
>>> +   Note that fmlib based integratin rely on underlying fmd driver in kernel,
>>> +   which is available as part of NXP kernel or NXP SDK.
>>
>> Hi Hemant,
>>
>> Thanks for the documentation.
>>
>> Can it be possible to give a little more detail related to the 'fmd' module, like
>> its folder etc..
>>
>> Another thing is with current patch 'dpaa' driver becomes completely
>> dependent to the 'fmd' kernel module [1] and PMD won't run without it, was
>> the dependency always there indirectly?
> 
> [Hemant]  I thought, I was clear in my documentation. 
> " This libray provides direct ioctl based interfaces for FMAN configuration as used by the FMC tool  as well"
> FMD always exist in the kernel for DPAA.  At present FMC is using it via IOCTL calls and FMC is indepdent tools with limitations for DPDK.
> Now, we have directly used those ioctl calls via fmlib in DPDK. 
> 
>> And does it make sense to support no 'fmd' kernel module case, as done
>> previously?
> [Hemant] FMD is already part of DPAA supported kernels.


OK, so what I understand is 'fmd' module was already an indirect dependency and
there is no point to support 'dpaa' for no 'fmd' available case.

Can you just give the location of the 'fmd' module, in case someone would like
to check the corresponding code for an ioctl call, it may help.


>>
>>
>> [1]
>> 'dpaa_fm_init()' will fail if the kernel driver is missing, which will cause PMD
>> driver probe, 'rte_dpaa_probe()' to fail.