Re: [dpdk-dev] [PATCH v7 11/21] eal/soc: implement probing of drivers

[Jianbo Liu <jianbo.liu@linaro.org>]

On 10 November 2016 at 14:10, Shreyansh Jain <shreyansh.jain@nxp.com> wrote:
> On Thursday 10 November 2016 09:00 AM, Jianbo Liu wrote:
>>
>> On 28 October 2016 at 20:26, Shreyansh Jain <shreyansh.jain@nxp.com>
>> wrote:
>>>
>>> Each SoC PMD registers a set of callback for scanning its own bus/infra
>>> and
>>> matching devices to drivers when probe is called.
>>> This patch introduces the infra for calls to SoC scan on
>>> rte_eal_soc_init()
>>> and match on rte_eal_soc_probe().
>>>
>>> Patch also adds test case for scan and probe.
>>>
>>> Signed-off-by: Jan Viktorin <viktorin@rehivetech.com>
>>> Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
>>> Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
>>> --
>>> v4:
>>>  - Update test_soc for descriptive test function names
>>>  - Comments over test functions
>>>  - devinit and devuninint --> probe/remove
>>>  - RTE_VERIFY at some places
>>> ---
>>>  app/test/test_soc.c                             | 205
>>> ++++++++++++++++++++++-
>>>  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |   4 +
>>>  lib/librte_eal/common/eal_common_soc.c          | 213
>>> +++++++++++++++++++++++-
>>>  lib/librte_eal/common/include/rte_soc.h         |  75 ++++++++-
>>>  lib/librte_eal/linuxapp/eal/eal.c               |   5 +
>>>  lib/librte_eal/linuxapp/eal/eal_soc.c           |  21 ++-
>>>  lib/librte_eal/linuxapp/eal/rte_eal_version.map |   4 +
>>>  7 files changed, 519 insertions(+), 8 deletions(-)
>>>
>> ......
>>
>>>  /**
>>> + * SoC device scan callback, called from rte_eal_soc_init.
>>> + * For various SoC, the bus on which devices are attached maynot be
>>> compliant
>>> + * to a standard platform (or platform bus itself). In which case, extra
>>> + * steps are implemented by PMD to scan over the bus and add devices to
>>> SoC
>>> + * device list.
>>> + */
>>> +typedef void (soc_scan_t)(void);
>>
>>
>> I'm still not sure about the purpose of soc_scan, and how to use it.
>
>
> For each device to be used by DPDK, which cannot be scanned/identified using
> the existing PCI/VDEV methods (sysfs/bus/pci), 'soc_scan_t' provides a way
> for driver to make those devices part of device lists.
>
> Ideally, 'scan' is not a function of a driver. It is a bus function - which
> is missing in this case.
>
>> If it's for each driver, it should at least struct rte_soc_driver * as
>> its parameter.
>
>
> Its for each driver - assuming that each non-PCI driver which implements it
> knows how to find devices which it can control (for example, special area in
> sysfs, or even platform bus).
>

Considering there are several drivers in a platform bus, each driver
call the scan function, like the rte_eal_soc_scan_platform_bus() you
implemented.
The first will add soc devices to the list, but the remaining calls
are redundant.

The other issue is adding the driver parameter. Do you need extra
information from driver to scan the bus?

>> If it's for each bus, why it is in rte_soc_driver?
>
>
> Short answer - lack of a better place. It should be in dev.h probably
> (rte_device/driver) but it would look out of place (as that represents PCI
> devices also which cannot implement it - all PCI devices are scanned in one
> go irrespective of driver)
>
>> I know you will implement bus driver in the future, but we need to
>> make it clear for current simplified implementation.
>
>
> Current implementation makes only a single assumption - that rather than
> relying on EAL for identifying devices (as being done now), next best option
> in existing framework (driver) should have control of finding devices.
>
> This is primarily to make the SoC work parallel to PCI implementation
> without much top-down changes in EAL.
>
> Bus model, improvises it by moving this implementation a little above in
> hierarchy - in rte_bus<-rte_driver<-PMD.
>
> I understand your apprehension - 'driver-scanning-for-devices' is indeed not
> correct real world analogy. It is just a place holder for enabling those
> drivers/PMDs which cannot work in absence of the right model.
> And that is still work in progress.
>
>
>>
>>> +
>>> +/**
>>> + * Custom device<=>driver match callback for SoC
>>> + * Unlike PCI, SoC devices don't have a fixed definition of device
>>> + * identification. PMDs can implement a specific matching function in
>>> which
>>> + * driver and device objects are provided to perform custom match.
>>> + */
>>> +typedef int (soc_match_t)(struct rte_soc_driver *, struct rte_soc_device
>>> *);
>>> +
>>> +/**
>>>   * A structure describing a SoC driver.
>>>   */
>>>  struct rte_soc_driver {
>>> @@ -104,6 +120,8 @@ struct rte_soc_driver {
>>>         struct rte_driver driver;          /**< Inherit core driver. */
>>>         soc_probe_t *probe;                /**< Device probe */
>>>         soc_remove_t *remove;              /**< Device remove */
>>> +       soc_scan_t *scan_fn;               /**< Callback for scanning SoC
>>> bus*/
>>> +       soc_match_t *match_fn;             /**< Callback to match
>>> dev<->drv */
>>>         const struct rte_soc_id *id_table; /**< ID table, NULL terminated
>>> */
>>>  };
>>>
>>> @@ -146,12 +164,63 @@ rte_eal_compare_soc_addr(const struct rte_soc_addr
>>> *a0,
>>>  }
>>>
>>>  /**
>>> + * Default function for matching the Soc driver with device. Each driver
>>> can
>>> + * either use this function or define their own soc matching function.
>>> + * This function relies on the compatible string extracted from sysfs.
>>> But,
>>> + * a SoC might have different way of identifying its devices. Such SoC
>>> can
>>> + * override match_fn.
>>> + *
>>> + * @return
>>> + *      0 on success
>>> + *     -1 when no match found
>>> +  */
>>> +int
>>> +rte_eal_soc_match_compat(struct rte_soc_driver *drv,
>>> +                        struct rte_soc_device *dev);
>>> +
>>> +/**
>>> + * Probe SoC devices for registered drivers.
>>> + *
>>> + * @return
>>> + *     0 on success
>>> + *     !0 in case of any failure in probe
>>> + */
>>> +int rte_eal_soc_probe(void);
>>> +
>>> +/**
>>> + * Probe the single SoC device.
>>> + */
>>> +int rte_eal_soc_probe_one(const struct rte_soc_addr *addr);
>>> +
>>> +/**
>>> + * Close the single SoC device.
>>> + *
>>> + * Scan the SoC devices and find the SoC device specified by the SoC
>>> + * address, then call the remove() function for registered driver
>>> + * that has a matching entry in its id_table for discovered device.
>>> + *
>>> + * @param addr
>>> + *     The SoC address to close.
>>> + * @return
>>> + *   - 0 on success.
>>> + *   - Negative on error.
>>> + */
>>> +int rte_eal_soc_detach(const struct rte_soc_addr *addr);
>>> +
>>> +/**
>>>   * Dump discovered SoC devices.
>>> + *
>>> + * @param f
>>> + *     File to dump device info in.
>>>   */
>>>  void rte_eal_soc_dump(FILE *f);
>>>
>>>  /**
>>>   * Register a SoC driver.
>>> + *
>>> + * @param driver
>>> + *     Object for SoC driver to register
>>> + * @return void
>>>   */
>>>  void rte_eal_soc_register(struct rte_soc_driver *driver);
>>>
>>> @@ -167,6 +236,10 @@ RTE_PMD_EXPORT_NAME(nm, __COUNTER__)
>>>
>>>  /**
>>>   * Unregister a SoC driver.
>>> + *
>>> + * @param driver
>>> + *     Object for SoC driver to unregister
>>> + * @return void
>>>   */
>>>  void rte_eal_soc_unregister(struct rte_soc_driver *driver);
>>>
>>> diff --git a/lib/librte_eal/linuxapp/eal/eal.c
>>> b/lib/librte_eal/linuxapp/eal/eal.c
>>> index 098ba02..bd775f3 100644
>>> --- a/lib/librte_eal/linuxapp/eal/eal.c
>>> +++ b/lib/librte_eal/linuxapp/eal/eal.c
>>> @@ -70,6 +70,7 @@
>>>  #include <rte_cpuflags.h>
>>>  #include <rte_interrupts.h>
>>>  #include <rte_pci.h>
>>> +#include <rte_soc.h>
>>>  #include <rte_dev.h>
>>>  #include <rte_devargs.h>
>>>  #include <rte_common.h>
>>> @@ -890,6 +891,10 @@ rte_eal_init(int argc, char **argv)
>>>         if (rte_eal_pci_probe())
>>>                 rte_panic("Cannot probe PCI\n");
>>>
>>> +       /* Probe & Initialize SoC devices */
>>> +       if (rte_eal_soc_probe())
>>> +               rte_panic("Cannot probe SoC\n");
>>> +
>>>         rte_eal_mcfg_complete();
>>>
>>>         return fctret;
>>> diff --git a/lib/librte_eal/linuxapp/eal/eal_soc.c
>>> b/lib/librte_eal/linuxapp/eal/eal_soc.c
>>> index 04848b9..3929a76 100644
>>> --- a/lib/librte_eal/linuxapp/eal/eal_soc.c
>>> +++ b/lib/librte_eal/linuxapp/eal/eal_soc.c
>>> @@ -44,13 +44,28 @@
>>>  #include <rte_log.h>
>>>  #include <rte_soc.h>
>>>
>>> -#include "eal_internal_cfg.h"
>>> -#include "eal_filesystem.h"
>>> -#include "eal_private.h"
>>> +#include <eal_internal_cfg.h>
>>> +#include <eal_filesystem.h>
>>> +#include <eal_private.h>
>>>
>>>  /* Init the SoC EAL subsystem */
>>>  int
>>>  rte_eal_soc_init(void)
>>>  {
>>> +       struct rte_soc_driver *drv;
>>> +
>>> +       /* SoC is disabled by default */
>>> +       if (!internal_config.enable_soc)
>>> +               return 0;
>>> +
>>> +       /* For each registered driver, call their scan routine to perform
>>> any
>>> +        * custom scan for devices (for example, custom buses)
>>> +        */
>>> +       TAILQ_FOREACH(drv, &soc_driver_list, next) {
>>> +               RTE_VERIFY(drv->scan_fn);
>>> +               drv->scan_fn();
>>> +               /* Ignore all errors from this */
>>> +       }
>>> +
>>>         return 0;
>>>  }
>>> diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>>> b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>>> index 0155025..c28e093 100644
>>> --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>>> +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>>> @@ -175,7 +175,11 @@ DPDK_16.11 {
>>>         rte_eal_dev_attach;
>>>         rte_eal_dev_detach;
>>>         rte_eal_map_resource;
>>> +       rte_eal_soc_detach;
>>>         rte_eal_soc_dump;
>>> +       rte_eal_soc_match;
>>> +       rte_eal_soc_probe;
>>> +       rte_eal_soc_probe_one;
>>>         rte_eal_soc_register;
>>>         rte_eal_soc_unregister;
>>>         rte_eal_unmap_resource;
>>> --
>>> 2.7.4
>>>
>>
>
>
> --
> -
> Shreyansh