Re: [dpdk-dev] [PATCH v11 00/23] Add DLB2 PMD

[Jerin Jacob <jerinjacobk@gmail.com>]

On Mon, Nov 2, 2020 at 5:07 AM Timothy McDaniel
<timothy.mcdaniel@intel.com> wrote:
>
> The following patch series adds support for a new eventdev PMD. The DLB2
> PMD adds support for the Intel Dynamic Load Balancer 2.0 (DLB2)
> hardware.
> The DLB2 is a PCIe device that provides load-balanced, prioritized
> scheduling of core-to-core communication. The device consists of
> queues and arbiters that connect producer and consumer cores, and
> implements load-balanced queueing features including:
> - Lock-free multi-producer/multi-consumer operation.
> - Multiple priority levels for varying traffic types.
> - 'Direct' traffic (i.e. multi-producer/single-consumer)
> - Simple unordered load-balanced distribution.
> - Atomic lock-free load balancing across multiple consumers.
> - Queue element reordering feature allowing ordered load-balanced
>   distribution.
>
> The DLB2 hardware supports both load balanced and directed ports and
> queues. Unlike other eventdev devices already in the repo,  not all
> DLB2 ports and queues are equally capable. In particular, directed
> ports are limited to a single link, and must be connected to a
> directed queue. Additionally, even though LDB ports may link multiple
> queues,
> the number of queues that may be linked is limited by hardware.
>
> While reviewing the code, please be aware that this PMD has full
> control over the DLB2 hardware. Intel will be extending the DLB2 PMD
> in the future (not as part of this first series) with a mode that we
> refer to as the bifurcated PMD. The bifurcated PMD communicates with a
> kernel driver to configure the device, ports, and queues, and memory
> maps device MMIO so datapath operations occur purely in user-space.
> Note that the DLB2 hardware is a successor of the DLB hardware, and
> as such is structured similarly, both in terms of code layout and
> implementation.
>
> The framework to support both the PF PMD and bifurcated PMD exists in
> this patchset, and is why the iface.[ch] layer is present.


Finally, Series applied to dpdk-next-eventdev/for-main with the
following fix[1]. Thanks.

Thanks, @David Marchand for your final review cycles.


[1]

 [for-main]dell[dpdk-next-eventdev] $ ./devtools/check-git-log.sh -n 23
Wrong tag:
        Reviewed-by: Chen, Mike Ximing <mike.ximing.chen@intel.com>



>
> Changes since V10
> =================
> - converted printfs in dlb2/pf/dlb2_main.c to DLB2_LOG
> - fixed a repeated word error in dlb2/pf/base/osdep_bitmap.h
> - caught up with marking the patches that Gage reviewed
>
> Changes since V9
> ================
> Address comments from David Marchand:
> - this patch-set is based on Nov 1, 2020 dpdk-next-eventdev
> - fix docs build (doxy-api.conf.in and doxy-api-index.md)
> - restore blank line in MAINTAINERS file
> - move dlb announcement in release_20_11.rst after ethdev
> - use headers = files() for exported meson public headers
> - fix a typo in 'add documentation ..." commit message
> - use eal version of cldemote
> - convert a couple of printfs to LOG messages
> - all patches build incrementally (gcc), and checkpatches reports
>   success
> - I am not able to run clang locally. If clang errors are still
>   present I will ask IT to install clang on a build server tomorrow.
>
> Changes since V8:
> =================
> - Fixed format errors in doc/api/doxy-api-index.md
> - Delayed introduction of dlb2_consume_qe_immediate until
>   add-dequeue-and-its-burst-variants.patch
> - Delayed introduction of dlb2_construct_token_pop_qe until
>   add-PMD-s-token-pop-public-interface.patch
>
> Changes since V7:
> ================
> - fix CENTOS build error: use __m128i instead of __v2di with
>   _mm_stream_si128
> - deleted unused dlb2_umwait and dlb2_umonitor functions
>
> Changes since V6:
> =================
> - removed unused function, fixing build error
> - fixed typo in port_setup commit message
> - this patch series is based on dpdk-next-eventdev
>
> Changes since V5:
> ================
> - convert to use rte_power_monitor patches
> - replace __builtin_ia32_movntdq() with _mm_stream_si128()
> - remove unused functions in dlb_selftest.c
> - workaround for RHEL 7 gcc brace bug
>
> Changes since V4:
> ================
> - moved introduction of dlb in relnotes_20_11 to first patch in series
> - fixed underlines in dlb.rst that were too short
> - note that the code still uses its private byte-encoded versions of
>   umonitor/umwait, rather than the new functions in the power
>   patch that are built on top of those intrinsics. This is intentional.
>
> Changes since V3:
> ================
> - updated MAINTAINERS file to alphabetically insert DLB
> - don't create RTE_ symbols in pmd
> - converted to use version.map scheme
> - converted to use .._master_lcore instead of .._main_lcore
> - this patch set is based on dpdk-next-eventdev
>
> Changes since V2:
> ================
> - fixed meson conditional build. Moved test into driver’s meson.build
>   file instead of event/meson.build
> - documentation is populated as associated code is introduced
> - add log_register in add dynamic logging patch
> - rename RTE_xxx symbol(s) as DLB2_xxx
> - replaced function ptr enqueue_four with direct call to movdir64b
> - remove unused port_pages
> - broke up probe patch into 3 smaller patches for easier review
> - changed param order of movdir64b/movntdq to match intrinsics
> - added self to MAINTAINERS files
> - squashed announcement of availability into last patch in series
> - correct spelling errors and delete repeated words
> - DPDK_21.0 -> DPDK 21 in map file
> - add experimental banner to public structs and APIs
>
> Changes since V1:
> =================
> - implement changes requested in code reviews by Gage Eads and Mike
>   Chen
> - fix a memzone leak
> - convert to use eal rte-cpuflags patch from Liang Ma
>
> Known Issues:
> - the documentation contained in dlb2.rst is not complete, and
>   should be updated to include command line parameters (class of service,
>   etc ...).
>
> Depends-on: patch-82202 ("eventdev: increase MAX QUEUES PER DEV to 255")
>
> Timothy McDaniel (23):
>   event/dlb2: add documentation and meson build infrastructure
>   event/dlb2: add dynamic logging
>   event/dlb2: add private data structures and constants
>   event/dlb2: add definitions shared with LKM or shared code
>   event/dlb2: add inline functions
>   event/dlb2: add eventdev probe
>   event/dlb2: add flexible interface
>   event/dlb2: add probe-time hardware init
>   event/dlb2: add xstats
>   event/dlb2: add infos get and configure
>   event/dlb2: add queue and port default conf
>   event/dlb2: add queue setup
>   event/dlb2: add port setup
>   event/dlb2: add port link
>   event/dlb2: add port unlink and port unlinks in progress
>   event/dlb2: add eventdev start
>   event/dlb2: add enqueue and its burst variants
>   event/dlb2: add dequeue and its burst variants
>   event/dlb2: add eventdev stop and close
>   event/dlb2: add PMD's token pop public interface
>   event/dlb2: add PMD self-tests
>   event/dlb2: add queue and port release
>   event/dlb2: add timeout ticks entry point
>
>  MAINTAINERS                                    |    5 +
>  app/test/test_eventdev.c                       |    7 +
>  config/rte_config.h                            |    7 +
>  doc/api/doxy-api-index.md                      |    3 +-
>  doc/api/doxy-api.conf.in                       |    1 +
>  doc/guides/eventdevs/dlb2.rst                  |  365 ++
>  doc/guides/eventdevs/index.rst                 |    1 +
>  doc/guides/rel_notes/release_20_11.rst         |    5 +
>  drivers/event/dlb2/dlb2.c                      | 3947 ++++++++++++++++
>  drivers/event/dlb2/dlb2_iface.c                |   74 +
>  drivers/event/dlb2/dlb2_iface.h                |   74 +
>  drivers/event/dlb2/dlb2_inline_fns.h           |   33 +
>  drivers/event/dlb2/dlb2_log.h                  |   25 +
>  drivers/event/dlb2/dlb2_priv.h                 |  578 +++
>  drivers/event/dlb2/dlb2_selftest.c             | 1524 ++++++
>  drivers/event/dlb2/dlb2_user.h                 |  679 +++
>  drivers/event/dlb2/dlb2_xstats.c               | 1235 +++++
>  drivers/event/dlb2/meson.build                 |   23 +
>  drivers/event/dlb2/pf/base/dlb2_hw_types.h     |  367 ++
>  drivers/event/dlb2/pf/base/dlb2_mbox.h         |  596 +++
>  drivers/event/dlb2/pf/base/dlb2_osdep.h        |  230 +
>  drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h |  440 ++
>  drivers/event/dlb2/pf/base/dlb2_osdep_list.h   |  131 +
>  drivers/event/dlb2/pf/base/dlb2_osdep_types.h  |   31 +
>  drivers/event/dlb2/pf/base/dlb2_regs.h         | 2527 ++++++++++
>  drivers/event/dlb2/pf/base/dlb2_resource.c     | 6027 ++++++++++++++++++++++++
>  drivers/event/dlb2/pf/base/dlb2_resource.h     | 1913 ++++++++
>  drivers/event/dlb2/pf/dlb2_main.c              |  673 +++
>  drivers/event/dlb2/pf/dlb2_main.h              |   97 +
>  drivers/event/dlb2/pf/dlb2_pf.c                |  728 +++
>  drivers/event/dlb2/rte_pmd_dlb2.c              |   39 +
>  drivers/event/dlb2/rte_pmd_dlb2.h              |   72 +
>  drivers/event/dlb2/version.map                 |    9 +
>  drivers/event/meson.build                      |    3 +-
>  34 files changed, 22467 insertions(+), 2 deletions(-)
>  create mode 100644 doc/guides/eventdevs/dlb2.rst
>  create mode 100644 drivers/event/dlb2/dlb2.c
>  create mode 100644 drivers/event/dlb2/dlb2_iface.c
>  create mode 100644 drivers/event/dlb2/dlb2_iface.h
>  create mode 100644 drivers/event/dlb2/dlb2_inline_fns.h
>  create mode 100644 drivers/event/dlb2/dlb2_log.h
>  create mode 100644 drivers/event/dlb2/dlb2_priv.h
>  create mode 100644 drivers/event/dlb2/dlb2_selftest.c
>  create mode 100644 drivers/event/dlb2/dlb2_user.h
>  create mode 100644 drivers/event/dlb2/dlb2_xstats.c
>  create mode 100644 drivers/event/dlb2/meson.build
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_hw_types.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_mbox.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_bitmap.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_list.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_osdep_types.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_regs.h
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.c
>  create mode 100644 drivers/event/dlb2/pf/base/dlb2_resource.h
>  create mode 100644 drivers/event/dlb2/pf/dlb2_main.c
>  create mode 100644 drivers/event/dlb2/pf/dlb2_main.h
>  create mode 100644 drivers/event/dlb2/pf/dlb2_pf.c
>  create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.c
>  create mode 100644 drivers/event/dlb2/rte_pmd_dlb2.h
>  create mode 100644 drivers/event/dlb2/version.map
>
> --
> 2.6.4
>