RE: [EXT] Re: [RFC 0/3] Introduce event link profiles

[Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>]

> On 2023-08-09 16:26, pbhagavatula@marvell.com wrote:
> > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >
> > A collection of event queues linked to an event port can be associated
> > with unique identifier called as a profile, multiple such profiles can
> > be configured based on the event device capability using the function
> > `rte_event_port_link_with_profile` which takes arguments similar to
> > `rte_event_port_link` in addition to the profile identifier.
> >
> 
> What is the overall goal with this new API? What problems does it intend
> to solve, that the old one doesn't.

Linking and unlinking currently has huge overhead and when it needs to be done
in fastpath, we have to wait for unlinks to complete and handle other corner cases.

This patch set solves it by avoiding linking/unlinking altogether in fastpath by
preconfigured set of link profiles out of which only one would be active and can 
be changed in fastpath with a simple function call. There is no link/unlink waiting for
unlink overhead.

> 
> > The maximum link profiles that are supported by an event device is
> > advertised through the structure member
> > `rte_event_dev_info::max_profiles_per_port`.
> >
> > By default, event ports are configured to use the link profile 0 on
> > initialization.
> >
> > Once multiple link profiles are set up and the event device is started, the
> > application can use the function `rte_event_port_change_profile` to
> change
> > the currently active profile on an event port. This effects the next
> > `rte_event_dequeue_burst` call, where the event queues associated with
> the
> > newly active link profile will participate in scheduling.
> >
> > Rudementary work flow would something like:
> >
> > Config path:
> >
> >      uint8_t lowQ[4] = {4, 5, 6, 7};
> >      uint8_t highQ[4] = {0, 1, 2, 3};
> >
> >      if (rte_event_dev_info.max_profiles_per_port < 2)
> >          return -ENOTSUP;
> >
> >      rte_event_port_link_with_profile(0, 0, highQ, NULL, 4, 0);
> >      rte_event_port_link_with_profile(0, 0, lowQ, NULL, 4, 1);
> >
> > Worker path:
> >
> >      empty_high_deq = 0;
> >      empty_low_deq = 0;
> >      is_low_deq = 0;
> >      while (1) {
> >          deq = rte_event_dequeue_burst(0, 0, &ev, 1, 0);
> >          if (deq == 0) {
> >              /**
> >               * Change link profile based on work activity on current
> >               * active profile
> >               */
> >              if (is_low_deq) {
> >                  empty_low_deq++;
> >                  if (empty_low_deq == MAX_LOW_RETRY) {
> >                      rte_event_port_change_profile(0, 0, 0);
> >                      is_low_deq = 0;
> >                      empty_low_deq = 0;
> >                  }
> >                  continue;
> >              }
> >
> >              if (empty_high_deq == MAX_HIGH_RETRY) {
> >                  rte_event_port_change_profile(0, 0, 1);
> >                  is_low_deq = 1;
> >                  empty_high_deq = 0;
> >              }
> >              continue;
> >          }
> >
> >          // Process the event received.
> >
> >          if (is_low_deq++ == MAX_LOW_EVENTS) {
> >              rte_event_port_change_profile(0, 0, 0);
> >              is_low_deq = 0;
> >          }
> >      }
> >
> 
> This thing looks like the application is asked to do work scheduling.
> That doesn't sound right. That's the job of the work scheduler (i.e.,
> the event device).
> 
> If this thing is merely a matter of changing what queues are linked to
> which ports, wouldn't a new call:
> rte_event_port_link_modify()
> suffice?


Some applications divide their available lcores into multiple types of 
workers which each work on a unique set of event queues, application might 
need to modify the worker ratio based on various parameters at run time
without a lot of overhead.

Modifying links wouldn’t work because we might want to restore previous links 
based on the new traffic pattern etc.,.

> 
> > An application could use heuristic data of load/activity of a given event
> > port and change its active profile to adapt to the traffic pattern.
> >
> > An unlink function `rte_event_port_unlink_with_profile` is provided to
> > modify the links associated to a profile, and
> > `rte_event_port_links_get_with_profile` can be used to retrieve the links
> > associated with a profile.
> >
> > Pavan Nikhilesh (3):
> >    eventdev: introduce link profiles
> >    event/cnxk: implement event link profiles
> >    test/event: add event link profile test
> >
> >   app/test/test_eventdev.c                   | 110 ++++++++++
> >   config/rte_config.h                        |   1 +
> >   doc/guides/eventdevs/cnxk.rst              |   1 +
> >   doc/guides/prog_guide/eventdev.rst         |  58 ++++++
> >   drivers/common/cnxk/roc_nix_inl_dev.c      |   4 +-
> >   drivers/common/cnxk/roc_sso.c              |  18 +-
> >   drivers/common/cnxk/roc_sso.h              |   8 +-
> >   drivers/common/cnxk/roc_sso_priv.h         |   4 +-
> >   drivers/event/cnxk/cn10k_eventdev.c        |  45 ++--
> >   drivers/event/cnxk/cn10k_worker.c          |  11 +
> >   drivers/event/cnxk/cn10k_worker.h          |   1 +
> >   drivers/event/cnxk/cn9k_eventdev.c         |  72 ++++---
> >   drivers/event/cnxk/cn9k_worker.c           |  22 ++
> >   drivers/event/cnxk/cn9k_worker.h           |   2 +
> >   drivers/event/cnxk/cnxk_eventdev.c         |  34 ++--
> >   drivers/event/cnxk/cnxk_eventdev.h         |  10 +-
> >   drivers/event/dlb2/dlb2.c                  |   1 +
> >   drivers/event/dpaa/dpaa_eventdev.c         |   1 +
> >   drivers/event/dpaa2/dpaa2_eventdev.c       |   2 +-
> >   drivers/event/dsw/dsw_evdev.c              |   1 +
> >   drivers/event/octeontx/ssovf_evdev.c       |   2 +-
> >   drivers/event/opdl/opdl_evdev.c            |   1 +
> >   drivers/event/skeleton/skeleton_eventdev.c |   1 +
> >   drivers/event/sw/sw_evdev.c                |   1 +
> >   lib/eventdev/eventdev_pmd.h                |  59 +++++-
> >   lib/eventdev/eventdev_private.c            |   9 +
> >   lib/eventdev/eventdev_trace.h              |  22 ++
> >   lib/eventdev/eventdev_trace_points.c       |   6 +
> >   lib/eventdev/rte_eventdev.c                | 146 ++++++++++---
> >   lib/eventdev/rte_eventdev.h                | 226 +++++++++++++++++++++
> >   lib/eventdev/rte_eventdev_core.h           |   4 +
> >   lib/eventdev/rte_eventdev_trace_fp.h       |   8 +
> >   lib/eventdev/version.map                   |   5 +
> >   33 files changed, 788 insertions(+), 108 deletions(-)
> >
> > --
> > 2.25.1
> >