Re: [PATCH v2] event/dsw: support explicit release only mode

[Jerin Jacob <jerinjacobk@gmail.com>]

On Wed, Jun 5, 2024 at 7:43 PM Mattias Rönnblom
<mattias.ronnblom@ericsson.com> wrote:
>
> Add the RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE capability to the
> DSW event device.
>
> This feature may be used by an EAL thread to pull more work from the
> work scheduler, without giving up the option to forward events
> originating from a previous dequeue batch. This in turn allows an EAL
> thread to be productive while waiting for a hardware accelerator to
> complete some operation.
>
> Prior to this change, DSW didn't make any distinction between
> RTE_EVENT_OP_FORWARD and RTE_EVENT_OP_NEW type events, other than that
> new events would be backpressured earlier.
>
> After this change, DSW tracks the number of released events (i.e.,
> events of type RTE_EVENT_OP_FORWARD and RTE_EVENT_OP_RELEASE) that has
> been enqueued.
>
> For efficiency reasons, DSW does not track the identity of individual
> events. This in turn implies that a certain stage in the flow
> migration process, DSW must wait for all pending releases (on the
> migration source port, only) to be received from the application, to
> assure that no event pertaining to any of the to-be-migrated flows are
> being processed.
>
> With this change, DSW starts making a distinction between forward and
> new type events for credit allocation purposes. Only RTE_EVENT_OP_NEW
> events needs credits. All events marked as RTE_EVENT_OP_FORWARD must
> have a corresponding dequeued event from a previous dequeue batch.
>
> Flow migration for flows on RTE_SCHED_TYPE_PARALLEL queues remains
> unaffected by this change.
>
> A side-effect of the tweaked DSW migration logic is that the migration
> latency is reduced, regardless if implicit release is enabled or not.
>
> Another side-effect is that migrated flows are now not processed
> during any part of the migration procedure. An upside of this change
> it reduces the load of the overloaded port. A downside is it
> introduces slightly more jitter for the migrated flows.
>
> This patch is contains various minor refactorings, improved
> formatting, fixed spelling, and the removal of unnessary memory
> barriers.
>
> v2:
>   * Remove redundant memory barriers.
>   * Discontinue processing of migrated flows throughout the migration
>     procedure. This is a part of a fix to address a reordering issue
>     v1 of this patch introduced.
>   * Added entry in the release notes.
>   * Fix spelling issues in commit message.
>
> Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>

meson --werror -Dc_args='-DRTE_ENABLE_ASSERT' -Denable_docs=true build


[1602/2784] Generating drivers/rte_net_af_xdp.pmd.c with a custom command
[1603/2784] Generating drivers/rte_net_ark.pmd.c with a custom command
[1604/2784] Generating drivers/rte_net_axgbe.pmd.c with a custom command
[1605/2784] Compiling C object
drivers/libtmp_rte_event_dsw.a.p/event_dsw_dsw_event.c.o
FAILED: drivers/libtmp_rte_event_dsw.a.p/event_dsw_dsw_event.c.o
ccache cc -Idrivers/libtmp_rte_event_dsw.a.p -Idrivers -I../drivers
-Idrivers/event/dsw -I../drivers/event/dsw -Ilib/eventdev
-I../lib/eventdev -I. -I.. -Iconfig -I../config -Ilib/eal/include
-I../lib/eal/include -Ilib/eal/linux/include
-I../lib/eal/linux/include -Ilib/eal/x86/include
-I../lib/eal/x86/include -Ilib/eal/common -I../lib/eal/common
-Ilib/eal -I../lib/eal -Ilib/kvargs -I../lib/kvargs -Ilib/log
-I../lib/log -Ilib/metrics -I../lib/metrics -Ilib/telemetry
-I../lib/telemetry -Ilib/ring -I../lib/ring -Ilib/ethdev
-I../lib/ethdev -Ilib/net -I../lib/net -Ilib/mbuf -I../lib/mbuf
-Ilib/mempool -I../lib/mempool -Ilib/meter -I../lib/meter -Ilib/hash
-I../lib/hash -Ilib/rcu -I../lib/rcu -Ilib/timer -I../lib/timer
-Ilib/cryptodev -I../lib/cryptodev -Ilib/dmadev -I../lib/dmadev
-Idrivers/bus/vdev -I../drivers/bus/vdev -fdiagnostics-color=always
-D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Werror -std=c11
-O3 -include rte_config.h -Wcast-qual -Wdeprecated -Wformat
-Wformat-nonliteral -Wformat-security -Wmissing-declarations
-Wmissing-prototypes -Wnested-externs -Wold-style-definition
-Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef
-Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned
-Wno-missing-field-initializers -Wno-zero-length-bounds -D_GNU_SOURCE
-DRTE_ENABLE_ASSERT -fPIC -march=native -mrtm -DALLOW_EXPERIMENTAL_API
-DALLOW_INTERNAL_API -Wno-format-truncation -Wno-format-nonliteral
-DRTE_LOG_DEFAULT_LOGTYPE=pmd.event.dsw -MD -MQ
drivers/libtmp_rte_event_dsw.a.p/event_dsw_dsw_event.c.o -MF
drivers/libtmp_rte_event_dsw.a.p/event_dsw_dsw_event.c.o.d -o
drivers/libtmp_rte_event_dsw.a.p/event_dsw_dsw_event.c.o -c
../drivers/event/dsw/dsw_event.c
In file included from ../lib/eal/include/rte_debug.h:18,
                 from ../lib/eal/include/rte_bitops.h:20,
                 from ../lib/eal/include/rte_memory.h:22,
                 from ../lib/eal/include/rte_malloc.h:16,
                 from ../lib/eventdev/eventdev_pmd.h:27,
                 from ../drivers/event/dsw/dsw_evdev.h:8,
                 from ../drivers/event/dsw/dsw_event.c:5:
../drivers/event/dsw/dsw_event.c: In function ‘dsw_port_abort_migration’:
../drivers/event/dsw/dsw_event.c:946:20: error: ‘port’ undeclared
(first use in this function)
  946 |         RTE_ASSERT(port->in_buffer_start == 0);
      |                    ^~~~
../lib/eal/include/rte_branch_prediction.h:47:45: note: in definition
of macro ‘unlikely’
   47 | #define unlikely(x)     __builtin_expect(!!(x), 0)
      |                                             ^
../lib/eal/include/rte_debug.h:47:25: note: in expansion of macro ‘RTE_VERIFY’
   47 | #define RTE_ASSERT(exp) RTE_VERIFY(exp)
      |                         ^~~~~~~~~~
../drivers/event/dsw/dsw_event.c:946:9: note: in expansion of macro ‘RTE_ASSERT’
  946 |         RTE_ASSERT(port->in_buffer_start == 0);
      |         ^~~~~~~~~~
../drivers/event/dsw/dsw_event.c:946:20: note: each undeclared
identifier is reported only once for each function it appears in
  946 |         RTE_ASSERT(port->in_buffer_start == 0);
      |                    ^~~~
../lib/eal/include/rte_branch_prediction.h:47:45: note: in definition
of macro ‘unlikely’
   47 | #define unlikely(x)     __builtin_expect(!!(x), 0)
      |                                             ^
../lib/eal/include/rte_debug.h:47:25: note: in expansion of macro ‘RTE_VERIFY’
   47 | #define RTE_ASSERT(exp) RTE_VERIFY(exp)
      |                         ^~~~~~~~~~
../drivers/event/dsw/dsw_event.c:946:9: note: in expansion of macro ‘RTE_ASSERT’
  946 |         RTE_ASSERT(port->in_buffer_start == 0);