From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7BDF7424F0; Mon, 4 Sep 2023 15:09:21 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6259540A7F; Mon, 4 Sep 2023 15:09:07 +0200 (CEST) Received: from EUR02-DB5-obe.outbound.protection.outlook.com (mail-db5eur02on2070.outbound.protection.outlook.com [40.107.249.70]) by mails.dpdk.org (Postfix) with ESMTP id F0D7040691; Mon, 4 Sep 2023 15:09:04 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q4Rgxl7Dh3MLZ0U3UuTY2FYaWF/k7Dol7Cv/gdIUyf+yh23P4vvogviXnIuF9NnVraO+yvX6a1h5jC8HBgchqn7Uxdai4FmaXfpm0I7s8Yh0e2yrBXXu07650oiEuMbFDQPaXzHf0bErfet85P18qg954fE7vepUnHYfgRh5jqNFXR362szUtUU0t2fnECpeixgY898f1Fj2hZ5aM0BSLVxFdxbOHG0uV3RR1gHdfTHPIDz9j+qd+s3FtbKQcvVcOUpx6skFo8JlElzt7y/klpvOio29JXgkB0UcFGCW/spsRt45Eaab27IeqtGreuKJxNMCxEJeg30AdBsJvOCMSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xbDAlr8M72jTTvoeGytCs6kmFUGh1uQ1/XzSot2Z8u0=; b=T+DFUHHOQ/buJ6gekNx5Jk1S/R9pay0OtBq1lfiOpBotRhVrhdWInuOkQI5sMa2RoPAUt/FkBFY+kB9ZbMBv/HkZLQbDuXLXIa20wwQpAVCCC5vKPhmwIyPUzWBSW0ak/EMVDXs3p/ZVOMXQdP+oo4xncWMwJKG6lJUJzsqSPPnLhM3x1Qy63BEg6AvxtiJva7qONbsiodknrHxVBkjgbIvUQGZVyOwKhHZb+mJ/+EKMMDJRS2wmGCHbVswO+jTt/MCFcJ7qP/MPfGt3rUFdedxoC4vQMlrbJsIzZoZdgO9Gp9eYBcOFVaBB/JHRD4pefkhuhUeDFkdt+2aMLf7H3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 192.176.1.74) smtp.rcpttodomain=dpdk.org smtp.mailfrom=ericsson.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ericsson.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xbDAlr8M72jTTvoeGytCs6kmFUGh1uQ1/XzSot2Z8u0=; b=W9nV+xjcn6lU5YxzBXdMSggMfA4g/BH6nFg79oDpxMuI5VZqPxo5v1N12uBoC3GKLnhqWBI23Te3TskZx1hW+VYWGbbLmtfoc6hlsDKLQ5wICB0Iqwy4yWcLehcYNfxEUZlaP8oBwCDuAaay+Cix1RLnXoMAJtqYmll+a+ZHyYo= Received: from AS9PR01CA0043.eurprd01.prod.exchangelabs.com (2603:10a6:20b:542::21) by DU2PR07MB8320.eurprd07.prod.outlook.com (2603:10a6:10:27d::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.30; Mon, 4 Sep 2023 13:09:03 +0000 Received: from AM4PEPF00027A66.eurprd04.prod.outlook.com (2603:10a6:20b:542:cafe::15) by AS9PR01CA0043.outlook.office365.com (2603:10a6:20b:542::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6745.32 via Frontend Transport; Mon, 4 Sep 2023 13:09:03 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 192.176.1.74) smtp.mailfrom=ericsson.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ericsson.com; Received-SPF: Pass (protection.outlook.com: domain of ericsson.com designates 192.176.1.74 as permitted sender) receiver=protection.outlook.com; client-ip=192.176.1.74; helo=oa.msg.ericsson.com; pr=C Received: from oa.msg.ericsson.com (192.176.1.74) by AM4PEPF00027A66.mail.protection.outlook.com (10.167.16.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.6768.25 via Frontend Transport; Mon, 4 Sep 2023 13:09:03 +0000 Received: from ESESSMB503.ericsson.se (153.88.183.164) by ESESSMB504.ericsson.se (153.88.183.192) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2507.32; Mon, 4 Sep 2023 15:09:02 +0200 Received: from seliicinfr00049.seli.gic.ericsson.se (153.88.183.153) by smtp.internal.ericsson.com (153.88.183.191) with Microsoft SMTP Server id 15.1.2507.32 via Frontend Transport; Mon, 4 Sep 2023 15:09:02 +0200 Received: from breslau.. (seliicwb00002.seli.gic.ericsson.se [10.156.25.100]) by seliicinfr00049.seli.gic.ericsson.se (Postfix) with ESMTP id 7A5B0380061; Mon, 4 Sep 2023 15:09:02 +0200 (CEST) From: =?UTF-8?q?Mattias=20R=C3=B6nnblom?= To: CC: Jerin Jacob , , , , Peter Nilsson , Heng Wang , "Naga Harish K S V" , Pavan Nikhilesh , Gujjar Abhinandan S , Erik Gabriel Carrillo , Shijith Thotton , "Hemant Agrawal" , Sachin Saxena , Liang Ma , Peter Mccarthy , Zhirun Yan , =?UTF-8?q?Mattias=20R=C3=B6nnblom?= Subject: [PATCH v3 3/3] doc: add dispatcher programming guide Date: Mon, 4 Sep 2023 15:03:13 +0200 Message-ID: <20230904130313.327809-4-mattias.ronnblom@ericsson.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230904130313.327809-1-mattias.ronnblom@ericsson.com> References: <20230616074041.159675-2-mattias.ronnblom@ericsson.com> <20230904130313.327809-1-mattias.ronnblom@ericsson.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM4PEPF00027A66:EE_|DU2PR07MB8320:EE_ X-MS-Office365-Filtering-Correlation-Id: 8df46329-e65f-4929-642f-08dbad481c83 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 32aeKKNlYua8hJwz6Woe9Z5UhkF7aVMz5NIGrtG9KtM8tWnvLX/OVnBNqo5kPePYuijQ0y5qkcaTRydZ/Zr4NXKPZIapqlFSkjhL/HTDLk7fe37g3rz2MpBVKpdQ4W0i7oJF3hl21Hm0vUJ/LnQa/gVv0PbjhQXkplDD5txrPxPHRPFfmF6K/MeNkX0QSmWN9GzFEiF+CUr49vV3+aw8soIqYdNYc9+rvvVvpgMgXjSjnDzzhRzU6xGRFpjzNZFai7khBSr+oH55vJ84htpmAboL4zfBkhJy5pZ5InJmgcfJaYQfOIdqb1H0CIzhS/UTCUg3WFme3qvocnm2t0cyAqqrRLy2xsSWhI7i2Br5aZ9gm2hm0JR2WTdxhKCnis4PXpTp2M8hFWjUIO5cdFHLC/EqSTrkycborQITnZ7Zt+OcTJFTSoSj2BdsDtug8ho7yoLIihKVP/SZhXU1hKzKO5osBzvoQl1U/TFoNbKDlwmpllTnelj9Th0zpN9F5CqByAQ5vWV/E9VDr0UvTywA9gT8I2dpWuSjTy9AAjK1JVCTEBkuifBZn3VzapbKdjDUPWAAWji8Lbz3Z8wqUteEqHD/r15ZWgGcXTOoQjtI/sGNWjyJZQ2CesUYtxf0gEejBBEXnJ/N3k2OD5rS2KW4bESnyRQ3ZdT7nxpd+KxusCva5i7Xz/YwTvnOMk3v6Z/uqnmaVgDKG+Sl4T0UbkUUIU6eDgkZpFMjxDbq1mmY7nvuMxhXC3vi9pvIM52KlGbPu8xlckEV+Kfl/nh8FNwHiIycL1u2AAB3cIqqqUPDHqQ= X-Forefront-Antispam-Report: CIP:192.176.1.74; CTRY:SE; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:oa.msg.ericsson.com; PTR:office365.se.ericsson.net; CAT:NONE; SFS:(13230031)(4636009)(136003)(39860400002)(396003)(376002)(346002)(451199024)(186009)(1800799009)(82310400011)(46966006)(40470700004)(36840700001)(6916009)(2906002)(36756003)(54906003)(86362001)(316002)(70206006)(70586007)(40480700001)(5660300002)(4326008)(8936002)(8676002)(41300700001)(40460700003)(107886003)(66574015)(47076005)(2616005)(83380400001)(336012)(26005)(1076003)(6266002)(36860700001)(30864003)(7416002)(66899024)(478600001)(7636003)(356005)(82740400003)(6666004)(82960400001); DIR:OUT; SFP:1101; X-OriginatorOrg: ericsson.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2023 13:09:03.0211 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8df46329-e65f-4929-642f-08dbad481c83 X-MS-Exchange-CrossTenant-Id: 92e84ceb-fbfd-47ab-be52-080c6b87953f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=92e84ceb-fbfd-47ab-be52-080c6b87953f; Ip=[192.176.1.74]; Helo=[oa.msg.ericsson.com] X-MS-Exchange-CrossTenant-AuthSource: AM4PEPF00027A66.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU2PR07MB8320 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Provide programming guide for the dispatcher library. Signed-off-by: Mattias Rönnblom -- PATCH v3: o Adapt guide to the dispatcher API name changes. PATCH: o Improve grammar and spelling. RFC v4: o Extend event matching section of the programming guide. o Improve grammar and spelling. --- MAINTAINERS | 1 + doc/api/doxy-api-index.md | 1 + doc/api/doxy-api.conf.in | 1 + doc/guides/prog_guide/dispatcher_lib.rst | 434 +++++++++++++++++++++++ doc/guides/prog_guide/index.rst | 1 + 5 files changed, 438 insertions(+) create mode 100644 doc/guides/prog_guide/dispatcher_lib.rst diff --git a/MAINTAINERS b/MAINTAINERS index 43890cad0e..ab35498204 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1730,6 +1730,7 @@ Dispatcher - EXPERIMENTAL M: Mattias Rönnblom F: lib/dispatcher/ F: app/test/test_dispatcher.c +F: doc/guides/prog_guide/dispatcher_lib.rst Test Applications ----------------- diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index fdeda13932..7d0cad9fed 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -155,6 +155,7 @@ The public API headers are grouped by topics: - **classification** [reorder](@ref rte_reorder.h), + [dispatcher](@ref rte_dispatcher.h), [distributor](@ref rte_distributor.h), [EFD](@ref rte_efd.h), [ACL](@ref rte_acl.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index a88accd907..59c679e621 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -34,6 +34,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/cmdline \ @TOPDIR@/lib/compressdev \ @TOPDIR@/lib/cryptodev \ + @TOPDIR@/lib/dispatcher \ @TOPDIR@/lib/distributor \ @TOPDIR@/lib/dmadev \ @TOPDIR@/lib/efd \ diff --git a/doc/guides/prog_guide/dispatcher_lib.rst b/doc/guides/prog_guide/dispatcher_lib.rst new file mode 100644 index 0000000000..d4f29ce7ba --- /dev/null +++ b/doc/guides/prog_guide/dispatcher_lib.rst @@ -0,0 +1,434 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2023 Ericsson AB. + +Dispatcher +========== + +Overview +-------- + +The purpose of the dispatcher is to help reduce coupling in an +:doc:`Eventdev `-based DPDK application. + +In particular, the dispatcher addresses a scenario where an +application's modules share the same event device and event device +ports, and performs work on the same lcore threads. + +The dispatcher replaces the conditional logic that follows an event +device dequeue operation, where events are dispatched to different +parts of the application, typically based on fields in the +``rte_event``, such as the ``queue_id``, ``sub_event_type``, or +``sched_type``. + +Below is an excerpt from a fictitious application consisting of two +modules; A and B. In this example, event-to-module routing is based +purely on queue id, where module A expects all events to a certain +queue id, and module B two other queue ids. [#Mapping]_ + +.. code-block:: c + + for (;;) { + struct rte_event events[MAX_BURST]; + unsigned int n; + + n = rte_event_dequeue_burst(dev_id, port_id, events, + MAX_BURST, 0); + + for (i = 0; i < n; i++) { + const struct rte_event *event = &events[i]; + + switch (event->queue_id) { + case MODULE_A_QUEUE_ID: + module_a_process(event); + break; + case MODULE_B_STAGE_0_QUEUE_ID: + module_b_process_stage_0(event); + break; + case MODULE_B_STAGE_1_QUEUE_ID: + module_b_process_stage_1(event); + break; + } + } + } + +The issue this example attempts to illustrate is that the centralized +conditional logic has knowledge of things that should be private to +the modules. In other words, this pattern leads to a violation of +module encapsulation. + +The shared conditional logic contains explicit knowledge about what +events should go where. In case, for example, the +``module_a_process()`` is broken into two processing stages — a +module-internal affair — the shared conditional code must be updated +to reflect this change. + +The centralized event routing code becomes an issue in larger +applications, where modules are developed by different organizations. +This pattern also makes module reuse across different application more +difficult. The part of the conditional logic relevant for a particular +application may need to be duplicated across many module +instantiations (e.g., applications and test setups). + +The dispatcher separates the mechanism (routing events to their +receiver) from the policy (which events should go where). + +The basic operation of the dispatcher is as follows: + +* Dequeue a batch of events from the event device. +* For each event determine which handler should receive the event, using + a set of application-provided, per-handler event matching callback + functions. +* Provide events matching a particular handler, to that handler, using + its process callback. + +If the above application would have made use of the dispatcher, the +code relevant for its module A may have looked something like this: + +.. code-block:: c + + static bool + module_a_match(const struct rte_event *event, void *cb_data) + { + return event->queue_id == MODULE_A_QUEUE_ID; + } + + static void + module_a_process_events(uint8_t event_dev_id, uint8_t event_port_id, + const struct rte_event *events, + uint16_t num, void *cb_data) + { + uint16_t i; + + for (i = 0; i < num; i++) + module_a_process_event(&events[i]); + } + + /* In the module's initialization code */ + rte_dispatcher_register(DISPATCHER_ID, module_a_match, NULL, + module_a_process_events, module_a_data); + +(Error handling is left out of this and future example code in this +chapter.) + +When the shared conditional logic is removed, a new question arise: +which part of the system actually runs the dispatching mechanism? Or +phrased differently, what is replacing the function hosting the shared +conditional logic (typically launched on all lcores using +``rte_eal_remote_launch()``)? To solve this issue, the dispatcher is a +run as a DPDK :doc:`Service `. + +The dispatcher is a layer between the application and the event device +in the receive direction. In the transmit (i.e., item of work +submission) direction, the application directly accesses the Eventdev +core API (e.g., ``rte_event_enqueue_burst()``) to submit new or +forwarded event to the event device. + +Dispatcher Creation +------------------- + +A dispatcher is created with using +``rte_dispatcher_create()``. + +The dispatcher id is provided by the application, and must be unique. + +The event device must be configured before the dispatcher is created. + +Usually, only one dispatcher is needed per event device. A dispatcher +handles exactly one event device. + +An dispatcher is freed using the ``rte_dispatcher_free()`` +function. The dispatcher's service functions must not be running on +any lcore at the point of this call. + +Event Port Binding +------------------ + +To be able to dequeue events, the dispatcher must know which event +ports are to be used, on all the lcores it uses. The application +provides this information using +``rte_dispatcher_bind_port_to_lcore()``. + +This call is typically made from the part of the application that +deals with deployment issues (e.g., iterating lcores and determining +which lcore does what), at the time of application initialization. + +The ``rte_dispatcher_unbind_port_from_lcore()`` is used to undo +this operation. + +Multiple lcore threads may not safely use the same event +port. [#Port-MT-Safety] + +Event ports cannot safely be bound or unbound while the dispatcher's +service function is running on any lcore. + +Event Handlers +-------------- + +The dispatcher handler is an interface between the dispatcher and an +application module, used to route events to the appropriate part of +the application. + +Handler Registration +^^^^^^^^^^^^^^^^^^^^ + +The event handler interface consists of two function pointers: + +* The ``rte_dispatcher_match_t`` callback, which job is to + decide if this event is to be the property of this handler. +* The ``rte_dispatcher_process_t``, which is used by the + dispatcher to deliver matched events. + +An event handler registration is valid on all lcores. + +The functions pointed to by the match and process callbacks resides in +the application's domain logic, with one or more handlers per +application module. + +A module may use more than one event handler, for convenience or to +further decouple sub-modules. However, the dispatcher may impose an +upper limit of the number handlers. In addition, installing a large +number of handlers increase dispatcher overhead, although this does +not nessarily translate to a system-level performance degradation. See +the section on :ref:`Event Clustering` for more information. + +Handler registration and unregistration cannot safely be done while +the dispatcher's service function is running on any lcore. + +Event Matching +^^^^^^^^^^^^^^ + +A handler's match callback function decides if an event should be +delivered to this handler, or not. + +An event is routed to no more than one handler. Thus, if a match +function returns true, no further match functions will be invoked for +that event. + +Match functions must not depend on being invocated in any particular +order (e.g., in the handler registration order). + +Events failing to match any handler are dropped, and the +``ev_drop_count`` counter is updated accordingly. + +Event Delivery +^^^^^^^^^^^^^^ + +The handler callbacks are invocated by the dispatcher's service +function, upon the arrival of events to the event ports bound to the +running service lcore. + +A particular event is delivery to at most one handler. + +The application must not depend on all match callback invocations for +a particular event batch being made prior to any process calls are +being made. For example, if the dispatcher dequeues two events from +the event device, it may choose to find out the destination for the +first event, and deliver it, and then continue to find out the +destination for the second, and then deliver that event as well. The +dispatcher may also choose a strategy where no event is delivered +until the destination handler for both events have been determined. + +The events provided in a single process call always belong to the same +event port dequeue burst. + +.. _Event Clustering: + +Event Clustering +^^^^^^^^^^^^^^^^ + +The dispatcher maintains the order of events destined for the same +handler. + +*Order* here refers to the order in which the events were delivered +from the event device to the dispatcher (i.e., in the event array +populated by ``rte_event_dequeue_burst()``), in relation to the order +in which the dispatcher deliveres these events to the application. + +The dispatcher *does not* guarantee to maintain the order of events +delivered to *different* handlers. + +For example, assume that ``MODULE_A_QUEUE_ID`` expands to the value 0, +and ``MODULE_B_STAGE_0_QUEUE_ID`` expands to the value 1. Then +consider a scenario where the following events are dequeued from the +event device (qid is short for event queue id). + +.. code-block:: + + [e0: qid=1], [e1: qid=1], [e2: qid=0], [e3: qid=1] + +The dispatcher may deliver the events in the following manner: + +.. code-block:: + + module_b_stage_0_process([e0: qid=1], [e1: qid=1]) + module_a_process([e2: qid=0]) + module_b_stage_0_process([e2: qid=1]) + +The dispatcher may also choose to cluster (group) all events destined +for ``module_b_stage_0_process()`` into one array: + +.. code-block:: + + module_b_stage_0_process([e0: qid=1], [e1: qid=1], [e3: qid=1]) + module_a_process([e2: qid=0]) + +Here, the event ``e2`` is reordered and placed behind ``e3``, from a +delivery order point of view. This kind of reshuffling is allowed, +since the events are destined for different handlers. + +The dispatcher may also deliver ``e2`` before the three events +destined for module B. + +An example of what the dispatcher may not do, is to reorder event +``e1`` so, that it precedes ``e0`` in the array passed to the module +B's stage 0 process callback. + +Although clustering requires some extra work for the dispatcher, it +leads to fewer process function calls. In addition, and likely more +importantly, it improves temporal locality of memory accesses to +handler-specific data structures in the application, which in turn may +lead to fewer cache misses and improved overall performance. + +Finalize +-------- + +The dispatcher may be configured to notify one or more parts of the +application when the matching and processing of a batch of events has +completed. + +The ``rte_dispatcher_finalize_register`` call is used to +register a finalize callback. The function +``rte_dispatcher_finalize_unregister`` is used to remove a +callback. + +The finalize hook may be used by a set of event handlers (in the same +modules, or a set of cooperating modules) sharing an event output +buffer, since it allows for flushing of the buffers at the last +possible moment. In particular, it allows for buffering of +``RTE_EVENT_OP_FORWARD`` events, which must be flushed before the next +``rte_event_dequeue_burst()`` call is made (assuming implicit release +is employed). + +The following is an example with an application-defined event output +buffer (the ``event_buffer``): + +.. code-block:: c + + static void + finalize_batch(uint8_t event_dev_id, uint8_t event_port_id, + void *cb_data) + { + struct event_buffer *buffer = cb_data; + unsigned lcore_id = rte_lcore_id(); + struct event_buffer_lcore *lcore_buffer = + &buffer->lcore_buffer[lcore_id]; + + event_buffer_lcore_flush(lcore_buffer); + } + + /* In the module's initialization code */ + rte_dispatcher_finalize_register(DISPATCHER_ID, finalize_batch, + shared_event_buffer); + +The dispatcher does not track any relationship between a handler and a +finalize callback, and all finalize callbacks will be called, if (and +only if) at least one event was dequeued from the event device. + +Finalize callback registration and unregistration cannot safely be +done while the dispatcher's service function is running on any lcore. + +Service +------- + +The dispatcher is a DPDK service, and is managed in a manner similar +to other DPDK services (e.g., an Event Timer Adapter). + +Below is an example of how to configure a particular lcore to serve as +a service lcore, and to map an already-configured dispatcher +(identified by ``DISPATCHER_ID``) to that lcore. + +.. code-block:: c + + static void + launch_dispatcher_core(unsigned lcore_id) + { + uint32_t service_id; + + rte_service_lcore_add(lcore_id); + + rte_dispatcher_service_id_get(DISPATCHER_ID, &service_id); + + rte_service_map_lcore_set(service_id, lcore_id, 1); + + rte_service_lcore_start(lcore_id); + + rte_service_runstate_set(service_id, 1); + } + +As the final step, the dispatcher must be started. + +.. code-block:: c + + rte_dispatcher_start(DISPATCHER_ID); + + +Multi Service Dispatcher Lcores +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In an Eventdev application, most (or all) compute-intensive and +performance-sensitive processing is done in an event-driven manner, +where CPU cycles spent on application domain logic is the direct +result of items of work (i.e., ``rte_event`` events) dequeued from an +event device. + +In the light of this, it makes sense to have the dispatcher service be +the only DPDK service on all lcores used for packet processing — at +least in principle. + +However, there is nothing in DPDK that prevents colocating other +services with the dispatcher service on the same lcore. + +Tasks that prior to the introduction of the dispatcher into the +application was performed on the lcore, even though no events were +received, are prime targets for being converted into such auxiliary +services, running on the dispatcher core set. + +An example of such a task would be the management of a per-lcore timer +wheel (i.e., calling ``rte_timer_manage()``). + +For applications employing :doc:`Read-Copy-Update (RCU) ` (or +similar technique), may opt for having quiescent state (e.g., calling +``rte_rcu_qsbr_quiescent()``) signaling factored out into a separate +service, to assure resource reclaimination occurs even in though some +lcores currently do not process any events. + +If more services than the dispatcher service is mapped to a service +lcore, it's important that the other service are well-behaved and +don't interfere with event processing to the extent the system's +throughput and/or latency requirements are at risk of not being met. + +In particular, to avoid jitter, they should have an small upper bound +for the maximum amount of time spent in a single service function +call. + +An example of scenario with a more CPU-heavy colocated service is a +low-lcore count deployment, where the event device lacks the +``RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT`` capability (and thus +require software to feed incoming packets into the event device). In +this case, the best performance may be achieved if the Event Ethernet +RX and/or TX Adapters are mapped to lcores also used by for event +dispatching, since otherwise the adapter lcores would have a lot of +idle CPU cycles. + +.. rubric:: Footnotes + +.. [#Mapping] + Event routing may reasonably be done based on other ``rte_event`` + fields (or even event user data). Indeed, that's the very reason to + have match callback functions, instead of a simple queue + id-to-handler mapping scheme. Queue id-based routing serves well in + a simple example. + +.. [#Port-MT-Safety] + This property (which is a feature, not a bug) is inherited from the + core Eventdev APIs. diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 52a6d9e7aa..ab05bd6074 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -60,6 +60,7 @@ Programmer's Guide event_ethernet_tx_adapter event_timer_adapter event_crypto_adapter + dispatcher_lib qos_framework power_man packet_classif_access_ctrl -- 2.34.1