From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 71BC342CAB; Tue, 13 Jun 2023 16:13:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id AB28942D62; Tue, 13 Jun 2023 16:12:44 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id 1CF6742D73 for ; Tue, 13 Jun 2023 16:12:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686665560; x=1718201560; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oTbke4D1HvMXlMrv0pmclDH+FYUB/W7nou//5qLmWZE=; b=dModznhARzkGESXjP8fU7sTP9vo8IuALFzjWre+3Z4A8EtsudYS9HHPY fIcBy1e77Ycw17sKOEs9jsnDxUxDhQ/wcBQZlN6N8Jl1yR8bQrxkTgo+c r1ELnVt6EgCrg/7Fr3YMjhiEUymxkoSIYTxjH4pMDwHs/FaUJmVTaurFm 65XdU9etaEFJ42U3XlcZdT6LU0o2pwvVrpUs41aZu86rUPyKboUy44iFG zBHrYhEtbL12NxIt7KgWa3gYJX9bk02owLWavCYzt6y1ekLAl0Utiv+/I 7ImBxCA+yUhQ0eSqsW0M9RippW9GZu4Gn/HcDPiC3h087KEa1wyD/Xw3V g==; X-IronPort-AV: E=McAfee;i="6600,9927,10740"; a="355839910" X-IronPort-AV: E=Sophos;i="6.00,239,1681196400"; d="scan'208";a="355839910" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Jun 2023 07:12:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10740"; a="741439430" X-IronPort-AV: E=Sophos;i="6.00,239,1681196400"; d="scan'208";a="741439430" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga008.jf.intel.com with ESMTP; 13 Jun 2023 07:12:36 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com, david.marchand@redhat.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v14 13/16] graph: enable graph multicore dispatch scheduler model Date: Tue, 13 Jun 2023 22:04:42 +0800 Message-Id: <20230613140445.2155542-14-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230613140445.2155542-1-zhirun.yan@intel.com> References: <20230613101404.1787790-1-zhirun.yan@intel.com> <20230613140445.2155542-1-zhirun.yan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch enables to chose new scheduler model. Must define RTE_GRAPH_MODEL_SELECT before including rte_graph_worker.h to enable specific model choosing. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob Acked-by: Pavan Nikhilesh --- doc/guides/prog_guide/graph_lib.rst | 71 ++++++++++++++++++++++++++--- lib/graph/rte_graph_worker.h | 13 ++++++ 2 files changed, 77 insertions(+), 7 deletions(-) diff --git a/doc/guides/prog_guide/graph_lib.rst b/doc/guides/prog_guide/graph_lib.rst index 07248f06c4..6f85e6faef 100644 --- a/doc/guides/prog_guide/graph_lib.rst +++ b/doc/guides/prog_guide/graph_lib.rst @@ -189,13 +189,70 @@ In the above example, A graph object will be created with ethdev Rx node of port 0 and queue 0, all ipv4* nodes in the system, and ethdev tx node of all ports. -Multicore graph processing -~~~~~~~~~~~~~~~~~~~~~~~~~~ -In the current graph library implementation, specifically, -``rte_graph_walk()`` and ``rte_node_enqueue*`` fast path API functions -are designed to work on single-core to have better performance. -The fast path API works on graph object, So the multi-core graph -processing strategy would be to create graph object PER WORKER. +Graph models +~~~~~~~~~~~~ +There are two different kinds of graph walking models. User can select the model using +``rte_graph_worker_model_set()`` API. If the application decides to use only one model, +the fast path check can be avoided by defining the model with RTE_GRAPH_MODEL_SELECT. +For example: + +.. code-block:: console + +#define RTE_GRAPH_MODEL_SELECT RTE_GRAPH_MODEL_RTC +#include "rte_graph_worker.h" + +RTC (Run-To-Completion) +^^^^^^^^^^^^^^^^^^^^^^^ +This is the default graph walking model. Specifically, ``rte_graph_walk_rtc()`` and +``rte_node_enqueue*`` fast path API functions are designed to work on single-core to +have better performance. The fast path API works on graph object, So the multi-core +graph processing strategy would be to create graph object PER WORKER. + +Example: + +Graph: node-0 -> node-1 -> node-2 @Core0. + +.. code-block:: diff + + + - - - - - - - - - - - - - - - - - - - - - + + ' Core #0 ' + ' ' + ' +--------+ +---------+ +--------+ ' + ' | Node-0 | --> | Node-1 | --> | Node-2 | ' + ' +--------+ +---------+ +--------+ ' + ' ' + + - - - - - - - - - - - - - - - - - - - - - + + +Dispatch model +^^^^^^^^^^^^^^ +The dispatch model enables a cross-core dispatching mechanism which employs +a scheduling work-queue to dispatch streams to other worker cores which +being associated with the destination node. + +Use ``rte_graph_model_mcore_dispatch_lcore_affinity_set()`` to set lcore affinity +with the node. +Each worker core will have a graph repetition. Use ``rte_graph_clone()`` to clone +graph for each worker and use``rte_graph_model_mcore_dispatch_core_bind()`` to +bind graph with the worker core. + +Example: + +Graph topo: node-0 -> Core1; node-1 -> node-2; node-2 -> node-3. +Config graph: node-0 @Core0; node-1/3 @Core1; node-2 @Core2. + +.. code-block:: diff + + + - - - - - -+ +- - - - - - - - - - - - - + + - - - - - -+ + ' Core #0 ' ' Core #1 ' ' Core #2 ' + ' ' ' ' ' ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' | Node-0 | - - - ->| Node-1 | | Node-3 |<- - - - | Node-2 | ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' ' ' | ' ' ^ ' + + - - - - - -+ +- - -|- - - - - - - - - - + + - - -|- - -+ + | | + + - - - - - - - - - - - - - - - - + + In fast path ~~~~~~~~~~~~ diff --git a/lib/graph/rte_graph_worker.h b/lib/graph/rte_graph_worker.h index 5b58f7bda9..6685600813 100644 --- a/lib/graph/rte_graph_worker.h +++ b/lib/graph/rte_graph_worker.h @@ -11,6 +11,7 @@ extern "C" { #endif #include "rte_graph_model_rtc.h" +#include "rte_graph_model_mcore_dispatch.h" /** * Perform graph walk on the circular buffer and invoke the process function @@ -25,7 +26,19 @@ __rte_experimental static inline void rte_graph_walk(struct rte_graph *graph) { +#if defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_RTC) rte_graph_walk_rtc(graph); +#elif defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_MCORE_DISPATCH) + rte_graph_walk_mcore_dispatch(graph); +#else + switch (rte_graph_worker_model_no_check_get(graph)) { + case RTE_GRAPH_MODEL_MCORE_DISPATCH: + rte_graph_walk_mcore_dispatch(graph); + break; + default: + rte_graph_walk_rtc(graph); + } +#endif } #ifdef __cplusplus -- 2.37.2