From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 320B542C5C; Thu, 8 Jun 2023 12:06:37 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0F48342D3B; Thu, 8 Jun 2023 12:05:48 +0200 (CEST) Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by mails.dpdk.org (Postfix) with ESMTP id 34CBE42D3B for ; Thu, 8 Jun 2023 12:05:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686218746; x=1717754746; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=smiop0Isdx3RfgQLVV8m4ueXrBnbMznKM/Bf7LcNiF8=; b=UayOmTx18vlGnLTafTef7vC/5JluecrsczCEw9cU3nW4Ka8ajqbmot/k jkTXkWXu1ziEEd7RVuURXjcySwhBmNfAtG/Ww2c4o+10y209BP0gTrbw3 R4yCYs3C4yil77Xlawc0O+O+LFmE2LXO7CBkZdPwbHxfhJbobV48JoAa2 zQxu8aIFxLJ5z8pAVU217YIU9pRhjCLkyNNDDuRoHMBe1vgYV1dqxnyC7 6rY3LM9OQGM+Tl8wHMw1lh3M0O0vIQXEVrGWvBDZ1DrFCWjxP6cjEBllG m3yIr58tdw2Gw+/bSmWfUIwcQj5bm2ox7uBB7krye1t/lvdjRftvSIMlE g==; X-IronPort-AV: E=McAfee;i="6600,9927,10734"; a="336896297" X-IronPort-AV: E=Sophos;i="6.00,226,1681196400"; d="scan'208";a="336896297" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 03:05:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10734"; a="743053768" X-IronPort-AV: E=Sophos;i="6.00,226,1681196400"; d="scan'208";a="743053768" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga001.jf.intel.com with ESMTP; 08 Jun 2023 03:05:42 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v10 13/16] graph: enable graph multicore dispatch scheduler model Date: Thu, 8 Jun 2023 17:57:56 +0800 Message-Id: <20230608095759.1800617-14-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608095759.1800617-1-zhirun.yan@intel.com> References: <20230607035144.1214492-1-zhirun.yan@intel.com> <20230608095759.1800617-1-zhirun.yan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch enables to chose new scheduler model. Must define RTE_GRAPH_MODEL_SELECT before including rte_graph_worker.h to enable specific model choosing. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan --- doc/guides/prog_guide/graph_lib.rst | 71 ++++++++++++++++++++++++++--- lib/graph/rte_graph_worker.h | 13 ++++++ 2 files changed, 77 insertions(+), 7 deletions(-) diff --git a/doc/guides/prog_guide/graph_lib.rst b/doc/guides/prog_guide/graph_lib.rst index 1cfdc86433..017cc25fd3 100644 --- a/doc/guides/prog_guide/graph_lib.rst +++ b/doc/guides/prog_guide/graph_lib.rst @@ -189,13 +189,70 @@ In the above example, A graph object will be created with ethdev Rx node of port 0 and queue 0, all ipv4* nodes in the system, and ethdev tx node of all ports. -Multicore graph processing -~~~~~~~~~~~~~~~~~~~~~~~~~~ -In the current graph library implementation, specifically, -``rte_graph_walk()`` and ``rte_node_enqueue*`` fast path API functions -are designed to work on single-core to have better performance. -The fast path API works on graph object, So the multi-core graph -processing strategy would be to create graph object PER WORKER. +Graph models +~~~~~~~~~~~~ +There are two different kinds of graph walking models. User can select the model using +``rte_graph_worker_model_set()`` API. If the application decides to use only one model, +the fast path check can be avoided by defining the model with RTE_GRAPH_MODEL_SELECT. +For example: + +.. code-block:: console + +#define RTE_GRAPH_MODEL_SELECT RTE_GRAPH_MODEL_RTC +#include "rte_graph_worker.h" + +RTC (Run-To-Completion) +^^^^^^^^^^^^^^^^^^^^^^^ +This is the default graph walking model. Specifically, ``rte_graph_walk_rtc()`` and +``rte_node_enqueue*`` fast path API functions are designed to work on single-core to +have better performance. The fast path API works on graph object, So the multi-core +graph processing strategy would be to create graph object PER WORKER. + +Example: + +Graph: node-0 -> node-1 -> node-2 @Core0. + +.. code-block:: diff + + + - - - - - - - - - - - - - - - - - - - - - + + ' Core #0 ' + ' ' + ' +--------+ +---------+ +--------+ ' + ' | Node-0 | --> | Node-1 | --> | Node-2 | ' + ' +--------+ +---------+ +--------+ ' + ' ' + + - - - - - - - - - - - - - - - - - - - - - + + +Dispatch model +^^^^^^^^^^^^^^ +The dispatch model enables a cross-core dispatching mechanism which employs +a scheduling work-queue to dispatch streams to other worker cores which +being associated with the destination node. + +Use ``rte_graph_model_mcore_dispatch_lcore_affinity_set()`` to set lcore affinity +with the node. +Each worker core will have a graph repetition. Use ``rte_graph_clone()`` to clone +graph for each worker and use``rte_graph_model_mcore_dispatch_core_bind()`` to +bind graph with the worker core. + +Example: + +Graph topo: node-0 -> Core1; node-1 -> node-2; node-2 -> node-3. +Config graph: node-0 @Core0; node-1/3 @Core1; node-2 @Core2. + +.. code-block:: diff + + + - - - - - -+ +- - - - - - - - - - - - - + + - - - - - -+ + ' Core #0 ' ' Core #1 ' ' Core #2 ' + ' ' ' ' ' ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' | Node-0 | - - - ->| Node-1 | | Node-3 |<- - - - | Node-2 | ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' ' ' | ' ' ^ ' + + - - - - - -+ +- - -|- - - - - - - - - - + + - - -|- - -+ + | | + + - - - - - - - - - - - - - - - - + + In fast path ~~~~~~~~~~~~ diff --git a/lib/graph/rte_graph_worker.h b/lib/graph/rte_graph_worker.h index 5b58f7bda9..6685600813 100644 --- a/lib/graph/rte_graph_worker.h +++ b/lib/graph/rte_graph_worker.h @@ -11,6 +11,7 @@ extern "C" { #endif #include "rte_graph_model_rtc.h" +#include "rte_graph_model_mcore_dispatch.h" /** * Perform graph walk on the circular buffer and invoke the process function @@ -25,7 +26,19 @@ __rte_experimental static inline void rte_graph_walk(struct rte_graph *graph) { +#if defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_RTC) rte_graph_walk_rtc(graph); +#elif defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_MCORE_DISPATCH) + rte_graph_walk_mcore_dispatch(graph); +#else + switch (rte_graph_worker_model_no_check_get(graph)) { + case RTE_GRAPH_MODEL_MCORE_DISPATCH: + rte_graph_walk_mcore_dispatch(graph); + break; + default: + rte_graph_walk_rtc(graph); + } +#endif } #ifdef __cplusplus -- 2.37.2