From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BF18942C61; Thu, 8 Jun 2023 17:27:13 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0B5FE42D0B; Thu, 8 Jun 2023 17:26:49 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by mails.dpdk.org (Postfix) with ESMTP id 4F9FF42D12 for ; Thu, 8 Jun 2023 17:26:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686238007; x=1717774007; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PbRAcqku2qwFKjQpRjCZ1/BAXJLK1K/Cj2WDWvQqwQU=; b=Mhbin9ZUTNzC117XUhhXSDtoAAO8KpEpVB4OjCvzbSW9cY9klUIREtA3 3ERuhH9T7uSfUW8SYjv9pXUDCkqrersP/gOwmOpdTjZMJWLbDN/sjY7cD VbnSqUc1m7nr3C7d3LOoOqe1XNco9Dk8yLLQNoEivonsVo0VyojK31Eau bKgfXrDc9X13kx2c/KZ7pQeuKZhgq/1QAHEKa1tlXyjfJknmo/c38IFZA GIS1ExuIFvpood8bnqdlF1CQH1W/PGJMWbmJQ0KXOKatqHG7OPlrIMQPF IuYvmXNrE7QXNqiu/9b1n9HzxWWgGDhaiLS+QiixFwg0ezMghJD9UFaSx g==; X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="385681571" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="385681571" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Jun 2023 08:26:26 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10735"; a="834233123" X-IronPort-AV: E=Sophos;i="6.00,227,1681196400"; d="scan'208";a="834233123" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga004.jf.intel.com with ESMTP; 08 Jun 2023 08:26:23 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v11 13/16] graph: enable graph multicore dispatch scheduler model Date: Thu, 8 Jun 2023 23:18:41 +0800 Message-Id: <20230608151844.1823783-14-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230608151844.1823783-1-zhirun.yan@intel.com> References: <20230608095759.1800617-1-zhirun.yan@intel.com> <20230608151844.1823783-1-zhirun.yan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch enables to chose new scheduler model. Must define RTE_GRAPH_MODEL_SELECT before including rte_graph_worker.h to enable specific model choosing. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob Acked-by: Pavan Nikhilesh --- doc/guides/prog_guide/graph_lib.rst | 71 ++++++++++++++++++++++++++--- lib/graph/rte_graph_worker.h | 13 ++++++ 2 files changed, 77 insertions(+), 7 deletions(-) diff --git a/doc/guides/prog_guide/graph_lib.rst b/doc/guides/prog_guide/graph_lib.rst index 1cfdc86433..017cc25fd3 100644 --- a/doc/guides/prog_guide/graph_lib.rst +++ b/doc/guides/prog_guide/graph_lib.rst @@ -189,13 +189,70 @@ In the above example, A graph object will be created with ethdev Rx node of port 0 and queue 0, all ipv4* nodes in the system, and ethdev tx node of all ports. -Multicore graph processing -~~~~~~~~~~~~~~~~~~~~~~~~~~ -In the current graph library implementation, specifically, -``rte_graph_walk()`` and ``rte_node_enqueue*`` fast path API functions -are designed to work on single-core to have better performance. -The fast path API works on graph object, So the multi-core graph -processing strategy would be to create graph object PER WORKER. +Graph models +~~~~~~~~~~~~ +There are two different kinds of graph walking models. User can select the model using +``rte_graph_worker_model_set()`` API. If the application decides to use only one model, +the fast path check can be avoided by defining the model with RTE_GRAPH_MODEL_SELECT. +For example: + +.. code-block:: console + +#define RTE_GRAPH_MODEL_SELECT RTE_GRAPH_MODEL_RTC +#include "rte_graph_worker.h" + +RTC (Run-To-Completion) +^^^^^^^^^^^^^^^^^^^^^^^ +This is the default graph walking model. Specifically, ``rte_graph_walk_rtc()`` and +``rte_node_enqueue*`` fast path API functions are designed to work on single-core to +have better performance. The fast path API works on graph object, So the multi-core +graph processing strategy would be to create graph object PER WORKER. + +Example: + +Graph: node-0 -> node-1 -> node-2 @Core0. + +.. code-block:: diff + + + - - - - - - - - - - - - - - - - - - - - - + + ' Core #0 ' + ' ' + ' +--------+ +---------+ +--------+ ' + ' | Node-0 | --> | Node-1 | --> | Node-2 | ' + ' +--------+ +---------+ +--------+ ' + ' ' + + - - - - - - - - - - - - - - - - - - - - - + + +Dispatch model +^^^^^^^^^^^^^^ +The dispatch model enables a cross-core dispatching mechanism which employs +a scheduling work-queue to dispatch streams to other worker cores which +being associated with the destination node. + +Use ``rte_graph_model_mcore_dispatch_lcore_affinity_set()`` to set lcore affinity +with the node. +Each worker core will have a graph repetition. Use ``rte_graph_clone()`` to clone +graph for each worker and use``rte_graph_model_mcore_dispatch_core_bind()`` to +bind graph with the worker core. + +Example: + +Graph topo: node-0 -> Core1; node-1 -> node-2; node-2 -> node-3. +Config graph: node-0 @Core0; node-1/3 @Core1; node-2 @Core2. + +.. code-block:: diff + + + - - - - - -+ +- - - - - - - - - - - - - + + - - - - - -+ + ' Core #0 ' ' Core #1 ' ' Core #2 ' + ' ' ' ' ' ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' | Node-0 | - - - ->| Node-1 | | Node-3 |<- - - - | Node-2 | ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' ' ' | ' ' ^ ' + + - - - - - -+ +- - -|- - - - - - - - - - + + - - -|- - -+ + | | + + - - - - - - - - - - - - - - - - + + In fast path ~~~~~~~~~~~~ diff --git a/lib/graph/rte_graph_worker.h b/lib/graph/rte_graph_worker.h index 5b58f7bda9..6685600813 100644 --- a/lib/graph/rte_graph_worker.h +++ b/lib/graph/rte_graph_worker.h @@ -11,6 +11,7 @@ extern "C" { #endif #include "rte_graph_model_rtc.h" +#include "rte_graph_model_mcore_dispatch.h" /** * Perform graph walk on the circular buffer and invoke the process function @@ -25,7 +26,19 @@ __rte_experimental static inline void rte_graph_walk(struct rte_graph *graph) { +#if defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_RTC) rte_graph_walk_rtc(graph); +#elif defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_MCORE_DISPATCH) + rte_graph_walk_mcore_dispatch(graph); +#else + switch (rte_graph_worker_model_no_check_get(graph)) { + case RTE_GRAPH_MODEL_MCORE_DISPATCH: + rte_graph_walk_mcore_dispatch(graph); + break; + default: + rte_graph_walk_rtc(graph); + } +#endif } #ifdef __cplusplus -- 2.37.2