From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9E13642CB8; Wed, 14 Jun 2023 18:15:09 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6B1C942D6C; Wed, 14 Jun 2023 18:13:57 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by mails.dpdk.org (Postfix) with ESMTP id C946C42D5E for ; Wed, 14 Jun 2023 18:13:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1686759232; x=1718295232; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oTbke4D1HvMXlMrv0pmclDH+FYUB/W7nou//5qLmWZE=; b=CLvOhH0qQZ1971zAv6ZqbsM8VqL/1GFqbJKPErwKuuNfsImXM2y+TdzP KbYCxMhGhe5zYqpVBFqgDZfB4IO/fDvNWSwyC6ZWlELGJ+ZoQIItJeKeV JcDBVhkVb60H/e/bx68ExHICgTdDQ11X8xuRb8IkcyfW3JrMBgp6QeJvi ewjU/YLtxK+Bkk0kif7R9aNQp3MnIPuG6whpknXZLkH+T99qGgwLdRg/i QJox9Ww+SI6K7MjT3idS1hvQ5n1V1r+qcYDbff48f5X2EEOSkCVH6saNk tAX3KDEMKjU33V3TL9hYiqh44OuH2SPfDvKOmtRFkaB5EMLUWnG6dgYe3 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10741"; a="358658087" X-IronPort-AV: E=Sophos;i="6.00,243,1681196400"; d="scan'208";a="358658087" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jun 2023 09:08:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10741"; a="662449616" X-IronPort-AV: E=Sophos;i="6.00,243,1681196400"; d="scan'208";a="662449616" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.94]) by orsmga003.jf.intel.com with ESMTP; 14 Jun 2023 09:08:49 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com, stephen@networkplumber.org, pbhagavatula@marvell.com, jerinjacobk@gmail.com, david.marchand@redhat.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, mattias.ronnblom@ericsson.com, Zhirun Yan Subject: [PATCH v15 13/16] graph: enable graph multicore dispatch scheduler model Date: Wed, 14 Jun 2023 23:58:59 +0800 Message-Id: <20230614155902.1530963-14-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230614155902.1530963-1-zhirun.yan@intel.com> References: <20230613140445.2155542-1-zhirun.yan@intel.com> <20230614155902.1530963-1-zhirun.yan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org This patch enables to chose new scheduler model. Must define RTE_GRAPH_MODEL_SELECT before including rte_graph_worker.h to enable specific model choosing. Signed-off-by: Haiyue Wang Signed-off-by: Cunming Liang Signed-off-by: Zhirun Yan Acked-by: Jerin Jacob Acked-by: Pavan Nikhilesh --- doc/guides/prog_guide/graph_lib.rst | 71 ++++++++++++++++++++++++++--- lib/graph/rte_graph_worker.h | 13 ++++++ 2 files changed, 77 insertions(+), 7 deletions(-) diff --git a/doc/guides/prog_guide/graph_lib.rst b/doc/guides/prog_guide/graph_lib.rst index 07248f06c4..6f85e6faef 100644 --- a/doc/guides/prog_guide/graph_lib.rst +++ b/doc/guides/prog_guide/graph_lib.rst @@ -189,13 +189,70 @@ In the above example, A graph object will be created with ethdev Rx node of port 0 and queue 0, all ipv4* nodes in the system, and ethdev tx node of all ports. -Multicore graph processing -~~~~~~~~~~~~~~~~~~~~~~~~~~ -In the current graph library implementation, specifically, -``rte_graph_walk()`` and ``rte_node_enqueue*`` fast path API functions -are designed to work on single-core to have better performance. -The fast path API works on graph object, So the multi-core graph -processing strategy would be to create graph object PER WORKER. +Graph models +~~~~~~~~~~~~ +There are two different kinds of graph walking models. User can select the model using +``rte_graph_worker_model_set()`` API. If the application decides to use only one model, +the fast path check can be avoided by defining the model with RTE_GRAPH_MODEL_SELECT. +For example: + +.. code-block:: console + +#define RTE_GRAPH_MODEL_SELECT RTE_GRAPH_MODEL_RTC +#include "rte_graph_worker.h" + +RTC (Run-To-Completion) +^^^^^^^^^^^^^^^^^^^^^^^ +This is the default graph walking model. Specifically, ``rte_graph_walk_rtc()`` and +``rte_node_enqueue*`` fast path API functions are designed to work on single-core to +have better performance. The fast path API works on graph object, So the multi-core +graph processing strategy would be to create graph object PER WORKER. + +Example: + +Graph: node-0 -> node-1 -> node-2 @Core0. + +.. code-block:: diff + + + - - - - - - - - - - - - - - - - - - - - - + + ' Core #0 ' + ' ' + ' +--------+ +---------+ +--------+ ' + ' | Node-0 | --> | Node-1 | --> | Node-2 | ' + ' +--------+ +---------+ +--------+ ' + ' ' + + - - - - - - - - - - - - - - - - - - - - - + + +Dispatch model +^^^^^^^^^^^^^^ +The dispatch model enables a cross-core dispatching mechanism which employs +a scheduling work-queue to dispatch streams to other worker cores which +being associated with the destination node. + +Use ``rte_graph_model_mcore_dispatch_lcore_affinity_set()`` to set lcore affinity +with the node. +Each worker core will have a graph repetition. Use ``rte_graph_clone()`` to clone +graph for each worker and use``rte_graph_model_mcore_dispatch_core_bind()`` to +bind graph with the worker core. + +Example: + +Graph topo: node-0 -> Core1; node-1 -> node-2; node-2 -> node-3. +Config graph: node-0 @Core0; node-1/3 @Core1; node-2 @Core2. + +.. code-block:: diff + + + - - - - - -+ +- - - - - - - - - - - - - + + - - - - - -+ + ' Core #0 ' ' Core #1 ' ' Core #2 ' + ' ' ' ' ' ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' | Node-0 | - - - ->| Node-1 | | Node-3 |<- - - - | Node-2 | ' + ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' + ' ' ' | ' ' ^ ' + + - - - - - -+ +- - -|- - - - - - - - - - + + - - -|- - -+ + | | + + - - - - - - - - - - - - - - - - + + In fast path ~~~~~~~~~~~~ diff --git a/lib/graph/rte_graph_worker.h b/lib/graph/rte_graph_worker.h index 5b58f7bda9..6685600813 100644 --- a/lib/graph/rte_graph_worker.h +++ b/lib/graph/rte_graph_worker.h @@ -11,6 +11,7 @@ extern "C" { #endif #include "rte_graph_model_rtc.h" +#include "rte_graph_model_mcore_dispatch.h" /** * Perform graph walk on the circular buffer and invoke the process function @@ -25,7 +26,19 @@ __rte_experimental static inline void rte_graph_walk(struct rte_graph *graph) { +#if defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_RTC) rte_graph_walk_rtc(graph); +#elif defined(RTE_GRAPH_MODEL_SELECT) && (RTE_GRAPH_MODEL_SELECT == RTE_GRAPH_MODEL_MCORE_DISPATCH) + rte_graph_walk_mcore_dispatch(graph); +#else + switch (rte_graph_worker_model_no_check_get(graph)) { + case RTE_GRAPH_MODEL_MCORE_DISPATCH: + rte_graph_walk_mcore_dispatch(graph); + break; + default: + rte_graph_walk_rtc(graph); + } +#endif } #ifdef __cplusplus -- 2.37.2