From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id D70B64285E; Wed, 29 Mar 2023 08:43:54 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B22F040EE7; Wed, 29 Mar 2023 08:43:54 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by mails.dpdk.org (Postfix) with ESMTP id D6C1F40EE7 for ; Wed, 29 Mar 2023 08:43:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1680072233; x=1711608233; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=T1fhSKOS7ST98cc8hgUjO55vKLXcZYF4Sb0MNrfY+eY=; b=ZIU5zX+1DFQvZctF+cOXHaOz7KAtme0ompGjcz8rlmo58nCA4nlyPmJ6 J+33i1K6dsn1gAKWPmuRBseJTdptmWp+SBZOIedfXjedNpCJITPuyewTY IPBstybRB3E8uFYDw4K08Kkh1zM81NxjzkRTo4VIPWB3jA2eiclJ+OA7G PA7EHiP9Q8e2DRfVay7OePoV8deBWhc8GKOg3vtFe13M5stOhpAJkZxlG HtSMgJAMNaGEGwn6tQyqW3XHMpaR351RboEHDNClBRWqanjyB/OuZbKVk jBKHbEhqs1ah3syTGn0y6Hi+oaYtajLq5SN5UlKipWdW9kC/n9mDMm4SY Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10663"; a="329260440" X-IronPort-AV: E=Sophos;i="5.98,300,1673942400"; d="scan'208";a="329260440" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2023 23:43:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10663"; a="795104903" X-IronPort-AV: E=Sophos;i="5.98,300,1673942400"; d="scan'208";a="795104903" Received: from dpdk-zhirun-lmm.sh.intel.com ([10.67.119.68]) by fmsmga002.fm.intel.com with ESMTP; 28 Mar 2023 23:43:49 -0700 From: Zhirun Yan To: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, ndabilpuram@marvell.com Cc: cunming.liang@intel.com, haiyue.wang@intel.com, Zhirun Yan Subject: [PATCH v3 00/15] graph enhancement for multi-core dispatch Date: Wed, 29 Mar 2023 15:43:25 +0900 Message-Id: <20230329064340.2550530-1-zhirun.yan@intel.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230324021622.1369006-1-zhirun.yan@intel.com> References: <20230324021622.1369006-1-zhirun.yan@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org V3: Fix CI build issues about TLS and typo. V2: Use git mv to keep git history. Use TLS for per-thread local storage. Change model name to mcore dispatch. Change API with specific mode name. Split big patch. Fix CI issues. Rebase l3fwd-graph example. Update doc and maintainers files. Currently, rte_graph supports RTC (Run-To-Completion) model within each of a single core. RTC is one of the typical model of packet processing. Others like Pipeline or Hybrid are lack of support. The patch set introduces a 'multicore dispatch' model selection which is a self-reacting scheme according to the core affinity. The new model enables a cross-core dispatching mechanism which employs a scheduling work-queue to dispatch streams to other worker cores which being associated with the destination node. When core flavor of the destination node is a default 'current', the stream can be continue executed as normal. Example: 3-node graph targets 3-core budget RTC: Graph: node-0 -> node-1 -> node-2 @Core0. + - - - - - - - - - - - - - - - - - - - - - + ' Core #0/1/2 ' ' ' ' +--------+ +---------+ +--------+ ' ' | Node-0 | --> | Node-1 | --> | Node-2 | ' ' +--------+ +---------+ +--------+ ' ' ' + - - - - - - - - - - - - - - - - - - - - - + Dispatch: Graph topo: node-0 -> Core1; node-1 -> node-2; node-2 -> node-3. Config graph: node-0 @Core0; node-1/3 @Core1; node-2 @Core2. .. code-block:: diff + - - - - - -+ +- - - - - - - - - - - - - + + - - - - - -+ ' Core #0 ' ' Core #1 ' ' Core #2 ' ' ' ' ' ' ' ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' ' | Node-0 | - - - ->| Node-1 | | Node-3 |<- - - - | Node-2 | ' ' +--------+ ' ' +--------+ +--------+ ' ' +--------+ ' ' ' ' | ' ' ^ ' + - - - - - -+ +- - -|- - - - - - - - - - + + - - -|- - -+ | | + - - - - - - - - - - - - - - - - + The patch set has been break down as below: 1. Split graph worker into common and default model part. 2. Inline graph node processing to make it reusable. 3. Add set/get APIs to choose worker model. 4. Introduce core affinity API to set the node run on specific worker core. (only use in new model) 5. Introduce graph affinity API to bind one graph with specific worker core. 6. Introduce graph clone API. 7. Introduce stream moving with scheduler work-queue in patch 8~12. 8. Add stats for new models. 9. Abstract default graph config process and integrate new model into example/l3fwd-graph. Add new parameters for model choosing. We could run with new worker model by this: ./dpdk-l3fwd-graph -l 8,9,10,11 -n 4 -- -p 0x1 --config="(0,0,9)" -P --model="dispatch" References: https://static.sched.com/hosted_files/dpdkuserspace22/a6/graph%20introduce%20remote%20dispatch%20for%20mult-core%20scaling.pdf Zhirun Yan (15): graph: rename rte_graph_work as common graph: split graph worker into common and default model graph: move node process into inline function graph: add get/set graph worker model APIs graph: introduce graph node core affinity API graph: introduce graph bind unbind API graph: introduce graph clone API for other worker core graph: add struct for stream moving between cores graph: introduce stream moving cross cores graph: enable create and destroy graph scheduling workqueue graph: introduce graph walk by cross-core dispatch graph: enable graph multicore dispatch scheduler model graph: add stats for corss-core dispatching examples/l3fwd-graph: introduce multicore dispatch worker model doc: update multicore dispatch model in graph guides MAINTAINERS | 1 + doc/guides/prog_guide/graph_lib.rst | 59 ++- examples/l3fwd-graph/main.c | 237 +++++++++--- lib/graph/graph.c | 179 +++++++++ lib/graph/graph_debug.c | 6 + lib/graph/graph_populate.c | 1 + lib/graph/graph_private.h | 44 +++ lib/graph/graph_stats.c | 74 +++- lib/graph/meson.build | 4 +- lib/graph/node.c | 1 + lib/graph/rte_graph.h | 44 +++ lib/graph/rte_graph_model_dispatch.c | 179 +++++++++ lib/graph/rte_graph_model_dispatch.h | 120 ++++++ lib/graph/rte_graph_model_rtc.h | 45 +++ lib/graph/rte_graph_worker.c | 54 +++ lib/graph/rte_graph_worker.h | 498 +------------------------ lib/graph/rte_graph_worker_common.h | 536 +++++++++++++++++++++++++++ lib/graph/version.map | 8 + 18 files changed, 1546 insertions(+), 544 deletions(-) create mode 100644 lib/graph/rte_graph_model_dispatch.c create mode 100644 lib/graph/rte_graph_model_dispatch.h create mode 100644 lib/graph/rte_graph_model_rtc.h create mode 100644 lib/graph/rte_graph_worker.c create mode 100644 lib/graph/rte_graph_worker_common.h -- 2.37.2