From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3C2BCA00C3; Tue, 20 Sep 2022 11:33:33 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1563040DFB; Tue, 20 Sep 2022 11:33:33 +0200 (CEST) Received: from mail-qk1-f176.google.com (mail-qk1-f176.google.com [209.85.222.176]) by mails.dpdk.org (Postfix) with ESMTP id 4865A4069B for ; Tue, 20 Sep 2022 11:33:32 +0200 (CEST) Received: by mail-qk1-f176.google.com with SMTP id 3so1189490qka.5 for ; Tue, 20 Sep 2022 02:33:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date; bh=pu2WXk+K1eJDZyYGUATJ6wJumtrbJYNGptyZI8w/ApQ=; b=eZNwhqvrGydVgL2s2hhI+d4DeQW514TIIOxCicbjlvUtQrjPTcH8qq+FMBIyYT7bB0 1zRV+IJSfDr8t/wjED2DzxSj8veFjWAyXGEAvtZUHvZWFnW/PDZJTtv+HMpf5DuU9Bcl QG6rO8KEXmyX3sNJzG3LwcFWE49dQd0uijTSgk3uGf0010uy3ILsGYnSSndKiqnk3D+F TtML+3jYzToOWD9KQ+2EzMHsgs188IIKvEB41Ks1nnJzyliOIZ81oS8OJxSpUT50S21R RpABxpb5Vlbib4Jcx16Jyo2qWxVx0yyectvpBcIEJRprUsPFj02hjjPhLDE1u1/BgsD8 h4xQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date; bh=pu2WXk+K1eJDZyYGUATJ6wJumtrbJYNGptyZI8w/ApQ=; b=biOxuwlkSzBDkg45ZDERN3WTX4KvkOxJj4UcIyFiHXSOrReIImUJZZQoh1iPMi8VRZ 7L1GVrFDhxyZZYisHt49fiFH9Bieu9TxlMQy6VOXB3xaMPeM49iYpejzdeftbzgoeovi STginan5elI6mJeq6yyLSjqslPNKMRzbmJNzNykT5DUROzZBvMWGrUQK48A5dTQcsQyl 5126e2MJmkLkUq8ZSK5TmOjVpQXpnguXlgtW0OVLjEmUvAUqhhPXyCvCjSy7rc5K/IvN qkaqHdZ9qKvok2kpZqPR7KNlJFCn7NrTnCQKozAL4H3p17TSnFzYysYsfkhc4DQ00aBE WcLQ== X-Gm-Message-State: ACrzQf3qGho3oKoUXhGqSpEPMEoNLcMnCk1N2UgrEM6OS/OxCAKN9WGM aW7pTk/IYYqTZtx0NxdTdmK17cOhmhDBRqqaC6c= X-Google-Smtp-Source: AMsMyM61wID+iyN9ezl7gdcPqjKQLswNH3IpgbJ8S4zUp1okZZ4h0qZqaJkiXj+k7Z2JQNNIuIrIe6DhxN3eXXboxyo= X-Received: by 2002:a05:620a:4248:b0:6be:7848:3ce5 with SMTP id w8-20020a05620a424800b006be78483ce5mr15741805qko.26.1663666411601; Tue, 20 Sep 2022 02:33:31 -0700 (PDT) MIME-Version: 1.0 References: <20220908020959.1675953-1-zhirun.yan@intel.com> In-Reply-To: <20220908020959.1675953-1-zhirun.yan@intel.com> From: Jerin Jacob Date: Tue, 20 Sep 2022 15:03:05 +0530 Message-ID: Subject: Re: [RFC, v1 0/6] graph enhancement for multi-core dispatch To: Zhirun Yan Cc: dev@dpdk.org, jerinj@marvell.com, kirankumark@marvell.com, cunming.liang@intel.com, haiyue.wang@intel.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Thu, Sep 8, 2022 at 7:40 AM Zhirun Yan wrote: > > Currently, the rte_graph_walk() and rte_node_enqueue* fast path API > functions in graph lib implementation are designed to work on single-core= . > > This solution(RFC) proposes usage of cross-core dispatching mechanism to > enhance the graph scaling strategy. We introduce Scheduler Workqueue > then we could directly dispatch streams to another worker core which is > affinity with a specific node. > > This RFC=EF=BC=9A > 1. Introduce core affinity API and graph clone API. > 2. Introduce key functions to enqueue/dequeue for dispatching streams. > 3. Enhance rte_graph_walk by cross-core dispatch. > 4. Add l2fwd-graph example and stats for cross-core dispatching. > > With this patch set, it could easily plan and orchestrate stream on > multi-core systems. > > Future work: > 1. Support to affinity lcore set for one node. > 2. Use l3fwd-graph instead of l2fwd-graph as example in patch 06. > 3. Add new parameter, like --node(nodeid, lcoreid) to config node for c= ore > affinity. > > Comments and suggestions are welcome. Thanks! Some top level comments. 1)Yes it makes sense to not create the l2fwd-graph, Please enhance the l3fwd-graph and compare the performance with multi core scenarios. 2) It is good to have multiple graph walk schemes like the one you have introduced now. Though I am not sure about performance aspects, specifically, it is used with multiple producers and multi consumers with node. If you have a use case for the new worker scheme, then we can add it. I think, it would call for a) We need to have separate rte_graph_worker.h for each implementation to avoid the performance impact for each other. That may boils down to i) Create lib/graph/rte_graph_worker_common.h ii) Treat existing rte_graph_worker.h as default scheme and include rte_graph_worker_common.h iii) Add new rte_graph_worker_xxx.h for the new scheme(diff between default worker) with leveraging te_graph_worker_common.h Application can select the worker by #define RTE_GRAPH_WORKER_MODEL_XXX //#define RTE_GRAPH_WORKER_MODEL_YYY #include b) Introduce a new enum rte_graph_model or so to express this new model and other models in feature c) Each core has its own node instance so we don't need explicit critical section management when dealing with node instances. In this new scheme, Can we leverage the existing node implementation? If not, we need to have separate node implementation for different graph models. It will be a maintenance issue. But if we really need to take this path, Probably on each node's capability, the node needs to declare the models supported(Use enum rte_graph_model). This can be used for sanity checking when we clone the graph etc and check the compatibility for creating the graph etc. I think this is the biggest issue with adding a new model. Where nodes need to be written based on the model. I think this could be the reason for VPP not adding other models. d) All new slowpath APIs like rte_node_set_lcore_affinity, rte_graph_clone, We need to fix the namespace by rte_graph_model__ or so to make sure application writer understand this APIs are only for this model.(Also we can use "enum rte_graph_model" for sanity check etc) >