From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 194CB42C3A; Tue, 6 Jun 2023 07:55:53 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E46FE40A84; Tue, 6 Jun 2023 07:55:52 +0200 (CEST) Received: from mail-vs1-f45.google.com (mail-vs1-f45.google.com [209.85.217.45]) by mails.dpdk.org (Postfix) with ESMTP id 784EF406B7 for ; Tue, 6 Jun 2023 07:55:51 +0200 (CEST) Received: by mail-vs1-f45.google.com with SMTP id ada2fe7eead31-43b45a0ae19so510007137.0 for ; Mon, 05 Jun 2023 22:55:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686030951; x=1688622951; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=3jtHCtIgVYvKtMcSkfZPeTlY1iNBgfSeVmlVFovA+7Q=; b=B+zabbX1Ke01vFFoZEliyB4CG4f72uV9h7+i9GLO4YNXbVSjMhO/xGnvwHXfhC14Ja RsTTYg1y/SrLnUzIhrErKQR0teXwlSe+oEuBBX3xFkoxN3lP8VnF+VAwe4KwEOxE9CLj a5lbyKLT7lO34lS5zemCPuTSXFPFur9HiqYNOom2K92Pp9gupjUZeh0l51T81fzuEDBW +ML5EgAtUJAGndbbnnj4Y2n6AOrnN/swJRxFBO1eVAsDdUx8OPNSbGx99GpBUYRbRKSM gS4xqKiTwCFYyZpfPpT5vrsAak6MR3+99r7n5AnNqeqwZ+f1j6g3RICwx8Lef1+rAZ2r tIzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686030951; x=1688622951; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3jtHCtIgVYvKtMcSkfZPeTlY1iNBgfSeVmlVFovA+7Q=; b=BKHuF5oIu1/SHtruqk9klFQeQc7slpyVP3o+XpI/7qDNBsbJVKdBFuP4I/iezkwmyV oUziMrrenzbS8A79ex5QAEdjbhhNgHLOPKo7XYFcDMU4gO+jAuX+YEN5yWMuAZtSvlhA J8hv8HeZUMP1lu2ie4+1fSvSZ8KPWLRy8D98yEmAxqrpctHqkwf+3KQ727lfSwX7IETh H8cjXjpErlhO8JWPUCpnwODsnLnPS8KFlwBlCErxn4aW6ulLNMROv5Gjdhc/aK8dOIN8 7TY/2KP9kHMHEmDHX+jUlqmh4EIb7quvllqf+ppNRVS0WY8IHm7XVy6BZO27oUJcsVQ7 JN3Q== X-Gm-Message-State: AC+VfDy5x5V49IpePPFNMF7Rp9yPGFsKCtZEIZ6xhmkNhWkPxVhTOXwt ZwyOwrwNsFg2XFMYImz8odvSnS2dmWRTM9hkixNsqwBLgZEOZpW4 X-Google-Smtp-Source: ACHHUZ6J2kgL1ikr2V3tirRvkj20gYFtBJNgfC8WEuktpGqCh6qXjsfrkaKer0/pqgIpJvFVTYWwOeqJL9JiQdvje3M= X-Received: by 2002:a67:f997:0:b0:42c:922e:65dd with SMTP id b23-20020a67f997000000b0042c922e65ddmr669688vsq.23.1686030950743; Mon, 05 Jun 2023 22:55:50 -0700 (PDT) MIME-Version: 1.0 References: <20230509060347.1237884-1-zhirun.yan@intel.com> <20230605111923.3772260-1-zhirun.yan@intel.com> <20230605111923.3772260-16-zhirun.yan@intel.com> In-Reply-To: From: Jerin Jacob Date: Tue, 6 Jun 2023 11:25:24 +0530 Message-ID: Subject: Re: [PATCH v7 15/17] examples/l3fwd-graph: introduce multicore dispatch worker model To: "Yan, Zhirun" Cc: "dev@dpdk.org" , "jerinj@marvell.com" , "kirankumark@marvell.com" , "ndabilpuram@marvell.com" , "stephen@networkplumber.org" , "pbhagavatula@marvell.com" , "Liang, Cunming" , "Wang, Haiyue" , "mattias.ronnblom" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Jun 6, 2023 at 10:41=E2=80=AFAM Yan, Zhirun = wrote: > > > > > -----Original Message----- > > From: Jerin Jacob > > Sent: Monday, June 5, 2023 9:42 PM > > To: Yan, Zhirun > > Cc: dev@dpdk.org; jerinj@marvell.com; kirankumark@marvell.com; > > ndabilpuram@marvell.com; stephen@networkplumber.org; > > pbhagavatula@marvell.com; Liang, Cunming ; Wan= g, > > Haiyue ; mattias.ronnblom > > > > Subject: Re: [PATCH v7 15/17] examples/l3fwd-graph: introduce multicore > > dispatch worker model > > > > On Mon, Jun 5, 2023 at 4:57=E2=80=AFPM Zhirun Yan wrote: > > > > > > Add new parameter "model" to choose mcore dispatch or rtc model. > > > And in dispatch model, the node will affinity to worker core successi= vely. > > > > > > Note: > > > RTE_GRAPH_MODEL_SELECT is set to GRAPH_MODEL_RTC by default. Must > > set > > > model the same as RTE_GRAPH_MODEL_SELECT If set it as rtc or mcore > > > dispatch explicitly. GRAPH_MODEL_MCORE_RUNTIME_SELECT means it could > > > choose by model in runtime. > > > Only support one RX node for mcore dispatch model in current > > > implementation. > > > > > > ./dpdk-l3fwd-graph -l 8,9,10,11 -n 4 -- -p 0x1 --config=3D"(0,0,9)" = -P > > > --model=3D"dispatch" > > > > > > Signed-off-by: Haiyue Wang > > > Signed-off-by: Cunming Liang > > > Signed-off-by: Zhirun Yan > > > --- > > > examples/l3fwd-graph/main.c | 231 +++++++++++++++++++++++++++++----= -- > > > lib/graph/rte_graph_worker.h | 3 + > > > 2 files changed, 196 insertions(+), 38 deletions(-) > > > > > > diff --git a/examples/l3fwd-graph/main.c b/examples/l3fwd-graph/main.= c > > > index 5feeab4f0f..4ecc6c9af4 100644 > > > --- a/examples/l3fwd-graph/main.c > > > +++ b/examples/l3fwd-graph/main.c > > > @@ -23,6 +23,12 @@ > > > #include > > > #include > > > #include > > > +#define GRAPH_MODEL_RTC 0 /* Run-to-completion model, set by default= . > > > +*/ #define GRAPH_MODEL_MCORE_DISPATCH 1 /* Dispatch model. */ > > #define > > > +GRAPH_MODEL_MCORE_RUNTIME_SELECT 2 /* Support to select model by > > */ > > > + /* parsing model in > > > +cmdline. */ > > > > After moving model to graph->model, Can you check the performance. > > In my env, I test l3fwd-graph, I got the same throughput.(slight improve = could be treated as jitter) > For graph_perf_autotest in test app, there is slight drop (About 0.2% cal= l, similar cycles/call) > Can it be treated as jitter? Most likely. Try in following in fasth path. const ... model =3D graph->model; > > Old: > +-------------------------------+---------------+---------------+--------= -------+---------------+---------------+-----------+ > |Node |calls |objs |realloc_= count |objs/call |objs/sec(10E6) |cycles/call| > +-------------------------------+---------------+---------------+--------= -------+---------------+---------------+-----------+ > |test_graph_perf_worker-0-0 |10175176 |2604845056 |1 = |256.000 |2015.394304 |27.0000 | > |test_graph_perf_worker-1-0 |10175542 |2604938752 |1 = |256.000 |2015.488000 |28.0000 | > |test_graph_perf_worker-2-0 |10175565 |2604944640 |1 = |256.000 |2015.493888 |28.0000 | > |test_graph_perf_worker-3-0 |10175593 |2604951808 |1 = |256.000 |2015.501056 |27.0000 | > |test_graph_perf_source-0 |10175623 |2604959488 |2 = |256.000 |2015.508480 |27.0000 | > |test_graph_perf_sink-0 |10175642 |2604964352 |1 = |256.000 |2015.513600 |27.0000 | > +-------------------------------+---------------+---------------+--------= -------+---------------+---------------+-----------+ > > New: > +-------------------------------+---------------+---------------+--------= -------+---------------+---------------+-----------+ > |Node |calls |objs |realloc_= count |objs/call |objs/sec(10E6) |cycles/call| > +-------------------------------+---------------+---------------+--------= -------+---------------+---------------+-----------+ > |test_graph_perf_worker-0-0 |10154953 |2599667968 |1 = |256.000 |2010.960128 |27.0000 | > |test_graph_perf_worker-1-0 |10155316 |2599760896 |1 = |256.000 |2011.053056 |27.0000 | > |test_graph_perf_worker-2-0 |10155338 |2599766528 |1 = |256.000 |2011.058688 |28.0000 | > |test_graph_perf_worker-3-0 |10155357 |2599771392 |1 = |256.000 |2011.063552 |28.0000 | > |test_graph_perf_source-0 |10155394 |2599780864 |2 = |256.000 |2011.072768 |27.0000 | > |test_graph_perf_sink-0 |10155422 |2599788032 |1 = |256.000 |2011.080192 |27.0000 | > +-------------------------------+---------------+---------------+--------= -------+---------------+---------------+-----------+ > > > This may not be needed for l3fwd > > > Do you mean graph->model? Yes. > > > or if there is not much code duplication, > > > > Do the following remove the limitation, > > #define RTE_GRAPH_MODEL_SELECT RTE_GRAPH_MODEL_RTC. > > > > graph_main_loop change to graph_main_rtc_loop > > > > #define RTE_GRAPH_MODEL_SELECT GRAPH_MODEL_MCORE_DISPATCH > > > > graph_main_loop change to graph_main_mcore_loop > > > > Select the following based on runtime option > > /* Launch per-lcore init on every worker lcore */ > > rte_eal_mp_remote_launch(graph_main_rtc_loop, NULL, SKIP_MAIN);= or > > rte_eal_mp_remote_launch(graph_main_mcore_loop, NULL, SKIP_MAIN= ); > > > > We want to 1. Use same API (rte_graph_walk()) for diff models. > 2. no performance drop for rtc (use RTE_GRAPH_MODEL_SELECT in compile tim= e) > > If I understand correctly, I need remove graph->model and only use > RTE_GRAPH_MODEL_SELECT to select models? > > And change it as > graph_main_rtc_loop() > { > While(1) > rte_graph_walk_rtc() > } > > But actually, I think graph->model is need, especially for config stage a= nd for runtime config > If set RTE_GRAPH_MODEL_SELECT_RUNTIME. Yes. Agree. If there is no MAJOR performance issues lets use RTE_GRAPH_MODEL_SELECT_RUNTIME for l3fwd. > We need the model type to decide to alloc workqueue and use RTE_GRAPH_MOD= EL_SELECT > to choose the walk. > > > > > memset(&rewrite_data, 0, sizeof(rewrite_data)); > > > rewrite_len =3D sizeof(rewrite_data); diff --git > > > a/lib/graph/rte_graph_worker.h b/lib/graph/rte_graph_worker.h index > > > 541c373cb1..19b4c1514f 100644 > > > --- a/lib/graph/rte_graph_worker.h > > > +++ b/lib/graph/rte_graph_worker.h > > > @@ -26,6 +26,9 @@ __rte_experimental > > > static inline void > > > rte_graph_walk(struct rte_graph *graph) { > > > +#define RTE_GRAPH_MODEL_RTC 0 > > > +#define RTE_GRAPH_MODEL_MCORE_DISPATCH 1 > > > > No need for duplicate enum. Please remove enum make this as in public = header > > file. > > > Yes, it will cause no defined warnings. > Thanks for your comments. > I will remove enum and define model type macros in public header. And als= o change > the related structs/APIs. Also add a comment in RTE_GRAPH_MODEL_MCORE_DISPATCH, If adding new entry, then update graph_is_valid API. > > > > > > + > > > > Add comment here, On how application uses this, aka. before inlcuding= the > > worker header file #define RTE_GRAPH_MODEL_SELECT > > RTE_GRAPH_MODEL_RTC. > > Please change the text as needed. > Yes, I will add comment and add the usage in document. > > > > > > > > #if !defined(RTE_GRAPH_MODEL_SELECT) || RTE_GRAPH_MODEL_SELECT =3D= =3D > > RTE_GRAPH_MODEL_RTC > > > rte_graph_walk_rtc(graph); > > > #elif RTE_GRAPH_MODEL_SELECT =3D=3D > > RTE_GRAPH_MODEL_MCORE_DISPATCH > > > -- > > > 2.37.2 > > >