From: Ajit Khaparde <ajit.khaparde@broadcom.com>
To: "Min Hu (Connor)" <humin29@huawei.com>
Cc: "Li, Xiaoyun" <xiaoyun.li@intel.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"Yigit, Ferruh" <ferruh.yigit@intel.com>
Subject: Re: [dpdk-dev] [PATCH v4] app/testpmd: support multi-process
Date: Thu, 25 Mar 2021 16:25:05 -0700 [thread overview]
Message-ID: <CACZ4nhvLtOEHHROdQ+cTAs5F-dUhJXZ+Wp2y1jrvHJj9CStFqw@mail.gmail.com> (raw)
In-Reply-To: <fbaec680-9aa1-b2a8-f21e-306f4606a54d@huawei.com>
[-- Attachment #1: Type: text/plain, Size: 6515 bytes --]
On Thu, Mar 25, 2021 at 6:32 AM Min Hu (Connor) <humin29@huawei.com> wrote:
>
>
>
> 在 2021/3/24 16:08, Li, Xiaoyun 写道:
> > Hi
> >
> >> -----Original Message-----
> >> From: dev <dev-bounces@dpdk.org> On Behalf Of Min Hu (Connor)
> >> Sent: Monday, March 22, 2021 15:07
> >> To: dev@dpdk.org
> >> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; ajit.khaparde@broadcom.com
> >> Subject: [dpdk-dev] [PATCH v4] app/testpmd: support multi-process
> >>
> >> From: Lijun Ou <oulijun@huawei.com>
> >>
> >> This patch adds multi-process support for testpmd.
> >> The test cmd example as follows:
> >> the primary cmd:
> >> ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i \
> >> --rxq=4 --txq=4 --num-procs=2 --proc-id=0
> >>
> >> the secondary cmd:
> >> ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i \
> >> --rxq=4 --txq=4 --num-procs=2 --proc-id=1
> >>
> >> Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
> >> Signed-off-by: Lijun Ou <oulijun@huawei.com>
> >> ---
> >> v4:
> >> * Fixed minimum vlaue of Rxq or Txq in doc.
> >>
> >> v3:
> >> * Fixed compiling error using gcc10.0.
> >>
> >> v2:
> >> * Added document for this patch.
> >> ---
> >> app/test-pmd/cmdline.c | 12 ++-
> >> app/test-pmd/config.c | 9 ++-
> >> app/test-pmd/parameters.c | 11 +++
> >> app/test-pmd/testpmd.c | 138 ++++++++++++++++++++++------------
> >> app/test-pmd/testpmd.h | 7 ++
> >> doc/guides/testpmd_app_ug/run_app.rst | 69 +++++++++++++++++
> >> 6 files changed, 196 insertions(+), 50 deletions(-)
> >>
> > <snip>
> >> + if (rte_eal_process_type() == RTE_PROC_PRIMARY)
> >> + rte_mp = rte_pktmbuf_pool_create(pool_name,
> >> + nb_mbuf, mb_mempool_cache, 0,
> >> + mbuf_seg_size, heap_socket);
> >> + else
> >> + rte_mp = rte_mempool_lookup(pool_name);
> >> +
> >> break;
> >> }
> >> case MP_ALLOC_XBUF:
> >
> > What about this one when users use external bufs? Why not addressing secondary process here?
> > If it works for all cases, you should add a condition at the start of this function, if it's secondary, goto err to check mp and return.
> >
> Yes, your are right, I have fixed it in v5, thanks.
> >> @@ -1994,6 +2013,12 @@ flush_fwd_rx_queues(void)
> >> uint64_t prev_tsc = 0, diff_tsc, cur_tsc, timer_tsc = 0;
> >> uint64_t timer_period;
> >>
> >> + if (num_procs > 1) {
> >> + printf("multi-process not support for flushing fwd rx "
> >> + "queues, skip the below lines and return.\n");
> >
> > <snip>
> >> +uint8_t f_quit;
> >> +int testpmd_fd_copy;
> >> +struct cmdline *testpmd_cl;
> >> +
> >
> > Please address the compilation failure on patchwork related to these variables (multiple definitions).
> >
> Done in v5.
> >> +.. code-block:: console
> >> +
> >> + primary process:
> >> + sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i --rxq=4
> >> +--txq=4 --num-procs=2 --proc-id=0
> >> +
> >> + secondary process:
> >> + sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i --rxq=4
> >> +--txq=4 --num-procs=2 --proc-id=1
> >> +
> > <snip>
> >> +* ``--rxq=N``
> >> +
> >> + Set the number of RX queues per port to N, where 1 <= N <= 65535.
> >> + The default value is 1. N is the sum of queues used by primary and secondary
> >> process.
> >> +
> >
> > Did you upstream wrong patch?
> > You said you would address the queue number issue Ajit Khaparde mentioned but you didn't in this patch.
> > The number of queues should be a multiple of the number of processes?
> >
> Done in v5.
> >> +* ``--txq=N``
> >> +
> >> + Set the number of TX queues per port to N, where 1 <= N <= 65535.
> >> + The default value is 1. N is the sum of queues used by primary and secondary
> >> process.
> >> +
> > Same as above.
> >
> >> +* ``--num-procs=N``
> > <snip>
> >> +Most dev ops is supported in primary and secondary process. While
> >> +secondary process is not permitted to allocate or release shared memory, so
> >> some ops are not supported as follows:
> >> +``dev_start``
> >> +``dev_stop``
> >> +``rx_queue_setup``
> >> +``tx_queue_setup``
> >> +``rx_queue_release``
> >> +``tx_queue_release``
> >
> > What about some config commands?
> > Such as "clear port stats all". Should this be allowed by secondary?
> >
> I think so, actually, all the queues is visible to primary and
> secondary. The only thing we do is to separate queues for different
> process for io (packets) in Rx/Tx. It is of for secondary "clear port
> stats all".
> > And like "port config all rxq". If primary hasn't started ports, should the secondary allowed to change traffic related stuff (offloads, rx/txd, rx/txq and so on)?
> >
> Yes, port config all rxq/txq/rxd/txd/offload is not supported in the
> secondary process. It has been done in v5.
> >> +
> >> +RTE_FLOW supported, it applies only on its own process on SW side, but all on
> >> HW size.
> >
> > About rte flow, what do you mean apply only on its own process on SW side?
> > If I set number-procs=2, rxq=4
> > Then on secondary process, I set a flow which directs 192.168.0.1 traffic to queue 0. It seems it will directs this kind of traffic to primary process. But I can't see this rule from primary process side.
> > Is this behavior right for multiple process?
> >
> According to doc rte_flow.rst, we maintain flow rules in process level:
> primary and secondary has its own flow list(but one flow list in HW).
> As previously mentioned, the two can see all the queues, so setting the
> flow rules for the other is OK.
> Of course, io(receive or transmit packets) in the queue in others is not
> permitted.
Can you add this behavior as well to the testpmd doc.
Further isolation of resources and operations between the primary and
secondary processes is possible.
But this is a good start. We can add more if needed.
> >> +stats supported, stats will not change when one quit and start, As they share
> >> the same buffer to store the stats.
> >> +RSS supported, Primary process and secondary process has separate queues to
> >> use, RSS will work in their own queues whether primary and secondary process.
> >> --
> >> 2.7.4
> >
next prev parent reply other threads:[~2021-03-25 23:25 UTC|newest]
Thread overview: 76+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-08 9:46 [dpdk-dev] [RFC] " Lijun Ou
2021-01-08 10:28 ` Ferruh Yigit
2021-01-09 9:54 ` oulijun
2021-01-10 12:32 ` Wisam Monther
2021-01-12 14:13 ` oulijun
2021-01-12 14:21 ` Wisam Monther
2021-01-14 2:46 ` oulijun
2021-01-20 14:06 ` [dpdk-dev] [RFC V2] " Lijun Ou
2021-03-05 1:04 ` [dpdk-dev] [PATCH] " Lijun Ou
2021-03-05 4:05 ` Ajit Khaparde
2021-03-10 11:11 ` Min Hu (Connor)
2021-03-11 2:47 ` [dpdk-dev] [PATCH v2] " Min Hu (Connor)
2021-03-22 2:27 ` Ajit Khaparde
2021-03-22 6:35 ` Min Hu (Connor)
2021-06-15 12:23 ` [dpdk-dev] [PATCH v14] " Min Hu (Connor)
2021-07-02 12:09 ` [dpdk-dev] [PATCH v15] " Andrew Rybchenko
2021-07-02 12:47 ` Andrew Rybchenko
2021-07-08 12:20 ` Min Hu (Connor)
2021-07-08 12:30 ` Andrew Rybchenko
2021-07-08 12:51 ` Min Hu (Connor)
2021-07-10 3:50 ` [dpdk-dev] [PATCH v16] " Min Hu (Connor)
2021-07-24 11:45 ` Thomas Monjalon
2021-07-26 0:26 ` Min Hu (Connor)
2021-07-26 6:30 ` Thomas Monjalon
2021-07-26 7:28 ` Min Hu (Connor)
2021-08-02 1:51 ` Min Hu (Connor)
2021-08-02 8:03 ` Thomas Monjalon
2021-08-16 18:12 ` Singh, Aman Deep
2021-08-24 12:18 ` Ferruh Yigit
2021-08-24 13:27 ` Min Hu (Connor)
2021-08-25 2:06 ` [dpdk-dev] [PATCH v17] " Min Hu (Connor)
2021-09-07 13:23 ` Ferruh Yigit
2021-09-08 0:48 ` Min Hu (Connor)
2021-03-11 9:07 ` [dpdk-dev] [PATCH v3] " Min Hu (Connor)
2021-03-20 0:58 ` Min Hu (Connor)
2021-03-22 7:07 ` [dpdk-dev] [PATCH v4] " Min Hu (Connor)
2021-03-22 11:19 ` Ferruh Yigit
2021-03-24 8:08 ` Li, Xiaoyun
2021-03-25 13:32 ` Min Hu (Connor)
2021-03-25 23:25 ` Ajit Khaparde [this message]
2021-03-26 6:46 ` Min Hu (Connor)
2021-03-25 13:17 ` [dpdk-dev] [PATCH v5] " Min Hu (Connor)
2021-03-26 6:46 ` [dpdk-dev] [PATCH v6] " Min Hu (Connor)
2021-03-26 8:52 ` [dpdk-dev] [PATCH v7] " Min Hu (Connor)
2021-03-29 7:51 ` Li, Xiaoyun
2021-03-30 1:48 ` Min Hu (Connor)
2021-03-30 1:48 ` [dpdk-dev] [PATCH v8] " Min Hu (Connor)
2021-03-30 2:17 ` Li, Xiaoyun
2021-03-30 6:36 ` Min Hu (Connor)
2021-03-30 3:11 ` Ajit Khaparde
2021-03-30 6:41 ` Min Hu (Connor)
2021-03-30 10:19 ` Ferruh Yigit
2021-03-30 10:43 ` Min Hu (Connor)
2021-04-08 10:32 ` Min Hu (Connor)
2021-04-08 13:27 ` Ferruh Yigit
2021-04-09 0:45 ` Min Hu (Connor)
2021-04-12 16:37 ` Ferruh Yigit
2021-04-15 7:54 ` Ferruh Yigit
2021-04-16 2:20 ` Min Hu (Connor)
2021-04-16 1:52 ` [dpdk-dev] [PATCH v9] " Min Hu (Connor)
2021-04-16 16:30 ` Ferruh Yigit
2021-04-17 6:12 ` Min Hu (Connor)
2021-04-17 6:12 ` [dpdk-dev] [PATCH v10] " Min Hu (Connor)
2021-04-17 22:21 ` Ferruh Yigit
2021-04-19 1:03 ` Min Hu (Connor)
2021-04-19 1:03 ` [dpdk-dev] [PATCH v11] " Min Hu (Connor)
2021-04-19 13:42 ` Ferruh Yigit
2021-04-21 9:08 ` Min Hu (Connor)
2021-04-21 8:36 ` [dpdk-dev] [PATCH v12] " Min Hu (Connor)
2021-04-22 1:18 ` [dpdk-dev] [PATCH v13] " Min Hu (Connor)
2021-06-08 8:42 ` Andrew Rybchenko
2021-06-08 10:22 ` Thomas Monjalon
2021-06-08 10:39 ` Andrew Rybchenko
2021-06-08 12:02 ` Thomas Monjalon
2021-06-08 12:36 ` Ferruh Yigit
2021-06-15 12:04 ` Min Hu (Connor)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACZ4nhvLtOEHHROdQ+cTAs5F-dUhJXZ+Wp2y1jrvHJj9CStFqw@mail.gmail.com \
--to=ajit.khaparde@broadcom.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=humin29@huawei.com \
--cc=xiaoyun.li@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).