DPDK patches and discussions
 help / color / mirror / Atom feed
From: "Min Hu (Connor)" <humin29@huawei.com>
To: Ajit Khaparde <ajit.khaparde@broadcom.com>
Cc: "Li, Xiaoyun" <xiaoyun.li@intel.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	"Yigit, Ferruh" <ferruh.yigit@intel.com>
Subject: Re: [dpdk-dev] [PATCH v4] app/testpmd: support multi-process
Date: Fri, 26 Mar 2021 14:46:43 +0800
Message-ID: <94e65b67-d7c2-40e9-9565-cecd953d80a5@huawei.com> (raw)
In-Reply-To: <CACZ4nhvLtOEHHROdQ+cTAs5F-dUhJXZ+Wp2y1jrvHJj9CStFqw@mail.gmail.com>



在 2021/3/26 7:25, Ajit Khaparde 写道:
> On Thu, Mar 25, 2021 at 6:32 AM Min Hu (Connor) <humin29@huawei.com> wrote:
>>
>>
>>
>> 在 2021/3/24 16:08, Li, Xiaoyun 写道:
>>> Hi
>>>
>>>> -----Original Message-----
>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Min Hu (Connor)
>>>> Sent: Monday, March 22, 2021 15:07
>>>> To: dev@dpdk.org
>>>> Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; ajit.khaparde@broadcom.com
>>>> Subject: [dpdk-dev] [PATCH v4] app/testpmd: support multi-process
>>>>
>>>> From: Lijun Ou <oulijun@huawei.com>
>>>>
>>>> This patch adds multi-process support for testpmd.
>>>> The test cmd example as follows:
>>>> the primary cmd:
>>>> ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i \
>>>> --rxq=4 --txq=4 --num-procs=2 --proc-id=0
>>>>
>>>> the secondary cmd:
>>>> ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i \
>>>> --rxq=4 --txq=4 --num-procs=2 --proc-id=1
>>>>
>>>> Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
>>>> Signed-off-by: Lijun Ou <oulijun@huawei.com>
>>>> ---
>>>> v4:
>>>> * Fixed minimum vlaue of Rxq or Txq in doc.
>>>>
>>>> v3:
>>>> * Fixed compiling error using gcc10.0.
>>>>
>>>> v2:
>>>> * Added document for this patch.
>>>> ---
>>>>    app/test-pmd/cmdline.c                |  12 ++-
>>>>    app/test-pmd/config.c                 |   9 ++-
>>>>    app/test-pmd/parameters.c             |  11 +++
>>>>    app/test-pmd/testpmd.c                | 138 ++++++++++++++++++++++------------
>>>>    app/test-pmd/testpmd.h                |   7 ++
>>>>    doc/guides/testpmd_app_ug/run_app.rst |  69 +++++++++++++++++
>>>>    6 files changed, 196 insertions(+), 50 deletions(-)
>>>>
>>> <snip>
>>>> +                    if (rte_eal_process_type() == RTE_PROC_PRIMARY)
>>>> +                            rte_mp = rte_pktmbuf_pool_create(pool_name,
>>>> +                                     nb_mbuf, mb_mempool_cache, 0,
>>>> +                                     mbuf_seg_size, heap_socket);
>>>> +                    else
>>>> +                            rte_mp = rte_mempool_lookup(pool_name);
>>>> +
>>>>                       break;
>>>>               }
>>>>       case MP_ALLOC_XBUF:
>>>
>>> What about this one when users use external bufs? Why not addressing secondary process here?
>>> If it works for all cases, you should add a condition at the start of this function, if it's secondary, goto err to check mp and return.
>>>
>> Yes, your are right, I have fixed it in v5, thanks.
>>>> @@ -1994,6 +2013,12 @@ flush_fwd_rx_queues(void)
>>>>       uint64_t prev_tsc = 0, diff_tsc, cur_tsc, timer_tsc = 0;
>>>>       uint64_t timer_period;
>>>>
>>>> +    if (num_procs > 1) {
>>>> +            printf("multi-process not support for flushing fwd rx "
>>>> +                   "queues, skip the below lines and return.\n");
>>>
>>> <snip>
>>>> +uint8_t f_quit;
>>>> +int testpmd_fd_copy;
>>>> +struct cmdline *testpmd_cl;
>>>> +
>>>
>>> Please address the compilation failure on patchwork related to these variables (multiple definitions).
>>>
>> Done in v5.
>>>> +.. code-block:: console
>>>> +
>>>> +    primary process:
>>>> +    sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i --rxq=4
>>>> +--txq=4 --num-procs=2 --proc-id=0
>>>> +
>>>> +    secondary process:
>>>> +    sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i --rxq=4
>>>> +--txq=4 --num-procs=2 --proc-id=1
>>>> +
>>> <snip>
>>>> +*   ``--rxq=N``
>>>> +
>>>> +    Set the number of RX queues per port to N, where 1 <= N <= 65535.
>>>> +    The default value is 1. N is the sum of queues used by primary and secondary
>>>> process.
>>>> +
>>>
>>> Did you upstream wrong patch?
>>> You said you would address the queue number issue Ajit Khaparde mentioned but you didn't in this patch.
>>> The number of queues should be a multiple of the number of processes?
>>>
>> Done in v5.
>>>> +*   ``--txq=N``
>>>> +
>>>> +    Set the number of TX queues per port to N, where 1 <= N <= 65535.
>>>> +    The default value is 1. N is the sum of queues used by primary and secondary
>>>> process.
>>>> +
>>> Same as above.
>>>
>>>> +*   ``--num-procs=N``
>>> <snip>
>>>> +Most dev ops is supported in primary and secondary process. While
>>>> +secondary process is not permitted to allocate or release shared memory, so
>>>> some ops are not supported as follows:
>>>> +``dev_start``
>>>> +``dev_stop``
>>>> +``rx_queue_setup``
>>>> +``tx_queue_setup``
>>>> +``rx_queue_release``
>>>> +``tx_queue_release``
>>>
>>> What about some config commands?
>>> Such as "clear port stats all". Should this be allowed by secondary?
>>   >
>> I think so, actually, all the queues is visible to primary and
>> secondary. The only thing we do is to separate queues for different
>> process for io (packets) in Rx/Tx. It is of for secondary "clear port
>> stats all".
>>> And like "port config all rxq". If primary hasn't started ports, should the secondary allowed to change traffic related stuff (offloads, rx/txd, rx/txq and so on)?
>>>
>> Yes, port config all rxq/txq/rxd/txd/offload is not supported in the
>> secondary process. It has been done in v5.
>>>> +
>>>> +RTE_FLOW supported, it applies only on its own process on SW side, but all on
>>>> HW size.
>>>
>>> About rte flow, what do you mean apply only on its own process on SW side?
>>> If I set number-procs=2, rxq=4
>>> Then on secondary process, I set a flow which directs 192.168.0.1 traffic to queue 0. It seems it will directs this kind of traffic to primary process. But I can't see this rule from primary process side.
>>> Is this behavior right for multiple process?
>>>
>> According to doc rte_flow.rst, we maintain flow rules in process level:
>> primary and secondary has its own flow list(but one flow list in HW).
>> As previously mentioned, the two can see all the queues, so setting the
>> flow rules for the other is OK.
>> Of course, io(receive or transmit packets) in the queue in others is not
>> permitted.
> Can you add this behavior as well to the testpmd doc.
 >
Hi, Ajit, done in v6, thanks.
> Further isolation of resources and operations between the primary and
> secondary processes is possible.
> But this is a good start. We can add more if needed.
> 
>>>> +stats supported, stats will not change when one quit and start, As they share
>>>> the same buffer to store the stats.
>>>> +RSS supported, Primary process and secondary process has separate queues to
>>>> use, RSS will work in their own queues whether primary and secondary process.
>>>> --
>>>> 2.7.4
>>>

  reply	other threads:[~2021-03-26  6:46 UTC|newest]

Thread overview: 76+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-01-08  9:46 [dpdk-dev] [RFC] " Lijun Ou
2021-01-08 10:28 ` Ferruh Yigit
2021-01-09  9:54   ` oulijun
2021-01-10 12:32 ` Wisam Monther
2021-01-12 14:13   ` oulijun
2021-01-12 14:21     ` Wisam Monther
2021-01-14  2:46       ` oulijun
2021-01-20 14:06 ` [dpdk-dev] [RFC V2] " Lijun Ou
2021-03-05  1:04   ` [dpdk-dev] [PATCH] " Lijun Ou
2021-03-05  4:05     ` Ajit Khaparde
2021-03-10 11:11       ` Min Hu (Connor)
2021-03-11  2:47     ` [dpdk-dev] [PATCH v2] " Min Hu (Connor)
2021-03-22  2:27       ` Ajit Khaparde
2021-03-22  6:35         ` Min Hu (Connor)
2021-06-15 12:23       ` [dpdk-dev] [PATCH v14] " Min Hu (Connor)
2021-07-02 12:09       ` [dpdk-dev] [PATCH v15] " Andrew Rybchenko
2021-07-02 12:47         ` Andrew Rybchenko
2021-07-08 12:20           ` Min Hu (Connor)
2021-07-08 12:30             ` Andrew Rybchenko
2021-07-08 12:51               ` Min Hu (Connor)
2021-07-10  3:50       ` [dpdk-dev] [PATCH v16] " Min Hu (Connor)
2021-07-24 11:45         ` Thomas Monjalon
2021-07-26  0:26           ` Min Hu (Connor)
2021-07-26  6:30             ` Thomas Monjalon
2021-07-26  7:28               ` Min Hu (Connor)
2021-08-02  1:51                 ` Min Hu (Connor)
2021-08-02  8:03                   ` Thomas Monjalon
2021-08-16 18:12                     ` Singh, Aman Deep
2021-08-24 12:18                       ` Ferruh Yigit
2021-08-24 13:27                         ` Min Hu (Connor)
2021-08-25  2:06       ` [dpdk-dev] [PATCH v17] " Min Hu (Connor)
2021-09-07 13:23         ` Ferruh Yigit
2021-09-08  0:48           ` Min Hu (Connor)
2021-03-11  9:07     ` [dpdk-dev] [PATCH v3] " Min Hu (Connor)
2021-03-20  0:58       ` Min Hu (Connor)
2021-03-22  7:07     ` [dpdk-dev] [PATCH v4] " Min Hu (Connor)
2021-03-22 11:19       ` Ferruh Yigit
2021-03-24  8:08       ` Li, Xiaoyun
2021-03-25 13:32         ` Min Hu (Connor)
2021-03-25 23:25           ` Ajit Khaparde
2021-03-26  6:46             ` Min Hu (Connor) [this message]
2021-03-25 13:17     ` [dpdk-dev] [PATCH v5] " Min Hu (Connor)
2021-03-26  6:46     ` [dpdk-dev] [PATCH v6] " Min Hu (Connor)
2021-03-26  8:52     ` [dpdk-dev] [PATCH v7] " Min Hu (Connor)
2021-03-29  7:51       ` Li, Xiaoyun
2021-03-30  1:48         ` Min Hu (Connor)
2021-03-30  1:48     ` [dpdk-dev] [PATCH v8] " Min Hu (Connor)
2021-03-30  2:17       ` Li, Xiaoyun
2021-03-30  6:36         ` Min Hu (Connor)
2021-03-30  3:11       ` Ajit Khaparde
2021-03-30  6:41         ` Min Hu (Connor)
2021-03-30 10:19           ` Ferruh Yigit
2021-03-30 10:43             ` Min Hu (Connor)
2021-04-08 10:32               ` Min Hu (Connor)
2021-04-08 13:27                 ` Ferruh Yigit
2021-04-09  0:45                   ` Min Hu (Connor)
2021-04-12 16:37       ` Ferruh Yigit
2021-04-15  7:54         ` Ferruh Yigit
2021-04-16  2:20           ` Min Hu (Connor)
2021-04-16  1:52     ` [dpdk-dev] [PATCH v9] " Min Hu (Connor)
2021-04-16 16:30       ` Ferruh Yigit
2021-04-17  6:12         ` Min Hu (Connor)
2021-04-17  6:12     ` [dpdk-dev] [PATCH v10] " Min Hu (Connor)
2021-04-17 22:21       ` Ferruh Yigit
2021-04-19  1:03         ` Min Hu (Connor)
2021-04-19  1:03     ` [dpdk-dev] [PATCH v11] " Min Hu (Connor)
2021-04-19 13:42       ` Ferruh Yigit
2021-04-21  9:08         ` Min Hu (Connor)
2021-04-21  8:36     ` [dpdk-dev] [PATCH v12] " Min Hu (Connor)
2021-04-22  1:18     ` [dpdk-dev] [PATCH v13] " Min Hu (Connor)
2021-06-08  8:42       ` Andrew Rybchenko
2021-06-08 10:22         ` Thomas Monjalon
2021-06-08 10:39           ` Andrew Rybchenko
2021-06-08 12:02             ` Thomas Monjalon
2021-06-08 12:36             ` Ferruh Yigit
2021-06-15 12:04         ` Min Hu (Connor)

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=94e65b67-d7c2-40e9-9565-cecd953d80a5@huawei.com \
    --to=humin29@huawei.com \
    --cc=ajit.khaparde@broadcom.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=xiaoyun.li@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git