From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 09554A0A02; Fri, 26 Mar 2021 07:46:50 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E16F140685; Fri, 26 Mar 2021 07:46:49 +0100 (CET) Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by mails.dpdk.org (Postfix) with ESMTP id 071C54067B for ; Fri, 26 Mar 2021 07:46:48 +0100 (CET) Received: from DGGEMS412-HUB.china.huawei.com (unknown [172.30.72.58]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4F6CBj234nzyNt1; Fri, 26 Mar 2021 14:44:45 +0800 (CST) Received: from [10.67.103.128] (10.67.103.128) by DGGEMS412-HUB.china.huawei.com (10.3.19.212) with Microsoft SMTP Server id 14.3.498.0; Fri, 26 Mar 2021 14:46:43 +0800 To: Ajit Khaparde CC: "Li, Xiaoyun" , "dev@dpdk.org" , "Yigit, Ferruh" References: <1614906276-34293-1-git-send-email-oulijun@huawei.com> <1616396846-19806-1-git-send-email-humin29@huawei.com> From: "Min Hu (Connor)" Message-ID: <94e65b67-d7c2-40e9-9565-cecd953d80a5@huawei.com> Date: Fri, 26 Mar 2021 14:46:43 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.3.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.67.103.128] X-CFilter-Loop: Reflected Subject: Re: [dpdk-dev] [PATCH v4] app/testpmd: support multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 在 2021/3/26 7:25, Ajit Khaparde 写道: > On Thu, Mar 25, 2021 at 6:32 AM Min Hu (Connor) wrote: >> >> >> >> 在 2021/3/24 16:08, Li, Xiaoyun 写道: >>> Hi >>> >>>> -----Original Message----- >>>> From: dev On Behalf Of Min Hu (Connor) >>>> Sent: Monday, March 22, 2021 15:07 >>>> To: dev@dpdk.org >>>> Cc: Yigit, Ferruh ; ajit.khaparde@broadcom.com >>>> Subject: [dpdk-dev] [PATCH v4] app/testpmd: support multi-process >>>> >>>> From: Lijun Ou >>>> >>>> This patch adds multi-process support for testpmd. >>>> The test cmd example as follows: >>>> the primary cmd: >>>> ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i \ >>>> --rxq=4 --txq=4 --num-procs=2 --proc-id=0 >>>> >>>> the secondary cmd: >>>> ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i \ >>>> --rxq=4 --txq=4 --num-procs=2 --proc-id=1 >>>> >>>> Signed-off-by: Min Hu (Connor) >>>> Signed-off-by: Lijun Ou >>>> --- >>>> v4: >>>> * Fixed minimum vlaue of Rxq or Txq in doc. >>>> >>>> v3: >>>> * Fixed compiling error using gcc10.0. >>>> >>>> v2: >>>> * Added document for this patch. >>>> --- >>>> app/test-pmd/cmdline.c | 12 ++- >>>> app/test-pmd/config.c | 9 ++- >>>> app/test-pmd/parameters.c | 11 +++ >>>> app/test-pmd/testpmd.c | 138 ++++++++++++++++++++++------------ >>>> app/test-pmd/testpmd.h | 7 ++ >>>> doc/guides/testpmd_app_ug/run_app.rst | 69 +++++++++++++++++ >>>> 6 files changed, 196 insertions(+), 50 deletions(-) >>>> >>> >>>> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) >>>> + rte_mp = rte_pktmbuf_pool_create(pool_name, >>>> + nb_mbuf, mb_mempool_cache, 0, >>>> + mbuf_seg_size, heap_socket); >>>> + else >>>> + rte_mp = rte_mempool_lookup(pool_name); >>>> + >>>> break; >>>> } >>>> case MP_ALLOC_XBUF: >>> >>> What about this one when users use external bufs? Why not addressing secondary process here? >>> If it works for all cases, you should add a condition at the start of this function, if it's secondary, goto err to check mp and return. >>> >> Yes, your are right, I have fixed it in v5, thanks. >>>> @@ -1994,6 +2013,12 @@ flush_fwd_rx_queues(void) >>>> uint64_t prev_tsc = 0, diff_tsc, cur_tsc, timer_tsc = 0; >>>> uint64_t timer_period; >>>> >>>> + if (num_procs > 1) { >>>> + printf("multi-process not support for flushing fwd rx " >>>> + "queues, skip the below lines and return.\n"); >>> >>> >>>> +uint8_t f_quit; >>>> +int testpmd_fd_copy; >>>> +struct cmdline *testpmd_cl; >>>> + >>> >>> Please address the compilation failure on patchwork related to these variables (multiple definitions). >>> >> Done in v5. >>>> +.. code-block:: console >>>> + >>>> + primary process: >>>> + sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 0-1 -- -i --rxq=4 >>>> +--txq=4 --num-procs=2 --proc-id=0 >>>> + >>>> + secondary process: >>>> + sudo ./dpdk-testpmd -a xxx --proc-type=auto -l 2-3 -- -i --rxq=4 >>>> +--txq=4 --num-procs=2 --proc-id=1 >>>> + >>> >>>> +* ``--rxq=N`` >>>> + >>>> + Set the number of RX queues per port to N, where 1 <= N <= 65535. >>>> + The default value is 1. N is the sum of queues used by primary and secondary >>>> process. >>>> + >>> >>> Did you upstream wrong patch? >>> You said you would address the queue number issue Ajit Khaparde mentioned but you didn't in this patch. >>> The number of queues should be a multiple of the number of processes? >>> >> Done in v5. >>>> +* ``--txq=N`` >>>> + >>>> + Set the number of TX queues per port to N, where 1 <= N <= 65535. >>>> + The default value is 1. N is the sum of queues used by primary and secondary >>>> process. >>>> + >>> Same as above. >>> >>>> +* ``--num-procs=N`` >>> >>>> +Most dev ops is supported in primary and secondary process. While >>>> +secondary process is not permitted to allocate or release shared memory, so >>>> some ops are not supported as follows: >>>> +``dev_start`` >>>> +``dev_stop`` >>>> +``rx_queue_setup`` >>>> +``tx_queue_setup`` >>>> +``rx_queue_release`` >>>> +``tx_queue_release`` >>> >>> What about some config commands? >>> Such as "clear port stats all". Should this be allowed by secondary? >> > >> I think so, actually, all the queues is visible to primary and >> secondary. The only thing we do is to separate queues for different >> process for io (packets) in Rx/Tx. It is of for secondary "clear port >> stats all". >>> And like "port config all rxq". If primary hasn't started ports, should the secondary allowed to change traffic related stuff (offloads, rx/txd, rx/txq and so on)? >>> >> Yes, port config all rxq/txq/rxd/txd/offload is not supported in the >> secondary process. It has been done in v5. >>>> + >>>> +RTE_FLOW supported, it applies only on its own process on SW side, but all on >>>> HW size. >>> >>> About rte flow, what do you mean apply only on its own process on SW side? >>> If I set number-procs=2, rxq=4 >>> Then on secondary process, I set a flow which directs 192.168.0.1 traffic to queue 0. It seems it will directs this kind of traffic to primary process. But I can't see this rule from primary process side. >>> Is this behavior right for multiple process? >>> >> According to doc rte_flow.rst, we maintain flow rules in process level: >> primary and secondary has its own flow list(but one flow list in HW). >> As previously mentioned, the two can see all the queues, so setting the >> flow rules for the other is OK. >> Of course, io(receive or transmit packets) in the queue in others is not >> permitted. > Can you add this behavior as well to the testpmd doc. > Hi, Ajit, done in v6, thanks. > Further isolation of resources and operations between the primary and > secondary processes is possible. > But this is a good start. We can add more if needed. > >>>> +stats supported, stats will not change when one quit and start, As they share >>>> the same buffer to store the stats. >>>> +RSS supported, Primary process and secondary process has separate queues to >>>> use, RSS will work in their own queues whether primary and secondary process. >>>> -- >>>> 2.7.4 >>>