From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id A4F52A0A02; Fri, 26 Mar 2021 00:25:23 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 319F940685; Fri, 26 Mar 2021 00:25:23 +0100 (CET) Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) by mails.dpdk.org (Postfix) with ESMTP id 6C5444067B for ; Fri, 26 Mar 2021 00:25:22 +0100 (CET) Received: by mail-qv1-f51.google.com with SMTP id cx5so2077476qvb.10 for ; Thu, 25 Mar 2021 16:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ICj+ZpXiLP8/nqfGB7fyHjiZl8oNa/P5WLbnV/IqD78=; b=AutwCdlFBB+Hx3QnZg7M/hDLEhT+ap8kfTtyNbOKmiUq+mk145z3KMnWwYBxqIP+t4 PHu548+nADSuMknSrahuiswFxSbGqzExuXyDzSt07Tu1Iz/ZA0lE78i7079kKr8WG0Eo 0cuP9oxzx+NBbO4CI7+MyfWNxmNVIzBU1dNDc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ICj+ZpXiLP8/nqfGB7fyHjiZl8oNa/P5WLbnV/IqD78=; b=qyrdFWrTXnx0Q1VJneZa1FFNIm1gMfYkEnBoBVlg6vX75ApFGDfpSsQrmLfOCfoc8Y hkwp9r8VoUo8oepT08t/l361VBQkY0Y+oiFdqgcyX/mh6FMbifuwmO6AqQmHJW0DHDH6 cKq+UDAGZSJWpzPO0Y3MyhUz51+RaC4BD6eCeomBIIw8FlEFuBIrhj1Ey8+mcB67odQG eVCS4FauViPLSgbq1Pq1kfwZrTKUJAJgJ38YwZlEX+lcCoMlT5E+pKHB+ExccRwbyqde t20OJhSNHdjaHyZKZhZyhx47MAPcoHyd1MOVlt10M7+V9Zbcf5aLn4jyRM7TGPBRCVkN v94A== X-Gm-Message-State: AOAM531KP53Lik7zD4lBxWZGECBRBnS8AZ7oKvI9ZYFkhE2YCMWQJHzd yaKb3gujcwIxl9G+4CjbWjNl6Wzxy+TR1Sk0K1WT+w== X-Google-Smtp-Source: ABdhPJzBD+96ZwPUKRyr1vaYrNmk5w5uyG016D3KdcnwBo2r/H172YysUD6/WTUnRJcTvl2vTGyhKSDRRrF4pjAKub0= X-Received: by 2002:a05:6214:184e:: with SMTP id d14mr10711609qvy.30.1616714721666; Thu, 25 Mar 2021 16:25:21 -0700 (PDT) MIME-Version: 1.0 References: <1614906276-34293-1-git-send-email-oulijun@huawei.com> <1616396846-19806-1-git-send-email-humin29@huawei.com> In-Reply-To: From: Ajit Khaparde Date: Thu, 25 Mar 2021 16:25:05 -0700 Message-ID: To: "Min Hu (Connor)" Cc: "Li, Xiaoyun" , "dev@dpdk.org" , "Yigit, Ferruh" Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="000000000000b53d8c05be64ba45" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [dpdk-dev] [PATCH v4] app/testpmd: support multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" --000000000000b53d8c05be64ba45 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Mar 25, 2021 at 6:32 AM Min Hu (Connor) wrote: > > > > =E5=9C=A8 2021/3/24 16:08, Li, Xiaoyun =E5=86=99=E9=81=93: > > Hi > > > >> -----Original Message----- > >> From: dev On Behalf Of Min Hu (Connor) > >> Sent: Monday, March 22, 2021 15:07 > >> To: dev@dpdk.org > >> Cc: Yigit, Ferruh ; ajit.khaparde@broadcom.com > >> Subject: [dpdk-dev] [PATCH v4] app/testpmd: support multi-process > >> > >> From: Lijun Ou > >> > >> This patch adds multi-process support for testpmd. > >> The test cmd example as follows: > >> the primary cmd: > >> ./dpdk-testpmd -a xxx --proc-type=3Dauto -l 0-1 -- -i \ > >> --rxq=3D4 --txq=3D4 --num-procs=3D2 --proc-id=3D0 > >> > >> the secondary cmd: > >> ./dpdk-testpmd -a xxx --proc-type=3Dauto -l 2-3 -- -i \ > >> --rxq=3D4 --txq=3D4 --num-procs=3D2 --proc-id=3D1 > >> > >> Signed-off-by: Min Hu (Connor) > >> Signed-off-by: Lijun Ou > >> --- > >> v4: > >> * Fixed minimum vlaue of Rxq or Txq in doc. > >> > >> v3: > >> * Fixed compiling error using gcc10.0. > >> > >> v2: > >> * Added document for this patch. > >> --- > >> app/test-pmd/cmdline.c | 12 ++- > >> app/test-pmd/config.c | 9 ++- > >> app/test-pmd/parameters.c | 11 +++ > >> app/test-pmd/testpmd.c | 138 ++++++++++++++++++++++--= ---------- > >> app/test-pmd/testpmd.h | 7 ++ > >> doc/guides/testpmd_app_ug/run_app.rst | 69 +++++++++++++++++ > >> 6 files changed, 196 insertions(+), 50 deletions(-) > >> > > > >> + if (rte_eal_process_type() =3D=3D RTE_PROC_PRIMAR= Y) > >> + rte_mp =3D rte_pktmbuf_pool_create(pool_n= ame, > >> + nb_mbuf, mb_mempool_cache, 0, > >> + mbuf_seg_size, heap_socket); > >> + else > >> + rte_mp =3D rte_mempool_lookup(pool_name); > >> + > >> break; > >> } > >> case MP_ALLOC_XBUF: > > > > What about this one when users use external bufs? Why not addressing se= condary process here? > > If it works for all cases, you should add a condition at the start of t= his function, if it's secondary, goto err to check mp and return. > > > Yes, your are right, I have fixed it in v5, thanks. > >> @@ -1994,6 +2013,12 @@ flush_fwd_rx_queues(void) > >> uint64_t prev_tsc =3D 0, diff_tsc, cur_tsc, timer_tsc =3D 0; > >> uint64_t timer_period; > >> > >> + if (num_procs > 1) { > >> + printf("multi-process not support for flushing fwd rx " > >> + "queues, skip the below lines and return.\n"); > > > > > >> +uint8_t f_quit; > >> +int testpmd_fd_copy; > >> +struct cmdline *testpmd_cl; > >> + > > > > Please address the compilation failure on patchwork related to these va= riables (multiple definitions). > > > Done in v5. > >> +.. code-block:: console > >> + > >> + primary process: > >> + sudo ./dpdk-testpmd -a xxx --proc-type=3Dauto -l 0-1 -- -i --rxq= =3D4 > >> +--txq=3D4 --num-procs=3D2 --proc-id=3D0 > >> + > >> + secondary process: > >> + sudo ./dpdk-testpmd -a xxx --proc-type=3Dauto -l 2-3 -- -i --rxq= =3D4 > >> +--txq=3D4 --num-procs=3D2 --proc-id=3D1 > >> + > > > >> +* ``--rxq=3DN`` > >> + > >> + Set the number of RX queues per port to N, where 1 <=3D N <=3D 65= 535. > >> + The default value is 1. N is the sum of queues used by primary an= d secondary > >> process. > >> + > > > > Did you upstream wrong patch? > > You said you would address the queue number issue Ajit Khaparde mention= ed but you didn't in this patch. > > The number of queues should be a multiple of the number of processes? > > > Done in v5. > >> +* ``--txq=3DN`` > >> + > >> + Set the number of TX queues per port to N, where 1 <=3D N <=3D 65= 535. > >> + The default value is 1. N is the sum of queues used by primary an= d secondary > >> process. > >> + > > Same as above. > > > >> +* ``--num-procs=3DN`` > > > >> +Most dev ops is supported in primary and secondary process. While > >> +secondary process is not permitted to allocate or release shared memo= ry, so > >> some ops are not supported as follows: > >> +``dev_start`` > >> +``dev_stop`` > >> +``rx_queue_setup`` > >> +``tx_queue_setup`` > >> +``rx_queue_release`` > >> +``tx_queue_release`` > > > > What about some config commands? > > Such as "clear port stats all". Should this be allowed by secondary? > > > I think so, actually, all the queues is visible to primary and > secondary. The only thing we do is to separate queues for different > process for io (packets) in Rx/Tx. It is of for secondary "clear port > stats all". > > And like "port config all rxq". If primary hasn't started ports, should= the secondary allowed to change traffic related stuff (offloads, rx/txd, r= x/txq and so on)? > > > Yes, port config all rxq/txq/rxd/txd/offload is not supported in the > secondary process. It has been done in v5. > >> + > >> +RTE_FLOW supported, it applies only on its own process on SW side, bu= t all on > >> HW size. > > > > About rte flow, what do you mean apply only on its own process on SW si= de? > > If I set number-procs=3D2, rxq=3D4 > > Then on secondary process, I set a flow which directs 192.168.0.1 traff= ic to queue 0. It seems it will directs this kind of traffic to primary pro= cess. But I can't see this rule from primary process side. > > Is this behavior right for multiple process? > > > According to doc rte_flow.rst, we maintain flow rules in process level: > primary and secondary has its own flow list(but one flow list in HW). > As previously mentioned, the two can see all the queues, so setting the > flow rules for the other is OK. > Of course, io(receive or transmit packets) in the queue in others is not > permitted. Can you add this behavior as well to the testpmd doc. Further isolation of resources and operations between the primary and secondary processes is possible. But this is a good start. We can add more if needed. > >> +stats supported, stats will not change when one quit and start, As th= ey share > >> the same buffer to store the stats. > >> +RSS supported, Primary process and secondary process has separate que= ues to > >> use, RSS will work in their own queues whether primary and secondary p= rocess. > >> -- > >> 2.7.4 > > --000000000000b53d8c05be64ba45--