From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 352CAA0547; Wed, 21 Apr 2021 11:57:11 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A9BEA419EC; Wed, 21 Apr 2021 11:57:10 +0200 (CEST) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mails.dpdk.org (Postfix) with ESMTP id EF1B9419D3 for ; Wed, 21 Apr 2021 11:57:08 +0200 (CEST) IronPort-SDR: lraqKE8/SF51RbzK165GTDsvmE3p++J2yzaRSvSy2PRvC0+Y1u0sJYzQIirMqW6QGDzcC2kM+H j8K9+mCeqLRA== X-IronPort-AV: E=McAfee;i="6200,9189,9960"; a="195698912" X-IronPort-AV: E=Sophos;i="5.82,238,1613462400"; d="scan'208,217";a="195698912" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 Apr 2021 02:57:07 -0700 IronPort-SDR: PSM0JIQXYxx+jOJ2ExBRyb/uv1HFhzM062WNDokR404zQbDqLW2puzJc8GK6DSGgs9HCO8Yr4m Ow77XP3m/A4g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.82,238,1613462400"; d="scan'208,217";a="463532179" Received: from orsmsx603.amr.corp.intel.com ([10.22.229.16]) by orsmga001.jf.intel.com with ESMTP; 21 Apr 2021 02:56:47 -0700 Received: from orsmsx608.amr.corp.intel.com (10.22.229.21) by ORSMSX603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Wed, 21 Apr 2021 02:56:46 -0700 Received: from orsmsx604.amr.corp.intel.com (10.22.229.17) by ORSMSX608.amr.corp.intel.com (10.22.229.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2; Wed, 21 Apr 2021 02:56:46 -0700 Received: from orsedg603.ED.cps.intel.com (10.7.248.4) by orsmsx604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2106.2 via Frontend Transport; Wed, 21 Apr 2021 02:56:46 -0700 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (104.47.73.47) by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2106.2; Wed, 21 Apr 2021 02:56:45 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B/XycVpWPpA8RznH3C6+y63i9b6H8oSS33pboKKX5j61gcYVZZ12bonG7BaOY8oUW100J61XU9cr7r+gafEbqnvuOakYHT5Ls2e6Yuh/Qn8oADe6199He3y1FdQX+/wb3gqjPCh1MffbmjjEBeG79VC47pVuJ+NGiU7q2yfpPzu9l1hfGpxtdkTIu3fHSKg6tAR4AarfJ0AMqBUPjUp0Ae8GkZXTP2zP2/D9/oE3g4HGmIHSHcMfst9JvniyEfuG9fqb2udy9x4AbTZ6P7+0z2wsaznweL64ngu8Cw+a3eLy92U10/f6K4Ex41kkVCqClkAT6/oo3Zc9CDRzwsB2LA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kywvSg5Bc1L8Rs5Wx4gBE+rVATmmosBGko/Tq+dFJVY=; b=h+zEPwKYyPiVICApVkBheTYhVzAIp5IoPvzH/89FeahjQb0WYJ9TiRjuMIi0LfYqK8Py7yVq29PpeOx1pTYBl0zd7b5NKQSfMXYE5X42CTXQl6jvFbP0UO1zCvR0g2T/Sly2ATmby9z+LkXScpTevCmYqRoPeiKL5pnnqK9obYZrERWaVersuLM24FeKsLc5qBO6r0PCRnGE+/ZwFaxpRQZttszdJWSe5Wxr+IeZp5TEKIJPUe8qzRAjlZ+yvud06J9zOuwZO4ehAh4U5O6wSF7c3J1O7bihDPCou84oFxO+ICIGl3J9nczdvm/1HBwhj/exnMAIsFh+/e+ftGbI7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kywvSg5Bc1L8Rs5Wx4gBE+rVATmmosBGko/Tq+dFJVY=; b=hHcXFl6gVavDCZHvysjuqIwMdNDeRqwfJgABdoeEdO8vKu3k9ZMluNZWBF/g1G0ZcXZrIOxwfabeSc2FpbcLd7UAmJzF3tdzJzNr6PeVn3cmuh33cyaahkw8/5L1nlRMQeipcdPRuiWaeKDQisldcH0Oi0e8bqnSyd500+ifZYE= Received: from SJ0PR11MB5007.namprd11.prod.outlook.com (2603:10b6:a03:2d2::8) by SJ0PR11MB4783.namprd11.prod.outlook.com (2603:10b6:a03:2af::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Wed, 21 Apr 2021 09:56:43 +0000 Received: from SJ0PR11MB5007.namprd11.prod.outlook.com ([fe80::dc03:cf6e:27ff:51c6]) by SJ0PR11MB5007.namprd11.prod.outlook.com ([fe80::dc03:cf6e:27ff:51c6%3]) with mapi id 15.20.4042.024; Wed, 21 Apr 2021 09:56:42 +0000 From: "Singh, Aman Deep" To: "Min Hu (Connor" , "'dev@dpdk.org'" CC: "Mcnamara, John" , "Kovacevic, Marko" , "Burakov, Anatoly" , "andrew.rybchenko@oktetlabs.ru" , "thomas@monjalon.net" , "Li, Xiaoyun" , "Yigit, Ferruh" Thread-Topic: [dpdk-dev] [PATCH v12] app/testpmd: support multi-process Thread-Index: Adc2kl8v+UkA6/kmRkKjL9hGuwO2Pw== Date: Wed, 21 Apr 2021 09:56:42 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-reaction: no-action dlp-version: 11.5.1.3 dlp-product: dlpe-windows authentication-results: huawei.com; dkim=none (message not signed) header.d=none;huawei.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [106.214.216.23] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 87ab41b7-05d5-4423-2413-08d904abc45c x-ms-traffictypediagnostic: SJ0PR11MB4783: x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7691; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: XqmJ8Pjy3QUbYLPyGmpHLcBibtjTh3KPmxB4rYQc/htSBghBpAoV7e0YyzLpYzLuiheKcEWpqxZY/UHg/BaILCQ7a4b7p/mgfv/PlqKBY96+6Ck5v4p+OjkeKooVWDUroJLr6iQ03+DGuQAuvCjOzgUe7RDysmxKleFNke/jYHMIpoiLLGnC366/qU9OJnurOG1M6tOhbjmxVhCaYlG80mCFWLSAkjS7YXGNl6HIF1veoeNqwTzadxCtFq2OH7133EwutY/ZKZYBSZAaxJ8zz03r+G0mSBrHGDj+8fLL4zwrn/mnKMGHYAdWhS/bcHeB7NUdEljfcA+tprao28l4VYLllQ8v5+nA0VFPVYtqoKWFBHd2ZAtyTUaWTBfjSOIOxCXGF5/0yMejPGcw4D4KKCDR97dzVjHvamvieqcoUNcwmxWN4rxCK40wwvNJDvaPyzxFURVNepo6cQo98nXL9KjL8HRGFXHerZLP8+oOLtCpPVnWxdbooHsqqNUnKp+RQ9JbMWBbv4VN7gc/KN1IqFbH8KOqaM0lc16WGtBkx1hsLD3NDcRg9vYl4kEqRC2Wt5P9HltDqq0LOuuxY2uRJ881LggmlGLRgj5ayTmUIzdDyaWcKj2A3tz9V/GY4vAZI9+8HIIBVpFEOmbo1uc8fSGFt+Vu+AdZbI4FYny0cbM= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR11MB5007.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(376002)(136003)(366004)(39860400002)(396003)(346002)(2906002)(26005)(107886003)(38100700002)(33656002)(4326008)(122000001)(55016002)(9686003)(8936002)(64756008)(66556008)(478600001)(71200400001)(5660300002)(66946007)(76116006)(52536014)(8676002)(316002)(7696005)(186003)(54906003)(83380400001)(110136005)(6506007)(86362001)(66446008)(66476007)(21314003)(491001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: =?us-ascii?Q?W+dAGeMPcpkZ7juP6fMeLUmdql/RoFFNJ/pBR43Ik70LdmyqurSBpol5L+bV?= =?us-ascii?Q?9JJiXFS5u3QJdkmiPiforvgm9bZp9TnMXZwm2XnGK+yjvl4VkkKH+G6dFKUi?= =?us-ascii?Q?cW3LePG/keiYM1DqUskGx94ejTLCXXJvpTKg5asr/sJX3LjdPU2e0zm9Po7s?= =?us-ascii?Q?mDhvhQot8TFUicfQz5KmI//aqSLyumt3+QDG9/QWU3NQy6vJWRfMcSO+y0SZ?= =?us-ascii?Q?g+gyTDpbzYSYY8vxbpncAGzQK5TbM53cVR+4vJJjR0xz5ZF57cW8Qwtf3Lr1?= =?us-ascii?Q?hmkhz5r63IUiCptgOuHMz3CeAJFqD8SrqFp8WxqwCTA5blHfoqGcjZ/9plpv?= =?us-ascii?Q?oOAO7rH78I8xwaUsDfTwIUQlxQbtkeuPoco0DxWU/ZpraR0XZzRYrf1TviNv?= =?us-ascii?Q?pLd4LSZp0zhNyh5C0slJG926rk6xrNBcNaImW1jYo0q6mXbByv6QxdczI4Nq?= =?us-ascii?Q?eU8NpD7SIpNQtyzkUIWu1bYDt66SlFxcbFmQ/MDG8gMo7P4sDZmP5EsbERLP?= =?us-ascii?Q?gz0CU7UBX0iGgmYcUcXduVGFUWiNL5GVj4S0GyTzNOgRwj2ohjJC5Hck9uDJ?= =?us-ascii?Q?v/rLy/mdF1vvBLO+/cO9VmJiO8OJ7QSrZiMN4B5yP+gEHOaKaqP1yVSYZxjm?= =?us-ascii?Q?qkWZQ6THj3//EQmluNPiidec/i2JzLvsoo2SyGm0JUyBSmXlOMtWwaIzLYhG?= =?us-ascii?Q?nJrFOb8cr17VJfW36P2a1mvTeuB4N2edQBfaHlgi9L9wgf5DBAx/C8RLwVdv?= =?us-ascii?Q?eA44yldk/5Vm5p2weA7H47S2o3fYLgeoJmYEU44fyMTFDvDNKo9c7ygOGJXd?= =?us-ascii?Q?gHnlAnEcuaM0nNi3PS04XByVdbwdZrbCdQyMikbsqqDdXuJPVR/A4RcawVp6?= =?us-ascii?Q?NOtb8I8k77tsnG8m6Qu/SKLI/EjTtFuiTbrGVpb74ziMs5OQWuoB23NsQp6t?= =?us-ascii?Q?hOHoM5oNsxHmHMFGttDkVEG+RCb20XHLZ9EGXVWX+MUOuBbyt8n8ZyAQRcic?= =?us-ascii?Q?ToHgPruRImPNl1kV1tQ1h1WoatmfTm8o8Ym3cdKArDrrHXaAeN7C6SLgIXbO?= =?us-ascii?Q?osQ8rhKdkdzMJx7RlNFRYC+txbeDUY5cp0fnZi0LL33wlhtfZwFP55iTp+CP?= =?us-ascii?Q?/Y3y8AuRCmbhGAPuB2vIL3LKIzuF1PwkhCZbp2rcw90PpEp28FDO/zW83VO/?= =?us-ascii?Q?ZPWwTdb4y1/Joc650IxDMmaZL6vrogHkE9KTTm+hCowDxqHLLlXsRx6JRpj3?= =?us-ascii?Q?tVB0MrTy/q/4vD2Nshyv7502u64uUzpMBrZh4BkRzfRKjCxyic9pSPjW2ilb?= =?us-ascii?Q?wuPecym9D0yKoyHE172kBZQx?= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SJ0PR11MB5007.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 87ab41b7-05d5-4423-2413-08d904abc45c X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Apr 2021 09:56:42.8354 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bhAfk25UB8vMV/MSeSV+5fGZw+QBSfbIC4vTP/i+6ghlYpOAgNtCYu8M90jce98cqgDmnDXK8Vl+IvK6ulfr8v9aqjYsoGOZ45a1vIjucTY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB4783 X-OriginatorOrg: intel.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [dpdk-dev] [PATCH v12] app/testpmd: support multi-process X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds multi-process support for testpmd. The test cmd example as follows: the primary cmd: ./dpdk-testpmd -a xxx --proc-type=3Dauto -l 0-1 -- -i \ --rxq=3D4 --txq=3D4 --num-procs=3D2 --proc-id=3D0 the secondary cmd: ./dpdk-testpmd -a xxx --proc-type=3Dauto -l 2-3 -- -i \ --rxq=3D4 --txq=3D4 --num-procs=3D2 --proc-id=3D1 Signed-off-by: Min Hu (Connor) > Signed-off-by: Lijun Ou > Acked-by: Xiaoyun Li > Acked-by: Ajit Khaparde > Reviewed-by: Ferruh Yigit > --- v12: * Updated doc info. <....> diff --git a/doc/guides/testpmd_app_ug/run_app.rst b/doc/guides/testpmd_app= _ug/run_app.rst index d062165..7414e85 100644 --- a/doc/guides/testpmd_app_ug/run_app.rst +++ b/doc/guides/testpmd_app_ug/run_app.rst @@ -543,3 +543,87 @@ The command line options are: bit 1 - two hairpin ports paired bit 0 - two hairpin ports loop The default value is 0. Hairpin will use single port mode and implicit= Tx flow mode. + +Testpmd Multi-Process Command-line Options +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following are the command-line options for testpmd multi-process suppo= rt: + +.. code-block:: console + + primary process: + sudo ./dpdk-testpmd -a xxx --proc-type=3Dauto -l 0-1 -- -i --r= xq=3D4 --txq=3D4 \ + --num-procs=3D2 --proc-id=3D0 + + secondary process: + sudo ./dpdk-testpmd -a xxx --proc-type=3Dauto -l 2-3 -- -i --r= xq=3D4 --txq=3D4 \ + --num-procs=3D2 --proc-id=3D1 + +The command line options are: + +* ``--rxq=3DN`` + + Set the number of Rx queues per port to N. N is the sum of queues used= by primary + and secondary process. Primary process and secondary process should ha= ve separate + queues, and each should occupy at least one queue. Where N should be t= he multiple // [Minor Comment]: , not required + of number of processes. + +* ``--txq=3DN`` + + Set the number of Tx queues per port to N. N is the sum of queues used= by primary + and secondary process. Primary process and secondary process should ha= ve separate + queues, and each should occupy at least one queue. Where N should be t= he multiple // [Minor Comment]: , not required + of number of processes. + +* ``--num-procs=3DN`` + + The number of processes which will be used. + +* ``--proc-id=3Did`` + + The id of the current process (id < num-procs). id should be different= in primary + process and secondary process, which starts from '0'. + +Calculation rule for queue: +All queues are allocated to different processes based on ``proc_num`` and = ``proc_id``. +Calculation rule for the testpmd to allocate queues to each process: +start(queue start id) =3D proc_id * nb_q / num_procs? +end(queue end id) =3D start + nb_q / num_procs? + +For example, if testpmp supports 4 Tx and Rx queues the 0~1 for primary // [Spelling] testpmp --> "testpmd", change Rx --> 4 Rx +process the 2~3 for secondary process + +The number of rings should be a multiple of the number of processes. If +not, redundant queues will exist after queues are allocated to +processes. After RSS is enabled, packet loss occurs when traffic is sent t= o all processes at the same time. +Some traffic enters redundant queues and cannot be forwarded. + +All the dev ops is supported in primary process. While secondary +process is not permitted to allocate or release shared memory, so some ops= are not supported as follows:: +``dev_configure`` +``dev_start`` +``dev_stop`` +``rx_queue_setup`` +``tx_queue_setup`` +``rx_queue_release`` +``tx_queue_release`` + +So, any command from testpmd which calls those APIs will not be +supported in secondary process, like:: +``port config all rxq|txq|rxd|txd `` ``port config +rx_offload xxx on/off `` ``port config tx_offload xxx +on/off`` etc. + +Flow API is supported, it applies only on its own process on SW side, but = all on HW size. // [Replace] all --> uniformly [Spelling] size --> side + +Stats is supported, stats will not change when one quit and start, As +they share the same buffer to store the stats. Flow rules are +maintained in process level: primary and secondary has its own flow +list (but one flow list in HW). The two can see all the queues, so +setting the flow rules for the other is OK. But in testpmd primary // [Replace] in --> if +process receives or transmits packets from the queue allocated for seconda= ry process is not permitted, and same for secondary process. // [Rephrase] is not permitted--> it is not permitted + +RSS is supported, primary process and secondary process has separate +queues to use, RSS will work in their own queues whether primary or second= ary process. -- 2.7.4 Tested-by: Aman Deep Singh >