From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B67B6A0350; Mon, 3 Jan 2022 10:14:16 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 4308A40042; Mon, 3 Jan 2022 10:14:16 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 819AF4003C for ; Mon, 3 Jan 2022 10:14:14 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with ESMTP id 2037tVtt029250; Mon, 3 Jan 2022 01:14:13 -0800 Received: from nam02-bn1-obe.outbound.protection.outlook.com (mail-bn1nam07lp2044.outbound.protection.outlook.com [104.47.51.44]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3dbw5w13rj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 03 Jan 2022 01:14:12 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BIHsjkXYhBpn5NRzqNTc626hVAaIQwlI3qylwDBl0wCM3x34EgMwuc2XlMswO99WFUK64mhdgUIE06GT4rcrxXZiGrakvsDLHIGBSzI8sTzY9EnO03Fj278Y0eyYSdE4Pg8nmGjDYWo6zKBFQJEqfk93OV2T2Cyp5xnMYT8VFnDp3CRtHokbGq9BPms18o58vv4Dr23QOGdUtBzErK8/qIIzpQ2jjUIDOprWHSadkUyAXROO6ibA8gVHLeB6Zpzx+tQh0EuwIgh+mSN865v2KUL1Qy6hfyHkNQBTHFDsIoVWn5QXbKAY50uhNNKNYQo43ylEGkpHMuS6Wc/y+V9vUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=azpQjJldTWcdnEN98p3aJbh0suT8k5tVPnJe+Nifo1c=; b=JJPhz52u6/c1unzFKWns3UTHMYjdDsXJwunTF+MIuri7hptq+hbpqRe32nMLOA5B7jLODBPDEimuC27EJa/pvEsxlTcQb7cC8P92Dz60yaBorTjIfvqL7RW6tJBZmBbWorByqUVXeWfY4MNzZ81xqJrWpIvGNF61oHITXHeY5uPQ0U/I1XS2X/YBojiguwxCrqQco4hakkWWweivYOF5j9ymjUxIMHC2vroPS8xGbCI7GtA0qgYlkfE/xsllyZWs8vW5lOXZUecjwQA+5FY2rY9QU1tPvty/qSnShEj/UvuI14yvKIHUDNgCQ1yRqoM/I4Aj9v+Hjz2KdHOdXHK/pA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=azpQjJldTWcdnEN98p3aJbh0suT8k5tVPnJe+Nifo1c=; b=bAYmC8iWJK2Fyq53x8JE4v+uh6IkdVE0e1WB2GaAY6iCJV2tutu8XcVLgu9cLSOxoFWIjgayniFI861zV6pKZVSFztRXu9goTZJ51elxg0+fA59t/03U37fserrViAyd/2Ox084TEGd86qBb/+dR+Vb99D9lPHDtkSjZb4ZoR9k= Received: from CO6PR18MB4418.namprd18.prod.outlook.com (2603:10b6:303:13a::23) by CO6PR18MB4433.namprd18.prod.outlook.com (2603:10b6:303:13b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4823.21; Mon, 3 Jan 2022 09:14:06 +0000 Received: from CO6PR18MB4418.namprd18.prod.outlook.com ([fe80::d5a6:6129:1e60:3afe]) by CO6PR18MB4418.namprd18.prod.outlook.com ([fe80::d5a6:6129:1e60:3afe%9]) with mapi id 15.20.4844.014; Mon, 3 Jan 2022 09:14:06 +0000 From: Shijith Thotton To: "dpdklab@iol.unh.edu" , "aconole@redhat.com" CC: Anoob Joseph , Pavan Nikhilesh Bhagavatula , Akhil Goyal , "Gujjar, Abhinandan S" , Jerin Jacob Kollanukkaran , "dev@dpdk.org" Subject: RE: [PATCH v2] app/eventdev: add crypto producer mode Thread-Topic: [PATCH v2] app/eventdev: add crypto producer mode Thread-Index: AQHX9kf0XRbwNw0ryEaL9tB7vcCBgKxK+/kAgAXcIiCAADgygIAAAL5g Date: Mon, 3 Jan 2022 09:14:05 +0000 Message-ID: References: <83932e16cff802f4701832c1798e31ec15b028f8.1640029478.git.sthotton@marvell.com> <5b7dadb222f6eb018c74983750052543eb5f0d8e.1640076426.git.sthotton@marvell.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: b51dc449-c8d1-42aa-6238-08d9ce9964a1 x-ms-traffictypediagnostic: CO6PR18MB4433:EE_ x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:33; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: cvVE/49bmXGZme4ahdR3lOxJjfvkiqDb0qoeAiL7gf1OkijzUQDHXEnxv1+Dj5G/YDdvZh9ryhk+CIHQtdHz8MVNm/xRN5nNjP76QtPfYs/C44pTW0VnlZtAf2I8uIUkVtC09BZsbveQGDRGCm/CfhYbNTypIyNEKLgHyvynNti/np6wA/LoYW+4L22hRsX63QVqZaBPw2sBhsSVSSQfD7PKgLUgoq8+tdcgbmIlWjbNYjiRCUNzsTtnJ+MPKBpx81PW+ZJuYiUAoJcBaxisw6KgwV8aiGIRJcTJNjwGR7BiYegq8JCeiZkgzfqQH4ECt76QPUZsEXmsOEb6pTcEcqUaB7YEFzYCOVuuKr8HKb67OLZlx3/fNWL4HrnMpwxLTBj81FAs84Z2lljQLne5GAxn3w3Q9BR/mq/bd+wZlBBiVfKPoOSIDQO24wM5AJjuJA55sh2ZBUprvuHrhkSdlDIQyw0wEbtyx+Jfu2PgEIBAxvU1wwzpm2z1u4X6soKW0NK2sk4FWN1N1yEyfeWBv4vKj7LSrhAjHoYTXKu3JLXjhuUZWskqzUyVzkCTKv9Kfn1tJgvMFDGoeLA01etwQQ/NJCgsX7/ppVjMMZ04ahgGGStn6Z1+1x7veDcpl0VtC6V/YlI//uAxf7y72+UvldPd8rhi30awYMVUifiFTiepoS2gee4wr9gXdqPgZUubdVkFvSF5yRYdvulxcqvg6/5OuhtpuB82xo8k71F29rZAzt1rfjIxHFsps3mhbOMVBllYpU6nbfLCBRmSKNIDuQknOfvATV239iJAiVSWTDY= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR18MB4418.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(76116006)(66446008)(66476007)(186003)(52536014)(64756008)(2906002)(122000001)(38100700002)(55236004)(71200400001)(30864003)(66946007)(316002)(9686003)(8936002)(508600001)(26005)(86362001)(83380400001)(8676002)(55016003)(6506007)(5660300002)(7696005)(38070700005)(54906003)(110136005)(66556008)(33656002)(4326008)(559001)(579004); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Mq4+H9mSGet6/pWtPds2xRJ57tuN4LUOXwDXxuuf9uU/Tzdp1QAJUp0IJQiZ?= =?us-ascii?Q?IhA5B4itXgY+erlyvA1x3qParF5Nr47RQRDznghPiSUJcRVCmsHSrEUn1tiS?= =?us-ascii?Q?ejYQAOzd7zNimXIhNp7bWg+0QoRmS9GQI86lX3W1bPs8eEDa/J8wVeRTkEqU?= =?us-ascii?Q?05UvKe47jV2hzqiExrSz5ZFiEjXucuwSXklRFqSCn+kpjY8tfwWv7e4MOOrQ?= =?us-ascii?Q?iSZh9sZrNTPaJEw3nhh+03kabwItOQ5LdGp3Obhjf7AjwpUPWl13NISPj7xL?= =?us-ascii?Q?mtk9YS1Kue3XE45v4WIDnqHCiSSUrcPFpCZX4Mzf0mdhmRqX7apysvbvMPJw?= =?us-ascii?Q?tX/ALPofxBNhetp1SYEr5mLL5IomwwIwZ6Rf///Xx6zQ6k+aLxWTFl3hp9hb?= =?us-ascii?Q?yei3O0q6xTsG2rWaa26YO3oKDW0E8Hyw04FQCm/0Fo+h1DCMV5ylYsxspKPL?= =?us-ascii?Q?ocZe4GvGqdV7qffmDe3Yl+hm8gVj+Shvha/yehqJAjoirPGkMUwXlmhZWp3T?= =?us-ascii?Q?G8eJl+J0ALJT3Rft93jiwTaQ8VUrD6TIEUOlCW1vFO7/ds0q63VgeTNHCuPq?= =?us-ascii?Q?/LjvhTyWoDY8B05CdispUWyQ0DWxfPTJyrj7IXVBQVYwsncILLSUWLGO1TSx?= =?us-ascii?Q?vlsjbM9HPmf7YnMA1j42/7jCr4bDtOLcpo/N3zR70xzTA9s71Z8KrVsJH3A0?= =?us-ascii?Q?tljy/QdkTY0NAw/02Qld3CLGIZsBOWGNRwQQBdkjNJP9fmEM19uuCUWIBwhU?= =?us-ascii?Q?j2cqZ7Ua3qHbcL1TKd/0nlSQu7CNqb06VIWgr1Gsf0LAE5TOQ34Ab3zKPJhh?= =?us-ascii?Q?1uhKShEFa9IXCYTPF6QSbl8d3lCwQKe+G8nzBDvdrMqJrWnWaJG3igp24Tdk?= =?us-ascii?Q?DRzVQMRnkzolcjgTldY1f/PBVm818ft3jT9dUXH1M2U4u8/xj9Sksg5jMwwm?= =?us-ascii?Q?7umlcqwuY3Y0k+s7xEkho34TU4J/6tV4jEgS6/4RfPMW6NG6wYRS3GCUtfMu?= =?us-ascii?Q?6x8ooU7epLhKd9C2a5hzQwClYlnw8IqmvzrXCom/vIc/AvKm8nbHodsi5+ib?= =?us-ascii?Q?vXzytxyObkhKf9stSu8zWHJeJEvcIRvhIPLr7adrd4iyhbN+SqRdJ1bScFgq?= =?us-ascii?Q?SUuWib6zdPbzyXGGG4+ULRYIRzDlY+wfOhQy1n2YTXD8NJ19guSHPszavBRT?= =?us-ascii?Q?wyue+mkcmUKrHmCv52Ud9ut0+dpoyPR2MPPS7MJnBoyUfrIm1Z/IgkqIwBTp?= =?us-ascii?Q?Tgi7zKBwCbmdJP3TzyHbPh0HBWwCVU5uQidq3UI4IEcCTdZsTvHvqP7Lnzo5?= =?us-ascii?Q?Q1JKLzoZ7j7yxwNBBbrS7eKx19tUri/qgPXU558MV0pia90IqIp15cZU5qAW?= =?us-ascii?Q?8TrIx+K6uGD4KWgdSniMbaYB35RIzEruN/93uKVem9tHszk1KXzh5kjEBsQz?= =?us-ascii?Q?KGzl130wOgk6liAhVhJrukcKwY4qQAQwbhZvqB+QVMsBmOEBN1+hCh7EP/W3?= =?us-ascii?Q?8ZS1tT+WGKJeF9F5Az/wbTd99IEznpJHkYWhC8OxJWNwHMtA1vdSSTN0Jt19?= =?us-ascii?Q?LNWLf0lDk+4iyrTH+dCYQZ9IyU9mgU3bMzQlRORG4q8kxFtpSw86mhjxZgBD?= =?us-ascii?Q?nZw+I76iLJ+J2Do7VXN9Kis=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO6PR18MB4418.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: b51dc449-c8d1-42aa-6238-08d9ce9964a1 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Jan 2022 09:14:06.0271 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: bDBT/HzMdH1R3KPzekrxCJlGzaDGNlcZEjDl3nRYZcHegag8gqrM0EcRkmOMkNU51JQ4qlOPow9PvykfV/bN4g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR18MB4433 X-Proofpoint-ORIG-GUID: mcHTIjeuu1NvRgPd36S8fB1KVQQPLHue X-Proofpoint-GUID: mcHTIjeuu1NvRgPd36S8fB1KVQQPLHue X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-01-03_03,2022-01-01_01,2021-12-02_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org >> >> >> >> In crypto producer mode, producer core enqueues cryptodev with >> >> software generated crypto ops and worker core dequeues crypto >> >> completion events from the eventdev. Event crypto metadata used for >> >> above processing is pre- populated in each crypto session. >> >> >> >> Parameter --prod_type_cryptodev can be used to enable crypto producer >> >mode. >> >> Parameter --crypto_adptr_mode can be set to select the crypto adapter >> >> mode, >> >0 >> >> for OP_NEW and 1 for OP_FORWARD. >> >> >> >> This mode can be used to measure the performance of crypto adapter. >> >> >> >> Example: >> >> ./dpdk-test-eventdev -l 0-2 -w -w -- \ >> >> --prod_type_cryptodev --crypto_adptr_mode 1 --test=3Dperf_atq \ >> >> --stlist=3Da --wlcores 1 --plcores 2 >> > >> >This patch has some perf failure as shown below. Could you please look = into >> this? >> >105300 --> performance testing fail >> > >> >Test environment and result as below: >> > >> >Ubuntu 20.04 >> >Kernel: 4.15.0-generic >> >Compiler: gcc 7.4 >> >NIC: Intel Corporation Ethernet Converged Network Adapter 82599ES 10000 >> >Mbps >> >Target: x86_64-native-linuxapp-gcc >> >Fail/Total: 0/4 >> > >> >Detail performance results: >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| frame_size | txd/rxd | num_cpus | num_threads | throughput >> >| difference from >> >| >> >| | | | | expected = | >> >>+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D >=3D >> >=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> >| 64 | 512 | 1 | 1 | 0.3% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| 64 | 2048 | 1 | 1 | -0.2% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| 64 | 512 | 1 | 2 | 0.0% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| 64 | 2048 | 1 | 2 | 0.3% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> > >> >Ubuntu 20.04 >> >Kernel: 4.15.0-generic >> >Compiler: gcc 7.4 >> >NIC: Intel Corporation Ethernet Converged Network Adapter XL710-QDA2 >> >40000 Mbps >> >Target: x86_64-native-linuxapp-gcc >> >Fail/Total: 1/4 >> > >> >Detail performance results: >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| frame_size | txd/rxd | num_cpus | num_threads | throughput >> >| difference from >> >| >> >| | | | | expected = | >> >>+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D >=3D >> >=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> >| 64 | 512 | 1 | 1 | 0.2% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| 64 | 2048 | 1 | 1 | -0.7% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| 64 | 512 | 1 | 2 | -1.5% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| 64 | 2048 | 1 | 2 | -5.3% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> > >> >Ubuntu 20.04 ARM >> >Kernel: 4.15.0-132-generic >> >Compiler: gcc 7.5 >> >NIC: Arm Intel Corporation Ethernet Converged Network Adapter >> >XL710-QDA2 >> >40000 Mbps >> >Target: x86_64-native-linuxapp-gcc >> >Fail/Total: 0/2 >> > >> >Detail performance results: >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| frame_size | txd/rxd | num_cpus | num_threads | throughput >> >| difference from >> >| >> >| | | | | expected = | >> >>+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D >=3D >> >=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D+ >> >| 64 | 512 | 1 | 1 | 0.1% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> >| 64 | 2048 | 1 | 1 | -0.5% = | >> >+------------+---------+----------+-------------+----------------------= --------+ >> > >> >To view detailed results, visit: >> >https://urldefense.proofpoint.com/v2/url?u=3Dhttps- >> >>3A__lab.dpdk.org_results_dashboard_patchsets_20534_&d=3DDwIFAg&c=3DnKjWe >> c2 >> >b6R0mOyPaz7xtfQ&r=3DG9w4KsPaQLACBfGCL35PtiRH996yqJDxAZwrWegU2qQ& >> m=3D >> >>dPDgmKiLC_KqhgjwZrodyFlBdlFPgckVr9IW0c7t2cIHpkw6u9pGqgElFqqwuU1v&s >> =3DE >> >ajYUFziZY27q6LB4p5sOGuFlty3VlOjbMsLXaQ5MeI&e=3D >> > >> >> Performance regression is reported by nic_single_core_perf, but is using >> testpmd to measure performance. As this patch does not touch testpmd, th= e >> issue could be with DPDK CI. Is it possible to re-trigger CI ? >I have don't have option to trigger CI from my side. >So, I have added dpdklab and Aaron to the loop. >Please check with them. > =20 Thanks Abhinandan. Aaron, >From DTS, I only see testpmd being used for nic_single_core_perf test. This patch is not touching testpmd, but still got failure for the perf test= . If possible, Please re-retrigger CI for this patch. >> >> >> >> >> Signed-off-by: Shijith Thotton >> >> --- >> >> v2: >> >> * Fix RHEL compilation warning. >> >> >> >> app/test-eventdev/evt_common.h | 3 + >> >> app/test-eventdev/evt_main.c | 13 +- >> >> app/test-eventdev/evt_options.c | 27 ++ >> >> app/test-eventdev/evt_options.h | 12 + >> >> app/test-eventdev/evt_test.h | 6 + >> >> app/test-eventdev/test_perf_atq.c | 51 ++++ >> >> app/test-eventdev/test_perf_common.c | 406 >> >++++++++++++++++++++++++++- >> >> app/test-eventdev/test_perf_common.h | 16 ++ app/test- >> >> eventdev/test_perf_queue.c | 52 ++++ >> >> doc/guides/tools/testeventdev.rst | 13 + >> >> 10 files changed, 592 insertions(+), 7 deletions(-) >> >> >> >> diff --git a/app/test-eventdev/evt_common.h b/app/test- >> >> eventdev/evt_common.h index f466434459..2f301a7e79 100644 >> >> --- a/app/test-eventdev/evt_common.h >> >> +++ b/app/test-eventdev/evt_common.h >> >> @@ -7,6 +7,7 @@ >> >> >> >> #include >> >> #include >> >> +#include >> >> #include >> >> #include >> >> >> >> @@ -39,6 +40,7 @@ enum evt_prod_type { >> >> EVT_PROD_TYPE_SYNT, /* Producer type Synthetic i.e. CPU. *= / >> >> EVT_PROD_TYPE_ETH_RX_ADPTR, /* Producer type Eth Rx Adapter. */ >> >> EVT_PROD_TYPE_EVENT_TIMER_ADPTR, /* Producer type Timer >> Adapter. >> >> */ >> >> + EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR, /* Producer type Crypto >> >> Adapter. */ >> >> EVT_PROD_TYPE_MAX, >> >> }; >> >> >> >> @@ -77,6 +79,7 @@ struct evt_options { >> >> uint64_t timer_tick_nsec; >> >> uint64_t optm_timer_tick_nsec; >> >> enum evt_prod_type prod_type; >> >> + enum rte_event_crypto_adapter_mode crypto_adptr_mode; >> >> }; >> >> >> >> static inline bool >> >> diff --git a/app/test-eventdev/evt_main.c >> >> b/app/test-eventdev/evt_main.c index 194c980c7a..cef0fb1382 100644 >> >> --- a/app/test-eventdev/evt_main.c >> >> +++ b/app/test-eventdev/evt_main.c >> >> @@ -161,12 +161,19 @@ main(int argc, char **argv) >> >> goto mempool_destroy; >> >> } >> >> } >> >> + /* Test specific cryptodev setup */ >> >> + if (test->ops.cryptodev_setup) { >> >> + if (test->ops.cryptodev_setup(test, &opt)) { >> >> + evt_err("%s: cryptodev setup failed", opt.test_name); >> >> + goto ethdev_destroy; >> >> + } >> >> + } >> >> >> >> /* Test specific eventdev setup */ >> >> if (test->ops.eventdev_setup) { >> >> if (test->ops.eventdev_setup(test, &opt)) { >> >> evt_err("%s: eventdev setup failed", opt.test_name); >> >> - goto ethdev_destroy; >> >> + goto cryptodev_destroy; >> >> } >> >> } >> >> >> >> @@ -197,6 +204,10 @@ main(int argc, char **argv) >> >> if (test->ops.eventdev_destroy) >> >> test->ops.eventdev_destroy(test, &opt); >> >> >> >> +cryptodev_destroy: >> >> + if (test->ops.cryptodev_destroy) >> >> + test->ops.cryptodev_destroy(test, &opt); >> >> + >> >> ethdev_destroy: >> >> if (test->ops.ethdev_destroy) >> >> test->ops.ethdev_destroy(test, &opt); diff --git a/app/test- >> >> eventdev/evt_options.c b/app/test-eventdev/evt_options.c index >> >> 753a7dbd7d..5ad1491020 100644 >> >> --- a/app/test-eventdev/evt_options.c >> >> +++ b/app/test-eventdev/evt_options.c >> >> @@ -122,6 +122,26 @@ evt_parse_timer_prod_type_burst(struct >> >> evt_options *opt, >> >> return 0; >> >> } >> >> >> >> +static int >> >> +evt_parse_crypto_prod_type(struct evt_options *opt, >> >> + const char *arg __rte_unused) >> >> +{ >> >> + opt->prod_type =3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR; >> >> + return 0; >> >> +} >> >> + >> >> +static int >> >> +evt_parse_crypto_adptr_mode(struct evt_options *opt, const char *arg= ) { >> >> + uint8_t mode; >> >> + int ret; >> >> + >> >> + ret =3D parser_read_uint8(&mode, arg); >> >> + opt->crypto_adptr_mode =3D mode ? >> >> RTE_EVENT_CRYPTO_ADAPTER_OP_FORWARD : >> >> + >> >> RTE_EVENT_CRYPTO_ADAPTER_OP_NEW; >> >> + return ret; >> >> +} >> >> + >> >> static int >> >> evt_parse_test_name(struct evt_options *opt, const char *arg) { @@ >> >> -335,6 >> >> +355,7 @@ usage(char *program) >> >> "\t--queue_priority : enable queue priority\n" >> >> "\t--deq_tmo_nsec : global dequeue timeout\n" >> >> "\t--prod_type_ethdev : use ethernet device as producer.\n" >> >> + "\t--prod_type_cryptodev : use crypto device as producer.\n" >> >> "\t--prod_type_timerdev : use event timer device as >> producer.\n" >> >> "\t expity_nsec would be the timeout\n" >> >> "\t in ns.\n" >> >> @@ -345,6 +366,8 @@ usage(char *program) >> >> "\t--timer_tick_nsec : timer tick interval in ns.\n" >> >> "\t--max_tmo_nsec : max timeout interval in ns.\n" >> >> "\t--expiry_nsec : event timer expiry ns.\n" >> >> + "\t--crypto_adptr_mode : 0 for OP_NEW mode (default) and\n" >> >> + "\t 1 for OP_FORWARD mode.\n" >> >> "\t--mbuf_sz : packet mbuf size.\n" >> >> "\t--max_pkt_sz : max packet size.\n" >> >> "\t--prod_enq_burst_sz : producer enqueue burst size.\n" >> >> @@ -415,8 +438,10 @@ static struct option lgopts[] =3D { >> >> { EVT_QUEUE_PRIORITY, 0, 0, 0 }, >> >> { EVT_DEQ_TMO_NSEC, 1, 0, 0 }, >> >> { EVT_PROD_ETHDEV, 0, 0, 0 }, >> >> + { EVT_PROD_CRYPTODEV, 0, 0, 0 }, >> >> { EVT_PROD_TIMERDEV, 0, 0, 0 }, >> >> { EVT_PROD_TIMERDEV_BURST, 0, 0, 0 }, >> >> + { EVT_CRYPTO_ADPTR_MODE, 1, 0, 0 }, >> >> { EVT_NB_TIMERS, 1, 0, 0 }, >> >> { EVT_NB_TIMER_ADPTRS, 1, 0, 0 }, >> >> { EVT_TIMER_TICK_NSEC, 1, 0, 0 }, >> >> @@ -455,8 +480,10 @@ evt_opts_parse_long(int opt_idx, struct >> >> evt_options >> >> *opt) >> >> { EVT_QUEUE_PRIORITY, evt_parse_queue_priority}, >> >> { EVT_DEQ_TMO_NSEC, evt_parse_deq_tmo_nsec}, >> >> { EVT_PROD_ETHDEV, evt_parse_eth_prod_type}, >> >> + { EVT_PROD_CRYPTODEV, evt_parse_crypto_prod_type}, >> >> { EVT_PROD_TIMERDEV, evt_parse_timer_prod_type}, >> >> { EVT_PROD_TIMERDEV_BURST, >> >> evt_parse_timer_prod_type_burst}, >> >> + { EVT_CRYPTO_ADPTR_MODE, evt_parse_crypto_adptr_mode}, >> >> { EVT_NB_TIMERS, evt_parse_nb_timers}, >> >> { EVT_NB_TIMER_ADPTRS, evt_parse_nb_timer_adptrs}, >> >> { EVT_TIMER_TICK_NSEC, evt_parse_timer_tick_nsec}, diff --git >> >> a/app/test-eventdev/evt_options.h b/app/test-eventdev/evt_options.h >> >> index >> >> 413d7092f0..2231c58801 100644 >> >> --- a/app/test-eventdev/evt_options.h >> >> +++ b/app/test-eventdev/evt_options.h >> >> @@ -9,6 +9,7 @@ >> >> #include >> >> >> >> #include >> >> +#include >> >> #include >> >> #include >> >> #include >> >> @@ -33,8 +34,10 @@ >> >> #define EVT_QUEUE_PRIORITY ("queue_priority") >> >> #define EVT_DEQ_TMO_NSEC ("deq_tmo_nsec") >> >> #define EVT_PROD_ETHDEV ("prod_type_ethdev") >> >> +#define EVT_PROD_CRYPTODEV ("prod_type_cryptodev") >> >> #define EVT_PROD_TIMERDEV ("prod_type_timerdev") >> >> #define EVT_PROD_TIMERDEV_BURST ("prod_type_timerdev_burst") >> >> +#define EVT_CRYPTO_ADPTR_MODE ("crypto_adptr_mode") >> >> #define EVT_NB_TIMERS ("nb_timers") >> >> #define EVT_NB_TIMER_ADPTRS ("nb_timer_adptrs") >> >> #define EVT_TIMER_TICK_NSEC ("timer_tick_nsec") >> >> @@ -249,6 +252,8 @@ evt_prod_id_to_name(enum evt_prod_type >> prod_type) >> >> return "Ethdev Rx Adapter"; >> >> case EVT_PROD_TYPE_EVENT_TIMER_ADPTR: >> >> return "Event timer adapter"; >> >> + case EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR: >> >> + return "Event crypto adapter"; >> >> } >> >> >> >> return ""; >> >> @@ -288,6 +293,13 @@ evt_dump_producer_type(struct evt_options *opt) >> >> evt_dump("timer_tick_nsec", "%"PRIu64"", >> >> opt->timer_tick_nsec); >> >> break; >> >> + case EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR: >> >> + snprintf(name, EVT_PROD_MAX_NAME_LEN, >> >> + "Event crypto adapter producers"); >> >> + evt_dump("crypto adapter mode", "%s", >> >> + opt->crypto_adptr_mode ? "OP_FORWARD" : >> >> "OP_NEW"); >> >> + evt_dump("nb_cryptodev", "%u", rte_cryptodev_count()); >> >> + break; >> >> } >> >> evt_dump("prod_type", "%s", name); >> >> } >> >> diff --git a/app/test-eventdev/evt_test.h >> >> b/app/test-eventdev/evt_test.h >> >index >> >> f07d2c3336..50fa474ec2 100644 >> >> --- a/app/test-eventdev/evt_test.h >> >> +++ b/app/test-eventdev/evt_test.h >> >> @@ -29,6 +29,8 @@ typedef int (*evt_test_mempool_setup_t) >> >> (struct evt_test *test, struct evt_options *opt); typedef int >> >> (*evt_test_ethdev_setup_t) >> >> (struct evt_test *test, struct evt_options *opt); >> >> +typedef int (*evt_test_cryptodev_setup_t) >> >> + (struct evt_test *test, struct evt_options *opt); >> >> typedef int (*evt_test_eventdev_setup_t) >> >> (struct evt_test *test, struct evt_options *opt); typedef int >> >> (*evt_test_launch_lcores_t) @@ -39,6 +41,8 @@ typedef void >> >> (*evt_test_eventdev_destroy_t) >> >> (struct evt_test *test, struct evt_options *opt); typedef void >> >> (*evt_test_ethdev_destroy_t) >> >> (struct evt_test *test, struct evt_options *opt); >> >> +typedef void (*evt_test_cryptodev_destroy_t) >> >> + (struct evt_test *test, struct evt_options *opt); >> >> typedef void (*evt_test_mempool_destroy_t) >> >> (struct evt_test *test, struct evt_options *opt); typedef void >> >> (*evt_test_destroy_t) @@ -52,10 +56,12 @@ struct evt_test_ops { >> >> evt_test_mempool_setup_t mempool_setup; >> >> evt_test_ethdev_setup_t ethdev_setup; >> >> evt_test_eventdev_setup_t eventdev_setup; >> >> + evt_test_cryptodev_setup_t cryptodev_setup; >> >> evt_test_launch_lcores_t launch_lcores; >> >> evt_test_result_t test_result; >> >> evt_test_eventdev_destroy_t eventdev_destroy; >> >> evt_test_ethdev_destroy_t ethdev_destroy; >> >> + evt_test_cryptodev_destroy_t cryptodev_destroy; >> >> evt_test_mempool_destroy_t mempool_destroy; >> >> evt_test_destroy_t test_destroy; >> >> }; >> >> diff --git a/app/test-eventdev/test_perf_atq.c b/app/test- >> >> eventdev/test_perf_atq.c index 8fd51004ee..dbb1dc8c0c 100644 >> >> --- a/app/test-eventdev/test_perf_atq.c >> >> +++ b/app/test-eventdev/test_perf_atq.c >> >> @@ -48,6 +48,22 @@ perf_atq_worker(void *arg, const int >> >> enable_fwd_latency) >> >> continue; >> >> } >> >> >> >> + if (prod_crypto_type && >> >> + (ev.event_type =3D=3D RTE_EVENT_TYPE_CRYPTODEV)) { >> >> + struct rte_crypto_op *op =3D ev.event_ptr; >> >> + >> >> + if (op->status =3D=3D RTE_CRYPTO_OP_STATUS_SUCCESS) { >> >> + if (op->sym->m_dst =3D=3D NULL) >> >> + ev.event_ptr =3D op->sym->m_src; >> >> + else >> >> + ev.event_ptr =3D op->sym->m_dst; >> >> + rte_crypto_op_free(op); >> >> + } else { >> >> + rte_crypto_op_free(op); >> >> + continue; >> >> + } >> >> + } >> >> + >> >> if (enable_fwd_latency && !prod_timer_type) >> >> /* first stage in pipeline, mark ts to compute fwd latency */ >> >> atq_mark_fwd_latency(&ev); >> >> @@ -87,6 +103,25 @@ perf_atq_worker_burst(void *arg, const int >> >> enable_fwd_latency) >> >> } >> >> >> >> for (i =3D 0; i < nb_rx; i++) { >> >> + if (prod_crypto_type && >> >> + (ev[i].event_type =3D=3D RTE_EVENT_TYPE_CRYPTODEV)) >> >> { >> >> + struct rte_crypto_op *op =3D ev[i].event_ptr; >> >> + >> >> + if (op->status =3D=3D >> >> + RTE_CRYPTO_OP_STATUS_SUCCESS) { >> >> + if (op->sym->m_dst =3D=3D NULL) >> >> + ev[i].event_ptr =3D >> >> + op->sym->m_src; >> >> + else >> >> + ev[i].event_ptr =3D >> >> + op->sym->m_dst; >> >> + rte_crypto_op_free(op); >> >> + } else { >> >> + rte_crypto_op_free(op); >> >> + continue; >> >> + } >> >> + } >> >> + >> >> if (enable_fwd_latency && !prod_timer_type) { >> >> rte_prefetch0(ev[i+1].event_ptr); >> >> /* first stage in pipeline. >> >> @@ -254,6 +289,20 @@ perf_atq_eventdev_setup(struct evt_test *test, >> >> struct evt_options *opt) >> >> return ret; >> >> } >> >> } >> >> + } else if (opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR) = { >> >> + uint8_t cdev_id, cdev_count; >> >> + >> >> + cdev_count =3D rte_cryptodev_count(); >> >> + for (cdev_id =3D 0; cdev_id < cdev_count; cdev_id++) { >> >> + ret =3D rte_cryptodev_start(cdev_id); >> >> + if (ret) { >> >> + evt_err("Failed to start cryptodev %u", >> >> + cdev_id); >> >> + return ret; >> >> + } >> >> + } >> >> + >> >> + rte_event_crypto_adapter_start(t->crypto_adptr.id); >> >> } >> >> >> >> return 0; >> >> @@ -295,12 +344,14 @@ static const struct evt_test_ops perf_atq =3D = { >> >> .opt_dump =3D perf_atq_opt_dump, >> >> .test_setup =3D perf_test_setup, >> >> .ethdev_setup =3D perf_ethdev_setup, >> >> + .cryptodev_setup =3D perf_cryptodev_setup, >> >> .mempool_setup =3D perf_mempool_setup, >> >> .eventdev_setup =3D perf_atq_eventdev_setup, >> >> .launch_lcores =3D perf_atq_launch_lcores, >> >> .eventdev_destroy =3D perf_eventdev_destroy, >> >> .mempool_destroy =3D perf_mempool_destroy, >> >> .ethdev_destroy =3D perf_ethdev_destroy, >> >> + .cryptodev_destroy =3D perf_cryptodev_destroy, >> >> .test_result =3D perf_test_result, >> >> .test_destroy =3D perf_test_destroy, >> >> }; >> >> diff --git a/app/test-eventdev/test_perf_common.c b/app/test- >> >> eventdev/test_perf_common.c index 9b73874151..7b964ab8dc 100644 >> >> --- a/app/test-eventdev/test_perf_common.c >> >> +++ b/app/test-eventdev/test_perf_common.c >> >> @@ -6,6 +6,8 @@ >> >> >> >> #include "test_perf_common.h" >> >> >> >> +#define NB_CRYPTODEV_DESCRIPTORS 128 >> >> + >> >> int >> >> perf_test_result(struct evt_test *test, struct evt_options *opt) { >> >> @@ -272,6 >> >> +274,121 @@ perf_event_timer_producer_burst(void *arg) >> >> return 0; >> >> } >> >> >> >> +static inline void >> >> +crypto_adapter_enq_op_new(struct prod_data *p) { >> >> + struct test_perf *t =3D p->t; >> >> + const uint32_t nb_flows =3D t->nb_flows; >> >> + const uint64_t nb_pkts =3D t->nb_pkts; >> >> + struct rte_mempool *pool =3D t->pool; >> >> + struct rte_crypto_sym_op *sym_op; >> >> + struct evt_options *opt =3D t->opt; >> >> + uint32_t flow_counter =3D 0; >> >> + struct rte_crypto_op *op; >> >> + struct rte_mbuf *m; >> >> + uint64_t count =3D 0; >> >> + uint16_t len; >> >> + >> >> + if (opt->verbose_level > 1) >> >> + printf("%s(): lcore %d queue %d cdev_id %u cdev_qp_id %u\n", >> >> + __func__, rte_lcore_id(), p->queue_id, p->cdev_id, >> >> + p->cdev_qp_id); >> >> + >> >> + len =3D opt->mbuf_sz ? opt->mbuf_sz : RTE_ETHER_MIN_LEN; >> >> + >> >> + while (count < nb_pkts && t->done =3D=3D false) { >> >> + m =3D rte_pktmbuf_alloc(pool); >> >> + if (m =3D=3D NULL) >> >> + continue; >> >> + >> >> + rte_pktmbuf_append(m, len); >> >> + op =3D rte_crypto_op_alloc(t->crypto_adptr.op_pool, >> >> + RTE_CRYPTO_OP_TYPE_SYMMETRIC); >> >> + sym_op =3D op->sym; >> >> + sym_op->m_src =3D m; >> >> + sym_op->cipher.data.offset =3D 0; >> >> + sym_op->cipher.data.length =3D len; >> >> + rte_crypto_op_attach_sym_session( >> >> + op, p->crypto_sess[flow_counter++ % nb_flows]); >> >> + while (rte_cryptodev_enqueue_burst(p->cdev_id, p- >> >> >cdev_qp_id, >> >> + &op, 1) !=3D 1) { >> >> + if (t->done) >> >> + break; >> >> + rte_pause(); >> >> + } >> >> + count++; >> >> + } >> >> +} >> >> + >> >> +static inline void >> >> +crypto_adapter_enq_op_fwd(struct prod_data *p) { >> >> + const uint8_t dev_id =3D p->dev_id; >> >> + const uint8_t port =3D p->port_id; >> >> + struct test_perf *t =3D p->t; >> >> + const uint32_t nb_flows =3D t->nb_flows; >> >> + const uint64_t nb_pkts =3D t->nb_pkts; >> >> + struct rte_mempool *pool =3D t->pool; >> >> + struct evt_options *opt =3D t->opt; >> >> + struct rte_crypto_sym_op *sym_op; >> >> + uint32_t flow_counter =3D 0; >> >> + struct rte_crypto_op *op; >> >> + struct rte_event ev; >> >> + struct rte_mbuf *m; >> >> + uint64_t count =3D 0; >> >> + uint16_t len; >> >> + >> >> + if (opt->verbose_level > 1) >> >> + printf("%s(): lcore %d port %d queue %d cdev_id %u cdev_qp_id >> >> %u\n", >> >> + __func__, rte_lcore_id(), port, p->queue_id, p->cdev_id, >> >> + p->cdev_qp_id); >> >> + >> >> + ev.event =3D 0; >> >> + ev.op =3D RTE_EVENT_OP_NEW; >> >> + ev.queue_id =3D p->queue_id; >> >> + ev.sched_type =3D RTE_SCHED_TYPE_ATOMIC; >> >> + ev.event_type =3D RTE_EVENT_TYPE_CPU; >> >> + len =3D opt->mbuf_sz ? opt->mbuf_sz : RTE_ETHER_MIN_LEN; >> >> + >> >> + while (count < nb_pkts && t->done =3D=3D false) { >> >> + m =3D rte_pktmbuf_alloc(pool); >> >> + if (m =3D=3D NULL) >> >> + continue; >> >> + >> >> + rte_pktmbuf_append(m, len); >> >> + op =3D rte_crypto_op_alloc(t->crypto_adptr.op_pool, >> >> + RTE_CRYPTO_OP_TYPE_SYMMETRIC); >> >> + sym_op =3D op->sym; >> >> + sym_op->m_src =3D m; >> >> + sym_op->cipher.data.offset =3D 0; >> >> + sym_op->cipher.data.length =3D len; >> >> + rte_crypto_op_attach_sym_session( >> >> + op, p->crypto_sess[flow_counter++ % nb_flows]); >> >> + ev.event_ptr =3D op; >> >> + while (rte_event_crypto_adapter_enqueue(dev_id, port, &ev, 1) >> >> !=3D >> >> + 1) { >> >> + if (t->done) >> >> + break; >> >> + rte_pause(); >> >> + } >> >> + count++; >> >> + } >> >> +} >> >> + >> >> +static inline int >> >> +perf_event_crypto_producer(void *arg) { >> >> + struct prod_data *p =3D arg; >> >> + struct evt_options *opt =3D p->t->opt; >> >> + >> >> + if (opt->crypto_adptr_mode =3D=3D >> >> RTE_EVENT_CRYPTO_ADAPTER_OP_NEW) >> >> + crypto_adapter_enq_op_new(p); >> >> + else >> >> + crypto_adapter_enq_op_fwd(p); >> >> + >> >> + return 0; >> >> +} >> >> + >> >> static int >> >> perf_producer_wrapper(void *arg) >> >> { >> >> @@ -298,6 +415,8 @@ perf_producer_wrapper(void *arg) >> >> else if (t->opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_TIMER_ADPTR >> && >> >> t->opt->timdev_use_burst) >> >> return perf_event_timer_producer_burst(arg); >> >> + else if (t->opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR) >> >> + return perf_event_crypto_producer(arg); >> >> return 0; >> >> } >> >> >> >> @@ -405,8 +524,10 @@ perf_launch_lcores(struct evt_test *test, struct >> >> evt_options *opt, >> >> if (remaining <=3D 0) { >> >> t->result =3D EVT_TEST_SUCCESS; >> >> if (opt->prod_type =3D=3D EVT_PROD_TYPE_SYNT >> >> || >> >> - opt->prod_type =3D=3D >> >> - >> >> EVT_PROD_TYPE_EVENT_TIMER_ADPTR) { >> >> + opt->prod_type =3D=3D >> >> + >> >> EVT_PROD_TYPE_EVENT_TIMER_ADPTR || >> >> + opt->prod_type =3D=3D >> >> + >> >> EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR) { >> >> t->done =3D true; >> >> break; >> >> } >> >> @@ -415,7 +536,8 @@ perf_launch_lcores(struct evt_test *test, struct >> >> evt_options *opt, >> >> >> >> if (new_cycles - dead_lock_cycles > dead_lock_sample && >> >> (opt->prod_type =3D=3D EVT_PROD_TYPE_SYNT || >> >> - opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_TIMER_ADPTR)) >> >> { >> >> + opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_TIMER_ADPTR >> >> || >> >> + opt->prod_type =3D=3D >> >> EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR)) { >> >> remaining =3D t->outstand_pkts - processed_pkts(t); >> >> if (dead_lock_remaining =3D=3D remaining) { >> >> rte_event_dev_dump(opt->dev_id, stdout); >> @@ -537,6 +659,96 @@ >> >> perf_event_timer_adapter_setup(struct test_perf *t) >> >> return 0; >> >> } >> >> >> >> +static int >> >> +perf_event_crypto_adapter_setup(struct test_perf *t, >> >> + struct rte_event_port_conf port_conf) { >> >> + struct evt_options *opt =3D t->opt; >> >> + uint8_t cdev_id, cdev_count; >> >> + int ret; >> >> + >> >> + t->crypto_adptr.id =3D 0; >> >> + ret =3D rte_event_crypto_adapter_create(t->crypto_adptr.id, opt->de= v_id, >> >> + &port_conf, 0); >> >> + if (ret) >> >> + return ret; >> >> + >> >> + cdev_count =3D rte_cryptodev_count(); >> >> + for (cdev_id =3D 0; cdev_id < cdev_count; cdev_id++) { >> >> + uint32_t cap; >> >> + >> >> + ret =3D rte_event_crypto_adapter_caps_get(opt->dev_id, >> >> cdev_id, >> >> + &cap); >> >> + if (ret) { >> >> + evt_err("Failed to get crypto adapter capabilities"); >> >> + return ret; >> >> + } >> >> + >> >> + if (((opt->crypto_adptr_mode =3D=3D >> >> + RTE_EVENT_CRYPTO_ADAPTER_OP_NEW) && >> >> + !(cap & >> >> + >> >> RTE_EVENT_CRYPTO_ADAPTER_CAP_INTERNAL_PORT_OP_NEW)) || >> >> + ((opt->crypto_adptr_mode =3D=3D >> >> + RTE_EVENT_CRYPTO_ADAPTER_OP_FORWARD) && >> >> + !(cap & >> >> + >> >> RTE_EVENT_CRYPTO_ADAPTER_CAP_INTERNAL_PORT_OP_FWD))) { >> >> + evt_err("crypto adapter %s mode unsupported\n", >> >> + opt->crypto_adptr_mode ? "OP_FORWARD" : >> >> + "OP_NEW"); >> >> + return -EINVAL; >> >> + } >> >> + >> >> + if (!(cap & >> >> + >> >> RTE_EVENT_CRYPTO_ADAPTER_CAP_SESSION_PRIVATE_DATA)) { >> >> + evt_err("Storing crypto session not supported"); >> >> + return -EINVAL; >> >> + } >> >> + >> >> + if (cap & >> >> + >> >> RTE_EVENT_CRYPTO_ADAPTER_CAP_INTERNAL_PORT_QP_EV_BIND) { >> >> + struct rte_event response_info; >> >> + >> >> + response_info.event =3D 0; >> >> + ret =3D rte_event_crypto_adapter_queue_pair_add( >> >> + t->crypto_adptr.id, cdev_id, -1, >> >> + &response_info); >> >> + } else { >> >> + ret =3D rte_event_crypto_adapter_queue_pair_add( >> >> + t->crypto_adptr.id, cdev_id, -1, NULL); >> >> + } >> >> + if (ret) >> >> + return ret; >> >> + } >> >> + >> >> + return 0; >> >> +} >> >> + >> >> +static struct rte_cryptodev_sym_session * >> >> +cryptodev_session_create(struct prod_data *p, struct test_perf *t) { >> >> + struct rte_crypto_sym_xform cipher_xform; >> >> + struct rte_cryptodev_sym_session *sess; >> >> + >> >> + cipher_xform.type =3D RTE_CRYPTO_SYM_XFORM_CIPHER; >> >> + cipher_xform.cipher.algo =3D RTE_CRYPTO_CIPHER_NULL; >> >> + cipher_xform.cipher.op =3D RTE_CRYPTO_CIPHER_OP_ENCRYPT; >> >> + cipher_xform.next =3D NULL; >> >> + >> >> + sess =3D rte_cryptodev_sym_session_create(t->crypto_adptr.sess_pool= ); >> >> + if (sess =3D=3D NULL) { >> >> + evt_err("Failed to create sym session"); >> >> + return NULL; >> >> + } >> >> + >> >> + if (rte_cryptodev_sym_session_init(p->cdev_id, sess, &cipher_xform, >> >> + t->crypto_adptr.sess_priv_pool)) { >> >> + evt_err("Failed to init session"); >> >> + return NULL; >> >> + } >> >> + >> >> + return sess; >> >> +} >> >> + >> >> int >> >> perf_event_dev_port_setup(struct evt_test *test, struct evt_options = *opt, >> >> uint8_t stride, uint8_t nb_queues, @@ -598,6 >> +810,55 @@ >> >> perf_event_dev_port_setup(struct evt_test *test, struct evt_options >> >> *opt, >> >> ret =3D perf_event_timer_adapter_setup(t); >> >> if (ret) >> >> return ret; >> >> + } else if (opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR) = { >> >> + uint8_t cdev_id =3D 0; >> >> + uint16_t qp_id =3D 0; >> >> + >> >> + prod =3D 0; >> >> + for (; port < perf_nb_event_ports(opt); port++) { >> >> + struct rte_cryptodev_sym_session *crypto_sess; >> >> + union rte_event_crypto_metadata m_data; >> >> + struct prod_data *p =3D &t->prod[port]; >> >> + uint32_t flow_id; >> >> + >> >> + if (qp_id =3D=3D rte_cryptodev_queue_pair_count(cdev_id)) >> >> { >> >> + cdev_id++; >> >> + qp_id =3D 0; >> >> + } >> >> + >> >> + p->dev_id =3D opt->dev_id; >> >> + p->port_id =3D port; >> >> + p->queue_id =3D prod * stride; >> >> + p->cdev_id =3D cdev_id; >> >> + p->cdev_qp_id =3D qp_id; >> >> + p->crypto_sess =3D rte_zmalloc_socket( >> >> + NULL, sizeof(crypto_sess) * t->nb_flows, >> >> + RTE_CACHE_LINE_SIZE, opt->socket_id); >> >> + >> >> + m_data.request_info.cdev_id =3D p->cdev_id; >> >> + m_data.request_info.queue_pair_id =3D p->cdev_qp_id; >> >> + m_data.response_info.op =3D RTE_EVENT_OP_NEW; >> >> + m_data.response_info.sched_type =3D >> >> RTE_SCHED_TYPE_ATOMIC; >> >> + m_data.response_info.event_type =3D >> >> RTE_EVENT_TYPE_CPU; >> >> + m_data.response_info.queue_id =3D p->queue_id; >> >> + for (flow_id =3D 0; flow_id < t->nb_flows; flow_id++) { >> >> + crypto_sess =3D cryptodev_session_create(p, t); >> >> + if (crypto_sess =3D=3D NULL) >> >> + return -ENOMEM; >> >> + >> >> + m_data.response_info.flow_id =3D flow_id; >> >> + rte_cryptodev_sym_session_set_user_data( >> >> + crypto_sess, &m_data, >> >> sizeof(m_data)); >> >> + p->crypto_sess[flow_id] =3D crypto_sess; >> >> + } >> >> + p->t =3D t; >> >> + qp_id++; >> >> + prod++; >> >> + } >> >> + >> >> + ret =3D perf_event_crypto_adapter_setup(t, *port_conf); >> >> + if (ret) >> >> + return ret; >> >> } else { >> >> prod =3D 0; >> >> for ( ; port < perf_nb_event_ports(opt); port++) { @@ -659,7 >> >> +920,8 @@ perf_opt_check(struct evt_options *opt, uint64_t nb_queues) >> >> } >> >> >> >> if (opt->prod_type =3D=3D EVT_PROD_TYPE_SYNT || >> >> - opt->prod_type =3D=3D >> >> EVT_PROD_TYPE_EVENT_TIMER_ADPTR) { >> >> + opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_TIMER_ADPTR || >> >> + opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR) { >> >> /* Validate producer lcores */ >> >> if (evt_lcores_has_overlap(opt->plcores, >> >> rte_get_main_lcore())) { >> >> @@ -767,8 +1029,7 @@ perf_ethdev_setup(struct evt_test *test, struct >> >> evt_options *opt) >> >> }, >> >> }; >> >> >> >> - if (opt->prod_type =3D=3D EVT_PROD_TYPE_SYNT || >> >> - opt->prod_type =3D=3D >> >> EVT_PROD_TYPE_EVENT_TIMER_ADPTR) >> >> + if (opt->prod_type !=3D EVT_PROD_TYPE_ETH_RX_ADPTR) >> >> return 0; >> >> >> >> if (!rte_eth_dev_count_avail()) { >> >> @@ -841,6 +1102,139 @@ void perf_ethdev_destroy(struct evt_test >> >> *test, struct evt_options *opt) >> >> } >> >> } >> >> >> >> +int >> >> +perf_cryptodev_setup(struct evt_test *test, struct evt_options *opt)= { >> >> + uint8_t cdev_count, cdev_id, nb_plcores, nb_qps; >> >> + struct test_perf *t =3D evt_test_priv(test); >> >> + unsigned int max_session_size; >> >> + uint32_t nb_sessions; >> >> + int ret; >> >> + >> >> + if (opt->prod_type !=3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR) >> >> + return 0; >> >> + >> >> + cdev_count =3D rte_cryptodev_count(); >> >> + if (cdev_count =3D=3D 0) { >> >> + evt_err("No crypto devices available\n"); >> >> + return -ENODEV; >> >> + } >> >> + >> >> + t->crypto_adptr.op_pool =3D rte_crypto_op_pool_create( >> >> + "crypto_op_pool", RTE_CRYPTO_OP_TYPE_SYMMETRIC, opt- >> >> >pool_sz, >> >> + 128, 0, rte_socket_id()); >> >> + if (t->crypto_adptr.op_pool =3D=3D NULL) { >> >> + evt_err("Failed to create crypto op pool"); >> >> + return -ENOMEM; >> >> + } >> >> + >> >> + nb_sessions =3D evt_nr_active_lcores(opt->plcores) * t->nb_flows; >> >> + t->crypto_adptr.sess_pool =3D rte_cryptodev_sym_session_pool_create= ( >> >> + "ca_sess_pool", nb_sessions, 0, 0, >> >> + sizeof(union rte_event_crypto_metadata), SOCKET_ID_ANY); >> >> + if (t->crypto_adptr.sess_pool =3D=3D NULL) { >> >> + evt_err("Failed to create sym session pool"); >> >> + ret =3D -ENOMEM; >> >> + goto err; >> >> + } >> >> + >> >> + max_session_size =3D 0; >> >> + for (cdev_id =3D 0; cdev_id < cdev_count; cdev_id++) { >> >> + unsigned int session_size; >> >> + >> >> + session_size =3D >> >> + rte_cryptodev_sym_get_private_session_size(cdev_id); >> >> + if (session_size > max_session_size) >> >> + max_session_size =3D session_size; >> >> + } >> >> + max_session_size +=3D sizeof(union rte_event_crypto_metadata); >> >> + t->crypto_adptr.sess_priv_pool =3D rte_mempool_create( >> >> + "ca_sess_priv_pool", nb_sessions, max_session_size, 0, 0, >> >> NULL, >> >> + NULL, NULL, NULL, SOCKET_ID_ANY, 0); >> >> + if (t->crypto_adptr.sess_priv_pool =3D=3D NULL) { >> >> + evt_err("failed to create sym session private pool"); >> >> + ret =3D -ENOMEM; >> >> + goto err; >> >> + } >> >> + >> >> + nb_plcores =3D evt_nr_active_lcores(opt->plcores); >> >> + nb_qps =3D (nb_plcores % cdev_count) ? (nb_plcores / cdev_count) + = 1 : >> >> + nb_plcores / cdev_count; >> >> + for (cdev_id =3D 0; cdev_id < cdev_count; cdev_id++) { >> >> + struct rte_cryptodev_qp_conf qp_conf; >> >> + struct rte_cryptodev_config conf; >> >> + struct rte_cryptodev_info info; >> >> + int qp_id; >> >> + >> >> + rte_cryptodev_info_get(cdev_id, &info); >> >> + if (nb_qps > info.max_nb_queue_pairs) { >> >> + evt_err("Not enough queue pairs per cryptodev (%u)", >> >> + nb_qps); >> >> + ret =3D -EINVAL; >> >> + goto err; >> >> + } >> >> + >> >> + conf.nb_queue_pairs =3D nb_qps; >> >> + conf.socket_id =3D SOCKET_ID_ANY; >> >> + conf.ff_disable =3D RTE_CRYPTODEV_FF_SECURITY; >> >> + >> >> + ret =3D rte_cryptodev_configure(cdev_id, &conf); >> >> + if (ret) { >> >> + evt_err("Failed to configure cryptodev (%u)", cdev_id); >> >> + goto err; >> >> + } >> >> + >> >> + qp_conf.nb_descriptors =3D NB_CRYPTODEV_DESCRIPTORS; >> >> + qp_conf.mp_session =3D t->crypto_adptr.sess_pool; >> >> + qp_conf.mp_session_private =3D t->crypto_adptr.sess_priv_pool; >> >> + >> >> + for (qp_id =3D 0; qp_id < conf.nb_queue_pairs; qp_id++) { >> >> + ret =3D rte_cryptodev_queue_pair_setup( >> >> + cdev_id, qp_id, &qp_conf, >> >> + rte_cryptodev_socket_id(cdev_id)); >> >> + if (ret) { >> >> + evt_err("Failed to setup queue pairs on >> >> cryptodev %u\n", >> >> + cdev_id); >> >> + goto err; >> >> + } >> >> + } >> >> + } >> >> + >> >> + return 0; >> >> +err: >> >> + rte_mempool_free(t->crypto_adptr.op_pool); >> >> + rte_mempool_free(t->crypto_adptr.sess_pool); >> >> + rte_mempool_free(t->crypto_adptr.sess_priv_pool); >> >> + >> >> + return ret; >> >> +} >> >> + >> >> +void >> >> +perf_cryptodev_destroy(struct evt_test *test, struct evt_options >> >> +*opt) { >> >> + uint8_t cdev_id, cdev_count =3D rte_cryptodev_count(); >> >> + struct test_perf *t =3D evt_test_priv(test); >> >> + >> >> + RTE_SET_USED(opt); >> >> + >> >> + if (opt->prod_type !=3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR) >> >> + return; >> >> + >> >> + rte_event_crypto_adapter_stop(t->crypto_adptr.id); >> >> + >> >> + for (cdev_id =3D 0; cdev_id < cdev_count; cdev_id++) { >> >> + rte_cryptodev_stop(cdev_id); >> >> + rte_event_crypto_adapter_queue_pair_del(t->crypto_adptr.id, >> >> + cdev_id, -1); >> >> + } >> >> + >> >> + rte_event_crypto_adapter_free(t->crypto_adptr.id); >> >> + >> >> + rte_mempool_free(t->crypto_adptr.op_pool); >> >> + rte_mempool_free(t->crypto_adptr.sess_pool); >> >> + rte_mempool_free(t->crypto_adptr.sess_priv_pool); >> >> +} >> >> + >> >> int >> >> perf_mempool_setup(struct evt_test *test, struct evt_options *opt) >> >> { diff --git a/app/test-eventdev/test_perf_common.h b/app/test- >> >> eventdev/test_perf_common.h index 14dcf80429..d9ad3a0b0d 100644 >> >> --- a/app/test-eventdev/test_perf_common.h >> >> +++ b/app/test-eventdev/test_perf_common.h >> >> @@ -9,9 +9,11 @@ >> >> #include >> >> #include >> >> >> >> +#include >> >> #include >> >> #include >> >> #include >> >> +#include >> >> #include #include >> >> #include @@ -37,9 +39,18 >> >> @@ struct prod_data { >> >> uint8_t dev_id; >> >> uint8_t port_id; >> >> uint8_t queue_id; >> >> + uint8_t cdev_id; >> >> + uint16_t cdev_qp_id; >> >> + struct rte_cryptodev_sym_session **crypto_sess; >> >> struct test_perf *t; >> >> } __rte_cache_aligned; >> >> >> >> +struct crypto_adptr_data { >> >> + uint8_t id; >> >> + struct rte_mempool *op_pool; >> >> + struct rte_mempool *sess_pool; >> >> + struct rte_mempool *sess_priv_pool; }; >> >> >> >> struct test_perf { >> >> /* Don't change the offset of "done". Signal handler use this >> >> memory @@ -58,6 +69,7 @@ struct test_perf { >> >> uint8_t sched_type_list[EVT_MAX_STAGES] __rte_cache_aligned; >> >> struct rte_event_timer_adapter *timer_adptr[ >> >> RTE_EVENT_TIMER_ADAPTER_NUM_MAX] >> >> __rte_cache_aligned; >> >> + struct crypto_adptr_data crypto_adptr; >> >> } __rte_cache_aligned; >> >> >> >> struct perf_elt { >> >> @@ -81,6 +93,8 @@ struct perf_elt { >> >> const uint8_t port =3D w->port_id;\ >> >> const uint8_t prod_timer_type =3D \ >> >> opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_TIMER_ADPTR;\ >> >> + const uint8_t prod_crypto_type =3D \ >> >> + opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR;\ >> >> uint8_t *const sched_type_list =3D &t->sched_type_list[0];\ >> >> struct rte_mempool *const pool =3D t->pool;\ >> >> const uint8_t nb_stages =3D t->opt->nb_stages;\ @@ -154,6 +168,7 @@ >> >> int perf_test_result(struct evt_test *test, struct evt_options *opt); >> >> int perf_opt_check(struct evt_options *opt, uint64_t nb_queues); int >> >> perf_test_setup(struct evt_test *test, struct evt_options *opt); int >> >> perf_ethdev_setup(struct evt_test *test, struct evt_options *opt); >> >> +int perf_cryptodev_setup(struct evt_test *test, struct evt_options >> >> +*opt); >> >> int perf_mempool_setup(struct evt_test *test, struct evt_options >> >> *opt); int perf_event_dev_port_setup(struct evt_test *test, struct >> evt_options *opt, >> >> uint8_t stride, uint8_t nb_queues, @@ -164,6 >> +179,7 @@ int >> >> perf_launch_lcores(struct evt_test *test, struct evt_options *opt, >> >> void perf_opt_dump(struct evt_options *opt, uint8_t nb_queues); void >> >> perf_test_destroy(struct evt_test *test, struct evt_options *opt); >> >> void perf_eventdev_destroy(struct evt_test *test, struct evt_options >> >> *opt); >> >> +void perf_cryptodev_destroy(struct evt_test *test, struct >> >> +evt_options *opt); >> >> void perf_ethdev_destroy(struct evt_test *test, struct evt_options >> >> *opt); void perf_mempool_destroy(struct evt_test *test, struct >> >> evt_options *opt); >> >> >> >> diff --git a/app/test-eventdev/test_perf_queue.c b/app/test- >> >> eventdev/test_perf_queue.c index f4ea3a795f..5657c8b3cd 100644 >> >> --- a/app/test-eventdev/test_perf_queue.c >> >> +++ b/app/test-eventdev/test_perf_queue.c >> >> @@ -49,6 +49,23 @@ perf_queue_worker(void *arg, const int >> >> enable_fwd_latency) >> >> rte_pause(); >> >> continue; >> >> } >> >> + >> >> + if (prod_crypto_type && >> >> + (ev.event_type =3D=3D RTE_EVENT_TYPE_CRYPTODEV)) { >> >> + struct rte_crypto_op *op =3D ev.event_ptr; >> >> + >> >> + if (op->status =3D=3D RTE_CRYPTO_OP_STATUS_SUCCESS) { >> >> + if (op->sym->m_dst =3D=3D NULL) >> >> + ev.event_ptr =3D op->sym->m_src; >> >> + else >> >> + ev.event_ptr =3D op->sym->m_dst; >> >> + rte_crypto_op_free(op); >> >> + } else { >> >> + rte_crypto_op_free(op); >> >> + continue; >> >> + } >> >> + } >> >> + >> >> if (enable_fwd_latency && !prod_timer_type) >> >> /* first q in pipeline, mark timestamp to compute fwd latency */ >> >> mark_fwd_latency(&ev, nb_stages); @@ -88,6 +105,25 >> @@ >> >> perf_queue_worker_burst(void *arg, const int >> >> enable_fwd_latency) >> >> } >> >> >> >> for (i =3D 0; i < nb_rx; i++) { >> >> + if (prod_crypto_type && >> >> + (ev[i].event_type =3D=3D RTE_EVENT_TYPE_CRYPTODEV)) >> >> { >> >> + struct rte_crypto_op *op =3D ev[i].event_ptr; >> >> + >> >> + if (op->status =3D=3D >> >> + RTE_CRYPTO_OP_STATUS_SUCCESS) { >> >> + if (op->sym->m_dst =3D=3D NULL) >> >> + ev[i].event_ptr =3D >> >> + op->sym->m_src; >> >> + else >> >> + ev[i].event_ptr =3D >> >> + op->sym->m_dst; >> >> + rte_crypto_op_free(op); >> >> + } else { >> >> + rte_crypto_op_free(op); >> >> + continue; >> >> + } >> >> + } >> >> + >> >> if (enable_fwd_latency && !prod_timer_type) { >> >> rte_prefetch0(ev[i+1].event_ptr); >> >> /* first queue in pipeline. >> >> @@ -269,6 +305,20 @@ perf_queue_eventdev_setup(struct evt_test *test, >> >> struct evt_options *opt) >> >> return ret; >> >> } >> >> } >> >> + } else if (opt->prod_type =3D=3D EVT_PROD_TYPE_EVENT_CRYPTO_ADPTR) = { >> >> + uint8_t cdev_id, cdev_count; >> >> + >> >> + cdev_count =3D rte_cryptodev_count(); >> >> + for (cdev_id =3D 0; cdev_id < cdev_count; cdev_id++) { >> >> + ret =3D rte_cryptodev_start(cdev_id); >> >> + if (ret) { >> >> + evt_err("Failed to start cryptodev %u", >> >> + cdev_id); >> >> + return ret; >> >> + } >> >> + } >> >> + >> >> + rte_event_crypto_adapter_start(t->crypto_adptr.id); >> >> } >> >> >> >> return 0; >> >> @@ -311,11 +361,13 @@ static const struct evt_test_ops perf_queue =3D= { >> >> .test_setup =3D perf_test_setup, >> >> .mempool_setup =3D perf_mempool_setup, >> >> .ethdev_setup =3D perf_ethdev_setup, >> >> + .cryptodev_setup =3D perf_cryptodev_setup, >> >> .eventdev_setup =3D perf_queue_eventdev_setup, >> >> .launch_lcores =3D perf_queue_launch_lcores, >> >> .eventdev_destroy =3D perf_eventdev_destroy, >> >> .mempool_destroy =3D perf_mempool_destroy, >> >> .ethdev_destroy =3D perf_ethdev_destroy, >> >> + .cryptodev_destroy =3D perf_cryptodev_destroy, >> >> .test_result =3D perf_test_result, >> >> .test_destroy =3D perf_test_destroy, >> >> }; >> >> diff --git a/doc/guides/tools/testeventdev.rst >> >> b/doc/guides/tools/testeventdev.rst >> >> index 48efb9ea6e..f7d813226d 100644 >> >> --- a/doc/guides/tools/testeventdev.rst >> >> +++ b/doc/guides/tools/testeventdev.rst >> >> @@ -120,6 +120,10 @@ The following are the application command-line >> >> options: >> >> >> >> Use burst mode event timer adapter as producer. >> >> >> >> +* ``--prod_type_cryptodev`` >> >> + >> >> + Use crypto device as producer. >> >> + >> >> * ``--timer_tick_nsec`` >> >> >> >> Used to dictate number of nano seconds between bucket >> >> traversal of the @@ -148,6 +152,11 @@ The following are the >> >> application command-line >> >> options: >> >> timeout is out of the supported range of event device it will= be >> >> adjusted to the highest/lowest supported dequeue timeout supp= orted. >> >> >> >> +* ``--crypto_adptr_mode`` >> >> + >> >> + Set crypto adapter mode. Use 0 for OP_NEW (default) and 1 fo= r >> >> + OP_FORWARD mode. >> >> + >> >> * ``--mbuf_sz`` >> >> >> >> Set packet mbuf size. Can be used to configure Jumbo Frames. >> >> Only @@ - >> >> 420,6 +429,7 @@ Supported application command line options are >> following:: >> >> --prod_type_ethdev >> >> --prod_type_timerdev_burst >> >> --prod_type_timerdev >> >> + --prod_type_cryptodev >> >> --prod_enq_burst_sz >> >> --timer_tick_nsec >> >> --max_tmo_nsec >> >> @@ -427,6 +437,7 @@ Supported application command line options are >> >> following:: >> >> --nb_timers >> >> --nb_timer_adptrs >> >> --deq_tmo_nsec >> >> + --crypto_adptr_mode >> >> >> >> Example >> >> ^^^^^^^ >> >> @@ -529,12 +540,14 @@ Supported application command line options are >> >> following:: >> >> --prod_type_ethdev >> >> --prod_type_timerdev_burst >> >> --prod_type_timerdev >> >> + --prod_type_cryptodev >> >> --timer_tick_nsec >> >> --max_tmo_nsec >> >> --expiry_nsec >> >> --nb_timers >> >> --nb_timer_adptrs >> >> --deq_tmo_nsec >> >> + --crypto_adptr_mode >> >> >> >> Example >> >> ^^^^^^^ >> >> -- >> >> 2.25.1