From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B863AA0C4B; Thu, 21 Oct 2021 14:30:28 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9E214411FB; Thu, 21 Oct 2021 14:30:28 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A9B154117A for ; Thu, 21 Oct 2021 14:30:27 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19LAkkCf021121; Thu, 21 Oct 2021 05:30:20 -0700 Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2104.outbound.protection.outlook.com [104.47.58.104]) by mx0b-0016f401.pphosted.com with ESMTP id 3bu6rhrg8s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Oct 2021 05:30:20 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WiU1q3Ajp+YXSfLTVUZUTQFwlfmzjdODojj1t2qifsYslQMAPw5pkCP59hfJ7vycv7LKUl+f89YWuFbTyR/bVxROL/JyZ1OZBhhDRzj1n7uaV8szC/DsP/bBxIY6WzXLF2L2WAcs5hW/6H6i22+AnnfmdpQRZZxd7GN0WaceifnMqPWNtFU1RwxYJJWdhBq0huG7MWRIX+2aYTx1YR/iQkDNjizztEiwxU2feJD36a7HOKZ68iEsIBHZB+yWMddsYL0PGLvuP7d9XEU8r1Rf3nEef06UAcMVRZ6zKCLfEj1kDK2bB7XQQUqqJQZrIKIOFc6eEqxum3+/x9aa5vKVqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eyRt7/vVMKuCUzDxWQ8/huLF5167ARgnsKGplqZgeh0=; b=BFfW3t4Qpd9YpnOOEKI0n7JjTEnrujAhYLcLoAlwBpwm46mHCd/60w3o0x2/1owr6KYngfc99F/aZyuTpFfe+P+Mk43xtubws66/4cMk+g71mLI4VBtzKex/Lnq9cK/Vo+SplTHZcGAqekg3mFof1UQDY/8xYXCHGaRmGitRyTT1Ywein96+s5FPvO7yO6WB6dhwDgLZ2pfCmsBN292PLW4H+imhk84SsKJjcHWFVr/zfaSYlTAkLnGE/sdfKcLQXsSLmwKx9DvITHnFHHgYyxIAb326rs/FCcbXjI3oTqMFS9pgadp/G7qh0umxpsa0PYfB8335TMtghTAUV+73FA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eyRt7/vVMKuCUzDxWQ8/huLF5167ARgnsKGplqZgeh0=; b=CUAqkdlf18DzD1cMCUOUG6OFDmXwv7frKd6nFut1OSP19ThfhS4I2qJxdVr82zkn79SSSqRLnpEMbEhq+frNi7Cy0le6ETubQhw48A17UelAjyNz6Yoxbbk0Vh1GuuxmoRvyZqq6COohS1vzh81Son+kJiJkv/XD2VGHWRXkcsc= Received: from CO6PR18MB4484.namprd18.prod.outlook.com (2603:10b6:5:359::9) by CO6PR18MB4436.namprd18.prod.outlook.com (2603:10b6:303:138::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.16; Thu, 21 Oct 2021 12:30:18 +0000 Received: from CO6PR18MB4484.namprd18.prod.outlook.com ([fe80::c41e:707:3f91:71b8]) by CO6PR18MB4484.namprd18.prod.outlook.com ([fe80::c41e:707:3f91:71b8%8]) with mapi id 15.20.4628.018; Thu, 21 Oct 2021 12:30:18 +0000 From: Akhil Goyal To: "Ananyev, Konstantin" , "dev@dpdk.org" , "Zhang, Roy Fan" CC: "thomas@monjalon.net" , "david.marchand@redhat.com" , "hemant.agrawal@nxp.com" , Anoob Joseph , "De Lara Guarch, Pablo" , "Trahe, Fiona" , "Doherty, Declan" , "matan@nvidia.com" , "g.singh@nxp.com" , "jianjay.zhou@huawei.com" , "asomalap@amd.com" , "ruifeng.wang@arm.com" , "Nicolau, Radu" , "ajit.khaparde@broadcom.com" , Nagadheeraj Rottela , Ankur Dwivedi , "Power, Ciara" , "Wang, Haiyue" , "jiawenwu@trustnetic.com" , "jianwang@trustnetic.com" , Jerin Jacob Kollanukkaran , Nithin Kumar Dabilpuram Thread-Topic: [PATCH v3 6/8] cryptodev: rework session framework Thread-Index: AQHXxGgefWweT0oRpE+oJQJPA2AoU6vcSFUAgAC9nRCAAEDUgIAAFFsg Date: Thu, 21 Oct 2021 12:30:18 +0000 Message-ID: References: <20211013192222.1582631-2-gakhil@marvell.com> <20211018213452.2734720-1-gakhil@marvell.com> <20211018213452.2734720-7-gakhil@marvell.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 01ca5841-b796-433b-1f33-08d9948e8ae2 x-ms-traffictypediagnostic: CO6PR18MB4436: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:7691; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: i4U1nbwQCqGWjNP57dqzk97usc3G/J27ULD+8MBLCvxMJ0Ed9LykNpsW4++P9G64tBnvleO7Frl2+G2UZInUUP4fkouohWn9ydluwM7G+zPSkHkTy6zCBUeOUHsgQ8vqvlTLjKYLqwQakQ3r41gWl0GwQw2XPAVyQXq03N6Nc4WDCfiaQi/rT3Ga4C0X88Z60Ls1+PSROP+TulhoAKXtDwEvHAuj3gCUUl12NfirBzAoH+4BKLhCRaRj8QzTzoZwGd0IGs0vO7gi3S9xKT23Ckr8/7pHlyN4XToJVz5c67TpQGdTRe5b1jsbJpS1f2YwOgbUZYCDThpEqKTMYvFJtcIy5s1hwJ87/FXz+zP9gbrme7MvO6PKgMiuI06fLLerp27xMxU7v3uUcvaGqNi0UqKMcpSbH12a06po0DBQffxqLXf7tWAmsHxEq+e++zN1/XGxsCOVbIgR7H98Yp+GdwBvg1om+oMLjfeJ/7V1DsrNSc5ioMc5nMabsKhGaLdaLICFZXZp9dCqmxGNkxOHeGBBrkjAiOn06BXaz/SZNbwnHjp973hIVM+Ss3K3+p1P9zUplBKIPyhTrqyvh5iCYdyOnpOMO+CcG3R/YvDmZtNCr5WY9aAILreAOmFN5PSwz17jjHpIwnsibzpCwSnE5Cat7zkxKMLfgCwD2auAcH6bDLr5anm30tWFAW0TWjKvqijmYxGhiiySRCa8GhhuGw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR18MB4484.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(5660300002)(6506007)(55236004)(71200400001)(186003)(52536014)(7696005)(33656002)(8676002)(83380400001)(508600001)(2906002)(38070700005)(66946007)(316002)(66556008)(66476007)(4326008)(7416002)(107886003)(110136005)(54906003)(8936002)(38100700002)(76116006)(122000001)(66446008)(86362001)(9686003)(55016002)(64756008)(26005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?koK9s717MvOxdRVJ1wVBfrHvuChi1qXsJftZXMO6NQP7n4Ah25XJGtui25Wg?= =?us-ascii?Q?M7mCsjSwqTZnSrAbKdAqje8/Wn1y3724rx/gQmfn2iOFFFONi/dhDQ9cOprW?= =?us-ascii?Q?ZiwZH1Lw+E6JvnzCK+4eLk2V/oW0SBkX820rfJv7s4jbr7TPEsD9k/DhbUJ2?= =?us-ascii?Q?tnZo+UJsz69STIiJK4e0umN/WaaYw5i7DA53LyHmtpWA/J+t7V3xoQMKvGen?= =?us-ascii?Q?o+31UKlmGkUWYAPv+dKP5RzEJCi1WAzoKGubW3stplO1gXbvc+RE40Uvvbbk?= =?us-ascii?Q?hDQ85F8V/bP2Z8jHv5EbflqEmVhtASZ+Rrnrhlc+23aFpe9TftsO7Md8q4qT?= =?us-ascii?Q?coK9ojNvmCWsSl5P2qd69Vd2IONofSre7uGcSFWc02WuPUb9hIvrz1iy0jdS?= =?us-ascii?Q?qLVbpE0I2N20p1CngXIZfQENa9ZCY/tkqZcxs8Ci9IYfvVtCo1W8vhuiXtb3?= =?us-ascii?Q?AOpIydhl7rXxazna2zjqloIFpUkcW3c1mPz97+VxuiPyFr+AY1QKqovaZ2N/?= =?us-ascii?Q?Af2LhxeEnCLcxFmAstVhhdKllnrhTtAYD/0OSF9BJcYdY8WDiZ3dAjY+IT+R?= =?us-ascii?Q?5L5J02s/OT8GQRXbT1zNOpoz1DOZGdD3UhQIlBZ1v260yNLLslz1ej63DJHJ?= =?us-ascii?Q?Po6jCmHaOp/DO9EE6EYH82ik2jhQdoQ1EzZ3LkVuAbsGQ4grEdY+Ajg+L5vj?= =?us-ascii?Q?gT+FDvG4Og04KBOTWa+rEMyX6jy0+JtOof86nlKXmNyAmkBrqM3IJQgBSE65?= =?us-ascii?Q?r6fc1Kn7kd8mblQaFWjYTAB8espUWT3zM4wZwR1anAZopqHPmtvSupusvewh?= =?us-ascii?Q?TscrSv01h+2ZhFs2+cN5htBZZriP808HyC7+tsCiY699P0SWllMssNyH8wwQ?= =?us-ascii?Q?5e8DDkjUdwT7a3IcVjfDMrYYfCnP19ZV8Tl/vDwt3f1rNUUABsFiOmp+GEfG?= =?us-ascii?Q?j595kjWy7PSj/+dC4VRYskZyclKDuQ+5wifzW4YmzTjxU9CPqhQAEiigC4aE?= =?us-ascii?Q?YGD8atAmu6GOJgp+2Vipjd0gFo5oNbHAq7DlmsHlvmZx2DQttIZPKlLnLPzH?= =?us-ascii?Q?Q1LIEb8pfJ0dlpURvKsnHqtGRQSpsLw2oV6nAFEIxVhNrsDUd9VCbtNRG9bQ?= =?us-ascii?Q?twgkqrdTmmq61Nfjvz65b1Ux0QyeLf5an68jDZJilaqLfQuNRqh/tV5HPd5m?= =?us-ascii?Q?SrO7oVrOoFws/WHHeqzdApFdk7YUeKfKXYPyUvz81WXBvDy7CHecgtjjcVHr?= =?us-ascii?Q?z8j9l4U0zcPCYjGIibGHoUjIkBEylxaPSQkAAvjLU0WXyOx01NOE+mqFl3Gc?= =?us-ascii?Q?js770LDikbQL1A0EBVRuLldUVW9qnTW8X5zxqbp+CUH63bYfg5UTHylP2QZ+?= =?us-ascii?Q?bS7/sWj64ws/JA5QVv4c140V9lY4RYgF+ZvD4cI/fHJGp0TN0c6KijfNe8fl?= =?us-ascii?Q?rjOFD4+2elw=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO6PR18MB4484.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 01ca5841-b796-433b-1f33-08d9948e8ae2 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Oct 2021 12:30:18.2562 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: gakhil@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR18MB4436 X-Proofpoint-GUID: gTNYqiZo-dPycE6FDsuHOSN2rnQvC0JD X-Proofpoint-ORIG-GUID: gTNYqiZo-dPycE6FDsuHOSN2rnQvC0JD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-21_03,2021-10-21_02,2020-04-07_01 Subject: Re: [dpdk-dev] [PATCH v3 6/8] cryptodev: rework session framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > The problem is that with new approach you proposed there is no simple way > for PMD to > fulfil that requirement. > In current version of DPDK: > - PMD reports size of private data, note that it reports extra space need= ed > to align its data properly inside provided buffer. > - Then it ss up to higher layer to allocate mempool with elements big eno= ugh > to hold > PMD private data. > - At session init that mempool is passed to PMD sym_session_confgure() an= d > it is > PMD responsibility to allocate buffer (from given mempool) for its priva= te > data > align it properly, and update sess->sess_data[].data. > With this patch: > - PMD still reports size of private data, but now it is cryptodev layer= who > allocates > memory for PMD private data and updates sess->sess_data[].data. >=20 > So PMD simply has no way to allocate/align its private data in a way it l= ikes > to. > Of course it can simply do alignment on the fly for each operation, somet= hing > like: >=20 > void *p =3D get_sym_session_private_data(sess, dev->driver_id); > sess_priv =3D RTE_PTR_ALIGN_FLOOR(p, PMD_SES_ALIGN); >=20 > But it is way too ugly and error-prone. >=20 > Another potential problem with that approach (when cryptodev allocates > memory for > PMD private session data and updates sess->sess_data[].data for it) - it = could > happen > that private data for different PMDs can endup on the same cache-line. > If we'll ever have a case with simultaneous session processing by multipl= e- > devices > it can cause all sorts of performance problems. To resolve above 2 issues(performance and pointer CEIL in PMD), can you che= ck If following diff in library would work? ---------------------------------------------------------------------------= ------------------------- diff --git a/lib/cryptodev/rte_cryptodev.c b/lib/cryptodev/rte_cryptodev.c index 9d5e08bba2..7beb5339ea 100644 --- a/lib/cryptodev/rte_cryptodev.c +++ b/lib/cryptodev/rte_cryptodev.c @@ -1731,12 +1731,13 @@ rte_cryptodev_sym_session_init(uint8_t dev_id, RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->sym_session_configure, -ENOT= SUP); if (sess->sess_data[index].refcnt =3D=3D 0) { - sess->sess_data[index].data =3D (void *)((uint8_t *)sess + + sess->sess_data[index].data =3D RTE_PTR_ALIGN_CEIL( + (void *)((uint8_t *)sess + rte_cryptodev_sym_get_header_session_size()= + - (index * sess->priv_sz)); - sess_iova =3D rte_mempool_virt2iova(sess) + + (index * sess->priv_sz)), RTE_CACHE_LINE_SI= ZE); + sess_iova =3D RTE_ALIGN_CEIL(rte_mempool_virt2iova(sess) + rte_cryptodev_sym_get_header_session_size()= + - (index * sess->priv_sz); + (index * sess->priv_sz), RTE_CACHE_LINE_SIZ= E); ret =3D dev->dev_ops->sym_session_configure(dev, xforms, sess->sess_data[index].data, sess_iova); if (ret < 0) { @@ -1805,7 +1806,7 @@ get_max_sym_sess_priv_sz(void) if (sz > max_sz) max_sz =3D sz; } - return max_sz; + return RTE_ALIGN_CEIL(max_sz,RTE_CACHE_LINE_SIZE); } struct rte_mempool * ---------------------------------------------------------------------------= ------------- >=20 > All in all - these changes for (remove second mempool, change the way we > allocate/setup > session private data) seems premature to me. > So, I think to go ahead with this series (hiding rte_cryptodev_sym_sessio= n) > for 21.11 > we need to drop changes for sess_data[] management allocation and keep > only changes > directly related to hide sym_session. > My apologies for not reviewing/testing properly that series earlier. >=20 The changes are huge and will affect a lot of people. We needed help >From all the pmd owners to look into this. We can drop this series, citing not enough review happened, but the issues that were raised could have been resolved till RC2 for the cases that are c= urrently broken. However, there is one more issue that was not highlighted here was that, in= case of Scheduler PMD there are a lot of inappropriate stuff which hampers these ch= anges. Because of which we will end up reserving huge memory space which will be u= nused if scheduler PMD is compiled in. We can have a simple single API for session creation similar to rte_securit= y. And let scheduler PMD manage all the memory by itself for all the PMDs whic= h it want to schedule. We can defer this series for now, and can work on Asymmetric crypto first (probably in 22.02) which is still in experimental state. This will h= elp in getting these changes matured enough for sym session which we can take up in 22.11. I believe Intel people are planning for new features in asymmetric crypto. It makes more sense that they can align it as per the discussed approach. Regards, Akhil