From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 91458A0C4B; Thu, 21 Oct 2021 08:54:35 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 62AA6410E2; Thu, 21 Oct 2021 08:54:35 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id A6B9340142 for ; Thu, 21 Oct 2021 08:54:34 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19KN0UL5003974; Wed, 20 Oct 2021 23:54:28 -0700 Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2173.outbound.protection.outlook.com [104.47.58.173]) by mx0b-0016f401.pphosted.com with ESMTP id 3btjwdmkhn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 20 Oct 2021 23:54:26 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OtDai4mpBBSY/ZA4y48/4mZ+WahAH8IMEH68UpaT9VpQl44P1csR27FlB5xCX0OJGSUX0qD1gPo+FvSbsOutRjoZKhAgx/63WSuFPMSeXc4xztQgiy+cK+9jP/KgwDrc1OrywNx4y7AjpJbBT/TZx9mDbmfo3+eJZK94sAHs1qwpXI9t8fOomfDnlkHzau1VXeBHGFYc4/a6A+6snm7Y+CbqPiME0z/yHTCxJzx2f0yH6TmAqbRjwv3mF+XoKQGOs+CuFiSGs3YUOwOUeggfj+xnpGp+lU26Dw2o/cQBKL5VkGkZLOluDgyYL7UozvKsY5REVrVRA1bNEqNNvQ7dzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rD5sPgAWYwPjLMrZS27LMAULJwrCUkXQk3KyFznvx0k=; b=mSchkPsD+sFIsihg/lhOHnUDXoKKFXAh///zklCyOfn45aHPixib+1GJhBd2Gz4Le8cdETsD5xJJKy1rwsDea7zIEVe79SSda5TNlOVs+QGdMybqiaupn7zeITLUk/02jP2m99oCL8+jSLardiYYAmFcNcCvZcRy8pUvT8hv7jd/eP5d7mUqplBP2wphJeDS0h5W+2qDIac70LxZ+lDghnJ94A4rV1hP7W7t9TAMgFNW8uJKYtwP87XPcmd+mfLFzaMkWwqdAORC43RPtyr2c7AO/9TZaBNve1ELmdUr9ICy7pCIx7x8z1lFKt0OBdbpaWixAPjH4Es9DjG05pKObg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rD5sPgAWYwPjLMrZS27LMAULJwrCUkXQk3KyFznvx0k=; b=c0q6xYIxZDrQtEEk5Mm64IsjFklHyBFZRwq1eTboAsVzk8DSchcqCHZGGKELqcZX614zLF0O+E6nVmM93h6MdpUtdBHkki3QStqYutjR4rw6g0g6JEOnSCUe3f8MPtivGMy37laV5aqKfuAW4eu3LP7kXEvnpfZ/OFkdLQk6iRU= Received: from CO6PR18MB4484.namprd18.prod.outlook.com (2603:10b6:5:359::9) by CO6PR18MB4433.namprd18.prod.outlook.com (2603:10b6:303:13b::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4628.16; Thu, 21 Oct 2021 06:53:50 +0000 Received: from CO6PR18MB4484.namprd18.prod.outlook.com ([fe80::c41e:707:3f91:71b8]) by CO6PR18MB4484.namprd18.prod.outlook.com ([fe80::c41e:707:3f91:71b8%8]) with mapi id 15.20.4608.019; Thu, 21 Oct 2021 06:53:50 +0000 From: Akhil Goyal To: "Ananyev, Konstantin" , "dev@dpdk.org" CC: "thomas@monjalon.net" , "david.marchand@redhat.com" , "hemant.agrawal@nxp.com" , Anoob Joseph , "De Lara Guarch, Pablo" , "Trahe, Fiona" , "Doherty, Declan" , "matan@nvidia.com" , "g.singh@nxp.com" , "Zhang, Roy Fan" , "jianjay.zhou@huawei.com" , "asomalap@amd.com" , "ruifeng.wang@arm.com" , "Nicolau, Radu" , "ajit.khaparde@broadcom.com" , Nagadheeraj Rottela , Ankur Dwivedi , "Power, Ciara" , "Wang, Haiyue" , "jiawenwu@trustnetic.com" , "jianwang@trustnetic.com" Thread-Topic: [PATCH v3 6/8] cryptodev: rework session framework Thread-Index: AQHXxGgefWweT0oRpE+oJQJPA2AoU6vcSFUAgAC9nRA= Date: Thu, 21 Oct 2021 06:53:50 +0000 Message-ID: References: <20211013192222.1582631-2-gakhil@marvell.com> <20211018213452.2734720-1-gakhil@marvell.com> <20211018213452.2734720-7-gakhil@marvell.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 007424a5-601d-4b74-26f9-08d9945f89e8 x-ms-traffictypediagnostic: CO6PR18MB4433: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: fq5FkWdDQFhJGoG+MGEnOMH6i4jLV5SoQx5OhXP5b48ID1DdH8H1tZ/gNyawcq86C6IABhr73s4jv1AlYY0R1045xqeeLEa/6/x3EwZWUTxc3OJbLuUus5IhBb3DHXhzQ1gxD9FP1id5NoDktgAlljp9banuB6JDDotSFCUvBQyLhMgh5mbGTJfskvAPIZsf+zUg+DgjK4LxYdnJdfb+z0xJGZ9nNzJ+4kQgpB6NaNv1C6lBAoJVcr2xjgNEtf8fuUvoqDW3fImnq34SBZ716ydw5DyJ26Kzxvy0ZHaU8LsoOPwF5wJ2c0h4XhTxvfGqvPlS9r2/QuA4gCkJtiplY/EXGuwTULFB0LeQpyl27/dPYoqLErxVejT+pkwTxxQPoOzwegrfdaAHyFX+XHH+0goadwbr7pG7x2Xx3UL9eUFfJLK55KH0h1ODgovLU/A/y84Vy5yMZr5CRJuhn7ZwJOHGsM8ZH8uRKKUAuMZcTSRoVWVO/WMMtoAbaMcwgLwQud7i3Kj8ELvVBCgy9RXK61JCAj/CvnIwC2jghRWUXQ0o8eOfyTzXEROSZfg7ZJ3dlvhpbd+d0EcZAnaiVwkaK1424cmnp+yq28OB93468nwsEfABMFP+t5uFioBZSODqoLuQqVSmxfs74qcW0AEN6tsbjFIJFsT1t9fCKZf0fABsO5eUZruxK35Yu2r6tGGY4VUGjgDUTMvo/cvy+rr3Ow== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR18MB4484.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(54906003)(186003)(86362001)(19627235002)(9686003)(2906002)(71200400001)(66476007)(66446008)(38100700002)(8936002)(316002)(26005)(33656002)(4326008)(7416002)(55236004)(110136005)(6506007)(122000001)(55016002)(76116006)(38070700005)(52536014)(83380400001)(508600001)(5660300002)(66946007)(7696005)(66556008)(8676002)(64756008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?vRctV61E9BKqxqVAwG8jwRWhU4mNa9B6hX2OH7yYPapu5jM9BOCfq75d1ACx?= =?us-ascii?Q?SNSaMeiT1jfp7zsALToeOAQUtGdTCqE4N0ViSEYuEMWrT/32XpouxzQxBh2i?= =?us-ascii?Q?m6PaxVSm9WhmPk+bdTVtBTHTe3BADo3YUwpgiCJ64XBn3N4343QInRPHXqA1?= =?us-ascii?Q?FwWXLTUEKcnnPzsdDbCL74z1KSE6Ao3TrmpGkKp9gYkWxpCd8N1H4Rv1aOM0?= =?us-ascii?Q?pf1Ip7tQAwpu34NTJidDWER6JHDplmsaCxjIcmUimOFVTW9LnZbfb1CAoTjV?= =?us-ascii?Q?sBWjTiOGHWzv4JKwR/6bsmkFvzPtqh+LaR2T1yO5PlJa99XpIK1mOsbDKyXg?= =?us-ascii?Q?zGx/GjL/P3eLU3J6kB0lTe6j01R38zQclB+lAyxL2jAQlgA0Z9kr/TR5J/2D?= =?us-ascii?Q?YcLAuxk5iJtaM+ZSkipT9vz0TNB8kLyp4uCK4OfQnM4rboUX5Tc517sy+RbO?= =?us-ascii?Q?zOHcq43r9oA3nudIyU5mPyfIO8v62Utl7WtdwXIT2n/7AMyZIy1Jhb2dxx0+?= =?us-ascii?Q?cKf98PDKn0Pr9YD+CUxO8XgsBDqH/hhMFXRgv26RXsf/5JJQAjd4aED1JzBX?= =?us-ascii?Q?NzMLhayM15Yai7d/jMulKjP1U55OP0QkPMN361HjJvbxzd/JI5o3GECMDXKE?= =?us-ascii?Q?JQZHIsr8B/ckCkEXNDVLqxO2krjK9wqj9UnaUSkkdmJrXB0IkCGigdMEbZv6?= =?us-ascii?Q?9RdXvezE/WqBZ98j6LJNZhBX0E3ZOsEoMk/hkqNOd816Y+/D6YLL1WSAti1S?= =?us-ascii?Q?Ib+2FzuId6dmM2lTEkvxFZZZjvhCNcQYYvD5Qgplw9shFfDcd14F8Vht1qnt?= =?us-ascii?Q?SGR0ysoRGumYHOre+wRSJS/CeH2qdQhpBmmmDCibqe7ZNoJDu2TwuxC2SyR6?= =?us-ascii?Q?9ToSXo2PhqMWILT3O4ElRmPJRBlvnw2g5eGsJHsL6hskMj46Ss62WCjKcmir?= =?us-ascii?Q?bxbKXnFXNzcPY/S3hXWD5oK7kPLWaWeKpXKTqqfX1a/Xx4fl6UGjlHVv2HMw?= =?us-ascii?Q?1fNoFLdtx4OK9saIDgZhSyzO+cLHZFDYSxOXq2fBFwx1j3OM/enjZplQCe6q?= =?us-ascii?Q?0MXptN75W7eCKIeTDTXt9CclF+kObAU9lGCGk+A1u8AxeCLRhkJLYZS3i+P4?= =?us-ascii?Q?RPMeJcoGwXKt/Ql8IbPn4pCcnmkNjJv8IroSEz760c5mmshRsDDDe3VENaJd?= =?us-ascii?Q?/T/sRoafhnCBIWl0sfPrRreWrpqfjukLOHmppaig1eYIUtHsp1s/wvrhNiGD?= =?us-ascii?Q?JGzfguIO5jDuKULNl5PFt4N+B7QpHcOaGytZfG2efmSXLC7/BeQHxOpNXJoJ?= =?us-ascii?Q?GxFVLqylHpLLrmNAIj1eg4JN1EmmSAnb+CvbofZwUc8zHx3meKue70Qb46CT?= =?us-ascii?Q?g8tZuN9znAjJV/xwNBUQ46Ia7GkaesYA9h3HuJA5av8HE432lwKc/VOujkP1?= =?us-ascii?Q?gbL/nNFn2Wc=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO6PR18MB4484.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 007424a5-601d-4b74-26f9-08d9945f89e8 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Oct 2021 06:53:50.4499 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: gakhil@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR18MB4433 X-Proofpoint-GUID: -M-dkGuUVUvz-KOknOqDg06Yn8evLnOi X-Proofpoint-ORIG-GUID: -M-dkGuUVUvz-KOknOqDg06Yn8evLnOi X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-21_02,2021-10-20_02,2020-04-07_01 Subject: Re: [dpdk-dev] [PATCH v3 6/8] cryptodev: rework session framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" =20 > > As per current design, rte_cryptodev_sym_session_create() and > > rte_cryptodev_sym_session_init() use separate mempool objects > > for a single session. > > And structure rte_cryptodev_sym_session is not directly used > > by the application, it may cause ABI breakage if the structure > > is modified in future. > > > > To address these two issues, the rte_cryptodev_sym_session_create > > will take one mempool object for both the session and session > > private data. The API rte_cryptodev_sym_session_init will now not > > take mempool object. > > rte_cryptodev_sym_session_create will now return an opaque session > > pointer which will be used by the app in rte_cryptodev_sym_session_init > > and other APIs. > > > > With this change, rte_cryptodev_sym_session_init will send > > pointer to session private data of corresponding driver to the PMD > > based on the driver_id for filling the PMD data. > > > > In data path, opaque session pointer is attached to rte_crypto_op > > and the PMD can call an internal library API to get the session > > private data pointer based on the driver id. > > > > Note: currently nb_drivers are getting updated in RTE_INIT which > > result in increasing the memory requirements for session. > > User can compile off drivers which are not in use to reduce the > > memory consumption of a session. > > > > Signed-off-by: Akhil Goyal > > --- >=20 > With that patch ipsec-secgw functional tests crashes for AES_GCM test-cas= es. > To be more specific: > examples/ipsec-secgw/test/run_test.sh -4 tun_aesgcm >=20 > [24126592.561071] traps: dpdk-ipsec-secg[3254860] general protection faul= t > ip:7f3ac2397027 sp:7ffeaade8848 error:0 in > libIPSec_MB.so.1.0.0[7f3ac238f000+2a20000] >=20 > Looking a bit deeper, it fails at: > #0 0x00007ff9274f4027 in aes_keyexp_128_enc_avx512 () > from /lib/libIPSec_MB.so.1 > #1 0x00007ff929f0ac97 in aes_gcm_pre_128_avx_gen4 () > from /lib/libIPSec_MB.so.1 > #2 0x0000561757073753 in aesni_gcm_session_configure > (mb_mgr=3D0x56175c5fe400, > session=3D0x17e3b72d8, xform=3D0x17e05d7c0) > at ../drivers/crypto/ipsec_mb/pmd_aesni_gcm.c:132 > #3 0x00005617570592af in ipsec_mb_sym_session_configure ( > dev=3D0x56175be0c940 , xform=3D0x17e05d7c0, > sess=3D0x17e3b72d8) at ../drivers/crypto/ipsec_mb/ipsec_mb_ops.c:330 > #4 0x0000561753b4d6ae in rte_cryptodev_sym_session_init (dev_id=3D0 > '\000', > sess_opaque=3D0x17e3b4940, xforms=3D0x17e05d7c0) > at ../lib/cryptodev/rte_cryptodev.c:1736 > #5 0x0000561752ef99b7 in create_lookaside_session ( > ipsec_ctx=3D0x56175aa6a210 , sa=3D0x17e05d140, > ips=3D0x17e05d140) at ../examples/ipsec-secgw/ipsec.c:145 > #6 0x0000561752f0cf98 in fill_ipsec_session (ss=3D0x17e05d140, > ctx=3D0x56175aa6a210 , sa=3D0x17e05d140) > at ../examples/ipsec-secgw/ipsec_process.c:89 > #7 0x0000561752f0d7dd in ipsec_process ( > ctx=3D0x56175aa6a210 , trf=3D0x7ffd192326a0) > at ../examples/ipsec-secgw/ipsec_process.c:300 > #8 0x0000561752f21027 in process_pkts_outbound ( > --Type for more, q to quit, c to continue without paging-- > ipsec_ctx=3D0x56175aa6a210 , > traffic=3D0x7ffd192326a0) > at ../examples/ipsec-secgw/ipsec-secgw.c:839 > #9 0x0000561752f21b2e in process_pkts ( > qconf=3D0x56175aa57340 , pkts=3D0x7ffd19233c20, > nb_pkts=3D1 '\001', portid=3D1) at ../examples/ipsec-secgw/ipsec-secg= w.c:1072 > #10 0x0000561752f224db in ipsec_poll_mode_worker () > at ../examples/ipsec-secgw/ipsec-secgw.c:1262 > #11 0x0000561752f38adc in ipsec_launch_one_lcore (args=3D0x56175c549700) > at ../examples/ipsec-secgw/ipsec_worker.c:654 > #12 0x0000561753cbc523 in rte_eal_mp_remote_launch ( > f=3D0x561752f38ab5 , arg=3D0x56175c549700, > call_main=3DCALL_MAIN) at ../lib/eal/common/eal_common_launch.c:64 > #13 0x0000561752f265ed in main (argc=3D12, argv=3D0x7ffd19234168) > at ../examples/ipsec-secgw/ipsec-secgw.c:2978 > (gdb) frame 2 > #2 0x0000561757073753 in aesni_gcm_session_configure > (mb_mgr=3D0x56175c5fe400, > session=3D0x17e3b72d8, xform=3D0x17e05d7c0) > at ../drivers/crypto/ipsec_mb/pmd_aesni_gcm.c:132 > 132 mb_mgr->gcm128_pre(key, &sess->gdata_key); >=20 > Because of un-expected unaligned memory access: > (gdb) disas > Dump of assembler code for function aes_keyexp_128_enc_avx512: > 0x00007ff9274f400b <+0>: endbr64 > 0x00007ff9274f400f <+4>: cmp $0x0,%rdi > 0x00007ff9274f4013 <+8>: je 0x7ff9274f41b4 > > 0x00007ff9274f4019 <+14>: cmp $0x0,%rsi > 0x00007ff9274f401d <+18>: je 0x7ff9274f41b4 > > 0x00007ff9274f4023 <+24>: vmovdqu (%rdi),%xmm1 > =3D> 0x00007ff9274f4027 <+28>: vmovdqa %xmm1,(%rsi) >=20 > (gdb) print/x $rsi > $12 =3D 0x17e3b72e8 >=20 > And this is caused because now AES_GCM session private data is not 16B-bi= ts > aligned anymore: > (gdb) print ((struct aesni_gcm_session *)sess->sess_data[index].data) > $29 =3D (struct aesni_gcm_session *) 0x17e3b72d8 >=20 > print &((struct aesni_gcm_session *)sess->sess_data[index].data)- > >gdata_key > $31 =3D (struct gcm_key_data *) 0x17e3b72e8 >=20 > As I understand the reason for that is that we changed the way how > sess_data[index].data > is populated. Now it is just: > sess->sess_data[index].data =3D (void *)((uint8_t *)sess + > rte_cryptodev_sym_get_header_session_size= () + > (index * sess->priv_sz)); >=20 > So, as I can see, there is no guarantee that PMD's private sess data will= be > aligned on 16B > as expected. >=20 Agreed, that there is no guarantee that the sess_priv will be aligned. I believe this is requirement from the PMD side for a particular alignment. Is it possible for the PMD to use __rte_aligned for the fields which are re= quired to Be aligned. For aesni_gcm it is 16B aligned requirement, for some other PMD= it may be 64B alignment.=20