From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6C2CFA0C43; Wed, 20 Oct 2021 21:28:07 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 59F8B40150; Wed, 20 Oct 2021 21:28:07 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by mails.dpdk.org (Postfix) with ESMTP id C637A40142 for ; Wed, 20 Oct 2021 21:28:05 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10143"; a="226326736" X-IronPort-AV: E=Sophos;i="5.87,167,1631602800"; d="scan'208";a="226326736" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Oct 2021 12:27:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.87,167,1631602800"; d="scan'208";a="551758432" Received: from fmsmsx604.amr.corp.intel.com ([10.18.126.84]) by fmsmga004.fm.intel.com with ESMTP; 20 Oct 2021 12:27:54 -0700 Received: from fmsmsx602.amr.corp.intel.com (10.18.126.82) by fmsmsx604.amr.corp.intel.com (10.18.126.84) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Wed, 20 Oct 2021 12:27:54 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Wed, 20 Oct 2021 12:27:54 -0700 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (104.47.51.46) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.12; Wed, 20 Oct 2021 12:27:54 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nEYj9sW5WCKq6jJ7utMguQqbCcEvTXx5A0pFvbOeXCCpzc9oh2EnVihwb0d1aqqeSB5bw4UhkLaiunYx7Ck2ODBgIWPPR7uAEsTzXhFYpUdovGTLYUcVXwa2YNBhhMP3H/34wmL5dEsqlItABgIlkjaWHt5xg7lo+xwZDQ55FWRDXYQj4iRlp7Hwya6EmZEc3JDfama9iYjwpTgevWD5KGhnN7DLP01HyekySRawwfZbdB7000ZVe6Kb+lwlakPvDoiDYIM80WJMjtKlznFDgO0Q7M/BFUXxmIJLfl3cIUdybfNRKpVbhbBU9wCT1w+dbz9h+h+16x8c72n0izvYhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pxZCkNlc1ZrTvOTPTfhU63Ljj9NBqV9MvYL3c2GAqZA=; b=YepGfRUSsWN8NnFbIseRj9zfmp5zNHjH7X3lWrBFGIDbK+MM92PxpMDgvoBfkIIYpl6UTcyYKBaL/3U+m7IoqnibDNZtyTc5WiG5/TE5qybB1IDU6b/2Vq+Yw0oQD9RppbtbGgPLBCICxhqYboGX93fwS7ltfUtaVyT7vQwZVrNV5rLtJhqDhA6+Q6XbRPIMEH/nbJRqbLQo3krZXTp0xppVDmL1THzbVXODswhQMtvRLR4FXq9eGzBFBnaHQQUTXQws6km7wL0eFsQUMF2iUfmyCJVOt/djrHwIPDlz147TZTJUslXIfvOXyEYh23kr/ob1QXsq22Ewsl/G0VxNDA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pxZCkNlc1ZrTvOTPTfhU63Ljj9NBqV9MvYL3c2GAqZA=; b=I9c/W7MdhmsPajuDFUYZPDOHjfaWydKNPYXdlGdDCIOC0Xbez4XH+1c87S8gNPpaR5Q+zjhFU9zT6DNjMQdD1+CLevUpZUgBemXunXwpVTLf4Ngx+7yPtbsiMU8ZW8NRYMdVRidkA7zoI50Jd6t/agYz3O3oXxCfdw4dVUMqquY= Received: from DM6PR11MB4491.namprd11.prod.outlook.com (2603:10b6:5:204::19) by DM6PR11MB4185.namprd11.prod.outlook.com (2603:10b6:5:195::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.18; Wed, 20 Oct 2021 19:27:52 +0000 Received: from DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::2c0c:5383:f814:3b4e]) by DM6PR11MB4491.namprd11.prod.outlook.com ([fe80::2c0c:5383:f814:3b4e%6]) with mapi id 15.20.4628.016; Wed, 20 Oct 2021 19:27:52 +0000 From: "Ananyev, Konstantin" To: Akhil Goyal , "dev@dpdk.org" CC: "thomas@monjalon.net" , "david.marchand@redhat.com" , "hemant.agrawal@nxp.com" , "anoobj@marvell.com" , "De Lara Guarch, Pablo" , "Trahe, Fiona" , "Doherty, Declan" , "matan@nvidia.com" , "g.singh@nxp.com" , "Zhang, Roy Fan" , "jianjay.zhou@huawei.com" , "asomalap@amd.com" , "ruifeng.wang@arm.com" , "Nicolau, Radu" , "ajit.khaparde@broadcom.com" , "rnagadheeraj@marvell.com" , "adwivedi@marvell.com" , "Power, Ciara" , "Wang, Haiyue" , "jiawenwu@trustnetic.com" , "jianwang@trustnetic.com" Thread-Topic: [PATCH v3 6/8] cryptodev: rework session framework Thread-Index: AQHXxGgs8uA5efxFxkuDtx/p8VXwcKvcQbSw Date: Wed, 20 Oct 2021 19:27:52 +0000 Message-ID: References: <20211013192222.1582631-2-gakhil@marvell.com> <20211018213452.2734720-1-gakhil@marvell.com> <20211018213452.2734720-7-gakhil@marvell.com> In-Reply-To: <20211018213452.2734720-7-gakhil@marvell.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.200.16 authentication-results: marvell.com; dkim=none (message not signed) header.d=none;marvell.com; dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9156b290-47a6-43f4-ebcb-08d993ffb5c2 x-ms-traffictypediagnostic: DM6PR11MB4185: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: r5bibzgJTMe5utCnnu5v/b1n/beiqte7DGVKbP0E4ueesZQLjt1/5cPbkIxD9t2sMsjShwx303DZlhLSWTeKu4NQnYYrZBKQYlM4ndGLBEWUJu3AJyly7kBBPbttC0TGZ18x5MNj4Oq5Yi87U45A/DDwtwvxArPAuXwhKcyI2emb6Cv6/VgCweKQHpL+KRqWijSmDAuxxm3Fh7sxYQSTuWsdIJg/mOjs1N65DfzEqA0RTvX/i6lbWVWm2W2RUgQKw/0efboh6VbYsHJmRmTJmqZvbq4v7g25XPm0Kyr/YNlci5AWLg97/TGzy+klAkjXfJsT1Lh5OtFVTRti0Nx2K/KwFwidGZ0P+aY+lcfwcVY6/KuyNrsYF5X6nov3kkFpyybpTUb2V76vXxdjnf0+5SuTxgHJaYE4SE47ANc/DNSmWyP75vpE+tfOiDJJakrLDYf50+sXozdFUyFf8QUv0OvfsbyYVHOUk5C0jFjXeiIVcNWxXMP/OdPmMDpU+NNTEslPTCeNMvd4acjQL8fHc8M3yM6lOt++FShVg0pIUqIhlOGbe9lYkhfM2hrllq71EzxJBPhScSwN1cv7E0liwXHYUJdCFqFZfE7Jqp1/X5v5iV/GXDVedTPTK+aoKPhkV1vfYVceW2PhHmWQ7qZT4lTqRQHlvY2cCFvRZmB82PKuiK2iqhopvJWW+OqXKPqUxpu4GarYPMacO6YkaBOavw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB4491.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(55236004)(26005)(19627235002)(66946007)(7416002)(76116006)(82960400001)(8936002)(316002)(54906003)(66476007)(66556008)(64756008)(2906002)(508600001)(38070700005)(55016002)(33656002)(186003)(5660300002)(6506007)(8676002)(7696005)(66446008)(4326008)(71200400001)(9686003)(86362001)(110136005)(52536014)(38100700002)(122000001)(83380400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?W80t4kc67FzZaVH9gBwONyu247n8Cg4utlxiqH8InZ5OqljP8AArY20T1KHV?= =?us-ascii?Q?zqzNuDBI8X+BG/Fu+vAXBsR6aTY1D1xitkRS/X6PzpO4UwVWorZndOSLRnQl?= =?us-ascii?Q?s5ZCznT7YLD8mB+jgXdQh5ZUECQk9jQrwQ1JFO6o/VUPw+1AOhTvEDEe9z0i?= =?us-ascii?Q?gueSaFTIgWQORMWKFEhd1JkZnx8ElpOwqGFbNy/pN3ukV6/V+PEGpjv105Tk?= =?us-ascii?Q?O/z7ImBIc6TCh/LcVPyFwmH/nLSAu9FSUki8LfCkIT8XCS7D428bqcyL/ft9?= =?us-ascii?Q?g10UAcc1n1kR0bHaU57xpHJ0rkIR9Tlm09aL5aMZaM5RYG/iH33F5zKbFRL/?= =?us-ascii?Q?d0ajhdf9k0tTbn/zNCRjWiSOD88blhfeNfNCGZuU/fj/TsRJzrU3H7htBZiL?= =?us-ascii?Q?yKVK74hPROyo4k0fGTaiL2QJukZkXmLM4mE4BjSS8aU751tHMhWaiS95x2GO?= =?us-ascii?Q?M+sFzitGHL7EI3iZZibtvQGT+0t/UhZW1n8g1Llcfz52jyLjWptBCNg57TfB?= =?us-ascii?Q?JVTE7LzpPsEhT8mqN5AokW/Vorg42JWDLNNs++8Th7GMv+Gqik0ZcTkOqF7Q?= =?us-ascii?Q?7DqjnMPbhhaYQwrYBajJCgWRPgdH+FjzegA7C0Po9awA/vkcJhe/jprPKrIU?= =?us-ascii?Q?EIx1tSe6mfShazAZBY1JairAr9nN2pgFzwbdMarAHIo4vQ6y2JplbqCvB9LZ?= =?us-ascii?Q?0sFgzQHoIoMCWudw7m0RXl8uykYc8u/2FXPWCIDu+aw2I8jb223GwK3wn9Lv?= =?us-ascii?Q?mXa/eGLIKswFXQ+aSwHiUrC2GT6qaIleSEE/dhHMlhu0Xnbv/+w9gsEqfF9f?= =?us-ascii?Q?zmi1qzn9gYyeuW9KV+FNR6g/8EMqoitmtIDIUwgb5nobCx4XG1rvAnJB9c13?= =?us-ascii?Q?pUTwDUCTS9C7lOPUT1exIqK0Cl7hBbcxrmHPRqF9mjWp/VLKTlak+S0WtUcX?= =?us-ascii?Q?7hW616Nn2JdYaGn8Uxc7jXB+VLnLK6qgvym/ADBCRJTSi54IMFQJtxcfBL72?= =?us-ascii?Q?c8Lm5VMcb3cWrsb1anL8vMX7ohtUgp70SVtFSEiTrSZOYnidnNVnocQP355P?= =?us-ascii?Q?yyFzmd67w6Uk92snaMKTswbbrdNfYF+lU0zUbbYjirOsTQ9M4y4VzMIJJlTq?= =?us-ascii?Q?+podMfQ9/nr7Jzso3zzvhTzbtiFCQF6GqLKxECB3Nf42kYbZsrUr7+t937r3?= =?us-ascii?Q?YBXRoiFfUQtDAonNZND6jMwcpHjWGNoNfdknIQo3cDngWmD46zv3G5r5Pybf?= =?us-ascii?Q?YdnlPYTtDgAfBP/RgvuO2A8hjlo/PM0QT05Tru1bGhirSQPU4Z1ZCmJom7OT?= =?us-ascii?Q?f0fUj+6tKxC4CvCGdgQzQmuP?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB4491.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9156b290-47a6-43f4-ebcb-08d993ffb5c2 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Oct 2021 19:27:52.3790 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: konstantin.ananyev@intel.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4185 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH v3 6/8] cryptodev: rework session framework X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Akhil, > As per current design, rte_cryptodev_sym_session_create() and > rte_cryptodev_sym_session_init() use separate mempool objects > for a single session. > And structure rte_cryptodev_sym_session is not directly used > by the application, it may cause ABI breakage if the structure > is modified in future. >=20 > To address these two issues, the rte_cryptodev_sym_session_create > will take one mempool object for both the session and session > private data. The API rte_cryptodev_sym_session_init will now not > take mempool object. > rte_cryptodev_sym_session_create will now return an opaque session > pointer which will be used by the app in rte_cryptodev_sym_session_init > and other APIs. >=20 > With this change, rte_cryptodev_sym_session_init will send > pointer to session private data of corresponding driver to the PMD > based on the driver_id for filling the PMD data. >=20 > In data path, opaque session pointer is attached to rte_crypto_op > and the PMD can call an internal library API to get the session > private data pointer based on the driver id. >=20 > Note: currently nb_drivers are getting updated in RTE_INIT which > result in increasing the memory requirements for session. > User can compile off drivers which are not in use to reduce the > memory consumption of a session. >=20 > Signed-off-by: Akhil Goyal > --- With that patch ipsec-secgw functional tests crashes for AES_GCM test-cases= . To be more specific: examples/ipsec-secgw/test/run_test.sh -4 tun_aesgcm [24126592.561071] traps: dpdk-ipsec-secg[3254860] general protection fault = ip:7f3ac2397027 sp:7ffeaade8848 error:0 in libIPSec_MB.so.1.0.0[7f3ac238f00= 0+2a20000] Looking a bit deeper, it fails at: #0 0x00007ff9274f4027 in aes_keyexp_128_enc_avx512 () from /lib/libIPSec_MB.so.1 #1 0x00007ff929f0ac97 in aes_gcm_pre_128_avx_gen4 () from /lib/libIPSec_MB.so.1 #2 0x0000561757073753 in aesni_gcm_session_configure (mb_mgr=3D0x56175c5fe= 400, session=3D0x17e3b72d8, xform=3D0x17e05d7c0) at ../drivers/crypto/ipsec_mb/pmd_aesni_gcm.c:132 #3 0x00005617570592af in ipsec_mb_sym_session_configure ( dev=3D0x56175be0c940 , xform=3D0x17e05d7c0, sess=3D0x17e3b72d8) at ../drivers/crypto/ipsec_mb/ipsec_mb_ops.c:330 #4 0x0000561753b4d6ae in rte_cryptodev_sym_session_init (dev_id=3D0 '\000'= , sess_opaque=3D0x17e3b4940, xforms=3D0x17e05d7c0) at ../lib/cryptodev/rte_cryptodev.c:1736 #5 0x0000561752ef99b7 in create_lookaside_session ( ipsec_ctx=3D0x56175aa6a210 , sa=3D0x17e05d140, ips=3D0x17e05d140) at ../examples/ipsec-secgw/ipsec.c:145 #6 0x0000561752f0cf98 in fill_ipsec_session (ss=3D0x17e05d140, ctx=3D0x56175aa6a210 , sa=3D0x17e05d140) at ../examples/ipsec-secgw/ipsec_process.c:89 #7 0x0000561752f0d7dd in ipsec_process ( ctx=3D0x56175aa6a210 , trf=3D0x7ffd192326a0) at ../examples/ipsec-secgw/ipsec_process.c:300 #8 0x0000561752f21027 in process_pkts_outbound ( --Type for more, q to quit, c to continue without paging-- ipsec_ctx=3D0x56175aa6a210 , traffic=3D0x7ffd192326= a0) at ../examples/ipsec-secgw/ipsec-secgw.c:839 #9 0x0000561752f21b2e in process_pkts ( qconf=3D0x56175aa57340 , pkts=3D0x7ffd19233c20, nb_pkts=3D1 '\001', portid=3D1) at ../examples/ipsec-secgw/ipsec-secgw.= c:1072 #10 0x0000561752f224db in ipsec_poll_mode_worker () at ../examples/ipsec-secgw/ipsec-secgw.c:1262 #11 0x0000561752f38adc in ipsec_launch_one_lcore (args=3D0x56175c549700) at ../examples/ipsec-secgw/ipsec_worker.c:654 #12 0x0000561753cbc523 in rte_eal_mp_remote_launch ( f=3D0x561752f38ab5 , arg=3D0x56175c549700, call_main=3DCALL_MAIN) at ../lib/eal/common/eal_common_launch.c:64 #13 0x0000561752f265ed in main (argc=3D12, argv=3D0x7ffd19234168) at ../examples/ipsec-secgw/ipsec-secgw.c:2978 (gdb) frame 2 #2 0x0000561757073753 in aesni_gcm_session_configure (mb_mgr=3D0x56175c5fe= 400, session=3D0x17e3b72d8, xform=3D0x17e05d7c0) at ../drivers/crypto/ipsec_mb/pmd_aesni_gcm.c:132 132 mb_mgr->gcm128_pre(key, &sess->gdata_key); Because of un-expected unaligned memory access: (gdb) disas Dump of assembler code for function aes_keyexp_128_enc_avx512: 0x00007ff9274f400b <+0>: endbr64 0x00007ff9274f400f <+4>: cmp $0x0,%rdi 0x00007ff9274f4013 <+8>: je 0x7ff9274f41b4 0x00007ff9274f4019 <+14>: cmp $0x0,%rsi 0x00007ff9274f401d <+18>: je 0x7ff9274f41b4 0x00007ff9274f4023 <+24>: vmovdqu (%rdi),%xmm1 =3D> 0x00007ff9274f4027 <+28>: vmovdqa %xmm1,(%rsi) (gdb) print/x $rsi $12 =3D 0x17e3b72e8 And this is caused because now AES_GCM session private data is not 16B-bits aligned anymore: (gdb) print ((struct aesni_gcm_session *)sess->sess_data[index].data) $29 =3D (struct aesni_gcm_session *) 0x17e3b72d8 print &((struct aesni_gcm_session *)sess->sess_data[index].data)->gdata_key $31 =3D (struct gcm_key_data *) 0x17e3b72e8 As I understand the reason for that is that we changed the way how sess_dat= a[index].data is populated. Now it is just: sess->sess_data[index].data =3D (void *)((uint8_t *)sess + rte_cryptodev_sym_get_header_session_size()= + (index * sess->priv_sz)); So, as I can see, there is no guarantee that PMD's private sess data will b= e aligned on 16B as expected.