From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6DC5F45545 for ; Mon, 1 Jul 2024 14:55:38 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 6AC7E427C0; Mon, 1 Jul 2024 14:55:31 +0200 (CEST) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.17]) by mails.dpdk.org (Postfix) with ESMTP id E51FA42686; Mon, 1 Jul 2024 14:55:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1719838528; x=1751374528; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version; bh=COicOgU8IPrw5+sI2FyGzN148IIOJnN6l1F4AALx+O0=; b=Lj51Mm+/VseIJO57o8aU+da1nq0yyoPZz/R9FXBnWFUPMzCZPSKSgg6+ eUp2iVQ3iBmrP2GPXxCIhm4CD9OE9JwcE75+hYdCmJPv02sf2WaXS7djl V3haVzTTbf+/RX5yFgm7GM5Zo2fygtVFVfAwAKrE3WG6q8KCobJCjSReX +t46pLAmXtnhpwrlZjZei1nH4ceLLo+THEI8983DwD22UsmQ4iE0nxnpX qt6MLTEEsDnwmMSgcX2Dkr+TRWCNIdyh5FOe0Kkr60MnlGplYZ3WJlgjy pB4lczEoLIB3SfsQvUpB0s9DAQ/FBRZiVbdiY4Pg+dzf+1vcPw+KRCDGC w==; X-CSE-ConnectionGUID: +ndp3tDHS76htXkfRXtOdw== X-CSE-MsgGUID: tXvxXBUIRsWFRcO92L9Vzw== X-IronPort-AV: E=McAfee;i="6700,10204,11120"; a="16834544" X-IronPort-AV: E=Sophos;i="6.09,176,1716274800"; d="scan'208,217";a="16834544" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by fmvoesa111.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2024 05:55:22 -0700 X-CSE-ConnectionGUID: 6sioktSfS6ep2RfdaKgxbQ== X-CSE-MsgGUID: lM462AFUT5OtjpGKLZe59A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,176,1716274800"; d="scan'208,217";a="50114739" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by fmviesa004.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 01 Jul 2024 05:55:22 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 1 Jul 2024 05:55:21 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Mon, 1 Jul 2024 05:55:21 -0700 Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39 via Frontend Transport; Mon, 1 Jul 2024 05:55:21 -0700 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.47) by edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 1 Jul 2024 05:55:21 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F72pdUEIwCiccsRIeCotMnWvY37JD+olc9k/jyHCgvMXhixcueoBKjTTXR0mVjDSGwxdezxAvbDYzE9S1joA74N9PiC/JfefFSOrts/EM+dewHadJDrjfdYOG1+cglkgj2sXtB+eYgWipa/c03nk5JQjYFw78iRinBvL3lNp0X4ag5AQSb3FfvxUWVhYzECCfMux1WoQqH7YBsQRBqaGN6ffgDDPWn0TQ/7/SqmOSOcvkO4Tkb3EmMKEmzG97+OwDvhIORtHabkAMV8ws6O+zCrumICaZWPWB+99Ylce36gWZmYlzeLu5Q11QsoJ8EDO1feJ0alI8K+MJvpt5AmLIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OEhlFT85+X64M89W4tBJfFIivr6KT13TCoWlcMmcAfs=; b=lGGHE23BYurMmwVIJ8c0qEBhkF4IE+ZYVXWQwTvuMiQ20TDXJQirhBYYgGnKq1WVmMQfuJ+sfncnGofBvaGubg3lHgXRNdqA60lcYc0vCeq/CuMhAmob2+0E5PAm2zlYekAD18iDkEwiMn3+2X2dUOfjlnzBb+6UK8dr+8Cc55sqgHkJW5D68tiFOZdAiMTRqe2xGogT4xeKfBPKungqtTv5/uUskAfY1KMf64+WGM4SEWlJUCpOCKYRA3Hoc2ZdTEm5CC+qhn1TKgIUh6m5Lkrt6OTFVM2DzfIR9XwZ7Xb+QfI3far15l2TP5BGMllvc1kYBikcY3HPnOBrA+uYfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DS0PR11MB7458.namprd11.prod.outlook.com (2603:10b6:8:145::13) by DM4PR11MB6408.namprd11.prod.outlook.com (2603:10b6:8:b7::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.32; Mon, 1 Jul 2024 12:55:18 +0000 Received: from DS0PR11MB7458.namprd11.prod.outlook.com ([fe80::1a9e:53a6:9603:8f79]) by DS0PR11MB7458.namprd11.prod.outlook.com ([fe80::1a9e:53a6:9603:8f79%5]) with mapi id 15.20.7719.028; Mon, 1 Jul 2024 12:55:18 +0000 From: "Ji, Kai" To: Jack Bond-Preston , Fan Zhang , Akhil Goyal CC: "dev@dpdk.org" , "stable@dpdk.org" , Wathsala Vithanage Subject: Re: [PATCH v4 1/5] crypto/openssl: fix GCM and CCM thread unsafe ctxs Thread-Topic: [PATCH v4 1/5] crypto/openssl: fix GCM and CCM thread unsafe ctxs Thread-Index: AQHauNj6GrHXfzmz2keUFEwWYYk5f7Hh+gA7 Date: Mon, 1 Jul 2024 12:55:17 +0000 Message-ID: References: <20240603160119.1279476-1-jack.bond-preston@foss.arm.com> <20240607124756.3968704-1-jack.bond-preston@foss.arm.com> <20240607124756.3968704-2-jack.bond-preston@foss.arm.com> In-Reply-To: <20240607124756.3968704-2-jack.bond-preston@foss.arm.com> Accept-Language: en-GB, en-US, en-IE Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: DS0PR11MB7458:EE_|DM4PR11MB6408:EE_ x-ms-office365-filtering-correlation-id: 8f8f8fe0-b084-4717-6356-08dc99cd0f1c x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|1800799024|376014|366016|38070700018; x-microsoft-antispam-message-info: =?iso-8859-1?Q?5Z8BZWTaShf43eydVxAlh+SVmGnVNwf1PnsQsMfQYVyny/jIS1wcuOzugt?= =?iso-8859-1?Q?hRxr2NuoVngY7MqJoz334H0rIcl3AI9tIMBA9pYy8qeCo+yHFbPY39kvtT?= =?iso-8859-1?Q?MoarXn38Nd79sUU1JvWpG4lpZ5SOpGVmfyn4nQIpvcSAPHSTEhQO5Etc9l?= =?iso-8859-1?Q?MyC5zGpFgkuJqsO0B8h4fiYAgWoSYquyGAWLnFmFUdJrm+Jl9aFt2U5Nfs?= =?iso-8859-1?Q?ZhusceIoJUzIIYkxhUMIB3aXYUopoerO8YkpupEXK3i1zT/A3qXHsCyLth?= =?iso-8859-1?Q?MYlagyovf7msieAnR33RYnrC2DHEVfXmMPH3zUeNi6NZjJWTZ2lKu8Yyw4?= =?iso-8859-1?Q?d4hDyb/Db91zyqpVD78/Zspv6QGDIY5dv0fNqFDI6kiE0bcnJ0uAzGPHEQ?= =?iso-8859-1?Q?fYkW56nEAbdlw0vLjo6tLxx0kjRk18PO0r4/uUdg/K70qaArHIc8yMNxCT?= =?iso-8859-1?Q?+WBgkMayrMSEpyRh57VaXLW2C17vF1FnK77V9fNAMJWs4cCkCEu65th3yj?= =?iso-8859-1?Q?S+wjGAq/ymLtOHHXKu8ZDQwyWNu2FSGIyqKlYKu1omYChXsHf0XpaaSs4R?= =?iso-8859-1?Q?xy9TwLgzcFiTDHjiBe0B9TLLfPPtksJ8w1Z3SebJnf7BDpR1z9xFsw9+Ke?= =?iso-8859-1?Q?bNMomCTOLGbI+6MVeutSducquqY1ruZO9w34f83cAEqW5ckNAeOl5/AXq3?= =?iso-8859-1?Q?gt4ag1Amvb3O7xrA+9A9dru0Gw1khwoYN7j6aS0Ul0FyhRW9+rnI6SNcCs?= =?iso-8859-1?Q?Ik6c0j9MhBxonrP+bX3zAm+/xdtvQZ1qzThvLeHhEO6A8HQF/F6eOT5iCT?= =?iso-8859-1?Q?tyVPL0npn4o9mV7AT834rCYZoQqsAiDezU3RviC3DgXx7s9n/al21jgSHL?= =?iso-8859-1?Q?ax2+EnlJgtrKJnLg0acFVfcUC6C4jG1E4pUu4GEziyjYiCjIl4dgZ1mbka?= =?iso-8859-1?Q?hRuFBb47/vKJKWr2rDjy7DcQ12x5qz3IT1HS+XtsA/TDaGKnFHmeiF5QBo?= =?iso-8859-1?Q?tXTLVObG21djKQIISwkobRtCeHGGCvNjznDLtfSxx5I4PvpVQ7ZzxoQPDV?= =?iso-8859-1?Q?QT0hN3tWFUco1OwwDagFmN2dbj0zr/jPe7x9MQQIYRTb3A+QNsMjtTCqj2?= =?iso-8859-1?Q?1Rud7F2US42EyEemRKoghaZtRViVKxUidRq8USSbaHegNvXIDcSQEZ3BzR?= =?iso-8859-1?Q?KSRNXokhOU/w/PydLiT9jGTiLpPvQhM0Zxg8zJsiMlCsGPuY6Djzcqt2WB?= =?iso-8859-1?Q?hbKWs4PnMBOvK7YlEhbk/Pd0DSCg9UdoboQmsLu5KWKKNs/q6vqHvR4N36?= =?iso-8859-1?Q?LlOz3o2Os8a2KWjv44B0MWtB1eXexdEuuFob9Z0oY4mOVfqMiRWt+EcZnJ?= =?iso-8859-1?Q?+Nf/TofyQUAI3FtblkpQDXw454qF3VgCxaM1vvuqIl9O3xCkby53tvHWMG?= =?iso-8859-1?Q?IOPh92ovLQ7pNwdwX2ynVRFfWL1uvI+i9qq4hQ=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7458.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(376014)(366016)(38070700018); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?5Jj47ViUX/qXJwlmGFzxZsH3HuAZ5xdCRBNz8AqKryMzURJ4rcFYkd+33W?= =?iso-8859-1?Q?xujsgFAMfFUcA23YeRpIP2DivNPsKLQp2dfFZpfpNbQah4nx4dAxqlnmFq?= =?iso-8859-1?Q?pAnd+20Wvnj4lCIbFF7Ea2uA8qf8LmL0Bj7ZQCg1aZBx+tJMFVO/pWCNZd?= =?iso-8859-1?Q?ookDw3q/XkxQ0FAVtsjsfVEFDVFM3Q61KLbX4WQ3EGX/JHE2hPiMHyHhUd?= =?iso-8859-1?Q?LEO+Gv6YxKAFDqMw2cTnKN+T+s0gZGfpBoTgz3NLwDHxHdwBXNOxaWohjZ?= =?iso-8859-1?Q?rJyrIm8RwS7ja3fYgS1nG42pPkx4HJMTHAw1IPEblI389vBW3twPwILdrz?= =?iso-8859-1?Q?0MpEKKmXDhednXdGHel7LuZhmVxurL+kkYCl6qPW+uytS69Xqu6mefPsRh?= =?iso-8859-1?Q?Om69hG17VYfqxdi7cY1y1mjCk2uidyx0WBFVc4qM0THNyXqMQhCNTY0Uhr?= =?iso-8859-1?Q?ad5wkO+oE//aO7uVUGqFHF4qvlTlxVvVAh8C9UPhqz0oFZEggLlbxaQfIJ?= =?iso-8859-1?Q?1vrPTygauhd/IBxfIzOajYz0jRMAi1PwMW9I1+8jR6fWMoV0Vd7aVyxDHk?= =?iso-8859-1?Q?zxPKGroHVLHfpulFDVLKOzWVQ3/WFf1hvnlP+kRITNbyuEJykqQjW896eK?= =?iso-8859-1?Q?AWwUr3rWOQBjjugmpvSCaeDfJqjUkq+oSMc25K+0raFom7jJy0z6o5+e+e?= =?iso-8859-1?Q?gsPwyxwMi/+z7+Fgg2tOgbiJla/3P47udgfF5DcqMjkOVxf+wJ6U4UTPWC?= =?iso-8859-1?Q?3AJG/JDnLHsFkxTNg6ebHbiVsibM0eNaziVQbSf+wOd0Fg8Nu4nPPzxcwu?= =?iso-8859-1?Q?OJMTyv7jdY+ovG8jHFiiB8yg9PhuXP2WusQLKtAFIivBiBrtxInJdy0ZK3?= =?iso-8859-1?Q?2+mYw64iocshe0jKmYSM3jqVnnr2ATn25GExtDYIKOwQPPysD6/8jzIaWw?= =?iso-8859-1?Q?9rL+CF+7ObpClWoAXC5AgAts05qw8m2mWXGCMWr5+5UDBigdOXv15o8W51?= =?iso-8859-1?Q?iPIbKDUR4w2dMCpG/J24jXOlhPpS/BAC4KbECCA108l4ROq1KJUesi0XZM?= =?iso-8859-1?Q?QKck93SzKkOROvCbPgEYd3L55hPv86qxCb7AHTdHWPUL6ZTvdKYuvQzbjV?= =?iso-8859-1?Q?eH4c2Mmzp9LhtemPmZlMm5dLTbJrjmHSlHpo9v3AqkzWN3NRk2ww+7Reoq?= =?iso-8859-1?Q?GwejPP/LqeCwpmT+NZ+B/XNEg6ju3e0LDBTTl0vwPIzLDabbbX0EsZyunB?= =?iso-8859-1?Q?bLgKDlepOfeRzWLxwC5njzAmwcSVeY9EMZMKcCnUY4aF7lLGGnuqDwWv63?= =?iso-8859-1?Q?dXsM7CKUPqgBBbcnGAvoBmdIVyEi7OWRsrV6TKknQ1m00KrSe1/se5d81V?= =?iso-8859-1?Q?TkdDzxKoV7NoesVyP2z0FLZ+Vi8E8IO/HJr5+/t/DYHpgkHy9/JUFvwWOS?= =?iso-8859-1?Q?vBsfiTlKMewELaxCU4V0UeO6yTuI1oldI8v3tX6qP5jWYGAzzD9LCprVyX?= =?iso-8859-1?Q?XrSHiZu1xbKeTge9JWEFr13GUGEMSZe0rZBZVUTkGoOwWJ757whrndoz98?= =?iso-8859-1?Q?rce/Z6FVtgLzPwkc/HMEEQQiPYaRthNNsSYw7quh7/a8Oosj+TQuN+Z9qZ?= =?iso-8859-1?Q?dUiQNbJYfbdYg=3D?= Content-Type: multipart/alternative; boundary="_000_DS0PR11MB74585B06A24B6A1FA6DB803A81D32DS0PR11MB7458namp_" MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7458.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8f8f8fe0-b084-4717-6356-08dc99cd0f1c X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2024 12:55:18.0082 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: owNGcYDuG2VKtXLzg38PNWJuDaXbu5Mf9iTY4bFHYW0bBUud2ZfJT4rNsJAZasocwvn6b4lePKt7jIsy7wVEUQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6408 X-OriginatorOrg: intel.com X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org --_000_DS0PR11MB74585B06A24B6A1FA6DB803A81D32DS0PR11MB7458namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Acked-by: Kai Ji ________________________________ From: Jack Bond-Preston Sent: 07 June 2024 13:47 To: Ji, Kai ; Fan Zhang ; Akhil G= oyal Cc: dev@dpdk.org ; stable@dpdk.org ; Wathsal= a Vithanage Subject: [PATCH v4 1/5] crypto/openssl: fix GCM and CCM thread unsafe ctxs Commit 67ab783b5d70 ("crypto/openssl: use local copy for session contexts") introduced a fix for concurrency bugs which could occur when using one OpenSSL PMD session across multiple cores simultaneously. The solution was to clone the EVP contexts per-buffer to avoid them being used concurrently. However, part of commit 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API") reverted this fix, only for combined ops (AES-GCM and AES-CCM). Fix the concurrency issue by cloning EVP contexts per-buffer. An extra workaround is required for OpenSSL versions which are >=3D 3.0.0, and <=3D 3.2.0. This is because, prior to OpenSSL 3.2.0, EVP_CIPHER_CTX_copy() is not implemented for AES-GCM or AES-CCM. When using these OpenSSL versions, create and initialise the context from scratch, per-buffer. Throughput performance uplift measurements for AES-GCM-128 encrypt on Ampere Altra Max platform: 1 worker lcore | buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift | |-----------------+---------------+--------------------+----------| | 64 | 2.60 | 1.31 | -49.5% | | 256 | 7.69 | 4.45 | -42.1% | | 1024 | 15.33 | 11.30 | -26.3% | | 2048 | 18.74 | 15.37 | -18.0% | | 4096 | 21.11 | 18.80 | -10.9% | 8 worker lcores | buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift | |-----------------+---------------+--------------------+----------| | 64 | 19.94 | 2.83 | -85.8% | | 256 | 58.84 | 11.00 | -81.3% | | 1024 | 119.71 | 42.46 | -64.5% | | 2048 | 147.69 | 80.91 | -45.2% | | 4096 | 167.39 | 121.25 | -27.6% | Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API"= ) Cc: stable@dpdk.org Signed-off-by: Jack Bond-Preston Reviewed-by: Wathsala Vithanage --- drivers/crypto/openssl/rte_openssl_pmd.c | 84 ++++++++++++++++++------ 1 file changed, 64 insertions(+), 20 deletions(-) diff --git a/drivers/crypto/openssl/rte_openssl_pmd.c b/drivers/crypto/open= ssl/rte_openssl_pmd.c index e8cb09defc..3f7f4d8c37 100644 --- a/drivers/crypto/openssl/rte_openssl_pmd.c +++ b/drivers/crypto/openssl/rte_openssl_pmd.c @@ -350,7 +350,8 @@ get_aead_algo(enum rte_crypto_aead_algorithm sess_algo,= size_t keylen, static int openssl_set_sess_aead_enc_param(struct openssl_session *sess, enum rte_crypto_aead_algorithm algo, - uint8_t tag_len, const uint8_t *key) + uint8_t tag_len, const uint8_t *key, + EVP_CIPHER_CTX **ctx) { int iv_type =3D 0; unsigned int do_ccm; @@ -378,7 +379,7 @@ openssl_set_sess_aead_enc_param(struct openssl_session = *sess, } sess->cipher.mode =3D OPENSSL_CIPHER_LIB; - sess->cipher.ctx =3D EVP_CIPHER_CTX_new(); + *ctx =3D EVP_CIPHER_CTX_new(); if (get_aead_algo(algo, sess->cipher.key.length, &sess->cipher.evp_algo) !=3D 0) @@ -388,19 +389,19 @@ openssl_set_sess_aead_enc_param(struct openssl_sessio= n *sess, sess->chain_order =3D OPENSSL_CHAIN_COMBINED; - if (EVP_EncryptInit_ex(sess->cipher.ctx, sess->cipher.evp_algo, + if (EVP_EncryptInit_ex(*ctx, sess->cipher.evp_algo, NULL, NULL, NULL) <=3D 0) return -EINVAL; - if (EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, iv_type, sess->iv.length, + if (EVP_CIPHER_CTX_ctrl(*ctx, iv_type, sess->iv.length, NULL) <=3D 0) return -EINVAL; if (do_ccm) - EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, EVP_CTRL_CCM_SET_TAG, + EVP_CIPHER_CTX_ctrl(*ctx, EVP_CTRL_CCM_SET_TAG, tag_len, NULL); - if (EVP_EncryptInit_ex(sess->cipher.ctx, NULL, NULL, key, NULL) <= =3D 0) + if (EVP_EncryptInit_ex(*ctx, NULL, NULL, key, NULL) <=3D 0) return -EINVAL; return 0; @@ -410,7 +411,8 @@ openssl_set_sess_aead_enc_param(struct openssl_session = *sess, static int openssl_set_sess_aead_dec_param(struct openssl_session *sess, enum rte_crypto_aead_algorithm algo, - uint8_t tag_len, const uint8_t *key) + uint8_t tag_len, const uint8_t *key, + EVP_CIPHER_CTX **ctx) { int iv_type =3D 0; unsigned int do_ccm =3D 0; @@ -437,7 +439,7 @@ openssl_set_sess_aead_dec_param(struct openssl_session = *sess, } sess->cipher.mode =3D OPENSSL_CIPHER_LIB; - sess->cipher.ctx =3D EVP_CIPHER_CTX_new(); + *ctx =3D EVP_CIPHER_CTX_new(); if (get_aead_algo(algo, sess->cipher.key.length, &sess->cipher.evp_algo) !=3D 0) @@ -447,24 +449,54 @@ openssl_set_sess_aead_dec_param(struct openssl_sessio= n *sess, sess->chain_order =3D OPENSSL_CHAIN_COMBINED; - if (EVP_DecryptInit_ex(sess->cipher.ctx, sess->cipher.evp_algo, + if (EVP_DecryptInit_ex(*ctx, sess->cipher.evp_algo, NULL, NULL, NULL) <=3D 0) return -EINVAL; - if (EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, iv_type, + if (EVP_CIPHER_CTX_ctrl(*ctx, iv_type, sess->iv.length, NULL) <=3D 0) return -EINVAL; if (do_ccm) - EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, EVP_CTRL_CCM_SET_TAG, + EVP_CIPHER_CTX_ctrl(*ctx, EVP_CTRL_CCM_SET_TAG, tag_len, NULL); - if (EVP_DecryptInit_ex(sess->cipher.ctx, NULL, NULL, key, NULL) <= =3D 0) + if (EVP_DecryptInit_ex(*ctx, NULL, NULL, key, NULL) <=3D 0) return -EINVAL; return 0; } +static int openssl_aesni_ctx_clone(EVP_CIPHER_CTX **dest, + struct openssl_session *sess) +{ +#if (OPENSSL_VERSION_NUMBER > 0x30200000L) + *dest =3D EVP_CIPHER_CTX_dup(sess->ctx); + return 0; +#elif (OPENSSL_VERSION_NUMBER > 0x30000000L) + /* OpenSSL versions 3.0.0 <=3D V < 3.2.0 have no dupctx() implement= ation + * for AES-GCM and AES-CCM. In this case, we have to create new emp= ty + * contexts and initialise, as we did the original context. + */ + if (sess->auth.algo =3D=3D RTE_CRYPTO_AUTH_AES_GMAC) + sess->aead_algo =3D RTE_CRYPTO_AEAD_AES_GCM; + + if (sess->cipher.direction =3D=3D RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return openssl_set_sess_aead_enc_param(sess, sess->aead_alg= o, + sess->auth.digest_length, sess->cipher.key.= data, + dest); + else + return openssl_set_sess_aead_dec_param(sess, sess->aead_alg= o, + sess->auth.digest_length, sess->cipher.key.= data, + dest); +#else + *dest =3D EVP_CIPHER_CTX_new(); + if (EVP_CIPHER_CTX_copy(*dest, sess->cipher.ctx) !=3D 1) + return -EINVAL; + return 0; +#endif +} + /** Set session cipher parameters */ static int openssl_set_session_cipher_parameters(struct openssl_session *sess, @@ -623,12 +655,14 @@ openssl_set_session_auth_parameters(struct openssl_se= ssion *sess, return openssl_set_sess_aead_enc_param(sess, RTE_CRYPTO_AEAD_AES_GCM, xform->auth.digest_length, - xform->auth.key.data); + xform->auth.key.data, + &sess->cipher.ctx); else return openssl_set_sess_aead_dec_param(sess, RTE_CRYPTO_AEAD_AES_GCM, xform->auth.digest_length, - xform->auth.key.data); + xform->auth.key.data, + &sess->cipher.ctx); break; case RTE_CRYPTO_AUTH_MD5: @@ -770,10 +804,12 @@ openssl_set_session_aead_parameters(struct openssl_se= ssion *sess, /* Select cipher direction */ if (xform->aead.op =3D=3D RTE_CRYPTO_AEAD_OP_ENCRYPT) return openssl_set_sess_aead_enc_param(sess, xform->aead.a= lgo, - xform->aead.digest_length, xform->aead.key.= data); + xform->aead.digest_length, xform->aead.key.= data, + &sess->cipher.ctx); else return openssl_set_sess_aead_dec_param(sess, xform->aead.a= lgo, - xform->aead.digest_length, xform->aead.key.= data); + xform->aead.digest_length, xform->aead.key.= data, + &sess->cipher.ctx); } /** Parse crypto xform chain and set private session parameters */ @@ -1590,6 +1626,12 @@ process_openssl_combined_op return; } + EVP_CIPHER_CTX *ctx; + if (openssl_aesni_ctx_clone(&ctx, sess) !=3D 0) { + op->status =3D RTE_CRYPTO_OP_STATUS_ERROR; + return; + } + iv =3D rte_crypto_op_ctod_offset(op, uint8_t *, sess->iv.offset); if (sess->auth.algo =3D=3D RTE_CRYPTO_AUTH_AES_GMAC) { @@ -1623,12 +1665,12 @@ process_openssl_combined_op status =3D process_openssl_auth_encryption_gcm( mbuf_src, offset, srclen, aad, aadlen, iv, - dst, tag, sess->cipher.ctx); + dst, tag, ctx); else status =3D process_openssl_auth_encryption_ccm( mbuf_src, offset, srclen, aad, aadlen, iv, - dst, tag, taglen, sess->cipher.ctx)= ; + dst, tag, taglen, ctx); } else { if (sess->auth.algo =3D=3D RTE_CRYPTO_AUTH_AES_GMAC || @@ -1636,14 +1678,16 @@ process_openssl_combined_op status =3D process_openssl_auth_decryption_gcm( mbuf_src, offset, srclen, aad, aadlen, iv, - dst, tag, sess->cipher.ctx); + dst, tag, ctx); else status =3D process_openssl_auth_decryption_ccm( mbuf_src, offset, srclen, aad, aadlen, iv, - dst, tag, taglen, sess->cipher.ctx)= ; + dst, tag, taglen, ctx); } + EVP_CIPHER_CTX_free(ctx); + if (status !=3D 0) { if (status =3D=3D (-EFAULT) && sess->auth.operation =3D=3D -- 2.34.1 --_000_DS0PR11MB74585B06A24B6A1FA6DB803A81D32DS0PR11MB7458namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Acked-by: Kai Ji <kai.ji@intel.com>

Commit 67ab783b5d70 ("crypto/openssl: = use local copy for session
contexts") introduced a fix for concurrency bugs which could occur whe= n
using one OpenSSL PMD session across multiple cores simultaneously. The
solution was to clone the EVP contexts per-buffer to avoid them being
used concurrently.

However, part of commit 75adf1eae44f ("crypto/openssl: update HMAC
routine with 3.0 EVP API") reverted this fix, only for combined ops (AES-GCM and AES-CCM).

Fix the concurrency issue by cloning EVP contexts per-buffer. An extra
workaround is required for OpenSSL versions which are >=3D 3.0.0, and <=3D 3.2.0. This is because, prior to OpenSSL 3.2.0, EVP_CIPHER_CTX_copy= ()
is not implemented for AES-GCM or AES-CCM. When using these OpenSSL
versions, create and initialise the context from scratch, per-buffer.

Throughput performance uplift measurements for AES-GCM-128 encrypt on
Ampere Altra Max platform:
1 worker lcore
|   buffer sz (B) |   prev (Gbps) |   optimis= ed (Gbps) |   uplift |
|-----------------+---------------+--------------------+----------|
|            &n= bsp; 64 |          2.60 | = ;            &n= bsp; 1.31 |   -49.5% |
|             2= 56 |          7.69 | &nbs= p;             = 4.45 |   -42.1% |
|            1024 |&= nbsp;        15.33 |   &n= bsp;          11.30 | &nb= sp; -26.3% |
|            2048 |&= nbsp;        18.74 |   &n= bsp;          15.37 | &nb= sp; -18.0% |
|            4096 |&= nbsp;        21.11 |   &n= bsp;          18.80 | &nb= sp; -10.9% |

8 worker lcores
|   buffer sz (B) |   prev (Gbps) |   optimis= ed (Gbps) |   uplift |
|-----------------+---------------+--------------------+----------|
|            &n= bsp; 64 |         19.94 | &nbs= p;             = 2.83 |   -85.8% |
|             2= 56 |         58.84 |  &nb= sp;           11.00 |&nbs= p;  -81.3% |
|            1024 |&= nbsp;       119.71 |    &= nbsp;         42.46 |   -= 64.5% |
|            2048 |&= nbsp;       147.69 |    &= nbsp;         80.91 |   -= 45.2% |
|            4096 |&= nbsp;       167.39 |    &= nbsp;        121.25 |   -27.6%= |

Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP= API")
Cc: stable@dpdk.org
Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
---
 drivers/crypto/openssl/rte_openssl_pmd.c | 84 ++++++++++++++++++-----= -
 1 file changed, 64 insertions(+), 20 deletions(-)

diff --git a/drivers/crypto/openssl/rte_openssl_pmd.c b/drivers/crypto/open= ssl/rte_openssl_pmd.c
index e8cb09defc..3f7f4d8c37 100644
--- a/drivers/crypto/openssl/rte_openssl_pmd.c
+++ b/drivers/crypto/openssl/rte_openssl_pmd.c
@@ -350,7 +350,8 @@ get_aead_algo(enum rte_crypto_aead_algorithm sess_algo,= size_t keylen,
 static int
 openssl_set_sess_aead_enc_param(struct openssl_session *sess,
            &nb= sp;    enum rte_crypto_aead_algorithm algo,
-            &n= bsp;  uint8_t tag_len, const uint8_t *key)
+            &n= bsp;  uint8_t tag_len, const uint8_t *key,
+            &n= bsp;  EVP_CIPHER_CTX **ctx)
 {
         int iv_type =3D 0;
         unsigned int do_ccm;
@@ -378,7 +379,7 @@ openssl_set_sess_aead_enc_param(struct openssl_session = *sess,
         }
 
         sess->cipher.mode =3D O= PENSSL_CIPHER_LIB;
-       sess->cipher.ctx =3D EVP_CIPHER_CT= X_new();
+       *ctx =3D EVP_CIPHER_CTX_new();
 
         if (get_aead_algo(algo, se= ss->cipher.key.length,
            &nb= sp;            &= sess->cipher.evp_algo) !=3D 0)
@@ -388,19 +389,19 @@ openssl_set_sess_aead_enc_param(struct openssl_sessio= n *sess,
 
         sess->chain_order =3D O= PENSSL_CHAIN_COMBINED;
 
-       if (EVP_EncryptInit_ex(sess->ciphe= r.ctx, sess->cipher.evp_algo,
+       if (EVP_EncryptInit_ex(*ctx, sess->= ;cipher.evp_algo,
            &nb= sp;            NULL,= NULL, NULL) <=3D 0)
            &nb= sp;    return -EINVAL;
 
-       if (EVP_CIPHER_CTX_ctrl(sess->ciph= er.ctx, iv_type, sess->iv.length,
+       if (EVP_CIPHER_CTX_ctrl(*ctx, iv_type= , sess->iv.length,
            &nb= sp;            NULL)= <=3D 0)
            &nb= sp;    return -EINVAL;
 
         if (do_ccm)
-            &n= bsp;  EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, EVP_CTRL_CCM_SET_TAG, +            &n= bsp;  EVP_CIPHER_CTX_ctrl(*ctx, EVP_CTRL_CCM_SET_TAG,
            &nb= sp;            =         tag_len, NULL);
 
-       if (EVP_EncryptInit_ex(sess->ciphe= r.ctx, NULL, NULL, key, NULL) <=3D 0)
+       if (EVP_EncryptInit_ex(*ctx, NULL, NU= LL, key, NULL) <=3D 0)
            &nb= sp;    return -EINVAL;
 
         return 0;
@@ -410,7 +411,8 @@ openssl_set_sess_aead_enc_param(struct openssl_session = *sess,
 static int
 openssl_set_sess_aead_dec_param(struct openssl_session *sess,
            &nb= sp;    enum rte_crypto_aead_algorithm algo,
-            &n= bsp;  uint8_t tag_len, const uint8_t *key)
+            &n= bsp;  uint8_t tag_len, const uint8_t *key,
+            &n= bsp;  EVP_CIPHER_CTX **ctx)
 {
         int iv_type =3D 0;
         unsigned int do_ccm =3D 0;=
@@ -437,7 +439,7 @@ openssl_set_sess_aead_dec_param(struct openssl_session = *sess,
         }
 
         sess->cipher.mode =3D O= PENSSL_CIPHER_LIB;
-       sess->cipher.ctx =3D EVP_CIPHER_CT= X_new();
+       *ctx =3D EVP_CIPHER_CTX_new();
 
         if (get_aead_algo(algo, se= ss->cipher.key.length,
            &nb= sp;            &= sess->cipher.evp_algo) !=3D 0)
@@ -447,24 +449,54 @@ openssl_set_sess_aead_dec_param(struct openssl_sessio= n *sess,
 
         sess->chain_order =3D O= PENSSL_CHAIN_COMBINED;
 
-       if (EVP_DecryptInit_ex(sess->ciphe= r.ctx, sess->cipher.evp_algo,
+       if (EVP_DecryptInit_ex(*ctx, sess->= ;cipher.evp_algo,
            &nb= sp;            NULL,= NULL, NULL) <=3D 0)
            &nb= sp;    return -EINVAL;
 
-       if (EVP_CIPHER_CTX_ctrl(sess->ciph= er.ctx, iv_type,
+       if (EVP_CIPHER_CTX_ctrl(*ctx, iv_type= ,
            &nb= sp;            sess-= >iv.length, NULL) <=3D 0)
            &nb= sp;    return -EINVAL;
 
         if (do_ccm)
-            &n= bsp;  EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, EVP_CTRL_CCM_SET_TAG, +            &n= bsp;  EVP_CIPHER_CTX_ctrl(*ctx, EVP_CTRL_CCM_SET_TAG,
            &nb= sp;            =         tag_len, NULL);
 
-       if (EVP_DecryptInit_ex(sess->ciphe= r.ctx, NULL, NULL, key, NULL) <=3D 0)
+       if (EVP_DecryptInit_ex(*ctx, NULL, NU= LL, key, NULL) <=3D 0)
            &nb= sp;    return -EINVAL;
 
         return 0;
 }
 
+static int openssl_aesni_ctx_clone(EVP_CIPHER_CTX **dest,
+            &n= bsp;  struct openssl_session *sess)
+{
+#if (OPENSSL_VERSION_NUMBER > 0x30200000L)
+       *dest =3D EVP_CIPHER_CTX_dup(sess->= ;ctx);
+       return 0;
+#elif (OPENSSL_VERSION_NUMBER > 0x30000000L)
+       /* OpenSSL versions 3.0.0 <=3D V &= lt; 3.2.0 have no dupctx() implementation
+        * for AES-GCM and AES-CCM. In t= his case, we have to create new empty
+        * contexts and initialise, as w= e did the original context.
+        */
+       if (sess->auth.algo =3D=3D RTE_CRY= PTO_AUTH_AES_GMAC)
+            &n= bsp;  sess->aead_algo =3D RTE_CRYPTO_AEAD_AES_GCM;
+
+       if (sess->cipher.direction =3D=3D = RTE_CRYPTO_CIPHER_OP_ENCRYPT)
+            &n= bsp;  return openssl_set_sess_aead_enc_param(sess, sess->aead_algo,=
+            &n= bsp;            = ;      sess->auth.digest_length, sess->ciphe= r.key.data,
+            &n= bsp;            = ;      dest);
+       else
+            &n= bsp;  return openssl_set_sess_aead_dec_param(sess, sess->aead_algo,=
+            &n= bsp;            = ;      sess->auth.digest_length, sess->ciphe= r.key.data,
+            &n= bsp;            = ;      dest);
+#else
+       *dest =3D EVP_CIPHER_CTX_new();
+       if (EVP_CIPHER_CTX_copy(*dest, sess-&= gt;cipher.ctx) !=3D 1)
+            &n= bsp;  return -EINVAL;
+       return 0;
+#endif
+}
+
 /** Set session cipher parameters */
 static int
 openssl_set_session_cipher_parameters(struct openssl_session *sess, @@ -623,12 +655,14 @@ openssl_set_session_auth_parameters(struct openssl_se= ssion *sess,
            &nb= sp;            retur= n openssl_set_sess_aead_enc_param(sess,
            &nb= sp;            =             &nb= sp;           RTE_CRYPTO_= AEAD_AES_GCM,
            &nb= sp;            =             &nb= sp;           xform->a= uth.digest_length,
-            &n= bsp;            = ;            &n= bsp;         xform->auth.key.dat= a);
+            &n= bsp;            = ;            &n= bsp;         xform->auth.key.dat= a,
+            &n= bsp;            = ;            &n= bsp;         &sess->cipher.c= tx);
            &nb= sp;    else
            &nb= sp;            retur= n openssl_set_sess_aead_dec_param(sess,
            &nb= sp;            =             &nb= sp;           RTE_CRYPTO_= AEAD_AES_GCM,
            &nb= sp;            =             &nb= sp;           xform->a= uth.digest_length,
-            &n= bsp;            = ;            &n= bsp;         xform->auth.key.dat= a);
+            &n= bsp;            = ;            &n= bsp;         xform->auth.key.dat= a,
+            &n= bsp;            = ;            &n= bsp;         &sess->cipher.c= tx);
            &nb= sp;    break;
 
         case RTE_CRYPTO_AUTH_MD5:<= br> @@ -770,10 +804,12 @@ openssl_set_session_aead_parameters(struct openssl_se= ssion *sess,
         /* Select cipher direction= */
         if (xform->aead.op =3D= =3D RTE_CRYPTO_AEAD_OP_ENCRYPT)
            &nb= sp;    return openssl_set_sess_aead_enc_param(sess, xform-&g= t;aead.algo,
-            &n= bsp;            = ;      xform->aead.digest_length, xform->aea= d.key.data);
+            &n= bsp;            = ;      xform->aead.digest_length, xform->aea= d.key.data,
+            &n= bsp;            = ;      &sess->cipher.ctx);
         else
            &nb= sp;    return openssl_set_sess_aead_dec_param(sess, xform-&g= t;aead.algo,
-            &n= bsp;            = ;      xform->aead.digest_length, xform->aea= d.key.data);
+            &n= bsp;            = ;      xform->aead.digest_length, xform->aea= d.key.data,
+            &n= bsp;            = ;      &sess->cipher.ctx);
 }
 
 /** Parse crypto xform chain and set private session parameters */ @@ -1590,6 +1626,12 @@ process_openssl_combined_op
            &nb= sp;    return;
         }
 
+       EVP_CIPHER_CTX *ctx;
+       if (openssl_aesni_ctx_clone(&ctx,= sess) !=3D 0) {
+            &n= bsp;  op->status =3D RTE_CRYPTO_OP_STATUS_ERROR;
+            &n= bsp;  return;
+       }
+
         iv =3D rte_crypto_op_ctod_= offset(op, uint8_t *,
            &nb= sp;            sess-= >iv.offset);
         if (sess->auth.algo =3D= =3D RTE_CRYPTO_AUTH_AES_GMAC) {
@@ -1623,12 +1665,12 @@ process_openssl_combined_op
            &nb= sp;            statu= s =3D process_openssl_auth_encryption_gcm(
            &nb= sp;            =             &nb= sp;   mbuf_src, offset, srclen,
            &nb= sp;            =             &nb= sp;   aad, aadlen, iv,
-            &n= bsp;            = ;            &n= bsp; dst, tag, sess->cipher.ctx);
+            &n= bsp;            = ;            &n= bsp; dst, tag, ctx);
            &nb= sp;    else
            &nb= sp;            statu= s =3D process_openssl_auth_encryption_ccm(
            &nb= sp;            =             &nb= sp;   mbuf_src, offset, srclen,
            &nb= sp;            =             &nb= sp;   aad, aadlen, iv,
-            &n= bsp;            = ;            &n= bsp; dst, tag, taglen, sess->cipher.ctx);
+            &n= bsp;            = ;            &n= bsp; dst, tag, taglen, ctx);
 
         } else {
            &nb= sp;    if (sess->auth.algo =3D=3D RTE_CRYPTO_AUTH_AES_GMA= C ||
@@ -1636,14 +1678,16 @@ process_openssl_combined_op
            &nb= sp;            statu= s =3D process_openssl_auth_decryption_gcm(
            &nb= sp;            =             &nb= sp;   mbuf_src, offset, srclen,
            &nb= sp;            =             &nb= sp;   aad, aadlen, iv,
-            &n= bsp;            = ;            &n= bsp; dst, tag, sess->cipher.ctx);
+            &n= bsp;            = ;            &n= bsp; dst, tag, ctx);
            &nb= sp;    else
            &nb= sp;            statu= s =3D process_openssl_auth_decryption_ccm(
            &nb= sp;            =             &nb= sp;   mbuf_src, offset, srclen,
            &nb= sp;            =             &nb= sp;   aad, aadlen, iv,
-            &n= bsp;            = ;            &n= bsp; dst, tag, taglen, sess->cipher.ctx);
+            &n= bsp;            = ;            &n= bsp; dst, tag, taglen, ctx);
         }
 
+       EVP_CIPHER_CTX_free(ctx);
+
         if (status !=3D 0) {
            &nb= sp;    if (status =3D=3D (-EFAULT) &&
            &nb= sp;            =         sess->auth.operation =3D=3D --
2.34.1

--_000_DS0PR11MB74585B06A24B6A1FA6DB803A81D32DS0PR11MB7458namp_--