From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <stable-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id BE949455AD
	for <public@inbox.dpdk.org>; Mon, 15 Jul 2024 17:28:03 +0200 (CEST)
Received: from mails.dpdk.org (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id BA83E40A71;
	Mon, 15 Jul 2024 17:28:03 +0200 (CEST)
Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com
 [209.85.128.53]) by mails.dpdk.org (Postfix) with ESMTP id B9102402DD
 for <stable@dpdk.org>; Mon, 15 Jul 2024 17:28:01 +0200 (CEST)
Received: by mail-wm1-f53.google.com with SMTP id
 5b1f17b1804b1-42726d6eca5so32375375e9.1
 for <stable@dpdk.org>; Mon, 15 Jul 2024 08:28:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1721057281; x=1721662081; darn=dpdk.org;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:from:to:cc:subject:date
 :message-id:reply-to;
 bh=xof3ds7sxZkgHFSLvV9cu8X8PvKFxWCRBj41pDMhwdI=;
 b=VzoGublg28aI6Ker/4bv1bmQXVNBMpu9BEGBvuAhgyn2Ar7CC5MNUO6YMOpYjbh15I
 jkxEvV8hy1vsksHCzoVBBaY5+gKj14Ap0kOZZQFl27FmkRbeyButddTbzZBR/nliBYzk
 UwOSb5c3BTJs9z7EbdeuNwDjxtTudIHnFbZB+hX2XCTYaNJgZc5M+QLZrpTbspubkU7P
 Ef/TTV3F7h4axNrSGBJWZd8aGXRx5iH8tIupgA7KvD1WOkMdPZJLF33OQCg49D3Hg4WL
 uiAUcqg6e5I23eDblehKdJg7cr2l60876H4iVRUzmpkQWqu+nJXM9n8o31BEChdqVwQC
 bWVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1721057281; x=1721662081;
 h=content-transfer-encoding:mime-version:references:in-reply-to
 :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=xof3ds7sxZkgHFSLvV9cu8X8PvKFxWCRBj41pDMhwdI=;
 b=tSMDY0EYcdlhtK1PZRjkAWzFmju6pEQMx3tqg15gxWfRgkLqaLZ/zxEo2CxUmszz0j
 iAuK5JzIzufx2GhyKcHR3Kplt5JSjc0y/E+udhabZvnn3OGg0mQHdb+6B5r1NjjW5/O7
 zVrh+9lccw6hvNfKNe6RoCYin6ECoXg5Ari7ikUeTb4ilOQQA42f1RoVsdSw354C0eiD
 g3jQ9RkVNCG9sk/mUsB/RsjEmjgs4ZZj3HuF9KGT7ULaKPA+5Sd1ajWn1ivbHjTVD+/m
 QaRL22w3Z1KPks3x/K7jGJcyGbhrAEhmdf8Vs9Nw0+6nfa/L4LQa79lXUA2GJblC0TEC
 krSw==
X-Forwarded-Encrypted: i=1;
 AJvYcCXzTO0ouAnOngSaaSVdUklqMpu7p1UkV3SNLZnXy3iPLfXvNTnxqB9UD3CaJoMdUIIMa2Yc0jpNm39YvD0vRNM=
X-Gm-Message-State: AOJu0YxTj8zTXTN0a1+aVKsjTErXXL5MQs4T1EI+ZosRHvWYZ7znFV0H
 eQnO77cw+mjh7EuN0NCPr/NsgTdEmsRiPA8QVzndxzScUz5VJ6L99cgXVPQN
X-Google-Smtp-Source: AGHT+IF3qZRGauhgSC0YFfaaKpHhzCRlEVMYiE+gUpL+8W+l4tABcQC0PQUxNdXOFlQdpc8OnEsabQ==
X-Received: by 2002:a05:600c:5347:b0:426:597d:d8d7 with SMTP id
 5b1f17b1804b1-426706c63a4mr128248585e9.1.1721057281269; 
 Mon, 15 Jul 2024 08:28:01 -0700 (PDT)
Received: from localhost ([2a01:4b00:d036:ae00:7aef:1aaa:3dff:d546])
 by smtp.gmail.com with ESMTPSA id
 5b1f17b1804b1-427a5edb41asm90498525e9.36.2024.07.15.08.28.00
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 15 Jul 2024 08:28:00 -0700 (PDT)
From: luca.boccassi@gmail.com
To: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Cc: Kai Ji <kai.ji@intel.com>, Wathsala Vithanage <wathsala.vithanage@arm.com>,
 dpdk stable <stable@dpdk.org>
Subject: patch 'crypto/openssl: fix GCM and CCM thread unsafe contexts' has
 been queued to stable release 22.11.6
Date: Mon, 15 Jul 2024 16:25:55 +0100
Message-Id: <20240715152704.2229503-17-luca.boccassi@gmail.com>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240715152704.2229503-1-luca.boccassi@gmail.com>
References: <20240624235907.885628-81-luca.boccassi@gmail.com>
 <20240715152704.2229503-1-luca.boccassi@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-BeenThere: stable@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: patches for DPDK stable branches <stable.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/stable>,
 <mailto:stable-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/stable/>
List-Post: <mailto:stable@dpdk.org>
List-Help: <mailto:stable-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/stable>,
 <mailto:stable-request@dpdk.org?subject=subscribe>
Errors-To: stable-bounces@dpdk.org

Hi,

FYI, your patch has been queued to stable release 22.11.6

Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet.
It will be pushed if I get no objections before 07/17/24. So please
shout if anyone has objections.

Also note that after the patch there's a diff of the upstream commit vs the
patch applied to the branch. This will indicate if there was any rebasing
needed to apply to the stable branch. If there were code changes for rebasing
(ie: not only metadata diffs), please double check that the rebase was
correctly done.

Queued patches are on a temporary branch at:
https://github.com/bluca/dpdk-stable

This queued commit can be viewed at:
https://github.com/bluca/dpdk-stable/commit/5154afdce867b5eca45e9239581dc2eca7b07777

Thanks.

Luca Boccassi

---
>From 5154afdce867b5eca45e9239581dc2eca7b07777 Mon Sep 17 00:00:00 2001
From: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Date: Wed, 3 Jul 2024 13:45:47 +0000
Subject: [PATCH] crypto/openssl: fix GCM and CCM thread unsafe contexts

[ upstream commit 78d7765f0acbb23168b7b25e25d775bea22c48ab ]

Commit 67ab783b5d70 ("crypto/openssl: use local copy for session
contexts") introduced a fix for concurrency bugs which could occur when
using one OpenSSL PMD session across multiple cores simultaneously. The
solution was to clone the EVP contexts per-buffer to avoid them being
used concurrently.

However, part of commit 75adf1eae44f ("crypto/openssl: update HMAC
routine with 3.0 EVP API") reverted this fix, only for combined ops
(AES-GCM and AES-CCM).

Fix the concurrency issue by cloning EVP contexts per-buffer. An extra
workaround is required for OpenSSL versions which are >= 3.0.0, and
<= 3.2.0. This is because, prior to OpenSSL 3.2.0, EVP_CIPHER_CTX_copy()
is not implemented for AES-GCM or AES-CCM. When using these OpenSSL
versions, create and initialise the context from scratch, per-buffer.

Throughput performance uplift measurements for AES-GCM-128 encrypt on
Ampere Altra Max platform:
1 worker lcore
|   buffer sz (B) |   prev (Gbps) |   optimised (Gbps) |   uplift |
|-----------------+---------------+--------------------+----------|
|              64 |          2.60 |               1.31 |   -49.5% |
|             256 |          7.69 |               4.45 |   -42.1% |
|            1024 |         15.33 |              11.30 |   -26.3% |
|            2048 |         18.74 |              15.37 |   -18.0% |
|            4096 |         21.11 |              18.80 |   -10.9% |

8 worker lcores
|   buffer sz (B) |   prev (Gbps) |   optimised (Gbps) |   uplift |
|-----------------+---------------+--------------------+----------|
|              64 |         19.94 |               2.83 |   -85.8% |
|             256 |         58.84 |              11.00 |   -81.3% |
|            1024 |        119.71 |              42.46 |   -64.5% |
|            2048 |        147.69 |              80.91 |   -45.2% |
|            4096 |        167.39 |             121.25 |   -27.6% |

Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API")

Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>
---
 drivers/crypto/openssl/rte_openssl_pmd.c | 84 ++++++++++++++++++------
 1 file changed, 64 insertions(+), 20 deletions(-)

diff --git a/drivers/crypto/openssl/rte_openssl_pmd.c b/drivers/crypto/openssl/rte_openssl_pmd.c
index 6ae31cb5cd..bf9a546f48 100644
--- a/drivers/crypto/openssl/rte_openssl_pmd.c
+++ b/drivers/crypto/openssl/rte_openssl_pmd.c
@@ -349,7 +349,8 @@ get_aead_algo(enum rte_crypto_aead_algorithm sess_algo, size_t keylen,
 static int
 openssl_set_sess_aead_enc_param(struct openssl_session *sess,
 		enum rte_crypto_aead_algorithm algo,
-		uint8_t tag_len, const uint8_t *key)
+		uint8_t tag_len, const uint8_t *key,
+		EVP_CIPHER_CTX **ctx)
 {
 	int iv_type = 0;
 	unsigned int do_ccm;
@@ -377,7 +378,7 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
 	}
 
 	sess->cipher.mode = OPENSSL_CIPHER_LIB;
-	sess->cipher.ctx = EVP_CIPHER_CTX_new();
+	*ctx = EVP_CIPHER_CTX_new();
 
 	if (get_aead_algo(algo, sess->cipher.key.length,
 			&sess->cipher.evp_algo) != 0)
@@ -387,19 +388,19 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
 
 	sess->chain_order = OPENSSL_CHAIN_COMBINED;
 
-	if (EVP_EncryptInit_ex(sess->cipher.ctx, sess->cipher.evp_algo,
+	if (EVP_EncryptInit_ex(*ctx, sess->cipher.evp_algo,
 			NULL, NULL, NULL) <= 0)
 		return -EINVAL;
 
-	if (EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, iv_type, sess->iv.length,
+	if (EVP_CIPHER_CTX_ctrl(*ctx, iv_type, sess->iv.length,
 			NULL) <= 0)
 		return -EINVAL;
 
 	if (do_ccm)
-		EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, EVP_CTRL_CCM_SET_TAG,
+		EVP_CIPHER_CTX_ctrl(*ctx, EVP_CTRL_CCM_SET_TAG,
 				tag_len, NULL);
 
-	if (EVP_EncryptInit_ex(sess->cipher.ctx, NULL, NULL, key, NULL) <= 0)
+	if (EVP_EncryptInit_ex(*ctx, NULL, NULL, key, NULL) <= 0)
 		return -EINVAL;
 
 	return 0;
@@ -409,7 +410,8 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
 static int
 openssl_set_sess_aead_dec_param(struct openssl_session *sess,
 		enum rte_crypto_aead_algorithm algo,
-		uint8_t tag_len, const uint8_t *key)
+		uint8_t tag_len, const uint8_t *key,
+		EVP_CIPHER_CTX **ctx)
 {
 	int iv_type = 0;
 	unsigned int do_ccm = 0;
@@ -436,7 +438,7 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess,
 	}
 
 	sess->cipher.mode = OPENSSL_CIPHER_LIB;
-	sess->cipher.ctx = EVP_CIPHER_CTX_new();
+	*ctx = EVP_CIPHER_CTX_new();
 
 	if (get_aead_algo(algo, sess->cipher.key.length,
 			&sess->cipher.evp_algo) != 0)
@@ -446,24 +448,54 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess,
 
 	sess->chain_order = OPENSSL_CHAIN_COMBINED;
 
-	if (EVP_DecryptInit_ex(sess->cipher.ctx, sess->cipher.evp_algo,
+	if (EVP_DecryptInit_ex(*ctx, sess->cipher.evp_algo,
 			NULL, NULL, NULL) <= 0)
 		return -EINVAL;
 
-	if (EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, iv_type,
+	if (EVP_CIPHER_CTX_ctrl(*ctx, iv_type,
 			sess->iv.length, NULL) <= 0)
 		return -EINVAL;
 
 	if (do_ccm)
-		EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, EVP_CTRL_CCM_SET_TAG,
+		EVP_CIPHER_CTX_ctrl(*ctx, EVP_CTRL_CCM_SET_TAG,
 				tag_len, NULL);
 
-	if (EVP_DecryptInit_ex(sess->cipher.ctx, NULL, NULL, key, NULL) <= 0)
+	if (EVP_DecryptInit_ex(*ctx, NULL, NULL, key, NULL) <= 0)
 		return -EINVAL;
 
 	return 0;
 }
 
+static int openssl_aesni_ctx_clone(EVP_CIPHER_CTX **dest,
+		struct openssl_session *sess)
+{
+#if (OPENSSL_VERSION_NUMBER >= 0x30200000L)
+	*dest = EVP_CIPHER_CTX_dup(sess->ctx);
+	return 0;
+#elif (OPENSSL_VERSION_NUMBER >= 0x30000000L)
+	/* OpenSSL versions 3.0.0 <= V < 3.2.0 have no dupctx() implementation
+	 * for AES-GCM and AES-CCM. In this case, we have to create new empty
+	 * contexts and initialise, as we did the original context.
+	 */
+	if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC)
+		sess->aead_algo = RTE_CRYPTO_AEAD_AES_GCM;
+
+	if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT)
+		return openssl_set_sess_aead_enc_param(sess, sess->aead_algo,
+				sess->auth.digest_length, sess->cipher.key.data,
+				dest);
+	else
+		return openssl_set_sess_aead_dec_param(sess, sess->aead_algo,
+				sess->auth.digest_length, sess->cipher.key.data,
+				dest);
+#else
+	*dest = EVP_CIPHER_CTX_new();
+	if (EVP_CIPHER_CTX_copy(*dest, sess->cipher.ctx) != 1)
+		return -EINVAL;
+	return 0;
+#endif
+}
+
 /** Set session cipher parameters */
 static int
 openssl_set_session_cipher_parameters(struct openssl_session *sess,
@@ -622,12 +654,14 @@ openssl_set_session_auth_parameters(struct openssl_session *sess,
 			return openssl_set_sess_aead_enc_param(sess,
 						RTE_CRYPTO_AEAD_AES_GCM,
 						xform->auth.digest_length,
-						xform->auth.key.data);
+						xform->auth.key.data,
+						&sess->cipher.ctx);
 		else
 			return openssl_set_sess_aead_dec_param(sess,
 						RTE_CRYPTO_AEAD_AES_GCM,
 						xform->auth.digest_length,
-						xform->auth.key.data);
+						xform->auth.key.data,
+						&sess->cipher.ctx);
 		break;
 
 	case RTE_CRYPTO_AUTH_MD5:
@@ -769,10 +803,12 @@ openssl_set_session_aead_parameters(struct openssl_session *sess,
 	/* Select cipher direction */
 	if (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT)
 		return openssl_set_sess_aead_enc_param(sess, xform->aead.algo,
-				xform->aead.digest_length, xform->aead.key.data);
+				xform->aead.digest_length, xform->aead.key.data,
+				&sess->cipher.ctx);
 	else
 		return openssl_set_sess_aead_dec_param(sess, xform->aead.algo,
-				xform->aead.digest_length, xform->aead.key.data);
+				xform->aead.digest_length, xform->aead.key.data,
+				&sess->cipher.ctx);
 }
 
 /** Parse crypto xform chain and set private session parameters */
@@ -1589,6 +1625,12 @@ process_openssl_combined_op
 		return;
 	}
 
+	EVP_CIPHER_CTX *ctx;
+	if (openssl_aesni_ctx_clone(&ctx, sess) != 0) {
+		op->status = RTE_CRYPTO_OP_STATUS_ERROR;
+		return;
+	}
+
 	iv = rte_crypto_op_ctod_offset(op, uint8_t *,
 			sess->iv.offset);
 	if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC) {
@@ -1622,12 +1664,12 @@ process_openssl_combined_op
 			status = process_openssl_auth_encryption_gcm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, sess->cipher.ctx);
+					dst, tag, ctx);
 		else
 			status = process_openssl_auth_encryption_ccm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, taglen, sess->cipher.ctx);
+					dst, tag, taglen, ctx);
 
 	} else {
 		if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC ||
@@ -1635,14 +1677,16 @@ process_openssl_combined_op
 			status = process_openssl_auth_decryption_gcm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, sess->cipher.ctx);
+					dst, tag, ctx);
 		else
 			status = process_openssl_auth_decryption_ccm(
 					mbuf_src, offset, srclen,
 					aad, aadlen, iv,
-					dst, tag, taglen, sess->cipher.ctx);
+					dst, tag, taglen, ctx);
 	}
 
+	EVP_CIPHER_CTX_free(ctx);
+
 	if (status != 0) {
 		if (status == (-EFAULT) &&
 				sess->auth.operation ==
-- 
2.39.2

---
  Diff of the applied patch vs upstream commit (please double-check if non-empty:
---
--- -	2024-07-15 16:19:35.657083047 +0100
+++ 0017-crypto-openssl-fix-GCM-and-CCM-thread-unsafe-context.patch	2024-07-15 16:19:34.476204499 +0100
@@ -1 +1 @@
-From 78d7765f0acbb23168b7b25e25d775bea22c48ab Mon Sep 17 00:00:00 2001
+From 5154afdce867b5eca45e9239581dc2eca7b07777 Mon Sep 17 00:00:00 2001
@@ -5,0 +6,2 @@
+[ upstream commit 78d7765f0acbb23168b7b25e25d775bea22c48ab ]
+
@@ -43 +44,0 @@
-Cc: stable@dpdk.org
@@ -53 +54 @@
-index e8cb09defc..3e547c2039 100644
+index 6ae31cb5cd..bf9a546f48 100644
@@ -56 +57 @@
-@@ -350,7 +350,8 @@ get_aead_algo(enum rte_crypto_aead_algorithm sess_algo, size_t keylen,
+@@ -349,7 +349,8 @@ get_aead_algo(enum rte_crypto_aead_algorithm sess_algo, size_t keylen,
@@ -66 +67 @@
-@@ -378,7 +379,7 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
+@@ -377,7 +378,7 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
@@ -75 +76 @@
-@@ -388,19 +389,19 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
+@@ -387,19 +388,19 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
@@ -99 +100 @@
-@@ -410,7 +411,8 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
+@@ -409,7 +410,8 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess,
@@ -109 +110 @@
-@@ -437,7 +439,7 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess,
+@@ -436,7 +438,7 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess,
@@ -118 +119 @@
-@@ -447,24 +449,54 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess,
+@@ -446,24 +448,54 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess,
@@ -177 +178 @@
-@@ -623,12 +655,14 @@ openssl_set_session_auth_parameters(struct openssl_session *sess,
+@@ -622,12 +654,14 @@ openssl_set_session_auth_parameters(struct openssl_session *sess,
@@ -194 +195 @@
-@@ -770,10 +804,12 @@ openssl_set_session_aead_parameters(struct openssl_session *sess,
+@@ -769,10 +803,12 @@ openssl_set_session_aead_parameters(struct openssl_session *sess,
@@ -209 +210 @@
-@@ -1590,6 +1626,12 @@ process_openssl_combined_op
+@@ -1589,6 +1625,12 @@ process_openssl_combined_op
@@ -222 +223 @@
-@@ -1623,12 +1665,12 @@ process_openssl_combined_op
+@@ -1622,12 +1664,12 @@ process_openssl_combined_op
@@ -237 +238 @@
-@@ -1636,14 +1678,16 @@ process_openssl_combined_op
+@@ -1635,14 +1677,16 @@ process_openssl_combined_op