From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BE949455AD for ; Mon, 15 Jul 2024 17:28:03 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BA83E40A71; Mon, 15 Jul 2024 17:28:03 +0200 (CEST) Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by mails.dpdk.org (Postfix) with ESMTP id B9102402DD for ; Mon, 15 Jul 2024 17:28:01 +0200 (CEST) Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-42726d6eca5so32375375e9.1 for ; Mon, 15 Jul 2024 08:28:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1721057281; x=1721662081; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xof3ds7sxZkgHFSLvV9cu8X8PvKFxWCRBj41pDMhwdI=; b=VzoGublg28aI6Ker/4bv1bmQXVNBMpu9BEGBvuAhgyn2Ar7CC5MNUO6YMOpYjbh15I jkxEvV8hy1vsksHCzoVBBaY5+gKj14Ap0kOZZQFl27FmkRbeyButddTbzZBR/nliBYzk UwOSb5c3BTJs9z7EbdeuNwDjxtTudIHnFbZB+hX2XCTYaNJgZc5M+QLZrpTbspubkU7P Ef/TTV3F7h4axNrSGBJWZd8aGXRx5iH8tIupgA7KvD1WOkMdPZJLF33OQCg49D3Hg4WL uiAUcqg6e5I23eDblehKdJg7cr2l60876H4iVRUzmpkQWqu+nJXM9n8o31BEChdqVwQC bWVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721057281; x=1721662081; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xof3ds7sxZkgHFSLvV9cu8X8PvKFxWCRBj41pDMhwdI=; b=tSMDY0EYcdlhtK1PZRjkAWzFmju6pEQMx3tqg15gxWfRgkLqaLZ/zxEo2CxUmszz0j iAuK5JzIzufx2GhyKcHR3Kplt5JSjc0y/E+udhabZvnn3OGg0mQHdb+6B5r1NjjW5/O7 zVrh+9lccw6hvNfKNe6RoCYin6ECoXg5Ari7ikUeTb4ilOQQA42f1RoVsdSw354C0eiD g3jQ9RkVNCG9sk/mUsB/RsjEmjgs4ZZj3HuF9KGT7ULaKPA+5Sd1ajWn1ivbHjTVD+/m QaRL22w3Z1KPks3x/K7jGJcyGbhrAEhmdf8Vs9Nw0+6nfa/L4LQa79lXUA2GJblC0TEC krSw== X-Forwarded-Encrypted: i=1; AJvYcCXzTO0ouAnOngSaaSVdUklqMpu7p1UkV3SNLZnXy3iPLfXvNTnxqB9UD3CaJoMdUIIMa2Yc0jpNm39YvD0vRNM= X-Gm-Message-State: AOJu0YxTj8zTXTN0a1+aVKsjTErXXL5MQs4T1EI+ZosRHvWYZ7znFV0H eQnO77cw+mjh7EuN0NCPr/NsgTdEmsRiPA8QVzndxzScUz5VJ6L99cgXVPQN X-Google-Smtp-Source: AGHT+IF3qZRGauhgSC0YFfaaKpHhzCRlEVMYiE+gUpL+8W+l4tABcQC0PQUxNdXOFlQdpc8OnEsabQ== X-Received: by 2002:a05:600c:5347:b0:426:597d:d8d7 with SMTP id 5b1f17b1804b1-426706c63a4mr128248585e9.1.1721057281269; Mon, 15 Jul 2024 08:28:01 -0700 (PDT) Received: from localhost ([2a01:4b00:d036:ae00:7aef:1aaa:3dff:d546]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-427a5edb41asm90498525e9.36.2024.07.15.08.28.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 15 Jul 2024 08:28:00 -0700 (PDT) From: luca.boccassi@gmail.com To: Jack Bond-Preston Cc: Kai Ji , Wathsala Vithanage , dpdk stable Subject: patch 'crypto/openssl: fix GCM and CCM thread unsafe contexts' has been queued to stable release 22.11.6 Date: Mon, 15 Jul 2024 16:25:55 +0100 Message-Id: <20240715152704.2229503-17-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240715152704.2229503-1-luca.boccassi@gmail.com> References: <20240624235907.885628-81-luca.boccassi@gmail.com> <20240715152704.2229503-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 22.11.6 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 07/17/24. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/5154afdce867b5eca45e9239581dc2eca7b07777 Thanks. Luca Boccassi --- >From 5154afdce867b5eca45e9239581dc2eca7b07777 Mon Sep 17 00:00:00 2001 From: Jack Bond-Preston Date: Wed, 3 Jul 2024 13:45:47 +0000 Subject: [PATCH] crypto/openssl: fix GCM and CCM thread unsafe contexts [ upstream commit 78d7765f0acbb23168b7b25e25d775bea22c48ab ] Commit 67ab783b5d70 ("crypto/openssl: use local copy for session contexts") introduced a fix for concurrency bugs which could occur when using one OpenSSL PMD session across multiple cores simultaneously. The solution was to clone the EVP contexts per-buffer to avoid them being used concurrently. However, part of commit 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API") reverted this fix, only for combined ops (AES-GCM and AES-CCM). Fix the concurrency issue by cloning EVP contexts per-buffer. An extra workaround is required for OpenSSL versions which are >= 3.0.0, and <= 3.2.0. This is because, prior to OpenSSL 3.2.0, EVP_CIPHER_CTX_copy() is not implemented for AES-GCM or AES-CCM. When using these OpenSSL versions, create and initialise the context from scratch, per-buffer. Throughput performance uplift measurements for AES-GCM-128 encrypt on Ampere Altra Max platform: 1 worker lcore | buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift | |-----------------+---------------+--------------------+----------| | 64 | 2.60 | 1.31 | -49.5% | | 256 | 7.69 | 4.45 | -42.1% | | 1024 | 15.33 | 11.30 | -26.3% | | 2048 | 18.74 | 15.37 | -18.0% | | 4096 | 21.11 | 18.80 | -10.9% | 8 worker lcores | buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift | |-----------------+---------------+--------------------+----------| | 64 | 19.94 | 2.83 | -85.8% | | 256 | 58.84 | 11.00 | -81.3% | | 1024 | 119.71 | 42.46 | -64.5% | | 2048 | 147.69 | 80.91 | -45.2% | | 4096 | 167.39 | 121.25 | -27.6% | Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API") Signed-off-by: Jack Bond-Preston Acked-by: Kai Ji Reviewed-by: Wathsala Vithanage --- drivers/crypto/openssl/rte_openssl_pmd.c | 84 ++++++++++++++++++------ 1 file changed, 64 insertions(+), 20 deletions(-) diff --git a/drivers/crypto/openssl/rte_openssl_pmd.c b/drivers/crypto/openssl/rte_openssl_pmd.c index 6ae31cb5cd..bf9a546f48 100644 --- a/drivers/crypto/openssl/rte_openssl_pmd.c +++ b/drivers/crypto/openssl/rte_openssl_pmd.c @@ -349,7 +349,8 @@ get_aead_algo(enum rte_crypto_aead_algorithm sess_algo, size_t keylen, static int openssl_set_sess_aead_enc_param(struct openssl_session *sess, enum rte_crypto_aead_algorithm algo, - uint8_t tag_len, const uint8_t *key) + uint8_t tag_len, const uint8_t *key, + EVP_CIPHER_CTX **ctx) { int iv_type = 0; unsigned int do_ccm; @@ -377,7 +378,7 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, } sess->cipher.mode = OPENSSL_CIPHER_LIB; - sess->cipher.ctx = EVP_CIPHER_CTX_new(); + *ctx = EVP_CIPHER_CTX_new(); if (get_aead_algo(algo, sess->cipher.key.length, &sess->cipher.evp_algo) != 0) @@ -387,19 +388,19 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, sess->chain_order = OPENSSL_CHAIN_COMBINED; - if (EVP_EncryptInit_ex(sess->cipher.ctx, sess->cipher.evp_algo, + if (EVP_EncryptInit_ex(*ctx, sess->cipher.evp_algo, NULL, NULL, NULL) <= 0) return -EINVAL; - if (EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, iv_type, sess->iv.length, + if (EVP_CIPHER_CTX_ctrl(*ctx, iv_type, sess->iv.length, NULL) <= 0) return -EINVAL; if (do_ccm) - EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, EVP_CTRL_CCM_SET_TAG, + EVP_CIPHER_CTX_ctrl(*ctx, EVP_CTRL_CCM_SET_TAG, tag_len, NULL); - if (EVP_EncryptInit_ex(sess->cipher.ctx, NULL, NULL, key, NULL) <= 0) + if (EVP_EncryptInit_ex(*ctx, NULL, NULL, key, NULL) <= 0) return -EINVAL; return 0; @@ -409,7 +410,8 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, static int openssl_set_sess_aead_dec_param(struct openssl_session *sess, enum rte_crypto_aead_algorithm algo, - uint8_t tag_len, const uint8_t *key) + uint8_t tag_len, const uint8_t *key, + EVP_CIPHER_CTX **ctx) { int iv_type = 0; unsigned int do_ccm = 0; @@ -436,7 +438,7 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess, } sess->cipher.mode = OPENSSL_CIPHER_LIB; - sess->cipher.ctx = EVP_CIPHER_CTX_new(); + *ctx = EVP_CIPHER_CTX_new(); if (get_aead_algo(algo, sess->cipher.key.length, &sess->cipher.evp_algo) != 0) @@ -446,24 +448,54 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess, sess->chain_order = OPENSSL_CHAIN_COMBINED; - if (EVP_DecryptInit_ex(sess->cipher.ctx, sess->cipher.evp_algo, + if (EVP_DecryptInit_ex(*ctx, sess->cipher.evp_algo, NULL, NULL, NULL) <= 0) return -EINVAL; - if (EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, iv_type, + if (EVP_CIPHER_CTX_ctrl(*ctx, iv_type, sess->iv.length, NULL) <= 0) return -EINVAL; if (do_ccm) - EVP_CIPHER_CTX_ctrl(sess->cipher.ctx, EVP_CTRL_CCM_SET_TAG, + EVP_CIPHER_CTX_ctrl(*ctx, EVP_CTRL_CCM_SET_TAG, tag_len, NULL); - if (EVP_DecryptInit_ex(sess->cipher.ctx, NULL, NULL, key, NULL) <= 0) + if (EVP_DecryptInit_ex(*ctx, NULL, NULL, key, NULL) <= 0) return -EINVAL; return 0; } +static int openssl_aesni_ctx_clone(EVP_CIPHER_CTX **dest, + struct openssl_session *sess) +{ +#if (OPENSSL_VERSION_NUMBER >= 0x30200000L) + *dest = EVP_CIPHER_CTX_dup(sess->ctx); + return 0; +#elif (OPENSSL_VERSION_NUMBER >= 0x30000000L) + /* OpenSSL versions 3.0.0 <= V < 3.2.0 have no dupctx() implementation + * for AES-GCM and AES-CCM. In this case, we have to create new empty + * contexts and initialise, as we did the original context. + */ + if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC) + sess->aead_algo = RTE_CRYPTO_AEAD_AES_GCM; + + if (sess->cipher.direction == RTE_CRYPTO_CIPHER_OP_ENCRYPT) + return openssl_set_sess_aead_enc_param(sess, sess->aead_algo, + sess->auth.digest_length, sess->cipher.key.data, + dest); + else + return openssl_set_sess_aead_dec_param(sess, sess->aead_algo, + sess->auth.digest_length, sess->cipher.key.data, + dest); +#else + *dest = EVP_CIPHER_CTX_new(); + if (EVP_CIPHER_CTX_copy(*dest, sess->cipher.ctx) != 1) + return -EINVAL; + return 0; +#endif +} + /** Set session cipher parameters */ static int openssl_set_session_cipher_parameters(struct openssl_session *sess, @@ -622,12 +654,14 @@ openssl_set_session_auth_parameters(struct openssl_session *sess, return openssl_set_sess_aead_enc_param(sess, RTE_CRYPTO_AEAD_AES_GCM, xform->auth.digest_length, - xform->auth.key.data); + xform->auth.key.data, + &sess->cipher.ctx); else return openssl_set_sess_aead_dec_param(sess, RTE_CRYPTO_AEAD_AES_GCM, xform->auth.digest_length, - xform->auth.key.data); + xform->auth.key.data, + &sess->cipher.ctx); break; case RTE_CRYPTO_AUTH_MD5: @@ -769,10 +803,12 @@ openssl_set_session_aead_parameters(struct openssl_session *sess, /* Select cipher direction */ if (xform->aead.op == RTE_CRYPTO_AEAD_OP_ENCRYPT) return openssl_set_sess_aead_enc_param(sess, xform->aead.algo, - xform->aead.digest_length, xform->aead.key.data); + xform->aead.digest_length, xform->aead.key.data, + &sess->cipher.ctx); else return openssl_set_sess_aead_dec_param(sess, xform->aead.algo, - xform->aead.digest_length, xform->aead.key.data); + xform->aead.digest_length, xform->aead.key.data, + &sess->cipher.ctx); } /** Parse crypto xform chain and set private session parameters */ @@ -1589,6 +1625,12 @@ process_openssl_combined_op return; } + EVP_CIPHER_CTX *ctx; + if (openssl_aesni_ctx_clone(&ctx, sess) != 0) { + op->status = RTE_CRYPTO_OP_STATUS_ERROR; + return; + } + iv = rte_crypto_op_ctod_offset(op, uint8_t *, sess->iv.offset); if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC) { @@ -1622,12 +1664,12 @@ process_openssl_combined_op status = process_openssl_auth_encryption_gcm( mbuf_src, offset, srclen, aad, aadlen, iv, - dst, tag, sess->cipher.ctx); + dst, tag, ctx); else status = process_openssl_auth_encryption_ccm( mbuf_src, offset, srclen, aad, aadlen, iv, - dst, tag, taglen, sess->cipher.ctx); + dst, tag, taglen, ctx); } else { if (sess->auth.algo == RTE_CRYPTO_AUTH_AES_GMAC || @@ -1635,14 +1677,16 @@ process_openssl_combined_op status = process_openssl_auth_decryption_gcm( mbuf_src, offset, srclen, aad, aadlen, iv, - dst, tag, sess->cipher.ctx); + dst, tag, ctx); else status = process_openssl_auth_decryption_ccm( mbuf_src, offset, srclen, aad, aadlen, iv, - dst, tag, taglen, sess->cipher.ctx); + dst, tag, taglen, ctx); } + EVP_CIPHER_CTX_free(ctx); + if (status != 0) { if (status == (-EFAULT) && sess->auth.operation == -- 2.39.2 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2024-07-15 16:19:35.657083047 +0100 +++ 0017-crypto-openssl-fix-GCM-and-CCM-thread-unsafe-context.patch 2024-07-15 16:19:34.476204499 +0100 @@ -1 +1 @@ -From 78d7765f0acbb23168b7b25e25d775bea22c48ab Mon Sep 17 00:00:00 2001 +From 5154afdce867b5eca45e9239581dc2eca7b07777 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 78d7765f0acbb23168b7b25e25d775bea22c48ab ] + @@ -43 +44,0 @@ -Cc: stable@dpdk.org @@ -53 +54 @@ -index e8cb09defc..3e547c2039 100644 +index 6ae31cb5cd..bf9a546f48 100644 @@ -56 +57 @@ -@@ -350,7 +350,8 @@ get_aead_algo(enum rte_crypto_aead_algorithm sess_algo, size_t keylen, +@@ -349,7 +349,8 @@ get_aead_algo(enum rte_crypto_aead_algorithm sess_algo, size_t keylen, @@ -66 +67 @@ -@@ -378,7 +379,7 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, +@@ -377,7 +378,7 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, @@ -75 +76 @@ -@@ -388,19 +389,19 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, +@@ -387,19 +388,19 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, @@ -99 +100 @@ -@@ -410,7 +411,8 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, +@@ -409,7 +410,8 @@ openssl_set_sess_aead_enc_param(struct openssl_session *sess, @@ -109 +110 @@ -@@ -437,7 +439,7 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess, +@@ -436,7 +438,7 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess, @@ -118 +119 @@ -@@ -447,24 +449,54 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess, +@@ -446,24 +448,54 @@ openssl_set_sess_aead_dec_param(struct openssl_session *sess, @@ -177 +178 @@ -@@ -623,12 +655,14 @@ openssl_set_session_auth_parameters(struct openssl_session *sess, +@@ -622,12 +654,14 @@ openssl_set_session_auth_parameters(struct openssl_session *sess, @@ -194 +195 @@ -@@ -770,10 +804,12 @@ openssl_set_session_aead_parameters(struct openssl_session *sess, +@@ -769,10 +803,12 @@ openssl_set_session_aead_parameters(struct openssl_session *sess, @@ -209 +210 @@ -@@ -1590,6 +1626,12 @@ process_openssl_combined_op +@@ -1589,6 +1625,12 @@ process_openssl_combined_op @@ -222 +223 @@ -@@ -1623,12 +1665,12 @@ process_openssl_combined_op +@@ -1622,12 +1664,12 @@ process_openssl_combined_op @@ -237 +238 @@ -@@ -1636,14 +1678,16 @@ process_openssl_combined_op +@@ -1635,14 +1677,16 @@ process_openssl_combined_op