patches for DPDK stable branches
 help / color / mirror / Atom feed
From: Akhil Goyal <gakhil@marvell.com>
To: Jack Bond-Preston <jack.bond-preston@foss.arm.com>,
	Kai Ji <kai.ji@intel.com>, Fan Zhang <fanzhang.oss@gmail.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
	"stable@dpdk.org" <stable@dpdk.org>,
	Wathsala Vithanage <wathsala.vithanage@arm.com>
Subject: RE: [EXTERNAL] [PATCH v4 1/5] crypto/openssl: fix GCM and CCM thread unsafe ctxs
Date: Tue, 2 Jul 2024 15:39:26 +0000	[thread overview]
Message-ID: <CO6PR18MB44849FA7EE39AA35E2D3E91BD8DC2@CO6PR18MB4484.namprd18.prod.outlook.com> (raw)
In-Reply-To: <20240607124756.3968704-2-jack.bond-preston@foss.arm.com>

> Commit 67ab783b5d70 ("crypto/openssl: use local copy for session
> contexts") introduced a fix for concurrency bugs which could occur when
> using one OpenSSL PMD session across multiple cores simultaneously. The
> solution was to clone the EVP contexts per-buffer to avoid them being
> used concurrently.
> 
> However, part of commit 75adf1eae44f ("crypto/openssl: update HMAC
> routine with 3.0 EVP API") reverted this fix, only for combined ops
> (AES-GCM and AES-CCM).
> 
> Fix the concurrency issue by cloning EVP contexts per-buffer. An extra
> workaround is required for OpenSSL versions which are >= 3.0.0, and
> <= 3.2.0. This is because, prior to OpenSSL 3.2.0, EVP_CIPHER_CTX_copy()
> is not implemented for AES-GCM or AES-CCM. When using these OpenSSL
> versions, create and initialise the context from scratch, per-buffer.
> 
> Throughput performance uplift measurements for AES-GCM-128 encrypt on
> Ampere Altra Max platform:
> 1 worker lcore
> |   buffer sz (B) |   prev (Gbps) |   optimised (Gbps) |   uplift |
> |-----------------+---------------+--------------------+----------|
> |              64 |          2.60 |               1.31 |   -49.5% |
> |             256 |          7.69 |               4.45 |   -42.1% |
> |            1024 |         15.33 |              11.30 |   -26.3% |
> |            2048 |         18.74 |              15.37 |   -18.0% |
> |            4096 |         21.11 |              18.80 |   -10.9% |
> 
> 8 worker lcores
> |   buffer sz (B) |   prev (Gbps) |   optimised (Gbps) |   uplift |
> |-----------------+---------------+--------------------+----------|
> |              64 |         19.94 |               2.83 |   -85.8% |
> |             256 |         58.84 |              11.00 |   -81.3% |
> |            1024 |        119.71 |              42.46 |   -64.5% |
> |            2048 |        147.69 |              80.91 |   -45.2% |
> |            4096 |        167.39 |             121.25 |   -27.6% |
> 
> Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API")
> Cc: stable@dpdk.org
> Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
> Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>


I am seeing below errors when it is compiled with openssl 3.0

[1/30] Compiling C object 'drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd_ops.c.o'.
FAILED: drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd_ops.c.o
ccache gcc -Idrivers/a715181@@tmp_rte_crypto_openssl@sta -Idrivers -I../drivers -Idrivers/crypto/openssl -I../drivers/crypto/openssl -Ilib/cryptodev -I../lib/cryptodev -I. -I../ -Iconfig -I../config -Ilib/eal/include -I../lib/eal/include -Ilib/eal/linux/include -I../lib/eal/linux/include -Ilib/eal/x86/include -I../lib/eal/x86/include -Ilib/eal/common -I../lib/eal/common -Ilib/eal -I../lib/eal -Ilib/kvargs -I../lib/kvargs -Ilib/log -I../lib/log -Ilib/telemetry/../metrics -I../lib/telemetry/../metrics -Ilib/telemetry -I../lib/telemetry -Ilib/mbuf -I../lib/mbuf -Ilib/mempool -I../lib/mempool -Ilib/ring -I../lib/ring -Ilib/rcu -I../lib/rcu -Idrivers/bus/vdev -I../drivers/bus/vdev -I/usr/local/include -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Werror -std=c11 -O2 -g -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned -Wno-missing-field-initializers -D_GNU_SOURCE -fPIC -march=native -mrtm -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DRTE_LOG_DEFAULT_LOGTYPE=pmd.crypto.openssl -MD -MQ 'drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd_ops.c.o' -MF 'drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd_ops.c.o.d' -o 'drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd_ops.c.o' -c ../drivers/crypto/openssl/rte_openssl_pmd_ops.c
In file included from ../drivers/crypto/openssl/rte_openssl_pmd_ops.c:12:
../drivers/crypto/openssl/compat.h: In function 'free_hmac_ctx':
../drivers/crypto/openssl/compat.h:24:2: error: 'HMAC_CTX_free' is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
   24 |  HMAC_CTX_free(ctx);
      |  ^~~~~~~~~~~~~
In file included from ../drivers/crypto/openssl/openssl_pmd_private.h:10,
                 from ../drivers/crypto/openssl/rte_openssl_pmd_ops.c:11:
/usr/local/include/openssl/hmac.h:35:28: note: declared here
   35 | OSSL_DEPRECATEDIN_3_0 void HMAC_CTX_free(HMAC_CTX *ctx);
      |                            ^~~~~~~~~~~~~
In file included from ../drivers/crypto/openssl/rte_openssl_pmd_ops.c:12:
../drivers/crypto/openssl/compat.h: In function 'free_cmac_ctx':
../drivers/crypto/openssl/compat.h:30:2: error: 'CMAC_CTX_free' is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
   30 |  CMAC_CTX_free(ctx);
      |  ^~~~~~~~~~~~~
In file included from ../drivers/crypto/openssl/openssl_pmd_private.h:9,
                 from ../drivers/crypto/openssl/rte_openssl_pmd_ops.c:11:
/usr/local/include/openssl/cmac.h:34:28: note: declared here
   34 | OSSL_DEPRECATEDIN_3_0 void CMAC_CTX_free(CMAC_CTX *ctx);
      |                            ^~~~~~~~~~~~~
cc1: all warnings being treated as errors
[2/30] Compiling C object 'drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd.c.o'.
FAILED: drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd.c.o
ccache gcc -Idrivers/a715181@@tmp_rte_crypto_openssl@sta -Idrivers -I../drivers -Idrivers/crypto/openssl -I../drivers/crypto/openssl -Ilib/cryptodev -I../lib/cryptodev -I. -I../ -Iconfig -I../config -Ilib/eal/include -I../lib/eal/include -Ilib/eal/linux/include -I../lib/eal/linux/include -Ilib/eal/x86/include -I../lib/eal/x86/include -Ilib/eal/common -I../lib/eal/common -Ilib/eal -I../lib/eal -Ilib/kvargs -I../lib/kvargs -Ilib/log -I../lib/log -Ilib/telemetry/../metrics -I../lib/telemetry/../metrics -Ilib/telemetry -I../lib/telemetry -Ilib/mbuf -I../lib/mbuf -Ilib/mempool -I../lib/mempool -Ilib/ring -I../lib/ring -Ilib/rcu -I../lib/rcu -Idrivers/bus/vdev -I../drivers/bus/vdev -I/usr/local/include -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Werror -std=c11 -O2 -g -include rte_config.h -Wcast-qual -Wdeprecated -Wformat -Wformat-nonliteral -Wformat-security -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpointer-arith -Wsign-compare -Wstrict-prototypes -Wundef -Wwrite-strings -Wno-address-of-packed-member -Wno-packed-not-aligned -Wno-missing-field-initializers -D_GNU_SOURCE -fPIC -march=native -mrtm -DALLOW_EXPERIMENTAL_API -DALLOW_INTERNAL_API -Wno-format-truncation -DRTE_LOG_DEFAULT_LOGTYPE=pmd.crypto.openssl -MD -MQ 'drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd.c.o' -MF 'drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd.c.o.d' -o 'drivers/a715181@@tmp_rte_crypto_openssl@sta/crypto_openssl_rte_openssl_pmd.c.o' -c ../drivers/crypto/openssl/rte_openssl_pmd.c
In file included from ../drivers/crypto/openssl/rte_openssl_pmd.c:19:
../drivers/crypto/openssl/compat.h: In function 'free_hmac_ctx':
../drivers/crypto/openssl/compat.h:24:2: error: 'HMAC_CTX_free' is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
   24 |  HMAC_CTX_free(ctx);
      |  ^~~~~~~~~~~~~
In file included from ../drivers/crypto/openssl/rte_openssl_pmd.c:14:
/usr/local/include/openssl/hmac.h:35:28: note: declared here
   35 | OSSL_DEPRECATEDIN_3_0 void HMAC_CTX_free(HMAC_CTX *ctx);
      |                            ^~~~~~~~~~~~~
In file included from ../drivers/crypto/openssl/rte_openssl_pmd.c:19:
../drivers/crypto/openssl/compat.h: In function 'free_cmac_ctx':
../drivers/crypto/openssl/compat.h:30:2: error: 'CMAC_CTX_free' is deprecated: Since OpenSSL 3.0 [-Werror=deprecated-declarations]
   30 |  CMAC_CTX_free(ctx);
      |  ^~~~~~~~~~~~~
In file included from ../drivers/crypto/openssl/rte_openssl_pmd.c:13:
/usr/local/include/openssl/cmac.h:34:28: note: declared here
   34 | OSSL_DEPRECATEDIN_3_0 void CMAC_CTX_free(CMAC_CTX *ctx);
      |                            ^~~~~~~~~~~~~
../drivers/crypto/openssl/rte_openssl_pmd.c: In function 'openssl_reset_session':
../drivers/crypto/openssl/rte_openssl_pmd.c:908:33: error: passing argument 1 of 'free_hmac_ctx' from incompatible pointer type [-Werror=incompatible-pointer-types]
  908 |    free_hmac_ctx(sess->qp_ctx[i].hmac);
      |                  ~~~~~~~~~~~~~~~^~~~~
      |                                 |
      |                                 EVP_MAC_CTX * {aka struct evp_mac_ctx_st *}
In file included from ../drivers/crypto/openssl/rte_openssl_pmd.c:19:
../drivers/crypto/openssl/compat.h:22:25: note: expected 'HMAC_CTX *' {aka 'struct hmac_ctx_st *'} but argument is of type 'EVP_MAC_CTX *' {aka 'struct evp_mac_ctx_st *'}
   22 | free_hmac_ctx(HMAC_CTX *ctx)
      |               ~~~~~~~~~~^~~
../drivers/crypto/openssl/rte_openssl_pmd.c:912:33: error: passing argument 1 of 'free_cmac_ctx' from incompatible pointer type [-Werror=incompatible-pointer-types]
  912 |    free_cmac_ctx(sess->qp_ctx[i].cmac);
      |                  ~~~~~~~~~~~~~~~^~~~~
      |                                 |
      |                                 EVP_MAC_CTX * {aka struct evp_mac_ctx_st *}
In file included from ../drivers/crypto/openssl/rte_openssl_pmd.c:19:
../drivers/crypto/openssl/compat.h:28:25: note: expected 'CMAC_CTX *' {aka 'struct CMAC_CTX_st *'} but argument is of type 'EVP_MAC_CTX *' {aka 'struct evp_mac_ctx_st *'}
   28 | free_cmac_ctx(CMAC_CTX *ctx)
      |               ~~~~~~~~~~^~~
../drivers/crypto/openssl/rte_openssl_pmd.c:925:32: error: passing argument 1 of 'free_hmac_ctx' from incompatible pointer type [-Werror=incompatible-pointer-types]
  925 |   free_hmac_ctx(sess->auth.hmac.ctx);
      |                 ~~~~~~~~~~~~~~~^~~~
      |                                |
      |                                EVP_MAC_CTX * {aka struct evp_mac_ctx_st *}
In file included from ../drivers/crypto/openssl/rte_openssl_pmd.c:19:
../drivers/crypto/openssl/compat.h:22:25: note: expected 'HMAC_CTX *' {aka 'struct hmac_ctx_st *'} but argument is of type 'EVP_MAC_CTX *' {aka 'struct evp_mac_ctx_st *'}
   22 | free_hmac_ctx(HMAC_CTX *ctx)
      |               ~~~~~~~~~~^~~
../drivers/crypto/openssl/rte_openssl_pmd.c:928:32: error: passing argument 1 of 'free_cmac_ctx' from incompatible pointer type [-Werror=incompatible-pointer-types]
  928 |   free_cmac_ctx(sess->auth.cmac.ctx);
      |                 ~~~~~~~~~~~~~~~^~~~
      |                                |
      |                                EVP_MAC_CTX * {aka struct evp_mac_ctx_st *}
In file included from ../drivers/crypto/openssl/rte_openssl_pmd.c:19:
../drivers/crypto/openssl/compat.h:28:25: note: expected 'CMAC_CTX *' {aka 'struct CMAC_CTX_st *'} but argument is of type 'EVP_MAC_CTX *' {aka 'struct evp_mac_ctx_st *'}
   28 | free_cmac_ctx(CMAC_CTX *ctx)
      |               ~~~~~~~~~~^~~
cc1: all warnings being treated as errors

  parent reply	other threads:[~2024-07-02 15:39 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <20240603160119.1279476-1-jack.bond-preston@foss.arm.com>
2024-06-03 16:01 ` [PATCH " Jack Bond-Preston
     [not found] ` <20240603184348.1310331-1-jack.bond-preston@foss.arm.com>
2024-06-03 18:43   ` [PATCH v2 " Jack Bond-Preston
     [not found] ` <20240603185939.1312680-1-jack.bond-preston@foss.arm.com>
2024-06-03 18:59   ` Jack Bond-Preston
     [not found] ` <20240606102043.2926695-1-jack.bond-preston@foss.arm.com>
2024-06-06 10:20   ` [PATCH v3 " Jack Bond-Preston
2024-06-06 10:44     ` [EXTERNAL] " Akhil Goyal
     [not found] ` <20240607124756.3968704-1-jack.bond-preston@foss.arm.com>
2024-06-07 12:47   ` [PATCH v4 " Jack Bond-Preston
2024-07-01 12:55     ` Ji, Kai
2024-07-02 15:39     ` Akhil Goyal [this message]
2024-07-03 10:49       ` [EXTERNAL] " Jack Bond-Preston
     [not found] ` <20240703134552.1439633-1-jack.bond-preston@foss.arm.com>
2024-07-03 13:45   ` [PATCH v5 " Jack Bond-Preston
2024-07-03 15:20     ` [EXTERNAL] " Akhil Goyal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CO6PR18MB44849FA7EE39AA35E2D3E91BD8DC2@CO6PR18MB4484.namprd18.prod.outlook.com \
    --to=gakhil@marvell.com \
    --cc=dev@dpdk.org \
    --cc=fanzhang.oss@gmail.com \
    --cc=jack.bond-preston@foss.arm.com \
    --cc=kai.ji@intel.com \
    --cc=stable@dpdk.org \
    --cc=wathsala.vithanage@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).