From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 25410489EF for ; Mon, 27 Oct 2025 17:24:04 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 1F64B40695; Mon, 27 Oct 2025 17:24:04 +0100 (CET) Received: from mail-wm1-f53.google.com (mail-wm1-f53.google.com [209.85.128.53]) by mails.dpdk.org (Postfix) with ESMTP id B906C4069D for ; Mon, 27 Oct 2025 17:24:02 +0100 (CET) Received: by mail-wm1-f53.google.com with SMTP id 5b1f17b1804b1-47112edf9f7so30468765e9.0 for ; Mon, 27 Oct 2025 09:24:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1761582242; x=1762187042; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=YcbuHcw2lO7V8rBqCm8NCr+N/gMLutnsVD2rLxox6pg=; b=hdlzbOdnC28szzjN/ZiPnPRGrUCXFImtgy9INYqVfhw56Qceq+SSpU+R/AujfmWU57 a3/QRynBLU2HyZkuYUqroh9zEBe/y7PinfPINXx7gFQHNl233g0pYKJL6EK6CxoipmIN Ht4pc9y+zTZCuLuU3oDgVOZuDRBkDT+9QqVaNFzleC6NWXf41fbwAf7xUzFrMA3U0Lfr BU09cOx7pr1oYEASFS2HlucEzg6gs4ae2Bgj5TRwg9Wl3ZYf5VWfLG9cb4K5yj8WFwCR vVxZSnV5neplGdoZGNgXTJDTlhcOcv8ggqWPlXVMRyy5K47GAbWRXNaiLtZZeXJg2SqZ mbow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1761582242; x=1762187042; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YcbuHcw2lO7V8rBqCm8NCr+N/gMLutnsVD2rLxox6pg=; b=oHF3DIqiEswhOBinejtEc7lb5qWVED9dOGzkyjSinWKQRfK2VuRASIPDthwFz1owIk x6SmNha/IHoUnMdUIeu4aRRFLllKxYubxdAK/lO98429SLV2sQpxa0ndtKwEPntWZ8Aa S0UDYfZq4iS5r8Da/gEzqG/JRUQEryCnzFkuzLm6wShXl7mJYjcotCdFb0EAKWL/uj0w 6cu0rj6j34G/9udBLgCVOXgs+m+VbSuj+w3bd4z0qGoRe/Xsloc6OCoIU6FeQ83a9Utw Fp1aAXlNbQ0r7oVXpefTzMuKmAflfCedH4B+4I8jcJaFx8fsyOmMG02YoCDzwc8TLytr H5VA== X-Forwarded-Encrypted: i=1; AJvYcCWojpmW5Iko+mTpHoMWVRaG2Xc7mDU82IT5VnHWtnA7Wqy9fAuqd4eNb6kQTEoxlQeTGpUPBDI=@dpdk.org X-Gm-Message-State: AOJu0YyHvtXWssHzFDOI/wogsT6xgMoJTLt3dpgQfqZIp/9XEt46uT3h 5gjAHRdaPgIS0J547Rx+bgOdTbJoyhQuvGHY2q5K0Ts1DGpBy786BCLOdguchWvN X-Gm-Gg: ASbGnctc4V66nlEd/tE5a/XurITlIM4A8R3DZQi46nN7KCYfAbbwLe1njy85jbZ4/Kn NQDhFg7OqPvk+EHJb1qidGyLHV/H7/AAD2ZJLPezDITn6GGKdtwBKKQYwL2TpP8K1KUXpNKWLl8 gZf25LFLHEuFh5o1DPgxV+surk1AuLsOswHJxzd3rqFpcGpFmHoY1B5k4NLMDzJPXc9ZBuYUlxL xBiBZJxHgDzlFJWFKdKvLklq/XydngsYELR4c3LqsPxzUJbX5P4i7qLXpayABLfQp4U6XfnC3Li NGPuVV78ZnvblZVDfUICOwz4xyLHFQpZADa4EcR4AgvgLUb23VxjF4r5g3qa0p6IYkiLM0kOkvo CeaZXPDTQhGX8zKA4Sd5xofAK5pqDNSNmKGY2zrW+9mRZD9D8DzcDwzzpZVvX8/GEoWxYz+gudb YMiHdl4O6tJ8BXdfeg X-Google-Smtp-Source: AGHT+IGy8MxpdWp4m0uBQuaQRyO24F8zlB5wbVQM6CpllFA9oHYHLIjDhnvY02Ou8OQQT6ij087Kyw== X-Received: by 2002:a05:600c:3b14:b0:475:dd9a:f786 with SMTP id 5b1f17b1804b1-47717e7ab64mr2060495e9.40.1761582242112; Mon, 27 Oct 2025 09:24:02 -0700 (PDT) Received: from localhost ([2a01:4b00:d036:ae00:6fc5:c3bc:147e:832c]) by smtp.gmail.com with UTF8SMTPSA id 5b1f17b1804b1-475dd47794asm162365865e9.1.2025.10.27.09.24.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Oct 2025 09:24:01 -0700 (PDT) From: luca.boccassi@gmail.com To: Radu Nicolau Cc: Kai Ji , dpdk stable Subject: patch 'crypto/qat: fix source buffer alignment' has been queued to stable release 22.11.11 Date: Mon, 27 Oct 2025 16:19:51 +0000 Message-ID: <20251027162001.3710450-73-luca.boccassi@gmail.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251027162001.3710450-1-luca.boccassi@gmail.com> References: <20251027162001.3710450-1-luca.boccassi@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org Hi, FYI, your patch has been queued to stable release 22.11.11 Note it hasn't been pushed to http://dpdk.org/browse/dpdk-stable yet. It will be pushed if I get no objections before 10/29/25. So please shout if anyone has objections. Also note that after the patch there's a diff of the upstream commit vs the patch applied to the branch. This will indicate if there was any rebasing needed to apply to the stable branch. If there were code changes for rebasing (ie: not only metadata diffs), please double check that the rebase was correctly done. Queued patches are on a temporary branch at: https://github.com/bluca/dpdk-stable This queued commit can be viewed at: https://github.com/bluca/dpdk-stable/commit/4c7cbd6bf29276e8f874343f8c262756e90c4f02 Thanks. Luca Boccassi --- >From 4c7cbd6bf29276e8f874343f8c262756e90c4f02 Mon Sep 17 00:00:00 2001 From: Radu Nicolau Date: Wed, 6 Aug 2025 14:48:32 +0000 Subject: [PATCH] crypto/qat: fix source buffer alignment [ upstream commit 253174309ff7abf9eaba58d1bccf90cca7e6d215 ] Fix performance regression resulting from using non cache-aligned source buffers when using cryptodev API. Fixes: fb3b9f492205 ("crypto/qat: rework burst data path") Signed-off-by: Radu Nicolau Acked-by: Kai Ji --- drivers/crypto/qat/dev/qat_crypto_pmd_gen3.c | 14 ++++++------ drivers/crypto/qat/dev/qat_crypto_pmd_gen4.c | 6 ++--- drivers/crypto/qat/dev/qat_crypto_pmd_gens.h | 21 ++++++++++++++++- drivers/crypto/qat/dev/qat_sym_pmd_gen1.c | 24 ++++++++++---------- 4 files changed, 42 insertions(+), 23 deletions(-) diff --git a/drivers/crypto/qat/dev/qat_crypto_pmd_gen3.c b/drivers/crypto/qat/dev/qat_crypto_pmd_gen3.c index 989caabf17..4a114a8a79 100644 --- a/drivers/crypto/qat/dev/qat_crypto_pmd_gen3.c +++ b/drivers/crypto/qat/dev/qat_crypto_pmd_gen3.c @@ -368,7 +368,7 @@ qat_sym_build_op_aead_gen3(void *in_op, struct qat_sym_session *ctx, } total_len = qat_sym_build_req_set_data(req, in_op, cookie, - in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num); + in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num, &ofs, op); if (unlikely(total_len < 0)) { op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; return -EINVAL; @@ -414,7 +414,7 @@ qat_sym_build_op_auth_gen3(void *in_op, struct qat_sym_session *ctx, } total_len = qat_sym_build_req_set_data(req, in_op, cookie, - in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num); + in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num, &ofs, op); if (unlikely(total_len < 0)) { op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; return -EINVAL; @@ -503,7 +503,7 @@ qat_sym_dp_enqueue_single_aead_gen3(void *qp_data, uint8_t *drv_ctx, rte_mov128((uint8_t *)req, (const uint8_t *)&(ctx->fw_req)); rte_prefetch0((uint8_t *)tx_queue->base_addr + tail); data_len = qat_sym_build_req_set_data(req, user_data, cookie, - data, n_data_vecs, NULL, 0); + data, n_data_vecs, NULL, 0, NULL, NULL); if (unlikely(data_len < 0)) return -1; @@ -555,12 +555,12 @@ qat_sym_dp_enqueue_aead_jobs_gen3(void *qp_data, uint8_t *drv_ctx, data_len = qat_sym_build_req_set_data(req, user_data[i], cookie, vec->src_sgl[i].vec, vec->src_sgl[i].num, - vec->dest_sgl[i].vec, vec->dest_sgl[i].num); + vec->dest_sgl[i].vec, vec->dest_sgl[i].num, NULL, NULL); } else { data_len = qat_sym_build_req_set_data(req, user_data[i], cookie, vec->src_sgl[i].vec, - vec->src_sgl[i].num, NULL, 0); + vec->src_sgl[i].num, NULL, 0, NULL, NULL); } if (unlikely(data_len < 0)) @@ -616,7 +616,7 @@ qat_sym_dp_enqueue_single_auth_gen3(void *qp_data, uint8_t *drv_ctx, rte_mov128((uint8_t *)req, (const uint8_t *)&(ctx->fw_req)); rte_prefetch0((uint8_t *)tx_queue->base_addr + tail); data_len = qat_sym_build_req_set_data(req, user_data, cookie, - data, n_data_vecs, NULL, 0); + data, n_data_vecs, NULL, 0, NULL, NULL); if (unlikely(data_len < 0)) return -1; @@ -679,7 +679,7 @@ qat_sym_dp_enqueue_auth_jobs_gen3(void *qp_data, uint8_t *drv_ctx, data_len = qat_sym_build_req_set_data(req, user_data[i], cookie, vec->src_sgl[i].vec, - vec->src_sgl[i].num, NULL, 0); + vec->src_sgl[i].num, NULL, 0, NULL, NULL); } if (unlikely(data_len < 0) || error) diff --git a/drivers/crypto/qat/dev/qat_crypto_pmd_gen4.c b/drivers/crypto/qat/dev/qat_crypto_pmd_gen4.c index 1ffc4528cf..54b3295647 100644 --- a/drivers/crypto/qat/dev/qat_crypto_pmd_gen4.c +++ b/drivers/crypto/qat/dev/qat_crypto_pmd_gen4.c @@ -207,7 +207,7 @@ qat_sym_build_op_aead_gen4(void *in_op, struct qat_sym_session *ctx, } total_len = qat_sym_build_req_set_data(qat_req, in_op, cookie, - in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num); + in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num, &ofs, op); if (unlikely(total_len < 0)) { op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; return -EINVAL; @@ -366,7 +366,7 @@ qat_sym_dp_enqueue_single_aead_gen4(void *qp_data, uint8_t *drv_ctx, rte_mov128((uint8_t *)req, (const uint8_t *)&(ctx->fw_req)); rte_prefetch0((uint8_t *)tx_queue->base_addr + tail); data_len = qat_sym_build_req_set_data(req, user_data, cookie, - data, n_data_vecs, NULL, 0); + data, n_data_vecs, NULL, 0, NULL, NULL); if (unlikely(data_len < 0)) return -1; @@ -426,7 +426,7 @@ qat_sym_dp_enqueue_aead_jobs_gen4(void *qp_data, uint8_t *drv_ctx, data_len = qat_sym_build_req_set_data(req, user_data[i], cookie, vec->src_sgl[i].vec, - vec->src_sgl[i].num, NULL, 0); + vec->src_sgl[i].num, NULL, 0, NULL, NULL); } if (unlikely(data_len < 0) || error) diff --git a/drivers/crypto/qat/dev/qat_crypto_pmd_gens.h b/drivers/crypto/qat/dev/qat_crypto_pmd_gens.h index 6f676a2c44..3201e1ead8 100644 --- a/drivers/crypto/qat/dev/qat_crypto_pmd_gens.h +++ b/drivers/crypto/qat/dev/qat_crypto_pmd_gens.h @@ -411,7 +411,8 @@ static __rte_always_inline int32_t qat_sym_build_req_set_data(struct icp_qat_fw_la_bulk_req *req, void *opaque, struct qat_sym_op_cookie *cookie, struct rte_crypto_vec *src_vec, uint16_t n_src, - struct rte_crypto_vec *dst_vec, uint16_t n_dst) + struct rte_crypto_vec *dst_vec, uint16_t n_dst, + union rte_crypto_sym_ofs *ofs, struct rte_crypto_op *op) { struct qat_sgl *list; uint32_t i; @@ -483,6 +484,24 @@ qat_sym_build_req_set_data(struct icp_qat_fw_la_bulk_req *req, dst_data_start = src_data_start; } + /* For crypto API only try to align the in-place buffers*/ + if (op != NULL && likely(n_dst == 0)) { + uint16_t offset = src_data_start & RTE_CACHE_LINE_MASK; + if (offset) { + rte_iova_t buff_addr = rte_mbuf_iova_get(op->sym->m_src); + /* make sure src_data_start is still within the buffer */ + if (src_data_start - offset >= buff_addr) { + src_data_start -= offset; + dst_data_start = src_data_start; + ofs->ofs.auth.head += offset; + ofs->ofs.cipher.head += offset; + tl_src += offset; + total_len_src = tl_src; + total_len_dst = tl_src; + } + } + } + req->comn_mid.src_data_addr = src_data_start; req->comn_mid.dest_data_addr = dst_data_start; req->comn_mid.src_length = total_len_src; diff --git a/drivers/crypto/qat/dev/qat_sym_pmd_gen1.c b/drivers/crypto/qat/dev/qat_sym_pmd_gen1.c index 1856770522..be50f5049f 100644 --- a/drivers/crypto/qat/dev/qat_sym_pmd_gen1.c +++ b/drivers/crypto/qat/dev/qat_sym_pmd_gen1.c @@ -236,7 +236,7 @@ qat_sym_build_op_cipher_gen1(void *in_op, struct qat_sym_session *ctx, } total_len = qat_sym_build_req_set_data(req, in_op, cookie, - in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num); + in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num, &ofs, op); if (unlikely(total_len < 0)) { op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; return -EINVAL; @@ -281,7 +281,7 @@ qat_sym_build_op_auth_gen1(void *in_op, struct qat_sym_session *ctx, } total_len = qat_sym_build_req_set_data(req, in_op, cookie, - in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num); + in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num, &ofs, op); if (unlikely(total_len < 0)) { op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; return -EINVAL; @@ -328,7 +328,7 @@ qat_sym_build_op_aead_gen1(void *in_op, struct qat_sym_session *ctx, } total_len = qat_sym_build_req_set_data(req, in_op, cookie, - in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num); + in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num, &ofs, op); if (unlikely(total_len < 0)) { op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; return -EINVAL; @@ -375,7 +375,7 @@ qat_sym_build_op_chain_gen1(void *in_op, struct qat_sym_session *ctx, } total_len = qat_sym_build_req_set_data(req, in_op, cookie, - in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num); + in_sgl.vec, in_sgl.num, out_sgl.vec, out_sgl.num, &ofs, op); if (unlikely(total_len < 0)) { op->status = RTE_CRYPTO_OP_STATUS_INVALID_ARGS; return -EINVAL; @@ -508,7 +508,7 @@ qat_sym_dp_enqueue_single_cipher_gen1(void *qp_data, uint8_t *drv_ctx, rte_prefetch0((uint8_t *)tx_queue->base_addr + tail); data_len = qat_sym_build_req_set_data(req, user_data, cookie, - data, n_data_vecs, NULL, 0); + data, n_data_vecs, NULL, 0, NULL, NULL); if (unlikely(data_len < 0)) return -1; @@ -569,7 +569,7 @@ qat_sym_dp_enqueue_cipher_jobs_gen1(void *qp_data, uint8_t *drv_ctx, data_len = qat_sym_build_req_set_data(req, user_data[i], cookie, vec->src_sgl[i].vec, - vec->src_sgl[i].num, NULL, 0); + vec->src_sgl[i].num, NULL, 0, NULL, NULL); } if (unlikely(data_len < 0 || error)) @@ -622,7 +622,7 @@ qat_sym_dp_enqueue_single_auth_gen1(void *qp_data, uint8_t *drv_ctx, rte_mov128((uint8_t *)req, (const uint8_t *)&(ctx->fw_req)); rte_prefetch0((uint8_t *)tx_queue->base_addr + tail); data_len = qat_sym_build_req_set_data(req, user_data, cookie, - data, n_data_vecs, NULL, 0); + data, n_data_vecs, NULL, 0, NULL, NULL); if (unlikely(data_len < 0)) return -1; @@ -690,7 +690,7 @@ qat_sym_dp_enqueue_auth_jobs_gen1(void *qp_data, uint8_t *drv_ctx, data_len = qat_sym_build_req_set_data(req, user_data[i], cookie, vec->src_sgl[i].vec, - vec->src_sgl[i].num, NULL, 0); + vec->src_sgl[i].num, NULL, 0, NULL, NULL); } if (unlikely(data_len < 0 || error)) @@ -749,7 +749,7 @@ qat_sym_dp_enqueue_single_chain_gen1(void *qp_data, uint8_t *drv_ctx, rte_mov128((uint8_t *)req, (const uint8_t *)&(ctx->fw_req)); rte_prefetch0((uint8_t *)tx_queue->base_addr + tail); data_len = qat_sym_build_req_set_data(req, user_data, cookie, - data, n_data_vecs, NULL, 0); + data, n_data_vecs, NULL, 0, NULL, NULL); if (unlikely(data_len < 0)) return -1; @@ -818,7 +818,7 @@ qat_sym_dp_enqueue_chain_jobs_gen1(void *qp_data, uint8_t *drv_ctx, data_len = qat_sym_build_req_set_data(req, user_data[i], cookie, vec->src_sgl[i].vec, - vec->src_sgl[i].num, NULL, 0); + vec->src_sgl[i].num, NULL, 0, NULL, NULL); } if (unlikely(data_len < 0 || error)) @@ -882,7 +882,7 @@ qat_sym_dp_enqueue_single_aead_gen1(void *qp_data, uint8_t *drv_ctx, rte_mov128((uint8_t *)req, (const uint8_t *)&(ctx->fw_req)); rte_prefetch0((uint8_t *)tx_queue->base_addr + tail); data_len = qat_sym_build_req_set_data(req, user_data, cookie, - data, n_data_vecs, NULL, 0); + data, n_data_vecs, NULL, 0, NULL, NULL); if (unlikely(data_len < 0)) return -1; @@ -942,7 +942,7 @@ qat_sym_dp_enqueue_aead_jobs_gen1(void *qp_data, uint8_t *drv_ctx, data_len = qat_sym_build_req_set_data(req, user_data[i], cookie, vec->src_sgl[i].vec, - vec->src_sgl[i].num, NULL, 0); + vec->src_sgl[i].num, NULL, 0, NULL, NULL); } if (unlikely(data_len < 0) || error) -- 2.47.3 --- Diff of the applied patch vs upstream commit (please double-check if non-empty: --- --- - 2025-10-27 15:54:37.440334027 +0000 +++ 0073-crypto-qat-fix-source-buffer-alignment.patch 2025-10-27 15:54:34.851950954 +0000 @@ -1 +1 @@ -From 253174309ff7abf9eaba58d1bccf90cca7e6d215 Mon Sep 17 00:00:00 2001 +From 4c7cbd6bf29276e8f874343f8c262756e90c4f02 Mon Sep 17 00:00:00 2001 @@ -5,0 +6,2 @@ +[ upstream commit 253174309ff7abf9eaba58d1bccf90cca7e6d215 ] + @@ -10 +11,0 @@ -Cc: stable@dpdk.org @@ -22 +23 @@ -index 0dcb5a7cb4..c196cf3cdb 100644 +index 989caabf17..4a114a8a79 100644 @@ -25 +26 @@ -@@ -422,7 +422,7 @@ qat_sym_build_op_aead_gen3(void *in_op, struct qat_sym_session *ctx, +@@ -368,7 +368,7 @@ qat_sym_build_op_aead_gen3(void *in_op, struct qat_sym_session *ctx, @@ -34 +35 @@ -@@ -466,7 +466,7 @@ qat_sym_build_op_auth_gen3(void *in_op, struct qat_sym_session *ctx, +@@ -414,7 +414,7 @@ qat_sym_build_op_auth_gen3(void *in_op, struct qat_sym_session *ctx, @@ -43 +44 @@ -@@ -564,7 +564,7 @@ qat_sym_dp_enqueue_single_aead_gen3(void *qp_data, uint8_t *drv_ctx, +@@ -503,7 +503,7 @@ qat_sym_dp_enqueue_single_aead_gen3(void *qp_data, uint8_t *drv_ctx, @@ -52 +53,7 @@ -@@ -623,7 +623,7 @@ qat_sym_dp_enqueue_aead_jobs_gen3(void *qp_data, uint8_t *drv_ctx, +@@ -555,12 +555,12 @@ qat_sym_dp_enqueue_aead_jobs_gen3(void *qp_data, uint8_t *drv_ctx, + data_len = qat_sym_build_req_set_data(req, + user_data[i], cookie, + vec->src_sgl[i].vec, vec->src_sgl[i].num, +- vec->dest_sgl[i].vec, vec->dest_sgl[i].num); ++ vec->dest_sgl[i].vec, vec->dest_sgl[i].num, NULL, NULL); + } else { @@ -60,2 +67,2 @@ - if (unlikely(data_len < 0) || error) -@@ -677,7 +677,7 @@ qat_sym_dp_enqueue_single_auth_gen3(void *qp_data, uint8_t *drv_ctx, + if (unlikely(data_len < 0)) +@@ -616,7 +616,7 @@ qat_sym_dp_enqueue_single_auth_gen3(void *qp_data, uint8_t *drv_ctx, @@ -70,7 +77 @@ -@@ -732,12 +732,12 @@ qat_sym_dp_enqueue_auth_jobs_gen3(void *qp_data, uint8_t *drv_ctx, - data_len = qat_sym_build_req_set_data(req, - user_data[i], cookie, - vec->src_sgl[i].vec, vec->src_sgl[i].num, -- vec->dest_sgl[i].vec, vec->dest_sgl[i].num); -+ vec->dest_sgl[i].vec, vec->dest_sgl[i].num, NULL, NULL); - } else { +@@ -679,7 +679,7 @@ qat_sym_dp_enqueue_auth_jobs_gen3(void *qp_data, uint8_t *drv_ctx, @@ -84 +85 @@ - if (unlikely(data_len < 0)) + if (unlikely(data_len < 0) || error) @@ -86 +87 @@ -index 843580af72..82c5a40501 100644 +index 1ffc4528cf..54b3295647 100644 @@ -89 +90 @@ -@@ -289,7 +289,7 @@ qat_sym_build_op_aead_gen4(void *in_op, struct qat_sym_session *ctx, +@@ -207,7 +207,7 @@ qat_sym_build_op_aead_gen4(void *in_op, struct qat_sym_session *ctx, @@ -98 +99 @@ -@@ -446,7 +446,7 @@ qat_sym_dp_enqueue_single_aead_gen4(void *qp_data, uint8_t *drv_ctx, +@@ -366,7 +366,7 @@ qat_sym_dp_enqueue_single_aead_gen4(void *qp_data, uint8_t *drv_ctx, @@ -107 +108 @@ -@@ -505,7 +505,7 @@ qat_sym_dp_enqueue_aead_jobs_gen4(void *qp_data, uint8_t *drv_ctx, +@@ -426,7 +426,7 @@ qat_sym_dp_enqueue_aead_jobs_gen4(void *qp_data, uint8_t *drv_ctx, @@ -117 +118 @@ -index 1f19c69f88..67dc889b50 100644 +index 6f676a2c44..3201e1ead8 100644 @@ -120 +121 @@ -@@ -430,7 +430,8 @@ static __rte_always_inline int32_t +@@ -411,7 +411,8 @@ static __rte_always_inline int32_t @@ -130 +131 @@ -@@ -502,6 +503,24 @@ qat_sym_build_req_set_data(struct icp_qat_fw_la_bulk_req *req, +@@ -483,6 +484,24 @@ qat_sym_build_req_set_data(struct icp_qat_fw_la_bulk_req *req, @@ -156 +157 @@ -index 8cb85fd8df..6da0f6c645 100644 +index 1856770522..be50f5049f 100644 @@ -159 +160 @@ -@@ -242,7 +242,7 @@ qat_sym_build_op_cipher_gen1(void *in_op, struct qat_sym_session *ctx, +@@ -236,7 +236,7 @@ qat_sym_build_op_cipher_gen1(void *in_op, struct qat_sym_session *ctx, @@ -168,2 +169,2 @@ -@@ -294,7 +294,7 @@ qat_sym_build_op_auth_gen1(void *in_op, struct qat_sym_session *ctx, - req->comn_hdr.serv_specif_flags, 0); +@@ -281,7 +281,7 @@ qat_sym_build_op_auth_gen1(void *in_op, struct qat_sym_session *ctx, + } @@ -177 +178 @@ -@@ -339,7 +339,7 @@ qat_sym_build_op_aead_gen1(void *in_op, struct qat_sym_session *ctx, +@@ -328,7 +328,7 @@ qat_sym_build_op_aead_gen1(void *in_op, struct qat_sym_session *ctx, @@ -186 +187 @@ -@@ -384,7 +384,7 @@ qat_sym_build_op_chain_gen1(void *in_op, struct qat_sym_session *ctx, +@@ -375,7 +375,7 @@ qat_sym_build_op_chain_gen1(void *in_op, struct qat_sym_session *ctx, @@ -195 +196 @@ -@@ -512,7 +512,7 @@ qat_sym_dp_enqueue_single_cipher_gen1(void *qp_data, uint8_t *drv_ctx, +@@ -508,7 +508,7 @@ qat_sym_dp_enqueue_single_cipher_gen1(void *qp_data, uint8_t *drv_ctx, @@ -204 +205 @@ -@@ -571,7 +571,7 @@ qat_sym_dp_enqueue_cipher_jobs_gen1(void *qp_data, uint8_t *drv_ctx, +@@ -569,7 +569,7 @@ qat_sym_dp_enqueue_cipher_jobs_gen1(void *qp_data, uint8_t *drv_ctx, @@ -213 +214 @@ -@@ -623,7 +623,7 @@ qat_sym_dp_enqueue_single_auth_gen1(void *qp_data, uint8_t *drv_ctx, +@@ -622,7 +622,7 @@ qat_sym_dp_enqueue_single_auth_gen1(void *qp_data, uint8_t *drv_ctx, @@ -231 +232 @@ -@@ -747,7 +747,7 @@ qat_sym_dp_enqueue_single_chain_gen1(void *qp_data, uint8_t *drv_ctx, +@@ -749,7 +749,7 @@ qat_sym_dp_enqueue_single_chain_gen1(void *qp_data, uint8_t *drv_ctx, @@ -240 +241 @@ -@@ -815,7 +815,7 @@ qat_sym_dp_enqueue_chain_jobs_gen1(void *qp_data, uint8_t *drv_ctx, +@@ -818,7 +818,7 @@ qat_sym_dp_enqueue_chain_jobs_gen1(void *qp_data, uint8_t *drv_ctx, @@ -249 +250 @@ -@@ -877,7 +877,7 @@ qat_sym_dp_enqueue_single_aead_gen1(void *qp_data, uint8_t *drv_ctx, +@@ -882,7 +882,7 @@ qat_sym_dp_enqueue_single_aead_gen1(void *qp_data, uint8_t *drv_ctx, @@ -258 +259 @@ -@@ -936,7 +936,7 @@ qat_sym_dp_enqueue_aead_jobs_gen1(void *qp_data, uint8_t *drv_ctx, +@@ -942,7 +942,7 @@ qat_sym_dp_enqueue_aead_jobs_gen1(void *qp_data, uint8_t *drv_ctx,