From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id A5B3DA04F1; Thu, 18 Jun 2020 19:50:41 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6A9611BFE3; Thu, 18 Jun 2020 19:50:40 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id AD80D1BFD1 for ; Thu, 18 Jun 2020 19:50:38 +0200 (CEST) IronPort-SDR: rAwJV9yltmulG7hTzrnlGqHY0h2zP5aznqbM+inG/Pz7+THVoNgwXWLDM2PLsv4RyiAfvPT62k rNf53Pw3TD1Q== X-IronPort-AV: E=McAfee;i="6000,8403,9656"; a="141745188" X-IronPort-AV: E=Sophos;i="5.75,252,1589266800"; d="scan'208";a="141745188" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Jun 2020 10:50:37 -0700 IronPort-SDR: LXaVLJNrwDJ7o2/DGo8qJtoT933CLgia5NU1oU5gmxekkDodR7JApK4ouwFQh2YRPLJjy52bvb hMCHjW6tEhkA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.75,252,1589266800"; d="scan'208";a="291849459" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga002.jf.intel.com with ESMTP; 18 Jun 2020 10:50:36 -0700 Received: from fmsmsx123.amr.corp.intel.com (10.18.125.38) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 18 Jun 2020 10:50:36 -0700 Received: from FMSEDG001.ED.cps.intel.com (10.1.192.133) by fmsmsx123.amr.corp.intel.com (10.18.125.38) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 18 Jun 2020 10:50:36 -0700 Received: from NAM04-CO1-obe.outbound.protection.outlook.com (104.47.45.53) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (TLS) id 14.3.439.0; Thu, 18 Jun 2020 10:50:36 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=CSwA2Q1ER4MuU7FnvV9dx7UTK114LV/RpIYUm/zLJm+2cHj7xcf+Pagre/wVIAyJulBeq5dSvE70e888MqPKI6u4xPX1RKXdBSV6wz77rkv1O7WKUkv6MLsOu3RKpfxn3CM0Pxl0TC7b2IZp37kyvr2Qkl3VuymlLIs8fxN/wK4tJCCMHMPy0Iw40yb9koNmOCEDO0nxlffsPEmTPuGyvNftx2UD5oDcmj5Yd0GEGnT+2+pjSTbZKJrvob9JcZmt4ifV+Clu8cnBEqiBWnyu+7YmzzKHPTa1qvsgCSpv2e9upA/eFUR8dUAwqBCBivTHOLe1hNl1mk85hy/hnN7Lpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=M1J+p8vwhxIpfL+U3bjq8OQgOv0HR+orqzrnxAdALhU=; b=k3BmOqeRFx2RzjLl31N/3GT7eCzYusq+TNxQ3SKeIaBIuXfr7JG5GVXFpxwvO507Sj43yFhpeggfwJPsjNFyWfYlJ6b/f1G5c1eS4uf4brTB5o2wt3ccKhdKv0QiVTCDkZvVXgRamp0E4QQ5EitXBeqpovNhUi1iCnhzHrLKs7M/rPE8M+nAfwZoWSqx+MpMglZ/iKhx/3iJm9WFXFM2TWRMWQvFnCJG1OI72kV+sHRLQutKaGnBFZ8ZzinP0fLzr/GvrBDB5TLApFu/zYk/9fTyeMx0AkoALpTGUguvMU+zfXz5YNkwzR/Pi9Ybw1zv9YzlA1OqY9z4pAyujDiFMQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=M1J+p8vwhxIpfL+U3bjq8OQgOv0HR+orqzrnxAdALhU=; b=EixmHxmyfKca21FFkWV7SScnvFJvQEJH/53AlYO6dYGnDGsa72mgJKZT9x62/2WsDYTX9jxYsWJ70fdjaZ1fp5IFM8vbSyv82LfDPBOf4aBiactuHyMAaWY7cMyolEWppwhnYSGkWunDHVwrYl3JQLMWXI9EXp0V1skN1WKV5I0= Received: from SN6PR11MB2880.namprd11.prod.outlook.com (2603:10b6:805:58::15) by SN6PR11MB3102.namprd11.prod.outlook.com (2603:10b6:805:d1::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Thu, 18 Jun 2020 17:50:34 +0000 Received: from SN6PR11MB2880.namprd11.prod.outlook.com ([fe80::c4ff:6258:78f7:ff90]) by SN6PR11MB2880.namprd11.prod.outlook.com ([fe80::c4ff:6258:78f7:ff90%5]) with mapi id 15.20.3109.021; Thu, 18 Jun 2020 17:50:34 +0000 From: "Trahe, Fiona" To: "Zhang, Roy Fan" , "dev@dpdk.org" CC: "akhil.goyal@nxp.com" , "Bronowski, PiotrX" , "Trahe, Fiona" Thread-Topic: [PATCH] crypto/qat: add data-path APIs Thread-Index: AQHWQMdcFnWbGX/2WUKKhZ7Vwkqf76jeooIg Date: Thu, 18 Jun 2020 17:50:34 +0000 Message-ID: References: <20200612143940.52851-1-roy.fan.zhang@intel.com> In-Reply-To: <20200612143940.52851-1-roy.fan.zhang@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.2.0.6 authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=intel.com; x-originating-ip: [192.198.151.182] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: de0fcb6a-c49e-4e35-d802-08d813b01a3f x-ms-traffictypediagnostic: SN6PR11MB3102: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4941; x-forefront-prvs: 0438F90F17 x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YAvg76t8rXYUp/qU3LtDLSp28DYX1xCqtO4udBcZ5uWh7Hlh5uFGsbBwrh4nFpv+Iyio3bKW29MQyEfjCUY3jQhOPvFlraahpFX0NSiJjKXMfIKYscY7PzqpleDtQWjuKN8jAmfBc/O1XvGrcSBoC8cWszauCJKgj86Hkc2s3TNChgboHIaTNNhSMFKvPK4DlwJXWs2+NowFRD33UgiweJbijoJ9QAT2oggS8FUKVzqOYrtiKbGeGJmB2Eo+/yTc61Itk90qH4NpQk3bWTRV2H6MFB60myO2ZRMZH1fosWtAUaYGprVfrXJnOCcRfxzCOQzx05BJ5rLvYOg+w5c/r9uUceLIWzBrUyCNjtje1MSKEY7capreODuWaIpVdOt+pg0MubWOFQ67ciAzMgdy1A== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR11MB2880.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(346002)(376002)(39860400002)(396003)(366004)(2906002)(86362001)(30864003)(66446008)(478600001)(66946007)(66556008)(316002)(26005)(186003)(66476007)(64756008)(54906003)(110136005)(76116006)(71200400001)(9686003)(966005)(55016002)(8936002)(107886003)(7696005)(8676002)(33656002)(52536014)(53546011)(5660300002)(83380400001)(4326008)(6506007)(579004); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: JRQngo++qWufvZqDkoFvnQkrw1BKt0mImX2WqfSanRKY/TqZ3eOlTEHi42FuyAKDJXftC+7kqWW7R91btFWfz7m1Brz++WNQtVTxKwLoS6nJboSJmd1fcaDdHUGSme1pJFK3c5/FpkZS277DCR5Z0IXVqyrsXJr7tVZpLeYAol/VQtfvgKHgYRVlFnQpaI2jWpfTTkx/zipVyQqz2PNgG4aDilv03VW57bbigkKJC2LDtQr4W3XadHkBDMfk+qfsyEfUOZx4TPP83KBco8B8Gx85WciXGGqU2+z5MjAMr0/ID8NVxQVgrZEbowVM6u4E2lZ3vvW/pY9beKlqCe0deMjXrMl5VwqdpTpa/6YCrjaM+wG21+l2qlqWYMNvkr1BRo3J9UoJQ2YLsJ2Cx9IkycVzrinJJyNPYEJW8rYsnpEhsjtyPNAjT4PZids7gYPI6hyeN2rb5EOcyxGiudwGhzyWKl2hOQIFrexaOzgVNqUjLRU4FX4K20RBSRXQin9D Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: de0fcb6a-c49e-4e35-d802-08d813b01a3f X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Jun 2020 17:50:34.6959 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: OwXUZc70kKzI7H7uvzi7imy7PW/wqB6y2mxjK45rt4XYjYZeh2J+caC+pmKoukufdUbEtpW3rLliiDrw3jZFwA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR11MB3102 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] crypto/qat: add data-path APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Zhang, Roy Fan > Sent: Friday, June 12, 2020 3:40 PM > To: dev@dpdk.org > Cc: akhil.goyal@nxp.com; Trahe, Fiona ; Zhang, Roy= Fan > ; Bronowski, PiotrX > Subject: [PATCH] crypto/qat: add data-path APIs >=20 > This patch adds data-path APIs to QAT symmetric dirver to support > raw data as input. >=20 > For applications/libraries that want to benefit from the data-path > encryption acceleration provided by QAT but not necessarily depends > on DPDK data-path structures (such as VPP), some performance > degradation is unavoidable to convert between their specific data > structure and DPDK cryptodev operation as well as mbufs. >=20 > This patch takes advantage of existing QAT implementations to form > symmetric data-path enqueue and dequeue APIs that support raw data > as input so that they can have wider usability towards those > applications/libraries without performance drop caused by the data > structure conversions. In the meantime the less performance-sensitive > cryptodev device and session management remains intact so that DPDK > cryptodev remains to be unified control path library for QAT. >=20 > Signed-off-by: Fan Zhang > Signed-off-by: Piotr Bronowski > --- > drivers/common/qat/Makefile | 4 +- > drivers/common/qat/qat_qp.c | 4 +- > drivers/common/qat/qat_qp.h | 3 + > drivers/compress/qat/rte_pmd_qat_version.map | 11 + > drivers/crypto/qat/meson.build | 5 + > drivers/crypto/qat/qat_sym_frame.c | 294 +++++++++++++++++++ > drivers/crypto/qat/qat_sym_frame.h | 237 +++++++++++++++ > 7 files changed, 555 insertions(+), 3 deletions(-) > create mode 100644 drivers/crypto/qat/qat_sym_frame.c > create mode 100644 drivers/crypto/qat/qat_sym_frame.h >=20 > diff --git a/drivers/common/qat/Makefile b/drivers/common/qat/Makefile > index 28bd5668f..3874f75ab 100644 > --- a/drivers/common/qat/Makefile > +++ b/drivers/common/qat/Makefile > @@ -39,6 +39,8 @@ ifeq ($(CONFIG_RTE_LIBRTE_PMD_QAT_SYM),y) > SRCS-y +=3D qat_sym.c > SRCS-y +=3D qat_sym_session.c > SRCS-y +=3D qat_sym_pmd.c > + SRCS-y +=3D qat_sym_frame.c > + > build_qat =3D yes > endif > endif > @@ -62,7 +64,7 @@ ifdef build_qat > LDLIBS +=3D -lrte_pci -lrte_bus_pci >=20 > # export include files > - SYMLINK-y-include +=3D > + SYMLINK-y-include +=3D qat_sym_frame.h >=20 > # versioning export map > EXPORT_MAP :=3D ../../compress/qat/rte_pmd_qat_version.map > diff --git a/drivers/common/qat/qat_qp.c b/drivers/common/qat/qat_qp.c > index 8e6dd04eb..06e2d8c8a 100644 > --- a/drivers/common/qat/qat_qp.c > +++ b/drivers/common/qat/qat_qp.c > @@ -547,8 +547,8 @@ txq_write_tail(struct qat_qp *qp, struct qat_queue *q= ) { > q->csr_tail =3D q->tail; > } >=20 > -static inline > -void rxq_free_desc(struct qat_qp *qp, struct qat_queue *q) > +void > +rxq_free_desc(struct qat_qp *qp, struct qat_queue *q) > { > uint32_t old_head, new_head; > uint32_t max_head; > diff --git a/drivers/common/qat/qat_qp.h b/drivers/common/qat/qat_qp.h > index 575d69059..8add1b049 100644 > --- a/drivers/common/qat/qat_qp.h > +++ b/drivers/common/qat/qat_qp.h > @@ -116,4 +116,7 @@ qat_comp_process_response(void **op __rte_unused, uin= t8_t *resp > __rte_unused, > void *op_cookie __rte_unused, > uint64_t *dequeue_err_count __rte_unused); >=20 > +void > +rxq_free_desc(struct qat_qp *qp, struct qat_queue *q); > + > #endif /* _QAT_QP_H_ */ > diff --git a/drivers/compress/qat/rte_pmd_qat_version.map > b/drivers/compress/qat/rte_pmd_qat_version.map > index f9f17e4f6..a9160b157 100644 > --- a/drivers/compress/qat/rte_pmd_qat_version.map > +++ b/drivers/compress/qat/rte_pmd_qat_version.map > @@ -1,3 +1,14 @@ > DPDK_20.0 { > local: *; > }; > + > +EXPERIMENTAL { > + global: > + > + qat_sym_get_qp; > + qat_sym_enqueue_frame_aead; > + qat_sym_enqueue_frame_cipher; > + qat_sym_enqueue_frame_auth; > + qat_sym_enqueue_frame_chain; > + qat_sym_dequeue_frame; > +}; > diff --git a/drivers/crypto/qat/meson.build b/drivers/crypto/qat/meson.bu= ild > index fc65923a7..8d53debcf 100644 > --- a/drivers/crypto/qat/meson.build > +++ b/drivers/crypto/qat/meson.build > @@ -13,9 +13,14 @@ if dep.found() > qat_sources +=3D files('qat_sym_pmd.c', > 'qat_sym.c', > 'qat_sym_session.c', > + 'qat_sym_frame.c', > 'qat_asym_pmd.c', > 'qat_asym.c') > qat_ext_deps +=3D dep > qat_cflags +=3D '-DBUILD_QAT_SYM' > qat_cflags +=3D '-DBUILD_QAT_ASYM' > + headers =3D files( > + 'qat_sym_frame.h', > + ) > + use_function_versioning =3D true > endif > diff --git a/drivers/crypto/qat/qat_sym_frame.c b/drivers/crypto/qat/qat_= sym_frame.c > new file mode 100644 > index 000000000..27656c970 > --- /dev/null > +++ b/drivers/crypto/qat/qat_sym_frame.c > @@ -0,0 +1,294 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2015-2019 Intel Corporation > + */ > + > +#include > + > +#include "adf_transport_access_macros.h" > +#include "icp_qat_fw.h" > +#include "icp_qat_fw_la.h" > + > +#include "qat_sym_pmd.h" > +#include "qat_sym_session.h" > +#include "qat_sym_frame.h" > +#include "qat_qp.h" > + > +void * > +qat_sym_get_qp(uint8_t dev_id, uint16_t qp_id) > +{ > + struct rte_cryptodev *dev; > + struct qat_qp *qp; > + const char *drv_name; > + > + /* make sure it is a QAT device */ > + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) > + return NULL; [Fiona] I'd suggest using the new API just pushed here: https://patches.dpdk.org/patch/71211/ Then you can leave out the above check and the nb_queue_pairs check below. > + dev =3D rte_cryptodev_pmd_get_dev(dev_id); > + drv_name =3D rte_cryptodev_driver_name_get(dev->driver_id); > + if ((strncmp(drv_name, RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD), > + sizeof(RTE_STR(CRYPTODEV_NAME_QAT_SYM_PMD))) !=3D 0) || > + (qp_id > dev->data->nb_queue_pairs)) > + return NULL; > + > + qp =3D dev->data->queue_pairs[qp_id]; [Fiona] a null check would be useful before dereferencing qp below, but if you use the fn above then no need. > + if (qp->service_type !=3D QAT_SERVICE_SYMMETRIC) > + return NULL; > + > + return (void *)qp; > +} > + [Fiona] Add function hdr with limitations/assumptions, e.g. assumes in-place, limit on how big the frame can be, relative to queue size, the frame APis must be used for enqueue /dequeu and can't be mixed with the other enqueue/dequeue APIs. Also explanation of frame param, etc > +int > +qat_sym_enqueue_frame_aead(void *qat_sym_qp, > + struct rte_cryptodev_sym_session *session, > + rte_iova_t data_iova, uint32_t cipher_ofs, uint32_t cipher_len, > + struct rte_crypto_vec *sgl, uint32_t n_sgl_vecs, [Fiona] could pass in an rte_crypto_sgl instead of both of these. > + uint8_t *iv, rte_iova_t tag_iova, rte_iova_t aad_iova, > + uint8_t is_first, uint8_t is_last, void *frame) > +{ > + struct qat_qp *qp =3D qat_sym_qp; > + struct qat_queue *tx_queue =3D &qp->tx_q; > + register uint8_t *msg =3D (uint8_t *)tx_queue->base_addr + > + tx_queue->tail; > + struct qat_sym_session *ctx; > + struct icp_qat_fw_la_bulk_req *req =3D > + (struct icp_qat_fw_la_bulk_req *)msg; > + struct icp_qat_fw_la_cipher_req_params *cipher_param; > + struct icp_qat_fw_la_auth_req_params *auth_param; > + > + ctx =3D (struct qat_sym_session *)get_sym_session_private_data( > + session, cryptodev_qat_driver_id); > + rte_mov128(msg, (const uint8_t *)&(ctx->fw_req)); > + > + cipher_param =3D (void *)&req->serv_specif_rqpars; > + auth_param =3D (void *)((uint8_t *)cipher_param + > + ICP_QAT_FW_HASH_REQUEST_PARAMETERS_OFFSET); > + req->comn_mid.src_data_addr =3D req->comn_mid.dest_data_addr =3D data_i= ova; > + req->comn_mid.src_length =3D req->comn_mid.dst_length =3D cipher_ofs + > + cipher_len; > + > + /* TODO: add support to non-gcm algorithms */ > + if (ctx->qat_hash_alg !=3D ICP_QAT_HW_AUTH_ALGO_GALOIS_128 && > + ctx->qat_hash_alg !=3D ICP_QAT_HW_AUTH_ALGO_GALOIS_64) > + return -1; > + > + /* since we know it is GCM, iv has to be 12 bytes */ > + ICP_QAT_FW_LA_GCM_IV_LEN_FLAG_SET( > + req->comn_hdr.serv_specif_flags, > + ICP_QAT_FW_LA_GCM_IV_LEN_12_OCTETS); > + > + if (unlikely(is_first !=3D 0)) > + req->comn_mid.opaque_data =3D (uintptr_t)frame; > + > + rte_memcpy_generic(cipher_param->u.cipher_IV_array, iv, > + ctx->auth_iv.length); > + > + if (ctx->is_single_pass) { > + cipher_param->spc_aad_addr =3D aad_iova; > + cipher_param->spc_auth_res_addr =3D tag_iova; > + } > + > + if (sgl) { > + if (!n_sgl_vecs) > + return -1; > + /* TODO: sgl process */ > + } else { > + cipher_param->cipher_offset =3D cipher_ofs; > + cipher_param->cipher_length =3D cipher_len; > + auth_param->auth_off =3D cipher_ofs; > + auth_param->auth_len =3D cipher_len; > + auth_param->u1.aad_adr =3D aad_iova; > + auth_param->auth_res_addr =3D tag_iova; > + } > + > + tx_queue->tail =3D (tx_queue->tail + tx_queue->msg_size) & > + tx_queue->modulo_mask; > + > + if (unlikely(is_last !=3D 0)) { > + qp->enqueued++; > + qp->stats.enqueued_count++; [Fiona] This needs to increment by the num descriptors sent. Frame size. Either need to keep track of this or calculate it here based on tail and or= ig tail Should make sure the ring is not filled and overwrapped is is_last never se= t. But avoid an extra check on data path - and at least document it in fn hdr. > + WRITE_CSR_RING_TAIL(qp->mmap_bar_addr, > + tx_queue->hw_bundle_number, > + tx_queue->hw_queue_number, > + tx_queue->tail); > + tx_queue->csr_tail =3D tx_queue->tail; > + } > + > + return 0; > +} > + > +int > +qat_sym_enqueue_frame_chain(__rte_unused void *qat_sym_qp, > + __rte_unused struct rte_cryptodev_sym_session *session, > + __rte_unused rte_iova_t data_iova, > + __rte_unused uint32_t cipher_ofs, > + __rte_unused uint32_t cipher_len, > + __rte_unused uint32_t auth_ofs, > + __rte_unused uint32_t auth_len, > + __rte_unused struct rte_crypto_vec *sgl, > + __rte_unused uint32_t n_sgl_vecs, > + __rte_unused uint8_t *iv, __rte_unused rte_iova_t digest_iova, > + __rte_unused uint8_t is_first, > + __rte_unused uint8_t is_last, __rte_unused void *frame) > +{ > + /* TODO: implement the body */ > + return 0; > +} > + > +int > +qat_sym_enqueue_frame_cipher(__rte_unused void *qat_sym_qp, > + __rte_unused struct rte_cryptodev_sym_session *session, > + __rte_unused rte_iova_t data_iova, > + __rte_unused uint32_t cipher_ofs, > + __rte_unused uint32_t cipher_len, > + __rte_unused struct rte_crypto_vec *sgl, > + __rte_unused uint32_t n_sgl_vecs, > + __rte_unused uint8_t *iv, > + __rte_unused uint8_t is_first, > + __rte_unused uint8_t is_last, __rte_unused void *frame) > +{ > + /* TODO: implement the body */ > + return 0; > +} > + > +int > +qat_sym_enqueue_frame_auth(__rte_unused void *qat_sym_qp, > + __rte_unused struct rte_cryptodev_sym_session *session, > + __rte_unused rte_iova_t data_iova, > + __rte_unused uint32_t auth_ofs, > + __rte_unused uint32_t auth_len, > + __rte_unused struct rte_crypto_vec *sgl, > + __rte_unused uint32_t n_sgl_vecs, > + __rte_unused uint8_t *iv, __rte_unused rte_iova_t digest_iova, > + __rte_unused uint8_t is_first, > + __rte_unused uint8_t is_last, __rte_unused void *frame) > +{ > + /* TODO: implement the body */ > + return 0; > +} > + > +#define get_rx_queue_message_at_index(q, h, i) \ > + (void *)((uint8_t *)q->base_addr + ((h + q->msg_size * (i)) & \ > + q->modulo_mask)) > + > +static __rte_always_inline int > +qat_is_rx_msg_ok(struct icp_qat_fw_comn_resp *resp_msg) > +{ > + return ICP_QAT_FW_COMN_STATUS_FLAG_OK =3D=3D > + ICP_QAT_FW_COMN_RESP_CRYPTO_STAT_GET( > + resp_msg->comn_hdr.comn_status); > +} > + > +int > +qat_sym_dequeue_frame(void *qat_sym_qp, void **frame, > + qat_qp_get_frame_n_element_t get_frame_n_elt, > + uint32_t first_status_offset, uint32_t element_interval, > + uint8_t element_status_success, uint8_t element_status_error) > +{ > + struct qat_qp *qp =3D qat_sym_qp; > + struct qat_queue *rx_queue =3D &qp->rx_q; > + struct icp_qat_fw_comn_resp *resp, *resp1, *resp2, *resp3; > + void *f =3D NULL; > + uint32_t n_elts, i; > + uint8_t *status, *status1, *status2, *status3; > + int n_fail =3D 0, n_fail1 =3D 0, n_fail2 =3D 0, n_fail3 =3D 0; > + uint32_t head =3D rx_queue->head; > + > + resp =3D (struct icp_qat_fw_comn_resp *)( > + (uint8_t *)rx_queue->base_addr + head); > + > + /* if message is not processed, return 0 */ > + if (*(uint32_t *)resp =3D=3D ADF_RING_EMPTY_SIG) { > + *frame =3D NULL; > + return -1; > + } > + > + f =3D (void *)(uintptr_t)resp->opaque_data; > + if (unlikely(f =3D=3D NULL)) { > + *frame =3D NULL; > + return -1; > + } > + > + *frame =3D f; > + status =3D (uint8_t *)f + first_status_offset; > + > + n_elts =3D (*get_frame_n_elt)(f); > + if (unlikely(n_elts =3D=3D 0)) > + return -1; [Fiona] I'd move this check up before the previous 2 lines.=20 But is there ever a case of 0. Surely this can be caught on the enqueue and= no check needed here? > + > + /* process the first message */ > + if (qat_is_rx_msg_ok(resp)) > + *status =3D element_status_success; > + else { > + *status =3D element_status_error; > + n_fail--; > + } > + [Fiona] isn't there a step missing? Need to make sure the whole frame is av= ailable before processing any reponses. > + status +=3D element_interval; > + > + /* fetch 4 messages in a loop */ > + for (i =3D 1; i < n_elts - 4; i +=3D 4) { > + resp =3D get_rx_queue_message_at_index(rx_queue, head, 0); > + resp1 =3D get_rx_queue_message_at_index(rx_queue, head, 1); > + resp2 =3D get_rx_queue_message_at_index(rx_queue, head, 2); > + resp3 =3D get_rx_queue_message_at_index(rx_queue, head, 3); > + > + status1 =3D status + element_interval; > + status2 =3D status + element_interval * 2; > + status3 =3D status + element_interval * 3; > + > + if (qat_is_rx_msg_ok(resp)) > + *status =3D element_status_success; > + else { > + *status =3D element_status_error; > + n_fail--; > + } > + > + if (qat_is_rx_msg_ok(resp1)) > + *status1 =3D element_status_success; > + else { > + *status1 =3D element_status_error; > + n_fail1--; > + } > + > + if (qat_is_rx_msg_ok(resp2)) > + *status2 =3D element_status_success; > + else { > + *status2 =3D element_status_error; > + n_fail2--; > + } > + > + if (qat_is_rx_msg_ok(resp3)) > + *status3 =3D element_status_success; > + else { > + *status3 =3D element_status_error; > + n_fail3--; > + } > + > + i +=3D 4; > + status =3D status3 + element_interval; > + head =3D (head + rx_queue->msg_size * 4) & rx_queue->modulo_mask; > + } > + > + for (; i < n_elts; i++) { > + resp =3D get_rx_queue_message_at_index(rx_queue, head, 0); > + if (qat_is_rx_msg_ok(resp)) > + *status =3D element_status_success; > + else { > + *status =3D element_status_error; > + n_fail--; > + } > + status +=3D element_interval; > + head =3D (head + rx_queue->msg_size * 4) & rx_queue->modulo_mask; > + } > + > + /* update queue pair head */ > + rx_queue->head =3D (rx_queue->head + i * rx_queue->msg_size) & > + rx_queue->modulo_mask; [Fiona] is this necessary? can head be different than the local var? It may be on the other path due to the head coalescing. But think you can leave that feature out here. > + rx_queue->nb_processed_responses +=3D i; > + qp->dequeued +=3D i; > + qp->stats.dequeued_count +=3D i; > + if (rx_queue->nb_processed_responses > QAT_CSR_HEAD_WRITE_THRESH) [Fiona] No if needed here. Shouldn't be necessary in frame case. > + rxq_free_desc(qp, rx_queue); > + > + return n_fail + n_fail1 + n_fail2 + n_fail3; > +} > diff --git a/drivers/crypto/qat/qat_sym_frame.h b/drivers/crypto/qat/qat_= sym_frame.h > new file mode 100644 > index 000000000..e378cacb8 > --- /dev/null > +++ b/drivers/crypto/qat/qat_sym_frame.h > @@ -0,0 +1,237 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2015-2018 Intel Corporation > + */ > + > +#ifndef _QAT_SYM_FRAME_H_ > +#define _QAT_SYM_FRAME_H_ > + > +#include > + > +/** > + * Get the QAT queue pair based on device id and queue pair id. > + * Checks if passed arguments are valid. > + * > + * @param dev_id > + * cryptodev device id. > + * @param qp_id > + * queue pair id > + * @return > + * pointer to queue pair if passed parameters are valid. > + * NULL pointer otherwise. > + **/ > +__rte_experimental > +void * > +qat_sym_get_qp(uint8_t devi_id, uint16_t qp_id); > + > +/** > + * enqueue one AEAD operation into QAT queue > + * > + * @param qat_sym_qp > + * queue pair data got from qat_sym_get_qp(). > + * @param session > + * configured cryptodev symmetric session data. > + * @param data_iova > + * iova address of data. > + * @param cipher_ofs > + * cipher offset start from data_iova. > + * @param cipher_len > + * cipher total length. > + * @param sgl > + * in case of SGL data, pointer to an array of sgl structure. > + * @param n_sgl_vecs > + * number of SGL vectors in sgl array, 0 for non-sgl input. > + * @param iv > + * pointer to iv data. > + * @param tag_iova > + * Tag iova address. > + * @param aad_iova > + * AAD iova address. > + * @param is_first > + * 1 if it is the first operation in the frame. > + * 0 otherwise. > + * @param is_last > + * 1 if the data is the last element in the frame. > + * 0 otherwise. > + * @param frame > + * if is_first is set the frame pointer will be written in to the mess= age. > + * > + * @return > + * 0 if operation is successful, negative value if otherwise. > + **/ > + > +__rte_experimental > +int > +qat_sym_enqueue_frame_aead(void *qat_sym_qp, > + struct rte_cryptodev_sym_session *session, > + rte_iova_t data_iova, uint32_t cipher_ofs, uint32_t cipher_len, > + struct rte_crypto_vec *sgl, uint32_t n_sgl_vecs, > + uint8_t *iv, rte_iova_t tag_iova, rte_iova_t aad_iova, > + uint8_t is_first, uint8_t is_last, void *frame); > + > +/** > + * enqueue one chaining operation (cipher and hash) into QAT queue > + * > + * @param qat_sym_qp > + * queue pair data got from qat_sym_get_qp(). > + * @param session > + * configured cryptodev symmetric session data. > + * @param data_iova > + * iova address of data. > + * @param cipher_ofs > + * cipher offset start from data_iova. > + * @param cipher_len > + * cipher total length. > + * @param sgl > + * in case of SGL data, pointer to an array of sgl structure. > + * @param n_sgl_vecs > + * number of SGL vectors in sgl array, 0 for non-sgl input. > + * @param iv > + * pointer to iv data. > + * @param digest_iova > + * Digest iova address. > + * @param is_first > + * 1 if it is the first operation in the frame so that opaque is to wr= itten > + * into QAT queue message that can be retrieved upon dequeue. > + * 0 otherwise. > + * @param is_last > + * 1 if the data is the last element in the frame, so that QAT queue t= ail > + * is kicked and the HW will start processing > + * 0 otherwise. > + * @param opaque > + * if is_first is set opaque will be written in to the message. > + * > + * @return > + * 0 if operation is successful, negative value if otherwise. > + **/ > +__rte_experimental > +int > +qat_sym_enqueue_frame_chain(void *qat_sym_qp, > + struct rte_cryptodev_sym_session *session, > + rte_iova_t data_iova, uint32_t cipher_ofs, uint32_t cipher_len, > + uint32_t auth_ofs, uint32_t auth_len, > + struct rte_crypto_vec *sgl, uint32_t n_sgl_vecs, > + uint8_t *iv, rte_iova_t digest_iova, > + uint8_t is_first, uint8_t is_last, void *frame); > + > +/** > + * enqueue one cipher-only operation into QAT queue > + * > + * @param qat_sym_qp > + * queue pair data got from qat_sym_get_qp(). > + * @param session > + * configured cryptodev symmetric session data. > + * @param data_iova > + * iova address of data. > + * @param cipher_ofs > + * cipher offset start from data_iova. > + * @param cipher_len > + * cipher total length. > + * @param sgl > + * in case of SGL data, pointer to an array of sgl structure. > + * @param n_sgl_vecs > + * number of SGL vectors in sgl array, 0 for non-sgl input. > + * @param iv > + * pointer to iv data. > + * @param is_first > + * 1 if it is the first operation in the frame. > + * 0 otherwise. > + * @param is_last > + * 1 if the data is the last element in the frame. > + * 0 otherwise. > + * @param frame > + * if is_first is set the frame pointer will be written in to the mess= age. > + * > + * @return > + * 0 if operation is successful, negative value if otherwise. > + **/ > + > +__rte_experimental > +int > +qat_sym_enqueue_frame_cipher(void *qat_sym_qp, > + struct rte_cryptodev_sym_session *session, > + rte_iova_t data_iova, uint32_t cipher_ofs, uint32_t cipher_len, > + struct rte_crypto_vec *sgl, uint32_t n_sgl_vecs, > + uint8_t *iv, uint8_t is_first, uint8_t is_last, void *frame); > + > +/** > + * enqueue one auth-only operation into QAT queue > + * > + * @param qat_sym_qp > + * queue pair data got from qat_sym_get_qp(). > + * @param session > + * configured cryptodev symmetric session data. > + * @param data_iova > + * iova address of data. > + * @param auth_ofs > + * authentication offset start from data_iova. > + * @param auth_len > + * authentication total length. > + * @param sgl > + * in case of SGL data, pointer to an array of sgl structure. > + * @param n_sgl_vecs > + * number of SGL vectors in sgl array, 0 for non-sgl input. > + * @param iv > + * pointer to iv data. > + * @param digest_iova > + * digest iova address. > + * @param is_first > + * 1 if it is the first operation in the frame. > + * 0 otherwise. > + * @param is_last > + * 1 if the data is the last element in the frame. > + * 0 otherwise. > + * @param frame > + * if is_first is set the frame pointer will be written in to the mess= age. > + * > + * @return > + * 0 if operation is successful, negative value if otherwise. > + **/ > + > +__rte_experimental > +int > +qat_sym_enqueue_frame_auth(void *qat_sym_qp, > + struct rte_cryptodev_sym_session *session, > + rte_iova_t data_iova, uint32_t auth_ofs, uint32_t auth_len, > + struct rte_crypto_vec *sgl, uint32_t n_sgl_vecs, > + uint8_t *iv, rte_iova_t digest_iova, > + uint8_t is_first, uint8_t is_last, void *frame); > + > +/** > + * Function prototype to get the number of elements in a frame in dequeu= e. > + * This function should be provided by the user. > + **/ > +typedef uint32_t (*qat_qp_get_frame_n_element_t)(void *frame); > + > +/** > + * Dequeue a frame from QAT queue > + * > + * @param qat_sym_qp > + * queue pair data got from qat_sym_get_qp(). > + * @param frame > + * return the frame dequeued. > + * @param get_frame_n_elt > + * callback function that gets opaque_data from the first processed me= ssage. > + * @param first_status_offset > + * the offset to status field of first frame element.. > + * @param element_interval > + * the size of frame element in the frame data, used to compute next > + * status field. > + * @param element_status_success > + * value to set for successfully processed frame element. > + * @param element_status_error > + * value to set for unsuccessfully processed frame element. > + * > + * @return > + * if a frame is retrieved from the queue pair it will be written > + * into "frame" parameter, otherwise "frame" will be written as NULL a= nd > + * -1 will be returned. If all elements are successful 0 will be retur= ned. > + * Negative number of failed elements will be returned. > + **/ > +__rte_experimental > +int > +qat_sym_dequeue_frame(void *qat_sym_qp, void **frame, > + qat_qp_get_frame_n_element_t get_frame_n_elt, > + uint32_t first_status_offset, uint32_t element_interval, > + uint8_t element_status_success, uint8_t element_status_error); > + > +#endif /* _QAT_SYM_FRAME_H_ */ > -- > 2.20.1