From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id DB9A2A04BC; Thu, 8 Oct 2020 16:26:13 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E2F8E1C119; Thu, 8 Oct 2020 16:26:11 +0200 (CEST) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140058.outbound.protection.outlook.com [40.107.14.58]) by dpdk.org (Postfix) with ESMTP id 77ADA1BB5C for ; Thu, 8 Oct 2020 16:26:09 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=aeDxuw97FcnDsckJZX5/aeOkRkhnxapmWWWI4x7V3NzqujrO+m/mW8IL0SzQprEKb2LlxJRwR76olB6ta95MIWbP42gfvzj58ZW6BHkr+jKHp6rgJ+O4mPNGdS8QbfIss8PrfVAT1FK4frx86gmyD99yNAI4x+6SK8F4i5yqcsn5QFSqocdH2HBWwTITpp+yP48WLzTQGSqoAexbpp9wN0PPjCDORfBggIp9wRqzVou5rkObfwwjhZI+2JUmIVG34KHJJAU47ABUCuTiCYtPxut8VXeoWCzm45ujO6PM1yJ2BIo8xKOoftS96Ppz7WR4JNsLgSG4pTAzsQNQm5ePvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N+pas99wVW747YGqKi1xaxp9DLAJNobrGEbhAiRlpUY=; b=S+CgOGkAiv9BYQV5MAZlcztWSSfDWIbrgG56/OsLeCI66OoE2l0fpmtSZjO6KTzA6UBQfOZWu4c7Ay/9jJEqnPg71rJDldAK2aExqcIta5FMPFKOVqO9suqO4V4wa4zkcdx5K3kNdj+Ajil08/k37HvCNUqYRygcQf+3jHHJd4M8REHEGC/NZH3z63Kx51pxoVi1e49P6m0NUi3TDw7zkSTqIbn3eXYTxuq28B/ZV309x23xVs/MJkBgQ8FE7EAjLKwT8ysRs/0DHKZsatxJz3f18NMRr0m1TySkTlBk3ymdPvvsZpmWemfRWTwrv5mXASJEko/1fIX3QQ7zk79xvg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=N+pas99wVW747YGqKi1xaxp9DLAJNobrGEbhAiRlpUY=; b=BvelNr6E4R9p8nVlDYPIMh37qOv0JAGu+6uW4ldfKysI1nxo9P9ikVPZq1g/ZqpEpkAWJ67o2PlcmYOplZg7rvbU5axeFCeQ7v7NfzRhTxaZ0fTVdPu6KwZtRpAFMJyXYUbPOR5c15KmWXhL1OEr1iDELaFcuRXIrnZ3KAjJRkg= Received: from VI1PR04MB3168.eurprd04.prod.outlook.com (2603:10a6:802:6::10) by VI1PR04MB4798.eurprd04.prod.outlook.com (2603:10a6:803:53::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3433.32; Thu, 8 Oct 2020 14:26:06 +0000 Received: from VI1PR04MB3168.eurprd04.prod.outlook.com ([fe80::9513:3b55:931f:216e]) by VI1PR04MB3168.eurprd04.prod.outlook.com ([fe80::9513:3b55:931f:216e%4]) with mapi id 15.20.3433.045; Thu, 8 Oct 2020 14:26:06 +0000 From: Akhil Goyal To: Fan Zhang , "dev@dpdk.org" CC: "fiona.trahe@intel.com" , "arkadiuszx.kusztal@intel.com" , "adamx.dybkowski@intel.com" , "anoobj@marvell.com" , "konstantin.ananyev@intel.com" , Piotr Bronowski Thread-Topic: [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs Thread-Index: AQHWkpCTO1BRq1qbhkKXgZJDe+ouAamNxjPg Date: Thu, 8 Oct 2020 14:26:06 +0000 Message-ID: References: <20200908084253.81022-1-roy.fan.zhang@intel.com> <20200924163417.49983-1-roy.fan.zhang@intel.com> <20200924163417.49983-3-roy.fan.zhang@intel.com> In-Reply-To: <20200924163417.49983-3-roy.fan.zhang@intel.com> Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: intel.com; dkim=none (message not signed) header.d=none;intel.com; dmarc=none action=none header.from=nxp.com; x-originating-ip: [122.180.231.103] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 293c3569-381b-4e50-3e7d-08d86b9617d7 x-ms-traffictypediagnostic: VI1PR04MB4798: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8882; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: YXNpIoPJC/NB6z7+HW7AJhgPdFJpiLvofpFLyqYhfoHqfvCZ3pSnaOPaUlufimraFWH4SP8W0X2GYZKhqcoPTXREePgSLyfCK5xBPJHwtVZCmtWbeEyFFW9K1+WswU59l3BG0CYq6XW2b122jVjNZsiWroepu+tY33UIks5Jst/LLE3OLNaKQocrWCb6p7PsApRn5L2oeHjOtBH8fD6kDYyyWxdzecoauhS0BMsSWDLkFsevDQI+XzAK35ZHry3onRsG/s/CGtHrvFEgzJFr2BXzRY508iM188r1wb+uGH6xsTMLl3vevKDaRrIf7aQK/GacxnsQ3DyCniEA57kzlA== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR04MB3168.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(39850400004)(396003)(346002)(136003)(376002)(8936002)(6506007)(55016002)(30864003)(478600001)(8676002)(9686003)(86362001)(5660300002)(44832011)(71200400001)(26005)(2906002)(66476007)(316002)(186003)(52536014)(64756008)(76116006)(66946007)(66446008)(83380400001)(7696005)(66556008)(110136005)(33656002)(54906003)(4326008)(579004)(559001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: 1aCqUSU6XRve0sjHhUpLdXQVKmm1H3eUbzqPecBij/3LEEbsbMuszt9mapi6NgJaz45zJQsEC6e+iKPpLIfZLmTH7kuzUvPU0aJWwwI/U5+68B92AaBf733YJFelj5RKwfrZCH6bytuQ0bHVihlnWizsQUjSYt+LtHnC/QpXC3jHms9qAkFmn03Tr1J/f4aQ5q35radPXTSmbiDLhd/BQFX6dx0d63HleorMcnRbpyDFYr972jQKj0KYOUAyaWL9jgQAbMjRXA6bTrUXqFtJBrnjqMX0pnBIyyqaUOyVe0XV3prRQDAwRpi7XFen2QuuaKAlhFxZbJpQPeUbBdLJdeQWT052xXC2m+LOdXiApvtdOL/CgpVSQsPLVpOIombaICQsP9b1Kj+dJ45FZ1m+y4N82qh3+cP5sJpg5ufrFbA2WK8+w4CPTWZ1GqWe5vr9wNntwoNJNV/jqYlARi/1/OdNEX5znulNKll82mW2Qp5bK0JNiJUM+T+LxAtbElOGhkqq7iRp5uoEffUuECvXcU/6Q1we/vXKTZdeqBCjQS7TzVvlx40OJrO6Vct5HvkMYxN1qB/yzNVWLs+TRRKyE/glL8rZ4FVpCz76Q1NE+oGocgQuMKo42TgGT3xPa91Hd8agbVCT+jfi26gH/L0oIA== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB3168.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 293c3569-381b-4e50-3e7d-08d86b9617d7 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Oct 2020 14:26:06.1074 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: ay+qIkTqtde0dTuB0ff3KKgO4yojBx4Y4GZN7o0nL0Lqz8eqT2wrEnJaEKBfmkdxNefBZvweYyq3K1d2GpzfZg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR04MB4798 Subject: Re: [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Fan, >=20 > This patch adds raw data-path APIs for enqueue and dequeue > operations to cryptodev. The APIs support flexible user-define > enqueue and dequeue behaviors. >=20 > Signed-off-by: Fan Zhang > Signed-off-by: Piotr Bronowski > --- > doc/guides/prog_guide/cryptodev_lib.rst | 93 +++++ > doc/guides/rel_notes/release_20_11.rst | 7 + > lib/librte_cryptodev/rte_crypto_sym.h | 2 +- > lib/librte_cryptodev/rte_cryptodev.c | 104 +++++ > lib/librte_cryptodev/rte_cryptodev.h | 354 +++++++++++++++++- > lib/librte_cryptodev/rte_cryptodev_pmd.h | 47 ++- > .../rte_cryptodev_version.map | 11 + > 7 files changed, 614 insertions(+), 4 deletions(-) >=20 > diff --git a/doc/guides/prog_guide/cryptodev_lib.rst > b/doc/guides/prog_guide/cryptodev_lib.rst > index e7ba35c2d..5fe6c3c24 100644 > --- a/doc/guides/prog_guide/cryptodev_lib.rst > +++ b/doc/guides/prog_guide/cryptodev_lib.rst > @@ -632,6 +632,99 @@ a call argument. Status different than zero must be > treated as error. > For more details, e.g. how to convert an mbuf to an SGL, please refer to= an > example usage in the IPsec library implementation. >=20 > +Cryptodev Raw Data-path APIs > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The Crypto Raw data-path APIs are a set of APIs are designed to enable The Crypto Raw data-path APIs are a set of APIs designed to enable external libraries/applications to leverage the cryptographic=20 > +externel libraries/applications which want to leverage the cryptographic > +processing provided by DPDK crypto PMDs through the cryptodev API but in= a > +manner that is not dependent on native DPDK data structures (eg. rte_mbu= f, > +rte_crypto_op, ... etc) in their data-path implementation. > + > +The raw data-path APIs have the following advantages: > +- External data structure friendly design. The new APIs uses the operati= on > + descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointe= r and > + IOVA addresses as input. Moreover, the APIs does not require the user = to > + allocate the descriptor from mempool, nor requiring mbufs to describe = input > + data's virtual and IOVA addresses. All these features made the transla= tion > + from user's own data structure into the descriptor easier and more eff= icient. > +- Flexible enqueue and dequeue operation. The raw data-path APIs gives t= he > + user more control to the enqueue and dequeue operations, including the > + capability of precious enqueue/dequeue count, abandoning enqueue or > dequeue > + at any time, and operation status translation and set on the fly. > + > +Cryptodev PMDs who supports the raw data-path APIs will have Cryptodev PMDs which support raw data-path APIs will have > +``RTE_CRYPTODEV_FF_SYM_HW_RAW_DP`` feature flag presented. To use this > +feature, the user should create a local ``struct rte_crypto_raw_dp_ctx`` > +buffer and extend to at least the length returned by > +``rte_cryptodev_raw_get_dp_context_size`` function call. The created buf= fer ``rte_cryptodev_get_raw_dp_ctx_size`` > +is then configured using ``rte_cryptodev_raw_configure_dp_context`` func= tion. rte_cryptodev _configure_raw_dp_ctx > +The library and the crypto device driver will then configure the buffer = and > +write necessary temporary data into the buffer for later enqueue and deq= ueue > +operations. The temporary data may be treated as the shadow copy of the > +driver's private queue pair data. > + > +After the ``struct rte_crypto_raw_dp_ctx`` buffer is initialized, it is = then > +attached either the cryptodev sym session, the rte_security session, or = the attached either with the cryptodev sym session, the rte_security session, o= r the > +cryptodev xform for session-less operation by > +``rte_cryptodev_raw_attach_session`` function. With the session or xform ``rte_cryptodev_attach_raw_session`` API > +information the driver will set the corresponding enqueue and dequeue > function > +handlers to the ``struct rte_crypto_raw_dp_ctx`` buffer. > + > +After the session is attached, the ``struct rte_crypto_raw_dp_ctx`` buff= er is > +now ready for enqueue and dequeue operation. There are two different > enqueue > +functions: ``rte_cryptodev_raw_enqueue`` to enqueue single descriptor, > +and ``rte_cryptodev_raw_enqueue_burst`` to enqueue multiple descriptors. > +In case of the application uses similar approach to > +``struct rte_crypto_sym_vec`` to manage its data burst but with differen= t > +data structure, using the ``rte_cryptodev_raw_enqueue_burst`` function m= ay > be > +less efficient as this is a situation where the application has to loop = over > +all crypto descriptors to assemble the ``struct rte_crypto_sym_vec`` buf= fer > +from its own data structure, and then the driver will loop over them aga= in to > +translate every crypto job to the driver's specific queue data. The > +``rte_cryptodev_raw_enqueue`` should be used to save one loop for each d= ata > +burst instead. > + > +During the enqueue, the cryptodev driver only sets the enqueued descript= ors > +into the device queue but not initiates the device to start processing t= hem. > +The temporary queue pair data changes in relation to the enqueued descri= ptors > +may be recorded in the ``struct rte_crypto_raw_dp_ctx`` buffer as the > reference > +to the next enqueue function call. When ``rte_cryptodev_raw_enqueue_done= `` > is > +called, the driver will initiate the processing of all enqueued descript= ors and > +merge the temporary queue pair data changes into the driver's private qu= eue > +pair data. Calling ``rte_cryptodev_raw_configure_dp_context`` twice with= out > +``rte_cryptodev_dp_enqueue_done`` call in between will invalidate the > temporary > +data stored in ``struct rte_crypto_raw_dp_ctx`` buffer. This feature is = useful > +when the user wants to abandon partially enqueued data for a failed enqu= eue > +burst operation and try enqueuing in a whole later. This feature may not be supported by all the HW PMDs, Can there be a way to= bypass this done API? > + > +Similar as enqueue, there are two dequeue functions: > +``rte_cryptodev_raw_dequeue`` for dequeing single descriptor, and > +``rte_cryptodev_raw_dequeue_burst`` for dequeuing a burst of descriptor.= The > +dequeue functions only writes back the user data that was passed to the = driver > +during inqueue, and inform the application the operation status. during enqueue, and inform the operation status to the application. > +Different than ``rte_cryptodev_dequeue_burst`` which the user can only > +set an expected dequeue count and needs to read from dequeued cryptodev > +operations' status field, the raw data-path dequeue burst function allow= s > +the user to provide callback functions to retrieve dequeue > +count from the enqueued user data, and write the expected status value t= o the > +user data on the fly. > + > +Same as enqueue, both ``rte_cryptodev_raw_dequeue`` and > +``rte_cryptodev_raw_dequeue_burst`` will not wipe the dequeued descripto= rs > +from cryptodev queue unless ``rte_cryptodev_dp_dequeue_done`` is called. > The > +dequeue related temporary queue data will be merged into the driver's pr= ivate > +queue data in the function call. > + > +There are a few limitations to the data path service: > + > +* Only support in-place operations. > +* APIs are NOT thread-safe. > +* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, = or > + vice versa. > + > +See *DPDK API Reference* for details on each API definitions. > + > Sample code > ----------- >=20 > diff --git a/doc/guides/rel_notes/release_20_11.rst > b/doc/guides/rel_notes/release_20_11.rst > index 20ebaef5b..d3d9f82f7 100644 > --- a/doc/guides/rel_notes/release_20_11.rst > +++ b/doc/guides/rel_notes/release_20_11.rst > @@ -55,6 +55,13 @@ New Features > Also, make sure to start the actual text at the margin. > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D >=20 > + * **Added raw data-path APIs for cryptodev library.** > + > + Cryptodev is added raw data-path APIs to accelerate external librar= ies or > + applications those want to avail fast cryptodev enqueue/dequeue > + operations but does not necessarily depends on mbufs and cryptodev > + operation mempool. Raw crypto data path APIs are added to accelerate external libraries or Applications which need to perform crypto processing on raw buffers and Not dependent on rte_mbufs or rte_crypto_op mempools. > + >=20 > Removed Items > ------------- > diff --git a/lib/librte_cryptodev/rte_crypto_sym.h > b/lib/librte_cryptodev/rte_crypto_sym.h > index 8201189e0..e1f23d303 100644 > --- a/lib/librte_cryptodev/rte_crypto_sym.h > +++ b/lib/librte_cryptodev/rte_crypto_sym.h > @@ -57,7 +57,7 @@ struct rte_crypto_sgl { > */ > struct rte_crypto_va_iova_ptr { > void *va; > - rte_iova_t *iova; > + rte_iova_t iova; > }; This should be part of 1/4 of this patchset. >=20 > /** > diff --git a/lib/librte_cryptodev/rte_cryptodev.c > b/lib/librte_cryptodev/rte_cryptodev.c > index 1dd795bcb..daeb5f504 100644 > --- a/lib/librte_cryptodev/rte_cryptodev.c > +++ b/lib/librte_cryptodev/rte_cryptodev.c > @@ -1914,6 +1914,110 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t > dev_id, > return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec); > } >=20 > +int > +rte_cryptodev_raw_get_dp_context_size(uint8_t dev_id) As suggested above raw_dp should be used as keyword in all the APIs Hence it should be rte_cryptodev_get_raw_dp_ctx_size > +{ > + struct rte_cryptodev *dev; > + int32_t size =3D sizeof(struct rte_crypto_raw_dp_ctx); > + int32_t priv_size; > + > + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) > + return -EINVAL; > + > + dev =3D rte_cryptodev_pmd_get_dev(dev_id); > + > + if (*dev->dev_ops->get_drv_ctx_size =3D=3D NULL || > + !(dev->feature_flags & > RTE_CRYPTODEV_FF_SYM_HW_RAW_DP)) { > + return -ENOTSUP; > + } > + > + priv_size =3D (*dev->dev_ops->get_drv_ctx_size)(dev); > + if (priv_size < 0) > + return -ENOTSUP; > + > + return RTE_ALIGN_CEIL((size + priv_size), 8); > +} > + > +int > +rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t qp_id, > + struct rte_crypto_raw_dp_ctx *ctx) rte_cryptodev_configure_raw_dp_ctx > +{ > + struct rte_cryptodev *dev; > + union rte_cryptodev_session_ctx sess_ctx =3D {NULL}; > + > + if (!rte_cryptodev_get_qp_status(dev_id, qp_id)) > + return -EINVAL; > + > + dev =3D rte_cryptodev_pmd_get_dev(dev_id); > + if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP) > + || dev->dev_ops->configure_dp_ctx =3D=3D NULL) > + return -ENOTSUP; > + > + return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id, > + RTE_CRYPTO_OP_WITH_SESSION, sess_ctx, ctx); > +} > + > +int > +rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id, > + struct rte_crypto_raw_dp_ctx *ctx, > + enum rte_crypto_op_sess_type sess_type, > + union rte_cryptodev_session_ctx session_ctx) > +{ > + struct rte_cryptodev *dev; > + > + if (!rte_cryptodev_get_qp_status(dev_id, qp_id)) > + return -EINVAL; > + > + dev =3D rte_cryptodev_pmd_get_dev(dev_id); > + if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_HW_RAW_DP) > + || dev->dev_ops->configure_dp_ctx =3D=3D NULL) > + return -ENOTSUP; > + return (*dev->dev_ops->configure_dp_ctx)(dev, qp_id, sess_type, > + session_ctx, ctx); What is the difference between rte_cryptodev_raw_configure_dp_context and rte_cryptodev_raw_attach_session? And if at all it is needed, then it should be rte_cryptodev_attach_raw_dp_s= ession. IMO attach is not needed, I am not clear about it. You are calling the same dev_ops for both - one with explicit session time = and other >From an argument. > +} > + > +uint32_t > +rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx, > + struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs, > + void **user_data) > +{ > + if (vec->num =3D=3D 1) { Why do we need this check? I think user is aware that for enqueuing 1 vecto= r, He should use the other API. Driver will be doing the enqueue operation onl= y one time. > + vec->status[0] =3D rte_cryptodev_raw_enqueue(ctx, vec->sgl- > >vec, > + vec->sgl->num, ofs, vec->iv, vec->digest, vec->aad, > + user_data[0]); > + return (vec->status[0] =3D=3D 0) ? 1 : 0; > + } > + > + return (*ctx->enqueue_burst)(ctx->qp_data, ctx->drv_ctx_data, vec, > + ofs, user_data); > +} Where are rte_cryptodev_raw_enqueue and rte_cryptodev_raw_dequeue ?? > + > +int > +rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx, > + uint32_t n) > +{ > + return (*ctx->enqueue_done)(ctx->qp_data, ctx->drv_ctx_data, n); > +} > + > +int > +rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx, > + uint32_t n) > +{ > + return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n); > +} > + > +uint32_t > +rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx, > + rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count, > + rte_cryptodev_raw_post_dequeue_t post_dequeue, > + void **out_user_data, uint8_t is_user_data_array, > + uint32_t *n_success_jobs) > +{ > + return (*ctx->dequeue_burst)(ctx->qp_data, ctx->drv_ctx_data, > + get_dequeue_count, post_dequeue, out_user_data, > + is_user_data_array, n_success_jobs); > +} > + > /** Initialise rte_crypto_op mempool element */ > static void > rte_crypto_op_init(struct rte_mempool *mempool, > diff --git a/lib/librte_cryptodev/rte_cryptodev.h > b/lib/librte_cryptodev/rte_cryptodev.h > index 7b3ebc20f..3579ab66e 100644 > --- a/lib/librte_cryptodev/rte_cryptodev.h > +++ b/lib/librte_cryptodev/rte_cryptodev.h > @@ -466,7 +466,8 @@ rte_cryptodev_asym_get_xform_enum(enum > rte_crypto_asym_xform_type *xform_enum, > /**< Support symmetric session-less operations */ > #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA (1ULL > << 23) > /**< Support operations on data which is not byte aligned */ > - > +#define RTE_CRYPTODEV_FF_SYM_HW_RAW_DP (1ULL > << 24) RTE_CRYPTODEV_FF_SYM_RAW_DP should be better Add this in doc/guides/cryptodevs/features/default.ini as well in this patc= h. > +/**< Support accelerated specific raw data-path APIs */ >=20 > /** > * Get the name of a crypto device feature flag > @@ -1351,6 +1352,357 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t > dev_id, > struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs, > struct rte_crypto_sym_vec *vec); >=20 > +/** > + * Get the size of the raw data-path context buffer. > + * > + * @param dev_id The device identifier. > + * > + * @return > + * - If the device supports raw data-path APIs, return the context siz= e. > + * - If the device does not support the APIs, return -1. > + */ > +__rte_experimental > +int > +rte_cryptodev_raw_get_dp_context_size(uint8_t dev_id); > + > +/** > + * Union of different crypto session types, including session-less xform > + * pointer. > + */ > +union rte_cryptodev_session_ctx { > + struct rte_cryptodev_sym_session *crypto_sess; > + struct rte_crypto_sym_xform *xform; > + struct rte_security_session *sec_sess; > +}; > + > +/** > + * Enqueue a data vector into device queue but the driver will not start > + * processing until rte_cryptodev_raw_enqueue_done() is called. > + * > + * @param qp Driver specific queue pair data. > + * @param drv_ctx Driver specific context data. > + * @param vec The array of descriptor vectors. > + * @param ofs Start and stop offsets for auth and cipher > + * operations. > + * @param user_data The array of user data for dequeue later. > + * @return > + * - The number of descriptors successfully submitted. > + */ > +typedef uint32_t (*cryptodev_dp_sym_enqueue_burst_t)( > + void *qp, uint8_t *drv_ctx, struct rte_crypto_sym_vec *vec, > + union rte_crypto_sym_ofs ofs, void *user_data[]); > + > +/** > + * Enqueue single descriptor into device queue but the driver will not s= tart > + * processing until rte_cryptodev_raw_enqueue_done() is called. > + * > + * @param qp Driver specific queue pair data. > + * @param drv_ctx Driver specific context data. > + * @param data_vec The buffer data vector. > + * @param n_data_vecs Number of buffer data vectors. > + * @param ofs Start and stop offsets for auth and cipher > + * operations. > + * @param iv IV virtual and IOVA addresses > + * @param digest digest virtual and IOVA addresses > + * @param aad_or_auth_iv AAD or auth IV virtual and IOVA addresses, > + * depends on the algorithm used. > + * @param user_data The user data. > + * @return > + * - On success return 0. > + * - On failure return negative integer. > + */ > +typedef int (*cryptodev_dp_sym_enqueue_t)( > + void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec, > + uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs, > + struct rte_crypto_va_iova_ptr *iv, > + struct rte_crypto_va_iova_ptr *digest, > + struct rte_crypto_va_iova_ptr *aad_or_auth_iv, > + void *user_data); > + > +/** > + * Inform the cryptodev queue pair to start processing or finish dequeui= ng all > + * enqueued/dequeued descriptors. > + * > + * @param qp Driver specific queue pair data. > + * @param drv_ctx Driver specific context data. > + * @param n The total number of processed descriptors. > + * @return > + * - On success return 0. > + * - On failure return negative integer. > + */ > +typedef int (*cryptodev_dp_sym_operation_done_t)(void *qp, uint8_t *drv_= ctx, > + uint32_t n); > + > +/** > + * Typedef that the user provided for the driver to get the dequeue coun= t. > + * The function may return a fixed number or the number parsed from the = user > + * data stored in the first processed descriptor. > + * > + * @param user_data Dequeued user data. > + **/ > +typedef uint32_t (*rte_cryptodev_raw_get_dequeue_count_t)(void > *user_data); > + > +/** > + * Typedef that the user provided to deal with post dequeue operation, s= uch > + * as filling status. > + * > + * @param user_data Dequeued user data. > + * @param index Index number of the processed descriptor. > + * @param is_op_success Operation status provided by the driver. > + **/ > +typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data, > + uint32_t index, uint8_t is_op_success); > + > +/** > + * Dequeue symmetric crypto processing of user provided data. > + * > + * @param qp Driver specific queue pair data. > + * @param drv_ctx Driver specific context data. > + * @param get_dequeue_count User provided callback function to > + * obtain dequeue count. > + * @param post_dequeue User provided callback function to > + * post-process a dequeued operation. > + * @param out_user_data User data pointer array to be retrieve > + * from device queue. In case of > + * *is_user_data_array* is set there > + * should be enough room to store all > + * user data. > + * @param is_user_data_array Set 1 if every dequeued user data will > + * be written into out_user_data* array. > + * @param n_success Driver written value to specific the > + * total successful operations count. > + * > + * @return > + * - Returns number of dequeued packets. > + */ > +typedef uint32_t (*cryptodev_dp_sym_dequeue_burst_t)(void *qp, > + uint8_t *drv_ctx, > + rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count, > + rte_cryptodev_raw_post_dequeue_t post_dequeue, > + void **out_user_data, uint8_t is_user_data_array, > + uint32_t *n_success); > + > +/** > + * Dequeue symmetric crypto processing of user provided data. > + * > + * @param qp Driver specific queue pair data. > + * @param drv_ctx Driver specific context data. > + * @param out_user_data User data pointer to be retrieve from > + * device queue. > + * > + * @return > + * - 1 if the user_data is dequeued and the operation is a success. > + * - 0 if the user_data is dequeued but the operation is failed. > + * - -1 if no operation is dequeued. > + */ > +typedef int (*cryptodev_dp_sym_dequeue_t)( > + void *qp, uint8_t *drv_ctx, void **out_user_data); > + > +/** > + * Context data for raw data-path API crypto process. The buffer of this > + * structure is to be allocated by the user application with the size eq= ual > + * or bigger than rte_cryptodev_raw_get_dp_context_size() returned value= . > + * > + * NOTE: the buffer is to be used and maintained by the cryptodev driver= , the > + * user should NOT alter the buffer content to avoid application or syst= em > + * crash. > + */ > +struct rte_crypto_raw_dp_ctx { > + void *qp_data; > + > + cryptodev_dp_sym_enqueue_t enqueue; > + cryptodev_dp_sym_enqueue_burst_t enqueue_burst; > + cryptodev_dp_sym_operation_done_t enqueue_done; > + cryptodev_dp_sym_dequeue_t dequeue; > + cryptodev_dp_sym_dequeue_burst_t dequeue_burst; > + cryptodev_dp_sym_operation_done_t dequeue_done; These function pointers are data path only. Why do we need to add explicit = dp in each one of them These should be cryptodev_sym_raw_** > + > + /* Driver specific context data */ > + __extension__ uint8_t drv_ctx_data[]; > +}; > + > +/** > + * Configure raw data-path context data. > + * > + * NOTE: > + * After the context data is configured, the user should call > + * rte_cryptodev_raw_attach_session() before using it in > + * rte_cryptodev_raw_enqueue/dequeue function call. I am not clear of the purpose of attach API? It looks an overhead to me. > + * > + * @param dev_id The device identifier. > + * @param qp_id The index of the queue pair from which to > + * retrieve processed packets. The value must be > + * in the range [0, nb_queue_pair - 1] previously > + * supplied to rte_cryptodev_configure(). > + * @param ctx The raw data-path context data. > + * @return > + * - On success return 0. > + * - On failure return negative integer. > + */ > +__rte_experimental > +int > +rte_cryptodev_raw_configure_dp_context(uint8_t dev_id, uint16_t qp_id, > + struct rte_crypto_raw_dp_ctx *ctx); > + > +/** > + * Attach a cryptodev session to a initialized raw data path context. > + * > + * @param dev_id The device identifier. > + * @param qp_id The index of the queue pair from which to > + * retrieve processed packets. The value must be > + * in the range [0, nb_queue_pair - 1] previously > + * supplied to rte_cryptodev_configure(). > + * @param ctx The raw data-path context data. > + * @param sess_type session type. > + * @param session_ctx Session context data. > + * @return > + * - On success return 0. > + * - On failure return negative integer. > + */ > +__rte_experimental > +int > +rte_cryptodev_raw_attach_session(uint8_t dev_id, uint16_t qp_id, > + struct rte_crypto_raw_dp_ctx *ctx, > + enum rte_crypto_op_sess_type sess_type, > + union rte_cryptodev_session_ctx session_ctx); > + > +/** > + * Enqueue single raw data-path descriptor. > + * > + * The enqueued descriptor will not be started processing until > + * rte_cryptodev_raw_enqueue_done() is called. > + * > + * @param ctx The initialized raw data-path context data. > + * @param data_vec The buffer vector. > + * @param n_data_vecs Number of buffer vectors. > + * @param ofs Start and stop offsets for auth and cipher > + * operations. > + * @param iv IV virtual and IOVA addresses > + * @param digest digest virtual and IOVA addresses > + * @param aad_or_auth_iv AAD or auth IV virtual and IOVA addresses, > + * depends on the algorithm used. > + * @param user_data The user data. > + * @return > + * - The number of descriptors successfully enqueued. > + */ > +__rte_experimental > +static __rte_always_inline int > +rte_cryptodev_raw_enqueue(struct rte_crypto_raw_dp_ctx *ctx, > + struct rte_crypto_vec *data_vec, uint16_t n_data_vecs, > + union rte_crypto_sym_ofs ofs, > + struct rte_crypto_va_iova_ptr *iv, > + struct rte_crypto_va_iova_ptr *digest, > + struct rte_crypto_va_iova_ptr *aad_or_auth_iv, > + void *user_data) > +{ > + return (*ctx->enqueue)(ctx->qp_data, ctx->drv_ctx_data, data_vec, > + n_data_vecs, ofs, iv, digest, aad_or_auth_iv, user_data); > +} > + > +/** > + * Enqueue a data vector of raw data-path descriptors. > + * > + * The enqueued descriptors will not be started processing until > + * rte_cryptodev_raw_enqueue_done() is called. > + * > + * @param ctx The initialized raw data-path context data. > + * @param vec The array of descriptor vectors. > + * @param ofs Start and stop offsets for auth and cipher > + * operations. > + * @param user_data The array of opaque data for dequeue. > + * @return > + * - The number of descriptors successfully enqueued. > + */ > +__rte_experimental > +uint32_t > +rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx, > + struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs, > + void **user_data); > + > +/** > + * Start processing all enqueued descriptors from last > + * rte_cryptodev_raw_configure_dp_context() call. > + * > + * @param ctx The initialized raw data-path context data. > + * @param n The total number of submitted descriptors. > + */ > +__rte_experimental > +int > +rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx, > + uint32_t n); > + > +/** > + * Dequeue a burst of raw crypto data-path operations and write the prev= iously > + * enqueued user data into the array provided. > + * > + * The dequeued operations, including the user data stored, will not be > + * wiped out from the device queue until rte_cryptodev_raw_dequeue_done(= ) > is > + * called. > + * > + * @param ctx The initialized raw data-path context > + * data. > + * @param get_dequeue_count User provided callback function to > + * obtain dequeue count. > + * @param post_dequeue User provided callback function to > + * post-process a dequeued operation. > + * @param out_user_data User data pointer array to be retrieve > + * from device queue. In case of > + * *is_user_data_array* is set there > + * should be enough room to store all > + * user data. > + * @param is_user_data_array Set 1 if every dequeued user data will > + * be written into *out_user_data* array. > + * @param n_success Driver written value to specific the > + * total successful operations count. // to specify the > + * > + * @return > + * - Returns number of dequeued packets. > + */ > +__rte_experimental > +uint32_t > +rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx, > + rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count, > + rte_cryptodev_raw_post_dequeue_t post_dequeue, > + void **out_user_data, uint8_t is_user_data_array, > + uint32_t *n_success); > + > +/** > + * Dequeue a raw crypto data-path operation and write the previously > + * enqueued user data. > + * > + * The dequeued operation, including the user data stored, will not be w= iped > + * out from the device queue until rte_cryptodev_raw_dequeue_done() is > called. > + * > + * @param ctx The initialized raw data-path context > + * data. > + * @param out_user_data User data pointer to be retrieve from > + * device queue. The driver shall support > + * NULL input of this parameter. > + * > + * @return > + * - 1 if the user data is dequeued and the operation is a success. > + * - 0 if the user data is dequeued but the operation is failed. > + * - -1 if no operation is ready to be dequeued. > + */ > +__rte_experimental > +static __rte_always_inline int Why is this function specifically inline and not others? > +rte_cryptodev_raw_dequeue(struct rte_crypto_raw_dp_ctx *ctx, > + void **out_user_data) > +{ > + return (*ctx->dequeue)(ctx->qp_data, ctx->drv_ctx_data, > out_user_data); > +} > + > +/** > + * Inform the queue pair dequeue operations finished. > + * > + * @param ctx The initialized raw data-path context data. > + * @param n The total number of jobs already dequeued. > + */ > +__rte_experimental > +int > +rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx, > + uint32_t n); > + > #ifdef __cplusplus > } > #endif > diff --git a/lib/librte_cryptodev/rte_cryptodev_pmd.h > b/lib/librte_cryptodev/rte_cryptodev_pmd.h > index 81975d72b..69a2a6d64 100644 > --- a/lib/librte_cryptodev/rte_cryptodev_pmd.h > +++ b/lib/librte_cryptodev/rte_cryptodev_pmd.h > @@ -316,6 +316,40 @@ typedef uint32_t > (*cryptodev_sym_cpu_crypto_process_t) > (struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess, > union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec); >=20 > +/** > + * Typedef that the driver provided to get service context private date = size. > + * > + * @param dev Crypto device pointer. > + * > + * @return > + * - On success return the size of the device's service context privat= e data. > + * - On failure return negative integer. > + */ > +typedef int (*cryptodev_dp_get_service_ctx_size_t)( > + struct rte_cryptodev *dev); > + > +/** > + * Typedef that the driver provided to configure data-path context. > + * > + * @param dev Crypto device pointer. > + * @param qp_id Crypto device queue pair index. > + * @param service_type Type of the service requested. > + * @param sess_type session type. > + * @param session_ctx Session context data. If NULL the driver > + * shall only configure the drv_ctx_data in > + * ctx buffer. Otherwise the driver shall only > + * parse the session_ctx to set appropriate > + * function pointers in ctx. > + * @param ctx The raw data-path context data. > + * @return > + * - On success return 0. > + * - On failure return negative integer. > + */ > +typedef int (*cryptodev_dp_configure_ctx_t)( > + struct rte_cryptodev *dev, uint16_t qp_id, > + enum rte_crypto_op_sess_type sess_type, > + union rte_cryptodev_session_ctx session_ctx, > + struct rte_crypto_raw_dp_ctx *ctx); These typedefs names are not matching with the corresponding API names. Can you fix it for all of them? >=20 > /** Crypto device operations function pointer table */ > struct rte_cryptodev_ops { > @@ -348,8 +382,17 @@ struct rte_cryptodev_ops { > /**< Clear a Crypto sessions private data. */ > cryptodev_asym_free_session_t asym_session_clear; > /**< Clear a Crypto sessions private data. */ > - cryptodev_sym_cpu_crypto_process_t sym_cpu_process; > - /**< process input data synchronously (cpu-crypto). */ > + union { > + cryptodev_sym_cpu_crypto_process_t sym_cpu_process; > + /**< process input data synchronously (cpu-crypto). */ > + __extension__ > + struct { > + cryptodev_dp_get_service_ctx_size_t get_drv_ctx_size; > + /**< Get data path service context data size. */ > + cryptodev_dp_configure_ctx_t configure_dp_ctx; > + /**< Initialize crypto service ctx data. */ > + }; > + }; > }; >=20 >=20 > diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map > b/lib/librte_cryptodev/rte_cryptodev_version.map > index 02f6dcf72..bc4cd1ea5 100644 > --- a/lib/librte_cryptodev/rte_cryptodev_version.map > +++ b/lib/librte_cryptodev/rte_cryptodev_version.map > @@ -105,4 +105,15 @@ EXPERIMENTAL { >=20 > # added in 20.08 > rte_cryptodev_get_qp_status; > + > + # added in 20.11 > + rte_cryptodev_raw_attach_session; > + rte_cryptodev_raw_configure_dp_context; > + rte_cryptodev_raw_get_dp_context_size; > + rte_cryptodev_raw_dequeue; > + rte_cryptodev_raw_dequeue_burst; > + rte_cryptodev_raw_dequeue_done; > + rte_cryptodev_raw_enqueue; > + rte_cryptodev_raw_enqueue_burst; > + rte_cryptodev_raw_enqueue_done; > }; > -- > 2.20.1