From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 2B60EA04B5;
Wed, 9 Sep 2020 18:21:13 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id BD7601BE0C;
Wed, 9 Sep 2020 18:21:12 +0200 (CEST)
Received: from mga05.intel.com (mga05.intel.com [192.55.52.43])
by dpdk.org (Postfix) with ESMTP id 341101DB8
for <dev@dpdk.org>; Wed, 9 Sep 2020 18:21:11 +0200 (CEST)
IronPort-SDR: S+h+gyAPtVaMjiOMInbDeDaCbUP727f498h426RHyfwQNT7dRmd7EUxIYmGbbabjJZ5o3fH3uP
v5dnvAnUagUQ==
X-IronPort-AV: E=McAfee;i="6000,8403,9739"; a="243174579"
X-IronPort-AV: E=Sophos;i="5.76,409,1592895600"; d="scan'208";a="243174579"
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga002.jf.intel.com ([10.7.209.21])
by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
09 Sep 2020 09:21:10 -0700
IronPort-SDR: E65drjZEDpXckCfJ3uGdX89GgSkU35NAzWZwxGNouMSP95GaPJrh+Ix63Hb/vuNM//eRhJ7hSX
rj8Pz6oqpEIw==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.76,409,1592895600"; d="scan'208";a="317595109"
Received: from fmsmsx606.amr.corp.intel.com ([10.18.126.86])
by orsmga002.jf.intel.com with ESMTP; 09 Sep 2020 09:21:09 -0700
Received: from fmsmsx603.amr.corp.intel.com (10.18.126.83) by
fmsmsx606.amr.corp.intel.com (10.18.126.86) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id
15.1.1713.5; Wed, 9 Sep 2020 09:21:09 -0700
Received: from fmsedg602.ED.cps.intel.com (10.1.192.136) by
fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5
via Frontend Transport; Wed, 9 Sep 2020 09:21:09 -0700
Received: from NAM02-CY1-obe.outbound.protection.outlook.com (104.47.37.58) by
edgegateway.intel.com (192.55.55.71) with Microsoft SMTP Server
(version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id
15.1.1713.5; Wed, 9 Sep 2020 09:21:08 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=HV8R75wjK86WM9IF6AuPGQkrXmcyf9u7BFWRluE8Lpat9UV4D8OFMje5ZHD+g1m18fAvEOGNwaAugnSZNcCUU7+V8ISlN/f3BSb19BsW9Y9dJGETp0uWGNFx/hlIwF0HIfzUf2boEMYJeSHzgTSxve6/xKqarvfPMK5ZX+Scbfyc1gD0fhbAlOvDV5t4hNiChPeh0QKTkq2h1B7J4KPVthez7t1GN86azXhfNdbbGr9Gtsa7GAd51ExLDVqD/ADdzCOWRaSOt4PWgJiXa876h105s5mPkWTIPn3X2qzqi6uDzJlQSveBHlvLUo2aHAaHr3/DNdsOSEnnx+onZeWAwg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=JV0xcK21zEJbv5xg897MLLN/JSz08C1mCvWMm4COSj0=;
b=mm73or8qOHqIX6+34w690M7MfVuixaVpO3y7XSERZdKMoOok/QVNHGqPPMvOAhQLMExeXl7pfu5JQLvs95xjRyFq+HBYOiC/6n7xRaAULFhxq8xEj5QdXsejbfm6c1tfeQBRHdzklUMcbzwPmXEa45stIt3428qHn80k1nsWu+V2HiQozKHpKD5rNyYOCR/uNV/FhtqkaPdhBByl/aKbeBUBNz+RPt+KvYObm9mZV7kHP3WXcOtrYAz4OaI/26qXayTicz2zG0GeuX1qj904h66Qw1ahF3Ub129KL5CslwlsU/xp5LGynj88hOmLnOgtPDrho/GnmuDwc89t95daHg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com;
s=selector2-intel-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=JV0xcK21zEJbv5xg897MLLN/JSz08C1mCvWMm4COSj0=;
b=OAvPsX/K44Cdjeu44ojQltv48UOMhIjcS1cLYCmlS9k8/83CKh1wx3sLJJAAoZvoR8j5YDQFBp6++rAqUcd98cBMU/EzKl4yaYGDhhbnCITmhpScG7EYGXl+bmzeKIzAwJWydLLyTCK+a2o5mADGMoiOWlxAxdEvFbgE1gkGdHU=
Received: from MWHPR11MB1838.namprd11.prod.outlook.com (2603:10b6:300:10c::11)
by MWHPR1101MB2077.namprd11.prod.outlook.com (2603:10b6:301:4d::16)
with Microsoft SMTP Server (version=TLS1_2,
cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3348.16; Wed, 9 Sep
2020 16:21:04 +0000
Received: from MWHPR11MB1838.namprd11.prod.outlook.com
([fe80::34c5:6550:a098:209f]) by MWHPR11MB1838.namprd11.prod.outlook.com
([fe80::34c5:6550:a098:209f%12]) with mapi id 15.20.3348.019; Wed, 9 Sep 2020
16:21:04 +0000
From: "Gujjar, Abhinandan S" <abhinandan.gujjar@intel.com>
To: "dev@dpdk.org" <dev@dpdk.org>
CC: "Doherty, Declan" <declan.doherty@intel.com>, "jerinj@marvell.com"
<jerinj@marvell.com>, "akhil.goyal@nxp.com" <akhil.goyal@nxp.com>, "Vangati,
Narender" <narender.vangati@intel.com>, Honnappa Nagarahalli
<Honnappa.Nagarahalli@arm.com>, "Ananyev, Konstantin"
<konstantin.ananyev@intel.com>
Thread-Topic: [v2 1/2] cryptodev: support enqueue callback functions
Thread-Index: AQHWhsTNpueHJn2Z1E+jIJevuNB9zqlgfJJw
Date: Wed, 9 Sep 2020 16:21:04 +0000
Message-ID: <MWHPR11MB183855A77FDD31DD6F1FBE04E8260@MWHPR11MB1838.namprd11.prod.outlook.com>
References: <1599549024-195051-1-git-send-email-abhinandan.gujjar@intel.com>
In-Reply-To: <1599549024-195051-1-git-send-email-abhinandan.gujjar@intel.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.5.1.3
authentication-results: dpdk.org; dkim=none (message not signed)
header.d=none;dpdk.org; dmarc=none action=none header.from=intel.com;
x-originating-ip: [103.5.135.70]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 2b1853fe-02f0-4c4a-8b02-08d854dc59a2
x-ms-traffictypediagnostic: MWHPR1101MB2077:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <MWHPR1101MB2077C782C1E1B6A85C0E4E37E8260@MWHPR1101MB2077.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: bzjduPBY12jodqJCn4xwznPM/2b611fg2DP6Dlr3VB4oOXLJh/1tUN4m72ZHPDkvhYLvXwQlgyXwNKN0C2CDaJ9lj7V0ltLVT+DrRqPRFdsjr/l9HzrJ4TIsFaUwad3Pn/M2H750ErSBDZhuHRW5W+8DRLKEY6uTtV13FOVUuipPGhU9NzfWPcAXIcpm9GlTkSPUJzkU6vmQQYtZqboyyCmNBZOGn9Ct9IKbbvJZIZME8JgIjy7s81RUWjWjw94IPJKk4WwSjVkAQSNV34MDXFJX5ixum3ZJd4l2N3btyUIdXt08a0/VYGXqNs+K6Ni9CnxilG4dvQ7bCSIxRplDBw==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:MWHPR11MB1838.namprd11.prod.outlook.com; PTR:; CAT:NONE;
SFS:(4636009)(396003)(136003)(376002)(366004)(346002)(39860400002)(186003)(316002)(53546011)(26005)(55236004)(6506007)(4326008)(55016002)(9686003)(33656002)(6916009)(8676002)(2906002)(54906003)(107886003)(8936002)(7696005)(71200400001)(52536014)(66446008)(83380400001)(30864003)(86362001)(66476007)(66946007)(5660300002)(66556008)(478600001)(64756008)(76116006);
DIR:OUT; SFP:1102;
x-ms-exchange-antispam-messagedata: QC7xNhle5U/BV50cZf6WS4XHeOJtCmATUbkg+DAwVqDh6JtzQ45Oovnx6uHJ/MNM7mLWNhayxX12/8I76FRfDK7a0SO/6cHf6q+xTFAp8kxlaBi5EVYcXENF1yjIFiPasQMof1oyXJCiT7Nu7tODYQT6HwTKGqvNBcvdGM/FYbT+gBfze+Wze4lvtFcPFsyWLjwfLW3e2ePZQ7eFsvtMIQabqaOqnGYB4Pa692i4U5kdP287uXvUQS860LSU5dt8DFcpjo/eG3CrH1RCxPRL2xIOZ3+LXTqXhAuesLmlYIl+XVgUWaj2pejs2AmeV25qfOSQI/2eL0Oor9I4jctqXZtzwVIGM8ovv2fatMe1x5tyxALH4HVe+gzyG4tLcnWxTHFapRkBCrWNR6buKUIHbu6+/Y/3OneX17cV3SdWH6pyyLklEjuV0lRuHaOzTOBQpC5sWJqSWFtw1BDsHOfSzFQ4sqvn/BF4xbtLmSXx9ebOEhEEdk2cFcluVHLuAKcb2xeA6Y0175E+KkKtyNWxhP18j75sOpgzExBtSyD/86a+YHfEF5l07gIPkz26GWc8FvjteVhKTYf2tqFg6nh3DAI0ItsWK+FihZhTdABUw7K+mN3KJozk8nPup8H0V+3jImH7EjrNVrDeeHjtqfGevA==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1838.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2b1853fe-02f0-4c4a-8b02-08d854dc59a2
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Sep 2020 16:21:04.3669 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: mPmGZ095ZxZiy1cEX1L4mt4tyDF0qe/hucvSe/v4krbQyuWN7TQ1tMJLZTxEwp10djzIxtTt/Tf4Wo9FdSHFpMUaa+SVu+tEzPw859+yNYM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1101MB2077
X-OriginatorOrg: intel.com
Subject: Re: [dpdk-dev] [v2 1/2] cryptodev: support enqueue callback
functions
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
+Konstantin & Honnappa
> -----Original Message-----
> From: Gujjar, Abhinandan S <abhinandan.gujjar@intel.com>
> Sent: Tuesday, September 8, 2020 12:40 PM
> To: dev@dpdk.org
> Cc: Doherty, Declan <declan.doherty@intel.com>; jerinj@marvell.com;
> akhil.goyal@nxp.com; Vangati, Narender <narender.vangati@intel.com>;
> Gujjar, Abhinandan S <abhinandan.gujjar@intel.com>
> Subject: [v2 1/2] cryptodev: support enqueue callback functions
>=20
> In an eventdev world, multiple workers (with ordered queue) will be worki=
ng
> on IPsec ESP processing. The ESP header's sequence number is unique and
> has to be sequentially incremented in an orderly manner.
> This rises a need for incrementing sequence number in crypto stage
> especially in event crypto adapter. By adding a user callback to cryptode=
v at
> enqueue burst, the user callback will get executed in the context of even=
t
> crypto adapter. This helps the application to increment the ESP sequence
> number atomically and orderly manner.
>=20
> This patch adds APIs to add/remove callback functions. The callback funct=
ion
> will be called for each burst of crypto ops received on a given crypto de=
vice
> queue pair.
>=20
> v1->v2:
> Moved callback related members to the end of cryptodev struct Added
> support for RCU
>=20
> Signed-off-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
> ---
> config/common_base | 1 +
> lib/librte_cryptodev/Makefile | 2 +-
> lib/librte_cryptodev/rte_cryptodev.c | 157
> +++++++++++++++++++++++++
> lib/librte_cryptodev/rte_cryptodev.h | 154
> +++++++++++++++++++++++-
> lib/librte_cryptodev/rte_cryptodev_version.map | 6 +
> 5 files changed, 318 insertions(+), 2 deletions(-)
>=20
> diff --git a/config/common_base b/config/common_base index
> fbf0ee7..f5ebde4 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -599,6 +599,7 @@
> CONFIG_RTE_LIBRTE_PMD_BBDEV_FPGA_5GNR_FEC=3Dy
> #
> CONFIG_RTE_LIBRTE_CRYPTODEV=3Dy
> CONFIG_RTE_CRYPTO_MAX_DEVS=3D64
> +CONFIG_RTE_CRYPTODEV_CALLBACKS=3Dy
>=20
> #
> # Compile PMD for ARMv8 Crypto device
> diff --git a/lib/librte_cryptodev/Makefile b/lib/librte_cryptodev/Makefil=
e
> index 73e77a2..514d552 100644
> --- a/lib/librte_cryptodev/Makefile
> +++ b/lib/librte_cryptodev/Makefile
> @@ -10,7 +10,7 @@ LIB =3D librte_cryptodev.a CFLAGS +=3D -O3 CFLAGS +=
=3D
> $(WERROR_FLAGS) LDLIBS +=3D -lrte_eal -lrte_mempool -lrte_ring -lrte_mbu=
f
> -LDLIBS +=3D -lrte_kvargs
> +LDLIBS +=3D -lrte_kvargs -lrte_rcu
>=20
> # library source files
> SRCS-y +=3D rte_cryptodev.c rte_cryptodev_pmd.c cryptodev_trace_points.c
> diff --git a/lib/librte_cryptodev/rte_cryptodev.c
> b/lib/librte_cryptodev/rte_cryptodev.c
> index 1dd795b..2fb3e35 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.c
> +++ b/lib/librte_cryptodev/rte_cryptodev.c
> @@ -38,6 +38,7 @@
> #include <rte_string_fns.h>
> #include <rte_compat.h>
> #include <rte_function_versioning.h>
> +#include <rte_rcu_qsbr.h>
>=20
> #include "rte_crypto.h"
> #include "rte_cryptodev.h"
> @@ -499,6 +500,10 @@ struct
> rte_cryptodev_sym_session_pool_private_data {
> return 0;
> }
>=20
> +#ifdef RTE_CRYPTODEV_CALLBACKS
> +/* spinlock for crypto device enq callbacks */ static rte_spinlock_t
> +rte_cryptodev_enq_cb_lock =3D RTE_SPINLOCK_INITIALIZER; #endif
>=20
> const char *
> rte_cryptodev_get_feature_name(uint64_t flag) @@ -1449,6 +1454,158
> @@ struct rte_cryptodev *
> rte_spinlock_unlock(&rte_cryptodev_cb_lock);
> }
>=20
> +#ifdef RTE_CRYPTODEV_CALLBACKS
> +int
> +rte_cryptodev_rcu_qsbr_add(uint8_t dev_id, struct rte_rcu_qsbr *qsbr) {
> +
> + struct rte_cryptodev *dev;
> +
> + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> + CDEV_LOG_ERR("Invalid dev_id=3D%" PRIu8, dev_id);
> + return -EINVAL;
> + }
> +
> + dev =3D &rte_crypto_devices[dev_id];
> + dev->qsbr =3D qsbr;
> + return 0;
> +}
> +
> +struct rte_cryptodev_enq_callback *
> +rte_cryptodev_add_enq_callback(uint8_t dev_id,
> + uint16_t qp_id,
> + rte_cryptodev_enq_cb_fn cb_fn,
> + void *cb_arg)
> +{
> + struct rte_cryptodev *dev;
> + struct rte_cryptodev_enq_callback *cb, *tail;
> +
> + if (!cb_fn)
> + return NULL;
> +
> + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> + CDEV_LOG_ERR("Invalid dev_id=3D%" PRIu8, dev_id);
> + return NULL;
> + }
> +
> + dev =3D &rte_crypto_devices[dev_id];
> + if (qp_id >=3D dev->data->nb_queue_pairs) {
> + CDEV_LOG_ERR("Invalid queue_pair_id=3D%d", qp_id);
> + return NULL;
> + }
> +
> + cb =3D rte_zmalloc(NULL, sizeof(*cb), 0);
> + if (cb =3D=3D NULL) {
> + CDEV_LOG_ERR("Failed to allocate memory for callback on "
> + "dev=3D%d, queue_pair_id=3D%d", dev_id, qp_id);
> + rte_errno =3D ENOMEM;
> + return NULL;
> + }
> +
> + cb->fn =3D cb_fn;
> + cb->arg =3D cb_arg;
> +
> + rte_spinlock_lock(&rte_cryptodev_enq_cb_lock);
> + if (dev->enq_cbs =3D=3D NULL) {
> + dev->enq_cbs =3D rte_zmalloc(NULL, sizeof(cb) *
> + dev->data->nb_queue_pairs, 0);
> + if (dev->enq_cbs =3D=3D NULL) {
> + CDEV_LOG_ERR("Failed to allocate memory for
> callbacks");
> + rte_errno =3D ENOMEM;
> + rte_free(cb);
> + return NULL;
> + }
> + }
> +
> + /* Add the callbacks in fifo order. */
> + tail =3D dev->enq_cbs[qp_id];
> + if (tail) {
> + while (tail->next)
> + tail =3D tail->next;
> + tail->next =3D cb;
> + } else
> + dev->enq_cbs[qp_id] =3D cb;
> +
> + rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> +
> + return cb;
> +}
> +
> +int
> +rte_cryptodev_remove_enq_callback(uint8_t dev_id,
> + uint16_t qp_id,
> + struct rte_cryptodev_enq_callback *cb) {
> + struct rte_cryptodev *dev;
> + struct rte_cryptodev_enq_callback **prev_cb, *curr_cb;
> + uint16_t qp;
> + int free_mem;
> + int ret;
> +
> + free_mem =3D 1;
> +
> + if (!cb) {
> + CDEV_LOG_ERR("cb is NULL");
> + return -EINVAL;
> + }
> +
> + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> + CDEV_LOG_ERR("Invalid dev_id=3D%" PRIu8, dev_id);
> + return -EINVAL;
> + }
> +
> + dev =3D &rte_crypto_devices[dev_id];
> + if (qp_id >=3D dev->data->nb_queue_pairs) {
> + CDEV_LOG_ERR("Invalid queue_pair_id=3D%d", qp_id);
> + return -EINVAL;
> + }
> +
> + if (!dev->qsbr) {
> + CDEV_LOG_ERR("Rcu qsbr is NULL");
> + return -EINVAL;
> + }
> +
> + rte_spinlock_lock(&rte_cryptodev_enq_cb_lock);
> + if (dev->enq_cbs =3D=3D NULL) {
> + rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> + return -EINVAL;
> + }
> +
> + prev_cb =3D &dev->enq_cbs[qp_id];
> + for (; *prev_cb !=3D NULL; prev_cb =3D &curr_cb->next) {
> + curr_cb =3D *prev_cb;
> + if (curr_cb =3D=3D cb) {
> + /* Remove the user cb from the callback list. */
> + *prev_cb =3D curr_cb->next;
> + ret =3D 0;
> + break;
> + }
> + }
> +
> + if (!ret) {
> + /* Call sync with invalid thread id as this is part of
> + * control plane API */
> + rte_rcu_qsbr_synchronize(dev->qsbr,
> + RTE_QSBR_THRID_INVALID);
> + rte_free(cb);
> + }
> +
> + for (qp =3D 0; qp < dev->data->nb_queue_pairs; qp++)
> + if (dev->enq_cbs[qp] !=3D NULL) {
> + free_mem =3D 0;
> + break;
> + }
> +
> + if (free_mem) {
> + rte_free(dev->enq_cbs);
> + dev->enq_cbs =3D NULL;
> + }
> +
> + rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> +
> + return ret;
> +}
> +#endif
>=20
> int
> rte_cryptodev_sym_session_init(uint8_t dev_id, diff --git
> a/lib/librte_cryptodev/rte_cryptodev.h
> b/lib/librte_cryptodev/rte_cryptodev.h
> index 7b3ebc2..2c7a47b 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.h
> +++ b/lib/librte_cryptodev/rte_cryptodev.h
> @@ -530,6 +530,32 @@ struct rte_cryptodev_qp_conf { };
>=20
> /**
> + * Function type used for pre processing crypto ops when enqueue burst
> +is
> + * called.
> + *
> + * The callback function is called on enqueue burst immediately
> + * before the crypto ops are put onto the hardware queue for processing.
> + *
> + * @param dev_id The identifier of the device.
> + * @param qp_id The index of the queue pair in which ops are
> + * to be enqueued for processing. The value
> + * must be in the range [0, nb_queue_pairs - 1]
> + * previously supplied to
> + * *rte_cryptodev_configure*.
> + * @param ops The address of an array of *nb_ops* pointers
> + * to *rte_crypto_op* structures which contain
> + * the crypto operations to be processed.
> + * @param nb_ops The number of operations to process.
> + * @param user_param The arbitrary user parameter passed in by the
> + * application when the callback was originally
> + * registered.
> + * @return The number of ops to be enqueued to the
> + * crypto device.
> + */
> +typedef uint16_t (*rte_cryptodev_enq_cb_fn)(uint16_t dev_id, uint16_t
> qp_id,
> + struct rte_crypto_op **ops, uint16_t nb_ops, void
> *user_param);
> +
> +/**
> * Typedef for application callback function to be registered by applica=
tion
> * software for notification of device events
> *
> @@ -853,7 +879,6 @@ struct rte_cryptodev_config {
> enum rte_cryptodev_event_type event,
> rte_cryptodev_cb_fn cb_fn, void *cb_arg);
>=20
> -
> typedef uint16_t (*dequeue_pkt_burst_t)(void *qp,
> struct rte_crypto_op **ops, uint16_t nb_ops);
> /**< Dequeue processed packets from queue pair of a device. */ @@ -870,6
> +895,17 @@ typedef uint16_t (*enqueue_pkt_burst_t)(void *qp,
> /** Structure to keep track of registered callbacks */
> TAILQ_HEAD(rte_cryptodev_cb_list, rte_cryptodev_callback);
>=20
> +/**
> + * @internal
> + * Structure used to hold information about the callbacks to be called
> +for a
> + * queue pair on enqueue.
> + */
> +struct rte_cryptodev_enq_callback {
> + struct rte_cryptodev_enq_callback *next;
> + rte_cryptodev_enq_cb_fn fn;
> + void *arg;
> +};
> +
> /** The data structure associated with each crypto device. */ struct
> rte_cryptodev {
> dequeue_pkt_burst_t dequeue_burst;
> @@ -898,6 +934,14 @@ struct rte_cryptodev {
> __extension__
> uint8_t attached : 1;
> /**< Flag indicating the device is attached */
> +
> +#ifdef RTE_CRYPTODEV_CALLBACKS
> + struct rte_cryptodev_enq_callback **enq_cbs;
> + /**< User application callback for pre enqueue processing */
> +
> + struct rte_rcu_qsbr *qsbr;
> + /** < RCU QSBR variable for rte_cryptodev_enq_callback */ #endif
> } __rte_cache_aligned;
>=20
> void *
> @@ -1019,6 +1063,18 @@ struct rte_cryptodev_data {
> struct rte_crypto_op **ops, uint16_t nb_ops) {
> struct rte_cryptodev *dev =3D &rte_cryptodevs[dev_id];
> +#ifdef RTE_CRYPTODEV_CALLBACKS
> + if (unlikely(dev->enq_cbs !=3D NULL && dev->enq_cbs[qp_id] !=3D
> NULL)) {
> + struct rte_cryptodev_enq_callback *cb =3D
> + dev->enq_cbs[qp_id];
> +
> + do {
> + nb_ops =3D cb->fn(dev_id, qp_id, ops, nb_ops,
> + cb->arg);
> + cb =3D cb->next;
> + } while (cb !=3D NULL);
> + }
> +#endif
>=20
> rte_cryptodev_trace_enqueue_burst(dev_id, qp_id, (void **)ops,
> nb_ops);
> return (*dev->enqueue_burst)(
> @@ -1351,6 +1407,102 @@ struct rte_cryptodev_asym_session *
> struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs
> ofs,
> struct rte_crypto_sym_vec *vec);
>=20
> +#ifdef RTE_CRYPTODEV_CALLBACKS
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Add a user callback for a given crypto device and queue pair which
> +will be
> + * called on crypto ops enqueue.
> + *
> + * This API configures a function to be called for each burst of crypto
> +ops
> + * received on a given crypto device queue pair. The return value is a
> +pointer
> + * that can be used later to remove the callback using
> + * rte_cryptodev_remove_enq_callback().
> + *
> + * Multiple functions are called in the order that they are added.
> + *
> + * @param dev_id The identifier of the device.
> + * @param qp_id The index of the queue pair in which ops are
> + * to be enqueued for processing. The value
> + * must be in the range [0, nb_queue_pairs - 1]
> + * previously supplied to
> + * *rte_cryptodev_configure*.
> + * @param cb_fn The callback function
> + * @param cb_arg A generic pointer parameter which will be
> passed
> + * to each invocation of the callback function on
> + * this crypto device and queue pair.
> + *
> + * @return
> + * NULL on error.
> + * On success, a pointer value which can later be used to remove the
> callback.
> + */
> +
> +__rte_experimental
> +struct rte_cryptodev_enq_callback *
> +rte_cryptodev_add_enq_callback(uint8_t dev_id,
> + uint16_t qp_id,
> + rte_cryptodev_enq_cb_fn cb_fn,
> + void *cb_arg);
> +
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Remove a user callback function for given crypto device and queue pai=
r.
> + *
> + * This function is used to removed callbacks that were added to a
> +crypto
> + * device queue pair using rte_cryptodev_add_enq_callback().
> + *
> + * Note: The callback expects a RCU QSBR to be configured to
> +synchronize
> + * to free the memory. Application is expected to configure RCU QSBR
> +after
> + * adding an enqueue callback.
> + *
> + *
> + * @param dev_id The identifier of the device.
> + * @param qp_id The index of the queue pair in which ops are
> + * to be enqueued for processing. The value
> + * must be in the range [0, nb_queue_pairs - 1]
> + * previously supplied to
> + * *rte_cryptodev_configure*.
> + * @param cb Pointer to user supplied callback created via
> + * rte_cryptodev_add_enq_callback().
> + *
> + * @return
> + * - 0: Success. Callback was removed.
> + * - -EINVAL: The dev_id or the qp_id is out of range, or the callbac=
k
> + * is NULL or not found for the crypto device queue pair.
> + */
> +
> +__rte_experimental
> +int rte_cryptodev_remove_enq_callback(uint8_t dev_id,
> + uint16_t qp_id,
> + struct rte_cryptodev_enq_callback *cb);
> +
> +
> +/**
> + * @warning
> + * @b EXPERIMENTAL: this API may change without prior notice
> + *
> + * Associate RCU QSBR variable with a cryptodev.
> + *
> + * This function is used to add RCU QSBR to a crypto device.
> + * The purpose of RCU is to help multiple threads to synchronize
> + * with each other before initiating adding/removing callback
> + * while dataplane threads are running enqueue callbacks.
> + *
> + * @param dev_id The identifier of the device.
> + * @param qsr RCU QSBR configuration
> + * @return
> + * On success - 0
> + * On error - EINVAL.
> + */
> +
> +__rte_experimental
> +int rte_cryptodev_rcu_qsbr_add(uint8_t dev_id, struct rte_rcu_qsbr
> +*qsbr); #endif
> +
> #ifdef __cplusplus
> }
> #endif
> diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map
> b/lib/librte_cryptodev/rte_cryptodev_version.map
> index 02f6dcf..46de3ca 100644
> --- a/lib/librte_cryptodev/rte_cryptodev_version.map
> +++ b/lib/librte_cryptodev/rte_cryptodev_version.map
> @@ -64,6 +64,7 @@ DPDK_20.0 {
> rte_cryptodev_sym_capability_get;
> };
>=20
> +
> EXPERIMENTAL {
> global:
>=20
> @@ -105,4 +106,9 @@ EXPERIMENTAL {
>=20
> # added in 20.08
> rte_cryptodev_get_qp_status;
> +
> + # added in 20.11
> + rte_cryptodev_add_enq_callback;
> + rte_cryptodev_remove_enq_callback;
> + rte_cryptodev_rcu_qsbr_add;
> };
> --
> 1.9.1