From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id DD359A00C2;
	Fri, 24 Apr 2020 17:10:58 +0200 (CEST)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 85C911C1C9;
	Fri, 24 Apr 2020 17:10:57 +0200 (CEST)
Received: from mga14.intel.com (mga14.intel.com [192.55.52.115])
 by dpdk.org (Postfix) with ESMTP id F20B82C6D
 for <dev@dpdk.org>; Fri, 24 Apr 2020 17:10:55 +0200 (CEST)
IronPort-SDR: hwPIHBY7HHSbCIsb+fRhx9RAhgSR1KY/6W7htU1VkQof7Vz2hVI9Z7KTx2oCZNwwLQpTpIC6Bs
 WhoikVKA5W8w==
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga005.fm.intel.com ([10.253.24.32])
 by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384;
 24 Apr 2020 08:10:51 -0700
IronPort-SDR: kjoY4OUC6yk+GL2k+IHcF8Y7avEvrHsiEPGKoEjNej56Vt3tcHm5F1rFWhMNyRtQhxEUTfjJgr
 jr+EF9LT3/SA==
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.73,311,1583222400"; d="scan'208";a="457371894"
Received: from orsmsx105.amr.corp.intel.com ([10.22.225.132])
 by fmsmga005.fm.intel.com with ESMTP; 24 Apr 2020 08:10:25 -0700
Received: from orsmsx111.amr.corp.intel.com (10.22.240.12) by
 ORSMSX105.amr.corp.intel.com (10.22.225.132) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Fri, 24 Apr 2020 08:10:25 -0700
Received: from ORSEDG001.ED.cps.intel.com (10.7.248.4) by
 ORSMSX111.amr.corp.intel.com (10.22.240.12) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Fri, 24 Apr 2020 08:10:25 -0700
Received: from NAM10-BN7-obe.outbound.protection.outlook.com (104.47.70.103)
 by edgegateway.intel.com (134.134.137.100) with Microsoft SMTP Server (TLS)
 id 14.3.439.0; Fri, 24 Apr 2020 08:10:25 -0700
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
 b=gyRyC0lshzZk0EwkO0l3DHFREFyjXsWCh499uHT9o8ypq6A7PTd0xyQTkLcttMi+AQjCE1lmK/uRNDTZHM1BeBN9J6UW1JRevSellOeu8W0p/QtTuyjprb8hJJBf7yBn2thgXHIbnBIbEdjej1l17sh3JxW4KQQUS95nuZinCA+ZNQnLnUvex3PhugmRXjwPIzqEwMJmtINHisnYK/H7UFrIFG1yUHo+8HvjJRf5YClXJt/XPzm06fv718M2iWRf3mty7ep+twmBbC/G4tO4OZcunKdUIBz/jwrYUvEzTkGvRzJjujYTQp4mxGJPBlANIth5oijkNYeS++7fs32Bjg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; 
 s=arcselector9901;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PH6jJMjrCSvctfGaNz55N/SMfD8Rw/WKU1ocupXgzPU=;
 b=EyNI1cGQN0r8giKSA56rQ+/4FvqFRSEZU/1ZhLV+iO7792wIjnVr4uGr5TEWRz4t8gNoIs/gR5i1LsmQXHK4ZUhmXy+PH6WYo1R9R9Bbp89hg1tclWV0+AYz+FOCrfJNfdaKewVrsPEEX5+h0bfB2ZZ45WqG0P08vsQJ5W0P/Ra60khEoon5jUSKiuNi3LsGj66QBRuIm7KL+Kpe7upp/9zGNFeZ/wU5G6uGNIAS3VopBdmFrpUKG/hfLLvureAhlaYbU8po+T+/knxrUbRT0xTRMm9tE6ZDKcMB1JV7pvINPj98ym2ZJ52hLUqyw6FATlTK7idqYNXzNckWqAp2qA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
 smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com;
 dkim=pass header.d=intel.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; 
 s=selector2-intel-onmicrosoft-com;
 h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
 bh=PH6jJMjrCSvctfGaNz55N/SMfD8Rw/WKU1ocupXgzPU=;
 b=JAWmwWQ/eUJNTpzjIUGwH8ArPmGeFHnoUcl5mKJ7t6Kee9s8zCTac/8yUXLCzrQ6J81UYx4imvGTlbiAcRf7JoL1COTFZ2AiEpcLeuWguYCPdy4eZ6csALauG5bga8saBYpV3TjBWX1E35Bg/n9N8ZA/ZEhNf17yXuTirp2H5eY=
Received: from BYAPR11MB3301.namprd11.prod.outlook.com (2603:10b6:a03:7f::26)
 by BYAPR11MB3014.namprd11.prod.outlook.com (2603:10b6:a03:91::17)
 with Microsoft SMTP Server (version=TLS1_2,
 cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2937.13; Fri, 24 Apr
 2020 15:10:21 +0000
Received: from BYAPR11MB3301.namprd11.prod.outlook.com
 ([fe80::f8cb:58cd:e958:fff4]) by BYAPR11MB3301.namprd11.prod.outlook.com
 ([fe80::f8cb:58cd:e958:fff4%6]) with mapi id 15.20.2937.020; Fri, 24 Apr 2020
 15:10:21 +0000
From: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>
To: "Gujjar, Abhinandan S" <abhinandan.gujjar@intel.com>, "Doherty, Declan"
 <declan.doherty@intel.com>, "jerinj@marvell.com" <jerinj@marvell.com>,
 "akhil.goyal@nxp.com" <akhil.goyal@nxp.com>, "dev@dpdk.org" <dev@dpdk.org>
CC: "Vangati, Narender" <narender.vangati@intel.com>, "Gujjar, Abhinandan S"
 <abhinandan.gujjar@intel.com>
Thread-Topic: [dpdk-dev] [PATCH] cryptodev: add support for user callback
 functions
Thread-Index: AQHWF6dnW1FIZPel1ki2JuBvaSroL6iIYciA
Date: Fri, 24 Apr 2020 15:10:21 +0000
Message-ID: <BYAPR11MB330154CB5020C6529DFDD8829AD00@BYAPR11MB3301.namprd11.prod.outlook.com>
References: <1587371610-19878-1-git-send-email-abhinandan.gujjar@intel.com>
In-Reply-To: <1587371610-19878-1-git-send-email-abhinandan.gujjar@intel.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.2.0.6
authentication-results: spf=none (sender IP is )
 smtp.mailfrom=konstantin.ananyev@intel.com; 
x-originating-ip: [192.198.151.170]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: f4a6e401-4f2e-46c2-c306-08d7e8619bca
x-ms-traffictypediagnostic: BYAPR11MB3014:
x-ld-processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <BYAPR11MB3014481CA224918B02BBB4909AD00@BYAPR11MB3014.namprd11.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-forefront-prvs: 03838E948C
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
 IPV:NLI; SFV:NSPM; H:BYAPR11MB3301.namprd11.prod.outlook.com; PTR:; CAT:NONE;
 SFTY:;
 SFS:(136003)(39860400002)(366004)(346002)(396003)(376002)(316002)(66556008)(2906002)(66446008)(64756008)(66476007)(55016002)(7696005)(186003)(6506007)(52536014)(478600001)(26005)(5660300002)(66946007)(8936002)(86362001)(81156014)(107886003)(9686003)(30864003)(54906003)(71200400001)(33656002)(4326008)(110136005)(76116006);
 DIR:OUT; SFP:1102; 
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: RklNhT0CgS1ow5eXqgZsvcuwee9o7kR/4Bn40BsHmQOAcg9itMLHB+FTX/jcdZyWhwVThfnl2uR4sw8jBeWwTfoscQ5Gq8KGGmH/VsQUVVFrW4EIvGjm4mNPCZkYkR0rxoFdxGWvSCGq928Fmsq81OFGOtfMUMCgbVZY9yvpvgE/5FGM8UVPAYE5dRswJOYkz9WBmH2xV4W8yUEw6rSYusoSzde8ZV9HFVYJQ6LClpn9+n6cuQKg7weXw597RSqqVKjzxYscAJ5cGJtOPzpJDjMujmb7bvXM3wHjyCjpVHqZFuNbkHC0ckWlxVx8r4UzEPJ+yOB4BEHFLZEXyhFYvjamZDdd7dC8vPcAe7GuQHS1fUBb2noTGuoWiH7q85dehW1QOKJGJVjYV/PttBZJmLGlceVYV3DB3634CHP371TJWFUWLXEOAdyuvTrMbwJQ
x-ms-exchange-antispam-messagedata: NX4AMSum7cpOuT0pRdp+5t2ewSkgGKA3V7GEOTGX/CFuHFtgabYLfjpLoCWEbQYdTLM8N7MUmv9d1seiOzElf3KeacHD8V7LUBzu64WM+ZDuCIwYLmyxV3oQVQSq0rpbmkVuPOpnINzAV/GJONkmGg==
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: f4a6e401-4f2e-46c2-c306-08d7e8619bca
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Apr 2020 15:10:21.8263 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: O6ZJzvUuWClZbjnHjEMLKQKoGzI3qB/DYd9BohMkVXcvRRv7i7w8W8sYXx+BiFWHLJ8Wg8VU0ATsmhhZ7X4sDWUIMwTggxUMCh6Wmev/8AM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB3014
X-OriginatorOrg: intel.com
Subject: Re: [dpdk-dev] [PATCH] cryptodev: add support for user
	callback	functions
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

=20
> In an eventdev world, multiple workers (with ordered queue) will be
> working on IPsec ESP processing. The ESP header's sequence number is
> unique and has to be sequentially incremented in an orderly manner.
> This rises a need for incrementing sequence number in crypto stage
> especially in event crypto adapter. By adding a user callback to
> cryptodev at enqueue burst, the user callback will get executed
> in the context of event crypto adapter. This helps the application
> to increment the ESP sequence number atomically and orderly manner.
>=20
> Signed-off-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
> ---
>  config/common_base                             |   1 +
>  lib/librte_cryptodev/rte_cryptodev.c           | 120 +++++++++++++++++++=
+++++
>  lib/librte_cryptodev/rte_cryptodev.h           | 125 +++++++++++++++++++=
+++++-
>  lib/librte_cryptodev/rte_cryptodev_version.map |   3 +
>  4 files changed, 248 insertions(+), 1 deletion(-)
>=20
> diff --git a/config/common_base b/config/common_base
> index 9ec689d..6f93acb 100644
> --- a/config/common_base
> +++ b/config/common_base
> @@ -586,6 +586,7 @@ CONFIG_RTE_LIBRTE_PMD_BBDEV_FPGA_5GNR_FEC=3Dy
>  #
>  CONFIG_RTE_LIBRTE_CRYPTODEV=3Dy
>  CONFIG_RTE_CRYPTO_MAX_DEVS=3D64
> +CONFIG_RTE_CRYPTODEV_CALLBACKS=3Dy
>=20
>  #
>  # Compile PMD for ARMv8 Crypto device
> diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/=
rte_cryptodev.c
> index 2849b2e..5a4cba9 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.c
> +++ b/lib/librte_cryptodev/rte_cryptodev.c
> @@ -56,6 +56,9 @@
>  /* spinlock for crypto device callbacks */
>  static rte_spinlock_t rte_cryptodev_cb_lock =3D RTE_SPINLOCK_INITIALIZER=
;
>=20
> +/* spinlock for crypto device enq callbacks */
> +static rte_spinlock_t rte_cryptodev_enq_cb_lock =3D RTE_SPINLOCK_INITIAL=
IZER;
> +
>=20
>  /**
>   * The user application callback description.
> @@ -1256,6 +1259,123 @@ struct rte_cryptodev *
>  	rte_spinlock_unlock(&rte_cryptodev_cb_lock);
>  }
>=20
> +const struct rte_cryptodev_enq_callback *__rte_experimental
> +rte_cryptodev_add_enq_callback(uint8_t dev_id,
> +			       uint16_t qp_id,
> +			       rte_cryptodev_enq_cb_fn cb_fn,
> +			       void *cb_arg)
> +{
> +	struct rte_cryptodev *dev;
> +	struct rte_cryptodev_enq_callback *cb, *tail;
> +
> +	if (!cb_fn)
> +		return NULL;
> +
> +	if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> +		CDEV_LOG_ERR("Invalid dev_id=3D%" PRIu8, dev_id);
> +		return NULL;
> +	}
> +
> +	dev =3D &rte_crypto_devices[dev_id];
> +	if (qp_id >=3D dev->data->nb_queue_pairs) {
> +		CDEV_LOG_ERR("Invalid queue_pair_id=3D%d", qp_id);
> +		return NULL;
> +	}
> +
> +	cb =3D rte_zmalloc(NULL, sizeof(*cb), 0);
> +	if (cb =3D=3D NULL) {
> +		CDEV_LOG_ERR("Failed to allocate memory for callback on "
> +			     "dev=3D%d, queue_pair_id=3D%d", dev_id, qp_id);
> +		rte_errno =3D ENOMEM;
> +		return NULL;
> +	}
> +
> +	cb->fn =3D cb_fn;
> +	cb->arg =3D cb_arg;
> +
> +	rte_spinlock_lock(&rte_cryptodev_enq_cb_lock);
> +	if (dev->enq_cbs =3D=3D NULL) {
> +		dev->enq_cbs =3D rte_zmalloc(NULL, sizeof(cb) *
> +					   dev->data->nb_queue_pairs, 0);
> +		if (dev->enq_cbs =3D=3D NULL) {
> +			CDEV_LOG_ERR("Failed to allocate memory for callbacks");
> +			rte_errno =3D ENOMEM;
> +			rte_free(cb);
> +			return NULL;
> +		}
> +	}
> +
> +	/* Add the callbacks in fifo order. */
> +	tail =3D dev->enq_cbs[qp_id];
> +	if (tail) {
> +		while (tail->next)
> +			tail =3D tail->next;
> +		tail->next =3D cb;
> +	} else
> +		dev->enq_cbs[qp_id] =3D cb;
> +
> +	rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> +
> +	return cb;
> +}
> +
> +int __rte_experimental
> +rte_cryptodev_remove_enq_callback(uint8_t dev_id,
> +				  uint16_t qp_id,
> +				  const struct rte_cryptodev_enq_callback *cb)
> +{
> +	struct rte_cryptodev *dev;
> +	struct rte_cryptodev_enq_callback **prev_cb, *curr_cb;
> +	uint16_t qp;
> +	int free_mem =3D 1;
> +	int ret =3D -EINVAL;
> +
> +	if (!cb)
> +		return ret;
> +
> +	if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) {
> +		CDEV_LOG_ERR("Invalid dev_id=3D%" PRIu8, dev_id);
> +		return ret;
> +	}
> +
> +	dev =3D &rte_crypto_devices[dev_id];
> +	if (qp_id >=3D dev->data->nb_queue_pairs) {
> +		CDEV_LOG_ERR("Invalid queue_pair_id=3D%d", qp_id);
> +		return ret;
> +	}
> +
> +	rte_spinlock_lock(&rte_cryptodev_enq_cb_lock);
> +	if (dev->enq_cbs =3D=3D NULL) {
> +		rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> +		return ret;
> +	}
> +
> +	prev_cb =3D &dev->enq_cbs[qp_id];
> +	for (; *prev_cb !=3D NULL; prev_cb =3D &curr_cb->next) {
> +		curr_cb =3D *prev_cb;
> +		if (curr_cb =3D=3D cb) {
> +			/* Remove the user cb from the callback list. */
> +			*prev_cb =3D curr_cb->next;
> +			ret =3D 0;
> +			break;
> +		}
> +	}
> +
> +	for (qp =3D 0; qp < dev->data->nb_queue_pairs; qp++)
> +		if (dev->enq_cbs[qp] !=3D NULL) {
> +			free_mem =3D 0;
> +			break;
> +		}
> +
> +	if (free_mem) {
> +		rte_free(dev->enq_cbs);
> +		dev->enq_cbs =3D NULL;
> +	}
> +
> +	rte_spinlock_unlock(&rte_cryptodev_enq_cb_lock);
> +
> +	return ret;
> +}

Pls don't re-implement pitfall we have with ethdev rx/tx callback:
right now with ethdev approach it is impossible to know when it is safe to =
free
removed CB and free used by CB resources if any.
Unless you do dev_stop() of course. =20
So majority of ethdev CB uses have to either leave CB allocated forever
after removal (memory leak), or invent some specific sync methods underneat=
h
that API.
We probably need to introduce some sync mechanism here straightway (RCU or =
so)
to avoid same issue.=20

>=20
>  int
>  rte_cryptodev_sym_session_init(uint8_t dev_id,
> diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/=
rte_cryptodev.h
> index f4846d2..2cf466b 100644
> --- a/lib/librte_cryptodev/rte_cryptodev.h
> +++ b/lib/librte_cryptodev/rte_cryptodev.h
> @@ -518,6 +518,32 @@ struct rte_cryptodev_qp_conf {
>  };
>=20
>  /**
> + * Function type used for pre processing crypto ops when enqueue burst i=
s
> + * called.
> + *
> + * The callback function is called on enqueue burst immediately
> + * before the crypto ops are put onto the hardware queue for processing.
> + *
> + * @param	dev_id		The identifier of the device.
> + * @param	qp_id		The index of the queue pair in which ops are
> + *				to be enqueued for processing. The value
> + *				must be in the range [0, nb_queue_pairs - 1]
> + *				previously supplied to
> + *				*rte_cryptodev_configure*.
> + * @param	ops		The address of an array of *nb_ops* pointers
> + *				to *rte_crypto_op* structures which contain
> + *				the crypto operations to be processed.
> + * @param	nb_ops		The number of operations to process.
> + * @param	user_param	The arbitrary user parameter passed in by the
> + *				application when the callback was originally
> + *				registered.
> + * @return			The number of ops to be enqueued to the
> + *				crypto device.
> + */
> +typedef uint16_t (*rte_cryptodev_enq_cb_fn)(uint16_t dev_id, uint16_t qp=
_id,
> +		struct rte_crypto_op **ops, uint16_t nb_ops, void *user_param);
> +
> +/**
>   * Typedef for application callback function to be registered by applica=
tion
>   * software for notification of device events
>   *
> @@ -800,7 +826,6 @@ struct rte_cryptodev_config {
>  		enum rte_cryptodev_event_type event,
>  		rte_cryptodev_cb_fn cb_fn, void *cb_arg);
>=20
> -
>  typedef uint16_t (*dequeue_pkt_burst_t)(void *qp,
>  		struct rte_crypto_op **ops,	uint16_t nb_ops);
>  /**< Dequeue processed packets from queue pair of a device. */
> @@ -817,6 +842,17 @@ typedef uint16_t (*enqueue_pkt_burst_t)(void *qp,
>  /** Structure to keep track of registered callbacks */
>  TAILQ_HEAD(rte_cryptodev_cb_list, rte_cryptodev_callback);
>=20
> +/**
> + * @internal
> + * Structure used to hold information about the callbacks to be called f=
or a
> + * queue pair on enqueue.
> + */
> +struct rte_cryptodev_enq_callback {
> +	struct rte_cryptodev_enq_callback *next;
> +	rte_cryptodev_enq_cb_fn fn;
> +	void *arg;
> +};
> +
>  /** The data structure associated with each crypto device. */
>  struct rte_cryptodev {
>  	dequeue_pkt_burst_t dequeue_burst;
> @@ -839,6 +875,9 @@ struct rte_cryptodev {
>  	struct rte_cryptodev_cb_list link_intr_cbs;
>  	/**< User application callback for interrupts if present */
>=20
> +	struct rte_cryptodev_enq_callback **enq_cbs;
> +	/**< User application callback for pre enqueue processing */
> +

Why not to put it at the very end of the structure?
Would be safer in terms of ABI breakage problem, etc.


>  	void *security_ctx;
>  	/**< Context for security ops */
>=20
> @@ -966,6 +1005,18 @@ struct rte_cryptodev_data {
>  {
>  	struct rte_cryptodev *dev =3D &rte_cryptodevs[dev_id];
>=20
> +#ifdef RTE_CRYPTODEV_CALLBACKS
> +	if (unlikely(dev->enq_cbs !=3D NULL && dev->enq_cbs[qp_id] !=3D NULL)) =
{
> +		struct rte_cryptodev_enq_callback *cb =3D
> +			dev->enq_cbs[qp_id];
> +
> +		do {
> +			nb_ops =3D cb->fn(dev_id, qp_id, ops, nb_ops,
> +					cb->arg);
> +			cb =3D cb->next;
> +		} while (cb !=3D NULL);
> +	}
> +#endif
>  	return (*dev->enqueue_burst)(
>  			dev->data->queue_pairs[qp_id], ops, nb_ops);
>  }
> @@ -1296,6 +1347,78 @@ struct rte_cryptodev_asym_session *
>  	struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs,
>  	struct rte_crypto_sym_vec *vec);
>=20
> +
> +/**
> + * Add a user callback for a given crypto device and queue pair which wi=
ll be
> + * called on crypto ops enqueue.
> + *
> + * This API configures a function to be called for each burst of crypto =
ops
> + * received on a given crypto device queue pair. The return value is a p=
ointer
> + * that can be used later to remove the callback using
> + * rte_cryptodev_remove_enq_callback().
> + *
> + * Multiple functions are called in the order that they are added.
> + *
> + * @param	dev_id		The identifier of the device.
> + * @param	qp_id		The index of the queue pair in which ops are
> + *				to be enqueued for processing. The value
> + *				must be in the range [0, nb_queue_pairs - 1]
> + *				previously supplied to
> + *				*rte_cryptodev_configure*.
> + * @param	cb_fn		The callback function
> + * @param	cb_arg		A generic pointer parameter which will be passed
> + *				to each invocation of the callback function on
> + *				this crypto device and queue pair.
> + *
> + * @return
> + *   NULL on error.
> + *   On success, a pointer value which can later be used to remove the c=
allback.
> + */
> +
> +const struct rte_cryptodev_enq_callback * __rte_experimental
> +rte_cryptodev_add_enq_callback(uint8_t dev_id,
> +			       uint16_t qp_id,
> +			       rte_cryptodev_enq_cb_fn cb_fn,
> +			       void *cb_arg);
> +
> +
> +/**
> + * Remove a user callback function for given crypto device and queue pai=
r.
> + *
> + * This function is used to removed callbacks that were added to a crypt=
o
> + * device queue pair using rte_cryptodev_add_enq_callback().
> + *
> + * Note: the callback is removed from the callback list but it isn't fre=
ed
> + * since the it may still be in use. The memory for the callback can be
> + * subsequently freed back by the application by calling rte_free().
> + *
> + * - Immediately - if the crypto device is stopped, or user knows that
> + *   no callbacks are in flight e.g. if called from the thread doing enq=
/deq
> + *   on that queue.
> + *
> + * - After a short delay - where the delay is sufficient to allow any
> + *   in-flight callbacks to complete.
> + *
> + * @param	dev_id		The identifier of the device.
> + * @param	qp_id		The index of the queue pair in which ops are
> + *				to be enqueued for processing. The value
> + *				must be in the range [0, nb_queue_pairs - 1]
> + *				previously supplied to
> + *				*rte_cryptodev_configure*.
> + * @param	cb		Pointer to user supplied callback created via
> + *				rte_cryptodev_add_enq_callback().
> + *
> + * @return
> + *   - 0: Success. Callback was removed.
> + *   - -EINVAL:  The dev_id or the qp_id is out of range, or the callbac=
k
> + *               is NULL or not found for the crypto device queue pair.
> + */
> +
> +int __rte_experimental
> +rte_cryptodev_remove_enq_callback(uint8_t dev_id,
> +				  uint16_t qp_id,
> +				  const struct rte_cryptodev_enq_callback *cb);
> +
>  #ifdef __cplusplus
>  }
>  #endif
> diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map b/lib/librte_=
cryptodev/rte_cryptodev_version.map
> index 6e41b4b..e8d3e77 100644
> --- a/lib/librte_cryptodev/rte_cryptodev_version.map
> +++ b/lib/librte_cryptodev/rte_cryptodev_version.map
> @@ -58,9 +58,11 @@ DPDK_20.0 {
>  	local: *;
>  };
>=20
> +
>  EXPERIMENTAL {
>  	global:
>=20
> +	rte_cryptodev_add_enq_callback;
>  	rte_cryptodev_asym_capability_get;
>  	rte_cryptodev_asym_get_header_session_size;
>  	rte_cryptodev_asym_get_private_session_size;
> @@ -71,6 +73,7 @@ EXPERIMENTAL {
>  	rte_cryptodev_asym_session_init;
>  	rte_cryptodev_asym_xform_capability_check_modlen;
>  	rte_cryptodev_asym_xform_capability_check_optype;
> +	rte_cryptodev_remove_enq_callback;
>  	rte_cryptodev_sym_cpu_crypto_process;
>  	rte_cryptodev_sym_get_existing_header_session_size;
>  	rte_cryptodev_sym_session_get_user_data;
> --
> 1.9.1