DPDK patches and discussions
 help / color / mirror / Atom feed
From: Fan Zhang <roy.fan.zhang@intel.com>
To: dev@dpdk.org
Cc: akhil.goyal@nxp.com, Fan Zhang <roy.fan.zhang@intel.com>,
	Piotr Bronowski <piotrx.bronowski@intel.com>,
	Adam Dybkowski <adamx.dybkowski@intel.com>
Subject: [dpdk-dev] [dpdk-dev v13 2/4] cryptodev: add raw crypto data-path APIs
Date: Sun, 11 Oct 2020 01:38:52 +0100
Message-ID: <20201011003854.54947-3-roy.fan.zhang@intel.com> (raw)
In-Reply-To: <20201011003854.54947-1-roy.fan.zhang@intel.com>

This patch adds raw data-path APIs for enqueue and dequeue
operations to cryptodev. The APIs support flexible user-define
enqueue and dequeue behaviors.

Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>
---
 doc/guides/cryptodevs/features/default.ini    |   1 +
 doc/guides/prog_guide/cryptodev_lib.rst       | 106 +++++
 doc/guides/rel_notes/release_20_11.rst        |   7 +
 lib/librte_cryptodev/rte_cryptodev.c          |  80 ++++
 lib/librte_cryptodev/rte_cryptodev.h          | 413 +++++++++++++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  51 ++-
 .../rte_cryptodev_version.map                 |  10 +
 7 files changed, 665 insertions(+), 3 deletions(-)

diff --git a/doc/guides/cryptodevs/features/default.ini b/doc/guides/cryptodevs/features/default.ini
index 133a246ee..17b177fc4 100644
--- a/doc/guides/cryptodevs/features/default.ini
+++ b/doc/guides/cryptodevs/features/default.ini
@@ -30,6 +30,7 @@ Asymmetric sessionless =
 CPU crypto             =
 Symmetric sessionless  =
 Non-Byte aligned data  =
+Sym raw data path API  =
 
 ;
 ; Supported crypto algorithms of a default crypto driver.
diff --git a/doc/guides/prog_guide/cryptodev_lib.rst b/doc/guides/prog_guide/cryptodev_lib.rst
index e7ba35c2d..bcf071326 100644
--- a/doc/guides/prog_guide/cryptodev_lib.rst
+++ b/doc/guides/prog_guide/cryptodev_lib.rst
@@ -632,6 +632,112 @@ a call argument. Status different than zero must be treated as error.
 For more details, e.g. how to convert an mbuf to an SGL, please refer to an
 example usage in the IPsec library implementation.
 
+Cryptodev Raw Data-path APIs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The Crypto Raw data-path APIs are a set of APIs designed to enable external
+libraries/applications to leverage the cryptographic processing provided by
+DPDK crypto PMDs through the cryptodev API but in a manner that is not
+dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc)
+in their data-path implementation.
+
+The raw data-path APIs have the following advantages:
+- External data structure friendly design. The new APIs uses the operation
+  descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and
+  IOVA addresses as input. Moreover, the APIs does not require the user to
+  allocate the descriptor from mempool, nor requiring mbufs to describe input
+  data's virtual and IOVA addresses. All these features made the translation
+  from user's own data structure into the descriptor easier and more efficient.
+- Flexible enqueue and dequeue operation. The raw data-path APIs gives the
+  user more control to the enqueue and dequeue operations, including the
+  capability of precious enqueue/dequeue count, abandoning enqueue or dequeue
+  at any time, and operation status translation and set on the fly.
+
+Cryptodev PMDs which support the raw data-path APIs will have
+``RTE_CRYPTODEV_FF_SYM_RAW_DP`` feature flag presented. To use this feature,
+the user shall create a local ``struct rte_crypto_raw_dp_ctx`` buffer and
+extend to at least the length returned by ``rte_cryptodev_get_raw_dp_ctx_size``
+function call. The created buffer is then initialized using
+``rte_cryptodev_configure_raw_dp_ctx`` function with the ``is_update``
+parameter as 0. The library and the crypto device driver will then set the
+buffer and attach either the cryptodev sym session, the rte_security session,
+or the cryptodev xform for session-less operation into the ctx buffer, and
+set the corresponding enqueue and dequeue function handlers based on the
+algorithm information stored in the session or xform. When the ``is_update``
+parameter passed into ``rte_cryptodev_configure_raw_dp_ctx`` is 1, the driver
+will not initialize the buffer but only update the session or xform and
+the function handlers accordingly.
+
+After the ``struct rte_crypto_raw_dp_ctx`` buffer is initialized, it is now
+ready for enqueue and dequeue operation. There are two different enqueue
+functions: ``rte_cryptodev_raw_enqueue`` to enqueue single raw data
+operation, and ``rte_cryptodev_raw_enqueue_burst`` to enqueue a descriptor
+with multiple operations. In case of the application uses similar approach to
+``struct rte_crypto_sym_vec`` to manage its data burst but with different
+data structure, using the ``rte_cryptodev_raw_enqueue_burst`` function may be
+less efficient as this is a situation where the application has to loop over
+all crypto operations to assemble the ``struct rte_crypto_sym_vec`` descriptor
+from its own data structure, and then the driver will loop over them again to
+translate every operation in the descriptor to the driver's specific queue data.
+The ``rte_cryptodev_raw_enqueue`` should be used to save one loop for each data
+burst instead.
+
+The ``rte_cryptodev_raw_enqueue`` and ``rte_cryptodev_raw_enqueue_burst``
+functions will return or set the enqueue status. ``rte_cryptodev_raw_enqueue``
+will return the status directly, ``rte_cryptodev_raw_enqueue_burst`` will
+return the number of operations enqueued or stored (explained as follows) and
+set the ``enqueue_status`` buffer provided by the user. The possible
+enqueue status values are:
+
+- ``1``: the operation(s) is/are enqueued successfully.
+- ``0``: the operation(s) is/are cached successfully in the crypto device queue
+  but is not actually enqueued. The user shall call
+  ``rte_cryptodev_raw_enqueue_done`` function after the expected operations
+  are stored. The crypto device will then start enqueuing all of them at
+  once.
+- The negative integer: error occurred during enqueue.
+
+Calling ``rte_cryptodev_configure_raw_dp_ctx`` with the parameter ``is_update``
+set as 0 twice without the enqueue function returning or setting enqueue status
+to 1 or ``rte_cryptodev_raw_enqueue_done`` function being called in between will
+invalidate any operation stored in the device queue but not enqueued. This
+feature is useful when the user wants to abandon partially enqueued operations
+for a failed enqueue burst operation and try enqueuing in a whole later.
+
+Similar as enqueue, there are two dequeue functions:
+``rte_cryptodev_raw_dequeue`` for dequeing single operation, and
+``rte_cryptodev_raw_dequeue_burst`` for dequeuing a burst of operations (e.g.
+all operations in a ``struct rte_crypto_sym_vec`` descriptor). The
+``rte_cryptodev_raw_dequeue_burst`` function allows the user to provide callback
+functions to retrieve dequeue count from the enqueued user data and write the
+expected status value to the user data on the fly. The dequeue functions also
+set the dequeue status:
+
+- ``1``: the operation(s) is/are dequeued successfully.
+- ``0``: the operation(s) is/are completed but is not actually dequeued (hence
+  still kept in the device queue). The user shall call the
+  ``rte_cryptodev_raw_dequeue_done`` function after the expected number of
+  operations (e.g. all operations in a descriptor) are dequeued. The crypto
+  device driver will then free them from the queue at once.
+- The negative integer: error occurred during dequeue.
+
+Calling ``rte_cryptodev_configure_raw_dp_ctx`` with the parameter ``is_update``
+set as 0 twice without the dequeue functions execution changed dequeue_status
+to 1 or ``rte_cryptodev_raw_dequeue_done`` function being called in between will
+revert the crypto device queue's dequeue effort to the moment when the
+``struct rte_crypto_raw_dp_ctx`` buffer is initialized. This feature is useful
+when the user wants to abandon partially dequeued data and try dequeuing again
+later in a whole.
+
+There are a few limitations to the raw data path APIs:
+
+* Only support in-place operations.
+* APIs are NOT thread-safe.
+* CANNOT mix the raw data-path API's enqueue with rte_cryptodev_enqueue_burst,
+  or vice versa.
+
+See *DPDK API Reference* for details on each API definitions.
+
 Sample code
 -----------
 
diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst
index 2973b2a33..85a07d86e 100644
--- a/doc/guides/rel_notes/release_20_11.rst
+++ b/doc/guides/rel_notes/release_20_11.rst
@@ -74,6 +74,13 @@ New Features
 
   * Added SR-IOV PF support
 
+* **Added raw data-path APIs for cryptodev library.**
+
+  Cryptodev is added with raw data-path APIs to accelerate external
+  libraries or applications which need to avail fast cryptodev
+  enqueue/dequeue operations but does not necessarily depends on
+  mbufs and cryptodev operation mempools.
+
 * **Updated the aesni_mb crypto PMD.**
 
   * Added support for AES-ECB 128, 192 and 256.
diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c
index cda160f61..3d95ac6ea 100644
--- a/lib/librte_cryptodev/rte_cryptodev.c
+++ b/lib/librte_cryptodev/rte_cryptodev.c
@@ -1772,6 +1772,86 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,
 	return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec);
 }
 
+int
+rte_cryptodev_get_raw_dp_ctx_size(uint8_t dev_id)
+{
+	struct rte_cryptodev *dev;
+	int32_t size = sizeof(struct rte_crypto_raw_dp_ctx);
+	int32_t priv_size;
+
+	if (!rte_cryptodev_pmd_is_valid_dev(dev_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+
+	if (*dev->dev_ops->sym_get_raw_dp_ctx_size == NULL ||
+		!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP)) {
+		return -ENOTSUP;
+	}
+
+	priv_size = (*dev->dev_ops->sym_get_raw_dp_ctx_size)(dev);
+	if (priv_size < 0)
+		return -ENOTSUP;
+
+	return RTE_ALIGN_CEIL((size + priv_size), 8);
+}
+
+int
+rte_cryptodev_configure_raw_dp_ctx(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	uint8_t is_update)
+{
+	struct rte_cryptodev *dev;
+
+	if (!rte_cryptodev_get_qp_status(dev_id, qp_id))
+		return -EINVAL;
+
+	dev = rte_cryptodev_pmd_get_dev(dev_id);
+	if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP)
+			|| dev->dev_ops->sym_configure_raw_dp_ctx == NULL)
+		return -ENOTSUP;
+
+	return (*dev->dev_ops->sym_configure_raw_dp_ctx)(dev, qp_id, ctx,
+			sess_type, session_ctx, is_update);
+}
+
+uint32_t
+rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **user_data, int *enqueue_status)
+{
+	return (*ctx->enqueue_burst)(ctx->qp_data, ctx->drv_ctx_data, vec,
+			ofs, user_data, enqueue_status);
+}
+
+int
+rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n)
+{
+	return (*ctx->enqueue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
+}
+
+uint32_t
+rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success_jobs, int *status)
+{
+	return (*ctx->dequeue_burst)(ctx->qp_data, ctx->drv_ctx_data,
+		get_dequeue_count, post_dequeue, out_user_data,
+		is_user_data_array, n_success_jobs, status);
+}
+
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n)
+{
+	return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n);
+}
+
 /** Initialise rte_crypto_op mempool element */
 static void
 rte_crypto_op_init(struct rte_mempool *mempool,
diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h
index 26abd0c52..b5db8fd81 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -458,7 +458,8 @@ rte_cryptodev_asym_get_xform_enum(enum rte_crypto_asym_xform_type *xform_enum,
 /**< Support symmetric session-less operations */
 #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA		(1ULL << 23)
 /**< Support operations on data which is not byte aligned */
-
+#define RTE_CRYPTODEV_FF_SYM_RAW_DP			(1ULL << 24)
+/**< Support accelerated specific symmetric raw data-path APIs */
 
 /**
  * Get the name of a crypto device feature flag
@@ -1319,6 +1320,416 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id,
 	struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs,
 	struct rte_crypto_sym_vec *vec);
 
+/**
+ * Get the size of the raw data-path context buffer.
+ *
+ * @param	dev_id		The device identifier.
+ *
+ * @return
+ *   - If the device supports raw data-path APIs, return the context size.
+ *   - If the device does not support the APIs, return -1.
+ */
+__rte_experimental
+int
+rte_cryptodev_get_raw_dp_ctx_size(uint8_t dev_id);
+
+/**
+ * Union of different crypto session types, including session-less xform
+ * pointer.
+ */
+union rte_cryptodev_session_ctx {
+	struct rte_cryptodev_sym_session *crypto_sess;
+	struct rte_crypto_sym_xform *xform;
+	struct rte_security_session *sec_sess;
+};
+
+/**
+ * Enqueue a vectorized operation descriptor into the device queue but the
+ * driver may or may not start processing until rte_cryptodev_raw_enqueue_done()
+ * is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	vec		Vectorized operation descriptor.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	user_data	The array of user data for dequeue later.
+ * @param	enqueue_status	Driver written value to specify the
+ *				enqueue status. Possible values:
+ *				- 1: The number of operations returned are
+ *				     enqueued successfully.
+ *				- 0: The number of operations returned are
+ *				     cached into the queue but are not processed
+ *				     until rte_cryptodev_raw_enqueue_done() is
+ *				     called.
+ *				- negative integer: Error occurred.
+ * @return
+ *   - The number of operations in the descriptor successfully enqueued or
+ *     cached into the queue but not enqueued yet, depends on the
+ *     "enqueue_status" value.
+ */
+typedef uint32_t (*cryptodev_sym_raw_enqueue_burst_t)(
+	void *qp, uint8_t *drv_ctx, struct rte_crypto_sym_vec *vec,
+	union rte_crypto_sym_ofs ofs, void *user_data[], int *enqueue_status);
+
+/**
+ * Enqueue single raw data vector into the device queue but the driver may or
+ * may not start processing until rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	data_vec	The buffer data vector.
+ * @param	n_data_vecs	Number of buffer data vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	iv		IV virtual and IOVA addresses
+ * @param	digest		digest virtual and IOVA addresses
+ * @param	aad_or_auth_iv	AAD or auth IV virtual and IOVA addresses,
+ *				depends on the algorithm used.
+ * @param	user_data	The user data.
+ * @return
+ *   - 1: The data vector is enqueued successfully.
+ *   - 0: The data vector is cached into the queue but is not processed
+ *        until rte_cryptodev_raw_enqueue_done() is called.
+ *   - negative integer: failure.
+ */
+typedef int (*cryptodev_sym_raw_enqueue_t)(
+	void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec,
+	uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs,
+	struct rte_crypto_va_iova_ptr *iv,
+	struct rte_crypto_va_iova_ptr *digest,
+	struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
+	void *user_data);
+
+/**
+ * Inform the cryptodev queue pair to start processing or finish dequeuing all
+ * enqueued/dequeued operations.
+ *
+ * @param	qp		Driver specific queue pair data.
+ * @param	drv_ctx		Driver specific context data.
+ * @param	n		The total number of processed operations.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_sym_raw_operation_done_t)(void *qp, uint8_t *drv_ctx,
+	uint32_t n);
+
+/**
+ * Typedef that the user provided for the driver to get the dequeue count.
+ * The function may return a fixed number or the number parsed from the user
+ * data stored in the first processed operation.
+ *
+ * @param	user_data	Dequeued user data.
+ * @return
+ *  - The number of operations to be dequeued.
+ **/
+typedef uint32_t (*rte_cryptodev_raw_get_dequeue_count_t)(void *user_data);
+
+/**
+ * Typedef that the user provided to deal with post dequeue operation, such
+ * as filling status.
+ *
+ * @param	user_data	Dequeued user data.
+ * @param	index		Index number of the processed descriptor.
+ * @param	is_op_success	Operation status provided by the driver.
+ **/
+typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data,
+	uint32_t index, uint8_t is_op_success);
+
+/**
+ * Dequeue a burst of symmetric crypto processing.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	drv_ctx			Driver specific context data.
+ * @param	get_dequeue_count	User provided callback function to
+ *					obtain dequeue operation count.
+ * @param	post_dequeue		User provided callback function to
+ *					post-process a dequeued operation.
+ * @param	out_user_data		User data pointer array to be retrieve
+ *					from device queue. In case of
+ *					*is_user_data_array* is set there
+ *					should be enough room to store all
+ *					user data.
+ * @param	is_user_data_array	Set 1 if every dequeued user data will
+ *					be written into out_user_data array.
+ *					Set 0 if only the first user data will
+ *					be written into out_user_data array.
+ * @param	n_success		Driver written value to specific the
+ *					total successful operations count.
+ * @param	dequeue_status		Driver written value to specify the
+ *					dequeue status. Possible values:
+ *					- 1: Successfully dequeued the number
+ *					     of operations returned. The user
+ *					     data previously set during enqueue
+ *					     is stored in the "out_user_data".
+ *					- 0: The number of operations returned
+ *					     are completed and the user data is
+ *					     stored in the "out_user_data", but
+ *					     they are not freed from the queue
+ *					     until
+ *					     rte_cryptodev_raw_dequeue_done()
+ *					     is called.
+ *					- negative integer: Error occurred.
+ * @return
+ *   - The number of operations dequeued or completed but not freed from the
+ *     queue, depends on "dequeue_status" value.
+ */
+typedef uint32_t (*cryptodev_sym_raw_dequeue_burst_t)(void *qp,
+	uint8_t *drv_ctx,
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success, int *dequeue_status);
+
+/**
+ * Dequeue a symmetric crypto processing.
+ *
+ * @param	qp			Driver specific queue pair data.
+ * @param	drv_ctx			Driver specific context data.
+ * @param	dequeue_status		Driver written value to specify the
+ *					dequeue status. Possible values:
+ *					- 1: Successfully dequeued a operation.
+ *					     The user data is returned.
+ *					- 0: The first operation in the queue
+ *					     is completed and the user data
+ *					     previously set during enqueue is
+ *					     returned, but it is not freed from
+ *					     the queue until
+ *					     rte_cryptodev_raw_dequeue_done() is
+ *					     called.
+ *					- negative integer: Error occurred.
+ * @param	op_status		Driver written value to specify
+ *					operation status.
+ * @return
+ *   - The user data pointer retrieved from device queue or NULL if no
+ *     operation is ready for dequeue.
+ */
+typedef void * (*cryptodev_sym_raw_dequeue_t)(
+		void *qp, uint8_t *drv_ctx, int *dequeue_status,
+		enum rte_crypto_op_status *op_status);
+
+/**
+ * Context data for raw data-path API crypto process. The buffer of this
+ * structure is to be allocated by the user application with the size equal
+ * or bigger than rte_cryptodev_get_raw_dp_ctx_size() returned value.
+ */
+struct rte_crypto_raw_dp_ctx {
+	void *qp_data;
+
+	cryptodev_sym_raw_enqueue_t enqueue;
+	cryptodev_sym_raw_enqueue_burst_t enqueue_burst;
+	cryptodev_sym_raw_operation_done_t enqueue_done;
+	cryptodev_sym_raw_dequeue_t dequeue;
+	cryptodev_sym_raw_dequeue_burst_t dequeue_burst;
+	cryptodev_sym_raw_operation_done_t dequeue_done;
+
+	/* Driver specific context data */
+	__extension__ uint8_t drv_ctx_data[];
+};
+
+/**
+ * Configure raw data-path context data.
+ *
+ * NOTE:
+ * After the context data is configured, the user should call
+ * rte_cryptodev_raw_attach_session() before using it in
+ * rte_cryptodev_raw_enqueue/dequeue function call.
+ *
+ * @param	dev_id		The device identifier.
+ * @param	qp_id		The index of the queue pair from which to
+ *				retrieve processed packets. The value must be
+ *				in the range [0, nb_queue_pair - 1] previously
+ *				supplied to rte_cryptodev_configure().
+ * @param	ctx		The raw data-path context data.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data.
+ * @param	is_update	Set 0 if it is to initialize the ctx.
+ *				Set 1 if ctx is initialized and only to update
+ *				session context data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_configure_raw_dp_ctx(uint8_t dev_id, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx,
+	uint8_t is_update);
+
+/**
+ * Enqueue a vectorized operation descriptor into the device queue but the
+ * driver may or may not start processing until rte_cryptodev_raw_enqueue_done()
+ * is called.
+ *
+ * @param	ctx		The initialized raw data-path context data.
+ * @param	vec		Vectorized operation descriptor.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	user_data	The array of user data for dequeue later.
+ * @param	enqueue_status	Driver written value to specify the
+ *				enqueue status. Possible values:
+ *				- 1: The number of operations returned are
+ *				     enqueued successfully.
+ *				- 0: The number of operations returned are
+ *				     cached into the queue but are not processed
+ *				     until rte_cryptodev_raw_enqueue_done() is
+ *				     called.
+ *				- negative integer: Error occurred.
+ * @return
+ *   - The number of operations in the descriptor successfully enqueued or
+ *     cached into the queue but not enqueued yet, depends on the
+ *     "enqueue_status" value.
+ */
+__rte_experimental
+uint32_t
+rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs,
+	void **user_data, int *enqueue_status);
+
+/**
+ * Enqueue single raw data vector into the device queue but the driver may or
+ * may not start processing until rte_cryptodev_raw_enqueue_done() is called.
+ *
+ * @param	ctx		The initialized raw data-path context data.
+ * @param	data_vec	The buffer data vector.
+ * @param	n_data_vecs	Number of buffer data vectors.
+ * @param	ofs		Start and stop offsets for auth and cipher
+ *				operations.
+ * @param	iv		IV virtual and IOVA addresses
+ * @param	digest		digest virtual and IOVA addresses
+ * @param	aad_or_auth_iv	AAD or auth IV virtual and IOVA addresses,
+ *				depends on the algorithm used.
+ * @param	user_data	The user data.
+ * @return
+ *   - 1: The data vector is enqueued successfully.
+ *   - 0: The data vector is cached into the queue but is not processed
+ *        until rte_cryptodev_raw_enqueue_done() is called.
+ *   - negative integer: failure.
+ */
+__rte_experimental
+static __rte_always_inline int
+rte_cryptodev_raw_enqueue(struct rte_crypto_raw_dp_ctx *ctx,
+	struct rte_crypto_vec *data_vec, uint16_t n_data_vecs,
+	union rte_crypto_sym_ofs ofs,
+	struct rte_crypto_va_iova_ptr *iv,
+	struct rte_crypto_va_iova_ptr *digest,
+	struct rte_crypto_va_iova_ptr *aad_or_auth_iv,
+	void *user_data)
+{
+	return (*ctx->enqueue)(ctx->qp_data, ctx->drv_ctx_data, data_vec,
+		n_data_vecs, ofs, iv, digest, aad_or_auth_iv, user_data);
+}
+
+/**
+ * Start processing all enqueued operations from last
+ * rte_cryptodev_configure_raw_dp_ctx() call.
+ *
+ * @param	ctx	The initialized raw data-path context data.
+ * @param	n	The number of operations cached.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n);
+
+/**
+ * Dequeue a burst of symmetric crypto processing.
+ *
+ * @param	ctx			The initialized raw data-path context
+ *					data.
+ * @param	get_dequeue_count	User provided callback function to
+ *					obtain dequeue operation count.
+ * @param	post_dequeue		User provided callback function to
+ *					post-process a dequeued operation.
+ * @param	out_user_data		User data pointer array to be retrieve
+ *					from device queue. In case of
+ *					*is_user_data_array* is set there
+ *					should be enough room to store all
+ *					user data.
+ * @param	is_user_data_array	Set 1 if every dequeued user data will
+ *					be written into out_user_data array.
+ *					Set 0 if only the first user data will
+ *					be written into out_user_data array.
+ * @param	n_success		Driver written value to specific the
+ *					total successful operations count.
+ * @param	dequeue_status		Driver written value to specify the
+ *					dequeue status. Possible values:
+ *					- 1: Successfully dequeued the number
+ *					     of operations returned. The user
+ *					     data previously set during enqueue
+ *					     is stored in the "out_user_data".
+ *					- 0: The number of operations returned
+ *					     are completed and the user data is
+ *					     stored in the "out_user_data", but
+ *					     they are not freed from the queue
+ *					     until
+ *					     rte_cryptodev_raw_dequeue_done()
+ *					     is called.
+ *					- negative integer: Error occurred.
+ * @return
+ *   - The number of operations dequeued or completed but not freed from the
+ *     queue, depends on "dequeue_status" value.
+ */
+__rte_experimental
+uint32_t
+rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx,
+	rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count,
+	rte_cryptodev_raw_post_dequeue_t post_dequeue,
+	void **out_user_data, uint8_t is_user_data_array,
+	uint32_t *n_success, int *dequeue_status);
+
+/**
+ * Dequeue a symmetric crypto processing.
+ *
+ * @param	ctx			The initialized raw data-path context
+ *					data.
+ * @param	dequeue_status		Driver written value to specify the
+ *					dequeue status. Possible values:
+ *					- 1: Successfully dequeued a operation.
+ *					     The user data is returned.
+ *					- 0: The first operation in the queue
+ *					     is completed and the user data
+ *					     previously set during enqueue is
+ *					     returned, but it is not freed from
+ *					     the queue until
+ *					     rte_cryptodev_raw_dequeue_done() is
+ *					     called.
+ *					- negative integer: Error occurred.
+ * @param	op_status		Driver written value to specify
+ *					operation status.
+ * @return
+ *   - The user data pointer retrieved from device queue or NULL if no
+ *     operation is ready for dequeue.
+ */
+__rte_experimental
+static __rte_always_inline void *
+rte_cryptodev_raw_dequeue(struct rte_crypto_raw_dp_ctx *ctx,
+		int *dequeue_status, enum rte_crypto_op_status *op_status)
+{
+	return (*ctx->dequeue)(ctx->qp_data, ctx->drv_ctx_data, dequeue_status,
+			op_status);
+}
+
+/**
+ * Inform the queue pair dequeue operations is finished.
+ *
+ * @param	ctx	The initialized raw data-path context data.
+ * @param	n	The number of operations.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+__rte_experimental
+int
+rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx,
+		uint32_t n);
+
 #ifdef __cplusplus
 }
 #endif
diff --git a/lib/librte_cryptodev/rte_cryptodev_pmd.h b/lib/librte_cryptodev/rte_cryptodev_pmd.h
index 1367222f7..9a8a7e632 100644
--- a/lib/librte_cryptodev/rte_cryptodev_pmd.h
+++ b/lib/librte_cryptodev/rte_cryptodev_pmd.h
@@ -317,6 +317,42 @@ typedef uint32_t (*cryptodev_sym_cpu_crypto_process_t)
 	(struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess,
 	union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec);
 
+/**
+ * Typedef that the driver provided to get service context private date size.
+ *
+ * @param	dev	Crypto device pointer.
+ *
+ * @return
+ *   - On success return the size of the device's service context private data.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_sym_get_raw_dp_ctx_size_t)(struct rte_cryptodev *dev);
+
+/**
+ * Typedef that the driver provided to configure raw data-path context.
+ *
+ * @param	dev		Crypto device pointer.
+ * @param	qp_id		Crypto device queue pair index.
+ * @param	service_type	Type of the service requested.
+ * @param	ctx		The raw data-path context data.
+ * @param	sess_type	session type.
+ * @param	session_ctx	Session context data. If NULL the driver
+ *				shall only configure the drv_ctx_data in
+ *				ctx buffer. Otherwise the driver shall only
+ *				parse the session_ctx to set appropriate
+ *				function pointers in ctx.
+ * @param	is_update	Set 0 if it is to initialize the ctx.
+ *				Set 1 if ctx is initialized and only to update
+ *				session context data.
+ * @return
+ *   - On success return 0.
+ *   - On failure return negative integer.
+ */
+typedef int (*cryptodev_sym_configure_raw_dp_ctx_t)(
+	struct rte_cryptodev *dev, uint16_t qp_id,
+	struct rte_crypto_raw_dp_ctx *ctx,
+	enum rte_crypto_op_sess_type sess_type,
+	union rte_cryptodev_session_ctx session_ctx, uint8_t is_update);
 
 /** Crypto device operations function pointer table */
 struct rte_cryptodev_ops {
@@ -349,8 +385,19 @@ struct rte_cryptodev_ops {
 	/**< Clear a Crypto sessions private data. */
 	cryptodev_asym_free_session_t asym_session_clear;
 	/**< Clear a Crypto sessions private data. */
-	cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
-	/**< process input data synchronously (cpu-crypto). */
+	union {
+		cryptodev_sym_cpu_crypto_process_t sym_cpu_process;
+		/**< process input data synchronously (cpu-crypto). */
+		__extension__
+		struct {
+			cryptodev_sym_get_raw_dp_ctx_size_t
+				sym_get_raw_dp_ctx_size;
+			/**< Get raw data path service context data size. */
+			cryptodev_sym_configure_raw_dp_ctx_t
+				sym_configure_raw_dp_ctx;
+			/**< Initialize raw data path context data. */
+		};
+	};
 };
 
 
diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map b/lib/librte_cryptodev/rte_cryptodev_version.map
index 7727286ac..7e4360ff0 100644
--- a/lib/librte_cryptodev/rte_cryptodev_version.map
+++ b/lib/librte_cryptodev/rte_cryptodev_version.map
@@ -99,4 +99,14 @@ EXPERIMENTAL {
 
 	# added in 20.08
 	rte_cryptodev_get_qp_status;
+
+	# added in 20.11
+	rte_cryptodev_configure_raw_dp_ctx;
+	rte_cryptodev_get_raw_dp_ctx_size;
+	rte_cryptodev_raw_dequeue;
+	rte_cryptodev_raw_dequeue_burst;
+	rte_cryptodev_raw_dequeue_done;
+	rte_cryptodev_raw_enqueue;
+	rte_cryptodev_raw_enqueue_burst;
+	rte_cryptodev_raw_enqueue_done;
 };
-- 
2.20.1


  parent reply	other threads:[~2020-10-11  0:39 UTC|newest]

Thread overview: 84+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-18 16:28 [dpdk-dev] [dpdk-dev v6 0/4] cryptodev: add data-path service APIs Fan Zhang
2020-08-18 16:28 ` [dpdk-dev] [dpdk-dev v6 1/4] cryptodev: add crypto " Fan Zhang
2020-08-18 16:28 ` [dpdk-dev] [dpdk-dev v6 2/4] crypto/qat: add crypto data-path service API support Fan Zhang
2020-08-18 16:28 ` [dpdk-dev] [dpdk-dev v6 3/4] test/crypto: add unit-test for cryptodev direct APIs Fan Zhang
2020-08-18 16:28 ` [dpdk-dev] [dpdk-dev v6 4/4] doc: add cryptodev service APIs guide Fan Zhang
2020-08-28 12:58 ` [dpdk-dev] [dpdk-dev v7 0/4] cryptodev: add data-path service APIs Fan Zhang
2020-08-28 12:58   ` [dpdk-dev] [dpdk-dev v7 1/4] cryptodev: add crypto " Fan Zhang
2020-08-31  6:23     ` Kusztal, ArkadiuszX
2020-08-31 12:21       ` Zhang, Roy Fan
2020-08-31 15:15       ` Zhang, Roy Fan
2020-08-28 12:58   ` [dpdk-dev] [dpdk-dev v7 2/4] crypto/qat: add crypto data-path service API support Fan Zhang
2020-08-28 12:58   ` [dpdk-dev] [dpdk-dev v7 3/4] test/crypto: add unit-test for cryptodev direct APIs Fan Zhang
2020-08-28 12:58   ` [dpdk-dev] [dpdk-dev v7 4/4] doc: add cryptodev service APIs guide Fan Zhang
2020-09-04 15:25   ` [dpdk-dev] [dpdk-dev v8 0/4] cryptodev: add data-path service APIs Fan Zhang
2020-09-04 15:25     ` [dpdk-dev] [dpdk-dev v8 1/4] cryptodev: add crypto " Fan Zhang
2020-09-07 12:36       ` Dybkowski, AdamX
2020-09-04 15:25     ` [dpdk-dev] [dpdk-dev v8 2/4] crypto/qat: add crypto data-path service API support Fan Zhang
2020-09-04 15:25     ` [dpdk-dev] [dpdk-dev v8 3/4] test/crypto: add unit-test for cryptodev direct APIs Fan Zhang
2020-09-04 15:25     ` [dpdk-dev] [dpdk-dev v8 4/4] doc: add cryptodev service APIs guide Fan Zhang
2020-09-08  8:42     ` [dpdk-dev] [dpdk-dev v9 0/4] cryptodev: add data-path service APIs Fan Zhang
2020-09-08  8:42       ` [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto " Fan Zhang
2020-09-18 21:50         ` Akhil Goyal
2020-09-21 10:40           ` Zhang, Roy Fan
2020-09-21 11:59             ` Akhil Goyal
2020-09-21 15:26               ` Zhang, Roy Fan
2020-09-21 15:41               ` Zhang, Roy Fan
2020-09-21 15:49                 ` Akhil Goyal
2020-09-22  8:08                   ` Zhang, Roy Fan
2020-09-22  8:21                   ` Zhang, Roy Fan
2020-09-22  8:48                     ` Ananyev, Konstantin
2020-09-22  9:05                       ` Akhil Goyal
2020-09-22  9:28                         ` Zhang, Roy Fan
2020-09-22 10:18                           ` Ananyev, Konstantin
2020-09-22 12:15                             ` Zhang, Roy Fan
2020-09-22 12:50                             ` Zhang, Roy Fan
2020-09-22 12:52                               ` Akhil Goyal
2020-09-08  8:42       ` [dpdk-dev] [dpdk-dev v9 2/4] crypto/qat: add crypto data-path service API support Fan Zhang
2020-09-08  8:42       ` [dpdk-dev] [dpdk-dev v9 3/4] test/crypto: add unit-test for cryptodev direct APIs Fan Zhang
2020-09-18 20:03         ` Akhil Goyal
2020-09-21 12:41           ` Zhang, Roy Fan
2020-09-08  8:42       ` [dpdk-dev] [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide Fan Zhang
2020-09-18 20:39         ` Akhil Goyal
2020-09-21 12:28           ` Zhang, Roy Fan
2020-09-23 13:37           ` Zhang, Roy Fan
2020-09-24 16:34       ` [dpdk-dev] [dpdk-dev v10 0/4] cryptodev: add raw data-path APIs Fan Zhang
2020-09-24 16:34         ` [dpdk-dev] [dpdk-dev v10 1/4] cryptodev: change crypto symmetric vector structure Fan Zhang
2020-09-25  8:03           ` Dybkowski, AdamX
2020-09-28 17:01           ` Ananyev, Konstantin
2020-09-24 16:34         ` [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs Fan Zhang
2020-09-25  8:04           ` Dybkowski, AdamX
2020-10-08 14:26           ` Akhil Goyal
2020-10-08 15:29             ` Zhang, Roy Fan
2020-10-08 16:07               ` Akhil Goyal
2020-10-08 16:24                 ` Zhang, Roy Fan
2020-10-09  8:32                 ` Zhang, Roy Fan
2020-10-08 14:37           ` Akhil Goyal
2020-09-24 16:34         ` [dpdk-dev] [dpdk-dev v10 3/4] crypto/qat: add raw crypto data-path API support Fan Zhang
2020-09-25  8:04           ` Dybkowski, AdamX
2020-09-24 16:34         ` [dpdk-dev] [dpdk-dev v10 4/4] test/crypto: add unit-test for cryptodev raw API test Fan Zhang
2020-09-25  8:05           ` Dybkowski, AdamX
2020-10-08 15:01           ` Akhil Goyal
2020-10-08 15:04         ` [dpdk-dev] [dpdk-dev v10 0/4] cryptodev: add raw data-path APIs Akhil Goyal
2020-10-08 15:30           ` Zhang, Roy Fan
2020-10-09 21:11         ` [dpdk-dev] [dpdk-dev v11 " Fan Zhang
2020-10-09 21:11           ` [dpdk-dev] [dpdk-dev v11 1/4] cryptodev: change crypto symmetric vector structure Fan Zhang
2020-10-09 21:11           ` [dpdk-dev] [dpdk-dev v11 2/4] cryptodev: add raw crypto data-path APIs Fan Zhang
2020-10-10 19:38             ` Akhil Goyal
2020-10-10 20:40               ` Zhang, Roy Fan
2020-10-09 21:11           ` [dpdk-dev] [dpdk-dev v11 3/4] crypto/qat: add raw crypto data-path API support Fan Zhang
2020-10-09 21:11           ` [dpdk-dev] [dpdk-dev v11 4/4] test/crypto: add unit-test for cryptodev raw API test Fan Zhang
2020-10-10 19:55             ` Akhil Goyal
2020-10-10 20:50               ` Zhang, Roy Fan
2020-10-10 21:03                 ` Akhil Goyal
2020-10-11  0:32           ` [dpdk-dev] [dpdk-dev v12 0/4] cryptodev: add raw data-path APIs Fan Zhang
2020-10-11  0:32             ` [dpdk-dev] [dpdk-dev v12 1/4] cryptodev: change crypto symmetric vector structure Fan Zhang
2020-10-11  0:32             ` [dpdk-dev] [dpdk-dev v12 2/4] cryptodev: add raw crypto data-path APIs Fan Zhang
2020-10-11  0:32             ` [dpdk-dev] [dpdk-dev v12 3/4] crypto/qat: add raw crypto data-path API support Fan Zhang
2020-10-11  0:32             ` [dpdk-dev] [dpdk-dev v12 4/4] test/crypto: add unit-test for cryptodev raw API test Fan Zhang
2020-10-11  0:38             ` [dpdk-dev] [dpdk-dev v13 0/4] cryptodev: add raw data-path APIs Fan Zhang
2020-10-11  0:38               ` [dpdk-dev] [dpdk-dev v13 1/4] cryptodev: change crypto symmetric vector structure Fan Zhang
2020-10-11  0:38               ` Fan Zhang [this message]
2020-10-11  0:38               ` [dpdk-dev] [dpdk-dev v13 3/4] crypto/qat: add raw crypto data-path API support Fan Zhang
2020-10-11  0:38               ` [dpdk-dev] [dpdk-dev v13 4/4] test/crypto: add unit-test for cryptodev raw API test Fan Zhang
2020-10-12 16:15               ` [dpdk-dev] [dpdk-dev v13 0/4] cryptodev: add raw data-path APIs Akhil Goyal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201011003854.54947-3-roy.fan.zhang@intel.com \
    --to=roy.fan.zhang@intel.com \
    --cc=adamx.dybkowski@intel.com \
    --cc=akhil.goyal@nxp.com \
    --cc=dev@dpdk.org \
    --cc=piotrx.bronowski@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git