From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9A124A04BC; Fri, 9 Oct 2020 23:12:33 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 509831D16D; Fri, 9 Oct 2020 23:11:56 +0200 (CEST) Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id CE6BE1BEA5 for ; Fri, 9 Oct 2020 23:11:52 +0200 (CEST) IronPort-SDR: 7B2+Aaz6rrkbYPyOd4fcDqwwxSgNBrtuYElI/d9lpGHY3MZBBqphGOIXXC9Ar+NY5UbrBltnsn 5rOScJ0kTIxw== X-IronPort-AV: E=McAfee;i="6000,8403,9769"; a="182984637" X-IronPort-AV: E=Sophos;i="5.77,356,1596524400"; d="scan'208";a="182984637" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Oct 2020 14:11:51 -0700 IronPort-SDR: i7tEhY6clg+CgETC8HVqN0JXrauK/FbZKQ2tF5BehlfrYbRp8Hysvvin1fpiMjI3j1Cmnk8G+P U0rFAMp5m/jg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,356,1596524400"; d="scan'208";a="329029395" Received: from silpixa00398673.ir.intel.com (HELO silpixa00398673.ger.corp.intel.com) ([10.237.223.136]) by orsmga002.jf.intel.com with ESMTP; 09 Oct 2020 14:11:49 -0700 From: Fan Zhang To: dev@dpdk.org Cc: akhil.goyal@nxp.com, Fan Zhang , Piotr Bronowski , Adam Dybkowski Date: Fri, 9 Oct 2020 22:11:39 +0100 Message-Id: <20201009211141.14435-3-roy.fan.zhang@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20201009211141.14435-1-roy.fan.zhang@intel.com> References: <20200924163417.49983-1-roy.fan.zhang@intel.com> <20201009211141.14435-1-roy.fan.zhang@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Subject: [dpdk-dev] [dpdk-dev v11 2/4] cryptodev: add raw crypto data-path APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch adds raw data-path APIs for enqueue and dequeue operations to cryptodev. The APIs support flexible user-define enqueue and dequeue behaviors. Signed-off-by: Fan Zhang Signed-off-by: Piotr Bronowski Acked-by: Adam Dybkowski --- doc/guides/cryptodevs/features/default.ini | 1 + doc/guides/cryptodevs/features/qat.ini | 1 + doc/guides/prog_guide/cryptodev_lib.rst | 97 +++++ doc/guides/rel_notes/release_20_11.rst | 7 + lib/librte_cryptodev/rte_cryptodev.c | 80 ++++ lib/librte_cryptodev/rte_cryptodev.h | 367 +++++++++++++++++- lib/librte_cryptodev/rte_cryptodev_pmd.h | 51 ++- .../rte_cryptodev_version.map | 10 + 8 files changed, 611 insertions(+), 3 deletions(-) diff --git a/doc/guides/cryptodevs/features/default.ini b/doc/guides/cryptodevs/features/default.ini index 133a246ee..17b177fc4 100644 --- a/doc/guides/cryptodevs/features/default.ini +++ b/doc/guides/cryptodevs/features/default.ini @@ -30,6 +30,7 @@ Asymmetric sessionless = CPU crypto = Symmetric sessionless = Non-Byte aligned data = +Sym raw data path API = ; ; Supported crypto algorithms of a default crypto driver. diff --git a/doc/guides/cryptodevs/features/qat.ini b/doc/guides/cryptodevs/features/qat.ini index 9e82f2886..6cc09cde7 100644 --- a/doc/guides/cryptodevs/features/qat.ini +++ b/doc/guides/cryptodevs/features/qat.ini @@ -17,6 +17,7 @@ Digest encrypted = Y Asymmetric sessionless = Y RSA PRIV OP KEY EXP = Y RSA PRIV OP KEY QT = Y +Sym raw data path API = Y ; ; Supported crypto algorithms of the 'qat' crypto driver. diff --git a/doc/guides/prog_guide/cryptodev_lib.rst b/doc/guides/prog_guide/cryptodev_lib.rst index e7ba35c2d..8ba800122 100644 --- a/doc/guides/prog_guide/cryptodev_lib.rst +++ b/doc/guides/prog_guide/cryptodev_lib.rst @@ -632,6 +632,103 @@ a call argument. Status different than zero must be treated as error. For more details, e.g. how to convert an mbuf to an SGL, please refer to an example usage in the IPsec library implementation. +Cryptodev Raw Data-path APIs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Crypto Raw data-path APIs are a set of APIs designed to enable external +libraries/applications to leverage the cryptographic processing provided by +DPDK crypto PMDs through the cryptodev API but in a manner that is not +dependent on native DPDK data structures (eg. rte_mbuf, rte_crypto_op, ... etc) +in their data-path implementation. + +The raw data-path APIs have the following advantages: +- External data structure friendly design. The new APIs uses the operation + descriptor ``struct rte_crypto_sym_vec`` that supports raw data pointer and + IOVA addresses as input. Moreover, the APIs does not require the user to + allocate the descriptor from mempool, nor requiring mbufs to describe input + data's virtual and IOVA addresses. All these features made the translation + from user's own data structure into the descriptor easier and more efficient. +- Flexible enqueue and dequeue operation. The raw data-path APIs gives the + user more control to the enqueue and dequeue operations, including the + capability of precious enqueue/dequeue count, abandoning enqueue or dequeue + at any time, and operation status translation and set on the fly. + +Cryptodev PMDs which support the raw data-path APIs will have +``RTE_CRYPTODEV_FF_SYM_RAW_DP`` feature flag presented. To use this feature, +the user shall create a local ``struct rte_crypto_raw_dp_ctx`` buffer and +extend to at least the length returned by ``rte_cryptodev_get_raw_dp_ctx_size`` +function call. The created buffer is then initialized using +``rte_cryptodev_configure_raw_dp_ctx`` function with the ``is_update`` +parameter as 0. The library and the crypto device driver will then set the +buffer and attach either the cryptodev sym session, the rte_security session, +or the cryptodev xform for session-less operation into the ctx buffer, and +set the corresponding enqueue and dequeue function handlers based on the +algorithm information stored in the session or xform. When the ``is_update`` +parameter passed into ``rte_cryptodev_configure_raw_dp_ctx`` is 1, the driver +will not initialize the buffer but only update the session or xform and +the function handlers accordingly. + +After the ``struct rte_crypto_raw_dp_ctx`` buffer is initialized, it is now +ready for enqueue and dequeue operation. There are two different enqueue +functions: ``rte_cryptodev_raw_enqueue`` to enqueue single descriptor, +and ``rte_cryptodev_raw_enqueue_burst`` to enqueue multiple descriptors. +In case of the application uses similar approach to +``struct rte_crypto_sym_vec`` to manage its data burst but with different +data structure, using the ``rte_cryptodev_raw_enqueue_burst`` function may be +less efficient as this is a situation where the application has to loop over +all crypto descriptors to assemble the ``struct rte_crypto_sym_vec`` buffer +from its own data structure, and then the driver will loop over them again to +translate every descriptor to the driver's specific queue data. The +``rte_cryptodev_raw_enqueue`` should be used to save one loop for each data +burst instead. + +The ``rte_cryptodev_raw_enqueue`` and ``rte_cryptodev_raw_enqueue_burst`` +functions will return or set the operation status (``rte_cryptodev_raw_enqueue`` +will return the status directly, ``rte_cryptodev_raw_enqueue_burst`` will set +the status buffer provided by the user): + +- The negative integer value of the status means something wrong happened + during enqueue, hence nothing is enqueued. +- The status ``1`` means the descriptor(s) is enqueued successfully. +- The status ``0`` indicates the descriptor(s) is stored successfully in the + crypto device queue but is not actually enqueued. The user shall call + ``rte_cryptodev_raw_enqueue_done`` function after the expected descriptors + are stored. The crypto device will then start enqueuing all of them at + once. + +Calling ``rte_cryptodev_configure_raw_dp_context`` with the parameter +``is_update`` set as 0 twice without the enqueue function returning status 1 or +``rte_cryptodev_dp_enqueue_done`` function call in between will invalidate any +descriptors stored in the device queue but not enqueued. This feature is useful +when the user wants to abandon partially enqueued data for a failed enqueue +burst operation and try enqueuing in a whole later. + +Similar as enqueue, there are two dequeue functions: +``rte_cryptodev_raw_dequeue`` for dequeing single descriptor, and +``rte_cryptodev_raw_dequeue_burst`` for dequeuing a burst of descriptors. The +raw data-path dequeue burst function allows the user to provide callback +functions to retrieve dequeue count from the enqueued user data, and write the +expected status value to the user data on the fly. The dequeue functions also +return or set the operation status: + +- The negative integer value of the status means something wrong happened during + dequeue, hence nothing is dequeued. +- The status ``1`` means the descriptors is/are dequeued successfully. +- The status ``0`` means the descriptors is/are fetched successfully from + crypto device queue but is not actually dequeued (hence still kept in the + device queue). The user shall call the ``rte_cryptodev_raw_dequeue_done`` + function after the expected number of descriptors are dequeued. The crypto + device driver will then free them from the queue at once. + +There are a few limitations to the raw data path APIs: + +* Only support in-place operations. +* APIs are NOT thread-safe. +* CANNOT mix the raw data-path API's enqueue with rte_cryptodev_enqueue_burst, + or vice versa. + +See *DPDK API Reference* for details on each API definitions. + Sample code ----------- diff --git a/doc/guides/rel_notes/release_20_11.rst b/doc/guides/rel_notes/release_20_11.rst index 2973b2a33..947671f09 100644 --- a/doc/guides/rel_notes/release_20_11.rst +++ b/doc/guides/rel_notes/release_20_11.rst @@ -165,6 +165,13 @@ New Features * Extern objects and functions can be plugged into the pipeline. * Transaction-oriented table updates. +* **Added raw data-path APIs for cryptodev library.** + + Cryptodev is added raw data-path APIs to accelerate external libraries or + applications those want to avail fast cryptodev enqueue/dequeue + operations but does not necessarily depends on mbufs and cryptodev + operation mempool. + Removed Items ------------- diff --git a/lib/librte_cryptodev/rte_cryptodev.c b/lib/librte_cryptodev/rte_cryptodev.c index cda160f61..7a143c4b9 100644 --- a/lib/librte_cryptodev/rte_cryptodev.c +++ b/lib/librte_cryptodev/rte_cryptodev.c @@ -1772,6 +1772,86 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, return dev->dev_ops->sym_cpu_process(dev, sess, ofs, vec); } +int +rte_cryptodev_get_raw_dp_ctx_size(uint8_t dev_id) +{ + struct rte_cryptodev *dev; + int32_t size = sizeof(struct rte_crypto_raw_dp_ctx); + int32_t priv_size; + + if (!rte_cryptodev_pmd_is_valid_dev(dev_id)) + return -EINVAL; + + dev = rte_cryptodev_pmd_get_dev(dev_id); + + if (*dev->dev_ops->sym_get_raw_dp_ctx_size == NULL || + !(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP)) { + return -ENOTSUP; + } + + priv_size = (*dev->dev_ops->sym_get_raw_dp_ctx_size)(dev); + if (priv_size < 0) + return -ENOTSUP; + + return RTE_ALIGN_CEIL((size + priv_size), 8); +} + +int +rte_cryptodev_configure_raw_dp_ctx(uint8_t dev_id, uint16_t qp_id, + struct rte_crypto_raw_dp_ctx *ctx, + enum rte_crypto_op_sess_type sess_type, + union rte_cryptodev_session_ctx session_ctx, + uint8_t is_update) +{ + struct rte_cryptodev *dev; + + if (!rte_cryptodev_get_qp_status(dev_id, qp_id)) + return -EINVAL; + + dev = rte_cryptodev_pmd_get_dev(dev_id); + if (!(dev->feature_flags & RTE_CRYPTODEV_FF_SYM_RAW_DP) + || dev->dev_ops->sym_configure_raw_dp_ctx == NULL) + return -ENOTSUP; + + return (*dev->dev_ops->sym_configure_raw_dp_ctx)(dev, qp_id, ctx, + sess_type, session_ctx, is_update); +} + +uint32_t +rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx, + struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs, + void **user_data, int *enqueue_status) +{ + return (*ctx->enqueue_burst)(ctx->qp_data, ctx->drv_ctx_data, vec, + ofs, user_data, enqueue_status); +} + +int +rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx, + uint32_t n) +{ + return (*ctx->enqueue_done)(ctx->qp_data, ctx->drv_ctx_data, n); +} + +int +rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx, + uint32_t n) +{ + return (*ctx->dequeue_done)(ctx->qp_data, ctx->drv_ctx_data, n); +} + +uint32_t +rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx, + rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count, + rte_cryptodev_raw_post_dequeue_t post_dequeue, + void **out_user_data, uint8_t is_user_data_array, + uint32_t *n_success_jobs, int *status) +{ + return (*ctx->dequeue_burst)(ctx->qp_data, ctx->drv_ctx_data, + get_dequeue_count, post_dequeue, out_user_data, + is_user_data_array, n_success_jobs, status); +} + /** Initialise rte_crypto_op mempool element */ static void rte_crypto_op_init(struct rte_mempool *mempool, diff --git a/lib/librte_cryptodev/rte_cryptodev.h b/lib/librte_cryptodev/rte_cryptodev.h index 26abd0c52..840a1c54c 100644 --- a/lib/librte_cryptodev/rte_cryptodev.h +++ b/lib/librte_cryptodev/rte_cryptodev.h @@ -458,7 +458,8 @@ rte_cryptodev_asym_get_xform_enum(enum rte_crypto_asym_xform_type *xform_enum, /**< Support symmetric session-less operations */ #define RTE_CRYPTODEV_FF_NON_BYTE_ALIGNED_DATA (1ULL << 23) /**< Support operations on data which is not byte aligned */ - +#define RTE_CRYPTODEV_FF_SYM_RAW_DP (1ULL << 24) +/**< Support accelerated specific raw data-path APIs */ /** * Get the name of a crypto device feature flag @@ -1319,6 +1320,370 @@ rte_cryptodev_sym_cpu_crypto_process(uint8_t dev_id, struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec); +/** + * Get the size of the raw data-path context buffer. + * + * @param dev_id The device identifier. + * + * @return + * - If the device supports raw data-path APIs, return the context size. + * - If the device does not support the APIs, return -1. + */ +__rte_experimental +int +rte_cryptodev_get_raw_dp_ctx_size(uint8_t dev_id); + +/** + * Union of different crypto session types, including session-less xform + * pointer. + */ +union rte_cryptodev_session_ctx { + struct rte_cryptodev_sym_session *crypto_sess; + struct rte_crypto_sym_xform *xform; + struct rte_security_session *sec_sess; +}; + +/** + * Enqueue a data vector into device queue but the driver will not start + * processing until rte_cryptodev_raw_enqueue_done() is called. + * + * @param qp Driver specific queue pair data. + * @param drv_ctx Driver specific context data. + * @param vec The array of descriptor vectors. + * @param ofs Start and stop offsets for auth and cipher + * operations. + * @param user_data The array of user data for dequeue later. + * @param enqueue_status Enqueue status written by the driver. + * @return + * - The number of descriptors successfully enqueued. + * - Possible enqueue status written by the driver: + * - 1: The descriptors are enqueued successfully. + * - 0: The descriptors are stored into device queue but are not processed + * until rte_cryptodev_raw_enqueue_done() is called. + * - negative integer: failure. + */ +typedef uint32_t (*cryptodev_sym_raw_enqueue_burst_t)( + void *qp, uint8_t *drv_ctx, struct rte_crypto_sym_vec *vec, + union rte_crypto_sym_ofs ofs, void *user_data[], int *enqueue_status); + +/** + * Enqueue single descriptor into device queue but the driver will not start + * processing until rte_cryptodev_raw_enqueue_done() is called. + * + * @param qp Driver specific queue pair data. + * @param drv_ctx Driver specific context data. + * @param data_vec The buffer data vector. + * @param n_data_vecs Number of buffer data vectors. + * @param ofs Start and stop offsets for auth and cipher + * operations. + * @param iv IV virtual and IOVA addresses + * @param digest digest virtual and IOVA addresses + * @param aad_or_auth_iv AAD or auth IV virtual and IOVA addresses, + * depends on the algorithm used. + * @param user_data The user data. + * @return + * - 1: The descriptor is enqueued successfully. + * - 0: The descriptor is cached into device queue but is not processed + * until rte_cryptodev_raw_enqueue_done() is called. + * - negative integer: failure. + */ +typedef int (*cryptodev_sym_raw_enqueue_t)( + void *qp, uint8_t *drv_ctx, struct rte_crypto_vec *data_vec, + uint16_t n_data_vecs, union rte_crypto_sym_ofs ofs, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *aad_or_auth_iv, + void *user_data); + +/** + * Inform the cryptodev queue pair to start processing or finish dequeuing all + * enqueued/dequeued descriptors. + * + * @param qp Driver specific queue pair data. + * @param drv_ctx Driver specific context data. + * @param n The total number of processed descriptors. + * @return + * - On success return 0. + * - On failure return negative integer. + */ +typedef int (*cryptodev_sym_raw_operation_done_t)(void *qp, uint8_t *drv_ctx, + uint32_t n); + +/** + * Typedef that the user provided for the driver to get the dequeue count. + * The function may return a fixed number or the number parsed from the user + * data stored in the first processed descriptor. + * + * @param user_data Dequeued user data. + **/ +typedef uint32_t (*rte_cryptodev_raw_get_dequeue_count_t)(void *user_data); + +/** + * Typedef that the user provided to deal with post dequeue operation, such + * as filling status. + * + * @param user_data Dequeued user data. + * @param index Index number of the processed descriptor. + * @param is_op_success Operation status provided by the driver. + **/ +typedef void (*rte_cryptodev_raw_post_dequeue_t)(void *user_data, + uint32_t index, uint8_t is_op_success); + +/** + * Dequeue a burst of symmetric crypto processing of user provided data. + * + * @param qp Driver specific queue pair data. + * @param drv_ctx Driver specific context data. + * @param get_dequeue_count User provided callback function to + * obtain dequeue count. + * @param post_dequeue User provided callback function to + * post-process a dequeued operation. + * @param out_user_data User data pointer array to be retrieve + * from device queue. In case of + * *is_user_data_array* is set there + * should be enough room to store all + * user data. + * @param is_user_data_array Set 1 if every dequeued user data will + * be written into out_user_data* array. + * @param n_success Driver written value to specific the + * total successful operations count. + * @param dequeue_status Dequeue status written by the driver. + * @return + * - The number of descriptors successfully dequeued. + * - Possible dequeue status written by the driver: + * - 1: The descriptors are dequeued successfully. + * - 0: The descriptors are fetched from queue pair but are not freed + * until rte_cryptodev_raw_dequeue_done() is called. + * - negative integer: Error occurred when dequeuing. + */ +typedef uint32_t (*cryptodev_sym_raw_dequeue_burst_t)(void *qp, + uint8_t *drv_ctx, + rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count, + rte_cryptodev_raw_post_dequeue_t post_dequeue, + void **out_user_data, uint8_t is_user_data_array, + uint32_t *n_success, int *dequeue_status); + +/** + * Dequeue symmetric crypto processing of user provided data. + * + * @param qp Driver specific queue pair data. + * @param drv_ctx Driver specific context data. + * @param dequeue_status Dequeue status written by the driver. + * @param op_status Operation status written by the driver. + * + * @return + * - The user data pointer retrieved from device queue or NULL if no + * descriptor is ready for dequeue. + * - Possible dequeue status written by the driver: + * - 1: The descriptors are dequeued successfully. + * - 0: The descriptors are fetched from queue pair but are not freed + * until rte_cryptodev_raw_dequeue_done() is called. + * - negative integer: Error occurred when dequeuing. + */ +typedef void * (*cryptodev_sym_raw_dequeue_t)( + void *qp, uint8_t *drv_ctx, int *dequeue_status, + enum rte_crypto_op_status *op_status); + +/** + * Context data for raw data-path API crypto process. The buffer of this + * structure is to be allocated by the user application with the size equal + * or bigger than rte_cryptodev_raw_get_dp_context_size() returned value. + * + * NOTE: the buffer is to be used and maintained by the cryptodev driver, the + * user should NOT alter the buffer content to avoid application or system + * crash. + */ +struct rte_crypto_raw_dp_ctx { + void *qp_data; + + cryptodev_sym_raw_enqueue_t enqueue; + cryptodev_sym_raw_enqueue_burst_t enqueue_burst; + cryptodev_sym_raw_operation_done_t enqueue_done; + cryptodev_sym_raw_dequeue_t dequeue; + cryptodev_sym_raw_dequeue_burst_t dequeue_burst; + cryptodev_sym_raw_operation_done_t dequeue_done; + + /* Driver specific context data */ + __extension__ uint8_t drv_ctx_data[]; +}; + +/** + * Configure raw data-path context data. + * + * NOTE: + * After the context data is configured, the user should call + * rte_cryptodev_raw_attach_session() before using it in + * rte_cryptodev_raw_enqueue/dequeue function call. + * + * @param dev_id The device identifier. + * @param qp_id The index of the queue pair from which to + * retrieve processed packets. The value must be + * in the range [0, nb_queue_pair - 1] previously + * supplied to rte_cryptodev_configure(). + * @param ctx The raw data-path context data. + * @param sess_type session type. + * @param session_ctx Session context data. + * @param is_update Set 0 if it is to initialize the ctx. + * Set 1 if ctx is initialized and only to update + * session context data. + * @return + * - On success return 0. + * - On failure return negative integer. + */ +__rte_experimental +int +rte_cryptodev_configure_raw_dp_ctx(uint8_t dev_id, uint16_t qp_id, + struct rte_crypto_raw_dp_ctx *ctx, + enum rte_crypto_op_sess_type sess_type, + union rte_cryptodev_session_ctx session_ctx, + uint8_t is_update); + +/** + * Enqueue single raw data-path descriptor. + * + * @param ctx The initialized raw data-path context data. + * @param data_vec The buffer vector. + * @param n_data_vecs Number of buffer vectors. + * @param ofs Start and stop offsets for auth and cipher + * operations. + * @param iv IV virtual and IOVA addresses + * @param digest digest virtual and IOVA addresses + * @param aad_or_auth_iv AAD or auth IV virtual and IOVA addresses, + * depends on the algorithm used. + * @param user_data The user data. + * @return + * - 1: The descriptor is enqueued successfully. + * - 0: The descriptor is cached into device queue but is not processed + * until rte_cryptodev_raw_enqueue_done() is called. + * - negative integer: failure. + */ +__rte_experimental +static __rte_always_inline int +rte_cryptodev_raw_enqueue(struct rte_crypto_raw_dp_ctx *ctx, + struct rte_crypto_vec *data_vec, uint16_t n_data_vecs, + union rte_crypto_sym_ofs ofs, + struct rte_crypto_va_iova_ptr *iv, + struct rte_crypto_va_iova_ptr *digest, + struct rte_crypto_va_iova_ptr *aad_or_auth_iv, + void *user_data) +{ + return (*ctx->enqueue)(ctx->qp_data, ctx->drv_ctx_data, data_vec, + n_data_vecs, ofs, iv, digest, aad_or_auth_iv, user_data); +} + +/** + * Enqueue a data vector of raw data-path descriptors. + * + * @param ctx The initialized raw data-path context data. + * @param vec The array of descriptor vectors. + * @param ofs Start and stop offsets for auth and cipher + * operations. + * @param user_data The array of opaque data for dequeue. + * @param enqueue_status Enqueue status written by the driver. + * @return + * - The number of descriptors successfully enqueued. + * - Possible operation status written by the driver: + * - 1: The descriptors are enqueued successfully. + * - 0: The descriptors are cached into device queue but are not processed + * until rte_cryptodev_raw_enqueue_done() is called. + * - negative integer: failure. + */ +__rte_experimental +uint32_t +rte_cryptodev_raw_enqueue_burst(struct rte_crypto_raw_dp_ctx *ctx, + struct rte_crypto_sym_vec *vec, union rte_crypto_sym_ofs ofs, + void **user_data, int *enqueue_status); + +/** + * Start processing all enqueued descriptors from last + * rte_cryptodev_raw_configure_dp_context() call. + * + * @param ctx The initialized raw data-path context data. + * @param n The total number of submitted descriptors. + * @return + * - On success return 0. + * - On failure return negative integer. + */ +__rte_experimental +int +rte_cryptodev_raw_enqueue_done(struct rte_crypto_raw_dp_ctx *ctx, + uint32_t n); + +/** + * Dequeue a burst of symmetric crypto processing of user provided data. + * + * @param ctx The initialized raw data-path context + * data. + * @param get_dequeue_count User provided callback function to + * obtain dequeue count. + * @param post_dequeue User provided callback function to + * post-process a dequeued operation. + * @param out_user_data User data pointer array to be retrieve + * from device queue. In case of + * *is_user_data_array* is set there + * should be enough room to store all + * user data. + * @param is_user_data_array Set 1 if every dequeued user data will + * be written into *out_user_data* array. + * @param n_success Driver written value to specify the + * total successful operations count. + * @param dequeue_status Dequeue status written by the driver. + * @return + * - The number of descriptors successfully dequeued. + * - Possible dequeue status written by the driver: + * - 1: The descriptors are dequeued successfully. + * - 0: The descriptors are fetched from queue pair but are not freed + * until rte_cryptodev_raw_dequeue_done() is called. + * - negative integer: Error occurred when dequeuing. + */ +__rte_experimental +uint32_t +rte_cryptodev_raw_dequeue_burst(struct rte_crypto_raw_dp_ctx *ctx, + rte_cryptodev_raw_get_dequeue_count_t get_dequeue_count, + rte_cryptodev_raw_post_dequeue_t post_dequeue, + void **out_user_data, uint8_t is_user_data_array, + uint32_t *n_success, int *dequeue_status); + +/** + * * Dequeue a symmetric crypto processing of user provided data. + * + * @param ctx The initialized raw data-path context + * data. + * @param dequeue_status Dequeue status written by the driver. + * @param op_status Operation status written by the driver. + * + * @return + * - The user data pointer retrieved from device queue or NULL if no + * descriptor is ready for dequeue. + * - Possible dequeue status written by the driver: + * - 1: The descriptors are dequeued successfully. + * - 0: The descriptors are fetched from queue pair but are not freed + * until rte_cryptodev_raw_dequeue_done() is called. + * - negative integer: Error occurred when dequeuing. + */ +__rte_experimental +static __rte_always_inline void * +rte_cryptodev_raw_dequeue(struct rte_crypto_raw_dp_ctx *ctx, + int *dequeue_status, enum rte_crypto_op_status *op_status) +{ + return (*ctx->dequeue)(ctx->qp_data, ctx->drv_ctx_data, dequeue_status, + op_status); +} + +/** + * Inform the queue pair dequeue operations finished. + * + * @param ctx The initialized raw data-path context data. + * @param n The total number of jobs already dequeued. + * @return + * - On success return 0. + * - On failure return negative integer. + */ +__rte_experimental +int +rte_cryptodev_raw_dequeue_done(struct rte_crypto_raw_dp_ctx *ctx, + uint32_t n); + #ifdef __cplusplus } #endif diff --git a/lib/librte_cryptodev/rte_cryptodev_pmd.h b/lib/librte_cryptodev/rte_cryptodev_pmd.h index 1367222f7..d6be69903 100644 --- a/lib/librte_cryptodev/rte_cryptodev_pmd.h +++ b/lib/librte_cryptodev/rte_cryptodev_pmd.h @@ -317,6 +317,42 @@ typedef uint32_t (*cryptodev_sym_cpu_crypto_process_t) (struct rte_cryptodev *dev, struct rte_cryptodev_sym_session *sess, union rte_crypto_sym_ofs ofs, struct rte_crypto_sym_vec *vec); +/** + * Typedef that the driver provided to get service context private date size. + * + * @param dev Crypto device pointer. + * + * @return + * - On success return the size of the device's service context private data. + * - On failure return negative integer. + */ +typedef int (*cryptodev_sym_get_raw_dp_ctx_size_t)(struct rte_cryptodev *dev); + +/** + * Typedef that the driver provided to configure data-path context. + * + * @param dev Crypto device pointer. + * @param qp_id Crypto device queue pair index. + * @param service_type Type of the service requested. + * @param ctx The raw data-path context data. + * @param sess_type session type. + * @param session_ctx Session context data. If NULL the driver + * shall only configure the drv_ctx_data in + * ctx buffer. Otherwise the driver shall only + * parse the session_ctx to set appropriate + * function pointers in ctx. + * @param is_update Set 0 if it is to initialize the ctx. + * Set 1 if ctx is initialized and only to update + * session context data. + * @return + * - On success return 0. + * - On failure return negative integer. + */ +typedef int (*cryptodev_sym_configure_raw_dp_ctx_t)( + struct rte_cryptodev *dev, uint16_t qp_id, + struct rte_crypto_raw_dp_ctx *ctx, + enum rte_crypto_op_sess_type sess_type, + union rte_cryptodev_session_ctx session_ctx, uint8_t is_update); /** Crypto device operations function pointer table */ struct rte_cryptodev_ops { @@ -349,8 +385,19 @@ struct rte_cryptodev_ops { /**< Clear a Crypto sessions private data. */ cryptodev_asym_free_session_t asym_session_clear; /**< Clear a Crypto sessions private data. */ - cryptodev_sym_cpu_crypto_process_t sym_cpu_process; - /**< process input data synchronously (cpu-crypto). */ + union { + cryptodev_sym_cpu_crypto_process_t sym_cpu_process; + /**< process input data synchronously (cpu-crypto). */ + __extension__ + struct { + cryptodev_sym_get_raw_dp_ctx_size_t + sym_get_raw_dp_ctx_size; + /**< Get data path service context data size. */ + cryptodev_sym_configure_raw_dp_ctx_t + sym_configure_raw_dp_ctx; + /**< Initialize crypto service ctx data. */ + }; + }; }; diff --git a/lib/librte_cryptodev/rte_cryptodev_version.map b/lib/librte_cryptodev/rte_cryptodev_version.map index 7727286ac..7e4360ff0 100644 --- a/lib/librte_cryptodev/rte_cryptodev_version.map +++ b/lib/librte_cryptodev/rte_cryptodev_version.map @@ -99,4 +99,14 @@ EXPERIMENTAL { # added in 20.08 rte_cryptodev_get_qp_status; + + # added in 20.11 + rte_cryptodev_configure_raw_dp_ctx; + rte_cryptodev_get_raw_dp_ctx_size; + rte_cryptodev_raw_dequeue; + rte_cryptodev_raw_dequeue_burst; + rte_cryptodev_raw_dequeue_done; + rte_cryptodev_raw_enqueue; + rte_cryptodev_raw_enqueue_burst; + rte_cryptodev_raw_enqueue_done; }; -- 2.20.1