DPDK patches and discussions
 help / color / mirror / Atom feed
From: Fan Zhang <roy.fan.zhang@intel.com>
To: dev@dpdk.org
Cc: akhil.goyal@nxp.com, fiona.trahe@intel.com,
	arkadiuszx.kusztal@intel.com, adamx.dybkowski@intel.com,
	roy.fan.zhang@intel.com
Subject: [dpdk-dev] [dpdk-dev v7 0/4] cryptodev: add data-path service APIs
Date: Fri, 28 Aug 2020 13:58:11 +0100
Message-ID: <20200828125815.21614-1-roy.fan.zhang@intel.com> (raw)
In-Reply-To: <20200818162833.20219-1-roy.fan.zhang@intel.com>

Direct crypto data-path service are a set of APIs that especially provided for
the external libraries/applications who want to take advantage of the rich
features provided by cryptodev, but not necessarily depend on cryptodev
operations, mempools, or mbufs in the their data-path implementations.

The direct crypto data-path service has the following advantages:
- Supports raw data pointer and physical addresses as input.
- Do not require specific data structure allocated from heap, such as
  cryptodev operation.
- Enqueue in a burst or single operation. The service allow enqueuing in
  a burst similar to ``rte_cryptodev_enqueue_burst`` operation, or only
  enqueue one job at a time but maintaining necessary context data locally for
  next single job enqueue operation. The latter method is especially helpful
  when the user application's crypto operations are clustered into a burst.
  Allowing enqueue one operation at a time helps reducing one additional loop
  and also reduced the cache misses during the double "looping" situation.
- Customerizable dequeue count. Instead of dequeue maximum possible operations
  as same as ``rte_cryptodev_dequeue_burst`` operation, the service allows the
  user to provide a callback function to decide how many operations to be
  dequeued. This is especially helpful when the expected dequeue count is
  hidden inside the opaque data stored during enqueue. The user can provide
  the callback function to parse the opaque data structure.
- Abandon enqueue and dequeue anytime. One of the drawbacks of
  ``rte_cryptodev_enqueue_burst`` and ``rte_cryptodev_dequeue_burst``
  operations are: once an operation is enqueued/dequeued there is no way to
  undo the operation. The service make the operation abandon possible by
  creating a local copy of the queue operation data in the service context
  data. The data will be written back to driver maintained operation data
  when enqueue or dequeue done function is called.

The cryptodev data-path service uses

Cryptodev PMDs who supports this feature will have
``RTE_CRYPTODEV_FF_SYM_HW_DIRECT_API`` feature flag presented. To use this
feature the function ``rte_cryptodev_get_dp_service_ctx_data_size`` should
be called to get the data path service context data size. The user should
creates a local buffer at least this size long and initialize it using
``rte_cryptodev_dp_configure_service`` function call.

The ``rte_cryptodev_dp_configure_service`` function call initialize or
updates the ``struct rte_crypto_dp_service_ctx`` buffer, in which contains the
driver specific queue pair data pointer and service context buffer, and a
set of function pointers to enqueue and dequeue different algorithms'
operations. The ``rte_cryptodev_dp_configure_service`` should be called when:

- Before enqueuing or dequeuing starts (set ``is_update`` parameter to 0).
- When different cryptodev session, security session, or session-less xform
  is used (set ``is_update`` parameter to 1).

Two different enqueue functions are provided.

- ``rte_cryptodev_dp_sym_submit_vec``: submit a burst of operations stored in
  the ``rte_crypto_sym_vec`` structure.
- ``rte_cryptodev_dp_submit_single_job``: submit single operation.

Either enqueue functions will not command the crypto device to start processing
until ``rte_cryptodev_dp_submit_done`` function is called. Before then the user
shall expect the driver only stores the necessory context data in the
``rte_crypto_dp_service_ctx`` buffer for the next enqueue operation. If the user
wants to abandon the submitted operations, simply call
``rte_cryptodev_dp_configure_service`` function instead with the parameter
``is_update`` set to 0. The driver will recover the service context data to
the previous state.

To dequeue the operations the user also have two operations:

- ``rte_cryptodev_dp_sym_dequeue``: fully customizable deuqueue operation. The
  user needs to provide the callback function for the driver to get the
  dequeue count and perform post processing such as write the status field.
- ``rte_cryptodev_dp_sym_dequeue_single_job``: dequeue single job.

Same as enqueue, the function ``rte_cryptodev_dp_dequeue_done`` is used to
merge user's local service context data with the driver's queue operation
data. Also to abandon the dequeue operation (still keep the operations in the
queue), the user shall avoid ``rte_cryptodev_dp_dequeue_done`` function call
but calling ``rte_cryptodev_dp_configure_service`` function with the parameter
``is_update`` set to 0.

There are a few limitations to the data path service:

* Only support in-place operations.
* APIs are NOT thread-safe.
* CANNOT mix the direct API's enqueue with rte_cryptodev_enqueue_burst, or
  vice versa.

v7:
- Fixed a few typos.
- Fixed length calculation bugs.

v6:
- Rebased on top of DPDK 20.08.
- Changed to service ctx and added single job submit/dequeue.

v5:
- Changed to use rte_crypto_sym_vec as input.
- Changed to use public APIs instead of use function pointer.

v4:
- Added missed patch.

v3:
- Instead of QAT only API, moved the API to cryptodev.
- Added cryptodev feature flags.

v2:
- Used a structure to simplify parameters.
- Added unit tests.
- Added documentation.

Fan Zhang (4):
  cryptodev: add crypto data-path service APIs
  crypto/qat: add crypto data-path service API support
  test/crypto: add unit-test for cryptodev direct APIs
  doc: add cryptodev service APIs guide

 app/test/test_cryptodev.c                     | 354 ++++++-
 app/test/test_cryptodev.h                     |   6 +
 app/test/test_cryptodev_blockcipher.c         |  50 +-
 doc/guides/prog_guide/cryptodev_lib.rst       |  90 ++
 drivers/common/qat/Makefile                   |   1 +
 drivers/crypto/qat/meson.build                |   1 +
 drivers/crypto/qat/qat_sym.h                  |  13 +
 drivers/crypto/qat/qat_sym_hw_dp.c            | 926 ++++++++++++++++++
 drivers/crypto/qat/qat_sym_pmd.c              |   9 +-
 lib/librte_cryptodev/rte_crypto.h             |   9 +
 lib/librte_cryptodev/rte_crypto_sym.h         |  44 +-
 lib/librte_cryptodev/rte_cryptodev.c          |  45 +
 lib/librte_cryptodev/rte_cryptodev.h          | 335 ++++++-
 lib/librte_cryptodev/rte_cryptodev_pmd.h      |  47 +-
 .../rte_cryptodev_version.map                 |  10 +
 15 files changed, 1892 insertions(+), 48 deletions(-)
 create mode 100644 drivers/crypto/qat/qat_sym_hw_dp.c

-- 
2.20.1


  parent reply	other threads:[~2020-08-28 12:58 UTC|newest]

Thread overview: 84+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-18 16:28 [dpdk-dev] [dpdk-dev v6 " Fan Zhang
2020-08-18 16:28 ` [dpdk-dev] [dpdk-dev v6 1/4] cryptodev: add crypto " Fan Zhang
2020-08-18 16:28 ` [dpdk-dev] [dpdk-dev v6 2/4] crypto/qat: add crypto data-path service API support Fan Zhang
2020-08-18 16:28 ` [dpdk-dev] [dpdk-dev v6 3/4] test/crypto: add unit-test for cryptodev direct APIs Fan Zhang
2020-08-18 16:28 ` [dpdk-dev] [dpdk-dev v6 4/4] doc: add cryptodev service APIs guide Fan Zhang
2020-08-28 12:58 ` Fan Zhang [this message]
2020-08-28 12:58   ` [dpdk-dev] [dpdk-dev v7 1/4] cryptodev: add crypto data-path service APIs Fan Zhang
2020-08-31  6:23     ` Kusztal, ArkadiuszX
2020-08-31 12:21       ` Zhang, Roy Fan
2020-08-31 15:15       ` Zhang, Roy Fan
2020-08-28 12:58   ` [dpdk-dev] [dpdk-dev v7 2/4] crypto/qat: add crypto data-path service API support Fan Zhang
2020-08-28 12:58   ` [dpdk-dev] [dpdk-dev v7 3/4] test/crypto: add unit-test for cryptodev direct APIs Fan Zhang
2020-08-28 12:58   ` [dpdk-dev] [dpdk-dev v7 4/4] doc: add cryptodev service APIs guide Fan Zhang
2020-09-04 15:25   ` [dpdk-dev] [dpdk-dev v8 0/4] cryptodev: add data-path service APIs Fan Zhang
2020-09-04 15:25     ` [dpdk-dev] [dpdk-dev v8 1/4] cryptodev: add crypto " Fan Zhang
2020-09-07 12:36       ` Dybkowski, AdamX
2020-09-04 15:25     ` [dpdk-dev] [dpdk-dev v8 2/4] crypto/qat: add crypto data-path service API support Fan Zhang
2020-09-04 15:25     ` [dpdk-dev] [dpdk-dev v8 3/4] test/crypto: add unit-test for cryptodev direct APIs Fan Zhang
2020-09-04 15:25     ` [dpdk-dev] [dpdk-dev v8 4/4] doc: add cryptodev service APIs guide Fan Zhang
2020-09-08  8:42     ` [dpdk-dev] [dpdk-dev v9 0/4] cryptodev: add data-path service APIs Fan Zhang
2020-09-08  8:42       ` [dpdk-dev] [dpdk-dev v9 1/4] cryptodev: add crypto " Fan Zhang
2020-09-18 21:50         ` Akhil Goyal
2020-09-21 10:40           ` Zhang, Roy Fan
2020-09-21 11:59             ` Akhil Goyal
2020-09-21 15:26               ` Zhang, Roy Fan
2020-09-21 15:41               ` Zhang, Roy Fan
2020-09-21 15:49                 ` Akhil Goyal
2020-09-22  8:08                   ` Zhang, Roy Fan
2020-09-22  8:21                   ` Zhang, Roy Fan
2020-09-22  8:48                     ` Ananyev, Konstantin
2020-09-22  9:05                       ` Akhil Goyal
2020-09-22  9:28                         ` Zhang, Roy Fan
2020-09-22 10:18                           ` Ananyev, Konstantin
2020-09-22 12:15                             ` Zhang, Roy Fan
2020-09-22 12:50                             ` Zhang, Roy Fan
2020-09-22 12:52                               ` Akhil Goyal
2020-09-08  8:42       ` [dpdk-dev] [dpdk-dev v9 2/4] crypto/qat: add crypto data-path service API support Fan Zhang
2020-09-08  8:42       ` [dpdk-dev] [dpdk-dev v9 3/4] test/crypto: add unit-test for cryptodev direct APIs Fan Zhang
2020-09-18 20:03         ` Akhil Goyal
2020-09-21 12:41           ` Zhang, Roy Fan
2020-09-08  8:42       ` [dpdk-dev] [dpdk-dev v9 4/4] doc: add cryptodev service APIs guide Fan Zhang
2020-09-18 20:39         ` Akhil Goyal
2020-09-21 12:28           ` Zhang, Roy Fan
2020-09-23 13:37           ` Zhang, Roy Fan
2020-09-24 16:34       ` [dpdk-dev] [dpdk-dev v10 0/4] cryptodev: add raw data-path APIs Fan Zhang
2020-09-24 16:34         ` [dpdk-dev] [dpdk-dev v10 1/4] cryptodev: change crypto symmetric vector structure Fan Zhang
2020-09-25  8:03           ` Dybkowski, AdamX
2020-09-28 17:01           ` Ananyev, Konstantin
2020-09-24 16:34         ` [dpdk-dev] [dpdk-dev v10 2/4] cryptodev: add raw crypto data-path APIs Fan Zhang
2020-09-25  8:04           ` Dybkowski, AdamX
2020-10-08 14:26           ` Akhil Goyal
2020-10-08 15:29             ` Zhang, Roy Fan
2020-10-08 16:07               ` Akhil Goyal
2020-10-08 16:24                 ` Zhang, Roy Fan
2020-10-09  8:32                 ` Zhang, Roy Fan
2020-10-08 14:37           ` Akhil Goyal
2020-09-24 16:34         ` [dpdk-dev] [dpdk-dev v10 3/4] crypto/qat: add raw crypto data-path API support Fan Zhang
2020-09-25  8:04           ` Dybkowski, AdamX
2020-09-24 16:34         ` [dpdk-dev] [dpdk-dev v10 4/4] test/crypto: add unit-test for cryptodev raw API test Fan Zhang
2020-09-25  8:05           ` Dybkowski, AdamX
2020-10-08 15:01           ` Akhil Goyal
2020-10-08 15:04         ` [dpdk-dev] [dpdk-dev v10 0/4] cryptodev: add raw data-path APIs Akhil Goyal
2020-10-08 15:30           ` Zhang, Roy Fan
2020-10-09 21:11         ` [dpdk-dev] [dpdk-dev v11 " Fan Zhang
2020-10-09 21:11           ` [dpdk-dev] [dpdk-dev v11 1/4] cryptodev: change crypto symmetric vector structure Fan Zhang
2020-10-09 21:11           ` [dpdk-dev] [dpdk-dev v11 2/4] cryptodev: add raw crypto data-path APIs Fan Zhang
2020-10-10 19:38             ` Akhil Goyal
2020-10-10 20:40               ` Zhang, Roy Fan
2020-10-09 21:11           ` [dpdk-dev] [dpdk-dev v11 3/4] crypto/qat: add raw crypto data-path API support Fan Zhang
2020-10-09 21:11           ` [dpdk-dev] [dpdk-dev v11 4/4] test/crypto: add unit-test for cryptodev raw API test Fan Zhang
2020-10-10 19:55             ` Akhil Goyal
2020-10-10 20:50               ` Zhang, Roy Fan
2020-10-10 21:03                 ` Akhil Goyal
2020-10-11  0:32           ` [dpdk-dev] [dpdk-dev v12 0/4] cryptodev: add raw data-path APIs Fan Zhang
2020-10-11  0:32             ` [dpdk-dev] [dpdk-dev v12 1/4] cryptodev: change crypto symmetric vector structure Fan Zhang
2020-10-11  0:32             ` [dpdk-dev] [dpdk-dev v12 2/4] cryptodev: add raw crypto data-path APIs Fan Zhang
2020-10-11  0:32             ` [dpdk-dev] [dpdk-dev v12 3/4] crypto/qat: add raw crypto data-path API support Fan Zhang
2020-10-11  0:32             ` [dpdk-dev] [dpdk-dev v12 4/4] test/crypto: add unit-test for cryptodev raw API test Fan Zhang
2020-10-11  0:38             ` [dpdk-dev] [dpdk-dev v13 0/4] cryptodev: add raw data-path APIs Fan Zhang
2020-10-11  0:38               ` [dpdk-dev] [dpdk-dev v13 1/4] cryptodev: change crypto symmetric vector structure Fan Zhang
2020-10-11  0:38               ` [dpdk-dev] [dpdk-dev v13 2/4] cryptodev: add raw crypto data-path APIs Fan Zhang
2020-10-11  0:38               ` [dpdk-dev] [dpdk-dev v13 3/4] crypto/qat: add raw crypto data-path API support Fan Zhang
2020-10-11  0:38               ` [dpdk-dev] [dpdk-dev v13 4/4] test/crypto: add unit-test for cryptodev raw API test Fan Zhang
2020-10-12 16:15               ` [dpdk-dev] [dpdk-dev v13 0/4] cryptodev: add raw data-path APIs Akhil Goyal

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200828125815.21614-1-roy.fan.zhang@intel.com \
    --to=roy.fan.zhang@intel.com \
    --cc=adamx.dybkowski@intel.com \
    --cc=akhil.goyal@nxp.com \
    --cc=arkadiuszx.kusztal@intel.com \
    --cc=dev@dpdk.org \
    --cc=fiona.trahe@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git