[dpdk-dev] [PATCH v8 00/10] Crypto API and device framework

[Declan Doherty <declan.doherty@intel.com>]

This series of patches defines a set of application burst oriented APIs for
asynchronous symmetric cryptographic functions within DPDK. It also contains a
poll mode driver cryptographic device framework for the implementation of
crypto devices within DPDK.

In the patch set we also have included 2 reference implementations of crypto
PMDs. Currently both implementations support AES-CBC with
HMAC_SHA1/SHA256/SHA512 authentication operations. The first device is a purely
software PMD based on Intel's multi-buffer library, which utilises both
AES-NI instructions and vector operations to accelerate crypto operations and
the second PMD utilises Intel's Quick Assist Technology (on DH895xxC) to
provide hardware accelerated crypto operations.

The API set supports two functional modes of operation:

1, A session oriented mode. In this mode the user creates a crypto session
which defines all the immutable data required to perform a particular crypto
operation in advance, including cipher/hash algorithms and operations to be
performed as well as the keys to used etc. The session is then referenced by
the crypto operation data structure which is a data structure specific to each
mbuf. It is contains all mutable data about the crypto operation to be
performed, such as data offsets and lengths into the mbuf's data payload for
cipher and hash operations to be performed.

2, A session-less mode. In this mode the user is able to provision crypto
operations on an mbuf without the need to have a cached session created in
advance, but at the cost of entailing the overhead of calculating
authentication pre-computes and preforming key expansions in-line with the
crypto operation. The crypto xform chain is directly attached to the op struct
in this mode, so the op struct now contains all of the immutable crypto
operation parameters that would be normally set within a session. Once all
mutable and immutable parameters are set the crypto operation data structure
can be attached to the specified mbuf and enqueued on a specified crypto device
for processing.

The patch set contains the following features:
 - Crypto device APIs and device framework
 - Implementation of a software crypto PMD based on multi-buffer library
 - Implementation of a hardware crypto PMD baed on Intel QAT(DH895xxC)
 - Unit and performance test's which give and example of utilising the crypto
 API's.
 - Sample application which performs crypto operations on the IP payload of the
   packets being forwarded

Current Status:
There is no support for chained mbuf's and as mentioned above the PMD's
have currently implemented support for AES128-CBC/AES192-CBC/AES256-CBC
and HMAC_SHA1/SHA256/SHA512 and AES_XCBC_MAC.

v8:
  - Doxygen comment fix for rte_pktmbuf_mtophys macro
  - Doxygen fixes for public headers in rte_crypto.h
  - QAT documentation tidy up based on J. McNamara comments
  - Detailed requirement to set YASM path when building multi-buffer  library
  - l2fwd-crypto: fix for 32-bit build; fix for possible memory leak if 
    rte_cryptodev_burst_enqueue fails; and handling for failure to  allocate
    rte_mbuf_offload.

v7:
  - Fix typos in commit message of eal: add __rte_packed /__rte_aligned macros patch
  - Include rte_mbuf_offload in doxygen build and updates file comments to clarify lib,
    usage. Also moved clean which was in wrong patch into this
    rte_mbuf_offload patch.
  - Tidy up map file for cryptodev library.
  - Add l2fwdc-crypto to main examples makefile.
v6:
  - Fix 32-bit build issue caused by casting in new  rte_pktmbuf_mtophys_offset macro
  - Fix truncation of log message by new  rte_pmd_debug_trace inline function

v5:
  - Making ethdev marcos for function pointer and port id checking public and
    available for use in by the cryptodev. The intialise to patches combine changes
    from original cryptodev patch and discussion in
    http://dpdk.org/ml/archives/dev/2015-November/027871.html
  - Split out changes to create new __rte_packed and __rte_aligned macros 
    into seperate patches form the main  cryptodev patch set for clairty
  - further code cleaning, removal of currently unsupported gcm code from  aesni_mb pmd
 
v4:
  - Some more EOF whitespace and  checkpatch fixes

v3:
  - Fixes a document build error, which I missed in the V2
  - Fixes for remaining checkpatch errors
  - Disables QAT and AESNI_MB PMD being build by default as they have external 
    library dependences 

v2: 
 - Introduces a new library to support attaching offload operations to a mbuf
 - Remove unused APIs from cryptodev
 - PMD code refactor due to  new rte_mbuf_offload structure
 - General bug fixes and code tidy up


Declan Doherty (10):
  ethdev: rename macros to have RTE_ prefix
  ethdev: make error checking macros public
  eal: add __rte_packed /__rte_aligned macros
  mbuf: add new marcos to get the physical address of data
  cryptodev: Initial DPDK Crypto APIs and device framework release
  mbuf_offload: library to support attaching offloads to a mbuf
  qat_crypto_pmd: Addition of a new QAT DPDK PMD.
  aesni_mb_pmd: Initial implementation of multi buffer based crypto
    device
  app/test: add cryptodev unit and performance tests
  l2fwd-crypto: crypto

 MAINTAINERS                                        |   14 +
 app/test/Makefile                                  |    4 +
 app/test/test.c                                    |   92 +-
 app/test/test.h                                    |   34 +-
 app/test/test_cryptodev.c                          | 1986 +++++++++++++++++++
 app/test/test_cryptodev.h                          |   68 +
 app/test/test_cryptodev_perf.c                     | 2062 ++++++++++++++++++++
 app/test/test_link_bonding.c                       |    6 +-
 app/test/test_link_bonding_mode4.c                 |    7 +-
 app/test/test_link_bonding_rssconf.c               |    7 +-
 config/common_bsdapp                               |   37 +-
 config/common_linuxapp                             |   37 +-
 doc/api/doxy-api-index.md                          |    2 +
 doc/api/doxy-api.conf                              |    2 +
 doc/guides/cryptodevs/aesni_mb.rst                 |   85 +
 doc/guides/cryptodevs/index.rst                    |   39 +
 doc/guides/cryptodevs/qat.rst                      |  219 +++
 doc/guides/index.rst                               |    1 +
 drivers/Makefile                                   |    1 +
 drivers/crypto/Makefile                            |   38 +
 drivers/crypto/aesni_mb/Makefile                   |   63 +
 drivers/crypto/aesni_mb/aesni_mb_ops.h             |  210 ++
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c         |  669 +++++++
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c     |  298 +++
 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h |  229 +++
 drivers/crypto/aesni_mb/rte_pmd_aesni_version.map  |    3 +
 drivers/crypto/qat/Makefile                        |   63 +
 .../qat/qat_adf/adf_transport_access_macros.h      |  174 ++
 drivers/crypto/qat/qat_adf/icp_qat_fw.h            |  316 +++
 drivers/crypto/qat/qat_adf/icp_qat_fw_la.h         |  404 ++++
 drivers/crypto/qat/qat_adf/icp_qat_hw.h            |  306 +++
 drivers/crypto/qat/qat_adf/qat_algs.h              |  125 ++
 drivers/crypto/qat/qat_adf/qat_algs_build_desc.c   |  601 ++++++
 drivers/crypto/qat/qat_crypto.c                    |  561 ++++++
 drivers/crypto/qat/qat_crypto.h                    |  124 ++
 drivers/crypto/qat/qat_logs.h                      |   78 +
 drivers/crypto/qat/qat_qp.c                        |  429 ++++
 drivers/crypto/qat/rte_pmd_qat_version.map         |    3 +
 drivers/crypto/qat/rte_qat_cryptodev.c             |  137 ++
 examples/Makefile                                  |    1 +
 examples/l2fwd-crypto/Makefile                     |   50 +
 examples/l2fwd-crypto/main.c                       | 1489 ++++++++++++++
 lib/Makefile                                       |    2 +
 lib/librte_cryptodev/Makefile                      |   60 +
 lib/librte_cryptodev/rte_crypto.h                  |  610 ++++++
 lib/librte_cryptodev/rte_cryptodev.c               | 1092 +++++++++++
 lib/librte_cryptodev/rte_cryptodev.h               |  651 ++++++
 lib/librte_cryptodev/rte_cryptodev_pmd.h           |  549 ++++++
 lib/librte_cryptodev/rte_cryptodev_version.map     |   32 +
 lib/librte_eal/common/include/rte_dev.h            |   53 +
 lib/librte_eal/common/include/rte_log.h            |    1 +
 lib/librte_eal/common/include/rte_memory.h         |   14 +-
 lib/librte_ether/rte_ethdev.c                      |  619 +++---
 lib/librte_ether/rte_ethdev.h                      |   26 +
 lib/librte_mbuf/rte_mbuf.h                         |   27 +
 lib/librte_mbuf_offload/Makefile                   |   52 +
 lib/librte_mbuf_offload/rte_mbuf_offload.c         |  100 +
 lib/librte_mbuf_offload/rte_mbuf_offload.h         |  302 +++
 .../rte_mbuf_offload_version.map                   |    7 +
 mk/rte.app.mk                                      |    9 +
 60 files changed, 14891 insertions(+), 389 deletions(-)
 create mode 100644 app/test/test_cryptodev.c
 create mode 100644 app/test/test_cryptodev.h
 create mode 100644 app/test/test_cryptodev_perf.c
 create mode 100644 doc/guides/cryptodevs/aesni_mb.rst
 create mode 100644 doc/guides/cryptodevs/index.rst
 create mode 100644 doc/guides/cryptodevs/qat.rst
 create mode 100644 drivers/crypto/Makefile
 create mode 100644 drivers/crypto/aesni_mb/Makefile
 create mode 100644 drivers/crypto/aesni_mb/aesni_mb_ops.h
 create mode 100644 drivers/crypto/aesni_mb/rte_aesni_mb_pmd.c
 create mode 100644 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_ops.c
 create mode 100644 drivers/crypto/aesni_mb/rte_aesni_mb_pmd_private.h
 create mode 100644 drivers/crypto/aesni_mb/rte_pmd_aesni_version.map
 create mode 100644 drivers/crypto/qat/Makefile
 create mode 100644 drivers/crypto/qat/qat_adf/adf_transport_access_macros.h
 create mode 100644 drivers/crypto/qat/qat_adf/icp_qat_fw.h
 create mode 100644 drivers/crypto/qat/qat_adf/icp_qat_fw_la.h
 create mode 100644 drivers/crypto/qat/qat_adf/icp_qat_hw.h
 create mode 100644 drivers/crypto/qat/qat_adf/qat_algs.h
 create mode 100644 drivers/crypto/qat/qat_adf/qat_algs_build_desc.c
 create mode 100644 drivers/crypto/qat/qat_crypto.c
 create mode 100644 drivers/crypto/qat/qat_crypto.h
 create mode 100644 drivers/crypto/qat/qat_logs.h
 create mode 100644 drivers/crypto/qat/qat_qp.c
 create mode 100644 drivers/crypto/qat/rte_pmd_qat_version.map
 create mode 100644 drivers/crypto/qat/rte_qat_cryptodev.c
 create mode 100644 examples/l2fwd-crypto/Makefile
 create mode 100644 examples/l2fwd-crypto/main.c
 create mode 100644 lib/librte_cryptodev/Makefile
 create mode 100644 lib/librte_cryptodev/rte_crypto.h
 create mode 100644 lib/librte_cryptodev/rte_cryptodev.c
 create mode 100644 lib/librte_cryptodev/rte_cryptodev.h
 create mode 100644 lib/librte_cryptodev/rte_cryptodev_pmd.h
 create mode 100644 lib/librte_cryptodev/rte_cryptodev_version.map
 create mode 100644 lib/librte_mbuf_offload/Makefile
 create mode 100644 lib/librte_mbuf_offload/rte_mbuf_offload.c
 create mode 100644 lib/librte_mbuf_offload/rte_mbuf_offload.h
 create mode 100644 lib/librte_mbuf_offload/rte_mbuf_offload_version.map

-- 
2.5.0