[dpdk-dev] [PATCH v2 0/4] introduce multi-function processing support

[David Coyle <david.coyle@intel.com>]

PLEASE NOTE: This patchset supercedes the following v1 patches which were
mistakenly added as stand-alone patches (apologies for any confusion this
may cause)

https://patchwork.dpdk.org/patch/66733/
https://patchwork.dpdk.org/patch/66735/
https://patchwork.dpdk.org/patch/66736/

PLEASE NOTE ALSO: Support for QAT, which the following patch addressed,
has been dropped from this patchset and is now targetted at the next
release (v20.08)

https://patchwork.dpdk.org/patch/66819/


Introduction
============

This patchset adds a new multi-function interface and a aesni_mb raw device
PMD which uses this interface.

This patchset has already been discussed as part of the following RFC:

http://mails.dpdk.org/archives/dev/2020-February/157045.html
http://mails.dpdk.org/archives/dev/2020-March/159189.html

The main aim of this interface and raw PMDs is to provide a flexible and
extensible way of combining one or more packet-processing functions into a
single operation, thereby allowing these to be performed in parallel in
optimized software libraries or in a hardware accelerator. These functions
can include cryptography, compression and CRC/checksum calculation, while
others can potentially be added in the future. Performing these functions
in parallel as a single operation can enable a significant performance
improvement.


Background
==========

There are a number of byte-wise operations which are present and common
across many access network data-plane pipelines, such as Cipher,
Authentication, CRC, Bit-Interleaved-Parity (BIP), other checksums etc.
Some prototyping has been done at Intel in relation to the 01.org
access-network-dataplanes project to prove that a significant performance
improvement is possible when such byte-wise operations are combined into a
single pass of packet data processing. This performance boost has been
prototyped for both XGS-PON MAC data-plane and DOCSIS MAC data-plane
pipelines.

The prototypes used some protocol-specific modifications to the DPDK
cryptodev library. In order to make this performance improvement consumable
by network access equipment vendors, a more extensible and correct solution
was required.

Hence, the introduction of a multi-function interface, initially for use by
raw devices. In this patchset, a new aesni_mb raw device has been created
which uses this interface.

NOTE: In a future DPDK release (currently targetting DPDK v20.08), a qat
raw device will also be added. As multiple raw devices will share the same
interface, the approach taken was to create a common interface
(i.e. multi-function) which can be used by these devices. This both cuts
down on code duplication across the devices and allows an application
access multiple devices using the same interface.


Use Cases
=========

The primary use cases for the multi-function interface and raw PMDs have
already been mentioned. These are:

- DOCSIS MAC: Crypto-CRC
	- Order:
		- Downstream: CRC, Encrypt
		- Upstream: Decrypt, CRC
	- Specifications:
		- Crypto: 128-bit AES-CFB encryption variant for DOCSIS as
		  described in section 11.1 of DOCSIS 3.1 Security
		  Specification
		  (https://apps.cablelabs.com/specification/CM-SP-SECv3.1)
		- CRC: Ethernet 32-bit CRC as defined in
		  Ethernet/[ISO/IEC 8802-3]

- XGS-PON MAC: Crypto-CRC-BIP
	- Order:
		- Downstream: CRC, Encrypt, BIP
		- Upstream: BIP, Decrypt, CRC
	- Specifications:
		- Crypto: AES-128 [NIST FIPS-197] cipher, used in counter
		  mode (AES-CTR), as described in [NIST SP800-38A].
		- CRC: Ethernet 32-bit CRC as defined in
		  Ethernet/[ISO/IEC 8802-3]
		- BIP: 4-byte bit-interleaved even parity (BIP) field
		  computed over the entire FS frame, refer to
		  ITU-T G.989.3, sections 8.1.1.5 and 8.1.2.3
		  (https://www.itu.int/rec/dologin_pub.asp?lang=e&id=
		   T-REC-G.989.3-201510-I!!PDF-E)

Note that support for both these chained operations is already available in
the Intel IPSec Multi-Buffer library.

However, it is not limited to these. The following are some of the other
possible use-cases, which multi-function will allow for:

- Storage:
	- Compression followed by Encryption
- IPSec over UDP:
	- UDP Checksum calculation followed by Encryption

While DPDK's rte_cryptodev and rte_compressdev allow many cryptographic and
compression algorithms to be chained together in one operation, there is no
way to chain these with any error detection or checksum algorithms. And
there is no way to chain crypto and compression algorithms together. The
multi-function  interface will allow these chains to be created, and also
allow any future type of operation to be easily added.


Architecture
============

The following diagram shows where the multi-function interface and raw
devices fit in an overall application architecture.

  +------------------------------------------------+
  |                                                |
  |                  Application                   |
  |   (e.g. vCMTS (DOCSIS), vOLT (XGS-PON), etc.)  |
  |                                                |
  +------------------------------------------------+
                          |
  +-----------------------|------------------------+
  |                       |                  DPDK  |
  |                       |                        |
  |             +---------------------+            |
  |             |                     |            |
  |             |     rte_rawdev      |            |
  |             |                     |            |            NOTE:
  |             +---------------------+        ____|______ 'MULTI-FUNCTION
  |                    /      \              /     |          INTERFACE'
  |                   /        \            /      |         is opaque to
  |                  /          \          /       |          rte_rawdev
  |       +--------------------------------+       |
  |       |    MULTI-FUNCTION INTERFACE    |       |
  |       +--------------------------------+       |
  |       +------------+      +------------+       |
  |       |   RAWDEV   |      |   RAWDEV   |       |
  |       |  AESNI-MB  |      |    QAT     |       |
  |       |    PMD     |      |    PMD     |       |
  |       +------------+      +------------+       |              NOTE:
  |              |                  |     \________|_____  'RAWDEV QAT PMD'
  +--------------|------------------|--------------+       will be added in
                 |                  |                         next release
          +------------+      +------------+
          |  AESNI-MB  |      |   QAT HW   |
          |   SW LIB   |      |            |
          +------------+      +------------+

David Coyle (4):
  raw/common: add multi-function interface
  raw/aesni_mb: add aesni_mb raw device
  test/rawdev: add aesni_mb raw device tests
  app/crypto-perf: add support for multi-function processing

 app/test-crypto-perf/Makefile                 |    5 +
 app/test-crypto-perf/cperf_ops.c              |  265 +++
 app/test-crypto-perf/cperf_options.h          |   37 +-
 app/test-crypto-perf/cperf_options_parsing.c  |  396 ++++-
 app/test-crypto-perf/cperf_test_common.c      |   88 +-
 app/test-crypto-perf/cperf_test_latency.c     |  176 +-
 .../cperf_test_pmd_cyclecount.c               |   96 +-
 app/test-crypto-perf/cperf_test_throughput.c  |  164 +-
 .../cperf_test_vector_parsing.c               |   35 +-
 app/test-crypto-perf/cperf_test_vectors.c     |   53 +
 app/test-crypto-perf/cperf_test_vectors.h     |    9 +
 app/test-crypto-perf/cperf_test_verify.c      |  205 ++-
 app/test-crypto-perf/main.c                   |  255 ++-
 app/test-crypto-perf/meson.build              |    6 +
 app/test/test_rawdev.c                        |   16 +
 config/common_base                            |   11 +
 drivers/meson.build                           |    5 +
 drivers/raw/Makefile                          |    3 +
 drivers/raw/aesni_mb/Makefile                 |   47 +
 drivers/raw/aesni_mb/aesni_mb_rawdev.c        | 1536 +++++++++++++++++
 drivers/raw/aesni_mb/aesni_mb_rawdev.h        |  112 ++
 drivers/raw/aesni_mb/aesni_mb_rawdev_test.c   | 1102 ++++++++++++
 .../aesni_mb/aesni_mb_rawdev_test_vectors.h   | 1183 +++++++++++++
 drivers/raw/aesni_mb/meson.build              |   26 +
 .../aesni_mb/rte_rawdev_aesni_mb_version.map  |    3 +
 drivers/raw/common/Makefile                   |    8 +
 drivers/raw/common/meson.build                |    7 +
 drivers/raw/common/multi_fn/Makefile          |   27 +
 drivers/raw/common/multi_fn/meson.build       |    9 +
 .../multi_fn/rte_common_multi_fn_version.map  |   11 +
 drivers/raw/common/multi_fn/rte_multi_fn.c    |  166 ++
 drivers/raw/common/multi_fn/rte_multi_fn.h    |  350 ++++
 .../raw/common/multi_fn/rte_multi_fn_driver.h |   55 +
 drivers/raw/meson.build                       |    3 +-
 meson.build                                   |    4 +
 mk/rte.app.mk                                 |    3 +
 36 files changed, 6270 insertions(+), 207 deletions(-)
 create mode 100644 drivers/raw/aesni_mb/Makefile
 create mode 100644 drivers/raw/aesni_mb/aesni_mb_rawdev.c
 create mode 100644 drivers/raw/aesni_mb/aesni_mb_rawdev.h
 create mode 100644 drivers/raw/aesni_mb/aesni_mb_rawdev_test.c
 create mode 100644 drivers/raw/aesni_mb/aesni_mb_rawdev_test_vectors.h
 create mode 100644 drivers/raw/aesni_mb/meson.build
 create mode 100644 drivers/raw/aesni_mb/rte_rawdev_aesni_mb_version.map
 create mode 100644 drivers/raw/common/Makefile
 create mode 100644 drivers/raw/common/meson.build
 create mode 100644 drivers/raw/common/multi_fn/Makefile
 create mode 100644 drivers/raw/common/multi_fn/meson.build
 create mode 100644 drivers/raw/common/multi_fn/rte_common_multi_fn_version.map
 create mode 100644 drivers/raw/common/multi_fn/rte_multi_fn.c
 create mode 100644 drivers/raw/common/multi_fn/rte_multi_fn.h
 create mode 100644 drivers/raw/common/multi_fn/rte_multi_fn_driver.h

-- 
2.17.1