Re: [dpdk-dev] [PATCH v2 0/4] introduce multi-function processing support

[Ferruh Yigit <ferruh.yigit@intel.com>]

On 4/3/2020 5:36 PM, David Coyle wrote:
> PLEASE NOTE: This patchset supercedes the following v1 patches which were
> mistakenly added as stand-alone patches (apologies for any confusion this
> may cause)
> 
> https://patchwork.dpdk.org/patch/66733/
> https://patchwork.dpdk.org/patch/66735/
> https://patchwork.dpdk.org/patch/66736/
> 
> PLEASE NOTE ALSO: Support for QAT, which the following patch addressed,
> has been dropped from this patchset and is now targetted at the next
> release (v20.08)
> 
> https://patchwork.dpdk.org/patch/66819/
> 
> 
> Introduction
> ============
> 
> This patchset adds a new multi-function interface and a aesni_mb raw device
> PMD which uses this interface.
> 
> This patchset has already been discussed as part of the following RFC:
> 
> http://mails.dpdk.org/archives/dev/2020-February/157045.html
> http://mails.dpdk.org/archives/dev/2020-March/159189.html
> 
> The main aim of this interface and raw PMDs is to provide a flexible and
> extensible way of combining one or more packet-processing functions into a
> single operation, thereby allowing these to be performed in parallel in
> optimized software libraries or in a hardware accelerator. These functions
> can include cryptography, compression and CRC/checksum calculation, while
> others can potentially be added in the future. Performing these functions
> in parallel as a single operation can enable a significant performance
> improvement.
> 
> 
> Background
> ==========
> 
> There are a number of byte-wise operations which are present and common
> across many access network data-plane pipelines, such as Cipher,
> Authentication, CRC, Bit-Interleaved-Parity (BIP), other checksums etc.
> Some prototyping has been done at Intel in relation to the 01.org
> access-network-dataplanes project to prove that a significant performance
> improvement is possible when such byte-wise operations are combined into a
> single pass of packet data processing. This performance boost has been
> prototyped for both XGS-PON MAC data-plane and DOCSIS MAC data-plane
> pipelines.
> 
> The prototypes used some protocol-specific modifications to the DPDK
> cryptodev library. In order to make this performance improvement consumable
> by network access equipment vendors, a more extensible and correct solution
> was required.
> 
> Hence, the introduction of a multi-function interface, initially for use by
> raw devices. In this patchset, a new aesni_mb raw device has been created
> which uses this interface.
> 
> NOTE: In a future DPDK release (currently targetting DPDK v20.08), a qat
> raw device will also be added. As multiple raw devices will share the same
> interface, the approach taken was to create a common interface
> (i.e. multi-function) which can be used by these devices. This both cuts
> down on code duplication across the devices and allows an application
> access multiple devices using the same interface.
> 
> 
> Use Cases
> =========
> 
> The primary use cases for the multi-function interface and raw PMDs have
> already been mentioned. These are:
> 
> - DOCSIS MAC: Crypto-CRC
> 	- Order:
> 		- Downstream: CRC, Encrypt
> 		- Upstream: Decrypt, CRC
> 	- Specifications:
> 		- Crypto: 128-bit AES-CFB encryption variant for DOCSIS as
> 		  described in section 11.1 of DOCSIS 3.1 Security
> 		  Specification
> 		  (https://apps.cablelabs.com/specification/CM-SP-SECv3.1)
> 		- CRC: Ethernet 32-bit CRC as defined in
> 		  Ethernet/[ISO/IEC 8802-3]
> 
> - XGS-PON MAC: Crypto-CRC-BIP
> 	- Order:
> 		- Downstream: CRC, Encrypt, BIP
> 		- Upstream: BIP, Decrypt, CRC
> 	- Specifications:
> 		- Crypto: AES-128 [NIST FIPS-197] cipher, used in counter
> 		  mode (AES-CTR), as described in [NIST SP800-38A].
> 		- CRC: Ethernet 32-bit CRC as defined in
> 		  Ethernet/[ISO/IEC 8802-3]
> 		- BIP: 4-byte bit-interleaved even parity (BIP) field
> 		  computed over the entire FS frame, refer to
> 		  ITU-T G.989.3, sections 8.1.1.5 and 8.1.2.3
> 		  (https://www.itu.int/rec/dologin_pub.asp?lang=e&id=
> 		   T-REC-G.989.3-201510-I!!PDF-E)
> 
> Note that support for both these chained operations is already available in
> the Intel IPSec Multi-Buffer library.
> 
> However, it is not limited to these. The following are some of the other
> possible use-cases, which multi-function will allow for:
> 
> - Storage:
> 	- Compression followed by Encryption
> - IPSec over UDP:
> 	- UDP Checksum calculation followed by Encryption
> 
> While DPDK's rte_cryptodev and rte_compressdev allow many cryptographic and
> compression algorithms to be chained together in one operation, there is no
> way to chain these with any error detection or checksum algorithms. And
> there is no way to chain crypto and compression algorithms together. The
> multi-function  interface will allow these chains to be created, and also
> allow any future type of operation to be easily added.

I was thinking if the cryptodev can be used instead but this paragraph already
seems explained it. But again can you please elaborate why rawdev is used?

> 
> 
> Architecture
> ============
> 
> The following diagram shows where the multi-function interface and raw
> devices fit in an overall application architecture.
> 
>   +------------------------------------------------+
>   |                                                |
>   |                  Application                   |
>   |   (e.g. vCMTS (DOCSIS), vOLT (XGS-PON), etc.)  |
>   |                                                |
>   +------------------------------------------------+
>                           |
>   +-----------------------|------------------------+
>   |                       |                  DPDK  |
>   |                       |                        |
>   |             +---------------------+            |
>   |             |                     |            |
>   |             |     rte_rawdev      |            |
>   |             |                     |            |            NOTE:
>   |             +---------------------+        ____|______ 'MULTI-FUNCTION
>   |                    /      \              /     |          INTERFACE'
>   |                   /        \            /      |         is opaque to
>   |                  /          \          /       |          rte_rawdev
>   |       +--------------------------------+       |
>   |       |    MULTI-FUNCTION INTERFACE    |       |
>   |       +--------------------------------+       |
>   |       +------------+      +------------+       |
>   |       |   RAWDEV   |      |   RAWDEV   |       |
>   |       |  AESNI-MB  |      |    QAT     |       |
>   |       |    PMD     |      |    PMD     |       |
>   |       +------------+      +------------+       |              NOTE:
>   |              |                  |     \________|_____  'RAWDEV QAT PMD'
>   +--------------|------------------|--------------+       will be added in
>                  |                  |                         next release
>           +------------+      +------------+
>           |  AESNI-MB  |      |   QAT HW   |
>           |   SW LIB   |      |            |
>           +------------+      +------------+
> 
> David Coyle (4):
>   raw/common: add multi-function interface
>   raw/aesni_mb: add aesni_mb raw device
>   test/rawdev: add aesni_mb raw device tests
>   app/crypto-perf: add support for multi-function processing
> 

<...>