Re: [dpdk-dev] [RFC] eventdev: add crypto adapter API header

[Jerin Jacob <jerin.jacob@caviumnetworks.com>]

-----Original Message-----
> Date: Thu, 9 Nov 2017 12:24:13 +0530
> From: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
> To: jerin.jacob@caviumnetworks.com
> CC: dev@dpdk.org, narender.vangati@intel.com, Abhinandan Gujjar
>  <abhinandan.gujjar@intel.com>, Nikhil Rao <nikhil.rao@intel.com>, Gage
>  Eads <gage.eads@intel.com>
> Subject: [RFC] eventdev: add crypto adapter API header
> X-Mailer: git-send-email 1.9.1
> 
> Signed-off-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
> Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
> Signed-off-by: Gage Eads <gage.eads@intel.com>
> ---
>  lib/librte_eventdev/Makefile                   |   1 +
>  lib/librte_eventdev/rte_event_crypto_adapter.h | 474 +++++++++++++++++++++++++
>  2 files changed, 475 insertions(+)
>  create mode 100644 lib/librte_eventdev/rte_event_crypto_adapter.h
> 
> diff --git a/lib/librte_eventdev/Makefile b/lib/librte_eventdev/Makefile
> index 5ac22cd..9cbe1a6 100644
> --- a/lib/librte_eventdev/Makefile
> +++ b/lib/librte_eventdev/Makefile
> @@ -53,6 +53,7 @@ SYMLINK-y-include += rte_eventdev_pmd_pci.h
>  SYMLINK-y-include += rte_eventdev_pmd_vdev.h
>  SYMLINK-y-include += rte_event_ring.h
>  SYMLINK-y-include += rte_event_eth_rx_adapter.h
> +SYMLINK-y-include += rte_event_crypto_adapter.h
>  
>  # versioning export map
>  EXPORT_MAP := rte_eventdev_version.map
> diff --git a/lib/librte_eventdev/rte_event_crypto_adapter.h b/lib/librte_eventdev/rte_event_crypto_adapter.h
> new file mode 100644
> index 0000000..080c3ed
> --- /dev/null
> +++ b/lib/librte_eventdev/rte_event_crypto_adapter.h
> @@ -0,0 +1,474 @@
> +/*
> + *   Copyright(c) 2017 Intel Corporation. All rights reserved.
> + *   All rights reserved.
> + *
> + *   Redistribution and use in source and binary forms, with or without
> + *   modification, are permitted provided that the following conditions
> + *   are met:
> + *
> + *     * Redistributions of source code must retain the above copyright
> + *       notice, this list of conditions and the following disclaimer.
> + *     * Redistributions in binary form must reproduce the above copyright
> + *       notice, this list of conditions and the following disclaimer in
> + *       the documentation and/or other materials provided with the
> + *       distribution.
> + *     * Neither the name of Intel Corporation nor the names of its
> + *       contributors may be used to endorse or promote products derived
> + *       from this software without specific prior written permission.
> + *
> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> + */
> +
> +#ifndef _RTE_EVENT_CRYPTO_ADAPTER_
> +#define _RTE_EVENT_CRYPTO_ADAPTER_
> +
> +/**
> + * This adapter adds support to enqueue crypto completion to event device.
> + * The packet flow from cryptodev to the event device can be accomplished
> + * using either HW or SW mechanisms.
> + * The adapter uses a EAL service core function for SW based packet transfer
> + * and uses the eventdev PMD functions to configure HW based packet transfer
> + * between the cryptodev and the event device.
> + *
> + * The event crypto adapter provides common APIs to configure the packet flow
> + * from the cryptodev to event devices on both HW and SW.
> + * The crypto event adapter's functions are:
> + *  - rte_event_crypto_adapter_create_ext()
> + *  - rte_event_crypto_adapter_create()
> + *  - rte_event_crypto_adapter_free()
> + *  - rte_event_crypto_adapter_queue_pair_add()
> + *  - rte_event_crypto_adapter_queue_pair_del()
> + *  - rte_event_crypto_adapter_start()
> + *  - rte_event_crypto_adapter_stop()
> + *  - rte_event_crypto_adapter_stats_get()
> + *  - rte_event_crypto_adapter_stats_reset()
> +
> + * The applicaton creates an instance using rte_event_crypto_adapter_create()
> + * or rte_event_crypto_adapter_create_ext().
> + *
> + * Cryptodev queue pair addition/deletion is done
> + * using rte_event_crypto_adapter_queue_pair_xxx() API.
> + *
> + * Adapter uses rte_event_crypto_queue_pair_conf to decide whether the event
> + * enqueue is based on RTE_EVENT_CRYPTO_ENQ_MULTI_EVENTQ or
> + * RTE_EVENT_CRYPTO_ENQ_MBUF_MULTI_EVENTQ.
> + * In case of RTE_EVENT_CRYPTO_ENQ_MULTI_EVENTQ,
> + * rte_event_crypto_queue_pair_conf::ev will be used for event enqueue.
> + * In case of RTE_EVENT_CRYPTO_ENQ_MBUF_MULTI_EVENTQ,
> + * members of rte_event_crypto_metadata will be used for event enqueue.

Adding Declan and Hemant.

IMO, RTE_EVENT_CRYPTO_ENQ_MULTI_EVENTQ may not be very useful
from application perceptive as the scope is very limited.
In real world use cases, we will be attaching destination event queue information
to the session, not to the queue pair.


IMO, RTE_EVENT_CRYPTO_ENQ_MBUF_MULTI_EVENTQ scheme may not very
convenient for application writers as
# it relies on mbuf private area memory. So it has additional memory alloc/free
requirements.
# additional overhead for application/PMD to write/read the event queue metadata
information per packet.

Since we already have meta data structure in the crypto
area, How about adding the destination event queue attributes
in the PMD crypto session area and for, _session less_, we can add it
in rte_crypto_op stricture? This will help in:

# Offloading HW specific meta data generation for event queue attributes
to slow path.
# From the application perspective, most likely the event queue parameters
will be different for each session not per packet nor per event queue
pair.

Something like below to share my view. Exact details may be we can work it out.

➜ [master][dpdk.org] $ git diff
diff --git a/lib/librte_cryptodev/rte_crypto.h
b/lib/librte_cryptodev/rte_crypto.h
index 3d672fe7d..b44ef673b 100644
--- a/lib/librte_cryptodev/rte_crypto.h
+++ b/lib/librte_cryptodev/rte_crypto.h
@@ -115,6 +115,9 @@ struct rte_crypto_op {
 
        uint8_t reserved[5];
        /**< Reserved bytes to fill 64 bits for future additions */

+#if 0
+ Crypto completion event attribute. For _session less_ crypto enqueue operation,
+ The will field shall be used by application to post the crypto completion event
+ upon the crypto enqueue operation complete.

+ Note: In the case of session based crypto operation, SW based crypto adapter can use
+ this memory to store crypto event completion attributes from the PMD
+ specific session area.
+
+ Note: ev.event_ptr will point to struct rte_crypto_op *op, So
+ that core can free the ops memory on event_dequeue().
+#endif
+
+       struct rte_event ev;

        struct rte_mempool *mempool;
        /**< crypto operation mempool which operation is allocated from
 * */
 
diff --git a/lib/librte_cryptodev/rte_cryptodev.h
b/lib/librte_cryptodev/rte_cryptodev.h
index dade5548f..540b29e66 100644
--- a/lib/librte_cryptodev/rte_cryptodev.h
+++ b/lib/librte_cryptodev/rte_cryptodev.h
@@ -945,6 +945,13 @@ rte_cryptodev_sym_session_init(uint8_t dev_id,
                        struct rte_crypto_sym_xform *xforms,
                        struct rte_mempool *mempool);

+#if 0
+ Create PMD specific session meta data for the destination event queue
+ attribute to post the crypto completion event on crypto work complete.
+#endif
+int
+rte_cryptodev_sym_session_event_init(uint8_t dev_id,
+                       struct rte_cryptodev_sym_session *sess,
+                       struct rte_crypto_sym_xform *xforms,
+                       struct rte_mempool *mempool,
+                       struct rte_event ev);
+
 /**
  * Frees private data for the device id, based on its device type,
  * returning it to its mempool.


> + *
> + * The metadata offset is used to configure the location of the
> + * rte_event_crypto_metadata structure within the mbuf's private metadata area.
> + *
> + * When the application sends crypto operations to the adapter,
> + * the crypto queue pair identifier needs to be specified, similarly eventdev
> + * parameters such as the flow id, scheduling type etc are needed by the
> + * adapter when it enqueues mbufs from completed crypto operations to eventdev.
> + */
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +#include <stdint.h>
> +#include <rte_service.h>
> +
> +#include "rte_eventdev.h"
> +
> +#define RTE_EVENT_CRYPTO_ADAPTER_MAX_INSTANCE 32
> +
> + /**
> + * @warning
> + * @b EXPERIMENTAL: this enum may change without prior notice
> + *
> + * Crypto event queue conf type
> + */
> +enum rte_event_crypto_conf_type {
> +	RTE_EVENT_CRYPTO_CONF_TYPE_EVENT = 1,
> +	/**< Refer RTE_EVENT_CRYPTO_ADAPTER_CAP_MULTI_EVENTQ */
> +	RTE_EVENT_CRYPTO_CONF_TYPE_MBUF,
> +	/**< Refer RTE_EVENT_CRYPTO_ADAPTER_CAP_MBUF_MULTI_EVENTQ */
> +	RTE_EVENT_CRYPTO_CONF_TYPE_MAX
> +};

See above.

> +
> + /**
> + * @warning
> + * @b EXPERIMENTAL: this enum may change without prior notice
> + *
> + * Crypto event adapter type
> + */
> +enum rte_event_crypto_adapter_type {
> +	RTE_EVENT_CRYPTO_ADAPTER_RX_ONLY = 1,
> +	/**< Start only Rx part of crypto adapter.
> +	* Packets dequeued from cryptodev are new to eventdev and
> +	* events will be treated as RTE_EVENT_OP_NEW */
> +	RTE_EVENT_CRYPTO_ADAPTER_RX_TX,
> +	/**< Start both Rx & Tx part of crypto adapter.
> +	* Packet's event context will be retained and
> +	* event will be treated as RTE_EVENT_OP_FORWARD */
> +};

How about leveraging ev.op based schematics as mentioned above?