Re: [dpdk-dev] [PATCH v5 2/8] eventdev: introduce event vector Rx capability

["Jayatheerthan, Jay" <jay.jayatheerthan@intel.com>]

> -----Original Message-----
> From: pbhagavatula@marvell.com <pbhagavatula@marvell.com>
> Sent: Wednesday, March 24, 2021 10:35 AM
> To: jerinj@marvell.com; Jayatheerthan, Jay <jay.jayatheerthan@intel.com>; Carrillo, Erik G <erik.g.carrillo@intel.com>; Gujjar,
> Abhinandan S <abhinandan.gujjar@intel.com>; McDaniel, Timothy <timothy.mcdaniel@intel.com>; hemant.agrawal@nxp.com; Van
> Haaren, Harry <harry.van.haaren@intel.com>; mattias.ronnblom <mattias.ronnblom@ericsson.com>; Ma, Liang J
> <liang.j.ma@intel.com>; Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
> Cc: dev@dpdk.org; Pavan Nikhilesh <pbhagavatula@marvell.com>
> Subject: [dpdk-dev] [PATCH v5 2/8] eventdev: introduce event vector Rx capability
> 
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Introduce event ethernet Rx adapter event vector capability.
> 
> If an event eth Rx adapter has the capability of
> RTE_EVENT_ETH_RX_ADAPTER_CAP_EVENT_VECTOR then a given Rx queue
> can be configured to enable event vectorization by passing the
> flag RTE_EVENT_ETH_RX_ADAPTER_QUEUE_EVENT_VECTOR to
> rte_event_eth_rx_adapter_queue_conf::rx_queue_flags while configuring
> Rx adapter through rte_event_eth_rx_adapter_queue_add().
> 
> The max vector size, vector timeout define the vector size and
> mempool used for allocating vector event are configured through
> rte_event_eth_rx_adapter_queue_add. The element size of the element
> in the vector pool should be equal to
>     sizeof(struct rte_event_vector) + (vector_sz * sizeof(uintptr_t))
> 
> Application can use `rte_event_vector_pool_create` to create the
> vector mempool used for
> rte_event_eth_rx_adapter_queue_conf::vector_mp.
> 
> The Rx adapter would be responsible for vectorizing the mbufs
> based on the flow, the vector limits configured by the application
> and add the vector event of mbufs to the event queue set via
> rte_event_eth_rx_adapter_queue_conf::ev::queue_id.
> It should also mark rte_event_vector::union_valid and fill
> rte_event_vector::port, rte_event_vector::queue.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> Acked-by: Jerin Jacob <jerinj@marvell.com>
> ---
>  .../prog_guide/event_ethernet_rx_adapter.rst  |  38 ++++++
>  lib/librte_eventdev/eventdev_pmd.h            |  53 ++++++++
>  .../rte_event_eth_rx_adapter.c                | 114 ++++++++++++++++++
>  .../rte_event_eth_rx_adapter.h                | 105 ++++++++++++++++
>  lib/librte_eventdev/rte_eventdev.h            |  30 ++++-
>  lib/librte_eventdev/version.map               |   2 +
>  6 files changed, 340 insertions(+), 2 deletions(-)
> 
> diff --git a/doc/guides/prog_guide/event_ethernet_rx_adapter.rst b/doc/guides/prog_guide/event_ethernet_rx_adapter.rst
> index cb44ce0e4..5eefef355 100644
> --- a/doc/guides/prog_guide/event_ethernet_rx_adapter.rst
> +++ b/doc/guides/prog_guide/event_ethernet_rx_adapter.rst
> @@ -186,3 +186,41 @@ the event buffer fill level is low. The
>  ``rte_event_eth_rx_adapter_cb_register()`` function allow the application
>  to register a callback that selects which packets to enqueue to the event
>  device.
> +
> +Rx event vectorization
> +~~~~~~~~~~~~~~~~~~~~~~
> +
> +The event devices, ethernet device pairs which support the capability
> +``RTE_EVENT_ETH_RX_ADAPTER_CAP_EVENT_VECTOR`` can aggregate packets based on
> +flow characteristics and generate a ``rte_event`` containing ``rte_event_vector``
> +whose event type is either ``RTE_EVENT_TYPE_ETHDEV_VECTOR`` or
> +``RTE_EVENT_TYPE_ETH_RX_ADAPTER_VECTOR``.
> +The aggregation size and timeout are configurable at a queue level and the
> +maximum, minimum vector sizes and timeouts vary based on the device capability
> +and can be queried using ``rte_event_eth_rx_adapter_vector_limits_get``.
> +The Rx adapter additionally might include useful data such as ethernet device
> +port and queue identifier in the ``rte_event_vector::port`` and
> +``rte_event_vector::queue`` and mark ``rte_event_vector::attr_valid`` as true.
> +
> +A loop processing ``rte_event_vector`` containing mbufs is shown below.
> +
> +.. code-block:: c
> +
> +        event = rte_event_dequeue_burst(event_dev, event_port, &event,
> +                                        1, 0);
> +        if (!event)
> +                continue;
> +
> +        switch (ev.event_type) {
> +        case RTE_EVENT_TYPE_ETH_RX_ADAPTER_VECTOR:
> +        case RTE_EVENT_TYPE_ETHDEV_VECTOR:
> +                struct rte_mbufs **mbufs;
> +
> +                mbufs = (struct rte_mbufs **)ev[i].vec->mbufs;
> +                for (i = 0; i < ev.vec->nb_elem; i++) {
> +                        /* Process each mbuf. */
> +                }
> +        break;
> +        case ...
> +        ...
> +        }
> diff --git a/lib/librte_eventdev/eventdev_pmd.h b/lib/librte_eventdev/eventdev_pmd.h
> index 7eb9a7739..9297f1433 100644
> --- a/lib/librte_eventdev/eventdev_pmd.h
> +++ b/lib/librte_eventdev/eventdev_pmd.h
> @@ -645,6 +645,53 @@ typedef int (*eventdev_eth_rx_adapter_stats_reset)
>   */
>  typedef int (*eventdev_selftest)(void);
> 
> +struct rte_event_eth_rx_adapter_vector_limits;
> +/**
> + * Get event vector limits for a given event, ethernet device pair.
> + *
> + * @param dev
> + *   Event device pointer
> + *
> + * @param eth_dev
> + *   Ethernet device pointer
> + *
> + * @param[out] limits
> + *   Pointer to the limits structure to be filled.
> + *
> + * @return
> + *   - 0: Success.
> + *   - <0: Error code returned by the driver function.
> + */
> +typedef int (*eventdev_eth_rx_adapter_vector_limits_get_t)(
> +	const struct rte_eventdev *dev, const struct rte_eth_dev *eth_dev,
> +	struct rte_event_eth_rx_adapter_vector_limits *limits);
> +
> +struct rte_event_eth_rx_adapter_event_vector_config;
> +/**
> + * Enable event vector on an given Rx queue of a ethernet devices belonging to
> + * the Rx adapter.
> + *
> + * @param dev
> + *   Event device pointer
> + *
> + * @param eth_dev
> + *   Ethernet device pointer
> + *
> + * @param rx_queue_id
> + *   The Rx queue identifier
> + *
> + * @param config
> + *   Pointer to the event vector configuration structure.
> + *
> + * @return
> + *   - 0: Success.
> + *   - <0: Error code returned by the driver function.
> + */
> +typedef int (*eventdev_eth_rx_adapter_event_vector_config_t)(
> +	const struct rte_eventdev *dev, const struct rte_eth_dev *eth_dev,
> +	int32_t rx_queue_id,
> +	const struct rte_event_eth_rx_adapter_event_vector_config *config);
> +
>  typedef uint32_t rte_event_pmd_selftest_seqn_t;
>  extern int rte_event_pmd_selftest_seqn_dynfield_offset;
> 
> @@ -1067,6 +1114,12 @@ struct rte_eventdev_ops {
>  	/**< Get ethernet Rx stats */
>  	eventdev_eth_rx_adapter_stats_reset eth_rx_adapter_stats_reset;
>  	/**< Reset ethernet Rx stats */
> +	eventdev_eth_rx_adapter_vector_limits_get_t
> +		eth_rx_adapter_vector_limits_get;
> +	/**< Get event vector limits for the Rx adapter */
> +	eventdev_eth_rx_adapter_event_vector_config_t
> +		eth_rx_adapter_event_vector_config;
> +	/**< Configure Rx adapter with event vector */
> 
>  	eventdev_timer_adapter_caps_get_t timer_adapter_caps_get;
>  	/**< Get timer adapter capabilities */
> diff --git a/lib/librte_eventdev/rte_event_eth_rx_adapter.c b/lib/librte_eventdev/rte_event_eth_rx_adapter.c
> index d8c635e99..ac8ba5bf0 100644
> --- a/lib/librte_eventdev/rte_event_eth_rx_adapter.c
> +++ b/lib/librte_eventdev/rte_event_eth_rx_adapter.c
> @@ -2263,6 +2263,120 @@ rte_event_eth_rx_adapter_queue_del(uint8_t id, uint16_t eth_dev_id,
>  	return ret;
>  }
> 
> +int
> +rte_event_eth_rx_adapter_queue_event_vector_config(
> +	uint8_t id, uint16_t eth_dev_id, int32_t rx_queue_id,
> +	struct rte_event_eth_rx_adapter_event_vector_config *config)
> +{
> +	struct rte_event_eth_rx_adapter_vector_limits limits;
> +	struct rte_event_eth_rx_adapter *rx_adapter;
> +	struct rte_eventdev *dev;
> +	uint32_t cap;
> +	int ret;
> +
> +	RTE_EVENT_ETH_RX_ADAPTER_ID_VALID_OR_ERR_RET(id, -EINVAL);
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(eth_dev_id, -EINVAL);
> +
> +	rx_adapter = rxa_id_to_adapter(id);
> +	if ((rx_adapter == NULL) || (config == NULL))
> +		return -EINVAL;
> +
> +	dev = &rte_eventdevs[rx_adapter->eventdev_id];
> +	ret = rte_event_eth_rx_adapter_caps_get(rx_adapter->eventdev_id,
> +						eth_dev_id, &cap);
> +	if (ret) {
> +		RTE_EDEV_LOG_ERR("Failed to get adapter caps edev %" PRIu8
> +				 "eth port %" PRIu16,
> +				 id, eth_dev_id);
> +		return ret;
> +	}
> +
> +	if (!(cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_EVENT_VECTOR)) {
> +		RTE_EDEV_LOG_ERR("Event vectorization is not supported,"
> +				 " eth port: %" PRIu16 " adapter id: %" PRIu8,
> +				 eth_dev_id, id);
> +		return -EINVAL;
> +	}
> +
> +	ret = rte_event_eth_rx_adapter_vector_limits_get(
> +		rx_adapter->eventdev_id, eth_dev_id, &limits);
> +	if (ret) {
> +		RTE_EDEV_LOG_ERR("Failed to get vector limits edev %" PRIu8
> +				 "eth port %" PRIu16,
> +				 rx_adapter->eventdev_id, eth_dev_id);
> +		return ret;
> +	}
> +
> +	if (config->vector_sz < limits.min_sz ||
> +	    config->vector_sz > limits.max_sz ||
> +	    config->vector_timeout_ns < limits.min_timeout_ns ||
> +	    config->vector_timeout_ns > limits.max_timeout_ns ||
> +	    config->vector_mp == NULL) {
> +		RTE_EDEV_LOG_ERR("Invalid event vector configuration,"
> +				 " eth port: %" PRIu16 " adapter id: %" PRIu8,
> +				 eth_dev_id, id);
> +		return -EINVAL;
> +	}
> +	if (config->vector_mp->elt_size <
> +	    (sizeof(struct rte_event_vector) +
> +	     (sizeof(uintptr_t) * config->vector_sz))) {
> +		RTE_EDEV_LOG_ERR("Invalid event vector configuration,"
> +				 " eth port: %" PRIu16 " adapter id: %" PRIu8,
> +				 eth_dev_id, id);
> +		return -EINVAL;
> +	}
> +
> +	if (cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT) {
> +		RTE_FUNC_PTR_OR_ERR_RET(
> +			*dev->dev_ops->eth_rx_adapter_event_vector_config,
> +			-ENOTSUP);
> +		ret = dev->dev_ops->eth_rx_adapter_event_vector_config(
> +			dev, &rte_eth_devices[eth_dev_id], rx_queue_id, config);
> +	} else {
> +		ret = -ENOTSUP;
> +	}

Trying to understand why non-INTERNAL_PORT based event device implementation returns ENOTSUP. Do you foresee any issues if such a device implements this function?

> +
> +	return ret;
> +}
> +
> +int
> +rte_event_eth_rx_adapter_vector_limits_get(
> +	uint8_t dev_id, uint16_t eth_port_id,
> +	struct rte_event_eth_rx_adapter_vector_limits *limits)
> +{
> +	struct rte_eventdev *dev;
> +	uint32_t cap;
> +	int ret;
> +
> +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> +	RTE_ETH_VALID_PORTID_OR_ERR_RET(eth_port_id, -EINVAL);
> +
> +	if (limits == NULL)
> +		return -EINVAL;
> +
> +	dev = &rte_eventdevs[dev_id];
> +
> +	ret = rte_event_eth_rx_adapter_caps_get(dev_id, eth_port_id, &cap);
> +	if (ret) {
> +		RTE_EDEV_LOG_ERR("Failed to get adapter caps edev %" PRIu8
> +				 "eth port %" PRIu16,
> +				 dev_id, eth_port_id);
> +		return ret;
> +	}
> +
> +	if (cap & RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT) {
> +		RTE_FUNC_PTR_OR_ERR_RET(
> +			*dev->dev_ops->eth_rx_adapter_vector_limits_get,
> +			-ENOTSUP);
> +		ret = dev->dev_ops->eth_rx_adapter_vector_limits_get(
> +			dev, &rte_eth_devices[eth_port_id], limits);
> +	} else {
> +		ret = -ENOTSUP;
> +	}
> +

Same here.

> +	return ret;
> +}
> +
>  int
>  rte_event_eth_rx_adapter_start(uint8_t id)
>  {
> diff --git a/lib/librte_eventdev/rte_event_eth_rx_adapter.h b/lib/librte_eventdev/rte_event_eth_rx_adapter.h
> index 21bb1e54c..7407cde00 100644
> --- a/lib/librte_eventdev/rte_event_eth_rx_adapter.h
> +++ b/lib/librte_eventdev/rte_event_eth_rx_adapter.h
> @@ -92,6 +92,10 @@ extern "C" {
>  /**< This flag indicates the flow identifier is valid
>   * @see rte_event_eth_rx_adapter_queue_conf::rx_queue_flags
>   */
> +#define RTE_EVENT_ETH_RX_ADAPTER_QUEUE_EVENT_VECTOR	0x2
> +/**< This flag indicates that mbufs arriving on the queue need to be vectorized
> + * @see rte_event_eth_rx_adapter_queue_conf::rx_queue_flags
> + */
> 
>  /**
>   * Adapter configuration structure that the adapter configuration callback
> @@ -169,6 +173,36 @@ struct rte_event_eth_rx_adapter_queue_conf {
>  	 */
>  };
> 
> +struct rte_event_eth_rx_adapter_event_vector_config {
> +	uint16_t vector_sz;
> +	/**<
> +	 * Indicates the maximum number for mbufs to combine and form a vector.
> +	 * Should be within
> +	 * @see rte_event_eth_rx_adapter_vector_limits::min_vector_sz
> +	 * @see rte_event_eth_rx_adapter_vector_limits::max_vector_sz
> +	 * Valid when RTE_EVENT_ETH_RX_ADAPTER_QUEUE_EVENT_VECTOR flag is set in
> +	 * @see rte_event_eth_rx_adapter_queue_conf::rx_queue_flags
> +	 */
> +	uint64_t vector_timeout_ns;
> +	/**<
> +	 * Indicates the maximum number of nanoseconds to wait for receiving
> +	 * mbufs. Should be within vectorization limits of the
> +	 * adapter
> +	 * @see rte_event_eth_rx_adapter_vector_limits::min_vector_ns
> +	 * @see rte_event_eth_rx_adapter_vector_limits::max_vector_ns
> +	 * Valid when RTE_EVENT_ETH_RX_ADAPTER_QUEUE_EVENT_VECTOR flag is set in
> +	 * @see rte_event_eth_rx_adapter_queue_conf::rx_queue_flags
> +	 */
> +	struct rte_mempool *vector_mp;
> +	/**<
> +	 * Indicates the mempool that should be used for allocating
> +	 * rte_event_vector container.
> +	 * Should be created by using `rte_event_vector_pool_create`.
> +	 * Valid when RTE_EVENT_ETH_RX_ADAPTER_QUEUE_EVENT_VECTOR flag is set in
> +	 * @see rte_event_eth_rx_adapter_queue_conf::rx_queue_flags.
> +	 */
> +};
> +
>  /**
>   * A structure used to retrieve statistics for an eth rx adapter instance.
>   */
> @@ -199,6 +233,32 @@ struct rte_event_eth_rx_adapter_stats {
>  	/**< Received packet count for interrupt mode Rx queues */
>  };
> 
> +/**
> + * A structure used to retrieve eth rx adapter vector limits.
> + */
> +struct rte_event_eth_rx_adapter_vector_limits {
> +	uint16_t min_sz;
> +	/**< Minimum vector limit configurable.
> +	 * @see rte_event_eth_rx_adapter_event_vector_config::vector_sz
> +	 */
> +	uint16_t max_sz;
> +	/**< Maximum vector limit configurable.
> +	 * @see rte_event_eth_rx_adapter_event_vector_config::vector_sz
> +	 */
> +	uint8_t log2_sz;
> +	/**< True if the size configured should be in log2.
> +	 * @see rte_event_eth_rx_adapter_event_vector_config::vector_sz
> +	 */
> +	uint64_t min_timeout_ns;
> +	/**< Minimum vector timeout configurable.
> +	 * @see rte_event_eth_rx_adapter_event_vector_config::vector_timeout_ns
> +	 */
> +	uint64_t max_timeout_ns;
> +	/**< Maximum vector timeout configurable.
> +	 * @see rte_event_eth_rx_adapter_event_vector_config::vector_timeout_ns
> +	 */
> +};
> +
>  /**
>   *
>   * Callback function invoked by the SW adapter before it continues
> @@ -467,6 +527,51 @@ int rte_event_eth_rx_adapter_cb_register(uint8_t id, uint16_t eth_dev_id,
>  					 rte_event_eth_rx_adapter_cb_fn cb_fn,
>  					 void *cb_arg);
> 
> +/**
> + * Retrieve vector limits for a given event dev and eth dev pair.
> + * @see rte_event_eth_rx_adapter_vector_limits
> + *
> + * @param dev_id
> + *  Event device identifier.
> + * @param eth_port_id
> + *  Port identifier of the ethernet device.
> + * @param [out] limits
> + *  A pointer to rte_event_eth_rx_adapter_vector_limits structure that has to
> + * be filled.
> + *
> + * @return
> + *  - 0: Success.
> + *  - <0: Error code on failure.
> + */
> +__rte_experimental
> +int rte_event_eth_rx_adapter_vector_limits_get(
> +	uint8_t dev_id, uint16_t eth_port_id,
> +	struct rte_event_eth_rx_adapter_vector_limits *limits);
> +
> +/**
> + * Configure event vectorization for a given ethernet device queue, that has
> + * been added to a event eth Rx adapter.
> + *
> + * @param id
> + *  The identifier of the ethernet Rx event adapter.
> + *
> + * @param eth_dev_id
> + *  The identifier of the ethernet device.
> + *
> + * @param rx_queue_id
> + *  Ethernet device receive queue index.
> + *  If rx_queue_id is -1, then all Rx queues configured for the ethernet device
> + *  are configured with event vectorization.
> + *
> + * @return
> + *  - 0: Success, Receive queue configured correctly.
> + *  - <0: Error code on failure.
> + */
> +__rte_experimental
> +int rte_event_eth_rx_adapter_queue_event_vector_config(
> +	uint8_t id, uint16_t eth_dev_id, int32_t rx_queue_id,
> +	struct rte_event_eth_rx_adapter_event_vector_config *config);
> +
>  #ifdef __cplusplus
>  }
>  #endif
> diff --git a/lib/librte_eventdev/rte_eventdev.h b/lib/librte_eventdev/rte_eventdev.h
> index aa4dd3959..678338247 100644
> --- a/lib/librte_eventdev/rte_eventdev.h
> +++ b/lib/librte_eventdev/rte_eventdev.h
> @@ -919,10 +919,28 @@ rte_event_dev_close(uint8_t dev_id);
>   * Event vector structure.
>   */
>  struct rte_event_vector {
> -	uint64_t nb_elem : 16;
> +	uint16_t nb_elem;
>  	/**< Number of elements in this event vector. */
> -	uint64_t rsvd : 48;
> +	uint16_t rsvd : 15;
>  	/**< Reserved for future use */
> +	uint16_t attr_valid : 1;
> +	/**< Indicates that the below union attributes have valid information.
> +	 */
> +	union {
> +		/* Used by Rx adapter.
> +		 * Indicates that all the elements in this vector belong to the
> +		 * same port and queue pair when originating from Rx adapter,
> +		 * valid only when event type is ETHDEV_VECTOR or
> +		 * ETH_RX_ADAPTER_VECTOR.
> +		 */
> +		struct {
> +			uint16_t port;
> +			/* Ethernet device port id. */
> +			uint16_t queue;
> +			/* Ethernet device queue id. */
> +		};
> +	};
> +	/**< Union to hold common attributes of the vector array. */
>  	uint64_t impl_opaque;
>  	/**< Implementation specific opaque value.
>  	 * An implementation may use this field to hold implementation specific
> @@ -1025,8 +1043,14 @@ struct rte_event_vector {
>   *		// Classify and handle event.
>   *	}
>   */
> +#define RTE_EVENT_TYPE_ETHDEV_VECTOR                                           \
> +	(RTE_EVENT_TYPE_VECTOR | RTE_EVENT_TYPE_ETHDEV)
> +/**< The event vector generated from ethdev subsystem */
>  #define RTE_EVENT_TYPE_CPU_VECTOR (RTE_EVENT_TYPE_VECTOR | RTE_EVENT_TYPE_CPU)
>  /**< The event vector generated from cpu for pipelining. */
> +#define RTE_EVENT_TYPE_ETH_RX_ADAPTER_VECTOR                                   \
> +	(RTE_EVENT_TYPE_VECTOR | RTE_EVENT_TYPE_ETH_RX_ADAPTER)
> +/**< The event vector generated from eth Rx adapter. */
> 
>  #define RTE_EVENT_TYPE_MAX              0x10
>  /**< Maximum number of event types */
> @@ -1171,6 +1195,8 @@ struct rte_event {
>   * @see struct rte_event_eth_rx_adapter_queue_conf::ev
>   * @see struct rte_event_eth_rx_adapter_queue_conf::rx_queue_flags
>   */
> +#define RTE_EVENT_ETH_RX_ADAPTER_CAP_EVENT_VECTOR	0x8
> +/**< Adapter supports event vectorization per ethdev. */
> 
>  /**
>   * Retrieve the event device's ethdev Rx adapter capabilities for the
> diff --git a/lib/librte_eventdev/version.map b/lib/librte_eventdev/version.map
> index a070ef56e..902df0ae3 100644
> --- a/lib/librte_eventdev/version.map
> +++ b/lib/librte_eventdev/version.map
> @@ -141,6 +141,8 @@ EXPERIMENTAL {
> 
>  	#added in 21.05
>  	rte_event_vector_pool_create;
> +	rte_event_eth_rx_adapter_vector_limits_get;
> +	rte_event_eth_rx_adapter_queue_event_vector_config;
>  };
> 
>  INTERNAL {
> --
> 2.17.1

More of informational queries.

Acked-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>