Re: [dpdk-dev] [PATCH v2 1/6] eventdev: introduce event driven programming model

[Bruce Richardson <bruce.richardson@intel.com>]

On Tue, Dec 06, 2016 at 09:22:15AM +0530, Jerin Jacob wrote:
> In a polling model, lcores poll ethdev ports and associated
> rx queues directly to look for packet. In an event driven model,
> by contrast, lcores call the scheduler that selects packets for
> them based on programmer-specified criteria. Eventdev library
> adds support for event driven programming model, which offer
> applications automatic multicore scaling, dynamic load balancing,
> pipelining, packet ingress order maintenance and
> synchronization services to simplify application packet processing.
> 
> By introducing event driven programming model, DPDK can support
> both polling and event driven programming models for packet processing,
> and applications are free to choose whatever model
> (or combination of the two) that best suits their needs.
> 
> This patch adds the eventdev specification header file.
> 
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> ---
<snip>
> +
> +/**
> + * The generic *rte_event* structure to hold the event attributes
> + * for dequeue and enqueue operation
> + */
> +struct rte_event {
> +	/** WORD0 */
> +	RTE_STD_C11
> +	union {
> +		uint64_t event;
> +		/** Event attributes for dequeue or enqueue operation */
> +		struct {
> +			uint64_t flow_id:20;
> +			/**< Targeted flow identifier for the enqueue and
> +			 * dequeue operation.
> +			 * The value must be in the range of
> +			 * [0, nb_event_queue_flows - 1] which
> +			 * previously supplied to rte_event_dev_configure().
> +			 */
> +			uint64_t sub_event_type:8;
> +			/**< Sub-event types based on the event source.
> +			 * @see RTE_EVENT_TYPE_CPU
> +			 */
> +			uint64_t event_type:4;
> +			/**< Event type to classify the event source.
> +			 * @see RTE_EVENT_TYPE_ETHDEV, (RTE_EVENT_TYPE_*)
> +			 */
> +			uint64_t sched_type:2;
> +			/**< Scheduler synchronization type (RTE_SCHED_TYPE_*)
> +			 * associated with flow id on a given event queue
> +			 * for the enqueue and dequeue operation.
> +			 */
> +			uint64_t queue_id:8;
> +			/**< Targeted event queue identifier for the enqueue or
> +			 * dequeue operation.
> +			 * The value must be in the range of
> +			 * [0, nb_event_queues - 1] which previously supplied to
> +			 * rte_event_dev_configure().
> +			 */
> +			uint64_t priority:8;
> +			/**< Event priority relative to other events in the
> +			 * event queue. The requested priority should in the
> +			 * range of  [RTE_EVENT_DEV_PRIORITY_HIGHEST,
> +			 * RTE_EVENT_DEV_PRIORITY_LOWEST].
> +			 * The implementation shall normalize the requested
> +			 * priority to supported priority value.
> +			 * Valid when the device has
> +			 * RTE_EVENT_DEV_CAP_FLAG_EVENT_QOS capability.
> +			 */
> +			uint64_t op:2;
> +			/**< The type of event enqueue operation - new/forward/
> +			 * etc.This field is not preserved across an instance
> +			 * and is undefined on dequeue.
> +			 *  @see RTE_EVENT_OP_NEW, (RTE_EVENT_OP_*)
> +			 */
> +			uint64_t impl_opaque:12;
> +			/**< Implementation specific opaque value.
> +			 * An implementation may use this field to hold
> +			 * implementation specific value to share between
> +			 * dequeue and enqueue operation.
> +			 * The application should not modify this field.
> +			 */
> +		};
> +	};
> +	/** WORD1 */
> +	RTE_STD_C11
> +	union {
> +		uint64_t u64;
> +		/**< Opaque 64-bit value */
> +		uintptr_t event_ptr;
> +		/**< Opaque event pointer */

Since we have a uint64_t member of the union, might this be better as a
void * rather than uintptr_t?

> +		struct rte_mbuf *mbuf;
> +		/**< mbuf pointer if dequeued event is associated with mbuf */
> +	};
> +};
> +
<snip>
> +/**
> + * Link multiple source event queues supplied in *rte_event_queue_link*
> + * structure as *queue_id* to the destination event port designated by its
> + * *port_id* on the event device designated by its *dev_id*.
> + *
> + * The link establishment shall enable the event port *port_id* from
> + * receiving events from the specified event queue *queue_id*
> + *
> + * An event queue may link to one or more event ports.
> + * The number of links can be established from an event queue to event port is
> + * implementation defined.
> + *
> + * Event queue(s) to event port link establishment can be changed at runtime
> + * without re-configuring the device to support scaling and to reduce the
> + * latency of critical work by establishing the link with more event ports
> + * at runtime.

I think this might need to be clarified. The device doesn't need to be
reconfigured, but does it need to be stopped? In SW implementation, this
affects how much we have to make things thread-safe. At minimum I think
we should limit this to having only one thread call the function at a
time, but we may allow enqueue dequeue ops from the data plane to run
in parallel.

> + *
> + * @param dev_id
> + *   The identifier of the device.
> + *
> + * @param port_id
> + *   Event port identifier to select the destination port to link.
> + *
> + * @param link
> + *   Points to an array of *nb_links* objects of type *rte_event_queue_link*
> + *   structure which contain the event queue to event port link establishment
> + *   attributes.
> + *   NULL value is allowed, in which case this function links all the configured
> + *   event queues *nb_event_queues* which previously supplied to
> + *   rte_event_dev_configure() to the event port *port_id* with normal servicing
> + *   priority(RTE_EVENT_DEV_PRIORITY_NORMAL).
> + *
> + * @param nb_links
> + *   The number of links to establish
> + *
> + * @return
> + * The number of links actually established. The return value can be less than
> + * the value of the *nb_links* parameter when the implementation has the
> + * limitation on specific queue to port link establishment or if invalid
> + * parameters are specified in a *rte_event_queue_link*.
> + * If the return value is less than *nb_links*, the remaining links at the end
> + * of link[] are not established, and the caller has to take care of them.
> + * If return value is less than *nb_links* then implementation shall update the
> + * rte_errno accordingly, Possible rte_errno values are
> + * (-EDQUOT) Quota exceeded(Application tried to link the queue configured with
> + *  RTE_EVENT_QUEUE_CFG_FLAG_SINGLE_LINK to more than one event ports)
> + * (-EINVAL) Invalid parameter
> + *
> + */
> +int
> +rte_event_port_link(uint8_t dev_id, uint8_t port_id,
> +		    const struct rte_event_queue_link link[],
> +		    uint16_t nb_links);
> +
> +/**
> + * Unlink multiple source event queues supplied in *queues* from the destination
> + * event port designated by its *port_id* on the event device designated
> + * by its *dev_id*.
> + *
> + * The unlink establishment shall disable the event port *port_id* from
> + * receiving events from the specified event queue *queue_id*
> + *
> + * Event queue(s) to event port unlink establishment can be changed at runtime
> + * without re-configuring the device.

Clarify, as above with link call.

> + *
> + * @param dev_id
> + *   The identifier of the device.
> + *
> + * @param port_id
> + *   Event port identifier to select the destination port to unlink.
> + *
> + * @param queues
> + *   Points to an array of *nb_unlinks* event queues to be unlinked
> + *   from the event port.
> + *   NULL value is allowed, in which case this function unlinks all the
> + *   event queue(s) from the event port *port_id*.
> + *
> + * @param nb_unlinks
> + *   The number of unlinks to establish
> + *
> + * @return
> + * The number of unlinks actually established. The return value can be less
> + * than the value of the *nb_unlinks* parameter when the implementation has the
> + * limitation on specific queue to port unlink establishment or
> + * if invalid parameters are specified.
> + * If the return value is less than *nb_unlinks*, the remaining queues at the
> + * end of queues[] are not established, and the caller has to take care of them.
> + * If return value is less than *nb_unlinks* then implementation shall update
> + * the rte_errno accordingly, Possible rte_errno values are
> + * (-EINVAL) Invalid parameter
> + *
> + */
> +int
> +rte_event_port_unlink(uint8_t dev_id, uint8_t port_id,
> +		      uint8_t queues[], uint16_t nb_unlinks);
> +
<snip>