[dpdk-dev] [pbhagavatula@caviumnetworks.com: Re: [PATCH 2/4] eventdev: add caps API and PMD callbacks for eth Tx adapter]

[dpdk-dev] [pbhagavatula@caviumnetworks.com: Re: [PATCH 2/4] eventdev: add caps API and PMD callbacks for eth Tx adapter]

[Pavan Nikhilesh]

Hi Nikhil,

On Mon, Jul 16, 2018 at 11:25:45AM +0530, Rao, Nikhil wrote:
> On 7/10/2018 4:26 PM, Pavan Nikhilesh wrote:
> > +int __rte_experimental
> > +rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint32_t *caps)
> > +{
> > The caps get API needs to be similar to rx adapter caps get i.e. it needs to
> > have the eth_port_id as a parameter so that the underlying event dev driver can
> > expose INTERNAL PORT capability as not all ethdev drivers have the capability
> > to interact with the eventdevs internal port.
> >
> > rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
> >                       uint32_t *caps);
> Hi Pavan,
>
> Is querying the INTERNAL PORT on a per ethdev basis useful to the
> application ?

If an application chooses to use 2 ports one with INTERNAL PORT capability and
one _without_ it then it would be useful to have per ethdev based
classification similar to Rx adapter. The application can classify events based
on event type RTE_EVENT_TYPE_ETHDEV & RTE_EVENT_TYPE_ETH_RX_ADAPTER to
segregate between INTERNAL & NON-INTERNAL port at ingress side and enqueue it
to either rte_event_eth_tx_adapter_enqueue or to the SINGLE link queue
respectively.

Also, I dont think eventdev should iterate over all probed ethdevs and give
the overall caps as an application might want only a specific ethdev to be
connected to the event tx adapter.

>
> For e.g., the txa_init() function in the adapter implementation can only
> decide to use the internal port if it is supported for all ethdevs,
> hence I left that upto the eventdev PMD to decide - i.e., it could
> iterate across txa->dev_count eth devices to make that determination.
> For caps in general, I agree it makes sense to pass in the ethdev, but
> the INTERNAL PORT didn't seem useful on a per ethdev basis.
>
> We could also replace caps_get with something like a
> rte_event_eth_tx_adapter_internal_port_check(dev_id) and add a per
> ethdev caps if needed later.
>
> Thanks,
> Nikhil

Thanks,
Pavan.

Re: [dpdk-dev] [pbhagavatula@caviumnetworks.com: Re: [PATCH 2/4] eventdev: add caps API and PMD callbacks for eth Tx adapter]

[Rao, Nikhil]

> -----Original Message-----
> From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com]
> Sent: Monday, July 16, 2018 5:03 PM
> To: Rao, Nikhil <nikhil.rao@intel.com>;
> jkollanukkaran@caviumnetworks.com; olivier.matz@6wind.com
> Cc: dev@dpdk.org
> Subject: [pbhagavatula@caviumnetworks.com: Re: [dpdk-dev] [PATCH 2/4]
> eventdev: add caps API and PMD callbacks for eth Tx adapter]
> 
> Hi Nikhil,
> 
> On Mon, Jul 16, 2018 at 11:25:45AM +0530, Rao, Nikhil wrote:
> > On 7/10/2018 4:26 PM, Pavan Nikhilesh wrote:
> > > +int __rte_experimental
> > > +rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint32_t *caps) {
> > > The caps get API needs to be similar to rx adapter caps get i.e. it
> > > needs to have the eth_port_id as a parameter so that the underlying
> > > event dev driver can expose INTERNAL PORT capability as not all
> > > ethdev drivers have the capability to interact with the eventdevs
> internal port.
> > >
> > > rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t
> eth_port_id,
> > >                       uint32_t *caps);
> > Hi Pavan,
> >
> > Is querying the INTERNAL PORT on a per ethdev basis useful to the
> > application ?
> 
> If an application chooses to use 2 ports one with INTERNAL PORT capability
> and one _without_ it then it would be useful to have per ethdev based
> classification similar to Rx adapter. The application can classify events based
> on event type RTE_EVENT_TYPE_ETHDEV &
> RTE_EVENT_TYPE_ETH_RX_ADAPTER to segregate between INTERNAL &
> NON-INTERNAL port at ingress side and enqueue it to either
> rte_event_eth_tx_adapter_enqueue or to the SINGLE link queue
> respectively.
> 
The current tx adapter is very similar to how the eventdev pipeline app decides between using the generic_tx v/s worker_tx, I guess what you are suggesting is using the 2 concurrently. I am fine with this, 
Would you always assume INTERNAL PORT on the Rx side to deduce INTERNAL PORT on the tx side ? Just curious if that was just an example, in the general case, you could have packets ingressing from an INTERNAL port and egressing on a port that is !INTERNAL ?

> Also, I dont think eventdev should iterate over all probed ethdevs and give
> the overall caps as an application might want only a specific ethdev to be
> connected to the event tx adapter.
> 
Agreed. The current adapter only supports either the generic_tx or the worker_tx models and selects either at the earliest point it is feasible to, 

I will update the patch series for  caps_get()

Thanks,
Nikhil

[dpdk-dev] [PATCH 2/4] eventdev: add caps API and PMD callbacks for eth Tx adapter

[Pavan Nikhilesh]

On Mon, Jul 16, 2018 at 02:17:40PM +0000, Rao, Nikhil wrote:
>
> > -----Original Message-----
> > From: Pavan Nikhilesh [mailto:pbhagavatula@caviumnetworks.com]
> > Sent: Monday, July 16, 2018 5:03 PM
> > To: Rao, Nikhil <nikhil.rao@intel.com>;
> > jkollanukkaran@caviumnetworks.com; olivier.matz@6wind.com
> > Cc: dev@dpdk.org
> > Subject: [pbhagavatula@caviumnetworks.com: Re: [dpdk-dev] [PATCH 2/4]
> > eventdev: add caps API and PMD callbacks for eth Tx adapter]
> >
> > Hi Nikhil,
> >
> > On Mon, Jul 16, 2018 at 11:25:45AM +0530, Rao, Nikhil wrote:
> > > On 7/10/2018 4:26 PM, Pavan Nikhilesh wrote:
> > > > +int __rte_experimental
> > > > +rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint32_t *caps) {
> > > > The caps get API needs to be similar to rx adapter caps get i.e. it
> > > > needs to have the eth_port_id as a parameter so that the underlying
> > > > event dev driver can expose INTERNAL PORT capability as not all
> > > > ethdev drivers have the capability to interact with the eventdevs
> > internal port.
> > > >
> > > > rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t
> > eth_port_id,
> > > >                       uint32_t *caps);
> > > Hi Pavan,
> > >
> > > Is querying the INTERNAL PORT on a per ethdev basis useful to the
> > > application ?
> >
> > If an application chooses to use 2 ports one with INTERNAL PORT capability
> > and one _without_ it then it would be useful to have per ethdev based
> > classification similar to Rx adapter. The application can classify events based
> > on event type RTE_EVENT_TYPE_ETHDEV &
> > RTE_EVENT_TYPE_ETH_RX_ADAPTER to segregate between INTERNAL &
> > NON-INTERNAL port at ingress side and enqueue it to either
> > rte_event_eth_tx_adapter_enqueue or to the SINGLE link queue
> > respectively.
> >
> The current tx adapter is very similar to how the eventdev pipeline app decides between using the generic_tx v/s worker_tx, I guess what you are suggesting is using the 2 concurrently. I am fine with this,
> Would you always assume INTERNAL PORT on the Rx side to deduce INTERNAL PORT on the tx side ? Just curious if that was just an example, in the general case, you could have packets ingressing from an INTERNAL port and egressing on a port that is !INTERNAL ?

I was just giving an example :). The application would be free to model the
pipeline accordingly based on event type and caps.

>
> > Also, I dont think eventdev should iterate over all probed ethdevs and give
> > the overall caps as an application might want only a specific ethdev to be
> > connected to the event tx adapter.
> >
> Agreed. The current adapter only supports either the generic_tx or the worker_tx models and selects either at the earliest point it is feasible to,
>
> I will update the patch series for  caps_get()
>
> Thanks,
> Nikhil

Thanks,
Pavan

Re: [dpdk-dev] [PATCH 2/4] eventdev: add caps API and PMD callbacks for eth Tx adapter

[Rao, Nikhil]

On 7/10/2018 4:26 PM, Pavan Nikhilesh wrote:
> +int __rte_experimental
> +rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint32_t *caps)
> +{
> The caps get API needs to be similar to rx adapter caps get i.e. it needs to
> have the eth_port_id as a parameter so that the underlying event dev driver can
> expose INTERNAL PORT capability as not all ethdev drivers have the capability
> to interact with the eventdevs internal port.
>
> rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
> 			uint32_t *caps);
Hi Pavan,

Is querying the INTERNAL PORT on a per ethdev basis useful to the 
application ?

For e.g., the txa_init() function in the adapter implementation can only 
decide to use the internal port if it is supported for all ethdevs, 
hence I left that upto the eventdev PMD to decide - i.e., it could 
iterate across txa->dev_count eth devices to make that determination. 
For caps in general, I agree it makes sense to pass in the ethdev, but 
the INTERNAL PORT didn't seem useful on a per ethdev basis.

We could also replace caps_get with something like a
rte_event_eth_tx_adapter_internal_port_check(dev_id) and add a per 
ethdev caps if needed later.

Thanks,
Nikhil

Re: [dpdk-dev] [PATCH 2/4] eventdev: add caps API and PMD callbacks for eth Tx adapter

[Pavan Nikhilesh]

Hi Nikhil,

On Fri, Jul 06, 2018 at 12:12:07PM +0530, Nikhil Rao wrote:
> The caps API allows the application to query if the transmit
> stage is implemented in the eventdev PMD or uses the common
> rte_service function. The PMD callbacks support the
> eventdev PMD implementation of the adapter.
>
> Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
> ---
>  lib/librte_eventdev/rte_eventdev.h     |  30 ++++-
>  lib/librte_eventdev/rte_eventdev_pmd.h | 193 +++++++++++++++++++++++++++++++++
>  lib/librte_eventdev/rte_eventdev.c     |  19 ++++
>  3 files changed, 241 insertions(+), 1 deletion(-)
>
<...>
>
> diff --git a/lib/librte_eventdev/rte_eventdev.c b/lib/librte_eventdev/rte_eventdev.c
> index 801810e..a29fae1 100644
> --- a/lib/librte_eventdev/rte_eventdev.c
> +++ b/lib/librte_eventdev/rte_eventdev.c
> @@ -175,6 +175,25 @@
>                 (dev, cdev, caps) : -ENOTSUP;
>  }
>
> +int __rte_experimental
> +rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint32_t *caps)
> +{

The caps get API needs to be similar to rx adapter caps get i.e. it needs to
have the eth_port_id as a parameter so that the underlying event dev driver can
expose INTERNAL PORT capability as not all ethdev drivers have the capability
to interact with the eventdevs internal port.

rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint16_t eth_port_id,
			uint32_t *caps);


> +       struct rte_eventdev *dev;
> +
> +       RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> +
> +       dev = &rte_eventdevs[dev_id];
> +
> +       if (caps == NULL)
> +               return -EINVAL;
> +       *caps = 0;
> +
> +       return dev->dev_ops->eth_tx_adapter_caps_get ?
> +                               (*dev->dev_ops->eth_tx_adapter_caps_get)(dev,
> +                                                                       caps)
> +                               : 0;
> +}
> +
>  static inline int
>  rte_event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
>  {
> --
> 1.8.3.1
>

Thanks,
Pavan.

[dpdk-dev] [PATCH 2/4] eventdev: add caps API and PMD callbacks for eth Tx adapter

[Nikhil Rao]

The caps API allows the application to query if the transmit
stage is implemented in the eventdev PMD or uses the common
rte_service function. The PMD callbacks support the
eventdev PMD implementation of the adapter.

Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
---
 lib/librte_eventdev/rte_eventdev.h     |  30 ++++-
 lib/librte_eventdev/rte_eventdev_pmd.h | 193 +++++++++++++++++++++++++++++++++
 lib/librte_eventdev/rte_eventdev.c     |  19 ++++
 3 files changed, 241 insertions(+), 1 deletion(-)

diff --git a/lib/librte_eventdev/rte_eventdev.h b/lib/librte_eventdev/rte_eventdev.h
index b6fd6ee..e8df526 100644
--- a/lib/librte_eventdev/rte_eventdev.h
+++ b/lib/librte_eventdev/rte_eventdev.h
@@ -1186,6 +1186,28 @@ struct rte_event {
 rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id,
 				  uint32_t *caps);
 
+/* Ethdev Tx adapter capability bitmap flags */
+#define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT	0x1
+/**< This flag is sent when the PMD supports a packet transmit callback
+ */
+
+/**
+ * Retrieve the event device's eth Tx adapter capabilities
+ *
+ * @param dev_id
+ *   The identifier of the device.
+ *
+ * @param[out] caps
+ *   A pointer to memory filled with eth Tx adapter capabilities.
+ *
+ * @return
+ *   - 0: Success, driver provides eth Tx adapter capabilities.
+ *   - <0: Error code returned by the driver function.
+ *
+ */
+int __rte_experimental
+rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint32_t *caps);
+
 struct rte_eventdev_ops;
 struct rte_eventdev;
 
@@ -1204,6 +1226,11 @@ typedef uint16_t (*event_dequeue_burst_t)(void *port, struct rte_event ev[],
 		uint16_t nb_events, uint64_t timeout_ticks);
 /**< @internal Dequeue burst of events from port of a device */
 
+typedef uint16_t (*event_tx_adapter_enqueue)(void *adapter,
+			const struct rte_eventdev *dev, void *port,
+			struct rte_event ev[], uint16_t nb_events);
+/**< @internal Enqueue burst of events on port of a device */
+
 #define RTE_EVENTDEV_NAME_MAX_LEN	(64)
 /**< @internal Max length of name of event PMD */
 
@@ -1266,7 +1293,8 @@ struct rte_eventdev {
 	/**< Pointer to PMD dequeue function. */
 	event_dequeue_burst_t dequeue_burst;
 	/**< Pointer to PMD dequeue burst function. */
-
+	event_tx_adapter_enqueue txa_enqueue;
+	/**< Pointer to PMD eth Tx adapter enqueue function. */
 	struct rte_eventdev_data *data;
 	/**< Pointer to device data */
 	struct rte_eventdev_ops *dev_ops;
diff --git a/lib/librte_eventdev/rte_eventdev_pmd.h b/lib/librte_eventdev/rte_eventdev_pmd.h
index 3fbb4d2..ccf07a8 100644
--- a/lib/librte_eventdev/rte_eventdev_pmd.h
+++ b/lib/librte_eventdev/rte_eventdev_pmd.h
@@ -789,6 +789,178 @@ typedef int (*eventdev_crypto_adapter_stats_reset)
 			(const struct rte_eventdev *dev,
 			 const struct rte_cryptodev *cdev);
 
+/**
+ * Retrieve the event device's eth Tx adapter capabilities.
+ *
+ * @param dev
+ *   Event device pointer
+ *
+ * @param[out] caps
+ *   A pointer to memory filled with eth Tx adapter capabilities.
+ *
+ * @return
+ *   - 0: Success, driver provides eth Tx adapter capabilities
+ *   - <0: Error code returned by the driver function.
+ *
+ */
+typedef int (*eventdev_eth_tx_adapter_caps_get_t)
+					(const struct rte_eventdev *dev,
+					uint32_t *caps);
+
+struct rte_event_eth_tx_adapter;
+
+/**
+ * Retrieve the adapter event port. The adapter creates an event port if
+ * the RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT is not set in the
+ * eth Tx capabilities of the event device.
+ *
+ * @param txa
+ *  Adapter pointer
+ *
+ * @param[out] event_port_id
+ *  Event port pointer
+ *
+ * @return
+ *   - 0: Success.
+ *   - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_event_port_get)
+					(struct rte_event_eth_tx_adapter *txa,
+					uint8_t *port);
+
+/**
+ * Initialize adapter PMD resources. This callback is invoked when
+ * adding the first Tx queue to the adapter.
+ *
+ * @param txa
+ *  Adapter pointer
+ *
+ * @return
+ *   - 0: Success.
+ *   - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_init_t)(
+					struct rte_event_eth_tx_adapter *txa);
+
+/**
+ * Free adapter PMD resources. This callback is invoked after the last queue
+ * has been deleted from the adapter.
+ *
+ * @param txa
+ *  Adapter pointer
+ *
+ * @return
+ *   - 0: Success.
+ *   - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_free_t)(
+					struct rte_event_eth_tx_adapter *txa);
+
+/**
+ * Add a Tx queue to the adapter.
+ * A queue value of -1 is used to indicate all
+ * queues within the device.
+ *
+ * @param txa
+ *  Adapter pointer
+ *
+ * @param eth_dev
+ *  Pointer to ethernet device
+ *
+ * @param tx_queue_id
+ *  Transmt queue index
+ *
+ * @return
+ *   - 0: Success.
+ *   - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_queue_add_t)(
+					struct rte_event_eth_tx_adapter *txa,
+					const struct rte_eth_dev *eth_dev,
+					int32_t tx_queue_id);
+
+/**
+ * Delete a Tx queue from the adapter.
+ * A queue value of -1 is used to indicate all
+ * queues within the device, that have been added to this
+ * adapter.
+ *
+ * @param txa
+ *  Adapter pointer
+ *
+ * @param eth_dev_id
+ *  Ethernet Port Identifier
+ *
+ * @param queue
+ *  Tx queue index
+ *
+ * @return
+ *  - 0: Success, Queues deleted successfully.
+ *  - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_queue_del_t)(
+					struct rte_event_eth_tx_adapter *txa,
+					const struct rte_eth_dev *eth_dev,
+					int32_t tx_queue_id);
+
+/**
+ * Start the adapter.
+ *
+ * @param txa
+ *  Adapter pointer
+ *
+ * @return
+ *  - 0: Success, Adapter started correctly.
+ *  - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_start_t)(
+					struct rte_event_eth_tx_adapter *txa);
+
+/**
+ * Stop the adapter.
+ *
+ * @param txa
+ *  Adapter pointer
+ *
+ * @return
+ *  - 0: Success.
+ *  - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_stop_t)(
+					struct rte_event_eth_tx_adapter *txa);
+
+struct rte_event_eth_tx_adapter_stats;
+
+/**
+ * Retrieve statistics for an adapter
+ *
+ * @param txa
+ *  Adapter Pointer
+ *
+ * @param [out] stats
+ *  A pointer to structure used to retrieve statistics for an adapter
+ *
+ * @return
+ *  - 0: Success, statistics retrieved successfully.
+ *  - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_stats_get_t)(
+				struct rte_event_eth_tx_adapter *txa,
+				struct rte_event_eth_tx_adapter_stats *stats);
+
+/**
+ * Reset statistics for an adapter
+ *
+ * @param txa
+ *  Adapter Pointer
+ *
+ * @return
+ *  - 0: Success, statistics retrieved successfully.
+ *  - <0: Error code on failure.
+ */
+typedef int (*eventdev_eth_tx_adapter_stats_reset_t)(
+					struct rte_event_eth_tx_adapter *txa);
+
 /** Event device operations function pointer table */
 struct rte_eventdev_ops {
 	eventdev_info_get_t dev_infos_get;	/**< Get device info. */
@@ -862,6 +1034,27 @@ struct rte_eventdev_ops {
 	eventdev_crypto_adapter_stats_reset crypto_adapter_stats_reset;
 	/**< Reset crypto stats */
 
+	eventdev_eth_tx_adapter_caps_get_t eth_tx_adapter_caps_get;
+	/**< Get ethernet Tx adapter capabilities */
+	eventdev_eth_tx_adapter_event_port_get eth_tx_adapter_event_port_get;
+	/**< Get event port */
+	eventdev_eth_tx_adapter_init_t eth_tx_adapter_init;
+	/**< Initialize eth Tx adapter */
+	eventdev_eth_tx_adapter_free_t eth_tx_adapter_free;
+	/**< Free ethernet Tx adapter resources */
+	eventdev_eth_tx_adapter_queue_add_t eth_tx_adapter_queue_add;
+	/**< Add Tx queues to the eth Tx adapter */
+	eventdev_eth_tx_adapter_queue_del_t eth_tx_adapter_queue_del;
+	/**< Delete Tx queues from the eth Tx adapter */
+	eventdev_eth_tx_adapter_start_t eth_tx_adapter_start;
+	/**< Start eth Tx adapter */
+	eventdev_eth_tx_adapter_stop_t eth_tx_adapter_stop;
+	/**< Stop eth Tx adapter */
+	eventdev_eth_tx_adapter_stats_get_t eth_tx_adapter_stats_get;
+	/**< Get eth Tx adapter statistics */
+	eventdev_eth_tx_adapter_stats_reset_t eth_tx_adapter_stats_reset;
+	/**< Reset eth Tx adapter statistics */
+
 	eventdev_selftest dev_selftest;
 	/**< Start eventdev Selftest */
 
diff --git a/lib/librte_eventdev/rte_eventdev.c b/lib/librte_eventdev/rte_eventdev.c
index 801810e..a29fae1 100644
--- a/lib/librte_eventdev/rte_eventdev.c
+++ b/lib/librte_eventdev/rte_eventdev.c
@@ -175,6 +175,25 @@
 		(dev, cdev, caps) : -ENOTSUP;
 }
 
+int __rte_experimental
+rte_event_eth_tx_adapter_caps_get(uint8_t dev_id, uint32_t *caps)
+{
+	struct rte_eventdev *dev;
+
+	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
+
+	dev = &rte_eventdevs[dev_id];
+
+	if (caps == NULL)
+		return -EINVAL;
+	*caps = 0;
+
+	return dev->dev_ops->eth_tx_adapter_caps_get ?
+				(*dev->dev_ops->eth_tx_adapter_caps_get)(dev,
+									caps)
+				: 0;
+}
+
 static inline int
 rte_event_dev_queue_config(struct rte_eventdev *dev, uint8_t nb_queues)
 {
-- 
1.8.3.1