Re: [dpdk-dev] [PATCH v5 1/2] eventdev: add rx queue conf get api

["Kundapura, Ganapati" <ganapati.kundapura@intel.com>]

Hi Jerin,

> -----Original Message-----
> From: Kundapura, Ganapati
> Sent: 20 September 2021 14:44
> To: Jerin Jacob <jerinjacobk@gmail.com>
> Cc: Jayatheerthan, Jay <jay.jayatheerthan@intel.com>; dpdk-dev
> <dev@dpdk.org>; Yigit, Ferruh <ferruh.yigit@intel.com>
> Subject: RE: [PATCH v5 1/2] eventdev: add rx queue conf get api
> 
> Hi Jerin,
> 
> > -----Original Message-----
> > From: Jerin Jacob <jerinjacobk@gmail.com>
> > Sent: 20 September 2021 12:00
> > To: Kundapura, Ganapati <ganapati.kundapura@intel.com>
> > Cc: Jayatheerthan, Jay <jay.jayatheerthan@intel.com>; dpdk-dev
> > <dev@dpdk.org>; Yigit, Ferruh <ferruh.yigit@intel.com>
> > Subject: Re: [PATCH v5 1/2] eventdev: add rx queue conf get api
> >
> > On Thu, Sep 16, 2021 at 6:21 PM Ganapati Kundapura
> > <ganapati.kundapura@intel.com> wrote:
> > >
> > > Added rte_event_eth_rx_adapter_queue_conf_get() API to get rx
> queue
> > > information - event queue identifier, flags for handling received
> > > packets, scheduler type, event priority, polling frequency of the
> > > receive queue and flow identifier in
> > > rte_event_eth_rx_adapter_queue_conf structure
> > >
> > > Signed-off-by: Ganapati Kundapura <ganapati.kundapura@intel.com>
> > >
> > > ---
> > > v5:
> > > * Filled queue_conf after memzone lookup
> > > * PMD callback if not NULL, invoked to override queue_conf struct
> > > * Added memzone lookup for stats_get(), stats_reset(), service_id_get()
> > >   api's called by secondary applications.
> > >
> > > v4:
> > > * squashed 1/3 and 3/3
> > > * reused rte_event_eth_rx_adapter_queue_conf structure in place of
> > >   rte_event_eth_rx_adapter_queue_info
> > > * renamed to rte_event_eth_rx_adapter_queue_info_get() to
> > >   rte_event_eth_rx_adapter_queue_conf_get to align with
> > >   rte_event_eth_rx_adapter_queue_conf structure
> > >
> > > v3:
> > > * Split single patch into implementaion, test and document updation
> > >   patches separately
> > >
> > > v2:
> > > * Fixed build issue due to missing entry in version.map
> > >
> > > v1:
> > > * Initial patch with implementaion, test and doc together
> > > ---
> > > ---
> > >  .../prog_guide/event_ethernet_rx_adapter.rst       |  8 ++
> > >  lib/eventdev/eventdev_pmd.h                        | 28 +++++++
> > >  lib/eventdev/rte_event_eth_rx_adapter.c            | 91
> > +++++++++++++++++++++-
> > >  lib/eventdev/rte_event_eth_rx_adapter.h            | 27 +++++++
> > >  lib/eventdev/version.map                           |  1 +
> > >  5 files changed, 154 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/doc/guides/prog_guide/event_ethernet_rx_adapter.rst
> > > b/doc/guides/prog_guide/event_ethernet_rx_adapter.rst
> > > index 0780b6f..ce23d8a 100644
> > > --- a/doc/guides/prog_guide/event_ethernet_rx_adapter.rst
> > > +++ b/doc/guides/prog_guide/event_ethernet_rx_adapter.rst
> > > @@ -146,6 +146,14 @@ if the callback is supported, and the counts
> > > maintained by the service function,  if one exists. The service
> > > function also maintains a count of cycles for which  it was not able
> > > to
> > enqueue to the event device.
> > >
> > > +Getting Adapter queue config
> > > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > +
> > > +The  ``rte_event_eth_rx_adapter_queue_conf_get()`` function reports
> > > +flags for handling received packets, event queue identifier,
> > > +scheduler type, event priority, polling frequency of the receive
> > > +queue and flow identifier in struct
> > ``rte_event_eth_rx_adapter_queue_conf``.
> > > +
> > >  Interrupt Based Rx Queues
> > >  ~~~~~~~~~~~~~~~~~~~~~~~~~~
> > >
> > > diff --git a/lib/eventdev/eventdev_pmd.h
> > b/lib/eventdev/eventdev_pmd.h
> > > index 63b3bc4..e69644b 100644
> > > --- a/lib/eventdev/eventdev_pmd.h
> > > +++ b/lib/eventdev/eventdev_pmd.h
> > > @@ -562,6 +562,32 @@ typedef int
> > (*eventdev_eth_rx_adapter_queue_del_t)
> > >                                         int32_t rx_queue_id);
> > >
> > >  /**
> > > + * Retrieve Rx adapter queue config information for the specified
> > > + * rx queue ID.
> > > + *
> > > + * @param dev
> > > + *  Event device pointer
> > > + *
> > > + * @param eth_dev
> > > + *  Ethernet device pointer
> > > + *
> > > + * @param rx_queue_id
> > > + *  Ethernet device receive queue index.
> > > + *
> > > + * @param[out] queue_conf
> > > + *  Pointer to rte_event_eth_rx_adapter_queue_conf structure
> > > + *
> > > + * @return
> > > + *  - 0: Success
> > > + *  - <0: Error code on failure.
> > > + */
> > > +typedef int (*eventdev_eth_rx_adapter_queue_conf_get_t)
> > > +                       (const struct rte_eventdev *dev,
> > > +                       const struct rte_eth_dev *eth_dev,
> > > +                       uint16_t rx_queue_id,
> > > +                       struct rte_event_eth_rx_adapter_queue_conf
> > > +*queue_conf);
> > > +
> > > +/**
> > >   * Start ethernet Rx adapter. This callback is invoked if
> > >   * the caps returned from eventdev_eth_rx_adapter_caps_get(..,
> > eth_port_id)
> > >   * has RTE_EVENT_ETH_RX_ADAPTER_CAP_INTERNAL_PORT set and Rx
> > queues
> > > @@ -1081,6 +1107,8 @@ struct rte_eventdev_ops {
> > >         /**< Add Rx queues to ethernet Rx adapter */
> > >         eventdev_eth_rx_adapter_queue_del_t
> eth_rx_adapter_queue_del;
> > >         /**< Delete Rx queues from ethernet Rx adapter */
> > > +       eventdev_eth_rx_adapter_queue_conf_get_t
> > eth_rx_adapter_queue_conf_get;
> > > +       /**< Get Rx adapter queue info */
> > >         eventdev_eth_rx_adapter_start_t eth_rx_adapter_start;
> > >         /**< Start ethernet Rx adapter */
> > >         eventdev_eth_rx_adapter_stop_t eth_rx_adapter_stop; diff
> > > --git a/lib/eventdev/rte_event_eth_rx_adapter.c
> > > b/lib/eventdev/rte_event_eth_rx_adapter.c
> > > index f2dc695..6cc4210 100644
> > > --- a/lib/eventdev/rte_event_eth_rx_adapter.c
> > > +++ b/lib/eventdev/rte_event_eth_rx_adapter.c
> > > @@ -40,6 +40,8 @@
> > >  /* Sentinel value to detect initialized file handle */
> > >  #define INIT_FD                -1
> > >
> > > +#define RXA_ADAPTER_ARRAY "rte_event_eth_rx_adapter_array"
> > > +
> > >  /*
> > >   * Used to store port and queue ID of interrupting Rx queue
> > >   */
> > > @@ -1347,7 +1349,7 @@ rxa_service_func(void *args)  static int
> > >  rte_event_eth_rx_adapter_init(void)
> > >  {
> > > -       const char *name = "rte_event_eth_rx_adapter_array";
> > > +       const char *name = RXA_ADAPTER_ARRAY;
> > >         const struct rte_memzone *mz;
> > >         unsigned int sz;
> > >
> > > @@ -1370,6 +1372,21 @@ rte_event_eth_rx_adapter_init(void)
> > >         return 0;
> > >  }
> > >
> > > +static int
> > > +rxa_memzone_lookup(void)
> > > +{
> > > +       const struct rte_memzone *mz;
> > > +
> > > +       if (event_eth_rx_adapter == NULL) {
> > > +               mz = rte_memzone_lookup(RXA_ADAPTER_ARRAY);
> >
> >
> >
> > Should this be global? There can be different config based on id and
> dev_id.
> > Right?
> > If so, we need to store memzone per unique space by appending
> > additional items in this base string via sprintf or so.
> >
> event_eth_rx_adapter is global array and allocated on memzone in
> rte_event_eth_rx_adapter_init() static struct rte_event_eth_rx_adapter
> **event_eth_rx_adapter;
> 
> Each rx adapter instance is allocated on heap using rte_zmalloc_socket() and
> stored in global memzone array and hence global array lookup.
> 
> event_eth_rx_adapter[id] = rx_adapter;
> 
> In queue_conf() api, memzone lookup for global array and rx_adapter
> instance is retrieved from the global array from adapter id.
> 
A gentle ping to review this
> 
> >
> > > +               if (mz == NULL)
> > > +                       return -ENOMEM;
> > > +               event_eth_rx_adapter = mz->addr;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
> > > +
> > >  static inline struct rte_event_eth_rx_adapter *
> > > rxa_id_to_adapter(uint8_t id)  { @@ -2631,6 +2648,9 @@
> > > rte_event_eth_rx_adapter_stats_get(uint8_t id,
> > >         uint32_t i;
> > >         int ret;
> > >
> > > +       if (rxa_memzone_lookup())
> > > +               return -ENOMEM;
> > > +
> > >         RTE_EVENT_ETH_RX_ADAPTER_ID_VALID_OR_ERR_RET(id, -
> EINVAL);
> > >
> > >         rx_adapter = rxa_id_to_adapter(id); @@ -2669,6 +2689,9 @@
> > > rte_event_eth_rx_adapter_stats_reset(uint8_t id)
> > >         struct eth_device_info *dev_info;
> > >         uint32_t i;
> > >
> > > +       if (rxa_memzone_lookup())
> > > +               return -ENOMEM;
> > > +
> > >         RTE_EVENT_ETH_RX_ADAPTER_ID_VALID_OR_ERR_RET(id, -
> EINVAL);
> > >
> > >         rx_adapter = rxa_id_to_adapter(id); @@ -2694,6 +2717,9 @@
> > > rte_event_eth_rx_adapter_service_id_get(uint8_t id, uint32_t
> > > *service_id)  {
> > >         struct rte_event_eth_rx_adapter *rx_adapter;
> > >
> > > +       if (rxa_memzone_lookup())
> > > +               return -ENOMEM;
> > > +
> > >         RTE_EVENT_ETH_RX_ADAPTER_ID_VALID_OR_ERR_RET(id, -
> EINVAL);
> > >
> > >         rx_adapter = rxa_id_to_adapter(id); @@ -2750,3 +2776,66 @@
> > > rte_event_eth_rx_adapter_cb_register(uint8_t id,
> > >
> > >         return 0;
> > >  }
> > > +
> > > +int
> > > +rte_event_eth_rx_adapter_queue_conf_get(uint8_t id,
> > > +                       uint16_t eth_dev_id,
> > > +                       uint16_t rx_queue_id,
> > > +                       struct rte_event_eth_rx_adapter_queue_conf
> > > +*queue_conf) {
> > > +       struct rte_eventdev *dev;
> > > +       struct rte_event_eth_rx_adapter *rx_adapter;
> > > +       struct eth_device_info *dev_info;
> > > +       struct eth_rx_queue_info *queue_info;
> > > +       struct rte_event *qi_ev;
> > > +       int ret;
> > > +
> > > +       if (rxa_memzone_lookup())
> > > +               return -ENOMEM;
> > > +
> > > +       RTE_EVENT_ETH_RX_ADAPTER_ID_VALID_OR_ERR_RET(id, -
> EINVAL);
> > > +       RTE_ETH_VALID_PORTID_OR_ERR_RET(eth_dev_id, -EINVAL);
> > > +
> > > +       if (rx_queue_id >= rte_eth_devices[eth_dev_id].data-
> > >nb_rx_queues) {
> > > +               RTE_EDEV_LOG_ERR("Invalid rx queue_id %u", rx_queue_id);
> > > +               return -EINVAL;
> > > +       }
> > > +
> > > +       if (queue_conf == NULL) {
> > > +               RTE_EDEV_LOG_ERR("Rx queue conf struct cannot be NULL");
> > > +               return -EINVAL;
> > > +       }
> > > +
> > > +       rx_adapter = rxa_id_to_adapter(id);
> > > +       if (rx_adapter == NULL)
> > > +               return -EINVAL;
> > > +
> > > +       dev_info = &rx_adapter->eth_devices[eth_dev_id];
> > > +       queue_info = &dev_info->rx_queue[rx_queue_id];
> > > +       if (!queue_info->queue_enabled) {
> > > +               RTE_EDEV_LOG_ERR("Rx queue %u not added", rx_queue_id);
> > > +               return -EINVAL;
> > > +       }
> > > +
> > > +       qi_ev = (struct rte_event *)&queue_info->event;
> > > +
> > > +       memset(queue_conf, 0, sizeof(*queue_conf));
> > > +       queue_conf->rx_queue_flags = 0;
> > > +       if (queue_info->flow_id_mask != 0)
> > > +               queue_conf->rx_queue_flags |=
> > > +                       RTE_EVENT_ETH_RX_ADAPTER_QUEUE_FLOW_ID_VALID;
> > > +       queue_conf->servicing_weight = queue_info->wt;
> > > +
> > > +       memcpy(&queue_conf->ev, qi_ev, sizeof(*qi_ev));
> > > +
> > > +       dev = &rte_eventdevs[rx_adapter->eventdev_id];
> > > +       if (dev->dev_ops->eth_rx_adapter_queue_conf_get != NULL) {
> > > +               ret = (*dev->dev_ops->eth_rx_adapter_queue_conf_get)(dev,
> > > +                                               &rte_eth_devices[eth_dev_id],
> > > +                                               rx_queue_id,
> > > +                                               queue_conf);
> > > +               return ret;
> > > +       }
> > > +
> > > +       return 0;
> > > +}
> > > diff --git a/lib/eventdev/rte_event_eth_rx_adapter.h
> > > b/lib/eventdev/rte_event_eth_rx_adapter.h
> > > index 3f8b362..470543e 100644
> > > --- a/lib/eventdev/rte_event_eth_rx_adapter.h
> > > +++ b/lib/eventdev/rte_event_eth_rx_adapter.h
> > > @@ -33,6 +33,7 @@
> > >   *  - rte_event_eth_rx_adapter_stop()
> > >   *  - rte_event_eth_rx_adapter_stats_get()
> > >   *  - rte_event_eth_rx_adapter_stats_reset()
> > > + *  - rte_event_eth_rx_adapter_queue_conf_get()
> > >   *
> > >   * The application creates an ethernet to event adapter using
> > >   * rte_event_eth_rx_adapter_create_ext() or
> > > rte_event_eth_rx_adapter_create() @@ -545,6 +546,32 @@ int
> > rte_event_eth_rx_adapter_vector_limits_get(
> > >         uint8_t dev_id, uint16_t eth_port_id,
> > >         struct rte_event_eth_rx_adapter_vector_limits *limits);
> > >
> > > +/**
> > > + * Retrieve Rx queue config information.
> > > + *
> > > + * @param id
> > > + *  Adapter identifier.
> > > +
> > > + * @param eth_dev_id
> > > + *  Port identifier of Ethernet device.
> > > +
> > > + * @param rx_queue_id
> > > + *  Ethernet device receive queue index.
> > > +
> > > + * @param[out] queue_conf
> > > + *  Pointer to struct rte_event_eth_rx_adapter_queue_conf
> > > +
> > > + * @return
> > > + *  - 0: Success, Receive queue added correctly.
> > > + *  - <0: Error code on failure.
> > > + */
> > > +__rte_experimental
> > > +int rte_event_eth_rx_adapter_queue_conf_get(uint8_t id,
> > > +                       uint16_t eth_dev_id,
> > > +                       uint16_t rx_queue_id,
> > > +                       struct rte_event_eth_rx_adapter_queue_conf
> > > +*queue_conf);
> > > +
> > > +
> > >  #ifdef __cplusplus
> > >  }
> > >  #endif
> > > diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map
> > > index
> > > cd86d2d..9f28016 100644
> > > --- a/lib/eventdev/version.map
> > > +++ b/lib/eventdev/version.map
> > > @@ -143,6 +143,7 @@ EXPERIMENTAL {
> > >         rte_event_vector_pool_create;
> > >         rte_event_eth_rx_adapter_vector_limits_get;
> > >         __rte_eventdev_trace_crypto_adapter_enqueue;
> > > +       rte_event_eth_rx_adapter_queue_conf_get;
> > >  };
> > >
> > >  INTERNAL {
> > > --
> > > 2.6.4
> > >