Re: [dpdk-dev] [PATCH v4 3/6] eventdev: implement the northbound APIs

[Nipun Gupta <nipun.gupta@nxp.com>]

> -----Original Message-----
> From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
> Sent: Thursday, February 02, 2017 20:02
> To: Nipun Gupta <nipun.gupta@nxp.com>
> Cc: dev@dpdk.org; thomas.monjalon@6wind.com;
> bruce.richardson@intel.com; Hemant Agrawal <hemant.agrawal@nxp.com>;
> gage.eads@intel.com; harry.van.haaren@intel.com
> Subject: Re: [dpdk-dev] [PATCH v4 3/6] eventdev: implement the northbound
> APIs
> 
> On Thu, Feb 02, 2017 at 11:19:45AM +0000, Nipun Gupta wrote:
> >
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Jerin Jacob
> > > Sent: Wednesday, December 21, 2016 14:55
> > > To: dev@dpdk.org
> > > Cc: thomas.monjalon@6wind.com; bruce.richardson@intel.com; Hemant
> > > Agrawal <hemant.agrawal@nxp.com>; gage.eads@intel.com;
> > > harry.van.haaren@intel.com; Jerin Jacob
> <jerin.jacob@caviumnetworks.com>
> > > Subject: [dpdk-dev] [PATCH v4 3/6] eventdev: implement the northbound
> APIs
> > >
> > > This patch implements northbound eventdev API interface using southbond
> > > driver interface
> > >
> > > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > > Acked-by: Bruce Richardson <bruce.richardson@intel.com>
> > > ---
> > >  config/common_base                           |   6 +
> > >  lib/Makefile                                 |   1 +
> > >  lib/librte_eal/common/include/rte_log.h      |   1 +
> > >  lib/librte_eventdev/Makefile                 |  57 ++
> > >  lib/librte_eventdev/rte_eventdev.c           | 986
> > > +++++++++++++++++++++++++++
> > >  lib/librte_eventdev/rte_eventdev.h           | 106 ++-
> > >  lib/librte_eventdev/rte_eventdev_pmd.h       | 109 +++
> > >  lib/librte_eventdev/rte_eventdev_version.map |  33 +
> > >  mk/rte.app.mk                                |   1 +
> > >  9 files changed, 1294 insertions(+), 6 deletions(-)  create mode 100644
> > > lib/librte_eventdev/Makefile  create mode 100644
> > > lib/librte_eventdev/rte_eventdev.c
> > >  create mode 100644 lib/librte_eventdev/rte_eventdev_version.map
> > >
> >
> > <Snip>
> >
> > > +static inline int
> > > +rte_event_dev_port_config(struct rte_eventdev *dev, uint8_t nb_ports) {
> > > +	uint8_t old_nb_ports = dev->data->nb_ports;
> > > +	void **ports;
> > > +	uint16_t *links_map;
> > > +	uint8_t *ports_dequeue_depth;
> > > +	uint8_t *ports_enqueue_depth;
> > > +	unsigned int i;
> > > +
> > > +	RTE_EDEV_LOG_DEBUG("Setup %d ports on device %u", nb_ports,
> > > +			 dev->data->dev_id);
> > > +
> > > +	/* First time configuration */
> > > +	if (dev->data->ports == NULL && nb_ports != 0) {
> > > +		dev->data->ports = rte_zmalloc_socket("eventdev->data-
> > > >ports",
> > > +				sizeof(dev->data->ports[0]) * nb_ports,
> > > +				RTE_CACHE_LINE_SIZE, dev->data->socket_id);
> > > +		if (dev->data->ports == NULL) {
> > > +			dev->data->nb_ports = 0;
> > > +			RTE_EDEV_LOG_ERR("failed to get mem for port meta
> > > data,"
> > > +					"nb_ports %u", nb_ports);
> > > +			return -(ENOMEM);
> > > +		}
> > > +
> > > +		/* Allocate memory to store ports dequeue depth */
> > > +		dev->data->ports_dequeue_depth =
> > > +			rte_zmalloc_socket("eventdev-
> > > >ports_dequeue_depth",
> > > +			sizeof(dev->data->ports_dequeue_depth[0]) *
> > > nb_ports,
> > > +			RTE_CACHE_LINE_SIZE, dev->data->socket_id);
> > > +		if (dev->data->ports_dequeue_depth == NULL) {
> > > +			dev->data->nb_ports = 0;
> > > +			RTE_EDEV_LOG_ERR("failed to get mem for port deq
> > > meta,"
> > > +					"nb_ports %u", nb_ports);
> > > +			return -(ENOMEM);
> > > +		}
> > > +
> > > +		/* Allocate memory to store ports enqueue depth */
> > > +		dev->data->ports_enqueue_depth =
> > > +			rte_zmalloc_socket("eventdev-
> > > >ports_enqueue_depth",
> > > +			sizeof(dev->data->ports_enqueue_depth[0]) *
> > > nb_ports,
> > > +			RTE_CACHE_LINE_SIZE, dev->data->socket_id);
> > > +		if (dev->data->ports_enqueue_depth == NULL) {
> > > +			dev->data->nb_ports = 0;
> > > +			RTE_EDEV_LOG_ERR("failed to get mem for port enq
> > > meta,"
> > > +					"nb_ports %u", nb_ports);
> > > +			return -(ENOMEM);
> > > +		}
> > > +
> > > +		/* Allocate memory to store queue to port link connection */
> > > +		dev->data->links_map =
> > > +			rte_zmalloc_socket("eventdev->links_map",
> > > +			sizeof(dev->data->links_map[0]) * nb_ports *
> > > +			RTE_EVENT_MAX_QUEUES_PER_DEV,
> > > +			RTE_CACHE_LINE_SIZE, dev->data->socket_id);
> > > +		if (dev->data->links_map == NULL) {
> > > +			dev->data->nb_ports = 0;
> > > +			RTE_EDEV_LOG_ERR("failed to get mem for port_map
> > > area,"
> > > +					"nb_ports %u", nb_ports);
> > > +			return -(ENOMEM);
> > > +		}
> >
> > I think we also need to set all the 'links map' to
> EVENT_QUEUE_SERVICE_PRIORITY_INVALID
> > on zmalloc.
> 
> Just after the port_setup, we are setting to
> EVENT_QUEUE_SERVICE_PRIORITY_INVALID in
> rte_event_port_unlink(). So it looks OK to me.
> 
>         diag = (*dev->dev_ops->port_setup)(dev, port_id, port_conf);
> 
>         /* Unlink all the queues from this port(default state after
> 	 * setup) */
>         if (!diag)
>                 diag = rte_event_port_unlink(dev_id, port_id, NULL, 0);
> 
> 

In case of NULL parameter as queues, in the rte_event_port_unlink(),
the number of 'links_map' which are being set to
EVENT_QUEUE_SERVICE_PRIORITY_INVALID equals the number of
configured queues:

	if (queues == NULL) {
		for (i = 0; i < dev->data->nb_queues; i++)
			all_queues[i] = i;
		queues = all_queues;
		nb_unlinks = dev->data->nb_queues;
	}

So, the EVENT_QUEUE_SERVICE_PRIORITY_INVALID does not gets set for
complete 'links_map' memory.

The API rte_event_port_links_get() will probably return wrong number of links
as the loop there is on RTE_EVENT_MAX_QUEUES_PER_DEV:

	for (i = 0; i < RTE_EVENT_MAX_QUEUES_PER_DEV; i++) {
		if (links_map[i] != EVENT_QUEUE_SERVICE_PRIORITY_INVALID) {
			queues[count] = i;
			priorities[count] = (uint8_t)links_map[i];
			++count;
		}
	}

> >
> > > +	} else if (dev->data->ports != NULL && nb_ports != 0) {/* re-config */
> > > +		RTE_FUNC_PTR_OR_ERR_RET(*dev->dev_ops->port_release, -
> > > ENOTSUP);
> > > +
> > > +		ports = dev->data->ports;
> > > +		ports_dequeue_depth = dev->data->ports_dequeue_depth;
> > > +		ports_enqueue_depth = dev->data->ports_enqueue_depth;
> > > +		links_map = dev->data->links_map;
> > > +
> >
> > <Snip>
> >
> > > +int
> > > +rte_event_port_setup(uint8_t dev_id, uint8_t port_id,
> > > +		     const struct rte_event_port_conf *port_conf) {
> > > +	struct rte_eventdev *dev;
> > > +	struct rte_event_port_conf def_conf;
> > > +	int diag;
> > > +
> > > +	RTE_EVENTDEV_VALID_DEVID_OR_ERR_RET(dev_id, -EINVAL);
> > > +	dev = &rte_eventdevs[dev_id];
> > > +
> > > +	if (!is_valid_port(dev, port_id)) {
> > > +		RTE_EDEV_LOG_ERR("Invalid port_id=%" PRIu8, port_id);
> > > +		return -EINVAL;
> > > +	}
> > > +
> > > +	/* Check new_event_threshold limit */
> > > +	if ((port_conf && !port_conf->new_event_threshold) ||
> > > +			(port_conf && port_conf->new_event_threshold >
> > > +				 dev->data->dev_conf.nb_events_limit)) {
> >
> > As mentioned in 'rte_eventdev.h', the 'new_event_threshold' is valid for
> *closed systems*,
> > so is the above check valid for *open systems*?
> 
> new_event_threshold is valid  only for *closed systems*. If you need any
> change then please suggest.

This is fine, but I think we also need to mention in the new_event_threshold
description in 'rte_eventdev.h' that for open systems this needs to be set to
'-1', because otherwise the check here will fail.

And this is also for ' struct rte_event_dev_config'->'nb_events_limit', which is
required to be set to '-1' for open systems by the application. Right?

I'll send a patch updating this in rte_eventdev.h file.

Regards,
Nipun

> 
> > Or is it implicit that for open systems the 'port_conf->new_event_threshold'
> should be
> > set to '-1' by the application just as it is for 'max_num_events' of 'struct
> rte_event_dev_info'.
> 
> 
> 
> >
> > > +		RTE_EDEV_LOG_ERR(
> > > +		   "dev%d port%d Invalid event_threshold=%d
> > > nb_events_limit=%d",
> > > +			dev_id, port_id, port_conf->new_event_threshold,
> > > +			dev->data->dev_conf.nb_events_limit);
> > > +		return -EINVAL;
> > > +	}
> > > +
> >
> > <Snip>
> >
> > Regards,
> > Nipun