From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by dpdk.org (Postfix) with ESMTP id 246DB692F for ; Tue, 9 May 2017 22:21:46 +0200 (CEST) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga104.jf.intel.com with ESMTP; 09 May 2017 13:21:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.38,316,1491289200"; d="scan'208";a="1128352047" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by orsmga001.jf.intel.com with ESMTP; 09 May 2017 13:21:45 -0700 Received: from fmsmsx157.amr.corp.intel.com (10.18.116.73) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.319.2; Tue, 9 May 2017 13:21:45 -0700 Received: from fmsmsx108.amr.corp.intel.com ([169.254.9.101]) by FMSMSX157.amr.corp.intel.com ([169.254.14.30]) with mapi id 14.03.0319.002; Tue, 9 May 2017 13:21:44 -0700 From: "Eads, Gage" To: Jerin Jacob CC: "dev@dpdk.org" , "thomas@monjalon.net" , "Richardson, Bruce" , "Van Haaren, Harry" , "hemant.agrawal@nxp.com" , "nipun.gupta@nxp.com" , "Vangati, Narender" Thread-Topic: [dpdk-dev] [PATCH] eventdev: abstract ethdev HW capability to inject packets Thread-Index: AQHSxaRXEEceKwbbnkaJ7g4lFmh3naHnSIU5gAUjlLA= Date: Tue, 9 May 2017 20:21:44 +0000 Message-ID: <9184057F7FC11744A2107296B6B8EB1E01EA9216@FMSMSX108.amr.corp.intel.com> References: <20170505133341.31138-1-jerin.jacob@caviumnetworks.com> <9184057F7FC11744A2107296B6B8EB1E01EA8327@FMSMSX108.amr.corp.intel.com> <20170506130806.GA9838@jerin> In-Reply-To: <20170506130806.GA9838@jerin> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.1.200.106] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH] eventdev: abstract ethdev HW capability to inject packets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 May 2017 20:21:48 -0000 > -----Original Message----- > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > Sent: Saturday, May 6, 2017 8:08 AM > To: Eads, Gage > Cc: dev@dpdk.org; thomas@monjalon.net; Richardson, Bruce > ; Van Haaren, Harry > ; hemant.agrawal@nxp.com; > nipun.gupta@nxp.com; Vangati, Narender > Subject: Re: [dpdk-dev] [PATCH] eventdev: abstract ethdev HW capability = to > inject packets > =20 > -----Original Message----- > > Date: Fri, 5 May 2017 19:02:59 +0000 > > From: "Eads, Gage" > > To: Jerin Jacob , "dev@dpdk.org" > > > > CC: "thomas@monjalon.net" , "Richardson, Bruce" > > , "Van Haaren, Harry" > > , "hemant.agrawal@nxp.com" > > , "nipun.gupta@nxp.com" > > , "Vangati, Narender" > > > > Subject: RE: [dpdk-dev] [PATCH] eventdev: abstract ethdev HW > > capability to inject packets > > > > Hi Jerin, > =20 > Hi Gage, > =20 > > > > > -----Original Message----- > > > From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com] > > > Sent: Friday, May 5, 2017 8:34 AM > > > To: dev@dpdk.org > > > Cc: thomas@monjalon.net; Richardson, Bruce > > > ; Van Haaren, Harry > > > ; hemant.agrawal@nxp.com; Eads, Gage > > > ; nipun.gupta@nxp.com; Vangati, Narender > > > ; Jerin Jacob > > > > > > Subject: [dpdk-dev] [PATCH] eventdev: abstract ethdev HW capability > > > to inject packets > > > > > > Some Ethdev Hardware is capable of injecting the events(Ethernet > > > packets) to eventdev without the need for dedicated service cores o= n Rx > path. > > > Since eventdev API is device capability agnostic, we need to > > > address three combinations of ethdev and eventdev PMD drivers. > > > > > > 1) Ethdev HW is not capable of injecting the packets and SW > > > eventdev driver(All existing ethdev PMD + drivers/event/sw PMD > > > combination) > > > 2) Ethdev HW is not capable of injecting the packets and not > > > compatible HW eventdev driver(All existing ethdev PMD + > > > driver/event/octeontx PMD > > > combination) > > > 3) Ethdev HW is capable of injecting the packet to compatible HW > > > eventdev driver. > > > > > > This patch abstract such capability disparity and have unified way > > > to get the functionality in the application. > > > > > > NOTE: The same infrastructure can be used _if_ an SW PMD wish to > > > create the default producer as service function with the service > > > core scheme in EAL to make it completely transparent to the > > > application for the default case. The selection of service core ena= blement > can be a vdev argument to SW PMD. > > > > > > > I'm not sure that a vdev argument/arguments is the right way to go for= cases > 1 and 2. It'll be rather complicated to specify the service core configu= ration per > event producer on the command line, and that configuration can't use run= time > information. > =20 > Yes. I didn't meant service core configuration needs to passed through v= dev > argument. I meant, only the true/false parameter in vdev command line to > enable default producer service function in PMD _if_ a PMD wish to do th= at > way. > A PMD can choose to return always FALSE for rte_event_dev_has_producer() > as it will have be hook to PMD driver. > =20 > Since service core proposal, talked only about "PMD" registering the ser= vice > core functions NOT _applications_, I thought you guys need to move defau= lt > producer handler in PMD and invoke them through service core infrastruct= ure > in application with application selected service core mask. > =20 > if you think, it always makes sense to return FALSE to > rte_event_dev_has_producer() on SW PMD then it is fine. > This patch does not dictate to return TRUE and it is completely in PMD c= ontrol. > I can remove NOTE: section in git commit section it is confusing. > =20 I'm fine either way. The NOTE's just a suggestion, so no harm leaving it in= . > > > > What do you think about this approach: We develop a library of device > producer code, with interfaces designed to run on service cores, that th= e app > can run if rte_event_dev_has_producer() returns FALSE. The device produc= er > code would, at minimum, have some configuration function (more or less > equivalent to the rte_event_queue_producer_conf) and a run function that= (at > a high level) processes eth/crypto/etc.-dev queues and injects events in= to an > event device. > =20 > That fine. > =20 > > > > For example, if rte_event_dev_has_producer() indicates there's no hard= ware > support for connecting an ethdev to the event device, the application > configures the ethdev->eventdev producer code and installs the run funct= ion > with the service core infrastructure. > > > > Plus, this approach allows an application to plug in their own device = producer > logic, if they require some app-specific functionality. In that case, th= e DPDK- > provided device producer could serve as a template for the app developer= . > =20 > Make sense. I believe, which is inline with rte_event_dev_has_producer() > comment in the patch > ------------------------------------------ > When application needs to setup an event producer to inject the events t= o > eventdev, application must check the event device producer capability by > invoking this function, On success, application can configure to inject = the > events to eventdev by setting up the event queue using > rte_event_queue_setup() with checked *conf*. > On failure due to lack of adequate capability in PMD or an application t= hat wish > to > ~~~~~~~~~~~~~~~~~~~= ~~~~~~~~~~~~~~~ > inject the event through application, it must create > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > additional event port by configuring the eventdev with rte_event_dev_con= fig() > and use rte_event_enqueue_burst() to inject the event through op =3D=3D > RTE_EVENT_OP_NEW operation > ------------------------------------------- > =20 > > If you agree this approach -- at least at a high level -- I'll put tog= ether an RFC > the library and a header file for an eth->eventdev block and we can work= on the > details. > =20 > Sure. Please send the RFC. > =20 Great -- I'll put it up on the ML shortly. > =20 > > > > > Detailed comments are added in the header file. > > > > > > Example API usage: > > > > > > - rte_eth_dev_configure(port,..); > > > - rte_eth_rx_queue_setup(port,..); > > > - rte_eth_dev_start(port,..); > > > > > > struct rte_event_queue_producer_conf ethdev_conf =3D { > > > .event_type =3D RTE_EVENT_TYPE_ETHDEV; > > > .sched_type =3D RTE_SCHED_TYPE_ATOMIC; > > > .priority =3D RTE_EVENT_DEV_PRIORITY_LOWEST; > > > .ethdev.ethdev_port =3D port; > > > .ethdev.rx_queue_id =3D -1; > > > }; > > > > > > - rte_event_dev_has_producer(dev_id, ðdev_conf); # configure > > > additional producer ports based on eventdev producer capability. > > > - rte_event_dev_configure(dev_id,..); > > > > > > struct rte_event_queue_conf =3D conf { > > > nb_producers =3D 1; > > > producers =3D ðdev_conf; > > > }; > > > # configure event queue based on eventdev producer capability > > > - rte_event_queue_setup(dev_id, &conf,..); > > > - rte_event_port_setup(dev_id,..); > > > - rte_event_dev_start(dev_id); > > > > > > Signed-off-by: Jerin Jacob > > > --- > > > This patch is based on the review commets from following RFC > > > http://dpdk.org/ml/archives/dev/2017-May/065176.html > > > --- > > > lib/librte_eventdev/rte_eventdev.h | 110 > > > +++++++++++++++++++++++++++++++++++++ > > > 1 file changed, 110 insertions(+) > > > > > > diff --git a/lib/librte_eventdev/rte_eventdev.h > > > b/lib/librte_eventdev/rte_eventdev.h > > > index 20e7293e0..202d03f61 100644 > > > --- a/lib/librte_eventdev/rte_eventdev.h > > > +++ b/lib/librte_eventdev/rte_eventdev.h > > > @@ -396,6 +396,100 @@ struct rte_event_dev_info { int > > > rte_event_dev_info_get(uint8_t dev_id, struct rte_event_dev_info > > > *dev_info); > > > > > > +/** Eventdev producer configuration structure. > > > + * The events are injected to event device through *enqueue* > > > operation +with + * op =3D=3D RTE_EVENT_OP_NEW by event producers = in > > > the system. If the +event + * producer is an Ethernet device then > > > eventdev PMD may operate in +conjunction + * with ethdev PMD to > > > injects the events(Ethernet packets) to eventdev. > > > + * The event injection can happen in HW or SW or the combination > > > of +these two + * based on the HW capabilities of target eventdev > > > and ethdev PMDs. > > > + * > > > + * @see rte_event_dev_has_producer() rte_event_queue_setup() + * > > > + */ +struct rte_event_dev_producer_conf { > > > + uint32_t event_type:4; > > > + /**< Event type to classify the event source. > > > + * @see RTE_EVENT_TYPE_ETHDEV, (RTE_EVENT_TYPE_*) > > > + */ > > > + uint8_t sched_type:2; > > > + /**< Scheduler synchronization type (RTE_SCHED_TYPE_*) > > > + * associated with flow id on a given event queue for the enqueue > > > + * operation. > > > + */ > > > + uint8_t priority; > > > + /**< Event priority relative to other events in the > > > + * event queue. The requested priority should in the > > > + * range of [RTE_EVENT_DEV_PRIORITY_HIGHEST, > > > + * RTE_EVENT_DEV_PRIORITY_LOWEST]. > > > + * The implementation shall normalize the requested > > > + * priority to supported priority value. > > > + * Valid when the device has > > > + * RTE_EVENT_DEV_CAP_EVENT_QOS capability. > > > + */ > > > + union { > > > + struct rte_event_ethdev_producer { > > > + uint16_t ethdev_port; > > > + /**< The port identifier of the Ethernet device */ > > > + int32_t rx_queue_id; > > > + /**< The index of the receive queue from which to > > > + * retrieve the input packets and inject to eventdev. > > > + * The value -1 denotes all the Rx queues configured > > > + * for the given ethdev_port are selected for retrieving > > > + * the input packets and then injecting the > > > + * events/packets to eventdev. > > > + * The rte_eth_rx_burst() result is undefined > > > + * if application invokes on bounded ethdev_port and > > > + * rx_queue_id. > > > + */ > > > + } ethdev; /* RTE_EVENT_TYPE_ETHDEV */ > > > + /**< Valid when event_type =3D=3D RTE_EVENT_TYPE_ETHDEV. > > > + * Implementation may use mbuff's rss->hash value as > > > + * flow_id for the enqueue operation. > > > + */ > > > + }; > > > +}; > > > + > > > +/** > > > + * Check for eventdev producer capability for the given eventdev > > > +producer *conf* + * on an event device. > > > + * > > > + * When application needs to setup an event producer to inject the > > > +events to + * eventdev, application must check the event device > > > producer +capability by + * invoking this function, On success, > > > application can configure to +inject the + * events to eventdev by > > > setting up the event queue using + * rte_event_queue_setup() with > > > checked *conf*. > > > + * On failure due to lack of adequate capability in PMD or an > > > +application that + * wish to inject the event through application, > > > it must create +additional + * event port by configuring the > > > eventdev with rte_event_dev_config() +and use + * > > > rte_event_enqueue_burst() to inject the event through + * op =3D=3D > > > RTE_EVENT_OP_NEW operation. > > > + * > > > + * @param dev_id > > > + * The identifier of the device. > > > + * @param conf > > > + * The eventdev producer configuration structure. > > > + * > > > + * @see rte_event_queue_setup() rte_event_dev_config() + * + * > > > @return > > > + * - TRUE (value different than 0) if the eventdev is capable of= producing > the > > > + * events for the given *conf* > > > + * - FALSE (value zero) if the eventdev is not capable of produc= ing the > events > > > + * for the given *conf* > > > + * > > > + * PMD may have partial support for the requested capability. > > > + * On return FALSE, PMD sets rte_errno to reflect the partial su= pport. > > > + * Possible rte_errno values include: > > > + * - -EOPNOTSUPP: The eventdev is not capable of pulling the eve= nts > from > > > + * the specific producer queue. On this errno, The caller may tr= y to > > > + * check the capability with rx_queue_id as -1 in *conf*. > > > > How do we indicate the PMD has zero support? E.g., return FALSE and se= t > rte_errno to 0? > =20 > Yes > =20 > > > > > + * > > > + */ > > > +int > > > +rte_event_dev_has_producer(uint8_t dev_id, > > > + struct rte_event_dev_producer_conf *conf); > > > > I interpret this to mean the user queries the presence of a particular= type of > producer by setting conf->event_type. Is that correct? > =20 > For other sources, say(crypto or timer) the events are generated based o= n > explicit application request(i.e for crypto: application needs to submit= the > crypto request and, on completion the event will be produced) not from w= ire The question is, who produces that completion event -- hardware or software= ? rte_event_dev_has_producer() can tell the app whether it needs to registe= r a (for example) service core that moves packets from crypto response queu= es to event queues. Otherwise, I'm not sure how apps could be made portable= across platforms with different crypto, timer, etc. capabilities. > like ethdev Rx case. > =20 > So I think, mostly for ethdev Rx case only we need this scheme on other = cases > when we submit the crypto job or timer job we specify where to inject th= e > event on work completion. > =20 > =20 > > > > For an ethdev producer, I assume the PMD's response would depend on co= nf- > >ethdev_port -- that is, it could return true for a hardware ethdev and = but > would always return false for a software (e.g. pcap) ethdev. > =20 > Yes. > =20 > > > > Thanks, > > Gage