From: "McDaniel, Timothy" <timothy.mcdaniel@intel.com> To: Jerin Jacob <jerinjacobk@gmail.com> Cc: Ray Kinsella <mdr@ashroe.eu>, Neil Horman <nhorman@tuxdriver.com>, "Jerin Jacob" <jerinj@marvell.com>, Mattias Rönnblom <mattias.ronnblom@ericsson.com>, dpdk-dev <dev@dpdk.org>, "Eads, Gage" <gage.eads@intel.com>, "Van Haaren, Harry" <harry.van.haaren@intel.com> Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites Date: Wed, 1 Jul 2020 16:48:48 +0000 Message-ID: <SN6PR11MB3103B2A2FA4587E10AAD73C29E6C0@SN6PR11MB3103.namprd11.prod.outlook.com> (raw) In-Reply-To: <CALBAE1Ome=wxMbPFfraA_BGOOqZq+4TXudBcTooW0i0-n=C_ng@mail.gmail.com> >-----Original Message----- >From: Jerin Jacob <jerinjacobk@gmail.com> >Sent: Tuesday, June 30, 2020 11:50 PM >To: McDaniel, Timothy <timothy.mcdaniel@intel.com> >Cc: Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; >Jerin Jacob <jerinj@marvell.com>; Mattias Rönnblom ><mattias.ronnblom@ericsson.com>; dpdk-dev <dev@dpdk.org>; Eads, Gage ><gage.eads@intel.com>; Van Haaren, Harry <harry.van.haaren@intel.com> >Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites > >On Wed, Jul 1, 2020 at 12:57 AM McDaniel, Timothy ><timothy.mcdaniel@intel.com> wrote: >> >> >-----Original Message----- >> >From: Jerin Jacob <jerinjacobk@gmail.com> >> >Sent: Tuesday, June 30, 2020 10:58 AM >> >To: McDaniel, Timothy <timothy.mcdaniel@intel.com> >> >Cc: Ray Kinsella <mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>; >> >Jerin Jacob <jerinj@marvell.com>; Mattias Rönnblom >> ><mattias.ronnblom@ericsson.com>; dpdk-dev <dev@dpdk.org>; Eads, Gage >> ><gage.eads@intel.com>; Van Haaren, Harry <harry.van.haaren@intel.com> >> >Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites >> > >> >On Tue, Jun 30, 2020 at 9:12 PM McDaniel, Timothy >> ><timothy.mcdaniel@intel.com> wrote: >> >> >> >> >-----Original Message----- >> >> >From: Jerin Jacob <jerinjacobk@gmail.com> >> >> >Sent: Monday, June 29, 2020 11:21 PM >> >> >To: McDaniel, Timothy <timothy.mcdaniel@intel.com> >> >> >Cc: Ray Kinsella <mdr@ashroe.eu>; Neil Horman ><nhorman@tuxdriver.com>; >> >> >Jerin Jacob <jerinj@marvell.com>; Mattias Rönnblom >> >> ><mattias.ronnblom@ericsson.com>; dpdk-dev <dev@dpdk.org>; Eads, >Gage >> >> ><gage.eads@intel.com>; Van Haaren, Harry ><harry.van.haaren@intel.com> >> >> >Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream >prerequisites >> >> > >> >> >On Tue, Jun 30, 2020 at 1:01 AM McDaniel, Timothy >> >> ><timothy.mcdaniel@intel.com> wrote: >> >> >> >> >> >> -----Original Message----- >> >> >> From: Jerin Jacob <jerinjacobk@gmail.com> >> >> >> Sent: Saturday, June 27, 2020 2:45 AM >> >> >> To: McDaniel, Timothy <timothy.mcdaniel@intel.com>; Ray Kinsella >> >> ><mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com> >> >> >> Cc: Jerin Jacob <jerinj@marvell.com>; Mattias Rönnblom >> >> ><mattias.ronnblom@ericsson.com>; dpdk-dev <dev@dpdk.org>; Eads, >Gage >> >> ><gage.eads@intel.com>; Van Haaren, Harry ><harry.van.haaren@intel.com> >> >> >> Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream >prerequisites >> >> >> >> >> >> > + >> >> >> > +/** Event port configuration structure */ >> >> >> > +struct rte_event_port_conf_v20 { >> >> >> > + int32_t new_event_threshold; >> >> >> > + /**< A backpressure threshold for new event enqueues on this >port. >> >> >> > + * Use for *closed system* event dev where event capacity is >limited, >> >> >> > + * and cannot exceed the capacity of the event dev. >> >> >> > + * Configuring ports with different thresholds can make higher >priority >> >> >> > + * traffic less likely to be backpressured. >> >> >> > + * For example, a port used to inject NIC Rx packets into the event >dev >> >> >> > + * can have a lower threshold so as not to overwhelm the device, >> >> >> > + * while ports used for worker pools can have a higher threshold. >> >> >> > + * This value cannot exceed the *nb_events_limit* >> >> >> > + * which was previously supplied to rte_event_dev_configure(). >> >> >> > + * This should be set to '-1' for *open system*. >> >> >> > + */ >> >> >> > + uint16_t dequeue_depth; >> >> >> > + /**< Configure number of bulk dequeues for this event port. >> >> >> > + * This value cannot exceed the *nb_event_port_dequeue_depth* >> >> >> > + * which previously supplied to rte_event_dev_configure(). >> >> >> > + * Ignored when device is not >RTE_EVENT_DEV_CAP_BURST_MODE >> >> >capable. >> >> >> > + */ >> >> >> > + uint16_t enqueue_depth; >> >> >> > + /**< Configure number of bulk enqueues for this event port. >> >> >> > + * This value cannot exceed the *nb_event_port_enqueue_depth* >> >> >> > + * which previously supplied to rte_event_dev_configure(). >> >> >> > + * Ignored when device is not >RTE_EVENT_DEV_CAP_BURST_MODE >> >> >capable. >> >> >> > + */ >> >> >> > uint8_t disable_implicit_release; >> >> >> > /**< Configure the port not to release outstanding events in >> >> >> > * rte_event_dev_dequeue_burst(). If true, all events received >through >> >> >> > @@ -733,6 +911,14 @@ struct rte_event_port_conf { >> >> >> > rte_event_port_default_conf_get(uint8_t dev_id, uint8_t port_id, >> >> >> > struct rte_event_port_conf *port_conf); >> >> >> > >> >> >> > +int >> >> >> > +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t port_id, >> >> >> > + struct rte_event_port_conf_v20 *port_conf); >> >> >> > + >> >> >> > +int >> >> >> > +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t port_id, >> >> >> > + struct rte_event_port_conf *port_conf); >> >> >> >> >> >> Hi Timothy, >> >> >> >> >> >> + ABI Maintainers (Ray, Neil) >> >> >> >> >> >> # As per my understanding, the structures can not be versioned, only >> >> >> function can be versioned. >> >> >> i.e we can not make any change to " struct rte_event_port_conf" >> >> >> >> >> >> # We have a similar case with ethdev and it deferred to next release >v20.11 >> >> >> http://patches.dpdk.org/patch/69113/ >> >> >> >> >> >> Regarding the API changes: >> >> >> # The slow path changes general looks good to me. I will review the >> >> >> next level in the coming days >> >> >> # The following fast path changes bothers to me. Could you share more >> >> >> details on below change? >> >> >> >> >> >> diff --git a/app/test-eventdev/test_order_atq.c >> >> >> b/app/test-eventdev/test_order_atq.c >> >> >> index 3366cfc..8246b96 100644 >> >> >> --- a/app/test-eventdev/test_order_atq.c >> >> >> +++ b/app/test-eventdev/test_order_atq.c >> >> >> @@ -34,6 +34,8 @@ >> >> >> continue; >> >> >> } >> >> >> >> >> >> + ev.flow_id = ev.mbuf->udata64; >> >> >> + >> >> >> # Since RC1 is near, I am not sure how to accommodate the API changes >> >> >> now and sort out ABI stuffs. >> >> >> # Other concern is eventdev spec get bloated with versioning files >> >> >> just for ONE release as 20.11 will be OK to change the ABI. >> >> >> # While we discuss the API change, Please send deprecation notice for >> >> >> ABI change for 20.11, >> >> >> so that there is no ambiguity of this patch for the 20.11 release. >> >> >> >> >> >> Hello Jerin, >> >> >> >> >> >> Thank you for the review comments. >> >> >> >> >> >> With regard to your comments regarding the fast path flow_id change, >the >> >Intel >> >> >DLB hardware >> >> >> is not capable of transferring the flow_id as part of the event itself. We >> >> >therefore require a mechanism >> >> >> to accomplish this. What we have done to work around this is to require >the >> >> >application to embed the flow_id >> >> >> within the data payload. The new flag, #define >> >> >RTE_EVENT_DEV_CAP_CARRY_FLOW_ID (1ULL << 9), can be used >> >> >> by applications to determine if they need to embed the flow_id, or if its >> >> >automatically propagated and present in the >> >> >> received event. >> >> >> >> >> >> What we should have done is to wrap the assignment with a conditional. >> >> >> >> >> >> if (!(device_capability_flags & RTE_EVENT_DEV_CAP_CARRY_FLOW_ID)) >> >> >> ev.flow_id = ev.mbuf->udata64; >> >> > >> >> >Two problems with this approach, >> >> >1) we are assuming mbuf udata64 field is available for DLB driver >> >> >2) It won't work with another adapter, eventdev has no dependency with >mbuf >> >> > >> >> >> >> This snippet is not intended to suggest that udata64 always be used to store >the >> >flow ID, but as an example of how an application could do it. Some >applications >> >won’t need to carry the flow ID through; others can select an unused field in >the >> >event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case) re- >generate >> >the flow ID in pipeline stages that require it. >> > >> >OK. >> >> >> >> >Question: >> >> >1) In the case of DLB hardware, on dequeue(), what HW returns? is it >> >> >only event pointer and not have any other metadata like schedule_type >> >> >etc. >> >> > >> >> >> >> The DLB device provides a 16B “queue entry” that consists of: >> >> >> >> * 8B event data >> >> * Queue ID >> >> * Priority >> >> * Scheduling type >> >> * 19 bits of carried-through data >> >> * Assorted error/debug/reserved bits that are set by the device (not >carried- >> >through) >> >> >> >> For the carried-through 19b, we use 12b for event_type and >sub_event_type. >> > >> >I can only think of TWO options to help >> >1) Since event pointer always cache aligned, You could grab LSB >> >6bits(2^6 = 64B ) and 7 bits from (19b - 12b) carried through >> >structure >> >2) Have separate mempool driver using existing drivers, ie "event >> >pointer" + or - some offset have any amount of custom data. >> > >> >> We can't guarantee that the event will contain a pointer -- it's possible that 8B >is inline data (i.e. struct rte_event's u64 field). >> >> It's really an application decision -- for example an app could allocate space in >the 'mbuf private data' to store the flow ID, if the event device lacks that carry- >flow-ID capability and the other mbuf fields can't be used for whatever reason. >> We modified the tests, sample apps to show how this might be done, not >necessarily how it must be done. > > >Yeah. If HW has limitation we can't do much. It is OK to change >eventdev spec to support new HW limitations. aka, >RTE_EVENT_DEV_CAP_CARRY_FLOW_ID is OK. >Please update existing drivers has this >RTE_EVENT_DEV_CAP_CARRY_FLOW_ID capability which is missing in the >patch(I believe) > >> >> > >> >> >> >> > >> >> >> >> >> >> This would minimize/eliminate any performance impact due to the >> >processor's >> >> >branch prediction logic. >> > >> >I think, If we need to change common fastpath, better we need to make >> >it template to create code for compile-time to have absolute zero >> >overhead >> >and use runtime. >> >See app/test-eventdev/test_order_atq.c: function: worker_wrapper() >> >_create_ worker at compile time based on runtime capability. >> > >> >> Yes, that would be perfect. Thanks for the example! > >Where ever you are making fastpath change, Please follow this scheme >and send the next version. >In order to have clean and reusable code, you could have template >function and with "if" and it can opt-out in _compile_ time. >i.e > >no_inline generic_worker(..., _const_ uint64_t flags) >{ >.. >.. > >if (! flags & CAP_CARRY_FLOW_ID) > .... > >} > >worker_with_out_carry_flow_id() >{ > generic_worker(.., CAP_CARRY_FLOW_ID) >} > >normal_worker() >{ > generic_worker(.., 0) >} > >No other controversial top-level comments with this patch series. >Once we sorted out the ABI issues then I can review and merge. > Thanks Jerin. I'll get these changes into the v3 patch set. > >> >> > >> > >> >> >> The assignment then becomes in essence a NOOP for all event devices >that >> >are >> >> >capable of carrying the flow_id as part of the event payload itself. >> >> >> >> >> >> Thanks, >> >> >> Tim >> >> >> >> >> >> >> >> >> >> >> >> Thanks, >> >> >> Tim
next prev parent reply other threads:[~2020-07-01 16:48 UTC|newest] Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-27 4:37 [dpdk-dev] [PATCH 00/27] event/dlb Intel DLB PMD Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites Tim McDaniel 2020-06-27 7:44 ` Jerin Jacob 2020-06-29 19:30 ` McDaniel, Timothy 2020-06-30 4:21 ` Jerin Jacob 2020-06-30 15:37 ` McDaniel, Timothy 2020-06-30 15:57 ` Jerin Jacob 2020-06-30 19:26 ` McDaniel, Timothy 2020-06-30 20:40 ` Pavan Nikhilesh Bhagavatula 2020-06-30 21:07 ` McDaniel, Timothy 2020-07-01 4:50 ` Jerin Jacob 2020-07-01 16:48 ` McDaniel, Timothy [this message] 2020-06-30 11:22 ` Kinsella, Ray 2020-06-30 11:30 ` Jerin Jacob 2020-06-30 11:36 ` Kinsella, Ray 2020-06-30 12:14 ` Jerin Jacob 2020-07-02 15:21 ` Kinsella, Ray 2020-07-02 16:35 ` McDaniel, Timothy 2020-06-27 4:37 ` [dpdk-dev] [PATCH 02/27] eventdev: do not pass disable_implicit_release bit to trace macro Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 03/27] event/dlb: add shared code version 10.7.9 Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 04/27] event/dlb: add make and meson build infrastructure Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 05/27] event/dlb: add DLB documentation Tim McDaniel 2020-07-09 3:29 ` Eads, Gage 2020-06-27 4:37 ` [dpdk-dev] [PATCH 06/27] event/dlb: add dynamic logging Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 07/27] event/dlb: add private data structures and constants Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 08/27] event/dlb: add definitions shared with LKM or shared code Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 09/27] event/dlb: add inline functions used in multiple files Tim McDaniel 2020-07-07 12:02 ` Bruce Richardson 2020-07-07 14:33 ` McDaniel, Timothy 2020-06-27 4:37 ` [dpdk-dev] [PATCH 10/27] event/dlb: add PFPMD-specific interface layer to shared code Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 11/27] event/dlb: add flexible PMD to device interfaces Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 12/27] event/dlb: add the PMD's public interfaces Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 13/27] event/dlb: add xstats support Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 14/27] event/dlb: add PMD self-tests Tim McDaniel 2020-07-10 20:42 ` Eads, Gage 2020-07-29 18:56 ` McDaniel, Timothy 2020-06-27 4:37 ` [dpdk-dev] [PATCH 15/27] event/dlb: add probe Tim McDaniel 2020-07-09 3:45 ` Eads, Gage 2020-06-27 4:37 ` [dpdk-dev] [PATCH 16/27] event/dlb: add infos_get and configure Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 17/27] event/dlb: add queue_def_conf and port_def_conf Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 18/27] event/dlb: add queue setup Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 19/27] event/dlb: add port_setup Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 20/27] event/dlb: add port_link Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 21/27] event/dlb: add queue_release and port_release Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 22/27] event/dlb: add port_unlink and port_unlinks_in_progress Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 23/27] event/dlb: add eventdev_start Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 24/27] event/dlb: add timeout_ticks, dump, xstats, and selftest Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 25/27] event/dlb: add enqueue and its burst variants Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 26/27] event/dlb: add dequeue, dequeue_burst, and variants Tim McDaniel 2020-06-27 4:37 ` [dpdk-dev] [PATCH 27/27] event/dlb: add eventdev_stop and eventdev_close Tim McDaniel [not found] <1593232671-5690-0-git-send-email-timothy.mcdaniel@intel.com> 2020-07-30 19:49 ` [dpdk-dev] [PATCH 00/27] Add Intel DLM PMD to 20.11 McDaniel, Timothy 2020-07-30 19:49 ` [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites McDaniel, Timothy 2020-08-11 17:44 ` Jerin Jacob -- strict thread matches above, loose matches on Subject: below -- 2020-06-12 21:24 [dpdk-dev] [PATCH 00/27] V1 event/dlb add Intel DLB PMD McDaniel, Timothy 2020-06-12 21:24 ` [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites McDaniel, Timothy 2020-06-13 3:59 ` Jerin Jacob 2020-06-13 10:43 ` Mattias Rönnblom 2020-06-18 15:51 ` McDaniel, Timothy 2020-06-18 15:44 ` McDaniel, Timothy
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=SN6PR11MB3103B2A2FA4587E10AAD73C29E6C0@SN6PR11MB3103.namprd11.prod.outlook.com \ --to=timothy.mcdaniel@intel.com \ --cc=dev@dpdk.org \ --cc=gage.eads@intel.com \ --cc=harry.van.haaren@intel.com \ --cc=jerinj@marvell.com \ --cc=jerinjacobk@gmail.com \ --cc=mattias.ronnblom@ericsson.com \ --cc=mdr@ashroe.eu \ --cc=nhorman@tuxdriver.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
DPDK patches and discussions This inbox may be cloned and mirrored by anyone: git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \ dev@dpdk.org public-inbox-index dev Example config snippet for mirrors. Newsgroup available over NNTP: nntp://inbox.dpdk.org/inbox.dpdk.dev AGPL code for this site: git clone https://public-inbox.org/public-inbox.git