DPDK patches and discussions
 help / color / mirror / Atom feed
From: Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com>
To: "McDaniel, Timothy" <timothy.mcdaniel@intel.com>,
	Jerin Jacob <jerinjacobk@gmail.com>
Cc: "Ray Kinsella" <mdr@ashroe.eu>,
	"Neil Horman" <nhorman@tuxdriver.com>,
	"Jerin Jacob Kollanukkaran" <jerinj@marvell.com>,
	"Mattias Rönnblom" <mattias.ronnblom@ericsson.com>,
	dpdk-dev <dev@dpdk.org>, "Eads, Gage" <gage.eads@intel.com>,
	"Van Haaren, Harry" <harry.van.haaren@intel.com>
Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites
Date: Tue, 30 Jun 2020 20:40:18 +0000	[thread overview]
Message-ID: <BN6PR18MB1140CE7C396DC68F562B5E87DE6F0@BN6PR18MB1140.namprd18.prod.outlook.com> (raw)
In-Reply-To: <SN6PR11MB31032CD6AEEF141FEEFCF8709E6F0@SN6PR11MB3103.namprd11.prod.outlook.com>



>-----Original Message-----
>From: dev <dev-bounces@dpdk.org> On Behalf Of McDaniel, Timothy
>Sent: Wednesday, July 1, 2020 12:57 AM
>To: Jerin Jacob <jerinjacobk@gmail.com>
>Cc: Ray Kinsella <mdr@ashroe.eu>; Neil Horman
><nhorman@tuxdriver.com>; Jerin Jacob Kollanukkaran
><jerinj@marvell.com>; Mattias Rönnblom
><mattias.ronnblom@ericsson.com>; dpdk-dev <dev@dpdk.org>; Eads,
>Gage <gage.eads@intel.com>; Van Haaren, Harry
><harry.van.haaren@intel.com>
>Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
>prerequisites
>
>>-----Original Message-----
>>From: Jerin Jacob <jerinjacobk@gmail.com>
>>Sent: Tuesday, June 30, 2020 10:58 AM
>>To: McDaniel, Timothy <timothy.mcdaniel@intel.com>
>>Cc: Ray Kinsella <mdr@ashroe.eu>; Neil Horman
><nhorman@tuxdriver.com>;
>>Jerin Jacob <jerinj@marvell.com>; Mattias Rönnblom
>><mattias.ronnblom@ericsson.com>; dpdk-dev <dev@dpdk.org>; Eads,
>Gage
>><gage.eads@intel.com>; Van Haaren, Harry
><harry.van.haaren@intel.com>
>>Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
>prerequisites
>>
>>On Tue, Jun 30, 2020 at 9:12 PM McDaniel, Timothy
>><timothy.mcdaniel@intel.com> wrote:
>>>
>>> >-----Original Message-----
>>> >From: Jerin Jacob <jerinjacobk@gmail.com>
>>> >Sent: Monday, June 29, 2020 11:21 PM
>>> >To: McDaniel, Timothy <timothy.mcdaniel@intel.com>
>>> >Cc: Ray Kinsella <mdr@ashroe.eu>; Neil Horman
><nhorman@tuxdriver.com>;
>>> >Jerin Jacob <jerinj@marvell.com>; Mattias Rönnblom
>>> ><mattias.ronnblom@ericsson.com>; dpdk-dev <dev@dpdk.org>;
>Eads, Gage
>>> ><gage.eads@intel.com>; Van Haaren, Harry
><harry.van.haaren@intel.com>
>>> >Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
>prerequisites
>>> >
>>> >On Tue, Jun 30, 2020 at 1:01 AM McDaniel, Timothy
>>> ><timothy.mcdaniel@intel.com> wrote:
>>> >>
>>> >> -----Original Message-----
>>> >> From: Jerin Jacob <jerinjacobk@gmail.com>
>>> >> Sent: Saturday, June 27, 2020 2:45 AM
>>> >> To: McDaniel, Timothy <timothy.mcdaniel@intel.com>; Ray
>Kinsella
>>> ><mdr@ashroe.eu>; Neil Horman <nhorman@tuxdriver.com>
>>> >> Cc: Jerin Jacob <jerinj@marvell.com>; Mattias Rönnblom
>>> ><mattias.ronnblom@ericsson.com>; dpdk-dev <dev@dpdk.org>;
>Eads, Gage
>>> ><gage.eads@intel.com>; Van Haaren, Harry
><harry.van.haaren@intel.com>
>>> >> Subject: Re: [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream
>prerequisites
>>> >>
>>> >> > +
>>> >> > +/** Event port configuration structure */
>>> >> > +struct rte_event_port_conf_v20 {
>>> >> > +       int32_t new_event_threshold;
>>> >> > +       /**< A backpressure threshold for new event enqueues on
>this port.
>>> >> > +        * Use for *closed system* event dev where event capacity
>is limited,
>>> >> > +        * and cannot exceed the capacity of the event dev.
>>> >> > +        * Configuring ports with different thresholds can make
>higher priority
>>> >> > +        * traffic less likely to  be backpressured.
>>> >> > +        * For example, a port used to inject NIC Rx packets into
>the event dev
>>> >> > +        * can have a lower threshold so as not to overwhelm the
>device,
>>> >> > +        * while ports used for worker pools can have a higher
>threshold.
>>> >> > +        * This value cannot exceed the *nb_events_limit*
>>> >> > +        * which was previously supplied to
>rte_event_dev_configure().
>>> >> > +        * This should be set to '-1' for *open system*.
>>> >> > +        */
>>> >> > +       uint16_t dequeue_depth;
>>> >> > +       /**< Configure number of bulk dequeues for this event
>port.
>>> >> > +        * This value cannot exceed the
>*nb_event_port_dequeue_depth*
>>> >> > +        * which previously supplied to rte_event_dev_configure().
>>> >> > +        * Ignored when device is not
>RTE_EVENT_DEV_CAP_BURST_MODE
>>> >capable.
>>> >> > +        */
>>> >> > +       uint16_t enqueue_depth;
>>> >> > +       /**< Configure number of bulk enqueues for this event
>port.
>>> >> > +        * This value cannot exceed the
>*nb_event_port_enqueue_depth*
>>> >> > +        * which previously supplied to rte_event_dev_configure().
>>> >> > +        * Ignored when device is not
>RTE_EVENT_DEV_CAP_BURST_MODE
>>> >capable.
>>> >> > +        */
>>> >> >         uint8_t disable_implicit_release;
>>> >> >         /**< Configure the port not to release outstanding events
>in
>>> >> >          * rte_event_dev_dequeue_burst(). If true, all events
>received through
>>> >> > @@ -733,6 +911,14 @@ struct rte_event_port_conf {
>>> >> >  rte_event_port_default_conf_get(uint8_t dev_id, uint8_t
>port_id,
>>> >> >                                 struct rte_event_port_conf *port_conf);
>>> >> >
>>> >> > +int
>>> >> > +rte_event_port_default_conf_get_v20(uint8_t dev_id, uint8_t
>port_id,
>>> >> > +                               struct rte_event_port_conf_v20 *port_conf);
>>> >> > +
>>> >> > +int
>>> >> > +rte_event_port_default_conf_get_v21(uint8_t dev_id, uint8_t
>port_id,
>>> >> > +                                     struct rte_event_port_conf *port_conf);
>>> >>
>>> >> Hi Timothy,
>>> >>
>>> >> + ABI Maintainers (Ray, Neil)
>>> >>
>>> >> # As per my understanding, the structures can not be versioned,
>only
>>> >> function can be versioned.
>>> >> i.e we can not make any change to " struct rte_event_port_conf"
>>> >>
>>> >> # We have a similar case with ethdev and it deferred to next
>release v20.11
>>> >> https://urldefense.proofpoint.com/v2/url?u=http-
>3A__patches.dpdk.org_patch_69113_&d=DwIGaQ&c=nKjWec2b6R0mO
>yPaz7xtfQ&r=1cjuAHrGh745jHNmj2fD85sUMIJ2IPIDsIJzo6FN6Z0&m=lL7
>dDlN7ICIpENvIB7_El27UclXA2tJLdOwbsirg1Dw&s=CNmSXDDn28U-
>OjEAaZgJI_A2fDmMKM6zb12sIE9L-Io&e=
>>> >>
>>> >> Regarding the API changes:
>>> >> # The slow path changes general looks good to me. I will review
>the
>>> >> next level in the coming days
>>> >> # The following fast path changes bothers to me. Could you share
>more
>>> >> details on below change?
>>> >>
>>> >> diff --git a/app/test-eventdev/test_order_atq.c
>>> >> b/app/test-eventdev/test_order_atq.c
>>> >> index 3366cfc..8246b96 100644
>>> >> --- a/app/test-eventdev/test_order_atq.c
>>> >> +++ b/app/test-eventdev/test_order_atq.c
>>> >> @@ -34,6 +34,8 @@
>>> >>                         continue;
>>> >>                 }
>>> >>
>>> >> +               ev.flow_id = ev.mbuf->udata64;
>>> >> +
>>> >> # Since RC1 is near, I am not sure how to accommodate the API
>changes
>>> >> now and sort out ABI stuffs.
>>> >> # Other concern is eventdev spec get bloated with versioning files
>>> >> just for ONE release as 20.11 will be OK to change the ABI.
>>> >> # While we discuss the API change, Please send deprecation
>notice for
>>> >> ABI change for 20.11,
>>> >> so that there is no ambiguity of this patch for the 20.11 release.
>>> >>
>>> >> Hello Jerin,
>>> >>
>>> >> Thank you for the review comments.
>>> >>
>>> >> With regard to your comments regarding the fast path flow_id
>change, the
>>Intel
>>> >DLB hardware
>>> >> is not capable of transferring the flow_id as part of the event
>itself. We
>>> >therefore require a mechanism
>>> >> to accomplish this. What we have done to work around this is to
>require the
>>> >application to embed the flow_id
>>> >> within the data payload. The new flag, #define
>>> >RTE_EVENT_DEV_CAP_CARRY_FLOW_ID (1ULL << 9), can be used
>>> >> by applications to determine if they need to embed the flow_id,
>or if its
>>> >automatically propagated and present in the
>>> >> received event.
>>> >>
>>> >> What we should have done is to wrap the assignment with a
>conditional.
>>> >>
>>> >> if (!(device_capability_flags &
>RTE_EVENT_DEV_CAP_CARRY_FLOW_ID))
>>> >>         ev.flow_id = ev.mbuf->udata64;
>>> >
>>> >Two problems with this approach,
>>> >1) we are assuming mbuf udata64 field is available for DLB driver
>>> >2) It won't work with another adapter, eventdev has no
>dependency with mbuf
>>> >
>>>
>>> This snippet is not intended to suggest that udata64 always be used
>to store the
>>flow ID, but as an example of how an application could do it. Some
>applications
>>won’t need to carry the flow ID through; others can select an unused
>field in the
>>event data (e.g. hash.rss or udata64 if using mbufs), or (worst-case)
>re-generate
>>the flow ID in pipeline stages that require it.
>>
>>OK.
>>>
>>> >Question:
>>> >1) In the case of DLB hardware, on dequeue(),  what HW returns? is
>it
>>> >only event pointer and not have any other metadata like
>schedule_type
>>> >etc.
>>> >
>>>
>>> The DLB device provides a 16B “queue entry” that consists of:
>>>
>>> *       8B event data
>>> *       Queue ID
>>> *       Priority
>>> *       Scheduling type
>>> *       19 bits of carried-through data
>>> *       Assorted error/debug/reserved bits that are set by the device
>(not carried-
>>through)
>>>
>>>  For the carried-through 19b, we use 12b for event_type and
>sub_event_type.
>>
>>I can only think of TWO options to help
>>1) Since event pointer always cache aligned, You could grab LSB
>>6bits(2^6 = 64B ) and 7 bits from (19b - 12b) carried through
>>structure
>>2) Have separate mempool driver using existing drivers, ie "event
>>pointer" + or - some offset have any amount of custom data.
>>
>
>We can't guarantee that the event will contain a pointer -- it's possible
>that 8B is inline data (i.e. struct rte_event's u64 field).
>
>It's really an application decision -- for example an app could allocate
>space in the 'mbuf private data' to store the flow ID, if the event device
>lacks that carry-flow-ID capability and the other mbuf fields can't be
>used for whatever reason.
>We modified the tests, sample apps to show how this might be done,
>not necessarily how it must be done.
>
>>
>>>
>>> >
>>> >>
>>> >> This would minimize/eliminate any performance impact due to
>the
>>processor's
>>> >branch prediction logic.
>>
>>I think, If we need to change common fastpath, better we need to
>make
>>it template to create code for compile-time to have absolute zero
>>overhead
>>and use runtime.
>>See app/test-eventdev/test_order_atq.c: function: worker_wrapper()
>>_create_ worker at compile time based on runtime capability.
>>
>
>Yes, that would be perfect.  Thanks for the example!

Just to  add instead of having if and else using a jumptbl would be much cleaner
Ex.
	const pipeline_atq_worker_t pipeline_atq_worker_single_stage[2][2][2] = {
		[0][0] = pipeline_atq_worker_single_stage_fwd,
		[0][1] = pipeline_atq_worker_single_stage_tx,
		[1][0] = pipeline_atq_worker_single_stage_burst_fwd,
		[1][1] = pipeline_atq_worker_single_stage_burst_tx,
	};

		return (pipeline_atq_worker_single_stage[burst][internal_port])(arg);

>
>>
>>
>>> >> The assignment then becomes in essence a NOOP for all event
>devices that
>>are
>>> >capable of carrying the flow_id as part of the event payload itself.
>>> >>
>>> >> Thanks,
>>> >> Tim
>>> >>
>>> >>
>>> >>
>>> >> Thanks,
>>> >> Tim

  reply	other threads:[~2020-06-30 20:40 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-27  4:37 [dpdk-dev] [PATCH 00/27] event/dlb Intel DLB PMD Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites Tim McDaniel
2020-06-27  7:44   ` Jerin Jacob
2020-06-29 19:30     ` McDaniel, Timothy
2020-06-30  4:21       ` Jerin Jacob
2020-06-30 15:37         ` McDaniel, Timothy
2020-06-30 15:57           ` Jerin Jacob
2020-06-30 19:26             ` McDaniel, Timothy
2020-06-30 20:40               ` Pavan Nikhilesh Bhagavatula [this message]
2020-06-30 21:07                 ` McDaniel, Timothy
2020-07-01  4:50               ` Jerin Jacob
2020-07-01 16:48                 ` McDaniel, Timothy
2020-06-30 11:22     ` Kinsella, Ray
2020-06-30 11:30       ` Jerin Jacob
2020-06-30 11:36         ` Kinsella, Ray
2020-06-30 12:14           ` Jerin Jacob
2020-07-02 15:21             ` Kinsella, Ray
2020-07-02 16:35               ` McDaniel, Timothy
2020-06-27  4:37 ` [dpdk-dev] [PATCH 02/27] eventdev: do not pass disable_implicit_release bit to trace macro Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 03/27] event/dlb: add shared code version 10.7.9 Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 04/27] event/dlb: add make and meson build infrastructure Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 05/27] event/dlb: add DLB documentation Tim McDaniel
2020-07-09  3:29   ` Eads, Gage
2020-06-27  4:37 ` [dpdk-dev] [PATCH 06/27] event/dlb: add dynamic logging Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 07/27] event/dlb: add private data structures and constants Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 08/27] event/dlb: add definitions shared with LKM or shared code Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 09/27] event/dlb: add inline functions used in multiple files Tim McDaniel
2020-07-07 12:02   ` Bruce Richardson
2020-07-07 14:33     ` McDaniel, Timothy
2020-06-27  4:37 ` [dpdk-dev] [PATCH 10/27] event/dlb: add PFPMD-specific interface layer to shared code Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 11/27] event/dlb: add flexible PMD to device interfaces Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 12/27] event/dlb: add the PMD's public interfaces Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 13/27] event/dlb: add xstats support Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 14/27] event/dlb: add PMD self-tests Tim McDaniel
2020-07-10 20:42   ` Eads, Gage
2020-07-29 18:56     ` McDaniel, Timothy
2020-06-27  4:37 ` [dpdk-dev] [PATCH 15/27] event/dlb: add probe Tim McDaniel
2020-07-09  3:45   ` Eads, Gage
2020-06-27  4:37 ` [dpdk-dev] [PATCH 16/27] event/dlb: add infos_get and configure Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 17/27] event/dlb: add queue_def_conf and port_def_conf Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 18/27] event/dlb: add queue setup Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 19/27] event/dlb: add port_setup Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 20/27] event/dlb: add port_link Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 21/27] event/dlb: add queue_release and port_release Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 22/27] event/dlb: add port_unlink and port_unlinks_in_progress Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 23/27] event/dlb: add eventdev_start Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 24/27] event/dlb: add timeout_ticks, dump, xstats, and selftest Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 25/27] event/dlb: add enqueue and its burst variants Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 26/27] event/dlb: add dequeue, dequeue_burst, and variants Tim McDaniel
2020-06-27  4:37 ` [dpdk-dev] [PATCH 27/27] event/dlb: add eventdev_stop and eventdev_close Tim McDaniel
     [not found] <1593232671-5690-0-git-send-email-timothy.mcdaniel@intel.com>
2020-07-30 19:49 ` [dpdk-dev] [PATCH 00/27] Add Intel DLM PMD to 20.11 McDaniel, Timothy
2020-07-30 19:49   ` [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites McDaniel, Timothy
2020-08-11 17:44     ` Jerin Jacob
  -- strict thread matches above, loose matches on Subject: below --
2020-06-12 21:24 [dpdk-dev] [PATCH 00/27] V1 event/dlb add Intel DLB PMD McDaniel, Timothy
2020-06-12 21:24 ` [dpdk-dev] [PATCH 01/27] eventdev: dlb upstream prerequisites McDaniel, Timothy
2020-06-13  3:59   ` Jerin Jacob
2020-06-13 10:43     ` Mattias Rönnblom
2020-06-18 15:51       ` McDaniel, Timothy
2020-06-18 15:44     ` McDaniel, Timothy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BN6PR18MB1140CE7C396DC68F562B5E87DE6F0@BN6PR18MB1140.namprd18.prod.outlook.com \
    --to=pbhagavatula@marvell.com \
    --cc=dev@dpdk.org \
    --cc=gage.eads@intel.com \
    --cc=harry.van.haaren@intel.com \
    --cc=jerinj@marvell.com \
    --cc=jerinjacobk@gmail.com \
    --cc=mattias.ronnblom@ericsson.com \
    --cc=mdr@ashroe.eu \
    --cc=nhorman@tuxdriver.com \
    --cc=timothy.mcdaniel@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).