From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6A8043A1E; Wed, 31 Jan 2024 14:45:36 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 8E4E2402D0; Wed, 31 Jan 2024 14:45:36 +0100 (CET) Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) by mails.dpdk.org (Postfix) with ESMTP id D885540275 for ; Wed, 31 Jan 2024 14:45:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706708735; x=1738244735; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=w6vXfYV5S9496sbqvC9c0yu4TnrN8hXcGS0Gk5V7C+c=; b=LEF0hV/gLKTxeImk+UQQmvJSK7BZAbXZNbKjlybiw70DcwDFj/CZe7nP GrUKDolmWhDGghVDrW+CHOlHcAtES5PieZqPFPlqgDN0Q0pwkCCacx3GL L5+NvNokVRY6yCIl8Rzc/QhsEYypyTWOPmoGsRvBFPzekR2sLcqibZ4ui HFHWoTy56Hy2OUTHrHMsLxAf+hbRJR8U1sJpsnJ4nju59373nlNdCXNcM and/2oDJ7WBIDZ104ma+a25cdhFlDYuN6fVfkIzleBgUSJc4GQQPMuf8V 3YM5B9JIzZUzNk+Dc/+WEpz+aWf6oA3muvrZUZK8PrHO9JfqsZDQkasId w==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="3441632" X-IronPort-AV: E=Sophos;i="6.05,231,1701158400"; d="scan'208";a="3441632" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 05:45:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="931846508" X-IronPort-AV: E=Sophos;i="6.05,231,1701158400"; d="scan'208";a="931846508" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmsmga001.fm.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 31 Jan 2024 05:45:33 -0800 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 31 Jan 2024 05:45:32 -0800 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Wed, 31 Jan 2024 05:45:32 -0800 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Wed, 31 Jan 2024 05:45:32 -0800 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (104.47.73.40) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 31 Jan 2024 05:45:32 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=m34+siaRfWx3lXcvh6YmQyF46RymebmBzEV8nxrJa/6vWg6fwrPqBDDj9XI5ahmXU9XSdDIg6OmNLec0kGtX5hQjkiMJ70k4QoWVeWp1Ni8MVYVYb0WvichSFa/Q7T8pgl5s4ftZ2ZiQVbWfCJPPLlAyRPdaSHMsYvgrRQ5j4qKBu5VTAZLv/cBuXlwZYVqqDmMlRl9RbR9uoRIKhTDTGJSMmAtVgt36lbqRPoZTpGpJfnFOQyqGzGWHxGsXFlRY5j7dveQ6k4DVltzw2vWq293aOi5x5MYB55cf1+HQwyuTtgpZZFre816FQMQPVxOAhZxyl0maJJeAlKj0+kgnJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hMmVloxy+0lEGbhqYEIBIbiX6l0n1XA3Aq2eSNqsn/c=; b=g8qWtHJvwzO8g8naqUwuuVixt1VrvzM5W/PFUyyaYf55SQhSawhAsGQ7Vd2pU+/03ExDggjsfNkX6FjNxs9xyGwySj/e9XGDbclR2gPvc/WOzTfO4zwJsxlHZIbvprSKMHb7spnUGV0N1zYb4//hub0o/4aa+xcQ+gZNsLcMW6911PX7CtMeIl/5R/nO0jD95oX7r5OOrd72kfRsumNKMLTPzDQqN62ofH/HHfeK2lOP6KqRUDpq6zJnwlFtwCTtw9/NZjaZcLOxCR+qkFHA4dRTRfrgaWl7c7chTGLtuF0/Z1RTly9EAVU7nqKzVDiRGxUxU7PmkyAbzNvQpZIX2w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; Received: from DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) by IA1PR11MB7890.namprd11.prod.outlook.com (2603:10b6:208:3ff::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7249.24; Wed, 31 Jan 2024 13:45:29 +0000 Received: from DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::df88:b743:97f8:516c]) by DS0PR11MB7309.namprd11.prod.outlook.com ([fe80::df88:b743:97f8:516c%5]) with mapi id 15.20.7249.017; Wed, 31 Jan 2024 13:45:29 +0000 Date: Wed, 31 Jan 2024 13:45:23 +0000 From: Bruce Richardson To: Mattias =?iso-8859-1?Q?R=F6nnblom?= CC: , , , , , , , Subject: Re: [PATCH v2 01/11] eventdev: improve doxygen introduction text Message-ID: References: <20240118134557.73172-1-bruce.richardson@intel.com> <20240119174346.108905-1-bruce.richardson@intel.com> <20240119174346.108905-2-bruce.richardson@intel.com> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: DB8PR06CA0033.eurprd06.prod.outlook.com (2603:10a6:10:100::46) To DS0PR11MB7309.namprd11.prod.outlook.com (2603:10b6:8:13e::17) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR11MB7309:EE_|IA1PR11MB7890:EE_ X-MS-Office365-Filtering-Correlation-Id: 8390eb08-adb2-4846-1c7f-08dc2262e344 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Z1N/2+G9ZCHMX2LosmIIIGbT0utEgKp1Wc07yQ4CF5HqgepZvDVqvq4iAAPipHhDmszyhEBRempYqu6DexJIw/dZxsuyXX3pngJV/Bom8QVeId3ASwTa5EW7jGOkZRki+ahXD57IvcYQqVl92vYyHcSkLqS/QvQCBX8UfTfdSiseWtboaBfSOk33KY32korIIOROiUa9TYS74599+fW8VsyfMHupz2jQ8vjELKwtoseR0U6Q5k7df5LEkqzA4Wpfd2BHOykANUxhZBYdBfShgTGjL4G2juQ8PuYeERmo2qNMQvelsQCVqwgMPtFW2Q9puHfKgJsonWC9HVISuFiAnB3a8hZ2eGO/3eWP0tkDsC7Ckdv2CQv9lZtON0JUDeOSEgq2DUdciEiz8cJXduKy6qdK1dRpTpZvIyqcFFnCgUfs8YUo1gmaa7px7swpgvq7U8xMnxIa19Aug7SqDQGmydQ33Dg1AmJUlECz8L2QSeIalAQAy+GBEEcbgxscAcbUWWLTtFY2hk+ElgFk+awnfB8NOoBNW4PoO6i6XuXVsosfUjNTDBSx8OqL3SiXAeCg X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS0PR11MB7309.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(136003)(346002)(396003)(376002)(39860400002)(366004)(230922051799003)(1800799012)(451199024)(186009)(64100799003)(66556008)(296002)(30864003)(2906002)(44832011)(5660300002)(82960400001)(6486002)(86362001)(38100700002)(478600001)(41300700001)(53546011)(6506007)(83380400001)(107886003)(66574015)(6512007)(26005)(6666004)(8936002)(316002)(4326008)(8676002)(6916009)(66946007)(66476007)(66899024); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?GGmWQYBbeySLboKXdrMXaSBfV/bFv/Cq+POnItJjDIW/agXTc/G3AJJClQ?= =?iso-8859-1?Q?aPQU47mUTTDqu1lsN4b5EeH4ohvZuoTKwbMp5YW5FP2R4sCN9oiH8okYIt?= =?iso-8859-1?Q?rxlfdhfiL7EE8NT9nkbMnv3NSBo8C+kbpw1lXMtgrR6skxp981lNPJXKa6?= =?iso-8859-1?Q?DoMW5mHQua8HSWgvyUWBCuXcBViU69dgCah4aK/PK0F53Fxn1cOnlOFdy/?= =?iso-8859-1?Q?er2u46EU8CfYm2p+S1HAJKJlYiDNMHuqwHO0B5Lw19V8gkQqRHtIFtb0BD?= =?iso-8859-1?Q?3yoI0xrUmIeOO7ck6zyfp9GH0Cv7Bu2Uj1RkNbqUH6h0BZZJWc9c09j0yv?= =?iso-8859-1?Q?FuBLsespZAXTDPyYSzMG6yHpmjwjQ9o4uvckmg83HYfq7HG61E69kfhMw1?= =?iso-8859-1?Q?oio+NMFlXP6MDk9badOo9aG2SZhoLtciheVZpz3kzERFdrfA0rMHdfXHyO?= =?iso-8859-1?Q?BUW+t5Qj2/xi2CSI4aHiT55ASORp2t4O4KtTIU+gQ+nA37SRRMjtPgHSjk?= =?iso-8859-1?Q?XIjkjx+xTT/hlXuhY0aHFPA2LD5vrYufvOW4BhoBtJZl659o5y84IQQbvL?= =?iso-8859-1?Q?IdcSFEPy9dXx96pyKaJm/Za4lpWGd26s3C/+NGgmN+OFXJj+nitroWM07I?= =?iso-8859-1?Q?QpuqOVuE/q9S51afwI4t66bDyxiqtCfGNFIwxDmEzV9cNwg5D4hXvgQKZ5?= =?iso-8859-1?Q?hGsiDcMdtZabh6r5t64IoxuhP7KN8VIIV3JgYmUBVKRkCczGZmRO0rxJ+Y?= =?iso-8859-1?Q?UUg6KSP/DibR9QdoXvNaY8PWEusXBzYs0xygHGbZLb0agHYXrTgsN/y8Z3?= =?iso-8859-1?Q?wGUqac5LMLX/AvcKCWf5P1NVcKiVKo0oIzn0qJirgk8ZzY5a0a2Z5a+K8o?= =?iso-8859-1?Q?ykeO1RzAWbFmZ0bOfNId25BW3dC35jEHsycSdDWlHzQ8EIXxmlRnkapmfR?= =?iso-8859-1?Q?wQ5AcPHAjUkf9B1GmzV1ArIhq+Y0RYsy13f6YzK5K7b8K37/yAEvP4yECW?= =?iso-8859-1?Q?l6fL5ItgoexhdI2itnFKzETm6zcIFkdP+1dnXBBjAhqrW/2uppg9tmrm0w?= =?iso-8859-1?Q?qvBePGLb/Rs4R6vv4306ZUf7Amag1CXapMJHD9yKrKJBM/OBKQ6l7iHjb0?= =?iso-8859-1?Q?HMUnc3zvg9TcajLB6TBkUEI7G3HJM8K0vm/ZfFD4GpPMfjqPVoxbU88YIP?= =?iso-8859-1?Q?9pMsJQ4kzAo49Rd/f2UlNjjevnIRHtzoNGtnd8VphO44g+eNAqKQlhVjd8?= =?iso-8859-1?Q?OcFm+lB+3aWKTOs+g2W6pEerDgFMmTFXXFVbVT5oqkl+CG292jZs0ahoVu?= =?iso-8859-1?Q?5XCV6eNU1dtmN5ec9TbEJRdXOm9JF8+oO8kkNqjcet7wlSGu8+Jj7POmyd?= =?iso-8859-1?Q?axu/dHyMB3TLxkTP5W4pCmeoUR+7UfodAJIGVwlRjm7ukQgAuMzehbAgTX?= =?iso-8859-1?Q?fprHCN3bP4qO+FJxv9LGAmlnNZxiDy7CsR9Ch4wTEI5hePfR6JgavZE2Vd?= =?iso-8859-1?Q?Us2k8OxNHH37OIsSl1MidA/FW2q5RC9CeOhJfhEQ2I5gKx7gUL5VASok7U?= =?iso-8859-1?Q?gKf834OzcVmY7YRuEIr33PqW3yS8bBICBU9foPV97hXeq15Nl0oKCeCC+M?= =?iso-8859-1?Q?NBKA62HdpUk2IqpmMB105hMwapHvrO8hSyLoFpyFTvRpM+mSIhtD1mKA?= =?iso-8859-1?Q?=3D=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 8390eb08-adb2-4846-1c7f-08dc2262e344 X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7309.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Jan 2024 13:45:29.6770 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GutuHQ4KFJuQjTHbZhB/ysEzv47q5U1dLvpV0358zfA3Yo4pAdTTxG6vE3qv3onWC9TVEVIK9izgos+31+2TdnDyqpXhSBrmixohAAuReXI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB7890 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Tue, Jan 23, 2024 at 09:57:58AM +0100, Mattias Rönnblom wrote: > On 2024-01-19 18:43, Bruce Richardson wrote: > > Make some textual improvements to the introduction to eventdev and event > > devices in the eventdev header file. This text appears in the doxygen > > output for the header file, and introduces the key concepts, for > > example: events, event devices, queues, ports and scheduling. > > > > Great stuff, Bruce. > > > This patch makes the following improvements: > > * small textual fixups, e.g. correcting use of singular/plural > > * rewrites of some sentences to improve clarity > > * using doxygen markdown to split the whole large block up into > > sections, thereby making it easier to read. > > > > No large-scale changes are made, and blocks are not reordered > > > > Signed-off-by: Bruce Richardson > > --- > > lib/eventdev/rte_eventdev.h | 112 +++++++++++++++++++++--------------- > > 1 file changed, 66 insertions(+), 46 deletions(-) > > > > diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h > > index ec9b02455d..a36c89c7a4 100644 > > --- a/lib/eventdev/rte_eventdev.h > > +++ b/lib/eventdev/rte_eventdev.h > > @@ -12,12 +12,13 @@ > > * @file > > * > > * RTE Event Device API > > + * ==================== > > * > > * In a polling model, lcores poll ethdev ports and associated rx queues > > "In a polling model, lcores pick up packets from Ethdev ports and associated > RX queues, runs the processing to completion, and enqueues the completed > packets to a TX queue. NIC-level receive-side scaling (RSS) may be used to > balance the load across multiple CPU cores." > > I thought it might be worth to be a little more verbose on what is the > reference model Eventdev is compared to. Maybe you can add "traditional" or > "archetypal", or "simple" as a prefix to the "polling model". (I think I > would call this a "simple run-to-completion model" rather than "polling > model".) > > "By contrast, in Eventdev, ingressing* packets are fed into an event device, > which schedules packets across available lcores, in accordance to its > configuration. This event-driven programming model offers applications > automatic multicore scaling, dynamic load balancing, pipelining, packet > order maintenance, synchronization, and quality of service." > > * Is this a word? > Ack, taking these suggestions with minor tweaks. Changed "ingressing" to "incoming", which should be clear enough and is definitely a word. > > - * directly to look for packet. In an event driven model, by contrast, lcores > > - * call the scheduler that selects packets for them based on programmer > > - * specified criteria. Eventdev library adds support for event driven > > - * programming model, which offer applications automatic multicore scaling, > > + * directly to look for packets. In an event driven model, in contrast, lcores > > + * call a scheduler that selects packets for them based on programmer > > + * specified criteria. The eventdev library adds support for the event driven > > + * programming model, which offers applications automatic multicore scaling, > > * dynamic load balancing, pipelining, packet ingress order maintenance and > > * synchronization services to simplify application packet processing. > > * > > @@ -25,12 +26,15 @@ > > * > > * - The application-oriented Event API that includes functions to setup > > * an event device (configure it, setup its queues, ports and start it), to > > - * establish the link between queues to port and to receive events, and so on. > > + * establish the links between queues and ports to receive events, and so on. > > * > > * - The driver-oriented Event API that exports a function allowing > > - * an event poll Mode Driver (PMD) to simultaneously register itself as > > + * an event poll Mode Driver (PMD) to register itself as > > * an event device driver. > > * > > + * Application-oriented Event API > > + * ------------------------------ > > + * > > * Event device components: > > * > > * +-----------------+ > > @@ -75,27 +79,33 @@ > > * | | > > * +-----------------------------------------------------------+ > > * > > - * Event device: A hardware or software-based event scheduler. > > + * **Event device**: A hardware or software-based event scheduler. > > * > > - * Event: A unit of scheduling that encapsulates a packet or other datatype > > - * like SW generated event from the CPU, Crypto work completion notification, > > - * Timer expiry event notification etc as well as metadata. > > - * The metadata includes flow ID, scheduling type, event priority, event_type, > > + * **Event**: A unit of scheduling that encapsulates a packet or other datatype, > > "Event: Represents an item of work and is the smallest unit of scheduling. > An event carries metadata, such as queue ID, scheduling type, and event > priority, and data such as one or more packets or other kinds of buffers. > Examples of events are a software-generated item of work originating from a > lcore carrying a packet to be processed, a crypto work completion > notification and a timer expiry notification." > > I've found "work scheduler" as helpful term describing what role an event > device serve in the system, and thus an event represent an item of work. > "Event" and "Event device" are also good names, but lead some people to > think libevent or event loop, which is not exactly right. > Ack. > > + * such as: SW generated event from the CPU, crypto work completion notification, > > + * timer expiry event notification etc., as well as metadata about the packet or data. > > + * The metadata includes a flow ID (if any), scheduling type, event priority, event_type, > > * sub_event_type etc. > > * > > - * Event queue: A queue containing events that are scheduled by the event dev. > > + * **Event queue**: A queue containing events that are scheduled by the event device. > > * An event queue contains events of different flows associated with scheduling > > * types, such as atomic, ordered, or parallel. > > + * Each event given to an eventdev must have a valid event queue id field in the metadata, > "eventdev" -> "event device" > > > + * to specify on which event queue in the device the event must be placed, > > + * for later scheduling to a core. > > Events aren't nessarily scheduled to cores, so remove the last part. > > > * > > - * Event port: An application's interface into the event dev for enqueue and > > + * **Event port**: An application's interface into the event dev for enqueue and > > * dequeue operations. Each event port can be linked with one or more > > * event queues for dequeue operations. > > - * > > - * By default, all the functions of the Event Device API exported by a PMD > > - * are lock-free functions which assume to not be invoked in parallel on > > - * different logical cores to work on the same target object. For instance, > > - * the dequeue function of a PMD cannot be invoked in parallel on two logical > > - * cores to operates on same event port. Of course, this function > > + * Each port should be associated with a single core (enqueue and dequeue is not thread-safe). > > Should, or must? > > Either it's a MT safety issue, and any lcore can access the port with the > proper serialization, or it's something where the lcore id used to store > state between invocations, or some other mechanism that prevents a port from > being used by multiple threads (lcore or not). > Rewording this to start with the fact that enqueue and dequeue functions are not "thread-safe", and then stating that the expected configuration is that each port is assigned to an lcore, otherwise sync mechanisms are needed. > > + * To schedule events to a core, the event device will schedule them to the event port(s) > > + * being polled by that core. > > "core" -> "lcore" ? > > > + * > > + * *NOTE*: By default, all the functions of the Event Device API exported by a PMD > > + * are lock-free functions, which must not be invoked on the same object in parallel on > > + * different logical cores. > > This is a one-sentence contradiction. The term "lock free" implies a data > structure which is MT safe, achieving this goal without the use of locks. A > lock-free object thus *may* be called from different threads, including > different lcore threads. > Changed lock-free to non-thread-safe. > Ports are not MT safe, and thus one port should not be acted upon by more > than one thread (either in parallel, or throughout the lifetime of the event > device/port; see above). > > The event device is MT safe, provided the different parallel callers use > different ports. > > A more subtle question and one with a less obvious answer is if the caller > of also *must* be an EAL thread, or if a registered non-EAL thread or even > an unregistered non-EAL thread may call the "fast path" functions (enqueue, > dequeue etc). > > For EAL threads, the event device implementation may safely use > non-preemption safe constructs (like the default ring variant and spin > locks). > > If the caller is a registered non-EAL thread or an EAL thread, the lcore id > may be used to index various data structures. > > If "lcore id"-less threads may call the fast path APIs, what are the MT > safety guarantees in that case? Like rte_random.h, or something else. > I don't know the answer to this. I believe right now that most/all eventdev functions are callable on non-EAL threads, but I'm not sure we want to guarantee that - e.g. some drivers may require registered threads. I think we need to resolve and document this, but I'm not going to do so in this patch(set). > > + * For instance, the dequeue function of a PMD cannot be invoked in parallel on two logical > > + * cores to operate on same event port. Of course, this function > > * can be invoked in parallel by different logical cores on different ports. > > * It is the responsibility of the upper level application to enforce this rule. > > * > > @@ -107,22 +117,19 @@ > > * > > * Event devices are dynamically registered during the PCI/SoC device probing > > * phase performed at EAL initialization time. > > - * When an Event device is being probed, a *rte_event_dev* structure and > > - * a new device identifier are allocated for that device. Then, the > > - * event_dev_init() function supplied by the Event driver matching the probed > > - * device is invoked to properly initialize the device. > > + * When an Event device is being probed, an *rte_event_dev* structure is allocated > > + * for it and the event_dev_init() function supplied by the Event driver > > + * is invoked to properly initialize the device. > > * > > - * The role of the device init function consists of resetting the hardware or > > - * software event driver implementations. > > + * The role of the device init function is to reset the device hardware or > > + * to initialize the software event driver implementation. > > * > > - * If the device init operation is successful, the correspondence between > > - * the device identifier assigned to the new device and its associated > > - * *rte_event_dev* structure is effectively registered. > > - * Otherwise, both the *rte_event_dev* structure and the device identifier are > > - * freed. > > + * If the device init operation is successful, the device is assigned a device > > + * id (dev_id) for application use. > > + * Otherwise, the *rte_event_dev* structure is freed. > > * > > * The functions exported by the application Event API to setup a device > > - * designated by its device identifier must be invoked in the following order: > > + * must be invoked in the following order: > > * - rte_event_dev_configure() > > * - rte_event_queue_setup() > > * - rte_event_port_setup() > > @@ -130,10 +137,15 @@ > > * - rte_event_dev_start() > > * > > * Then, the application can invoke, in any order, the functions > > - * exported by the Event API to schedule events, dequeue events, enqueue events, > > - * change event queue(s) to event port [un]link establishment and so on. > > - * > > - * Application may use rte_event_[queue/port]_default_conf_get() to get the > > + * exported by the Event API to dequeue events, enqueue events, > > + * and link and unlink event queue(s) to event ports. > > + * > > + * Before configuring a device, an application should call rte_event_dev_info_get() > > + * to determine the capabilities of the event device, and any queue or port > > + * limits of that device. The parameters set in the various device configuration > > + * structures may need to be adjusted based on the max values provided in the > > + * device information structure returned from the info_get API. > > + * An application may use rte_event_[queue/port]_default_conf_get() to get the > > * default configuration to set up an event queue or event port by > > * overriding few default values. > > * > > @@ -145,7 +157,11 @@ > > * when the device is stopped. > > * > > * Finally, an application can close an Event device by invoking the > > - * rte_event_dev_close() function. > > + * rte_event_dev_close() function. Once closed, a device cannot be > > + * reconfigured or restarted. > > + * > > + * Driver-Oriented Event API > > + * ------------------------- > > * > > * Each function of the application Event API invokes a specific function > > * of the PMD that controls the target device designated by its device > > @@ -164,10 +180,13 @@ > > * supplied in the *event_dev_ops* structure of the *rte_event_dev* structure. > > * > > * For performance reasons, the address of the fast-path functions of the > > - * Event driver is not contained in the *event_dev_ops* structure. > > + * Event driver are not contained in the *event_dev_ops* structure. > > It's one address, so it should remain "is"? I think it should be "addresses of the functions", so adjusting that and keeping it as "are". Next sentence already uses "they" in the plural too, so then everything aligns nicely. > > > * Instead, they are directly stored at the beginning of the *rte_event_dev* > > * structure to avoid an extra indirect memory access during their invocation. > > * > > + * Event Enqueue, Dequeue and Scheduling > > + * ------------------------------------- > > + * > > * RTE event device drivers do not use interrupts for enqueue or dequeue > > * operation. Instead, Event drivers export Poll-Mode enqueue and dequeue > > * functions to applications. > > @@ -179,21 +198,22 @@ > > * crypto work completion notification etc > > * > > * The *dequeue* operation gets one or more events from the event ports. > > - * The application process the events and send to downstream event queue through > > - * rte_event_enqueue_burst() if it is an intermediate stage of event processing, > > - * on the final stage, the application may use Tx adapter API for maintaining > > - * the ingress order and then send the packet/event on the wire. > > + * The application processes the events and sends them to a downstream event queue through > > + * rte_event_enqueue_burst(), if it is an intermediate stage of event processing. > > + * On the final stage of processing, the application may use the Tx adapter API for maintaining > > + * the event ingress order while sending the packet/event on the wire via NIC Tx. > > * > > * The point at which events are scheduled to ports depends on the device. > > * For hardware devices, scheduling occurs asynchronously without any software > > * intervention. Software schedulers can either be distributed > > * (each worker thread schedules events to its own port) or centralized > > * (a dedicated thread schedules to all ports). Distributed software schedulers > > - * perform the scheduling in rte_event_dequeue_burst(), whereas centralized > > - * scheduler logic need a dedicated service core for scheduling. > > - * The RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag is not set > > - * indicates the device is centralized and thus needs a dedicated scheduling > > - * thread that repeatedly calls software specific scheduling function. > > + * perform the scheduling inside the enqueue or dequeue functions, whereas centralized > > + * software schedulers need a dedicated service core for scheduling. > > + * The absence of the RTE_EVENT_DEV_CAP_DISTRIBUTED_SCHED capability flag > > + * indicates that the device is centralized and thus needs a dedicated scheduling > > + * thread, generally a service core, > > + * that repeatedly calls the software specific scheduling function. > > In the SW case, what you have is a service that needs to be mapped to a > service lcore. > > "generally a RTE service that should be mapped to one or more service > lcores" > Ack, will use that rewording.