Re: [dpdk-dev] [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues

["Van Haaren, Harry" <harry.van.haaren@intel.com>]

> From: Nipun Gupta [mailto:nipun.gupta@nxp.com]
> Sent: Monday, July 24, 2017 2:07 PM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>; Rao, Nikhil <nikhil.rao@intel.com>;
> jerin.jacob@caviumnetworks.com
> Cc: Eads, Gage <gage.eads@intel.com>; dev@dpdk.org; thomas@monjalon.net; Richardson, Bruce
> <bruce.richardson@intel.com>; Hemant Agrawal <hemant.agrawal@nxp.com>; Vangati, Narender
> <narender.vangati@intel.com>; Gujjar, Abhinandan S <abhinandan.gujjar@intel.com>
> Subject: RE: [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues
> 
> 
> 
> > -----Original Message-----
> > From: Van Haaren, Harry [mailto:harry.van.haaren@intel.com]
> > Sent: Monday, July 24, 2017 16:03
> > To: Nipun Gupta <nipun.gupta@nxp.com>; Rao, Nikhil <nikhil.rao@intel.com>;
> > jerin.jacob@caviumnetworks.com
> > Cc: Eads, Gage <gage.eads@intel.com>; dev@dpdk.org;
> > thomas@monjalon.net; Richardson, Bruce <bruce.richardson@intel.com>;
> > Hemant Agrawal <hemant.agrawal@nxp.com>; Vangati, Narender
> > <narender.vangati@intel.com>; Gujjar, Abhinandan S
> > <abhinandan.gujjar@intel.com>
> > Subject: RE: [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues
> >
> > > From: Nipun Gupta [mailto:nipun.gupta@nxp.com]
> > > Sent: Monday, July 24, 2017 11:11 AM
> > > To: Rao, Nikhil <nikhil.rao@intel.com>; jerin.jacob@caviumnetworks.com
> > > Cc: Eads, Gage <gage.eads@intel.com>; dev@dpdk.org;
> > thomas@monjalon.net; Richardson, Bruce
> > > <bruce.richardson@intel.com>; Van Haaren, Harry
> > <harry.van.haaren@intel.com>; Hemant
> > > Agrawal <hemant.agrawal@nxp.com>; Vangati, Narender
> > <narender.vangati@intel.com>; Gujjar,
> > > Abhinandan S <abhinandan.gujjar@intel.com>
> > > Subject: RE: [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues
> > >
> > > Hi Nikhil/Edas,
> >
> > Hi All,
> >
> > I'll reply on the service-cores topic, and leave the config to those more involved
> > and familiar with them than I :)
> >
> > > > -----Original Message-----
> > > > From: Nikhil Rao [mailto:nikhil.rao@intel.com]
> > > > Sent: Friday, July 07, 2017 3:23
> > > > To: jerin.jacob@caviumnetworks.com
> > > > Cc: gage.eads@intel.com; dev@dpdk.org; thomas@monjalon.net;
> > > > bruce.richardson@intel.com; harry.van.haaren@intel.com; Hemant Agrawal
> > > > <hemant.agrawal@nxp.com>; Nipun Gupta <nipun.gupta@nxp.com>;
> > > > narender.vangati@intel.com; Nikhil Rao <nikhil.rao@intel.com>;
> > Abhinandan
> > > > Gujjar <abhinandan.gujjar@intel.com>
> > > > Subject: [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues
> > > >
> > > > Eventdev-based networking applications require a component to dequeue
> > > > packets from NIC Rx queues and inject them into eventdev queues[1]. While
> > > > some platforms (e.g. Cavium Octeontx) do this operation in hardware, other
> > > > platforms use software.
> > > >
> > > > This patchset introduces an ethernet Rx event adapter that dequeues packets
> > > > from ethernet devices and enqueues them to event devices. It is based on
> > > > a previous RFC[2].
> > > >
> > > > The adapter is designed to work with the EAL service core[3]. If
> > > > an application determines that the adapter is required, it can register and
> > > > launch it on a service core. Alternatively, this adapter can serve as a
> > > > template for applications to design customer ethernet Rx event adapters
> > > > better suited to their needs.
> > > >
> > > > The adapter can service multiple ethernet devices and queues. Each queue is
> > > > configured with a servicing weight to control the relative frequency with
> > > > which the adapter polls the queue, and the event fields to use when
> > > > constructing packet events. The adapter has two modes for programming an
> > > > event's flow ID: use a static per-queue user-specified value or use the RSS
> > > > hash.
> > > >
> > > > A detailed description of the adapter is contained in the header's
> > > > comments.
> > > >
> > > > [1] http://dpdk.org/ml/archives/dev/2017-May/065341.html
> > > > [2] http://dpdk.org/ml/archives/dev/2017-May/065539.html
> > > > [3] http://dpdk.org/ml/archives/dev/2017-July/069782.html
> > > >
> > > > Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
> > > > Signed-off-by: Gage Eads <gage.eads@intel.com>
> > > > Signed-off-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
> > > > ---
> > > >
> > > > v2:
> > > > Thanks Jerin for review - below is a list of changes you
> > > > suggested.
> > > >
> > > > - all public symbols are started with rte_event_.
> > > > - Add Doxygen reference with @see.
> > > > - Mention setting of ev.event_type.
> > > > - Mention adapter to service function mapping.
> > > > - Remove rte_eth_rx_event_adapter_dev_add/del().
> > > > - Change rx_queuee_id to int32_t and use -1 to denote all Rx queues.
> > > > - Add rte_eth_event_rx_queue_del().
> > > >
> > > > Other changes
> > > > - Remove adapter's run function (rte_event_eth_rx_adapter_run()) from
> > > >   the public interface. The adapter internally uses it to create a
> > > >   service.
> > > > - Add a blocked cycle count to stats. Further description is contained
> > > >   in the header.
> > > > - Minor struct renames rte_event_eth_rx_adapter_config -> .._conf
> > > > ---
> > > >  lib/librte_eventdev/rte_event_eth_rx_adapter.h | 301 +++++++++
> > > >  lib/librte_eventdev/rte_event_eth_rx_adapter.c | 825
> > > > +++++++++++++++++++++++++
> > > >  lib/librte_eventdev/rte_eventdev_version.map   |  13 +
> > > >  lib/Makefile                                   |   2 +-
> > > >  lib/librte_eventdev/Makefile                   |   2 +
> > > >  5 files changed, 1142 insertions(+), 1 deletion(-)
> > > >  create mode 100644 lib/librte_eventdev/rte_event_eth_rx_adapter.h
> > > >  create mode 100644 lib/librte_eventdev/rte_event_eth_rx_adapter.c
> > > >
> > > > diff --git a/lib/librte_eventdev/rte_event_eth_rx_adapter.h
> > > > b/lib/librte_eventdev/rte_event_eth_rx_adapter.h
> > > > new file mode 100644
> > > > index 000000000..5ccd0bd24
> > > > --- /dev/null
> > > > +++ b/lib/librte_eventdev/rte_event_eth_rx_adapter.h
> > > > @@ -0,0 +1,301 @@
> > > > +/*
> > > > + *   Copyright(c) 2017 Intel Corporation. All rights reserved.
> > > > + *   All rights reserved.
> > > > + *
> > > > + *   Redistribution and use in source and binary forms, with or without
> > > > + *   modification, are permitted provided that the following conditions
> > > > + *   are met:
> > > > + *
> > > > + *     * Redistributions of source code must retain the above copyright
> > > > + *       notice, this list of conditions and the following disclaimer.
> > > > + *     * Redistributions in binary form must reproduce the above copyright
> > > > + *       notice, this list of conditions and the following disclaimer in
> > > > + *       the documentation and/or other materials provided with the
> > > > + *       distribution.
> > > > + *     * Neither the name of Intel Corporation nor the names of its
> > > > + *       contributors may be used to endorse or promote products derived
> > > > + *       from this software without specific prior written permission.
> > > > + *
> > > > + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> > > > CONTRIBUTORS
> > > > + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> > NOT
> > > > + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> > > > FITNESS FOR
> > > > + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> > > > COPYRIGHT
> > > > + *   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> > > > INCIDENTAL,
> > > > + *   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
> > BUT
> > > > NOT
> > > > + *   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
> > LOSS
> > > > OF USE,
> > > > + *   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
> > AND
> > > > ON ANY
> > > > + *   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
> > > > TORT
> > > > + *   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
> > OF
> > > > THE USE
> > > > + *   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> > > > DAMAGE.
> > > > + */
> > > > +
> > > > +#ifndef _RTE_EVENT_ETH_RX_ADAPTER_
> > > > +#define _RTE_EVENT_ETH_RX_ADAPTER_
> > > > +
> > > > +/**
> > > > + * @file
> > > > + *
> > > > + * RTE Event Ethernet Rx Adapter
> > > > + *
> > > > + * An eventdev-based packet processing application enqueues/dequeues
> > mbufs
> > > > + * to/from the event device. The ethernet Rx event adapter's role is to
> > transfer
> > > > + * mbufs from the ethernet receive queues managed by DPDK to an event
> > > > device.
> > > > + * The application uses the adapter APIs to configure the packet flow
> > between
> > > > + * the ethernet devices and event devices. The adapter is designed to work
> > with
> > > > + * the EAL service cores. The adapter's work can be parallelized by dividing
> > the
> > > > + * NIC Rx queues among multiple adapter services that run in parallel.
> > >
> > > First of all, apologies for commenting late on this. I am going through this
> > patchset and
> > > have some concerns over this.
> > > It is mentioned that "The adapter is designed to work with * the EAL service
> > cores"
> > > Does this mean that the adapter cannot function without service core?
> > > In the case where Ethdev HW is capable of injecting the packet to compatible
> > HW eventdev
> > > driver, there is no service
> > > required. It seems this patchset does not take care of this use-case or maybe I
> > am missing
> > > something here?
> >
> >
> > The "service cores" is a method of abstracting lcore requirements. In short, with
> > a SW PMD, a service core is required to perform the event scheduling. With the
> > HW use case as you describe, no core is required. So in the HW case, no service
> > cores are allocated in DPDK, and as a result, all your cores are available to the
> > application.
> >
> > If I understand correctly, the generic "Event Ethernet RX Adapter" will register a
> > service which needs to be run (by a service core) in order to perform SW polling
> > of the NIC, and enqueue to the eventdev.
> >
> > In order to support the hardware use-case, no service is registered and no
> > service lcores required. The HW is allowed to perform the Ethdev -> Eventdev
> > enqueue.
> >
> > Hope that makes sense! -Harry
> 
> With function pointer based approach the adapter calls will be propagated to eventdev PMD
> so
> here won't this way the event driver will be responsible for registering the service? I
> don't think the
> generic 'Event ethernet Rx adapter' would register a service with this model.

Good point - I should dig into code before replying. Yes, with the function-pointer / PMD model allows the PMD to decide if it requires a service.

Thanks for the correction!


> Thanks,
> Nipun