From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0059.outbound.protection.outlook.com [104.47.2.59]) by dpdk.org (Postfix) with ESMTP id 129437CF1 for ; Mon, 24 Jul 2017 12:10:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=FpRVJJc9key0hVc8cTEsL6YIp7MPJQG7lmj+eP2PJR8=; b=hEAdTDJR356c/pHLerPh0n+QZxTGfFBDTjZ2RXL3X5n3fb6rAl+KryT//HBBrWQPKXe8Rus6qH7MsdbZPP8T3Msui/RoT3Mv2H2dpHIS+cRQ3CgSard9IRtYbrkd+tRHNjPKinbG2lJbGMgUY9iCEU/xxLJKdkDIg33dw1z5orU= Received: from HE1PR0401MB2425.eurprd04.prod.outlook.com (10.168.33.22) by AM2PR04MB0756.eurprd04.prod.outlook.com (10.160.56.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1282.10; Mon, 24 Jul 2017 10:10:52 +0000 Received: from HE1PR0401MB2425.eurprd04.prod.outlook.com ([fe80::a06b:de0:6c53:1fc1]) by HE1PR0401MB2425.eurprd04.prod.outlook.com ([fe80::a06b:de0:6c53:1fc1%18]) with mapi id 15.01.1282.017; Mon, 24 Jul 2017 10:10:51 +0000 From: Nipun Gupta To: Nikhil Rao , "jerin.jacob@caviumnetworks.com" CC: "gage.eads@intel.com" , "dev@dpdk.org" , "thomas@monjalon.net" , "bruce.richardson@intel.com" , "harry.van.haaren@intel.com" , Hemant Agrawal , "narender.vangati@intel.com" , Abhinandan Gujjar Thread-Topic: [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues Thread-Index: AQHS9loW9CPBh9hook6mT2JxEF8VJqJihdpQ Date: Mon, 24 Jul 2017 10:10:50 +0000 Message-ID: References: <29140c16-909a-1b9a-7391-481f900bd13c@intel.com> <1499377952-5306-1-git-send-email-nikhil.rao@intel.com> In-Reply-To: <1499377952-5306-1-git-send-email-nikhil.rao@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=nipun.gupta@nxp.com; x-originating-ip: [192.88.169.1] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM2PR04MB0756; 7:M/QRT9fgA/q0gkaTP+0TiC2GA+LoXS+GjMCEeDUCAO1eXPU8dtMLiCJEUUaFuua6ujanpslgu1KaPAb3H7c7NUJ5PRLinYZk2W3g+JrJP2P1iATraCDuKM+DcRpwE8r/VYD0nJJIf46qO6++fiejr1u33a354ZUAqa9OyMFHqsYbPbj8owIi1Fnm5jt/P1NbJrfCwIfndWmYdAA2GdeK62hACQ+2K1E93dHqiA6dsh0zdWvJD84THJSXENWsynJyazlVCBoko35nMqkqiGFee7hyJPDs+rD8E/raO2ob07xHSfGj5Q+8Dv/zk6RsGei+8HS+/tBQ6cJSxU4f1+J5WKrarRaq85ittZMIClELjS4JRn5H3KBIeQZ4AzbJ8DZc+G1FBl3WK52fBMJwm96SY7HmBERmpTij21HgS7O2pGawfSFK6vnztw6b+prwOmVM0FWFv5W7B5/Q+qIgFUCVFS2nnZPX2cVjylYm7OxDeX93JEAL9MKlB77f7RDBnX0EOTxE7qD4i5CLLUCIu28oX0IeVrN7+WqFUEf8wigh8oiWaVYN6iyaUic7ddd+SXRBxLx8RL7qdea9+7AMlwwF86e+7sHd0duiwIT4cSqrvW2AlflOFhAhtv3gbxEe2O5yH+rM+4foAFJtZ2+u1a5d6TOJlUhNnbaUCriza36MXv3XGnAMU4AyQCVmoh1fNlbnI+LFFH1xiGJgjq121V5/WJugROp8YK5/td6bO9S+9tabk3sorOuOcWliJtw2mJTvLG3pXoYnDRTAew3OaUredPqw/qPq+GFB3OCKUYz0uAs= x-forefront-antispam-report: SFV:SKI; SCL:-1; SFV:NSPM; SFS:(10009020)(6009001)(39410400002)(39400400002)(39850400002)(39860400002)(39840400002)(39450400003)(43544003)(199003)(189002)(13464003)(106356001)(14454004)(229853002)(5660300001)(3280700002)(3660700001)(102836003)(966005)(50986999)(25786009)(6116002)(86362001)(575784001)(3846002)(76176999)(54356999)(6506006)(53546010)(7696004)(4326008)(2906002)(105586002)(68736007)(478600001)(6436002)(7736002)(2900100001)(33656002)(55016002)(53936002)(101416001)(66066001)(2501003)(54906002)(305945005)(99286003)(81166006)(97736004)(53376002)(8936002)(2950100002)(9686003)(5250100002)(6306002)(38730400002)(6246003)(189998001)(74316002)(81156014)(5890100001)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM2PR04MB0756; H:HE1PR0401MB2425.eurprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; x-ms-office365-filtering-correlation-id: 373ca221-ce97-437e-5baf-08d4d27c435b x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254075)(48565401081)(300000503095)(300135400095)(2017052603031)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095); SRVR:AM2PR04MB0756; x-ms-traffictypediagnostic: AM2PR04MB0756: x-exchange-antispam-report-test: UriScan:(278428928389397)(185117386973197)(228905959029699); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(100000703101)(100105400095)(3002001)(6055026)(6041248)(20161123562025)(20161123558100)(20161123560025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(20161123564025)(6072148)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:AM2PR04MB0756; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:AM2PR04MB0756; x-forefront-prvs: 0378F1E47A received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jul 2017 10:10:51.1420 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR04MB0756 Subject: Re: [dpdk-dev] [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jul 2017 10:10:55 -0000 Hi Nikhil/Edas, > -----Original Message----- > From: Nikhil Rao [mailto:nikhil.rao@intel.com] > Sent: Friday, July 07, 2017 3:23 > To: jerin.jacob@caviumnetworks.com > Cc: gage.eads@intel.com; dev@dpdk.org; thomas@monjalon.net; > bruce.richardson@intel.com; harry.van.haaren@intel.com; Hemant Agrawal > ; Nipun Gupta ; > narender.vangati@intel.com; Nikhil Rao ; Abhinandan > Gujjar > Subject: [PATCH 1/2] eventdev: add event adapter for ethernet Rx queues >=20 > Eventdev-based networking applications require a component to dequeue > packets from NIC Rx queues and inject them into eventdev queues[1]. While > some platforms (e.g. Cavium Octeontx) do this operation in hardware, othe= r > platforms use software. >=20 > This patchset introduces an ethernet Rx event adapter that dequeues packe= ts > from ethernet devices and enqueues them to event devices. It is based on > a previous RFC[2]. >=20 > The adapter is designed to work with the EAL service core[3]. If > an application determines that the adapter is required, it can register a= nd > launch it on a service core. Alternatively, this adapter can serve as a > template for applications to design customer ethernet Rx event adapters > better suited to their needs. >=20 > The adapter can service multiple ethernet devices and queues. Each queue = is > configured with a servicing weight to control the relative frequency with > which the adapter polls the queue, and the event fields to use when > constructing packet events. The adapter has two modes for programming an > event's flow ID: use a static per-queue user-specified value or use the R= SS > hash. >=20 > A detailed description of the adapter is contained in the header's > comments. >=20 > [1] http://dpdk.org/ml/archives/dev/2017-May/065341.html > [2] http://dpdk.org/ml/archives/dev/2017-May/065539.html > [3] http://dpdk.org/ml/archives/dev/2017-July/069782.html >=20 > Signed-off-by: Nikhil Rao > Signed-off-by: Gage Eads > Signed-off-by: Abhinandan Gujjar > --- >=20 > v2: > Thanks Jerin for review - below is a list of changes you > suggested. >=20 > - all public symbols are started with rte_event_. > - Add Doxygen reference with @see. > - Mention setting of ev.event_type. > - Mention adapter to service function mapping. > - Remove rte_eth_rx_event_adapter_dev_add/del(). > - Change rx_queuee_id to int32_t and use -1 to denote all Rx queues. > - Add rte_eth_event_rx_queue_del(). >=20 > Other changes > - Remove adapter's run function (rte_event_eth_rx_adapter_run()) from > the public interface. The adapter internally uses it to create a > service. > - Add a blocked cycle count to stats. Further description is contained > in the header. > - Minor struct renames rte_event_eth_rx_adapter_config -> .._conf > --- > lib/librte_eventdev/rte_event_eth_rx_adapter.h | 301 +++++++++ > lib/librte_eventdev/rte_event_eth_rx_adapter.c | 825 > +++++++++++++++++++++++++ > lib/librte_eventdev/rte_eventdev_version.map | 13 + > lib/Makefile | 2 +- > lib/librte_eventdev/Makefile | 2 + > 5 files changed, 1142 insertions(+), 1 deletion(-) > create mode 100644 lib/librte_eventdev/rte_event_eth_rx_adapter.h > create mode 100644 lib/librte_eventdev/rte_event_eth_rx_adapter.c >=20 > diff --git a/lib/librte_eventdev/rte_event_eth_rx_adapter.h > b/lib/librte_eventdev/rte_event_eth_rx_adapter.h > new file mode 100644 > index 000000000..5ccd0bd24 > --- /dev/null > +++ b/lib/librte_eventdev/rte_event_eth_rx_adapter.h > @@ -0,0 +1,301 @@ > +/* > + * Copyright(c) 2017 Intel Corporation. All rights reserved. > + * All rights reserved. > + * > + * Redistribution and use in source and binary forms, with or without > + * modification, are permitted provided that the following conditions > + * are met: > + * > + * * Redistributions of source code must retain the above copyright > + * notice, this list of conditions and the following disclaimer. > + * * Redistributions in binary form must reproduce the above copyrig= ht > + * notice, this list of conditions and the following disclaimer in > + * the documentation and/or other materials provided with the > + * distribution. > + * * Neither the name of Intel Corporation nor the names of its > + * contributors may be used to endorse or promote products derived > + * from this software without specific prior written permission. > + * > + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND > CONTRIBUTORS > + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT > + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND > FITNESS FOR > + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE > COPYRIGHT > + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, > INCIDENTAL, > + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT > NOT > + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS > OF USE, > + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND > ON ANY > + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR > TORT > + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF > THE USE > + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH > DAMAGE. > + */ > + > +#ifndef _RTE_EVENT_ETH_RX_ADAPTER_ > +#define _RTE_EVENT_ETH_RX_ADAPTER_ > + > +/** > + * @file > + * > + * RTE Event Ethernet Rx Adapter > + * > + * An eventdev-based packet processing application enqueues/dequeues mbu= fs > + * to/from the event device. The ethernet Rx event adapter's role is to = transfer > + * mbufs from the ethernet receive queues managed by DPDK to an event > device. > + * The application uses the adapter APIs to configure the packet flow be= tween > + * the ethernet devices and event devices. The adapter is designed to wo= rk with > + * the EAL service cores. The adapter's work can be parallelized by divi= ding the > + * NIC Rx queues among multiple adapter services that run in parallel. First of all, apologies for commenting late on this. I am going through thi= s patchset and have some concerns over this. It is mentioned that "The adapter is designed to work with * the EAL servic= e cores" Does this mean that the adapter cannot function without service core? In the case where Ethdev HW is capable of injecting the packet to compatibl= e HW eventdev driver, there is no service required. It seems this patchset does not take care of this use-case or may= be I am missing something here? > + * > + * Before using the adapter, the application needs to enumerate and conf= igure > + * the ethernet devices that it wishes to use. This is typically done us= ing the > + * following DPDK ethdev functions: > + * - rte_eth_dev_configure() > + * - rte_eth_tx_queue_setup() > + * - rte_eth_rx_queue_setup() > + * - rte_eth_dev_start() > + * > + * The application also configures an event device and creates event por= ts > + * to interface with the event device. In addition to the event ports us= ed by > + * its packet processing functions, the application creates an event por= t > + * to be used by this adapter. > + * > + * The ethernet Rx event adapter's functions are: > + * - rte_event_eth_rx_adapter_create() > + * - rte_event_eth_rx_adapter_free() > + * - rte_event_eth_rx_adapter_queue_add() > + * - rte_event_eth_rx_adapter_queue_del() > + * - rte_event_eth_rx_adapter_stats_get() > + * - rte_event_eth_rx_adapter_stats_reset() > + * > + * The applicaton creates an event to ethernet adapter using > + * rte_event_eth_rx_adapter_create(). The event device and event port > + * identifiers are passed to this function. > + * > + * The adapter needs to know which ethernet rx queues to poll for mbufs = as > well > + * as event device parameters such as the event queue identifier, event > + * priority and scheduling type that the adapter should use when constru= cting > + * events. The rte_event_eth_rx_adapter_queue_add() function is provided= for > + * this purpose. > + * > + * At the time of adding an ethernet device receive queue, the applicati= on can > + * also specify a static event flow id and set the > + * RTE_ETH_RX_EVENT_ADAPTER_QUEUE_FLOW_ID_VALID bit of the > rx_queue_flags > + * member of the rte_event_eth_rx_adapter_queue_conf structure. If the > + * RTE_ETH_RX_EVENT_ADAPTER_QUEUE_FLOW_ID_VALID isn't set, the flow > id is > + * assigned the value of the RSS hash. The adapter generates the RSS has= h if it > + * hasn't been already computed by the NIC, based on source and destinat= ion > + * IPv4/6 addresses, using the rte_softrss_be() routine included in the = DPDK. > + * > + * The servicing weight parameter in the > rte_event_eth_rx_adapter_queue_conf > + * intended to provide application control of the polling frequency of e= thernet > + * device receive queues, for example, the application may want to poll = higher > + * priority queues with a higher frequency but at the same time not star= ve > + * lower priority queues completely. If this parameter is zero and the r= eceive > + * interrupt is enabled when configuring the device, the receive queue i= s > + * interrupt driven; else, the queue is assigned a servicing weight of o= ne. > + * > + * Note: Interrupt driven receive queues are currentely unimplemented. > + */ > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include > +#include > + > +#include "rte_eventdev.h" > + > + > +#define RTE_MAX_EVENT_ETH_RX_ADAPTER_INSTANCE 32 > + > +/* struct rte_event_eth_rx_adapter_queue_conf flags definitions */ > +#define RTE_EVENT_ETH_RX_ADAPTER_QUEUE_FLOW_ID_VALID 0x1 > +/**< This flag indicates the flow identifier is valid > + * @see rte_event_eth_rx_adapter_queue_conf > + */ > + > +static int > +_rte_event_eth_rx_adapter_queue_add(struct rte_event_eth_rx_adapter > *rx_adapter, > + struct eth_device_info *dev_info, > + uint16_t rx_queue_id, > + const struct > rte_event_eth_rx_adapter_queue_conf *conf) > +{ > + int ret; > + struct eth_rx_queue_info *queue_info; > + const struct rte_event *ev; > + > + if (rx_queue_id >=3D dev_info->dev->data->nb_rx_queues) > + return -EINVAL; > + > + queue_info =3D &dev_info->rx_queue[rx_queue_id]; > + if (queue_info->queue_enabled) > + return -EEXIST; > + > + ev =3D &conf->ev; > + memset(queue_info, 0, sizeof(*queue_info)); > + queue_info->event_queue_id =3D ev->queue_id; > + queue_info->sched_type =3D ev->sched_type; > + queue_info->priority =3D ev->priority; > + queue_info->wt =3D conf->servicing_weight; > + > + if (queue_info->wt =3D=3D 0) { > + struct rte_eth_dev_data *data =3D dev_info->dev->data; > + > + /* If Rx interrupts are disabled set wt =3D 1 */ > + queue_info->wt =3D !data->dev_conf.intr_conf.rxq; > + } > + > + if (conf-> > + rx_queue_flags & > RTE_EVENT_ETH_RX_ADAPTER_QUEUE_FLOW_ID_VALID) { > + queue_info->flow_id =3D ev->flow_id; > + queue_info->flow_id_mask =3D ~0; > + } > + > + queue_info->queue_enabled =3D true; > + rx_adapter->num_rx_polled++; > + ret =3D eth_poll_wrr_calc(rx_adapter); > + if (ret) { > + rx_adapter->num_rx_polled--; > + queue_info->queue_enabled =3D false; > + return ret; > + } > + > + return 0; > +} > + > +int > +rte_event_eth_rx_adapter_queue_add(uint8_t id, > + uint8_t eth_dev_id, > + int32_t rx_queue_id, > + const struct > rte_event_eth_rx_adapter_queue_conf *conf) > +{ > + int ret =3D 0; > + struct rte_event_eth_rx_adapter *rx_adapter; > + struct eth_device_info *dev_info; > + uint32_t i, j; > + > + rx_adapter =3D rte_event_eth_rx_adapter[id]; > + if (!valid_id(id) || !rte_event_eth_rx_adapter || !rx_adapter || > + eth_dev_id >=3D rte_eth_dev_count() || !conf) > + return -EINVAL; > + > + rte_spinlock_lock(&rx_adapter->rx_lock); > + > + dev_info =3D &rx_adapter->eth_devices[eth_dev_id]; > + if (rx_queue_id =3D=3D -1) { > + for (i =3D 0; i < dev_info->dev->data->nb_rx_queues; i++) { > + ret =3D > + _rte_event_eth_rx_adapter_queue_add(rx_adapter, > dev_info, i, > + conf); > + if (ret) { > + for (j =3D 0; j < i; j++) > + > _rte_event_eth_rx_adapter_queue_del(rx_adapter, > + > dev_info, > + j); > + } > + } > + > + } else { > + ret =3D _rte_event_eth_rx_adapter_queue_add(rx_adapter, > dev_info, > + > (uint16_t)rx_queue_id, > + conf); > + } > + > + rte_spinlock_unlock(&rx_adapter->rx_lock); > + > + return ret; > +} Looking at the rte_event_eth_rx_adapter_queue_add & event_eth_rx_adapter_se= rvice_func it seems that this indeed will not fit with the cases where ethdev is capab= le of enqueing packets to the eventdev (as was mentioned in Jerin's first RFC). In case of hardware based eventdev and queues, these function should also i= nvoke respective PMD HW configs. e.g. In queue case - rte_eventdev and rte_ethdev - both PMDs at= hw level shall be configured. A typical eventdev hardware will require queues of eth devices will be conf= igured to directly attach to eventdev in the hardware. Mapping it to NXP eventdev, enabling this functionality requires some confi= guration where dev private information of both the devices (event dev and eth dev) is required at the = same time, and the final configuration is provided via eth device to H/W. So, this req= uire inter device communication in DPDK. Jerin, I have an impression that Cavium hardware has H/W capability to inject pack= ets from Ethernet devices to event devices? If yes, how do you plan to support it? Thanks, Nipun