From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0050.outbound.protection.outlook.com [104.47.1.50]) by dpdk.org (Postfix) with ESMTP id 1EC5847D1 for ; Mon, 7 Nov 2016 13:30:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=cfZVy40ui5nYvLY5+9qCG7WkWi5VWNoX4Na3Usp6/Go=; b=m+8a8ek3yXvKM+FXSPYWX43fUhQDig4J0uFC654LqBU+jsh8rRNLHlqTyuythFge7CGm8dfYiZQf1zTnDOlCxfyQBIxGhzWg7QlPytyq/QYNXaYEaaoP8Sl2d/Q9zjjTsfLo+o0cwj78G8s94SjpyM3PPHZnRQz0tqaQArDs6YM= Received: from DB5PR04MB1605.eurprd04.prod.outlook.com (10.164.38.147) by DB5PR04MB1606.eurprd04.prod.outlook.com (10.164.38.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.707.6; Mon, 7 Nov 2016 12:30:26 +0000 Received: from DB5PR04MB1605.eurprd04.prod.outlook.com ([10.164.38.147]) by DB5PR04MB1605.eurprd04.prod.outlook.com ([10.164.38.147]) with mapi id 15.01.0707.006; Mon, 7 Nov 2016 12:30:26 +0000 From: Hemant Agrawal To: Olivier Matz CC: "dev@dpdk.org" , "jerin.jacob@caviumnetworks.com" , "david.hunt@intel.com" Thread-Topic: [PATCH v3 2/2] mempool: pktmbuf pool default fallback for mempool ops error Thread-Index: AQHSJhQK19GhEpRKXUi0B/Amkg4mI6DNlr0w Date: Mon, 7 Nov 2016 12:30:26 +0000 Message-ID: References: <1473959607-1951-1-git-send-email-hemant.agrawal@nxp.com> <1474044395-11627-1-git-send-email-hemant.agrawal@nxp.com> <1474044395-11627-2-git-send-email-hemant.agrawal@nxp.com> <7e26965e-173b-1cf6-0beb-c94712ad615a@nxp.com> <412b3d71-f6df-7ee5-6189-ed7af1268fed@6wind.com> In-Reply-To: <412b3d71-f6df-7ee5-6189-ed7af1268fed@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=hemant.agrawal@nxp.com; x-originating-ip: [192.88.169.1] x-ms-office365-filtering-correlation-id: 77e3712c-513c-49f4-7420-08d40709da72 x-microsoft-exchange-diagnostics: 1; DB5PR04MB1606; 7:YwDo1jUySQixkgTKtldX8r3K8fdsr8iHL4liIdOYSTMtfw9Y5pt65Uj2fbnGBHbtAHKmpKrfJ6RfXmQpb31Gcez/1iyiE8ibwwhP7wqrjnGuTiLJgxVKssqosjMVo86sYQi0YfRhCfh8PyspydDaxC5ToPfbC3uW817ha/Wl4dfffYOT3qTgkgEJMXU/CGyiXTMwbkEWwq7LQTFTSW02d+lIAeRTQCTqSwFKLOKIiyypTOCbJGP1ONzG7CsLW+LQRbIbV2Ki6lA151dAOba4Tx3U66/HI01msQcGRcK74x+Qrw7NPQmp/Vu0KD+r4aUb0FvsErfEGTDsMV2/qVMcMKBNJkV3ELw/mRRHqNllf+M= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB5PR04MB1606; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(278428928389397)(185117386973197); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6045074)(6060229)(6040176)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6055026)(6061226)(6046074); SRVR:DB5PR04MB1606; BCL:0; PCL:0; RULEID:; SRVR:DB5PR04MB1606; x-forefront-prvs: 0119DC3B5E x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(199003)(13464003)(189002)(377454003)(24454002)(43544003)(93886004)(2950100002)(77096005)(33656002)(122556002)(6916009)(15975445007)(92566002)(9686002)(4326007)(586003)(189998001)(76176999)(54356999)(50986999)(6116002)(2906002)(8936002)(101416001)(2900100001)(97736004)(105586002)(10400500002)(106356001)(86362001)(575784001)(106116001)(87936001)(7696004)(110136003)(305945005)(5660300001)(11100500001)(68736007)(3846002)(102836003)(5002640100001)(7846002)(7736002)(76576001)(3280700002)(8676002)(3660700001)(81166006)(81156014)(66066001)(15395725005)(74316002)(19580395003)(19580405001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR04MB1606; H:DB5PR04MB1605.eurprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Nov 2016 12:30:26.5995 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR04MB1606 Subject: Re: [dpdk-dev] [PATCH v3 2/2] mempool: pktmbuf pool default fallback for mempool ops error X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Nov 2016 12:30:29 -0000 Hi Olivier, =09 > -----Original Message----- > From: Olivier Matz [mailto:olivier.matz@6wind.com] > Sent: Friday, October 14, 2016 5:41 PM > > On 9/22/2016 6:42 PM, Hemant Agrawal wrote: > >> Hi Olivier > >> > >> On 9/19/2016 7:27 PM, Olivier Matz wrote: > >>> Hi Hemant, > >>> > >>> On 09/16/2016 06:46 PM, Hemant Agrawal wrote: > >>>> In the rte_pktmbuf_pool_create, if the default external mempool is > >>>> not available, the implementation can default to "ring_mp_mc", > >>>> which is an software implementation. > >>>> > >>>> Signed-off-by: Hemant Agrawal > >>>> --- > >>>> Changes in V3: > >>>> * adding warning message to say that falling back to default sw > >>>> pool > >>>> --- > >>>> lib/librte_mbuf/rte_mbuf.c | 8 ++++++++ > >>>> 1 file changed, 8 insertions(+) > >>>> > >>>> diff --git a/lib/librte_mbuf/rte_mbuf.c > >>>> b/lib/librte_mbuf/rte_mbuf.c index 4846b89..8ab0eb1 100644 > >>>> --- a/lib/librte_mbuf/rte_mbuf.c > >>>> +++ b/lib/librte_mbuf/rte_mbuf.c > >>>> @@ -176,6 +176,14 @@ rte_pktmbuf_pool_create(const char *name, > >>>> unsigned n, > >>>> > >>>> rte_errno =3D rte_mempool_set_ops_byname(mp, > >>>> RTE_MBUF_DEFAULT_MEMPOOL_OPS, NULL); > >>>> + > >>>> + /* on error, try falling back to the software based default > >>>> pool */ > >>>> + if (rte_errno =3D=3D -EOPNOTSUPP) { > >>>> + RTE_LOG(WARNING, MBUF, "Default HW Mempool not supported. " > >>>> + "falling back to sw mempool \"ring_mp_mc\""); > >>>> + rte_errno =3D rte_mempool_set_ops_byname(mp, "ring_mp_mc", > >>>> NULL); > >>>> + } > >>>> + > >>>> if (rte_errno !=3D 0) { > >>>> RTE_LOG(ERR, MBUF, "error setting mempool handler\n"); > >>>> return NULL; > >>>> > >>> > >>> Without adding a new method ".supported()", the first call to > >>> rte_mempool_populate() could return the same error ENOTSUP. In this > >>> case, it is still possible to fallback. > >>> > >> It will be bit late. > >> > >> On failure, than we have to set the default ops and do a goto before > >> rte_pktmbuf_pool_init(mp, &mbp_priv); >=20 > I still think we can do the job without adding the .supported() method. > The following code is just an (untested) example: >=20 > struct rte_mempool * > rte_pktmbuf_pool_create(const char *name, unsigned n, > unsigned cache_size, uint16_t priv_size, uint16_t data_room_size, > int socket_id) > { > struct rte_mempool *mp; > struct rte_pktmbuf_pool_private mbp_priv; > unsigned elt_size; > int ret; > const char *ops[] =3D { > RTE_MBUF_DEFAULT_MEMPOOL_OPS, "ring_mp_mc", NULL, > }; > const char **op; >=20 > if (RTE_ALIGN(priv_size, RTE_MBUF_PRIV_ALIGN) !=3D priv_size) { > RTE_LOG(ERR, MBUF, "mbuf priv_size=3D%u is not aligned\n", > priv_size); > rte_errno =3D EINVAL; > return NULL; > } > elt_size =3D sizeof(struct rte_mbuf) + (unsigned)priv_size + > (unsigned)data_room_size; > mbp_priv.mbuf_data_room_size =3D data_room_size; > mbp_priv.mbuf_priv_size =3D priv_size; >=20 > for (op =3D &ops[0]; *op !=3D NULL; op++) { > mp =3D rte_mempool_create_empty(name, n, elt_size, cache_size, > sizeof(struct rte_pktmbuf_pool_private), socket_id, 0); > if (mp =3D=3D NULL) > return NULL; >=20 > ret =3D rte_mempool_set_ops_byname(mp, *op, NULL); > if (ret !=3D 0) { > RTE_LOG(ERR, MBUF, "error setting mempool handler\n"); > rte_mempool_free(mp); > if (ret =3D=3D -ENOTSUP) > continue; > rte_errno =3D -ret; > return NULL; > } > rte_pktmbuf_pool_init(mp, &mbp_priv); >=20 > ret =3D rte_mempool_populate_default(mp); > if (ret < 0) { > rte_mempool_free(mp); > if (ret =3D=3D -ENOTSUP) > continue; > rte_errno =3D -ret; > return NULL; > } > } >=20 > rte_mempool_obj_iter(mp, rte_pktmbuf_init, NULL); >=20 > return mp; > } >=20 >=20 [Hemant] This look fine to me. Please submit a patch for the same.=20 > >>> I've just submitted an RFC, which I think is quite linked: > >>> http://dpdk.org/ml/archives/dev/2016-September/046974.html > >>> Assuming a new parameter "mempool_ops" is added to > >>> rte_pktmbuf_pool_create(), would it make sense to fallback to > >>> "ring_mp_mc"? What about just returning ENOTSUP? The application > >>> could do the job and decide which sw fallback to use. > >> > >> We ran into this issue when trying to run the standard DPDK examples > >> (l3fwd) in VM. Do you think, is it practical to add fallback handling > >> in each of the DPDK examples? >=20 > OK. What is still unclear for me, is how the software is aware of the dif= ferent > hardware-assisted handlers. Moreover, we could imagine more software > handlers, which could be used depending on the use case. >=20 > I think this choice has to be made by the user or the application: >=20 > - the application may want to use a specific (sw or hw) handler: in > this case, it want to be notified if it fails, instead of having > a quiet fallback to ring_mp_mc > - if several handlers are available, the application may want to > try them in a specific order > - maybe some handlers will have some limitations with some > configurations or driver? The application could decide to use > a different handler according the configuration. >=20 > So that's why I think this is an application decision. >=20 [Hemant] We should simplify it: if the application has supplied the handl= er, it is application's responsibility to take care of failure. Only if the= application want to use the default handler, the implementation can fallba= ck. The fallback handler can again be configurable.=20 > >> Typically when someone is writing a application on host, he need not > >> worry non-availability of the hw offloaded mempool. He may also want > >> to run the same binary in virtual machine. In VM, it is not > >> guaranteed that hw offloaded mempools will be available. >=20 > Running the same binary is of course a need. But if your VM does not prov= ide > the same virtualized hardware than the host, I think the command line opt= ion > makes sense. >=20 > AFAIU, on the host, you can use a hw mempool handler or a sw one, and on = the > guest, only a sw one. So you need an option to select the behavior you wa= nt on > the host, without recompiling. >=20 > I understand that modifying all the applications is not a good option eit= her. I'm > thinking a a EAL parameter that would allow to configure a library (mbuf = in this > case). Something like kernel boot options. > Example: testpmd -l 2,4 --opt mbuf.handler=3D"ring_mp_mc" -- [app args] >=20 > I don't know if this is feasible, this has to be discussed first, but wha= t do you > think on the principle? The value could also be an ordered list if we wan= t a > fallback, and this option could be overriden by the application before cr= eating > the mbuf pool. There was some discussions about a kind of dpdk global > configuration some time ago. >=20 [Hemant] I agree that command line option provide a better control in this = case.=20 On the flipside, We need to be careful that we do not end up having too man= y command line options.=20 >=20 > Regards, > Olivier