From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3on0075.outbound.protection.outlook.com [157.55.234.75]) by dpdk.org (Postfix) with ESMTP id 45A78ADFA for ; Wed, 8 Jun 2016 15:49:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Ms2EohVwnOpwUU6LXcLcc0yEG6H544jOH6DpNKB1DoQ=; b=u+JJmiG/gtQ0Tcl+n5tl4CXP4XDZZghbVwAgRXHF+AirbtQfEu1Lmu2vfpD1ZMEb6vNdH0rrsUIaQ+TZmqXyoUmPNP9/yjEmTix6PIVmxAJzuR1zr8mfIedIVbZJVKYD75tkL7yzy397v0Gk03Ex1rqT4GX2IEZkADo9KN4ZXt4= Received: from DB5PR0401MB2054.eurprd04.prod.outlook.com (10.166.11.137) by DB5PR0401MB2053.eurprd04.prod.outlook.com (10.166.11.136) with Microsoft SMTP Server (TLS) id 15.1.517.8; Wed, 8 Jun 2016 13:49:05 +0000 Received: from DB5PR0401MB2054.eurprd04.prod.outlook.com ([10.166.11.137]) by DB5PR0401MB2054.eurprd04.prod.outlook.com ([10.166.11.137]) with mapi id 15.01.0511.010; Wed, 8 Jun 2016 13:49:05 +0000 From: Shreyansh Jain To: "Hunt, David" , "dev@dpdk.org" CC: "olivier.matz@6wind.com" , "viktorin@rehivetech.com" , "jerin.jacob@caviumnetworks.com" Thread-Topic: [dpdk-dev] [PATCH v8 1/3] mempool: support external mempool operations Thread-Index: AQHRwJ6WYtTJj5Q9pk+CUYpxLcIab5/fWYJQ Date: Wed, 8 Jun 2016 13:48:45 +0000 Deferred-Delivery: Wed, 8 Jun 2016 13:47:47 +0000 Message-ID: References: <1464874043-67467-1-git-send-email-david.hunt@intel.com> <1464965906-108927-1-git-send-email-david.hunt@intel.com> <1464965906-108927-2-git-send-email-david.hunt@intel.com> <5756931C.70805@intel.com> In-Reply-To: <5756931C.70805@intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shreyansh.jain@nxp.com; x-originating-ip: [192.88.169.1] x-ms-office365-filtering-correlation-id: ecb8edf3-9bfc-47f1-edd7-08d38fa3a804 x-microsoft-exchange-diagnostics: 1; DB5PR0401MB2053; 6:+TukcHEgyZ7N0QNOW/lB5xFsKue3KtHuYuq16YSloSZzC+m/+6Qc6AxbvvFBYJEdk7mIopTOhrfM5U99xV1v8zufQNi88lUaFQ9z09BgmBpxzOulWLLkwmqVWbJold5h0UkdUR0jLY3FqETKSxJ/wHv16xhaDkK8BAw6wj28sC2fJoTFEabhlpA+hUIuRHLdBguWVKVknKR93SjnM11hyWhQTOCDONaCQ1qEOKYNUlof6p8ff5rooQ25BZX14pmBjuH3j3K0dJQRHC2Dk0eqYrGc16P2D8OSBhfd36KhwlZLA4AoUvcUHt+RtGQ8heX7; 5:MMlTD99r31kuf5u2DiXk43N4t6tImSljpmDeqVWnq4YuITDkVHolfzA1+Nezrkk2+FJkG/l6yVoQpss9YFe/xB6gisdfcYja5rg0lOwfgIxvESpg7FZGibVt/wyjfKZ/P00Oxh4Zadk/8/hxOtB2Vg==; 24:nqILWwsM6MfdxY7cDg64YmIsiBCf1O4jyP7+pG6IpwOo7AGE94bL+PXrdwM4ycTpZRd0Kup4eadnvrZP8d6n4vx7vCQbdJL9Ympd5lq2bps=; 7:2HGqpD9dCGHOzRE+2va5p9R0VkJwSFiaoZDnrPea2VNKCQt1zDm35gDKgZrDy+GJdtb1TMeiVRAykWwKedXKpcfnGfwH7bE57PCL0dBvXuooptgw3Ct1VjnWXA4jhawHWiPZhf9ycvIAyA23lAC9lA3R03t3+4u0aWuCui3DTVJhUyQ1nJBR46zlam4T9u2Uz6Wya3rgmklSL+MoUnxdWsAOpEDJI6ImBfnwDg37TlQ= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB5PR0401MB2053; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(278428928389397)(185117386973197)(788757137089)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6055026); SRVR:DB5PR0401MB2053; BCL:0; PCL:0; RULEID:; SRVR:DB5PR0401MB2053; x-forefront-prvs: 0967749BC1 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(199003)(377454003)(13464003)(189002)(24454002)(51914003)(122556002)(86362001)(74316001)(87936001)(76576001)(11100500001)(189998001)(2906002)(5008740100001)(10400500002)(106116001)(106356001)(105586002)(50986999)(54356999)(33656002)(5004730100002)(76176999)(81166006)(5001770100001)(97736004)(101416001)(5002640100001)(4326007)(9686002)(81156014)(8676002)(66066001)(8936002)(102836003)(19580395003)(3660700001)(92566002)(3846002)(5003600100002)(19580405001)(2501003)(6116002)(68736007)(3280700002)(93886004)(77096005)(2900100001)(586003)(2950100001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR0401MB2053; H:DB5PR0401MB2054.eurprd04.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; CAT:NONE; LANG:en; CAT:NONE; received-spf: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Jun 2016 13:49:04.8833 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR0401MB2053 Subject: Re: [dpdk-dev] [PATCH v8 1/3] mempool: support external mempool operations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jun 2016 13:49:06 -0000 Hi David, Thanks for explanation. I have some comments inline... > -----Original Message----- > From: Hunt, David [mailto:david.hunt@intel.com] > Sent: Tuesday, June 07, 2016 2:56 PM > To: Shreyansh Jain ; dev@dpdk.org > Cc: olivier.matz@6wind.com; viktorin@rehivetech.com; > jerin.jacob@caviumnetworks.com > Subject: Re: [dpdk-dev] [PATCH v8 1/3] mempool: support external mempool > operations >=20 > Hi Shreyansh, >=20 > On 6/6/2016 3:38 PM, Shreyansh Jain wrote: > > Hi, > > > > (Apologies for overly-eager email sent on this thread earlier. Will be = more > careful in future). > > > > This is more of a question/clarification than a comment. (And I have ta= ken > only some snippets from original mail to keep it cleaner) > > > > > >> +MEMPOOL_REGISTER_OPS(ops_mp_mc); > >> +MEMPOOL_REGISTER_OPS(ops_sp_sc); > >> +MEMPOOL_REGISTER_OPS(ops_mp_sc); > >> +MEMPOOL_REGISTER_OPS(ops_sp_mc); > > > > > > From the above what I understand is that multiple packet pool handlers= can > be created. > > > > I have a use-case where application has multiple pools but only the pac= ket > pool is hardware backed. Using the hardware for general buffer requiremen= ts > would prove costly. > > From what I understand from the patch, selection of the pool is based = on > the flags below. >=20 > The flags are only used to select one of the default handlers for > backward compatibility through > the rte_mempool_create call. If you wish to use a mempool handler that > is not one of the > defaults, (i.e. a new hardware handler), you would use the > rte_create_mempool_empty > followed by the rte_mempool_set_ops_byname call. > So, for the external handlers, you create and empty mempool, then set > the operations (ops) > for that particular mempool. I am concerned about the existing applications (for example, l3fwd). Explicit calls to 'rte_create_mempool_empty->rte_mempool_set_ops_byname' mo= del would require modifications to these applications. Ideally, without any modifications, these applications should be able to us= e packet pools (backed by hardware) and buffer pools (backed by ring/others= ) - transparently. If I go by your suggestions, what I understand is, doing the above without = modification to applications would be equivalent to: struct rte_mempool_ops custom_hw_allocator =3D {...} thereafter, in config/common_base: CONFIG_RTE_DEFAULT_MEMPOOL_OPS=3D"custom_hw_allocator" calls to rte_pktmbuf_pool_create would use the new allocator. But, another problem arises here. There are two distinct paths for allocations of a memory pool: 1. A 'pkt' pool: rte_pktmbuf_pool_create =20 \- rte_mempool_create_empty | \- rte_mempool_set_ops_byname(..ring_mp_mc..) | `- rte_mempool_set_ops_byname (...RTE_MBUF_DEFAULT_MEMPOOL_OPS..) /* Override default 'ring_mp_mc' of * rte_mempool_create */ 2. Through generic mempool create API rte_mempool_create \- rte_mempool_create_empty (passing pktmbuf and pool constructors) =20 I found various instances in example applications where rte_mempool_create(= ) is being called directly for packet pools - bypassing the more semantical= ly correct call to rte_pktmbuf_* for packet pools. In (2) control path, RTE_MBUF_DEFAULT_MEMPOOLS_OPS wouldn't be able to repl= ace custom handler operations for packet buffer allocations. >>From a performance point-of-view, Applications should be able to select bet= ween packet pools and non-packet pools. >=20 > > > >> + /* > >> + * Since we have 4 combinations of the SP/SC/MP/MC examine the flags= to > >> + * set the correct index into the table of ops structs. > >> + */ > >> + if (flags & (MEMPOOL_F_SP_PUT | MEMPOOL_F_SC_GET)) > >> + rte_mempool_set_ops_byname(mp, "ring_sp_sc"); > >> + else if (flags & MEMPOOL_F_SP_PUT) > >> + rte_mempool_set_ops_byname(mp, "ring_sp_mc"); > >> + else if (flags & MEMPOOL_F_SC_GET) > >> + rte_mempool_set_ops_byname(mp, "ring_mp_sc"); > >> + else > >> + rte_mempool_set_ops_byname(mp, "ring_mp_mc"); > >> + My suggestion is to have an additional flag, 'MEMPOOL_F_PKT_ALLOC', which, = if specified, would: ... #define MEMPOOL_F_SC_GET 0x0008 #define MEMPOOL_F_PKT_ALLOC 0x0010 ... in rte_mempool_create_empty: ... after checking the other MEMPOOL_F_* flags... if (flags & MEMPOOL_F_PKT_ALLOC) rte_mempool_set_ops_byname(mp, RTE_MBUF_DEFAULT_MEMPOOL_OPS) And removing the redundant call to rte_mempool_set_ops_byname() in rte_pktm= buf_create_pool(). Thereafter, rte_pktmbuf_pool_create can be changed to: ... mp =3D rte_mempool_create_empty(name, n, elt_size, cache_size, - sizeof(struct rte_pktmbuf_pool_private), socket_id, 0); + sizeof(struct rte_pktmbuf_pool_private), socket_id, + MEMPOOL_F_PKT_ALLOC); if (mp =3D=3D NULL) return NULL; > > Is there any way I can achieve the above use case of multiple pools whi= ch > can be selected by an application - something like a run-time toggle/flag= ? > > > > - > > Shreyansh >=20 > Yes, you can create multiple pools, some using the default handlers, and > some using external handlers. > There is an example of this in the autotests (app/test/test_mempool.c). > This test creates multiple > mempools, of which one is a custom malloc based mempool handler. The > test puts and gets mbufs > to/from each mempool all in the same application. Thanks for the explanation. >=20 > Regards, > Dave. >=20 - Shreyansh