From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-CO1-obe.outbound.protection.outlook.com (mail-co1nam03on0080.outbound.protection.outlook.com [104.47.40.80]) by dpdk.org (Postfix) with ESMTP id 641FC2956 for ; Wed, 31 Aug 2016 10:38:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=harmonic.onmicrosoft.com; s=selector1-harmonicinc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=kTV6DhSO4IqbXr5J4DPonQrLfp3rmR45Rx9yGbAecxM=; b=MRoPjXQQe9JbArXkmb5k7ecfzFT0yGwTF1hVDeuVzWuE76WtQ3Jv8o6k9xddKVNdc630fy6vd5RNryKsE+qTcpfRXqLXY/VoAi715Xg4R/yFBQdh72OwDTNu3dqjULvCvyeIZ6385LtmGhOcCWFwim//rDMPY5ur/whLMUF2iMc= Received: from BN6PR11MB1347.namprd11.prod.outlook.com (10.173.32.147) by BN6PR11MB1346.namprd11.prod.outlook.com (10.173.32.146) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.599.9; Wed, 31 Aug 2016 08:38:05 +0000 Received: from BN6PR11MB1347.namprd11.prod.outlook.com ([10.173.32.147]) by BN6PR11MB1347.namprd11.prod.outlook.com ([10.173.32.147]) with mapi id 15.01.0587.009; Wed, 31 Aug 2016 08:38:06 +0000 From: Vladyslav Buslov To: "Ananyev, Konstantin" CC: "dev@dpdk.org" Thread-Topic: [PATCH] acl: use rte_calloc for temporary memory allocation Thread-Index: AQHR98bMloFFvwZovkmV/gPzRtdsvaBiVocggAB/W0A= Date: Wed, 31 Aug 2016 08:38:06 +0000 Message-ID: References: <20160816140128.10149-1-vladyslav.buslov@harmonicinc.com> <20160816140128.10149-2-vladyslav.buslov@harmonicinc.com> <2601191342CEEE43887BDE71AB97725836B94E0C@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB97725836B94E0C@irsmsx105.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Vladyslav.Buslov@harmonicinc.com; x-originating-ip: [95.67.66.62] x-ms-office365-filtering-correlation-id: dfe84739-8e26-45f3-b22b-08d3d17a219f x-microsoft-exchange-diagnostics: 1; BN6PR11MB1346; 20:Ainn0Q1TocF0XLJ1alJuFMJvYCuiEePGKTnUGtyiB8nEQEbDiYMVld9o3xu1/34ukwzjHckcdY49RNe2oL9HURtFOLSoSjPmOpvH2oGp052ihgEIdwvqO7/EHWzfQcOPniTZlB367psJEeGBxkeYdSboqwtPxPDnhgIMXMGJ0M0= x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:BN6PR11MB1346; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(228905959029699)(17755550239193); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6055026); SRVR:BN6PR11MB1346; BCL:0; PCL:0; RULEID:; SRVR:BN6PR11MB1346; x-forefront-prvs: 00514A2FE6 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(13464003)(377454003)(189002)(199003)(2900100001)(5002640100001)(189998001)(110136002)(5660300001)(7696003)(68736007)(101416001)(86362001)(3660700001)(7846002)(99286002)(3280700002)(7736002)(6116002)(305945005)(586003)(3846002)(77096005)(10400500002)(106116001)(97736004)(105586002)(50986999)(76576001)(19580405001)(19580395003)(76176999)(54356999)(4326007)(74316002)(81156014)(2906002)(2950100001)(8676002)(102836003)(92566002)(87936001)(33656002)(8936002)(122556002)(81166006)(9686002)(106356001)(66066001); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR11MB1346; H:BN6PR11MB1347.namprd11.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: harmonicinc.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: harmonicinc.com X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Aug 2016 08:38:06.5307 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 19294cf8-3352-4dde-be9e-7f47b9b6b73d X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1346 Subject: Re: [dpdk-dev] [PATCH] acl: use rte_calloc for temporary memory allocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Aug 2016 08:38:11 -0000 Hi Konstantin, Thanks for your feedback. Would you accept this change as config file compile-time parameter with lib= c calloc as default? It is one line change only so it is easy to ifdef. Regards, Vlad -----Original Message----- From: Ananyev, Konstantin [mailto:konstantin.ananyev@intel.com]=20 Sent: Wednesday, August 31, 2016 4:28 AM To: Vladyslav Buslov Cc: dev@dpdk.org Subject: RE: [PATCH] acl: use rte_calloc for temporary memory allocation Hi Vladyslav, > -----Original Message----- > From: Vladyslav Buslov [mailto:vladyslav.buslov@harmonicinc.com] > Sent: Tuesday, August 16, 2016 3:01 PM > To: Ananyev, Konstantin > Cc: dev@dpdk.org > Subject: [PATCH] acl: use rte_calloc for temporary memory allocation >=20 > Acl build process uses significant amount of memory which degrades=20 > performance by causing page walks when memory is allocated on regular hea= p using libc calloc. >=20 > This commit changes tb_mem to allocate temporary memory on huge pages wit= h rte_calloc. We deliberately used standard system memory allocation routines (calloc/fre= e) here. With current design build phase was never considered to be an 'RT' phase op= eration. It is pretty cpu and memory expensive. So if we'll use RTE memory for build phase it could easily consume all (or = most) of it, and might cause DPDK process failure or degradation. If you really feel that you (and other users) would benefit from using rte_= calloc here (I personally still in doubt), then at least it should be a new= field inside rte_acl_config, that would allow user to control that behavio= r. Though, as I said above, librte_acl was never designed to ' to allocate ten= s of thousands of ACLs at runtime'. To add ability to add/delete rules at runtime while keeping lookup time rea= sonably low some new approach need to be introduced. =20 Konstantin >=20 > Signed-off-by: Vladyslav Buslov > --- > lib/librte_acl/tb_mem.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) >=20 > diff --git a/lib/librte_acl/tb_mem.c b/lib/librte_acl/tb_mem.c index=20 > 157e608..c373673 100644 > --- a/lib/librte_acl/tb_mem.c > +++ b/lib/librte_acl/tb_mem.c > @@ -52,7 +52,7 @@ tb_pool(struct tb_mem_pool *pool, size_t sz) > size_t size; >=20 > size =3D sz + pool->alignment - 1; > - block =3D calloc(1, size + sizeof(*pool->block)); > + block =3D rte_calloc("ACL_TBMEM_BLOCK", 1, size +=20 > +sizeof(*pool->block), 0); > if (block =3D=3D NULL) { > RTE_LOG(ERR, MALLOC, "%s(%zu)\n failed, currently allocated " > "by pool: %zu bytes\n", __func__, sz, pool->alloc); > -- > 2.8.3