From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id E3DD832A5 for ; Fri, 26 Feb 2016 13:39:18 +0100 (CET) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga103.fm.intel.com with ESMTP; 26 Feb 2016 04:39:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.22,498,1449561600"; d="scan'208";a="753723995" Received: from irsmsx151.ger.corp.intel.com ([163.33.192.59]) by orsmga003.jf.intel.com with ESMTP; 26 Feb 2016 04:39:17 -0800 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.237]) by IRSMSX151.ger.corp.intel.com ([169.254.4.72]) with mapi id 14.03.0248.002; Fri, 26 Feb 2016 12:39:07 +0000 From: "Ananyev, Konstantin" To: "Rapelly, Varun" , "dev@dpdk.org" Thread-Topic: ACL memory allocation failures Thread-Index: AdFwf4kdB1whTVNNQG6hirscCjj/gAAANPTAAADEVXAAAf5KEAABgkhQ Date: Fri, 26 Feb 2016 12:39:06 +0000 Message-ID: <2601191342CEEE43887BDE71AB97725836B0BADD@irsmsx105.ger.corp.intel.com> References: <2601191342CEEE43887BDE71AB97725836B0BA25@irsmsx105.ger.corp.intel.com> In-Reply-To: Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiMWM3YmY0OTktMzE1YS00ZjFkLWEzZTEtNjVhZGUxYjc3ZDBmIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE1LjkuNi42IiwiVHJ1c3RlZExhYmVsSGFzaCI6IitQQm1nSnlSeUdTYVVERDlDTWZDWkxZaU40SGpQNUhIN1pBTHNtTW1lTlU9In0= x-ctpclassification: CTP_IC x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] ACL memory allocation failures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2016 12:39:19 -0000 >=20 > Thanks Konstantin. >=20 > Previous allocation error was coming with 1024 huge pages of 2 MB size. >=20 > After increasing the huge pages to 2048, I was able to add another ~140 r= ules [IPv4 rule data--> with src, dst IP address & port, next > header ] more, ie., 950 rules were added. That's strange according to your log, all you need is ~13MB of hugepage mem= ory: ACL: allocation of 12966528 bytes on socket 0 for ipv4_acl_table1 Wonder what consumed rest of 4GB? Again do you re-build your table after every rule you add? If so, then it seems a bit strange approach (and definitely not the fastest= one). What you can do instead: create context; add all your rules into it; build;= =20 >=20 > Logically it did not increase number of rules [expected 2*817, but only 9= 50 were added]. Is it really using huge pages memory only? >=20 > From the code it looks like heap memory. [ ret =3D malloc_heap_alloc(&mcf= g->malloc_heaps[i], type, size, 0, align =3D=3D 0 ? 1 : align, 0) ] As I can see from the log it fails at GEN phase, when trying to allocate hu= gepages for RT table. At lib/librte_acl/acl_gen.c:509 rte_acl_gen(struct rte_acl_ctx *ctx, struct rte_acl_trie *trie, struct rte_acl_bld_trie *node_bld_trie, uint32_t num_tries, uint32_t num_categories, uint32_t data_index_sz, size_t max_size) { ... mem =3D rte_zmalloc_socket(ctx->name, total_size, RTE_CACHE_LINE_SIZE, ctx->socket_id); if (mem =3D=3D NULL) { RTE_LOG(ERR, ACL, "allocation of %zu bytes on socket %d for %s failed= \n", total_size, ctx->socket_id, ctx->name); return -ENOMEM; } Konstantin >=20 > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Rapelly, Varun > > Sent: Friday, February 26, 2016 10:28 AM > > To: dev@dpdk.org > > Subject: Re: [dpdk-dev] ACL memory allocation failures > > > > Hi All, > > > > When I'm trying to configure some 5000+ ACL rules with different > > source IP addresses, getting ACL memory allocation failure. I'm using D= PDK 2.1. > > > > [root@ACLISSUE log_2015_10_26_08_19_42]# vim np.log match nodes/bytes > > used: 816/104448 > > total: 12940832 bytes > > ACL: Build phase for ACL "ipv4_acl_table2": > > memory consumed: 947913495 > > ACL: trie 0: number of rules: 816 > > ACL: allocation of 12966528 bytes on socket 0 for ipv4_acl_table1 > > failed > > ACL: Build phase for ACL "ipv4_acl_table1": > > memory consumed: 947913495 > > ACL: trie 0: number of rules: 817 > > EAL: Error - exiting with code: 1 > > Cause: Failed to build ACL trie > > > > Again sourced the ACL config file. After adding around 77 again the sam= e error came. > > > > total: 14912784 bytes > > ACL: Build phase for ACL "ipv4_acl_table1": > > memory consumed: 1040188260 > > ACL: trie 0: number of rules: 893 > > ACL: allocation of 14938480 bytes on socket 0 for ipv4_acl_table2 > > failed >=20 > You are running out of hugepages memory. >=20 > > ACL: Build phase for ACL "ipv4_acl_table2": > > memory consumed: 1040188260 > > ACL: trie 0: number of rules: 894 > > EAL: Error - exiting with code: 1 > > Cause: Failed to build ACL trie > > > > Where to increase the memory to avoid this issue? >=20 > Refer to: > http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html#running-dpdk-applicati= ons > Section 2.3.2 >=20 > Konstantin