From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 3EED2952; Mon, 6 Mar 2017 10:34:47 +0100 (CET) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout2.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0OMD00MQRZXXBL10@mailout2.w1.samsung.com>; Mon, 06 Mar 2017 09:34:45 +0000 (GMT) Received: from eusmges1.samsung.com (unknown [203.254.199.239]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170306093445eucas1p1ad68671acc8826325f02a5a1fa45876e~pQT2_Dj6m0967409674eucas1p1Z; Mon, 6 Mar 2017 09:34:45 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1.samsung.com (EUCPMTA) with SMTP id F5.82.16908.73D2DB85; Mon, 6 Mar 2017 09:34:47 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170306093444eucas1p2b4f88cb91fb5fa6d001f84d2215ca2dd~pQT2IKfh61252512525eucas1p2u; Mon, 6 Mar 2017 09:34:44 +0000 (GMT) X-AuditID: cbfec7ef-f79d26d00000420c-7e-58bd2d372630 Received: from eusync4.samsung.com ( [203.254.199.214]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 9C.36.10233.E3D2DB85; Mon, 6 Mar 2017 09:34:54 +0000 (GMT) Received: from [106.109.129.180] by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OMD00JJSZXUP290@eusync4.samsung.com>; Mon, 06 Mar 2017 09:34:44 +0000 (GMT) To: dev@dpdk.org, David Marchand , Sergio Gonzalez Monroy Cc: Heetae Ahn , Yuanhan Liu , Jianfeng Tan , Neil Horman , Yulong Pei , stable@dpdk.org, Thomas Monjalon , Bruce Richardson From: Ilya Maximets Message-id: <50517d4c-5174-f4b2-e77e-143f7aac2c00@samsung.com> Date: Mon, 06 Mar 2017 12:34:42 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-version: 1.0 In-reply-to: <1487250070-13973-1-git-send-email-i.maximets@samsung.com> Content-type: text/plain; charset=windows-1252 Content-transfer-encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA02SbUhTYRTHee69u/e6nNzmsoNawshCUVMovIRoQdA+WZmk+MVWXqb4yq6K K4jJyNdS0XAy0xbzdUYrHTpIwaYlak3TRZpLohBfILO01ArJeTX89jvn+Z/zf/4PD41LLSJf Oi0rl1NnKTPkpJjofrU5FhoZ2pcY3mYLYafMMWxbSTHBLv/owdja1RmKdRZvUmx5/RrJftAN k2xb1SBit0r+Uuxap55k31eNi9hfn9bxs56Kt5UupPhtbBYpTL2LmKJx+LKiwmpGCud6B7pE JomjUriMtHxOfTL6mjh12rRC5rz2KnhptFFapD9QhjxoYE7Bn/k7SGAfGJ+1kGVITEuZFgTF 0/2YUKwiqCxcoPYmLBuj2H9VxcNlkVAsIHjUMIG5Vd7MGegerdhhGVMA2tIOws04Y8PAZZW7 mWRCYKRjcNubpiVMNDx3RrrbBBMITyqLduSHmESwTd0l3SxhDsJGzexO34O5AJ9rrJSwMhzq 7g9gAgdA1+OvuPs+wDgocM2WU+79wByBzn5cCHAeflodhMDesDRk3Q3mD5M15YQwq0OgNU8g oahCsKZrxQRVDIxMv9t184Lqbj0uGEigpEgqSBRgf9pMCnwOvozdo4QHMiConhzCq1CAYV8g w74Qhn0hjAg3IxmXx2eqOD4ijFdm8nlZqrAb2ZmdaPsbjW4NfbOhOV28HTE0kntKSq73JkpF ynxek2lHQONymeSFdbslSVFqbnLq7GR1XgbH25EfTcgPS/qMzgQpo1Lmcukcl8Op904x2sNX i1QzgfUf5xvwJCo5qNra3pY3kbXiwwU4amOj+LFn37eaOMti0/Tpnjmf4Li4xmFj2jJ7dCDn lpjWn+gKue0TP9GOpftnmzRXtGb1LLucI6sjNFd1CeEPdIWlQceXWn1d4jdJBbqplg1nxjHn Wqy338X6/Nxsg97Y7TDFrjfJCT5VGRGMq3nlP0SQPIpCAwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrFIsWRmVeSWpSXmKPExsVy+t/xa7p2unsjDLbNFrG4screYkVHO4vF u0/bmSymfb7NbnGl/Se7RffsL2wWt5pPslmsmHCE0eJfxx92iy+bprNZXJ9wgdXi24PvzA48 Hhf77zB6/FqwlNVj8Z6XTB7zTgZ69G1Zxehx5ftqxgC2KDebjNTElNQihdS85PyUzLx0W6XQ EDddCyWFvMTcVFulCF3fkCAlhbLEnFIgz8gADTg4B7gHK+nbJbhl3Fz8ga3gDF/F0QU72BsY p3N3MXJySAiYSKz/cZoJwhaTuHBvPRuILSSwhFHi80VuCPsFo8Tkv9ogtrCAlcS2031g9SIC FRLTXq8EqucCqpnDKNH3qp0VJMEssItJYsvVFBCbTUBH4tTqI4xdjBwcvAJ2EruvmIOEWQRU Jdb1t7GA2KICERLzn64Cm8krICjxY/I9sDingLvEo8lb2EFamQX0JO5f1IKYLi+xec1b5gmM ArOQdMxCqJqFpGoBI/MqRpHU0uLc9NxiI73ixNzi0rx0veT83E2MwHjcduznlh2MXe+CDzEK cDAq8fA+SNkTIcSaWFZcmXuIUYKDWUmE9+AWoBBvSmJlVWpRfnxRaU5q8SFGU6AXJjJLiSbn A1NFXkm8oYmhuaWhkbGFhbmRkZI479QPV8KFBNITS1KzU1MLUotg+pg4OKUaGKV1FpyJlJCO Y4i6NXPFrCAFq+Zv3fObtuZP8letXqgUlLl0/cW/2q+mvpytnj5hMbNlTVjhOhfxgB02rHLu ydXzgn4aFb5c5b+0Sf7VBKYctpwaQ42b89Pe8RsZZgo+ifv84CPHlliXvrwdfZ7cey6JrX12 15a156ikbGCXmixvkaaHZNk9JZbijERDLeai4kQAJr+u/d0CAAA= X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170306093444eucas1p2b4f88cb91fb5fa6d001f84d2215ca2dd X-Msg-Generator: CA X-Sender-IP: 182.198.249.180 X-Local-Sender: =?UTF-8?B?SWx5YSBNYXhpbWV0cxtTUlItVmlydHVhbGl6YXRpb24gTGFi?= =?UTF-8?B?G+yCvOyEseyghOyekBtMZWFkaW5nIEVuZ2luZWVy?= X-Global-Sender: =?UTF-8?B?SWx5YSBNYXhpbWV0cxtTUlItVmlydHVhbGl6YXRpb24gTGFi?= =?UTF-8?B?G1NhbXN1bmcgRWxlY3Ryb25pY3MbTGVhZGluZyBFbmdpbmVlcg==?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170216130139eucas1p2512567d6f5db9eaac5ee840b56bf920a X-RootMTR: 20170216130139eucas1p2512567d6f5db9eaac5ee840b56bf920a References: <1487250070-13973-1-git-send-email-i.maximets@samsung.com> Subject: Re: [dpdk-dev] [PATCH] mem: balanced allocation of hugepages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Mar 2017 09:34:47 -0000 Hi all. So, what about this change? Best regards, Ilya Maximets. On 16.02.2017 16:01, Ilya Maximets wrote: > Currently EAL allocates hugepages one by one not paying > attention from which NUMA node allocation was done. > > Such behaviour leads to allocation failure if number of > available hugepages for application limited by cgroups > or hugetlbfs and memory requested not only from the first > socket. > > Example: > # 90 x 1GB hugepages availavle in a system > > cgcreate -g hugetlb:/test > # Limit to 32GB of hugepages > cgset -r hugetlb.1GB.limit_in_bytes=34359738368 test > # Request 4GB from each of 2 sockets > cgexec -g hugetlb:test testpmd --socket-mem=4096,4096 ... > > EAL: SIGBUS: Cannot mmap more hugepages of size 1024 MB > EAL: 32 not 90 hugepages of size 1024 MB allocated > EAL: Not enough memory available on socket 1! > Requested: 4096MB, available: 0MB > PANIC in rte_eal_init(): > Cannot init memory > > This happens beacause all allocated pages are > on socket 0. > > Fix this issue by setting mempolicy MPOL_PREFERRED for each > hugepage to one of requested nodes in a round-robin fashion. > In this case all allocated pages will be fairly distributed > between all requested nodes. > > New config option RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES > introduced and disabled by default because of external > dependency from libnuma. > > Cc: > Fixes: 77988fc08dc5 ("mem: fix allocating all free hugepages") > > Signed-off-by: Ilya Maximets > --- > config/common_base | 1 + > lib/librte_eal/Makefile | 4 ++ > lib/librte_eal/linuxapp/eal/eal_memory.c | 66 ++++++++++++++++++++++++++++++++ > mk/rte.app.mk | 3 ++ > 4 files changed, 74 insertions(+) >