From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 845F51B5E9 for ; Mon, 26 Nov 2018 13:46:15 +0100 (CET) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20181126124614euoutp0269025dd8465d5b71b00d40f08fb1cbad~qrU5gAW530616206162euoutp02S for ; Mon, 26 Nov 2018 12:46:14 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20181126124614euoutp0269025dd8465d5b71b00d40f08fb1cbad~qrU5gAW530616206162euoutp02S DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1543236374; bh=TGi0FkgTx2ahqvvJyiKLAfWawpTzIpOs8rXksB6Iuok=; h=Subject:From:To:Date:In-Reply-To:References:From; b=vEahiGm+J7Gm+/LILu/rKyHWpEwAjrQ0yBjTtJ6xsluYzjMirpZQ9l4amGQ9+SFeu tc6D7ji4b/H7E2Jc0Fowb03fW+u0qwNPc7+tOiC/ytAEqZdbQ3vnklsbPoyKXm2m4u wKBRjNfFrJL8nzrx308i9oh4Fhh9tUoz22q7YamA= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20181126124613eucas1p235883f39ae9be9abac89483562f1e54c~qrU5D2hkP1248712487eucas1p27; Mon, 26 Nov 2018 12:46:13 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id B5.FA.04441.51BEBFB5; Mon, 26 Nov 2018 12:46:13 +0000 (GMT) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20181126124612eucas1p2190d344005d9c4922d1578968b749a34~qrU4HrxHE2971329713eucas1p2G; Mon, 26 Nov 2018 12:46:12 +0000 (GMT) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20181126124612eusmtrp1c4cee67d14c209d0254cf24ffc66600a~qrU35XDzX1500115001eusmtrp1t; Mon, 26 Nov 2018 12:46:12 +0000 (GMT) X-AuditID: cbfec7f2-5e3ff70000001159-ad-5bfbeb153da7 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 33.39.04284.41BEBFB5; Mon, 26 Nov 2018 12:46:12 +0000 (GMT) Received: from [106.109.129.180] (unknown [106.109.129.180]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20181126124612eusmtip2125e06d65874846a53b5025883b84cb8~qrU3mkjhH2964329643eusmtip20; Mon, 26 Nov 2018 12:46:12 +0000 (GMT) From: Ilya Maximets To: dev@dpdk.org, Asaf Sinai , Anatoly Burakov Message-ID: <96769dd6-9583-97f4-e29e-1f9105bec306@samsung.com> Date: Mon, 26 Nov 2018 15:46:11 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprBKsWRmVeSWpSXmKPExsWy7djP87qir39HG/SsVrd4dG8xs0XjgeUs Fu8+bWeyuNL+k92BxePXgqWsHov3vGTyuP3xPKtH35ZVjAEsUVw2Kak5mWWpRfp2CVwZPbsv sBVcEqmYefUYSwPjQsEuRk4OCQETiWNfFjJ2MXJxCAmsYJTY93MulPOFUeLN8WdsEM5noMyK p0wwLesn3GaBSCxnlLg0dS+U85FR4uXsF+wgVcIC6RLbm4+CdbAJ6EicWn2EEcQWEUiQOH+7 iQ3E5hWwk/g9/xVYnEVAVWJf4yewXlGBCImO+6uhagQlTs58wgJicwrYS2y83AtmMwuISzR9 WckKYctLbH87hxnkCAmBbnaJxoXnoE51kdh99CwbhC0s8er4FnYIW0bi9OQeFgi7XuJ+y0tG iOYORonph/5BNdtLbHl9DqiBA2iDpsT6XfoQYUeJjysfsoCEJQT4JG68FYS4gU9i0rbpzBBh XomONiGIahWJ3weXM0PYUhI3332GusBD4veW36wTGBVnIflyFpLPZiH5bBbCDQsYWVYxiqeW FuempxYb5qWW6xUn5haX5qXrJefnbmIEppfT/45/2sH49VLSIUYBDkYlHt4Jv35FC7EmlhVX 5h5ilOBgVhLh9V3yO1qINyWxsiq1KD++qDQntfgQozQHi5I4bzXDg2ghgfTEktTs1NSC1CKY LBMHp1QDo8pfgVWqJxiKX+vcun9rQsaOL6mVT+ZOvv3RfV11akRo82W1NT/XsCwtM21U+S2q 8uGMjSpnLo+t67He+ccYxGRDXtqeNFd5f/n7IVE7/VV9Gr2tESyv677H+zlNmnBp8nZWy60H Ilknapv/PVXmr3Ks02Ke1MNzu2U/PUmUzbnIWvWmO2oToxJLcUaioRZzUXEiANFrIdgrAwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprPIsWRmVeSWpSXmKPExsVy+t/xe7oir39HG6z4rG3x6N5iZovGA8tZ LN592s5kcaX9J7sDi8evBUtZPRbvecnkcfvjeVaPvi2rGANYovRsivJLS1IVMvKLS2yVog0t jPQMLS30jEws9QyNzWOtjEyV9O1sUlJzMstSi/TtEvQyenZfYCu4JFIx8+oxlgbGhYJdjJwc EgImEusn3GbpYuTiEBJYyijxadd0ZoiElMSPXxdYIWxhiT/Xutggit4zSjzoncECkhAWSJf4 t+YoWBGbgI7EqdVHGEFsEYEEiZtvetghGl4xSnx+dxxsKq+AncTv+a/AilgEVCX2NX5iB7FF BSIkzr5cxwhRIyhxcuYTsAWcAvYSGy/3gtnMAuoSf+ZdYoawxSWavqxkhbDlJba/ncM8gVFw FpL2WUhaZiFpmYWkZQEjyypGkdTS4tz03GJDveLE3OLSvHS95PzcTYzAyNl27OfmHYyXNgYf YhTgYFTi4X3x51e0EGtiWXFl7iFGCQ5mJRFe3yW/o4V4UxIrq1KL8uOLSnNSiw8xmgI9N5FZ SjQ5HxjVeSXxhqaG5haWhubG5sZmFkrivOcNKqOEBNITS1KzU1MLUotg+pg4OKUaGP2SJn5i y1p485TNboMOZd3GO958lU8T3m9Rmu1wu4ttdeDOG4+kxB/J/SuUlznLvHyF4I141uDtVzc+ 7lzY+0xG90du8AT3sGU6XZsDVW8tLLvL9mb3qycCD29J3DZZcn1NVaxn08an7Cbr1t4oUVG7 d3S26Nbp2nIL/7is4Tq/TqRW5fDpmSuVWIozEg21mIuKEwF5d3ljsgIAAA== X-CMS-MailID: 20181126124612eucas1p2190d344005d9c4922d1578968b749a34 X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20181126122321eucas1p1c8bfe7e1b74fc5cd71eec3a3c8929f5d X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20181126122321eucas1p1c8bfe7e1b74fc5cd71eec3a3c8929f5d References: Subject: Re: [dpdk-dev] CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES: no difference in memory pool allocations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Nov 2018 12:46:15 -0000 On 26.11.2018 15:23, Ilya Maximets wrote: >> Hi, >> >> We have 2 NUMAs in our system, and we try to allocate a single DPDK memory pool on each NUMA. >> However, we see no difference when enabling/disabling "CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES" configuration. >> We expected that disabling it will allocate pools only on one NUMA (probably NUMA0), but it actually allocates pools on both NUMAs, according to "socket_id" parameter passed to "rte_mempool_create" API. >> We have 192GB memory, so NUMA1 memory starts from address: 0x1800000000. >> As you can see below, "undDpdkPoolNameSocket_1" was indeed allocated on NUMA1, as we wanted, although "CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES" is disabled: >> >> CONFIG_RTE_LIBRTE_VHOST_NUMA=n >> CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n >> >> created poolName=undDpdkPoolNameSocket_0, nbufs=887808, bufferSize=2432, total=2059MB >> (memZone: name=MP_undDpdkPoolNameSocket_0, socket_id=0, vaddr=0x1f2c0427d00-0x1f2c05abe00, paddr=0x178e627d00-0x178e7abe00, len=1589504, hugepage_sz=2MB) >> created poolName=undDpdkPoolNameSocket_1, nbufs=887808, bufferSize=2432, total=2059MB >> (memZone: name=MP_undDpdkPoolNameSocket_1, socket_id=1, vaddr=0x1f57fa7be40-0x1f57fbfff40, paddr=0x2f8247be40-0x2f825fff40, len=1589504, hugepage_sz=2MB) >> >> Does anyone know what is "CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES" configuration used for? >> > > This config option was introduced to force DPDK to allocate memory > from NUMA nodes that was requested by 'socket_id'. That is exactly > what you're observing. I meant '--socket-mem' (not the 'socket_id'), of course. > > Look at the commit 1b72605d2416 ("mem: balanced allocation of hugepages") > for the original issue fixed by this option. > > ---- > >> Hi Asaf, >> >> I cannot reproduce this behavior. Just tried running testpmd with DPDK >> 18.08 as well as latest master [1], and DPDK could not successfully >> allocate a mempool on socket 1. > > I think that this is a bug. Because, if option enabled, you should successfully > allocate the memory from the requested NUMA if it's available. Anyway, we can't guarantee the failure here because if option disabled the kernel will decide from which NUMA to allocate memory. And it could eventually allocate it from the requested one. And I guess, the meaning of this option was a bit changed with a new memory model. > > If option disabled, we just requesting pages from the kernel and it could > return them from any NUMA node. With option enabled, we're trying to > force kernel to allocate from the nodes we need. > > Best regards, Ilya Maximets.