From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id EEB741B5CC for ; Mon, 26 Nov 2018 13:23:23 +0100 (CET) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20181126122322euoutp0155b47cdb69bfb331be5a300a379113e3~qrA73zdgo2489424894euoutp01Q for ; Mon, 26 Nov 2018 12:23:22 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20181126122322euoutp0155b47cdb69bfb331be5a300a379113e3~qrA73zdgo2489424894euoutp01Q DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1543235002; bh=APEqsLOfh8ihfVy1fNNyeZuBlv1xH7Yxm8hnC1hy4gg=; h=To:Subject:From:Date:In-Reply-To:References:From; b=VhTT9+5ToR6NjGexu2No8x4erJUdjBo6thXWA41VdGUnWSQEbdBHwnDKJbgBbgbQx Wx5OfXC1gcVkog0jPGdf/oQ76q/4/JASBSO/qL0TS6jrW+5McU36GQ1E96sqLT5uVG whXQ8LBgltx7jEG0Hqzn/LW2obCV6BjzSxSCew/U= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20181126122322eucas1p1be2ef934563dbffef29ff396f62fd0fa~qrA7elXLc0221902219eucas1p1L; Mon, 26 Nov 2018 12:23:22 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id 47.CC.04294.9B5EBFB5; Mon, 26 Nov 2018 12:23:21 +0000 (GMT) Received: from eusmtrp1.samsung.com (unknown [182.198.249.138]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20181126122321eucas1p1c8bfe7e1b74fc5cd71eec3a3c8929f5d~qrA6yB9KW0221902219eucas1p1K; Mon, 26 Nov 2018 12:23:21 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp1.samsung.com (KnoxPortal) with ESMTP id 20181126122321eusmtrp1e09436456765ecc27e1914471e49dda3~qrA6jZqis0800608006eusmtrp1W; Mon, 26 Nov 2018 12:23:21 +0000 (GMT) X-AuditID: cbfec7f4-835ff700000010c6-68-5bfbe5b90ca1 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 78.89.04128.9B5EBFB5; Mon, 26 Nov 2018 12:23:21 +0000 (GMT) Received: from [106.109.129.180] (unknown [106.109.129.180]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20181126122320eusmtip12e72031e0cee54ebb9ef0400853a0101~qrA6L6lcZ2298722987eusmtip1X; Mon, 26 Nov 2018 12:23:20 +0000 (GMT) To: dev@dpdk.org, Asaf Sinai , Anatoly Burakov From: Ilya Maximets Message-ID: Date: Mon, 26 Nov 2018 15:23:16 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA01Sa0hTYRju2zlzx8vs86jtxYpwlGSklkTsR2iB0P4YmgTiojzmaVpuyo53 K2/lTC1Eo+EoE1R0ajcxlyYslyW21D8ahqBZhqgpTYfgJpLbWeS/530ufM8DH0XQ9cIgKl2d zWrUTIbUw4vs/bQ5Ftb3y6E4MTAlk/2YaSZkpe/bSNnqmlEgm9Buis6ScntTq1DePLAokE9b x4Xyhz0dKI5M8jqTymak57KaiKhkrzRdy42sx375ixPPPErQoG8V8qQAn4KJkVFRFfKiaNyO YGvTQPCHDYGpo47kj3UE1bWL6F9Ea61GvNCGoML+R+AUaGxF8KBmrxMH4GQYny7zcGJ/rARj +UeXxwMfh8+dQzthihLjKDC0+zhpEh+B168WXJZAnAiVs52uqBj7wUjDPOm0e+IUMPEVCCyB MptByONDYFx5QvDVtCIYLRXxOAYmJ/tJHvvD0nCPmz8AlvoaN18Ms3cXXVMAVyLQmbcFvBAN PctjIue7BA6Fl/0RPH0OrIY5Vx3AvjC14sdX8IW6Xh3B02KorKB592FwDLa5mwXBt9V1dwM5 OHocwloUrN+1Ub9rmH7XMP3/Dk2I7EASNodTKVkuUs3mhXOMistRK8OvZaq60c5XsWwP296i /q0UM8IUkvqIa+12BS1kcrkClRkBRUgDxLEtDgUtTmUKCllN5lVNTgbLmdF+ipRKxEV7vito rGSy2Zssm8Vq/qkCyjOoBMmDvZPl1jvKsbiDX233uUl5Xiwdp2XnLOXpquh7lu6F1syZNF3R 2MViZd1Up2z20unCL41dCV0hL8Jr8sN+Jy5pl951DyWsnq/aXl6zrxzdMA03eps2ZIHW65L5 0JWQK/v6HW8aYp7nfYg3Xr5d67iQpLxlj6yLnw/7Gdz3iHkqJbk05uQxQsMxfwG/avv9JgMA AA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprNIsWRmVeSWpSXmKPExsVy+t/xu7o7n/6ONri/ns3i0b3FzBaNB5az WLz7tJ3J4kr7T3YHFo9fC5ayeize85LJ4/bH86wefVtWMQawROnZFOWXlqQqZOQXl9gqRRta GOkZWlroGZlY6hkam8daGZkq6dvZpKTmZJalFunbJehlTF+SVTBNsOLllflsDYwH+boYOTkk BEwk2j92M3YxcnEICSxllPh+spkRIiEl8ePXBVYIW1jiz7UuNoii94wSU18sYwdJiAgkSNx8 0wNmCwkkStxpescEYgsLpEv8W3MUrJlNQEfi1OojQEM5OHgF7CRWruABCbMIqEps3PAcrFxU IELi7Mt1YHt5BQQlTs58wgJSzimQJLH/JViYWUBd4s+8S8wQtrhE05eVrBC2vMT2t3OYJzAK zkLSPQtJyywkLbOQtCxgZFnFKJJaWpybnltspFecmFtcmpeul5yfu4kRGDHbjv3csoOx613w IUYBDkYlHt4Xf35FC7EmlhVX5h5ilOBgVhLh9V3yO1qINyWxsiq1KD++qDQntfgQoynQbxOZ pUST84HRnFcSb2hqaG5haWhubG5sZqEkznveoDJKSCA9sSQ1OzW1ILUIpo+Jg1OqgbFd5110 KOudzObzencW2S3v4rGpXrntVZy4L9uieE3Z16XXV3pfkfU+fP5DxrZAt/TGlvP7jvZ2ZpTd iTvYkeAye5KNpWelqFWB5o3zyvcak/WO/kpKDPdUdf5lOSU5VvWoxZzIj5kHtKWrjVV3lNp+ Ps4n/8f7m/VHlpIC30usIYkuSWlblViKMxINtZiLihMBYp9C1K4CAAA= X-CMS-MailID: 20181126122321eucas1p1c8bfe7e1b74fc5cd71eec3a3c8929f5d X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20181126122321eucas1p1c8bfe7e1b74fc5cd71eec3a3c8929f5d X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20181126122321eucas1p1c8bfe7e1b74fc5cd71eec3a3c8929f5d References: Subject: Re: [dpdk-dev] CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES: no difference in memory pool allocations X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Nov 2018 12:23:24 -0000 > Hi, > > We have 2 NUMAs in our system, and we try to allocate a single DPDK memory pool on each NUMA. > However, we see no difference when enabling/disabling "CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES" configuration. > We expected that disabling it will allocate pools only on one NUMA (probably NUMA0), but it actually allocates pools on both NUMAs, according to "socket_id" parameter passed to "rte_mempool_create" API. > We have 192GB memory, so NUMA1 memory starts from address: 0x1800000000. > As you can see below, "undDpdkPoolNameSocket_1" was indeed allocated on NUMA1, as we wanted, although "CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES" is disabled: > > CONFIG_RTE_LIBRTE_VHOST_NUMA=n > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n > > created poolName=undDpdkPoolNameSocket_0, nbufs=887808, bufferSize=2432, total=2059MB > (memZone: name=MP_undDpdkPoolNameSocket_0, socket_id=0, vaddr=0x1f2c0427d00-0x1f2c05abe00, paddr=0x178e627d00-0x178e7abe00, len=1589504, hugepage_sz=2MB) > created poolName=undDpdkPoolNameSocket_1, nbufs=887808, bufferSize=2432, total=2059MB > (memZone: name=MP_undDpdkPoolNameSocket_1, socket_id=1, vaddr=0x1f57fa7be40-0x1f57fbfff40, paddr=0x2f8247be40-0x2f825fff40, len=1589504, hugepage_sz=2MB) > > Does anyone know what is "CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES" configuration used for? > This config option was introduced to force DPDK to allocate memory from NUMA nodes that was requested by 'socket_id'. That is exactly what you're observing. Look at the commit 1b72605d2416 ("mem: balanced allocation of hugepages") for the original issue fixed by this option. ---- > Hi Asaf, > > I cannot reproduce this behavior. Just tried running testpmd with DPDK > 18.08 as well as latest master [1], and DPDK could not successfully > allocate a mempool on socket 1. I think that this is a bug. Because, if option enabled, you should successfully allocate the memory from the requested NUMA if it's available. If option disabled, we just requesting pages from the kernel and it could return them from any NUMA node. With option enabled, we're trying to force kernel to allocate from the nodes we need. Best regards, Ilya Maximets.