From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from demumfd001.nsn-inter.net (demumfd001.nsn-inter.net [93.183.12.32]) by dpdk.org (Postfix) with ESMTP id 4BCCD231C for ; Fri, 30 May 2014 13:45:10 +0200 (CEST) Received: from demuprx016.emea.nsn-intra.net ([10.150.129.55]) by demumfd001.nsn-inter.net (8.14.3/8.14.3) with ESMTP id s4UBjLUA029294 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Fri, 30 May 2014 11:45:21 GMT Received: from localhost.localdomain ([10.154.193.102]) by demuprx016.emea.nsn-intra.net (8.12.11.20060308/8.12.11) with ESMTP id s4UBjLtT023216 for ; Fri, 30 May 2014 13:45:21 +0200 Message-ID: <53886F51.6060807@nsn.com> Date: Fri, 30 May 2014 14:45:21 +0300 From: Kypriotis Angelos Organization: Nokia Solutions and Networks User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130805 Thunderbird/17.0.8 MIME-Version: 1.0 To: dev@dpdk.org X-purgate-type: clean X-purgate-Ad: Categorized by eleven eXpurgate (R) http://www.eleven.de X-purgate: clean X-purgate: This mail is considered clean (visit http://www.eleven.de for further information) X-purgate-size: 7610 X-purgate-ID: 151667::1401450322-00001326-5E6D6CEB/0/0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] buffer allocation failure in NUMA platform X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 11:45:10 -0000 Hi, Here is my topology. ************************************************************* NUMA Topology ************************************************************* NUMA domains: 2 ------------------------------------------------------------- Domain 0: Processors: 0 1 2 3 8 9 10 11 Relative distance to nodes: 10 21 Memory: 5815.68 MB free of total 5977.69 MB ------------------------------------------------------------- Domain 1: Processors: 4 5 6 7 12 13 14 15 Relative distance to nodes: 21 10 Memory: 4893.15 MB free of total 6058.52 MB I use dpdk-1.6.0r1 and I have setup 1024 pages of 2M per NUMA node, as early as possible ( boot time ) using the hugepages=2048 parameter. cat /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages --> 1024 cat /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages --> 1024 When I try to pin my process to specific nodes like this : numactl --physcpubind=12-15 -- ./my_simulator -c f000 -n 2 -b 0000:01:00.0 --socket-mem=0,2048 -- my_simulator.conf , I get 'RING:cannot reserve memory' error. Here is the relevant part : EAL: Setting up memory... EAL: Ask a virtual area of 0x1200000 bytes EAL: Virtual area found at 0x7f217a600000 (size = 0x1200000) EAL: Ask a virtual area of 0x3ec00000 bytes EAL: Virtual area found at 0x7f213b800000 (size = 0x3ec00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f213b400000 (size = 0x200000) EAL: Ask a virtual area of 0x3fc00000 bytes EAL: Virtual area found at 0x7f20fb600000 (size = 0x3fc00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f20fb200000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f20fae00000 (size = 0x200000) EAL: Ask a virtual area of 0x7fc00000 bytes EAL: Virtual area found at 0x7f207b000000 (size = 0x7fc00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f207ac00000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f207a800000 (size = 0x200000) EAL: Requesting 1024 pages of size 2MB from socket 1 EAL: TSC frequency is ~1600000 KHz EAL: Master core 12 is ready (tid=81c69840) EAL: Core 13 is ready (tid=7fd55700) EAL: Core 15 is ready (tid=7ed53700) EAL: Core 14 is ready (tid=7f554700) RING: Cannot reserve memory Failed to allocate packet buffer pool. If I do not use --socket-mem=0,2048 memory ( 1024 pages ) is requested from both sockets 0 and 1 and it works, but I want to use node 1 only. In addition in order to request for 1024 pages in node 1 I must pass the --socket-mem parameter as above. I would expect that what I should pass is --socket-mem=0,1024, but then the actual memory requested is 512. Am I missing something here or is it a bug? Regards, -- Angel Kypriotis MBB LC EPC GW RD Athens PET GR Nokia 14 Km Athinon-Lamias National Road 145 64 N. Kifissia / Athens / Hellas