From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f175.google.com (mail-pd0-f175.google.com [209.85.192.175]) by dpdk.org (Postfix) with ESMTP id 1BC6330E for ; Fri, 31 May 2013 17:45:35 +0200 (CEST) Received: by mail-pd0-f175.google.com with SMTP id 4so407236pdd.20 for ; Fri, 31 May 2013 08:45:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type:content-transfer-encoding :x-gm-message-state; bh=O9w3uS0X5SS9T8t4LewVcf0sZQuNNk90ohABbL3faWw=; b=lit7gSJoRL/mOkSXj+H5Dj8zmPVv9jmSbqGfaxC3r8Fstdu7poBf84ddvWxnBgdixR 4s/klyIy54BuqyxEoch3yioXsedqEOc7IFiN8hjsfzlTClXlqm6b2p4YTyyjTSANrljT JpJFAbBJinbbREQPJGGgi9Z9C0zuqSh0YIpRkfNMPVgMS4F8L/n84YZ5oPLCpX/D4s9L wx80nK+UFu30NBdWMWbCmKcQa8v9QWzqrSE76HicAne/E8R3b3Az8IpyfDx1t3uHUWVD dfLqL5vq1O6P2o3Xq4r1IV4QnMuEwL6dBgrP5ys8OJpOgXcPAlh16cYfiaqgDYc78E6W MN+A== X-Received: by 10.66.150.168 with SMTP id uj8mr14288297pab.34.1370015142228; Fri, 31 May 2013 08:45:42 -0700 (PDT) Received: from nehalam.linuxnetplumber.net (static-50-53-71-109.bvtn.or.frontiernet.net. [50.53.71.109]) by mx.google.com with ESMTPSA id lq4sm50303626pab.19.2013.05.31.08.45.39 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Fri, 31 May 2013 08:45:40 -0700 (PDT) Date: Fri, 31 May 2013 08:45:29 -0700 From: Stephen Hemminger To: Damien Millescamps Message-ID: <20130531084529.5c63d4c0@nehalam.linuxnetplumber.net> In-Reply-To: <51A86D65.2090003@6wind.com> References: <20130530171234.301927271@vyatta.com> <51A86D65.2090003@6wind.com> X-Mailer: Claws Mail 3.8.1 (GTK+ 2.24.10; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Gm-Message-State: ALoCoQmFCI7eDuUiOoSaSnpzei2gLasa4GVyzJgZ6yByqoO5Ngg+poqw5LdgjUjA2tUPl0aKPY2u Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH 0/7] Vyatta patches X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 May 2013 15:45:36 -0000 On Fri, 31 May 2013 11:29:09 +0200 Damien Millescamps wrote: > On 05/30/2013 07:12 PM, Stephen Hemminger wrote: > > One fix not included is the NUMA cpu assignment. The original Intel code > > in 1.2 incorrectly used /proc/cpuinfo to try and assign CPU's to NUMA socket. > > The problem is that /proc/cpuinfo physical_id corresponds to what the BIOS > > tells the kernel and is intended for messages only. For example, on our > > Dell boxes the first CPU and only CPU is reported as physical_id 1! > > The fix is to use sysfs instead, > Hi Stephen, > > Are you using the > /sys/devices/system/node/nodeX/cpuX/topology/physical_package_id special > file ? > > If so, then it is only usable starting from Kernel 3.3 according to this > fix: > http://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/commit/?id=64be4c1c2428e148de6081af235e2418e6a66dda > > The value returned for kernel prior to 3.3 is really not better than the > one from /proc/cpuinfo, so that's kind of a robbing Peter to pay Paul > example... > > There is obviously a problem with the NUMA node ID detection right now > since both /proc/cpuinfo and /sys can return incorrect values, however > the node and cpu numbering is always good in the kernel boot log from > what I know. So there might be a better way to find the real node ID > whatever the kernel version used. > > Cheers, You need to use /sys/devices/system/cpu/cpuN/topology/physical_package_id as documented in kernel (Documentation/cputopology.txt). This was confirmed by several kernel developers including Andi Kleen from Intel. The value in /proc/cpuinfo comes from the APCI tables and is the value reported by the BIOS. There are machines that report socket 1 and 2. Haven't played with older kernels, but the topology information in sysfs has existed since 2.6.16. The final Intel solution was to use physical_packate_id and fall back to /proc/cpuinfo as last resort.