From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7EB64A09E4; Sat, 30 Jan 2021 00:09:26 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F04644069B; Sat, 30 Jan 2021 00:09:25 +0100 (CET) Received: from mx.quacker.org (mx.quacker.org [129.97.75.228]) by mails.dpdk.org (Postfix) with ESMTP id CC38340395 for ; Fri, 29 Jan 2021 23:25:04 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 5DD1A326BB8A; Fri, 29 Jan 2021 17:24:48 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quacker.org; s=dkim; t=1611959092; h=from:subject:date:message-id:to:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=jXTJyTvPA71lUZT2JfPGn11ue58LH0WefkHcopZnyxE=; b=cGYLhmw3/3bAQe6zopCszdDbFTISlp/ygCR8qMNpxn5YyJmsaYaA0W9t5p73C3SeTPKBvC M0dgqR75wrk4EtuyxBCU1ZhxQcQiuhw1E+B6I1lGF37hbJmUYzzFNwzIMOfsqTDGe8ZELN 8VA6uk2nihSQXGucZ7yjcBpZDMFcJ47wU4NQZsfBGvmrpzmeN3QSvc3kbcpC8umFleFAP3 6PURk0RAiJWC3OlpCLXrPNG9prf/bGdIcG4I96YqJoHj4gmpl4+PZihIF6CQ+9bp7F1Jm2 vdIY0FigkmVrmejFgn+TDNo/Md7DE8X4Opd2aqzO4aqP1iUAsP/Vw6p+erM8sg== MIME-Version: 1.0 Date: Fri, 29 Jan 2021 22:24:48 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: "Oscar Zhao" Message-ID: To: "Bruce Richardson" , "Burakov, Anatoly" Cc: dev@dpdk.org In-Reply-To: <20210129134511.GF929@bricha3-MOBL.ger.corp.intel.com> References: <20210129134511.GF929@bricha3-MOBL.ger.corp.intel.com> <20210129100537.GB929@bricha3-MOBL.ger.corp.intel.com> <8b287cd6-7ea0-96e9-5a48-89e99d84b1f9@intel.com> X-Last-TLS-Session-Version: TLSv1.2 X-Mailman-Approved-At: Sat, 30 Jan 2021 00:09:24 +0100 Subject: Re: [dpdk-dev] Feature Discussion: CPU topology detection on FreeBSD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" FreeBSD rolled their own NUMA memory APIs on 12.0 (see https://www.freebs= d.org/cgi/man.cgi?query=3Dcpuset_getdomain&sektion=3D2&apropos=3D0&manpat= h=3DFreeBSD+12.2-RELEASE+and+Ports). These syscalls allow binding of a pr= ocess/thread/jail memory allocation to a specific NUMA domain.=0A=0AI thi= nk the first step is to support correct NUMA detection. The "de facto" wa= y of doing that is parsing the XML output from the cpu topology sysctl. I= 'm not sure how viable it is in DPDK without introducing external depende= ncies. I've also been playing with the hwloc library which supposedly off= ers portable topology detection.=0A=0AIf both options are not feasible ma= ybe something can be done on the FreeBSD side first.=0A=0AJanuary 29, 202= 1 8:45 AM, "Bruce Richardson" wrote:=0A=0A> = On Fri, Jan 29, 2021 at 12:52:50PM +0000, Burakov, Anatoly wrote:=0A> =0A= >> On 29-Jan-21 10:05 AM, Bruce Richardson wrote:=0A>> On Thu, Jan 28, 20= 21 at 04:07:07AM +0000, Oscar Zhao wrote:=0A>>> Hello dpdk devs,=0A>>> = =0A>>> I would like to discuss the feasibility of implementing CPU topolo= gy detection on FreeBSD.=0A>> Currently both eal_cpu_core_id() and eal_cp= u_socket_id() in rte_eal library always return 0 on=0A>> FreeBSD, making = NUMA-aware development impossible without resorting to external libraries= or system=0A>> APIs.=0A>>> =0A>>> The CPU topology information is availa= ble via sysct kern.sched.topology_spec. The only issue is=0A>> that the r= eturn value is an XML formatted object (see=0A>> https://forums.freebsd.o= rg/threads/number-of-cpus-and-cores.41299).=0A>> (https://forums.freebsd.= org/threads/number-of-cpus-and-cores.41299) I'm not sure how feasible it = is=0A>> to parse XML inside DPDK.=0A>>> =0A>>> On a side note, obtaining = the physical NUMA node id of a core is easy - sysctl=0A>> dev.cpu.[cpu#].= %domain returns the corresponding node id but I have yet found a straight= forward way=0A>> to detect hyperthreads besides the XML thing.=0A>>> =0A>= >> Oscar=0A>> =0A>> Even adding in the numa node information would be a w= elcome start.=0A>> =0A>> There's little value in it though, because as fa= r as i'm aware we can't=0A>> allocate memory on specific NUMA nodes in Fr= eeBSD.=0A> =0A> That is because the NUMA support in FreeBSD was not in pl= ace when we first=0A> did the DPDK port. However, I still see no reason n= ot to start adding it=0A> now, even if it is only for information purpose= s at this point.