From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pd0-f172.google.com (mail-pd0-f172.google.com [209.85.192.172]) by dpdk.org (Postfix) with ESMTP id 200957E62 for ; Fri, 24 Oct 2014 14:58:24 +0200 (CEST) Received: by mail-pd0-f172.google.com with SMTP id r10so1449635pdi.3 for ; Fri, 24 Oct 2014 06:06:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=+JvTGd66NVJb/Jvt5u40X6y77BHFxnayP0s5rGivxEs=; b=Vz0VM+aGvgZ6dCyUcuaRtFKAMru/isx+DADry8XsK5745kyni1IrE2UIYtNvwnywo1 tn97fLBLQxHJObuqRUE7Tnr8O5sXHxVsWCiD3Ut3DoMq9hLHwIEXMwk+kykbzwfAW8xt Jnw9Uu6ilyOLMeCip0jmUN/X8sx6TAV783nuigM1CcJDQFfCN4LDxvRrKGk4gbrBt6yV OBdSWJwGpIhKPmeEk/hS/u974XDQEgC8tkuA+yNEsXibkVmbCfFSaNiIoYSPktHeGMyu 7NEHt232Hct0IolAHQMQ2YP4cnYKQIPgaCgP7DHihDlHrnAL4V1Yr66Gj/m86uIx7bo0 dtWQ== X-Gm-Message-State: ALoCoQniuAIQ2BNEUqTkYRw8G+JPk2KhlQ7gsJ1Bz97Rf55JdzTuSkG0Ti2M+5o9HlKx48dscbpu X-Received: by 10.66.140.76 with SMTP id re12mr1494321pab.147.1414156013463; Fri, 24 Oct 2014 06:06:53 -0700 (PDT) Received: from uryu.home.lan ([14.141.84.68]) by mx.google.com with ESMTPSA id w15sm3880435pbt.59.2014.10.24.06.06.51 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Fri, 24 Oct 2014 06:06:53 -0700 (PDT) Date: Fri, 24 Oct 2014 18:36:29 +0530 From: Stephen Hemminger To: Matthew Hall Message-ID: <20141024183629.0740fd72@uryu.home.lan> In-Reply-To: <20141006091344.GA14759@mhcomputing.net> References: <20141006091344.GA14759@mhcomputing.net> X-Mailer: Claws Mail 3.10.1 (GTK+ 2.24.10; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Possible bug in eal_pci pci_scan_one X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Oct 2014 12:58:26 -0000 On Mon, 6 Oct 2014 02:13:44 -0700 Matthew Hall wrote: > Hi Guys, > > I'm doing my development on kind of a cheap machine with no NUMA support... > but several years ago I used DPDK to build a NUMA box that could do 40 gbits > bidirectional L4-L7 stateful traffic replay. > > So given the past experiences I had before, I wanted to clean the code up so > it'd work well if some crazy guy tried my code on one of these huge boxes, > too, but then I ran into some weird issues. > > 1) When I call rte_eth_dev_socket_id() I get back -1. But the call can return > -1 if the port_id is bogus or if pci_scan_one didn't get a numa_node (because > you're on a non-NUMA box for example). > > int rte_eth_dev_socket_id(uint8_t port_id) > { > if (port_id >= nb_ports) > return -1; > return rte_eth_devices[port_id].pci_dev->numa_node; > } > > So you couldn't tell the different between non-NUMA or a bad port value, etc. > > 2) The code's behavior and comments disagree with one another. In the > pci_scan_one function, there's this code: > > /* get numa node */ > snprintf(filename, sizeof(filename), "%s/numa_node", > dirname); > if (access(filename, R_OK) != 0) { > /* if no NUMA support just set node to 0 */ > dev->numa_node = -1; > } else { > if (eal_parse_sysfs_value(filename, &tmp) < 0) { > free(dev); > return -1; > } > dev->numa_node = tmp; > } > > It says, just use NUMA node 0 if there is no NUMA support. But then proceeds > to set the value to -1 in disagreement with the comment, and also stomping on > the other meaning for -1 in the higher function rte_eth_dev_socket_id. > > 3) In conclusion, it seems like some stuff is missing... first there needs to > be a function that will tell you the number of NUMA nodes present on the box > so you can create the right number of mbuf_pools, but I couldn't find that function. > > Then if you have the function, you can do some magic and shuffle the NICs > around to get them hooked to a core on the same NUMA, and the mbuf_pool on the > same NUMA. > > When NUMA is not present, can we return 0 instead of -1, or return a specific > error code that the client can use to know he should just use Socket 0? Right > now I can't tell apart any potential errors or weird values from correct > values. > > 4) I'm willing to help make and test some patches... but first I want to > understand what is happening with these funny functions before doing things > blindly. > > Thanks, > Matthew. The code is fairly consistent in returning -1 for cases of not a NUMA socket, bogus port value. It is interpreted as SOCKET_ID_ANY in several places. The examples mostly check for -1 and use socket 0 as a fallback. Probably not worth introducing more return values and breaking existing applications.