From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f170.google.com (mail-io0-f170.google.com [209.85.223.170]) by dpdk.org (Postfix) with ESMTP id 3F0698E70 for ; Mon, 19 Oct 2015 09:51:29 +0200 (CEST) Received: by iofz202 with SMTP id z202so36909348iof.2 for ; Mon, 19 Oct 2015 00:51:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=rewZ/Mx6JsMMN8qHilVaJOcvo3OiHC91qOk7qFruJHc=; b=yAki2OeY6Z9G/eiUYmA4/mAJffhSP5Fwa7zXxwHY4Q1iUCC1KjZELUAsN3W2nGpNbF mDg3Z+DUYa7d+MoSKUm02dz+Fb34tsc4A1A7IWFnzewYEibR7ncgRSNohR5yqfB8OhwI TkepFv+J0GylQ0BLZUksFaeJkai6c7KquIJlUMJNYLVZQAe2g8P1XjFnxKaPtSyKwSSa 9oCVtPPBGBtgCyvpWz8lTQ7Exu67woQwUIA6/7mcsp3sHgx5YO8NbUhK01ISopOASKyO xpK6F6MvvBG0yrekeJkCEWYQ+12UV5rrXokWzKtF/tJFH6DjDiz/fCOCyP8srAgKZ+uo dEsw== MIME-Version: 1.0 X-Received: by 10.107.7.218 with SMTP id g87mr8765238ioi.7.1445241088635; Mon, 19 Oct 2015 00:51:28 -0700 (PDT) Received: by 10.79.33.206 with HTTP; Mon, 19 Oct 2015 00:51:28 -0700 (PDT) In-Reply-To: References: <20151016134320.GE9980@bricha3-MOBL3> Date: Mon, 19 Oct 2015 16:51:28 +0900 Message-ID: From: Moon-Sang Lee To: Bruce Richardson Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [Q] l2fwd in examples directory X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Oct 2015 07:51:29 -0000 Let me clarify my mixed stuffs. My processor is L5520, family 6, model 26 that is based on Nehalem microarchitecture according to wikipedia ( https://en.wikipedia.org/wiki/Nehalem_(microarchitecture)), it does not have PCI interface on chipset. Therefore, "rte_eth_dev_socket_id(portid) always returns -1" seems no problem. My understanding of the lstopo result might be wrong. Thanks anyway. On Mon, Oct 19, 2015 at 4:39 PM, Moon-Sang Lee wrote: > > My NUT has Xeon L5520 that is based on Nehalem microarchitecture. > Does Nehalem supports PCIe interface on chipset? > > Anyhow, 'lstopo' shows as below and it seems that my PCI devices are > connected to socket #0. > I'm still wondering why rte_eth_dev_socket_id(portid) always returns -1. > > mslee@myhost:~$ lstopo > Machine (31GB) > NUMANode L#0 (P#0 16GB) + Socket L#0 + L3 L#0 (8192KB) > L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 > PU L#0 (P#0) > PU L#1 (P#8) > L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 > PU L#2 (P#2) > PU L#3 (P#10) > L2 L#2 (256KB) + L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2 > PU L#4 (P#4) > PU L#5 (P#12) > L2 L#3 (256KB) + L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3 > PU L#6 (P#6) > PU L#7 (P#14) > NUMANode L#1 (P#1 16GB) + Socket L#1 + L3 L#1 (8192KB) > L2 L#4 (256KB) + L1d L#4 (32KB) + L1i L#4 (32KB) + Core L#4 > PU L#8 (P#1) > PU L#9 (P#9) > L2 L#5 (256KB) + L1d L#5 (32KB) + L1i L#5 (32KB) + Core L#5 > PU L#10 (P#3) > PU L#11 (P#11) > L2 L#6 (256KB) + L1d L#6 (32KB) + L1i L#6 (32KB) + Core L#6 > PU L#12 (P#5) > PU L#13 (P#13) > L2 L#7 (256KB) + L1d L#7 (32KB) + L1i L#7 (32KB) + Core L#7 > PU L#14 (P#7) > PU L#15 (P#15) > HostBridge L#0 > PCIBridge > PCI 14e4:163b > Net L#0 "em1" > PCI 14e4:163b > Net L#1 "em2" > PCIBridge > PCI 1000:0058 > Block L#2 "sda" > Block L#3 "sdb" > PCIBridge > PCIBridge > PCIBridge > PCI 8086:10e8 > PCI 8086:10e8 > PCIBridge > PCI 8086:10e8 > PCI 8086:10e8 > PCIBridge > PCI 102b:0532 > PCI 8086:3a20 > PCI 8086:3a26 > Block L#4 "sr0" > mslee@myhost:~$ > > > > On Sun, Oct 18, 2015 at 2:51 PM, Moon-Sang Lee wrote: > >> >> thanks bruce. >> >> I didn't know that PCI slots have direct socket affinity. >> is it static or configurable through PCI configuration space? >> well, my NUT, two node NUMA, seems always returns -1 on calling >> rte_eth_dev_socket_id(portid) whenever portid is 0, 1, or other values. >> I appreciate if you explain more about getting the affinity. >> >> p.s. >> I'm using intel Xeon processor and 1G NIC(82576). >> >> >> >> >> On Fri, Oct 16, 2015 at 10:43 PM, Bruce Richardson < >> bruce.richardson@intel.com> wrote: >> >>> On Thu, Oct 15, 2015 at 11:08:57AM +0900, Moon-Sang Lee wrote: >>> > There is codes as below in examples/l2fwd/main.c and I think >>> > rte_eth_dev_socket_id(portid) >>> > always returns -1(SOCKET_ID_ANY) since there is no association code >>> between >>> > port and >>> > lcore in the example codes. >>> >>> Can you perhaps clarify what you mean here. On modern NUMA systems, such >>> as those >>> from Intel :-), the PCI slots are directly connected to the CPU sockets, >>> so the >>> ethernet ports do indeed have a direct NUMA affinity. It's not something >>> that >>> the app needs to specify. >>> >>> /Bruce >>> >>> > (i.e. I need to find a matching lcore from >>> > lcore_queue_conf[] with portid >>> > and call rte_lcore_to_socket_id(lcore_id).) >>> > >>> > /* init one RX queue */ >>> > fflush(stdout); >>> > ret = rte_eth_rx_queue_setup(portid, 0, nb_rxd, >>> > rte_eth_dev_socket_id(portid), >>> > NULL, >>> > l2fwd_pktmbuf_pool); >>> > if (ret < 0) >>> > rte_exit(EXIT_FAILURE, "rte_eth_rx_queue_setup:err=%d, >>> > port=%u\n", >>> > ret, (unsigned) portid); >>> > >>> > It works fine even though memory is allocated in different NUMA node. >>> But I >>> > wonder there is >>> > a DPDK API that associates inlcore to port internally thus >>> > rte_eth_devices[portid].pci_dev->numa_node >>> > contains proper node. >>> > >>> > >>> > -- >>> > Moon-Sang Lee, SW Engineer >>> > Email: sang0627@gmail.com >>> > Wisdom begins in wonder. *Socrates* >>> >> >> >> >> -- >> Moon-Sang Lee, SW Engineer >> Email: sang0627@gmail.com >> Wisdom begins in wonder. *Socrates* >> > > > > -- > Moon-Sang Lee, SW Engineer > Email: sang0627@gmail.com > Wisdom begins in wonder. *Socrates* > -- Moon-Sang Lee, SW Engineer Email: sang0627@gmail.com Wisdom begins in wonder. *Socrates*