From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f171.google.com (mail-io0-f171.google.com [209.85.223.171]) by dpdk.org (Postfix) with ESMTP id 363B5C4C8 for ; Thu, 16 Jun 2016 16:36:52 +0200 (CEST) Received: by mail-io0-f171.google.com with SMTP id t74so44436072ioi.0 for ; Thu, 16 Jun 2016 07:36:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=sJ7jJDqqiTYaadJUUrgje+fa3rDn60MgTNm0YWznF3o=; b=HptEGUA0MXuDbnyjguD6kt3gusolJCLFZhnqr9dWo5Jn2PVjc60YEZAdq7z1axKGHc U8Yfv7k0R5s2+HKuf8gpmhqZiPRaD31SW9xuCIAzHFSpBEmfDiNNemPJWTQPSnF/uTp1 xmZ9cN3NKaSDq/EXTYQFmz9mbhbmWmhBQyNKla3jezHzFoYWqKMPIRM6p6BLiR91ZKQ7 QtNOHBCvCSbQJd4pjj+RB0bBXfYBgVaumFlRYoAngnDoKf5UaCEsszDDW6II9RUy5OYU OHtMcR515hDg76y9SJW26zerBbi8m4zjPd09iBWVgtH93bMU98UdavBhMgN34XK9HhJb GgBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=sJ7jJDqqiTYaadJUUrgje+fa3rDn60MgTNm0YWznF3o=; b=QY7XlpXySB5R6JiVr0DBhHsVkVCBHm2QpwE+7nxh8Ds5FDwB1nd50hvMBwKk+O1WuQ uQv7pRf8yR0uqkKs7W4WX8tD0/1fL7ZsHZpTG2ySFE3Daff/XssDwj/ryY4GQqr+KhAJ zb65/pXINGlqUY7A+s/tcLhgKBpoWXIU4cNwXqPiSbgrZ1u79HoNdOaZz+ZlTNyIhpP4 Ys17e9CotIn4iiaBOxt4rBQvNRDGevgw2Yb085Ykb+1yGM19hpeyLLUr9ZRXXAyQ+ZB7 MtoMPjWfVx0mQ5wJxBJzc8ZEIMf2U8lywvSYl+MSG9GppwrGbIomvkMllf112rtWFwq1 dFyA== X-Gm-Message-State: ALyK8tIV9eBCjJ44A+nzFg5iq9KGOPwWmAqpTjHcvXOFtAcY27a2saOjhmi1RI5pUj+pv86BDAiFAXb+TxchMw== X-Received: by 10.107.173.196 with SMTP id m65mr8736027ioo.131.1466087811152; Thu, 16 Jun 2016 07:36:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.20.197 with HTTP; Thu, 16 Jun 2016 07:36:31 -0700 (PDT) In-Reply-To: <2A957DA5-72A6-45B0-8B76-FE0DBDE758FD@intel.com> References: <1DB024A9-9185-41C8-9FA5-67C41891189A@intel.com> <2A957DA5-72A6-45B0-8B76-FE0DBDE758FD@intel.com> From: Take Ceara Date: Thu, 16 Jun 2016 16:36:31 +0200 Message-ID: To: "Wiles, Keith" Cc: "dev@dpdk.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] Performance hit - NICs on different CPU sockets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jun 2016 14:36:52 -0000 Hi Keith, On Tue, Jun 14, 2016 at 3:47 PM, Wiles, Keith wrote= : >>> Normally the limitation is in the hardware, basically how the PCI bus i= s connected to the CPUs (or sockets). How the PCI buses are connected to th= e system depends on the Mother board design. I normally see the buses attac= hed to socket 0, but you could have some of the buses attached to the other= sockets or all on one socket via a PCI bridge device. >>> >>> No easy way around the problem if some of your PCI buses are split or a= ll on a single socket. Need to look at your system docs or look at lspci it= has an option to dump the PCI bus as an ASCII tree, at least on Ubuntu. >> >>This is the motherboard we use on our system: >> >>http://www.supermicro.com/products/motherboard/Xeon/C600/X10DRX.cfm >> >>I need to swap some NICs around (as now we moved everything on socket >>1) before I can share the lspci output. > > FYI: the option for lspci is =E2=80=98lspci =E2=80=93tv=E2=80=99, but may= be more options too. > I retested with two 10G X710 ports connected back to back: port 0: 0000:01:00.3 - socket 0 port 1: 0000:81:00.3 - socket 1 I ran the following scenarios: - assign 16 threads from CPU 0 on socket 0 to port 0 and 16 threads from CPU 1 to port 1 =3D> setup rate of 1.6M sess/s - assign only the 16 threads from CPU0 for both ports (so 8 threads on socket 0 for port 0 and 8 threads on socket 0 for port 1) =3D> setup rate of 3M sess/s - assign only the 16 threads from CPU1 for both ports (so 8 threads on socket 1 for port 0 and 8 threads on socket 1 for port 1) =3D> setup rate of 3M sess/s I also tried a scenario with two machines connected back to back each of which had a NIC on socket 1. I assigned 16 threads from socket 1 on each machine to the port and performance scaled to 6M sess/s as expected. I double checked all our memory allocations and, at least in the tested scenario, we never use memory that's not on the same socket as the core. I pasted below the output of lspci -tv. I see that 0000:01:00.3 and 0000:81:00.3 are connected to different PCI bridges but on each of those bridges there are also "Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel " devices. It would be great if you could also take a look in case I missed/misunderstood something. Thanks, Dumitru # lspci -tv -+-[0000:ff]-+-08.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 0 | +-08.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 0 | +-08.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 0 | +-09.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 1 | +-09.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 1 | +-09.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 1 | +-0b.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring | +-0b.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring | +-0b.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring | +-0c.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0d.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0d.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0f.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring Agent | +-0f.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring Agent | +-0f.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring Agent | +-0f.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring Agent | +-0f.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers | +-0f.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers | +-0f.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers | +-10.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface | +-10.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface | +-10.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers | +-10.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers | +-10.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers | +-12.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 0 | +-12.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 0 | +-12.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 1 | +-12.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 1 | +-13.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Target Address, Thermal & RAS Registers | +-13.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Target Address, Thermal & RAS Registers | +-13.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder | +-13.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder | +-13.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Channel 0/1 Broadcast | +-13.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Global Broadcast | +-14.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 0 Thermal Control | +-14.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 1 Thermal Control | +-14.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 0 ERROR Registers | +-14.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 1 ERROR Registers | +-14.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 | +-14.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 | +-14.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 | +-14.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 | +-16.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Target Address, Thermal & RAS Registers | +-16.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Target Address, Thermal & RAS Registers | +-16.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel Target Address Decoder | +-16.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel Target Address Decoder | +-16.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Channel 2/3 Broadcast | +-16.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Global Broadcast | +-17.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 0 Thermal Control | +-17.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 1 Thermal Control | +-17.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 0 ERROR Registers | +-17.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 1 ERROR Registers | +-17.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 | +-17.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 | +-17.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 | +-17.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 | +-1e.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1e.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1e.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1e.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1e.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1f.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 VCU | \-1f.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 VCU +-[0000:80]-+-02.0-[81]--+-00.0 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ | | +-00.1 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ | | +-00.2 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ | | \-00.3 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ | +-03.0-[82]----00.0 Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ | +-03.2-[83]----00.0 Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ | +-04.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 0 | +-04.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 1 | +-04.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 2 | +-04.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 3 | +-04.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 4 | +-04.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 5 | +-04.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 6 | +-04.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 7 | +-05.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Address Map, VTd_Misc, System Management | +-05.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Hot Pl= ug | +-05.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 RAS, Control Status and Global Errors | \-05.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 I/O AP= IC +-[0000:7f]-+-08.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 0 | +-08.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 0 | +-08.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 0 | +-09.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 1 | +-09.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 1 | +-09.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 QPI Li= nk 1 | +-0b.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring | +-0b.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring | +-0b.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 R3 QPI Link 0 & 1 Monitoring | +-0c.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0c.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0d.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0d.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Unicast Registers | +-0f.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring Agent | +-0f.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring Agent | +-0f.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring Agent | +-0f.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Buffered Ring Agent | +-0f.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers | +-0f.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers | +-0f.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 System Address Decoder & Broadcast Registers | +-10.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface | +-10.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 PCIe Ring Interface | +-10.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers | +-10.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers | +-10.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Scratchpad & Semaphore Registers | +-12.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 0 | +-12.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 0 | +-12.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 1 | +-12.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Home Agent 1 | +-13.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Target Address, Thermal & RAS Registers | +-13.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Target Address, Thermal & RAS Registers | +-13.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder | +-13.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel Target Address Decoder | +-13.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Channel 0/1 Broadcast | +-13.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Global Broadcast | +-14.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 0 Thermal Control | +-14.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 1 Thermal Control | +-14.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 0 ERROR Registers | +-14.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 0 Channel 1 ERROR Registers | +-14.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 | +-14.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 | +-14.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 | +-14.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 0 & 1 | +-16.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Target Address, Thermal & RAS Registers | +-16.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Target Address, Thermal & RAS Registers | +-16.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel Target Address Decoder | +-16.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel Target Address Decoder | +-16.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Channel 2/3 Broadcast | +-16.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO Global Broadcast | +-17.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 0 Thermal Control | +-17.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 1 Thermal Control | +-17.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 0 ERROR Registers | +-17.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Integrated Memory Controller 1 Channel 1 ERROR Registers | +-17.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 | +-17.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 | +-17.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 | +-17.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DDRIO (VMSE) 2 & 3 | +-1e.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1e.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1e.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1e.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1e.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Power Control Unit | +-1f.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 VCU | \-1f.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 VCU \-[0000:00]-+-00.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMI2 +-01.0-[01]--+-00.0 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ | +-00.1 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ | +-00.2 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ | \-00.3 Intel Corporation Ethernet Controller X710 for 10GbE SFP+ +-02.0-[02]--+-00.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection | \-00.1 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection +-02.2-[03]--+-00.0 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection | \-00.1 Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection +-03.0-[04]-- +-03.2-[05]-- +-04.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 0 +-04.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 1 +-04.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 2 +-04.3 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 3 +-04.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 4 +-04.5 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 5 +-04.6 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 6 +-04.7 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 DMA Channel 7 +-05.0 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Address Map, VTd_Misc, System Management +-05.1 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 Hot Pl= ug +-05.2 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 RAS, Control Status and Global Errors +-05.4 Intel Corporation Xeon E7 v3/Xeon E5 v3/Core i7 I/O AP= IC +-11.0 Intel Corporation C610/X99 series chipset SPSR +-11.4 Intel Corporation C610/X99 series chipset sSATA Controller [AHCI mode] +-14.0 Intel Corporation C610/X99 series chipset USB xHCI Host Controller +-16.0 Intel Corporation C610/X99 series chipset MEI Controll= er #1 +-16.1 Intel Corporation C610/X99 series chipset MEI Controll= er #2 +-1a.0 Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 +-1c.0-[06]-- +-1c.3-[07-08]----00.0-[08]----00.0 ASPEED Technology, Inc. ASPEED Graphics Family +-1c.4-[09]--+-00.0 Intel Corporation I350 Gigabit Network Connection | \-00.1 Intel Corporation I350 Gigabit Network Connection +-1d.0 Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 +-1f.0 Intel Corporation C610/X99 series chipset LPC Controll= er +-1f.2 Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] +-1f.3 Intel Corporation C610/X99 series chipset SMBus Contro= ller \-1f.6 Intel Corporation C610/X99 series chipset Thermal Subs= ystem