From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f45.google.com (mail-it0-f45.google.com [209.85.214.45]) by dpdk.org (Postfix) with ESMTP id 3FCF437AF for ; Tue, 14 Jun 2016 09:46:20 +0200 (CEST) Received: by mail-it0-f45.google.com with SMTP id e5so60889595ith.0 for ; Tue, 14 Jun 2016 00:46:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=qm5saFivyepyykBLVI9vX6YNjSBHp0d6QHwOqmUt4uM=; b=aNkVC9XvFmwHcwLRp+wDISOi37ybWrWIRF7+WQzcsYcT/pV6oEkMzz5SMtVB8m9w6a KAexNBLgncaOVmpwUxT+brOv9nq99GKjRLK8Ov9eelF9R2Nzcl2TyIlF8BpCa2DTwH1C rra9N76z2NHDVbboKHjYFO9He6Dk5gBREIchgS49CBP9lNhZoFrdCeMGcFwJrQrFccYZ 1RSF5Z3RGax5QjjJZxAZcVWUlhxlP3WtSKoykMYXf0k/XnOikfS/xDyWLBqXpS17H/8A yU7338me7ZM8Lumbt9YNmZsaKwzVyhahVg9XfO3bDGAJZdoBSQTrypCD1X+IrBWTqbKr EtGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=qm5saFivyepyykBLVI9vX6YNjSBHp0d6QHwOqmUt4uM=; b=FAx9uju10V7ixab/WgUgQcuIIsr1CKi6Zfm+5V8epvX352gU459tE76tMxYeGLF6v4 6bw1GY42F9HKRndxswTI51mjbrslBKSXNZshy/3cQ7fB6BsoedOnOzPG2kGqreyl2ofV pYu5HA/YJfNg34Mj6F4vsCyzSumt0pJp+t5folIQEoX3/qiFC4xZbHWpKNEZTvn6J19r JMqVrn16mj2LTgwMHaeulqAgMX1gSPoYxQVzqaqhR1mJDAUHRjKqnxMQWZ/cfl8US+g+ 17/VpaofCV1Pvb5fbJ+18rTj/yq4ZIBfje2ji4gyJuUovt3uxlkPXO07ElYA3iWzIWEk vHCQ== X-Gm-Message-State: ALyK8tLE5E2ynnJUqSl1uE/nooErXPl1fYhhnbkpyk32o/H7Z470p8eJBCLYJGekvALBUnQ8COfJa8D0lpIgxg== X-Received: by 10.36.227.12 with SMTP id d12mr5691759ith.49.1465890379620; Tue, 14 Jun 2016 00:46:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.20.197 with HTTP; Tue, 14 Jun 2016 00:46:00 -0700 (PDT) In-Reply-To: <1DB024A9-9185-41C8-9FA5-67C41891189A@intel.com> References: <1DB024A9-9185-41C8-9FA5-67C41891189A@intel.com> From: Take Ceara Date: Tue, 14 Jun 2016 09:46:00 +0200 Message-ID: To: "Wiles, Keith" Cc: "dev@dpdk.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] Performance hit - NICs on different CPU sockets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jun 2016 07:46:20 -0000 Hi Keith, On Mon, Jun 13, 2016 at 9:35 PM, Wiles, Keith wrote= : > > On 6/13/16, 9:07 AM, "dev on behalf of Take Ceara" wrote: > >>Hi, >> >>I'm reposting here as I didn't get any answers on the dpdk-users mailing = list. >> >>We're working on a stateful traffic generator (www.warp17.net) using >>DPDK and we would like to control two XL710 NICs (one on each socket) >>to maximize CPU usage. It looks that we run into the following >>limitation: >> >>http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html >>section 7.2, point 3 >> >>We completely split memory/cpu/NICs across the two sockets. However, >>the performance with a single CPU and both NICs on the same socket is >>better. >>Why do all the NICs have to be on the same socket, is there a >>driver/hw limitation? > > Normally the limitation is in the hardware, basically how the PCI bus is = connected to the CPUs (or sockets). How the PCI buses are connected to the = system depends on the Mother board design. I normally see the buses attache= d to socket 0, but you could have some of the buses attached to the other s= ockets or all on one socket via a PCI bridge device. > > No easy way around the problem if some of your PCI buses are split or all= on a single socket. Need to look at your system docs or look at lspci it h= as an option to dump the PCI bus as an ASCII tree, at least on Ubuntu. This is the motherboard we use on our system: http://www.supermicro.com/products/motherboard/Xeon/C600/X10DRX.cfm I need to swap some NICs around (as now we moved everything on socket 1) before I can share the lspci output. Thanks, Dumitru