From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <bruce.richardson@intel.com>
Received: from mga11.intel.com (mga11.intel.com [192.55.52.93])
 by dpdk.org (Postfix) with ESMTP id 6CF1F37A6
 for <dev@dpdk.org>; Mon, 13 Jun 2016 16:28:40 +0200 (CEST)
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
 by fmsmga102.fm.intel.com with ESMTP; 13 Jun 2016 07:28:39 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.26,466,1459839600"; d="scan'208";a="120978820"
Received: from lculbert-sp4.amr.corp.intel.com ([10.252.8.8])
 by fmsmga004.fm.intel.com with SMTP; 13 Jun 2016 07:28:37 -0700
Received: by  (sSMTP sendmail emulation); Mon, 13 Jun 2016 15:28:36 +0025
Date: Mon, 13 Jun 2016 15:28:36 +0100
From: Bruce Richardson <bruce.richardson@intel.com>
To: Take Ceara <dumitru.ceara@gmail.com>
Cc: dev@dpdk.org
Message-ID: <20160613142836.GH15736@bricha3-MOBL3>
References: <CAKKV4w-FYcK8XLv-HKUBJghNr-Fpsm-_EUqPSCb09zu2aZwxvQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAKKV4w-FYcK8XLv-HKUBJghNr-Fpsm-_EUqPSCb09zu2aZwxvQ@mail.gmail.com>
Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?=
 =?iso-8859-1?Q?opment?= Ireland Ltd.
User-Agent: Mutt/1.5.23 (2014-03-12)
Subject: Re: [dpdk-dev] Performance hit - NICs on different CPU sockets
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Mon, 13 Jun 2016 14:28:40 -0000

On Mon, Jun 13, 2016 at 04:07:37PM +0200, Take Ceara wrote:
> Hi,
> 
> I'm reposting here as I didn't get any answers on the dpdk-users mailing list.
> 
> We're working on a stateful traffic generator (www.warp17.net) using
> DPDK and we would like to control two XL710 NICs (one on each socket)
> to maximize CPU usage. It looks that we run into the following
> limitation:
> 
> http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html
> section 7.2, point 3
> 
> We completely split memory/cpu/NICs across the two sockets. However,
> the performance with a single CPU and both NICs on the same socket is
> better.
> Why do all the NICs have to be on the same socket, is there a
> driver/hw limitation?
> 
Hi,

so long as each thread only ever accesses the NIC on it's own local socket, then
there is no performance penalty. It's only when a thread on one socket works
using a NIC on a remote socket that you start seeing a penalty, with all
NIC-core communication having to go across QPI.

/Bruce