From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f41.google.com (mail-it0-f41.google.com [209.85.214.41]) by dpdk.org (Postfix) with ESMTP id B22BBC49A for ; Thu, 16 Jun 2016 17:16:30 +0200 (CEST) Received: by mail-it0-f41.google.com with SMTP id a5so137467732ita.1 for ; Thu, 16 Jun 2016 08:16:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fm1TKyU1BY5vy2EHFCrdVBrPE/0A8XYTDvKBk7duol4=; b=VOIPLIIbsqRWGj0o/UJ8pF2KDvUyuxZ4eam1q7PCfDXq8KMjQ2g29tKD+PzKSgYSN/ mxyLyre07HJedIBAvCPYm/UO/hxaznQWIeYE+Wmx4+Ay0ku5WK3qsuVvFHfPOZiDW51U bjvAgrN86VKQ41tXVWAlchWX3u5QlPnWJFG2vJX3kStvPcZvW61NDRaMckmVgqBuYYXs eRqGKE0U3obHaSUIzeQGtz6iVXnYs57ME6j4HZ9Wu2OD8r3Nfx55EhW88SoPEmTV+XgR TQKcW/7t1bA7vwZ6zI7jmSReMha6mY1E4Lipq1l9iPrueIxfeOOp4JTh2YXwEney2J7t h/Yw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fm1TKyU1BY5vy2EHFCrdVBrPE/0A8XYTDvKBk7duol4=; b=V/rGWXr9UEspSJ1O7u95aAymS5YLdrQ536RlQMDTnzkTANvD091v68zt4FuJeKVdZh w4vcTtDuzYnoFUmUY9tZhw9c0AYW1N3XOJLo3PHbnVldopdPwLxB8Qt7c7b7JzvOtLzn kcyVZHUjAkJIJp9w2vV32GKFnaXb/pu8QGB3u2MyG5RAzQm8STHgPUnle2VUYHQbHkK0 xxWE+5r35YtcNGODXceDbA5BsY1E32gWwAofJCskffpJylcyYPChknu9b/AlrgIAK+rx NNN4uqnV1d2UuCPqQNQkg0HVWPphbOFmn6p0hiL6Xw9gvJfq7BXll0d/TkFnrci/agh/ nP6A== X-Gm-Message-State: ALyK8tLRWU8szG+wVuatRmWhXPFtIjCd6Zdz9gqbRqg46+qtlP8yM/uTHfuhP3+v/Glg4F3L34JJ1YOnKsLt2Q== X-Received: by 10.36.14.76 with SMTP id 73mr8510739ite.70.1466090190052; Thu, 16 Jun 2016 08:16:30 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.20.197 with HTTP; Thu, 16 Jun 2016 08:16:10 -0700 (PDT) In-Reply-To: References: <1DB024A9-9185-41C8-9FA5-67C41891189A@intel.com> <2A957DA5-72A6-45B0-8B76-FE0DBDE758FD@intel.com> From: Take Ceara Date: Thu, 16 Jun 2016 17:16:10 +0200 Message-ID: To: "Wiles, Keith" Cc: "dev@dpdk.org" Content-Type: text/plain; charset=UTF-8 Subject: Re: [dpdk-dev] Performance hit - NICs on different CPU sockets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jun 2016 15:16:31 -0000 On Thu, Jun 16, 2016 at 4:58 PM, Wiles, Keith wrote: > > From the output below it appears the x710 devices 01:00.[0-3] are on socket 0 > And the x710 devices 02:00.[0-3] sit on socket 1. > I assume there's a mistake here. The x710 devices on socket 0 are: $ lspci | grep -ie "01:.*x710" 01:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) 01:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) 01:00.2 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) 01:00.3 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) and the X710 devices on socket 1 are: $ lspci | grep -ie "81:.*x710" 81:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) 81:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) 81:00.2 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) 81:00.3 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) > This means the ports on 01.00.xx should be handled by socket 0 CPUs and 02:00.xx should be handled by Socket 1. I can not tell if that is the case for you here. The CPUs or lcores from the cpu_layout.py should help understand the layout. > That was the first scenario I tried: - assign 16 CPUs from socket 0 to port 0 (01:00.3) - assign 16 CPUs from socket 1 to port 1 (81:00.3) Our performance measurements show then a setup rate of 1.6M sess/s which is less then half of what I get when i install both X710 on socket 1 and use only 16 CPUs from socket 1 for both ports. I double checked the cpu layout. We also have our own CLI and warnings when using cores that are not on the same socket as the port they're assigned too so the mapping should be fine. Thanks, Dumitru