From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f48.google.com (mail-it0-f48.google.com [209.85.214.48]) by dpdk.org (Postfix) with ESMTP id 5A7355598 for ; Tue, 14 Jun 2016 09:48:12 +0200 (CEST) Received: by mail-it0-f48.google.com with SMTP id a5so70188378ita.1 for ; Tue, 14 Jun 2016 00:48:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=vgD2a3KGmZLgvkxCJjQua8u4RgU2Y5ao5lnce605jEg=; b=xhzksc2HbK5N+p+t8OpOwDpw1RhPtZs0yRQjctrnOz9zDQgLvkcvBg+5ShpNas5qqW bliXVmd6znv8Dy4/cYsJl49WmID8nbm6EvK/WEacYmFtiuYD1te99BOrlQexrT/920bK YlC0eCOLfNcvQ2cSnFVZ8sB1PgZRUvi4dC7/vS0uGpj9b81oS9jOdRx4L4Gd+tUTdOnB zWHk6HLxYqyBRW8a23vyFpw10G3COrEygPv/BMkOLMS/m4/Rzo2Dj5LVyXBSFrKpeoZ0 S3FQ9Ddr/94/HcReH3aF4AL714khCMxQoMwmh/XzqPf1PkM4Vi09jAsvabqcVQRua8eB O+1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=vgD2a3KGmZLgvkxCJjQua8u4RgU2Y5ao5lnce605jEg=; b=gq2tQCQ6xS0Kvoj7xsXGSVCGPyXomAMJp87lC2x+Zx9GE8zZQdjm/PCoE6e7e7ufzW 5ar1CyKytjHgZIhDwhrGhBbtg+v2+4YXg22iwAucFOxj0VNgqe5fkWcM58X2yUTlZ56j KY3YZ5+Zq90kd2HxObzCuivF9i4xZBbH7hBjtx+SLJ9dFcO9Kgv+FecR6SKLLqsm7IoY BP041CTp6m+DWPCoYpEnnEzsGN/j6frBLiEFUikriNIbCmpfqNLELUBEIK0MQPmA11RE NYZ7lG3L/22bUgrwQu2T5xUx22Dhoi1w5eCeIyf4XF1lfFEEIh6TFQuAzWePMVWL6MW/ sagw== X-Gm-Message-State: ALyK8tL4B3Y4uC+8gfBlxvi8UhcIK74M5dmcDNSX+gMFx7Znmz1RwnO+9sPpBRZ5ODfKAJQzFTTYGxkVsqKZnw== X-Received: by 10.36.50.201 with SMTP id j192mr5700179ita.70.1465890491840; Tue, 14 Jun 2016 00:48:11 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.20.197 with HTTP; Tue, 14 Jun 2016 00:47:52 -0700 (PDT) In-Reply-To: <20160613142836.GH15736@bricha3-MOBL3> References: <20160613142836.GH15736@bricha3-MOBL3> From: Take Ceara Date: Tue, 14 Jun 2016 09:47:52 +0200 Message-ID: To: Bruce Richardson Cc: dev@dpdk.org Content-Type: text/plain; charset=UTF-8 Subject: Re: [dpdk-dev] Performance hit - NICs on different CPU sockets X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jun 2016 07:48:12 -0000 Hi Bruce, On Mon, Jun 13, 2016 at 4:28 PM, Bruce Richardson wrote: > On Mon, Jun 13, 2016 at 04:07:37PM +0200, Take Ceara wrote: >> Hi, >> >> I'm reposting here as I didn't get any answers on the dpdk-users mailing list. >> >> We're working on a stateful traffic generator (www.warp17.net) using >> DPDK and we would like to control two XL710 NICs (one on each socket) >> to maximize CPU usage. It looks that we run into the following >> limitation: >> >> http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html >> section 7.2, point 3 >> >> We completely split memory/cpu/NICs across the two sockets. However, >> the performance with a single CPU and both NICs on the same socket is >> better. >> Why do all the NICs have to be on the same socket, is there a >> driver/hw limitation? >> > Hi, > > so long as each thread only ever accesses the NIC on it's own local socket, then > there is no performance penalty. It's only when a thread on one socket works > using a NIC on a remote socket that you start seeing a penalty, with all > NIC-core communication having to go across QPI. > > /Bruce Thanks for the confirmation. We'll go through our code again to double check that no thread accesses the NIC or memory on a remote socket. Regards, Dumitru