From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ee0-x234.google.com (mail-ee0-x234.google.com [IPv6:2a00:1450:4013:c00::234]) by dpdk.org (Postfix) with ESMTP id 2CE571F3 for ; Sun, 22 Sep 2013 16:31:38 +0200 (CEST) Received: by mail-ee0-f52.google.com with SMTP id c41so1208720eek.11 for ; Sun, 22 Sep 2013 07:32:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=mDWCrS9ilFzM9XPqxySHcYNRSTMi6vxwknkRxfbq+tA=; b=jt7WT7nHAWILMCtUJOx5AyNwkDW/NTuFGh3CPCBQ8A9cEmtr8LQ4xEee5z9lsM6SNC ABsCLxANjyxi+jB6z3b7egkeZHAnvAmT9nwWWQNTYKy4zDtyz38I+IUorc+3xCBVmT9p qk6M4wrnLo823GlG/CYOcoRud8BeyL97S1O7DABa7oRPqVmzvYSgYD7uxzB0qKgs46n3 vtD/1Ms1KfB8Csf6zHxTW3/BFdoo7vtetdtscapJy0EVf7bRNBLRjSb4p9ZzxXRZnWyw EmvkgtKQND8PqyT/K0Y4RmIUEynX3yRASQmK8EgP/AoPm0d91Y3sacfT//2N46b8yOdc lNWA== X-Received: by 10.15.75.193 with SMTP id l41mr224961eey.58.1379860337615; Sun, 22 Sep 2013 07:32:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.14.9.130 with HTTP; Sun, 22 Sep 2013 07:31:37 -0700 (PDT) From: Chris Pappas Date: Sun, 22 Sep 2013 16:31:37 +0200 Message-ID: To: dev@dpdk.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Question regarding throughput number with DPDK l2fwd with Wind River System's pktgen X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 14:31:38 -0000 Hi, We have a question about the performance numbers we are getting measured through the pktgen application provided by Wind River Systems. The current setup is the following: We have two machines, each equipped with 6 dual-port 10 GbE NICs (with a total of 12 ports). Machine 0 runs DPDK L2FWD code, and Machine 1 runs Wind River System's pktgen. L2FWD is modified to forward the incoming packets to other statically assigned output port. Our machines have two Intel Xeon E5-2600 CPUs connected via QPI, and has two riser slots each having three 10Gbps NICs. Two NICS in riser slot 1 (NIC0 and NIC1) is connected to CPU 1 via PCIe Gen3, while the remaining NIC2 is connected to CPU2 also via PCIe Gen3. In riser slot 2, all NICs (NICs 3,4, and 5) are connected to CPU2 via PCIe Gen3. We were careful to assign the NIC ports to cores of CPU sockets that have direct physical connection to achieve max performance. With this setup, we are getting 120 Gbps throughput measured by pktgen with packet size 1500 Bytes. For 64 Byte packets, we are getting around 80 Gbps. Do these performance numbers make sense? We are reading related papers in this domain, and seems like our numbers are unusually high. We did our theoretical calculation and find that it should theoretically be possible because it does not hit the PCIe bandwidth or our machine, nor does it exceed QPI bandwidth when packets are forwarded over the NUMA node. Can you share your thoughts / experience with this? Thank you, Chris