From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-f48.google.com (mail-la0-f48.google.com [209.85.215.48]) by dpdk.org (Postfix) with ESMTP id 8D0845A58 for ; Wed, 1 Jul 2015 14:06:08 +0200 (CEST) Received: by lagc2 with SMTP id c2so39178673lag.3 for ; Wed, 01 Jul 2015 05:06:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=p8N/l6+bSnzlUQD6x79e5605AOx+sOecGoVQlaShtFw=; b=ggqq9VAUc43fwtcOPnfy6a4YDL48qwzuM/WmZ0hq9npFg0NTaOu7O7zyZ2wSSZwOq3 N9ynZZEPJ0yef/96dFYko4O5ogf7sh5dYLYmQhFm0j4mGRE2rNogGKZ2oY/AEoYmJWOx xnoVota3PMDNXa44GO00ltzlc+XGPPMfpaAbqXG2+xjWjn7gi3NCfTy3blesC8aXccti vUu4jYBwSBDUyGxY7SF85MoJXDtMDq/Gq19qrRfBgPJTNKycfQ+vciruesvDUS/jgRBm YqFKLbJ593h3BPU2zoMwUskvaw3yCj+0jZrjAlGDnT0PHnsgMIXVrSYQrVZeeCl8JZ5D cgcA== MIME-Version: 1.0 X-Received: by 10.112.130.68 with SMTP id oc4mr24372943lbb.87.1435752368130; Wed, 01 Jul 2015 05:06:08 -0700 (PDT) Received: by 10.114.10.229 with HTTP; Wed, 1 Jul 2015 05:06:08 -0700 (PDT) In-Reply-To: References: Date: Wed, 1 Jul 2015 15:06:08 +0300 Message-ID: From: Vladimir Medvedkin To: Pavel Odintsov Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Could not achieve wire speed for 40GE with any DPDK version on XL710 NIC's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Jul 2015 12:06:08 -0000 Hi Pavel, Looks like you ran into pcie bottleneck. So let's calculate xl710 rx only case. Assume we have 32byte descriptors (if we want more offload). DMA makes one pcie transaction with packet payload, one descriptor writeback and one memory request for free descriptors for every 4 packets. For Transaction Layer Packet (TLP) there is 30 bytes overhead (4 PHY + 6 DLL + 16 header + 4 ECRC). So for 1 rx packet dma sends 30 + 64(packet itself) + 30 + 32 (writeback descriptor) + (16 / 4) (read request for new descriptors). Note that we do not take into account PCIe ACK/NACK/FC Update DLLP. So we have 160 bytes per packet. One lane PCIe 3.0 transmits 1 byte in 1 ns, so x8 transmits 8 bytes in 1 ns. 1 packet transmits in 20 ns. Thus in theory pcie 3.0 x8 may transfer not more than 50mpps. Correct me if I'm wrong. Regards, Vladimir 2015-06-29 18:41 GMT+03:00 Pavel Odintsov : > Hello, Andrew! > > What NIC have you used? Is it XL710? > > On Mon, Jun 29, 2015 at 6:38 PM, Andrew Theurer > wrote: > > > > > > On Mon, Jun 29, 2015 at 10:06 AM, Keunhong Lee > wrote: > >> > >> I have not used XL710 or i40e. > >> I have no opinion for those NICs. > >> > >> Keunhong. > >> > >> 2015-06-29 15:59 GMT+09:00 Pavel Odintsov : > >> > >> > Hello! > >> > > >> > Lee, thank you so much for sharing your experience! What do you think > >> > about 40GE version of 82599? > >> > > >> > On Mon, Jun 29, 2015 at 2:35 AM, Keunhong Lee > >> > wrote: > >> > > DISCLAIMER: This information is not verified. This is truly my > >> > > personal > >> > > opinion. > >> > > > >> > > As I know, intel 82599 is the only 10G NIC which supports line rate > >> > > with > >> > > minimum sized packets (64 byte). > >> > > According to our internal tests, Mellanox's 40G NICs even support > less > >> > than > >> > > 30Mpps. > >> > > I think 40 Mpps is the hardware capacity. > > > > > > This is approximately what I see as well. > > > >> > >> > > > >> > > Keunhong. > >> > > > >> > > > >> > > > >> > > 2015-06-28 19:34 GMT+09:00 Pavel Odintsov >: > >> > >> > >> > >> Hello, folks! > >> > >> > >> > >> We have execute bunch of tests for receive data with Intel XL710 > 40GE > >> > >> NIC. We want to achieve wire speed on this platform for traffic > >> > >> capture. > >> > >> > >> > >> But we definitely can't do it. We tried with different versions of > >> > >> DPDK: 1.4, 1.6, 1.8, 2.0. And have not success. > >> > >> > >> > >> We achieved only 40Mpps and could do more. > >> > >> > >> > >> Could anybody help us with this issue? Looks like this NIC's could > >> > >> not > >> > >> work on wire speed :( > >> > >> > >> > >> Platform: Intel Xeon E5 e5 2670 + XL 710. > >> > >> > >> > >> -- > >> > >> Sincerely yours, Pavel Odintsov > >> > > > >> > > > >> > > >> > > >> > > >> > -- > >> > Sincerely yours, Pavel Odintsov > >> > > > > > -Andrew > > > > > > > > -- > Sincerely yours, Pavel Odintsov >