From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pavel.odintsov@gmail.com> Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by dpdk.org (Postfix) with ESMTP id 887BC5A6B for <dev@dpdk.org>; Wed, 1 Jul 2015 14:44:57 +0200 (CEST) Received: by wiga1 with SMTP id a1so124559821wig.0 for <dev@dpdk.org>; Wed, 01 Jul 2015 05:44:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=PKNkydeIx44tBoyRriOiByVfbJE7WLc8x8Q4Esy1I9w=; b=z9pv9gEU9fXnJh9sUGgsJoe3SY+/nZLpKv5h2wzSoGoGmCgDFcaY6Ujl4q6KqGcFLi DwfxxhDNnFEkVu7CI+YzKKxkeTko/8QNnm7h1GMJpT2pYaEqSIzDdluoBD1XGZMVWL/z 6vVhOIeLe2iBxXQYDM1OaY5E5/kSTblh8hKmlj4DoRnhHaLn2AywwpmD4TihdKeJiu9j mPS9SM5L0uU6K9aerN0uLuELQL+c3Kf6ufRXip4huLpar3D5IRh/3DgZj9r1H2y+453y P4FFROdTy+HQBgzijNUoOkuoYc55eeblOhJBSjpRvOoAIfYcEQ2TitRkBG+RoQjiR8TV mRMw== MIME-Version: 1.0 X-Received: by 10.194.100.104 with SMTP id ex8mr36821418wjb.69.1435754697204; Wed, 01 Jul 2015 05:44:57 -0700 (PDT) Received: by 10.27.178.129 with HTTP; Wed, 1 Jul 2015 05:44:57 -0700 (PDT) In-Reply-To: <CANDrEHkz+F9HTVNwQrJ7f=93qUb2AJgUKmFKJ2BWctEp-0H4fA@mail.gmail.com> References: <CALgsdbemVaahegO0uhE1dG52PYOxSvCtT-iD9RD2y7PkKus7Cg@mail.gmail.com> <CAKBXKmC3vcMg-QhDaRKh4iu+SPuc8kzwcyruW+-H1mEdvJwz=g@mail.gmail.com> <CALgsdbe2-K8buJXz_oQVMaojWaj3G1Z=DHWybn9vRkK7-ZuHJg@mail.gmail.com> <CAKBXKmCCPQ0Mw9hZxDn2aRsCySvZW1GwUOXYr0x0tzbWyhrH1w@mail.gmail.com> <CAD5-U7TuQJsEqcrXQFaXr0=hO8gHKx_V_J+dwJw9zZ0bg4L5uw@mail.gmail.com> <CALgsdbd__EBf+hnWxTNeNO-c8Dikyt1SM4eQQMsndSu2m-G+9w@mail.gmail.com> <CANDrEHkz+F9HTVNwQrJ7f=93qUb2AJgUKmFKJ2BWctEp-0H4fA@mail.gmail.com> Date: Wed, 1 Jul 2015 15:44:57 +0300 Message-ID: <CALgsdbc93MvV+yiHjSKM-cxQ=P+4LvsKqsBfA8Z=0d648t0+Fw@mail.gmail.com> From: Pavel Odintsov <pavel.odintsov@gmail.com> To: Vladimir Medvedkin <medvedkinv@gmail.com> Content-Type: text/plain; charset=UTF-8 Cc: "dev@dpdk.org" <dev@dpdk.org> Subject: Re: [dpdk-dev] Could not achieve wire speed for 40GE with any DPDK version on XL710 NIC's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Wed, 01 Jul 2015 12:44:57 -0000 Thanks for answer, Vladimir! So we need look for x16 NIC if we want achieve 40GE line rate... On Wed, Jul 1, 2015 at 3:06 PM, Vladimir Medvedkin <medvedkinv@gmail.com> wrote: > Hi Pavel, > > Looks like you ran into pcie bottleneck. So let's calculate xl710 rx only > case. > Assume we have 32byte descriptors (if we want more offload). > DMA makes one pcie transaction with packet payload, one descriptor writeback > and one memory request for free descriptors for every 4 packets. For > Transaction Layer Packet (TLP) there is 30 bytes overhead (4 PHY + 6 DLL + > 16 header + 4 ECRC). So for 1 rx packet dma sends 30 + 64(packet itself) + > 30 + 32 (writeback descriptor) + (16 / 4) (read request for new > descriptors). Note that we do not take into account PCIe ACK/NACK/FC Update > DLLP. So we have 160 bytes per packet. One lane PCIe 3.0 transmits 1 byte in > 1 ns, so x8 transmits 8 bytes in 1 ns. 1 packet transmits in 20 ns. Thus > in theory pcie 3.0 x8 may transfer not more than 50mpps. > Correct me if I'm wrong. > > Regards, > Vladimir > > > 2015-06-29 18:41 GMT+03:00 Pavel Odintsov <pavel.odintsov@gmail.com>: >> >> Hello, Andrew! >> >> What NIC have you used? Is it XL710? >> >> On Mon, Jun 29, 2015 at 6:38 PM, Andrew Theurer <atheurer@redhat.com> >> wrote: >> > >> > >> > On Mon, Jun 29, 2015 at 10:06 AM, Keunhong Lee <dlrmsghd@gmail.com> >> > wrote: >> >> >> >> I have not used XL710 or i40e. >> >> I have no opinion for those NICs. >> >> >> >> Keunhong. >> >> >> >> 2015-06-29 15:59 GMT+09:00 Pavel Odintsov <pavel.odintsov@gmail.com>: >> >> >> >> > Hello! >> >> > >> >> > Lee, thank you so much for sharing your experience! What do you think >> >> > about 40GE version of 82599? >> >> > >> >> > On Mon, Jun 29, 2015 at 2:35 AM, Keunhong Lee <dlrmsghd@gmail.com> >> >> > wrote: >> >> > > DISCLAIMER: This information is not verified. This is truly my >> >> > > personal >> >> > > opinion. >> >> > > >> >> > > As I know, intel 82599 is the only 10G NIC which supports line rate >> >> > > with >> >> > > minimum sized packets (64 byte). >> >> > > According to our internal tests, Mellanox's 40G NICs even support >> >> > > less >> >> > than >> >> > > 30Mpps. >> >> > > I think 40 Mpps is the hardware capacity. >> > >> > >> > This is approximately what I see as well. >> > >> >> >> >> > > >> >> > > Keunhong. >> >> > > >> >> > > >> >> > > >> >> > > 2015-06-28 19:34 GMT+09:00 Pavel Odintsov >> >> > > <pavel.odintsov@gmail.com>: >> >> > >> >> >> > >> Hello, folks! >> >> > >> >> >> > >> We have execute bunch of tests for receive data with Intel XL710 >> >> > >> 40GE >> >> > >> NIC. We want to achieve wire speed on this platform for traffic >> >> > >> capture. >> >> > >> >> >> > >> But we definitely can't do it. We tried with different versions of >> >> > >> DPDK: 1.4, 1.6, 1.8, 2.0. And have not success. >> >> > >> >> >> > >> We achieved only 40Mpps and could do more. >> >> > >> >> >> > >> Could anybody help us with this issue? Looks like this NIC's could >> >> > >> not >> >> > >> work on wire speed :( >> >> > >> >> >> > >> Platform: Intel Xeon E5 e5 2670 + XL 710. >> >> > >> >> >> > >> -- >> >> > >> Sincerely yours, Pavel Odintsov >> >> > > >> >> > > >> >> > >> >> > >> >> > >> >> > -- >> >> > Sincerely yours, Pavel Odintsov >> >> > >> > >> > -Andrew >> > >> > >> >> >> >> -- >> Sincerely yours, Pavel Odintsov > > -- Sincerely yours, Pavel Odintsov