From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f42.google.com (mail-wm0-f42.google.com [74.125.82.42]) by dpdk.org (Postfix) with ESMTP id C76572C2A for ; Fri, 16 Sep 2016 09:14:09 +0200 (CEST) Received: by mail-wm0-f42.google.com with SMTP id k186so24021247wmd.0 for ; Fri, 16 Sep 2016 00:14:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to; bh=vm94cgQR2+hEYVMxmm15KbqXB8iNe20itJihGrhT3tY=; b=po5hu0C1cdHWAgHyjVot+/VBAi806stPqq2oJ1DwjqacpgY3EllGMQrmAnJu3sqAce gXhwKApGJ38xUwghoDWltoIkgDYpSpRemHgwQ7NmKl7ICwP2P1rymddlfE1htC+iH97x a+E9wDJpK/HrQByWRuL//mbWRfWZbmbPkvH2yLPIPBwDeVa6UzvM9wzQw+8t6aPXj2J2 vm523ZFqhx9+QX6BYL77XHLMCSILPGsxIr65/Ro00tZm76t26UFhnODR9vU6bJq0W/69 wYWhu/U8oV2j5oKTOF63n4uW1G2pR+4P6x33ZL6hdnQ/NSmTTmOALKjxkzmVkUiMv2Cz 9+Kw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to; bh=vm94cgQR2+hEYVMxmm15KbqXB8iNe20itJihGrhT3tY=; b=dOzjPbxqmxX7jR+H5IEck1GTzbSCHjXTYk/ZOtGKCen6nKrc8QQdo/lYZ2CLii09TM GFaF0HPs/LnLqjbbg+j/i2WgjKVH+u5NZfqh5u+d/ME6sLopttBCiud8y+dootB90jVK R1m5k0ekgErLglNVnEcsL+38X6DElO6pV6XRYIyoOvPQGRHhzjCUi8/yC7TbsFIaLPmj BmNpspLiODqwVVh0gi8ssh8WPdUR81gIDMdsNuN+B2ZRL4Hi9PEuwwGGyr1AX6ul2MKm zlc1QiOhTZpiexMApCKeIqqQc+aUycnGF5PNizjkxuFn7MT8X3uvuYyGpORMLnpcIURu IFNw== X-Gm-Message-State: AE9vXwO++RDuLCHJBcLwvIpflWFW4D43yoRddg/rT/6yhstry6WlIjVsqRPcHxjMfBpUOhyn X-Received: by 10.28.209.142 with SMTP id i136mr6888864wmg.0.1474010049522; Fri, 16 Sep 2016 00:14:09 -0700 (PDT) Received: from 6wind.com (guy78-3-82-239-227-177.fbx.proxad.net. [82.239.227.177]) by smtp.gmail.com with ESMTPSA id j10sm6950541wjy.13.2016.09.16.00.14.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 16 Sep 2016 00:14:08 -0700 (PDT) Date: Fri, 16 Sep 2016 09:14:03 +0200 From: Adrien Mazarguil To: Luke Gorrie Cc: "dev@dpdk.org" Message-ID: <20160916071403.GW17252@6wind.com> Mail-Followup-To: Luke Gorrie , "dev@dpdk.org" References: <20160914143016.GT17252@6wind.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [dpdk-dev] Possible bug in mlx5_tx_burst_mpw? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Sep 2016 07:14:10 -0000 On Wed, Sep 14, 2016 at 09:33:18PM +0200, Luke Gorrie wrote: > Hi Adrien, > > On 14 September 2016 at 16:30, Adrien Mazarguil > wrote: > > > Your interpretation is correct (this is intentional and not a bug). > > > > Thanks very much for clarifying. > > This is interesting to me because I am also working on a ConnectX-4 (Lx) > driver based on the newly released driver interface specification [1] and I > am wondering how interested I should be in this MPW feature that is > currently not documented. Seems like this document only describes established features whose interface won't be subject to firmware evolutions, I think MPW is not one of them. AFAIK currently MPW cannot be used with LSO which we intend to support soon. Our implementation is a stripped down version of the code found in libmlx5. I guess you could ask Mellanox directly if you need more information. > In the event successive packets share a few properties (length, number of > > segments, offload flags), these can be factored out as an optimization to > > lower the amount of traffic on the PCI bus. This feature is currently > > supported by the ConnectX-4 Lx family of adapters. > > > > I have a concern here that I hope you will forgive me for voicing. > > This optimization seems to run the risk of inflating scores on > constant-packet-size IXIA-style benchmarks like [2] and making them less > useful for predicting real-world performance. That seems like a negative to > me as an application developer. I wonder if I am overlooking some practical > benefits that motivate implementing this in silicon and in the driver and > enabling it by default? Your concern is understandable, no offense taken. You are obviously right about benchmarks with constant packets, whose results can be improved by MPW. Performance-wise, with the right traffic patterns MPW allows ConnectX-4 Lx adapters to outperform their non-Lx counterparts (e.g. comparing 40G EN Lx PCIe 8x vs. 40G EN PCIe 8x) when measuring traffic rate (Mpps), not throughput. Disabling MPW yields comparable results, which is why it is considered to be an optimization. Since processing MPW consumes a few additional CPU cycles, it can be disabled at runtime with the txq_mpw_en switch (documented in mlx5.rst). Now about the real-world scenario, we are not talking about needing millions of identical packets to notice an improvement. MPW is effective from 2 to at most 5 consecutive packets that share some meta-data (length, number of segments and offload flags), all within the same burst. Just to be clear, neither their destination nor their payload need to be the same, it would have been useless otherwise. Sending a few packets at once with such similar properties is common occurrence in the real world, think about forwarding TCP traffic that has been shaped to a constant size by LSO or MTU. Like many optimizations, this one targets a specific yet common use-case. If you would rather get a constant rate out of any traffic pattern for predictable latency, DPDK which is burst-oriented is probably not what your application needs if used as-is. > [1] > http://www.mellanox.com/related-docs/user_manuals/Ethernet_Adapters_Programming_Manual.pdf > [2] > https://www.mellanox.com/blog/2016/06/performance-beyond-numbers-stephen-curry-style-server-io/ -- Adrien Mazarguil 6WIND