From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.droids-corp.org (zoll.droids-corp.org [94.23.50.67]) by dpdk.org (Postfix) with ESMTP id EE47C5957 for ; Mon, 26 May 2014 13:59:19 +0200 (CEST) Received: from was59-1-82-226-113-214.fbx.proxad.net ([82.226.113.214] helo=[192.168.0.10]) by mail.droids-corp.org with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1WotaT-0001Zb-UC; Mon, 26 May 2014 14:01:13 +0200 Message-ID: <53832CA4.8060204@6wind.com> Date: Mon, 26 May 2014 13:59:32 +0200 From: Olivier MATZ User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Icedove/24.4.0 MIME-Version: 1.0 To: "Venkatesan, Venky" , Thomas Monjalon , "Ananyev, Konstantin" , "Shaw, Jeffrey B" , "Richardson, Bruce" , "nhorman@tuxdriver.com" , "stephen@networkplumber.org" References: <1400507789-18453-1-git-send-email-olivier.matz@6wind.com> <466924555.DZ26nc55Es@xps13> <1FD9B82B8BF2CF418D9A1000154491D9740B9074@ORSMSX102.amr.corp.intel.com> <537F599F.20009@6wind.com> <1FD9B82B8BF2CF418D9A1000154491D9740C60E2@ORSMSX102.amr.corp.intel.com> In-Reply-To: <1FD9B82B8BF2CF418D9A1000154491D9740C60E2@ORSMSX102.amr.corp.intel.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] [PATCH v2 00/17] add TSO support X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 11:59:20 -0000 Hi Venky, >> my testpmd iofwd test with the txqflags option disabling many mbuf >> features is not representative of a real world application. > [Venky] I did see your test reports. I also made the point that the > tests we have are insufficient for testing the impact. If you look at > data_ofs, it actually has an impact on two sides - the driver and the > upper layer. We do not at this time have a test for the upper > layer/accessor. Secondly, there is a whole class of apps (fast path > route for example) that aren't in your purview that do not need > txqflags. Calling it not representative of a real world application is > incorrect. I was talking about "iofwd with txqflags", not "txqflags". The iofwd mode means that the mbuf data is not accessed by the application, so I think we should not rely on it. I agree that "txqflags" could be useful in some situations, when the user does not need multi-segment. Anyway, the tests I've posted on the list do not show a real impact on this use case. Finally, as stated in the initial TSO cover letter [1], I've tested the upper layer impact that you are talking about with the 6WINDGate stack and there is no visible regression. > Secondly, your testpmd run baseline performance should be higher. At > this point, it is about 40% off from the numbers we see on the baseline > on the same CPU. If the baseline is incorrect, I cannot judge anything > more on the performance. We need to get the baseline performance the > same, and then compare impact. I tested with the default testpmd configuration, I though it was close enough to the best performance. > [Venky] I exclude VLAN because it is something explicitly set by the Rx > side of the driver. Having Rx access a second cache line will generate a > performance impact (can be mitigated by a prefetch, but it will cost > more instructions, and cannot be deterministically controlled). The rest > of the structure is on the transmit side - which is going to be cache > hot - at least in LLC anyway. There are cases where this will not be in > LLC - and we have a few of those. Those however, we can mitigate. If we add another cache line to the mbuf, if the code accesses it (whatever rx or tx side), it will at least increase the required memory bandwidth, and in the worst case will result in an additional cache miss. This is difficult to predict and depends on the use case. I think each solution would have its drawbacks in specific cases. I think the tests I've provided are representative enough to assume that there is no need to search for an alternative, knowing the fact that it also clean the mbuf structure and give more room for later use. > [Venky] I don't think reworking core data structures (especially > regressing core data structures) is a good thing. We have kept this > relatively stable over 5 releases, sometimes at the impact of > performance, and churning data structures is not a good thing. Sorry, but I don't think this is a good argument: we cannot say "it was stable during 5 releases so we cannot modify it". I would say that the code modifications should be guided by technical reasons. Keeping the API stable could be a good technical reason, but as far as I understood, the DPDK is not at this stage today. The mbuf rework brings 9 additional bytes in the structure, which help for TSO today, but may also help for next features. For instance, having another checksum flag as proposed by Stephen and me [2]. Moreover, the patches make the rte_mbuf structure clearer: the current separation rte_mbuf / rte_pktmbuf is a bit strange, for instance the ol_flags field which is clearly related to a pktmuf is located in the rte_mbuf. Now, I'm ready to make some concessions and discuss about an alternative solution. Thomas, you are the maintainer ;) what are your plans? Regards, Olivier [1] http://dpdk.org/ml/archives/dev/2014-May/002322.html [2] http://dpdk.org/ml/archives/dev/2014-May/002339.html