From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id CFE5CA09E0; Fri, 13 Nov 2020 18:58:49 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 78EFBC868; Fri, 13 Nov 2020 18:58:47 +0100 (CET) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) by dpdk.org (Postfix) with ESMTP id 7864C4C96 for ; Fri, 13 Nov 2020 18:58:44 +0100 (CET) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id E15C6789; Fri, 13 Nov 2020 12:58:42 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 13 Nov 2020 12:58:43 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm2; bh= RUG1Tiz3AKJWRlE/UGPFTfSc92gGDvMN30yt3Y50isI=; b=Yo8G3k2wQcX3d4yX 4gEZi7bwXiQbxPS9/dxliTJNSJ2RipNDEwFt8B8udTPPefbfoKbO00qcC69y+IEt WFxkF+Hr0aI4Gqpazj1H8HfJi+QroG/JwK6XlUX8SCxdZBWkfBBkYDbWqc6/pfDu lF1U6CTJLxSIELkAV1r3kyQwlNN0EysWvNvEvAqSRnqwwQCtpdvJUo2Vn0LJKn3e mxYEHxCu7xAph85jJklUTXJkWTs38XDLya10ihUyt1WtCVk6QNxJk7KoHSWubMXV Y3BTLX2Ly9Onr/G8nVIx2b1qFULiBL7/vNcjQ449syTLi/q14atTR30Q/ke2zk8s QMvT6g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=RUG1Tiz3AKJWRlE/UGPFTfSc92gGDvMN30yt3Y50i sI=; b=AHOXxYW9Sozim8xYej2lNdE/VV4MU5uCbgdSsEGcDg/Ocr1WVgvc/NAdr oBbykfjdYWBsrNCx0BJUJj0rFm2dPWvmvz1p7K7o3znm1OzzS65GuC8eMHvzj1I7 lyIs6GBGbSMEohAt7NvxQNg61MzNp43pHFewneoQJxdfS3EQu8ivHEKwABsV7fTk BJEXRIEe5VIXN/8YO0NSZmwtIOV2EuJ+mL6xTWOiYnf6jDPAk9PvmDgnCiMOcS9O +QxO7CWbfWs2eiPaeGkb8aflRDvwxUiWbmIOOboVXeED1VS9NqP+ik7XXQvxBkYC NW7598Ut7+d5ogrhA8gY51FAkMrmA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedruddvhedguddtjecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhm rghsucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenuc ggtffrrghtthgvrhhnpedugefgvdefudfftdefgeelgffhueekgfffhfeujedtteeutdej ueeiiedvffegheenucfkphepjeejrddufeegrddvtdefrddukeegnecuvehluhhsthgvrh fuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepthhhohhmrghssehmohhnjhgr lhhonhdrnhgvth X-ME-Proxy: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id 866413280060; Fri, 13 Nov 2020 12:58:41 -0500 (EST) From: Thomas Monjalon To: Alexander Kozyrev Cc: "dev@dpdk.org" , Raslan Darawsheh , Matan Azrad , Slava Ovsiienko , asafp@nvidia.com Date: Fri, 13 Nov 2020 18:58:36 +0100 Message-ID: <2341599.Mj1GBqBveQ@thomas> In-Reply-To: References: <20201106000442.26059-1-akozyrev@nvidia.com> <20201108042357.28542-1-akozyrev@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v2] net/mlx5: improve vMPRQ descriptors allocation locality X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > > There is a performance penalty for the replenish scheme used in vectorized Rx > > burst for both MPRQ and SPRQ. > > Mbuf elements are being filled at the end of the mbufs array and being > > replenished at the beginning. That leads to an increase in cache misses and the > > performance drop. > > The more Rx descriptors are used the worse the situation. > > > > Change the allocation scheme for vectorized MPRQ Rx burst: > > allocate new mbufs only when consumed mbufs are almost depleted (always > > have one burst gap between allocated and consumed indices). Keeping a small > > number of mbufs allocated improves cache locality and improves performance > > a lot. > > > > Unfortunately, this approach cannot be applied to SPRQ Rx burst routine. In > > MPRQ Rx burst we simply copy packets from external MPRQ buffers or attach > > these buffers to mbufs. > > In SPRQ Rx burst we allow the NIC to fill mbufs for us. > > Hence keeping a small number of allocated mbufs will limit NIC ability to fill as > > many buffers as possible. This fact offsets the advantage of better cache > > locality. > > > > Fixes: 0f20acbf5e ("net/mlx5: implement vectorized MPRQ burst") > > > > Signed-off-by: Alexander Kozyrev > Acked-by: Viacheslav Ovsiienko Applied in next-net-mlx, thanks.