From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 893ADA04E7; Sun, 1 Nov 2020 17:38:24 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 0D9902BF9; Sun, 1 Nov 2020 17:38:23 +0100 (CET) Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com [66.111.4.230]) by dpdk.org (Postfix) with ESMTP id 5676D29C6 for <dev@dpdk.org>; Sun, 1 Nov 2020 17:38:20 +0100 (CET) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.nyi.internal (Postfix) with ESMTP id F261E580556; Sun, 1 Nov 2020 11:38:18 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Sun, 01 Nov 2020 11:38:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm2; bh= LBZPTd0JzsPn5NX8B3qgXCEDHTXYvKs0rcTK4jF1idc=; b=uoZ1hEQKwnCkN3Xy xPOOk/IikDe5I0EVXdKtIu8RCuMo04fKSrQE71VVkF9T9m46FkAkcRKo6daV6g8N nF8bQIPQGhwkcm+uhT7iQrcQ4sB+nex3WTAubiAyBUNT1CXsMOnPxWTYuoFCcZcG sKGLkYhpzbo8yJpKBM7I0mKmYz6mPm9sMaxbOUpqHPyz2oTgMANegMGJFISyrI3P CGa5HPj9fZKinxAVaMFlwGZZ5n58YUGytkPIimq0LWmLYHm2o0C0/yDXuUqQ4grC yPuTQGZc2iK1JiJzJkdG0S/Sz8OAA3KzhcUVf4nI8zpAORUMw2jSptHBdXvJpZMz XVkPTQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=LBZPTd0JzsPn5NX8B3qgXCEDHTXYvKs0rcTK4jF1i dc=; b=D51Zrnf0y8i0xzTZkCrlbtOcLWF9YWkfct1Xorkb/vg7mWlBupwhmXQqB 7b0w2lFur7mOukehBR833rf8x8P/gujPm30lvnSfxtUzgzHHG07nOmU2vZ3yAWmp ydPagT7LS8gLccuTaTdgdIemPaFdTgMjOrhN/cnbG47Mlq07kNVRkPb0cCKaOXZU 6y+7UEC7oI79Du5IMDhNDCqjmJ6tT0mcE2lm7MmCSlXjynLafXmXkpkIlNfdIC7f z7CeztwqyfbPj5Dql+utxx768PxblHpNkrcXksZFsQSlIECMiJSRAGsIYFGYH/6i cDw6w1LvJup+yL9rtYCYFj/5qVaqw== X-ME-Sender: <xms:eeSeX84vALR6MElrRnRB7Yx-vKR427NXUUCJ1mD0aR5BMkxNwFdU8g> <xme:eeSeX96K5cylihDSEOd1R_--zTOo4BmJIxuNmsEkxI9xJ_tzFo13oB-iZwS1IoEEI ENUGXGyeiHAcHgnHA> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrleelgdelgecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthhqredttddtudenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucggtf frrghtthgvrhhnpeefgeffiefhfeettdfhvdfgteekffffudekvedtvedtvdfgveeuudev gedvgeegtdenucfkphepjeejrddufeegrddvtdefrddukeegnecuvehluhhsthgvrhfuih iivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepthhhohhmrghssehmohhnjhgrlhho nhdrnhgvth X-ME-Proxy: <xmx:eeSeX7fsh_i5jGxJJ7UCaAIGTCC5Cy7WVHylRbJWodQDmF50yTKing> <xmx:eeSeXxKgIyD506OQjx-7qljnh8jsFbKkklcWftddtUz2N2mIKcPkWQ> <xmx:eeSeXwIY1xnpJULSeoug2tYAaI_xRTqJN4GQ85uru_KyXcBBVkldOQ> <xmx:euSeXyhvqDyWNov6H2tb2Ai3yCg1cVahD0G_I1Ez1q5n-oD6fbippw> Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id D9516306467E; Sun, 1 Nov 2020 11:38:15 -0500 (EST) From: Thomas Monjalon <thomas@monjalon.net> To: Morten =?ISO-8859-1?Q?Br=F8rup?= <mb@smartsharesystems.com> Cc: dev@dpdk.org, Ajit Khaparde <ajit.khaparde@broadcom.com>, "Ananyev, Konstantin" <konstantin.ananyev@intel.com>, Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>, dev@dpdk.org, "Yigit, Ferruh" <ferruh.yigit@intel.com>, david.marchand@redhat.com, "Richardson, Bruce" <bruce.richardson@intel.com>, olivier.matz@6wind.com, jerinj@marvell.com, viacheslavo@nvidia.com, honnappa.nagarahalli@arm.com, maxime.coquelin@redhat.com, stephen@networkplumber.org, hemant.agrawal@nxp.com, viacheslavo@nvidia.com, Matan Azrad <matan@nvidia.com>, Shahaf Shuler <shahafs@nvidia.com> Date: Sun, 01 Nov 2020 17:38:14 +0100 Message-ID: <3086227.yllCKDRCEA@thomas> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35C613CB@smartserver.smartshare.dk> References: <20201029092751.3837177-1-thomas@monjalon.net> <3458411.u7HY7OY5Un@thomas> <98CBD80474FA8B44BF855DF32C47DC35C613CB@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" Subject: Re: [dpdk-dev] [PATCH 15/15] mbuf: move pool pointer in hotterfirst half X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> 01/11/2020 10:12, Morten Br=F8rup: > One thing has always puzzled me: > Why do we use 64 bits to indicate which memory pool > an mbuf belongs to? > The portid only uses 16 bits and an indirection index. > Why don't we use the same kind of indirection index for mbuf pools? I wonder what would be the cost of indirection. Probably neglectible. I think it is a good proposal... =2E.. for next year, after a deprecation notice. > I can easily imagine using one mbuf pool (or perhaps a few pools) > per CPU socket (or per physical memory bus closest to an attached NIC), > but not more than 256 mbuf memory pools in total. > So, let's introduce an mbufpoolid like the portid, > and cut this mbuf field down from 64 to 8 bits. >=20 > If we also cut down m->pkt_len from 32 to 24 bits, Who is using packets larger than 64k? Are 16 bits enough? > we can get the 8 bit mbuf pool index into the first cache line > at no additional cost. I like the idea. It means we don't need to move the pool pointer now, i.e. it does not have to replace the timestamp field. > In other words: This would free up another 64 bit field in the mbuf struc= ture! That would be great! > And even though the m->next pointer for scattered packets resides > in the second cache line, the libraries and application knows > that m->next is NULL when m->nb_segs is 1. > This proves that my suggestion would make touching > the second cache line unnecessary (in simple cases), > even for re-initializing the mbuf. So you think the "next" pointer should stay in the second half of mbuf? I feel you would like to move the Tx offloads in the first half to improve performance of very simple apps. I am thinking the opposite: we could have some dynamic fields space in the first half to improve performance of complex Rx. Note: we can add a flag hint for field registration in this first half. > And now I will proceed out on a tangent with two more > independent thoughts, so feel free to ignore. >=20 > Consider a multi CPU socket system with one mbuf pool > per CPU socket, the NICs attached to each CPU socket > use an RX mbuf pool with RAM on the same CPU socket. > I would imagine that (re-)initializing these mbufs could be faster > if performed only on a CPU on the same socket. > If this is the case, mbufs should be re-initialized > as part of the RX preparation at ingress, > not as part of the mbuf free at egress. >=20 > Perhaps some microarchitectures are faster to compare > nb_segs=3D=3D0 than nb_segs=3D=3D1. > If so, nb_segs could be redefined to mean number of > additional segments, rather than number of segments.