From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 45D83A04DD; Wed, 28 Oct 2020 11:44:04 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1E097C9C0; Wed, 28 Oct 2020 11:44:03 +0100 (CET) Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com [66.111.4.221]) by dpdk.org (Postfix) with ESMTP id 30692C8F8 for <dev@dpdk.org>; Wed, 28 Oct 2020 11:44:01 +0100 (CET) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.nyi.internal (Postfix) with ESMTP id BB218580404; Wed, 28 Oct 2020 06:43:59 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Wed, 28 Oct 2020 06:43:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding:content-type; s=fm2; bh= Siogu/PA/lioP9GTXLpWcMm1SqSYdzFTOAJDQR81BFM=; b=nqPyLvhJGoLWkvJc 2lgfnxQ6tcOzB9v1gWCQX1k3xKJTCCL6aF+E6NcHtFMMAfemgq7sXDF9kAO+XeG5 Dt9LT6jLF62Ec/xw4kvzL34CwN6W7mDVE42yqjr2ddZIkUOQdo3wQQfmEF8eCpVb a8HQl+mCMIw0vV184MMM3PbUgCASjveD1J/x6Bx3tc63Z6yj7Yj6LRAFIx4rDjjl 4jDfqVWnxmR8muLa/LEmOUX2PdqaXqD74Yuz1Nl7svuCaMVUNvI2m3ZM7eIg+Hd6 /B3WmS5kkRp38pwcE+d0htvVn2mEw9m9TBOyQ8MboDyQ+P1Eceugo8aTNmtGxaIC Ck+Fpw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=Siogu/PA/lioP9GTXLpWcMm1SqSYdzFTOAJDQR81B FM=; b=lMxvZBbwTb0ZewXS/T6rQsNySWHbFCFE5zzlwhMDRa12KTyjPX+EtMwke IRufm3yn0S8MtXIP/M58EkfERT9YbQ9sNDW+3krS8WbnefyWzGHTGYh9/a/aC8Xx WmsF+Y1Db60bdeX4YlCeDZeOWHpZQ5Rdcw3uUWpbOxQJzdlTMlL2WZ4kwcy2eM6a PPE15DuRLdSDmAMYbhjHlRszZ4vFaOhhiiOQKyAnOS7DhJviUuwcA+Ic6Mg0Tk5s yoHdxEwiBtLNyeAOLLcJ8+wxHLOIlEaewWJS6HCom22E0H9ct5ilwzR08fA7Qm5p 39KgwmNDt26UK8oDHg+Q0wLb3g6fg== X-ME-Sender: <xms:bkuZX1IT-6zWmoLdrl-0Z75ZPnLv6n_6DalqFRgosI03easADLb4MQ> <xme:bkuZXxIC0W35lchnmVKmKbU1hVAAdR6XiAUUcrrHfJtRn7DLZ9veKfRQKbSAfMEE4 _vYx5Du_ECDsY37bQ> X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrledugddukecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhmrghs ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucggtf frrghtthgvrhhnpedugefgvdefudfftdefgeelgffhueekgfffhfeujedtteeutdejueei iedvffegheenucfkphepjeejrddufeegrddvtdefrddukeegnecuvehluhhsthgvrhfuih iivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepthhhohhmrghssehmohhnjhgrlhho nhdrnhgvth X-ME-Proxy: <xmx:bkuZX9tl0gXG5dJ55UkUe5zxqkYDdFQrnEHMQASNHkA89u6Grh5eZw> <xmx:bkuZX2aM35bHe_tdtgUwFr_9OBUWZsQQXLOCLTgNjc4PHG0Rw2q6-A> <xmx:bkuZX8YvnjMea2Be_hsHmaG7I0EW272gGDMDRJf8Nxv5N5fwU7yaEw> <xmx:b0uZXyN7Ip5LKSaoweONn5IMJDMLKPL2UxmKx8drTEslF7tIKCALKw> Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id CACCF3064682; Wed, 28 Oct 2020 06:43:57 -0400 (EDT) From: Thomas Monjalon <thomas@monjalon.net> To: "Van Haaren, Harry" <harry.van.haaren@intel.com>, Nithin Dabilpuram <nithind1988@gmail.com> Cc: Pavan Nikhilesh <pbhagavatula@marvell.com>, Jerin Jacob <jerinj@marvell.com>, Ruifeng Wang <ruifeng.wang@arm.com>, "Richardson, Bruce" <bruce.richardson@intel.com>, "Ananyev, Konstantin" <konstantin.ananyev@intel.com>, "kirankumark@marvell.com" <kirankumark@marvell.com>, "dev@dpdk.org" <dev@dpdk.org>, "david.marchand@redhat.com" <david.marchand@redhat.com>, "olivier.matz@6wind.com" <olivier.matz@6wind.com> Date: Wed, 28 Oct 2020 11:43:56 +0100 Message-ID: <2588040.FO0brCVOmV@thomas> In-Reply-To: <X5lLD/E0eUXkDw68@gmail.com> References: <3705096.qAGAdPRMt2@thomas> <BYAPR11MB31435E2095DB6E93D35C0381D7170@BYAPR11MB3143.namprd11.prod.outlook.com> <X5lLD/E0eUXkDw68@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v4] node: switch IPv4 metadata to dynamic mbuf field X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> 28/10/2020 11:42, Nithin Dabilpuram: > On Wed, Oct 28, 2020 at 10:24:01AM +0000, Van Haaren, Harry wrote: > > From: Thomas Monjalon > > > 28/10/2020 10:30, Nithin Dabilpuram: > > > > From: Thomas Monjalon <thomas@monjalon.net> > > > > > > > > The node_mbuf_priv1 was stored in the deprecated mbuf field udata64. > > > > It is moved to a dynamic field in order to allow removal of udata64. > > > > > > > > Signed-off-by: Thomas Monjalon <thomas@monjalon.net> > > > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com> > > > [...] > > > > + IP4_LOOKUP_NODE_PRIV1_OFF(node->ctx) = > > > node_mbuf_priv1_dynfield_offset; > > > > > > That's interesting. > > > You copy the offset in the node context for better performance. > > > How much is it better than with global offset variable? > > > How much it decreases compared to a static mbuf field? > > > > Also interested in this topic, I'll offer the logical/theory point of view; > > > > With a static field, the offset into the mbuf can be encoded in the instruction > > stream, meaning there are no d-cache loads to identify particular dynamic field. > > > > With a static/global variable, the cache line where the value resides is presumably > > not hot in cache per burst (assuming an application that does significant work, so not > > in cache since last burst). Hence overhead estimate could be 1x cache line load per burst. > > > > With the data copied into the node, the offset is presumably on a hot cache line as the > > node is using other data-members of its context. As a result, perhaps a cold static cache > > line load is converted to a hot node-context line re-use. > > > > Real world overhead likely depends on A) does the application cache-trash enough to make > > the static/global line fall out of cache - causing perf degradation due to reload, and B) does > > the node->ctx still fit in the same number of lines as before if the value is copied there. > > Agreed, node->ctx is already referred to get other data (lpm pointer). So > referening another 4 bytes might even convert that to load pair which is at > no extra cost. > > Number's wise, > it decreases by ~1.4 % from static mbuf field to global offset variable > and it decreases by ~1% from static mbuf field to node context field > cached per process call OK thanks for providing these numbers.