From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 45D83A04DD;
	Wed, 28 Oct 2020 11:44:04 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
	by dpdk.org (Postfix) with ESMTP id 1E097C9C0;
	Wed, 28 Oct 2020 11:44:03 +0100 (CET)
Received: from new1-smtp.messagingengine.com (new1-smtp.messagingengine.com
 [66.111.4.221]) by dpdk.org (Postfix) with ESMTP id 30692C8F8
 for <dev@dpdk.org>; Wed, 28 Oct 2020 11:44:01 +0100 (CET)
Received: from compute2.internal (compute2.nyi.internal [10.202.2.42])
 by mailnew.nyi.internal (Postfix) with ESMTP id BB218580404;
 Wed, 28 Oct 2020 06:43:59 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
 by compute2.internal (MEProxy); Wed, 28 Oct 2020 06:43:59 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h=
 from:to:cc:subject:date:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding:content-type; s=fm2; bh=
 Siogu/PA/lioP9GTXLpWcMm1SqSYdzFTOAJDQR81BFM=; b=nqPyLvhJGoLWkvJc
 2lgfnxQ6tcOzB9v1gWCQX1k3xKJTCCL6aF+E6NcHtFMMAfemgq7sXDF9kAO+XeG5
 Dt9LT6jLF62Ec/xw4kvzL34CwN6W7mDVE42yqjr2ddZIkUOQdo3wQQfmEF8eCpVb
 a8HQl+mCMIw0vV184MMM3PbUgCASjveD1J/x6Bx3tc63Z6yj7Yj6LRAFIx4rDjjl
 4jDfqVWnxmR8muLa/LEmOUX2PdqaXqD74Yuz1Nl7svuCaMVUNvI2m3ZM7eIg+Hd6
 /B3WmS5kkRp38pwcE+d0htvVn2mEw9m9TBOyQ8MboDyQ+P1Eceugo8aTNmtGxaIC
 Ck+Fpw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:content-transfer-encoding:content-type
 :date:from:in-reply-to:message-id:mime-version:references
 :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
 :x-sasl-enc; s=fm1; bh=Siogu/PA/lioP9GTXLpWcMm1SqSYdzFTOAJDQR81B
 FM=; b=lMxvZBbwTb0ZewXS/T6rQsNySWHbFCFE5zzlwhMDRa12KTyjPX+EtMwke
 IRufm3yn0S8MtXIP/M58EkfERT9YbQ9sNDW+3krS8WbnefyWzGHTGYh9/a/aC8Xx
 WmsF+Y1Db60bdeX4YlCeDZeOWHpZQ5Rdcw3uUWpbOxQJzdlTMlL2WZ4kwcy2eM6a
 PPE15DuRLdSDmAMYbhjHlRszZ4vFaOhhiiOQKyAnOS7DhJviUuwcA+Ic6Mg0Tk5s
 yoHdxEwiBtLNyeAOLLcJ8+wxHLOIlEaewWJS6HCom22E0H9ct5ilwzR08fA7Qm5p
 39KgwmNDt26UK8oDHg+Q0wLb3g6fg==
X-ME-Sender: <xms:bkuZX1IT-6zWmoLdrl-0Z75ZPnLv6n_6DalqFRgosI03easADLb4MQ>
 <xme:bkuZXxIC0W35lchnmVKmKbU1hVAAdR6XiAUUcrrHfJtRn7DLZ9veKfRQKbSAfMEE4
 _vYx5Du_ECDsY37bQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrledugddukecutefuodetggdotefrodftvf
 curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
 uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc
 fjughrpefhvffufffkjghfggfgtgesthfuredttddtvdenucfhrhhomhepvfhhohhmrghs
 ucfoohhnjhgrlhhonhcuoehthhhomhgrshesmhhonhhjrghlohhnrdhnvghtqeenucggtf
 frrghtthgvrhhnpedugefgvdefudfftdefgeelgffhueekgfffhfeujedtteeutdejueei
 iedvffegheenucfkphepjeejrddufeegrddvtdefrddukeegnecuvehluhhsthgvrhfuih
 iivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepthhhohhmrghssehmohhnjhgrlhho
 nhdrnhgvth
X-ME-Proxy: <xmx:bkuZX9tl0gXG5dJ55UkUe5zxqkYDdFQrnEHMQASNHkA89u6Grh5eZw>
 <xmx:bkuZX2aM35bHe_tdtgUwFr_9OBUWZsQQXLOCLTgNjc4PHG0Rw2q6-A>
 <xmx:bkuZX8YvnjMea2Be_hsHmaG7I0EW272gGDMDRJf8Nxv5N5fwU7yaEw>
 <xmx:b0uZXyN7Ip5LKSaoweONn5IMJDMLKPL2UxmKx8drTEslF7tIKCALKw>
Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184])
 by mail.messagingengine.com (Postfix) with ESMTPA id CACCF3064682;
 Wed, 28 Oct 2020 06:43:57 -0400 (EDT)
From: Thomas Monjalon <thomas@monjalon.net>
To: "Van Haaren, Harry" <harry.van.haaren@intel.com>,
 Nithin Dabilpuram <nithind1988@gmail.com>
Cc: Pavan Nikhilesh <pbhagavatula@marvell.com>,
 Jerin Jacob <jerinj@marvell.com>, Ruifeng Wang <ruifeng.wang@arm.com>,
 "Richardson, Bruce" <bruce.richardson@intel.com>, "Ananyev,
 Konstantin" <konstantin.ananyev@intel.com>,
 "kirankumark@marvell.com" <kirankumark@marvell.com>,
 "dev@dpdk.org" <dev@dpdk.org>,
 "david.marchand@redhat.com" <david.marchand@redhat.com>,
 "olivier.matz@6wind.com" <olivier.matz@6wind.com>
Date: Wed, 28 Oct 2020 11:43:56 +0100
Message-ID: <2588040.FO0brCVOmV@thomas>
In-Reply-To: <X5lLD/E0eUXkDw68@gmail.com>
References: <3705096.qAGAdPRMt2@thomas>
 <BYAPR11MB31435E2095DB6E93D35C0381D7170@BYAPR11MB3143.namprd11.prod.outlook.com>
 <X5lLD/E0eUXkDw68@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
Subject: Re: [dpdk-dev] [PATCH v4] node: switch IPv4 metadata to dynamic
	mbuf field
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>

28/10/2020 11:42, Nithin Dabilpuram:
> On Wed, Oct 28, 2020 at 10:24:01AM +0000, Van Haaren, Harry wrote:
> > From: Thomas Monjalon
> > > 28/10/2020 10:30, Nithin Dabilpuram:
> > > > From: Thomas Monjalon <thomas@monjalon.net>
> > > >
> > > > The node_mbuf_priv1 was stored in the deprecated mbuf field udata64.
> > > > It is moved to a dynamic field in order to allow removal of udata64.
> > > >
> > > > Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
> > > > Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
> > > [...]
> > > > +	IP4_LOOKUP_NODE_PRIV1_OFF(node->ctx) =
> > > node_mbuf_priv1_dynfield_offset;
> > > 
> > > That's interesting.
> > > You copy the offset in the node context for better performance.
> > > How much is it better than with global offset variable?
> > > How much it decreases compared to a static mbuf field?
> > 
> > Also interested in this topic, I'll offer the logical/theory point of view;
> > 
> > With a static field, the offset into the mbuf can be encoded in the instruction
> > stream, meaning there are no d-cache loads to identify particular dynamic field.
> > 
> > With a static/global variable, the cache line where the value resides is presumably
> > not hot in cache per burst (assuming an application that does significant work, so not
> > in cache since last burst). Hence overhead estimate could be 1x cache line load per burst.
> > 
> > With the data copied into the node, the offset is presumably on a hot cache line as the
> > node is using other data-members of its context. As a result, perhaps a cold static cache
> > line load is converted to a hot node-context line re-use. 
> > 
> > Real world overhead likely depends on A) does the application cache-trash enough to make
> > the static/global line fall out of cache - causing perf degradation due to reload, and B) does
> > the node->ctx still fit in the same number of lines as before if the value is copied there.
> 
> Agreed, node->ctx is already referred to get other data (lpm pointer). So
> referening another 4 bytes might even convert that to load pair which is at
> no extra cost.
> 
> Number's wise, 
> it decreases by ~1.4 % from static mbuf field to global offset variable 
> and it decreases by ~1% from static mbuf field to node context field
> cached per process call

OK thanks for providing these numbers.