From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 27B62A04DD; Wed, 28 Oct 2020 11:42:39 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 7EF03C9C2; Wed, 28 Oct 2020 11:42:36 +0100 (CET) Received: from mail-pl1-f194.google.com (mail-pl1-f194.google.com [209.85.214.194]) by dpdk.org (Postfix) with ESMTP id B4FC8C9C0 for ; Wed, 28 Oct 2020 11:42:34 +0100 (CET) Received: by mail-pl1-f194.google.com with SMTP id b12so2277593plr.4 for ; Wed, 28 Oct 2020 03:42:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=nTkOQSaPkSCizd8agPdTdLIgxPNGLqlAePgsZbEnBL0=; b=eMAj/TxA70ql3BP6ONJ6UDus52/0hpIO0PO5kpF2fhqK6+7ctp+Qu4XtT7lLlAoma5 CWjJKiGzdQsADPRyMlqbWHVxcCRyIHD/gnCFVCmH93YIWMLP2x+4Ks747qXMeGhojJCi 9DyxYW4tv1lETt2JvWp13P1zji+M8YCzrzR+VxaYBsmvHPAVx4D3s1Zgh2u4qmGKbNpX Jw7c0T67KVN8vycZHUD9ygCqnFqwj9NZtdz7lNBc1I6SNgJWFfp6IeYuGZqCYRaEUCub VQ+bDQQLtcUXyq2Uq5w6E8K57+SKJNt5zNMFSUk/JlqKyLe0ob2kAppyByvY9SKjHDnS bdLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=nTkOQSaPkSCizd8agPdTdLIgxPNGLqlAePgsZbEnBL0=; b=EBPvPOImeZEfE6uiPTj96KrboiRorC9bComVWaNT2jwML//njajgmkUCs8V25fcLT2 j7DkHkwhuZW3SWr9Hqfxvs9RL7FHxg/tDjqBDH55kSV1d2fNirWQDWa+sSa+cMaKXgw8 gCtXqihL9oUTRZj2I/NDo09P/hOr8pP7LP8+HLcO1phcU9r48yo90zNXjbO+5s+Aotsh KHGGNZnC/ToTYAcZvi9ko1pEGla+/yFPQ5SUnYYgc8j7wQP/p8i6S8Ip3LkGLIO1koXJ PTUUdh1h6K6iJe9PvZYMyqAS2OfunHE2SmBZoKzerbOjmAS3TYhGaPaO8q0WKLHWnvEW WDPA== X-Gm-Message-State: AOAM5322QjpNmydW8D5llQJS+vERs0P7WtbkaurV0UORjP/XhuxIT24o R1j8MjtysoPPS7888tQxS0M= X-Google-Smtp-Source: ABdhPJxSxUeJabmoyHMIK3AXjMiSowsO0N8LkFx+VtgglqG8mKi7ftUEHn/bzEPZI4tC2Zg+2nF5qg== X-Received: by 2002:a17:90a:2e03:: with SMTP id q3mr6444955pjd.118.1603881752633; Wed, 28 Oct 2020 03:42:32 -0700 (PDT) Received: from gmail.com ([1.6.215.26]) by smtp.gmail.com with ESMTPSA id t28sm5312990pfe.2.2020.10.28.03.42.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Oct 2020 03:42:32 -0700 (PDT) Date: Wed, 28 Oct 2020 16:12:23 +0530 From: Nithin Dabilpuram To: "Van Haaren, Harry" Cc: Thomas Monjalon , Pavan Nikhilesh , Jerin Jacob , Ruifeng Wang , "Richardson, Bruce" , "Ananyev, Konstantin" , "kirankumark@marvell.com" , "dev@dpdk.org" , "david.marchand@redhat.com" , "olivier.matz@6wind.com" Message-ID: References: <3705096.qAGAdPRMt2@thomas> <20201028093003.29564-1-ndabilpuram@marvell.com> <2153992.0QQoBXgI6F@thomas> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [dpdk-dev] [PATCH v4] node: switch IPv4 metadata to dynamic mbuf field X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, Oct 28, 2020 at 10:24:01AM +0000, Van Haaren, Harry wrote: > > -----Original Message----- > > From: dev On Behalf Of Thomas Monjalon > > Sent: Wednesday, October 28, 2020 10:09 AM > > To: Nithin Dabilpuram > > Cc: Pavan Nikhilesh ; Jerin Jacob > > ; Ruifeng Wang ; Richardson, Bruce > > ; Ananyev, Konstantin > > ; kirankumark@marvell.com; dev@dpdk.org; > > david.marchand@redhat.com; olivier.matz@6wind.com > > Subject: Re: [dpdk-dev] [PATCH v4] node: switch IPv4 metadata to dynamic mbuf > > field > > > > 28/10/2020 10:30, Nithin Dabilpuram: > > > From: Thomas Monjalon > > > > > > The node_mbuf_priv1 was stored in the deprecated mbuf field udata64. > > > It is moved to a dynamic field in order to allow removal of udata64. > > > > > > Signed-off-by: Thomas Monjalon > > > Signed-off-by: Nithin Dabilpuram > > [...] > > > + IP4_LOOKUP_NODE_PRIV1_OFF(node->ctx) = > > node_mbuf_priv1_dynfield_offset; > > > > That's interesting. > > You copy the offset in the node context for better performance. > > How much is it better than with global offset variable? > > How much it decreases compared to a static mbuf field? > > Also interested in this topic, I'll offer the logical/theory point of view; > > With a static field, the offset into the mbuf can be encoded in the instruction > stream, meaning there are no d-cache loads to identify particular dynamic field. > > With a static/global variable, the cache line where the value resides is presumably > not hot in cache per burst (assuming an application that does significant work, so not > in cache since last burst). Hence overhead estimate could be 1x cache line load per burst. > > With the data copied into the node, the offset is presumably on a hot cache line as the > node is using other data-members of its context. As a result, perhaps a cold static cache > line load is converted to a hot node-context line re-use. > > Real world overhead likely depends on A) does the application cache-trash enough to make > the static/global line fall out of cache - causing perf degradation due to reload, and B) does > the node->ctx still fit in the same number of lines as before if the value is copied there. Agreed, node->ctx is already referred to get other data (lpm pointer). So referening another 4 bytes might even convert that to load pair which is at no extra cost. Number's wise, it decreases by ~1.4 % from static mbuf field to global offset variable and it decreases by ~1% from static mbuf field to node context field cached per process call