From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 18E3BA054F; Wed, 7 Sep 2022 15:55:39 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id F173E40143; Wed, 7 Sep 2022 15:55:38 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 23852400D6 for ; Wed, 7 Sep 2022 15:55:38 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2877FOP8024241; Wed, 7 Sep 2022 06:55:33 -0700 Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2176.outbound.protection.outlook.com [104.47.56.176]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3jc6epxhnn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 07 Sep 2022 06:55:33 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PM8E6AAqYDjEp2JV7sEKKP5mMJuk63s0+yuHghFwcpPIEna3utmxO2uxoI6g8CNSfjIezE9ID5gqP0dlS1O47Gr6WW0oyJBMcs9XoI0u8EIkH5NT8LCvtA4YZ6VO2H427f/of6JGWEt1TbNSEyfz+5nitQn4EoXPGPhYUS1RJW2SJv2G9Qo/BvvHEcuUyFIrJ4uDAj6QfjCD8XnC7NnCNN0gVWaREc7mogj822C0X9lEZEGrBMaKH4ALz5F5cgkb0u7AiC1ydlH+pelP5mM9Uf4FNrulQBga9P0Ww2iCR8d5Av1TWbfiezijGuIcvmKJjHe97NRy/OtRCiLS2bV08w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kxA9Qt42FG6iPxW2x0FhN188VBeaKJDqqRbJmr+JMmc=; b=FZw+9YJUWLJqjNUlYkPupX2M2Kp8289l+360pufxVUPeBP6sHIVJz6kZcc+cXnQOCISV+R9T4EBakfMoNFE92MQRmfO9oogUhEni8J88vXvyOt7CtD9ugdygkBIXxZ0oEZSZBOhr2XieVqIrV+xkNqZX9KetWWJL4vLqh9vQjdKmgetXwAA8ZNXLrgK885w2pcK40SJ+znEPDHlns/JknoFarpSNYxcvNwXYsLopmPmhSsL6pIokK+BPqRaeTdwssFPcdrT3mOo4IhhGDKzCdPAsnIv/3DY/7yrjp5Hi10iLncuo9mAIN9h03EJW3c8u2jelqkAHfn+eqM0FWTqndQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kxA9Qt42FG6iPxW2x0FhN188VBeaKJDqqRbJmr+JMmc=; b=GblqUDQ/gUiEhopY2HFW2ced0ildhBTNSoz7+txg9pITv9vzSPcqtl/smEev7v9Oigv9UkBO65giuz2ay2DVZsYjqwQpIGJstvqpOGbrjmUDRrCpY9ZSdZOTRVggYsl8I4CkJbw+Abp+ZOu+eAqMjDnHk7oIFwIyyLUWow+ngDU= Received: from PH0PR18MB4425.namprd18.prod.outlook.com (2603:10b6:510:ef::13) by PH0PR18MB5191.namprd18.prod.outlook.com (2603:10b6:510:166::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.17; Wed, 7 Sep 2022 13:55:30 +0000 Received: from PH0PR18MB4425.namprd18.prod.outlook.com ([fe80::d1f:57fd:a78b:fd6e]) by PH0PR18MB4425.namprd18.prod.outlook.com ([fe80::d1f:57fd:a78b:fd6e%5]) with mapi id 15.20.5612.014; Wed, 7 Sep 2022 13:55:29 +0000 From: Shijith Thotton To: Honnappa Nagarahalli , Pavan Nikhilesh Bhagavatula , Bruce Richardson , =?iso-8859-1?Q?Morten_Br=F8rup?= CC: "dev@dpdk.org" , Jerin Jacob Kollanukkaran , "olivier.matz@6wind.com" , "stephen@networkplumber.org" , "thomas@monjalon.net" , nd Subject: RE: [EXT] Re: [PATCH v1 2/4] mbuf: add second dynamic field member for VA only build Thread-Topic: [EXT] Re: [PATCH v1 2/4] mbuf: add second dynamic field member for VA only build Thread-Index: AQHYu7riln90huFf1kSEWyxdDOYSya3GM+IAgADrsQCAAAGxgIAATlWAgAybhSA= Date: Wed, 7 Sep 2022 13:55:29 +0000 Message-ID: References: <20220829151626.2101336-3-sthotton@marvell.com> <98CBD80474FA8B44BF855DF32C47DC35D872D1@smartserver.smartshare.dk> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PH0PR18MB4425:EE_|PH0PR18MB5191:EE_ x-ms-office365-filtering-correlation-id: 9e8cb59c-49d0-47a0-44b4-08da90d8a007 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: W95I9DBK0aZBuVi0VkPe4X4S7Vo39Ran9QPSgNkQm61blkRV5D+QN5JagGKkqWvz82UM39Csi8EPnH1sJ7yDzKhBzBqEwqHQEh5nyB8Qo3KA4QProGYRPOegnyrTIt0VcArdGxohy99TzY59J2uAtBQni0QdvxCGGG1TNH36vGk3xz/ssPXh1YRvZPdZlwH8aUMcJOpZtomQhLPvX4CGeaUkqiYExPxAIiuk6hJIGNGdYjKwDEADsFEO/SItzS/rvSiPmYb89nD7aqZYvhoTe5ihIM3DBzwdiaZaPIjd0fqbfw6iQeSEOF+yHQbyodp/LDJ3QWFiBpoUj55S4bWfrgIzt8qdBwWBjoGrxjpRtyvvxSizMSrKnbF1MBgemP5ObWMdQ4/ffAh+7iNBNVRv4t2KsmLhkZgN+pWLMM8aiV4Usvwy92xSX/J8h+1s15cKVW1vNAEtJos1mP9X5k85fKhFo0dmLNA/tgUBPPmYHOQ6NYBUbWH1wMce8FY3cE76AI37G5/ra4rq+ecLOP2+6eDP2oKrsqS+7KqBK1ZCqJSPfk4UBCypqu4IaYYG3Kj+aA6hRjb1VQGvnJLfY4nYH6BScrvxClponB6hFyYJbR+x5P8NjC8YFbU4x9DEYzgdIg46ZIsem93w2A1YRRIFz2SbD6e/ORCZDWIMgA27YUF90Ucm/woujodVXdyy3b/4sLfPnPQwIIBfJGWKta/c6EakyLkhLf35yVSFj4k1OlYvJSjfEvI7aXslk+V71zP2BbR4YGRXpXFhYAUYTXYcNYv7tigZ74Gi4XxQecTr+to= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR18MB4425.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(346002)(366004)(39860400002)(396003)(136003)(376002)(186003)(122000001)(478600001)(38100700002)(966005)(6506007)(7696005)(83380400001)(26005)(71200400001)(2906002)(66574015)(41300700001)(8936002)(52536014)(76116006)(5660300002)(55016003)(66476007)(86362001)(66446008)(66946007)(66556008)(64756008)(9686003)(8676002)(4326008)(33656002)(38070700005)(316002)(54906003)(110136005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?B0YWxq3lVhxzzi0ZYTrNkOCKR0NWSa2L967JY44+XWETMEdpGuiyEC6S/3?= =?iso-8859-1?Q?xX6Mn3nwSehgRO/BGtbMC+1T16Mq5XHwoREQZQLBvkcImVy/npGrYYvMeE?= =?iso-8859-1?Q?UFXpY8VpSTTVq6rVrTfnmxCnTEs+EITz/B9H4Nc/27uDAdXcyTbFJ/QEwq?= =?iso-8859-1?Q?FvWtGpyaqT4H1woQLBQdCZgCLsDGGBXI/ZLUsuL6FLJnrXfZ/tSf3uxmC2?= =?iso-8859-1?Q?/U+1D1YgZ30eNGK18igCwoJ8a+kt6KvA137gKv5l/JdnvxlgG0SpCyQdfh?= =?iso-8859-1?Q?t4CXtb4/34sgsxBp2NqWeTbwfX2l0+sqsVFQ0NBYKGzTfX/uy4JsbLExh8?= =?iso-8859-1?Q?YHbBkanx/BMCj1IqE9k41WLWE0k2G4pGcs/zIHNPW/7BBKB+FE2knGGGen?= =?iso-8859-1?Q?VH2Vh7JS9G5Fk8qmOeVNW0ZhHmSZPCQ/CgMPombLDNG++Suz/wIk0RLu2V?= =?iso-8859-1?Q?QI4nxPA9qtH+/bRQvVqBYI2t7aR2ifPckHoVT5ARuXiNfBwV4jxsDgrIZd?= =?iso-8859-1?Q?5qkHC63EKfgewzbcXPXg83qInWy0A+4dsL32Q3qNW+71TmgiwiI/uC1A5s?= =?iso-8859-1?Q?lmZ2Nn/7KeE2yEIrWbq6vk1AYLYgfma+X8506iwEHQFkTE/2ag6gPY+vNI?= =?iso-8859-1?Q?Qe/F6xdK+/uQgrTQUK7+lfI5jgp/ynhn+FJ4Q4ldc9waw2D+luGC3vOWJs?= =?iso-8859-1?Q?0mUJ9Q+7a6ngSXKQDn39QxL5loK3mh4IJ83anZD/2feWw6O5yg4WZTX2Pa?= =?iso-8859-1?Q?gNrOP59oajs9/pBFM2Ogdvh14Ji3It+5nCwms3WQYQjARK08cLvqJvMDJy?= =?iso-8859-1?Q?EHZ8pxGoVcQ2v3vxPhjPc9QrylPRy349mhwQreRLROkn7LwfmS7uUnfBii?= =?iso-8859-1?Q?yVZSa7vb3D4r87s7S0B9KtIb4lrlhM1kkL8d6FWWJhcexcj1Y5DDmqn6rX?= =?iso-8859-1?Q?tjE1fhX2sfGiJsEuhcmXa4b3YkP9kefFyEQFcnD8S5Upxwuz2E39/geZyi?= =?iso-8859-1?Q?M0gjxzw7Pk3SAHwDwRGhyoycbWi+2QU1KwBnDG8YyDmVpe+BTgLrqraxcW?= =?iso-8859-1?Q?qIi6QLqV9kLnin2MCpdG4tQRlj6boR1IyM3JFfMNDHi7LnDm034MHknN02?= =?iso-8859-1?Q?0Dps/A2hBv3xp2hgGbplbZ5Eivz0fdlfvGsT1/OkmgWA5+KY2c20eyCIjQ?= =?iso-8859-1?Q?oTyr9r22HswQpJ/WYre1nGK4y29qjjwKDUjRLdNFz01mpwdfaAyk0UbkUu?= =?iso-8859-1?Q?833DQJvKF5+89hXQsxp0MVGZMVt7wBGrbkZzEB5Al4lLuXG97ORU7i1fNA?= =?iso-8859-1?Q?dAjZskuQ15RzStdsp5BqsLAhMvl4Q+kv3/lf3/Pw8mlk8qCpWDVDyEM2nL?= =?iso-8859-1?Q?Cc1sxj69G8HLlzwCSlugEWw8IM7GhC34zn4/sQRAEGa3S9qEfP4XDUoixK?= =?iso-8859-1?Q?2eFXaxa+Pjtbk73FCnFuGhTNSvwU84mmzxnfFsFf2R6UaM0X+LsL40mehS?= =?iso-8859-1?Q?s4bxzaBK/dtmwbwjuX4z6SCwkt2dmuklHwCYdYWmIM+U1T8c29yQ8kv8Q9?= =?iso-8859-1?Q?MEq/wuI6qwZWfn9DbxzEPJwxunXyvCBVjO6idKUgRqXkoDJ9jLP1WTn0K4?= =?iso-8859-1?Q?tk+BQTQtWHN5bF+y33P0B5Op7YLWyNBWu8?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR18MB4425.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9e8cb59c-49d0-47a0-44b4-08da90d8a007 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2022 13:55:29.8200 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: RZ0n/EnkM4++I/XaSDdo8COvgoNopV9Ntt3YxsbuZOdJmFOEqXI2Hs9MHsebcJhbjYrh9bpg3JFCzfJ8hTjFVA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR18MB5191 X-Proofpoint-ORIG-GUID: FVy35-bqKWUh6JQsIXEXf_mMLafSaWJd X-Proofpoint-GUID: FVy35-bqKWUh6JQsIXEXf_mMLafSaWJd X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-09-07_08,2022-09-07_02,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org >> > >> > ---------------------------------------------------------------------- >> > On Mon, Aug 29, 2022 at 08:32:20PM +0200, Morten Br=F8rup wrote: >> > > >> > > > From: Shijith Thotton [mailto:sthotton@marvell.com] >> > > > Sent: Monday, 29 August 2022 17.16 >> > > > >> > > > mbuf physical address field is not used in builds which only uses = VA. >> > > > It is used to expand the dynamic field area. >> > > > >> > > > Signed-off-by: Shijith Thotton >> > > > --- >> > > > lib/mbuf/rte_mbuf_core.h | 26 +++++++++++++++++--------- >> > > > lib/mbuf/rte_mbuf_dyn.c | 2 ++ >> > > > 2 files changed, 19 insertions(+), 9 deletions(-) >> > > > >> > > > diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h >> > > > index 81cb07c2e4..98ce62fd6a 100644 >> > > > --- a/lib/mbuf/rte_mbuf_core.h >> > > > +++ b/lib/mbuf/rte_mbuf_core.h >> > > > @@ -579,15 +579,23 @@ struct rte_mbuf { >> > > > RTE_MARKER cacheline0; >> > > > >> > > > void *buf_addr; /**< Virtual address of segment buffer= . >> > > > */ >> > > > - /** >> > > > - * Physical address of segment buffer. >> > > > - * This field is invalid if the build is configured to use only >> > > > - * virtual address as IOVA (i.e. RTE_IOVA_AS_VA is defined). >> > > > - * Force alignment to 8-bytes, so as to ensure we have the exact >> > > > - * same mbuf cacheline0 layout for 32-bit and 64-bit. This makes >> > > > - * working on vector drivers easier. >> > > > - */ >> > > > - rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t)); >> > > > + RTE_STD_C11 >> > > > + union { >> > > > + /** >> > > > + * Physical address of segment buffer. >> > > > + * This field is invalid if the build is configured to use >> > > > only >> > > > + * virtual address as IOVA (i.e. RTE_IOVA_AS_VA is >> > > > defined). >> > > > + * Force alignment to 8-bytes, so as to ensure we have the >> > > > exact >> > > > + * same mbuf cacheline0 layout for 32-bit and 64-bit. This >> > > > makes >> > > > + * working on vector drivers easier. >> > > > + */ >> > > > + rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t)); >> > > > + /** >> > > > + * Reserved for dynamic field in builds where physical >> > > > address >> > > > + * field is invalid. >> > > > + */ >> > > > + uint64_t dynfield2; >> > > > + }; >> > > > >> > > > /* next 8 bytes are initialised on RX descriptor rearm */ >> > > > RTE_MARKER64 rearm_data; >> > > >> > > I know that the intention here is to keep the rte_mbuf structure >> > > intact, >> > which will certainly improve the probability of getting this patch >> > series into DPDK. >> > > >> > > So, I will add a comment for the benefit of the other participants >> > > in the >> > discussion: >> > > >> > > With this patch, and in RTE_IOVA_AS_VA mode, it becomes possible to >> > move m->next into the first cache line, so rte_pktmbuf_prefree_seg() >> > does not have to touch the second cache line, thus potentially >> > improving performance by eliminating one cache miss per freed packet >> > segment. (I also recall someone mentioning that some PMDs set m->next >> > on RX... If that is the case, a cache miss per packet might also be >> > avoidable in those PMDs.) >> > > >> > > Obviously, moving m->next to the first cache line is not related to >> > > this patch >> > series, but would belong in a completely different patch. >> > > >> > >> > +1 to that, with the exception that if it is decided to move the next >> > pointer rather than use this as dynamic space, I think it *should* be >> > in this patch series, rather than mucking about with mbuf twice. :-) >> >> +1 When RTE_IOVA_AS_VA is set we can set mbuf->next as the dynamic field >> and move it to mbuf->buf_iova. >> mbuf->next write is one of the prominent hotspot in arm platforms. >+1 for reducing the cachelines that need to be touched =20 Added a new patch to move next pointer to first cache line in v2. Please re= view. https://patchwork.dpdk.org/project/dpdk/patch/20220907134340.3629224-4-stho= tton@marvell.com/