From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E36CEA034C; Tue, 30 Aug 2022 10:42:09 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 14215427ED; Tue, 30 Aug 2022 10:42:06 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 9391E40F17 for ; Tue, 30 Aug 2022 10:42:04 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 27U1UVLl005534; Tue, 30 Aug 2022 01:42:00 -0700 Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2169.outbound.protection.outlook.com [104.47.57.169]) by mx0b-0016f401.pphosted.com (PPS) with ESMTPS id 3j7jsnbaej-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 30 Aug 2022 01:42:00 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H1VuYye/GbLtTuY1jcIaH0n9yr87CpwhwlZOTe+tAhgnmvCS4bMVXyxPJm6+lrmoG84Ncj97MZ+7wWi64rCDGmuMjzuxtAv0q2AU03d3tlKjhqV5GeUKqf3fKzMkG7U2rO27/1NNuLEam6aq+1nZ/ZSPx3R/ZR/VKkhuhFfJZbYqXR3rQjQdGfw/UXGaQa8lvmHKFDnqloAygk5FSVS8P+9tyQIpxon33y+Sm7qmCDqCnLRhgxoJDGGJfZmbNQIoP9Om1LeT6CeKXdNzHScQbWiGJSBrpJWd34UZneEyqnaFWdzpcpTymHXuaxtmVX7jU3qg2JBDfHBSf0gLAtbk+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1EMT0gmiuHSRsqrRbYfighFDp3kbbJyOALgYS8SPHmo=; b=KnunBcj1u5Ouajxd+BVBNY7+xouEGLugwahKzHuzOO3sAon3GVE09cTLIoLy0WguYAof6zSFss0WXrTxnNEHVdlyMOH86l1iYRmaSZZ8yYM+4lIqVSQBDKterKRsMHGQsa/Tej3g/S/LiLIAeADuJ672UwNU074S9WdFKd3Wjido0X7xJbDaFUTsUvG9lUu8c1lf91xKGkwwyedNia61RS6XCiLgY9IU7kxMeRZ8Gr1qNG2Q9U0YzN71Ax995WKTf3iwCRodA4EciTtVnyj8RCg3nmF71ln2d26obGt3Vx/wBD2LlgMq54OsHWhIFQK673d0kwAfBdU+S3/aCkYKFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1EMT0gmiuHSRsqrRbYfighFDp3kbbJyOALgYS8SPHmo=; b=HRYN/YZ9thT/ocD+uXo9K7X79P66Q+Q+4mTJI/4hTIr+PQXiWml3GdUmOpK8z29Fqk/RuVENbPAw+1SUkXNh1jMrtHZE3WSZujPOZVzWxu0R9ecyPF6+/ytpg+Y/n8FxHnFviH2lQVFiifXhUgZfYyzhJJVOAR1SV3tJXjz6AjQ= Received: from PH0PR18MB4086.namprd18.prod.outlook.com (2603:10b6:510:3::9) by MW5PR18MB5119.namprd18.prod.outlook.com (2603:10b6:303:1cc::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.21; Tue, 30 Aug 2022 08:41:57 +0000 Received: from PH0PR18MB4086.namprd18.prod.outlook.com ([fe80::3c67:e351:cf7a:c1fa]) by PH0PR18MB4086.namprd18.prod.outlook.com ([fe80::3c67:e351:cf7a:c1fa%2]) with mapi id 15.20.5566.021; Tue, 30 Aug 2022 08:41:57 +0000 From: Pavan Nikhilesh Bhagavatula To: Bruce Richardson , =?iso-8859-1?Q?Morten_Br=F8rup?= CC: Shijith Thotton , "dev@dpdk.org" , "Honnappa.Nagarahalli@arm.com" , Jerin Jacob Kollanukkaran , "olivier.matz@6wind.com" , "stephen@networkplumber.org" , "thomas@monjalon.net" Subject: RE: [EXT] Re: [PATCH v1 2/4] mbuf: add second dynamic field member for VA only build Thread-Topic: [EXT] Re: [PATCH v1 2/4] mbuf: add second dynamic field member for VA only build Thread-Index: AQHYu9WxSeCqnz8pKEu1BzZ72QDg5a3HH10AgAAAy5A= Date: Tue, 30 Aug 2022 08:41:57 +0000 Message-ID: References: <20220829151626.2101336-3-sthotton@marvell.com> <98CBD80474FA8B44BF855DF32C47DC35D872D1@smartserver.smartshare.dk> In-Reply-To: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 35e46da2-78d4-4212-3a97-08da8a637f99 x-ms-traffictypediagnostic: MW5PR18MB5119:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: eB30F0U14xUgQAEWTT0DxUc5ejNZsXkIrdPBQCyThMEG2KNLxWsW1pePaQCJhuvGj/jxcV8syXHJC8GmQjciu3QTMSeSWKRkKvB0+5rtSYWzitn1R+vLWUN9y9gw+xs0AbPFqHhvQCs1rOBZNg7WXx/ZrBgPesEehIVVGxS8OZAzFMJ40Iuedjk8JkFM5/f1dFt84ICnVE8fCZtsQU/YOnFaZPCfX2r0chW4JXR/wsZyJd+7DB7SHZ/xjo6Tl8AkWZMSXqY5HhxNE4owEk8NBcLa8FgWk0Y9U5gP2VkXg/8lyv6hGMOTzQwyrkqjqhCcYBNgNmvDUPUjJo3N4bQe4D3tts3/yALGf+k3nijqY7V9vAGusMWYIgw2Pw1JcGG1KovStRiRHy+QPIdQEpTj7TqB0NgiWKOzgOeC0gE7n65W5CjFd9ptXU0VkUMy/QYaMPiFO4TE4UwWFtq380nA9iRnwNSswsq2Q20nOzB/3Dss5keHCypQghg1q7IwUStFaEJZNasMGlbiMvSCt8aKFfC42ZKS/XmEehT8O6xHj9L1P+ySoZwc7rTwqUKmtw3qioHP1hc3MZ8Ol4GJtZix8aEs/1+NxUxYGnhzzG8qhPH0TcRMQS7OTBcPMjtrSooe/V2Lnr3NsSYvC3xcNYeaibTcv2ibucAuF8jtW5KrD/5+qeBd9mAIzn4/H8DgLH5wrw3Wjyye5g55g8/dZ85bd+UHf57CfY9ccmbxTNw5YcybDrOLDID3SJDxaPNb9IC/VmfH4zk2yNyzAPxaw3F70g== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR18MB4086.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(376002)(346002)(39860400002)(396003)(366004)(136003)(26005)(71200400001)(8676002)(110136005)(54906003)(38070700005)(38100700002)(478600001)(41300700001)(53546011)(186003)(9686003)(66574015)(6506007)(7696005)(55016003)(86362001)(33656002)(52536014)(316002)(4326008)(5660300002)(66556008)(66946007)(66446008)(66476007)(64756008)(76116006)(2906002)(83380400001)(122000001)(8936002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?qF8ESK4uzqTzZT7EMMQ1cqFHEefxZvu3ChAvzOqV79PHdrPCSqJt46IjEv?= =?iso-8859-1?Q?F8D6jOxXGLxrwE8jbN0Qtvi9CJo+37RC0RUq4J2OEXSiJM8zXHeBr3eWrD?= =?iso-8859-1?Q?c2YulT3E440b+l5P03F5xHJOmNyJZaPAEuvmU2VYwTWJKNg73pOZ3Nlx6Y?= =?iso-8859-1?Q?F5rH47+nbBPDqJVreloECFLKelEgKHtTvtVblW30HqkppzfQ+ELbHtUp8/?= =?iso-8859-1?Q?mfqwwmPgWtCY2FaYvHRpMs/LAqpsQDiFQsfnMWJUmsw0//tzv8XbmN687Y?= =?iso-8859-1?Q?o45elJRrO1qSxYhXlhTovdmK0g8cY+vizlbrvbtr0BOxg9/wvImgyEAUxB?= =?iso-8859-1?Q?njc0/dqh2DzzxxuOiUxf72VbQAZmEJVuIt2rd/5faKmz5VV7ncrGBje2x0?= =?iso-8859-1?Q?l8opxmdVgiDM61akuhh6CJSMXohLwxCvhmQ5hnDOV9w/N9LBWsNZWzceUO?= =?iso-8859-1?Q?vvwi8W3K+I/MvY17TZdpS1pFdn4JcMgQRSdMc0GAeWD3F756y/VeIW+ToH?= =?iso-8859-1?Q?qCVjUNMkCHtnC4vvNwt3OC15gSg3yQpNHQ4Za+2VsCJpk5mXnb8xKscvRS?= =?iso-8859-1?Q?sllPXjumF6ZMaF/80yuZjUBctK8COPqdSbUlDQEpA01CKrPGxeWYKA79uh?= =?iso-8859-1?Q?yuZ8XKKJcyH+Pd4lMp42m4JvqIp5OaKCKCyll0VYvcosS3HZeCavXsKHUX?= =?iso-8859-1?Q?6LY+h02TTw9Ukmnq5zwDdxsNsKOPXdCL20S12mM6GnSltxI8OR8Nw/3aRz?= =?iso-8859-1?Q?eqMHST48Ce8jJCDjxO4qL4qCvebhbF9ci7EShghcOsP/zXjCkcGeJ0TIL2?= =?iso-8859-1?Q?t/2084lykcaKXKgn/0vxtxwFGaQwqPlc0Qnl1OWimwKxPac9egzBirb6C2?= =?iso-8859-1?Q?kYZEkKjn+13DzgsxQcTNUqMLBEAJKRnHyG9zF5pmgucVx56Talo7vnVE8b?= =?iso-8859-1?Q?AFBMkOzicV0Ovgt6XP5nb3Hv8pFKaw028dZvoizFDMxaZUsG2z/MXdFTvq?= =?iso-8859-1?Q?mdST41nEF2iJg0TNszPBmHDVN90VbqLmfC/VCMIvE66Y9T62f9t2ttklax?= =?iso-8859-1?Q?kNeI0ADOrRNbCWNqELD8m43X2YpOPwhqlmaRMW3hGII4EBa/N5B8G66cs1?= =?iso-8859-1?Q?UPUlSkWj6f4zkNdPay4cBkqZYgO69oHmQeVafWszz+Z9ByzTIb0gYvU1p1?= =?iso-8859-1?Q?dLMlNxTV3hS1qk2ByPXjq1vhPlk/iv+pVsa7RhJtHp2+JX79/EzMqKccqa?= =?iso-8859-1?Q?MpHy2UjgUyhi+sI1UluihgCWVFa1fWRmiVxMw2pdW5aexN2/QhbScKmVqt?= =?iso-8859-1?Q?3JQZodnywmuyHQCWtYsfAgcQnmC2lXAN8S4VSbsPLJdXwqINLnuHPC41dq?= =?iso-8859-1?Q?Vbpc+3J+dGCoXlB/ZF6tNQLw/iXMk2hFOLrdMYXaYtaxkg2puj99O9blUq?= =?iso-8859-1?Q?0Slj5zQpCeqksFq14aM+tsnXtcrTP+iDTExQ1imgKqPqMJQBMK11MGznVw?= =?iso-8859-1?Q?ryn5+1obf6XBqfSewgoC5Y7FRaeiDWQHEkzfIs8hzCsYu6F79JxpMRNxNT?= =?iso-8859-1?Q?79FutCMxOd6inLPelr5hkBUMWUBd6Vawy1ZgPD8DxIUAtTrDOsqJp+bhDJ?= =?iso-8859-1?Q?aQ1Y8AcqeyBfnLs4HrRjUDdB04moQ+kG2+?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR18MB4086.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 35e46da2-78d4-4212-3a97-08da8a637f99 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Aug 2022 08:41:57.3334 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Oby3FT/+tjzJNImExl3PHK9YZZxX6STfdQpa4ZDKTiZnxc4B5hCdNYbT1xXJoDTwPEkunajDsvkSkkgdlUZpoNgUIdgehVYU0QRxMKpDJuo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR18MB5119 X-Proofpoint-ORIG-GUID: _f1J2OFQDmZAXtJma8Ihec1Z94uJ52Sa X-Proofpoint-GUID: _f1J2OFQDmZAXtJma8Ihec1Z94uJ52Sa X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-08-30_04,2022-08-25_01,2022-06-22_01 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Bruce Richardson > Sent: Tuesday, August 30, 2022 2:06 PM > To: Morten Br=F8rup > Cc: Shijith Thotton ; dev@dpdk.org; Pavan > Nikhilesh Bhagavatula ; > Honnappa.Nagarahalli@arm.com; Jerin Jacob Kollanukkaran > ; olivier.matz@6wind.com; > stephen@networkplumber.org; thomas@monjalon.net > Subject: [EXT] Re: [PATCH v1 2/4] mbuf: add second dynamic field member > for VA only build >=20 > External Email >=20 > ---------------------------------------------------------------------- > On Mon, Aug 29, 2022 at 08:32:20PM +0200, Morten Br=F8rup wrote: > > > > > From: Shijith Thotton [mailto:sthotton@marvell.com] > > > Sent: Monday, 29 August 2022 17.16 > > > > > > mbuf physical address field is not used in builds which only uses VA. > > > It is used to expand the dynamic field area. > > > > > > Signed-off-by: Shijith Thotton > > > --- > > > lib/mbuf/rte_mbuf_core.h | 26 +++++++++++++++++--------- > > > lib/mbuf/rte_mbuf_dyn.c | 2 ++ > > > 2 files changed, 19 insertions(+), 9 deletions(-) > > > > > > diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h > > > index 81cb07c2e4..98ce62fd6a 100644 > > > --- a/lib/mbuf/rte_mbuf_core.h > > > +++ b/lib/mbuf/rte_mbuf_core.h > > > @@ -579,15 +579,23 @@ struct rte_mbuf { > > > RTE_MARKER cacheline0; > > > > > > void *buf_addr; /**< Virtual address of segment buffer. > > > */ > > > - /** > > > - * Physical address of segment buffer. > > > - * This field is invalid if the build is configured to use only > > > - * virtual address as IOVA (i.e. RTE_IOVA_AS_VA is defined). > > > - * Force alignment to 8-bytes, so as to ensure we have the exact > > > - * same mbuf cacheline0 layout for 32-bit and 64-bit. This makes > > > - * working on vector drivers easier. > > > - */ > > > - rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t)); > > > + RTE_STD_C11 > > > + union { > > > + /** > > > + * Physical address of segment buffer. > > > + * This field is invalid if the build is configured to use > > > only > > > + * virtual address as IOVA (i.e. RTE_IOVA_AS_VA is > > > defined). > > > + * Force alignment to 8-bytes, so as to ensure we have the > > > exact > > > + * same mbuf cacheline0 layout for 32-bit and 64-bit. This > > > makes > > > + * working on vector drivers easier. > > > + */ > > > + rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t)); > > > + /** > > > + * Reserved for dynamic field in builds where physical > > > address > > > + * field is invalid. > > > + */ > > > + uint64_t dynfield2; > > > + }; > > > > > > /* next 8 bytes are initialised on RX descriptor rearm */ > > > RTE_MARKER64 rearm_data; > > > > I know that the intention here is to keep the rte_mbuf structure intact= , > which will certainly improve the probability of getting this patch series= into > DPDK. > > > > So, I will add a comment for the benefit of the other participants in t= he > discussion: > > > > With this patch, and in RTE_IOVA_AS_VA mode, it becomes possible to > move m->next into the first cache line, so rte_pktmbuf_prefree_seg() does > not have to touch the second cache line, thus potentially improving > performance by eliminating one cache miss per freed packet segment. (I al= so > recall someone mentioning that some PMDs set m->next on RX... If that is > the case, a cache miss per packet might also be avoidable in those PMDs.) > > > > Obviously, moving m->next to the first cache line is not related to thi= s patch > series, but would belong in a completely different patch. > > >=20 > +1 to that, with the exception that if it is decided to move the next > pointer rather than use this as dynamic space, I think it *should* be in > this patch series, rather than mucking about with mbuf twice. :-) +1 When RTE_IOVA_AS_VA is set we can set mbuf->next as the dynamic field an= d move it to mbuf->buf_iova. mbuf->next write is one of the prominent hotspot in arm platforms.