From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C0165A034C; Tue, 30 Aug 2022 15:22:37 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5DFD340F18; Tue, 30 Aug 2022 15:22:36 +0200 (CEST) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-eopbgr130071.outbound.protection.outlook.com [40.107.13.71]) by mails.dpdk.org (Postfix) with ESMTP id 67E9740F17 for ; Tue, 30 Aug 2022 15:22:35 +0200 (CEST) ARC-Seal: i=2; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=pass; b=cE14YJWx/WbfvkKPX9Jds0yjk9d4kDLDCmj/EsYnsQ7VfXSjDYM5uiUTBcGS4DRXmlTfaWUTkfPbzhfGBVOOJ8G065eM5dc1u+atXJmMolfVjvJo2rlZo8y51BCyNPokpbn1P/D0Qq473msxu8MnEQBknQKTldIvZYqk4DNwVae+rFULt2hQMFHtph9/TSAV7RxvJBMVYz4+f2WfJ6mQDX/64yDBoRC4miu7k2ZCXuJqjQwAkbHBnmO49fmlCWn4GM7XqjJ0ZR8c4Vmjwgepozya+Wm4t1Ja0N/RBIzFNB6VasZ55UXw2kJL/l9uheJgTxfgZlQsPrY1mRNwdxBaUg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jDJheSjb276dgIim4/Y9cZWI7TvScZ5hVyji0Bl2bBQ=; b=bXBgEK7eJTbv+yDCO4jmw09ITJARAMpTJ72ev4Il5Q1JtE853WFeKLJTjw9g6Z3m8txBHDpe5FL2SKN7rZDpSAKVJ6cRemXZUxEmTjpAQnsLxKxVq7+AFbu6i+a+M8IjZZKDFxSqWpom8iQ+rRJt/fr50DsFBabstdaag7uPRGyqNuYpF+dGrfrDs8+S7cO+Yr4AJTEsGutt+cRER1Vfgl4jEK/7oi2WqZUYsg7K+pR40ylSa0xw17aX2fm8RjToKtjz9kyoqg4wg5QzoIbibHgRW8PXjIGKoQNLgPqNJMEAsflEv3yTZvtYDUFz6dFYlDaDYkqWZQEZwD0CJjPAsw== ARC-Authentication-Results: i=2; mx.microsoft.com 1; spf=pass (sender ip is 63.35.35.123) smtp.rcpttodomain=dpdk.org smtp.mailfrom=arm.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com; arc=pass (0 oda=1 ltdi=1 spf=[1,1,smtp.mailfrom=arm.com] dkim=[1,1,header.d=arm.com] dmarc=[1,1,header.from=arm.com]) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jDJheSjb276dgIim4/Y9cZWI7TvScZ5hVyji0Bl2bBQ=; b=469HaZNcmktHbFqL4aepe9CYiXWPqC1bEBUQgyfFbboVTYZRFdePEVLM15ndHgPF5MmcdYb3FAVcpwFv9K+HsHpd2AQR+ngNgG00DOrIk/xKIKrR342SbXUCJth8F/vRd1AT9fi+At0+0E5Arn13N2V50JybVqvEYq7enEfEyQs= Received: from AS9PR06CA0455.eurprd06.prod.outlook.com (2603:10a6:20b:49a::16) by DBAPR08MB5848.eurprd08.prod.outlook.com (2603:10a6:10:1b1::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5588.10; Tue, 30 Aug 2022 13:22:33 +0000 Received: from VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com (2603:10a6:20b:49a:cafe::5d) by AS9PR06CA0455.outlook.office365.com (2603:10a6:20b:49a::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.15 via Frontend Transport; Tue, 30 Aug 2022 13:22:31 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dmarc=pass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; pr=C Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT041.mail.protection.outlook.com (10.152.19.163) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.15 via Frontend Transport; Tue, 30 Aug 2022 13:22:30 +0000 Received: ("Tessian outbound 73dd6a25223d:v123"); Tue, 30 Aug 2022 13:22:29 +0000 X-CR-MTA-TID: 64aa7808 Received: from d78e58cb5e92.1 by 64aa7808-outbound-1.mta.getcheckrecipient.com id 75978E0F-AD04-4B30-924D-ABC1FD7A312E.1; Tue, 30 Aug 2022 13:22:23 +0000 Received: from EUR05-AM6-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id d78e58cb5e92.1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 30 Aug 2022 13:22:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZJvhMJtPB4s32TVjHxLwFupNv0h+43BIGFcnJKaev+TI0RbnjKZOqX4iL6xDXFBi8oNPkdkgMiUtcT7SqxtcwhYD3xebpFQZJVv90ybbZEQKvkNwycnbMGEZMS3g9WdLXKfNoTa7OT+N7NPjWU2Q9Oyu/nO5XhYueGbtlMM5ZzBsPvUXE3L8ljg8Py17xriPIBop7fudUI4NIyJgIKMU3OCqeMbdYaeUdBym3gFuYT7lICawjsdNgWTyKt43Cejuq9sPJqcWPCX5Q9kayedTiqhQMzQnsLl1IBwwSMvEaKVtXT0IZ9ELtzn7bq4DmzTE1sy0BKvJG2h0meRv76x71Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jDJheSjb276dgIim4/Y9cZWI7TvScZ5hVyji0Bl2bBQ=; b=Rtfo003JRGRkKERv8CLcYEMOb0LXto6rZ1IQmMEfQCL3zNi+B7+YynKJnLREEZAicmSnC0IocMeJY0cW3Ql83ZL+UIT6+8iRQsq6Pvxo5RnkA40dHUF4MUMilRC6cSVch6cdXvLnszxwUtoKAHUCfu8PRuqdRiV99o6js50+iS46OIR9Wd7m/uhOFB3LHTqfUVjSi3P0D5Wle7Ov2+7M/VhtJBLZjhvYLG1Jr/TMR+uszptxPA8wkiCv9ksajtBQ47XjhDXqlPK5wtUdqvHgUG29lBudnLIj0t2vtIbAYRt9R1MMXemoABxFTNYYmOvDdUjZFMFv6jszxdEmkCvPwA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jDJheSjb276dgIim4/Y9cZWI7TvScZ5hVyji0Bl2bBQ=; b=469HaZNcmktHbFqL4aepe9CYiXWPqC1bEBUQgyfFbboVTYZRFdePEVLM15ndHgPF5MmcdYb3FAVcpwFv9K+HsHpd2AQR+ngNgG00DOrIk/xKIKrR342SbXUCJth8F/vRd1AT9fi+At0+0E5Arn13N2V50JybVqvEYq7enEfEyQs= Received: from DBAPR08MB5814.eurprd08.prod.outlook.com (2603:10a6:10:1b1::6) by VE1PR08MB4832.eurprd08.prod.outlook.com (2603:10a6:802:a7::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5566.15; Tue, 30 Aug 2022 13:22:20 +0000 Received: from DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::1415:460c:634:3d62]) by DBAPR08MB5814.eurprd08.prod.outlook.com ([fe80::1415:460c:634:3d62%7]) with mapi id 15.20.5566.021; Tue, 30 Aug 2022 13:22:20 +0000 From: Honnappa Nagarahalli To: Pavan Nikhilesh Bhagavatula , Bruce Richardson , =?iso-8859-1?Q?Morten_Br=F8rup?= CC: Shijith Thotton , "dev@dpdk.org" , "jerinj@marvell.com" , "olivier.matz@6wind.com" , "stephen@networkplumber.org" , "thomas@monjalon.net" , nd Subject: RE: [EXT] Re: [PATCH v1 2/4] mbuf: add second dynamic field member for VA only build Thread-Topic: [EXT] Re: [PATCH v1 2/4] mbuf: add second dynamic field member for VA only build Thread-Index: AQHYvEuVHx5xTflbSEOPOWPJiec6/K3HICKAgABN0EA= Date: Tue, 30 Aug 2022 13:22:19 +0000 Message-ID: References: <20220829151626.2101336-3-sthotton@marvell.com> <98CBD80474FA8B44BF855DF32C47DC35D872D1@smartserver.smartshare.dk> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: B3F37A9D1DBE4D498D934F66E572FC23.0 x-checkrecipientchecked: true Authentication-Results-Original: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-MS-Office365-Filtering-Correlation-Id: 114c3e55-8a0f-4feb-c06d-08da8a8ab111 x-ms-traffictypediagnostic: VE1PR08MB4832:EE_|VE1EUR03FT041:EE_|DBAPR08MB5848:EE_ x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-checkrecipientrouted: true nodisclaimer: true X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: JqAHVDI13zLQPRayqZCsf9hPHdKSefPoJlgwibn3fDORliz5UAoZqLsLI+/HV5YF+Xaa3kETGwrNfkuYU4CSftWknzxk5RFGkzUpumzZKRepCJunudgTUFQ5bZYfou2FsY/bTR2R8Z56upcOujA+cjXvKDzpCMcT/yofJYcbjoN9prj/mWFOLJdrvh3/IJfWlQ8Ydx8N2DSL8zV7T0oJtrTwdz4JyhueQb667xrTR81qXmGQO/orXEYv6VLT13i9QkSLV9XdWyyHFA+nHlx/UVOHZExRruJT7ka/utDtmF8Ay/ivGabMg0oDYRqwf0qrrNSAocZ91N4hDSIqs1Uvb+Rxc3FdaBM2q8UXH0ZwTycZaAvUkFDV1lggZDCLUGARKukg8DIx3NoBTejjXAYoI+thdeloFMDOt/9uTOpf04o1evAniyJSAqAouI0Lii8LXuTXOBwDTen16Oik572d2nwsotq05+nXHRLaidPKJY57VRm19hmoXH6F7iFJjVIeuieSQ+E6l6sQXL1XVIVb7G6l9/WgoN/CUZ2LT6sIeY9UJJC4MiC9dnwEev6OsKihWAZQuTplWSO8z2xXwGy0DM+67LWH7FOIkL3hUKxSYOWcnDozokuq/C1bTQxBgNPvULU0GLlzkiiRuAEB41Qs8zCqrePSJtALnO557HUDrDw0LNJP1QSIRxgq5YzL0DDHs/Avx+YzQowB26ChSp95kInwGT80LEBLXIjQs+XX5vrZmWN6+hOmXOZg4FG1eHBs9tgknlccYnV4STN1zWg20w== X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DBAPR08MB5814.eurprd08.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(366004)(39860400002)(136003)(376002)(396003)(346002)(33656002)(8936002)(55016003)(41300700001)(5660300002)(2906002)(66574015)(7696005)(186003)(86362001)(6506007)(26005)(9686003)(52536014)(316002)(38100700002)(54906003)(71200400001)(122000001)(110136005)(66476007)(38070700005)(478600001)(8676002)(66556008)(83380400001)(76116006)(66946007)(64756008)(66446008)(4326008); DIR:OUT; SFP:1101; Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR08MB4832 Original-Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 36c0d6cd-ae70-419c-e854-08da8a8aaa99 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OtvejERfnLp2ITHn3qqGxcThjUPBnkHElKJIraGNo6FmMMleEOnc2Ltvgu7a9bcMnjAaI30rZT/KRF/tGeidvXmKuJsD4g7I7tH4k3JYnhoFk0tslgSrObmVO786jDQgZh2D1z4WRRq4hwdjG0xCI0kW5u41YF2Q0rBQIZP72A3qKOHQsT7SHqKmnEbr9RZqDWRUGuuWRnOEXfO02m9+6/V+jF0LXg3rd5HC46nFt/BQ6H/L1MaliXtg3Pp5FKHXbPyq3w39rJZp/s8R/CjZyfOkYzFEMnzs4G7EhqnisUI54O7bbaQq/luKgxvrsDkCK+PU2/2XhsUeHOzlYdxBvTREU+r9jWB76x9L7oxU93lWBqOx5Fv0Kq1dL8seY0/hWBdXqGmLn6W3MYe4KlBbUFjwsl3Cncy/VLp2JIp4oCRZpSGrKBkonXvLenFgvfXVu+4a8VKTAlKHQoWSnjvucu6VeGIsE4VKT7hLZWGTu+uXyiVCWh3Q5qg2ymCfVplSDWZFpSvG0iEwER3Qj34JLjHwS/7Qr4z3zaO4IMLjGneVaj+OlVw8EiS2v9XMigxfupepGFev9qR/jg9UBWHtomtbaaO1tSr3hVV9tCV5TxKMdsw//K2Wj33x2cRJaIcvrUPMXxGhYdNBO0GVcOFf+YvIGe4cJhtqdjE4NrANhrYOxpjY12MKye92ASlBfcdGtTPY8KHdsRRDtP+rqPd4Sz0dFNBrRNzf12ABkXEvZloftqmbr6Ta7YEHROnbSVA37HDJleIafY86U81GmjAlXQ== X-Forefront-Antispam-Report: CIP:63.35.35.123; CTRY:IE; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:64aa7808-outbound-1.mta.getcheckrecipient.com; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; CAT:NONE; SFS:(13230016)(4636009)(136003)(376002)(39860400002)(346002)(396003)(46966006)(40470700004)(36840700001)(356005)(110136005)(82310400005)(82740400003)(54906003)(316002)(36860700001)(81166007)(55016003)(40480700001)(26005)(2906002)(9686003)(70206006)(4326008)(8676002)(70586007)(478600001)(33656002)(40460700003)(5660300002)(86362001)(186003)(336012)(7696005)(47076005)(66574015)(6506007)(41300700001)(52536014)(8936002)(83380400001); DIR:OUT; SFP:1101; X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Aug 2022 13:22:30.5944 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 114c3e55-8a0f-4feb-c06d-08da8a8ab111 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-AuthSource: VE1EUR03FT041.eop-EUR03.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR08MB5848 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > > > > ---------------------------------------------------------------------- > > On Mon, Aug 29, 2022 at 08:32:20PM +0200, Morten Br=F8rup wrote: > > > > > > > From: Shijith Thotton [mailto:sthotton@marvell.com] > > > > Sent: Monday, 29 August 2022 17.16 > > > > > > > > mbuf physical address field is not used in builds which only uses V= A. > > > > It is used to expand the dynamic field area. > > > > > > > > Signed-off-by: Shijith Thotton > > > > --- > > > > lib/mbuf/rte_mbuf_core.h | 26 +++++++++++++++++--------- > > > > lib/mbuf/rte_mbuf_dyn.c | 2 ++ > > > > 2 files changed, 19 insertions(+), 9 deletions(-) > > > > > > > > diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h > > > > index 81cb07c2e4..98ce62fd6a 100644 > > > > --- a/lib/mbuf/rte_mbuf_core.h > > > > +++ b/lib/mbuf/rte_mbuf_core.h > > > > @@ -579,15 +579,23 @@ struct rte_mbuf { > > > > RTE_MARKER cacheline0; > > > > > > > > void *buf_addr; /**< Virtual address of segment buffer. > > > > */ > > > > - /** > > > > - * Physical address of segment buffer. > > > > - * This field is invalid if the build is configured to use only > > > > - * virtual address as IOVA (i.e. RTE_IOVA_AS_VA is defined). > > > > - * Force alignment to 8-bytes, so as to ensure we have the exact > > > > - * same mbuf cacheline0 layout for 32-bit and 64-bit. This makes > > > > - * working on vector drivers easier. > > > > - */ > > > > - rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t)); > > > > + RTE_STD_C11 > > > > + union { > > > > + /** > > > > + * Physical address of segment buffer. > > > > + * This field is invalid if the build is configured to use > > > > only > > > > + * virtual address as IOVA (i.e. RTE_IOVA_AS_VA is > > > > defined). > > > > + * Force alignment to 8-bytes, so as to ensure we have the > > > > exact > > > > + * same mbuf cacheline0 layout for 32-bit and 64-bit. This > > > > makes > > > > + * working on vector drivers easier. > > > > + */ > > > > + rte_iova_t buf_iova __rte_aligned(sizeof(rte_iova_t)); > > > > + /** > > > > + * Reserved for dynamic field in builds where physical > > > > address > > > > + * field is invalid. > > > > + */ > > > > + uint64_t dynfield2; > > > > + }; > > > > > > > > /* next 8 bytes are initialised on RX descriptor rearm */ > > > > RTE_MARKER64 rearm_data; > > > > > > I know that the intention here is to keep the rte_mbuf structure > > > intact, > > which will certainly improve the probability of getting this patch > > series into DPDK. > > > > > > So, I will add a comment for the benefit of the other participants > > > in the > > discussion: > > > > > > With this patch, and in RTE_IOVA_AS_VA mode, it becomes possible to > > move m->next into the first cache line, so rte_pktmbuf_prefree_seg() > > does not have to touch the second cache line, thus potentially > > improving performance by eliminating one cache miss per freed packet > > segment. (I also recall someone mentioning that some PMDs set m->next > > on RX... If that is the case, a cache miss per packet might also be > > avoidable in those PMDs.) > > > > > > Obviously, moving m->next to the first cache line is not related to > > > this patch > > series, but would belong in a completely different patch. > > > > > > > +1 to that, with the exception that if it is decided to move the next > > pointer rather than use this as dynamic space, I think it *should* be > > in this patch series, rather than mucking about with mbuf twice. :-) >=20 > +1 When RTE_IOVA_AS_VA is set we can set mbuf->next as the dynamic field > and move it to mbuf->buf_iova. > mbuf->next write is one of the prominent hotspot in arm platforms. +1 for reducing the cachelines that need to be touched