From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6037BA04AB; Tue, 12 Nov 2019 03:37:01 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 98A04271; Tue, 12 Nov 2019 03:37:00 +0100 (CET) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-eopbgr140042.outbound.protection.outlook.com [40.107.14.42]) by dpdk.org (Postfix) with ESMTP id 14C51235 for ; Tue, 12 Nov 2019 03:37:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lx4JuPyuG5VSYs6djxTvu/wSl+29vGp3Twdi/vICcNU=; b=21+ZIwazKcDxhvzyNjiaWFyRM68F3QO8HYg+RDWBhXbDuKE4eYtuuPW9r6SZ7Rl3ECttuQqYdo0Gn2bi8+yh9F/seypjZVXcMuV1GCzICIqZaysaLRzRnUhNpq84nxH141bl2otT5x86+okjGCPyO0YqUTaY0FIrtYQAVGMTdBg= Received: from VI1PR08CA0113.eurprd08.prod.outlook.com (2603:10a6:800:d4::15) by DB6PR08MB2936.eurprd08.prod.outlook.com (2603:10a6:6:25::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2430.20; Tue, 12 Nov 2019 02:36:58 +0000 Received: from VE1EUR03FT021.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::204) by VI1PR08CA0113.outlook.office365.com (2603:10a6:800:d4::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2430.20 via Frontend Transport; Tue, 12 Nov 2019 02:36:58 +0000 Authentication-Results: spf=fail (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: Fail (protection.outlook.com: domain of arm.com does not designate 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT021.mail.protection.outlook.com (10.152.18.117) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2430.21 via Frontend Transport; Tue, 12 Nov 2019 02:36:57 +0000 Received: ("Tessian outbound 081de437afc7:v33"); Tue, 12 Nov 2019 02:36:57 +0000 X-CR-MTA-TID: 64aa7808 Received: from 55bebbfca9f0.3 (cr-mta-lb-1.cr-mta-net [104.47.5.57]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 04DE233E-9A67-40D0-B694-5DFDE4688580.1; Tue, 12 Nov 2019 02:36:52 +0000 Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-he1eur02lp2057.outbound.protection.outlook.com [104.47.5.57]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 55bebbfca9f0.3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Tue, 12 Nov 2019 02:36:52 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UQ4zlg+nTvlnQlU1yZFOuEIxSiCfav/gP0z7LXh/6tVKRlLZptFi8Uq/RBtm00t8JARR89lGYKkLGkdqYSWmZldigmKVGf8XnkkMpzXTIuq+EXhZEbfiVkmZv/tm0WbZMIbk0H3150ekkNoRHVrDVERoe1dKGdVfPGeq2AcvKyZ/TbyI+Zf+mPNmhk8M4PXmZc1gh/k9oK0ROh7vexu/OZRqYlBU/VmpAViCnB+RhsJfeht+1xMjEFHLs2xjl8vhEASXy3vGXm5bM7r4k0MApYw9PrM+hgCk2AzQeHjzm2JCcmqnec4DueQb4h1CyXjPhSBC6ae4lLkrEvrXgcG/Cg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lx4JuPyuG5VSYs6djxTvu/wSl+29vGp3Twdi/vICcNU=; b=MECVv26VtXbsUsp4OohZ/VUHCv96JA+HKKugRp3+zV/pEh/CtXCLRrtsLAJMBZTiXDEREVCKVqk3rNc0ojZBm77AnbXlfXPZ34FIsPu2Xx0U1+orDdd68XR19csrorKUafIDwTBsKUj1nEwwF3xrUbLxRQL4E8AyIbxo4rI9FptbT3+tv4sW7uf2DeYCoNGbzjJiep3+IRDCTaZpA1KqSzXoXYkQ0XbeIGffgxJQ/1tCS3XGzEmsaJmII02X/Eo1wLwayjQAwwyTWqmkm8fWbfmX4aXbuBVjwCZ7xGzvW5PwR/C9P8Ae5loiHnoJa//0tkFhfyt57Wfs194HgbyALw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Lx4JuPyuG5VSYs6djxTvu/wSl+29vGp3Twdi/vICcNU=; b=21+ZIwazKcDxhvzyNjiaWFyRM68F3QO8HYg+RDWBhXbDuKE4eYtuuPW9r6SZ7Rl3ECttuQqYdo0Gn2bi8+yh9F/seypjZVXcMuV1GCzICIqZaysaLRzRnUhNpq84nxH141bl2otT5x86+okjGCPyO0YqUTaY0FIrtYQAVGMTdBg= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (10.255.196.79) by VI1PR08MB5422.eurprd08.prod.outlook.com (52.133.245.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2430.24; Tue, 12 Nov 2019 02:36:46 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::709e:c2c9:139d:9df3]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::709e:c2c9:139d:9df3%4]) with mapi id 15.20.2451.018; Tue, 12 Nov 2019 02:36:45 +0000 From: "Gavin Hu (Arm Technology China)" To: "Gavin Hu (Arm Technology China)" , "jerinj@marvell.com" , "dev@dpdk.org" CC: Olivier Matz , Andrew Rybchenko , David Christensen , "bruce.richardson@intel.com" , "konstantin.ananyev@intel.com" , "hemant.agrawal@nxp.com" , Shahaf Shuler , Honnappa Nagarahalli , "viktorin@rehivetech.com" , "anatoly.burakov@intel.com" , Steve Capper , Ola Liljedahl , nd , nd Thread-Topic: [dpdk-dev] Mbuf memory alignment constraints for (micro)architectures Thread-Index: AQHVmQIGCpjUU0pJ0EqqRDCZ478qEA== Date: Tue, 12 Nov 2019 02:36:45 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 79210942-633e-4f1e-a049-d1f69f38525c.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: c336d12d-0a9a-4a62-a24a-08d767193053 X-MS-TrafficTypeDiagnostic: VI1PR08MB5422:|VI1PR08MB5422:|DB6PR08MB2936: X-MS-Exchange-PUrlCount: 2 x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:8882;OLM:8882; x-forefront-prvs: 021975AE46 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(396003)(136003)(346002)(366004)(376002)(39860400002)(189003)(199004)(13464003)(52536014)(55016002)(9686003)(2501003)(476003)(6306002)(2201001)(446003)(66446008)(66066001)(66476007)(6506007)(66556008)(64756008)(53546011)(33656002)(76116006)(7696005)(66946007)(5660300002)(76176011)(186003)(11346002)(229853002)(110136005)(102836004)(55236004)(6116002)(6436002)(3846002)(99286004)(25786009)(256004)(26005)(966005)(478600001)(8936002)(14454004)(54906003)(6246003)(2906002)(7416002)(7736002)(316002)(4326008)(305945005)(486006)(71190400001)(71200400001)(74316002)(81156014)(81166006)(86362001); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB5422; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: gh346na+fta4iQtTfpdjIguIr/RpakMU5fWScgk3PdDuZUaLt3/Mo5ENzup8rWQ817P2y6eHSem2/aRloiYkJCnOwHzFTiy8S3QlDby3TLFNSR7t7V13b0Vs9oV/OTwUk0bSHGgK2aaim/JBATS3ECOTMSVBPj/hh0URjnfkNA+0kbUieYUMIo9J0dobmCrmjeTAntmXAXsTtVetOiEKy6X8JaKDv4FenxabxjkdRmlHp8aYynQS/6xIIwYBBX4yI70/F7Nk9btttvxmC9dl1JEuNl3fE35Y726xm1VG+gDTWGYdgoYGY7GqT72qh5JZssG41pYE8NhSaTCFiYmfOXIegf9Xck16iL3lXRCtdpK38Qt/E3iG6orxkmAAZTVHB9IyF/lsJZ3hEBwxctem6TQxD70kFi0iqrbB0LECQXkYdwCSN5JecS5j953VmTpFkhhqgrODllJ9FoVTEVCUE6SmE70M4SaKAjVHqOF/rCw= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB5422 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT021.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(346002)(396003)(376002)(136003)(39860400002)(1110001)(339900001)(199004)(189003)(13464003)(54906003)(4326008)(81156014)(81166006)(316002)(46406003)(70206006)(110136005)(70586007)(2501003)(305945005)(6306002)(9686003)(55016002)(2201001)(966005)(7736002)(14454004)(86362001)(6246003)(36906005)(478600001)(97756001)(33656002)(76130400001)(26826003)(66066001)(76176011)(53546011)(7696005)(102836004)(25786009)(6506007)(356004)(99286004)(50466002)(47776003)(186003)(22756006)(8936002)(74316002)(52536014)(126002)(8746002)(2906002)(229853002)(26005)(3846002)(336012)(23726003)(446003)(6116002)(105606002)(476003)(486006)(11346002)(5660300002); DIR:OUT; SFP:1101; SCL:1; SRVR:DB6PR08MB2936; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Fail; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 74f2457b-0e89-4538-865e-08d767192929 NoDisclaimer: True X-Forefront-PRVS: 021975AE46 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jZAecPWaR0zm94ghrt76D698j32I5SLmssWLWifXGWMUF177W7af2sLKWYu/uINjwwk8ufOyyE5IWkrkUkMhA7nb3+2JH3pr9x+aWwlaBznfZHuxVUa2OrBqo3muncLSmMXqwN3bbZRpZIOpstw4RXCw5zOeL6bBuQGAdDQIMbN4S7sueBoqYdYP9SjxwADQs2pdOjMgnBOY9lFZFJzTbzcSp0T6SUjOUZY+LM7VMSJqBlefUBYnd+auM7aTN60hpriPFxY8HoTLRPuYO/pEuBMyWl4CHv287gNDJ1B22hkS8cXrmp+Q6vSJO3OYapvAa7VK4fa98nNlhg93k3sbAv61cjoWBZqOPuJt5UaJedbp7jhphce8ID9aAr9M5wemo4L+h97JedmEXutX3/+Y71e3a/XgfbuwJxIhMG50DnOLuLboDnvkYei5SobxUkOcC0bFxGoqBcFQx9jlouKvZAWQ+F5dxzClRn47bTyXRvw= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Nov 2019 02:36:57.7384 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c336d12d-0a9a-4a62-a24a-08d767193053 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR08MB2936 Subject: Re: [dpdk-dev] Mbuf memory alignment constraints for (micro)architectures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Jerin, > -----Original Message----- > From: dev On Behalf Of Gavin Hu (Arm Technology > China) > Sent: Monday, November 11, 2019 10:01 PM > To: jerinj@marvell.com; dev@dpdk.org > Cc: Olivier Matz ; Andrew Rybchenko > ; David Christensen ; > bruce.richardson@intel.com; konstantin.ananyev@intel.com; > hemant.agrawal@nxp.com; Shahaf Shuler ; > Honnappa Nagarahalli ; > viktorin@rehivetech.com; anatoly.burakov@intel.com; Steve Capper > ; Ola Liljedahl ; nd > > Subject: Re: [dpdk-dev] Mbuf memory alignment constraints for > (micro)architectures >=20 > Hi Jerin, >=20 > > -----Original Message----- > > From: Jerin Jacob Kollanukkaran > > Sent: Thursday, October 31, 2019 2:02 AM > > To: dev@dpdk.org > > Cc: Olivier Matz ; Andrew Rybchenko > > ; David Christensen > ; > > bruce.richardson@intel.com; konstantin.ananyev@intel.com; > > hemant.agrawal@nxp.com; Shahaf Shuler ; > > Honnappa Nagarahalli ; Gavin Hu (Arm > > Technology China) ; viktorin@rehivetech.com; > > anatoly.burakov@intel.com > > Subject: Mbuf memory alignment constraints for (micro)architectures > > > > CC: Arch and platform maintainers > > > > While reviewing the mempool objection allocation requirements in the co= de, > > > > A) it's found that in the default case, mempool objects have padding > > in the object trailer to have start addresses of objects among the diff= erent > > channels, > > to enable equally load on the DRAM channel to have better performance > > > > # More documentation is here > > https://doc.dpdk.org/guides/prog_guide/mempool_lib.html > > in section 8.3. Memory Alignment Constraints > > > > B) The optimize_object_size() does the channel distribution requirement > > by the following formula > > > > new_obj_size =3D (obj_size + RTE_MEMPOOL_ALIGN_MASK) / > > RTE_MEMPOOL_ALIGN; > > while (get_gcd(new_obj_size, nrank * nchan) !=3D 1) > > new_obj_size++; > > > > > > C) The formula mentioned in the (B) is NOT generic. At least of the oct= eontx2 > > SoC > > The memory/DDR controller works in different way. Where by: > > # It does XOR operation of some of physical address lines(not the user= space > > VA address) > > to compute the hash and that the function defines the actual channel. > > > > The XOR(kind of CRC) scheme is useful because there is natural channel > > distribution > > based on the address i.e No need to have padding to waste memory > > > > So, in short the padding scheme does not need for some SoC. I trying to= send > > the patch > > to fix it. So the questions is, > > > > # Is PPC and other ARM SoC has formula (B) to compute DRAM channel > > distribution ? or > > Is it specific to x86? That would define where the hooks needs to added= to > have > > proper fix. > Reading through some documents, both x86 and arm, and having internal > discussion, > it looks like this is specific to x86, x86 spreads adjacent virtual addre= sses within > a page across multiple memory devices, > the interleaving was done per one or two cache lines. > https://software.intel.com/en-us/articles/how-memory-is-accessed >=20 > Arm leaves flexibility to implementations, no fixed pattern for interleav= ing and > thus it can hardly be generalized. Same conclusion, but more words for this topic(from Arm internally): "Interleaving (or stripping) happens at the interconnect/memory controller = level, so on Arm-based systems it's going to be highly dependent on the giv= en SoC's integration and probably the system configuration too. Arm own int= erconnect and DMC IPs generally offer various options to support stripping,= but even then it's the integrator's choice how to use them, and obviously = there are multitudes of alternative third-party IPs too. In summary, this really depends on the system's interconnect and memory con= troller capabilities and how it has been configured." /Gavin > > > > > > > > > > > > > > > >