From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id C5953A04B9; Mon, 11 Nov 2019 15:01:18 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 8B924237; Mon, 11 Nov 2019 15:01:18 +0100 (CET) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00087.outbound.protection.outlook.com [40.107.0.87]) by dpdk.org (Postfix) with ESMTP id D75C091 for ; Mon, 11 Nov 2019 15:01:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RcoOT6wzRh/tHSoOpMfHreiHIOOEjDcIX48ScTkZWqk=; b=PC4eB9vCPIN7QBMUWgLHxN2/CWGFdSsjYUBtlp6j80HPYLsMmkfYDDpjUVR7ku9J7zkuaMqadwHbmAk8PN366jW92nG1BpbybiY3vmUvMqlOWkNiT7GxoydEJx47xCFBqIswVsDFG39wWkHCEuYP95zqCOmmb/vmZXq5JsTK4dA= Received: from DB6PR0802CA0028.eurprd08.prod.outlook.com (2603:10a6:4:a3::14) by DBBPR08MB4774.eurprd08.prod.outlook.com (2603:10a6:10:d5::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2430.24; Mon, 11 Nov 2019 14:01:14 +0000 Received: from DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e0a::204) by DB6PR0802CA0028.outlook.office365.com (2603:10a6:4:a3::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2430.22 via Frontend Transport; Mon, 11 Nov 2019 14:01:14 +0000 Authentication-Results: spf=fail (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=none action=none header.from=arm.com; Received-SPF: Fail (protection.outlook.com: domain of arm.com does not designate 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by DB5EUR03FT026.mail.protection.outlook.com (10.152.20.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2430.21 via Frontend Transport; Mon, 11 Nov 2019 14:01:14 +0000 Received: ("Tessian outbound e4042aced47b:v33"); Mon, 11 Nov 2019 14:01:12 +0000 X-CR-MTA-TID: 64aa7808 Received: from ae09b86fb1a1.2 (cr-mta-lb-1.cr-mta-net [104.47.14.59]) by 64aa7808-outbound-1.mta.getcheckrecipient.com id 5AD3B6ED-0236-49E6-9A14-EF93C0E3FFE6.1; Mon, 11 Nov 2019 14:01:07 +0000 Received: from EUR04-VI1-obe.outbound.protection.outlook.com (mail-vi1eur04lp2059.outbound.protection.outlook.com [104.47.14.59]) by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id ae09b86fb1a1.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Mon, 11 Nov 2019 14:01:07 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HNiDYdfqbMEM5BxXvv67yCTcw2s8rITyxbbaQYO4MVzvYLzUEz24pp/gnjg9VefDvHaZNeqSKaF//5XaaLvDl/w0UKKpq3QVyq5IOXEEZ15Knrlkn+JgM/pZ+3WfFaIijiMyqt1xie1F4vkL1vfsO8b774Eux5lLh5hEMuvWNiXvTWorsz1olDYUnSbEiZIqga21+AwXxLpq0IdkL7ETpzNqKHn5PwBoRhuc6SJ9ahkzD28lr8kQ3lrYp9e8q6QJWq4MDaUbTME/auDBqo7VeVipUFb5RofjvMICIpghjVo78ZWpBTG0DJwQP2b7udOFlKP2PPpwKkFN/zxst2NlJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RcoOT6wzRh/tHSoOpMfHreiHIOOEjDcIX48ScTkZWqk=; b=mam1mEpZ3brdwzfYsddxAYofWPCBHt/Q4sKOnK2TUMJ4iXdXh6NiWCqKNgzN/CF7xWT9ObP4E9hbtLMOWvPZ0xDPEOOuh17GCd/p3C0gX97UCD0kugE0PfBWVCGrheuG3tw05LYAKgDOLJ8628vtHVTa39hPB94COIo9Z5jA68ZJRxtH9h2tcslpRG8uHZVZV+8THXtqasdMcYCwAp+122HWe2gIb5YRlCPQg52XY9OInm3ieHW1gY9QmJVw4ISlvgduwRv6mujP8ibCWiddSEEGmQlh7vjMsoroN3UEwSUcl6M8lRJRvulO8ylcu6YmKGotnW4IRLyGQUf9hNFbmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RcoOT6wzRh/tHSoOpMfHreiHIOOEjDcIX48ScTkZWqk=; b=PC4eB9vCPIN7QBMUWgLHxN2/CWGFdSsjYUBtlp6j80HPYLsMmkfYDDpjUVR7ku9J7zkuaMqadwHbmAk8PN366jW92nG1BpbybiY3vmUvMqlOWkNiT7GxoydEJx47xCFBqIswVsDFG39wWkHCEuYP95zqCOmmb/vmZXq5JsTK4dA= Received: from VI1PR08MB5376.eurprd08.prod.outlook.com (10.255.196.79) by VI1PR08MB3359.eurprd08.prod.outlook.com (52.134.31.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2430.20; Mon, 11 Nov 2019 14:01:05 +0000 Received: from VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::709e:c2c9:139d:9df3]) by VI1PR08MB5376.eurprd08.prod.outlook.com ([fe80::709e:c2c9:139d:9df3%4]) with mapi id 15.20.2430.027; Mon, 11 Nov 2019 14:01:04 +0000 From: "Gavin Hu (Arm Technology China)" To: "jerinj@marvell.com" , "dev@dpdk.org" CC: Olivier Matz , Andrew Rybchenko , David Christensen , "bruce.richardson@intel.com" , "konstantin.ananyev@intel.com" , "hemant.agrawal@nxp.com" , Shahaf Shuler , Honnappa Nagarahalli , "viktorin@rehivetech.com" , "anatoly.burakov@intel.com" , Steve Capper , Ola Liljedahl , nd Thread-Topic: Mbuf memory alignment constraints for (micro)architectures Thread-Index: AdWPS2iWx/Kv7sS0Txqhe4GeZ0d67AJFAK+Q Date: Mon, 11 Nov 2019 14:01:04 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 88d0781e-9c23-4582-a344-dd7edc16aa7d.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: bb21f8a5-87ea-4976-d1d3-08d766af9db2 X-MS-TrafficTypeDiagnostic: VI1PR08MB3359:|VI1PR08MB3359:|DBBPR08MB4774: X-MS-Exchange-PUrlCount: 2 x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:7691;OLM:7691; x-forefront-prvs: 0218A015FA X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(366004)(346002)(39860400002)(136003)(396003)(376002)(199004)(189003)(13464003)(7416002)(25786009)(476003)(11346002)(81156014)(81166006)(66446008)(446003)(66476007)(76116006)(64756008)(2906002)(66946007)(66556008)(8676002)(2501003)(966005)(33656002)(53546011)(6506007)(478600001)(256004)(7696005)(52536014)(14454004)(8936002)(55236004)(76176011)(486006)(102836004)(5660300002)(55016002)(186003)(7736002)(99286004)(74316002)(316002)(9686003)(6306002)(110136005)(54906003)(305945005)(26005)(6436002)(229853002)(71200400001)(71190400001)(4326008)(6246003)(86362001)(66066001)(3846002)(6116002); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB3359; H:VI1PR08MB5376.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: mi1DrkixQJs2h9t5t4ZAxFXTE9cfcseb55zQNkbpkdRvRiEJRbkqnfXFDZsaCnxDiK+0Hst+DCMGxd20kwdcfMTNJExFYFDr9oCumrnQXXIJ9zu18FaWJOiy22TF5aiTRFCIa0wu2NxSVwHaO6Em1wqzCalnK1sPuj7HeH1+v/oYTqg3kOQI+Kdd2WQ2Lj7PEJOpDb+dTZel7xpzeDSIjGErHteJ+oCk4coCqqcjkyVr7zXxHy5IKvhwBdb+P9jYQ+Ra0L5MGp5fspysbW/advG2SKecTPIGT5gWfA/hzEICchk8eZ2a2RekFg5lbvRJYcW1e+AjJ3vHDJH8nuZe+uzq1mPYCw55TWyjCrjbCBiEyBixbeTsUx0iILT80fkzblawrljqzied6RStxeBsvyK63GLolt0mp3fGTGbBXbTOKjXIDmYaQ/SPJH99mskQeZqz2TMSIGDB2QkLWtdhQW4tvmB6J+kwG/RZfPAKf20= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3359 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Gavin.Hu@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: DB5EUR03FT026.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(396003)(39860400002)(136003)(376002)(346002)(1110001)(339900001)(189003)(199004)(13464003)(6246003)(6306002)(50466002)(9686003)(23726003)(476003)(486006)(8676002)(126002)(3846002)(6116002)(22756006)(66066001)(305945005)(7736002)(14454004)(5660300002)(110136005)(229853002)(81156014)(86362001)(54906003)(52536014)(316002)(97756001)(76130400001)(26005)(33656002)(81166006)(478600001)(70586007)(70206006)(25786009)(47776003)(46406003)(26826003)(2906002)(105606002)(356004)(2501003)(966005)(55016002)(4326008)(6506007)(7696005)(53546011)(76176011)(186003)(74316002)(102836004)(8936002)(336012)(8746002)(99286004)(446003)(11346002); DIR:OUT; SFP:1101; SCL:1; SRVR:DBBPR08MB4774; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Fail; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; MX:1; A:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: 6f779a7b-0f09-4352-4d82-08d766af978b NoDisclaimer: True X-Forefront-PRVS: 0218A015FA X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Gp6q7KbKpR5CFI/sxQBxJG7O2dZvjIYYImYNIqChiGffMsDe8BDcWklKxrnMF477VEmBx+OuYvZGQhChj5DxxUod+ibrSAKhLcblAjX5/FCoNsbH6FLmkiai87FaQUW6bgaZq++6k8uy8cJ7ieOmQ43LOn2UprA/0/MvbH1vLC/oWfCEOrIiolKflFuleKsfIcVTIxoVb/Ka6d9hf1FlcNPyou75wBH0FiEYz9zRg560E/Sy2yoAi9sYqtzNzwkPQB48T+NN4KedTewl3/C0Y4jYZEYSeoBKsoN6OswwS4q9rivCq2DLk/pgC82SDDnRxCa+scG5r9HGDa6f6oxmdCcI8CxmD6s1N8ql/q70EjL9UkkqSxzMRjA/dDT5E8XBIsFN9jMRPpKX5Zihog7O9KxSx1VyWe1byqQWwDbTW+pOAnGOsrVHAZ5Hey9KoUWo6xcjzfdXl+s5ELF0JhsfKxubMUQdvXWyp0DW8hAoGPg= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Nov 2019 14:01:14.6495 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bb21f8a5-87ea-4976-d1d3-08d766af9db2 X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBBPR08MB4774 Subject: Re: [dpdk-dev] Mbuf memory alignment constraints for (micro)architectures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Jerin, > -----Original Message----- > From: Jerin Jacob Kollanukkaran > Sent: Thursday, October 31, 2019 2:02 AM > To: dev@dpdk.org > Cc: Olivier Matz ; Andrew Rybchenko > ; David Christensen ; > bruce.richardson@intel.com; konstantin.ananyev@intel.com; > hemant.agrawal@nxp.com; Shahaf Shuler ; > Honnappa Nagarahalli ; Gavin Hu (Arm > Technology China) ; viktorin@rehivetech.com; > anatoly.burakov@intel.com > Subject: Mbuf memory alignment constraints for (micro)architectures >=20 > CC: Arch and platform maintainers >=20 > While reviewing the mempool objection allocation requirements in the code= , >=20 > A) it's found that in the default case, mempool objects have padding > in the object trailer to have start addresses of objects among the differ= ent > channels, > to enable equally load on the DRAM channel to have better performance >=20 > # More documentation is here > https://doc.dpdk.org/guides/prog_guide/mempool_lib.html > in section 8.3. Memory Alignment Constraints >=20 > B) The optimize_object_size() does the channel distribution requirement > by the following formula >=20 > new_obj_size =3D (obj_size + RTE_MEMPOOL_ALIGN_MASK) / > RTE_MEMPOOL_ALIGN; > while (get_gcd(new_obj_size, nrank * nchan) !=3D 1) > new_obj_size++; >=20 >=20 > C) The formula mentioned in the (B) is NOT generic. At least of the octeo= ntx2 > SoC > The memory/DDR controller works in different way. Where by: > # It does XOR operation of some of physical address lines(not the user s= pace > VA address) > to compute the hash and that the function defines the actual channel. >=20 > The XOR(kind of CRC) scheme is useful because there is natural channel > distribution > based on the address i.e No need to have padding to waste memory >=20 > So, in short the padding scheme does not need for some SoC. I trying to s= end > the patch > to fix it. So the questions is, >=20 > # Is PPC and other ARM SoC has formula (B) to compute DRAM channel > distribution ? or > Is it specific to x86? That would define where the hooks needs to added t= o have > proper fix. Reading through some documents, both x86 and arm, and having internal discu= ssion, it looks like this is specific to x86, x86 spreads adjacent virtual address= es within a page across multiple memory devices,=20 the interleaving was done per one or two cache lines. https://software.inte= l.com/en-us/articles/how-memory-is-accessed =20 Arm leaves flexibility to implementations, no fixed pattern for interleavin= g and thus it can hardly be generalized.=20 /Gavin >=20 >=20 >=20 >=20 >=20 >=20 >=20 >=20