From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 529F8A04F5; Fri, 20 Dec 2019 16:55:51 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3C42C1F1C; Fri, 20 Dec 2019 16:55:50 +0100 (CET) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20050.outbound.protection.outlook.com [40.107.2.50]) by dpdk.org (Postfix) with ESMTP id 889101C01; Fri, 20 Dec 2019 16:55:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cad7aA5VofY02kp++4TYb9CyzP4RBySEcV1AhRb3uqg=; b=iqXqKEhagoZN/QWDJBgSdI1ROXRpY/3nFgObK/Hm49zcs8KBGBVqZnB502OLN1j4yzPOMpbPVZ0XF2Gj1PqtDD3qJjsl3UO27cBImhn8Sgbv/iEwQNlRnPg9+YzF9Kecugp02aSTDaY+KeIrvG71IbbbELNZEDBIcYMWgv7X8E0= Received: from VI1PR08CA0105.eurprd08.prod.outlook.com (2603:10a6:800:d3::31) by VI1PR08MB3533.eurprd08.prod.outlook.com (2603:10a6:803:84::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2559.16; Fri, 20 Dec 2019 15:55:47 +0000 Received: from VE1EUR03FT009.eop-EUR03.prod.protection.outlook.com (2a01:111:f400:7e09::203) by VI1PR08CA0105.outlook.office365.com (2603:10a6:800:d3::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2559.14 via Frontend Transport; Fri, 20 Dec 2019 15:55:47 +0000 Authentication-Results: spf=pass (sender IP is 63.35.35.123) smtp.mailfrom=arm.com; dpdk.org; dkim=pass (signature was verified) header.d=armh.onmicrosoft.com;dpdk.org; dmarc=bestguesspass action=none header.from=arm.com; Received-SPF: Pass (protection.outlook.com: domain of arm.com designates 63.35.35.123 as permitted sender) receiver=protection.outlook.com; client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com; Received: from 64aa7808-outbound-1.mta.getcheckrecipient.com (63.35.35.123) by VE1EUR03FT009.mail.protection.outlook.com (10.152.18.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2559.14 via Frontend Transport; Fri, 20 Dec 2019 15:55:47 +0000 Received: ("Tessian outbound 28955e0c1ca8:v40"); Fri, 20 Dec 2019 15:55:46 +0000 X-CR-MTA-TID: 64aa7808 Received: from 9fbfafbff41b.2 by 64aa7808-outbound-1.mta.getcheckrecipient.com id B2357D10-1F06-4951-9D16-826289256776.1; Fri, 20 Dec 2019 15:55:41 +0000 Received: from EUR01-HE1-obe.outbound.protection.outlook.com by 64aa7808-outbound-1.mta.getcheckrecipient.com with ESMTPS id 9fbfafbff41b.2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384); Fri, 20 Dec 2019 15:55:41 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=axBTMIlTwrmV125CHKgjFUWHMwgIjSN5UujbPQQJP4QDtVoLfp8RNGxsU3Z/GssrZeKKhNDvqpvjL64hYpDxjmsDVTNbHn1BVvPuKQV6JKf36ixzWbQrPTdrDSVHqQPZw1QGOWwduZP3ZFd4FwQWl5xYkKcE2gWnOI5st8UFO1K2CFo3JTwdxCZb+hucGUQs6sUdr5y18RPiog+JPCycoavKr5p/nJYHj2gtPMIUkxCzXZpIWzs9rMBmCV1XXl3hLgJUcQAUBEuzTs2HCVrKAZEPoc2OBjzBgupICyQc9xRguIfvWeIQ08TtNj9rLAkw00ONT/Jcg+fSVjc1wSlmtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cad7aA5VofY02kp++4TYb9CyzP4RBySEcV1AhRb3uqg=; b=drvZ3sTPt4zaabQA4CpFdadeLiyrFOwYqskje/pUu5DxAF+hdPqbaC5rddQDn6B2QHgMkiG2jm6H6WTU4tUCc+gKlwTu5h9JAyIukU4rC+RgIJPbh4KKT50xEBUkhW4/FyVALT4nbpkt9IVM8oT3UcpzMyTeiHyguNykg/mgUyEMVRG33xeXs7PI7ufKIvjDwhgQRDaoVvjgKd/XpsNPBkFAHh7lcVVX5En4TbYNJ80G7PNW8ZAKBgzMShMz+lcJt8gtyeZ+3VyJZpEJFUlKPRpRXpWdUAmnHETN7727Bge328chQbACtuIfQx/YE4lkMcZ/zXx4jvQoIb2ieEQ4bg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=arm.com; dmarc=pass action=none header.from=arm.com; dkim=pass header.d=arm.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector2-armh-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cad7aA5VofY02kp++4TYb9CyzP4RBySEcV1AhRb3uqg=; b=iqXqKEhagoZN/QWDJBgSdI1ROXRpY/3nFgObK/Hm49zcs8KBGBVqZnB502OLN1j4yzPOMpbPVZ0XF2Gj1PqtDD3qJjsl3UO27cBImhn8Sgbv/iEwQNlRnPg9+YzF9Kecugp02aSTDaY+KeIrvG71IbbbELNZEDBIcYMWgv7X8E0= Received: from VE1PR08MB5149.eurprd08.prod.outlook.com (20.179.30.27) by VE1SPR01MB0003.eurprd08.prod.outlook.com (10.255.115.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2559.16; Fri, 20 Dec 2019 15:55:38 +0000 Received: from VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::9df4:7af6:b62d:f94e]) by VE1PR08MB5149.eurprd08.prod.outlook.com ([fe80::9df4:7af6:b62d:f94e%7]) with mapi id 15.20.2538.022; Fri, 20 Dec 2019 15:55:37 +0000 From: Honnappa Nagarahalli To: "jerinj@marvell.com" , "dev@dpdk.org" CC: "thomas@monjalon.net" , "olivier.matz@6wind.com" , "arybchenko@solarflare.com" , "bruce.richardson@intel.com" , "konstantin.ananyev@intel.com" , "hemant.agrawal@nxp.com" , "shahafs@mellanox.com" , Gavin Hu , "viktorin@rehivetech.com" , "drc@linux.vnet.ibm.com" , "anatoly.burakov@intel.com" , "jerinj@marvell.com" , "stable@dpdk.org" , nd , nd Thread-Topic: [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for non x86 Thread-Index: AQHVtnIW1PTtutFsN0aj3sCVDCCb8afDLUsw Date: Fri, 20 Dec 2019 15:55:37 +0000 Message-ID: References: <20191219134227.3841799-1-jerinj@marvell.com> In-Reply-To: <20191219134227.3841799-1-jerinj@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ts-tracking-id: 01323055-dc2d-42fe-a32c-f6022463aa06.0 x-checkrecipientchecked: true Authentication-Results-Original: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; x-originating-ip: [217.140.111.135] x-ms-publictraffictype: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 0ff90d3a-aadd-4c6b-3a58-08d78565144a X-MS-TrafficTypeDiagnostic: VE1SPR01MB0003:|VE1SPR01MB0003:|VI1PR08MB3533: x-ld-processed: f34e5979-57d9-4aaa-ad4d-b122a662184d,ExtAddr x-ms-exchange-transport-forked: True X-Microsoft-Antispam-PRVS: x-checkrecipientrouted: true x-ms-oob-tlc-oobclassifiers: OLM:9508;OLM:9508; x-forefront-prvs: 025796F161 X-Forefront-Antispam-Report-Untrusted: SFV:NSPM; SFS:(10009020)(4636009)(346002)(366004)(136003)(39860400002)(396003)(376002)(189003)(199004)(6506007)(7696005)(8936002)(52536014)(55016002)(9686003)(64756008)(66446008)(81166006)(81156014)(4326008)(66476007)(66946007)(76116006)(66556008)(8676002)(33656002)(2906002)(86362001)(316002)(966005)(110136005)(54906003)(71200400001)(186003)(478600001)(7416002)(5660300002)(26005); DIR:OUT; SFP:1101; SCL:1; SRVR:VE1SPR01MB0003; H:VE1PR08MB5149.eurprd08.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam-Untrusted: BCL:0; X-Microsoft-Antispam-Message-Info-Original: VUT+F8EQjabhHgUspeHLypYHh70/dAS4Ni6k7XAyNy5Tja/i0YVKf3OzdUVUZXODaXvCIqhtBksFIXOiRO/SiX4tFz0RL38yRnirI6+15s0dPJkeGB+FDx1QuPf4WhyrY5JL503jnob8OrY5Yr/nPt9UhxXZy+Rme5AJpq0gXYg6GUTLAGJB0CUwOfIcLsTbFxAPE7OXdJRJoFpbSCOthtdiczIuMqAwbrtqPamjENVuuEXlzKNXqxG8BvQbWE4GZCvmLXV5bZ8ebLiSWjrEqNRd3Jd3cxw3xsHBPh+kZiExGGk0wMFSdqTWpsX4f4BsmA5fpRRAm+YHC7xf7WlkhMVJD2qFCqTZG3MP7JVhsDMLe4UtDDO9KbX7Zrb3Bg4fxv9Macr3LIKepQZiftc93i1ijtq9QBkiep9UotBmSyeu4EUXohAJLAe+huPbtdWzUIWSVsZ/k6+yR1kV4vy3DO8I58VFJMNOLmkFiDpoSho= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1SPR01MB0003 Original-Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Honnappa.Nagarahalli@arm.com; X-EOPAttributedMessage: 0 X-MS-Exchange-Transport-CrossTenantHeadersStripped: VE1EUR03FT009.eop-EUR03.prod.protection.outlook.com X-Forefront-Antispam-Report: CIP:63.35.35.123; IPV:CAL; SCL:-1; CTRY:IE; EFV:NLI; SFV:NSPM; SFS:(10009020)(4636009)(136003)(396003)(39860400002)(376002)(346002)(199004)(189003)(81156014)(81166006)(336012)(8676002)(9686003)(86362001)(5660300002)(4326008)(26005)(2906002)(26826003)(110136005)(7696005)(8936002)(478600001)(316002)(36906005)(186003)(33656002)(356004)(450100002)(70206006)(52536014)(966005)(6506007)(55016002)(76130400001)(54906003)(70586007); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR08MB3533; H:64aa7808-outbound-1.mta.getcheckrecipient.com; FPR:; SPF:Pass; LANG:en; PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com; A:1; MX:1; X-MS-Office365-Filtering-Correlation-Id-Prvs: dc737c0e-d199-486b-a9a8-08d785650e95 NoDisclaimer: True X-Forefront-PRVS: 025796F161 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: DtwW7dAOSDUA8eqL1a+fLy6tzzV7e03uWzUrB/ipb7pTT2TinUlhLmcUnmnNItTQYKuEVbiz3CUvfuUkpOGLZTdGnEXkkdZ5Hd4Sc1rwtljcFxjwEdhq/VMbU3Gt60MoEY9oGwZdMcLHR/hd2RadD2QdCfPKFXwqOjXeEGhxLw8Fq4TeoIR2y53xikd+RgwoR0zDPdK7A4KCt7aKXd8c8j+WgZE+tNBKXoUsG5dsD/LpGvwkXryPB8SUiPPGwJreydjNEQTRNntX4W27dAakLrAWlZHK4wMIEXe9iXn9latSM6neuZ4VLXRMoboXtZdti9HbADKwh7cLBu8Kpg33x2kRanCPbmyKGiBCsvmrh3Rc9A5cz/FcWkrkzvKuHcf4I43S4xebt9teiKZ8muNmF4Z6xMDkdBV7Xw+rsQeZvDEb2QtAZ1ddNpHb5YUDuCnEpfnyjhFsoQ2gfahSmvchfUDyh/uT04nWTuG0pHy6laE= X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Dec 2019 15:55:47.3363 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0ff90d3a-aadd-4c6b-3a58-08d78565144a X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d; Ip=[63.35.35.123]; Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR08MB3533 Subject: Re: [dpdk-dev] [PATCH] mempool: fix mempool obj alignment for non x86 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >=20 > From: Jerin Jacob >=20 > The exiting optimize_object_size() function address the memory object > alignment constraint on x86 for better performance. >=20 > Different (Mirco) architecture may have different memory alignment > constraint for better performance and it not same as the existing > optimize_object_size() function. Some use, XOR(kind of CRC) scheme to > enable DRAM channel distribution based on the address and some may have > a different formula. If I understand correctly, address interleaving is the characteristic of th= e memory controller and not the CPU. For ex: different SoCs using the same Arm architecture might have different= memory controllers. So, the solution should not be architecture specific, = but SoC specific. >=20 > Introducing arch_mem_object_align() function to abstract the differences = in > different (mirco) architectures and avoid wasting memory for mempool > object alignment for the architecture the existing optimize_object_size()= is > not valid. >=20 > Additional details: > https://www.mail-archive.com/dev@dpdk.org/msg149157.html >=20 > Fixes: af75078fece3 ("first public release") > Cc: stable@dpdk.org >=20 > Signed-off-by: Jerin Jacob > --- > doc/guides/prog_guide/mempool_lib.rst | 6 +++--- > lib/librte_mempool/rte_mempool.c | 17 +++++++++++++---- > 2 files changed, 16 insertions(+), 7 deletions(-) >=20 > diff --git a/doc/guides/prog_guide/mempool_lib.rst > b/doc/guides/prog_guide/mempool_lib.rst > index 3bb84b0a6..eea7a2906 100644 > --- a/doc/guides/prog_guide/mempool_lib.rst > +++ b/doc/guides/prog_guide/mempool_lib.rst > @@ -27,10 +27,10 @@ In debug mode > (CONFIG_RTE_LIBRTE_MEMPOOL_DEBUG is enabled), statistics about get > from/put in the pool are stored in the mempool structure. > Statistics are per-lcore to avoid concurrent access to statistics counte= rs. >=20 > -Memory Alignment Constraints > ----------------------------- > +Memory Alignment Constraints on X86 architecture > +------------------------------------------------ >=20 > -Depending on hardware memory configuration, performance can be greatly > improved by adding a specific padding between objects. > +Depending on hardware memory configuration on X86 architecture, > performance can be greatly improved by adding a specific padding between > objects. > The objective is to ensure that the beginning of each object starts on a > different channel and rank in memory so that all channels are equally loa= ded. >=20 > This is particularly true for packet buffers when doing L3 forwarding or= flow > classification. > diff --git a/lib/librte_mempool/rte_mempool.c > b/lib/librte_mempool/rte_mempool.c > index 78d8eb941..871894525 100644 > --- a/lib/librte_mempool/rte_mempool.c > +++ b/lib/librte_mempool/rte_mempool.c > @@ -45,6 +45,7 @@ EAL_REGISTER_TAILQ(rte_mempool_tailq) > #define CALC_CACHE_FLUSHTHRESH(c) \ > ((typeof(c))((c) * CACHE_FLUSHTHRESH_MULTIPLIER)) >=20 > +#if defined(RTE_ARCH_X86) > /* > * return the greatest common divisor between a and b (fast algorithm) > * > @@ -74,12 +75,13 @@ static unsigned get_gcd(unsigned a, unsigned b) } >=20 > /* > - * Depending on memory configuration, objects addresses are spread > + * Depending on memory configuration on x86 arch, objects addresses are > + spread > * between channels and ranks in RAM: the pool allocator will add > * padding between objects. This function return the new size of the > * object. > */ > -static unsigned optimize_object_size(unsigned obj_size) > +static unsigned > +arch_mem_object_align(unsigned obj_size) > { > unsigned nrank, nchan; > unsigned new_obj_size; > @@ -99,6 +101,13 @@ static unsigned optimize_object_size(unsigned > obj_size) > new_obj_size++; > return new_obj_size * RTE_MEMPOOL_ALIGN; } > +#else This applies to add Arm (PPC as well) SoCs which might have different schem= es depending on the memory controller. IMO, this should not be architecture= specific. > +static unsigned > +arch_mem_object_align(unsigned obj_size) { > + return obj_size; > +} > +#endif >=20 > struct pagesz_walk_arg { > int socket_id; > @@ -234,8 +243,8 @@ rte_mempool_calc_obj_size(uint32_t elt_size, > uint32_t flags, > */ > if ((flags & MEMPOOL_F_NO_SPREAD) =3D=3D 0) { > unsigned new_size; > - new_size =3D optimize_object_size(sz->header_size + sz- > >elt_size + > - sz->trailer_size); > + new_size =3D arch_mem_object_align > + (sz->header_size + sz->elt_size + sz->trailer_size); > sz->trailer_size =3D new_size - sz->header_size - sz->elt_size; > } >=20 > -- > 2.24.1