From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40057.outbound.protection.outlook.com [40.107.4.57]) by dpdk.org (Postfix) with ESMTP id 6A7CD14EC for ; Sat, 19 Jan 2019 08:09:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4XZZcHray7q7bp5I/o11HLQ+WbBNipH1jjTlHuvktOE=; b=I61oF6eoqPmDGs2ihDHi2/tdn3AYb1/wXC6N+qw2M9fCDwmHsWHYdOJKOyMeBZ0gNwuEr54Y8CPVh8mlZ6RloBfI4YKhbb15o4Q+Rpu5ILDWXFFJIIm0dW65KqafS/y7rA5Aeu/+MIna6WLRM919EyeYt5JwvD/2ep4gtzPzxeQ= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB4043.eurprd05.prod.outlook.com (52.134.68.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1537.26; Sat, 19 Jan 2019 07:09:45 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::1da8:cd09:4e78:111c]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::1da8:cd09:4e78:111c%2]) with mapi id 15.20.1537.028; Sat, 19 Jan 2019 07:09:45 +0000 From: Yongseok Koh To: Honnappa Nagarahalli CC: Thomas Monjalon , "jerinj@marvell.com" , Shahaf Shuler , "Gavin Hu (Arm Technology China)" , "tspeier@qti.qualcomm.com" , "bluca@debian.org" , "dev@dpdk.org" , nd Thread-Topic: [EXT] [PATCH] config: change default cache line size for ARMv8 with meson Thread-Index: AQHUqANwYgSIDCOae0ardNgcuhiCBaWmuneAgAAHfACAAAreAIAAFkAAgAAL/oCAAA6ZAIAACYIAgActIoCAADXEgIACxPcAgAUMz4A= Date: Sat, 19 Jan 2019 07:09:45 +0000 Message-ID: <20190119070934.GA21963@yongseok-MBP.local> References: <20190109093915.40882-1-yskoh@mellanox.com> <3649611.6SvQ7ZztEu@xps> <6f5a14e478d7c92d1f08a749afac8bb785b3b492.camel@marvell.com> <4346565.rU6Rjy1soH@xps> <04F82086-5151-459B-9026-008BFD89074A@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR01CA0010.prod.exchangelabs.com (2603:10b6:a02:80::23) To DB3PR0502MB3980.eurprd05.prod.outlook.com (2603:10a6:8:10::27) authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [216.9.110.8] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB3PR0502MB4043; 6:Y+dGVuXKq20dwsqGaiNxIIAhgAEmYNEI8QG5p6xBW7CMqCAZ5Q2GgfWSf4cB6nnOKhKow/0sH80fuj0/YicTMzgm3wOUV9kIG3WKvjH19Q6KqBW4wgn2yaJoN4LDeCPiVG5qmuoP/ybAi4Zi5N+rLs9vqI7tbx7f0B7iajBnpruf/38Zgd84fqfFB/JkeeJMze646PYf3zgbWloGi4HpYUKEenLzYJ0RlXyE7aNlebBW/VjajMFfYsjkbrmVuXjm3AAtLN4BhtV3KCRZDY3/AePTOky3Wycrhi5lCvJ6sV6MVxRcKQJ4WThzxShJdvEJAhc63oHHUUdgMlBFrFBBKbBp8Z/7WWFkpxOiarXbd5K7DY23ZXo82h6LoWCqM3r4La8ya8plWEY6YraAvNulpJNPFLsx8D/Prybd/LlJT0MHwuP6D3ZActhiUe4dKTzQf3LM9lYWUQ/4U0HHL3jYyw==; 5:zvj5gxEXk8zAcvTbZ87w/GoYOAvsZEidDst2zEjfbHO9JfP+0TnN5X8Txa9rGKelIUQv/uLtgkMC1Vgas+J1UeA0f2NBRc6uxrUiX2bDOX0bcogrtx+BFeOMrqzpBIUNi3lFQppW/qK8KBVzsVFWsWapB9OCfHY9u7sXNeDzHhasQrOVbaENxX8DqF5RYrWsx7GrRye4i8FrjsB6Cp0i/w==; 7:PIy3gwMDEJEZPd3c0gqU2HpOlYBpGjFsRtB50QmXk1APTgBBnhpgFKyK3mHULUQuorPEEFNXw5OQ63rAzdtVayAHtknswzTj6w6+zTTeSDU1FQY0e8wTgfCikb8mNBKrZtADKc/Vn6S+QoZVPI577w== x-ms-office365-filtering-correlation-id: d5493ede-50e7-4ee5-7806-08d67ddd175d x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB3PR0502MB4043; x-ms-traffictypediagnostic: DB3PR0502MB4043: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-forefront-prvs: 09222B39F5 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(396003)(376002)(39860400002)(136003)(346002)(189003)(199004)(102836004)(6512007)(9686003)(561944003)(76176011)(52116002)(68736007)(256004)(386003)(6506007)(93886005)(11346002)(105586002)(6306002)(476003)(446003)(106356001)(186003)(53546011)(5660300001)(26005)(6116002)(98436002)(316002)(33656002)(25786009)(6486002)(3846002)(486006)(33896004)(6916009)(4326008)(99286004)(97736004)(54906003)(6436002)(71200400001)(71190400001)(478600001)(6246003)(53936002)(966005)(2906002)(8676002)(229853002)(14454004)(7736002)(305945005)(86362001)(1076003)(81166006)(45080400002)(8936002)(66066001)(81156014); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB4043; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: NN5AHPZkgs9zc5LOTzMxK2Yh/tIIfevGL5gsBNsalbsMsJGOxpo+AToucX/5+aZOmYurHdPDTsmmjyVo8NCgiJGuQY1ADVso8x/5FKWTesFG92J23uMkgfYkbeXTz9i7s6m/OrDHTEb6gDfWegk+X4pqxZP4Exd4LkQax72VcTtiK/sCpqjkJiHJI6l4hMaT8/keaaRglOY3iQ3McGd2YuM/2Wm4t7C2QQot1KkYH15m28tQxiozqpUvDyMx6z2mHVgGDAzHADkopCjXf41svsQKIIZWmJkr9HLyoNxZSrKStxl05ji7yfBW2sGql2jB9QuCTlBYW4tvf4yDR08cVHSl7x7uUVRE5krldcNT1u9ykvEjxXgKFhtPeEdk03iIrKxXXJasJXy5UWbL55hZMi7dRV6pDMGg36xj0Z2+ti8= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: d5493ede-50e7-4ee5-7806-08d67ddd175d X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Jan 2019 07:09:42.3739 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB4043 Subject: Re: [dpdk-dev] [EXT] [PATCH] config: change default cache line size for ARMv8 with meson X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jan 2019 07:09:48 -0000 On Wed, Jan 16, 2019 at 02:02:26AM +0000, Honnappa Nagarahalli wrote: > > > >>>>>> On Wed, 2019-01-09 at 10:22 +0000, Yongseok Koh wrote: > > > >>>>>>> On Jan 9, 2019, at 2:09 AM, Jerin Jacob Kollanukkaran wrote: > > > >>>>>>>> I think, I way forward is to add > > > >>>>>>>> config/arm/arm64_a72_linuxapp_gcc for meson. This config can > > > >>>>>>>> be used for all SoC with A72 > > > >>>>>>>> armv8 > > > >>>>>>>> implementation and may have sym link to specfific SoC to > > > >>>>>>>> avoid confusion to end users. > > > >>>>>>> > > > >>>>>>> Is config/arm/arm64_a72_linuxapp_gcc valid? Others have > > > >>>>>> > > > >>>>>> Yes. For cross compiling for A72. > > > >>>>> > > > >>>>> Any cross-compilation with meson requires a config file. > > > >>>>> The default Arm cross-compilation is done with > > > >>>>> config/arm/arm64_armv8_linuxapp_gcc > > > >>>>> which set implementor_id =3D 'generic' > > > >>>>> > > > >>>>> For native compilation, implementor_id is detected from > > > >>>>> /sys/devices/system/cpu/cpu0/regs/identification/midr_el1 > > > >>>>> > > > >>>>> So each Arm machine needs 2 things: > > > >>>>> - a cross-compilation file > > > >>>>> - settings based on implementor_id in > > config/arm/meson.build > > > >>>> > > > >>>> Yes. config/arm/arm64_armv8_linuxapp_gcc sets the implementor_id > > > >>>> =3D 'generic' which assumed to generic across all the armv8 plat= form. > > > >>>> If tomorrow there is new core from ARM which A100 with armv8.2 > > > >>>> specific we can not tune the generic params armv8.2 as it will > > > >>>> break other CPU. > > > >>>> > > > >>>> > > > >>>>>> Having not seperate IMPLEMENTOR ID is a chip design issue. > > > >>>>> > > > >>>>> No I don't think it's a design issue. > > > >>>>> If the Arm core has no modification, it does not need to be > > > >>>>> specially identified. > > > >>>> > > > >>>> Thats right. It does not need to be specially identified, then > > > >>>> should have default config is enough. > > > >>>> > > > >>>> > > > >>>>>> I think it can work around by creating > > > >>>>>> config/arm/arm64__linuxapp_gcc > > > >>>>>> and build on x86 or arm64 through > > > >>>>>> > > > >>>>>> meson build --cross-file > > > >>>>>> config/arm/arm64__linuxapp_gcc > > > >>>>> > > > >>>>> No, it is a real A72, so it should work with default settings. > > > >>>>> > > > >>>>> The only issue we have is that the default cache line size for > > > >>>>> Aarch64 > > > >>>>> is set to 128 in config/arm/meson.build, and this is wrong. > > > >>>>> The default cache line is 64 bits. > > > >>>> > > > >>>> The cache line size as per ARM spec it is IMPLEMENTATION DEFINED= . > > > >>> > > > >>> In A72 spec, it is said > > > >>> "Returns 0b010 to indicate that the cache line size is 64 bytes." > > > >>> But I guess we cannot say it is always true for all models. > > > >>> So let's assume there is no default. > > > >> > > > >> Please note, A72 is not armv8 spec. A72 is just an IMPLEMENTATION > > > >> of armv8. > > > > > > > > Yes, this my understanding. That's why I agree with you. > > > > > > > >>>> So no default there. So the default is something work on all > > > >>>> platforms. > > > >>>> Actually Cavium has machine with 64B and 128B CL and same image > > > >>>> should work on both for generic build. > > > >>>> > > > >>>>> This is already overriden for Cavium machines which have 128-bi= t > > > >>>>> cache lines. > > > >>>>> It may be needed to do the same change for other machines > > > >>>>> (Qualcomm?) > > > >>>>> having Arm core modified to 128-bit cache lines. > > > >>>> > > > >>>> Assume you meant 128B here. > > > >>> > > > >>> Yes, sorry I mixed bits and bytes :) > > > >>> > > > >>>> Building the image Naively(on 128B CL > > > >>>> machine) and cross compile (on x86) is not an issue. > > > >>>> > > > >>>>> The other concern is about running a generic Arm build. > > > >>>> > > > >>>> Yes. That's the ONLY concern. > > > >>>> > > > >>>>> Given 64-bit should be the default, generic builds will have > > > >>>>> this value. > > > >>>>> Is it a big issue for running generic 64-bit build on Cavium > > > >>>>> machines? > > > >>>> > > > >>>> Cavium has both 64B and 128B CL machines. So putting generic > > > >>>> form, > > > >>>> > > > >>>> You can run 128B configured image on 64B machine, It will waste > > > >>>> some memory not beyond that. Other way around will result in HW > > > >>>> misbehavior. > > > >>>> ie Running 64B CL image on 128B target. > > > >>> > > > >>> Indeed it is the main concern. > > > >>> Running DPDK tuned for 128 bytes on a core having 64 bytes cache > > > >>> line will result in lower performances. It is less an issue than > > > >>> HW misbehavior. > > > >> > > > >> Do you see performance issue or it more memory usage? It nothing d= o > > > >> with thread just of out curosity. Becase, our 64CL machine does > > > >> take more memory, performance seems to same for both. Note we are > > > >> using 512MB hugepage size. > > > > > > > > Yes, we see better performance with 64B cache line on Bluefield. > > > > > > > >>> If we agree to keep 128 bytes as generic cache line size for Arm, > > > >>> we need a way to get 64 bytes size for unmodified cores. > > > >>> In other words, the generic build settings must be different of > > > >>> the default settings. > > > >> > > > >> Please send a patch. > > > >> > > > >> If MIDR value is set to A72, we can set to 64B cache, no issue. > > > >> > > > >>> Please make a difference between default 'armv8' and 'generic' > > > >>> as implementor_id in config/arm/meson.build. > > > >>> I propose arm64_armv8_linuxapp_gcc being the default config (for > > > >>> armv8) > > > >>> and creating arm64_generic_linuxapp_gcc for the generic build (fo= r > > > >>> distros). > > > >> > > > >> It should be inline with how distro guys build the image. I guess > > > >> we dont want DPDK to be a exception. > > > > > > > > The machine option is specific to DPDK, so we can define it as we w= ant. > > > > > > > >> Please check below thread and patch. > > > >> > > > >> > > > https://emea01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fm= ai > > > >> ls.dpdk.org%2Farchives%2Fdev%2F2019- > > > January%2F122676.html&data=3D02 > > > >> %7C01%7Cyskoh%40mellanox.com%7C96576e66c4b6434b47ad08d6764 > > 2 > > > be9a%7Ca65 > > > >> > > > > > 2971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636826426371146146&a > > > mp;sdata=3D > > > >> > > > > > MNMzpjs7e71l4vZAmoyqicpElp7UIFO48UuamggQWHQ%3D&reserved=3D0 > > > >> > > > https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2F= pa > > > >> > > > > > tches.dpdk.org%2Fpatch%2F49477%2F&data=3D02%7C01%7Cyskoh%40m > > > ellanox > > > >> .com%7C96576e66c4b6434b47ad08d67642be9a%7Ca652971c7d2e4d9 > > ba > > > 6a4d149256 > > > >> > > > > > f461b%7C0%7C0%7C636826426371146146&sdata=3DokIrz7Idc8t7nMbFkc > > > RnjxZg > > > >> 2wMn9ZTjqaTLlEXCnaU%3D&reserved=3D0 > > > >> > > > >> Debian folks are building like this for the _generic_ image. > > > >> What ever works for every distros, I am fine with that. > > > >> > > > >> meson configure -Dmachine=3Ddefault > > > >> meson build > > > >> cd build > > > >> ninja > > > >> ninja install > > > > > > > > I think we agree on the idea of having different configs for > > > > unmodified A72 core and generic build working for all. > > > > The remaining bits to discuss are: > > > > - do we want to use the armv8 config for unmodified A72? > > > > - what should be the name of the generic config? > > > > > > > > When digging more the config files in meson, I found this: > > > > > > > > > > > > > https://emea01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fmes= o > > > > nbuild.com%2FCross-compilation.html%23cross-file- > > > locations&data=3D02 > > > > %7C01%7Cyskoh%40mellanox.com%7C96576e66c4b6434b47ad08d6764 > > 2b > > > e9a%7Ca652 > > > > > > > > > 971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636826426371146146&am > > > p;sdata=3DJ8 > > > > XiCovgwqxm8HHRCJ5bSUbx4yTHCO2YuZz2ryZJx8I%3D&reserved=3D0 > > > > It says that distros or compilers should provide some config files. > > > > It means we should check if some standard names are emerging and tr= y > > > > to follow the same naming, or even re-use existing config files. > > > > > > I'll come up with a new patch based on the discussion here. > > > A few things noted, > > > - we still want it to be 128B for generic build > > > - we at least agreed on changing it to 64B for A72 > > How will this be done? Will you add > > config/arm/arm64_bluefield_linuxapp_gcc? > I asked this question as there was a proposal containing 'a72' in the fil= e > name. IMO, the file name should contain 'bluefield', not on a72. Sorry for late reply. It's been buried for some reason. :-) Nope, I won't create such a file. That's for cross-compiler AFAIK. I'm thinking about changing meson.build. Currently, one CL size is applied = to all kinds of cores. Consequently, for armv8a, both 'default' and 'a72' have= to have the same CL size. And one more thing raised from ARM was that 'crypto'= in -march can't be a default. > > > - As Qualcomm Centriq CPU has 128B cache line with A72, they should > > > create a profile in meson.build based on their impl_id. > > Qualcomm is not A72 core. Can it not use RTE_CACHE_LINE_SIZE from > > 'flags_generic' in config/arm/meson.build? We can add flags_qualcomm for their processor. Thanks, Yongseok