From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-HE1-obe.outbound.protection.outlook.com (mail-eopbgr10073.outbound.protection.outlook.com [40.107.1.73]) by dpdk.org (Postfix) with ESMTP id B774C1B445 for ; Mon, 14 Jan 2019 05:32:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RjX51Zi94OIvjhY0KYjE4b07lvbbrK8/Ex2nDuxAXQo=; b=iJ9omMed57wlbTTnGsWX3Le4r5//TPi3dUvU5JCLzfJQlwJ0UG5VTwHBn7UrLInbkIxXh3FcJuohyvesyrdF/AUrgcW7y59U80cJZOoc9DVKR3OUYUnCmH8thz0/B/dzJEbVKHkV332V2T7HCNEBRedg6pX0V7h7dl2ma+ucfB0= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB3946.eurprd05.prod.outlook.com (52.134.71.23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1516.18; Mon, 14 Jan 2019 04:32:32 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::1da8:cd09:4e78:111c]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::1da8:cd09:4e78:111c%2]) with mapi id 15.20.1516.019; Mon, 14 Jan 2019 04:32:32 +0000 From: Yongseok Koh To: Thomas Monjalon , Jerin Jacob Kollanukkaran CC: Shahaf Shuler , "honnappa.nagarahalli@arm.com" , "Gavin.Hu@arm.com" , "tspeier@qti.qualcomm.com" , "bluca@debian.org" , "dev@dpdk.org" Thread-Topic: [EXT] [PATCH] config: change default cache line size for ARMv8 with meson Thread-Index: AQHUqANwYgSIDCOae0ardNgcuhiCBaWmuneAgAAHfACAAAreAIAAFkAAgAAL/oCAAA6ZAIAACYIAgActIoA= Date: Mon, 14 Jan 2019 04:32:32 +0000 Message-ID: <04F82086-5151-459B-9026-008BFD89074A@mellanox.com> References: <20190109093915.40882-1-yskoh@mellanox.com> <3649611.6SvQ7ZztEu@xps> <6f5a14e478d7c92d1f08a749afac8bb785b3b492.camel@marvell.com> <4346565.rU6Rjy1soH@xps> In-Reply-To: <4346565.rU6Rjy1soH@xps> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-originating-ip: [69.181.245.183] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB3PR0502MB3946; 6:1TiVQ84AVJkHD10Z1arei6IngtIOUNUTa5bYfVXWU8d5ymPsoAfdCZ7FTSHcoeLEPXX/ec7lMdvdRZvAeqq8F6+DMCPIP3K474epGA1pZdB5OKw80my9oM6mRpmbcrxSo8jEBiSUlDXeiGHZzWmvmKcROywy5zOVSL7GicSMGe/F8+dkLVwvIPej9Kstkh1nnC/Xq016vXuJbbjTYa7SRUEiaoK8WHm9PO/Dn9FflUMyYMCrZBrS538LEm3slwvwIxt64KKE8YPEOMM0bHDtqjXp7uyFqhB7Mpgk4IkTpKALvhf87SO+ReV6E0vJ65UPQ6FSSqnIxcIR5vXY9XLlRK8SYXvdRrUTpPa90wKQY1ZdDgyO/Xqyd1sz7oNjx7u+riGpfoku5NNAVUvTlSUZf5chReMXxKB4pWc3Tm9fhIyEENeiQRp2lQeRpnzM4VEjsVRA/VBH4d1uCy6knm0Wkw==; 5:YD4hnenlpy8+dSpHGkBxBWwXYNawQyTWYaVbuisnm4iCKF9q194w/u+ZIT7FeI5v3cK79MqxQyL2Okw054OrsYBbhMHUWABHHlmwKhQM62w8kTwscSjOqDiFzPftAPKCbwoaPonqyNTmdJ5C1ph9nyy+gwG4x0llsbZZ0HRWLIy368rwshJhBrAqxFg5rJiOBnprlY8ZA6UgeUIiL1VAJg==; 7:y/gnZ3XrGE14CjQk+SO7MWIPZabVbd2mveskwDhnQU4uMk/JS5WrNY5T9hvSJAouj5fInKSSeVKliiCLDKO0yf9JChmdHUP+RJAKWuGeAjiGXUGpzG3NR9G8sz/pvydz2s2aSyxJIBvjsddqHiV8Ew== x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: 9b2a06f6-4b5b-4a78-d2dc-08d679d94ce9 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600109)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB3PR0502MB3946; x-ms-traffictypediagnostic: DB3PR0502MB3946: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-forefront-prvs: 0917DFAC67 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(366004)(136003)(376002)(396003)(39860400002)(189003)(199004)(25786009)(229853002)(36756003)(3846002)(86362001)(7736002)(93886005)(6116002)(6436002)(4326008)(110136005)(316002)(966005)(54906003)(11346002)(8936002)(8676002)(102836004)(97736004)(26005)(83716004)(71190400001)(81166006)(2616005)(446003)(305945005)(81156014)(71200400001)(68736007)(476003)(186003)(486006)(53546011)(6506007)(45080400002)(76176011)(478600001)(2906002)(33656002)(106356001)(6246003)(99286004)(82746002)(6486002)(14454004)(5660300001)(66066001)(6306002)(105586002)(6512007)(53936002)(256004); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB3946; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: DG915Pt8c1hg/AqlbV/ClWcP9lEzpyHZlDsQzbkFv4A7HnGfJiqVRgZzWGFZSZxbmOgIYW1S9aERXCzYHZeXCykAPb5nnIsSNsCNCti3Jn42vUnbmhWk6TGUUCIqcIXbnTVqZo5LsL7Kcy8FhLeuNkN4OSCnQz9wk4zCA/NnzsEEZA8/8vSUxVV1NK9IDzvMMfyfOnqYFu4jwlaYsq3wQfcsskFq8fG1TgezgWnLLIlhr5kU5kEjvrAELmzrCvAGZMtbUL7XAinLOunIsMVnm8c4+ABdtEumYn3LNfFeUQ1PgJ7tVjoJdnwv/UwAOMPIwKihCM+gLZ/3aYNSzUemXKdWOBw31Kba1/IROupwRXu9jkDAj2l1TCaBhTCZCagn4+C0wq5V4xzQt7TSNPg0Y8VDW2EILsYSqocUGVthq/U= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b2a06f6-4b5b-4a78-d2dc-08d679d94ce9 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Jan 2019 04:32:32.2729 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB3946 Subject: Re: [dpdk-dev] [EXT] [PATCH] config: change default cache line size for ARMv8 with meson X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jan 2019 04:32:37 -0000 > On Jan 9, 2019, at 6:57 AM, Thomas Monjalon wrote: >=20 > 09/01/2019 15:23, Jerin Jacob Kollanukkaran: >> On Wed, 2019-01-09 at 14:30 +0100, Thomas Monjalon wrote: >>> 09/01/2019 13:47, Jerin Jacob Kollanukkaran: >>>> On Wed, 2019-01-09 at 12:28 +0100, Thomas Monjalon wrote: >>>>> 09/01/2019 11:49, Jerin Jacob Kollanukkaran: >>>>>> On Wed, 2019-01-09 at 10:22 +0000, Yongseok Koh wrote: >>>>>>> On Jan 9, 2019, at 2:09 AM, Jerin Jacob Kollanukkaran wrote: >>>>>>>> I think, I way forward is to add >>>>>>>> config/arm/arm64_a72_linuxapp_gcc >>>>>>>> for meson. This config can be used for all SoC with A72 >>>>>>>> armv8=20 >>>>>>>> implementation and may have sym link to specfific SoC to >>>>>>>> avoid >>>>>>>> confusion to end users. >>>>>>>=20 >>>>>>> Is config/arm/arm64_a72_linuxapp_gcc valid? Others have=20 >>>>>>=20 >>>>>> Yes. For cross compiling for A72. >>>>>=20 >>>>> Any cross-compilation with meson requires a config file. >>>>> The default Arm cross-compilation is done with >>>>> config/arm/arm64_armv8_linuxapp_gcc >>>>> which set implementor_id =3D 'generic' >>>>>=20 >>>>> For native compilation, implementor_id is detected from >>>>> /sys/devices/system/cpu/cpu0/regs/identification/midr_el1 >>>>>=20 >>>>> So each Arm machine needs 2 things: >>>>> - a cross-compilation file >>>>> - settings based on implementor_id in config/arm/meson.build >>>>=20 >>>> Yes. config/arm/arm64_armv8_linuxapp_gcc sets the implementor_id =3D >>>> 'generic' which assumed to generic across all the armv8 platform. >>>> If tomorrow there is new core from ARM which A100 with armv8.2 >>>> specific >>>> we can not tune the generic params armv8.2 as it will break other=20 >>>> CPU. >>>>=20 >>>>=20 >>>>>> Having not seperate IMPLEMENTOR ID is a chip design issue. >>>>>=20 >>>>> No I don't think it's a design issue. >>>>> If the Arm core has no modification, it does not need to be >>>>> specially identified. >>>>=20 >>>> Thats right. It does not need to be specially identified, >>>> then should have default config is enough.=20 >>>>=20 >>>>=20 >>>>>> I think it can work around by creating >>>>>> config/arm/arm64__linuxapp_gcc >>>>>> and build on x86 or arm64 through >>>>>>=20 >>>>>> meson build --cross-file >>>>>> config/arm/arm64__linuxapp_gcc >>>>>=20 >>>>> No, it is a real A72, so it should work with default settings. >>>>>=20 >>>>> The only issue we have is that the default cache line size for >>>>> Aarch64 >>>>> is set to 128 in config/arm/meson.build, and this is wrong. >>>>> The default cache line is 64 bits. >>>>=20 >>>> The cache line size as per ARM spec it is IMPLEMENTATION DEFINED. >>>=20 >>> In A72 spec, it is said >>> "Returns 0b010 to indicate that the cache line size is 64 bytes." >>> But I guess we cannot say it is always true for all models. >>> So let's assume there is no default. >>=20 >> Please note, A72 is not armv8 spec. A72 is just an IMPLEMENTATION of >> armv8. >=20 > Yes, this my understanding. That's why I agree with you. >=20 >>>> So no default there. So the default is something work on all >>>> platforms. >>>> Actually Cavium has machine with 64B and 128B CL and same image >>>> should >>>> work on both for generic build. >>>>=20 >>>>> This is already overriden for Cavium machines which have 128-bit >>>>> cache lines. >>>>> It may be needed to do the same change for other machines >>>>> (Qualcomm?) >>>>> having Arm core modified to 128-bit cache lines. >>>>=20 >>>> Assume you meant 128B here. >>>=20 >>> Yes, sorry I mixed bits and bytes :) >>>=20 >>>> Building the image Naively(on 128B CL >>>> machine) and cross compile (on x86) is not an issue. >>>>=20 >>>>> The other concern is about running a generic Arm build. >>>>=20 >>>> Yes. That's the ONLY concern. >>>>=20 >>>>> Given 64-bit should be the default, generic builds will have this >>>>> value. >>>>> Is it a big issue for running generic 64-bit build on Cavium >>>>> machines? >>>>=20 >>>> Cavium has both 64B and 128B CL machines. So putting generic form, >>>>=20 >>>> You can run 128B configured image on 64B machine, It will waste >>>> some >>>> memory not beyond that. Other way around will result in HW >>>> misbehavior. >>>> ie Running 64B CL image on 128B target. >>>=20 >>> Indeed it is the main concern. >>> Running DPDK tuned for 128 bytes on a core having 64 bytes cache line >>> will result in lower performances. It is less an issue than HW >>> misbehavior. >>=20 >> Do you see performance issue or it more memory usage? It nothing >> do with thread just of out curosity. Becase, our 64CL machine does >> take more memory, performance seems to same for both. Note we are >> using 512MB hugepage size. >=20 > Yes, we see better performance with 64B cache line on Bluefield. >=20 >>> If we agree to keep 128 bytes as generic cache line size for Arm, >>> we need a way to get 64 bytes size for unmodified cores. >>> In other words, the generic build settings must be different of >>> the default settings. >>=20 >> Please send a patch. >>=20 >> If MIDR value is set to A72, we can set to 64B cache, no issue. >>=20 >>> Please make a difference between default 'armv8' and 'generic' >>> as implementor_id in config/arm/meson.build. >>> I propose arm64_armv8_linuxapp_gcc being the default config (for >>> armv8) >>> and creating arm64_generic_linuxapp_gcc for the generic build (for >>> distros). >>=20 >> It should be inline with how distro guys build the image. I guess >> we dont want DPDK to be a exception. >=20 > The machine option is specific to DPDK, so we can define it as we want. >=20 >> Please check below thread and patch. >>=20 >> https://emea01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fmail= s.dpdk.org%2Farchives%2Fdev%2F2019-January%2F122676.html&data=3D02%7C01= %7Cyskoh%40mellanox.com%7C96576e66c4b6434b47ad08d67642be9a%7Ca652971c7d2e4d= 9ba6a4d149256f461b%7C0%7C0%7C636826426371146146&sdata=3DMNMzpjs7e71l4vZ= AmoyqicpElp7UIFO48UuamggQWHQ%3D&reserved=3D0 >> https://emea01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fpat= ches.dpdk.org%2Fpatch%2F49477%2F&data=3D02%7C01%7Cyskoh%40mellanox.com%= 7C96576e66c4b6434b47ad08d67642be9a%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7= C0%7C636826426371146146&sdata=3DokIrz7Idc8t7nMbFkcRnjxZg2wMn9ZTjqaTLlEX= CnaU%3D&reserved=3D0 >>=20 >> Debian folks are building like this for the _generic_ image. >> What ever works for every distros, I am fine with that. >>=20 >> meson configure -Dmachine=3Ddefault >> meson build >> cd build >> ninja >> ninja install >=20 > I think we agree on the idea of having different configs > for unmodified A72 core and generic build working for all. > The remaining bits to discuss are: > - do we want to use the armv8 config for unmodified A72? > - what should be the name of the generic config? >=20 > When digging more the config files in meson, I found this: > https://emea01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Fmeso= nbuild.com%2FCross-compilation.html%23cross-file-locations&data=3D02%7C= 01%7Cyskoh%40mellanox.com%7C96576e66c4b6434b47ad08d67642be9a%7Ca652971c7d2e= 4d9ba6a4d149256f461b%7C0%7C0%7C636826426371146146&sdata=3DJ8XiCovgwqxm8= HHRCJ5bSUbx4yTHCO2YuZz2ryZJx8I%3D&reserved=3D0 > It says that distros or compilers should provide some config files. > It means we should check if some standard names are emerging > and try to follow the same naming, or even re-use existing config files. I'll come up with a new patch based on the discussion here. A few things noted, - we still want it to be 128B for generic build - we at least agreed on changing it to 64B for A72 - As Qualcomm Centriq CPU has 128B cache line with A72, they should create a profile in meson.build based on their impl_id. I talked to Thomas and we'll shoot it for 19.05. Thanks, Yongseok=