From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0078.outbound.protection.outlook.com [104.47.42.78]) by dpdk.org (Postfix) with ESMTP id 27B062BBE for ; Tue, 24 Jan 2017 13:57:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=hJ+nZe3GvLV2SYd2pCsP1ipxCDTWFLS1DZIfCNwUfBE=; b=RoB6Jl1LMLx3AQls7q5Jrs/BUNRCZ5Dz9+B7Q98t1ddaNJ8/wSAprYliIGE+3ht8xqoV4G2El8kju1tSJUPnxOgZwP2/+BGYfK8KDs8KtACMhVkiTzLwG9y3Wd2/LY9TiYSVqFdxkfI+QlvJCQWn+yRje7KDoQSr/7F+H3dXRvo= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Jerin.Jacob@cavium.com; Received: from localhost.localdomain (111.93.218.67) by CY1PR0701MB1726.namprd07.prod.outlook.com (10.163.21.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.860.13; Tue, 24 Jan 2017 12:57:13 +0000 Date: Tue, 24 Jan 2017 18:26:53 +0530 From: Jerin Jacob To: Bruce Richardson CC: Olivier Matz , , , Message-ID: <20170124125652.GA7561@localhost.localdomain> References: <20170123181813.00567e47@platinum> <1485191953-2613-1-git-send-email-olivier.matz@6wind.com> <20170124114013.GA4609@localhost.localdomain> <20170124121506.GA160692@bricha3-MOBL3.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20170124121506.GA160692@bricha3-MOBL3.ger.corp.intel.com> User-Agent: Mutt/1.7.1 (2016-10-04) X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: MA1PR01CA0113.INDPRD01.PROD.OUTLOOK.COM (10.174.56.157) To CY1PR0701MB1726.namprd07.prod.outlook.com (10.163.21.140) X-MS-Office365-Filtering-Correlation-Id: aa111b3f-4ea4-40dc-0122-08d444588645 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:CY1PR0701MB1726; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 3:223mVvFhj1LFsuQ89a7xbq8D0Wj32zEqDpzym7WaiTUb6SEtTWSoITzUUyJ5ftLqZ77GF3n3ilIJeTn2NKMnFHsF1ewpzXzt8Z+yogo9tztMSP9Iph5jvWSdRZT4XtQ0GJ7/rr/oTgwIHy52kHhlyXcGN+xPHSoPi8h4ZnJhVelpCaBCRwrxAiJyAuZqiDtlxKOZVWwKnULnBHnK6jVNIZCSq9hVdYZVvEJszEvu/p2AiyPHBEYOZpd6IfAr2OXYnd9wZ8fv6SEhnNJDcCpLJg==; 25:FBHqExZbFWSk2yi07f+Vdn9UkD0l9SwBEVqqAysaG1ETt+4tZ/K+23hTwBGiXOxR05GOy9THqmvLA6WmISgCtjR93PNa1NvYtberSWHgX03set3s30x5gyUBFCmbcxCJdfzqopvk3oI0DpjNN/GPN3RgAT4Pz8DblBFz7YUfkzNaZLHjJiqKN7pWQkJQ63MNrXsMpbwhPOeNo9BYZByyQ3lEI07dpBNhrPeogCn5rsmcQpbGFnu+MqljE7qJS+nVEHlj1l6kaSFGi4yCHmdWoG8IxGv7bzTn/7/n8SLpCY4Gh7uyJ1QgoIqo9Q23RX1KdoHgPykNUDED50HXr/O87vFktzsxq7N54dgdeH1BRo4bNlAbex4UHOuxi8DcT2l/4qw5wVdm3GixiFHgCAUxIf7aQ3RFq6ULlBZhFicPI8TcfazmohaqURiH/8Ci8HaJagN2Moyag9UaU3CVB9haSA== X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 31:CHBtdyjSUiDyw5sy6qsiMIVXD2j8NVBREaBfASSXLa5jVQtkURY3rsx6STB6S/J3/uPKrA6c6iyoSbWf54i5buywRRUqGO00pZ9dL5Cl4prM1+4K3dqzP/mBPi9GoCsvZ5FPLTWGbfqCOm5c8YypiLmFzuI8KSFW3hTHQmxGvzyEKHd8NvNMjGXRrVAbGYrXq7l3qUHsEP2JHg4+bm1Q7r/NBpWFREk0lYI0+I8SQmpZoIPraDCVx2Rbb72DWKQiKqBar+SeN0oMh8dBtLBFE17fBlfV+BuTlpjeFhCtgeI=; 20:B4yJGb24ctlOyeDhSUyuqCg6hj74cuptIZPxw9n4yAMKGr+3hTjjjSXtpH+ddw2JHEUNpac2dmlzXp7FTCcslDcMnS7b1WEYuIJCXrYir5OwjQntP7tfJznBOWhs7e5ZQlotY++2BZFedNCqGBA/qB5kbigBfgN0SNmkF4Gm2AWx6/7JaMdcM5SleZTHJRxPWLvKoPriQiuPAdjqX5Pm0Iinu88KhbsGeAnmn3flflDUlMX/jXLwWHFywlNh/0m40LMaYfmAwO2Zv6tfO0i73N5g4WPrvtMMIHLfvtl9Om7HI45u2MSBaCS9TIJRtHXuVWmzNGIaVsFL60gv+i1i8lLB/Nz2P0tfW/vFnyzogeaLl+mgWl6pjsOdJYejpNCSRJMHqg1gGbsOFOHy7tY8gUnBs6UjtfNs5eRxTEpDrZVn11UuLmsgNBBxf6Zq52R7cVfNqL7SGi2UCAlynp7xlxYZ0je2c5Ypx5p8lcic5DFh/+l3JpJH/X9yDQWN2QeTRIuACRaBfvAcUZEe2+Q552aMHsY0oJu0YP+MspDUuROJzYEkj4NN73UWwEMVkJD6HsZ43lvVqJXl0gHTMgjljdef9rcZ9WVFMoAxb4DDimI= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(20161123560025)(20161123562025)(20161123555025)(20161123564025)(6072148); SRVR:CY1PR0701MB1726; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0701MB1726; X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 4:odXFnYzxlBgvrpZwq5LLLDpkIP4VXdUG/Z5hAsJr1kfYrZJ0yOlvQsClK9mzEgXvJjkvqWzO0w46kiHjB6OqCJDQSkQFYKKDLBv0JNDUnBv/RufjEqitZiyWqr/Qou26fhdVPkNjNZ8NS6VJpmr5xC6aJhmVVWeh09KcUJG2EsZpVxkd5L23lywb65ghpQxoaf9H8D4zGucv8G+88hoSMfHMVrJWM4DL5gDuO8A91iXOV0gcDNUoBdWrABNMaLH1l8tITUN+4jqd3ILxJEQHjoQe3q0oboXF186nPMsrccZ71iX4IMVshsZFtBAJbkK2to74k3ZIBASsI5n6INeUey8JuVzF6CMBmEJxUOJ4sVviCcHCMwXIBGZ9wo61v9jOyFqnmBk6IO0/pb0Qn2UwUU3MO7WGpvznkM0pE/Wl5JSeBxc+cGXEBDASYz9DqCBzXl5+JfaSWW+Aa6XflE/EqcgzqlQ2ATwbSf/Is/hu5TMJk+NP65x6aoL0TE2d36OAccFNCB8qZxh8pUE9DGJg1hEMYpxYUyPp6/BHixoncrJw0rZX9B2MMP25pDirZ0WBhCYuaBW0R+AjYrhp2KkHzg== X-Forefront-PRVS: 0197AFBD92 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(7916002)(39450400003)(199003)(50944005)(189002)(24454002)(3846002)(6116002)(1076002)(4001350100001)(189998001)(106356001)(92566002)(23676002)(105586002)(97736004)(50466002)(42186005)(81166006)(8676002)(81156014)(33656002)(76176999)(5660300001)(53936002)(93886004)(54356999)(101416001)(6916009)(42882006)(110136003)(2950100002)(47776003)(6666003)(66066001)(229853002)(50986999)(83506001)(6506006)(7736002)(25786008)(9686003)(55016002)(54906002)(61506002)(305945005)(5009440100003)(68736007)(2870700001)(2906002)(4326007)(38730400001)(7099028)(18370500001); DIR:OUT; SFP:1101; SCL:1; SRVR:CY1PR0701MB1726; H:localhost.localdomain; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtDWTFQUjA3MDFNQjE3MjY7MjM6dm9weEQ1d0gvZnRlajlDRERGYkJ6cmRk?= =?utf-8?B?RGUvVTAwbW1PNkVVcUxGZWhJVE5IdkZSbW5mTlJGSCs4UEp0cEFzUjQybEc1?= =?utf-8?B?bVhQOThqYTN1VTRINlg4ODg1NVJUbjBlTFNDQzlvajhOSURIbzVoeFNQN1p3?= =?utf-8?B?V0R3UmlwZGNyMkcyUnNYU3ozd1h3a0V4WmRVZjFOb3BVRlMwZVExOUVxU3lz?= =?utf-8?B?UlNPb3FEYnF0VHdHaEhFQmNSVVI5TTJuRW9sWCt0NlkxRHRDVHFuZlBDZDVR?= =?utf-8?B?bEZTK0VlMkU2bjYrODZYYmljQjZ1cTVrYndJRWhDWm1RZTVWdStTblltR0Mx?= =?utf-8?B?UGpPTTdac1V6bnkwMkxPb1lLK1k2NW1Sc2Vrc1p6WWxRbmlXQ2RJWi8xNU1k?= =?utf-8?B?VmoycldjTEI2aTlhbyt0bnNneGFIeEhGdW9hdXdQRDlPL2RsM3BWbVVQbmJp?= =?utf-8?B?dmhSRTN0elNtR2pmdFRMRndLYmZkVjFSdzIyR2wyVHlQY1BPemhyT1Q1S0g3?= =?utf-8?B?OVh6WmJuRHMyandhMFVwalFGR1pvbFo0dU0zUnQ3bXE3WUhiMDl4aFphSmp4?= =?utf-8?B?RFdUbUgxTzNHMlpOK1pwc05INzFJc0Q3YWlDR3ZYTHhiZyt0N3VlSE02SkpD?= =?utf-8?B?Q0hvSHdPSWZ4M09EUWFGNmZsaVpmYkprbk4wckNSRXY5NGptR3FyM2U1UlEy?= =?utf-8?B?SlgzZVR6Rkp6M0pzR3QyWjVEenNPQTdzc00wY0FpUVpSc1ZKTUtaSEU1L2RJ?= =?utf-8?B?RnhmTGt1MW5mUldWbGJkRUp4OG9YMnNpU2M3bmlzLzhDd0duVnpWZnA3dUJP?= =?utf-8?B?dWVvb0QwUktUeHJVejRnUEkxRW1EbXhMUjFIdGl2bGEwclQyNWZacnJmVXBv?= =?utf-8?B?NTRJQkpSRjI4UkxGNUZlaGRac0NwNnMxaStxRktqVXFZcEp1SUltUnpySkhG?= =?utf-8?B?ZU5LVG5PeDR3U1lOUzExckVnVGQ0MVJ0bTVNUTJxSXRPNExMSjN4dDR0anBT?= =?utf-8?B?clQzYU5NOThnaFZock5xK0dWSkNTNjY0YXZsRXVjaklpMng2NmZWbElYbEtJ?= =?utf-8?B?QWlBNG1PeXVWWEZWMzF6YkhIVUdKRUY1N2FvcGdxdFJFVytOYmQ1VlR1cVlP?= =?utf-8?B?cmh0VENUcW5mOW0yK0NQWjJJL2g5K3dGOGlDYTBQZDRFQlF6R3drdHBwTVVu?= =?utf-8?B?blU5R2NyWjYyUk1qbE1ielRRY0FKVm1kei9mY3hxdkQ2c3BWWW5za0djZC9Q?= =?utf-8?B?VmhLb3ZhNGVNdjFCR1VTSmpCbTFXTkkyZ2ltc2ZpTTM3VUFVaTNEanpQZXB2?= =?utf-8?B?WVg3dEtIc1JQcFFxWUl0R2kzd3JWZ3Q4VHhEU0lXTVdlZEs4UHM4am02MENi?= =?utf-8?B?L3VqM2JpSWJXOFJQWUs1Y1lWUGl6d0xlKzIybU1YL1h5ZXdRUmEzUEZ5YmtQ?= =?utf-8?B?WFVHekt2YWtRZURudmM1aHhKRHM0QmFmT2hnWEZ0WXRHbUtCTVNCakcwQXg4?= =?utf-8?B?Z1hrcDNKWWVtN1lWK3hzL2RQTkkyNGdTbnExd2ZpMHQ2YjZSS1BjdzcrM1hL?= =?utf-8?B?ZzFRclJkb0I0eWltQkhrekxDV2tiOVpHaXUyd1loMzRKaXNKYmQ3K3B4cGxK?= =?utf-8?B?emtCb2xVOTR4MlRPam9NRGRRN21ueDhxWnpONEIrdDhJcDdGV3lwMmpISnBu?= =?utf-8?B?MTRlRlM3VmtQUGduaVVJcXdVS01vVTMwTDd0SjJDdHlOaHhBQWFnTFo4bXdF?= =?utf-8?B?Z295R3VFUWZUaTZzOVhpaDcvOEFHM2RwbWpKTlZTbVNNQktvbVNUYlQ1aHBK?= =?utf-8?B?QUhyZWl4NWhEWHlET1hCWGdZNkFRTEc5MlNCbFpmMkxsZmNJNUgrUVhWKzh0?= =?utf-8?Q?7WQddenMVGXgg=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 6:/xd8JYpZk1WaB0a+W9rrSVau+iO3Vicee8QXTcJRB0B08EqTyNelvQolNGFyzD/qr4fkdjsp21M9Et00yfUNARNwZbf2kmd9+66v6nP/6WEYrTf7QNHYioBT0NdK+aIxz8TjuzaUyeFHKmi1O3SLunglNroOiPyWLxPH+cyXzvnYY2g3+QGaF/R7BTptLAXCGpQQy3h+K8q444rOY49ajC3CVG04VomG8274lV6VH7moe/SJMYcVrUgTDaOz4Sps1RL8jUi/SgxGCue2nVcD/mUyVZAvC1IcuoykdZAmquzOi7+VRrFca1y/OcDni4dojtfJxMkMyNiFgi6iH4RIw/bFnsZNIFiZ81uovzwJP8RYueO4ad71tnhhQWFWTh+a4KQekPTIKDCarr2XoGwYvjUD5HrMpPkRhJ0zBSU2gfA=; 5:mC0O/r51QYdG2QD6QhUqs34hFPhdq3NckHvMVZDL/+5W3bYuB/wLlXiTDMlErCtn3BoBc5O3qOus5HjdVcOesQbCRs6RLisSxW80CFgPhm/ExfnAYv/vdxvkg2Et2m70zLRwmnkrsPCVL2HNgPIZc0sYMCPsu5Fsp5Kfg97yNXc=; 24:CFpq6VZuVL3HgCwiVh0NQeIrV8FXLvopQWzQLOr/Fedgu2HRUgA/HHR5IlrVnQ4QqKLapHgBM/xTCRrCsjrcjI8X+i3DGhE3YTVf0elT2F4= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY1PR0701MB1726; 7:btD+/wMxSXXB6dk+ExctIGi8EkBfmP87BhGnd1a3FagD9W2ZTOXIsZt3cE5l31SAgZ7SLDzmEKljFSVP37gu4EUDwgyACPsK/0UTw0sCDun1wVBdEYtiRthnKVa3sJSWAk98khDRTILKe44/0qXfp/EjDFBuaNtdRK2jZ5Jx39i1S/dukxHTVsB9NytsXYGmJ0eg6mjSWYQ6GJxYL9mrv1+rFk/BauzVFPOCLxpb8esmjc4Kx3r7AaWT2CpuUWAgkRTkfgVHr/q+p9ElO0azt97IW8eFGJRBaORFUhCcj51aj32KLpq/HXG1Em/Che0pCAH9iNu3nQzdlSqyiZp61K7fyWtOfvmz84cDM7dH0WKxigaxmD95+n4A9uax83kLY1BrbMg8qh+bhdrYFlYgQspeF9CCBBpVKjlnPfHso2VjKbNpOfc3MvGn7JFqkZ1ZQMhFvv1qPNAU6+jSQD37lA== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jan 2017 12:57:13.7498 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0701MB1726 Subject: Re: [dpdk-dev] [PATCH] mk: optimize directory dependencies X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2017 12:57:18 -0000 On Tue, Jan 24, 2017 at 12:15:06PM +0000, Bruce Richardson wrote: > On Tue, Jan 24, 2017 at 05:10:15PM +0530, Jerin Jacob wrote: > > On Mon, Jan 23, 2017 at 06:19:13PM +0100, Olivier Matz wrote: > > > Before this patch, the management of dependencies between directories > > > had several issues: > > > > > > - the generation of .depdirs, done at configuration is slow: it can take > > > more than one minute on some slow targets (usually ~10s on a standard > > > PC). > > > > > > - for instance, it is possible to expressed a dependency like: > > > - app/foo depends on lib/librte_foo > > > - and lib/librte_foo depends on app/bar > > > But this won't work because the directories are traversed with a > > > depth-first algorithm, so we have to choose between doing 'app' before > > > or after 'lib'. > > > > > > - the script depdirs-rule.sh is too complex. > > > > > > - we cannot use "make -d" for debug, because the output of make is used for > > > the generation of .depdirs. > > > > > > This patch moves the DEPDIRS-* variables in the upper Makefile, making > > > the dependencies much easier to calculate. A DEPDIRS variable is still > > > used to process library dependencies in LDLIBS. > > > > > > After this commit, "make config" is almost immediate. > > > > > > Signed-off-by: Olivier Matz > > > > Tested both approach on ThunderX. This patch looks better > > > > Tested-by: Jerin Jacob > > > > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc > > Configuration done > > > > real 0m18.112s > > user 0m2.810s > > sys 0m0.660s > > > > ➜ [master]GB-2S [dpdk-master] $ pwclient git-am 19859 > > Applying patch #19859 using 'git am' > > Description: [dpdk-dev] mk: parallelize make config > > Applying: mk: parallelize make config > > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 8 > > Configuration done > > > > real 0m2.812s > > user 0m3.020s > > sys 0m0.870s > > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 16 > > Configuration done > > > > real 0m1.748s > > user 0m3.040s > > sys 0m1.020s > > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 32 > > Configuration done > > > > real 0m1.422s > > user 0m3.380s > > sys 0m1.080s > > > > ➜ [master]GB-2S [dpdk-master] $ pwclient git-am 19918 > > Applying patch #19918 using 'git am' > > Description: [dpdk-dev] mk: optimize directory dependencies > > Applying: mk: optimize directory dependencies > > > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc > > Configuration done > > > > real 0m0.064s > > user 0m0.000s > > sys 0m0.000s > > ➜ [master]GB-2S [dpdk-master] $ rm -rf build > > ➜ [master]GB-2S [dpdk-master] $ time make config T=arm64-thunderx-linuxapp-gcc -j 8 > > Configuration done > > > > real 0m0.055s > > user 0m0.000s > > sys 0m0.000s > > > > I agree that Olivier's patch is faster. However, I think I prefer having > the library dependencies in the makefiles for the libs themselves rather > than up a level. Given that we are only looking at ~2second of a > difference here in your tests - assuming -j flag - what is the actual > build time differences? My suspicion is that after Ferruh's simpler Without patch - 18sec With patch -j1 - 18 sec With patch -j2 - 9.2 sec With patch -j4 - 4.9 sec With patch -j8 - 2.8 sec With patch -j16 - 1.7 sec With patch -j32 - 1.4 sec > patch is applied, the config time becomes such a small part of the > build, that the extra benefits from Oliviers work is not worth the extra > complexity. The low-end embedded SoCs (SoC with 2 to 4 cores) will be get benefited out of bring this extra complexity.My take is, if it is manageable complexity then take the most optimized one.