From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 248055699 for ; Fri, 14 Jul 2017 18:17:46 +0200 (CEST) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 14 Jul 2017 09:17:46 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.40,359,1496127600"; d="scan'208";a="108173016" Received: from fmsmsx106.amr.corp.intel.com ([10.18.124.204]) by orsmga004.jf.intel.com with ESMTP; 14 Jul 2017 09:17:46 -0700 Received: from fmsmsx118.amr.corp.intel.com (10.18.116.18) by FMSMSX106.amr.corp.intel.com (10.18.124.204) with Microsoft SMTP Server (TLS) id 14.3.319.2; Fri, 14 Jul 2017 09:17:45 -0700 Received: from fmsmsx113.amr.corp.intel.com ([169.254.13.214]) by fmsmsx118.amr.corp.intel.com ([10.18.116.18]) with mapi id 14.03.0319.002; Fri, 14 Jul 2017 09:17:45 -0700 From: "Wiles, Keith" To: Thomas Monjalon CC: Stephen Hemminger , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH 01/19] devtools: add simple script to find duplicate includes Thread-Index: AQHS/Ly520LBajjAkk+rZBxLPunlSA== Date: Fri, 14 Jul 2017 16:17:45 +0000 Message-ID: References: <20170711185546.26138-1-stephen@networkplumber.org> <7177425.p9cP4Rfg1a@xps> <12511927.qublii9NrP@xps> <62513523.FtO1afHum7@xps> In-Reply-To: <62513523.FtO1afHum7@xps> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.254.72.76] Content-Type: text/plain; charset="us-ascii" Content-ID: <9A74FA0082C21E4BAFBADA1F8D49CA29@intel.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH 01/19] devtools: add simple script to find duplicate includes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Jul 2017 16:17:47 -0000 > On Jul 14, 2017, at 10:54 AM, Thomas Monjalon wrote= : >=20 > 14/07/2017 17:39, Thomas Monjalon: >> 13/07/2017 08:56, Thomas Monjalon: >>> 12/07/2017 23:59, Stephen Hemminger: >>>> On Tue, 11 Jul 2017 22:33:55 +0200 >>>> Thomas Monjalon wrote: >>>>=20 >>>>> Thank you for this script, but... it is written in Perl! >>>>> I don't think it is a good idea to add yet another language to DPDK. >>>>> We already have shell and python scripts. >>>>> And I am not sure a lot of (young) people are able to parse it ;) >>>>>=20 >>>>> I would like to propose this shell script: >> [...] >>>=20 >>>> plus shell is 7x slower. >>>>=20 >>>> $ time bash -c "find . -name '*.c' | xargs /tmp/dupinc.sh" >>>> real 0m0.765s >>>> user 0m1.220s >>>> sys 0m0.155s >>>> $time bash -c "find . -name '*.c' | xargs ~/bin/dup_inc.pl" >>>> real 0m0.131s >>>> user 0m0.118s >>>> sys 0m0.014s >>>=20 >>> I don't think speed is really relevant here :) >>=20 >> I did my own benchmark (recreation time): >>=20 >> # time sh -c 'for file in $(git ls-files app buildtools drivers examples= lib test) ; do devtools/dup_include.pl $file ; done' >> 4,41s user 1,32s system 101% cpu 5,667 total >> # time devtools/check-duplicate-includes.sh >> 5,48s user 1,00s system 153% cpu 4,222 total >>=20 >> The shell version is reported as faster on my computer! >>=20 >> It is faster when filtering only .c and .h files: >>=20 >> for file in $(git ls-files '*.[ch]') ; do >> dups=3D$(sed -rn "s,$pattern,\1,p" $file | sort | uniq -d) >> [ -z "$dups" ] || echo "$dups" | sed "s,^,$file: duplicated include: = ," >> done >>=20 >> # time sh -c 'for file in $(git ls-files "*.[ch]") ; do devtools/dup_inc= lude.pl $file ; done' >> 3,65s user 1,05s system 100% cpu 4,668 total >> # time devtools/check-duplicate-includes.sh >> 4,72s user 0,80s system 153% cpu 3,603 total >>=20 >> I prefer this version using only pipes, which is well parallelized: >>=20 >> for file in $(git ls-files '*.[ch]') ; do >> sed -rn "s,$pattern,\1,p" $file | sort | uniq -d | >> sed "s,^,$file: duplicated include: ," >> done >>=20 >> 7,40s user 1,49s system 231% cpu 3,847 total >=20 > And now, the big shell optimization: > export LC_ALL=3DC > Result is impressive: > 2,99s user 0,72s system 258% cpu 1,436 total >=20 > I'm sure you will agree to integrate my version now :) >=20 hands down winner, where is the patch? :-) >=20 Regards, Keith