From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ernst.netinsight.se (ernst.netinsight.se [194.16.221.21]) by dpdk.org (Postfix) with SMTP id 33BFD5949 for ; Mon, 2 Nov 2015 08:54:39 +0100 (CET) Received: from [10.100.1.152] (unverified [10.100.1.152]) by ernst.netinsight.se (EMWAC SMTPRS 0.83) with SMTP id ; Mon, 02 Nov 2015 08:54:31 +0100 To: "Liu, Yong" , "dev@dpdk.org" References: <86228AFD5BCD8E4EBFD2B90117B5E81E10F379B0@SHSMSX103.ccr.corp.intel.com> From: =?UTF-8?Q?Simon_K=c3=a5gstr=c3=b6m?= Message-ID: <563716B5.4090809@netinsight.net> Date: Mon, 2 Nov 2015 08:54:29 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E10F379B0@SHSMSX103.ccr.corp.intel.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] dpdk pre-patch testing introduction X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2015 07:54:39 -0000 Hi Liu! I think this is a really, really great addition! On 2015-10-29 16:19, Liu, Yong wrote: > Currently the patch testing only run unit test and basic function test on this platform. > It can make sure that new patch doesn't break original code and functions. > Due to coverage limitation, it can't verify the functionality of new patch. > If you want to verify your patch's new function, you need to check with tester and perform specific testing on it. Speaking of coverage, I do have a few suggestions: 1. Build and test both with and without optimizations (build with EXTRA_CFLAGS="-O0"). While uncommon, I've seen code break in the non-optimized case because the compiler simply removes some unused code which would otherwise cause link errors. 2. I think it would be very good to collect code coverage for patches as well. (Yes, this is pushing my own tool) I've written a code coverage tool called kcov [1], which only needs debugging information (-g) in the binary to collect code coverage. It outputs results in various formats: lcov-style HTML, cobertura XML and can post directly to coveralls.io. It's also simple to run: kcov --include-pattern=dpdk /tmp/kcov-output ./testpmd [args...] To the mail report, I think it would be good to add information about code coverage increase/decrease with the patch and ideally which lines of the patch which were covered Coverage: 69.2% (+5 lines with the patch) Coverage for the patch: 5/7 lines both of these should be fairly simple to parse out from the XML output. I'd be happy to help you if think it's a good idea! // Simon [1] https://github.com/SimonKagstrom/kcov