Re: [dpdk-dev] tools brainstorming

["Cao, Waterman" <waterman.cao@intel.com>]

On 2015/3/20 22:52, Thomas Monjalon wrote:
> Hi,
>
> As you probably know, a MAINTAINERS file is being filled, which is a great
> help to request patch reviews and discuss design with the knowledgeable people
> of this young DPDK community:
> 	http://dpdk.org/browse/dpdk/tree/MAINTAINERS
>
> The next step is to clearly define what are the guidelines to review a patch
> and accept it. So let's write a new document CONTRIBUTING (or another
> capitalized file ;). It will help contributors to do the right checks
> before submitting, and will help reviewers.
>
> As we are lazy developers, writing guidelines is not enough. It must be
> coupled with the integration of some tools. Let's work on these ones:
> 	- make autotests easier and faster to run for smoke testing
> 	- automated basic testpmd check
> 	- build check with various options combinations
> 	- abi check (started with validate-abi.sh)
> 	- static analyze (clang, free online coverity)
> 	- comment check (doxygen, codespell, kerspell)
> 	- format check (customized checkpatch)
>
> I'm sure this last item will trigger a lot of debate.
> Actually, format checking can be of two kinds:
> 	- commit message formatting (how to write the title, how and when adding
> 	Fixes tag, Signed-off-by tag, etc);
> 	- coding style might deserve its own document.
>
> At the end, we should be able to pass a "make check" on the whole code and
> a "make checkpatch" before submitting.
> Then the result of these tools could be automatically checked and displayed
> in patchwork or in an adapted version of qemu's patchew. But this is
> obviously a later step.
> When all automatic lights are green and human design review is properly done,
> the patch can be acknowledged by one or many reviewers. Speaking about that,
> it would be helpful to have a column in our patchwork to summarize the counts
> of tests, reviews and acknowledgements.
>
> Comments and contributions are more than welcome!
>
Hi Thomas,

     That's good idea to check patch before merging it into branch.
     We can perform basic test per each patch and improve the quality of
patch.
    
    As you knew, currently Intel DPDK test team maintained automation
test tool to perform build check and smoke test on a lot of mainstream
platforms.
    It will a good chance to share these knowledge with whole DPDK
community.

- Daily Build Test
     So far, Intel test team run daily build test on CentOS6.5, Fedora
18/20/21, RedHat 6.5/7.0, SUSE 11 SP2/SP3, Ubuntu 12/14, Oracle Linux
6.4 and FreeBSD 10.
     In addition, we also verified with different compilers, kernels and
DPDK build options.
     Since Our daily build test is focus on master branch and only
monitor latest code changes.
     Maybe we don't need to check so much OS per each patch, just make
quick build check with short list.
     We can share our build script with contributors/maintainer. they
can use it to verify their patch set.

- Automated Smoke Test
   Based on DTS (DPDK test suite), we already built up automated smoke
test on FC16/18/20/21/ , Ubuntu and Redhat. it's composed of unit test
and function test for dpdk sample application.
    I think that it's easy to build up automated smoke test based on
patch, we just need to define which test cases should include in the
list, and make sure if it can achieve at shortest time.

- Bug Tracking
    During our test cycle, we found some defects in release candidates. 
But it's difficult to track/report them without public bug tool.
    It's really helpful to get one formal tool to manage these
information and speed up bug fixing.
   
In addition, I think that patchwork is a good tool, which provides a
place to show test result for each patch.
But patchwork is focus on patch level, we need to think how to test
latest code branch in package level.
Finally, we are eager to share our experience of validation with DPDK
community.
We would like to contribute tool and script,  and help to improve
quality of DPDK release.

regards

Waterman