[dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Thomas F Herbert]

This is a proposal to create a patch-test dpdk tree. I welcome feedback 
from the dpdk community on this proposal.

This tree will consist of pre-integrated patches submitted to dpdk-dev 
mailing list. The purpose of this tree is to help reduce the effort in 
reviewing patches and providing feedback to submitters of patches. Also 
integrators of dpdk can benefit by having a "linux-next" like tree to 
clone and or try new features for performance and breadth of hardware 
support. It is the hope that the patch-test tree will provide early 
feedback to patch reviewers as well as an assurance of quality to the 
project maintainer.

Work Flow and Process:

All patches will be taken from from public submissions to dpdk-dev.org 
scraped from dpdk patchwork. Patches will be applied to the patch-test 
tree and tested against HEAD as they are received. The feedback from the 
testing will be provided to the community. The patch-test tree will 
periodically be git pull'ed from dpdk.

Longer term goal:

Initially, the patches will be applied along with some simple smoke 
tests. The longer term goal is to automate this process, apply more 
extensive tests and post the results in dpdk patchwork, 
http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an 
accompanying mailing list for distribution of a results summary of the 
tests.

Submissions of test suites will be encouraged from the community 
including testing against compilers, architectures, devices, options 
etc. The purpose is to try to evolve to widen test coverage for 
submitted patches.

--Thomas Herbert

Re: [dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Vincent JARDIN]

On 27/05/2015 22:48, Thomas F Herbert wrote:
> Work Flow and Process:
>
> All patches will be taken from from public submissions to dpdk-dev.org
> scraped from dpdk patchwork. Patches will be applied to the patch-test
> tree and tested against HEAD as they are received. The feedback from the
> testing will be provided to the community. The patch-test tree will
> periodically be git pull'ed from dpdk.
>
> Longer term goal:
>
> Initially, the patches will be applied along with some simple smoke
> tests. The longer term goal is to automate this process, apply more
> extensive tests and post the results in dpdk patchwork,
> http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an
> accompanying mailing list for distribution of a results summary of the
> tests.

thanks for helping.

It could be broken into two parts:
   - patch-test-net-next,
http://dpdk.org/browse/dpdk/tree/MAINTAINERS#n190
http://dpdk.org/browse/dpdk/tree/drivers/net

   - patch-test-other-next,
all, excepted drivers

Best regards,
   Vincent

Re: [dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Thomas F Herbert]

On 5/27/15 5:06 PM, Matthew Hall wrote:
> On Wed, May 27, 2015 at 04:48:47PM -0400, Thomas F Herbert wrote:
>> Submissions of test suites will be encouraged from the community including
>> testing against compilers, architectures, devices, options etc. The purpose
>> is to try to evolve to widen test coverage for submitted patches.
>
> Hi Thomas,
>
> This overall idea is extremely awesome and will help me a lot so I won't have
> to try and sit there yanking in random patches from email threads.
Thanks!

In the meantime, you can yank  patches from 
http://dpdk.org/dev/patchwork/project/dpdk/list/ which might be easier 
then sorting through the emails depending on your email client.
>
> I am not sure if it will be possible or not, but I'd like to see some test
> suites against some applications as well. They do more complex operations in
> my experience than the operations the DPDK tests do.
I will accept submitted test suites toward this goal if you would like 
to write a test suite.
>
> Perhaps something like an L4-L7 firewall app or an IDS / IPS app or an L3-L4
> cloud switch / router app. Etc.
>
> Matthew.
>

Re: [dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Simon Kågström]

I spot a can of worms to be opened here :-)

On 2015-05-27 22:48, Thomas F Herbert wrote:
> Work Flow and Process:
> 
> All patches will be taken from from public submissions to dpdk-dev.org
> scraped from dpdk patchwork. Patches will be applied to the patch-test
> tree and tested against HEAD as they are received. The feedback from the
> testing will be provided to the community. The patch-test tree will
> periodically be git pull'ed from dpdk.
> 
> Longer term goal:
> 
> Initially, the patches will be applied along with some simple smoke
> tests. The longer term goal is to automate this process, apply more
> extensive tests and post the results in dpdk patchwork,
> http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an
> accompanying mailing list for distribution of a results summary of the
> tests.

Actually, github and services such as travis-ci and coveralls already
provide this functionality (with very little setup). So when someone
sends a pull request, the continuous integration service travis-ci will
notice it, and start a build and (possibly) run a test suite on the code
- with the patches applied.

If code coverage is collected in the process [1], it's uploaded to the
coveralls site. Both travis-ci and coveralls will add a note to the pull
request saying something like "Build failed with this patch, be careful"
or "Build OK, everything is fine" and "Coverage decreased with 5% with
this patch" etc etc.

Of course, github provides an API so it's entirely possible to add your
own continuous integration support with the same functionality as
travis-ci (customized for DPDK).



So before venturing into implementing something like this, I think the
DPDK project should at least consider the existing alternatives.

And with that, I close the can of worms again. I hope no worms were hurt
in the process! :-)

// Simon

[1] https://github.com/SimonKagstrom/kcov - yes my personal project

Re: [dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Thomas Monjalon]

2015-05-27 16:48, Thomas F Herbert:
> This is a proposal to create a patch-test dpdk tree. I welcome feedback 
> from the dpdk community on this proposal.
> 
> This tree will consist of pre-integrated patches submitted to dpdk-dev 
> mailing list. The purpose of this tree is to help reduce the effort in 
> reviewing patches and providing feedback to submitters of patches. Also 
> integrators of dpdk can benefit by having a "linux-next" like tree to 
> clone and or try new features for performance and breadth of hardware 
> support. It is the hope that the patch-test tree will provide early 
> feedback to patch reviewers as well as an assurance of quality to the 
> project maintainer.

Thanks for the proposal. This tree may be hosted here:
	http://dpdk.org/browse
You just have to give the name of the repository and a SSH key.

When it will be done, we could discuss which tests should be passed.

Re: [dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Wiles, Keith]

On 5/28/15, 1:54 AM, "Simon Kågström" <simon.kagstrom@netinsight.net>
wrote:

>I spot a can of worms to be opened here :-)
>
>On 2015-05-27 22:48, Thomas F Herbert wrote:
>> Work Flow and Process:
>> 
>> All patches will be taken from from public submissions to dpdk-dev.org
>> scraped from dpdk patchwork. Patches will be applied to the patch-test
>> tree and tested against HEAD as they are received. The feedback from the
>> testing will be provided to the community. The patch-test tree will
>> periodically be git pull'ed from dpdk.
>> 
>> Longer term goal:
>> 
>> Initially, the patches will be applied along with some simple smoke
>> tests. The longer term goal is to automate this process, apply more
>> extensive tests and post the results in dpdk patchwork,
>> http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an
>> accompanying mailing list for distribution of a results summary of the
>> tests.
>
>Actually, github and services such as travis-ci and coveralls already
>provide this functionality (with very little setup). So when someone
>sends a pull request, the continuous integration service travis-ci will
>notice it, and start a build and (possibly) run a test suite on the code
>- with the patches applied.
>
>If code coverage is collected in the process [1], it's uploaded to the
>coveralls site. Both travis-ci and coveralls will add a note to the pull
>request saying something like "Build failed with this patch, be careful"
>or "Build OK, everything is fine" and "Coverage decreased with 5% with
>this patch" etc etc.
>
>Of course, github provides an API so it's entirely possible to add your
>own continuous integration support with the same functionality as
>travis-ci (customized for DPDK).
>
>
>
>So before venturing into implementing something like this, I think the
>DPDK project should at least consider the existing alternatives.

Using the above process seems very reasonable and we already have github
account we can mirror the code and run these processes. This gives us a
more standard method instead of a home grown method, but as Simon states
we could enhance the process via the API.

I may have bruised a few worms here, so very sorry :-)
>
>And with that, I close the can of worms again. I hope no worms were hurt
>in the process! :-)
>
>// Simon
>
>[1] https://github.com/SimonKagstrom/kcov - yes my personal project

Re: [dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Thomas F Herbert]

On 5/27/15 5:22 PM, Vincent JARDIN wrote:
> On 27/05/2015 22:48, Thomas F Herbert wrote:
>> Work Flow and Process:
>>
>> All patches will be taken from from public submissions to dpdk-dev.org
>> scraped from dpdk patchwork. Patches will be applied to the patch-test
>> tree and tested against HEAD as they are received. The feedback from the
>> testing will be provided to the community. The patch-test tree will
>> periodically be git pull'ed from dpdk.
>>
>> Longer term goal:
>>
>> Initially, the patches will be applied along with some simple smoke
>> tests. The longer term goal is to automate this process, apply more
>> extensive tests and post the results in dpdk patchwork,
>> http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an
>> accompanying mailing list for distribution of a results summary of the
>> tests.
>
> thanks for helping.
>
> It could be broken into two parts:
>    - patch-test-net-next,
> http://dpdk.org/browse/dpdk/tree/MAINTAINERS#n190
> http://dpdk.org/browse/dpdk/tree/drivers/net
>
>    - patch-test-other-next,
> all, excepted drivers
Vincent, I don't understand what is in patch-test-next vs 
patch-test-other-next. Is patch-test-other-next to be for pmd drivers 
that are not included in the standard dpdk release?
>
> Best regards,
>    Vincent

Re: [dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Thomas Monjalon]

2015-05-28 10:39, Thomas F Herbert:
> On 5/28/15 5:48 AM, Thomas Monjalon wrote:
> > 2015-05-27 16:48, Thomas F Herbert:
> >> This is a proposal to create a patch-test dpdk tree. I welcome feedback
> >> from the dpdk community on this proposal.
> >>
> >> This tree will consist of pre-integrated patches submitted to dpdk-dev
> >> mailing list. The purpose of this tree is to help reduce the effort in
> >> reviewing patches and providing feedback to submitters of patches. Also
> >> integrators of dpdk can benefit by having a "linux-next" like tree to
> >> clone and or try new features for performance and breadth of hardware
> >> support. It is the hope that the patch-test tree will provide early
> >> feedback to patch reviewers as well as an assurance of quality to the
> >> project maintainer.
> >
> > Thanks for the proposal. This tree may be hosted here:
> > 	http://dpdk.org/browse
> > You just have to give the name of the repository and a SSH key.
> 
> Shall we call the repo patch-test-next?

Why not dpdk-next?
With this description? Apply and test every patches before acceptance

> I am attaching two public keys to this email

Received, thanks

Re: [dpdk-dev] DPDK: Proposal for a patch patch-test integration tree

[Thomas F Herbert]

On 5/28/15 10:44 AM, Thomas Monjalon wrote:
> 2015-05-28 10:39, Thomas F Herbert:
>> On 5/28/15 5:48 AM, Thomas Monjalon wrote:
>>> 2015-05-27 16:48, Thomas F Herbert:
>>>> This is a proposal to create a patch-test dpdk tree. I welcome feedback
>>>> from the dpdk community on this proposal.
>>>>
>>>> This tree will consist of pre-integrated patches submitted to dpdk-dev
>>>> mailing list. The purpose of this tree is to help reduce the effort in
>>>> reviewing patches and providing feedback to submitters of patches. Also
>>>> integrators of dpdk can benefit by having a "linux-next" like tree to
>>>> clone and or try new features for performance and breadth of hardware
>>>> support. It is the hope that the patch-test tree will provide early
>>>> feedback to patch reviewers as well as an assurance of quality to the
>>>> project maintainer.
>>>
>>> Thanks for the proposal. This tree may be hosted here:
>>> 	http://dpdk.org/browse
>>> You just have to give the name of the repository and a SSH key.
>>
>> Shall we call the repo patch-test-next?
>
> Why not dpdk-next?
> With this description? Apply and test every patches before acceptance
Sounds good to me.
>
>> I am attaching two public keys to this email
>
> Received, thanks
>