Re: [dpdk-dev] [dpdk-techboard] Consider improving the DPDK contribution processes

["Wiles, Keith" <keith.wiles@intel.com>]

> On May 26, 2020, at 4:33 AM, Burakov, Anatoly <anatoly.burakov@intel.com> wrote:
> 
> On 25-May-20 8:26 PM, Tom Barbette wrote:
>> Le 25/05/2020 à 19:50, Wiles, Keith a écrit :
>>> 
>>>> On May 25, 2020, at 12:32 PM, Thomas Monjalon <thomas@monjalon.net> wrote:
>>>> 
>>>> 25/05/2020 18:57, Wiles, Keith:
>>>>> On May 25, 2020, at 11:28 AM, Thomas Monjalon <thomas@monjalon.net> wrote:
>>>>>> 25/05/2020 18:09, Burakov, Anatoly:
>>>>>>> On 25-May-20 5:04 PM, Maxime Coquelin wrote:
>>>>>>>> On 5/25/20 5:59 PM, Burakov, Anatoly wrote:
>>>>>>>>> On 25-May-20 4:52 PM, Maxime Coquelin wrote:
>>>>>>>>>> On 5/25/20 5:35 PM, Jerin Jacob wrote:
>>>>>>>>>>> On May 25, 2020 Thomas Monjalon wrote:
>>>>>>>>>>>> My concern about clarity is the history of the discussion.
>>>>>>>>>>>> When we post a new versions in GitHub, it's very hard to keep track
>>>>>>>>>>>> of the history.
>>>>>>>>>>>> As a maintainer, I need to see the history to understand what happened,
>>>>>>>>>>>> what we are waiting for, and what should be merged.
>>>>>>>>>>> IMO, The complete history is available per pull request URL.
>>>>>>>>>>> I think, Github also email notification mechanism those to prefer to see
>>>>>>>>>>> comments in the email too.
>>>>>>>>>>> 
>>>>>>>>>>> In addition to that, Bugzilla, patchwork, CI stuff all integrated into
>>>>>>>>>>> one place.
>>>>>>>>>>> I am quite impressed with vscode community collaboration.
>>>>>>>>>>> https://github.com/Microsoft/vscode/pulls
>>>>>>>>>> Out of curiosity, just checked the git history and I'm not that
>>>>>>>>>> impressed. For example last commit on the master branch:
>>>>>>>>>> 
>>>>>>>>>> https://github.com/microsoft/vscode/commit/2a4cecf3f2f72346d06990feeb7446b3915d6148 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Commit title: " Fix #98530 "
>>>>>>>>>> Commit message empty, no explanation on what the patch is doing.
>>>>>>>>>> 
>>>>>>>>>> Then, let's check the the issue it is pointed to:
>>>>>>>>>> https://github.com/microsoft/vscode/issues/98530
>>>>>>>>>> 
>>>>>>>>>> Issue is created 15 minutes before the patch is being merged. All that
>>>>>>>>>> done by the same contributor, without any review.
>>>>>>>>>> 
>>>>>>>>> Just because they do it wrong doesn't mean we can't do it right :) This
>>>>>>>>> says more about Microsoft's lack of process around VSCode than it does
>>>>>>>>> about Github the tool.
>>>>>>>>> 
>>>>>>>> True. I was just pointing out that is not the kind of process I would
>>>>>>>> personally want to adopt.
>>>>>>>> 
>>>>>>> You won't find disagreement here, but this "process" is not due to the
>>>>>>> tool. You can just as well allow Thomas to merge stuff without any
>>>>>>> review because he has commit rights, no Github needed - and you would be
>>>>>>> faced with the same problem.
>>>>>>> 
>>>>>>> So, i don't think Jerin was suggesting that we degrade our merge/commit
>>>>>>> rules. Rather, the point was that (whatever you think of VSCode's
>>>>>>> review/merge process) there are a lot of pull requests and there is
>>>>>>> healthy community collaboration. I'm not saying we don't have that,
>>>>>> Yes, recent survey said the process was fine:
>>>>>>     http://mails.dpdk.org/archives/announce/2019-June/000268.html
>>>>> IMO the survey is not a great tool for these types of things. The tech board and others that fully understand the process should decide. From my experience using Github or Gitlab is much easy and a single tool to submit patches to a project. Anatoly and others stated it very well and we should convert IMO, as I have always stated in the past.
>>>>>> 
>>>>>>> obviously, but i have a suspicion that we'll get more of it if we lower
>>>>>>> the barrier for entry (not the barrier for merge!). I think there is a
>>>>>>> way to lower the secondary skill level needed to contribute to DPDK
>>>>>>> without lowering coding/merge standards with it.
>>>>>> About the barrier for entry, maybe it is not obvious because I don't
>>>>>> communicate a lot about it, but please be aware that I (and other
>>>>>> maintainers I think) are doing a lot of changes in newcomer patches
>>>>>> to avoid asking them knowing the whole process from the beginning.
>>>>>> Then frequent contributors get educated on the way.
>>>>>> 
>>>>>> I think the only real barrier we have is to sign the patch
>>>>>> with a real name and send an email to right list.
>>>>>> The ask for SoB real name is probably what started this thread
>>>>>> in Morten's mind. And the SoB requirement will *never* change.
>>>>> Would it not free up your time and energies by have the tools
>>>>> do most of the work. then you can focus on what matters the patch
>>>>> and developing more features?
>>>> No, GitHub is not helping to track root cause and define what should be backported.
>>>> It does not help to track Coverity issues.
>>>> It does not add Acks automatically (but patchwork does).
>>>> It does not send a notification when enough review is done (judgement needed here).
>>>> It does not split patches when different bugs are fixed.
>>>> etc...
>>> Thanks for reading my emails and I am trying to help DPDK as a whole.
>>> 
>>> All of these seem to be supported by GitHub or GitLab in one way or another, but other more versed in these tools can correct me.
>>> 
>>> - We use Coverity and other tools attached to GitLab and they seem to be doing the job. I agree we will always find issues and these tools are not a complete answer and no tool is today.
>>> - Acks can be done via the merge rules (at least in GitLab FWIW not used GitHub much).
>>> - cherry-picking a merge request into multiple commit or different merge request appear to be supported.
>>> - Notifications are part of the process with merge rules if I understand your comment.
>>> 
>>> We need to drag DPDK kicking and screaming into the year 2020 :-)
>> Maybe we could find something that allows to "git push" to the patchwork, where it kind of appears already as a github-like discussion?  It doesn't miss a lot to enable writing/discussion from the website directly.
>> Personnaly I've put a lot of efforts to fix simple comments, be sure that I wrote "v2" here, sign-off there, cc-ed the right person, not mess my dozen format-patch versions, changed only the cover letter, ... Quite afraid of bothering that big mailing list for nothing (though It's true people have gently helped). It would be much easier with a git push, a fast online review of the diff, as on github/gitlab, and done. Also, github allows online edits, and therefore allows "elders" to do small fixes directly in the "patch". Some fixes are not worth the discussion and the chain of mails. That's what I'm missing the most personnaly. Doable from patchwork too I guess.
> 
> The problem is, we would then have to maintain these changes to patchwork :) So despite the pain of switching should we choose to do so, i think in the long run it's easier to switch to a solution that already does support all of this and is maintained by someone else.

+1, lets not keep patching our current process or adding yet another tool.
> 
>>> 
>>>> But yes GitHub provides a beautiful interface,
>>>> and can help with reviews (even if not my taste).
>>>> 
>>>> One more thing I experience sometimes, GitHub requires only one account
>>>> for all hosted projects, so it helps leaving quick comments in projects
>>>> we are not familiar with.
>>>> 
>>>> 
>>>>> There is a reasons millions of developer use one of these two tools, instead of emailing patch around. We are a fairly small project compared to Linux Kernel and we are not developing code for the Linux kernel. Some of the process like coding standard is great, but the rest is just legacy IMO and not required to get the job done. Having tools to keep track of the minutia should free up more of your time for the real development.
>>>>> 
>>>>> Yes, it will be a learning curve for some and nailing down the process or rules for merge requests needs to be done.
>>>>> 
>>>>> All in all it will be a huge improvement for contributors.
> 
> 
> -- 
> Thanks,
> Anatoly