Re: [dpdk-dev] [dpdk-techboard] Consider improving the DPDK contribution processes

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

On 25-May-20 8:26 PM, Tom Barbette wrote:
> 
> Le 25/05/2020 à 19:50, Wiles, Keith a écrit :
>>
>>> On May 25, 2020, at 12:32 PM, Thomas Monjalon <thomas@monjalon.net> 
>>> wrote:
>>>
>>> 25/05/2020 18:57, Wiles, Keith:
>>>> On May 25, 2020, at 11:28 AM, Thomas Monjalon <thomas@monjalon.net> 
>>>> wrote:
>>>>> 25/05/2020 18:09, Burakov, Anatoly:
>>>>>> On 25-May-20 5:04 PM, Maxime Coquelin wrote:
>>>>>>> On 5/25/20 5:59 PM, Burakov, Anatoly wrote:
>>>>>>>> On 25-May-20 4:52 PM, Maxime Coquelin wrote:
>>>>>>>>> On 5/25/20 5:35 PM, Jerin Jacob wrote:
>>>>>>>>>> On May 25, 2020 Thomas Monjalon wrote:
>>>>>>>>>>> My concern about clarity is the history of the discussion.
>>>>>>>>>>> When we post a new versions in GitHub, it's very hard to keep 
>>>>>>>>>>> track
>>>>>>>>>>> of the history.
>>>>>>>>>>> As a maintainer, I need to see the history to understand what 
>>>>>>>>>>> happened,
>>>>>>>>>>> what we are waiting for, and what should be merged.
>>>>>>>>>> IMO, The complete history is available per pull request URL.
>>>>>>>>>> I think, Github also email notification mechanism those to 
>>>>>>>>>> prefer to see
>>>>>>>>>> comments in the email too.
>>>>>>>>>>
>>>>>>>>>> In addition to that, Bugzilla, patchwork, CI stuff all 
>>>>>>>>>> integrated into
>>>>>>>>>> one place.
>>>>>>>>>> I am quite impressed with vscode community collaboration.
>>>>>>>>>> https://github.com/Microsoft/vscode/pulls
>>>>>>>>> Out of curiosity, just checked the git history and I'm not that
>>>>>>>>> impressed. For example last commit on the master branch:
>>>>>>>>>
>>>>>>>>> https://github.com/microsoft/vscode/commit/2a4cecf3f2f72346d06990feeb7446b3915d6148 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Commit title: " Fix #98530 "
>>>>>>>>> Commit message empty, no explanation on what the patch is doing.
>>>>>>>>>
>>>>>>>>> Then, let's check the the issue it is pointed to:
>>>>>>>>> https://github.com/microsoft/vscode/issues/98530
>>>>>>>>>
>>>>>>>>> Issue is created 15 minutes before the patch is being merged. 
>>>>>>>>> All that
>>>>>>>>> done by the same contributor, without any review.
>>>>>>>>>
>>>>>>>> Just because they do it wrong doesn't mean we can't do it right 
>>>>>>>> :) This
>>>>>>>> says more about Microsoft's lack of process around VSCode than 
>>>>>>>> it does
>>>>>>>> about Github the tool.
>>>>>>>>
>>>>>>> True. I was just pointing out that is not the kind of process I 
>>>>>>> would
>>>>>>> personally want to adopt.
>>>>>>>
>>>>>> You won't find disagreement here, but this "process" is not due to 
>>>>>> the
>>>>>> tool. You can just as well allow Thomas to merge stuff without any
>>>>>> review because he has commit rights, no Github needed - and you 
>>>>>> would be
>>>>>> faced with the same problem.
>>>>>>
>>>>>> So, i don't think Jerin was suggesting that we degrade our 
>>>>>> merge/commit
>>>>>> rules. Rather, the point was that (whatever you think of VSCode's
>>>>>> review/merge process) there are a lot of pull requests and there is
>>>>>> healthy community collaboration. I'm not saying we don't have that,
>>>>> Yes, recent survey said the process was fine:
>>>>>     http://mails.dpdk.org/archives/announce/2019-June/000268.html
>>>> IMO the survey is not a great tool for these types of things. The 
>>>> tech board and others that fully understand the process should 
>>>> decide. From my experience using Github or Gitlab is much easy and a 
>>>> single tool to submit patches to a project. Anatoly and others 
>>>> stated it very well and we should convert IMO, as I have always 
>>>> stated in the past.
>>>>>
>>>>>> obviously, but i have a suspicion that we'll get more of it if we 
>>>>>> lower
>>>>>> the barrier for entry (not the barrier for merge!). I think there 
>>>>>> is a
>>>>>> way to lower the secondary skill level needed to contribute to DPDK
>>>>>> without lowering coding/merge standards with it.
>>>>> About the barrier for entry, maybe it is not obvious because I don't
>>>>> communicate a lot about it, but please be aware that I (and other
>>>>> maintainers I think) are doing a lot of changes in newcomer patches
>>>>> to avoid asking them knowing the whole process from the beginning.
>>>>> Then frequent contributors get educated on the way.
>>>>>
>>>>> I think the only real barrier we have is to sign the patch
>>>>> with a real name and send an email to right list.
>>>>> The ask for SoB real name is probably what started this thread
>>>>> in Morten's mind. And the SoB requirement will *never* change.
>>>> Would it not free up your time and energies by have the tools
>>>> do most of the work. then you can focus on what matters the patch
>>>> and developing more features?
>>> No, GitHub is not helping to track root cause and define what should 
>>> be backported.
>>> It does not help to track Coverity issues.
>>> It does not add Acks automatically (but patchwork does).
>>> It does not send a notification when enough review is done (judgement 
>>> needed here).
>>> It does not split patches when different bugs are fixed.
>>> etc...
>> Thanks for reading my emails and I am trying to help DPDK as a whole.
>>
>> All of these seem to be supported by GitHub or GitLab in one way or 
>> another, but other more versed in these tools can correct me.
>>
>> - We use Coverity and other tools attached to GitLab and they seem to 
>> be doing the job. I agree we will always find issues and these tools 
>> are not a complete answer and no tool is today.
>> - Acks can be done via the merge rules (at least in GitLab FWIW not 
>> used GitHub much).
>> - cherry-picking a merge request into multiple commit or different 
>> merge request appear to be supported.
>> - Notifications are part of the process with merge rules if I 
>> understand your comment.
>>
>> We need to drag DPDK kicking and screaming into the year 2020 :-)
> 
> 
> Maybe we could find something that allows to "git push" to the 
> patchwork, where it kind of appears already as a github-like 
> discussion?  It doesn't miss a lot to enable writing/discussion from the 
> website directly.
> 
> Personnaly I've put a lot of efforts to fix simple comments, be sure 
> that I wrote "v2" here, sign-off there, cc-ed the right person, not mess 
> my dozen format-patch versions, changed only the cover letter, ... Quite 
> afraid of bothering that big mailing list for nothing (though It's true 
> people have gently helped). It would be much easier with a git push, a 
> fast online review of the diff, as on github/gitlab, and done. Also, 
> github allows online edits, and therefore allows "elders" to do small 
> fixes directly in the "patch". Some fixes are not worth the discussion 
> and the chain of mails. That's what I'm missing the most personnaly. 
> Doable from patchwork too I guess.

The problem is, we would then have to maintain these changes to 
patchwork :) So despite the pain of switching should we choose to do so, 
i think in the long run it's easier to switch to a solution that already 
does support all of this and is maintained by someone else.

> 
>>
>>> But yes GitHub provides a beautiful interface,
>>> and can help with reviews (even if not my taste).
>>>
>>> One more thing I experience sometimes, GitHub requires only one account
>>> for all hosted projects, so it helps leaving quick comments in projects
>>> we are not familiar with.
>>>
>>>
>>>> There is a reasons millions of developer use one of these two tools, 
>>>> instead of emailing patch around. We are a fairly small project 
>>>> compared to Linux Kernel and we are not developing code for the 
>>>> Linux kernel. Some of the process like coding standard is great, but 
>>>> the rest is just legacy IMO and not required to get the job done. 
>>>> Having tools to keep track of the minutia should free up more of 
>>>> your time for the real development.
>>>>
>>>> Yes, it will be a learning curve for some and nailing down the 
>>>> process or rules for merge requests needs to be done.
>>>>
>>>> All in all it will be a huge improvement for contributors.
> 


-- 
Thanks,
Anatoly