[dpdk-dev] Notes from project governance meeting at DPDK Userspace

[dpdk-dev] Notes from project governance meeting at DPDK Userspace

[Dave Neary]

Hi everyone,

I took some notes from a discussion we had at the end of DPDK Userspace
in Dublin, concerning the growth and project structure for DPDK. If I
missed anyone's name, I apologise - there were many active contributors,
including most prominently Venky, Jim St Leger, Bruce Richardson,
Stephen Hemminger, Chris Wright, myself, Keith Wiles, Cristian
Dumitrescu, Tim O'Driscoll, Thomas Monjalon, and (until he had to leave)
Vincent Jardin. There were a few others from Intel, ARM, and others, but
I didn't get all the names during the discussion. If you see a comment
you made and would like attribution, reply - especially if it doesn't
quite match your view.

The discussion was wide ranging and we didn't quite stay on one topic
until we reached a conclusion, so some of these notes are not in strict
time order.

These are a mixture of notes and proposals for the project coming out of
the meeting - comments are welcome, all proposals are up for discussion,
and nothing has been decided on the basis of this meeting. However, all
present expressed agreement that there are issues we need to address in
the near future.

Apologies for the non-linear note taking, for those who were not there I
hope they're useful.

Thanks,
Dave.



Topic 1: Legal entity
=====================

Do we need/want a legal entity independent of a commercial vendor who
can represent the project?

Things a legal entity could do:
- Sign contracts and raise money for events
- Organise events
- Own the trademark
- Own project infrastructure like DNS, website infrastructure
- Centralised pool for marketing budget?
- Brand awareness?

There was agreement that legal governance should be lightweight, and
completely independent of technical governance. Vincent insisted on the
low cost structure for entities like 6WIND who would not be able to
justify a 6 figure membership fee.

Topic 2: Technical governance (roadmap, project scope)
======================================================

Jim: What if soeone proposes a feature that competes with a commercial
offering from an incumbent? Is it rejected, accepted, ignored?

Thomas: Issue is one of trust - is Thomas a good maintainer or not?
(language moderated ;-) If we trust the maintainer, then no problem. If
people disagree with Thomas as maintainer, then there are multiple ways
around it - gang up on Thomas & change maintainer, or fork.

Scope is a big question - is (say) a TCP stack in scope, or not? Website
says no. Thomas replies that website can be changed by a patch - patch
submission would be the start of a scope discussion, and the community
will decide. Venky: Scope narrowed compared to his initial goal -
satisfy needs of high volume servers.

How about ODP/DPDK convergence/co-operation? (nothing major noted,
beyond "the topic came up")

Do we want/need a TSC (Technical Steering Committee)? Do we need a
public roadmap?

Dave: If we have a TSC, then its membership should be from the technical
leadership of the project - users voices should be heard, and perhaps we
should have an official user advisory group, but the technical goals and
roadmap of the project should be owned by people actively developing the
project.

Topic 3: Day to day technical process
=====================================

Keith raised the issue of general throughput of patches, and the number
of unreviewed/wait state patches: need to actively scale the process.

Stephen: How about identifying reviewers, and Thomas designates lead
reviewer for each patch? Allows scaling of review, while allowing Thomas
to have last word. Alternative proposals are to have joint "maintainers"
all of whom can commit patches, subject to certain constraints (don't
commit your own code), or subsystem maintainers, establishing a
hierarchy of trust so that Thomas can just scan and accept reviews 90%
of the time.

Thomas: Need many reviewers, but there should be one committer per tree.

Stephen: Dave Miller acks minor changes, and for major changes drives
discussion - no need to wait for consensus for a small, localised
change. Activst maintainer means maintainer takes responsibility for
converging patch discussions.
Venky: Merge windows are a bad idea - batching patch merge just make it
harder to find out which patch slowed down performance - in a
performance driven project you want CI & testing to track changes across
individual patchsets.
Venky also pointed out the cost when you need to sync patches across
multiple projects, with multi-month offsets in release dates. It can
take months for a full feature to be enabled in a distro if an OpenStack
patch depends on a QEMU, OVS or DPDK patch.

3 proposals from Stephen:

1. "Clone Thomas" - multiple top-level maintainers
2. Delegate - maintainer names reviewer for each patch, and trusts review
3. Subsystem maintainers - based on trust, maintainers are on the hook
for their tree.

Chris: Do we have enough ppl with skills to be reviewers? If not, how do
we get to that point?

Venky: Performance requires a specific skill set, not everyone has the
skills, but slow patch review has an opportunity cost, and we can
minimise cost of "bad" reviews with good CI performance tests that catch
regressions

How do we maintain review velocity when maintainer is unavailable (eg.
on vacation)? How does it happen in Linux?

- Linus delegates - names "locum" maintainer while he is away. Stephen
volunteered to do it for a week or two when necessary.

Thomas: Subsystem maintainers carry review weight when maintainer is out.
Stephen: Subsystem maintainers are reviewers, need high level of trust
to be a top level maintainer.

Thomas - Subtrees can live in DPDK git tree, makes it easier to manage
merges afterwards. 2 volunteer subtree maintainers - Declan for crypto,
Kevin for (not noted). Is it an issue if only Intel people are subtree
maintainers? Venky: Yes, should have others.

Discussion summary
==================

At this point, we synthesised the conversation and tried to converge on
a firm proposal for community discussion & action:

- Legal entity:
  - Propose moving DPDK to Linux Foundation, with low overhead option
  - Minimal governance, event, marketing budget
  - Legal governance around project name, trademark

- Project leadership (roadmap, scope)
  - Venky to write a proposal for broadening scope as a patch to the website
  - Thomas proposes several smaller projects, rather than one umbrella
project as scope broadens
  - Some discussion around whether we should limit to one project per
stack? Venky: Prefers not to
  - Chris: How do decisions on new projects get made?
  - Stephen: Proposed that in the event of contention, in-person debate
and consensus decides
  - Jim proposed documenting current decision process, and improving on
it - documenting it will help make it better.

  - Proposed criteria which should be met by any technical governance model:
   1. Everyone has a voice
   2. Some voices carry more weight than others, based on technical
seniority and participation in the community
   3. Decisions should be time bound - after community debate, decision
should converge quickly one way or the other to give clarity

Further discussion on the composition and process for choosing a TSC:
=====================================================================

- Why not just have all the maintainers act as the TSC?
  - Too many, but not open enough if community voice is not heard.
Maintainers may be tightly focused on one area, and not have overall
visibility of the project.

Bruce: We should have a TSC. Elected by contributors, with some minimum
bar for voting rights & being a candidate (eg. 10 patches)

Christian: Thinks it makes sense to first define scope of what a TSC
would own, before deciding on a format and process for voting

 - Bruce: Anything that doesn't reach community consensus gets kicked to TSC
 - Cristian: Better to only have TSC decide on things which do not have
a clear decision path elsewhere (don't 2nd guess maintainer decisions)

Dave: Recommended that TSC stay small (4-5 people)

Tim will resurface his TSC proposal, revised based on this discussion,
for community review and comment.

Stable/long-term releases
=========================

We finished the discussion with a brief conversation on whether we
should maintain certain branches for a long time.

Bruce: Adds a lot of overhead
Venky: Can do point releases on some branches
Stephen: Prepared to "volunteer" someone from Brocade, because they need
to maintain an older release anyway.

Proposal: How many parallel long-term branches?
 - One per year (roughly one in 3 releases)
 - 2 years maintenance after release (3 maintained branches in parallel
- 2 long-term, plus trunk)

Thanks,
Dave.
-- 
Dave Neary - NFV/SDN Community Strategy
Open Source and Standards, Red Hat - http://community.redhat.com
Ph: +1-978-399-2182 / Cell: +1-978-799-3338

Re: [dpdk-dev] Notes from project governance meeting at DPDK Userspace

[Thomas Monjalon]

Thanks Dave for the good summary.
Some comments below.

2015-10-11 01:36, Dave Neary:
> Keith raised the issue of general throughput of patches, and the number
> of unreviewed/wait state patches: need to actively scale the process.
> 
> Stephen: How about identifying reviewers, and Thomas designates lead
> reviewer for each patch? Allows scaling of review, while allowing Thomas
> to have last word. Alternative proposals are to have joint "maintainers"
> all of whom can commit patches, subject to certain constraints (don't
> commit your own code), or subsystem maintainers, establishing a
> hierarchy of trust so that Thomas can just scan and accept reviews 90%
> of the time.
> 
> Thomas: Need many reviewers, but there should be one committer per tree.

Some maintainers are already identified in this file:
	http://dpdk.org/browse/dpdk/tree/MAINTAINERS
It is expected that the maintainers review every patches touching their
area (with exceptions for minor changes). They should also participate
in discussions to get consensus.
Reviews from non-maintainers are also very welcome.
Important features and fixes which are not properly reviewed should not
be integrated.
Someone in the room said it is the role of the patch author to get a
review. I agree.
Until now, patches from maintainers were accepted even if not reviewed.
Should it be changed?

In order to scale, some maintainers could become committers of a sub-tree.
The granularity of the sub-trees can evolve with growth of the project.
It seems a good idea to start with trees for drivers/net, drivers/crypto
and packet framework. It is not clear yet how EAL and core libraries can
benefit from any sub-tree.

> Chris: Do we have enough ppl with skills to be reviewers? If not, how do
> we get to that point?

Contributors must be motivated enough to get some time and, in some cases,
to get the green light from their management.
Some private discussions revealed that the lack of involvement is often due
to some management issues.

> Venky: Performance requires a specific skill set, not everyone has the
> skills, but slow patch review has an opportunity cost, and we can
> minimise cost of "bad" reviews with good CI performance tests that catch
> regressions

Yes having a good distributed CI will help. But it cannot replace reviews
for the quality and design considerations.

> How do we maintain review velocity when maintainer is unavailable (eg.
> on vacation)? How does it happen in Linux?
> 
> - Linus delegates - names "locum" maintainer while he is away. Stephen
> volunteered to do it for a week or two when necessary.

Thanks Stephen.

> Thomas: Subsystem maintainers carry review weight when maintainer is out.
> Stephen: Subsystem maintainers are reviewers, need high level of trust
> to be a top level maintainer.

Having a -next tree is another way of basing its work on an up-to-date tree
without waiting reviews.

> Thomas - Subtrees can live in DPDK git tree, makes it easier to manage
> merges afterwards. 2 volunteer subtree maintainers - Declan for crypto,
> Kevin for (not noted). Is it an issue if only Intel people are subtree
> maintainers? Venky: Yes, should have others.

About company sharing: it would be very nice to have more patches reviewed
by someone working for a different company than the author's one.
Considering making at least one review, when waiting for integration of its
patch, would be a good start.

>   - Some discussion around whether we should limit to one project per
> stack? Venky: Prefers not to
>   - Chris: How do decisions on new projects get made?

My understanding is that some new projects/layers can be added under the
dpdk.org umbrella. So different implementations of a DPDK layer will be
hosted in different dpdk.org git trees. But some implementations cannot
move there, e.g. ovs-dpdk.
What about https://01.org/spdk?
http://www.seastar-project.org?
https://github.com/rumpkernel/drv-netif-dpdk?
https://01.org/intel-data-plane-performance-demonstrators?

> We finished the discussion with a brief conversation on whether we
> should maintain certain branches for a long time.

1.8.0, 2.0.0, 2.1.0, the last digit can be used for stable branches.
 
> Bruce: Adds a lot of overhead
> Venky: Can do point releases on some branches
> Stephen: Prepared to "volunteer" someone from Brocade, because they need
> to maintain an older release anyway.

You just need to send me a SSH key and the version to maintain in order to
have a dedicated git tree.
Then the fixes should be preferably backported from the main tree.

Thanks everybody for the useful discussions.

[dpdk-dev] Proposals from project governance meeting at DPDK Userspace (was Notes from ...)

[Dave Neary]

Hi,

To explicitly call out the proposals and action items from the meeting:

- Legal entity proposal:
   - PROPOSAL: Chris proposes moving DPDK to Linux Foundation, with low
overhead option
   - Minimal governance, event, marketing budget
   - Legal governance around project name, trademark

- Project leadership proposal (roadmap, scope)
   - ACTION: Venky to write a proposal for broadening scope as a patch
to the website
   - PROPOSAL: Thomas proposes several smaller projects, rather than one
umbrella project as scope broadens
   - ACTION: Jim proposed documenting current decision process, and
improving on it - documenting it will help make it better.
   - ACTION: Tim proposed to resurface his TSC proposal and drive it to
agreement and action
   - Proposed criteria which should be met by any technical governance
model:
    1. Everyone has a voice
    2. Some voices carry more weight than others, based on technical
seniority and participation in the community
    3. Decisions should be time bound - after community debate, decision
should converge quickly one way or the other to give clarity

- Day to day patch review:
   - PROPOSAL: Thomas: Create hierarchical review process with
maintainers responsible for sub-trees (to be housed in DPDK Git)
   - ACTION: (without owner?) Subtrees and maintainers to be identified,
-next, crypto and (drivers, IIRC?) to be first trees
   - PROPOSAL: Thomas to identify replacement maintainers short-term
when he is unable to do it (vacation, sickness, etc)

- Stable patch maintenance
   - PROPOSAL: Maintain one release per year as a long term release,
with point releases being made regularly (based on patch volume), with
branches maintained for 2 years (2 stable branches + 1 devel branch
active at all times).

In addition to Thomas's notes, does this cover all of the conclusions
that came out of the meeting?

Thanks,
Dave.

On 10/11/2015 01:36 AM, Dave Neary wrote:
> Hi everyone,
> 
> I took some notes from a discussion we had at the end of DPDK Userspace
> in Dublin, concerning the growth and project structure for DPDK. If I
> missed anyone's name, I apologise - there were many active contributors,
> including most prominently Venky, Jim St Leger, Bruce Richardson,
> Stephen Hemminger, Chris Wright, myself, Keith Wiles, Cristian
> Dumitrescu, Tim O'Driscoll, Thomas Monjalon, and (until he had to leave)
> Vincent Jardin. There were a few others from Intel, ARM, and others, but
> I didn't get all the names during the discussion. If you see a comment
> you made and would like attribution, reply - especially if it doesn't
> quite match your view.
> 
> The discussion was wide ranging and we didn't quite stay on one topic
> until we reached a conclusion, so some of these notes are not in strict
> time order.
> 
> These are a mixture of notes and proposals for the project coming out of
> the meeting - comments are welcome, all proposals are up for discussion,
> and nothing has been decided on the basis of this meeting. However, all
> present expressed agreement that there are issues we need to address in
> the near future.
> 
> Apologies for the non-linear note taking, for those who were not there I
> hope they're useful.
> 
> Thanks,
> Dave.
> 
> 
> 
> Topic 1: Legal entity
> =====================
> 
> Do we need/want a legal entity independent of a commercial vendor who
> can represent the project?
> 
> Things a legal entity could do:
> - Sign contracts and raise money for events
> - Organise events
> - Own the trademark
> - Own project infrastructure like DNS, website infrastructure
> - Centralised pool for marketing budget?
> - Brand awareness?
> 
> There was agreement that legal governance should be lightweight, and
> completely independent of technical governance. Vincent insisted on the
> low cost structure for entities like 6WIND who would not be able to
> justify a 6 figure membership fee.
> 
> Topic 2: Technical governance (roadmap, project scope)
> ======================================================
> 
> Jim: What if soeone proposes a feature that competes with a commercial
> offering from an incumbent? Is it rejected, accepted, ignored?
> 
> Thomas: Issue is one of trust - is Thomas a good maintainer or not?
> (language moderated ;-) If we trust the maintainer, then no problem. If
> people disagree with Thomas as maintainer, then there are multiple ways
> around it - gang up on Thomas & change maintainer, or fork.
> 
> Scope is a big question - is (say) a TCP stack in scope, or not? Website
> says no. Thomas replies that website can be changed by a patch - patch
> submission would be the start of a scope discussion, and the community
> will decide. Venky: Scope narrowed compared to his initial goal -
> satisfy needs of high volume servers.
> 
> How about ODP/DPDK convergence/co-operation? (nothing major noted,
> beyond "the topic came up")
> 
> Do we want/need a TSC (Technical Steering Committee)? Do we need a
> public roadmap?
> 
> Dave: If we have a TSC, then its membership should be from the technical
> leadership of the project - users voices should be heard, and perhaps we
> should have an official user advisory group, but the technical goals and
> roadmap of the project should be owned by people actively developing the
> project.
> 
> Topic 3: Day to day technical process
> =====================================
> 
> Keith raised the issue of general throughput of patches, and the number
> of unreviewed/wait state patches: need to actively scale the process.
> 
> Stephen: How about identifying reviewers, and Thomas designates lead
> reviewer for each patch? Allows scaling of review, while allowing Thomas
> to have last word. Alternative proposals are to have joint "maintainers"
> all of whom can commit patches, subject to certain constraints (don't
> commit your own code), or subsystem maintainers, establishing a
> hierarchy of trust so that Thomas can just scan and accept reviews 90%
> of the time.
> 
> Thomas: Need many reviewers, but there should be one committer per tree.
> 
> Stephen: Dave Miller acks minor changes, and for major changes drives
> discussion - no need to wait for consensus for a small, localised
> change. Activst maintainer means maintainer takes responsibility for
> converging patch discussions.
> Venky: Merge windows are a bad idea - batching patch merge just make it
> harder to find out which patch slowed down performance - in a
> performance driven project you want CI & testing to track changes across
> individual patchsets.
> Venky also pointed out the cost when you need to sync patches across
> multiple projects, with multi-month offsets in release dates. It can
> take months for a full feature to be enabled in a distro if an OpenStack
> patch depends on a QEMU, OVS or DPDK patch.
> 
> 3 proposals from Stephen:
> 
> 1. "Clone Thomas" - multiple top-level maintainers
> 2. Delegate - maintainer names reviewer for each patch, and trusts review
> 3. Subsystem maintainers - based on trust, maintainers are on the hook
> for their tree.
> 
> Chris: Do we have enough ppl with skills to be reviewers? If not, how do
> we get to that point?
> 
> Venky: Performance requires a specific skill set, not everyone has the
> skills, but slow patch review has an opportunity cost, and we can
> minimise cost of "bad" reviews with good CI performance tests that catch
> regressions
> 
> How do we maintain review velocity when maintainer is unavailable (eg.
> on vacation)? How does it happen in Linux?
> 
> - Linus delegates - names "locum" maintainer while he is away. Stephen
> volunteered to do it for a week or two when necessary.
> 
> Thomas: Subsystem maintainers carry review weight when maintainer is out.
> Stephen: Subsystem maintainers are reviewers, need high level of trust
> to be a top level maintainer.
> 
> Thomas - Subtrees can live in DPDK git tree, makes it easier to manage
> merges afterwards. 2 volunteer subtree maintainers - Declan for crypto,
> Kevin for (not noted). Is it an issue if only Intel people are subtree
> maintainers? Venky: Yes, should have others.
> 
> Discussion summary
> ==================
> 
> At this point, we synthesised the conversation and tried to converge on
> a firm proposal for community discussion & action:
> 
> - Legal entity:
>   - Propose moving DPDK to Linux Foundation, with low overhead option
>   - Minimal governance, event, marketing budget
>   - Legal governance around project name, trademark
> 
> - Project leadership (roadmap, scope)
>   - Venky to write a proposal for broadening scope as a patch to the website
>   - Thomas proposes several smaller projects, rather than one umbrella
> project as scope broadens
>   - Some discussion around whether we should limit to one project per
> stack? Venky: Prefers not to
>   - Chris: How do decisions on new projects get made?
>   - Stephen: Proposed that in the event of contention, in-person debate
> and consensus decides
>   - Jim proposed documenting current decision process, and improving on
> it - documenting it will help make it better.
> 
>   - Proposed criteria which should be met by any technical governance model:
>    1. Everyone has a voice
>    2. Some voices carry more weight than others, based on technical
> seniority and participation in the community
>    3. Decisions should be time bound - after community debate, decision
> should converge quickly one way or the other to give clarity
> 
> Further discussion on the composition and process for choosing a TSC:
> =====================================================================
> 
> - Why not just have all the maintainers act as the TSC?
>   - Too many, but not open enough if community voice is not heard.
> Maintainers may be tightly focused on one area, and not have overall
> visibility of the project.
> 
> Bruce: We should have a TSC. Elected by contributors, with some minimum
> bar for voting rights & being a candidate (eg. 10 patches)
> 
> Christian: Thinks it makes sense to first define scope of what a TSC
> would own, before deciding on a format and process for voting
> 
>  - Bruce: Anything that doesn't reach community consensus gets kicked to TSC
>  - Cristian: Better to only have TSC decide on things which do not have
> a clear decision path elsewhere (don't 2nd guess maintainer decisions)
> 
> Dave: Recommended that TSC stay small (4-5 people)
> 
> Tim will resurface his TSC proposal, revised based on this discussion,
> for community review and comment.
> 
> Stable/long-term releases
> =========================
> 
> We finished the discussion with a brief conversation on whether we
> should maintain certain branches for a long time.
> 
> Bruce: Adds a lot of overhead
> Venky: Can do point releases on some branches
> Stephen: Prepared to "volunteer" someone from Brocade, because they need
> to maintain an older release anyway.
> 
> Proposal: How many parallel long-term branches?
>  - One per year (roughly one in 3 releases)
>  - 2 years maintenance after release (3 maintained branches in parallel
> - 2 long-term, plus trunk)
> 
> Thanks,
> Dave.
> 

-- 
Dave Neary - NFV/SDN Community Strategy
Open Source and Standards, Red Hat - http://community.redhat.com
Ph: +1-978-399-2182 / Cell: +1-978-799-3338