Re: [dpdk-dev] Why nothing since 1.8.0?

[Neil Horman <nhorman@tuxdriver.com>]

On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote:
> 2015-01-15 08:06, Neil Horman:
> > On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon wrote:
> > > 2015-01-15 04:27, Ouyang, Changchun:
> > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Zhang, Helin
> > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman
> > > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen Hemminger wrote:
> > > > > > > Ok, so 1.8.0 came out almost a month ago and none of the patches
> > > > > > > that were deferred waiting for the release got merged since then.
> > > > > > > Last commit in git is the 1.8.0 release.
> > > > > > >
> > > > > > > Where is the post-merge window bundle, where are the later commits?
> > > > > > > Lots of patches are sitting rotting in patchwork...
> > > > > >
> > > > > > +1, I've had the same questions.
> > > > > > Neil
> > > > > 
> > > > > +1, Some patch set might be ready for being merged.
> > > > 
> > > > +1,  the earlier some patches are merged into mainline, and the easier those
> > > > sequent patch sets can resolve their conflicts.
> > > 
> > > +1, there are some patches which are properly reviewed
> > > 
> > > Reminder: sub-tree to manage specific part of DPDK can be open on request
> > 
> > Ok, I think what you're saying here is you're too busy to handle all the patches
> > comming in at the moment.  As such I'd like to propose a sub-tree encompassing
> > all the pmds in DPDK.  I would envision that including all the acutal pmd's in
> > the tree, as well as the infrastructure that is used to interface them to the
> > core (i.e. the ethdev/rte_ether library).  I'll gladly maintain the patch pool
> > and send you pull requests.
> 
> The list of PMDs is increasing:
> 	librte_pmd_af_packet
> 	librte_pmd_bond
> 	librte_pmd_e1000
> 	librte_pmd_enic
> 	librte_pmd_i40e
> 	librte_pmd_ixgbe
> 	librte_pmd_pcap
> 	librte_pmd_ring
> 	librte_pmd_virtio
> 	librte_pmd_vmxnet3
> 	librte_pmd_xenvirt
> There is already some sub-trees for bnx2x, fm10k and i40e:
> 	http://dpdk.org/browse/
> 
Yes, and I've mentioned before that that is an absolutely silly way to break out
subtrees.  You have to find a balance of workload distribution and developer
convienience.

I also note that these are problematic because you're not merging anything
from them. Is it your intention to keep bnx2 and fm10k separate in perpituity?
If so, thats a real problem, because then we effectively just have several out
of tree drivers, and thats just unacceptible.

> > If you could set me up with a login to dpdk.org, I'd appreciate it.
> 
> It is preferred to have 1 sub-tree per module.
> What do you think of managing contributions for af_packet and/or virtio?
> It would make sense as virtio is a RedHat technology.
> Maybe it could include vhost lib and example.
> 
No, for reasons I've mentioned before.  If you take each pmd/library and create
a subtree for it, you've created the most fine grained control of subtrees you
could ask for, but you've created a nighmare of a burden on developers who want
to update any code, especially if they have patches that hit multiple trees.

Look at some of the stats in the dpdk tree:

Library		Commits between 1.7.0 and 1.8.0
librte_acl		5
librte_cfgfile		0
librte_cmdline		4
librte_compat		0
librte_distributor 	5
librte_eal 		125
librte_ether 		31
librte_hash 		1
librte_ip_frag 		5
librte_ivshmem 		0
librte_kni 		2
librte_kvargs 		0
librte_lpm 		1
librte_malloc 		1
librte_mbuf 		39
librte_mempool 		4
librte_meter 		0
librte_net 		4
librte_pipeline 	0
librte_pmd_af_packet 	4
librte_pmd_bond 	20
librte_pmd_e1000 	21
librte_pmd_enic 	12
librte_pmd_i40e 	90
librte_pmd_ixgbe 	83
librte_pmd_pcap 	4
librte_pmd_ring 	0
librte_pmd_virtio 	21
librte_pmd_vmxnet3 	21
librte_pmd_xenvirt 	6
librte_port 		6
librte_power 		3
librte_ring 		2
librte_sched 		1
librte_table 		7
librte_timer 		0
librte_vhost 		30

If you look at all of the pmds in the dpdk tree, we're talking about ~300
patches per release.  If you look at the net-next tree for the linux kernel,
Dave Miller merged 569 patches on his own (based on the following command:
git log --pretty=format:%H v3.17..v3.18 -- drivers/net/ethernet/ net/core/ | wc
-l)

And that doesn't account for the ~500 patches that come in via pull request from
the wireless subtree.  Nor does it account for the merge window for net-next
being 2 months instead of dpdk's 6 months.  Theres no need in any way for 12
maintainers to be twiddling their thumbs waiting on ~20 patches each, and for
that split, you've forced developers to potentially develop patches against 12
trees (12 being the current number of PMD's that are in the dpdk).

The right answer here is balance.  Let me split out the pmd's and ethernet
infrastructure libraries to a subtree.  I'll pull in patches posted regarding
pmd's and librte_ether/ip_frag etc, and send you a pull requests after each
release so you get all the latest bits, and then pulls for stabilization on each
-rc. I can manage 300 patches without issue, and that takes a load off your
shoulders.  I'll get fm10k integrated, as well as bnx2.  That gives us a single
alternate tree for developers to go to for pmd and pmd infrastructure updates.
Its a win-win.

Regards
Neil