From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <tim.o'driscoll@intel.com> Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 2C3B712A8 for <dev@dpdk.org>; Sun, 18 Jan 2015 22:48:37 +0100 (CET) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga103.fm.intel.com with ESMTP; 18 Jan 2015 13:43:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="4.97,862,1389772800"; d="scan'208";a="442048338" Received: from irsmsx106.ger.corp.intel.com ([163.33.3.31]) by FMSMGA003.fm.intel.com with ESMTP; 18 Jan 2015 13:35:24 -0800 Received: from irsmsx102.ger.corp.intel.com ([169.254.2.28]) by IRSMSX106.ger.corp.intel.com ([169.254.8.94]) with mapi id 14.03.0195.001; Sun, 18 Jan 2015 21:48:33 +0000 From: "O'driscoll, Tim" <tim.o'driscoll@intel.com> To: Neil Horman <nhorman@tuxdriver.com> Thread-Topic: [dpdk-dev] Why nothing since 1.8.0? Thread-Index: AQHQMDgkRICerg8+NU+uGsyD3xhjm5zAGjKAgAB5GwCAAANIgIAAWrAAgAA2YQCAAEhygIAAF++AgAAuIsCAAUKzgIABxIRwgAF6XACAABMFsA== Date: Sun, 18 Jan 2015 21:48:33 +0000 Message-ID: <26FA93C7ED1EAA44AB77D62FBE1D27BA54CA4FD5@IRSMSX102.ger.corp.intel.com> References: <20150114122352.63ef79eb@urahara> <1717148.0lUn7GOqmp@xps13> <20150115130616.GA22455@hmsreliant.think-freely.org> <12253538.YOLaLVcskf@xps13> <20150115185112.GC22455@hmsreliant.think-freely.org> <26FA93C7ED1EAA44AB77D62FBE1D27BA54C9978E@IRSMSX102.ger.corp.intel.com> <20150116165119.GC27496@hmsreliant.think-freely.org> <26FA93C7ED1EAA44AB77D62FBE1D27BA54CA4BCC@IRSMSX102.ger.corp.intel.com> <20150118182508.GA21891@hmsreliant.think-freely.org> In-Reply-To: <20150118182508.GA21891@hmsreliant.think-freely.org> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: "dev@dpdk.org" <dev@dpdk.org> Subject: Re: [dpdk-dev] Why nothing since 1.8.0? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> X-List-Received-Date: Sun, 18 Jan 2015 21:48:38 -0000 > -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: Sunday, January 18, 2015 6:25 PM > To: O'driscoll, Tim > Cc: Thomas Monjalon; dev@dpdk.org > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? >=20 > On Sat, Jan 17, 2015 at 07:57:04PM +0000, O'driscoll, Tim wrote: > > I'm going to risk the wrath of the open source purists amongst you by t= op- > posting. The reason is that there has been lots of email on this subject,= and I > want to summarise the key issue and clearly state our position on it. > Hoperfully nobody is offended by this! > > > Acutally, I am a bit upset by your doing this. While top posting can be = an > acceptible response form, doing so in an interleaved thread gives you the > opportunity to effectively rewrite the conversation on your own terms. > While > you might have summarized your position accurately in your mind, you've > discarded all the evidence that I presented in opposition. I don't appre= ciate > that. But we can rebuild from here, no worries. >=20 > > This thread has generated lots of debate, which is good, and there are = a > number of items that I believe everybody agrees on (having a MAINTAINERS > file, better tools etc.). However, the key issue that we do not agree on = is the > granularity of the repositories within DPDK. > > > No, thats not really the crux of the debate in my mind anymore, though th= at > is > certainly part of it, as it effects the convienience of developers to con= tribute > to the project. More to the point, the area of disagreement here is the = best, > most efficient way to integrate patches to various pieces of the dpdk tha= t > balances developer convienience, effiency and transparency (I've not > ennumerated > that last part before, as I've not thought of the right word, and that ma= y still > not be quite right, but more on it later). >=20 > > The current state is: > > - One master repository > > - A small number of sub-repositories, each with a separate > maintainer/SME, including: i40e, fm10k, bnx2x, docs, dts. These are used = to > generate pull requests to the main repo. > > > No, this is not the current state. The current state is: > - One master repository We have a repository for docs, with a separate maintainer that was used to = generate pull requests for 1.8. From our perspective that worked well. For i40e, 1.8 development was done in the main repository, and we agreed to= transition to a separate repo for 2.0. 2.0 development is currently in pro= gress. We haven't upstreamed anything for fm10k yet, but that will be done in its = own repo from the start, for the 2.0 release. > We have lots of cloned dpdk trees on dpdk.org, but there is no path from > them to > the master repository. Nothing has been comitted to any of the subtrees, > despite having been open for a few months now. The plan is to use the i40e and fm10k repos for 2.0 development, which is i= n progress. > I don't see any > documentation > indicating who the maintainer of each tree is, and so don't have the fogg= iest > idea who to contact if I want to get something merged to these trees. Th= ey > aren't subtrees, they're just clones of the master repository. A MAINTAINERS file to clarify responsibilities is a good idea. > > You're proposing the following: > > - Remove the individual PMD repositories, and replace them with a singl= e > repository containing all PMDs, plus parts of the core code that are clos= ely > linked to the PMDs, with you as the maintainer and SMEs for each PMD. > > > Not necesarily me, mind you (though yes, I've volunteered). I'm happy fo= r > someone else to take on this role if they volunteer. The point is not to= have > a > separate repo to integrate patches for any one PMD (as theres no need, an= d > it > makes development harder). I want one repository that I can use to targe= t > development against all PMDs, just like we do with net/net-next in the li= nux > community. >=20 > > As I've said before, and as Venky has also explained in private email o= n this, > we do not agree with this proposed change. We believe it would be a > backward step, and would have an adverse impact on DPDK. > > > You've asserted that several times, but not once have you provided any > supporting evidence or data to back that assertion. Conversely I've prov= ided > several bits of evidence to support my assertion that using the linux > workflow > model would work perfectly well here and handle all our needs (which > include, > but are not limited to, yours). The reason is to give as much control as possible to the people most famili= ar with the PMD code and the corresponding NIC hardware. > > The key issue here is that we've deliberately tried to give as much con= trol > as possible to those who are intimately familiar with a particular PMD an= d the > underlying hardware. In our view, that's just common sense. What you're > proposing is to take some of that control away, and give it to someone el= se > (in this case you) who isn't familiar with the details. We don't agree wi= th that > approach. > > > Yes, you have proposed that. The question is, why? People intimately > familiar > with the code should be writing and reviewing code. Why do they need the > responsibility for integration as well? Thats really the question. Answ= er > that, and perhaps we can make some process on this issue. Tell me how on= e > of > the people most familiar with a given piece of hardware and the software > that > drives it having the added responsibility of handling the trivial operati= ons of > SCM helps you, or anyone else? I'll point out again here that there are = 6 I40e > patches on the DPDK list, some as old as November 20th of last year. 4 h= ave > been ACK'd, but none have been integrated, with no additional commentary > from > the experts who are tasked with the patch management role. How is that > efficient, transparent or condusive to development? We're doing a lot of work on i40e and fm10k at the moment. In both cases, s= omebody needs to fill the maintainer role. That could be someone familiar w= ith the PMD code and NIC hardware, or it could be somebody without this dep= th of knowledge. The current situation is the first option. It's true that = it doesn't have to be done that way, but it seems sensible to give as much = control as possible to the people with the expertise in these areas. We jus= t believe that this will allow for more efficient development as the mainta= iner will have a detailed understanding of what they're applying. > > The arguments I've heard in favour of your proposal are summarised > below. Apologies for paraphrasing, and for any errors, but the email thre= ad is > too big and too convoluted to address these all separately. I've also inc= luded > an explanation for each item to say why we don't believe they're sufficie= nt to > justify your proposed change. > > > I thought we were doing just fine with the conversation. The paraphrasin= g > here > is why I was upset by your top post. You convieniently ignored all my > supporting evidence. >=20 > > 1. Your proposal is consistent with the Linux kernel, but current state= is not. > > In both cases (current state and your proposal), the workflow is exactl= y the > same as the Linux kernel. The difference we're discussing is simply the > granularity of the repositories. > > > Thats simply not true. My proposal is consistent with the linux kernel m= odel, > but what you want in no way is. The graunularity is diffferent as you no= te, > but > you're adding in this requirement whereby a developer for a given PMD > must be > the tree maintainer for a subtree solely for the purpose of maintaining t= hat > PMD. That in no way shape or form resembles the linux kernel model. If = you > wish to do that internally to Intel, thats great, and you have all the ab= ility > to do that, but to mandate it as part of the community project is anathem= a to > the way the linux workflow works, and open source projects in general. We're not mandating that at all. What we're saying is that this is the appr= oach that's in place for fm10k and i40e, and that we don't agree with your = proposal to change this. For other PMDs, the model could be different. > > DPDK is much smaller than the kernel, so the granularity is naturally g= oing > to be different. The kernel may combine drivers into a single repository = due > to its size and complexity, but that doesn't mean we need to do the same = for > DPDK. > > > Why? You're right in the most general of senses, diffferent projects can > work > differently, but being different in one way (size/complexity), doesn't > mandate > that it be different in another way (workflow). We can (and are) explori= ng > different ways to implement workflows here, but the question is one of > reason. > The kernel combines drivers into a single respository because its a natur= al > functional separation that allows Linus to divide the workload to > subordinates, > while still giving contributing developers a canonical place to go for th= eir > development target needs. Its efficient (As I calculated before in the > interleaved section below, Dave Miller integrates either from pull reques= ts or > individual posts, over 1000 patches on average every 2 month release cycl= e), > fast (even though every patch goes to the mailing list, gets reviewed, an= d > acked), and convienient for all developers. >=20 > Conversely, the I40e driver has seen 114 patches in the last 6 month peri= od > to > the DPDK. What is it about dpdk PMD's that makes a process that is as > efficient, fast and transparent as the upstream kenrel's unacceptible to = you? I still believe the workflow and process are the same in both cases. In bot= h cases there's a subtree to which a maintainer applies patches and then ge= nerates pull requests to the main repo. The difference in your proposal is = the granularity of the subtree. > > 2. Maintainer and SME are separate roles and should not be combined. > > We understand that they're separate roles. For the PMDs that we're > developing, the most efficient way for us to manage the work is to combin= e > these and have one of our SMEs act as maintainer as well. That's an inter= nal > decision that suits the structure of our development team and the current > state of the i40e and fm10k PMDs. Others are obviously free to make their > own choices for PMDs that they're developing. > > > Thats a really interesting thing to say. You made an internal decision, = but > this isn't an internal project. This is a community project. If you wan= t an > internal tree to commit patches too, thats fine, you can and should do th= at > for > whatever internal workflow suits your needs, but before they get merged t= o > the > community project (which includes the I40e driver), they need to get post= ed > to > the community list to allow any and all an opportunity to review them (ev= en > if > they choose not to, they deserve the opportunity).=20 All of our work is posted to the mailing list and available to anybody in t= he community to review. That's been the case since the 1.7 release. > It seems that, while you > haven't said it in so many words, you are looking for ways to accelerate = that > process, and potentially cutting out the community in the process. Absolutely not. > Let me ask a question here: Do you intend to post all the I40e patches th= at > you > plan to commit to the I40e tree to the DPDK list? Or do you plan to inte= grate > them to the tree using only internal review, and then send a pull request= to > the > list? If you are planning on doing the latter, that would explain your d= esires > to merge the tree maintainer and SME role, but would be a huge step > backwards > from all of the progress you've made toward making this project a true > community > project. As Stephen indicates, what you're then doing is creating severa= l > out-of-tree drivers, and that would be totally unacceptable. You're of c= ourse > welcome to do so, but I would not accept any workflow in which changes to > code > did not have patches posted to the list. As above, all of our work has been, and will continue to be, posted to the = mailing list and is open to anybody in the community to review. > > 3. A maintainer can handle a higher volume of patches than will be pres= ent > in any individual PMD. > > That's true, but it's also not relevant. Our goal is not to make the > SME/maintainer for i40e, fm10k etc. a full-time position. Our goal is to = have > an expert in this position, so that we can move quickly and still ensure = good > quality software. > > > Please re-read your above statement. I think you're contradicting yoursel= f > 1) You agree that a tree maintainer can handle a higher volume of patches > than > any one PMD presents, implying that a tree maintainer can, when properly > focused, efficiently take the feedback of SME's and integrate many patche= s > quickly. >=20 > 2) You claim (1) is irellevant because because your goal is to put an exp= ert in > the position of tree maintainer so that you can move more quickly. >=20 > If you agree that (1) allows for a fast, efficient and transparent workfl= ow, > how > can you both claim it is both irrelevant and that you need a merged SME/t= ree > maintainer role to achieve the same goal? >=20 > Question: How exactly do you believe putting an SME in the role of tree > maintainer will improve code quality and make code integration faster? I think Thomas is a good example here. Theoretically, someone in his role w= ould not need any knowledge of DPDK, since they would just be fulfilling an= SCM function. However, I believe his knowledge and understanding of DPDK h= ave been crucial in building the community and getting the project to the s= tage it's at now. The same applies for i40e and fm10k maintainers. We belie= ve that having experts in these roles is the most efficient way to make pro= gress. > > 4. We shouldn't give maintainer work to an SME as it detracts from thei= r > other tasks. > > We don't anticipate a problem with workload for the people that we have > in these positions. > You're having that problem right now, you just refuse to acknoweldge it. = Let > me > present it for you: > http://dpdk.org/dev/patchwork/project/dpdk/list/?q=3DI40e >=20 > That shows the only 6 patches that have been posted for I40e since 1.8 > released, > ranging dating back as far as November 20th. These patches have been > sittting > on the list since then unacted upon. If fact, taking a closer look, its = a bit > worse than that. The X710 patch in that list has been integrated, but a > version > different from the one in patchwork. Its probably just an oversight, but= the > fact remains, whoever is doing your subtree maintenence is focused on you= r > internal needs and is ignoring their community responsibilities. Thats a > problem. This is work in progress. We're confident that we'll have the necessary i40= e changes in place for 2.0. > > 5. There will be extra overhead for developers who want to implement > changes spanning multiple PMDs. > > That's true, but it's also something of a hypothetical argument. The pe= ople > who've spoken against your proposal on the mailing list are from Intel an= d > 6Wind, who are also the people contributing to most of these PMDs. I had = a > quick scan of the commits to see if I could spot anything from another > contributor that might fall into this category, and I couldn't (admittedl= y it > wasn't an exhaustive search, which someone else is free to do if they wan= t). > If this situation does arise, then Thomas has previously outlined how it = can be > handled. >=20 > Its not a hypothetical argument at all. We have this situation upstream = every > time someone makes a patch that crosses subsystems, and its managed by > the > maintainers co-ordinating their efforts when merge time comes along. Tha= ts > why > its so important to find the right granularity for subtrees, so that extr= a > effort is minimized. >=20 > And yes, you're right, most of the contributors currently are from 6wind = and > Intel. The goal here is to spread that participation beyond just the two= of > you. If you don't have a tree-maintainer that is actively handling patch= es on > the list, getting things integrated in a timely fashion, no one is going = to > bother participating. >=20 > As for handling it using the workaround thomas has proposed, I've made my > arguments to that effect already several times, but you've neglected to > summarize them here with your top post, so let me re-iterate: > Patch sets that cross subtrees are a challenge no matter how you slice th= em. > If > you have subtrees at a reasonable granularity, subtree maintainers can wo= rk > together to ensure that the upstream merge goes smoothly. If you force a > tree > spanning patch to be done in the parent tree you will have merge work to = do > for > each subtree that sends you a pull request. Having a few subtree > maintainers > handle that work is preferable than having to do merge fixups in the main > tree. > The main tree should whenever possible have clean merges. My comment was saying that we should give weight to the views of the people= who are currently doing the work in these areas. These people are strongly= of the opinion that it's better to continue with the current approach. > > In terms of where we go from here, I'd suggest the following: > > - Thomas has already asked us about a maintainer for older Intel NICs, > which we're looking into. We may choose to have a single repository with = a > single maintainer/SME for e1000/igb/ixgbe combined, which would limit the > overall number of repositories. > > - You could pursue a path of having a single repository for non-Intel N= ICs. > This would obviously need to be worked with those responsible (Stephen, > Sujith etc.). > > - As Thomas previously suggested, we should review this in future, poss= ibly > after 2.0. Maybe you'll be proven right and we'll all have to apologise f= or > disagreeing! >=20 > It sounds like you've already made a decision. Thats really disheartenin= g. > You've gone to alot of effort to make this project more open, and I'd lik= e to > encourage you further in that direction. But your comments above really > make > me think that you're hoping to isolate your development efforts, which is= a > step > in the wrong direction. What I'm doing is stating our position on your proposal, and giving my sugg= estions for how we move forward. There's no decision made that I'm aware of= , and we're not in a position to make a decision in isolation anyway. I suspect this is something that we will never agree on. That's fine, as we= all have different opinions, and diverse opinions make for a healthy commu= nity. In this situation, our view would be that we should continue with the= current approach and review again in future, as Thomas has suggested. > Neil >=20 >=20 > > > -----Original Message----- > > > From: Neil Horman [mailto:nhorman@tuxdriver.com] > > > Sent: Friday, January 16, 2015 4:51 PM > > > To: O'driscoll, Tim > > > Cc: Thomas Monjalon; dev@dpdk.org > > > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > > > > > On Thu, Jan 15, 2015 at 09:55:00PM +0000, O'driscoll, Tim wrote: > > > > > -----Original Message----- > > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Neil Horman > > > > > Sent: Thursday, January 15, 2015 6:51 PM > > > > > To: Thomas Monjalon > > > > > Cc: dev@dpdk.org > > > > > Subject: Re: [dpdk-dev] Why nothing since 1.8.0? > > > > > > > > > > On Thu, Jan 15, 2015 at 06:25:33PM +0100, Thomas Monjalon wrote: > > > > > > 2015-01-15 08:06, Neil Horman: > > > > > > > On Thu, Jan 15, 2015 at 10:51:38AM +0100, Thomas Monjalon > wrote: > > > > > > > > 2015-01-15 04:27, Ouyang, Changchun: > > > > > > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of > Zhang, > > > Helin > > > > > > > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of > Neil > > > > > Horman > > > > > > > > > > > On Wed, Jan 14, 2015 at 12:23:52PM -0800, Stephen > > > Hemminger > > > > > wrote: > > > > > > > > > > > > Ok, so 1.8.0 came out almost a month ago and none o= f > the > > > > > patches > > > > > > > > > > > > that were deferred waiting for the release got merg= ed > since > > > > > then. > > > > > > > > > > > > Last commit in git is the 1.8.0 release. > > > > > > > > > > > > > > > > > > > > > > > > Where is the post-merge window bundle, where are th= e > > > later > > > > > commits? > > > > > > > > > > > > Lots of patches are sitting rotting in patchwork... > > > > > > > > > > > > > > > > > > > > > > +1, I've had the same questions. > > > > > > > > > > > Neil > > > > > > > > > > > > > > > > > > > > +1, Some patch set might be ready for being merged. > > > > > > > > > > > > > > > > > > +1, the earlier some patches are merged into mainline, a= nd > the > > > easier > > > > > those > > > > > > > > > sequent patch sets can resolve their conflicts. > > > > > > > > > > > > > > > > +1, there are some patches which are properly reviewed > > > > > > > > > > > > > > > > Reminder: sub-tree to manage specific part of DPDK can be > open on > > > > > request > > > > > > > > > > > > > > Ok, I think what you're saying here is you're too busy to han= dle all > the > > > > > patches > > > > > > > comming in at the moment. As such I'd like to propose a sub-= tree > > > > > encompassing > > > > > > > all the pmds in DPDK. I would envision that including all th= e acutal > > > pmd's > > > > > in > > > > > > > the tree, as well as the infrastructure that is used to inter= face > them to > > > the > > > > > > > core (i.e. the ethdev/rte_ether library). I'll gladly mainta= in the > patch > > > pool > > > > > > > and send you pull requests. > > > > > > > > > > > > The list of PMDs is increasing: > > > > > > librte_pmd_af_packet > > > > > > librte_pmd_bond > > > > > > librte_pmd_e1000 > > > > > > librte_pmd_enic > > > > > > librte_pmd_i40e > > > > > > librte_pmd_ixgbe > > > > > > librte_pmd_pcap > > > > > > librte_pmd_ring > > > > > > librte_pmd_virtio > > > > > > librte_pmd_vmxnet3 > > > > > > librte_pmd_xenvirt > > > > > > There is already some sub-trees for bnx2x, fm10k and i40e: > > > > > > http://dpdk.org/browse/ > > > > > > > > > > > Yes, and I've mentioned before that that is an absolutely silly w= ay to > > > break > > > > > out > > > > > subtrees. You have to find a balance of workload distribution an= d > > > developer > > > > > convienience. > > > > > > > > > > I also note that these are problematic because you're not merging > > > anything > > > > > from them. Is it your intention to keep bnx2 and fm10k separate i= n > > > > > perpituity? > > > > > If so, thats a real problem, because then we effectively just hav= e > several > > > out > > > > > of tree drivers, and thats just unacceptible. > > > > > > > > > > > > If you could set me up with a login to dpdk.org, I'd apprecia= te it. > > > > > > > > > > > > It is preferred to have 1 sub-tree per module. > > > > > > What do you think of managing contributions for af_packet and/o= r > > > virtio? > > > > > > It would make sense as virtio is a RedHat technology. > > > > > > Maybe it could include vhost lib and example. > > > > > > > > > > > No, for reasons I've mentioned before. If you take each pmd/libr= ary > and > > > > > create > > > > > a subtree for it, you've created the most fine grained control of > subtrees > > > you > > > > > could ask for, but you've created a nighmare of a burden on > developers > > > who > > > > > want > > > > > to update any code, especially if they have patches that hit mult= iple > > > trees. > > > > > > > > > > Look at some of the stats in the dpdk tree: > > > > > > > > > > Library Commits between 1.7.0 and 1.8.0 > > > > > librte_acl 5 > > > > > librte_cfgfile 0 > > > > > librte_cmdline 4 > > > > > librte_compat 0 > > > > > librte_distributor 5 > > > > > librte_eal 125 > > > > > librte_ether 31 > > > > > librte_hash 1 > > > > > librte_ip_frag 5 > > > > > librte_ivshmem 0 > > > > > librte_kni 2 > > > > > librte_kvargs 0 > > > > > librte_lpm 1 > > > > > librte_malloc 1 > > > > > librte_mbuf 39 > > > > > librte_mempool 4 > > > > > librte_meter 0 > > > > > librte_net 4 > > > > > librte_pipeline 0 > > > > > librte_pmd_af_packet 4 > > > > > librte_pmd_bond 20 > > > > > librte_pmd_e1000 21 > > > > > librte_pmd_enic 12 > > > > > librte_pmd_i40e 90 > > > > > librte_pmd_ixgbe 83 > > > > > librte_pmd_pcap 4 > > > > > librte_pmd_ring 0 > > > > > librte_pmd_virtio 21 > > > > > librte_pmd_vmxnet3 21 > > > > > librte_pmd_xenvirt 6 > > > > > librte_port 6 > > > > > librte_power 3 > > > > > librte_ring 2 > > > > > librte_sched 1 > > > > > librte_table 7 > > > > > librte_timer 0 > > > > > librte_vhost 30 > > > > > > > > > > If you look at all of the pmds in the dpdk tree, we're talking ab= out > ~300 > > > > > patches per release. If you look at the net-next tree for the li= nux > kernel, > > > > > Dave Miller merged 569 patches on his own (based on the following > > > > > command: > > > > > git log --pretty=3Dformat:%H v3.17..v3.18 -- drivers/net/ethernet= / > > > net/core/ | > > > > > wc > > > > > -l) > > > > > > > > > > And that doesn't account for the ~500 patches that come in via pu= ll > > > request > > > > > from > > > > > the wireless subtree. Nor does it account for the merge window f= or > net- > > > > > next > > > > > being 2 months instead of dpdk's 6 months. Theres no need in any > way > > > for > > > > > 12 > > > > > maintainers to be twiddling their thumbs waiting on ~20 patches e= ach, > > > and > > > > > for > > > > > that split, you've forced developers to potentially develop patch= es > > > against 12 > > > > > trees (12 being the current number of PMD's that are in the dpdk)= . > > > > > > > > > > The right answer here is balance. Let me split out the pmd's and > ethernet > > > > > infrastructure libraries to a subtree. I'll pull in patches post= ed > regarding > > > > > pmd's and librte_ether/ip_frag etc, and send you a pull requests = after > > > each > > > > > release so you get all the latest bits, and then pulls for stabil= ization on > > > each > > > > > -rc. I can manage 300 patches without issue, and that takes a loa= d off > your > > > > > shoulders. I'll get fm10k integrated, as well as bnx2. That giv= es us a > single > > > > > alternate tree for developers to go to for pmd and pmd infrastruc= ture > > > > > updates. > > > > > Its a win-win. > > > > > > > > > > Regards > > > > > Neil > > > > > > > > I agree with Thomas on this. The approach we've been taking for PMD= s > for > > > newer Intel NICs is to have a separate sub-repository with a maintain= er > > > who's an expert in the area. This offloads some work from Thomas, > ensures > > > that the maintainer is very familiar with the PMD code and the > corresponding > > > hardware, > > > As I mentioned to you in a private note, you're convoluting two roles= into > a > > > single one here to the detriment of both: > > > > > > 1) A tree maintainer is a machine, who is responsible for the mechani= cm > of > > > SCM > > > and tree maintenence. They are responsible for merging patches, > making > > > sure > > > that merged patches make it to their upstream tree, ensuring conflict= s > get > > > resolved, and above all, making sure that SME's review patches before > they > > > get > > > merged > > > > > > 2) An SME (subject matter expert), is just that, someone who is > intimately > > > familiar with the hardware and software of a certain bit of a project= . > Their > > > only responsibility is to ensure that proposed changes are legitimate= and > > > safe. > > > They review patches within their purview and provide acks to the tree > > > maintainer. > > > > > > When you merge these roles, you take the person most responsible for > the > > > stability of their code, and distract them with 1000 patch management > > > operations, and from the work of developing new code for their area o= f > > > expertise. > > > > > > The separation of these roles has evolved in several communities > because of > > > exactly the above. Linus knows alot about the kernel, but he's never > opened > > > the > > > I40e datasheet, and he doesn't have to because he trusts Dave M to > ensure > > > that > > > code in his pull requests is legitimate. Dave in turn relies on some= one at > > > Intel to ack every I40e patch on the list before he merges it. He do= es > exactly > > > the same thing with every single other sub-component as listed in the > > > MAINTAINERS file. Thats how he is able to manage twice as many > patches as > > > the > > > dpdk can in half the amount of time. If its efficiency you're after,= thats > the > > > route to go. Please don't ignore all that evolutionary refinement. > > > > > > > > > >and doesn't involve too much additional work for the developers > involved > > > (as you said, there isn't a huge volume of commits for any individual > PMD). > > > Patch volume isn't where the additional effort comes in. Its in the > > > determination of what tree to develop against. > > > > > > Lets take an example. I'm doing work on the I40e card, so I have to = look > to > > > see > > > if theres a separate tree for it. I figure out there is, so I have t= o clone > > > that tree, and develop a patch against it. So far so good. > > > > > > A few days later I notice a bug in the pcap pmd, so I want to write a= patch > to > > > fix it. If we carry your model out to the point where each pmd has i= ts > own > > > subtree, I have to go find that new tree, and clone it as well (or ad= d a new > > > remote to my existing tree), pull those changes, branch from the prop= er > > > master, > > > and continue my work). > > > > > > If later I find a bug in the virtio pmd, I have to repeat the above p= rocess > > > again, cloning a new tree, or adding a new remote for that drivers tr= ee. > > > > > > It doesn't sound like alot when you're just sitting in a room talking= about > it, > > > but for the developers actually working on the code, its a significan= t > > > inconvienience, since it means that any pmd you want to touch require= s > you > > > to go > > > through this lookup process, ensuring your branching from the right > master > > > branch in the proper tree. > > > > > > Conversely, if you put all pmds in a single subtree, it doesn't matte= r what > > > pmd a person is working on, they only have to clone the pmd subtree, > and > > > they're > > > good to go. > > > > > > > > > > We have this in place for i40e now, and will be applying this to fm= 10k, > which > > > hasn't been upstreamed yet but will be in time for the 2.0 release. B= ased > on > > > our experiences with those, we may well look at extending the model t= o > > > include PMDs for other Intel NICs, and possibly other libraries, as w= ell. > > > > > > > You really haven't. You have an i40e tree, but you have dozens of I4= 0e > > > patches > > > on the list, and all of them thus far have been integrated into the m= ain > tree. > > > Ditto with the bnx2x tree and others that have been separated. Pleas= e > > > remember > > > subject matter expert !=3D repo maintainer. The roles can be, and sh= ould > be, > > > separeated. > > > > > > > As you said, there's a balance to be struck, and too many subtrees = may > > > become unmanageable. With respect to your concern about developers > > > having to potentially develop patches against multiple subtrees, this= has > > > never been raised as a concern by any of our development team. Is the= re > > > any historical data on the number of changes that would fall into thi= s > > > category so we can see if it's a real problem or not? > > > Look at the linux kernel if you want historical data on where the bal= ance > is. > > > Major subsystems has become the natural breaking point for subtree > > > maintenence. > > > Every net driver goes through Millers net or net-next tree. All scsi > updates > > > go > > > through Bottomleys scsi tree. FCoE changes used go through what used > to > > > be Rob > > > Loves FCoE tree. It just makes sense. I'm sure there isn't recorded= data > to > > > show anything more granular than that because no one ever seriously > > > considered > > > breaking out all 301 network drivers into their own subtree, because = its > > > obviously unmanageable at that scale. Even taken all as a single unit= (as > > > the kernel does for network drivers), a single maintainer can still h= andle > the > > > patch volume with the right subject matter experts doing timely revie= ws. > > > > > > > > > The point of this very long message is that maintainers !=3D repos. = Just > > > because > > > you're a subject matter expert in a given bit of hardware/pmd, in no = way > > > means > > > you need to be responsible for patch merging/maintenence. Doing so > just > > > makes > > > everything harder. > > > > > > Neil > > > >