From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.mhcomputing.net (master.mhcomputing.net [74.208.46.186]) by dpdk.org (Postfix) with ESMTP id 1DBAA6942 for ; Tue, 28 Apr 2015 08:22:57 +0200 (CEST) Received: from [192.168.1.160] (99-34-229-174.lightspeed.sntcca.sbcglobal.net [99.34.229.174]) by mail.mhcomputing.net (Postfix) with ESMTPSA id 9C12280C003; Mon, 27 Apr 2015 23:21:17 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) From: Matthew Hall In-Reply-To: <20150425121030.GA26734@neilslaptop.think-freely.org> Date: Mon, 27 Apr 2015 23:22:53 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <26FA93C7ED1EAA44AB77D62FBE1D27BA54D1A917@IRSMSX102.ger.corp.intel.com> <20150424185123.GD32445@hmsreliant.think-freely.org> <20150425121030.GA26734@neilslaptop.think-freely.org> To: "dev@dpdk.org" X-Mailer: Apple Mail (2.1878.6) Subject: Re: [dpdk-dev] Beyond DPDK 2.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Apr 2015 06:22:57 -0000 On Apr 25, 2015, at 5:10 AM, Neil Horman wrote: > I'm more focused on why that level of participation is not higher Hi Neal, This mail is probably way too long, but here is what I saw about = participation, in my case I used DPDK on two projects so far: 1) proprietary project for a L4-L7 stateful replay DPI firewall = performance tester at a large network test and measurement corporation = using the old original 6WIND DPDK 1.X under NDA before it became = available to a wider channel 2) open-source self-created threat intelligence sensor project using = DPDK 1.7.X, under development in my spare time I think the things that making more DPDK contributions are ironically = more technical and social in nature than legal or bureaucratic in nature = as one would normally suspect, and as you theorized in your original = mail. Let me go through some things and see what people think. At the social level, there are not very many people in the world who are = familiar and comfortable with the LKML style embedded coding workflow = (heavy mailing list usage, sending and reviewing patches in emails, = putting specific people in To or Cc to get patches approved and ACKed in = subtrees, etc.) I happen to know some if not most of this tribally, = because I used Linux since 1997, but very few developers have any clue = about this stuff. However I never participated in the actual LKML flow = any further than tester / bug reporter, and I actually use DPDK very = deliberately, to avoid fighting with the headaches of the linux-net code = and flamewars. This is why I was proposing that we try to find a way to allow = contributions via Github or Bitbucket... their fork-and-pull model is = much simpler for outsiders to comprehend and make quick patches when = they find little bugs or issues as they integrate with the library... = given we are BSD licensed, the low barrier to entry is the ultimate way = to keep the patch velocity as high as possible, and keep the community = going. At the technical level, I see two or three difficulties: 1) A lot of the various performance enhancements one can use are kind of = "magical" or "jigsaw puzzle" and not presented in a unified way, where I = can methodically go through a checklist and enable everything my app = should use even though I have no clue what they all are. For example, 1) = let's talk about hashing... there is RSS hashing (symmetric or = asymmetric), JHASH, CRC hash, ... not sure how many different ones. 2) = Let's talk about CPU models... SSE, SSE4, SSE4.1, SSE4.2, etc. I don't = know what I have myself, much less what my users will have, much less = what I actually need or should use, without guidance from some processor = people. 3) Let's talk about PCIe bus... there is DCA, some other non-DCA = acceleration that's faster if you are on the same NUMA node as the PCIe = slot, but fails to work if you aren't... etc. A lot of the people from Intel, 6WIND, and the kernel people are just = thinking "this stuff is obvious... we've used it since 200X and it's = 2015!" That's true if you are a processor / kernel hacker... but if you = spent your whole career on packet processing or network security like a = lot of us app developers might have done, that's very orthogonal to = Intel-specific and compiler-specific and hardware-specific performance = hacks... so a lot of us have no real clue how to configure and test them = all, much less enhance them and make some patches to it. We just blindly = trust 6WIND and Intel to get it right, because as far as we can see, all = the DPDK code is pretty clean and readable, and we're pretty sure we = don't know anything better than what's already coded and put into the = repository, if we don't have some checklists to follow to enable and = test every combination, and find any more bugfixes to suggest. 2) A lot of the network adapters DPDK uses, especially when you begin = using the more crazy accelerations, are either hard to obtain or = expensive... for example from what I saw in my jobs, the 10 gigabit = boards were a minimum ~$250 USD in manufacturing quantities. The 2-port = gigabit latest-gen Intel board I got was $100 USD... I think a lot of = higher-ed students and overseas people from less developed nations might = have a hard time paying for some of this stuff to start hacking... some = kind of program to get these sort of people some sample gear might help. I also hit difficulties with the virtio-net driver... it doesn't work = with the virtio-net adapter in Virtualbox... which makes it harder for = me to use cool, convenient environments configured by the tool Vagrant, = to make very simple dev VM environments to quickly get new hackers up to = speed on my open source running just a command or two, and by extension = harder for me to show them why they should use DPDK for all their cool = new network ${GADGET}s. The same difficulty comes into place if I wanted to do some performance = patches... I don't have the money to buy the VTune profiler for my spare = time project, that you would want if you're going to get in there and = start dumping out all the CPU time and cache counters, so I could help = you guys to squash the performance bugs more quickly by testing the = behavior in a real packet processing app and not just some random = microbenchmark that doesn't do any real work on real stuff. 3) The DPDK library itself only covers Layer 2 which makes it pretty = esoteric for average Joes. This meant that I had to implement ARP, NDP, = IPv4, IPv6, UDP, and TCP myself, which was very educational but probably = not super efficient for attracting community attention. All except TCP = weren't too bad... TCP was a ton of work... I had to go to the = university library downtown in my city and get hold of Tanenbaum and = some of the other networking textbooks and write quite a lot of paper = notes, just to implement enough of the server half of TCP to process the = traffic and pull out the data stream... and as Jay Rolette can attest = since he gave me pointers... I cheated like hell to do it, totally = ignoring reordering and retransmissions... but I am proud to say it = seems to work great for how half-baked of a job I did coding it. I would = gladly trade a first-born child for a better TCP implementation = available with the DPDK. ;) But if I look at the world of coders out there... a lot of guys aren't = going to have any idea what to do with an L2-only network stack that = doesn't support the two most popular Transport protocols out of the = box... they only know about what to do with TCP and UDP, not how to redo = those themselves from the ground up. In my career I only knew a handful = of guys who would be good enough and patient enough to go pound that = stuff out... of course if I start my own company someday that's = precisely who I'd try to hire, but there aren't so many of these people = to go around. Matthew.=