From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mo2.mail-out.ovh.net (7.mo2.mail-out.ovh.net [188.165.48.182]) by dpdk.org (Postfix) with ESMTP id 123BF156 for ; Wed, 4 Dec 2013 14:38:08 +0100 (CET) Received: from mail422.ha.ovh.net (gw6.ovh.net [213.251.189.206]) by mo2.mail-out.ovh.net (Postfix) with SMTP id C7698FF8B89 for ; Wed, 4 Dec 2013 14:39:05 +0100 (CET) Received: from b0.ovh.net (HELO queueout) (213.186.33.50) by b0.ovh.net with SMTP; 4 Dec 2013 15:39:13 +0200 Received: from lneuilly-152-23-9-75.w193-252.abo.wanadoo.fr (HELO pcdeff) (ff@ozog.com@193.252.40.75) by ns0.ovh.net with SMTP; 4 Dec 2013 15:39:11 +0200 From: =?iso-8859-1?Q?Fran=E7ois-Fr=E9d=E9ric_Ozog?= To: "'Jason Vassbender'" References: <035f01cef0ca$a0d7a470$e286ed50$@com> In-Reply-To: Date: Wed, 4 Dec 2013 14:37:28 +0100 Message-ID: <03b101cef0f5$faa2cc70$efe86550$@com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Ac7w0ra9BGbw8/PSTVGyZ7DoabzcdwAIcixQ Content-Language: fr X-Ovh-Tracer-Id: 3494230361967876313 X-Ovh-Remote: 193.252.40.75 (lneuilly-152-23-9-75.w193-252.abo.wanadoo.fr) X-Ovh-Local: 213.186.33.20 (ns0.ovh.net) X-OVH-SPAMSTATE: OK X-OVH-SPAMSCORE: -100 X-OVH-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeeiledrkeehucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd X-Spam-Check: DONE|U 0.5/N X-VR-SPAMSTATE: OK X-VR-SPAMSCORE: -100 X-VR-SPAMCAUSE: gggruggvucftvghtrhhoucdtuddrfeeiledrkeehucetufdoteggodetrfcurfhrohhfihhlvgemucfqggfjnecuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Decoupling DPDK from EAL X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Dec 2013 13:38:08 -0000 Hi, I think I get the picture. DPDK is not really flexible at memory = allocation (nor the Linux kernel which requires boot parameters for 1GB huge = pages)... Let's assume that "static" memory configuration is acceptable.=20 Is the thread model integration issue related to the fact we set = affinity ATFER thread creation? (I agree this is complex, but you can still = create an adaptation layer in your thread API to accommodate DPDK thread model). Fran=E7ois-Fr=E9d=E9ric > -----Message d'origine----- > De=A0: Jason Vassbender [mailto:jason.vassbender@gmail.com] > Envoy=E9=A0: mercredi 4 d=E9cembre 2013 10:25 > =C0=A0: Fran=E7ois-Fr=E9d=E9ric Ozog > Cc=A0: dev@dpdk.org > Objet=A0: Re: [dpdk-dev] Decoupling DPDK from EAL >=20 > Hey, >=20 > I guess the main hurdle is that we already have our own multi-threaded > architecture and ways to control thread startup/shutdown, priorities = and > affinities and they are all balanced very delicately (our application = is > latency sensitive, runs on rt_preempt, boots with isolcpus, etc). In > addition, we are already using the command line to initialize some of = our > things, and part of the configuration for the application does not = even > come from the command line, but from eg. > XML configuration file over the network. So ideally what I would have > preferred is that EAL initialization could be done by other means (for > example a simple initialization function with a dictionary as to be = more > flexible) and thread creation/shutdown could be left to the = application if > it so desires, provided it meets the execution conditions expected by DPDK. >=20 > Essentially, at its current state, DPDK offers a complete solution to = your > problem including the entire surrounding framework. But for most big > applications they already have their own frameworks in place and > integrating DPDK becomes harder than it should be. So if DPDK were to = be > decoupled from EAL, made more modular, and some of the functions optionally > left to applications to provide if they already have the facilities = for > them would make integration much easier and more flexible. >=20 > -Jason >=20 > On Wed, Dec 4, 2013 at 10:27 AM, Fran=E7ois-Fr=E9d=E9ric Ozog = > wrote: > > Hi, > > > > I just completed such a consulting mission for a customer. They were > > using libpcap as the network back end and the most challenging = hurdle > > was to transform a single threaded capture architecture to a > > multi-threaded one with DPDK. The other key take away, is that DPDK > > capture helps to get only 20% of the 20 times performance boost I > > managed to achieve: most of the latency is due to "application" and other > internal communication mechanisms. > > > > So I agree that DPDK is not light, but I think most of the power of > > DPDK comes from EAL thread management and "IPC"... > > > > Having said all that, I may have missed a critical point, so, what = is > > the specific major hurdle you see in the integration? > > > > Fran=E7ois-Fr=E9d=E9ric > > > > > >> -----Message d'origine----- > >> De : dev [mailto:dev-bounces@dpdk.org] De la part de Jason = Vassbender > >> Envoy=E9 : mardi 3 d=E9cembre 2013 22:51 =C0 : dev@dpdk.org Objet : > >> [dpdk-dev] Decoupling DPDK from EAL > >> > >> Hello, > >> > >> I am trying to integrate DPDK into an existing application in order > >> to improve packet processing latency, but it is proving rather > >> difficult because of DPDK's dependency on EAL's thread management = and > >> bootstrap mechanism. Our application already has its own framework > >> for managing threads and their affinities/priorities, IPC, timers = and > >> its own bootstrap mechanism (not necessarily via command line > >> arguments), we wish to integrate DPDK as an alternative network > >> back-end, but it wants to to take over our entire way of doing = things. > >> > >> Are there any plans to decouple DPDK's core functionality away from > >> EAL so that it can be more easily integrated into existing applications? > >> > >> -Jason > >