From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 77B7CAFD1 for ; Wed, 17 Sep 2014 00:01:50 +0200 (CEST) Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga101.jf.intel.com with ESMTP; 16 Sep 2014 15:07:01 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,536,1406617200"; d="scan'208";a="603833023" Received: from fmsmsx104.amr.corp.intel.com ([10.18.124.202]) by orsmga002.jf.intel.com with ESMTP; 16 Sep 2014 15:06:36 -0700 Received: from fmsmsx113.amr.corp.intel.com (10.18.116.7) by fmsmsx104.amr.corp.intel.com (10.18.124.202) with Microsoft SMTP Server (TLS) id 14.3.195.1; Tue, 16 Sep 2014 15:06:33 -0700 Received: from bgsmsx152.gar.corp.intel.com (10.224.48.50) by FMSMSX113.amr.corp.intel.com (10.18.116.7) with Microsoft SMTP Server (TLS) id 14.3.195.1; Tue, 16 Sep 2014 15:06:32 -0700 Received: from bgsmsx102.gar.corp.intel.com ([169.254.2.43]) by BGSMSX152.gar.corp.intel.com ([169.254.6.175]) with mapi id 14.03.0195.001; Wed, 17 Sep 2014 03:36:30 +0530 From: "Ramia, Kannan Babu" To: "Dumitrescu, Cristian" , Olivier MATZ , "Richardson, Bruce" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata Thread-Index: AQHPy11R3D9dFAO/RUGocjSLSkflH5v4JlIAgAU74YCAAESWgIAGOfWAgAB8WLA= Date: Tue, 16 Sep 2014 22:06:29 +0000 Message-ID: <682698A055A0F44AA47192B20141149711B1FFE6@BGSMSX102.gar.corp.intel.com> References: <1409759378-10113-1-git-send-email-bruce.richardson@intel.com> <1409759378-10113-8-git-send-email-bruce.richardson@intel.com> <540D9B95.3020504@6wind.com> <59AF69C657FD0841A61C55336867B5B0343EFAA3@IRSMSX103.ger.corp.intel.com> <3EB4FA525960D640B5BDFFD6A3D891262E070D42@IRSMSX108.ger.corp.intel.com> <54135F63.2090401@6wind.com> <3EB4FA525960D640B5BDFFD6A3D891262E071FE6@IRSMSX108.ger.corp.intel.com> In-Reply-To: <3EB4FA525960D640B5BDFFD6A3D891262E071FE6@IRSMSX108.ger.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.223.10.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Sep 2014 22:01:51 -0000 I completely agree with Cristian here, instead of leaving to applications w= here to place their meta data, we can provide a guidance by having this fie= ld about placement of application meta while maintaining transparency on th= e contents of application meta information.=20 Regards Kannan Babu Ramia Sr.System Architect Communication Storage Infrastructure Group DCG -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Dumitrescu, Cristian Sent: Wednesday, September 17, 2014 1:37 AM To: Olivier MATZ; Richardson, Bruce; dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the m= buf metadata Hi Olivier, I agree that your suggested approach for application-dependent metadata mak= es sense, in fact the two approaches work in exactly the same way (packet m= etadata immediately after the regular mbuf), there is only a subtle differe= nce, which is related to defining consistent DPDK usage guidelines. 1. Advertising the presence of application-dependent meta-data as supported= mechanism If we explicitly have a metadata zero-size field at the end of t= he mbuf, we basically tell people that adding their own application meta-da= ta at the end of the mandatory meta-data (mbuf structure) is a mechanism th= at DPDK allows and supports, and will continue to do so for the foreseeable= future. In other words, we guarantee that an application doing so will con= tinue to build successfully with future releases of DPDK, and we will not i= ntroduce changes in DPDK that could potentially break this mechanism. It is= also a hint to people of where to put their application dependent meta-dat= a. 2. Defining a standard base address for the application-dependent metadata - There are also libraries in DPDK that work with application dependent met= a-data, currently these are the Packet Framework libraries: librte_port, li= brte_table, librte_pipeline. Of course, the library does not have the knowl= edge of the application dependent meta-data format, so they treat it as opa= que array of bytes, with the offset and size of the array given as argument= s. In my opinion, it is safer (and more elegant) if these libraries (and ot= hers) can rely on an mbuf API to access the application dependent meta-data= (in an opaque way) rather than make an assumption about the mbuf (i.e. the= location of custom metadata relative to the mbuf) that is not clearly supp= orted/defined by the mbuf library.=20 - By having this API, we basically say: we define the custom meta-data base= address (first location where custom metadata _could_ be placed) immediate= ly after the mbuf, so libraries and apps accessing custom meta-data should = do so by using a relative offset from this base rather than each applicatio= n defining its own base: immediately after mbuf, or 128 bytes after mbuf, o= r 64 bytes before the end of the buffer, or other. More (minor) comments inline below. Thanks, Cristian -----Original Message----- From: Olivier MATZ [mailto:olivier.matz@6wind.com] Sent: Friday, September 12, 2014 10:02 PM To: Dumitrescu, Cristian; Richardson, Bruce; dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the m= buf metadata Hello Cristian, > What is the reason to remove this field? Please explain the rationale=20 > of removing this field. The rationale is explained in http://dpdk.org/ml/archives/dev/2014-September/005232.html "The format of the metadata is up to the application". The type of data the application stores after the mbuf has not to be define= d in the mbuf. These macros limits the types of metadata to uint8, uint16, = uint32, uint64? What should I do if I need a void*, a struct foo ? Should w= e add a macro for each possible type? [Cristian] Actually, this is not correct, as macros to access metadata thro= ugh pointers (to void or scalar types) are provided as well. This pointer c= an be converted by the application to the format is defines. > We previously agreed we need to provide an easy and standard mechanism=20 > for applications to extend the mandatory per buffer metadata (mbuf)=20 > with optional application-dependent metadata. Defining a structure in the application which does not pollute the rte_mbuf= structure is "easy and standard(TM)" too. [Cristian] I agree, both approaches work the same really, it is just the di= fference in advertising the presence of meta-data as supported mechanism an= d defining a standard base address for it. > This field just provides a clean way for the apps to know where is the=20 > end of the mandatory metadata, i.e. the first location in the packet=20 > buffer where the app can add its own metadata (of course, the app has=20 > to manage the headroom space before the first byte of packet data). A=20 > zero-size field is the standard mechanism that DPDK uses extensively=20 > in pretty much every library to access memory immediately after a=20 > header structure. Having the following is clean too: struct metadata { ... }; struct app_mbuf { struct rte_mbuf mbuf; struct metadata metadata; }; There is no need to define anything in the mbuf structure. [Cristian] I agree, both approaches work the same really, it is just the di= fference in advertising the presence of meta-data as supported mechanism an= d defining a standard base address for it. >=20 > The impact of removing this field is that there is no standard way to=20 > identify where the end of the mandatory metadata is, so each=20 > application will have to reinvent this. With no clear convention, we=20 > will end up with a lot of non-standard ways. Every time the format of=20 > the mbuf structure is going to be changed, this can potentially break=20 > applications that use custom metadata, while using this simple=20 > standard mechanism would prevent this. So why remove this? Waow. Five occurences of "standard" until now.=20 [Cristian] I am sorry :) Could you give a reference to the standard you're refering to? :) [Cristian] See the IEFT Service Function Chaining link below, the environme= nt is different (data center pipeline vs. CPU core-level pipeline), but the= concepts are very similar. Our application defines private metadata in mbufs in the way described abov= e, we never changed that since we're supporting the dpdk. So I don't unders= tand when you say that each time mbuf is reformatted it breaks the applicat= ion. > Having applications define their optional meta-data is a real need. Sure. This patch does not prevent this at all. You can continue to do exact= ly the same, but in the concerned application, not in the generic mbuf stru= cture. > Please take a look at the Service Chaining IEFT emerging protocols=20 > (https://datatracker.ietf.org/wg/sfc/documents/), which provide=20 > standard mechanisms for applications to define their own Six :) I'm not sure these documents define the way to extend a packet structure wi= th metadata in a C program. Again, Bruce's patch does not prevent to do wha= t you need, it just moves it at the proper place. > packet meta-data and share it between the elements of the processing=20 > pipeline (for Service Chaining, these are typically virtual machines=20 > scattered amongst the data center). >=20 > And, in my opinion, there is no negative impact/cost associated with=20 > keeping this field. To summarize what I think: - this patch does not prevent to do what you want to do - removing the macros help to have a shorter and more comprehensible mbuf structure - the previous approach does not scale because it would require a macro per type > -------------------------------------------------------------- > Intel Shannon Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County=20 > Kildare Registered Number: 308263 Business address: Dromore House,=20 > East Park, Shannon, Co. Clare >=20 > This e-mail and any attachments may contain confidential material for the= sole use of the intended recipient(s). Any review or distribution by other= s is strictly prohibited. If you are not the intended recipient, please con= tact the sender and delete all copies. This is a public mailing list, this disclaimer is invalid. Regards, Olivier -------------------------------------------------------------- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Reg= istered Number: 308263 Business address: Dromore House, East Park, Shannon,= Co. Clare This e-mail and any attachments may contain confidential material for the s= ole use of the intended recipient(s). Any review or distribution by others = is strictly prohibited. If you are not the intended recipient, please conta= ct the sender and delete all copies.