From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 5B3706A1D for ; Tue, 16 Sep 2014 22:02:03 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP; 16 Sep 2014 13:07:30 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,535,1406617200"; d="scan'208";a="600573034" Received: from irsmsx104.ger.corp.intel.com ([163.33.3.159]) by fmsmga002.fm.intel.com with ESMTP; 16 Sep 2014 13:07:29 -0700 Received: from irsmsx108.ger.corp.intel.com ([169.254.11.157]) by IRSMSX104.ger.corp.intel.com ([163.33.3.159]) with mapi id 14.03.0195.001; Tue, 16 Sep 2014 21:07:27 +0100 From: "Dumitrescu, Cristian" To: Olivier MATZ , "Richardson, Bruce" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata Thread-Index: AQHPzszq5sDkvNeJ6UiyApy8tXytDZwEKlCA Date: Tue, 16 Sep 2014 20:07:27 +0000 Message-ID: <3EB4FA525960D640B5BDFFD6A3D891262E071FE6@IRSMSX108.ger.corp.intel.com> References: <1409759378-10113-1-git-send-email-bruce.richardson@intel.com> <1409759378-10113-8-git-send-email-bruce.richardson@intel.com> <540D9B95.3020504@6wind.com> <59AF69C657FD0841A61C55336867B5B0343EFAA3@IRSMSX103.ger.corp.intel.com> <3EB4FA525960D640B5BDFFD6A3D891262E070D42@IRSMSX108.ger.corp.intel.com> <54135F63.2090401@6wind.com> In-Reply-To: <54135F63.2090401@6wind.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.180] Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Sep 2014 20:02:04 -0000 Hi Olivier, I agree that your suggested approach for application-dependent metadata mak= es sense, in fact the two approaches work in exactly the same way (packet m= etadata immediately after the regular mbuf), there is only a subtle differe= nce, which is related to defining consistent DPDK usage guidelines. 1. Advertising the presence of application-dependent meta-data as supported= mechanism If we explicitly have a metadata zero-size field at the end of the mbuf, we= basically tell people that adding their own application meta-data at the e= nd of the mandatory meta-data (mbuf structure) is a mechanism that DPDK all= ows and supports, and will continue to do so for the foreseeable future. In= other words, we guarantee that an application doing so will continue to bu= ild successfully with future releases of DPDK, and we will not introduce ch= anges in DPDK that could potentially break this mechanism. It is also a hin= t to people of where to put their application dependent meta-data. 2. Defining a standard base address for the application-dependent metadata - There are also libraries in DPDK that work with application dependent met= a-data, currently these are the Packet Framework libraries: librte_port, li= brte_table, librte_pipeline. Of course, the library does not have the knowl= edge of the application dependent meta-data format, so they treat it as opa= que array of bytes, with the offset and size of the array given as argument= s. In my opinion, it is safer (and more elegant) if these libraries (and ot= hers) can rely on an mbuf API to access the application dependent meta-data= (in an opaque way) rather than make an assumption about the mbuf (i.e. the= location of custom metadata relative to the mbuf) that is not clearly supp= orted/defined by the mbuf library. = - By having this API, we basically say: we define the custom meta-data base= address (first location where custom metadata _could_ be placed) immediate= ly after the mbuf, so libraries and apps accessing custom meta-data should = do so by using a relative offset from this base rather than each applicatio= n defining its own base: immediately after mbuf, or 128 bytes after mbuf, o= r 64 bytes before the end of the buffer, or other. More (minor) comments inline below. Thanks, Cristian -----Original Message----- From: Olivier MATZ [mailto:olivier.matz@6wind.com] = Sent: Friday, September 12, 2014 10:02 PM To: Dumitrescu, Cristian; Richardson, Bruce; dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the m= buf metadata Hello Cristian, > What is the reason to remove this field? Please explain the > rationale of removing this field. The rationale is explained in http://dpdk.org/ml/archives/dev/2014-September/005232.html "The format of the metadata is up to the application". The type of data the application stores after the mbuf has not to be defined in the mbuf. These macros limits the types of metadata to uint8, uint16, uint32, uint64? What should I do if I need a void*, a struct foo ? Should we add a macro for each possible type? [Cristian] Actually, this is not correct, as macros to access metadata thro= ugh pointers (to void or scalar types) are provided as well. This pointer c= an be converted by the application to the format is defines. > We previously agreed we need to provide an easy and standard > mechanism for applications to extend the mandatory per buffer > metadata (mbuf) with optional application-dependent > metadata. Defining a structure in the application which does not pollute the rte_mbuf structure is "easy and standard(TM)" too. [Cristian] I agree, both approaches work the same really, it is just the di= fference in advertising the presence of meta-data as supported mechanism an= d defining a standard base address for it. > This field just provides a clean way for the apps to > know where is the end of the mandatory metadata, i.e. the first > location in the packet buffer where the app can add its own > metadata (of course, the app has to manage the headroom space > before the first byte of packet data). A zero-size field is the > standard mechanism that DPDK uses extensively in pretty much > every library to access memory immediately after a header > structure. Having the following is clean too: struct metadata { ... }; struct app_mbuf { struct rte_mbuf mbuf; struct metadata metadata; }; There is no need to define anything in the mbuf structure. [Cristian] I agree, both approaches work the same really, it is just the di= fference in advertising the presence of meta-data as supported mechanism an= d defining a standard base address for it. > = > The impact of removing this field is that there is no standard > way to identify where the end of the mandatory metadata is, so > each application will have to reinvent this. With no clear > convention, we will end up with a lot of non-standard ways. Every > time the format of the mbuf structure is going to be changed, > this can potentially break applications that use custom metadata, > while using this simple standard mechanism would prevent this. So > why remove this? Waow. Five occurences of "standard" until now. = [Cristian] I am sorry :) Could you give a reference to the standard you're refering to? :) [Cristian] See the IEFT Service Function Chaining link below, the environme= nt is different (data center pipeline vs. CPU core-level pipeline), but the= concepts are very similar. Our application defines private metadata in mbufs in the way described above, we never changed that since we're supporting the dpdk. So I don't understand when you say that each time mbuf is reformatted it breaks the application. > Having applications define their optional meta-data is a real > need. = Sure. This patch does not prevent this at all. You can continue to do exactly the same, but in the concerned application, not in the generic mbuf structure. > Please take a look at the Service Chaining IEFT emerging > protocols (https://datatracker.ietf.org/wg/sfc/documents/), which > provide standard mechanisms for applications to define their own Six :) I'm not sure these documents define the way to extend a packet structure with metadata in a C program. Again, Bruce's patch does not prevent to do what you need, it just moves it at the proper place. > packet meta-data and share it between the elements of the > processing pipeline (for Service Chaining, these are typically > virtual machines scattered amongst the data center). > = > And, in my opinion, there is no negative impact/cost associated > with keeping this field. To summarize what I think: - this patch does not prevent to do what you want to do - removing the macros help to have a shorter and more comprehensible mbuf structure - the previous approach does not scale because it would require a macro per type > -------------------------------------------------------------- > Intel Shannon Limited > Registered in Ireland > Registered Office: Collinstown Industrial Park, Leixlip, County Kildare > Registered Number: 308263 > Business address: Dromore House, East Park, Shannon, Co. Clare > = > This e-mail and any attachments may contain confidential material for the= sole use of the intended recipient(s). Any review or distribution by other= s is strictly prohibited. If you are not the intended recipient, please con= tact the sender and delete all copies. This is a public mailing list, this disclaimer is invalid. Regards, Olivier -------------------------------------------------------------- Intel Shannon Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare This e-mail and any attachments may contain confidential material for the s= ole use of the intended recipient(s). Any review or distribution by others = is strictly prohibited. If you are not the intended recipient, please conta= ct the sender and delete all copies.