From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 204A75911 for ; Fri, 24 Feb 2017 15:21:19 +0100 (CET) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 24 Feb 2017 06:21:19 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.35,201,1484035200"; d="scan'208";a="1115131220" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.221.61]) by fmsmga001.fm.intel.com with SMTP; 24 Feb 2017 06:21:16 -0800 Received: by (sSMTP sendmail emulation); Fri, 24 Feb 2017 14:21:16 +0000 Date: Fri, 24 Feb 2017 14:21:16 +0000 From: Bruce Richardson To: Olivier Matz Cc: "Ananyev, Konstantin" , Jan Blunck , "dev@dpdk.org" Message-ID: <20170224142116.GN106392@bricha3-MOBL3.ger.corp.intel.com> References: <20170217151708.20bf4a49@platinum> <20170221105400.2eba4747@glumotte.dev.6wind.com> <20170221163808.GA213576@bricha3-MOBL3.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772583F11B4CC@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772583F11B633@irsmsx105.ger.corp.intel.com> <20170224150053.279e718d@platinum> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170224150053.279e718d@platinum> Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?= =?iso-8859-1?Q?opment?= Ireland Ltd. User-Agent: Mutt/1.7.2 (2016-11-26) Subject: Re: [dpdk-dev] [RFC 0/8] mbuf: structure reorganization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Feb 2017 14:21:20 -0000 On Fri, Feb 24, 2017 at 03:00:53PM +0100, Olivier Matz wrote: > Hi, > > On Tue, 21 Feb 2017 20:30:57 +0000, "Ananyev, Konstantin" > wrote: > > > -----Original Message----- > > > From: jblunck@gmail.com [mailto:jblunck@gmail.com] On Behalf Of Jan > > > Blunck Sent: Tuesday, February 21, 2017 7:18 PM > > > To: Ananyev, Konstantin > > > Cc: Richardson, Bruce ; Olivier MATZ > > > ; dev@dpdk.org Subject: Re: [dpdk-dev] [RFC > > > 0/8] mbuf: structure reorganization > > > > > > On Tue, Feb 21, 2017 at 6:26 PM, Ananyev, Konstantin > > > wrote: > > > > Hi Jan, > > > > > > > >> -----Original Message----- > > > >> From: jblunck@gmail.com [mailto:jblunck@gmail.com] On Behalf Of > > > >> Jan Blunck Sent: Tuesday, February 21, 2017 5:05 PM > > > >> To: Richardson, Bruce > > > >> Cc: Olivier MATZ ; Ananyev, Konstantin > > > >> ; dev@dpdk.org Subject: Re: > > > >> [dpdk-dev] [RFC 0/8] mbuf: structure reorganization > > > >> > > > >> On Tue, Feb 21, 2017 at 5:38 PM, Bruce Richardson > > > >> wrote: > > > >> > On Tue, Feb 21, 2017 at 05:12:12PM +0100, Jan Blunck wrote: > > > >> >> > > > >> >> Access through PMD specific function pointers should be > > > >> >> relatively fast on access. Modern architecture optimize that > > > >> >> use case well enough. > > > >> >> > > > >> > The cost of doing a function call per packet to access data > > > >> > starts to add up very, very fast. For the app, once the data > > > >> > is written to the mbuf, it should be in the L1 cache, giving > > > >> > very fast access to it in a few cycles. However, if a function > > > >> > call has to be made in order to do the read, that makes the > > > >> > read of that field many times more expensive. > > > >> > > > >> Exactly. Right now the timestamp normalization is done before > > > >> writing to each mbuf. Timestamps are usually read at most > > > >> once ... if at all. > > > > > > > > Well we don't know for sure right? > > > > Someone can argue that there are plenty of scenarios when > > > > other fields might also never be used/updated (rss, vlan, etc). > > > > > > > > So, are you suggesting to do normalization later? > > > > If so, then what would be the benefit (data still need to be in > > > > mbuf)? > > > > > > Yes, postponing normalization prevents you from doing unnecessary > > > work upfront. AFAIK not all NICs store timestamp data OOB, e.g. in > > > CQ. > > > > Yes, postponing normalization might help a bit (though I don't think > > much) in terms of calculations performed inside PMD. > > But we still need 8B inside mbuf to store the timestamp value, > > either normalized or raw one. > > So to clarify where is the disagreement: > > 1. timestamp position: > > mbufs 1-st cacheline vs 2-nd cacheline > > > In my opinion, if we have the room in the first cache line, we should > put it there. The only argument I see against is "we may find something > more important in the future, and we won't have room for it in the > first cache line". I don't feel we should penalize today's use cases for > hypothetic future use cases. > > > > > 2. timestamp normalization point > > inside PMD RX vs somewhere later as user needs it (extra > > function in dev_ops?). > > This point could be changed. My initial proposition tries to provide a > generic API for timestamp. Let me remind it here: > > a- the timestamp is in nanosecond > b- the reference is always the same for a given path: if the timestamp > is set in a PMD, all the packets for this PMD will have the same > reference, but for 2 different PMDs (or a sw lib), the reference > would not be the same. > > We may remove a-, and just have: > - the reference and the unit are always the same for a given path: if > the timestamp is set in a PMD, all the packets for this PMD will have > the same reference and unit, but for 2 different PMDs (or a sw lib), > they would not be the same. > > In both cases, we would need a conversion code (maybe in a library) if > the application wants to work with timestamps from several sources. The > second solution removes the normalization code in the PMD when not > needed, it is probably better. > > > About having the timestamp in the packet data, I don't think it is > a good solution for a generic API in DPDK. The timestamp is a metadata, > it has to go in the mbuf metadata. The packet data should not be > modified when the timestamp is enabled. > > But this would not prevent to have driver-specific features to do that. > In that case, the application will be aware that it is using this > specific driver and that it will receive a timestamp in the packet data. > > To summarize, the generic API could be: > - an ethdev API to enable the timestamp in a PMD for received packets > - a mbuf flag "timestamp present" > - a mbuf 64b field to store the timestamp value > > Additionally, a driver-specific API can be added for a given PMD. > Example: > - the generic timestamp ethdev is disabled (or not supported) > - a driver-specific feature "put timestamp in packet" is enabled > It would have no additional cost compared to what we have today, since > the timestamp in mbuf is not read/written. > All seems reasonable to me. /Bruce