From: Bruce Richardson <bruce.richardson@intel.com>
To: Olivier Matz <olivier.matz@6wind.com>
Cc: "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
Jan Blunck <jblunck@infradead.org>, "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [RFC 0/8] mbuf: structure reorganization
Date: Fri, 24 Feb 2017 14:21:16 +0000 [thread overview]
Message-ID: <20170224142116.GN106392@bricha3-MOBL3.ger.corp.intel.com> (raw)
In-Reply-To: <20170224150053.279e718d@platinum>
On Fri, Feb 24, 2017 at 03:00:53PM +0100, Olivier Matz wrote:
> Hi,
>
> On Tue, 21 Feb 2017 20:30:57 +0000, "Ananyev, Konstantin"
> <konstantin.ananyev@intel.com> wrote:
> > > -----Original Message-----
> > > From: jblunck@gmail.com [mailto:jblunck@gmail.com] On Behalf Of Jan
> > > Blunck Sent: Tuesday, February 21, 2017 7:18 PM
> > > To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> > > Cc: Richardson, Bruce <bruce.richardson@intel.com>; Olivier MATZ
> > > <olivier.matz@6wind.com>; dev@dpdk.org Subject: Re: [dpdk-dev] [RFC
> > > 0/8] mbuf: structure reorganization
> > >
> > > On Tue, Feb 21, 2017 at 6:26 PM, Ananyev, Konstantin
> > > <konstantin.ananyev@intel.com> wrote:
> > > > Hi Jan,
> > > >
> > > >> -----Original Message-----
> > > >> From: jblunck@gmail.com [mailto:jblunck@gmail.com] On Behalf Of
> > > >> Jan Blunck Sent: Tuesday, February 21, 2017 5:05 PM
> > > >> To: Richardson, Bruce <bruce.richardson@intel.com>
> > > >> Cc: Olivier MATZ <olivier.matz@6wind.com>; Ananyev, Konstantin
> > > >> <konstantin.ananyev@intel.com>; dev@dpdk.org Subject: Re:
> > > >> [dpdk-dev] [RFC 0/8] mbuf: structure reorganization
> > > >>
> > > >> On Tue, Feb 21, 2017 at 5:38 PM, Bruce Richardson
> > > >> <bruce.richardson@intel.com> wrote:
> > > >> > On Tue, Feb 21, 2017 at 05:12:12PM +0100, Jan Blunck wrote:
> > > >> >>
> > > >> >> Access through PMD specific function pointers should be
> > > >> >> relatively fast on access. Modern architecture optimize that
> > > >> >> use case well enough.
> > > >> >>
> > > >> > The cost of doing a function call per packet to access data
> > > >> > starts to add up very, very fast. For the app, once the data
> > > >> > is written to the mbuf, it should be in the L1 cache, giving
> > > >> > very fast access to it in a few cycles. However, if a function
> > > >> > call has to be made in order to do the read, that makes the
> > > >> > read of that field many times more expensive.
> > > >>
> > > >> Exactly. Right now the timestamp normalization is done before
> > > >> writing to each mbuf. Timestamps are usually read at most
> > > >> once ... if at all.
> > > >
> > > > Well we don't know for sure right?
> > > > Someone can argue that there are plenty of scenarios when
> > > > other fields might also never be used/updated (rss, vlan, etc).
> > > >
> > > > So, are you suggesting to do normalization later?
> > > > If so, then what would be the benefit (data still need to be in
> > > > mbuf)?
> > >
> > > Yes, postponing normalization prevents you from doing unnecessary
> > > work upfront. AFAIK not all NICs store timestamp data OOB, e.g. in
> > > CQ.
> >
> > Yes, postponing normalization might help a bit (though I don't think
> > much) in terms of calculations performed inside PMD.
> > But we still need 8B inside mbuf to store the timestamp value,
> > either normalized or raw one.
> > So to clarify where is the disagreement:
> > 1. timestamp position:
> > mbufs 1-st cacheline vs 2-nd cacheline
>
>
> In my opinion, if we have the room in the first cache line, we should
> put it there. The only argument I see against is "we may find something
> more important in the future, and we won't have room for it in the
> first cache line". I don't feel we should penalize today's use cases for
> hypothetic future use cases.
>
>
>
> > 2. timestamp normalization point
> > inside PMD RX vs somewhere later as user needs it (extra
> > function in dev_ops?).
>
> This point could be changed. My initial proposition tries to provide a
> generic API for timestamp. Let me remind it here:
>
> a- the timestamp is in nanosecond
> b- the reference is always the same for a given path: if the timestamp
> is set in a PMD, all the packets for this PMD will have the same
> reference, but for 2 different PMDs (or a sw lib), the reference
> would not be the same.
>
> We may remove a-, and just have:
> - the reference and the unit are always the same for a given path: if
> the timestamp is set in a PMD, all the packets for this PMD will have
> the same reference and unit, but for 2 different PMDs (or a sw lib),
> they would not be the same.
>
> In both cases, we would need a conversion code (maybe in a library) if
> the application wants to work with timestamps from several sources. The
> second solution removes the normalization code in the PMD when not
> needed, it is probably better.
>
>
> About having the timestamp in the packet data, I don't think it is
> a good solution for a generic API in DPDK. The timestamp is a metadata,
> it has to go in the mbuf metadata. The packet data should not be
> modified when the timestamp is enabled.
>
> But this would not prevent to have driver-specific features to do that.
> In that case, the application will be aware that it is using this
> specific driver and that it will receive a timestamp in the packet data.
>
> To summarize, the generic API could be:
> - an ethdev API to enable the timestamp in a PMD for received packets
> - a mbuf flag "timestamp present"
> - a mbuf 64b field to store the timestamp value
>
> Additionally, a driver-specific API can be added for a given PMD.
> Example:
> - the generic timestamp ethdev is disabled (or not supported)
> - a driver-specific feature "put timestamp in packet" is enabled
> It would have no additional cost compared to what we have today, since
> the timestamp in mbuf is not read/written.
>
All seems reasonable to me.
/Bruce
next prev parent reply other threads:[~2017-02-24 14:21 UTC|newest]
Thread overview: 155+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-24 15:19 Olivier Matz
2017-01-24 15:19 ` [dpdk-dev] [RFC 1/8] mbuf: make segment prefree function public Olivier Matz
2017-01-24 15:19 ` [dpdk-dev] [RFC 2/8] mbuf: make raw free " Olivier Matz
2017-01-24 15:19 ` [dpdk-dev] [RFC 3/8] mbuf: set mbuf fields while in pool Olivier Matz
2017-01-24 15:50 ` Bruce Richardson
2017-02-28 14:51 ` Olivier Matz
2017-01-24 15:19 ` [dpdk-dev] [RFC 4/8] net: don't touch mbuf next or nb segs on Rx Olivier Matz
2017-01-24 15:19 ` [dpdk-dev] [RFC 5/8] mbuf: make rearm data address naturally aligned Olivier Matz
2017-01-24 15:19 ` [dpdk-dev] [RFC 6/8] mbuf: use 2 bytes for port and nb segments Olivier Matz
2017-01-24 15:19 ` [dpdk-dev] [RFC 7/8] mbuf: move sequence number in second cache line Olivier Matz
2017-01-24 15:19 ` [dpdk-dev] [RFC 8/8] mbuf: add a timestamp field Olivier Matz
2017-01-24 15:59 ` [dpdk-dev] [RFC 0/8] mbuf: structure reorganization Bruce Richardson
2017-01-24 16:16 ` Olivier MATZ
2017-02-06 18:41 ` Ananyev, Konstantin
2017-02-09 16:20 ` Morten Brørup
2017-02-09 16:56 ` Ananyev, Konstantin
2017-02-16 13:48 ` Olivier Matz
2017-02-16 15:46 ` Bruce Richardson
2017-02-16 16:14 ` Olivier Matz
2017-02-21 14:20 ` Morten Brørup
2017-02-21 14:28 ` Bruce Richardson
2017-02-21 15:04 ` Olivier MATZ
2017-02-21 15:18 ` Bruce Richardson
2017-02-21 15:18 ` Morten Brørup
2017-02-19 19:04 ` Chilikin, Andrey
2017-02-21 9:53 ` Olivier MATZ
2017-02-16 17:26 ` Jan Blunck
2017-02-17 10:51 ` Olivier Matz
2017-02-17 12:49 ` Nélio Laranjeiro
2017-02-17 13:51 ` Jan Blunck
2017-02-18 5:48 ` Andrew Rybchenko
2017-02-17 13:38 ` Jan Blunck
2017-02-17 14:17 ` Olivier Matz
2017-02-17 18:42 ` Ananyev, Konstantin
2017-02-21 9:53 ` Olivier MATZ
2017-02-21 10:28 ` Ananyev, Konstantin
2017-02-20 9:27 ` Jan Blunck
2017-02-21 9:54 ` Olivier MATZ
2017-02-21 16:12 ` Jan Blunck
2017-02-21 16:38 ` Bruce Richardson
2017-02-21 17:04 ` Jan Blunck
2017-02-21 17:26 ` Ananyev, Konstantin
2017-02-21 19:17 ` Jan Blunck
2017-02-21 20:30 ` Ananyev, Konstantin
2017-02-21 21:51 ` Morten Brørup
2017-02-24 14:11 ` Olivier Matz
2017-02-24 14:00 ` Olivier Matz
2017-02-24 14:21 ` Bruce Richardson [this message]
2017-02-28 8:55 ` Jan Blunck
2017-02-28 9:05 ` Ananyev, Konstantin
2017-02-28 9:23 ` Olivier Matz
2017-02-28 9:33 ` Jan Blunck
2017-02-28 10:29 ` Ananyev, Konstantin
2017-02-28 10:50 ` Olivier Matz
2017-02-28 11:48 ` Ananyev, Konstantin
2017-02-28 12:28 ` Olivier Matz
2017-02-28 22:53 ` Ananyev, Konstantin
2017-03-02 16:46 ` Olivier Matz
2017-03-08 11:11 ` Ananyev, Konstantin
2017-03-20 9:00 ` Olivier Matz
2017-03-22 17:42 ` Ananyev, Konstantin
2017-03-24 8:35 ` Jerin Jacob
2017-03-24 13:35 ` Olivier Matz
2017-02-28 9:25 ` Jan Blunck
2017-02-19 23:45 ` Ananyev, Konstantin
2017-02-21 9:22 ` Morten Brørup
2017-02-21 9:54 ` Olivier MATZ
2017-03-08 9:41 ` [dpdk-dev] [PATCH 0/9] " Olivier Matz
2017-03-08 9:41 ` [dpdk-dev] [PATCH 1/9] mbuf: make segment prefree function public Olivier Matz
2017-03-08 9:41 ` [dpdk-dev] [PATCH 2/9] mbuf: make raw free " Olivier Matz
2017-03-08 9:41 ` [dpdk-dev] [PATCH 3/9] mbuf: set mbuf fields while in pool Olivier Matz
2017-03-31 11:21 ` Bruce Richardson
2017-03-31 11:51 ` Ananyev, Konstantin
2017-03-08 9:41 ` [dpdk-dev] [PATCH 4/9] drivers/net: don't touch mbuf next or nb segs on Rx Olivier Matz
2017-03-08 9:41 ` [dpdk-dev] [PATCH 5/9] mbuf: make rearm data address naturally aligned Olivier Matz
2017-03-08 9:41 ` [dpdk-dev] [PATCH 6/9] mbuf: use 2 bytes for port and nb segments Olivier Matz
2017-03-08 9:41 ` [dpdk-dev] [PATCH 7/9] mbuf: move sequence number in second cache line Olivier Matz
2017-03-08 9:42 ` [dpdk-dev] [PATCH 8/9] mbuf: add a timestamp field Olivier Matz
2017-04-04 10:29 ` [dpdk-dev] [PATCH 0/2] reduce writes to mbuf in ixgbe vRX Konstantin Ananyev
2017-04-07 15:13 ` Ferruh Yigit
2017-04-07 15:44 ` Ferruh Yigit
2017-04-09 22:56 ` Ananyev, Konstantin
2017-04-04 10:29 ` [dpdk-dev] [PATCH 1/2] net/ixgbe: eliminate mbuf write on rearm Konstantin Ananyev
2017-04-10 15:59 ` [dpdk-dev] [PATCH v2 0/2] reduce writes to mbuf in ixgbe vRX Konstantin Ananyev
2017-04-10 16:17 ` Ferruh Yigit
2017-04-10 15:59 ` [dpdk-dev] [PATCH v2 1/2] net/ixgbe: eliminate mbuf write on rearm Konstantin Ananyev
2017-04-10 15:59 ` [dpdk-dev] [PATCH v2 2/2] net/ixgbe: remove option to disable offload flags Konstantin Ananyev
2017-04-04 10:29 ` [dpdk-dev] [PATCH " Konstantin Ananyev
2017-03-08 9:42 ` [dpdk-dev] [PATCH 9/9] mbuf: reorder VLAN tci and buffer len fields Olivier Matz
2017-03-29 15:56 ` [dpdk-dev] [PATCH 0/9] mbuf: structure reorganization Olivier Matz
2017-03-29 16:03 ` Morten Brørup
2017-03-29 20:09 ` Bruce Richardson
2017-03-30 9:31 ` Bruce Richardson
2017-03-30 12:02 ` Olivier Matz
2017-03-30 12:23 ` Bruce Richardson
2017-03-30 16:45 ` Ananyev, Konstantin
2017-03-30 16:47 ` Ananyev, Konstantin
2017-03-30 18:06 ` Ananyev, Konstantin
2017-03-31 8:41 ` Olivier Matz
2017-03-31 9:58 ` Ananyev, Konstantin
2017-03-31 1:00 ` Ananyev, Konstantin
2017-03-31 7:21 ` Morten Brørup
2017-03-31 8:26 ` Olivier Matz
2017-03-31 8:41 ` Bruce Richardson
2017-03-31 8:59 ` Olivier Matz
2017-03-31 9:18 ` Ananyev, Konstantin
2017-03-31 9:36 ` Olivier Matz
2017-04-03 16:15 ` Thomas Monjalon
2017-04-04 7:58 ` Olivier MATZ
2017-04-04 8:53 ` Bruce Richardson
2017-03-31 9:23 ` Bruce Richardson
2017-03-31 11:18 ` Nélio Laranjeiro
2017-03-30 14:54 ` Andrew Rybchenko
2017-03-30 15:12 ` Jerin Jacob
2017-04-04 16:27 ` [dpdk-dev] [PATCH v2 0/8] " Olivier Matz
2017-04-04 16:28 ` [dpdk-dev] [PATCH v2 1/8] mbuf: make segment prefree function public Olivier Matz
2017-04-04 16:28 ` [dpdk-dev] [PATCH v2 2/8] mbuf: make raw free " Olivier Matz
2017-04-04 16:28 ` [dpdk-dev] [PATCH v2 3/8] mbuf: set mbuf fields while in pool Olivier Matz
2017-04-04 16:28 ` [dpdk-dev] [PATCH v2 4/8] drivers/net: don't touch mbuf next or nb segs on Rx Olivier Matz
2017-04-04 16:28 ` [dpdk-dev] [PATCH v2 5/8] mbuf: make rearm data address naturally aligned Olivier Matz
2017-04-04 16:28 ` [dpdk-dev] [PATCH v2 6/8] mbuf: use 2 bytes for port and nb segments Olivier Matz
2017-04-06 5:45 ` Yuanhan Liu
2017-04-18 13:03 ` Olivier MATZ
2017-07-04 7:54 ` Wang, Zhihong
2017-07-10 8:00 ` Olivier Matz
2017-07-10 8:15 ` Morten Brørup
2017-07-11 13:25 ` Wiles, Keith
2017-07-11 13:30 ` Morten Brørup
2017-07-11 15:05 ` Thomas Monjalon
2017-07-11 15:23 ` [dpdk-dev] [PATCH v2 6/8] mbuf: use 2 bytes for port and nbsegments Morten Brørup
2017-07-11 16:48 ` Wiles, Keith
2017-07-12 7:25 ` Morten Brørup
2017-07-12 9:02 ` Yang, Zhiyong
2017-07-12 9:50 ` [dpdk-dev] [PATCH v2 6/8] mbuf: use 2 bytes for port andnbsegments Morten Brørup
2017-07-12 15:35 ` Stephen Hemminger
2017-07-12 15:57 ` Morten Brørup
2017-07-12 16:23 ` Thomas Monjalon
2017-07-12 18:20 ` Wiles, Keith
2017-07-21 15:03 ` Bruce Richardson
2017-07-12 15:34 ` [dpdk-dev] [PATCH v2 6/8] mbuf: use 2 bytes for port and nbsegments Wiles, Keith
2017-07-11 13:34 ` [dpdk-dev] [PATCH v2 6/8] mbuf: use 2 bytes for port and nb segments Wiles, Keith
2017-07-11 13:46 ` Olivier MATZ
2017-04-04 16:28 ` [dpdk-dev] [PATCH v2 7/8] mbuf: move sequence number in second cache line Olivier Matz
2017-04-04 16:28 ` [dpdk-dev] [PATCH v2 8/8] mbuf: add a timestamp field Olivier Matz
2017-04-05 9:37 ` [dpdk-dev] [PATCH v2 0/8] mbuf: structure reorganization Thomas Monjalon
2017-04-05 9:46 ` Olivier MATZ
2017-04-05 9:48 ` Richardson, Bruce
2017-04-05 12:06 ` Ferruh Yigit
2017-04-14 13:10 ` Ferruh Yigit
2017-04-18 13:04 ` Olivier MATZ
2017-04-19 9:39 ` Thomas Monjalon
2017-04-19 12:28 ` Olivier MATZ
2017-04-19 12:56 ` Thomas Monjalon
2017-04-19 13:03 ` Ferruh Yigit
2017-04-19 13:12 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170224142116.GN106392@bricha3-MOBL3.ger.corp.intel.com \
--to=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=jblunck@infradead.org \
--cc=konstantin.ananyev@intel.com \
--cc=olivier.matz@6wind.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).