DPDK patches and discussions
 help / color / mirror / Atom feed
From: Bruce Richardson <bruce.richardson@intel.com>
To: Yerden Zhumabekov <e_zhumabekov@sts.kz>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] [PATCH 11/13] mbuf: move l2_len and l3_len to second cache line
Date: Thu, 4 Sep 2014 12:55:49 +0100
Message-ID: <20140904115549.GA13208@sivswdev02.ir.intel.com> (raw)
In-Reply-To: <5408463C.8040805@sts.kz>

On Thu, Sep 04, 2014 at 05:00:12PM +0600, Yerden Zhumabekov wrote:
> I get your point. I've also read throught the code of various PMDs and
> have found no indication of setting l2_len/l3_len fields as well.
> 
> As for testing, we'd be happy to test the patchset but we are now in
> process of building our testing facilities so we are not ready to
> provide enough workload for the hardware/software. I was also wondering
> if anyone has run some test and can provide some numbers on that matter.
> 
> Personally, I don't think frag/reassemly app is a perfect example for
> evaluating 2nd cache line performance penalty. The offsets to L3 and L4
> headers need to be calculated for all TCP/IP traffic and fragmented
> traffic is not representative in this case. Maybe it would be better to
> write an app which calculates these offsets for different set of mbufs
> and provides some stats. For example, l2fwd/l3fwd + additional l2_len
> and l3_len calculation.
> 
> And I'm also figuring out how to rewrite our app/libs (prefetch etc) to
> reflect the future changes in mbuf, hence my concerns :)
>
Just a final point on this. Note that the second cache line is always being
read by the TX leg of the code to free back mbufs to their mbuf pool post-
transmit. The overall fast-path RX+TX benchmarks show no performance
degradation due to that access.

For sample apps, you make a good point indeed about the existing app not being
very useful as they work on larger packets. I'll see what I can throw together
here to make a more realistic test.

/Bruce
 
> 
> 04.09.2014 16:27, Bruce Richardson ??????????:
> > Hi Yerden,
> >
> > I understand your concerns and it's good to have this discussion.
> >
> > There are a number of reasons why I've moved these particular fields
> > to the second cache line. Firstly, the main reason is that, obviously enough,
> > not all fields will fit in cache line 0, and we need to prioritize what does
> > get stored there. The guiding principle behind what fields get moved or not
> > that I've chosen to use for this patch set is to move fields that are not
> > used on the receive path (or the fastpath receive path, more specifically -
> > so that we can move fields only used by jumbo frames that span mbufs) to the
> > second cache line. From a search through the existing codebase, there are no
> > drivers which set the l2/l3 length fields on RX, this is only used in
> > reassembly libraries/apps and by the drivers on TX.
> >
> > The other reason for moving it to the second cache line is that it logically
> > belongs with all the other length fields that we need to add to enable
> > tunneling support. [To get an idea of the extra fields that I propose adding
> > to the mbuf, please see the RFC patchset I sent out previously as "[RFC 
> > PATCH 00/14] Extend the mbuf structure"]. While we probably can fit the 16-bits
> > needed for l2/l3 length on the mbuf line 0, there is not enough room for all
> > the lengths so we would end up splitting them with other fields in between.
> >
> > So, in terms of what do to about this particular issue. I would hope that for
> > applications that use these fields the impact should be small and/or possible
> > to work around e.g. maybe prefetch second cache line on RX in driver. If not,
> > then I'm happy to see about withdrawing this particular change and seeing if
> > we can keep l2/l3 lengths on cache line zero, with other length fields being
> > on cache line 1.
> >
> > Question: would you consider the ip fragmentation and reassembly example apps
> > in the Intel DPDK releases good examples to test to see the impacts of this
> > change, or is there some other test you would prefer that I look to do? 
> > Can you perhaps test out the patch sets for the mbuf that I've upstreamed so
> > far and let me know what regressions, if any, you see in your use-case
> > scenarios?
> >
> > Regards,
> > /Bruce
> >
> -- 
> Sincerely,
> 
> Yerden Zhumabekov
> STS, ACI
> Astana, KZ
> 

  reply	other threads:[~2014-09-04 11:51 UTC|newest]

Thread overview: 62+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-09-03 15:49 [dpdk-dev] [PATCH 00/13] Mbuf Structure Rework, part 2 Bruce Richardson
2014-09-03 15:49 ` [dpdk-dev] [PATCH 01/13] mbuf: replace data pointer by an offset Bruce Richardson
2014-09-08  9:52   ` Olivier MATZ
2014-09-08  9:55     ` Olivier MATZ
2014-09-03 15:49 ` [dpdk-dev] [PATCH 02/13] mbuf: reorder fields by time of use Bruce Richardson
2014-09-08 10:17   ` Olivier MATZ
2014-09-03 15:49 ` [dpdk-dev] [PATCH 03/13] mbuf: add packet_type field Bruce Richardson
2014-09-08 10:17   ` Olivier MATZ
2014-09-08 10:33     ` Yerden Zhumabekov
2014-09-08 11:17       ` Olivier MATZ
2014-09-09  3:59         ` Zhang, Helin
     [not found]           ` <540EB428.9060706@6wind.com>
2014-09-09  8:45             ` Zhang, Helin
2014-09-09  9:47             ` Richardson, Bruce
2014-09-09 15:05         ` Jim Thompson
2014-09-03 15:49 ` [dpdk-dev] [PATCH 04/13] mbuf: expand ol_flags field to 64-bits Bruce Richardson
2014-09-08 10:25   ` Olivier MATZ
2014-09-09  9:00     ` Richardson, Bruce
2014-09-03 15:49 ` [dpdk-dev] [PATCH 05/13] mbuf: introduce a flag to indicate a control mbuf Bruce Richardson
2014-09-08 11:53   ` Olivier MATZ
2014-09-03 15:49 ` [dpdk-dev] [PATCH 06/13] mbuf: minor changes for readability Bruce Richardson
2014-09-08 12:03   ` Olivier MATZ
2014-09-03 15:49 ` [dpdk-dev] [PATCH 07/13] mbuf: use macros only to access the mbuf metadata Bruce Richardson
2014-09-08 12:05   ` Olivier MATZ
2014-09-09  9:01     ` Richardson, Bruce
2014-09-12 16:56       ` Dumitrescu, Cristian
2014-09-12 21:02         ` Olivier MATZ
2014-09-16 20:07           ` Dumitrescu, Cristian
2014-09-16 22:06             ` Ramia, Kannan Babu
2014-09-17 10:31               ` Richardson, Bruce
2014-09-17 14:01                 ` Thomas Monjalon
2014-09-10 15:09     ` Bruce Richardson
2014-09-10 15:31       ` Olivier MATZ
2014-09-03 15:49 ` [dpdk-dev] [PATCH 08/13] mbuf: add named points inside the mbuf structure Bruce Richardson
2014-09-08 12:08   ` Olivier MATZ
2014-09-03 15:49 ` [dpdk-dev] [PATCH 09/13] ixgbe: rework vector pmd following mbuf changes Bruce Richardson
2014-09-03 15:49 ` [dpdk-dev] [PATCH 10/13] mbuf: split mbuf across two cache lines Bruce Richardson
2014-09-08 12:10   ` Olivier MATZ
2014-09-03 15:49 ` [dpdk-dev] [PATCH 11/13] mbuf: move l2_len and l3_len to second cache line Bruce Richardson
2014-09-04  5:08   ` Yerden Zhumabekov
2014-09-04 10:27     ` Bruce Richardson
2014-09-04 11:00       ` Yerden Zhumabekov
2014-09-04 11:55         ` Bruce Richardson [this message]
2014-09-03 15:49 ` [dpdk-dev] [PATCH 12/13] ixgbe: Fix perf regression due to moved pool ptr Bruce Richardson
2014-09-03 15:49 ` [dpdk-dev] [PATCH 13/13] ixgbe: Improve slow-path perf: vector scattered RX Bruce Richardson
2014-09-11 13:15 ` [dpdk-dev] [PATCH v2 00/13] Mbuf Structure Rework, part 2 Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 01/13] mbuf: replace data pointer by an offset Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 02/13] mbuf: reorder fields by time of use Bruce Richardson
2014-09-15  7:11     ` Liu, Jijiang
2014-09-15  8:19       ` Richardson, Bruce
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 03/13] mbuf: expand ol_flags field to 64-bits Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 04/13] mbuf: introduce a flag to indicate a control mbuf Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 05/13] mbuf: minor changes for readability Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 06/13] mbuf: use macros only to access the mbuf metadata Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 07/13] mbuf: move metadata macros to rte_port library Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 08/13] mbuf: add named points inside the mbuf structure Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 09/13] ixgbe: rework vector pmd following mbuf changes Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 10/13] mbuf: split mbuf across two cache lines Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 11/13] mbuf: move l2_len and l3_len to second cache line Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 12/13] ixgbe: Fix perf regression due to moved pool ptr Bruce Richardson
2014-09-15 16:20     ` [dpdk-dev] [PATCH v3 " Bruce Richardson
2014-09-11 13:15   ` [dpdk-dev] [PATCH v2 13/13] ixgbe: Improve slow-path perf: vector scattered RX Bruce Richardson
2014-09-17 22:35   ` [dpdk-dev] [PATCH v2 00/13] Mbuf Structure Rework, part 2 Thomas Monjalon

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140904115549.GA13208@sivswdev02.ir.intel.com \
    --to=bruce.richardson@intel.com \
    --cc=dev@dpdk.org \
    --cc=e_zhumabekov@sts.kz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ https://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git