From: Patrick Mahan <mahan@mahan.org>
To: Damien Millescamps <damien.millescamps@6wind.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>
Subject: Re: [dpdk-dev] Best example for showing throughput?
Date: Wed, 29 May 2013 11:24:51 -0700 [thread overview]
Message-ID: <9DE4A69B-DAF4-423A-99B3-95E8FDBB17D3@mahan.org> (raw)
In-Reply-To: <51A60BA0.7000700@6wind.com>
On May 29, 2013, at 7:07 AM, Damien Millescamps <damien.millescamps@6wind.com> wrote:
> On 05/28/2013 09:15 PM, Patrick Mahan wrote:
>> So the overhead cost is almost 70%?
>>
>> Can this ever do line rate? Under what conditions? It has been my experience that the industry standard is testing throughput using these 64 byte packets.
>
> This overhead can actually be explained considering the PCIe 2.1[1]
> standard and 82599 Specifications[2].
>
Damien,
Thanks very much for this explanation of the overhead costs associated with the 64-byte packet size (and the references). I am just recently started looking at what it takes to do 10GE using off the shelf components and having the PCIe overhead explained so clearly helps, hugely!
Patrick
> To sum up, for each packet the adapter needs to first send a read
> request on a 16 Bytes packet descriptor (cf. [2]), to which it will
> receive a read answer. Then the adapter must issue either a read or
> write request to the packet physical address for the size of the packet.
> The frame format for PCIe read and write request is composed with a
> Start of frame, a Sequence Number, a Header, the Data, an LRC and an End
> of frame (cf. [1]). The overhead we are talking about here is more than
> 16 Bytes per PCIe message. In addition to that, the PCIe physical layer
> uses a 10bits per bytes encoding, thus adding to the overhead.
> Now if you apply this to the 64 Bytes packet, you should notice that the
> overhead is way above 70% (4 messages plus descriptor and data size
> times 10b/8b encoding which should be around 83% if I didn't miss anything).
>
> However, if we end up with a limited overhead it is because the 82599
> implements thresholds in order to be able to batch the packet descriptor
> reading / writing back (cf. [2] WTHRESH for example) thus reducing the
> overhead to a little more than 70% with the default DPDK parameters.
>
> You can achieve line-rate for 64 Bytes packets on each port
> independently. When using both port simultaneously you can achieve
> line-rate using packet size above 64Bytes. In the post to which I
> redirected you, Alexander talked about 256Bytes packets. But if you take
> the time to compute the total throughput needed on the PCIe as a
> function of the packet size, you'll probably end up with a lower minimum
> packet size than 256B to achieve line-rate simultaneously on both port.
>
> [1]
> http://www.pcisig.com/members/downloads/specifications/pciexpress/PCI_Express_Base_r2_1_04Mar09.pdf
> [2]
> http://www.intel.com/content/dam/doc/datasheet/82599-10-gbe-controller-datasheet.pdf
>
> --
> Damien Millescamps
next prev parent reply other threads:[~2013-05-29 18:24 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-24 14:11 Patrick Mahan
2013-05-24 14:41 ` Thomas Monjalon
2013-05-24 15:45 ` Thomas Monjalon
2013-05-24 18:51 ` Patrick Mahan
2013-05-25 19:23 ` Damien Millescamps
2013-05-25 20:59 ` Damien Millescamps
2013-05-28 19:15 ` Patrick Mahan
2013-05-29 14:07 ` Damien Millescamps
2013-05-29 18:24 ` Patrick Mahan [this message]
2013-05-24 18:32 ` Patrick Mahan
2013-05-24 20:03 ` Olivier MATZ
2013-05-24 20:44 ` Patrick Mahan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=9DE4A69B-DAF4-423A-99B3-95E8FDBB17D3@mahan.org \
--to=mahan@mahan.org \
--cc=damien.millescamps@6wind.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).