From: Luke Gorrie <luke@snabb.co>
To: zhihong.wang@intel.com
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"snabb-devel@googlegroups.com" <snabb-devel@googlegroups.com>
Subject: Re: [dpdk-dev] [PATCH 0/4] DPDK memcpy optimization
Date: Sun, 25 Jan 2015 15:50:27 +0100 [thread overview]
Message-ID: <CAA2XHbfxoc9DDgbNUQJJT4TRfhHc5FbXWTnTfwDO7wEjF3y-Qw@mail.gmail.com> (raw)
In-Reply-To: <1421632414-10027-1-git-send-email-zhihong.wang@intel.com>
Hi John,
On 19 January 2015 at 02:53, <zhihong.wang@intel.com> wrote:
> This patch set optimizes memcpy for DPDK for both SSE and AVX platforms.
> It also extends memcpy test coverage with unaligned cases and more test
> points.
>
I am really interested in this work you are doing on memory copies
optimized for packet data. I would like to understand it in more depth. I
have a lot of questions and ideas but let me try to keep it simple for now
:-)
How do you benchmark? where does the "factor of 2-8" cited elsewhere in the
thread come from? how can I reproduce? what results are you seeing compared
with libc?
I did a quick benchmark this weekend based on cachebench
<http://icl.cs.utk.edu/projects/llcbench/cachebench.html>. This seems like
a fairly weak benchmark (always L1 cache, always same alignment, always
predictable branches). Do you think this is relevant? How does this compare
with your results?
I compared:
rte_memcpy (the new optimized one compiled with gcc-4.9 and -march=native
and -O3)
memcpy from glibc 2.19 (ubuntu 14.04)
memcpy from glibc 2.20 (arch linux)
on hardware:
E5-2620v3 (Haswell)
E5-2650 (Sandy Bridge)
running cachebench like this:
./cachebench -p -e1 -x1 -m14
rte_memcpy.h on Haswell:
Memory Copy Library Cache Test
C Size Nanosec MB/sec % Chnge
------- ------- ------- -------
256 0.01 89191.88 1.00
384 0.01 96505.43 0.92
512 0.01 96509.19 1.00
768 0.01 91475.72 1.06
1024 0.01 96293.82 0.95
1536 0.01 96521.66 1.00
2048 0.01 96522.87 1.00
3072 0.01 96525.53 1.00
4096 0.01 96522.79 1.00
6144 0.01 96507.71 1.00
8192 0.01 94584.41 1.02
12288 0.01 95062.80 0.99
16384 0.01 80493.46 1.18
libc 2.20 on Haswell:
Memory Copy Library Cache Test
C Size Nanosec MB/sec % Chnge
------- ------- ------- -------
256 0.01 65978.64 1.00
384 0.01 100249.01 0.66
512 0.01 123476.55 0.81
768 0.01 144699.86 0.85
1024 0.01 159459.88 0.91
1536 0.01 168001.92 0.95
2048 0.01 80738.31 2.08
3072 0.01 80270.02 1.01
4096 0.01 84239.84 0.95
6144 0.01 90600.13 0.93
8192 0.01 89767.94 1.01
12288 0.01 92085.98 0.97
16384 0.01 92719.95 0.99
libc 2.19 on Haswell:
Memory Copy Library Cache Test
C Size Nanosec MB/sec % Chnge
------- ------- ------- -------
256 0.02 59871.69 1.00
384 0.01 68545.94 0.87
512 0.01 72674.23 0.94
768 0.01 79257.47 0.92
1024 0.01 79740.43 0.99
1536 0.01 85483.67 0.93
2048 0.01 87703.68 0.97
3072 0.01 86685.71 1.01
4096 0.01 87147.84 0.99
6144 0.01 68622.96 1.27
8192 0.01 70591.25 0.97
12288 0.01 72621.28 0.97
16384 0.01 67713.63 1.07
rte_memcpy on Sandy Bridge:
Memory Copy Library Cache Test
C Size Nanosec MB/sec % Chnge
------- ------- ------- -------
256 0.02 62158.19 1.00
384 0.01 73256.41 0.85
512 0.01 82032.16 0.89
768 0.01 73919.92 1.11
1024 0.01 75937.51 0.97
1536 0.01 78280.20 0.97
2048 0.01 79562.54 0.98
3072 0.01 80800.93 0.98
4096 0.01 81453.71 0.99
6144 0.01 81915.84 0.99
8192 0.01 82427.98 0.99
12288 0.01 82789.82 1.00
16384 0.01 67519.66 1.23
libc 2.20 on Sandy Bridge:
Memory Copy Library Cache Test
C Size Nanosec MB/sec % Chnge
------- ------- ------- -------
256 0.02 48651.20 1.00
384 0.02 57653.91 0.84
512 0.01 67909.77 0.85
768 0.01 71177.75 0.95
1024 0.01 72519.48 0.98
1536 0.01 76686.24 0.95
2048 0.19 4975.55 15.41
3072 0.19 5091.97 0.98
4096 0.19 5152.38 0.99
6144 0.18 5211.26 0.99
8192 0.18 5245.27 0.99
12288 0.18 5276.50 0.99
16384 0.18 5209.80 1.01
libc 2.19 on Sandy Bridge:
Memory Copy Library Cache Test
C Size Nanosec MB/sec % Chnge
------- ------- ------- -------
256 0.02 44970.51 1.00
384 0.02 51922.46 0.87
512 0.02 57230.56 0.91
768 0.02 63438.96 0.90
1024 0.01 67506.58 0.94
1536 0.01 72579.25 0.93
2048 0.01 75722.25 0.96
3072 0.01 71039.19 1.07
4096 0.01 73946.17 0.96
6144 0.02 40969.79 1.80
8192 0.02 41396.05 0.99
12288 0.02 41830.01 0.99
16384 0.02 42032.40 1.00
Last question: Why is rte_memcpy inline? (Would making it a library
function give you smaller code, comparable performance, and fast compiles?)
Cheers!
-Luke
next prev parent reply other threads:[~2015-01-25 14:50 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-01-19 1:53 zhihong.wang
2015-01-19 1:53 ` [dpdk-dev] [PATCH 1/4] app/test: Disabled VTA for memcpy test in app/test/Makefile zhihong.wang
2015-01-19 1:53 ` [dpdk-dev] [PATCH 2/4] app/test: Removed unnecessary test cases in test_memcpy.c zhihong.wang
2015-01-19 1:53 ` [dpdk-dev] [PATCH 3/4] app/test: Extended test coverage in test_memcpy_perf.c zhihong.wang
2015-01-19 1:53 ` [dpdk-dev] [PATCH 4/4] lib/librte_eal: Optimized memcpy in arch/x86/rte_memcpy.h for both SSE and AVX platforms zhihong.wang
2015-01-20 17:15 ` Stephen Hemminger
2015-01-20 19:16 ` Neil Horman
2015-01-21 3:18 ` Wang, Zhihong
2015-01-25 20:02 ` Jim Thompson
2015-01-26 14:43 ` Wodkowski, PawelX
2015-01-27 5:12 ` Wang, Zhihong
2015-01-19 13:02 ` [dpdk-dev] [PATCH 0/4] DPDK memcpy optimization Neil Horman
2015-01-20 3:01 ` Wang, Zhihong
2015-01-20 15:11 ` Neil Horman
2015-01-20 16:14 ` Bruce Richardson
2015-01-21 3:44 ` Wang, Zhihong
2015-01-21 11:40 ` Bruce Richardson
2015-01-21 12:02 ` Ananyev, Konstantin
2015-01-21 12:38 ` Neil Horman
2015-01-23 3:26 ` Wang, Zhihong
2015-01-21 12:36 ` Marc Sune
2015-01-21 13:02 ` Bruce Richardson
2015-01-21 13:21 ` Marc Sune
2015-01-21 13:26 ` Bruce Richardson
2015-01-21 19:49 ` Stephen Hemminger
2015-01-21 20:54 ` Neil Horman
2015-01-21 21:25 ` Jim Thompson
2015-01-22 0:53 ` Stephen Hemminger
2015-01-22 9:06 ` Luke Gorrie
2015-01-22 13:29 ` Jay Rolette
2015-01-22 18:27 ` Luke Gorrie
2015-01-22 19:36 ` Jay Rolette
2015-01-22 18:21 ` EDMISON, Kelvin (Kelvin)
2015-01-27 8:22 ` Wang, Zhihong
2015-01-28 21:48 ` EDMISON, Kelvin (Kelvin)
2015-01-29 1:53 ` Wang, Zhihong
2015-01-23 6:52 ` Wang, Zhihong
2015-01-26 18:29 ` Ananyev, Konstantin
2015-01-27 1:42 ` Wang, Zhihong
2015-01-27 11:30 ` Ananyev, Konstantin
2015-01-27 12:19 ` Ananyev, Konstantin
2015-01-28 2:06 ` Wang, Zhihong
2015-01-25 14:50 ` Luke Gorrie [this message]
2015-01-26 1:30 ` Wang, Zhihong
2015-01-26 8:03 ` Luke Gorrie
2015-01-27 7:19 ` Wang, Zhihong
2015-01-27 13:57 ` [dpdk-dev] [snabb-devel] " Luke Gorrie
2015-01-29 3:42 ` [dpdk-dev] " Fu, JingguoX
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAA2XHbfxoc9DDgbNUQJJT4TRfhHc5FbXWTnTfwDO7wEjF3y-Qw@mail.gmail.com \
--to=luke@snabb.co \
--cc=dev@dpdk.org \
--cc=snabb-devel@googlegroups.com \
--cc=zhihong.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).