From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.sakura.ad.jp (mail.sakura.ad.jp [210.224.172.11]) by dpdk.org (Postfix) with ESMTP id DD5D130E for ; Wed, 29 May 2013 05:09:20 +0200 (CEST) Received: from [172.31.5.208] ([172.31.5.208]) by mail.sakura.ad.jp (8.14.5/8.14.5) with ESMTP id r4T39Lmr085362; Wed, 29 May 2013 12:09:21 +0900 (JST) (envelope-from n-matsumoto@sakura.ad.jp) Date: Wed, 29 May 2013 12:09:26 +0900 From: Naoto MATSUMOTO To: Shinae Woo In-Reply-To: References: <20130528113005.5E6D.C42C3789@sakura.ad.jp> Message-Id: <20130529120925.DD69.C42C3789@sakura.ad.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Becky! ver. 2.65 [ja] Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Performances are not scale with multiple ports X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 May 2013 03:09:21 -0000 Hi shiane and all Emre and Stephen, thanks your suggest for us. We are not testing multi-port traffic gen using NETMAP though DPDK l3fw, but we understood some packet recive-side without NETMAP Linux. Please try it using stephen's pktgen if you have more traffic gen/recive server resources or Using IXIA. best regards, -- Naoto On Tue, 28 May 2013 12:05:50 +0900 Shinae Woo wrote: > Thanks for sharing Naoto. > > So in your experiments, the forwarding performance still does not reach the > line rate. > > Your perf record shows that the cpu spend most of the time in polling for > receiving packets, > and no other heavy operation. > Even though the application polling packets in its best, > the forwarder miss some packets from elsewhere from the application-side. > > The dpdk document shows that 160Mpps forwarding performance in 2 sockets, > but I can only reach the 13 Mpps in 2 ports. > Even doubling the number of ports to 4 ports, the performance is still less > than 17Mpps. > > I want to know where is the bottleneck lies in my environments, or > how I can reprocuce the same performance as the dpdk published. > > Thank you, > Shinae > > > > On Tue, May 28, 2013 at 11:30 AM, Naoto MATSUMOTO > wrote: > > > > > FYI: Disruptive IP Networking with Intel DPDK on Linux > > http://slidesha.re/SeVFZo > > > > > > On Tue, 28 May 2013 11:26:30 +0900 > > Shinae Woo wrote: > > > > > Hello, all. > > > > > > I play the dpdk-1.2.3r1 with examples. > > > > > > But I can not achieve the line-rate packet receive performance, > > > and the performance is not scale with multiple ports. > > > > > > For example, in example l2fwd, I have tested two cases with 2 ports, and > > 4 > > > ports, > > > using belowed command line each > > > > > > ./build/l2fwd -cf -n3 -- -p3 > > > ./build/l2fwd -cf -n3 -- -pf > > > > > > But both cases, the aggregated performance are not scale. > > > > > > == experiments environments == > > > - Two Intel 82599 NICs (total 4 ports) > > > - Intel Xeon X5690 @ 3.47GHz * 2 (total 12 cores) > > > - 1024 * 2MB hugepages > > > - Linux 2.6.38-15-server > > > - Each ports receiving 10Gbps of traffic of 64 bytes packets, 14.88Mpps. > > > > > > *1. Packet forwarding performance* > > > > > > In 2 ports case, receive performance is 13Mpps, > > > In 4 ports case, not 26Mbps, only 16.8Mpps. > > > > > > Port statistics ==================================== > > > Statistics for port 0 ------------------------------ > > > Packets sent: 4292256 > > > Packets received: 6517396 > > > Packets dropped: 2224776 > > > Statistics for port 1 ------------------------------ > > > Packets sent: 4291840 > > > Packets received: 6517044 > > > Packets dropped: 2225556 > > > Aggregate statistics =============================== > > > Total packets sent: 8584128 > > > Total packets received: 13034472 > > > Total packets dropped: 4450332 > > > ==================================================== > > > > > > Port statistics ==================================== > > > Statistics for port 0 ------------------------------ > > > Packets sent: 1784064 > > > Packets received: 2632700 > > > Packets dropped: 848128 > > > Statistics for port 1 ------------------------------ > > > Packets sent: 1784104 > > > Packets received: 2632196 > > > Packets dropped: 848596 > > > Statistics for port 2 ------------------------------ > > > Packets sent: 3587616 > > > Packets received: 5816344 > > > Packets dropped: 2200176 > > > Statistics for port 3 ------------------------------ > > > Packets sent: 3587712 > > > Packets received: 5787848 > > > Packets dropped: 2228684 > > > Aggregate statistics =============================== > > > Total packets sent: 10743560 > > > Total packets received: 16869152 > > > Total packets dropped: 6125608 > > > ==================================================== > > > > > > *2. Packet receiving performance* > > > I fix the codes for only receiving packets (not forwarding), > > > the performance is still not scalable as each 13.3Mpps, 18Mpps. > > > > > > Port statistics ==================================== > > > Statistics for port 0 ------------------------------ > > > Packets sent: 0 > > > Packets received: 6678860 > > > Packets dropped: 0 > > > Statistics for port 1 ------------------------------ > > > Packets sent: 0 > > > Packets received: 6646120 > > > Packets dropped: 0 > > > Aggregate statistics =============================== > > > Total packets sent: 0 > > > Total packets received: 13325012 > > > Total packets dropped: 0 > > > ==================================================== > > > > > > Port statistics ==================================== > > > Statistics for port 0 ------------------------------ > > > Packets sent: 0 > > > Packets received: 3129624 > > > Packets dropped: 0 > > > Statistics for port 1 ------------------------------ > > > Packets sent: 0 > > > Packets received: 3131292 > > > Packets dropped: 0 > > > Statistics for port 2 ------------------------------ > > > Packets sent: 0 > > > Packets received: 6260908 > > > Packets dropped: 0 > > > Statistics for port 3 ------------------------------ > > > Packets sent: 0 > > > Packets received: 6238764 > > > Packets dropped: 0 > > > Aggregate statistics =============================== > > > Total packets sent: 0 > > > Total packets received: 18760640 > > > Total packets dropped: 0 > > > ==================================================== > > > > > > The question is that > > > 1. How I can achieve each port receiving full 14.88Mpps ? > > > What might be the bottleneck in current environment? > > > 2. Why the performance using multiple ports is not scale? > > > I guess doubling ports shows the doubling the receiving performance, > > > but it shows not. I am curious about what is limiting the packet > > > receivng performance. > > > > > > Thanks, > > > Shinae > > > > -- > > SAKURA Internet Inc. / Senior Researcher > > Naoto MATSUMOTO > > SAKURA Internet Research Center > > > > -- SAKURA Internet Inc. / Senior Researcher Naoto MATSUMOTO SAKURA Internet Research Center