From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pb0-x22d.google.com (mail-pb0-x22d.google.com [IPv6:2607:f8b0:400e:c01::22d]) by dpdk.org (Postfix) with ESMTP id AEEC530E for ; Tue, 28 May 2013 04:26:25 +0200 (CEST) Received: by mail-pb0-f45.google.com with SMTP id mc17so7306395pbc.32 for ; Mon, 27 May 2013 19:26:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=VGzw/CNZnpgC6y1fFwjtLTfXkSAT2qcBC1piQqCNpbE=; b=fiZuKFf/unD7I9m43AZTy09fV1rtzeRX/tCOsGM55GA1ChjHKqIlDWFH2sKxpG+8ym 7isXSdXeR1mLAOvGW2QHzKBZDyLnFEp7ArumFa1hZn79AD5TRWnypOIAdgkUSqzkVcHY W+V91UM4BhuIc0SEEGmTsWXJtv4MBhvZ3lZ6+T89HeZoOdcY10GKIRyjKnIWW0pJTt/s HaWqDxOjEMXvaYdZ72XOEpngApMpuVABui8q6iOpf3/PGjaVUNEX8Z5Nut4yB6d3kep+ O/AoUNFJN6dA7q/TurCOPbf7DjlcGLBHfxVPOPi4XATGaE+lQL1TzeQ4ZvYuF+Q/Hl4j 3IWg== MIME-Version: 1.0 X-Received: by 10.66.245.110 with SMTP id xn14mr32024452pac.130.1369707990747; Mon, 27 May 2013 19:26:30 -0700 (PDT) Received: by 10.66.155.170 with HTTP; Mon, 27 May 2013 19:26:30 -0700 (PDT) Date: Tue, 28 May 2013 11:26:30 +0900 Message-ID: From: Shinae Woo To: dev@dpdk.org Content-Type: multipart/alternative; boundary=047d7b111acd3c0f7f04ddbdfeca Subject: [dpdk-dev] Performances are not scale with multiple ports X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 02:26:26 -0000 --047d7b111acd3c0f7f04ddbdfeca Content-Type: text/plain; charset=ISO-8859-1 Hello, all. I play the dpdk-1.2.3r1 with examples. But I can not achieve the line-rate packet receive performance, and the performance is not scale with multiple ports. For example, in example l2fwd, I have tested two cases with 2 ports, and 4 ports, using belowed command line each ./build/l2fwd -cf -n3 -- -p3 ./build/l2fwd -cf -n3 -- -pf But both cases, the aggregated performance are not scale. == experiments environments == - Two Intel 82599 NICs (total 4 ports) - Intel Xeon X5690 @ 3.47GHz * 2 (total 12 cores) - 1024 * 2MB hugepages - Linux 2.6.38-15-server - Each ports receiving 10Gbps of traffic of 64 bytes packets, 14.88Mpps. *1. Packet forwarding performance* In 2 ports case, receive performance is 13Mpps, In 4 ports case, not 26Mbps, only 16.8Mpps. Port statistics ==================================== Statistics for port 0 ------------------------------ Packets sent: 4292256 Packets received: 6517396 Packets dropped: 2224776 Statistics for port 1 ------------------------------ Packets sent: 4291840 Packets received: 6517044 Packets dropped: 2225556 Aggregate statistics =============================== Total packets sent: 8584128 Total packets received: 13034472 Total packets dropped: 4450332 ==================================================== Port statistics ==================================== Statistics for port 0 ------------------------------ Packets sent: 1784064 Packets received: 2632700 Packets dropped: 848128 Statistics for port 1 ------------------------------ Packets sent: 1784104 Packets received: 2632196 Packets dropped: 848596 Statistics for port 2 ------------------------------ Packets sent: 3587616 Packets received: 5816344 Packets dropped: 2200176 Statistics for port 3 ------------------------------ Packets sent: 3587712 Packets received: 5787848 Packets dropped: 2228684 Aggregate statistics =============================== Total packets sent: 10743560 Total packets received: 16869152 Total packets dropped: 6125608 ==================================================== *2. Packet receiving performance* I fix the codes for only receiving packets (not forwarding), the performance is still not scalable as each 13.3Mpps, 18Mpps. Port statistics ==================================== Statistics for port 0 ------------------------------ Packets sent: 0 Packets received: 6678860 Packets dropped: 0 Statistics for port 1 ------------------------------ Packets sent: 0 Packets received: 6646120 Packets dropped: 0 Aggregate statistics =============================== Total packets sent: 0 Total packets received: 13325012 Total packets dropped: 0 ==================================================== Port statistics ==================================== Statistics for port 0 ------------------------------ Packets sent: 0 Packets received: 3129624 Packets dropped: 0 Statistics for port 1 ------------------------------ Packets sent: 0 Packets received: 3131292 Packets dropped: 0 Statistics for port 2 ------------------------------ Packets sent: 0 Packets received: 6260908 Packets dropped: 0 Statistics for port 3 ------------------------------ Packets sent: 0 Packets received: 6238764 Packets dropped: 0 Aggregate statistics =============================== Total packets sent: 0 Total packets received: 18760640 Total packets dropped: 0 ==================================================== The question is that 1. How I can achieve each port receiving full 14.88Mpps ? What might be the bottleneck in current environment? 2. Why the performance using multiple ports is not scale? I guess doubling ports shows the doubling the receiving performance, but it shows not. I am curious about what is limiting the packet receivng performance. Thanks, Shinae --047d7b111acd3c0f7f04ddbdfeca Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: base64 PGRpdiBkaXI9Imx0ciI+PGRpdj48ZGl2PjxkaXY+PGRpdj5IZWxsbywgYWxsLjxicj48YnI+PC9k aXY+SSBwbGF5IHRoZSBkcGRrLTEuMi4zcjEgd2l0aCBleGFtcGxlcy4gPGJyPjxicj5CdXQgSSBj YW4gbm90IGFjaGlldmUgdGhlIGxpbmUtcmF0ZSBwYWNrZXQgcmVjZWl2ZSBwZXJmb3JtYW5jZSw8 YnI+YW5kIHRoZSBwZXJmb3JtYW5jZSBpcyBub3Qgc2NhbGUgd2l0aCBtdWx0aXBsZSBwb3J0cy48 YnI+DQo8L2Rpdj48YnI+PC9kaXY+Rm9yIGV4YW1wbGUsIGluIGV4YW1wbGUgbDJmd2QsIEkgaGF2 ZSB0ZXN0ZWQgdHdvIGNhc2VzIHdpdGggMiBwb3J0cywgYW5kIDQgcG9ydHMsPGJyPjwvZGl2PnVz aW5nIGJlbG93ZWQgY29tbWFuZCBsaW5lIGVhY2g8YnI+PGRpdj48YnI+Li9idWlsZC9sMmZ3ZCAt Y2YgLW4zIC0tIC1wMzxicj4uL2J1aWxkL2wyZndkIC1jZiAtbjMgLS0gLXBmPGJyPjxicj48L2Rp dj4NCjxkaXY+QnV0IGJvdGggY2FzZXMsIHRoZSBhZ2dyZWdhdGVkIHBlcmZvcm1hbmNlIGFyZSBu b3Qgc2NhbGUuPGJyPjxicj48L2Rpdj48ZGl2Pj09IGV4cGVyaW1lbnRzIGVudmlyb25tZW50cyA9 PTxicj48L2Rpdj48ZGl2Pi0gVHdvIEludGVsIDgyNTk5IE5JQ3MgKHRvdGFsIDQgcG9ydHMpPGJy Pi0gSW50ZWwgWGVvbiBYNTY5MKAgQCAzLjQ3R0h6ICogMiAodG90YWwgMTIgY29yZXMpPGJyPg0K PC9kaXY+PGRpdj4tIDEwMjQgKiAyTUIgaHVnZXBhZ2VzPGJyPi0gTGludXggMi42LjM4LTE1LXNl cnZlcjxicj4tIEVhY2ggcG9ydHMgcmVjZWl2aW5nIDEwR2JwcyBvZiB0cmFmZmljIG9mIDY0IGJ5 dGVzIHBhY2tldHMsIDE0Ljg4TXBwcy48YnI+PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj48Yj4x LiBQYWNrZXQgZm9yd2FyZGluZyBwZXJmb3JtYW5jZTwvYj48YnI+PGJyPjwvZGl2PjxkaXY+DQpJ biAyIHBvcnRzIGNhc2UsoCByZWNlaXZlIHBlcmZvcm1hbmNlIGlzIDEzTXBwcywgPGJyPjwvZGl2 PjxkaXY+SW4gNCBwb3J0cyBjYXNlLKAgbm90IDI2TWJwcywgb25seSAxNi44TXBwcy48YnI+PC9k aXY+PGRpdj48YnI+UG9ydCBzdGF0aXN0aWNzID09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PTxicj5TdGF0aXN0aWNzIGZvciBwb3J0IDAgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tPGJyPg0KUGFja2V0cyBzZW50OqCgoKCgoKCgoKCgoKCgoKCgIDQyOTIyNTY8YnI+UGFj a2V0cyByZWNlaXZlZDqgoKCgoKCgoKCgoKCgIDY1MTczOTY8YnI+UGFja2V0cyBkcm9wcGVkOqCg oKCgoKCgoKCgoKCgIDIyMjQ3NzY8YnI+U3RhdGlzdGljcyBmb3IgcG9ydCAxIC0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLTxicj5QYWNrZXRzIHNlbnQ6oKCgoKCgoKCgoKCgoKCgoKAgNDI5 MTg0MDxicj4NClBhY2tldHMgcmVjZWl2ZWQ6oKCgoKCgoKCgoKCgoCA2NTE3MDQ0PGJyPlBhY2tl dHMgZHJvcHBlZDqgoKCgoKCgoKCgoKCgoCAyMjI1NTU2PGJyPkFnZ3JlZ2F0ZSBzdGF0aXN0aWNz ID09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08YnI+VG90YWwgcGFja2V0cyBzZW50OqCg oKCgoKCgoKCgIDg1ODQxMjg8YnI+VG90YWwgcGFja2V0cyByZWNlaXZlZDqgoKCgoKAgMTMwMzQ0 NzI8YnI+DQpUb3RhbCBwYWNrZXRzIGRyb3BwZWQ6oKCgoKCgoKAgNDQ1MDMzMjxicj49PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PGJyPjxicj5Qb3J0 IHN0YXRpc3RpY3MgPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PGJyPlN0YXRp c3RpY3MgZm9yIHBvcnQgMCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS08YnI+UGFja2V0 cyBzZW50OqCgoKCgoKCgoKCgoKCgoKCgIDE3ODQwNjQ8YnI+DQpQYWNrZXRzIHJlY2VpdmVkOqCg oKCgoKCgoKCgoKAgMjYzMjcwMDxicj5QYWNrZXRzIGRyb3BwZWQ6oKCgoKCgoKCgoKCgoKCgIDg0 ODEyODxicj5TdGF0aXN0aWNzIGZvciBwb3J0IDEgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tPGJyPlBhY2tldHMgc2VudDqgoKCgoKCgoKCgoKCgoKCgoCAxNzg0MTA0PGJyPlBhY2tldHMg cmVjZWl2ZWQ6oKCgoKCgoKCgoKCgoCAyNjMyMTk2PGJyPg0KUGFja2V0cyBkcm9wcGVkOqCgoKCg oKCgoKCgoKCgoCA4NDg1OTY8YnI+U3RhdGlzdGljcyBmb3IgcG9ydCAyIC0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLTxicj5QYWNrZXRzIHNlbnQ6oKCgoKCgoKCgoKCgoKCgoKAgMzU4NzYx Njxicj5QYWNrZXRzIHJlY2VpdmVkOqCgoKCgoKCgoKCgoKAgNTgxNjM0NDxicj5QYWNrZXRzIGRy b3BwZWQ6oKCgoKCgoKCgoKCgoKAgMjIwMDE3Njxicj4NClN0YXRpc3RpY3MgZm9yIHBvcnQgMyAt LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS08YnI+UGFja2V0cyBzZW50OqCgoKCgoKCgoKCg oKCgoKCgIDM1ODc3MTI8YnI+UGFja2V0cyByZWNlaXZlZDqgoKCgoKCgoKCgoKCgIDU3ODc4NDg8 YnI+UGFja2V0cyBkcm9wcGVkOqCgoKCgoKCgoKCgoKCgIDIyMjg2ODQ8YnI+QWdncmVnYXRlIHN0 YXRpc3RpY3MgPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PTxicj4NClRvdGFsIHBhY2tl dHMgc2VudDqgoKCgoKCgoKCgIDEwNzQzNTYwPGJyPlRvdGFsIHBhY2tldHMgcmVjZWl2ZWQ6oKCg oKCgIDE2ODY5MTUyPGJyPlRvdGFsIHBhY2tldHMgZHJvcHBlZDqgoKCgoKCgoCA2MTI1NjA4PGJy Pj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08YnI+ PGJyPjxiPjIuIFBhY2tldCByZWNlaXZpbmcgcGVyZm9ybWFuY2U8L2I+PGJyPg0KPC9kaXY+PGRp dj5JIGZpeCB0aGUgY29kZXMgZm9yIG9ubHkgcmVjZWl2aW5nIHBhY2tldHMgKG5vdCBmb3J3YXJk aW5nKSw8YnI+PC9kaXY+PGRpdj50aGUgcGVyZm9ybWFuY2UgaXMgc3RpbGwgbm90IHNjYWxhYmxl IGFzIGVhY2ggMTMuM01wcHMsIDE4TXBwcy48YnI+PC9kaXY+PGRpdj48YnI+UG9ydCBzdGF0aXN0 aWNzID09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PTxicj4NClN0YXRpc3RpY3Mg Zm9yIHBvcnQgMCAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS08YnI+UGFja2V0cyBzZW50 OqCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgIDA8YnI+UGFja2V0cyByZWNlaXZlZDqgoKCgoKCgoKCg oKCgIDY2Nzg4NjA8YnI+UGFja2V0cyBkcm9wcGVkOqCgoKCgoKCgoKCgoKCgoKCgoKCgIDA8YnI+ U3RhdGlzdGljcyBmb3IgcG9ydCAxIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLTxicj4N ClBhY2tldHMgc2VudDqgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoCAwPGJyPlBhY2tldHMgcmVjZWl2 ZWQ6oKCgoKCgoKCgoKCgoCA2NjQ2MTIwPGJyPlBhY2tldHMgZHJvcHBlZDqgoKCgoKCgoKCgoKCg oKCgoKCgoCAwPGJyPkFnZ3JlZ2F0ZSBzdGF0aXN0aWNzID09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT08YnI+VG90YWwgcGFja2V0cyBzZW50OqCgoKCgoKCgoKCgoKCgoKCgIDA8YnI+DQpU b3RhbCBwYWNrZXRzIHJlY2VpdmVkOqCgoKCgoCAxMzMyNTAxMjxicj5Ub3RhbCBwYWNrZXRzIGRy b3BwZWQ6oKCgoKCgoKCgoKCgoKAgMDxicj49PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PGJyPjxicj5Qb3J0IHN0YXRpc3RpY3MgPT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PGJyPlN0YXRpc3RpY3MgZm9yIHBvcnQgMCAtLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS08YnI+DQpQYWNrZXRzIHNlbnQ6oKCgoKCgoKCgoKCgoKCg oKCgoKCgoKAgMDxicj5QYWNrZXRzIHJlY2VpdmVkOqCgoKCgoKCgoKCgoKAgMzEyOTYyNDxicj5Q YWNrZXRzIGRyb3BwZWQ6oKCgoKCgoKCgoKCgoKCgoKCgoKAgMDxicj5TdGF0aXN0aWNzIGZvciBw b3J0IDEgLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tPGJyPlBhY2tldHMgc2VudDqgoKCg oKCgoKCgoKCgoKCgoKCgoKCgoCAwPGJyPg0KUGFja2V0cyByZWNlaXZlZDqgoKCgoKCgoKCgoKCg IDMxMzEyOTI8YnI+UGFja2V0cyBkcm9wcGVkOqCgoKCgoKCgoKCgoKCgoKCgoKCgIDA8YnI+U3Rh dGlzdGljcyBmb3IgcG9ydCAyIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLTxicj5QYWNr ZXRzIHNlbnQ6oKCgoKCgoKCgoKCgoKCgoKCgoKCgoKAgMDxicj5QYWNrZXRzIHJlY2VpdmVkOqCg oKCgoKCgoKCgoKAgNjI2MDkwODxicj4NClBhY2tldHMgZHJvcHBlZDqgoKCgoKCgoKCgoKCgoKCg oKCgoCAwPGJyPlN0YXRpc3RpY3MgZm9yIHBvcnQgMyAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS08YnI+UGFja2V0cyBzZW50OqCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgIDA8YnI+UGFja2V0 cyByZWNlaXZlZDqgoKCgoKCgoKCgoKCgIDYyMzg3NjQ8YnI+UGFja2V0cyBkcm9wcGVkOqCgoKCg oKCgoKCgoKCgoKCgoKCgIDA8YnI+DQpBZ2dyZWdhdGUgc3RhdGlzdGljcyA9PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PGJyPlRvdGFsIHBhY2tldHMgc2VudDqgoKCgoKCgoKCgoKCgoKCg oCAwPGJyPlRvdGFsIHBhY2tldHMgcmVjZWl2ZWQ6oKCgoKCgIDE4NzYwNjQwPGJyPlRvdGFsIHBh Y2tldHMgZHJvcHBlZDqgoKCgoKCgoKCgoKCgoCAwPGJyPj09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08YnI+DQo8YnI+PC9kaXY+PGRpdj5UaGUgcXVl c3Rpb24gaXMgdGhhdDxicj48L2Rpdj48ZGl2PjEuIEhvdyBJIGNhbiBhY2hpZXZlIGVhY2ggcG9y dCByZWNlaXZpbmcgZnVsbCAxNC44OE1wcHMgPzxicj48L2Rpdj48ZGl2PqCgoCBXaGF0IG1pZ2h0 IGJlIHRoZSBib3R0bGVuZWNrIGluIGN1cnJlbnQgZW52aXJvbm1lbnQ/PGJyPjwvZGl2PjxkaXY+ Mi4gV2h5IHRoZSBwZXJmb3JtYW5jZSB1c2luZyBtdWx0aXBsZSBwb3J0cyBpcyBub3Qgc2NhbGU/ PGJyPg0KPC9kaXY+PGRpdj6goKAgSSBndWVzcyBkb3VibGluZyBwb3J0cyBzaG93cyB0aGUgZG91 YmxpbmcgdGhlIHJlY2VpdmluZyBwZXJmb3JtYW5jZSwgPGJyPjwvZGl2PjxkaXY+oKCgIGJ1dCBp dCBzaG93cyBub3QuIEkgYW0gY3VyaW91cyBhYm91dCB3aGF0IGlzIGxpbWl0aW5nIHRoZSBwYWNr ZXQgcmVjZWl2bmcgcGVyZm9ybWFuY2UuPGJyPjxicj48L2Rpdj48ZGl2PlRoYW5rcyw8YnI+U2hp bmFlPGJyPg0KPC9kaXY+PC9kaXY+DQo= --047d7b111acd3c0f7f04ddbdfeca--