From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ex1.cas-well.com (unknown [122.147.166.54]) by dpdk.org (Postfix) with ESMTP id D97C22E89 for ; Mon, 2 Sep 2013 05:21:44 +0200 (CEST) Received: from [172.16.1.178] (122.147.166.57) by ex1.cas-well.com (192.168.200.10) with Microsoft SMTP Server id 14.2.247.3; Mon, 2 Sep 2013 11:24:28 +0800 Message-ID: <52240466.7050907@cas-well.com> Date: Mon, 2 Sep 2013 11:22:14 +0800 From: Zachary Organization: Cas-Well User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130803 Thunderbird/17.0.8 MIME-Version: 1.0 To: Content-Type: multipart/alternative; boundary="------------030907080403080602000004" X-Originating-IP: [122.147.166.57] Cc: =?UTF-8?B?Illhbm5pYy5DaG91ICjlkajlk7LmraMpIDogNjg=?= =?UTF-8?B?MDgi?= , =?UTF-8?B?IkFsYW4gWXUgKOS/nuS6puWBiSkgOiA2NjMyIg==?= Subject: [dpdk-dev] DPDK & QPI performance issue in Romley platform. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 03:21:46 -0000 --------------030907080403080602000004 Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: quoted-printable Hi~ I have a question about DPDK & QPI performance issue in Romley platform. Recently, I use DPDK example, l2fwd, to test DPDK's performance in my Romle= y platform. When I try to do the test, crossing used CPU, I find the performance dramat= ically decrease. Is it true? Or any method can prove the phenomenon? In my opinion, there should be no this kind of issue here due to QPI have e= nough bandwidth to deal the kinds of case. Thus, I am so amaze in our results and can not explain it. Could someone can help me to solve this problem. Thank a lot! My testing environment describe as below: Platform: Romley CPU: E5-2643 * 2 RAM: Transcend 8GB PC3-1600 DDR3 * 8 OS: Fedora core 14 DPDK: v1.3.1r2, example/l2fwd Slot setting: SlotA is controled by CPU1 directly. SlotB is controled by CPU0 directly. DPDK pre-setting: a. BIOS setting: HT=3Ddisable b. Kernel paramaters isolcpus=3D2,3,6,7 default_hugepagesz=3D1024M hugepagesz=3D1024M hugepages=3D16 c. OS setting: service avahi-daemon stop service NetworkManager stop service iptables stop service acpid stop selinux disable Example program Command: a. SlotB(CPU0) -> CPU1 #>./l2fwd -c 0xc -n 4 -- -q 1 -p 0xc b. SlotA(CPU1) -> CPU0 #>./l2fwd -c 0xc0 -n 4 -- -q 1 -p 0xc0 Results: use frame size 128 bytes CPU Affinity Slot A (CPU1) Slot B (CPU0) CPU0 15.9% 96.49% CPU1 90.88% 24.78% =E6=9C=AC=E4=BF=A1=E4=BB=B6=E5=8F=AF=E8=83=BD=E5=8C=85=E5=90=AB=E7=91=9E=E7= =A5=BA=E9=9B=BB=E9=80=9A=E6=A9=9F=E5=AF=86=E8=B3=87=E8=A8=8A=EF=BC=8C=E9=9D= =9E=E6=8C=87=E5=AE=9A=E4=B9=8B=E6=94=B6=E4=BB=B6=E8=80=85=EF=BC=8C=E8=AB=8B= =E5=8B=BF=E4=BD=BF=E7=94=A8=E6=88=96=E6=8F=AD=E9=9C=B2=E6=9C=AC=E4=BF=A1=E4= =BB=B6=E5=85=A7=E5=AE=B9=EF=BC=8C=E4=B8=A6=E8=AB=8B=E9=8A=B7=E6=AF=80=E6=AD= =A4=E4=BF=A1=E4=BB=B6=E3=80=82 This email may contain confidential informat= ion. Please do not use or disclose it in any way and delete it if you are n= ot the intended recipient. --------------030907080403080602000004 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi~

I have a question about DPDK & QPI performance issue in Romley pl= atform.
Recently, I use DPDK example, l2fwd, to test DPDK's performance in my Romle= y platform.
When I try to do the test, crossing used CPU, I find the performance dramat= ically decrease.
Is it true? Or any method can prove the phenomenon?

In my opinion, there should be no this kind of issue here due to QPI have e= nough bandwidth to deal the kinds of case.
Thus, I am so amaze in our results and can not explain it.
Could someone can help me to solve this problem.

Thank a lot!

My testing environment describe as below:

Platform:         Romley
CPU:            = ;    E5-2643 * 2
RAM:            = ;   Transcend 8GB PC3-1600 DDR3 * 8
OS:           &= nbsp;     Fedora core 14
DPDK:           = ; v1.3.1r2, example/l2fwd
Slot setting:
            &nb= sp;         SlotA is controled by C= PU1 directly.
          &nbs= p;           SlotB is con= troled by CPU0 directly.

= DPDK pre-setting:
a. BIOS setting:
    HT=3Ddisable
b. Kernel paramaters
    isolcpus=3D2,3,6,7
    default_hugepagesz=3D1024M
    hugepagesz=3D1024M
    hugepages=3D16
c. OS setting:
    service avahi-daemon stop
    service NetworkManager stop
    service iptables stop
    service acpid stop
    selinux disable

Example program Command:
a. SlotB(CPU0) -> CPU1
    #>./l2fwd -c 0xc -n 4 -- -q 1 -p 0xc

b. SlotA(CPU1) -> CPU0
    #>./l2fwd -c 0xc0 -n 4 -- -q 1 -p 0xc0 =

Results:
     use frame size 128 bytes

CPU Affinity	Slot = A (CPU1)	Slot = B (CPU0)
CPU0<= /span>	15.9%=	96.49= %
CPU1<= /span>	90.88= %	24.78= %

=E6=9C=AC=E4=BF=A1=E4=BB=B6=E5=8F=AF=E8=83=BD=E5=8C=85=E5=90=AB=E7=91=9E=E7= =A5=BA=E9=9B=BB=E9=80=9A=E6=A9=9F=E5=AF=86=E8=B3=87=E8=A8=8A=EF=BC=8C=E9=9D= =9E=E6=8C=87=E5=AE=9A=E4=B9=8B=E6=94=B6=E4=BB=B6=E8=80=85=EF=BC=8C=E8=AB=8B= =E5=8B=BF=E4=BD=BF=E7=94=A8=E6=88=96=E6=8F=AD=E9=9C=B2=E6=9C=AC=E4=BF=A1=E4= =BB=B6=E5=85=A7=E5=AE=B9=EF=BC=8C=E4=B8=A6=E8=AB=8B=E9=8A=B7=E6=AF=80=E6=AD= =A4=E4=BF=A1=E4=BB=B6=E3=80=82 This email may contain confidential informat= ion. Please do not use or disclose it in any way and delete it if you are n= ot the intended recipient. --------------030907080403080602000004-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) by dpdk.org (Postfix) with ESMTP id 39D891F3 for ; Mon, 2 Sep 2013 18:09:42 +0200 (CEST) Received: by mail-pa0-f50.google.com with SMTP id fb10so5363171pad.37 for ; Mon, 02 Sep 2013 09:10:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=WJheowdGKoyFQufjzZMFcwAWF0of9XPwF1uZnN7HEpY=; b=Y4iLNcpUnaEqgQX4PZ1HJnkhrB69nVWQdtxDc43Gzb1pNRz4hg2pKynIPaxtboSkOO Ch9qdZxyMK1U7pTciFZlrZIXm7qXY6bdUy6qEhaVxwsePfP/RYa8PomaLmHFQb9KTx5R SRtSn70dbigZXi9AcF0UIZ8Sw+q/cGVcoskgYz9VPAb4vDzARHsf6YKPmTqmPl9s9mf7 RSh555J8xe1hWSE+oSpnDSH/YCr8vJLpxtaxuVyP5UdKw2MPVSInzHoN/tMnUVgGEQrX 5xjROwyuDNqBHA4HgrJJLhk7DxaDD5WRNVRK7kB6e2duQOEbHnx9co4k0alp95lgcaWW XtNg== X-Gm-Message-State: ALoCoQml9nPjfg8UnX8WqA9OqeUL8eSiYiNL/yY1ZedybG1D5w4l9XFs6ak4K3FWKEQb9U2mlbf/ X-Received: by 10.68.14.234 with SMTP id s10mr4215282pbc.139.1378138215336; Mon, 02 Sep 2013 09:10:15 -0700 (PDT) Received: from nehalam.linuxnetplumber.net (static-50-53-69-237.bvtn.or.frontiernet.net. [50.53.69.237]) by mx.google.com with ESMTPSA id ta10sm18048127pab.5.1969.12.31.16.00.00 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 02 Sep 2013 09:10:15 -0700 (PDT) Date: Mon, 2 Sep 2013 09:10:12 -0700 From: Stephen Hemminger To: Zachary Message-ID: <20130902091012.2e68b88e@nehalam.linuxnetplumber.net> In-Reply-To: <52240466.7050907@cas-well.com> References: <52240466.7050907@cas-well.com> X-Mailer: Claws Mail 3.8.1 (GTK+ 2.24.10; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org, "Yannic.Chou \(=?utf-8?B?5ZGo5ZOy5q2j?=\) : 6808" , "Alan Yu \(=?utf-8?B?5L+e5Lqm5YGJ?=\) : 6632" Subject: Re: [dpdk-dev] DPDK & QPI performance issue in Romley platform. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Sep 2013 16:09:42 -0000 On Mon, 2 Sep 2013 11:22:14 +0800 Zachary wrote: > Hi~ > > I have a question about DPDK & QPI performance issue in Romley platform. > Recently, I use DPDK example, l2fwd, to test DPDK's performance in my Romley platform. > When I try to do the test, crossing used CPU, I find the performance dramatically decrease. > Is it true? Or any method can prove the phenomenon? > > In my opinion, there should be no this kind of issue here due to QPI have enough bandwidth to deal the kinds of case. > Thus, I am so amaze in our results and can not explain it. > Could someone can help me to solve this problem. > > Thank a lot! Many DPDK API's have NUMA socket as one of the parameters. In order to get good performance it is up to the application to be NUMA aware and use socket local resources. One example we do is to have a packet mbuf pool per socket, and assign each device to the correct pool. Also, you may want to choose which lcore's to assign to which function based on socket locality. For example threads that are polling receiver should be on same socket as that NIC. Remember the example applications are demo toys, and don't do all the things a real application would need to do. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtpbg63.qq.com (smtpbg63.qq.com [103.7.29.150]) by dpdk.org (Postfix) with SMTP id D33132E89 for ; Tue, 3 Sep 2013 18:19:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qq.com; s=s201307; t=1378225168; bh=89DEWWfo2cQw5B5ScR0ED54UUeo++aIFul030SqfvWI=; h=X-QQ-SSF:X-HAS-ATTACH:X-QQ-BUSINESS-ORIGIN:X-Originating-IP: In-Reply-To:References:X-QQ-STYLE:X-QQ-mid:From:To:Subject: Mime-Version:Content-Type:Content-Transfer-Encoding:Date: X-Priority:Message-ID:X-QQ-MIME:X-Mailer:X-QQ-Mailer: X-QQ-ReplyHash:X-QQ-SENDSIZE; b=Hk2A9c7qDsEC88VLae4L/SdIVkX0ebBt74+Esd4fU59kzv2UYQ8RjUBEGN/eYQ3V/ 9Rwvpm+ssrzoZUH2Dmkdg4Nv16G5BR4VqN3gV0IseL/bXQbPCxjksPigU1NgMBSbFL NpqEaHkVbqFlGxYnH7xm8suasUvqErXe+W24r1hI= X-QQ-SSF: 00000000000000F000000000000000Z X-HAS-ATTACH: no X-QQ-BUSINESS-ORIGIN: 2 X-Originating-IP: 211.155.19.91 In-Reply-To: <52240466.7050907@cas-well.com> References: <52240466.7050907@cas-well.com> X-QQ-STYLE: X-QQ-mid: webmail658t1378225168t9340803 From: "=?gb18030?B?Qm9iIENoZW4=?=" To: "=?gb18030?B?WmFjaGFyeQ==?=" , "=?gb18030?B?ZGV2?=" Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_52260C10_0944D010_165657A1" Content-Transfer-Encoding: 8Bit Date: Wed, 4 Sep 2013 00:19:28 +0800 X-Priority: 3 Message-ID: X-QQ-MIME: TCMime 1.0 by Tencent X-Mailer: QQMail 2.x X-QQ-Mailer: QQMail 2.x X-QQ-ReplyHash: 1567667119 X-QQ-SENDSIZE: 520 Subject: [dpdk-dev] =?gb18030?b?u9i4tKO6IERQREsgJiBRUEkgcGVyZm9ybWFuY2Ug?= =?gb18030?q?issue_in_Romley_platform=2E?= X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Sep 2013 16:19:47 -0000 This is a multi-part message in MIME format. ------=_NextPart_52260C10_0944D010_165657A1 Content-Type: text/plain; charset="gb18030" Content-Transfer-Encoding: base64 UVBJIGJhbmR3aWR0aCBpcyBkZWZpbml0ZWx5IGxhcmdlIGVub3VnaCwgYnV0IGl0IHNlZW1z IHRoYXQgUVBJIGlzIG9ubHkgcmVzcG9uc2libGUgZm9yIHRoZSBjb21tdW5pY2F0aW9uIGJl dHdlZW4gc2VwYXJhdGUgQ1BVIGNoaXBzLiBXaGF0IHlvdSBuZWVkIHRvIGRvIGlzIGFjdHVh bGx5IGFjY2Vzc2luZyB0aGUgbWVtb3J5IG9uIHRoZSBvdGhlciBwYXJ0LCBwcm9iYWJseSBu b3QgZXZlbiBoaXQgdGhlIGJhbmR3aWR0aC4gVGhlIGxhdGVuY3kgY2FuIGJlIGNhdXNlZCBi eSBhIGxvdCBvZiBmYWN0cyBkdXJpbmcgYSBOVU1BIG9wZXJhdGlvbi4NCg0KL0JvYg0KDQoN Ci0tLS0tLS0tLS0tLS0tLS0tLSDUrcq808q8/iAtLS0tLS0tLS0tLS0tLS0tLS0NCreivP7I yzogIlphY2hhcnkiOzx6YWNoYXJ5LmplbkBjYXMtd2VsbC5jb20+Ow0Kt6LLzcqxvOQ6IDIw MTPE6jnUwjLI1SjQx8ba0rspINbQzucxMToyMg0KytW8/sjLOiAiZGV2IjxkZXZAZHBkay5v cmc+OyANCrOty806ICIgIllhbm5pYy5DaG91ICjW3NXc1f0pIDogNjgwOCIgPHlhbm5pYy5j aG91QGNhcy13ZWxsLmNvbT47ICJBbGFuIFl1INPh0uCCpSA6IDY2MzIiIjxBbGFuLll1QGNh cy13ZWxsLmNvbT47IA0K1vfM4jogW2RwZGstZGV2XSBEUERLICYgUVBJIHBlcmZvcm1hbmNl IGlzc3VlIGluIFJvbWxleSBwbGF0Zm9ybS4NCg0KDQoNCiBIaX4NCiANCiBJIGhhdmUgYSBx dWVzdGlvbiBhYm91dCBEUERLICYgUVBJIHBlcmZvcm1hbmNlIGlzc3VlIGluIFJvbWxleSAg cGxhdGZvcm0uDQogUmVjZW50bHksIEkgdXNlIERQREsgZXhhbXBsZSwgbDJmd2QsIHRvIHRl c3QgRFBESydzIHBlcmZvcm1hbmNlIGluIG15IFJvbWxleSBwbGF0Zm9ybS4NCiBXaGVuIEkg dHJ5IHRvIGRvIHRoZSB0ZXN0LCBjcm9zc2luZyB1c2VkIENQVSwgSSBmaW5kIHRoZSBwZXJm b3JtYW5jZSBkcmFtYXRpY2FsbHkgZGVjcmVhc2UuDQogSXMgaXQgdHJ1ZT8gT3IgYW55IG1l dGhvZCBjYW4gcHJvdmUgdGhlIHBoZW5vbWVub24/DQogDQogSW4gbXkgb3BpbmlvbiwgdGhl cmUgc2hvdWxkIGJlIG5vIHRoaXMga2luZCBvZiBpc3N1ZSBoZXJlIGR1ZSB0byBRUEkgaGF2 ZSBlbm91Z2ggYmFuZHdpZHRoIHRvIGRlYWwgdGhlIGtpbmRzIG9mIGNhc2UuDQogVGh1cywg SSBhbSBzbyBhbWF6ZSBpbiBvdXIgcmVzdWx0cyBhbmQgY2FuIG5vdCBleHBsYWluIGl0Lg0K IENvdWxkIHNvbWVvbmUgY2FuIGhlbHAgbWUgdG8gc29sdmUgdGhpcyBwcm9ibGVtLg0KIA0K IFRoYW5rIGEgbG90IQ0KIA0KIA0KIE15IHRlc3RpbmcgZW52aXJvbm1lbnQgZGVzY3JpYmUg YXMgYmVsb3c6DQogDQogUGxhdGZvcm06ICAgICAgICAgUm9tbGV5DQogQ1BVOiAgICAgICAg ICAgICAgICBFNS0yNjQzICogMg0KIFJBTTogICAgICAgICAgICAgICBUcmFuc2NlbmQgIDhH QiBQQzMtMTYwMCBERFIzICogOA0KIE9TOiAgICAgICAgICAgICAgICAgRmVkb3JhIGNvcmUg MTQNCiBEUERLOiAgICAgICAgICAgIHYxLjMuMXIyLCBleGFtcGxlL2wyZndkDQogU2xvdCBz ZXR0aW5nOg0KICAgICAgICAgICAgICAgICAgICAgICBTbG90QSBpcyBjb250cm9sZWQgYnkg Q1BVMSBkaXJlY3RseS4NCiAgICAgICAgICAgICAgICAgICAgICAgU2xvdEIgaXMgY29udHJv bGVkIGJ5IENQVTAgZGlyZWN0bHkuDQogDQogRFBESyBwcmUtc2V0dGluZzoNCiBhLiBCSU9T IHNldHRpbmc6DQogICAgIEhUPWRpc2FibGUNCiBiLiBLZXJuZWwgcGFyYW1hdGVycyANCiAg ICAgaXNvbGNwdXM9MiwzLDYsNw0KICAgICBkZWZhdWx0X2h1Z2VwYWdlc3o9MTAyNE0NCiAg ICAgaHVnZXBhZ2Vzej0xMDI0TQ0KICAgICBodWdlcGFnZXM9MTYNCiBjLiBPUyBzZXR0aW5n Og0KICAgICBzZXJ2aWNlIGF2YWhpLWRhZW1vbiBzdG9wDQogICAgIHNlcnZpY2UgTmV0d29y a01hbmFnZXIgc3RvcA0KICAgICBzZXJ2aWNlIGlwdGFibGVzIHN0b3ANCiAgICAgc2Vydmlj ZSBhY3BpZCBzdG9wDQogICAgIHNlbGludXggZGlzYWJsZQ0KIA0KIA0KIEV4YW1wbGUgcHJv Z3JhbSBDb21tYW5kOg0KIGEuIFNsb3RCKENQVTApIC0+IENQVTENCiAgICAgIz4uL2wyZndk IC1jIDB4YyAtbiA0IC0tIC1xIDEgLXAgMHhjDQogDQogYi4gU2xvdEEoQ1BVMSkgLT4gQ1BV MA0KICAgICAjPi4vbDJmd2QgLWMgMHhjMCAtbiA0IC0tIC1xIDEgLXAgMHhjMCANCiANCiBS ZXN1bHRzOg0KICAgICAgdXNlIGZyYW1lIHNpemUgMTI4IGJ5dGVzDQogICAgIA0KQ1BVIEFm ZmluaXR5DQogICANClNsb3QgQSAoQ1BVMSkNCiAgIA0KU2xvdCBCIChDUFUwKQ0KICANCiAg IA0KQ1BVMA0KICAgDQoxNS45JQ0KICAgDQo5Ni40OSUNCiAgDQogICANCkNQVTENCiAgIA0K OTAuODglDQogICANCjI0Ljc4JQ0KICANCiAgIA0KIA0KILG+0MW8/r/JxNyw/LqsyPDs9+uK zaiZQ8Pc2VnTjaOst8fWuLao1q7K1bz+1d+jrNWIzvDKudPDu/K90sK2sb7Qxbz+g8jI3aOs gUvViOROmqe0y9DFvP6hoyBUaGlzIGVtYWlsIG1heSBjb250YWluIGNvbmZpZGVudGlhbCBp bmZvcm1hdGlvbi4gUGxlYXNlIGRvIG5vdCB1c2Ugb3IgZGlzY2xvc2UgaXQgaW4gYW55IHdh eSBhbmQgZGVsZXRlIGl0IGlmIHlvdSBhcmUgbm90IHRoZSBpbnRlbmRlZCByZWNpcGllbnQu ------=_NextPart_52260C10_0944D010_165657A1 Content-Type: text/html; charset="gb18030" Content-Transfer-Encoding: base64 PGRpdj48cCBjbGFzcz0iTXNvTm9ybWFsIj48YSBuYW1lPSJPTEVfTElOSzEiPjxzcGFuIGxh bmc9IkVOLVVTIiBzdHlsZT0iZm9udC1mYW1pbHk6IFZlcmRhbmEsIHNhbnMtc2VyaWY7IGJh Y2tncm91bmQtY29sb3I6IHdoaXRlOyBiYWNrZ3JvdW5kLXBvc2l0aW9uOiBpbml0aWFsIGlu aXRpYWw7IGJhY2tncm91bmQtcmVwZWF0OiBpbml0aWFsIGluaXRpYWw7ICI+UVBJCmJhbmR3 aWR0aCBpcyBkZWZpbml0ZWx5IGxhcmdlIGVub3VnaDwvc3Bhbj4sIGJ1dCBpdCBzZWVtcyB0 aGF0IFFQSSBpcyBvbmx5CnJlc3BvbnNpYmxlIGZvciB0aGUgY29tbXVuaWNhdGlvbiBiZXR3 ZWVuIHNlcGFyYXRlIENQVSBjaGlwcy4gV2hhdCB5b3UgbmVlZCB0bwpkbyBpcyBhY3R1YWxs eSBhY2Nlc3NpbmcgdGhlIG1lbW9yeSBvbiB0aGUgb3RoZXIgcGFydCwgcHJvYmFibHkgbm90 IGV2ZW4gaGl0CnRoZSBiYW5kd2lkdGguIFRoZSBsYXRlbmN5IGNhbiBiZSBjYXVzZWQgYnkg YSBsb3Qgb2YgZmFjdHMgZHVyaW5nIGEgTlVNQSBvcGVyYXRpb24uPC9hPjwvcD48L2Rpdj48 ZGl2PjxkaXY+L0JvYjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtc2l6 ZTogMTJweDtmb250LWZhbWlseTogQXJpYWwgTmFycm93O3BhZGRpbmc6MnB4IDAgMnB4IDA7 Ij4tLS0tLS0tLS0tLS0tLS0tLS0mbmJzcDvUrcq808q8/iZuYnNwOy0tLS0tLS0tLS0tLS0t LS0tLTwvZGl2PjxkaXYgc3R5bGU9ImZvbnQtc2l6ZTogMTJweDtiYWNrZ3JvdW5kOiNlZmVm ZWY7cGFkZGluZzo4cHg7Ij48ZGl2PjxiPreivP7Iyzo8L2I+Jm5ic3A7IlphY2hhcnkiOyZs dDt6YWNoYXJ5LmplbkBjYXMtd2VsbC5jb20mZ3Q7OzwvZGl2PjxkaXY+PGI+t6LLzcqxvOQ6 PC9iPiZuYnNwOzIwMTPE6jnUwjLI1SjQx8ba0rspINbQzucxMToyMjwvZGl2PjxkaXY+PGI+ ytW8/sjLOjwvYj4mbmJzcDsiZGV2IiZsdDtkZXZAZHBkay5vcmcmZ3Q7OyA8d2JyPjwvZGl2 PjxkaXY+PGI+s63LzTo8L2I+Jm5ic3A7IiAiWWFubmljLkNob3UgKNbc1dzV/SkgOiA2ODA4 IiAmbHQ7eWFubmljLmNob3VAY2FzLXdlbGwuY29tJmd0OzsgIkFsYW4gWXUg0+HS4IKlIDog NjYzMiIiJmx0O0FsYW4uWXVAY2FzLXdlbGwuY29tJmd0OzsgPHdicj48L2Rpdj48ZGl2Pjxi Ptb3zOI6PC9iPiZuYnNwO1tkcGRrLWRldl0gRFBESyAmYW1wOyBRUEkgcGVyZm9ybWFuY2Ug aXNzdWUgaW4gUm9tbGV5IHBsYXRmb3JtLjwvZGl2PjwvZGl2PjxkaXY+PGJyPjwvZGl2PgoK CgoKSGl+PGJyPgo8YnI+CkkgaGF2ZSBhIHF1ZXN0aW9uIGFib3V0IERQREsgJmFtcDsgUVBJ IHBlcmZvcm1hbmNlIGlzc3VlIGluIFJvbWxleSZuYnNwOyBwbGF0Zm9ybS48YnI+ClJlY2Vu dGx5LCBJIHVzZSBEUERLIGV4YW1wbGUsIGwyZndkLCB0byB0ZXN0IERQREsncyBwZXJmb3Jt YW5jZSBpbiBteSBSb21sZXkgcGxhdGZvcm0uPGJyPgpXaGVuIEkgdHJ5IHRvIGRvIHRoZSB0 ZXN0LCBjcm9zc2luZyB1c2VkIENQVSwgSSBmaW5kIHRoZSBwZXJmb3JtYW5jZSBkcmFtYXRp Y2FsbHkgZGVjcmVhc2UuPGJyPgpJcyBpdCB0cnVlPyBPciBhbnkgbWV0aG9kIGNhbiBwcm92 ZSB0aGUgcGhlbm9tZW5vbj88YnI+Cjxicj4KSW4gbXkgb3BpbmlvbiwgdGhlcmUgc2hvdWxk IGJlIG5vIHRoaXMga2luZCBvZiBpc3N1ZSBoZXJlIGR1ZSB0byBRUEkgaGF2ZSBlbm91Z2gg YmFuZHdpZHRoIHRvIGRlYWwgdGhlIGtpbmRzIG9mIGNhc2UuPGJyPgpUaHVzLCBJIGFtIHNv IGFtYXplIGluIG91ciByZXN1bHRzIGFuZCBjYW4gbm90IGV4cGxhaW4gaXQuPGJyPgpDb3Vs ZCBzb21lb25lIGNhbiBoZWxwIG1lIHRvIHNvbHZlIHRoaXMgcHJvYmxlbS48YnI+Cjxicj4K VGhhbmsgYSBsb3QhPGJyPgo8YnI+Cjxicj4KTXkgdGVzdGluZyBlbnZpcm9ubWVudCBkZXNj cmliZSBhcyBiZWxvdzo8YnI+Cjxicj4KUGxhdGZvcm06Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7ICZuYnNwOyBSb21sZXk8YnI+CkNQVTombmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz cDsmbmJzcDsgJm5ic3A7IEU1LTI2NDMgKiAyPGJyPgpSQU06Jm5ic3A7Jm5ic3A7Jm5ic3A7 Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7 ICZuYnNwOyBUcmFuc2NlbmQgPHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6 MTAuNXB0OyBmb250LWZhbWlseTpTaW1TdW47IGNvbG9yOmJsYWNrIj4KPGJpZz44R0IgUEMz LTE2MDAgRERSMyAqIDg8L2JpZz48YnI+CjxiaWc+T1M6Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5ic3A7Jm5i c3A7Jm5ic3A7ICZuYnNwOyBGZWRvcmEgY29yZSAxNDwvYmlnPjxicj4KPGJpZz5EUERLOiZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyAm bmJzcDsgdjEuMy4xcjIsIGV4YW1wbGUvbDJmd2Q8YnI+ClNsb3Qgc2V0dGluZzo8YnI+CiZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyAmbmJzcDsgU2xvdEEgaXMgY29udHJvbGVkIGJ5IENQVTEgZGlyZWN0bHkuPC9iaWc+ PC9zcGFuPjxicj4KPHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTAuNXB0 OyBmb250LWZhbWlseTpTaW1TdW47IGNvbG9yOmJsYWNrIj48YmlnPiZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZu YnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyZuYnNwOyAmbmJzcDsg U2xvdEIgaXMgY29udHJvbGVkIGJ5IENQVTAgZGlyZWN0bHkuPGJyPgo8YmlnPjxicj4KPC9i aWc+PC9iaWc+PC9zcGFuPjxiaWc+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNp emU6MTAuNXB0OyBmb250LWZhbWlseTpTaW1TdW47IGNvbG9yOmJsYWNrIj48c3BhbiBsYW5n PSJFTi1VUyIgc3R5bGU9ImNvbG9yOiMwMDk5MDAiPkRQREsgcHJlLXNldHRpbmc6PC9zcGFu PjxzcGFuIGxhbmc9IkVOLVVTIj48YnI+CmEuIEJJT1Mgc2V0dGluZzo8YnI+CiZuYnNwOyAm bmJzcDsgSFQ9ZGlzYWJsZTxicj4KYi4gS2VybmVsIHBhcmFtYXRlcnMgPGJyPgombmJzcDsg Jm5ic3A7IGlzb2xjcHVzPTIsMyw2LDc8YnI+CiZuYnNwOyAmbmJzcDsgZGVmYXVsdF9odWdl cGFnZXN6PTEwMjRNPGJyPgombmJzcDsgJm5ic3A7IGh1Z2VwYWdlc3o9MTAyNE08YnI+CiZu YnNwOyAmbmJzcDsgaHVnZXBhZ2VzPTE2PGJyPgpjLiBPUyBzZXR0aW5nOjxicj4KJm5ic3A7 ICZuYnNwOyBzZXJ2aWNlIGF2YWhpLWRhZW1vbiBzdG9wPGJyPgombmJzcDsgJm5ic3A7IHNl cnZpY2UgTmV0d29ya01hbmFnZXIgc3RvcDxicj4KJm5ic3A7ICZuYnNwOyBzZXJ2aWNlIGlw dGFibGVzIHN0b3A8YnI+CiZuYnNwOyAmbmJzcDsgc2VydmljZSBhY3BpZCBzdG9wPGJyPgom bmJzcDsgJm5ic3A7IHNlbGludXggZGlzYWJsZTxicj4KPGJyPgo8L3NwYW4+PC9zcGFuPjxi cj4KPHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJmb250LXNpemU6MTAuNXB0OyBmb250LWZh bWlseTpTaW1TdW47IGNvbG9yOmJsYWNrIj48c3BhbiBsYW5nPSJFTi1VUyI+PHNwYW4gbGFu Zz0iRU4tVVMiPjxzcGFuIHN0eWxlPSJjb2xvcjojMDA5OTAwIj5FeGFtcGxlIHByb2dyYW0g Q29tbWFuZDwvc3Bhbj46PGJyPgphLiBTbG90QihDUFUwKSAtJmd0OyBDUFUxPGJyPgombmJz cDsgJm5ic3A7ICMmZ3Q7Li9sMmZ3ZCAtYyAweGMgLW4gNCAtLSAtcSAxIC1wIDB4Yzxicj4K PGJyPgpiLiBTbG90QShDUFUxKSAtJmd0OyBDUFUwPGJyPgombmJzcDsgJm5ic3A7ICMmZ3Q7 Li9sMmZ3ZCAtYyAweGMwIC1uIDQgLS0gLXEgMSAtcCAweGMwPC9zcGFuPjwvc3Bhbj4gPGJy Pgo8YnI+Cjwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMC41 cHQ7IGZvbnQtZmFtaWx5OlNpbVN1bjsgY29sb3I6YmxhY2siPjxzcGFuIGxhbmc9IkVOLVVT Ij48c3BhbiBsYW5nPSJFTi1VUyI+PHNwYW4gc3R5bGU9ImNvbG9yOiMwMDk5MDAiPlJlc3Vs dHM8L3NwYW4+Ojwvc3Bhbj48L3NwYW4+PC9zcGFuPjxicj4KJm5ic3A7Jm5ic3A7ICZuYnNw OyA8c21hbGw+dXNlIGZyYW1lIHNpemUgMTI4IGJ5dGVzPC9zbWFsbD48L2JpZz48YnI+Cjx0 YWJsZSBjbGFzcz0iTXNvTm9ybWFsVGFibGUiIGJvcmRlcj0iMCIgY2VsbHBhZGRpbmc9IjAi IGNlbGxzcGFjaW5nPSIwIiBzdHlsZT0ibWFyZ2luLWxlZnQ6MjAuOXB0OyBib3JkZXItY29s bGFwc2U6Y29sbGFwc2UiPgo8dGJvZHk+Cjx0cj4KPHRkIHZhbGlnbj0idG9wIiB3aWR0aD0i MjA3IiBzdHlsZT0id2lkdGg6MTI0LjBwdDsgYm9yZGVyOnNvbGlkIHdpbmRvd3RleHQKICAg ICAgICAgICAgMS4wcHQ7IHBhZGRpbmc6MGNtIDUuNHB0IDBjbSA1LjRwdCI+CjxwIGNsYXNz PSJNc29Ob3JtYWwiIHN0eWxlPSJ0ZXh0LWFsaWduOmp1c3RpZnk7IHRleHQtanVzdGlmeTpp bnRlci1pZGVvZ3JhcGgiPjxzcGFuIHN0eWxlPSIgIDsgOyA7ICI+Q1BVCjwvc3Bhbj48c3Bh biBsYW5nPSJFTi1VUyI+QWZmaW5pdHk8L3NwYW4+PC9wPgo8L3RkPgo8dGQgdmFsaWduPSJ0 b3AiIHdpZHRoPSIyNDIiIHN0eWxlPSJ3aWR0aDoxNDQuOTVwdDsgYm9yZGVyOnNvbGlkIHdp bmRvd3RleHQKICAgICAgICAgICAgMS4wcHQ7IGJvcmRlci1sZWZ0Om5vbmU7IHBhZGRpbmc6 MGNtIDUuNHB0IDBjbSA1LjRwdCI+CjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJ0ZXh0 LWFsaWduOmp1c3RpZnk7IHRleHQtanVzdGlmeTppbnRlci1pZGVvZ3JhcGgiPjxzcGFuIGxh bmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjEwLjVwdDsgY29sb3I6IzAwMjA2MCI+U2xv dCBBIChDUFUxKTwvc3Bhbj48c3BhbiBsYW5nPSJFTi1VUyI+PC9zcGFuPjwvcD4KPC90ZD4K PHRkIHZhbGlnbj0idG9wIiB3aWR0aD0iMjQyIiBzdHlsZT0id2lkdGg6MTQ0Ljk1cHQ7IGJv cmRlcjpzb2xpZCB3aW5kb3d0ZXh0CiAgICAgICAgICAgIDEuMHB0OyBib3JkZXItbGVmdDpu b25lOyBwYWRkaW5nOjBjbSA1LjRwdCAwY20gNS40cHQiPgo8cCBjbGFzcz0iTXNvTm9ybWFs IiBzdHlsZT0idGV4dC1hbGlnbjpqdXN0aWZ5OyB0ZXh0LWp1c3RpZnk6aW50ZXItaWRlb2dy YXBoIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMC41cHQ7IGNvbG9y OiMwMDIwNjAiPlNsb3QgQiAoQ1BVMCk8L3NwYW4+PHNwYW4gbGFuZz0iRU4tVVMiPjwvc3Bh bj48L3A+CjwvdGQ+CjwvdHI+Cjx0cj4KPHRkIHZhbGlnbj0idG9wIiB3aWR0aD0iMjA3IiBz dHlsZT0id2lkdGg6MTI0LjBwdDsgYm9yZGVyOnNvbGlkIHdpbmRvd3RleHQKICAgICAgICAg ICAgMS4wcHQ7IGJvcmRlci10b3A6bm9uZTsgcGFkZGluZzowY20gNS40cHQgMGNtIDUuNHB0 Ij4KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9InRleHQtYWxpZ246anVzdGlmeTsgdGV4 dC1qdXN0aWZ5OmludGVyLWlkZW9ncmFwaCI+PHNwYW4gbGFuZz0iRU4tVVMiIHN0eWxlPSJm b250LXNpemU6MTAuNXB0OyBjb2xvcjojMDAyMDYwIj5DUFUwPC9zcGFuPjxzcGFuIGxhbmc9 IkVOLVVTIj48L3NwYW4+PC9wPgo8L3RkPgo8dGQgdmFsaWduPSJ0b3AiIHdpZHRoPSIyNDIi IHN0eWxlPSJ3aWR0aDoxNDQuOTVwdDsgYm9yZGVyLXRvcDpub25lOyBib3JkZXItbGVmdDpu b25lOyBib3JkZXItYm90dG9tOnNvbGlkCiAgICAgICAgICAgIHdpbmRvd3RleHQgMS4wcHQ7 IGJvcmRlci1yaWdodDpzb2xpZCB3aW5kb3d0ZXh0CiAgICAgICAgICAgIDEuMHB0OyBwYWRk aW5nOjBjbSA1LjRwdCAwY20gNS40cHQiPgo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0i dGV4dC1hbGlnbjpqdXN0aWZ5OyB0ZXh0LWp1c3RpZnk6aW50ZXItaWRlb2dyYXBoIj48c3Bh biBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6ZToxMC41cHQ7IGNvbG9yOiMwMDIwNjAi PjE1LjklPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIj48L3NwYW4+PC9wPgo8L3RkPgo8dGQg dmFsaWduPSJ0b3AiIHdpZHRoPSIyNDIiIHN0eWxlPSJ3aWR0aDoxNDQuOTVwdDsgYm9yZGVy LXRvcDpub25lOyBib3JkZXItbGVmdDpub25lOyBib3JkZXItYm90dG9tOnNvbGlkCiAgICAg ICAgICAgIHdpbmRvd3RleHQgMS4wcHQ7IGJvcmRlci1yaWdodDpzb2xpZCB3aW5kb3d0ZXh0 CiAgICAgICAgICAgIDEuMHB0OyBwYWRkaW5nOjBjbSA1LjRwdCAwY20gNS40cHQiPgo8cCBj bGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0idGV4dC1hbGlnbjpqdXN0aWZ5OyB0ZXh0LWp1c3Rp Znk6aW50ZXItaWRlb2dyYXBoIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9ImZvbnQtc2l6 ZToxMC41cHQ7IGNvbG9yOiMwMDIwNjAiPjk2LjQ5JTwvc3Bhbj48c3BhbiBsYW5nPSJFTi1V UyI+PC9zcGFuPjwvcD4KPC90ZD4KPC90cj4KPHRyPgo8dGQgdmFsaWduPSJ0b3AiIHdpZHRo PSIyMDciIHN0eWxlPSJ3aWR0aDoxMjQuMHB0OyBib3JkZXI6c29saWQgd2luZG93dGV4dAog ICAgICAgICAgICAxLjBwdDsgYm9yZGVyLXRvcDpub25lOyBwYWRkaW5nOjBjbSA1LjRwdCAw Y20gNS40cHQiPgo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0idGV4dC1hbGlnbjpqdXN0 aWZ5OyB0ZXh0LWp1c3RpZnk6aW50ZXItaWRlb2dyYXBoIj48c3BhbiBsYW5nPSJFTi1VUyIg c3R5bGU9ImZvbnQtc2l6ZToxMC41cHQ7IGNvbG9yOiMwMDIwNjAiPkNQVTE8L3NwYW4+PHNw YW4gbGFuZz0iRU4tVVMiPjwvc3Bhbj48L3A+CjwvdGQ+Cjx0ZCB2YWxpZ249InRvcCIgd2lk dGg9IjI0MiIgc3R5bGU9IndpZHRoOjE0NC45NXB0OyBib3JkZXItdG9wOm5vbmU7IGJvcmRl ci1sZWZ0Om5vbmU7IGJvcmRlci1ib3R0b206c29saWQKICAgICAgICAgICAgd2luZG93dGV4 dCAxLjBwdDsgYm9yZGVyLXJpZ2h0OnNvbGlkIHdpbmRvd3RleHQKICAgICAgICAgICAgMS4w cHQ7IHBhZGRpbmc6MGNtIDUuNHB0IDBjbSA1LjRwdCI+CjxwIGNsYXNzPSJNc29Ob3JtYWwi IHN0eWxlPSJ0ZXh0LWFsaWduOmp1c3RpZnk7IHRleHQtanVzdGlmeTppbnRlci1pZGVvZ3Jh cGgiPjxzcGFuIGxhbmc9IkVOLVVTIiBzdHlsZT0iZm9udC1zaXplOjEwLjVwdDsgY29sb3I6 IzAwMjA2MCI+OTAuODglPC9zcGFuPjxzcGFuIGxhbmc9IkVOLVVTIj48L3NwYW4+PC9wPgo8 L3RkPgo8dGQgdmFsaWduPSJ0b3AiIHdpZHRoPSIyNDIiIHN0eWxlPSJ3aWR0aDoxNDQuOTVw dDsgYm9yZGVyLXRvcDpub25lOyBib3JkZXItbGVmdDpub25lOyBib3JkZXItYm90dG9tOnNv bGlkCiAgICAgICAgICAgIHdpbmRvd3RleHQgMS4wcHQ7IGJvcmRlci1yaWdodDpzb2xpZCB3 aW5kb3d0ZXh0CiAgICAgICAgICAgIDEuMHB0OyBwYWRkaW5nOjBjbSA1LjRwdCAwY20gNS40 cHQiPgo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0idGV4dC1hbGlnbjpqdXN0aWZ5OyB0 ZXh0LWp1c3RpZnk6aW50ZXItaWRlb2dyYXBoIj48c3BhbiBsYW5nPSJFTi1VUyIgc3R5bGU9 ImZvbnQtc2l6ZToxMC41cHQ7IGNvbG9yOiMwMDIwNjAiPjI0Ljc4JTwvc3Bhbj48c3BhbiBs YW5nPSJFTi1VUyI+PC9zcGFuPjwvcD4KPC90ZD4KPC90cj4KPC90Ym9keT4KPC90YWJsZT4K PGJyPgo8YnI+CrG+0MW8/r/JxNyw/LqsyPDs9+uKzaiZQ8Pc2VnTjaOst8fWuLao1q7K1bz+ 1d+jrNWIzvDKudPDu/K90sK2sb7Qxbz+g8jI3aOsgUvViOROmqe0y9DFvP6hoyBUaGlzIGVt YWlsIG1heSBjb250YWluIGNvbmZpZGVudGlhbCBpbmZvcm1hdGlvbi4gUGxlYXNlIGRvIG5v dCB1c2Ugb3IgZGlzY2xvc2UgaXQgaW4gYW55IHdheSBhbmQgZGVsZXRlIGl0IGlmIHlvdSBh cmUgbm90IHRoZSBpbnRlbmRlZCByZWNpcGllbnQuPC9kaXY+ ------=_NextPart_52260C10_0944D010_165657A1-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ex1.cas-well.com (unknown [122.147.166.54]) by dpdk.org (Postfix) with ESMTP id AEAF02E89 for ; Fri, 6 Sep 2013 09:31:15 +0200 (CEST) Received: from [172.16.1.178] (122.147.166.57) by ex1.cas-well.com (192.168.200.10) with Microsoft SMTP Server id 14.2.247.3; Fri, 6 Sep 2013 15:34:00 +0800 Message-ID: <522984E2.8020802@cas-well.com> Date: Fri, 6 Sep 2013 15:31:46 +0800 From: Zachary Organization: Cas-Well User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130803 Thunderbird/17.0.8 MIME-Version: 1.0 To: References: <52289F01.7010503@cas-well.com> In-Reply-To: <52289F01.7010503@cas-well.com> X-Forwarded-Message-Id: <52289F01.7010503@cas-well.com> X-Originating-IP: [122.147.166.57] Content-Type: text/plain; charset="GB18030"; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] DPDK & QPI performance issue in Romley platform. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2013 07:31:17 -0000 Hi~ Bob, Thanks for your response! So, you think it is memory usage problem rather than QPI issue? That means if I improve the memory usage issue, may the preformance will ra= ise to my expected? BTW, Have anyone every use DPDK in NUMA and use crossing CPU like my case? If yes, could you tell me how to solve the question? If no, I would to know the DPDK allow user to this kinds of case in their a= pp or not? If the answer is true, I need change a way to use DPDK. above, it has lots of questions. I hope someone can help me to answer the q= uestions. On 09/04/2013 12:19 AM, Bob Chen wrote: QPI bandwidth is definitely large enough, but it seems that QPI is only res= ponsible for the communication between separate CPU chips. What you need to= do is actually accessing the memory on the other part, probably not even h= it the bandwidth. The latency can be caused by a lot of facts during a NUMA= operation. /Bob ------------------ =D4=AD=CA=BC=D3=CA =BC=FE ------------------ =B7=A2=BC=FE=C8=CB: "Zachary";; =B7=A2=CB=CD=CA=B1=BC=E4: 2013=C4=EA9=D4=C22=C8=D5(=D0=C7=C6=DA=D2=BB) =D6= =D0=CE=E711:22 =CA=D5=BC=FE=C8=CB: "dev"; =B3=AD=CB=CD: " "Yannic.Chou (=D6=DC=D5=DC=D5=FD) : 6808"

; "Alan Yu =D3=E1=D2=E0=82=A5 : 6= 632""

; =D6=F7=CC=E2: [dpdk-dev] DPDK & QPI performance issue in Romley platform. Hi~ I have a question about DPDK & QPI performance issue in Romley platform. Recently, I use DPDK example, l2fwd, to test DPDK's performance in my Romle= y platform. When I try to do the test, crossing used CPU, I find the performance dramat= ically decrease. Is it true? Or any method can prove the phenomenon? In my opinion, there should be no this kind of issue here due to QPI have e= nough bandwidth to deal the kinds of case. Thus, I am so amaze in our results and can not explain it. Could someone can help me to solve this problem. Thank a lot! My testing environment describe as below: Platform: Romley CPU: E5-2643 * 2 RAM: Transcend 8GB PC3-1600 DDR3 * 8 OS: Fedora core 14 DPDK: v1.3.1r2, example/l2fwd Slot setting: SlotA is controled by CPU1 directly. SlotB is controled by CPU0 directly. DPDK pre-setting: a. BIOS setting: HT=3Ddisable b. Kernel paramaters isolcpus=3D2,3,6,7 default_hugepagesz=3D1024M hugepagesz=3D1024M hugepages=3D16 c. OS setting: service avahi-daemon stop service NetworkManager stop service iptables stop service acpid stop selinux disable Example program Command: a. SlotB(CPU0) -> CPU1 #>./l2fwd -c 0xc -n 4 -- -q 1 -p 0xc b. SlotA(CPU1) -> CPU0 #>./l2fwd -c 0xc0 -n 4 -- -q 1 -p 0xc0 Results: use frame size 128 bytes CPU Affinity Slot A (CPU1) Slot B (CPU0) CPU0 15.9% 96.49% CPU1 90.88% 24.78% =B1=BE=D0=C5=BC=FE=BF=C9=C4=DC=B0=FC=BA=AC=C8=F0=EC=F7=EB=8A=CD=A8=99C=C3= =DC=D9Y=D3=8D=A3=AC=B7=C7=D6=B8=B6=A8=D6=AE=CA=D5=BC=FE=D5=DF=A3=AC=D5=88= =CE=F0=CA=B9=D3=C3=BB=F2=BD=D2=C2=B6=B1=BE=D0=C5=BC=FE=83=C8=C8=DD=A3=AC=81= K=D5=88=E4N=9A=A7=B4=CB=D0=C5=BC=FE=A1=A3 This email may contain confidenti= al information. Please do not use or disclose it in any way and delete it i= f you are not the intended recipient. -- Best Regards, Zachary Jen Software RD CAS-WELL Inc. 8th Floor, No. 242, Bo-Ai St., Shu-Lin City, Taipei County 238, Taiwan Tel: +886-2-7731-8888#6305 Fax: +886-2-7731-9988 =B1=BE=D0=C5=BC=FE=BF=C9=C4=DC=B0=FC=BA=AC=C8=F0=EC=F7=EB=8A=CD=A8=99C=C3= =DC=D9Y=D3=8D=A3=AC=B7=C7=D6=B8=B6=A8=D6=AE=CA=D5=BC=FE=D5=DF=A3=AC=D5=88= =CE=F0=CA=B9=D3=C3=BB=F2=BD=D2=C2=B6=B1=BE=D0=C5=BC=FE=83=C8=C8=DD=A3=AC=81= K=D5=88=E4N=9A=A7=B4=CB=D0=C5=BC=FE=A1=A3 This email may contain confidenti= al information. Please do not use or disclose it in any way and delete it i= f you are not the intended recipient.