* [dpdk-users] 2VMs with dpdk app-testpmd effect each other @ 2016-09-01 0:47 鄭凱懋 2016-09-01 1:35 ` Xu, Qian Q 0 siblings, 1 reply; 6+ messages in thread From: 鄭凱懋 @ 2016-09-01 0:47 UTC (permalink / raw) To: users Hi all I am facing a wierd situation when i start testpmd app inside two VMs( Case 3 ). Please see the following setup. Hardware Information System Dell Inc. PowerEdge R630 BIOS Dell Inc. version1.0.4 CPU Intel(R) Xeon(R) x86_64 E5-2630 v3 2.40 Ghz Processor(s): 16 Cache: 20480 KB Memory Size: 16 GB*6 Type: DDR4 Speed: 2133MHz Ethernet Controller Intel Corporation Ethernet 10G 2P X520 Adapter *2 Software information Qemu = qemu-2.5.1.1 DPDK = dpdk-2.2.0 OVS = ovs-branch-2.5 Input stream : 10Gbps(5G with vlan 1001 and 5G with vlan 1002) which is generated by Spirent testcenter | VM1 | | VM2 | ↑1001↓ ↑1002↓ ----------------------- | OVS | ----------------------- ↑ ↓ ----------------------- TestCenter | ----------------------- ovs setup: modprobe openvswitch rm -rf /usr/local/var/run/openvswitch/* rm -rf /opt/ovs-vswitchd.log modprobe uio insmod $DPDK_BUILD/kmod/igb_uio.ko $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p1 $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p2 $DPDK_DIR/tools/dpdk_nic_bind.py --status rm /usr/local/etc/openvswitch/conf.db -rf mkdir -p /usr/local/etc/openvswitch mkdir -p /usr/local/var/run/openvswitch cd $OVS_DIR ./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db ./vswitchd/vswitch.ovsschema ./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach ./utilities/ovs-vsctl --no-wait init ./utilities/ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true ./vswitchd/ovs-vswitchd --dpdk --socket-mem 4096,4096 -n 4 -- unix:/usr/local/var/run/openvswitch/db.sock --log-file=$OVS_LOG --pidfile --detach Case 1: VM1 start testpmd, fwd = iofwd VM2 start testpmd,fwd = iofwd Result = Spirent testcenter show output stream is reaching line speed 10G. Case 2: VM1 testpmd = started, fwd = iofwd VM2 testpmd=stopped,fwd = none Result = Spirent testcenter show output stream is reaching 5G. Which is reasonable. Case 3: VM1 testpmd = started, fwd = iofwd VM2 testpmd = started ,fwd = none Result = Spirent testcenter show output stream is less than 200Mbps. I am confused about the case 3. What is the reason VM2 effects VM1 so significantly? How can i solve such problem? I will appreciate for your answer. Thx ============================ 鄭凱懋 Kai-Mao Cheng 中華電信研究院 寬頻網路研究所 231計畫 電話:03-4244093 手機:0912314847 E-mail:kmcheng@cht.com.tw ============================ Please be advised that this email message (including any attachments) contains confidential information and may be legally privileged. If you are not the intended recipient, please destroy this message and all attachments from your system and do not further collect, process, or use them. Chunghwa Telecom and all its subsidiaries and associated companies shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. If you are the intended recipient, please protect the confidential and/or personal information contained in this email with due care. Any unauthorized use, disclosure or distribution of this message in whole or in part is strictly prohibited. Also, please self-inspect attachments and hyperlinks contained in this email to ensure the information security and to protect personal information. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] 2VMs with dpdk app-testpmd effect each other 2016-09-01 0:47 [dpdk-users] 2VMs with dpdk app-testpmd effect each other 鄭凱懋 @ 2016-09-01 1:35 ` Xu, Qian Q 2016-09-02 5:37 ` 鄭凱懋 0 siblings, 1 reply; 6+ messages in thread From: Xu, Qian Q @ 2016-09-01 1:35 UTC (permalink / raw) To: kmcheng, users Maybe VM1 and VM2 are using the same core for the forwarding? I guess 2VMs are using the same core for fwd. What do you mean by testpmd fwd=none, if you don't set the fwd mode, it's IO fwd. -----Original Message----- From: users [mailto:users-bounces@dpdk.org] On Behalf Of ??? Sent: Thursday, September 1, 2016 8:47 AM To: users@dpdk.org Subject: [dpdk-users] 2VMs with dpdk app-testpmd effect each other Hi all I am facing a wierd situation when i start testpmd app inside two VMs( Case 3 ). Please see the following setup. Hardware Information System Dell Inc. PowerEdge R630 BIOS Dell Inc. version1.0.4 CPU Intel(R) Xeon(R) x86_64 E5-2630 v3 2.40 Ghz Processor(s): 16 Cache: 20480 KB Memory Size: 16 GB*6 Type: DDR4 Speed: 2133MHz Ethernet Controller Intel Corporation Ethernet 10G 2P X520 Adapter *2 Software information Qemu = qemu-2.5.1.1 DPDK = dpdk-2.2.0 OVS = ovs-branch-2.5 Input stream : 10Gbps(5G with vlan 1001 and 5G with vlan 1002) which is generated by Spirent testcenter | VM1 | | VM2 | ↑1001↓ ↑1002↓ ----------------------- | OVS | ----------------------- ↑ ↓ ----------------------- TestCenter | ----------------------- ovs setup: modprobe openvswitch rm -rf /usr/local/var/run/openvswitch/* rm -rf /opt/ovs-vswitchd.log modprobe uio insmod $DPDK_BUILD/kmod/igb_uio.ko $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p1 $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p2 $DPDK_DIR/tools/dpdk_nic_bind.py --status rm /usr/local/etc/openvswitch/conf.db -rf mkdir -p /usr/local/etc/openvswitch mkdir -p /usr/local/var/run/openvswitch cd $OVS_DIR ./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db ./vswitchd/vswitch.ovsschema ./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach ./utilities/ovs-vsctl --no-wait init ./utilities/ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true ./vswitchd/ovs-vswitchd --dpdk --socket-mem 4096,4096 -n 4 -- unix:/usr/local/var/run/openvswitch/db.sock --log-file=$OVS_LOG --pidfile --detach Case 1: VM1 start testpmd, fwd = iofwd VM2 start testpmd,fwd = iofwd Result = Spirent testcenter show output stream is reaching line speed 10G. Case 2: VM1 testpmd = started, fwd = iofwd VM2 testpmd=stopped,fwd = none Result = Spirent testcenter show output stream is reaching 5G. Which is reasonable. Case 3: VM1 testpmd = started, fwd = iofwd VM2 testpmd = started ,fwd = none Result = Spirent testcenter show output stream is less than 200Mbps. I am confused about the case 3. What is the reason VM2 effects VM1 so significantly? How can i solve such problem? I will appreciate for your answer. Thx ============================ 鄭凱懋 Kai-Mao Cheng 中華電信研究院 𥶡頻網路研究所 231計畫 電話:03-4244093 手機:0912314847 E-mail:kmcheng@cht.com.tw ============================ Please be advised that this email message (including any attachments) contains confidential information and may be legally privileged. If you are not the intended recipient, please destroy this message and all attachments from your system and do not further collect, process, or use them. Chunghwa Telecom and all its subsidiaries and associated companies shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. If you are the intended recipient, please protect the confidential and/or personal information contained in this email with due care. Any unauthorized use, disclosure or distribution of this message in whole or in part is strictly prohibited. Also, please self-inspect attachments and hyperlinks contained in this email to ensure the information security and to protect personal information. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] 2VMs with dpdk app-testpmd effect each other 2016-09-01 1:35 ` Xu, Qian Q @ 2016-09-02 5:37 ` 鄭凱懋 2016-09-07 2:03 ` 鄭凱懋 0 siblings, 1 reply; 6+ messages in thread From: 鄭凱懋 @ 2016-09-02 5:37 UTC (permalink / raw) To: Xu, Qian Q, users I am sure that VM1 and VM2 are using different core with different coremask setting. Yes, I know that default fwd enging is IO fwd. And fwd engine will start while testpmd is started. I used "stop" command to stop fwd engine inside testpmd. The I can see effect ion on the other VM. -----Original Message----- From: Xu, Qian Q [mailto:qian.q.xu@intel.com] Sent: Thursday, September 01, 2016 9:36 AM To: 鄭凱懋; users@dpdk.org Subject: RE: 2VMs with dpdk app-testpmd effect each other Maybe VM1 and VM2 are using the same core for the forwarding? I guess 2VMs are using the same core for fwd. What do you mean by testpmd fwd=none, if you don't set the fwd mode, it's IO fwd. -----Original Message----- From: users [mailto:users-bounces@dpdk.org] On Behalf Of ??? Sent: Thursday, September 1, 2016 8:47 AM To: users@dpdk.org Subject: [dpdk-users] 2VMs with dpdk app-testpmd effect each other Hi all I am facing a wierd situation when i start testpmd app inside two VMs( Case 3 ). Please see the following setup. Hardware Information System Dell Inc. PowerEdge R630 BIOS Dell Inc. version1.0.4 CPU Intel(R) Xeon(R) x86_64 E5-2630 v3 2.40 Ghz Processor(s): 16 Cache: 20480 KB Memory Size: 16 GB*6 Type: DDR4 Speed: 2133MHz Ethernet Controller Intel Corporation Ethernet 10G 2P X520 Adapter *2 Software information Qemu = qemu-2.5.1.1 DPDK = dpdk-2.2.0 OVS = ovs-branch-2.5 Input stream : 10Gbps(5G with vlan 1001 and 5G with vlan 1002) which is generated by Spirent testcenter | VM1 | | VM2 | ↑1001↓ ↑1002↓ ----------------------- | OVS | ----------------------- ↑ ↓ ----------------------- TestCenter | ----------------------- ovs setup: modprobe openvswitch rm -rf /usr/local/var/run/openvswitch/* rm -rf /opt/ovs-vswitchd.log modprobe uio insmod $DPDK_BUILD/kmod/igb_uio.ko $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p1 $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p2 $DPDK_DIR/tools/dpdk_nic_bind.py --status rm /usr/local/etc/openvswitch/conf.db -rf mkdir -p /usr/local/etc/openvswitch mkdir -p /usr/local/var/run/openvswitch cd $OVS_DIR ./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db ./vswitchd/vswitch.ovsschema ./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach ./utilities/ovs-vsctl --no-wait init ./utilities/ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true ./vswitchd/ovs-vswitchd --dpdk --socket-mem 4096,4096 -n 4 -- unix:/usr/local/var/run/openvswitch/db.sock --log-file=$OVS_LOG --pidfile --detach Case 1: VM1 start testpmd, fwd = iofwd VM2 start testpmd,fwd = iofwd Result = Spirent testcenter show output stream is reaching line speed 10G. Case 2: VM1 testpmd = started, fwd = iofwd VM2 testpmd=stopped,fwd = none Result = Spirent testcenter show output stream is reaching 5G. Which is reasonable. Case 3: VM1 testpmd = started, fwd = iofwd VM2 testpmd = started ,fwd = none Result = Spirent testcenter show output stream is less than 200Mbps. I am confused about the case 3. What is the reason VM2 effects VM1 so significantly? How can i solve such problem? I will appreciate for your answer. Thx ============================ 鄭凱懋 Kai-Mao Cheng 中華電信研究院 寬頻網路研究所 231計畫 電話:03-4244093 手機:0912314847 E-mail:kmcheng@cht.com.tw ============================ Please be advised that this email message (including any attachments) contains confidential information and may be legally privileged. If you are not the intended recipient, please destroy this message and all attachments from your system and do not further collect, process, or use them. Chunghwa Telecom and all its subsidiaries and associated companies shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. If you are the intended recipient, please protect the confidential and/or personal information contained in this email with due care. Any unauthorized use, disclosure or distribution of this message in whole or in part is strictly prohibited. Also, please self-inspect attachments and hyperlinks contained in this email to ensure the information security and to protect personal information. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] 2VMs with dpdk app-testpmd effect each other 2016-09-02 5:37 ` 鄭凱懋 @ 2016-09-07 2:03 ` 鄭凱懋 2016-09-07 2:11 ` Xu, Qian Q 0 siblings, 1 reply; 6+ messages in thread From: 鄭凱懋 @ 2016-09-07 2:03 UTC (permalink / raw) To: Xu, Qian Q, users So my question is : is it normal ? Is there any way i can solve the problem? Is the problem related to qemu-img? ________________________________________ 從: 鄭凱懋 寄件日期: 2016年9月2日 下午 01:37 至: Xu, Qian Q; users@dpdk.org 主旨: RE: 2VMs with dpdk app-testpmd effect each other I am sure that VM1 and VM2 are using different core with different coremask setting. Yes, I know that default fwd enging is IO fwd. And fwd engine will start while testpmd is started. I used "stop" command to stop fwd engine inside testpmd. The I can see effect ion on the other VM. -----Original Message----- From: Xu, Qian Q [mailto:qian.q.xu@intel.com] Sent: Thursday, September 01, 2016 9:36 AM To: 鄭凱懋; users@dpdk.org Subject: RE: 2VMs with dpdk app-testpmd effect each other Maybe VM1 and VM2 are using the same core for the forwarding? I guess 2VMs are using the same core for fwd. What do you mean by testpmd fwd=none, if you don't set the fwd mode, it's IO fwd. -----Original Message----- From: users [mailto:users-bounces@dpdk.org] On Behalf Of ??? Sent: Thursday, September 1, 2016 8:47 AM To: users@dpdk.org Subject: [dpdk-users] 2VMs with dpdk app-testpmd effect each other Hi all I am facing a wierd situation when i start testpmd app inside two VMs( Case 3 ). Please see the following setup. Hardware Information System Dell Inc. PowerEdge R630 BIOS Dell Inc. version1.0.4 CPU Intel(R) Xeon(R) x86_64 E5-2630 v3 2.40 Ghz Processor(s): 16 Cache: 20480 KB Memory Size: 16 GB*6 Type: DDR4 Speed: 2133MHz Ethernet Controller Intel Corporation Ethernet 10G 2P X520 Adapter *2 Software information Qemu = qemu-2.5.1.1 DPDK = dpdk-2.2.0 OVS = ovs-branch-2.5 Input stream : 10Gbps(5G with vlan 1001 and 5G with vlan 1002) which is generated by Spirent testcenter | VM1 | | VM2 | ↑1001↓ ↑1002↓ ----------------------- | OVS | ----------------------- ↑ ↓ ----------------------- TestCenter | ----------------------- ovs setup: modprobe openvswitch rm -rf /usr/local/var/run/openvswitch/* rm -rf /opt/ovs-vswitchd.log modprobe uio insmod $DPDK_BUILD/kmod/igb_uio.ko $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p1 $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p2 $DPDK_DIR/tools/dpdk_nic_bind.py --status rm /usr/local/etc/openvswitch/conf.db -rf mkdir -p /usr/local/etc/openvswitch mkdir -p /usr/local/var/run/openvswitch cd $OVS_DIR ./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db ./vswitchd/vswitch.ovsschema ./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach ./utilities/ovs-vsctl --no-wait init ./utilities/ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true ./vswitchd/ovs-vswitchd --dpdk --socket-mem 4096,4096 -n 4 -- unix:/usr/local/var/run/openvswitch/db.sock --log-file=$OVS_LOG --pidfile --detach Case 1: VM1 start testpmd, fwd = iofwd VM2 start testpmd,fwd = iofwd Result = Spirent testcenter show output stream is reaching line speed 10G. Case 2: VM1 testpmd = started, fwd = iofwd VM2 testpmd=stopped,fwd = none Result = Spirent testcenter show output stream is reaching 5G. Which is reasonable. Case 3: VM1 testpmd = started, fwd = iofwd VM2 testpmd = started ,fwd = none Result = Spirent testcenter show output stream is less than 200Mbps. I am confused about the case 3. What is the reason VM2 effects VM1 so significantly? How can i solve such problem? I will appreciate for your answer. Thx ============================ 鄭凱懋 Kai-Mao Cheng 中華電信研究院 寬頻網路研究所 231計畫 電話:03-4244093 手機:0912314847 E-mail:kmcheng@cht.com.tw ============================ Please be advised that this email message (including any attachments) contains confidential information and may be legally privileged. If you are not the intended recipient, please destroy this message and all attachments from your system and do not further collect, process, or use them. Chunghwa Telecom and all its subsidiaries and associated companies shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. If you are the intended recipient, please protect the confidential and/or personal information contained in this email with due care. Any unauthorized use, disclosure or distribution of this message in whole or in part is strictly prohibited. Also, please self-inspect attachments and hyperlinks contained in this email to ensure the information security and to protect personal information. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] 2VMs with dpdk app-testpmd effect each other 2016-09-07 2:03 ` 鄭凱懋 @ 2016-09-07 2:11 ` Xu, Qian Q 2016-09-09 8:18 ` 鄭凱懋 0 siblings, 1 reply; 6+ messages in thread From: Xu, Qian Q @ 2016-09-07 2:11 UTC (permalink / raw) To: kmcheng, users It's not normal, and I think first we need figure out it's DPDK issue or OVS issue or some configuration issue. Are you using 2 10G port to inject traffic? What's your qemu launch commands for each VM? Can you use "top -H" command to check the core usage when u running the case2 and case3? -----Original Message----- From: 鄭凱懋 [mailto:kmcheng@cht.com.tw] Sent: Wednesday, September 7, 2016 10:04 AM To: Xu, Qian Q <qian.q.xu@intel.com>; users@dpdk.org Subject: RE: 2VMs with dpdk app-testpmd effect each other So my question is : is it normal ? Is there any way i can solve the problem? Is the problem related to qemu-img? ________________________________________ 從: 鄭凱懋 寄件日期: 2016年9月2日 下午 01:37 至: Xu, Qian Q; users@dpdk.org 主旨: RE: 2VMs with dpdk app-testpmd effect each other I am sure that VM1 and VM2 are using different core with different coremask setting. Yes, I know that default fwd enging is IO fwd. And fwd engine will start while testpmd is started. I used "stop" command to stop fwd engine inside testpmd. The I can see effect ion on the other VM. -----Original Message----- From: Xu, Qian Q [mailto:qian.q.xu@intel.com] Sent: Thursday, September 01, 2016 9:36 AM To: 鄭凱懋; users@dpdk.org Subject: RE: 2VMs with dpdk app-testpmd effect each other Maybe VM1 and VM2 are using the same core for the forwarding? I guess 2VMs are using the same core for fwd. What do you mean by testpmd fwd=none, if you don't set the fwd mode, it's IO fwd. -----Original Message----- From: users [mailto:users-bounces@dpdk.org] On Behalf Of ??? Sent: Thursday, September 1, 2016 8:47 AM To: users@dpdk.org Subject: [dpdk-users] 2VMs with dpdk app-testpmd effect each other Hi all I am facing a wierd situation when i start testpmd app inside two VMs( Case 3 ). Please see the following setup. Hardware Information System Dell Inc. PowerEdge R630 BIOS Dell Inc. version1.0.4 CPU Intel(R) Xeon(R) x86_64 E5-2630 v3 2.40 Ghz Processor(s): 16 Cache: 20480 KB Memory Size: 16 GB*6 Type: DDR4 Speed: 2133MHz Ethernet Controller Intel Corporation Ethernet 10G 2P X520 Adapter *2 Software information Qemu = qemu-2.5.1.1 DPDK = dpdk-2.2.0 OVS = ovs-branch-2.5 Input stream : 10Gbps(5G with vlan 1001 and 5G with vlan 1002) which is generated by Spirent testcenter | VM1 | | VM2 | ↑1001↓ ↑1002↓ ----------------------- | OVS | ----------------------- ↑ ↓ ----------------------- TestCenter | ----------------------- ovs setup: modprobe openvswitch rm -rf /usr/local/var/run/openvswitch/* rm -rf /opt/ovs-vswitchd.log modprobe uio insmod $DPDK_BUILD/kmod/igb_uio.ko $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p1 $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p2 $DPDK_DIR/tools/dpdk_nic_bind.py --status rm /usr/local/etc/openvswitch/conf.db -rf mkdir -p /usr/local/etc/openvswitch mkdir -p /usr/local/var/run/openvswitch cd $OVS_DIR ./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db ./vswitchd/vswitch.ovsschema ./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach ./utilities/ovs-vsctl --no-wait init ./utilities/ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true ./vswitchd/ovs-vswitchd --dpdk --socket-mem 4096,4096 -n 4 -- unix:/usr/local/var/run/openvswitch/db.sock --log-file=$OVS_LOG --pidfile --detach Case 1: VM1 start testpmd, fwd = iofwd VM2 start testpmd,fwd = iofwd Result = Spirent testcenter show output stream is reaching line speed 10G. Case 2: VM1 testpmd = started, fwd = iofwd VM2 testpmd=stopped,fwd = none Result = Spirent testcenter show output stream is reaching 5G. Which is reasonable. Case 3: VM1 testpmd = started, fwd = iofwd VM2 testpmd = started ,fwd = none Result = Spirent testcenter show output stream is less than 200Mbps. I am confused about the case 3. What is the reason VM2 effects VM1 so significantly? How can i solve such problem? I will appreciate for your answer. Thx ============================ 鄭凱懋 Kai-Mao Cheng 中華電信研究院 𥶡頻網路研究所 231計畫 電話:03-4244093 手機:0912314847 E-mail:kmcheng@cht.com.tw ============================ Please be advised that this email message (including any attachments) contains confidential information and may be legally privileged. If you are not the intended recipient, please destroy this message and all attachments from your system and do not further collect, process, or use them. Chunghwa Telecom and all its subsidiaries and associated companies shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. If you are the intended recipient, please protect the confidential and/or personal information contained in this email with due care. Any unauthorized use, disclosure or distribution of this message in whole or in part is strictly prohibited. Also, please self-inspect attachments and hyperlinks contained in this email to ensure the information security and to protect personal information. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dpdk-users] 2VMs with dpdk app-testpmd effect each other 2016-09-07 2:11 ` Xu, Qian Q @ 2016-09-09 8:18 ` 鄭凱懋 0 siblings, 0 replies; 6+ messages in thread From: 鄭凱懋 @ 2016-09-09 8:18 UTC (permalink / raw) To: Xu, Qian Q, users 1.Are you using 2 10G port to inject traffic? Ans: No. one for in, one for out. i used opeflow rule to separate the incoming stream into two VMs. 2.What's your qemu launch commands for each VM? Ans: taskset -c 8,10 ./qemu-2.5.1.1/x86_64-softmmu/qemu-system-x86_64 -cpu host -enable-kvm -m 4096M -smp 4 -nographic -k en-us -vnc :1 -pidfile /usr/local/var/run/openvswitch/vm1.pid,snapshot=off -monitor unix:/usr/local/var/run/openvswitch/vm1monitor,server,nowait\ -object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc \ -chardev socket,id=char0,path=/usr/local/var/run/openvswitch/vhost-user-1 \ -netdev type=vhost-user,id=mynet1,chardev=char0,vhostforce \ -device virtio-net-pci,netdev=mynet1,mac=52:54:00:02:d9:01 \ -chardev socket,id=char1,path=/usr/local/var/run/openvswitch/vhost-user-2 \ -netdev type=vhost-user,id=mynet2,chardev=char1,vhostforce \ -device virtio-net-pci,netdev=mynet2,mac=52:54:00:02:d9:02 \ vm1.qcow2 taskset -c 12,14 ./qemu-2.5.1.1/x86_64-softmmu/qemu-system-x86_64 -cpu host -enable-kvm -m 4096M -smp 4 -nographic -k en-us -vnc :2 -pidfile /usr/local/var/run/openvswitch/vm2.pid,snapshot=off -monitor unix:/usr/local/var/run/openvswitch/vm1monitor,server,nowait\ -object memory-backend-file,id=mem,size=4096M,mem-path=/dev/hugepages,share=on -numa node,memdev=mem -mem-prealloc \ -chardev socket,id=char2,path=/usr/local/var/run/openvswitch/vhost-user-3 \ -netdev type=vhost-user,id=mynet3,chardev=char2,vhostforce \ -device virtio-net-pci,netdev=mynet3,mac=52:54:00:02:d9:03 \ -chardev socket,id=char3,path=/usr/local/var/run/openvswitch/vhost-user-4 \ -netdev type=vhost-user,id=mynet4,chardev=char3,vhostforce \ -device virtio-net-pci,netdev=mynet4,mac=52:54:00:02:d9:04 \ vm2.qcow2 3.Can you use "top -H" command to check the core usage when u running the case2 and case3? Ans: Case2: 105561 root 20 0 41.660g 27388 2300 R 99.9 0.0 1487:40 pmd39 105562 root 20 0 41.660g 27388 2300 R 99.9 0.0 1488:13 pmd38 105563 root 20 0 41.660g 27388 2300 R 99.9 0.0 1488:12 pmd37 105564 root 20 0 41.660g 27388 2300 R 99.9 0.0 1488:13 pmd36 119509 root 20 0 4929700 57736 5296 R 99.9 0.1 7:01.16 qemu-system-x86 Case3: 119509 root 20 0 4929700 57736 5296 R 99.9 0.1 8:44.03 qemu-system-x86 105561 root 20 0 41.660g 27388 2300 R 99.9 0.0 1489:23 pmd39 105562 root 20 0 41.660g 27388 2300 R 99.9 0.0 1489:55 pmd38 105563 root 20 0 41.660g 27388 2300 R 99.9 0.0 1489:55 pmd37 105564 root 20 0 41.660g 27388 2300 R 99.9 0.0 1489:55 pmd36 case2 and case 3 seems nothing different~~ -----Original Message----- From: Xu, Qian Q [mailto:qian.q.xu@intel.com] Sent: Wednesday, September 07, 2016 10:11 AM To: 鄭凱懋; users@dpdk.org Subject: RE: 2VMs with dpdk app-testpmd effect each other It's not normal, and I think first we need figure out it's DPDK issue or OVS issue or some configuration issue. Are you using 2 10G port to inject traffic? What's your qemu launch commands for each VM? Can you use "top -H" command to check the core usage when u running the case2 and case3? -----Original Message----- From: 鄭凱懋 [mailto:kmcheng@cht.com.tw] Sent: Wednesday, September 7, 2016 10:04 AM To: Xu, Qian Q <qian.q.xu@intel.com>; users@dpdk.org Subject: RE: 2VMs with dpdk app-testpmd effect each other So my question is : is it normal ? Is there any way i can solve the problem? Is the problem related to qemu-img? ________________________________________ 從: 鄭凱懋 寄件日期: 2016年9月2日 下午 01:37 至: Xu, Qian Q; users@dpdk.org 主旨: RE: 2VMs with dpdk app-testpmd effect each other I am sure that VM1 and VM2 are using different core with different coremask setting. Yes, I know that default fwd enging is IO fwd. And fwd engine will start while testpmd is started. I used "stop" command to stop fwd engine inside testpmd. The I can see effect ion on the other VM. -----Original Message----- From: Xu, Qian Q [mailto:qian.q.xu@intel.com] Sent: Thursday, September 01, 2016 9:36 AM To: 鄭凱懋; users@dpdk.org Subject: RE: 2VMs with dpdk app-testpmd effect each other Maybe VM1 and VM2 are using the same core for the forwarding? I guess 2VMs are using the same core for fwd. What do you mean by testpmd fwd=none, if you don't set the fwd mode, it's IO fwd. -----Original Message----- From: users [mailto:users-bounces@dpdk.org] On Behalf Of ??? Sent: Thursday, September 1, 2016 8:47 AM To: users@dpdk.org Subject: [dpdk-users] 2VMs with dpdk app-testpmd effect each other Hi all I am facing a wierd situation when i start testpmd app inside two VMs( Case 3 ). Please see the following setup. Hardware Information System Dell Inc. PowerEdge R630 BIOS Dell Inc. version1.0.4 CPU Intel(R) Xeon(R) x86_64 E5-2630 v3 2.40 Ghz Processor(s): 16 Cache: 20480 KB Memory Size: 16 GB*6 Type: DDR4 Speed: 2133MHz Ethernet Controller Intel Corporation Ethernet 10G 2P X520 Adapter *2 Software information Qemu = qemu-2.5.1.1 DPDK = dpdk-2.2.0 OVS = ovs-branch-2.5 Input stream : 10Gbps(5G with vlan 1001 and 5G with vlan 1002) which is generated by Spirent testcenter | VM1 | | VM2 | ↑1001↓ ↑1002↓ ----------------------- | OVS | ----------------------- ↑ ↓ ----------------------- TestCenter | ----------------------- ovs setup: modprobe openvswitch rm -rf /usr/local/var/run/openvswitch/* rm -rf /opt/ovs-vswitchd.log modprobe uio insmod $DPDK_BUILD/kmod/igb_uio.ko $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p1 $DPDK_DIR/tools/dpdk_nic_bind.py --bind=igb_uio p3p2 $DPDK_DIR/tools/dpdk_nic_bind.py --status rm /usr/local/etc/openvswitch/conf.db -rf mkdir -p /usr/local/etc/openvswitch mkdir -p /usr/local/var/run/openvswitch cd $OVS_DIR ./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db ./vswitchd/vswitch.ovsschema ./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_vSwitch,manager_options --private-key=db:Open_vSwitch,SSL,private_key --certificate=db:Open_vSwitch,SSL,certificate --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach ./utilities/ovs-vsctl --no-wait init ./utilities/ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true ./vswitchd/ovs-vswitchd --dpdk --socket-mem 4096,4096 -n 4 -- unix:/usr/local/var/run/openvswitch/db.sock --log-file=$OVS_LOG --pidfile --detach Case 1: VM1 start testpmd, fwd = iofwd VM2 start testpmd,fwd = iofwd Result = Spirent testcenter show output stream is reaching line speed 10G. Case 2: VM1 testpmd = started, fwd = iofwd VM2 testpmd=stopped,fwd = none Result = Spirent testcenter show output stream is reaching 5G. Which is reasonable. Case 3: VM1 testpmd = started, fwd = iofwd VM2 testpmd = started ,fwd = none Result = Spirent testcenter show output stream is less than 200Mbps. I am confused about the case 3. What is the reason VM2 effects VM1 so significantly? How can i solve such problem? I will appreciate for your answer. Thx ============================ 鄭凱懋 Kai-Mao Cheng 中華電信研究院 寬頻網路研究所 231計畫 電話:03-4244093 手機:0912314847 E-mail:kmcheng@cht.com.tw ============================ Please be advised that this email message (including any attachments) contains confidential information and may be legally privileged. If you are not the intended recipient, please destroy this message and all attachments from your system and do not further collect, process, or use them. Chunghwa Telecom and all its subsidiaries and associated companies shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. If you are the intended recipient, please protect the confidential and/or personal information contained in this email with due care. Any unauthorized use, disclosure or distribution of this message in whole or in part is strictly prohibited. Also, please self-inspect attachments and hyperlinks contained in this email to ensure the information security and to protect personal information. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-09-09 8:18 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-09-01 0:47 [dpdk-users] 2VMs with dpdk app-testpmd effect each other 鄭凱懋 2016-09-01 1:35 ` Xu, Qian Q 2016-09-02 5:37 ` 鄭凱懋 2016-09-07 2:03 ` 鄭凱懋 2016-09-07 2:11 ` Xu, Qian Q 2016-09-09 8:18 ` 鄭凱懋
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).