* [dts] [PATCH V1]userspace_ethtool: update test script and test plan @ 2016-05-05 2:59 Yufen Mo 2016-05-05 2:59 ` Yufen Mo 0 siblings, 1 reply; 6+ messages in thread From: Yufen Mo @ 2016-05-05 2:59 UTC (permalink / raw) To: dts; +Cc: yufengmx From: yufengmx <yufengx.mo@intel.com> yufengmx (1): userspace_ethtool: update test script and test plan test_plans/userspace_ethtool_test_plan.rst | 167 ++++++++++++++++++----------- tests/TestSuite_userspace_ethtool.py | 85 +++++++++------ 2 files changed, 156 insertions(+), 96 deletions(-) -- 1.9.3 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [dts] [PATCH V1]userspace_ethtool: update test script and test plan 2016-05-05 2:59 [dts] [PATCH V1]userspace_ethtool: update test script and test plan Yufen Mo @ 2016-05-05 2:59 ` Yufen Mo 2016-05-23 20:29 ` Liu, Yong 0 siblings, 1 reply; 6+ messages in thread From: Yufen Mo @ 2016-05-05 2:59 UTC (permalink / raw) To: dts; +Cc: yufengmx From: yufengmx <yufengx.mo@intel.com> change vlan packet check number when doing vlan testing. Add process to resize linux ethtool eeprom dump data size as the size dump by dpdk ethtool. Ignore auto test link connect status on FVL. update related test cases' test plan, add testing configuration and requirements. Signed-off-by: yufengmx <yufengx.mo@intel.com> --- test_plans/userspace_ethtool_test_plan.rst | 167 ++++++++++++++++++----------- tests/TestSuite_userspace_ethtool.py | 85 +++++++++------ 2 files changed, 156 insertions(+), 96 deletions(-) diff --git a/test_plans/userspace_ethtool_test_plan.rst b/test_plans/userspace_ethtool_test_plan.rst index 5ee79a1..83e7fac 100644 --- a/test_plans/userspace_ethtool_test_plan.rst +++ b/test_plans/userspace_ethtool_test_plan.rst @@ -34,31 +34,42 @@ Userspace Ethtool ================== -This feature is designed to provide one rte_ethtool shim layer based on -rte_ethdev API. The implementation also along with a command prompt driven -demonstration application. It only contained of 18 popular used Ethtool and -Netdevice ops as described in rte_ethtool.h. +This feature is designed to provide one rte_ethtool shim layer based on +rte_ethdev API. The Ethtool sample application shows an implementation +of an ethtool-like API and provides a console environment that allows +its use to query and change Ethernet card parameters. The Ethtool sample +is based upon a simple L2 frame reflector. Prerequisites ============= - +notice: On FVL, test case "test_dump_driver_info" need a physical link disconnect, +this case must do manually at this condition. + Assume port 0 and 1 are connected to the traffic generator, to run the test -application in linuxapp environment with 4 lcores, 2 ports. +application in linux app environment with 4 lcores, 2 ports. ethtool -c f -n 4 -The sample should be validated on Forville, Niantic and i350 Nics. +The sample should be validated on Forville, Niantic and i350 Nics. + +other requirements: +*. igxbe driver (version >= 4.3.13). +*. ethtool of linux is the default reference tool. +*. md5sum is a tool to do dumped bin format file comparison. +*. insert two nic cards on No.0 socket -Test Case: Dump driver infor test +Test Case: Dump driver information test ================================= -User "drvinfo" command to dump driver information and then check that dumped -information was exactly the same as fact. +User "drvinfo" command to dump driver information and then check that +dumped information, which are dumped separately by dpdk's ethtool and +linux's ethtool, were exactly the same. EthApp> drvinfo Port 0 driver: rte_ixgbe_pmd (ver: RTE 2.1.0) Port 1 driver: rte_ixgbe_pmd (ver: RTE 2.1.0) Use "link" command to dump all ports link status. +Notice:: On FVL, link detect need a physical link disconnect. EthApp> link Port 0: Up Port 1: Up @@ -68,7 +79,7 @@ Change tester port link status to down and re-check link status. Port 0: Down Port 1: Down -Send few packets to l2fwd and check that command "portstats" dumped correct +Send a few packets to l2fwd and check that command "portstats" dumps correct port statistics. EthApp> portstats 0 Port 0 stats @@ -78,34 +89,38 @@ port statistics. Test Case: Retrieve eeprom test =============================== Unbind ports from igb_uio and bind them to default driver. -Dump eeprom binary by ethtool. +Dump eeprom binary by linux's ethtool and dpdk's ethtool separately. ethtool --eeprom-dump INTF_0 raw on > ethtool_eeprom_0.bin ethtool --eeprom-dump INTF_1 raw on > ethtool_eeprom_1.bin -Retrieve eeprom on specified port and compare csum with the file dumped by ethtool. +Retrieve eeprom on specified port using dpdk's ethtool and +compare csum with the file dumped by ethtool. EthApp> eeprom 0 eeprom_0.bin EthApp> eeprom 1 eeprom_1.bin -md5sum ethtool_eeprom_0.bin -md5sum eeprom_0.bin > eeprom_0.bin - -diff ethtool_eeprom_0.hex eeprom_0.hex + md5sum ethtool_eeprom_0.bin + md5sum eeprom_0.bin + +compare md5sum value of the two bin files. Test Case: Retrieve register test =============================== -Retrieve register on specified port, do not known how to check the binary? +Retrieve register on specified port. EthApp> regs 0 reg_0.bin EthApp> regs 1 reg_1.bin Unbind ports from igb_uio and bind them to default driver. + + dpdk/tools/dpdk_nic_bind.py --bind=ixgbe x:xx.x + Check that dumped register information is correct. -ethtool -d INTF_0 raw on file reg_0.bin -ethtool -d INTF_1 raw on file reg_0.bin - +ethtool -d INTF_0 raw off file reg_0.bin +ethtool -d INTF_1 raw off file reg_0.bin + Test Case: Ring param test ========================== Dump port 0 ring size by ringparam command and check numbers are correct. @@ -126,52 +141,30 @@ Port 0 ring paramaeters Rx Pending: 256 (256 max) Tx Pending: 2048 (4096 max) -Test Case: Pause test -===================== -Enable port 0 Rx pause frame and then create two packets flows in IXIA port. -One flow is 100000 normally packet and the second flow is pause frame. -Check that port 0 Rx speed dropped. For example, niantic will drop from -14.8Mpps to 7.49Mpps. - - EthApp> pause 0 rx - -Use "parse" command to print port pause status, check that port 0 rx has been -paused. - EthApp> pause 0 - Port 0: Rx Paused - -Unpause port 0 rx and then restart port0, check that packets Rx speed is normal. - EthApp> pause 0 none - EthApp> +send packet by scapy on Tester -Pause port 0 TX pause frame. - EthApp> pause 0 tx - -Use "parse" command to print port pause status, check that port 1 tx has been -paused. - EthApp> pause 0 - Port 0: Tx Paused - -Enable flow control in IXIA port and send packets from IXIA with line rate. -Check that IXIA receive flow control packets and IXIA transmit speed dropped. -IXIA Rx packets more then Tx packets to check that received pause frame. - -Unpause port 0 tx and restart port 0. Then send packets to port0, check that -packets forwarded normally from port 0. - EthApp> pause 0 none - EthApp> stop 0 - EthApp> open 0 +check tx/rx packets +EthApp> portstats 0 Test Case: Vlan test ==================== +enable vlan filter flag in main.c of dpdk's ethtool + + sed -i -e '/cfg_port.txmode.mq_mode = ETH_MQ_TX_NONE;$/a\\cfg_port.rxmode.hw_vlan_filter=1;' examples/ethtool/ethtool-app/main.c + +re-compile examples/ethtool + + make -C examples/ethtool + Add vlan 0 to port 0 and vlan 1 to port1, send packet without vlan to port0,1 Verify port0 and port1 recevied vlan packets + EthApp> vlan 0 add 0 VLAN vid 0 added EthApp> vlan 1 add 1 VLAN vid 1 added - + Send packet with vlan0,1 to port0&1. Verify port0 and port1 received vlan packets @@ -189,7 +182,11 @@ port0 and port1 can not receive vlan packet. Test Case: Mac address test =========================== Use "macaddr" command to dump port mac address and then check that dumped -information is exactly the same as fact. +information is exactly the same as ifconfig do. + +set a new mac address by dpdk's ethtool, send and sniff packet and check packet +forwared status + EthApp> macaddr 0 Port 0 MAC Address: XX:XX:XX:XX:XX:XX EthApp> macaddr 1 @@ -211,8 +208,8 @@ Use "macaddr" command to change port mac address and then check mac changed. MAC address changed EthApp> macaddr 0 Port 0 MAC Address: 00:10:00:00:00:00 - -Verified mac adress in forwarded packets has been changed. + +Verified mac adress in forwarded packets has been changed. Test Case: Port config test =========================== @@ -224,18 +221,58 @@ Use "open" command to re-enable port0. Send packets to port0 and verify packets received and forwarded. EthApp> open 0 - Test case: Mtu config test ========================== -Use "mtu" command to change port 0 mtu from default 1518 to 1000. +Use "mtu" command to change port 0 mtu from default 1519 to 9000 on Tester's port. -Send packet size over 1000 and check that packet will be detected as error. +Send packet size over 1519 and check that packet will be detected as error. - EthApp> mtu 0 1000 + EthApp> mtu 0 1519 Port 0 stats In: 0 (0 bytes) Out: 0 (0 bytes) Err: 1 -Change mtu to default value and send packet size over 1000 and check that -packet will normally received. +Change mtu to default value and send packet size over 1519 and check that +packet will normally be received. + +Test Case: Pause tx/rx test(performance test) +===================== +Enable port 0 Rx pause frame and then create two packets flows on IXIA port. +One flow is 100000 normally packet and the second flow is pause frame. +Check that dut's port 0 Rx speed dropped status. For example, niantic will drop +from 14.8Mpps to 7.49Mpps. + + EthApp> pause 0 rx + +Use "pause" command to print dut's port pause status, check that dut's port 0 rx +has been paused. + + EthApp> pause 0 + Port 0: Rx Paused + +Release pause status of port 0 rx and then restart port 0, check that packets Rx +speed is normal. + EthApp> pause 0 none + EthApp> + +Pause port 0 TX pause frame. + EthApp> pause 0 tx + +Use "pause" command to print port pause status, check that port 0 tx has been +paused. + EthApp> pause 0 + Port 0: Tx Paused + +Enable flow control in IXIA port and send packets from IXIA with line rate. +Record line rate before send packet. +Check that IXIA receive flow control packets and IXIA transmit speed dropped. +IXIA Rx packets more then Tx packets to check that received pause frame.Compare +the line rates in the time before and after the Pause packets are injected + +Unpause port 0 tx and restart port 0. Then send packets to port0, check that +packets forwarded normally from port 0. + + EthApp> pause 0 none + EthApp> stop 0 + EthApp> open 0 diff --git a/tests/TestSuite_userspace_ethtool.py b/tests/TestSuite_userspace_ethtool.py index c479093..c23f841 100644 --- a/tests/TestSuite_userspace_ethtool.py +++ b/tests/TestSuite_userspace_ethtool.py @@ -38,12 +38,12 @@ import dts import time import re from test_case import TestCase -from pmd_output import PmdOutput from packet import Packet, sniff_packets, load_sniff_packets import random from etgen import IxiaPacketGenerator from settings import HEADER_SIZE from settings import SCAPY2IXIA +from utils import RED class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): @@ -127,6 +127,22 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): else: return 1518 + def resize_linux_eeprom_file(self, dpdk_eeprom_file, linux_eeprom_file): + basePath = self.dut.base_dir + with open( basePath + dpdk_eeprom_file, 'rb') as fpDpdk: + dpdk_bytes = fpDpdk.read() + dpdk_length = len(dpdk_bytes) + + with open( basePath + linux_eeprom_file, 'rb') as fplinux: + linux_bytes = fplinux.read() + linux_length = len(linux_bytes) + + self.verify(dpdk_length <= linux_length, + "linux ethtool haven't dump out enough data as dpdk ethtool") + + with open( basePath + linux_eeprom_file, 'wb') as fplinux: + fplinux.write(linux_bytes[:dpdk_length]) + def strip_md5(self, filename): md5_info = self.dut.send_expect("md5sum %s" % filename, "# ") md5_pattern = r"(\w+) (\w+)" @@ -155,30 +171,33 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): version = m.group(3) print dts.GREEN("Detect port %s with %s driver\n" % (port, driver)) - # check link status dump function - for port in self.ports: - tester_port = self.tester.get_local_port(port) - intf = self.tester.get_interface(tester_port) - self.tester.send_expect("ip link set dev %s down" % intf, "# ") - # wait for link stable - time.sleep(5) - - out = self.dut.send_expect("link", "EthApp>", 60) - link_pattern = r"Port (\d+): (.*)" - link_infos = out.split("\r\n") - for link_info in link_infos: - m = re.match(link_pattern, link_info) - if m: - port = m.group(1) - status = m.group(2) - self.verify(status == "Down", "Userspace tool failed to detect link down") - - for port in self.ports: - tester_port = self.tester.get_local_port(port) - intf = self.tester.get_interface(tester_port) - self.tester.send_expect("ip link set dev %s up" % intf, "# ") - # wait for link stable - time.sleep(5) + # ethtool doesn't support port disconnect by tools of linux + # only detect physical link disconnect status + if self.nic.startswith("fortville") == False: + # check link status dump function + for port in self.ports: + tester_port = self.tester.get_local_port(port) + intf = self.tester.get_interface(tester_port) + self.tester.send_expect("ip link set dev %s down" % intf, "# ") + # wait for link stable + time.sleep(5) + + out = self.dut.send_expect("link", "EthApp>", 60) + link_pattern = r"Port (\d+): (.*)" + link_infos = out.split("\r\n") + for link_info in link_infos: + m = re.match(link_pattern, link_info) + if m: + port = m.group(1) + status = m.group(2) + self.verify(status == "Down", "Userspace tool failed to detect link down") + + for port in self.ports: + tester_port = self.tester.get_local_port(port) + intf = self.tester.get_interface(tester_port) + self.tester.send_expect("ip link set dev %s up" % intf, "# ") + # wait for link stable + time.sleep(5) # check port stats function pkt = Packet(pkt_type='UDP') @@ -271,8 +290,10 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): for index in range(len(self.ports)): md5 = self.strip_md5(portsinfo[index]['eeprom_file']) + self.resize_linux_eeprom_file( portsinfo[index]['eeprom_file'], portsinfo[index]['ethtool_eeprom']) md5_ref = self.strip_md5(portsinfo[index]['ethtool_eeprom']) - print dts.GREEN("Reference eeprom md5 %s" % md5_ref) + print dts.GREEN("Reference eeprom md5 %s" % md5) + print dts.GREEN("Reference eeprom md5_ref %s" % md5_ref) self.verify(md5 == md5_ref, "Dumped eeprom not same as linux dumped") def test_ring_parameter(self): @@ -323,7 +344,6 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): tester_port = self.tester.get_local_port(port) intf = self.tester.get_interface(tester_port) pkt.send_pkt(tx_port=intf) - time.sleep(2) rx_pkts, tx_pkts = self.strip_portstats(port) self.verify(rx_pkts == ori_rx_pkts + 1, "Failed to Rx vlan packet") self.verify(tx_pkts == ori_tx_pkts + 1, "Failed to Tx vlan packet") @@ -372,8 +392,8 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): inst = sniff_packets(intf, timeout=5) pkt.send_pkt(tx_port=intf) pkts = load_sniff_packets(inst) - self.verify(len(pkts) == 2, "Packet not forwarded as expected") - src_mac = pkts[1].strip_layer_element("layer2", "src") + self.verify(len(pkts) == 1, "Packet not forwarded as expected") + src_mac = pkts[0].strip_layer_element("layer2", "src") self.verify(src_mac == valid_mac, "Forwarded packet not match default mac") # check multicase will not be valid mac @@ -405,9 +425,11 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): rx_pkts, tx_pkts = self.strip_portstats(index) self.verify(rx_pkts == ori_rx_pkts, "Failed to stop port") # restart port and check packet can normally forwarded + time.sleep(2) self.dut.send_expect("open %d" % index, "EthApp>") # wait few time for port ready - time.sleep(0.5) + rx_pkts, tx_pkts = self.strip_portstats(index) + time.sleep(2) pkt.send_pkt(tx_port=intf) rx_pkts_open, tx_pkts_open = self.strip_portstats(index) self.verify(rx_pkts_open == rx_pkts + 1, "Failed to reopen port rx") @@ -430,11 +452,12 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): self.tester.send_expect("ifconfig %s mtu 9000" % (intf), "# ") for mtu in mtus: self.dut.send_expect("mtu %d %d" % (index, mtu), "EthApp>") + ori_rx_pkts, _ = self.strip_portstats(index) pkt_size = mtu + HEADER_SIZE['eth'] pkt = Packet(pkt_len=pkt_size) pkt.send_pkt(tx_port=intf) rx_pkts, _ = self.strip_portstats(index) - self.verify(rx_pkts == 1, "Packet match mtu not forwarded as expected") + self.verify(rx_pkts == ori_rx_pkts, "Packet match mtu not forwarded as expected") pkt = Packet(pkt_len=mtu + 1) pkt.send_pkt(tx_port=intf) rx_pkts_over, _ = self.strip_portstats(index) -- 1.9.3 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dts] [PATCH V1]userspace_ethtool: update test script and test plan 2016-05-05 2:59 ` Yufen Mo @ 2016-05-23 20:29 ` Liu, Yong 0 siblings, 0 replies; 6+ messages in thread From: Liu, Yong @ 2016-05-23 20:29 UTC (permalink / raw) To: Mo, YufengX, dts; +Cc: Mo, YufengX Applied, thanks. > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Yufen Mo > Sent: Wednesday, May 04, 2016 7:59 PM > To: dts@dpdk.org > Cc: Mo, YufengX > Subject: [dts] [PATCH V1]userspace_ethtool: update test script and test > plan > > From: yufengmx <yufengx.mo@intel.com> > > change vlan packet check number when doing vlan testing. Add process to > resize linux ethtool eeprom dump > data size as the size dump by dpdk ethtool. Ignore auto test link connect > status on FVL. update related > test cases' test plan, add testing configuration and requirements. > > Signed-off-by: yufengmx <yufengx.mo@intel.com> > --- > test_plans/userspace_ethtool_test_plan.rst | 167 ++++++++++++++++++------ > ----- > tests/TestSuite_userspace_ethtool.py | 85 +++++++++------ > 2 files changed, 156 insertions(+), 96 deletions(-) > > diff --git a/test_plans/userspace_ethtool_test_plan.rst > b/test_plans/userspace_ethtool_test_plan.rst > index 5ee79a1..83e7fac 100644 > --- a/test_plans/userspace_ethtool_test_plan.rst > +++ b/test_plans/userspace_ethtool_test_plan.rst > @@ -34,31 +34,42 @@ > Userspace Ethtool > ================== > > -This feature is designed to provide one rte_ethtool shim layer based on > -rte_ethdev API. The implementation also along with a command prompt > driven > -demonstration application. It only contained of 18 popular used Ethtool > and > -Netdevice ops as described in rte_ethtool.h. > +This feature is designed to provide one rte_ethtool shim layer based on > +rte_ethdev API. The Ethtool sample application shows an implementation > +of an ethtool-like API and provides a console environment that allows > +its use to query and change Ethernet card parameters. The Ethtool sample > +is based upon a simple L2 frame reflector. > > Prerequisites > ============= > - > +notice: On FVL, test case "test_dump_driver_info" need a physical link > disconnect, > +this case must do manually at this condition. > + > Assume port 0 and 1 are connected to the traffic generator, to run the > test > -application in linuxapp environment with 4 lcores, 2 ports. > +application in linux app environment with 4 lcores, 2 ports. > > ethtool -c f -n 4 > > -The sample should be validated on Forville, Niantic and i350 Nics. > +The sample should be validated on Forville, Niantic and i350 Nics. > + > +other requirements: > +*. igxbe driver (version >= 4.3.13). > +*. ethtool of linux is the default reference tool. > +*. md5sum is a tool to do dumped bin format file comparison. > +*. insert two nic cards on No.0 socket > > -Test Case: Dump driver infor test > +Test Case: Dump driver information test > ================================= > -User "drvinfo" command to dump driver information and then check that > dumped > -information was exactly the same as fact. > +User "drvinfo" command to dump driver information and then check that > +dumped information, which are dumped separately by dpdk's ethtool and > +linux's ethtool, were exactly the same. > > EthApp> drvinfo > Port 0 driver: rte_ixgbe_pmd (ver: RTE 2.1.0) > Port 1 driver: rte_ixgbe_pmd (ver: RTE 2.1.0) > > Use "link" command to dump all ports link status. > +Notice:: On FVL, link detect need a physical link disconnect. > EthApp> link > Port 0: Up > Port 1: Up > @@ -68,7 +79,7 @@ Change tester port link status to down and re-check link > status. > Port 0: Down > Port 1: Down > > -Send few packets to l2fwd and check that command "portstats" dumped > correct > +Send a few packets to l2fwd and check that command "portstats" dumps > correct > port statistics. > EthApp> portstats 0 > Port 0 stats > @@ -78,34 +89,38 @@ port statistics. > Test Case: Retrieve eeprom test > =============================== > Unbind ports from igb_uio and bind them to default driver. > -Dump eeprom binary by ethtool. > +Dump eeprom binary by linux's ethtool and dpdk's ethtool separately. > > ethtool --eeprom-dump INTF_0 raw on > ethtool_eeprom_0.bin > ethtool --eeprom-dump INTF_1 raw on > ethtool_eeprom_1.bin > > -Retrieve eeprom on specified port and compare csum with the file dumped > by ethtool. > +Retrieve eeprom on specified port using dpdk's ethtool and > +compare csum with the file dumped by ethtool. > > EthApp> eeprom 0 eeprom_0.bin > EthApp> eeprom 1 eeprom_1.bin > > -md5sum ethtool_eeprom_0.bin > -md5sum eeprom_0.bin > eeprom_0.bin > - > -diff ethtool_eeprom_0.hex eeprom_0.hex > + md5sum ethtool_eeprom_0.bin > + md5sum eeprom_0.bin > + > +compare md5sum value of the two bin files. > > Test Case: Retrieve register test > =============================== > -Retrieve register on specified port, do not known how to check the binary? > +Retrieve register on specified port. > > EthApp> regs 0 reg_0.bin > EthApp> regs 1 reg_1.bin > > Unbind ports from igb_uio and bind them to default driver. > + > + dpdk/tools/dpdk_nic_bind.py --bind=ixgbe x:xx.x > + > Check that dumped register information is correct. > > -ethtool -d INTF_0 raw on file reg_0.bin > -ethtool -d INTF_1 raw on file reg_0.bin > - > +ethtool -d INTF_0 raw off file reg_0.bin > +ethtool -d INTF_1 raw off file reg_0.bin > + > Test Case: Ring param test > ========================== > Dump port 0 ring size by ringparam command and check numbers are correct. > @@ -126,52 +141,30 @@ Port 0 ring paramaeters > Rx Pending: 256 (256 max) > Tx Pending: 2048 (4096 max) > > -Test Case: Pause test > -===================== > -Enable port 0 Rx pause frame and then create two packets flows in IXIA > port. > -One flow is 100000 normally packet and the second flow is pause frame. > -Check that port 0 Rx speed dropped. For example, niantic will drop from > -14.8Mpps to 7.49Mpps. > - > - EthApp> pause 0 rx > - > -Use "parse" command to print port pause status, check that port 0 rx has > been > -paused. > - EthApp> pause 0 > - Port 0: Rx Paused > - > -Unpause port 0 rx and then restart port0, check that packets Rx speed is > normal. > - EthApp> pause 0 none > - EthApp> > +send packet by scapy on Tester > > -Pause port 0 TX pause frame. > - EthApp> pause 0 tx > - > -Use "parse" command to print port pause status, check that port 1 tx has > been > -paused. > - EthApp> pause 0 > - Port 0: Tx Paused > - > -Enable flow control in IXIA port and send packets from IXIA with line > rate. > -Check that IXIA receive flow control packets and IXIA transmit speed > dropped. > -IXIA Rx packets more then Tx packets to check that received pause frame. > - > -Unpause port 0 tx and restart port 0. Then send packets to port0, check > that > -packets forwarded normally from port 0. > - EthApp> pause 0 none > - EthApp> stop 0 > - EthApp> open 0 > +check tx/rx packets > +EthApp> portstats 0 > > Test Case: Vlan test > ==================== > +enable vlan filter flag in main.c of dpdk's ethtool > + > + sed -i -e '/cfg_port.txmode.mq_mode = > ETH_MQ_TX_NONE;$/a\\cfg_port.rxmode.hw_vlan_filter=1;' > examples/ethtool/ethtool-app/main.c > + > +re-compile examples/ethtool > + > + make -C examples/ethtool > + > Add vlan 0 to port 0 and vlan 1 to port1, send packet without vlan to > port0,1 > Verify port0 and port1 recevied vlan packets > + > EthApp> vlan 0 add 0 > VLAN vid 0 added > > EthApp> vlan 1 add 1 > VLAN vid 1 added > - > + > Send packet with vlan0,1 to port0&1. Verify port0 and port1 received vlan > packets > > @@ -189,7 +182,11 @@ port0 and port1 can not receive vlan packet. > Test Case: Mac address test > =========================== > Use "macaddr" command to dump port mac address and then check that dumped > -information is exactly the same as fact. > +information is exactly the same as ifconfig do. > + > +set a new mac address by dpdk's ethtool, send and sniff packet and check > packet > +forwared status > + > EthApp> macaddr 0 > Port 0 MAC Address: XX:XX:XX:XX:XX:XX > EthApp> macaddr 1 > @@ -211,8 +208,8 @@ Use "macaddr" command to change port mac address and > then check mac changed. > MAC address changed > EthApp> macaddr 0 > Port 0 MAC Address: 00:10:00:00:00:00 > - > -Verified mac adress in forwarded packets has been changed. > + > +Verified mac adress in forwarded packets has been changed. > > Test Case: Port config test > =========================== > @@ -224,18 +221,58 @@ Use "open" command to re-enable port0. Send packets > to port0 and verify > packets received and forwarded. > EthApp> open 0 > > - > Test case: Mtu config test > ========================== > -Use "mtu" command to change port 0 mtu from default 1518 to 1000. > +Use "mtu" command to change port 0 mtu from default 1519 to 9000 on > Tester's port. > > -Send packet size over 1000 and check that packet will be detected as > error. > +Send packet size over 1519 and check that packet will be detected as > error. > > - EthApp> mtu 0 1000 > + EthApp> mtu 0 1519 > Port 0 stats > In: 0 (0 bytes) > Out: 0 (0 bytes) > Err: 1 > > -Change mtu to default value and send packet size over 1000 and check that > -packet will normally received. > +Change mtu to default value and send packet size over 1519 and check that > +packet will normally be received. > + > +Test Case: Pause tx/rx test(performance test) > +===================== > +Enable port 0 Rx pause frame and then create two packets flows on IXIA > port. > +One flow is 100000 normally packet and the second flow is pause frame. > +Check that dut's port 0 Rx speed dropped status. For example, niantic > will drop > +from 14.8Mpps to 7.49Mpps. > + > + EthApp> pause 0 rx > + > +Use "pause" command to print dut's port pause status, check that dut's > port 0 rx > +has been paused. > + > + EthApp> pause 0 > + Port 0: Rx Paused > + > +Release pause status of port 0 rx and then restart port 0, check that > packets Rx > +speed is normal. > + EthApp> pause 0 none > + EthApp> > + > +Pause port 0 TX pause frame. > + EthApp> pause 0 tx > + > +Use "pause" command to print port pause status, check that port 0 tx has > been > +paused. > + EthApp> pause 0 > + Port 0: Tx Paused > + > +Enable flow control in IXIA port and send packets from IXIA with line > rate. > +Record line rate before send packet. > +Check that IXIA receive flow control packets and IXIA transmit speed > dropped. > +IXIA Rx packets more then Tx packets to check that received pause > frame.Compare > +the line rates in the time before and after the Pause packets are > injected > + > +Unpause port 0 tx and restart port 0. Then send packets to port0, check > that > +packets forwarded normally from port 0. > + > + EthApp> pause 0 none > + EthApp> stop 0 > + EthApp> open 0 > diff --git a/tests/TestSuite_userspace_ethtool.py > b/tests/TestSuite_userspace_ethtool.py > index c479093..c23f841 100644 > --- a/tests/TestSuite_userspace_ethtool.py > +++ b/tests/TestSuite_userspace_ethtool.py > @@ -38,12 +38,12 @@ import dts > import time > import re > from test_case import TestCase > -from pmd_output import PmdOutput > from packet import Packet, sniff_packets, load_sniff_packets > import random > from etgen import IxiaPacketGenerator > from settings import HEADER_SIZE > from settings import SCAPY2IXIA > +from utils import RED > > > class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): > @@ -127,6 +127,22 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > else: > return 1518 > > + def resize_linux_eeprom_file(self, dpdk_eeprom_file, > linux_eeprom_file): > + basePath = self.dut.base_dir > + with open( basePath + dpdk_eeprom_file, 'rb') as fpDpdk: > + dpdk_bytes = fpDpdk.read() > + dpdk_length = len(dpdk_bytes) > + > + with open( basePath + linux_eeprom_file, 'rb') as fplinux: > + linux_bytes = fplinux.read() > + linux_length = len(linux_bytes) > + > + self.verify(dpdk_length <= linux_length, > + "linux ethtool haven't dump out enough data as dpdk > ethtool") > + > + with open( basePath + linux_eeprom_file, 'wb') as fplinux: > + fplinux.write(linux_bytes[:dpdk_length]) > + > def strip_md5(self, filename): > md5_info = self.dut.send_expect("md5sum %s" % filename, "# ") > md5_pattern = r"(\w+) (\w+)" > @@ -155,30 +171,33 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > version = m.group(3) > print dts.GREEN("Detect port %s with %s driver\n" % (port, > driver)) > > - # check link status dump function > - for port in self.ports: > - tester_port = self.tester.get_local_port(port) > - intf = self.tester.get_interface(tester_port) > - self.tester.send_expect("ip link set dev %s down" % intf, "# > ") > - # wait for link stable > - time.sleep(5) > - > - out = self.dut.send_expect("link", "EthApp>", 60) > - link_pattern = r"Port (\d+): (.*)" > - link_infos = out.split("\r\n") > - for link_info in link_infos: > - m = re.match(link_pattern, link_info) > - if m: > - port = m.group(1) > - status = m.group(2) > - self.verify(status == "Down", "Userspace tool failed to > detect link down") > - > - for port in self.ports: > - tester_port = self.tester.get_local_port(port) > - intf = self.tester.get_interface(tester_port) > - self.tester.send_expect("ip link set dev %s up" % intf, "# ") > - # wait for link stable > - time.sleep(5) > + # ethtool doesn't support port disconnect by tools of linux > + # only detect physical link disconnect status > + if self.nic.startswith("fortville") == False: > + # check link status dump function > + for port in self.ports: > + tester_port = self.tester.get_local_port(port) > + intf = self.tester.get_interface(tester_port) > + self.tester.send_expect("ip link set dev %s down" % intf, > "# ") > + # wait for link stable > + time.sleep(5) > + > + out = self.dut.send_expect("link", "EthApp>", 60) > + link_pattern = r"Port (\d+): (.*)" > + link_infos = out.split("\r\n") > + for link_info in link_infos: > + m = re.match(link_pattern, link_info) > + if m: > + port = m.group(1) > + status = m.group(2) > + self.verify(status == "Down", "Userspace tool failed > to detect link down") > + > + for port in self.ports: > + tester_port = self.tester.get_local_port(port) > + intf = self.tester.get_interface(tester_port) > + self.tester.send_expect("ip link set dev %s up" % intf, > "# ") > + # wait for link stable > + time.sleep(5) > > # check port stats function > pkt = Packet(pkt_type='UDP') > @@ -271,8 +290,10 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > > for index in range(len(self.ports)): > md5 = self.strip_md5(portsinfo[index]['eeprom_file']) > + > self.resize_linux_eeprom_file( portsinfo[index]['eeprom_file'], > portsinfo[index]['ethtool_eeprom']) > md5_ref = self.strip_md5(portsinfo[index]['ethtool_eeprom']) > - print dts.GREEN("Reference eeprom md5 %s" % md5_ref) > + print dts.GREEN("Reference eeprom md5 %s" % md5) > + print dts.GREEN("Reference eeprom md5_ref %s" % md5_ref) > self.verify(md5 == md5_ref, "Dumped eeprom not same as linux > dumped") > > def test_ring_parameter(self): > @@ -323,7 +344,6 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > tester_port = self.tester.get_local_port(port) > intf = self.tester.get_interface(tester_port) > pkt.send_pkt(tx_port=intf) > - time.sleep(2) > rx_pkts, tx_pkts = self.strip_portstats(port) > self.verify(rx_pkts == ori_rx_pkts + 1, "Failed to Rx vlan > packet") > self.verify(tx_pkts == ori_tx_pkts + 1, "Failed to Tx vlan > packet") > @@ -372,8 +392,8 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > inst = sniff_packets(intf, timeout=5) > pkt.send_pkt(tx_port=intf) > pkts = load_sniff_packets(inst) > - self.verify(len(pkts) == 2, "Packet not forwarded as > expected") > - src_mac = pkts[1].strip_layer_element("layer2", "src") > + self.verify(len(pkts) == 1, "Packet not forwarded as > expected") > + src_mac = pkts[0].strip_layer_element("layer2", "src") > self.verify(src_mac == valid_mac, "Forwarded packet not match > default mac") > > # check multicase will not be valid mac > @@ -405,9 +425,11 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > rx_pkts, tx_pkts = self.strip_portstats(index) > self.verify(rx_pkts == ori_rx_pkts, "Failed to stop port") > # restart port and check packet can normally forwarded > + time.sleep(2) > self.dut.send_expect("open %d" % index, "EthApp>") > # wait few time for port ready > - time.sleep(0.5) > + rx_pkts, tx_pkts = self.strip_portstats(index) > + time.sleep(2) > pkt.send_pkt(tx_port=intf) > rx_pkts_open, tx_pkts_open = self.strip_portstats(index) > self.verify(rx_pkts_open == rx_pkts + 1, "Failed to reopen > port rx") > @@ -430,11 +452,12 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > self.tester.send_expect("ifconfig %s mtu 9000" % (intf), "# ") > for mtu in mtus: > self.dut.send_expect("mtu %d %d" % (index, mtu), > "EthApp>") > + ori_rx_pkts, _ = self.strip_portstats(index) > pkt_size = mtu + HEADER_SIZE['eth'] > pkt = Packet(pkt_len=pkt_size) > pkt.send_pkt(tx_port=intf) > rx_pkts, _ = self.strip_portstats(index) > - self.verify(rx_pkts == 1, "Packet match mtu not forwarded > as expected") > + self.verify(rx_pkts == ori_rx_pkts, "Packet match mtu not > forwarded as expected") > pkt = Packet(pkt_len=mtu + 1) > pkt.send_pkt(tx_port=intf) > rx_pkts_over, _ = self.strip_portstats(index) > -- > 1.9.3 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [dts] [PATCH V1]userspace_ethtool: update test script and test plan @ 2016-05-05 2:23 Yufen Mo 2016-05-05 2:23 ` Yufen Mo 0 siblings, 1 reply; 6+ messages in thread From: Yufen Mo @ 2016-05-05 2:23 UTC (permalink / raw) To: dts; +Cc: yufengmx From: yufengmx <yufengx.mo@intel.com> yufengmx (1): userspace_ethtool: update test script and test plan test_plans/userspace_ethtool_test_plan.rst | 167 ++++++++++++++++++----------- tests/TestSuite_userspace_ethtool.py | 89 +++++++++------ 2 files changed, 160 insertions(+), 96 deletions(-) -- 1.9.3 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [dts] [PATCH V1]userspace_ethtool: update test script and test plan 2016-05-05 2:23 Yufen Mo @ 2016-05-05 2:23 ` Yufen Mo 2016-05-05 2:35 ` Liu, Yong 0 siblings, 1 reply; 6+ messages in thread From: Yufen Mo @ 2016-05-05 2:23 UTC (permalink / raw) To: dts; +Cc: yufengmx From: yufengmx <yufengx.mo@intel.com> change vlan packet check number when doing vlan testing.Add process to resize linux ethtool eeprom dump data content as the size dump by dpdk ethtool. Ignore auto test link connect status on FVL. update related test cases' test plan, add testing configuration and requirements. Signed-off-by: yufengmx <yufengx.mo@intel.com> --- test_plans/userspace_ethtool_test_plan.rst | 167 ++++++++++++++++++----------- tests/TestSuite_userspace_ethtool.py | 89 +++++++++------ 2 files changed, 160 insertions(+), 96 deletions(-) diff --git a/test_plans/userspace_ethtool_test_plan.rst b/test_plans/userspace_ethtool_test_plan.rst index 5ee79a1..83e7fac 100644 --- a/test_plans/userspace_ethtool_test_plan.rst +++ b/test_plans/userspace_ethtool_test_plan.rst @@ -34,31 +34,42 @@ Userspace Ethtool ================== -This feature is designed to provide one rte_ethtool shim layer based on -rte_ethdev API. The implementation also along with a command prompt driven -demonstration application. It only contained of 18 popular used Ethtool and -Netdevice ops as described in rte_ethtool.h. +This feature is designed to provide one rte_ethtool shim layer based on +rte_ethdev API. The Ethtool sample application shows an implementation +of an ethtool-like API and provides a console environment that allows +its use to query and change Ethernet card parameters. The Ethtool sample +is based upon a simple L2 frame reflector. Prerequisites ============= - +notice: On FVL, test case "test_dump_driver_info" need a physical link disconnect, +this case must do manually at this condition. + Assume port 0 and 1 are connected to the traffic generator, to run the test -application in linuxapp environment with 4 lcores, 2 ports. +application in linux app environment with 4 lcores, 2 ports. ethtool -c f -n 4 -The sample should be validated on Forville, Niantic and i350 Nics. +The sample should be validated on Forville, Niantic and i350 Nics. + +other requirements: +*. igxbe driver (version >= 4.3.13). +*. ethtool of linux is the default reference tool. +*. md5sum is a tool to do dumped bin format file comparison. +*. insert two nic cards on No.0 socket -Test Case: Dump driver infor test +Test Case: Dump driver information test ================================= -User "drvinfo" command to dump driver information and then check that dumped -information was exactly the same as fact. +User "drvinfo" command to dump driver information and then check that +dumped information, which are dumped separately by dpdk's ethtool and +linux's ethtool, were exactly the same. EthApp> drvinfo Port 0 driver: rte_ixgbe_pmd (ver: RTE 2.1.0) Port 1 driver: rte_ixgbe_pmd (ver: RTE 2.1.0) Use "link" command to dump all ports link status. +Notice:: On FVL, link detect need a physical link disconnect. EthApp> link Port 0: Up Port 1: Up @@ -68,7 +79,7 @@ Change tester port link status to down and re-check link status. Port 0: Down Port 1: Down -Send few packets to l2fwd and check that command "portstats" dumped correct +Send a few packets to l2fwd and check that command "portstats" dumps correct port statistics. EthApp> portstats 0 Port 0 stats @@ -78,34 +89,38 @@ port statistics. Test Case: Retrieve eeprom test =============================== Unbind ports from igb_uio and bind them to default driver. -Dump eeprom binary by ethtool. +Dump eeprom binary by linux's ethtool and dpdk's ethtool separately. ethtool --eeprom-dump INTF_0 raw on > ethtool_eeprom_0.bin ethtool --eeprom-dump INTF_1 raw on > ethtool_eeprom_1.bin -Retrieve eeprom on specified port and compare csum with the file dumped by ethtool. +Retrieve eeprom on specified port using dpdk's ethtool and +compare csum with the file dumped by ethtool. EthApp> eeprom 0 eeprom_0.bin EthApp> eeprom 1 eeprom_1.bin -md5sum ethtool_eeprom_0.bin -md5sum eeprom_0.bin > eeprom_0.bin - -diff ethtool_eeprom_0.hex eeprom_0.hex + md5sum ethtool_eeprom_0.bin + md5sum eeprom_0.bin + +compare md5sum value of the two bin files. Test Case: Retrieve register test =============================== -Retrieve register on specified port, do not known how to check the binary? +Retrieve register on specified port. EthApp> regs 0 reg_0.bin EthApp> regs 1 reg_1.bin Unbind ports from igb_uio and bind them to default driver. + + dpdk/tools/dpdk_nic_bind.py --bind=ixgbe x:xx.x + Check that dumped register information is correct. -ethtool -d INTF_0 raw on file reg_0.bin -ethtool -d INTF_1 raw on file reg_0.bin - +ethtool -d INTF_0 raw off file reg_0.bin +ethtool -d INTF_1 raw off file reg_0.bin + Test Case: Ring param test ========================== Dump port 0 ring size by ringparam command and check numbers are correct. @@ -126,52 +141,30 @@ Port 0 ring paramaeters Rx Pending: 256 (256 max) Tx Pending: 2048 (4096 max) -Test Case: Pause test -===================== -Enable port 0 Rx pause frame and then create two packets flows in IXIA port. -One flow is 100000 normally packet and the second flow is pause frame. -Check that port 0 Rx speed dropped. For example, niantic will drop from -14.8Mpps to 7.49Mpps. - - EthApp> pause 0 rx - -Use "parse" command to print port pause status, check that port 0 rx has been -paused. - EthApp> pause 0 - Port 0: Rx Paused - -Unpause port 0 rx and then restart port0, check that packets Rx speed is normal. - EthApp> pause 0 none - EthApp> +send packet by scapy on Tester -Pause port 0 TX pause frame. - EthApp> pause 0 tx - -Use "parse" command to print port pause status, check that port 1 tx has been -paused. - EthApp> pause 0 - Port 0: Tx Paused - -Enable flow control in IXIA port and send packets from IXIA with line rate. -Check that IXIA receive flow control packets and IXIA transmit speed dropped. -IXIA Rx packets more then Tx packets to check that received pause frame. - -Unpause port 0 tx and restart port 0. Then send packets to port0, check that -packets forwarded normally from port 0. - EthApp> pause 0 none - EthApp> stop 0 - EthApp> open 0 +check tx/rx packets +EthApp> portstats 0 Test Case: Vlan test ==================== +enable vlan filter flag in main.c of dpdk's ethtool + + sed -i -e '/cfg_port.txmode.mq_mode = ETH_MQ_TX_NONE;$/a\\cfg_port.rxmode.hw_vlan_filter=1;' examples/ethtool/ethtool-app/main.c + +re-compile examples/ethtool + + make -C examples/ethtool + Add vlan 0 to port 0 and vlan 1 to port1, send packet without vlan to port0,1 Verify port0 and port1 recevied vlan packets + EthApp> vlan 0 add 0 VLAN vid 0 added EthApp> vlan 1 add 1 VLAN vid 1 added - + Send packet with vlan0,1 to port0&1. Verify port0 and port1 received vlan packets @@ -189,7 +182,11 @@ port0 and port1 can not receive vlan packet. Test Case: Mac address test =========================== Use "macaddr" command to dump port mac address and then check that dumped -information is exactly the same as fact. +information is exactly the same as ifconfig do. + +set a new mac address by dpdk's ethtool, send and sniff packet and check packet +forwared status + EthApp> macaddr 0 Port 0 MAC Address: XX:XX:XX:XX:XX:XX EthApp> macaddr 1 @@ -211,8 +208,8 @@ Use "macaddr" command to change port mac address and then check mac changed. MAC address changed EthApp> macaddr 0 Port 0 MAC Address: 00:10:00:00:00:00 - -Verified mac adress in forwarded packets has been changed. + +Verified mac adress in forwarded packets has been changed. Test Case: Port config test =========================== @@ -224,18 +221,58 @@ Use "open" command to re-enable port0. Send packets to port0 and verify packets received and forwarded. EthApp> open 0 - Test case: Mtu config test ========================== -Use "mtu" command to change port 0 mtu from default 1518 to 1000. +Use "mtu" command to change port 0 mtu from default 1519 to 9000 on Tester's port. -Send packet size over 1000 and check that packet will be detected as error. +Send packet size over 1519 and check that packet will be detected as error. - EthApp> mtu 0 1000 + EthApp> mtu 0 1519 Port 0 stats In: 0 (0 bytes) Out: 0 (0 bytes) Err: 1 -Change mtu to default value and send packet size over 1000 and check that -packet will normally received. +Change mtu to default value and send packet size over 1519 and check that +packet will normally be received. + +Test Case: Pause tx/rx test(performance test) +===================== +Enable port 0 Rx pause frame and then create two packets flows on IXIA port. +One flow is 100000 normally packet and the second flow is pause frame. +Check that dut's port 0 Rx speed dropped status. For example, niantic will drop +from 14.8Mpps to 7.49Mpps. + + EthApp> pause 0 rx + +Use "pause" command to print dut's port pause status, check that dut's port 0 rx +has been paused. + + EthApp> pause 0 + Port 0: Rx Paused + +Release pause status of port 0 rx and then restart port 0, check that packets Rx +speed is normal. + EthApp> pause 0 none + EthApp> + +Pause port 0 TX pause frame. + EthApp> pause 0 tx + +Use "pause" command to print port pause status, check that port 0 tx has been +paused. + EthApp> pause 0 + Port 0: Tx Paused + +Enable flow control in IXIA port and send packets from IXIA with line rate. +Record line rate before send packet. +Check that IXIA receive flow control packets and IXIA transmit speed dropped. +IXIA Rx packets more then Tx packets to check that received pause frame.Compare +the line rates in the time before and after the Pause packets are injected + +Unpause port 0 tx and restart port 0. Then send packets to port0, check that +packets forwarded normally from port 0. + + EthApp> pause 0 none + EthApp> stop 0 + EthApp> open 0 diff --git a/tests/TestSuite_userspace_ethtool.py b/tests/TestSuite_userspace_ethtool.py index c479093..a0f4018 100644 --- a/tests/TestSuite_userspace_ethtool.py +++ b/tests/TestSuite_userspace_ethtool.py @@ -38,12 +38,12 @@ import dts import time import re from test_case import TestCase -from pmd_output import PmdOutput from packet import Packet, sniff_packets, load_sniff_packets import random from etgen import IxiaPacketGenerator from settings import HEADER_SIZE from settings import SCAPY2IXIA +from utils import RED class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): @@ -127,6 +127,22 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): else: return 1518 + def resize_linux_eeprom_file(self, dpdk_eeprom_file, linux_eeprom_file): + basePath = "/root/dpdk/" + with open( basePath + dpdk_eeprom_file, 'rb') as fpDpdk: + dpdk_bytes = fpDpdk.read() + dpdk_length = len(dpdk_bytes) + + with open( basePath + linux_eeprom_file, 'rb') as fplinux: + linux_bytes = fplinux.read() + linux_length = len(linux_bytes) + + self.verify(dpdk_length <= linux_length, + "linux ethtool haven't dump out enough data as dpdk ethtool") + + with open( basePath + linux_eeprom_file, 'wb') as fplinux: + fplinux.write(linux_bytes[:dpdk_length]) + def strip_md5(self, filename): md5_info = self.dut.send_expect("md5sum %s" % filename, "# ") md5_pattern = r"(\w+) (\w+)" @@ -155,30 +171,33 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): version = m.group(3) print dts.GREEN("Detect port %s with %s driver\n" % (port, driver)) - # check link status dump function - for port in self.ports: - tester_port = self.tester.get_local_port(port) - intf = self.tester.get_interface(tester_port) - self.tester.send_expect("ip link set dev %s down" % intf, "# ") - # wait for link stable - time.sleep(5) - - out = self.dut.send_expect("link", "EthApp>", 60) - link_pattern = r"Port (\d+): (.*)" - link_infos = out.split("\r\n") - for link_info in link_infos: - m = re.match(link_pattern, link_info) - if m: - port = m.group(1) - status = m.group(2) - self.verify(status == "Down", "Userspace tool failed to detect link down") - - for port in self.ports: - tester_port = self.tester.get_local_port(port) - intf = self.tester.get_interface(tester_port) - self.tester.send_expect("ip link set dev %s up" % intf, "# ") - # wait for link stable - time.sleep(5) + # ethtool doesn't support port disconnect by tools of linux + # only detect physical link disconnect status + if self.nic.startswith("fortville") == False: + # check link status dump function + for port in self.ports: + tester_port = self.tester.get_local_port(port) + intf = self.tester.get_interface(tester_port) + self.tester.send_expect("ip link set dev %s down" % intf, "# ") + # wait for link stable + time.sleep(5) + + out = self.dut.send_expect("link", "EthApp>", 60) + link_pattern = r"Port (\d+): (.*)" + link_infos = out.split("\r\n") + for link_info in link_infos: + m = re.match(link_pattern, link_info) + if m: + port = m.group(1) + status = m.group(2) + self.verify(status == "Down", "Userspace tool failed to detect link down") + + for port in self.ports: + tester_port = self.tester.get_local_port(port) + intf = self.tester.get_interface(tester_port) + self.tester.send_expect("ip link set dev %s up" % intf, "# ") + # wait for link stable + time.sleep(5) # check port stats function pkt = Packet(pkt_type='UDP') @@ -271,8 +290,10 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): for index in range(len(self.ports)): md5 = self.strip_md5(portsinfo[index]['eeprom_file']) + self.resize_linux_eeprom_file( portsinfo[index]['eeprom_file'], portsinfo[index]['ethtool_eeprom']) md5_ref = self.strip_md5(portsinfo[index]['ethtool_eeprom']) - print dts.GREEN("Reference eeprom md5 %s" % md5_ref) + print dts.GREEN("Reference eeprom md5 %s" % md5) + print dts.GREEN("Reference eeprom md5_ref %s" % md5_ref) self.verify(md5 == md5_ref, "Dumped eeprom not same as linux dumped") def test_ring_parameter(self): @@ -323,7 +344,6 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): tester_port = self.tester.get_local_port(port) intf = self.tester.get_interface(tester_port) pkt.send_pkt(tx_port=intf) - time.sleep(2) rx_pkts, tx_pkts = self.strip_portstats(port) self.verify(rx_pkts == ori_rx_pkts + 1, "Failed to Rx vlan packet") self.verify(tx_pkts == ori_tx_pkts + 1, "Failed to Tx vlan packet") @@ -372,8 +392,8 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): inst = sniff_packets(intf, timeout=5) pkt.send_pkt(tx_port=intf) pkts = load_sniff_packets(inst) - self.verify(len(pkts) == 2, "Packet not forwarded as expected") - src_mac = pkts[1].strip_layer_element("layer2", "src") + self.verify(len(pkts) == 1, "Packet not forwarded as expected") + src_mac = pkts[0].strip_layer_element("layer2", "src") self.verify(src_mac == valid_mac, "Forwarded packet not match default mac") # check multicase will not be valid mac @@ -405,11 +425,17 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): rx_pkts, tx_pkts = self.strip_portstats(index) self.verify(rx_pkts == ori_rx_pkts, "Failed to stop port") # restart port and check packet can normally forwarded + time.sleep(2) self.dut.send_expect("open %d" % index, "EthApp>") # wait few time for port ready - time.sleep(0.5) + rx_pkts, tx_pkts = self.strip_portstats(index) + time.sleep(2) pkt.send_pkt(tx_port=intf) rx_pkts_open, tx_pkts_open = self.strip_portstats(index) + if rx_pkts_open != rx_pkts + 1: + RED( "Failed to reopen port rx") + if tx_pkts_open != tx_pkts + 1: + RED( "Failed to reopen port tx") self.verify(rx_pkts_open == rx_pkts + 1, "Failed to reopen port rx") self.verify(tx_pkts_open == tx_pkts + 1, "Failed to reopen port tx") @@ -430,11 +456,12 @@ class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): self.tester.send_expect("ifconfig %s mtu 9000" % (intf), "# ") for mtu in mtus: self.dut.send_expect("mtu %d %d" % (index, mtu), "EthApp>") + ori_rx_pkts, _ = self.strip_portstats(index) pkt_size = mtu + HEADER_SIZE['eth'] pkt = Packet(pkt_len=pkt_size) pkt.send_pkt(tx_port=intf) rx_pkts, _ = self.strip_portstats(index) - self.verify(rx_pkts == 1, "Packet match mtu not forwarded as expected") + self.verify(rx_pkts == ori_rx_pkts, "Packet match mtu not forwarded as expected") pkt = Packet(pkt_len=mtu + 1) pkt.send_pkt(tx_port=intf) rx_pkts_over, _ = self.strip_portstats(index) -- 1.9.3 ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [dts] [PATCH V1]userspace_ethtool: update test script and test plan 2016-05-05 2:23 ` Yufen Mo @ 2016-05-05 2:35 ` Liu, Yong 0 siblings, 0 replies; 6+ messages in thread From: Liu, Yong @ 2016-05-05 2:35 UTC (permalink / raw) To: Mo, YufengX, dts; +Cc: Mo, YufengX Yufen, please check comments in previous email. > -----Original Message----- > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Yufen Mo > Sent: Wednesday, May 04, 2016 7:23 PM > To: dts@dpdk.org > Cc: Mo, YufengX > Subject: [dts] [PATCH V1]userspace_ethtool: update test script and test > plan > > From: yufengmx <yufengx.mo@intel.com> > > change vlan packet check number when doing vlan testing.Add process to > resize linux ethtool eeprom dump > data content as the size dump by dpdk ethtool. Ignore auto test link > connect status on FVL. update related > test cases' test plan, add testing configuration and requirements. > > Signed-off-by: yufengmx <yufengx.mo@intel.com> > --- > test_plans/userspace_ethtool_test_plan.rst | 167 ++++++++++++++++++------ > ----- > tests/TestSuite_userspace_ethtool.py | 89 +++++++++------ > 2 files changed, 160 insertions(+), 96 deletions(-) > > diff --git a/test_plans/userspace_ethtool_test_plan.rst > b/test_plans/userspace_ethtool_test_plan.rst > index 5ee79a1..83e7fac 100644 > --- a/test_plans/userspace_ethtool_test_plan.rst > +++ b/test_plans/userspace_ethtool_test_plan.rst > @@ -34,31 +34,42 @@ > Userspace Ethtool > ================== > > -This feature is designed to provide one rte_ethtool shim layer based on > -rte_ethdev API. The implementation also along with a command prompt > driven > -demonstration application. It only contained of 18 popular used Ethtool > and > -Netdevice ops as described in rte_ethtool.h. > +This feature is designed to provide one rte_ethtool shim layer based on > +rte_ethdev API. The Ethtool sample application shows an implementation > +of an ethtool-like API and provides a console environment that allows > +its use to query and change Ethernet card parameters. The Ethtool sample > +is based upon a simple L2 frame reflector. > > Prerequisites > ============= > - > +notice: On FVL, test case "test_dump_driver_info" need a physical link > disconnect, > +this case must do manually at this condition. > + > Assume port 0 and 1 are connected to the traffic generator, to run the > test > -application in linuxapp environment with 4 lcores, 2 ports. > +application in linux app environment with 4 lcores, 2 ports. > > ethtool -c f -n 4 > > -The sample should be validated on Forville, Niantic and i350 Nics. > +The sample should be validated on Forville, Niantic and i350 Nics. > + > +other requirements: > +*. igxbe driver (version >= 4.3.13). > +*. ethtool of linux is the default reference tool. > +*. md5sum is a tool to do dumped bin format file comparison. > +*. insert two nic cards on No.0 socket > > -Test Case: Dump driver infor test > +Test Case: Dump driver information test > ================================= > -User "drvinfo" command to dump driver information and then check that > dumped > -information was exactly the same as fact. > +User "drvinfo" command to dump driver information and then check that > +dumped information, which are dumped separately by dpdk's ethtool and > +linux's ethtool, were exactly the same. > > EthApp> drvinfo > Port 0 driver: rte_ixgbe_pmd (ver: RTE 2.1.0) > Port 1 driver: rte_ixgbe_pmd (ver: RTE 2.1.0) > > Use "link" command to dump all ports link status. > +Notice:: On FVL, link detect need a physical link disconnect. > EthApp> link > Port 0: Up > Port 1: Up > @@ -68,7 +79,7 @@ Change tester port link status to down and re-check link > status. > Port 0: Down > Port 1: Down > > -Send few packets to l2fwd and check that command "portstats" dumped > correct > +Send a few packets to l2fwd and check that command "portstats" dumps > correct > port statistics. > EthApp> portstats 0 > Port 0 stats > @@ -78,34 +89,38 @@ port statistics. > Test Case: Retrieve eeprom test > =============================== > Unbind ports from igb_uio and bind them to default driver. > -Dump eeprom binary by ethtool. > +Dump eeprom binary by linux's ethtool and dpdk's ethtool separately. > > ethtool --eeprom-dump INTF_0 raw on > ethtool_eeprom_0.bin > ethtool --eeprom-dump INTF_1 raw on > ethtool_eeprom_1.bin > > -Retrieve eeprom on specified port and compare csum with the file dumped > by ethtool. > +Retrieve eeprom on specified port using dpdk's ethtool and > +compare csum with the file dumped by ethtool. > > EthApp> eeprom 0 eeprom_0.bin > EthApp> eeprom 1 eeprom_1.bin > > -md5sum ethtool_eeprom_0.bin > -md5sum eeprom_0.bin > eeprom_0.bin > - > -diff ethtool_eeprom_0.hex eeprom_0.hex > + md5sum ethtool_eeprom_0.bin > + md5sum eeprom_0.bin > + > +compare md5sum value of the two bin files. > > Test Case: Retrieve register test > =============================== > -Retrieve register on specified port, do not known how to check the binary? > +Retrieve register on specified port. > > EthApp> regs 0 reg_0.bin > EthApp> regs 1 reg_1.bin > > Unbind ports from igb_uio and bind them to default driver. > + > + dpdk/tools/dpdk_nic_bind.py --bind=ixgbe x:xx.x > + > Check that dumped register information is correct. > > -ethtool -d INTF_0 raw on file reg_0.bin > -ethtool -d INTF_1 raw on file reg_0.bin > - > +ethtool -d INTF_0 raw off file reg_0.bin > +ethtool -d INTF_1 raw off file reg_0.bin > + > Test Case: Ring param test > ========================== > Dump port 0 ring size by ringparam command and check numbers are correct. > @@ -126,52 +141,30 @@ Port 0 ring paramaeters > Rx Pending: 256 (256 max) > Tx Pending: 2048 (4096 max) > > -Test Case: Pause test > -===================== > -Enable port 0 Rx pause frame and then create two packets flows in IXIA > port. > -One flow is 100000 normally packet and the second flow is pause frame. > -Check that port 0 Rx speed dropped. For example, niantic will drop from > -14.8Mpps to 7.49Mpps. > - > - EthApp> pause 0 rx > - > -Use "parse" command to print port pause status, check that port 0 rx has > been > -paused. > - EthApp> pause 0 > - Port 0: Rx Paused > - > -Unpause port 0 rx and then restart port0, check that packets Rx speed is > normal. > - EthApp> pause 0 none > - EthApp> > +send packet by scapy on Tester > > -Pause port 0 TX pause frame. > - EthApp> pause 0 tx > - > -Use "parse" command to print port pause status, check that port 1 tx has > been > -paused. > - EthApp> pause 0 > - Port 0: Tx Paused > - > -Enable flow control in IXIA port and send packets from IXIA with line > rate. > -Check that IXIA receive flow control packets and IXIA transmit speed > dropped. > -IXIA Rx packets more then Tx packets to check that received pause frame. > - > -Unpause port 0 tx and restart port 0. Then send packets to port0, check > that > -packets forwarded normally from port 0. > - EthApp> pause 0 none > - EthApp> stop 0 > - EthApp> open 0 > +check tx/rx packets > +EthApp> portstats 0 > > Test Case: Vlan test > ==================== > +enable vlan filter flag in main.c of dpdk's ethtool > + > + sed -i -e '/cfg_port.txmode.mq_mode = > ETH_MQ_TX_NONE;$/a\\cfg_port.rxmode.hw_vlan_filter=1;' > examples/ethtool/ethtool-app/main.c > + > +re-compile examples/ethtool > + > + make -C examples/ethtool > + > Add vlan 0 to port 0 and vlan 1 to port1, send packet without vlan to > port0,1 > Verify port0 and port1 recevied vlan packets > + > EthApp> vlan 0 add 0 > VLAN vid 0 added > > EthApp> vlan 1 add 1 > VLAN vid 1 added > - > + > Send packet with vlan0,1 to port0&1. Verify port0 and port1 received vlan > packets > > @@ -189,7 +182,11 @@ port0 and port1 can not receive vlan packet. > Test Case: Mac address test > =========================== > Use "macaddr" command to dump port mac address and then check that dumped > -information is exactly the same as fact. > +information is exactly the same as ifconfig do. > + > +set a new mac address by dpdk's ethtool, send and sniff packet and check > packet > +forwared status > + > EthApp> macaddr 0 > Port 0 MAC Address: XX:XX:XX:XX:XX:XX > EthApp> macaddr 1 > @@ -211,8 +208,8 @@ Use "macaddr" command to change port mac address and > then check mac changed. > MAC address changed > EthApp> macaddr 0 > Port 0 MAC Address: 00:10:00:00:00:00 > - > -Verified mac adress in forwarded packets has been changed. > + > +Verified mac adress in forwarded packets has been changed. > > Test Case: Port config test > =========================== > @@ -224,18 +221,58 @@ Use "open" command to re-enable port0. Send packets > to port0 and verify > packets received and forwarded. > EthApp> open 0 > > - > Test case: Mtu config test > ========================== > -Use "mtu" command to change port 0 mtu from default 1518 to 1000. > +Use "mtu" command to change port 0 mtu from default 1519 to 9000 on > Tester's port. > > -Send packet size over 1000 and check that packet will be detected as > error. > +Send packet size over 1519 and check that packet will be detected as > error. > > - EthApp> mtu 0 1000 > + EthApp> mtu 0 1519 > Port 0 stats > In: 0 (0 bytes) > Out: 0 (0 bytes) > Err: 1 > > -Change mtu to default value and send packet size over 1000 and check that > -packet will normally received. > +Change mtu to default value and send packet size over 1519 and check that > +packet will normally be received. > + > +Test Case: Pause tx/rx test(performance test) > +===================== > +Enable port 0 Rx pause frame and then create two packets flows on IXIA > port. > +One flow is 100000 normally packet and the second flow is pause frame. > +Check that dut's port 0 Rx speed dropped status. For example, niantic > will drop > +from 14.8Mpps to 7.49Mpps. > + > + EthApp> pause 0 rx > + > +Use "pause" command to print dut's port pause status, check that dut's > port 0 rx > +has been paused. > + > + EthApp> pause 0 > + Port 0: Rx Paused > + > +Release pause status of port 0 rx and then restart port 0, check that > packets Rx > +speed is normal. > + EthApp> pause 0 none > + EthApp> > + > +Pause port 0 TX pause frame. > + EthApp> pause 0 tx > + > +Use "pause" command to print port pause status, check that port 0 tx has > been > +paused. > + EthApp> pause 0 > + Port 0: Tx Paused > + > +Enable flow control in IXIA port and send packets from IXIA with line > rate. > +Record line rate before send packet. > +Check that IXIA receive flow control packets and IXIA transmit speed > dropped. > +IXIA Rx packets more then Tx packets to check that received pause > frame.Compare > +the line rates in the time before and after the Pause packets are > injected > + > +Unpause port 0 tx and restart port 0. Then send packets to port0, check > that > +packets forwarded normally from port 0. > + > + EthApp> pause 0 none > + EthApp> stop 0 > + EthApp> open 0 > diff --git a/tests/TestSuite_userspace_ethtool.py > b/tests/TestSuite_userspace_ethtool.py > index c479093..a0f4018 100644 > --- a/tests/TestSuite_userspace_ethtool.py > +++ b/tests/TestSuite_userspace_ethtool.py > @@ -38,12 +38,12 @@ import dts > import time > import re > from test_case import TestCase > -from pmd_output import PmdOutput > from packet import Packet, sniff_packets, load_sniff_packets > import random > from etgen import IxiaPacketGenerator > from settings import HEADER_SIZE > from settings import SCAPY2IXIA > +from utils import RED > > > class TestUserspaceEthtool(TestCase, IxiaPacketGenerator): > @@ -127,6 +127,22 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > else: > return 1518 > > + def resize_linux_eeprom_file(self, dpdk_eeprom_file, > linux_eeprom_file): > + basePath = "/root/dpdk/" > + with open( basePath + dpdk_eeprom_file, 'rb') as fpDpdk: > + dpdk_bytes = fpDpdk.read() > + dpdk_length = len(dpdk_bytes) > + > + with open( basePath + linux_eeprom_file, 'rb') as fplinux: > + linux_bytes = fplinux.read() > + linux_length = len(linux_bytes) > + > + self.verify(dpdk_length <= linux_length, > + "linux ethtool haven't dump out enough data as dpdk > ethtool") > + > + with open( basePath + linux_eeprom_file, 'wb') as fplinux: > + fplinux.write(linux_bytes[:dpdk_length]) > + > def strip_md5(self, filename): > md5_info = self.dut.send_expect("md5sum %s" % filename, "# ") > md5_pattern = r"(\w+) (\w+)" > @@ -155,30 +171,33 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > version = m.group(3) > print dts.GREEN("Detect port %s with %s driver\n" % (port, > driver)) > > - # check link status dump function > - for port in self.ports: > - tester_port = self.tester.get_local_port(port) > - intf = self.tester.get_interface(tester_port) > - self.tester.send_expect("ip link set dev %s down" % intf, "# > ") > - # wait for link stable > - time.sleep(5) > - > - out = self.dut.send_expect("link", "EthApp>", 60) > - link_pattern = r"Port (\d+): (.*)" > - link_infos = out.split("\r\n") > - for link_info in link_infos: > - m = re.match(link_pattern, link_info) > - if m: > - port = m.group(1) > - status = m.group(2) > - self.verify(status == "Down", "Userspace tool failed to > detect link down") > - > - for port in self.ports: > - tester_port = self.tester.get_local_port(port) > - intf = self.tester.get_interface(tester_port) > - self.tester.send_expect("ip link set dev %s up" % intf, "# ") > - # wait for link stable > - time.sleep(5) > + # ethtool doesn't support port disconnect by tools of linux > + # only detect physical link disconnect status > + if self.nic.startswith("fortville") == False: > + # check link status dump function > + for port in self.ports: > + tester_port = self.tester.get_local_port(port) > + intf = self.tester.get_interface(tester_port) > + self.tester.send_expect("ip link set dev %s down" % intf, > "# ") > + # wait for link stable > + time.sleep(5) > + > + out = self.dut.send_expect("link", "EthApp>", 60) > + link_pattern = r"Port (\d+): (.*)" > + link_infos = out.split("\r\n") > + for link_info in link_infos: > + m = re.match(link_pattern, link_info) > + if m: > + port = m.group(1) > + status = m.group(2) > + self.verify(status == "Down", "Userspace tool failed > to detect link down") > + > + for port in self.ports: > + tester_port = self.tester.get_local_port(port) > + intf = self.tester.get_interface(tester_port) > + self.tester.send_expect("ip link set dev %s up" % intf, > "# ") > + # wait for link stable > + time.sleep(5) > > # check port stats function > pkt = Packet(pkt_type='UDP') > @@ -271,8 +290,10 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > > for index in range(len(self.ports)): > md5 = self.strip_md5(portsinfo[index]['eeprom_file']) > + > self.resize_linux_eeprom_file( portsinfo[index]['eeprom_file'], > portsinfo[index]['ethtool_eeprom']) > md5_ref = self.strip_md5(portsinfo[index]['ethtool_eeprom']) > - print dts.GREEN("Reference eeprom md5 %s" % md5_ref) > + print dts.GREEN("Reference eeprom md5 %s" % md5) > + print dts.GREEN("Reference eeprom md5_ref %s" % md5_ref) > self.verify(md5 == md5_ref, "Dumped eeprom not same as linux > dumped") > > def test_ring_parameter(self): > @@ -323,7 +344,6 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > tester_port = self.tester.get_local_port(port) > intf = self.tester.get_interface(tester_port) > pkt.send_pkt(tx_port=intf) > - time.sleep(2) > rx_pkts, tx_pkts = self.strip_portstats(port) > self.verify(rx_pkts == ori_rx_pkts + 1, "Failed to Rx vlan > packet") > self.verify(tx_pkts == ori_tx_pkts + 1, "Failed to Tx vlan > packet") > @@ -372,8 +392,8 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > inst = sniff_packets(intf, timeout=5) > pkt.send_pkt(tx_port=intf) > pkts = load_sniff_packets(inst) > - self.verify(len(pkts) == 2, "Packet not forwarded as > expected") > - src_mac = pkts[1].strip_layer_element("layer2", "src") > + self.verify(len(pkts) == 1, "Packet not forwarded as > expected") > + src_mac = pkts[0].strip_layer_element("layer2", "src") > self.verify(src_mac == valid_mac, "Forwarded packet not match > default mac") > > # check multicase will not be valid mac > @@ -405,11 +425,17 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > rx_pkts, tx_pkts = self.strip_portstats(index) > self.verify(rx_pkts == ori_rx_pkts, "Failed to stop port") > # restart port and check packet can normally forwarded > + time.sleep(2) > self.dut.send_expect("open %d" % index, "EthApp>") > # wait few time for port ready > - time.sleep(0.5) > + rx_pkts, tx_pkts = self.strip_portstats(index) > + time.sleep(2) > pkt.send_pkt(tx_port=intf) > rx_pkts_open, tx_pkts_open = self.strip_portstats(index) > + if rx_pkts_open != rx_pkts + 1: > + RED( "Failed to reopen port rx") > + if tx_pkts_open != tx_pkts + 1: > + RED( "Failed to reopen port tx") > self.verify(rx_pkts_open == rx_pkts + 1, "Failed to reopen > port rx") > self.verify(tx_pkts_open == tx_pkts + 1, "Failed to reopen > port tx") > > @@ -430,11 +456,12 @@ class TestUserspaceEthtool(TestCase, > IxiaPacketGenerator): > self.tester.send_expect("ifconfig %s mtu 9000" % (intf), "# ") > for mtu in mtus: > self.dut.send_expect("mtu %d %d" % (index, mtu), > "EthApp>") > + ori_rx_pkts, _ = self.strip_portstats(index) > pkt_size = mtu + HEADER_SIZE['eth'] > pkt = Packet(pkt_len=pkt_size) > pkt.send_pkt(tx_port=intf) > rx_pkts, _ = self.strip_portstats(index) > - self.verify(rx_pkts == 1, "Packet match mtu not forwarded > as expected") > + self.verify(rx_pkts == ori_rx_pkts, "Packet match mtu not > forwarded as expected") > pkt = Packet(pkt_len=mtu + 1) > pkt.send_pkt(tx_port=intf) > rx_pkts_over, _ = self.strip_portstats(index) > -- > 1.9.3 ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-05-23 20:30 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-05-05 2:59 [dts] [PATCH V1]userspace_ethtool: update test script and test plan Yufen Mo 2016-05-05 2:59 ` Yufen Mo 2016-05-23 20:29 ` Liu, Yong -- strict thread matches above, loose matches on Subject: below -- 2016-05-05 2:23 Yufen Mo 2016-05-05 2:23 ` Yufen Mo 2016-05-05 2:35 ` Liu, Yong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).