* [dts] [PATCH v1 1/2] test_plans: add test plan for distributor library and sample @ 2017-07-26 2:56 Marvin Liu 2017-07-26 2:56 ` [dts] [PATCH v1 2/2] tests: add test suite " Marvin Liu 2017-07-26 3:41 ` [dts] [PATCH v2 0/2] support distributor unit and function test Marvin Liu 0 siblings, 2 replies; 5+ messages in thread From: Marvin Liu @ 2017-07-26 2:56 UTC (permalink / raw) To: dts; +Cc: Marvin Liu Signed-off-by: Marvin Liu <yong.liu@intel.com> diff --git a/test_plans/distributor_test_plan.rst b/test_plans/distributor_test_plan.rst new file mode 100644 index 0000000..cd4e71b --- /dev/null +++ b/test_plans/distributor_test_plan.rst @@ -0,0 +1,154 @@ +.. Copyright (c) <2017>, Intel Corporation + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + - Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + - Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + + - Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + OF THE POSSIBILITY OF SUCH DAMAGE. + +================== +Packet distributor +================== +Packet Distributor library is a library designed to be used for dynamic +load balancing of traffic while supporting single packet at a time operation. +When using this library, the logical cores in use are to be considered in +several roles: + +rx lcore: responsible for receive packets from different ports and enqueue + +distributor lcore: responsible for load balancing or distributing packets + +worker locres: responsible for receiving the packets from the distributor + and operating on them. + +tx lcore: responsible for dequeue packets from distrbutor and transmit them + + +Test Case: Distributor unit test +================================ +Start test application and run distributor unit test:: + + test -c f -n 4 -- -i + RTE>>distributor_autotest + +Verify burst distributor API unit test passed + +Test Case: Distributor performance unit test +============================================ +Start test application and run distributor unit test:: + + test -c f -n 4 -- -i + RTE>>distributor_perf_autotest + +Compared CPU cycles for normal distributor and burst API + +Verify burst distributor API cost much less cycles then normal version + +Test Case: Distributor packet check +=================================== +Start distributor sample with one worker:: + + distributor_app -c 0x7c -n 4 -- -p 0x1 + +Send few packets (less then burst size) with sequence index which indicated in +ip dst address + +Check forwarded packets are all in sequence and content not changed + +Send packets equal to burst size with sequence index + +Check forwarded packets are all in sequence and content not changed + +Send packets over burst size with sequence index + +Check forwarded packets are all in sequence and content not changed + +Test Case: Distributor with workers +=================================== +Start distributor sample with two workers:: + + distributor_app -c 0xfc -n 4 -- -p 0x1 + +Send several packets with ip address increasing + +Check packets distributed to different workers + +Check all packets have been sent back from tx lcore + +Repeat step 1 to step4 with 4(3fc)/8(3ffc)/16(0x3ffffc)/32(0xffff0003ffffc) +workers + +Test case: Distribute with maximum workers +========================================== +Start distributor sample with 63(0xeffffffffffffffff0) workers + +Send several packets with ip address increasing + +Check packets distributed to different workers + +Check all packets have been sent back from tx lcore + +Test Case: Distributor with multiple input ports +================================================ +Start distributor sample with two workers and two ports:: + + distributor_app -c 0x7c -n 4 -- -p 0x3 + +Send packets with sequence indicated in udp port id + +Check forwarded packets are all in sequence and content not changed + +Test case: Distribute performance +================================= +The number of workers are configured through the command line interface of the +application: + +The test report should provide the measurements(mpps and % of the line rate) +for each action in lcores as listed in the table below: + + +----+---------+------------------+------------------+------------------+------------------+------------------+------------------+ + | # |Number of| Throughput Rate | Throughput Rate | Throughput Rate | Throughput Rate | Throughput Rate | Throughput Rate | + | |workers | Rx received | Rx core enqueued | Distributor sent | Tx core dequeued | Tx transmitted | Pkts out | + | | +------------------+------------------+------------------+------------------+------------------+------------------+ + | | | mpps | % | mpps | % | mpps | % | mpps | % | mpps | % | mpps | % | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 1 | 1 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 2 | 2 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 3 | 3 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 4 | 4 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 5 | 8 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 6 | 16 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 7 | 32 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 8 | 63 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ -- 1.9.3 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts] [PATCH v1 2/2] tests: add test suite for distributor library and sample 2017-07-26 2:56 [dts] [PATCH v1 1/2] test_plans: add test plan for distributor library and sample Marvin Liu @ 2017-07-26 2:56 ` Marvin Liu 2017-07-26 3:41 ` [dts] [PATCH v2 0/2] support distributor unit and function test Marvin Liu 1 sibling, 0 replies; 5+ messages in thread From: Marvin Liu @ 2017-07-26 2:56 UTC (permalink / raw) To: dts; +Cc: Marvin Liu Signed-off-by: Marvin Liu <yong.liu@intel.com> diff --git a/tests/TestSuite_distributor.py b/tests/TestSuite_distributor.py new file mode 100644 index 0000000..63edd52 --- /dev/null +++ b/tests/TestSuite_distributor.py @@ -0,0 +1,286 @@ +# BSD LICENSE +# +# Copyright(c) 2010-2017 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +""" +DPDK Test suite. +""" +import re +import utils +from test_case import TestCase +from etgen import IxiaPacketGenerator + + +class TestDistributor(TestCase, IxiaPacketGenerator): + def set_up_all(self): + """ + Run at the start of each test suite. + """ + self.tester.extend_external_packet_generator(TestDistributor, self) + + out = self.dut.send_expect("make -C test -j", "#") + self.verify("Error" not in out, "Compilation error") + self.verify("No such" not in out, "Compilation error") + + # reduce tx queues for enable many workers + self.dut.send_expect("sed -i -e 's/.*txRings = .*/\\tconst uint16_t rxRings = 1, txRings = 1;/' ./examples/distributor/main.c", "#") + out = self.dut.build_dpdk_apps("./examples/distributor") + self.verify("Error" not in out, "Compilation error") + self.verify("No such" not in out, "Compilation error") + + self.dut_ports = self.dut.get_ports() + + + def set_up(self): + """ + Run before each test case. + """ + pass + + def test_distributor_unit(self): + """ + Run distributor unit test + """ + self.dut.send_expect("./test/test/test -n 1 -c f", "RTE>>", 60) + out = self.dut.send_expect("distributor_autotest", "RTE>>", 30) + self.dut.send_expect("quit", "# ") + self.verify("Test OK" in out, "Test failed") + + def test_distributor_unit_perf(self): + """ + Run distributor unit perf test + """ + self.dut.send_expect("./test/test/test -n 1 -c f", "RTE>>", 60) + out = self.dut.send_expect("distributor_perf_autotest", "RTE>>", 120) + cycles_single = self.strip_cycles(out, "single") + cycles_burst = self.strip_cycles(out, "burst") + self.logger.info("Cycles for single mode is %d burst mode is %d" + % (cycles_single, cycles_burst)) + self.dut.send_expect("quit", "# ") + self.verify("Test OK" in out, "Test failed") + self.verify(cycles_single > cycles_burst * 2, + "Burst performance should be much better") + + def test_perf_distributor(self): + """ + Run distributor perf test, recorded statistic of Rx/Enqueue/Sent/Dequeue/Tx + """ + self.verify(len(self.dut_ports) >=1, "Not enough ports") + workers = [1, 2, 3, 4, 8, 16, 32] + table_header = ["Number of workers", + "Throughput Rate Rx received", + "Throughput Rate Rx core enqueued", + "Throughput Rate Distributor Sent", + "Throughput Rate Tx core dequeued", + "Throughput Rate Tx transmitted", + "Throughput Rate Pkts out", + "Throughput Rate Pkts out line rate"] + + app = "./examples/distributor/build/distributor_app" + # output port is calculated from overall ports number + cmd_fmt = "%s -c %s -n %d -w %s -- -p 0x1" + socket = self.dut.get_numa_id(self.dut_ports[0]) + + self.tester.scapy_append('wrpcap("distributor.pcap", [Ether()/IP()/("X"*26)])') + self.tester.scapy_execute() + tgen_input = [] + rx_port = self.tester.get_local_port(self.dut_ports[0]) + tx_port = self.tester.get_local_port(self.dut_ports[0]) + tgen_input.append((tx_port, rx_port, "distributor.pcap")) + + self.result_table_create(table_header) + for worker_num in workers: + # Rx core/distributor core/Tx core/stats core + cores = self.dut.get_core_list("1S/%dC/1T" % (worker_num + 4), socket) + # If can't get enough core from one socket, just use all lcores + if len(cores) < (worker_num + 4): + cores = self._get_thread_lcore(worker_num + 4) + + cmd = cmd_fmt % (app, utils.create_mask(cores), + self.dut.get_memory_channels(), + self.dut.get_port_pci(self.dut_ports[0])) + + self.dut.send_expect(cmd, "doing packet RX", timeout=30) + + self.tester.ixia_packet_gen.hook_transmission_func = self.hook_transmission_func + _, pps = self.tester.traffic_generator_throughput(tgen_input, delay=2) + + self.dut.send_expect("^C", "#") + + pps/= 1000000.0 + rx, enq, sent, deq, trans = self.strip_performance_data(self.app_output) + rate = pps * 100 / float(self.wirespeed(self.nic, 64, 1)) + self.result_table_add([worker_num, rx, enq, sent, deq, trans, pps, float('%.3f' % rate)]) + + self.result_table_print() + + def test_maximum_workers(self): + """ + Check distributor app work fine with maximum workers + """ + self.verify(len(self.dut_ports) >=1, "Not enough ports") + app = "./examples/distributor/build/distributor_app" + cmd_fmt = "%s -c %s -n %d -w %s -- -p 0x1" + + out = self.dut.send_expect("sed -n '/#define RTE_DISTRIB_MAX_WORKERS/p' lib/librte_distributor/rte_distributor_private.h", "# ") + reg_match = r"#define RTE_DISTRIB_MAX_WORKERS (.*)" + m = re.match(reg_match, out) + self.verify(m, "Can't find maximum worker number") + + max_workers = int(m.group(1)) + cores = self._get_thread_lcore(max_workers - 1 + 4) + + cmd = cmd_fmt % (app, utils.create_mask(cores), + self.dut.get_memory_channels(), + self.dut.get_port_pci(self.dut_ports[0])) + + self.dut.send_expect(cmd, "doing packet RX", timeout=30) + + tx_port = self.tester.get_local_port(self.dut_ports[0]) + tgen_input = [(tx_port, tx_port)] + self.tester.check_random_pkts(tgen_input, pktnum=256, seq_check=True) + + self.dut.send_expect("^C", "#") + + def test_multiple_ports(self): + """ + Check distributor app work fine with multiple ports + """ + app = "./examples/distributor/build/distributor_app" + self.verify(len(self.dut_ports) >=2, "Not enough ports") + cmd_fmt = "%s -c %s -n %d -w %s -w %s -- -p 0x3" + socket = self.dut.get_numa_id(self.dut_ports[0]) + cores = self.dut.get_core_list("1S/%dC/1T" % (2 + 4), socket) + + cmd = cmd_fmt % (app, utils.create_mask(cores), + self.dut.get_memory_channels(), + self.dut.get_port_pci(self.dut_ports[0]), + self.dut.get_port_pci(self.dut_ports[1])) + + self.dut.send_expect(cmd, "doing packet RX", timeout=30) + + tx_port = self.tester.get_local_port(self.dut_ports[0]) + rx_port = self.tester.get_local_port(self.dut_ports[1]) + tgen_input = [(tx_port, rx_port)] + self.tester.check_random_pkts(tgen_input, pktnum=256, seq_check=True) + + tgen_input = [(rx_port, tx_port)] + self.tester.check_random_pkts(tgen_input, pktnum=256, seq_check=True) + + self.dut.send_expect("^C", "#") + + def _get_thread_lcore(self, core_num): + def strip_core(x): + return(int(x['thread'])) + cores = map(strip_core, self.dut.cores[0:core_num]) + return cores + + def hook_transmission_func(self): + self.app_output = self.dut.session.get_session_before(timeout=2) + + def strip_performance_data(self, output=""): + """ + Strip throughput of each stage in threads + RX Thread: + Port 0 Pktsin : + - Received: + - Returned: + - Enqueued: + - Dropped: + Distributor thread: + - In: + - Returned: + - Sent: + - Dropped: + TX thread: + - Dequeued: + Port 0 Pktsout: + - Transmitted: + - Dropped: + """ + # skip the last one, we use the next one + output = output[:output.rfind("RX Thread")] + output = output[output.rfind("RX Thread"):] + rec_rate = 0.0 + enq_rate = 0.0 + sent_rate = 0.0 + deq_rate = 0.0 + trans_rate = 0.0 + for line in output.splitlines(): + if "Received" in line: + rec_rate = float(line.split()[2]) + elif "Enqueued" in line: + enq_rate = float(line.split()[2]) + elif "Sent" in line: + sent_rate = float(line.split()[2]) + elif "Dequeued" in line: + deq_rate = float(line.split()[2]) + elif "Transmitted" in line: + trans_rate = float(line.split()[2]) + + return (rec_rate, enq_rate, sent_rate, deq_rate, trans_rate) + + def strip_cycles(self, out="", mode="single"): + """ + Strip per packet cycles from output like: + Time per burst: 12542 + Time per packet: 195 + """ + out = out[out.index("%s mode" % mode):] + lines = out.splitlines() + cycles = lines[2].split()[3] + return int(cycles) + + def ip(self, port, frag, src, proto, tos, dst, chksum, len, options, version, flags, ihl, ttl, id): + self.add_tcl_cmd("protocol config -name ip") + self.add_tcl_cmd('ip config -sourceIpAddr "%s"' % src) + self.add_tcl_cmd("ip config -sourceIpAddrMode ipIdle") + self.add_tcl_cmd('ip config -destIpAddr "%s"' % dst) + self.add_tcl_cmd("ip config -destIpAddrMode ipRandom") + self.add_tcl_cmd("ip config -ttl %d" % ttl) + self.add_tcl_cmd("ip config -totalLength %d" % len) + self.add_tcl_cmd("ip config -fragment %d" % frag) + self.add_tcl_cmd("ip config -ipProtocol ipV4ProtocolReserved255") + self.add_tcl_cmd("ip config -identifier %d" % id) + self.add_tcl_cmd("stream config -framesize %d" % (len + 18)) + self.add_tcl_cmd("ip set %d %d %d" % (self.chasId, port['card'], port['port'])) + + def tear_down(self): + """ + Run after each test case. + """ + pass + + def tear_down_all(self): + """ + Run after each test suite. + """ + self.dut.send_expect("sed -i -e 's/.*txRings = .*/\\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;/' ./examples/distributor/main.c", "#") + pass -- 1.9.3 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts] [PATCH v2 0/2] support distributor unit and function test 2017-07-26 2:56 [dts] [PATCH v1 1/2] test_plans: add test plan for distributor library and sample Marvin Liu 2017-07-26 2:56 ` [dts] [PATCH v1 2/2] tests: add test suite " Marvin Liu @ 2017-07-26 3:41 ` Marvin Liu 2017-07-26 3:42 ` [dts] [PATCH v2 1/2] test_plans: add test plan for distributor library and sample Marvin Liu 2017-07-26 3:42 ` [dts] [PATCH v2 2/2] tests: add test suite " Marvin Liu 1 sibling, 2 replies; 5+ messages in thread From: Marvin Liu @ 2017-07-26 3:41 UTC (permalink / raw) To: dts v2: fix pep8 issue Marvin Liu (2): test_plans: add test plan for distributor library and sample tests: add test suite for distributor library and sample test_plans/distributor_test_plan.rst | 154 +++++++++++++++++++ tests/TestSuite_distributor.py | 283 +++++++++++++++++++++++++++++++++++ 2 files changed, 437 insertions(+) create mode 100644 test_plans/distributor_test_plan.rst create mode 100644 tests/TestSuite_distributor.py -- 1.9.3 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts] [PATCH v2 1/2] test_plans: add test plan for distributor library and sample 2017-07-26 3:41 ` [dts] [PATCH v2 0/2] support distributor unit and function test Marvin Liu @ 2017-07-26 3:42 ` Marvin Liu 2017-07-26 3:42 ` [dts] [PATCH v2 2/2] tests: add test suite " Marvin Liu 1 sibling, 0 replies; 5+ messages in thread From: Marvin Liu @ 2017-07-26 3:42 UTC (permalink / raw) To: dts; +Cc: Marvin Liu Signed-off-by: Marvin Liu <yong.liu@intel.com> diff --git a/test_plans/distributor_test_plan.rst b/test_plans/distributor_test_plan.rst new file mode 100644 index 0000000..cd4e71b --- /dev/null +++ b/test_plans/distributor_test_plan.rst @@ -0,0 +1,154 @@ +.. Copyright (c) <2017>, Intel Corporation + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + - Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + + - Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + + - Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE + COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, + INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES + (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, + STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED + OF THE POSSIBILITY OF SUCH DAMAGE. + +================== +Packet distributor +================== +Packet Distributor library is a library designed to be used for dynamic +load balancing of traffic while supporting single packet at a time operation. +When using this library, the logical cores in use are to be considered in +several roles: + +rx lcore: responsible for receive packets from different ports and enqueue + +distributor lcore: responsible for load balancing or distributing packets + +worker locres: responsible for receiving the packets from the distributor + and operating on them. + +tx lcore: responsible for dequeue packets from distrbutor and transmit them + + +Test Case: Distributor unit test +================================ +Start test application and run distributor unit test:: + + test -c f -n 4 -- -i + RTE>>distributor_autotest + +Verify burst distributor API unit test passed + +Test Case: Distributor performance unit test +============================================ +Start test application and run distributor unit test:: + + test -c f -n 4 -- -i + RTE>>distributor_perf_autotest + +Compared CPU cycles for normal distributor and burst API + +Verify burst distributor API cost much less cycles then normal version + +Test Case: Distributor packet check +=================================== +Start distributor sample with one worker:: + + distributor_app -c 0x7c -n 4 -- -p 0x1 + +Send few packets (less then burst size) with sequence index which indicated in +ip dst address + +Check forwarded packets are all in sequence and content not changed + +Send packets equal to burst size with sequence index + +Check forwarded packets are all in sequence and content not changed + +Send packets over burst size with sequence index + +Check forwarded packets are all in sequence and content not changed + +Test Case: Distributor with workers +=================================== +Start distributor sample with two workers:: + + distributor_app -c 0xfc -n 4 -- -p 0x1 + +Send several packets with ip address increasing + +Check packets distributed to different workers + +Check all packets have been sent back from tx lcore + +Repeat step 1 to step4 with 4(3fc)/8(3ffc)/16(0x3ffffc)/32(0xffff0003ffffc) +workers + +Test case: Distribute with maximum workers +========================================== +Start distributor sample with 63(0xeffffffffffffffff0) workers + +Send several packets with ip address increasing + +Check packets distributed to different workers + +Check all packets have been sent back from tx lcore + +Test Case: Distributor with multiple input ports +================================================ +Start distributor sample with two workers and two ports:: + + distributor_app -c 0x7c -n 4 -- -p 0x3 + +Send packets with sequence indicated in udp port id + +Check forwarded packets are all in sequence and content not changed + +Test case: Distribute performance +================================= +The number of workers are configured through the command line interface of the +application: + +The test report should provide the measurements(mpps and % of the line rate) +for each action in lcores as listed in the table below: + + +----+---------+------------------+------------------+------------------+------------------+------------------+------------------+ + | # |Number of| Throughput Rate | Throughput Rate | Throughput Rate | Throughput Rate | Throughput Rate | Throughput Rate | + | |workers | Rx received | Rx core enqueued | Distributor sent | Tx core dequeued | Tx transmitted | Pkts out | + | | +------------------+------------------+------------------+------------------+------------------+------------------+ + | | | mpps | % | mpps | % | mpps | % | mpps | % | mpps | % | mpps | % | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 1 | 1 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 2 | 2 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 3 | 3 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 4 | 4 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 5 | 8 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 6 | 16 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 7 | 32 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ + | 8 | 63 | | | | | | | | | | | | | + +----+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+--------+---------+ -- 1.9.3 ^ permalink raw reply [flat|nested] 5+ messages in thread
* [dts] [PATCH v2 2/2] tests: add test suite for distributor library and sample 2017-07-26 3:41 ` [dts] [PATCH v2 0/2] support distributor unit and function test Marvin Liu 2017-07-26 3:42 ` [dts] [PATCH v2 1/2] test_plans: add test plan for distributor library and sample Marvin Liu @ 2017-07-26 3:42 ` Marvin Liu 1 sibling, 0 replies; 5+ messages in thread From: Marvin Liu @ 2017-07-26 3:42 UTC (permalink / raw) To: dts; +Cc: Marvin Liu Signed-off-by: Marvin Liu <yong.liu@intel.com> diff --git a/tests/TestSuite_distributor.py b/tests/TestSuite_distributor.py new file mode 100644 index 0000000..a900f5e --- /dev/null +++ b/tests/TestSuite_distributor.py @@ -0,0 +1,283 @@ +# BSD LICENSE +# +# Copyright(c) 2010-2017 Intel Corporation. All rights reserved. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in +# the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Intel Corporation nor the names of its +# contributors may be used to endorse or promote products derived +# from this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +""" +DPDK Test suite. +""" +import re +import utils +from test_case import TestCase +from etgen import IxiaPacketGenerator + + +class TestDistributor(TestCase, IxiaPacketGenerator): + def set_up_all(self): + """ + Run at the start of each test suite. + """ + self.tester.extend_external_packet_generator(TestDistributor, self) + + out = self.dut.send_expect("make -C test -j", "#") + self.verify("Error" not in out, "Compilation error") + self.verify("No such" not in out, "Compilation error") + + # reduce tx queues for enable many workers + self.dut.send_expect("sed -i -e 's/.*txRings = .*/\\tconst uint16_t rxRings = 1, txRings = 1;/' ./examples/distributor/main.c", "#") + out = self.dut.build_dpdk_apps("./examples/distributor") + self.verify("Error" not in out, "Compilation error") + self.verify("No such" not in out, "Compilation error") + + self.dut_ports = self.dut.get_ports() + self.app = "./examples/distributor/build/distributor_app" + + def set_up(self): + """ + Run before each test case. + """ + pass + + def test_distributor_unit(self): + """ + Run distributor unit test + """ + self.dut.send_expect("./test/test/test -n 1 -c f", "RTE>>", 60) + out = self.dut.send_expect("distributor_autotest", "RTE>>", 30) + self.dut.send_expect("quit", "# ") + self.verify("Test OK" in out, "Test failed") + + def test_distributor_unit_perf(self): + """ + Run distributor unit perf test + """ + self.dut.send_expect("./test/test/test -n 1 -c f", "RTE>>", 60) + out = self.dut.send_expect("distributor_perf_autotest", "RTE>>", 120) + cycles_single = self.strip_cycles(out, "single") + cycles_burst = self.strip_cycles(out, "burst") + self.logger.info("Cycles for single mode is %d burst mode is %d" + % (cycles_single, cycles_burst)) + self.dut.send_expect("quit", "# ") + self.verify("Test OK" in out, "Test failed") + self.verify(cycles_single > cycles_burst * 2, + "Burst performance should be much better") + + def test_perf_distributor(self): + """ + Run distributor perf test, recorded statistic of Rx/Enqueue/Sent/Dequeue/Tx + """ + self.verify(len(self.dut_ports) >= 1, "Not enough ports") + workers = [1, 2, 3, 4, 8, 16, 32] + table_header = ["Number of workers", + "Throughput Rate Rx received", + "Throughput Rate Rx core enqueued", + "Throughput Rate Distributor Sent", + "Throughput Rate Tx core dequeued", + "Throughput Rate Tx transmitted", + "Throughput Rate Pkts out", + "Throughput Rate Pkts out line rate"] + + # output port is calculated from overall ports number + cmd_fmt = "%s -c %s -n %d -w %s -- -p 0x1" + socket = self.dut.get_numa_id(self.dut_ports[0]) + + self.tester.scapy_append('wrpcap("distributor.pcap", [Ether()/IP()/("X"*26)])') + self.tester.scapy_execute() + tgen_input = [] + rx_port = self.tester.get_local_port(self.dut_ports[0]) + tx_port = self.tester.get_local_port(self.dut_ports[0]) + tgen_input.append((tx_port, rx_port, "distributor.pcap")) + + self.result_table_create(table_header) + for worker_num in workers: + # Rx core/distributor core/Tx core/stats core + cores = self.dut.get_core_list("1S/%dC/1T" % (worker_num + 4), socket) + # If can't get enough core from one socket, just use all lcores + if len(cores) < (worker_num + 4): + cores = self._get_thread_lcore(worker_num + 4) + + cmd = cmd_fmt % (self.app, utils.create_mask(cores), + self.dut.get_memory_channels(), + self.dut.get_port_pci(self.dut_ports[0])) + + self.dut.send_expect(cmd, "doing packet RX", timeout=30) + + self.tester.ixia_packet_gen.hook_transmission_func = self.hook_transmission_func + _, pps = self.tester.traffic_generator_throughput(tgen_input, delay=2) + + self.dut.send_expect("^C", "#") + + pps /= 1000000.0 + rx, enq, sent, deq, trans = self.strip_performance_data(self.app_output) + rate = pps * 100 / float(self.wirespeed(self.nic, 64, 1)) + self.result_table_add([worker_num, rx, enq, sent, deq, trans, pps, float('%.3f' % rate)]) + + self.result_table_print() + + def test_maximum_workers(self): + """ + Check distributor app work fine with maximum workers + """ + self.verify(len(self.dut_ports) >= 1, "Not enough ports") + cmd_fmt = "%s -c %s -n %d -w %s -- -p 0x1" + + out = self.dut.send_expect("sed -n '/#define RTE_DISTRIB_MAX_WORKERS/p' lib/librte_distributor/rte_distributor_private.h", "# ") + reg_match = r"#define RTE_DISTRIB_MAX_WORKERS (.*)" + m = re.match(reg_match, out) + self.verify(m, "Can't find maximum worker number") + + max_workers = int(m.group(1)) + cores = self._get_thread_lcore(max_workers - 1 + 4) + + cmd = cmd_fmt % (self.app, utils.create_mask(cores), + self.dut.get_memory_channels(), + self.dut.get_port_pci(self.dut_ports[0])) + + self.dut.send_expect(cmd, "doing packet RX", timeout=30) + + tx_port = self.tester.get_local_port(self.dut_ports[0]) + tgen_input = [(tx_port, tx_port)] + self.tester.check_random_pkts(tgen_input, pktnum=256, seq_check=True) + + self.dut.send_expect("^C", "#") + + def test_multiple_ports(self): + """ + Check distributor app work fine with multiple ports + """ + self.verify(len(self.dut_ports) >= 2, "Not enough ports") + cmd_fmt = "%s -c %s -n %d -w %s -w %s -- -p 0x3" + socket = self.dut.get_numa_id(self.dut_ports[0]) + cores = self.dut.get_core_list("1S/%dC/1T" % (2 + 4), socket) + + cmd = cmd_fmt % (self.app, utils.create_mask(cores), + self.dut.get_memory_channels(), + self.dut.get_port_pci(self.dut_ports[0]), + self.dut.get_port_pci(self.dut_ports[1])) + + self.dut.send_expect(cmd, "doing packet RX", timeout=30) + + tx_port = self.tester.get_local_port(self.dut_ports[0]) + rx_port = self.tester.get_local_port(self.dut_ports[1]) + tgen_input = [(tx_port, rx_port)] + self.tester.check_random_pkts(tgen_input, pktnum=256, seq_check=True) + + tgen_input = [(rx_port, tx_port)] + self.tester.check_random_pkts(tgen_input, pktnum=256, seq_check=True) + + self.dut.send_expect("^C", "#") + + def _get_thread_lcore(self, core_num): + def strip_core(x): + return(int(x['thread'])) + cores = map(strip_core, self.dut.cores[0:core_num]) + return cores + + def hook_transmission_func(self): + self.app_output = self.dut.session.get_session_before(timeout=2) + + def strip_performance_data(self, output=""): + """ + Strip throughput of each stage in threads + RX Thread: + Port 0 Pktsin : + - Received: + - Returned: + - Enqueued: + - Dropped: + Distributor thread: + - In: + - Returned: + - Sent: + - Dropped: + TX thread: + - Dequeued: + Port 0 Pktsout: + - Transmitted: + - Dropped: + """ + # skip the last one, we use the next one + output = output[:output.rfind("RX Thread")] + output = output[output.rfind("RX Thread"):] + rec_rate = 0.0 + enq_rate = 0.0 + sent_rate = 0.0 + deq_rate = 0.0 + trans_rate = 0.0 + for line in output.splitlines(): + if "Received" in line: + rec_rate = float(line.split()[2]) + elif "Enqueued" in line: + enq_rate = float(line.split()[2]) + elif "Sent" in line: + sent_rate = float(line.split()[2]) + elif "Dequeued" in line: + deq_rate = float(line.split()[2]) + elif "Transmitted" in line: + trans_rate = float(line.split()[2]) + + return (rec_rate, enq_rate, sent_rate, deq_rate, trans_rate) + + def strip_cycles(self, out="", mode="single"): + """ + Strip per packet cycles from output like: + Time per burst: 12542 + Time per packet: 195 + """ + out = out[out.index("%s mode" % mode):] + lines = out.splitlines() + cycles = lines[2].split()[3] + return int(cycles) + + def ip(self, port, frag, src, proto, tos, dst, chksum, len, options, version, flags, ihl, ttl, id): + self.add_tcl_cmd("protocol config -name ip") + self.add_tcl_cmd('ip config -sourceIpAddr "%s"' % src) + self.add_tcl_cmd("ip config -sourceIpAddrMode ipIdle") + self.add_tcl_cmd('ip config -destIpAddr "%s"' % dst) + self.add_tcl_cmd("ip config -destIpAddrMode ipRandom") + self.add_tcl_cmd("ip config -ttl %d" % ttl) + self.add_tcl_cmd("ip config -totalLength %d" % len) + self.add_tcl_cmd("ip config -fragment %d" % frag) + self.add_tcl_cmd("ip config -ipProtocol ipV4ProtocolReserved255") + self.add_tcl_cmd("ip config -identifier %d" % id) + self.add_tcl_cmd("stream config -framesize %d" % (len + 18)) + self.add_tcl_cmd("ip set %d %d %d" % (self.chasId, port['card'], port['port'])) + + def tear_down(self): + """ + Run after each test case. + """ + pass + + def tear_down_all(self): + """ + Run after each test suite. + """ + self.dut.send_expect("sed -i -e 's/.*txRings = .*/\\tconst uint16_t rxRings = 1, txRings = rte_lcore_count() - 1;/' ./examples/distributor/main.c", "#") + pass -- 1.9.3 ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2017-07-26 3:44 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2017-07-26 2:56 [dts] [PATCH v1 1/2] test_plans: add test plan for distributor library and sample Marvin Liu 2017-07-26 2:56 ` [dts] [PATCH v1 2/2] tests: add test suite " Marvin Liu 2017-07-26 3:41 ` [dts] [PATCH v2 0/2] support distributor unit and function test Marvin Liu 2017-07-26 3:42 ` [dts] [PATCH v2 1/2] test_plans: add test plan for distributor library and sample Marvin Liu 2017-07-26 3:42 ` [dts] [PATCH v2 2/2] tests: add test suite " Marvin Liu
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).