From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48]) by dpdk.org (Postfix) with ESMTP id 4227528EE for ; Thu, 30 Nov 2017 10:05:34 +0100 (CET) Received: by mail-wm0-f48.google.com with SMTP id 9so11508093wme.4 for ; Thu, 30 Nov 2017 01:05:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=wYzcZzs83J8UWKgpEk2ymyr9fGWZBUZO5Mfv7LO+njk=; b=owc3Rc8dgszZisGv6OfdNiET7yQO7ektMjfadalVizjZQFlVU7juWptVRS3MxOGx4v 7xTR7UH6v1VVkkokGJk4G2udT5rPZ3ubj2fEgUNlAugH0yJvNeD4dwytcdJELn9/odzF nf0AJkxl0MFkBQcDWt9V1HklnkuM3b5agdxl0lJNjQj67c1XBv7jeQzSb4nd3DjivNGv 1fBIOPY8ewOCRS/IrmLLpRtmx/PgR2ZhndOKze9FhYbuclIbyochZdPWOVIsNkro+Rv6 Qq+f3yzHDMvATyTxjaUU9rXw+ktGvRzmDc2GjZ/QJ68RCkvyYTymwCF78IBp3+m2nr+Y z33A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=wYzcZzs83J8UWKgpEk2ymyr9fGWZBUZO5Mfv7LO+njk=; b=pyCqXF7BKDnabgP/ht4PSEOQyeol068pRoZtImpHmAEyx5kskPaJ9XV49fIc878ZFC WiTpa8oYqiHNR5kQumG8Wq2Le/pKUETBUe5pyAWBcikPwlnAgMp0fij2mqlUmA6F+brj Ru4UPTiQSBHakRM8qVQGGRYYuxfNPHLLglBA1ujbiBmQ3ZNcBYAzbSMkyP8E5viVMibS od0+/aNhBqNdpQHUkIVMQ4vacaIUhZeVkUCV7B/rubBlWYg40vbbUjFFrcqgjJvOOJXQ fnrHUPchHc4hSbJygf2UMiTqmz2zlYx8MJ8qjyfYsWQ1zTrA59wckg2NZF26EIq8SWHw rRqw== X-Gm-Message-State: AJaThX5bvhdlopWsc7Bd+g4l4uLCfNnuMeUTuGf9V2NpSWOYPZMorBY4 TdOrL7PFQ9iswrR4BdD5/m86 X-Google-Smtp-Source: AGs4zMZj08yTKoQwYu581gZeFGe9kIZ5vh792TdU/+3R8xjkasyqJ3Z4nngpL15Dqjf8j8aQYKPwPQ== X-Received: by 10.80.226.12 with SMTP id n12mr12158912edl.177.1512032733783; Thu, 30 Nov 2017 01:05:33 -0800 (PST) Received: from bra-l27t7p12.vyatta.net ([213.251.34.151]) by smtp.gmail.com with ESMTPSA id i16sm2895707edj.77.2017.11.30.01.05.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 30 Nov 2017 01:05:33 -0800 (PST) From: alangordondewar@gmail.com X-Google-Original-From: alan.dewar@att.com To: cristian.dumitrescu@intel.com Cc: dev@dpdk.org, Alan Dewar Date: Thu, 30 Nov 2017 09:05:26 +0000 Message-Id: <1512032726-30807-1-git-send-email-alan.dewar@att.com> X-Mailer: git-send-email 2.1.4 Subject: [dpdk-dev] [PATCH v2] test: new sched WRR unit-test X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Nov 2017 09:05:34 -0000 From: Alan Dewar New unit-test for the librte_sched WRR weighting code. With the standard 17.11 code, the first three sub-tests pass, but the last three fail due to bugs in the WRR weighting code. With v1 of the "sched: fix overflow errors in WRR weighting code" patch the first five sub-tests pass, and the last sub-test fails badly. With v2 of the "sched: fix overflow errors in WRR weighting code" patch the first five sub-tests pass, and the last sub-test is a very near miss (i.e. measured packets counts are one away from the expected counts). Signed-off-by: Alan Dewar --- v2 - add new 255-254-253-1 weightings sub-test test/test/Makefile | 1 + test/test/test_sched_wrr.c | 491 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 492 insertions(+) create mode 100644 test/test/test_sched_wrr.c diff --git a/test/test/Makefile b/test/test/Makefile index bb54c98..0ab0ed3 100644 --- a/test/test/Makefile +++ b/test/test/Makefile @@ -173,6 +173,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_NET) += test_crc.c ifeq ($(CONFIG_RTE_LIBRTE_SCHED),y) SRCS-y += test_red.c SRCS-y += test_sched.c +SRCS-y += test_sched_wrr.c endif SRCS-$(CONFIG_RTE_LIBRTE_METER) += test_meter.c diff --git a/test/test/test_sched_wrr.c b/test/test/test_sched_wrr.c new file mode 100644 index 0000000..df5a231 --- /dev/null +++ b/test/test/test_sched_wrr.c @@ -0,0 +1,491 @@ +/*- + * BSD LICENSE + * + * Copyright(c) 2010-2014 Intel Corporation. All rights reserved. + * Copyright(c) 2017 ATT Intellectual Property. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * Neither the name of Intel Corporation nor the names of its + * contributors may be used to endorse or promote products derived + * from this software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + */ + +#include +#include +#include +#include +#include + +#include "test.h" + +#include +#include +#include +#include +#include + + +#define SUBPORT 0 +#define PIPE 1 +#define TC 0 +#define QUEUE 3 + +static struct rte_sched_subport_params subport_param[] = { + { + .tb_rate = 1250000000, + .tb_size = 1000000, + + .tc_rate = {1250000000, 1250000000, 1250000000, 1250000000}, + .tc_period = 10, + }, +}; + +static struct rte_sched_pipe_params pipe_profile[] = { + { /* Profile #0 */ + .tb_rate = 3051750, + .tb_size = 1000000, + + .tc_rate = {3051750, 3051750, 3051750, 3051750}, + .tc_period = 160, + + .wrr_weights = {1, 1, 1, 1, + 1, 1, 1, 1, + 1, 1, 1, 1, + 1, 1, 1, 1}, + }, +}; + +static struct rte_sched_port_params port_param = { + .socket = 0, /* computed */ + .rate = 0, /* computed */ + .mtu = 1522, + .frame_overhead = RTE_SCHED_FRAME_OVERHEAD_DEFAULT, + .n_subports_per_port = 1, + .n_pipes_per_subport = 1024, + .qsize = {32, 32, 32, 32}, + .pipe_profiles = pipe_profile, + .n_pipe_profiles = 1, +}; + +#define NB_MBUF (4 * 32) +#define MBUF_DATA_SZ (2048 + RTE_PKTMBUF_HEADROOM) +#define MEMPOOL_CACHE_SZ 0 +#define SOCKET 0 + + +static struct rte_mempool * +create_mempool(uint32_t total_packets) +{ + struct rte_mempool *mp; + + mp = rte_mempool_lookup("test_sched wrr"); + if (!mp) + mp = rte_pktmbuf_pool_create("test_sched wrr", total_packets, + MEMPOOL_CACHE_SZ, 0, MBUF_DATA_SZ, SOCKET); + + return mp; +} + +static void +delete_mempool(struct rte_mempool *mp) +{ + rte_mempool_free(mp); +} + +static void +prepare_pkt(struct rte_mbuf *mbuf, uint32_t tc, uint32_t queue) +{ + struct ether_hdr *eth_hdr; + struct vlan_hdr *vlan1, *vlan2; + struct ipv4_hdr *ip_hdr; + + /* Simulate a classifier */ + eth_hdr = rte_pktmbuf_mtod(mbuf, struct ether_hdr *); + vlan1 = (struct vlan_hdr *)(ð_hdr->ether_type); + vlan2 = (struct vlan_hdr *)((uintptr_t)ð_hdr->ether_type + + sizeof(struct vlan_hdr)); + eth_hdr = (struct ether_hdr *)((uintptr_t)ð_hdr->ether_type + + 2 * sizeof(struct vlan_hdr)); + ip_hdr = (struct ipv4_hdr *)((uintptr_t)eth_hdr + + sizeof(eth_hdr->ether_type)); + + vlan1->vlan_tci = rte_cpu_to_be_16(SUBPORT); + vlan2->vlan_tci = rte_cpu_to_be_16(PIPE); + eth_hdr->ether_type = rte_cpu_to_be_16(ETHER_TYPE_IPv4); + ip_hdr->dst_addr = IPv4(0, 0, TC, QUEUE); + + + rte_sched_port_pkt_write(mbuf, SUBPORT, PIPE, tc, queue, + e_RTE_METER_YELLOW); + + /* 64 byte packet */ + mbuf->pkt_len = 60; + mbuf->data_len = 60; +} + +/* + * This function carries out the core of the enqueue/dequeue testing. + * This is where we should detect failures if the WRR code is broken. + */ +static int +test_sched_wrr_enqueue_dequeue(const char *subtest_name, + struct rte_mempool *mp, + struct rte_sched_port *port, + struct rte_mbuf **in_mbufs, + struct rte_mbuf **out_mbufs, + int32_t enqueue_packets, + int32_t dequeue_packets, + uint32_t *expected_counts) +{ + uint32_t wrr_counts[RTE_SCHED_QUEUES_PER_TRAFFIC_CLASS] = { 0 }; + uint32_t pipe; + int32_t i; + int err; + + /* + * Create the packets to be enqueued, spread them evenly across + * each of the four WRR queues of the test-TC + */ + for (i = 0; i < enqueue_packets; i++) { + in_mbufs[i] = rte_pktmbuf_alloc(mp); + TEST_ASSERT_NOT_NULL(in_mbufs[i], + "%s: Packet allocation failed on packet " + "%d\n", subtest_name, i); + prepare_pkt(in_mbufs[i], TC, (i & 0x3)); + } + + /* + * Queue all the enqueue packets, none should be dropped as the + * four queues should be long enough. + */ + err = rte_sched_port_enqueue(port, in_mbufs, enqueue_packets); + TEST_ASSERT_EQUAL(err, enqueue_packets, + "%s: Enqueue failed, err: %d != %d\n", + subtest_name, err, enqueue_packets); + + /* + * Dequeue the required number of packets. + */ + err = rte_sched_port_dequeue(port, out_mbufs, dequeue_packets); + TEST_ASSERT_EQUAL(err, dequeue_packets, + "%s: Dequeue failed, err: %d != %d\n", + subtest_name, err, dequeue_packets); + + /* + * Check each packet and count which WRR queue it came from. + */ + for (i = 0; i < dequeue_packets; i++) { + enum rte_meter_color color; + uint32_t subport, traffic_class, queue; + + color = rte_sched_port_pkt_read_color(out_mbufs[i]); + TEST_ASSERT_EQUAL(color, e_RTE_METER_YELLOW, + "%s: Wrong color\n", subtest_name); + + rte_sched_port_pkt_read_tree_path(out_mbufs[i], + &subport, &pipe, &traffic_class, &queue); + + TEST_ASSERT_EQUAL(subport, SUBPORT, "%s: Wrong subport\n", + subtest_name); + TEST_ASSERT_EQUAL(pipe, PIPE, "%s: Wrong pipe\n", subtest_name); + TEST_ASSERT_EQUAL(traffic_class, TC, + "%s: Wrong traffic-class\n", subtest_name); + wrr_counts[queue]++; + rte_pktmbuf_free(out_mbufs[i]); + } + + /* + * Check the number of packets dequeued from each WRR queue + * against the expected counts. + */ + err = 0; + for (i = 0; i < RTE_SCHED_QUEUES_PER_TRAFFIC_CLASS; i++) { + printf("%s - WRR queue %d, dequeued: %u, expected: %u\n", + subtest_name, i, wrr_counts[i], expected_counts[i]); + if (wrr_counts[i] != expected_counts[i]) + err = -1; + } + return err; +} + +/* + * This function does the test config set-up and tear-down. + * If we see failures in here it is probably due to the test configuration. + */ +static int +test_sched_wrr_test(const char *subtest_name, uint16_t *tc_qlengths, + uint8_t *wrr_weights, int32_t enqueue_packets, + int32_t dequeue_packets, uint32_t *expected_counts) +{ + struct rte_mbuf **in_mbufs; + struct rte_mbuf **out_mbufs; + struct rte_mempool *mp = NULL; + struct rte_sched_port *port = NULL; + int32_t total_expected = 0; + uint32_t pipe; + int32_t i; + int err; + + /* + * Some inbound argument checking + */ + TEST_ASSERT_EQUAL((tc_qlengths[TC] * 4), enqueue_packets, + "%s: Queue length/Enqueue packet mismatch - " + "%u vs %d\n", + subtest_name, (tc_qlengths[TC] * 4), enqueue_packets); + TEST_ASSERT((dequeue_packets <= enqueue_packets), + "%s: Dequeue packets %d > Enqueue packets %d\n", + subtest_name, dequeue_packets, enqueue_packets); + + for (i = 0; i < RTE_SCHED_QUEUES_PER_TRAFFIC_CLASS; i++) + total_expected += expected_counts[i]; + + TEST_ASSERT((dequeue_packets == total_expected), + "%s: Dequeue packets %d != Total expected %u\n", + subtest_name, dequeue_packets, total_expected); + + /* + * Create the mempool and allocate arrays to hold the rte_mbuf pointers + */ + mp = create_mempool(enqueue_packets); + TEST_ASSERT_NOT_NULL(mp, "%s: Error creating mempool\n", subtest_name); + + in_mbufs = malloc(sizeof(struct rte_mbuf *) * enqueue_packets); + TEST_ASSERT_NOT_NULL(in_mbufs, "%s: Error creating in_mbuf array\n", + subtest_name); + + out_mbufs = malloc(sizeof(struct rte_mbuf *) * dequeue_packets); + TEST_ASSERT_NOT_NULL(out_mbufs, "%s: Error creating out_mbuf array\n", + subtest_name); + + /* + * Set up the port and pipe profiles with the TC's queue lengths + * and the WRR queue weightings + */ + port_param.socket = 0; + port_param.rate = (uint64_t) 10000 * 1000 * 1000 / 8; + for (i = 0; i < RTE_SCHED_TRAFFIC_CLASSES_PER_PIPE; i++) + port_param.qsize[i] = tc_qlengths[i]; + + for (i = 0; i < RTE_SCHED_QUEUES_PER_PIPE; i++) + pipe_profile[0].wrr_weights[i] = wrr_weights[i]; + + port = rte_sched_port_config(&port_param); + TEST_ASSERT_NOT_NULL(port, "%s: Error config sched port\n", + subtest_name); + + err = rte_sched_subport_config(port, SUBPORT, subport_param); + TEST_ASSERT_SUCCESS(err, "%s: Error config sched, err=%d\n", + subtest_name, err); + + for (pipe = 0; pipe < port_param.n_pipes_per_subport; pipe++) { + err = rte_sched_pipe_config(port, SUBPORT, pipe, 0); + TEST_ASSERT_SUCCESS(err, + "%s: Error config sched pipe %u, err=%d\n", + subtest_name, pipe, err); + } + + /* + * Enqueue and dequeue packets checking that each WRR queue dequeued + * the correct number of packets. + */ + err = test_sched_wrr_enqueue_dequeue(subtest_name, mp, port, in_mbufs, + out_mbufs, enqueue_packets, + dequeue_packets, + expected_counts); + + /* + * Free up allocated resources + */ + free(in_mbufs); + free(out_mbufs); + rte_sched_port_free(port); + delete_mempool(mp); + + return err; +} + +static int +test_sched_wrr_even_weights(void) +{ + /* + * Even weighting - each WRR queue should dequeue the same number of + * packets + */ + uint16_t tc_qlengths[] = { 64, 64, 64, 64 }; + uint8_t wrr_weights[] = { 1, 1, 1, 1, // TC-0 + 1, 1, 1, 1, // TC-1 + 1, 1, 1, 1, // TC-2 + 1, 1, 1, 1 }; // TC-3 + int32_t enqueue_packets = 64 * 4; + int32_t dequeue_packets = 32; + uint32_t expected_counts[] = { 8, 8, 8, 8 }; + + return test_sched_wrr_test("wrr-even-weights", tc_qlengths, wrr_weights, + enqueue_packets, dequeue_packets, + expected_counts); +} + +static int +test_sched_wrr_8_4_2_1_weights(void) +{ + /* + * Uneven weightings but with a low LCM of 8 * 4 * 2 * 1 = 64 + */ + uint16_t tc_qlengths[] = { 64, 64, 64, 64 }; + uint8_t wrr_weights[] = { 8, 4, 2, 1, // TC-0 + 1, 1, 1, 1, + 1, 1, 1, 1, + 1, 1, 1, 1 }; + int32_t enqueue_packets = 64 * 4; + int32_t dequeue_packets = 15; + uint32_t expected_counts[] = { 8, 4, 2, 1 }; + + return test_sched_wrr_test("wrr-8-4-2-1-weights", tc_qlengths, + wrr_weights, enqueue_packets, + dequeue_packets, expected_counts); +} + +static int +test_sched_wrr_1_2_3_4_weights(void) +{ + /* + * Uneven weightings with a low LCM, but weightings in reverse order + * from previous test + */ + uint16_t tc_qlengths[] = { 64, 64, 64, 64 }; + uint8_t wrr_weights[] = { 1, 2, 3, 4, + 1, 1, 1, 1, + 1, 1, 1, 1, + 1, 1, 1, 1 }; + int32_t enqueue_packets = 64 * 4; + int32_t dequeue_packets = 40; + uint32_t expected_counts[] = { 4, 8, 12, 16 }; + + return test_sched_wrr_test("wrr-1-2-3-4-weights", tc_qlengths, + wrr_weights, enqueue_packets, + dequeue_packets, expected_counts); +} + +static int +test_sched_wrr_11_7_5_3_weights(void) +{ + /* + * This test generates a LCM of 11 * 7 * 5 * 3 = 1155. + * 1155 / 3 = 385 which is more than can be fitted in a uint8_t + */ + uint16_t tc_qlengths[] = { 64, 64, 64, 64 }; + uint8_t wrr_weights[] = { 11, 7, 5, 3, + 1, 1, 1, 1, + 1, 1, 1, 1, + 1, 1, 1, 1 }; + int32_t enqueue_packets = 64 * 4; + int32_t dequeue_packets = 26; + uint32_t expected_counts[] = { 11, 7, 5, 3 }; + + return test_sched_wrr_test("wrr-11-7-5-3-weights", tc_qlengths, + wrr_weights, enqueue_packets, + dequeue_packets, expected_counts); +} + +static int +test_sched_wrr_100_to_97_weights(void) +{ + /* + * The ratios between the weightings is small how well will the + * pseudo floating point wrr_cost perform. + */ + uint16_t tc_qlengths[] = { 128, 1, 1, 1 }; + uint8_t wrr_weights[] = { 100, 99, 98, 97, + 1, 1, 1, 1, + 1, 1, 1, 1, + 1, 1, 1, 1 }; + int32_t enqueue_packets = 128 * 4; + int32_t dequeue_packets = 394; + uint32_t expected_counts[] = { 100, 99, 98, 97 }; + + return test_sched_wrr_test("wrr-100-to-97-weights", tc_qlengths, + wrr_weights, enqueue_packets, + dequeue_packets, expected_counts); +} + +static int +test_sched_wrr_255_254_253_1_weights(void) +{ + /* + * This test generates a LCM of 255 * 254 * 253 * 1 = 16386810 + * 16386810 / 255 = 64262 = 0xFB06 + * 16386810 / 1 = 16386810 = 0xFA0AFF + * 0xFA0AFF needs to be shifted 16 bits left to fit into a uint8_t + * but shifting 0xFB06 16 bits left results in a zero wrr_cost. + * + * This test can fail in two different ways. + * + * Very badly when the dequeued packet counts aren't even close to the + * expected counts due a bug in a recent WRR patch. + * + * A very near miss, when the dequeued packet counts are just one or + * two packets away from the expected counts. This happens because + * the weights of 255, 254, 254 and 1 get converted into wrr_costs + * of 1, 1, 1, 250 respectively. I think that this happens due to + * rounding errors in the pseudo floating point code used by WRR. + */ + uint16_t tc_qlengths[] = { 1024, 1, 1, 1 }; + uint8_t wrr_weights[] = { 254, 1, 253, 255, + 1, 1, 1, 1, + 1, 1, 1, 1, + 1, 1, 1, 1 }; + int32_t enqueue_packets = 1024 * 4; + int32_t dequeue_packets = 255 + 254 + 253 + 1; + uint32_t expected_counts[] = { 254, 1, 253, 255 }; + + return test_sched_wrr_test("wrr-255-254-253-1-weights", tc_qlengths, + wrr_weights, enqueue_packets, + dequeue_packets, expected_counts); +} + +/** + * WRR test main entrance for library sched + */ +static int +test_sched_wrr(void) +{ + TEST_ASSERT_SUCCESS(test_sched_wrr_even_weights(), + "even-weight test failed\n"); + TEST_ASSERT_SUCCESS(test_sched_wrr_8_4_2_1_weights(), + "8-4-2-1-weight test failed\n"); + TEST_ASSERT_SUCCESS(test_sched_wrr_1_2_3_4_weights(), + "1-2-3-4-weight test failed\n"); + TEST_ASSERT_SUCCESS(test_sched_wrr_11_7_5_3_weights(), + "11-7-5-3-weight test failed\n"); + TEST_ASSERT_SUCCESS(test_sched_wrr_100_to_97_weights(), + "100-to-97-weight test failed\n"); + TEST_ASSERT_SUCCESS(test_sched_wrr_255_254_253_1_weights(), + "255-254-253-1-weight test failed\n"); + return 0; +} + +REGISTER_TEST_COMMAND(sched_wrr_test, test_sched_wrr); -- 2.1.4