From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <3chas3@gmail.com> Received: from mail-qt1-f196.google.com (mail-qt1-f196.google.com [209.85.160.196]) by dpdk.org (Postfix) with ESMTP id BADF52C38 for ; Sun, 7 Oct 2018 22:22:15 +0200 (CEST) Received: by mail-qt1-f196.google.com with SMTP id e22-v6so11971906qto.6 for ; Sun, 07 Oct 2018 13:22:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=4CHlyjombDUiplQ/IKUXkcswAZ82PiTktLjT8AMabXs=; b=QZFJ5bQDIo8rlxIGVcx50eAOXYdBQyhzHb/bm93jzTKvFwnbHXBVx7sH2886CRS7/H ms0rQ/BuJZ91sqxZ5ZmYvqdo3TZ0Gx//W2OmHt+8kvPhZX1fhTARFU0SaY8sRKxyC4DK Kd3yJUNiVy9adJmn3AJiONMCqFeSC+Lt3eIj9/3uF2YDGxrhJs/PLsXoANSrG7wg1eLv AkRgMs09VKYu9nSuwNdku+NqJQZer0mhijaj5UB6I4COr8JaYJRGfFIDvvu6tVcNFFWJ qsJ6VbHmJxjlqk9PxtsU042II4MFPDYbtL+2k2UkPLHj5XQss8zusDpweJxfjIuE2VDg 7gKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=4CHlyjombDUiplQ/IKUXkcswAZ82PiTktLjT8AMabXs=; b=n9yMLGWx0LS7Rg15LyjdRAR7Q7MGhVt7Tqq0O70Jg5WG+7oP55KdFzihGWTpTv6YvK RUbCDLnR8ezj/P/dHzkt2DnW4erBQWp0VAwT0l4kiOYVcc/DAxppmYGBgEByFrgfQoow sR0EVxxQiHdGTg25yfF6B1OtJZ3pK8aDJXr/jUdO2T3Ehv0GlOLoS5byi7UVAH5cFAPt xCVO6lrl8lLekfPaLPTL1BFC1hg8xlxTS+yWRFiBNbFshcfQVIJEoN1LlAUaGwDzaoMc nzkJKQSmPxWJYha1ugUZOc7XLfMjrny2xzUON5RK1ytrn8z2zGkzUzUKYlHJhF7B10uk 3aRg== X-Gm-Message-State: ABuFfoj5M+Vh2gj8qn7cPDzfLkztxl0w2itkS6DlnG0kQlYOSujH6H/I jD3OMZ1mUmnxJKTH8Oyd+qgvNQZQ X-Google-Smtp-Source: ACcGV60ep/FW2WJArphBAkFPgDLQVI9IVV9c//gw6PWzgNydROVI2wQ2jnVWd0wwH2kyNy+W4rj+2g== X-Received: by 2002:ac8:592:: with SMTP id a18-v6mr17387087qth.184.1538943735064; Sun, 07 Oct 2018 13:22:15 -0700 (PDT) Received: from monolith.fios-router.home (pool-96-255-82-34.washdc.fios.verizon.net. [96.255.82.34]) by smtp.gmail.com with ESMTPSA id a72-v6sm6962406qkc.19.2018.10.07.13.22.14 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 07 Oct 2018 13:22:14 -0700 (PDT) From: Chas Williams <3chas3@gmail.com> To: stable@dpdk.org Cc: bluca@debian.org, Keith Wiles Date: Sun, 7 Oct 2018 16:22:01 -0400 Message-Id: <20181007202202.7648-2-3chas3@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20181007202202.7648-1-3chas3@gmail.com> References: <20181007202202.7648-1-3chas3@gmail.com> Subject: [dpdk-stable] [PATCH 16.11 1/2] net/bonding: reduce slave starvation on Rx poll X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Oct 2018 20:22:16 -0000 From: Keith Wiles [ upstream commit ae2a04864a9a3878f74e66e3ae0fdebe77223a09 ] When polling the bonded ports for RX packets the old driver would always start with the first slave in the list. If the requested number of packets is filled on the first port in a two port config then the second port could be starved or have larger number of missed packet errors. The code attempts to start with a different slave each time RX poll is done to help eliminate starvation of slave ports. The effect of the previous code was much lower performance for two slaves in the bond then just the one slave. The performance drop was detected when the application can not poll the rings of Rx packets fast enough and the packets per second for two or more ports was at the threshold throughput of the application. At this threshold the slaves would see very little or no drops in the case of one slave. Then enable the second slave you would see a large drop rate on the two slave bond and reduction in throughput. Signed-off-by: Keith Wiles Acked-by: Declan Doherty --- drivers/net/bonding/rte_eth_bond_pmd.c | 21 +++++++++++++++------ drivers/net/bonding/rte_eth_bond_private.h | 3 ++- 2 files changed, 17 insertions(+), 7 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index c672f0560..15e893240 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -1,7 +1,7 @@ /*- * BSD LICENSE * - * Copyright(c) 2010-2015 Intel Corporation. All rights reserved. + * Copyright(c) 2010-2017 Intel Corporation. All rights reserved. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -146,7 +146,7 @@ bond_ethdev_rx_burst_8023ad(void *queue, struct rte_mbuf **bufs, const uint16_t ether_type_slow_be = rte_be_to_cpu_16(ETHER_TYPE_SLOW); uint16_t num_rx_total = 0; /* Total number of received packets */ uint8_t slaves[RTE_MAX_ETHPORTS]; - uint8_t slave_count; + uint8_t slave_count, idx; uint8_t collecting; /* current slave collecting status */ const uint8_t promisc = internals->promiscuous_en; @@ -160,12 +160,18 @@ bond_ethdev_rx_burst_8023ad(void *queue, struct rte_mbuf **bufs, memcpy(slaves, internals->active_slaves, sizeof(internals->active_slaves[0]) * slave_count); + idx = internals->active_slave; + if (idx >= slave_count) { + internals->active_slave = 0; + idx = 0; + } for (i = 0; i < slave_count && num_rx_total < nb_pkts; i++) { j = num_rx_total; - collecting = ACTOR_STATE(&mode_8023ad_ports[slaves[i]], COLLECTING); + collecting = ACTOR_STATE(&mode_8023ad_ports[slaves[idx]], + COLLECTING); /* Read packets from this slave */ - num_rx_total += rte_eth_rx_burst(slaves[i], bd_rx_q->queue_id, + num_rx_total += rte_eth_rx_burst(slaves[idx], bd_rx_q->queue_id, &bufs[num_rx_total], nb_pkts - num_rx_total); for (k = j; k < 2 && k < num_rx_total; k++) @@ -188,8 +194,8 @@ bond_ethdev_rx_burst_8023ad(void *queue, struct rte_mbuf **bufs, !is_same_ether_addr(&bond_mac, &hdr->d_addr)))) { if (hdr->ether_type == ether_type_slow_be) { - bond_mode_8023ad_handle_slow_pkt(internals, slaves[i], - bufs[j]); + bond_mode_8023ad_handle_slow_pkt( + internals, slaves[idx], bufs[j]); } else rte_pktmbuf_free(bufs[j]); @@ -202,8 +208,11 @@ bond_ethdev_rx_burst_8023ad(void *queue, struct rte_mbuf **bufs, } else j++; } + if (unlikely(++idx == slave_count)) + idx = 0; } + internals->active_slave = idx; return num_rx_total; } diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h index d95d440b4..8c963ddb2 100644 --- a/drivers/net/bonding/rte_eth_bond_private.h +++ b/drivers/net/bonding/rte_eth_bond_private.h @@ -1,7 +1,7 @@ /*- * BSD LICENSE * - * Copyright(c) 2010-2015 Intel Corporation. All rights reserved. + * Copyright(c) 2010-2017 Intel Corporation. All rights reserved. * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -144,6 +144,7 @@ struct bond_dev_private { uint16_t nb_rx_queues; /**< Total number of rx queues */ uint16_t nb_tx_queues; /**< Total number of tx queues*/ + uint8_t active_slave; /**< Next active_slave to poll */ uint8_t active_slave_count; /**< Number of active slaves */ uint8_t active_slaves[RTE_MAX_ETHPORTS]; /**< Active slave list */ -- 2.14.4