From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <3chas3@gmail.com> Received: from mail-qt0-f196.google.com (mail-qt0-f196.google.com [209.85.216.196]) by dpdk.org (Postfix) with ESMTP id ADE796C9B; Thu, 20 Sep 2018 14:52:32 +0200 (CEST) Received: by mail-qt0-f196.google.com with SMTP id l42-v6so8280709qtf.13; Thu, 20 Sep 2018 05:52:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=hCkNiR43GgMg+9aStxrr3xUesDfiP38MSIqZ89S2JW8=; b=YKttsPX6kJMaVYHkKVTRBNkP2wvdDQy5MbVZkKQeOcd1GbnL/3CnLc1V5kvjZVnpjA gIEm+sxWT1Bfabw7+/iwbnDL613OWGDIF3ERwA8MoznwC7P4aFkOpu/PxW1jqMWTaFQ2 Vd1g0wSwsi8T+8BHT/trQXHlA/EDxAcDTcHlBCpx5OrkWgxrt2xm9Z5Zz3nIdnmrjclj jGs8+ochA8bnISLyXedewEtui7bhSyAjNXgyOhh076wsgct0LfnlVLcsGnL2yr9L7+31 +D5Jh6uT35qP5Rv6MbVyp6nA7KGfoSATkXiI9wkqr8Ys/kgxj3JGpYxKCgR4P63mqk2x 6dJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=hCkNiR43GgMg+9aStxrr3xUesDfiP38MSIqZ89S2JW8=; b=HHzPTKxmutlbPi7Qyvv4roa1rHhBIljiOgnYZWkxgngKkUL4aayUsuoJYiuwpsXGwC AO5jvKhG46ntf76WQanftY2MxhV6PBdwfJ06eIKqJPKHfpHb2aHghJBo3nu2LswXOupn DKXDkJg6BQ//Mp/brzORz73k0deCoFRhnMO+BWZTYsGNHXTiP/qiGbgOF+fS/2aDo5RR eF77Nyc2ufrbs6SbFZ6hlLD1F0bi9p2dKmMlCdGE2z80s4WBW0l5trYO+VgYNWFJS4p9 VQRSacMkGWRW2xzvs6fmcvj8cWBRXQvyziXry56bdw+TSa3OeOKOLLmALKVIEco03uWs BEqg== X-Gm-Message-State: APzg51DtX20/h1Fe9O67/HXN9gHdJvx0SjTZM+35KuQiv9xB0U1H96cU T9QsqXGB49uhFdsXAomkNsrxXoXWvHI= X-Google-Smtp-Source: ANB0VdaMpmNao44q8gYG5/3Y+zXgnmneDfb8kqF8DjJmCv0x5AKmHkRh9sA/NunBdQRT+i4iUndjyA== X-Received: by 2002:ac8:2f24:: with SMTP id j33-v6mr27064491qta.93.1537447951821; Thu, 20 Sep 2018 05:52:31 -0700 (PDT) Received: from monolith.fios-router.home (pool-173-79-169-217.washdc.fios.verizon.net. [173.79.169.217]) by smtp.gmail.com with ESMTPSA id u184-v6sm14301770qkc.87.2018.09.20.05.52.31 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 20 Sep 2018 05:52:31 -0700 (PDT) From: Chas Williams <3chas3@gmail.com> To: dev@dpdk.org Cc: declan.doherty@intel.com, matan@mellanox.com, ehkinzie@gmail.com, Chas Williams , stable@dpdk.org Date: Thu, 20 Sep 2018 08:52:26 -0400 Message-Id: <20180920125226.11904-1-3chas3@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20180919154825.5183-1-3chas3@gmail.com> References: <20180919154825.5183-1-3chas3@gmail.com> Subject: [dpdk-dev] [PATCH v2] net/bonding: fix RX slave fairness X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Sep 2018 12:52:33 -0000 From: Chas Williams Some PMDs, especially ones with vector receives, require a minimum number of receive buffers in order to receive any packets. If the first slave read leaves less than this number available, a read from the next slave may return 0 implying that the slave doesn't have any packets which results in skipping over that slave as the next active slave. To fix this, implement round robin for the slaves during receive that is only advanced to the next slave at the end of each receive burst. This is also done to provide some additional fairness in processing in other bonding RX burst routines as well. Fixes: 2efb58cbab6e ("bond: new link bonding library") Cc: stable@dpdk.org Signed-off-by: Chas Williams Acked-by: Luca Boccassi Acked-by: Matan Azrad --- v2: - Reworded title and commit message - Fix checkpatch issue drivers/net/bonding/rte_eth_bond_pmd.c | 53 ++++++++++++++++++++++------------ 1 file changed, 34 insertions(+), 19 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c index b84f32263..5efd046a1 100644 --- a/drivers/net/bonding/rte_eth_bond_pmd.c +++ b/drivers/net/bonding/rte_eth_bond_pmd.c @@ -58,28 +58,34 @@ bond_ethdev_rx_burst(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) { struct bond_dev_private *internals; - uint16_t num_rx_slave = 0; uint16_t num_rx_total = 0; - + uint16_t slave_count; + uint16_t active_slave; int i; /* Cast to structure, containing bonded device's port id and queue id */ struct bond_rx_queue *bd_rx_q = (struct bond_rx_queue *)queue; - internals = bd_rx_q->dev_private; + slave_count = internals->active_slave_count; + active_slave = internals->active_slave; + for (i = 0; i < slave_count && nb_pkts; i++) { + uint16_t num_rx_slave; - for (i = 0; i < internals->active_slave_count && nb_pkts; i++) { /* Offset of pointer to *bufs increases as packets are received * from other slaves */ - num_rx_slave = rte_eth_rx_burst(internals->active_slaves[i], - bd_rx_q->queue_id, bufs + num_rx_total, nb_pkts); - if (num_rx_slave) { - num_rx_total += num_rx_slave; - nb_pkts -= num_rx_slave; - } + num_rx_slave = + rte_eth_rx_burst(internals->active_slaves[active_slave], + bd_rx_q->queue_id, + bufs + num_rx_total, nb_pkts); + num_rx_total += num_rx_slave; + nb_pkts -= num_rx_slave; + if (++active_slave == slave_count) + active_slave = 0; } + if (++internals->active_slave == slave_count) + internals->active_slave = 0; return num_rx_total; } @@ -258,25 +264,32 @@ bond_ethdev_rx_burst_8023ad_fast_queue(void *queue, struct rte_mbuf **bufs, uint16_t num_rx_total = 0; /* Total number of received packets */ uint16_t slaves[RTE_MAX_ETHPORTS]; uint16_t slave_count; - - uint16_t i, idx; + uint16_t active_slave; + uint16_t i; /* Copy slave list to protect against slave up/down changes during tx * bursting */ slave_count = internals->active_slave_count; + active_slave = internals->active_slave; memcpy(slaves, internals->active_slaves, sizeof(internals->active_slaves[0]) * slave_count); - for (i = 0, idx = internals->active_slave; - i < slave_count && num_rx_total < nb_pkts; i++, idx++) { - idx = idx % slave_count; + for (i = 0; i < slave_count && nb_pkts; i++) { + uint16_t num_rx_slave; /* Read packets from this slave */ - num_rx_total += rte_eth_rx_burst(slaves[idx], bd_rx_q->queue_id, - &bufs[num_rx_total], nb_pkts - num_rx_total); + num_rx_slave = rte_eth_rx_burst(slaves[active_slave], + bd_rx_q->queue_id, + bufs + num_rx_total, nb_pkts); + num_rx_total += num_rx_slave; + nb_pkts -= num_rx_slave; + + if (++active_slave == slave_count) + active_slave = 0; } - internals->active_slave = idx; + if (++internals->active_slave == slave_count) + internals->active_slave = 0; return num_rx_total; } @@ -459,7 +472,9 @@ bond_ethdev_rx_burst_8023ad(void *queue, struct rte_mbuf **bufs, idx = 0; } - internals->active_slave = idx; + if (++internals->active_slave == slave_count) + internals->active_slave = 0; + return num_rx_total; } -- 2.14.4