From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f194.google.com (mail-qk0-f194.google.com [209.85.220.194]) by dpdk.org (Postfix) with ESMTP id 63BBC5A17 for ; Mon, 1 Aug 2016 22:45:04 +0200 (CEST) Received: by mail-qk0-f194.google.com with SMTP id y127so2510152qkd.3 for ; Mon, 01 Aug 2016 13:45:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lPdlVbTjg/1D4bUpOzkW0V7CTCiOon/BJaW+f0PT4d8=; b=EFtvGvm0eGKOSIxoFpKjCwHKezwaV0a4YxOdxt57So1IfipkWpSX5CNkjOe/RK/re/ jcH/CcrorGUM7sHFy2mnqn+78wHWNn31DsdnwZYRUKh/S73vBmEv+nVe4Nu62/wNlz6o XlP2EuE8chVfAnR3+ePY7nCj+OYMStCAsPFbaYwsE/GYbLT76Zd1y5deeXxukIPM3ti+ 1fJCY+zOSyawyRTiIleK69Yv5PNcmUisoo8ngGS6VjwprsKOYcDD87b/H5EHzgpj5F3Q sSK/WdX488CUgOjOGfJLgL0NJDjJow/CMdSjW0t4UkdXhq2e62G9p1q2diVESF1J5dCA LH1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lPdlVbTjg/1D4bUpOzkW0V7CTCiOon/BJaW+f0PT4d8=; b=eBfNeog1vzuINoUZ3idFJ04limBguyuJyzxFVFWkQcswBxuMk/pbJJuPQlTcVSY2Ao xP8PFF9daBFWOwf6ntEPrDQhFsdqM5LM4clFUoHrahZkBFBoQAAMATmZ0lZukhHBzeww tf0tjBePh50NNSKAgXjfT6204/HC7LB9+E172q+/uiSG15N6CxrNH6xc2bWE5fP9wkOI L/nmzMdP/+sw/Lcl/0SwKbBvlfslETnqKi9p3wHQMuseCCCOW9cEf386DY4Gszi9IZsV 2Q6b9NwPaKqgGmM9mzLA57UE1Csz88+lHKuoeDoNCjFqxr/BjRxdMYhWjAMbIqIpOiVP EQJg== X-Gm-Message-State: AEkooutaPf+fiD+HwuRpx1iybf2EKLylmwE5olBHTDZrj2h+6RTw4+dmcjjVa4J1zyoGeA== X-Received: by 10.55.188.198 with SMTP id m189mr67967343qkf.134.1470084303799; Mon, 01 Aug 2016 13:45:03 -0700 (PDT) Received: from localhost.localdomain ([23.79.237.14]) by smtp.gmail.com with ESMTPSA id i65sm18534359qtb.18.2016.08.01.13.45.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Aug 2016 13:45:03 -0700 (PDT) From: Robert Sanford X-Google-Original-From: Robert Sanford To: dev@dpdk.org Cc: declan.doherty@intel.com, pablo.de.lara.guarch@intel.com, olivier.matz@6wind.com Date: Mon, 1 Aug 2016 16:42:55 -0400 Message-Id: <1470084176-79932-4-git-send-email-rsanford@akamai.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1470084176-79932-1-git-send-email-rsanford@akamai.com> References: <1470084176-79932-1-git-send-email-rsanford@akamai.com> Subject: [dpdk-dev] [PATCH 3/4] net/bonding: another fix to LACP mempool size X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2016 20:45:04 -0000 The following log message may appear after a slave is idle (or nearly idle) for a few minutes: "PMD: Failed to allocate LACP packet from pool". Problem: All mbufs from a slave's private pool (used exclusively for transmitting LACPDUs) have been allocated and are still sitting in the device's tx descriptor ring and other cores' mempool caches. Solution: Ensure that each slaves' tx (LACPDU) mempool owns more than n-tx-queues * (n-tx-descriptors + per-core-mempool-flush-threshold) mbufs. Note that the LACP tx machine function is the only code that allocates from a slave's private pool. It runs in the context of the interrupt thread, and thus it has no mempool cache of its own. Signed-off-by: Robert Sanford --- drivers/net/bonding/rte_eth_bond_8023ad.c | 10 +++++++--- 1 files changed, 7 insertions(+), 3 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_8023ad.c b/drivers/net/bonding/rte_eth_bond_8023ad.c index 2f7ae70..1207896 100644 --- a/drivers/net/bonding/rte_eth_bond_8023ad.c +++ b/drivers/net/bonding/rte_eth_bond_8023ad.c @@ -854,6 +854,8 @@ bond_mode_8023ad_activate_slave(struct rte_eth_dev *bond_dev, uint8_t slave_id) char mem_name[RTE_ETH_NAME_MAX_LEN]; int socket_id; unsigned element_size; + unsigned cache_size; + unsigned cache_flushthresh; uint32_t total_tx_desc; struct bond_tx_queue *bd_tx_q; uint16_t q_id; @@ -890,19 +892,21 @@ bond_mode_8023ad_activate_slave(struct rte_eth_dev *bond_dev, uint8_t slave_id) element_size = sizeof(struct slow_protocol_frame) + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM; + cache_size = RTE_MEMPOOL_CACHE_MAX_SIZE >= 32 ? + 32 : RTE_MEMPOOL_CACHE_MAX_SIZE; + cache_flushthresh = RTE_MEMPOOL_CALC_CACHE_FLUSHTHRESH(cache_size); /* The size of the mempool should be at least: * the sum of the TX descriptors + BOND_MODE_8023AX_SLAVE_TX_PKTS */ total_tx_desc = BOND_MODE_8023AX_SLAVE_TX_PKTS; for (q_id = 0; q_id < bond_dev->data->nb_tx_queues; q_id++) { bd_tx_q = (struct bond_tx_queue*)bond_dev->data->tx_queues[q_id]; - total_tx_desc += bd_tx_q->nb_tx_desc; + total_tx_desc += bd_tx_q->nb_tx_desc + cache_flushthresh; } snprintf(mem_name, RTE_DIM(mem_name), "slave_port%u_pool", slave_id); port->mbuf_pool = rte_mempool_create(mem_name, - total_tx_desc, element_size, - RTE_MEMPOOL_CACHE_MAX_SIZE >= 32 ? 32 : RTE_MEMPOOL_CACHE_MAX_SIZE, + total_tx_desc, element_size, cache_size, sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, NULL, rte_pktmbuf_init, NULL, socket_id, MEMPOOL_F_NO_SPREAD); -- 1.7.1