From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <dev-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0A807A00C4; Sat, 5 Mar 2022 08:09:56 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A0DA440683; Sat, 5 Mar 2022 08:09:55 +0100 (CET) Received: from m12-11.163.com (m12-11.163.com [220.181.12.11]) by mails.dpdk.org (Postfix) with ESMTP id 9F95A40042 for <dev@dpdk.org>; Sat, 5 Mar 2022 08:09:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=163.com; s=s110527; h=From:Subject:Date:Message-Id:MIME-Version; bh=vlaG1 oApEotSQa8tc8ALeoXZS+C4e6f5T6OYgzp/T/s=; b=D+V82XYiLW5jqpcIz7nMV g1/gOzUH9EOfesA2VVibtualf241xQ7ykIqJQMhpNYoOPM9I3UV9SiN3ANsVm2eR z9QAL5+QubebkKaJq23FYPqupfA/3JWrAz96zpd0IqGi/5YDXP6cnyCvsX/zYGBn Pz8dlhTj5UmbkF6dlddu20= Received: from DESKTOP-ONA2IA7.localdomain (unknown [211.138.116.222]) by smtp7 (Coremail) with SMTP id C8CowAAXD6KwDCNiDFsQBg--.11247S4; Sat, 05 Mar 2022 15:09:47 +0800 (CST) From: Gaoxiang Liu <gaoxiangliu0@163.com> To: chas3@att.com, humin29@huawei.com Cc: dev@dpdk.org, liugaoxiang@huawei.com, Gaoxiang Liu <gaoxiangliu0@163.com> Subject: [PATCH v3] net/bonding: another fix to LACP mempool size Date: Sat, 5 Mar 2022 15:09:22 +0800 Message-Id: <20220305070922.308-1-gaoxiangliu0@163.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20220305022630.153-1-gaoxiangliu0@163.com> References: <20220305022630.153-1-gaoxiangliu0@163.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CM-TRANSID: C8CowAAXD6KwDCNiDFsQBg--.11247S4 X-Coremail-Antispam: 1Uf129KBjvJXoW7uw1rJF45ur4xCF4rCw47twb_yoW8tw4Upr W7Ga15GF1UZ347u3W7X3WxC395u348X3y7Ka98Zas5Z3yxAFn0kr17try5urW8GrWktrs0 vF4UZasagF15CrJanT9S1TB71UUUUUUqnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x0pRl_McUUUUU= X-Originating-IP: [211.138.116.222] X-CM-SenderInfo: xjdr5xxdqjzxjxq6il2tof0z/1tbi6xu6OlXlzyYo6AAAsp X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions <dev.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org The following log message may appear after a slave is idle(or nearly idle) for a few minutes:"PMD: Failed to allocate LACP packet from pool". And bond mode 4 negotiation may fail. Problem: All mbufs from a slave' private pool(used exclusively for transmitting LACPDUs) have been allocated and are still sitting in the device's tx descriptor ring and other cores' mempool caches. Solution: Ensure that each slave'tx (LACPDU) mempool owns more than n-tx-queues * n-tx-descriptor + fwd_core_num * per-core-mmempool-flush-threshold mbufs. Note that the LACP tx machine fuction is the only code that allocates from a slave's private pool. It runs in the context of the interrupt thread, and thus it has no mempool cache of its own. Signed-off-by: Gaoxiang Liu <liugaoxiang@huawei.com> --- v2: * Fixed compile issues. v3: * delete duplicate code. --- drivers/net/bonding/rte_eth_bond_8023ad.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/net/bonding/rte_eth_bond_8023ad.c b/drivers/net/bonding/rte_eth_bond_8023ad.c index ca50583d62..d73038b018 100644 --- a/drivers/net/bonding/rte_eth_bond_8023ad.c +++ b/drivers/net/bonding/rte_eth_bond_8023ad.c @@ -1050,6 +1050,7 @@ bond_mode_8023ad_activate_slave(struct rte_eth_dev *bond_dev, uint32_t total_tx_desc; struct bond_tx_queue *bd_tx_q; uint16_t q_id; + uint32_t cache_size; /* Given slave mus not be in active list */ RTE_ASSERT(find_slave_by_id(internals->active_slaves, @@ -1100,11 +1101,12 @@ bond_mode_8023ad_activate_slave(struct rte_eth_dev *bond_dev, total_tx_desc += bd_tx_q->nb_tx_desc; } + cache_size = RTE_MEMPOOL_CACHE_MAX_SIZE >= 32 ? + 32 : RTE_MEMPOOL_CACHE_MAX_SIZE; + total_tx_desc = rte_lcore_count() * cache_size * 1.5; snprintf(mem_name, RTE_DIM(mem_name), "slave_port%u_pool", slave_id); port->mbuf_pool = rte_pktmbuf_pool_create(mem_name, total_tx_desc, - RTE_MEMPOOL_CACHE_MAX_SIZE >= 32 ? - 32 : RTE_MEMPOOL_CACHE_MAX_SIZE, - 0, element_size, socket_id); + cache_size, 0, element_size, socket_id); /* Any memory allocation failure in initialization is critical because * resources can't be free, so reinitialization is impossible. */ -- 2.32.0