From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1AA09A0093; Tue, 8 Mar 2022 02:34:06 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D2406410F4; Tue, 8 Mar 2022 02:34:05 +0100 (CET) Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by mails.dpdk.org (Postfix) with ESMTP id E13D9410DD for ; Tue, 8 Mar 2022 02:34:04 +0100 (CET) Received: from dggeme756-chm.china.huawei.com (unknown [172.30.72.54]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4KCHT03yM9zBrd0; Tue, 8 Mar 2022 09:15:48 +0800 (CST) Received: from [10.67.103.128] (10.67.103.128) by dggeme756-chm.china.huawei.com (10.3.19.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.21; Tue, 8 Mar 2022 09:17:42 +0800 Subject: Re: [PATCH v3] net/bonding: another fix to LACP mempool size To: Gaoxiang Liu , CC: , References: <20220305022630.153-1-gaoxiangliu0@163.com> <20220305070922.308-1-gaoxiangliu0@163.com> From: "Min Hu (Connor)" Message-ID: <787b7c59-c3a1-af18-3de5-71fbad05bb78@huawei.com> Date: Tue, 8 Mar 2022 09:17:42 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.3.1 MIME-Version: 1.0 In-Reply-To: <20220305070922.308-1-gaoxiangliu0@163.com> Content-Type: text/plain; charset="gbk"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.67.103.128] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggeme756-chm.china.huawei.com (10.3.19.102) X-CFilter-Loop: Reflected X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org HI, ÔÚ 2022/3/5 15:09, Gaoxiang Liu дµÀ: > The following log message may appear after a slave is idle(or nearly > idle) > for a few minutes:"PMD: Failed to allocate LACP packet from pool". > And bond mode 4 negotiation may fail. > > Problem: All mbufs from a slave' private pool(used exclusively for > transmitting LACPDUs) > have been allocated and are still sitting in the device's tx descriptor > ring and > other cores' mempool caches. > > Solution: Ensure that each slave'tx (LACPDU) mempool owns more than > n-tx-queues * n-tx-descriptor + fwd_core_num * > per-core-mmempool-flush-threshold mbufs. How do you get this way to calculate 'total_tx_desc'? I want to know the causal logic. > > Note that the LACP tx machine fuction is the only code that allocates > from a slave's private pool. It runs in the context of the interrupt > thread, and thus it has no mempool cache of its own. > > Signed-off-by: Gaoxiang Liu > > --- > v2: > * Fixed compile issues. > > v3: > * delete duplicate code. > --- > drivers/net/bonding/rte_eth_bond_8023ad.c | 8 +++++--- > 1 file changed, 5 insertions(+), 3 deletions(-) > > diff --git a/drivers/net/bonding/rte_eth_bond_8023ad.c b/drivers/net/bonding/rte_eth_bond_8023ad.c > index ca50583d62..d73038b018 100644 > --- a/drivers/net/bonding/rte_eth_bond_8023ad.c > +++ b/drivers/net/bonding/rte_eth_bond_8023ad.c > @@ -1050,6 +1050,7 @@ bond_mode_8023ad_activate_slave(struct rte_eth_dev *bond_dev, > uint32_t total_tx_desc; > struct bond_tx_queue *bd_tx_q; > uint16_t q_id; > + uint32_t cache_size; > > /* Given slave mus not be in active list */ > RTE_ASSERT(find_slave_by_id(internals->active_slaves, > @@ -1100,11 +1101,12 @@ bond_mode_8023ad_activate_slave(struct rte_eth_dev *bond_dev, > total_tx_desc += bd_tx_q->nb_tx_desc; > } > > + cache_size = RTE_MEMPOOL_CACHE_MAX_SIZE >= 32 ? > + 32 : RTE_MEMPOOL_CACHE_MAX_SIZE; > + total_tx_desc = rte_lcore_count() * cache_size * 1.5; According to your comment, is it wrong? or: 'total_tx_desc += rte_lcore_count() * cache_size * 1.5;' > snprintf(mem_name, RTE_DIM(mem_name), "slave_port%u_pool", slave_id); > port->mbuf_pool = rte_pktmbuf_pool_create(mem_name, total_tx_desc, > - RTE_MEMPOOL_CACHE_MAX_SIZE >= 32 ? > - 32 : RTE_MEMPOOL_CACHE_MAX_SIZE, > - 0, element_size, socket_id); > + cache_size, 0, element_size, socket_id); > > /* Any memory allocation failure in initialization is critical because > * resources can't be free, so reinitialization is impossible. */ >