From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B67F3A0501; Sat, 26 Mar 2022 13:57:26 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 587B441611; Sat, 26 Mar 2022 13:57:26 +0100 (CET) Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by mails.dpdk.org (Postfix) with ESMTP id 9F84D410F3 for ; Sat, 26 Mar 2022 13:57:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648299444; x=1679835444; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=0szzQu9CgOFs0mSrEnFYRCxtCDF5Nxdlck8CYnzFMhA=; b=IIskNb8fw3MoScEp6UGvx0ZiqQs7hCoG/X/X/YzS9H7QOLnPRwj/P2B4 vO0CGUm3IcpTWJpEB/ou4zNpA6UzmtAu3jmIpVU3faiEufVUDRgxPZ6Bz 9aXpkav9GW+DpgrmteMCgmFyvbPGeV162ldleUKqNj8/0L0gWq4lIA3iK Qns62j1phKrrfGQDqY+Pi/cbwzFDpHF0PgZh0QcKN1/xzfdg3d2yQI3PX Fu6p9vFUHnAlIXv4H+H/seUYoUThhsoriQh4CmMgwuTr2cWNoCQ7DK9Tg kx3qB/GmfUhA7ysqBg4nF7j52w4jMqnSCfZV228n4IM7N5YeayRS9lu4H g==; X-IronPort-AV: E=McAfee;i="6200,9189,10297"; a="345216823" X-IronPort-AV: E=Sophos;i="5.90,213,1643702400"; d="scan'208";a="345216823" Received: from orsmga006.jf.intel.com ([10.7.209.51]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Mar 2022 05:57:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.90,213,1643702400"; d="scan'208";a="520490158" Received: from fmsmsx605.amr.corp.intel.com ([10.18.126.85]) by orsmga006.jf.intel.com with ESMTP; 26 Mar 2022 05:57:22 -0700 Received: from fmsmsx612.amr.corp.intel.com (10.18.126.92) by fmsmsx605.amr.corp.intel.com (10.18.126.85) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Sat, 26 Mar 2022 05:57:22 -0700 Received: from fmsmsx608.amr.corp.intel.com (10.18.126.88) by fmsmsx612.amr.corp.intel.com (10.18.126.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27; Sat, 26 Mar 2022 05:57:21 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx608.amr.corp.intel.com (10.18.126.88) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.27 via Frontend Transport; Sat, 26 Mar 2022 05:57:21 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.45) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.21; Sat, 26 Mar 2022 05:57:21 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KlZ/RUuvazsDFdP5xkHRqFFpGhtdcakZyigZyMKwpRGpnF/v+8mq3dBB+0zGnMk4lpa01sfNOrMyNP2H4HAjM4oQBmMyUb6QU2IKLoFZRwcfLY+cuA8qFlpFvEQYxeOmZLo89rBliYCx8BD1D7AJeNTAG1X+tk91KX0yCCW8crA/cqUIE4tMphny3jlQY0uHoXyigFRt5NjPMm2U5kzg9lkSIS1swqKPAVKpOSwjnMRBqFo4gPmHj4F1nszdZkRgh8RK750PkplWkih2+cYUWdFGIkMFpZjbmckoC2ofK5F9aa9eLtb5CQX9ExYGu6Tg64rTcT0p21Q2xqmYo3gZ1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=d+aypy0fjdJPW58ERA21KdvLhZcMH32TwsNu1rvrBx4=; b=atb6H8UqLTbnHqhF90p520OEYoCeBHEwz0KaaMQ5x6aUygSZaEEPZjIrOzbijL99we7mfPYSRPbLyw/xuwGcuHNaWVnjDxYoq5XASwEGPkmE+YWbj7/mW5OP/kbr1kxUwl7Djim48uufuP46YlXxu/RATjA/zSicyq1TH+4dpuXECqtQYFNRDMoDy1M/bvIhx7sFwqduomEjfQofJJCghupArP4TAZwar8O83dwmBLpRvot5JnmK+N03i72lwUp1bs3oRlDshQmk9FtguEajGDzgJvBNNOjApjm99UzJhdpmkM4Un1EDGtNMHC7fyb2qTS7cvTRNfloZZ4k7B8Uaxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BYAPR11MB3495.namprd11.prod.outlook.com (2603:10b6:a03:8a::14) by BY5PR11MB4290.namprd11.prod.outlook.com (2603:10b6:a03:1ca::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5102.21; Sat, 26 Mar 2022 12:57:20 +0000 Received: from BYAPR11MB3495.namprd11.prod.outlook.com ([fe80::cc6f:9806:bca7:2297]) by BYAPR11MB3495.namprd11.prod.outlook.com ([fe80::cc6f:9806:bca7:2297%7]) with mapi id 15.20.5081.022; Sat, 26 Mar 2022 12:57:20 +0000 From: "Wang, Haiyue" To: =?iso-8859-1?Q?Morten_Br=F8rup?= , Gaoxiang Liu , "chas3@att.com" , "humin29@huawei.com" CC: "dev@dpdk.org" , "liugaoxiang@huawei.com" , "olivier.matz@6wind.com" , "andrew.rybchenko@oktetlabs.ru" Subject: RE: [PATCH v5] net/bonding: another fix to LACP mempool size Thread-Topic: [PATCH v5] net/bonding: another fix to LACP mempool size Thread-Index: AQHYQEFpbDnRGBYwskiYkwFeaN2J0qzQEGaAgAADd4CAAYuv4A== Date: Sat, 26 Mar 2022 12:57:20 +0000 Message-ID: References: <20220325121023.746-1-gaoxiangliu0@163.com> <20220325130135.2207-1-gaoxiangliu0@163.com> <98CBD80474FA8B44BF855DF32C47DC35D86F73@smartserver.smartshare.dk> In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC35D86F73@smartserver.smartshare.dk> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-product: dlpe-windows dlp-reaction: no-action dlp-version: 11.6.401.20 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 80d63117-e5f0-4a10-10ac-08da0f2829ea x-ms-traffictypediagnostic: BY5PR11MB4290:EE_ x-microsoft-antispam-prvs: x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: U2alRDj6ozRZsZSusFs+eWkOzfiMXt76vX7ndSsXVxzP360OiCeu0r255ddeRmrdsB2fHyH0OaT2pJbW2MTEKzpEcTf0jjVYuM06Y+LZB8C2RoXbxztj9hDNjLcaN4LWZULhqDimeb8jqImr0Jjg8mM5vjU4G2hM1aWtvTJEXKaz7AtGVdIl8tG2HvV0NyL2YFstnD1b7uJ6lzGwEUlX3xrfEIBR6bi5bo+iJrIPADkOUKon019UAyXm8BE+dotw3WdYQNWobwc6O9cEPJGl0vXnEF8z7xufIvPOJK1AntR92kDH3VwnkE7MsN8tW02mXZSif45ohyum+ZPnS99e/KtHTCx5oU0Fu3s3EXBqqqCevSDuz6D59Ih+ZRPoLf6ZtdbfWnt2TRawu9cPcBns3/lcmopX252gVF/JT/XHUYIsh3EPIeDb0ap8OWdz12l26/TDnt0cLHPQXPkkUU1XXVSEgOUCKs2xTofw2c31b6tnqZesV/qCAdOnfwEiSXJGN9r+9Q/Fztn4BQ4KepXmO+x1xSPHZQ/c8GNOgTh/rXwX2mws8nh+ROuTMg/gKd3I6U7bqALZI+6VLFUVf37EVOp2myUSss6eVQVJF3qFlT7p6pf/5P6ktoYYcsiBbtHUE8dx7FmFJs0Fdf6fz7WeXuPbc1D4F4fLKz7L8tNS0PKSkJJRjP7mCfPEFVWfztjZDfkDGNiT2ZR9/7Xfy3bUUw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BYAPR11MB3495.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(366004)(52536014)(110136005)(5660300002)(8936002)(66476007)(66556008)(76116006)(53546011)(9686003)(508600001)(316002)(186003)(54906003)(26005)(66574015)(66946007)(86362001)(38070700005)(38100700002)(82960400001)(71200400001)(33656002)(83380400001)(55016003)(7696005)(6506007)(64756008)(8676002)(122000001)(2906002)(4326008)(66446008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?oBa00NID1w7HbdxBBWqdqRXMXoRxvL1DUBVN9SwA7uFckE5ouAParxZxWV?= =?iso-8859-1?Q?s2HMLTs6ohffdxrHRghZ4wP9TEn0rgGQn1JBqA/JscNo+p4WDex+P1WgS6?= =?iso-8859-1?Q?Kq1IP0BSbvUiOI4AbgNN/3/y+2ujbdQC1CDBaMyFoCRZpM4NM9osYyZ1Qi?= =?iso-8859-1?Q?Uzps2SeFm40GdKbJ/t6is8N4VbcDRbJJ6ieZiQBL4+QUpXVuBKEJvoJtw2?= =?iso-8859-1?Q?QYtjkYF9K/x4tLAW2FM3c6qrcePyLeYlLlZWRnkvcF3TBqAWmI/N6bRiVt?= =?iso-8859-1?Q?vfjIh3iK3EWAB0OhJ4/OfrMhlAHdEHsAirAre6+cZQJ1/1mQI8CEP5qo3H?= =?iso-8859-1?Q?p1XpY64Wh2C3dy6H79EeDPMivvLEnhkndYFmdw5mcDa1mwlF666IMoPY5m?= =?iso-8859-1?Q?PFZuVssWYENZFY2SB7YpfemCcWWHJL9SVBXmHV0FDF6SjVeebhVs8081Zz?= =?iso-8859-1?Q?Yv6VbBxtN/6hxUaxIOLQjZol0dlEwbntMZoO0hPKDMFk0o7A3KtdOovor6?= =?iso-8859-1?Q?AfMdRO7eRsPJtFOTYUaKEEJH5u5F3sBTOitd48fMazMs8P20Pd3sLH0B5B?= =?iso-8859-1?Q?w2J8JvyeZNjzdWzNQCMkfRK30yc+P348qzsVsH357qjTXz3w5AXtEaTxt+?= =?iso-8859-1?Q?lXMpGHr0stQuDTasvLMnMe7a3Nt3jtKPT36s5fw9vTXPz4fR3qPHqiPTa1?= =?iso-8859-1?Q?D+VKGQdvgyUb0i2WuToXp6rizpTHMQQ8yF9+G7dEV7e9mW8shBdB2PX9TB?= =?iso-8859-1?Q?Dm3LviYdPSlgxghnY1YI/aTorC2dCPduefmJ7EezwIloaEqFmUcDJ1avyf?= =?iso-8859-1?Q?TNnPgDzyoyE57cCXyhApzJWPzTmKcdrZtmzlUefjCcbxd/D+Daiq3NmKMn?= =?iso-8859-1?Q?xdI1SogZvoy9jykJ8bDtdSh4BF02Oo+IjjoIFtgnRpfaFdf28OykR1J/yP?= =?iso-8859-1?Q?LJqmMdG/KtldwkRGgtjDIYdNEkBCCbXhOkTxgHjf58z0yq0Zuw9qbozN5L?= =?iso-8859-1?Q?bTcot6PHOvcwat1ZrwFP7kClZf7L33ozq6ECO8eX7PFJ5v+/Weo4JFF5b1?= =?iso-8859-1?Q?oWJwTb/9dWVlZRRB+jXpT7/AzPKGJmEqRG1/UEflVzeKy+hQaJMWbas1Kk?= =?iso-8859-1?Q?+vVdWXPS0dzRH3KMMcZwGv3abDuUNnpe4+UxjiPk9WJvqcXKaCiDJH24mM?= =?iso-8859-1?Q?Zx84Oxyc/oYLzMS8DehzCd8M3ZrEhMUNSWYnPPdcC2eywmCWQlIj6xh9ck?= =?iso-8859-1?Q?jKL/LKlGwZl6Rv1n4Dee4wcmgClbuYLE6ho/CwA+KDH1upyd9sHXmVqdcM?= =?iso-8859-1?Q?iVMw1gyHGB6WMsoEksn6NeixWaeK9UUvDhz2Fo10u8m8brT4s4xMdVJZ1y?= =?iso-8859-1?Q?UkgPqIGIdB1OITBHeBmvkhvYCRKO4kqWabJxHd49SXLpjdAS9q05fgKB4B?= =?iso-8859-1?Q?ES0USV3Uy0/zJyfxIyREUkFv5P/M48WC+MmoUYObg28nXTbA47opY0EsAc?= =?iso-8859-1?Q?MAvyXPI05Fv29dcE2WTq0/aL6O9oRH2eieQAvYE80g3kJ+5xr/vJbsVEF+?= =?iso-8859-1?Q?D19w1VMN9EvhSWXbmdmq3nPCNxRUenjxkm/aOQmj85tuDBTpaS/dcvZ4h6?= =?iso-8859-1?Q?lyuX16u5Op8/AyXp5nCTowCOhK/DEfMn2c76GkMWlKQ6Vm8QY2Nlgc6Kqa?= =?iso-8859-1?Q?UWYonin1mrAMklPrzxKYOMmRZQMG5za2D+TtwlPdHkbwo9Nq5bV011yFiu?= =?iso-8859-1?Q?6MkxYR/4Wiac40qO0qCYKzesMtBRstn+JFdxVftpuO/WoSBdyUVkiI3GmJ?= =?iso-8859-1?Q?J2jKFbVI4g=3D=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3495.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 80d63117-e5f0-4a10-10ac-08da0f2829ea X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Mar 2022 12:57:20.1879 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: oP42mfvv0yN2Ws/WR4OKd7ioYkASHRWASXUEQDTP/1AHRQIDC/3rhB5daB9qKcCf5bBQ780iLI9BrCbsVypjkA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR11MB4290 X-OriginatorOrg: intel.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Morten Br=F8rup > Sent: Friday, March 25, 2022 21:14 > To: Gaoxiang Liu ; chas3@att.com; humin29@huawei.co= m > Cc: dev@dpdk.org; liugaoxiang@huawei.com; olivier.matz@6wind.com; andrew.= rybchenko@oktetlabs.ru > Subject: RE: [PATCH v5] net/bonding: another fix to LACP mempool size >=20 > +CC mempool maintainers >=20 > > From: Gaoxiang Liu [mailto:gaoxiangliu0@163.com] > > Sent: Friday, 25 March 2022 14.02 > > > > The following log message may appear after a slave is idle(or nearly > > idle) > > for a few minutes:"PMD: Failed to allocate LACP packet from pool". > > And bond mode 4 negotiation may fail. > > > > Problem:When bond mode 4 has been chosed and delicated queue has > > not been enable, all mbufs from a slave' private pool(used > > exclusively for transmitting LACPDUs) have been allocated in > > interrupt thread, and are still sitting in the device's tx > > descriptor ring and other cores' mempool caches in fwd thread. > > Thus the interrupt thread can not alloc LACP packet from pool. > > > > Solution: Ensure that each slave'tx (LACPDU) mempool owns more than > > n-tx-queues * n-tx-descriptor + fwd_core_num * > > per-core-mmempool-flush-threshold mbufs. > > > > Note that the LACP tx machine fuction is the only code that allocates > > from a slave's private pool. It runs in the context of the interrupt > > thread, and thus it has no mempool cache of its own. > > > > Signed-off-by: Gaoxiang Liu > > > > --- > > v2: > > * Fixed compile issues. > > > > v3: > > * delete duplicate code. > > > > v4; > > * Fixed some issues. > > 1. total_tx_desc should use +=3D > > 2. add detailed logs > > > > v5: > > * Fixed some issues. > > 1. move CACHE_FLUSHTHRESH_MULTIPLIER to rte_eth_bond-8023ad.c > > 2. use RTE_MIN > > --- > > drivers/net/bonding/rte_eth_bond_8023ad.c | 11 ++++++++--- > > > > +/* BONDING_8023AD_CACHE_FLUSHTHRESH_MULTIPLIER is the same as > > + * CACHE_FLUSHTHRESH_MULTIPLIER already defined in rte_mempool.c */ > > +#define BONDING_8023AD_CACHE_FLUSHTHRESH_MULTIPLIER 1.5 >=20 > Very important comment. Thank you! >=20 > May I suggest that a similar comment is added to the rte_mempool.c file, = so if > CACHE_FLUSHTHRESH_MULTIPLIER is changed there, we don't forget to change = the copy-pasted code in the > rte_eth_bond_8023ad.c file too. It has previously been discussed changing= it from 1.5 to 2 for > symmetry reasons. Then, introduce some kind of public API macro, like RTE_MEMPOOL_CACHE_MAX_FLUSHTHRESH_MULTIPLIER as RTE_MEMPOOL_CACHE_MAX_SIZE = does ? So that when calling mempool create API, it can do other kind of calculatio= n, like RTE_MIN(user's new flush multiper, RTE_MEMPOOL_CACHE_MAX_FLUSHTHRESH_MULTIP= LIER). Just a suggestion, so that no need add strange comments like BONDING_8023AD= in an lib. >=20 > > + > > + cache_size =3D RTE_MIN(RTE_MEMPOOL_CACHE_MAX_SIZE, 32); > > + total_tx_desc +=3D rte_lcore_count() * cache_size * > > BONDING_8023AD_CACHE_FLUSHTHRESH_MULTIPLIER; > > snprintf(mem_name, RTE_DIM(mem_name), "slave_port%u_pool", > > slave_id); > > port->mbuf_pool =3D rte_pktmbuf_pool_create(mem_name, > > total_tx_desc, > > - RTE_MEMPOOL_CACHE_MAX_SIZE >=3D 32 ? > > - 32 : RTE_MEMPOOL_CACHE_MAX_SIZE, > > - 0, element_size, socket_id); > > + cache_size, 0, element_size, socket_id); > > > > /* Any memory allocation failure in initialization is critical > > because > > * resources can't be free, so reinitialization is impossible. */ > > -- > > 2.32.0 > >