From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 664F046556 for ; Thu, 10 Apr 2025 23:15:50 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 388A44067A; Thu, 10 Apr 2025 23:15:50 +0200 (CEST) Received: from mx0a-00196b01.pphosted.com (mx0a-00196b01.pphosted.com [67.231.149.170]) by mails.dpdk.org (Postfix) with ESMTP id 3967D402ED for ; Thu, 10 Apr 2025 23:15:48 +0200 (CEST) Received: from pps.filterd (m0096263.ppops.net [127.0.0.1]) by mx0a-00196b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53AKp45t005868; Thu, 10 Apr 2025 17:15:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netscout.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= netscout.com.09.24.2020; bh=l0OvhoppK1Xy6znwfvEj6/GVhNWbAEC4/1LS 1dQLATA=; b=G4P4r100dg+1OODI5vKQpxkQ7uoCYSe5KfENQ2R0tlKIqgJJP58h nnBJncln2aPJEtGa/GQs69DirUjcYfJWTn5oGktiSXRynp5t0Oxkz0LuDKW5HXST 4C4opfd13Ge9buTPJSbORE9j/02fxfaK3pvWgszpTm7n9Giyycx/ZSuCyXQY2Kfj jC1Hjn2MkXKLtwE68+f/UffkrXWQnjDbcqvGmpq5r/wFDJDiQqR5H3HKthBJHYwM MH+ZdftoFgVtahpLHu/0K2PtJ8h+DYyuFl8PgW2Va0eti/Cl+6qigN4M+bnW9oOx bsTB+uqq4oCN+QORB50WmjYG5XQx0dd3Og== Received: from ch5pr02cu005.outbound.protection.outlook.com (mail-northcentralusazlp17012050.outbound.protection.outlook.com [40.93.20.50]) by mx0a-00196b01.pphosted.com (PPS) with ESMTPS id 45u18extcy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 10 Apr 2025 17:15:46 -0400 (EDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ot/yXR7pDxATRy5+h6ZZbfKhMUbZZjPlfrtF2VG9XknfK/YjXmvOlZHFBYwuEqVrzIG663ateHGqa/gckvHhn6mNMHiGWLvew8hcIiDMmighLolIB94Tf8TS1acZil76307hfS4UFu/haUftEZfvOFXRWuhKR9pcgejusA77eAPT9iBPfdUvxDZzDeThwAhq65XbTgnMuZgKNAiU2qN9sU9OqzsSouKI0AZCsQRlzaK50/MYMcguWhxrlmXbmh+ZDmWS3nptkhVod2tWdz6JgF9o7D76ZL4QRVkqxozpR9da1CuCBgGRrBEzVwZxwR77/PyetfHdeh26YN+2GNutwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=l0OvhoppK1Xy6znwfvEj6/GVhNWbAEC4/1LS1dQLATA=; b=p+tSbY449lVVJHmrmocHx4RU/Sbyt1Vdo2JiDDXAgvB+6QlE3bGoJBnOjd5BNXnkjKxTqnd/a1RXj6QYqQrkTJNBmu7afNYUdBra2asZVHqrnVN+4BAhISOy8SP0NurN9ENYGe+qxaYk4+RyZ/wZVWDhy3lDWWDjPzAXytrTjV+JY4UXFrPgCp5Bt5VnqiI4DyVK4oJJBVe0nbZ4fL2oAT9LGs0lxIESimjIxB5aKd2fJwGsR/OCvJ2UDZL72IwDb1PoLhY/l9VVxKApxNhaHfP882IvGalJouSgON1WXv2h8ZuhqA6Vy71yK84hdFQwioe0hL2phtU5/WVwH9MIgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netscout.com; dmarc=pass action=none header.from=netscout.com; dkim=pass header.d=netscout.com; arc=none Received: from CH3PR01MB8470.prod.exchangelabs.com (2603:10b6:610:1a4::21) by CO1PR01MB7276.prod.exchangelabs.com (2603:10b6:303:153::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.33; Thu, 10 Apr 2025 21:15:42 +0000 Received: from CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd]) by CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd%5]) with mapi id 15.20.8606.029; Thu, 10 Apr 2025 21:15:41 +0000 From: "Lombardo, Ed" To: Stephen Hemminger CC: Dmitry Kozlyuk , "users@dpdk.org" Subject: RE: mbuf refcnt issue Thread-Topic: mbuf refcnt issue Thread-Index: AdulnN5IfTxeCHcXQsa3xaXr7ECZhQAFBrWAAGffZNAAOS6kYAAzcomAAAGN4hAAGKknAAALaIUgAAuwQwAAAPmcQAAYOK0w Date: Thu, 10 Apr 2025 21:15:41 +0000 Message-ID: References: <20250408205341.68d37d0a@hermes.local> <20250409092418.70c5eb3d@hermes.local> <20250409202538.46a08ba8@hermes.local> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CH3PR01MB8470:EE_|CO1PR01MB7276:EE_ x-ms-office365-filtering-correlation-id: 98373257-78fe-4f35-9077-08dd7874d9a1 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|376014|1800799024|366016|38070700018; x-microsoft-antispam-message-info: =?us-ascii?Q?JUPacgc0dsZFNuYxVbJ7IepvBPmWVrrQh3OaXIWBOvIOQq9ONnquKAGNam2x?= =?us-ascii?Q?wy+szUYSgbv69AkUt0ew25GFdyQEN0QNWaUbDtOJKBY1m6SDil5GQa/zvT8y?= =?us-ascii?Q?bWT+weQpiAlUrEKe8SwOc2ZeSAxiFNbRRgTSvd5MGCjmJXIF95bAUBBAUWqN?= =?us-ascii?Q?zUucdxq5flK+vlxdnXD1Pm72P+df5Hu8xBy6rdpNEU60fi/ie6M35+lujMQd?= =?us-ascii?Q?IUhfpVIGv4h/9v8e6KiESTGg4jufsa9F1PL//91DaVgpxOAexWnbmlRMFuG0?= =?us-ascii?Q?wfanKf3D9BHIJVILaohWau8QxuVI0lDxzWGtEDhMnc6/iBQO9QRQl7K+rjNe?= =?us-ascii?Q?dqbY78/YchtUWc4OEjJ0QfH6S0A7RAivzmFqhc2MtjdfmpNgrk79Ozqxq51F?= =?us-ascii?Q?umGVusWBX7B2inZh4qAKzKgRfLFwzRtdwC69c+rCUz9sG8vI2M2cusLW0/g5?= =?us-ascii?Q?YsQtzoPoSZSCP/4jjjGd6l+qBZnknCWSdDaZ95CnfiqHkHRgDz2CebUPyI1V?= =?us-ascii?Q?5ImJBYy0lh2ve/1m25+dXV6LInyN6PU1gNd39rvViibljaVrzfZ2fnfmgbs/?= =?us-ascii?Q?8UFDNpZ0xOalFyx4u79VNULBvZjFFqksqUh5QpwXGB4BSj5hU5iVitdSupNY?= =?us-ascii?Q?8v3V39LzenB8HgQZ0RN3VdDNW7h53FPkI6s5FKIxDM5mHYax+OThc8SPqGB3?= =?us-ascii?Q?kA1/IeB8z/h+0Iz9yaypKkvWDlhu9PKDAAVnnZGVBtYY/7zz0k7OvKARTs3M?= =?us-ascii?Q?Le4HProLFsmXt1ndNTy4PosuJcKYe+zRiiAY4hGJ2oPpvReUFKv9EAilRZco?= =?us-ascii?Q?YdvqtaTXnp1nJZDy25lHt1m3D3+8aDbN/1+0JROA1uIQ+5HNvUnlYl4wZV8i?= =?us-ascii?Q?nRRgMgICemBI5siCsw7K6ikB4ubMlMTRa1NXVThfSi1fPxrfUXr5D8ATauWs?= =?us-ascii?Q?ao0D7jOltUPRwsu4HydUBCDzZUVHQYswJwziPGQG8NdrnY2eqrFkwRL+77r2?= =?us-ascii?Q?fboGS/VrzEv2euAqnZqpNDPvwEYnmt1b3ryL04cEc4sijcWRFvWtWR7d09OD?= =?us-ascii?Q?8Jz6lNjLIHYyxTJB5MeIvcdIxd4GfqU7ONHMt1BFgxYqgA6VVMxlTbVG2rPh?= =?us-ascii?Q?ZKKWiTpSaqpSgH0sUsG5jlS6tTpsp9GubdLXWOwzD/agMqfKu7yo5l60LHS0?= =?us-ascii?Q?HxsAKvCZsUnT5JivXoRFdkJ46WeqokAjM/uviaUE3bHGbVg98tZHgzpFnaU/?= =?us-ascii?Q?4waBnNzBKoo/kp+dQ3Ftp1t6UIMxMhc/3rrePhPjCEGA/TA4Rtf6TM8Q2ud0?= =?us-ascii?Q?oTda0LIHAirJOwuFvsine4DRXiIVRrIzbnSfDDcao1e6mtCZPUSRJe+vpeOG?= =?us-ascii?Q?X/66Gy9zxfHd1LY7M9RIzFtYt0ZDEVuWTvZkj2MbxPY2WpRDIXl/2hUZ8P/b?= =?us-ascii?Q?L8+l0vFyOm5Myc+BgPBSwLdrib43+cs90MKClZdItuRrFNp78fMjuQ=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR01MB8470.prod.exchangelabs.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(1800799024)(366016)(38070700018); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?g7q0RnGdI/fL5qckfyitJaAbQl9yAG/00az0ZkLCuWx164k/yjXgZDwl5OrA?= =?us-ascii?Q?VDyNqaVe52dpY9TNG88oP8MBw6iAgQIJCAfL3n0x1r3mSNBN0aqYn3Ms681H?= =?us-ascii?Q?HwdB5TX2rCo+VxXATjxvseGUojvU7v5jVakDTZTbnTaGgkM6tM3n0BGzF1Np?= =?us-ascii?Q?7ysSESamf41I61xaMRGbhiQmspBQQ5GdzNmtoqRQTJjPs+/uRFmDQLWjvADM?= =?us-ascii?Q?T+MuMlmTc3nbmRLT7vXr5By+AjVHG5SiIvsxY7WIGrU2Cfo0KxTsPPsnvmNH?= =?us-ascii?Q?yejW1NI3lKTVs3Ks6d6LDue32BmX9TT1Tq1JnMZhp4ePIUHBtnwJMurkLLbc?= =?us-ascii?Q?rACNY58h2t0zd9/mr7Gdh/v0cdGYnZJ8CjAh8y393APX3XlCVEdHLIsyVZD6?= =?us-ascii?Q?gk7ODM0BMOaYOSdneRsbj+Jo9NHSfKR2tFovTQ78jtG8m8vsDGJzPOqXgNBu?= =?us-ascii?Q?vKsjmMdVV7RWKaJkzFbwFN0CMtMFnVis0zKzMVr9csWiUrDpEvSQqw7kOgKY?= =?us-ascii?Q?dLYlW1nKOldNqwkDZIzYt6D6mdrLGy0C9P0w3BiZoEKozfQ7ODzs6cl7Yh3v?= =?us-ascii?Q?3cbCqgcBWA6zpsoxmcHf7qoWoh+xfGKx3cWGBzkRkjsSzFa//a09vFxMsKOp?= =?us-ascii?Q?x/29HkXljW7BWmEhSfKQ8Qzf9DNM9Cfm1nx6D5fM4Kb381sSPI0MuuVoL9mY?= =?us-ascii?Q?EzL9Db2Lg9M6fgKCFwE3wnvpy8jlUPZQ9K6w9JFsE7bND2J6KyJzFnzdHzRG?= =?us-ascii?Q?i+3cm6RabX8Tl8GyRb9JcIjYwNjklauanKjYATL6TYSFVESRcsiOGJNlYiZ4?= =?us-ascii?Q?bMCGzf+3fxJF5JrwDOewZRxyv5E94Qwz4AW4Ps8OSN8m1j7EZi+kck1AcJvZ?= =?us-ascii?Q?/fy88wL5rkpVoDb94UA5iIdiKRC2cTXEMfL/V8mBRmwPQqIi4R9noKIWjOwX?= =?us-ascii?Q?Lkz573W1z5+lWICW8/ekWEkHt55L/C5qzlOKGfJUQ4rs4SkZvaRlOYIq040k?= =?us-ascii?Q?GTRsZGZhIk1/fIdPqS280lng7a2Y8Fd0yUfrrijP8mgpkpktPoCCXz1gZUSu?= =?us-ascii?Q?JZr5hoR6BJceXJSxrqX1wP1u7aMNIUcXb1Pd3VctQ5/atzYpN92ji58Xjh9u?= =?us-ascii?Q?rPFFT3EAn1CquGj8G89ioz4stJXuEzivdka7NNgvjfmc8qdbDwXI+oVtClMF?= =?us-ascii?Q?SxBGQ96T8a3X+fXoLw0GYnkotUGjcRLc3yM9w3+P1Fxd+aaib/uxp4xO+w+u?= =?us-ascii?Q?W1k/m9eAYGJtPl5zOlwo5wzq7t8Qf13hPmwCAA1NTJz53v75d1mUqHq2iJV1?= =?us-ascii?Q?HM+M7Y/LtzgvTvDRFvMDd1eXxGQatqUYHCKvXlO0p7QPqwdlROllKoyaL8aL?= =?us-ascii?Q?N5h/NoFta5oUANT5QbJwhdP4/31yttIHsRteoHKHjrVao7t2TaL/Hb7rKsT9?= =?us-ascii?Q?rUFePmgl8EKJCwmchdRWjm00QaOgPXSbKali6CpjhvS0JmgEyx+30L0ciORP?= =?us-ascii?Q?tQ6ldtJQ5qN29Ysgq5JK2r8fhoUEJyP6vgw21Vccn5NSo0KEpNYUzcvbWR9s?= =?us-ascii?Q?wNiN9E+ax7JdWMLcAvHG+ldwhswETQ53u0dFICbs?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: jwlI5n0ocTyuRAbFkSamH+mvFDbVWnlIBluJTssOikzj1GEMLqsqpcJ3sjt3RP7q3fcW7QVMV3RvJj0aWk1+ROsq/muwMMwDXloSrLJLB/IUlI13FrD6cgMeQRguiM1q7cZXCcfFotDY7j3lHvWJdrHIm2H1tSAEfeYuVNygvDpL2hecJgZelHDO3LTzI3KO5zciPgN3ZS9pLETPFJxrD9d7lat07YkeCWkmfhJb1cNVuHl3+iNgEuB6gxiQ6755XcXPVnHzLVqLRSF/qECBjvFmvo3oevu87LeaRya2HdRZ9l4oguKNUxw+fvmPryGTNLzHVB6jTmF5jHKJycXnSiViWMlY0l3e4mYh54Zlvaz9Z8RdQWmKaEAG3vxoWg8DUdljuiKKzuaYzccswcG4cBQRyG9hu5tiIJebbsaIlLovr5BZpJLIv6EglqXTo48DFcrMkEgRTJfQxjpp/ff+cYfSQLz9muQ1dXsGrAGBL7endPihxG1/d8HRVCbyLnV7O3HPyEZG9Yiy/ZTWbOoUsH6hih1kJNzmD3DbIoI7QC8FwwMLJrCthGPTXH+azQYlQ8UDQsI+pcB0eH+btwoMoImksk+pB47zF0P+TnK9B+88QNmh69dQlKnAJStLoFH8 X-OriginatorOrg: netscout.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH3PR01MB8470.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: 98373257-78fe-4f35-9077-08dd7874d9a1 X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Apr 2025 21:15:41.8655 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 54f11205-d4aa-4809-bd36-0b542199c5b2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: WWHgQtbUEFZ91hrP8R54D8G2wWAXZE1W6FKXjQpWRKiIej6G+Rvky+OcGv0gg4AQ4HqekNQFxtaV6lF6CiYnGg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR01MB7276 X-Proofpoint-GUID: io-QxGqrikT0bjMRPqSitJnFx8SUqkS5 X-Proofpoint-ORIG-GUID: io-QxGqrikT0bjMRPqSitJnFx8SUqkS5 X-Authority-Analysis: v=2.4 cv=VcP3PEp9 c=1 sm=1 tr=0 ts=67f83502 cx=c_pps a=Al84TKQ7ImdPUwhB59KusQ==:117 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=kj9zAlcOel0A:10 a=XR8D0OoHHMoA:10 a=jZVsG21pAAAA:8 a=pGLkceISAAAA:8 a=8rWy6zfcAAAA:8 a=jQOgFn-ZAAAA:8 a=7CdMl5eaQ7zADWMlK_8A:9 a=CjuIK1q_8ugA:10 a=3Sh2lD0sZASs_lUdrUhf:22 a=YjdVzJdQTyZRADMV7wFX:22 a=mT82qxFQzDvLIExZS32s:22 cc=ntf X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 malwarescore=0 suspectscore=0 phishscore=0 lowpriorityscore=0 adultscore=0 mlxlogscore=999 mlxscore=0 bulkscore=0 priorityscore=1501 clxscore=1015 impostorscore=0 spamscore=0 classifier=spam authscore=0 authtc=n/a authcc=notification route=outbound adjust=0 reason=mlx scancount=1 engine=8.21.0-2502280000 definitions=main-2504100154 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi Stephen, The overnight tx test passed with no problems. I modified our application = DPDK driver this morning, I set the mbuf refcnt in the second stage of our = pipeline, I then enqueued the mbuf pointers to the ack ring. Then I added = the same mbufs to the tx ring. =20 Then I provided Mobile traffic and the same MEMPOOL log was seen but in add= ition it failed to transmit. I checked the rte_mempool_create() has the flags setting to value of 0. Th= e flags setting to 0 is not an issue for my usecase? I removed the high layer application, if it was corrupting the mbufs and th= is has been eliminated. I then tested the multi-thread tx with rte_eth_tx_burst() and second thread= doing the rte_pktmbuf_free() and I see problems again. =20 Pretty convincing that these two DPDK APIs can not be called by multiple th= reads and maybe the rte_pktmbuf_free() must be called after the mbuf is tra= nsmitted. If this is the case I am back to square one. I verified that when rte_pktmbuf_free() on a mbuf that has refcnt=3D=3D 2 i= s decremented to refcnt=3D=3D1. The rte_eth_tx_burst() should also decrement the mbuf refcnt by 1 if seen a= s 2. In theory I believe that the last thread that decrements the mbuf refcnt wh= en seen as value of 1 should trigger that mbuf to be returned to the mempoo= l. Is my understanding incorrect? Thanks, Ed -----Original Message----- From: Lombardo, Ed=20 Sent: Wednesday, April 9, 2025 11:58 PM To: Stephen Hemminger Cc: Dmitry Kozlyuk ; users@dpdk.org Subject: RE: mbuf refcnt issue Hi Stephen, I should mention that I added mbuf refcnt checks at all pipeline stages, an= d what caught my eye was the first stage of the pipeline the mbuf provided = by the rte_eth_rx_burst() I logged mbuf refcnt set to 2 when should be 1. = S,o something went bad after the event that triggered the error to cause th= is. The mbufs kept arriving so the mempool was making available the mbufs = for receive of packets from the NIC interface. So weird. I will try the patch your provided. -----Original Message----- From: Stephen Hemminger Sent: Wednesday, April 9, 2025 11:26 PM To: Lombardo, Ed Cc: Dmitry Kozlyuk ; users@dpdk.org Subject: Re: mbuf refcnt issue External Email: This message originated outside of NETSCOUT. Do not click l= inks or open attachments unless you recognize the sender and know the conte= nt is safe. On Wed, 9 Apr 2025 23:22:50 +0000 "Lombardo, Ed" wrote: > Hi, > I just finished modifying and testing our application to just do transmit= of packets received on an NIC interface and let the rte_eth_tx_burst () fr= ee the mbuf and all works fine for both traffic types. This proves to me t= hat my implementation of processing the packets and queueing them to tx rin= g and transmit from the tx ring is not buggy, which I had carefully verifie= d in gdb early on. I still believe there is a problem with our application= with many threads that can do rte_pktmbuf_free() on the same mbuf. =20 >=20 > I added these lines in my driver source file: > #define RTE_LITRTE_MBUF_DEBUG 1 > #define RTE_LIBRTE_MEMPOOL_DEBUG 1 > #define RTE_ENABLE_ASSERT 1 >=20 > I don't see any asserts occur during my tx packet testing. >=20 > The dpdk header files show the Atomic ifdef checks=20 > rte_build_config.h:#define RTE_MBUF_REFCNT_ATOMIC > rte_mbuf_core.h: * or non-atomic) is controlled b= y the RTE_MBUF_REFCNT_ATOMIC flag. > rte_mbuf.h:#ifdef RTE_MBUF_REFCNT_ATOMIC rte_mbuf.h:#else /* !=20 > RTE_MBUF_REFCNT_ATOMIC */ rte_mbuf.h:#endif /* RTE_MBUF_REFCNT_ATOMIC=20 > */ >=20 > I verified in building our application with DPDK rte_mbuf.h header file t= hat the atomic functions for mbuf refcnt read/writes are turned ON. I adde= d junk characters, and the compiler spotted syntax errors. >=20 > So, I am back to the question as to why I get mbuf issue with multiple th= reads processing the same mbuf? >=20 > Any more suggestions. >=20 > Thanks, > Ed >=20 > -----Original Message----- > From: Stephen Hemminger > Sent: Wednesday, April 9, 2025 12:24 PM > To: Lombardo, Ed > Cc: Dmitry Kozlyuk ; users@dpdk.org > Subject: Re: mbuf refcnt issue >=20 > External Email: This message originated outside of NETSCOUT. Do not click= links or open attachments unless you recognize the sender and know the con= tent is safe. >=20 > On Wed, 9 Apr 2025 04:46:09 +0000 > "Lombardo, Ed" wrote: >=20 > > Hi Stephen, > > I am looking a the rte_mbuf.h file for rte_pktmbuf_free() and it is not= clear to me that it checks if the mbuf refcnt is 1 before decrementing it = and allowing the mbuf and segments (if any) to be returned to free pool. > >=20 > > Could my application issue be I have tx threads that transmit packets a= nd does rte_pktmbuf_free(), while one other thread will perform rte_pktmbuf= _free() on the same mbuf? I ensured I bump the mbuf refcnt to 2 before oth= er threads can process the same mbuf. > >=20 > > Thanks, > > Ed >=20 > It doesn't need to check refcnt there. The check is done later (since mbu= f can be multi segment). >=20 > rte_pktmbuf_free > -> rte_pktmbuf_free_seg > -> rte_pktmbuf_prefree_seg > =09 > static __rte_always_inline struct rte_mbuf * rte_pktmbuf_prefree_seg(stru= ct rte_mbuf *m) { > __rte_mbuf_sanity_check(m, 0); >=20 > if (likely(rte_mbuf_refcnt_read(m) =3D=3D 1)) { > normal fast path. breaks the chain. > } else if (__rte_mbuf_refcnt_update(m, -1) =3D=3D 0) { > refcnt > 1 logic >=20 > Note, the refcnt doesn't always go to zero when the mbuf is put back in t= he pool. > The refcnt for a freed mbuf (in the pool) doesn't matter, it is free, it = is dead. > The refcnt is reset to 1 when mbuf is extracted from the pool. >=20 >=20 >=20 You might find something by poisoning the mbuf when it is freed, so that an= y attempt to use the data would get junk. Also, add check at start of rte_p= ktmbuf_free_seg() to catch dup free. Something like this, but only compile tested. It will break some of the functional tests, because they tend to make bogus= dummy mbufs. diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h index 06ab7502a5..60= 88b34506 100644 --- a/lib/mbuf/rte_mbuf.h +++ b/lib/mbuf/rte_mbuf.h @@ -1423,10 +1423,11 @@ static inline int __rte_pktmbuf_pinned_extbuf_decre= f(struct rte_mbuf *m) static __rte_always_inline struct rte_mbuf * rte_pk= tmbuf_prefree_seg(struct rte_mbuf *m) { - __rte_mbuf_sanity_check(m, 0); =20 - if (likely(rte_mbuf_refcnt_read(m) =3D=3D 1)) { + __rte_mbuf_sanity_check(m, 0); =20 + unsigned int refcnt =3D rte_mbuf_refcnt_read(m); + if (likely(refcnt =3D=3D 1)) { if (!RTE_MBUF_DIRECT(m)) { rte_pktmbuf_detach(m); if (RTE_MBUF_HAS_EXTBUF(m) && @@ -1435,13 +1436,15 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m) return NULL; } =20 + m->refcnt =3D 0; if (m->next !=3D NULL) m->next =3D NULL; if (m->nb_segs !=3D 1) m->nb_segs =3D 1; =20 return m; - + } else if (unlikely(refcnt =3D=3D 0 || refcnt >=3D UINT16_MAX - 1)) { + rte_panic("mbuf refcnt underflow %u\n", refcnt); } else if (__rte_mbuf_refcnt_update(m, -1) =3D=3D 0) { =20 if (!RTE_MBUF_DIRECT(m)) { @@ -1452,6 +1455,7 @@ rte_pktmbuf_prefree_seg(struct rte_mbuf *m) return NULL; } =20 + m->refcnt =3D 0; if (m->next !=3D NULL) m->next =3D NULL; if (m->nb_segs !=3D 1)