From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6EDFF4650B for ; Sat, 5 Apr 2025 00:01:07 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0EE3040144; Sat, 5 Apr 2025 00:01:07 +0200 (CEST) Received: from mx0a-00196b01.pphosted.com (mx0a-00196b01.pphosted.com [67.231.149.170]) by mails.dpdk.org (Postfix) with ESMTP id 5EDD9400EF for ; Sat, 5 Apr 2025 00:01:05 +0200 (CEST) Received: from pps.filterd (m0072398.ppops.net [127.0.0.1]) by mx0a-00196b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 534Kcs49027526 for ; Fri, 4 Apr 2025 18:01:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netscout.com; h= content-type:date:from:message-id:mime-version:subject:to; s= netscout.com.09.24.2020; bh=amggairr/gzxjVTweR9Dy0lGo5MbvM6QMYaY CgZ4xbU=; b=WXRiHJJZ7XqZ+qU9BHf/u7oQ/A5yUw1U3BjE7CAG0Oedq5VTi4ny osYI8w8+XswwR6UlUh15GNTxhDUjiImE3icNC60jlAMe5y/Zg1BVio3zHVIsBD+p CmpFp8AWFNw+rVbq/OgXWUNYT1fRSAY4TnvbF/BW/PuDg+FRqskOeK6xpsvQ/Bh4 hSWsGPgIlIZfLFMZ4fdkIWMNkCmXJfv9qoMpD1IFZA1aLvQV03Kob9iDf6OAElse h6cohJk83GyHXoz+HJJPz8lT8nm0gQr8o3fG6KMF6GPukwZ6Jv/N8cDnOcQfv5XH +3EdS5rdsC+eJy7tNj37StbKbgmxhiGyXg== Received: from sj2pr03cu002.outbound.protection.outlook.com (mail-westusazlp17013078.outbound.protection.outlook.com [40.93.1.78]) by mx0a-00196b01.pphosted.com (PPS) with ESMTPS id 45sqvfj1wd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 04 Apr 2025 18:01:03 -0400 (EDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wwyvmr1N5d1fnAfF1VYQlsQT+9hxMiWe3V4p5lD47jjxjDhdZITGJin+1+JuyDqEz1r8ikYhuSESgwNWLb1CUBcrswAa9a+AIMKq9pqHHhGrLEk5SFlmmlir0225xRzTVATIwTuWO+Ce5SdG+Kjq7/3FsdikyPYYI0ubMQ7KkIk5Dv3F+k02LEg19CIH2Wp5hSuGsbey/bMhx+dkoGl0iVYO6OZhZEXSAqG5suZLh/QvcmlqLxKKQoEJeLxgEAd3dmtgkVjR8KonDTCusXjx30hDrUMuB5vQWjtS1Rm2z4/yzSohQkIFn57DgOzEnoFX0lD3wxVb0+fcy+3J0CarKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=amggairr/gzxjVTweR9Dy0lGo5MbvM6QMYaYCgZ4xbU=; b=HOJC3m2P2aQZBbCaUmU7DK2ccWQ28rq3mElI2vmYY6oKs8lOAFjn8FupWeGDG46dPcsXih4MWWIBK3T87kUtmXNEcG0dnixeVeZlz8nkrGTo7+SZ8HwB3zcDKb1aVsC6Ba7G66Imo8rfsvcsY8l4o+g38tnPMJSdsRNN4Q0R80iqVxwc9WnTTxALfgN2vRpsipuH7YZQXTi5CNICMhAR9gRJU+Ah4U87TElo9KcKEzSnfPHdWArz21WgzgWxLUmUvU0MvPnrwJZdSf4hRy4/RB/qTfFWDvslHBZMhWaENyvKJD3Nb+75TtJGBkv56mVggwoUmWpVoRgnihEUB6c+8Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netscout.com; dmarc=pass action=none header.from=netscout.com; dkim=pass header.d=netscout.com; arc=none Received: from CH3PR01MB8470.prod.exchangelabs.com (2603:10b6:610:1a4::21) by CH5PR01MB9078.prod.exchangelabs.com (2603:10b6:610:212::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.52; Fri, 4 Apr 2025 22:00:59 +0000 Received: from CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd]) by CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd%5]) with mapi id 15.20.8583.041; Fri, 4 Apr 2025 22:00:59 +0000 From: "Lombardo, Ed" To: "users@dpdk.org" Subject: mbuf refcnt issue Thread-Topic: mbuf refcnt issue Thread-Index: AdulnN5IfTxeCHcXQsa3xaXr7ECZhQ== Date: Fri, 4 Apr 2025 22:00:59 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CH3PR01MB8470:EE_|CH5PR01MB9078:EE_ x-ms-office365-filtering-correlation-id: 013cb15d-d0ca-4ca6-cc41-08dd73c42ec7 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|366016|1800799024|376014|38070700018|8096899003; x-microsoft-antispam-message-info: =?us-ascii?Q?+XAaqNirh5DzeBL1pZ5YG/Ru3lXmWyMezdMonnm1DKOK365SippVV1dLK+2q?= =?us-ascii?Q?7XoOxCtcPdl9rxCwgZPnfg6E1iImJndKS1fp3/Zq5fwJiW87Ah8qBG2VgCrN?= =?us-ascii?Q?BJt2s+KJghvZfMpHDIdQMnOGZl52p4xtuHNGnX8aHXorpyyycoRyTwLPckTV?= =?us-ascii?Q?UVosf/nyI8FUpf3uHA/wT6UjcV/1dR6zs9lupFqkhWZoMaWFBrDG3++p695+?= =?us-ascii?Q?36f8a0JFd4e9uvcu15a4RNJEa6ieLepeb+FzZSBIeGsCImkpBRzl9hMg1GVy?= =?us-ascii?Q?Ua0h7DBzVSw34K8MQihQaByI2h/oAAeiD9RG88FtILvnSitQMDLbIyeJR5lm?= =?us-ascii?Q?UtDxkykIwZ7MehZOqTOXfWuSmtkUbY8BiywmmZZ7BXDL72u2UuxPbKso6dVl?= =?us-ascii?Q?UDRAjIfFxF7SgH/mKDJxql9JRZdVTLhTdaxghsB9whAb5gkAxF97/TtgNmog?= =?us-ascii?Q?OsBBFbfamdPDZqol6KD22jAHFtPV8sFmtipQM/lj9/Z/5VdWoCiqMAo16+g5?= =?us-ascii?Q?MnE2qpiBAFrIZ+BFs81aMLBYTkAyIyM3RyuW5/luISDlIYAOtWC8XYmZXBwY?= =?us-ascii?Q?7W5rgbzOlszteQTRgXmYkbQZufDBGDA4ZpL4S75pqd25Vq6Q73Kc0y+zhGVC?= =?us-ascii?Q?llT1zEC9RAO1V1+pdI8XPrZVysaOkOFOeW87y+1MSPLKpt3QzAWEkCZ9m6zw?= =?us-ascii?Q?boHMCxYlgh66CMEg5ivjQTPTv7dgMHFTr5seJPIBDxuLW2NCmvibmePhsT6a?= =?us-ascii?Q?frjdIEmC9YQRUMVGT2cdNF6YVCAXHFOV1AnUImJMOn7Zl4BqWf5O8flVdZp/?= =?us-ascii?Q?fnpSFlGaDodyk/feB2uj8I5yZU9SCBMG59rNOxM7vY6X8TJplEs5a2X87RhB?= =?us-ascii?Q?EDFdyDgDq4lora6bUYNtlNqMWw6xato1KVLfjU8yRw00yLqbBDZzqwZqQnUJ?= =?us-ascii?Q?Yt4hkCnqYiIESBxJH17C0+uFMgcf/pxAATvqXKrEJ0alJH7nttRZVRn/pqCE?= =?us-ascii?Q?TY3/Ia2i/mIYHWZaIScTY18fcwAkmY0PtzA23c5Ioea6cZRnnsWPU6Ul5zez?= =?us-ascii?Q?TKFUS215XGjDphEieLfnzxDwkM7N3s8ZW97D5Vp6mCHT0S80elHSKCj8EJ89?= =?us-ascii?Q?Vt74P44qCGsdRpMM7r9677FIFsKG9Z9fzBPf3EU9IIjs/97QJb65QHog9MJI?= =?us-ascii?Q?qmWo9XeVH29Cvmc0QX51R4h115Kh43lTMQ4OFQds8d+A82Hg02esb6OLb8Gc?= =?us-ascii?Q?g3/E73NS82VLlxzXg9LDGgkiHiEHrt7eOdzt73hdDdg2VuOjuFENqb6Ks/bm?= =?us-ascii?Q?4LHQsseFhUpsW5r3n90v2+tGnDlN3kbTuYQvn+GXK0Yk74fAf4T1K4aMQ3R4?= =?us-ascii?Q?P7EdI7tEDlNZRZVmG5G6adxc4aK5xpKnOhHmmKDRuuXVbZN3BKZ6LkBgUuPD?= =?us-ascii?Q?FJSWm77Gl0tD/BrVJg7STtI28StrSQjHqnl9V2SK9s/n7gfeP0tXEw=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR01MB8470.prod.exchangelabs.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(376014)(38070700018)(8096899003); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?XsHKvwKwkx9GdGfAgwOJEVUSaOgg2maA3MQH53lDQ/q6z+FDa89NteTHZZNR?= =?us-ascii?Q?unIEQd1nVKt+WsdYveovnJlWlqYkr7PnqQLqRsIYZg1u4zGLT19uJKxKh1Nl?= =?us-ascii?Q?uBLhJuBeyNnBp37ZXQBUXlTcS+R7vNciyVErlYTh7OIzjZ2vPUWr8+5D3xpo?= =?us-ascii?Q?MnYIWPU7w3hGdsXMDSo12hL0QEtAqwkw+MHx47YSbZfcFRaeg61o0ynuN/Y0?= =?us-ascii?Q?KV2zDSkA+H1q6AeLBEXKI/GnfH1On3qnGj2myGkYG6onS6zJ9uQWwhOH7bkf?= =?us-ascii?Q?zJEr7N5Rj7Cnin3WrNLEHQKo0+h9P3aLZ9OiSIds6g7qJRS+jEEMzysyfn9o?= =?us-ascii?Q?UkEGadSgZ+/nRlSzu0av3JIAAeSY89VSYhHUujTvpY8tIIpytrWFQI2Jm5ZE?= =?us-ascii?Q?ZRdlavCZdIM0hPZGtMAoJO46H+duy93BbPtwWzc48xRF37Wy5Ix5atSgUfdK?= =?us-ascii?Q?yvl4S110ZrxTmGabr2YLCT7hDkLjGcQbfXeUsSqOgEiWVhH0nHkJ9Tl/hmyi?= =?us-ascii?Q?596Kx2yzXl4dQ26+2UIUYXH7++RZX/fc0xgtkVCkfU3eOUgg6UsrGzWYB5Tf?= =?us-ascii?Q?DeZPWblPlvjh5ny89UxIwxiRQrdeBBuvjToTmeRxenqrNYM4JPLQ7eZgJtbX?= =?us-ascii?Q?/QBzswsibuue6EDpyQn7vy+FOZ0isRwQJ0Ps2t6KKk9+iGOEjOdcu1a4OOq8?= =?us-ascii?Q?eHeRNfYGywCa24R/ZQVTe1GAdSYy7TcTDrYjSWprLd7nul3tqkLp9SD11sVM?= =?us-ascii?Q?e5A1OColSTVAca6ySsVyFAy6MZwObvvNbedqD8p/K8lmtN2cS8NgYVS5wvMC?= =?us-ascii?Q?s+Hcqw2RRwJBhO0VLqFtVCaFBAVI5Z+VIYveeba+QyrSZdjJLo6SoCw9b3nb?= =?us-ascii?Q?XMhhw0MI65kF8Zvp9GS2TXwVsJYY2uzrTFZW63C3gYF5jae6Im5jd6H8JLq2?= =?us-ascii?Q?HWfc+5QIArjks0Nm1kdy5LusxB9xDsoXoazJpbGWbz+qSfdJy7tdFfinilCM?= =?us-ascii?Q?NiFDnlinyjACy/GSpFo30fRO+8CZZwN+KpUdxKyS5BGe1sywZlNAvIb2yxYr?= =?us-ascii?Q?4RfDsJZq2RQ2ll4Jy3fy6caufnIJYn/0Xu0Rrc3CJDKYeA5Szz3dGJzTiA/b?= =?us-ascii?Q?QKgLguKYrmeQ5qB8WEHO4YtWCG9tfAYza/TmYpDCLXWdXRZGp4SMEqgcTdFL?= =?us-ascii?Q?kTUgeX3hIROpy2mD8ZwE8ItobXdTW+2j5Nhriz9pSzHHN7ljS7MTHmgh398o?= =?us-ascii?Q?ZngNkRYT6NEh7H2vneTT5XCatfKN7RpalkwfNJjRBcL5uzC44NRVT0UAXVgz?= =?us-ascii?Q?i9OKeh6TKq5+6Rn4uuzPpSirDzO9C+eErsdTeIae1cT5HSD/wBZAVtBhOC0J?= =?us-ascii?Q?vLsm+zmlX/ZKh6y7dmhrOjIy5cLclWVHA9UD5MuvML2/7FZolTD6A60YFkD1?= =?us-ascii?Q?vDy5U+S4YMhbCy2MVD9xCllmuCutVa37mPOAOgV5CmeexTLgSCKpdJPiDWH4?= =?us-ascii?Q?sUKDyf0pFOIGWOWbAZ8QjadFvsT1+40uPV5S5Vjy5PRsQ4TKN+u8MzV+gxdc?= =?us-ascii?Q?Cb08RLGUxLxyZVmq0itv1xaM4f5JWt06EDEbecSs?= Content-Type: multipart/alternative; boundary="_000_CH3PR01MB847068FB4D2669E00351EAA38FA92CH3PR01MB8470prod_" MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: s0rkx4eivsIhdWchN6ZfNKgAhzp0cMFx0iKgZiextlrKs8AC/gwmcl5OrBEvAr/4UNjS46+jrcxzD2+nAkYxJjkLk+BOmPJiUQWQKrb3nqRHf7Aioxvom3e8Mc36Qt0goAW4ENnOQxRGSzCqKwCx6P9Aae56gvTdR8oCaTwz8sYuDDH6FzR8GO+O58HP73GqMNJMoyl9H6Sw8NQGbJ6BNIogy7y3LwmdO53zZYw5ZaAwk4J0dgbA+EjKEjIzjHbNK+9kcJF8njQ+CwHiwnRmtfrUHjQdmvMoFcrYWkmUy9Wf1ZRx6A92QHzutY7mR3301R32RCGz/pfE0hc3loAuTGMd6L/6Oie9MRWnKc+nOQSSXwOANDVamt202b5r/CIkUHSoQL+lPChHGmnpUp6p35R8uvfmDbowZjXk/lXHF30sJPXVVt8b9NfUB9UnOeJPo+a4J5BHFAfIWBLBf0o4Dn9pcklH6hDPfUFGWQs32E1yZ7znhJDbMZWmnke1C1vLjsR1bQ4mQGdYAya0ASRxTafrctYnVPfAqdhmv3q6Ma0lPAc00JBsA9iI0BM98giz/npw9dkTsScv+ObXDFmyt5sNg2mqKV7b5UVFlwd2tH6f4NaLX1SpfTeIIkhK81r0 X-OriginatorOrg: netscout.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH3PR01MB8470.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: 013cb15d-d0ca-4ca6-cc41-08dd73c42ec7 X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Apr 2025 22:00:59.1750 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 54f11205-d4aa-4809-bd36-0b542199c5b2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: wRPpXvnyEa/qxNiVeo53duveGOzOLQOMDpjtvYIEr4/7THigVaMXRfhRrhRP6+0YKj8ZbjVcE1KKfcG8/8ESyQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH5PR01MB9078 X-Proofpoint-GUID: 07H9hrr-k6KrfB4XLaov_TvmSstFBEal X-Proofpoint-ORIG-GUID: 07H9hrr-k6KrfB4XLaov_TvmSstFBEal X-Authority-Analysis: v=2.4 cv=cJTgskeN c=1 sm=1 tr=0 ts=67f0569f cx=c_pps a=l9cCpZ6fIogKje1+qubiRw==:117 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=XR8D0OoHHMoA:10 a=4hB67t2KgUFhEHKIYxQA:9 a=CjuIK1q_8ugA:10 a=yMhMjlubAAAA:8 a=SSmOFEACAAAA:8 a=V5_0iL-FRsSiX5QdRdMA:9 a=gKO2Hq4RSVkA:10 a=UiCQ7L4-1S4A:10 a=hTZeC7Yk6K0A:10 a=frz4AuCg-hUA:10 cc=ntf X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 lowpriorityscore=0 adultscore=0 clxscore=1015 impostorscore=0 phishscore=0 malwarescore=0 suspectscore=0 mlxscore=0 bulkscore=0 spamscore=0 mlxlogscore=748 priorityscore=1501 classifier=spam authscore=0 authtc=n/a authcc=notification route=outbound adjust=0 reason=mlx scancount=1 engine=8.21.0-2502280000 definitions=main-2504040151 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --_000_CH3PR01MB847068FB4D2669E00351EAA38FA92CH3PR01MB8470prod_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi, I have an application where we receive packets and transmit them. The pack= et data is inspected and later mbuf is freed to mempool. The pipeline is such that the Rx packet mbuf is saved to rx worker ring, th= en the application threads process the packets and decides if to transmit t= he packet and if true then increments the mbuf to a value of 2. The batch = of mbufs to transmit are put in a Tx ring queue for the Tx thread to pull f= rom and call the DPDK rte_eth_tx_burst() with the batch of mbufs (limited t= o 400 mbufs). In theory the transmit operation will decrement the mbuf ref= cnt. In our application we could see the tx of the mbuf followed by anothe= r application thread that calls to free the mbufs, or vice versa. We have = no way to synchronize these threads. Is the mbuf refcnt updates thread safe to allow un-deterministic handling o= f the mbufs among multiple threads? The decision to transmit the mbuf and = increment the mbuf refcnt and load in the tx ring is completed before the a= pplication says it is finished and frees the mbufs. I am seeing in my error checking code the mbuf refcnt contains large values= like 65520, 65529, 65530, 65534, 65535 in the early pipeline stage refcnt = checks. I read online and in the DPDK code that the mbuf refcnt update is atomic, a= nd is thread safe; so, this is good. Now this part is unclear to me and that is when the rte_eth_tx_burst() is c= alled and returns the number of packets transmitted , does this mean that = transmit of the packets are completed and mbuf refcnt is decremented by 1 o= n return, or maybe the Tx engine queue is populated and mbuf refcnt is not = decremented until it is actually transmitted, or much worse later in time. Is the DPDK Tx operation intended to be the last stage of any pipeline that= will free the mbuf if successfully transmitted? Any help is appreciated to help me resolve my issue. Thanks, Ed --_000_CH3PR01MB847068FB4D2669E00351EAA38FA92CH3PR01MB8470prod_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi,

I have an application where we receive packets and t= ransmit them.  The packet data is inspected and later mbuf is freed to= mempool.

 

The pipeline is such that the Rx packet mbuf is save= d to rx worker ring, then the application threads process the packets and d= ecides if to transmit the packet and if true then increments the mbuf to a = value of 2.  The batch of mbufs to transmit are put in a Tx ring queue for the Tx thread to pull from and cal= l the DPDK rte_eth_tx_burst() with the batch of mbufs (limited to 400 mbufs= ).  In theory the transmit operation will decrement the mbuf refcnt.&n= bsp; In our application we could see the tx of the mbuf followed by another application thread that calls to free the = mbufs, or vice versa.  We have no way to synchronize these threads.

 

Is the mbuf refcnt updates thread safe to allow un-d= eterministic handling of the mbufs among multiple threads?  The decisi= on to transmit the mbuf and increment the mbuf refcnt and load in the tx ri= ng is completed before the application says it is finished and frees the mbufs.

 

I am seeing in my error checking code the mbuf refcn= t contains large values like 65520, 65529, 65530, 65534, 65535 in the early= pipeline stage refcnt checks.

 

I read online and in the DPDK code that the mbuf ref= cnt update is atomic, and is thread safe; so, this is good.

 

Now this part is unclear to me and that is when the = rte_eth_tx_burst() is called and returns the number of packets transmitted = , does this  mean that transmit of the packets are completed and mbuf = refcnt is decremented by 1 on return, or maybe the Tx engine queue is populated and mbuf refcnt is not decremented = until it is actually transmitted, or much worse later in time. 

Is the DPDK Tx operation intended to be the last sta= ge of any pipeline that will free the mbuf if successfully transmitted?

 

 

Any help is appreciated to help me resolve my issue.=

 

Thanks,

Ed

 

 

--_000_CH3PR01MB847068FB4D2669E00351EAA38FA92CH3PR01MB8470prod_--