From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 19B544624E for ; Mon, 17 Feb 2025 19:57:08 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E14D140151; Mon, 17 Feb 2025 19:57:07 +0100 (CET) Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by mails.dpdk.org (Postfix) with ESMTP id F406A400EF for ; Mon, 17 Feb 2025 19:57:05 +0100 (CET) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 51HEMoJr017119; Mon, 17 Feb 2025 18:57:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h= content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to; s=corp-2023-11-20; bh=7eBmWJzTcOksvGiQw7 NfUp8LZsHxaA3364IQRKTnVoM=; b=nk/E/hfuyhk8/RnrWpKjyv8HTKClymvKrW pGoeWeZwygeD0w8H3B9lz0HZtlSo7aohWIChIfvmPvPSGWfovHXYtf9aGajR73f1 8O8C3gioq83U9nS7GgMRLFnllimBeb+LxOiTWekYMkNy/OWuh9RG37Xp3fiJ+uu0 RVsOatl+oSlCbdlwUUadOlhWXOWfabvM8/X9tgX4iEGdwEV3Uy/IDfv5jOjOr/wx eYBXrxOCMcr6ydy8+HZJJDz4y7RXsXNapYox6oSqg8L7HXG5yJavvRU2WreLxpUP 74sBiEhoUF3k9xDfy/rhluozbWPwQhQtpOUGMWn0ACT+qID3IPmg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 44thq2n00u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Feb 2025 18:57:04 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 51HHgQIL028401; Mon, 17 Feb 2025 18:57:04 GMT Received: from nam10-mw2-obe.outbound.protection.outlook.com (mail-mw2nam10lp2047.outbound.protection.outlook.com [104.47.55.47]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 44thc7ya35-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 17 Feb 2025 18:57:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aiNp09/VkpayH3n4mcZ08B2oRlDO5R0Bno4TVthjQ2e1qOoLZsHvkXNdw8nmfkT+2ZjfY9+EZ2Dnpkr2Mx9vHed0zWGrc4OUz5+P605pP+awpvai3v0A6r1KLYuQvwX/iUS16IzyVD7EQV7C3XhDbOsybqEvEKOJPaxxsl7FDIeh2KHv4w3wDJFw0G0juAQ/4VAS8IVsxSgAQWy67UCr7QDR/7U9Qm9NCLheLKk2YdW+sllVI2lRgGSFU8xp5UTvLPh1BJLdXQjMI3Zn4FuXD1BqELLtnkP3qlTd3uwto5BkNLLRCMBOmScF+WGjd4Tlo1A0eM0f/Y6Q3G5p5FrSbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7eBmWJzTcOksvGiQw7NfUp8LZsHxaA3364IQRKTnVoM=; b=c4T96cckicv/XPElewX05xadwAjHyiimK5/T+fF54M1YfyY2eGT/4cyRXPjIzzMGFEHkOUpORFo59Y6gKn490QuXFmsVar+kZMLZbKr1IQMEKf/8XBG71WafHeCw2uzXQ20CMWeCVkUUfQcUu77qjwTQWxw3fcsm3SUrsFuXu1YXuvbLZXPvThJTS7KmFEjAkoRlHBSC7OgqMXfXR9qRK6vO8GuLTV6RGYmePCeijLvprxm3eEJgV4KofQxIF3E7k5kFrjWl/JSejaIQ5F/buTfBegJY/7/Dro5aAePcbuHpkZAw61mSAGXeraWd/DCuAxsyvQgRrwSsIOEmKAGg/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7eBmWJzTcOksvGiQw7NfUp8LZsHxaA3364IQRKTnVoM=; b=aPXPZw6j6Td9BJKrOh1yLcKWMfXIlrjggdSEc2BMtfXnhvGcnAYM0j4QLviRzydsS3w9A/HtTKjKeCP43Te9KzxXq0jKPj8caoVn9/d1WW209qjk8zC3tlFCCXdlyvhd7KS8y23IG335t8TvrUpCiVgX4wcHyiMMejSLyHa497s= Received: from CO1PR10MB4756.namprd10.prod.outlook.com (2603:10b6:303:9b::17) by DS0PR10MB6080.namprd10.prod.outlook.com (2603:10b6:8:c8::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8445.13; Mon, 17 Feb 2025 18:57:00 +0000 Received: from CO1PR10MB4756.namprd10.prod.outlook.com ([fe80::8a56:dd51:c0e3:5958]) by CO1PR10MB4756.namprd10.prod.outlook.com ([fe80::8a56:dd51:c0e3:5958%3]) with mapi id 15.20.8445.013; Mon, 17 Feb 2025 18:57:00 +0000 From: Changchun Zhang To: "Van Haaren, Harry" , NAGENDRA BALAGANI , "users@dpdk.org" Subject: Re: [External] : Re: Query Regarding Race Condition Between Packet Reception and Device Stop in DPDK Thread-Topic: [External] : Re: Query Regarding Race Condition Between Packet Reception and Device Stop in DPDK Thread-Index: AQHbfsIt2fDpuCfIZU29/h2zQSBn1bNL2SC0 Date: Mon, 17 Feb 2025 18:57:00 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CO1PR10MB4756:EE_|DS0PR10MB6080:EE_ x-ms-office365-filtering-correlation-id: 32d4eb89-babb-4280-80c9-08dd4f84dc4a x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|376014|366016|1800799024|13003099007|38070700018|7053199007|8096899003; x-microsoft-antispam-message-info: =?iso-8859-1?Q?GBDZdm1QwIjCKvghnlUP4kHaFzOii6G7obzKNV7l23seJ+P4/RRF/q7OlG?= =?iso-8859-1?Q?5h65V5k9iv1rRgirf5YORu2hy7sBqoLURuTbX3o3Gny2c2OR5+MYNAOPPa?= =?iso-8859-1?Q?Xiv6z+q4nvWbBCZF1Cwn3Q8mhiUxL9jzpg5G3ecdbSpkp2En2L6SJpnWhm?= =?iso-8859-1?Q?uDEy7uIPOV5b5crXNIZymvfaTdbdeVXKNX+jso79L+td8V52PzSPU8V5bD?= =?iso-8859-1?Q?P8GNEf1rmSjmpB7melad5tIj4nSvQu4eFCToHgD+WDRy20C3D5pnB78/MB?= =?iso-8859-1?Q?fJdhM4+g1+9itv0F3ZCI6XGIEq8Y29x7m0H7iWGrMqEaVParStxSOSa1H/?= =?iso-8859-1?Q?Zufhs4hDQuH1ioIbdiagwelgl5L1sQQwoRifSdTruUEedEZg0ue+3MUFjj?= =?iso-8859-1?Q?9K/QDRPjQo5KTeFgZdbQ2JLGRmfoPXd0Q/EtkSCpsV4d62gaZ2udGSbXT7?= =?iso-8859-1?Q?RxWvv6g9A9nrxZuNnAvsg7Wx/tL7plRQ7wNzShKf4qCfO3SEFIzDiw83A7?= =?iso-8859-1?Q?47qZpUuT4q3LyZC8GDGMvwuB8+BJwnDSn2hK64M5xHoqHhDTcXlpXU3UcO?= =?iso-8859-1?Q?DxH6g8JWzUwQUt+8Veg5qUtIM0KDok7jPfvaPxud9TNeWH5ooC+CxIbxYE?= =?iso-8859-1?Q?IBnfo8WP2ITD6eK4LtqCfh0rlPtmeDxzbYYw7zWI3DoXUR/JWVzG5NyAO9?= =?iso-8859-1?Q?vg/VxGxDXwUZX2MFTD0fxPAuRQL56QJNJt/lzB2slK5LyOxZbgQi2B/+Rq?= =?iso-8859-1?Q?sIlvp+ZWReqVwIq+e9tkGCGxrBw1SQxOSaWGhl3CTMqPhXEn05pPursMWX?= =?iso-8859-1?Q?xVvZuSbs59XifcKhHY4ZCQt9SRGxigwAEVGjlCupw4d/jjTRxXtP2edkmY?= =?iso-8859-1?Q?QcPsy94RtlVkmzazBMwJivjpEhU8RJ8LPO+43OfWIW6PJY/xIxxfpBBuby?= =?iso-8859-1?Q?xyvqUSLKX9GKq+zgHUbLixMqzMwJ8DQs12bM1GKRZwcZ73YScIzdarQ3EX?= =?iso-8859-1?Q?H/wTCIJVaxLzpsYEvqL3nslUXVU3GzwuHJy5P84y11TML2/BbFxuYZKNYP?= =?iso-8859-1?Q?bFco4XS8ZN7/HWaxbVxYcxtNTO8jaWUkgRd5uSQEvN26jbgZRVY6IMzI5p?= =?iso-8859-1?Q?kP5saplfm0HUFJ9PNvJ/0BlZOiPzrYzI5z0RPsRhM6YeMBtykkaB5NacmV?= =?iso-8859-1?Q?FwT17a0EeRfTH/uBIaf/m7K3m8nEbbBly6dYbjFjQuL/QMpokWrzTfX6zN?= =?iso-8859-1?Q?F+oe+oWCn2WnOAkrRHIKsRX9sb5XvuHLX+kZgSjV/JK5ZXiCWwLbTbmDDV?= =?iso-8859-1?Q?J1l9ehZMYRA+tYmQUbIfzXX1rwlFvbQp9QtF7kSi8N/35ieABwUi7kbdRu?= =?iso-8859-1?Q?GevZZO8AwLVJLsUVFGtzSrRw0Jj3OKkxTiz+n0Nz+Y7wJEC/avLIOPag3f?= =?iso-8859-1?Q?E5yInHSW1gwJrZXPcl4g+9f+8DqZGJ/uB1pc/Q=3D=3D?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO1PR10MB4756.namprd10.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(376014)(366016)(1800799024)(13003099007)(38070700018)(7053199007)(8096899003); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?5bIWu2p7hYPz3NkBkDJnYm4O9OPt3kBt7ibJgx+jGZdm1HJr5AvXABli1w?= =?iso-8859-1?Q?QyDKxqAeii61n/fD2LaDVgQzVPFzkRgHS84ai/AJlxZ+PfrZ07UWz8C+Ib?= =?iso-8859-1?Q?Vhv/mYhlgrSNnN8YacIED/ZON6p/t7olXSqABkh/aFUIPYhRl+xSdxm1il?= =?iso-8859-1?Q?Co4v9of36nupQ7ORr+ELs24n5nPWsyqNmIFWH9nfNMQu2urEsOJK7tLXgW?= =?iso-8859-1?Q?deDTfbZ8ZYyz6SVqXWMotAhkW3fl0ZzmnL4NoUuPGngTPJUDrXFtRNqHue?= =?iso-8859-1?Q?QdADUbL7B0iheJafoeLjDcBwdwsIDRc9BC5flbWOYDVe2e0tJ8og8bvAev?= =?iso-8859-1?Q?Y59YXzjF3LB05i26d9wLJf6RaEeBx4eoohtBoNygWz1ykRmC62pHoCmoWn?= =?iso-8859-1?Q?7HvRy3qCo02NXL48S9fEZTaEwIW/JJfvZKzNmmt34IWVFu5hRxsJwBfpI0?= =?iso-8859-1?Q?s7o84tiFTdh65yEDHUliQpdjIVUUweTsTH+6Ca9VyYj9zsxJwzVk4afz7M?= =?iso-8859-1?Q?J/KCaitDERRTZ/HdU1B7d4fjFe9oLAzvcWbAUqLQ/Z5IG7RBTFULoLqLzD?= =?iso-8859-1?Q?3ZOc43yevEuMFnwA90YFT2gZBz5XettmEMcs8fCkuTH5vOpRf1kKSFq+32?= =?iso-8859-1?Q?8DcTFK/lLVYkrld0Hykh/mPNwRzMHvN3OzHWOUeXlRyz/MO4ZxItg3ln+B?= =?iso-8859-1?Q?1AlQH/10xZZbRVWBztn8pYFVkJYXKv496H3wVvOEOVM+YMwcJo7uPnizFz?= =?iso-8859-1?Q?nwFvWBu7Gw1ytYVwhOPosc/MKiHkME3H3GEKmTgqDfy8DtbPSp3NnfHwan?= =?iso-8859-1?Q?EYOyRwptzosrbuHAsVvRn3q3+In4iF0uB+Q3YFVnKetSwDhKB5IMCfNUSi?= =?iso-8859-1?Q?gtFbXONNe73XyJcnmsotkrDFigEhLnd9wg3Wa6FTYTk6+qD78PgLhcWTKY?= =?iso-8859-1?Q?6PCUJq5sRqXMZr5UFDyVjFfmZShjH+dsq+9ZXhI1zX4ZiAeXBhbQ4z10TQ?= =?iso-8859-1?Q?h1WdysJgPUaS3XGCLhUI+Z/WUxrJ6tLQ6ktxqcHWA3GeZ8PKLXrDP9ppdk?= =?iso-8859-1?Q?4u4jnnfnH8Myc9Ah0eog/I7eJ5h3/q8amqPPBvLkyAtTrtlD3vtgSw9/p+?= =?iso-8859-1?Q?N4t5A/OL9Y9fvJof1r9e4Vx1/JGOmmoYj7Uq+20tG7FMUCYvl8jPRMtojN?= =?iso-8859-1?Q?8SgK+v+x0ZMY3mrv556cSiSqJ2RfDAIgT0o3qtX0TlMb7XpG4ae322SU9F?= =?iso-8859-1?Q?gHauaXJ/eOQCWJsPTkTEnNVTLGBnvL3iuTMx0nyDmFvfMQ0YVGEH6dxxnZ?= =?iso-8859-1?Q?2QdLqsRMEUIEknJJE8VPTHVeQ/c+qny+L5Nb85beCizH6hsbYyQndVDPdw?= =?iso-8859-1?Q?b1Lv5U9d8gqp9ckQ+UHEJ0tIXVEjdyvAz2l1nz8ic7/MyP60m9leiS1A8O?= =?iso-8859-1?Q?qeOVAxF4Bnp8s0WAi46yuM+K77znUqd+ZTqoO8h+KTPW4trX+sREwXIU2f?= =?iso-8859-1?Q?aBTPTkySr4WZLAOGOTSpsVOHZw/+7hW0qwcUWuRVxyeS7FRJCqjO2B0ae1?= =?iso-8859-1?Q?GvH91sLHlCQfbU0NR5BNdyBBm1zdQ8QcZrpNBfbQ6V8+4ZYfLMQAXPz1/R?= =?iso-8859-1?Q?oO+RLVV70JFqAMqLGfsGrl1eLbxRcHy3/U?= Content-Type: multipart/alternative; boundary="_000_CO1PR10MB475637CF7A1B9548E765958584FB2CO1PR10MB4756namp_" MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: GCfAfCvc6wwMsYZbcVWHGXeJJKgN1F7+JWagAWYhJCVLf/sSV59oUAOBbgPbHhnC0o7cCnqWg9b1qNRm9pjSPFGmvG+4XPGSCztXUQJ9FMitb2tAAFmtJhvgEdnRQC7Wq8Tfg9Bnd2iu89hFICLsTtYD/7sdemKEyHErKq+9ddKbfremyZnomw62B3jEr7BOZKAq/SlWYZxCo+O2pkpOj51IWdl7whBDF2Wz18esXynnnhUby0hK1yqeltlUzuMVI4yhxxbWWc1yuIJobgGeiqjYBpMHMurDKgu2WtoorgGKxQ2ZGUFwk8+Gt6x14W5724d+7F+ElSaRRmYLoBVZTkKrGe03p51DFvzmNijvM56BGdSo8QmCsa8QuK7vEqfBkXrx8aQsFghQlNyo1gHNmppfyNRoUBIEpNoPrkBd5A4qj6ys7sy5NkmNU+6gZ5l/xOn/UxBhcNsx5O9ozHe0LzqEW0N0Ow4P7hoAn+ybX9F7ahWAco50Qx6lxlTbDHzUHxN71h/7V/K0BdTRpVb9igTxfmXOwmfZPMdkw4/yDw5AyAB4TLAUMyqfxsvAPQhPHETcX4Svwf5eXBvGVl9j2JsKTl3a9uuTATxp0AEbBeY= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4756.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 32d4eb89-babb-4280-80c9-08dd4f84dc4a X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Feb 2025 18:57:00.6457 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: z5rnGLF6b6tSvcPVL2ydDqTPeOwgDkPwPABlrAPLzVPQ0axvsFrWorfGsT0XukVPjWJi8aftTnTYyGfo+EEQKXwy1e5dS/4EhtvidFSEHZU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR10MB6080 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-02-17_08,2025-02-13_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999 suspectscore=0 phishscore=0 mlxscore=0 bulkscore=0 malwarescore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2501170000 definitions=main-2502170148 X-Proofpoint-ORIG-GUID: uWXliBLP358RdTl92Q3w8pzMw_t789Dw X-Proofpoint-GUID: uWXliBLP358RdTl92Q3w8pzMw_t789Dw X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --_000_CO1PR10MB475637CF7A1B9548E765958584FB2CO1PR10MB4756namp_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi Harry, Can we call rte_eth_dev_rx_queue_stop() on a rx queue when a fast path is s= till polling the queue? The sequence on control and fast path cores would l= ike: Control path: rte_eth_dev_rx_queue_stop(rx_queue_id); ...waiting for draining of rx_queue... rte_eth_dev_stop() .... Fast path: Keep calling rte_eth_rx_burst() (I am expecting it will return 0 if queue is already drained and stopped) Thanks, Changchun ________________________________ From: Van Haaren, Harry Sent: Friday, February 14, 2025 4:22 AM To: NAGENDRA BALAGANI ; users@dpdk.org Subject: [External] : Re: Query Regarding Race Condition Between Packet Rec= eption and Device Stop in DPDK > From: NAGENDRA BALAGANI > Sent: Friday, February 14, 2025 8:43 AM > To: users@dpdk.org > Subject: Query Regarding Race Condition Between Packet Reception and Devi= ce Stop in DPDK > > Hi Team, Ni Nagendra, > We are facing a race condition in our DPDK application where one thread i= s reading packets from queue using rte_eth_rx_burst() , while another threa= d is attempting to stop the device using rte_eth_dev_stop(). This is causin= g instability, as the reading thread may still be accessing queues while th= e device is being stopped. This is as expected - it is not valid to stop a device while other cores ar= e using it. > Could you please suggest the best way to mitigate this race condition wit= hout impacting fast path performance? We want to ensure safe synchronizatio= n while maintaining high throughput. There are many implementations possible, but the end result of them all is = "ensure that the dataplane core is NOT polling a device that is stopping". 1) One implementation is using a "force_quit" boolean value (see dpdk/examp= les/l2fwd/main.c for example). This approach changes the lcore's "while (1)= " polling loop, and turns it into a "while (!force_quit)". (Note some nuanc= e around "volatile" keyword for the boolean to ensure reloading on each ite= ration, but that's off topic). 2) Another more flexible/powerful implementation could be some form of mess= age passing. For example imagine the dataplane thread and control plane (st= opping ethdev) thread are capable of communicating by sending an "event" to= eachother. When a "stop polling" event is recieved by the dataplane thread= , it disables polling just that eth device/queue, and responds with a "stop= ped polling" reply. On recieving the "stopped polling" event, the thread th= at wants to stop the eth device can now safely do so. Both of these implementations will have no datapath performance impact: 1) a single boolean value check (shared state cache-line, likely in the cor= e's cache) per iteration of polling of the app is super lightweight 2) an "event ringbuffer" check (when empty, also shared-state, likely in ca= che) per iteration is also super light. General notes on the above: There's even an option to only check the boolean/event-ringbuffer once ever= y N iterations: this will cause even less overhead, but will increase the l= atency of event action/reply on the datapath thread. As almost always, it d= epends on what's important for your use-case! The main difference between implementation 1) and 2) above can be captured = by this phrase: "Do not communicate by sharing memory; instead, share memor= y by communicating.", which I read at the Rust docs here: https://doc.rust-= lang.org/book/ch16-02-message-passing.html. 1) literally shares memory (both threads access th= e force_quit value directly). 2) focusses on communicating: which enables a= voiding the race condition in a more powerful/elegant way (and future proof= too - it allows adding new event types cleanly, which the force_quit bool = value does not.) I like this design-mentality, as it is a good/high-perform= ance/scalable way of interacting between threads, and scales to future need= s too: so I recommend approach 2. > Looking forward to your insights. > > Regards, > Nagendra Regards, -Harry --_000_CO1PR10MB475637CF7A1B9548E765958584FB2CO1PR10MB4756namp_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hi Harry, 

Can we call rte_eth_dev_rx_queue_stop() on a rx queue when a fast path is s= till polling the queue? The sequence on control and fast path cores would l= ike:
Control path:
rte_eth_dev_rx_queue_stop(rx_queue_id);
...waiting for draining of rx_queue...
rte_eth_dev_stop()
....

Fast path:
Keep calling rte_eth_rx_burst()
(I am expecting it will return 0 if queue is already drained and stopped)

 

Thanks,

Changchun

 


From: Van Ha= aren, Harry <harry.van.haaren@intel.com>
Sent: Friday, February 14, 2025 4:22 AM
To: NAGENDRA BALAGANI <nagendra.balagani@oracle.com>; use= rs@dpdk.org <users@dpdk.org>
Subject: [External] : Re: Query Regarding Race Condition Betwee= n Packet Reception and Device Stop in DPDK
 
> From: NAGENDRA BALAGANI <nagendra.balagani@oracle.com>
> Sent: Friday, February 14, 2025 8:43 AM
> To: users@dpdk.org <users@dpdk.org>
> Subject: Query Regarding Race Condition Between Packet Reception and D= evice Stop in DPDK
>  
> Hi Team,

Ni Nagendra,

> We are facing a race condition in our DPDK application where one threa= d is reading packets from queue using rte_eth_rx_burst() , while another th= read is attempting to stop the device using rte_eth_dev_stop(). This is cau= sing instability, as the reading thread may still be accessing queues while the device is being stopped.

This is as expected - it is not valid to stop a device while other cores ar= e using it.

> Could you please suggest the best way to mitigate this race condition = without impacting fast path performance? We want to ensure safe synchroniza= tion while maintaining high throughput.

There are many implementations possible, but the end result of them all is = "ensure that the dataplane core is NOT polling a device that is stoppi= ng".

1) One implementation is using a "force_quit" boolean value (see = dpdk/examples/l2fwd/main.c for example). This approach changes the lcore's = "while (1)" polling loop, and turns it into a "while (!force= _quit)". (Note some nuance around "volatile" keyword for the boolean to ensure reloading on each iteration, but that's off topic).<= /div>

2) Another more flexible/powerful implementation could be some form of mess= age passing. For example imagine the dataplane thread and control plane (st= opping ethdev) thread are capable of communicating by sending an "even= t" to eachother. When a "stop polling" event is recieved by the dataplane thread, it disables polling just that e= th device/queue, and responds with a "stopped polling" reply. On = recieving the "stopped polling" event, the thread that wants to s= top the eth device can now safely do so.

Both of these implementations will have no datapath performance impact:
1) a single boolean value check (shared state cache-line, likely in the cor= e's cache) per iteration of polling of the app is super lightweight
2) an "event ringbuffer" check (when empty, also shared-state, li= kely in cache) per iteration is also super light.

General notes on the above:
There's even an option to only check the boolean/event-ringbuffer once ever= y N iterations: this will cause even less overhead, but will increase the l= atency of event action/reply on the datapath thread. As almost always, it d= epends on what's important for your use-case!

The main difference between implementation 1) and 2) above can be captured = by this phrase: "Do not communicate by sharing memory; instead, share = memory by communicating.", which I read at the Rust docs here: https://doc.rust-lang.org/book/ch16-02-message-passing.html. 1) literal= ly shares memory (both threads access the force_quit value directly). 2) fo= cusses on communicating: which enables avoiding the race condition in a mor= e powerful/elegant way (and future proof too - it allows adding new event types cleanly, which the force_quit= bool value does not.) I like this design-mentality, as it is a good/high-p= erformance/scalable way of interacting between threads, and scales to futur= e needs too: so I recommend approach 2.

> Looking forward to your insights.
>
> Regards,
> Nagendra  

Regards, -Harry
--_000_CO1PR10MB475637CF7A1B9548E765958584FB2CO1PR10MB4756namp_--