From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id BDCFE46B22 for ; Tue, 8 Jul 2025 06:10:15 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 57D6D402A0; Tue, 8 Jul 2025 06:10:15 +0200 (CEST) Received: from mx0a-00196b01.pphosted.com (mx0a-00196b01.pphosted.com [67.231.149.170]) by mails.dpdk.org (Postfix) with ESMTP id E47E840287 for ; Tue, 8 Jul 2025 06:10:12 +0200 (CEST) Received: from pps.filterd (m0096263.ppops.net [127.0.0.1]) by mx0a-00196b01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 567Jbnah006901; Tue, 8 Jul 2025 00:10:09 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netscout.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= netscout.com.09.24.2020; bh=ryDB6HEkvL0KNIfyQHJL7nvNogfkWUk5W0MV mp+fzik=; b=tufpLJNoB+ma2WI6uP2Bn5m7ARJiXRpV//vR3nnapVrINgO5hhZZ EdWN17zfRumLEapM1jlfeevPo3yWMiUo7nn2SeSQsZLSjFBe/N2iGFZjluTkDPkw KE0jcRosNwHZdnotc18T7A7rvP6p89S2hwEnu9WYYt6f9KrfTHatJ2WxQ6H16//c vvbnlGMgWQQqibLphh+7xODjxsuTGJhhZ9/7WfiVEF8v9DKAeSQyEGmxbZEkxxp7 M1VEfPhB3eLsrVyJhngu32tKTiCCS0CQlqEGqagqnu3QLOEGjr5hoAYz8owo720N WMMcVEFSIx26nQHefLYytiRrmTtlvXKcow== Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2176.outbound.protection.outlook.com [104.47.56.176]) by mx0a-00196b01.pphosted.com (PPS) with ESMTPS id 47rf5m8txt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 08 Jul 2025 00:10:08 -0400 (EDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=svHRCag7i0m0sMt5fzU1lpvuT28jALQyS/FoxqmuNJX6E2+7zFPKygc4k88xKkyPpHmgh/y+BRlK55DtmsA/7VvO+YcjYjKpCfDAWbc24Y4I6TuhN3k5ISHpXVqAoKnlEtNrD2/4w+OW0f7PfMpG2yiYnDyrHxhq452zCUU+SPuE9wBX2WB+E/7s0y6IS4x6WFwtYjRShJTIDaBco/dcoPiaX53nytu0OhokfJnCaB7T/t4+5ZPyGSUe30aeMZX8bJZHC5B+dUlBPqKnd/Sz5MjRkixRiqfXyqkz53hIm1D0mvovRRTQCIKvHrUry/5GMah+WhBXtOt+hWX0FmbcPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ryDB6HEkvL0KNIfyQHJL7nvNogfkWUk5W0MVmp+fzik=; b=G/+P95B5+cJPJBOdoaOUNP2840dl7mc5rhefKWyRQxlJUknN8xIVo1Lee/r6t/uRzZdmJf+AeoijcCH+H11qDLxpZTKimOEsiUgmPIg5MYtan/diRweoNs+8896pqfm3+uYaotkb004pRDARzStiT49ZkPjl2ThWcAnJE52nN94PfCj3B/EI0L1qrR6+Rnsk0CJV7x1DdaLZ+7NjTlPjHMSK7aQYtKntHE77ThvuH59JQxRIGbG1Kkm+MQKxEbEoLHNAAhP8+6+XmdWUkVtZjXDPxWEGvhgMhKKjwnJMmFGkUKIDDhk/bLkLR4P2F/gP0YHfwQuUl0/OtLYTZIUmoQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=netscout.com; dmarc=pass action=none header.from=netscout.com; dkim=pass header.d=netscout.com; arc=none Received: from CH3PR01MB8470.prod.exchangelabs.com (2603:10b6:610:1a4::21) by MN0PR01MB7850.prod.exchangelabs.com (2603:10b6:208:37e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8901.24; Tue, 8 Jul 2025 04:10:06 +0000 Received: from CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd]) by CH3PR01MB8470.prod.exchangelabs.com ([fe80::80c4:7216:f070:e5fd%3]) with mapi id 15.20.8901.021; Tue, 8 Jul 2025 04:10:05 +0000 From: "Lombardo, Ed" To: Stephen Hemminger , Ivan Malov CC: users Subject: RE: dpdk Tx falling short Thread-Topic: dpdk Tx falling short Thread-Index: AdvsS4XiFhfr9tUWTZG/9hrfPmeo0QAjYaoAAAZ3EoAAOAaP4AADTEIAAAkqa1AAIqDrAAADVbOwAC9nmnAAC62DAAACmVaAAApYvfA= Date: Tue, 8 Jul 2025 04:10:05 +0000 Message-ID: References: <20250704074957.5848175a@hermes.local> <20250705120834.78849e56@hermes.local> <20250706090232.635bd36e@hermes.local> <9ae56e38-0d29-4c7c-0bc2-f92912146da2@arknetworks.am> <20250707160409.75fbc2f1@hermes.local> In-Reply-To: <20250707160409.75fbc2f1@hermes.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: CH3PR01MB8470:EE_|MN0PR01MB7850:EE_ x-ms-office365-filtering-correlation-id: b3e2a643-b394-42d0-b998-08ddbdd551ec x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; ARA:13230040|366016|1800799024|376014|38070700018; x-microsoft-antispam-message-info: =?us-ascii?Q?KG5NIEG3icVNTLA3Tmm9Eni34xVCeAm5F2hE/YYOVCgVz6T9wdisOeCF1JNK?= =?us-ascii?Q?bSJWhY2SYwoQTvMPThKeRjAfp3CO1X8D0iOZuoXRDBxhQIf+3BiKQRVc766Z?= =?us-ascii?Q?uV+8fuljBHRlE1LUe1Yx4HFB/j2258ykrBcHMIK8agia8uk7xOq0VbY1xyFl?= =?us-ascii?Q?Yxdz5LcjO6Phkp3jhyFEtl4G6zsXrT40GNw3zoAFsXgyZaeq5rGSqmmYXr6X?= =?us-ascii?Q?FCvfEeS4AecKsfGijSlyAaU/BmlcMPVhgxTtpV5+XKy/fyaL6McCc3qxaKNL?= =?us-ascii?Q?e/9Gv2TOB66C/41ZRGplOoRujDE9ayeTWXv/3alfAHGwFnu+XhKoNnTRP/Fx?= =?us-ascii?Q?0QyBr63LKUkUBmBjld6/i95OLCgcmqkomjlr52LuxeBmpVzhVqwsIDIcAnhO?= =?us-ascii?Q?/Y9DaUKS1b02lTFfcS+9INely+O18dRaTshof3TL6//5ctGRjQybCdYdRSgN?= =?us-ascii?Q?2APjCln9kuYMPWlpHSH9t/Bfh6pgQFHPzfgtYNIESxOmjuql1WXB7Mj3IuKK?= =?us-ascii?Q?xnWqUCFD2JJGP8ArF8082LZSKqCTfKtxs5Yq7DQe8EFLK+dSlqyo8mTR82MR?= =?us-ascii?Q?yiCXTvgD9k5JLkvZKptCSn1QSLMUtwMo4wfuMyg5Yqbv5fVw33UI/ZdUCACs?= =?us-ascii?Q?ZEvxmzwWgOeqsKL5IRKrA0ff/ALpaO0P9LV5Tmr0Vh1VHdHU0WM++XEyHO2c?= =?us-ascii?Q?HbKsLEWqX3fWGl/rVLtNhwseZBB89Lyk0BT3/MblULDBnYw9+Lu7cH/bPJvu?= =?us-ascii?Q?iv8bJfAiWOPbxMhXMbhg6CzTzL2ZKQgmvxxeY2G4QSTjnBoTMFwHEtIyeuTE?= =?us-ascii?Q?ZpJ9PJJgpfqf/k3CtDynY8kCc6ZJL5ht7tlXWmyyxpUOVA4G/NEHFTnXmn3v?= =?us-ascii?Q?VN4dUFaBdsjRtbTI8+97p/JczPl+ccDGc1RZbc1Dikf5BLu5oI1Q+y+fHzds?= =?us-ascii?Q?LLxa4NiVfsvTgIBUVUK4fK18/9r+AjPglTA6HVRt4oFK7FDveFPH+2WfdxR8?= =?us-ascii?Q?s43U/fkd11qqdirftdp4mGZk7FOXXbD2KHzg2lYnW2ywb6lYI6BBx7vPgL+O?= =?us-ascii?Q?oRjItEBHZp4cpfXHCYlrVbiULKMnN9eivx0hEEllqPvk6QdPTySt6r3GXgcP?= =?us-ascii?Q?uh5xNdhjsKg8acatJJnTQaRRyNi8HrwLwt06lnRAx5AcmSl2DrKrPrmUBtN/?= =?us-ascii?Q?JV+N0c0n9znzwQowLGe2Q/POJVH4HCW068llsk5vGpsWDKWwx0RkmFbTf5qv?= =?us-ascii?Q?74mqpZ/GTfbLy3C3YWKWZto4E90o3NbgvWb+Y66QlkYTSDsQpz3ev+lxKD1i?= =?us-ascii?Q?faeOWYApXT1MWvVRt1NMUHjOIBLP63DbFvWAaZ8bdEQDSYzMtC+sj+fNcfWm?= =?us-ascii?Q?wtnNuVREMud3zaoekSTTuoeMboL1MA/2DAkORpC587UICOQOeiPytcQ+An/F?= =?us-ascii?Q?hvtDyn94R8DhvGlfWeu9tXmGintQcUb5?= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR01MB8470.prod.exchangelabs.com; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(376014)(38070700018); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?HQ7CMa1bpcyg6gar5JbC9/ADL175mH5rl9YEkV+799tyacXVgVeB1z00I41q?= =?us-ascii?Q?kjqrxV0Nc8F+YSWtADslAPW1J7hytuLhuZtRaD1uYYGkqDpjwBLX+fURdyk8?= =?us-ascii?Q?N6eCr0V0xW+4Y+t54rvK5jBE8M8UCx+2XFZUKP202tyyLGyUxkQ8zaDE+5dW?= =?us-ascii?Q?KR1Ma+3LY6zDsaJuo9bZEIJitbECjEPlRqkISJdwP6MIThZpXRGSnrQ1Kj1V?= =?us-ascii?Q?uh1pnmOlz7lIbhY3xNDB98k4Hq9LUngi31fUwVz0f1wq1VV0249dYqV+BIoh?= =?us-ascii?Q?+ikFAbEnHNwi8uCvrwU8Q/xg/InUrNDo1dHkxN4Be6zA9915Td73OzdO4/kY?= =?us-ascii?Q?krzxo+YjIdauAjHZTLVqfZ25b7kf/Td/+1tjeV+Jk9LmXaybhWmL2M2OJf4c?= =?us-ascii?Q?0y5y/HumqNTgFnU/dDK0ShSWP9UUybbUGBFtKgukhL+CjmpTmZfA2Oqx/8c/?= =?us-ascii?Q?2yvaGCdOU8umFssnssIEZPwVShlbvpNoRNVsYaIvaetzbUOti4tEoviwIYfj?= =?us-ascii?Q?cGr0Jmhh0BA4/Q1Ai3F004+mAhb6DddQ/Po5zkaSGL0xFllhZ261Oz1hNwYj?= =?us-ascii?Q?HGWu+2mFnduhSbP/6EkFt0VKkZPmYdnPUAaNcST0a+vtBUmnKF+SWpmy/iQB?= =?us-ascii?Q?Q5VEQQkvoAWxwLG30mB+4f70YVc0q/kNwhCe6Lcnt8xpr1GuA/r4ucGw38Fg?= =?us-ascii?Q?JKIUzm+Q4M9ZnqWc/lMoYu9jJ2jQzz+oE6h5DRHwG80cx8QqDwNHzqvhEXU3?= =?us-ascii?Q?nBzUkZ98Fz31RIy6Uvbh/cN2C7/qTBEaMkFMSbE2alhSZ4LS8NNqbQAOzlGC?= =?us-ascii?Q?pjZGd70dTeP2ABD1f1nSCW13CT7k6/19H10jvgwFHlaNGcK8HO/KXpCOgY9K?= =?us-ascii?Q?yixAdslAVVXqqBBDjJM5bITiXcHCOTyls7ls9JzSBpJ1ZhgOzvb6G/XOjIc2?= =?us-ascii?Q?ULpf1umXjVIySeCVLUUkv24ULihIOrH9vhukH+XHoDYED24giTSVcAL0Hr/W?= =?us-ascii?Q?5HZarNKUl4sv9E4f/tq+n2ToCAnbWZY+520H2tA+4rEhRtFMyn/4tR/xpI+O?= =?us-ascii?Q?QVAIYyc70xrr5nW0z3DHqZD447R7CZoSuZCnSn9WyaW/tK+gLJOU4NP99+8G?= =?us-ascii?Q?cYBuzHb5yd17HzqJE/UYd9WGAMtoW37A52yo8n+WpqIZOjUXyjTIsTRuz6oJ?= =?us-ascii?Q?OYXzeVhDtvvWhad2qSI1q3oCcJF15Wd/jBjP9kYbQjvq60ak/KO27TRBmNri?= =?us-ascii?Q?4Lqdy/zxQ/skElG3rCPQaYCmGBkorvPzTy0ba4RkPolztcobNw3dHXohhXTI?= =?us-ascii?Q?HraCu4ollcSEwa1MsIDXN6oGlQKlzV0vP19WqnxjDI9au604bSyT88oFhy0Q?= =?us-ascii?Q?Ry9Eju5SQUNXywKfHpk6gvPswziDZQhpjBvZ1IHa3NRiFEgu7GJ5/Hm/g2V2?= =?us-ascii?Q?sKBHF/J3ATphTlYG2Fa+PX4m/rDKe0krrl9v1+nVgnkz0ccAoQcRLrZwM2PI?= =?us-ascii?Q?gTnVxvJbeUH/L99+E1cRL7Z2QYojDL5BtWM/y98Az/4YjOTsmPYSBoZqspzW?= =?us-ascii?Q?eEa8XDvrM7I4yZyQtrYn3XeRbOda/a7j1NulutNC?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: YNgV0+f/A0Kk0qDikxRLbB7upQTRXqoYVt5ey8gmF020i61aefaGkjBnXq0fTPt2tGTmNXKqZy/ReQNq/ezSprV+sjCmTpK8v9o7W1ushJPfeA36RCGzHWNBHKuYwn2wYW44u0/kujqNr6sN8mg1UOYzlOGLZP3lCJZW9qu+zlxVVZHekXqkdA1yt092LPNqh1aCaGkT+BrlxwsYcc5/EqYYieiwTiFC15oYWUdIYz4nZkpb0LGsEf0NCuGSTHGsjEMxdqubmp1qICQ24ltOqKAtfizUvzbXlRHSL7OKrczU8/Zx8KZIb93JGpAlSj6j9M0R9E13vGs+ZJCab1NDkET4tywM0JI5YfzCB8xk3y+fImqfU/iIkdb9JM8h7047xLJy5tC/mSUhywUzsFFRm0kW2B13dnFIqQNAZPQAXBfhB0pU9bpzZR/Rdh4u+UgMZtjOTFeunRVt9Kez2Mbd0OyHisfD/P+x0tw4qrICarZMEynWFv1oGZTklsfMgdkNllIV9YtWhFOli3M9LowoIH6xEaKdoJwddBC8kzPal7YqLmSJAHP873Uh+1XQWk8QhQcYtgyS1jHSLYlcaa24r8u0n70pzOSv4LBCfxdblFSkSPEdN7NX0tniGIjYfBrN X-OriginatorOrg: netscout.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: CH3PR01MB8470.prod.exchangelabs.com X-MS-Exchange-CrossTenant-Network-Message-Id: b3e2a643-b394-42d0-b998-08ddbdd551ec X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Jul 2025 04:10:05.6277 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 54f11205-d4aa-4809-bd36-0b542199c5b2 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: eh387nRL3AWduCqIGyEoPhY5wEP1UxZ7RREDHHXVOTCT6RG5RUYL+WXj2/PUcnfLpqMhunUSPx1uX7HYGht+lw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR01MB7850 X-Authority-Analysis: v=2.4 cv=NInV+16g c=1 sm=1 tr=0 ts=686c9a20 cx=c_pps a=/1KN1z/xraQh0Fnb7pnMZA==:117 a=lCpzRmAYbLLaTzLvsPZ7Mbvzbb8=:19 a=wKuvFiaSGQ0qltdbU6+NXLB8nM8=:19 a=Ol13hO9ccFRV9qXi2t6ftBPywas=:19 a=xqWC_Br6kY4A:10 a=kj9zAlcOel0A:10 a=Wb1JkmetP80A:10 a=uherdBYGAAAA:8 a=8rWy6zfcAAAA:8 a=jZVsG21pAAAA:8 a=jQOgFn-ZAAAA:8 a=lGSfbmVVSQVwxxZrwbMA:9 a=CjuIK1q_8ugA:10 a=YjdVzJdQTyZRADMV7wFX:22 a=3Sh2lD0sZASs_lUdrUhf:22 a=mT82qxFQzDvLIExZS32s:22 cc=ntf X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwNzA4MDAzMCBTYWx0ZWRfX6xQLXtjHOaz8 yhqDkpOk2BOvZ9AG5Cz74J/rcahN47PSsyj9NXF8NvyWykhY8N0nU68gzgZmnZnzSV6uBRQfmjB ilE6MgvzxGZoaOjJuH6zV741ztLw20+2oKB05w7n2aD/KVDFCxriFI6HHNbcEg/77L1j2GCeRxL PcuVToCZBUOIbtUuELiVqtEJKFO2xahBTFJTKAZhBvZ19uDgHdrMnF+1jefs8PrEGD/EE9KjLxh eM3qZcet8h1osAC1s7NmjY+11wAD646+aMEGGmv0dMzmtfstTDUUL95qXijkDA76MotxJ3KWk2R Reo5D3coNjW9IhojF6bjgc6vio8tNpVvngeE2Qx5cfEDtTnqIvZOoCJbHghsDZnXSdUg0j/rEFJ NXj9sv0NJ9wSJK0ku7BshQjwTpoxcn2zRkdpS3+MSs4I83OJMESc157XXoZKDqA3v2MxHYgZ X-Proofpoint-GUID: A4C6Tdl99Pw5gNNKmATu8VjGF-euc3G4 X-Proofpoint-ORIG-GUID: A4C6Tdl99Pw5gNNKmATu8VjGF-euc3G4 X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 lowpriorityscore=0 mlxscore=0 mlxlogscore=999 spamscore=0 adultscore=0 suspectscore=0 impostorscore=0 clxscore=1015 bulkscore=0 malwarescore=0 priorityscore=1501 phishscore=0 classifier=spam authscore=0 authtc=n/a authcc=notification route=outbound adjust=0 reason=mlx scancount=1 engine=8.21.0-2505280000 definitions=main-2507080030 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi Stephen, I ensured that in every pipeline stage that enqueue or dequeues mbufs it us= es the burst version, perf showed the repercussions of doing one mbuf deque= ue and enqueue. For the receive stage rte_eth_rx_burst() is used and Tx stage we use rte_et= h_tx_burst(). The burst size used in tx_thread for dequeue burst is 512 Mb= ufs. Thanks, Ed -----Original Message----- From: Stephen Hemminger =20 Sent: Monday, July 7, 2025 7:04 PM To: Ivan Malov Cc: Lombardo, Ed ; users Subject: Re: dpdk Tx falling short External Email: This message originated outside of NETSCOUT. Do not click l= inks or open attachments unless you recognize the sender and know the conte= nt is safe. On Tue, 8 Jul 2025 01:49:44 +0400 (+04) Ivan Malov wrote: > Hi Ed, >=20 > On Mon, 7 Jul 2025, Lombardo, Ed wrote: >=20 > > Hi Stephen, > > I ran a perf diff on two perf records and reveals the real problem with= the tx thread in transmitting packets. > > > > The comparison is traffic received on ifn3 and transmit ifn4 to traffic= received on ifn3, ifn5 and transmit on ifn4, ifn6. > > When transmit packets on one port the performance is better, however wh= en transmit on two ports the performance across the two drops dramatically. > > > > There is increase of 55.29% of the CPU spent in common_ring_mp_enqueue = and 54.18% less time in i40e_xmit_pkts (was E810 tried x710). > > The common_ring_mp_enqueue is multi-producer, is the enqueue of mbuf p= ointers passed in to rte_eth_tx_burst() have to be multi-producer? =20 >=20 > I may be wrong, but rte_eth_tx_burst(), as part of what is known as "reap= " > process, should check for "done" Tx descriptors resulting from=20 > previous invocations and free (enqueue) the associated mbufs into respect= ive mempools. > In your case, you say you only have a single mempool shared between=20 > the port pairs, which, as I understand, are served by concurrent=20 > threads, so it might be logical to use a multi-producer mempool in this c= ase. Or am I missing something? >=20 > The pktmbuf API for mempool allocation is a wrapper around generic API=20 > and it might request multi-producer multi-consumer by default (see [1], '= flags'). > According to your original mempool monitor printout, the per-lcore=20 > cache size is 512. On the premise that separate lcores serve the two=20 > port pairs, and taking into account the burst size, it should be OK,=20 > yet you may want to play with the per-lcore cache size argument when crea= ting the pool. Does it change anything? >=20 > Regarding separate mempools, -- I saw Stephen's response about those=20 > making CPU cache behaviour worse and not better. Makes sense and I=20 > won't argue. And yet, why not just try an make sure this indeed holds=20 > in this particular case? Also, since you're seeking single-producer=20 > behaviour, having separate per-port-pair mempools might allow to=20 > create such (again, see 'flags' at [1]), provided that API [1] is used fo= r mempool creation. Please correct me in case I'm mistaken. >=20 > Also, PMDs can support "fast free" Tx offload. Please see [2] to check=20 > whether the application asks for this offload flag or not. It may be wort= h enabling. >=20 > [1]=20 > https://urldefense.com/v3/__https://doc.dpdk.org/api-25.03/rte__mempoo > l_8h.html*a0b64d611bc140a4d2a0c94911580efd5__;Iw!!Nzg7nt7_!EvEznHI_mP3 > GsiSVrhbQfDE2va8UxZ5-8okSD-Cq_gTm9nP0Q34d6XPWYoQhUGqoJifjjk4Na1a8j5EZH > SqWzqXGztg$ [2]=20 > https://urldefense.com/v3/__https://doc.dpdk.org/api-25.03/rte__ethdev > _8h.html*a43f198c6b59d965130d56fd8f40ceac1__;Iw!!Nzg7nt7_!EvEznHI_mP3G > siSVrhbQfDE2va8UxZ5-8okSD-Cq_gTm9nP0Q34d6XPWYoQhUGqoJifjjk4Na1a8j5EZHS > qWypWjs8A$ >=20 > Thank you. >=20 > > > > Is there a way to change dpdk to use single-producer? > > > > # Event 'cycles' > > # > > # Baseline Delta Abs Shared Object Symbol > > # ........ ......... ................. .............................= ......... > > # > > 36.37% +55.29% test [.] common_ring_mp_en= queue > > 62.36% -54.18% test [.] i40e_xmit_pkts > > 1.10% -0.94% test [.] dpdk_tx_threa= d > > 0.01% -0.01% [kernel.kallsyms] [k] native_sched_clock > > +0.00% [kernel.kallsyms] [k] fill_pmd > > +0.00% [kernel.kallsyms] [k] perf_sample_event_= took > > 0.00% +0.00% [kernel.kallsyms] [k] __flush_smp_call_functio= n_queue > > 0.02% [kernel.kallsyms] [k] __intel_pmu_enabl= e_all.constprop.0 > > 0.02% [kernel.kallsyms] [k] native_irq_return= _iret > > 0.02% [kernel.kallsyms] [k] native_tss_update= _io_bitmap > > 0.01% [kernel.kallsyms] [k] ktime_get > > 0.01% [kernel.kallsyms] [k] perf_adjust_freq_= unthr_context > > 0.01% [kernel.kallsyms] [k] __update_blocked_= fair > > 0.01% [kernel.kallsyms] [k] perf_adjust_freq_= unthr_events > > > > Thanks, > > Ed > > > > -----Original Message----- > > From: Lombardo, Ed > > Sent: Sunday, July 6, 2025 1:45 PM > > To: Stephen Hemminger > > Cc: Ivan Malov ; users > > Subject: RE: dpdk Tx falling short > > > > Hi Stephen, > > If using dpdk rings comes with this penalty then what should I use, is = there an alterative to rings. We do not want to use shared memory and do b= uffer copies? > > > > Thanks, > > Ed > > > > -----Original Message----- > > From: Stephen Hemminger > > Sent: Sunday, July 6, 2025 12:03 PM > > To: Lombardo, Ed > > Cc: Ivan Malov ; users > > Subject: Re: dpdk Tx falling short > > > > External Email: This message originated outside of NETSCOUT. Do not cli= ck links or open attachments unless you recognize the sender and know the c= ontent is safe. > > > > On Sun, 6 Jul 2025 00:03:16 +0000 > > "Lombardo, Ed" wrote: > > =20 > >> Hi Stephen, > >> Here are comments to the list of obvious causes of cache misses you me= ntiond. > >> > >> Obvious cache misses. > >> - passing packets to worker with ring - we use lots of rings to pass = mbuf pointers. If I skip the rte_eth_tx_burst() and just free mbuf bulk, t= he tx ring does not fill up. > >> - using spinlocks (cost 16ns) - The driver does not use spinlocks, o= ther than what dpdk uses. > >> - fetching TSC - We don't do this, we let Rx offload timestamp packe= ts. > >> - syscalls? - No syscalls are done in our driver fast path. > >> > >> You mention "passing packets to worker with ring", do you mean using r= ings to pass mbuf pointers causes cache misses and should be avoided? =20 > > > > Rings do cause data to be modified by one core and examined by another = so they are a cache miss. > > > > =20 How many packets is your application seeing per-burst? Ideally it should be= getting chunks not single packet a time. And then the driver can use defe= r free to put back bursts. If you have multi-stage pipeline it helps if you pass a burst to each stage= rather than looping over the burst in the outer loop. Imagine getting a bu= rst of 16 packets. If you pass an array down the pipeline, then there is on= e call per burst. If you process packets one at a time, it can mean 16 call= s, and if the pipeline exceeds the instruction cache it can mean 16 cache m= isses. The point is bursting is a big win in data and instruction cache. If you really want to tune investigate prefetching like VPP does.