From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 0782245515 for ; Fri, 28 Jun 2024 11:23:26 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E02F542DC3; Fri, 28 Jun 2024 11:23:25 +0200 (CEST) Received: from CWXP265CU008.outbound.protection.outlook.com (mail-ukwestazon11020125.outbound.protection.outlook.com [52.101.195.125]) by mails.dpdk.org (Postfix) with ESMTP id 4D75240EE7 for ; Fri, 28 Jun 2024 11:22:45 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=asG8LIXHrAqMhVBvI0Ja4qwbMJJ+Hb7uerZ3Dr/H3z5XeaQSUV/A3TMN9Pjhv+sEhqy0gcOqFlphAzBjFxgTy0VYGw2leN+Ems2FUYlg3WDgD3Qu3i3tdlNqYq3oj94Iq6V4/pPN39I/+K9lHr6+42cK5yu0hQ/Pqg9MOgTNIhq9ehtaPehpYPc9ts/RX+3oqVcc66WQHVbyoPiyZdVTP5n6WvtusSdrR84YBg0iAaX0SCW+tJubyiM8ewGN/if4nwfKZloQJsCAJhVrKUhY9nXit7w8+Orbk6NCApyg0W0GK1MQgvDIX5xOcOEeApWdak/FbTVWWyDw4+cJ8K38Lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yvL2SNUrvHypkUDakdsaDc6qViWg2QkFbo8vzJzyBMQ=; b=mB1WKWaka1KnpcyjXLFULxhBO1MCzlv0iWGWBVco/IGVg6ua9tqiNTU9XrJaXxS4ITa2Fp74kOS6J/jWI6Wayi69qQ4ABktsKEM/FNRN1pwb3BIowAacFqZQvQT9GoSDYOmoaS45wDZyRiZRrQD4beHNTVyC6cuHyH1YXSYwbO7dnrYwzQifIjVlYLWLxIkzKQjY+hCjVGZaHLHJnAO6K+7jztERZYab4dGJEi1WXJOo0bJ8cdOOcoudw1hPLg33+oNttEia5BfbrrRHWDSNWKSs02HnskcBaGF5Wl+L2vpsq6o1n90Jq8tKpoDVnZ9niaZYqRkWq8sBfNMeX+5aKA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 51.138.85.117) smtp.rcpttodomain=dpdk.org smtp.mailfrom=calnexsol.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=calnexsol.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=calnexsol.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yvL2SNUrvHypkUDakdsaDc6qViWg2QkFbo8vzJzyBMQ=; b=cyCVJasxYNfbH8wmrNgws/gAd8uWLECFeTmxPDL0lqSHt0a2f5xS/ZEvWst9d2S414zMBdoI62MDUwwcnVvV9DIgaF+knqvNCSECzxtUiOZBS22B2e8L8AfSbPdSd/5vDfLZDDhY8nmA7TZAOKCebeJS3gMZV29SJvzQu2fhUpc= Received: from DU2PR04CA0280.eurprd04.prod.outlook.com (2603:10a6:10:28c::15) by LO4P265MB6716.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2f1::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.26; Fri, 28 Jun 2024 09:22:43 +0000 Received: from DU6PEPF00009523.eurprd02.prod.outlook.com (2603:10a6:10:28c:cafe::df) by DU2PR04CA0280.outlook.office365.com (2603:10a6:10:28c::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.25 via Frontend Transport; Fri, 28 Jun 2024 09:22:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 51.138.85.117) smtp.mailfrom=calnexsol.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=calnexsol.com; Received-SPF: Pass (protection.outlook.com: domain of calnexsol.com designates 51.138.85.117 as permitted sender) receiver=protection.outlook.com; client-ip=51.138.85.117; helo=xwaresmtpservice-58d846c49b-sfk9f.xware-prod-eu-k1.o365.crossware.co.nz; pr=C Received: from xwaresmtpservice-58d846c49b-sfk9f.xware-prod-eu-k1.o365.crossware.co.nz (51.138.85.117) by DU6PEPF00009523.mail.protection.outlook.com (10.167.8.4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.15 via Frontend Transport; Fri, 28 Jun 2024 09:22:42 +0000 Received: from LO3P265CU004.outbound.protection.outlook.com (40.93.67.7) by xwaresmtpservice-58d846c49b-sfk9f.xware-prod-eu-k1.o365.crossware.co.nz with Crossware for Office365 id 33cfb276-4f6b-45c6-89a1-9f5653692bd5; Fri, 28 Jun 2024 09:22:41 +0000 Received: from CWLP265MB6499.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:1db::12) by CWLP265MB2674.GBRP265.PROD.OUTLOOK.COM (2603:10a6:400:a1::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.25; Fri, 28 Jun 2024 09:22:40 +0000 Received: from CWLP265MB6499.GBRP265.PROD.OUTLOOK.COM ([fe80::1666:5ac9:b829:205d]) by CWLP265MB6499.GBRP265.PROD.OUTLOOK.COM ([fe80::1666:5ac9:b829:205d%4]) with mapi id 15.20.7698.033; Fri, 28 Jun 2024 09:22:40 +0000 X-MAIL_SIG_CONFIGNAME: Calnex Standard Signature (current) X-MAIL_SIG_CONFIGNAMEPLIED: Calnex Standard Signature (current) From: Ken Andrews To: "users@dpdk.org" Subject: Mellanox - Unexpected CEQ error, rx stops receiving packets Thread-Topic: Mellanox - Unexpected CEQ error, rx stops receiving packets Thread-Index: AQHayTZvXdTRG8YAUUi1PrqLUS1Ypw== Date: Fri, 28 Jun 2024 09:22:39 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: X-MAIL_SIG_CACHEID: 13162 x-ms-traffictypediagnostic: CWLP265MB6499:EE_|CWLP265MB2674:EE_|DU6PEPF00009523:EE_|LO4P265MB6716:EE_ X-MS-Office365-Filtering-Correlation-Id: b0ffa9f6-959f-47da-5973-08dc9753dd23 X-MAIL_SIG_ADMINUNIT: none X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam-Untrusted: BCL:0; ARA:13230040|366016|1800799024|376014|38070700018; X-Microsoft-Antispam-Message-Info-Original: =?iso-8859-1?Q?w5qgX4ZN+axs6stVeeudZJv5fgw/1DDEmUccCuVe7SDnS3KobfZnGSofgI?= =?iso-8859-1?Q?E+G+zR4Rc4TswW1RMuHsYF76VTEsNJ25qs2euM1XNSHB4O/ImzIDTYiabF?= =?iso-8859-1?Q?UJssY11aMBh2SFIW+kkC1yoELe4Rssn42qdJ2HZGyvg8QeYfV7cOBqAgxK?= =?iso-8859-1?Q?5VBR3JeZjbejSCWlr5gzyPrib4f2j8d0YcytqZfak4iJDZvFIfxnxH+mSt?= =?iso-8859-1?Q?hJ4GyoWVqTEPTvIIaI/UDgL1AXmHHMmYqnk7oWJQYzLOVk7UtcGGeoXcU/?= =?iso-8859-1?Q?1y5Msrhx/Koc87DEknqqn/RYoQzvXII3G1pExwxGuBWQyZ1DparbD7MHlu?= =?iso-8859-1?Q?Z2jNDHgNdUcWGt8uBM6Tszsn8R5jpkhSg/N2MRZwMpfNSg/vROJzeUIyz1?= =?iso-8859-1?Q?Xgm24cuAE0/R4cjw8yO6BZIKCkNIxFY6yky9zxU51RD5JAnNmBPkmiByqZ?= =?iso-8859-1?Q?TJY7kdJwsaXDSWVkq146+gEvYhxGjB/PMnuRTn8xr4DkPB+z4nPcy4Mz+2?= =?iso-8859-1?Q?2Z/xYnULGjZZSOr0KXSzxzlqHU/5y6yjYpczeZJxsZRL6i1qtUtbajnakc?= =?iso-8859-1?Q?j5K8BE5cZzBjmoLviWVsvSuiWBza38PI8Cc0+U5+vBMulvj/0hDm3C9tQJ?= =?iso-8859-1?Q?P3QB0QcUf77xAOpVjWCmQvCjeig630dscSWiFlaaXNJcRP0hTEChBJzyu5?= =?iso-8859-1?Q?oTN8P5PYjMeVBbkJfv8uQ5dJJZKJtuSuhgQstRm+uvtlPMVllVEO9maBN+?= =?iso-8859-1?Q?rqBaa6IchHTqox/QN/FAnIGt0uwzjMhfFQ4zax2IAScQ1fhaSwxUd8we9M?= =?iso-8859-1?Q?2ggbxrC0uWBquLLjdY8mc3yzKAMwrbBSLbFL9o37MQEJMjFm96AmBw4HaI?= =?iso-8859-1?Q?k5bsH/b72bYwgHV6RJzTxDIQuKyZGGv8met8JxkBTPd1stjBHtP8SnY5Ji?= =?iso-8859-1?Q?9rZJptTxYiHbImPyBXejwJCa+Gvs+3q8R8fmRfuGyEiI3jMprWd0kqGliz?= =?iso-8859-1?Q?kUyDSy0bsa3qP1egfxbkJGFrLGklwyvuQWBxx6cETghhTBynmIC9vVTRwp?= =?iso-8859-1?Q?ydB2WUB1PFXKwFCpXmVW8FFgDMqA8iX9DJSoAj/JDSlBuao5mgKRJZD3eQ?= =?iso-8859-1?Q?+8g8nrVjrHGkxZUZ+3q+ptT3TL4+FthA5Nl78gjxccf2r71OjWCsxUmz1P?= =?iso-8859-1?Q?A4DqNfB3jR3RYHApf0qmktZugrGV/baRh0EnedPwQuW6qcXAlSTF3CoEzn?= =?iso-8859-1?Q?v8duxX4uEuMQ5lWOh7FNpBUfWCVeomrwgqobncrFq8YUpWwjrqFPLHYLId?= =?iso-8859-1?Q?mUbLxXuDZ4pPrmv1hupX8TXEOOHVgpmXKqcUC8rumAE3y+QGuUKzaExlCx?= =?iso-8859-1?Q?Wo+xIkMapZ5y475x+cNQUc3gTuxW3yLBLplTEnBnFRrrhdXQ6YMzQ=3D?= X-Forefront-Antispam-Report-Untrusted: CIP:255.255.255.255; CTRY:; LANG:; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CWLP265MB6499.GBRP265.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(13230040)(366016)(1800799024)(376014)(38070700018); DIR:OUT; SFP:1102; X-CWesigProcessed: Y X-MAIL_SIG_VERSION: 0.2.546 X-MAIL_SIG_SERVER: xwaresmtpservice-58d846c49b-sfk9f.xware-prod-eu-k1.o365.crossware.co.nz Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CWLP265MB2674 X-EOPAttributedMessage: 0 X-CrossPremisesHeadersPromoted: DU6PEPF00009523.eurprd02.prod.outlook.com X-CrossPremisesHeadersFiltered: DU6PEPF00009523.eurprd02.prod.outlook.com X-MS-Exchange-Transport-CrossTenantHeadersStripped: DU6PEPF00009523.eurprd02.prod.outlook.com X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id-Prvs: 96b6e089-65b6-4139-19ec-08dc9753db7c X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|1800799024|36860700013|35042699022|376014; X-Microsoft-Antispam-Message-Info: =?iso-8859-1?Q?PzZuqwJhVxqjo5RXUa4YTmdOkKpvWHw+NjdPUgwgGNPS6+PQZLx8CTlyy/?= =?iso-8859-1?Q?+AYRx47khspSF92gZDV9BpnyMtqEgSG69/ELtVlq39S+2iRiplD1kWX2Ql?= =?iso-8859-1?Q?shSu5vGMUBht7PofThnxKmYqYfzAOacrER++xm1MKqIH+DxmdqB5XBCuTn?= =?iso-8859-1?Q?oaw2pe1T6S9Pp2va35eCJY8bZTwPwYY3YZnDyzr6sVuEbAzXMqeW4foptS?= =?iso-8859-1?Q?wm9hDHDHuM1C7SdiiOelSQ/mEmhbDlbSGC02Lwd0DothuKXSs8lZsnud3N?= =?iso-8859-1?Q?zCvxrvXGRAc77SUD6TqDw6tjuLgrzN5t2bEI5qxd0cguJcM4hxpYJxVNgQ?= =?iso-8859-1?Q?zyxeEmkGbTKd89+P0Yac0WcckR8pRv1qjDkH/WoKQN1VkNZqI/85n1BRIE?= =?iso-8859-1?Q?IVLk0nSWRBP/daTAKGuECc7Op97reZNX3VnF1amUzor86NWWqsPRP1LWhj?= =?iso-8859-1?Q?CCxfCPoIUlQbanrTgeL236FWbLfFgPUn/SfMt2PilfBnIrtmvYfdJt+v4I?= =?iso-8859-1?Q?C1BH3vnVQ18MzqZm40aUT3sgMsfPVGIECMr8lkdpWjefhheg6WmLaCru2S?= =?iso-8859-1?Q?sOVP9mpNoF3QNN/aMNfW2Bqi68S1VBO9kv6rt1DS6vNbWTnbdcT1Tz+3MU?= =?iso-8859-1?Q?9rKzSKoj9dFEf1xdbvFDN1EBDPlgLvItVAIWnBDaw/+XsXQ+4wo/l7v+RW?= =?iso-8859-1?Q?aSO0A/D5lDLqM/SvGEakGeVBAC98KBa3tsLq8Id/zENbnyFRzrfMItkwF7?= =?iso-8859-1?Q?WU2XLxh+X1m1vRaXkTocFHnBIbExDPY9A8guGWl5B2bGYBqOCjD+urkKWY?= =?iso-8859-1?Q?f96P7iwbTfgl1tVnuiQtWhvCgKi9JjnRK8bKk4wcfLm40KeveQnTQQUfkh?= =?iso-8859-1?Q?uDGFXO8CKLzmTY1gVHnGDC/tt0jjZ7IxKX7jVXJwCXrAPllLzTq+tvCoFl?= =?iso-8859-1?Q?mL7DrmKAkmd1Fo7m/vBTOvx88U5hofhAs0bzxG/A5jJ+gfNcfWgM0RiB2a?= =?iso-8859-1?Q?kE2eSJLoqhsLVa/yVONok9IATekkuoOjyMrAIOqQ2pPPNJ1EMgWmbY2mdG?= =?iso-8859-1?Q?0g/e9uys/BvPyKqJ5WUV1e9k3aFwvMKlviFFeFcJT4PytXIoB5m7O0o3NX?= =?iso-8859-1?Q?K+ruMoxXuKpsJGdy06cPyP+2/joPlgeS3MgMVkUVIylD6cNhV1ntxRBzRg?= =?iso-8859-1?Q?TnNP6USftorV0PSzi5KAvOkuscRjwJ5DEwmY8tbvhrB0vDOP+1m5FJPEu7?= =?iso-8859-1?Q?mEhkeQfyx6hnN34IpQDJsi5dgSppJ9YL6WlSPBFGIy18nelGb+Ph2Q4M4A?= =?iso-8859-1?Q?x327gZzjYUFtKCjjAEayw5F+dZI1xkvot/A5D5fexhqPmddhAZDP5IlI3S?= =?iso-8859-1?Q?lloIWKH1inf7y68ET2ETY9Xf4JtGQeb8QDQts1pK1dRvCp7abXg5r51PZH?= =?iso-8859-1?Q?uqlXbaEf4IHVoqjg?= X-Forefront-Antispam-Report: CIP:51.138.85.117; CTRY:NL; LANG:; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xwaresmtpservice-58d846c49b-sfk9f.xware-prod-eu-k1.o365.crossware.co.nz; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(82310400026)(1800799024)(36860700013)(35042699022)(376014); DIR:OUT; SFP:1102; X-OriginatorOrg: calnexsol.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jun 2024 09:22:42.3866 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b0ffa9f6-959f-47da-5973-08dc9753dd23 X-MS-Exchange-CrossTenant-AuthSource: DU6PEPF00009523.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-Id: fbb4e2fb-f802-4d55-beca-b7149551e928 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=fbb4e2fb-f802-4d55-beca-b7149551e928; Ip=[51.138.85.117]; Helo=[xwaresmtpservice-58d846c49b-sfk9f.xware-prod-eu-k1.o365.crossware.co.nz] X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LO4P265MB6716 X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi, I'm seeing an issue previously mentioned on this list in 2022, where my Mel= lanox NIC is logging an Unexpected CEQ error syndrome. Once this condition = is hit, the=A0 rxq->err_state var in the mlx5 PMD is never cleared, and the rx just loops = around never receiving any further packets.=20 It's not clear what's causing the initial CEQ error, as it can take upwards= of an hour to occur. The full log entry is: Unexpected CQE error syndrome 0x04 CQN =3D 256 SQN =3D 4679 wqe_counter =3D= 2149 wq_ci =3D 3294 cq_ci =3D 42609 MLX5 Error CQ: at [0x292d26000], len=3D16384 The NIC is: =A0NVIDIA ConnectX-7 HHHL Adapter card, 400GbE / NDR IB (defaul= t mode), Single-port OSFP, PCIe 5.0 x16, Crypto Enabled, Secure Boot Enable= d Part number: MCX75310AAC-NEA_Ax Firmware: 28.36.1010 OFED Version: 24.04-0.6.6.0 DPDK Version: 23.11.0 This issue was previously mentioned in this post: https://mails.dpdk.org/ar= chives/users/2022-October/006779.html Can anyone please help shed some light on this? Thanks, Ken AndrewsKen Andrews R&D Departmentt: +44 1506 671416e: ken.andrews@calne= xsol.comw: calnexsol.comNew Product The SNE-X is a total solution to the problem of real-world Ethernet testing= . It combines comprehensive and efficient network emulation for 5G, Data Ce= nter, and Cloud applications. Click for more information. Calnex Solutions Oracle Campus Linlithgow EH49 7LR United KingdomCalnex Solutions plc is registered in Scotland. Registration = number: SC299625. Registered office: Oracle Campus, Linlithgow, Scotland, E= H49 7LR, United Kingdom.