From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0069.outbound.protection.outlook.com [104.47.1.69]) by dpdk.org (Postfix) with ESMTP id 8CFF61B216 for ; Tue, 10 Oct 2017 16:28:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=E3/k6OjaNNESnDgl5xRJoYAL8AG4LOExb1eaIUVqM34=; b=jKt8gzGfgK0ljj//ypwzDQKRrt234mOrenAWgT5CsAPv0eIF1UwTAuB03TbbgSrCQvUAxHH8Rh57AMt2rtXd/JNwasPUoqLfSp3d2T9jxBgUysuCd3JvhYqBBovvHC5IHJtD6NAQoGNXWcGsVSaYgcn9y2i/D64TvlEdRpVxXkY= Received: from VI1PR0501MB2045.eurprd05.prod.outlook.com (10.167.195.147) by VI1PR0501MB2048.eurprd05.prod.outlook.com (10.167.195.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Tue, 10 Oct 2017 14:28:27 +0000 Received: from VI1PR0501MB2045.eurprd05.prod.outlook.com ([fe80::84ed:9505:3e7f:9722]) by VI1PR0501MB2045.eurprd05.prod.outlook.com ([fe80::84ed:9505:3e7f:9722%13]) with mapi id 15.20.0077.018; Tue, 10 Oct 2017 14:28:27 +0000 From: Yongseok Koh To: Martin Weiser CC: Adrien Mazarguil , =?iso-8859-1?Q?N=E9lio_Laranjeiro?= , "dev@dpdk.org" , Ferruh Yigit Thread-Topic: [dpdk-dev] Mellanox ConnectX-5 crashes and mbuf leak Thread-Index: AQHTNqktVwg/YRPpzEiZDb3J3tSb96LV2VeAgAES/ACAAIvqAIABjw0AgAPJ+ICAAGl6AA== Date: Tue, 10 Oct 2017 14:28:26 +0000 Message-ID: <7DB3162E-9A29-4668-BCB3-693CCB4D772A@mellanox.com> References: <5d1f07c4-5933-806d-4d11-8fdfabc701d7@allegro-packets.com> <374F8C13-CFB0-42FD-8993-BF7F0401F891@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-originating-ip: [73.222.116.174] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VI1PR0501MB2048; 6:gKcwqXy2ev3kQ0Lvh9HVV/W6GYm+/hPnZPzzxOuWEs/xB/oquxDYlqeLDcEsnU2k4KlqyFm7s65rcbdc/j7DJp8hEXBWL6CkfX71VUizbPCiv1sTzHRKKbDCMOfY+vTaThb8NXxBVAybEHlXpBjkiW3dVL5kROCB89fFaawlOMudktLtYX3shiCfk2utWc4qpH5MYOK5nVIjyjV4PVWYaw0KjLXyoaJ61IbaHZzmYJa1D928mOCZuNE5qSFJIzHpt5ZTkC6todMvT8QiHZGYjzIjvMAZSGbIqR7sW8H8nNaufv4S38jAiixbx5mpJi6u3drX1tnntyLkXlHjZVIFzQ==; 5:3A2ShjXKaboGQmFuUFclzcpAMVbNZmEe7vF/Un3kbAP+0/xwOBmUinWol+hc4XV+HXGqJSSS/0dr3l7toHHtcKoHBQWvKQCDKEk7/Z4solxZ4YDxPIYU7PaZkpNxJ2JCHMTcTVdhgkClSV9WYIKSMA==; 24:f20gQUr/EZF/Ve461SccdH1XrQZxmf9ESfx5/hTSaen+rkGgTWui+YRMi9s+qIRfx8vibZ6Rx1gIilhdyl01hMsVz+v8LCjkioDMsq1syRc=; 7:B9AzzgAyCsVCSQZfg9vCjawLUxK77jaVzWB1C/REGacQ/tp/bYgo/pe9vJPH8cDBQhYS+YZpTiTD8mLKqhdg+FaIKR35+QNEN0Mcu1QBEXfQXn/c6r1NyoONar+JsYW0yZN+nMMtBk5nxJQIM6Zp7yIQrAnIqkUuIsM3nwo6TJ4XMNvF9C496wFddEvi8MkDfymsecBmDvgvROU4SqEOug33FBSjqmo6MJ9/Xz7rTZQ= x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 56114be0-fcf6-4bb5-88b3-08d50feb2c12 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(48565401081)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:VI1PR0501MB2048; x-ms-traffictypediagnostic: VI1PR0501MB2048: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-exchange-antispam-report-test: UriScan:(189930954265078)(45079756050767); x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(10201501046)(3002001)(100000703101)(100105400095)(93006095)(93001095)(6055026)(6041248)(20161123555025)(20161123564025)(20161123560025)(20161123558100)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:VI1PR0501MB2048; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:VI1PR0501MB2048; x-forefront-prvs: 04569283F9 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(39860400002)(377454003)(189002)(199003)(24454002)(42274003)(6512007)(316002)(6116002)(25786009)(83716003)(99286003)(105586002)(97736004)(53936002)(81156014)(81166006)(106356001)(966005)(6246003)(2906002)(54356999)(54906003)(76176999)(101416001)(5890100001)(5660300001)(45080400002)(3660700001)(189998001)(6486002)(68736007)(229853002)(2950100002)(6916009)(33656002)(53546010)(93886005)(478600001)(86362001)(50986999)(8676002)(2900100001)(6506006)(6436002)(14454004)(4326008)(5250100002)(3280700002)(7736002)(8936002)(345774005)(102836003)(305945005)(36756003)(3846002)(66066001)(6306002)(82746002); DIR:OUT; SFP:1101; SCL:1; SRVR:VI1PR0501MB2048; H:VI1PR0501MB2045.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Oct 2017 14:28:27.0030 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0501MB2048 Subject: Re: [dpdk-dev] Mellanox ConnectX-5 crashes and mbuf leak X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Oct 2017 14:28:28 -0000 Glad to hear that helped! Then the patch will get merged soon. Thanks, Yongseok > On Oct 10, 2017, at 1:10 AM, Martin Weiser wrote: >=20 > Hi Yongseok, >=20 > I can confirm that this patch fixes the crashes and freezing in my tests > so far. >=20 > We still see an issue that once the mbufs run low and reference counts > are used as well as freeing of mbufs in processing lcores happens we > suddenly lose a large amount of mbufs that will never return to the > pool. But I can also reproduce this with ixgbe so this is not specific > to the mlx5 driver but rather an issue of the current dpdk-net-next > state. I will write up a separate mail with details how to reproduce this= . >=20 > Thank you for your support! >=20 > Best regards, > Martin >=20 >=20 > On 08.10.17 00:19, Yongseok Koh wrote: >>> On Oct 6, 2017, at 3:30 PM, Yongseok Koh wrote: >>>=20 >>> Hi, Martin >>>=20 >>> Even though I had done quite serious tests before sending out the patch= , >>> I figured out deadlock could happen if the Rx queue size is smaller. It= is 128 >>> by default in testpmd while I usually use 256. >>>=20 >>> I've fixed the bug and submitted a new patch [1], which actually revert= s the >>> previous patch. So, you can apply the attached with disregarding the ol= d one. >>>=20 >>> And I have also done extensive tests for this new patch but please let = me know >>> your test results. >>>=20 >>> [1] >>> "net/mlx5: fix deadlock due to buffered slots in Rx SW ring" >>> at https://emea01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2F= dpdk.org%2Fdev%2Fpatchwork%2Fpatch%2F29847&data=3D02%7C01%7Cyskoh%40mellano= x.com%7Cd026493e00eb429cb6b608d50fb673ed%7Ca652971c7d2e4d9ba6a4d149256f461b= %7C0%7C0%7C636432198675778556&sdata=3DYOZQFrSeIM%2BFw42CvqazNDYSv98jKB%2F2b= MRSqrYo2a8%3D&reserved=3D0 >> Hi Martin >>=20 >> I've submitted v2 of the patch [1]. I just replaced vector insns with re= gular >> statements. This is just for ease of maintenance because I'm about to a= dd >> vectorized PMD for ARM NEON. In terms of functionality and performance = it is >> identical. >>=20 >> Please proceed your testing with this and let me know the result. >>=20 >> [1] >> [dpdk-dev,v2] net/mlx5: fix deadlock due to buffered slots in Rx SW ring >> , which is at https://emea01.safelinks.protection.outlook.com/?url=3Dhtt= p%3A%2F%2Fdpdk.org%2Fdev%2Fpatchwork%2Fpatch%2F29879%2F&data=3D02%7C01%7Cys= koh%40mellanox.com%7Cd026493e00eb429cb6b608d50fb673ed%7Ca652971c7d2e4d9ba6a= 4d149256f461b%7C0%7C0%7C636432198675778556&sdata=3DIrrEKKLWYRarrbE2McSzytYa= Q4zdh1nAnsWErgijd%2Fg%3D&reserved=3D0 >>=20 >> Thanks, >> Yongseok