From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 69D49A04E6; Wed, 9 Dec 2020 15:15:40 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AA48BBE75; Wed, 9 Dec 2020 15:15:38 +0100 (CET) Received: from nat-hk.nvidia.com (nat-hk.nvidia.com [203.18.50.4]) by dpdk.org (Postfix) with ESMTP id 7D9F198 for ; Wed, 9 Dec 2020 15:15:36 +0100 (CET) Received: from HKMAIL101.nvidia.com (Not Verified[10.18.92.77]) by nat-hk.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Wed, 09 Dec 2020 22:15:35 +0800 Received: from HKMAIL102.nvidia.com (10.18.16.11) by HKMAIL101.nvidia.com (10.18.16.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 9 Dec 2020 14:15:32 +0000 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.174) by HKMAIL102.nvidia.com (10.18.16.11) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Wed, 9 Dec 2020 14:15:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YfizYaBWfyN3Vp0uEkl6zLXxEe55KOs3bGrvyGhTbl6s7uztK6dbIv8JZFrSMk0SyAe44ApeHPszolG4rOd5HFmPzZksnrApGB8CTdPJ5RjzX0Pb4nFXKHHl2i1fgockXLaWvk5oVCF17nn8V/lGvFqUgqtyw2qat4XJrPZLcFDZCyULZBP6f0HcW9TDRToCZkJ4SkGc1x6S235dgy8k6xlshbdDpyAE3pyvc9POyApT2GamL0hITYOFzBTZnHbD4BXAsZ3MFBr1P1rfX3BYyYmPNRqCfWYr1VitbkZRE0PYOs+qNy4HR8eZ39TSVpMfVlB0rc+Jp07gx74J0mrVFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4Sji1G7AmSbMkGag8VcfuvoI5VWRLcU7S08cxF27uD8=; b=gXprrkodlpwdwnOPldi7lLV6rkYKJBy9DNseGgb300yZutYr1Yqtwo3FRS/plCv7JgCHtAHUGLrA+MWNjpeOr4totYqYYSZkZ3ZOWSBcgMILacJ63OLBRFKcVWfIybvgte1n4R8euhg6HYOhbWar3tJuaqosH9nym4+M92O5o6OVgFOp7+bvMrJj3LRnkomMRkLPRdqOpbmuT+UmJS9+7AuY3E8MonAm3eWVtWMsyjwJie9xwG6zUxm2nCVp4TLpPSLCnPT+rrpG8qu+ZZJKR7Xr8Tw1pT2HLjyZAmsv7x//yOdWLy1F9Dj/+fENUF1/jigc61IpGP8Q69UcPJIA5A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from DM6PR12MB3945.namprd12.prod.outlook.com (2603:10b6:5:1c2::27) by DM5PR12MB1915.namprd12.prod.outlook.com (2603:10b6:3:10c::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3654.12; Wed, 9 Dec 2020 14:15:30 +0000 Received: from DM6PR12MB3945.namprd12.prod.outlook.com ([fe80::fc92:3e29:29d1:27b2]) by DM6PR12MB3945.namprd12.prod.outlook.com ([fe80::fc92:3e29:29d1:27b2%6]) with mapi id 15.20.3654.012; Wed, 9 Dec 2020 14:15:30 +0000 From: Tal Shnaiderman To: Dmitry Kozlyuk , "Dmitry Malloy (MESHCHANINOV)" , Narcisa Ana Maria Vasile CC: Eilon Greenstein , Omar Cardona , Rani Sharoni , Odi Assli , Harini Ramakrishnan , NBU-Contact-Thomas Monjalon , "dev@dpdk.org" Thread-Topic: Windows DPDK real-time priority threads causing thread starvation Thread-Index: AdbONYWvtK4WZhcIQAeTWkc2TKlT0A== Date: Wed, 9 Dec 2020 14:15:30 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [77.137.115.184] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 4244967b-a95b-41e0-dc52-08d89c4ce2a2 x-ms-traffictypediagnostic: DM5PR12MB1915: x-ld-processed: 43083d15-7273-40c1-b7db-39efd9ccc17a,ExtAddr x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 1hqLyl5rfnMY7rb9y8yWjUyJmz5lJ5oKbr3avg4MTO0DERBuB4lesHVzG+XWT2n9a87awDMXCKpkbtT87C/HsEUjqRMiAqDwK5SlVYHeysfgYHpmOVrET+S2O/HB8ifG1TVbGcP5QykEsxLqhlWhfGlqFV4oYXExFJMKe/KWATJGzQJkb6Rab3k+foUmUvNNTqKUSFzXuGYfx52P0dnCzoZQclIW+ayuHukNFaGvls2C88fAt0lNrYlkEJpUpxQUYWuqG2aSu+732sq+78cf79GPdk1XuA2sZbugx4zc7MzZj4UIb2mo9NXgkBHMxRGakC5fUmfh/TAIzSiGMqfs/fyme4Hl0hX63qvzuXcS+D561l2GdpfH70v48gKQ/V0xhJaJkpUtlvR6gK3fmsuCgQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR12MB3945.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(136003)(366004)(346002)(54906003)(66476007)(86362001)(83380400001)(64756008)(76116006)(52536014)(5660300002)(66556008)(66446008)(110136005)(186003)(508600001)(26005)(66946007)(7696005)(2906002)(71200400001)(55016002)(966005)(8676002)(6506007)(9686003)(4326008)(8936002)(33656002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?rse2lUApwgRUOeOkoNk8aoW3ATj45NzKRSjS8PMkorLg75Qp5Daymtdony?= =?iso-8859-1?Q?U+uZJ4cDFKg7o93JP+2eQugQkLDq2fkZzfHPSz/2UwHd6gqagnvcerVaE9?= =?iso-8859-1?Q?1v7espXcsBpWJKCFurB07jVRF259izURZj0Kobvkkj6M6tyXpI1HvWruI0?= =?iso-8859-1?Q?tDD+p2vUhyBFyNxhFK8jS/P2uVPTICygCq+EbsPk0mA3H4/1GQrd29/qYq?= =?iso-8859-1?Q?EXcNImeWueOEHVAudEZnfVsCRkIUQDggcwVgmlOEpv0uXV/jEY2a64003d?= =?iso-8859-1?Q?8cl/8FOC3+STAJWXBBJvsardj6KKi8diKqnJctno3BxbSKq237Sg57bXIE?= =?iso-8859-1?Q?3jA9539L2OyZeX6Lbg5cw0yVHM770wV/m+G6wke05zV6mnkt/0fxq6UwM2?= =?iso-8859-1?Q?0qT48/w1eOGsmSXeaEjecxTPfXqVBY24QWPrQhWmHd0z+b+Ra+149PTTjH?= =?iso-8859-1?Q?ANCoMlCWQ5j9yAOH8UWCpGZFkgxEU0p7qBMTfWKjCYJAcO4UXFNq/aIYXH?= =?iso-8859-1?Q?wy9KNPu2MlrpefiHTfbbuhF5kVx07CfEIjvMs5DNMkRRHEjCCR4V9KZBs9?= =?iso-8859-1?Q?GClXPhKqm8WLtXMuuzI69jVK3Bolgbk+smqhJCqt5NTPlNzYmrZrZQi0nM?= =?iso-8859-1?Q?jirT1aEjk3MzmWijsMFjL2KoiDvBygN53JXgDNxVrtU05UiNxWJYmAeL4H?= =?iso-8859-1?Q?GNP5z9yV5OlelghG5Pfn+6sXWffXXKHmceIHu58XTbWIamPzmMlFzi2s6J?= =?iso-8859-1?Q?bNwUro8+N4745DZqffgHE6JWcn94nnUrB62ue9TsyHQ9nOboFGsuXas3CE?= =?iso-8859-1?Q?bWVZl/LaSjfsbUuBqZsTrseOo3BRrzXtSMUjoKSsGmWweAi1nysKvT21iP?= =?iso-8859-1?Q?zgWUTLlHZ7QTSlY/3VzlR1gUMbM5Gt0nar1pYBXhkkpvr/PGtG4WgS6HmR?= =?iso-8859-1?Q?jr/RSwHFpQbCe9b9nc3QKP+WumkU7exXqlXb3aJimIsMzaHSfMqR5AKoiS?= =?iso-8859-1?Q?bcnTeIfNwAYF4K2iE=3D?= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3945.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4244967b-a95b-41e0-dc52-08d89c4ce2a2 X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Dec 2020 14:15:30.5757 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: UUVfDrXmmPsLqwVDM4Bb8MCe/eERx0hExAhOoa1NqtHAC2rz5rivMLl2cN3niaeIcSns0bEsmvMBYGX9Runrcg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1915 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1607523335; bh=4Sji1G7AmSbMkGag8VcfuvoI5VWRLcU7S08cxF27uD8=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID: Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ld-processed: x-ms-exchange-transport-forked:x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers:x-ms-exchange-senderadcheck: x-microsoft-antispam:x-microsoft-antispam-message-info: x-forefront-antispam-report:x-ms-exchange-antispam-messagedata: Content-Type:Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=qyeu5lZMEIMfiG9Rxvvav+TbW6DeNYK6qTBJP7rOcARoLSpYgkIJtoGNmZ3pH/on9 yWTuLBGSDEXUseKLAfZ8x/MTzSXN8H0rR5gWQJbwlgwlB8CE4z30eYLnHanNpvOn5n vhpXBk7O0ZQP4tRXhTtmq23LHqTz8dwbLbvKS4sZXomiHmN+oxiO1AE3rPPEX1q47J T58IMoZLu2FZcHyYSZr2+SyxmvhFgoEeAc191aWhpbPqvdPhgKPycKQ5iTc29itNRF em+UMdI5yMb4+ipT2B6n4xezU9Dy9s+RHM4dAaFuzkCMjUwbY0C8ueVVYx+wUfGtjI JvCNb9Vkqg3fQ== Subject: [dpdk-dev] Windows DPDK real-time priority threads causing thread starvation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, During our verification tests on Windows DPDK we've noticed that DPDK polli= ng threads, which run in REALTIME_PRIORITY_CLASS are causing starvation to = other threads from the OS which need to change affinity and run in lower pr= iority. While running an application for a while we see the OS thread waits for 2:3= 0 minutes and raises a bugcheck, see below example of such flow: 1) DPDK thread running on core-0 in real-time high priority(24) polling mod= e. 2) The thread is blocking the system function=A0NtSetSystemInformation (Exp= UpdateTimerConfiguration) in another thread from=20 =A0 =A0switching to core-0 via=A0KeSetSystemGroupAffinityThread since the c= alling thread is priority 15.=A0 3) NtSetSystemInformation exclusively=A0acquired system-wide lock (ExpTimeR= efreshLock) hence=20 =A0 =A0 it blocks other threads (e.g. calling=A0NtQuerySystemInformation). We've seen this behavior only while running on Windows 2019 VMs, maybe on n= ative machines OS scheduling of such flow is done differently?=20 Below is usage explanation from the documentation of SetPriorityClass [1]: - REALTIME_PRIORITY_CLASS Process that has the highest possible priority. The threads of the process = preempt the threads of all other processes, including operating system proc= esses performing important tasks. For example, a real-time process that exe= cutes for more than a very brief interval can cause disk caches not to flus= h or cause the mouse to be unresponsive.=20 So I assume using this kind of thread for a long period as we do can cause = unstable behavior. How do you think we can resolve this? Are there such cases in Linux? [1] - https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/= nf-processthreadsapi-setpriorityclass Thanks, Tal.