From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 782F3A0032; Fri, 14 Jan 2022 14:54:01 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id EE8C64114A; Fri, 14 Jan 2022 14:54:00 +0100 (CET) Received: from mga17.intel.com (mga17.intel.com [192.55.52.151]) by mails.dpdk.org (Postfix) with ESMTP id C28C140C35 for ; Fri, 14 Jan 2022 14:53:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1642168439; x=1673704439; h=from:to:cc:subject:date:message-id:references: in-reply-to:mime-version:content-transfer-encoding; bh=9skvzSd7sELuUceGc4hefXht5c5c4ZB4DjeA/rgIIqw=; b=WmlFvL6D40lrChoaBjM68oIlnauhcYKdic2fMWIKiSRpIiU4v2v8VgDt 9W+CF7dt9VwUas21J4QhQ1UiNkm5VnAwj4AkB/laIQqyIVRQm4iIt1zk4 gHOJuI/QAX1t0S/yHO0dKtXo2jra+JjX28BPUS9u8qvWLyIgIa0WFs8f8 K8GqtbK/hrAvUWLb+a3202WD6uho0eRcKZjB+1++UCytTR4lFpLKEF54y gPWVsyMLF81zO1qbgwFMUcdr80vkvC3WYmQJlKpfMAKC1XaLqDEeWjYoZ 7xkBdHxngneM/PbHkxqBgnxclrSWH77dhgyGdsy4iMeBI2gDB911tXgX9 w==; X-IronPort-AV: E=McAfee;i="6200,9189,10226"; a="224934992" X-IronPort-AV: E=Sophos;i="5.88,288,1635231600"; d="scan'208";a="224934992" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jan 2022 05:53:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.88,288,1635231600"; d="scan'208";a="473657245" Received: from orsmsx604.amr.corp.intel.com ([10.22.229.17]) by orsmga003.jf.intel.com with ESMTP; 14 Jan 2022 05:53:54 -0800 Received: from orsmsx604.amr.corp.intel.com (10.22.229.17) by ORSMSX604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Fri, 14 Jan 2022 05:53:54 -0800 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20 via Frontend Transport; Fri, 14 Jan 2022 05:53:54 -0800 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (104.47.55.168) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2308.20; Fri, 14 Jan 2022 05:53:54 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DkMcRankFL/lfku21GCgBvAj3aNll9XiUjVd3t9MTssAIMxFr2JKbQfIKxMYv0LlIJe9fZE7xjB/yF2sBEVq5xaqkbUp26jqr7XyGBvoxiCBdplreOThX24BLbkBMdq88pSckuQMC+OP+UouADEp5rqBOcHDxaZ8d8Q7P/MioPh4u/003Fj+8KfzIw1f5Gib6KMrnXkurIHU36sOk3Vop6RNJX9dh+bKpg5n4mXJ7br6bZPnu9xq8LEUH5fzzitzRdqcAPsBKCTflU3pOPE0QpJz5pkxxVmVFjPzeH1dy3R60RD6rb9YReuA2X2nrjpWvEHJHWChwP4FxdGStcZvbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=K4Vn/YyzU57bwV1FkGcE7gBBLebg1B6QsB0krtsU0Xg=; b=V/ktpiLi0uqwkXG9hi2T13wX6LemEqpg2D+dcY7Atv48fW5ZQllKeJppJrGqIL+U8kVio5dlP4H6NSDLsR90G8aWJ365LEGTZnF0jaVOtrfmzwxTzNPI8JYygX5eXITYAeMRAbCtYWyzDZCuw8GRiRMWWyF0HLMw9F6xj+0IRoTL247SS34MYa5Kk4seL84fd7Xdv27kaT+ciQaP5fYDCNiOs//CTrHto5SvhJv127+enf7b8WRxlLKeZLARs8xRgyWsX/blEZYvrv4wXGoLVqwZeNnG3Yv5DoZotwgF42WzTb1VoaUAL6u/yEEFNaxX/U68Ps0XZRjsYd/f1DthCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DM6PR11MB3738.namprd11.prod.outlook.com (2603:10b6:5:139::25) by CY4PR11MB1335.namprd11.prod.outlook.com (2603:10b6:903:26::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4888.10; Fri, 14 Jan 2022 13:53:52 +0000 Received: from DM6PR11MB3738.namprd11.prod.outlook.com ([fe80::6cc6:2c99:fc5f:355]) by DM6PR11MB3738.namprd11.prod.outlook.com ([fe80::6cc6:2c99:fc5f:355%5]) with mapi id 15.20.4888.012; Fri, 14 Jan 2022 13:53:51 +0000 From: "Connolly, Padraig J" To: Tudor Cornea , "Yigit, Ferruh" , "Connolly, Padraig J" CC: "thomas@monjalon.net" , "stephen@networkplumber.org" , "Zhang, Helin" , "dev@dpdk.org" Subject: RE: [PATCH v4] kni: allow configuring the kni thread granularity Thread-Topic: [PATCH v4] kni: allow configuring the kni thread granularity Thread-Index: AQHX4Wj2+ugmDi5qC0i0oCVYWTmTp6xi1REQ Date: Fri, 14 Jan 2022 13:53:51 +0000 Message-ID: References: <1636366413-57455-1-git-send-email-tudor.cornea@gmail.com> <1637781854-74761-1-git-send-email-tudor.cornea@gmail.com> In-Reply-To: <1637781854-74761-1-git-send-email-tudor.cornea@gmail.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: dlp-version: 11.6.200.16 dlp-product: dlpe-windows dlp-reaction: no-action authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 0a729ed1-4526-445f-b1ab-08d9d7654c28 x-ms-traffictypediagnostic: CY4PR11MB1335:EE_ x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: lAU/9aYl7fg06Oyu5be1xRDo6Qtz5I95wjmdOyHDQ5bsDY9sPGi1WTIFizgq0SMLKjy1klpZFF6peZnheR8qXLYDTyj5ijjV7tqvr9EVZa4G3t+dD5qdQT9ULTzN/BLpujepZDvpGwUlC1nkX9yE85OTXlFiocNt9I9bOuekYNSW/et7LmLV00K//4dA2p8VEYKyXFuSarwqbVrrVe356pDOhm/7inQcEIfgWCOhcMW7aDiI5XbJ/mLZzJsxhRTSM2h+TR8NXmDGLg2g2dhphePPjCTZ+U/vSTA/zgsuBjB2KWKS2vNlXLyYROHka0gHSQhpitnjJZkm5Ux/V3a1irDGh/SjKZesSjF049DTiw/FVOMmkas14/FWz4t7iNzeJRfIZdOH72EFu6HDDltAjf3v5VjK+B8eB62gLTMGcv0VMrPpQFcgmMc19pPhdp2YMmB0jGGMX50FX1q02yldpMaHqEx1w3BNLMSkKgO4PEEONkRJxT0/+K0428GARaAbtexMvpkMT7/f5NETnckQTQXZ50I8efb8tBR43/pPKJhyOtGaMd4/nI6lVBiQKAEwGslS6hNe3HqDg16dyIlvBHN0/DfONtMv34XO50uzWhpIiy6gbiubZ1RuqNjSq8PtZ73x92dVXf+PuQ8mRAfGrpfRfZ19eG6IKb5+pWnFBVO4363bFuWKZaKphw8Xbn89cXN+PHEfViQ4DxAmEdix7gcYaijtEfo5JOtEmwHLDYBVt+7L8ozeFPr15k/iCIWULMGYjrAecq65d51s2/LvPDmACwaMKYRGDTK1Jg1aqYg= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR11MB3738.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(366004)(38070700005)(966005)(52536014)(7696005)(186003)(6506007)(86362001)(55016003)(508600001)(38100700002)(82960400001)(122000001)(5660300002)(53546011)(2906002)(26005)(33656002)(30864003)(9686003)(66556008)(316002)(83380400001)(66946007)(54906003)(8936002)(66476007)(66446008)(64756008)(110136005)(8676002)(71200400001)(76116006)(4326008); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?gnXn56u767tdwJUF6eQdO0yAcwTKcWgG/H8is2iKq5LLiADk9gEDqmQUsgyI?= =?us-ascii?Q?egVdgkARFSeXF+hl1t6FQlj5sS9DkoiSafKvYpZc2neVq+QT7dYJI3PnXVEE?= =?us-ascii?Q?67C5qkRm4Pz4uoJYS40YmVvB+mueF7FkCr+QrjsGcDGz5PEu6oOnMnV2OCs7?= =?us-ascii?Q?0ww/YBN9mcc5IAlBFxxIJt6uctAKX8and422S8W1TCZr+YmSfNjuf3BZkUro?= =?us-ascii?Q?Vtdr0AKuQsA5vs5/5Sc3bSF/y/2TB8BXyKreNK3sa3taF/NWNrn/9OCXqY25?= =?us-ascii?Q?H4WZdld4pv+fDlGmH8rtM5mk4tLmF2Ve23v2bRcSwBuJlvILi2JAOvQK3HAH?= =?us-ascii?Q?5YtyneYxhrSn2s2g52W3rEk1giexHQ4b564sCE1PLLhYYvKtEqjfIyWYWxH9?= =?us-ascii?Q?oLxZOzZmlheAD7dXr+DmDhTxSXK7trxoCkDkzaQmnz8410VU8fOT8fEA/BFy?= =?us-ascii?Q?AARS2Eqrzun/URndO/35rYzs7TP/uf9XYKY3KoCjkNHpngSNjWxqTsoruYKB?= =?us-ascii?Q?6jjSmIWZL4B/DzYGjLqdvIwJCKNONX93OPgUE3s5efOKBtV4/35jD8kbxyvz?= =?us-ascii?Q?YCzFUzAw0ATR1Nc2lWggGryxRdLdss8X+ZLyVeJXVIQieWIVUKlU6VyWZ/ig?= =?us-ascii?Q?M709uYf/Au9BuubRly1FDVu0PpMnsv3N6xsZkp9pef/82FS5Go8PvoThmSo1?= =?us-ascii?Q?WOS03HVBaVXLxpULlVqXf/0BnxM/iOJE6vNyphAQYucUVvhXYWWPIntqOpPA?= =?us-ascii?Q?tL+BCZkYd8RvR6NTESkyLMqG4UQ0WzwIz4uuKshRdb9q3G30cwigMSj0Pl7N?= =?us-ascii?Q?1hNVWlt8t7SJ1rt2THRTEpNVcembm+HSMYMjR9paHqGMmCseNu0LN+3hnktN?= =?us-ascii?Q?9vL50WUdaX86qgN5tSmAeYGvLV2AvRFslcZrloKgJWMBpLlXKS1lEL1Szlig?= =?us-ascii?Q?fFzwYwGwSvtRwEslxS5JeitBnjkvSkqJ+ehYcng6jDbcbES9zxMt8nXBGTOE?= =?us-ascii?Q?8i0hyP8VJEyhvaNND5B6KYBXWHXin3ZRBVHe9ozQan216BTobpZdHSkYv/Uj?= =?us-ascii?Q?GSGwP9jZLI4Zm8NqEBDO1xe2XCDXO4pjsafSl9mGK7MY0/SXe55wLVse4oJl?= =?us-ascii?Q?XpUKyaYx2CmTyV9CLUEfAJgU3wxIMAVkJHU408aFAG4DtOKbwNWKm871TNDW?= =?us-ascii?Q?ldWs3CzLxCmdbvL268b/MYAqCbuptZ0A1iiHLkO6GNO7TJZrta6ebTCViaJE?= =?us-ascii?Q?f0mkcOR9vCYaOR3xJWCOuAWXoeE3XaHqBZBOtABKpvijwn3kr9rMwSEyMZPP?= =?us-ascii?Q?c6z+bbwy/vye0wmWUAlJvpgIVKixIV/hSuxvSkAfMrO6r0IrYODDaAvTReTw?= =?us-ascii?Q?+55v8Cu1P6RlzI6AB3Mes4bwA0JuwhOHLAcaoW3BsDpSeUPeDS70w0q7KpSW?= =?us-ascii?Q?MlUSTvzDqxmq3XQix4eNx51sqIOJt0wM/eoNZ7qAYqzvyfhyKIwWS+ORNuLg?= =?us-ascii?Q?urE295IHNITzapyV2Suppu2BeCeqUVHHc7vpLrRt2pxfNN/s2V1/qc6iCvOs?= =?us-ascii?Q?9iBC/dyZ7KOfqLGXVug8DHjXOG/nNpo6kAmotBjAEtPUbr3eJKShRJQVtMO+?= =?us-ascii?Q?aqKRYBZt9sH/NmAT8z80B35G5C9S45p2zJ5ZfmdMGuxncV8k4jQHniBczt9G?= =?us-ascii?Q?HaGQ7Q=3D=3D?= Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3738.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0a729ed1-4526-445f-b1ab-08d9d7654c28 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Jan 2022 13:53:51.7789 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: kLVMEqgoCDQtrHTUJnlZ70ZazYN8f8on2+0QDVnEQHLoWxeVvNQ0++svk6VXUY1Vee5V20uOXFFLluduwHxzBOGQmGJbMQkIpY7wNk6XnFU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR11MB1335 X-OriginatorOrg: intel.com Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > -----Original Message----- > From: Tudor Cornea > Sent: Wednesday, November 24, 2021 7:24 PM > To: Yigit, Ferruh > Cc: thomas@monjalon.net; stephen@networkplumber.org; Zhang, Helin > ; dev@dpdk.org; Tudor Cornea > > Subject: [PATCH v4] kni: allow configuring the kni thread granularity > = > The Kni kthreads seem to be re-scheduled at a granularity of roughly > 1 millisecond right now, which seems to be insufficient for performing te= sts > involving a lot of control plane traffic. > = > Even if KNI_KTHREAD_RESCHEDULE_INTERVAL is set to 5 microseconds, it > seems that the existing code cannot reschedule at the desired granularily, > due to precision constraints of schedule_timeout_interruptible(). > = > In our use case, we leverage the Linux Kernel for control plane, and it i= s not > uncommon to have 60K - 100K pps for some signaling protocols. > = > Since we are not in atomic context, the usleep_range() function seems to = be > more appropriate for being able to introduce smaller controlled delays, i= n the > range of 5-10 microseconds. Upon reading the existing code, it would seem > that this was the original intent. Adding sub-millisecond delays, seems > unfeasible with a call to schedule_timeout_interruptible(). > = > KNI_KTHREAD_RESCHEDULE_INTERVAL 5 /* us */ > schedule_timeout_interruptible( > usecs_to_jiffies(KNI_KTHREAD_RESCHEDULE_INTERVAL)); > = > Below, we attempted a brief comparison between the existing > implementation, which uses schedule_timeout_interruptible() and > usleep_range(). > = > We attempt to measure the CPU usage, and RTT between two Kni interfaces, > which are created on top of vmxnet3 adapters, connected by a vSwitch. > = > insmod rte_kni.ko kthread_mode=3Dsingle carrier=3Don > = > schedule_timeout_interruptible(usecs_to_jiffies(5)) > kni_single CPU Usage: 2-4 % > [root@localhost ~]# ping 1.1.1.2 -I eth1 PING 1.1.1.2 (1.1.1.2) from 1.1.= 1.1 > eth1: 56(84) bytes of data. > 64 bytes from 1.1.1.2: icmp_seq=3D1 ttl=3D64 time=3D2.70 ms > 64 bytes from 1.1.1.2: icmp_seq=3D2 ttl=3D64 time=3D1.00 ms > 64 bytes from 1.1.1.2: icmp_seq=3D3 ttl=3D64 time=3D1.99 ms > 64 bytes from 1.1.1.2: icmp_seq=3D4 ttl=3D64 time=3D0.985 ms > 64 bytes from 1.1.1.2: icmp_seq=3D5 ttl=3D64 time=3D1.00 ms > = > usleep_range(5, 10) > kni_single CPU usage: 50% > 64 bytes from 1.1.1.2: icmp_seq=3D1 ttl=3D64 time=3D0.338 ms > 64 bytes from 1.1.1.2: icmp_seq=3D2 ttl=3D64 time=3D0.150 ms > 64 bytes from 1.1.1.2: icmp_seq=3D3 ttl=3D64 time=3D0.123 ms > 64 bytes from 1.1.1.2: icmp_seq=3D4 ttl=3D64 time=3D0.139 ms > 64 bytes from 1.1.1.2: icmp_seq=3D5 ttl=3D64 time=3D0.159 ms > = > usleep_range(20, 50) > kni_single CPU usage: 24% > 64 bytes from 1.1.1.2: icmp_seq=3D1 ttl=3D64 time=3D0.202 ms > 64 bytes from 1.1.1.2: icmp_seq=3D2 ttl=3D64 time=3D0.170 ms > 64 bytes from 1.1.1.2: icmp_seq=3D3 ttl=3D64 time=3D0.171 ms > 64 bytes from 1.1.1.2: icmp_seq=3D4 ttl=3D64 time=3D0.248 ms > 64 bytes from 1.1.1.2: icmp_seq=3D5 ttl=3D64 time=3D0.185 ms > = > usleep_range(50, 100) > kni_single CPU usage: 13% > 64 bytes from 1.1.1.2: icmp_seq=3D1 ttl=3D64 time=3D0.537 ms > 64 bytes from 1.1.1.2: icmp_seq=3D2 ttl=3D64 time=3D0.257 ms > 64 bytes from 1.1.1.2: icmp_seq=3D3 ttl=3D64 time=3D0.231 ms > 64 bytes from 1.1.1.2: icmp_seq=3D4 ttl=3D64 time=3D0.143 ms > 64 bytes from 1.1.1.2: icmp_seq=3D5 ttl=3D64 time=3D0.200 ms > = > usleep_range(100, 200) > kni_single CPU usage: 7% > 64 bytes from 1.1.1.2: icmp_seq=3D1 ttl=3D64 time=3D0.716 ms > 64 bytes from 1.1.1.2: icmp_seq=3D2 ttl=3D64 time=3D0.167 ms > 64 bytes from 1.1.1.2: icmp_seq=3D3 ttl=3D64 time=3D0.459 ms > 64 bytes from 1.1.1.2: icmp_seq=3D4 ttl=3D64 time=3D0.455 ms > 64 bytes from 1.1.1.2: icmp_seq=3D5 ttl=3D64 time=3D0.252 ms > = > usleep_range(1000, 1100) > kni_single CPU usage: 2% > 64 bytes from 1.1.1.2: icmp_seq=3D1 ttl=3D64 time=3D2.22 ms > 64 bytes from 1.1.1.2: icmp_seq=3D2 ttl=3D64 time=3D1.17 ms > 64 bytes from 1.1.1.2: icmp_seq=3D3 ttl=3D64 time=3D1.17 ms > 64 bytes from 1.1.1.2: icmp_seq=3D4 ttl=3D64 time=3D1.17 ms > 64 bytes from 1.1.1.2: icmp_seq=3D5 ttl=3D64 time=3D1.15 ms > = > Upon testing, usleep_range(1000, 1100) seems roughly equivalent in latency > and cpu usage to the variant with schedule_timeout_interruptible(), while > usleep_range(100, 200) seems to give a decent tradeoff between latency > and cpu usage, while allowing users to tweak the limits for improved > precision if they have such use cases. > = > Disabling RTE_KNI_PREEMPT_DEFAULT, interestingly seems to lead to a > softlockup on my kernel. > = > Kernel panic - not syncing: softlockup: hung tasks > CPU: 0 PID: 1226 Comm: kni_single Tainted: G W O 3.10 #1 > [] dump_stack+0x19/0x1b [] > panic+0xcd/0x1e0 [] watchdog_timer_fn+0x160/0x160 > [] __run_hrtimer.isra.4+0x42/0xd0 [] > hrtimer_interrupt+0xe7/0x1f0 [] > smp_apic_timer_interrupt+0x67/0xa0 > [] apic_timer_interrupt+0x6d/0x80 > = > This patch also attempts to remove this option. > = > References: > [1] https://www.kernel.org/doc/Documentation/timers/timers-howto.txt > = > Signed-off-by: Tudor Cornea > = > --- > v4: > * Removed RTE_KNI_PREEMPT_DEFAULT configuration option > v3: > * Fixed unwrapped commit description warning > * Changed from hrtimers to Linux High Precision Timers in docs > * Added two tabs at the beginning of the new params description. > Stephen correctly pointed out that the descriptions of the parameters > for the Kni module are nonstandard w.r.t existing kernel code. > I was thinking to preserve compatibility with the existing parameters > of the Kni module for the moment, while an additional clean-up patch > could format the descriptions to be closer to the kernel standard. > v2: > * Fixed some spelling errors > --- > config/rte_config.h | 3 --- > doc/guides/prog_guide/kernel_nic_interface.rst | 33 > +++++++++++++++++++++++++ > kernel/linux/kni/kni_dev.h | 2 +- > kernel/linux/kni/kni_misc.c | 34 ++++++++++++++++++++= ------ > 4 files changed, 60 insertions(+), 12 deletions(-) > = > diff --git a/config/rte_config.h b/config/rte_config.h index cab4390..91d= 96ee > 100644 > --- a/config/rte_config.h > +++ b/config/rte_config.h > @@ -95,9 +95,6 @@ > #define RTE_SCHED_PORT_N_GRINDERS 8 > #undef RTE_SCHED_VECTOR > = > -/* KNI defines */ > -#define RTE_KNI_PREEMPT_DEFAULT 1 > - > /* rte_graph defines */ > #define RTE_GRAPH_BURST_SIZE 256 > #define RTE_LIBRTE_GRAPH_STATS 1 > diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst > b/doc/guides/prog_guide/kernel_nic_interface.rst > index 1ce03ec..fce3667 100644 > --- a/doc/guides/prog_guide/kernel_nic_interface.rst > +++ b/doc/guides/prog_guide/kernel_nic_interface.rst > @@ -56,6 +56,10 @@ can be specified when the module is loaded to control > its behavior: > off Interfaces will be created with carrier state = set to off. > on Interfaces will be created with carrier state = set to on. > (charp) > + parm: min_scheduling_interval: "Kni thread min scheduling = interval > (default=3D100 microseconds): > + (long) > + parm: max_scheduling_interval: "Kni thread max scheduling = interval > (default=3D200 microseconds): > + (long) > = > Loading the ``rte_kni`` kernel module without any optional parameters is > the typical way a DPDK application gets packets into and out of the kernel > @@ -174,6 +178,35 @@ To set the default carrier state to *off*: > If the ``carrier`` parameter is not specified, the default carrier state= of KNI > interfaces will be set to *off*. > = > +KNI Kthread Scheduling > +~~~~~~~~~~~~~~~~~~~~~~ > + > +The ``min_scheduling_interval`` and ``max_scheduling_interval`` > +parameters control the rescheduling interval of the KNI kthreads. > + > +This might be useful if we have use cases in which we require improved > +latency or performance for control plane traffic. > + > +The implementation is backed by Linux High Precision Timers, and uses > ``usleep_range``. > +Hence, it will have the same granularity constraints as this Linux subsy= stem. > + > +For Linux High Precision Timers, you can check the following resource: > +`Kernel Timers > +`_ > + > +To set the ``min_scheduling_interval`` to a value of 100 microseconds: > + > +.. code-block:: console > + > + # insmod /kernel/linux/kni/rte_kni.ko > + min_scheduling_interval=3D100 > + > +To set the ``max_scheduling_interval`` to a value of 200 microseconds: > + > +.. code-block:: console > + > + # insmod /kernel/linux/kni/rte_kni.ko > + max_scheduling_interval=3D200 > + > +If the ``min_scheduling_interval`` and ``max_scheduling_interval`` > +parameters are not specified, the default interval limits will be set to= *100* > and *200* respectively. > + > KNI Creation and Deletion > ------------------------- > = > diff --git a/kernel/linux/kni/kni_dev.h b/kernel/linux/kni/kni_dev.h index > c15da311..bb4d891 100644 > --- a/kernel/linux/kni/kni_dev.h > +++ b/kernel/linux/kni/kni_dev.h > @@ -27,7 +27,7 @@ > #include > = > #include > -#define KNI_KTHREAD_RESCHEDULE_INTERVAL 5 /* us */ > +#define KNI_KTHREAD_MAX_RESCHEDULE_INTERVAL 1000000 /* us */ > = > #define MBUF_BURST_SZ 32 > = > diff --git a/kernel/linux/kni/kni_misc.c b/kernel/linux/kni/kni_misc.c in= dex > f4944e1..23132bb 100644 > --- a/kernel/linux/kni/kni_misc.c > +++ b/kernel/linux/kni/kni_misc.c > @@ -41,6 +41,10 @@ static uint32_t multiple_kthread_on; static char > *carrier; uint32_t kni_dflt_carrier; > = > +/* Kni thread scheduling interval */ > +static long min_scheduling_interval =3D 100; /* us */ static long > +max_scheduling_interval =3D 200; /* us */ > + > #define KNI_DEV_IN_USE_BIT_NUM 0 /* Bit number for device in use */ > = > static int kni_net_id; > @@ -128,11 +132,8 @@ kni_thread_single(void *data) > } > } > up_read(&knet->kni_list_lock); > -#ifdef RTE_KNI_PREEMPT_DEFAULT > /* reschedule out for a while */ > - schedule_timeout_interruptible( > - > usecs_to_jiffies(KNI_KTHREAD_RESCHEDULE_INTERVAL)); > -#endif > + usleep_range(min_scheduling_interval, > max_scheduling_interval); > } > = > return 0; > @@ -149,10 +150,7 @@ kni_thread_multiple(void *param) > kni_net_rx(dev); > kni_net_poll_resp(dev); > } > -#ifdef RTE_KNI_PREEMPT_DEFAULT > - schedule_timeout_interruptible( > - > usecs_to_jiffies(KNI_KTHREAD_RESCHEDULE_INTERVAL)); > -#endif > + usleep_range(min_scheduling_interval, > max_scheduling_interval); > } > = > return 0; > @@ -590,6 +588,14 @@ kni_init(void) > else > pr_debug("Default carrier state set to on.\n"); > = > + if (min_scheduling_interval < 0 || max_scheduling_interval < 0 || > + min_scheduling_interval > > KNI_KTHREAD_MAX_RESCHEDULE_INTERVAL || > + max_scheduling_interval > > KNI_KTHREAD_MAX_RESCHEDULE_INTERVAL || > + min_scheduling_interval >=3D max_scheduling_interval) { > + pr_err("Invalid parameters for scheduling interval\n"); > + return -EINVAL; > + } > + > #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS > rc =3D register_pernet_subsys(&kni_net_ops); > #else > @@ -656,3 +662,15 @@ MODULE_PARM_DESC(carrier, > "\t\ton Interfaces will be created with carrier state set to on.\n" > "\t\t" > ); > + > +module_param(min_scheduling_interval, long, 0644); > +MODULE_PARM_DESC(min_scheduling_interval, > +"\t\tKni thread min scheduling interval (default=3D100 microseconds):\n" > +"\t\t" > +); > + > +module_param(max_scheduling_interval, long, 0644); > +MODULE_PARM_DESC(max_scheduling_interval, > +"\t\tKni thread max scheduling interval (default=3D200 microseconds):\n" > +"\t\t" > +); This patch looks good to me. Tested with /kernel/linux/kni/rte_kni.ko min_scheduling_interval= =3D20 max_scheduling_interval=3D50 Results: KNI Perf before patch was added: 64 bytes from 5.5.5.6: icmp_seq=3D1 ttl=3D64 time=3D4.79 ms 64 bytes from 5.5.5.6: icmp_seq=3D2 ttl=3D64 time=3D2.97 ms 64 bytes from 5.5.5.6: icmp_seq=3D3 ttl=3D64 time=3D1.90 ms 64 bytes from 5.5.5.6: icmp_seq=3D4 ttl=3D64 time=3D7.94 ms 64 bytes from 5.5.5.6: icmp_seq=3D5 ttl=3D64 time=3D6.85 ms KNI Perf after patch was added (min_scheduling_interval=3D20 max_scheduling= _interval=3D50): 64 bytes from 5.5.5.6: icmp_seq=3D1 ttl=3D64 time=3D0.106 ms 64 bytes from 5.5.5.6: icmp_seq=3D2 ttl=3D64 time=3D0.055 ms 64 bytes from 5.5.5.6: icmp_seq=3D3 ttl=3D64 time=3D0.059 ms 64 bytes from 5.5.5.6: icmp_seq=3D4 ttl=3D64 time=3D0.056 ms 64 bytes from 5.5.5.6: icmp_seq=3D5 ttl=3D64 time=3D0.061 ms Acked-by: Padraig Connolly > -- > 2.7.4 -------------------------------------------------------------- Intel Research and Development Ireland Limited Registered in Ireland Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 This e-mail and any attachments may contain confidential material for the s= ole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact = the sender and delete all copies.