From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0067.outbound.protection.outlook.com [104.47.37.67]) by dpdk.org (Postfix) with ESMTP id E57E72BAC for ; Sat, 10 Sep 2016 15:50:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=harmonic.onmicrosoft.com; s=selector1-harmonicinc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=54JAlGkvwuAFWos4bI1pXnJVI3DAZdfQjUtZBqqyYzw=; b=G6GOTEGslE2kviG9JdkM5qMvRBOqG3C0+MCgFjPbSt4bq/5RjaB+nkY4EqenejFcotWRdSKMoep36GI8U0nY02K7fCcG8DjVXIyCim/E7buhExAhgVGtVqDQd15JfGR3wzLmfrO/rLbKCGhEyz6DOp40NpdmvCUShZ4hL6IkBto= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Vladyslav.Buslov@harmonicinc.com; Received: from kms.harmonicinc.com (62.205.135.152) by MWHPR11MB1360.namprd11.prod.outlook.com (10.169.235.22) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.1.609.9; Sat, 10 Sep 2016 13:50:55 +0000 From: Vladyslav Buslov To: Date: Sat, 10 Sep 2016 16:50:16 +0300 Message-ID: <20160910135016.6468-2-vladyslav.buslov@harmonicinc.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20160910135016.6468-1-vladyslav.buslov@harmonicinc.com> References: <20160906112513.26090-2-vladyslav.buslov@harmonicinc.com> <20160910135016.6468-1-vladyslav.buslov@harmonicinc.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [62.205.135.152] X-ClientProxiedBy: AM4PR01CA0056.eurprd01.prod.exchangelabs.com (10.164.74.24) To MWHPR11MB1360.namprd11.prod.outlook.com (10.169.235.22) X-MS-Office365-Filtering-Correlation-Id: 6027524a-d0e1-40a5-a223-08d3d9817cd8 X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 2:yczn6zMIrkCJspcxbaKV33ZM+17nNEGs6Vp3b2zc9KcxHK3xKuI1cwIBFfmtMA4H95ZAWBfBaNnxvBZsoy5H0Vg8a9TmD8XOHBCutSkWloXwvSTrO7fjjBrdYnTnaFqDTT99z5JIs66DofwSHxRlID0K3Kr6MTdhPwvtDgHh5nFPz8qcmD8cKn1icbHDUdqg; 3:ivSdYmbWbOm/CGi1h2JUfJ9OiWmeabftH/pgZbgI3zZkVNofCTpstzrJcRxOa8Dps1itIDxGF0kqwGPkytVPlcSCgK45oIT+d0m2urJtfO1klzM46gouayhwRXraBLMU; 25:UZ1cyeIrSVn5yYV7XOpJ+KbrQNdw23qAbonQDW4J13Uv2AF4J6B9ghkz+AoFwPA79MrAfhbFXL1B7L3YCQuZyGNkgaBb6u8Kmi4jxPo/GjXnbK90GjfGfnWfzz5IsG0LUJCiqB6QoctrTsZM7onJuRNj7B7qaXkhiOCFBKkCbcudzTKQkByswwMprHveCj0nWSGOPw9f7aUsvBCU0DqogvnxVd1oPQrnzLq5fIh2+k0Bz0h2RfhxzC5hXHag8qT+IYy6Rxm5n3g6gFbR0ounJRZ+KWhbTFPuJyAg5lhZdxNEVUnUoJF6FgxWWpQHtj4yGA9E+xNKoIxrtXWjqcQb0Vy6Dvu4Oaux7RdlH9lBpnhpOWfVrFcMWBIgrB/73EyGiwOcDLKBSOKGhLnn9on4oPUEMnu6ijrJvBvXa0Mhjus= X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:MWHPR11MB1360; X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 31:Rxm+Af3Wxc8dmTYY1txD/wlBoSBJ5oLFYfsS6rUfV5JoqRUcaUZklXzqa/DF//yAQhlqE8t0NCDrR3xekOcpiTbL+T5t0jPoQvUxKA/PV8DFA9sfJc6cvkLBCM53bZU5+zz/r5lxtTkb1CNIqbrWLEsl8jXFtnD+2lCrdoiTynlKavIcrvcNKZ/2kIcCh++ICUVrt4Coi39stRGXU+vk6dYHX7UjmxBXoYoszgqeYd0=; 20:j/veu6aBsEPqmuza75s6Lt8H6y0AnH+s6b7tU8x3qRzzPHKzw3+Xd59TSPf+JJqBj1TrzeltNXRTUXPQxGtimDiJBRbryZMTffbo9Vz9SDbvNaT0Qwy9H3p7C1A6atkhtiAJ9JM+lcFx8TYipL+H9VGVdsUsspbUFnZX0X6hu6FTSWFgCEnmCOjx5uxGlV7np+l/4UeXiBn1jH+ve6XvFcw+/KMMqz4kDypFtSMDS4YbmbwVrUcZSl5JZiXTX2pUqc3FjUeDVNontktc/J7fnNttOC5V407veuVPR6KqmcvLVjPQoDjiVz9NS5Co3RGSkYBSsHnN52V6XzPhKf3/PSz41JSen/Gldm2WQWPcvte2bjY6LQyn/IWOjFMuAMLIXHtqGkvkg349Y2b3B4mjfqwLHyTZtMpdBWuPoJpMkZb0Y7gufHP1QbiKHDz4tiKGSR2rczmcGFsFnP1ZuXqFWWmGZuRICZiwQcue2JHvSlAbifw82roXCv/fikoRtuZG X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(209352067349851); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026); SRVR:MWHPR11MB1360; BCL:0; PCL:0; RULEID:; SRVR:MWHPR11MB1360; X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 4:2S8PUdnMerU2OsMXYZFB4D/IW3euVTAleBznWI2NgjpezEkhyXojaBKp39qodBkCl/+6Pv/NCb7c1cwjz67vVkNKr3wukonBwJuW56+dofhc9uMz5dKFvf5GQO2XXvAFF8tPIkyybFwcqpWIs40H5uX2cd+u9Q+9C8wZXkx5qNvxMXKFuSLBBBk0aOhOyaLeYNcJwhIzI4pkN6uIUl+JWQEnA83HmHURLaVED6x3t3suNwO5GaaXl3tS0FRDb9BNKmbnuHg5xvDwNhIrMBSGmDSlpG2y1D2sntK5BcTF9KkG9Xcc3cCRWF1uR6xoudlfOAeeo/YP1C/nZobRHgbALJFR31g9fwrQOGps52IdPa5ZuqhV6NPO5uGftpwH/R7YH97JZpn3OQOHRquQkwMHXJbhfUOulNE4Rfzrf0e5Zw9cQfZIJtP58OllTK2Nxz76crAgMG6RBhNrXpoqrOIRaw== X-Forefront-PRVS: 0061C35778 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(189002)(199003)(47776003)(42186005)(92566002)(81166006)(7736002)(97736004)(586003)(2906002)(1076002)(81156014)(50226002)(105586002)(68736007)(2351001)(7846002)(107886002)(106356001)(189998001)(53416004)(305945005)(19580395003)(5660300001)(101416001)(2950100001)(36756003)(48376002)(5003940100001)(50466002)(110136002)(66066001)(76176999)(6116002)(8676002)(50986999)(3846002)(86362001)(69596002)(19580405001)(77096005)(33646002)(450100001)(229853001); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR11MB1360; H:kms.harmonicinc.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: harmonicinc.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR11MB1360; 23:ULSju8+wXByxsPiwY9AA755/l01GSWxKKL7r+Leoz?= =?us-ascii?Q?ooni1xViYq3r+KTxjpPBbKiWKxpzj+Tvt8+qCNAUD//a5UsrPauVcvBWZvBA?= =?us-ascii?Q?ZAMj91WqWeY+NTON88/UOYbOzOzTrOPeZkNpuUj6jim8F+hxth6ebOibCrnM?= =?us-ascii?Q?e5ajJ377wICDYj7J0H3FfJnCrj/0xmFFIBNuDnlEQ6ea1HNLOeTsLikdo/5h?= =?us-ascii?Q?rZmweuQ0w4VFoA6XN/hsGp1q507nKFMPIKCK+d6nG2deDCfe3UWsnr5NkfiL?= =?us-ascii?Q?b7Uu/m2H2AOVsQUGZeAP8+kUpjliMTJc2ZNbrj/m9NO3rbgmjejQ9UAYGkkw?= =?us-ascii?Q?mLMiH2rIJeRtLoVfpm4DpxCtr8G7luJvL3wUgw/3/b8I/nTCzxAGPSi45gAu?= =?us-ascii?Q?3URI0mp8Ik36oI0X7fD/94yW72pEYhkkuZOl+tGC2u5Z5SG5Mu552BoqDyMs?= =?us-ascii?Q?iIAY3OAX3G1ssUA9PvAaABqydUHeuJlIAQjB0IOta+R/I3qlRvP+WmO+HQ8/?= =?us-ascii?Q?dKQb1WmKTW/sI4sE+ElXOq+N3MBK27lEqIl7fi/9oaCwcq3GbDij0fcSLOh2?= =?us-ascii?Q?+vK73sFYqUJ6IXiHSiofIltjfYDBBNwxi0ACG94aN+iuQ9kdy2b9K96G7U0y?= =?us-ascii?Q?IhQ8JVBTyTqAkJ6+BkGzNvFea3DWmpctPckuezZg1oQYdwuWujlJEvDJIRXK?= =?us-ascii?Q?2g6kaYOzUcZxb1Zka0yI1jLlMSZwHfZuvWJqjwv7bpdbjU2w7oAmWYWeUHhA?= =?us-ascii?Q?3R24LCBzC+zW4rykwyY+cOYMzCcABTNXwVdw4STzsnF8ISrqZrKrDN599i10?= =?us-ascii?Q?nGiEYrEqFkJfxFeluzVi4qTz2qyhdXxfh0IFYrYBqPo4Wo/ROVMD0Ba+k5F8?= =?us-ascii?Q?A0eN1APvM32bHd4qhYobJ++hVb8rkYvYt6js/0l23/3wrUbl3kxDx2VWNDjO?= =?us-ascii?Q?RlCir8xyzk3TRQt+OROdKPUi2+QdI3kvuX1lurIPMIICTJf7FqSHE+OiJvsC?= =?us-ascii?Q?aYoOVpNe0GaixvWaIGRs9UH8aIQv6f9lu6pSZGq6fnBydp12rUpC7iGJ2jst?= =?us-ascii?Q?eLL9fCoZjkti2y1vFGB3P0NROtY0T3mayRStw8aEBmiZaOaoR+XdgrikBeBr?= =?us-ascii?Q?f1gztGvMrWuS31xq9sWybWs8s/aTgrlBMNDa7YMV80rF6Q23K9gTCIY6psEg?= =?us-ascii?Q?DP3x23rCR3pYLM=3D?= X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 6:CbZltctRAkmi8VGOzD8pJe1e/MVj1nlARxQjmUaRPnQrvzZbpEzv5mI+YwY5fBxk5HoYSOVFobVV24ZdL/G8fKeHnfimD+Q7fPt9/rCew7LINkoX1YxtpQosADpH35ztgJhZFTDrjojm/o9Vbeg1XiChtexRmB4uDLmiyjBmJ6SJ7YsATaS0hvfM/qLa86GIw5Rc6VBMNj0ECYiAkrA2gBs4Xs/a+TvLzJxnavL5XPYtPPaYlyYcXWU4PmL68V67xI7jmAaZHpzAcz2z6jpEZop2/iUNveWzc0xB5zIkr22FUNTO+ctwV73XH+pwxEVNCtaTWydX2dHx53BR/fpyNw==; 5:W4vx5NKqXmf63Xo1ThrlN4QKSLc/2apZSPtW32Ts1m7LbSLjW8lKtligioaJSiouwPknNxtOvb7hkEh8lWMMHD2E504gHKUKHTWncCUNuGGPSbACp5CIN7hmSm+NfSO0+Bc4QeZPCvcbHPxiACoOpQ==; 24:p2KDU+YYk8xYvPbWUU49iVNPMqqLaxso7FqtVdJKy4NDObqLoJ4bPgLS3EJQN3XcKflegxX189jRZ+/S0XqIgAFDoZnAszAyNRB8rUtDOiM=; 7:Ma77LZrdkSPNLH2rVyZa5MDobCPyO+0Vsd/CRVPhpuns4eCSD9RNSb6qAVw5vHW0cvMD7gILlvxvTYKAJglUwmjr5qsQlhqlOq2VGxWQ9gujfikp8JiMjm/h7yu78AJdRDYenEEm1FkziU5ruG2JNcksXctefZ6N4zWZMsC69wiC6AgghlB84R8aIoF4m839eabE843X/n/+euGUOua5zP/Iqqozahql9SJ2dAsy/zh80f5QOgwX6r5AfVxkBTAw SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 20:vGkMWnmyPOqQr/+w2l8kkxyg78kpOPbTjVvCzqd/rypxpeJr92iPRpgzWL0/BYFeXMb05iJniW6X8YoKDkM/mGPq2N1bJ813gOo3u0DWgAr48ZzRhX7+AGPn1yOCpe4e70zsQd+h6gvOyH8qTWHQnx2eJAfHNLyztOvoOhI/IpQ= X-OriginatorOrg: harmonicinc.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Sep 2016 13:50:55.1937 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR11MB1360 Subject: [dpdk-dev] [PATCH v2 2/2] kni: add support for core_id param in single threaded mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Sep 2016 13:50:57 -0000 Allow binding KNI thread to specific core in single threaded mode by setting core_id and force_bind config parameters. Signed-off-by: Vladyslav Buslov --- v2: * Fixed formatting. * Refactored kthread create/bind functionality into separate function. * Moved thread mode print into kni_init. * Added short description to KNI Programmer's Gude doc. * Fixed outdated mbuf processing description in KNI Programmer's Gude doc. doc/guides/prog_guide/kernel_nic_interface.rst | 5 +- lib/librte_eal/linuxapp/kni/kni_misc.c | 72 +++++++++++++++++--------- 2 files changed, 51 insertions(+), 26 deletions(-) diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index fac1960..0fdc307 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -102,6 +102,9 @@ Refer to rte_kni_common.h in the DPDK source code for more details. The physical addresses will be re-mapped into the kernel address space and stored in separate KNI contexts. +The affinity of kernel RX thread (both single and multi-threaded modes) is controlled by force_bind and +core_id config parameters. + The KNI interfaces can be deleted by a DPDK application dynamically after being created. Furthermore, all those KNI interfaces not deleted will be deleted on the release operation of the miscellaneous device (when the DPDK application is closed). @@ -128,7 +131,7 @@ Use Case: Ingress On the DPDK RX side, the mbuf is allocated by the PMD in the RX thread context. This thread will enqueue the mbuf in the rx_q FIFO. The KNI thread will poll all KNI active devices for the rx_q. -If an mbuf is dequeued, it will be converted to a sk_buff and sent to the net stack via netif_rx(). +If an mbuf is dequeued, it will be converted to a sk_buff and sent to the net stack via netif_rx_ni(). The dequeued mbuf must be freed, so the same pointer is sent back in the free_q FIFO. The RX thread, in the same main loop, polls this FIFO and frees the mbuf after dequeuing it. diff --git a/lib/librte_eal/linuxapp/kni/kni_misc.c b/lib/librte_eal/linuxapp/kni/kni_misc.c index 5e7cf21..c79f5a8 100644 --- a/lib/librte_eal/linuxapp/kni/kni_misc.c +++ b/lib/librte_eal/linuxapp/kni/kni_misc.c @@ -172,6 +172,11 @@ kni_init(void) return -EINVAL; } + if (multiple_kthread_on == 0) + KNI_PRINT("Single kernel thread for all KNI devices\n"); + else + KNI_PRINT("Multiple kernel thread mode enabled\n"); + #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS rc = register_pernet_subsys(&kni_net_ops); #else @@ -240,12 +245,6 @@ kni_open(struct inode *inode, struct file *file) if (test_and_set_bit(KNI_DEV_IN_USE_BIT_NUM, &knet->device_in_use)) return -EBUSY; - /* Create kernel thread for single mode */ - if (multiple_kthread_on == 0) - KNI_PRINT("Single kernel thread for all KNI devices\n"); - else - KNI_PRINT("Multiple kernel thread mode enabled\n"); - file->private_data = get_net(net); KNI_PRINT("/dev/kni opened\n"); @@ -391,6 +390,32 @@ kni_check_param(struct kni_dev *kni, struct rte_kni_device_info *dev) return 0; } +__printf(5, 6) static struct task_struct * +kni_run_thread(int (*threadfn)(void *data), + void *data, + uint8_t force_bind, + unsigned core_id, + const char namefmt[], ...) +{ + struct task_struct *kni_thread = NULL; + char task_comm[TASK_COMM_LEN]; + va_list args; + + va_start(args, namefmt); + vsnprintf(task_comm, sizeof(task_comm), namefmt, args); + va_end(args); + + kni_thread = kthread_create(threadfn, data, task_comm); + if (IS_ERR(kni_thread)) + return NULL; + + if (force_bind) + kthread_bind(kni_thread, core_id); + wake_up_process(kni_thread); + + return kni_thread; +} + static int kni_ioctl_create(struct net *net, unsigned int ioctl_num, unsigned long ioctl_param) @@ -419,8 +444,7 @@ kni_ioctl_create(struct net *net, /** * Check if the cpu core id is valid for binding. */ - if (dev_info.force_bind && - !cpu_online(dev_info.core_id)) { + if (dev_info.force_bind && !cpu_online(dev_info.core_id)) { KNI_ERR("cpu %u is not online\n", dev_info.core_id); return -EINVAL; } @@ -572,31 +596,29 @@ kni_ioctl_create(struct net *net, * and finally wake it up. */ if (multiple_kthread_on) { - kni->pthread = kthread_create(kni_thread_multiple, - (void *)kni, - "kni_%s", kni->name); - if (IS_ERR(kni->pthread)) { + kni->pthread = kni_run_thread(kni_thread_multiple, + (void *)kni, + dev_info.force_bind, + kni->core_id, + "kni_%s", kni->name); + if (kni->pthread == NULL) { kni_dev_remove(kni); return -ECANCELED; } - if (dev_info.force_bind) - kthread_bind(kni->pthread, kni->core_id); - wake_up_process(kni->pthread); } else { mutex_lock(&knet->kni_kthread_lock); if (knet->kni_kthread == NULL) { - knet->kni_kthread = kthread_create(kni_thread_single, - (void *)knet, - "kni_single"); - if (IS_ERR(knet->kni_kthread)) { - kni_dev_remove(kni); - return -ECANCELED; - } - if (dev_info.force_bind) - kthread_bind(knet->kni_kthread, kni->core_id); - wake_up_process(knet->kni_kthread); + knet->kni_kthread = kni_run_thread(kni_thread_single, + (void *)knet, + dev_info.force_bind, + kni->core_id, + "kni_single"); } mutex_unlock(&knet->kni_kthread_lock); + if (knet->kni_kthread == NULL) { + kni_dev_remove(kni); + return -ECANCELED; + } } down_write(&knet->kni_list_lock); -- 2.8.3