From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-by2nam03on0059.outbound.protection.outlook.com [104.47.42.59]) by dpdk.org (Postfix) with ESMTP id 89246FFA for ; Tue, 20 Sep 2016 20:17:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=harmonic.onmicrosoft.com; s=selector1-harmonicinc-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=gHYA3b2CXCePPd+i3fcsAdxSQ6dCVAK1utcZLJge610=; b=UDY/GrQbla9qQvss2tdxFqSWVqKlRi7PuUDV7zS1Jb1TiJM+tz336+aks9gHe0STtEykmJ7OXoO3e7N9uqvSfV4JAAz28J3yw9//RVklEbr1atTOrU/N6jFcgtliRjtWNwGJyCDfpOdFNBLFjQZ1IOXC4FyLlV5jVZi1kod9nvk= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Vladyslav.Buslov@harmonicinc.com; Received: from kms.harmonicinc.com (95.67.66.62) by MWHPR11MB1360.namprd11.prod.outlook.com (10.169.235.22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.629.8; Tue, 20 Sep 2016 18:17:00 +0000 From: Vladyslav Buslov To: CC: Date: Tue, 20 Sep 2016 21:16:37 +0300 Message-ID: <20160920181637.26778-1-vladyslav.buslov@harmonicinc.com> X-Mailer: git-send-email 2.8.3 In-Reply-To: <20160910135016.6468-2-vladyslav.buslov@harmonicinc.com> References: <20160910135016.6468-2-vladyslav.buslov@harmonicinc.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [95.67.66.62] X-ClientProxiedBy: AMSPR02CA0050.eurprd02.prod.outlook.com (10.242.225.178) To MWHPR11MB1360.namprd11.prod.outlook.com (10.169.235.22) X-MS-Office365-Filtering-Correlation-Id: 4b4e011a-5e8d-4d1c-810a-08d3e1825150 X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 2:zFQ7TmACind3RhCUZ6OAnjxabQ7uD8ynAySVOxAmGdBfgXEUXydR4OCZMkHeZ2gpIeS1D2YoKSDPtTVcatH3wUyMPO+mv7gxpoUdwGAOHktIKQXRsMRgOicW0QAxX5qiAnpa+EG/4oZsjMiHOBvKeWl0q3dyT01wGFG/c0TH2xGIIsYDRQJnZGp90TDbMncD; 3:tnCUSf7DOAY2CEfez+rvBgpTr3u9jE3bNANXrBGtYqbkdaqgfCLdIOmwOqBWEedJbHnZP7UMriqlAA9q+9S2fK7krN+MFgw6fGewz0dh1elhDz+8wdTOy8r1X9oG4tvv X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:MWHPR11MB1360; X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 25:zQutKa8UAXb04QGTa0hFjsDNV4trRjnNo1ocMrEpxYYJci0iayJdhSC8CHkdq8+D2CJO6D7NnJmh8hv0QQqTBlqJ1Sga2XZkhs42vwm9+zcn3yHFG40iCiCsZr1egNK/F77IJQGCcQGiZhAaCG4FbpKj2Mqv0kFGHDIZkZLg9shx1nfVZ+1VFzG7A0jI/VBrc38bSrtIYdDekDiHDxBJO8pt/yv8HoZ3r61AKXOMF5OlcO11h5dhKXGj9DrvId6X4MjqObTVGEULBpENm3QaM3XzZE+T/ZB/R8nc5VPSiGYvBtKQHMhlGUTwoMsFt8xjS+b8/02HI7q93DLK5SnIdQRgb2/dAa7AEHaDuDI/Y7U5d6/r+dvOv185V/SX32sLwWaGW0hsrNzF8Wyo/oZZM9M33sDabVOxf8s5Trtf6myMusLCd1DwZBaF5v6M/NDD9Mwz/3WIjPuhixQL1DJS5bu50vKDp00e4iF228R6r6cgIl3XDxxA/jkS+1YbVp+KxN/KKxnS3s/D/LoQA7WYTngtNbSfg8aQ1pVfu2b86p04pIg0VzNoX7QR0lduy7udvz2yPFIbCWT2ZBn103ynsbGhTbtHrZ3DzUfuabyxOWxHY2PdsYg4TeUCJlOYAV7WeoVtph+Z3sDUPriXFenlCt+kW2p3T0V3bQvASMm3azF+RKJJGZinDtZopprPfx/y; 31:31ihWO7CvaKLtPhL4CEHWa4Tx9tjoN500kt4o9Gqf2zfccGZrsB9fTAieNdzgoZEyZTYMG3ukfKiCUu8uMWLJlcomfQDzTVojq1k2hWdRa/x+cyd6/91J6SE/LSE/P4UeDLIeabxfWdaUYh1aFanUc6A3vyDsz8gVwY/biak3doia2aH8+IU0xxOCyw3abKokhroCbJwTSv6NrLemIpYmP0S85sJLxIinjg+tn/136Y= X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 20:hzA4Lvv6FkOh27B79IP+KENqg3Lq87zKqgCuUUQA/QEaylgqqTYUEX/rlA0PAkeUVyo5QZkydmSrLxaNjwTWXaEdWx388lgTyjS9DvF/Bt5fG75G1g2+aOjeIhhHUWdHR1gw/C9y2MRC47NuSK/qC0MKYCIcAYG9iMqZZhROxyztw+koW5Xv/JKCqFuRXRUQAW+59xQlM2UsLDVzLnVMh6iVjQpZB6K9xr616RYnpbfLXGH/MLD5l4ncTpnO9HcgKacGaOqmewmt8ODyC4zjEiMvc7XSTdUIUGhnzZmCbbb7krd/90CEKAVVODRlUtc6vmgi6O6BeqHuCLi/b9yNtXPK5sAe/nNI5Z+ioXETo+DZE/LPHpu8P4o9jjmpUj1zqbUISjYL2ndKFrj9LmfaTN0Esb9H9eAShpDKY4kfI+71wNtLDqxhUwtEsTgBaW/n73TkHYp4fuYdqJBJ7T94sb1DkXJjscXK2BRnixqpCsmHx3pDDJpbaqvncBjNq/Ci X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(209352067349851); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040176)(601004)(2401047)(8121501046)(5005006)(3002001)(10201501046)(6055026); SRVR:MWHPR11MB1360; BCL:0; PCL:0; RULEID:; SRVR:MWHPR11MB1360; X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 4:Yd3juGG7Na0SD/S975341Ph7SpeGfVVl5WpcA6LapMDxviKTYPboOKF/PHLdNcttD+H1qVKfsHFwy3+u9qOMZM9FxAHUnT24VYgHSsTQ/j0uvJGXUEynnpGXuUIjuqzcAWeQQKt6WAXsydmqml0pUUMQmf/3zdDmhd3MAnoXgUPlh59I7h7kasccQdQZo0CAFb3nBAdG81/auFjdPN9sVTOtNyyN6oduE0Z/PCuDlYKUHjc3UZcrwB/dKdSMzkAgwskJm6Dlje43cXbh2h5P7ccAG9VhwSfAWILSffiaZNLWDfT9mbNx6Tcc+Jkz6h6leHcmUcHX6Lr7la8CbXuWpfDtPRFqZbo7chrJkFYKTdQip0xFR2DWmzIFUlgYYKdPikNGTMH2KTLIez0rp6vEt3X4J1kVoCSUf8LEpuAl/TmWj3Qp8WnNMnx35Xfs4JvjdZRmsgW6Ytplv6dXHj5mqw== X-Forefront-PRVS: 0071BFA85B X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6009001)(7916002)(199003)(189002)(69596002)(106356001)(68736007)(77096005)(586003)(105586002)(36756003)(50226002)(66066001)(47776003)(3846002)(48376002)(4326007)(1076002)(5003940100001)(50466002)(6116002)(92566002)(81166006)(110136003)(97736004)(8676002)(81156014)(42186005)(101416001)(2950100001)(2351001)(229853001)(305945005)(5660300001)(19580405001)(86362001)(76176999)(19580395003)(7846002)(53416004)(189998001)(2906002)(7736002)(50986999)(33646002); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR11MB1360; H:kms.harmonicinc.com; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: harmonicinc.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR11MB1360; 23:WDtqlKEI2XmDQyeDa8hHq/+Uk8F9r4tWEzMcs3B+n?= =?us-ascii?Q?eqbDJGyaN78Fkk1eQf8IYja/yVinDmiTPzPtGrjDlIZFtd9AzANcsleaYkgR?= =?us-ascii?Q?ocIuvsAS8PhFa1CQNhCsLRxUHAlWuiE1PRONNCyH3C296XpYGQBQvm3w8QGX?= =?us-ascii?Q?UvVyIYWVXEyT/hfM0x5S7vhmpIkiRnqJFnOff9ihUJp1TC4SD1BiF9GOL5rA?= =?us-ascii?Q?lpDndlCiUVpGsJFAvLUgTDf4F9eBh78O8+OoZdPA3tiPsxUOQ2dw5U6fTMK0?= =?us-ascii?Q?f7v7eBIUokzT1vg8u4WT9H3M47rIchInqlznNr4DXFTGaWZSvoJe4MWvRhC0?= =?us-ascii?Q?OHhw8DCWQ7H2zxOPGGc8koLNsSoKmKO5U10qajH94NAZS4YaTtHCoeDxwzA+?= =?us-ascii?Q?CheII64Qp6ZbR9hVE6nF+lr5UNsperwNGAcZeB8CbdXQyOHuC/r0sIyiS1gB?= =?us-ascii?Q?tE75bem+6Gjl75vaFWAxrsZwG9ZqaKoyRdFRy04r5kmZeaA22qzGXLOYI+L1?= =?us-ascii?Q?rJf6FtiYdpAIOc9YTaTvy8GUBE/R6Ec0AeStdbsZAos+KSSH67Qc/yk6xIh2?= =?us-ascii?Q?OkJdxOQ7mnutq9xR01OKQ73CtXqaLTdzHC/2WsFy2SEe059Iav3OUJK7dvXW?= =?us-ascii?Q?mvv/UwP6nupSNQykcKHmV62AGYhGDqWDTCHqM3D/lgwLHfU4PFCzoVi//Hse?= =?us-ascii?Q?1mf5aG1mx1QxZ7Sn4agQn2RRspAvx9fmrqgV2DoGHYqtkIu5urIThWWzMXnq?= =?us-ascii?Q?jOYD8oLQ5pX8tXFgRR8t93VcxnmJGNuP+QndkI1/KO6G9doa3HOrZuRkIFp9?= =?us-ascii?Q?7MAZUDSjL61CPiC2jzeVSvXcrrYL4zHOXHSJO3sfFwq0T47nZdrjyoA1DDo2?= =?us-ascii?Q?2egYVXkroC1OUoqXqBOonlr5Y4Mt7ZBi4f2RzsmU96MhXtZb0Ml5fHg70Ngk?= =?us-ascii?Q?yrcqCp0hltk3Wb+1uMAcB4bEeEU/Dy7hpCIzioNo1r64GPdREWi2GqAdHVO6?= =?us-ascii?Q?bR5wrFrd4hZlz617xbBwFdNywLab6FC7zViSYyHk77Ag7aAcqhXLfEAKYRuR?= =?us-ascii?Q?qmwlwl6zjCI0SwFPxqPbyDVTEgKGPHY9tpCUcKa38pndAXiWq191wNIsM/+q?= =?us-ascii?Q?Mw6jacYNxTyAEoE47K6LfXfHUfmcVFjlYkRy2kaXQ3tWnBfvPoBoQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 6:IX0cYd25LQ0pSDxV4LpqxA4jfAGFFlT9P5qpOnlUj6qW/mL9QDVwlyUE5FSKhO6RE0CtLxsd7c57j47ZngkqOLnCsiiHSF0p6Ve13w4L1OTZosjeuEVoPh34ZLKL7W72JIvJKeESlqzAlnFBLirozdtskFw/9QUv4ciVeoh9V99DF8IiMecfUP/Wutt1IGxAczyec/D9YG93rf2W62tlPSq+BwWdaSGIFFttcLxaAeKk1Gaa1cra161njDjVQFU6MWUx8qc4pJ65CVGNHkzoAH3nHDmzRimtN0Sq+9TeAhUxSizbYZQWFIEPMqDYej7sP6+bypIoFCW4lJ+JIjHaSQ==; 5:JX43ClAUjy7YJ/ky6TmZEdQERzKxaq6rGjIbvv91KebRBsUDD4cTCcUU71NNpBDrhzpENT2x94sVJZ1WW3iiCVQrUTM6VoQdmXJyrqGY/qWCKrttbf4vOC5jFOu1tuXGRVF/SfzJk9PJA/OykAQEkg==; 24:B3KkwCDFS+dF+OqcrhKYwvfOO3WU2sisH4Aj+cqW7WcZDpsOMXZJsm9ttqwJH+1sMPCM5CBhg6AnckNKrrab5qPrece23RZZS3bUCS9UKBc=; 7:84h2UtvAcB3p8dYhkwNZ6XejwcSsbw8GI6paWIo4WQ42LglzcSw/4QKPTvVcbVMcx5Q2iDs/d8sbfIp1xaEx0vOEM/snjnlqYyhhjOr21ywFcqS4B5Aq0+5yN7KsrjQ+HdTKzlRwtALUoA56eh2YjbKGSAuGm1GtnCcv3GrzkfCf1+NNtIcQnRdvD8eik01hRYykCfqVD2yIc7AvjkWSW4SkIZ7uLdXQuGj/V7ITgM07IIeDv7cLTCroxHNbFv608EaBsyNjO6T9lk640clH6cB1GwuLbDTygN4vSkW85VDWqp7MalDJMjOYWhMP5cbA SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR11MB1360; 20:GCgSXfCUgOoqdR2ArYZMzhQ+fXN5uH89WZtg2dk9EX3VjfxiByZj7P/kvlMLHkV7b48Wy6fzJ93pZzti4HbuNeeDz5dIma9h3tG2hEcBXyWsRq0L2LKgpExuBt+58HBFZi8B+HkVV9tX1BZc1U/OPZpCpK9HfMuY3bre2x1JyGw= X-OriginatorOrg: harmonicinc.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2016 18:17:00.8134 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR11MB1360 Subject: [dpdk-dev] [PATCH] kni: add support for core_id param in single threaded mode X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Sep 2016 18:17:04 -0000 Allow binding KNI thread to specific core in single threaded mode by setting core_id and force_bind config parameters. Signed-off-by: Vladyslav Buslov --- doc/guides/prog_guide/kernel_nic_interface.rst | 3 + lib/librte_eal/linuxapp/kni/kni_misc.c | 101 ++++++++++++++++--------- 2 files changed, 67 insertions(+), 37 deletions(-) diff --git a/doc/guides/prog_guide/kernel_nic_interface.rst b/doc/guides/prog_guide/kernel_nic_interface.rst index fac1960..eb16e2e 100644 --- a/doc/guides/prog_guide/kernel_nic_interface.rst +++ b/doc/guides/prog_guide/kernel_nic_interface.rst @@ -102,6 +102,9 @@ Refer to rte_kni_common.h in the DPDK source code for more details. The physical addresses will be re-mapped into the kernel address space and stored in separate KNI contexts. +The affinity of kernel RX thread (both single and multi-threaded modes) is controlled by force_bind and +core_id config parameters. + The KNI interfaces can be deleted by a DPDK application dynamically after being created. Furthermore, all those KNI interfaces not deleted will be deleted on the release operation of the miscellaneous device (when the DPDK application is closed). diff --git a/lib/librte_eal/linuxapp/kni/kni_misc.c b/lib/librte_eal/linuxapp/kni/kni_misc.c index 59d15ca..056e13b 100644 --- a/lib/librte_eal/linuxapp/kni/kni_misc.c +++ b/lib/librte_eal/linuxapp/kni/kni_misc.c @@ -30,6 +30,7 @@ #include #include #include +#include #include #include #include @@ -100,6 +101,7 @@ static int kni_net_id; struct kni_net { unsigned long device_in_use; /* device in use flag */ + struct mutex kni_kthread_lock; struct task_struct *kni_kthread; struct rw_semaphore kni_list_lock; struct list_head kni_list_head; @@ -123,6 +125,9 @@ static int __net_init kni_init_net(struct net *net) /* Clear the bit of device in use */ clear_bit(KNI_DEV_IN_USE_BIT_NUM, &knet->device_in_use); + mutex_init(&knet->kni_kthread_lock); + knet->kni_kthread = NULL; + init_rwsem(&knet->kni_list_lock); INIT_LIST_HEAD(&knet->kni_list_head); @@ -139,9 +144,9 @@ static int __net_init kni_init_net(struct net *net) static void __net_exit kni_exit_net(struct net *net) { -#ifndef HAVE_SIMPLIFIED_PERNET_OPERATIONS struct kni_net *knet = net_generic(net, kni_net_id); - + mutex_destroy(&knet->kni_kthread_lock); +#ifndef HAVE_SIMPLIFIED_PERNET_OPERATIONS kfree(knet); #endif } @@ -167,6 +172,11 @@ kni_init(void) return -EINVAL; } + if (multiple_kthread_on == 0) + KNI_PRINT("Single kernel thread for all KNI devices\n"); + else + KNI_PRINT("Multiple kernel thread mode enabled\n"); + #ifdef HAVE_SIMPLIFIED_PERNET_OPERATIONS rc = register_pernet_subsys(&kni_net_ops); #else @@ -235,19 +245,6 @@ kni_open(struct inode *inode, struct file *file) if (test_and_set_bit(KNI_DEV_IN_USE_BIT_NUM, &knet->device_in_use)) return -EBUSY; - /* Create kernel thread for single mode */ - if (multiple_kthread_on == 0) { - KNI_PRINT("Single kernel thread for all KNI devices\n"); - /* Create kernel thread for RX */ - knet->kni_kthread = kthread_run(kni_thread_single, (void *)knet, - "kni_single"); - if (IS_ERR(knet->kni_kthread)) { - KNI_ERR("Unable to create kernel threaed\n"); - return PTR_ERR(knet->kni_kthread); - } - } else - KNI_PRINT("Multiple kernel thread mode enabled\n"); - file->private_data = get_net(net); KNI_PRINT("/dev/kni opened\n"); @@ -263,9 +260,13 @@ kni_release(struct inode *inode, struct file *file) /* Stop kernel thread for single mode */ if (multiple_kthread_on == 0) { + mutex_lock(&knet->kni_kthread_lock); /* Stop kernel thread */ - kthread_stop(knet->kni_kthread); - knet->kni_kthread = NULL; + if (knet->kni_kthread != NULL) { + kthread_stop(knet->kni_kthread); + knet->kni_kthread = NULL; + } + mutex_unlock(&knet->kni_kthread_lock); } down_write(&knet->kni_list_lock); @@ -390,6 +391,47 @@ kni_check_param(struct kni_dev *kni, struct rte_kni_device_info *dev) } static int +kni_run_thread(struct kni_net *knet, struct kni_dev *kni, uint8_t force_bind) +{ + /** + * Create a new kernel thread for multiple mode, set its core affinity, + * and finally wake it up. + */ + if (multiple_kthread_on) { + kni->pthread = kthread_create(kni_thread_multiple, + (void *)kni, "kni_%s", kni->name); + if (IS_ERR(kni->pthread)) { + kni_dev_remove(kni); + return -ECANCELED; + } + + if (force_bind) + kthread_bind(kni->pthread, kni->core_id); + wake_up_process(kni->pthread); + } else { + mutex_lock(&knet->kni_kthread_lock); + + if (knet->kni_kthread == NULL) { + knet->kni_kthread = kthread_create(kni_thread_single, + (void *)knet, "kni_single"); + if (IS_ERR(knet->kni_kthread)) { + mutex_unlock(&knet->kni_kthread_lock); + kni_dev_remove(kni); + return -ECANCELED; + } + + if (force_bind) + kthread_bind(knet->kni_kthread, kni->core_id); + wake_up_process(knet->kni_kthread); + } + + mutex_unlock(&knet->kni_kthread_lock); + } + + return 0; +} + +static int kni_ioctl_create(struct net *net, unsigned int ioctl_num, unsigned long ioctl_param) { @@ -415,11 +457,9 @@ kni_ioctl_create(struct net *net, } /** - * Check if the cpu core id is valid for binding, - * for multiple kernel thread mode. + * Check if the cpu core id is valid for binding. */ - if (multiple_kthread_on && dev_info.force_bind && - !cpu_online(dev_info.core_id)) { + if (dev_info.force_bind && !cpu_online(dev_info.core_id)) { KNI_ERR("cpu %u is not online\n", dev_info.core_id); return -EINVAL; } @@ -566,22 +606,9 @@ kni_ioctl_create(struct net *net, kni_vhost_init(kni); #endif - /** - * Create a new kernel thread for multiple mode, set its core affinity, - * and finally wake it up. - */ - if (multiple_kthread_on) { - kni->pthread = kthread_create(kni_thread_multiple, - (void *)kni, - "kni_%s", kni->name); - if (IS_ERR(kni->pthread)) { - kni_dev_remove(kni); - return -ECANCELED; - } - if (dev_info.force_bind) - kthread_bind(kni->pthread, kni->core_id); - wake_up_process(kni->pthread); - } + ret = kni_run_thread(knet, kni, dev_info.force_bind); + if (ret != 0) + return ret; down_write(&knet->kni_list_lock); list_add(&kni->list, &knet->kni_list_head); -- 2.8.3