From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from prod-mail-xrelay07.akamai.com (prod-mail-xrelay07.akamai.com [72.246.2.115]) by dpdk.org (Postfix) with ESMTP id 38D5D595B for ; Fri, 14 Nov 2014 21:55:42 +0100 (CET) Received: from prod-mail-xrelay07.akamai.com (localhost.localdomain [127.0.0.1]) by postfix.imss70 (Postfix) with ESMTP id 61DCB475D3; Fri, 14 Nov 2014 21:05:46 +0000 (GMT) Received: from prod-mail-relay06.akamai.com (prod-mail-relay06.akamai.com [172.17.120.126]) by prod-mail-xrelay07.akamai.com (Postfix) with ESMTP id 5219347584; Fri, 14 Nov 2014 21:05:46 +0000 (GMT) Received: from email.msg.corp.akamai.com (ustx2ex-cas3.msg.corp.akamai.com [172.27.25.32]) by prod-mail-relay06.akamai.com (Postfix) with ESMTP id 3634F2029; Fri, 14 Nov 2014 21:05:46 +0000 (GMT) Received: from ustx2ex-cashub7.dfw01.corp.akamai.com (172.27.25.73) by ustx2ex-dag1mb3.msg.corp.akamai.com (172.27.27.103) with Microsoft SMTP Server (TLS) id 15.0.913.22; Fri, 14 Nov 2014 15:05:46 -0600 Received: from USMBX2.msg.corp.akamai.com ([169.254.1.74]) by ustx2ex-cashub7.dfw01.corp.akamai.com ([172.27.25.73]) with mapi; Fri, 14 Nov 2014 15:05:45 -0600 From: "Sanford, Robert" To: Thomas Monjalon , "dev@dpdk.org" Date: Fri, 14 Nov 2014 15:05:46 -0600 Thread-Topic: Enhance KNI DPDK-app-side to be Multi-Producer/Consumer Thread-Index: AdAATsFuOFSVQkZQSEOfgayUigXcoQ== Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.3.140616 acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: [dpdk-dev] Enhance KNI DPDK-app-side to be Multi-Producer/Consumer X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Nov 2014 20:55:42 -0000 Hello Thomas, I want to discuss a small enhancement to KNI that we developed. We wanted to send/receive mbufs between one KNI device and multiple cores, but we wanted to keep the changes simple. So, here were our requirements: 1. Don't use heavy synchronization when reading/writing the FIFOs in shared memory. 2. Don't make any one core (data or control plane) perform too much work shuffling mbufs to/from the FIFOs. 3. Don't use an intermediate RTE ring to drop off mbufs when another core is already accessing the same KNI FIFO. 4. Since (for our private use case) we don't need MP/MC on the kernel side, don't change the kernel KNI code at all. 5. Don't change the request/reply logic. It stays single-threaded on both sides. Here is what we came up with: 1. Borrow heavily from the librte_ring implementation. 2. In librte_kni structure rte_kni, supplement each rte_kni_fifo (tx, rx, alloc, and free q) with another private, corresponding structure that contains a head, tail, mask, and size field. 3. Create kni_fifo_put_mp function with merged logic of kni_fifo_put and __rte_ring_mp_do_enqueue. After we update the tail index (which is private to the DPDK-app), we update the FIFO write index (shared between app and kernel). 4. Create kni_fifo_get_mc function with merged logic of kni_fifo_get and __rte_ring_mc_do_dequeue. After we update the tail index, update the FIFO read index. 5. In rte_kni_tx_burst and kni_alloc_mbufs, call kni_fifo_put_mp instead of kni_fifo_put. 6. In rte_kni_rx_burst and kni_free_bufs, call kni_fifo_get_mc instead of kni_fifo_get. We believe this is a common use case, and thus would like to contribute it to dpdk.org. Questions/comments: 1. Are you interested for us to send the changes as an RFC? 2. Do you agree with this approach, or would it be better, say, to rewrite both sides of the interface to be more like librte_ring? 3. Perhaps we could improve upon kni_allocate_mbufs, as it blindly attempts to allocate and enqueue a constant number of mbufs. We have not really focused on the kernel =3D=3D> app data path, because we were only interested in app =3D=3D> kernel. -- Regards, Robert Sanford