From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from forward1m.mail.yandex.net (forward1m.mail.yandex.net [37.140.138.1]) by dpdk.org (Postfix) with ESMTP id 508735A1F for ; Wed, 21 Jan 2015 20:25:42 +0100 (CET) Received: from web4m.yandex.ru (web4m.yandex.ru [37.140.138.95]) by forward1m.mail.yandex.net (Yandex) with ESMTP id 973AE1221BAD for ; Wed, 21 Jan 2015 22:25:41 +0300 (MSK) Received: from 127.0.0.1 (localhost [127.0.0.1]) by web4m.yandex.ru (Yandex) with ESMTP id 160173161C4B; Wed, 21 Jan 2015 22:25:40 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.com; s=mail; t=1421868341; bh=RhzMG/6glBpazWtuubq1N3GOu7douDYym626KGXnAHg=; h=From:To:In-Reply-To:References:Subject:Date; b=jZkTzFILeI/2XtyszKaAdYaygtkS/4EBGMIgHi/64jnQj5lLFc4kVRaZWFC1iWk8M 75YixUX0E6qVfLN5CMN2QAVy+ijQr8apCTL2aQnJVIdTHP8BhNVOmRH421HTYz68oD b80nlKD59BlFtSh/P1OJJ8OurtCzJpyjQU9vC7fE= Received: by web4m.yandex.ru with HTTP; Wed, 21 Jan 2015 22:25:40 +0300 From: deco33000 Jog To: dev@dpdk.org In-Reply-To: <3621191421867764@web4m.yandex.ru> References: <2523541421848850@web15h.yandex.ru> <20150121163040.GA11720@bricha3-MOBL3> <3621191421867764@web4m.yandex.ru> MIME-Version: 1.0 Message-Id: <3643611421868340@web4m.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Wed, 21 Jan 2015 20:25:40 +0100 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=koi8-r Subject: Re: [dpdk-dev] DPDK - TX from lcore in packet distributor configuration X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Jan 2015 19:25:42 -0000 21.01.2015, 20:16, "deco33000 Jog" : > 21.01.2015, 17:43, "Bruce Richardson" : >> šOn Wed, Jan 21, 2015 at 03:00:50PM +0100, deco33000 Jog wrote: >>> ššHello, >>> >>> šš-- PROBLEM >>> ššI have a AF_PACKET socket which is in promiscuous mode to get all the NIC traffic and let my apps do the whole stuff. >>> >>> ššSo I have one receiver and need to communicate the packet to different threads/processes (lcore) so that they can process the rest of the packet (tcp/udp...) >>> >>> šš-- QUESTION >>> ššI read about the packet distributor architecture which seems to answer that need. >>> ššhttp://dpdk.org/doc/guides/prog_guide/packet_distrib_lib.html >>> >>> ššBUT i fear that it be slow at resending the packet to the distributor which may already be overloaded by inputs from the net. Why pass back the answer to the distributor if the lcore could send to the wire directly ? >>> >>> ššMy problem is going back to the distributor after the packet processing. i would a direct send to the tx ring. >>> >>> ššIs it possible ? How ? šBy passing the TX pointer to the lcore ? >> šIf you sent up the NIC so that it has multiple queues for each thread you can >> šhave each worker send directly to the NIC. Without multiple queues, they could >> šstill send directly, they will just have to use locking or some other access >> šmechanism to mediate access to the common TX queue. >> šThe other problem with this approach is that sending packets individually is >> šalmost always slower than sending them in bursts. To mitigate against this, >> šyou could look to buffer packets inside the workers before transmitting them >> šback out, but that could lead to packets being sent out of order - not sure if >> šthat is a problem for you or not. >> >> šThe reason the distributor sample app sends the packets back to the distributor >> šafter worker processing is to overcome these limitations. There is no additional >> šcross-core round trip involved in sending a packet back to the distributor along >> šwith the request, and having the distributor re-gather the packets ensures ordering >> šwithin flows/tags is maintained. Thereafter packets can be burst-sent out an >> šethernet port. >> >> šOut of interest, given you are using the AF_PACKET driver, what rate of packets >> šper second are you looking at? >> >> š/Bruce > > Bruce, thanks, > > 1) > I am looking for saturation of a 1 Gb/s link not a 10 Gb/s yet. > I want to feel at home before going higher. > > I work with raw packets for security purposes. > Do you think AF_PACKET is not good enough ? Do DPDK enables another (more efficient) solution (I may have missed it)? > > 2) > I tried Netmap and PF_RING too; i am currently seeking for my "weapon" :) > I want to use it in a socket context for a custom webserver. > > At first i thought the packet return would incur a cross core operation. > But the problem remains : > the distributor has to deal with Rx data and packets returned from the workers as well. Even if it is fast, I think I prefer to buffer + send directly form the worker to release the distributor from doing it. > i decided to avoid the Nagle algo as much as possible because i want the least latency as possible for my app. I modulate to to balance the trade-off though.