Re: [dpdk-dev] DPDK - TX from lcore in packet distributor configuration

[Bruce Richardson <bruce.richardson@intel.com>]

On Wed, Jan 21, 2015 at 03:00:50PM +0100, deco33000 Jog wrote:
> Hello,
> 
> -- PROBLEM
> I have a AF_PACKET socket which is in promiscuous mode to get all the NIC traffic and let my apps do the whole stuff.
> 
> So I have one receiver and need to communicate the packet to different threads/processes (lcore) so that they can process the rest of the packet (tcp/udp...)
> 
> -- QUESTION
> I read about the packet distributor architecture which seems to answer that need.
> http://dpdk.org/doc/guides/prog_guide/packet_distrib_lib.html
> 
> BUT i fear that it be slow at resending the packet to the distributor which may already be overloaded by inputs from the net. Why pass back the answer to the distributor if the lcore could send to the wire directly ?
> 
> My problem is going back to the distributor after the packet processing. i would a direct send to the tx ring.
> 
> Is it possible ? How ?  By passing the TX pointer to the lcore ?

If you sent up the NIC so that it has multiple queues for each thread you can
have each worker send directly to the NIC. Without multiple queues, they could
still send directly, they will just have to use locking or some other access
mechanism to mediate access to the common TX queue.
The other problem with this approach is that sending packets individually is
almost always slower than sending them in bursts. To mitigate against this,
you could look to buffer packets inside the workers before transmitting them
back out, but that could lead to packets being sent out of order - not sure if
that is a problem for you or not.

The reason the distributor sample app sends the packets back to the distributor
after worker processing is to overcome these limitations. There is no additional
cross-core round trip involved in sending a packet back to the distributor along
with the request, and having the distributor re-gather the packets ensures ordering
within flows/tags is maintained. Thereafter packets can be burst-sent out an
ethernet port.

Out of interest, given you are using the AF_PACKET driver, what rate of packets
per second are you looking at? 

/Bruce