Re: [dpdk-dev] tcpdump support in DPDK 2.3

[Matthew Hall <mhall@mhcomputing.net>]

FYI your last name comes in as a corrupt character for me. You might have to 
think about converting it from ISO 8859-1 / 8859-15 to UTF-8.

On Mon, Dec 14, 2015 at 10:57:10AM +0100, Morten B wrote:
> Check out the new "extcap" feature of Wireshark. It uses named pipes for the 
> packets, already mentioned by Stephen Hemminger.

I looked at it a bit. I wasn't 100% clear if there is a way to pass down the 
BPF expression for compilation and usage inside the DPDK application.

> Tcpdump is an open source application, so it should be possible to define an 
> efficient interface between DPDK and tcpdump, and implement it in both DPDK 
> and tcpdump. The same goes for libpcap.

Easier said than done. A whole ton of libpcap assumes it's talking to a very 
specific kernel interface, and the code is quite complicated.

> It possibly also has a secondary feature: passing a BPF program 
> from tcpdump/libpcap to DPDK, so packets can be filtered in DPDK and don't 
> need to be passed on to tcpdump/libpcap.

If we can figure out how to get this feature to work in extcap, I think that 
will be the winning solution by far.

> [A]dd a BPF library (librte_bpf) to DPDK, preferably with a compiler. The 
> application initially calls the library's BPF compiler function once with 
> the BPF program to compile it, and in the fast path the application calls a 
> library function that takes an mbuf and the compiled BPF program and returns 
> an integer value indicating how many bytes of the packet should be mirrored 
> by the capturing application. +1 to Matthew Hall for taking this direction!

Yes, performance wise I think this is the only way that will really work 100% 
of the time. Otherwise I think we end up in the very bad situation where the 
guy who tries to make a capture of a single flow for debugging on i40e ends up 
crashing his system or dropping all his traffic when the capture system 
unhelpfully redirects a storm of unfiltered traffic outside of DPDK to KNI or 
some pipe devices or another place it does not belong.

There is one complexity though... the list of BPF filters should probably be a 
linked list, where they get added and removed, or you can't do > 1 filter at a 
time. I know how to code some of this stuff but I only work on DPDK in my 
spare time so I don't have the cycles to do all of the work.

> The pcap file format contains a header in front of each packet, which is 
> extremely simple. But it has a timestamp (which uses 32 bit for tv_sec and 
> tv_usec in files), so it needs to be considered how to handle this 
> efficiently.

I already wrote some C code for generating the original pcap format files a 
while ago which I think could be donated. For the timestamps to work at 
highest efficiency we'd need to run an rte_timer every X microseconds that 
updates a global volatile copy of tv_sec and tv_usec.

Or make some code that calculates the offset of rte_rdtsc from 01 January 1970 
00:00:00 UTC and uses TSC value to generate the right tv_sec and tv_usec would 
also work fine.

Matthew.