From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.mhcomputing.net (master.mhcomputing.net [74.208.228.170]) by dpdk.org (Postfix) with ESMTP id 0056D8E8D for ; Mon, 14 Dec 2015 19:29:34 +0100 (CET) Received: by mail.mhcomputing.net (Postfix, from userid 1000) id BD377419; Mon, 14 Dec 2015 13:29:31 -0500 (EST) Date: Mon, 14 Dec 2015 13:29:31 -0500 From: Matthew Hall To: Morten B Message-ID: <20151214182931.GA17279@mhcomputing.net> References: <98CBD80474FA8B44BF855DF32C47DC358AF758@smartserver.smartshare.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <98CBD80474FA8B44BF855DF32C47DC358AF758@smartserver.smartshare.dk> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: dev@dpdk.org Subject: Re: [dpdk-dev] tcpdump support in DPDK 2.3 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Dec 2015 18:29:34 -0000 FYI your last name comes in as a corrupt character for me. You might have to think about converting it from ISO 8859-1 / 8859-15 to UTF-8. On Mon, Dec 14, 2015 at 10:57:10AM +0100, Morten B wrote: > Check out the new "extcap" feature of Wireshark. It uses named pipes for the > packets, already mentioned by Stephen Hemminger. I looked at it a bit. I wasn't 100% clear if there is a way to pass down the BPF expression for compilation and usage inside the DPDK application. > Tcpdump is an open source application, so it should be possible to define an > efficient interface between DPDK and tcpdump, and implement it in both DPDK > and tcpdump. The same goes for libpcap. Easier said than done. A whole ton of libpcap assumes it's talking to a very specific kernel interface, and the code is quite complicated. > It possibly also has a secondary feature: passing a BPF program > from tcpdump/libpcap to DPDK, so packets can be filtered in DPDK and don't > need to be passed on to tcpdump/libpcap. If we can figure out how to get this feature to work in extcap, I think that will be the winning solution by far. > [A]dd a BPF library (librte_bpf) to DPDK, preferably with a compiler. The > application initially calls the library's BPF compiler function once with > the BPF program to compile it, and in the fast path the application calls a > library function that takes an mbuf and the compiled BPF program and returns > an integer value indicating how many bytes of the packet should be mirrored > by the capturing application. +1 to Matthew Hall for taking this direction! Yes, performance wise I think this is the only way that will really work 100% of the time. Otherwise I think we end up in the very bad situation where the guy who tries to make a capture of a single flow for debugging on i40e ends up crashing his system or dropping all his traffic when the capture system unhelpfully redirects a storm of unfiltered traffic outside of DPDK to KNI or some pipe devices or another place it does not belong. There is one complexity though... the list of BPF filters should probably be a linked list, where they get added and removed, or you can't do > 1 filter at a time. I know how to code some of this stuff but I only work on DPDK in my spare time so I don't have the cycles to do all of the work. > The pcap file format contains a header in front of each packet, which is > extremely simple. But it has a timestamp (which uses 32 bit for tv_sec and > tv_usec in files), so it needs to be considered how to handle this > efficiently. I already wrote some C code for generating the original pcap format files a while ago which I think could be donated. For the timestamps to work at highest efficiency we'd need to run an rte_timer every X microseconds that updates a global volatile copy of tv_sec and tv_usec. Or make some code that calculates the offset of rte_rdtsc from 01 January 1970 00:00:00 UTC and uses TSC value to generate the right tv_sec and tv_usec would also work fine. Matthew.