Re: [PATCH 05/12] gen: add raw packet data API and tests

[Thomas Monjalon <thomas@monjalon.net>]

20/12/2021 11:21, Van Haaren, Harry:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 17/12/2021 12:40, Van Haaren, Harry:
> > > I could ramble on a bit more, but mostly diminishing returns I think...
> > > I'll just use this email as a reply to Thomas' tweet;
> > > https://twitter.com/tmonjalo/status/1337313985662771201
> > 
> > My original question was to know available applications,
> > not integrating such application in the DPDK repository.
> > 
> > I may me miss something obvious,
> > but I don't understand why trying to add a user app inside DPDK repo.
> 
> There are likely a few points-of-view on this particular topic; and I'm glad
> you mention it so we can discuss it clearly here.
> 
> There are two main parts to this patchset, the first is a packet generation library,
> with an easy to use string-based syntax. The *library* is designed to be extended in
> future to a range of "useful stuff" to do while generating packets.

The text syntax would be specific to this application
and not usable somewhere else, so it doesn't make sense as a lib.

> The packet generation
> *application* should have minimal features, and focus on ease-of-use (as suggested below).

It would be either a limited application,
or an ever-growing application.
If the latter, it should not be in the main DPDK repository in my opinion.

By the way, I don't think it is the responsibility of DPDK to generate packets.
I would prefer having an application using the already known scapy
or a graphical interface like Ostinato.
There are tons of approach to define packets to send (pCraft is another one).
DPDK should only manage the Tx part, and optionally Rx of forwarded packets.

> In order to test the DPDK code, we need a variety of unit tests, and a sample-application to show
> users how to use the library (as well as docs etc). For me, the interesting part is that it is a small
> step from a simple sample-app just for testing to a minimal tool for high-rate packet generation.
> 
> I think many users of DPDK first install DPDK, then wish for a tool to generate high traffic rates
> to test DPDK, and end up with a usability problem; DPDK does not include a usable packet generator.

I don't see any usability problem in using an external well known tool.
Learning a new tool provided by DPDK *is* a usabilty difficulty.

> To highlight this point; our own DPDK Docs simply ignore the requirement of packet-generation to
> actually have packets processed by skeleton: http://doc.dpdk.org/guides/sample_app_ug/skeleton.html 
> Our "quick start" on the website uses PCAP vdevs (avoiding the problem)  https://core.dpdk.org/doc/quick-start/
> Even searching the entire docs for "generate packet" doesn't give any relevant/useful results:
> http://doc.dpdk.org/guides/search.html?q=generate+packet&check_keywords=yes&area=default# 
> 
> Users could internet-search & find pktgen, moongen, trex, or similar tools. These tools are fantastic for experienced
> developers such as devs on this mailing list - we should *NOT* replicate these complex tools in DPDK itself. However,
> building any tool outside of DPDK repo requires more effort; another git-clone, another set of dependencies to install,
> perhaps another build-system to get used to. Particularly for people starting out with DPDK (who are likely finding
> it difficult to learn the various hugepage/PCI-binding etc), this is yet another problem to solve, or to give up.
> 
> So my proposal is as follows; let us add a simple DPDK traffic generator to DPDK. We can define its scope
> and its intended use, limiting the scope and capabilities. As before, I do NOT think it a good idea to build a
> complex and feature-rich packet generator. I do feel it useful to have an easy-to-use application in DPDK that
> is particularly designed for generating specific packets, at specific line-rates, and reports mpps returned.
> 
> Thoughts on adding an small scope-limited application to DPDK enabling ease-of-packet-generation for new users?

So you want a simple packet generator for simple benchmarks?
And for complex benchmarks, we use another tool?