Re: [dpdk-dev] L3fwd mode in testpmd

[Honnappa Nagarahalli <Honnappa.Nagarahalli@arm.com>]

<snip>

> > > > > > > > > > On Thu, Mar 11, 2021 at 12:01 AM Honnappa Nagarahalli
> > > > > > > > > > <Honnappa.Nagarahalli@arm.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hello,
> > > > > > > > > > >         Performance of L3fwd example application is
> > > > > > > > > > > one of the key
> > > > > > > > > > benchmarks in DPDK. However, the application does not
> > > > > > > > > > have many debugging statistics to understand the
> > > > > > > > > > performance issues. We have added L3fwd as another
> > > > > > > > > > mode/stream to testpmd which provides
> > > > > > > > enough
> > > > > > > > > > statistics at various levels. This has allowed us to
> > > > > > > > > > debug the performance issues effectively.
> > > > > > > > > > >
> > > > > > > > > > > There is more work to be done to get it to
> > > > > > > > > > > upstreamable state. I am
> > > > > > > > > > wondering if such a patch is helpful for others and if
> > > > > > > > > > the community would be interested in taking a look.
> > > > > > > > > > Please let me know
> > > > > > > what you think.
> > > > > > > > > >
> > > > > > > > > > We are using app/proc-info/ to attach and analyze the
> > > performance.
> > > > > > > > > > That helps to analyze the unmodified application. I
> > > > > > > > > > think, if something is missing in proc-info app, in my
> > > > > > > > > > opinion it is better to enhance proc-info so that it
> > > > > > > > > > can help other third-party
> > > > > applications.
> > > > > > > > > >
> > > > > > > > > > Just my 2c.
> > > > > > > > > Thanks Jerin. We will explore that.
> > > > > > > >
> > > > > > > > I agree it is dangerous to rely too much on testpmd for
> everything.
> > > > > > > > Please tell us what in testpmd could be useful out of it.
> > > > > > > >
> > > > > > > Things that are very helpful in testpmd are: 1) HW
> > > > > > > statistics from the NIC 2) Forwarding stats 3) Burst stats
> > > > > > > (indication of headroom
> > > > > > > availability) 4) Easy to set parameters like RX and TX queue
> > > > > > > depths (among others) without having to recompile.
> > > > > >
> > > > > > [Kathleen Capella]
> > > > > > Thank you for the suggestion of app/proc-info. I've tried it
> > > > > > out with l3fwd and see that it does have the HW stats from the
> > > > > > NIC and the forwarding
> > > > > stats.
> > > > > > However, it does not have the burst stats testpmd offers, nor
> > > > > > the
> > > > >
> > > > > One option to see such  level of debugging would be to have
> > > > > - Create a memzone in the primary process
> > > > > - Application under test can update the stats in memzone based
> > > > > on the code flow
> > > > > - proc-info can read the counters updated by application under
> > > > > test using the memzone object got through rte_memzone_lookup()
> > > > Agreed. Currently, using app/proc-info does not provide this
> > > > ability. We
> > > cannot add this capability to app/proc-info as these stats would be
> > > specific to L3fwd application.
> > >
> > > I meant creating generic counter-read/write infra via memzone to not
> > > make it as l3fwd specific.
> > Currently, app/proc-info is able to print the stats as they are standardized
> via the API. But for statistics that are generated in the application, they are
> very specific to that application. For ex: burst stats in testpmd are very
> specific to it and another application might implement the same in a very
> different manner.
> >
> > In needs to be something like the app/proc-info just needs to be a dumb
> displaying utility and the application has to do all the heavy lifting of copying
> the exact display strings to the memory.
> 
> Yes.
> 
> >
> > > > >
> > > > > Another approach will be using rte_trace()[1] for
> > > > > debugging/tracing by adding tracepoints in l3fwd for such events.
> > > > > It has a timestamp and the trace format is opensource trace
> > > > > format(CTF(Common trace format)), so that we can use post
> > > > > posting tools to analyze.
> > > > > [1]
> > > > > https://doc.dpdk.org/guides/prog_guide/trace_lib.html
> > > > This is good for analyzing an incident. I think it is an overhead
> > > > for
> > > development purposes.
> > >
> > > Consider if one wants to add burst stats, one can add stats
> > > increment under RTE_TRACE_POINT_FP, it will be emitted whenever
> code
> > > flow through that path. Set of events of can be viewed in trace
> > > viewer[1]. Would that be enough?
> > > Adding traces to l3fwd can be upstreamed as it is useful for others
> > > for debugging.
> > >
> > > [1]
> > > https://github.com/jerinjacobk/share/blob/master/dpdk_trace.JPG
> > This needs post processing of the trace info to derive the information, is it
> correct? For ex: for burst stats, there will be several traces generated
> collecting the number of packets returned by rte_eth_rx_burst which needs
> to be post processed.
> 
> Or You can have an additional variable to acculate it.
> 
> > Also, adding traces is equivalent to adding statistics in L3fwd.
> 
> Yes.
> 
> If the sole purpose only stats then it is better to add status in l3fwd without
> performance impact. I thought some thing else.
Stats as well as ability to change the configuration parameters without having to recompile.

> 
> >
> > > > >
> > > > > > ability to easily change parameters without having to
> > > > > > recompile, which helps reduce debugging time significantly.
> > We will not be able to fix this above issue.
> 
> It depends on what you want to debug. Trace can be disabled at runtime.
We need to be able to identify the best configurations for various parameters like RX/TX queue depths, burst size etc