Re: Updating examples which use coremask parameters

[Stephen Hemminger <stephen@networkplumber.org>]

On Thu, 2 Nov 2023 16:58:52 +0000
Bruce Richardson <bruce.richardson@intel.com> wrote:

> On Thu, Nov 02, 2023 at 05:28:42PM +0100, Thomas Monjalon wrote:
> > 02/11/2023 15:56, Bruce Richardson:  
> > > Hi all,
> > > 
> > > looking to start a discussion and get some input here.
> > > 
> > > There are a number of our examples in DPDK which still track core usage via
> > > a 64-bit bitmask, and, as such, cannot run on cores between 64 and
> > > RTE_MAX_LCORE. Two examples I have recently come across with this issue are
> > > "eventdev_pipeline" and "qos_sched", but I am sure there are others. The
> > > former is a good example (or bad example depending on your viewpoint) of
> > > this as it takes multiple coremask parameters - for RX cores, for TX cores,
> > > for worker cores and optionally for scheduler cores.
> > > 
> > > Now, the simple solution to this is to just expand the 64-bit bitmask to
> > > 128 bit or more, but I think that is just making things harder for the
> > > user, since dealing with long bitmasks is very awkward and unwieldy. Better
> > > instead to convert all examples using coremasks to using core lists
> > > instead.
> > > 
> > > First step should be to take our EAL corelist processing code and refactor
> > > it into a function that can be made public, so that it can be used by all
> > > apps for parsing core lists. Simple enough!  
> > 
> > OK to add some command line parsing helpers.
> > It should probably be the start of a new library for command line.
> >   
> 
> Funnily enough, separate to this I had already been working on an
> "rte_args" library to have some functions for working on argc/argv
> parameters. I'm planning on pushing out an RFC for 24.03 fairly shortly.
> 
> However, pulling in functions for arg parsing is a different set of
> functionality to what I had in mind, so we may yet get two libraries out of
> this. [Merging may be tricky due to issues around circular dependencies
> with EAL. My arg management library is designed to "sit on top of" EAL,
> while any lib offering e.g. coremask, corelist parsing functions would need
> to "sit beneath" EAL, so EAL can re-use it's functions].
> 
> Let's postpone the details of both these to when we get some RFCs out
> though.
> 
> 
> > > The next part I'm looking for input on is - how do we switch the apps from
> > > coremasks to core lists? Some options:
> > > 
> > > 1. Add in new commandline parameters for each app to work with core lists.
> > >   This is what we did in the past with EAL, by adding -l as a replacement
> > >   for -c. The advantage is that we maintain backward compatibility, but the
> > >   downside is that it becomes hard to find new suitable letter options for
> > >   the core lists. Taking eventdev_pipeline again, we would need "new"
> > >   options for "-r", "-t", "-w" and "-s" parameters. Using the capitalized
> > >   versions of these would be a simple alternative, but "-W" is already used
> > >   as an app parameter so we can't do that.
> > > 
> > > 2. Just break backward compatibility and switch the apps to taking
> > >   core lists instead of masks. Advantage is that it gives us the cleanest
> > >   solution, but the downside is that and testing done using these examples,
> > >   or any users who may have run them in the past, get different behaviour.  
> > 
> > We don't need to offer backward compatibility in examples.
> > So this option is acceptable.
> >   
> 
> Glad to hear it. Makes life simpler.
> 
> > > 3. An interesting further alternative is to allow apps to take both
> > > coremasks and corelists and use heuristics to determine which is which.
> > > For example, anything starting with "0x" is a mask, anything containing
> > > "-" or "," is a list. There would be ambiguous values such as e.g. 2,
> > > which could be either, but we can always find ways to disambiguate
> > > these, e.g. allow trailing commas in lists, so that "0x2" is the
> > > coremask, and "2," is the corelist. [Could be other alternatives]. This
> > > largely keeps backward compatibility and also allows use of corelists.  
> > 
> > The option 3 can be interesting as well.
> >  
> Yep. If we start offering a library of arg-parsing functions, one of those
> could be a function using heuristics to identify core-mask, core-list or
> ambiguous values. Then each app can decide what to do in the latter case.
> Since we don't care about backward compatibility, the examples can just parse
> ambiguous values as core-list. User could then still use coremasks by
> prefixing them with "0x".

Noticed that lib/graph is using 64 bit coremask internally.
Wonder if others have the same issue.
Would be good if DPDK had a library to handle cpusets better, something
like the Linux kernel cpuset which uses comma separated list of cpu masks.