Re: Updating examples which use coremask parameters

[Bruce Richardson <bruce.richardson@intel.com>]

On Thu, Nov 02, 2023 at 05:28:42PM +0100, Thomas Monjalon wrote:
> 02/11/2023 15:56, Bruce Richardson:
> > Hi all,
> > 
> > looking to start a discussion and get some input here.
> > 
> > There are a number of our examples in DPDK which still track core usage via
> > a 64-bit bitmask, and, as such, cannot run on cores between 64 and
> > RTE_MAX_LCORE. Two examples I have recently come across with this issue are
> > "eventdev_pipeline" and "qos_sched", but I am sure there are others. The
> > former is a good example (or bad example depending on your viewpoint) of
> > this as it takes multiple coremask parameters - for RX cores, for TX cores,
> > for worker cores and optionally for scheduler cores.
> > 
> > Now, the simple solution to this is to just expand the 64-bit bitmask to
> > 128 bit or more, but I think that is just making things harder for the
> > user, since dealing with long bitmasks is very awkward and unwieldy. Better
> > instead to convert all examples using coremasks to using core lists
> > instead.
> > 
> > First step should be to take our EAL corelist processing code and refactor
> > it into a function that can be made public, so that it can be used by all
> > apps for parsing core lists. Simple enough!
> 
> OK to add some command line parsing helpers.
> It should probably be the start of a new library for command line.
> 

Funnily enough, separate to this I had already been working on an
"rte_args" library to have some functions for working on argc/argv
parameters. I'm planning on pushing out an RFC for 24.03 fairly shortly.

However, pulling in functions for arg parsing is a different set of
functionality to what I had in mind, so we may yet get two libraries out of
this. [Merging may be tricky due to issues around circular dependencies
with EAL. My arg management library is designed to "sit on top of" EAL,
while any lib offering e.g. coremask, corelist parsing functions would need
to "sit beneath" EAL, so EAL can re-use it's functions].

Let's postpone the details of both these to when we get some RFCs out
though.


> > The next part I'm looking for input on is - how do we switch the apps from
> > coremasks to core lists? Some options:
> > 
> > 1. Add in new commandline parameters for each app to work with core lists.
> >   This is what we did in the past with EAL, by adding -l as a replacement
> >   for -c. The advantage is that we maintain backward compatibility, but the
> >   downside is that it becomes hard to find new suitable letter options for
> >   the core lists. Taking eventdev_pipeline again, we would need "new"
> >   options for "-r", "-t", "-w" and "-s" parameters. Using the capitalized
> >   versions of these would be a simple alternative, but "-W" is already used
> >   as an app parameter so we can't do that.
> > 
> > 2. Just break backward compatibility and switch the apps to taking
> >   core lists instead of masks. Advantage is that it gives us the cleanest
> >   solution, but the downside is that and testing done using these examples,
> >   or any users who may have run them in the past, get different behaviour.
> 
> We don't need to offer backward compatibility in examples.
> So this option is acceptable.
> 

Glad to hear it. Makes life simpler.

> > 3. An interesting further alternative is to allow apps to take both
> > coremasks and corelists and use heuristics to determine which is which.
> > For example, anything starting with "0x" is a mask, anything containing
> > "-" or "," is a list. There would be ambiguous values such as e.g. 2,
> > which could be either, but we can always find ways to disambiguate
> > these, e.g. allow trailing commas in lists, so that "0x2" is the
> > coremask, and "2," is the corelist. [Could be other alternatives]. This
> > largely keeps backward compatibility and also allows use of corelists.
> 
> The option 3 can be interesting as well.
>
Yep. If we start offering a library of arg-parsing functions, one of those
could be a function using heuristics to identify core-mask, core-list or
ambiguous values. Then each app can decide what to do in the latter case.
Since we don't care about backward compatibility, the examples can just parse
ambiguous values as core-list. User could then still use coremasks by
prefixing them with "0x".

/Bruce