Re: [dpdk-dev] [PATCH v1 1/6] build: increase default of max lcores to 512

[David Hunt <david.hunt@intel.com>]

On 14/9/2021 11:00 AM, David Marchand wrote:
> On Tue, Sep 14, 2021 at 11:34 AM David Hunt <david.hunt@intel.com> wrote:
>>
>> On 10/9/2021 9:24 AM, Thomas Monjalon wrote:
>>> 10/09/2021 10:06, David Marchand:
>>>> On Fri, Sep 10, 2021 at 9:54 AM Bruce Richardson
>>>> <bruce.richardson@intel.com> wrote:
>>>>> On Fri, Sep 10, 2021 at 08:51:04AM +0200, David Marchand wrote:
>>>>>> On Thu, Sep 9, 2021 at 4:38 PM Bruce Richardson
>>>>>> <bruce.richardson@intel.com> wrote:
>>>>>>> On Thu, Sep 09, 2021 at 02:45:06PM +0100, David Hunt wrote:
>>>>>>>> Modern processors are coming with an ever increasing number of cores,
>>>>>>>> and 128 does not seem like a sensible max limit any more, especially
>>>>>>>> when you consider multi-socket systems with Hyper-Threading enabled.
>>>>>>>>
>>>>>>>> This patch increases max_lcores default from 128 to 512.
>>>>>>>>
>>>>>>>> Signed-off-by: David Hunt <david.hunt@intel.com>
>>>>>> Why should we need this?
>>>>>>
>>>>>> --lcores makes it possible to pin 128 lcores to any physical core on
>>>>>> your system.
>>>>>> And for applications that have their own thread management, they can
>>>>>> pin thread, then use rte_thread_register.
>>>>>>
>>>>>> Do you have applications that require more than 128 lcores?
>>>>>>
>>>>> The trouble is that using the --lcores syntax for mapping high core numbers
>>>>> to low lcore ids is much more awkward to use. Every case of DPDK use I've
>>>>> seen uses -c with a coremask, or -l with just giving a few core numbers on
>>>>> it. This simple scheme won't work with core numbers greater than 128, and
>>>>> there are already systems available with more than that number of cores.
>>>>>
>>>>> Apart from the memory footprint issues - which this patch is already making
>>>>> a good start in addressing, why would we not increase the default
>>>>> max_lcores to that seen on real systems?
>>>> The memory footprint is a major issue to me, and reserving all those
>>>> lcores won't be needed in any system.
>>>> We will also have to decide on a "640k ought to be enough" value to
>>>> avoid ABI issue with the next processor that comes out and has more
>>>> than 512 cores.
>>>>
>>>> Could we wire the -c / -l options to --lcores behavior ?
>>>> It breaks the 1:1 lcore/physical core assumption, but it solves your
>>>> usability issue.
>>> Why would we change existing options while we already have an option
>>> (--lcores) which solves the issue above?
>>> I think the only issue is to educate users.
>>> Is there something to improve in the documentation?
>>>
>> Hi all,
>> I agree that it’s a good idea to switch to using the “--lcrores” option
> Let's avoid typo in the error message you'll add :-).
>
>
>> for cores above the default, that’s already future proofed.
>> However, I’m still a little concerned about usability, if our users are
>> accustomed to the “-c” and “-l” options, I suggest that we add a warning
>> to suggest using the “--lcores” option if any of the cores provided on
>> the command line are above RTE_MAX_LCORE. That would help them with the
>> solution to using physical cores above 128 (or whatever the compiled
>> default is).
>>
>> Example:
>>
>> “ERROR: logical core 212 is above the maximum lcore number permitted.
>> Please use the --lcores option to map lcores onto physical cores, e.g.
>> --lcores="(0-3)@(212-215).”
> If you could directly provide the right --lcores syntax based on what
> user provided with -c or -l, it would be even better.
> This should be not that difficult.


Agreed. I now have something working that when given "-l 12-16,130,132", 
will output the following:

EAL: One of the 7 cores provided exceeds RTE_MAX_LCORE (128)
EAL: Please use --lcores instead, e.g. --lcores "(0-6)@(12-16,130,132)"

So you can just cut-and-paste that option into your command line. Makes 
it very easy for users to migrate.


>
>> I’ll replace the first patch in the set with a patch that adds the
>> additional information in the error message.
>
>