Re: [PATCH v5] enhance NUMA affinity heuristic

["Burakov, Anatoly" <anatoly.burakov@intel.com>]

On 2/15/2023 2:22 PM, Burakov, Anatoly wrote:
> On 2/1/2023 12:20 PM, Kaisen You wrote:
>> Trying to allocate memory on the first detected numa node has less
>> chance to find some memory actually available rather than on the main
>> lcore numa node (especially when the DPDK application is started only
>> on one numa node).
>>
>> Fixes: 705356f0811f ("eal: simplify control thread creation")
>> Fixes: bb0bd346d5c1 ("eal: suggest using --lcores option")
>> Cc: stable@dpdk.org
>>
>> Signed-off-by: David Marchand <david.marchand@redhat.com>
>> Signed-off-by: Kaisen You <kaisenx.you@intel.com>
>> ---
>> Changes since v4:
>> - mod the patch title,
>>
>> Changes since v3:
>> - add the assignment of socket_id in thread initialization,
>>
>> Changes since v2:
>> - add uncommitted local change and fix compilation,
>>
>> Changes since v1:
>> - accomodate for configurations with main lcore running on multiples
>>    physical cores belonging to different numa,
>> ---
>>   lib/eal/common/eal_common_thread.c | 1 +
>>   lib/eal/common/malloc_heap.c       | 4 ++++
>>   2 files changed, 5 insertions(+)
>>
>> diff --git a/lib/eal/common/eal_common_thread.c 
>> b/lib/eal/common/eal_common_thread.c
>> index 38d83a6885..21bff971f8 100644
>> --- a/lib/eal/common/eal_common_thread.c
>> +++ b/lib/eal/common/eal_common_thread.c
>> @@ -251,6 +251,7 @@ static void *ctrl_thread_init(void *arg)
>>       void *routine_arg = params->arg;
>>       __rte_thread_init(rte_lcore_id(), cpuset);
>> +    RTE_PER_LCORE(_socket_id) = SOCKET_ID_ANY;
>>       params->ret = rte_thread_set_affinity_by_id(rte_thread_self(), 
>> cpuset);
>>       if (params->ret != 0) {
>>           __atomic_store_n(&params->ctrl_thread_status,
>> diff --git a/lib/eal/common/malloc_heap.c b/lib/eal/common/malloc_heap.c
>> index d7c410b786..3ee19aee15 100644
>> --- a/lib/eal/common/malloc_heap.c
>> +++ b/lib/eal/common/malloc_heap.c
>> @@ -717,6 +717,10 @@ malloc_get_numa_socket(void)
>>               return socket_id;
>>       }
>> +    socket_id = rte_lcore_to_socket_id(rte_get_main_lcore());
>> +    if (socket_id != (unsigned int)SOCKET_ID_ANY)
>> +        return socket_id;
>> +
>>       return rte_socket_id_by_idx(0);
>>   }
> 
> I may be lacking context, but I don't quite get the suggested change. 
>  From what I understand, the original has to do with assigning lcore 
> cpusets in such a way that an lcore ends up having two socket ID's 
> (because it's been assigned to CPU's on different sockets). Why is this 
> allowed in the first place? It seems like a user error to me, as it 
> breaks many of the fundamental assumptions DPDK makes.
> 
> I'm fine with using main lcore socket for control threads, I just don't 
> think the `socket_id != SOCKET_ID_ANY` thing should be checked here, 
> because it apparently tries to compensate for a problem with cpuset of 
> the main thread, which shouldn't have happened to begin with.
> 
Just to be clear: I don't have any objections to this patch otherwise.

Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>

-- 
Thanks,
Anatoly