Re: [dpdk-dev] [PATCH v5 0/2] Balanced allocation of hugepages

[Hemant Agrawal <hemant.agrawal@nxp.com>]

On 6/21/2017 4:52 PM, Jerin Jacob wrote:
> -----Original Message-----
>> Date: Wed, 21 Jun 2017 13:36:58 +0300
>> From: Ilya Maximets <i.maximets@samsung.com>
>> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>, Thomas Monjalon
>>  <thomas@monjalon.net>
>> CC: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>, Hemant
>>  Agrawal <hemant.agrawal@nxp.com>, dev@dpdk.org, Bruce Richardson
>>  <bruce.richardson@intel.com>, David Marchand <david.marchand@6wind.com>,
>>  Heetae Ahn <heetae82.ahn@samsung.com>, Yuanhan Liu <yliu@fridaylinux.org>,
>>  Jianfeng Tan <jianfeng.tan@intel.com>, Neil Horman
>>  <nhorman@tuxdriver.com>, Yulong Pei <yulong.pei@intel.com>
>> Subject: Re: [PATCH v5 0/2] Balanced allocation of hugepages
>> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
>>  Thunderbird/45.8.0
>>
>> On 21.06.2017 13:29, Jerin Jacob wrote:
>>> -----Original Message-----
>>>> Date: Wed, 21 Jun 2017 11:58:12 +0200
>>>> From: Thomas Monjalon <thomas@monjalon.net>
>>>> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>>>> Cc: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>, Hemant
>>>>  Agrawal <hemant.agrawal@nxp.com>, Ilya Maximets <i.maximets@samsung.com>,
>>>>  dev@dpdk.org, Bruce Richardson <bruce.richardson@intel.com>, David
>>>>  Marchand <david.marchand@6wind.com>, Heetae Ahn
>>>>  <heetae82.ahn@samsung.com>, Yuanhan Liu <yliu@fridaylinux.org>, Jianfeng
>>>>  Tan <jianfeng.tan@intel.com>, Neil Horman <nhorman@tuxdriver.com>, Yulong
>>>>  Pei <yulong.pei@intel.com>
>>>> Subject: Re: [PATCH v5 0/2] Balanced allocation of hugepages
>>>>
>>>> 21/06/2017 11:27, Jerin Jacob:
>>>>> -----Original Message-----
>>>>>> Date: Wed, 21 Jun 2017 10:49:14 +0200
>>>>>> From: Thomas Monjalon <thomas@monjalon.net>
>>>>>> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>>>>>> Cc: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>, Hemant
>>>>>>  Agrawal <hemant.agrawal@nxp.com>, Ilya Maximets <i.maximets@samsung.com>,
>>>>>>  dev@dpdk.org, Bruce Richardson <bruce.richardson@intel.com>, David
>>>>>>  Marchand <david.marchand@6wind.com>, Heetae Ahn
>>>>>>  <heetae82.ahn@samsung.com>, Yuanhan Liu <yliu@fridaylinux.org>, Jianfeng
>>>>>>  Tan <jianfeng.tan@intel.com>, Neil Horman <nhorman@tuxdriver.com>, Yulong
>>>>>>  Pei <yulong.pei@intel.com>
>>>>>> Subject: Re: [PATCH v5 0/2] Balanced allocation of hugepages
>>>>>>
>>>>>> 21/06/2017 10:41, Jerin Jacob:
>>>>>>>>> 1. There are many machines (arm/ppc), which do not support NUMA.
>>>>>>>>>
>>>>>>>>> https://wiki.linaro.org/LEG/Engineering/Kernel/NUMA
>>>>>>>>>
>>>>>>>>
>>>>>>>> I did find that link too, last modified 4 years ago.
>>>>>>>> Despite that, I could not find any ARM references in libnuma sources, but
>>>>>>>> Jerin proved that there is support for it.
>>>>>>>>
>>>>>>>> http://oss.sgi.com/projects/libnuma/
>>>>>>>> https://github.com/numactl/numactl
>>>>>>>
>>>>>>> Those Linaro links are very old. ARM64 NUMA supported has been added in 4.7 kernel.
>>>>>>> I guess we are talking about build time time dependency with libnuma here.
>>>>>>> Correct? I think, Even with old arm64 kernel(< 4.6), You can build against
>>>>>>> libnuma if it is present in rootfs. Just that at runtime, it will return
>>>>>>> NUMA support not available. Correct?
>>>>>>>
>>>>>>> How hard is detect the presence of "numaif.h" if existing build system does not
>>>>>>> support it? If it trivial, we can enable RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES
>>>>>>> if build environment has "numaif.h".
>>>>>>>
>>>>>>> Some example in linux kernel build system:
>>>>>>> http://lxr.linux.no/linux+v4.10.1/scripts/gcc-goto.sh
>>>>>>
>>>>>> I think we should not try to detect numaif.h, because it should be
>>>>>> an error on platform supporting NUMA.
>>>>>
>>>>> I have installed libnuma on a NUMA and non NUMA machine.
>>>>> Compiled and ran following code on those machine and it could detect
>>>>> the numa availability. Could you add more details on the "error on
>>>>> platform supporting NUMA".
>>>>
>>>> I was saying that we do not need to detect NUMA.
>>>> If we are building DPDK for a NUMA architecture and libnuma is not
>>>> available, then it will be a problem that the user must catch.
>>>> The easiest way to catch it, is to fail on the include of numaif.h.
>>>
>>> libnuma is not really _architecture_ depended.
>>>
>>> Ilya Maximets patch disables NUMA support in common arm64 config.I
>>> think, It is not correct, We should not disable on any archs generic config.
>>>
>>> IMO, It should be enabled by default in common config and then we can
>>> detect the presence of numaif.h, if not available OR a target does not need it
>>> explicitly, proceed with disabling
>>> RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES. I think, That is more portable.
>>
>> Detecting of headers is impossible until dpdk doesn't have dynamic build
>> configuration system like autotools, CMake or meson.
>> Right now we just can't do that.
>
> I agree. Unless if we do something like linux kernel does it below
> http://elixir.free-electrons.com/linux/latest/source/scripts/kconfig/lxdialog/check-lxdialog.sh
>
> Either way, I think, you can enable RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES in
> generic arm64 config and disable on defconfig_arm64-dpaa2-linuxapp-gcc(as Hemant requested) or
> any sub arch target that does not need in RTE_LIBRTE_EAL_NUMA_AWARE_HUGEPAGES.

No, this is not acceptable. it should not be enabled in generic arm64. 
It can be enabled in specific ARM platforms, which support NUMA 
architecture.
We also use generic ARM code on various of our platform when running 
with non-dpaa and/or virtio-net. So enabling it will break all those 
platforms.


>
>>
>>> No strong opinion on "failing the build" vs "printing a warning" in the
>>> absence of numaif.h
>