Re: [dpdk-dev] [PATCH v8 2/2] config: enable vhost numa awareness by default

[Jerin Jacob <jerin.jacob@caviumnetworks.com>]

-----Original Message-----
> Date: Tue, 27 Jun 2017 17:47:44 +0530
> From: Hemant Agrawal <hemant.agrawal@nxp.com>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> CC: Thomas Monjalon <thomas@monjalon.net>, Ilya Maximets
>  <i.maximets@samsung.com>, dev@dpdk.org, David Marchand
>  <david.marchand@6wind.com>, Sergio Gonzalez Monroy
>  <sergio.gonzalez.monroy@intel.com>, Heetae Ahn <heetae82.ahn@samsung.com>,
>  Yuanhan Liu <yliu@fridaylinux.org>, Jianfeng Tan <jianfeng.tan@intel.com>,
>  Neil Horman <nhorman@tuxdriver.com>, Yulong Pei <yulong.pei@intel.com>,
>  Bruce Richardson <bruce.richardson@intel.com>
> Subject: Re: [PATCH v8 2/2] config: enable vhost numa awareness by default
> User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
>  Thunderbird/45.8.0
> 
> On 6/27/2017 3:29 PM, Jerin Jacob wrote:
> > -----Original Message-----
> > > Date: Tue, 27 Jun 2017 15:11:07 +0530
> > > From: Hemant Agrawal <hemant.agrawal@nxp.com>
> > > To: Thomas Monjalon <thomas@monjalon.net>
> > > CC: Ilya Maximets <i.maximets@samsung.com>, dev@dpdk.org, David Marchand
> > >  <david.marchand@6wind.com>, Sergio Gonzalez Monroy
> > >  <sergio.gonzalez.monroy@intel.com>, Heetae Ahn <heetae82.ahn@samsung.com>,
> > >  Yuanhan Liu <yliu@fridaylinux.org>, Jianfeng Tan <jianfeng.tan@intel.com>,
> > >  Neil Horman <nhorman@tuxdriver.com>, Yulong Pei <yulong.pei@intel.com>,
> > >  Bruce Richardson <bruce.richardson@intel.com>, Jerin Jacob
> > >  <jerin.jacob@caviumnetworks.com>
> > > Subject: Re: [PATCH v8 2/2] config: enable vhost numa awareness by default
> > > User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101
> > >  Thunderbird/45.8.0
> > > 
> > > On 6/27/2017 2:51 PM, Thomas Monjalon wrote:
> > > > 27/06/2017 11:18, Hemant Agrawal:
> > > > > On 6/27/2017 2:16 PM, Ilya Maximets wrote:
> > > > > > It is safe to enable LIBRTE_VHOST_NUMA by default for all
> > > > > > configurations where libnuma is already a default dependency.
> > > > > > 
> > > > > > Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
> > > > > > ---
> > > > > >  config/common_linuxapp                    | 1 +
> > > > > >  config/defconfig_arm-armv7a-linuxapp-gcc  | 1 +
> > > > > >  config/defconfig_arm64-dpaa2-linuxapp-gcc | 1 +
> > > > > >  3 files changed, 3 insertions(+)
> > > > [...]
> > > > > > --- a/config/defconfig_arm64-dpaa2-linuxapp-gcc
> > > > > > +++ b/config/defconfig_arm64-dpaa2-linuxapp-gcc
> > > > > > @@ -47,6 +47,7 @@ CONFIG_RTE_PKTMBUF_HEADROOM=256
> > > > > > 
> > > > > >  # Doesn't support NUMA
> > > > > >  CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y
> > > > > > +CONFIG_RTE_LIBRTE_VHOST_NUMA=n
> > > > > > 
> > > > > >  #
> > > > > >  # Compile Support Libraries for DPAA2
> > > > > > 
> > > > > 
> > > > > -1
> > > > > It should also be disabled for generic ARM64. This patch is breaking
> > > > > generic arm64 config tests on our platforms and creating a unnecessary
> > > > > dependency.
> > > > 
> > > > What do you mean? Which ARM64 platform is it breaking?
> > > > We can specifically disable it on more platforms.
> > > > 
> > > Unlike x86, ARM only represent a core architecture.
> > > Different platforms can integrate these cores differently in their SoCs.
> > > The stock ARM v8 cores do not provide support for NUMA in my knowledge.
> > 
> > A72 is just _an_ implementation of armv8. Not ARMv8 specification
> > itself. By specification it is NUMA capable and there are NUMA
> > implementation too.
> > 
> > > Some vendors have modified ARM cores (e.g. Cavium) to support NUMA
> > > architecture. However that is not a common phenomena.
> > > NUMA config should not be default for generic ARM config. It should be
> > > enabled only for architecture supporting it.
> > 
> > It just an build time dependency. Right? If you feed the libnuma package,
> > it will NON NUMA as well. Right? ARM64 libnuma package is already
> > available for major distributions.
> 
> yes, libnuma will work for non-NUMA.
> > 
> > My point is, I don't want to make arm64 generic config an exceptional case,
> > If DPDK common config creates libnuma dependency then there is no reason
> > for arm64 not have it. It is same for x86 and powerpc, non numa systems
> > too. Right?
> 
> x86 and powerpc configs are single vendor based.
> Common should be common and generic.

Yes. What I understand by common is that it should work on functionality on _all_ the
armv8 targets. If you don't include NUMA then it will have functionality issue
with NUMA targets.

The ARM64 Linux kernel took the similar approach. The default config has all
options and NUMA is _enabled_ even it is not supported on A72.

http://elixir.free-electrons.com/linux/latest/source/arch/arm64/configs/defconfig#L77


> 
> Why to create a unnecessary dependency, when we know that the support is not
> uniform?  It adds difficulties e.g. For the ARM cross compilation, will also
> have to cross compile libnuma-dev. Makefile will need a path for specifying
> the lib and include paths for libnuma and numa.h.

Yes. I agree. Cross compilation needs additional step. On the other
hand, If we don't include NUMA in common config, We need to add new targets on
all new SoCs(like thunderx2). IMO, In order to reduce the config, I think,
this is the better way.(and it is not hard to disable NUMA for cross
compilation mode if not interested)

> 
> 
> > 
> > > 
> > > So, *arm64-armv8a-linuxapp-gcc* config is being used by several vendors
> > > include NXP. e.g. We use this config on several of our low end systems
> > > (non-dpaa). Also, we use it when running in VM with virtio interfaces on all
> > > of our different platforms (non-dpaa, dpaa1, dpaa2 etc).
> > 
> > On the same note, arm64-armv8a-linuxapp-gcc used by other vendors for Server machines
> > with NUMA and if want to keep creating new targets there is no end to it.
> > 
> > How hard is to install libnuma on VM? There is already package for it.
> > 
> > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > 
> 
>