Re: [dpdk-dev] [PATCH 3/4] mk: allow kernel artifacts to be located in output directory

[Scott Branden <scott.branden@broadcom.com>]

Hi Ferruh,

We dropped this patch from the series - comment inline.


On 18-04-10 07:59 AM, Ferruh Yigit wrote:
> On 4/6/2018 11:32 PM, Scott Branden wrote:
>>
>> On 18-04-06 02:31 PM, Ferruh Yigit wrote:
>>> On 4/6/2018 7:10 PM, Scott Branden wrote:
>>>> Hi Ferruh,
>>>>
>>>> Somehow I did not receive your other email so Ajit fowarded it to me.  So I am
>>>> responding to it here.
>>>>
>>>> Sorry for the previous confusion:  RTE_KERNELDIR_OUT specifies the location of
>>>> the output of the kernel artifacts, not the output of the DPDK build artifacts.
>>> Ahh, this makes more sense now J Thanks.
>>>
>>>> Here is an example to use RTE_KERNELDIR_OUT:
>>>>
>>>> export ARCH=arm64
>>>> export CROSS_COMPILE=~/gcc-linaro-aarch64-linux-gnu/bin/aarch64-linux-gnu-
>>>> export CROSS=aarch64-linux-gnu-
>>>>
>>>> export RTE_KERNELDIR_OUT=~/test/linux-out
>>>> export RTE_KERNELDIR=~/test/linux
>>>> cd $RTE_KERNELDIR
>>>> make defconfig O=$RTE_KERNELDIR_OUT
>>>> make -j 8 O=$RTE_KERNELDIR_OUT
>>>>
>>>> cd ~/test/dpdk
>>>> make config T=arm64-stingray-linuxapp-gcc
>>>> make
>>> I guess using "~/test/linux-out" as RTE_KERNELDIR also works (because of
>>> "source" link), but your patch looks more proper way.
>> Actually, you may be correct.
>> Perhaps all this requires is a documentation change.
>> Does any of DPDK build system actually require to know where the kernel
>> source directory is?
>> Or, does DPDK only require to know where the kernel artifacts exist (ie.
>> generated headers)
> Both are required, kernel source is required because of some headers, there are
> multiple components depends on kernel headers, also kernel artifacts are
> required like kernel config mainly for building kernel modules.
>> Looking at the documentation it is a little confusing.  Each one
>> describes something different.
>> guides/linux_gsg/build_dpdk.rst
>> 99:    the ``RTE_KERNELDIR`` environment variable should be used to
>> point the compilation at a copy of the kernel version to be used on the
>> target machine.
>>
>> guides/prog_guide/dev_kit_build_system.rst
>> 297:*   RTE_KERNELDIR: This variable contains the absolute path to the
>> kernel sources that will be used to compile the kernel modules.
>>
>> build-sdk-quick.txt
>> 24:    RTE_KERNELDIR    linux headers path
>>
>> In guides/prog_guide/dev_kit_build_system.rst.  If "kernel sources" was
>> changed to "linux headers" my change may not be needed.  I'm doing some
>> more testing now with RTE_KERNELDIR set to linux-out in other DPDK builds.
> I believe it is not just "linux headers", it is more like "compiled kernel source"
Yes - everything is provided in the linux-out directory.
I have no need for this patch and we have dropped it from the patch series.
Setting RTE_KERNELDIR to the linux-out directory allows DPDK to build.
>
>>> There are a few documents for build parameters, can you please update them too:
>>> doc/guides/prog_guide/dev_kit_build_system.rst
>>> doc/guides/linux_gsg/build_dpdk.rst <-- not sure about this one, please check
>>> doc/build-sdk-quick.txt  <-- this is "make help" output
>>>
>>>
>>>> Note:
>>>>
>>>> With this example you will likely hit a few compile errors as latest dpdk has
>>>> compile bugs with unused variables/type mismatches in the code.
>>> I didn't get the error, if you observe build error please report or better a
>>> patch to fix is always welcome :)
>> Yes - we have patches being created for 64-bit build warnings/errors
>> that we will submit.
>>>> Regards,
>>>>
>>>>    Scott
>>>>
>>>>
>>>> On 18-04-05 11:58 AM, Ajit Khaparde wrote:
>>>>> Scott, are you looking at this?
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: *Ferruh Yigit* <ferruh.yigit@intel.com <mailto:ferruh.yigit@intel.com>>
>>>>> Date: Tue, Apr 3, 2018 at 7:18 AM
>>>>> Subject: Re: [PATCH 3/4] mk: allow kernel artifacts to be located in output
>>>>> directory
>>>>> To: Scott Branden <scott.branden@broadcom.com
>>>>> <mailto:scott.branden@broadcom.com>>, Ajit Khaparde
>>>>> <ajit.khaparde@broadcom.com <mailto:ajit.khaparde@broadcom.com>>, dev@dpdk.org
>>>>> <mailto:dev@dpdk.org>
>>>>>
>>>>>
>>>>> On 3/30/2018 7:23 PM, Scott Branden wrote:
>>>>>> Hi Ferruh,
>>>>>>
>>>>>>
>>>>>> On 18-03-30 03:35 AM, Ferruh Yigit wrote:
>>>>>>> On 3/21/2018 6:06 PM, Ajit Khaparde wrote:
>>>>>>>> From: Scott Branden <scott.branden@broadcom.com
>>>>> <mailto:scott.branden@broadcom.com>>
>>>>>>>> Allow kernel artifacts to be located in output directory specific
>>>>>>>> by RTE_KERNELDIR_OUT.
>>>>>>> How to use this variable, I tried following but kernel modules are still in
>>>>> same
>>>>>>> place:
>>>>>> export RTE_KERNELDIR and RTE_KERNELDIR_OUT and it works fine.
>>>>>> RTE_KERNELDIR already exists in Makefile.  Are you able to override it?
>>>>> Please help me a little more, can you please list your commands?
>>>>>
>>>>> Before your patch, what I do:
>>>>>
>>>>> export RTE_KERNELDIR=~/development/linux/linux
>>>>> make defconfig
>>>>> make    <--- After this kernel modules in ./build/kmod/*.ko
>>>>> make DESTDIR=/tmp/for-scott install
>>>>>
>>>>> kernel modules are installed into /tmp/for-scott/lib/modules/`uname
>>>>> -r`/extra/dpdk/*.ko
>>>>>
>>>>> Here `uname -r` is wrong but your patch is not related this.
>>>>>
>>>>> I am using RTE_KERNELDIR a lot since I am compiling with various kernel
>>>>> versions, not seen .ko files goes into source folder, what is your command line?
>>>>>
>>>>>
>>>>>
>>>>> Or _after_ your patch:
>>>>> export RTE_KERNELDIR=~/development/linux/linux
>>>>> export RTE_KERNELDIR_OUT=/tmp/kmod
>>>>> make defconfig
>>>>> make    <--- After this kernel modules in ./build/kmod/*.ko
>>>>>
>>>>> Getting following build error:
>>>>> /bin/sh: scripts/mod/modpost: No such file or directory
>>>>> make[7]: *** [.../development/linux/linux/scripts/Makefile.modpost:92:
>>>>> __modpost] Error 127
>>>>> make[6]: *** [.../development/linux/linux/Makefile:1558: modules] Error 2
>>>>> make[5]: *** [Makefile:146: sub-make] Error 2
>>>>> make[4]: *** [../mk/rte.module.mk:54 <http://rte.module.mk:54>: igb_uio.ko]
>>>>> Error 2
>>>>>
>>>>> Can you please describe how this variable intended to be used?
>>>>>
>>>>>
>>>>>>> - make RTE_KERNELDIR_OUT=/tmp/kmod O=ferruh DESTDIR=/tmp/ferruh
>>>>>>> - make RTE_KERNELDIR_OUT=/tmp/kmod O=ferruh DESTDIR=/tmp/ferruh install
>>>>>>> - make RTE_KERNELDIR_OUT=/tmp/kmod O=ferruh DESTDIR=/tmp/ferruh modules_install
>>>>>>>
>>>>>>> /tmp/kmod not created.
>>>>>>>
>>>>>>> And what we are fixing here, what was wrong with old code?
>>>>>> Old code puts the intermediate object files in the kernel source
>>>>>> directory.  This doesn't work when building an external module in
>>>>>> openembedded recipes.  We need the Makefile change to be able to able to
>>>>>> specify the kernel and output directories separately.
>>>>>>>> If RTE_KERNELDIR_OUT not specified than
>>>>>>>> source directory RTE_KERNEL is used.
>>>>>>> ${RTE_KERNELDIR} is used as external Linux kernel source folder, why we are
>>>>> want
>>>>>>> to put artifacts into kernel source?
>>>>>> I just kept the default behaviour the same as what it does today.
>>>>>>>> Signed-off-by: Scott Branden <scott.branden@broadcom.com
>>>>> <mailto:scott.branden@broadcom.com>>
>>>>>>>> Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com
>>>>> <mailto:ajit.khaparde@broadcom.com>>
>>>>>>>> ---
>>>>>>>>      mk/rte.module.mk <http://rte.module.mk> | 9 ++++++---
>>>>>>>>      1 file changed, 6 insertions(+), 3 deletions(-)
>>>>>>>>
>>>>>>>> diff --git a/mk/rte.module.mk <http://rte.module.mk> b/mk/rte.module.mk
>>>>> <http://rte.module.mk>
>>>>>>>> index 1ada528a0..e2e1af716 100644
>>>>>>>> --- a/mk/rte.module.mk <http://rte.module.mk>
>>>>>>>> +++ b/mk/rte.module.mk <http://rte.module.mk>
>>>>>>>> @@ -39,6 +39,9 @@ _postinstall: build
>>>>>>>>      .PHONY: build
>>>>>>>>      build: _postbuild
>>>>>>>>
>>>>>>>> +# Allow kernel artifacts to be located in a different location that source
>>>>>>>> +RTE_KERNELDIR_OUT ?= ${RTE_KERNELDIR}
>>>>>>>> +
>>>>>>>>      # Link all sources in build directory
>>>>>>>>      %_link: FORCE
>>>>>>>>        $(if $(call compare,$(notdir $*),$*),\
>>>>>>>> @@ -48,7 +51,7 @@ build: _postbuild
>>>>>>>>      # build module
>>>>>>>>      $(MODULE).ko: $(SRCS_LINKS)
>>>>>>>>        @if [ ! -f $(notdir Makefile) ]; then ln -nfs $(SRCDIR)/Makefile . ; fi
>>>>>>>> -   @$(MAKE) -C $(RTE_KERNELDIR) M=$(CURDIR) O=$(RTE_KERNELDIR) \
>>>>>>>> +   @$(MAKE) -C $(RTE_KERNELDIR) M=$(CURDIR) O=$(RTE_KERNELDIR_OUT) \
>>>>>>>>                CC="$(KERNELCC)" CROSS_COMPILE=$(CROSS) V=$(if $V,1,0)
>>>>>>>>
>>>>>>>>      # install module in $(RTE_OUTPUT)/kmod
>>>>>>>> @@ -59,7 +62,7 @@ $(RTE_OUTPUT)/kmod/$(MODULE).ko: $(MODULE).ko
>>>>>>>>
>>>>>>>>      # install module
>>>>>>>>      modules_install:
>>>>>>>> -   @$(MAKE) -C $(RTE_KERNELDIR) M=$(CURDIR) O=$(RTE_KERNELDIR) \
>>>>>>>> +   @$(MAKE) -C $(RTE_KERNELDIR) M=$(CURDIR) O=$(RTE_KERNELDIR_OUT) \
>>>>>>>>                modules_install
>>>>>>>>
>>>>>>>>      .PHONY: clean
>>>>>>>> @@ -69,7 +72,7 @@ clean: _postclean
>>>>>>>>      .PHONY: doclean
>>>>>>>>      doclean:
>>>>>>>>        @if [ ! -f $(notdir Makefile) ]; then ln -nfs $(SRCDIR)/Makefile . ; fi
>>>>>>>> -   $(Q)$(MAKE) -C $(RTE_KERNELDIR) M=$(CURDIR) O=$(RTE_KERNELDIR) clean
>>>>>>>> +   $(Q)$(MAKE) -C $(RTE_KERNELDIR) M=$(CURDIR) O=$(RTE_KERNELDIR_OUT) clean
>>>>>>>>        @$(foreach FILE,$(SRCS-y) $(SRCS-n) $(SRCS-),\
>>>>>>>>                if [ -h $(notdir $(FILE)) ]; then rm -f $(notdir $(FILE)) ; fi ;)
>>>>>>>>        @if [ -h $(notdir Makefile) ]; then rm -f $(notdir Makefile) ; fi
>>>>>>>>
>>>>>> Regards,
>>>>>>      Scott
>>>>>>