Re: [dpdk-dev] [PATCH v3] build: add dockerfile for building docker image

[Yasufumi Ogawa <yasufum.o@gmail.com>]

On 2019/12/09 12:23, Ruifeng Wang (Arm Technology China) wrote:
>> -----Original Message-----
>> From: Halim, Abdul <abdul.halim@intel.com>
>> Sent: Friday, December 6, 2019 19:13
>> To: Yasufumi Ogawa <yasufum.o@gmail.com>; Ruifeng Wang (Arm
>> Technology China) <Ruifeng.Wang@arm.com>; dev@dpdk.org
>> Cc: Kinsella, Ray <ray.kinsella@intel.com>; nd <nd@arm.com>; Richardson,
>> Bruce <bruce.richardson@intel.com>
>> Subject: RE: [dpdk-dev] [PATCH v3] build: add dockerfile for building docker
>> image
>>
>>
>>
>>> -----Original Message-----
>>> From: Yasufumi Ogawa <yasufum.o@gmail.com>
>>> Sent: Thursday, December 5, 2019 7:52 PM
>>> To: Ruifeng Wang (Arm Technology China) <Ruifeng.Wang@arm.com>;
>> Halim,
>>> Abdul <abdul.halim@intel.com>; dev@dpdk.org
>>> Cc: Kinsella, Ray <ray.kinsella@intel.com>; nd <nd@arm.com>
>>> Subject: Re: [dpdk-dev] [PATCH v3] build: add dockerfile for building
>>> docker image
>>>
>>> On 2019/12/05 23:13, Ruifeng Wang (Arm Technology China) wrote:
>>>>
>>>>> -----Original Message-----
>>>>> From: dev <dev-bounces@dpdk.org> On Behalf Of Abdul Halim
>>>>> Sent: Tuesday, December 3, 2019 19:42
>>>>> To: dev@dpdk.org
>>>>> Cc: ray.kinsella@intel.com; yasufum.o@gmail.com; Abdul Halim
>>>>> <abdul.halim@intel.com>
>>>>> Subject: [dpdk-dev] [PATCH v3] build: add dockerfile for building
>>>>> docker image
>>>>>
>>>>> Adding a Dockerfile with Ubuntu bionic base image to build dpdk as
>>>>> shared library. This docker image could be used as base image to
>>>>> build and run dpdk applications in containers.
>>>>>
>>>>> Signed-off-by: Abdul Halim <abdul.halim@intel.com>
>>>>>
>>> [...]
>>>>> diff --git a/extras/README.md b/extras/README.md new file mode
>>> 100644
>>>>> index 0000000..f38d7f1
>>>>> --- /dev/null
>>>>> +++ b/extras/README.md
>>>>> @@ -0,0 +1,52 @@
>>>>> +# Build DPDK Docker image
>>>>> +
>>>>> +To build a docker image run the following command from dpdk root
>>>>> directory.
>>>>> +
>>>>> +```
>>>>> +DOCKER_TAG="dpdk"
>>>>> +docker build -t ${DOCKER_TAG} -f extras/Dockerfile.bionic .
>>>>> +```
>>>>> +
>>>>> +# Example of how to use this dpdk library image
>>>>> +
>>>>> +The following steps shows how to use the dpdk shared library
>>>>> +container to build and run a dpdk application without having to
>>>>> +build dpdk library for each application.
>>>>> +
>>>>> +## Create a dpdk sample app docker file with 'dpdk' as the base
>>>>> +image
>>>>> +
>>>>> +Create a docker file to build the dpdk helloworld application.
>>>>> +Since, we are creating a docker file for dpdk helloworld app we
>>>>> +need to add the dpdk source files, thus create the following
>>>>> +docker file in dpdk root
>>>>> directory.
>>>>> +
>>>>> +```
>>>>> +cat << EOF > Dockerfile.dpdkSampleApp FROM dpdk
>>>>> +
>>>>> +ADD . /opt/dpdk
>>>>> +
>>>>> +WORKDIR /opt/dpdk/examples/helloworld RUN make && cp
>>>>> +build/helloworld-shared /usr/local/bin/helloworld EOF ```
>>>>> +
>>>>> +## Build sample app docker image
>>>>> +
>>>>> +```
>>>>> +DOCKERAPP_TAG="dpdk-helloworld"
>>>>> +docker build -t ${DOCKERAPP_TAG} -f Dockerfile.dpdkSampleApp .
>>>>> +```
>>>>
>>>> Hi Abdul,
>>>>
>>>> I tried the steps on AArch64 platform, and hit error as below:
>>>>
>>>> $ sudo docker build -t ${DOCKERAPP_TAG} -f Dockerfile.dpdkSampleApp .
>>>> Sending build context to Docker daemon   2.55GB
>>>> Step 1/4 : FROM dpdk
>>>>    ---> 955448007987
>>>> Step 2/4 : ADD . /opt/dpdk
>>>>    ---> d8b58019a7e2
>>>> Step 3/4 : WORKDIR /opt/dpdk/examples/helloworld
>>>>    ---> Running in 14fc89f7d3cd
>>>> Removing intermediate container 14fc89f7d3cd
>>>>    ---> 065a682c58fd
>>>> Step 4/4 : RUN make && cp build/helloworld-shared
>>> /usr/local/bin/helloworld
>>>>    ---> Running in 11e755a7180b
>>>> Makefile:44: *** "Please define RTE_SDK environment variable".  Stop.
>>>> The command '/bin/sh -c make && cp build/helloworld-shared
>>>> /usr/local/bin/helloworld' returned a non-zero code: 2
>>>>
>>>> Missing define of RTE_SDK and RTE_TARGET?
>>>
>>> Hi Ruifeng,
>>>
>>> I think you run you run the command in dpdk/extras. However, this
>>> 'Dockerfile.dpdkSampleApp' is expected to be run in dpdk's root dir so
>>> that it is mounted as '/opt/dpdk' in the second step above. I have
>>> tested this Dockerfile on Ubuntu 18.04 and compiled without any error.
>>> RTE_SDK is set correctly, but dpdk's directory is not mounted in the
>>> container.
>>>
> Hi Yasufumi,
> 
> I ran the command in dpdk root dir which should be correct.
> The issue was due to shared library image not been correctly built. See below.
Hi Ruifeng,

I've misunderstood about RTE_SDK as Abdul said. Sorry.
> 
> Thanks.
> /Ruifeng
>>> Abdul,
>>>
>>>   >> +docker build -t ${DOCKERAPP_TAG} -f Dockerfile.dpdkSampleApp .
>>>
>>> I think this line should be corrected as following, and make it clear
>>> it should be run in dpdk's root.
>>>
>>>     docker build -t ${DOCKERAPP_TAG} -f extras/Dockerfile.dpdkSampleApp .
>>>
>>> Even if the container image is built successfully, there is another
>>> problem in running app because it isn't run in privileged mode.
>>>
>>> root@0d2a309dfd2c:/opt/dpdk/examples/helloworld# helloworld
>>> EAL: Detected 16 lcore(s)
>>> EAL: Detected 1 NUMA nodes
>>> ...
>>> EAL: Failed to get current mempolicy: Operation not permitted.
>>> Assuming MPOL_DEFAULT.
>>> set_mempolicy: Operation not permitted
>>> set_mempolicy: Operation not permitted
>>> EAL: error allocating rte services array
>>> EAL: FATAL: rte_service_init() failed
>>> EAL: rte_service_init() failed
>>> PANIC in main():
>>> Cannot init EAL
>>> 5: [helloworld(+0x84a) [0x55555555484a]]
>>> 4: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)
>>> [0x7ffff7721b97]]
>>> 3: [helloworld(+0x818) [0x555555554818]]
>>> 2:
>>> [/usr/local/lib/x86_64-linux-gnu/librte_eal.so.20.1(__rte_panic+0xbd)
>>> [0x7ffff7afb410]]
>>> 1:
>>> [/usr/local/lib/x86_64-linux-gnu/librte_eal.so.20.1(rte_dump_stack+0x2
>>> e)
>>> [0x7ffff7b1598e]]
>>> Aborted (core dumped)
>>>
>>> I think '--privileged' option should be added to avoid the error.
>>>
>>> $ docker run --rm --privileged -it  -v /dev/hugepages:/dev/hugepages
>>> dpdk- helloworld
>>>
>>> I have one more suggestion. You might have added $USER to docker group
>>> and run docker without sudo like as following.
>>>
>>> $ sudo groupadd docker
>>> $ sudo usermod -aG docker $USER
>>>
>>> I wounder it is better to use sudo in your examples, or add the
>>> instruction for users not familiar with docker.
>>>
>>> Regards,
>>> Yasufumi
>>
>> Hi Yasufumi,
>> Thank you for your feedback.
>> The steps for creating the sample app docker file explains that that we are
>> creating the file at dpdk root directory. So the assumption here is the docker
>> run command also run from there. Not sure if we need to repeat this later
>> also.
>>
>> The 'cat' command above creates the docker file in dpdk root directory  for
>> simplicity. Actually, we just needed the examples/helloworld source code
>> from there. As for other user application, the docker file could be anywhere,
>> not necessarily in dpdk tree at all. User need to run docker build from where
>> their own docker file is.
>>
>> The dpdk 'base' container should be used as shared-lib to build dpdk
>> application with libdpdk. So, the dpdk source code, RTE_SDK or RTE_TARGET
>> is not needed unless the pkg-config is unable to find libdpdk.
>>
>> I will update the patch with suggested '--privileged' flag on docker run
>> command.
>> Not sure if we should cover the docker permissions and docker specific
>> configurations on  this doc though. I am sure user can find those resources
>> somewhere else if needed.
>>
>> Hi Ruifeng,
>> Unfortunately I could not create Aarch64 environment to test this. Could you
>> please run the following command in your env and see if you can get output
>> as below:
>>
>> $ docker run --rm dpdk pkg-config --list-all | grep libdpdk
>> libdpdk          DPDK - The Data Plane Development Kit (DPDK).
>>
> Hi Abdul,
> 
> Yes, the issue was due to pkg-config could not find libdpdk.
> I didn't get expected libdpdk info in pkg-config output.
> 
> Just found that even my building of dpdk shared library image was failed.
> Shared library image build failure was due to local 'build' directory in my
> dpdk source tree. And the 'build' impacted meson configuration in container.
> 
> Not sure if it is a worth to mention that base dpdk tree should be clean.
I am not sure the problem is always happened if the tree is not cleaned, 
but might be helpful.

Regards,
Yasufumi
> 
> With '--privileged' flag suggested by Yasufumi:
> Tested-by: Ruifeng Wang <ruifeng.wang@arm.com>
> 
> Thanks.
> /Ruifeng
>>
>> Regards,
>> Abdul
>>
>>>
>>>>
>>>>> +
>>>>> +This sample app now can be run like any other applicaiton in a
>>>>> +docker
>>>>> container.
>>>>> +
>>>>> +```
>>>>> +$ docker run --rm -it  -v /dev/hugepages:/dev/hugepages
>>>>> +dpdk-helloworld ```
>>>>> +
>>>>> +## Running the sample app
>>>>> +Once inside the container run helloword binary
>>>>> +
>>>>> +```
>>>>> +$ root@11233ed2e69c # helloworld
>>>>> +```
>>>>> +
>>>>> --
>>>>> 1.8.3.1
>>>>>
>>>>> --------------------------------------------------------------
>>>>> Intel Research and Development Ireland Limited Registered in
>>>>> Ireland Registered Office: Collinstown Industrial Park, Leixlip,
>>>>> County Kildare Registered Number: 308263
>>>>>
>>>>>
>>>>> This e-mail and any attachments may contain confidential material
>>>>> for the sole use of the intended recipient(s). Any review or
>>>>> distribution by others is strictly prohibited. If you are not the
>>>>> intended recipient, please contact the sender and delete all copies.
>>>>
>> --------------------------------------------------------------
>> Intel Research and Development Ireland Limited Registered in Ireland
>> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
>> Registered Number: 308263
>>
>>
>> This e-mail and any attachments may contain confidential material for the
>> sole use of the intended recipient(s). Any review or distribution by others is
>> strictly prohibited. If you are not the intended recipient, please contact the
>> sender and delete all copies.