Re: [dpdk-dev] Question on huge pages and running DPDK apps inside containers

DPDK patches and discussions
 help / color / mirror / Atom feed

* Re: [dpdk-dev] Question on huge pages and running DPDK apps inside containers
       [not found] <CFDAB971.2ABC8%anjali@juniper.net>
@ 2014-07-03 14:53 ` Shaw, Jeffrey B
  2014-07-03 15:31   ` Anjali Kulkarni
  2014-07-03 15:33   ` Anjali Kulkarni
  2014-07-03 16:02 ` Richardson, Bruce
  1 sibling, 2 replies; 5+ messages in thread
From: Shaw, Jeffrey B @ 2014-07-03 14:53 UTC (permalink / raw)
  To: Anjali Kulkarni, dev

Do you know if the host's hugepages are mapped into the container?
Seeing as containers are meant to provide isolation, it seems to make sense that the host would not automatically share hugepages with a container, but I'm not sure.

Jeff

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anjali Kulkarni
Sent: Thursday, July 03, 2014 7:41 AM
To: dev@dpdk.org
Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside containers

Hi,

I am encountering a problem running DPDK apps inside containers, and need your help:

I create containers by doing:

lxc-create -t ubuntu -n fpc0
lxc-start -n fpc0

(Reserve hugepages and follow all other steps to run the DPDK app on host)

Then run a standard dpdk app, on host, it works fine, inside container, it runs into this problem:

ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 -- -i rte_eal_init : 857 opt : 99 opt : 110
EAL: No free hugepages reported in hugepages-2048kB PANIC in rte_eal_init():
Cannot get hugepage information
6: [./testpmd() [0x402941]]
5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f7f14f50ec5]]
4: [./testpmd() [0x40237e]]
3: [./testpmd() [0x478edb]]
2: [./testpmd() [0x4022ad]]
1: [./testpmd() [0x47e3c3]]
Aborted
ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$

However, running this same app on the host, or doing lxc-execute works.

While trying to debug my own dpdk app, I see that reading the same file for no of huge pages (

/sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages

 ), on the host, its value is read 2048, while inside the container it is 0. This is same file.
Also, if you cat the contents of this file, it is actually 0! (nr_hugepages is 2048, but the DPDK code does not read this file.)

cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
0

In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are added by me in DPDL eal library.

On host:
rte_eal_init : 859
opt : 99
opt : 110
EAL: Using free_hugepages
EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
EAL: No of hugepages is 2048

Inside container:
pt : 99
opt : 110
EAL: Using free_hugepages
EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
EAL: No of hugepages is 0

Has anyone run into this problem and know what's the issue?

Anjali

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] Question on huge pages and running DPDK apps inside containers
  2014-07-03 14:53 ` [dpdk-dev] Question on huge pages and running DPDK apps inside containers Shaw, Jeffrey B
@ 2014-07-03 15:31   ` Anjali Kulkarni
  2014-07-03 15:33   ` Anjali Kulkarni
  1 sibling, 0 replies; 5+ messages in thread
From: Anjali Kulkarni @ 2014-07-03 15:31 UTC (permalink / raw)
  To: Shaw, Jeffrey B, dev

Thanks - I got a confirmation from some Intel folks earlier that Intel
DPDK apps do work inside containers. Also, the weird thing is, testpmd
application runs in the container using lxc-execute. Lxc-execute runs the
app inside container. It works, see below:

root@anjali:/dpdk/x86_64-default-linuxapp-gcc/app# lxc-execute -n foo --
./testpmd -c f -n 4 -- -i
lxc: call to cgmanager_move_pid_abs_sync(cpuset) failed: Escape request
from different namespace requires a proxy
rte_eal_init : 859 
opt : 99 
opt : 110 
EAL: Using free_hugepages
EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
EAL: No of hugepages is 2048
rte_eal_init : 884 
EAL: No of hugepages is 0
EAL: No of hugepages is 1
EAL: No of hugepages is 2
EAL: No of hugepages is 3
EAL: Setting up hugepage memory...
EAL: Ask a virtual area of 0x8388608 bytes
EAL: Virtual area found at 0x7f7c39c00000 (size = 0x800000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7c39800000 (size = 0x200000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7c39400000 (size = 0x200000)
EAL: Ask a virtual area of 0x293601280 bytes
EAL: Virtual area found at 0x7f7c27a00000 (size = 0x11800000)
EAL: Ask a virtual area of 0x1841299456 bytes
EAL: Virtual area found at 0x7f7bb9c00000 (size = 0x6dc00000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7bb9800000 (size = 0x200000)
EAL: Ask a virtual area of 0x2143289344 bytes
EAL: Virtual area found at 0x7f7b39a00000 (size = 0x7fc00000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7b39600000 (size = 0x200000)
EAL: Requesting 1024 pages of size 2MB from socket 0
EAL: Requesting 1024 pages of size 2MB from socket 1
EAL: TSC frequency is ~2593752 KHz
EAL: No of hugepages is 32902
EAL: No of hugepages is 3584
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3586
EAL: No of hugepages is 32902
EAL: No of hugepages is 3586
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3587
EAL: No of hugepages is 32902
EAL: No of hugepages is 3587
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3588
EAL: No of hugepages is 32902
EAL: No of hugepages is 3588
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3592
EAL: No of hugepages is 32902
EAL: No of hugepages is 3592
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3594
EAL: No of hugepages is 32902
EAL: No of hugepages is 3594
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3624
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3626
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3628
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7486
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7482
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7483
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7469
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7440
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7462
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 9294
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7489
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7426
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7458
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7460
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 5409
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 5409
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 4347
EAL: No of hugepages is 5421
EAL: No of hugepages is 35198
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 4347
EAL: No of hugepages is 5421
EAL: No of hugepages is 35198
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 7531
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 6659
EAL: No of hugepages is 4432
EAL: No of hugepages is 6659
EAL: No of hugepages is 4432
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3592
EAL: No of hugepages is 32902
EAL: No of hugepages is 3592
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3624
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3626
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3628
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 4347
EAL: No of hugepages is 32902
EAL: No of hugepages is 3
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 4347
EAL: No of hugepages is 32902
EAL: No of hugepages is 3
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3712
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3728
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3760
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3761
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3762
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3763
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3764
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3765
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3766
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3767
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3613
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3636
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3713
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3638
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3784
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3785
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3786
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3776
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3777
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3778
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3779
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3614
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3615
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3808
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3810
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3812
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3814
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3809
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3811
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3813
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3815
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3744
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3632
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3752
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3697
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3754
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3755
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3756
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 32902
EAL: No of hugepages is 3757
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 1
EAL: No of hugepages is 6659
EAL: No of hugepages is 8192
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3712
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3728
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3760
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3761
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3762
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3763
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3764
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3765
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3766
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3767
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3613
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3636
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3713
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3638
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3784
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3785
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3786
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3776
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3777
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3778
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3779
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3614
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3615
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3808
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3810
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3812
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3814
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3809
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3811
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3813
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3815
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3744
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3632
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3752
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3697
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3754
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3755
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3756
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
EAL: No of hugepages is 32902
EAL: No of hugepages is 3757
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0

...

When I logged inside the container and ran the app, I saw this issue.

Anjali

On 7/3/14 7:53 AM, "Shaw, Jeffrey B" <jeffrey.b.shaw@intel.com> wrote:

>Do you know if the host's hugepages are mapped into the container?
>Seeing as containers are meant to provide isolation, it seems to make
>sense that the host would not automatically share hugepages with a
>container, but I'm not sure.
>
>Jeff
>
>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anjali Kulkarni
>Sent: Thursday, July 03, 2014 7:41 AM
>To: dev@dpdk.org
>Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside
>containers
>
>Hi,
>
>I am encountering a problem running DPDK apps inside containers, and need
>your help:
>
>I create containers by doing:
>
>lxc-create -t ubuntu -n fpc0
>lxc-start -n fpc0
>
>(Reserve hugepages and follow all other steps to run the DPDK app on host)
>
>Then run a standard dpdk app, on host, it works fine, inside container,
>it runs into this problem:
>
>ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 --
>-i rte_eal_init : 857 opt : 99 opt : 110
>EAL: No free hugepages reported in hugepages-2048kB PANIC in
>rte_eal_init():
>Cannot get hugepage information
>6: [./testpmd() [0x402941]]
>5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)
>[0x7f7f14f50ec5]]
>4: [./testpmd() [0x40237e]]
>3: [./testpmd() [0x478edb]]
>2: [./testpmd() [0x4022ad]]
>1: [./testpmd() [0x47e3c3]]
>Aborted
>ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$
>
>However, running this same app on the host, or doing lxc-execute works.
>
>While trying to debug my own dpdk app, I see that reading the same file
>for no of huge pages (
>
>/sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>
> ), on the host, its value is read 2048, while inside the container it is
>0. This is same file.
>Also, if you cat the contents of this file, it is actually 0!
>(nr_hugepages is 2048, but the DPDK code does not read this file.)
>
>cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>0
>
>
>In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are
>added by me in DPDL eal library.
>
>On host:
>rte_eal_init : 859
>opt : 99
>opt : 110
>EAL: Using free_hugepages
>EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>EAL: No of hugepages is 2048
>
>Inside container:
>pt : 99
>opt : 110
>EAL: Using free_hugepages
>EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>EAL: No of hugepages is 0
>
>
>Has anyone run into this problem and know what's the issue?
>
>Anjali

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] Question on huge pages and running DPDK apps inside containers
  2014-07-03 14:53 ` [dpdk-dev] Question on huge pages and running DPDK apps inside containers Shaw, Jeffrey B
  2014-07-03 15:31   ` Anjali Kulkarni
@ 2014-07-03 15:33   ` Anjali Kulkarni
  1 sibling, 0 replies; 5+ messages in thread
From: Anjali Kulkarni @ 2014-07-03 15:33 UTC (permalink / raw)
  To: Shaw, Jeffrey B, dev

Ignore the previous email, it was too long. Pls see this one only:

Thanks - I got a confirmation from some Intel folks earlier that Intel
DPDK apps do work inside containers. Also, the weird thing is, testpmd
application runs in the container using lxc-execute. Lxc-execute runs the
app inside container. It works, see below:
root@anjali:/dpdk/x86_64-default-linuxapp-gcc/app# lxc-execute -n foo --
./testpmd -c f -n 4 -- -i
lxc: call to cgmanager_move_pid_abs_sync(cpuset) failed: Escape request
from different namespace requires a proxy
rte_eal_init : 859
opt : 99
opt : 110
EAL: Using free_hugepages
EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
EAL: No of hugepages is 2048
rte_eal_init : 884
EAL: No of hugepages is 0
EAL: No of hugepages is 1
EAL: No of hugepages is 2
EAL: No of hugepages is 3
EAL: Setting up hugepage memory...
EAL: Ask a virtual area of 0x8388608 bytes
EAL: Virtual area found at 0x7f7c39c00000 (size = 0x800000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7c39800000 (size = 0x200000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7c39400000 (size = 0x200000)
EAL: Ask a virtual area of 0x293601280 bytes
EAL: Virtual area found at 0x7f7c27a00000 (size = 0x11800000)
EAL: Ask a virtual area of 0x1841299456 bytes
EAL: Virtual area found at 0x7f7bb9c00000 (size = 0x6dc00000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7bb9800000 (size = 0x200000)
EAL: Ask a virtual area of 0x2143289344 bytes
EAL: Virtual area found at 0x7f7b39a00000 (size = 0x7fc00000)
EAL: Ask a virtual area of 0x2097152 bytes
EAL: Virtual area found at 0x7f7b39600000 (size = 0x200000)
EAL: Requesting 1024 pages of size 2MB from socket 0
EAL: Requesting 1024 pages of size 2MB from socket 1
EAL: TSC frequency is ~2593752 KHz
EAL: No of hugepages is 32902
EAL: No of hugepages is 3584
EAL: No of hugepages is 5421
EAL: No of hugepages is 35207
EAL: No of hugepages is 0
Š

Š

When I logged inside the container and ran the app, I saw this issue.

Anjali

On 7/3/14 7:53 AM, "Shaw, Jeffrey B" <jeffrey.b.shaw@intel.com> wrote:

>Do you know if the host's hugepages are mapped into the container?
>Seeing as containers are meant to provide isolation, it seems to make
>sense that the host would not automatically share hugepages with a
>container, but I'm not sure.
>
>Jeff
>
>-----Original Message-----
>From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anjali Kulkarni
>Sent: Thursday, July 03, 2014 7:41 AM
>To: dev@dpdk.org
>Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside
>containers
>
>Hi,
>
>I am encountering a problem running DPDK apps inside containers, and need
>your help:
>
>I create containers by doing:
>
>lxc-create -t ubuntu -n fpc0
>lxc-start -n fpc0
>
>(Reserve hugepages and follow all other steps to run the DPDK app on host)
>
>Then run a standard dpdk app, on host, it works fine, inside container,
>it runs into this problem:
>
>ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 --
>-i rte_eal_init : 857 opt : 99 opt : 110
>EAL: No free hugepages reported in hugepages-2048kB PANIC in
>rte_eal_init():
>Cannot get hugepage information
>6: [./testpmd() [0x402941]]
>5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)
>[0x7f7f14f50ec5]]
>4: [./testpmd() [0x40237e]]
>3: [./testpmd() [0x478edb]]
>2: [./testpmd() [0x4022ad]]
>1: [./testpmd() [0x47e3c3]]
>Aborted
>ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$
>
>However, running this same app on the host, or doing lxc-execute works.
>
>While trying to debug my own dpdk app, I see that reading the same file
>for no of huge pages (
>
>/sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>
> ), on the host, its value is read 2048, while inside the container it is
>0. This is same file.
>Also, if you cat the contents of this file, it is actually 0!
>(nr_hugepages is 2048, but the DPDK code does not read this file.)
>
>cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>0
>
>
>In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are
>added by me in DPDL eal library.
>
>On host:
>rte_eal_init : 859
>opt : 99
>opt : 110
>EAL: Using free_hugepages
>EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>EAL: No of hugepages is 2048
>
>Inside container:
>pt : 99
>opt : 110
>EAL: Using free_hugepages
>EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>EAL: No of hugepages is 0
>
>
>Has anyone run into this problem and know what's the issue?
>
>Anjali

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] Question on huge pages and running DPDK apps inside containers
       [not found] <CFDAB971.2ABC8%anjali@juniper.net>
  2014-07-03 14:53 ` [dpdk-dev] Question on huge pages and running DPDK apps inside containers Shaw, Jeffrey B
@ 2014-07-03 16:02 ` Richardson, Bruce
  2014-07-03 18:09   ` Anjali Kulkarni
  1 sibling, 1 reply; 5+ messages in thread
From: Richardson, Bruce @ 2014-07-03 16:02 UTC (permalink / raw)
  To: Anjali Kulkarni, dev

Can you try deleting the hugepage files on the host before trying to run the app in a container?

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anjali Kulkarni
> Sent: Thursday, July 03, 2014 7:41 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside
> containers
> 
> Hi,
> 
> I am encountering a problem running DPDK apps inside containers, and need
> your help:
> 
> I create containers by doing:
> 
> lxc-create -t ubuntu -n fpc0
> lxc-start -n fpc0
> 
> (Reserve hugepages and follow all other steps to run the DPDK app on host)
> 
> Then run a standard dpdk app, on host, it works fine, inside container, it runs
> into this problem:
> 
> ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4 -- -i
> rte_eal_init : 857
> opt : 99
> opt : 110
> EAL: No free hugepages reported in hugepages-2048kB
> PANIC in rte_eal_init():
> Cannot get hugepage information
> 6: [./testpmd() [0x402941]]
> 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f7f14f50ec5]]
> 4: [./testpmd() [0x40237e]]
> 3: [./testpmd() [0x478edb]]
> 2: [./testpmd() [0x4022ad]]
> 1: [./testpmd() [0x47e3c3]]
> Aborted
> ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$
> 
> However, running this same app on the host, or doing lxc-execute works.
> 
> While trying to debug my own dpdk app, I see that reading the same file for no
> of huge pages (
> 
> /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
> 
>  ), on the host, its value is read 2048, while inside the container it is 0. This is
> same file.
> Also, if you cat the contents of this file, it is actually 0! (nr_hugepages is 2048,
> but the DPDK code does not read this file.)
> 
> cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
> 0
> 
> 
> In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are added by me
> in DPDL eal library.
> 
> On host:
> rte_eal_init : 859
> opt : 99
> opt : 110
> EAL: Using free_hugepages
> EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
> EAL: No of hugepages is 2048
> 
> Inside container:
> pt : 99
> opt : 110
> EAL: Using free_hugepages
> EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
> EAL: No of hugepages is 0
> 
> 
> Has anyone run into this problem and know what's the issue?
> 
> Anjali

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [dpdk-dev] Question on huge pages and running DPDK apps inside containers
  2014-07-03 16:02 ` Richardson, Bruce
@ 2014-07-03 18:09   ` Anjali Kulkarni
  0 siblings, 0 replies; 5+ messages in thread
From: Anjali Kulkarni @ 2014-07-03 18:09 UTC (permalink / raw)
  To: Richardson, Bruce, dev

Richard, thanks.

Looking at lxc-execute, I see this, which seems to be the key in running
the applications created via lxc-create and those run lxc-execute:
Note
"lxc-execute is intended to run application containers that share the
host's
        root file system, and not to run system containers that you create
using
          lxc-create. Use lxc-start to run system containers."

Anjali




On 7/3/14 9:02 AM, "Richardson, Bruce" <bruce.richardson@intel.com> wrote:

>Can you try deleting the hugepage files on the host before trying to run
>the app in a container?
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Anjali Kulkarni
>> Sent: Thursday, July 03, 2014 7:41 AM
>> To: dev@dpdk.org
>> Subject: [dpdk-dev] Question on huge pages and running DPDK apps inside
>> containers
>> 
>> Hi,
>> 
>> I am encountering a problem running DPDK apps inside containers, and
>>need
>> your help:
>> 
>> I create containers by doing:
>> 
>> lxc-create -t ubuntu -n fpc0
>> lxc-start -n fpc0
>> 
>> (Reserve hugepages and follow all other steps to run the DPDK app on
>>host)
>> 
>> Then run a standard dpdk app, on host, it works fine, inside container,
>>it runs
>> into this problem:
>> 
>> ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$ ./testpmd -c f -n 4
>>-- -i
>> rte_eal_init : 857
>> opt : 99
>> opt : 110
>> EAL: No free hugepages reported in hugepages-2048kB
>> PANIC in rte_eal_init():
>> Cannot get hugepage information
>> 6: [./testpmd() [0x402941]]
>> 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)
>>[0x7f7f14f50ec5]]
>> 4: [./testpmd() [0x40237e]]
>> 3: [./testpmd() [0x478edb]]
>> 2: [./testpmd() [0x4022ad]]
>> 1: [./testpmd() [0x47e3c3]]
>> Aborted
>> ubuntu@fpc0:/dpdk/x86_64-default-linuxapp-gcc/app$
>> 
>> However, running this same app on the host, or doing lxc-execute works.
>> 
>> While trying to debug my own dpdk app, I see that reading the same file
>>for no
>> of huge pages (
>> 
>> /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>> 
>>  ), on the host, its value is read 2048, while inside the container it
>>is 0. This is
>> same file.
>> Also, if you cat the contents of this file, it is actually 0!
>>(nr_hugepages is 2048,
>> but the DPDK code does not read this file.)
>> 
>> cat /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>> 0
>> 
>> 
>> In the below logs, the 2 prints "EAL: Using .." and "EAL: path ..." are
>>added by me
>> in DPDL eal library.
>> 
>> On host:
>> rte_eal_init : 859
>> opt : 99
>> opt : 110
>> EAL: Using free_hugepages
>> EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>> EAL: No of hugepages is 2048
>> 
>> Inside container:
>> pt : 99
>> opt : 110
>> EAL: Using free_hugepages
>> EAL: path /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages
>> EAL: No of hugepages is 0
>> 
>> 
>> Has anyone run into this problem and know what's the issue?
>> 
>> Anjali

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2014-07-03 18:09 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <CFDAB971.2ABC8%anjali@juniper.net>
2014-07-03 14:53 ` [dpdk-dev] Question on huge pages and running DPDK apps inside containers Shaw, Jeffrey B
2014-07-03 15:31   ` Anjali Kulkarni
2014-07-03 15:33   ` Anjali Kulkarni
2014-07-03 16:02 ` Richardson, Bruce
2014-07-03 18:09   ` Anjali Kulkarni

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).