[dpdk-users] Running DPDK as non-root

[dpdk-users] Running DPDK as non-root

[Jez Higgins]

Hello,

Is it possible to run applications that use DPDK on Linux as anything
other than root?  I think the answer is no, but I wonder if anyone has
worked it out.

5.2 of the getting started guide says

  with a number of small permission adjustments it is possible to run
these applications as a user other than “root”

but this seems insufficient.  I have changed the permissions on the
hugepages mountpoint (in my case /dev/hugepages), on /dev/uio0, on
/sys/class/uio/uio0/device/config, and on
/sys/class/uio/uio0/device/resource*

Making these changes is sufficient initialise the application.  However,
a non-root user does not seem to be able to read /proc/self/pagemap.
More specifically, a non-root user can open and read /proc/self/pagemap,
but only ever reads zeros.  This means the mapping the hugepages virtual
to physical addresses isn't correct, and so actually allocating even
reasonably small amounts of memory (for example the log buffer) is
liable to fail.

Apparently, access to /proc/self/pagemap is controlled by the
CAP_SYS_ADMIN capability.  However, if I use setcap to enable that
capability, I can't even open it, never mind read it.

I bodged around this by setting my default hugepage size to 1GB, rather
than 2MB.  The virtual to physical mapping is still wrong, but I can at
least get DPDK to initialise.  I doubt it's correct though.

At that point, if I try to initialise a port with rte_eth_dev_start it
fails.  I'm running in a vm, so using the vmxnet3 driver.  It's failing
to activate the device down in vmxnet3_dev_start.

If I run as root, however, everything fires up and runs correctly.

Is it possible to get DPDK up and running as non-root - if so, can
anyone guide me to what I'm missing? Or should I be giving this up as a
bad job?

Thanks in advance,
Jez

Re: [dpdk-users] Running DPDK as non-root

[Thomas Monjalon]

Hi

2016-07-07 16:47, Jez Higgins:
> Is it possible to get DPDK up and running as non-root - if so, can
> anyone guide me to what I'm missing? Or should I be giving this up as a
> bad job?

You can try the --no-huge option.
But most of drivers won't work without hugepage currently.
A rework of the memory allocation is needed to make it work better.

Re: [dpdk-users] Running DPDK as non-root

[Wiles, Keith]

> On Jul 7, 2016, at 11:17 AM, Thomas Monjalon <thomas.monjalon@6wind.com> wrote:
> 
> Hi
> 
> 2016-07-07 16:47, Jez Higgins:
>> Is it possible to get DPDK up and running as non-root - if so, can
>> anyone guide me to what I'm missing? Or should I be giving this up as a
>> bad job?
> 
> You can try the --no-huge option.
> But most of drivers won't work without hugepage currently.
> A rework of the memory allocation is needed to make it work better.

Last time I looked DPDK must be run as root as the virtual to physical translation in the startup needs to be root, unless the no-huge option alters the requirement. As I understand the virtual to physical translation needing to be run a root is a Linux restriction. I do not know of any work arounds.

Re: [dpdk-users] Running DPDK as non-root

[Tan, Jianfeng]

Hi,

> -----Original Message-----
> From: users [mailto:users-bounces@dpdk.org] On Behalf Of Wiles, Keith
> Sent: Friday, July 8, 2016 2:38 AM
> To: Thomas Monjalon
> Cc: Jez Higgins; users@dpdk.org; olivier.matz@6wind.com; Gonzalez Monroy,
> Sergio
> Subject: Re: [dpdk-users] Running DPDK as non-root
> 
> 
> > On Jul 7, 2016, at 11:17 AM, Thomas Monjalon
> <thomas.monjalon@6wind.com> wrote:
> >
> > Hi
> >
> > 2016-07-07 16:47, Jez Higgins:
> >> Is it possible to get DPDK up and running as non-root - if so, can
> >> anyone guide me to what I'm missing? Or should I be giving this up as a
> >> bad job?
> >
> > You can try the --no-huge option.
> > But most of drivers won't work without hugepage currently.
> > A rework of the memory allocation is needed to make it work better.
> 
> Last time I looked DPDK must be run as root as the virtual to physical
> translation in the startup needs to be root, unless the no-huge option alters
> the requirement. As I understand the virtual to physical translation needing
> to be run a root is a Linux restriction. I do not know of any work arounds.


I had a try long time ago. Firstly, to avoid using physical memory, I once sent a patch, http://dpdk.org/ml/archives/dev/2016-January/031180.html. The second, how to use DPDK with physical NIC without physical address, just use virtual address as IOVA, which needs the help of vfio-pci with iommu on.

Thanks,
Jianfeng

Re: [dpdk-users] Running DPDK as non-root

[Adrien Mazarguil]

On Thu, Jul 07, 2016 at 06:17:21PM +0200, Thomas Monjalon wrote:
> Hi
> 
> 2016-07-07 16:47, Jez Higgins:
> > Is it possible to get DPDK up and running as non-root - if so, can
> > anyone guide me to what I'm missing? Or should I be giving this up as a
> > bad job?
> 
> You can try the --no-huge option.
> But most of drivers won't work without hugepage currently.
> A rework of the memory allocation is needed to make it work better.

A little late but just for the record, mlx4 and mlx5 should run fine with
--no-huge since these devices use virtual memory addresses directly
(although I must admit this feature is not tested very often).

There is still a requirement for root in order to create raw Ethernet TX and
RX queues for security reasons, as it would be like setting a network
interface in promiscuous mode to sniff traffic (tcpdump) or creating raw
sockets (ping) as a mere user, but this may be possibly achieved (not sure,
to be verified) by running applications with the CAP_NET_ADMIN capability.

Raw Ethernet QP creation can be allowed by default without special user
capabilities by installing MLNX_OFED with the --vma flag.

A few features will still be missing such as setting interfaces up and down,
updating MTU of the physical link and so on, basically all netdevice
operations that are not queries. Those can be performed as super-user on the
related netdevice directly from a separate process, even before starting the
DPDK application.

-- 
Adrien Mazarguil
6WIND

Re: [dpdk-users] Running DPDK as non-root

[Singh, Satish 1. (Nokia - IN/Bangalore)]

Hi,

This is possible, with few permission for few filesystem as non-root user.
See section 4.3.
http://dpdk.org/doc/guides-1.8/freebsd_gsg/build_sample_apps.html



Regards………....:)- 
---------------------------------------------------- 
SATISH SINGH 
RCP-Data Plane Domain / UMW
-------------------------------------------------------------------- 
Mob. No. +91-9535568802 
-------------------------------------------------------------------- 
“I will never Lose, Either I will Win or I will Learn”

-----Original Message-----
From: users [mailto:users-bounces@dpdk.org] On Behalf Of Jez Higgins
Sent: Thursday, July 07, 2016 9:17 PM
To: users@dpdk.org
Subject: [dpdk-users] Running DPDK as non-root

Hello,

Is it possible to run applications that use DPDK on Linux as anything other than root?  I think the answer is no, but I wonder if anyone has worked it out.

5.2 of the getting started guide says

  with a number of small permission adjustments it is possible to run these applications as a user other than “root”

but this seems insufficient.  I have changed the permissions on the hugepages mountpoint (in my case /dev/hugepages), on /dev/uio0, on /sys/class/uio/uio0/device/config, and on
/sys/class/uio/uio0/device/resource*

Making these changes is sufficient initialise the application.  However, a non-root user does not seem to be able to read /proc/self/pagemap.
More specifically, a non-root user can open and read /proc/self/pagemap, but only ever reads zeros.  This means the mapping the hugepages virtual to physical addresses isn't correct, and so actually allocating even reasonably small amounts of memory (for example the log buffer) is liable to fail.

Apparently, access to /proc/self/pagemap is controlled by the CAP_SYS_ADMIN capability.  However, if I use setcap to enable that capability, I can't even open it, never mind read it.

I bodged around this by setting my default hugepage size to 1GB, rather than 2MB.  The virtual to physical mapping is still wrong, but I can at least get DPDK to initialise.  I doubt it's correct though.

At that point, if I try to initialise a port with rte_eth_dev_start it fails.  I'm running in a vm, so using the vmxnet3 driver.  It's failing to activate the device down in vmxnet3_dev_start.

If I run as root, however, everything fires up and runs correctly.

Is it possible to get DPDK up and running as non-root - if so, can anyone guide me to what I'm missing? Or should I be giving this up as a bad job?

Thanks in advance,
Jez

Re: [dpdk-users] Running DPDK as non-root

[Singh, Satish 1. (Nokia - IN/Bangalore)]

Hi,

This is possible, with few permission for few filesystem as non-root user.
See section 4.3.
http://dpdk.org/doc/guides-1.8/freebsd_gsg/build_sample_apps.html



-----Original Message-----
From: users [mailto:users-bounces@dpdk.org] On Behalf Of Jez Higgins
Sent: Thursday, July 07, 2016 9:17 PM
To: users@dpdk.org
Subject: [dpdk-users] Running DPDK as non-root

Hello,

Is it possible to run applications that use DPDK on Linux as anything other than root?  I think the answer is no, but I wonder if anyone has worked it out.

5.2 of the getting started guide says

  with a number of small permission adjustments it is possible to run these applications as a user other than “root”

but this seems insufficient.  I have changed the permissions on the hugepages mountpoint (in my case /dev/hugepages), on /dev/uio0, on /sys/class/uio/uio0/device/config, and on
/sys/class/uio/uio0/device/resource*

Making these changes is sufficient initialise the application.  However, a non-root user does not seem to be able to read /proc/self/pagemap.
More specifically, a non-root user can open and read /proc/self/pagemap, but only ever reads zeros.  This means the mapping the hugepages virtual to physical addresses isn't correct, and so actually allocating even reasonably small amounts of memory (for example the log buffer) is liable to fail.

Apparently, access to /proc/self/pagemap is controlled by the CAP_SYS_ADMIN capability.  However, if I use setcap to enable that capability, I can't even open it, never mind read it.

I bodged around this by setting my default hugepage size to 1GB, rather than 2MB.  The virtual to physical mapping is still wrong, but I can at least get DPDK to initialise.  I doubt it's correct though.

At that point, if I try to initialise a port with rte_eth_dev_start it fails.  I'm running in a vm, so using the vmxnet3 driver.  It's failing to activate the device down in vmxnet3_dev_start.

If I run as root, however, everything fires up and runs correctly.

Is it possible to get DPDK up and running as non-root - if so, can anyone guide me to what I'm missing? Or should I be giving this up as a bad job?

Thanks in advance,
Jez

Re: [dpdk-users] Running DPDK as non-root

[Jez Higgins]

On 07/07/16 16:47, Jez Higgins wrote:
> Is it possible to get DPDK up and running as non-root - if so, can
> anyone guide me to what I'm missing? Or should I be giving this up as a
> bad job?

Thanks to everyone who replied - all very helpful.  We've poked around
further here and now spinning up as root, but setgid/setuid once DPDK
intialisation is complete.  That seems to satisfy both DPDK on the one
side, and our clients on the other :)

Jez

[dpdk-users] Running DPDK as non-root

[John Ousterhout]

I'm trying to run a DPDK application as a normal user without root
privilege, and saw the instructions here:

http://dpdk.org/doc/guides/linux_gsg/enable_func.html

However, when I try this I get the following log messages:

EAL: Detected 8 lcore(s)
EAL: Probing VFIO support...
EAL: Cannot open /proc/self/pagemap: Operation not permitted. virt2phys
address translation will not work
PANIC in rte_eal_init():
Cannot init memory

I tried to change the permissions on /proc/self/pagemap, but that is
disallowed, even if I try it as root.

Any suggestions on how to get around this problem?

Thanks in advance for the help.

-John-

Re: [dpdk-users] Running DPDK as non-root

[Wiles, Keith]

Regards,
Keith

> On Aug 19, 2016, at 1:16 PM, John Ousterhout <ouster@cs.stanford.edu> wrote:
> 
> I'm trying to run a DPDK application as a normal user without root
> privilege, and saw the instructions here:
> 
> http://dpdk.org/doc/guides/linux_gsg/enable_func.html
> 
> However, when I try this I get the following log messages:
> 
> EAL: Detected 8 lcore(s)
> EAL: Probing VFIO support...
> EAL: Cannot open /proc/self/pagemap: Operation not permitted. virt2phys
> address translation will not work
> PANIC in rte_eal_init():
> Cannot init memory
> 
> I tried to change the permissions on /proc/self/pagemap, but that is
> disallowed, even if I try it as root.
> 
> Any suggestions on how to get around this problem?

I have tried a couple times to make this happen for my development, but without any luck.

I believe one of the big problems is related the virt2phy address translation is done in the kernel and it is seen as a security hole if a user level application is able to access that information.


> 
> Thanks in advance for the help.
> 
> -John-

Re: [dpdk-users] Running DPDK as non-root

[Vincent JARDIN]

John,

> EAL: Detected 8 lcore(s)
> EAL: Probing VFIO support...
> EAL: Cannot open /proc/self/pagemap: Operation not permitted. virt2phys
> address translation will not work
> PANIC in rte_eal_init():
> Cannot init memory
>
> I tried to change the permissions on /proc/self/pagemap, but that is
> disallowed, even if I try it as root.
>
> Any suggestions on how to get around this problem?

assuming that you are using a Mellanox CX3 or CX4 PCI boards, you maybe 
able to update the code to avoid access to pagemap. pagemap is used to 
get and then cache the physaddr. However, since Verbs are used to 
register memory by these 2 PMDs, the physical address is not needed.

A dirty hack could be to store any dummy values, and you should be good 
almost to go.

best regards,
   Vincent