Re: [dpdk-users] DPDK mlx4 PMD on Azure VM

[Hui Ling <kelvin.brookletling@gmail.com>]

Ophir,

Here it is:

root@myVM:# ./build/app/testpmd -l 1-2 -n 4 -w 0003:00:02.0 -- -i
--port-topology=chained
EAL: Detected 4 lcore(s)
EAL: 2 hugepages of size 1073741824 reserved, but no mounted hugetlbfs
found for that size
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
unreliable clock cycles !
EAL: PCI device 0003:00:02.0 on NUMA socket 0
EAL:   probe driver: 15b3:1004 net_mlx4
PMD: mlx4.c:465: mlx4_pci_probe(): PCI information matches, using
device "mlx4_2" (VF: true)
PMD: mlx4.c:492: mlx4_pci_probe(): 1 port(s) detected
PMD: mlx4.c:586: mlx4_pci_probe(): port 1 MAC address is 00:0d:3a:f9:08:0b
Interactive-mode selected
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456,
size=2176, socket=0
Configuring Port 0 (socket 0)
PMD: mlx4_rxq.c:811: mlx4_rx_queue_setup(): 0xde0740: MR creation
failure: Operation not permitted
Fail to configure port 0 rx queues
EAL: Error - exiting with code: 1
  Cause: Start ports failed

Same "Operation not permitted" error as before.

Do you have a working setup for MLX4 in DPDK 17.11? I can start from
scratch to build a new VM.
Giving all the information I gathered so far from DPDK, I can't seem
to make it work in my VM.

This is my VM info in case it is needed.
=======================================================================================================
A Standard_DS3_v2 instance from Azure. (one of these models support AN)

Kernel 4.11.0-1016-azure,  (I think from somewhere I saw that 4.14
kernel is required for the DPDK solution in
MS Azure to work. So I tried to update kernel to 4.14, but It didn't
seem to help solve the MLX4 driver problem)

Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial

ii  ibverbs-providers:amd64             42mlnx2-1.42120
            amd64        User space provider drivers for libibverbs
ii  ibverbs-utils                       42mlnx2-1.42120
            amd64        Examples for the libibverbs library
ii  libibverbs-dev:amd64                42mlnx2-1.42120
            amd64        Development files for the libibverbs library
ii  libibverbs1:amd64                   42mlnx2-1.42120
            amd64        Library for direct userspace use of RDMA
(InfiniBand/iWARP)
ii  librdmacm-dev                       42mlnx2-1.42120
            amd64        Development files for the librdmacm library
ii  librdmacm1                          42mlnx2-1.42120
            amd64        Library for managing RDMA connections
ii  rdma-core                           42mlnx2-1.42120
            amd64        RDMA core userspace infrastructure and
documentation
ii  rdmacm-utils                        42mlnx2-1.42120
            amd64        Examples for the librdmacm library

DPDK 17.11

and some mem info:

HugePages_Total:    1024
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
=========================================================================================================

On Wed, Dec 20, 2017 at 12:29 AM, Ophir Munk <ophirmu@mellanox.com> wrote:
> Hi Hui,
> Can you please let know if running testpmd with just one PCI device you are getting the same error?
>
>> -----Original Message-----
>> From: Thomas Monjalon [mailto:thomas@monjalon.net]
>> Sent: Tuesday, December 19, 2017 6:22 PM
>> To: Hui Ling <kelvin.brookletling@gmail.com>
>> Cc: users@dpdk.org; Ophir Munk <ophirmu@mellanox.com>
>> Subject: Re: [dpdk-users] DPDK mlx4 PMD on Azure VM
>>
>> 19/12/2017 08:14, Hui Ling:
>> > I installed DPDK 17.11 on Ubuntu 16.04. And I downloaded MLNX OFED
>> > 4.2-1.2.0.0 and installed up-stream libs with ./mlnxofedinstall
>> > --guest --dpdk --upstream-libs
>> >
>> > MLX4 PMD in DPDK doesn't seem to work with lib from ubuntu repository
>> > and install OFED allows me to compile DPDK mlx4 PMD without any
>> > compilation problem.
>> >
>> > Then I tried to see if the mlx4 PMD works or not by running:
>> >
>> > root@myVM:
>> > ./build/app/testpmd -l 1-2 -n 4 -w 0003:00:02.0 -w 0004:00:02.0 --
>> > --rxq=2 --txq=2 -i
>> [...]
>> > Configuring Port 0 (socket 0)
>> > PMD: mlx4_rxq.c:811: mlx4_rx_queue_setup(): 0xde0740: MR creation
>> > failure: Operation not permitted
>>
>> [...]
>> > I also tried to run DPDK 17.11 on Ubuntu 17.10. It didn't work either.
>> > the testpmd hangs during "configuring Port 0" forever.
>>
>> So you see 2 different errors on Ubuntu 16.04 and 17.10.
>> What are the Linux kernel versions?
>>
>> > Can someone from MS or Mellanox help me figure out why? and how to
>> > make mlx4 PMD work on Azure VM?
>>
>> Mellanox will support you.