[dpdk-users] chelsio T-580_CR FreeBSD DPDK link down

[dpdk-users] chelsio T-580_CR FreeBSD DPDK link down

[Jordan Caraballo]

Hi guys,

We currently trying to implement DPDK in FreeBSD 11-STABLE to a Dell
PE R530 equipped with
a Chelsio T-580-CR port in an 8x slot (firmware version: 1.16.45.0).
At the time of running testpmd to check the interfaces, ports are
detected but none of them are up.
We were expecting an output like the one from
http://dpdk.org/doc/guides/nics/cxgbe.html.

###############################

Installed dpdk from /usr/ports/net/dpdk in a FreeBSD 11-STABLE system.
Also tried same approach with the stable version from the website.
Unloaded every module related to dpdk and cxgbe. Then:
# kldload if_cxgbe
# dmesg | grep "t5nex"
# pciconf -l | grep "t5nex"
Note: The result given from this command is primarily  t5nex0@pci0:4:0:4:
# kldunload if_cxgbe

Added to /boot/loader.conf below configurations and rebooted the system:
reserve 2 x 1G blocks of contiguous memory using contigmem driver
# hw.contigmem.num_buffers=2
# hw.contigmem.buffer_size=1073741824

loaded contigmem module during boot process

# contigmem_load="YES"
Once the system is rebooted we proceed to attach the pci address to
nic_uio module.

# kenv hw.nic_uio.bdfs="4:0:4"
# kldload nic_uio.ko

# pciconf -l
The result of pciconf -l at the t5nex0 pci is  nic_uio@pci0:4:0:4.
Note that once I load the nic module cxlX interfaces are no longer
present at ifconfig.

Tried to run:
# /usr/local/share/dpdk/x86_64-native-bsdapp-clang/app/testpmd -l 0-3 -n 4
-w 0000:4:00.4 -- -i

And script runs but links are down. Output below. At interactive mode
I use "set link-up port 0” but it fails to bring any
of the ports up with an error of "set link-up fail".

root@router1:~/dpdk-stable-17.05.1 #
./x86_64-native-bsdapp-clang/app/testpmd -l 0-3 -n 4 -w 0000:04:00.4
-- -i
EAL: Sysctl reports 36 cpus
EAL: Detected 36 lcore(s)
EAL: Contigmem driver has 2 buffers, each of size 1GB
EAL: Mapped memory segment 1 @ 0x802400000: physaddr:0x140000000,len 1073741824
EAL: Mapped memory segment 2 @ 0x842400000: physaddr:0x180000000,len 1073741824
EAL: PCI scan found 212 devices
EAL: PCI device 0000:04:00.4 on NUMA socket 0
EAL:   probe driver: 1425:5410 net_cxgbe
PMD: rte_cxgbe_pmd: fw: 1.16.45.0, TP: 0.1.4.9
PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
PMD: rte_cxgbe_pmd: Successfully configured using Firmware
Configuration File "On Flash", version 0x1000026, computed checksum
0x2e1a8475
PMD: rte_cxgbe_pmd:  0000:04:00.4 Chelsio rev 0 40GBASE-QSFP
PMD: rte_cxgbe_pmd:  0000:04:00.4 Chelsio rev 0 40GBASE-QSFP
Interactive-mode selected
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456,
size=2176, socket=0
Configuring Port 0 (socket 0) Port 0: 00:07:43:2C:AC:50
Configuring Port 1 (socket 0) Port 1: 00:07:43:2C:AC:58
Checking link statuses...
Done

testpmd>

###################################

- Any advice or idea?
- Am I missing something?
- My aim is to increase packets per second forwarding in a router, are
we in the right track with DPDK?

In case of needing additional log information do not hesitate to ask.

Thanks in advance,

Jordan

Re: [dpdk-users] chelsio T-580_CR FreeBSD DPDK link down

[Don maillist]

I am in the midst of a debug of DPDK for a different card type so I can't
say for sure and I am not expert but, the testpmd looks fine. And the
disappearing ports is to be expected and would be replaced during use
(testpmd is a driver test only). You need something riding on top of it
like in our case OVS+DPDK which later creates a dpdk0 port. I think you are
on the right track but you are early in the process.

Best Regards,
Don

On Sat, Jul 15, 2017 at 9:05 AM, Jordan Caraballo <
jordancaraballo87@gmail.com> wrote:

> Hi guys,
>
> We currently trying to implement DPDK in FreeBSD 11-STABLE to a Dell
> PE R530 equipped with
> a Chelsio T-580-CR port in an 8x slot (firmware version: 1.16.45.0).
> At the time of running testpmd to check the interfaces, ports are
> detected but none of them are up.
> We were expecting an output like the one from
> http://dpdk.org/doc/guides/nics/cxgbe.html.
>
> ###############################
>
> Installed dpdk from /usr/ports/net/dpdk in a FreeBSD 11-STABLE system.
> Also tried same approach with the stable version from the website.
> Unloaded every module related to dpdk and cxgbe. Then:
> # kldload if_cxgbe
> # dmesg | grep "t5nex"
> # pciconf -l | grep "t5nex"
> Note: The result given from this command is primarily  t5nex0@pci0:4:0:4:
> # kldunload if_cxgbe
>
> Added to /boot/loader.conf below configurations and rebooted the system:
> reserve 2 x 1G blocks of contiguous memory using contigmem driver
> # hw.contigmem.num_buffers=2
> # hw.contigmem.buffer_size=1073741824
>
> loaded contigmem module during boot process
> # contigmem_load="YES"
> Once the system is rebooted we proceed to attach the pci address to
> nic_uio module.
>
> # kenv hw.nic_uio.bdfs="4:0:4"
> # kldload nic_uio.ko
> # pciconf -l
> The result of pciconf -l at the t5nex0 pci is  nic_uio@pci0:4:0:4.
> Note that once I load the nic module cxlX interfaces are no longer
> present at ifconfig.
>
> Tried to run:
> # /usr/local/share/dpdk/x86_64-native-bsdapp-clang/app/testpmd -l 0-3 -n 4
> -w 0000:4:00.4 -- -i
>
> And script runs but links are down. Output below. At interactive mode
> I use "set link-up port 0” but it fails to bring any
> of the ports up with an error of "set link-up fail".
>
> root@router1:~/dpdk-stable-17.05.1 #
> ./x86_64-native-bsdapp-clang/app/testpmd -l 0-3 -n 4 -w 0000:04:00.4
> -- -i
> EAL: Sysctl reports 36 cpus
> EAL: Detected 36 lcore(s)
> EAL: Contigmem driver has 2 buffers, each of size 1GB
> EAL: Mapped memory segment 1 @ 0x802400000: physaddr:0x140000000,len
> 1073741824
> EAL: Mapped memory segment 2 @ 0x842400000: physaddr:0x180000000,len
> 1073741824
> EAL: PCI scan found 212 devices
> EAL: PCI device 0000:04:00.4 on NUMA socket 0
> EAL:   probe driver: 1425:5410 net_cxgbe
> PMD: rte_cxgbe_pmd: fw: 1.16.45.0, TP: 0.1.4.9
> PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
> PMD: rte_cxgbe_pmd: Successfully configured using Firmware
> Configuration File "On Flash", version 0x1000026, computed checksum
> 0x2e1a8475
> PMD: rte_cxgbe_pmd:  0000:04:00.4 Chelsio rev 0 40GBASE-QSFP
> PMD: rte_cxgbe_pmd:  0000:04:00.4 Chelsio rev 0 40GBASE-QSFP
> Interactive-mode selected
> USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456,
> size=2176, socket=0
> Configuring Port 0 (socket 0) Port 0: 00:07:43:2C:AC:50
> Configuring Port 1 (socket 0) Port 1: 00:07:43:2C:AC:58
> Checking link statuses...
> Done
>
> testpmd>
>
> ###################################
>
> - Any advice or idea?
> - Am I missing something?
> - My aim is to increase packets per second forwarding in a router, are
> we in the right track with DPDK?
>
> In case of needing additional log information do not hesitate to ask.
>
> Thanks in advance,
>
> Jordan
>

Re: [dpdk-users] chelsio T-580_CR FreeBSD DPDK link down

[Caraballo-Vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP]]

Hi Don,

Thanks for the feedback. I will proceed then to further steps.

- Jordan



On 7/15/17 2:52 PM, Don maillist wrote:
> I am in the midst of a debug of DPDK for a different card type so I
> can't say for sure and I am not expert but, the testpmd looks fine.
> And the disappearing ports is to be expected and would be replaced
> during use (testpmd is a driver test only). You need something riding
> on top of it like in our case OVS+DPDK which later creates a dpdk0
> port. I think you are on the right track but you are early in the
> process. 
>
> Best Regards,
> Don
>
> On Sat, Jul 15, 2017 at 9:05 AM, Jordan Caraballo
> <jordancaraballo87@gmail.com <mailto:jordancaraballo87@gmail.com>> wrote:
>
>     Hi guys,
>
>     We currently trying to implement DPDK in FreeBSD 11-STABLE to a Dell
>     PE R530 equipped with
>     a Chelsio T-580-CR port in an 8x slot (firmware version: 1.16.45.0).
>     At the time of running testpmd to check the interfaces, ports are
>     detected but none of them are up.
>     We were expecting an output like the one from
>     http://dpdk.org/doc/guides/nics/cxgbe.html
>     <http://dpdk.org/doc/guides/nics/cxgbe.html>.
>
>     ###############################
>
>     Installed dpdk from /usr/ports/net/dpdk in a FreeBSD 11-STABLE system.
>     Also tried same approach with the stable version from the website.
>     Unloaded every module related to dpdk and cxgbe. Then:
>     # kldload if_cxgbe
>     # dmesg | grep "t5nex"
>     # pciconf -l | grep "t5nex"
>     Note: The result given from this command is primarily 
>     t5nex0@pci0:4:0:4:
>     # kldunload if_cxgbe
>
>     Added to /boot/loader.conf below configurations and rebooted the
>     system:
>     reserve 2 x 1G blocks of contiguous memory using contigmem driver
>     # hw.contigmem.num_buffers=2
>     # hw.contigmem.buffer_size=1073741824
>
>     loaded contigmem module during boot process
>     # contigmem_load="YES"
>     Once the system is rebooted we proceed to attach the pci address to
>     nic_uio module.
>
>     # kenv hw.nic_uio.bdfs="4:0:4"
>     # kldload nic_uio.ko
>     # pciconf -l
>     The result of pciconf -l at the t5nex0 pci is  nic_uio@pci0:4:0:4.
>     Note that once I load the nic module cxlX interfaces are no longer
>     present at ifconfig.
>
>     Tried to run:
>     # /usr/local/share/dpdk/x86_64-native-bsdapp-clang/app/testpmd -l
>     0-3 -n 4
>     -w 0000:4:00.4 -- -i
>
>     And script runs but links are down. Output below. At interactive mode
>     I use "set link-up port 0” but it fails to bring any
>     of the ports up with an error of "set link-up fail".
>
>     root@router1:~/dpdk-stable-17.05.1 #
>     ./x86_64-native-bsdapp-clang/app/testpmd -l 0-3 -n 4 -w 0000:04:00.4
>     -- -i
>     EAL: Sysctl reports 36 cpus
>     EAL: Detected 36 lcore(s)
>     EAL: Contigmem driver has 2 buffers, each of size 1GB
>     EAL: Mapped memory segment 1 @ 0x802400000:
>     physaddr:0x140000000,len 1073741824
>     EAL: Mapped memory segment 2 @ 0x842400000:
>     physaddr:0x180000000,len 1073741824
>     EAL: PCI scan found 212 devices
>     EAL: PCI device 0000:04:00.4 on NUMA socket 0
>     EAL:   probe driver: 1425:5410 net_cxgbe
>     PMD: rte_cxgbe_pmd: fw: 1.16.45.0, TP: 0.1.4.9
>     PMD: rte_cxgbe_pmd: Coming up as MASTER: Initializing adapter
>     PMD: rte_cxgbe_pmd: Successfully configured using Firmware
>     Configuration File "On Flash", version 0x1000026, computed checksum
>     0x2e1a8475
>     PMD: rte_cxgbe_pmd:  0000:04:00.4 Chelsio rev 0 40GBASE-QSFP
>     PMD: rte_cxgbe_pmd:  0000:04:00.4 Chelsio rev 0 40GBASE-QSFP
>     Interactive-mode selected
>     USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456,
>     size=2176, socket=0
>     Configuring Port 0 (socket 0) Port 0: 00:07:43:2C:AC:50
>     Configuring Port 1 (socket 0) Port 1: 00:07:43:2C:AC:58
>     Checking link statuses...
>     Done
>
>     testpmd>
>
>     ###################################
>
>     - Any advice or idea?
>     - Am I missing something?
>     - My aim is to increase packets per second forwarding in a router, are
>     we in the right track with DPDK?
>
>     In case of needing additional log information do not hesitate to ask.
>
>     Thanks in advance,
>
>     Jordan
>
>