[dpdk-dev] Distributor example app not working for 1G ports (I354)

[dpdk-dev] Distributor example app not working for 1G ports (I354)

[Aws Ismail]

Hi DPDKers,

I have a strange case where the Distributor example application is not
working when I use Intel's I35x (I354 specifically) 1Gig ports. The
app initializes fine but the port always comes with link down:

<snippet of the output>

[root@localhost dpdk]# ./tools/dpdk_nic_bind.py -b igb_uio eth1 eth2

[root@localhost dpdk]# ./tools/dpdk_nic_bind.py --status



Network devices using DPDK-compatible driver

============================================

0000:00:14.1 'Ethernet Connection I354' drv=igb_uio unused=igb

0000:00:14.2 'Ethernet Connection I354' drv=igb_uio unused=igb



Network devices using kernel driver

===================================

0000:00:14.0 'Ethernet Connection I354' if=eth0 drv=igb unused=igb_uio *Active*

0000:00:14.3 'Ethernet Connection I354' if=eth3 drv=igb unused=igb_uio



Other network devices

=====================

0000:04:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection'
unused=ixgbe,igb_uio

0000:04:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection'
unused=ixgbe,igb_uio



[root@localhost dpdk]# cd examples/distributor/

[root@localhost distributor]# ./build/distributor_app -c f -n 4 -- -p 3

EAL: Detected lcore 0 as core 0 on socket 0

EAL: Detected lcore 1 as core 1 on socket 0

EAL: Detected lcore 2 as core 2 on socket 0

EAL: Detected lcore 3 as core 3 on socket 0

EAL: Detected lcore 4 as core 4 on socket 0

EAL: Detected lcore 5 as core 5 on socket 0

EAL: Detected lcore 6 as core 6 on socket 0

EAL: Detected lcore 7 as core 7 on socket 0

EAL: Support maximum 128 logical core(s) by configuration.

EAL: Detected 8 lcore(s)

EAL: VFIO modules not all loaded, skip VFIO support...

EAL: Setting up memory...

EAL: Ask a virtual area of 0x400000 bytes

EAL: Virtual area found at 0x7fa5ff000000 (size = 0x400000)

EAL: Ask a virtual area of 0x1400000 bytes

EAL: Virtual area found at 0x7fa5fda00000 (size = 0x1400000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7fa5fd600000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7fa5fd200000 (size = 0x200000)

EAL: Ask a virtual area of 0x28c00000 bytes

EAL: Virtual area found at 0x7fa5d4400000 (size = 0x28c00000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7fa5d4000000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7fa5d3c00000 (size = 0x200000)

EAL: Ask a virtual area of 0x55000000 bytes

EAL: Virtual area found at 0x7fa57ea00000 (size = 0x55000000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7fa57e600000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7fa57e200000 (size = 0x200000)

EAL: Requesting 1024 pages of size 2MB from socket 0

EAL: TSC frequency is ~2399971 KHz

EAL: Master lcore 0 is ready (tid=13568c0;cpuset=[0])

PMD: ENICPMD trace: rte_enic_pmd_init

EAL: lcore 3 is ready (tid=7d1fd700;cpuset=[3])

EAL: lcore 2 is ready (tid=7d9fe700;cpuset=[2])

EAL: lcore 1 is ready (tid=7e1ff700;cpuset=[1])

EAL: PCI device 0000:00:14.0 on NUMA socket -1

EAL:   probe driver: 8086:1f41 rte_igb_pmd

EAL:   Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:00:14.1 on NUMA socket -1

EAL:   probe driver: 8086:1f41 rte_igb_pmd

EAL:   PCI memory mapped at 0x7fa5ff400000

EAL:   PCI memory mapped at 0x7fa5ff420000

PMD: eth_igb_dev_init(): port_id 0 vendorID=0x8086 deviceID=0x1f41

EAL: PCI device 0000:00:14.2 on NUMA socket -1

EAL:   probe driver: 8086:1f41 rte_igb_pmd

EAL:   PCI memory mapped at 0x7fa5ff424000

EAL:   PCI memory mapped at 0x7fa5ff444000

PMD: eth_igb_dev_init(): port_id 1 vendorID=0x8086 deviceID=0x1f41

EAL: PCI device 0000:00:14.3 on NUMA socket -1

EAL:   probe driver: 8086:1f41 rte_igb_pmd

EAL:   Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:04:00.0 on NUMA socket -1

EAL:   probe driver: 8086:10fb rte_ixgbe_pmd

EAL:   Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:04:00.1 on NUMA socket -1

EAL:   probe driver: 8086:10fb rte_ixgbe_pmd

EAL:   Not managed by a supported kernel driver, skipped

Initializing port 0... done

PMD: eth_igb_rx_queue_setup(): sw_ring=0x7fa5fe4f3500
hw_ring=0x7fa5ff3a9580 dma_addr=0x27a9580

PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance,
consider setting the TX WTHRESH value to 4, 8, or 16.

PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4f1400
hw_ring=0x7fa5ff3b9580 dma_addr=0x27b9580

PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance,
consider setting the TX WTHRESH value to 4, 8, or 16.

PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4ef300
hw_ring=0x7fa5ff3c9580 dma_addr=0x27c9580

PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance,
consider setting the TX WTHRESH value to 4, 8, or 16.

PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4ed200
hw_ring=0x7fa5ff3d9580 dma_addr=0x27d9580

PMD: eth_igb_start(): <<

Link down on port 0

Initializing port 1... done

PMD: eth_igb_rx_queue_setup(): sw_ring=0x7fa5fe4ec800
hw_ring=0x7fa5ff3e9580 dma_addr=0x27e9580

PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance,
consider setting the TX WTHRESH value to 4, 8, or 16.

PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4ea700
hw_ring=0x7fa5fd602080 dma_addr=0x36e02080

PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance,
consider setting the TX WTHRESH value to 4, 8, or 16.

PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4e8600
hw_ring=0x7fa5fd612080 dma_addr=0x36e12080

PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance,
consider setting the TX WTHRESH value to 4, 8, or 16.

PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4e6500
hw_ring=0x7fa5fd622080 dma_addr=0x36e22080

PMD: eth_igb_start(): <<

Link down on port 1



Core 1 acting as worker core.



Core 2 acting as worker core.



Core 3 doing packet TX.



Core 0 doing packet RX.

^CExiting on signal 2



RX thread stats:

- Received:    0

- Processed:   0

- Enqueued:    0



TX thread stats:

- Dequeued:    0

- Transmitted: 0



Port 0 stats:

- Pkts in:   0

- Pkts out:  0

- In Errs:   0

- Out Errs:  0

- Mbuf Errs: 0



Port 1 stats:

- Pkts in:   0

- Pkts out:  0

- In Errs:   0

- Out Errs:  0

- Mbuf Errs: 0





I say strange because the same example works fine with the 10Gig 82599
ports shown in the dpdk_nic_status output above:

(root@localhost) # ./distributor_app -c 0xf0 -n 4 -- -p 3

EAL: Detected lcore 0 as core 0 on socket 0

EAL: Detected lcore 1 as core 1 on socket 0

EAL: Detected lcore 2 as core 2 on socket 0

EAL: Detected lcore 3 as core 3 on socket 0

EAL: Detected lcore 4 as core 4 on socket 0

EAL: Detected lcore 5 as core 5 on socket 0

EAL: Detected lcore 6 as core 6 on socket 0

EAL: Detected lcore 7 as core 7 on socket 0

EAL: Support maximum 128 logical core(s) by configuration.

EAL: Detected 8 lcore(s)

EAL: VFIO modules not all loaded, skip VFIO support...

EAL: Setting up memory...

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f50f0000000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f50efc00000 (size = 0x200000)

EAL: Ask a virtual area of 0x1c400000 bytes

EAL: Virtual area found at 0x7f50d3600000 (size = 0x1c400000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f50d3200000 (size = 0x200000)

EAL: Ask a virtual area of 0xe2400000 bytes

EAL: Virtual area found at 0x7f4ff0c00000 (size = 0xe2400000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f4ff0800000 (size = 0x200000)

EAL: Ask a virtual area of 0x400000 bytes

EAL: Virtual area found at 0x7f4ff0200000 (size = 0x400000)

EAL: Ask a virtual area of 0x800000 bytes

EAL: Virtual area found at 0x7f4fef800000 (size = 0x800000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f4fef400000 (size = 0x200000)

EAL: Ask a virtual area of 0x200000 bytes

EAL: Virtual area found at 0x7f4fef000000 (size = 0x200000)

EAL: Requesting 2048 pages of size 2MB from socket 0

EAL: TSC frequency is ~2399999 KHz

EAL: Master lcore 4 is ready (tid=f2568900;cpuset=[4])

PMD: ENICPMD trace: rte_enic_pmd_init

EAL: lcore 5 is ready (tid=eefff700;cpuset=[5])

EAL: lcore 7 is ready (tid=edffd700;cpuset=[7])

EAL: lcore 6 is ready (tid=ee7fe700;cpuset=[6])

EAL: PCI device 0000:00:14.0 on NUMA socket -1

EAL:   probe driver: 8086:1f41 rte_igb_pmd

EAL:   Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:00:14.1 on NUMA socket -1

EAL:   probe driver: 8086:1f41 rte_igb_pmd

EAL:   Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:00:14.2 on NUMA socket -1

EAL:   probe driver: 8086:1f41 rte_igb_pmd

EAL:   Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:00:14.3 on NUMA socket -1

EAL:   probe driver: 8086:1f41 rte_igb_pmd

EAL:   Not managed by a supported kernel driver, skipped

EAL: PCI device 0000:04:00.0 on NUMA socket -1

EAL:   probe driver: 8086:154d rte_ixgbe_pmd

EAL:   PCI memory mapped at 0x7f50f0200000

EAL:   PCI memory mapped at 0x7f50f0280000

PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 17, SFP+: 5

PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 deviceID=0x154d

EAL: PCI device 0000:04:00.1 on NUMA socket -1

EAL:   probe driver: 8086:154d rte_ixgbe_pmd

EAL:   PCI memory mapped at 0x7f50f0284000

EAL:   PCI memory mapped at 0x7f50f0304000

PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 17, SFP+: 6

PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x154d

Initializing port 0... done

PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f50d40ed480
hw_ring=0x7f4ff05a9580 dma_addr=0x3823a9580

PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40eb340
hw_ring=0x7f4ff05b9600 dma_addr=0x3823b9600

PMD: ixgbe_set_tx_function(): Using simple tx code path

PMD: ixgbe_set_tx_function(): Vector tx enabled.

PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40e9200
hw_ring=0x7f4ff05c9600 dma_addr=0x3823c9600

PMD: ixgbe_set_tx_function(): Using simple tx code path

PMD: ixgbe_set_tx_function(): Vector tx enabled.

PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40e70c0
hw_ring=0x7f4ff05d9600 dma_addr=0x3823d9600

PMD: ixgbe_set_tx_function(): Using simple tx code path

PMD: ixgbe_set_tx_function(): Vector tx enabled.

PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX
burst size no less than 32.

Port 0 MAC: a0 36 9f 48 c1 c0

Initializing port 1... done

PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f50d40e6340
hw_ring=0x7f4ff05e9600 dma_addr=0x3823e9600

PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40e4200
hw_ring=0x7f50f0002080 dma_addr=0x36c02080

PMD: ixgbe_set_tx_function(): Using simple tx code path

PMD: ixgbe_set_tx_function(): Vector tx enabled.

PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40e20c0
hw_ring=0x7f50f0012080 dma_addr=0x36c12080

PMD: ixgbe_set_tx_function(): Using simple tx code path

PMD: ixgbe_set_tx_function(): Vector tx enabled.

PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40dff80
hw_ring=0x7f50f0022080 dma_addr=0x36c22080

PMD: ixgbe_set_tx_function(): Using simple tx code path

PMD: ixgbe_set_tx_function(): Vector tx enabled.

PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX
burst size no less than 32.

Port 1 MAC: a0 36 9f 48 c1 c2



Core 5 acting as worker core.



Core 6 acting as worker core.



Core 7 doing packet TX.



Core 4 doing packet RX.







Then I send a SIGINT signal to the distributor process to dump the
stats (root@localhost:~# kill -SIGINT 6315):



Exiting on signal 2



RX thread stats:

- Received:    109348320

- Processed:   109348311

- Enqueued:    109348311



TX thread stats:

- Dequeued:    109348311

- Transmitted: 109348311



Port 0 stats:

- Pkts in:   54674335

- Pkts out:  54674167

- In Errs:   44718519

- Out Errs:  0

- Mbuf Errs: 0



Port 1 stats:

- Pkts in:   54674367

- Pkts out:  54674144

- In Errs:   46955093

- Out Errs:  0

- Mbuf Errs: 0


Has anyone seen this behavior before? I am building  DPDK from the
v2.0.0 tag in the git repo.

Re: [dpdk-dev] Distributor example app not working for 1G ports (I354)

[Bruce Richardson]

On Fri, Jun 26, 2015 at 03:37:08PM -0400, Aws Ismail wrote:
> Hi DPDKers,
> 
> I have a strange case where the Distributor example application is not
> working when I use Intel's I35x (I354 specifically) 1Gig ports. The
> app initializes fine but the port always comes with link down:
>
Do other apps, e.g. testpmd, or l2fwd work with the 1G port, or is it only the
distributor app which is affected by this?

/Bruce

Re: [dpdk-dev] Distributor example app not working for 1G ports (I354)

[Aws Ismail]

Hi Bruce,

As far as I have checked, testpmd and l2fwd apps work fine.

Aws\

On Mon, Jun 29, 2015 at 5:19 AM, Bruce Richardson
<bruce.richardson@intel.com> wrote:
> On Fri, Jun 26, 2015 at 03:37:08PM -0400, Aws Ismail wrote:
>> Hi DPDKers,
>>
>> I have a strange case where the Distributor example application is not
>> working when I use Intel's I35x (I354 specifically) 1Gig ports. The
>> app initializes fine but the port always comes with link down:
>>
> Do other apps, e.g. testpmd, or l2fwd work with the 1G port, or is it only the
> distributor app which is affected by this?
>
> /Bruce