From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qk0-f175.google.com (mail-qk0-f175.google.com [209.85.220.175]) by dpdk.org (Postfix) with ESMTP id D6C4CC9EA for ; Fri, 26 Jun 2015 21:37:27 +0200 (CEST) Received: by qkeo142 with SMTP id o142so60410113qke.1 for ; Fri, 26 Jun 2015 12:37:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=7Vl82bUpH4ywywZOKvdAF1FcHw4ElWptr4pnz1y/kXc=; b=TR1j/aM4KEWdmdbirDRXg712nraKQ+bD6qDYR2c3WcW6REyVGEd5v+SG0BxjIYWY0C Ls8LwlBvm5iUaHu3RQ0x8rg6iln3709H6BLypNkaTzphlOW2qNwj8ar9gh6VcSs0lsIB 6WX5WWfexsI4n1fbyIwAYb17e/CrSXvhBrT8FKQbCp3KPWwOw/lvjdqegQ0cZadvVf1E dRLNJDIb2K4j7dsUNBwuuUx8uKlwZWspXPrfpLk4apH0CGlzdIEByN8G8c8hAuJIbit2 wsz4+vX9huHKKGXN/6p9p1i5YQL0ObKzxiZfwuOekyeItXEGcnsruVvAqeBjGDRrxWvy Kv3g== X-Received: by 10.55.21.42 with SMTP id f42mr7316783qkh.96.1435347447437; Fri, 26 Jun 2015 12:37:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.140.40.112 with HTTP; Fri, 26 Jun 2015 12:37:08 -0700 (PDT) From: Aws Ismail Date: Fri, 26 Jun 2015 15:37:08 -0400 Message-ID: To: dev@dpdk.org Content-Type: text/plain; charset=UTF-8 Subject: [dpdk-dev] Distributor example app not working for 1G ports (I354) X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2015 19:37:28 -0000 Hi DPDKers, I have a strange case where the Distributor example application is not working when I use Intel's I35x (I354 specifically) 1Gig ports. The app initializes fine but the port always comes with link down: [root@localhost dpdk]# ./tools/dpdk_nic_bind.py -b igb_uio eth1 eth2 [root@localhost dpdk]# ./tools/dpdk_nic_bind.py --status Network devices using DPDK-compatible driver ============================================ 0000:00:14.1 'Ethernet Connection I354' drv=igb_uio unused=igb 0000:00:14.2 'Ethernet Connection I354' drv=igb_uio unused=igb Network devices using kernel driver =================================== 0000:00:14.0 'Ethernet Connection I354' if=eth0 drv=igb unused=igb_uio *Active* 0000:00:14.3 'Ethernet Connection I354' if=eth3 drv=igb unused=igb_uio Other network devices ===================== 0000:04:00.0 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=ixgbe,igb_uio 0000:04:00.1 '82599ES 10-Gigabit SFI/SFP+ Network Connection' unused=ixgbe,igb_uio [root@localhost dpdk]# cd examples/distributor/ [root@localhost distributor]# ./build/distributor_app -c f -n 4 -- -p 3 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 6 on socket 0 EAL: Detected lcore 7 as core 7 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 8 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up memory... EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7fa5ff000000 (size = 0x400000) EAL: Ask a virtual area of 0x1400000 bytes EAL: Virtual area found at 0x7fa5fda00000 (size = 0x1400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa5fd600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa5fd200000 (size = 0x200000) EAL: Ask a virtual area of 0x28c00000 bytes EAL: Virtual area found at 0x7fa5d4400000 (size = 0x28c00000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa5d4000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa5d3c00000 (size = 0x200000) EAL: Ask a virtual area of 0x55000000 bytes EAL: Virtual area found at 0x7fa57ea00000 (size = 0x55000000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa57e600000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7fa57e200000 (size = 0x200000) EAL: Requesting 1024 pages of size 2MB from socket 0 EAL: TSC frequency is ~2399971 KHz EAL: Master lcore 0 is ready (tid=13568c0;cpuset=[0]) PMD: ENICPMD trace: rte_enic_pmd_init EAL: lcore 3 is ready (tid=7d1fd700;cpuset=[3]) EAL: lcore 2 is ready (tid=7d9fe700;cpuset=[2]) EAL: lcore 1 is ready (tid=7e1ff700;cpuset=[1]) EAL: PCI device 0000:00:14.0 on NUMA socket -1 EAL: probe driver: 8086:1f41 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:00:14.1 on NUMA socket -1 EAL: probe driver: 8086:1f41 rte_igb_pmd EAL: PCI memory mapped at 0x7fa5ff400000 EAL: PCI memory mapped at 0x7fa5ff420000 PMD: eth_igb_dev_init(): port_id 0 vendorID=0x8086 deviceID=0x1f41 EAL: PCI device 0000:00:14.2 on NUMA socket -1 EAL: probe driver: 8086:1f41 rte_igb_pmd EAL: PCI memory mapped at 0x7fa5ff424000 EAL: PCI memory mapped at 0x7fa5ff444000 PMD: eth_igb_dev_init(): port_id 1 vendorID=0x8086 deviceID=0x1f41 EAL: PCI device 0000:00:14.3 on NUMA socket -1 EAL: probe driver: 8086:1f41 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:04:00.0 on NUMA socket -1 EAL: probe driver: 8086:10fb rte_ixgbe_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:04:00.1 on NUMA socket -1 EAL: probe driver: 8086:10fb rte_ixgbe_pmd EAL: Not managed by a supported kernel driver, skipped Initializing port 0... done PMD: eth_igb_rx_queue_setup(): sw_ring=0x7fa5fe4f3500 hw_ring=0x7fa5ff3a9580 dma_addr=0x27a9580 PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance, consider setting the TX WTHRESH value to 4, 8, or 16. PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4f1400 hw_ring=0x7fa5ff3b9580 dma_addr=0x27b9580 PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance, consider setting the TX WTHRESH value to 4, 8, or 16. PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4ef300 hw_ring=0x7fa5ff3c9580 dma_addr=0x27c9580 PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance, consider setting the TX WTHRESH value to 4, 8, or 16. PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4ed200 hw_ring=0x7fa5ff3d9580 dma_addr=0x27d9580 PMD: eth_igb_start(): << Link down on port 0 Initializing port 1... done PMD: eth_igb_rx_queue_setup(): sw_ring=0x7fa5fe4ec800 hw_ring=0x7fa5ff3e9580 dma_addr=0x27e9580 PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance, consider setting the TX WTHRESH value to 4, 8, or 16. PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4ea700 hw_ring=0x7fa5fd602080 dma_addr=0x36e02080 PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance, consider setting the TX WTHRESH value to 4, 8, or 16. PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4e8600 hw_ring=0x7fa5fd612080 dma_addr=0x36e12080 PMD: eth_igb_tx_queue_setup(): To improve 1G driver performance, consider setting the TX WTHRESH value to 4, 8, or 16. PMD: eth_igb_tx_queue_setup(): sw_ring=0x7fa5fe4e6500 hw_ring=0x7fa5fd622080 dma_addr=0x36e22080 PMD: eth_igb_start(): << Link down on port 1 Core 1 acting as worker core. Core 2 acting as worker core. Core 3 doing packet TX. Core 0 doing packet RX. ^CExiting on signal 2 RX thread stats: - Received: 0 - Processed: 0 - Enqueued: 0 TX thread stats: - Dequeued: 0 - Transmitted: 0 Port 0 stats: - Pkts in: 0 - Pkts out: 0 - In Errs: 0 - Out Errs: 0 - Mbuf Errs: 0 Port 1 stats: - Pkts in: 0 - Pkts out: 0 - In Errs: 0 - Out Errs: 0 - Mbuf Errs: 0 I say strange because the same example works fine with the 10Gig 82599 ports shown in the dpdk_nic_status output above: (root@localhost) # ./distributor_app -c 0xf0 -n 4 -- -p 3 EAL: Detected lcore 0 as core 0 on socket 0 EAL: Detected lcore 1 as core 1 on socket 0 EAL: Detected lcore 2 as core 2 on socket 0 EAL: Detected lcore 3 as core 3 on socket 0 EAL: Detected lcore 4 as core 4 on socket 0 EAL: Detected lcore 5 as core 5 on socket 0 EAL: Detected lcore 6 as core 6 on socket 0 EAL: Detected lcore 7 as core 7 on socket 0 EAL: Support maximum 128 logical core(s) by configuration. EAL: Detected 8 lcore(s) EAL: VFIO modules not all loaded, skip VFIO support... EAL: Setting up memory... EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f50f0000000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f50efc00000 (size = 0x200000) EAL: Ask a virtual area of 0x1c400000 bytes EAL: Virtual area found at 0x7f50d3600000 (size = 0x1c400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f50d3200000 (size = 0x200000) EAL: Ask a virtual area of 0xe2400000 bytes EAL: Virtual area found at 0x7f4ff0c00000 (size = 0xe2400000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f4ff0800000 (size = 0x200000) EAL: Ask a virtual area of 0x400000 bytes EAL: Virtual area found at 0x7f4ff0200000 (size = 0x400000) EAL: Ask a virtual area of 0x800000 bytes EAL: Virtual area found at 0x7f4fef800000 (size = 0x800000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f4fef400000 (size = 0x200000) EAL: Ask a virtual area of 0x200000 bytes EAL: Virtual area found at 0x7f4fef000000 (size = 0x200000) EAL: Requesting 2048 pages of size 2MB from socket 0 EAL: TSC frequency is ~2399999 KHz EAL: Master lcore 4 is ready (tid=f2568900;cpuset=[4]) PMD: ENICPMD trace: rte_enic_pmd_init EAL: lcore 5 is ready (tid=eefff700;cpuset=[5]) EAL: lcore 7 is ready (tid=edffd700;cpuset=[7]) EAL: lcore 6 is ready (tid=ee7fe700;cpuset=[6]) EAL: PCI device 0000:00:14.0 on NUMA socket -1 EAL: probe driver: 8086:1f41 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:00:14.1 on NUMA socket -1 EAL: probe driver: 8086:1f41 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:00:14.2 on NUMA socket -1 EAL: probe driver: 8086:1f41 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:00:14.3 on NUMA socket -1 EAL: probe driver: 8086:1f41 rte_igb_pmd EAL: Not managed by a supported kernel driver, skipped EAL: PCI device 0000:04:00.0 on NUMA socket -1 EAL: probe driver: 8086:154d rte_ixgbe_pmd EAL: PCI memory mapped at 0x7f50f0200000 EAL: PCI memory mapped at 0x7f50f0280000 PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 17, SFP+: 5 PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 deviceID=0x154d EAL: PCI device 0000:04:00.1 on NUMA socket -1 EAL: probe driver: 8086:154d rte_ixgbe_pmd EAL: PCI memory mapped at 0x7f50f0284000 EAL: PCI memory mapped at 0x7f50f0304000 PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 17, SFP+: 6 PMD: eth_ixgbe_dev_init(): port 1 vendorID=0x8086 deviceID=0x154d Initializing port 0... done PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f50d40ed480 hw_ring=0x7f4ff05a9580 dma_addr=0x3823a9580 PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40eb340 hw_ring=0x7f4ff05b9600 dma_addr=0x3823b9600 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40e9200 hw_ring=0x7f4ff05c9600 dma_addr=0x3823c9600 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40e70c0 hw_ring=0x7f4ff05d9600 dma_addr=0x3823d9600 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 32. Port 0 MAC: a0 36 9f 48 c1 c0 Initializing port 1... done PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f50d40e6340 hw_ring=0x7f4ff05e9600 dma_addr=0x3823e9600 PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40e4200 hw_ring=0x7f50f0002080 dma_addr=0x36c02080 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40e20c0 hw_ring=0x7f50f0012080 dma_addr=0x36c12080 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f50d40dff80 hw_ring=0x7f50f0022080 dma_addr=0x36c22080 PMD: ixgbe_set_tx_function(): Using simple tx code path PMD: ixgbe_set_tx_function(): Vector tx enabled. PMD: ixgbe_set_rx_function(): Vector rx enabled, please make sure RX burst size no less than 32. Port 1 MAC: a0 36 9f 48 c1 c2 Core 5 acting as worker core. Core 6 acting as worker core. Core 7 doing packet TX. Core 4 doing packet RX. Then I send a SIGINT signal to the distributor process to dump the stats (root@localhost:~# kill -SIGINT 6315): Exiting on signal 2 RX thread stats: - Received: 109348320 - Processed: 109348311 - Enqueued: 109348311 TX thread stats: - Dequeued: 109348311 - Transmitted: 109348311 Port 0 stats: - Pkts in: 54674335 - Pkts out: 54674167 - In Errs: 44718519 - Out Errs: 0 - Mbuf Errs: 0 Port 1 stats: - Pkts in: 54674367 - Pkts out: 54674144 - In Errs: 46955093 - Out Errs: 0 - Mbuf Errs: 0 Has anyone seen this behavior before? I am building DPDK from the v2.0.0 tag in the git repo.