From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 1E327A054F for ; Tue, 18 Feb 2020 09:59:38 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 04BA81C295; Tue, 18 Feb 2020 09:59:37 +0100 (CET) Received: from mail-qt1-f175.google.com (mail-qt1-f175.google.com [209.85.160.175]) by dpdk.org (Postfix) with ESMTP id 0CBC81DBD for ; Mon, 17 Feb 2020 06:57:29 +0100 (CET) Received: by mail-qt1-f175.google.com with SMTP id r5so11271731qtt.9 for ; Sun, 16 Feb 2020 21:57:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+5/VCQ2lZh+AIQenvGtg2lEbEz47EGINVMpH7DhumQ8=; b=Z4EUK4XcfRSQwMh7nfCyjY8e6dIa/OBo9goHAJuWcVch02Py8VztvARS66Gk51SgqM acbWSssn2kz36Dx7WDZnGMo17VWcbtzCq0XlNZtFkP/wDRmHZ6ObV/gpuNh/PIKqYiY0 keVawN1J3d8Kt0LN1LNIpA7cVTzSUIrXtIhZPqH7JMwNZLKgaZbfYp3fGGFaYJzH6ktw 74VVAsOrWJ8QdcMAVGXIgvs/7vKmu5gP4pGK55hAv7NvvPO7lR+KVeGVKSDrQNN0dP2E Dh/JZSAvLeLr5pvzfmM1czDET6pzpfOHWmV57CIDPlxazpLWJXuKlN3h5kwgYIPfMVrl A0wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+5/VCQ2lZh+AIQenvGtg2lEbEz47EGINVMpH7DhumQ8=; b=LZXrNW1fB+bEuv/2zliHJ/H6FHeP4LgCGVkWG0NZRmecSS50Yn+axT8oMpoloQo0Zn z9PD0KcXjP4EUHNeZLoMKjcuTYOrVsyTdIOniapeZGsqga5d2Oq0egu0I25SlAbYtIxQ O+ZOhxeD15tzln9MGatySmelxcKsw7jFL+vg+x8tpr94GGlBEtnzA9BaUPajMxMDyH6t eRvgsorQvx4nV4soY8BMAfEqHHRo8aWUand8uB34WMmS/z2WqFAQVVtzdevRUU/UUX+/ 4DWVNKLEMHTJQi4P9GANAWF7Tbx+JElCcivEKr+6k9kLau89DU8zGE9LsH+7or2yRWGE VUow== X-Gm-Message-State: APjAAAW0elbwC6L+yHV4FmDsY3spVCAZIVI46vSz2khlcIlE7+RmYMLi w/OeDe6lzQqRgKarp5afvURg/bQW9QxBX4DTVGQ= X-Google-Smtp-Source: APXvYqz+EfQPVfeHA2zz6LqtuFP88PKMFi74Qcp2nFpI2Zu1ceOs/EKEq1UrZ7gGwlu5M0G5XUr2ODnS53cBcLQu6bY= X-Received: by 2002:ac8:4a02:: with SMTP id x2mr11989326qtq.388.1581919048239; Sun, 16 Feb 2020 21:57:28 -0800 (PST) MIME-Version: 1.0 References: <20200216144525.1886c176@hermes.lan> In-Reply-To: <20200216144525.1886c176@hermes.lan> From: Soumen Chandra Laha Date: Mon, 17 Feb 2020 11:27:16 +0530 Message-ID: To: Stephen Hemminger Cc: users@dpdk.org X-Mailman-Approved-At: Tue, 18 Feb 2020 09:59:35 +0100 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] L3fwd-power queue utilization issue. X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Sender: "users" Hi Stephen, Thanks for your email. Tried binding the NIC's to vfio-pci. Observed still queue 0 of Port 0, Port 1 has been utilized. No interrupts for queue 1. ./dpdk-devbind.py -s Network devices using DPDK-compatible driver ============================================ 0000:17:00.0 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=vfio-pci unused=i40e,igb_uio 0000:17:00.1 'Ethernet Controller XXV710 for 25GbE SFP28 158b' drv=vfio-pci unused=i40e,igb_uio ------------------------------------------------------------------------------ ./l3fwd-power -l 2,3,4,5 -n 6 --socket-mem 512,0 --file-prefix l3fwd -w 17:00.0 -w 17:00.1 -- -p 0x3 --config '(0,0,2),(0,1,3),(1,0,4),(1,1,5)' -P EAL: Detected 48 lcore(s) EAL: Detected 1 NUMA nodes EAL: Multi-process socket /var/run/dpdk/l3fwd/mp_socket EAL: Probing VFIO support... EAL: VFIO support initialized EAL: PCI device 0000:17:00.0 on NUMA socket 0 EAL: probe driver: 8086:158b net_i40e EAL: using IOMMU type 1 (Type 1) EAL: PCI device 0000:17:00.1 on NUMA socket 0 EAL: probe driver: 8086:158b net_i40e Promiscuous mode selected POWER: Attempting to initialise ACPI cpufreq power management... POWER: Power management governor of lcore 2 has been set to user space successfully POWER: Initialized successfully for lcore 2 power management POWER: Power management governor of lcore 3 has been set to user space successfully POWER: Initialized successfully for lcore 3 power management POWER: Power management governor of lcore 4 has been set to user space successfully POWER: Initialized successfully for lcore 4 power management POWER: Power management governor of lcore 5 has been set to user space successfully POWER: Initialized successfully for lcore 5 power management Initializing port 0 ... Creating queues: nb_rxq=2 nb_txq=4... Port 0 modified RSS hash function based on hardware support,requested:0x20820 configured:0x820 Address:00:E0:ED:96:2B:AA, Allocated mbuf pool on socket 0 LPM: Adding route 0x01010100 / 24 (0) LPM: Adding route 0x02010100 / 24 (1) LPM: Adding route 0x03010100 / 24 (2) LPM: Adding route 0x04010100 / 24 (3) LPM: Adding route 0x05010100 / 24 (4) LPM: Adding route 0x06010100 / 24 (5) LPM: Adding route 0x07010100 / 24 (6) LPM: Adding route 0x08010100 / 24 (7) txq=2,0,0 txq=3,1,0 txq=4,2,0 txq=5,3,0 Initializing port 1 ... Creating queues: nb_rxq=2 nb_txq=4... Port 1 modified RSS hash function based on hardware support,requested:0x20820 configured:0x820 Address:00:E0:ED:96:2B:AB, txq=2,0,0 txq=3,1,0 txq=4,2,0 txq=5,3,0 Initializing rx queues on lcore 2 ... rxq=0,0,0 Initializing rx queues on lcore 3 ... rxq=0,1,0 Initializing rx queues on lcore 4 ... rxq=1,0,0 Initializing rx queues on lcore 5 ... rxq=1,1,0 Checking link status.....done Port 0 Link Up - speed 25000 Mbps - full-duplex Port 1 Link Up - speed 25000 Mbps - full-duplex L3FWD_POWER: entering main loop on lcore 3 L3FWD_POWER: -- lcoreid=3 portid=0 rxqueueid=1 L3FWD_POWER: entering main loop on lcore 4 L3FWD_POWER: entering main loop on lcore 5 L3FWD_POWER: entering main loop on lcore 2 L3FWD_POWER: -- lcoreid=5 portid=1 rxqueueid=1 L3FWD_POWER: -- lcoreid=4 portid=1 rxqueueid=0 L3FWD_POWER: -- lcoreid=2 portid=0 rxqueueid=0 L3FWD_POWER: lcore 5 sleeps until interrupt triggers L3FWD_POWER: lcore 4 sleeps until interrupt triggers L3FWD_POWER: lcore 2 sleeps until interrupt triggers L3FWD_POWER: lcore 3 sleeps until interrupt triggers L3FWD_POWER: lcore 4 is waked up from rx interrupt on port 1 queue 0 L3FWD_POWER: lcore 2 is waked up from rx interrupt on port 0 queue 0 L3FWD_POWER: lcore 2 sleeps until interrupt triggers L3FWD_POWER: lcore 2 is waked up from rx interrupt on port 0 queue 0 L3FWD_POWER: lcore 2 sleeps until interrupt triggers L3FWD_POWER: lcore 4 sleeps until interrupt triggers L3FWD_POWER: lcore 4 is waked up from rx interrupt on port 1 queue 0 L3FWD_POWER: lcore 4 sleeps until interrupt triggers Does anybody observed similar behavior, any help & suggestions will be appreciated. Thanks, Soumen Chandra Laha On Mon, Feb 17, 2020 at 4:15 AM Stephen Hemminger < stephen@networkplumber.org> wrote: > On Fri, 14 Feb 2020 16:36:04 +0530 > Soumen Chandra Laha wrote: > > > Hi, > > > > I'm trying to run l3fwd-power application for 2 Port with queue. Always > > only queue 1 of port 0 and port 1 are utilized. > > CPU C-state, hpet are enabled, ports are binded to igb_uio. > > Tried with NIC's x772 (10 GbE) and XXV710(25 GbE), Observed same > behavior. > > > > Igb_uio does not support per-queue interrupts. That needs MSI-X which is > only supported by VFIO-pci. > >