From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pg0-f45.google.com (mail-pg0-f45.google.com [74.125.83.45]) by dpdk.org (Postfix) with ESMTP id 125B53DC for ; Tue, 13 Dec 2016 23:22:49 +0100 (CET) Received: by mail-pg0-f45.google.com with SMTP id 3so389972pgd.0 for ; Tue, 13 Dec 2016 14:22:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/xQmELw+OLEwYPuSljE6EKCp8udAdMMOxWwD00SRsJ0=; b=S3pGFtTHETmEr3+WdlZ4GgQtnkYBMPX0W/WySAIbNULLOf9WwuhsiNoHIB5MD8kScd lKwqfYrz1RxnQRSCz/VzDCcsnqrYtR83zQAoiIFUP/kakT3L3HDbOHgOwFPLXHJxLIbr O5xKtCXT1e1dYYmJofJ4pzlTujN0cRzYIklXeIBA2z119i1DSYAtJbAZLOBcpEzyfJVh PMrJSIc57ehjniwbovv5hst8Gq0Z/Vzb1DTP26UKCssDt/491x+tkMcaQXfkh0CnKA3t FXDu8dy0edi8fY1ZaIcVnePliah4jZ2DZyXXjDBztlX+mF1bnOpb11QOBHJh0PC1cRTb ek3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/xQmELw+OLEwYPuSljE6EKCp8udAdMMOxWwD00SRsJ0=; b=U7WySo85dN0u1pkSHD8zv/NjnWbkO9cXBSg2Ia2RYMyGJBVd9fHJeRO/vObHS8LRQM Ghx4xbjy+mxoAkKTJSEBIKYwsvKD5YTXv/WgilKhyUttUHyT4ogq6qqtMMuP7yfBF+jB xJozQC+cvdZIs8MMiuDSxyoaDIubx8lWQw5DpbugVU1FFQe+h/MjpeAGrNQkVPA1PZZg IVkPsl3Fepg1/tY18KLkzMdyZWAFaXPzaZ6lI1ynfj8BHUePBfL3dlEIuoCtpOZ5nhqo jfCiD/FsGVucZepkT3TCHbhdKp1tCbHR2BZjiW5xmOo4jDciPmeXRAMQABiP93SeRtYu qIzg== X-Gm-Message-State: AKaTC01k2Xncy1A5LQZnDoqhcBGntboYnStnYuFvlSVypNQ6qpCKwc+jGp+5SrFf0fNh9Q== X-Received: by 10.84.136.1 with SMTP id 1mr202133689plk.152.1481667769057; Tue, 13 Dec 2016 14:22:49 -0800 (PST) Received: from xeon-e3 (204-195-18-65.wavecable.com. [204.195.18.65]) by smtp.gmail.com with ESMTPSA id f132sm82117501pfa.72.2016.12.13.14.22.48 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 13 Dec 2016 14:22:48 -0800 (PST) Date: Tue, 13 Dec 2016 14:22:41 -0800 From: Stephen Hemminger To: Prem Chaitanya Cc: users@dpdk.org Message-ID: <20161213142241.0eef9572@xeon-e3> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-users] Facing issues with Interrupt mode (IGB,VFIO) X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 22:22:50 -0000 On Tue, 13 Dec 2016 14:43:41 +0530 Prem Chaitanya wrote: > Hi All, > > We modified the l3fwd application to run in interrupt+poll mode. The driver > we used was IGB with Intel I350 NIC. > When we ran with 1 lcore and 1 queue, there were no issues. When we tried > to run with 2 lcores and 2 Rx-queues we are not getting interrupts. > In one of the documentation(DPDK-201.pdf), it was mentioned that IGB driver > doesn't support interrupt mode with multiple lcores since it supports only > 1 epoll-fd. > Can you please confirm if this is the case? igb_uio device driver can not support an interrupt per queue (MSI). > Then we went ahead and loaded the VFIO driver and tried to run the same > application. > Now we get an interrupt once and after that when we go back to waiting on > epoll, we dont receive any further interrupts. > But when we print the stats for rx-packets we see that there are packets > waiting in the NIC Rx queue. > Same behavior is observed even when we run application with 1 lcore and 1 > Rx queue. You need to do some careful coding to handle interrupt+poll mode. Look at l3fwd-power example, and even that isn't enough. Basically, you need to dynamically go in and out of interrupt mode and use the rx_queue_count function to check for races. The application must have poll loop like: while() { rte_epoll_wait() if (rx_event) { readmore: rte_eth_dev_rx_intr_disable() while ( (n = rte_eth_rx_queue_burst()) > 0) { process n packets; } rte_eth_dev_rx_intr_enable(); if (rte_eth_dev_rx_queue_count() > 0) { // lost race goto readmore; } } }