From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx0b-000f0801.pphosted.com (mx0b-000f0801.pphosted.com [67.231.152.113]) by dpdk.org (Postfix) with ESMTP id 5BC335A54 for ; Wed, 10 Jun 2015 01:59:47 +0200 (CEST) Received: from pps.filterd (m0000700.ppops.net [127.0.0.1]) by mx0b-000f0801.pphosted.com (8.14.7/8.14.7) with SMTP id t59NWvPQ027915; Tue, 9 Jun 2015 16:59:46 -0700 Received: from brmwp-exchub02.corp.brocade.com ([208.47.132.227]) by mx0b-000f0801.pphosted.com with ESMTP id 1ux6xe0bcr-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 09 Jun 2015 16:59:46 -0700 Received: from BRMWP-EXMB11.corp.brocade.com (172.16.59.77) by BRMWP-EXCHUB02.corp.brocade.com (172.16.187.99) with Microsoft SMTP Server (TLS) id 14.3.123.3; Tue, 9 Jun 2015 17:59:45 -0600 Received: from urahara (10.252.8.11) by BRMWP-EXMB11.corp.brocade.com (172.16.59.77) with Microsoft SMTP Server (TLS) id 15.0.1044.25; Tue, 9 Jun 2015 17:59:44 -0600 Date: Tue, 9 Jun 2015 16:59:48 -0700 From: Stephen Hemminger To: Cunming Liang Message-ID: <20150609165948.7890ed9b@urahara> In-Reply-To: <1433741351-27005-1-git-send-email-cunming.liang@intel.com> References: <1433492410-30832-1-git-send-email-cunming.liang@intel.com> <1433741351-27005-1-git-send-email-cunming.liang@intel.com> Organization: Brocade MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-ClientProxiedBy: hq1wp-excas12.corp.brocade.com (10.70.38.22) To BRMWP-EXMB11.corp.brocade.com (172.16.59.77) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151, 1.0.33, 0.0.0000 definitions=2015-06-10_01:2015-06-09,2015-06-09,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=1 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1506090395 Cc: dev@dpdk.org, liang-min.wang@intel.com Subject: Re: [dpdk-dev] [PATCH v12 00/14] Interrupt mode PMD X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jun 2015 23:59:47 -0000 On Mon, 8 Jun 2015 13:28:57 +0800 Cunming Liang wrote: > v12 changes > - bsd cleanup for unused variable warning > - fix awkward line split in debug message > > v11 changes > - typo cleanup and check kernel style > > v10 changes > - code rework to return actual error code > - bug fix for lsc when using uio_pci_generic > > v9 changes > - code rework to fix open comment > - bug fix for igb lsc when both lsc and rxq are enabled in vfio-msix > - new patch to turn off the feature by default so as to avoid v2.1 abi broken > > v8 changes > - remove condition check for only vfio-msix > - add multiplex intr support when only one intr vector allowed > - lsc and rxq interrupt runtime enable decision > - add safe event delete while the event wakeup execution happens > > v7 changes > - decouple epoll event and intr operation > - add condition check in the case intr vector is disabled > - renaming some APIs > > v6 changes > - split rte_intr_wait_rx_pkt into two APIs 'wait' and 'set'. > - rewrite rte_intr_rx_wait/rte_intr_rx_set. > - using vector number instead of queue_id as interrupt API params. > - patch reorder and split. > > v5 changes > - Rebase the patchset onto the HEAD > - Isolate ethdev from EAL for new-added wait-for-rx interrupt function > - Export wait-for-rx interrupt function for shared libraries > - Split-off a new patch file for changed struct rte_intr_handle that > other patches depend on, to avoid breaking git bisect > - Change sample applicaiton to accomodate EAL function spec change > accordingly > > v4 changes > - Export interrupt enable/disable functions for shared libraries > - Adjust position of new-added structure fields and functions to > avoid breaking ABI > > v3 changes > - Add return value for interrupt enable/disable functions > - Move spinlok from PMD to L3fwd-power > - Remove unnecessary variables in e1000_mac_info > - Fix miscelleous review comments > > v2 changes > - Fix compilation issue in Makefile for missed header file. > - Consolidate internal and community review comments of v1 patch set. > > The patch series introduce low-latency one-shot rx interrupt into DPDK with > polling and interrupt mode switch control example. > > DPDK userspace interrupt notification and handling mechanism is based on UIO > with below limitation: > 1) It is designed to handle LSC interrupt only with inefficient suspended > pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling thread > which then wakes up DPDK polling thread). In this way, it introduces > non-deterministic wakeup latency for DPDK polling thread as well as packet > latency if it is used to handle Rx interrupt. > 2) UIO only supports a single interrupt vector which has to been shared by > LSC interrupt and interrupts assigned to dedicated rx queues. > > This patchset includes below features: > 1) Enable one-shot rx queue interrupt in ixgbe PMD(PF & VF) and igb PMD(PF only). > 2) Build on top of the VFIO mechanism instead of UIO, so it could support > up to 64 interrupt vectors for rx queue interrupts. > 3) Have 1 DPDK polling thread handle per Rx queue interrupt with a dedicated > VFIO eventfd, which eliminates non-deterministic pthread wakeup latency in > user space. > 4) Demonstrate interrupts control APIs and userspace NAIP-like polling/interrupt > switch algorithms in L3fwd-power example. > > Known limitations: > 1) It does not work for UIO due to a single interrupt eventfd shared by LSC > and rx queue interrupt handlers causes a mess. [FIXED] > 2) LSC interrupt is not supported by VF driver, so it is by default disabled > in L3fwd-power now. Feel free to turn in on if you want to support both LSC > and rx queue interrupts on a PF. > > Cunming Liang (14): > eal/linux: add interrupt vectors support in intr_handle > eal/linux: add rte_epoll_wait/ctl support > eal/linux: add API to set rx interrupt event monitor > eal/linux: fix comments typo on vfio msi > eal/linux: add interrupt vectors handling on VFIO > eal/linux: standalone intr event fd create support > eal/linux: fix lsc read error in uio_pci_generic > eal/bsd: dummy for new intr definition > eal/bsd: fix inappropriate linuxapp referred in bsd > ethdev: add rx intr enable, disable and ctl functions > ixgbe: enable rx queue interrupts for both PF and VF > igb: enable rx queue interrupts for PF > l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode > switch > abi: fix v2.1 abi broken issue > > drivers/net/e1000/igb_ethdev.c | 311 ++++++++++-- > drivers/net/ixgbe/ixgbe_ethdev.c | 519 ++++++++++++++++++++- > drivers/net/ixgbe/ixgbe_ethdev.h | 4 + > examples/l3fwd-power/main.c | 206 ++++++-- > lib/librte_eal/bsdapp/eal/eal_interrupts.c | 30 ++ > .../bsdapp/eal/include/exec-env/rte_interrupts.h | 91 +++- > lib/librte_eal/bsdapp/eal/rte_eal_version.map | 5 + > lib/librte_eal/linuxapp/eal/eal_interrupts.c | 361 ++++++++++++-- > .../linuxapp/eal/include/exec-env/rte_interrupts.h | 219 +++++++++ > lib/librte_eal/linuxapp/eal/rte_eal_version.map | 8 + > lib/librte_ether/rte_ethdev.c | 109 +++++ > lib/librte_ether/rte_ethdev.h | 132 ++++++ > lib/librte_ether/rte_ether_version.map | 4 + > 13 files changed, 1871 insertions(+), 128 deletions(-) > Acked-by: Stephen Hemminger This still needs more work in lots more drivers (like bonding) and in several other subsystems (like pipeline) before it is widely useful. But this is a great first step.