From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <shemming@Brocade.com>
Received: from mx0b-000f0801.pphosted.com (mx0b-000f0801.pphosted.com
 [67.231.152.113]) by dpdk.org (Postfix) with ESMTP id 5BC335A54
 for <dev@dpdk.org>; Wed, 10 Jun 2015 01:59:47 +0200 (CEST)
Received: from pps.filterd (m0000700.ppops.net [127.0.0.1])
 by mx0b-000f0801.pphosted.com (8.14.7/8.14.7) with SMTP id t59NWvPQ027915;
 Tue, 9 Jun 2015 16:59:46 -0700
Received: from brmwp-exchub02.corp.brocade.com ([208.47.132.227])
 by mx0b-000f0801.pphosted.com with ESMTP id 1ux6xe0bcr-1
 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT);
 Tue, 09 Jun 2015 16:59:46 -0700
Received: from BRMWP-EXMB11.corp.brocade.com (172.16.59.77) by
 BRMWP-EXCHUB02.corp.brocade.com (172.16.187.99) with Microsoft SMTP Server
 (TLS) id 14.3.123.3; Tue, 9 Jun 2015 17:59:45 -0600
Received: from urahara (10.252.8.11) by BRMWP-EXMB11.corp.brocade.com
 (172.16.59.77) with Microsoft SMTP Server (TLS) id 15.0.1044.25; Tue, 9 Jun
 2015 17:59:44 -0600
Date: Tue, 9 Jun 2015 16:59:48 -0700
From: Stephen Hemminger <shemming@brocade.com>
To: Cunming Liang <cunming.liang@intel.com>
Message-ID: <20150609165948.7890ed9b@urahara>
In-Reply-To: <1433741351-27005-1-git-send-email-cunming.liang@intel.com>
References: <1433492410-30832-1-git-send-email-cunming.liang@intel.com>
 <1433741351-27005-1-git-send-email-cunming.liang@intel.com>
Organization: Brocade
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
X-ClientProxiedBy: hq1wp-excas12.corp.brocade.com (10.70.38.22) To
 BRMWP-EXMB11.corp.brocade.com (172.16.59.77)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.14.151, 1.0.33,
 0.0.0000
 definitions=2015-06-10_01:2015-06-09,2015-06-09,1970-01-01 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0
 suspectscore=1 phishscore=0
 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=7.0.1-1402240000 definitions=main-1506090395
Cc: dev@dpdk.org, liang-min.wang@intel.com
Subject: Re: [dpdk-dev] [PATCH v12 00/14] Interrupt mode PMD
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: patches and discussions about DPDK <dev.dpdk.org>
List-Unsubscribe: <http://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <http://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
X-List-Received-Date: Tue, 09 Jun 2015 23:59:47 -0000

On Mon,  8 Jun 2015 13:28:57 +0800
Cunming Liang <cunming.liang@intel.com> wrote:

> v12 changes
>  - bsd cleanup for unused variable warning
>  - fix awkward line split in debug message
> 
> v11 changes
>  - typo cleanup and check kernel style
> 
> v10 changes
>  - code rework to return actual error code
>  - bug fix for lsc when using uio_pci_generic
> 
> v9 changes
>  - code rework to fix open comment
>  - bug fix for igb lsc when both lsc and rxq are enabled in vfio-msix
>  - new patch to turn off the feature by default so as to avoid v2.1 abi broken
> 
> v8 changes
>  - remove condition check for only vfio-msix
>  - add multiplex intr support when only one intr vector allowed
>  - lsc and rxq interrupt runtime enable decision
>  - add safe event delete while the event wakeup execution happens
> 
> v7 changes
>  - decouple epoll event and intr operation
>  - add condition check in the case intr vector is disabled
>  - renaming some APIs
> 
> v6 changes
>  - split rte_intr_wait_rx_pkt into two APIs 'wait' and 'set'.
>  - rewrite rte_intr_rx_wait/rte_intr_rx_set.
>  - using vector number instead of queue_id as interrupt API params.
>  - patch reorder and split.
> 
> v5 changes
>  - Rebase the patchset onto the HEAD
>  - Isolate ethdev from EAL for new-added wait-for-rx interrupt function
>  - Export wait-for-rx interrupt function for shared libraries
>  - Split-off a new patch file for changed struct rte_intr_handle that
>    other patches depend on, to avoid breaking git bisect
>  - Change sample applicaiton to accomodate EAL function spec change
>    accordingly
> 
> v4 changes
>  - Export interrupt enable/disable functions for shared libraries
>  - Adjust position of new-added structure fields and functions to
>    avoid breaking ABI
>  
> v3 changes
>  - Add return value for interrupt enable/disable functions
>  - Move spinlok from PMD to L3fwd-power
>  - Remove unnecessary variables in e1000_mac_info
>  - Fix miscelleous review comments
>  
> v2 changes
>  - Fix compilation issue in Makefile for missed header file.
>  - Consolidate internal and community review comments of v1 patch set.
>  
> The patch series introduce low-latency one-shot rx interrupt into DPDK with
> polling and interrupt mode switch control example.
>  
> DPDK userspace interrupt notification and handling mechanism is based on UIO
> with below limitation:
> 1) It is designed to handle LSC interrupt only with inefficient suspended
>    pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling thread
>    which then wakes up DPDK polling thread). In this way, it introduces
>    non-deterministic wakeup latency for DPDK polling thread as well as packet
>    latency if it is used to handle Rx interrupt.
> 2) UIO only supports a single interrupt vector which has to been shared by
>    LSC interrupt and interrupts assigned to dedicated rx queues.
>  
> This patchset includes below features:
> 1) Enable one-shot rx queue interrupt in ixgbe PMD(PF & VF) and igb PMD(PF only).
> 2) Build on top of the VFIO mechanism instead of UIO, so it could support
>    up to 64 interrupt vectors for rx queue interrupts.
> 3) Have 1 DPDK polling thread handle per Rx queue interrupt with a dedicated
>    VFIO eventfd, which eliminates non-deterministic pthread wakeup latency in
>    user space.
> 4) Demonstrate interrupts control APIs and userspace NAIP-like polling/interrupt
>    switch algorithms in L3fwd-power example.
> 
> Known limitations:
> 1) It does not work for UIO due to a single interrupt eventfd shared by LSC
>    and rx queue interrupt handlers causes a mess. [FIXED]
> 2) LSC interrupt is not supported by VF driver, so it is by default disabled
>    in L3fwd-power now. Feel free to turn in on if you want to support both LSC
>    and rx queue interrupts on a PF.
> 
> Cunming Liang (14):
>   eal/linux: add interrupt vectors support in intr_handle
>   eal/linux: add rte_epoll_wait/ctl support
>   eal/linux: add API to set rx interrupt event monitor
>   eal/linux: fix comments typo on vfio msi
>   eal/linux: add interrupt vectors handling on VFIO
>   eal/linux: standalone intr event fd create support
>   eal/linux: fix lsc read error in uio_pci_generic
>   eal/bsd: dummy for new intr definition
>   eal/bsd: fix inappropriate linuxapp referred in bsd
>   ethdev: add rx intr enable, disable and ctl functions
>   ixgbe: enable rx queue interrupts for both PF and VF
>   igb: enable rx queue interrupts for PF
>   l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode
>     switch
>   abi: fix v2.1 abi broken issue
> 
>  drivers/net/e1000/igb_ethdev.c                     | 311 ++++++++++--
>  drivers/net/ixgbe/ixgbe_ethdev.c                   | 519 ++++++++++++++++++++-
>  drivers/net/ixgbe/ixgbe_ethdev.h                   |   4 +
>  examples/l3fwd-power/main.c                        | 206 ++++++--
>  lib/librte_eal/bsdapp/eal/eal_interrupts.c         |  30 ++
>  .../bsdapp/eal/include/exec-env/rte_interrupts.h   |  91 +++-
>  lib/librte_eal/bsdapp/eal/rte_eal_version.map      |   5 +
>  lib/librte_eal/linuxapp/eal/eal_interrupts.c       | 361 ++++++++++++--
>  .../linuxapp/eal/include/exec-env/rte_interrupts.h | 219 +++++++++
>  lib/librte_eal/linuxapp/eal/rte_eal_version.map    |   8 +
>  lib/librte_ether/rte_ethdev.c                      | 109 +++++
>  lib/librte_ether/rte_ethdev.h                      | 132 ++++++
>  lib/librte_ether/rte_ether_version.map             |   4 +
>  13 files changed, 1871 insertions(+), 128 deletions(-)
> 

Acked-by: Stephen Hemminger <stephen@networkplumber.org>

This still needs more work in lots more drivers (like bonding) and in several
other subsystems (like pipeline) before it is widely useful. But this is
a great first step.