From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 987BEA0471 for ; Wed, 17 Jul 2019 07:56:04 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 1D65A2BF4; Wed, 17 Jul 2019 07:56:03 +0200 (CEST) Received: from rcdn-iport-6.cisco.com (rcdn-iport-6.cisco.com [173.37.86.77]) by dpdk.org (Postfix) with ESMTP id 9B3C92BC7 for ; Wed, 17 Jul 2019 07:56:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=7395; q=dns/txt; s=iport; t=1563342961; x=1564552561; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=wih1tEidtX15GSpddfWSoEHHDMZ6f6A3FPrknD0Yhbg=; b=BEb3MRluqo0k2ZL6nobzgr39+foCp1QKJ8kj0moV2axSNv/fRFDoVFrK zKN4hzETdr6NX1PPL0QNzO1vJamQeWY1AAduMAfsxMnlWoUgqm17RhxWn QvAxrv9oHKtyFraDZOkJr+E/9XcbVMUJg76tM4pAIA5dlggUQsN1TtRMQ g=; IronPort-PHdr: =?us-ascii?q?9a23=3A3oC3kBQ2II7qELY6Q/JJaq4ucNpsv++ubAcI9p?= =?us-ascii?q?oqja5Pea2//pPkeVbS/uhpkESXBdfA8/wRje3QvuigQmEG7Zub+FE6OJ1XH1?= =?us-ascii?q?5g640NmhA4RsuMCEn1NvnvOiwrG8JBVVpN9HCgOk8TE8H7NBXf?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0CpAACQty5d/5tdJa1mGgEBAQEBAgE?= =?us-ascii?q?BAQEHAgEBAQGBZ4FEKScDgUIgBAsqh2MDjk+CW36WUoFCgRADVAkBAQEMAQE?= =?us-ascii?q?tAgEBhEACgj0jOBMBAwEBBAEBAgEFbYU8DIVKAQEBAQIBEhUTBgEBNwEEBwQ?= =?us-ascii?q?CAQgOAwQBAR8QMhcBBQgCBAENBQgagjWCNgMODwGhYwKBOIhggXAzgnkBAQW?= =?us-ascii?q?BBgGECRiCEwmBNIdug3EXgUA/gRABRoIXNT6ELBokgxeCJowCIDaeDwkCghm?= =?us-ascii?q?UJ5gKjTWXUAIEAgQFAg4BAQWBZyGBWHAVgyeCQQwXg06KHAE2coEpjQ0BAQ?= X-IronPort-AV: E=Sophos;i="5.64,273,1559520000"; d="scan'208";a="596501131" Received: from rcdn-core-4.cisco.com ([173.37.93.155]) by rcdn-iport-6.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 17 Jul 2019 05:55:59 +0000 Received: from XCH-RCD-009.cisco.com (xch-rcd-009.cisco.com [173.37.102.19]) by rcdn-core-4.cisco.com (8.15.2/8.15.2) with ESMTPS id x6H5txBR029209 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=FAIL); Wed, 17 Jul 2019 05:55:59 GMT Received: from xhs-aln-003.cisco.com (173.37.135.120) by XCH-RCD-009.cisco.com (173.37.102.19) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 17 Jul 2019 00:55:59 -0500 Received: from xhs-rcd-003.cisco.com (173.37.227.248) by xhs-aln-003.cisco.com (173.37.135.120) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 17 Jul 2019 00:55:59 -0500 Received: from NAM01-BY2-obe.outbound.protection.outlook.com (72.163.14.9) by xhs-rcd-003.cisco.com (173.37.227.248) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Wed, 17 Jul 2019 00:55:58 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XxP2rCfAUH5D9FlQMgABmCXbYtSB1HJtEtped3qPqAdQKsmTUUqzyFUI8Sh5iALtbA15rawP6c4DxpzBGj+A+YEyONe+RUWSActUaDMGPJh3ySE6PYj8WAYPHIjmpxKJ1EhQJUGufVgsPUnbsnAkSmC7oxapwO3JUhl5BopGjSdMfV45pco7Iu8SmhUf58aV/W1XDMmabNfVtEQTnTESsRri8yqQMdGjM/OzQdO9uhHynrsZ3vGmPBfXqpstllF5zeBt+bEIhK4biB0vNcgWYhJS8TZAtF/S7WmYGaZSj962+sjSZFS1eVDMRj+p5sQCqztmoWKryXja9xuKQrgH2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QdizE3qCS526fLs6K4t2WW2oWgSJqrK3k4bfihMYf9c=; b=grkvv4sYXovr6Af6dzIway+Z6oPnRP2AknuDs6cj06VhwwIOF44mKXMMpmpqoc4JIn+S3bI/lVHih5ai4EGeyO3ZN5YSPpyINXV2Ft/omJ16kgI9ZTaZ/8L8cRtCb5iiYHF55FQfl3gPojUuNOITKeSGOzZKBUq6qnPq6uqhAn5+iT9M/663Bvrkltmeqia6HgIn5hc/0aABxaOzx3hH0ZuS9uE/U1brBJulpnimLpvOZcyTSFzHfohb+anegRXCW055bZo+XW/JzTvIcxNNhST1f8w/RcR/ZeWUOxfvMdhekO+qblP97cdmoB+/mw6Cjqol07oCoE+LI1iKkHYmnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1;spf=pass smtp.mailfrom=cisco.com;dmarc=pass action=none header.from=cisco.com;dkim=pass header.d=cisco.com;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cisco.onmicrosoft.com; s=selector2-cisco-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QdizE3qCS526fLs6K4t2WW2oWgSJqrK3k4bfihMYf9c=; b=ahd/0gncXAY8OAUu5dPDYROK1lj6Q43rGEVYJT9t2oQwVRTIkmw/fkna+v2RUqoAgJrhTD74RqadeRqtsHLpDYstEWU05U8rkYmYAJMWECgkzqowTecg0vfKx4uv1tokRVrUoYAHMe2piwwTjn99+0Y5xd84rFPofK40SV4lYm0= Received: from MWHPR11MB1839.namprd11.prod.outlook.com (10.175.53.12) by MWHPR11MB1837.namprd11.prod.outlook.com (10.175.53.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2073.11; Wed, 17 Jul 2019 05:55:57 +0000 Received: from MWHPR11MB1839.namprd11.prod.outlook.com ([fe80::3cef:35b5:8800:39f1]) by MWHPR11MB1839.namprd11.prod.outlook.com ([fe80::3cef:35b5:8800:39f1%6]) with mapi id 15.20.2094.011; Wed, 17 Jul 2019 05:55:57 +0000 From: "Hyong Youb Kim (hyonkim)" To: Nithin Dabilpuram , David Marchand , Thomas Monjalon , "Ferruh Yigit" , Bruce Richardson CC: "jerinj@marvell.com" , "John Daley (johndale)" , Shahed Shaikh , "dev@dpdk.org" Thread-Topic: [RFC PATCH v3 2/3] eal: add mask and unmask interrupt APIs Thread-Index: AQHVO/XevRYbQEFR2kiByXrzVb6deqbOSisw Date: Wed, 17 Jul 2019 05:55:56 +0000 Message-ID: References: <20190716164424.16776-1-ndabilpuram@marvell.com> <20190716164424.16776-2-ndabilpuram@marvell.com> In-Reply-To: <20190716164424.16776-2-ndabilpuram@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=hyonkim@cisco.com; x-originating-ip: [2001:420:c0dc:1001::133] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 7be3c261-fde1-4987-12d8-08d70a7b6fdf x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600148)(711020)(4605104)(1401327)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:MWHPR11MB1837; x-ms-traffictypediagnostic: MWHPR11MB1837: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 01018CB5B3 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(4636009)(396003)(346002)(39860400002)(376002)(366004)(136003)(13464003)(189003)(199004)(316002)(6116002)(110136005)(54906003)(14454004)(76176011)(6506007)(53546011)(7696005)(46003)(229853002)(102836004)(186003)(99286004)(25786009)(4326008)(9686003)(33656002)(55016002)(53936002)(478600001)(6246003)(68736007)(8676002)(71200400001)(71190400001)(81166006)(81156014)(486006)(7736002)(74316002)(476003)(6436002)(446003)(2906002)(64756008)(86362001)(14444005)(256004)(11346002)(66946007)(5660300002)(66556008)(52536014)(66446008)(305945005)(76116006)(8936002)(66476007); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR11MB1837; H:MWHPR11MB1839.namprd11.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: cisco.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: haA+G3R5STIArhF9SBJFJapV8x0kbQHaY1hRzM/h6hvg2d99K/b0bWZuPfV/Dl5Hvdxt9EBR1SqXAmRjSuv7C9l0LpIuz+8yJrqkX0L6/NzQGthpMn2crmVWzVRdbxvRscg2Bw5odRiJAeuVkskY37VkROkaOF0eMyp888UAiCpqpwlTQQ3ykxlKE6YkmECIyk75p7D7VExEyo/0sA3/XYkaFTAAfCvEnc+yxMTEOQSm9slE923nG/MDKR0SszwDVOPq+H5fzaUklqE4ltwgwHdCbebJWsrBBBBSHB0KRKwepQwd1IQX0yEc9c050jW0Z4dkTE1Q+1xUly6dVbLy3KSy5+wzSIiHrUixuGu+IDitbNgcfKGN13f6IL6/FnUEqELEBMT3eEOIIE8Y8f2Yjf++FDVtcXTsWvN1nLWRNc8= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: 7be3c261-fde1-4987-12d8-08d70a7b6fdf X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jul 2019 05:55:56.7427 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 5ae1af62-9505-4097-a69a-c1553ef7840e X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: hyonkim@cisco.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR11MB1837 X-OriginatorOrg: cisco.com X-Outbound-SMTP-Client: 173.37.102.19, xch-rcd-009.cisco.com X-Outbound-Node: rcdn-core-4.cisco.com Subject: Re: [dpdk-dev] [RFC PATCH v3 2/3] eal: add mask and unmask interrupt APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Nithin Dabilpuram > Sent: Wednesday, July 17, 2019 1:44 AM > To: Hyong Youb Kim (hyonkim) ; David Marchand > ; Thomas Monjalon > ; Ferruh Yigit ; Bruce > Richardson > Cc: jerinj@marvell.com; John Daley (johndale) ; > Shahed Shaikh ; dev@dpdk.org; Nithin Dabilpuram > > Subject: [RFC PATCH v3 2/3] eal: add mask and unmask interrupt APIs >=20 > Add new mask and unmask interrupt APIs to avoid using > VFIO_IRQ_SET_ACTION_TRIGGER for masking/unmasking purpose for VFIO > based handlers. This implementation is specific to Linux. >=20 > Using action trigger for masking and unmasking has below issues >=20 > * Time consuming to do for every interrupt received as it will > free_irq() followed by request_irq() and all other initializations > * A race condition because of a window between free_irq() and > request_irq() with packet reception still on and device still > enabled and would throw warning messages like below. > [158764.159833] do_IRQ: 9.34 No irq handler for vector >=20 > In this patch, mask/unmask is a no-op for VFIO_MSIX/VFIO_MSI interrupts > as they are edge triggered and kernel would not mask the interrupt before > delivering the event to userspace. >=20 > Signed-off-by: Nithin Dabilpuram > --- > v3: > * Re-org patch to move driver change to 3/3 > * Add stub implementation for freebsd. > * Fix version map file for new apis. > v2: > Make change in qede driver for unmask > lib/librte_eal/common/include/rte_interrupts.h | 23 ++++ > lib/librte_eal/freebsd/eal/eal_interrupts.c | 54 +++++++++ > lib/librte_eal/linux/eal/eal_interrupts.c | 155 > +++++++++++++++++++++++++ > lib/librte_eal/rte_eal_version.map | 2 + > 4 files changed, 234 insertions(+) >=20 > diff --git a/lib/librte_eal/common/include/rte_interrupts.h > b/lib/librte_eal/common/include/rte_interrupts.h > index c1e912c..b0675be 100644 > --- a/lib/librte_eal/common/include/rte_interrupts.h > +++ b/lib/librte_eal/common/include/rte_interrupts.h > @@ -118,6 +118,29 @@ int rte_intr_enable(const struct rte_intr_handle > *intr_handle); > */ > int rte_intr_disable(const struct rte_intr_handle *intr_handle); >=20 > +/** > + * It masks the interrupt for the specified handle. > + * > + * @param intr_handle > + * pointer to the interrupt handle. > + * > + * @return > + * - On success, zero. > + * - On failure, a negative value. > + */ > +int rte_intr_mask(const struct rte_intr_handle *intr_handle); > + > +/** > + * It unmasks the interrupt for the specified handle. > + * > + * @param intr_handle > + * pointer to the interrupt handle. > + * > + * @return > + * - On success, zero. > + * - On failure, a negative value. > + */ > +int rte_intr_unmask(const struct rte_intr_handle *intr_handle); > #ifdef __cplusplus > } > #endif [...] > diff --git a/lib/librte_eal/linux/eal/eal_interrupts.c > b/lib/librte_eal/linux/eal/eal_interrupts.c > index 79ad5e8..f619981 100644 > --- a/lib/librte_eal/linux/eal/eal_interrupts.c > +++ b/lib/librte_eal/linux/eal/eal_interrupts.c [...] > int > +rte_intr_mask(const struct rte_intr_handle *intr_handle) > +{ > + if (intr_handle && intr_handle->type =3D=3D RTE_INTR_HANDLE_VDEV) > + return 0; > + > + if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0) > + return -1; > + > + switch (intr_handle->type){ > + /* Both masking and disabling are same for UIO */ > + case RTE_INTR_HANDLE_UIO: > + if (uio_intr_disable(intr_handle)) > + return -1; > + break; > + case RTE_INTR_HANDLE_UIO_INTX: > + if (uio_intx_intr_disable(intr_handle)) > + return -1; > + break; > + /* not used at this moment */ > + case RTE_INTR_HANDLE_ALARM: > + return -1; > +#ifdef VFIO_PRESENT > + case RTE_INTR_HANDLE_VFIO_MSIX: > + case RTE_INTR_HANDLE_VFIO_MSI: > + return 0; Isn't this a little confusing? It returns success, but irq is not masked. As is, return code 0 means... - igb_uio: irq is masked for INTx, MSI, MSI-X - vfio-pci + INTx: irq is masked - vfio-pci + MSI/MSI-X: no changes Masking is useful only for INTx, IMO... Masking MSI/MSI-X via PCI-defined mechanisms (e.g. Mask bit in MSI-X Table) has no practical use for drivers. Handshaking/masking/unmasking is done via device/vendor specific ways, as needed. See all those ack/block/unblock/credit/... mechanisms used in various drivers/NICs to control interrupts their own way. A long time ago in early PCIe days, the linux kernel did auto-masking for MSI/MSI-X (i.e. mask before calling netdev irq handler). It was soon removed as it was unnecessary overhead (expensive PIOs to NIC for every interrupt). Windows and FreeBSD do not do auto-masking either. Thanks. -Hyong > + case RTE_INTR_HANDLE_VFIO_LEGACY: > + if (vfio_mask_intx(intr_handle)) > + return -1; > + break; > +#ifdef HAVE_VFIO_DEV_REQ_INTERFACE > + case RTE_INTR_HANDLE_VFIO_REQ: > + return -1; > +#endif > +#endif > + /* not used at this moment */ > + case RTE_INTR_HANDLE_DEV_EVENT: > + return -1; > + /* unknown handle type */ > + default: > + RTE_LOG(ERR, EAL, > + "Unknown handle type of fd %d\n", > + intr_handle->fd); > + return -1; > + } > + > + return 0; > +} > + > +int > +rte_intr_unmask(const struct rte_intr_handle *intr_handle) > +{ > + if (intr_handle && intr_handle->type =3D=3D RTE_INTR_HANDLE_VDEV) > + return 0; > + > + if (!intr_handle || intr_handle->fd < 0 || intr_handle->uio_cfg_fd < 0) > + return -1; > + > + switch (intr_handle->type){ > + /* Both unmasking and disabling are same for UIO */ > + case RTE_INTR_HANDLE_UIO: > + if (uio_intr_enable(intr_handle)) > + return -1; > + break; > + case RTE_INTR_HANDLE_UIO_INTX: > + if (uio_intx_intr_enable(intr_handle)) > + return -1; > + break; > + /* not used at this moment */ > + case RTE_INTR_HANDLE_ALARM: > + return -1; > +#ifdef VFIO_PRESENT > + case RTE_INTR_HANDLE_VFIO_MSIX: > + case RTE_INTR_HANDLE_VFIO_MSI: > + return 0; > + case RTE_INTR_HANDLE_VFIO_LEGACY: > + if (vfio_unmask_intx(intr_handle)) > + return -1; > + break; > +#ifdef HAVE_VFIO_DEV_REQ_INTERFACE > + case RTE_INTR_HANDLE_VFIO_REQ: > + return -1; > +#endif > +#endif > + /* not used at this moment */ > + case RTE_INTR_HANDLE_DEV_EVENT: > + return -1; > + /* unknown handle type */ > + default: > + RTE_LOG(ERR, EAL, > + "Unknown handle type of fd %d\n", > + intr_handle->fd); > + return -1; > + } > + > + return 0; > +} > + > +int > rte_intr_disable(const struct rte_intr_handle *intr_handle) > { > if (intr_handle && intr_handle->type =3D=3D RTE_INTR_HANDLE_VDEV) > diff --git a/lib/librte_eal/rte_eal_version.map > b/lib/librte_eal/rte_eal_version.map > index 1892d9e..b3b2df4 100644 > --- a/lib/librte_eal/rte_eal_version.map > +++ b/lib/librte_eal/rte_eal_version.map > @@ -309,6 +309,8 @@ DPDK_19.08 { > rte_service_lcore_attr_reset_all; > rte_service_may_be_active; > rte_srand; > + rte_intr_unmask; > + rte_intr_mask; >=20 > } DPDK_19.05; >=20 > -- > 2.8.4