From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id D32EBA04B5; Wed, 28 Oct 2020 21:37:03 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 240074C9B; Wed, 28 Oct 2020 21:37:01 +0100 (CET) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id B121D378B for ; Wed, 28 Oct 2020 21:36:59 +0100 (CET) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 09SKaSGP020207; Wed, 28 Oct 2020 13:36:56 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=date : from : to : cc : subject : message-id : references : content-type : in-reply-to : mime-version; s=pfpt0220; bh=Hxny41yZa6xUAc0gbiMBzGXC7p+pSiJqI09LwAWABUY=; b=ZEixegMwhkVhpYkAYGwoCdbB3A4uPv413l3euC00/yyJfEOwfxxPdIY3j0kRcesIXmH5 tTG7HuyS1YTvaV7swrkN2vYfgv4gTLhbLpuFDGMzOMw3wkRVB4aPGOkT74ClCOCQ3VaY wBBSJS5wcc9W0BlXjBkv6cnnfg3P07JeEXC6AHondEXO9jPv5VYOxeN2DemabaYCmapP Vt5YKZXuhVcNXsXgnFOgKettSYNmv+VrPY2YyjNW4rqVVZR89mepaeY5ll2omsVVjSD4 TZ/QPiXLZQlfLZmhWDLiS/WxmvJ4Xng1YsRGuX7B7QKdAPN3D6edcOBm69kdCXD3XXwq gQ== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 34cksmh723-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Wed, 28 Oct 2020 13:36:55 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 28 Oct 2020 13:36:53 -0700 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (104.47.55.103) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Wed, 28 Oct 2020 13:36:53 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Z+95+RDqR65e6IC7vwhMBYyciD98X7tqvhHLLEYP5hMx847RnLTQEt4vTRZb6GMRNOFbxaguh+LoStXy3RPVhO/K7H8XgmB3qWWq2ul5+KQQrgloF/OJuUvVvnbN9a8KhBqnq+OyHPi2/z7FUqS2AP0EIQe8vVI7Zezz9Pmqwdy9vpt0qBMqKRhGG9xmO9P2E1pi0VFyzOMyDMVe9RcUEtLQWj2XbSdkMQY4D4PVfy3SfY62l8AzH6G1ROsXdJ/luXEx3Mf13aGFGJn4GGoZO2MUceeXKrp/q0vCAz5rd/g9BXpaP6wWjhhBGaY6s3kM5Y6w90tS/6c9ZWlt3hdOpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Hxny41yZa6xUAc0gbiMBzGXC7p+pSiJqI09LwAWABUY=; b=QIRfIO5m24YemdHw7CpgIcnUjnqKn+mk5BlacH0/PWRrri/SoXKfr1ncUUXM6sB2vsyXG7ZtqFkOh2jg6xSeVLx23dDrlBZ4fMldd1j9C49YDZpKPNBT4VUOhZMHPqL1GUR0OdD/439reEODPEaFAGN2YHe1pOySY4QflmPX+TFfJKw/qOB90rNecvqKN1tvv4q4ia5jvp1lzqI4Hq+Z62Srpk6XrfO62FZVINoPAmzbJBtN0Qo64QIr+jtW0555xXVHXL+s3ev2fYysV6Tiw0Om9hBiP4zjFtMhYKM9lL6M9SWj1MpvGmEabfeF74Azt3SE+ljhOBBtV1OzfToQAg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Hxny41yZa6xUAc0gbiMBzGXC7p+pSiJqI09LwAWABUY=; b=AVGMYl6xVxpfKzW2uPJmm77YT3FUJcONf+535F9uGR96CHPi5P3GgAgWBa6szR6jTX4uYoPRV+Ydeeqczo9PeFefs5JeMKBtTMnG6KGptb/HscD0zqMRtdQxiOWVtqAK+wPoQZChadSdUxWhdT3FE1pz4QHhyb0T2ZhUMR7DuUo= Authentication-Results: ekinops.com; dkim=none (message not signed) header.d=none;ekinops.com; dmarc=none action=none header.from=marvell.com; Received: from CO6PR18MB3842.namprd18.prod.outlook.com (2603:10b6:5:341::16) by MW3PR18MB3531.namprd18.prod.outlook.com (2603:10b6:303:2d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.26; Wed, 28 Oct 2020 20:36:52 +0000 Received: from CO6PR18MB3842.namprd18.prod.outlook.com ([fe80::b960:6770:a1:2a19]) by CO6PR18MB3842.namprd18.prod.outlook.com ([fe80::b960:6770:a1:2a19%7]) with mapi id 15.20.3499.027; Wed, 28 Oct 2020 20:36:51 +0000 Date: Thu, 29 Oct 2020 02:06:33 +0530 From: Harman Kalra To: Renata Saiakhova CC: Anatoly Burakov , Bruce Richardson , Ray Kinsella , Neil Horman , Message-ID: <20201028203632.GA137989@outlook.office365.com> References: <20200817140828.9769-1-Renata.Saiakhova@ekinops.com> <20200817140828.9769-2-Renata.Saiakhova@ekinops.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200817140828.9769-2-Renata.Saiakhova@ekinops.com> User-Agent: Mutt/1.10.0 (2018-05-17) X-Originating-IP: [1.6.215.26] X-ClientProxiedBy: BMXPR01CA0023.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::33) To CO6PR18MB3842.namprd18.prod.outlook.com (2603:10b6:5:341::16) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from outlook.office365.com (1.6.215.26) by BMXPR01CA0023.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:d::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3499.18 via Frontend Transport; Wed, 28 Oct 2020 20:36:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: cb0bb27b-732e-4ab0-8d83-08d87b81334d X-MS-TrafficTypeDiagnostic: MW3PR18MB3531: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Qt0wd+5lY+eY1bLPq0mczk0ea6N+l55R/bpiQzb3ITQYf30ocnRDefD/1cgH75MsgBqj2DeVBHHKL3QhdIsWgU/LteUbNlkzfNhLaftPz6Ufbd4OcPOSoygXcmp5/96OMpQkdxHoSMiuT7h8Q644yoCvJBnYkXT2lj+2WDvzCodzs0NpZz00/5XMn2zIhxyfxWi/zlywvgZJ/njXNyVxRKVgMD+77eknykI0wZJNn4hPB2sLxdPGuYrnlJQ37kALjPVImNE3LRfgiWA+dNVOFmNXQ7cnBTK9P+8FffEnNQI+qB11Vigbkiu0HltyY7MBHpZ9X4dg0OpeKHDxcpKpA3W5WgzfZWKCJY4g1ia4gn2zLYr90oPJ6miwzfbqY06bSHKzKY5PCd3nQacvtfVS6w== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CO6PR18MB3842.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(376002)(39860400002)(366004)(346002)(396003)(1006002)(66476007)(8676002)(5660300002)(52116002)(6506007)(6666004)(55016002)(8936002)(7696005)(66946007)(26005)(54906003)(316002)(83380400001)(6916009)(186003)(478600001)(86362001)(66556008)(956004)(9686003)(1076003)(33656002)(4326008)(16526019)(2906002)(36456003)(42976004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: X3C415Jgc25YmX3a3RLWJkOA4beElEm09v/ItcERMzS0Uikke0QUQF2bV7ITKl/HpZSzrMEvaG/lmWVXPdtfhcHPunRnXzxVe93N+szzsgyEfzmYbzzHqOtZnCH+9TSVC3dWTFF5JPwSHOwUscKXJXJyRknyDPwCIoosSX8RbyfWjL4H8ciiGBxa6bA9o3QxEx3aJaymkvv3oQGrBVezoPovfZicG7I93Owi2A82WqdNVhudy29mBk+9dA+0freVlXrottozM4nssA2hg4q7ac2/SotInvd8cr1tK02b+FlzSUrjKxACi0Tjdnk+CvkhOMIQb4aneFuLefmzeboRkXwRE40Epu5OIeJDOJzp6KDRSd+uKeqRPn8+t9HokyINxBo/kMYEeH/TJBqOlUlqnDJe4qV0Jr1fg58zkfWv4Q6f83zuorqhUSLoha6hivEIEIKKNP4Y3vQsodeM0uMfzb00SB4kOsY8XR/pfxeTdCRB5K8g8W/RCXIyBx7i8shH8P/MBriqmO+yxllnkYYiER75CPdTBUV1vxujBzWc46to0+Xzis5YggGdCmax7de15ccIF6azs6DKaFIKM6rlfC2VJmTvKNbZ8Zi1dt28tHoO4iAdu0mTSWEoZh1kj8Og+d0Ch7u6s1xfLNlAutP3Yg== X-MS-Exchange-CrossTenant-Network-Message-Id: cb0bb27b-732e-4ab0-8d83-08d87b81334d X-MS-Exchange-CrossTenant-AuthSource: CO6PR18MB3842.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Oct 2020 20:36:51.6425 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cuP+d0imvlhfVRPQemQvPH9bC0g6mnPIiUOcUYuOKycYgGTe12S97mrAmDSmApmWGjj3iF5NaPOvbRlES62/tA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR18MB3531 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.312, 18.0.737 definitions=2020-10-28_09:2020-10-28, 2020-10-28 signatures=0 Subject: Re: [dpdk-dev] [EXT] [PATCH v2 1/1] librte_eal: rte_intr_callback_unregister_sync() - wrapper around rte_intr_callback_unregister(). X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Aug 17, 2020 at 04:08:27PM +0200, Renata Saiakhova wrote: > External Email > > ---------------------------------------------------------------------- > Avoid race with unregister interrupt hanlder if interrupt > source has some active callbacks at the moment, use wrapper > around rte_intr_callback_unregister() to check for -EAGAIN > return value and to loop until rte_intr_callback_unregister() > succeeds. > Hi Renata, Just trying to understand the scenario, as you mentioned "while removing the device by rte_dev_remove()" are you calling rte_eal_hotplug_remove or kernel has sent an event to remove the device. As far as I know vfio notifier mechanism is used by kernel vfio driver to notify user to release the resources and as you are observing EAGAIN means same callback is executing. Regarding the tight polling loop in the patch, I think its good to have a fixed retry logic to avoid any unidentified corner case which might lead to infinite looping. Thanks Harman > Signed-off-by: Renata Saiakhova > --- > drivers/bus/pci/linux/pci_vfio.c | 2 +- > lib/librte_eal/freebsd/eal_interrupts.c | 12 ++++++++++++ > lib/librte_eal/include/rte_interrupts.h | 25 +++++++++++++++++++++++++ > lib/librte_eal/linux/eal_interrupts.c | 12 ++++++++++++ > lib/librte_eal/rte_eal_version.map | 1 + > 5 files changed, 51 insertions(+), 1 deletion(-) > > diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c > index 07e072e13..a4bfdf553 100644 > --- a/drivers/bus/pci/linux/pci_vfio.c > +++ b/drivers/bus/pci/linux/pci_vfio.c > @@ -415,7 +415,7 @@ pci_vfio_disable_notifier(struct rte_pci_device *dev) > return -1; > } > > - ret = rte_intr_callback_unregister(&dev->vfio_req_intr_handle, > + ret = rte_intr_callback_unregister_sync(&dev->vfio_req_intr_handle, > pci_vfio_req_handler, > (void *)&dev->device); > if (ret < 0) { > diff --git a/lib/librte_eal/freebsd/eal_interrupts.c b/lib/librte_eal/freebsd/eal_interrupts.c > index 6d53d33c8..7d99bdaff 100644 > --- a/lib/librte_eal/freebsd/eal_interrupts.c > +++ b/lib/librte_eal/freebsd/eal_interrupts.c > @@ -345,6 +345,18 @@ rte_intr_callback_unregister(const struct rte_intr_handle *intr_handle, > return ret; > } > > +int > +rte_intr_callback_unregister_sync(const struct rte_intr_handle *intr_handle, > + rte_intr_callback_fn cb_fn, void *cb_arg) > +{ > + int ret = 0; > + > + while ((ret = rte_intr_callback_unregister(intr_handle, cb_fn, cb_arg)) == -EAGAIN) > + rte_pause(); > + > + return ret; > +} > + > int > rte_intr_enable(const struct rte_intr_handle *intr_handle) > { > diff --git a/lib/librte_eal/include/rte_interrupts.h b/lib/librte_eal/include/rte_interrupts.h > index e3b406abc..cc3bf45d8 100644 > --- a/lib/librte_eal/include/rte_interrupts.h > +++ b/lib/librte_eal/include/rte_interrupts.h > @@ -94,6 +94,31 @@ rte_intr_callback_unregister_pending(const struct rte_intr_handle *intr_handle, > rte_intr_callback_fn cb_fn, void *cb_arg, > rte_intr_unregister_callback_fn ucb_fn); > > +/** > + * @warning > + * @b EXPERIMENTAL: this API may change without prior notice > + * > + * Loop until rte_intr_callback_unregister() succeeds. > + * After a call to this function, > + * the callback provided by the specified interrupt handle is unregistered. > + * > + * @param intr_handle > + * pointer to the interrupt handle. > + * @param cb > + * callback address. > + * @param cb_arg > + * address of parameter for callback, (void *)-1 means to remove all > + * registered which has the same callback address. > + * > + * @return > + * - On success, return the number of callback entities removed. > + * - On failure, a negative value. > + */ > +__rte_experimental > +int > +rte_intr_callback_unregister_sync(const struct rte_intr_handle *intr_handle, > + rte_intr_callback_fn cb, void *cb_arg); > + > /** > * It enables the interrupt for the specified handle. > * > diff --git a/lib/librte_eal/linux/eal_interrupts.c b/lib/librte_eal/linux/eal_interrupts.c > index 13db5c4e8..c99d5dbd4 100644 > --- a/lib/librte_eal/linux/eal_interrupts.c > +++ b/lib/librte_eal/linux/eal_interrupts.c > @@ -662,6 +662,18 @@ rte_intr_callback_unregister(const struct rte_intr_handle *intr_handle, > return ret; > } > > +int > +rte_intr_callback_unregister_sync(const struct rte_intr_handle *intr_handle, > + rte_intr_callback_fn cb_fn, void *cb_arg) > +{ > + int ret = 0; > + > + while ((ret = rte_intr_callback_unregister(intr_handle, cb_fn, cb_arg)) == -EAGAIN) > + rte_pause(); > + > + return ret; > +} > + > int > rte_intr_enable(const struct rte_intr_handle *intr_handle) > { > diff --git a/lib/librte_eal/rte_eal_version.map b/lib/librte_eal/rte_eal_version.map > index bf0c17c23..b1d824f59 100644 > --- a/lib/librte_eal/rte_eal_version.map > +++ b/lib/librte_eal/rte_eal_version.map > @@ -325,6 +325,7 @@ EXPERIMENTAL { > rte_fbarray_find_rev_biggest_free; > rte_fbarray_find_rev_biggest_used; > rte_intr_callback_unregister_pending; > + rte_intr_callback_unregister_sync; > rte_realloc_socket; > > # added in 19.08 > -- > 2.17.2 >