From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E2F88A0C52; Wed, 13 Oct 2021 19:54:36 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id CEBED4118D; Wed, 13 Oct 2021 19:54:36 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 712784111A for ; Wed, 13 Oct 2021 19:54:35 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19DEsY0L013092; Wed, 13 Oct 2021 10:54:32 -0700 Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2104.outbound.protection.outlook.com [104.47.58.104]) by mx0b-0016f401.pphosted.com with ESMTP id 3bp0h3s795-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Oct 2021 10:54:32 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Q/BaZUlrQEYd5V2d1wp4xGTMwhs1PI3+IA62PFZwFIaHZzfPcfVodLKpXUtTSwQC//piALv2P57u1j2HOr7lrhW60kEd2O2qxr+FY3nTcQsTp+SSX4iU/EcBTSRCwO1FM00Tv0oi/7a5J09EQWM+oxQZ8ycf2M7iOuUPIjXoJv/70JDto+dRcmANptDqPrFeIazlvjBmeSfeuxwpTPBvhAssTjHw9V/XGD2zzbFk6oJxqAdyiRekknSmZ/+ZokwSrpXmTYQtgJfpqjybif6c1Wfo/T3CNukcJR934JAGGT6V46PjHqsGIx3VhKPqaUie1JJKzWl8cksrlS2pVl9alQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=g4TaIItusOnaPBbHmRaSILMdg6ns1LzMayGyvMtlVrk=; b=fxKWrnD0M5tz6ll+5t1laCr3D/8X+tnq+2InZdu+Z3eANiZxeIxUvq8MRAUijN49epABklnNVINngX/jT3aEyl4LgKKAx+H62Tv7T8sUId5eDQSiG8iwDruCbUw9bowkDfQV112Jo6FrHuximvD2lU8KG70Y6Q5ILWLMXFjcFoJLdl7LBy8BXWYG464jSxgA/tv7zkL5r0GOyf9AfOLNQXr1jZMSI81HPbUDVcDxCUM+Oxe3wnIPjh97dymlKxKlFi0TuJ64zEslipLMLrysOhOT1zgRBoqZ5LBXF+V3yvDw7vdtgU0aZkblV/N3OKPK3UWzCRulWMh3F0syDdjf2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=g4TaIItusOnaPBbHmRaSILMdg6ns1LzMayGyvMtlVrk=; b=FhujjpkHQlsav23jdH21cwG2Z35tObVTv6dyyYkMStyK2VXXoD+4kS+DF3BnceNus3ymMpFhsBPDBc4BlIVwGoGj5cKh4pDVVBf9tzHAESGwcwKUKM3JXznUcc8FsuC5c8FKvZrPmn1xYc3pTCXpPHJlGCZBIBnv85MN/eno6D4= Received: from BN9PR18MB4204.namprd18.prod.outlook.com (2603:10b6:408:119::18) by BN6PR18MB1634.namprd18.prod.outlook.com (2603:10b6:404:12e::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.16; Wed, 13 Oct 2021 17:54:29 +0000 Received: from BN9PR18MB4204.namprd18.prod.outlook.com ([fe80::29f4:8e3d:264f:26b1]) by BN9PR18MB4204.namprd18.prod.outlook.com ([fe80::29f4:8e3d:264f:26b1%8]) with mapi id 15.20.4608.016; Wed, 13 Oct 2021 17:54:29 +0000 From: Harman Kalra To: Thomas Monjalon CC: Raslan Darawsheh , "dev@dpdk.org" , Ray Kinsella , Dmitry Kozlyuk , David Marchand , "viacheslavo@nvidia.com" , "matan@nvidia.com" Thread-Topic: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implement get set APIs Thread-Index: AQHXv3z/vwDxAXEBi0ulm3+8amQc96vRLyBg Date: Wed, 13 Oct 2021 17:54:29 +0000 Message-ID: References: <20210826145726.102081-1-hkalra@marvell.com> <14591983.bG23H4Sf1D@thomas> In-Reply-To: <14591983.bG23H4Sf1D@thomas> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 046824e1-ab51-4949-9a18-08d98e72812f x-ms-traffictypediagnostic: BN6PR18MB1634: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: i4kp2s0JiZ0EiWE+Kv2OMgVDgaFSdqqtdJxoryl+KerOgyYiI5FLHWPiTAJA9CizYiy/w4rq4C/x8cggljsk2uqN6vv5gZ0jvxxJZeT6DHLPv97TeNczaugO3Gi1thL/nRdRDs0bIxstU//2UPHPd+vJRrJG+fyysBTjOD3GpuFfLWonBNl6OeQlB7c/0FwpgiWygTYswVuaHBGcGRkeqOR8PhG0a+isj+1MW/P6upaj8KKusHCQ0i0q1fLInNVZAGRP+12HxRBTvvYQOKDHpkrlFkKmozH/EylnRaVzTxGrEqbadVjUtetz8ZKjmeETZJ+8M7YfpnBPcdwx++FQ+Jk4vIlk5K/+w5ycBNR8ep21AHZX7NXXxOvyQmdQPSYv4NQDsK4OaKsTxCCgrOqCuznrrw54+c+ZlpZbrOD8JjvPSR4CGzPXrL1CB53J64IAohb9RgFAAaNKsadmYoV1sA9t+Px8vGNmcmFLR2juWmXNjBjZEjfZ+e1WzJ/4XLFAf6J01Fqeld1Qd1+qqnEQPKCHz9lHTFW0ZXn9u/F9TczUot+YazBMkoMLw6mG67yDSLqXzlwD1f0pCUe8CHhsrTBeFCHDBbkoSPgYsQQoJg/0lplic/bJ2aF3RmTLlCNd83OThCDZlXNS1e1v7PdVi16bus4V4nKLLSCtyobJqh93SpTXT9Iqo94TtsvLdGP8IdTgph/TDzqI5k2gLViTtw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR18MB4204.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(4326008)(7696005)(66556008)(55016002)(66476007)(66946007)(76116006)(53546011)(8676002)(6506007)(26005)(33656002)(6916009)(71200400001)(186003)(64756008)(66446008)(2906002)(86362001)(9686003)(5660300002)(8936002)(52536014)(316002)(54906003)(38070700005)(83380400001)(122000001)(38100700002)(508600001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?YqMXo+hllm5c+WDeDHxgFs7PjJuIuFLU+5b0eyTJWeZmodWmdGq55eELAYL0?= =?us-ascii?Q?7xmF1brZDp18Umi1W4MJm+kTAHEeeHa30K+R95xb+EzaszBEZiMq7UuPadz0?= =?us-ascii?Q?IeAh3epwMtNB5AyErRKK5Mem/BOZmrGzARj55r19tEu3/qAja/UrCzH3L66O?= =?us-ascii?Q?KdREmGro+S528yxpsxSnN0GGrFBYQIiULvRgwbLXXnW8H+R3tCQsvOUK9wvM?= =?us-ascii?Q?eSA4R54Hhb/AbcrDgO6ZbAaBLRcBEmAgk2AdJTU/wDGt885vJYC9GgJEYz6C?= =?us-ascii?Q?nx9YO+lNF8ycddBcG2dp60ZvhA1MW1advDsz/GAlc9pjDDpn3xfnaB0qiWhi?= =?us-ascii?Q?pfHxfdCXvQUmkYXsC50u2F0Swe6DLPoX9qxEJIb6LAIXxUXGveTm20rUicWX?= =?us-ascii?Q?K05GeFaDmiBDxNJMyyrkPyBkGB1kPbYTt35T9LKtT6DDBVJzMxWh9WQjlyFO?= =?us-ascii?Q?FFRzD8mlmjF3wHvPlIivcOQrTu9YgSJuuWtb3UxlhVvTDYJdlPag4HI1l7qj?= =?us-ascii?Q?p9gaipEwG/P4MhhqJt/5/jaVbCBaLf6/pwnd7+vj6BmMXdQipPaqDi0jcJWO?= =?us-ascii?Q?CiRt0nuBom9seebmORtDvMXegXX8b/Zro7DY3/GcVbdQiLgdGSs8gGidcQGA?= =?us-ascii?Q?Qj+66YfLjtzjlrhA3/qWC62hpewuIpcMkGO9xidXlSTjXiXRF/vP67YqDNFG?= =?us-ascii?Q?4opum8eMvBa8U59mOUvy0PQQJAEsmswrGmKJPG67D2NPM8W64BQjLi+0gBHq?= =?us-ascii?Q?z4UYFuA0aWHH9Pzi2wj0rw3RwXNr/s/YfpkJjwQ3XJBAVpnSGBkH3XJF9wMQ?= =?us-ascii?Q?HS/kwwlZgIAVgO9Zgh1o2t3P0BQxEu2cWD/uVrAjqCeEPBbPFIYOyYs9/0dT?= =?us-ascii?Q?lha+FzTaJJHiVgHBcIdtVORENR+9Snca8jKITlcwqGmX+jT+coK8yiezz+ma?= =?us-ascii?Q?RCHZQig0VgtdhjK7RJOyRzq8nun28aKQkcAlx+jhuXVF3LPg8fzf6P3QgoSE?= =?us-ascii?Q?3obwtmmlqn/7CIfCBBwJzRtg5YG3NoBKXx3V1JV2T+DB07L2skCXO17P1Vu8?= =?us-ascii?Q?SxrsmfdQG9wn3ZIsKexu3ATRRWlJM0Lk9ya7axA4FBc9TdwCixXXKtV+VccT?= =?us-ascii?Q?9yLuYQoQFYM1ouVMcPm1IWtmrbLG/Lia1jv4r+ZsFgPfowMzFTkoevYbTOw9?= =?us-ascii?Q?QsjfDf7N5cE+XMDxNJdklWWIk1U79Jv6t6q+gsuRkVDhuOouR8/pr8ts3ksa?= =?us-ascii?Q?1j4wkBnyzzep7n/HvWpvxO63Xbh6Vj8yjQ2Z4cpQSwCYixb76BRZDFsRQsPR?= =?us-ascii?Q?tPKvlAqrMWjXVhCapExj52GI?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR18MB4204.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 046824e1-ab51-4949-9a18-08d98e72812f X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Oct 2021 17:54:29.2978 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 2eh0bcCk5uTL1C5Caqge1wxJ9n1P5d9DKJbS7seN9ihzN4S1iEoPxyBMUkTSoTlJLbDab2A8Ygh2lMMQ1DFSXw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR18MB1634 X-Proofpoint-GUID: dgnMAYxGp-sa6p0gWB4Kb1LeRsmwK0XB X-Proofpoint-ORIG-GUID: dgnMAYxGp-sa6p0gWB4Kb1LeRsmwK0XB X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-13_06,2021-10-13_02,2020-04-07_01 Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implement get set APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Thomas Monjalon > Sent: Tuesday, October 12, 2021 8:53 PM > To: Harman Kalra > Cc: Raslan Darawsheh ; dev@dpdk.org; Ray Kinsella > ; Dmitry Kozlyuk ; David > Marchand ; viacheslavo@nvidia.com; > matan@nvidia.com > Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implemen= t > get set APIs >=20 > 04/10/2021 11:57, David Marchand: > > On Mon, Oct 4, 2021 at 10:51 AM Harman Kalra > wrote: > > > > > +struct rte_intr_handle *rte_intr_handle_instance_alloc(int size, > > > > > + bool > > > > > +from_hugepage) { > > > > > + struct rte_intr_handle *intr_handle; > > > > > + int i; > > > > > + > > > > > + if (from_hugepage) > > > > > + intr_handle =3D rte_zmalloc(NULL, > > > > > + size * sizeof(struct rt= e_intr_handle), > > > > > + 0); > > > > > + else > > > > > + intr_handle =3D calloc(1, size * sizeof(struct > > > > > + rte_intr_handle)); > > > > > > > > We can call DPDK allocator in all cases. > > > > That would avoid headaches on why multiprocess does not work in > > > > some rarely tested cases. > > > > Wdyt? > > > > > > > > Plus "from_hugepage" is misleading, you could be in --no-huge > > > > mode, rte_zmalloc still works. > > > > > > In mellanox 5 driver interrupt handle instance is freed in > > > destructor " mlx5_pmd_interrupt_handler_uninstall()" while DPDK > > > memory allocators are already cleaned up in "rte_eal_cleanup". Hence > > > I allocated interrupt instances for such cases from normal heap. > > > There could be other such cases so I think its ok to keep this suppor= t. > > > > This is surprising. > > Why would the mlx5 driver wait to release in a destructor? > > It should be done once no interrupt handler is necessary (like when > > stopping all ports), and that would be before rte_eal_cleanup(). >=20 > I agree with David. > I prefer a simpler API which always use rte_malloc, and make sure interru= pts > are always handled between rte_eal_init and rte_eal_cleanup. > The mlx5 PMD could be reworked to match this requirement. > In any case we should not any memory management in > constructors/destructors. Hi Thomas, David There are couple of more dependencies on glibc heap APIs: 1. "rte_eal_alarm_init()" allocates an interrupt instance which is used for= timerfd,=20 is called before "rte_eal_memory_init()" which does the memseg init. Not sure what all challenges we may face in moving alarm_init after memory_= init as it might break some subsystem inits. Other option could be to allocate interrupt instance for timerfd on first a= larm_setup call.=20 2. Currently interrupt handle field inside struct rte_pci_device is static = which is changed to a pointer in this series(as struct rte_intr_handle is hidden inside a c = file and size is unknown outside). I am allocating the memory for this interrupt instance inside "rte_pci_prob= e_one_driver()" just before "pci_vfio_map_resource()" which sets up vfio resources and calls " pci_vfio= _setup_interrupts()" which setups the interrupt support. Here challenge is "rte_bus_probe()" also gets called= before "rte_eal_memory_init()". There are many other drivers which statically declares the interrupt handle= s inside their respective private structures and memory for those structure was allocated from heap. For such= drivers I allocated interrupt instances also using glibc heap APIs. Thanks Harman =20 >=20