From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B30DAA0C55; Wed, 13 Oct 2021 19:57:08 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A21274118E; Wed, 13 Oct 2021 19:57:08 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by mails.dpdk.org (Postfix) with ESMTP id 6D55A41140 for ; Wed, 13 Oct 2021 19:57:07 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19DEsXT3013078; Wed, 13 Oct 2021 10:57:05 -0700 Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2174.outbound.protection.outlook.com [104.47.59.174]) by mx0b-0016f401.pphosted.com with ESMTP id 3bp0h3s7mx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Oct 2021 10:57:04 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kZTBVxNWhwtnAVhwPtYltcszz+C3uRmNtQiTvWEx0MFMFJAMAiC/4khthGYsbS0rnrHIFIMPk8XoKV55CqJiPmDR8wQezsH48OfpuK96awFGluPZT9Ffr8F9fSICL3eGSXTeNoYsRlvK/wVM1rT9JjqY6JU0kNCxxcTO75RlyhcFDPIJgXjNkpwySD+ISfEjGHOYnIcYtfCu0eWT8JE/YPmY2tH2RW7iml9gIjTO39teKOoBFMjaKigOaHPrFVrKrlUiOEfoG/3yGQbwcbcPWmt7yzCD/DdFlUp/cwDi+L7fLBr4jzzpjV+oB3+6wsdXeqDmqQCTIbj65tznZl8/mQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P5Zz0h10wlCa30+3zzSagg9zutFiipMdKZj7DziF5e8=; b=SD5WdKb9LMw7FKi7alNzRiJtmGX9NAoRF140yHjRtZVk0fyZS7GMG7VhsMvXA4FDDSZwangKdPnjN6LwdVfwZCXS4W2BlxXlutISe66TcwL14K4vue33KfnMNr0laYJsg1qRuT/h1R6Fq/RLy4BLBUi2ow7GPxOd9JVo7g+9255j9s8UgOF0dKLNEYVDwjinh38nJBbTi8HoqBmzypNe7Uun9lDZM2yuwTcPWiHO6+2OWw5W1rOR6EJolCRyrUwkZzMyPEU9b4d9uB+zn9JWP/sP+3j4j8kWuOEJZ/oxq8uwSxrWGqNS34cFIcReovaKAR0uCZ11t1kximFZfMpn6w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P5Zz0h10wlCa30+3zzSagg9zutFiipMdKZj7DziF5e8=; b=h5hOW8sWmQtSiN9AJODFW1OwC++4U62GWM2hnp4ixU6432kfNFBO61EGGiWl919B5rVp+LMMxohCuhUlUIIgZiIqgjhyaNW8S/n9OvASP8vsAvAb6gtV31rU2P96wOATcDgc76UVneOhRFAP1VM40UzMP+Byq3jvXFk1ywkVwLw= Received: from BN9PR18MB4204.namprd18.prod.outlook.com (2603:10b6:408:119::18) by BN8PR18MB2723.namprd18.prod.outlook.com (2603:10b6:408:76::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4608.15; Wed, 13 Oct 2021 17:57:02 +0000 Received: from BN9PR18MB4204.namprd18.prod.outlook.com ([fe80::29f4:8e3d:264f:26b1]) by BN9PR18MB4204.namprd18.prod.outlook.com ([fe80::29f4:8e3d:264f:26b1%8]) with mapi id 15.20.4608.016; Wed, 13 Oct 2021 17:57:02 +0000 From: Harman Kalra To: Harman Kalra , Thomas Monjalon CC: Raslan Darawsheh , "dev@dpdk.org" , Ray Kinsella , Dmitry Kozlyuk , David Marchand , "viacheslavo@nvidia.com" , "matan@nvidia.com" Thread-Topic: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implement get set APIs Thread-Index: AQHXv3z/vwDxAXEBi0ulm3+8amQc96vRLyBggAAJB3A= Date: Wed, 13 Oct 2021 17:57:01 +0000 Message-ID: References: <20210826145726.102081-1-hkalra@marvell.com> <14591983.bG23H4Sf1D@thomas> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 128b33cc-5797-4fbc-9969-08d98e72dc31 x-ms-traffictypediagnostic: BN8PR18MB2723: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Puvme3KruILSS9w+k1im8nKcK0he7BsMMazCm7eAumYpxO3oafgb4tvEq5z4X7UQWyEsVthRh7V4nlI+XQiSM6BbCdAQAcUoeHKp+wgRmcXspAV7JL1EILv1Ftb+0V8pU3j6E9MkWD+jIqfS6bBQf07LX4V4oaZMYCbzXxbp5PhhcYr+tGOZ5IxgZCOjQ5/lzxrzrXthmAzdPhs+ZvfpxkqLJwDSQss/zr0KrSrDqWcMf/gvfrb+UZ4dRvrL/UpvAZyKbmcme9IUVHKe9mH76E7TUNHINn2kOB2ajQpLmVGr/A2fkAhMwuteqsKLIJ98oeDJV5VpMHTfQmDVcMIyqxlnskqEWIwkol4XiGnNxYaxTMIMHYGz6nL4EZPiAqnxTPnWT6Sd2gji/IT5kpPpO+iDB95cNkeOJBLaXaGMcg8I2GeuiTTNtd4DhsN4pDO+/SXRArI7+XKQAIOwaY0Hn8/rmGcUy5UR5TpIH156QOClOA3krNAripr8MHGlrQAVjWGDZo2SJlTWpRDjoy2Pdmw9f7Z7HwDfpaxsTdaEKJp4Ud/l/RDS3ltPJlxii3+FCgEhGbaqRWRk4fsYh0P4wJxrsmCUIonle4k2Pb3Ts7j3uCwTGWBeZAwHQR9Ed1QzmnKX/1PKn5CQRwN6gu5AbPieAOnRf2dVcwarBGDapDyeJ9vD+ERfhGnALODq4J8cSox+ns7w7yPvwKvkiP+dhw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR18MB4204.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(186003)(66476007)(66556008)(64756008)(7696005)(66946007)(5660300002)(76116006)(71200400001)(26005)(38070700005)(33656002)(122000001)(316002)(4326008)(2940100002)(55016002)(38100700002)(54906003)(52536014)(53546011)(2906002)(508600001)(83380400001)(9686003)(110136005)(8936002)(6506007)(8676002)(66446008)(86362001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?SGiB2uETXkXVcFDg33ygoMJhuXUj0JZefTS1nlLkEJCaLNtSUxwwe5pJkNoK?= =?us-ascii?Q?+RhwEAnUtBKOBcjdt1wSJFg74luAEDhX+Syvd8JQDWAyUBv1LHhkV+YfWFyI?= =?us-ascii?Q?PSyUoVKjUxmmx4dHLt/9VSaynZSFeJn1WqXflm1yozzPv/3CQMdNozAn9iof?= =?us-ascii?Q?IkNXHmazJud/U0YYHWmrZenUb0G75jXYsfhNvxoweNzgPqAQbD7aG3VEUdVQ?= =?us-ascii?Q?vvhWP3fwXd3E15hF9G4v4/2KP8qONlDXKoO03qyGpE60VbsDXKu6OnULlyyd?= =?us-ascii?Q?Oqut2wQgR2vfBYUhasLM7RksJf1TnnxDaZUGe70x+anUHYFnTL42e6zqfsnb?= =?us-ascii?Q?51DX32ElhR9fWbZySiiu/m4a/Cmluz5E7kTnLJ9n8glIsY0GNE5ciW0VWoNE?= =?us-ascii?Q?ZqhyFR4SrCPa5DmmHgfctDMzVBwXDJzxga+BdqaBopqdHh25XOaDPC6fIGbb?= =?us-ascii?Q?ua/MF9eee1buqLzf+FRGbZA/rGnrpcy5BVs+XP1pA7R3bAMOVghOnTh4KN/M?= =?us-ascii?Q?oGcI2f1Do2bhrzalk8OYxKY3Q9XDQkyHtuIFsQQX8cZ0yrxmkc4Ck7ZTOm4T?= =?us-ascii?Q?+AQO3m7ThCApZjJskvZ4ROT+Sr2kM1vDPrAFYXSHWgzJ1O87gEIwB0K9hJTo?= =?us-ascii?Q?0sIgf7ttLVkPj50Cts1I9Qodnjbb+HjShBh/w2BK0ckVMtgZ8k8ff4kQ12TS?= =?us-ascii?Q?EtCZihZm/yfgP5T/3VBsa+1gFM+pX2k1DgUjtQUxtOlCDg08RZ052NTbVxWt?= =?us-ascii?Q?XKpiDt/mrR1BXWk1TfLsqdVQUh1O6e6oq9xzdTwWZYr5a/XVBqkS5HDYbiqZ?= =?us-ascii?Q?mjBmWTIBJGdNNgCpKUEroqcyvPCrwbt28yznnwFxuL7LjFHm9nZ3wI3w2RLz?= =?us-ascii?Q?oEmj6dEIP7Nt+bQ9bNyTjRDmJaB3Zx38Wgnmxv7t3CJuUriF6Bszy5dcdxQ9?= =?us-ascii?Q?lhAhzAU+qjhiNVDnsbwQGYrss3SAsv+x9VNgerdyKFBQLMJiQogvfaTb7cdJ?= =?us-ascii?Q?/tkTpIVqXusR8rGJy8SAC0aAxKYktP0u16TyoJRlxpE8MJ9jlTrZeDC0YMoR?= =?us-ascii?Q?ZxZDsk+fZXBvvSw0/MDINWqIpngsx2BRFCNk8YK1yTv0uXg41tK6jxzkJlDW?= =?us-ascii?Q?ngRdu5bh+NZT7p95KBsqcvPEmUKch9O5xeEPbYE6nk1TVNyxfbaKobsSem8B?= =?us-ascii?Q?C2ny0nEWz/4oWGx5tykrAi8Pr7MN9M2o18wJrXOM7NKaHIWS217vsJVbW0Up?= =?us-ascii?Q?hRAkxTJOhuECdCQolR8Xv7jhMWxwaLPGJTAM7qQawSA5xZnFVDxPoPjLLirw?= =?us-ascii?Q?6bPPLKg/XcPCdwObYvP5wb8o?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR18MB4204.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 128b33cc-5797-4fbc-9969-08d98e72dc31 X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Oct 2021 17:57:02.0025 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: SmgF4K2yjNhjuComRA0Yzp0dZgWK6kT7qLa+xZSjTcfPsgZA8gV4OzkeJqMbUentqmeRzJwXIEuAL3P9Y1xeHw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR18MB2723 X-Proofpoint-GUID: LkICdYflHF8vc4wVuGm0yZ2bj11EhV1f X-Proofpoint-ORIG-GUID: LkICdYflHF8vc4wVuGm0yZ2bj11EhV1f X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-13_06,2021-10-13_02,2020-04-07_01 Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implement get set APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: dev On Behalf Of Harman Kalra > Sent: Wednesday, October 13, 2021 11:24 PM > To: Thomas Monjalon > Cc: Raslan Darawsheh ; dev@dpdk.org; Ray Kinsella > ; Dmitry Kozlyuk ; David > Marchand ; viacheslavo@nvidia.com; > matan@nvidia.com > Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implemen= t > get set APIs >=20 >=20 >=20 > > -----Original Message----- > > From: Thomas Monjalon > > Sent: Tuesday, October 12, 2021 8:53 PM > > To: Harman Kalra > > Cc: Raslan Darawsheh ; dev@dpdk.org; Ray Kinsella > > ; Dmitry Kozlyuk ; David > > Marchand ; viacheslavo@nvidia.com; > > matan@nvidia.com > > Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: > > implement get set APIs > > > > 04/10/2021 11:57, David Marchand: > > > On Mon, Oct 4, 2021 at 10:51 AM Harman Kalra > > wrote: > > > > > > +struct rte_intr_handle *rte_intr_handle_instance_alloc(int siz= e, > > > > > > + bool > > > > > > +from_hugepage) { > > > > > > + struct rte_intr_handle *intr_handle; > > > > > > + int i; > > > > > > + > > > > > > + if (from_hugepage) > > > > > > + intr_handle =3D rte_zmalloc(NULL, > > > > > > + size * sizeof(struct = rte_intr_handle), > > > > > > + 0); > > > > > > + else > > > > > > + intr_handle =3D calloc(1, size * sizeof(struct > > > > > > + rte_intr_handle)); > > > > > > > > > > We can call DPDK allocator in all cases. > > > > > That would avoid headaches on why multiprocess does not work in > > > > > some rarely tested cases. > > > > > Wdyt? > > > > > > > > > > Plus "from_hugepage" is misleading, you could be in --no-huge > > > > > mode, rte_zmalloc still works. > > > > > > > > In mellanox 5 driver interrupt handle instance is freed in > > > > destructor " mlx5_pmd_interrupt_handler_uninstall()" while DPDK > > > > memory allocators are already cleaned up in "rte_eal_cleanup". > > > > Hence I allocated interrupt instances for such cases from normal he= ap. > > > > There could be other such cases so I think its ok to keep this supp= ort. > > > > > > This is surprising. > > > Why would the mlx5 driver wait to release in a destructor? > > > It should be done once no interrupt handler is necessary (like when > > > stopping all ports), and that would be before rte_eal_cleanup(). > > > > I agree with David. > > I prefer a simpler API which always use rte_malloc, and make sure > > interrupts are always handled between rte_eal_init and rte_eal_cleanup. > > The mlx5 PMD could be reworked to match this requirement. > > In any case we should not any memory management in > > constructors/destructors. >=20 >=20 > Hi Thomas, David >=20 > There are couple of more dependencies on glibc heap APIs: > 1. "rte_eal_alarm_init()" allocates an interrupt instance which is used f= or > timerfd, is called before "rte_eal_memory_init()" which does the memseg > init. > Not sure what all challenges we may face in moving alarm_init after > memory_init as it might break some subsystem inits. > Other option could be to allocate interrupt instance for timerfd on first > alarm_setup call. >=20 > 2. Currently interrupt handle field inside struct rte_pci_device is stati= c which > is changed to a pointer in this series(as struct rte_intr_handle is hidde= n inside > a c file and size is unknown outside). > I am allocating the memory for this interrupt instance inside > "rte_pci_probe_one_driver()" just before "pci_vfio_map_resource()" which > sets up vfio resources and calls " pci_vfio_setup_interrupts()" which set= ups > the interrupt support. Here challenge is "rte_bus_probe()" also gets call= ed > before "rte_eal_memory_init()". Sorry my bad, bus probing happens after "rte_eal_memory_init()", so second = point Is invalid. >=20 > There are many other drivers which statically declares the interrupt hand= les > inside their respective private structures and memory for those structure > was allocated from heap. For such drivers I allocated interrupt instances= also > using glibc heap APIs. >=20 > Thanks > Harman >=20 >=20 >=20 > >