From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 63CE3A0C47; Thu, 14 Oct 2021 12:44:47 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E343E411FC; Thu, 14 Oct 2021 12:44:46 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 0E5F740041 for ; Thu, 14 Oct 2021 12:44:44 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 19E89Hnu018493; Thu, 14 Oct 2021 03:44:41 -0700 Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2175.outbound.protection.outlook.com [104.47.57.175]) by mx0a-0016f401.pphosted.com with ESMTP id 3bpgsr8nsu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 14 Oct 2021 03:44:41 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HfTiDEUNy1Yh1IlrEeOomMaQw56b+3mPXYYRDX7jEaZwk+vKLGnrS0nyM/sWudXrpVAcwkhcQGyYA02Pelnm2lxvWee4deiAyIAPsonBLqxkxmX/B9bOju2AHDT4dsS+5acjzVDgPrTIutGyavb01kEsib0G6/KC9G12cnzu/1QRTQSR3teI3trxKDjN3nv27jb77RTX3jGUJBXB9S/C/0VrZQGx8ATxTRKLGMCucBzkydkVdZIVAM8mmkWx8Qprs/JrpfqGCKg5nVqfk17pPvBTvugym4X+UvlT2kmRR/WcMCR3Wa8ezs+ZSEkZD4QoKIAuz7lcuSZI9QuNVj6Cbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3BpFO7J+l5a9S6r5b8G8NVitsGSDT5s9d/E2dKIOpL0=; b=NN+g42ILXY5DvDPXPax7rx+7lwfZa6r1vpzmewSPuWpmApYAwH7YR5F93Yhg2aJVCIUOZcjfynSnBUiabfAY8CO7j3VomOFKa/wK1HOnFiw+Z/NkmIAiHCYEA7QUHX+ZjKHA5vxYKW+faWfUq+W1lnFnXD5h7yKziDA74x82bn/lm77/2MVWDPemA/Tf0qzqnd3fnd9P+N0/ETIkc0BEnhTGyqugOU+DrnsbcFZxastu77Tktq96KaWdDIwsEL0B5T2hZlEUNAqJIAOkQPDr3XGK6Z8ys4BanRmV+JOUlDwQfEbmcJmXkv2q0fpMWAq4urDn6U99gnfP8ExLQdlraA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3BpFO7J+l5a9S6r5b8G8NVitsGSDT5s9d/E2dKIOpL0=; b=evwAmepl6woHhrHd7zReUv0/qye6+PWG8yPh/BHFMzHlUZztbzSWHBk7EVHvExPf+CJ7cz2DctMP2JwZswQvda5NU736UtJZhJw/2xKuFYsFZdrVhlcUNbnft8+EOViyoUsWTIq+RsRlSr7PwhP4+Iw+5pL1sEU+dKyMGQK7ASE= Received: from BN9PR18MB4204.namprd18.prod.outlook.com (2603:10b6:408:119::18) by BN6PR1801MB1938.namprd18.prod.outlook.com (2603:10b6:405:63::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.24; Thu, 14 Oct 2021 10:44:39 +0000 Received: from BN9PR18MB4204.namprd18.prod.outlook.com ([fe80::29f4:8e3d:264f:26b1]) by BN9PR18MB4204.namprd18.prod.outlook.com ([fe80::29f4:8e3d:264f:26b1%8]) with mapi id 15.20.4608.016; Thu, 14 Oct 2021 10:44:39 +0000 From: Harman Kalra To: Thomas Monjalon CC: David Marchand , "dev@dpdk.org" , Raslan Darawsheh , Ray Kinsella , Dmitry Kozlyuk , "viacheslavo@nvidia.com" , "matan@nvidia.com" Thread-Topic: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implement get set APIs Thread-Index: AQHXv3z/vwDxAXEBi0ulm3+8amQc96vRLyBggAAJB3CAAA+/gIAA4m4AgAAQGMCAAAXhgIAAC0YggAAD4gCAAACAkA== Date: Thu, 14 Oct 2021 10:44:39 +0000 Message-ID: References: <20210826145726.102081-1-hkalra@marvell.com> <4395254.EVvzvEdfqG@thomas> <2130106.uiXJXLAz8d@thomas> In-Reply-To: <2130106.uiXJXLAz8d@thomas> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 985194e8-0976-4423-97b5-08d98eff9fd2 x-ms-traffictypediagnostic: BN6PR1801MB1938: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: MNrQwV37fh0i5Npo1ZT08fo7A2TJoF2pKwXN1hvcupoex2Q95oxTtmLs3VWAG5tI1aG1fFXKXFubO2qPO1/4Bayk5H3cJz1ID38PHMEsd85RnluJ1aH4eNX3wqqI7PvKEdOBHOqWsGUYqsK4PtpJ22IswKDMd2CV6Rlh6Ipyg8CwBaX2aH3CBcyb32uCq0tZU1SrM3aqr/BC3a9gCzIJenkSqUecQ6wxXGHXf1Sc9dYkSLTFxUS8nHS8Vmg94m7wxtsntyiv/s+0OkWgmMUssy2cSpfx1ldwoimuh9bPANSVFKw/hdT94C9BpuXjF5feOxhCdAAAlfRzdbXrsz5E33PfyJKw5lkxWmsqmzloercN+KAbOvR+qtbLKK5yR8S6u48ARWpUaTh+qPp1j+EPfTMp0g0G8iFagRqs2/9l9ncALCEASv1f7+xiakLDPmkBTMLKQ4aHcVsPC6KX8Q15UxoQRpLRTTwYP0ZUW0tODz6AEou3l4bd48fRiB4pdT3YbqRQbYI/KNAPBCO7qqgrN4riWDkK5y5Rb17TvGtSDJjnX5RAiXucoFFjgGCl3PUoYrLnSJJXLkGGNGX06DX59xOr2xFXaKETmAGH/OhvQQeT57/gP9ntpq3o1AdPhhdFrlnHXZmUcIB+uVSX0h/AFZCIVZ2Eu/KA/45CxcpKBmYN+62etKVhpQ6VPjz8IBMBj4VOoRfoMBYPevpfUMC36g== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN9PR18MB4204.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(316002)(52536014)(76116006)(71200400001)(4326008)(54906003)(8936002)(33656002)(186003)(86362001)(8676002)(66446008)(64756008)(66476007)(9686003)(66946007)(66556008)(508600001)(6916009)(5660300002)(38070700005)(38100700002)(122000001)(55016002)(2906002)(83380400001)(7696005)(53546011)(6506007)(26005); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?3N3srB7zwr5f+/ysWR0OUUVtVq0fMMpq24nnge057v4Me/B5fKOxXrxtmu4l?= =?us-ascii?Q?RNbZ+Oj+/4tmsBCnSRGd+qlJQHc9Zx+eLwlMY15DVM45FF/eQdGMXkkms1qV?= =?us-ascii?Q?41nIfHYLkLFxgl/6MEV2r2kzk6nUS4LIjR8PYZu1q7aDKNMM3VGYEMdw1iOL?= =?us-ascii?Q?MH/F1SRpvUadc7mXSy5rg+K31JyT/76Z27SxWJE/ANkeIBAzYwLeVzjDWE46?= =?us-ascii?Q?PamPQdE8u6YJVM/omXMLFKAUN1j0J8CqMwyNFTwSDIi3tgEEeHfJWCVLTnxm?= =?us-ascii?Q?OY46ZVGAy2drLtYWQE8Rwq8PtzBtKkgxbJZRr2c9wERlkrCNAswnrHI/iwMJ?= =?us-ascii?Q?ncz4hCqaxfNCvAUxKE6+sTS88k/1onjudDN4YdzNqkbJpQZ9uHJ8kVbv3u8M?= =?us-ascii?Q?gt+/zdPLD9TwGSrqtedfe6llyJCNCO2HQbYeqPQQCCvkF54Q1I9IFTyL/8iE?= =?us-ascii?Q?lODrOEgkVr3wOnNoSANh4QhrByt2/DxQrdVhQB+URz42EWuDbvWdbqiiOvri?= =?us-ascii?Q?q6LXPKNbS+22T1VlXs0NhNRbRdTyz6fwsaxhJNAvvpYsrIzH0aTSIRuYLuV9?= =?us-ascii?Q?aPJFrJwPq9V/JjiWaY1Jbqq3ZIdBJDvhweg9uy+kCp5DJ3z/bazXBtn3hJqw?= =?us-ascii?Q?tQoGrEn9zZJuLdoGN5Y+G7wKDO5eCtI9E5jk53Ei99cVrTKjUCIvgJ51+Ccg?= =?us-ascii?Q?pwZZvH5NE4dDE5WlhzAyZlQMBO7Cc19QE8sIZNBFSrc7Ye/N68oRdA4XrAfh?= =?us-ascii?Q?8Ihj+NGlaE1YlXqVcnG4pKaCJYn6JeCYav9JhNjZsRAArjf4EM/g9/YnNAWw?= =?us-ascii?Q?diIeRkp2ICKjoa3MlThCZof4j2crBt/+dnyUz4DZOSRuNcgWfY0iq4O6kSvj?= =?us-ascii?Q?wM90ckzO+3sTDN3q2qNET9pCIW6XzzVbwqxpztcSrGDz2hZhdSINvRZTg+oC?= =?us-ascii?Q?YGOjIussrUvqyd/qNb6wwZ2qarGADqzDTPUbOV0z7RgXAPmh1lap6WQnniHV?= =?us-ascii?Q?MqodKMkwE3Y8s66ka+PHOXE+NroWVJdijSCowJ5XazY4XiziP9w0ipXCQrfX?= =?us-ascii?Q?6UCNLd14OR0qsvR4kjEhviSImg8s2OfTN3aIoTEbwNIEp9bSsaDXT64wZLiJ?= =?us-ascii?Q?6VIqwCUgA7tfhlZBvmwGd8E94WsR15WSPH0x6gIWIhewnA0rcQuK47/gB+C+?= =?us-ascii?Q?2RmUriW0C+aT5EyQWmZ1AY25F+ZmQe+DMoHfbCPQwx6EtWeA5muhVwmc08Bw?= =?us-ascii?Q?ofLF7zlF9o3tEG3xu7QCX5VhifLL0Ek1k3eaf/VcIfN0b/0RRVPjjYblcJ/u?= =?us-ascii?Q?R2tnGsEjS+Ostoziik1FgJUL?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR18MB4204.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 985194e8-0976-4423-97b5-08d98eff9fd2 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Oct 2021 10:44:39.6867 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: FLzhtflVWH3ouu1VEycpGgPltrPturkVHPGwzgIdNEYbDrToebXKhHIUCN8avUbZyFdLjMJT8TTHY2zC/Mg1pQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1801MB1938 X-Proofpoint-GUID: KN2-7iv3_0STqhc_iFlQFflGqoLzgtJ0 X-Proofpoint-ORIG-GUID: KN2-7iv3_0STqhc_iFlQFflGqoLzgtJ0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.0.607.475 definitions=2021-10-14_03,2021-10-14_01,2020-04-07_01 Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implement get set APIs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Thomas Monjalon > Sent: Thursday, October 14, 2021 4:06 PM > To: Harman Kalra > Cc: David Marchand ; dev@dpdk.org; Raslan > Darawsheh ; Ray Kinsella ; Dmitry > Kozlyuk ; viacheslavo@nvidia.com; > matan@nvidia.com > Subject: Re: [dpdk-dev] [EXT] Re: [PATCH v1 2/7] eal/interrupts: implemen= t > get set APIs >=20 > 14/10/2021 12:31, Harman Kalra: > > From: Thomas Monjalon > > > 14/10/2021 11:31, Harman Kalra: > > > > From: Thomas Monjalon > > > > > 13/10/2021 20:52, Thomas Monjalon: > > > > > > 13/10/2021 19:57, Harman Kalra: > > > > > > > From: dev On Behalf Of Harman Kalra > > > > > > > > From: Thomas Monjalon > > > > > > > > > 04/10/2021 11:57, David Marchand: > > > > > > > > > > On Mon, Oct 4, 2021 at 10:51 AM Harman Kalra > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > +struct rte_intr_handle > > > > > > > > > > > > > +*rte_intr_handle_instance_alloc(int > > > > > size, > > > > > > > > > > > > > + > > > > > > > > > > > > > +bool > > > > > > > > > > > > > +from_hugepage) { > > > > > > > > > > > > > + struct rte_intr_handle *intr_handle; > > > > > > > > > > > > > + int i; > > > > > > > > > > > > > + > > > > > > > > > > > > > + if (from_hugepage) > > > > > > > > > > > > > + intr_handle =3D rte_zmalloc(NULL, > > > > > > > > > > > > > + size * = sizeof(struct > rte_intr_handle), > > > > > > > > > > > > > + 0); > > > > > > > > > > > > > + else > > > > > > > > > > > > > + intr_handle =3D calloc(1, size * > > > > > > > > > > > > > + sizeof(struct rte_intr_handle)); > > > > > > > > > > > > > > > > > > > > > > > > We can call DPDK allocator in all cases. > > > > > > > > > > > > That would avoid headaches on why multiprocess > > > > > > > > > > > > does not work in some rarely tested cases. > > > > > [...] > > > > > > > > > I agree with David. > > > > > > > > > I prefer a simpler API which always use rte_malloc, and > > > > > > > > > make sure interrupts are always handled between > > > > > > > > > rte_eal_init and > > > > > rte_eal_cleanup. > > > > > [...] > > > > > > > > There are couple of more dependencies on glibc heap APIs: > > > > > > > > 1. "rte_eal_alarm_init()" allocates an interrupt instance > > > > > > > > which is used for timerfd, is called before > > > > > > > > "rte_eal_memory_init()" which does the memseg init. > > > > > > > > Not sure what all challenges we may face in moving > > > > > > > > alarm_init after memory_init as it might break some subsyst= em > inits. > > > > > > > > Other option could be to allocate interrupt instance for > > > > > > > > timerfd on first alarm_setup call. > > > > > > > > > > > > Indeed it is an issue. > > > > > > > > > > > > [...] > > > > > > > > > > > > > > There are many other drivers which statically declares the > > > > > > > > interrupt handles inside their respective private > > > > > > > > structures and memory for those structure was allocated > > > > > > > > from heap. For such drivers I allocated interrupt > > > > > > > > instances also using glibc heap > > > APIs. > > > > > > > > > > > > Could you use rte_malloc in these drivers? > > > > > > > > > > If we take the direction of 2 different allocations mode for the > > > > > interrupts, I suggest we make it automatic without any API parame= ter. > > > > > We don't have any function to check rte_malloc readiness I think. > > > > > But we can detect whether shared memory is ready with this check: > > > > > rte_eal_get_configuration()->mem_config->magic =3D=3D RTE_MAGIC T= his > > > > > check is true at the end of rte_eal_init, so it is false during p= robing. > > > > > Would it be enough? Or should we implement rte_malloc_is_ready()? > > > > > > > > Hi Thomas, > > > > > > > > It's a very good suggestion. Let's implement "rte_malloc_is_ready()= " > > > > which could be as simple as " > > > >rte_eal_get_configuration()->mem_config- > > > >magic =3D=3D RTE_MAGIC" check. > > > > There may be more consumers for this API in future. > > > > > > You cannot rely on the magic because it is set only after probing. > > > For such API you need to have another internal flag to check that > > > malloc is setup. > > > > Yeah, got that. You mean in case of bus probing although rte_malloc is > > setup but eal_mcfg_complete() is calledt done yet. So we should set > > another malloc specific flag at the end of rte_eal_memory_init(). Corre= ct? >=20 > I think the new internal flag should be at the end of > rte_eal_malloc_heap_init(). > Then a rte_internal function rte_malloc_is_ready() should check this flag= . Sure. >=20 > > But just for understanding, as David suggested that we preserve keep > > this flag then why not use it, have rte_malloc and malloc bits and mak= e a > decision. > > Let driver has the flexibility to choose. Do you see any harm in this? >=20 > Which flag? In V2, I have replaced the bool arg with an 32bit flag in alloc api: struct rte_intr_handle *rte_intr_instance_alloc(uint32_t flags); Declared some flags which can be passed by the consumer /** Interrupt instance allocation flags * @see rte_intr_instance_alloc */ /** Allocate interrupt instance from traditional heap */ #define RTE_INTR_ALLOC_TRAD_HEAP 0x00000000 /** Allocate interrupt instance using DPDK memory management APIs */ #define RTE_INTR_ALLOC_DPDK_ALLOCATOR 0x00000001 As a future enhancement, if more options to the allocation is required by u= ser, new flags can be added.=20 Thanks Harman >=20 >=20