From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 95C17A0C41; Fri, 8 Oct 2021 07:38:31 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 187B040040; Fri, 8 Oct 2021 07:38:31 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id 14E924003C for ; Fri, 8 Oct 2021 07:38:28 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 1984s2Wj001282; Thu, 7 Oct 2021 22:38:28 -0700 Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2040.outbound.protection.outlook.com [104.47.66.40]) by mx0a-0016f401.pphosted.com with ESMTP id 3bjfc984ch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 07 Oct 2021 22:38:27 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LBGNhIDPwr0lzBRWypWXqQUeL+jBgIJlBHBISf+bTDclPBzb0mCNa38pJiys8/KiuzwPQpXofcGNvRlF91XIDTuCdUgVg1a7d5xk5lHMSgo2ZQLoVBA75lUYgdIFPZYITYDrbGwMx9BOEN5x/Se6XBLcv05yxWRKx1RKIWSmgzAuUynCZDBLPXOz1BHOBqob6bEFl9Ug/BZscpBMogZzp/k0BJl21RzVoilAin18XlbXmh86hRuPadQsDAoyoP/WuWGiIJz5mgOLPS5ZDJqdj4NoI2/wzNCaWpqthrTrHSUfwZ5m8zh/pZLLMIEawTIKKdyQtZosdzat4r+GKTGdFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NVIerZsWyChvX6CZT7e6LLlaoerUJ95H/oNTdwIWn2g=; b=j6LAdhdzkprFj6b7UFjP4QIrVaFu6xBd4t6LXoqlQ7oRZ87ct7DMGcNwzv+6a1EPRoA+n717J8Gna8l/F9t3V3KbcrwpKBr4ve0dB0cEGgAXiflpw+T9dqlcZ9TGg+EpxH9ivjP9nZReJE1rbFQDbout/CgzaWfxT5hBlpoNt8DW8iBYTniiNyq1reFgnbAJSsskdoGKlJN2p9BZ5hlSu7b73aDEdwOMWjVgaJPyc+/MLcZj8RGaYnRODLHqW4DxHGk/U28gxmLTKq7mrS9toQrr+6PumdyhqFafMVOkPwxd8+QCMeLJbodCvrYCFfHxk+fRSCYFKEqKXWlv7mRuLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NVIerZsWyChvX6CZT7e6LLlaoerUJ95H/oNTdwIWn2g=; b=h5AuQ7FfakSbEQxIWkSIEQOw+ZHELkAGngd5zsycxfdT+B9VlWZL/glu8gddLW2dRbTOhji06Dh6NzYfihGzV/QvH6BbUbcvu0Owrw3dCbgX2scuxKxG3GHXt2p40UzuvqQ40cqfU0xNcMnX3GQFMnYnlRvyMoAvryb7HuSPw5o= Received: from PH0PR18MB4086.namprd18.prod.outlook.com (2603:10b6:510:3::9) by PH0PR18MB4103.namprd18.prod.outlook.com (2603:10b6:510:3::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4587.19; Fri, 8 Oct 2021 05:38:26 +0000 Received: from PH0PR18MB4086.namprd18.prod.outlook.com ([fe80::6012:f5c4:8b1b:5289]) by PH0PR18MB4086.namprd18.prod.outlook.com ([fe80::6012:f5c4:8b1b:5289%8]) with mapi id 15.20.4587.020; Fri, 8 Oct 2021 05:38:26 +0000 From: Pavan Nikhilesh Bhagavatula To: "Carrillo, Erik G" , Jerin Jacob Kollanukkaran CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v3 11/14] eventdev: move timer adapters memory to hugepage Thread-Index: AQHXun6kwe2M72iOPUKyHv0wmQO6t6vIBJmAgACSpgA= Date: Fri, 8 Oct 2021 05:38:25 +0000 Message-ID: References: <20211003082710.8398-1-pbhagavatula@marvell.com> <20211006065012.16508-1-pbhagavatula@marvell.com> <20211006065012.16508-11-pbhagavatula@marvell.com> In-Reply-To: Accept-Language: en-IN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2d66b1fd-420e-49ae-6860-08d98a1dd9b2 x-ms-traffictypediagnostic: PH0PR18MB4103: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:8273; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ofsjs+TdPL1kmk3ResmD4GApbqOdtfix9ank75Ie+/mK2dv/moNY2i00ixzpmUEJpyf3CwUUvikymlOzqOo9V2/ckEyNShkEUKbhK40JPUCqsEhZDc+7B99ZMm1u4y69MUSRvOkgi04wUq4gdWm+RrThcb2p87MRhlA+i4goao627MkirpMbbtSxBg17Qtls5Tadu/ABE0PYCQA/QySCwFKdwcvY+sPxK1f9G6fx79lOHSIUFdlX1YUrVJ/s+vvg+qVZf264kdbZsa1pp50jJGqW733C62B6uTKDJKfjmcEc5G39Z7b7RBpHPE6/9bC7RHGn3wTMx7PW/XjjVaU0U9+Ri3OG7iqW3zJgYJyv6ptbmQEN0jKH16q0qvlgV2TXLHzcieqpDdVzN0ieHQMqO+N1ql4Ovvk1Q7e21WfZoFQchnUJxn8w5SL/6jlVoKqiEOJ4+1Uva7bion+h5UHgsSd4ueWm4R6/WZpDgoj+Pd8MgmpHiP9yFxO/YkFE0Q5vqyjk1tbf7BdzJTzEtt6ylOGIIETcYw/WV3kWgSOwhXsBrGYD40Gv5CBEkYHWNWZzo1VhCB94KV52iDsEKOWt07o5k2O9iugb0mvC93SNmUzoloFW5wxYxAvDKUw2g1BkHrQ2OipPWhqxW6ZFgzQfHGE6b6Mj2N1w+xGNRbleOomxF/s4ZhARnTr/eIEVQAgB6cHeYWL/lsgu/NoXCBAvXQ== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR18MB4086.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(316002)(66946007)(64756008)(508600001)(83380400001)(122000001)(8676002)(38100700002)(66476007)(66556008)(110136005)(4326008)(66446008)(8936002)(186003)(76116006)(6636002)(6506007)(38070700005)(5660300002)(52536014)(53546011)(71200400001)(2906002)(55016002)(86362001)(26005)(9686003)(7696005)(33656002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?lUBp0cEoAp7dsurcpUskh75bf6TTUdaYCsoA1590QiZn7WtKq3apMzPP7IW5?= =?us-ascii?Q?QkzvwMc47zNn+oHhKS+vr77+vYn2f1Q8G8m5NRcpF48I4JNLmF89JcEdHGHZ?= =?us-ascii?Q?fTZDZOGr7FZJGa2Df4yVzNnP4OrO0GoZSoR3bYw+m3vPgcdaiJKBjnLmCdYm?= =?us-ascii?Q?C9ePT8gLhbmw+PrgWeXMI77BumhxNuzxWYk/ro/mHHZhhrH8BFSKNDmK19A/?= =?us-ascii?Q?HCIPeifaee4XtKZpj3HubZZDEzp/x744GnaNM40gVwA5TfjinX983A2WRgWx?= =?us-ascii?Q?RvDrMdi3HDEJnNZlrKZS7Erb7Lt7aVZldjfvQtCGAbQS2Clv4oCUrmdLH2MA?= =?us-ascii?Q?M9St0ynn+7ByEgRrDMuAnQfglBTpDq8o8FzNI1H4w9n61fh90wgJyYrba1RK?= =?us-ascii?Q?BCHGhgB7U2l/eKLwpnYzIaehkzhriwWMx+AF5apk77cyfxE+piUSuLsti/4I?= =?us-ascii?Q?LgUFTE93kgoaP8aEwzjMak9FIGUT97iKdQcHJ7EfniPwUotJHNM+Jvv50PyP?= =?us-ascii?Q?BwJXxmZSDozarnxZuRpw53fd/kwrg2vM19CquO4oyw8xFbhDrc+ymqitUKbi?= =?us-ascii?Q?lhwAiQssO9fKCXmLKNmn+Jxua6vJf4HAr2NvtyksAWeDGm51dYhbtkzlFGwh?= =?us-ascii?Q?2nC5hyV2PrMzu/frcNuxubIwp6Eq8xtzGT70DM850jug9U0YJ05v8pnaBLKF?= =?us-ascii?Q?s4u4xqt6eQc/NOawnbOedLgmnh6AFCuziY8jFg+mJVi+9x/Y6wTXXcVnD+a8?= =?us-ascii?Q?rF6PgWhx4pAP0XdL9rr/ZvyoDzGn1MfWUMUPVyBp1Owpi+hBJBRY650MDsyr?= =?us-ascii?Q?2CmIVSRcQVHd5UVgwfG7nSDTzi/f/jT04ncWe+4/8uZ6UH70tHwPx16W7IYR?= =?us-ascii?Q?TxqIpG8Y9r6wVnuBsxTR/mZ+BibhnEG6uyA43va6cbTtIJbrHjjOvieP2Gih?= =?us-ascii?Q?m0lbvJ6AzVjRXuiN9PAoTGwE5UfPWsomoEMUlb0ulzIK2PyTufEh2I4OoZhH?= =?us-ascii?Q?u/7V0BGY18YBgCvGezqEJ5ezmAThKsZyDBg4Sq4NtG7NYq8RJ/K2bRj7Vq6U?= =?us-ascii?Q?r71gMRwEv2LcMYWs/sMG+yk1qzIpqRXsaOJHYJfi1WyaCI8ovAxRdl5+48BI?= =?us-ascii?Q?NsT19HL7FT5ktkj70KFnraKjy+F3OoQPi75lbrVNP6YnLkvtM8TfUgY9eRDf?= =?us-ascii?Q?9Mrqnkc3Nvw1AXOGuzPfQWynFUza+kkQIYfAfovOhfefGGPydcZGDJw7CWFt?= =?us-ascii?Q?Utt7fg7zgI6LE5GsLd8MnVFuMZeL3bkL2Q6W6Pc5xW8VwrRvqV2uZYDN5vO8?= =?us-ascii?Q?JdZBaASEGxMN6Qgl7FQYfORd?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: marvell.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PH0PR18MB4086.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2d66b1fd-420e-49ae-6860-08d98a1dd9b2 X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Oct 2021 05:38:25.8946 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: XtN3tSFuUCO27dRhLITJbMvbCfWcYqaAv+wcfJqTr+AK4GUmmQP0Vl3Ck3altATvoxvlgwbY8kxZYdsYAVgciR1rXHJtQs8Ukr0WB5933yU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR18MB4103 X-Proofpoint-GUID: RR_C3OPeJWpbdSkyIvTihJb0oIoVc2ha X-Proofpoint-ORIG-GUID: RR_C3OPeJWpbdSkyIvTihJb0oIoVc2ha X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.182.1,Aquarius:18.0.790,Hydra:6.0.391,FMLib:17.0.607.475 definitions=2021-10-07_05,2021-10-07_02,2020-04-07_01 Subject: Re: [dpdk-dev] [PATCH v3 11/14] eventdev: move timer adapters memory to hugepage X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Erik, >Hi Pavan, > >Some comments below: > >> -----Original Message----- >> From: pbhagavatula@marvell.com >> Sent: Wednesday, October 6, 2021 1:50 AM >> To: jerinj@marvell.com; Carrillo, Erik G >> Cc: dev@dpdk.org; Pavan Nikhilesh >> Subject: [dpdk-dev] [PATCH v3 11/14] eventdev: move timer adapters >> memory to hugepage >> >> From: Pavan Nikhilesh >> >> Move memory used by timer adapters to hugepage. >> Allocate memory on the first adapter create or lookup to address both >> primary and secondary process usecases. >> This will prevent TLB misses if any and aligns to memory structure of >other >> subsystems. >> >> Signed-off-by: Pavan Nikhilesh >> --- >> lib/eventdev/rte_event_timer_adapter.c | 24 >> +++++++++++++++++++++++- >> 1 file changed, 23 insertions(+), 1 deletion(-) >> >> diff --git a/lib/eventdev/rte_event_timer_adapter.c >> b/lib/eventdev/rte_event_timer_adapter.c >> index ae55407042..c4dc7a5fd4 100644 >> --- a/lib/eventdev/rte_event_timer_adapter.c >> +++ b/lib/eventdev/rte_event_timer_adapter.c >> @@ -33,7 +33,7 @@ RTE_LOG_REGISTER_SUFFIX(evtim_logtype, >> adapter.timer, NOTICE); >> RTE_LOG_REGISTER_SUFFIX(evtim_buffer_logtype, adapter.timer, >NOTICE); >> RTE_LOG_REGISTER_SUFFIX(evtim_svc_logtype, adapter.timer.svc, >> NOTICE); >> >> -static struct rte_event_timer_adapter >> adapters[RTE_EVENT_TIMER_ADAPTER_NUM_MAX]; >> +static struct rte_event_timer_adapter *adapters; >> >> static const struct event_timer_adapter_ops swtim_ops; >> >> @@ -138,6 +138,17 @@ rte_event_timer_adapter_create_ext( >> int n, ret; >> struct rte_eventdev *dev; >> >> + if (adapters =3D=3D NULL) { >> + adapters =3D rte_zmalloc("Eventdev", >> + sizeof(struct >rte_event_timer_adapter) * >> + >> RTE_EVENT_TIMER_ADAPTER_NUM_MAX, >> + RTE_CACHE_LINE_SIZE); >> + if (adapters =3D=3D NULL) { >> + rte_errno =3D ENOMEM; >> + return NULL; >> + } >> + } >> + >> if (conf =3D=3D NULL) { >> rte_errno =3D EINVAL; >> return NULL; >> @@ -312,6 +323,17 @@ rte_event_timer_adapter_lookup(uint16_t >> adapter_id) >> int ret; >> struct rte_eventdev *dev; >> >> + if (adapters =3D=3D NULL) { >> + adapters =3D rte_zmalloc("Eventdev", >> + sizeof(struct >rte_event_timer_adapter) * >> + >> RTE_EVENT_TIMER_ADAPTER_NUM_MAX, >> + RTE_CACHE_LINE_SIZE); >> + if (adapters =3D=3D NULL) { >> + rte_errno =3D ENOMEM; >> + return NULL; >> + } >> + } >> + >> if (adapters[adapter_id].allocated) >> return &adapters[adapter_id]; /* Adapter is already >loaded >> */ >> >> -- >> 2.17.1 > >The rte_event_timer_adapter struct has several fields that have per- >process values. > >For example, there are three fast path function pointers and each will >be assigned distinct addresses for each process in a multi-process >scenario. The "allocated" field is also per-process. With the changes >above, if a secondary process did a lookup() after a primary process did >a create(), the secondary would get a reference to an object with >function pointers that are invalid in the secondary process. > I understand, the current patch doesn't unify the memory between processes. Instead, we zmalloc the per-process 'array' that holds the adapter objects = when=20 ever the process calls either create or lookup and initialize the per-proce= ss data structure. The pointer to the adapter array is static, so when ever a process is initi= alized it will be NULL. >To fully move the adapter object table into shared hugepage memory, >those "per-process" members would need to be collected into a per- >process data structure that could be independently allocated for each >process. However, that would add one more pointer dereference to >get to the fast path functions, and avoiding that was the original reason >to put those pointers there. This is similar to the rte_eventdev struct. > >Thanks, >Erik