From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50049.outbound.protection.outlook.com [40.107.5.49]) by dpdk.org (Postfix) with ESMTP id 59DDA1B19 for ; Tue, 19 Mar 2019 07:54:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OPWWE9ISdR2QDfKOD6KWQEVrH+zPH5Y9XHTapWa9b6A=; b=mjhCtB+5YR3InwNgnA/QDUkB3NLFuPyX7fLHK48q/9pBN9EqbkYfIt+t7Nb0dFicmURzmMwD8Sn72yuA7NKa86ZziQokXPG9EvSIfodqHxhjUq6yf3L9UEXakZDn8x8xBbde63wiJpIoFhl1eXvHT1wMWKiw03Pwin47tiwnwBA= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB3604.eurprd05.prod.outlook.com (52.133.46.29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1709.13; Tue, 19 Mar 2019 06:54:13 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1709.015; Tue, 19 Mar 2019 06:54:13 +0000 From: Shahaf Shuler To: Yongseok Koh CC: "dev@dpdk.org" Thread-Topic: [PATCH 3/4] net/mlx5: rework PMD global data init Thread-Index: AQHU1LgX7zYIOT32k0ySUzf2696uVKYK78KQgAcH2ACAAJ/c0A== Date: Tue, 19 Mar 2019 06:54:13 +0000 Message-ID: References: <20190307073314.18324-1-yskoh@mellanox.com> <20190307073314.18324-4-yskoh@mellanox.com> <20190318212107.GA37866@yongseok-MBP.local> In-Reply-To: <20190318212107.GA37866@yongseok-MBP.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5a0c32f4-b4a7-4e53-3b03-08d6ac37b230 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB3604; x-ms-traffictypediagnostic: AM0PR0502MB3604: x-microsoft-antispam-prvs: x-forefront-prvs: 0981815F2F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(39860400002)(346002)(136003)(376002)(396003)(199004)(189003)(14454004)(6116002)(256004)(55016002)(3846002)(4326008)(9686003)(5660300002)(102836004)(93886005)(71190400001)(53936002)(25786009)(99286004)(2906002)(316002)(26005)(14444005)(6436002)(76176011)(71200400001)(33656002)(6506007)(105586002)(97736004)(7696005)(74316002)(305945005)(7736002)(66066001)(186003)(8676002)(52536014)(229853002)(6636002)(486006)(86362001)(478600001)(106356001)(476003)(446003)(6246003)(6862004)(8936002)(81156014)(81166006)(68736007)(11346002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3604; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: xKVnlsypTx2nU2QWeAYdXYIyH6fSjzkIPkkOB9m1d/Q+InTz2CCskR+H0l5xIciPacQ2LVrW5/32mVukn1JbESX+z0jF4KVqavdL3DOEUR+go3RM73wqTPPtMDXafWlTLWcN7UaXee0g6EqLKGsYxQO+/4CANfHkQrf8bPO97W4ifkpPrYonkwU1mBWDpIMRVu0eCsJ+yFlfxHzaw9+yx8MGMbXUBUNgiI5M4CxjHFx3C9eXdLh+Gyho1ms/+hM1104/OwWvRbV87PyEMrky7jcN/8/tz0PN7nFn+ewosFXECmBgr42uvMG5qg00IFScIlaMLeeg6d5O13jngwxl9RfaCmbn7eXvWBjWS0bfak3WqmqF7oazP0N32a2X4nhkmrGpDs/dZ7d20pIjkA9zeyY8NPiVNJnt4k1Ffte/hm8= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5a0c32f4-b4a7-4e53-3b03-08d6ac37b230 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Mar 2019 06:54:13.1167 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3604 Subject: Re: [dpdk-dev] [PATCH 3/4] net/mlx5: rework PMD global data init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Mar 2019 06:54:14 -0000 Monday, March 18, 2019 11:21 PM, Yongseok Koh: > Subject: Re: [PATCH 3/4] net/mlx5: rework PMD global data init >=20 > On Thu, Mar 14, 2019 at 05:36:28AM -0700, Shahaf Shuler wrote: > > Hi Koh, > > > > Thursday, March 7, 2019 9:33 AM, Yongseok Koh: > > > Subject: [PATCH 3/4] net/mlx5: rework PMD global data init > > > > > > There's more need to have PMD global data structure. It should be > > > initialized once per a process regardless of how many PMD instances a= re > probed. > > > mlx5_init_once() is called during probing and make sure all the init > > > functions are called once per a process. The existing shared memory > > > gets more extensively used for this purpose. As there could be > > > multiple secondary processes, a static storage (local to process) is = also > added. > > > > It is hard to understand from the commit log what was missing on the ol= d > design. >=20 > Okay, will add more comments. >=20 > > > As the reserved virtual address for UAR remap is a PMD global > > > resource, this doesn't need to be stored in the device priv > > > structure, but in the PMD global data. > > > > I thought we agreed to drop those and have different VA for each proces= s. > > If so, is the extra work on the UAR here is needed? >=20 > My plan was to do that in a separate patch for performance regression. > Let me know if you want it to be done in this patchset. I prefer to see how the UAR will look in the final design. If you can inclu= de such patch it wil be great. Otherwise lets keep it as is. >=20 > > > Signed-off-by: Yongseok Koh > > > --- > > > drivers/net/mlx5/mlx5.c | 250 > ++++++++++++++++++++++++++++++++-- > > > ---------- > > > drivers/net/mlx5/mlx5.h | 19 +++- > > > drivers/net/mlx5/mlx5_mp.c | 19 +++- > > > drivers/net/mlx5/mlx5_txq.c | 7 +- > > > 4 files changed, 217 insertions(+), 78 deletions(-) > > > > > > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > > > 6ed2418106..ea8fd55ee6 100644 > > > --- a/drivers/net/mlx5/mlx5.c > > > +++ b/drivers/net/mlx5/mlx5.c > > > @@ -128,16 +128,26 @@ struct mlx5_shared_data *mlx5_shared_data; > > > /* Spinlock for mlx5_shared_data allocation. */ static > > > rte_spinlock_t mlx5_shared_data_lock =3D RTE_SPINLOCK_INITIALIZER; > > > > > > +/* Process local data for secondary processes. */ static struct > > > +mlx5_local_data mlx5_local_data; > > > > Why not storing this context as part of ethdev-> process_private instea= d of > declaring it static? >=20 > Because it is not per-device data but per-PMD data. >=20 > Will also have to rebase my patchsets when I send out v2. >=20 >=20 > Thanks, > Yongseok From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 806C1A05FE for ; Tue, 19 Mar 2019 07:54:16 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id AC6C025A1; Tue, 19 Mar 2019 07:54:15 +0100 (CET) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50049.outbound.protection.outlook.com [40.107.5.49]) by dpdk.org (Postfix) with ESMTP id 59DDA1B19 for ; Tue, 19 Mar 2019 07:54:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OPWWE9ISdR2QDfKOD6KWQEVrH+zPH5Y9XHTapWa9b6A=; b=mjhCtB+5YR3InwNgnA/QDUkB3NLFuPyX7fLHK48q/9pBN9EqbkYfIt+t7Nb0dFicmURzmMwD8Sn72yuA7NKa86ZziQokXPG9EvSIfodqHxhjUq6yf3L9UEXakZDn8x8xBbde63wiJpIoFhl1eXvHT1wMWKiw03Pwin47tiwnwBA= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB3604.eurprd05.prod.outlook.com (52.133.46.29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1709.13; Tue, 19 Mar 2019 06:54:13 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1709.015; Tue, 19 Mar 2019 06:54:13 +0000 From: Shahaf Shuler To: Yongseok Koh CC: "dev@dpdk.org" Thread-Topic: [PATCH 3/4] net/mlx5: rework PMD global data init Thread-Index: AQHU1LgX7zYIOT32k0ySUzf2696uVKYK78KQgAcH2ACAAJ/c0A== Date: Tue, 19 Mar 2019 06:54:13 +0000 Message-ID: References: <20190307073314.18324-1-yskoh@mellanox.com> <20190307073314.18324-4-yskoh@mellanox.com> <20190318212107.GA37866@yongseok-MBP.local> In-Reply-To: <20190318212107.GA37866@yongseok-MBP.local> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5a0c32f4-b4a7-4e53-3b03-08d6ac37b230 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:AM0PR0502MB3604; x-ms-traffictypediagnostic: AM0PR0502MB3604: x-microsoft-antispam-prvs: x-forefront-prvs: 0981815F2F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(39860400002)(346002)(136003)(376002)(396003)(199004)(189003)(14454004)(6116002)(256004)(55016002)(3846002)(4326008)(9686003)(5660300002)(102836004)(93886005)(71190400001)(53936002)(25786009)(99286004)(2906002)(316002)(26005)(14444005)(6436002)(76176011)(71200400001)(33656002)(6506007)(105586002)(97736004)(7696005)(74316002)(305945005)(7736002)(66066001)(186003)(8676002)(52536014)(229853002)(6636002)(486006)(86362001)(478600001)(106356001)(476003)(446003)(6246003)(6862004)(8936002)(81156014)(81166006)(68736007)(11346002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB3604; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: xKVnlsypTx2nU2QWeAYdXYIyH6fSjzkIPkkOB9m1d/Q+InTz2CCskR+H0l5xIciPacQ2LVrW5/32mVukn1JbESX+z0jF4KVqavdL3DOEUR+go3RM73wqTPPtMDXafWlTLWcN7UaXee0g6EqLKGsYxQO+/4CANfHkQrf8bPO97W4ifkpPrYonkwU1mBWDpIMRVu0eCsJ+yFlfxHzaw9+yx8MGMbXUBUNgiI5M4CxjHFx3C9eXdLh+Gyho1ms/+hM1104/OwWvRbV87PyEMrky7jcN/8/tz0PN7nFn+ewosFXECmBgr42uvMG5qg00IFScIlaMLeeg6d5O13jngwxl9RfaCmbn7eXvWBjWS0bfak3WqmqF7oazP0N32a2X4nhkmrGpDs/dZ7d20pIjkA9zeyY8NPiVNJnt4k1Ffte/hm8= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5a0c32f4-b4a7-4e53-3b03-08d6ac37b230 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Mar 2019 06:54:13.1167 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB3604 Subject: Re: [dpdk-dev] [PATCH 3/4] net/mlx5: rework PMD global data init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190319065413.sWQm0pQlDxdY6vxbyRaIZ9LIXVsEjXvNOSGjnQ7oSRM@z> Monday, March 18, 2019 11:21 PM, Yongseok Koh: > Subject: Re: [PATCH 3/4] net/mlx5: rework PMD global data init >=20 > On Thu, Mar 14, 2019 at 05:36:28AM -0700, Shahaf Shuler wrote: > > Hi Koh, > > > > Thursday, March 7, 2019 9:33 AM, Yongseok Koh: > > > Subject: [PATCH 3/4] net/mlx5: rework PMD global data init > > > > > > There's more need to have PMD global data structure. It should be > > > initialized once per a process regardless of how many PMD instances a= re > probed. > > > mlx5_init_once() is called during probing and make sure all the init > > > functions are called once per a process. The existing shared memory > > > gets more extensively used for this purpose. As there could be > > > multiple secondary processes, a static storage (local to process) is = also > added. > > > > It is hard to understand from the commit log what was missing on the ol= d > design. >=20 > Okay, will add more comments. >=20 > > > As the reserved virtual address for UAR remap is a PMD global > > > resource, this doesn't need to be stored in the device priv > > > structure, but in the PMD global data. > > > > I thought we agreed to drop those and have different VA for each proces= s. > > If so, is the extra work on the UAR here is needed? >=20 > My plan was to do that in a separate patch for performance regression. > Let me know if you want it to be done in this patchset. I prefer to see how the UAR will look in the final design. If you can inclu= de such patch it wil be great. Otherwise lets keep it as is. >=20 > > > Signed-off-by: Yongseok Koh > > > --- > > > drivers/net/mlx5/mlx5.c | 250 > ++++++++++++++++++++++++++++++++-- > > > ---------- > > > drivers/net/mlx5/mlx5.h | 19 +++- > > > drivers/net/mlx5/mlx5_mp.c | 19 +++- > > > drivers/net/mlx5/mlx5_txq.c | 7 +- > > > 4 files changed, 217 insertions(+), 78 deletions(-) > > > > > > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > > > 6ed2418106..ea8fd55ee6 100644 > > > --- a/drivers/net/mlx5/mlx5.c > > > +++ b/drivers/net/mlx5/mlx5.c > > > @@ -128,16 +128,26 @@ struct mlx5_shared_data *mlx5_shared_data; > > > /* Spinlock for mlx5_shared_data allocation. */ static > > > rte_spinlock_t mlx5_shared_data_lock =3D RTE_SPINLOCK_INITIALIZER; > > > > > > +/* Process local data for secondary processes. */ static struct > > > +mlx5_local_data mlx5_local_data; > > > > Why not storing this context as part of ethdev-> process_private instea= d of > declaring it static? >=20 > Because it is not per-device data but per-PMD data. >=20 > Will also have to rebase my patchsets when I send out v2. >=20 >=20 > Thanks, > Yongseok