From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50084.outbound.protection.outlook.com [40.107.5.84]) by dpdk.org (Postfix) with ESMTP id 4E49DA3 for ; Mon, 18 Mar 2019 22:21:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CFnEOdf3tz8jhaUkkU+eReABR0jVAFKyL/miKcomyF8=; b=NCi2sLK/XwR6Qnt0pTJutLdcjF3tIRKnH/JNedTkBfYBZK5uXyhASVZNT/ejQ17cFYKYL7MWkgtgUH+ZIzbe6Lt5GVV5g1m70j/hS3E61oBdh5mf7zIjQaRm201+8fQklzhMOzHUOE0JQNScNoi4sIXAkcPrLsWbKCJFnrkTFEg= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB3995.eurprd05.prod.outlook.com (52.134.72.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1709.14; Mon, 18 Mar 2019 21:21:17 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::580c:ae7b:8278:cc50]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::580c:ae7b:8278:cc50%3]) with mapi id 15.20.1709.015; Mon, 18 Mar 2019 21:21:17 +0000 From: Yongseok Koh To: Shahaf Shuler CC: "dev@dpdk.org" Thread-Topic: [PATCH 3/4] net/mlx5: rework PMD global data init Thread-Index: AQHU1LgXU/0ejgjXUECxzghobJFwFqYLG6YAgAbb6gA= Date: Mon, 18 Mar 2019 21:21:16 +0000 Message-ID: <20190318212107.GA37866@yongseok-MBP.local> References: <20190307073314.18324-1-yskoh@mellanox.com> <20190307073314.18324-4-yskoh@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR21CA0006.namprd21.prod.outlook.com (2603:10b6:a03:114::16) To DB3PR0502MB3980.eurprd05.prod.outlook.com (2603:10a6:8:10::27) authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [209.116.155.178] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1a84bbb7-e00c-48fb-fee6-08d6abe7a7a6 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB3PR0502MB3995; x-ms-traffictypediagnostic: DB3PR0502MB3995: x-microsoft-antispam-prvs: x-forefront-prvs: 098076C36C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(136003)(346002)(376002)(396003)(39860400002)(189003)(199004)(25786009)(5660300002)(76176011)(52116002)(305945005)(478600001)(476003)(486006)(446003)(7736002)(11346002)(68736007)(8676002)(81166006)(386003)(26005)(6506007)(102836004)(86362001)(8936002)(186003)(81156014)(66066001)(105586002)(97736004)(3846002)(6116002)(6246003)(229853002)(9686003)(71200400001)(1076003)(6512007)(2906002)(6636002)(106356001)(316002)(71190400001)(98436002)(99286004)(14454004)(6436002)(6862004)(53936002)(6486002)(14444005)(256004)(33656002)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB3995; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Jm6/Kt75jW5ASIbWLkeXRdTQ6zb9KlpjMVYNdAuebwd6V0mKbQzg2R6KTJgda2GyS51T9TEvrpnY2Caur01GypAxgJyVCl23FAw6GkluP9ysGE6vHuBLSW/VEMCaCHhqtMNNip/AGlCa+yFxz+dNRahJLrVZ50ukN07ldD5ZlCVTQMOPh3upvnclcDAsE+qMSiiq609RwMjNTpQd+d1M/h3WNsbx4NTwF3/jmC5IeF+HjQ59qq/l7ZWfUw93ckfQMDXpkHYq4Fttu6YXPeGCtmF+quoHsZxUl1PUQUyPRiBU3QrsunaWnTHsV0b61Kf5bujwrNQmErp6IkdHxfnkVH3Hvufbwwit/59Kr8icOMluutJ1VIZxBroG14YnhDiAW9XocgAbX3263hheJq83DygJq+5BqNIwhbmpeBSEuoM= Content-Type: text/plain; charset="us-ascii" Content-ID: <411BB5053A6F1749B5C1B944C96F7389@eurprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1a84bbb7-e00c-48fb-fee6-08d6abe7a7a6 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Mar 2019 21:21:16.9386 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB3995 Subject: Re: [dpdk-dev] [PATCH 3/4] net/mlx5: rework PMD global data init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2019 21:21:18 -0000 On Thu, Mar 14, 2019 at 05:36:28AM -0700, Shahaf Shuler wrote: > Hi Koh, >=20 > Thursday, March 7, 2019 9:33 AM, Yongseok Koh: > > Subject: [PATCH 3/4] net/mlx5: rework PMD global data init > >=20 > > There's more need to have PMD global data structure. It should be initi= alized > > once per a process regardless of how many PMD instances are probed. > > mlx5_init_once() is called during probing and make sure all the init fu= nctions > > are called once per a process. The existing shared memory gets more > > extensively used for this purpose. As there could be multiple secondary > > processes, a static storage (local to process) is also added. >=20 > It is hard to understand from the commit log what was missing on the old = design.=20 Okay, will add more comments. > > As the reserved virtual address for UAR remap is a PMD global resource,= this > > doesn't need to be stored in the device priv structure, but in the PMD = global > > data. >=20 > I thought we agreed to drop those and have different VA for each process.= =20 > If so, is the extra work on the UAR here is needed?=20 My plan was to do that in a separate patch for performance regression. Let me know if you want it to be done in this patchset. > > Signed-off-by: Yongseok Koh > > --- > > drivers/net/mlx5/mlx5.c | 250 ++++++++++++++++++++++++++++++++-- > > ---------- > > drivers/net/mlx5/mlx5.h | 19 +++- > > drivers/net/mlx5/mlx5_mp.c | 19 +++- > > drivers/net/mlx5/mlx5_txq.c | 7 +- > > 4 files changed, 217 insertions(+), 78 deletions(-) > >=20 > > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > > 6ed2418106..ea8fd55ee6 100644 > > --- a/drivers/net/mlx5/mlx5.c > > +++ b/drivers/net/mlx5/mlx5.c > > @@ -128,16 +128,26 @@ struct mlx5_shared_data *mlx5_shared_data; > > /* Spinlock for mlx5_shared_data allocation. */ static rte_spinlock_t > > mlx5_shared_data_lock =3D RTE_SPINLOCK_INITIALIZER; > >=20 > > +/* Process local data for secondary processes. */ static struct > > +mlx5_local_data mlx5_local_data; >=20 > Why not storing this context as part of ethdev-> process_private instead = of declaring it static?=20 Because it is not per-device data but per-PMD data. Will also have to rebase my patchsets when I send out v2. Thanks, Yongseok From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id CA006A05FE for ; Mon, 18 Mar 2019 22:21:20 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id CF9802C30; Mon, 18 Mar 2019 22:21:19 +0100 (CET) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50084.outbound.protection.outlook.com [40.107.5.84]) by dpdk.org (Postfix) with ESMTP id 4E49DA3 for ; Mon, 18 Mar 2019 22:21:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CFnEOdf3tz8jhaUkkU+eReABR0jVAFKyL/miKcomyF8=; b=NCi2sLK/XwR6Qnt0pTJutLdcjF3tIRKnH/JNedTkBfYBZK5uXyhASVZNT/ejQ17cFYKYL7MWkgtgUH+ZIzbe6Lt5GVV5g1m70j/hS3E61oBdh5mf7zIjQaRm201+8fQklzhMOzHUOE0JQNScNoi4sIXAkcPrLsWbKCJFnrkTFEg= Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com (52.134.72.27) by DB3PR0502MB3995.eurprd05.prod.outlook.com (52.134.72.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1709.14; Mon, 18 Mar 2019 21:21:17 +0000 Received: from DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::580c:ae7b:8278:cc50]) by DB3PR0502MB3980.eurprd05.prod.outlook.com ([fe80::580c:ae7b:8278:cc50%3]) with mapi id 15.20.1709.015; Mon, 18 Mar 2019 21:21:17 +0000 From: Yongseok Koh To: Shahaf Shuler CC: "dev@dpdk.org" Thread-Topic: [PATCH 3/4] net/mlx5: rework PMD global data init Thread-Index: AQHU1LgXU/0ejgjXUECxzghobJFwFqYLG6YAgAbb6gA= Date: Mon, 18 Mar 2019 21:21:16 +0000 Message-ID: <20190318212107.GA37866@yongseok-MBP.local> References: <20190307073314.18324-1-yskoh@mellanox.com> <20190307073314.18324-4-yskoh@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: BYAPR21CA0006.namprd21.prod.outlook.com (2603:10b6:a03:114::16) To DB3PR0502MB3980.eurprd05.prod.outlook.com (2603:10a6:8:10::27) authentication-results: spf=none (sender IP is ) smtp.mailfrom=yskoh@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [209.116.155.178] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 1a84bbb7-e00c-48fb-fee6-08d6abe7a7a6 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600127)(711020)(4605104)(4618075)(2017052603328)(7153060)(7193020); SRVR:DB3PR0502MB3995; x-ms-traffictypediagnostic: DB3PR0502MB3995: x-microsoft-antispam-prvs: x-forefront-prvs: 098076C36C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(136003)(346002)(376002)(396003)(39860400002)(189003)(199004)(25786009)(5660300002)(76176011)(52116002)(305945005)(478600001)(476003)(486006)(446003)(7736002)(11346002)(68736007)(8676002)(81166006)(386003)(26005)(6506007)(102836004)(86362001)(8936002)(186003)(81156014)(66066001)(105586002)(97736004)(3846002)(6116002)(6246003)(229853002)(9686003)(71200400001)(1076003)(6512007)(2906002)(6636002)(106356001)(316002)(71190400001)(98436002)(99286004)(14454004)(6436002)(6862004)(53936002)(6486002)(14444005)(256004)(33656002)(4326008); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR0502MB3995; H:DB3PR0502MB3980.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Jm6/Kt75jW5ASIbWLkeXRdTQ6zb9KlpjMVYNdAuebwd6V0mKbQzg2R6KTJgda2GyS51T9TEvrpnY2Caur01GypAxgJyVCl23FAw6GkluP9ysGE6vHuBLSW/VEMCaCHhqtMNNip/AGlCa+yFxz+dNRahJLrVZ50ukN07ldD5ZlCVTQMOPh3upvnclcDAsE+qMSiiq609RwMjNTpQd+d1M/h3WNsbx4NTwF3/jmC5IeF+HjQ59qq/l7ZWfUw93ckfQMDXpkHYq4Fttu6YXPeGCtmF+quoHsZxUl1PUQUyPRiBU3QrsunaWnTHsV0b61Kf5bujwrNQmErp6IkdHxfnkVH3Hvufbwwit/59Kr8icOMluutJ1VIZxBroG14YnhDiAW9XocgAbX3263hheJq83DygJq+5BqNIwhbmpeBSEuoM= Content-Type: text/plain; charset="UTF-8" Content-ID: <411BB5053A6F1749B5C1B944C96F7389@eurprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1a84bbb7-e00c-48fb-fee6-08d6abe7a7a6 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Mar 2019 21:21:16.9386 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR0502MB3995 Subject: Re: [dpdk-dev] [PATCH 3/4] net/mlx5: rework PMD global data init X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190318212116.YHLApKypzOlSwwt3hZZahAEg6Behi5eQ54w977VwTQ8@z> On Thu, Mar 14, 2019 at 05:36:28AM -0700, Shahaf Shuler wrote: > Hi Koh, >=20 > Thursday, March 7, 2019 9:33 AM, Yongseok Koh: > > Subject: [PATCH 3/4] net/mlx5: rework PMD global data init > >=20 > > There's more need to have PMD global data structure. It should be initi= alized > > once per a process regardless of how many PMD instances are probed. > > mlx5_init_once() is called during probing and make sure all the init fu= nctions > > are called once per a process. The existing shared memory gets more > > extensively used for this purpose. As there could be multiple secondary > > processes, a static storage (local to process) is also added. >=20 > It is hard to understand from the commit log what was missing on the old = design.=20 Okay, will add more comments. > > As the reserved virtual address for UAR remap is a PMD global resource,= this > > doesn't need to be stored in the device priv structure, but in the PMD = global > > data. >=20 > I thought we agreed to drop those and have different VA for each process.= =20 > If so, is the extra work on the UAR here is needed?=20 My plan was to do that in a separate patch for performance regression. Let me know if you want it to be done in this patchset. > > Signed-off-by: Yongseok Koh > > --- > > drivers/net/mlx5/mlx5.c | 250 ++++++++++++++++++++++++++++++++-- > > ---------- > > drivers/net/mlx5/mlx5.h | 19 +++- > > drivers/net/mlx5/mlx5_mp.c | 19 +++- > > drivers/net/mlx5/mlx5_txq.c | 7 +- > > 4 files changed, 217 insertions(+), 78 deletions(-) > >=20 > > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > > 6ed2418106..ea8fd55ee6 100644 > > --- a/drivers/net/mlx5/mlx5.c > > +++ b/drivers/net/mlx5/mlx5.c > > @@ -128,16 +128,26 @@ struct mlx5_shared_data *mlx5_shared_data; > > /* Spinlock for mlx5_shared_data allocation. */ static rte_spinlock_t > > mlx5_shared_data_lock =3D RTE_SPINLOCK_INITIALIZER; > >=20 > > +/* Process local data for secondary processes. */ static struct > > +mlx5_local_data mlx5_local_data; >=20 > Why not storing this context as part of ethdev-> process_private instead = of declaring it static?=20 Because it is not per-device data but per-PMD data. Will also have to rebase my patchsets when I send out v2. Thanks, Yongseok