From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20065.outbound.protection.outlook.com [40.107.2.65]) by dpdk.org (Postfix) with ESMTP id 9A6875F44 for ; Tue, 2 Apr 2019 21:09:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GFuaLH9leIqgtHLOJ9LKRv/ZRhCbBhS4f7Bjhg3kwpk=; b=pN2If2v8990ouDIY1Wwx5itSRDxpSXbNTv7zsDE836SAsrTRn9pmUYyhOUMxS19JXNZgpJg54uU+wRfyOAzYSKlB1vxHOoDieras47EXOcZj/1Qn2eJ0NHq50K9kjfmoWNfS8aIVlW4eqbEuMZOtZcdDI9K1Gb2FGh+vxEmJggk= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB4050.eurprd05.prod.outlook.com (52.133.37.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.20; Tue, 2 Apr 2019 19:09:48 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1750.014; Tue, 2 Apr 2019 19:09:48 +0000 From: Shahaf Shuler To: Slava Ovsiienko , "dev@dpdk.org" Thread-Topic: [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures Thread-Index: AQHU6Rx/GOpAjTIZ6Ey+SXO/9HMZcaYpNQKQ Date: Tue, 2 Apr 2019 19:09:48 +0000 Message-ID: References: <1554186157-29455-1-git-send-email-viacheslavo@mellanox.com> <1554186157-29455-3-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1554186157-29455-3-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e58fb3d8-0c06-48db-22b9-08d6b79ec6b4 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM0PR0502MB4050; x-ms-traffictypediagnostic: AM0PR0502MB4050: x-microsoft-antispam-prvs: x-forefront-prvs: 0995196AA2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(396003)(39860400002)(376002)(136003)(366004)(189003)(199004)(97736004)(55016002)(71200400001)(68736007)(14454004)(9686003)(53936002)(26005)(6116002)(6246003)(71190400001)(6506007)(3846002)(99286004)(476003)(446003)(186003)(76176011)(2501003)(6436002)(7696005)(52536014)(11346002)(102836004)(486006)(305945005)(106356001)(33656002)(316002)(7736002)(110136005)(14444005)(86362001)(229853002)(74316002)(66066001)(105586002)(5660300002)(81156014)(8936002)(81166006)(25786009)(2906002)(8676002)(256004)(478600001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB4050; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: fJWLcKrOWA/+FmrN0ws0Ui46M3j0p95Waf9C8cDJSF1eZqasTT6z+sD6RqHnhl4nfyft7fBFdOABOlP4Gzfro4XtjRu5HjmLaVOSFUNrfxLS1eohfavg5Pk9UKzeWeOTzmz4yLbjqsDmfjlV4zGMxSd4z5nuvZ8XhEhcqkOvXFOGVKDyMDX28cQvKtW5KkjW5m9atfqSoVKUv9mWYLij31ryGotwlmOwO/cQ9rbxtiqAUHguuDxzBebrOPMBwiYH1caXBnxKCtdS1BZxyFlp17CLtzVsU9PUXBN9jN6pTwiNqguqbABmm+Yacf7oDe6msgkGVK2FdyLHSPGdhINm4AOripwWfUdlgrDkeQDnPSa2scf6+jNiypM0DAMoSvp+ng1++jdMEna8z6cDwBoyk+py5IFRtzFZWSj6t0kxruE= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: e58fb3d8-0c06-48db-22b9-08d6b79ec6b4 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Apr 2019 19:09:48.4762 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB4050 Subject: Re: [dpdk-dev] [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2019 19:09:50 -0000 Tuesday, April 2, 2019 9:23 AM, Viacheslav Ovsiienko: > Subject: [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures Same comment about the title.=20 >=20 > This patch introduces the reference counter for DV/DR flow engine > structure, which we are going to share between master and representors in > E-Switch configurations over multiport Infiniband device. >=20 > Signed-off-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/mlx5.c | 26 ++++++++++++++++++++++++-- > drivers/net/mlx5/mlx5.h | 4 ++++ > 2 files changed, 28 insertions(+), 2 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > 9de122d..79e0c17 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -299,7 +299,8 @@ struct mlx5_dev_spawn_data { #ifdef > HAVE_MLX5DV_DR > /** > * Initialize DV/DR related data within private structure. > - * This is preparation step for the data sharing. > + * Routine checks the reference counter and does actual > + * resources creation/iniialization only if counter is zero. > * > * @param[in] priv > * Pointer to the private device data structure. > @@ -314,6 +315,14 @@ struct mlx5_dev_spawn_data { > int err =3D 0; > void *ns; >=20 > + assert(sh); > + if (sh->dv_refcnt) { > + /* Shared DV/DR structures is already initialized. */ > + sh->dv_refcnt++; > + priv->dv_shared =3D 1; > + return 0; > + } > + /* Reference counter is zero, we should initialize structures. */ > ns =3D mlx5dv_dr_create_ns(sh->ctx, > MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS); > if (!ns) { > DRV_LOG(ERR, "ingress mlx5dv_dr_create_ns failed"); @@ - > 328,6 +337,8 @@ struct mlx5_dev_spawn_data { > goto error; > } > priv->tx_ns =3D ns; > + sh->dv_refcnt++; > + priv->dv_shared =3D 1; > return 0; >=20 > error: > @@ -352,6 +363,16 @@ struct mlx5_dev_spawn_data { static void > mlx5_free_shared_dv(struct mlx5_priv *priv) { > + struct mlx5_ibv_shared *sh; > + > + if (!priv->dv_shared) > + return; > + priv->dv_shared =3D 0; > + sh =3D priv->sh; > + assert(sh); > + assert(sh->dv_refcnt); > + if (sh->dv_refcnt && --sh->dv_refcnt) > + return; > if (priv->rx_ns) { > mlx5dv_dr_destroy_ns(priv->rx_ns); > priv->rx_ns =3D NULL; > @@ -1491,7 +1512,8 @@ struct mlx5_dev_spawn_data { > error: > if (priv) { > #ifdef HAVE_MLX5DV_DR > - mlx5_free_shared_dv(priv); > + if (priv->sh) > + mlx5_free_shared_dv(priv); > #endif > if (priv->nl_socket_route >=3D 0) > close(priv->nl_socket_route); > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > a3d5f8e..56a2c61 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -213,6 +213,9 @@ struct mlx5_ibv_shared { > char ibdev_name[IBV_SYSFS_NAME_MAX]; /* IB device name. */ > char ibdev_path[IBV_SYSFS_PATH_MAX]; /* IB device path for > secondary */ > struct ibv_device_attr_ex device_attr; /* Device properties. */ > + /* Shared DV/DR flow data section. */ > + uint32_t dv_refcnt; /* DV/DR data reference counter. */ > + /* Shared interrupt handler section. */ > pthread_mutex_t intr_mutex; /* Interrupt config mutex. */ > uint32_t intr_cnt; /* Interrupt handler reference counter. */ > struct rte_intr_handle intr_handle; /* Interrupt handler for device. > */ @@ -244,6 +247,7 @@ struct mlx5_priv { > unsigned int isolated:1; /* Whether isolated mode is enabled. */ > unsigned int representor:1; /* Device is a port representor. */ > unsigned int master:1; /* Device is a E-Switch master. */ > + unsigned int dv_shared:1; /* DV/DR data is shared. */ Why this flags is needed? Aren't we always going to share?=20 > uint16_t domain_id; /* Switch domain identifier. */ > uint16_t vport_id; /* Associated VF vport index (if any). */ > int32_t representor_id; /* Port representor identifier. */ > -- > 1.8.3.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id D646AA0679 for ; Tue, 2 Apr 2019 21:09:55 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D6686683E; Tue, 2 Apr 2019 21:09:51 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20065.outbound.protection.outlook.com [40.107.2.65]) by dpdk.org (Postfix) with ESMTP id 9A6875F44 for ; Tue, 2 Apr 2019 21:09:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GFuaLH9leIqgtHLOJ9LKRv/ZRhCbBhS4f7Bjhg3kwpk=; b=pN2If2v8990ouDIY1Wwx5itSRDxpSXbNTv7zsDE836SAsrTRn9pmUYyhOUMxS19JXNZgpJg54uU+wRfyOAzYSKlB1vxHOoDieras47EXOcZj/1Qn2eJ0NHq50K9kjfmoWNfS8aIVlW4eqbEuMZOtZcdDI9K1Gb2FGh+vxEmJggk= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB4050.eurprd05.prod.outlook.com (52.133.37.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.20; Tue, 2 Apr 2019 19:09:48 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1750.014; Tue, 2 Apr 2019 19:09:48 +0000 From: Shahaf Shuler To: Slava Ovsiienko , "dev@dpdk.org" Thread-Topic: [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures Thread-Index: AQHU6Rx/GOpAjTIZ6Ey+SXO/9HMZcaYpNQKQ Date: Tue, 2 Apr 2019 19:09:48 +0000 Message-ID: References: <1554186157-29455-1-git-send-email-viacheslavo@mellanox.com> <1554186157-29455-3-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1554186157-29455-3-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e58fb3d8-0c06-48db-22b9-08d6b79ec6b4 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM0PR0502MB4050; x-ms-traffictypediagnostic: AM0PR0502MB4050: x-microsoft-antispam-prvs: x-forefront-prvs: 0995196AA2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(396003)(39860400002)(376002)(136003)(366004)(189003)(199004)(97736004)(55016002)(71200400001)(68736007)(14454004)(9686003)(53936002)(26005)(6116002)(6246003)(71190400001)(6506007)(3846002)(99286004)(476003)(446003)(186003)(76176011)(2501003)(6436002)(7696005)(52536014)(11346002)(102836004)(486006)(305945005)(106356001)(33656002)(316002)(7736002)(110136005)(14444005)(86362001)(229853002)(74316002)(66066001)(105586002)(5660300002)(81156014)(8936002)(81166006)(25786009)(2906002)(8676002)(256004)(478600001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB4050; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: fJWLcKrOWA/+FmrN0ws0Ui46M3j0p95Waf9C8cDJSF1eZqasTT6z+sD6RqHnhl4nfyft7fBFdOABOlP4Gzfro4XtjRu5HjmLaVOSFUNrfxLS1eohfavg5Pk9UKzeWeOTzmz4yLbjqsDmfjlV4zGMxSd4z5nuvZ8XhEhcqkOvXFOGVKDyMDX28cQvKtW5KkjW5m9atfqSoVKUv9mWYLij31ryGotwlmOwO/cQ9rbxtiqAUHguuDxzBebrOPMBwiYH1caXBnxKCtdS1BZxyFlp17CLtzVsU9PUXBN9jN6pTwiNqguqbABmm+Yacf7oDe6msgkGVK2FdyLHSPGdhINm4AOripwWfUdlgrDkeQDnPSa2scf6+jNiypM0DAMoSvp+ng1++jdMEna8z6cDwBoyk+py5IFRtzFZWSj6t0kxruE= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: e58fb3d8-0c06-48db-22b9-08d6b79ec6b4 X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Apr 2019 19:09:48.4762 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB4050 Subject: Re: [dpdk-dev] [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190402190948.jBohJ0QanxZwAlqJlfC3KOdyC0hHobiM9UtzSsDdM8Y@z> Tuesday, April 2, 2019 9:23 AM, Viacheslav Ovsiienko: > Subject: [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures Same comment about the title.=20 >=20 > This patch introduces the reference counter for DV/DR flow engine > structure, which we are going to share between master and representors in > E-Switch configurations over multiport Infiniband device. >=20 > Signed-off-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/mlx5.c | 26 ++++++++++++++++++++++++-- > drivers/net/mlx5/mlx5.h | 4 ++++ > 2 files changed, 28 insertions(+), 2 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > 9de122d..79e0c17 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -299,7 +299,8 @@ struct mlx5_dev_spawn_data { #ifdef > HAVE_MLX5DV_DR > /** > * Initialize DV/DR related data within private structure. > - * This is preparation step for the data sharing. > + * Routine checks the reference counter and does actual > + * resources creation/iniialization only if counter is zero. > * > * @param[in] priv > * Pointer to the private device data structure. > @@ -314,6 +315,14 @@ struct mlx5_dev_spawn_data { > int err =3D 0; > void *ns; >=20 > + assert(sh); > + if (sh->dv_refcnt) { > + /* Shared DV/DR structures is already initialized. */ > + sh->dv_refcnt++; > + priv->dv_shared =3D 1; > + return 0; > + } > + /* Reference counter is zero, we should initialize structures. */ > ns =3D mlx5dv_dr_create_ns(sh->ctx, > MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS); > if (!ns) { > DRV_LOG(ERR, "ingress mlx5dv_dr_create_ns failed"); @@ - > 328,6 +337,8 @@ struct mlx5_dev_spawn_data { > goto error; > } > priv->tx_ns =3D ns; > + sh->dv_refcnt++; > + priv->dv_shared =3D 1; > return 0; >=20 > error: > @@ -352,6 +363,16 @@ struct mlx5_dev_spawn_data { static void > mlx5_free_shared_dv(struct mlx5_priv *priv) { > + struct mlx5_ibv_shared *sh; > + > + if (!priv->dv_shared) > + return; > + priv->dv_shared =3D 0; > + sh =3D priv->sh; > + assert(sh); > + assert(sh->dv_refcnt); > + if (sh->dv_refcnt && --sh->dv_refcnt) > + return; > if (priv->rx_ns) { > mlx5dv_dr_destroy_ns(priv->rx_ns); > priv->rx_ns =3D NULL; > @@ -1491,7 +1512,8 @@ struct mlx5_dev_spawn_data { > error: > if (priv) { > #ifdef HAVE_MLX5DV_DR > - mlx5_free_shared_dv(priv); > + if (priv->sh) > + mlx5_free_shared_dv(priv); > #endif > if (priv->nl_socket_route >=3D 0) > close(priv->nl_socket_route); > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > a3d5f8e..56a2c61 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -213,6 +213,9 @@ struct mlx5_ibv_shared { > char ibdev_name[IBV_SYSFS_NAME_MAX]; /* IB device name. */ > char ibdev_path[IBV_SYSFS_PATH_MAX]; /* IB device path for > secondary */ > struct ibv_device_attr_ex device_attr; /* Device properties. */ > + /* Shared DV/DR flow data section. */ > + uint32_t dv_refcnt; /* DV/DR data reference counter. */ > + /* Shared interrupt handler section. */ > pthread_mutex_t intr_mutex; /* Interrupt config mutex. */ > uint32_t intr_cnt; /* Interrupt handler reference counter. */ > struct rte_intr_handle intr_handle; /* Interrupt handler for device. > */ @@ -244,6 +247,7 @@ struct mlx5_priv { > unsigned int isolated:1; /* Whether isolated mode is enabled. */ > unsigned int representor:1; /* Device is a port representor. */ > unsigned int master:1; /* Device is a E-Switch master. */ > + unsigned int dv_shared:1; /* DV/DR data is shared. */ Why this flags is needed? Aren't we always going to share?=20 > uint16_t domain_id; /* Switch domain identifier. */ > uint16_t vport_id; /* Associated VF vport index (if any). */ > int32_t representor_id; /* Port representor identifier. */ > -- > 1.8.3.1