From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40053.outbound.protection.outlook.com [40.107.4.53]) by dpdk.org (Postfix) with ESMTP id 9F8A61B39A for ; Wed, 3 Apr 2019 15:27:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CZtfFm7rxgQrHppiJ9YKfKzxHcubykuyJB+FEyxrBes=; b=iDPdBV637r5sqRycjZ/oSHBiXZ20vZ6swBDMzEbpIcw0TWDl+JCkYqj4zXBItTmmjxC/c0VicAYDadfT1ZmBwUd2chu/ASFM6JZ/ITuaWWJBDdaGaXQkOsxnGz138qdhIecfkba6lFt3hbTK9EFPkzIROpKF72wL2hUzeXAfs1g= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.188.154) by AM4PR05MB3284.eurprd05.prod.outlook.com (10.171.186.157) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.15; Wed, 3 Apr 2019 13:27:51 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::11b0:de86:8d93:8b02]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::11b0:de86:8d93:8b02%3]) with mapi id 15.20.1750.021; Wed, 3 Apr 2019 13:27:50 +0000 From: Slava Ovsiienko To: Shahaf Shuler , "dev@dpdk.org" Thread-Topic: [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures Thread-Index: AQHU6Yekst0zYrhCFUKlPNx4uHXz9qYqaOrg Date: Wed, 3 Apr 2019 13:27:50 +0000 Message-ID: References: <1554186157-29455-1-git-send-email-viacheslavo@mellanox.com> <1554186157-29455-3-git-send-email-viacheslavo@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-originating-ip: [95.67.35.250] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9dbcd850-0f4c-4ca1-b281-08d6b8382ba3 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM4PR05MB3284; x-ms-traffictypediagnostic: AM4PR05MB3284: x-microsoft-antispam-prvs: x-forefront-prvs: 0996D1900D x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(376002)(39860400002)(136003)(396003)(346002)(189003)(13464003)(199004)(106356001)(71200400001)(7696005)(68736007)(74316002)(305945005)(71190400001)(3846002)(2501003)(186003)(14444005)(486006)(11346002)(6436002)(7736002)(99286004)(8936002)(66066001)(476003)(9686003)(55016002)(2906002)(25786009)(97736004)(110136005)(81156014)(478600001)(33656002)(6506007)(14454004)(446003)(256004)(8676002)(52536014)(6116002)(53936002)(53546011)(105586002)(316002)(102836004)(6246003)(229853002)(76176011)(26005)(81166006)(86362001)(5660300002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3284; H:AM4PR05MB3265.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: gcaHXnRqdUNcfvN90V7o8dGHEBP15TC8S3PCC9AKrOBs7Kea/IzUhwzYumUv685SqSYrgMNa7NqWQUf4/PnSmCfPTkqJT82mYHzM/EIQXvK0kg3q45qg0gxR5w1I7M8EROCJdQVACcWmQdQ+pGMgR5wx785UDit0WI8Ub6e3wpPDEoekI92R0/StBZ4sJq1sCdLoramznlh+cP5x4UzWDekReu/Jt7y3JEBPIoyCBrj/pns1gILASJjmlaquPCKEHFijro7DdSNPh1K39Rg6F/qS9RlYEI/JsWLcNH8Z9vf7E8umA1LFkhP7sHFfivexKeI5EX4OxHXNKnWTwop4nxCzuN5aieE0XxE8ycB7/4aIHGqQhUl5gbOEu9DGx3EBvzQ8VnhhlSpS/2SV4IEjB0POFSILUk5NUtA49KL4KUo= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9dbcd850-0f4c-4ca1-b281-08d6b8382ba3 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Apr 2019 13:27:50.7300 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3284 Subject: Re: [dpdk-dev] [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Apr 2019 13:27:52 -0000 > -----Original Message----- > From: Shahaf Shuler > Sent: Tuesday, April 2, 2019 22:10 > To: Slava Ovsiienko ; dev@dpdk.org > Subject: RE: [PATCH 2/4] net/mlx5: add reference counter for DV/DR > structures >=20 > Tuesday, April 2, 2019 9:23 AM, Viacheslav Ovsiienko: > > Subject: [PATCH 2/4] net/mlx5: add reference counter for DV/DR > > structures >=20 > Same comment about the title. >=20 > > > > This patch introduces the reference counter for DV/DR flow engine > > structure, which we are going to share between master and representors > > in E-Switch configurations over multiport Infiniband device. > > > > Signed-off-by: Viacheslav Ovsiienko > > --- > > drivers/net/mlx5/mlx5.c | 26 ++++++++++++++++++++++++-- > > drivers/net/mlx5/mlx5.h | 4 ++++ > > 2 files changed, 28 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > > 9de122d..79e0c17 100644 > > --- a/drivers/net/mlx5/mlx5.c > > +++ b/drivers/net/mlx5/mlx5.c > > @@ -299,7 +299,8 @@ struct mlx5_dev_spawn_data { #ifdef > > HAVE_MLX5DV_DR > > /** > > * Initialize DV/DR related data within private structure. > > - * This is preparation step for the data sharing. > > + * Routine checks the reference counter and does actual > > + * resources creation/iniialization only if counter is zero. > > * > > * @param[in] priv > > * Pointer to the private device data structure. > > @@ -314,6 +315,14 @@ struct mlx5_dev_spawn_data { > > int err =3D 0; > > void *ns; > > > > + assert(sh); > > + if (sh->dv_refcnt) { > > + /* Shared DV/DR structures is already initialized. */ > > + sh->dv_refcnt++; > > + priv->dv_shared =3D 1; > > + return 0; > > + } > > + /* Reference counter is zero, we should initialize structures. */ > > ns =3D mlx5dv_dr_create_ns(sh->ctx, > > MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS); > > if (!ns) { > > DRV_LOG(ERR, "ingress mlx5dv_dr_create_ns failed"); @@ - > > 328,6 +337,8 @@ struct mlx5_dev_spawn_data { > > goto error; > > } > > priv->tx_ns =3D ns; > > + sh->dv_refcnt++; > > + priv->dv_shared =3D 1; > > return 0; > > > > error: > > @@ -352,6 +363,16 @@ struct mlx5_dev_spawn_data { static void > > mlx5_free_shared_dv(struct mlx5_priv *priv) { > > + struct mlx5_ibv_shared *sh; > > + > > + if (!priv->dv_shared) > > + return; > > + priv->dv_shared =3D 0; > > + sh =3D priv->sh; > > + assert(sh); > > + assert(sh->dv_refcnt); > > + if (sh->dv_refcnt && --sh->dv_refcnt) > > + return; > > if (priv->rx_ns) { > > mlx5dv_dr_destroy_ns(priv->rx_ns); > > priv->rx_ns =3D NULL; > > @@ -1491,7 +1512,8 @@ struct mlx5_dev_spawn_data { > > error: > > if (priv) { > > #ifdef HAVE_MLX5DV_DR > > - mlx5_free_shared_dv(priv); > > + if (priv->sh) > > + mlx5_free_shared_dv(priv); > > #endif > > if (priv->nl_socket_route >=3D 0) > > close(priv->nl_socket_route); > > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > > a3d5f8e..56a2c61 100644 > > --- a/drivers/net/mlx5/mlx5.h > > +++ b/drivers/net/mlx5/mlx5.h > > @@ -213,6 +213,9 @@ struct mlx5_ibv_shared { > > char ibdev_name[IBV_SYSFS_NAME_MAX]; /* IB device name. */ > > char ibdev_path[IBV_SYSFS_PATH_MAX]; /* IB device path for > secondary > > */ > > struct ibv_device_attr_ex device_attr; /* Device properties. */ > > + /* Shared DV/DR flow data section. */ > > + uint32_t dv_refcnt; /* DV/DR data reference counter. */ > > + /* Shared interrupt handler section. */ > > pthread_mutex_t intr_mutex; /* Interrupt config mutex. */ > > uint32_t intr_cnt; /* Interrupt handler reference counter. */ > > struct rte_intr_handle intr_handle; /* Interrupt handler for device. > > */ @@ -244,6 +247,7 @@ struct mlx5_priv { > > unsigned int isolated:1; /* Whether isolated mode is enabled. */ > > unsigned int representor:1; /* Device is a port representor. */ > > unsigned int master:1; /* Device is a E-Switch master. */ > > + unsigned int dv_shared:1; /* DV/DR data is shared. */ >=20 > Why this flags is needed? Aren't we always going to share? I think we can get rid of this flag, because it is used for correct cleanup in device spawning routines If something goes wrong (on error exit) >=20 >=20 > > uint16_t domain_id; /* Switch domain identifier. */ > > uint16_t vport_id; /* Associated VF vport index (if any). */ > > int32_t representor_id; /* Port representor identifier. */ > > -- > > 1.8.3.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 35178A0679 for ; Wed, 3 Apr 2019 15:27:55 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 376381B3B9; Wed, 3 Apr 2019 15:27:54 +0200 (CEST) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-eopbgr40053.outbound.protection.outlook.com [40.107.4.53]) by dpdk.org (Postfix) with ESMTP id 9F8A61B39A for ; Wed, 3 Apr 2019 15:27:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CZtfFm7rxgQrHppiJ9YKfKzxHcubykuyJB+FEyxrBes=; b=iDPdBV637r5sqRycjZ/oSHBiXZ20vZ6swBDMzEbpIcw0TWDl+JCkYqj4zXBItTmmjxC/c0VicAYDadfT1ZmBwUd2chu/ASFM6JZ/ITuaWWJBDdaGaXQkOsxnGz138qdhIecfkba6lFt3hbTK9EFPkzIROpKF72wL2hUzeXAfs1g= Received: from AM4PR05MB3265.eurprd05.prod.outlook.com (10.171.188.154) by AM4PR05MB3284.eurprd05.prod.outlook.com (10.171.186.157) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.15; Wed, 3 Apr 2019 13:27:51 +0000 Received: from AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::11b0:de86:8d93:8b02]) by AM4PR05MB3265.eurprd05.prod.outlook.com ([fe80::11b0:de86:8d93:8b02%3]) with mapi id 15.20.1750.021; Wed, 3 Apr 2019 13:27:50 +0000 From: Slava Ovsiienko To: Shahaf Shuler , "dev@dpdk.org" Thread-Topic: [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures Thread-Index: AQHU6Yekst0zYrhCFUKlPNx4uHXz9qYqaOrg Date: Wed, 3 Apr 2019 13:27:50 +0000 Message-ID: References: <1554186157-29455-1-git-send-email-viacheslavo@mellanox.com> <1554186157-29455-3-git-send-email-viacheslavo@mellanox.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=viacheslavo@mellanox.com; x-originating-ip: [95.67.35.250] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 9dbcd850-0f4c-4ca1-b281-08d6b8382ba3 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM4PR05MB3284; x-ms-traffictypediagnostic: AM4PR05MB3284: x-microsoft-antispam-prvs: x-forefront-prvs: 0996D1900D x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(376002)(39860400002)(136003)(396003)(346002)(189003)(13464003)(199004)(106356001)(71200400001)(7696005)(68736007)(74316002)(305945005)(71190400001)(3846002)(2501003)(186003)(14444005)(486006)(11346002)(6436002)(7736002)(99286004)(8936002)(66066001)(476003)(9686003)(55016002)(2906002)(25786009)(97736004)(110136005)(81156014)(478600001)(33656002)(6506007)(14454004)(446003)(256004)(8676002)(52536014)(6116002)(53936002)(53546011)(105586002)(316002)(102836004)(6246003)(229853002)(76176011)(26005)(81166006)(86362001)(5660300002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM4PR05MB3284; H:AM4PR05MB3265.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: gcaHXnRqdUNcfvN90V7o8dGHEBP15TC8S3PCC9AKrOBs7Kea/IzUhwzYumUv685SqSYrgMNa7NqWQUf4/PnSmCfPTkqJT82mYHzM/EIQXvK0kg3q45qg0gxR5w1I7M8EROCJdQVACcWmQdQ+pGMgR5wx785UDit0WI8Ub6e3wpPDEoekI92R0/StBZ4sJq1sCdLoramznlh+cP5x4UzWDekReu/Jt7y3JEBPIoyCBrj/pns1gILASJjmlaquPCKEHFijro7DdSNPh1K39Rg6F/qS9RlYEI/JsWLcNH8Z9vf7E8umA1LFkhP7sHFfivexKeI5EX4OxHXNKnWTwop4nxCzuN5aieE0XxE8ycB7/4aIHGqQhUl5gbOEu9DGx3EBvzQ8VnhhlSpS/2SV4IEjB0POFSILUk5NUtA49KL4KUo= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9dbcd850-0f4c-4ca1-b281-08d6b8382ba3 X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Apr 2019 13:27:50.7300 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR05MB3284 Subject: Re: [dpdk-dev] [PATCH 2/4] net/mlx5: add reference counter for DV/DR structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190403132750.x5uPNecDstnR2niOqhoIm1iIbx_AfZh8I8TeADIk174@z> > -----Original Message----- > From: Shahaf Shuler > Sent: Tuesday, April 2, 2019 22:10 > To: Slava Ovsiienko ; dev@dpdk.org > Subject: RE: [PATCH 2/4] net/mlx5: add reference counter for DV/DR > structures >=20 > Tuesday, April 2, 2019 9:23 AM, Viacheslav Ovsiienko: > > Subject: [PATCH 2/4] net/mlx5: add reference counter for DV/DR > > structures >=20 > Same comment about the title. >=20 > > > > This patch introduces the reference counter for DV/DR flow engine > > structure, which we are going to share between master and representors > > in E-Switch configurations over multiport Infiniband device. > > > > Signed-off-by: Viacheslav Ovsiienko > > --- > > drivers/net/mlx5/mlx5.c | 26 ++++++++++++++++++++++++-- > > drivers/net/mlx5/mlx5.h | 4 ++++ > > 2 files changed, 28 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > > 9de122d..79e0c17 100644 > > --- a/drivers/net/mlx5/mlx5.c > > +++ b/drivers/net/mlx5/mlx5.c > > @@ -299,7 +299,8 @@ struct mlx5_dev_spawn_data { #ifdef > > HAVE_MLX5DV_DR > > /** > > * Initialize DV/DR related data within private structure. > > - * This is preparation step for the data sharing. > > + * Routine checks the reference counter and does actual > > + * resources creation/iniialization only if counter is zero. > > * > > * @param[in] priv > > * Pointer to the private device data structure. > > @@ -314,6 +315,14 @@ struct mlx5_dev_spawn_data { > > int err =3D 0; > > void *ns; > > > > + assert(sh); > > + if (sh->dv_refcnt) { > > + /* Shared DV/DR structures is already initialized. */ > > + sh->dv_refcnt++; > > + priv->dv_shared =3D 1; > > + return 0; > > + } > > + /* Reference counter is zero, we should initialize structures. */ > > ns =3D mlx5dv_dr_create_ns(sh->ctx, > > MLX5DV_DR_NS_DOMAIN_INGRESS_BYPASS); > > if (!ns) { > > DRV_LOG(ERR, "ingress mlx5dv_dr_create_ns failed"); @@ - > > 328,6 +337,8 @@ struct mlx5_dev_spawn_data { > > goto error; > > } > > priv->tx_ns =3D ns; > > + sh->dv_refcnt++; > > + priv->dv_shared =3D 1; > > return 0; > > > > error: > > @@ -352,6 +363,16 @@ struct mlx5_dev_spawn_data { static void > > mlx5_free_shared_dv(struct mlx5_priv *priv) { > > + struct mlx5_ibv_shared *sh; > > + > > + if (!priv->dv_shared) > > + return; > > + priv->dv_shared =3D 0; > > + sh =3D priv->sh; > > + assert(sh); > > + assert(sh->dv_refcnt); > > + if (sh->dv_refcnt && --sh->dv_refcnt) > > + return; > > if (priv->rx_ns) { > > mlx5dv_dr_destroy_ns(priv->rx_ns); > > priv->rx_ns =3D NULL; > > @@ -1491,7 +1512,8 @@ struct mlx5_dev_spawn_data { > > error: > > if (priv) { > > #ifdef HAVE_MLX5DV_DR > > - mlx5_free_shared_dv(priv); > > + if (priv->sh) > > + mlx5_free_shared_dv(priv); > > #endif > > if (priv->nl_socket_route >=3D 0) > > close(priv->nl_socket_route); > > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > > a3d5f8e..56a2c61 100644 > > --- a/drivers/net/mlx5/mlx5.h > > +++ b/drivers/net/mlx5/mlx5.h > > @@ -213,6 +213,9 @@ struct mlx5_ibv_shared { > > char ibdev_name[IBV_SYSFS_NAME_MAX]; /* IB device name. */ > > char ibdev_path[IBV_SYSFS_PATH_MAX]; /* IB device path for > secondary > > */ > > struct ibv_device_attr_ex device_attr; /* Device properties. */ > > + /* Shared DV/DR flow data section. */ > > + uint32_t dv_refcnt; /* DV/DR data reference counter. */ > > + /* Shared interrupt handler section. */ > > pthread_mutex_t intr_mutex; /* Interrupt config mutex. */ > > uint32_t intr_cnt; /* Interrupt handler reference counter. */ > > struct rte_intr_handle intr_handle; /* Interrupt handler for device. > > */ @@ -244,6 +247,7 @@ struct mlx5_priv { > > unsigned int isolated:1; /* Whether isolated mode is enabled. */ > > unsigned int representor:1; /* Device is a port representor. */ > > unsigned int master:1; /* Device is a E-Switch master. */ > > + unsigned int dv_shared:1; /* DV/DR data is shared. */ >=20 > Why this flags is needed? Aren't we always going to share? I think we can get rid of this flag, because it is used for correct cleanup in device spawning routines If something goes wrong (on error exit) >=20 >=20 > > uint16_t domain_id; /* Switch domain identifier. */ > > uint16_t vport_id; /* Associated VF vport index (if any). */ > > int32_t representor_id; /* Port representor identifier. */ > > -- > > 1.8.3.1