From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id BA48BA04B0; Sun, 18 Oct 2020 13:58:24 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 90C3EC9D2; Sun, 18 Oct 2020 13:58:23 +0200 (CEST) Received: from hqnvemgate25.nvidia.com (hqnvemgate25.nvidia.com [216.228.121.64]) by dpdk.org (Postfix) with ESMTP id 4634B4C9B for ; Sun, 18 Oct 2020 13:58:22 +0200 (CEST) Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Sun, 18 Oct 2020 04:57:35 -0700 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 18 Oct 2020 11:58:20 +0000 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (104.47.38.56) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Sun, 18 Oct 2020 11:58:20 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Navq3OUn2b2ec/p7uxuLMIKoOq0r4ltqFNGEju0hWDBa1lylisiuhR74ElqP71drIyjAHEzo7viEHM1Pw86vXuNcBbZnKDB/GuALilx3l963Wb++iUwdL1pzEvXXXa/vTJ8DpmsVjUYPyjsYfDSpf+01B5iCYYToBUhuQ5fO8fOxffvj+1NITD9MSiEob/MtwNxN0sDxy0tjc2o+EKxgSVD9qNFFJctOn1aLAcUjY7mX+VUjbPW4cuVcGy+SPeN7vZYj/+oqThClUEhGYm6YL8P1lBxRTXH30LETeol6ZaD4w/XBhc6KXBGD72cO0vk/81IbJ3kLqHytw03dJxiqVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ewjfu82cgeiRKqk5X/5UDVTqY1aVhqzFN0oPuIcyvZs=; b=nNR3nIFEQh+OPtVa5M31rnMi5bY0ga7lA/rk9jsO+1jf642KLOJ92tsv2UCLRGh4ms12MIuCJz2tI2YWCMX++vhjQmmyUGIc+KEFEV7wRxkRUJEv1APjLdaHg2I36ice/QBxzTj0Ms0QaT3cDIrT9f50EueW2qzbvc3kC+ZxPkEn2cGMr0o+rt+HUAf7sCwuTYoyNsUWAujRkTKdISk8nmlhBQWjBcZMsAmEKXhXE6ZPYRC1x5i4gaH5ei2NDutiish2S/va4xv8scsq7iojGq/IVkwBQE7wvsQM7mohMNg63ybtaEuNkbO6W/tZ+RSmv6x+4LKrXj8+5OVnRkttyw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from BN7PR12MB2737.namprd12.prod.outlook.com (2603:10b6:408:30::18) by BN6PR1201MB0209.namprd12.prod.outlook.com (2603:10b6:405:4d::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3477.21; Sun, 18 Oct 2020 11:58:18 +0000 Received: from BN7PR12MB2737.namprd12.prod.outlook.com ([fe80::8c7e:529e:6821:b788]) by BN7PR12MB2737.namprd12.prod.outlook.com ([fe80::8c7e:529e:6821:b788%5]) with mapi id 15.20.3477.027; Sun, 18 Oct 2020 11:58:18 +0000 From: Raslan Darawsheh To: Matan Azrad , "dev@dpdk.org" CC: Slava Ovsiienko Thread-Topic: [dpdk-dev] [PATCH 1/4] net/mlx5: fix Rx queue release Thread-Index: AQHWor3mJQWiVY0KsU2RQtoTg0XaSKmdRljQ Date: Sun, 18 Oct 2020 11:58:18 +0000 Message-ID: References: <1602743893-345348-1-git-send-email-matan@nvidia.com> In-Reply-To: <1602743893-345348-1-git-send-email-matan@nvidia.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: nvidia.com; dkim=none (message not signed) header.d=none;nvidia.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [212.29.221.74] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bb1707b9-32f1-4716-1495-08d8735d1a7c x-ms-traffictypediagnostic: BN6PR1201MB0209: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4941; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: /Te9AMCHag7KUSWPA4FmcP0hA9D+AjHfflmzX19oUeaye3erphV/CqFFE0+HIAsGwDoAYJjF8pGaoaSjLDNoS2rj0jONvlC8LBoxenOZG0AzY7s1Qx0Dfeaj/WXdxGsW6ejWM/B3VNns1XAQmgKe8yYbkZx88jlxOShCr7kjv6nR8TVoYfKzqd5c/EKmoaiBIGSwfLtjBEjx/C79wKUwC2k45ZuqTZP6otcIhP/5RmrRAb2nY8fbKJ2SKqQtakYrAmUScnO8PG6c90tBVu5wP705prDOZ64deVmv8cM/Lm3ewMC4qGYTWvt1HPVU8jTi x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BN7PR12MB2737.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(39850400004)(136003)(396003)(376002)(366004)(346002)(86362001)(66556008)(8936002)(64756008)(66446008)(76116006)(9686003)(316002)(4326008)(66946007)(66476007)(107886003)(33656002)(110136005)(83380400001)(55016002)(5660300002)(53546011)(6506007)(52536014)(71200400001)(26005)(7696005)(186003)(8676002)(478600001)(2906002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: HqNS0GznK6s4hDjmmT4PuGWV23OLQl3u187MZFOTJSKcunQkWXZTe9Q2LndQACFaPnzfPUlptKp3KeJMLeJ/pc1ExyjMDYtZ7ppKYvLrtu8aoXqiJD/qdyjk+3pJ3RdiFi3ufw+A5gtIJZ4d1F4miQOQbQSDwsbInvgKJgtoYHLL5A4J0NnnNqgcGxZj5ms5AanJNntDQJ8bbbEr5qwfZ1ZEL2i03mKKRTlczYrnNjWTZ1h2OezoQxHUFB+uJHWnzBet8CFpoRWPgkqg6a7Cit5RRCF0oL2KfkG0qqlaVGJZIPWcNL9wx8iBpV5isLGX6GKq4Jr8KEqn2XUhum+7ROu0Vd58XGjZ2AEWt9SWFAiY+eWFDWeF7/P+7529zBYbd3L/9RX0jSYUnMBvMDQ0WvzVQftRrriOehhSbwAmiIAm9S+0PKYZvz1wf1NZf7MAI+PvCqna2mOGz/OVX4LHTyED9eCTQjl+jic5/Vd+pUiRkwag6pbdWX+0+iSWgayMjl/qUNqP+DCBKGdBJ/Psgs9ky/7Og4LT38JUVWHwufcwnsP+BZWsinkRHY/YSson/NiDJHil2EqP/+0GiDDi/CAq77doo3ozRWIS+gvM65DXl6tH2aS69NUCl+NbE9exkhI3M/9dPMWG9rdxC9pEdQ== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN7PR12MB2737.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: bb1707b9-32f1-4716-1495-08d8735d1a7c X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Oct 2020 11:58:18.5237 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: WI1bGq5xYR5NLqwxPm4CcWbReP3Elvi+Clx72TVTymM7kUK0Rb3kNX5J3lMaFultr1PS0CAFIqklzLMSw4vjQw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB0209 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1603022255; bh=ewjfu82cgeiRKqk5X/5UDVTqY1aVhqzFN0oPuIcyvZs=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ms-exchange-transport-forked: x-microsoft-antispam-prvs:x-ms-oob-tlc-oobclassifiers: x-ms-exchange-senderadcheck:x-microsoft-antispam: x-microsoft-antispam-message-info:x-forefront-antispam-report: x-ms-exchange-antispam-messagedata:Content-Type: Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=CTf1hGh+Jjaas074ZtwleQeYM+YpgmbwKPxUQTzBukGWTMZd3GNclwyjVKknD9Eik mOUICTg6YmNVrgLh5oXDijiENMi4k6822YhnEMmbTdwIlMuMnxOQRMaBHKxp7Czkyq zPPW1A7k6HBdLo9hioyBwHsCY9KqJNsSnko1HQ7Z44CMyIJRJAA7LNheYPkpvARpnm cnwlwUbpg1R3eUsW5fC4Dl728exm540utBJxQtdUdAN3weg+CBdW/wu8j6vxg2h7ob EmdkCDLCk+bRgwKFGllgBoAfObYX7whQGND+vLgrNo/bYWKlFJlGw1Z1yD6fUXCeCS aJ1PmmE7O9/Ng== Subject: Re: [dpdk-dev] [PATCH 1/4] net/mlx5: fix Rx queue release X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, > -----Original Message----- > From: dev On Behalf Of Matan Azrad > Sent: Thursday, October 15, 2020 9:38 AM > To: dev@dpdk.org > Cc: Slava Ovsiienko > Subject: [dpdk-dev] [PATCH 1/4] net/mlx5: fix Rx queue release >=20 > The HW objects of the Rx queue is created/destroyed in the device > start\stop stage while the ethdev configurations for the Rx queue > starts from the rx_queue_setup stage. > The PMD should save all the last configurations it got from the ethdev > and to apply them to the device in the dev_start operation. >=20 > Wrongly, last code added to mitigate the reference counters didn't take > into account the above rule and combined the configurations and HW > objects to be created\destroyed together. >=20 > This causes to memory leak and other memory issues. >=20 > Make sure the HW object is released in stop operation when there is no > any reference to it while the configurations stay saved. >=20 > Fixes: 24e4b650badc ("net/mlx5: mitigate Rx queue reference counters") >=20 > Signed-off-by: Matan Azrad > Acked-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/mlx5_rxq.c | 23 +++++++++++++---------- > drivers/net/mlx5/mlx5_rxtx.h | 2 +- > 2 files changed, 14 insertions(+), 11 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5_rxq.c b/drivers/net/mlx5/mlx5_rxq.c > index f1d8373..e1783ba 100644 > --- a/drivers/net/mlx5/mlx5_rxq.c > +++ b/drivers/net/mlx5/mlx5_rxq.c > @@ -447,7 +447,8 @@ > return -rte_errno; > } > rxq_ctrl =3D container_of((*priv->rxqs)[idx], struct mlx5_rxq_ctrl, rxq= ); > - return (rte_atomic32_read(&rxq_ctrl->refcnt) =3D=3D 1); > + return (__atomic_load_n(&rxq_ctrl->refcnt, __ATOMIC_RELAXED) > =3D=3D 1); > + > } >=20 > /* Fetches and drops all SW-owned and error CQEs to synchronize CQ. */ > @@ -1541,7 +1542,7 @@ struct mlx5_rxq_ctrl * > tmpl->rxq.uar_lock_cq =3D &priv->sh->uar_lock_cq; > #endif > tmpl->rxq.idx =3D idx; > - rte_atomic32_inc(&tmpl->refcnt); > + __atomic_add_fetch(&tmpl->refcnt, 1, __ATOMIC_RELAXED); > LIST_INSERT_HEAD(&priv->rxqsctrl, tmpl, next); > return tmpl; > error: > @@ -1588,7 +1589,7 @@ struct mlx5_rxq_ctrl * > tmpl->rxq.mr_ctrl.cache_bh =3D (struct mlx5_mr_btree) { 0 }; > tmpl->hairpin_conf =3D *hairpin_conf; > tmpl->rxq.idx =3D idx; > - rte_atomic32_inc(&tmpl->refcnt); > + __atomic_add_fetch(&tmpl->refcnt, 1, __ATOMIC_RELAXED); > LIST_INSERT_HEAD(&priv->rxqsctrl, tmpl, next); > return tmpl; > } > @@ -1613,7 +1614,7 @@ struct mlx5_rxq_ctrl * >=20 > if (rxq_data) { > rxq_ctrl =3D container_of(rxq_data, struct mlx5_rxq_ctrl, rxq); > - rte_atomic32_inc(&rxq_ctrl->refcnt); > + __atomic_add_fetch(&rxq_ctrl->refcnt, 1, > __ATOMIC_RELAXED); > } > return rxq_ctrl; > } > @@ -1638,7 +1639,7 @@ struct mlx5_rxq_ctrl * > if (!(*priv->rxqs)[idx]) > return 0; > rxq_ctrl =3D container_of((*priv->rxqs)[idx], struct mlx5_rxq_ctrl, rxq= ); > - if (!rte_atomic32_dec_and_test(&rxq_ctrl->refcnt)) > + if (__atomic_sub_fetch(&rxq_ctrl->refcnt, 1, __ATOMIC_RELAXED) > > 1) > return 1; > if (rxq_ctrl->obj) { > priv->obj_ops.rxq_obj_release(rxq_ctrl->obj); > @@ -1646,13 +1647,15 @@ struct mlx5_rxq_ctrl * > mlx5_free(rxq_ctrl->obj); > rxq_ctrl->obj =3D NULL; > } > - if (rxq_ctrl->type =3D=3D MLX5_RXQ_TYPE_STANDARD) { > - mlx5_mr_btree_free(&rxq_ctrl->rxq.mr_ctrl.cache_bh); > + if (rxq_ctrl->type =3D=3D MLX5_RXQ_TYPE_STANDARD) > rxq_free_elts(rxq_ctrl); > + if (!__atomic_load_n(&rxq_ctrl->refcnt, __ATOMIC_RELAXED)) { > + if (rxq_ctrl->type =3D=3D MLX5_RXQ_TYPE_STANDARD) > + mlx5_mr_btree_free(&rxq_ctrl- > >rxq.mr_ctrl.cache_bh); > + LIST_REMOVE(rxq_ctrl, next); > + mlx5_free(rxq_ctrl); > + (*priv->rxqs)[idx] =3D NULL; > } > - LIST_REMOVE(rxq_ctrl, next); > - mlx5_free(rxq_ctrl); > - (*priv->rxqs)[idx] =3D NULL; > return 0; > } >=20 > diff --git a/drivers/net/mlx5/mlx5_rxtx.h b/drivers/net/mlx5/mlx5_rxtx.h > index 674296e..c3734e3 100644 > --- a/drivers/net/mlx5/mlx5_rxtx.h > +++ b/drivers/net/mlx5/mlx5_rxtx.h > @@ -165,7 +165,7 @@ enum mlx5_rxq_type { > struct mlx5_rxq_ctrl { > struct mlx5_rxq_data rxq; /* Data path structure. */ > LIST_ENTRY(mlx5_rxq_ctrl) next; /* Pointer to the next element. */ > - rte_atomic32_t refcnt; /* Reference counter. */ > + uint32_t refcnt; /* Reference counter. */ > struct mlx5_rxq_obj *obj; /* Verbs/DevX elements. */ > struct mlx5_priv *priv; /* Back pointer to private data. */ > enum mlx5_rxq_type type; /* Rxq type. */ > -- > 1.8.3.1 Series applied to next-net-mlx, Kindest regards, Raslan Darawsheh