From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6E2DCA04DD; Thu, 19 Nov 2020 02:01:42 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D4128C87E; Thu, 19 Nov 2020 02:01:39 +0100 (CET) Received: from nat-hk.nvidia.com (nat-hk.nvidia.com [203.18.50.4]) by dpdk.org (Postfix) with ESMTP id 7F7EF5AA7 for ; Thu, 19 Nov 2020 02:01:37 +0100 (CET) Received: from HKMAIL102.nvidia.com (Not Verified[10.18.92.9]) by nat-hk.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Thu, 19 Nov 2020 09:01:34 +0800 Received: from HKMAIL101.nvidia.com (10.18.16.10) by HKMAIL102.nvidia.com (10.18.16.11) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Thu, 19 Nov 2020 01:01:34 +0000 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (104.47.58.176) by HKMAIL101.nvidia.com (10.18.16.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Thu, 19 Nov 2020 01:01:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RaDoHk0svyaoHzXz4ZM+8ZLzuvjnt9X93faHCPumblmO1Pl+mG/0q8uYnBijZghlsel80hzbDrWThCFhVPrSkXX0ae87a7R7KKvRjFLJN0tNR1NgyFBwpma/gwWrllMtONAY/iblmCpS2ixrbujHle035kKSbbXok4kc8wbkZDUXxk6IAj+CnDzLsHNJ+M6o1rJve+oZmswkVzamXXbbUXvPITyRHcDIZF8j+fkmMZtBEZ2l6NKudFMHBNT2/kXz7PPMiyHtRaXcU2E4UPBHeMK9cJclLyO2oXIP4cCMeRXumWmIFjAZLGRYk0hNKDp30CJ46G7HCrj+epzlHStXCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IGGuLVlWsjyq1gCIvJIyU++m9/+5dPTiIUg/lzT03Lc=; b=SLJDSR7IpOO0VVugt2zZ5uUcWy7uYRqZ6Wj36QsOmnzQEmGccObUKqmDZke2XxTezq3mK4KbmJA9Lj1VoHITFcK+qpyq2xrx9mWtgO5WGPSjRX0IIaECj73afbYFPbXppuDIi6G7rDrfnaDU+beppQZ63aGqVfhwYReRVVOtUKd0uSfjI2QtP6mG7tDZLhSZBJ7X/Wxqcy0k/e6NSjbUC+kwcknCd3MofaVHVbFO2Yp8kJguQVD5wd09UVEI65GbOubkv/O8QX+fIiOP2KtkwcPxPy7tPuzu7bUapLWH7OJZfIfym8viDwMabBnkK4YV3j0dPeXsDoSBU+KgKUcsOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MWHPR1201MB0189.namprd12.prod.outlook.com (2603:10b6:301:59::19) by MWHPR1201MB2477.namprd12.prod.outlook.com (2603:10b6:300:e6::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3564.25; Thu, 19 Nov 2020 01:01:30 +0000 Received: from MWHPR1201MB0189.namprd12.prod.outlook.com ([fe80::406c:cecb:1e3a:80e4]) by MWHPR1201MB0189.namprd12.prod.outlook.com ([fe80::406c:cecb:1e3a:80e4%11]) with mapi id 15.20.3589.020; Thu, 19 Nov 2020 01:01:30 +0000 From: Suanming Mou To: Weifeng Li , Matan Azrad CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH v2] net/mlx5: make flow operation thread safe Thread-Index: AQHWtaTN9UCrnnR4n0GgJ39/Felrh6nOs3rw Date: Thu, 19 Nov 2020 01:01:30 +0000 Message-ID: References: <20201108073714.1448-1-liweifeng96@126.com> <20201108075330.1502-1-liweifeng96@126.com> In-Reply-To: <20201108075330.1502-1-liweifeng96@126.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: 126.com; dkim=none (message not signed) header.d=none;126.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [36.24.141.208] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 175de673-46f4-4b23-1409-08d88c26a6c3 x-ms-traffictypediagnostic: MWHPR1201MB2477: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4125; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: QGInEYPKLCx8a43iDt5ln+FtBRUzRvnrnrIGljsF9Vwopg1xSDbf54TxcLOsVzcrZQ5ZCCsZPTCwx+0OAg8Ct+xHTy6RehPfgIbHzoa879pXS26ePNE8wKxt4q+NNAdr6HNwKJl7NbxHLfGL3Z15RtRrwPW/hJwh+ACPd77XQsKxZTtZx9bxAOzUcSdjO0XdHZAyTvt4/qTJEJ4nVj4jpS/E30VQ+ZcjWUPJcrssNjfk7UVjFMZE879LOl4Uehnp2nC5KH5A3F6KEk+miVkc08xEDnESCU93RSZEVnuDgqDnNPUHJcixwY6QoLkHSxAMsZn6+q91stoUlalr1IMf6w== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR1201MB0189.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(39860400002)(136003)(376002)(346002)(366004)(316002)(55016002)(53546011)(9686003)(6506007)(6636002)(26005)(186003)(8676002)(5660300002)(71200400001)(64756008)(66556008)(66476007)(52536014)(86362001)(8936002)(2906002)(66446008)(66946007)(83380400001)(76116006)(110136005)(7696005)(33656002)(478600001)(4326008); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: JzCfdQHCNvSldW5kaws7TapuP65Vo64xTvfhYAess7APP0nytO8jsxPkNcoZeWCTntMNg7rKf1s+k/XQ3YwPvUJNGqvAwu6NzANW3ECRR6s9ifiW56HWoJ1tf2ChnHZqvX1S6h8LiF4y+tYIh/33p2mZ80nyGOCZY7Mj85kdhBlfVroMFpWFz5dgqbXyyl4PUFn7yfl5NQbAD4zhJovv2YIrRaT+7NI2DrD6PIOKQ8pBNQXtZqYLFbSc7lsr+YhdcnSTABbP2lHGTFpzQ5RQfIkJN4sQQBNSN4Ce4TO57xTssxb/HvhJj8qPAPxfOIVLP76g1IK9MvhVgtAgaJ5rh8QUE0GN3NN8SUVtMivZTqNxz7XpwKJbHS4CuPISHiYIujCqAIyyz1JBXnZEtnr5IM9yIFoOLqvfcUBYPnJzO1n2OgFriAlscd9Vl43ma2tJ2xJV2A0Nc3LQ6N8QuwKPovGvymY2TVZjQ4Tr0vaAEXI3WHplFqTHHv+hgJAoT0WeiKMndFXImehgfKLfO6+CdGqyPh2YiqqdTGVt4PzOg3kYf+mxG910SqbvTgqG9bdN6BnE07N8muYWEK/ARSOmxq5qyzqobdwjF4rbfcLN1fCxQjlobHTf0CJhJSS0vNlQUern2nha9laSHFt2XeI7+ZxGM4uTW5erZUjW7khBaUkFB95WTtcVOTRDT6eVpuvzaj9u+TD23my8gprS8+7590piC1GJkVZ4/Ca63f4p6xiWELqQqYcS40wEeZsir5+YCeBAsx4mqJKNDAEF7JtgoO1W2V7LqOjWzweWo2UBREAWjBia//8Jeo7T9a3jJUEZ0VrdGNrcNgG8TH9JcSh18bGsgX28I+7RzOpkNyRn1ekOHWRmA9DCt0lq7elUiJe7lm4oSPW++NkkdehKxOcBVg== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR1201MB0189.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 175de673-46f4-4b23-1409-08d88c26a6c3 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Nov 2020 01:01:30.6633 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: /PTFiBikss6Ia8MAaMc8qGjBWmjUGVVqnd5iNypRrb28QWLIw3dRSowP/Y3lECakj9KjhQCGSADZMtW7reOouw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB2477 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1605747694; bh=IGGuLVlWsjyq1gCIvJIyU++m9/+5dPTiIUg/lzT03Lc=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ms-exchange-transport-forked: x-microsoft-antispam-prvs:x-ms-oob-tlc-oobclassifiers: x-ms-exchange-senderadcheck:x-microsoft-antispam: x-microsoft-antispam-message-info:x-forefront-antispam-report: x-ms-exchange-antispam-messagedata:Content-Type: Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=pABOxKNd1mFyf1HbceOzfzvbhYQnOzc+cpVnBbjRpWChLW3S6VmbBRkcxF0jje7y6 teMD56YUc5Iof7u38MtPJrfT3skAtyHMg2vAH+YGwQZGZkc3I5foY8qStpOYjkw+Rl Y7y+Bj4FPGEl466pClpSazSZI8TtoZc2eyP6F3RYDgn469icb5bUQjrgwsM4rJXF7c eNea+DutRZwZTbm+r/GipO/qxcYW8eoFir5iPZOYXZrmgRvuKjlIJ4i3NIqFxheOmV 83FC58NfwgA+bi6jQ4brp+axcaVBlkcLkpU8ZiS1xwmVoZvs08H7xQRkAqg4jcdf9k Ig6M+rD53392A== Subject: Re: [dpdk-dev] [PATCH v2] net/mlx5: make flow operation thread safe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi, (Seems I replied to the wrong version yesterday.) Since the issue happens only in the flow flush when the port status change. The root cause is the LSC callback should not be called before the port sta= rt is done. BR, SuanmingMou > -----Original Message----- > From: dev On Behalf Of Weifeng Li > Sent: Sunday, November 8, 2020 3:54 PM > To: Matan Azrad > Cc: dev@dpdk.org; Weifeng Li > Subject: [dpdk-dev] [PATCH v2] net/mlx5: make flow operation thread safe >=20 > Does it need a lock for flow about below scene. > Thread1: flow_list_destroy----flow_list_create > Thread2: ---------flow_list_destroy---- > Maybe the same flow can be operate at the same time. >=20 > When i start mlx5 bond and trigger LSC at the same time. > It is possible to assert in mlx5_rx_queue_release func and print "port 4 = Rx queue > 0 is still used by a flow and cannot be removed". I use dpdk-testpmd to s= imulate > the test. >=20 > Signed-off-by: Weifeng Li > --- > v2: adjust coding style issue. > --- > drivers/net/mlx5/linux/mlx5_os.c | 1 + > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_flow.c | 13 +++++++++++-- > 3 files changed, 13 insertions(+), 2 deletions(-) >=20 > diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/ml= x5_os.c > index c78d56f..59c074e 100644 > --- a/drivers/net/mlx5/linux/mlx5_os.c > +++ b/drivers/net/mlx5/linux/mlx5_os.c > @@ -1426,6 +1426,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, > MLX5_MAX_MAC_ADDRESSES); > priv->flows =3D 0; > priv->ctrl_flows =3D 0; > + rte_spinlock_init(&priv->flow_lock); > rte_spinlock_init(&priv->flow_list_lock); > TAILQ_INIT(&priv->flow_meters); > TAILQ_INIT(&priv->flow_meter_profiles); > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > b43a8c9..860bf2f 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -963,6 +963,7 @@ struct mlx5_priv { > unsigned int reta_idx_n; /* RETA index size. */ > struct mlx5_drop drop_queue; /* Flow drop queues. */ > uint32_t flows; /* RTE Flow rules. */ > + rte_spinlock_t flow_lock; > uint32_t ctrl_flows; /* Control flow rules. */ > rte_spinlock_t flow_list_lock; > struct mlx5_obj_ops obj_ops; /* HW objects operations. */ diff --git > a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index > e9c0ddd..69d8159 100644 > --- a/drivers/net/mlx5/mlx5_flow.c > +++ b/drivers/net/mlx5/mlx5_flow.c > @@ -5577,6 +5577,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t > *list, > external, hairpin_flow, error); > if (ret < 0) > goto error_before_hairpin_split; > + rte_spinlock_lock(&priv->flow_lock); > flow =3D mlx5_ipool_zmalloc(priv->sh->ipool[MLX5_IPOOL_RTE_FLOW], > &idx); > if (!flow) { > rte_errno =3D ENOMEM; > @@ -5598,8 +5599,10 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t > *list, > memset(rss_desc, 0, offsetof(struct mlx5_flow_rss_desc, queue)); > rss =3D flow_get_rss_action(p_actions_rx); > if (rss) { > - if (flow_rss_workspace_adjust(wks, rss_desc, rss->queue_num)) > + if (flow_rss_workspace_adjust(wks, rss_desc, rss->queue_num)) > { > + rte_spinlock_unlock(&priv->flow_lock); > return 0; > + } > /* > * The following information is required by > * mlx5_flow_hashfields_adjust() in advance. > @@ -5723,6 +5726,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t > *list, > __atomic_add_fetch(&tunnel->refctn, 1, __ATOMIC_RELAXED); > mlx5_free(default_miss_ctx.queue); > } > + rte_spinlock_unlock(&priv->flow_lock); > return idx; > error: > MLX5_ASSERT(flow); > @@ -5738,6 +5742,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t > *list, > wks->flow_nested_idx =3D 0; > error_before_hairpin_split: > rte_free(translated_actions); > + rte_spinlock_unlock(&priv->flow_lock); > return 0; > } >=20 > @@ -5877,11 +5882,14 @@ flow_list_destroy(struct rte_eth_dev *dev, > uint32_t *list, > uint32_t flow_idx) > { > struct mlx5_priv *priv =3D dev->data->dev_private; > + rte_spinlock_lock(&priv->flow_lock); > struct rte_flow *flow =3D mlx5_ipool_get(priv->sh->ipool > [MLX5_IPOOL_RTE_FLOW], > flow_idx); >=20 > - if (!flow) > + if (!flow) { > + rte_spinlock_unlock(&priv->flow_lock); > return; > + } > /* > * Update RX queue flags only if port is started, otherwise it is > * already clean. > @@ -5908,6 +5916,7 @@ flow_list_destroy(struct rte_eth_dev *dev, uint32_t > *list, > } > flow_mreg_del_copy_action(dev, flow); > mlx5_ipool_free(priv->sh->ipool[MLX5_IPOOL_RTE_FLOW], flow_idx); > + rte_spinlock_unlock(&priv->flow_lock); > } >=20 > /** > -- > 2.9.5