From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 23C4DA04E6; Wed, 18 Nov 2020 03:10:01 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id D4C9F58C4; Wed, 18 Nov 2020 03:09:58 +0100 (CET) Received: from nat-hk.nvidia.com (nat-hk.nvidia.com [203.18.50.4]) by dpdk.org (Postfix) with ESMTP id 865464C90 for ; Wed, 18 Nov 2020 03:09:55 +0100 (CET) Received: from HKMAIL103.nvidia.com (Not Verified[10.18.92.100]) by nat-hk.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Wed, 18 Nov 2020 10:09:53 +0800 Received: from HKMAIL104.nvidia.com (10.18.16.13) by HKMAIL103.nvidia.com (10.18.16.12) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 18 Nov 2020 02:09:52 +0000 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.41) by HKMAIL104.nvidia.com (10.18.16.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Wed, 18 Nov 2020 02:09:52 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A0VopFUvIrVfuC2xjJqDpFbR1/RSvB/Q70ySNY0TclG0SU+L8D0Pk5JdMSqc1Vm8bNBQ92zJNlE7F5GJ82RG7O2pSGEVpWj8D7oypVDWwrF57M+CWvR0EIUmRCeUCDIcxfF3Kp7Pfva+8XDkBdjkUcmWoo76IhZjop1juXD5/JgiM8dp2D3obg9mk9frFpHnsnnYk1POfS3gQ1X0Gn1Zls0p9aBE0fRSYZVt0g88/8sLs1G5C6Wav4xxWGdVwLAh8Fm+x+5U3O7hXBVKmxSj8H9vAn3RvxjOLwrOIzm+pyq+aq5ulo0fVGlcVg6u5CIT/Dh1tk8PuvseUzmbMZK1uA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5VrRLyQYRs1y/TpbETvawWZqLZAVm1DK+eWTo8/Ksas=; b=ghOsLWV1VCQd6sb7n1ZdCVcbXZUOloyHxvhoBe3Knv5nHejYqTq3teuwMebwXp/U+Zzp6orAJTxUibtjqgViws5nXuvTsvUsO98UumbzG3fP+WVs5sIOrs3Do+JWtgcMLcdH5sCrESFDHXw3mG7VcGbBm/QYvSJVz72obJJf+QrrEjQsedI0r42gX3QpIGZ7+MBxavJUighe2xsbNw4OcH1zqmB+UyXmohr+0HapPdsgjFQ4wtxPLIMxcpF8FxCbWNdFkiUO8qWqETmyMSrKk92sacCSsIKgm36Yr6IfMAhovrUKA1jVWg42dw4rrEcw2GMYCPs0/Gvm2NmyoKKllA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none Received: from MWHPR1201MB0189.namprd12.prod.outlook.com (2603:10b6:301:59::19) by MWHPR1201MB0190.namprd12.prod.outlook.com (2603:10b6:301:55::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3564.28; Wed, 18 Nov 2020 02:09:49 +0000 Received: from MWHPR1201MB0189.namprd12.prod.outlook.com ([fe80::406c:cecb:1e3a:80e4]) by MWHPR1201MB0189.namprd12.prod.outlook.com ([fe80::406c:cecb:1e3a:80e4%11]) with mapi id 15.20.3589.020; Wed, 18 Nov 2020 02:09:48 +0000 From: Suanming Mou To: Weifeng Li , Matan Azrad CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH] net/mlx5: make flow opration thread safe Thread-Index: AQHWtaKHpaEH4fTZjUCq2RXzRpJqaKnNM8cw Date: Wed, 18 Nov 2020 02:09:48 +0000 Message-ID: References: <20201108073714.1448-1-liweifeng96@126.com> In-Reply-To: <20201108073714.1448-1-liweifeng96@126.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: 126.com; dkim=none (message not signed) header.d=none;126.com; dmarc=none action=none header.from=nvidia.com; x-originating-ip: [36.24.141.208] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 91b66356-270a-4e0f-b727-08d88b6706f6 x-ms-traffictypediagnostic: MWHPR1201MB0190: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4125; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: WKTjjGzszsvA4njdbAGUHR20wgoGHhnfDjTKrPkxod4l6P++uVjuHSWbG1/UMDSMwssFyx6Bz4k/K27zL6bbbW6X8DBuKfX2os7M3csjvFI8CoyaOaAL4SmvyrERtvT4uuOktVzezexPFiBQfri/9S/hHh1cFiTkV7OxT2n0av5OzK4A3RAg5Xb+UlTt/CSCFUEd8uiFBpB83IkBa1Olf6FRcFTUgRT1HHQmiMtbLqBak+Crxta40+DJBkftnRh2jwRx6NAQDPNFQpgkt8H8TkDRt39k97at2bohd5Brd3B9Y72Nxf1YmDl4E0jMpaAlT2cPj4DzlN2e5W4B4UegVw== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR1201MB0189.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(396003)(346002)(39860400002)(376002)(366004)(66446008)(110136005)(66946007)(76116006)(66476007)(66556008)(316002)(5660300002)(6636002)(33656002)(52536014)(7696005)(64756008)(4326008)(71200400001)(8936002)(2906002)(6506007)(53546011)(55016002)(86362001)(9686003)(8676002)(83380400001)(186003)(26005)(478600001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: curYoW3eV8iXjvWXYfozkBs+IMAnG2Y/FOG86wEyJQ1X/NxjBVQpTznl/3Lyb6ypOYBljetdXiVF7DJE4Svb87kksSpiWra5yBFMNymJzkBa7w2fqm/8308icTU6zsuno1f70AElUKAscJv1iIk+djNIyaFBJkXbV7BZ59SJQHCbgk1TjbmzTn4IR7LF2HwU4rEyNRrVxM2mM7QOImKlwf0433vsdVh8uzMCuSc6piY17jK5qMD5xa173YySd+dGxXq6rxLjwPh3EfT3usBecflLWxw6FYCgNpXszLJwlRQA2MOA27qK3vKWMrUliXnSFs0Lrwtm3/Qi+otOiJWpTnMX2UPTPzo0NnoSkbeRvCk6CmpFgwXuNgW6/AsHTfK0F/eaEsuxMsbZ5Pg022MqWa0xW3Ut6yazg8Xs7QXtwFaoee+RgaCoyuDLSUsQpIUSzAbnQpgUBhqo13FpbbKVrbiC3Uny11hWPJyNRZeGjRZCURKmnqI8OoR+WhdKo+bJaSQf3+9mc9Ouus9CvFC+/EFbGxkYPy4a8+njq/abjslAg3k+omNLqEtrGMI0JWXpo7yUyfhU4TW+HIk8DRA6lDwR1L0h3TMTfdyqk7AFLDx2HeKgQxij/4bP6aZ6JgKfdzZoCiUsw0whyh+K1SRET6rdXPPICg0I6zghrBPD9ImEph8phE30TiFKPIp6/fx2UHu26ne2jPM4bEJsTKrYrQ1ty3G38bumrj/A7FG/Ju7/NpKkmc576p0TDSYVY7+DK08qxm20zeBMOUdErLXmAT2cvQRMxknkFFKsTY2ois4F3UcwVOHNInSQmeg/grFfeGWb6Xa142gOhnPY8Gr3YXeq33p5Awp/ell3t7r67grdVeXXAJY4VQMdzlAOqzZFPJtGQavoRdz5y/HlCtm5wA== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: MWHPR1201MB0189.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 91b66356-270a-4e0f-b727-08d88b6706f6 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Nov 2020 02:09:48.6590 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: hrSFtCwPWzF4ODaWtiNhqgzJm4ttJkbSJwG9HSPett8+2V8zVTMa6+DMC/SIXIB3cFNYJ7EA8YbFrTNiBM3p/w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0190 X-OriginatorOrg: Nvidia.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1605665393; bh=5VrRLyQYRs1y/TpbETvawWZqLZAVm1DK+eWTo8/Ksas=; h=ARC-Seal:ARC-Message-Signature:ARC-Authentication-Results:From:To: CC:Subject:Thread-Topic:Thread-Index:Date:Message-ID:References: In-Reply-To:Accept-Language:Content-Language:X-MS-Has-Attach: X-MS-TNEF-Correlator:authentication-results:x-originating-ip: x-ms-publictraffictype:x-ms-office365-filtering-correlation-id: x-ms-traffictypediagnostic:x-ms-exchange-transport-forked: x-microsoft-antispam-prvs:x-ms-oob-tlc-oobclassifiers: x-ms-exchange-senderadcheck:x-microsoft-antispam: x-microsoft-antispam-message-info:x-forefront-antispam-report: x-ms-exchange-antispam-messagedata:Content-Type: Content-Transfer-Encoding:MIME-Version: X-MS-Exchange-CrossTenant-AuthAs: X-MS-Exchange-CrossTenant-AuthSource: X-MS-Exchange-CrossTenant-Network-Message-Id: X-MS-Exchange-CrossTenant-originalarrivaltime: X-MS-Exchange-CrossTenant-fromentityheader: X-MS-Exchange-CrossTenant-id:X-MS-Exchange-CrossTenant-mailboxtype: X-MS-Exchange-CrossTenant-userprincipalname: X-MS-Exchange-Transport-CrossTenantHeadersStamped:X-OriginatorOrg; b=LjK16+lEzuHw0AND30h81Dae5VfjvpY0fOuCARRB/i9JHNlYOIlxTP1hARVdSy5eD VN/NxtktXyqGLBCefAM1jiHFjMgHDWtYDRhW1IpXK5N/PLojfPv3k0wN0pZpLPERTm 4dcSJ4h/ra2UJBMl3nSoS5fuM0MUP0cmShmwJ86Ow0XCf+tsO3mwG2XsiEp/PGttnm zJ6Xi3XqAeQsNG2WM+mDmNefnekhZCHtzfN3UJCEYDGWuPIwMWZTMofiOtMGpBPOmY u8mfKD6DFLwXxlGS3B5hNPZRurIkYFJAWcReyqLOJEnYjp+U/CxVLXS5BeX8qRFaUi mrwb01VItN++w== Subject: Re: [dpdk-dev] [PATCH] net/mlx5: make flow opration thread safe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Weifeng, Since the issue happens only in the flow flush when the port status change. The root cause is the LSC callback should not be called before the port sta= rt is done. BR, SuanmingMou > -----Original Message----- > From: dev On Behalf Of Weifeng Li > Sent: Sunday, November 8, 2020 3:37 PM > To: Matan Azrad > Cc: dev@dpdk.org; Weifeng Li > Subject: [dpdk-dev] [PATCH] net/mlx5: make flow opration thread safe >=20 > Does it neet a lock for flow about below scene. > Thread1: flow_list_destroy----flow_list_create > Thread2: ---------flow_list_destroy---- > Maybe the same flow can be operate at the same time. >=20 > When i start mlx5 bond and trigger LSC at the same time. > It is possible to assert in mlx5_rx_queue_release func and print "port 4 = Rx queu 0 > is still used by a flow and cannot be removed". I use dpdk-testpmd to sim= ulate > the test. >=20 > Signed-off-by: Weifeng Li > --- > drivers/net/mlx5/linux/mlx5_os.c | 1 + > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_flow.c | 13 +++++++++++-- > 3 files changed, 13 insertions(+), 2 deletions(-) >=20 > diff --git a/drivers/net/mlx5/linux/mlx5_os.c b/drivers/net/mlx5/linux/ml= x5_os.c > index c78d56f..59c074e 100644 > --- a/drivers/net/mlx5/linux/mlx5_os.c > +++ b/drivers/net/mlx5/linux/mlx5_os.c > @@ -1426,6 +1426,7 @@ mlx5_dev_spawn(struct rte_device *dpdk_dev, > MLX5_MAX_MAC_ADDRESSES); > priv->flows =3D 0; > priv->ctrl_flows =3D 0; > + rte_spinlock_init(&priv->flow_lock); > rte_spinlock_init(&priv->flow_list_lock); > TAILQ_INIT(&priv->flow_meters); > TAILQ_INIT(&priv->flow_meter_profiles); > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > b43a8c9..860bf2f 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -963,6 +963,7 @@ struct mlx5_priv { > unsigned int reta_idx_n; /* RETA index size. */ > struct mlx5_drop drop_queue; /* Flow drop queues. */ > uint32_t flows; /* RTE Flow rules. */ > + rte_spinlock_t flow_lock; > uint32_t ctrl_flows; /* Control flow rules. */ > rte_spinlock_t flow_list_lock; > struct mlx5_obj_ops obj_ops; /* HW objects operations. */ diff --git > a/drivers/net/mlx5/mlx5_flow.c b/drivers/net/mlx5/mlx5_flow.c index > e9c0ddd..69d8159 100644 > --- a/drivers/net/mlx5/mlx5_flow.c > +++ b/drivers/net/mlx5/mlx5_flow.c > @@ -5577,6 +5577,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t > *list, > external, hairpin_flow, error); > if (ret < 0) > goto error_before_hairpin_split; > + rte_spinlock_lock(&priv->flow_lock); > flow =3D mlx5_ipool_zmalloc(priv->sh->ipool[MLX5_IPOOL_RTE_FLOW], > &idx); > if (!flow) { > rte_errno =3D ENOMEM; > @@ -5598,8 +5599,10 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t > *list, > memset(rss_desc, 0, offsetof(struct mlx5_flow_rss_desc, queue)); > rss =3D flow_get_rss_action(p_actions_rx); > if (rss) { > - if (flow_rss_workspace_adjust(wks, rss_desc, rss->queue_num)) > + if (flow_rss_workspace_adjust(wks, rss_desc, rss->queue_num)) > { > + rte_spinlock_unlock(&priv->flow_lock); > return 0; > + } > /* > * The following information is required by > * mlx5_flow_hashfields_adjust() in advance. > @@ -5723,6 +5726,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t > *list, > __atomic_add_fetch(&tunnel->refctn, 1, __ATOMIC_RELAXED); > mlx5_free(default_miss_ctx.queue); > } > + rte_spinlock_unlock(&priv->flow_lock); > return idx; > error: > MLX5_ASSERT(flow); > @@ -5738,6 +5742,7 @@ flow_list_create(struct rte_eth_dev *dev, uint32_t > *list, > wks->flow_nested_idx =3D 0; > error_before_hairpin_split: > rte_free(translated_actions); > + rte_spinlock_unlock(&priv->flow_lock); > return 0; > } >=20 > @@ -5877,11 +5882,14 @@ flow_list_destroy(struct rte_eth_dev *dev, > uint32_t *list, > uint32_t flow_idx) > { > struct mlx5_priv *priv =3D dev->data->dev_private; > + rte_spinlock_lock(&priv->flow_lock); > struct rte_flow *flow =3D mlx5_ipool_get(priv->sh->ipool > [MLX5_IPOOL_RTE_FLOW], > flow_idx); >=20 > - if (!flow) > + if (!flow) { > + rte_spinlock_unlock(&priv->flow_lock); > return; > + } > /* > * Update RX queue flags only if port is started, otherwise it is > * already clean. > @@ -5908,6 +5916,7 @@ flow_list_destroy(struct rte_eth_dev *dev, uint32_t > *list, > } > flow_mreg_del_copy_action(dev, flow); > mlx5_ipool_free(priv->sh->ipool[MLX5_IPOOL_RTE_FLOW], flow_idx); > + rte_spinlock_unlock(&priv->flow_lock); > } >=20 > /** > -- > 2.9.5