From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 5761EA0679 for ; Tue, 2 Apr 2019 21:10:15 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 628A01B0F9; Tue, 2 Apr 2019 21:10:00 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20067.outbound.protection.outlook.com [40.107.2.67]) by dpdk.org (Postfix) with ESMTP id 2D0375F72 for ; Tue, 2 Apr 2019 21:09:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VndO6iCucmuN5urm2btBJtpjOSSRvFHdWLnelmApwqI=; b=TBamRYkhuyDnOTR0LAwqC0mgDW+pi94gv+Jx5D9B7wMkKos1hRGlcN3whMle3LrIDHEf3espufu1lSV4uCNJ6P24V66CWk6hBBQMSzByFfYF8EFakSiiX9zlU1K/j2o/XXx/3m7p1SUjLVIZjzX2JX1A7NR0SfRdQSUGFQKrcjo= Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com (52.133.45.150) by AM0PR0502MB4050.eurprd05.prod.outlook.com (52.133.37.26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1750.20; Tue, 2 Apr 2019 19:09:58 +0000 Received: from AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003]) by AM0PR0502MB3795.eurprd05.prod.outlook.com ([fe80::84f3:7e92:7a51:1003%2]) with mapi id 15.20.1750.014; Tue, 2 Apr 2019 19:09:58 +0000 From: Shahaf Shuler To: Slava Ovsiienko , "dev@dpdk.org" Thread-Topic: [PATCH 4/4] net/mlx5: add mutex for shared DV/DR structures Thread-Index: AQHU6RyBk0wyi9HQOUKzgfuF2Mzx4KYpObYg Date: Tue, 2 Apr 2019 19:09:57 +0000 Message-ID: References: <1554186157-29455-1-git-send-email-viacheslavo@mellanox.com> <1554186157-29455-5-git-send-email-viacheslavo@mellanox.com> In-Reply-To: <1554186157-29455-5-git-send-email-viacheslavo@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=shahafs@mellanox.com; x-originating-ip: [31.154.10.105] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 65d0839c-7a3b-48e0-500a-08d6b79ecc5a x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600139)(711020)(4605104)(4618075)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7193020); SRVR:AM0PR0502MB4050; x-ms-traffictypediagnostic: AM0PR0502MB4050: x-microsoft-antispam-prvs: x-forefront-prvs: 0995196AA2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(346002)(396003)(39860400002)(376002)(136003)(366004)(189003)(199004)(97736004)(55016002)(71200400001)(68736007)(14454004)(9686003)(53936002)(26005)(6116002)(6246003)(71190400001)(6506007)(3846002)(99286004)(476003)(446003)(186003)(76176011)(2501003)(6436002)(7696005)(52536014)(11346002)(102836004)(486006)(305945005)(106356001)(33656002)(316002)(7736002)(110136005)(14444005)(86362001)(229853002)(74316002)(66066001)(105586002)(5660300002)(81156014)(8936002)(81166006)(25786009)(2906002)(8676002)(256004)(478600001); DIR:OUT; SFP:1101; SCL:1; SRVR:AM0PR0502MB4050; H:AM0PR0502MB3795.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: IdTmRkMVcvOfSU7lBqznmYkDPGw6QKWiDU9FbqlVUXlMyPJ4SWeQTiVQfRF3GtgG9bPA7F/MMIqXWsAqJaEv4xzHYSsu0uSvFaCKYMePo9V9Pl7OgGwztCF/T6GLI/2mDvaM/xT+IfXipgix+F2I3s/X7i4202pPHMUw1EMxS1UIQPsefvUGvxjFNCHBp1BvQxLa4aDRm9/ry1E3snxybwH/2eh39gNfkJOrYGGpr24hXxBpbCAqC0UkqSqvFBKggTMrNvtdfLKQi4w64ujR/8LagpEp/sQ0KXinrVzfCPdUwB2K5G0a3VAM8mdI2ppyL6osgmFoeel02fgOHME9DQ6lfce5Rkg/erDafnzlf2l48FOFl/v1SWwv1nLgsDvJoyc3rVimk/ULWrFkHZVTPfNMV8mLJu/f6zqPXkmuzhY= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65d0839c-7a3b-48e0-500a-08d6b79ecc5a X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Apr 2019 19:09:57.9679 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM0PR0502MB4050 Subject: Re: [dpdk-dev] [PATCH 4/4] net/mlx5: add mutex for shared DV/DR structures X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190402190957.BmFfd9xsNvYGxwALzDAysbtvfZqZSrYmQ5fxNx5yids@z> Tuesday, April 2, 2019 9:23 AM, Viacheslav Ovsiienko: > Subject: [PATCH 4/4] net/mlx5: add mutex for shared DV/DR structures Same comment about title.=20 >=20 > This patch introduces the mutex for shared DV/DR structures. > Application may have multiple threads (but single dedicated thread per po= rt). > Due to sharing the IB context in the multiport IB device configurations w= e > should synchronize access to the shared DV/DR flow structures. >=20 > Signed-off-by: Viacheslav Ovsiienko > --- > drivers/net/mlx5/mlx5.c | 6 +++ > drivers/net/mlx5/mlx5.h | 1 + > drivers/net/mlx5/mlx5_flow_dv.c | 103 > ++++++++++++++++++++++++++++++++++++++-- > 3 files changed, 106 insertions(+), 4 deletions(-) >=20 > diff --git a/drivers/net/mlx5/mlx5.c b/drivers/net/mlx5/mlx5.c index > 369b698..96ad4c6 100644 > --- a/drivers/net/mlx5/mlx5.c > +++ b/drivers/net/mlx5/mlx5.c > @@ -312,6 +312,7 @@ struct mlx5_dev_spawn_data { > mlx5_alloc_shared_dv(struct mlx5_priv *priv) { > struct mlx5_ibv_shared *sh =3D priv->sh; > + pthread_mutexattr_t mattr; > int err =3D 0; > void *ns; >=20 > @@ -336,6 +337,10 @@ struct mlx5_dev_spawn_data { > err =3D errno; > goto error; > } > + pthread_mutexattr_init(&mattr); > + pthread_mutexattr_settype(&mattr, > PTHREAD_MUTEX_RECURSIVE); Please add inline comment about why you use this type of mutex (so that you= can use a single mutex for the entire flow engine logic).=20 > + pthread_mutex_init(&sh->dv_mutex, &mattr); > + pthread_mutexattr_destroy(&mattr); > sh->tx_ns =3D ns; > sh->dv_refcnt++; > priv->dv_shared =3D 1; > @@ -381,6 +386,7 @@ struct mlx5_dev_spawn_data { > mlx5dv_dr_destroy_ns(sh->tx_ns); > sh->tx_ns =3D NULL; > } > + pthread_mutex_destroy(&sh->dv_mutex); > } > #endif >=20 > diff --git a/drivers/net/mlx5/mlx5.h b/drivers/net/mlx5/mlx5.h index > e67227f..4f6c1b7 100644 > --- a/drivers/net/mlx5/mlx5.h > +++ b/drivers/net/mlx5/mlx5.h > @@ -223,6 +223,7 @@ struct mlx5_ibv_shared { > char ibdev_path[IBV_SYSFS_PATH_MAX]; /* IB device path for > secondary */ > struct ibv_device_attr_ex device_attr; /* Device properties. */ > /* Shared DV/DR flow data section. */ > + pthread_mutex_t dv_mutex; /* DV context mutex. */ > uint32_t dv_refcnt; /* DV/DR data reference counter. */ > void *rx_ns; /* RX Direct Rules name space handle. */ > struct mlx5_flow_tbl_resource rx_tbl[MLX5_MAX_TABLES]; diff --git > a/drivers/net/mlx5/mlx5_flow_dv.c b/drivers/net/mlx5/mlx5_flow_dv.c > index 4912fc8..737011a 100644 > --- a/drivers/net/mlx5/mlx5_flow_dv.c > +++ b/drivers/net/mlx5/mlx5_flow_dv.c > @@ -125,6 +125,45 @@ struct field_modify_info modify_tcp[] =3D { }; >=20 > /** > + * Acquire the synchronizing object to protect multithreaded access > + * to shared dv context. Lock occurs only if context is actually > + * shared, i.e. we have multiport IB device and representors are > + * created. > + * > + * @param[in] dev > + * Pointer to the rte_eth_dev structure. > + */ > +static void > +flow_d_shared_lock(struct rte_eth_dev *dev) { > + struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_ibv_shared *sh =3D priv->sh; > + > + if (sh->dv_refcnt > 1) { > + int ret; > + > + ret =3D pthread_mutex_lock(&sh->dv_mutex); > + assert(!ret); > + (void)ret; > + } > +} > + > +static void > +flow_d_shared_unlock(struct rte_eth_dev *dev) { > + struct mlx5_priv *priv =3D dev->data->dev_private; > + struct mlx5_ibv_shared *sh =3D priv->sh; > + > + if (sh->dv_refcnt > 1) { > + int ret; > + > + ret =3D pthread_mutex_unlock(&sh->dv_mutex); > + assert(!ret); > + (void)ret; > + } > +} > + > +/** > * Convert modify-header action to DV specification. > * > * @param[in] item > @@ -3958,14 +3997,70 @@ struct field_modify_info modify_tcp[] =3D { > return ret; > } >=20 > +/* > + * Mutex-protected thunk to flow_dv_translate(). > + */ > +static int > +flow_d_translate(struct rte_eth_dev *dev, > + struct mlx5_flow *dev_flow, > + const struct rte_flow_attr *attr, > + const struct rte_flow_item items[], > + const struct rte_flow_action actions[], > + struct rte_flow_error *error) > +{ > + int ret; > + > + flow_d_shared_lock(dev); > + ret =3D flow_dv_translate(dev, dev_flow, attr, items, actions, error); > + flow_d_shared_unlock(dev); > + return ret; > +} > + > +/* > + * Mutex-protected thunk to flow_dv_apply(). > + */ > +static int > +flow_d_apply(struct rte_eth_dev *dev, > + struct rte_flow *flow, > + struct rte_flow_error *error) > +{ > + int ret; > + > + flow_d_shared_lock(dev); > + ret =3D flow_dv_apply(dev, flow, error); > + flow_d_shared_unlock(dev); > + return ret; > +} > + > +/* > + * Mutex-protected thunk to flow_dv_remove(). > + */ > +static void > +flow_d_remove(struct rte_eth_dev *dev, struct rte_flow *flow) { > + flow_d_shared_lock(dev); > + flow_dv_remove(dev, flow); > + flow_d_shared_unlock(dev); > +} > + > +/* > + * Mutex-protected thunk to flow_dv_destroy(). > + */ > +static void > +flow_d_destroy(struct rte_eth_dev *dev, struct rte_flow *flow) { > + flow_d_shared_lock(dev); > + flow_dv_destroy(dev, flow); > + flow_d_shared_unlock(dev); > +} >=20 > const struct mlx5_flow_driver_ops mlx5_flow_dv_drv_ops =3D { > .validate =3D flow_dv_validate, > .prepare =3D flow_dv_prepare, > - .translate =3D flow_dv_translate, > - .apply =3D flow_dv_apply, > - .remove =3D flow_dv_remove, > - .destroy =3D flow_dv_destroy, > + .translate =3D flow_d_translate, > + .apply =3D flow_d_apply, > + .remove =3D flow_d_remove, > + .destroy =3D flow_d_destroy, > .query =3D flow_dv_query, > }; >=20 > -- > 1.8.3.1