From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr60055.outbound.protection.outlook.com [40.107.6.55]) by dpdk.org (Postfix) with ESMTP id 1A2461B040 for ; Sun, 7 Jan 2018 10:58:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=DkdVsq13qa7PK1Z8fwi88Nbh5U+CLZ3jcaql+Mlgu9I=; b=FUTcTl0qov0OkbdKCfGZXmW1DSSg+SMUh9uNbz/TysSdlYrNkjYxCXDwktlpJvW8XFtmXqsMpZCiQSZnkYEtQAPbHy/I239iyhpou3RI2DLNBQViFopVZwapgzE0/Mx3rVXDCfB5+dbDYqGbYxIeylSzdA2d+xdt1JMKYnmK49U= Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com (52.133.21.26) by AM6PR0502MB3798.eurprd05.prod.outlook.com (52.133.21.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.386.5; Sun, 7 Jan 2018 09:58:18 +0000 Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com ([fe80::b4b4:7de8:cf70:aa3a]) by AM6PR0502MB3797.eurprd05.prod.outlook.com ([fe80::b4b4:7de8:cf70:aa3a%13]) with mapi id 15.20.0386.006; Sun, 7 Jan 2018 09:58:18 +0000 From: Matan Azrad To: Matan Azrad , Thomas Monjalon , Gaetan Rivet , Jingjing Wu CC: "dev@dpdk.org" , Neil Horman , "Bruce Richardson" , Konstantin Ananyev Thread-Topic: [dpdk-dev] [PATCH v2 3/6] ethdev: synchronize port allocation Thread-Index: AQHTh5x8h2XvX7oBJUm5iQisr2XbFqNoLH2w Date: Sun, 7 Jan 2018 09:58:17 +0000 Message-ID: References: <1511870281-15282-1-git-send-email-matan@mellanox.com> <1515318351-4756-1-git-send-email-matan@mellanox.com> <1515318351-4756-4-git-send-email-matan@mellanox.com> In-Reply-To: <1515318351-4756-4-git-send-email-matan@mellanox.com> Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; x-originating-ip: [193.47.165.251] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR0502MB3798; 7:Sr00bLlf5V7Nz8tPmqtaw9DOJ+ZNp8NrjJe3KftXRmY9WQZEvm2mCT8TmKxGGLE22cgGYJQ6Fjjpf1J+07CPR5xilPHwRtlrcb4zJamiSSTUlPeOv1WlLrCj3OVqBbpRwYdS+dqMtMsRW2eAgO9/6IoiDp28JDIM1va2sCsoev9a8aAVDWxznSl71Ul4L1YCg7c0fbEmijHCBxiu7x5Vsjgmdc2dsGmmxLlOr6Coa2yjPfMbu8c9uxFaV8C93Jrg x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 1ef43d28-8dcc-4d1f-aae3-08d555b52d65 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(48565401081)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(5600026)(4604075)(3008032)(2017052603307)(7153060)(7193020); SRVR:AM6PR0502MB3798; x-ms-traffictypediagnostic: AM6PR0502MB3798: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(60795455431006)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(8121501046)(5005006)(3231023)(944501075)(3002001)(10201501046)(93006095)(93001095)(6055026)(6041268)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123558120)(20161123562045)(20161123564045)(6072148)(201708071742011); SRVR:AM6PR0502MB3798; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:AM6PR0502MB3798; x-forefront-prvs: 0545EFAC9A x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(39380400002)(376002)(366004)(346002)(39860400002)(189003)(199004)(13464003)(59450400001)(99286004)(81156014)(3280700002)(102836004)(5250100002)(54906003)(105586002)(76176011)(106356001)(6246003)(5890100001)(6436002)(81166006)(55016002)(8676002)(97736004)(2900100001)(33656002)(3660700001)(110136005)(7696005)(25786009)(86362001)(14454004)(74316002)(53936002)(8936002)(305945005)(478600001)(2950100002)(5660300001)(3846002)(66066001)(316002)(53546011)(6506007)(6116002)(2906002)(229853002)(68736007)(4326008)(9686003)(7736002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR0502MB3798; H:AM6PR0502MB3797.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: Sekw94IDgs0umLEW9MF9uoGvQsBn8Mw6h25XkNjOy660sqV6pfbpIgNzB75WnJdoxxLHntXOncV+gQhr7dMCGA== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1ef43d28-8dcc-4d1f-aae3-08d555b52d65 X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jan 2018 09:58:18.0153 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0502MB3798 Subject: Re: [dpdk-dev] [PATCH v2 3/6] ethdev: synchronize port allocation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jan 2018 09:58:20 -0000 Self-review. > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Matan Azrad > Sent: Sunday, January 7, 2018 11:46 AM > To: Thomas Monjalon ; Gaetan Rivet > ; Jingjing Wu > Cc: dev@dpdk.org; Neil Horman ; Bruce > Richardson ; Konstantin Ananyev > > Subject: [dpdk-dev] [PATCH v2 3/6] ethdev: synchronize port allocation >=20 > Ethernet port allocation was not thread safe, means 2 threads which tried= to > allocate a new port at the same time might get an identical port identifi= er and > caused to memory overwrite. > Actually, all the port configurations were not thread safe from ethdev po= int > of view. >=20 > The port ownership mechanism added to the ethdev is a good point to > redefine the synchronization rules in ethdev: >=20 > 1. The port allocation and port release synchronization will be > managed by ethdev. > 2. The port usage synchronization will be managed by the port owner. > 3. The port ownership synchronization will be managed by ethdev. >=20 > Add port allocation synchronization to complete the new rules. >=20 > Signed-off-by: Matan Azrad > --- > config/common_base | 4 ++-- > lib/librte_ether/rte_ethdev.c | 39 ++++++++++++++++++++++++++++------ > ----- > 2 files changed, 30 insertions(+), 13 deletions(-) >=20 > diff --git a/config/common_base b/config/common_base index > b8ee8f9..980ae3b 100644 > --- a/config/common_base > +++ b/config/common_base > @@ -94,7 +94,7 @@ CONFIG_RTE_MAX_MEMSEG=3D256 > CONFIG_RTE_MAX_MEMZONE=3D2560 > CONFIG_RTE_MAX_TAILQ=3D32 > CONFIG_RTE_ENABLE_ASSERT=3Dn > -CONFIG_RTE_LOG_LEVEL=3DRTE_LOG_INFO > +CONFIG_RTE_LOG_LEVEL=3DRTE_LOG_DEBUG Wrong change. Will fix in the next version. > CONFIG_RTE_LOG_DP_LEVEL=3DRTE_LOG_INFO > CONFIG_RTE_LOG_HISTORY=3D256 > CONFIG_RTE_BACKTRACE=3Dy > @@ -136,7 +136,7 @@ CONFIG_RTE_LIBRTE_KVARGS=3Dy # Compile generic > ethernet library # CONFIG_RTE_LIBRTE_ETHER=3Dy - > CONFIG_RTE_LIBRTE_ETHDEV_DEBUG=3Dn > +CONFIG_RTE_LIBRTE_ETHDEV_DEBUG=3Dy Wrong change. Will fix in the next version. > CONFIG_RTE_MAX_ETHPORTS=3D32 > CONFIG_RTE_MAX_QUEUES_PER_PORT=3D1024 > CONFIG_RTE_LIBRTE_IEEE1588=3Dn > diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.= c > index 0e12452..d30d61f 100644 > --- a/lib/librte_ether/rte_ethdev.c > +++ b/lib/librte_ether/rte_ethdev.c > @@ -85,6 +85,9 @@ > /* spinlock for add/remove tx callbacks */ static rte_spinlock_t > rte_eth_tx_cb_lock =3D RTE_SPINLOCK_INITIALIZER; >=20 > +/* spinlock for shared data allocation */ static rte_spinlock_t > +rte_eth_share_data_alloc =3D RTE_SPINLOCK_INITIALIZER; > + > /* spinlock for eth device ownership management stored in shared memory > */ static rte_spinlock_t *rte_eth_dev_ownership_lock; >=20 > @@ -234,21 +237,27 @@ struct rte_eth_dev * rte_eth_dev_allocate(const > char *name) { > uint16_t port_id; > - struct rte_eth_dev *eth_dev; > + struct rte_eth_dev *eth_dev =3D NULL; > + > + /* Synchronize share data one time allocation between local threads. > */ > + rte_spinlock_lock(&rte_eth_share_data_alloc); > + if (rte_eth_dev_data =3D=3D NULL) > + rte_eth_dev_share_data_alloc(); > + rte_spinlock_unlock(&rte_eth_share_data_alloc); > + > + /* Synchronize port creation between primary and secondary > threads. */ > + rte_spinlock_lock(rte_eth_dev_ownership_lock); >=20 > port_id =3D rte_eth_dev_find_free_port(); > if (port_id =3D=3D RTE_MAX_ETHPORTS) { > RTE_PMD_DEBUG_TRACE("Reached maximum number of > Ethernet ports\n"); > - return NULL; > + goto unlock; > } >=20 > - if (rte_eth_dev_data =3D=3D NULL) > - rte_eth_dev_share_data_alloc(); > - > if (rte_eth_dev_allocated(name) !=3D NULL) { > RTE_PMD_DEBUG_TRACE("Ethernet Device with name %s > already allocated!\n", > name); > - return NULL; > + goto unlock; > } >=20 > eth_dev =3D eth_dev_get(port_id); > @@ -256,6 +265,8 @@ struct rte_eth_dev * > eth_dev->data->port_id =3D port_id; > eth_dev->data->mtu =3D ETHER_MTU; >=20 > +unlock: > + rte_spinlock_unlock(rte_eth_dev_ownership_lock); > return eth_dev; > } >=20 > @@ -268,10 +279,16 @@ struct rte_eth_dev * > rte_eth_dev_attach_secondary(const char *name) { > uint16_t i; > - struct rte_eth_dev *eth_dev; > + struct rte_eth_dev *eth_dev =3D NULL; >=20 > + /* Synchronize share data one time attachment between local > threads. */ > + rte_spinlock_lock(&rte_eth_share_data_alloc); > if (rte_eth_dev_data =3D=3D NULL) > rte_eth_dev_share_data_alloc(); > + rte_spinlock_unlock(&rte_eth_share_data_alloc); > + > + /* Synchronize port attachment to primary port creation and release. > */ > + rte_spinlock_lock(rte_eth_dev_ownership_lock); >=20 > for (i =3D 0; i < RTE_MAX_ETHPORTS; i++) { > if (strcmp(rte_eth_dev_data[i].name, name) =3D=3D 0) @@ - > 281,12 +298,12 @@ struct rte_eth_dev * > RTE_PMD_DEBUG_TRACE( > "device %s is not driven by the primary process\n", > name); > - return NULL; > + } else { > + eth_dev =3D eth_dev_get(i); > + RTE_ASSERT(eth_dev->data->port_id =3D=3D i); > } >=20 > - eth_dev =3D eth_dev_get(i); > - RTE_ASSERT(eth_dev->data->port_id =3D=3D i); > - > + rte_spinlock_unlock(rte_eth_dev_ownership_lock); > return eth_dev; > } >=20 > -- > 1.8.3.1