From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-eopbgr00040.outbound.protection.outlook.com [40.107.0.40]) by dpdk.org (Postfix) with ESMTP id 933881B03F for ; Wed, 17 Jan 2018 18:58:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=iQ6ZRuHpBLsWLYkjmQ5lnApimABLxQudk1le3Y1p9nw=; b=SIHGD1/EyhdXGtd7GebzBD0c4iCDFbq7JYTnfLAZb3cQUPhCzRQqoxDSX2/CNuATme5Jd1W452qLTirZYyOszRTqro62XeidUF8CC9UaHATkWt0IZh6UF7z1NR6gLqikJc6F49w9aCG9bfZT74IjntlDtu7CXFppTBXAlGtI8qk= Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com (52.133.21.26) by AM6PR0502MB3608.eurprd05.prod.outlook.com (52.133.20.29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.407.7; Wed, 17 Jan 2018 17:58:07 +0000 Received: from AM6PR0502MB3797.eurprd05.prod.outlook.com ([fe80::6c28:c6b3:de94:a733]) by AM6PR0502MB3797.eurprd05.prod.outlook.com ([fe80::6c28:c6b3:de94:a733%13]) with mapi id 15.20.0407.012; Wed, 17 Jan 2018 17:58:07 +0000 From: Matan Azrad To: Neil Horman CC: "Ananyev, Konstantin" , Thomas Monjalon , Gaetan Rivet , "Wu, Jingjing" , "dev@dpdk.org" , "Richardson, Bruce" Thread-Topic: [PATCH v2 2/6] ethdev: add port ownership Thread-Index: AQHTihf/M9xg8LYorUSRFqZtTc27hqNtNdVQgAFomACAAAOCwIAAuwQAgABvY+CABQwJgIAAEk3ggABinoCAANUIAIAAxQGAgAAGCnCAAQnKAIAAAmNwgAApSwCAAC0cwA== Date: Wed, 17 Jan 2018 17:58:07 +0000 Message-ID: References: <2601191342CEEE43887BDE71AB9772588627CCB0@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627DC25@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627DE30@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627E954@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627EE60@irsmsx105.ger.corp.intel.com> <20180117140020.GA5432@hmswarspite.think-freely.org> In-Reply-To: <20180117140020.GA5432@hmswarspite.think-freely.org> Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; x-originating-ip: [85.64.81.213] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM6PR0502MB3608; 7:2EUUSdCGc4FDBAkb5Su7hwwzbyHZ1IBvlfZGDo2/wq77GlZIWvt/QtXHovGocIwrCHxKwIFimmmKHB1L3+/iCxpWL7ARS1aE7nwvwVGVnlDOYWvI3Rn9DtREOTIaslqDNRU2urSsXGNeo55OaqpXWAT0gGffwDcmu4+mb+Q8L+nhI0FRTOWwQdR0DMGoyFmD9FFeJ4cofooFJViLWoqIUMujHCZn0v4idBDvbJas+9LELZK+n6IhYzIQh7FfVEiY x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-correlation-id: 898420af-8219-43b0-3fcd-08d55dd3dd5a x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(48565401081)(2017052603307)(7153060)(7193020); SRVR:AM6PR0502MB3608; x-ms-traffictypediagnostic: AM6PR0502MB3608: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(60795455431006); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(8121501046)(5005006)(3231023)(944501161)(10201501046)(93006095)(93001095)(3002001)(6055026)(6041268)(20161123564045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(6072148)(201708071742011); SRVR:AM6PR0502MB3608; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:AM6PR0502MB3608; x-forefront-prvs: 0555EC8317 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39380400002)(366004)(376002)(39860400002)(396003)(346002)(189003)(199004)(24454002)(2950100002)(2906002)(106356001)(86362001)(14454004)(105586002)(74316002)(33656002)(26005)(93886005)(66066001)(97736004)(5660300001)(2900100001)(5250100002)(54906003)(6916009)(7736002)(76176011)(59450400001)(4326008)(99286004)(8676002)(25786009)(478600001)(6506007)(305945005)(53936002)(6246003)(81156014)(81166006)(102836004)(316002)(8936002)(229853002)(3660700001)(3280700002)(6116002)(6436002)(68736007)(3846002)(7696005)(55016002)(9686003); DIR:OUT; SFP:1101; SCL:1; SRVR:AM6PR0502MB3608; H:AM6PR0502MB3797.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: cKYnKdtmm90dfMh//P+l475YDpzG8HwhUZyEqscRIW68U7/zQT5CV/GEqRn9I+Wykz1W2H4SSVDe7ad0yI0EKQ== spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 898420af-8219-43b0-3fcd-08d55dd3dd5a X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jan 2018 17:58:07.4861 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM6PR0502MB3608 Subject: Re: [dpdk-dev] [PATCH v2 2/6] ethdev: add port ownership X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 17:58:08 -0000 Hi Neil From: Neil Horman, Wednesday, January 17, 2018 4:00 PM > On Wed, Jan 17, 2018 at 12:05:42PM +0000, Matan Azrad wrote: > > > > Hi Konstantin > > From: Ananyev, Konstantin, Sent: Wednesday, January 17, 2018 1:24 PM > > > Hi Matan, > > > > > > > Hi Konstantin > > > > > > > > From: Ananyev, Konstantin, Tuesday, January 16, 2018 9:11 PM > > > > > Hi Matan, > > > > > > > > > > > > > > > > > Hi Konstantin > > > > > > From: Ananyev, Konstantin, Monday, January 15, 2018 8:44 PM > > > > > > > Hi Matan, > > > > > > > > Hi Konstantin > > > > > > > > From: Ananyev, Konstantin, Monday, January 15, 2018 1:45 > > > > > > > > PM > > > > > > > > > Hi Matan, > > > > > > > > > > Hi Konstantin > > > > > > > > > > From: Ananyev, Konstantin, Friday, January 12, 2018 > > > > > > > > > > 2:02 AM > > > > > > > > > > > Hi Matan, > > > > > > > > > > > > Hi Konstantin > > > > > > > > > > > > From: Ananyev, Konstantin, Thursday, January 11, > > > > > > > > > > > > 2018 > > > > > > > > > > > > 2:40 PM > > > > > > > > > > > > > Hi Matan, > > > > > > > > > > > > > > Hi Konstantin > > > > > > > > > > > > > > From: Ananyev, Konstantin, Wednesday, January > > > > > > > > > > > > > > 10, > > > > > > > > > > > > > > 2018 > > > > > > > > > > > > > > 3:36 PM > > > > > > > > > > > > > > > Hi Matan, > > > > > > > > > > > > > > > > > > > > > It is good to see that now scanning/updating > > > > > > > > > > > > > > > rte_eth_dev_data[] is lock protected, but it > > > > > > > > > > > > > > > might be not very plausible to protect both > > > > > > > > > > > > > > > data[] and next_owner_id using the > > > > > > > > > same lock. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I guess you mean to the owner structure in > > > > > > > > > rte_eth_dev_data[port_id]. > > > > > > > > > > > > > > The next_owner_id is read by ownership > > > > > > > > > > > > > > APIs(for owner validation), so it > > > > > > > > > > > > > makes sense to use the same lock. > > > > > > > > > > > > > > Actually, why not? > > > > > > > > > > > > > > > > > > > > > > > > > > Well to me next_owner_id and rte_eth_dev_data[] > > > > > > > > > > > > > are not directly > > > > > > > > > > > related. > > > > > > > > > > > > > You may create new owner_id but it doesn't mean > > > > > > > > > > > > > you would update rte_eth_dev_data[] immediately. > > > > > > > > > > > > > And visa-versa - you might just want to update > > > > > > > > > > > > > rte_eth_dev_data[].name or .owner_id. > > > > > > > > > > > > > It is not very good coding practice to use same > > > > > > > > > > > > > lock for non-related data structures. > > > > > > > > > > > > > > > > > > > > > > > > > I see the relation like next: > > > > > > > > > > > > Since the ownership mechanism synchronization is > > > > > > > > > > > > in ethdev responsibility, we must protect against > > > > > > > > > > > > user mistakes as much as we can by > > > > > > > > > > > using the same lock. > > > > > > > > > > > > So, if user try to set by invalid owner (exactly > > > > > > > > > > > > the ID which currently is > > > > > > > > > > > allocated) we can protect on it. > > > > > > > > > > > > > > > > > > > > > > Hmm, not sure why you can't do same checking with > > > > > > > > > > > different lock or atomic variable? > > > > > > > > > > > > > > > > > > > > > The set ownership API is protected by ownership lock > > > > > > > > > > and checks the owner ID validity By reading the next ow= ner > ID. > > > > > > > > > > So, the owner ID allocation and set API should use the > > > > > > > > > > same atomic > > > > > > > > > mechanism. > > > > > > > > > > > > > > > > > > Sure but all you are doing for checking validity, is > > > > > > > > > check that owner_id > 0 &&& owner_id < next_ownwe_id, > right? > > > > > > > > > As you don't allow owner_id overlap (16/3248 bits) you > > > > > > > > > can safely do same check with just > atomic_get(&next_owner_id). > > > > > > > > > > > > > > > > > It will not protect it, scenario: > > > > > > > > - current next_id is X. > > > > > > > > - call set ownership of port A with owner id X by thread > > > > > > > > 0(by user > > > > > mistake). > > > > > > > > - context switch > > > > > > > > - allocate new id by thread 1 and get X and change next_id > > > > > > > > to > > > > > > > > X+1 > > > > > > > atomically. > > > > > > > > - context switch > > > > > > > > - Thread 0 validate X by atomic_read and succeed to take > > > ownership. > > > > > > > > - The system loosed the port(or will be managed by two > > > > > > > > entities) - > > > > > crash. > > > > > > > > > > > > > > > > > > > > > Ok, and how using lock will protect you with such scenario? > > > > > > > > > > > > The owner set API validation by thread 0 should fail because > > > > > > the owner > > > > > validation is included in the protected section. > > > > > > > > > > Then your validation function would fail even if you'll use > > > > > atomic ops instead of lock. > > > > No. > > > > With atomic this specific scenario will cause the validation to pas= s. > > > > > > Can you explain to me how? > > > > > > rte_eth_is_valid_owner_id(uint16_t owner_id) { > > > int32_t cur_owner_id =3D > > > RTE_MIN(rte_atomic32_get(next_owner_id), > > > UINT16_MAX); > > > > > > if (owner_id =3D=3D RTE_ETH_DEV_NO_OWNER || owner > > > > cur_owner_id) { > > > RTE_LOG(ERR, EAL, "Invalid owner_id=3D%d.\n", owner_id); > > > return 0; > > > } > > > return 1; > > > } > > > > > > Let say your next_owne_id=3D=3DX, and you invoke > > > rte_eth_is_valid_owner_id(owner_id=3DX+1) - it would fail. > > > > Explanation: > > The scenario with locks: > > next_owner_id =3D X. > > Thread 0 call to set API(with invalid owner Y=3DX) and take lock. > > Context switch. > > Thread 1 call to owner_new and stuck in the lock. > > Context switch. > > Thread 0 does owner id validation and failed(Y>=3DX) - unlock the lock = and > return failure to the user. > > Context switch. > > Thread 1 take the lock and update X to X+1, then, unlock the lock. > > Everything is OK! > > > > The same scenario with atomics: > > next_owner_id =3D X. > > Thread 0 call to set API(with invalid owner Y=3DX) and take lock. > > Context switch. > > Thread 1 call to owner_new and change X to X+1(atomically). > > Context switch. > > Thread 0 does owner id validation and success(Y<(atomic)X+1) - unlock t= he > lock and return success to the user. > > Problem! > > >=20 >=20 > Matan is correct here, there is no way to preform parallel set operations > using just and atomic variable here, because multiple reads of > next_owner_id need to be preformed while it is stable. That is to say > rte_eth_next_owner_id must be compared to RTE_ETH_DEV_NO_OWNER > and owner_id in rte_eth_is_valid_owner_id. If you were to only use an > atomic_read on such a variable, it could be incremented by the owner_new > function between the checks and an invalid owner value could become valid > because a third thread incremented the next value. The state of > next_owner_id must be kept stable during any validity checks >=20 > That said, I really have to wonder why ownership ids are really needed he= re > at all. It seems this design could be much simpler with the addition of = a per- > port lock (and optional ownership record). The API could consist of thre= e > operations: >=20 > ownership_set > ownership_tryset > ownership_release > ownership_get >=20 >=20 > The first call simply tries to take the per-port lock (blocking if its al= ready > locked) >=20 Per port lock is not good because the ownership mechanism must to be synchr= onized with the port creation\release. So the port creation and port ownership should use the same lock. I didn't find precedence for blocking function in ethdev. > The second call is a non-blocking version of the first >=20 > The third unlocks the port, allowing others to take ownership >=20 > The fourth returns whatever ownership record you want to encode with the > lock. >=20 > The addition of all this id checking seems a bit overcomplicated You miss the identification of the owner - we want to allow info of the own= er for printing and easy debug. And it is makes sense to manage the owner uniqueness by unique ID. The API already discussed a lot in the previous version, Do you really want= , now, to open it again? =20 =20 > Neil