From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 228681B04C for ; Wed, 17 Jan 2018 18:01:14 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 17 Jan 2018 09:01:14 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,372,1511856000"; d="scan'208";a="10998334" Received: from irsmsx153.ger.corp.intel.com ([163.33.192.75]) by fmsmga007.fm.intel.com with ESMTP; 17 Jan 2018 09:01:12 -0800 Received: from irsmsx155.ger.corp.intel.com (163.33.192.3) by IRSMSX153.ger.corp.intel.com (163.33.192.75) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 17 Jan 2018 17:01:11 +0000 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.236]) by irsmsx155.ger.corp.intel.com ([169.254.14.169]) with mapi id 14.03.0319.002; Wed, 17 Jan 2018 17:01:11 +0000 From: "Ananyev, Konstantin" To: Neil Horman , Matan Azrad CC: Thomas Monjalon , Gaetan Rivet , "Wu, Jingjing" , "dev@dpdk.org" , "Richardson, Bruce" Thread-Topic: [PATCH v2 2/6] ethdev: add port ownership Thread-Index: AQHTh5xreoY7ZC2VXUKPLFrmK4AmzKNtEwCwgABGVwCAATRvEIAAOkGAgACPrvCAAIW5gIAE/GJggAAbIoCAAD5+YIAA/qMAgACx27CAAB8nAIAA79jQgAAU2ACAACAHAIAAMFhg Date: Wed, 17 Jan 2018 17:01:10 +0000 Message-ID: <2601191342CEEE43887BDE71AB9772588627F0E9@irsmsx105.ger.corp.intel.com> References: <2601191342CEEE43887BDE71AB9772588627CCB0@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627DC25@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627DE30@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627E954@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627EE60@irsmsx105.ger.corp.intel.com> <20180117140020.GA5432@hmswarspite.think-freely.org> In-Reply-To: <20180117140020.GA5432@hmswarspite.think-freely.org> Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiOWQwMTA2M2QtMTk4My00MmQ1LThlOGEtNjhmYTgxMTE1YTcyIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6Ik9yM0ZiTDdZRThDYml6TG9OOXRxbVI0bnZNMCtLTmZnWStpZldZVmJFcmc9In0= x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.181] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 2/6] ethdev: add port ownership X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Jan 2018 17:01:15 -0000 > -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: Wednesday, January 17, 2018 2:00 PM > To: Matan Azrad > Cc: Ananyev, Konstantin ; Thomas Monjalon <= thomas@monjalon.net>; Gaetan Rivet > ; Wu, Jingjing ; dev@dpdk.= org; Richardson, Bruce > Subject: Re: [PATCH v2 2/6] ethdev: add port ownership >=20 > On Wed, Jan 17, 2018 at 12:05:42PM +0000, Matan Azrad wrote: > > > > Hi Konstantin > > From: Ananyev, Konstantin, Sent: Wednesday, January 17, 2018 1:24 PM > > > Hi Matan, > > > > > > > Hi Konstantin > > > > > > > > From: Ananyev, Konstantin, Tuesday, January 16, 2018 9:11 PM > > > > > Hi Matan, > > > > > > > > > > > > > > > > > Hi Konstantin > > > > > > From: Ananyev, Konstantin, Monday, January 15, 2018 8:44 PM > > > > > > > Hi Matan, > > > > > > > > Hi Konstantin > > > > > > > > From: Ananyev, Konstantin, Monday, January 15, 2018 1:45 PM > > > > > > > > > Hi Matan, > > > > > > > > > > Hi Konstantin > > > > > > > > > > From: Ananyev, Konstantin, Friday, January 12, 2018 2:0= 2 > > > > > > > > > > AM > > > > > > > > > > > Hi Matan, > > > > > > > > > > > > Hi Konstantin > > > > > > > > > > > > From: Ananyev, Konstantin, Thursday, January 11, 20= 18 > > > > > > > > > > > > 2:40 PM > > > > > > > > > > > > > Hi Matan, > > > > > > > > > > > > > > Hi Konstantin > > > > > > > > > > > > > > From: Ananyev, Konstantin, Wednesday, January 1= 0, > > > > > > > > > > > > > > 2018 > > > > > > > > > > > > > > 3:36 PM > > > > > > > > > > > > > > > Hi Matan, > > > > > > > > > > > > > > > > > > > > > It is good to see that now scanning/updating > > > > > > > > > > > > > > > rte_eth_dev_data[] is lock protected, but it > > > > > > > > > > > > > > > might be not very plausible to protect both > > > > > > > > > > > > > > > data[] and next_owner_id using the > > > > > > > > > same lock. > > > > > > > > > > > > > > > > > > > > > > > > > > > > I guess you mean to the owner structure in > > > > > > > > > rte_eth_dev_data[port_id]. > > > > > > > > > > > > > > The next_owner_id is read by ownership APIs(for > > > > > > > > > > > > > > owner validation), so it > > > > > > > > > > > > > makes sense to use the same lock. > > > > > > > > > > > > > > Actually, why not? > > > > > > > > > > > > > > > > > > > > > > > > > > Well to me next_owner_id and rte_eth_dev_data[] a= re > > > > > > > > > > > > > not directly > > > > > > > > > > > related. > > > > > > > > > > > > > You may create new owner_id but it doesn't mean y= ou > > > > > > > > > > > > > would update rte_eth_dev_data[] immediately. > > > > > > > > > > > > > And visa-versa - you might just want to update > > > > > > > > > > > > > rte_eth_dev_data[].name or .owner_id. > > > > > > > > > > > > > It is not very good coding practice to use same l= ock > > > > > > > > > > > > > for non-related data structures. > > > > > > > > > > > > > > > > > > > > > > > > > I see the relation like next: > > > > > > > > > > > > Since the ownership mechanism synchronization is in > > > > > > > > > > > > ethdev responsibility, we must protect against user > > > > > > > > > > > > mistakes as much as we can by > > > > > > > > > > > using the same lock. > > > > > > > > > > > > So, if user try to set by invalid owner (exactly th= e > > > > > > > > > > > > ID which currently is > > > > > > > > > > > allocated) we can protect on it. > > > > > > > > > > > > > > > > > > > > > > Hmm, not sure why you can't do same checking with > > > > > > > > > > > different lock or atomic variable? > > > > > > > > > > > > > > > > > > > > > The set ownership API is protected by ownership lock an= d > > > > > > > > > > checks the owner ID validity By reading the next owner = ID. > > > > > > > > > > So, the owner ID allocation and set API should use the > > > > > > > > > > same atomic > > > > > > > > > mechanism. > > > > > > > > > > > > > > > > > > Sure but all you are doing for checking validity, is che= ck > > > > > > > > > that owner_id > 0 &&& owner_id < next_ownwe_id, right? > > > > > > > > > As you don't allow owner_id overlap (16/3248 bits) you ca= n > > > > > > > > > safely do same check with just atomic_get(&next_owner_id)= . > > > > > > > > > > > > > > > > > It will not protect it, scenario: > > > > > > > > - current next_id is X. > > > > > > > > - call set ownership of port A with owner id X by thread 0(= by > > > > > > > > user > > > > > mistake). > > > > > > > > - context switch > > > > > > > > - allocate new id by thread 1 and get X and change next_id = to > > > > > > > > X+1 > > > > > > > atomically. > > > > > > > > - context switch > > > > > > > > - Thread 0 validate X by atomic_read and succeed to take > > > ownership. > > > > > > > > - The system loosed the port(or will be managed by two > > > > > > > > entities) - > > > > > crash. > > > > > > > > > > > > > > > > > > > > > Ok, and how using lock will protect you with such scenario? > > > > > > > > > > > > The owner set API validation by thread 0 should fail because th= e > > > > > > owner > > > > > validation is included in the protected section. > > > > > > > > > > Then your validation function would fail even if you'll use atomi= c > > > > > ops instead of lock. > > > > No. > > > > With atomic this specific scenario will cause the validation to pas= s. > > > > > > Can you explain to me how? > > > > > > rte_eth_is_valid_owner_id(uint16_t owner_id) { > > > int32_t cur_owner_id =3D RTE_MIN(rte_atomic32_get(next_= owner_id), > > > UINT16_MAX); > > > > > > if (owner_id =3D=3D RTE_ETH_DEV_NO_OWNER || owner > > > > cur_owner_id) { > > > RTE_LOG(ERR, EAL, "Invalid owner_id=3D%d.\n", owner_id); > > > return 0; > > > } > > > return 1; > > > } > > > > > > Let say your next_owne_id=3D=3DX, and you invoke > > > rte_eth_is_valid_owner_id(owner_id=3DX+1) - it would fail. > > > > Explanation: > > The scenario with locks: > > next_owner_id =3D X. > > Thread 0 call to set API(with invalid owner Y=3DX) and take lock. > > Context switch. > > Thread 1 call to owner_new and stuck in the lock. > > Context switch. > > Thread 0 does owner id validation and failed(Y>=3DX) - unlock the lock = and return failure to the user. > > Context switch. > > Thread 1 take the lock and update X to X+1, then, unlock the lock. > > Everything is OK! > > > > The same scenario with atomics: > > next_owner_id =3D X. > > Thread 0 call to set API(with invalid owner Y=3DX) and take lock. > > Context switch. > > Thread 1 call to owner_new and change X to X+1(atomically). > > Context switch. > > Thread 0 does owner id validation and success(Y<(atomic)X+1) - unlock t= he lock and return success to the user. > > Problem! > > >=20 >=20 > Matan is correct here, there is no way to preform parallel set operations= using > just and atomic variable here, because multiple reads of next_owner_id ne= ed to > be preformed while it is stable. That is to say rte_eth_next_owner_id mu= st be > compared to RTE_ETH_DEV_NO_OWNER and owner_id in rte_eth_is_valid_owner_i= d. If > you were to only use an atomic_read on such a variable, it could be incre= mented > by the owner_new function between the checks and an invalid owner value c= ould > become valid because a third thread incremented the next value. The sta= te of > next_owner_id must be kept stable during any validity checks It could still be incremented between the checks - if let say different thr= ead will invoke new_onwer_id, grab the lock update counter, release the lock - all t= hat before the check. But ok, there is probably no point to argue on that one any longer - let's keep the lock here, nothing will be broken with it for sure. >=20 > That said, I really have to wonder why ownership ids are really needed he= re at > all. It seems this design could be much simpler with the addition of a p= er-port > lock (and optional ownership record). The API could consist of three > operations: >=20 > ownership_set > ownership_tryset > ownership_release > ownership_get >=20 Ok, but how to distinguish who is the current owner of the port? To make sure that only owner is allowed to perform control ops? Konstantin >=20 > The first call simply tries to take the per-port lock (blocking if its al= ready > locked) >=20 > The second call is a non-blocking version of the first >=20 > The third unlocks the port, allowing others to take ownership >=20 > The fourth returns whatever ownership record you want to encode with the = lock. >=20 > The addition of all this id checking seems a bit overcomplicated >=20 > Neil