From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by dpdk.org (Postfix) with ESMTP id 7D54C71B5 for ; Mon, 15 Jan 2018 12:45:07 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 15 Jan 2018 03:45:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,363,1511856000"; d="scan'208";a="195594660" Received: from irsmsx110.ger.corp.intel.com ([163.33.3.25]) by fmsmga006.fm.intel.com with ESMTP; 15 Jan 2018 03:45:04 -0800 Received: from irsmsx105.ger.corp.intel.com ([169.254.7.236]) by irsmsx110.ger.corp.intel.com ([163.33.3.25]) with mapi id 14.03.0319.002; Mon, 15 Jan 2018 11:45:04 +0000 From: "Ananyev, Konstantin" To: Matan Azrad , Thomas Monjalon , Gaetan Rivet , "Wu, Jingjing" CC: "dev@dpdk.org" , Neil Horman , "Richardson, Bruce" Thread-Topic: [PATCH v2 2/6] ethdev: add port ownership Thread-Index: AQHTh5xreoY7ZC2VXUKPLFrmK4AmzKNtEwCwgABGVwCAATRvEIAAOkGAgACPrvCAAIW5gIAE/GJg Date: Mon, 15 Jan 2018 11:45:03 +0000 Message-ID: <2601191342CEEE43887BDE71AB9772588627DC25@irsmsx105.ger.corp.intel.com> References: <1511870281-15282-1-git-send-email-matan@mellanox.com> <1515318351-4756-1-git-send-email-matan@mellanox.com> <1515318351-4756-3-git-send-email-matan@mellanox.com> <2601191342CEEE43887BDE71AB97725880E3B9D6@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627B12A@irsmsx105.ger.corp.intel.com> <2601191342CEEE43887BDE71AB9772588627CCB0@irsmsx105.ger.corp.intel.com> In-Reply-To: Accept-Language: en-IE, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiZDcyOTk3ZTctZTE0Mi00OTMyLWE2ZWMtMDc3Y2NkOWJmYTJiIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX05UIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE2LjUuOS4zIiwiVHJ1c3RlZExhYmVsSGFzaCI6Ik1UQVJCa2RNRjFDZTRieXl2VFM0ZFYzYTRSYU1LcGdEMzdzcVwva2NZUVk4PSJ9 x-ctpclassification: CTP_NT dlp-product: dlpe-windows dlp-version: 11.0.0.116 dlp-reaction: no-action x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH v2 2/6] ethdev: add port ownership X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Jan 2018 11:45:08 -0000 Hi Matan, >=20 >=20 > Hi Konstantin >=20 > From: Ananyev, Konstantin, Friday, January 12, 2018 2:02 AM > > Hi Matan, > > > > > > > > Hi Konstantin > > > > > > From: Ananyev, Konstantin, Thursday, January 11, 2018 2:40 PM > > > > Hi Matan, > > > > > > > > > > > > > > Hi Konstantin > > > > > > > > > > From: Ananyev, Konstantin, Wednesday, January 10, 2018 3:36 PM > > > > > > Hi Matan, > > > > > > > Few comments from me below. > > > > > > BTW, do you plan to add ownership mandatory check in control > > > > > > path functions that change port configuration? > > > > > > > > > > No. > > > > > > > > So it still totally voluntary usage and application nneds to be > > > > changed to exploit it? > > > > Apart from RTE_FOR_EACH_DEV() change proposed by Gaetan? > > > > > > > > > > Also RTE_FOR_EACH_DEV() change proposed by Gaetan is not protected > > because 2 DPDK entities can get the same port while using it. > > > > I am not talking about racing condition here. > > Right now even from the same thread - I can call dev_configure() for th= e port > > which I don't own (let say it belongs to failsafe port), and that would= remain, > > correct? > > > Yes. Ok, thanks for clarification. I think that makes current approach sort of incomplete, but might be it is = a=20 subject of separate discussion. >=20 > > > As I wrote in the log\docs and as discussed a lot in the first versio= n: > > > The new synchronization rules are: > > > 1. The port allocation and port release synchronization will be > > > managed by ethdev. > > > 2. The port usage synchronization will be managed by the port owner. > > > 3. The port ownership API synchronization(also with port creation) wi= ll be > > managed by ethdev. > > > 4. DPDK entity which want to use a port must take ownership before. > > > > > > Ethdev should not protect 2 and 4 according these rules. > > > > > > > > > Konstantin > > > > > > > > > > > > > -----Original Message----- > > > > > > > From: Matan Azrad [mailto:matan@mellanox.com] > > > > I mean the documentation about the needed alignment for spinlock. > > Where is it? > > > > https://emea01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Finf= o > > center.arm.com%2Fhelp%2Findex.jsp%3Ftopic%3D%2Fcom.arm.doc.faqs%2 > > Fka15414.html&data=3D02%7C01%7Cmatan%40mellanox.com%7Cb3c329ae9db > > f4bd29a7008d5594fb776%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C1 > > %7C636513121294703050&sdata=3D40v3b4wk5f4qEyIY5jdDv8S47LjgXK0t9TPtav > > XIMOk%3D&reserved=3D0 > > https://emea01.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%2Finf= o > > center.arm.com%2Fhelp%2Findex.jsp%3Ftopic%3D%2Fcom.arm.doc.dht000 > > 8a%2FCJAGCFAF.html&data=3D02%7C01%7Cmatan%40mellanox.com%7Cb3c32 > > 9ae9dbf4bd29a7008d5594fb776%7Ca652971c7d2e4d9ba6a4d149256f461b%7 > > C0%7C1%7C636513121294703050&sdata=3DB7pEZjFJntVp3Il8fS9wr%2FlxABgNX > > FSr9PE4emEPLQE%3D&reserved=3D0 > > > > Might be ARM and PPC guys can provide you some more complete/recent > > docs. > Thanks. > > > > > > > It is good to see that now scanning/updating rte_eth_dev_data[] > > > > > > is lock protected, but it might be not very plausible to protec= t > > > > > > both data[] and next_owner_id using the same lock. > > > > > > > > > > I guess you mean to the owner structure in rte_eth_dev_data[port_= id]. > > > > > The next_owner_id is read by ownership APIs(for owner validation)= , > > > > > so it > > > > makes sense to use the same lock. > > > > > Actually, why not? > > > > > > > > Well to me next_owner_id and rte_eth_dev_data[] are not directly > > related. > > > > You may create new owner_id but it doesn't mean you would update > > > > rte_eth_dev_data[] immediately. > > > > And visa-versa - you might just want to update > > > > rte_eth_dev_data[].name or .owner_id. > > > > It is not very good coding practice to use same lock for non-relate= d > > > > data structures. > > > > > > > I see the relation like next: > > > Since the ownership mechanism synchronization is in ethdev > > > responsibility, we must protect against user mistakes as much as we c= an by > > using the same lock. > > > So, if user try to set by invalid owner (exactly the ID which current= ly is > > allocated) we can protect on it. > > > > Hmm, not sure why you can't do same checking with different lock or ato= mic > > variable? > > > The set ownership API is protected by ownership lock and checks the owner= ID validity > By reading the next owner ID. > So, the owner ID allocation and set API should use the same atomic mechan= ism. Sure but all you are doing for checking validity, is check that=20 owner_id > 0 &&& owner_id < next_ownwe_id, right? As you don't allow owner_id overlap (16/3248 bits) you can safely do same c= heck with just atomic_get(&next_owner_id).=20 > The set(and others) ownership APIs already uses the ownership lock so I t= hink it makes sense to use the same lock also in ID allocation. >=20 > > > > > > In fact, for next_owner_id, you don't need a lock - just > > > > > > rte_atomic_t should be enough. > > > > > > > > > > I don't think so, it is problematic in next_owner_id wraparound > > > > > and may > > > > complicate the code in other places which read it. > > > > > > > > IMO it is not that complicated, something like that should work I t= hink. > > > > > > > > /* init to 0 at startup*/ > > > > rte_atomic32_t *owner_id; > > > > > > > > int new_owner_id(void) > > > > { > > > > int32_t x; > > > > x =3D rte_atomic32_add_return(&owner_id, 1); > > > > if (x > UINT16_MAX) { > > > > rte_atomic32_dec(&owner_id); > > > > return -EOVERWLOW; > > > > } else > > > > return x; > > > > } > > > > > > > > > > > > > Why not just to keep it simple and using the same lock? > > > > > > > > Lock is also fine, I just think it better be a separate one - that > > > > would protext just next_owner_id. > > > > Though if you are going to use uuid here - all that probably not > > > > relevant any more. > > > > > > > > > > I agree about the uuid but still think the same lock should be used f= or both. > > > > But with uuid you don't need next_owner_id at all, right? > > So lock will only be used for rte_eth_dev_data[] fields anyway. > > > Sorry, I meant uint64_t, not uuid. Ah ok, my thought uuid_t is better as with it you don't need to support you= r own code to allocate new owner_id, but rely on system libs instead. But wouldn't insist here. >=20 > > > > > > Another alternative would be to use 2 locks - one for > > > > > > next_owner_id second for actual data[] protection. > > > > > > > > > > > > Another thing - you'll probably need to grab/release a lock > > > > > > inside > > > > > > rte_eth_dev_allocated() too. > > > > > > It is a public function used by drivers, so need to be protecte= d too. > > > > > > > > > > > > > > > > Yes, I thought about it, but decided not to use lock in next: > > > > > rte_eth_dev_allocated > > > > > rte_eth_dev_count > > > > > rte_eth_dev_get_name_by_port > > > > > rte_eth_dev_get_port_by_name > > > > > maybe more... > > > > > > > > As I can see in patch #3 you protect by lock access to > > > > rte_eth_dev_data[].name (which seems like a good thing). > > > > So I think any other public function that access > > > > rte_eth_dev_data[].name should be protected by the same lock. > > > > > > > > > > I don't think so, I can understand to use the ownership lock here(as = in port > > creation) but I don't think it is necessary too. > > > What are we exactly protecting here? > > > Don't you think it is just timing?(ask in the next moment and you ma= y > > > get another answer) I don't see optional crash. > > > > Not sure what you mean here by timing... > > As I understand rte_eth_dev_data[].name unique identifies device and is > > used by port allocation/release/find functions. > > As you stated above: > > "1. The port allocation and port release synchronization will be manag= ed by > > ethdev." > > To me it means that ethdev layer has to make sure that all accesses to > > rte_eth_dev_data[].name are atomic. > > Otherwise what would prevent the situation when one process does > > rte_eth_dev_allocate()->snprintf(rte_eth_dev_data[x].name, ...) while > > second one does rte_eth_dev_allocated(rte_eth_dev_data[x].name, ...) ? > > > The second will get True or False and that is it. Under race condition - in the worst case it might crash, though for that yo= u'll have to be really unlucky. Though in most cases as you said it would just not operate correctly. I think if we start to protect dev->name by lock we need to do it for all i= nstances (both read and write). =20 > Maybe if it had been called just a moment after, It might get different a= nswer. > Because these APIs don't change ethdev structure(just read), it can be OK= . > But again, I can understand to use ownership lock also here. >=20 Konstantin