From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0049.outbound.protection.outlook.com [104.47.0.49]) by dpdk.org (Postfix) with ESMTP id B9FE91B2D5 for ; Thu, 21 Dec 2017 20:37:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=ubJOxvoxfNAZSd3KW2yTC+VxP28eRUiM7z/N50z1aXs=; b=PoK5o/1D6UR1mtLaOnSlEqGWPzesJ8t4tZO6+5Cjem14n7qPC7H/YMOY5fQIYXTGBl+lHSiACS5gJydkxRwWEJiG9HaZgTa9EfM4DhIp/9bdWiNnuv2c+GIDEumCXxQogEyVZzjWw5xEuLVdDNw+WpQ/FmvnyDcY+ZJkAF+2x8w= Received: from HE1PR0502MB3659.eurprd05.prod.outlook.com (10.167.127.17) by HE1PR0502MB3659.eurprd05.prod.outlook.com (10.167.127.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.345.14; Thu, 21 Dec 2017 19:37:07 +0000 Received: from HE1PR0502MB3659.eurprd05.prod.outlook.com ([fe80::adf6:87de:2761:8e4]) by HE1PR0502MB3659.eurprd05.prod.outlook.com ([fe80::adf6:87de:2761:8e4%13]) with mapi id 15.20.0345.013; Thu, 21 Dec 2017 19:37:07 +0000 From: Matan Azrad To: Neil Horman , Thomas Monjalon CC: "dev@dpdk.org" , Bruce Richardson , "Ananyev, Konstantin" , =?iso-8859-1?Q?Ga=EBtan_Rivet?= , "Wu, Jingjing" Thread-Topic: [dpdk-dev] [PATCH 2/5] ethdev: add port ownership Thread-Index: AQHTaEBDeXaioloFskiHncvIK1YNTKMs3xqAgAANj4CAAX1kAIAC1eJggAA+JgCAABoCUIAByaUAgAAcouCAAFA1gIAAeA4QgABKFgCABNAHAIAAD8IAgBS7KwCAAAoiAIAADZrA Date: Thu, 21 Dec 2017 19:37:06 +0000 Message-ID: References: <20171130123611.GA20914@hmswarspite.think-freely.org> <5212147.QN8ImyqEg2@xps> <20171208123142.GA6955@hmswarspite.think-freely.org> <1567916.dnd6Z652YM@xps> <20171221174304.GB23958@hmswarspite.think-freely.org> In-Reply-To: <20171221174304.GB23958@hmswarspite.think-freely.org> Accept-Language: en-US, he-IL Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=matan@mellanox.com; x-originating-ip: [85.64.136.190] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0502MB3659; 6:ZLxCzvKTDNTpdTnTh7n1QFuL6G6aVn+nB/hk7ssNViTcnFJvHo2+pID5UapRCP1jsYEq9R2BLfaPUVzmPfD5Bl1HiakVZJRIoo+7QHU6oUS4rVFKMMWZQf6Fv/OC7qgsiAX9xegBasVgzYB0A/6SjfZBd1Gvnmmf/EhrWr3kfQQMj1JDhDowXN0L9kV4TWeF7vQPRjGupMFrk6/o0oCBobC8c6Sub7ulM4WpiQU0tkaRGYh0EoNx0KFs4dDw6qiFGgQIsL9x3E67dhbCm3bX9xzL3wjnGVsJredFStqZiTEzmLGJkD2tOLHCcjsGxc+BxXg+EfHReAx/o2KjJlTqEcInMTSNt7vX9Ekm67THY3o=; 5:V6sUXypd05DZfLconNqpS0A1PS/L4csYwJQDbRrGEhEqFm0HuxuTFcjsJvukboz6wZDPh6qAEwC/hP4dYj/e8aSt5/Sd5FWI8oM2ZwA4y8EltX8yb80QUSWq38Keazc3Ztpd3sA6yBekSNCK7MBsYV+8uklAfyrRDvemC4NrDb4=; 24:jH2iIeh/wf1YJ5CUAeXnhq8Ziw4U5E26htPhfNl2R1XM9TFznlCJt0wPBeepyfEaYK33Iy0OSKspIg0YHer23TjZkQ1euc19kIU2/ba/zQU=; 7:A+N43KOoeZXBCYFzhZvE9lyollCILxMf8Bp3L4LJMqjrAwuULH0klibl/foTbwkaIXgboHBKeyOx34oF6/DuQltliBMSuVeqtGqnWjMOItY4YSgsMJDluLpy5cS6cWHiEMDPHoPIBO1TEJEZftydTiXL13b2k6JT16qlyiky2KDtSxCCP6+j8Kfx8D7Dyo4QpD4AUdZlddpOB9z67m9fnpR9eE3fXh5/D/kdisIAW9suNb7gAzb+07nvyYzqpIhZ x-ms-exchange-antispam-srfa-diagnostics: SSOS; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 1ce1fa5e-7563-4413-6d5f-08d548aa3871 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(48565401081)(5600026)(4604075)(3008031)(2017052603307)(7153060); SRVR:HE1PR0502MB3659; x-ms-traffictypediagnostic: HE1PR0502MB3659: x-ld-processed: a652971c-7d2e-4d9b-a6a4-d149256f461b,ExtAddr x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(60795455431006)(278428928389397)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(8121501046)(5005006)(10201501046)(3002001)(3231023)(93006095)(93001095)(6055026)(6041268)(20161123564045)(20161123562045)(20161123560045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:HE1PR0502MB3659; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:HE1PR0502MB3659; x-forefront-prvs: 0528942FD8 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(346002)(366004)(376002)(39860400002)(39380400002)(13464003)(76104003)(189003)(199004)(24454002)(55016002)(9686003)(6436002)(5660300001)(6116002)(14454004)(3846002)(93886005)(74316002)(97736004)(305945005)(316002)(54906003)(110136005)(7736002)(86362001)(478600001)(6246003)(66066001)(4326008)(6506007)(76176011)(53546011)(102836004)(59450400001)(5250100002)(7696005)(106356001)(33656002)(105586002)(99286004)(25786009)(2900100001)(2906002)(81166006)(229853002)(81156014)(8936002)(8676002)(2950100002)(53936002)(3660700001)(3280700002)(68736007); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0502MB3659; H:HE1PR0502MB3659.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1ce1fa5e-7563-4413-6d5f-08d548aa3871 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Dec 2017 19:37:06.9047 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0502MB3659 Subject: Re: [dpdk-dev] [PATCH 2/5] ethdev: add port ownership X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Dec 2017 19:37:10 -0000 Hi > -----Original Message----- > From: Neil Horman [mailto:nhorman@tuxdriver.com] > Sent: Thursday, December 21, 2017 7:43 PM > To: Thomas Monjalon > Cc: dev@dpdk.org; Bruce Richardson ; Matan > Azrad ; Ananyev, Konstantin > ; Ga=EBtan Rivet ; > Wu, Jingjing > Subject: Re: [dpdk-dev] [PATCH 2/5] ethdev: add port ownership >=20 > On Thu, Dec 21, 2017 at 06:06:48PM +0100, Thomas Monjalon wrote: > > 08/12/2017 13:31, Neil Horman: > > > On Fri, Dec 08, 2017 at 12:35:18PM +0100, Thomas Monjalon wrote: > > > > 05/12/2017 11:05, Bruce Richardson: > > > > > > I think you suggest to make all the ethdev configuration race > > > > > > safe, it is behind to this thread. Current ethdev > > > > > > implementation leave the race management to applications, so > > > > > > port ownership as any other port configurations should be desig= ned > in the same method. > > > > > > > > > > One key difference, though, being that port ownership itself > > > > > could be used to manage the thread-safety of the ethdev > > > > > configuration. It's also a little different from other APIs in > > > > > that I find it hard to come up with a scenario where it would be > > > > > very useful to an application without also having some form of > > > > > locking present in it. For other config/control APIs we can have > > > > > the control plane APIs work without locks e.g. by having a > > > > > single designated thread/process manage all configuration > > > > > updates. However, as Neil points out, in such a scenario, the > > > > > ownership concept doesn't provide any additional benefit so can > > > > > be skipped completely. I'd view it a bit like the reference > > > > > counting of mbufs - we can provide a lockless/non-atomic version, > but for just about every real use-case, you want the atomic version. > > > > > > > > I think we need to clearly describe what is the tread-safety > > > > policy in DPDK (especially in ethdev as a first example). > > > > Let's start with obvious things: > > > > > > > > 1/ A queue is not protected for races with multiple Rx or Tx > > > > - no planned change because of performance > purpose > > > > 2/ The list of devices is racy > > > > - to be fixed with atomics > > > > 3/ The configuration of different devices is thread-safe > > > > - the configurations are different per-device > > > > 4/ The configuration of a given device is racy > > > > - can be managed by the owner of the device > > > > 5/ The device ownership is racy > > > > - to be fixed with atomics > > > > > > > > What am I missing? > > > > Thank you Thomas for this order. Actually the port ownership is a good opportunity to redefine the synchroni= zation rules in ethdev :) > > > There is fan out to consider here: > > > > > > 1) Is device configuration racy with ownership? That is to say, can > > > I change ownership of a device safely while another thread that > > > currently owns it modifies its configuration? > > > > If an entity steals ownership to another one, either it is agreed > > earlier, or it is done by a central authority. > > When it is acked that ownership can be moved, there should not be any > > configuration in progress. > > So it is more a communication issue than a race. > > > But if thats the case (specifically that mutual exclusion between port > ownership and configuration is an exercize left to an application develop= er), > then port ownership itself is largely meaningless within the dpdk, becaus= e > the notion of who owns the port needs to be codified within the applicati= on > anyway. >=20 Bruce, As I understand it, only the dpdk entity who took ownership of a por= t successfully can configure the device by default, if other dpdk entities = want to configure it too they must to be synchronized with the port owner w= hile it is not recommended after the port ownership integration. So, for example, if the dpdk entity is an application, the application sho= uld take ownership of the port and manage the synchronization of this port = configuration between the application threads and its EAL host thread callb= acks, no other dpdk entity should configure the same port because they shou= ld fail when they try to take ownership of the same port too. Each dpdk entity which wants to take ownership must to be able to synchroni= ze the port configuration in its level.=20 >=20 > > > 2) Is device configuration racy with device addition/removal? That > > > is to say, can one thread remove a device while another configures it= . > > > > I think it is the same as two threads configuring the same device > > (item 4/ above). It can be managed with port ownership. > > > Only if you assert that application is required to have the owning port b= e > responsible for the ports deletion, which we can say, but that leads to t= he > issue above again. >=20 >=20 As Thomas said in item 2 the port creation must be synchronized by ethdev a= nd we need to add it there.=20 I think it is obvious that port removal must to be done only by the port ow= ner. =20 I think we need to add synchronization for port ownership management in thi= s patch V2 and add port creation synchronization in ethdev in separate patc= h to fill the new rules Thomas suggested. What do you think? > > > There are probably many subsystem interactions that need to be > addressed here. > > > > > > Neil > > > > > > > I am also wondering whether the device ownership can be a separate > > > > library used in several device class interfaces? > > > > > >