From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by dpdk.org (Postfix) with ESMTP id BAAD61B01C for ; Thu, 25 Jan 2018 11:56:00 +0100 (CET) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 3AB88211E7; Thu, 25 Jan 2018 05:56:00 -0500 (EST) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Thu, 25 Jan 2018 05:56:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=monjalon.net; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=mesmtp; bh=HI2v4rm2T8ZsLox+Iax08p3UCS rBjPYi/x2zG9nSNzc=; b=c86tT6lupbDXNAvfFKhGFteftpfDSjNkyyL6HJorsk INoKjRCaA6yob9Htf6t/BjTQq8P2Hx6KbfsS0Bu1Q5E8PBhLUOtwUnKQY0F+lsU1 P00g505kFu6xWM636m/jv65h/tNA8rKef157NPTKt8Lto9SfmyrYxFlKqAZC+78G s= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=HI2v4r m2T8ZsLox+Iax08p3UCSrBjPYi/x2zG9nSNzc=; b=LCNJYYAzusv5D3q1tGCr1b RBZKA4jGCU9D0GN5ODV3wIuiXGXqrYPU6a1r9TwrjrOcowNCTFGlKUOxzBbPZzlN 7xS7JhvejMKG3Q9BFG2eVLyXoEnARe4yuKEyrXbMTEZgK/tT3Jbe88sJAkIM63pG Tz9NU7SLBkusnFkbm8pfwC0BVhwqa4JtzmgRehlgjk51n/sx6IaqHpVTPmW7ZbD7 l0GQKJrbtBjopqYxpm9gAB50I4NJ6NRrJiXa5rQL9aTcrJ6ChI1i5Tot0UWFcU8m U350Ltkf1PAIH0o0dTGWcil7bw7iemtFDjYJs1EYY33Jv4j2C5okEr7dFKX5i8VQ == X-ME-Sender: Received: from xps.localnet (184.203.134.77.rev.sfr.net [77.134.203.184]) by mail.messagingengine.com (Postfix) with ESMTPA id D0DA37E12E; Thu, 25 Jan 2018 05:55:59 -0500 (EST) From: Thomas Monjalon To: "Ananyev, Konstantin" Cc: Matan Azrad , =?ISO-8859-1?Q?Ga=EBtan?= Rivet , "Wu, Jingjing" , dev@dpdk.org, Neil Horman , "Richardson, Bruce" Date: Thu, 25 Jan 2018 11:55:18 +0100 Message-ID: <42152390.b8nyHhbJZJ@xps> In-Reply-To: <2601191342CEEE43887BDE71AB9772588628308D@irsmsx105.ger.corp.intel.com> References: <1515318351-4756-1-git-send-email-matan@mellanox.com> <1682675.gfxK9aSxec@xps> <2601191342CEEE43887BDE71AB9772588628308D@irsmsx105.ger.corp.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Subject: Re: [dpdk-dev] [PATCH v3 7/7] app/testpmd: adjust ethdev port ownership X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jan 2018 10:56:01 -0000 24/01/2018 19:30, Ananyev, Konstantin: > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > 23/01/2018 22:18, Ananyev, Konstantin: > > > > > > > > 23/01/2018 16:18, Ananyev, Konstantin: > > > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ananyev, Konstantin > > > > > > From: Thomas Monjalon [mailto:thomas@monjalon.net] > > > > > > > 23/01/2018 14:34, Ananyev, Konstantin: > > > > > > > > If that' s the use case, then I think you need to set device ownership at creation time - > > > > > > > > inside dev_allocate(). > > > > > > > > Again that would avoid such racing conditions inside testpmd. > > > > > > > > > > > > > > The devices must be allocated at a low level layer. > > > > > > > > > > > > No one arguing about that. > > > > > > But we can provide owner id information to the low level. > > > > > > > > Sorry, you did not get it. > > > > > > Might be. > > > > > > > We cannot provide owner id at the low level > > > > because it is not yet decided who will be the owner > > > > before the port is allocated. > > > > > > Why is that? > > > What prevents us decide who will manage that device *before* allocating port of it? > > > IMO we do have all needed information at that stage. > > > > We don't have the information. > > At that point we do have dev name and all parameters, right? We just have the PCI id. > Plus we do have blacklist/whitelist, etc. > So what else are we missing to make the decision at that point? It depends on the ownership policy. Example: we can decide to take ownership based on a MAC address. Another example: it can be decided to take ownership of a given driver. We don't have these informations with the PCI id. > > It is a new device, it is matched by a driver which allocates a port. > > I don't see where the higher level can interact here. > > And even if you manage a trick, the higher level needs to read the port > > informations to decide the ownership. > > Could you specify what particular port information it needs? Replied to the same question above :) > > > > > > > When a new device appears (hotplug), an ethdev port should be allocated > > > > > > > automatically if it passes the whitelist/blacklist policy test. > > > > > > > Then we must decide who will manage this device. > > > > > > > I suggest notifying the DPDK libs first. > > > > > > > So a DPDK lib or PMD like failsafe can have the priority to take the > > > > > > > ownership in its notification callback. > > > > > > > > > > > > Possible, but seems a bit overcomplicated. > > > > > > Why not just: > > > > > > > > > > > > Have a global variable process_default_owner_id, that would be init once at startup. > > > > > > Have an LTS variable default_owner_id. > > > > > > It will be used by rte_eth_dev_allocate() caller can set dev->owner_id at creation time, > > > > > > so port allocation and setting ownership - will be an atomic operation. > > > > > > At the exit rte_eth_dev_allocate() will always reset default_owner_id=0: > > > > > > > > > > > > rte_eth_dev_allocate(...) > > > > > > { > > > > > > lock(owner_lock); > > > > > > > > > > > > owner = RTE_PER_LCORE(default_owner_id); > > > > > > if (owner == 0) > > > > > > owner = process_default_owner_id; > > > > > > set_owner(port, ..., owner); > > > > > > unlock(owner_lock); > > > > > > RTE_PER_LCORE(default_owner_id) = 0; > > > > > > > > > > Or probably better to leave default_owner_id reset to the caller. > > > > > Another thing - we can use same LTS variable in all control ops to > > > > > allow/disallow changing of port configuration based on ownership. > > > > > Konstantin > > > > > > > > > > > } > > > > > > > > > > > > So callers who don't need any special ownership - don't need to do anything. > > > > > > Special callers (like failsafe) can set default_owenr_id just before calling hotplug > > > > > > handling routine. > > > > > > > > No, hotplug will not be a routine. > > > > I am talking about real hotplug, like a device which appears in the VM. > > > > This new device must be handled by EAL and probed automatically if > > > > comply with whitelist/blacklist policy given by the application or user. > > > > Real hotplug is asynchronous. > > > > > > By 'asynchronous' here you mean it would be handled in the EAL interrupt thread > > > or something different? > > > > Yes, we receive an hotplug event which is processed in the event thread. > > > > > Anyway, I suppose you do need a function inside DPDK that will do the actual work in response > > > on hotplug event, right? > > > > Yes > > Ok, btw why that function has to be always called from interrupt thread? > Why not to allow user to decide? Absolutely, the user must decide. In the example of failsafe, the user instructs a policy to decide which devices will be owned, so failsafe takes the decision based on user inputs. > Some apps have their own thread dedicated for control ops (like testpmd) > For them it might be plausible to call that function from their own control thread context. > Konstantin > > > > > > That's what I refer to as 'hotplug routine' above. > > > > > > > We will just receive notifications that it appeared. > > > > > > > > Note: there is some temporary code in failsafe to manage some hotplug. > > > > This code must be removed when it will be properly handled in EAL. > > > > > > Ok, if it is just a temporary code, that would be removed soon - > > > then it definitely seems wrong to modify tespmd (or any other user app) > > > to comply with that temporary solution. > > > > It will be modified when EAL hotplug will be implemented. > > > > However, the ownership issue will be the same: > > we don't know the owner when allocating a port.