From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by dpdk.org (Postfix) with ESMTP id BD8579E7 for ; Tue, 27 Sep 2016 15:01:35 +0200 (CEST) Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga104.fm.intel.com with ESMTP; 27 Sep 2016 06:01:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.30,405,1470726000"; d="scan'208";a="13722628" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.220.48]) by fmsmga005.fm.intel.com with SMTP; 27 Sep 2016 06:01:22 -0700 Received: by (sSMTP sendmail emulation); Tue, 27 Sep 2016 14:01:21 +0025 Date: Tue, 27 Sep 2016 14:01:21 +0100 From: Bruce Richardson To: "Iremonger, Bernard" Cc: Thomas Monjalon , "dev@dpdk.org" , Jerin Jacob , "Shah, Rahul R" , "Lu, Wenzhuo" , azelezniak Message-ID: <20160927130121.GA34240@bricha3-MOBL3> References: <1471528125-26357-1-git-send-email-bernard.iremonger@intel.com> <8452736.eEWzj5BUlI@xps13> <8CEF83825BEC744B83065625E567D7C21A08D383@IRSMSX108.ger.corp.intel.com> <2212551.lClP9YTigq@xps13> <8CEF83825BEC744B83065625E567D7C21A08D615@IRSMSX108.ger.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8CEF83825BEC744B83065625E567D7C21A08D615@IRSMSX108.ger.corp.intel.com> Organization: Intel Research and =?iso-8859-1?Q?De=ACvel?= =?iso-8859-1?Q?opment?= Ireland Ltd. User-Agent: Mutt/1.5.23 (2014-03-12) Subject: Re: [dpdk-dev] [RFC PATCH v2 3/5] librte_ether: add API's for VF management X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Sep 2016 13:01:36 -0000 On Tue, Sep 27, 2016 at 11:31:06AM +0100, Iremonger, Bernard wrote: > Hi Thomas, Bruce, > > > > > Subject: Re: [dpdk-dev] [RFC PATCH v2 3/5] librte_ether: add API's for VF > > management > > > > 2016-09-26 15:37, Iremonger, Bernard: > > > Hi Thomas, Bruce, > > > > > > > > > > > > > Subject: Re: [dpdk-dev] [RFC PATCH v2 3/5] librte_ether: add API's > > > > for VF management > > > > > > > > 2016-09-23 17:02, Iremonger, Bernard: > > > > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > > > > > 2016-09-23 09:53, Richardson, Bruce: > > > > > > > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > > > > > > > > 2016-09-23 10:20, Bruce Richardson: > > > > > > > > > On Thu, Sep 22, 2016 at 07:04:37PM +0200, Thomas Monjalon > > wrote: > > > > > > > > > > 2016-09-15 16:46, Iremonger, Bernard: > > > > > > > > > > > > > > Do we really need to expose VF specific functions > > here? > > > > > > > > > > > > > > It can be generic(PF/VF) function indexed only > > > > > > > > > > > > > > through > > > > > > > > port_id. > > > > > > > > > > > > > > (example: as > > > > > > > > > > > > > > rte_eth_dev_set_vlan_anti_spoof(uint8_t > > > > > > > > > > > > > > port_id, uint8_t on)) For instance, In Thunderx > > > > > > > > > > > > > > PMD, We are not exposing a separate port_id for > > > > > > > > > > > > > > PF. We only enumerate 0..N VFs as 0..N ethdev > > > > > > > > > > > > > > port_id > > > > > > > > > > > > > > > > > > > > > > > > > > Our intention with this patch is to control the VF from the > > PF. > > > > > > > > > > > > > > > > > > > > > > > > > > The following librte_ether functions already work > > > > > > > > > > > > > in a similar > > > > > > > > way: > > > > > > > > > > > > > > > > > > > > > > > > > > rte_eth_dev_set_vf_rxmode(uint8_t port_id, > > > > > > > > > > > > > uint16_t vf, uint16_t rx_mode, uint8_t on) > > > > > > > > > > > > > > > > > > > > > > > > > > rte_eth_dev_set_vf_rx(uint8_t port_id, uint16_t > > > > > > > > > > > > > vf, uint8_t > > > > > > > > > > > > > on) > > > > > > > > > > > > > > > > > > > > > > > > > > rte_eth_dev_set_vf_tx(uint8_t port_id, uint16_t > > > > > > > > > > > > > vf, uint8_t > > > > > > > > > > > > > on) > > > > > > > > > > > > > > > > > > > > > > > > > > int rte_eth_set_vf_rate_limit(uint8_t port_id, > > > > > > > > > > > > > uint16_t vf, uint16_t tx_rate, uint64_t q_msk) > > > > > > > > > > > > > > > > > > > > > > > > I have a bad feeling with these functions dedicated > > > > > > > > > > > > to VF from > > > > PF. > > > > > > > > > > > > Are we sure there is no other way? > > > > > > > > > > > > I mean we just need to know the VF with a port ID. > > > > > > > > > > > > > > > > > > > > > > When the VF is used in a VM the port ID of the VF is > > > > > > > > > > > not visible to > > > > > > > > the PF. > > > > > > > > > > > I don't think there is another way to do this. > > > > > > > > > > > > > > > > > > > > I don't understand why we could not assign a port id to > > > > > > > > > > the VF from the host instead of having the couple PF port id / > > VF id. > > > > > > > > > > Can we enumerate all the VFs associated to a PF? > > > > > > > > > > Then can we allocate them a port id in the array > > rte_eth_devices? > > > > > > > > > > > > > > > > > > Hi Thomas, > > > > > > > > > > > > > > > > > > The VF is not a port visible to DPDK, though, so it > > > > > > > > > shouldn't have a port id IMHO. DPDK can't actually do anything > > with it. > > > > > > > > > > > > > > > > You say the contrary below. > > > > > > > > > > > > > > Well, yes and no. The driver can manipulate things for the VF, > > > > > > > but DPDK > > > > > > doesn't actually have a device that corresponds to the VF. There > > > > > > are no PCI bar mappings for it, DPDK can't do RX and TX with it etc.? > > > > > > > > > > > > Very good point. > > > > > > There are only few ethdev functions which are supported by every > > > > > > drivers, like Rx/Tx and would not be available for VF from PF > > interface. > > > > > > > > > > > > > > > The PCI device for the VF is likely passed through to a > > > > > > > > > different VM and being used there. Unfortunately, the VF > > > > > > > > > still needs certain things done for it by the PF, so if > > > > > > > > > the PF is under DPDK control, it needs to provide the > > > > > > > > > functionality to assist > > > > the VF. > > > > > > > > > > > > > > > > Why not have a VF_from_PF driver which does the mailbox > > things? > > > > > > > > So you can manage the VF from the PF with a simple port id. > > > > > > > > It really seems to be the cleanest design to me. > > > > > > > > > > > > > > While I see your point, and it could work, I just want to be > > > > > > > sure that we are > > > > > > ok with the results of that. Suppose we do create ethdevs for > > > > > > the VFs controlled by the PF. Does the new VF get counted in the > > > > > > rte_eth_dev_count() value (I assume yes)? How are apps meant to > > > > > > use the port? Do they have to put in a special case when > > > > > > iterating through all the port ids to check that it's not a > > > > > > pseudo port that can't do anything. None of the standard ethdev > > > > > > calls from an app will work on it, you can't configure nb rx/tx > > > > > > queues on it, you can't start or > > > > stop it, you can't do rx or tx on it, etc, etc. > > > > > > > > > > > > Yes these devices would be special because their supported API > > > > > > would be quite different. I was thinking that in the future you > > > > > > could add most of the configuration functions through the VF > > mailbox. > > > > > > But the Intel mailbox currently support only some special > > > > > > configurations which are not supported by other devices even its > > > > > > own VF device (except setting MAC address). > > > > > > And when I read "set drop enable bit in the VF split rx control > > > > > > register", it becomes clear it is really specific and has > > > > > > nothing to do in the generic ethdev API. > > > > > > That's why it is a NACK. > > > > > > > > > > > > When we want to use these very specific features we are aware of > > > > > > the underlying device and driver. So we can directly include a > > > > > > header from the driver. I suggest to retrieve a handler for the > > > > > > device which is not a port id and will allow to call ixgbe functions > > directly. > > > > > > It could be achieved by adding an ethdev function like discussed here: > > > > > > http://dpdk.org/ml/archives/dev/2016-September/047392.html > > > > > > > > > > > > > > > > I have been reading the net/vhost mail thread above. The following > > > > > quote > > > > is from this thread. > > > > > > > > > > "It means I would be in favor of introducing API in drivers for > > > > > very specific > > > > features." > > > > > > > > > > At present all the PMD functions are accessed through the > > > > > eth_dev_ops > > > > structure, there are no PMD API's. > > > > > > > > > > Is your proposal to add API(s) to the DPDK ixgbe PMD (similar to a > > > > > driver > > > > ioctl API) which can be accessed through a generic API in the ethdev? > > > > > > > > Not exactly. I'm thinking about a PMD specific API. > > > > The only ethdev API you need would be a function to retrieve a > > > > handler (an opaque pointer on the device struct) from the port id. > > > > Then you can include rte_ixgbe.h and directly call the specific > > > > ixgbe function, passing the device handler. > > > > How does it sound? > > > > > > I have been prototyping this proposed solution, it appears to work. > > > > > > I have added the following function: > > > > > > int rte_eth_dev_get_pmd_handle(uint8_t port_id, void** pmd_handle); > > > > > > The pmd_handle is a pointer to a dev_ops structure containing driver > > specific functions. > > > > > > Using the pmd_handle the driver specific functions can be called > > > (without having them in struct eth_dev_ops) > > > > > > Has this proposal been superseded by the discussion on the following > > patch? > > > > > > [PATCH] net/vhost: Add function to retreive the 'vid' for a given port > > > id > > > > Maybe, it can be superseded by this discussion, yes. > > Bruce thinks we do not need rte_eth_dev_get_pmd_handle(). > > What is your opinion about using port_id directly and retrieving the structs > > from the driver via rte_eth_devices? > > Looking at the code in rte_eth_devices[] > > struct rte_eth_dev rte_eth_devices[RTE_MAX_ETHPORTS]; > > struct rte_eth_dev { > > ... > > const struct eth_dev_ops *dev_ops; /**< Functions exported by PMD */ > > ... > > void *pmd_ops; /** < exported PMD specific functions */ > > } > > The PMD functions are only accessible at present if they are in struct eth_dev_ops. > > Adding a pmd_ops field to struct rte_eth_dev {} makes the PMD functions accessible and is a simpler solution than using rte_eth_dev_get_pmd_handle() to get access to the PMD functions. > > Regards, > Why would an ops structure be needed? If it's a private API for a driver, there should be no need for function pointers, and instead the driver can define regular functions in it's header file, no? /Bruce