Re: [dpdk-dev] [RFC PATCH v2 3/5] librte_ether: add API's for VF management

[Thomas Monjalon <thomas.monjalon@6wind.com>]

2016-09-23 09:53, Richardson, Bruce:
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> > 2016-09-23 10:20, Bruce Richardson:
> > > On Thu, Sep 22, 2016 at 07:04:37PM +0200, Thomas Monjalon wrote:
> > > > 2016-09-15 16:46, Iremonger, Bernard:
> > > > > > > > Do we really need to expose VF specific functions here?
> > > > > > > > It can be generic(PF/VF) function indexed only through
> > port_id.
> > > > > > > > (example: as rte_eth_dev_set_vlan_anti_spoof(uint8_t
> > > > > > > > port_id, uint8_t on)) For instance, In Thunderx PMD, We are
> > > > > > > > not exposing a separate port_id for PF. We only enumerate
> > > > > > > > 0..N VFs as 0..N ethdev port_id
> > > > > > >
> > > > > > > Our intention with this patch is to control the VF from the PF.
> > > > > > >
> > > > > > > The following librte_ether functions already work in a similar
> > way:
> > > > > > >
> > > > > > > rte_eth_dev_set_vf_rxmode(uint8_t port_id,  uint16_t vf,
> > > > > > > uint16_t rx_mode, uint8_t on)
> > > > > > >
> > > > > > > rte_eth_dev_set_vf_rx(uint8_t port_id, uint16_t vf, uint8_t
> > > > > > > on)
> > > > > > >
> > > > > > > rte_eth_dev_set_vf_tx(uint8_t port_id, uint16_t vf, uint8_t
> > > > > > > on)
> > > > > > >
> > > > > > > int rte_eth_set_vf_rate_limit(uint8_t port_id, uint16_t vf,
> > > > > > > uint16_t tx_rate, uint64_t q_msk)
> > > > > >
> > > > > > I have a bad feeling with these functions dedicated to VF from PF.
> > > > > > Are we sure there is no other way?
> > > > > > I mean we just need to know the VF with a port ID.
> > > > >
> > > > > When the VF is used in a VM the port ID of the VF is not visible to
> > the PF.
> > > > > I don't think there is another way to do this.
> > > >
> > > > I don't understand why we could not assign a port id to the VF from
> > > > the host instead of having the couple PF port id / VF id.
> > > > Can we enumerate all the VFs associated to a PF?
> > > > Then can we allocate them a port id in the array rte_eth_devices?
> > >
> > > Hi Thomas,
> > >
> > > The VF is not a port visible to DPDK, though, so it shouldn't have a
> > > port id IMHO. DPDK can't actually do anything with it.
> > 
> > You say the contrary below.
> 
> Well, yes and no. The driver can manipulate things for the VF, but DPDK doesn't actually have a device that corresponds to the VF. There are no PCI bar mappings for it, DPDK can't do RX and TX with it etc.?

Very good point.
There are only few ethdev functions which are supported by every drivers,
like Rx/Tx and would not be available for VF from PF interface.

> > > The PCI device for the VF is likely passed through to a different VM
> > > and being used there. Unfortunately, the VF still needs certain things
> > > done for it by the PF, so if the PF is under DPDK control, it needs to
> > > provide the functionality to assist the VF.
> > 
> > Why not have a VF_from_PF driver which does the mailbox things?
> > So you can manage the VF from the PF with a simple port id.
> > It really seems to be the cleanest design to me.
> 
> While I see your point, and it could work, I just want to be sure that we are ok with the results of that. Suppose we do create ethdevs for the VFs controlled by the PF. Does the new VF get counted in the rte_eth_dev_count() value (I assume yes)? How are apps meant to use the port? Do they have to put in a special case when iterating through all the port ids to check that it's not a pseudo port that can't do anything. None of the standard ethdev calls from an app will work on it, you can't configure nb rx/tx queues on it, you can't start or stop it, you can't do rx or tx on it, etc, etc.

Yes these devices would be special because their supported API would be
quite different. I was thinking that in the future you could add most
of the configuration functions through the VF mailbox.
But the Intel mailbox currently support only some special configurations
which are not supported by other devices even its own VF device (except
setting MAC address).
And when I read "set drop enable bit in the VF split rx control register",
it becomes clear it is really specific and has nothing to do in the
generic ethdev API.
That's why it is a NACK.

When we want to use these very specific features we are aware of the
underlying device and driver. So we can directly include a header from
the driver. I suggest to retrieve a handler for the device which is not
a port id and will allow to call ixgbe functions directly.
It could be achieved by adding an ethdev function like discussed here:
	http://dpdk.org/ml/archives/dev/2016-September/047392.html