Re: [dpdk-dev] [Question] How pmd virtio works without UIO?

[Stephen Hemminger <stephen@networkplumber.org>]

On Thu, 24 Dec 2015 11:30:27 +0800
Yuanhan Liu <yuanhan.liu@linux.intel.com> wrote:

> On Wed, Dec 23, 2015 at 11:26:17PM +0100, Thomas Monjalon wrote:
> > 2015-12-23 10:09, Yuanhan Liu:
> > > On Wed, Dec 23, 2015 at 09:55:54AM +0800, Peter Xu wrote:
> > > > On Tue, Dec 22, 2015 at 04:38:30PM +0000, Xie, Huawei wrote:
> > > > > On 12/22/2015 7:39 PM, Peter Xu wrote:
> > > > > > I tried to unbind one of the virtio net device, I see the PCI entry
> > > > > > still there.
> > > > > >
> > > > > > Before unbind:
> > > > > >
> > > > > > [root@vm proc]# lspci -k -s 00:03.0
> > > > > > 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
> > > > > >         Subsystem: Red Hat, Inc Device 0001
> > > > > >         Kernel driver in use: virtio-pci
> > > > > > [root@vm proc]# cat /proc/ioports | grep c060-c07f
> > > > > >   c060-c07f : 0000:00:03.0
> > > > > >     c060-c07f : virtio-pci
> > > > > >
> > > > > > After unbind:
> > > > > >
> > > > > > [root@vm proc]# lspci -k -s 00:03.0
> > > > > > 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
> > > > > >         Subsystem: Red Hat, Inc Device 0001
> > > > > > [root@vm proc]# cat /proc/ioports | grep c060-c07f
> > > > > >   c060-c07f : 0000:00:03.0
> > > > > >
> > > > > > So... does this means that it is an alternative to black list
> > > > > > solution?
> > > > > Oh, we could firstly check if this port is manipulated by kernel driver
> > > > > in virtio_resource_init/eth_virtio_dev_init, as long as it is not too late.
> > > 
> > > Why can't we simply quit at pci_scan_one, once finding that it's not
> > > bond to uio (or similar stuff)? That would be generic enough, that we
> > > don't have to do similar checks for each new pmd driver.
> > > 
> > > Or, am I missing something?
> > 
> > UIO is not needed to make virtio works (without interrupt support).
> > Sometimes it may be required to avoid using kernel modules.
> 
> While dig the git history, I found the commit:
> 
>     commit da978dfdc43b59e290a46d7ece5fd19ce79a1162
>     Author: Ouyang Changchun <changchun.ouyang@intel.com>
>     Date:   Mon Feb 9 09:14:06 2015 +0800
>     
>         virtio: use port IO to get PCI resource
>     
>         Make virtio not require UIO for some security reasons, this is to match
>         6WIND's virtio-net-pmd.
>     
>         Signed-off-by: Changchun Ouyang <changchun.ouyang@intel.com>
>         Acked-by: Huawei Xie <huawei.xie@intel.com>
> 
> The commit log is not well written, giving no explanation about the
> "some security reasons".
> 
> Anyway, I see it now that it's kind of a design.
> 
> 
> Note that my first patch set about enabling virtio 1.0 [0] sets the
> RTE_PCI_DRV_NEED_MAPPING flag, which somehow implies that uio is a
> must, otherwise, eal init would fail at pci_map_device().
> 
> So that my pathset breaks the un-documented rule, and I need fix it.
> How about adding a wrapper, say rte_pci_map_device(), and exporting
> it, so that virtio pmd driver could map resources when necessary?
> 
> [0]: http://dpdk.org/dev/patchwork/bundle/yliu/virtio-1.0-pmd/
> 
> 
> > > > I guess there might be two problems? Which are:
> > > > 
> > > > 1. How user avoid DPDK taking over virtio devices that they do not
> > > >    want for IO (chooses which device to use)
> > > 
> > > Isn't that what's the 'binding/unbinding' for?
> > 
> > Binding is, sometimes, required.
> 
> We may need fix the doc then. As the doc says it's a must:
> 
>     3.6. Binding and Unbinding Network Ports to/from the Kernel Modules
> 
>     Instead, all ports that are to be used by an DPDK application
> ==> must be bound to the uio_pci_generic, igb_uio or vfio-pci
>     module before the application is run. Any network ports under
>     Linux* control will be ignored by the DPDK poll-mode drivers
>     and cannot be used by the application.
> 
> 
> 	--yliu
> 
> > But does it mean DPDK should use every available ports?
> > That's the default and may be configured with blacklist/whitelist.
> > 
> > > > 2. Driver conflict between virtio PMD in DPDK, and virtio-pci in
> > > >    kernel (happens on every virtio device that DPDK uses)
> > > 
> > > If you unbinded the kernel driver first, which is the suggested (or
> > > must?) way to use DPDK, that will not happen.


As far as I remember; there are some environments where DPDK but guests are not
allowed to load their own kernel modules. But since virtio only needs access to
I/O ports on x86, the driver can accomodate. I haven't run into these environments.

But the added code in virtio driver causes issues. Mostly because it causes
an unnecessary duplication of the initialization code, and is missing many of
the protections and interfaces that exist in the base code. For example,
there are lots of corner cases in interrupt support which are related to
this non-UIO mode of operation.