From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f46.google.com (mail-pa0-f46.google.com [209.85.220.46]) by dpdk.org (Postfix) with ESMTP id 1A7998DED for ; Thu, 24 Dec 2015 18:56:45 +0100 (CET) Received: by mail-pa0-f46.google.com with SMTP id jx14so124583921pad.2 for ; Thu, 24 Dec 2015 09:56:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; bh=BJmWqTIWld1eyao7I2fFGWuweVdqkZ6nvhKPwwjqzy4=; b=Mf8rWXF5tlZzXmAWfXNrCrl3FZAVTdycYgRDGYlEXHI7DKzXAXW+NEZ6cL1AzDSu6K MjwV1xCYo3/CRFCxsGfjMClXMf08mJK5Ij6kMFcAZGTrPxYcUgppENKccnx+c0HDRMf8 RjSA6mxIvRH1/4DqlaXoq7/nDNp6HmfOU7JsSr0JP7Iw1c0D7xRQ3nkycJzEDA5Go9Dv rHkeDjJBHfLEyfrMZoCM4NlyJMnde5GZYlurFSy0Su7rjW+pW0lBdFSpS5nftd6+//G+ F7nOzVcJ2yowsgNWD+QKrlXxtIksVcf7N2rnTMFqN672oYirGE1x9CtPArOmMG5I1Q3i dzkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=BJmWqTIWld1eyao7I2fFGWuweVdqkZ6nvhKPwwjqzy4=; b=aEJukQ8VK/vJw3YVMSLHpdF9KWVAwzamWYEwmdipWHHI/NYjxwAEWGZG28RIU8Iaim m8IveLaFpYwQjtoeh+NjR4/KCsYcEtI0KTL4pVl9Cn239cHf5qm/7bxtkPBOtVU1lBg4 jaXHlM+l85J+priMw4QGAQ1LA2ATJLtPmdGPbYck4VaFjkN6DRrn+Z7Sj1jX3RfFMqLp DSyk2aHqI0Ah2joIPRMSUPo05K5JZm1Ldig4gpZ312x0nreTvRm7Zf7Hg02coEP1M8Vv N1DM9I9tLLO7ZFbU41IXx22sgA3RROGk9HystG7Uf/kS85uTJyfjEoUbdgIN49BnARN0 T6SQ== X-Gm-Message-State: ALoCoQkQ7n5GZR/is2+QU/c/F/UV1vev6L5LI1N6+f5OWeSh+NGy++lix3uJqb7PiUWhVle+m3x5KRUEcPKlWZ8ammfJxrx9ow== X-Received: by 10.67.3.196 with SMTP id by4mr53435226pad.67.1450979804202; Thu, 24 Dec 2015 09:56:44 -0800 (PST) Received: from xeon-e3 (static-50-53-82-155.bvtn.or.frontiernet.net. [50.53.82.155]) by smtp.gmail.com with ESMTPSA id m1sm55475364pfi.27.2015.12.24.09.56.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 Dec 2015 09:56:44 -0800 (PST) Date: Thu, 24 Dec 2015 09:56:52 -0800 From: Stephen Hemminger To: Yuanhan Liu Message-ID: <20151224095652.5a2adfe6@xeon-e3> In-Reply-To: <20151224033027.GY18863@yliu-dev.sh.intel.com> References: <20151222035041.GA7532@pxdev.xzpeter.org> <20151223015554.GC1694@pxdev.xzpeter.org> <20151223020949.GV18863@yliu-dev.sh.intel.com> <2150122.EUSmcjyJtA@xps13> <20151224033027.GY18863@yliu-dev.sh.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [Question] How pmd virtio works without UIO? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Dec 2015 17:56:45 -0000 On Thu, 24 Dec 2015 11:30:27 +0800 Yuanhan Liu wrote: > On Wed, Dec 23, 2015 at 11:26:17PM +0100, Thomas Monjalon wrote: > > 2015-12-23 10:09, Yuanhan Liu: > > > On Wed, Dec 23, 2015 at 09:55:54AM +0800, Peter Xu wrote: > > > > On Tue, Dec 22, 2015 at 04:38:30PM +0000, Xie, Huawei wrote: > > > > > On 12/22/2015 7:39 PM, Peter Xu wrote: > > > > > > I tried to unbind one of the virtio net device, I see the PCI entry > > > > > > still there. > > > > > > > > > > > > Before unbind: > > > > > > > > > > > > [root@vm proc]# lspci -k -s 00:03.0 > > > > > > 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device > > > > > > Subsystem: Red Hat, Inc Device 0001 > > > > > > Kernel driver in use: virtio-pci > > > > > > [root@vm proc]# cat /proc/ioports | grep c060-c07f > > > > > > c060-c07f : 0000:00:03.0 > > > > > > c060-c07f : virtio-pci > > > > > > > > > > > > After unbind: > > > > > > > > > > > > [root@vm proc]# lspci -k -s 00:03.0 > > > > > > 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device > > > > > > Subsystem: Red Hat, Inc Device 0001 > > > > > > [root@vm proc]# cat /proc/ioports | grep c060-c07f > > > > > > c060-c07f : 0000:00:03.0 > > > > > > > > > > > > So... does this means that it is an alternative to black list > > > > > > solution? > > > > > Oh, we could firstly check if this port is manipulated by kernel driver > > > > > in virtio_resource_init/eth_virtio_dev_init, as long as it is not too late. > > > > > > Why can't we simply quit at pci_scan_one, once finding that it's not > > > bond to uio (or similar stuff)? That would be generic enough, that we > > > don't have to do similar checks for each new pmd driver. > > > > > > Or, am I missing something? > > > > UIO is not needed to make virtio works (without interrupt support). > > Sometimes it may be required to avoid using kernel modules. > > While dig the git history, I found the commit: > > commit da978dfdc43b59e290a46d7ece5fd19ce79a1162 > Author: Ouyang Changchun > Date: Mon Feb 9 09:14:06 2015 +0800 > > virtio: use port IO to get PCI resource > > Make virtio not require UIO for some security reasons, this is to match > 6WIND's virtio-net-pmd. > > Signed-off-by: Changchun Ouyang > Acked-by: Huawei Xie > > The commit log is not well written, giving no explanation about the > "some security reasons". > > Anyway, I see it now that it's kind of a design. > > > Note that my first patch set about enabling virtio 1.0 [0] sets the > RTE_PCI_DRV_NEED_MAPPING flag, which somehow implies that uio is a > must, otherwise, eal init would fail at pci_map_device(). > > So that my pathset breaks the un-documented rule, and I need fix it. > How about adding a wrapper, say rte_pci_map_device(), and exporting > it, so that virtio pmd driver could map resources when necessary? > > [0]: http://dpdk.org/dev/patchwork/bundle/yliu/virtio-1.0-pmd/ > > > > > > I guess there might be two problems? Which are: > > > > > > > > 1. How user avoid DPDK taking over virtio devices that they do not > > > > want for IO (chooses which device to use) > > > > > > Isn't that what's the 'binding/unbinding' for? > > > > Binding is, sometimes, required. > > We may need fix the doc then. As the doc says it's a must: > > 3.6. Binding and Unbinding Network Ports to/from the Kernel Modules > > Instead, all ports that are to be used by an DPDK application > ==> must be bound to the uio_pci_generic, igb_uio or vfio-pci > module before the application is run. Any network ports under > Linux* control will be ignored by the DPDK poll-mode drivers > and cannot be used by the application. > > > --yliu > > > But does it mean DPDK should use every available ports? > > That's the default and may be configured with blacklist/whitelist. > > > > > > 2. Driver conflict between virtio PMD in DPDK, and virtio-pci in > > > > kernel (happens on every virtio device that DPDK uses) > > > > > > If you unbinded the kernel driver first, which is the suggested (or > > > must?) way to use DPDK, that will not happen. As far as I remember; there are some environments where DPDK but guests are not allowed to load their own kernel modules. But since virtio only needs access to I/O ports on x86, the driver can accomodate. I haven't run into these environments. But the added code in virtio driver causes issues. Mostly because it causes an unnecessary duplication of the initialization code, and is missing many of the protections and interfaces that exist in the base code. For example, there are lots of corner cases in interrupt support which are related to this non-UIO mode of operation.