From: Maxime Coquelin <maxime.coquelin@redhat.com>
To: "谢华伟(此时此刻）" <huawei.xhw@alibaba-inc.com>, ferruh.yigit@intel.com
Cc: dev@dpdk.org, anatoly.burakov@intel.com,
david.marchand@redhat.com, zhihong.wang@intel.com,
chenbo.xia@intel.com, grive@u256.net
Subject: Re: [dpdk-dev] [PATCH v5 3/3] PCI: don't use vfio ioctl call to access PIO resource
Date: Thu, 21 Jan 2021 09:29:54 +0100 [thread overview]
Message-ID: <e24f7bfd-e071-0b51-bbc9-21e0b61a69cf@redhat.com> (raw)
In-Reply-To: <18871462-4d25-302a-2716-99ebec65c3ac@alibaba-inc.com>
On 1/20/21 3:54 PM, 谢华伟(此时此刻） wrote:
>
> On 2021/1/13 0:58, Maxime Coquelin wrote:
>>
>> On 1/12/21 10:37 AM, Maxime Coquelin wrote:
>>> bus/pci: ...
>>>
>>> On 10/22/20 5:51 PM, 谢华伟(此时此刻） wrote:
>>>> From: "huawei.xhw" <huawei.xhw@alibaba-inc.com>
>>>>
>>>> VFIO should use the same way to map/read/write PORT IO as UIO, for
>>>> virtio PMD.
>>> Please provide more details in the commit message on why the way VFIO
>>> works today is wrong (The cover letter is lost once applied).
> ok
>>>
>>>> Signed-off-by: huawei.xhw <huawei.xhw@alibaba-inc.com>
>>> Same comment about name format as on previous patches.
>>>
>>>> ---
>>>> drivers/bus/pci/linux/pci.c | 8 ++++----
>>>> drivers/bus/pci/linux/pci_uio.c | 4 +++-
>>>> 2 files changed, 7 insertions(+), 5 deletions(-)
>>>>
>>>> diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c
>>>> index 0dc99e9..2ed9f2b 100644
>>>> --- a/drivers/bus/pci/linux/pci.c
>>>> +++ b/drivers/bus/pci/linux/pci.c
>>>> @@ -687,7 +687,7 @@ int rte_pci_write_config(const struct
>>>> rte_pci_device *device,
>>>> #ifdef VFIO_PRESENT
>>>> case RTE_PCI_KDRV_VFIO:
>>>> if (pci_vfio_is_enabled())
>>>> - ret = pci_vfio_ioport_map(dev, bar, p);
>>>> + ret = pci_uio_ioport_map(dev, bar, p);
>>> Doesn't it create a regression with regards to needed capabilities?
>>> My understanding is that before this patch we don't need to call iopl(),
>>> whereas once applied it is required, correct?
>> I did some testing today, and think it is not a regression with para-
>> virtualized Virtio devices.
>>
>> Indeed, I thought it would be a regression with Legacy devices when
>> IOMMU is enabled and the program is run as non-root (IOMMU enabled
>> just to suport IOVA as VA mode). But it turns out para-virtualized
>> Virtio legacy device and vIOMMU enabled is not a supported configuration
>> by QEMU.
>>
>> Note that when noiommu mode is enabled, the app needs cap_sys_rawio, so
>> same as iopl(). No regression in this case too.
>>
>> That said, with real (non para-virtualized) Virtio device using PIO like
>> yours, doesn't your patch introduce a restriction for your device that
>> it will require cap_sys_rawio whereas it would not be needed?
>
> I don't catch the regression issue.
>
> With real virtio device(hardware implemented), if it is using MMIO, no
> cap_sys_rawio is required.
>
> If it is using PIO, iopl is required always.
My understanding of the Kernel VFIO driver is that cap_sys_rawio is only
necessary in noiommu mode, i.e. when VFIO is loaded with
enable_unsafe_noiommu parameter set. The doc for this parameters seems
to validate my understanding of the code:
"
MODULE_PARM_DESC(enable_unsafe_noiommu_mode, "Enable UNSAFE, no-IOMMU
mode. This mode provides no device isolation, no DMA translation, no
host kernel protection, cannot be used for device assignment to virtual
machines, requires RAWIO permissions, and will taint the kernel. If you
do not know what this is for, step away. (default: false)");
"
I think that using inb/outb in the case of VFIO with IOMMU enabled won't
work without cap_sys_rawio, and using it in the case of VFIO with IOMMU
disabled just bypasses VFIO and so is not correct.
In my opinion, what we should do is to add something like this in the
DPDK documentation:
- MMIO BAR: VFIO with IOMMU enabled recommended. Equivalent performance
as with IGB UIO or VFIO with NOIOMMU. VFIO with IOMMU is recommended for
security reasons.
- PIO BAR: VFIO with IOMMU enabled is recommended for security reasons,
providing proper isolation and not requiring cap_sys_rawio. However, use
of IOMMU is not always possible in some cases (e.g. para-virtualized
Virtio-net legacy device). Also, performance of using VFIO for PIO BARs
accesses has an impact on performance as it uses pread/pwrite syscalls,
whereas UIO drivers use inb/outb. If security is not a concern or IOMMU
is not available, one might consider using UIO driver in this case for
performance reasons.
What do you think?
Thanks,
Maxime
>
>> Thanks,
>> Maxime
>>
>>> Regards,
>>> Maxime
>>>
>
next prev parent reply other threads:[~2021-01-21 8:30 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-30 14:59 [dpdk-dev] [PATCH v2] pci: support both PIO and MMIO BAR for legacy virtio on x86 谢华伟(此时此刻）
2020-10-01 10:22 ` Burakov, Anatoly
2020-10-02 5:44 ` 谢华伟(此时此刻）
2020-10-09 8:36 ` [dpdk-dev] [PATCH v3] " 谢华伟(此时此刻）
2020-10-13 8:41 ` [dpdk-dev] [PATCH v4] support both PIO and MMIO bar for virtio pci device 谢华伟(此时此刻）
2020-10-13 8:41 ` [dpdk-dev] [PATCH v4] pci: support both PIO and MMIO BAR for legacy virtio on x86 谢华伟(此时此刻）
2020-10-13 12:34 ` 谢华伟(此时此刻）
2020-10-21 8:46 ` 谢华伟(此时此刻）
2020-10-21 11:49 ` Ferruh Yigit
2020-10-21 12:32 ` 谢华伟(此时此刻）
2020-10-21 17:24 ` Ferruh Yigit
2020-10-22 9:15 ` 谢华伟(此时此刻）
2020-10-22 9:44 ` Ferruh Yigit
2020-10-22 9:57 ` 谢华伟(此时此刻）
2020-10-22 15:51 ` [dpdk-dev] [PATCH v5 0/3] support both PIO and MMIO BAR for virtio PMD 谢华伟(此时此刻）
2020-10-22 15:51 ` [dpdk-dev] [PATCH v5 1/3] PCI: use PCI standard sysfs entry to get PIO address 谢华伟(此时此刻）
2021-01-12 8:07 ` Maxime Coquelin
2021-01-14 18:23 ` 谢华伟(此时此刻）
2021-01-24 15:10 ` Xueming(Steven) Li
2020-10-22 15:51 ` [dpdk-dev] [PATCH v5 2/3] PCI: support MMIO in rte_pci_ioport_map/unap/read/write 谢华伟(此时此刻）
2021-01-12 8:23 ` Maxime Coquelin
2021-01-21 6:30 ` 谢华伟(此时此刻）
2021-01-24 15:22 ` Xueming(Steven) Li
2021-01-25 3:08 ` 谢华伟(此时此刻）
2021-01-27 10:40 ` Ferruh Yigit
2021-01-27 15:34 ` 谢华伟(此时此刻）
2021-01-27 16:45 ` Ferruh Yigit
2020-10-22 15:51 ` [dpdk-dev] [PATCH v5 3/3] PCI: don't use vfio ioctl call to access PIO resource 谢华伟(此时此刻）
2021-01-12 9:37 ` Maxime Coquelin
2021-01-12 16:58 ` Maxime Coquelin
2021-01-20 14:54 ` 谢华伟(此时此刻）
2021-01-21 8:29 ` Maxime Coquelin [this message]
2021-01-21 14:57 ` 谢华伟(此时此刻）
2021-01-21 15:00 ` 谢华伟(此时此刻）
2021-01-21 15:38 ` Maxime Coquelin
2021-01-22 7:25 ` 谢华伟(此时此刻）
2021-01-26 10:44 ` Maxime Coquelin
2021-01-27 10:32 ` Ferruh Yigit
2021-01-27 12:17 ` Maxime Coquelin
2021-01-27 14:43 ` 谢华伟(此时此刻）
2021-01-27 16:45 ` Ferruh Yigit
2021-01-28 13:43 ` 谢华伟(此时此刻）
2021-01-26 12:30 ` 谢华伟(此时此刻）
2021-01-26 12:35 ` Maxime Coquelin
2021-01-26 14:24 ` 谢华伟(此时此刻）
2020-10-27 8:50 ` [dpdk-dev] [PATCH v5 0/3] support both PIO and MMIO BAR for virtio PMD 谢华伟(此时此刻）
2020-10-28 3:48 ` 谢华伟(此时此刻）
2020-11-02 11:56 ` 谢华伟(此时此刻）
2020-11-10 12:35 ` 谢华伟(此时此刻）
2020-11-10 12:42 ` David Marchand
2020-11-12 13:35 ` 谢华伟(此时此刻）
2020-12-14 14:24 ` 谢华伟(此时此刻）
2020-12-16 7:54 ` Maxime Coquelin
2021-01-12 17:37 ` Maxime Coquelin
2021-01-14 18:19 ` 谢华伟(此时此刻）
2021-01-21 4:12 ` 谢华伟(此时此刻）
2021-01-21 8:47 ` Maxime Coquelin
2021-01-21 13:51 ` 谢华伟(此时此刻）
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e24f7bfd-e071-0b51-bbc9-21e0b61a69cf@redhat.com \
--to=maxime.coquelin@redhat.com \
--cc=anatoly.burakov@intel.com \
--cc=chenbo.xia@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=grive@u256.net \
--cc=huawei.xhw@alibaba-inc.com \
--cc=zhihong.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).