From: Ferruh Yigit <ferruh.yigit@intel.com>
To: "谢华伟(此时此刻)" <huawei.xhw@alibaba-inc.com>,
"Maxime Coquelin" <maxime.coquelin@redhat.com>
Cc: dev@dpdk.org, anatoly.burakov@intel.com,
david.marchand@redhat.com, grive@u256.net,
zhihong.wang@intel.com, chenbo.xia@intel.com
Subject: Re: [dpdk-dev] [PATCH v4] pci: support both PIO and MMIO BAR for legacy virtio on x86
Date: Wed, 21 Oct 2020 18:24:21 +0100 [thread overview]
Message-ID: <e23712f1-7dbb-ce3c-2694-5648eb8369ae@intel.com> (raw)
In-Reply-To: <5d26db1a-6970-6e3a-c2e5-8dc540922028@alibaba-inc.com>
On 10/21/2020 1:32 PM, 谢华伟(此时此刻) wrote:
>
> On 2020/10/21 19:49, Ferruh Yigit wrote:
>> On 10/13/2020 9:41 AM, 谢华伟(此时此刻) wrote:
>>> From: "huawei.xhw" <huawei.xhw@alibaba-inc.com>
>>>
>>> Legacy virtio-pci only supports PIO BAR resource. As we need to create lots of
>>> virtio devices and PIO resource on x86 is very limited, we expose MMIO BAR.
>>>
>>> Kernel supports both PIO and MMIO BAR for legacy virtio-pci device. We handles
>>> different type of BAR in the similar way.
>>>
>>> In previous implementation, with igb_uio we get PIO address from igb_uio
>>> sysfs entry; with uio_pci_generic, we get PIO address from
>>> /proc/ioports.
>>> For PIO/MMIO RW, there is different path for different drivers and arch.
>>> For VFIO, PIO/MMIO RW is through syscall, which has big performance
>>> issue.
>>> On X86, it assumes only PIO is supported.
>>>
>>> All of the above is too much twisted.
>>> This patch unifies the way to get both PIO and MMIO address for different driver
>>> and arch, all from standard resource attr under pci sysfs.
>>>
>>
>> As mentined above this patch does multiple things.
>>
>> The main target is, as far as I understand, you have a legacy virtio device
>> which supports "memory-mapped I/O" and "port-mapped I/O", but virtio logic
>> forces legacy devices to use the PIO but you want to be able to use the MMIO
>> with this device.
> yes.
>>
>> The solution below is adding MMIO support in the PIO funciton, and distinguish
>> MMIO or PIO based on their address check.
> Yes, kernel does this in the similar way.
>>
>>
>> Instead of this, can't this be resolved in the virtio side, like if the legacy
>> device supports MMIO (detect this somehow) use the MMIO istead of hacking PIO
>> mapping to support MMIO?
>
> Get your concern.
>
> 1>
>
> If we move, I think we should move all those PCI codes into virtio side, not
> just the mmio part.
>
> Without my patch, those PCI codes are virtio-pci device specific, not generic.
>
> With this patch, those pci ioport map/rw code could also be used for other
> devices if they support both PIO and MMIO.
>
I was not suggesting moving any code into virtio, but within 'vtpci_init()' what
happens when "hw->modern = 1;" is set?
And if this is set for your device, will it work without change?
> Every option is ok. Hope i make myself clear.
>
> 2> I don't think this is hacking. for rte_pci_ioport_map/read/write, if ioport
> could be both PIO and MMIO, then everything is reasonable.
>
> Take how kernel does port map for example:
>
> vp_dev->ioaddr = pci_iomap(pci_dev, 0, 0);
>
> Here io doesn't mean PIO only. It could also be MMIO. Kernel then uses
> ioread/write to access PIO/MMIO port.
>
> Actually we are pretty much the same in the interface.
>
> I think this patch extends rather then hacks the ioport interface to support MMIO.
>
>>
>> I have other concerns, specially mergin VFIO mapping too, but lets clarify
>> above first.
>
> vfio doesn't affect other driver but only virtio.
>
Why it doesn't affect other drivers, can't there be other driver using PIO?
> igb_uio, uio_pci_generic and vfio-pci all uses the same way to map/rw ioport.
>
For vfio, code changes 'pci_vfio_ioport_read()' to the direct address read,
first I don't know if this is always safe, and my question why there is a
syscall introduced at first place if you can read from address directly?
Is your device works as expected when vfio-pci kernel module used? Since it is
not suffering from PIO limitation, right?
And I wonder if the patch can be done as three patches to simply it, as:
1) Combine 'RTE_PCI_KDRV_IGB_UIO' & 'RTE_PCI_KDRV_UIO_GENERIC' (remove
pci_ioport_map)
2) Update 'pci_uio_ioport_map()' to add memory map support (and update
read/write functions according)
3) Combine vfio & uio
>>
>> Thanks,
>> ferruh
>>
>>
>>
>>> We distinguish PIO and MMIO by their address like how kernel does. It is ugly
>>> but works.
>>>
>>> Signed-off-by: huawei.xhw <huawei.xhw@alibaba-inc.com>
<...>
next prev parent reply other threads:[~2020-10-21 17:24 UTC|newest]
Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-30 14:59 [dpdk-dev] [PATCH v2] " 谢华伟(此时此刻)
2020-10-01 10:22 ` Burakov, Anatoly
2020-10-02 5:44 ` 谢华伟(此时此刻)
2020-10-09 8:36 ` [dpdk-dev] [PATCH v3] " 谢华伟(此时此刻)
2020-10-13 8:41 ` [dpdk-dev] [PATCH v4] support both PIO and MMIO bar for virtio pci device 谢华伟(此时此刻)
2020-10-13 8:41 ` [dpdk-dev] [PATCH v4] pci: support both PIO and MMIO BAR for legacy virtio on x86 谢华伟(此时此刻)
2020-10-13 12:34 ` 谢华伟(此时此刻)
2020-10-21 8:46 ` 谢华伟(此时此刻)
2020-10-21 11:49 ` Ferruh Yigit
2020-10-21 12:32 ` 谢华伟(此时此刻)
2020-10-21 17:24 ` Ferruh Yigit [this message]
2020-10-22 9:15 ` 谢华伟(此时此刻)
2020-10-22 9:44 ` Ferruh Yigit
2020-10-22 9:57 ` 谢华伟(此时此刻)
2020-10-22 15:51 ` [dpdk-dev] [PATCH v5 0/3] support both PIO and MMIO BAR for virtio PMD 谢华伟(此时此刻)
2020-10-22 15:51 ` [dpdk-dev] [PATCH v5 1/3] PCI: use PCI standard sysfs entry to get PIO address 谢华伟(此时此刻)
2021-01-12 8:07 ` Maxime Coquelin
2021-01-14 18:23 ` 谢华伟(此时此刻)
2021-01-24 15:10 ` Xueming(Steven) Li
2020-10-22 15:51 ` [dpdk-dev] [PATCH v5 2/3] PCI: support MMIO in rte_pci_ioport_map/unap/read/write 谢华伟(此时此刻)
2021-01-12 8:23 ` Maxime Coquelin
2021-01-21 6:30 ` 谢华伟(此时此刻)
2021-01-24 15:22 ` Xueming(Steven) Li
2021-01-25 3:08 ` 谢华伟(此时此刻)
2021-01-27 10:40 ` Ferruh Yigit
2021-01-27 15:34 ` 谢华伟(此时此刻)
2021-01-27 16:45 ` Ferruh Yigit
2020-10-22 15:51 ` [dpdk-dev] [PATCH v5 3/3] PCI: don't use vfio ioctl call to access PIO resource 谢华伟(此时此刻)
2021-01-12 9:37 ` Maxime Coquelin
2021-01-12 16:58 ` Maxime Coquelin
2021-01-20 14:54 ` 谢华伟(此时此刻)
2021-01-21 8:29 ` Maxime Coquelin
2021-01-21 14:57 ` 谢华伟(此时此刻)
2021-01-21 15:00 ` 谢华伟(此时此刻)
2021-01-21 15:38 ` Maxime Coquelin
2021-01-22 7:25 ` 谢华伟(此时此刻)
2021-01-26 10:44 ` Maxime Coquelin
2021-01-27 10:32 ` Ferruh Yigit
2021-01-27 12:17 ` Maxime Coquelin
2021-01-27 14:43 ` 谢华伟(此时此刻)
2021-01-27 16:45 ` Ferruh Yigit
2021-01-28 13:43 ` 谢华伟(此时此刻)
2021-01-26 12:30 ` 谢华伟(此时此刻)
2021-01-26 12:35 ` Maxime Coquelin
2021-01-26 14:24 ` 谢华伟(此时此刻)
2020-10-27 8:50 ` [dpdk-dev] [PATCH v5 0/3] support both PIO and MMIO BAR for virtio PMD 谢华伟(此时此刻)
2020-10-28 3:48 ` 谢华伟(此时此刻)
2020-11-02 11:56 ` 谢华伟(此时此刻)
2020-11-10 12:35 ` 谢华伟(此时此刻)
2020-11-10 12:42 ` David Marchand
2020-11-12 13:35 ` 谢华伟(此时此刻)
2020-12-14 14:24 ` 谢华伟(此时此刻)
2020-12-16 7:54 ` Maxime Coquelin
2021-01-12 17:37 ` Maxime Coquelin
2021-01-14 18:19 ` 谢华伟(此时此刻)
2021-01-21 4:12 ` 谢华伟(此时此刻)
2021-01-21 8:47 ` Maxime Coquelin
2021-01-21 13:51 ` 谢华伟(此时此刻)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e23712f1-7dbb-ce3c-2694-5648eb8369ae@intel.com \
--to=ferruh.yigit@intel.com \
--cc=anatoly.burakov@intel.com \
--cc=chenbo.xia@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=grive@u256.net \
--cc=huawei.xhw@alibaba-inc.com \
--cc=maxime.coquelin@redhat.com \
--cc=zhihong.wang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).