Re: [dpdk-dev] Query : Does Binding with vfio-pci is supported inside qemu-kvm guest/vm instance.?

[John Sucaet <john.sucaet@oneaccess-net.com>]

Hi Maxime,

Thanks for your answer. I want to add that I have this problem both with 
18.08 and 18.11. Also I am using qemu with machine type pc-q35-2.10 (and 
pcie-root). When I change it to machine type pc-i440fx-2.4 (and 
pci-root), I don't see the problem. Ports are in that case, detected in 
the 32-bit dpdk application, as expected.

Hope this helps in finding a reason why mmap failed.

Thanks

John

On 04/03/2019 09:54 AM, Maxime Coquelin wrote:
> Hi John,
>
> On 4/3/19 9:49 AM, John Sucaet wrote:
>> Thanks, Anatoly.
>>
>> Maxime, could you enlighten me a bit? I basically would like to know 
>> whether I should be able to make my 32-bit dpdk application work with 
>> virtio-pci-net and vfio-pci (with a 64-bit kernel), or if I should do 
>> the effort to port to 64-bit (which I would like to avoid for now).
>
> I think that it should work, but that's not something I have tried.
> I will try to reproduce it this week to get a precise idea of what is
> going wrong.
>
> Thanks for reporting the issue,
> Maxime
>
>> Thank you
>> John
>>
>> On 04/02/2019 03:38 PM, Burakov, Anatoly wrote:
>>> On 02-Apr-19 11:38 AM, John Sucaet wrote:
>>>> Hi Anatoly,
>>>>
>>>> As you said: There's no reason to use igb_uio, ever!
>>>
>>> That was partly tongue in cheek, but point taken :)
>>>
>>>> I would like to ask whether vfio-pci with or without vIOMMU 
>>>> should/could work for virtio-pci net devices in the case of a 
>>>> 32-bit dpdk application, on a 64-bit kernel (4.9) inside a guest VM 
>>>> (qemu-2.10.2-1.fc27)?
>>>>
>>>> I tried both a 64-bit and a 32-bit version of the same application, 
>>>> but only in the case of the 64-bit application, the port was found 
>>>> by eal. The 32-bit application gave errors like:
>>>>
>>>> EAL: pci_map_resource(): cannot mmap(16, 0xf4a01000, 0x4000, 0x0): 
>>>> Invalid argument (0xffffffff)
>>>> EAL: Failed to map pci BAR4
>>>> EAL:   0000:00:02.0 mapping BAR4 failed: Invalid argument
>>>> EAL: Can't write to PCI bar (0) : offset (12)
>>>> EAL: Can't read from PCI bar (0) : offset (12)
>>>> EAL: Can't read from PCI bar (0) : offset (12)
>>>> EAL: Can't write to PCI bar (0) : offset (12)
>>>> EAL: Can't read from PCI bar (0) : offset (12)
>>>> EAL: Can't write to PCI bar (0) : offset (12)
>>>> EAL: Can't read from PCI bar (0) : offset (0)
>>>> EAL: Can't write to PCI bar (0) : offset (4)
>>>> EAL: Can't write to PCI bar (0) : offset (14)
>>>> EAL: Can't read from PCI bar (0) : offset (14)
>>>> EAL: Can't read from PCI bar (0) : offset (1a)
>>>> EAL: Can't read from PCI bar (0) : offset (1c)
>>>> EAL: Can't write to PCI bar (0) : offset (e)
>>>> EAL: Can't read from PCI bar (0) : offset (c)
>>>> virtio_init_queue(): virtqueue size is not powerof 2
>>>> EAL: Requested device 0000:00:02.0 cannot be used
>>>>
>>>> Maybe you have an idea what went wrong here?
>>>>
>>>> By preference, I would like to continue to use the 32-bit 
>>>> application which worked fine with the igb_uio driver.
>>>
>>> Unfortunately, i am not very familiar with virtio and wouldn't know 
>>> whether it's supposed to work under these conditions. Perhaps Maxime 
>>> would be of more help here (CC'd).
>>>
>>>>
>>>>
>>>> Thanks
>>>>
>>>> John
>>>>
>>>>
>>>> On 03/12/2019 11:57 AM, Burakov, Anatoly wrote:
>>>>> On 12-Mar-19 10:20 AM, Bruce Richardson wrote:
>>>>>> On Tue, Mar 12, 2019 at 05:54:39PM +0800, Jason Wang wrote:
>>>>>>>
>>>>>>> On 2019/3/12 下午5:42, Thanneeru Srinivasulu wrote:
>>>>>>>> Thanks Bruce..
>>>>>>>>
>>>>>>>> On Tue, Mar 12, 2019 at 3:08 PM Bruce Richardson
>>>>>>>> <bruce.richardson@intel.com> wrote:
>>>>>>>>> On Tue, Mar 12, 2019 at 10:57:55AM +0530, Thanneeru 
>>>>>>>>> Srinivasulu wrote:
>>>>>>>>>> Hi Everyone.
>>>>>>>>>>
>>>>>>>>>> I did attached pice  to Guest VM using vfio-pci with qemu 
>>>>>>>>>> command, and then
>>>>>>>>>> tried binding the pcie bdf with vfio-pci, observing binding 
>>>>>>>>>> failure with
>>>>>>>>>> vfio-pci.
>>>>>>>>>>
>>>>>>>>>> Where as when tryied with igb_uio, everything works fine.
>>>>>>>>>>
>>>>>>>>>> Does Binding with vfio-pci is supported inside VM/guest?
>>>>>>>>>>
>>>>>>>>> vfio support requires the presence of an IOMMU, and you 
>>>>>>>>> generally don't
>>>>>>>>> have an IOMMU available in a VM.
>>>>>>>>>
>>>>>>>>> /Bruce
>>>>>>>
>>>>>>>
>>>>>>> Actually, Qemu support vIOMMU + VFIO in guest[1], all you need 
>>>>>>> is to add a
>>>>>>> intel IOMMU and enabling caching mode.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>> https://www.lfasiallc.com/wp-content/uploads/2017/11/Device-Assignment-with-Nested-Guests-and-DPDK_Peter-Xu.pdf 
>>>>>>>
>>>>>>>
>>>>>> Thanks for the info.
>>>>>>
>>>>>> /Bruce
>>>>>>
>>>>>
>>>>> One more thing: even without vIOMMU, VFIO has no-IOMMU mode which 
>>>>> can be enabled (for a recent-enough kernel). This will make VFIO 
>>>>> work even in cases where the guest doesn't have IOMMU emulation. 
>>>>> See? There's no reason to use igb_uio, ever! :D
>>>>>
>>>>
>>>>
>>>
>>>
>>
> .
>