Re: [dpdk-dev] [PATCH v1 1/1] kernel/linux: introduce vfio_pf kernel module

[Andrew Rybchenko <arybchenko@solarflare.com>]

On 9/25/19 7:06 AM, Vamsi Krishna Attunuru wrote:
> Hi,
>
> We would like to have these support in 19.11. Please let us know if there are any comments or suggestions, we can discuss and address accordingly.
>
> -----Original Message-----
> From: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> Sent: Friday, September 6, 2019 6:58 PM
> To: Thomas Monjalon <thomas@monjalon.net>; Vamsi Krishna Attunuru <vattunuru@marvell.com>
> Cc: dev@dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v1 1/1] kernel/linux: introduce vfio_pf kernel module
>
>> -----Original Message-----
>> From: Thomas Monjalon <thomas@monjalon.net>
>> Sent: Friday, September 6, 2019 3:15 PM
>> To: Vamsi Krishna Attunuru <vattunuru@marvell.com>
>> Cc: dev@dpdk.org; Jerin Jacob Kollanukkaran <jerinj@marvell.com>
>> Subject: Re: [dpdk-dev] [PATCH v1 1/1] kernel/linux: introduce vfio_pf
>> kernel module
>>
>> 06/09/2019 11:12, vattunuru@marvell.com:
>>> From: Vamsi Attunuru <vattunuru@marvell.com>
>>>
>>> The DPDK use case such as VF representer or OVS offload etc would
>>> call for PF and VF PCIe devices to bind vfio-pci module to enable
>>> IOMMU protection.
>>>
>>> In addition to vSwitch use case, unlike, other PCI class of devices,
>>> Network class of PCIe devices would have additional responsibility
>>> on the PF devices such as promiscuous mode support etc.
>>>
>>> The above use cases demand VFIO needs bound to PF and its VF devices.
>>> This is use case is not supported in Linux kernel, due to a security
>>> issue where it is possible to have DoS in case if VF attached to
>>> guest over vfio-pci and netdev kernel driver runs on it and which
>>> something VF representer would like to enable it.
>>>
>>> Since we can not differentiate, the vfio-pci bounded VF devices runs
>>> DPDK application or netdev driver in guest, we can not introduce any
>>> scheme to fix DoS case and therefore not have proper support of this
>>> in the upstream kernel.
>>>
>>> The igb_uio enables such PF and VF binding support for non-iommu
>>> devices to make VF representer or OVS offload run on non-iommu
>>> devices with DoS vulnerability for netdev driver as VF.
>>>
>>> This kernel module, facilitate to enable SRIOV on PF devices,
>>> therefore, to run both PF and VF devices in VFIO mode knowing its
>>> impacts like igb_uio driver functions of non-iommu devices.
>>>
>>> Signed-off-by: Vamsi Attunuru <vattunuru@marvell.com>
>>> Signed-off-by: Jerin Jacob <jerinj@marvell.com>
>> Sorry I fail to properly understand the explanation above.
>> Please try to split in shorter sentences.
>>
>> About the request to add an out-of-tree Linux kernel driver, I guess
>> Jerin is well aware that we don't want such anymore.
> Yes. I am aware of it. I don't like the out of tree modules either. But, This case, I suggested Vamsi to have out of tree module.
>
> Let me describe the issue and let us discuss how to tackle the  problem:
>
> # Linux kernel wont allow VFIO PF to have SRIOV enable.
>
> Patches and on going discussion are here:
> https://patchwork.kernel.org/patch/10522381/
> https://lwn.net/Articles/748526/
>
> Based on my understanding the reason for NOT allowing the VFIO PF to have SRIOV enable is genuine from kernel point of View but not from DPDK point of view.
>
> Here is the sequence  to describe the problem
> 1) Consider Linux kernel allowed VFIO PCI SRIOV enable
> 2) PF bound to vfio-pci
> 3) using SRIOV infrastructure of vfio-pci  PF driver, VFs  are created
> 4) DPDK application bound to PF and VF, No issue here.
> 5) Assume DPDK application bound to PF and VF bound To netdev kernel driver. Now, there is a genuine  concern From kernel point of view that, DPDK PF can intercept, VF mailbox message or so and deny the Kernel request Or what if DPDK PF application crashes?
>
> To avoid the case (5), (3) is not allowed in stock kernel.
> Which makes sense IMO.
>
> Now, From DPDK PoV, step 5 is valid as we have Rte_flow's VF action etc used to enable such case.
> Where, user can program the PF's rte_flow to steer Some traffic to VF, where VF can be, DPDK application or Linux kernel netdev driver.
>
> This patch enables the step (3) to enable step (5) from DPDK PoV. i.e DPDK needs to allow PF to bind to DPDK with VFs.
>
> Why this issue now:
> - igb_uio kernel driver is used as enabling step (3) See store_max_vfs() kernel/linux/igb_uio/igb_uio.c  This is fine for non-iommu device, IOMMU devices needs VFIO.
> - We would like support VFIO for IOMMU protection And enable step (5) as DPDK supports form the spec level.
> i.e need to fix feature disparity between iommu vs non-iommu based devices.
>
> Note:
> We may not need a  brand new kernel module, we could move this logic to igb_uio if maintenance is concern.

It is really a problem for not bifurcated drivers and I agree with Jerin 
above.

One driver could be better, but I'm not sure and I'm afraid it will make too
many changes in igb_uio and result in instabilities/bugs. I have no strong
opinion if we can take the risk with igb_uio - simply have no enough
information if it is widely used or Linux in-kernel drivers are preferred.

Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>