Re: [dpdk-dev] [PATCH 2/2] net: enable IOVA mode for PMDs

[Maxime Coquelin <maxime.coquelin@redhat.com>]

On 01/05/2018 11:32 AM, Maxime Coquelin wrote:
> Hi Jianfeng,
> 
> On 10/11/2017 12:33 PM, Jianfeng Tan wrote:
>> If we want to enable IOVA mode, introduced by
>> commit 93878cf0255e ("eal: introduce helper API for IOVA mode"),
>> we need PMDs (for PCI devices) to expose this flag.
>>
>> Signed-off-by: Jianfeng Tan<jianfeng.tan@intel.com>
>> ---
>>   drivers/net/e1000/em_ethdev.c     | 3 ++-
>>   drivers/net/e1000/igb_ethdev.c    | 5 +++--
>>   drivers/net/fm10k/fm10k_ethdev.c  | 3 ++-
>>   drivers/net/i40e/i40e_ethdev.c    | 3 ++-
>>   drivers/net/i40e/i40e_ethdev_vf.c | 2 +-
>>   drivers/net/ixgbe/ixgbe_ethdev.c  | 5 +++--
>>   6 files changed, 13 insertions(+), 8 deletions(-)
> 
> This patch introduces a regression when doing device assignment in
> guest, because current VT-d emulation only supports 39bits guest address
> width [0].
> 
> In the Bz, Peter suggest we could have an IOVA allocator algorithm,
> which could start to allocate IOVAs from 0. I think it could solve the
> --no-huge case your series address, do you agree?
> 
> But it would be a long term solution, we need to fix this in stable.
> 
> Is the --no-huge option used in production, or is it only for testing?
> If the latter do you think we could revert your patch while we find a
> solution that makes all cases to work?

It seems that we can get Intel IOMMU's Guest Address Width from the
sysfs, as the CAP register is exposed.

So we can get the SAGAW value (see [1], page 217):

On Bare Metal:
# echo $(((0x`cat /sys/class/iommu/dmar0/intel-iommu/cap` >> 8) & 0x1f))
4
=> 48bits

In guest:
# echo $(((0x`cat /sys/class/iommu/dmar0/intel-iommu/cap` >> 8) & 0x1f))
2
=> 39bits

Using this, we could or not allow the VA mode when using Intel IOMMU.
Any thoughts?

Regards,
Maxime

[1]: 
https://software.intel.com/sites/default/files/managed/c5/15/vt-directed-io-spec.pdf

> Ferruh, I see you also faced problems with KNI, how did you solved it?
> 
> Thanks,
> Maxime
> 
> [0]: https://bugzilla.redhat.com/show_bug.cgi?id=1530957#c3