[dpdk-dev] AWS i3.metal and IOMMU

[dpdk-dev] AWS i3.metal and IOMMU

[fwefew 4t4tg]

As per https://doc.dpdk.org/guides/nics/ena.html section 17.9
enabling IOMMU on an AWS i3 metal instance is as simple as adding "iommu=1
intel_iommu=on" to /etc/default/grub, update-grub, and reboot.

I can't get this to work. Once done, I cannot ssh back into the instance;
EC2 console can't get good status checks. It has to be terminated. My
config:

* Ubuntu 20.04 LTS x86 64-bit stock AWS AMI
* i3.metal (spot)

But more importantly: if one has provisioned a i3.metal instance, why
enable IOMMU at all? Can't one just run vfio-pci in
enable_unsafe_noiommu_mode?

Even if IOMMU runs in enable_unsafe_noiommu_mode, I'd like to reconfirm the
steps to make DPDK work on an AWS i3.metal instance. My steps are:

* git clone https://github.com/amzn/amzn-drivers.git
* git clone git://dpdk.org/dpdk and checkout tag 'v19.11-rc4'
* in dpdk git am in the patches from amzn-drivers 19.11 userspace dir
* build dpdk with:  make install T=x86_64-native-linuxapp-gcc
DESTDIR=/home/ubuntu/local
* then follow the instructions at
https://github.com/amzn/amzn-drivers/tree/master/userspace/dpdk/enav2-vfio-patch
to build and install an updated vfio kernel module

Is that complete and correct?

Re: [dpdk-dev] AWS i3.metal and IOMMU

[David Marchand]

On Wed, Oct 27, 2021 at 2:55 AM fwefew 4t4tg <7532yahoo@gmail.com> wrote:
>
> As per https://doc.dpdk.org/guides/nics/ena.html section 17.9
> enabling IOMMU on an AWS i3 metal instance is as simple as adding "iommu=1
> intel_iommu=on" to /etc/default/grub, update-grub, and reboot.
>
> I can't get this to work. Once done, I cannot ssh back into the instance;
> EC2 console can't get good status checks. It has to be terminated. My
> config:
>
> * Ubuntu 20.04 LTS x86 64-bit stock AWS AMI
> * i3.metal (spot)
>
> But more importantly: if one has provisioned a i3.metal instance, why
> enable IOMMU at all? Can't one just run vfio-pci in
> enable_unsafe_noiommu_mode?
>
> Even if IOMMU runs in enable_unsafe_noiommu_mode, I'd like to reconfirm the
> steps to make DPDK work on an AWS i3.metal instance. My steps are:
>
> * git clone https://github.com/amzn/amzn-drivers.git
> * git clone git://dpdk.org/dpdk and checkout tag 'v19.11-rc4'
> * in dpdk git am in the patches from amzn-drivers 19.11 userspace dir
> * build dpdk with:  make install T=x86_64-native-linuxapp-gcc
> DESTDIR=/home/ubuntu/local
> * then follow the instructions at
> https://github.com/amzn/amzn-drivers/tree/master/userspace/dpdk/enav2-vfio-patch
> to build and install an updated vfio kernel module
>
> Is that complete and correct?

- Let's ask the ENA DPDK maintainers.
They should know best.


- While at it, what is the status of upstreaming this write combined
support in vfio?
I found a RFC from 2017, and did not find anything else.
We are trying to stick to upstream support, so mentioning in DPDK a
non upstream kernel patch is not cool.


-- 
David Marchand