Re: [dpdk-dev] [PATCH] eal: force IOVA PA mode if KNI module inserted

[Ferruh Yigit <ferruh.yigit@intel.com>]

On 11/1/2017 8:58 PM, santosh wrote:
> Hi Ferruh,
> 
> 
> On Thursday 02 November 2017 05:36 AM, Ferruh Yigit wrote:
>> Fix kernel crash with KNI because KNI requires physical addresses.
>>
>> When IOVA VA mode used, memzones and mbufs physical address fields
>> contain virtual addresses. But KNI relies on these fields to enable
>> kernel access for buffers. Those fields having virtual address cause
>> crash in kernel.
>>
>> This is a workaround until KNI fixed properly to work with virtual
>> addresses.
>>
>> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy")
>>
>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
>> ---
>> Cc: Jianfeng Tan <jianfeng.tan@intel.com>
>> Cc: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>> Cc: Santosh Shukla <santosh.shukla@caviumnetworks.com>
>> Cc: Thomas Monjalon <thomas@monjalon.net>
>>
>> This patch superseded following one:
>> http://dpdk.org/dev/patchwork/patch/31071/
>> ---
>>  lib/librte_eal/linuxapp/eal/eal.c | 9 +++++++++
>>  1 file changed, 9 insertions(+)
>>
>> diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
>> index 017c402ed..29912a4e5 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal.c
>> @@ -808,6 +808,15 @@ rte_eal_init(int argc, char **argv)
>>  	/* autodetect the iova mapping mode (default is iova_pa) */
>>  	rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class();
>>  
>> +	/* Workaround for KNI which requires physical address to work */
>> +	if (rte_eal_get_configuration()->iova_mode == RTE_IOVA_VA &&
>> +			rte_eal_check_module("rte_kni") == 1) {
>> +		rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA;
>> +		RTE_LOG(WARNING, EAL,
>> +			"Some devices want iova as va but pa will be used because.. "
>> +			"KNI module inserted\n");
>> +	}
>> +
>>  	if (internal_config.no_hugetlbfs == 0 &&
>>  			internal_config.process_type != RTE_PROC_SECONDARY &&
>>  			eal_hugepage_info_init() < 0) {
> 
> Rather checking for KNI module at linuxapp/eal, I was suggesting to move
> KNI detection code in bus layer like below:

I was thinking same at first, but later was not sure about add this check into
bus related file, kni module check is so unrelated with pci.c .

Also this is not part of a regular process of detecting iova mode, which putting
into rte_pci_get_iommu_class() will look like it is. Instead this is a
workaround and we are overwriting the result of regular method for this special
case, so I think putting this into linux eal.c level after detection is good for
stressing the workaround.

So lets not spoil the logic in rte_pci_get_iommu_class() with kni module checks.

> 
> diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c
> index cdf8106..971586c 100644
> --- a/drivers/bus/pci/linux/pci.c
> +++ b/drivers/bus/pci/linux/pci.c
> @@ -586,11 +586,13 @@ rte_pci_get_iommu_class(void)
>         bool is_vfio_noiommu_enabled = true;
>         bool has_iova_va;
>         bool is_bound_uio;
> +       bool has_kni;
>  
>         is_bound = pci_one_device_is_bound();
>         if (!is_bound)
>                 return RTE_IOVA_DC;
>  
> +       has_kni = rte_eal_check_module("rte_kni");
>         has_iova_va = pci_one_device_has_iova_va();
>         is_bound_uio = pci_one_device_bound_uio();
>  #ifdef VFIO_PRESENT
> @@ -598,7 +600,7 @@ rte_pci_get_iommu_class(void)
>                                         true : false;
>  #endif
>  
> -       if (has_iova_va && !is_bound_uio && !is_vfio_noiommu_enabled)
> +       if (has_iova_va && !is_bound_uio && !has_kni && !is_vfio_noiommu_enabled)
>                 return RTE_IOVA_VA;
>  
>         if (has_iova_va) {
> @@ -607,6 +609,9 @@ rte_pci_get_iommu_class(void)
>                         RTE_LOG(WARNING, EAL, "vfio-noiommu mode configured\n");
>                 if (is_bound_uio)
>                         RTE_LOG(WARNING, EAL, "few device bound to UIO\n");
> +               if (has_kni)
> +                       RTE_LOG(WARNING, EAL, "Some devices want iova as va but pa will be used because.."
> +                                       "KNI module inserted\n");
>         }
>  
>         return RTE_IOVA_PA;
> It builds but I haven;t tested, can you please give it a try.
> 
> Thanks.
>