From: Ferruh Yigit <ferruh.yigit@intel.com>
To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Cc: Thomas Monjalon <thomas@monjalon.net>,
Bruce Richardson <bruce.richardson@intel.com>,
Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>,
dev@dpdk.org, Jianfeng Tan <jianfeng.tan@intel.com>,
Santosh Shukla <santosh.shukla@caviumnetworks.com>
Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default
Date: Tue, 31 Oct 2017 23:21:18 -0700 [thread overview]
Message-ID: <80719d01-7859-e6ac-ee1e-964a04357098@intel.com> (raw)
In-Reply-To: <20171101040251.GA13569@jerin>
On 10/31/2017 9:02 PM, Jerin Jacob wrote:
> -----Original Message-----
>> Date: Wed, 1 Nov 2017 01:07:26 +0000
>> From: Ferruh Yigit <ferruh.yigit@intel.com>
>> To: Thomas Monjalon <thomas@monjalon.net>, Bruce Richardson
>> <bruce.richardson@intel.com>, Sergio Gonzalez Monroy
>> <sergio.gonzalez.monroy@intel.com>
>> CC: dev@dpdk.org, Ferruh Yigit <ferruh.yigit@intel.com>, Jianfeng Tan
>> <jianfeng.tan@intel.com>, Santosh Shukla
>> <santosh.shukla@caviumnetworks.com>
>> Subject: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default
>> X-Mailer: git-send-email 2.13.6
>>
>> Fix kernel crash with KNI because KNI requires physical addresses.
>
> The actual fix would be to make KNI IOMMU aware based on the DPDK mode.
>
> ie. On slow path,
>
> /* Get iommu domain for iova to physical addr conversion */
> if (rte_eal_iova_mode() == RTE_IOVA_VA)
> kni->iommu_domain = iommu_get_domain_for_dev(dev);
> else
> kni->iommu_domain = NULL;
>
> On fast path,
>
> static inline u64 kni_iova_to_phys(struct ... *kni, dma_addr_t dma_addr)
> {
> /* Translation is installed only when IOMMU is present */
> if (kni->iommu_domain)
> return iommu_iova_to_phys(kni->iommu_domain, dma_addr);
> return dma_addr;
> }
>
>>
>> A config option introduced to disable IOVA mode detection and to set it
>> to physical address by default. Disabling config option will enable IOVA
>> mode detection.
>>
>> When there is no intension to use KNI, it is safe to enable detection.
>>
>> Config option disable IOVA mode detection by default to be sure only who
>> is aware of result enable it.
>>
>> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy")
>>
>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
>> ---
>> Cc: Jianfeng Tan <jianfeng.tan@intel.com>
>> Cc: Santosh Shukla <santosh.shukla@caviumnetworks.com>
>> Cc: Thomas Monjalon <thomas@monjalon.net>
>> ---
>> config/common_base | 5 +++++
>> lib/librte_eal/bsdapp/eal/eal.c | 4 ++++
>> lib/librte_eal/linuxapp/eal/eal.c | 4 ++++
>> 3 files changed, 13 insertions(+)
>>
>> diff --git a/config/common_base b/config/common_base
>> index 82ee75456..903e7685b 100644
>> --- a/config/common_base
>> +++ b/config/common_base
>> @@ -107,6 +107,11 @@ CONFIG_RTE_MALLOC_DEBUG=n
>> CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n
>>
>> #
>> +# Disabling PHYS_IOVA may crash kernel for KNI, use with caution
>> +#
>> +CONFIG_RTE_EAL_USE_PHYS_IOVA=y
>
> Defeat the purpose of all dynamic probing scheme.
> Either we can fix the KNI or revert the following patch for this release.
>
> http://dpdk.org/commit/f37dfab2
This commit just enables IOVA VA mode for Intel drivers, that is how I can able
to observe the issue, but it is not the source of the problem. Reverting that
commit will not solve KNI crash with any other PMD that enables IOVA VA mode.
Related to the KNI, iommu is not involved at all, I am not clear with your above
suggestion, physical address is required for kernelspace - userspace communication.
KNI uses physical address for two things:
1- Creates a buffer in userspace, a memzone, and shares its physical address to
the kernel, so that both kernel and app can access same buffer.
The question is, even all devices supports IOVA VA mode, why creating a memzone
that has real physical address info is not possible anymore?
For KNI case this is not related at all with what devices supports.
2- For each mbuf that will be sent to kernel, dpdk app puts physical address of
that mbuf into shared buffer, so that kernel can access it. So that kernel can
access to mbuf data that can be coming from any mempool.
For this the physical address of the mbuf is required.
Overall KNI needs to know physical address of mbufs and memzone.
This patch provides a way to have old behavior with a config option, so that KNI
can work and anyone who needs to create memzone with physical address can a way
to have it, and all PMDs will work fine.
If there is no need for these, it is possible to disable config and dynamic
probing scheme is already there.
Thanks,
ferruh
next prev parent reply other threads:[~2017-11-01 6:21 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-01 1:07 Ferruh Yigit
2017-11-01 2:17 ` Tan, Jianfeng
2017-11-01 6:32 ` Ferruh Yigit
2017-11-01 6:37 ` Tan, Jianfeng
2017-11-01 7:29 ` Ferruh Yigit
2017-11-01 7:55 ` Tan, Jianfeng
2017-11-01 3:54 ` santosh
2017-11-01 4:02 ` Jerin Jacob
2017-11-01 6:21 ` Ferruh Yigit [this message]
2017-11-01 6:54 ` Jerin Jacob
2017-11-01 7:39 ` Tan, Jianfeng
2017-11-01 8:26 ` Jerin Jacob
2017-11-01 8:53 ` Tan, Jianfeng
2017-11-01 18:31 ` Ferruh Yigit
2017-11-01 8:12 ` Ferruh Yigit
2017-11-01 10:31 ` Thomas Monjalon
2017-11-01 11:06 ` santosh
2017-11-01 18:22 ` Ferruh Yigit
2017-11-01 18:11 ` Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=80719d01-7859-e6ac-ee1e-964a04357098@intel.com \
--to=ferruh.yigit@intel.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=jerin.jacob@caviumnetworks.com \
--cc=jianfeng.tan@intel.com \
--cc=santosh.shukla@caviumnetworks.com \
--cc=sergio.gonzalez.monroy@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).