From: Jerin Jacob <jerin.jacob@caviumnetworks.com>
To: "Tan, Jianfeng" <jianfeng.tan@intel.com>
Cc: Ferruh Yigit <ferruh.yigit@intel.com>,
Thomas Monjalon <thomas@monjalon.net>,
Bruce Richardson <bruce.richardson@intel.com>,
Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>,
dev@dpdk.org, Santosh Shukla <santosh.shukla@caviumnetworks.com>
Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default
Date: Wed, 1 Nov 2017 13:56:44 +0530 [thread overview]
Message-ID: <20171101082642.GA30967@jerin> (raw)
In-Reply-To: <d6755d4c-a60a-c50d-1b0b-647b73d2c87d@intel.com>
-----Original Message-----
> Date: Wed, 1 Nov 2017 15:39:01 +0800
> From: "Tan, Jianfeng" <jianfeng.tan@intel.com>
> To: Jerin Jacob <jerin.jacob@caviumnetworks.com>, Ferruh Yigit
> <ferruh.yigit@intel.com>
> CC: Thomas Monjalon <thomas@monjalon.net>, Bruce Richardson
> <bruce.richardson@intel.com>, Sergio Gonzalez Monroy
> <sergio.gonzalez.monroy@intel.com>, dev@dpdk.org, Santosh Shukla
> <santosh.shukla@caviumnetworks.com>
> Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default
> User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101
> Thunderbird/45.8.0
>
>
>
> On 11/1/2017 2:54 PM, Jerin Jacob wrote:
> > -----Original Message-----
> > > Date: Tue, 31 Oct 2017 23:21:18 -0700
> > > From: Ferruh Yigit <ferruh.yigit@intel.com>
> > > To: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > > CC: Thomas Monjalon <thomas@monjalon.net>, Bruce Richardson
> > > <bruce.richardson@intel.com>, Sergio Gonzalez Monroy
> > > <sergio.gonzalez.monroy@intel.com>, dev@dpdk.org, Jianfeng Tan
> > > <jianfeng.tan@intel.com>, Santosh Shukla
> > > <santosh.shukla@caviumnetworks.com>
> > > Subject: Re: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default
> > > User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
> > > Thunderbird/52.4.0
> > >
> > > On 10/31/2017 9:02 PM, Jerin Jacob wrote:
> > > > -----Original Message-----
> > > > > Date: Wed, 1 Nov 2017 01:07:26 +0000
> > > > > From: Ferruh Yigit <ferruh.yigit@intel.com>
> > > > > To: Thomas Monjalon <thomas@monjalon.net>, Bruce Richardson
> > > > > <bruce.richardson@intel.com>, Sergio Gonzalez Monroy
> > > > > <sergio.gonzalez.monroy@intel.com>
> > > > > CC: dev@dpdk.org, Ferruh Yigit <ferruh.yigit@intel.com>, Jianfeng Tan
> > > > > <jianfeng.tan@intel.com>, Santosh Shukla
> > > > > <santosh.shukla@caviumnetworks.com>
> > > > > Subject: [dpdk-dev] [PATCH] eal: disable IOVA mode detection by default
> > > > > X-Mailer: git-send-email 2.13.6
> > > > >
> > > > > Fix kernel crash with KNI because KNI requires physical addresses.
> > > > The actual fix would be to make KNI IOMMU aware based on the DPDK mode.
> > > >
> > > > ie. On slow path,
> > > >
> > > > /* Get iommu domain for iova to physical addr conversion */
> > > > if (rte_eal_iova_mode() == RTE_IOVA_VA)
> > > > kni->iommu_domain = iommu_get_domain_for_dev(dev);
> > > > else
> > > > kni->iommu_domain = NULL;
> > > >
> > > > On fast path,
> > > >
> > > > static inline u64 kni_iova_to_phys(struct ... *kni, dma_addr_t dma_addr)
> > > > {
> > > > /* Translation is installed only when IOMMU is present */
> > > > if (kni->iommu_domain)
> > > > return iommu_iova_to_phys(kni->iommu_domain, dma_addr);
> > > > return dma_addr;
> > > > }
> > > >
> > > > > A config option introduced to disable IOVA mode detection and to set it
> > > > > to physical address by default. Disabling config option will enable IOVA
> > > > > mode detection.
> > > > >
> > > > > When there is no intension to use KNI, it is safe to enable detection.
> > > > >
> > > > > Config option disable IOVA mode detection by default to be sure only who
> > > > > is aware of result enable it.
> > > > >
> > > > > Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy")
> > > > >
> > > > > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > > > > ---
> > > > > Cc: Jianfeng Tan <jianfeng.tan@intel.com>
> > > > > Cc: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> > > > > Cc: Thomas Monjalon <thomas@monjalon.net>
> > > > > ---
> > > > > config/common_base | 5 +++++
> > > > > lib/librte_eal/bsdapp/eal/eal.c | 4 ++++
> > > > > lib/librte_eal/linuxapp/eal/eal.c | 4 ++++
> > > > > 3 files changed, 13 insertions(+)
> > > > >
> > > > > diff --git a/config/common_base b/config/common_base
> > > > > index 82ee75456..903e7685b 100644
> > > > > --- a/config/common_base
> > > > > +++ b/config/common_base
> > > > > @@ -107,6 +107,11 @@ CONFIG_RTE_MALLOC_DEBUG=n
> > > > > CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n
> > > > > #
> > > > > +# Disabling PHYS_IOVA may crash kernel for KNI, use with caution
> > > > > +#
> > > > > +CONFIG_RTE_EAL_USE_PHYS_IOVA=y
> > > > Defeat the purpose of all dynamic probing scheme.
> > > > Either we can fix the KNI or revert the following patch for this release.
> > > >
> > > > http://dpdk.org/commit/f37dfab2
> > > This commit just enables IOVA VA mode for Intel drivers, that is how I can able
> > > to observe the issue, but it is not the source of the problem. Reverting that
> > > commit will not solve KNI crash with any other PMD that enables IOVA VA mode.
> > I don't understand why a PMD needs to enable IOVA_VA if it can support IOVA_PA.
> > IMO, IOVA_VA should be enabled only for those device it can WORK ONLY on
> > IOVA_VA mode.
>
> IOVA_VA is also necessary if we don't want/have root privilege to run DPDK
> app.
OK.
>
> > Forget about KNI, If we set CONFIG_RTE_EAL_USE_PHYS_IOVA
> > as y then the normal stuff wont work for if PMD can operate only in
> > IOVA_VA mode(like octeontx).
>
> Yes, agreed.
>
> > Regarding the KNI crash, it can be avoid by first checking the exiting
> > mode(rte_eal_iova_mode()). i.e since legacy driver like KNI need real
> > physical address to work "now", it can grace full exit on the init time if
> > mode == IOVA_VA;
>
> For the KNI crash, do we really need to correct the case: make those PMDs
> (requires IOVA_VA) also work well with KNI?
> - If no, we just need to report RTE_PCI_DRV_IOVA_AS_VA smartly.
> - If yes, we shall make KNI work with VA.
I would say "yes". But for this immediate release, I would say "no".
>
> Thanks,
> Jianfeng
>
next prev parent reply other threads:[~2017-11-01 8:27 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-11-01 1:07 Ferruh Yigit
2017-11-01 2:17 ` Tan, Jianfeng
2017-11-01 6:32 ` Ferruh Yigit
2017-11-01 6:37 ` Tan, Jianfeng
2017-11-01 7:29 ` Ferruh Yigit
2017-11-01 7:55 ` Tan, Jianfeng
2017-11-01 3:54 ` santosh
2017-11-01 4:02 ` Jerin Jacob
2017-11-01 6:21 ` Ferruh Yigit
2017-11-01 6:54 ` Jerin Jacob
2017-11-01 7:39 ` Tan, Jianfeng
2017-11-01 8:26 ` Jerin Jacob [this message]
2017-11-01 8:53 ` Tan, Jianfeng
2017-11-01 18:31 ` Ferruh Yigit
2017-11-01 8:12 ` Ferruh Yigit
2017-11-01 10:31 ` Thomas Monjalon
2017-11-01 11:06 ` santosh
2017-11-01 18:22 ` Ferruh Yigit
2017-11-01 18:11 ` Ferruh Yigit
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20171101082642.GA30967@jerin \
--to=jerin.jacob@caviumnetworks.com \
--cc=bruce.richardson@intel.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=jianfeng.tan@intel.com \
--cc=santosh.shukla@caviumnetworks.com \
--cc=sergio.gonzalez.monroy@intel.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).