DPDK patches and discussions
 help / color / Atom feed
From: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: "Burakov, Anatoly" <anatoly.burakov@intel.com>,
	David Marchand <david.marchand@redhat.com>,
	"dev@dpdk.org" <dev@dpdk.org>,
	John McNamara <john.mcnamara@intel.com>,
	Marko Kovacevic <marko.kovacevic@intel.com>,
	"Igor Russkikh" <igor.russkikh@aquantia.com>,
	Pavel Belous <pavel.belous@aquantia.com>,
	Ajit Khaparde <ajit.khaparde@broadcom.com>,
	Somnath Kotur <somnath.kotur@broadcom.com>,
	Wenzhuo Lu <wenzhuo.lu@intel.com>,
	John Daley <johndale@cisco.com>,
	Hyong Youb Kim <hyonkim@cisco.com>,
	Qi Zhang <qi.z.zhang@intel.com>,
	Xiao Wang <xiao.w.wang@intel.com>,
	Beilei Xing <beilei.xing@intel.com>,
	Jingjing Wu <jingjing.wu@intel.com>,
	Qiming Yang <qiming.yang@intel.com>,
	"Konstantin Ananyev" <konstantin.ananyev@intel.com>,
	Matan Azrad <matan@mellanox.com>,
	Shahaf Shuler <shahafs@mellanox.com>,
	Yongseok Koh <yskoh@mellanox.com>,
	Viacheslav Ovsiienko <viacheslavo@mellanox.com>,
	Alejandro Lucero <alejandro.lucero@netronome.com>,
	Nithin Kumar Dabilpuram <ndabilpuram@marvell.com>,
	Kiran Kumar Kokkilagadda <kirankumark@marvell.com>,
	Rasesh Mody <rmody@marvell.com>,
	Shahed Shaikh <shshaikh@marvell.com>,
	Bruce Richardson <bruce.richardson@intel.com>,
	"alialnu@mellanox.com" <alialnu@mellanox.com>,
	"aconole@redhat.com" <aconole@redhat.com>
Subject: Re: [dpdk-dev] [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drivers
Date: Mon, 15 Jul 2019 16:27:52 +0000
Message-ID: <BYAPR18MB24243202E0928C6A3C3C9C7AC8CF0@BYAPR18MB2424.namprd18.prod.outlook.com> (raw)
In-Reply-To: <1689145.QdohnT2lii@xps>

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Monday, July 15, 2019 9:36 PM
> To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>
> Cc: Burakov, Anatoly <anatoly.burakov@intel.com>; David Marchand
> <david.marchand@redhat.com>; dev@dpdk.org; John McNamara
> <john.mcnamara@intel.com>; Marko Kovacevic
> <marko.kovacevic@intel.com>; Igor Russkikh
> <igor.russkikh@aquantia.com>; Pavel Belous <pavel.belous@aquantia.com>;
> Ajit Khaparde <ajit.khaparde@broadcom.com>; Somnath Kotur
> <somnath.kotur@broadcom.com>; Wenzhuo Lu <wenzhuo.lu@intel.com>;
> John Daley <johndale@cisco.com>; Hyong Youb Kim <hyonkim@cisco.com>;
> Qi Zhang <qi.z.zhang@intel.com>; Xiao Wang <xiao.w.wang@intel.com>;
> Beilei Xing <beilei.xing@intel.com>; Jingjing Wu <jingjing.wu@intel.com>;
> Qiming Yang <qiming.yang@intel.com>; Konstantin Ananyev
> <konstantin.ananyev@intel.com>; Matan Azrad <matan@mellanox.com>;
> Shahaf Shuler <shahafs@mellanox.com>; Yongseok Koh
> <yskoh@mellanox.com>; Viacheslav Ovsiienko
> <viacheslavo@mellanox.com>; Alejandro Lucero
> <alejandro.lucero@netronome.com>; Nithin Kumar Dabilpuram
> <ndabilpuram@marvell.com>; Kiran Kumar Kokkilagadda
> <kirankumark@marvell.com>; Rasesh Mody <rmody@marvell.com>; Shahed
> Shaikh <shshaikh@marvell.com>; Bruce Richardson
> <bruce.richardson@intel.com>; alialnu@mellanox.com;
> aconole@redhat.com
> Subject: Re: [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drivers
> 
> 15/07/2019 17:35, Jerin Jacob Kollanukkaran:
> > From: Thomas Monjalon <thomas@monjalon.net>
> > > 15/07/2019 16:26, Jerin Jacob Kollanukkaran:
> > > > > > Is there any specific reason why we always prefer PA if
> > > > > > physical addresses are available? Since we're already assuming
> > > > > > that all devices support PA and VA anyway, what's the harm in
> > > > > > enabling VA by
> > > default?
> > > > >
> > > > > If PA is available, it means we are running as root.
> > > > > We can assume that using root is a choice, probably related to a
> > > > > preference for PA.
> > > >
> > > > # Even if we are running as root, Why to choose PA in case of DC?
> > > > ie. Following logic is not need
> > > >                 if (iova_mode == RTE_IOVA_DC) {
> > > >                         iova_mode = phys_addrs ? RTE_IOVA_PA : RTE_IOVA_VA;
> > > >                         RTE_LOG(DEBUG, EAL,
> > > >                                 "Buses did not request a specific IOVA mode, using '%s'
> > > based on physical addresses availability.\n",
> > > >                                 phys_addrs ? "PA" : "VA");
> > > >                 }
> > >
> > > Why running as root if using VA anyway?
> > > We can assume the user knows what he is doing, so it is a user choice.
> > > We want to allow the user choosing, right?
> >
> > The user can override iova=pa/va as eal argument if user needs to run a
> specific mode.
> > Running as root for various other reason(just be lazy) etc. it is not
> > or it should not be connected to set the mode as PA.
> 
> Good point.
> I tend to prefer avoiding the use of EAL arguments because they may be
> unavailable, depending on the application.

Yes. The default case suffice the requirement here.ie when it DC chosen from
bus layer select the VA. I don't see any point in overriding that.
It is a good default. Do you think any case where it need to be "changed"?
If not, let stick with VA i.e until unless if there no HARD requirement for PA.
i.e Stayaway from PA WHEN possible.

> 
> > > > # When DPDK running on guest, Anyway it can not access the real
> > > > PA, It will
> > > be IPA.
> > >
> > > What is IPA? Isn't it a beer?
> >
> > There may a beer with that name. In this context, it is "Intermediate
> physical address"
> >
> > > > So I don't understand logic behind choose PA when DC.
> > > > To me, it make sense to choose PA when DC.
> > >
> > > You probably mean "choose VA".
> >
> > Yup.
> >
> > > > # To align with RTE_PCI_DRV_NEED_MAPPING flag and reflect it "need"
> > > > rather than support, I think, flag can be changed to
> > > > RTE_PCI_DRV_NEED_IOVA_AS_VA
> > >
> > > I think the most important is to have a good documentation of this
> > > flag (it was not done properly when Cavium introduced it initially).
> > > If you want to rename the flag, you can do it in a separate patch.
> > > If renaming, I really would like to get an answer to an old question:
> > > Why IO adress is called IOVA? The name "IOVA_AS_VA" looks strange.
> >
> > IOVA = IO virtual address
> > Since IOVA can be PA or VA, the name IOVA_AS_VA as chosen
> 
> We could also call it "bus address" or "device address".
> I think the word "IOVA" was enforced by Linux.
> Anyway, my real issue when using "virtual" is that we don't really know what
> we are talking about: is it an IOMMU translated address on the device side or
> an MMU translated address on the application side?

Actually in linux kernel, it creates the same mapping for device and CPU so 
user both can access the very same address. In this context, IOVA means,
Virtual address for device. The OS can do same mapping in CPU MMU tables as well.

> 
> I think we should better explain things.
> One diagram which can help:
> https://en.wikipedia.org/wiki/Input%E2%80%93output_memory_managem
> ent_unit#/media/File:MMU_and_IOMMU.svg
> 
> > > For reference, one description of addressing:
> > > https://lists.linuxfoundation.org/pipermail/iommu/2018-May/027686.ht
> > > ml
> > >
> > > About the naming, do you remember how I insisted to have a correct
> > > naming of all related stuff in DPDK? It was hard to get it accepted,
> > > the discussion was not nice and I stopped insisting to get all details fine
> because I just got bored.
> > > It was a really bad experience.
> >
> > I agree.
> > To me that bad experience was due to mostly not having enough
> > technical comments On the proposal. Though I am not the author/owner of
> it.
> >
> > > You can ask why I remind this now? Because we must take care of all
> > > details, make sure our messages are well understood, and be
> cooperative.
> >
> > No disagreement.
> > If we see the history the meaning got changed/updated in this commit
> > By adding intel drivers to it. I would nt say  it is big ideal, It
> > just C code, It can be changed based on the need. I think, what really
> > import is, maintain the the feature and commitment towards fixing any
> issue.
> >
> > commit f37dfab21c988d2d0ecb3c82be4ba9738c7e51c7
> > Author: Jianfeng Tan <jianfeng.tan@intel.com>
> > Date:   Wed Oct 11 10:33:48 2017 +0000
> >
> >     drivers/net: enable IOVA mode for Intel PMDs
> >
> >     If we want to enable IOVA mode, introduced by
> >     commit 93878cf0255e ("eal: introduce helper API for IOVA mode"),
> >     we need PMDs (for PCI devices) to expose this flag.
> >
> >     Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> >     Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> >     Reviewed-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> 
> The doxygen meaning did not change from day one:
> 	/** Device driver supports IOVA as VA */ But the commit log
> meaning was:
> 	"Flag used when driver needs to operate in iova=va mode."
> And the Intel commit log had a different understanding:
> 	"If we want to enable IOVA mode, [..] we need PMDs [..] to expose
> this flag."
> 
> Anyway we agree on the new meaning to be the original one the author had
> in mind (i.e. "driver needs").
> 
> > > > Other than above points,
> > > > Reviewed this patch and tested on octeontx2, It looks good to me.
> 
> 


  reply index

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-10 21:48 [dpdk-dev] [PATCH 0/2] Fixes on IOVA mode selection David Marchand
2019-07-10 21:48 ` [dpdk-dev] [PATCH 1/2] Revert "bus/pci: add Mellanox kernel driver type" David Marchand
2019-07-16 10:37   ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-10 21:48 ` [dpdk-dev] [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drivers David Marchand
2019-07-11 14:40   ` Thomas Monjalon
2019-07-12  8:05     ` Jerin Jacob Kollanukkaran
2019-07-12 11:03   ` Burakov, Anatoly
2019-07-12 12:43     ` Thomas Monjalon
2019-07-12 12:58       ` Burakov, Anatoly
2019-07-12 13:19         ` Bruce Richardson
2019-07-15 14:26       ` Jerin Jacob Kollanukkaran
2019-07-15 15:03         ` Thomas Monjalon
2019-07-15 15:35           ` Jerin Jacob Kollanukkaran
2019-07-15 16:06             ` Thomas Monjalon
2019-07-15 16:27               ` Jerin Jacob Kollanukkaran [this message]
2019-07-16 13:46 ` [dpdk-dev] [PATCH v2 0/4] Fixes on IOVA mode selection jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 1/4] Revert "bus/pci: add Mellanox kernel driver type" jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 2/4] eal: fix IOVA mode selection as VA for pci drivers jerinj
2019-07-16 14:26     ` Burakov, Anatoly
2019-07-16 15:07       ` Jerin Jacob Kollanukkaran
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 3/4] eal: change RTE_PCI_DRV_IOVA_AS_VA flag name jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 4/4] eal: select IOVA mode as VA for default case jerinj
2019-07-16 14:33     ` Burakov, Anatoly
2019-07-17  8:33       ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-17 12:38         ` Burakov, Anatoly
2019-07-17 14:04           ` Jerin Jacob Kollanukkaran
2019-07-18  6:45   ` [dpdk-dev] [PATCH v3 0/4] Fixes on IOVA mode selection jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 1/4] Revert "bus/pci: add Mellanox kernel driver type" jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 2/4] eal: fix IOVA mode selection as VA for pci drivers jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 3/4] eal: change RTE_PCI_DRV_IOVA_AS_VA flag name jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 4/4] eal: select IOVA mode as VA for default case jerinj
2019-07-22 11:28     ` [dpdk-dev] [PATCH v3 0/4] Fixes on IOVA mode selection David Marchand
2019-07-22 12:56 ` [dpdk-dev] [PATCH v4 " David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 1/4] Revert "bus/pci: add Mellanox kernel driver type" David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers David Marchand
2019-11-25  9:33     ` Ferruh Yigit
2019-11-25 10:22       ` Thomas Monjalon
2019-11-25 12:03         ` Ferruh Yigit
2019-11-25 12:36           ` David Marchand
2019-11-25 12:58             ` Burakov, Anatoly
2019-11-25 14:29               ` Thomas Monjalon
2019-11-25 11:07       ` Jerin Jacob
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 3/4] drivers: change IOVA as VA PCI flag name David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 4/4] eal: select IOVA as VA mode for default case David Marchand
2019-07-22 15:53   ` [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode selection Thomas Monjalon
2019-07-23  3:35     ` Stojaczyk, Dariusz
2019-07-23  4:18       ` Jerin Jacob Kollanukkaran
2019-07-23  4:54         ` Stojaczyk, Dariusz
2019-07-23  5:27           ` Jerin Jacob Kollanukkaran
2019-07-23  7:21             ` Thomas Monjalon
2019-07-23  9:57             ` Burakov, Anatoly
2019-07-23 10:25               ` Thomas Monjalon
2019-07-23 13:56                 ` Burakov, Anatoly
2019-07-23 14:24                   ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-23 14:29                   ` [dpdk-dev] " Burakov, Anatoly
2019-07-23 14:36                     ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-23 15:47                       ` Burakov, Anatoly

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BYAPR18MB24243202E0928C6A3C3C9C7AC8CF0@BYAPR18MB2424.namprd18.prod.outlook.com \
    --to=jerinj@marvell.com \
    --cc=aconole@redhat.com \
    --cc=ajit.khaparde@broadcom.com \
    --cc=alejandro.lucero@netronome.com \
    --cc=alialnu@mellanox.com \
    --cc=anatoly.burakov@intel.com \
    --cc=beilei.xing@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=hyonkim@cisco.com \
    --cc=igor.russkikh@aquantia.com \
    --cc=jingjing.wu@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=johndale@cisco.com \
    --cc=kirankumark@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=matan@mellanox.com \
    --cc=ndabilpuram@marvell.com \
    --cc=pavel.belous@aquantia.com \
    --cc=qi.z.zhang@intel.com \
    --cc=qiming.yang@intel.com \
    --cc=rmody@marvell.com \
    --cc=shahafs@mellanox.com \
    --cc=shshaikh@marvell.com \
    --cc=somnath.kotur@broadcom.com \
    --cc=thomas@monjalon.net \
    --cc=viacheslavo@mellanox.com \
    --cc=wenzhuo.lu@intel.com \
    --cc=xiao.w.wang@intel.com \
    --cc=yskoh@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox