DPDK patches and discussions
 help / color / Atom feed
From: Jerin Jacob <jerinjacobk@gmail.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>
Cc: David Marchand <david.marchand@redhat.com>,
	Ajit Khaparde <ajit.khaparde@broadcom.com>,
	 Alejandro Lucero <alejandro.lucero@netronome.com>,
	 Anatoly Burakov <anatoly.burakov@intel.com>,
	Beilei Xing <beilei.xing@intel.com>,
	 "Richardson, Bruce" <bruce.richardson@intel.com>,
	dpdk-dev <dev@dpdk.org>,  Hyong Youb Kim <hyonkim@cisco.com>,
	igor.russkikh@aquantia.com,  Jerin Jacob <jerinj@marvell.com>,
	Jingjing Wu <jingjing.wu@intel.com>,
	 John McNamara <john.mcnamara@intel.com>,
	John Daley <johndale@cisco.com>,
	 Kiran Kumar K <kirankumark@marvell.com>,
	 "Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
	Marko Kovacevic <marko.kovacevic@intel.com>,
	 Matan Azrad <matan@mellanox.com>,
	Nithin Dabilpuram <ndabilpuram@marvell.com>,
	pavel.belous@aquantia.com,  Qi Zhang <qi.z.zhang@intel.com>,
	Qiming Yang <qiming.yang@intel.com>,
	 Rasesh Mody <rmody@marvell.com>,
	Shahaf Shuler <shahafs@mellanox.com>,
	 Shahed Shaikh <shshaikh@marvell.com>,
	Somnath Kotur <somnath.kotur@broadcom.com>,
	 Thomas Monjalon <thomas@monjalon.net>,
	Slava Ovsiienko <viacheslavo@mellanox.com>,
	 Wenzhuo Lu <wenzhuo.lu@intel.com>,
	Xiao Wang <xiao.w.wang@intel.com>,
	 Yongseok Koh <yskoh@mellanox.com>
Subject: Re: [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers
Date: Mon, 25 Nov 2019 20:07:35 +0900
Message-ID: <CALBAE1Mvei0W3rPd7jguBcLXgZQ9RJU21gSBOcK-43cNRfRrhg@mail.gmail.com> (raw)
In-Reply-To: <20191125093313.61700-1-ferruh.yigit@intel.com>

On Mon, Nov 25, 2019 at 6:33 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
>
> On 7/22/2019 1:56 PM, David Marchand wrote:
> > The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which
> > was intended to mean "driver only supports VA" but had been understood
> > as "driver supports both PA and VA" by most net drivers and used to let
> > dpdk processes to run as non root (which do not have access to physical
> > addresses on recent kernels).
> >
> > The check on physical addresses actually closed the gap for those
> > drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this
> > flag can retain its intended meaning.
> > Document explicitly its meaning.
> >
> > We can check that a driver requirement wrt to IOVA mode is fulfilled
> > before trying to probe a device.
> >
> > Finally, document the heuristic used to select the IOVA mode and hope
> > that we won't break it again.
> >
> > Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode")
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > Reviewed-by: Jerin Jacob <jerinj@marvell.com>
> > Tested-by: Jerin Jacob <jerinj@marvell.com>
> > Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> <...>
>
> > diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
> > index d2af472..9794552 100644
> > --- a/drivers/bus/pci/pci_common.c
> > +++ b/drivers/bus/pci/pci_common.c
> > @@ -169,8 +169,22 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
> >        * This needs to be before rte_pci_map_device(), as it enables to use
> >        * driver flags for adjusting configuration.
> >        */
> > -     if (!already_probed)
> > +     if (!already_probed) {
> > +             enum rte_iova_mode dev_iova_mode;
> > +             enum rte_iova_mode iova_mode;
> > +
> > +             dev_iova_mode = pci_device_iova_mode(dr, dev);
> > +             iova_mode = rte_eal_iova_mode();
> > +             if (dev_iova_mode != RTE_IOVA_DC &&
> > +                 dev_iova_mode != iova_mode) {
> > +                     RTE_LOG(ERR, EAL, "  Expecting '%s' IOVA mode but current mode is '%s', not initializing\n",
> > +                             dev_iova_mode == RTE_IOVA_PA ? "PA" : "VA",
> > +                             iova_mode == RTE_IOVA_PA ? "PA" : "VA");
> > +                     return -EINVAL;
> > +             }
> > +
>
> OvS reported an error while hotplugging a device.
>
> It looks like DPDK application initialized as IOVA=VA, and the new device is bound to 'igb_uio' which forces it to PA, fails on above check.

Why they are binding to igb_uio if there is NO need for it?

>
> I would like to get your comment on the issue.
>
> For the OvS mode, hopefully binding the device to 'vfio-pci' can be a solution, but for the cases we don't have that option, can/should we force the DPDK to PA mode after initialization?

On the other side, If we are forcing DPDK to PA then the same fate
will be for VFIO only devices.
There are two cases:
1) The system has a limitation on the specific mode
2) The devices have a limitation on the specific mod.e.

Case (1), It is not applicable for hotplug cases as the system can run
only one mode. We should able to detect in the first pass(Before the
hotplug devices runs)
Case (2), Is there any devices that can work ONLY in IOVA as PA mode?
If yes, Please enumerate. Maybe something in the storage domain.



>

  parent reply index

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-10 21:48 [dpdk-dev] [PATCH 0/2] Fixes on IOVA mode selection David Marchand
2019-07-10 21:48 ` [dpdk-dev] [PATCH 1/2] Revert "bus/pci: add Mellanox kernel driver type" David Marchand
2019-07-16 10:37   ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-10 21:48 ` [dpdk-dev] [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drivers David Marchand
2019-07-11 14:40   ` Thomas Monjalon
2019-07-12  8:05     ` Jerin Jacob Kollanukkaran
2019-07-12 11:03   ` Burakov, Anatoly
2019-07-12 12:43     ` Thomas Monjalon
2019-07-12 12:58       ` Burakov, Anatoly
2019-07-12 13:19         ` Bruce Richardson
2019-07-15 14:26       ` Jerin Jacob Kollanukkaran
2019-07-15 15:03         ` Thomas Monjalon
2019-07-15 15:35           ` Jerin Jacob Kollanukkaran
2019-07-15 16:06             ` Thomas Monjalon
2019-07-15 16:27               ` Jerin Jacob Kollanukkaran
2019-07-16 13:46 ` [dpdk-dev] [PATCH v2 0/4] Fixes on IOVA mode selection jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 1/4] Revert "bus/pci: add Mellanox kernel driver type" jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 2/4] eal: fix IOVA mode selection as VA for pci drivers jerinj
2019-07-16 14:26     ` Burakov, Anatoly
2019-07-16 15:07       ` Jerin Jacob Kollanukkaran
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 3/4] eal: change RTE_PCI_DRV_IOVA_AS_VA flag name jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 4/4] eal: select IOVA mode as VA for default case jerinj
2019-07-16 14:33     ` Burakov, Anatoly
2019-07-17  8:33       ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-17 12:38         ` Burakov, Anatoly
2019-07-17 14:04           ` Jerin Jacob Kollanukkaran
2019-07-18  6:45   ` [dpdk-dev] [PATCH v3 0/4] Fixes on IOVA mode selection jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 1/4] Revert "bus/pci: add Mellanox kernel driver type" jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 2/4] eal: fix IOVA mode selection as VA for pci drivers jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 3/4] eal: change RTE_PCI_DRV_IOVA_AS_VA flag name jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 4/4] eal: select IOVA mode as VA for default case jerinj
2019-07-22 11:28     ` [dpdk-dev] [PATCH v3 0/4] Fixes on IOVA mode selection David Marchand
2019-07-22 12:56 ` [dpdk-dev] [PATCH v4 " David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 1/4] Revert "bus/pci: add Mellanox kernel driver type" David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers David Marchand
2019-11-25  9:33     ` Ferruh Yigit
2019-11-25 10:22       ` Thomas Monjalon
2019-11-25 12:03         ` Ferruh Yigit
2019-11-25 12:36           ` David Marchand
2019-11-25 12:58             ` Burakov, Anatoly
2019-11-25 14:29               ` Thomas Monjalon
2019-11-25 11:07       ` Jerin Jacob [this message]
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 3/4] drivers: change IOVA as VA PCI flag name David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 4/4] eal: select IOVA as VA mode for default case David Marchand
2019-07-22 15:53   ` [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode selection Thomas Monjalon
2019-07-23  3:35     ` Stojaczyk, Dariusz
2019-07-23  4:18       ` Jerin Jacob Kollanukkaran
2019-07-23  4:54         ` Stojaczyk, Dariusz
2019-07-23  5:27           ` Jerin Jacob Kollanukkaran
2019-07-23  7:21             ` Thomas Monjalon
2019-07-23  9:57             ` Burakov, Anatoly
2019-07-23 10:25               ` Thomas Monjalon
2019-07-23 13:56                 ` Burakov, Anatoly
2019-07-23 14:24                   ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-23 14:29                   ` [dpdk-dev] " Burakov, Anatoly
2019-07-23 14:36                     ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-23 15:47                       ` Burakov, Anatoly

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CALBAE1Mvei0W3rPd7jguBcLXgZQ9RJU21gSBOcK-43cNRfRrhg@mail.gmail.com \
    --to=jerinjacobk@gmail.com \
    --cc=ajit.khaparde@broadcom.com \
    --cc=alejandro.lucero@netronome.com \
    --cc=anatoly.burakov@intel.com \
    --cc=beilei.xing@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=hyonkim@cisco.com \
    --cc=igor.russkikh@aquantia.com \
    --cc=jerinj@marvell.com \
    --cc=jingjing.wu@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=johndale@cisco.com \
    --cc=kirankumark@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=matan@mellanox.com \
    --cc=ndabilpuram@marvell.com \
    --cc=pavel.belous@aquantia.com \
    --cc=qi.z.zhang@intel.com \
    --cc=qiming.yang@intel.com \
    --cc=rmody@marvell.com \
    --cc=shahafs@mellanox.com \
    --cc=shshaikh@marvell.com \
    --cc=somnath.kotur@broadcom.com \
    --cc=thomas@monjalon.net \
    --cc=viacheslavo@mellanox.com \
    --cc=wenzhuo.lu@intel.com \
    --cc=xiao.w.wang@intel.com \
    --cc=yskoh@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox