DPDK patches and discussions
 help / color / Atom feed
From: Ferruh Yigit <ferruh.yigit@intel.com>
To: david.marchand@redhat.com
Cc: ajit.khaparde@broadcom.com, alejandro.lucero@netronome.com,
	anatoly.burakov@intel.com, beilei.xing@intel.com,
	bruce.richardson@intel.com, dev@dpdk.org, hyonkim@cisco.com,
	igor.russkikh@aquantia.com, jerinj@marvell.com,
	jingjing.wu@intel.com, john.mcnamara@intel.com,
	johndale@cisco.com, kirankumark@marvell.com,
	konstantin.ananyev@intel.com, marko.kovacevic@intel.com,
	matan@mellanox.com, ndabilpuram@marvell.com,
	pavel.belous@aquantia.com, qi.z.zhang@intel.com,
	qiming.yang@intel.com, rmody@marvell.com, shahafs@mellanox.com,
	shshaikh@marvell.com, somnath.kotur@broadcom.com,
	thomas@monjalon.net, viacheslavo@mellanox.com,
	wenzhuo.lu@intel.com, xiao.w.wang@intel.com, yskoh@mellanox.com
Subject: [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers
Date: Mon, 25 Nov 2019 09:33:13 +0000
Message-ID: <20191125093313.61700-1-ferruh.yigit@intel.com> (raw)
In-Reply-To: <1563800213-29839-3-git-send-email-david.marchand@redhat.com>


On 7/22/2019 1:56 PM, David Marchand wrote:
> The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which
> was intended to mean "driver only supports VA" but had been understood
> as "driver supports both PA and VA" by most net drivers and used to let
> dpdk processes to run as non root (which do not have access to physical
> addresses on recent kernels).
> 
> The check on physical addresses actually closed the gap for those
> drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this
> flag can retain its intended meaning.
> Document explicitly its meaning.
> 
> We can check that a driver requirement wrt to IOVA mode is fulfilled
> before trying to probe a device.
> 
> Finally, document the heuristic used to select the IOVA mode and hope
> that we won't break it again.
> 
> Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode")
> 
> Signed-off-by: David Marchand <david.marchand@redhat.com>
> Reviewed-by: Jerin Jacob <jerinj@marvell.com>
> Tested-by: Jerin Jacob <jerinj@marvell.com>
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
<...>

> diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
> index d2af472..9794552 100644
> --- a/drivers/bus/pci/pci_common.c
> +++ b/drivers/bus/pci/pci_common.c
> @@ -169,8 +169,22 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
>  	 * This needs to be before rte_pci_map_device(), as it enables to use
>  	 * driver flags for adjusting configuration.
>  	 */
> -	if (!already_probed)
> +	if (!already_probed) {
> +		enum rte_iova_mode dev_iova_mode;
> +		enum rte_iova_mode iova_mode;
> +
> +		dev_iova_mode = pci_device_iova_mode(dr, dev);
> +		iova_mode = rte_eal_iova_mode();
> +		if (dev_iova_mode != RTE_IOVA_DC &&
> +		    dev_iova_mode != iova_mode) {
> +			RTE_LOG(ERR, EAL, "  Expecting '%s' IOVA mode but current mode is '%s', not initializing\n",
> +				dev_iova_mode == RTE_IOVA_PA ? "PA" : "VA",
> +				iova_mode == RTE_IOVA_PA ? "PA" : "VA");
> +			return -EINVAL;
> +		}
> +

OvS reported an error while hotplugging a device.

It looks like DPDK application initialized as IOVA=VA, and the new device is bound to 'igb_uio' which forces it to PA, fails on above check.

I would like to get your comment on the issue.

For the OvS mode, hopefully binding the device to 'vfio-pci' can be a solution, but for the cases we don't have that option, can/should we force the DPDK to PA mode after initialization?


  reply index

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-10 21:48 [dpdk-dev] [PATCH 0/2] Fixes on IOVA mode selection David Marchand
2019-07-10 21:48 ` [dpdk-dev] [PATCH 1/2] Revert "bus/pci: add Mellanox kernel driver type" David Marchand
2019-07-16 10:37   ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-10 21:48 ` [dpdk-dev] [PATCH 2/2] eal: fix IOVA mode selection as VA for pci drivers David Marchand
2019-07-11 14:40   ` Thomas Monjalon
2019-07-12  8:05     ` Jerin Jacob Kollanukkaran
2019-07-12 11:03   ` Burakov, Anatoly
2019-07-12 12:43     ` Thomas Monjalon
2019-07-12 12:58       ` Burakov, Anatoly
2019-07-12 13:19         ` Bruce Richardson
2019-07-15 14:26       ` Jerin Jacob Kollanukkaran
2019-07-15 15:03         ` Thomas Monjalon
2019-07-15 15:35           ` Jerin Jacob Kollanukkaran
2019-07-15 16:06             ` Thomas Monjalon
2019-07-15 16:27               ` Jerin Jacob Kollanukkaran
2019-07-16 13:46 ` [dpdk-dev] [PATCH v2 0/4] Fixes on IOVA mode selection jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 1/4] Revert "bus/pci: add Mellanox kernel driver type" jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 2/4] eal: fix IOVA mode selection as VA for pci drivers jerinj
2019-07-16 14:26     ` Burakov, Anatoly
2019-07-16 15:07       ` Jerin Jacob Kollanukkaran
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 3/4] eal: change RTE_PCI_DRV_IOVA_AS_VA flag name jerinj
2019-07-16 13:46   ` [dpdk-dev] [PATCH v2 4/4] eal: select IOVA mode as VA for default case jerinj
2019-07-16 14:33     ` Burakov, Anatoly
2019-07-17  8:33       ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-17 12:38         ` Burakov, Anatoly
2019-07-17 14:04           ` Jerin Jacob Kollanukkaran
2019-07-18  6:45   ` [dpdk-dev] [PATCH v3 0/4] Fixes on IOVA mode selection jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 1/4] Revert "bus/pci: add Mellanox kernel driver type" jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 2/4] eal: fix IOVA mode selection as VA for pci drivers jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 3/4] eal: change RTE_PCI_DRV_IOVA_AS_VA flag name jerinj
2019-07-18  6:45     ` [dpdk-dev] [PATCH v3 4/4] eal: select IOVA mode as VA for default case jerinj
2019-07-22 11:28     ` [dpdk-dev] [PATCH v3 0/4] Fixes on IOVA mode selection David Marchand
2019-07-22 12:56 ` [dpdk-dev] [PATCH v4 " David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 1/4] Revert "bus/pci: add Mellanox kernel driver type" David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers David Marchand
2019-11-25  9:33     ` Ferruh Yigit [this message]
2019-11-25 10:22       ` Thomas Monjalon
2019-11-25 12:03         ` Ferruh Yigit
2019-11-25 12:36           ` David Marchand
2019-11-25 12:58             ` Burakov, Anatoly
2019-11-25 14:29               ` Thomas Monjalon
2019-11-25 11:07       ` Jerin Jacob
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 3/4] drivers: change IOVA as VA PCI flag name David Marchand
2019-07-22 12:56   ` [dpdk-dev] [PATCH v4 4/4] eal: select IOVA as VA mode for default case David Marchand
2019-07-22 15:53   ` [dpdk-dev] [PATCH v4 0/4] Fixes on IOVA mode selection Thomas Monjalon
2019-07-23  3:35     ` Stojaczyk, Dariusz
2019-07-23  4:18       ` Jerin Jacob Kollanukkaran
2019-07-23  4:54         ` Stojaczyk, Dariusz
2019-07-23  5:27           ` Jerin Jacob Kollanukkaran
2019-07-23  7:21             ` Thomas Monjalon
2019-07-23  9:57             ` Burakov, Anatoly
2019-07-23 10:25               ` Thomas Monjalon
2019-07-23 13:56                 ` Burakov, Anatoly
2019-07-23 14:24                   ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-23 14:29                   ` [dpdk-dev] " Burakov, Anatoly
2019-07-23 14:36                     ` [dpdk-dev] [EXT] " Jerin Jacob Kollanukkaran
2019-07-23 15:47                       ` Burakov, Anatoly

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191125093313.61700-1-ferruh.yigit@intel.com \
    --to=ferruh.yigit@intel.com \
    --cc=ajit.khaparde@broadcom.com \
    --cc=alejandro.lucero@netronome.com \
    --cc=anatoly.burakov@intel.com \
    --cc=beilei.xing@intel.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=hyonkim@cisco.com \
    --cc=igor.russkikh@aquantia.com \
    --cc=jerinj@marvell.com \
    --cc=jingjing.wu@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=johndale@cisco.com \
    --cc=kirankumark@marvell.com \
    --cc=konstantin.ananyev@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=matan@mellanox.com \
    --cc=ndabilpuram@marvell.com \
    --cc=pavel.belous@aquantia.com \
    --cc=qi.z.zhang@intel.com \
    --cc=qiming.yang@intel.com \
    --cc=rmody@marvell.com \
    --cc=shahafs@mellanox.com \
    --cc=shshaikh@marvell.com \
    --cc=somnath.kotur@broadcom.com \
    --cc=thomas@monjalon.net \
    --cc=viacheslavo@mellanox.com \
    --cc=wenzhuo.lu@intel.com \
    --cc=xiao.w.wang@intel.com \
    --cc=yskoh@mellanox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK patches and discussions

Archives are clonable:
	git clone --mirror http://inbox.dpdk.org/dev/0 dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dev dev/ http://inbox.dpdk.org/dev \
		dev@dpdk.org
	public-inbox-index dev


Newsgroup available over NNTP:
	nntp://inbox.dpdk.org/inbox.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/ public-inbox