From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <dev-bounces@dpdk.org>
Received: from dpdk.org (dpdk.org [92.243.14.124])
by inbox.dpdk.org (Postfix) with ESMTP id 2755BA04C2;
Mon, 25 Nov 2019 12:07:55 +0100 (CET)
Received: from [92.243.14.124] (localhost [127.0.0.1])
by dpdk.org (Postfix) with ESMTP id 4B4A22952;
Mon, 25 Nov 2019 12:07:54 +0100 (CET)
Received: from mail-il1-f196.google.com (mail-il1-f196.google.com
[209.85.166.196]) by dpdk.org (Postfix) with ESMTP id BBD6A28EE
for <dev@dpdk.org>; Mon, 25 Nov 2019 12:07:52 +0100 (CET)
Received: by mail-il1-f196.google.com with SMTP id q15so13775383ils.8
for <dev@dpdk.org>; Mon, 25 Nov 2019 03:07:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=psVOReWBeVNSxsGZzbq1/khehcJlZg1ANbSUYrwPp1M=;
b=uBO/ht1s+U/1WuIjy61ikyOAXVVAP5retjNhXILGI013Epm7LU/6i8m9mxyrt0u/X5
L84P2mru27bUGZzJ4kP5v4IlEQTAnqzRdR1029ikLyN4/BVuYa2xLiMOB7IBLXpLNz+9
p5Q88RxDmCCIIqDOy2KhoYzLfrSmyjgBi7e3yN1SvVsHERpCzweGyhqJ3FCf8DLh3xMu
EB8DXLyhm3fRFzapDnkg7V57GTgXaMU+L6Hg7daw4yZCCt6X8wlYEMLw8e4CsfGT8Av2
5/NfwFv+7tR8VGTHyvAhSSPvK9msoNiW5xUYY0r+McivA2mAD8H5QEae8kYhMyS84tUe
WgqA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=psVOReWBeVNSxsGZzbq1/khehcJlZg1ANbSUYrwPp1M=;
b=MTCSVuohsaKuTjvupgHGp4nCDRiJwoQpfDDrrpCNFeC3G0GZGhE1BkZ14OpY2ZRluV
fEI9+a4JlTAAuVtnAEqxV4PANMuHDZffHArmH2AFje68VfVI3rmXWc45T8eQr6Gn/ULR
ODX+HeGz0WPkkxQbcG74jzJvJn32ljPNUf+O02+tj5McUip2DgJFwhRYSTAMeIrmPvxX
wLnoEPi7alh0BUIZO2ilEvuvvjp8w1o32gHVXAqh6jM4h7l7kcSA3lKKkSpqyXTjFNDr
lRpv7IngHGDFist6XOgMHGgvw7kSkAyt+CUzmqvjQ0tcvGBO0F2JJJSCYnfu0Pyfsb87
ZIVg==
X-Gm-Message-State: APjAAAVqjBA+xOerwMH7AQcAwBNkfGoiFVxZdIVwMKI4bVUWpoRBu+Xr
na+kk0LbHwDuAtZnVXyenn540jfKsVUzYapsRUs=
X-Google-Smtp-Source: APXvYqxlf9hkAk0I8gd7Mwbn+nDgRKvRRfKRQejDPd1c1c73PwlMB6/btnAmM0tJnTAU4HRhEMWuRlUJEGe+u+ZP0+k=
X-Received: by 2002:a92:aa48:: with SMTP id j69mr33043781ili.162.1574680071719;
Mon, 25 Nov 2019 03:07:51 -0800 (PST)
MIME-Version: 1.0
References: <1563800213-29839-3-git-send-email-david.marchand@redhat.com>
<20191125093313.61700-1-ferruh.yigit@intel.com>
In-Reply-To: <20191125093313.61700-1-ferruh.yigit@intel.com>
From: Jerin Jacob <jerinjacobk@gmail.com>
Date: Mon, 25 Nov 2019 20:07:35 +0900
Message-ID: <CALBAE1Mvei0W3rPd7jguBcLXgZQ9RJU21gSBOcK-43cNRfRrhg@mail.gmail.com>
To: Ferruh Yigit <ferruh.yigit@intel.com>
Cc: David Marchand <david.marchand@redhat.com>,
Ajit Khaparde <ajit.khaparde@broadcom.com>,
Alejandro Lucero <alejandro.lucero@netronome.com>,
Anatoly Burakov <anatoly.burakov@intel.com>,
Beilei Xing <beilei.xing@intel.com>,
"Richardson, Bruce" <bruce.richardson@intel.com>, dpdk-dev <dev@dpdk.org>,
Hyong Youb Kim <hyonkim@cisco.com>, igor.russkikh@aquantia.com,
Jerin Jacob <jerinj@marvell.com>, Jingjing Wu <jingjing.wu@intel.com>,
John McNamara <john.mcnamara@intel.com>, John Daley <johndale@cisco.com>,
Kiran Kumar K <kirankumark@marvell.com>,
"Ananyev, Konstantin" <konstantin.ananyev@intel.com>,
Marko Kovacevic <marko.kovacevic@intel.com>,
Matan Azrad <matan@mellanox.com>, Nithin Dabilpuram <ndabilpuram@marvell.com>,
pavel.belous@aquantia.com,
Qi Zhang <qi.z.zhang@intel.com>, Qiming Yang <qiming.yang@intel.com>,
Rasesh Mody <rmody@marvell.com>, Shahaf Shuler <shahafs@mellanox.com>,
Shahed Shaikh <shshaikh@marvell.com>,
Somnath Kotur <somnath.kotur@broadcom.com>,
Thomas Monjalon <thomas@monjalon.net>,
Slava Ovsiienko <viacheslavo@mellanox.com>,
Wenzhuo Lu <wenzhuo.lu@intel.com>, Xiao Wang <xiao.w.wang@intel.com>,
Yongseok Koh <yskoh@mellanox.com>
Content-Type: text/plain; charset="UTF-8"
Subject: Re: [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA
for PCI drivers
X-BeenThere: dev@dpdk.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/dev>,
<mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/dev>,
<mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>
On Mon, Nov 25, 2019 at 6:33 PM Ferruh Yigit <ferruh.yigit@intel.com> wrote:
>
>
> On 7/22/2019 1:56 PM, David Marchand wrote:
> > The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which
> > was intended to mean "driver only supports VA" but had been understood
> > as "driver supports both PA and VA" by most net drivers and used to let
> > dpdk processes to run as non root (which do not have access to physical
> > addresses on recent kernels).
> >
> > The check on physical addresses actually closed the gap for those
> > drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this
> > flag can retain its intended meaning.
> > Document explicitly its meaning.
> >
> > We can check that a driver requirement wrt to IOVA mode is fulfilled
> > before trying to probe a device.
> >
> > Finally, document the heuristic used to select the IOVA mode and hope
> > that we won't break it again.
> >
> > Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode")
> >
> > Signed-off-by: David Marchand <david.marchand@redhat.com>
> > Reviewed-by: Jerin Jacob <jerinj@marvell.com>
> > Tested-by: Jerin Jacob <jerinj@marvell.com>
> > Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> <...>
>
> > diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c
> > index d2af472..9794552 100644
> > --- a/drivers/bus/pci/pci_common.c
> > +++ b/drivers/bus/pci/pci_common.c
> > @@ -169,8 +169,22 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev)
> > * This needs to be before rte_pci_map_device(), as it enables to use
> > * driver flags for adjusting configuration.
> > */
> > - if (!already_probed)
> > + if (!already_probed) {
> > + enum rte_iova_mode dev_iova_mode;
> > + enum rte_iova_mode iova_mode;
> > +
> > + dev_iova_mode = pci_device_iova_mode(dr, dev);
> > + iova_mode = rte_eal_iova_mode();
> > + if (dev_iova_mode != RTE_IOVA_DC &&
> > + dev_iova_mode != iova_mode) {
> > + RTE_LOG(ERR, EAL, " Expecting '%s' IOVA mode but current mode is '%s', not initializing\n",
> > + dev_iova_mode == RTE_IOVA_PA ? "PA" : "VA",
> > + iova_mode == RTE_IOVA_PA ? "PA" : "VA");
> > + return -EINVAL;
> > + }
> > +
>
> OvS reported an error while hotplugging a device.
>
> It looks like DPDK application initialized as IOVA=VA, and the new device is bound to 'igb_uio' which forces it to PA, fails on above check.
Why they are binding to igb_uio if there is NO need for it?
>
> I would like to get your comment on the issue.
>
> For the OvS mode, hopefully binding the device to 'vfio-pci' can be a solution, but for the cases we don't have that option, can/should we force the DPDK to PA mode after initialization?
On the other side, If we are forcing DPDK to PA then the same fate
will be for VFIO only devices.
There are two cases:
1) The system has a limitation on the specific mode
2) The devices have a limitation on the specific mod.e.
Case (1), It is not applicable for hotplug cases as the system can run
only one mode. We should able to detect in the first pass(Before the
hotplug devices runs)
Case (2), Is there any devices that can work ONLY in IOVA as PA mode?
If yes, Please enumerate. Maybe something in the storage domain.
>