From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 2755BA04C2; Mon, 25 Nov 2019 12:07:55 +0100 (CET) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 4B4A22952; Mon, 25 Nov 2019 12:07:54 +0100 (CET) Received: from mail-il1-f196.google.com (mail-il1-f196.google.com [209.85.166.196]) by dpdk.org (Postfix) with ESMTP id BBD6A28EE for ; Mon, 25 Nov 2019 12:07:52 +0100 (CET) Received: by mail-il1-f196.google.com with SMTP id q15so13775383ils.8 for ; Mon, 25 Nov 2019 03:07:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=psVOReWBeVNSxsGZzbq1/khehcJlZg1ANbSUYrwPp1M=; b=uBO/ht1s+U/1WuIjy61ikyOAXVVAP5retjNhXILGI013Epm7LU/6i8m9mxyrt0u/X5 L84P2mru27bUGZzJ4kP5v4IlEQTAnqzRdR1029ikLyN4/BVuYa2xLiMOB7IBLXpLNz+9 p5Q88RxDmCCIIqDOy2KhoYzLfrSmyjgBi7e3yN1SvVsHERpCzweGyhqJ3FCf8DLh3xMu EB8DXLyhm3fRFzapDnkg7V57GTgXaMU+L6Hg7daw4yZCCt6X8wlYEMLw8e4CsfGT8Av2 5/NfwFv+7tR8VGTHyvAhSSPvK9msoNiW5xUYY0r+McivA2mAD8H5QEae8kYhMyS84tUe WgqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=psVOReWBeVNSxsGZzbq1/khehcJlZg1ANbSUYrwPp1M=; b=MTCSVuohsaKuTjvupgHGp4nCDRiJwoQpfDDrrpCNFeC3G0GZGhE1BkZ14OpY2ZRluV fEI9+a4JlTAAuVtnAEqxV4PANMuHDZffHArmH2AFje68VfVI3rmXWc45T8eQr6Gn/ULR ODX+HeGz0WPkkxQbcG74jzJvJn32ljPNUf+O02+tj5McUip2DgJFwhRYSTAMeIrmPvxX wLnoEPi7alh0BUIZO2ilEvuvvjp8w1o32gHVXAqh6jM4h7l7kcSA3lKKkSpqyXTjFNDr lRpv7IngHGDFist6XOgMHGgvw7kSkAyt+CUzmqvjQ0tcvGBO0F2JJJSCYnfu0Pyfsb87 ZIVg== X-Gm-Message-State: APjAAAVqjBA+xOerwMH7AQcAwBNkfGoiFVxZdIVwMKI4bVUWpoRBu+Xr na+kk0LbHwDuAtZnVXyenn540jfKsVUzYapsRUs= X-Google-Smtp-Source: APXvYqxlf9hkAk0I8gd7Mwbn+nDgRKvRRfKRQejDPd1c1c73PwlMB6/btnAmM0tJnTAU4HRhEMWuRlUJEGe+u+ZP0+k= X-Received: by 2002:a92:aa48:: with SMTP id j69mr33043781ili.162.1574680071719; Mon, 25 Nov 2019 03:07:51 -0800 (PST) MIME-Version: 1.0 References: <1563800213-29839-3-git-send-email-david.marchand@redhat.com> <20191125093313.61700-1-ferruh.yigit@intel.com> In-Reply-To: <20191125093313.61700-1-ferruh.yigit@intel.com> From: Jerin Jacob Date: Mon, 25 Nov 2019 20:07:35 +0900 Message-ID: To: Ferruh Yigit Cc: David Marchand , Ajit Khaparde , Alejandro Lucero , Anatoly Burakov , Beilei Xing , "Richardson, Bruce" , dpdk-dev , Hyong Youb Kim , igor.russkikh@aquantia.com, Jerin Jacob , Jingjing Wu , John McNamara , John Daley , Kiran Kumar K , "Ananyev, Konstantin" , Marko Kovacevic , Matan Azrad , Nithin Dabilpuram , pavel.belous@aquantia.com, Qi Zhang , Qiming Yang , Rasesh Mody , Shahaf Shuler , Shahed Shaikh , Somnath Kotur , Thomas Monjalon , Slava Ovsiienko , Wenzhuo Lu , Xiao Wang , Yongseok Koh Content-Type: text/plain; charset="UTF-8" Subject: Re: [dpdk-dev] [PATCH v4 2/4] eal: fix IOVA mode selection as VA for PCI drivers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Mon, Nov 25, 2019 at 6:33 PM Ferruh Yigit wrote: > > > On 7/22/2019 1:56 PM, David Marchand wrote: > > The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which > > was intended to mean "driver only supports VA" but had been understood > > as "driver supports both PA and VA" by most net drivers and used to let > > dpdk processes to run as non root (which do not have access to physical > > addresses on recent kernels). > > > > The check on physical addresses actually closed the gap for those > > drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this > > flag can retain its intended meaning. > > Document explicitly its meaning. > > > > We can check that a driver requirement wrt to IOVA mode is fulfilled > > before trying to probe a device. > > > > Finally, document the heuristic used to select the IOVA mode and hope > > that we won't break it again. > > > > Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode") > > > > Signed-off-by: David Marchand > > Reviewed-by: Jerin Jacob > > Tested-by: Jerin Jacob > > Acked-by: Anatoly Burakov > <...> > > > diff --git a/drivers/bus/pci/pci_common.c b/drivers/bus/pci/pci_common.c > > index d2af472..9794552 100644 > > --- a/drivers/bus/pci/pci_common.c > > +++ b/drivers/bus/pci/pci_common.c > > @@ -169,8 +169,22 @@ static struct rte_devargs *pci_devargs_lookup(struct rte_pci_device *dev) > > * This needs to be before rte_pci_map_device(), as it enables to use > > * driver flags for adjusting configuration. > > */ > > - if (!already_probed) > > + if (!already_probed) { > > + enum rte_iova_mode dev_iova_mode; > > + enum rte_iova_mode iova_mode; > > + > > + dev_iova_mode = pci_device_iova_mode(dr, dev); > > + iova_mode = rte_eal_iova_mode(); > > + if (dev_iova_mode != RTE_IOVA_DC && > > + dev_iova_mode != iova_mode) { > > + RTE_LOG(ERR, EAL, " Expecting '%s' IOVA mode but current mode is '%s', not initializing\n", > > + dev_iova_mode == RTE_IOVA_PA ? "PA" : "VA", > > + iova_mode == RTE_IOVA_PA ? "PA" : "VA"); > > + return -EINVAL; > > + } > > + > > OvS reported an error while hotplugging a device. > > It looks like DPDK application initialized as IOVA=VA, and the new device is bound to 'igb_uio' which forces it to PA, fails on above check. Why they are binding to igb_uio if there is NO need for it? > > I would like to get your comment on the issue. > > For the OvS mode, hopefully binding the device to 'vfio-pci' can be a solution, but for the cases we don't have that option, can/should we force the DPDK to PA mode after initialization? On the other side, If we are forcing DPDK to PA then the same fate will be for VFIO only devices. There are two cases: 1) The system has a limitation on the specific mode 2) The devices have a limitation on the specific mod.e. Case (1), It is not applicable for hotplug cases as the system can run only one mode. We should able to detect in the first pass(Before the hotplug devices runs) Case (2), Is there any devices that can work ONLY in IOVA as PA mode? If yes, Please enumerate. Maybe something in the storage domain. >