From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id D39921B2D2; Mon, 6 Nov 2017 19:41:20 +0100 (CET) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Nov 2017 10:41:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.44,353,1505804400"; d="scan'208";a="918189495" Received: from fyigit-mobl1.ger.corp.intel.com (HELO [10.241.225.127]) ([10.241.225.127]) by FMSMGA003.fm.intel.com with ESMTP; 06 Nov 2017 10:41:16 -0800 To: Stephen Hemminger Cc: "Roberts, Lee A." , "Tan, Jianfeng" , Thomas Monjalon , "dev@dpdk.org" , "stable@dpdk.org" , "Wu, Jingjing" , Shijith Thotton , Gregory Etelson , Harish Patil , George Prekas , "Gonzalez Monroy, Sergio" , Rasesh Mody References: <20171103003005.44339-1-ferruh.yigit@intel.com> <20171103005100.44633-1-ferruh.yigit@intel.com> From: Ferruh Yigit Message-ID: Date: Mon, 6 Nov 2017 10:41:15 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] [PATCH v2] igb_uio: add config option to control reset X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Nov 2017 18:41:21 -0000 On 11/4/2017 3:08 AM, Stephen Hemminger wrote: > > > On Nov 4, 2017 01:03, "Ferruh Yigit" > wrote: > > On 11/3/2017 12:42 PM, Roberts, Lee A. wrote: > > > > > >> -----Original Message----- > >> From: dev [mailto:dev-bounces@dpdk.org ] On > Behalf Of Tan, Jianfeng > >> Sent: Thursday, November 02, 2017 8:57 PM > >> To: Ferruh Yigit >; Thomas Monjalon > > >> Cc: dev@dpdk.org ; stable@dpdk.org > ; Jingjing Wu >; Shijith Thotton > >> >; Gregory Etelson > >; Harish Patil > >> >; George Prekas > >; Sergio Gonzalez Monroy > >> >; Rasesh Mody > > > >> Subject: Re: [dpdk-dev] [PATCH v2] igb_uio: add config option to control > reset > >> > >> > >> > >> On 11/3/2017 8:51 AM, Ferruh Yigit wrote: > >>> Adding a compile time configuration option to control device reset done > >>> during DPDK application exit. > >>> > >>> Config option is CONFIG_RTE_EAL_IGB_UIO_RESET and enabled by default, > >>> so by default reset will happen. Having this reset is safer to be sure > >>> device left in a proper case. > >>> > >>> But for special cases [1] it is possible to disable the config option > >>> to prevent the device reset. > >>> > >>> [1] > >>> http://dpdk.org/ml/archives/dev/2017-November/080927.html > > >>> > >>> Fixes: b58eedfc7dd5 ("igb_uio: issue FLR during open and release of > device file") > >>> Cc: stable@dpdk.org > >>> > >>> Signed-off-by: Ferruh Yigit > > >> > >> Realize that we do have a pci_clear_master() in the release() to disable > >> the DMA from device until the next open() will enable the DMA again . > >> Here is my: > >> > >> Reviewed-by: Jianfeng Tan > > >> > >> Thanks, > >> Jianfeng > >> > >>> --- > >>> Cc: Jianfeng Tan > > >>> Cc: Jingjing Wu > > >>> Cc: Shijith Thotton > > >>> Cc: Gregory Etelson > > >>> Cc: Harish Patil > > >>> Cc: George Prekas > > >>> Cc: Sergio Gonzalez Monroy > > >>> Cc: Rasesh Mody > > >>> > >>> v2: > >>> * fix typo in commit log > >>> --- > >>>   config/common_base                        | 1 + > >>>   config/common_linuxapp                    | 1 + > >>>   lib/librte_eal/linuxapp/igb_uio/igb_uio.c | 2 ++ > >>>   3 files changed, 4 insertions(+) > >>> > >>> diff --git a/config/common_base b/config/common_base > >>> index 82ee75456..2a9947420 100644 > >>> --- a/config/common_base > >>> +++ b/config/common_base > >>> @@ -102,6 +102,7 @@ CONFIG_RTE_LIBEAL_USE_HPET=n > >>>   CONFIG_RTE_EAL_ALLOW_INV_SOCKET_ID=n > >>>   CONFIG_RTE_EAL_ALWAYS_PANIC_ON_ERROR=n > >>>   CONFIG_RTE_EAL_IGB_UIO=n > >>> +CONFIG_RTE_EAL_IGB_UIO_RESET=n > >>>   CONFIG_RTE_EAL_VFIO=n > >>>   CONFIG_RTE_MALLOC_DEBUG=n > >>>   CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=n > >>> diff --git a/config/common_linuxapp b/config/common_linuxapp > >>> index 74c7d64ec..b3a602909 100644 > >>> --- a/config/common_linuxapp > >>> +++ b/config/common_linuxapp > >>> @@ -37,6 +37,7 @@ CONFIG_RTE_EXEC_ENV_LINUXAPP=y > >>> > >>>   CONFIG_RTE_EAL_NUMA_AWARE_HUGEPAGES=y > >>>   CONFIG_RTE_EAL_IGB_UIO=y > >>> +CONFIG_RTE_EAL_IGB_UIO_RESET=y > >>>   CONFIG_RTE_EAL_VFIO=y > >>>   CONFIG_RTE_KNI_KMOD=y > >>>   CONFIG_RTE_LIBRTE_KNI=y > >>> diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > >>> index fd320d87d..0325722c0 100644 > >>> --- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > >>> +++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > >>> @@ -360,7 +360,9 @@ igbuio_pci_release(struct uio_info *info, struct > inode *inode) > >>>   /* stop the device from further DMA */ > >>>   pci_clear_master(dev); > >>> > >>> +#ifdef RTE_EAL_IGB_UIO_RESET > >>>   pci_reset_function(dev); > >>> +#endif > >>> > >>>   return 0; > >>>   } > > > > A compile time configuration option makes life very difficult for > application providers. > > > > Consider the case where an application such as Open vSwitch with DPDK > support is being provided > > with a Linux distribution.  One would want the Open vSwitch binary to > support as many vendor NICs > > as possible---without the need to recompile.  With a change such as this, > one would need to have > > different versions of the kernel igb_uio module to support different NICs. > > Agreed, I am against adding more compile time options although I am end up > sending a few of them these days. > > > The Linux kernel is already aware of, and provides work-arounds for, > various PCI quirks. > > For example, see linux/drivers/pci/quirks.c > (http://elixir.free-electrons.com/linux/latest/source/drivers/pci/quirks.c > ). > > > > At this point in igb_uio.c, one is aware of the struct pci_dev "dev" for > the device in question. > > Access to the vendor and device information should be simple: > > > > struct pci_dev { > > struct list_head bus_list;/* node in per-bus list */ > > struct pci_bus*bus;/* bus this device is on */ > > struct pci_bus*subordinate;/* bus this device bridges to */ > > > > void*sysdata;/* hook for sys-specific extension */ > > struct proc_dir_entry *procent;/* device entry in /proc/bus/pci */ > > struct pci_slot*slot;/* Physical slot this device is in */ > > > > unsigned intdevfn;/* encoded device & function index */ > > unsigned shortvendor; > > unsigned shortdevice; > > unsigned shortsubsystem_vendor; > > unsigned shortsubsystem_device; > > ... > > > > One could imagine using logic to implement corresponding PCI quirks that > can be evaluated > > at runtime.  For example (in pseudocode), > > > >      if not (vendor = "Cavium" and device = "bnx2x") > >         then pci_reset_function(dev); > > It wouldn't be nice to add device specific checks into generic igb_uio module, > but also it is not nice to add compile time option, comparing two I would be OK > to device checks. > > What do you think about following? > If there is no objection and Rasesh confirms that patch is working, I can send a > proper patch for it. > > > > diff --git a/lib/librte_eal/linuxapp/igb_uio/compat.h > b/lib/librte_eal/linuxapp/igb_uio/compat.h > index 30508f35c..264206af3 100644 > --- a/lib/librte_eal/linuxapp/igb_uio/compat.h > +++ b/lib/librte_eal/linuxapp/igb_uio/compat.h > @@ -134,3 +134,21 @@ static bool pci_check_and_mask_intx(struct pci_dev *pdev) >  #endif > > > +#define BROADCOM_PCI_VENDOR_ID 0x14E4 > +static const struct pci_device_id no_reset_pci_tbl[] = { > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x168a) }, /* 57800 */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x164f) }, /* 57711 */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x168e) }, /* 57810 */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x163d) }, /* 57811 */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x168d) }, /* 57840_OBS */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16a1) }, /* 57840_4_10 */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16a2) }, /* 57840_2_20 */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16ae) }, /* 57810_MF */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x163e) }, /* 57811_MF */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16a4) }, /* 57840_MF */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16a9) }, /* 57800_VF */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16af) }, /* 57810_VF */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x163f) }, /* 57811_VF */ > +       { PCI_DEVICE(BROADCOM_PCI_VENDOR_ID, 0x16ad) }, /* 57840_VF */ > +       { 0 }, > +}; > diff --git a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > index fd320d87d..b0d92b51e 100644 > --- a/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > +++ b/lib/librte_eal/linuxapp/igb_uio/igb_uio.c > @@ -348,6 +348,14 @@ igbuio_pci_open(struct uio_info *info, struct inode *inode) >         return 0; >  } > > +static int is_device_excluded_from_reset(struct pci_dev *pdev) > +{ > +       if (pci_match_id(no_reset_pci_tbl, pdev)) > +               return 1; > + > +       return 0; > +} > + > > > Personal preference is for more concise: > static book is_device_excluded(const struct pci_dev *pdev) > { >        return pci_match_id(no_reset_pci_tbl, pdev); > > } I will update function, but I am for keeping function name to clarify what device is excluded from. > >  static int >  igbuio_pci_release(struct uio_info *info, struct inode *inode) >  { > @@ -360,7 +368,8 @@ igbuio_pci_release(struct uio_info *info, struct inode > *inode) >         /* stop the device from further DMA */ >         pci_clear_master(dev); > > -       pci_reset_function(dev); > +       if (!is_device_excluded_from_reset(dev)) > +               pci_reset_function(dev); > >         return 0; >  } > > > > > > There are other possible implementations.  If there are enough quirks, one > might have action > > functions defined---and a table of function pointers associated with each > PMD to select the > > proper action. > > > >                                                 - Lee Roberts > > > > > > > > > >