Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open and release of device file

["Yang, Qiming" <qiming.yang@intel.com>]

Yes, I have tried, and it works. But I think removing this function also makes the patch meaningless.

Qiming
From: Gregory Etelson [mailto:gregory@weka.io]
Sent: Sunday, September 17, 2017 10:50 AM
To: Yang, Qiming <qiming.yang@intel.com>
Cc: thomas@monjalon.net; dev@dpdk.org; Tan, Jianfeng <jianfeng.tan@intel.com>; Thotton, Shijith <Shijith.Thotton@cavium.com>; Hu, Xuekun <xuekun.hu@intel.com>; Yigit, Ferruh <ferruh.yigit@intel.com>; Patil, Harish <Harish.Patil@cavium.com>
Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open and release of device file


Hello Qiming,



Did you try to remove PCI reset at igbuio_pci_open ?

Reverting the patch entirely will get us back to VF issues



Regards,

Gregory





On Friday, 15 September 2017 11:04:16 IDT Yang, Qiming wrote:

> Hi, Thomas

> We meet a urgent issue because of Shijith's patch.

> When use DPDK version 17.08, DPDK VF driver and kernel PF driver, i40e, ixgbe and qede(Harish said) VF all meet initial failure.

> This issue only occurred when use 17.08 igb_uio, vfio-pci and 17.05 igb_uio all works well.

> Considering this issue will block our vf driver develop and test work, could we revert this patch first and find another appropriate way?

>

> Qiming

>

> > -----Original Message-----

> > From: Patil, Harish [mailto:Harish.Patil@cavium.com]

> > Sent: Thursday, September 14, 2017 3:45 AM

> > To: Yigit, Ferruh <ferruh.yigit@intel.com<mailto:ferruh.yigit@intel.com>>; Hu, Xuekun <xuekun.hu@intel.com<mailto:xuekun.hu@intel.com>>;

> > Thotton, Shijith <Shijith.Thotton@cavium.com<mailto:Shijith.Thotton@cavium.com>>; Yang, Qiming

> > <qiming.yang@intel.com<mailto:qiming.yang@intel.com>>; Gregory Etelson <gregory@weka.io<mailto:gregory@weka.io>>

> > Cc: dev@dpdk.org<mailto:dev@dpdk.org>; Tan, Jianfeng <jianfeng.tan@intel.com<mailto:jianfeng.tan@intel.com>>

> > Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open

> > and release of device file

> >

> >

> >

> > -----Original Message-----

> > From: dev <dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>> on behalf of Ferruh Yigit

> > <ferruh.yigit@intel.com<mailto:ferruh.yigit@intel.com>>

> > Date: Wednesday, September 13, 2017 at 10:06 AM

> > To: "Hu, Xuekun" <xuekun.hu@intel.com<mailto:xuekun.hu@intel.com>>, "Thotton, Shijith"

> > <Shijith.Thotton@cavium.com<mailto:Shijith.Thotton@cavium.com>>, "Yang, Qiming" <qiming.yang@intel.com<mailto:qiming.yang@intel.com>>,

> > Gregory Etelson <gregory@weka.io<mailto:gregory@weka.io>>

> > Cc: "dev@dpdk.org<mailto:dev@dpdk.org>" <dev@dpdk.org<mailto:dev@dpdk.org>>, "Tan, Jianfeng" <jianfeng.tan@intel.com<mailto:jianfeng.tan@intel.com>>

> > Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open

> > and release of device file

> >

> > >On 9/13/2017 3:25 PM, Hu, Xuekun wrote:

> > >> I met the same issue too, only with i40e 2.1.26 PF kernel driver.

> > >>

> > >> -----Original Message-----

> > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ferruh Yigit

> > >> Sent: Wednesday, September 13, 2017 7:04 PM

> > >> To: Shijith Thotton <shijith.thotton@caviumnetworks.com<mailto:shijith.thotton@caviumnetworks.com>>; Yang,

> > >>Qiming <qiming.yang@intel.com<mailto:qiming.yang@intel.com>>; Gregory Etelson <gregory@weka.io<mailto:gregory@weka.io>>

> > >> Cc: dev@dpdk.org<mailto:dev@dpdk.org>; Tan, Jianfeng <jianfeng.tan@intel.com<mailto:jianfeng.tan@intel.com>>

> > >> Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR

> > >>during open and release of device file

> > >>

> > >> On 9/13/2017 11:48 AM, Shijith Thotton wrote:

> > >>> On Wed, Sep 13, 2017 at 07:51:30AM +0000, Yang, Qiming wrote:

> > >>>> Hi, Shijith

> > >>>>

> > >>>>

> > >>>>

> > >>>> VF init error will happen after apply your patch, error log as

> > >>>>below. If

> > >>>> revert your commit, all things work well. And this issue is not

> > >>>>only occur

> > >>>> in i40 VF but also ixgbe.

> > >

> > >Hi Qiming,

> > >

> > >I can reproduce the issue for the case:

> > >

> > >1- i40e 2.1.26 PF kernel driver (NOT reproduced with 1.6.27-k)

> > >2- DPDK app run on host, so both PF and VF are in host. Linux driver

> > >for PF and DPDK for VF. (When VF used in a VM, Linux PF on host and

> > >DPDK VF on guest, this works fine)

> > >

> > >And I confirm the function igbuio_pci_open() added with below patch is

> > >causing it.

> > >

> > >igbuio_pci_open() does:

> > > pci_reset_function(dev);

> > > pci_set_master(dev);

> > >

> > >Don't know yet root cause of the error, will dig more.

> > >

> > >>>>

> > >>>> Could you help to check it soon?

> > >>>>

> > >>>>

> > >>>>

> > >>>> [root@localhost app]# ./testpmd -c 7 -n 4 -- -i

> > >>>>

> > >>>> EAL: Detected 10 lcore(s)

> > >>>>

> > >>>> EAL: No free hugepages reported in hugepages-1048576kB

> > >>>>

> > >>>> EAL: Probing VFIO support...

> > >>>>

> > >>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using

> > >>>> unreliable clock cycles !

> > >>>>

> > >>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1

> > >>>>

> > >>>> EAL: Invalid NUMA socket, default to 0

> > >>>>

> > >>>> EAL: probe driver: 8086:154c net_i40e_vf

> > >>>>

> > >>>> i40evf_init_vf(): init_adminq failed: -53

> > >>>>

> > >>>> i40evf_dev_init(): Init vf failed

> > >>>>

> > >>>> EAL: Requested device 0000:00:03.0 cannot be used

> > >>>>

> > >>>>

> > >>>>

> > >>>> commit b58eedfc7dd57eef6d12e2c654a52c834f36084a

> > >>>> Author: Shijith Thotton <shijith.thotton@caviumnetworks.com<mailto:shijith.thotton@caviumnetworks.com>>

> > >>>> Date: Fri Jul 7 16:43:51 2017 +0530

> > >>>>

> > >>>> igb_uio: issue FLR during open and release of device file

> > >>>>

> > >>>> Set UIO info device file operations open and release. Call pci

> > >>>>reset

> > >>>> function inside open and release to clear device state at start

> > >>>>and end.

> > >>>> Copied this behaviour from vfio_pci kernel module code. With

> > >>>>this patch,

> > >>>> it is not mandatory to issue FLR by PMD's during init and close.

> > >>>>

> > >>>> Bus master enable and disable are added in open and release

> > >>>>respectively

> > >>>> to take care of device DMA.

> > >>>>

> > >>>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com<mailto:shijith.thotton@caviumnetworks.com>>

> > >>>> Reviewed-by: Jianfeng Tan <jianfeng.tan@intel.com<mailto:jianfeng.tan@intel.com>>

> > >>>> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com<mailto:ferruh.yigit@intel.com>>

> > >>>> Acked-by: Gregory Etelson <gregory@weka.io<mailto:gregory@weka.io>>

> > >>>>

> > >>>>

> > >>>>

> > >>>> Best Regard,

> > >>>>

> > >>>> Yang Qiming

> > >>>>

> > >>>>

> > >>>

> > >>> Hi Yang Qiming,

> > >>>

> > >>> We didn't face this issue while preparing patch. I think, Gregory

> > >>> tested it for Intel NICs. Adding more people to the loop for help.

> > >>>

> > >>

> > >> I will try to reproduce the issue.

> > >>

> > >>> Shijith

> >

> > [Harish] We are seeing similar issue with the bringup of qede VF devices due to

> > this patch.

> > Could we revert this and leave it to individual PMDs to manage it?

> > >>

>

>