From: Shijith Thotton <shijith.thotton@caviumnetworks.com>
To: "Yang, Qiming" <qiming.yang@intel.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"Tan, Jianfeng" <jianfeng.tan@intel.com>,
"Hu, Xuekun" <xuekun.hu@intel.com>,
Gregory Etelson <gregory@weka.io>,
"Yigit, Ferruh" <ferruh.yigit@intel.com>,
"Patil, Harish" <Harish.Patil@cavium.com>
Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open and release of device file
Date: Thu, 14 Sep 2017 12:30:55 +0530 [thread overview]
Message-ID: <20170914070053.GA23787@localhost.localdomain> (raw)
In-Reply-To: <F5DF4F0E3AFEF648ADC1C3C33AD4DBF16F7E9B61@SHSMSX151.ccr.corp.intel.com>
On Thu, Sep 14, 2017 at 03:16:05AM +0000, Yang, Qiming wrote:
> > -----Original Message-----
> > From: Yigit, Ferruh
> > Sent: Thursday, September 14, 2017 1:06 AM
> > To: Hu, Xuekun <xuekun.hu@intel.com>; Shijith Thotton
> > <shijith.thotton@caviumnetworks.com>; Yang, Qiming
> > <qiming.yang@intel.com>; Gregory Etelson <gregory@weka.io>
> > Cc: dev@dpdk.org; Tan, Jianfeng <jianfeng.tan@intel.com>
> > Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open
> > and release of device file
> >
> > On 9/13/2017 3:25 PM, Hu, Xuekun wrote:
> > > I met the same issue too, only with i40e 2.1.26 PF kernel driver.
> > >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ferruh Yigit
> > > Sent: Wednesday, September 13, 2017 7:04 PM
> > > To: Shijith Thotton <shijith.thotton@caviumnetworks.com>; Yang, Qiming
> > > <qiming.yang@intel.com>; Gregory Etelson <gregory@weka.io>
> > > Cc: dev@dpdk.org; Tan, Jianfeng <jianfeng.tan@intel.com>
> > > Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR
> > > during open and release of device file
> > >
> > > On 9/13/2017 11:48 AM, Shijith Thotton wrote:
> > >> On Wed, Sep 13, 2017 at 07:51:30AM +0000, Yang, Qiming wrote:
> > >>> Hi, Shijith
> > >>>
> > >>>
> > >>>
> > >>> VF init error will happen after apply your patch, error log as below. If
> > >>> revert your commit, all things work well. And this issue is not only occur
> > >>> in i40 VF but also ixgbe.
> >
> > Hi Qiming,
> >
> > I can reproduce the issue for the case:
> >
> > 1- i40e 2.1.26 PF kernel driver (NOT reproduced with 1.6.27-k)
> > 2- DPDK app run on host, so both PF and VF are in host. Linux driver for PF and
> > DPDK for VF. (When VF used in a VM, Linux PF on host and DPDK VF on guest,
> > this works fine)
> >
> > And I confirm the function igbuio_pci_open() added with below patch is causing
> > it.
> >
> Hi, Shijith
> Could you tell why this adding is necessary? And confirm why this function will cause the vf init issue. As Harish said, it seems not only reproduced in intel NICs.
Reset was added in open to make the device ready for init. I'm not sure about
the exact cause of this issue and was unable to reproduce the same with LiquidIO
VFs.
Reset happens during open and if VFs are not ready by the time init happens in
PMD, it can fail. Issue could be in firmware as we don't see it for all versions.
If there are no other way to fix this, I think, it is best to remove reset
during open as it breaks PMD from multiple vendors.
Please try removing igbuio_pci_open.
Thanks,
Shijith
> > igbuio_pci_open() does:
> > pci_reset_function(dev);
> > pci_set_master(dev);
> >
> > Don't know yet root cause of the error, will dig more.
> >
> > >>>
> > >>> Could you help to check it soon?
> > >>>
> > >>>
> > >>>
> > >>> [root@localhost app]# ./testpmd -c 7 -n 4 -- -i
> > >>>
> > >>> EAL: Detected 10 lcore(s)
> > >>>
> > >>> EAL: No free hugepages reported in hugepages-1048576kB
> > >>>
> > >>> EAL: Probing VFIO support...
> > >>>
> > >>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using
> > >>> unreliable clock cycles !
> > >>>
> > >>> EAL: PCI device 0000:00:03.0 on NUMA socket -1
> > >>>
> > >>> EAL: Invalid NUMA socket, default to 0
> > >>>
> > >>> EAL: probe driver: 8086:154c net_i40e_vf
> > >>>
> > >>> i40evf_init_vf(): init_adminq failed: -53
> > >>>
> > >>> i40evf_dev_init(): Init vf failed
> > >>>
> > >>> EAL: Requested device 0000:00:03.0 cannot be used
> > >>>
> > >>>
> > >>>
> > >>> commit b58eedfc7dd57eef6d12e2c654a52c834f36084a
> > >>> Author: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> > >>> Date: Fri Jul 7 16:43:51 2017 +0530
> > >>>
> > >>> igb_uio: issue FLR during open and release of device file
> > >>>
> > >>> Set UIO info device file operations open and release. Call pci reset
> > >>> function inside open and release to clear device state at start and end.
> > >>> Copied this behaviour from vfio_pci kernel module code. With this patch,
> > >>> it is not mandatory to issue FLR by PMD's during init and close.
> > >>>
> > >>> Bus master enable and disable are added in open and release respectively
> > >>> to take care of device DMA.
> > >>>
> > >>> Signed-off-by: Shijith Thotton <shijith.thotton@caviumnetworks.com>
> > >>> Reviewed-by: Jianfeng Tan <jianfeng.tan@intel.com>
> > >>> Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > >>> Acked-by: Gregory Etelson <gregory@weka.io>
> > >>>
> > >>>
> > >>>
> > >>> Best Regard,
> > >>>
> > >>> Yang Qiming
> > >>>
> > >>>
> > >>
> > >> Hi Yang Qiming,
> > >>
> > >> We didn't face this issue while preparing patch. I think, Gregory
> > >> tested it for Intel NICs. Adding more people to the loop for help.
> > >>
> > >
> > > I will try to reproduce the issue.
> > >
> > >> Shijith
> > >>
> > >
>
next prev parent reply other threads:[~2017-09-14 7:01 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-13 7:51 Yang, Qiming
2017-09-13 10:48 ` Shijith Thotton
2017-09-13 11:03 ` Ferruh Yigit
2017-09-13 14:25 ` Hu, Xuekun
2017-09-13 17:06 ` Ferruh Yigit
2017-09-13 19:44 ` Patil, Harish
2017-09-15 8:04 ` Yang, Qiming
2017-09-15 8:42 ` Thomas Monjalon
2017-09-15 9:18 ` Yang, Qiming
2017-09-15 9:25 ` Ferruh Yigit
2017-09-15 9:31 ` Shijith Thotton
2017-09-17 2:49 ` Gregory Etelson
2017-09-18 2:21 ` Yang, Qiming
2017-09-18 2:39 ` Zhang, Helin
2017-09-18 3:50 ` Yang, Qiming
2017-09-18 6:49 ` Shijith Thotton
2017-09-18 19:33 ` Ferruh Yigit
2017-09-18 22:43 ` Patil, Harish
2017-09-14 3:16 ` Yang, Qiming
2017-09-14 7:00 ` Shijith Thotton [this message]
2017-09-14 7:15 ` Shijith Thotton
2017-09-14 1:23 ` Yang, Qiming
2017-09-19 10:24 ` [dpdk-dev] [PATCH] igb_uio: remove PCI reset during uio device open Shijith Thotton
2017-09-20 16:52 ` Ferruh Yigit
2017-09-21 10:00 ` Luca Boccassi
2017-09-22 2:47 ` Yang, Qiming
2017-09-29 12:57 ` Wu, Jingjing
2017-10-02 18:24 ` Shijith Thotton
2017-10-03 11:35 ` Gregory Etelson
2017-10-13 14:36 ` Shijith Thotton
2017-10-13 14:51 ` [dpdk-dev] [PATCH] igb_uio: revert open and release operations Thomas Monjalon
2017-10-13 21:05 ` Ferruh Yigit
2017-10-13 21:11 ` Thomas Monjalon
2017-10-13 21:17 ` Thomas Monjalon
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170914070053.GA23787@localhost.localdomain \
--to=shijith.thotton@caviumnetworks.com \
--cc=Harish.Patil@cavium.com \
--cc=dev@dpdk.org \
--cc=ferruh.yigit@intel.com \
--cc=gregory@weka.io \
--cc=jianfeng.tan@intel.com \
--cc=qiming.yang@intel.com \
--cc=xuekun.hu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).