From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by dpdk.org (Postfix) with ESMTP id 88D871396 for ; Sun, 17 Sep 2017 04:49:45 +0200 (CEST) Received: by mail-wm0-f43.google.com with SMTP id v142so15252502wmv.5 for ; Sat, 16 Sep 2017 19:49:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=weka.io; s=google; h=from:to:cc:subject:date:message-id:organization:in-reply-to :references:mime-version:content-transfer-encoding; bh=5X2QtkNrl0zn5iYbXY57beyUJCybv60E0uNa5ARyWO0=; b=Ih6HAfoI85q03/adEp8Tb6FekfkasJ3vypJXOrxBpy/Gx1WFgdUYjuvsSV3UtPeKUD vj7IhR3sOYISo0cXeWcfADksJKD+i2s/vkx6CWh0YqX7W0UFHz0ekjMCO0azRzBDBH09 8UJgbJz0kwkkwBaMCYLrkiv8jJahoxejOVFIA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:organization :in-reply-to:references:mime-version:content-transfer-encoding; bh=5X2QtkNrl0zn5iYbXY57beyUJCybv60E0uNa5ARyWO0=; b=OHmGGiOlJYsn9SldmQO0lC6UqBAcehvx6sN9b04buAREsaOGRmGSNzb2KqAlR143bx vWW6XEnVfxwEGjEIvfOWe+s3oVAgB8v/WiV/YOWc3KiaspYTrhYc8I2d6mdf2gr3vSeP Z9kHSTIsoMFDFJsPxD9nMRFgcSDij1S9nHttESCKBB+sz+PccB9vqZlaW0E/kgfzaBRa JPg7ultWFvhlRfXgwsmwYaidjtpxYyRhiONF61fimZ6BSEdeeOL4bKQfiFn8KklvyfMk AutzsDhqoRoO0cFHWv405+dLy7jPt5yz/71dXvjdeQOBAUB7fTky0oigffY/4+yo1zwU ftJQ== X-Gm-Message-State: AHPjjUjoSuNDv/8f5kQ8IWSyxWnBCOJ+4p8rcY1j+upsRB5lrKDDXsbg v30OGkX2hjGIKZz5 X-Google-Smtp-Source: AOwi7QCxp1QM3cTZY6tUpy1HZKhAneJ2E+mR1ep0xYeLF88FrnI5XQCsUAlMU9Q+jFOP9arCl3qqOQ== X-Received: by 10.28.94.85 with SMTP id s82mr5352619wmb.13.1505616585208; Sat, 16 Sep 2017 19:49:45 -0700 (PDT) Received: from polaris.localnet (bzq-82-81-85-138.red.bezeqint.net. [82.81.85.138]) by smtp.gmail.com with ESMTPSA id j4sm3719576wrg.96.2017.09.16.19.49.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 16 Sep 2017 19:49:43 -0700 (PDT) From: Gregory Etelson To: "Yang, Qiming" Cc: "thomas@monjalon.net" , "dev@dpdk.org" , "Tan, Jianfeng" , "Thotton, Shijith" , "Hu, Xuekun" , "Yigit, Ferruh" , "Patil, Harish" Date: Sun, 17 Sep 2017 05:49:41 +0300 Message-ID: <2430472.fsKqsvm3Ry@polaris> Organization: Weka.IO In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7Bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open and release of device file X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Sep 2017 02:49:45 -0000 Hello Qiming, Did you try to remove PCI reset at igbuio_pci_open ? Reverting the patch entirely will get us back to VF issues Regards, Gregory On Friday, 15 September 2017 11:04:16 IDT Yang, Qiming wrote: > Hi, Thomas > We meet a urgent issue because of Shijith's patch. > When use DPDK version 17.08, DPDK VF driver and kernel PF driver, i40e, ixgbe and qede(Harish said) VF all meet initial failure. > This issue only occurred when use 17.08 igb_uio, vfio-pci and 17.05 igb_uio all works well. > Considering this issue will block our vf driver develop and test work, could we revert this patch first and find another appropriate way? > > Qiming > > > -----Original Message----- > > From: Patil, Harish [mailto:Harish.Patil@cavium.com] > > Sent: Thursday, September 14, 2017 3:45 AM > > To: Yigit, Ferruh ; Hu, Xuekun ; > > Thotton, Shijith ; Yang, Qiming > > ; Gregory Etelson > > Cc: dev@dpdk.org; Tan, Jianfeng > > Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open > > and release of device file > > > > > > > > -----Original Message----- > > From: dev on behalf of Ferruh Yigit > > > > Date: Wednesday, September 13, 2017 at 10:06 AM > > To: "Hu, Xuekun" , "Thotton, Shijith" > > , "Yang, Qiming" , > > Gregory Etelson > > Cc: "dev@dpdk.org" , "Tan, Jianfeng" > > Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR during open > > and release of device file > > > > >On 9/13/2017 3:25 PM, Hu, Xuekun wrote: > > >> I met the same issue too, only with i40e 2.1.26 PF kernel driver. > > >> > > >> -----Original Message----- > > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ferruh Yigit > > >> Sent: Wednesday, September 13, 2017 7:04 PM > > >> To: Shijith Thotton ; Yang, > > >>Qiming ; Gregory Etelson > > >> Cc: dev@dpdk.org; Tan, Jianfeng > > >> Subject: Re: [dpdk-dev] vf init issue with patch igb_uio: issue FLR > > >>during open and release of device file > > >> > > >> On 9/13/2017 11:48 AM, Shijith Thotton wrote: > > >>> On Wed, Sep 13, 2017 at 07:51:30AM +0000, Yang, Qiming wrote: > > >>>> Hi, Shijith > > >>>> > > >>>> > > >>>> > > >>>> VF init error will happen after apply your patch, error log as > > >>>>below. If > > >>>> revert your commit, all things work well. And this issue is not > > >>>>only occur > > >>>> in i40 VF but also ixgbe. > > > > > >Hi Qiming, > > > > > >I can reproduce the issue for the case: > > > > > >1- i40e 2.1.26 PF kernel driver (NOT reproduced with 1.6.27-k) > > >2- DPDK app run on host, so both PF and VF are in host. Linux driver > > >for PF and DPDK for VF. (When VF used in a VM, Linux PF on host and > > >DPDK VF on guest, this works fine) > > > > > >And I confirm the function igbuio_pci_open() added with below patch is > > >causing it. > > > > > >igbuio_pci_open() does: > > > pci_reset_function(dev); > > > pci_set_master(dev); > > > > > >Don't know yet root cause of the error, will dig more. > > > > > >>>> > > >>>> Could you help to check it soon? > > >>>> > > >>>> > > >>>> > > >>>> [root@localhost app]# ./testpmd -c 7 -n 4 -- -i > > >>>> > > >>>> EAL: Detected 10 lcore(s) > > >>>> > > >>>> EAL: No free hugepages reported in hugepages-1048576kB > > >>>> > > >>>> EAL: Probing VFIO support... > > >>>> > > >>>> EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using > > >>>> unreliable clock cycles ! > > >>>> > > >>>> EAL: PCI device 0000:00:03.0 on NUMA socket -1 > > >>>> > > >>>> EAL: Invalid NUMA socket, default to 0 > > >>>> > > >>>> EAL: probe driver: 8086:154c net_i40e_vf > > >>>> > > >>>> i40evf_init_vf(): init_adminq failed: -53 > > >>>> > > >>>> i40evf_dev_init(): Init vf failed > > >>>> > > >>>> EAL: Requested device 0000:00:03.0 cannot be used > > >>>> > > >>>> > > >>>> > > >>>> commit b58eedfc7dd57eef6d12e2c654a52c834f36084a > > >>>> Author: Shijith Thotton > > >>>> Date: Fri Jul 7 16:43:51 2017 +0530 > > >>>> > > >>>> igb_uio: issue FLR during open and release of device file > > >>>> > > >>>> Set UIO info device file operations open and release. Call pci > > >>>>reset > > >>>> function inside open and release to clear device state at start > > >>>>and end. > > >>>> Copied this behaviour from vfio_pci kernel module code. With > > >>>>this patch, > > >>>> it is not mandatory to issue FLR by PMD's during init and close. > > >>>> > > >>>> Bus master enable and disable are added in open and release > > >>>>respectively > > >>>> to take care of device DMA. > > >>>> > > >>>> Signed-off-by: Shijith Thotton > > >>>> Reviewed-by: Jianfeng Tan > > >>>> Acked-by: Ferruh Yigit > > >>>> Acked-by: Gregory Etelson > > >>>> > > >>>> > > >>>> > > >>>> Best Regard, > > >>>> > > >>>> Yang Qiming > > >>>> > > >>>> > > >>> > > >>> Hi Yang Qiming, > > >>> > > >>> We didn't face this issue while preparing patch. I think, Gregory > > >>> tested it for Intel NICs. Adding more people to the loop for help. > > >>> > > >> > > >> I will try to reproduce the issue. > > >> > > >>> Shijith > > > > [Harish] We are seeing similar issue with the bringup of qede VF devices due to > > this patch. > > Could we revert this and leave it to individual PMDs to manage it? > > >> > >