From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dispatch1-us1.ppe-hosted.com (dispatch1-us1.ppe-hosted.com [67.231.154.164]) by dpdk.org (Postfix) with ESMTP id 53BB22FDD for ; Tue, 4 Apr 2017 18:10:52 +0200 (CEST) Received: from pure.maildistiller.com (unknown [10.110.50.29]) by dispatch1-us1.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTP id EB84E60071; Tue, 4 Apr 2017 16:10:51 +0000 (UTC) X-Virus-Scanned: Proofpoint Essentials engine Received: from mx4-us3.ppe-hosted.com (unknown [10.110.49.251]) by pure.maildistiller.com (Proofpoint Essentials ESMTP Server) with ESMTPS id 1B45A60049; Tue, 4 Apr 2017 16:10:51 +0000 (UTC) Received: from webmail.solarflare.com (uk.solarflare.com [193.34.186.16]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx4-us3.ppe-hosted.com (Proofpoint Essentials ESMTP Server) with ESMTPS id EA4AA800B5; Tue, 4 Apr 2017 16:10:48 +0000 (UTC) Received: from [192.168.38.17] (84.52.89.52) by ukex01.SolarFlarecom.com (10.17.10.4) with Microsoft SMTP Server (TLS) id 15.0.1044.25; Tue, 4 Apr 2017 17:10:44 +0100 To: "Burakov, Anatoly" , "dev@dpdk.org" , Alejandro Lucero References: <694e35cd-8c9d-715b-9b6b-482b5c802846@solarflare.com> From: Andrew Rybchenko Message-ID: Date: Tue, 4 Apr 2017 19:10:39 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: X-Originating-IP: [84.52.89.52] X-ClientProxiedBy: ocex03.SolarFlarecom.com (10.20.40.36) To ukex01.SolarFlarecom.com (10.17.10.4) X-TM-AS-Product-Ver: SMEX-11.0.0.1191-8.100.1062-22984.003 X-TM-AS-Result: No--12.633600-0.000000-31 X-TM-AS-User-Approved-Sender: Yes X-TM-AS-User-Blocked-Sender: No X-MDID: 1491322251-q4bu+lkM+iKV Content-Type: text/plain; charset="utf-8"; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] vfio: failed to select IOMMU type X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 16:10:52 -0000 On 04/04/2017 06:52 PM, Burakov, Anatoly wrote: > Hi Andrew, > >> I think a key to the main problem is the same IOMMU group used for both PCI functions. >> It tries to set IOMMU type using the same file descriptor twice. The second set is dummy, since the same value is set, but still fails, I guess, because it is already in use. >> See logs with debug enabled and few extra logs below: > Yes you're right. Specifically, eal_vfio.c:vfio_setup_device() at line 311 (where we check for number of groups) - the code always assumes that one > active group means we've just initialized a new group, which may not necessarily be the case if there's more than one device per group. > > Alejandro, please correct me if I'm wrong, but I think this raises another issue: vfio_release_device() seems to attempt to close group fd unconditionally, > which is probably a bad idea if there are more than one device per group. If it is true, please, care about it. > Would you be so kind to come up with a patch to fix this oversight, or should I do it? :) Please, take a look at http://dpdk.org/dev/patchwork/patch/23202/ Basically it just moves IOMMU type set under previous container-not-set condition. Also my testing is very restricted, no hotplug, no different IOMMU groups. Andrew.