From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vk0-f47.google.com (mail-vk0-f47.google.com [209.85.213.47]) by dpdk.org (Postfix) with ESMTP id 2C5591E20 for ; Wed, 5 Apr 2017 09:12:34 +0200 (CEST) Received: by mail-vk0-f47.google.com with SMTP id z204so2359826vkd.1 for ; Wed, 05 Apr 2017 00:12:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netronome-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=4NIUSMme1Z+dqqf1EAxmuknxAw0VeLX77Q3RHxBdi5s=; b=j7myJR9Vs4HleV9Y6q7ln2+9AhedhOaV45Q1Ekakgp1dHg0QfIDjUzI9r1KTwnz6RI 8aSOsOFwJBO968ANzPaZA8iENC/umDpK2PDTsDsk9VUhgfsim0D/X4JbPeCYK/p2PAgp cDuNbO66ruskGSwGal1tGZ44s3LIRTdWm1uiEBWiZHlDTkQZfxVDxLY81B4x36Cw4SkE 5L+Kj7Or9WYIK7ITxVScswO6Ytus3hyhpiFmdnKZ3NZ0kbMpFfq1Ah1b2OLL+IqOh8PH bNstiupecEB2X/PPC/liHHN9doag0XXMgrwVKrT28tAR2++th15gq09duHWRlOFZip7G Q53A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=4NIUSMme1Z+dqqf1EAxmuknxAw0VeLX77Q3RHxBdi5s=; b=WW7b40RvNU/SrPVMqB8bOQwNg1G6mv1yx0rKiqC3oUbsngujqQ2vkEuAm4BV8eWOQp 9fUP8IWS5wM+PZGdTwv0R3yGZgF6xpLEuD8fBKiHYWD4Age7tGO/FiihdFcrmZJRpig1 osHHNXLr7kPhAAn/odkMaWEGozDqKnosdkx8+VuV82TmWXIsq4Zd3u/ZItZmveErdpwY z0zR7tvzgb/FnPI8mHhZ2M+6HL8itdLVTW/FRAHpb5//5aQCN1zsvezhhHQ05pwOUgjO IuYwTx5vkv+VYb90ttPBWuhv2+WVwz3fE3cNCse4KvROUZebcK4L2PtpVgcRqmPrGdOk cCZA== X-Gm-Message-State: AFeK/H2ruVGmI9eq4KgnleSubLGb4NhTII1GQQnJWk7VRRt5F84CVcXHT5+x1nS2yvnYivbxmP0/UxZw2UOKYAHv X-Received: by 10.31.1.7 with SMTP id 7mr12120559vkb.0.1491376354161; Wed, 05 Apr 2017 00:12:34 -0700 (PDT) MIME-Version: 1.0 Received: by 10.103.43.4 with HTTP; Wed, 5 Apr 2017 00:12:33 -0700 (PDT) In-Reply-To: References: <694e35cd-8c9d-715b-9b6b-482b5c802846@solarflare.com> From: Alejandro Lucero Date: Wed, 5 Apr 2017 08:12:33 +0100 Message-ID: To: "Burakov, Anatoly" Cc: Andrew Rybchenko , "dev@dpdk.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] vfio: failed to select IOMMU type X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Apr 2017 07:12:35 -0000 Hi Andrew, Anatoly, On Tue, Apr 4, 2017 at 4:52 PM, Burakov, Anatoly wrote: > Hi Andrew, > > > I think a key to the main problem is the same IOMMU group used for both > PCI functions. > > It tries to set IOMMU type using the same file descriptor twice. The > second set is dummy, since the same value is set, but still fails, I guess, > because it is already in use. > > See logs with debug enabled and few extra logs below: > > Yes you're right. Specifically, eal_vfio.c:vfio_setup_device() at line > 311 (where we check for number of groups) - the code always assumes that one > active group means we've just initialized a new group, which may not > necessarily be the case if there's more than one device per group. > > Yes, the code was not aware of that possibility. Being honest, I knew about that, but not in my mind when implementing the code. I have just cards where each PF even VF have their own VFIO group. This could be a problem for testing. Anatoly, do you have a system with that peculiarity? > Alejandro, please correct me if I'm wrong, but I think this raises another > issue: vfio_release_device() seems to attempt to close group fd > unconditionally, > which is probably a bad idea if there are more than one device per group. > > Yes, I think so, but not completely sure. Doing a quick look at kernel VFIO code, that seems a problem, but need more time for studying the code again. Again, not having hardware with this peculiarity will make the process harder. > Would you be so kind to come up with a patch to fix this oversight, or > should I do it? :) > > Yes, I will work on this as a priority and send a patch asap. Andrew, could you test this hotplug option in some way with your systems? Thanks > Thanks, > Anatoly >