From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 9BAA34319A; Wed, 18 Oct 2023 15:45:01 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 86DC4402CD; Wed, 18 Oct 2023 15:45:01 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id 37A4840261 for ; Wed, 18 Oct 2023 15:45:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1697636699; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3E88y4c4aNKIMX3PAJmFKsceotJ8dgZgM3ZuA9c+9+U=; b=WK5rv4Yw6lEo/njWhATKLLWJCb3/ErrOLjoM7+a0AoPC70y+FWKGlsF7p3y3PwOKxZxLKG 7kEHfpDVGlbJPEjbpqFR3CUvGPiM/nrRNZ0GteabYmIPl0jrgh6x9nv8MQXMDz8fuMUDV1 wNEXBuUUnA3rRtejclfiPaIQiqgECtg= Received: from mail-lj1-f197.google.com (mail-lj1-f197.google.com [209.85.208.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-257-mGQsCP91O3WY9Vr_ynsUjw-1; Wed, 18 Oct 2023 09:44:53 -0400 X-MC-Unique: mGQsCP91O3WY9Vr_ynsUjw-1 Received: by mail-lj1-f197.google.com with SMTP id 38308e7fff4ca-2c5047f94bdso39692931fa.1 for ; Wed, 18 Oct 2023 06:44:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697636692; x=1698241492; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3E88y4c4aNKIMX3PAJmFKsceotJ8dgZgM3ZuA9c+9+U=; b=wS34iatFNygh2KudcirYihmAzIkK9Cw8R3j4c4Hso6YW7Q4YCHlJKXVYsmWc2dWHUu +cQlXtmUujXljpwAFygafYNS1Ks5Bu7jslua39r+wBkkHg1jJjqJFU6ujczncZmJkUay Zpz+LazmbUXr6QdTMTP8VW5L135NGElk3Wmsfk/9g+KPQHQNLFPE3iniE0vi+qyOTOar h7l4GWcSN4hC9pZi89tanaPQpflfAwXN0sYcuaqsd+miOaGYTQSz67vJM4I2sO49IIjo KQKR7Q97Dha5OmBvjknreyXRL0K/WyWEeDeG00ha6TaN30wnCAOM5KSJvRSvj+WWOYxl LjNQ== X-Gm-Message-State: AOJu0YzR8bfRSoA/4PlwYsbIMGgPGuxOtMsgREsZltRRrW9tr5MGF76b coxni0Pr+bYW3PHvZLsqo9sy3glybrxAgu72PjFfTCn48rNIBoJmmW7jxhChV5nsbmezZ6URaFu M0UxjC48B2AEqCuhRYfc= X-Received: by 2002:a2e:b5da:0:b0:2c5:25f3:8e1c with SMTP id g26-20020a2eb5da000000b002c525f38e1cmr3541517ljn.21.1697636692143; Wed, 18 Oct 2023 06:44:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE5W9pCa04hJ+ZDg5CHQ6rfU+6oCpW1YuAJUhSu9XA05mmSgVf3/Ra+CwVzgr55e1F3cDSE5To/yunVk2MTdrA= X-Received: by 2002:a2e:b5da:0:b0:2c5:25f3:8e1c with SMTP id g26-20020a2eb5da000000b002c525f38e1cmr3541495ljn.21.1697636691708; Wed, 18 Oct 2023 06:44:51 -0700 (PDT) MIME-Version: 1.0 References: <20230807015820.1329972-1-wenwux.ma@intel.com> <20230830050713.58247-1-wenwux.ma@intel.com> <2ba09d01-8241-43a9-be90-5fbbfc75b2e4@amd.com> In-Reply-To: <2ba09d01-8241-43a9-be90-5fbbfc75b2e4@amd.com> From: David Marchand Date: Wed, 18 Oct 2023 15:44:39 +0200 Message-ID: Subject: Re: [PATCH v4] bus/pci: fix legacy device IO port map in secondary process To: "Gupta, Nipun" Cc: "Ma, WenwuX" , "chenbo.xia@outlook.com" , "dev@dpdk.org" , "maxime.coquelin@redhat.com" , "Li, Miao" , "Ling, WeiX" , "stable@dpdk.org" X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org On Wed, Oct 18, 2023 at 2:39=E2=80=AFPM Gupta, Nipun = wrote: > On 10/18/2023 3:35 PM, David Marchand wrote: > > On Mon, Oct 9, 2023 at 5:06=E2=80=AFAM Ma, WenwuX = wrote: > >>> From a pci bus API pov, nothing prevents a driver from mixing memory > >>> mapped with vfio and ioport resources (iow, calls to > >>> rte_pci_map_resource() and rte_pci_ioport_map()). > >>> IOW, it may not be the case with the net/virtio driver but, in theory= , > >>> rte_pci_ioport_map()/pci_vfio_ioport_map() may be called after a > >>> rte_pci_map_resource() call. > >>> > >>> In a similar manner, from the API pov, > >>> rte_pci_ioport_map()/pci_vfio_ioport_map() may be called for multiple= bars. > >>> > >>> In summary, nothing in this patch checks that vfio has been configure= d already > >>> and I think we need a refcount to handle those situations. > >>> > >> We call rte_vfio_setup_device just to get device info, we can call rte= _vfio_release_device as soon as pci_vfio_fill_regions is done. > >> This avoids reference counting operations, do you think it works? > > > > Afaics, rte_vfio_setup_device should not be called if a call to > > rte_pci_map_device for this device was successful (rte_pci_map_device > > itself calls rte_vfio_setup_device). > > And as a consequence, calling rte_vfio_release_device cannot be done > > unconditionnally neither. > > Hi David, > > AFAIU, c() is written as re-entrant and does not > create the DMA mapping again if it is already done for the iommu group. > > When this API is called again either for a device within the same group > or from the device for which it is already called, it mainly only does > the work for device info get. Though not the best thing to use like > this, but if this is called multiple times it should not have any > negative impact. Even if rte_vfio_setup_device() is reentrant, there is still the question when to call rte_vfio_release_device(). > > As Wenmu mention that they need only device info from VFIO, a separate > API to get device info can be added in eal_vfio.c/h. The device info > portion of rte_vfio_setup_device() can be moved out to a new API, and > rte_vfio_setup_device() can call this new API too? Ok, I think I understand your suggestion. Do we have a reference to the vfio device fd stored somewhere in the pci device object? I don't think it is the case, but if the pci layer keeps a reference to it (it would be populated/reset during rte_pci_map_device/rte_pci_unmap_device), then the ioport code can call the VFIO_DEVICE_GET_INFO ioctl() similarly to what is done for irq msi info, and there is no need for a new EAL api. For the case when this device fd is not available (no previous call to rte_pci_map_device()), then the ioport code can call rte_vfio_setup_device() / rte_vfio_release_device(). Is this what you have in mind? --=20 David Marchand