Re: [PATCH v4] bus/pci: fix legacy device IO port map in secondary process

[David Marchand <david.marchand@redhat.com>]

On Wed, Aug 30, 2023 at 7:07 AM Wenwu Ma <wenwux.ma@intel.com> wrote:
>
> When doing IO port mapping for legacy device in secondary process, the
> region information is missing, so, we need to refill it.
>
> Fixes: 4b741542ecde ("bus/pci: avoid depending on private kernel value")
> Cc: stable@dpdk.org
>
> Signed-off-by: Wenwu Ma <wenwux.ma@intel.com>
>
> ---
>  drivers/bus/pci/linux/pci_vfio.c | 43 ++++++++++++++++++++++++++++++--
>  1 file changed, 41 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c
> index e634de8322..5ef26c98d1 100644
> --- a/drivers/bus/pci/linux/pci_vfio.c
> +++ b/drivers/bus/pci/linux/pci_vfio.c
> @@ -1314,6 +1314,27 @@ pci_vfio_ioport_map(struct rte_pci_device *dev, int bar,
>                 return -1;
>         }
>
> +       if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
> +               struct vfio_device_info device_info = { .argsz = sizeof(device_info) };
> +               char pci_addr[PATH_MAX];
> +               int vfio_dev_fd;
> +               struct rte_pci_addr *loc = &dev->addr;
> +               int ret;
> +               /* store PCI address string */
> +               snprintf(pci_addr, sizeof(pci_addr), PCI_PRI_FMT,
> +                               loc->domain, loc->bus, loc->devid, loc->function);
> +
> +               ret = rte_vfio_setup_device(rte_pci_get_sysfs_path(), pci_addr,
> +                                               &vfio_dev_fd, &device_info);

From a pci bus API pov, nothing prevents a driver from mixing memory
mapped with vfio and ioport resources (iow, calls to
rte_pci_map_resource() and rte_pci_ioport_map()).
IOW, it may not be the case with the net/virtio driver but, in theory,
rte_pci_ioport_map()/pci_vfio_ioport_map() may be called after a
rte_pci_map_resource() call.

In a similar manner, from the API pov,
rte_pci_ioport_map()/pci_vfio_ioport_map() may be called for multiple
bars.

In summary, nothing in this patch checks that vfio has been configured
already and I think we need a refcount to handle those situations.

Nipun, Chenbo, WDYT?


> +               if (ret)
> +                       return -1;

ret value is not used, so there is no need for this variable.

if (rte_vfio_setup_device(rte_pci_get_sysfs_path(), pci_addr,
        &vfio_dev_fd, &device_info) != 0)
    return -1;

> +
> +               ret = pci_vfio_fill_regions(dev, vfio_dev_fd, &device_info);
> +               if (ret)
> +                       return -1;

Same here, ret is not needed.


> +
> +       }
> +
>         if (pci_vfio_get_region(dev, bar, &size, &offset) != 0) {
>                 RTE_LOG(ERR, EAL, "Cannot get offset of region %d.\n", bar);
>                 return -1;
> @@ -1361,8 +1382,26 @@ pci_vfio_ioport_write(struct rte_pci_ioport *p,
>  int
>  pci_vfio_ioport_unmap(struct rte_pci_ioport *p)
>  {
> -       RTE_SET_USED(p);
> -       return -1;
> +       char pci_addr[PATH_MAX] = {0};
> +       struct rte_pci_addr *loc = &p->dev->addr;
> +       int ret, vfio_dev_fd;
> +
> +       /* store PCI address string */
> +       snprintf(pci_addr, sizeof(pci_addr), PCI_PRI_FMT,
> +                       loc->domain, loc->bus, loc->devid, loc->function);
> +
> +       vfio_dev_fd = rte_intr_dev_fd_get(p->dev->intr_handle);
> +       if (vfio_dev_fd < 0)
> +               return -1;

This check is odd and does not seem related.


> +
> +       ret = rte_vfio_release_device(rte_pci_get_sysfs_path(), pci_addr,
> +                                     vfio_dev_fd);
> +       if (ret < 0) {
> +               RTE_LOG(ERR, EAL, "Cannot release VFIO device\n");
> +               return ret;
> +       }


-- 
David Marchand