From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id DDF7AA055F; Fri, 27 May 2022 10:39:06 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 7072F40E5A; Fri, 27 May 2022 10:39:06 +0200 (CEST) Received: from mail-pj1-f48.google.com (mail-pj1-f48.google.com [209.85.216.48]) by mails.dpdk.org (Postfix) with ESMTP id D20CC4003F for ; Fri, 27 May 2022 10:39:04 +0200 (CEST) Received: by mail-pj1-f48.google.com with SMTP id cs3-20020a17090af50300b001e0808b5838so3787143pjb.1 for ; Fri, 27 May 2022 01:39:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=cCAYPJz5DpOQsiqk8u0yzx/S/+oBzxXi3qqBp3F/v6U=; b=HzNT5kUbwbYY2sTnHHL3i9WUbrHCYNDTb/rdjRIH+HQv1SP5ZulTXUIs8eWAjjtDTZ wf6TxAYM37DUBromfBLo8TCgD9aEfXnXK6iLfan5r/2HRCO3iU9LNOklfxP2FemUOcVG r1p3IEY55eZaI0bg6m3P4JA3X+6LLfY04b9w99JinuWp5bB1DPmnRaSQZnW9bgGeR11J QOR2MWeiK+cdMVHRiB5yVLmcNWZblNlROMrPE9axiOHUhufINa/cnhwQMgoJDnmGnX6m uW08ZxfhNSUtAEkaCgbvIv4rmrb40B6w9XoDBt0xzRxlZSsXSclqw+MtPDjGs8qCnOa/ c6QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cCAYPJz5DpOQsiqk8u0yzx/S/+oBzxXi3qqBp3F/v6U=; b=NA51zesV5gKFQySiuSgIczahx2m97Cz2k//Fwuwuho8l6PqcHM6ygKCqJA2hi/Flo1 MIdLrfMw0A7lIamTAYKydOWRxuVoKIlt535jjdDfHEM3aSy5QLO3FyBw6DwwnrmQzKbg 6z9n5H1vSVBrvZTPa8yQWmKGHeDf1tvZ4isWw8pqrRtTz6hHM9GFaPK9tZz1pNgxaBGN kGo1PyKHozdOIcNRFhZr8eCv4IkwDJZRMG45klI/79SNVfkZzB5HE+7i+ovtyb3yevkU 7St+DtJpNkqjcLRWq32hhMVX2y+xabJFIHYvDnEDXBArYMAxMJnTeMDPz/pRJCmrsvr0 ZcVw== X-Gm-Message-State: AOAM530ledjCKzChy0fF+Q2JGDacGgiZOsgchk4crJkuFB00/OMfkomV MdLXuHABhkJ5sH1zkhVPVcGeBlESPltkk0SMJpv5oQ== X-Google-Smtp-Source: ABdhPJz5SXD+3ckVK8wxkyfNw75mUY+iy01PCGWe+2oKQ6S5+245amJeyD43iIByhVkx84DwdhlAdFuMxWS0hDmbM4g= X-Received: by 2002:a17:90b:350d:b0:1e0:436a:d553 with SMTP id ls13-20020a17090b350d00b001e0436ad553mr7074384pjb.147.1653640744069; Fri, 27 May 2022 01:39:04 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Romain Delhomel Date: Fri, 27 May 2022 10:38:52 +0200 Message-ID: Subject: Re: [RFC] bus/fslmc: fix invalid use of default vfio config To: hemant.agrawal@nxp.com, sachin.saxena@oss.nxp.com Cc: dev@dpdk.org, Olivier Matz , stephane gonauer , Guillaume Gaudonville , Elias Boutaleb Content-Type: multipart/alternative; boundary="00000000000023028005dffa3d02" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org --00000000000023028005dffa3d02 Content-Type: text/plain; charset="UTF-8" On Fri, May 27, 2022 at 10:34 AM Romain Delhomel wrote: > > > On Fri, May 27, 2022 at 10:28 AM Romain Delhomel < > romain.delhomel@6wind.com> wrote: > >> At device probe, the fslmc bus driver calls rte_vfio_get_group_fd() to >> get a fd associated to a vfio group. This function first checks if the >> group is already opened, else it opens /dev/vfio/%u, and increases the >> number of active groups in default_vfio_cfg (which references the >> default vfio container). >> >> When adding the first group to a vfio_cfg, the caller is supposed to >> pick an IOMMU type and set up DMA mappings for container, as it's done >> by pci bus, but it is not done here. Instead, a new container is created >> and used. >> >> This prevents the pci bus driver, which use the default_vfio_cfg >> container, to configure the container because >> default_vfio_cfg->active_group > 1. >> >> This patch fixes the issue by always creating a new container (and its >> associated vfio_cfg) and bind the group to it. >> >> Fixes: a69f79300262 ("bus/fslmc: support multi VFIO group") >> Signed-off-by: Romain Delhomel >> --- >> drivers/bus/fslmc/fslmc_vfio.c | 10 +++++++++- >> 1 file changed, 9 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/bus/fslmc/fslmc_vfio.c >> b/drivers/bus/fslmc/fslmc_vfio.c >> index b4704eeae4e2..abe1cab2ee20 100644 >> --- a/drivers/bus/fslmc/fslmc_vfio.c >> +++ b/drivers/bus/fslmc/fslmc_vfio.c >> @@ -979,6 +979,7 @@ fslmc_vfio_setup_group(void) >> { >> int groupid; >> int ret; >> + int vfio_container_fd; >> struct vfio_group_status status = { .argsz = sizeof(status) }; >> >> /* if already done once */ >> @@ -997,8 +998,15 @@ fslmc_vfio_setup_group(void) >> return 0; >> } >> >> + ret = rte_vfio_container_create(); >> + if (ret < 0) { >> + DPAA2_BUS_ERR("Failed to open VFIO container"); >> + return ret; >> + } >> + vfio_container_fd = ret; >> + >> /* Get the actual group fd */ >> - ret = rte_vfio_get_group_fd(groupid); >> + ret = rte_vfio_container_group_bind(vfio_container_fd, groupid); >> if (ret < 0) >> return ret; >> vfio_group.fd = ret; >> -- >> 2.30.2 >> > --00000000000023028005dffa3d02 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Fri, May 27, 2022 at 10:34 AM Romain Delhomel <<= a href=3D"mailto:romain.delhomel@6wind.com">romain.delhomel@6wind.com&g= t; wrote:


On Fri, May 27, 2022 at 10:28 AM Romain Delhomel <romain.delhomel@6w= ind.com> wrote:
At device probe, the fslmc bus driver calls rte_vfi= o_get_group_fd() to
get a fd associated to a vfio group. This function f= irst checks if the
group is already opened, else it opens /dev/vfio/%u, = and increases the
number of active groups in default_vfio_cfg (which ref= erences the
default vfio container).

When adding the first group = to a vfio_cfg, the caller is supposed to
pick an IOMMU type and set up D= MA mappings for container, as it's done
by pci bus, but it is not do= ne here. Instead, a new container is created
and used.

This preve= nts the pci bus driver, which use the default_vfio_cfg
container, to con= figure the container because
default_vfio_cfg->active_group > 1.
This patch fixes the issue by always creating a new container (and it= s
associated vfio_cfg) and bind the group to it.

Fixes: a69f79300= 262 ("bus/fslmc: support multi VFIO group")
Signed-off-by: Rom= ain Delhomel <romain.delhomel@6wind.com>
---
=C2=A0drivers/bus/fslmc/fs= lmc_vfio.c | 10 +++++++++-
=C2=A01 file changed, 9 insertions(+), 1 dele= tion(-)

diff --git a/drivers/bus/fslmc/fslmc_vfio.c b/drivers/bus/fs= lmc/fslmc_vfio.c
index b4704eeae4e2..abe1cab2ee20 100644
--- a/driver= s/bus/fslmc/fslmc_vfio.c
+++ b/drivers/bus/fslmc/fslmc_vfio.c
@@ -979= ,6 +979,7 @@ fslmc_vfio_setup_group(void)
=C2=A0{
=C2=A0 int groupid;=
=C2=A0 int ret;
+ int vfio_container_fd;
=C2=A0 struct vfio_group= _status status =3D { .argsz =3D sizeof(status) };
=C2=A0
=C2=A0 /* if= already done once */
@@ -997,8 +998,15 @@ fslmc_vfio_setup_group(void)<= br>=C2=A0 return 0;
=C2=A0 }
=C2=A0
+ ret =3D rte_vfio_container_= create();
+ if (ret < 0) {
+ DPAA2_BUS_ERR("Failed to open V= FIO container");
+ return ret;
+ }
+ vfio_container_fd =3D r= et;
+
=C2=A0 /* Get the actual group fd */
- ret =3D rte_vfio_get_= group_fd(groupid);
+ ret =3D rte_vfio_container_group_bind(vfio_containe= r_fd, groupid);
=C2=A0 if (ret < 0)
=C2=A0 return ret;
=C2=A0 = vfio_group.fd =3D ret;
--
2.30.2
--00000000000023028005dffa3d02--