From: "Gupta, Nipun" <Nipun.Gupta@amd.com>
To: "Burakov, Anatoly" <anatoly.burakov@intel.com>,
David Marchand <david.marchand@redhat.com>
Cc: "dev@dpdk.org" <dev@dpdk.org>,
"thomas@monjalon.net" <thomas@monjalon.net>,
"Yigit, Ferruh" <Ferruh.Yigit@amd.com>,
"Agarwal, Nikhil" <nikhil.agarwal@amd.com>
Subject: RE: [PATCH v2] vfio: do not coalesce DMA mappings
Date: Wed, 5 Apr 2023 15:06:29 +0000 [thread overview]
Message-ID: <DM6PR12MB3082A56754C5C011C2A80020E8909@DM6PR12MB3082.namprd12.prod.outlook.com> (raw)
In-Reply-To: <c82cc0a1-4545-2000-0027-15bc7df1b50e@intel.com>
[AMD Official Use Only - General]
> -----Original Message-----
> From: Burakov, Anatoly <anatoly.burakov@intel.com>
> Sent: Wednesday, April 5, 2023 7:46 PM
> To: Gupta, Nipun <Nipun.Gupta@amd.com>; David Marchand
> <david.marchand@redhat.com>
> Cc: dev@dpdk.org; thomas@monjalon.net; Yigit, Ferruh
> <Ferruh.Yigit@amd.com>; Agarwal, Nikhil <nikhil.agarwal@amd.com>
> Subject: Re: [PATCH v2] vfio: do not coalesce DMA mappings
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> On 4/4/2023 5:32 PM, Nipun Gupta wrote:
> >
> >
> > On 4/4/2023 8:43 PM, Burakov, Anatoly wrote:
> >> Caution: This message originated from an External Source. Use proper
> >> caution when opening attachments, clicking links, or responding.
> >>
> >>
> >> On 2/7/2023 8:56 AM, Gupta, Nipun wrote:
> >>> [AMD Official Use Only - General]
> >>>
> >>> Hi David,
> >>>
> >>> I agree that change is not straightforward to review, but it should
> >>> not cause any functional issue as we are still creating all the
> >>> memory mappings, but one by one for each segment.
> >>> For hot plug case this causes issue as mentioned, that VFIO does not
> >>> allow unmap of the individual segments in case mapping was created of
> >>> a single coalesced segment.
> >>>
> >>> But yes, I am not sure why this code was added, which Anatoly may
> >>> have more understanding on.
> >>
> >> The motivation behind this code was that Linux allows limited amount of
> >> page mappings, so we were trying to save on those. However, since then
> >> there have been a few changes related to partial unmaps that may make it
> >> so that this code is not only no longer necessary, but is in fact
> >> actively harmful. I agree that this at least warrants a second look.
> >>
> >>>
> >>> Anatoly,
> >>>
> >>> Can you please provide your feedback on this change?
> >>
> >> The patch probably shouldn't include the mailmap changes :)
> >
> > Sure, will send a separate patch for it.
> >
> >>
> >> Could you please provide some steps to reproduce the hotplug issue
> >> you're having? It would be great to have a test case for this patchset
> >> to put it in context.
> >
> > I am working on CDX bus
> > (http://patchwork.dpdk.org/project/dpdk/patch/20230124140746.594066-2-
> nipun.gupta@amd.com/) and trying out some cases for plug/unplug.
> >
> > The test is as follows:
> > # Run testpmd application
> > ./dpdk-testpmd -c 0x3 -- -i --nb-cores=1
> >
> > # Bind to VFIO
> > echo "vfio-cdx" > /sys/bus/cdx/devices/cdx-00\:00/driver_override
> > echo "cdx-00:00" > /sys/bus/cdx/drivers_probe
> >
> > # Plug a device
> > testpmd> port attach cdx:cdx-00:00
> >
> > #quit testpmd
> > testpmd> quit
> >
> > This gave error at testpmd exit that memory cannot be freed. On
> > debugging I updated this code and seems it should be seen with any of
> > the device.
> >
> > I see similar test case (without quit) mentioned
> > https://doc.dpdk.org/dts/test_plans/hotplug_test_plan.html, but the
> > difference is that it is with igb_uio and issue is being observed with
> > VFIO.
> >
> > Please note the device/bus mentioned in the commands is not yet
> > upstreamed in DPDK, but patches would be sent out soon.
> >
> > Thanks,
> > Nipun
> >
>
> Thanks, I can reproduce this issue with regular devices too (run testpmd
> with no devices, bind a NIC to VFIO, attach it, then quit). You're
> correct in that since the initial mapping was done with mapping large
> contiguous zones (such as when mempools are created before attach), any
> subsequent freeing of memory will cause these errors to happen.
Thanks for trying on other devices too, and good to know that this is also
reproduced with them :)
Regards,
Nipun
>
> I don't think this can be fixed by anything other than not doing the
> contiguous mapping thing, so provisionally, I think this patch should be
> accepted. I'll play around with it some more and get back to you :)
>
> >>
> >> --
> >> Thanks,
> >> Anatoly
> >>
>
> --
> Thanks,
> Anatoly
next prev parent reply other threads:[~2023-04-05 15:06 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-30 9:58 [PATCH] " Nipun Gupta
2023-01-04 5:19 ` [PATCH v2] " Nipun Gupta
2023-02-02 10:48 ` David Marchand
2023-02-07 8:56 ` Gupta, Nipun
2023-04-04 14:57 ` Gupta, Nipun
2023-04-04 15:13 ` Burakov, Anatoly
2023-04-04 16:32 ` Nipun Gupta
2023-04-05 14:16 ` Burakov, Anatoly
2023-04-05 15:06 ` Gupta, Nipun [this message]
2023-04-24 15:22 ` David Marchand
2023-04-24 16:10 ` Stephen Hemminger
2023-04-24 16:16 ` Gupta, Nipun
2023-05-10 12:58 ` Nipun Gupta
2023-05-11 14:08 ` Burakov, Anatoly
2023-04-11 15:13 ` Thomas Monjalon
2023-04-11 16:51 ` Gupta, Nipun
2023-04-07 6:13 ` Nipun Gupta
2023-04-07 7:26 ` David Marchand
2023-05-15 11:16 ` David Marchand
2023-06-29 8:21 ` [PATCH] " Ding, Xuan
2023-06-30 1:45 ` Nipun Gupta
2023-06-30 5:58 ` Ding, Xuan
2023-07-04 5:13 ` Ding, Xuan
2023-07-04 6:53 ` Gupta, Nipun
2023-07-04 8:06 ` Ding, Xuan
2023-07-04 9:23 ` Gupta, Nipun
2023-07-04 14:09 ` Thomas Monjalon
2023-07-04 16:39 ` Gupta, Nipun
2023-07-04 15:11 ` Ding, Xuan
2023-07-04 16:42 ` Gupta, Nipun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM6PR12MB3082A56754C5C011C2A80020E8909@DM6PR12MB3082.namprd12.prod.outlook.com \
--to=nipun.gupta@amd.com \
--cc=Ferruh.Yigit@amd.com \
--cc=anatoly.burakov@intel.com \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=nikhil.agarwal@amd.com \
--cc=thomas@monjalon.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).