From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id C6000429DC; Mon, 24 Apr 2023 17:23:01 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A299E40ED6; Mon, 24 Apr 2023 17:23:01 +0200 (CEST) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by mails.dpdk.org (Postfix) with ESMTP id F31B040DF8 for ; Mon, 24 Apr 2023 17:23:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1682349780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EV6XAujhNEooH2ryCACHTPlNkRNzEgLPRT2pAjiKWqQ=; b=HcxIIsRxixgBEySyHD/S2M711EcJdbBt9cs4H4ij4ay1Qz7triZfBGcUzCoS0ZAbHg6yeX 0mYiF6aVQ4BnWUJLfGN0onuqIxTbEMZM3B94Uyehi9+4435hZ9mqAhoacDE+gV3+9IEuUJ iRB7krvZQBz8gvp5aMPHxVhvs6LvpaU= Received: from mail-pj1-f70.google.com (mail-pj1-f70.google.com [209.85.216.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-619-UxV_QW2YMpmwpQBkW6QTYg-1; Mon, 24 Apr 2023 11:22:59 -0400 X-MC-Unique: UxV_QW2YMpmwpQBkW6QTYg-1 Received: by mail-pj1-f70.google.com with SMTP id 98e67ed59e1d1-247c0ee87aaso2726712a91.0 for ; Mon, 24 Apr 2023 08:22:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682349778; x=1684941778; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EV6XAujhNEooH2ryCACHTPlNkRNzEgLPRT2pAjiKWqQ=; b=Wte8Z/rNjgvIefw2hnCThrVoSuZ4c6vgWq0qTpxlcgz4aHolSlZKE+DTyiv6WPjxIC WXkgDbaNaIIiEKsuIk6HMHk1X1nrBUOelswXWwKIzvDh1d8J4mMdzGN7Jo5ksv9kcJM6 D2hrTzjEYpwNikklgOPRRBsCVLI4/N2N5hzvkO7bEE3RvppokHD/E3ZpH4UipOIL2U/z dMFAJwR6RMLr/t+rCkRJwiImXKcNkkodYT5YV1YnlbOmzzax5Fq57kKbyY4ybUVk1k7N xrqQCMAz9cAo8yuaI198mdGYxVN0rpUrAiIdKam14dDWqWcWezrcyXBaPpa+9n8Jkpe4 3lvQ== X-Gm-Message-State: AAQBX9datjgQ6wGpXILPG8EzMVdr6FlyY5L/hj7AU0T7URiTOoCjcGBH ZIHlHxelC2AB6SJ6JpyQ37TuhH6z/EOx/R3r+SkuKpEmvpDmu9pcvt4XXpXa9Do7yzrdS3ewOsc wLR6XhAycMN6ZXmuaEek= X-Received: by 2002:a17:90a:4989:b0:247:bdeb:594f with SMTP id d9-20020a17090a498900b00247bdeb594fmr14337395pjh.15.1682349778354; Mon, 24 Apr 2023 08:22:58 -0700 (PDT) X-Google-Smtp-Source: AKy350YLdY0gaSgOcBjGlOJv21RPssPE7kmU41l/h6m44IHpfM9FSo4iUhB4Np+uArF6gU6nLIAhFHa+jOe4G9gM9wg= X-Received: by 2002:a17:90a:4989:b0:247:bdeb:594f with SMTP id d9-20020a17090a498900b00247bdeb594fmr14337377pjh.15.1682349778027; Mon, 24 Apr 2023 08:22:58 -0700 (PDT) MIME-Version: 1.0 References: <20221230095853.1323616-1-nipun.gupta@amd.com> <20230104051936.2456411-1-nipun.gupta@amd.com> <0d356823-b198-ebd1-7a28-b678ae9edaf9@intel.com> <4e912b50-c17f-cfb4-c8e3-f3db4f96178c@amd.com> In-Reply-To: From: David Marchand Date: Mon, 24 Apr 2023 17:22:46 +0200 Message-ID: Subject: Re: [PATCH v2] vfio: do not coalesce DMA mappings To: "Burakov, Anatoly" Cc: Nipun Gupta , "dev@dpdk.org" , "thomas@monjalon.net" , "Yigit, Ferruh" , "Agarwal, Nikhil" X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Hello Anatoly, On Wed, Apr 5, 2023 at 4:17=E2=80=AFPM Burakov, Anatoly wrote: > >> Could you please provide some steps to reproduce the hotplug issue > >> you're having? It would be great to have a test case for this patchset > >> to put it in context. > > > > I am working on CDX bus > > (http://patchwork.dpdk.org/project/dpdk/patch/20230124140746.594066-2-n= ipun.gupta@amd.com/) and trying out some cases for plug/unplug. > > > > The test is as follows: > > # Run testpmd application > > ./dpdk-testpmd -c 0x3 -- -i --nb-cores=3D1 > > > > # Bind to VFIO > > echo "vfio-cdx" > /sys/bus/cdx/devices/cdx-00\:00/driver_override > > echo "cdx-00:00" > /sys/bus/cdx/drivers_probe > > > > # Plug a device > > testpmd> port attach cdx:cdx-00:00 > > > > #quit testpmd > > testpmd> quit > > > > This gave error at testpmd exit that memory cannot be freed. On > > debugging I updated this code and seems it should be seen with any of > > the device. > > > > I see similar test case (without quit) mentioned > > https://doc.dpdk.org/dts/test_plans/hotplug_test_plan.html, but the > > difference is that it is with igb_uio and issue is being observed with > > VFIO. > > > > Please note the device/bus mentioned in the commands is not yet > > upstreamed in DPDK, but patches would be sent out soon. > > > > Thanks, > > Nipun > > > > Thanks, I can reproduce this issue with regular devices too (run testpmd > with no devices, bind a NIC to VFIO, attach it, then quit). You're > correct in that since the initial mapping was done with mapping large > contiguous zones (such as when mempools are created before attach), any > subsequent freeing of memory will cause these errors to happen. > > I don't think this can be fixed by anything other than not doing the > contiguous mapping thing, so provisionally, I think this patch should be > accepted. I'll play around with it some more and get back to you :) Can we conclude on this topic? It is best we merge this kind of change the sooner possible for a release. Thanks. --=20 David Marchand