From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id 6548C93B8 for ; Mon, 5 Oct 2015 22:20:31 +0200 (CEST) Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (Postfix) with ESMTPS id BB8F4A2C29; Mon, 5 Oct 2015 20:20:30 +0000 (UTC) Received: from redhat.com (ovpn-116-21.ams2.redhat.com [10.36.116.21]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with SMTP id t95KKSej006951; Mon, 5 Oct 2015 16:20:28 -0400 Date: Mon, 5 Oct 2015 23:20:27 +0300 From: "Michael S. Tsirkin" To: "Xie, Huawei" Message-ID: <20151005231323-mutt-send-email-mst@redhat.com> References: <20150929161628.GA3810@redhat.com> <20150930003531-mutt-send-email-mst@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Cc: "dev@dpdk.org" Subject: Re: [dpdk-dev] Unlinking hugepage backing file after initialiation X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2015 20:20:31 -0000 On Mon, Oct 05, 2015 at 01:08:52PM +0000, Xie, Huawei wrote: > On 9/30/2015 5:36 AM, Michael S. Tsirkin wrote: > > On Tue, Sep 29, 2015 at 05:50:00PM +0000, shesha Sreenivasamurthy (shesha) wrote: > >> Sure. Then, is there any real reason why the backing files should not be > >> unlinked ? > > AFAIK qemu unlinks them already. > Sorry, i didn't make it clear. Let us take the physical Ethernet > controller in the host for example > > 1) DPDK app1 unlinked huge page after initialization. > 2) DPDK app1 crashed or got killed unexpectedly. > 3) The nic device is just DMAing to the buffer memory allocated from > the huge page. > 4) Another app2 started, allocated memory from the hugetlbfs, and the > memory allocated happened to be the buffer memory. > Ok, the nic device dmaed to memory of app2, which corrupted app2. > Btw, the window opened is very very narrow, but we could avoid this > corruption if we don't unlink huge page immediately. We could > reinitialize the nic through binding operation and then remove the huge > page. > > I mentioned virtio at the first time. For its case, the one who does DMA > is vhost and i am talking about the guest huge page not the huge pages > used to back guest memory. > > So we had better not unlink huge pages unless we have other solution to > avoid the corruption. Oh, I get it now. It's when you (ab)use UIO to bypass all normal kernel protections. There's no problem when using VFIO. So kernel doesn't protect you in case of a crash, but I guess you can try to protect yourself. For example, write a separate service that you can pass the hugepage FDs and the device FDs to. Have it hold on to them, and when it detects your app crashed, have it reset the device before closing the FDs. Just make sure that one doesn't crash :). But really, people should just use VFIO. -- MST