From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by dpdk.org (Postfix) with ESMTP id E5EB7FFA for ; Tue, 12 Sep 2017 22:29:13 +0200 (CEST) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 31BB54ACA5; Tue, 12 Sep 2017 20:29:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 31BB54ACA5 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=aconole@redhat.com Received: from dhcp-25-97.bos.redhat.com (unknown [10.18.25.172]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3623C5E1C9; Tue, 12 Sep 2017 20:29:11 +0000 (UTC) From: Aaron Conole To: Thomas Monjalon Cc: Aaron Conole , Eelco Chaudron , dev@dpdk.org, jingjing.wu@intel.com, john.mcnamara@intel.com References: <22990026376b08418cb0eb6f028840c03e89f47f.1505221429.git.echaudro@redhat.com> <1863612.973jloI4LL@xps> Date: Tue, 12 Sep 2017 16:29:11 -0400 In-Reply-To: <1863612.973jloI4LL@xps> (Thomas Monjalon's message of "Tue, 12 Sep 2017 22:14:01 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.0.50 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 12 Sep 2017 20:29:13 +0000 (UTC) Subject: Re: [dpdk-dev] [PATCH] app/testpmd: adds mlockall() to fix pages X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Sep 2017 20:29:14 -0000 Thomas Monjalon writes: > 12/09/2017 16:50, Aaron Conole: >> Eelco Chaudron writes: >> >> > Call the mlockall() function, to attempt to lock all of its process >> > memory into physical RAM, and preventing the kernel from paging any >> > of its memory to disk. >> > >> > When using testpmd for performance testing, depending on the code path >> > taken, we see a couple of page faults in a row. These faults effect >> > the overall drop-rate of testpmd. On Linux the mlockall() call will >> > prefault all the pages of testpmd (and the DPDK libraries if linked >> > dynamically), even without LD_BIND_NOW. >> > >> > Signed-off-by: Eelco Chaudron >> >> Acked-by: Aaron Conole > > It is interesting, but why make it in testpmd? > > Maybe it should be documented in this guide: > http://dpdk.org/doc/guides/linux_gsg/nic_perf_intel_platform.html Well, I'm not sure what the user would be able to do to get the prefaulting performance without having a library they use with LD_PRELOAD and a function with the constructor attribute which does the same thing, AND export LD_BIND_NOW before linking starts. The LD_BIND_NOW simply does the symbol resolution, but there's no guarantee that it will fault all the code pages in to process space, and without an mlockall(), I'm not sure that there's any kind of guarantee that they don't get swapped out of resident memory (which also leads to later page faults). Maybe I misunderstood the question?