From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id A0902333 for ; Tue, 1 Jul 2014 01:43:06 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga102.fm.intel.com with ESMTP; 30 Jun 2014 16:43:05 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.01,578,1400050800"; d="scan'208";a="563259844" Received: from ecsmtp.pdx.intel.com (HELO plxs0284.pdx.intel.com) ([10.25.97.128]) by fmsmga002.fm.intel.com with ESMTP; 30 Jun 2014 16:42:29 -0700 Received: from plxv1142.pdx.intel.com (plxv1142.pdx.intel.com [10.25.98.49]) by plxs0284.pdx.intel.com with ESMTP id s5UNgTRS004449; Mon, 30 Jun 2014 16:42:29 -0700 Received: from plxv1142.pdx.intel.com (localhost [127.0.0.1]) by plxv1142.pdx.intel.com with ESMTP id s5UNgTNp008471; Mon, 30 Jun 2014 16:42:29 -0700 Received: (from jbshaw@localhost) by plxv1142.pdx.intel.com with œ id s5UNgT81008431; Mon, 30 Jun 2014 16:42:29 -0700 Date: Mon, 30 Jun 2014 16:42:29 -0700 From: Jeff Shaw To: Matt Laswell Message-ID: <20140630234229.GA19646@plxv1142.pdx.intel.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: dev@dpdk.org Subject: Re: [dpdk-dev] Ability to/impact of running with smaller page sizes X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jun 2014 23:43:08 -0000 Hi Matt, On Mon, Jun 30, 2014 at 05:43:39PM -0500, Matt Laswell wrote: > Hey Folks, > > In my application, I'm seeing some design considerations in a project I'm > working on that push me towards the use of smaller memory page sizes. I'm > curious - is it possible in practical terms to run DPDK without hugepages? Yes, but I do not believe an implementation exists. > If so, does anybody have any practical experience (or a > back-of-the-envelop estimate) of how badly such a configuration would hurt > performance? For sake of argument, assume that virtually all of the memory > being used is in pre-allocated mempools (e.g lots of rte_mempool_create(), > very little rte_malloc(). > It is possible, though not recommended if you want "good performance", to use smaller memory page sizes. Poor performance results from penalties incurred due to DTLB misses. Please consider the following example. An application pre-allocates several thousand buffers to use for packet reception and transmission using 4KB pages. Each buffer contains 2KB worth of data space, or enough to store the typical maximum Ethernet frame size. Since the page size is only 4KB, each DTLB entry can cache a maximum of two packet buffer address translations. If the first level DTLB has, for instance, 64 x 4KB entries, you would only be able to cache about 128 address translations at any given time (+1,024 if you include the second level DTLB). With 32 x 2MB entries, each DTLB entry can cache address translations for 32K packet buffers at any given time. If you believe that your application performance will be negatively impacted by latencies incurred due to DTLB misses, it is recommended to take steps which would maximize the DTLB hit rate. Of course, you will not know how this impacts performance for you application unless it is tried under realistic conditions. If you end up doing so, could you please update the list? Thanks, Jeff