From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 9AB123B5 for ; Fri, 18 Dec 2015 13:07:45 +0100 (CET) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga101.jf.intel.com with ESMTP; 18 Dec 2015 04:07:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,446,1444719600"; d="scan'208";a="620137474" Received: from bricha3-mobl3.ger.corp.intel.com ([10.237.208.64]) by FMSMGA003.fm.intel.com with SMTP; 18 Dec 2015 04:07:42 -0800 Received: by (sSMTP sendmail emulation); Fri, 18 Dec 2015 12:07:41 +0025 Date: Fri, 18 Dec 2015 12:07:41 +0000 From: Bruce Richardson To: =?utf-8?B?5byg5Lyf?= Message-ID: <20151218120740.GA11116@bricha3-MOBL3> References: <44e6e2b3.65ad.151ae293aa7.Coremail.zhangwqh@126.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <44e6e2b3.65ad.151ae293aa7.Coremail.zhangwqh@126.com> Organization: Intel Shannon Ltd. User-Agent: Mutt/1.5.23 (2014-03-12) Cc: dev@dpdk.org Subject: Re: [dpdk-dev] dpdk multi process increase the number of mbufs, throughput gets dropped X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Dec 2015 12:07:46 -0000 On Thu, Dec 17, 2015 at 12:18:36PM +0800, 张伟 wrote: > Hi all, > > > When running the multi process example, does anybody know that why increasing the number of mbufs, the performance gets dropped. > > > In multi process example, there are two macros which are related to the number of mbufs > > > #defineMBUFS_PER_CLIENT1536 > | > | #defineMBUFS_PER_PORT1536 | > | | > > > If increasing these two numbers by 8 times, the performance drops about 10%. Does anybody know why? > > | constunsigned num_mbufs = (num_clients * MBUFS_PER_CLIENT) \ | > | | + (ports->num_ports * MBUFS_PER_PORT); | > | pktmbuf_pool = rte_mempool_create(PKTMBUF_POOL_NAME, num_mbufs, | > | | MBUF_SIZE, MBUF_CACHE_SIZE, | > | | sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init, | > | | NULL, rte_pktmbuf_init, NULL, rte_socket_id(), NO_FLAGS ); | One possible explanation could be due to the memory footprint of the memory pool. While the per-lcore mempool caches of buffers operate in a LIFO (i.e. stack) manner, when mbufs are allocated on one core and freed on another, they pass through a FIFO (i.e. ring) inside the mempool. This means that you iterate through all buffers in the pool in this case, which can cause a slowdown if the mempool size is bigger than your cache. /Bruce