From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi0-f53.google.com (mail-oi0-f53.google.com [209.85.218.53]) by dpdk.org (Postfix) with ESMTP id 7452E7EB0 for ; Tue, 2 Dec 2014 03:35:53 +0100 (CET) Received: by mail-oi0-f53.google.com with SMTP id x69so8219954oia.26 for ; Mon, 01 Dec 2014 18:35:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sidebandnetworks.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=PmmeWVjGlAGld7V7c03c7W5c7wMBio/R+GGWEQmE83c=; b=Zj8sAPVObg6xMvJJHYRpj6WhPn+7wuzx7CHJJqGbyd/IX/OfYsXZ6PWco0YMSQoOHr ghwgK25MSh6ZSdHS+8CgtZ8l5RZCS3lL0pTrg2yB1tyNon25nl0WGFrnRfxdMi2KtEua r7PWCdDIkLUnp73gWg5RQXzssrMRYqBTtIZso= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=PmmeWVjGlAGld7V7c03c7W5c7wMBio/R+GGWEQmE83c=; b=lMsSvPqg3beAms9KREJajp5EKvP0/JEwRNtwu78Y3ei7qVKQFbnqlGAIzw3mYeORq+ qREZf57QwIPR9LvVpgq/8d8ACtnPE0BnWwp9wiBZzyaswtKpi/+fGbSZITw3maWJO8kt /ea3pow5uCXa2ryFlU2fHcHy0bP5D7P6YHHUaUlW3ePgiwRYRGuxAj6FKHq5w3J4B+jz Q8Id1/wNBuktm6nFFDHdmWA+EqWnUo1t7q4CMglwJnXXZeDevLF1Mb4VFXoRXZK89nLn RJfraw6dkYzzZ6aBJt0rAG/lL+ePNy+1RnEuolS4ac5L5/6JYJAKvboluT+ErFNRjJ70 XzAQ== X-Gm-Message-State: ALoCoQmRsO9vUDNwpbLeqaAPRR6ekdlYPQceSyRcNWSELNBhZ7YOYJMeDXVM024hdDONdyfThyfl MIME-Version: 1.0 X-Received: by 10.202.229.76 with SMTP id c73mr33945315oih.69.1417487752423; Mon, 01 Dec 2014 18:35:52 -0800 (PST) Received: by 10.202.48.79 with HTTP; Mon, 1 Dec 2014 18:35:52 -0800 (PST) In-Reply-To: <547C3BB7.5010309@6wind.com> References: <547C3BB7.5010309@6wind.com> Date: Mon, 1 Dec 2014 21:35:52 -0500 Message-ID: From: Kamraan Nasim To: Olivier MATZ Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: dev@dpdk.org, Steve Noble , Ashish Juneja Subject: Re: [dpdk-dev] RTE mempool "used count" steadily goes down to zero despite steady packet throughput X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Dec 2014 02:35:54 -0000 Thanks Oliver for the helpful tip, that settles the confusion as to why Free count grows and used shrinks. I've modified the rte_mempool library to maintain atomic pktmbuf malloc and free stats, hopefully that would help pinpoint the leak. Thanks again for your help! --Kam On Mon, Dec 1, 2014 at 4:58 AM, Olivier MATZ wrote: > Hi Kam, > > On 11/28/2014 06:34 PM, Kamraan Nasim wrote: > > I have ~15Gbps of traffic flowing through two 10GE ports and been > profiling > > the rte mempool(or rather the pktmbuf mempool) memory consumption: > > > > I have per lcore caching disabled(cache_size is 0) > > > > I have noticed that: > > - Mempool FREE cnt(as given byt rte_mempool_free_count()) increases > > - Mempool USED cnt(as given by rte_mempool_used_count() decreases and > > eventually drops to 0. When this happens, mempool reports itself as EMPTY > > - rx_nombuf stats for the eth ports start climbing > > - Valgrind Memcheck does not indicate any obvious leaks in RTE mempool or > > my application. > > > > > > I was wondering if others have come across this issue? Or if people here > > have used ways, besides Valgrind to profile the mempool or the pkt mbuf > > pool? > > First, be careful with rte_mempool_free_count(): it returns the number > of free entries in the internal ring of the mempool, which actually > corresponds to the number of allocated objects from the mempool point > of view. See: > http://dpdk.org/browse/dpdk/tree/lib/librte_mempool/rte_mempool.h > > If you have the number of allocated objects increasing, and the > statistics rx_nombuf increasing when the mbuf pool is empty, it means > that you have a mbuf leak in your application. Valgrind won't see it > since it does not know about mempool alloc/free functions. > > Regards, > Olivier >