From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout3.w1.samsung.com (mailout3.w1.samsung.com [210.118.77.13]) by dpdk.org (Postfix) with ESMTP id 0F99CA2F for ; Thu, 21 Apr 2016 13:01:29 +0200 (CEST) Received: from eucpsbgm2.samsung.com (unknown [203.254.199.245]) by mailout3.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0O5Z0049WDAFEJ30@mailout3.w1.samsung.com> for dev@dpdk.org; Thu, 21 Apr 2016 12:01:27 +0100 (BST) X-AuditID: cbfec7f5-f792a6d000001302-75-5718b307bfcb Received: from eusync3.samsung.com ( [203.254.199.213]) by eucpsbgm2.samsung.com (EUCPMTA) with SMTP id 0A.B4.04866.703B8175; Thu, 21 Apr 2016 12:01:27 +0100 (BST) Received: from [106.109.129.180] by eusync3.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0O5Z00MAQDAEJW70@eusync3.samsung.com>; Thu, 21 Apr 2016 12:01:27 +0100 (BST) To: dev@dpdk.org, christian.ehrhardt@canonical.com References: Cc: Daniele Di Proietto , "dev@openvswitch.org" , "yuanhan.liu@linux.intel.com" , Dyasly Sergey From: Ilya Maximets Message-id: <5718B306.5070801@samsung.com> Date: Thu, 21 Apr 2016 14:01:26 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-version: 1.0 In-reply-to: Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrDLMWRmVeSWpSXmKPExsVy+t/xq7rsmyXCDe6cFbSYu3oDu8W7T9uZ LI6e3sNs8bL7HrvFlfaf7BaTZ0tZXJ9wgdWB3WNWQy+bx68FS1k95p0M9Hh28z+jR9+WVYwe 7+a/ZQtgi+KySUnNySxLLdK3S+DKOH3sJHPBUYGKc41PmRoYJ/B2MXJySAiYSJyY/44NwhaT uHBvPZDNxSEksJRRonPzbSYI5wWjxKl9O9m7GDk4RASMJHaezQQxhQQCJHoWcYD0Cgt4Shyd 3MgEYjMLHGOU6FroBmKzCehInFp9hBHE5hXQklh7YD8ziM0ioCrx9PhzsHpRgQiJw51d7BA1 ghI/Jt9jAbE5BYIl1r5YD7aVWUBdYsqUXIjx8hKb17xlnsAoMAtJxyyEqllIqhYwMq9iFE0t TS4oTkrPNdIrTswtLs1L10vOz93ECAnzrzsYlx6zOsQowMGoxMO7QUYiXIg1say4MvcQowQH s5IIb9YmoBBvSmJlVWpRfnxRaU5q8SFGaQ4WJXHembvehwgJpCeWpGanphakFsFkmTg4pRoY q+dJ9an1T5/RLrxLYsbhFM4rrGujs6Y0XP4SprDQLeNGxwKVEx3aW+xP+jkJb7tRFSz7M/Jc U4bbT7Vv3yUrCzwYZ9kejHKINXQs9XqzReXpzx/fTyv9XyNntkVcVjvqV83RSIe5S4+9f/ft rdxXg7Wvtz2/fCD9aSSzmJzQjSv7tN3PhKuWKLEUZyQaajEXFScCAAdIyYhvAgAA Subject: Re: [dpdk-dev] Memory leak when adding/removing vhost_user ports X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Apr 2016 11:01:29 -0000 Hi, Christian. You're, likely, using tar archive with openvswitch from openvswitch.org. It doesn't contain many bug fixes from git/branch-2.5 unfortunately. The problem that you are facing has been solved in branch-2.5 by commit d9df7b9206831631ddbd90f9cbeef1b4fc5a8e89 Author: Ilya Maximets Date: Thu Mar 3 11:30:06 2016 +0300 netdev-dpdk: Fix memory leak in netdev_dpdk_vhost_destruct(). Fixes: 4573fbd38fa1 ("netdev-dpdk: Add vhost-user multiqueue support") Signed-off-by: Ilya Maximets Acked-by: Flavio Leitner Acked-by: Daniele Di Proietto Best regards, Ilya Maximets. > I assume there is a leak somewhere on adding/removing vhost_user ports. > Although it could also be "only" a fragmentation issue. > > Reproduction is easy: > I set up a pair of nicely working OVS-DPDK connected KVM Guests. > Then in a loop I > - add up to more 512 ports > - test connectivity between the two guests > - remove up to 512 ports > > Depending on memory and the amount of multiqueue/rxq I use it seems to > slightly change when exactly it breaks. But for my default setup of 4 > queues and 5G Hugepages initialized by DPDK it always breaks at the sixth > iteration. > Here a link to the stack trace indicating a memory shortage (TBC): > https://launchpadlibrarian.net/253916410/apport-retrace.log > > Known Todos: > - I want to track it down more, and will try to come up with a non > openvswitch based looping testcase that might show it as well to simplify > debugging. > - in use were Openvswitch-dpdk 2.5 and DPDK 2.2; Retest with DPDK 16.04 and > Openvswitch master is planned. > > I will go on debugging this and let you know, but I wanted to give a heads > up to everyone. > In case this is a known issue for some of you please let me know. > > Kind Regards, > Christian Ehrhardt > Software Engineer, Ubuntu Server > Canonical Ltd > > P.S. I think it is a dpdk issue, but adding Daniele on CC to represent > ovs-dpdk as well.