From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qg0-f54.google.com (mail-qg0-f54.google.com [209.85.192.54]) by dpdk.org (Postfix) with ESMTP id 6FB372BFE for ; Mon, 18 Apr 2016 19:18:25 +0200 (CEST) Received: by mail-qg0-f54.google.com with SMTP id f52so120355829qga.3 for ; Mon, 18 Apr 2016 10:18:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to:cc; bh=ZhNmNhGbC0QfdjU0iw3eN+xtsDDnfI+8fDcVpaZvC/E=; b=Fnh7v4gTwyD/sa/R1qmihWtXEOJ+ZpTSZsaSiof1e5nUHcu4UywhF2m5ow2jnT3Lok JO797p0lkyyLW6CiBZzecffWW57ix4l6BwghVptJvtS/BSJUOWP1Oam5qWey9mdc+bBy AZM1AhpvwLMbbVcfs/0Qno9oqtlGLZ9Hghdv6Dj/ooQ19b/i7KkGqccyYUtFvEotY6bM va1arj3VSCPlM2+8VIeRjWScjzTExq+UfvQo1Fcrc8GUC0AunVY+xx9kcf8FMsdNQgVG tq/wVvDZ4qKUIiDe5ztvHu4qI3WEL4CK7SJj/g+CRYZbtqQtSsq/LdJxNPDoGjSw3Uf9 5uVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=ZhNmNhGbC0QfdjU0iw3eN+xtsDDnfI+8fDcVpaZvC/E=; b=UKl9zphZnVQt89hPLDSgBeLQEuoxSWDaJtmXMw31UgfGGQEl+hikhdvfrguMM8UCGr pjMqoVUE8JMpC6N3NpuVr0BvYjf5SsxldNxMVSKxmLaKpaxICS7WoWGNFp5y0FKhTINT UdQlOBkhhSBv0xmPCiS08FOaEGW3JKB/ZrHb3/VfCnQlaM0cBwNtjsdpC1fJWvLWwXGD R5N6bEM8T9q1rXENPnRaRcLrXXev+BOeTtvXWnfsCkuopG/A2ZlKZ7Wk/Gi5+9npagQ6 rUYXKXo5Z6X+I3lO008BW3Nt8W2j0I4BwAorlDNviKwTas343093bzNTCUcQ5cv9x227 Ivyw== X-Gm-Message-State: AOPr4FVTzB8wiCilUU9baeapW8MwQf1lznjB0RsMiKLKK04atEsLYLVjz8ZCGJZjIDiQf5q6MJTEwystQwQaVe91 X-Received: by 10.55.79.17 with SMTP id d17mr45793578qkb.56.1460999904773; Mon, 18 Apr 2016 10:18:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.212.91 with HTTP; Mon, 18 Apr 2016 10:18:05 -0700 (PDT) From: Christian Ehrhardt Date: Mon, 18 Apr 2016 19:18:05 +0200 Message-ID: To: dev Cc: Daniele Di Proietto Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Memory leak when adding/removing vhost_user ports X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Apr 2016 17:18:25 -0000 I assume there is a leak somewhere on adding/removing vhost_user ports. Although it could also be "only" a fragmentation issue. Reproduction is easy: I set up a pair of nicely working OVS-DPDK connected KVM Guests. Then in a loop I - add up to more 512 ports - test connectivity between the two guests - remove up to 512 ports Depending on memory and the amount of multiqueue/rxq I use it seems to slightly change when exactly it breaks. But for my default setup of 4 queues and 5G Hugepages initialized by DPDK it always breaks at the sixth iteration. Here a link to the stack trace indicating a memory shortage (TBC): https://launchpadlibrarian.net/253916410/apport-retrace.log Known Todos: - I want to track it down more, and will try to come up with a non openvswitch based looping testcase that might show it as well to simplify debugging. - in use were Openvswitch-dpdk 2.5 and DPDK 2.2; Retest with DPDK 16.04 and Openvswitch master is planned. I will go on debugging this and let you know, but I wanted to give a heads up to everyone. In case this is a known issue for some of you please let me know. Kind Regards, Christian Ehrhardt Software Engineer, Ubuntu Server Canonical Ltd P.S. I think it is a dpdk issue, but adding Daniele on CC to represent ovs-dpdk as well.