From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-vs1-f66.google.com (mail-vs1-f66.google.com [209.85.217.66]) by dpdk.org (Postfix) with ESMTP id 62B7C37AF for ; Tue, 30 Apr 2019 17:39:13 +0200 (CEST) Received: by mail-vs1-f66.google.com with SMTP id b23so619862vso.1 for ; Tue, 30 Apr 2019 08:39:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=5VXKzGYO4Qu7iWNiE2TzYvIdKwrsxIyznNKEekVASKQ=; b=X8YDMJei3SI8xrTGQFf49tY+E/HRZ1PehhLEPSitTl0hikYwYOEKk8T/IxrkspyFu4 5krh+o17ydi5or/ziP/LVxwmX0if1gK2Pph3mWC2L0l1e7W+fooXEfXAYbSLKhO78OQc rarmzdNlkNOdcyyzKhN0txEPstpgvYwSteR5A/KVKVg/hA2km/5n5W9RW65xtWXdJrjZ 9KXhzlr3iVwBC1XTqs4TvDzmYbSMKZo0w6QpleyAbHuPv2AE6/JHvjKyLgNaJDpSxzwO 7nzI26fxX1S8rP2CALfL0qQ2r97imZCeuMkPLa8BJ4RAvyuQ6tNq6saphFQsQbIFuPxq 5NfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=5VXKzGYO4Qu7iWNiE2TzYvIdKwrsxIyznNKEekVASKQ=; b=PXsb6YGVEFiJpo+VNh7NTkN65swaM79OiO2uNFO6ePJxNOBrc8uv2TqfqoB90ruukK ibcQ8rRCPnZnqRx//BACcSZafBJUMPhvnHqqcjaDOqA40EtHujq8qSIb1G3GG8W2h+ZE j+JVgIx7nRurFY/+Yb4Q7iDLbawmhY+4YsgJxyRPuKuFZ+ONcHeyaZUN6nH+aUFpHYP2 zsrWxAdW+i2Zr0oj/jdr8ScvhbS2YRwdsvytPOum2ZbJdUFHno+YFcR49DhZydcwLsQw jAlRFqlhPgTkB/YLpTnXNUJfmhn4ATvEkj4slPYBi8PmiqOsnyPkYWeZ52jAIrIw+soV uDbg== X-Gm-Message-State: APjAAAXwQQMuRLu5ztYtIXUomN5YOo4tgpUNhcy2zBp4kBT2zye61uyh vH8bKm213bSvqBdxnJi3PAU4D9rsGRyjcZbV70QTJYDW X-Google-Smtp-Source: APXvYqwHoMXxDzwIRUqXy+QTVoznnpjftSWu5HV14MxUGbpmdBlUxJ/lj6OyfZ+E3BH12S8juuRHOg4zOBYm5j/sMc8= X-Received: by 2002:a67:7cd1:: with SMTP id x200mr5376545vsc.144.1556638752888; Tue, 30 Apr 2019 08:39:12 -0700 (PDT) MIME-Version: 1.0 References: <9c7588c8-b1b8-7363-5b44-209a2bf1fa0e@intel.com> In-Reply-To: From: Paras Jha Date: Tue, 30 Apr 2019 11:39:01 -0400 Message-ID: To: Ferruh Yigit Cc: dev@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] rte_eth_tx_burst improperly freeing mbufs from KNI mbuf pool X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 15:39:13 -0000 Sorry, I meant that "the mbufs will not be freed as the threshold for freeing seems to not be based on the pool the mbuf originated from, but based on the pool the PMD is configured to use" On Tue, Apr 30, 2019 at 11:37 AM Paras Jha wrote: > Hi, > > I think this issue seems to be due to how the PMD frees mbufs. If the PMD > is configured with pool X, and KNI configured with pool Y, and pool Y has > far fewer mbufs available than pool X, when an application calls tx_burst > on the PMD, the mbufs will not be freed as the threshold for freeing seems > to be based on how many mbufs are in the pool the mbuf originated from. > Setting the amount of mbufs in pool X and Y to be equal resolved the issue, > even with arbitrarily small or large counts. This kind of seems like a > "gotcha" where it isn't immediately clear without experimentation. > > On Tue, Apr 30, 2019 at 10:59 AM Ferruh Yigit > wrote: > >> On 4/10/2019 2:10 PM, Paras Jha wrote: >> > Hi all, >> > >> > I've been chasing down a strange issue related to rte_kni_tx_burst. >> > >> > My application calls rte_kni_rx_burst, which allocates from a discrete >> mbuf >> > pool using kni_allocate_mbufs. That traffic is immediately sent to >> > rte_eth_tx_burst which does not seem to be freeing mbufs even upon >> > succesful completion. >> > >> > My application follows the standard model of freeing mbufs only if the >> > number of tx mbufs is less than the rx mbufs - however, after sending as >> > many mbufs as there are in the pool, I get KNI: Out of memory soon after >> > when calling rte_kni_rx_burst >> > >> > My concern is that if I free all mbufs allocated by the KNI during >> > rte_kni_rx_burst the application seems to work as intended without >> memory >> > leaks, even though this goes against how the actual PMDs work. Is this a >> > bug, or intended behavior? The documented examples on the DPDK website >> seem >> > to only free mbufs if it fails to send, even with the KNI example. >> >> The behavior in the KNI sample application is correct thing to do, free >> only the >> mbufs failed to Tx. As far as I understand you are doing same thing as >> the kni >> sample app, so it should be OK, sample app works fine. >> >> 'rte_eth_tx_burst()' sends packets to the kernel, so it shouldn't free the >> mbufs, right, userspace can't know when kernel side will be done with >> them. >> When kernel side is done, it puts the mbufs into 'free_q' fifo, >> 'kni_free_mbufs()' pulls the mbuf from 'free_q' fifo and frees them. >> So the mbufs sent via 'rte_eth_tx_burst()' freed asynchronously. >> I hope this helps. >> >> From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from dpdk.org (dpdk.org [92.243.14.124]) by dpdk.space (Postfix) with ESMTP id 5FC73A0679 for ; Tue, 30 Apr 2019 17:39:14 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 36DDF4CC3; Tue, 30 Apr 2019 17:39:14 +0200 (CEST) Received: from mail-vs1-f66.google.com (mail-vs1-f66.google.com [209.85.217.66]) by dpdk.org (Postfix) with ESMTP id 62B7C37AF for ; Tue, 30 Apr 2019 17:39:13 +0200 (CEST) Received: by mail-vs1-f66.google.com with SMTP id b23so619862vso.1 for ; Tue, 30 Apr 2019 08:39:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=5VXKzGYO4Qu7iWNiE2TzYvIdKwrsxIyznNKEekVASKQ=; b=X8YDMJei3SI8xrTGQFf49tY+E/HRZ1PehhLEPSitTl0hikYwYOEKk8T/IxrkspyFu4 5krh+o17ydi5or/ziP/LVxwmX0if1gK2Pph3mWC2L0l1e7W+fooXEfXAYbSLKhO78OQc rarmzdNlkNOdcyyzKhN0txEPstpgvYwSteR5A/KVKVg/hA2km/5n5W9RW65xtWXdJrjZ 9KXhzlr3iVwBC1XTqs4TvDzmYbSMKZo0w6QpleyAbHuPv2AE6/JHvjKyLgNaJDpSxzwO 7nzI26fxX1S8rP2CALfL0qQ2r97imZCeuMkPLa8BJ4RAvyuQ6tNq6saphFQsQbIFuPxq 5NfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=5VXKzGYO4Qu7iWNiE2TzYvIdKwrsxIyznNKEekVASKQ=; b=PXsb6YGVEFiJpo+VNh7NTkN65swaM79OiO2uNFO6ePJxNOBrc8uv2TqfqoB90ruukK ibcQ8rRCPnZnqRx//BACcSZafBJUMPhvnHqqcjaDOqA40EtHujq8qSIb1G3GG8W2h+ZE j+JVgIx7nRurFY/+Yb4Q7iDLbawmhY+4YsgJxyRPuKuFZ+ONcHeyaZUN6nH+aUFpHYP2 zsrWxAdW+i2Zr0oj/jdr8ScvhbS2YRwdsvytPOum2ZbJdUFHno+YFcR49DhZydcwLsQw jAlRFqlhPgTkB/YLpTnXNUJfmhn4ATvEkj4slPYBi8PmiqOsnyPkYWeZ52jAIrIw+soV uDbg== X-Gm-Message-State: APjAAAXwQQMuRLu5ztYtIXUomN5YOo4tgpUNhcy2zBp4kBT2zye61uyh vH8bKm213bSvqBdxnJi3PAU4D9rsGRyjcZbV70QTJYDW X-Google-Smtp-Source: APXvYqwHoMXxDzwIRUqXy+QTVoznnpjftSWu5HV14MxUGbpmdBlUxJ/lj6OyfZ+E3BH12S8juuRHOg4zOBYm5j/sMc8= X-Received: by 2002:a67:7cd1:: with SMTP id x200mr5376545vsc.144.1556638752888; Tue, 30 Apr 2019 08:39:12 -0700 (PDT) MIME-Version: 1.0 References: <9c7588c8-b1b8-7363-5b44-209a2bf1fa0e@intel.com> In-Reply-To: From: Paras Jha Date: Tue, 30 Apr 2019 11:39:01 -0400 Message-ID: To: Ferruh Yigit Cc: dev@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] rte_eth_tx_burst improperly freeing mbufs from KNI mbuf pool X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Message-ID: <20190430153901.vPS37WXctzbc_Y-_GCdOKwl76Tcd_CPEjvxhwoYvnlU@z> Sorry, I meant that "the mbufs will not be freed as the threshold for freeing seems to not be based on the pool the mbuf originated from, but based on the pool the PMD is configured to use" On Tue, Apr 30, 2019 at 11:37 AM Paras Jha wrote: > Hi, > > I think this issue seems to be due to how the PMD frees mbufs. If the PMD > is configured with pool X, and KNI configured with pool Y, and pool Y has > far fewer mbufs available than pool X, when an application calls tx_burst > on the PMD, the mbufs will not be freed as the threshold for freeing seems > to be based on how many mbufs are in the pool the mbuf originated from. > Setting the amount of mbufs in pool X and Y to be equal resolved the issue, > even with arbitrarily small or large counts. This kind of seems like a > "gotcha" where it isn't immediately clear without experimentation. > > On Tue, Apr 30, 2019 at 10:59 AM Ferruh Yigit > wrote: > >> On 4/10/2019 2:10 PM, Paras Jha wrote: >> > Hi all, >> > >> > I've been chasing down a strange issue related to rte_kni_tx_burst. >> > >> > My application calls rte_kni_rx_burst, which allocates from a discrete >> mbuf >> > pool using kni_allocate_mbufs. That traffic is immediately sent to >> > rte_eth_tx_burst which does not seem to be freeing mbufs even upon >> > succesful completion. >> > >> > My application follows the standard model of freeing mbufs only if the >> > number of tx mbufs is less than the rx mbufs - however, after sending as >> > many mbufs as there are in the pool, I get KNI: Out of memory soon after >> > when calling rte_kni_rx_burst >> > >> > My concern is that if I free all mbufs allocated by the KNI during >> > rte_kni_rx_burst the application seems to work as intended without >> memory >> > leaks, even though this goes against how the actual PMDs work. Is this a >> > bug, or intended behavior? The documented examples on the DPDK website >> seem >> > to only free mbufs if it fails to send, even with the KNI example. >> >> The behavior in the KNI sample application is correct thing to do, free >> only the >> mbufs failed to Tx. As far as I understand you are doing same thing as >> the kni >> sample app, so it should be OK, sample app works fine. >> >> 'rte_eth_tx_burst()' sends packets to the kernel, so it shouldn't free the >> mbufs, right, userspace can't know when kernel side will be done with >> them. >> When kernel side is done, it puts the mbufs into 'free_q' fifo, >> 'kni_free_mbufs()' pulls the mbuf from 'free_q' fifo and frees them. >> So the mbufs sent via 'rte_eth_tx_burst()' freed asynchronously. >> I hope this helps. >> >>