From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wi0-f180.google.com (mail-wi0-f180.google.com [209.85.212.180]) by dpdk.org (Postfix) with ESMTP id 80C1D5A37 for ; Mon, 1 Jun 2015 19:51:06 +0200 (CEST) Received: by wizo1 with SMTP id o1so114813395wiz.1 for ; Mon, 01 Jun 2015 10:51:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=sY1CfphIVSIZZ4oVSKAmQ9gLWrtnBg8cjHz34YCuI84=; b=OEqweriLApMAixo1AdRKmpNGHxd9SE9blpgQDJWZfifnuKqzREvmyB0eVcEFRzCTns /rBux9ls/gBwBtN6ZPMJGMSUb6FFKHL5fcavQZUxkx3NqH3lSaQMzD4LxcxnkziAkdCw RlJ8+9OvBbXoxPojNQlStIGKR1teg7QT0h0UT8XYIdAbRthkgig2alGo6NztVGjBlmzs 3XvgVlxL6ehGbFHOxRLO+u9y+oTsznzIxFdUVXK6t3mm2+4a0zLC3gVQII7z6CxrWhQp I59cFrsaGt3iQ38i5WCED2/Ytk33NbG2q6wSIb2N0chs3mF1ugcYWO4jzC1npe7dI2Nx amDA== X-Gm-Message-State: ALoCoQke7cyWw4Nx4XKyIkF9J7Z3oECMkid9tSUnij8uHzQXkBj6zagH1jb+wxzAN81iuvsLTZ2X X-Received: by 10.180.107.70 with SMTP id ha6mr22888395wib.20.1433181066376; Mon, 01 Jun 2015 10:51:06 -0700 (PDT) Received: from [192.168.0.101] ([90.152.119.35]) by mx.google.com with ESMTPSA id bw5sm2592040wjc.31.2015.06.01.10.51.05 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 01 Jun 2015 10:51:05 -0700 (PDT) Message-ID: <556C9B88.9080409@linaro.org> Date: Mon, 01 Jun 2015 18:51:04 +0100 From: Zoltan Kiss User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Andriy Berestovskyy , dev@dpdk.org References: <55689B42.6060800@linaro.org> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] Free up completed TX buffers X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Jun 2015 17:51:06 -0000 On 01/06/15 09:50, Andriy Berestovskyy wrote: > Hi Zoltan, > > On Fri, May 29, 2015 at 7:00 PM, Zoltan Kiss wrote: >> The easy way is just to increase your buffer pool's size to make >> sure that doesn't happen. > > Go for it! I went for it, my question is whether is it a good and worthwhile idea to give the applications a last resort option for rainy days? It's a problem which probably won't occur very often, but when it does, I think it can take painfully long until you figure out what's wrong. > >> But there is no bulletproof way to calculate such >> a number > > Yeah, there are many places for mbufs to stay :( I would try: > > Mempool size = sum(numbers of all TX descriptors) > + sum(rx_free_thresh) > + (mempool cache size * (number of lcores - 1)) > + (burst size * number of lcores) It heavily depends on what your application does, and I think it's easy to make a mistake in these calculations. > >> I'm thinking about a foolproof way, which is exposing functions like >> ixgbe_tx_free_bufs from the PMDs, so the application can call it as a last >> resort to avoid deadlock. > > Have a look at rte_eth_dev_tx_queue_stop()/start(). Some NICs (i.e. > ixgbe) do reset the queue and free all the mbufs. That's a bit drastic, I just want to flush the finished TX buffers, even if tx_free_thresh were not reached. An easy option would be to use rte_eth_tx_burst(..., nb_pkts=0), I'm already using this to enforce TX completion if it's really needed. It checks for tx_free_thresh, like this: /* Check if the descriptor ring needs to be cleaned. */ if ((txq->nb_tx_desc - txq->nb_tx_free) > txq->tx_free_thresh) i40e_xmit_cleanup(txq); My idea is to extend this condition and add " || nb_pkts == 0", so you can force cleanup. But there might be others who uses this same way to do manual TX completion, and they expect that it only happens when tx_free_thresh is reached. > > Regards, > Andriy >