From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mx3.ddos-guard.net (mx3.ddos-guard.net [190.115.16.1]) by dpdk.org (Postfix) with ESMTP id 9EA158E7B for ; Sun, 1 Nov 2015 13:06:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ddos-guard.net; s=ddos-guard; t=1446379617; bh=jUF9UtBQSM62lLW1LP38bR+00eENEmjo+z8gomBkam4=; h=From:To:Subject:Date:From; b=LtXeoXntwuH8yaeK/1+pwQa0Qk52O2kUcj63e15Fl8DM6wPI5IgmzScJzLoWlU6Is oEUaR4IiZm8o1wVAXUM2E7GIhBPITJIDm6gVw9Vx9yfsM4h1AzOcEkUbcG/mKPVN/Z 3SIIQ46L4ck+hlIutHtX7z5hq7fKTU+3I1loK94c= From: Evgeniy Marchenko To: dev@dpdk.org Date: Sun, 01 Nov 2015 15:06:55 +0300 Message-ID: <5105036.dxuAVOzRX1@mentor> User-Agent: KMail/5.0.2 (Linux/4.2.0-17-lowlatency; KDE/5.15.0; x86_64; ; ) MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="utf-8" Subject: [dpdk-dev] mbuf cleanup in i40e/ixgbe X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Nov 2015 12:06:57 -0000 Hello I'm checking mbuf consumption issues in TX path and it looks like i40e and ixgbe drivers consume all mbufs in "full featured" path and free them one-by- one only after TX queue wraps. Upstream drivers are more conservative with memory consumption and free up to 256 SKBs on every napi_poll invocation. And this makes sense because there is indeed not too much work for cleanup and freeing as much memory buffers as possible would lower memory pressure and memory requirements and allow bigger TX bursts without cleanup procedures and better CPU cache utilization. Why cannot we bulk free mbuf in i40e_xmit_cleanup ? Why do we need nb_tx_to_clean calculations? Isn't it always equal to txq->tx_rs_thresh? Here is a proposed patch for i40e PMD to bulk free unused mbufs: ----------------------------------- CUT -------------------------------------- diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c index 8731712..9e3a333 100644 --- a/drivers/net/i40e/i40e_rxtx.c +++ b/drivers/net/i40e/i40e_rxtx.c @@ -883,8 +883,12 @@ i40e_xmit_cleanup(struct i40e_tx_queue *txq) uint16_t nb_tx_desc = txq->nb_tx_desc; uint16_t desc_to_clean_to; uint16_t nb_tx_to_clean; + struct i40e_tx_entry *txe; + int i; - desc_to_clean_to = (uint16_t)(last_desc_cleaned + txq->tx_rs_thresh); + nb_tx_to_clean = txq->tx_rs_thresh; + + desc_to_clean_to = (uint16_t)(last_desc_cleaned + nb_tx_to_clean); if (desc_to_clean_to >= nb_tx_desc) desc_to_clean_to = (uint16_t)(desc_to_clean_to - nb_tx_desc); @@ -898,12 +902,18 @@ i40e_xmit_cleanup(struct i40e_tx_queue *txq) return -1; } - if (last_desc_cleaned > desc_to_clean_to) - nb_tx_to_clean = (uint16_t)((nb_tx_desc - last_desc_cleaned) + - desc_to_clean_to); - else - nb_tx_to_clean = (uint16_t)(desc_to_clean_to - - last_desc_cleaned); + i = last_desc_cleaned; + while (i++ != desc_to_clean_to) { + if (i >= nb_tx_desc) + i -= nb_tx_desc; + + txe = &sw_ring[i]; + RTE_MBUF_PREFETCH_TO_FREE(txe->mbuf); + if (txe->mbuf) { + rte_pktmbuf_free_seg(txe->mbuf); + txe->mbuf = NULL; + } + } txd[desc_to_clean_to].cmd_type_offset_bsz = 0; ----------------------------------- CUT -------------------------------------- What do you think about cleaning not just txq->tx_rs_thresh mbufs but as many as possible? Regards, Evgeniy Marchenko DDoS-Guard.net