From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.druidsoftware.com (unknown [148.251.52.124]) by dpdk.org (Postfix) with ESMTP id 7D4929AAC for ; Tue, 24 Feb 2015 14:00:21 +0100 (CET) Received: from localhost (unknown [127.0.0.1]) by mail.druidsoftware.com (Postfix) with ESMTP id EF93317BB947 for ; Tue, 24 Feb 2015 13:00:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at www.druidsoftware.com Received: from mail.druidsoftware.com ([127.0.0.1]) by localhost (www.druidsoftware.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W3IoeH0xsJ1i for ; Tue, 24 Feb 2015 13:00:15 +0000 (GMT) Received: from [172.28.2.37] (unknown [95.45.250.213]) (Authenticated sender: odeme@druidsoftware.com) by mail.druidsoftware.com (Postfix) with ESMTPSA id 6E1A417BB93B for ; Tue, 24 Feb 2015 13:00:15 +0000 (GMT) Message-ID: <54EC75CF.20102@druidsoftware.com> Date: Tue, 24 Feb 2015 12:59:59 +0000 From: Olivier Deme User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: dev@dpdk.org Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Subject: [dpdk-dev] rte_kni_rx_burst issues X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 13:00:21 -0000 All, I know that an issue was already raised with regards to the efficiency of rte_kni_rx_burst but I think that there is more to it than previously discussed. As previously pointed out rte_kni_rx_burst invokes kni_allocate_mbufs every single time. In turn, it looks like kni_allocate_mbufs allocates 32 mbufs (MAX_MBUF_BURST_NUM) and attempt to enqueue these to the alloc_q. If alloc_q is full (1024 buffers: KNI_FIFO_COUNT_MAX), kni_allocate_mbufs frees all buffers that couldn't be enqueued. Further to the fact that this is very inefficient, it looks like invoking rte_kni_rx_burst in a loop is guaranteed to fill the alloc_q to its maximum capacity (1024) unless packets are read from the kernel faster than they are enqueued by rte_kni_rx_burst. In my application, I hit the "Out of memory" error in kni_allocate_mbufs almost straight away because there is very little egress traffic from the kernel and my memory pool wasn't big enough to cater for the kni thread and other dpdk queues. I would think the kni_allocate_mbufs should take a "buffer_count" parameter which is the number of desired mbufs to allocate and add to the alloc_q. With this, rte_kni_rx_burst would be able to request allocating as many mbufs that are dequeued from the tx_q, so that the total number of buffers across alloc_q and tx_q remains constant. I also noticed that none of these functions are declared inline. This is not great as the thread that forwards packets between a NIC and the kernel may also be the same thread that forwards packets between 2 NICs. As such it would be better to avoid too many function calls to forward packets between NICs and kernel. Kind Regards, Olivier.