From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 589DE2C06 for ; Mon, 24 Oct 2016 05:33:21 +0200 (CEST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga103.fm.intel.com with ESMTP; 23 Oct 2016 20:33:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.31,392,1473145200"; d="scan'208";a="1058075559" Received: from fmsmsx105.amr.corp.intel.com ([10.18.124.203]) by fmsmga001.fm.intel.com with ESMTP; 23 Oct 2016 20:33:20 -0700 Received: from fmsmsx158.amr.corp.intel.com (10.18.116.75) by FMSMSX105.amr.corp.intel.com (10.18.124.203) with Microsoft SMTP Server (TLS) id 14.3.248.2; Sun, 23 Oct 2016 20:33:19 -0700 Received: from shsmsx101.ccr.corp.intel.com (10.239.4.153) by fmsmsx158.amr.corp.intel.com (10.18.116.75) with Microsoft SMTP Server (TLS) id 14.3.248.2; Sun, 23 Oct 2016 20:33:19 -0700 Received: from shsmsx103.ccr.corp.intel.com ([169.254.4.139]) by SHSMSX101.ccr.corp.intel.com ([169.254.1.104]) with mapi id 14.03.0248.002; Mon, 24 Oct 2016 11:33:17 +0800 From: "Zhang, Helin" To: yingzhi CC: "dev@dpdk.org" Thread-Topic: [dpdk-dev] rte_kni_tx_burst() hangs because of no free descriptors Thread-Index: AQHSLTGcc2dJoX9z7UC/eekbL29vOqC28w9w Date: Mon, 24 Oct 2016 03:33:16 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.239.127.40] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] rte_kni_tx_burst() hangs because of no free descriptors X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Oct 2016 03:33:22 -0000 Hi Yingzhi Thank you for the reporting! The description is not so clear at least for m= e. Please help to narrown down the issue by youself. How many packets would it have for calling TX function? Why it would return 0 after calling TX function? No memory? Or return from = else? Have you found anything? Regards, Helin > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of yingzhi > Sent: Sunday, October 23, 2016 9:30 PM > To: users; dev@dpdk.org > Subject: [dpdk-dev] rte_kni_tx_burst() hangs because of no free descripto= rs >=20 > - > Hi Experts, >=20 > Background: >=20 > We are using DPDK to develop a LoadBalancer following below logic: When > a new packet is received: > 1. if the dst_addr is management IP, forward to KNI. 2. if the dst_addr = is in > VIP list, select backend and forward(modify dst mac address). 3. otherwis= e > drop the packet. >=20 > At this stage, we use one single thread for KNI forwarding and another fo= r > VIP forwarding(forward to eth). >=20 > DPDK version: 16.07 > NIC: 82599ES 10-Gigabit SFI/SFP+ Network Connection > Linux: 14.04.1-Ubuntu x64 >=20 > Promblem description: >=20 > The program runs correctly for sometime(around 2 hours for 400Mb traffic)= . > But it it will hang. When problem happens, rte_eth_tx_burst() will not ab= le to > send out any packets(always returns 0). We tracked into that function and > noticed it is actually calling ixgbe driver's ixgbe_xmit_pkts_vec() funct= ion in > our environment, because we use default tx queue configuration, after > printing some info, we found if the free function works fine: > tx_rs_thresh: 32, tx_free_thresh: 32, nb_tx_free: 31 >=20 > it will trigger free and make 32 more free descriptors: > tx_rs_thresh: 32, tx_free_thresh: 32, nb_tx_free: 62 >=20 > but when something going wrong, it will no longer free anything: > tx_rs_thresh: 32, tx_free_thresh: 32, nb_tx_free: 0 tx_rs_thresh: 32, > tx_free_thresh: 32, nb_tx_free: 0 >=20 > It may related with the DD flag of the descriptor but we are not quite su= re. >=20 > Our program logic: >=20 > create two mbuf pools on socket 0, one for rx_queue and one for kni. (all > lcore threads runs on socket0) >=20 > init kni interface with rte_kni_alloc() >=20 >=20 > init one NIC interface with > rte_eth_dev_configure(); rte_eth_rx_queue_setup(); > rte_eth_tx_queue_setup(); rte_eth_dev_start(); >=20 >=20 >=20 > in the eth main loop: (code is simplified) > while(1) { n =3D rte_eth_rx_burst(packets); for (i =3D 0; i < n;= ++i) { if > (SEND_TO_KNI) { m =3D rte_kni_tx_burst(packets[i]); = if (m !=3D 1)) > { rte_pktmbuf_free(packets[i]); } } = if (SEND_TO_ETH) > { // after modify the packet m =3D rte_eth_tx_bur= st(packets[i]); > if (m !=3D 1)) { rte_pktmbuf_free(packets[i]); = } } // > otherwise drop the packet rte_pktmbuf_free(packets[i]); } } >=20 >=20 > Please advise if I'm using DPDK in a wrong way. Sorry if I missed somethi= ng > basic, I'm new to DPDK. >=20 > Thanks in advance > Best regards