From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id D4ACF98 for ; Thu, 9 Aug 2018 11:31:22 +0200 (CEST) X-Amp-Result: UNSCANNABLE X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 09 Aug 2018 02:31:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,214,1531810800"; d="scan'208";a="75108486" Received: from bricha3-mobl.ger.corp.intel.com ([10.237.221.107]) by fmsmga002.fm.intel.com with SMTP; 09 Aug 2018 02:31:19 -0700 Received: by (sSMTP sendmail emulation); Thu, 09 Aug 2018 10:31:16 +0100 Date: Thu, 9 Aug 2018 10:31:16 +0100 From: Bruce Richardson To: Hui Liu Cc: dev@dpdk.org Message-ID: <20180809093115.GA23324@bricha3-MOBL.ger.corp.intel.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Organization: Intel Research and Development Ireland Ltd. User-Agent: Mutt/1.10.1 (2018-07-13) Subject: Re: [dpdk-dev] Questions about TX descriptors run out occasionally X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2018 09:31:23 -0000 On Thu, Jul 26, 2018 at 07:13:50PM -0700, Hui Liu wrote: > Hi Experts, > > I'm developing my own dpdk-based application via Intel 82599ES port. My > Application is doing a job to send ICMP requests (packet size varies from > 64 bytes to 1472 bytes, 200,000 pps, 1.1Gbps) and receive responses, with > ARP request/response and ICMP response handling when necessary. It was > working pretty fine in 5 hours to 10 days randomly and then TX descriptors > run out and cannot be freed by ixgbe_tx_free_bufs() due to DD bit is not > set: > > /* check DD bit on threshold descriptor */ > status = txq->tx_ring[txq->tx_next_dd].wb.status; > if (!(status & IXGBE_ADVTXD_STAT_DD)) > return 0; > > My tx queue setup is: > tx_conf->tx_thresh.pthresh = 64; > tx_conf->tx_thresh.hthresh = 0; > tx_conf->tx_thresh.wthresh = 0; > tx_conf->tx_free_thresh = 256; > tx_conf->tx_rs_thresh = 32; > tx_conf->txq_flags = ETH_TXQ_FLAGS_NOMULTSEGS | > ETH_TXQ_FLAGS_NOOFFLOADS; > > > I tried to read code to see if there is any case to take these descriptors > and never set IXGBE_ADVTXD_STAT_DD back but no luck yet. And I have not > even found the related code when IXGBE_ADVTXD_STAT_DD is set/unset when > descriptor is taken/released other than reset queues... So may I ask: > 1. where do we set/unset IXGBE_ADVTXD_STAT_DD when descriptor is > taken/released? For RX and TX, the DD bit is never set by software, only by hardware. When writing a descriptor to memory for the NIC to read, the DD bit is cleared. Software knows the NIC has finished with that descriptor by checking for the DD bit being set by the NIC. If the DD bit is not being set, then the problem is likely on the NIC side. [Potential software issues that could cause this might be buffer e.g. overflows where we overwrite a DD bit set by the NIC, or where we are polling an incorrect address, etc. etc.] Regards, /Bruce