From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by dpdk.org (Postfix) with ESMTP id 204CCC3FC for ; Mon, 20 Jul 2015 11:47:24 +0200 (CEST) Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP; 20 Jul 2015 02:47:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.15,507,1432623600"; d="scan'208";a="527054126" Received: from irsmsx109.ger.corp.intel.com ([163.33.3.23]) by FMSMGA003.fm.intel.com with ESMTP; 20 Jul 2015 02:47:23 -0700 Received: from irsmsx103.ger.corp.intel.com ([169.254.3.216]) by IRSMSX109.ger.corp.intel.com ([169.254.13.200]) with mapi id 14.03.0224.002; Mon, 20 Jul 2015 10:47:19 +0100 From: "Richardson, Bruce" To: "Ananyev, Konstantin" , "dev@dpdk.org" Thread-Topic: [dpdk-dev] [PATCH 2/2] ixgbe: check mbuf refcnt when clearing RX/TX ring Thread-Index: AQHQws+gUH5ZaQvoFUyIp6x5SWDUkJ3kG9iA Date: Mon, 20 Jul 2015 09:47:19 +0000 Message-ID: <59AF69C657FD0841A61C55336867B5B03457C0DA@IRSMSX103.ger.corp.intel.com> References: <1435938006-22254-1-git-send-email-bruce.richardson@intel.com> <1435938006-22254-3-git-send-email-bruce.richardson@intel.com> <2601191342CEEE43887BDE71AB97725836A26FB2@irsmsx105.ger.corp.intel.com> In-Reply-To: <2601191342CEEE43887BDE71AB97725836A26FB2@irsmsx105.ger.corp.intel.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [163.33.239.182] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Subject: Re: [dpdk-dev] [PATCH 2/2] ixgbe: check mbuf refcnt when clearing RX/TX ring X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jul 2015 09:47:25 -0000 > -----Original Message----- > From: Ananyev, Konstantin > Sent: Monday, July 20, 2015 10:37 AM > To: Richardson, Bruce; dev@dpdk.org > Subject: RE: [dpdk-dev] [PATCH 2/2] ixgbe: check mbuf refcnt when clearin= g > RX/TX ring >=20 > Hi Bruce, >=20 > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Bruce Richardson > > Sent: Friday, July 03, 2015 4:40 PM > > To: dev@dpdk.org > > Subject: [dpdk-dev] [PATCH 2/2] ixgbe: check mbuf refcnt when clearing > > RX/TX ring > > > > The function to clear the TX ring when a port was being closed, e.g. > > on exit in testpmd, was not checking the mbuf refcnt before freeing it. > > Since the function in the vector driver to clear the ring after TX > > does not set the pointer to NULL post-free, this caused crashes if > > mbuf debugging was turned on. > > > > To reproduce the issue, ensure the follow config variables are set: > > RTE_IXGBE_INC_VECTOR > > RTE_LIBRTE_MBUF_DEBUG > > Then compile up and run testpmd using 10G ports with the vector driver. > > Start traffic and let some flow through, then type "stop" and "quit" > > at the testpmd prompt, and crash will occur. Output below: > > > > testpmd> quit > > Stopping port 0...done > > Stopping port 1...PANIC in rte_mbuf_sanity_check(): > > bad ref cnt > > [New Thread 0x7fffabfff700 (LWP 145312)] > > [New Thread 0x7fffb47fe700 (LWP 145311)] > > [New Thread 0x7fffb4fff700 (LWP 145310)] > > [New Thread 0x7ffff6cd5700 (LWP 145309)] > > 18: [/home/bruce/dpdk.org/x86_64-native-linuxapp- > gcc/app/testpmd(_start+0x29) > > <....snip for brevity...> > > Program received signal SIGABRT, Aborted. > > 0x00007ffff7120a98 in raise () from /lib64/libc.so.6 > > > > A similar error occurs when clearing the RX ring, which is also fixed > > by this patch. > > > > Signed-off-by: Bruce Richardson > > --- > > drivers/net/ixgbe/ixgbe_rxtx.c | 3 ++- > > drivers/net/ixgbe/ixgbe_rxtx_vec.c | 8 +++++++- > > 2 files changed, 9 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c > > b/drivers/net/ixgbe/ixgbe_rxtx.c index 41a062e..12e25b7 100644 > > --- a/drivers/net/ixgbe/ixgbe_rxtx.c > > +++ b/drivers/net/ixgbe/ixgbe_rxtx.c > > @@ -2108,7 +2108,8 @@ ixgbe_rx_queue_release_mbufs(struct > > ixgbe_rx_queue *rxq) > > > > if (rxq->sw_ring !=3D NULL) { > > for (i =3D 0; i < rxq->nb_rx_desc; i++) { > > - if (rxq->sw_ring[i].mbuf !=3D NULL) { > > + if (rxq->sw_ring[i].mbuf !=3D NULL && > > + rte_mbuf_refcnt_read(rxq->sw_ring[i].mbuf)) > { > > rte_pktmbuf_free_seg(rxq->sw_ring[i].mbuf); > > rxq->sw_ring[i].mbuf =3D NULL; > > } >=20 >=20 > Sorry for late review, but I am afraid your changes don't fix the problem= . > After sw_ring[].mbuf was freed by RX path, entity that manages that RX > queue shouldn't touch that mbuf. > (unless it was allocated by the same RX queue again). > As same mbuf could be already allocated by something else. > As an example by another RX/TX queue and is in active use. > Same story for TX below. Good point, I'd forgotten about that scenario. >=20 > As I can see the proper fix could be one of 2: > 1. Make RX/TX vector functions to reset sw_ring[].mbuf to 0. > 2. At queue_release_mbufs(), don't go through all sw_ring[] entries, but > only though ones which might contain valid mbufs. > For RX: entries between rx_tail and rxrearm_start only (which implies a > special queue_release_mbufs() for vector rx). > For TX: from tx_next_dd - (tx_rs_thresh - 1) and no more then nb_tx_desc = - > nb_tx_free Option 2 seems a better choice for a fix. I'll look at it when I get a chan= ce. /Bruce >=20 > Konstantin >=20 > > diff --git a/drivers/net/ixgbe/ixgbe_rxtx_vec.c > > b/drivers/net/ixgbe/ixgbe_rxtx_vec.c > > index 0edac82..7e633d3 100644 > > --- a/drivers/net/ixgbe/ixgbe_rxtx_vec.c > > +++ b/drivers/net/ixgbe/ixgbe_rxtx_vec.c > > @@ -665,7 +665,13 @@ ixgbe_tx_queue_release_mbufs(struct ixgbe_tx_queue > *txq) > > nb_free < max_desc && i !=3D txq->tx_tail; > > i =3D (i + 1) & max_desc) { > > txe =3D (struct ixgbe_tx_entry_v *)&txq->sw_ring[i]; > > - if (txe->mbuf !=3D NULL) > > + /* > > + *check for already freed packets. > > + * Note: ixgbe_tx_free_bufs does not NULL after free, > > + * so we actually have to check the reference count. > > + */ > > + if (txe->mbuf !=3D NULL && > > + rte_mbuf_refcnt_read(txe->mbuf) !=3D 0) > > rte_pktmbuf_free_seg(txe->mbuf); > > } > > /* reset tx_entry */ > > -- > > 2.4.3