From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yk0-f172.google.com (mail-yk0-f172.google.com [209.85.160.172]) by dpdk.org (Postfix) with ESMTP id 686DE532D for ; Tue, 10 Nov 2015 12:50:38 +0100 (CET) Received: by ykba77 with SMTP id a77so13330298ykb.2 for ; Tue, 10 Nov 2015 03:50:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=callistech_com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=59uQJpvJRwJA0eDDm6rF2YJDDMFSyLOJQ2MXvAD2l4g=; b=YwapjajUE2JlclKTD1AMdmXOQcHjAQFaCmCYHcfVLB2BsJ2YcFNLcJxHvvRHR7yc6F 5LcXSnVBB9q1by9bk9o2hGWRIG9wG7oa+hb1z1rZOlFrCGnLgpjrvpwIgDhrO6ilhzsq GoxnCRV619VtORNn9+jcw09dTUmDFR9BRfNvWRDOhnpIxBHE+JH7syhs3ybJb4gZttRt vWQcp1m/SHflWBEsOGaR/ETY6oLT14wddidCBF1RLkk8wBcoBVzqlKqjMNJQrxyEoBUH aZ3ckfZkpXFs389Cx8A3E7avU/ytn3eNKSNRpIk0d6xA2xY+cA2qWKFwDwLpbMq0fY8B C2MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=59uQJpvJRwJA0eDDm6rF2YJDDMFSyLOJQ2MXvAD2l4g=; b=iK1rmjDrqRAk+tCnXsqPcGQrDwQgsagHqcDWLeWrg9KSKndM2M9ZxbIuHYeADUeKqp 4P/g6rWYRS0VxIpc6nnn00aDVoTlS7qTWEkv33M9osoJJpozN7SHC22WGdCodhWvnnPg vOdUGYmLDbEVGFOlQpt7ycDd8efi9/GQ9KQ8Vx2oUtqpTG/jKA8IvGhRtt5FeTrdk82F HKnEvm1Q1Iy31Qqif80WLgF8UUOFGsLJtwG+mc6Uxd9DFqrv+HdBm70hBKeY89ff/50c S2dn8l0sGV2bw1fPb4C1lb4s3f5UDAxtcf8VqVaGSceICjs/RD79/y2UEtW9v1etExSz XeuA== X-Gm-Message-State: ALoCoQmDB+Zdr1aCjPsIDUUl1Z6i8Nn5TGm4ym177fdUonw/t3T2akcwt1POMipO4WK9OxT3xxsp MIME-Version: 1.0 X-Received: by 10.13.234.70 with SMTP id t67mr2705041ywe.321.1447156237861; Tue, 10 Nov 2015 03:50:37 -0800 (PST) Received: by 10.13.204.203 with HTTP; Tue, 10 Nov 2015 03:50:37 -0800 (PST) Received: by 10.13.204.203 with HTTP; Tue, 10 Nov 2015 03:50:37 -0800 (PST) In-Reply-To: <20151110105417.GD29836@bricha3-MOBL3> References: <20151110105417.GD29836@bricha3-MOBL3> Date: Tue, 10 Nov 2015 08:50:37 -0300 Message-ID: From: Ariel Rodriguez To: Bruce Richardson Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Cc: dev@dpdk.org Subject: Re: [dpdk-dev] URGENT please help. Issue on ixgbe_tx_free_bufs version 2.0.0 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Nov 2015 11:50:38 -0000 Thank you very much for your rapid response. 1) IO part is the same as load balancer. The worker part is different. The tx part use qos scheduler framework also. I will try to run the example and see what happends. 2) yes i can. I will do that too. The nic is 82599ES 10-Gigabit SFI/SFP+ with tapped traffic (is a hardware bypass device silicom vendor). I develop a similar app without the tx part. It just received a copy of the traffic (around 6gbps and 400000 concurrent flows) and then free the mbufs. It works like a charm. Is strange this issue ... If i disabled the qos scheduler code and the tx code dropping all packets instead of rte_eth_tx_burst ( is like disabling tx core) the issue is happening in rte_eth_rx_burst returning corrupted mbuf (rx core) Could the nic behave anormally? I will try the 2 things you comment before. Regards . Ariel Horacio Rodriguez On Tue, Nov 10, 2015 at 01:35:21AM -0300, Ariel Rodriguez wrote: > Dear dpdk experts. > > Im having a recurrent segmentation fault under the > function ixgbe_tx_free_bufs (ixgbe_rxtx.c:150) (i enable -g3 -O0). > > Surfing the core dump i find out this: > > txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh - 1)]); > > txq->tx_next_dd = 31 > txq->txq->tx_rs_thresh=32 > > Obviosly txep points out to the first element but > > *(txep).mbuf == INVALID MBUF ADDRESS > > The same applies to > > *(txep+1).mbuf ; *(txep +2).mbuf;*(txep+3).mbuf > > from *(txep+4) .mbuf to *(txep+31).mbuf seems to be valid because im able > to derefence the mbuf's > > > Note: > > I disable CONFIG_RTE_IXGBE_INC_VECTOR because i gets similiar behavior , I > thought the problem would disappear disabling that feature. > > > the program always runs well up to 4 or 5 hours and then crash ... always > in the same line. > > this is the backtrace of the program: > > #0 0x0000000000677a64 in rte_atomic16_read (v=0x47dc14c18b14) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/generic/rte_atomic.h:151 > #1 0x0000000000677c1d in rte_mbuf_refcnt_read (m=0x47dc14c18b00) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:411 > #2 0x000000000067a13c in __rte_pktmbuf_prefree_seg (m=0x47dc14c18b00) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:778 > #3 rte_pktmbuf_free_seg (m=0x47dc14c18b00) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_mbuf.h:810 > #4 ixgbe_tx_free_bufs (txq=0x7ffb40ae52c0) at > /opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:150 > #5 tx_xmit_pkts (tx_queue=0x7ffb40ae52c0, tx_pkts=0x64534770 , > nb_pkts=32) at /opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:256 > #6 0x000000000067c6f3 in ixgbe_xmit_pkts_simple (tx_queue=0x7ffb40ae52c0, > tx_pkts=0x64534570 , nb_pkts=80) at > /opt/dpdk-2.0.0/lib/librte_pmd_ixgbe/ixgbe_rxtx.c:343 > #7 0x00000000004ec93d in rte_eth_tx_burst (port_id=1 '\001', queue_id=0, > tx_pkts=0x64534570 , nb_pkts=144) at > /opt/dpdk-2.0.0/x86_64-native-linuxapp-gcc/include/rte_ethdev.h:2572 > Hi, I'd like a bit more information to help debug your problem: * what application are you running when you see this crash? If it's an app of your own making, can you reproduce the crash using one of the standard DPDK apps, or example apps, e.g. testpmd, l2fwd, etc. * Can you also try to verify if the crash occurs with the latest DPDK code available in git from dpdk.org? Regards, /Bruce