From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B91A445D07 for ; Thu, 14 Nov 2024 19:41:16 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9F6614025D; Thu, 14 Nov 2024 19:41:16 +0100 (CET) Received: from mail-4325.protonmail.ch (mail-4325.protonmail.ch [185.70.43.25]) by mails.dpdk.org (Postfix) with ESMTP id 3D4494021F for ; Thu, 14 Nov 2024 19:41:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1731609674; x=1731868874; bh=4ATETrz6O3V1JjRz7X8kCyxXkuEV2Z9cGTojt09VWeg=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector: List-Unsubscribe:List-Unsubscribe-Post; b=mXK0NQTJ/mt5J3gM8nmOy5ngayJ4LrkFd3C6ZcB3REjvPB67FN/nxE/dn3JVtqYHv tSh5rMHsc6yj+vN2OyNPklTp0Esri4cHg3UI0SLk+N4SGvONHE7gAMsXSOOrIceO9t IVfQAIsG17X7sktAY4lg2B7yN6SU7vA96/SuhtHKFnPNM8hTWnXIO8n9fSxR8mOicP ghFztrcCBhG73YpL/opdwpBseJNpquTtM+mmDRBKbhNfAAHncpL42OI2QO71dhn9ei ZwCMhqxAKlXvcfHwx2AgJCjcNZAasL2nhEptc6ECW2sXE+G8UeT3SHOMripTHaR4il RBc2uXy0cqyMg== Date: Thu, 14 Nov 2024 18:41:09 +0000 To: "users@dpdk.org" From: Fabio Fernandes Subject: Re: mbufs getting reused despite nonzero refcnt (Alan Beadle) Message-ID: Feedback-ID: 46522295:user:proton X-Pm-Message-ID: 216596b1e99e58b225359891ded361f705390b4b MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi Alan, Are you using RTE_ETH_TX_OFFLOAD_MBUF_FAST_FREE? Drivers are not required to handle mbuf refcount when this flag is enabled. Regards, Fabio Sent from Proton Mail Android -------- Original Message -------- On 13/11/2024 08:00, wrote: > Send users mailing list submissions to > =09users@dpdk.org > =20 > To subscribe or unsubscribe via the World Wide Web, visit > =09https://mails.dpdk.org/listinfo/users > or, via email, send a message with subject or body 'help' to > =09users-request@dpdk.org > =20 > You can reach the person managing the list at > =09users-owner@dpdk.org > =20 > When replying, please edit your Subject line so it is more specific > than "Re: Contents of users digest..." > =20 > =20 > Today's Topics: > =20 > 1. Re: mbufs getting reused despite nonzero refcnt (Alan Beadle) > =20 > =20 > ---------------------------------------------------------------------- > =20 > Message: 1 > Date: Tue, 12 Nov 2024 08:02:06 -0500 > From: Alan Beadle > To: Stephen Hemminger > Cc: users@dpdk.org > Subject: Re: mbufs getting reused despite nonzero refcnt > Message-ID: > =09 > Content-Type: text/plain; charset=3D"UTF-8" > =20 > Is there anything in the usage I described in my previous email which > might explain this problem? Is there anything else wrong with what I'm > doing driver-wise? > =20 > On Sun, Nov 10, 2024 at 12:31?PM Alan Beadle wrote= : > > > > I'm using the vfio-pci module with Intel X550-T2 NICs. I believe this > > means it will use the ixgbe driver? To be honest, I am a bit confused > > about the use of drivers in DPDK. I am using the first setup that I > > got to work and send/receive packets. Additional tips would be greatly > > appreciated. After loading the vfio-pci module I run dpdk-devbind.py > > --bind vfio-pci 65:00.1 and then I just use the standard DPDK API > > calls in my app. I was meaning to revisit this once my app was more > > complete. > > > > On Sun, Nov 10, 2024 at 12:12?PM Stephen Hemminger > > wrote: > > > > > > On Sun, 10 Nov 2024 11:23:29 -0500 > > > Alan Beadle wrote: > > > > > > > Hi everyone, > > > > > > > > I am using DPDK to send two-way traffic between a pair of machines= . My > > > > application has local readers, remote acknowledgments, as well as > > > > automatic retries when a packet is lost. For these reasons I am us= ing > > > > rte_mbuf_refcnt_update() to prevent the NIC from freeing the packe= t > > > > and recycling the mbuf before my local readers are done and the re= mote > > > > reader has acknowledged the message. I was advised to do this in a= n > > > > earlier thread on this mailing list. > > > > > > > > However, this does not seem to be working. After running my app fo= r > > > > awhile and exchanging about 1000 messages in this way, my queue of > > > > unacknowledged mbufs is getting corrupted. The mbufs attached to m= y > > > > queue seem to contain data for newer messages than what is suppose= d to > > > > be in them, and in some cases contains a totally different type of > > > > packet (an acknack for instance). Obviously this results in retrie= s of > > > > those messages failing to send the correct data and my application > > > > gets stuck. > > > > > > > > I have ensured that the refcount is not reaching 0. Each new mbuf > > > > immediately has the refcnt incremented by 1. I was concerned that > > > > retries might need the refcnt bumped again, but if I bump the refc= ount > > > > every time I resend a specific mbuf to the NIC, the refcounts just > > > > keep getting higher. So it looks like re-bumping it on a resend is= not > > > > necessary. > > > > > > > > I have ruled out other possible explanations. The mbufs are being > > > > reused by rte_pktmbuf_alloc. I even tried playing with the EAL > > > > settings related to the number of mbuf descriptors and saw my chan= ges > > > > directly correlate with how long it takes this problem to occur. H= ow > > > > do I really prevent the driver from reusing packets that I still m= ight > > > > need to resend? > > > > > > > > Thanks in advance, > > > > -Alan > > > > > > Which driver, could be a driver bug. > > > > > > Also, you should be able to trace mbuf functions, either with rte_tr= ace > > > or by other facility. > =20 > =20 > End of users Digest, Vol 466, Issue 2 > ************************************* >