From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 05D4645CDD for ; Sun, 10 Nov 2024 18:12:51 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id B20BE40264; Sun, 10 Nov 2024 18:12:50 +0100 (CET) Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by mails.dpdk.org (Postfix) with ESMTP id B1AB6400EF for ; Sun, 10 Nov 2024 18:12:48 +0100 (CET) Received: by mail-pg1-f173.google.com with SMTP id 41be03b00d2f7-7eab7622b61so2608025a12.1 for ; Sun, 10 Nov 2024 09:12:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1731258768; x=1731863568; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=6qpvQWjNUDfqoj4J7LPC09RHLHSpAbrIvjq4dQxWPww=; b=liWW6V0sCT6ZxIHmSDcLvjgAYAZ/p3pBcTWLJJzQci5KcdTo81PYPDOwNoTVJ9P7+f zHQUakQ3omVB3LVQusA+zuWufLWJ1GpsmTOMfMxOGABRdDUFHiYlP7xi5dC/cqoqEvfG zYe2rKwuT4Td3VWuZ79H53Mb5jrTAEmixWhooZm9Oc97orT+ycDCrPBDgvC7Al8nKCEA Ik8GxC1lbKS7/uWzRk6On9OAV7MK85mFWkOppilPsSbt/bQfR7mxnEi4bdPf0jxGBITx wrSVXvsk/bPwi5aKUdJki9nceYb2n6kDtzHbhhnFXQbwM9g7O/mOIHNRpbkAlOqO+EUo iCDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731258768; x=1731863568; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6qpvQWjNUDfqoj4J7LPC09RHLHSpAbrIvjq4dQxWPww=; b=NJgDuVvq2dIq873UHpf7AFlOLa3hWDeOpPYXjl/hIVUOIpQ8V9nxLQ6X9v0mPbH/EI l2zSsFOkNU0CJ78+A1Mux/ZSxTd2Kxm4lWLGbAX424pe6KQS4a8coZf1p7S/Meq9qSY/ zgcbWAzIG4e7bPKyZipVnA1+ajoGV5y19rlpzIkqDOI3NX0F8u+S7rR5VrQg6/7oWN/V 2Pn2Dndf40CBrtPp4GLMo1bqAFrvBhXPCRQJkHL1C1zJDl6T+X2PiDNn1griLb0eHeNb 75ZI4Zxp4Lmbefud1UdEBw0redErQXuYxSzu2owCShPDT8jGnGY7USJEGqYJKySOEOKc AiWA== X-Gm-Message-State: AOJu0YwPb98CpXNGwd0djCqfvcA1sZN3UIQT0lI29zihL1OAVVhxjgRI 659ciNr2skYeUsEgbl+dOs4xDtK6DWqv+CeB10irpBzut60T6quIZQJRDlS0Jpg= X-Google-Smtp-Source: AGHT+IEBuHrYUJr62U6nPNng+8jAFMT3mUFX0UwgDx1MJFKhJ19k9XyTUSwKj57Z2+3SOsJlgE8Ctg== X-Received: by 2002:a17:90b:1a8c:b0:2e2:cef9:4d98 with SMTP id 98e67ed59e1d1-2e9b177bad9mr14066780a91.25.1731258767630; Sun, 10 Nov 2024 09:12:47 -0800 (PST) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2e9a5fd154asm6969973a91.40.2024.11.10.09.12.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 10 Nov 2024 09:12:47 -0800 (PST) Date: Sun, 10 Nov 2024 09:12:45 -0800 From: Stephen Hemminger To: Alan Beadle Cc: users@dpdk.org Subject: Re: mbufs getting reused despite nonzero refcnt Message-ID: <20241110091245.677a3983@hermes.local> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org On Sun, 10 Nov 2024 11:23:29 -0500 Alan Beadle wrote: > Hi everyone, > > I am using DPDK to send two-way traffic between a pair of machines. My > application has local readers, remote acknowledgments, as well as > automatic retries when a packet is lost. For these reasons I am using > rte_mbuf_refcnt_update() to prevent the NIC from freeing the packet > and recycling the mbuf before my local readers are done and the remote > reader has acknowledged the message. I was advised to do this in an > earlier thread on this mailing list. > > However, this does not seem to be working. After running my app for > awhile and exchanging about 1000 messages in this way, my queue of > unacknowledged mbufs is getting corrupted. The mbufs attached to my > queue seem to contain data for newer messages than what is supposed to > be in them, and in some cases contains a totally different type of > packet (an acknack for instance). Obviously this results in retries of > those messages failing to send the correct data and my application > gets stuck. > > I have ensured that the refcount is not reaching 0. Each new mbuf > immediately has the refcnt incremented by 1. I was concerned that > retries might need the refcnt bumped again, but if I bump the refcount > every time I resend a specific mbuf to the NIC, the refcounts just > keep getting higher. So it looks like re-bumping it on a resend is not > necessary. > > I have ruled out other possible explanations. The mbufs are being > reused by rte_pktmbuf_alloc. I even tried playing with the EAL > settings related to the number of mbuf descriptors and saw my changes > directly correlate with how long it takes this problem to occur. How > do I really prevent the driver from reusing packets that I still might > need to resend? > > Thanks in advance, > -Alan Which driver, could be a driver bug. Also, you should be able to trace mbuf functions, either with rte_trace or by other facility.