From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 3357B45CDA for ; Sun, 10 Nov 2024 17:23:45 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0EEA740264; Sun, 10 Nov 2024 17:23:43 +0100 (CET) Received: from mail-oa1-f51.google.com (mail-oa1-f51.google.com [209.85.160.51]) by mails.dpdk.org (Postfix) with ESMTP id 01163400EF for ; Sun, 10 Nov 2024 17:23:40 +0100 (CET) Received: by mail-oa1-f51.google.com with SMTP id 586e51a60fabf-288d74b3a91so2468643fac.0 for ; Sun, 10 Nov 2024 08:23:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731255820; x=1731860620; darn=dpdk.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=uBQY1Y2v0JVNBKkk6OqWq4ZYS6YVDyqnl6d14PgqsQg=; b=cBS+FVGk6Ly2C5P7v12WTb5a4BcOIE7MSLd6TNSawbWmjtQGn9xDe1QSMhbIAVBlxD JraQwyqO9BRn9u1wCeJ5TPBQ2pHd41gAfMnZ5dPtU4Ei+K4Q4dInpGHW1RXhv1WG8hyj 8oygaktbd78u0mNtMEF8UwTUmJwZU7xxFbd0uy+YK2NioNiAZWQd7YB6NdUkzvAoroAW rmQlDoXGDiI+N5hng+e9fHsjsGW3PNdBL2fRCL4HA03soPNfFWTnPcDXEAr/sugiFm3p ckAoJIq42H7c7taq54dEGuAugxrkhPPWqPftMfm5AHhNWM0jx6T/diVtxtmzCDLyQkyV waBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731255820; x=1731860620; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=uBQY1Y2v0JVNBKkk6OqWq4ZYS6YVDyqnl6d14PgqsQg=; b=nUMMLDLHiCHwPNj6nD5panURO+TFXBm7aJrnA6qWDkQaknT9EUiJscBV9b5oGu/gU1 TzLDjzfXfMcRteUJsR3EsYrM9CabO+A8FF3OInKldtGfHP0Fo4aUD9PSpKxKepyWa2N6 qk1JxvRaFjSUjBEFxB7kS+QAe82dqM9ZvMCRSGG/w8eXKHFAVPiD7DnGTkrw6dAlfx2t uA3PFCGHTtUZl4nCHVnq1WYt5j84Kh4hwDtVzrT2aUn5bdpQzXYm9968RF1USYC8wGSF O3zoCOkZk+zPx4mYwWr462c5OgXNaNm4ZU6wCqMPhQt4TXO8IqPOVh674fiWq9P8tidw E7vQ== X-Gm-Message-State: AOJu0YxZtLF3ao1WgubqbuWBQloLC2OYn+cqXU5qnc6lPNcl9uLz5+Cr hAVFKxxScdwkmw5Fj9iyfmbmdJvItTtTz6HL+VpN/WMHzk76yE0mKPwDv7GeMX3n3M4SDg8az7F xdVmMmHA9Z3yC8x5x/qMSNh4qJsM3RQ== X-Google-Smtp-Source: AGHT+IHUIBDNrqYXYNtLQFa8j1BN35ZFniXNFS0zXNWNdWJJ6SJqP4Ly+6Tm4NFzZ19yzzK7bOAUyogPNI04xV10vF0= X-Received: by 2002:a05:6870:e0c9:b0:27c:a414:b907 with SMTP id 586e51a60fabf-29560260457mr7717697fac.33.1731255819977; Sun, 10 Nov 2024 08:23:39 -0800 (PST) MIME-Version: 1.0 From: Alan Beadle Date: Sun, 10 Nov 2024 11:23:29 -0500 Message-ID: Subject: mbufs getting reused despite nonzero refcnt To: users@dpdk.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org Hi everyone, I am using DPDK to send two-way traffic between a pair of machines. My application has local readers, remote acknowledgments, as well as automatic retries when a packet is lost. For these reasons I am using rte_mbuf_refcnt_update() to prevent the NIC from freeing the packet and recycling the mbuf before my local readers are done and the remote reader has acknowledged the message. I was advised to do this in an earlier thread on this mailing list. However, this does not seem to be working. After running my app for awhile and exchanging about 1000 messages in this way, my queue of unacknowledged mbufs is getting corrupted. The mbufs attached to my queue seem to contain data for newer messages than what is supposed to be in them, and in some cases contains a totally different type of packet (an acknack for instance). Obviously this results in retries of those messages failing to send the correct data and my application gets stuck. I have ensured that the refcount is not reaching 0. Each new mbuf immediately has the refcnt incremented by 1. I was concerned that retries might need the refcnt bumped again, but if I bump the refcount every time I resend a specific mbuf to the NIC, the refcounts just keep getting higher. So it looks like re-bumping it on a resend is not necessary. I have ruled out other possible explanations. The mbufs are being reused by rte_pktmbuf_alloc. I even tried playing with the EAL settings related to the number of mbuf descriptors and saw my changes directly correlate with how long it takes this problem to occur. How do I really prevent the driver from reusing packets that I still might need to resend? Thanks in advance, -Alan