From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <users-bounces@dpdk.org> Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 38EA446192 for <public@inbox.dpdk.org>; Wed, 5 Feb 2025 00:26:40 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id BF67A4021E; Wed, 5 Feb 2025 00:26:39 +0100 (CET) Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by mails.dpdk.org (Postfix) with ESMTP id 681D1400EF for <users@dpdk.org>; Wed, 5 Feb 2025 00:26:38 +0100 (CET) Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-21644aca3a0so144286975ad.3 for <users@dpdk.org>; Tue, 04 Feb 2025 15:26:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=networkplumber-org.20230601.gappssmtp.com; s=20230601; t=1738711597; x=1739316397; darn=dpdk.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=ET65lsVuTs5B2YY5GKi+goIS2SHd7Y5y3o7RgDk3k10=; b=VmK9Ozy9lEYOcpDPD/bNuStkwoXCiUZb4gBvfdiKkawmGglda1d0IOniBYweVHK3oS fWaJPJwronRcmrJoW5EcFG4+10eCe1PQ7HbnlrQjsLt47iGFV4Us4QfShS6oGRJzNj7g nXsCzpAoEyt+trQwSfjPYlUGl2/Uww225jCeTnJVHbIPUyWl5qaoDsWctGJ3Pics3fMx irpV8jUYms28/ABJqFsM9U9+mlosdZ2UYd1jqaVzu095iS1yh0EWML1YLUd4shwHwEAQ x9jHjjCduRFYpjTskBoMdqMgI4fXfDg383UoDODWGoqRR4WGLaPuFaiQBSM5Gv+PeMKj UzFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738711597; x=1739316397; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ET65lsVuTs5B2YY5GKi+goIS2SHd7Y5y3o7RgDk3k10=; b=mdp+P5cBXD976dTmdaMAuLPmDVWGwAuh9f/bC8E2cgIU4DLxRkmWfPo9Gkd3pehV74 bNJm13SnijvIF0y5xlV+ENJpHwtXVsQ11IKcjcBHcPOjP0EpyygAOkqViMmIEk7gl0Q6 2CGDDCKqDmIK+LxOPq+bn5yK4VbfGoDbuePhcaVciNXl4W7JxtEV2/SbGJjcNy0LPMm6 cjDEoaN6oWtRjS7Hwzz5/nV7DADn89Eg7tzZ5ZQvhzGTAkzhGmfwKhDX7aMrj8c4hP1h oQAinl7jo9m9Smkv9BTvZhR8HzQXBp3TDMvMnffZckXxy7sdRybpQKBNNdARVfZoSPQr s9wg== X-Gm-Message-State: AOJu0YxkDJrWYl+kYJ8EkcUG/PH5p2RgQjzYSb8tt5FtdSiQ8+OL1wNS sn8oA0GPblxPLxgUcD9SjTcpW38mb7hi6CXFcjQi8p2jYjbbdATKZZyKndBevuE= X-Gm-Gg: ASbGncu43cV0JMTSL7N2vIcLntWzoKPbMY3PtCCU2UPTo1WZF7zJf9V1hzyeStbdZ2A NguWgAQRURHKcN4cZkiXaaUkfcCyC6EtnOPjJnpX3uzK8LjZ5CYmM9ie+pIKQyY2DjTE07uAdxZ I+17mzTKB7nlvGrrRPEN1NJJspmPQLDsyAsw5mi8T2qexZoej4gF8NiQz3FSZgXioN7UUCqXV1g cQ3m2l43shp3LDUg7tpCUAgM+fhocfGRKgXyiq7qQ4gjjPz6Kn77EZRWsJOrCHfIUkQKcdyPS36 bpYSu8Es15kQ6xd6BS6gYXCYNIiNziWJzR8RcGx4RMwpD+NNFEhDvkJl06LWPXIt8CFA X-Google-Smtp-Source: AGHT+IEcIE7eKu7LJSAVaYj917j3NklMNfNGzhL+EPc5DDxh/EvlDIRiRkNzr45U8w8mi4R68bB/eA== X-Received: by 2002:a05:6a20:2d27:b0:1e1:ac4f:d322 with SMTP id adf61e73a8af0-1ede882f0e3mr1326427637.14.1738711597347; Tue, 04 Feb 2025 15:26:37 -0800 (PST) Received: from hermes.local (204-195-96-226.wavecable.com. [204.195.96.226]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-acec096ac4asm10708581a12.63.2025.02.04.15.26.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2025 15:26:37 -0800 (PST) Date: Tue, 4 Feb 2025 15:26:35 -0800 From: Stephen Hemminger <stephen@networkplumber.org> To: Alan Beadle <ab.beadle@gmail.com> Cc: users@dpdk.org Subject: Re: Corrupted payload when sending many individual mbufs in loop Message-ID: <20250204152635.19deced0@hermes.local> In-Reply-To: <CANTAOdy7sNDDHDgL_Ow7xBJWeYEgbbik7Ab7x4wm0uFmgzFsXQ@mail.gmail.com> References: <CANTAOdy7sNDDHDgL_Ow7xBJWeYEgbbik7Ab7x4wm0uFmgzFsXQ@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions <users.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/users>, <mailto:users-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/users/> List-Post: <mailto:users@dpdk.org> List-Help: <mailto:users-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/users>, <mailto:users-request@dpdk.org?subject=subscribe> Errors-To: users-bounces@dpdk.org On Tue, 4 Feb 2025 17:36:31 -0500 Alan Beadle <ab.beadle@gmail.com> wrote: > Hello, > > I am working on an application that involves fragmenting a large > buffer into many smaller packets/mbufs and sending them with DPDK. The > fragment packets for each data buffer are currently sent in a loop > without any throttling (I plan to add some sort of rate limiting > later). I am not using any of the special DPDK features for IP > fragmention. I am simply preparing and sending mbufs for each > fragment. > > The problem I am having is that when I need to send large buffers, > which involve many packets being sent in a tight loop, sometimes part > of the payload is corrupted when received on the other machine. This > is rare and only happens above a certain data rate. I would like to > have a reliable way to detect (or even better, prevent) this > corruption. > > Out of paranoia I placed an rte_compiler_barrier() and then an > rte_mb() (a memory barrier) between the code that prepares the mbuf > and the actual tx_burst(). It did not help and I think it should be > unnecessary. > > I found some information about the function rte_mbuf_check() and have > started using it to check every received packet on the other end. It > fails to detect this corruption. > > Any advice would be appreciated. > -Alan Are you using refcounting of the mbuf. Are you reusing the data? DPDK has no callback or mechanism to indicate when data has actually been sent; ie the mbuf may still be in use for some amount of time while sitting in the device transmit queue and being DMA'd. Fragmenting can be complex to do.