From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-it0-f42.google.com (mail-it0-f42.google.com [209.85.214.42]) by dpdk.org (Postfix) with ESMTP id C609BD558 for ; Fri, 11 Nov 2016 14:45:03 +0100 (CET) Received: by mail-it0-f42.google.com with SMTP id u205so123327904itc.0 for ; Fri, 11 Nov 2016 05:45:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infinite-io.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=INiAf8+g8YeCq/1I9azdJums2TCes7LxF8oxPPygoD4=; b=BU3E5NZdLBQrs0Ud4pBMcmIixCDPV+jvEV9lniMtUCEmJAjt15/a/ULSgvXxqLhN3o 5SrUasfJnyMAEHLZ1HNoo+3O3vC23m/NrGOytRa4RCVU8hkDj1UPf23Yh4bDuQmSBSBd KjKA8exviVg9GtZa7+iTLMwKn6rxkJbMwXR67+JtSRUhQSWc4O+l8OoHcpWdandaxLFC w7alsEd0xL1+sMmE2Rt8on1rukTBVXH/yd7sDgzOB7UKFtKXkxHbxUpw5X6eegl8Ajpp f+jGnNb2q1yIcC1FukSvB+VqTRLT7E718tMB260k2N0h+aIV+1BF3ZvfTKj4meoJHR9l vdsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=INiAf8+g8YeCq/1I9azdJums2TCes7LxF8oxPPygoD4=; b=lWVoD4QkbssbuhmhwuFSdni7BoL8N9ODPsuOwCxnj0amJpIBlrd3n538j1J9uGlx3f xu8HJQs/EKMH2wiz8hA8BOK6Ka3qTSEzVMPZ+h2fS0y/ttttjGyGLDi+pPvgMWSpSCly S1FmqB4b7Z5MA83MzR9dO1MmMxj+aq0PdgtcVkV2FroRcHAypEiWPJ2oduUlyblTg6ag rAI+rPNeHsk99NPim6+gu9xvgJ+ee4IrcWDP836U6JtGZKO6+QL082WtILN+XgcIOe2W uLkqFpeqG9vnLfPl7tDFqoxui9wroFFDPvvdkc0HDDMF4QsRGMJYxyyYoO9KY02wzAbE zTdw== X-Gm-Message-State: ABUngvcpStHqSYI1cazyz9iYooX6H+/uXYxZIOlpssOSpCZmI5qg13HnEIDmAnWbjbKdhBsR+HaN9KTJ8AChdA== X-Received: by 10.36.93.83 with SMTP id w80mr8920688ita.90.1478871903163; Fri, 11 Nov 2016 05:45:03 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.19.202 with HTTP; Fri, 11 Nov 2016 05:45:02 -0800 (PST) In-Reply-To: <750170fd-492a-588c-de6a-27c0547d1cbd@voipfuture.com> References: <750170fd-492a-588c-de6a-27c0547d1cbd@voipfuture.com> From: Matt Laswell Date: Fri, 11 Nov 2016 07:45:02 -0600 Message-ID: To: Philipp Beyer Cc: users Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Beginners question: rte_eth_tx_burst, rte_mbuf access synchronization X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Nov 2016 13:45:04 -0000 Hi Philipp, I'm a little unclear what you mean with your comments about adjusting the refcnt in your mbufs. You are absolutely correct that rte_eth_tx_burst doesn't synchronously transmit the packets. Instead, it puts them in a ring that is serviced by the poll mode driver. Eventually, they are handed off to the NIC, which copies them into its buffer and ultimately sends them on the wire. The architecture you've described won't work for the reasons you've surmised - when you hand a pointer to the pack to the device driver, you are giving it control of the memory pointed to. If you continue to modify its contents at that point, the results will be unpredictable. Also, it sounds as though you might really just have 16 pointers to a single packet, with a reference count of 16. Since you don't actually have 16 buffers, if you modify the contents of any one packet, you're modifying them all. Let me suggest that you might want to rethink your scheme. Rather than trying to reverse engineer a way to either make the PMD behave synchronously or to give you a callback, I would consider prebuilding packet contents at init time, then allocating mbufs and copying the contents in. I suspect you've avoided an approach like this because you'd like to not copy mostly the same data over and over when you only want to modify one byte. An alternative approach would be to use indirect mbufs. In essence, each packet you want to send might be made up of three mbufs. The first is an indirect mbuf that points to one that contains the common data at the start of your packets. The second contains the one byte that you wish to change. The third is an indirect mbuf that points to the common data at the end of your packets. I haven't used this approach myself, but I suspect it would let you avoid copying so much data. - Matt On Fri, Nov 11, 2016 at 3:49 AM, Philipp Beyer wrote: > Hi! > > I am just writing my first code using dpdk, a traffic generator, for which > I started with the l2fwd example. > > Basically, I need to send the same packet over a single interface, over an > over again, with single bytes changed each time. > I use rte_eth_tx_burst to send 16 packets at once. As I want to re-use the > same buffers in a very simple way, I just increment the refcnt > accordingly. > > My current code prepares all 16 buffers, calls rte_eth_tx_burst until all > 16 packets are stored in the transmit ring, and starts over again, > adjusting the buffers to send the next 16 packets. > > Currently I observe duplicate packets, although every packet should be > individual due to single byte adjustments. > > My current problem is, as I guess, that rte_eth_tx_burst does not > synchnolously transmit the count of packets, which is returned to the > caller, but just stores them in transmit queue. So, I am not allowed to > instantly re-use these buffers again. > > My question is: How do I know when to re-use buffers passed to > rte_eth_tx_burst. Of course, I can check their refcnt member, and this > would be perfectly fine. Apparently, I should have at least BURST_SIZE*2 > buffers, passing BURST_SIZE buffers at once, so I can manipulate one set of > buffers while the other is transmitted. But I am missing the idea of the > best synchronization scheme here: How should I wait on this refcnt to drop? > > Some blind guessing: > If I take the documentation of rte_eth_tx_burst literally, I could get the > idea that refcounts of buffers are only decreased (buffers are 'freed'), > while rte_eth_tx_burst is executed, but one function call might free > buffers used by previous function calls. If this is correct, I still do not > see a complete synchronization scheme. There is still a chance that I end > up without any buffers left, which means I do not have a chance to call > rte_eth_tx_burst again to free buffers. > > Thanks for any help, > Philipp > >