From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.voipfuture.com (mail.voipfuture.com [213.238.34.195]) by dpdk.org (Postfix) with ESMTP id 3D76BD552 for ; Fri, 11 Nov 2016 15:06:44 +0100 (CET) X-Footer: dm9pcGZ1dHVyZS5jb20= Received: from [192.168.1.155] ([192.168.1.1]) (authenticated user pbeyer@voipfuture.com) by mail.voipfuture.com (Kerio Connect 9.1.1) with ESMTPSA (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128 bits)); Fri, 11 Nov 2016 15:06:43 +0100 To: Matt Laswell References: <750170fd-492a-588c-de6a-27c0547d1cbd@voipfuture.com> Cc: users From: Philipp Beyer Message-ID: Date: Fri, 11 Nov 2016 15:06:43 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-users] Beginners question: rte_eth_tx_burst, rte_mbuf access synchronization X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Nov 2016 14:06:44 -0000 Hi Matt, Thanks for your answers. This helps as I am still stabbing in the dark quite a lot. I actually use 16, not one, distinct buffers to be sent in one burst. But still, your conclusion is correct: I mess with the refcount, adjust payload after calling rte_eth_tx_burst, and therefore get undefined behaviour. Your answer pretty much sound like you understood my point, so it seems the solution I am looking for does not exist. Unfortunately, it is not really only one byte i am changing. This was just a simplification, its a few byte actually, but still a small portion of the payload. So your idea won't really work. But I might have found another idea: What about preparing all buffers of a memory pool with the same payload? I should than get a pre-filled buffer from rte_pktmbuf_alloc, right? Let's say, I initialize a buffer for transmittion, the transmitting code free's this buffer, and I get the same buffer back from rte_pktmbuf_alloc. What do I have to re-initialize to have the same buffer again? Only the payload length? Is this approach feasible, based on documented/specified behaviour? Philipp Am 11.11.2016 um 14:45 schrieb Matt Laswell: > Hi Philipp, > > I'm a little unclear what you mean with your comments about adjusting > the refcnt in your mbufs. You are absolutely correct that > rte_eth_tx_burst doesn't synchronously transmit the packets. Instead, > it puts them in a ring that is serviced by the poll mode driver. > Eventually, they are handed off to the NIC, which copies them into its > buffer and ultimately sends them on the wire. > > The architecture you've described won't work for the reasons you've > surmised - when you hand a pointer to the pack to the device driver, > you are giving it control of the memory pointed to. If you continue > to modify its contents at that point, the results will be > unpredictable. Also, it sounds as though you might really just have > 16 pointers to a single packet, with a reference count of 16. Since > you don't actually have 16 buffers, if you modify the contents of any > one packet, you're modifying them all. > > Let me suggest that you might want to rethink your scheme. Rather than > trying to reverse engineer a way to either make the PMD behave > synchronously or to give you a callback, I would consider prebuilding > packet contents at init time, then allocating mbufs and copying the > contents in. I suspect you've avoided an approach like this because > you'd like to not copy mostly the same data over and over when you > only want to modify one byte. > > An alternative approach would be to use indirect mbufs. In essence, > each packet you want to send might be made up of three mbufs. The > first is an indirect mbuf that points to one that contains the common > data at the start of your packets. The second contains the one byte > that you wish to change. The third is an indirect mbuf that points to > the common data at the end of your packets. I haven't used this > approach myself, but I suspect it would let you avoid copying so much > data. > > - Matt > > On Fri, Nov 11, 2016 at 3:49 AM, Philipp Beyer > wrote: > > Hi! > > I am just writing my first code using dpdk, a traffic generator, > for which I started with the l2fwd example. > > Basically, I need to send the same packet over a single interface, > over an over again, with single bytes changed each time. > I use rte_eth_tx_burst to send 16 packets at once. As I want to > re-use the same buffers in a very simple way, I just increment the > refcnt > accordingly. > > My current code prepares all 16 buffers, calls rte_eth_tx_burst > until all 16 packets are stored in the transmit ring, and starts > over again, adjusting the buffers to send the next 16 packets. > > Currently I observe duplicate packets, although every packet > should be individual due to single byte adjustments. > > My current problem is, as I guess, that rte_eth_tx_burst does not > synchnolously transmit the count of packets, which is returned to > the caller, but just stores them in transmit queue. So, I am not > allowed to instantly re-use these buffers again. > > My question is: How do I know when to re-use buffers passed to > rte_eth_tx_burst. Of course, I can check their refcnt member, and > this would be perfectly fine. Apparently, I should have at least > BURST_SIZE*2 buffers, passing BURST_SIZE buffers at once, so I can > manipulate one set of buffers while the other is transmitted. But > I am missing the idea of the best synchronization scheme here: How > should I wait on this refcnt to drop? > > Some blind guessing: > If I take the documentation of rte_eth_tx_burst literally, I could > get the idea that refcounts of buffers are only decreased (buffers > are 'freed'), while rte_eth_tx_burst is executed, but one function > call might free buffers used by previous function calls. If this > is correct, I still do not see a complete synchronization scheme. > There is still a chance that I end up without any buffers left, > which means I do not have a chance to call rte_eth_tx_burst again > to free buffers. > > Thanks for any help, > Philipp > > -- Philipp Beyer Software Developer ** ** ** ** Voipfuture GmbH Wendenstr. 4 20097 Hamburg Germany Phone +49 40 688 9001 69 Fax +49 40 688 9001 99 www.voipfuture.com Managing Directors Jan Bastian Eyal Ullert Commercial Court AG Hamburg HRB 109896 VAT ID DE263738086