From: Philipp Beyer <pbeyer@voipfuture.com>
To: Matt Laswell <laswell@infinite.io>
Cc: users <users@dpdk.org>
Subject: Re: [dpdk-users] Beginners question: rte_eth_tx_burst, rte_mbuf access synchronization
Date: Fri, 11 Nov 2016 15:06:43 +0100 [thread overview]
Message-ID: <e284a6c9-3354-cb4d-d6b8-96a9f5c4cedd@voipfuture.com> (raw)
In-Reply-To: <CA+GnqAqzkwopcmg3CyKe0GTAOB+umZha560LqBkc=UvWFwCQ=Q@mail.gmail.com>
Hi Matt,
Thanks for your answers. This helps as I am still stabbing in the dark
quite a lot.
I actually use 16, not one, distinct buffers to be sent in one burst.
But still, your conclusion is correct: I mess with the refcount, adjust
payload after calling rte_eth_tx_burst, and therefore get undefined
behaviour.
Your answer pretty much sound like you understood my point, so it seems
the solution I am looking for does not exist. Unfortunately, it is not
really only one byte i am changing. This was just a simplification, its
a few byte actually, but still a small portion of the payload. So your
idea won't really work.
But I might have found another idea: What about preparing all buffers of
a memory pool with the same payload? I should than get a pre-filled
buffer from rte_pktmbuf_alloc, right? Let's say, I initialize a buffer
for transmittion, the transmitting code free's this buffer, and I get
the same buffer back from rte_pktmbuf_alloc. What do I have to
re-initialize to have the same buffer again? Only the payload length? Is
this approach feasible, based on documented/specified behaviour?
Philipp
Am 11.11.2016 um 14:45 schrieb Matt Laswell:
> Hi Philipp,
>
> I'm a little unclear what you mean with your comments about adjusting
> the refcnt in your mbufs. You are absolutely correct that
> rte_eth_tx_burst doesn't synchronously transmit the packets. Instead,
> it puts them in a ring that is serviced by the poll mode driver.
> Eventually, they are handed off to the NIC, which copies them into its
> buffer and ultimately sends them on the wire.
>
> The architecture you've described won't work for the reasons you've
> surmised - when you hand a pointer to the pack to the device driver,
> you are giving it control of the memory pointed to. If you continue
> to modify its contents at that point, the results will be
> unpredictable. Also, it sounds as though you might really just have
> 16 pointers to a single packet, with a reference count of 16. Since
> you don't actually have 16 buffers, if you modify the contents of any
> one packet, you're modifying them all.
>
> Let me suggest that you might want to rethink your scheme. Rather than
> trying to reverse engineer a way to either make the PMD behave
> synchronously or to give you a callback, I would consider prebuilding
> packet contents at init time, then allocating mbufs and copying the
> contents in. I suspect you've avoided an approach like this because
> you'd like to not copy mostly the same data over and over when you
> only want to modify one byte.
>
> An alternative approach would be to use indirect mbufs. In essence,
> each packet you want to send might be made up of three mbufs. The
> first is an indirect mbuf that points to one that contains the common
> data at the start of your packets. The second contains the one byte
> that you wish to change. The third is an indirect mbuf that points to
> the common data at the end of your packets. I haven't used this
> approach myself, but I suspect it would let you avoid copying so much
> data.
>
> - Matt
>
> On Fri, Nov 11, 2016 at 3:49 AM, Philipp Beyer <pbeyer@voipfuture.com
> <mailto:pbeyer@voipfuture.com>> wrote:
>
> Hi!
>
> I am just writing my first code using dpdk, a traffic generator,
> for which I started with the l2fwd example.
>
> Basically, I need to send the same packet over a single interface,
> over an over again, with single bytes changed each time.
> I use rte_eth_tx_burst to send 16 packets at once. As I want to
> re-use the same buffers in a very simple way, I just increment the
> refcnt
> accordingly.
>
> My current code prepares all 16 buffers, calls rte_eth_tx_burst
> until all 16 packets are stored in the transmit ring, and starts
> over again, adjusting the buffers to send the next 16 packets.
>
> Currently I observe duplicate packets, although every packet
> should be individual due to single byte adjustments.
>
> My current problem is, as I guess, that rte_eth_tx_burst does not
> synchnolously transmit the count of packets, which is returned to
> the caller, but just stores them in transmit queue. So, I am not
> allowed to instantly re-use these buffers again.
>
> My question is: How do I know when to re-use buffers passed to
> rte_eth_tx_burst. Of course, I can check their refcnt member, and
> this would be perfectly fine. Apparently, I should have at least
> BURST_SIZE*2 buffers, passing BURST_SIZE buffers at once, so I can
> manipulate one set of buffers while the other is transmitted. But
> I am missing the idea of the best synchronization scheme here: How
> should I wait on this refcnt to drop?
>
> Some blind guessing:
> If I take the documentation of rte_eth_tx_burst literally, I could
> get the idea that refcounts of buffers are only decreased (buffers
> are 'freed'), while rte_eth_tx_burst is executed, but one function
> call might free buffers used by previous function calls. If this
> is correct, I still do not see a complete synchronization scheme.
> There is still a chance that I end up without any buffers left,
> which means I do not have a chance to call rte_eth_tx_burst again
> to free buffers.
>
> Thanks for any help,
> Philipp
>
>
--
Philipp Beyer
Software Developer
**
** **
**
Voipfuture GmbH Wendenstr. 4 20097 Hamburg Germany
Phone +49 40 688 9001 69 Fax +49 40 688 9001 99 www.voipfuture.com
<http://www.voipfuture.com/>
Managing Directors Jan Bastian Eyal Ullert
Commercial Court AG Hamburg HRB 109896 VAT ID DE263738086
next prev parent reply other threads:[~2016-11-11 14:06 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-11-11 9:49 Philipp Beyer
2016-11-11 12:35 ` Anupam Kapoor
2016-11-11 13:09 ` Philipp Beyer
2016-11-11 13:45 ` Matt Laswell
2016-11-11 14:06 ` Philipp Beyer [this message]
[not found] ` <9754A038-DB66-417F-8958-2DDDE317E7A2@net.in.tum.de>
2016-11-11 14:16 ` Paul Emmerich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e284a6c9-3354-cb4d-d6b8-96a9f5c4cedd@voipfuture.com \
--to=pbeyer@voipfuture.com \
--cc=laswell@infinite.io \
--cc=users@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).