[dpdk-dev] Packet Cloning

[dpdk-dev] Packet Cloning

[Padam Jeet Singh]

Hello,

Is there a function in DPDK to completely clone a pkt_mbuf including the segments? 

I am trying to build a packet mirroring application which sends packet out through two separate interfaces, but the packet payload needs to be altered before send.

Thanks,
Padam

Re: [dpdk-dev] Packet Cloning

[Stephen Hemminger]

On Thu, 28 May 2015 17:15:42 +0530
Padam Jeet Singh <padam.singh@inventum.net> wrote:

> Hello,
> 
> Is there a function in DPDK to completely clone a pkt_mbuf including the segments? 
> 
> I am trying to build a packet mirroring application which sends packet out through two separate interfaces, but the packet payload needs to be altered before send.
> 
> Thanks,
> Padam
> 
> 

Isn't this what you want?

/**
 * Creates a "clone" of the given packet mbuf.
 *
 * Walks through all segments of the given packet mbuf, and for each of them:
 *  - Creates a new packet mbuf from the given pool.
 *  - Attaches newly created mbuf to the segment.
 * Then updates pkt_len and nb_segs of the "clone" packet mbuf to match values
 * from the original packet mbuf.
 *
 * @param md
 *   The packet mbuf to be cloned.
 * @param mp
 *   The mempool from which the "clone" mbufs are allocated.
 * @return
 *   - The pointer to the new "clone" mbuf on success.
 *   - NULL if allocation fails.
 */
static inline struct rte_mbuf *rte_pktmbuf_clone(struct rte_mbuf *md,
		struct rte_mempool *mp)

Re: [dpdk-dev] Packet Cloning

[Matt Laswell]

Since Padam is going to be altering payload, he likely cannot use that API.
The rte_pktmbuf_clone() API doesn't make a copy of the payload.  Instead,
it gives you a second mbuf whose payload pointer points back to the
contents of the first (and also increments the reference counter on the
first so that it isn't actually freed until all clones are accounted for).
This is very fast, which is good.  However, since there's only really one
buffer full of payload, changes in the original also affect the clone and
vice versa.  This can have surprising and unpleasant side effects that may
not show up until you are under load, which is awesome*.

For what it's worth, if you need to be able to modify the copy while
leaving the original alone, I don't believe that there's a good solution
within DPDK.   However, writing your own API to copy rather than clone a
packet mbuf isn't difficult.

-- 
Matt Laswell
infinite io, inc.
laswell@infiniteio.com

* Don't ask me how I know how much awesome fun this can be, though I
suspect you can guess.

On Thu, May 28, 2015 at 9:52 AM, Stephen Hemminger <
stephen@networkplumber.org> wrote:

> On Thu, 28 May 2015 17:15:42 +0530
> Padam Jeet Singh <padam.singh@inventum.net> wrote:
>
> > Hello,
> >
> > Is there a function in DPDK to completely clone a pkt_mbuf including the
> segments?
> >
> > I am trying to build a packet mirroring application which sends packet
> out through two separate interfaces, but the packet payload needs to be
> altered before send.
> >
> > Thanks,
> > Padam
> >
> >
>
> Isn't this what you want?
>
> /**
>  * Creates a "clone" of the given packet mbuf.
>  *
>  * Walks through all segments of the given packet mbuf, and for each of
> them:
>  *  - Creates a new packet mbuf from the given pool.
>  *  - Attaches newly created mbuf to the segment.
>  * Then updates pkt_len and nb_segs of the "clone" packet mbuf to match
> values
>  * from the original packet mbuf.
>  *
>  * @param md
>  *   The packet mbuf to be cloned.
>  * @param mp
>  *   The mempool from which the "clone" mbufs are allocated.
>  * @return
>  *   - The pointer to the new "clone" mbuf on success.
>  *   - NULL if allocation fails.
>  */
> static inline struct rte_mbuf *rte_pktmbuf_clone(struct rte_mbuf *md,
>                 struct rte_mempool *mp)
>

Re: [dpdk-dev] Packet Cloning

[Kyle Larose]

I'm fairly new to dpdk, so I may be completely out to lunch on this, but
here's an idea to possibly improve performance compared to a straight copy
of the entire packet. If this idea makes sense, perhaps it could be added
to the mbuf library as an extension of the clone functionality?

If you are only modifying the headers (say the Ethernet header), is it
possible to make a copy of only the first N bytes (say 32 bytes)?

For example, you make two new "main" mbufs, which contain duplicate
metadata, and a copy of the first 32 bytes of the packet. Call them A and
B. Have both A and B chain to the original mbuf (call it O), which is
reference counted as with the normal clone functionality. Then, you adjust
the O such that its start data is 32 bytes into the packet.

When you transmit A, it will send its own copy of the 32 bytes, plus the
unaltered remaining data contained in O. A will be freed, and the refcount
of O decremented. When you transmit B, it will work the same as with the
previous one, except that when the refcount on O is decremented, it reaches
zero and it is freed as well.

I'm not sure if this makes sense in all cases (for example, maybe it's just
faster to allocate separate mbufs for 64-byte packets). Perhaps that could
also be handled transparently underneath the hood.

Thoughts?

Thanks,

Kyle

On Thu, May 28, 2015 at 11:10 AM, Matt Laswell <laswell@infiniteio.com>
wrote:

> Since Padam is going to be altering payload, he likely cannot use that API.
> The rte_pktmbuf_clone() API doesn't make a copy of the payload.  Instead,
> it gives you a second mbuf whose payload pointer points back to the
> contents of the first (and also increments the reference counter on the
> first so that it isn't actually freed until all clones are accounted for).
> This is very fast, which is good.  However, since there's only really one
> buffer full of payload, changes in the original also affect the clone and
> vice versa.  This can have surprising and unpleasant side effects that may
> not show up until you are under load, which is awesome*.
>
> For what it's worth, if you need to be able to modify the copy while
> leaving the original alone, I don't believe that there's a good solution
> within DPDK.   However, writing your own API to copy rather than clone a
> packet mbuf isn't difficult.
>
> --
> Matt Laswell
> infinite io, inc.
> laswell@infiniteio.com
>
> * Don't ask me how I know how much awesome fun this can be, though I
> suspect you can guess.
>
> On Thu, May 28, 2015 at 9:52 AM, Stephen Hemminger <
> stephen@networkplumber.org> wrote:
>
> > On Thu, 28 May 2015 17:15:42 +0530
> > Padam Jeet Singh <padam.singh@inventum.net> wrote:
> >
> > > Hello,
> > >
> > > Is there a function in DPDK to completely clone a pkt_mbuf including
> the
> > segments?
> > >
> > > I am trying to build a packet mirroring application which sends packet
> > out through two separate interfaces, but the packet payload needs to be
> > altered before send.
> > >
> > > Thanks,
> > > Padam
> > >
> > >
> >
> > Isn't this what you want?
> >
> > /**
> >  * Creates a "clone" of the given packet mbuf.
> >  *
> >  * Walks through all segments of the given packet mbuf, and for each of
> > them:
> >  *  - Creates a new packet mbuf from the given pool.
> >  *  - Attaches newly created mbuf to the segment.
> >  * Then updates pkt_len and nb_segs of the "clone" packet mbuf to match
> > values
> >  * from the original packet mbuf.
> >  *
> >  * @param md
> >  *   The packet mbuf to be cloned.
> >  * @param mp
> >  *   The mempool from which the "clone" mbufs are allocated.
> >  * @return
> >  *   - The pointer to the new "clone" mbuf on success.
> >  *   - NULL if allocation fails.
> >  */
> > static inline struct rte_mbuf *rte_pktmbuf_clone(struct rte_mbuf *md,
> >                 struct rte_mempool *mp)
> >
>

Re: [dpdk-dev] Packet Cloning

[Matt Laswell]

Hey Kyle,

That's one way you can handle it, though I suspect you'll end up with some
complexity elsewhere in your code to deal with remembering whether you
should look at the original data or the copied and modified data.  Another
way is just to make a copy of the original mbuf, but have your copy API
stop after it reaches some particular point.  Perhaps just the L2-L4
headers, perhaps a few hundred bytes into payload, or perhaps something
else entirely. This all gets very application dependent, of course.  How
much is "enough" is going to depend heavily on what you're trying to
accomplish.

-- 
Matt Laswell
infinite io, inc.
laswell@infiniteio.com


On Thu, May 28, 2015 at 10:38 AM, Kyle Larose <eomereadig@gmail.com> wrote:

> I'm fairly new to dpdk, so I may be completely out to lunch on this, but
> here's an idea to possibly improve performance compared to a straight copy
> of the entire packet. If this idea makes sense, perhaps it could be added
> to the mbuf library as an extension of the clone functionality?
>
> If you are only modifying the headers (say the Ethernet header), is it
> possible to make a copy of only the first N bytes (say 32 bytes)?
>
> For example, you make two new "main" mbufs, which contain duplicate
> metadata, and a copy of the first 32 bytes of the packet. Call them A and
> B. Have both A and B chain to the original mbuf (call it O), which is
> reference counted as with the normal clone functionality. Then, you adjust
> the O such that its start data is 32 bytes into the packet.
>
> When you transmit A, it will send its own copy of the 32 bytes, plus the
> unaltered remaining data contained in O. A will be freed, and the refcount
> of O decremented. When you transmit B, it will work the same as with the
> previous one, except that when the refcount on O is decremented, it reaches
> zero and it is freed as well.
>
> I'm not sure if this makes sense in all cases (for example, maybe it's
> just faster to allocate separate mbufs for 64-byte packets). Perhaps that
> could also be handled transparently underneath the hood.
>
> Thoughts?
>
> Thanks,
>
> Kyle
>
> On Thu, May 28, 2015 at 11:10 AM, Matt Laswell <laswell@infiniteio.com>
> wrote:
>
>> Since Padam is going to be altering payload, he likely cannot use that
>> API.
>> The rte_pktmbuf_clone() API doesn't make a copy of the payload.  Instead,
>> it gives you a second mbuf whose payload pointer points back to the
>> contents of the first (and also increments the reference counter on the
>> first so that it isn't actually freed until all clones are accounted for).
>> This is very fast, which is good.  However, since there's only really one
>> buffer full of payload, changes in the original also affect the clone and
>> vice versa.  This can have surprising and unpleasant side effects that may
>> not show up until you are under load, which is awesome*.
>>
>> For what it's worth, if you need to be able to modify the copy while
>> leaving the original alone, I don't believe that there's a good solution
>> within DPDK.   However, writing your own API to copy rather than clone a
>> packet mbuf isn't difficult.
>>
>> --
>> Matt Laswell
>> infinite io, inc.
>> laswell@infiniteio.com
>>
>> * Don't ask me how I know how much awesome fun this can be, though I
>> suspect you can guess.
>>
>> On Thu, May 28, 2015 at 9:52 AM, Stephen Hemminger <
>> stephen@networkplumber.org> wrote:
>>
>> > On Thu, 28 May 2015 17:15:42 +0530
>> > Padam Jeet Singh <padam.singh@inventum.net> wrote:
>> >
>> > > Hello,
>> > >
>> > > Is there a function in DPDK to completely clone a pkt_mbuf including
>> the
>> > segments?
>> > >
>> > > I am trying to build a packet mirroring application which sends packet
>> > out through two separate interfaces, but the packet payload needs to be
>> > altered before send.
>> > >
>> > > Thanks,
>> > > Padam
>> > >
>> > >
>> >
>> > Isn't this what you want?
>> >
>> > /**
>> >  * Creates a "clone" of the given packet mbuf.
>> >  *
>> >  * Walks through all segments of the given packet mbuf, and for each of
>> > them:
>> >  *  - Creates a new packet mbuf from the given pool.
>> >  *  - Attaches newly created mbuf to the segment.
>> >  * Then updates pkt_len and nb_segs of the "clone" packet mbuf to match
>> > values
>> >  * from the original packet mbuf.
>> >  *
>> >  * @param md
>> >  *   The packet mbuf to be cloned.
>> >  * @param mp
>> >  *   The mempool from which the "clone" mbufs are allocated.
>> >  * @return
>> >  *   - The pointer to the new "clone" mbuf on success.
>> >  *   - NULL if allocation fails.
>> >  */
>> > static inline struct rte_mbuf *rte_pktmbuf_clone(struct rte_mbuf *md,
>> >                 struct rte_mempool *mp)
>> >
>>
>
>

Re: [dpdk-dev] Packet Cloning

[Marc Sune]

On 28/05/15 18:06, Matt Laswell wrote:
> Hey Kyle,
>
> That's one way you can handle it, though I suspect you'll end up with some
> complexity elsewhere in your code to deal with remembering whether you
> should look at the original data or the copied and modified data.  Another
> way is just to make a copy of the original mbuf, but have your copy API
> stop after it reaches some particular point.  Perhaps just the L2-L4
> headers, perhaps a few hundred bytes into payload, or perhaps something
> else entirely. This all gets very application dependent, of course.  How
> much is "enough" is going to depend heavily on what you're trying to
> accomplish.

mbufs can be chained in multiple segments. So you could first split into 
two segments leaving the big chunk in the original mbuf (chunk2) and 
copy chunk1 into the new mbuf (check prepend, adj and trim).

Marc

[1] http://dpdk.org/doc/api/rte__mbuf_8h.html

>

Re: [dpdk-dev] Packet Cloning

[Padam Jeet Singh]

Thanks Marc, Matt, Kyle, and Stephen for your inputs!

I have a possibly good solution of splitting the mbuf into parts to correctly mirror to multiple interfaces, else a fallback of copying the full payload into a new mbuf.

Though the library doesn’t offer a standard method to copy, I can submit a rte_mbuf_copy() patch that does a full deep copy.

Regards,
Padam


> On 28-May-2015, at 9:43 pm, Marc Sune <marc.sune@bisdn.de> wrote:
> 
> 
> 
> On 28/05/15 18:06, Matt Laswell wrote:
>> Hey Kyle,
>> 
>> That's one way you can handle it, though I suspect you'll end up with some
>> complexity elsewhere in your code to deal with remembering whether you
>> should look at the original data or the copied and modified data.  Another
>> way is just to make a copy of the original mbuf, but have your copy API
>> stop after it reaches some particular point.  Perhaps just the L2-L4
>> headers, perhaps a few hundred bytes into payload, or perhaps something
>> else entirely. This all gets very application dependent, of course.  How
>> much is "enough" is going to depend heavily on what you're trying to
>> accomplish.
> 
> mbufs can be chained in multiple segments. So you could first split into two segments leaving the big chunk in the original mbuf (chunk2) and copy chunk1 into the new mbuf (check prepend, adj and trim).
> 
> Marc
> 
> [1] http://dpdk.org/doc/api/rte__mbuf_8h.html
> 
>> 
> 

Padam J. Singh | VP Engg. | Inventum Technologies Private Limited
www.inventum.net <http://www.inventum.net/>
+91.9810146640
 
Address:
C-70, Phase II Extension,
Noida 201305, UP,
India
 
Tel     : +91.120.4647000
 
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. No employee or agent is authorized to conclude any binding agreement on behalf of [Inventum Technologies Private Limited] with another party by email without express written confirmation by a whole-time Director of the Company who has been authorized vide a valid board resolution to the effect. Our company accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. Employees of [Inventum Technologies Private Limited] are expressly required not to make defamatory statements and not to infringe or authorize any infringement of copyright or any other legal right by email communications. Any such communication is contrary to company policy and outside the scope of the employment of the individual concerned. The company will not accept any liability in respect of such communication, and the employee responsible will be personally liable for any damages or other liability arising.