[dpdk-users] ipv4 fragmentation

[dpdk-users] ipv4 fragmentation

[Александр Киселев]

Hi!

I am trying to use rte_ipv4_fragment_packet() function, but so far all
attemps has been unsuccessful.

Fragment function gets a packet, successfuly splits it to the two packets,
then I add an L2 header to the each of resulting fragments
and send the fragments using rte_eth_tx_burst() to the wire. All steps of
the above process are succesfully completed without any error, but I don't
see the fragments on the wire and PMD stat function also reports that no
packets were transmitted.

I've rechecked many times that L2 header of each fragments is correct, that
values of l2_len and l3_len rte_mbuf members are correct,
that the rte_eth_tx_burst() result is correct (2 packets). The sizes of
fragments are also correct.

What did I miss?

I am using DPDK 2.2.0 and ixgbe PMD driver.

This is some log data of a test run.
==
==
The original packet that has to be fragmented (including L2 header data)

dump mbuf at 0x0x7f7d103870c0, phys=215787140, buf_len=2176
  pkt_len=2546, ol_flags=40000000000003, nb_segs=2, in_port=0
  segment at 0x0x7f7d103870c0, data=0x0x7f7d103871c0, data_len=1518
  Dump data at [0x7f7d103871c0], len=50
00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 | ..!<iE....<.....
00000010: 08 00 45 00 09 E0 7E D1 00 00 40 01 00 00 C0 A8 | ..E...~...@.....
00000020: 7D 01 C0 A8 7D 02 00 00 FD FD 0B 2D 00 01 56 FF | }...}......-..V.
00000030: AD 57 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | .W
  segment at 0x0x7f7d10386780, data=0x0x7f7d103868a6, data_len=1028


fragmentation is needed. mtu 1500, pkt len 2546, data len 1518, l2 len 18,
l3 len 20
num fragment needed: 2

==
==
Fragments (including L2 header data):

fragment #0, pkt len 1518, l2 len 18, l3 len 20, data len 38

dump mbuf at 0x0x7f7d10388340, phys=2157883c0, buf_len=2176
  pkt_len=1518, ol_flags=40000000000000, nb_segs=2, in_port=255
  segment at 0x0x7f7d10388340, data=0x0x7f7d1038842e, data_len=38
  Dump data at [0x7f7d1038842e], len=38
00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 | ..!<iE....<.....
00000010: 08 00 45 00 05 DC 7E D1 20 00 40 01 00 00 C0 A8 | ..E...~. .@.....
00000020: 7D 01 C0 A8 7D 02 |  |  |  |  |  |  |  |  |  |  | }...}.
  segment at 0x0x7f7d0ef87dc0, data=0x0x7f7d103871e6, data_len=1480
  Dump data at [0x7f7d103871e6], len=12
00000000: 00 00 FD FD 0B 2D 00 01 56 FF AD 57 |  |  |  |  | .....-..V..W


fragment #1, pkt len 1066, l2 len 18, l3 len 20, data len 38

dump mbuf at 0x0x7f7d10388c80, phys=215788d00, buf_len=2176
  pkt_len=1066, ol_flags=40000000000000, nb_segs=2, in_port=255
  segment at 0x0x7f7d10388c80, data=0x0x7f7d10388d6e, data_len=38
  Dump data at [0x7f7d10388d6e], len=38
00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 | ..!<iE....<.....
00000010: 08 00 45 00 04 18 7E D1 00 B9 40 01 00 00 C0 A8 | ..E...~...@.....
00000020: 7D 01 C0 A8 7D 02 |  |  |  |  |  |  |  |  |  |  | }...}.
  segment at 0x0x7f7d0ef87480, data=0x0x7f7d103868a6, data_len=1028
  Dump data at [0x7f7d103868a6], len=12
00000000: C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 CA CB |  |  |  |  | ............
lcore 1 SEND

==
==
sending 2 packets
sending result: 2


-- 
--
Kiselev Alexander

Re: [dpdk-users] ipv4 fragmentation

[Matt Laswell]

I've found that if both the packet length and data length fields in an mbuf
aren't set properly, DPDK will quietly drop it on the floor when I try to
send it.  I don't know if that's the cause of your problems, but it's one
of the first places that I would look.

--
Matt Laswell
Principal Software Engineer, infinite io
laswell@infinite.io


On Fri, Aug 12, 2016 at 12:32 PM, Александр Киселев <kiselev99@gmail.com>
wrote:

> Hi!
>
> I am trying to use rte_ipv4_fragment_packet() function, but so far all
> attemps has been unsuccessful.
>
> Fragment function gets a packet, successfuly splits it to the two packets,
> then I add an L2 header to the each of resulting fragments
> and send the fragments using rte_eth_tx_burst() to the wire. All steps of
> the above process are succesfully completed without any error, but I don't
> see the fragments on the wire and PMD stat function also reports that no
> packets were transmitted.
>
> I've rechecked many times that L2 header of each fragments is correct, that
> values of l2_len and l3_len rte_mbuf members are correct,
> that the rte_eth_tx_burst() result is correct (2 packets). The sizes of
> fragments are also correct.
>
> What did I miss?
>
> I am using DPDK 2.2.0 and ixgbe PMD driver.
>
> This is some log data of a test run.
> ==
> ==
> The original packet that has to be fragmented (including L2 header data)
>
> dump mbuf at 0x0x7f7d103870c0, phys=215787140, buf_len=2176
>   pkt_len=2546, ol_flags=40000000000003, nb_segs=2, in_port=0
>   segment at 0x0x7f7d103870c0, data=0x0x7f7d103871c0, data_len=1518
>   Dump data at [0x7f7d103871c0], len=50
> 00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 |
> ..!<iE....<.....
> 00000010: 08 00 45 00 09 E0 7E D1 00 00 40 01 00 00 C0 A8 |
> ..E...~...@.....
> 00000020: 7D 01 C0 A8 7D 02 00 00 FD FD 0B 2D 00 01 56 FF |
> }...}......-..V.
> 00000030: AD 57 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | .W
>   segment at 0x0x7f7d10386780, data=0x0x7f7d103868a6, data_len=1028
>
>
> fragmentation is needed. mtu 1500, pkt len 2546, data len 1518, l2 len 18,
> l3 len 20
> num fragment needed: 2
>
> ==
> ==
> Fragments (including L2 header data):
>
> fragment #0, pkt len 1518, l2 len 18, l3 len 20, data len 38
>
> dump mbuf at 0x0x7f7d10388340, phys=2157883c0, buf_len=2176
>   pkt_len=1518, ol_flags=40000000000000, nb_segs=2, in_port=255
>   segment at 0x0x7f7d10388340, data=0x0x7f7d1038842e, data_len=38
>   Dump data at [0x7f7d1038842e], len=38
> 00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 |
> ..!<iE....<.....
> 00000010: 08 00 45 00 05 DC 7E D1 20 00 40 01 00 00 C0 A8 | ..E...~.
> .@.....
> 00000020: 7D 01 C0 A8 7D 02 |  |  |  |  |  |  |  |  |  |  | }...}.
>   segment at 0x0x7f7d0ef87dc0, data=0x0x7f7d103871e6, data_len=1480
>   Dump data at [0x7f7d103871e6], len=12
> 00000000: 00 00 FD FD 0B 2D 00 01 56 FF AD 57 |  |  |  |  | .....-..V..W
>
>
> fragment #1, pkt len 1066, l2 len 18, l3 len 20, data len 38
>
> dump mbuf at 0x0x7f7d10388c80, phys=215788d00, buf_len=2176
>   pkt_len=1066, ol_flags=40000000000000, nb_segs=2, in_port=255
>   segment at 0x0x7f7d10388c80, data=0x0x7f7d10388d6e, data_len=38
>   Dump data at [0x7f7d10388d6e], len=38
> 00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 |
> ..!<iE....<.....
> 00000010: 08 00 45 00 04 18 7E D1 00 B9 40 01 00 00 C0 A8 |
> ..E...~...@.....
> 00000020: 7D 01 C0 A8 7D 02 |  |  |  |  |  |  |  |  |  |  | }...}.
>   segment at 0x0x7f7d0ef87480, data=0x0x7f7d103868a6, data_len=1028
>   Dump data at [0x7f7d103868a6], len=12
> 00000000: C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 CA CB |  |  |  |  | ............
> lcore 1 SEND
>
> ==
> ==
> sending 2 packets
> sending result: 2
>
>
> --
> --
> Kiselev Alexander
>

Re: [dpdk-users] ipv4 fragmentation

[Александр Киселев]

It looks like the packet length and data length are set properly.

According to my log data I have two fragments:

#1 fragment
pkt_len=1518, 2 segments:
data_len=38 and data_len=1480

#2 fragment
pkt_len=1066, 2 segments:
data_len=38 and data_len=1028

For each fragment sum of the data lengths of all segmens equal to the
pkt_len value.



2016-08-12 20:44 GMT+03:00 Matt Laswell <laswell@infinite.io>:

> I've found that if both the packet length and data length fields in an
> mbuf aren't set properly, DPDK will quietly drop it on the floor when I try
> to send it.  I don't know if that's the cause of your problems, but it's
> one of the first places that I would look.
>
> --
> Matt Laswell
> Principal Software Engineer, infinite io
> laswell@infinite.io
>
>
> On Fri, Aug 12, 2016 at 12:32 PM, Александр Киселев <kiselev99@gmail.com>
> wrote:
>
>> Hi!
>>
>> I am trying to use rte_ipv4_fragment_packet() function, but so far all
>> attemps has been unsuccessful.
>>
>> Fragment function gets a packet, successfuly splits it to the two packets,
>> then I add an L2 header to the each of resulting fragments
>> and send the fragments using rte_eth_tx_burst() to the wire. All steps of
>> the above process are succesfully completed without any error, but I don't
>> see the fragments on the wire and PMD stat function also reports that no
>> packets were transmitted.
>>
>> I've rechecked many times that L2 header of each fragments is correct,
>> that
>> values of l2_len and l3_len rte_mbuf members are correct,
>> that the rte_eth_tx_burst() result is correct (2 packets). The sizes of
>> fragments are also correct.
>>
>> What did I miss?
>>
>> I am using DPDK 2.2.0 and ixgbe PMD driver.
>>
>> This is some log data of a test run.
>> ==
>> ==
>> The original packet that has to be fragmented (including L2 header data)
>>
>> dump mbuf at 0x0x7f7d103870c0, phys=215787140, buf_len=2176
>>   pkt_len=2546, ol_flags=40000000000003, nb_segs=2, in_port=0
>>   segment at 0x0x7f7d103870c0, data=0x0x7f7d103871c0, data_len=1518
>>   Dump data at [0x7f7d103871c0], len=50
>> 00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 |
>> ..!<iE....<.....
>> 00000010: 08 00 45 00 09 E0 7E D1 00 00 40 01 00 00 C0 A8 |
>> ..E...~...@.....
>> 00000020: 7D 01 C0 A8 7D 02 00 00 FD FD 0B 2D 00 01 56 FF |
>> }...}......-..V.
>> 00000030: AD 57 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | .W
>>   segment at 0x0x7f7d10386780, data=0x0x7f7d103868a6, data_len=1028
>>
>>
>> fragmentation is needed. mtu 1500, pkt len 2546, data len 1518, l2 len 18,
>> l3 len 20
>> num fragment needed: 2
>>
>> ==
>> ==
>> Fragments (including L2 header data):
>>
>> fragment #0, pkt len 1518, l2 len 18, l3 len 20, data len 38
>>
>> dump mbuf at 0x0x7f7d10388340, phys=2157883c0, buf_len=2176
>>   pkt_len=1518, ol_flags=40000000000000, nb_segs=2, in_port=255
>>   segment at 0x0x7f7d10388340, data=0x0x7f7d1038842e, data_len=38
>>   Dump data at [0x7f7d1038842e], len=38
>> 00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 |
>> ..!<iE....<.....
>> 00000010: 08 00 45 00 05 DC 7E D1 20 00 40 01 00 00 C0 A8 | ..E...~.
>> .@.....
>> 00000020: 7D 01 C0 A8 7D 02 |  |  |  |  |  |  |  |  |  |  | }...}.
>>   segment at 0x0x7f7d0ef87dc0, data=0x0x7f7d103871e6, data_len=1480
>>   Dump data at [0x7f7d103871e6], len=12
>> 00000000: 00 00 FD FD 0B 2D 00 01 56 FF AD 57 |  |  |  |  | .....-..V..W
>>
>>
>> fragment #1, pkt len 1066, l2 len 18, l3 len 20, data len 38
>>
>> dump mbuf at 0x0x7f7d10388c80, phys=215788d00, buf_len=2176
>>   pkt_len=1066, ol_flags=40000000000000, nb_segs=2, in_port=255
>>   segment at 0x0x7f7d10388c80, data=0x0x7f7d10388d6e, data_len=38
>>   Dump data at [0x7f7d10388d6e], len=38
>> 00000000: 00 1B 21 3C 69 45 90 E2 BA 00 3C CC 81 00 00 86 |
>> ..!<iE....<.....
>> 00000010: 08 00 45 00 04 18 7E D1 00 B9 40 01 00 00 C0 A8 |
>> ..E...~...@.....
>> 00000020: 7D 01 C0 A8 7D 02 |  |  |  |  |  |  |  |  |  |  | }...}.
>>   segment at 0x0x7f7d0ef87480, data=0x0x7f7d103868a6, data_len=1028
>>   Dump data at [0x7f7d103868a6], len=12
>> 00000000: C0 C1 C2 C3 C4 C5 C6 C7 C8 C9 CA CB |  |  |  |  | ............
>> lcore 1 SEND
>>
>> ==
>> ==
>> sending 2 packets
>> sending result: 2
>>
>>
>> --
>> --
>> Kiselev Alexander
>>
>
>


-- 
--
Kiselev Alexander

Re: [dpdk-users] ipv4 fragmentation

[Stephen Hemminger]

On Fri, 12 Aug 2016 20:59:19 +0300
Александр Киселев <kiselev99@gmail.com> wrote:

> > On Fri, Aug 12, 2016 at 12:32 PM, Александр Киселев <kiselev99@gmail.com>
> > wrote:
> >  
> >> Hi!
> >>
> >> I am trying to use rte_ipv4_fragment_packet() function, but so far all
> >> attemps has been unsuccessful.
> >>
> >> Fragment function gets a packet, successfuly splits it to the two packets,
> >> then I add an L2 header to the each of resulting fragments
> >> and send the fragments using rte_eth_tx_burst() to the wire. All steps of
> >> the above process are succesfully completed without any error, but I don't
> >> see the fragments on the wire and PMD stat function also reports that no
> >> packets were transmitted.

Did you account for the fact that some of these might be indirect mbuf's
where refcount > 1? If refcount is not 1 then mbuf is shared, and you can
not safely write your L2 header directly.

Re: [dpdk-users] ipv4 fragmentation

[Александр Киселев]

2016-08-12 22:42 GMT+03:00 Stephen Hemminger <stephen@networkplumber.org>:

> > >> Fragment function gets a packet, successfuly splits it to the two
> packets,
> > >> then I add an L2 header to the each of resulting fragments
> > >> and send the fragments using rte_eth_tx_burst() to the wire. All
> steps of
> > >> the above process are succesfully completed without any error, but I
> don't
> > >> see the fragments on the wire and PMD stat function also reports that
> no
> > >> packets were transmitted.
>
> Did you account for the fact that some of these might be indirect mbuf's
> where refcount > 1? If refcount is not 1 then mbuf is shared, and you can
> not safely write your L2 header directly.
>

No, I was following the ip_reassembly example.

Since rte_ipv4_fragment_packet() function requires as input parameter an
mbuf without L2 header,
it produces fragments without L2 too, so L2 header should be added. That is
exactly what the example application does.

And I've just checked the mbufs that are created by
the rte_ipv4_fragment_packet(). The are direct mbuf. They have non
zero buf_len size.

Also I checked the sources of rte_ipv4_fragment_packet() function. For each
fragment it creates direct mbuf,
then reserves space for the IP header in it and then adds one or more
indirect mbuf to it. Therefore it should be safe to prepend
L2 header to each resulting fragment, shouldn't be?

-- 
--
Kiselev Alexander

Re: [dpdk-users] ipv4 fragmentation

[Александр Киселев]

>
>
>> No, I was following the ip_reassembly example.
>
I am sorry, I was following the ip_fragmentation example.

Re: [dpdk-users] ipv4 fragmentation

[Александр Киселев]

I've switched from a 10G nic (IXGBE)  to a 1G nic (IGB driver) and now my
test works fine.

Have I found a bug in the ixgbe driver?

It's very strange that rte_eth_tx_burst function reports that packets have
been sent but there are no packets on the wire
and PMD stat fuctions reports that no packes have been sent.

P.S.
Should I post this on the dev list?

-- 
--
Kiselev Alexander