[dpdk-users] IP fragmentation chunk getting dropped on egress

[Steve Banville <steve.banville@oracle.com>]

Hi,

 

I'm seeing an issue with DPDK and IP frag where the first chunk is getting dropped on egress. I'm using the igb_e1000 driver with DPDK v18.x. My scenario is that I'm allocating a new buffer to store an outer L3 tunnel header, I clone the original received buffer which is a little over 3K, I link the newly cloned buffer to the allocated buffers nextptr and copy the meta data from the original packet buffer to the new allocated buffer. After calling   rte_ipv4_fragment_packet, I notice that the number of chunks created form the jumbo packet seems correct. What's strange is that the 1st fragmented chunk has a segment with a data_len = 0. All subsequent chunks appear to be fine. I've ran rte_mbuf_sanity_check against my new packet which succeeds. After adjusting the packet for L2 and adding the Ethernet header, I send the packet out but only chunks 1 + 2 are received off the wire. The first chunk (0) is never received and the NICs eth stats reflect this as well.

 

 

I'm hoping someone could shed some light on this issue and provide a solution.

 

Thanks,

 

Steve

 

Code example:

 

                  copybuf = rte_pktmbuf_alloc(getMbufPool());

        // Add L3 tunnel header

        Clone_buffer = rte_pktmbuf_clone(pkt_info->mblk, getClonePool());

                

        

        // Link the outer trace header

        copybuf->next = clone_buffer;

        copybuf->pkt_len += clone_buffer->pkt_len;           // update overall packet length

        clone_buffer->pkt_len = clone_buffer->data_len;      // packet len only in first segment            

        copybuf->nb_segs = (uint8_t)(clone_buffer->nb_segs + 1);

        

        

        /* copy metadata from source packet */

        copybuf->vlan_tci = pkt_info->mblk->vlan_tci;

        copybuf->vlan_tci_outer = pkt_info->mblk->vlan_tci_outer;

        copybuf->port = pkt_info->mblk->port;

        copybuf->tx_offload = pkt_info->mblk->tx_offload;        

        copybuf->hash = pkt_info->mblk->hash;

        copybuf->ol_flags = pkt_info->mblk->ol_flags;

 

        rte_mbuf_sanity_check(copybuf, 1);

 

 

Rice# dump mbuf at 0x3100ae8e56c0, iova=42e8e5880, buf_len=2176

  pkt_len=3687, ol_flags=182, nb_segs=4, in_port=0

  segment at 0x3100ae8e56c0, data=0x3100ae8e5900, data_len=20

  Dump data at [0x3100ae8e5900], len=20

00000000: 45 00 0E 67 00 00 00 00 80 04 82 0B C0 A8 94 6E | E..g...........n

00000010: C0 A8 94 C8 |  |  |  |  |  |  |  |  |  |  |  |  | ....

  segment at 0x3100a6f319c0, data=0x3100a73b9b0e, data_len=1500

  Dump data at [0x3100a73b9b0e], len=80

00000000: 45 00 0E 53 27 22 00 00 40 11 C4 DE AC 10 94 0A | E..S'"..@.......

00000010: AC 10 94 6E 13 C4 13 C4 0E 3F 00 1F 49 4E 56 49 | ...n.....?..INVI

00000020: 54 45 20 73 69 70 3A 73 65 72 76 69 63 65 40 31 | TE sip:service@1

00000030: 37 32 2E 31 36 2E 31 34 38 2E 31 31 30 3A 35 30 | 72.16.148.110:50

00000040: 36 30 20 53 49 50 2F 32 2E 30 0D 0A 56 69 61 3A | 60 SIP/2.0..Via:

  segment at 0x3100a6f31900, data=0x3100a73b90a2, data_len=1480

  segment at 0x3100a6f31840, data=0x3100a73b8622, data_len=687

 

Chunk 0: l2 len 0 l3 len 20, pkt len 1514, data len 34, nsegs 3

dump mbuf at 0x3100ae8e4c40, iova=42e8e4e00, buf_len=2176

  pkt_len=1514, ol_flags=40000000000000, nb_segs=3, in_port=65535

  segment at 0x3100ae8e4c40, data=0x3100ae8e4e72, data_len=34

  Dump data at [0x3100ae8e4e72], len=34

00000000: 00 21 F6 69 02 A3 00 08 25 25 63 A4 08 00 45 00 | .!.i....%%c...E.

00000010: 05 DC FA CF 20 00 80 04 6F C6 C0 A8 94 6E C0 A8 | .... ...o....n..

00000020: 94 C8 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | ..

  segment at 0x3100ae8e41c0, data=0x3100ae8e5914, data_len=0

  segment at 0x3100ae8e3740, data=0x3100a73b9b0e, data_len=1480

  Dump data at [0x3100a73b9b0e], len=66

00000000: 45 00 0E 53 27 22 00 00 40 11 C4 DE AC 10 94 0A | E..S'"..@.......

00000010: AC 10 94 6E 13 C4 13 C4 0E 3F 00 1F 49 4E 56 49 | ...n.....?..INVI

00000020: 54 45 20 73 69 70 3A 73 65 72 76 69 63 65 40 31 | TE sip:service@1

00000030: 37 32 2E 31 36 2E 31 34 38 2E 31 31 30 3A 35 30 | 72.16.148.110:50

00000040: 36 30 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | 60

Sending packet: txQ 1, L2 len 14, L3 len 20, protocol 4, flags 0 mtu 1500 egress intf 537919488

 

Chunk 1: l2 len 0 l3 len 20, pkt len 1514, data len 34, nsegs 3

dump mbuf at 0x3100ae8e2cc0, iova=42e8e2e80, buf_len=2176

  pkt_len=1514, ol_flags=40000000000000, nb_segs=3, in_port=65535

  segment at 0x3100ae8e2cc0, data=0x3100ae8e2ef2, data_len=34

  Dump data at [0x3100ae8e2ef2], len=34

00000000: 00 21 F6 69 02 A3 00 08 25 25 63 A4 08 00 45 00 | .!.i....%%c...E.

00000010: 05 DC FA CF 20 B9 80 04 6F 0D C0 A8 94 6E C0 A8 | .... ...o....n..

00000020: 94 C8 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | ..

  segment at 0x3100ae8e2240, data=0x3100a73ba0d6, data_len=20

  Dump data at [0x3100a73ba0d6], len=20

00000000: 37 32 2E 31 36 2E 31 34 38 2E 31 30 3A 35 30 36 | 72.16.148.10:506

00000010: 30 3E 3B 72 |  |  |  |  |  |  |  |  |  |  |  |  | 0>;r

  segment at 0x3100ae8e17c0, data=0x3100a73b90a2, data_len=1460

  Dump data at [0x3100a73b90a2], len=46

00000000: 65 61 73 6F 6E 3D 74 69 6D 65 2D 6F 66 2D 64 61 | eason=time-of-da

00000010: 79 0D 0A 44 69 76 65 72 73 69 6F 6E 3A 20 3C 73 | y..Diversion: <s

00000020: 69 70 3A 73 69 70 70 30 30 30 31 38 40 31 |  |  | ip:sipp00018@1

Sending packet: txQ 1, L2 len 14, L3 len 20, protocol 4, flags 0 mtu 1500 egress intf 537919488

 

Chunk 2: l2 len 0 l3 len 20, pkt len 741, data len 34, nsegs 3

dump mbuf at 0x3100ae8e0d40, iova=42e8e0f00, buf_len=2176

  pkt_len=741, ol_flags=40000000000000, nb_segs=3, in_port=65535

  segment at 0x3100ae8e0d40, data=0x3100ae8e0f72, data_len=34

  Dump data at [0x3100ae8e0f72], len=34

00000000: 00 21 F6 69 02 A3 00 08 25 25 63 A4 08 00 45 00 | .!.i....%%c...E.

00000010: 02 D7 FA CF 01 72 80 04 91 59 C0 A8 94 6E C0 A8 | .....r...Y...n..

00000020: 94 C8 |  |  |  |  |  |  |  |  |  |  |  |  |  |  | ..

  segment at 0x3100ae8e02c0, data=0x3100a73b9656, data_len=20

  Dump data at [0x3100a73b9656], len=20

00000000: 6D 65 2D 6F 66 2D 64 61 79 0D 0A 44 69 76 65 72 | me-of-day..Diver

00000010: 73 69 6F 6E |  |  |  |  |  |  |  |  |  |  |  |  | sion

  segment at 0x3100ae8df840, data=0x3100a73b8622, data_len=687

  Dump data at [0x3100a73b8622], len=46

00000000: 3A 20 3C 73 69 70 3A 73 69 70 70 30 30 30 34 30 | : <sip:sipp00040

00000010: 40 31 37 32 2E 31 36 2E 31 34 38 2E 31 30 3A 35 | @172.16.148.10:5

00000020: 30 36 30 3E 3B 72 65 61 73 6F 6E 3D 74 69 |  |  | 060>;reason=ti

Sending packet: txQ 1, L2 len 14, L3 len 20, protocol 4, flags 0 mtu 1500 egress intf 537919488

 

 

 



Stephen Banville | Consulting Software Engineer

Phone: +1 781-538-7670

100 Crosby Drive | Bedford, MA 01730

CGBU DEV - Platform Engineering

https://oracle.zoom.us/my/steve.banville

  _____