Regarding multi-segmented mbufs

Regarding multi-segmented mbufs

[fwefew 4t4tg]

The DPDK documentation
https://doc.dpdk.org/guides/prog_guide/mbuf_lib.html provides helpful
information but comes up a tad short re: multi-segmented mbufs. See
Fig. 14.3 An mbuf with Three Segments.

See https://doc.dpdk.org/api/structrte__mbuf.html for mbuf struct
field description.

Consider a 16Kb message which can fit into 16 packets (to keep the
numbers nice here). Now one way to play to game TX side is:

1. Allocate 16 mbufs from a mempool
2. Chain mbuf 0 to mbuf 1 by updating mbuf0's m->ptr.next pointer as per diagram
3. Chain mbuf 1 to mbuf 2 by updating mbuf1's m->ptr.next pointer as per diagram
4. etc.;
5. Update ONLY mbuf 0's nb_seg to 16 so DPDK has some idea there's 16
related packets
6. Write headers, payloads into each of the 16 mbuf's data room
7. Transmit

However, the doc misses at least two points:

1. The diagram mentions 'm->pkt.next'. However mbuf does not have a
pkt field. It does have a 'next' field described as "Next segment of
scattered packet. This field is valid when physical address field is
undefined." Should the diagram read 'm->next
2. Once I get the 16 mbufs linked together, can I send them one at a
time, or will DPDK make me burst TX all 16 in one shot? Note, the TX
ring may not have capacity for all 16 the first time code tries to
send them.

I'm guessing that, more or less, the next pointers are handy for app
devs because it records which mbufs are related to a larger overall
message while, on the other hand, DPDK doesn't care. As far as DPDK is
concerned, I can burst send all 16, transmit 1 at a time, or heck,
even send them in reverse order one at a time. But then I don't know
why DPDK cares about nb_seg.

Or perhaps the next pointers, nb_seg cary through on TX, and are
intended to help code RX side. For example, once the RX code sees a
packet with nb_seg = 16, it'll know there are 15 more related packets.

Re: Regarding multi-segmented mbufs

[Stephen Hemminger]

On Wed, 3 Jul 2024 14:15:58 -0400
fwefew 4t4tg <7532yahoo@gmail.com> wrote:

> The DPDK documentation
> https://doc.dpdk.org/guides/prog_guide/mbuf_lib.html provides helpful
> information but comes up a tad short re: multi-segmented mbufs. See
> Fig. 14.3 An mbuf with Three Segments.
> 
> See https://doc.dpdk.org/api/structrte__mbuf.html for mbuf struct
> field description.
> 
> Consider a 16Kb message which can fit into 16 packets (to keep the
> numbers nice here). Now one way to play to game TX side is:
> 
> 1. Allocate 16 mbufs from a mempool
> 2. Chain mbuf 0 to mbuf 1 by updating mbuf0's m->ptr.next pointer as per diagram
> 3. Chain mbuf 1 to mbuf 2 by updating mbuf1's m->ptr.next pointer as per diagram
> 4. etc.;
> 5. Update ONLY mbuf 0's nb_seg to 16 so DPDK has some idea there's 16
> related packets
> 6. Write headers, payloads into each of the 16 mbuf's data room
> 7. Transmit
> 
> However, the doc misses at least two points:
> 
> 1. The diagram mentions 'm->pkt.next'. However mbuf does not have a
> pkt field. It does have a 'next' field described as "Next segment of
> scattered packet. This field is valid when physical address field is
> undefined." Should the diagram read 'm->next

The next field used to be in pkt.next, got changed many releases ago;
this is just a documentation bug.


> 2. Once I get the 16 mbufs linked together, can I send them one at a
> time, or will DPDK make me burst TX all 16 in one shot? Note, the TX
> ring may not have capacity for all 16 the first time code tries to
> send them.

A mbuf represents a single packet, a multi-segment mbuf still represents
a single packet on the network. If the HW TX ring does not have space
for that many segments, then tx_burst should return that the packet
can not be sent. For a burst with only a single packet that would be
zero. Also tx_burst preserves packet order so if a packet could
not fit, then that ends the burst.

> 
> I'm guessing that, more or less, the next pointers are handy for app
> devs because it records which mbufs are related to a larger overall
> message while, on the other hand, DPDK doesn't care. As far as DPDK is
> concerned, I can burst send all 16, transmit 1 at a time, or heck,
> even send them in reverse order one at a time. But then I don't know
> why DPDK cares about nb_seg.

No, next pointers are not for apps handling bursts of packets.
If the application needs to keep track of a chain of packets
then it needs to define its own pointer (using dynamic field)
or have meta data wrapper. It is also worth noting that linked
lists are slower than arrays.

> 
> Or perhaps the next pointers, nb_seg cary through on TX, and are
> intended to help code RX side. For example, once the RX code sees a
> packet with nb_seg = 16, it'll know there are 15 more related packets.