DPDK usage discussions
 help / color / mirror / Atom feed
* [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
@ 2020-07-01 10:23 David Aldrich
  2020-07-01 12:57 ` Cliff Burdick
  0 siblings, 1 reply; 11+ messages in thread
From: David Aldrich @ 2020-07-01 10:23 UTC (permalink / raw)
  To: users

 Hi,

I have a problem transmitting udp packets with dpdk-stable-18.11.8. I have
posted a question about it on stackoverflow:

https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload

I would be grateful for any comments either there or in this group please.

Best regards

David

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 10:23 [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload? David Aldrich
@ 2020-07-01 12:57 ` Cliff Burdick
  2020-07-01 13:04   ` David Aldrich
  2020-07-01 13:40   ` David Aldrich
  0 siblings, 2 replies; 11+ messages in thread
From: Cliff Burdick @ 2020-07-01 12:57 UTC (permalink / raw)
  To: David Aldrich; +Cc: users

Are you setting data_len and packet_len in the mbuf before sending?


On Wed, Jul 1, 2020, 03:23 David Aldrich <david.aldrich.ntml@gmail.com>
wrote:

>  Hi,
>
> I have a problem transmitting udp packets with dpdk-stable-18.11.8. I have
> posted a question about it on stackoverflow:
>
>
> https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload
>
> I would be grateful for any comments either there or in this group please.
>
> Best regards
>
> David
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 12:57 ` Cliff Burdick
@ 2020-07-01 13:04   ` David Aldrich
  2020-07-01 13:08     ` Cliff Burdick
  2020-07-01 13:40   ` David Aldrich
  1 sibling, 1 reply; 11+ messages in thread
From: David Aldrich @ 2020-07-01 13:04 UTC (permalink / raw)
  To: Cliff Burdick; +Cc: users

>
> > Are you setting data_len and packet_len in the mbuf before sending?
>

 Yes, and I believe that the mbuf dump and the tcpdump that I included show
that these lengths are set correctly.  The problem is that udp payload is
present but consists of all 0's.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 13:04   ` David Aldrich
@ 2020-07-01 13:08     ` Cliff Burdick
  0 siblings, 0 replies; 11+ messages in thread
From: Cliff Burdick @ 2020-07-01 13:08 UTC (permalink / raw)
  To: David Aldrich; +Cc: users

Sorry, I didn't see those below, but it looks incorrect. Those numbers
should match if you're using a single segment. My guess is the 42 is
truncating it to the size of the udp header part only.


On Wed, Jul 1, 2020, 06:04 David Aldrich <david.aldrich.ntml@gmail.com>
wrote:

> > Are you setting data_len and packet_len in the mbuf before sending?
>>
>
>  Yes, and I believe that the mbuf dump and the tcpdump that I included
> show that these lengths are set correctly.  The problem is that udp payload
> is present but consists of all 0's.
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 12:57 ` Cliff Burdick
  2020-07-01 13:04   ` David Aldrich
@ 2020-07-01 13:40   ` David Aldrich
  2020-07-01 14:59     ` Cliff Burdick
  1 sibling, 1 reply; 11+ messages in thread
From: David Aldrich @ 2020-07-01 13:40 UTC (permalink / raw)
  To: Cliff Burdick; +Cc: users

I think this is where my lack of understanding is my problem.  Looking at
the mbuf dump:

dump mbuf at 0x53e4e92c0, iova=73e4e9340, buf_len=2176
  pkt_len=57, ol_flags=c0000000000000, nb_segs=2, in_port=65535
  segment at 0x53e4e92c0, data=0x53e4e9396, data_len=42
  Dump data at [0x53e4e9396], len=42
00000000: 94 40 C9 1F C4 DB 94 40 C9 1E 13 7D 08 00 45 B8 | .@.....@...}..E.
00000010: 00 2B 00 00 00 00 40 11 F5 E8 C0 A8 01 69 C0 A8 | .+....@......i..
00000020: 01 68 0B B9 0B B8 00 17 64 2D |  |  |  |  |  |  | .h......d-
  segment at 0x53e146e40, data=0x53e4fbbc0, data_len=15
  Dump data at [0x53e4fbbc0], len=15
00000000: 48 65 6C 6C 6F 20 66 72 6F 6D 20 64 70 64 6B |  | Hello from dpdk

It consists of two parts (segments?), of lengths 42 and 15. This makes
sense to me as I first
put the payload in the mbuf (15 bytes) and then added the Ethernet and
L3 headers (42 bytes) by calling rte_pktmbuf_prepend().

I guess only the first segment is getting transmitted?


On Wed, Jul 1, 2020 at 1:57 PM Cliff Burdick <shaklee3@gmail.com> wrote:

> Are you setting data_len and packet_len in the mbuf before sending?
>
>
> On Wed, Jul 1, 2020, 03:23 David Aldrich <david.aldrich.ntml@gmail.com>
> wrote:
>
>>  Hi,
>>
>> I have a problem transmitting udp packets with dpdk-stable-18.11.8. I have
>> posted a question about it on stackoverflow:
>>
>>
>> https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload
>>
>> I would be grateful for any comments either there or in this group please.
>>
>> Best regards
>>
>> David
>>
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 13:40   ` David Aldrich
@ 2020-07-01 14:59     ` Cliff Burdick
  2020-07-01 15:06       ` David Aldrich
  0 siblings, 1 reply; 11+ messages in thread
From: Cliff Burdick @ 2020-07-01 14:59 UTC (permalink / raw)
  To: David Aldrich; +Cc: users

If you look at the code for  rte_pktmbuf_prepend  it appears to be just
incrementing data_len and pkt_len by the same amount. My guess is that
those fields were not set to the correct values before you called
rte_pktmbuf_prepend().

On Wed, Jul 1, 2020 at 6:41 AM David Aldrich <david.aldrich.ntml@gmail.com>
wrote:

> I think this is where my lack of understanding is my problem.  Looking at
> the mbuf dump:
>
> dump mbuf at 0x53e4e92c0, iova=73e4e9340, buf_len=2176
>   pkt_len=57, ol_flags=c0000000000000, nb_segs=2, in_port=65535
>   segment at 0x53e4e92c0, data=0x53e4e9396, data_len=42
>   Dump data at [0x53e4e9396], len=42
> 00000000: 94 40 C9 1F C4 DB 94 40 C9 1E 13 7D 08 00 45 B8 | .@.....@...}..E.
> 00000010: 00 2B 00 00 00 00 40 11 F5 E8 C0 A8 01 69 C0 A8 | .+....@......i..
> 00000020: 01 68 0B B9 0B B8 00 17 64 2D |  |  |  |  |  |  | .h......d-
>   segment at 0x53e146e40, data=0x53e4fbbc0, data_len=15
>   Dump data at [0x53e4fbbc0], len=15
> 00000000: 48 65 6C 6C 6F 20 66 72 6F 6D 20 64 70 64 6B |  | Hello from dpdk
>
> It consists of two parts (segments?), of lengths 42 and 15. This makes sense to me as I first
> put the payload in the mbuf (15 bytes) and then added the Ethernet and L3 headers (42 bytes) by calling rte_pktmbuf_prepend().
>
> I guess only the first segment is getting transmitted?
>
>
> On Wed, Jul 1, 2020 at 1:57 PM Cliff Burdick <shaklee3@gmail.com> wrote:
>
>> Are you setting data_len and packet_len in the mbuf before sending?
>>
>>
>> On Wed, Jul 1, 2020, 03:23 David Aldrich <david.aldrich.ntml@gmail.com>
>> wrote:
>>
>>>  Hi,
>>>
>>> I have a problem transmitting udp packets with dpdk-stable-18.11.8. I
>>> have
>>> posted a question about it on stackoverflow:
>>>
>>>
>>> https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload
>>>
>>> I would be grateful for any comments either there or in this group
>>> please.
>>>
>>> Best regards
>>>
>>> David
>>>
>>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 14:59     ` Cliff Burdick
@ 2020-07-01 15:06       ` David Aldrich
  2020-07-01 15:38         ` Cliff Burdick
  2020-07-01 15:56         ` Matt Laswell
  0 siblings, 2 replies; 11+ messages in thread
From: David Aldrich @ 2020-07-01 15:06 UTC (permalink / raw)
  To: Cliff Burdick; +Cc: users

>If you look at the code for  rte_pktmbuf_prepend  it appears to be just
>incrementing data_len and pkt_len by the same amount.

I'm sorry, but I don't understand. My calls to rte_pktmbuf_prepend are:


p_udp_hdr = (struct udp_hdr*)rte_pktmbuf_prepend(ptMbuf,
(uint16_t)sizeof(struct udp_hdr));
p_ip_hdr  = (struct ipv4_hdr*)rte_pktmbuf_prepend(ptMbuf,
(uint16_t)sizeof(struct ipv4_hdr));

are you saying that those calls are wrong?  When you say 'if you look
at the code' are you referring to this code?

(I thought it best that I should ask for clarification rather than guess).

Thanks


On Wed, Jul 1, 2020 at 3:59 PM Cliff Burdick <shaklee3@gmail.com> wrote:

> If you look at the code for  rte_pktmbuf_prepend  it appears to be just
> incrementing data_len and pkt_len by the same amount. My guess is that
> those fields were not set to the correct values before you called
> rte_pktmbuf_prepend().
>
> On Wed, Jul 1, 2020 at 6:41 AM David Aldrich <david.aldrich.ntml@gmail.com>
> wrote:
>
>> I think this is where my lack of understanding is my problem.  Looking at
>> the mbuf dump:
>>
>> dump mbuf at 0x53e4e92c0, iova=73e4e9340, buf_len=2176
>>   pkt_len=57, ol_flags=c0000000000000, nb_segs=2, in_port=65535
>>   segment at 0x53e4e92c0, data=0x53e4e9396, data_len=42
>>   Dump data at [0x53e4e9396], len=42
>> 00000000: 94 40 C9 1F C4 DB 94 40 C9 1E 13 7D 08 00 45 B8 | .@.....@...}..E.
>> 00000010: 00 2B 00 00 00 00 40 11 F5 E8 C0 A8 01 69 C0 A8 | .+....@......i..
>> 00000020: 01 68 0B B9 0B B8 00 17 64 2D |  |  |  |  |  |  | .h......d-
>>   segment at 0x53e146e40, data=0x53e4fbbc0, data_len=15
>>   Dump data at [0x53e4fbbc0], len=15
>> 00000000: 48 65 6C 6C 6F 20 66 72 6F 6D 20 64 70 64 6B |  | Hello from dpdk
>>
>> It consists of two parts (segments?), of lengths 42 and 15. This makes sense to me as I first
>> put the payload in the mbuf (15 bytes) and then added the Ethernet and L3 headers (42 bytes) by calling rte_pktmbuf_prepend().
>>
>> I guess only the first segment is getting transmitted?
>>
>>
>> On Wed, Jul 1, 2020 at 1:57 PM Cliff Burdick <shaklee3@gmail.com> wrote:
>>
>>> Are you setting data_len and packet_len in the mbuf before sending?
>>>
>>>
>>> On Wed, Jul 1, 2020, 03:23 David Aldrich <david.aldrich.ntml@gmail.com>
>>> wrote:
>>>
>>>>  Hi,
>>>>
>>>> I have a problem transmitting udp packets with dpdk-stable-18.11.8. I
>>>> have
>>>> posted a question about it on stackoverflow:
>>>>
>>>>
>>>> https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload
>>>>
>>>> I would be grateful for any comments either there or in this group
>>>> please.
>>>>
>>>> Best regards
>>>>
>>>> David
>>>>
>>>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 15:06       ` David Aldrich
@ 2020-07-01 15:38         ` Cliff Burdick
  2020-07-01 15:56         ` Matt Laswell
  1 sibling, 0 replies; 11+ messages in thread
From: Cliff Burdick @ 2020-07-01 15:38 UTC (permalink / raw)
  To: David Aldrich; +Cc: users

How are you setting up ptMbuf before you call those? It seems that if you
had a payload beyond the UDP header you would need to set those
appropriately in the data/pkt_len fields prior to those calls.

On Wed, Jul 1, 2020 at 8:06 AM David Aldrich <david.aldrich.ntml@gmail.com>
wrote:

> >If you look at the code for  rte_pktmbuf_prepend  it appears to be just
> >incrementing data_len and pkt_len by the same amount.
>
> I'm sorry, but I don't understand. My calls to rte_pktmbuf_prepend are:
>
>
> p_udp_hdr = (struct udp_hdr*)rte_pktmbuf_prepend(ptMbuf, (uint16_t)sizeof(struct udp_hdr));
> p_ip_hdr  = (struct ipv4_hdr*)rte_pktmbuf_prepend(ptMbuf, (uint16_t)sizeof(struct ipv4_hdr));
>
> are you saying that those calls are wrong?  When you say 'if you look at the code' are you referring to this code?
>
> (I thought it best that I should ask for clarification rather than guess).
>
> Thanks
>
>
> On Wed, Jul 1, 2020 at 3:59 PM Cliff Burdick <shaklee3@gmail.com> wrote:
>
>> If you look at the code for  rte_pktmbuf_prepend  it appears to be just
>> incrementing data_len and pkt_len by the same amount. My guess is that
>> those fields were not set to the correct values before you called
>> rte_pktmbuf_prepend().
>>
>> On Wed, Jul 1, 2020 at 6:41 AM David Aldrich <
>> david.aldrich.ntml@gmail.com> wrote:
>>
>>> I think this is where my lack of understanding is my problem.  Looking
>>> at the mbuf dump:
>>>
>>> dump mbuf at 0x53e4e92c0, iova=73e4e9340, buf_len=2176
>>>   pkt_len=57, ol_flags=c0000000000000, nb_segs=2, in_port=65535
>>>   segment at 0x53e4e92c0, data=0x53e4e9396, data_len=42
>>>   Dump data at [0x53e4e9396], len=42
>>> 00000000: 94 40 C9 1F C4 DB 94 40 C9 1E 13 7D 08 00 45 B8 | .@.....@...}..E.
>>> 00000010: 00 2B 00 00 00 00 40 11 F5 E8 C0 A8 01 69 C0 A8 | .+....@......i..
>>> 00000020: 01 68 0B B9 0B B8 00 17 64 2D |  |  |  |  |  |  | .h......d-
>>>   segment at 0x53e146e40, data=0x53e4fbbc0, data_len=15
>>>   Dump data at [0x53e4fbbc0], len=15
>>> 00000000: 48 65 6C 6C 6F 20 66 72 6F 6D 20 64 70 64 6B |  | Hello from dpdk
>>>
>>> It consists of two parts (segments?), of lengths 42 and 15. This makes sense to me as I first
>>> put the payload in the mbuf (15 bytes) and then added the Ethernet and L3 headers (42 bytes) by calling rte_pktmbuf_prepend().
>>>
>>> I guess only the first segment is getting transmitted?
>>>
>>>
>>> On Wed, Jul 1, 2020 at 1:57 PM Cliff Burdick <shaklee3@gmail.com> wrote:
>>>
>>>> Are you setting data_len and packet_len in the mbuf before sending?
>>>>
>>>>
>>>> On Wed, Jul 1, 2020, 03:23 David Aldrich <david.aldrich.ntml@gmail.com>
>>>> wrote:
>>>>
>>>>>  Hi,
>>>>>
>>>>> I have a problem transmitting udp packets with dpdk-stable-18.11.8. I
>>>>> have
>>>>> posted a question about it on stackoverflow:
>>>>>
>>>>>
>>>>> https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload
>>>>>
>>>>> I would be grateful for any comments either there or in this group
>>>>> please.
>>>>>
>>>>> Best regards
>>>>>
>>>>> David
>>>>>
>>>>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 15:06       ` David Aldrich
  2020-07-01 15:38         ` Cliff Burdick
@ 2020-07-01 15:56         ` Matt Laswell
  2020-07-01 16:48           ` David Aldrich
  1 sibling, 1 reply; 11+ messages in thread
From: Matt Laswell @ 2020-07-01 15:56 UTC (permalink / raw)
  To: David Aldrich; +Cc: users

OK, this makes sense.  Understand that the mbuf data structure in DPDK
contains an offset that tells other consumers of the mbuf where to find
content.  When you call rte_pktmbuf_prepend(), you're adjusting this
offset, in this case moving it forward by 42 bytes.

Based on what I've seen you say here and on Stack Overflow, I think perhaps
your code is doing roughly this:
1. You have an MBuf that contains your payload
2. You create a second MBuf that contains your headers, and link it to the
payload mbuf via the ->next pointer and nb_segs
3. But you then call rte_pktmbuf_prepend() on the payload MBuf, making
space in the payload MBuf for headers.
4. You send the header payload, implicitly also sending the payload one
(since they're linked)

If that's an accurate description of what you're doing, step 3 is the
problem.  You're telling the transmitting NIC to start pulling 15 bytes of
payload from a spot that's 42 bytes ahead of where the payload actually
lives.  Hence, you get whatever happens to be in the buffer at that point.
In this case, it's zeroes.  It could be whatever garbage was lying around
from the last user of the MBuf.

Let me know if that's an accurate description of what your application is
doing, or if I've misunderstood.

On Wed, Jul 1, 2020 at 10:07 AM David Aldrich <david.aldrich.ntml@gmail.com>
wrote:

> >If you look at the code for  rte_pktmbuf_prepend  it appears to be just
> >incrementing data_len and pkt_len by the same amount.
>
> I'm sorry, but I don't understand. My calls to rte_pktmbuf_prepend are:
>
>
> p_udp_hdr = (struct udp_hdr*)rte_pktmbuf_prepend(ptMbuf,
> (uint16_t)sizeof(struct udp_hdr));
> p_ip_hdr  = (struct ipv4_hdr*)rte_pktmbuf_prepend(ptMbuf,
> (uint16_t)sizeof(struct ipv4_hdr));
>
> are you saying that those calls are wrong?  When you say 'if you look
> at the code' are you referring to this code?
>
> (I thought it best that I should ask for clarification rather than guess).
>
> Thanks
>
>
> On Wed, Jul 1, 2020 at 3:59 PM Cliff Burdick <shaklee3@gmail.com> wrote:
>
> > If you look at the code for  rte_pktmbuf_prepend  it appears to be just
> > incrementing data_len and pkt_len by the same amount. My guess is that
> > those fields were not set to the correct values before you called
> > rte_pktmbuf_prepend().
> >
> > On Wed, Jul 1, 2020 at 6:41 AM David Aldrich <
> david.aldrich.ntml@gmail.com>
> > wrote:
> >
> >> I think this is where my lack of understanding is my problem.  Looking
> at
> >> the mbuf dump:
> >>
> >> dump mbuf at 0x53e4e92c0, iova=73e4e9340, buf_len=2176
> >>   pkt_len=57, ol_flags=c0000000000000, nb_segs=2, in_port=65535
> >>   segment at 0x53e4e92c0, data=0x53e4e9396, data_len=42
> >>   Dump data at [0x53e4e9396], len=42
> >> 00000000: 94 40 C9 1F C4 DB 94 40 C9 1E 13 7D 08 00 45 B8 |
> .@.....@...}..E.
> >> 00000010: 00 2B 00 00 00 00 40 11 F5 E8 C0 A8 01 69 C0 A8 |
> .+....@......i..
> >> 00000020: 01 68 0B B9 0B B8 00 17 64 2D |  |  |  |  |  |  | .h......d-
> >>   segment at 0x53e146e40, data=0x53e4fbbc0, data_len=15
> >>   Dump data at [0x53e4fbbc0], len=15
> >> 00000000: 48 65 6C 6C 6F 20 66 72 6F 6D 20 64 70 64 6B |  | Hello from
> dpdk
> >>
> >> It consists of two parts (segments?), of lengths 42 and 15. This makes
> sense to me as I first
> >> put the payload in the mbuf (15 bytes) and then added the Ethernet and
> L3 headers (42 bytes) by calling rte_pktmbuf_prepend().
> >>
> >> I guess only the first segment is getting transmitted?
> >>
> >>
> >> On Wed, Jul 1, 2020 at 1:57 PM Cliff Burdick <shaklee3@gmail.com>
> wrote:
> >>
> >>> Are you setting data_len and packet_len in the mbuf before sending?
> >>>
> >>>
> >>> On Wed, Jul 1, 2020, 03:23 David Aldrich <david.aldrich.ntml@gmail.com
> >
> >>> wrote:
> >>>
> >>>>  Hi,
> >>>>
> >>>> I have a problem transmitting udp packets with dpdk-stable-18.11.8. I
> >>>> have
> >>>> posted a question about it on stackoverflow:
> >>>>
> >>>>
> >>>>
> https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload
> >>>>
> >>>> I would be grateful for any comments either there or in this group
> >>>> please.
> >>>>
> >>>> Best regards
> >>>>
> >>>> David
> >>>>
> >>>
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 15:56         ` Matt Laswell
@ 2020-07-01 16:48           ` David Aldrich
  2020-07-01 17:57             ` Cliff Burdick
  0 siblings, 1 reply; 11+ messages in thread
From: David Aldrich @ 2020-07-01 16:48 UTC (permalink / raw)
  To: Matt Laswell; +Cc: users

Hi Matt
Thanks very much for your help. I was about to write out in more detail
what our code does but I've just noticed that it uses a zero-copy method to
attach a data buffer to the mbuf using a call to:

rte_pktmbuf_attach()

The docs says for this function: "Right now, not supported:"

So maybe it doesn't work. Anyway, I need to revisit this code and take the
simpler approach of just copying the payload into the mbuf. Maybe I can get
the zero copy method working later. I'll see how this goes and come back if
I need more help.

David

On Wed, Jul 1, 2020 at 4:56 PM Matt Laswell <laswell@infinite.io> wrote:

> OK, this makes sense.  Understand that the mbuf data structure in DPDK
> contains an offset that tells other consumers of the mbuf where to find
> content.  When you call rte_pktmbuf_prepend(), you're adjusting this
> offset, in this case moving it forward by 42 bytes.
>
> Based on what I've seen you say here and on Stack Overflow, I think
> perhaps your code is doing roughly this:
> 1. You have an MBuf that contains your payload
> 2. You create a second MBuf that contains your headers, and link it to the
> payload mbuf via the ->next pointer and nb_segs
> 3. But you then call rte_pktmbuf_prepend() on the payload MBuf, making
> space in the payload MBuf for headers.
> 4. You send the header payload, implicitly also sending the payload one
> (since they're linked)
>
> If that's an accurate description of what you're doing, step 3 is the
> problem.  You're telling the transmitting NIC to start pulling 15 bytes of
> payload from a spot that's 42 bytes ahead of where the payload actually
> lives.  Hence, you get whatever happens to be in the buffer at that point.
> In this case, it's zeroes.  It could be whatever garbage was lying around
> from the last user of the MBuf.
>
> Let me know if that's an accurate description of what your application is
> doing, or if I've misunderstood.
>
> On Wed, Jul 1, 2020 at 10:07 AM David Aldrich <
> david.aldrich.ntml@gmail.com> wrote:
>
>> >If you look at the code for  rte_pktmbuf_prepend  it appears to be just
>> >incrementing data_len and pkt_len by the same amount.
>>
>> I'm sorry, but I don't understand. My calls to rte_pktmbuf_prepend are:
>>
>>
>> p_udp_hdr = (struct udp_hdr*)rte_pktmbuf_prepend(ptMbuf,
>> (uint16_t)sizeof(struct udp_hdr));
>> p_ip_hdr  = (struct ipv4_hdr*)rte_pktmbuf_prepend(ptMbuf,
>> (uint16_t)sizeof(struct ipv4_hdr));
>>
>> are you saying that those calls are wrong?  When you say 'if you look
>> at the code' are you referring to this code?
>>
>> (I thought it best that I should ask for clarification rather than guess).
>>
>> Thanks
>>
>>
>> On Wed, Jul 1, 2020 at 3:59 PM Cliff Burdick <shaklee3@gmail.com> wrote:
>>
>> > If you look at the code for  rte_pktmbuf_prepend  it appears to be just
>> > incrementing data_len and pkt_len by the same amount. My guess is that
>> > those fields were not set to the correct values before you called
>> > rte_pktmbuf_prepend().
>> >
>> > On Wed, Jul 1, 2020 at 6:41 AM David Aldrich <
>> david.aldrich.ntml@gmail.com>
>> > wrote:
>> >
>> >> I think this is where my lack of understanding is my problem.  Looking
>> at
>> >> the mbuf dump:
>> >>
>> >> dump mbuf at 0x53e4e92c0, iova=73e4e9340, buf_len=2176
>> >>   pkt_len=57, ol_flags=c0000000000000, nb_segs=2, in_port=65535
>> >>   segment at 0x53e4e92c0, data=0x53e4e9396, data_len=42
>> >>   Dump data at [0x53e4e9396], len=42
>> >> 00000000: 94 40 C9 1F C4 DB 94 40 C9 1E 13 7D 08 00 45 B8 |
>> .@.....@...}..E.
>> >> 00000010: 00 2B 00 00 00 00 40 11 F5 E8 C0 A8 01 69 C0 A8 |
>> .+....@......i..
>> >> 00000020: 01 68 0B B9 0B B8 00 17 64 2D |  |  |  |  |  |  | .h......d-
>> >>   segment at 0x53e146e40, data=0x53e4fbbc0, data_len=15
>> >>   Dump data at [0x53e4fbbc0], len=15
>> >> 00000000: 48 65 6C 6C 6F 20 66 72 6F 6D 20 64 70 64 6B |  | Hello from
>> dpdk
>> >>
>> >> It consists of two parts (segments?), of lengths 42 and 15. This makes
>> sense to me as I first
>> >> put the payload in the mbuf (15 bytes) and then added the Ethernet and
>> L3 headers (42 bytes) by calling rte_pktmbuf_prepend().
>> >>
>> >> I guess only the first segment is getting transmitted?
>> >>
>> >>
>> >> On Wed, Jul 1, 2020 at 1:57 PM Cliff Burdick <shaklee3@gmail.com>
>> wrote:
>> >>
>> >>> Are you setting data_len and packet_len in the mbuf before sending?
>> >>>
>> >>>
>> >>> On Wed, Jul 1, 2020, 03:23 David Aldrich <
>> david.aldrich.ntml@gmail.com>
>> >>> wrote:
>> >>>
>> >>>>  Hi,
>> >>>>
>> >>>> I have a problem transmitting udp packets with dpdk-stable-18.11.8. I
>> >>>> have
>> >>>> posted a question about it on stackoverflow:
>> >>>>
>> >>>>
>> >>>>
>> https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload
>> >>>>
>> >>>> I would be grateful for any comments either there or in this group
>> >>>> please.
>> >>>>
>> >>>> Best regards
>> >>>>
>> >>>> David
>> >>>>
>> >>>
>>
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload?
  2020-07-01 16:48           ` David Aldrich
@ 2020-07-01 17:57             ` Cliff Burdick
  0 siblings, 0 replies; 11+ messages in thread
From: Cliff Burdick @ 2020-07-01 17:57 UTC (permalink / raw)
  To: David Aldrich; +Cc: Matt Laswell, users

Another option is to use rte_pktmbuf_chain(), and chain the buffers
together. That sounds more like what you were intending to do in the first
place. If you do that, make sure your driver supports it (mlx5 does), and
set the data/pkt_len appropriately for each part of the chain. This also is
zero-copy.

On Wed, Jul 1, 2020 at 9:48 AM David Aldrich <david.aldrich.ntml@gmail.com>
wrote:

> Hi Matt
> Thanks very much for your help. I was about to write out in more detail
> what our code does but I've just noticed that it uses a zero-copy method to
> attach a data buffer to the mbuf using a call to:
>
> rte_pktmbuf_attach()
>
> The docs says for this function: "Right now, not supported:"
>
> So maybe it doesn't work. Anyway, I need to revisit this code and take the
> simpler approach of just copying the payload into the mbuf. Maybe I can get
> the zero copy method working later. I'll see how this goes and come back if
> I need more help.
>
> David
>
> On Wed, Jul 1, 2020 at 4:56 PM Matt Laswell <laswell@infinite.io> wrote:
>
> > OK, this makes sense.  Understand that the mbuf data structure in DPDK
> > contains an offset that tells other consumers of the mbuf where to find
> > content.  When you call rte_pktmbuf_prepend(), you're adjusting this
> > offset, in this case moving it forward by 42 bytes.
> >
> > Based on what I've seen you say here and on Stack Overflow, I think
> > perhaps your code is doing roughly this:
> > 1. You have an MBuf that contains your payload
> > 2. You create a second MBuf that contains your headers, and link it to
> the
> > payload mbuf via the ->next pointer and nb_segs
> > 3. But you then call rte_pktmbuf_prepend() on the payload MBuf, making
> > space in the payload MBuf for headers.
> > 4. You send the header payload, implicitly also sending the payload one
> > (since they're linked)
> >
> > If that's an accurate description of what you're doing, step 3 is the
> > problem.  You're telling the transmitting NIC to start pulling 15 bytes
> of
> > payload from a spot that's 42 bytes ahead of where the payload actually
> > lives.  Hence, you get whatever happens to be in the buffer at that
> point.
> > In this case, it's zeroes.  It could be whatever garbage was lying around
> > from the last user of the MBuf.
> >
> > Let me know if that's an accurate description of what your application is
> > doing, or if I've misunderstood.
> >
> > On Wed, Jul 1, 2020 at 10:07 AM David Aldrich <
> > david.aldrich.ntml@gmail.com> wrote:
> >
> >> >If you look at the code for  rte_pktmbuf_prepend  it appears to be just
> >> >incrementing data_len and pkt_len by the same amount.
> >>
> >> I'm sorry, but I don't understand. My calls to rte_pktmbuf_prepend are:
> >>
> >>
> >> p_udp_hdr = (struct udp_hdr*)rte_pktmbuf_prepend(ptMbuf,
> >> (uint16_t)sizeof(struct udp_hdr));
> >> p_ip_hdr  = (struct ipv4_hdr*)rte_pktmbuf_prepend(ptMbuf,
> >> (uint16_t)sizeof(struct ipv4_hdr));
> >>
> >> are you saying that those calls are wrong?  When you say 'if you look
> >> at the code' are you referring to this code?
> >>
> >> (I thought it best that I should ask for clarification rather than
> guess).
> >>
> >> Thanks
> >>
> >>
> >> On Wed, Jul 1, 2020 at 3:59 PM Cliff Burdick <shaklee3@gmail.com>
> wrote:
> >>
> >> > If you look at the code for  rte_pktmbuf_prepend  it appears to be
> just
> >> > incrementing data_len and pkt_len by the same amount. My guess is that
> >> > those fields were not set to the correct values before you called
> >> > rte_pktmbuf_prepend().
> >> >
> >> > On Wed, Jul 1, 2020 at 6:41 AM David Aldrich <
> >> david.aldrich.ntml@gmail.com>
> >> > wrote:
> >> >
> >> >> I think this is where my lack of understanding is my problem.
> Looking
> >> at
> >> >> the mbuf dump:
> >> >>
> >> >> dump mbuf at 0x53e4e92c0, iova=73e4e9340, buf_len=2176
> >> >>   pkt_len=57, ol_flags=c0000000000000, nb_segs=2, in_port=65535
> >> >>   segment at 0x53e4e92c0, data=0x53e4e9396, data_len=42
> >> >>   Dump data at [0x53e4e9396], len=42
> >> >> 00000000: 94 40 C9 1F C4 DB 94 40 C9 1E 13 7D 08 00 45 B8 |
> >> .@.....@...}..E.
> >> >> 00000010: 00 2B 00 00 00 00 40 11 F5 E8 C0 A8 01 69 C0 A8 |
> >> .+....@......i..
> >> >> 00000020: 01 68 0B B9 0B B8 00 17 64 2D |  |  |  |  |  |  |
> .h......d-
> >> >>   segment at 0x53e146e40, data=0x53e4fbbc0, data_len=15
> >> >>   Dump data at [0x53e4fbbc0], len=15
> >> >> 00000000: 48 65 6C 6C 6F 20 66 72 6F 6D 20 64 70 64 6B |  | Hello
> from
> >> dpdk
> >> >>
> >> >> It consists of two parts (segments?), of lengths 42 and 15. This
> makes
> >> sense to me as I first
> >> >> put the payload in the mbuf (15 bytes) and then added the Ethernet
> and
> >> L3 headers (42 bytes) by calling rte_pktmbuf_prepend().
> >> >>
> >> >> I guess only the first segment is getting transmitted?
> >> >>
> >> >>
> >> >> On Wed, Jul 1, 2020 at 1:57 PM Cliff Burdick <shaklee3@gmail.com>
> >> wrote:
> >> >>
> >> >>> Are you setting data_len and packet_len in the mbuf before sending?
> >> >>>
> >> >>>
> >> >>> On Wed, Jul 1, 2020, 03:23 David Aldrich <
> >> david.aldrich.ntml@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>>>  Hi,
> >> >>>>
> >> >>>> I have a problem transmitting udp packets with
> dpdk-stable-18.11.8. I
> >> >>>> have
> >> >>>> posted a question about it on stackoverflow:
> >> >>>>
> >> >>>>
> >> >>>>
> >>
> https://stackoverflow.com/questions/62674596/why-is-udp-packet-sent-by-dpdk-received-with-zerod-payload
> >> >>>>
> >> >>>> I would be grateful for any comments either there or in this group
> >> >>>> please.
> >> >>>>
> >> >>>> Best regards
> >> >>>>
> >> >>>> David
> >> >>>>
> >> >>>
> >>
> >
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2020-07-01 17:57 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-01 10:23 [dpdk-users] Why is udp packet, sent by dpdk, received with zero'd payload? David Aldrich
2020-07-01 12:57 ` Cliff Burdick
2020-07-01 13:04   ` David Aldrich
2020-07-01 13:08     ` Cliff Burdick
2020-07-01 13:40   ` David Aldrich
2020-07-01 14:59     ` Cliff Burdick
2020-07-01 15:06       ` David Aldrich
2020-07-01 15:38         ` Cliff Burdick
2020-07-01 15:56         ` Matt Laswell
2020-07-01 16:48           ` David Aldrich
2020-07-01 17:57             ` Cliff Burdick

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).