Re: [dpdk-dev] phantom old packets received in new mbuf

[Helmut Sim <simhelmut@gmail.com>]

Hi,

I still can't get over this issue and need some help here.

in order to investigate this issue i wrote a very simple app that do:
        rte_eth_rx_burst(port[i].portid, 0, pkts_burst, MAX_PKT_BURST)
        printf the udp pkt content
        rte_pktmbuf_free(pkts_burst[j]);

on my host i wrote a simple udp app:
        for i=0 to n, do
            sprintf(buf, "PKT %d", i++);
            send buf over udp (i tried several lengths)
            sleep(i have tried delay values from 0 to 1000 us)
        end

then i see that from time to time i get a packet that was already received
earlier instead of receiving the expected packet.
the output is:
PKT 12201
PKT 12202
PKT 12203
PKT 12204
PKT 12205
PKT 12206
PKT 12202
PKT 12208

total received pkts is identical to the total delivered pkts, but part of
the pkts are being received twice...
the testpmd application doens't give me tools to check it as there are no
content aware tests.

as smaller is the delay between pkts the probability to see this behaviour
is higher.
I've tried this with other NIC as well (82575) but got similar behavior.
I've tried it over my desktop as well as over a VM on my XEON, same
behaviour...

i've printed the pktmbuf pointer values and found that the duplicated pkts
use different pktmbufs.

My question is what tools do I have to start investigating it.

I've played with the rte_eth_rxconf possible values but it didn't help.

Since it looks like a basic issue I'm pretty sure it is a kind of
configuration problem.

any idea?


Thanks in advance,




On Sun, Jul 6, 2014 at 6:19 PM, Helmut Sim <simhelmut@gmail.com> wrote:

> Hi All,
>
> I face a very strange behavior.
> I do:
> rte_eth_rx_burst(port[i].portid, 0, pkts_burst, MAX_PKT_BURST)
>
> then I reassemble the received UDP segments into one packet (i used the
> ip_reassembly example), manipulate its content and send it back to the
> network after fragmenting the packet.
> This works great, ...well only most of the time it works great...
>
> sometimes it happens that a packet is reassembled and retransmitted to the
> network, and after a while I receive one of its segment instead of a a
> segment of a totally different packet...
>
> i.e.
> rx seg 1 of pkt.id x
> rx seg 2 of pkt.id x
> rx seg 3 of pkt.id x
> rx seg 4 of pkt.id x
>
> packet is reassembled and sent to the network.
>
> then i get:
> rx seg 1 of pkt.id y
> rx seg 2 of pkt.id y
> rx seg 3 of pkt.id y
> rx seg 2 of pkt.id x
>
> ....
>
> it looks like the pmd gives me a mbuf that points to an old  freed buffer
> instead of a new allocated buffer with the new segment data.
> or alternatively, the pmd allocates the buffer but doesn't copy the rx
> data into it...
>
> the input packets were checked with a wireshark and it fits my description
> above.
>
> I checked it with different level of delay between the packets, but it
> still happens.
>
> I am using dpdk 1.6.0 r2.
> specifically running over an ubuntu 14.04 server on a xen hypervisor using
> the NIC 82576 as pci passthrough mode.
> using one single port and one single queue.
>
> obviously I suspect that the problem is at my side, but couldn't find it,
> ...yet.
>
> Any idea or known issue?
>
>  Thanks
> Sim
>