[dpdk-dev] packet loss: multi-queue (RSS enabled)

[Hamid Ramazani <mrramazani@gmail.com>]

Hi,
I tried a lot (more than a week) to solve the problem myself and not
to bother the list, but I didn't succeed. Maybe other people have the
same problem.

I have a simple program attached. It is intended for simple packet
capturing; captures from interface and writes to memory, and frees the
memory in the next loop.

I have a 10G 82599EB Intel SFI/SFP+ network interface for capturing packets.
As you may know, this network card supports up to 128 RSS queues.

This is just a test, so the packets being sent at 820Kpps (kilo packet
per second). Each packet is 1500B (fixed size); it is 9.16 Gbit per
second.
Of course when the packet per seconds goes up and packet size goes
down (e.g. 400B per packet), it gets much worse.

When using one queue to receive, I receive all the packets, with no loss.
When I use more than one queue (e.g. 8 queues), with each thread
running on a dedicated core, I have a considerable amount of loss.

Please note that:
1. The computer has 12 * 2.67GHz cores, and it does nothing else but
capturing packets. The CPU is Intel Xeon X5650.
2. The operating system is Ubuntu 12.04.3 LTS

Attached file includes:
main.h
main.c
Makefile
./run.sh

It is configured to be run with 8 queues.

If you want to change the number of receive queues, please:
1. in main.c, change the value assigned to nb_rx_q_of_dev to the desired value.
2. change core mask in run.sh file (since there is SKIP_MASTER, you
should give a core containing one more CPU than given number of
queues).

I think there might be following problems:
1. the port configuration is not fine.
2. freeing memory has a considerable amount of overhead, and may be I
shouldn't do that. But If I don't the pool will be full, won't be? Is
there any other way?

Please help.
Thanks a lot in advance for your help and comments.

All the Best,
Hamid