From: Dmitry Vyal <dmitryvyal@gmail.com>
To: "dev@dpdk.org" <dev@dpdk.org>
Subject: [dpdk-dev] DPDK delaying individual packets infinitely
Date: Fri, 06 Dec 2013 17:22:37 +0400 [thread overview]
Message-ID: <52A1CF9D.30708@gmail.com> (raw)
Hello list,
For some time I've been writing a custom packet generator coupled with a
packet receiver using DPDK.
It works greatly when used for generating millions of packets, but
unexpectedly It has troubles generating a several dozens of packets or so.
The application consists of two threads, first one sends the packets
from one port and second receives them on another. These are 2 ports of
four-ports 1Gb NIC. It's detected as Intel Corporation 82576 Gigabit
Network Connection (rev 01). Ports are connected with a patch cord.
My experiment runs like the following:
After initializing NICs both generator and receiver wait for 1 second.
Generator sends N packets by calling rte_eth_tx_burst for each
individual packet. It waits for 100000000 cpu ticks between bursts.
rte_eth_tx_burst reports all packets are sent.
Receiver repeats calling rte_eth_rx_burst and waits for 50000000 ticks
if the function returns zero. After generators sends all the packets,
the receiver continues polling for several seconds.
I'm observing the following behavior:
If N is small, say 20, than no packets a received. See the logs.
Receiver prints Z when it got zero packets and R if it got at least one
packet.
Starting experiment
receiver 0 started main loop
generator 0 sitting on socket 0 is waiting for experiment start
generator 0 started main loop
Zgenerating mbuf for file 0, port 0, addr 0 on socket 0
free_count on pool 0 = 100000000
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 1 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 2 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 3 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 4 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 5 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 6 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 7 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 8 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 9 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 10 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 11 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 12 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 13 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 14 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 15 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 16 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 17 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 18 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 19 on socket 0
sent 1 in queue 0 freed 0
Stopping after reaching 20 packets limit
ZZwaiting for receivers to
stopZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ*********
Statistics:
Seconds elapsed: 6.123671
If I made N bigger, say 25, than all of a sudden some packets a
received, like so:
Starting experiment
receiver 0 started main loop
generator 0 sitting on socket 0 is waiting for experiment start
generator 0 started main loop
generating mbuf for file 0, port 0, addr 0 on socket 0
Zfree_count on pool 0 = 100000000
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 1 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 2 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 3 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 4 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 5 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 6 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 7 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 8 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 9 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 10 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 11 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 12 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 13 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 14 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 15 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 16 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 17 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 18 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 19 on socket 0
sent 1 in queue 0 freed 0
ZZgenerating mbuf for file 0, port 0, addr 20 on socket 0
sent 1 in queue 0 freed 0
Rtotal received packets on queue 0: 21; received: 21; zero_iters: 41
Zgenerating mbuf for file 0, port 0, addr 21 on socket 0
free_count on pool 0 = 6113
sent 1 in queue 0 freed 0
Rtotal received packets on queue 0: 22; received: 1; zero_iters: 1
Zgenerating mbuf for file 0, port 0, addr 22 on socket 0
sent 1 in queue 0 freed 0
Rtotal received packets on queue 0: 23; received: 1; zero_iters: 1
Zgenerating mbuf for file 0, port 0, addr 23 on socket 0
sent 1 in queue 0 freed 0
Rtotal received packets on queue 0: 24; received: 1; zero_iters: 1
Zgenerating mbuf for file 0, port 0, addr 24 on socket 0
sent 1 in queue 0 freed 0
Stopping after reaching 25 packets limit
Rtotal received packets on queue 0: 25; received: 1; zero_iters: 1
Zwaiting for receivers to
stopZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ*********
Statistics:
Seconds elapsed: 6.293315
So looks like DPDK is buffering first 21 packets and doesn't flush the
buffer in the first run even if I wait for several seconds after last
packet is sent. In second run we see it forwards packets one by one
after first burst of size 21.
I tried on DPDK-1.3.1 and DPDK-1.5.1 observing similar results. Is it
expected behavior?
next reply other threads:[~2013-12-06 13:21 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-12-06 13:22 Dmitry Vyal [this message]
2014-01-02 13:59 ` Thomas Monjalon
2014-01-02 16:43 ` Stephen Hemminger
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52A1CF9D.30708@gmail.com \
--to=dmitryvyal@gmail.com \
--cc=dev@dpdk.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).