From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) by dpdk.org (Postfix) with ESMTP id EE1A59DE for ; Fri, 6 Dec 2013 14:21:42 +0100 (CET) Received: by mail-la0-f52.google.com with SMTP id y1so271700lam.39 for ; Fri, 06 Dec 2013 05:22:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=WAOvpY1pAeaIJyyKRsgiIVqcBMot4azgNgc5ueQ+ag8=; b=etEUlCHWb15QmtndnN4WbG/DZu4leLolC+Lws8IHpL7bHTciPm7HtryQ8VTW1CY9iM q99BB8N+kPRRV/yPVXMPXeImhi9R16sk7IThCnU/QJemGVy7RNp8xXfRa+d3GMEotxKI nXMQyv324e6w4hKvsBjIYC3o8VXNVumrX6ZbHqrpo6Gxq53RijxmkuuBalZ1qzqafAWj UFW8oKGCo9L7u5KkSYLaNjabtlLVfzElmIiGG8ZV8yv+tqY+lyjE2BYbmnFQGP5zz2rM NtM9Sr6p8bexpbka3HcwqRprJF3cdHVItLJ+YCKTKXn9JhXQPtxCXlaZ1+eu2M32/weC 4UtQ== X-Received: by 10.112.135.165 with SMTP id pt5mr867379lbb.33.1386336165454; Fri, 06 Dec 2013 05:22:45 -0800 (PST) Received: from [192.168.15.116] ([87.236.23.254]) by mx.google.com with ESMTPSA id r10sm97866392lag.7.2013.12.06.05.22.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 06 Dec 2013 05:22:44 -0800 (PST) Message-ID: <52A1CF9D.30708@gmail.com> Date: Fri, 06 Dec 2013 17:22:37 +0400 From: Dmitry Vyal User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: "dev@dpdk.org" Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [dpdk-dev] DPDK delaying individual packets infinitely X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Dec 2013 13:21:43 -0000 Hello list, For some time I've been writing a custom packet generator coupled with a packet receiver using DPDK. It works greatly when used for generating millions of packets, but unexpectedly It has troubles generating a several dozens of packets or so. The application consists of two threads, first one sends the packets from one port and second receives them on another. These are 2 ports of four-ports 1Gb NIC. It's detected as Intel Corporation 82576 Gigabit Network Connection (rev 01). Ports are connected with a patch cord. My experiment runs like the following: After initializing NICs both generator and receiver wait for 1 second. Generator sends N packets by calling rte_eth_tx_burst for each individual packet. It waits for 100000000 cpu ticks between bursts. rte_eth_tx_burst reports all packets are sent. Receiver repeats calling rte_eth_rx_burst and waits for 50000000 ticks if the function returns zero. After generators sends all the packets, the receiver continues polling for several seconds. I'm observing the following behavior: If N is small, say 20, than no packets a received. See the logs. Receiver prints Z when it got zero packets and R if it got at least one packet. Starting experiment receiver 0 started main loop generator 0 sitting on socket 0 is waiting for experiment start generator 0 started main loop Zgenerating mbuf for file 0, port 0, addr 0 on socket 0 free_count on pool 0 = 100000000 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 1 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 2 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 3 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 4 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 5 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 6 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 7 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 8 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 9 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 10 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 11 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 12 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 13 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 14 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 15 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 16 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 17 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 18 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 19 on socket 0 sent 1 in queue 0 freed 0 Stopping after reaching 20 packets limit ZZwaiting for receivers to stopZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ********* Statistics: Seconds elapsed: 6.123671 If I made N bigger, say 25, than all of a sudden some packets a received, like so: Starting experiment receiver 0 started main loop generator 0 sitting on socket 0 is waiting for experiment start generator 0 started main loop generating mbuf for file 0, port 0, addr 0 on socket 0 Zfree_count on pool 0 = 100000000 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 1 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 2 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 3 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 4 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 5 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 6 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 7 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 8 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 9 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 10 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 11 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 12 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 13 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 14 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 15 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 16 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 17 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 18 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 19 on socket 0 sent 1 in queue 0 freed 0 ZZgenerating mbuf for file 0, port 0, addr 20 on socket 0 sent 1 in queue 0 freed 0 Rtotal received packets on queue 0: 21; received: 21; zero_iters: 41 Zgenerating mbuf for file 0, port 0, addr 21 on socket 0 free_count on pool 0 = 6113 sent 1 in queue 0 freed 0 Rtotal received packets on queue 0: 22; received: 1; zero_iters: 1 Zgenerating mbuf for file 0, port 0, addr 22 on socket 0 sent 1 in queue 0 freed 0 Rtotal received packets on queue 0: 23; received: 1; zero_iters: 1 Zgenerating mbuf for file 0, port 0, addr 23 on socket 0 sent 1 in queue 0 freed 0 Rtotal received packets on queue 0: 24; received: 1; zero_iters: 1 Zgenerating mbuf for file 0, port 0, addr 24 on socket 0 sent 1 in queue 0 freed 0 Stopping after reaching 25 packets limit Rtotal received packets on queue 0: 25; received: 1; zero_iters: 1 Zwaiting for receivers to stopZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ********* Statistics: Seconds elapsed: 6.293315 So looks like DPDK is buffering first 21 packets and doesn't flush the buffer in the first run even if I wait for several seconds after last packet is sent. In second run we see it forwards packets one by one after first burst of size 21. I tried on DPDK-1.3.1 and DPDK-1.5.1 observing similar results. Is it expected behavior?