* [dpdk-dev] NIC Stops Transmitting
@ 2013-07-26 19:39 Scott Talbert
2013-07-26 19:53 ` Stephen Hemminger
0 siblings, 1 reply; 10+ messages in thread
From: Scott Talbert @ 2013-07-26 19:39 UTC (permalink / raw)
To: dev
Hi,
I'm writing an application using DPDK that transmits a large number of
packets (it doesn't receive any). When I transmit at 2 Gb/sec, everything
will run fine for several seconds (receiver is receiving at correct rate),
but then the NIC appears to get 'stuck' and doesn't transmit any more
packets. In this state, rte_eth_tx_burst() is returning zero (suggesting
that there are no available transmit descriptors), but even if I sleep()
for a second and try again, rte_eth_tx_burst() still returns 0. It almost
appears as if a packet gets stuck in the transmit ring and keeps
everything from flowing. I'm using an Intel 82599EB NIC.
Does anyone have any ideas of what might be going on?
Thanks,
Scott
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] NIC Stops Transmitting
2013-07-26 19:39 [dpdk-dev] NIC Stops Transmitting Scott Talbert
@ 2013-07-26 19:53 ` Stephen Hemminger
2013-07-26 20:04 ` Scott Talbert
0 siblings, 1 reply; 10+ messages in thread
From: Stephen Hemminger @ 2013-07-26 19:53 UTC (permalink / raw)
To: Scott Talbert; +Cc: dev
On Fri, 26 Jul 2013 15:39:18 -0400 (EDT)
Scott Talbert <swt@techie.net> wrote:
> Hi,
>
> I'm writing an application using DPDK that transmits a large number of
> packets (it doesn't receive any). When I transmit at 2 Gb/sec, everything
> will run fine for several seconds (receiver is receiving at correct rate),
> but then the NIC appears to get 'stuck' and doesn't transmit any more
> packets. In this state, rte_eth_tx_burst() is returning zero (suggesting
> that there are no available transmit descriptors), but even if I sleep()
> for a second and try again, rte_eth_tx_burst() still returns 0. It almost
> appears as if a packet gets stuck in the transmit ring and keeps
> everything from flowing. I'm using an Intel 82599EB NIC.
>
> Does anyone have any ideas of what might be going on?
>
> Thanks,
> Scott
Make sure there is enough memory for mbufs.
Also what is your ring size and transmit free threshold?
It is easy to instrument the driver to see where it is saying "no space left"
Also be careful with threshold values, many values of pthresh/hthresh/wthresh
don't work. I would check the Intel reference manual for your hardware.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] NIC Stops Transmitting
2013-07-26 19:53 ` Stephen Hemminger
@ 2013-07-26 20:04 ` Scott Talbert
2013-07-26 22:31 ` jinho hwang
0 siblings, 1 reply; 10+ messages in thread
From: Scott Talbert @ 2013-07-26 20:04 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: dev
On Fri, 26 Jul 2013, Stephen Hemminger wrote:
>> I'm writing an application using DPDK that transmits a large number of
>> packets (it doesn't receive any). When I transmit at 2 Gb/sec, everything
>> will run fine for several seconds (receiver is receiving at correct rate),
>> but then the NIC appears to get 'stuck' and doesn't transmit any more
>> packets. In this state, rte_eth_tx_burst() is returning zero (suggesting
>> that there are no available transmit descriptors), but even if I sleep()
>> for a second and try again, rte_eth_tx_burst() still returns 0. It almost
>> appears as if a packet gets stuck in the transmit ring and keeps
>> everything from flowing. I'm using an Intel 82599EB NIC.
>>
> Make sure there is enough memory for mbufs.
> Also what is your ring size and transmit free threshold?
> It is easy to instrument the driver to see where it is saying "no space left"
> Also be careful with threshold values, many values of pthresh/hthresh/wthresh
> don't work. I would check the Intel reference manual for your hardware.
Thanks for the tips. I don't think I'm running out of mbufs, but I'll
check that again. I am using these values from one of the examples -
which claim to be correct for the 82599EB.
/*
* These default values are optimized for use with the Intel(R) 82599 10
GbE
* Controller and the DPDK ixgbe PMD. Consider using other values for
other
* network controllers and/or network drivers.
*/
#define TX_PTHRESH 36 /**< Default values of TX prefetch threshold reg. */
#define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */
#define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg.
*/
static const struct rte_eth_txconf tx_conf = {
.tx_thresh = {
.pthresh = TX_PTHRESH,
.hthresh = TX_HTHRESH,
.wthresh = TX_WTHRESH,
},
.tx_free_thresh = 0, /* Use PMD default values */
.tx_rs_thresh = 0, /* Use PMD default values */
};
/*
* Configurable number of RX/TX ring descriptors
*/
#define RTE_TEST_TX_DESC_DEFAULT 512
static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] NIC Stops Transmitting
2013-07-26 20:04 ` Scott Talbert
@ 2013-07-26 22:31 ` jinho hwang
2013-07-26 23:26 ` Scott Talbert
0 siblings, 1 reply; 10+ messages in thread
From: jinho hwang @ 2013-07-26 22:31 UTC (permalink / raw)
To: Scott Talbert; +Cc: dev
On Fri, Jul 26, 2013 at 4:04 PM, Scott Talbert <swt@techie.net> wrote:
> On Fri, 26 Jul 2013, Stephen Hemminger wrote:
>
>>> I'm writing an application using DPDK that transmits a large number of
>>> packets (it doesn't receive any). When I transmit at 2 Gb/sec,
>>> everything
>>> will run fine for several seconds (receiver is receiving at correct
>>> rate),
>>> but then the NIC appears to get 'stuck' and doesn't transmit any more
>>> packets. In this state, rte_eth_tx_burst() is returning zero (suggesting
>>> that there are no available transmit descriptors), but even if I sleep()
>>> for a second and try again, rte_eth_tx_burst() still returns 0. It
>>> almost
>>> appears as if a packet gets stuck in the transmit ring and keeps
>>> everything from flowing. I'm using an Intel 82599EB NIC.
>>>
>> Make sure there is enough memory for mbufs.
>> Also what is your ring size and transmit free threshold?
>> It is easy to instrument the driver to see where it is saying "no space
>> left"
>> Also be careful with threshold values, many values of
>> pthresh/hthresh/wthresh
>> don't work. I would check the Intel reference manual for your hardware.
>
>
> Thanks for the tips. I don't think I'm running out of mbufs, but I'll check
> that again. I am using these values from one of the examples - which claim
> to be correct for the 82599EB.
>
> /*
> * These default values are optimized for use with the Intel(R) 82599 10 GbE
> * Controller and the DPDK ixgbe PMD. Consider using other values for other
> * network controllers and/or network drivers.
> */
> #define TX_PTHRESH 36 /**< Default values of TX prefetch threshold reg. */
> #define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */
> #define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */
>
> static const struct rte_eth_txconf tx_conf = {
> .tx_thresh = {
> .pthresh = TX_PTHRESH,
> .hthresh = TX_HTHRESH,
> .wthresh = TX_WTHRESH,
> },
> .tx_free_thresh = 0, /* Use PMD default values */
> .tx_rs_thresh = 0, /* Use PMD default values */
> };
>
> /*
> * Configurable number of RX/TX ring descriptors
> */
> #define RTE_TEST_TX_DESC_DEFAULT 512
> static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
>
Scott,
I am wondering whether you use multiple cores accessing the same
receive queue. I had this problem before, but after I make the same
number of receiving queues as the number of receiving cores, the
problem disappeared. I did not dig more since I did not care how many
receive queues I have did not matter.
Jinho
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] NIC Stops Transmitting
2013-07-26 22:31 ` jinho hwang
@ 2013-07-26 23:26 ` Scott Talbert
2013-07-29 17:02 ` Scott Talbert
0 siblings, 1 reply; 10+ messages in thread
From: Scott Talbert @ 2013-07-26 23:26 UTC (permalink / raw)
To: jinho hwang; +Cc: dev
On Fri, 26 Jul 2013, jinho hwang wrote:
>> Thanks for the tips. I don't think I'm running out of mbufs, but I'll check
>> that again. I am using these values from one of the examples - which claim
>> to be correct for the 82599EB.
>>
>> /*
>> * These default values are optimized for use with the Intel(R) 82599 10 GbE
>> * Controller and the DPDK ixgbe PMD. Consider using other values for other
>> * network controllers and/or network drivers.
>> */
>> #define TX_PTHRESH 36 /**< Default values of TX prefetch threshold reg. */
>> #define TX_HTHRESH 0 /**< Default values of TX host threshold reg. */
>> #define TX_WTHRESH 0 /**< Default values of TX write-back threshold reg. */
>>
>> static const struct rte_eth_txconf tx_conf = {
>> .tx_thresh = {
>> .pthresh = TX_PTHRESH,
>> .hthresh = TX_HTHRESH,
>> .wthresh = TX_WTHRESH,
>> },
>> .tx_free_thresh = 0, /* Use PMD default values */
>> .tx_rs_thresh = 0, /* Use PMD default values */
>> };
>>
>> /*
>> * Configurable number of RX/TX ring descriptors
>> */
>> #define RTE_TEST_TX_DESC_DEFAULT 512
>> static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
>>
> I am wondering whether you use multiple cores accessing the same
> receive queue. I had this problem before, but after I make the same
> number of receiving queues as the number of receiving cores, the
> problem disappeared. I did not dig more since I did not care how many
> receive queues I have did not matter.
Jinho,
Thanks. I have only one queue (should I be using more?) but as far as I
know, I'm only using one core to transmit as well.
Scott
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] NIC Stops Transmitting
2013-07-26 23:26 ` Scott Talbert
@ 2013-07-29 17:02 ` Scott Talbert
2013-07-31 2:43 ` [dpdk-dev] miss include file in DPDK 1.3.1r2 Jia.Sui
0 siblings, 1 reply; 10+ messages in thread
From: Scott Talbert @ 2013-07-29 17:02 UTC (permalink / raw)
To: jinho hwang; +Cc: dev
On Fri, 26 Jul 2013, Scott Talbert wrote:
>> I am wondering whether you use multiple cores accessing the same
>> receive queue. I had this problem before, but after I make the same
>> number of receiving queues as the number of receiving cores, the
>> problem disappeared. I did not dig more since I did not care how many
>> receive queues I have did not matter.
>
> Thanks. I have only one queue (should I be using more?) but as far as I
> know, I'm only using one core to transmit as well.
All,
It turns out that the problem appears to have been caused by me using a
signal handler to control my data rates. After switching to use RTE
timers instead, I am able to transmit close to 10 Gb/sec reliably.
Thanks,
Scott
^ permalink raw reply [flat|nested] 10+ messages in thread
* [dpdk-dev] miss include file in DPDK 1.3.1r2
2013-07-29 17:02 ` Scott Talbert
@ 2013-07-31 2:43 ` Jia.Sui
2013-07-31 9:18 ` [dpdk-dev] [PATCH] mem: fix include in rte_malloc Thomas Monjalon
0 siblings, 1 reply; 10+ messages in thread
From: Jia.Sui(贾睢) @ 2013-07-31 2:43 UTC (permalink / raw)
To: dev
Hi All,
In DPDK 1.3.1r2, When I include 'rte_malloc.h' in my app, and then compile it
Will cause this issue:
/root/dpdk/x86_64-default-linuxapp-gcc/include/rte_malloc.h:333: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘rte_malloc_virt2phy’
It looks it miss the definition of 'phys_addr_t' which is defined in 'rte_memory.h'
Thanks
BRs
Jia Sui
^ permalink raw reply [flat|nested] 10+ messages in thread
* [dpdk-dev] [PATCH] mem: fix include in rte_malloc
2013-07-31 2:43 ` [dpdk-dev] miss include file in DPDK 1.3.1r2 Jia.Sui
@ 2013-07-31 9:18 ` Thomas Monjalon
2013-07-31 11:36 ` didier.pallard
0 siblings, 1 reply; 10+ messages in thread
From: Thomas Monjalon @ 2013-07-31 9:18 UTC (permalink / raw)
To: dev
The function rte_malloc_virt2phy has a dependency on rte_memory.h:
phys_addr_t must be defined.
The dependency handling for apps was wrong in the commit 8c86825cbf.
Let's revert this part.
Reported-by: Jia Sui <jia.sui@advantech.com.cn>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
---
app/test/test_hash.c | 2 +-
app/test/test_hash_perf.c | 2 +-
app/test/test_memcpy.c | 1 -
app/test/test_memcpy_perf.c | 1 -
lib/librte_malloc/rte_malloc.h | 1 +
5 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/app/test/test_hash.c b/app/test/test_hash.c
index 4de8cb1..5437e05 100644
--- a/app/test/test_hash.c
+++ b/app/test/test_hash.c
@@ -41,10 +41,10 @@
#include <sys/queue.h>
#include <rte_common.h>
+#include <rte_malloc.h>
#include <rte_cycles.h>
#include <rte_random.h>
#include <rte_memory.h>
-#include <rte_malloc.h>
#include <rte_memzone.h>
#include <rte_tailq.h>
#include <rte_eal.h>
diff --git a/app/test/test_hash_perf.c b/app/test/test_hash_perf.c
index 311c2bd..8a0feb3 100644
--- a/app/test/test_hash_perf.c
+++ b/app/test/test_hash_perf.c
@@ -42,10 +42,10 @@
#include <rte_common.h>
#include <rte_lcore.h>
+#include <rte_malloc.h>
#include <rte_cycles.h>
#include <rte_random.h>
#include <rte_memory.h>
-#include <rte_malloc.h>
#include <rte_memzone.h>
#include <rte_tailq.h>
#include <rte_eal.h>
diff --git a/app/test/test_memcpy.c b/app/test/test_memcpy.c
index 729a2ff..d7777a8 100644
--- a/app/test/test_memcpy.c
+++ b/app/test/test_memcpy.c
@@ -41,7 +41,6 @@
#include <cmdline_parse.h>
#include <rte_cycles.h>
#include <rte_random.h>
-#include <rte_memory.h>
#include <rte_malloc.h>
#include <rte_memcpy.h>
diff --git a/app/test/test_memcpy_perf.c b/app/test/test_memcpy_perf.c
index 1e75f53..236b295 100644
--- a/app/test/test_memcpy_perf.c
+++ b/app/test/test_memcpy_perf.c
@@ -41,7 +41,6 @@
#include <cmdline_parse.h>
#include <rte_cycles.h>
#include <rte_random.h>
-#include <rte_memory.h>
#include <rte_malloc.h>
#include <rte_memcpy.h>
diff --git a/lib/librte_malloc/rte_malloc.h b/lib/librte_malloc/rte_malloc.h
index e68187a..fe85689 100644
--- a/lib/librte_malloc/rte_malloc.h
+++ b/lib/librte_malloc/rte_malloc.h
@@ -42,6 +42,7 @@
*/
#include <stddef.h>
+#include <rte_memory.h>
#ifdef __cplusplus
extern "C" {
--
1.7.10.4
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH] mem: fix include in rte_malloc
2013-07-31 9:18 ` [dpdk-dev] [PATCH] mem: fix include in rte_malloc Thomas Monjalon
@ 2013-07-31 11:36 ` didier.pallard
2013-07-31 11:48 ` Thomas Monjalon
0 siblings, 1 reply; 10+ messages in thread
From: didier.pallard @ 2013-07-31 11:36 UTC (permalink / raw)
To: Thomas Monjalon; +Cc: dev
On 07/31/2013 11:18 AM, Thomas Monjalon wrote:
> The function rte_malloc_virt2phy has a dependency on rte_memory.h:
> phys_addr_t must be defined.
>
> The dependency handling for apps was wrong in the commit 8c86825cbf.
> Let's revert this part.
>
> Reported-by: Jia Sui <jia.sui@advantech.com.cn>
> Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Didier Pallard <didier.pallard@6wind.com>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH] mem: fix include in rte_malloc
2013-07-31 11:36 ` didier.pallard
@ 2013-07-31 11:48 ` Thomas Monjalon
0 siblings, 0 replies; 10+ messages in thread
From: Thomas Monjalon @ 2013-07-31 11:48 UTC (permalink / raw)
To: Didier Pallard, Jia Sui; +Cc: dev
31/07/2013 13:36, didier.pallard :
> On 07/31/2013 11:18 AM, Thomas Monjalon wrote:
> > The function rte_malloc_virt2phy has a dependency on rte_memory.h:
> > phys_addr_t must be defined.
> >
> > The dependency handling for apps was wrong in the commit 8c86825cbf.
> > Let's revert this part.
> >
> > Reported-by: Jia Sui <jia.sui@advantech.com.cn>
> > Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
>
> Acked-by: Didier Pallard <didier.pallard@6wind.com>
pushed
Thanks to Jia Sui for reporting.
Side note: please do not reply to an old email when creating new thread.
(see http://dpdk.org/ml/archives/dev/2013-July/thread.html)
--
Thomas
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-07-31 11:47 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-07-26 19:39 [dpdk-dev] NIC Stops Transmitting Scott Talbert
2013-07-26 19:53 ` Stephen Hemminger
2013-07-26 20:04 ` Scott Talbert
2013-07-26 22:31 ` jinho hwang
2013-07-26 23:26 ` Scott Talbert
2013-07-29 17:02 ` Scott Talbert
2013-07-31 2:43 ` [dpdk-dev] miss include file in DPDK 1.3.1r2 Jia.Sui
2013-07-31 9:18 ` [dpdk-dev] [PATCH] mem: fix include in rte_malloc Thomas Monjalon
2013-07-31 11:36 ` didier.pallard
2013-07-31 11:48 ` Thomas Monjalon
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).