From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <users-bounces@dpdk.org>
Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124])
	by inbox.dpdk.org (Postfix) with ESMTP id 786D3A0032
	for <public@inbox.dpdk.org>; Tue, 15 Mar 2022 20:29:09 +0100 (CET)
Received: from [217.70.189.124] (localhost [127.0.0.1])
	by mails.dpdk.org (Postfix) with ESMTP id 01BFC40395;
	Tue, 15 Mar 2022 20:29:09 +0100 (CET)
Received: from mail-vk1-f169.google.com (mail-vk1-f169.google.com
 [209.85.221.169])
 by mails.dpdk.org (Postfix) with ESMTP id D4F874014F
 for <users@dpdk.org>; Tue, 15 Mar 2022 20:29:07 +0100 (CET)
Received: by mail-vk1-f169.google.com with SMTP id m84so136726vke.1
 for <users@dpdk.org>; Tue, 15 Mar 2022 12:29:07 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:from:date:message-id:subject:to;
 bh=cr2tvs7+79cyLYT8k6Ppf6nr+L3Z/duv/LrxGPuluo4=;
 b=o+CtBTZZKXEBsLHnYvmGvvV1+Kp6fro2jrhNtPeX10sRpJX+dgH0PGL+xVkeXO9wzU
 /QAlJfGGlOLZVqs9C4YHpx0Ei2MQ7RgwOMYcKQnj422sOYTQyhT6Y7J4TvsH11uXfWpL
 TD0DnPBdnjgSDcdJZSvA0YvCO7jbzEW558ZPNHCMwdyrrP48VHbcYobUBwp765p/X3dt
 WQxPPdarrw9iHcFgvuYX/uS7lzecOkeww8vVDIDlB1Q/1RAP5qa0R1R4AkOpvxkwO8FB
 K/0/gZhQzwr9xQwaG4sY3H3NwHn9fn8ovKRx685dNM0ZQfxEIdt3DRUFuYGWDanE9zge
 x6+g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=cr2tvs7+79cyLYT8k6Ppf6nr+L3Z/duv/LrxGPuluo4=;
 b=LQeYLDd3Pm9O3/YSshx/IbbCmMwqmrlsO/7x5Gg4oYkBrdkRtVyIxGJ7zboiGU/ZOA
 YUBXBOYxaLZy0U5lOXB3blAMUTzpgVPbzxAyePiASjWVhv1qLi90NH9vaFCdUZ3sc/NZ
 r3bDgmGtii8ui8ajlKfZrE0yWJowxkdDv7wKdGEJx5CtOCaWYxUHO9oZx1BG0Eo8aAsA
 r2Q2+8nt/Ntrz9CxmC/+mmKSxLL/xTf1ohA3CYqTOHvfk+hv9RvSib3xS0Jiaw5jk4So
 gWEFMkouwk4gm8aGse66TJfY9O3ROtariM24/APvPo3LObwzb2Zo+wweK3H7h45r2lj1
 hVXQ==
X-Gm-Message-State: AOAM533nExCf0fXQOnGhhGJxYwaz+/u280c+lpw4KDReuydTwK0VoqXL
 9+K/yRhJVbA/naLOfm5rR4SMVx4TRgw0RYZmZvgY44olEDk=
X-Google-Smtp-Source: ABdhPJyblp1tDGuJGQRCzzEGh8kHHjkXf2GK36eTlpmVhc832gJ97i0iSHFTpT7KJY2s6af99ma2PGuWRjZd1wi6Ce4=
X-Received: by 2002:a1f:9895:0:b0:331:7132:72cf with SMTP id
 a143-20020a1f9895000000b00331713272cfmr12377617vke.7.1647372546847; Tue, 15
 Mar 2022 12:29:06 -0700 (PDT)
MIME-Version: 1.0
From: fwefew 4t4tg <7532yahoo@gmail.com>
Date: Tue, 15 Mar 2022 15:28:55 -0400
Message-ID: <CA+Tq66UpwJ6B3Jztw5YHdScNsvyzVQ_iPfssU_WoPWj=CAo7Kg@mail.gmail.com>
Subject: (AWS ENA NIC) queued v. out packets reported by any of the DPDK stats
 functions
To: users@dpdk.org
Content-Type: multipart/alternative; boundary="00000000000077b99a05da46cf6b"
X-BeenThere: users@dpdk.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: DPDK usage discussions <users.dpdk.org>
List-Unsubscribe: <https://mails.dpdk.org/options/users>,
 <mailto:users-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://mails.dpdk.org/archives/users/>
List-Post: <mailto:users@dpdk.org>
List-Help: <mailto:users-request@dpdk.org?subject=help>
List-Subscribe: <https://mails.dpdk.org/listinfo/users>,
 <mailto:users-request@dpdk.org?subject=subscribe>
Errors-To: users-bounces@dpdk.org

--00000000000077b99a05da46cf6b
Content-Type: text/plain; charset="UTF-8"

On DPDK head commit (917229) I have a simple program creating two TXQs each
sending 15000 IPV4 UDP packets.

As far as I can see there are no errors anymore. I have looked. I burst TX
them in blocks of 15; rte_eth_tx_burst always returns 15.

The lcores are launched with rte_eal_mp_remote_launch, and main() blocks
until rte_eal_mp_wait_lcore() returns.

However, at any time after rte_eal_mp_wait_lcore returns showing the number
of out packets as per rte_eth_stats_get or xstats per rte_eth_xstats_get I
see two things:

- the reported number of queued packets per TXQ is always 15000
- the number of out packets or "good packets" is close to 15000*2 but never
30000

It appears like the TXQs still have work in-queue even after the lcore's
thread returns and rte_eal_mp_wait_lcore returns. I've tried to
rte_eth_tx_done_cleanup and/or rte_eth_dev_tx_queue_stop before I get the
stats. That doesn't seem to help.

The TXQ config is default except for checksum offloads:

000000.336378246 DEBUG reinvent_dpdk_initaws.cpp:1341 TXQ conf:
{"tx_thresh": { "pthresh": 0, "hthresh": 0, "wthresh": 0 }, "tx_rs_thresh":
0, "tx_free_thresh": 0, "tx_deferred_start": 0, "tx_offloads": 00008006, }

Is this expected behavior? I've look through code; I haven't seen clear
signs of a TXQ oriented lcore flushed or waiting for the output queue to be
actually written onto the wire before exiting.

lcoreId: 01, txqIndex: 1, packetsQueued: 15000, sleeps: 0,
totalSleepTimeNs: 0

lcoreId: 00, txqIndex: 0, packetsQueued: 15000, sleeps: 0,
totalSleepTimeNs: 0

000000.351345192 INFO  reinvent_dpdk_udp_integration_test.cpp:585 waiting
for DPDK worker threads to stop

in  packets : 0

out packets : 29000

in  bytes   : 0

out bytes   : 1566000

missed pkts : 0

in err pkts : 0

out err pkts: 0

rx allc errs: 0

rx_good_packets                 : 0

tx_good_packets                 : 29952

rx_good_bytes                   : 0

tx_good_bytes                   : 1617408

rx_missed_errors                : 0

rx_errors                       : 0

tx_errors                       : 0

rx_mbuf_allocation_errors       : 0

tx_q0_packets                   : 15000

tx_q0_bytes                     : 810000

tx_q1_packets                   : 15000

tx_q1_bytes                     : 810000

wd_expired                      : 0

dev_start                       : 1

dev_stop                        : 0

tx_drops                        : 0

bw_in_allowance_exceeded        : 0

bw_out_allowance_exceeded       : 0

pps_allowance_exceeded          : 0

conntrack_allowance_exceeded    : 0

linklocal_allowance_exceeded    : 0

tx_q0_cnt                       : 15000

tx_q1_cnt                       : 15000

tx_q0_bytes                     : 810000

tx_q1_bytes                     : 810000

tx_q0_prepare_ctx_err           : 0

tx_q1_prepare_ctx_err           : 0

tx_q0_tx_poll                   : 54669

tx_q1_tx_poll                   : 54316

tx_q0_doorbells                 : 2311

tx_q1_doorbells                 : 2321

tx_q0_bad_req_id                : 0

tx_q1_bad_req_id                : 0

tx_q0_available_desc            : 15

tx_q1_available_desc            : 7

tx_q0_missed_tx                 : 0

tx_q1_missed_tx                 : 0

ena_tx_queue_release(): Tx queue 0:0 released

ena_tx_queue_release(): Tx queue 0:1 released

--00000000000077b99a05da46cf6b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">On DPDK head commit (917229) I have a simple program creat=
ing two TXQs each sending 15000 IPV4 UDP packets.=C2=A0<div><br></div><div>=
As far as I can see there are no errors anymore. I have looked. I burst TX =
them in blocks of 15; rte_eth_tx_burst always returns 15.<br><br>The lcores=
 are launched with rte_eal_mp_remote_launch, and main() blocks until=C2=A0r=
te_eal_mp_wait_lcore() returns.=C2=A0<br><br>However, at any time after rte=
_eal_mp_wait_lcore returns showing the number of out packets as per=C2=A0rt=
e_eth_stats_get or xstats per=C2=A0rte_eth_xstats_get I see two things:<br>=
<br>- the reported number of queued packets per TXQ is always 15000</div><d=
iv>- the number of out packets or &quot;good packets&quot; is close to 1500=
0*2 but never 30000</div><div><br></div><div>It appears like the TXQs still=
 have work in-queue even after the lcore&#39;s thread returns and rte_eal_m=
p_wait_lcore returns. I&#39;ve tried to rte_eth_tx_done_cleanup and/or rte_=
eth_dev_tx_queue_stop before I get the stats. That doesn&#39;t seem to help=
.</div><div><br></div><div>The TXQ config is default except for checksum=C2=
=A0offloads:<br><br>000000.336378246 DEBUG reinvent_dpdk_initaws.cpp:1341 T=
XQ conf: {&quot;tx_thresh&quot;: { &quot;pthresh&quot;: 0, &quot;hthresh&qu=
ot;: 0, &quot;wthresh&quot;: 0 }, &quot;tx_rs_thresh&quot;: 0, &quot;tx_fre=
e_thresh&quot;: 0, &quot;tx_deferred_start&quot;: 0, &quot;tx_offloads&quot=
;: 00008006, }<br><br></div><div>Is this expected behavior? I&#39;ve look t=
hrough code; I haven&#39;t seen clear signs of a TXQ oriented lcore flushed=
 or waiting for the output queue to be actually written onto the wire befor=
e exiting.</div><div><br></div><div>





<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
lcoreId: 01, txqIndex: 1, packetsQueued: 15000, sleeps: 0, totalSleepTimeNs=
: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
lcoreId: 00, txqIndex: 0, packetsQueued: 15000, sleeps: 0, totalSleepTimeNs=
: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
000000.351345192 INFO<span class=3D"gmail-Apple-converted-space">=C2=A0 </s=
pan>reinvent_dpdk_udp_integration_test.cpp:585 waiting for DPDK worker thre=
ads to stop</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
in<span class=3D"gmail-Apple-converted-space">=C2=A0 </span>packets : 0</sp=
an></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
out packets : 29000</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
in<span class=3D"gmail-Apple-converted-space">=C2=A0 </span>bytes <span cla=
ss=3D"gmail-Apple-converted-space">=C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
out bytes <span class=3D"gmail-Apple-converted-space">=C2=A0 </span>: 15660=
00</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
missed pkts : 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
in err pkts : 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
out err pkts: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
rx allc errs: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
rx_good_packets <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_good_packets <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 29952</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
rx_good_bytes <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_good_bytes <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 1617408</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
rx_missed_errors<span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
rx_errors <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></=
p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_errors <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></=
p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
rx_mbuf_allocation_errors <span class=3D"gmail-Apple-converted-space">=C2=
=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_packets <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 15000</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_bytes <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 810000</span>=
</p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_packets <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 15000</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_bytes <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 810000</span>=
</p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
wd_expired<span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></=
p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
dev_start <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 1</span></=
p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
dev_stop<span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</=
span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_drops<span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</=
span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
bw_in_allowance_exceeded<span class=3D"gmail-Apple-converted-space">=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
bw_out_allowance_exceeded <span class=3D"gmail-Apple-converted-space">=C2=
=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
pps_allowance_exceeded<span class=3D"gmail-Apple-converted-space">=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
conntrack_allowance_exceeded<span class=3D"gmail-Apple-converted-space">=C2=
=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
linklocal_allowance_exceeded<span class=3D"gmail-Apple-converted-space">=C2=
=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_cnt <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 15000</spa=
n></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_cnt <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 15000</spa=
n></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_bytes <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 810000</span>=
</p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_bytes <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 810000</span>=
</p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_prepare_ctx_err <span class=3D"gmail-Apple-converted-space">=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_prepare_ctx_err <span class=3D"gmail-Apple-converted-space">=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_tx_poll <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 54669</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_tx_poll <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 54316</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_doorbells <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 2311</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_doorbells <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 2321</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_bad_req_id<span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_bad_req_id<span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_available_desc<span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 15</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_available_desc<span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 7</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q0_missed_tx <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
tx_q1_missed_tx <span class=3D"gmail-Apple-converted-space">=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>: 0</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
ena_tx_queue_release(): Tx queue 0:0 released</span></p>
<p class=3D"gmail-p1" style=3D"margin:0px;font-variant-numeric:normal;font-=
variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:no=
rmal;font-family:Menlo;color:rgb(33,255,6);background-color:rgb(0,0,0)"><sp=
an class=3D"gmail-s1" style=3D"font-variant-ligatures:no-common-ligatures">=
ena_tx_queue_release(): Tx queue 0:1 released</span></p></div>





</div>

--00000000000077b99a05da46cf6b--