From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 8C07F42EB9 for ; Wed, 19 Jul 2023 17:30:16 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 024DF40DFD; Wed, 19 Jul 2023 17:30:16 +0200 (CEST) Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com [209.85.219.173]) by mails.dpdk.org (Postfix) with ESMTP id 6946B40685 for ; Wed, 19 Jul 2023 17:30:14 +0200 (CEST) Received: by mail-yb1-f173.google.com with SMTP id 3f1490d57ef6-cf0211e2059so1460767276.2 for ; Wed, 19 Jul 2023 08:30:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1689780613; x=1692372613; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=6P7jmDyLSmx290a8CM2KANYYFbkCbyH54bOAvZaOWmA=; b=pbmNz+13D6cGRwSGbMreTg9jXB+d0gVOFVl2KxVxx5HpRKD0P0AR7c6vl3svYmkW/d 365QFD/WsaWdPAMK1v21dhcW5obbVU8xhCRgY+Ox2GigzL5//HRhYRF4VDsMwuqG7F7v gh694nQv+F+3zpvnG/98wShW9jVZjaPhSwKet65GPLxr8hp0np4nN3wCzP8MGc85DdIP WYTSTC4vgTesaR3gbNM0J7+vj8NrN3ac5NB9O7Ltn5Bwqq0P6hEPWO4o0KqGVAyqeICN W4xZXpRdhCFr/cWIgooU2Eivodh8cnuofeCVgUHSZ3yNKwqRSCRD/5icgajrKuyF9yd8 1SHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689780613; x=1692372613; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=6P7jmDyLSmx290a8CM2KANYYFbkCbyH54bOAvZaOWmA=; b=gPzDtyMSsBIJgeb2PXu3etn3S+cSWRrmaHaRW28HmAXBwhukaoAYJIZdsU8drQb4My 4uIPcMR27un2gzA8m7F6GX1u3RLarFp75rD8lVdhSRU2GSINpXpY4205sVZA9KPiSLT6 8HLT6Gqqo9S2XydoRH0pC+qWE6h3KIwvIKd08TCFwZPT1Anbn+OKHB4RUqI/HdUfseQe B54DNgQ0bfGVv+G6YL2YFOS1xLeoO8aY7nkgOY+1Wz4qcHXKP/dW+CYtGIPK52jygC/Y 1OvbV8Je5/lKqz/WROX06Dg/Vouvf829leZp2jDXWNF4OCFqUU24MswToJPp0jFaFj6Q b79w== X-Gm-Message-State: ABy/qLa6VsINBSWoyMJB4tnzbeyJo+2awNnCgutLeYMZnyvwDIbfPHrA jYTqgpKhhVfzXo+jgLc5HEVuDX9r7zgpYVifL2dOj5Rh4Tg= X-Google-Smtp-Source: APBJJlFx1HKIPupbFMm30IdFlB+orC9rAdBn9ATGmHVlIVGksZ7jBV76qBJ8g/uWUjcWvkAQwJuFNq6h1dbPamLj/zk= X-Received: by 2002:a25:db82:0:b0:cee:1956:7b73 with SMTP id g124-20020a25db82000000b00cee19567b73mr2568981ybf.38.1689780613502; Wed, 19 Jul 2023 08:30:13 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Hari Haran Date: Wed, 19 Jul 2023 21:00:02 +0530 Message-ID: Subject: Re: Inflight value shown invalid in Event Dev Queue To: "Van Haaren, Harry" Cc: "users@dpdk.org" Content-Type: multipart/alternative; boundary="000000000000375b9c0600d8b576" X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK usage discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: users-bounces@dpdk.org --000000000000375b9c0600d8b576 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi Harry Haaren (Yes :) ) I have given more details below, please check this. *Device Configuration:* Event Dev Queue : 1 Number of ports : 3 Queue 0 depth - 32k Port 0, 1 amd 2 : Enqueue depth 4096, Dequeue depth 128 *Cores: * Rx core - 1 Workers cores - 2 *Port 2:* Used in Rx core - Used to post packets from Rx core to worker cores using event dev queue . So port 2 used to post packets only. API used: rte_event_enqueue_burst() *Port 0 and 1* linked with Event Dev Q 0 to dequeue the packets. These ports used to dequeue the packets only. Port 0 used in Worker core 1 - Only to receive the packets from Rx core using event dev queue Port 1 used in worker core 2 - Only to receive the packets from Rx core using event dev queue API used: rte_event_dequeue_burst() *Expected behaviour*: Port 2 enqueue packets to event dev Q in Rx core Port 0 and 1 dequeue packets from event dev Q in two workers Event dev scheduler of queue 0, will schedule received packets in port 2 to port 0 and 1. *Problem Description:* Port 0 - only received 4096 packets through event dev Q, after that no packets available for this. API used: rte_event_dequeue_burst() Port 2 - Successfully enqueued 32k packets through event dev Q, after that enqueue failure observed. API used: rte_event_enqueue_burst() Looks like, event dev queue stalled at this point. Also why port 0 stats show inflight as 4096? *Port 0 Stats:* rx 0 drop 0 tx 4096 inflight 4096 *All Stats:* Dev=3D0 Port=3D1EventDev todo-fix-name: ports 3, qids 1 rx 32768 drop 0 tx 4096 sched calls: 628945658 sched cq/qid call: 628964843 sched no IQ enq: 628926401 sched no CQ enq: 628942982 inflight 32768, credits: 0 Port 0 rx 0 drop 0 tx 4096 inflight 4096 Max New: 32768 Avg cycles PP: 0 Credits: 0 Receive burst distribution: 0:100% 1-4:0.00% 5-8:0.00% 9-12:0.00% rx ring used: 0 free: 4096 cq ring used: 0 free: 128 Port 1 rx 0 drop 0 tx 0 inflight 0 Max New: 32768 Avg cycles PP: 0 Credits: 0 Receive burst distribution: 0:100% rx ring used: 0 free: 4096 cq ring used: 0 free: 128 Port 2 rx 32768 drop 0 tx 0 inflight 0 Max New: 32768 Avg cycles PP: 0 Credits: 0 Receive burst distribution: 0:-nan% rx ring used: 0 free: 4096 cq ring used: 0 free: 128 Queue 0 (Atomic) rx 32768 drop 0 tx 4096 Per Port Stats: Port 0: Pkts: 4096 Flows: 1 Port 1: Pkts: 0 Flows: 0 Port 2: Pkts: 0 Flows: 0 Port 3: Pkts: 0 Flows: 0 iq 0: Used 28672 Regards, Hariharan On Wed, Jul 19, 2023 at 6:30=E2=80=AFPM Van Haaren, Harry < harry.van.haaren@intel.com> wrote: > > From: Hari Haran > > Sent: Wednesday, July 19, 2023 1:39 PM > > To: users@dpdk.org > > Subject: Inflight value shown invalid in Event Dev Queue > > > > Hi All, > > Hi Hari Haran, > > > Once packets dequeued from port 0, still inflight stats shown same as > dequeued count. > > After that, enqueue failure happens for port 2 from another core due to > it reaches Max Enqueue depth . > > This describes what happens -> it would be helpful to know what you are > expecting to happen. > Would you describe what each of port 0,1,2 are actually used for, and how > events are expected > to flow from RX to a Port through a Queue, to another Port, until TX? > > Describing the expectation and then comparing that to your "problem > description" in this email often > leads to the root cause & solution. > > Keep in mind that the event/sw implementation has capacity limitations. I= t > seems to be too high in your configuration > (inflight =3D 32768 is a indicator of an issue, as SW_INFLIGHT_EVENTS_TOT= AL > is 4096 in sw_evdev.h). > > > Port 0 Stats: > > > > Below case, port 0 dequeued 4096 packets and still inflight showed the > same value. > > How are the events being re-enqueued? > > > Port 0 > > rx 0 drop 0 tx 4096 inflight 4096 > > > > Full Stats: > > > > Dev=3D0 Port=3D1EventDev todo-fix-name: ports 3, qids 1 > > rx 32768 > > drop 0 > > tx 4096 > > sched calls: 628945658 > > sched cq/qid call: 628964843 > > sched no IQ enq: 628926401 > > sched no CQ enq: 628942982 > > inflight 32768, credits: 0 > > > > Port 0 > > rx 0 drop 0 tx 4096 inflight 4096 > > Max New: 32768 Avg cycles PP: 0 Credits: 0 > > Receive burst distribution: > > 0:100% 1-4:0.00% 5-8:0.00% 9-12:0.00% > > rx ring used: 0 free: 4096 > > cq ring used: 0 free: 128 > > Port 1 > > rx 0 drop 0 tx 0 inflight 0 > > Max New: 32768 Avg cycles PP: 0 Credits: 0 > > Receive burst distribution: > > 0:100% > > rx ring used: 0 free: 4096 > > cq ring used: 0 free: 128 > > Port 2 > > rx 32768 drop 0 tx 0 inflight 0 > > Max New: 32768 Avg cycles PP: 0 Credits: 0 > > Receive burst distribution: > > 0:-nan% > > rx ring used: 0 free: 4096 > > cq ring used: 0 free: 128 > > > > Queue 0 (Atomic) > > rx 32768 drop 0 tx 4096 > > Per Port Stats: > > Port 0: Pkts: 4096 Flows: 1 > > Port 1: Pkts: 0 Flows: 0 > > Port 2: Pkts: 0 Flows: 0 > > Port 3: Pkts: 0 Flows: 0 > > iq 0: Used 28672 > > This issue will be resolved once system level restart is done. > > Kindly give insight on this issue, if you found any clues. > > TIA. > > > > Regards > > Hariharan > > Regards, -Harry van Haaren (PS: our names are surprisingly similar! : ) > --000000000000375b9c0600d8b576 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Harry Haaren (Yes :) )

I have given = more details below, please check this.=C2=A0

Device Config= uration:
Event Dev Queue : 1
Number of ports : 3

Queue 0 d= epth - 32k
Port 0, 1 amd 2 : Enqueue depth 4096, Dequeue depth 128
Cores:
Rx core - 1
Workers cores - 2

Port 2:<= br>Used in Rx core - Used to post packets from Rx core to worker cores usin= g event dev queue .
So port 2 used to post packets only. =C2=A0
API u= sed: rte_event_enqueue_burst()

Port 0 and 1 linked with Event= Dev Q 0 to dequeue the packets. These ports used to dequeue the packets on= ly.
Port 0 used in Worker core 1 - Only to receive the packets from Rx = core using event dev queue
Port 1 used in worker core 2 - Only to receiv= e the packets from Rx core using event dev queue
API used: rte_event_deq= ueue_burst()

Expected behaviour:

Port 2 enqueue packet= s to event dev Q in Rx core
Port 0 and 1 dequeue packets from event dev= Q in two workers

Event dev scheduler of queue 0, will schedule rec= eived packets in port 2 to port 0 and 1.


Problem Description= :

Port 0 - only received 4096 packets through event dev Q, after= that no packets available for this.
API used: rte_event_dequeue_burst(= )

Port 2 - Successfully enqueued 32k packets through event dev Q, af= ter that enqueue failure observed.
API used: rte_event_enqueue_burst()<= br>
Looks like, event dev queue stalled at this point.
Also why port 0 stats show inflight as 4096?

Port 0 Stats:
<= /b>=C2=A0 rx=C2=A0=C2=A0 0=C2=A0 drop 0=C2=A0 tx=C2=A0=C2=A0 4096=C2=A0=C2= =A0 inflight 4096

All Stats:
Dev=3D0 Port=3D1EventDev todo= -fix-name: ports 3, qids 1
rx=C2=A0=C2=A0 32768
drop 0
tx=C2=A0=C2= =A0 4096
sched calls: 628945658
sched cq/qid call: 628964843
sched= no IQ enq: 628926401
sched no CQ enq: 628942982
inflight 32768, cred= its: 0


Port 0
=C2=A0 rx=C2=A0=C2=A0 0=C2=A0 drop 0=C2=A0 tx= =C2=A0=C2=A0 4096=C2=A0=C2=A0 inflight 4096
=C2=A0 Max New: 32768=C2=A0 = Avg cycles PP: 0=C2=A0=C2=A0=C2=A0 Credits: 0
=C2=A0 Receive burst distr= ibution:
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0:100% 1-4:0.00% 5-8:0.00% 9-12:= 0.00%
=C2=A0 rx ring used:=C2=A0=C2=A0=C2=A0 0 free: 4096
=C2=A0 cq r= ing used:=C2=A0=C2=A0=C2=A0 0 free:=C2=A0 128
Port 1
=C2=A0 rx=C2=A0= =C2=A0 0=C2=A0 drop 0=C2=A0 tx=C2=A0=C2=A0 0=C2=A0 inflight 0
=C2=A0 Max= New: 32768=C2=A0 Avg cycles PP: 0=C2=A0=C2=A0=C2=A0 Credits: 0
=C2=A0 R= eceive burst distribution:
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0:100%
=C2= =A0 rx ring used:=C2=A0=C2=A0=C2=A0 0 free: 4096
=C2=A0 cq ring used:=C2= =A0=C2=A0=C2=A0 0 free:=C2=A0 128
Port 2
=C2=A0 rx=C2=A0=C2=A0 32768= =C2=A0 drop 0=C2=A0 tx=C2=A0=C2=A0 0=C2=A0 inflight 0
=C2=A0 Max New: 32= 768=C2=A0 Avg cycles PP: 0=C2=A0=C2=A0=C2=A0 Credits: 0
=C2=A0 Receive b= urst distribution:
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0:-nan%
=C2=A0 rx r= ing used:=C2=A0=C2=A0=C2=A0 0 free: 4096
=C2=A0 cq ring used:=C2=A0=C2= =A0=C2=A0 0 free:=C2=A0 128

Queue 0 (Atomic)
=C2=A0 rx=C2=A0=C2= =A0 32768=C2=A0 drop 0=C2=A0 tx=C2=A0=C2=A0 4096
=C2=A0 Per Port Stats:<= br>=C2=A0=C2=A0=C2=A0 Port 0: Pkts: 4096=C2=A0=C2=A0=C2=A0 Flows: 1
=C2= =A0=C2=A0=C2=A0 Port 1: Pkts: 0=C2=A0=C2=A0 Flows: 0
=C2=A0=C2=A0=C2=A0 = Port 2: Pkts: 0=C2=A0=C2=A0 Flows: 0
=C2=A0=C2=A0=C2=A0 Port 3: Pkts: 0= =C2=A0=C2=A0 Flows: 0
=C2=A0 iq 0: Used 28672

Regards,
Hariharan


On Wed, Jul 19, 2023 at= 6:30=E2=80=AFPM Van Haaren, Harry <harry.van.haaren@intel.com> wrote:
> From: Hari Haran <info2hariharan@gmail.com>
> Sent: Wednesday, July 19, 2023 1:39 PM
> To: users@dpdk.org=
> Subject: Inflight value shown invalid in Event Dev Queue
>
> Hi All,

Hi Hari Haran,

> Once packets dequeued from port 0, still inflight stats shown same as = dequeued count.
> After that, enqueue failure happens for port 2 from another core due t= o it reaches Max Enqueue depth .

This describes what happens -> it would be helpful to know what you are = expecting to happen.
Would you describe what each of port 0,1,2 are actually used for, and how e= vents are expected
to flow from RX to a Port through a Queue, to another Port, until TX?

Describing the expectation and then comparing that to your "problem de= scription" in this email often
leads to the root cause & solution.

Keep in mind that the event/sw implementation has capacity limitations. It = seems to be too=C2=A0 high in your configuration
(inflight =3D 32768 is a indicator of an issue, as SW_INFLIGHT_EVENTS_TOTAL= is 4096 in sw_evdev.h).

> Port 0 Stats:
>
> Below case, port 0 dequeued 4096 packets and still inflight showed the= same value.

How are the events being re-enqueued?

> Port 0
>=C2=A0 =C2=A0rx=C2=A0 =C2=A00=C2=A0 drop 0=C2=A0 tx=C2=A0 =C2=A04096=C2= =A0 =C2=A0inflight 4096=C2=A0
>
> Full Stats:
>
> Dev=3D0 Port=3D1EventDev todo-fix-name: ports 3, qids 1
> rx=C2=A0 =C2=A032768
> drop 0
> tx=C2=A0 =C2=A04096
> sched calls: 628945658
> sched cq/qid call: 628964843
> sched no IQ enq: 628926401
> sched no CQ enq: 628942982
> inflight 32768, credits: 0
>
> Port 0
>=C2=A0 =C2=A0rx=C2=A0 =C2=A00=C2=A0 drop 0=C2=A0 tx=C2=A0 =C2=A04096=C2= =A0 =C2=A0inflight 4096
>=C2=A0 =C2=A0Max New: 32768=C2=A0 Avg cycles PP: 0=C2=A0 =C2=A0 Credits= : 0
>=C2=A0 =C2=A0Receive burst distribution:
>=C2=A0 =C2=A0 =C2=A0 =C2=A00:100% 1-4:0.00% 5-8:0.00% 9-12:0.00%
>=C2=A0 =C2=A0rx ring used:=C2=A0 =C2=A0 0 free: 4096
>=C2=A0 =C2=A0cq ring used:=C2=A0 =C2=A0 0 free:=C2=A0 128
> Port 1
>=C2=A0 =C2=A0rx=C2=A0 =C2=A00=C2=A0 drop 0=C2=A0 tx=C2=A0 =C2=A00=C2=A0= inflight 0
>=C2=A0 =C2=A0Max New: 32768=C2=A0 Avg cycles PP: 0=C2=A0 =C2=A0 Credits= : 0
>=C2=A0 =C2=A0Receive burst distribution:
>=C2=A0 =C2=A0 =C2=A0 =C2=A00:100%
>=C2=A0 =C2=A0rx ring used:=C2=A0 =C2=A0 0 free: 4096
>=C2=A0 =C2=A0cq ring used:=C2=A0 =C2=A0 0 free:=C2=A0 128
> Port 2
>=C2=A0 =C2=A0rx=C2=A0 =C2=A032768=C2=A0 drop 0=C2=A0 tx=C2=A0 =C2=A00= =C2=A0 inflight 0
>=C2=A0 =C2=A0Max New: 32768=C2=A0 Avg cycles PP: 0=C2=A0 =C2=A0 Credits= : 0
>=C2=A0 =C2=A0Receive burst distribution:
>=C2=A0 =C2=A0 =C2=A0 =C2=A00:-nan%
>=C2=A0 =C2=A0rx ring used:=C2=A0 =C2=A0 0 free: 4096
>=C2=A0 =C2=A0cq ring used:=C2=A0 =C2=A0 0 free:=C2=A0 128
>
> Queue 0 (Atomic)
>=C2=A0 =C2=A0rx=C2=A0 =C2=A032768=C2=A0 drop 0=C2=A0 tx=C2=A0 =C2=A0409= 6
>=C2=A0 =C2=A0Per Port Stats:
>=C2=A0 =C2=A0 =C2=A0Port 0: Pkts: 4096=C2=A0 =C2=A0 Flows: 1
>=C2=A0 =C2=A0 =C2=A0Port 1: Pkts: 0=C2=A0 =C2=A0Flows: 0
>=C2=A0 =C2=A0 =C2=A0Port 2: Pkts: 0=C2=A0 =C2=A0Flows: 0
>=C2=A0 =C2=A0 =C2=A0Port 3: Pkts: 0=C2=A0 =C2=A0Flows: 0
>=C2=A0 =C2=A0iq 0: Used 28672
> This issue will be resolved once system level restart is done.
> Kindly give insight on this issue, if you found any clues.
> TIA.
>
> Regards
> Hariharan

Regards, -Harry van Haaren (PS: our names are surprisingly similar! : )
--000000000000375b9c0600d8b576--