DPDK CI discussions
 help / color / mirror / Atom feed
From: "Liguzinski, WojciechX" <wojciechx.liguzinski@intel.com>
To: "Dumitrescu, Cristian" <cristian.dumitrescu@intel.com>,
	David Marchand <david.marchand@redhat.com>,
	Lincoln Lavoie <lylavoie@iol.unh.edu>,
	"Ajmera, Megha" <megha.ajmera@intel.com>,
	"Singh, Jasvinder" <jasvinder.singh@intel.com>
Cc: dev <dev@dpdk.org>, Aaron Conole <aconole@redhat.com>,
	Thomas Monjalon <thomas@monjalon.net>,
	"Yigit, Ferruh" <ferruh.yigit@intel.com>,
	"ci@dpdk.org" <ci@dpdk.org>,
	"Zegota, AnnaX" <annax.zegota@intel.com>
Subject: RE: [dpdk-dev] [Bug 826] red_autotest random failures
Date: Thu, 18 Nov 2021 22:10:46 +0000	[thread overview]
Message-ID: <BN9PR11MB53724D41DC05875D1D7F8E12949B9@BN9PR11MB5372.namprd11.prod.outlook.com> (raw)
In-Reply-To: <DM4SPR01MB00033233A5A39A2336D5D38394989@DM4SPR01MB0003.namprd11.prod.outlook.com>

Hi,

I was trying to reproduce this test failure, but for me RED tests are passing. 
I was running the exact test command like the one described in Bug 826 - 'red_autotest' on the current main branch.

Here is an example when DPDK is build without RTE_SCHED_CMAN enabled, but with this flag set to true tests are also not failing.

root@silpixa00400629:~/wojtek/dpdk/build/app/test# ./dpdk-test '-l 0-15' --file-prefix=red_autotest
EAL: Detected CPU lcores: 96
EAL: Detected NUMA nodes: 2
EAL: Detected static linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/red_autotest/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: VFIO support initialized
TELEMETRY: No legacy callbacks, legacy socket not created
APP: HPET is not enabled, using TSC as default timer
RTE>>red_autotest

--------------------------------------------------------------------------------
functional test 1 : use one rte_red configuration,
                    increase average queue size to various levels,
                    compare drop rate to drop probability

                avg queue size enqueued       dropped        drop prob %    drop rate %    diff %         tolerance %    
                6              10000          0              0.0000         0.0000         0.0000         50.0000        
                12             10000          0              0.0000         0.0000         0.0000         50.0000        
                18             10000          0              0.0000         0.0000         0.0000         50.0000        
                24             10000          0              0.0000         0.0000         0.0000         50.0000        
                30             10000          0              0.0000         0.0000         0.0000         50.0000        
                36             9961           39             0.4167         0.3900         0.0000         50.0000        
                42             9898           102            1.0417         1.0200         0.0000         50.0000        
                48             9835           165            1.6667         1.6500         0.0000         50.0000        
                54             9785           215            2.2917         2.1500         0.0000         50.0000        
                60             9703           297            2.9167         2.9700         0.0000         50.0000        
                66             9627           373            3.5417         3.7300         0.0000         50.0000        
                72             9580           420            4.1667         4.2000         0.0000         50.0000        
                78             9511           489            4.7917         4.8900         0.0000         50.0000        
                84             9462           538            5.4167         5.3800         0.0000         50.0000        
                90             9398           602            6.0417         6.0200         0.0000         50.0000        
                96             9366           634            6.6667         6.3400         0.0000         50.0000        
                102            9267           733            7.2917         7.3300         0.0000         50.0000        
                108            9212           788            7.9167         7.8800         0.0000         50.0000        
                114            9146           854            8.5417         8.5400         0.0000         50.0000        
                120            9102           898            9.1667         8.9800         0.0000         50.0000        
                126            8984           1016           9.7917         10.1600        0.0000         50.0000        
                132            0              10000          100.0000       100.0000       0.0000         50.0000        
                138            0              10000          100.0000       100.0000       0.0000         50.0000        
                144            0              10000          100.0000       100.0000       0.0000         50.0000        
-------------------------------------<pass>-------------------------------------

--------------------------------------------------------------------------------
functional test 2 : use several RED configurations,
                    increase average queue size to just below maximum threshold,
                    compare drop rate to drop probability

RED config     avg queue size min threshold  max threshold  drop prob %    drop rate %    diff %         tolerance %    
0              127            32             128            9.8958         10.0100        0.0000         50.0000        
1              127            32             128            4.9479         4.9700         0.0000         50.0000        
2              127            32             128            3.2986         2.6800         0.0000         50.0000        
3              127            32             128            2.4740         1.7000         0.0000         50.0000        
4              127            32             128            1.9792         1.2700         0.0000         50.0000        
5              127            32             128            1.6493         1.0500         0.0000         50.0000        
6              127            32             128            1.4137         0.8100         0.0000         50.0000        
7              127            32             128            1.2370         0.7100         0.0000         50.0000        
8              127            32             128            1.0995         0.6200         0.0000         50.0000        
9              127            32             128            0.9896         0.5600         0.0000         50.0000        
-------------------------------------<pass>-------------------------------------

--------------------------------------------------------------------------------
functional test 3 : use one RED configuration,
                    increase average queue size to target level,
                    dequeue all packets until queue is empty,
                    confirm that average queue size is computed correctly while queue is empty

q avg before   q avg after    expected       difference %   tolerance %    result        
1022.0000      1022.0000      1016.0627      0.5843         5.0000         pass           
1022.0000      1022.0000      1016.0627      0.5843         5.0000         pass           
1022.0000      1022.0000      1016.0627      0.5843         5.0000         pass           
1022.0000      1022.0000      1016.0627      0.5843         5.0000         pass           
1022.0000      1022.0000      1016.0627      0.5843         5.0000         pass           
-------------------------------------<pass>-------------------------------------

--------------------------------------------------------------------------------
functional test 5 : use several queues (each with its own run-time data),
                    use several RED configurations (such that each configuration is shared by multiple queues),
                    increase average queue size to just below maximum threshold,
                    compare drop rate to drop probability,
                    (this is a larger scale version of functional test 2)

queue          config         avg queue size min threshold  max threshold  drop prob %    drop rate %    diff %         tolerance %    
0              0              127            32             128            9.8958         9.9200         0.0000         50.0000        
1              0              127            32             128            9.8958         9.9700         0.0000         50.0000        
2              1              127            32             128            4.9479         4.8600         0.0000         50.0000        
3              1              127            32             128            4.9479         4.9400         0.0000         50.0000        
-------------------------------------<pass>-------------------------------------

--------------------------------------------------------------------------------
functional test 6 : use several queues (each with its own run-time data),
                    use several RED configurations (such that each configuration is shared by multiple queues),
                    increase average queue size to target level,
                    dequeue all packets until queue is empty,
                    confirm that average queue size is computed correctly while queue is empty
                    (this is a larger scale version of functional test 3)

queue          config         q avg before   q avg after    expected       difference %   tolerance %    result  
0              0              1022.0000      1022.0000      1016.0627      0.5843         5.0000         pass           
1              0              1022.0000      1022.0000      1016.0627      0.5843         5.0000         pass           
2              1              1022.0000      1022.0000      1010.1483      1.1733         5.0000         pass           
3              1              1022.0000      1022.0000      1010.1483      1.1733         5.0000         pass           
-------------------------------------<pass>-------------------------------------

--------------------------------------------------------------------------------
overflow test 1 : use one RED configuration,
                  increase average queue size to target level,
                  check maximum number of bits requirte_red to represent avg_s

avg queue size  wq_log2  fraction bits  max queue avg  num bits  enqueued  dropped   drop prob %  drop rate %  
1023            12       10             0xffc00000     32        0         941366    100.00       100.00       
-------------------------------------<pass>-------------------------------------
[total: 6, pass: 6]
Test OK
RTE>>quit


Kind Regards,
Wojtek


-----Original Message-----
From: Liguzinski, WojciechX 
Sent: Monday, November 15, 2021 6:27 PM
To: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; David Marchand <david.marchand@redhat.com>; Lincoln Lavoie <lylavoie@iol.unh.edu>; Ajmera, Megha <megha.ajmera@intel.com>; Singh, Jasvinder <jasvinder.singh@intel.com>
Cc: dev <dev@dpdk.org>; Aaron Conole <aconole@redhat.com>; Thomas Monjalon <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; ci@dpdk.org
Subject: RE: [dpdk-dev] [Bug 826] red_autotest random failures

Hi,

Sure, I will have a look.

Best Regards,
Wojciech


-----Original Message-----
From: Dumitrescu, Cristian <cristian.dumitrescu@intel.com> 
Sent: Monday, November 15, 2021 12:51 PM
To: David Marchand <david.marchand@redhat.com>; Lincoln Lavoie <lylavoie@iol.unh.edu>; Liguzinski, WojciechX <wojciechx.liguzinski@intel.com>; Ajmera, Megha <megha.ajmera@intel.com>; Singh, Jasvinder <jasvinder.singh@intel.com>
Cc: dev <dev@dpdk.org>; Aaron Conole <aconole@redhat.com>; Thomas Monjalon <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; ci@dpdk.org
Subject: RE: [dpdk-dev] [Bug 826] red_autotest random failures



> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Friday, November 12, 2021 2:16 PM
> To: Lincoln Lavoie <lylavoie@iol.unh.edu>
> Cc: Dumitrescu, Cristian <cristian.dumitrescu@intel.com>; dev 
> <dev@dpdk.org>; Aaron Conole <aconole@redhat.com>; Thomas Monjalon 
> <thomas@monjalon.net>; Yigit, Ferruh <ferruh.yigit@intel.com>; 
> ci@dpdk.org
> Subject: Re: [dpdk-dev] [Bug 826] red_autotest random failures
> 
> On Fri, Nov 12, 2021 at 3:11 PM Lincoln Lavoie <lylavoie@iol.unh.edu> wrote:
> >> This failure keeps on popping in the CI.
> >> The bug report is one month old, with no reply.
> >>
> >>
> >> I sent a proposal of removing red_autotest from the list executed 
> >> by the
> CI.
> >> https://patchwork.dpdk.org/project/dpdk/patch/20211027140458.2502-
> 2-david.marchand@redhat.com/
> >>
> >> It might be the best solution waiting for an analysis.
> >>
> >>
> >> --
> >> David Marchand
> >>
> >
> > Hi David,
> >
> > My understanding is, removing the test would require removing it 
> > from the
> DPDK unit tests, we are just running the fast-tests suite for the unit tests.
> DPDK's unit test structure / framework does not allow removing or 
> customizing the suite of tests beyond the suites.
> 
> https://patchwork.dpdk.org/project/dpdk/patch/20211027140458.2502-2-
> david.marchand@redhat.com/
> 
> 
> >
> > In the lab, Brandon has been looking into and trying different
> configurations for running the tests within the containers along the 
> lines of the CPU pinning requirements that might be assumed by the 
> unit tests. So far, everything he has tried has still had the similar 
> failures / issues.  We are still looking into it, so the bug is not 
> sitting without action, just no final resolution.
> 
> The mail I sent was not a comment for the investigation on UNH side.
> The ask is for Cristian to have a look too.
> 
> 
> --
> David Marchand

Wojciech, Megha,

Are you able to take a look at why is the RED autotest failing, please?

Thanks,
Cristian



  reply	other threads:[~2021-11-18 22:10 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <bug-826-3@http.bugs.dpdk.org/>
2021-11-12 13:51 ` David Marchand
2021-11-12 14:10   ` Lincoln Lavoie
2021-11-12 14:15     ` David Marchand
2021-11-15 11:51       ` Dumitrescu, Cristian
2021-11-15 17:26         ` Liguzinski, WojciechX
2021-11-18 22:10           ` Liguzinski, WojciechX [this message]
2021-11-19  7:26             ` Thomas Monjalon
2021-11-19 16:53               ` Dumitrescu, Cristian
2021-11-19 17:25                 ` Lincoln Lavoie
     [not found]                   ` <BN9PR11MB53729251C262EEBB1134A61194619@BN9PR11MB5372.namprd11.prod.outlook.com>
2021-11-29 17:58                     ` Brandon Lo
2021-11-30  7:51                       ` Liguzinski, WojciechX
2021-12-10 13:31                         ` Liguzinski, WojciechX
     [not found]                     ` <SA0PR11MB46708D32B6B2EC31D3DCE17F975A9@SA0PR11MB4670.namprd11.prod.outlook.com>
     [not found]                       ` <BY5PR11MB3926999DD139D10AD76D177F8F5B9@BY5PR11MB3926.namprd11.prod.outlook.com>
     [not found]                         ` <BY5PR11MB39261E9379E18C67BB4FB9938F5B9@BY5PR11MB3926.namprd11.prod.outlook.com>
     [not found]                           ` <BY5PR11MB3926DF1466F5815D5D2FEC798F259@BY5PR11MB3926.namprd11.prod.outlook.com>
     [not found]                             ` <CAOE1vsPcKAiTMPGH1VYwoTccWi7b=9DJdObdPJZhKQvqNQsFmw@mail.gmail.com>
2022-02-02 14:51                               ` Brandon Lo
2022-02-02 17:07                                 ` Danilewicz, MarcinX
2022-02-03 23:31                                   ` Danilewicz, MarcinX
2022-02-04  0:11                                     ` Brandon Lo
2022-03-09 10:01                                       ` Danilewicz, MarcinX
2022-03-09 14:48                                         ` Brandon Lo
2022-03-10 17:25                                           ` Danilewicz, MarcinX
2021-11-22  8:17                 ` David Marchand
2021-11-22 13:34                   ` Lincoln Lavoie

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BN9PR11MB53724D41DC05875D1D7F8E12949B9@BN9PR11MB5372.namprd11.prod.outlook.com \
    --to=wojciechx.liguzinski@intel.com \
    --cc=aconole@redhat.com \
    --cc=annax.zegota@intel.com \
    --cc=ci@dpdk.org \
    --cc=cristian.dumitrescu@intel.com \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=ferruh.yigit@intel.com \
    --cc=jasvinder.singh@intel.com \
    --cc=lylavoie@iol.unh.edu \
    --cc=megha.ajmera@intel.com \
    --cc=thomas@monjalon.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).