[Bug 937] [dpdk-22.03] unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest

[Bug 937] [dpdk-22.03] unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest

[bugzilla]

https://bugs.dpdk.org/show_bug.cgi?id=937

            Bug ID: 937
           Summary: [dpdk-22.03] unit_tests_mempool/test_mempool_perf:
                    timeout on mempool_perf_autotest
           Product: DPDK
           Version: unspecified
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: normal
          Priority: Normal
         Component: other
          Assignee: dev@dpdk.org
          Reporter: linglix.chen@intel.com
  Target Milestone: ---

dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40

Reproduce Step:

1.launch dpdk-test
x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4
2. RTE>>mempool_perf_autotest

Expect results: mempool_perf_autotest timeout in 2000s.

Actual results: mempool_perf_autotest need run 1 or more hours.

Is this issue a regression: Y
Version the regression was introduced: Specify git id if known.
 First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
Author: Morten Brørup <mb@smartsharesystems.com>
Date: Mon Jan 24 15:59:53 2022 +0100

mempool: test performance with constant n

"What gets measured gets done."

This patch adds mempool performance tests where the number of objects to
put and get is constant at compile time, which may significantly improve
the performance of these functions. [*]

Also, it is ensured that the array holding the object used for testing
is cache line aligned, for maximum performance.

And finally, the following entries are added to the list of tests:
- Number of kept objects: 512
- Number of objects to get and to put: The number of pointers fitting
into a cache line, i.e. 8 or 16

[*] Some example performance test (with cache) results:

get_bulk=4 put_bulk=4 keep=128 constant_n=false rate_persec=280480972
get_bulk=4 put_bulk=4 keep=128 constant_n=true rate_persec=622159462

get_bulk=8 put_bulk=8 keep=128 constant_n=false rate_persec=477967155
get_bulk=8 put_bulk=8 keep=128 constant_n=true rate_persec=917582643

get_bulk=32 put_bulk=32 keep=32 constant_n=false rate_persec=871248691
get_bulk=32 put_bulk=32 keep=32 constant_n=true rate_persec=1134021836

Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>

-- 
You are receiving this mail because:
You are the assignee for the bug.

RE: [Bug 937] [dpdk-22.03] unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest

[Morten Brørup]

> From: bugzilla@dpdk.org [mailto:bugzilla@dpdk.org]
> Sent: Thursday, 17 February 2022 10.11
> 
> https://bugs.dpdk.org/show_bug.cgi?id=937
> 

[...]


> 
> dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
> 
> Reproduce Step:
> 
> 1.launch dpdk-test
> x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4
> 2. RTE>>mempool_perf_autotest
> 
> Expect results: mempool_perf_autotest timeout in 2000s.
> 
> Actual results: mempool_perf_autotest need run 1 or more hours.
> 
> Is this issue a regression: Y
> Version the regression was introduced: Specify git id if known.
>  First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
> Author: Morten Brørup <mb@smartsharesystems.com>
> Date: Mon Jan 24 15:59:53 2022 +0100
> 
> mempool: test performance with constant n
> 
> "What gets measured gets done."
> 
> This patch adds mempool performance tests where the number of objects
> to
> put and get is constant at compile time, which may significantly
> improve
> the performance of these functions. [*]
> 
> Also, it is ensured that the array holding the object used for testing
> is cache line aligned, for maximum performance.
> 
> And finally, the following entries are added to the list of tests:
> - Number of kept objects: 512
> - Number of objects to get and to put: The number of pointers fitting
> into a cache line, i.e. 8 or 16
> 

A few more test cases were added, and each test is now run in two variants, one variant with variable N, and one variant with constant N.

To reduce the very long test running time, we could consider removing some of the tests. However, a few undocumented magic numbers are used throughout DPDK, e.g. a burst size of 32. These magic numbers do not have names, but are just used numerically. So I am not sure if we can safely remove the tests with burst size 4 and/or 128 kept objects. E.g. I suspect that 4 could be a magic number in some NIC drivers, and thus still relevant to test.


> [*] Some example performance test (with cache) results:
> 
> get_bulk=4 put_bulk=4 keep=128 constant_n=false rate_persec=280480972
> get_bulk=4 put_bulk=4 keep=128 constant_n=true rate_persec=622159462
> 
> get_bulk=8 put_bulk=8 keep=128 constant_n=false rate_persec=477967155
> get_bulk=8 put_bulk=8 keep=128 constant_n=true rate_persec=917582643
> 
> get_bulk=32 put_bulk=32 keep=32 constant_n=false rate_persec=871248691
> get_bulk=32 put_bulk=32 keep=32 constant_n=true rate_persec=1134021836
> 
> Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> 
> --
> You are receiving this mail because:
> You are the assignee for the bug.

RE: [Bug 937] [dpdk-22.03] unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest

[Jiang, YuX]

> -----Original Message-----
> From: Morten Brørup <mb@smartsharesystems.com>
> Sent: Thursday, February 17, 2022 6:16 PM
> To: bugzilla@dpdk.org; dev@dpdk.org
> Subject: RE: [Bug 937] [dpdk-22.03]
> unit_tests_mempool/test_mempool_perf: timeout on
> mempool_perf_autotest
> 
> > From: bugzilla@dpdk.org [mailto:bugzilla@dpdk.org]
> > Sent: Thursday, 17 February 2022 10.11
> >
> > https://bugs.dpdk.org/show_bug.cgi?id=937
> >
> 
> [...]
> 
> 
> >
> > dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
> >
> > Reproduce Step:
> >
> > 1.launch dpdk-test
> > x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4 2.
> > RTE>>mempool_perf_autotest
> >
> > Expect results: mempool_perf_autotest timeout in 2000s.
> >
> > Actual results: mempool_perf_autotest need run 1 or more hours.
> >
> > Is this issue a regression: Y
> > Version the regression was introduced: Specify git id if known.
> >  First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
> > Author: Morten Brørup <mb@smartsharesystems.com>
> > Date: Mon Jan 24 15:59:53 2022 +0100
> >
> > mempool: test performance with constant n
> >
> > "What gets measured gets done."
> >
> > This patch adds mempool performance tests where the number of objects
> > to put and get is constant at compile time, which may significantly
> > improve the performance of these functions. [*]
> >
> > Also, it is ensured that the array holding the object used for testing
> > is cache line aligned, for maximum performance.
> >
> > And finally, the following entries are added to the list of tests:
> > - Number of kept objects: 512
> > - Number of objects to get and to put: The number of pointers fitting
> > into a cache line, i.e. 8 or 16
> >
> 
> A few more test cases were added, and each test is now run in two variants,
> one variant with variable N, and one variant with constant N.
> 
> To reduce the very long test running time, we could consider removing some
> of the tests. However, a few undocumented magic numbers are used
> throughout DPDK, e.g. a burst size of 32. These magic numbers do not have
> names, but are just used numerically. So I am not sure if we can safely
> remove the tests with burst size 4 and/or 128 kept objects. E.g. I suspect that
> 4 could be a magic number in some NIC drivers, and thus still relevant to test.
> 
> 
Do you mean that this is normal behavior due to this merged patch? How long this mempool_perf_autotest will take? 1 more hours or several hours?

> > [*] Some example performance test (with cache) results:
> >
> > get_bulk=4 put_bulk=4 keep=128 constant_n=false
> rate_persec=280480972
> > get_bulk=4 put_bulk=4 keep=128 constant_n=true rate_persec=622159462
> >
> > get_bulk=8 put_bulk=8 keep=128 constant_n=false
> rate_persec=477967155
> > get_bulk=8 put_bulk=8 keep=128 constant_n=true rate_persec=917582643
> >
> > get_bulk=32 put_bulk=32 keep=32 constant_n=false
> rate_persec=871248691
> > get_bulk=32 put_bulk=32 keep=32 constant_n=true
> rate_persec=1134021836
> >
> > Signed-off-by: Morten Brørup <mb@smartsharesystems.com>
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> >
> > --
> > You are receiving this mail because:
> > You are the assignee for the bug.

RE: [Bug 937] [dpdk-22.03] unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest

[Morten Brørup]

> From: Jiang, YuX [mailto:yux.jiang@intel.com]
> Sent: Thursday, 17 February 2022 11.31
> 
> > From: Morten Brørup <mb@smartsharesystems.com>
> > Sent: Thursday, February 17, 2022 6:16 PM
> >
> > > From: bugzilla@dpdk.org [mailto:bugzilla@dpdk.org]
> > > Sent: Thursday, 17 February 2022 10.11
> > >
> > > https://bugs.dpdk.org/show_bug.cgi?id=937
> > >
> >
> > [...]
> >
> >
> > >
> > > dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
> > >
> > > Reproduce Step:
> > >
> > > 1.launch dpdk-test
> > > x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4 2.
> > > RTE>>mempool_perf_autotest
> > >
> > > Expect results: mempool_perf_autotest timeout in 2000s.
> > >
> > > Actual results: mempool_perf_autotest need run 1 or more hours.
> > >
> > > Is this issue a regression: Y
> > > Version the regression was introduced: Specify git id if known.
> > >  First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
> > > Author: Morten Brørup <mb@smartsharesystems.com>
> > > Date: Mon Jan 24 15:59:53 2022 +0100
> > >
> > > mempool: test performance with constant n
> > >
> > > "What gets measured gets done."
> > >
> > > This patch adds mempool performance tests where the number of
> objects
> > > to put and get is constant at compile time, which may significantly
> > > improve the performance of these functions. [*]
> > >
> > > Also, it is ensured that the array holding the object used for
> testing
> > > is cache line aligned, for maximum performance.
> > >
> > > And finally, the following entries are added to the list of tests:
> > > - Number of kept objects: 512
> > > - Number of objects to get and to put: The number of pointers
> fitting
> > > into a cache line, i.e. 8 or 16
> > >
> >
> > A few more test cases were added, and each test is now run in two
> variants,
> > one variant with variable N, and one variant with constant N.
> >
> > To reduce the very long test running time, we could consider removing
> some
> > of the tests. However, a few undocumented magic numbers are used
> > throughout DPDK, e.g. a burst size of 32. These magic numbers do not
> have
> > names, but are just used numerically. So I am not sure if we can
> safely
> > remove the tests with burst size 4 and/or 128 kept objects. E.g. I
> suspect that
> > 4 could be a magic number in some NIC drivers, and thus still
> relevant to test.
> >
> >
> Do you mean that this is normal behavior due to this merged patch? How
> long this mempool_perf_autotest will take? 1 more hours or several
> hours?

It was running for 2-3 hours on our server.

-Morten

RE: [Bug 937] [dpdk-22.03] unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest

[Jiang, YuX]

> -----Original Message-----
> From: Morten Brørup <mb@smartsharesystems.com>
> Sent: Friday, February 18, 2022 4:10 PM
> To: Jiang, YuX <yux.jiang@intel.com>; bugzilla@dpdk.org; dev@dpdk.org
> Subject: RE: [Bug 937] [dpdk-22.03]
> unit_tests_mempool/test_mempool_perf: timeout on
> mempool_perf_autotest
> 
> > From: Jiang, YuX [mailto:yux.jiang@intel.com]
> > Sent: Thursday, 17 February 2022 11.31
> >
> > > From: Morten Brørup <mb@smartsharesystems.com>
> > > Sent: Thursday, February 17, 2022 6:16 PM
> > >
> > > > From: bugzilla@dpdk.org [mailto:bugzilla@dpdk.org]
> > > > Sent: Thursday, 17 February 2022 10.11
> > > >
> > > > https://bugs.dpdk.org/show_bug.cgi?id=937
> > > >
> > >
> > > [...]
> > >
> > >
> > > >
> > > > dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
> > > >
> > > > Reproduce Step:
> > > >
> > > > 1.launch dpdk-test
> > > > x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4 2.
> > > > RTE>>mempool_perf_autotest
> > > >
> > > > Expect results: mempool_perf_autotest timeout in 2000s.
> > > >
> > > > Actual results: mempool_perf_autotest need run 1 or more hours.
> > > >
> > > > Is this issue a regression: Y
> > > > Version the regression was introduced: Specify git id if known.
> > > >  First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
> > > > Author: Morten Brørup <mb@smartsharesystems.com>
> > > > Date: Mon Jan 24 15:59:53 2022 +0100
> > > >
> > > > mempool: test performance with constant n
> > > >
> > > > "What gets measured gets done."
> > > >
> > > > This patch adds mempool performance tests where the number of
> > objects
> > > > to put and get is constant at compile time, which may
> > > > significantly improve the performance of these functions. [*]
> > > >
> > > > Also, it is ensured that the array holding the object used for
> > testing
> > > > is cache line aligned, for maximum performance.
> > > >
> > > > And finally, the following entries are added to the list of tests:
> > > > - Number of kept objects: 512
> > > > - Number of objects to get and to put: The number of pointers
> > fitting
> > > > into a cache line, i.e. 8 or 16
> > > >
> > >
> > > A few more test cases were added, and each test is now run in two
> > variants,
> > > one variant with variable N, and one variant with constant N.
> > >
> > > To reduce the very long test running time, we could consider
> > > removing
> > some
> > > of the tests. However, a few undocumented magic numbers are used
> > > throughout DPDK, e.g. a burst size of 32. These magic numbers do not
> > have
> > > names, but are just used numerically. So I am not sure if we can
> > safely
> > > remove the tests with burst size 4 and/or 128 kept objects. E.g. I
> > suspect that
> > > 4 could be a magic number in some NIC drivers, and thus still
> > relevant to test.
> > >
> > >
> > Do you mean that this is normal behavior due to this merged patch? How
> > long this mempool_perf_autotest will take? 1 more hours or several
> > hours?
> 
> It was running for 2-3 hours on our server.
> 
> -Morten
Thanks Morten, so you mean this is normal behavior, right?

RE: [Bug 937] [dpdk-22.03] unit_tests_mempool/test_mempool_perf: timeout on mempool_perf_autotest

[Morten Brørup]

> From: Jiang, YuX [mailto:yux.jiang@intel.com]
> Sent: Friday, 18 February 2022 10.52
> 
> > From: Morten Brørup <mb@smartsharesystems.com>
> > Sent: Friday, February 18, 2022 4:10 PM
> >
> > > From: Jiang, YuX [mailto:yux.jiang@intel.com]
> > > Sent: Thursday, 17 February 2022 11.31
> > >
> > > > From: Morten Brørup <mb@smartsharesystems.com>
> > > > Sent: Thursday, February 17, 2022 6:16 PM
> > > >
> > > > > From: bugzilla@dpdk.org [mailto:bugzilla@dpdk.org]
> > > > > Sent: Thursday, 17 February 2022 10.11
> > > > >
> > > > > https://bugs.dpdk.org/show_bug.cgi?id=937
> > > > >
> > > >
> > > > [...]
> > > >
> > > >
> > > > >
> > > > > dpdk-22.03-rc1: 2e07139b66a810883871ff20a5f31e4c222e5b40
> > > > >
> > > > > Reproduce Step:
> > > > >
> > > > > 1.launch dpdk-test
> > > > > x86_64-native-linuxapp-gcc/app/test/dpdk-test -l 1-111 -n 4 2.
> > > > > RTE>>mempool_perf_autotest
> > > > >
> > > > > Expect results: mempool_perf_autotest timeout in 2000s.
> > > > >
> > > > > Actual results: mempool_perf_autotest need run 1 or more hours.
> > > > >
> > > > > Is this issue a regression: Y
> > > > > Version the regression was introduced: Specify git id if known.
> > > > >  First bad commit:ed579e50c66db90485593c6c79f5366a57145f2a
> > > > > Author: Morten Brørup <mb@smartsharesystems.com>
> > > > > Date: Mon Jan 24 15:59:53 2022 +0100
> > > > >
> > > > > mempool: test performance with constant n
> > > > >
> > > > > "What gets measured gets done."
> > > > >
> > > > > This patch adds mempool performance tests where the number of
> > > objects
> > > > > to put and get is constant at compile time, which may
> > > > > significantly improve the performance of these functions. [*]
> > > > >
> > > > > Also, it is ensured that the array holding the object used for
> > > testing
> > > > > is cache line aligned, for maximum performance.
> > > > >
> > > > > And finally, the following entries are added to the list of
> tests:
> > > > > - Number of kept objects: 512
> > > > > - Number of objects to get and to put: The number of pointers
> > > fitting
> > > > > into a cache line, i.e. 8 or 16
> > > > >
> > > >
> > > > A few more test cases were added, and each test is now run in two
> > > variants,
> > > > one variant with variable N, and one variant with constant N.
> > > >
> > > > To reduce the very long test running time, we could consider
> > > > removing
> > > some
> > > > of the tests. However, a few undocumented magic numbers are used
> > > > throughout DPDK, e.g. a burst size of 32. These magic numbers do
> not
> > > have
> > > > names, but are just used numerically. So I am not sure if we can
> > > safely
> > > > remove the tests with burst size 4 and/or 128 kept objects. E.g.
> I
> > > suspect that
> > > > 4 could be a magic number in some NIC drivers, and thus still
> > > relevant to test.
> > > >
> > > >
> > > Do you mean that this is normal behavior due to this merged patch?
> How
> > > long this mempool_perf_autotest will take? 1 more hours or several
> > > hours?
> >
> > It was running for 2-3 hours on our server.
> >
> > -Morten
> Thanks Morten, so you mean this is normal behavior, right?

Yes, unfortunately.