https://bugs.dpdk.org/show_bug.cgi?id=1507

            Bug ID: 1507
           Summary: Fixed test traffic patterns make test results
                    unreliable
           Product: DTS
           Version: unspecified
          Hardware: All
                OS: All
            Status: UNCONFIRMED
          Severity: major
          Priority: Normal
         Component: tests
          Assignee: dts@dpdk.org
          Reporter: mb@smartsharesystems.com
  Target Milestone: ---

It has recently come to my attention that a lot of testing is performed with
fixed packet size and type.

This has two bad consequences:

1. The same code path in the SUT is taken every time.
This is bad, because it reduces the code coverage of functional testing.
Worst case, newly added code (intended to be tested) may be in a non-taken code
path, and thus not tested at all.

2. When the same code path is taken every time, the CPU's branch predictor
achieves near-perfect hit rate, and thus the performance test results do not
reflect real-life performance.
This is very bad, because performance results are often used to quantify the
benefit of suggested performance optimizations, and they might in fact have the
opposite effect in real life (with real-life traffic mix).

Testing should be performed with a realistic mix of packets, as seen on the
internet.
For some cases, packet size matters, and for some cases, packet type matters.

-- 
You are receiving this mail because:
You are the assignee for the bug.