Bug ID 1507
Summary Fixed test traffic patterns make test results unreliable
Product DTS
Version unspecified
Hardware All
OS All
Status UNCONFIRMED
Severity major
Priority Normal
Component tests
Assignee dts@dpdk.org
Reporter mb@smartsharesystems.com
Target Milestone ---

It has recently come to my attention that a lot of testing is performed with
fixed packet size and type.

This has two bad consequences:

1. The same code path in the SUT is taken every time.
This is bad, because it reduces the code coverage of functional testing.
Worst case, newly added code (intended to be tested) may be in a non-taken code
path, and thus not tested at all.

2. When the same code path is taken every time, the CPU's branch predictor
achieves near-perfect hit rate, and thus the performance test results do not
reflect real-life performance.
This is very bad, because performance results are often used to quantify the
benefit of suggested performance optimizations, and they might in fact have the
opposite effect in real life (with real-life traffic mix).

Testing should be performed with a realistic mix of packets, as seen on the
internet.
For some cases, packet size matters, and for some cases, packet type matters.
          


You are receiving this mail because: