Hello Thomas,
I was the one who asked for that feature, and I wanted to explain my reasoning.
First, it is not possible to run every test in DTS for every patch with a reasonable amount of hardware. The last time I tried to run everything it took well over 2 days. As such, the DTS working group decided that it would be best to target our testing. I wrote a script that walks the python module paths and will use the files changed in a patch to determine which DTS tests could possibly be affected by the change. However, since many of the tests in DTS require special configuration or particular hardware, the Intel Lab has a limited set of tests that can be run. I asked that these warnings be sent out for a test that could be affected by a change, but which was not run. In part, this is due to a situation that can occur like with a recent patch I made to the rte flow test suite, where I basically rewrote a test suite but it was not tested in CI. All of the smoke tests pass, so it looks fine, but given how DTS is set up, I could have placed code that wouldn't compile in the test suite and it would still have passed CI. Since we don't have a good way to prevent that, these notifications are a way for maintainers to quickly check if there were changes in a test suite that was not tested, and either manually test the patch or pay special attention to it.
As for your concern about noise in patchwork, this would be run on DTS patches only, and would not report onto the main DPDK section. There is very little going on in the DTS patchworks right now (many patches have no tests at all) and this provides valuable information for maintainers to help streamline patch review. The only time this would generate more than 1 or 2 warnings would be if a change is made to somewhere very deep in the DTS framework. When that happens, I personally would prefer that it generates a lot of output and makes itself noticable, since framework level changes need to be very carefully reviewed to avoid breaking DPDK CI.
Owen Hilyard