Re: [dpdk-ci] Community CI Meeting Minutes

[Thomas Monjalon <thomas@monjalon.net>]

Thanks for the explanation.
OK for extra warnings in DTS patchwork only.


01/09/2021 14:49, Owen Hilyard:
> Hello Thomas,
> 
> I was the one who asked for that feature, and I wanted to explain my
> reasoning.
> 
> First, it is not possible to run every test in DTS for every patch with a
> reasonable amount of hardware. The last time I tried to run everything it
> took well over 2 days. As such, the DTS working group decided that it would
> be best to target our testing. I wrote a script that walks the python
> module paths and will use the files changed in a patch to determine which
> DTS tests could possibly be affected by the change. However, since many of
> the tests in DTS require special configuration or particular hardware, the
> Intel Lab has a limited set of tests that can be run. I asked that these
> warnings be sent out for a test that could be affected by a change, but
> which was not run. In part, this is due to a situation that can occur like
> with a recent patch I made to the rte flow test suite, where I basically
> rewrote a test suite but it was not tested in CI. All of the smoke tests
> pass, so it looks fine, but given how DTS is set up, I could have placed
> code that wouldn't compile in the test suite and it would still have passed
> CI. Since we don't have a good way to prevent that, these notifications are
> a way for maintainers to quickly check if there were changes in a test
> suite that was not tested, and either manually test the patch or pay
> special attention to it.
> 
> As for your concern about noise in patchwork, this would be run on DTS
> patches only, and would not report onto the main DPDK section. There is
> very little going on in the DTS patchworks right now (many patches have no
> tests at all) and this provides valuable information for maintainers to
> help streamline patch review. The only time this would generate more than 1
> or 2 warnings would be if a change is made to somewhere very deep in the
> DTS framework. When that happens, I personally would prefer that it
> generates a lot of output and makes itself noticable, since framework level
> changes need to be very carefully reviewed to avoid breaking DPDK CI.
> 
> Owen Hilyard
> 
> On Tue, Aug 31, 2021 at 11:47 AM Thomas Monjalon <thomas@monjalon.net>
> wrote:
> 
> > Then it should not be reported in patchwork.
> > Please let's not add more noise in patchwork.
> >
> >
> > 26/08/2021 18:27, Lincoln Lavoie:
> > > This is specific to patches for DTS, where Intel doesn't have the
> > > infrastructure to run every possible test suite that is included in DTS.
> > > So, the warning is a notice to the submitter and the maintainer(s) the
> > > patch couldn't be tested. It's not really an issue related to the content
> > > of the patch itself (i.e. not about a breaking change or something like
> > > that).
> > >
> > > Cheers,
> > > Lincoln
> > >
> > > On Thu, Aug 26, 2021 at 12:05 PM Thomas Monjalon <thomas@monjalon.net>
> > > wrote:
> > >
> > > > 26/08/2021 15:33, Lincoln Lavoie:
> > > > > * For DTS CI, should authors be notified of skipped testing (i.e. CI
> > > > infrastructure doesn’t support that test suite)?  Should this be
> > marked as
> > > > a warning in patchwork?  Agreed it should do both of these actions,
> > notify
> > > > the author and mark the patch as warning in patchworks.
> > > >
> > > > Not sure to understand.
> > > > If a test is not supported, it is not an issue of the patch,
> > > > so why would it be reported in patchwork?