Re: [dpdk-ci] [PATCH v2 10/10] tools: skip the IDs we already fetched

[Thomas Monjalon <thomas@monjalon.net>]

11/10/2021 21:30, Ali Alnubani:
> From: Thomas Monjalon <thomas@monjalon.net>
> > 21/09/2021 16:35, alialnu@nvidia.com:
> > > From: Ali Alnubani <alialnu@nvidia.com>
> > >
> > > Store the IDs we already fetched in a file and don't
> > > run 'callcmd' again for them.
> > 
> > We store all IDs. Should we manually remove olds one from time to time?
> > 
> 
> Do you have a suggestion for when should we clear this file? Maybe each time the script starts?

Yes at each start, we can remove the very old entries, like more than 10 hours old.

> > We need an explanation about the strategy, why it is needed.
> > I think it is because filtering by date is not enough.
> > In order to not miss any patch, we should request a date earlier
> > than the previous fetch and skip those already fetched.
> 
> The reason this change was made isn't because filtering by date is not enough, it's because
> I want to avoid feeding the same ID to 'callcmd' more than once.
> This can happen if a patchwork ID was created between recording date_now and fetching the API.
> I don't think we are missing any IDs, even without this change.
> 
> > Where the "earlier date" is defined?
> 
> There are 2 variables, "date_now", which is recorded right before fetching from the API, and then gets written
> to the file, and "since", which is the last date that was written to the file.

OK please update the commit log.