* [dpdk-ci] What is the reliability of the community lab @ 2021-04-14 12:10 Thomas Monjalon 2021-04-15 18:26 ` [dpdk-ci] [dpdklab] " Brandon Lo 0 siblings, 1 reply; 4+ messages in thread From: Thomas Monjalon @ 2021-04-14 12:10 UTC (permalink / raw) To: dpdklab; +Cc: ci We still have some severe reliability issues with the community lab. One under investigation, waiting for a bisect, is making the lab crashing. Another one I would like to understand is about the ABI check. Why the test is failing (which is normal) in Travis and GitHub but passing in the community lab? See this example: https://patches.dpdk.org/project/dpdk/patch/1618283653-16510-6-git-send-email-xuemingl@nvidia.com/ ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-ci] [dpdklab] What is the reliability of the community lab 2021-04-14 12:10 [dpdk-ci] What is the reliability of the community lab Thomas Monjalon @ 2021-04-15 18:26 ` Brandon Lo 2021-04-15 18:33 ` Thomas Monjalon 0 siblings, 1 reply; 4+ messages in thread From: Brandon Lo @ 2021-04-15 18:26 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dpdklab, ci Hi Thomas, For the ABI check failure, we were working on a few migration-related changes that caused this issue. We have fixed the underlying cause and the rolling migration will be completed soon. The test will be rerun and reported back up to patchworks as well. We are also in the process of adding more monitoring for failure results that are not limited to only infrastructure failures, i.e. failures caused by a test's return value will alert us in a separate category which will allow us to validate the result. Thanks, Brandon On Wed, Apr 14, 2021 at 8:10 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > We still have some severe reliability issues with the community lab. > One under investigation, waiting for a bisect, is making the lab crashing. > > Another one I would like to understand is about the ABI check. > Why the test is failing (which is normal) in Travis and GitHub > but passing in the community lab? > > See this example: > https://patches.dpdk.org/project/dpdk/patch/1618283653-16510-6-git-send-email-xuemingl@nvidia.com/ > > -- Brandon Lo UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 blo@iol.unh.edu www.iol.unh.edu ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-ci] [dpdklab] What is the reliability of the community lab 2021-04-15 18:26 ` [dpdk-ci] [dpdklab] " Brandon Lo @ 2021-04-15 18:33 ` Thomas Monjalon 2021-04-16 21:03 ` Brandon Lo 0 siblings, 1 reply; 4+ messages in thread From: Thomas Monjalon @ 2021-04-15 18:33 UTC (permalink / raw) To: Brandon Lo; +Cc: dpdklab, ci, aconole 15/04/2021 20:26, Brandon Lo: > Hi Thomas, > > For the ABI check failure, we were working on a few migration-related > changes that caused this issue. We have fixed the underlying cause and > the rolling migration will be completed soon. The test will be rerun > and reported back up to patchworks as well. You should avoid running tests when the platform is not ready. It was giving false positives. > We are also in the process of adding more monitoring for failure > results that are not limited to only infrastructure failures, i.e. > failures caused by a test's return value will alert us in a separate > category which will allow us to validate the result. The lab seems stopped currently. There are issues for a week now. What are the news today? > On Wed, Apr 14, 2021 at 8:10 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > > We still have some severe reliability issues with the community lab. > > One under investigation, waiting for a bisect, is making the lab crashing. > > > > Another one I would like to understand is about the ABI check. > > Why the test is failing (which is normal) in Travis and GitHub > > but passing in the community lab? > > > > See this example: > > https://patches.dpdk.org/project/dpdk/patch/1618283653-16510-6-git-send-email-xuemingl@nvidia.com/ ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [dpdk-ci] [dpdklab] What is the reliability of the community lab 2021-04-15 18:33 ` Thomas Monjalon @ 2021-04-16 21:03 ` Brandon Lo 0 siblings, 0 replies; 4+ messages in thread From: Brandon Lo @ 2021-04-16 21:03 UTC (permalink / raw) To: Thomas Monjalon; +Cc: dpdklab, ci, Aaron Conole We have completed our rolling migration to containers, so all of our compilation jobs are running and catching up. Our team has also worked on getting the DTS side of things back up and running since the DTS issue had caused the pipelines to hang a bit. Thanks, Brandon On Thu, Apr 15, 2021 at 2:33 PM Thomas Monjalon <thomas@monjalon.net> wrote: > > 15/04/2021 20:26, Brandon Lo: > > Hi Thomas, > > > > For the ABI check failure, we were working on a few migration-related > > changes that caused this issue. We have fixed the underlying cause and > > the rolling migration will be completed soon. The test will be rerun > > and reported back up to patchworks as well. > > You should avoid running tests when the platform is not ready. > It was giving false positives. > > > We are also in the process of adding more monitoring for failure > > results that are not limited to only infrastructure failures, i.e. > > failures caused by a test's return value will alert us in a separate > > category which will allow us to validate the result. > > The lab seems stopped currently. > There are issues for a week now. > What are the news today? > > > > On Wed, Apr 14, 2021 at 8:10 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > > > > We still have some severe reliability issues with the community lab. > > > One under investigation, waiting for a bisect, is making the lab crashing. > > > > > > Another one I would like to understand is about the ABI check. > > > Why the test is failing (which is normal) in Travis and GitHub > > > but passing in the community lab? > > > > > > See this example: > > > https://patches.dpdk.org/project/dpdk/patch/1618283653-16510-6-git-send-email-xuemingl@nvidia.com/ > > > -- Brandon Lo UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 blo@iol.unh.edu www.iol.unh.edu On Thu, Apr 15, 2021 at 2:33 PM Thomas Monjalon <thomas@monjalon.net> wrote: > > 15/04/2021 20:26, Brandon Lo: > > Hi Thomas, > > > > For the ABI check failure, we were working on a few migration-related > > changes that caused this issue. We have fixed the underlying cause and > > the rolling migration will be completed soon. The test will be rerun > > and reported back up to patchworks as well. > > You should avoid running tests when the platform is not ready. > It was giving false positives. > > > We are also in the process of adding more monitoring for failure > > results that are not limited to only infrastructure failures, i.e. > > failures caused by a test's return value will alert us in a separate > > category which will allow us to validate the result. > > The lab seems stopped currently. > There are issues for a week now. > What are the news today? > > > > On Wed, Apr 14, 2021 at 8:10 AM Thomas Monjalon <thomas@monjalon.net> wrote: > > > > > > We still have some severe reliability issues with the community lab. > > > One under investigation, waiting for a bisect, is making the lab crashing. > > > > > > Another one I would like to understand is about the ABI check. > > > Why the test is failing (which is normal) in Travis and GitHub > > > but passing in the community lab? > > > > > > See this example: > > > https://patches.dpdk.org/project/dpdk/patch/1618283653-16510-6-git-send-email-xuemingl@nvidia.com/ > > > -- Brandon Lo UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 blo@iol.unh.edu www.iol.unh.edu ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-04-16 21:03 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-04-14 12:10 [dpdk-ci] What is the reliability of the community lab Thomas Monjalon 2021-04-15 18:26 ` [dpdk-ci] [dpdklab] " Brandon Lo 2021-04-15 18:33 ` Thomas Monjalon 2021-04-16 21:03 ` Brandon Lo
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).