Hi Thomas, Adam and I discussed and I agree with his points. His "patchseries apply" script rewrite will already alleviate this issue to a significant degree, but it will be even better if we can also use next-* mirrors on GitHub. Do you know who I can contact about this? Also, as of today our LTS testing is now moved over to GitHub, which should help a little. On Mon, Nov 27, 2023 at 4:30 PM Adam Hassick wrote: > Hi Thomas, > > Yes, mirroring the next-* branches would help to reduce the complexity of > creating patch series artifacts. > As Patrick wrote, moving the clones for the periodic testing on DPDK main > and DPDK LTS should be a trivial change. > > However, most of the load we put on that server originates from our > testing on new patch series. > I am working on a new script for creating the tarball artifacts from > incoming patch series. It only pulls down either 1 or 2 repositories rather > than main and all next-* branches for every single patch. > Once deployed, I expect this script to reduce the load on git.dpdk.org > significantly. > Also, I anticipate that I can modify this new script to pull main down > from GitHub. Then, only incoming patch series for next-* branches will > produce any load on git.dpdk.org. > > Thanks, > Adam Hassick > > On Mon, Nov 27, 2023 at 3:58 PM Thomas Monjalon > wrote: > >> 27/11/2023 17:09, Patrick Robb: >> > Hello, >> > >> > Thanks for the heads up Thomas. For the Community Lab, as far as I can >> > tell, every git clone/fetch for DPDK main and DPDK LTS should be able >> to be >> > moved to the GitHub repo. >> > >> > On the other hand, I don't think next-* branches are mirrored on GitHub >> (at >> > least for now), so unless I'm mistaken, we will still have to pull from >> > git.dpdk.org. But, between DPDK main and LTS, it should still be a >> sizable >> > reduction on the load put on your server. I'll discuss with the team >> today >> > and we will update here. >> >> We should mirror next-* trees as well I think, good point. >> >> >> > On Mon, Nov 27, 2023 at 6:21 AM Thomas Monjalon >> wrote: >> > >> > > Hello, >> > > >> > > The server git.dpdk.org is not strong enough to hold all CI requests. >> > > When a script is doing a git fetch or clone, it is consuming workload >> on >> > > the server. >> > > Please could you favor the mirror on GitHub in any CI automation? >> > > https://github.com/DPDK/dpdk.git >> > > This mirror is updated as frequently as git.dpdk.org: >> > > We push to git.dpdk.org where a git hook is replicating the same on >> > > GitHub. >> > > >> > > Hope we can reduce the workload on our main server. >> > > Thank you everyone for collaborating. >> >> >> >> -- Patrick Robb Technical Service Manager UNH InterOperability Laboratory 21 Madbury Rd, Suite 100, Durham, NH 03824 www.iol.unh.edu