RE: [PATCH v7 0/7] Introduce support for LoongArch architecture

[Ali Alnubani <alialnu@nvidia.com>]

> -----Original Message-----
> From: David Marchand <david.marchand@redhat.com>
> Sent: Friday, September 30, 2022 5:21 PM
> To: zhoumin <zhoumin@loongson.cn>
> Cc: NBU-Contact-Thomas Monjalon (EXTERNAL) <thomas@monjalon.net>;
> bruce.richardson@intel.com; anatoly.burakov@intel.com;
> qiming.yang@intel.com; Yuying.Zhang@intel.com; jgrajcia@cisco.com;
> konstantin.v.ananyev@yandex.ru; dev@dpdk.org; maobibo@loongson.cn;
> Aaron Conole <aconole@redhat.com>; Ali Alnubani <alialnu@nvidia.com>;
> dpdklab <dpdklab@iol.unh.edu>; ci@dpdk.org
> Subject: Re: [PATCH v7 0/7] Introduce support for LoongArch architecture
> 
> On Fri, Sep 30, 2022 at 12:05 PM zhoumin <zhoumin@loongson.cn> wrote:
> > On Fri, Sep 30, 2022 at 16:13, David Marchand wrote:
> > > On Fri, Sep 30, 2022 at 10:02 AM Min Zhou <zhoumin@loongson.cn>
> wrote:
> > >>
> > >> The online documents of LoongArch architecture are here:
> > >>      https://loongson.github.io/LoongArch-Documentation/README-
> EN.html
> > >>
> > >> The latest build tools for LoongArch (binary) can be downloaded from:
> > >>      https://github.com/loongson/build-tools
> > > Could you confirm which sources have been used to generate it? and
> > > instructions to compile it?
> >
> > Only the cross compiler [1] is required. The instructions can be found in
> > the new added file cross_build_dpdk_for_loongarch.rst. I had added the
> > CI job for cross compiling DPDK for LoongArch in patch v7 7/7. The CI job
> > can run successfully if without the GCC warning caused by vhost.
> 
> - Sorry, but those instructions are not useful.
> 
> Is this architecture support in upstream gcc not functional?
> 
> Maybe I missed the information.. I spent some time at the different
> links in the docs and in github, but I always end up with a set of
> headers, or binaries, and no reference to the exact sources that were
> used.
> I have limited trust in binaries uploaded somewhere in github.
> I don't want to spend more time on this.
> 
> What I ask for, is clear instructions how to get the toolchain
> sources, and how to generate this toolchain.
> 
> 
> - About the vhost compilation issue, a fix in the same area of the
> code is in progress.
> It will take some time to get the fix.
> I will postpone merging the last patch until the vhost fix is ready.
> (I am rather confident all of this will be resolved by the time 22.11
> is released).
> 
> 
> >
> > >> v7:
> > >>      - rebase the patchset on the main repository
> > >>      - add errno.h to rte_power_intrinsics.c according with
> > >>        commit 72b452c5f259
> > > Thanks, I will look at this last revision.
> > >
> > >
> > > There is still one aspect that is unclear to me.
> > > How will the DPDK community make sure changes won't break this
> > > architecture? (I mean, runtime checks, not only compilation)
> > > IOW, what do you plan to hook to our CI to test patches submitted to
> > > the mailing list?
> >
> > We can send our machine to UNH lab, but it may take a long time.
> >
> > GHA seems to be a good choice. However, I found that the codes of CI
> > runner of GHA [2] are arch-specific. So the CI runner currently cannot
> > run on
> > LoongArch machine.
> 
> I see.
> 
> The better solution is probably to go with "your" own CI so that that
> LoongArch has runtime non regression (functional and performance)
> tests.
> See below.
> 
> 
> >
> > Are there other CI clients which are not arch-specific and can be used
> > for DPDK?
> > We can provide machines accessible by the public network. These
> machines run
> > Loongnix-server system which was built based on the source rpms of
> CentOS 8.
> > We can deploy DPDK CI client on these machines.
> 
> There is no "DPDK CI client" per se.
> 
> The DPDK project has a distributed CI made of at least 3 CI entities.
> 
> Those CI test patches and post reports via mail: the ovsrobot, Intel
> CI and UNH lab.
> A CI retrieves patches from patchwork, a set of scripts is available
> in https://git.dpdk.org/tools/dpdk-ci/ (especially the poll-pw
> script).
> 
> Then the way the patches are tested is something each CI handles on its side:
> - the ovsrobot creates a branch per series under the ovsrobot/dpdk
> github repository, and let GitHub action run (this is how your current
> series has been tested in GHA),
> - Intel CI have their own tool for which I have little detail,
> - UNH lab have their infrastructure too, using some Jenkins iirc. They
> provide a dashboard for reports
> https://lab.dpdk.org/results/dashboard/ and to get all details and
> logs.
> 
> The common point is that, at the end of testing a series, a test
> report is sent to the (sender-restricted) test-report@ mailing list.
> 
> Those reports could be done per patch, but given the amount of patches
> on the dev@ mailing list, the consensus is to test the whole series
> and report back against the last patch of a series.
> 
> All of this is gathered by patchwork (the details of how it is done
> are not 100% clear to me, maybe Ali can confirm later if a
> modification is required).

A few more things to add:
Labs can either use "dpdk-ci:tools/poll-pw" to pull the patches/patchsets from the Events API endpoint (https://patches.dpdk.org/api/events), or they can use their own scripts. Events API objects should be filtered by the categories "patch-completed" or "series-completed".
The script "dpdk-ci:tools/send-patch-report.sh" can be used to send reports to the mailing list in the expected report format.
The dpdk.org servers take care of adding the report results to Patchwork once they are in the test-report archives.

> 
> If you look at your v7 series, you will see:
> https://patchwork.dpdk.org/project/dpdk/list/?series=24929&state=%2A&a
> rchive=both
> - ovsrobot: ci/github-robot link
> http://mails.dpdk.org/archives/test-report/2022-September/310836.html
> - Intel CI: ci/Intel-* links, for example on the compilation test
> http://mails.dpdk.org/archives/test-report/2022-September/310822.html
> - UNH lab: all ci/iol-* links, for example on the compilation test
> http://mails.dpdk.org/archives/test-report/2022-September/310834.html
> 
> So what LoongSoon could do is setup some Loongnix systems with a
> similar infrastructure and provide (native?) compilation and runtime
> test reports.
> 
> I Cc'd a few people involved in all this.
> And there is the ci@ mailing list where all CI people can discuss.
> 
> 
> --
> David Marchand