[dpdk-users] Integration of DPDK into Bazel

[dpdk-users] Integration of DPDK into Bazel

[Jinkun Geng]

Hi, all.
Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community. The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.

It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.


Jinkun

Re: [dpdk-users] Integration of DPDK into Bazel

[Stephen Hemminger]

On Tue, 9 Mar 2021 01:32:16 +0000
Jinkun Geng <gjk1994@stanford.edu> wrote:

> Hi, all.
> Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community. The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> 
> It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> 
> 
> Jinkun

DPDK is on meson now. The core team is unlikely to change build systems again.

Re: [dpdk-users] Integration of DPDK into Bazel

[Jinkun Geng]

Too bad. :<
________________________________
From: Stephen Hemminger <stephen@networkplumber.org>
Sent: Monday, March 8, 2021 6:31 PM
To: Jinkun Geng <gjk1994@stanford.edu>
Cc: users@dpdk.org <users@dpdk.org>; ci@dpdk.org <ci@dpdk.org>
Subject: Re: [dpdk-users] Integration of DPDK into Bazel

On Tue, 9 Mar 2021 01:32:16 +0000
Jinkun Geng <gjk1994@stanford.edu> wrote:

> Hi, all.
> Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community. The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
>
> It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
>
>
> Jinkun

DPDK is on meson now. The core team is unlikely to change build systems again.

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

09/03/2021 05:11, Jinkun Geng:
> Too bad. :<

Why is it too bad?
How the choice of an internal build system
can affect other projects?


From: Stephen Hemminger <stephen@networkplumber.org>
> On Tue, 9 Mar 2021 01:32:16 +0000
> Jinkun Geng <gjk1994@stanford.edu> wrote:
> 
> > Hi, all.
> > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.

Why the DPDK community would support building with Bazel?
What is the benefit?
Bazel projects cannot just link with DPDK using pkg-config?


> > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> >
> > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> >
> > Jinkun
> 
> DPDK is on meson now. The core team is unlikely to change build systems again.

DPDK supports library standards for compiling, installing and linking.
What else is needed?

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Jinkun Geng]

For any project using bazel, if we want to use DPDK, then we need to compile DPDK stuff into bazel by ourselves. It is not a trivial thing and the bazelment (https://github.com/bazelment/dpdk) guys have spent much effort extracting the core files in DPDK and write the BUILD files for DPDK. But now it seems they have stopped maintaining that repo since DPDK 16.04. Even in that version, it has some runtime failure when we use DPDK in our bazel project.


________________________________
From: Thomas Monjalon <thomas@monjalon.net>
Sent: Wednesday, March 10, 2021 3:36 PM
To: Jinkun Geng <gjk1994@stanford.edu>
Cc: Stephen Hemminger <stephen@networkplumber.org>; ci@dpdk.org <ci@dpdk.org>; users@dpdk.org <users@dpdk.org>
Subject: Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel

09/03/2021 05:11, Jinkun Geng:
> Too bad. :<

Why is it too bad?
How the choice of an internal build system
can affect other projects?


From: Stephen Hemminger <stephen@networkplumber.org>
> On Tue, 9 Mar 2021 01:32:16 +0000
> Jinkun Geng <gjk1994@stanford.edu> wrote:
>
> > Hi, all.
> > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.

Why the DPDK community would support building with Bazel?
What is the benefit?
Bazel projects cannot just link with DPDK using pkg-config?


> > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> >
> > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> >
> > Jinkun
>
> DPDK is on meson now. The core team is unlikely to change build systems again.

DPDK supports library standards for compiling, installing and linking.
What else is needed?

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

11/03/2021 00:42, Jinkun Geng:
> For any project using bazel, if we want to use DPDK, then we need to compile DPDK stuff into bazel by ourselves. It is not a trivial thing and the bazelment (https://github.com/bazelment/dpdk) guys have spent much effort extracting the core files in DPDK and write the BUILD files for DPDK. But now it seems they have stopped maintaining that repo since DPDK 16.04. Even in that version, it has some runtime failure when we use DPDK in our bazel project.

Sorry I don't know bazel.
Do you mean all components must be built inside bazel?
You are not able to call meson/ninja commands from bazel?
How do you do with other libraries?
Does bazel usually reimplement what is packaged with autotools?


> ________________________________
From: Thomas Monjalon <thomas@monjalon.net>
> 09/03/2021 05:11, Jinkun Geng:
> > Too bad. :<
> 
> Why is it too bad?
> How the choice of an internal build system
> can affect other projects?
> 
> 
> From: Stephen Hemminger <stephen@networkplumber.org>
> > On Tue, 9 Mar 2021 01:32:16 +0000
> > Jinkun Geng <gjk1994@stanford.edu> wrote:
> >
> > > Hi, all.
> > > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.
> 
> Why the DPDK community would support building with Bazel?
> What is the benefit?
> Bazel projects cannot just link with DPDK using pkg-config?
> 
> 
> > > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> > >
> > > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> > >
> > > Jinkun
> >
> > DPDK is on meson now. The core team is unlikely to change build systems again.
> 
> DPDK supports library standards for compiling, installing and linking.
> What else is needed?

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Jinkun Geng]

Do you mean all components must be built inside bazel?
>> Sort of. We have a project that is built on bazel. Now, we need to use the core functions of DPDK to replace the network primitives in our project, so that we can improve the performance of our project.

You are not able to call meson/ninja commands from bazel?
>> I have found any examples using meson inside bazel. According to my understanding, Bazel and meson are two parallel building systems and I haven't seen anyone could use them together.

How do you do with other libraries? Does bazel usually reimplement what is packaged with autotools?
>> That is the reason why https://github.com/bazelment exists. If some libraries are popular and do not support bazel, then these guys will help generate some modified version of these libraries, so that developers can integrate the library into bazel. It is not completely reimplementation, but it indeed needs much extraction to adapt the previous library so that it can be used in bazel.
Unfortunately, these guys stop supporting DPDK atop bazel now.


________________________________
From: Thomas Monjalon <thomas@monjalon.net>
Sent: Wednesday, March 10, 2021 4:06 PM
To: Jinkun Geng <gjk1994@stanford.edu>
Cc: Stephen Hemminger <stephen@networkplumber.org>; ci@dpdk.org <ci@dpdk.org>; users@dpdk.org <users@dpdk.org>
Subject: Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel

11/03/2021 00:42, Jinkun Geng:
> For any project using bazel, if we want to use DPDK, then we need to compile DPDK stuff into bazel by ourselves. It is not a trivial thing and the bazelment (https://github.com/bazelment/dpdk) guys have spent much effort extracting the core files in DPDK and write the BUILD files for DPDK. But now it seems they have stopped maintaining that repo since DPDK 16.04. Even in that version, it has some runtime failure when we use DPDK in our bazel project.

Sorry I don't know bazel.
Do you mean all components must be built inside bazel?
You are not able to call meson/ninja commands from bazel?
How do you do with other libraries?
Does bazel usually reimplement what is packaged with autotools?


> ________________________________
From: Thomas Monjalon <thomas@monjalon.net>
> 09/03/2021 05:11, Jinkun Geng:
> > Too bad. :<
>
> Why is it too bad?
> How the choice of an internal build system
> can affect other projects?
>
>
> From: Stephen Hemminger <stephen@networkplumber.org>
> > On Tue, 9 Mar 2021 01:32:16 +0000
> > Jinkun Geng <gjk1994@stanford.edu> wrote:
> >
> > > Hi, all.
> > > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.
>
> Why the DPDK community would support building with Bazel?
> What is the benefit?
> Bazel projects cannot just link with DPDK using pkg-config?
>
>
> > > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> > >
> > > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> > >
> > > Jinkun
> >
> > DPDK is on meson now. The core team is unlikely to change build systems again.
>
> DPDK supports library standards for compiling, installing and linking.
> What else is needed?

Re: [dpdk-users] [dpdk-ci] Integration of DPDK into Bazel

[Thomas Monjalon]

11/03/2021 01:17, Jinkun Geng:
> Do you mean all components must be built inside bazel?
> >> Sort of. We have a project that is built on bazel. Now, we need to use the core functions of DPDK to replace the network primitives in our project, so that we can improve the performance of our project.

You don't need to compile DPDK inside your project.
Just install the package or compile it externally.

> You are not able to call meson/ninja commands from bazel?
> >> I have found any examples using meson inside bazel. According to my understanding, Bazel and meson are two parallel building systems and I haven't seen anyone could use them together.
> 
> How do you do with other libraries? Does bazel usually reimplement what is packaged with autotools?
> >> That is the reason why https://github.com/bazelment exists. If some libraries are popular and do not support bazel, then these guys will help generate some modified version of these libraries, so that developers can integrate the library into bazel. It is not completely reimplementation, but it indeed needs much extraction to adapt the previous library so that it can be used in bazel.
> Unfortunately, these guys stop supporting DPDK atop bazel now.

That's crazy trying to replace the build system of projects.
Even if you find new volunteers, it will not be always up-to-date.

You should look for how to link an existing library from bazel.


> ________________________________
From: Thomas Monjalon <thomas@monjalon.net>
> 11/03/2021 00:42, Jinkun Geng:
> > For any project using bazel, if we want to use DPDK, then we need to compile DPDK stuff into bazel by ourselves. It is not a trivial thing and the bazelment (https://github.com/bazelment/dpdk) guys have spent much effort extracting the core files in DPDK and write the BUILD files for DPDK. But now it seems they have stopped maintaining that repo since DPDK 16.04. Even in that version, it has some runtime failure when we use DPDK in our bazel project.
> 
> Sorry I don't know bazel.
> Do you mean all components must be built inside bazel?
> You are not able to call meson/ninja commands from bazel?
> How do you do with other libraries?
> Does bazel usually reimplement what is packaged with autotools?
> 
> 
> > ________________________________
> From: Thomas Monjalon <thomas@monjalon.net>
> > 09/03/2021 05:11, Jinkun Geng:
> > > Too bad. :<
> >
> > Why is it too bad?
> > How the choice of an internal build system
> > can affect other projects?
> >
> >
> > From: Stephen Hemminger <stephen@networkplumber.org>
> > > On Tue, 9 Mar 2021 01:32:16 +0000
> > > Jinkun Geng <gjk1994@stanford.edu> wrote:
> > >
> > > > Hi, all.
> > > > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.
> >
> > Why the DPDK community would support building with Bazel?
> > What is the benefit?
> > Bazel projects cannot just link with DPDK using pkg-config?
> >
> >
> > > > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> > > >
> > > > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> > > >
> > > > Jinkun
> > >
> > > DPDK is on meson now. The core team is unlikely to change build systems again.
> >
> > DPDK supports library standards for compiling, installing and linking.
> > What else is needed?
> 
> 
> 
> 
> 
>