DPDK CI discussions
 help / color / mirror / Atom feed
* [dpdk-ci] Integration of DPDK into Bazel
@ 2021-03-09  1:32 Jinkun Geng
  2021-03-09  2:31 ` [dpdk-ci] [dpdk-users] " Stephen Hemminger
  0 siblings, 1 reply; 8+ messages in thread
From: Jinkun Geng @ 2021-03-09  1:32 UTC (permalink / raw)
  To: users, ci

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

Hi, all.
Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community. The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.

It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.


Jinkun

[-- Attachment #2: Type: text/html, Size: 1917 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel
  2021-03-09  1:32 [dpdk-ci] Integration of DPDK into Bazel Jinkun Geng
@ 2021-03-09  2:31 ` Stephen Hemminger
  2021-03-09  4:11   ` Jinkun Geng
  0 siblings, 1 reply; 8+ messages in thread
From: Stephen Hemminger @ 2021-03-09  2:31 UTC (permalink / raw)
  To: Jinkun Geng; +Cc: users, ci

On Tue, 9 Mar 2021 01:32:16 +0000
Jinkun Geng <gjk1994@stanford.edu> wrote:

> Hi, all.
> Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community. The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> 
> It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> 
> 
> Jinkun

DPDK is on meson now. The core team is unlikely to change build systems again.

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel
  2021-03-09  2:31 ` [dpdk-ci] [dpdk-users] " Stephen Hemminger
@ 2021-03-09  4:11   ` Jinkun Geng
  2021-03-10 23:36     ` Thomas Monjalon
  0 siblings, 1 reply; 8+ messages in thread
From: Jinkun Geng @ 2021-03-09  4:11 UTC (permalink / raw)
  To: Stephen Hemminger; +Cc: users, ci

[-- Attachment #1: Type: text/plain, Size: 1283 bytes --]

Too bad. :<
________________________________
From: Stephen Hemminger <stephen@networkplumber.org>
Sent: Monday, March 8, 2021 6:31 PM
To: Jinkun Geng <gjk1994@stanford.edu>
Cc: users@dpdk.org <users@dpdk.org>; ci@dpdk.org <ci@dpdk.org>
Subject: Re: [dpdk-users] Integration of DPDK into Bazel

On Tue, 9 Mar 2021 01:32:16 +0000
Jinkun Geng <gjk1994@stanford.edu> wrote:

> Hi, all.
> Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community. The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
>
> It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
>
>
> Jinkun

DPDK is on meson now. The core team is unlikely to change build systems again.

[-- Attachment #2: Type: text/html, Size: 2152 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel
  2021-03-09  4:11   ` Jinkun Geng
@ 2021-03-10 23:36     ` Thomas Monjalon
  2021-03-10 23:42       ` Jinkun Geng
  0 siblings, 1 reply; 8+ messages in thread
From: Thomas Monjalon @ 2021-03-10 23:36 UTC (permalink / raw)
  To: Jinkun Geng; +Cc: Stephen Hemminger, ci, users

09/03/2021 05:11, Jinkun Geng:
> Too bad. :<

Why is it too bad?
How the choice of an internal build system
can affect other projects?


From: Stephen Hemminger <stephen@networkplumber.org>
> On Tue, 9 Mar 2021 01:32:16 +0000
> Jinkun Geng <gjk1994@stanford.edu> wrote:
> 
> > Hi, all.
> > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.

Why the DPDK community would support building with Bazel?
What is the benefit?
Bazel projects cannot just link with DPDK using pkg-config?


> > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> >
> > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> >
> > Jinkun
> 
> DPDK is on meson now. The core team is unlikely to change build systems again.

DPDK supports library standards for compiling, installing and linking.
What else is needed?



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel
  2021-03-10 23:36     ` Thomas Monjalon
@ 2021-03-10 23:42       ` Jinkun Geng
  2021-03-11  0:06         ` Thomas Monjalon
  0 siblings, 1 reply; 8+ messages in thread
From: Jinkun Geng @ 2021-03-10 23:42 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: Stephen Hemminger, ci, users

[-- Attachment #1: Type: text/plain, Size: 2252 bytes --]

For any project using bazel, if we want to use DPDK, then we need to compile DPDK stuff into bazel by ourselves. It is not a trivial thing and the bazelment (https://github.com/bazelment/dpdk) guys have spent much effort extracting the core files in DPDK and write the BUILD files for DPDK. But now it seems they have stopped maintaining that repo since DPDK 16.04. Even in that version, it has some runtime failure when we use DPDK in our bazel project.


________________________________
From: Thomas Monjalon <thomas@monjalon.net>
Sent: Wednesday, March 10, 2021 3:36 PM
To: Jinkun Geng <gjk1994@stanford.edu>
Cc: Stephen Hemminger <stephen@networkplumber.org>; ci@dpdk.org <ci@dpdk.org>; users@dpdk.org <users@dpdk.org>
Subject: Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel

09/03/2021 05:11, Jinkun Geng:
> Too bad. :<

Why is it too bad?
How the choice of an internal build system
can affect other projects?


From: Stephen Hemminger <stephen@networkplumber.org>
> On Tue, 9 Mar 2021 01:32:16 +0000
> Jinkun Geng <gjk1994@stanford.edu> wrote:
>
> > Hi, all.
> > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.

Why the DPDK community would support building with Bazel?
What is the benefit?
Bazel projects cannot just link with DPDK using pkg-config?


> > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> >
> > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> >
> > Jinkun
>
> DPDK is on meson now. The core team is unlikely to change build systems again.

DPDK supports library standards for compiling, installing and linking.
What else is needed?



[-- Attachment #2: Type: text/html, Size: 3662 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel
  2021-03-10 23:42       ` Jinkun Geng
@ 2021-03-11  0:06         ` Thomas Monjalon
  2021-03-11  0:17           ` Jinkun Geng
  0 siblings, 1 reply; 8+ messages in thread
From: Thomas Monjalon @ 2021-03-11  0:06 UTC (permalink / raw)
  To: Jinkun Geng; +Cc: Stephen Hemminger, ci, users

11/03/2021 00:42, Jinkun Geng:
> For any project using bazel, if we want to use DPDK, then we need to compile DPDK stuff into bazel by ourselves. It is not a trivial thing and the bazelment (https://github.com/bazelment/dpdk) guys have spent much effort extracting the core files in DPDK and write the BUILD files for DPDK. But now it seems they have stopped maintaining that repo since DPDK 16.04. Even in that version, it has some runtime failure when we use DPDK in our bazel project.

Sorry I don't know bazel.
Do you mean all components must be built inside bazel?
You are not able to call meson/ninja commands from bazel?
How do you do with other libraries?
Does bazel usually reimplement what is packaged with autotools?


> ________________________________
From: Thomas Monjalon <thomas@monjalon.net>
> 09/03/2021 05:11, Jinkun Geng:
> > Too bad. :<
> 
> Why is it too bad?
> How the choice of an internal build system
> can affect other projects?
> 
> 
> From: Stephen Hemminger <stephen@networkplumber.org>
> > On Tue, 9 Mar 2021 01:32:16 +0000
> > Jinkun Geng <gjk1994@stanford.edu> wrote:
> >
> > > Hi, all.
> > > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.
> 
> Why the DPDK community would support building with Bazel?
> What is the benefit?
> Bazel projects cannot just link with DPDK using pkg-config?
> 
> 
> > > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> > >
> > > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> > >
> > > Jinkun
> >
> > DPDK is on meson now. The core team is unlikely to change build systems again.
> 
> DPDK supports library standards for compiling, installing and linking.
> What else is needed?






^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel
  2021-03-11  0:06         ` Thomas Monjalon
@ 2021-03-11  0:17           ` Jinkun Geng
  2021-03-11  8:07             ` Thomas Monjalon
  0 siblings, 1 reply; 8+ messages in thread
From: Jinkun Geng @ 2021-03-11  0:17 UTC (permalink / raw)
  To: Thomas Monjalon; +Cc: Stephen Hemminger, ci, users

[-- Attachment #1: Type: text/plain, Size: 3747 bytes --]

Do you mean all components must be built inside bazel?
>> Sort of. We have a project that is built on bazel. Now, we need to use the core functions of DPDK to replace the network primitives in our project, so that we can improve the performance of our project.

You are not able to call meson/ninja commands from bazel?
>> I have found any examples using meson inside bazel. According to my understanding, Bazel and meson are two parallel building systems and I haven't seen anyone could use them together.

How do you do with other libraries? Does bazel usually reimplement what is packaged with autotools?
>> That is the reason why https://github.com/bazelment exists. If some libraries are popular and do not support bazel, then these guys will help generate some modified version of these libraries, so that developers can integrate the library into bazel. It is not completely reimplementation, but it indeed needs much extraction to adapt the previous library so that it can be used in bazel.
Unfortunately, these guys stop supporting DPDK atop bazel now.


________________________________
From: Thomas Monjalon <thomas@monjalon.net>
Sent: Wednesday, March 10, 2021 4:06 PM
To: Jinkun Geng <gjk1994@stanford.edu>
Cc: Stephen Hemminger <stephen@networkplumber.org>; ci@dpdk.org <ci@dpdk.org>; users@dpdk.org <users@dpdk.org>
Subject: Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel

11/03/2021 00:42, Jinkun Geng:
> For any project using bazel, if we want to use DPDK, then we need to compile DPDK stuff into bazel by ourselves. It is not a trivial thing and the bazelment (https://github.com/bazelment/dpdk) guys have spent much effort extracting the core files in DPDK and write the BUILD files for DPDK. But now it seems they have stopped maintaining that repo since DPDK 16.04. Even in that version, it has some runtime failure when we use DPDK in our bazel project.

Sorry I don't know bazel.
Do you mean all components must be built inside bazel?
You are not able to call meson/ninja commands from bazel?
How do you do with other libraries?
Does bazel usually reimplement what is packaged with autotools?


> ________________________________
From: Thomas Monjalon <thomas@monjalon.net>
> 09/03/2021 05:11, Jinkun Geng:
> > Too bad. :<
>
> Why is it too bad?
> How the choice of an internal build system
> can affect other projects?
>
>
> From: Stephen Hemminger <stephen@networkplumber.org>
> > On Tue, 9 Mar 2021 01:32:16 +0000
> > Jinkun Geng <gjk1994@stanford.edu> wrote:
> >
> > > Hi, all.
> > > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.
>
> Why the DPDK community would support building with Bazel?
> What is the benefit?
> Bazel projects cannot just link with DPDK using pkg-config?
>
>
> > > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> > >
> > > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> > >
> > > Jinkun
> >
> > DPDK is on meson now. The core team is unlikely to change build systems again.
>
> DPDK supports library standards for compiling, installing and linking.
> What else is needed?






[-- Attachment #2: Type: text/html, Size: 6749 bytes --]

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [dpdk-ci] [dpdk-users] Integration of DPDK into Bazel
  2021-03-11  0:17           ` Jinkun Geng
@ 2021-03-11  8:07             ` Thomas Monjalon
  0 siblings, 0 replies; 8+ messages in thread
From: Thomas Monjalon @ 2021-03-11  8:07 UTC (permalink / raw)
  To: Jinkun Geng; +Cc: Stephen Hemminger, ci, users

11/03/2021 01:17, Jinkun Geng:
> Do you mean all components must be built inside bazel?
> >> Sort of. We have a project that is built on bazel. Now, we need to use the core functions of DPDK to replace the network primitives in our project, so that we can improve the performance of our project.

You don't need to compile DPDK inside your project.
Just install the package or compile it externally.

> You are not able to call meson/ninja commands from bazel?
> >> I have found any examples using meson inside bazel. According to my understanding, Bazel and meson are two parallel building systems and I haven't seen anyone could use them together.
> 
> How do you do with other libraries? Does bazel usually reimplement what is packaged with autotools?
> >> That is the reason why https://github.com/bazelment exists. If some libraries are popular and do not support bazel, then these guys will help generate some modified version of these libraries, so that developers can integrate the library into bazel. It is not completely reimplementation, but it indeed needs much extraction to adapt the previous library so that it can be used in bazel.
> Unfortunately, these guys stop supporting DPDK atop bazel now.

That's crazy trying to replace the build system of projects.
Even if you find new volunteers, it will not be always up-to-date.

You should look for how to link an existing library from bazel.


> ________________________________
From: Thomas Monjalon <thomas@monjalon.net>
> 11/03/2021 00:42, Jinkun Geng:
> > For any project using bazel, if we want to use DPDK, then we need to compile DPDK stuff into bazel by ourselves. It is not a trivial thing and the bazelment (https://github.com/bazelment/dpdk) guys have spent much effort extracting the core files in DPDK and write the BUILD files for DPDK. But now it seems they have stopped maintaining that repo since DPDK 16.04. Even in that version, it has some runtime failure when we use DPDK in our bazel project.
> 
> Sorry I don't know bazel.
> Do you mean all components must be built inside bazel?
> You are not able to call meson/ninja commands from bazel?
> How do you do with other libraries?
> Does bazel usually reimplement what is packaged with autotools?
> 
> 
> > ________________________________
> From: Thomas Monjalon <thomas@monjalon.net>
> > 09/03/2021 05:11, Jinkun Geng:
> > > Too bad. :<
> >
> > Why is it too bad?
> > How the choice of an internal build system
> > can affect other projects?
> >
> >
> > From: Stephen Hemminger <stephen@networkplumber.org>
> > > On Tue, 9 Mar 2021 01:32:16 +0000
> > > Jinkun Geng <gjk1994@stanford.edu> wrote:
> > >
> > > > Hi, all.
> > > > Since bazel building system is becoming more and more popular, sometimes we need to integrate DPDK library into a bazel project. However, it seems there is no much support for bazel from DPDK community.
> >
> > Why the DPDK community would support building with Bazel?
> > What is the benefit?
> > Bazel projects cannot just link with DPDK using pkg-config?
> >
> >
> > > > The only support at https://github.com/bazelment/dpdk has been outdated. Based on our experience, it can only compile successfully with dpdk-16.04 (i.e. the bazel-16.04 branch). Now DPDK has developed to DPDK 21.02, but the bazel support fails to catch up.
> > > >
> > > > It would be great if the experts in DPDK community can provide some portable BUILD files to facilitate the integration of the newest DPDK into bazel project (just like bazelment). After all, writing the bazel files can be really challenging, especially if we do not have a very deep understanding of the whole DPDK codes.
> > > >
> > > > Jinkun
> > >
> > > DPDK is on meson now. The core team is unlikely to change build systems again.
> >
> > DPDK supports library standards for compiling, installing and linking.
> > What else is needed?
> 
> 
> 
> 
> 
> 






^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-03-11  8:07 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-09  1:32 [dpdk-ci] Integration of DPDK into Bazel Jinkun Geng
2021-03-09  2:31 ` [dpdk-ci] [dpdk-users] " Stephen Hemminger
2021-03-09  4:11   ` Jinkun Geng
2021-03-10 23:36     ` Thomas Monjalon
2021-03-10 23:42       ` Jinkun Geng
2021-03-11  0:06         ` Thomas Monjalon
2021-03-11  0:17           ` Jinkun Geng
2021-03-11  8:07             ` Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).