Wrapping DPDK log messages with an application logger

Wrapping DPDK log messages with an application logger

[Lukáš Šišmiš]

Hi all,


I would like to ask you if you know of a way how could I wrap log 
messages coming from the DPDK by an application logger so that output of 
the application is unified. I would think there could be opportunity to 
pass my logging callback into DPDK but I have not found it. The only 
thing that I found was setting the stream (rte_openlog_stream()). So I 
think I could use a Linux pipe to which DPDK would write to and the 
application would read the contents of it, parsing it into messages and 
logging it with the application logger. The proposed solution is 
described in this ticket: 
https://redmine.openinfosecfoundation.org/issues/6114

Is there any alternative solution?

To provide an example of what I am talking about, currently the 
application output is as follows:

"""
Config: affinity: Found affinity definition for "worker-cpu-set" 
[AffinitySetupLoadFromConfig:util-affinity.c:201]
Config: affinity: Using default prio 'medium' for set 'worker-cpu-set' 
[AffinitySetupLoadFromConfig:util-affinity.c:249] <- colored log from 
Suricata
EAL: No free 2048 kB hugepages reported on node 0 <- DPDK log

"""

Here is an example of the ideal output:

"""

Config: affinity: Found affinity definition for "worker-cpu-set" 
[AffinitySetupLoadFromConfig:util-affinity.c:201]
Config: affinity: Using default prio 'medium' for set 'worker-cpu-set' 
[AffinitySetupLoadFromConfig:util-affinity.c:249] <- colored log from 
Suricata
Info: EAL: No free 2048 kB hugepages reported on node 0 <- also colored 
log from Suricata

"""

Thank you in advance.


Cheers,
Lukas

Re: Wrapping DPDK log messages with an application logger

[Dmitry Kozlyuk]

Hi,

2023-06-22 17:45 (UTC+0200), Lukáš Šišmiš:
> I would think there could be opportunity to  pass my logging callback into
> DPDK but I have not found it.

FWIW, I too think a callback would be better than what DPDK offers now,
because FILE* argument in rte_openlog_stream() is specific to C (libc).

> The only thing that I found was setting the stream (rte_openlog_stream()).

Correct, this is what you're supposed to use.

> So I think I could use a Linux pipe to which DPDK would write to and the 
> application would read the contents of it, parsing it into messages and 
> logging it with the application logger.

> Is there any alternative solution?

https://man7.org/linux/man-pages/man3/fopencookie.3.html

It's effectively a way to pass the callback you want, just wrapped in a FILE*.

Re: Wrapping DPDK log messages with an application logger

[Stephen Hemminger]

On Thu, 22 Jun 2023 19:08:25 +0300
Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> wrote:

> Hi,
> 
> 2023-06-22 17:45 (UTC+0200), Lukáš Šišmiš:
> > I would think there could be opportunity to  pass my logging callback into
> > DPDK but I have not found it.  
> 
> FWIW, I too think a callback would be better than what DPDK offers now,
> because FILE* argument in rte_openlog_stream() is specific to C (libc).
> 
> > The only thing that I found was setting the stream (rte_openlog_stream()).  
> 
> Correct, this is what you're supposed to use.
> 
> > So I think I could use a Linux pipe to which DPDK would write to and the 
> > application would read the contents of it, parsing it into messages and 
> > logging it with the application logger.  
> 
> > Is there any alternative solution?  
> 
> https://man7.org/linux/man-pages/man3/fopencookie.3.html
> 
> It's effectively a way to pass the callback you want, just wrapped in a FILE*.


I have use fopencookie and rte_logstream to handle this in the past.
Probably the code is still there in Danos and other projects.