How to enable logging and where to find the logs

How to enable logging and where to find the logs

[Sindhura Bandi]

[-- Attachment #1: Type: text/plain, Size: 1253 bytes --]

Hi Everyone,

I might be asking a basic question here. I'm facing some issues while bringing up a dpdk interface on ovs-dpdk on debian-12. Dpdk version is 22.11.1.
Dpdk seems to fail to allocate a mempool, even though there seem to be enough free huge page memory available on the machine.

I tried to enable dpdk debugs using the following command,
ovs-vsctl set Open_vSwitch . other_config:dpdk-extra="--log-level=lib.mbuf:debug"

The command was accepted, but I don't see any logs from dpdk anywhere in my /var/log.

I get the following information from ovs logs.

Feb 22 01:03:42 cep ovs-vswitchd[108468]: ovs|00237|netdev_dpdk|ERR|Failed to create mempool "ovs013337e900021580020512" with a request of 20512 mbufs
Feb 22 01:03:42 cep ovs-vswitchd[108468]: ovs|00238|netdev_dpdk|ERR|Failed to create memory pool for netdev vhost-user0, with MTU 1500 on socket 0: No such file or directory
Feb 22 01:03:42 cep ovs-vswitchd[108468]: ovs|00239|dpif_netdev|ERR|Failed to set interface vhost-user0 new configuration

Checked ovs code, and its callingvrte_pktmbuf_pool_create(), which is failing with "No such file or directory". I would like to enable debugs for lib.mbuf.
Please let me know if I'm missing anything.

Thanks,
Sindhu

[-- Attachment #2: Type: text/html, Size: 5374 bytes --]

RE: How to enable logging and where to find the logs

[Tummala, Sivaprasad]

[-- Attachment #1: Type: text/plain, Size: 2624 bytes --]

[AMD Official Use Only - General]

Hi Sindhu,

You can try enabling the dpdk logging with:

  ovs-appctl vlog/set file:dpdk:dbg

DPDK logs should be in ovs-vswitchd log itself as indicated below.

2024-02-29T11:02:54.327Z|00115|dpif_netdev|INFO|Core 10 on numa node 0 assigned port 'dpdk0' rx queue 0 (measured processing cycles 0).
2024-02-29T11:02:54.332Z|00116|dpif|WARN|netdev@ovs-netdev: port_set_config failed (Invalid argument)
2024-02-29T11:02:54.332Z|00117|ofproto|WARN|br0: datapath configuration on port 1 (dpdk0) failed (Invalid argument)
2024-02-29T11:02:54.335Z|00118|dpdk|INFO|Device with port_id=1 already stopped
2024-02-29T11:02:54.350Z|00119|dpdk|INFO|ice_set_rx_function(): Using AVX2 OFFLOAD Vector Rx (port 1).
2024-02-29T11:02:54.813Z|00120|netdev_dpdk|INFO|Port 1: 6c:fe:54:6b:4f:11
2024-02-29T11:02:54.814Z|00121|netdev_dpdk|INFO|dpdk1: rx-steering: default rss
2024-02-29T11:02:54.814Z|00122|dpif_netdev|INFO|Performing pmd to rx queue assignment using cycles algorithm.

Thanks & Regards,
Sivaprasad


From: Sindhura Bandi <sindhura.bandi@certesnetworks.com>
Sent: Thursday, February 22, 2024 6:52 AM
To: users@dpdk.org
Subject: How to enable logging and where to find the logs

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi Everyone,

I might be asking a basic question here. I'm facing some issues while bringing up a dpdk interface on ovs-dpdk on debian-12. Dpdk version is 22.11.1.
Dpdk seems to fail to allocate a mempool, even though there seem to be enough free huge page memory available on the machine.

I tried to enable dpdk debugs using the following command,
ovs-vsctl set Open_vSwitch . other_config:dpdk-extra="--log-level=lib.mbuf:debug"

The command was accepted, but I don't see any logs from dpdk anywhere in my /var/log.

I get the following information from ovs logs.

Feb 22 01:03:42 cep ovs-vswitchd[108468]: ovs|00237|netdev_dpdk|ERR|Failed to create mempool "ovs013337e900021580020512" with a request of 20512 mbufs
Feb 22 01:03:42 cep ovs-vswitchd[108468]: ovs|00238|netdev_dpdk|ERR|Failed to create memory pool for netdev vhost-user0, with MTU 1500 on socket 0: No such file or directory
Feb 22 01:03:42 cep ovs-vswitchd[108468]: ovs|00239|dpif_netdev|ERR|Failed to set interface vhost-user0 new configuration

Checked ovs code, and its callingvrte_pktmbuf_pool_create(), which is failing with "No such file or directory". I would like to enable debugs for lib.mbuf.
Please let me know if I'm missing anything.

Thanks,
Sindhu

[-- Attachment #2: Type: text/html, Size: 10792 bytes --]

Re: How to enable logging and where to find the logs

[David Marchand]

Hello,

On Thu, Feb 29, 2024 at 9:51 AM Sindhura Bandi
<sindhura.bandi@certesnetworks.com> wrote:
>
> I might be asking a basic question here. I'm facing some issues while bringing up a dpdk interface on ovs-dpdk on debian-12. Dpdk version is 22.11.1.
> Dpdk seems to fail to allocate a mempool, even though there seem to be enough free huge page memory available on the machine.
>

Well, this is more a OVS question than a DPDK one :-)
There are two things to configure here.


> I tried to enable dpdk debugs using the following command,
> ovs-vsctl set Open_vSwitch . other_config:dpdk-extra="--log-level=lib.mbuf:debug"

First, as you tried, you must ask DPDK to emit debug logs for the mbuf library.
The above configuration is fine.

Just a small note that since OVS 2.14 you can enable logs dynamically
without having to restart OVS / setting a dpdk-extra param in ovsdb.
For example, I have no dpdk-extra config.

# ovs-appctl dpdk/log-list | grep mbuf
id 16: lib.mbuf, level is info

# ovs-appctl dpdk/log-set lib.mbuf:debug

# ovs-appctl dpdk/log-list | grep mbuf
id 16: lib.mbuf, level is debug

>
> The command was accepted, but I don't see any logs from dpdk anywhere in my /var/log.

That is because OVS log framework applies a filter.

# ovs-appctl vlog/list | grep dpdk
dpdk               OFF        ERR       INFO
netdev_dpdk        OFF        ERR       INFO
netdev_offload_dpdk   OFF        ERR       INFO

The dpdk facility here is only logging ERR level messages to syslog,
and INFO level to the /var/log file.

You probably want to set:
# ovs-appctl vlog/set FILE:dpdk:dbg

# ovs-appctl vlog/list | grep dpdk
dpdk               OFF        ERR        DBG
netdev_dpdk        OFF        ERR       INFO
netdev_offload_dpdk   OFF        ERR       INFO


Hope that helps.

-- 
David Marchand