DPDK 22.11 Troubleshooting

DPDK 22.11 Troubleshooting

[Gilbert Carrillo]

[-- Attachment #1.1: Type: text/plain, Size: 1315 bytes --]

Hello,

I installed DPDK version 22.11 and the QDMA DPDK driver. However, I am having trouble compiling the test applications.

I have a c++ program that has an external buffer and my end goal is to attach/map an mbuf to my external buffer for zero-copy DMA.

Currently I use CMAKE to compile my program, so I was curious if this application has to be run on meson/ninja or is there a cmake option? I tried compiling the test applications with the makefile but had no luck, I could only compile using meson/ninja.


Thank you,
---------------------------------------------------------
Name       : Gilbert Carrillo | Software Engineer
Company : Ampex Data Systems
Address    : 26460 Corporate Ave, Ste 200
Hayward, CA 94545
Office        : +1-650-367-2011
Mobile      : +1-575-312-7477
Website   : ampex.com

[A picture containing text, clock  Description automatically generated]

The Delta Family of Companies
Acroamatics - Telemetry Processing and Display Systems
Ampex Data Systems Corp - Rugged Data Systems & Encryption
Delta Digital Video - Video Compression Products
GDP - Aerospace Telemetry Systems
Telemetry and Communications Systems - Tracking Antenna Systems
Wideband Systems - Advanced Recording Solutions
--------------------------------------------------------------


[-- Attachment #1.2: Type: text/html, Size: 6224 bytes --]

[-- Attachment #2: image001.png --]
[-- Type: image/png, Size: 11437 bytes --]

Re: DPDK 22.11 Troubleshooting

[Bruce Richardson]

On Mon, May 01, 2023 at 10:27:05PM +0000, Gilbert Carrillo wrote:
>    Hello,
> 
> 
>    I installed DPDK version 22.11 and the QDMA DPDK driver. However, I am
>    having trouble compiling the test applications.
> 
> 
>    I have a c++ program that has an external buffer and my end goal is to
>    attach/map an mbuf to my external buffer for zero-copy DMA.
> 
> 
>    Currently I use CMAKE to compile my program, so I was curious if this
>    application has to be run on meson/ninja or is there a cmake option? I
>    tried compiling the test applications with the makefile but had no
>    luck, I could only compile using meson/ninja.
> 

If you have installed DPDK on your system, then a pkg-config file for it
should be available, allowing applications to be built against it using any
build system. The makefiles for the sample applications demonstrate how
this can be done.

If building the example applications with make is failing, can you share
the error messages got here, as it should work ok, once DPDK is correctly
installed on the system. An additional test you can run is "pkg-config
--path libdpdk" to check where DPDK is installed [though not all versions
of pkg-config support --path, I think].

Regards,
/Bruce

RE: DPDK 22.11 Troubleshooting

[Gilbert Carrillo]

Hi Bruce, 

Thank you for the response.

There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.

For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.

pkg-config --path libdpdk returns: /usr/lib64/pkgconfig/libdpdk.pc (I am using Redat 8.7)


V/r,
--------------------------------------------------------- 
Name       : Gilbert Carrillo | Software Engineer
Company : Ampex Data Systems
Address    : 26460 Corporate Ave, Ste 200
           Hayward, CA 94545
Office        : +1-650-367-2011
Mobile      : +1-575-312-7477
Website   : ampex.com 



-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Wednesday, May 3, 2023 2:28 AM
To: Gilbert Carrillo <gcarrillo@ampex.com>
Cc: dev@dpdk.org
Subject: Re: DPDK 22.11 Troubleshooting

On Mon, May 01, 2023 at 10:27:05PM +0000, Gilbert Carrillo wrote:
>    Hello,
> 
> 
>    I installed DPDK version 22.11 and the QDMA DPDK driver. However, I am
>    having trouble compiling the test applications.
> 
> 
>    I have a c++ program that has an external buffer and my end goal is to
>    attach/map an mbuf to my external buffer for zero-copy DMA.
> 
> 
>    Currently I use CMAKE to compile my program, so I was curious if this
>    application has to be run on meson/ninja or is there a cmake option? I
>    tried compiling the test applications with the makefile but had no
>    luck, I could only compile using meson/ninja.
> 

If you have installed DPDK on your system, then a pkg-config file for it should be available, allowing applications to be built against it using any build system. The makefiles for the sample applications demonstrate how this can be done.

If building the example applications with make is failing, can you share the error messages got here, as it should work ok, once DPDK is correctly installed on the system. An additional test you can run is "pkg-config --path libdpdk" to check where DPDK is installed [though not all versions of pkg-config support --path, I think].

Regards,
/Bruce

Re: DPDK 22.11 Troubleshooting

[Bruce Richardson]

On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> Hi Bruce, 
> 
> Thank you for the response.
> 
> There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> 
> For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> 

Note that by default the meson build will statically link the examples,
while the makefile will dynamically load the drivers at runtime. That may
explain the difference. Can you try building and running using "make
static" rather than just "make".

/Bruce

RE: DPDK 22.11 Troubleshooting

[Gilbert Carrillo]

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

Make static returns an error (see attached).

v/R,
Gilbert

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Wednesday, May 3, 2023 10:35 AM
To: Gilbert Carrillo <gcarrillo@ampex.com>
Cc: dev@dpdk.org
Subject: Re: DPDK 22.11 Troubleshooting

On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> Hi Bruce,
> 
> Thank you for the response.
> 
> There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> 
> For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> 

Note that by default the meson build will statically link the examples, while the makefile will dynamically load the drivers at runtime. That may explain the difference. Can you try building and running using "make static" rather than just "make".

/Bruce

[-- Attachment #2: make static.PNG --]
[-- Type: image/png, Size: 102488 bytes --]

Re: DPDK 22.11 Troubleshooting

[Bruce Richardson]

On Wed, May 03, 2023 at 04:53:20PM +0000, Gilbert Carrillo wrote:
> Make static returns an error (see attached).
> 
> v/R,
> Gilbert
> 

JFYI, best practice is to reply below last message when replying in mailing
list. It just makes it easier for readers to follow the conversation as it
flows from top to bottom.

Looks like we are missing libelf in the linker command when using "make
static". May need some investigation.

> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com> 
> Sent: Wednesday, May 3, 2023 10:35 AM
> To: Gilbert Carrillo <gcarrillo@ampex.com>
> Cc: dev@dpdk.org
> Subject: Re: DPDK 22.11 Troubleshooting
> 
> On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> > Hi Bruce,
> > 
> > Thank you for the response.
> > 
> > There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> > 
> > For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> > 
> 
> Note that by default the meson build will statically link the examples, while the makefile will dynamically load the drivers at runtime. That may explain the difference. Can you try building and running using "make static" rather than just "make".
> 
> /Bruce

Re: DPDK 22.11 Troubleshooting

[Bruce Richardson]

On Wed, May 03, 2023 at 04:53:20PM +0000, Gilbert Carrillo wrote:
> Make static returns an error (see attached).
> 
> v/R,
> Gilbert
>

To help investigate this issue, can you perhaps include the text of the
full build output when you run "make static". On my system I see libelf
listed on the linker flags when I run "make static", and things link
properly. I'm wondering how my setup may differ from yours.

/Bruce
 
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com> 
> Sent: Wednesday, May 3, 2023 10:35 AM
> To: Gilbert Carrillo <gcarrillo@ampex.com>
> Cc: dev@dpdk.org
> Subject: Re: DPDK 22.11 Troubleshooting
> 
> On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> > Hi Bruce,
> > 
> > Thank you for the response.
> > 
> > There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> > 
> > For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> > 
> 
> Note that by default the meson build will statically link the examples, while the makefile will dynamically load the drivers at runtime. That may explain the difference. Can you try building and running using "make static" rather than just "make".
> 
> /Bruce

RE: DPDK 22.11 Troubleshooting

[Gilbert Carrillo]

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Wednesday, May 3, 2023 11:18 AM
To: Gilbert Carrillo <gcarrillo@ampex.com>
Cc: dev@dpdk.org
Subject: Re: DPDK 22.11 Troubleshooting

On Wed, May 03, 2023 at 04:53:20PM +0000, Gilbert Carrillo wrote:
> Make static returns an error (see attached).
> 
> v/R,
> Gilbert
>

To help investigate this issue, can you perhaps include the text of the full build output when you run "make static". On my system I see libelf listed on the linker flags when I run "make static", and things link properly. I'm wondering how my setup may differ from yours.

/Bruce
 
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Wednesday, May 3, 2023 10:35 AM
> To: Gilbert Carrillo <gcarrillo@ampex.com>
> Cc: dev@dpdk.org
> Subject: Re: DPDK 22.11 Troubleshooting
> 
> On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> > Hi Bruce,
> > 
> > Thank you for the response.
> > 
> > There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> > 
> > For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> > 
> 
> Note that by default the meson build will statically link the examples, while the makefile will dynamically load the drivers at runtime. That may explain the difference. Can you try building and running using "make static" rather than just "make".
> 
> /Bruce

Below is the full output after running make static from the ethtool folder in examples.
[root@localhost ethtool]# make static
make -C lib static
make[1]: Entering directory '/home/ampex/dpdk-23.03/dpdk-23.03/examples/ethtool/lib'
cc -O3 -fPIC -DALLOW_EXPERIMENTAL_API -O3 -I/usr/include/dpdk -include rte_config.h -march=corei7 -mno-avx512f  -c rte_ethtool.c -o build/rte_ethtool.c.o
In file included from rte_ethtool.c:7:
rte_ethtool.c: In function 'rte_ethtool_get_drvinfo':
rte_ethtool.c:52:29: warning: implicit declaration of function 'rte_dev_name'; did you mean 'rte_dev_remove'? [-Wimplicit-function-declaration]
  strlcpy(drvinfo->bus_info, rte_dev_name(dev_info.device),
                             ^~~~~~~~~~~~
/usr/include/dpdk/rte_string_fns.h:90:50: note: in definition of macro 'strlcpy'
 #define strlcpy(dst, src, size) rte_strlcpy(dst, src, size)
                                                  ^~~
rte_ethtool.c:52:29: warning: passing argument 2 of 'rte_strlcpy' makes pointer from integer without a cast [-Wint-conversion]
  strlcpy(drvinfo->bus_info, rte_dev_name(dev_info.device),
                             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/dpdk/rte_string_fns.h:90:50: note: in definition of macro 'strlcpy'
 #define strlcpy(dst, src, size) rte_strlcpy(dst, src, size)
                                                  ^~~
/usr/include/dpdk/rte_string_fns.h:59:36: note: expected 'const char *' but argument is of type 'int'
 rte_strlcpy(char *dst, const char *src, size_t size)
                        ~~~~~~~~~~~~^~~
ar -cr build/librte_ethtool.a build/*.o
make[1]: Leaving directory '/home/ampex/dpdk-23.03/dpdk-23.03/examples/ethtool/lib'
make -C ethtool-app static
make[1]: Entering directory '/home/ampex/dpdk-23.03/dpdk-23.03/examples/ethtool/ethtool-app'
cc -I../lib -O3 -I/usr/include/dpdk -include rte_config.h -march=corei7 -mno-avx512f  -DALLOW_EXPERIMENTAL_API main.c ethapp.c -o build/ethtool-static -L../lib/build -l:librte_ethtool.a -Wl,--whole-archive -l:librte_common_iavf.a -l:librte_bus_auxiliary.a -l:librte_bus_pci.a -l:librte_bus_vdev.a -l:librte_bus_vmbus.a -l:librte_common_mlx5.a -l:librte_mempool_ring.a -l:librte_net_bnxt.a -l:librte_net_e1000.a -l:librte_net_enic.a -l:librte_net_failsafe.a -l:librte_net_i40e.a -l:librte_net_iavf.a -l:librte_net_ice.a -l:librte_net_ixgbe.a -l:librte_net_mlx4.a -l:librte_net_mlx5.a -l:librte_net_netvsc.a -l:librte_net_nfp.a -l:librte_net_qede.a -l:librte_net_ring.a -l:librte_net_tap.a -l:librte_net_vdev_netvsc.a -l:librte_net_vhost.a -l:librte_net_virtio.a -l:librte_node.a -l:librte_graph.a -l:librte_flow_classify.a -l:librte_pipeline.a -l:librte_table.a -l:librte_pdump.a -l:librte_port.a -l:librte_fib.a -l:librte_ipsec.a -l:librte_vhost.a -l:librte_stack.a -l:librte_security.a -l:librte_sched.a -l:librte_reorder.a -l:librte_rib.a -l:librte_dmadev.a -l:librte_regexdev.a -l:librte_rawdev.a -l:librte_pcapng.a -l:librte_member.a -l:librte_lpm.a -l:librte_latencystats.a -l:librte_ip_frag.a -l:librte_gso.a -l:librte_gro.a -l:librte_eventdev.a -l:librte_efd.a -l:librte_distributor.a -l:librte_cryptodev.a -l:librte_compressdev.a -l:librte_cfgfile.a -l:librte_bpf.a -l:librte_bitratestats.a -l:librte_bbdev.a -l:librte_acl.a -l:librte_timer.a -l:librte_hash.a -l:librte_metrics.a -l:librte_cmdline.a -l:librte_pci.a -l:librte_ethdev.a -l:librte_meter.a -l:librte_net.a -l:librte_mbuf.a -l:librte_mempool.a -l:librte_rcu.a -l:librte_ring.a -l:librte_eal.a -l:librte_telemetry.a -l:librte_kvargs.a -Wl,--no-whole-archive -Wl,--export-dynamic -Wl,--as-needed -lrte_node -lrte_graph -lrte_flow_classify -lrte_pipeline -lrte_table -lrte_pdump -lrte_port -lrte_fib -lrte_ipsec -lrte_vhost -lrte_stack -lrte_security -lrte_sched -lrte_reorder -lrte_rib -lrte_dmadev -lrte_regexdev -lrte_rawdev -lrte_pcapng -lrte_member -lrte_lpm -lrte_latencystats -lrte_ip_frag -lrte_gso -lrte_gro -lrte_eventdev -lrte_efd -lrte_distributor -lrte_cryptodev -lrte_compressdev -lrte_cfgfile -lrte_bpf -lrte_bitratestats -lrte_bbdev -lrte_acl -lrte_timer -lrte_hash -lrte_metrics -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs -pthread -lm -ldl -lnuma -lmlx5 -lpthread -lpthread -libverbs -lpthread -lmlx4 -lpthread -libverbs -lpthread 
//usr/local/lib64/librte_bpf.a(bpf_bpf_load_elf.c.o): In function `rte_bpf_elf_load':
bpf_load_elf.c:(.text+0x5e): undefined reference to `elf_version'
bpf_load_elf.c:(.text+0x6e): undefined reference to `elf_begin'
bpf_load_elf.c:(.text+0x79): undefined reference to `elf64_getehdr'
bpf_load_elf.c:(.text+0xc7): undefined reference to `elf_nextscn'
bpf_load_elf.c:(.text+0xdb): undefined reference to `elf64_getshdr'
bpf_load_elf.c:(.text+0xec): undefined reference to `elf_strptr'
bpf_load_elf.c:(.text+0x126): undefined reference to `elf_getdata'
bpf_load_elf.c:(.text+0x14f): undefined reference to `elf_ndxscn'
bpf_load_elf.c:(.text+0x15e): undefined reference to `elf_nextscn'
bpf_load_elf.c:(.text+0x197): undefined reference to `elf_nextscn'
bpf_load_elf.c:(.text+0x1ab): undefined reference to `elf64_getshdr'
bpf_load_elf.c:(.text+0x1c8): undefined reference to `elf_getdata'
bpf_load_elf.c:(.text+0x20e): undefined reference to `elf64_getehdr'
bpf_load_elf.c:(.text+0x21e): undefined reference to `elf_getscn'
bpf_load_elf.c:(.text+0x228): undefined reference to `elf_getdata'
bpf_load_elf.c:(.text+0x2a5): undefined reference to `elf_strptr'
bpf_load_elf.c:(.text+0x3d9): undefined reference to `elf_nextscn'
bpf_load_elf.c:(.text+0x3f5): undefined reference to `elf_end'
bpf_load_elf.c:(.text+0x493): undefined reference to `elf_nextscn'
bpf_load_elf.c:(.text+0x584): undefined reference to `elf_end'
bpf_load_elf.c:(.text+0x5db): undefined reference to `elf_errno'
bpf_load_elf.c:(.text+0x5e4): undefined reference to `elf_errmsg'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:48: build/ethtool-static] Error 1
make[1]: Leaving directory '/home/ampex/dpdk-23.03/dpdk-23.03/examples/ethtool/ethtool-app'
make: *** [Makefile:11: ethtool-app] Error 2

-Gilbert

RE: DPDK 22.11 Troubleshooting

[Gilbert Carrillo]

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Wednesday, May 3, 2023 11:18 AM
To: Gilbert Carrillo <gcarrillo@ampex.com>
Cc: dev@dpdk.org
Subject: Re: DPDK 22.11 Troubleshooting

On Wed, May 03, 2023 at 04:53:20PM +0000, Gilbert Carrillo wrote:
> Make static returns an error (see attached).
> 
> v/R,
> Gilbert
>

To help investigate this issue, can you perhaps include the text of the full build output when you run "make static". On my system I see libelf listed on the linker flags when I run "make static", and things link properly. I'm wondering how my setup may differ from yours.

/Bruce
 
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Wednesday, May 3, 2023 10:35 AM
> To: Gilbert Carrillo <gcarrillo@ampex.com>
> Cc: dev@dpdk.org
> Subject: Re: DPDK 22.11 Troubleshooting
> 
> On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> > Hi Bruce,
> > 
> > Thank you for the response.
> > 
> > There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> > 
> > For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> > 
> 
> Note that by default the meson build will statically link the examples, while the makefile will dynamically load the drivers at runtime. That may explain the difference. Can you try building and running using "make static" rather than just "make".
> 
> /Bruce

Bruce,

I had multiple versions of DPDK installed and I was linking the wrong one. I was able to compile my application with CMAKE successfully.

I had one last quick question. I am trying to achieve zero copy DMA from my FPGA to my external buffer. Is this possible? I saw methods such as attaching an mbuf to the external buffer? But I wasn't sure if this was truly zero copy.

-Gilbert

Re: DPDK 22.11 Troubleshooting

[Bruce Richardson]

On Mon, May 08, 2023 at 11:26:59PM +0000, Gilbert Carrillo wrote:
> 
> 
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com> 
> Sent: Wednesday, May 3, 2023 11:18 AM
> To: Gilbert Carrillo <gcarrillo@ampex.com>
> Cc: dev@dpdk.org
> Subject: Re: DPDK 22.11 Troubleshooting
> 
> On Wed, May 03, 2023 at 04:53:20PM +0000, Gilbert Carrillo wrote:
> > Make static returns an error (see attached).
> > 
> > v/R,
> > Gilbert
> >
> 
> To help investigate this issue, can you perhaps include the text of the full build output when you run "make static". On my system I see libelf listed on the linker flags when I run "make static", and things link properly. I'm wondering how my setup may differ from yours.
> 
> /Bruce
>  
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Wednesday, May 3, 2023 10:35 AM
> > To: Gilbert Carrillo <gcarrillo@ampex.com>
> > Cc: dev@dpdk.org
> > Subject: Re: DPDK 22.11 Troubleshooting
> > 
> > On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> > > Hi Bruce,
> > > 
> > > Thank you for the response.
> > > 
> > > There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> > > 
> > > For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> > > 
> > 
> > Note that by default the meson build will statically link the examples, while the makefile will dynamically load the drivers at runtime. That may explain the difference. Can you try building and running using "make static" rather than just "make".
> > 
> > /Bruce
> 
> Bruce,
> 
> I had multiple versions of DPDK installed and I was linking the wrong one. I was able to compile my application with CMAKE successfully.
> 
> I had one last quick question. I am trying to achieve zero copy DMA from my FPGA to my external buffer. Is this possible? I saw methods such as attaching an mbuf to the external buffer? But I wasn't sure if this was truly zero copy.
> 
Attaching an mbuf to an external buffer would not involve any copy of the
data itself, so should be zero-copy. Each mbuf header contains a buffer
pointer, which normally points just to the end of the header structure, but
which can point to any other location in memory.

/Bruce

RE: DPDK 22.11 Troubleshooting

[Gilbert Carrillo]

-----Original Message-----
From: Bruce Richardson <bruce.richardson@intel.com> 
Sent: Tuesday, May 9, 2023 3:43 AM
To: Gilbert Carrillo <gcarrillo@ampex.com>
Cc: dev@dpdk.org
Subject: Re: DPDK 22.11 Troubleshooting

On Mon, May 08, 2023 at 11:26:59PM +0000, Gilbert Carrillo wrote:
> 
> 
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com>
> Sent: Wednesday, May 3, 2023 11:18 AM
> To: Gilbert Carrillo <gcarrillo@ampex.com>
> Cc: dev@dpdk.org
> Subject: Re: DPDK 22.11 Troubleshooting
> 
> On Wed, May 03, 2023 at 04:53:20PM +0000, Gilbert Carrillo wrote:
> > Make static returns an error (see attached).
> > 
> > v/R,
> > Gilbert
> >
> 
> To help investigate this issue, can you perhaps include the text of the full build output when you run "make static". On my system I see libelf listed on the linker flags when I run "make static", and things link properly. I'm wondering how my setup may differ from yours.
> 
> /Bruce
>  
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Wednesday, May 3, 2023 10:35 AM
> > To: Gilbert Carrillo <gcarrillo@ampex.com>
> > Cc: dev@dpdk.org
> > Subject: Re: DPDK 22.11 Troubleshooting
> > 
> > On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> > > Hi Bruce,
> > > 
> > > Thank you for the response.
> > > 
> > > There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> > > 
> > > For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> > > 
> > 
> > Note that by default the meson build will statically link the examples, while the makefile will dynamically load the drivers at runtime. That may explain the difference. Can you try building and running using "make static" rather than just "make".
> > 
> > /Bruce
> 
> Bruce,
> 
> I had multiple versions of DPDK installed and I was linking the wrong one. I was able to compile my application with CMAKE successfully.
> 
> I had one last quick question. I am trying to achieve zero copy DMA from my FPGA to my external buffer. Is this possible? I saw methods such as attaching an mbuf to the external buffer? But I wasn't sure if this was truly zero copy.
> 
Attaching an mbuf to an external buffer would not involve any copy of the data itself, so should be zero-copy. Each mbuf header contains a buffer pointer, which normally points just to the end of the header structure, but which can point to any other location in memory.

/Bruce


Thank you, Bruce. Is attaching an mbuf to my external buffer the only way to achieve this zero copy DMA? I also read that there is a function rte_pktmbuf_read() that can read the data from my FPGA directly to my external buffer, but I don't think this method is zero copy since it requires the CPU to copy the data. 

/Gilbert

Re: DPDK 22.11 Troubleshooting

[Bruce Richardson]

On Tue, May 09, 2023 at 02:16:39PM +0000, Gilbert Carrillo wrote:
> 
> 
> -----Original Message-----
> From: Bruce Richardson <bruce.richardson@intel.com> 
> Sent: Tuesday, May 9, 2023 3:43 AM
> To: Gilbert Carrillo <gcarrillo@ampex.com>
> Cc: dev@dpdk.org
> Subject: Re: DPDK 22.11 Troubleshooting
> 
> On Mon, May 08, 2023 at 11:26:59PM +0000, Gilbert Carrillo wrote:
> > 
> > 
> > -----Original Message-----
> > From: Bruce Richardson <bruce.richardson@intel.com>
> > Sent: Wednesday, May 3, 2023 11:18 AM
> > To: Gilbert Carrillo <gcarrillo@ampex.com>
> > Cc: dev@dpdk.org
> > Subject: Re: DPDK 22.11 Troubleshooting
> > 
> > On Wed, May 03, 2023 at 04:53:20PM +0000, Gilbert Carrillo wrote:
> > > Make static returns an error (see attached).
> > > 
> > > v/R,
> > > Gilbert
> > >
> > 
> > To help investigate this issue, can you perhaps include the text of the full build output when you run "make static". On my system I see libelf listed on the linker flags when I run "make static", and things link properly. I'm wondering how my setup may differ from yours.
> > 
> > /Bruce
> >  
> > > -----Original Message-----
> > > From: Bruce Richardson <bruce.richardson@intel.com>
> > > Sent: Wednesday, May 3, 2023 10:35 AM
> > > To: Gilbert Carrillo <gcarrillo@ampex.com>
> > > Cc: dev@dpdk.org
> > > Subject: Re: DPDK 22.11 Troubleshooting
> > > 
> > > On Wed, May 03, 2023 at 04:22:05PM +0000, Gilbert Carrillo wrote:
> > > > Hi Bruce,
> > > > 
> > > > Thank you for the response.
> > > > 
> > > > There is no errors when I run the makefile, however I do see a difference in the programs. I don't believe the makefile is linking all the libraries together as intended.
> > > > 
> > > > For example, when I run the ethtool sample program and compile it using meson, it works fine and rte_eth_dev_count_avail() returns the correct amount. However, when I compile ethtool with the makefile and run it rte_eth_dev_count_avail() returns 0.
> > > > 
> > > 
> > > Note that by default the meson build will statically link the examples, while the makefile will dynamically load the drivers at runtime. That may explain the difference. Can you try building and running using "make static" rather than just "make".
> > > 
> > > /Bruce
> > 
> > Bruce,
> > 
> > I had multiple versions of DPDK installed and I was linking the wrong one. I was able to compile my application with CMAKE successfully.
> > 
> > I had one last quick question. I am trying to achieve zero copy DMA from my FPGA to my external buffer. Is this possible? I saw methods such as attaching an mbuf to the external buffer? But I wasn't sure if this was truly zero copy.
> > 
> Attaching an mbuf to an external buffer would not involve any copy of the data itself, so should be zero-copy. Each mbuf header contains a buffer pointer, which normally points just to the end of the header structure, but which can point to any other location in memory.
> 
> /Bruce
> 
> 
> Thank you, Bruce. Is attaching an mbuf to my external buffer the only way to achieve this zero copy DMA? I also read that there is a function rte_pktmbuf_read() that can read the data from my FPGA directly to my external buffer, but I don't think this method is zero copy since it requires the CPU to copy the data. 
> 
Can you set up your FPGA to DMA directly to a DPDK buffer? This is the
method used by all DPDK devices: NICS, crypto accelerators, DMA
accelerators, etc.

/Bruce